Logback SDK v2

Prev Next

Available in Classic and VPC

In Logback SDK 2, it describes how to use Logback v2, which is provided by Effective Log Search & Analytics 2.0 service.

The following describes how to use SDK:

1. Preparation
2. Add dependency
3. Set basic appender
4. Set AsyncAppender
5. Use SDK

1. Preparation

The following describes the preparations for installing Logback SDK:

  1. In the NAVER Cloud Platform's console, Create project.
  2. Check the txtToken value in Project details page.
  3. Download Logback SDK.

2. Add dependency

To install Logback SDK v2:

  1. Unzip the downloaded SDK file.
  2. Add dependency to the pom.xml file as follows:
    • Add directories of core module (nelo2-java-sdk-core-1.6.6.jar) and log4j module (nelo2-java-sdk-logback-1.6.6.jar) to <systemPath>.
     <dependencies>
         <dependency>
             <groupId>nelo2-java-sdk-core</groupId>
             <artifactId>nelo2-java-sdk-core</artifactId>
             <version>1.6.6</version>
             <scope>system</scope>
             <systemPath>/nelo2-java-sdk-core-1.6.6.jar</systemPath>
         </dependency>
         <dependency>
             <groupId>nelo2-java-sdk</groupId>
             <artifactId>nelo2-java-sdk-logback</artifactId>
             <version>1.6.6</version>
             <scope>system</scope>
             <systemPath>/nelo2-java-sdk-logback-1.6.6.jar</systemPath>
         </dependency>
         <dependency>
             <groupId>ch.qos.logback</groupId>
             <artifactId>logback-classic</artifactId>
             <version>1.1.7</version>
         </dependency>
         <dependency>
             <groupId>org.apache.thrift</groupId>
             <artifactId>libthrift</artifactId>
             <version>0.9.3</version>
         </dependency>
         <dependency>
             <groupId>org.apache.httpcomponents</groupId>
             <artifactId>httpclient</artifactId>
             <version>4.2.6</version>
         </dependency>
         <dependency>
             <groupId>com.fasterxml.jackson.core</groupId>
             <artifactId>jackson-databind</artifactId>
             <version>2.3.1</version>
         </dependency>
 </dependencies>
Note

This SDK includes and provides logback-classic, the SLF4J binding library of Logback.

  • Since SLF4J supports only one binding at a time, you must not use it together with other SLF4J binding libraries.
  • If the existing reference library overlaps with those referenced by the NELO2 Logback SDK, problems may occur. In this case, it is recommended to use the later version.

3. Set basic appender

To set the basic appender:

  • Logback searches for configuration files in the following order: logback.groovy → logback-test.xml → logback.xml.
    • It is described based on logback.xml in this document.
  • File name: logback.xml
  1. Enter configuration as the following example:
    <?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN" shutDownHook="disable">
    <Appenders>
        <Console name="console" target="SYSTEM_OUT">
            <PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
        </Console>
        <ThriftAppender>
            <name>nelo</name>
            <collectorUrl>COLLECTOR_HOST</collectorUrl>
            <port>COLLECTOR_PORT</port>
            <projectName>PROJECT_KEY</projectName>
            <version>PROJECT_VERSION</version>
            <logType>LOG_TYPE</logType>
            <logSource>LOG_SOURCE</logSource>
        </ThriftAppender>
        <HttpAppender>
            <name>http</name>
            <collectorUrl>COLLECTOR_HOST</collectorUrl>
            <port>COLLECTOR_PORT</port>
            <projectName>PROJECT_KEY</projectName>
            <version>PROJECT_VERSION</version>
            <logType>LOG_TYPE</logType>
            <logSource>LOG_SOURCE</logSource>
        </HttpAppender>
    </Appenders>
    <Loggers>
        <Logger name="nelo" level="error" additivity="false">
            <AppenderRef ref="nelo"/>
            <AppenderRef ref="console"/>
        </Logger>
        <Root level="warn">
            <AppenderRef ref="nelo"/>
            <AppenderRef ref="console"/>
        </Root>
    </Loggers>
</Configuration>
  2. Enter appender as the following example:
    <!-- define nelo appender -->
<appender name="nelo" class="com.naver.nelo2.log4j.ThriftAppender">
    <param name="Threshold" value="ERROR"/>
    <param name="projectName" value="%YOUR_PROJECT_KEY%"/>
    <param name="collectorUrl" value="elsa-v2-col.ncloud.com"/>
    <param name="port" value="10006"/>
    <param name="timeout" value="1200"/>
    <param name="enable" value="true"/>
    <param name="errorCodeType" value="default"/>
</appender>

The following describes each setting option:

Setting options Description
Appender Select the appender class name based on the transfer protocol.
  • Thrift appender: com.naver.nelo2.logback.ThriftAppender
  • HTTP appender: com.naver.nelo2.logback.HttpAppender
projectName Project key.
version Project version. (Only English letters, numbers, -, _, . are allowed, and it should start with an English letter, number, or _.)
collectorUrl Log collecting server URL.
  • Thrift appender: elsa-v2-col.ncloud.com
  • HTTP appender: http://elsa-v2-col.ncloud.com/_store
port Collection server port.
  • Thrift appender: 10006
  • HTTP appender: 80
enable Status of using SDK (default: true).
logType Set logType.
logSource Set logSource.
errorCodeType Error code type.
  • default: Uses exception information from Log4j basic information.
    • If exception information is not delivered (for example, when the log is recorded in the log.error(message) format), the entire error message is used as the error code.
    • Example: When a NullPointerException occurs: NullPointerException
  • message: Uses only the part from the beginning of the error message up to the first whitespace.
    • Example: If the error message is 다운로드에러 다운로드가 실패했습니다., use only "Download error."
  • mdc: Uses errorCode value set in SLF4J MDC.
    • Example: If set to MDC.put("errorCode", "Login"), use Login.
debug Display debugging information of Effective Log Search & Analytics 2.0 (default: false).
  • This option applies globally, and true takes precedence over false. In other words, when various appenders are declared and the debug value of one of them is true, all appenders export debug logs to the console.
timeout Socket timeout used for transmission. The default value is 5000 ms (5 seconds).
keepAlive Socket keepAlive timeout used for transmission. The default value is 60,000 ms (1 minute), and the maximum value is 180,000 ms (3 minutes).
isBulkEnabled Set whether to use bulk mode (default: true).
  • If set to false, logs are transferred individually (single mode).
bulkSize Maximum number of logs to be sent in a single bulk request when using bulk mode. The default value is 1000, and the maximum value is 100,000.
bulkInterval Cycle to call bulk requests when using bulk mode. The default value is 1000 ms (1 second), and the maximum value is 10,000 ms (10 seconds).
alwaysIncludeLocation Set whether to add the Location field to all logs (default: true).
  • If set to false: Check and set the Location field only for logs with logLevel ERROR.
  • If set to true: Check and set the Location field for all logs. Compared to false, it can adversely affect the logging performance.
mdcConversionRule Set rule for renaming MDC keys.
  • Format: key1:newKey1;key2:newkey2;...
  • Example: time:date;fullname:name (When renaming time of MDC keys to date, and fullname to name.)
Caution

Note the following in regards to the memory usage:

  • The provided SDK uses bulk transfer mode by default, and the default value of bulkSize is 1000. As each Effective Log Search & Analytics 2.0 log includes multiple fields such as projectName, it occupies about 1 KB of memory even when the log body is very short. Therefore, with the default settings, a NELO2 bulk occupies (로그 사이즈 + 1 kb)x1000 of memory.
  • When starting a Java process, you can specify the maximum heap size using the -Xmx option. At this point, consider additional memory usage.
  • Be more cautious when you set the bulkSize option even larger.

Bulk and single modes

The NELO2 Logback SDK supports both single mode, which sends logs individually, and bulk mode, which sends logs in batches. You can select bulk or single mode by setting isBulkEnabled to true or false in xml appender settings (default: true(bulk mode)).
Following the protocol, note the following performance indicator:

  • Throughput when sending 1 KB logs from a single thread for 1 minute
    • thrift
      • Single mode: 2587.93 logs/sec
      • Bulk mode: 6492.99 logs/sec
    • http
      • Single mode: 595.86 logs/sec
      • Bulk mode: 4617.98 logs/sec
Note
  • The specifications for the equipment used in the performance test above are as follows:
    • Log transfer server: 2 GHZ, 12 core CPU, and 48 G memory.
    • Log collecting server: 2.26 GHZ, 12 core CPU, and 48 G memory.
  • The transfer performance may differ depending on the load.
    • The test was run with no load, and it shows relatively low performance when transferring with the actual instance in use.
    • Different performance due to the instance load can be perceived more clearly in the single mode than the bulk mode. Therefore, use the bulk mode, which is the default (recommended).
    • The maximum size of the packet that the collecting server allows is 30 MB. Configure an appropriate bulkSize based on the client server's log pattern. (Default: 1000)

4. Set AsyncAppender

In addition to the default appender, you can configure Logback's AsyncAppender to perform log transmission in a separate thread.

To configure AsyncAppender, refer to the following example.

Caution

Set AsyncAppender's includeLocation to true. Otherwise, an error occurs during the transfer.

Note

For more information on AsyncAppender settings, see Logback manual.

Example:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN" shutDownHook="disable">
    <Appenders>
        <Console name="console" target="SYSTEM_OUT">
            <PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
        </Console>
        <ThriftAppender>
            <name>nelo</name>
            <collectorUrl>COLLECTOR_HOST</collectorUrl>
            <port>COLLECTOR_PORT</port>
            <projectName>PROJECT_KEY</projectName>
            <version>PROJECT_VERSION</version>
            <logType>LOG_TYPE</logType>
            <logSource>LOG_SOURCE</logSource>
        </ThriftAppender>
        <HttpAppender>
            <name>http</name>
            <collectorUrl>COLLECTOR_HOST</collectorUrl>
            <port>COLLECTOR_PORT</port>
            <projectName>PROJECT_KEY</projectName>
            <version>PROJECT_VERSION</version>
            <logType>LOG_TYPE</logType>
            <logSource>LOG_SOURCE</logSource>
        </HttpAppender>
        <Async name="async">
            <AppenderRef ref="nelo"/>
            <includeLocation>true</includeLocation>
        </Async>
    </Appenders>
    <Loggers>
        <Logger name="nelo" level="error" additivity="false">
            <AppenderRef ref="async"/>
            <AppenderRef ref="console"/>
        </Logger>
        <Root level="warn">
            <AppenderRef ref="nelo"/>
            <AppenderRef ref="console"/>
        </Root>
    </Loggers>
</Configuration>

5. Use SDK

Refer to the following actual codes that use Effective Log Search & Analytics 2.0 Logback SDK and transfer logs to the projects:

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

private static final Logger logger = LoggerFactory.getLogger("nelo");
...
    logger.debug("Effective Log Search & Analytics logback SDK Debug Message");
    try {
        String npe = null;
        npe.toString();
    } catch(Exception e) {
        logger.error("Effective Log Search & Analytics logback SDK Exception", e);
    }
Note

Effective Log Search & Analytics 2.0 Logback SDK's service limits are as follows:

  • While using AsyncAppender, when the log occurrence speed is faster than the transfer speed, any logs that exceed queueSize (default: 128) are not transferred.
  • When the includeCallerData default value is false, the AsyncAppender ignores the location information of the log.
  • Select between AsyncAppender and the basic appender depending on the following criteria (recommended):
    • Basic appender: When you want to minimize log loss.
    • AsyncAppende: When you are worried about the performance degradation in an application when there are system failures in Effective Log Search & Analytics 2.0 service.

Logback SDK v2 troubleshooting

Check potential problems and solutions related to Logback SDK v2.

When logs are transferred to the Effective Log Search & Analytics 2.0 server but you cannot view them from the web

When you cannot view the transferred logs from the web, take the following measures:

  1. Check if the projectName is correct when there is no error in the result message after transferring the logs to Effective Log Search & Analytics 2.0's collecting server.
  2. Check if the actual error date is transferred.
    • Check if the following transfer log is exported after setting the Effective Log Search & Analytics 2.0 NELO appender's debug attribute to true in the configuration file (logback.xml).
  <!-- NELO2 Logback Appender -->
  <appender name="nelo-logback" class="com.naver.nelo2.logback.ThriftAppender">
      <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
          <level>ERROR</level>
      </filter>
      <param name="projectName" value="%YOUR_PROJECT_KEY%"/>
      <param name="collectorUrl" value="elsa-v2-col.ncloud.com"/>
      <param name="port" value="10006"/>
      <param name="timeout" value="1200" />
      <param name="enable" value="true" />
      <param name="errorCodeType" value="default" />
      <param name="debug" value="true" />
  </appender>

  [NELO2] Log Append : sent event, return value :
  …