Logback SDK v2

release/20240425
English

Logback SDK v2

Article Summary

Share feedback

Thanks for sharing your feedback!

Available in Classic and VPC

In logback SDK v2, it describes how to use logback SDK, which is provided by Effective Log Search & Analytics 2.0 service.

The following describes how to use SDK.

1. Preparation
2. Add dependency
3. Set basic appender
4. Set AsyncAppender
5. Use SDK

1. Preparation

The following describes the preparations for installing logback SDK.

In NAVER Cloud Platform's console, Create project.
Check the txtToken value in Project detail page.
Download logback SDK.

2. Add dependency

The following describes how to install logback SDK v2.

Unzip the downloaded SDK file.

Add dependency to the pom.xml file as follows:

Add directories of core module(nelo2-java-sdk-core-1.6.6.jar) and log4j module(nelo2-java-sdk-logback-1.6.6.jar) to <systemPath>

 <dependencies>
         <dependency>
             <groupId>nelo2-java-sdk-core</groupId>
             <artifactId>nelo2-java-sdk-core</artifactId>
             <version>1.6.6</version>
             <scope>system</scope>
             <systemPath>/nelo2-java-sdk-core-1.6.6.jar</systemPath>
         </dependency>
         <dependency>
             <groupId>nelo2-java-sdk</groupId>
             <artifactId>nelo2-java-sdk-logback</artifactId>
             <version>1.6.6</version>
             <scope>system</scope>
             <systemPath>/nelo2-java-sdk-logback-1.6.6.jar</systemPath>
         </dependency>
         <dependency>
             <groupId>ch.qos.logback</groupId>
             <artifactId>logback-classic</artifactId>
             <version>1.1.7</version>
         </dependency>
         <dependency>
             <groupId>org.apache.thrift</groupId>
             <artifactId>libthrift</artifactId>
             <version>0.9.3</version>
         </dependency>
         <dependency>
             <groupId>org.apache.httpcomponents</groupId>
             <artifactId>httpclient</artifactId>
             <version>4.2.6</version>
         </dependency>
         <dependency>
             <groupId>com.fasterxml.jackson.core</groupId>
             <artifactId>jackson-databind</artifactId>
             <version>2.3.1</version>
         </dependency>
 </dependencies>

Note

This SDK includes and provides logback-classic, the slf4j binding library of logback.

As slf4j provides one binding at a time due to its nature, any other library for slf4j binding cannot be used together.
When the existing reference library and libraries that nelo2 logback SDK refers to overlap, a problem can occur. In that case, use a higher version (recommended).

3. Set basic appender

The following describes how to set the basic appender.

As for logback, find the configuration file in logback.groovy → logback-test.xml → logback.xml order.
- It is described based on logback.xml in this document.
File name: logback.xml

Enter configuration as the following example:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN" shutDownHook="disable">
    <Appenders>
        <Console name="console" target="SYSTEM_OUT">
            <PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
        </Console>
        <ThriftAppender>
            <name>nelo</name>
            <collectorUrl>COLLECTOR_HOST</collectorUrl>
            <port>COLLECTOR_PORT</port>
            <projectName>PROJECT_ID</projectName>
            <version>PROJECT_VERSION</version>
            <logType>LOG_TYPE</logType>
            <logSource>LOG_SOURCE</logSource>
        </ThriftAppender>
        <HttpAppender>
            <name>http</name>
            <collectorUrl>COLLECTOR_HOST</collectorUrl>
            <port>COLLECTOR_PORT</port>
            <projectName>PROJECT_ID</projectName>
            <version>PROJECT_VERSION</version>
            <logType>LOG_TYPE</logType>
            <logSource>LOG_SOURCE</logSource>
        </HttpAppender>
    </Appenders>
    <Loggers>
        <Logger name="nelo" level="error" additivity="false">
            <AppenderRef ref="nelo"/>
            <AppenderRef ref="console"/>
        </Logger>
        <Root level="warn">
            <AppenderRef ref="nelo"/>
            <AppenderRef ref="console"/>
        </Root>
    </Loggers>
</Configuration>

Enter appender as the following example:

<!-- define nelo appender -->
<appender name="nelo" class="com.naver.nelo2.log4j.ThriftAppender">
    <param name="Threshold" value="ERROR"/>
    <param name="projectName" value="%YOUR_PROJECT_ID%"/>
    <param name="collectorUrl" value="elsa-v2-col.ncloud.com"/>
    <param name="port" value="10006"/>
    <param name="timeout" value="1200"/>
    <param name="enable" value="true"/>
    <param name="errorCodeType" value="default"/>
</appender>

The following describes each setting option.

Setting options	Description
`Appender`	Select the class name of the appender depending on the transfer protocol Thrift Appender: `com.naver.nelo2.logback.ThriftAppender` Http Appender: `com.naver.nelo2.logback.HttpAppender`
`projectName`	Project ID
`version`	Project version (English letters, numbers, -, _, . are allowed only, and it should start with an English letter, number, or _)
`collectorUrl`	Log collecting server's URL Thrift Appender: `elsa-v2-col.ncloud.com` Http Appender: `http://elsa-v2-col.ncloud.com/_store`
`port`	Collecting server's port Thrift Appender: `10006` Http Appender: `80`
`enable`	Status of using SDK (Default: `true`)
`logType`	Set logType
`logSource`	Set logSource
`errorCodeType`	Error code type `default`: use exception information among log4j basic information When the exception information is not delivered (When the logs are recorded in log.error(message) format), the entire error message is used as the error code <example> When NullPointerException occurs: `NullPointerException` `message`: use from the beginning of the error message till the space <example> When the error message is `다운로드에러 다운로드가 실패했습니다.`, use 'downloaderror' only `mdc`: use the `errorCode` value set to SLF4J MDC <example> When it is set to `MDC.put("errorCode", "Login")`, use `Login`
`debug`	Displays Effective Log Search & Analytics 2.0's debugging information (Default: `false`) This option value is used globally, and `true` has priority over `false`. In other words, when various appenders are declared and the `debug` value of one of them is `true`, all appenders export debug logs to the console
`timeout`	Timeout of the socket used when transferring (Default: 5000 ms(5 seconds))
`keepAlive`	KeepAlive timeout of the socket used when transferring (Default: 60,000 ms (1 minute), Maximum value: 180,000 ms (3 minutes))
`isBulkEnabled`	Set whether to use the bulk mode (Default: `true`) When the set value is `false`, logs are transferred one by one (Single mode)
`bulkSize`	When using the bulk mode, the maximum number of logs to be transferred for a single bulk request (Default: 1000, maximum value: 100,000)
`bulkInterval`	When using the bulk mode, the cycle to call bulk request (Default: 1000 ms (1 second), maximum value: 10,000 ms (10 seconds))
`alwaysIncludeLocation`	Set whether to add the `Location` field to all logs (Default: `true`) When set to `false`: set it after checking the `Location` field of the logs for which the logLevel is `ERROR` When set to `true`: set it after checking the `Location` field for all logs. Compared to `false`, it can adversely affect the logging performance.
`mdcConversionRule`	Set MDC key rename rule Format: `key1:newKey1;key2:newkey2;...` <example> `time:date;fullname:name` (When renaming MDC key's `time` to `date` and `fullname` to `name`)

Caution

Please note the following in regards to the memory usage:

The provided SDK uses bulk transfer mode by default, and the default value of bulkSize is 1000. As one Effective Log Search & Analytics 2.0 log includes various fields, such as projectName, it occupies about 1 kb of memory even when the log body is very short. Therefore, in the default setup, Nelo2 bulk occupies (로그 사이즈 + 1 kb)x1000 of memory.
When running the Java process, you can set the maximum heap to -Xmx option. At this point, consider additional memory usage.
Be more cautious when you set the bulkSize option even larger.

Bulk and single modes

NELO2 logback SDK supports the single mode, where logs are transferred one by one, and the bulk mode, where logs are transferred in bulk. You can select bulk or single mode by setting isBulkEnabled to true or false in xml appender settings (Default: true(bulk mode)).
Following the protocol, note the following performance indicator:

The throughput when a 1 kb-sized log is transferred in a single thread for a minute
- thrift
  - Single mode: 2587.93 logs/sec
  - Bulk mode: 6492.99 logs/sec
- http
  - Single mode: 595.86 logs/sec
  - Bulk mode: 4617.98 logs/sec

Note

The specifications for the equipment used in the performance test above are as follows:
- Log transfer server: 2 GHZ, 12 core CPU, and 48 G memory
- Log collecting server: 2.26 GHZ, 12 core CPU, and 48 G memory
The transfer performance may differ depending on the load.
- The test was run with no load, and it shows relatively low performance when transferring with the actual instance in use.
- Different performance due to the instance load can be perceived more clearly in the single mode than the bulk mode. Therefore, use the bulk mode, which is the default (recommended).
- The maximum size of the packet that the collecting server allows is 30 mb. Set proper bulkSize in consideration of the log pattern of the client server (Default: 1000).

4. Set AsyncAppender

You can execute the actual log transfer task in a separate thread using AsyncAppender, supported by logback, in addition to the appender by default.

Set AsyncAppender referring to the following example.

Caution

Set AsyncAppender's includeLocation to true. Otherwise, an error will occur during the transfer.

Note

For more information on AsyncAppender settings, see logback manual.

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN" shutDownHook="disable">
    <Appenders>
        <Console name="console" target="SYSTEM_OUT">
            <PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
        </Console>
        <ThriftAppender>
            <name>nelo</name>
            <collectorUrl>COLLECTOR_HOST</collectorUrl>
            <port>COLLECTOR_PORT</port>
            <projectName>PROJECT_ID</projectName>
            <version>PROJECT_VERSION</version>
            <logType>LOG_TYPE</logType>
            <logSource>LOG_SOURCE</logSource>
        </ThriftAppender>
        <HttpAppender>
            <name>http</name>
            <collectorUrl>COLLECTOR_HOST</collectorUrl>
            <port>COLLECTOR_PORT</port>
            <projectName>PROJECT_ID</projectName>
            <version>PROJECT_VERSION</version>
            <logType>LOG_TYPE</logType>
            <logSource>LOG_SOURCE</logSource>
        </HttpAppender>
        <Async name="async">
            <AppenderRef ref="nelo"/>
            <includeLocation>true</includeLocation>
        </Async>
    </Appenders>
    <Loggers>
        <Logger name="nelo" level="error" additivity="false">
            <AppenderRef ref="async"/>
            <AppenderRef ref="console"/>
        </Logger>
        <Root level="warn">
            <AppenderRef ref="nelo"/>
            <AppenderRef ref="console"/>
        </Root>
    </Loggers>
</Configuration>

5. Use SDK

Refer to the following actual codes that use Effective Log Search & Analytics 2.0 logback SDK and transfer logs to the projects.

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

private static final Logger logger = LoggerFactory.getLogger("nelo");
...
    logger.debug("Effective Log Search & Analytics logback SDK Debug Message");
    try {
        String npe = null;
        npe.toString();
    } catch(Exception e) {
        logger.error("Effective Log Search & Analytics logback SDK Exception", e);
    }

Note

Effective Log Search & Analytics 2.0 logback SDK's service limits are as follows:

While using AsyncAppender, when the log occurrence speed is faster than the transfer speed, any logs that exceed queueSize (Default: 128) are not transferred.
When the includeCallerData default value is false, AsyncAppender ignores the location where the log occurs.
Select between AsyncAppender and the basic appender depending on the following criteria (recommended).
- Basic appender: when you want to minimize log loss
- AsyncAppende: when you are worried about the performance degradation in an application when there are system failures in Effective Log Search & Analytics 2.0 service.

Troubleshooting logback SDK v2

Check potential problems and the solutions in relation to logback SDK v2.

When logs are transferred to the Effective Log Search & Analytics 2.0 server but you cannot view them from the web

When you view the transferred logs from the web, please take the following measures:

Check if the projectName is correct when there is no error in the result message after transferring the logs to Effective Log Search & Analytics 2.0's collecting server.
Check if the actual error date is transferred.
- Check if the following transfer log is exported after setting the Effective Log Search & Analytics 2.0 NELO appender's debug attribute to true in the configuration file (logback.xml).

  <!-- NELO2 Logback Appender -->
  <appender name="nelo-logback" class="com.naver.nelo2.logback.ThriftAppender">
      <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
          <level>ERROR</level>
      </filter>
      <param name="projectName" value="%YOUR_PROJECT_ID%"/>
      <param name="collectorUrl" value="elsa-v2-col.ncloud.com"/>
      <param name="port" value="10006"/>
      <param name="timeout" value="1200" />
      <param name="enable" value="true" />
      <param name="errorCodeType" value="default" />
      <param name="debug" value="true" />
  </appender>

  [NELO2] Log Append : sent event, return value :
  …

Was this article helpful?

What's Next

Log4j2 SDK v2

Table of contents

1. Preparation
2. Add dependency
3. Set basic appender
4. Set AsyncAppender
5. Use SDK
Troubleshooting logback SDK v2