Log4j SDK v2

release/20240425
English

Log4j SDK v2

Article Summary

Share feedback

Thanks for sharing your feedback!

Available in Classic and VPC

In log4j SDK v2, it describes how to use log4j SDK v2, which is provided by Effective Log Search & Analytics 2.0 service.

The following describes how to use SDK.

1. Preparation
2. Add dependency
3. Set basic appender
4. Set AsyncAppender
5. Use SDK

1. Preparation

The following describes the preparations for installing log4j SDK.

In NAVER Cloud Platform's console, Create project.
Check the txtToken value in Project detail page.
Download log4j SDK.

2. Add dependency

The following describes how to install log4j SDK v2.

Unzip the downloaded SDK file.

Add dependency to the pom.xml file as follows:

Add directories of core module(nelo2-java-sdk-core-1.6.6.jar) and log4j module(nelo2-java-sdk-log4j-1.6.6.jar) to <systemPath>

 <dependencies>
     <dependency>
         <groupId>nelo2-java-sdk-core</groupId>
         <artifactId>nelo2-java-sdk-core</artifactId>
         <version>1.6.6</version>
         <scope>system</scope>
         <systemPath>/nelo2-java-sdk-core-1.6.6.jar</systemPath>
     </dependency>
     <dependency>
         <groupId>nelo2-java-sdk-log4j</groupId>
         <artifactId>nelo2-java-sdk-log4j</artifactId>
         <version>1.6.6</version>
         <scope>system</scope>
         <systemPath>/nelo2-java-sdk-log4j-1.6.6.jar</systemPath>
     </dependency>
     <dependency>
         <groupId>log4j</groupId>
         <artifactId>log4j</artifactId>
         <version>1.2.17</version>
     </dependency>
     <dependency>
         <groupId>org.slf4j</groupId>
         <artifactId>slf4j-log4j12</artifactId>
         <version>1.7.2</version>
     </dependency>
     <dependency>
         <groupId>org.apache.thrift</groupId>
         <artifactId>libthrift</artifactId>
         <version>0.9.3</version>
     </dependency>
     <dependency>
         <groupId>org.apache.httpcomponents</groupId>
         <artifactId>httpclient</artifactId>
         <version>4.2.6</version>
     </dependency>
     <dependency>
         <groupId>com.fasterxml.jackson.core</groupId>
         <artifactId>jackson-databind</artifactId>
         <version>2.3.1</version>
     </dependency>
 </dependencies>

Note

This SDK includes and provides slf4j-log4j12, the slf4j binding library of log4j.

As slf4j provides one binding at a time due to its nature, any other library for slf4j binding cannot be used together.
When the existing reference library and libraries that nelo2 log4j SDK refers to overlap, a problem can occur. In that case, use a higher version (recommended).

3. Set basic appender

The following describes how to set the basic appender.

File name: log4j.xml

Enter configuration as follows:

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration PUBLIC "" "log4j.dtd">
<log4j:configuration xmlns:log4j='http://jakarta.apache.org/log4j/'>
    <!-- // define appenders // -->
    <appender name="STDOUT" class="org.apache.log4j.ConsoleAppender">
        <param name="Threshold" value="DEBUG"/>
        <layout class="org.apache.log4j.PatternLayout">
            <param name="ConversionPattern" value="%m%n"/>
        </layout>
    </appender>
    <!-- // define loggers // -->
    <logger name="com" additivity="false">
        <level value="INFO"/>
        <appender-ref ref="STDOUT"/>
    </logger>
    <!-- // define root // -->
    <root>
        <level value="WARN"/>
        <appender-ref ref="STDOUT"/>
    </root>
</log4j:configuration>

Enter appender in the following format:

<!-- define nelo appender -->
<appender name="nelo" class="com.naver.nelo2.log4j.ThriftAppender">
    <param name="Threshold" value="ERROR"/>
    <param name="projectName" value="%YOUR_PROJECT_ID%"/>
    <param name="collectorUrl" value="elsa-v2-col.ncloud.com"/>
    <param name="port" value="10006"/>
    <param name="timeout" value="1200"/>
    <param name="enable" value="true"/>
    <param name="errorCodeType" value="default"/>
</appender>

The following describes each setting option.

Setting options	Description
`Appender`	Select the class name of the appender depending on the transfer protocol Thrift Appender: `com.naver.nelo2.log4j.ThriftAppender` Http Appender: `com.naver.nelo2.log4j.HttpAppender`
`projectName`	Project ID
`version`	Project version (English letters, numbers, -, _, . are allowed only, and it should start with an English letter, number, or _)
`collectorUrl`	Log collecting server's URL Thrift Appender: `elsa-v2-col.ncloud.com` Http Appender: `http://elsa-v2-col.ncloud.com/_store`
`port`	Collecting server's port Thrift Appender: `10006` Http Appender: `80`
`enable`	Status of using SDK (Default: `true`)
`logType`	Set logType
`logSource`	Set logSource
`errorCodeType`	Error code type `default`: use exception information among log4j basic information When the exception information is not delivered (When the logs are recorded in log.error(message) format), the entire error message is used as the error code <example> When NullPointerException occurs: `NullPointerException` `message`: use from the beginning of the error message till the space <example> When the error message is `다운로드에러 다운로드가 실패했습니다.`, use 'downloaderror' only `action`: URL's path information (Lucy intercepter has to be applied) <example> When URL is `http://xxx.com/board/list?id=100`, the path is '/board/list' `mdc`: use the `errorCode` set to SLF4J MDC <example> When it is set to `MDC.put("errorCode", "Login")`, use `Login`
`debug`	Displays Effective Log Search & Analytics 2.0's debugging information (Default: `false`) This option value is used globally, and `true` has priority over `false`. In other words, when various appenders are declared and the `debug` value of one of them is `true`, all appenders export debug logs to the console
`timeout`	Timeout of the socket used when transferring (Default: 5000 ms(5 seconds))
`keepAlive`	KeepAlive timeout of the socket used when transferring (Default: 60,000 ms (1 minute), Maximum value: 180,000 ms (3 minutes))
`isBulkEnabled`	Set whether to use the bulk mode (Default: `true`) When the set value is `false`, logs are transferred one by one (Single mode)
`bulkSize`	When using the bulk mode, the maximum number of logs to be transferred for a single bulk request (Default: 1000, maximum value: 100,000)
`bulkInterval`	When using the bulk mode, the cycle to call bulk request (Default: 1000 ms (1 second), maximum value: 10,000 ms (10 seconds))
`alwaysIncludeLocation`	Set whether to add the `Location` field to all logs (Default: `true`) When set to `false`: set it after checking the `Location` field of the logs for which the logLevel is `ERROR` When set to `true`: set it after checking the `Location` field for all logs. Compared to `false`, it can adversely affect the logging performance.
`mdcConversionRule`	Set MDC key rename rule Format: `key1:newKey1;key2:newkey2;...` <example> `time:date;fullname:name` (When renaming MDC key's `time` to `date` and `fullname` to `name`)

Caution

Please note the following in regards to the memory usage:

The provided SDK uses bulk transfer mode by default, and the default value of bulkSize is 1000. As one Effective Log Search & Analytics 2.0 log includes various fields, such as projectName, it occupies about 1 kb of memory even when the log body is very short. Therefore, in the default setup, Nelo2 bulk occupies (로그 사이즈 + 1 kb)x1000 of memory.
When running the Java process, you can set the maximum heap to -Xmx option. At this point, consider additional memory usage.
Be more cautious when you set the bulkSize option even larger.

Bulk and single modes

NELO2 log4j SDK supports the single mode, where logs are transferred one by one, and the bulk mode, where logs are transferred in bulk. You can select bulk or single mode by setting isBulkEnabled to true or false in xml appender settings (Default: true(bulk mode)).
Following the protocol, note the following performance indicator:

The throughput when a 1 kb-sized log is transferred in a single thread for a minute
- thrift
  - Single mode: 2615.54 logs/sec
  - Bulk mode: 6642.97 logs/sec
- http
  - Single mode: 592.97 logs/sec
  - Bulk mode: 4665.26 logs/sec

Note

The specifications for the equipment used in the performance test above are as follows:
- Log transfer server: 2 GHZ, 12 core CPU, 48 G memory, and located in Gasan IDC
- Log collecting server: 2.26 GHZ, 12 core CPU, 48 G memory, and located in Gasan IDC
The transfer performance may differ depending on the load.
- The test was run with no load, and it shows relatively low performance when transferring with the actual instance in use.
- Different performance due to the instance load can be perceived more clearly in the single mode than the bulk mode. Therefore, use the bulk mode, which is the default (recommended).
- The maximum size of the packet that the collecting server allows is 30 mb. Set proper bulkSize in consideration of the log pattern of the client server (Default: 1000).

4. Set AsyncAppender

You can get the same type of results as the basic appender by using AsyncAppender supported by the basic Appenderlog4j.

Set AsyncAppender referring to the following example.

Caution

When using AsyncAppender, please be cautious with the following setting values.

bufferSize's default value is 128, but it may not be sufficient for some applications.
locationInfo's default value is false, and in this case, AsyncAppender ignores the location where the log occurs.

Note

For more information on AsyncAppender settings, see log4j manual.

<appender name="nelo-async" class="org.apache.log4j.AsyncAppender">
    <param name="Threshold" value="ERROR"/>
    <param name="blocking" value="false"/>
    <param name="locationInfo" value="true"/>
    <param name="bufferSize" value="2048"/>
    <appender-ref ref="nelo"/>
</appender>

To execute the actual log transfer task in a separate thread using Nelo2AsyncAppender in addition to ThriftAppender by default, refer to the following examples and set:

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration PUBLIC "" "log4j.dtd">
<log4j:configuration xmlns:log4j='http://jakarta.apache.org/log4j/'>
    <!-- // define appenders // -->
    <appender name="STDOUT" class="org.apache.log4j.ConsoleAppender">
        <param name="Threshold" value="DEBUG"/>
        <layout class="org.apache.log4j.PatternLayout">
            <param name="ConversionPattern" value="%m%n"/>
        </layout>
    </appender>
    <!-- define nelo appender -->
    <appender name="nelo" class="com.naver.nelo2.log4j.ThriftAppender">
        <param name="Threshold" value="ERROR"/>
        <param name="projectName" value="ProjectID"/>
        <param name="collectorUrl" value="elsa-v2-col.ncloud.com"/>
        <param name="port" value="10006"/>
        <param name="timeout" value="1200"/>
        <param name="enable" value="true"/>
        <param name="errorCodeType" value="default"/>
    </appender>
    <!-- define nelo-async appender -->
    <appender name="nelo-async"
              class="org.apache.log4j.AsyncAppender">
        <param name="Threshold" value="ERROR"/>
        <param name="blocking" value="false"/>
        <param name="locationInfo" value="true"/>
        <param name="bufferSize" value="2048"/>
        <appender-ref ref="nelo"/>
    </appender>
    <!-- // define loggers // -->
    <logger name="com" additivity="false">
        <level value="ERROR"/>
        <appender-ref ref="STDOUT"/>
        <appender-ref ref="nelo-async"/>
    </logger>
    <!-- // define root // -->
    <root>
        <level value="WARN"/>
        <appender-ref ref="STDOUT"/>
        <appender-ref ref="nelo-async"/>
    </root>
</log4j:configuration>

5. Use SDK

Refer to the following actual codes that use Effective Log Search & Analytics 2.0 log4j SDK and transfer logs to the projects.

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

private static Logger logger = LoggerFactory.getLogger(log4jTest.class);
...
    logger.debug("Effective Log Search & Analytics log4j SDK Debug Message");
    try {
        String npe = null;
        npe.toString();
    } catch(Exception e) {
        logger.error("Effective Log Search & Analytics log4j SDK Exception", e);
    }

Note

Effective Log Search & Analytics 2.0 log4j SDK's service limits are as follows:

In log4j 2.0 version, the content of the guide above is not officially supported.
While using AsyncAppender, when the log occurrence speed is faster than the transfer speed, any logs that exceed queueSize (Default: 128) are not transferred.
Select between AsyncAppender and the basic appender depending on the following criteria (recommended).
- Basic appender: when you want to minimize log loss
- AsyncAppende: when you are worried about the performance degradation in an application when there are system failures in Effective Log Search & Analytics 2.0 service.

Troubleshooting log4j SDK v2

Check potential problems and the solutions in relation to log4j SDK v2.

When logs are transferred to the Effective Log Search & Analytics 2.0 server but you cannot view them from the web

When you view the transferred logs from the web, please take the following measures:

Check if the projectName is correct when there is no error in the result message after transferring the logs to Effective Log Search & Analytics 2.0's collecting server.

Check if the actual error date is transferred.

Check if the following transfer log is exported after setting the Effective Log Search & Analytics 2.0 NELO appender's debug attribute to true in the configuration file (log4j.xml).

<!-- define nelo appender -->
<appender name="nelo" class="com.naver.nelo2.log4j.ThriftAppender">
     <param name="Threshold" value="ERROR"/>
     <param name="projectName" value="%YOUR_PROJECT_ID%"/>
     <param name="collectorUrl" value="elsa-v2-col.ncloud.com"/>
     <param name="port" value="10006"/>
     <param name="timeout" value="1200"/>
     <param name="enable" value="true"/>
     <param name="errorCodeType" value="default"/>
     <param name="debug" value="true"/>
</appender>

[NELO2] Log Append : sent event, return value :
…

Was this article helpful?

What's Next

Logback SDK v2

Table of contents

1. Preparation
2. Add dependency
3. Set basic appender
4. Set AsyncAppender
5. Use SDK
Troubleshooting log4j SDK v2