Log4j SDK v2

Prev Next

Available in Classic and VPC

In Log4j SDK v2, it describes how to use Log4j v2, which is provided by Effective Log Search & Analytics 2.0 service.

The following describes how to use SDK:

1. Preparation
2. Add dependency
3. Set basic appender
4. Set AsyncAppender
5. Use SDK

1. Preparation

The following describes the preparations for installing Log4j SDK:

  1. In the NAVER Cloud Platform's console, Create project.
  2. Check the txtToken value in Project details page.
  3. Download Log4j SDK.

2. Add dependency

To install Log4j SDK v2:

  1. Unzip the downloaded SDK file.
  2. Add dependency to the pom.xml file as follows:
    • Add directories of core module (nelo2-java-sdk-core-1.6.6.jar) and Log4j module (nelo2-java-sdk-log4j-1.6.6.jar) to <systemPath>.
     <dependencies>
     <dependency>
         <groupId>nelo2-java-sdk-core</groupId>
         <artifactId>nelo2-java-sdk-core</artifactId>
         <version>1.6.6</version>
         <scope>system</scope>
         <systemPath>/nelo2-java-sdk-core-1.6.6.jar</systemPath>
     </dependency>
     <dependency>
         <groupId>nelo2-java-sdk-log4j</groupId>
         <artifactId>nelo2-java-sdk-log4j</artifactId>
         <version>1.6.6</version>
         <scope>system</scope>
         <systemPath>/nelo2-java-sdk-log4j-1.6.6.jar</systemPath>
     </dependency>
     <dependency>
         <groupId>log4j</groupId>
         <artifactId>log4j</artifactId>
         <version>1.2.17</version>
     </dependency>
     <dependency>
         <groupId>org.slf4j</groupId>
         <artifactId>slf4j-log4j12</artifactId>
         <version>1.7.2</version>
     </dependency>
     <dependency>
         <groupId>org.apache.thrift</groupId>
         <artifactId>libthrift</artifactId>
         <version>0.9.3</version>
     </dependency>
     <dependency>
         <groupId>org.apache.httpcomponents</groupId>
         <artifactId>httpclient</artifactId>
         <version>4.2.6</version>
     </dependency>
     <dependency>
         <groupId>com.fasterxml.jackson.core</groupId>
         <artifactId>jackson-databind</artifactId>
         <version>2.3.1</version>
     </dependency>
 </dependencies>
Note

This SDK includes slf4j-log4j12, which is the SLF4J binding library for Log4j.

  • Since SLF4J supports only one binding at a time, you must not use it together with other SLF4J binding libraries.
  • If the existing reference library overlaps with those referenced by the NELO2 Log4j SDK, problems may occur. In this case, it is recommended to use the later version.

3. Set basic appender

To set the basic appender:

  • File name: log4j.xml
  1. Enter configuration as follows:
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration PUBLIC "" "log4j.dtd">
<log4j:configuration xmlns:log4j='http://jakarta.apache.org/log4j/'>
    <!-- // define appenders // -->
    <appender name="STDOUT" class="org.apache.log4j.ConsoleAppender">
        <param name="Threshold" value="DEBUG"/>
        <layout class="org.apache.log4j.PatternLayout">
            <param name="ConversionPattern" value="%m%n"/>
        </layout>
    </appender>
    <!-- // define loggers // -->
    <logger name="com" additivity="false">
        <level value="INFO"/>
        <appender-ref ref="STDOUT"/>
    </logger>
    <!-- // define root // -->
    <root>
        <level value="WARN"/>
        <appender-ref ref="STDOUT"/>
    </root>
</log4j:configuration>
  1. Enter appender in the following format:
<!-- define nelo appender -->
<appender name="nelo" class="com.naver.nelo2.log4j.ThriftAppender">
    <param name="Threshold" value="ERROR"/>
    <param name="projectName" value="%YOUR_PROJECT_KEY%"/>
    <param name="collectorUrl" value="elsa-v2-col.ncloud.com"/>
    <param name="port" value="10006"/>
    <param name="timeout" value="1200"/>
    <param name="enable" value="true"/>
    <param name="errorCodeType" value="default"/>
</appender>

The following describes each setting option:

Setting options Description
Appender Select the appender class name based on the transfer protocol.
  • Thrift appender: com.naver.nelo2.log4j.ThriftAppender
  • HTTP appender: com.naver.nelo2.log4j.HttpAppender
projectName Project key.
version Project version. (Only English letters, numbers, -, _, . are allowed, and it should start with an English letter, number, or _.)
collectorUrl Log collecting server URL.
  • Thrift appender: elsa-v2-col.ncloud.com
  • HTTP appender: http://elsa-v2-col.ncloud.com/_store
port Collection server port.
  • Thrift appender: 10006
  • HTTP appender: 80
enable Status of using SDK (default: true).
logType Set logType.
logSource Set logSource.
errorCodeType Error code type.
  • default: Uses exception information from Log4j basic information.
    • If exception information is not delivered (for example, when the log is recorded in the log.error(message) format), the entire error message is used as the error code.
    • Example: When a NullPointerException occurs: NullPointerException
  • message: Uses only the part from the beginning of the error message up to the first whitespace.
    • Example: If the error message is 다운로드에러 다운로드가 실패했습니다., use only "Download error."
  • action: Path information of the URL. (Lucy interceptor must be applied.)
    • Example: If the URL is http://xxx.com/board/list?id=100, the path is "/board/list."
  • mdc: Uses errorCode value set in SLF4J MDC.
    • Example: If set to MDC.put("errorCode", "Login"), use Login.
debug Display debugging information of Effective Log Search & Analytics 2.0 (default: false).
  • This option applies globally, and true takes precedence over false. In other words, when various appenders are declared and the debug value of one of them is true, all appenders export debug logs to the console.
timeout Socket timeout used for transmission. The default value is 5000 ms (5 seconds).
keepAlive Socket keepAlive timeout used for transmission. The default value is 60,000 ms (1 minute), and the maximum value is 180,000 ms (3 minutes).
isBulkEnabled Set whether to use bulk mode (default: true).
  • If set to false, logs are transferred individually (single mode).
bulkSize Maximum number of logs to be sent in a single bulk request when using bulk mode. The default value is 1000, and the maximum value is 100,000.
bulkInterval Cycle to call bulk requests when using bulk mode. The default value is 1000 ms (1 second), and the maximum value is 10,000 ms (10 seconds).
alwaysIncludeLocation Set whether to add the Location field to all logs (default: true).
  • If set to false: Check and set the Location field only for logs with logLevel ERROR.
  • If set to true: Check and set the Location field for all logs. Compared to false, it can adversely affect the logging performance.
mdcConversionRule Set rule for renaming MDC keys.
  • Format: key1:newKey1;key2:newkey2;...
  • Example: time:date;fullname:name (When renaming time of MDC keys to date, and fullname to name.)
Caution

Note the following in regards to the memory usage:

  • The provided SDK uses bulk transfer mode by default, and the default value of bulkSize is 1000. As each Effective Log Search & Analytics 2.0 log includes multiple fields such as projectName, it occupies about 1 KB of memory even when the log body is very short. Therefore, with the default settings, a NELO2 bulk occupies (로그 사이즈 + 1 kb)x1000 of memory.
  • When starting a Java process, you can specify the maximum heap size using the -Xmx option. At this point, consider additional memory usage.
  • Be more cautious when you set the bulkSize option even larger.

Bulk and single modes

The NELO2 Log4j SDK supports both single mode, which sends logs individually, and bulk mode, which sends logs in batches. You can select bulk or single mode by setting isBulkEnabled to true or false in xml appender settings (default: true(bulk mode)).
Following the protocol, note the following performance indicator:

  • Throughput when sending 1 KB logs from a single thread for 1 minute
    • thrift
      • Single mode: 2615.54 logs/sec
      • Bulk mode: 6642.97 logs/sec
    • http
      • Single mode: 592.97 logs/sec
      • Bulk mode: 4665.26 logs/sec
Note
  • The specifications for the equipment used in the performance test above are as follows:
    • Log transfer server: 2 GHZ, 12 core CPU, 48 G memory, and located in Gasan IDC.
    • Log collecting server: 2.26 GHZ, 12 core CPU, 48 G memory, and located in Gasan IDC.
  • The transfer performance may differ depending on the load.
    • The test was run with no load, and it shows relatively low performance when transferring with the actual instance in use.
    • Different performance due to the instance load can be perceived more clearly in the single mode than the bulk mode. Therefore, use the bulk mode, which is the default (recommended).
    • The maximum size of the packet that the collecting server allows is 30 MB. Configure an appropriate bulkSize based on the client server's log pattern. (Default: 1000)

4. Set AsyncAppender

You can get the same type of results as the basic appender by using AsyncAppender provided by the basic Appenderlog4j.

To configure AsyncAppender, refer to the following example.

Caution

When using AsyncAppender, be cautious with the following setting values:

  • bufferSize's default value is 128, but this may not be sufficient for some applications.
  • locationInfo's default value is false, and in this case, the AsyncAppender ignores the location information of the log.
Note

For more information on AsyncAppender settings, see Log4j manual.

Example:

<appender name="nelo-async" class="org.apache.log4j.AsyncAppender">
    <param name="Threshold" value="ERROR"/>
    <param name="blocking" value="false"/>
    <param name="locationInfo" value="true"/>
    <param name="bufferSize" value="2048"/>
    <appender-ref ref="nelo"/>
</appender>

To perform log transmission in a separate thread using Nelo2AsyncAppender in addition to the default ThriftAppender, refer to the following examples and set:

Example:

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration PUBLIC "" "log4j.dtd">
<log4j:configuration xmlns:log4j='http://jakarta.apache.org/log4j/'>
    <!-- // define appenders // -->
    <appender name="STDOUT" class="org.apache.log4j.ConsoleAppender">
        <param name="Threshold" value="DEBUG"/>
        <layout class="org.apache.log4j.PatternLayout">
            <param name="ConversionPattern" value="%m%n"/>
        </layout>
    </appender>
    <!-- define nelo appender -->
    <appender name="nelo" class="com.naver.nelo2.log4j.ThriftAppender">
        <param name="Threshold" value="ERROR"/>
        <param name="projectName" value="프로젝트 키"/>
        <param name="collectorUrl" value="elsa-v2-col.ncloud.com"/>
        <param name="port" value="10006"/>
        <param name="timeout" value="1200"/>
        <param name="enable" value="true"/>
        <param name="errorCodeType" value="default"/>
    </appender>
    <!-- define nelo-async appender -->
    <appender name="nelo-async"
              class="org.apache.log4j.AsyncAppender">
        <param name="Threshold" value="ERROR"/>
        <param name="blocking" value="false"/>
        <param name="locationInfo" value="true"/>
        <param name="bufferSize" value="2048"/>
        <appender-ref ref="nelo"/>
    </appender>
    <!-- // define loggers // -->
    <logger name="com" additivity="false">
        <level value="ERROR"/>
        <appender-ref ref="STDOUT"/>
        <appender-ref ref="nelo-async"/>
    </logger>
    <!-- // define root // -->
    <root>
        <level value="WARN"/>
        <appender-ref ref="STDOUT"/>
        <appender-ref ref="nelo-async"/>
    </root>
</log4j:configuration>

5. Use SDK

Refer to the following actual codes that use Effective Log Search & Analytics 2.0 Log4j SDK and transfer logs to the projects:

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

private static Logger logger = LoggerFactory.getLogger(log4jTest.class);
...
    logger.debug("Effective Log Search & Analytics log4j SDK Debug Message");
    try {
        String npe = null;
        npe.toString();
    } catch(Exception e) {
        logger.error("Effective Log Search & Analytics log4j SDK Exception", e);
    }
Note

Effective Log Search & Analytics 2.0 Log4j SDK's service limits are as follows:

  • The contents of the above guide are not officially supported in Log4j version 2.0.
  • While using AsyncAppender, when the log occurrence speed is faster than the transfer speed, any logs that exceed queueSize (default: 128) are not transferred.
  • Select between AsyncAppender and the basic appender depending on the following criteria (recommended):
    • Basic appender: When you want to minimize log loss.
    • AsyncAppende: When you are worried about the performance degradation in an application when there are system failures in Effective Log Search & Analytics 2.0 service.

Log4j SDK v2 troubleshooting

Check potential problems and solutions related to Log4j SDK v2.

When logs are transferred to the Effective Log Search & Analytics 2.0 server but you cannot view them from the web

When you cannot view the transferred logs from the web, take the following measures:

  1. Check if the projectName is correct when there is no error in the result message after transferring the logs to Effective Log Search & Analytics 2.0's collecting server.
  2. Check if the actual error date is transferred.
    • Check if the following transfer log is exported after setting the Effective Log Search & Analytics 2.0 Nelo appender's debug attribute to true in the configuration file (log4j.xml):
      <!-- define nelo appender -->
<appender name="nelo" class="com.naver.nelo2.log4j.ThriftAppender">
    <param name="Threshold" value="ERROR"/>
    <param name="projectName" value="%YOUR_PROJECT_KEY%"/>
    <param name="collectorUrl" value="elsa-v2-col.ncloud.com"/>
    <param name="port" value="10006"/>
    <param name="timeout" value="1200"/>
    <param name="enable" value="true"/>
    <param name="errorCodeType" value="default"/>
    <param name="debug" value="true"/>
</appender>

[NELO2] Log Append : sent event, return value :
…