Log4j2 SDK v2

Prev Next

Available in Classic and VPC

In log4j2 SDK v2, it describes how to use log4j2 SDK v2, which is provided by Effective Log Search & Analytics 2.0 service.

The following describes how to use SDK.

1. Preparation
2. Add dependency
3. Set basic appender
4. Set AsyncAppender
5. Use SDK

1. Preparation

The following describes the preparations for installing log4j2 SDK.

  1. In NAVER Cloud Platform's console, Create project.
  2. Check the txtToken value in Project detail page.
  3. Download log4j2 SDK.

2. Add dependency

The following describes how to install log4j2 SDK v2.

  1. Unzip the downloaded SDK file.

  2. Add dependency to the pom.xml file as follows:

    • Add directories of core module(nelo2-java-sdk-core-1.6.6.jar) and log4j2 module(nelo2-java-sdk-log4j2-2.8.5.jar) to <systemPath>
     <dependencies>
            <dependency>
                <groupId>nelo2-java-sdk-core</groupId>
                <artifactId>nelo2-java-sdk-core</artifactId>
                <version>1.6.6</version>
                <scope>system</scope>
                <systemPath>/nelo2-java-sdk-core-1.6.6.jar</systemPath>
            </dependency>
            <dependency>
                <groupId>nelo2-java-sdk</groupId>
                <artifactId>nelo2-java-sdk-log4j2</artifactId>
                <version>2.8.5</version>
                <scope>system</scope>
                <systemPath>/nelo2-java-sdk-log4j2-2.8.5.jar</systemPath>
            </dependency>
            <dependency>
                <groupId>org.apache.logging.log4j</groupId>
                <artifactId>log4j-api</artifactId>
                <version>2.8.2</version>
            </dependency>
            <dependency>
                <groupId>org.apache.logging.log4j</groupId>
                <artifactId>log4j-core</artifactId>
                <version>2.8.2</version>
            </dependency>
            <dependency>
                <groupId>org.apache.logging.log4j</groupId>
                <artifactId>log4j-slf4j-impl</artifactId>
                <version>2.8.2</version>
            </dependency>
            <dependency>
                <groupId>org.apache.thrift</groupId>
                <artifactId>libthrift</artifactId>
                <version>0.9.3</version>
            </dependency>
            <dependency>
                <groupId>org.apache.httpcomponents</groupId>
                <artifactId>httpclient</artifactId>
                <version>4.2.6</version>
            </dependency>
            <dependency>
                <groupId>com.fasterxml.jackson.core</groupId>
                <artifactId>jackson-databind</artifactId>
                <version>2.3.1</version>
            </dependency>
    </dependencies>
    
Note

This SDK includes and provides log4j-slf4j-impl, the slf4j binding library of log4j2.

  • As slf4j provides one binding at a time due to its nature, any other library for slf4j binding cannot be used together.
  • When the existing reference library and libraries that nelo2 log4j2 SDK refers to overlap, a problem can occur. In that case, use a higher version (recommended).

3. Set basic appender

The following describes how to set the basic appender.

  • File name: log4j2.xml

Enter configuration as the following example:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN" shutDownHook="disable">
    <Appenders>
        <Console name="console" target="SYSTEM_OUT">
            <PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
        </Console>
        <ThriftAppender>
            <name>nelo</name>
            <collectorUrl>COLLECTOR_HOST</collectorUrl>
            <port>COLLECTOR_PORT</port>
            <projectName>PROJECT_ID</projectName>
            <version>PROJECT_VERSION</version>
            <logType>LOG_TYPE</logType>
            <logSource>LOG_SOURCE</logSource>
        </ThriftAppender>
        <HttpAppender>
            <name>http</name>
            <collectorUrl>COLLECTOR_HOST</collectorUrl>
            <port>COLLECTOR_PORT</port>
            <projectName>PROJECT_ID</projectName>
            <version>PROJECT_VERSION</version>
            <logType>LOG_TYPE</logType>
            <logSource>LOG_SOURCE</logSource>
        </HttpAppender>
    </Appenders>
    <Loggers>
        <Logger name="nelo" level="error" additivity="false">
            <AppenderRef ref="nelo"/>
            <AppenderRef ref="console"/>
        </Logger>
        <Root level="warn">
            <AppenderRef ref="nelo"/>
            <AppenderRef ref="console"/>
        </Root>
    </Loggers>
</Configuration>

The following describes each setting option.

Setting options Description
Appender Select the class name of the appender depending on the transfer protocol
  • Thrift Appender: ThriftAppender
  • Http Appender: HttpAppender
projectName Project ID
version Project version (English letters, numbers, -, _, . are allowed only, and it should start with an English letter, number, or _)
collectorUrl Log collecting server's URL
  • Thrift Appender: elsa-v2-col.ncloud.com
  • Http Appender: http://elsa-v2-col.ncloud.com/_store
port Collecting server's port
  • Thrift Appender: 10006
  • Http Appender: 80
enable Status of using SDK (Default: true)
logType Set logType
logSource Set logSource
errorCodeType Error code type
  • default: use exception information among log4j basic information
    • When the exception information is not delivered (When the logs are recorded in log.error(message) format), the entire error message is used as the error code
    • <example> When NullPointerException occurs: NullPointerException
  • message: use from the beginning of the error message till the space
    • <example> When the error message is 다운로드에러 다운로드가 실패했습니다., use 'downloaderror' only
  • mdc: use the errorCode value set to SLF4J MDC
    • <example> When it is set to MDC.put("errorCode", "Login"), use Login
debug Displays Effective Log Search & Analytics 2.0's debugging information (Default: false)
  • This option value is used globally, and true has priority over false. In other words, when various appenders are declared and the debug value of one of them is true, all appenders export debug logs to the console
timeout Timeout of the socket used when transferring (Default: 5000 ms(5 seconds))
keepAlive KeepAlive timeout of the socket used when transferring (Default: 60,000 ms (1 minute), Maximum value: 180,000 ms (3 minutes))
isBulkEnabled Set whether to use the bulk mode (Default: true)
  • When the set value is false, logs are transferred one by one (Single mode)
bulkSize When using the bulk mode, the maximum number of logs to be transferred for a single bulk request (Default: 1000, maximum value: 100,000)
bulkInterval When using the bulk mode, the cycle to call bulk request (Default: 1000 ms (1 second), maximum value: 10,000 ms (10 seconds))
alwaysIncludeLocation Set whether to add the Location field to all logs (Default: true)
  • When set to false: set it after checking the Location field of the logs for which the logLevel is ERROR
  • When set to true: set it after checking the Location field for all logs. Compared to false, it can adversely affect the logging performance.
mdcConversionRule Set MDC key rename rule
  • Format: key1:newKey1;key2:newkey2;...
  • <example> time:date;fullname:name (When renaming MDC key's time to date and fullname to name)
Caution

Please note the following in regards to the memory usage:

  • The provided SDK uses bulk transfer mode by default, and the default value of bulkSize is 1000. As one Effective Log Search & Analytics 2.0 log includes various fields, such as projectName, it occupies about 1 kb of memory even when the log body is very short. Therefore, in the default setup, Nelo2 bulk occupies (로그 사이즈 + 1 kb)x1000 of memory.
  • When running the Java process, you can set the maximum heap to -Xmx option. At this point, consider additional memory usage.
  • Be more cautious when you set the bulkSize option even larger.

Bulk and single modes

NELO2 log4j2 SDK supports the single mode, where logs are transferred one by one, and the bulk mode, where logs are transferred in bulk. You can select bulk or single mode by setting isBulkEnabled to true or false in xml appender settings (Default: true(bulk mode)).
Following the protocol, note the following performance indicator:

  • The throughput when a 1 kb-sized log is transferred in a single thread for a minute
    • thrift
      • Single mode: 2636.00 logs/sec
      • Bulk mode: 6369.10 logs/sec
    • http
      • Single mode: 583.36 logs/sec
      • Bulk mode: 4618.90 logs/sec
Note
  • The specifications for the equipment used in the performance test above are as follows:
    • Log transfer server: 2 GHZ, 12 core CPU, and 48 G memory
    • Log collecting server: 2.26 GHZ, 12 core CPU, and 48 G memory
  • The transfer performance may differ depending on the load.
    • The test was run with no load, and it shows relatively low performance when transferring with the actual instance in use.
    • Different performance due to the instance load can be perceived more clearly in the single mode than the bulk mode. Therefore, use the bulk mode, which is the default (recommended).
    • The maximum size of the packet that the collecting server allows is 30 mb. Set proper bulkSize in consideration of the log pattern of the client server (Default: 1000).

4. Set AsyncAppender

You can execute the actual log transfer task in a separate thread using AsyncAppender, supported by log4j2, in addition to the appender by default.

Set AsyncAppender referring to the following example.

Caution

Set AsyncAppender's includeLocation to true. Otherwise, an error will occur during the transfer.

Note

For more information on AsyncAppender settings, see log4j2 manual.

<example>

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN" shutDownHook="disable">
    <Appenders>
        <Console name="console" target="SYSTEM_OUT">
            <PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
        </Console>
        <ThriftAppender>
            <name>nelo</name>
            <collectorUrl>COLLECTOR_HOST</collectorUrl>
            <port>COLLECTOR_PORT</port>
            <projectName>PROJECT_ID</projectName>
            <version>PROJECT_VERSION</version>
            <logType>LOG_TYPE</logType>
            <logSource>LOG_SOURCE</logSource>
        </ThriftAppender>
        <HttpAppender>
            <name>http</name>
            <collectorUrl>COLLECTOR_HOST</collectorUrl>
            <port>COLLECTOR_PORT</port>
            <projectName>PROJECT_ID</projectName>
            <version>PROJECT_VERSION</version>
            <logType>LOG_TYPE</logType>
            <logSource>LOG_SOURCE</logSource>
        </HttpAppender>
        <Async name="async">
            <AppenderRef ref="nelo"/>
            <includeLocation>true</includeLocation>
        </Async>
    </Appenders>
    <Loggers>
        <Logger name="nelo" level="error" additivity="false">
            <AppenderRef ref="async"/>
            <AppenderRef ref="console"/>
        </Logger>
        <Root level="warn">
            <AppenderRef ref="nelo"/>
            <AppenderRef ref="console"/>
        </Root>
    </Loggers>
</Configuration>

5. Use SDK

Refer to the following actual codes that use Effective Log Search & Analytics 2.0 log4j2 SDK and transfer logs to the projects.

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

private static final Logger logger = LoggerFactory.getLogger("nelo");
...
    logger.debug("Effective Log Search & Analytics log4j2 SDK Debug Message");
    try {
        String npe = null;
        npe.toString();
    } catch(Exception e) {
        logger.error("Effective Log Search & Analytics log4j2 SDK Exception", e);
    }
Note

Effective Log Search & Analytics 2.0 log4j2 SDK's service limits are as follows:

  • While using AsyncAppender, when the log occurrence speed is faster than the transfer speed, any logs that exceed queueSize (Default: 128) are not transferred.
  • Select between AsyncAppender and the basic appender depending on the following criteria (recommended).
    • Basic appender: when you want to minimize log loss
    • AsyncAppende: when you are worried about the performance degradation in an application when there are system failures in Effective Log Search & Analytics 2.0.

Troubleshooting log4j2 SDK v2

Check potential problems and the solutions in relation to log4j2 SDK v2.

When logs are transferred to the Effective Log Search & Analytics 2.0 server but you cannot view them from the web

When you view the transferred logs from the web, please take the following measures:

  1. Check if the projectName is correct when there is no error in the result message after transferring the logs to Effective Log Search & Analytics 2.0's collecting server.
  2. Check if the actual error date is transferred.
    • Check if the following transfer log is exported after setting the Effective Log Search & Analytics 2.0 NELO appender's debug attribute to true in the configuration file (log4j.xml).
  <!-- define nelo appender -->
  <ThriftAppender>
      <name>nelo</name>
      <collectorUrl>COLLECTOR_HOST</collectorUrl>
      <port>COLLECTOR_PORT</port>
      <projectName>PROJECT_ID</projectName>
      <version>PROJECT_VERSION</version>
      <logType>LOG_TYPE</logType>
      <logSource>LOG_SOURCE</logSource>
      <debug>true</debug>
  </ThriftAppender>

  [NELO2] Log Append : sent event, return value :
  …