---
title: "Integrate Cloud Data Streaming Service using Logstash"
slug: "ses-example01"
tags: ["Search Engine Service", "Cloud Data Streaming Service"]
updated: 2026-09-17T09:18:20Z
published: 2026-09-17T09:18:20Z
canonical: "guide.ncloud-docs.com/ses-example01"
stale: true
---

> ## Documentation Index
> Fetch the complete documentation index at: https://guide.ncloud-docs.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Integrate Cloud Data Streaming Service using Logstash

Available in VPC

You can use Logstash to send data from Cloud Data Streaming Service to Search Engine Service and verify the results.

## Preliminary tasks

Before following this guide, complete these prerequisites:

- Create a VPC and server.
- Create a Cloud Data Streaming Service cluster.
- Create a Search Engine Service cluster.

The example shows how to run Logstash on a **Server** and send **Kafka** data to **Search Engine Service**.

## Configure network

This section introduces a network configuration example.

### STEP 1. Configure ACG

To configure the ACG to allow access to port 9092 of the Cloud Data Streaming Service broker node:

1. In the VPC environment on the NAVER Cloud Platform console, navigate to **Menu** > **All Services** > **Compute** > **Server** > **ACG**.
2. Select "cdss-b-xxxxx" from the ACG list and click **[Set ACG]**.
3. Enter ACG rules, and then click **[Add]**.  

![cdss-5-4_ko](https://cdn.document360.io/6998976f-9d95-4df8-b847-d375892b92c2/Images/Documentation/cdss-5-4_en.png?sv=2026-02-06&spr=https&st=2026-10-27T15%3A39%3A59Z&se=2026-10-27T16%3A39%3A59Z&sr=c&sp=r&sig=sv90MVAYhEc2403ViTIbM1tcpLzSv5IAiBl9v%2BGN9KY%3D)
  - Protocol: TCP
  - Access source: IP of the server where Logstash is to run
  - Allowed port: 9092
4. Click **[Apply]**.

To configure ACG to allow access to port 9200 of the Search Engine Service manager node:

1. In the VPC environment on the NAVER Cloud Platform console, navigate to **Menu** > **All Services** > **Compute** > **Server** > **ACG**.
2. Select "searchengine-m-xxxxx" from the ACG list and click **[Set ACG]**.
3. Enter ACG rules, and then click **[Add]**.  

![cdss-5-6_ko](https://cdn.document360.io/6998976f-9d95-4df8-b847-d375892b92c2/Images/Documentation/cdss-5-6_en.png?sv=2026-02-06&spr=https&st=2026-10-27T15%3A39%3A59Z&se=2026-10-27T16%3A39%3A59Z&sr=c&sp=r&sig=sv90MVAYhEc2403ViTIbM1tcpLzSv5IAiBl9v%2BGN9KY%3D)
  - Protocol: TCP
  - Access source: IP of the server where Logstash is to run
  - Allowed port: 9200

## Install Logstash

This section introduces an example of installing Logstash on a server. The installation process includes steps for both Elasticsearch and OpenSearch. Install the version that matches the service you are using to ensure proper testing.

### STEP 1. Install Java

1. Enter the following command to install Java:

```
yum install java-devel -y
```

### STEP 2. Install Logstash

To install Logstash:

1. Enter the following command to download Logstash to the `/root` directory.

```
# For Elasticsearch version (install OSS version)
wget https://artifacts.elastic.co/downloads/logstash/logstash-oss-7.7.0.rpm

# For OpenSearch version
wget https://artifacts.opensearch.org/logstash/logstash-oss-with-opensearch-output-plugin-7.16.3-linux-x64.tar.gz
```

1. Enter the following command to install the downloaded file.

```
# For Elasticsearch version
rpm -ivh logstash-oss-7.7.0.rpm

# For OpenSearch version
tar -zxvf logstash-oss-with-opensearch-output-plugin-7.16.3-linux-x64.tar.gz
```

1. Before starting Logstash, edit the `logstash.conf` file.

- For Elasticsearch version

```
mv /etc/logstash/logstash-sample.conf /etc/logstash/conf.d/logstash.conf
vi /etc/logstash/conf.d/logstash.conf
```

- Elasticsearch version logstash.conf

```
input {
 kafka {
  bootstrap_servers => "${bootstrap_servers}"
  topics => "cdss_topic"
 }
}

output {
  elasticsearch {
    hosts => ["http://${ses manager node1 ip}:9200", "http://${ses manager node2 ip}:9200"]
    index => "cdss-%{+YYYY.MM.dd}"
  }
}
```

- For OpenSearch version

```
# When installing to /root/, the {installation path} is /root/logstash-7.16.3. 
mv {installation path}/config/logstash-sample.conf {installation path}/config/logstash.conf
vi {installation path}/config/logstash.conf
```

- OpenSearch version logstash.conf

```
input {
 kafka {
  bootstrap_servers => "${bootstrap_servers}"
  topics => "cdss_topic"
 }
}

output {
  opensearch {
    hosts => ["https://${ses manager node1 ip}:9200", "https://${ses manager node2 ip}:9200"]
    index => "cdss-%{+YYYY.MM.dd}"
    user => ${userID}
    password => ${password}
    ssl_certificate_verification => false
  }
}
```

- Logstash Conf Comment

```
${bootstrap_servers} - Enter the IP address and Kafka port of the Cloud Data Streaming Service broker node.  Example: 172.16.19.6:9092,172.16.19.7:9092,172.16.19.8:9092
${ses manager node1 ip} - Enter the IP address of the Search Engine Service manager node.
${ses manager node2 ip} - Enter the IP address of the second Search Engine Service manager node, or leave it blank if the manager nodes are not redundant.
${userID} - For OpenSearch, enter the user ID you set when creating the cluster.
${password} - For OpenSearch, enter the password you set when creating the cluster.
```

## Run Logstash

Example commands for running Logstash are as follows.

```
# For Elasticsearch version
systemctl start logstash

# For OpenSearch version
# Use nohup for background execution.
# You need to specify the path for logstash.conf by using the -f option.
nohup {installation path}/bin/logstash -f ~{installation path}/config/logstash.conf &
```

## Build CDSS integration environment

Example commands for running Logstash are as follows.

- Install Java

```
yum install java-devel -y
```

- Install Kafka Binary

```
wget https://archive.apache.org/dist/kafka/2.4.0/kafka_2.12-2.4.0.tgz
tar -zxvf kafka_2.12-2.4.0.tgz
```

- Produce

```
./kafka_2.12-2.4.0/bin/kafka-console-producer.sh --broker-list ${broker list} --topic cdss_topic
>this is my first message
>this is my second message

# For ${broker list}, enter the IP address and Kafka port of the broker node. Example: 172.16.19.6:9092,172.16.19.7:9092,172.16.19.8:9092
```

## View Cloud Data Streaming Service data

The following example demonstrates how to query Cloud Data Streaming Service data in Search Engine Service.

```
GET cdss-2022.08.08/_search
```

```
{
  "took" : 3,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 2,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "cdss-2022.08.08",
        "_type" : "_doc",
        "_id" : "VtmKe4IBicE7MyrTaKJ5",
        "_score" : 1.0,
        "_source" : {
          "@version" : "1",
          "@timestamp" : "2022-08-08T03:40:44.335Z",
          "message" : "this is my first message"
        }
      },
      {
        "_index" : "cdss-2022.08.08",
        "_type" : "_doc",
        "_id" : "V9mKe4IBicE7MyrTg6IW",
        "_score" : 1.0,
        "_source" : {
          "@version" : "1",
          "@timestamp" : "2022-08-08T03:40:51.248Z",
          "message" : "this is my second message"
        }
      }
    ]
  }
}
```

## Use Kafka with SSL

When using SSL with Cloud Data Streaming Service, you can configure SSL by adding certificates.

- INPUT

```
input {
  kafka {
    bootstrap_servers => “{BrokerNode-HostName}:9093” 
    topics => "test“
    ssl_truststore_location => "/etc/logstash/conf.d/kafka.client.truststore.jks"
    ssl_truststore_password => “${password}"
    security_protocol => "SSL"
  }  
}
```
  - `${BrokerNode-HostName}` - This is available in Cluster list -> Broker node info -> Details -> TLS. Example: `"networktest12-d-251:9093,networktest12-d-252:9093,networktest12-d-253:9093"`
  - `${password}` - Separate password for the certificate.