Cloud Data Streaming Service concept

Available in VPC

This document explains a few concepts that'd be useful to know before using Cloud Data Streaming Service.

Apache Kafka

Open-source Apache Kafka is an open-source platform to build real-time streaming data pipelines and applications. With NAVER Cloud Platform's Cloud Data Streaming Service, you can build a Kafka cluster in just a few minutes. You don't have to install or maintain a software program separately.
NAVER Cloud Platform provides open-source Cluster Manager for Apache Kafka (CMAK), which is a management tool for Apache Kafka clusters. The service is provided as a cluster format with Apache 2.0 licensed Apache Kafka that is already installed and set up.

Kafka Connect

Kafka Connect is a component that connects external systems such as a database, key-value store, and file system to Kafka. It is open-source software provided by Apache Kafka. Kafka Connect enables you to connect different products to the system, and build your data pipeline with ease.

CMAK

Cluster Manager for Apache Kafka (CMAK) is an open-source platform which provides the Apache Kafka cluster management functions, such as creating and changing clusters and topics and checking consumer groups.

Cloud Data Streaming Service components

A Cloud Data Streaming Service cluster consists of at least four servers: one manager nodes and three or more broker nodes. The number of broker nodes can be increased as needed.

• Manager node
  The manager node manages Apache Kafka clusters through CMAK. It provides convenience for Kafka operation and management such as creating and deleting topics, adding partitions, etc. You can access CMAK through a public domain.

• Broker node
  A broker node is a server where actual data is saved. To access a broker node, the user's VM must be located in the same VPC as the VPC where the Cloud Data Streaming Service cluster is created.

• Topic
  It's a unit of identification for messages exchanged between the producer and consumer through Kafka.

• Partition
  It's topics divided for fast handling of large-volume messages.