Create and manage cluster

Available in VPC

Cluster interface

The basics of using the cluster menu are as follows:

Create cluster

1. From the NAVER Cloud Platform console's VPC environment, navigate to > Services > Big Data & Analytics > Cloud Data Streaming Service.
2. Click [Create cluster] in the Cloud Data Streaming Service menu.
3. Enter the cluster settings information, and then click [Next].
   
   • Cluster name: Enter the cluster name.
   • Application version: Select Kafka version and CMAK version.
   • ACG setting: Automatically create ACG with cdss-cluster code.
   • Config Group setting: Select the Config Group to apply to the cluster. For information on how to create Config Groups, see Create and manage Config Group.
   • Kafka log collection: Information text for collecting Kafka log. Kafka logs are available in Cloud Log Analytics.
   • Kafka broker port: Port used for communicating with the broker. It is automatically configured as 9092, and cannot be changed.
   • Kafka broker TLS port: Enables encrypted communication with the broker. It is automatically configured as 9093, and cannot be changed.
   • Zookeeper port: Port for communicating with the Zookeeper client. It is automatically configured as 2181, and cannot be changed.
   • CMAK port: It is automatically configured as 9000, and cannot be changed.
   • CMAK access permission: Enter the ID and password for access to CMAK. The ID cannot be changed.
4. Enter the node information in the set node step, and then click [Next].
   
   • OS type: Select the OS type to use.
   • VPC: Select the VPC to use.
   • Manager node subnet: Select the subnet of the manager node server. You can select from public/private subnets.
   • Number of manager nodes: The number of manager node servers. The default value 1 is automatically set, and cannot be changed.
   • Manager node server type: The server types you can choose from.
   • Broker node subnet: Select the subnet of the broker node. You can select from private subnets.
   • Number of broker nodes: The number of broker node servers. The default value is 3, and between 3 to 10 nodes can be created.
   • Broker node server type: The server types you can choose from.
   • Broker node storage capacity: Set the storage capacity for broker nodes. It uses a separate Block Storage rather than the OS storage, and up to 2,000 GB of Block Storage per broker node can be used.
5. Check the information entered, and then click [Create cluster].
   • It may take several minutes or longer to complete the cluster creation.

Manage cluster

This section describes how to add a broker node to a cluster, how to change node specifications, how to change the public endpoint of a broker node to access CMAK, and how to change the CMAK password.

Add broker node

If you need additional broker nodes while operating a cluster, you can add broker nodes. The number of added broker nodes cannot be decreased. You can only add new broker nodes.

To add a broker node:

1. From the NAVER Cloud Platform console's VPC environment, navigate to > Services > Big Data & Analytics > Cloud Data Streaming Service.
2. Select the check box of the cluster to which you want to add a broker node, and then click Manage cluster > Add broker node.
3. When the Add broker node window appears, enter the number of nodes you want to add.
   • If you select 1 for the number of nodes to add, then 1 broker node is added to the existing 3 nodes.
   • Up to 10 nodes can be added at once.
     
4. Click [OK].
   • The broker node expansion starts, and the cluster's server status is displayed as Changing.
5. Check if the cluster's server status changes to Running.

Change node specifications

If you need to change the specifications of the manager or broker node during cluster operation, you can change the specifications of the node. You may upgrade the current CPU and memory specifications to a higher level, but lowering the specifications is not allowed.

To change the node specifications:

1. From the NAVER Cloud Platform console's VPC environment, navigate to > Services > Big Data & Analytics > Cloud Data Streaming Service.
2. After clicking the check box of the cluster to which you want to change the node specification, click Manage cluster > Change Node specifications.
3. When you see the Change node specifications window, select the node specification to change.
   • You can select the specifications of manager and broker nodes respectively.
   • You cannot select specifications lower than those currently in use.
4. Click [OK].
   • The cluster's server status is displayed as Changing.
   • When the node specification change is complete, the cluster's server status is displayed as Running.

Node disk size

If you need to change the disk size of a broker node while operating a cluster, you can modify the node's disk size.

To change the disk size of a broker node:

1. From the NAVER Cloud Platform console's VPC environment, navigate to > Services > Big Data & Analytics > Cloud Data Streaming Service.
2. After clicking the check box of the cluster to which you want to change the node disk size, click Manage cluster > Change node disk size.
3. When the Change node disk size window appears, enter the desired disk size for the node.
   • Disk sizes can only be adjusted in increments of 10 GB.
   • You cannot change the disk size to a value smaller than those currently in use.
4. Click [OK].
5. When a popup window for a node disk size change appears, and when the message The change has been applied. is displayed, click [OK].
   • The cluster's server status is displayed as Changing.
   • During the node disk size change process, the cluster's service will be interrupted.
   • When the node disk size change is complete, the cluster's server status is displayed as Running.

Change broker node public endpoint settings

You can add a dedicated public endpoint listener and advertised listener to the broker node's configuration file. Before changing the broker node public endpoint settings, you must complete the load balancer setting by referring to Load Balancer setting.

To change the public endpoint settings of a broker node:

1. From the NAVER Cloud Platform console's VPC environment, navigate to > Services > Big Data & Analytics > Cloud Data Streaming Service > Cluster.
2. Select the check box of the cluster of which you want to change the public endpoint settings, and then click Manage cluster > Change broker node public endpoint settings.
   • It displays the load balancer list that you can set as the public endpoint of the broker node.
3. Select a load balancer.
   
   • You can only select load balancers using Network Type and Public IP.
   • You cannot select a load balancer being used in another cluster.
4. Click [OK].

Access CMAK

For more information, see Access CMAK.

Reset CMAK connection password

If you have forgotten the CMAK access password, then you can change it.
To change the CMAK access password:

1. From the NAVER Cloud Platform console's VPC environment, navigate to > Services > Big Data & Analytics > Cloud Data Streaming Service > Cluster.
2. Click Manage cluster > Reset CMAK access password.
3. Enter the new password, and then click [OK].
   • It may take several minutes for the new password to be applied.

Change CMAK access domain settings

Before accessing CMAK, you need to activate the public domain that can access the CMAK server.
To change the CMAK access domain settings:

1. From the NAVER Cloud Platform console's VPC environment, navigate to > Services > Big Data & Analytics > Cloud Data Streaming Service > Cluster.
2. Click Manage cluster > Change CMAK access domain settings.
3. When a popup window appears, check the text for activation or deactivation, and click [OK].
   
   • If the public domain is inactive, you can activate the public domain.
   • If the public domain is active, you can deactivate the public domain. External connections to CMAK will be blocked.

Restart cluster

If an issue occurs on the cluster status, you can restart CMAK, Kafka, and Zookeeper.

Restart all services

To restart Kafka, Zookeeper, and CMAK at once:

1. From the NAVER Cloud Platform console's VPC environment, navigate to > Services > Big Data & Analytics > Cloud Data Streaming Service > Cluster.
2. Click Restart cluster > Restart all services.

Restart Kafka/Zookeeper

To restart Kafka and Zookeeper:

1. From the NAVER Cloud Platform console's VPC environment, navigate to > Services > Big Data & Analytics > Cloud Data Streaming Service > Cluster.
2. Click Restart cluster > Restart Kafka & Zookeeper.

Restart Kafka by node

To restart Kafka by node:

1. From the NAVER Cloud Platform console's VPC environment, navigate to > Services > Big Data & Analytics > Cloud Data Streaming Service > Cluster.
2. Click Restart cluster > Restart Kafka by node.
3. Click the check box of the node to restart Kafka and click [OK].
   
   • Kafka of the selected node is restarting.

Restart CMAK

To restart CMAK:

1. From the NAVER Cloud Platform console's VPC environment, navigate to > Services > Big Data & Analytics > Cloud Data Streaming Service > Cluster.
2. Click Restart cluster > Restart CMAK.

Load balancer settings and ACG settings

The following describes how to create and set up a load balancer to set as a public endpoint of a broker node.

Create target group

To create a target group:

1. From the NAVER Cloud Platform console's VPC environment, navigate to > Services > Big Data & Analytics > Cloud Data Streaming Service.
2. From Cluster details, click [View details] under Broker node information.
3. When the broker node information window appears, check the information of Public Endpoint PlainText Listener Port.
   • Check the port corresponding to each broker node.
   • Check the information of the Public Endpoint TLS Listener Port in case of using encrypted communication.
     
4. From the NAVER Cloud Platform console's VPC environment, navigate to > Services > Networking > Load Balancer > Target Group.
5. Click Create target group.
6. Enter the target group information, and then click [Next].
   
   • Target group name: Enter the target group's name.
   • Target group type: Select target group type and select VPC Server.
   • VPC: Select the same VPC as the VPC where the cluster was created.
   • Protocol: Select the TCP protocol.
   • Port: Enter one of the ports corresponding to the broker node as checked in Step 3.
     • You need target groups created with the ports corresponding to their broker nodes.
     • If there are 3 broker nodes, then you'd need 3 target groups in total.
7. Enter information for the health check setting, and then click [Next].
   
   • Protocol: Select the TCP protocol.
   • Port: Enter the port you entered in the previous step.
8. Select the target you want to apply to the target group, and then click >.
   
   • Select the broker node corresponding to the port entered in the previous step.
   • The corresponding ports to the broker nodes can be viewed by clicking [View details] under Broker node information.
9. Click [Next].
10. After checking the information you configured, click [Create target group].
11. Repeat the above process as many times as the number of broker nodes to create target groups.

Create load balancer

To create a load balancer:

1. From the NAVER Cloud Platform console's VPC environment, navigate to > Services > Networking > Load Balancer > Load Balancer.
2. Click [Create load balancer] > [Create network load balancer].
   • If you create another type of load balancer, it will not be exposed to Cloud Data Streaming Service.
3. Enter the load balancer creation information, and then click [Next].
   
   • Load balancer name: Enter the load balancer name.
   • Network: Select Public IP.
   • Load balancing performance: Select load balancer size depending on the load balancing performance.
   • Target VPC: Choose the same VPC as the one you selected when creating the Cloud Data Streaming Service Cluster.
   • Select subnet: Select the subnet to be used by the load balancer.
4. In the set listener field, enter one of the ports that correspond to broker nodes, and then click [Add].
   • For the remaining ports, add them using the add listener feature after completing the load balancer creation.
     
5. Click [Next].
6. Select the target group that corresponds to the load balancer that you registered in the set listener step, and then click [Next].
   
7. Check if the created information is correct, and then click [Create load balancer].

Set target group listener

You need to complete the listener settings for the remaining target groups.

1. From the NAVER Cloud Platform console's VPC environment, navigate to > Services > Networking > Load Balancer > Load Balancer.
2. Select the check box load balancer to change the listener settings for, and then click [Change listener settings].
3. Click [Add listener].
   
4. When the add listener interface appears, enter the information.
   
   • Protocol: Select TCP.
   • Port: Enter the port corresponding to the broker node.
   • Target group: Select the target group corresponding to the port.
5. Click [OK]. A listener is added.
6. Add listeners for all target groups created in the Create target group step.

Configure ACG

You need to set ACG rules for IPs so that the broker nodes can be accessed from the load balancer and the outside.

To configure ACG rules:

1. From the NAVER Cloud Platform console's VPC environment, navigate to > Services > Big Data & Analytics > Cloud Data Streaming Service > Cluster.
2. From the details component of the cluster you want to change the ACG settings for, click the image of broker node ACG.
   • Move to set ACG interface.
     
3. Select the broker node ACG from the ACG list and click [Set ACG].
   • The set ACG rule window appears.
     
4. Create an ACG rule for the IP to access the load balancer and the broker node from the outside.
   
   • Protocol: Select TCP.
   • Access source: Enter the IP range that needs access to the broker nodes.
     • Enter the IP address range of the load balancer dedicated subnet to allow access from the load balancer.
     • [myip]: You can check your public IP address and register the IP to the access source.
   • Allowed port: Enter the allowed ports. You can either enter a specific port or a range.
5. After checking the added rules, click [Apply].
   • The rule is applied to ACG.

Delete cluster

To delete a cluster:

1. From the NAVER Cloud Platform console's VPC environment, navigate to > Services > Big Data & Analytics > Cloud Data Streaming Service.
2. In the Cluster menu, select the check box of the cluster to delete, and then click [Delete].
3. When the Delete cluster window appears, click [OK].