Create and manage cluster

Available in VPC

Cluster page

The basics of using the cluster menu are as follows:

Create cluster

1. Access NAVER Cloud Platform console.
2. Click VPC from the Platform menu to switch to the VPC environment.
3. Click Services > Big Data & Analytics > Cloud Data Streaming Service in order.
4. Click [Create cluster] in the Cloud Data Streaming Service menu.
5. Enter the cluster settings information, and then click the [Next] button.
   
   • Cluster name: enter the cluster name
   • Application version: select Kafka version, CMAK version
   • ACG setting: automatically create ACG with cdss-cluster code
   • Config group settings: select the config group to apply to the cluster. For information on how to create config groups, see Create and manage config groups
   • Kafka log collection: information text for collecting Kafka log. Kafka logs are available in Cloud Log Analytics
   • Kafka broker port: it is the port for communicating with the broker. It is automatically configured as 9092, and can't be changed
   • Kafka broker TLS port: it is a port for encrypting communication with the broker. It is automatically configured as 9093, and can't be changed
   • Zookeeper port: it is the port for communicating with the Zookeeper client. It is automatically configured as 2181, and can't be changed
   • CMAK port: it is automatically configured as 9000, and can't be changed
   • CMAK access permission: enter the ID and password for the access to CMAK. The ID cannot be changed.
6. Enter the node information in the set node step, and then click the [Next] button
   
   • OS type: select the OS type to use
   • VPC: select the VPC to use
   • Manager node subnet: select the subnet of the manager node server. You can select from public/private subnets
   • Number of manager nodes: it refers to the number of manager node servers. The default value 1 is automatically set, and can't be changed
   • Manager node server type: it is the types of servers you can select
   • Broker node subnet: select the subnet of the broker node server. You can select from private subnets
   • Number of broker nodes: it refers to the number of broker node servers. The default value is 3, and between 3 to 10 nodes can be created
   • Broker node server type: it is the types of servers you can select
   • Broker node storage capacity: set the storage capacity for broker nodes. It uses a separate Block Storage rather than the OS storage, and up to 2000 GB of Block Storage per broker node can be used
7. Check the information entered, and then click the [Create cluster] button.
   • It may take several minutes or longer for the completion of cluster creation.

Manage cluster

It describes how to add a broker node to a cluster, how to change node specifications, how to change the public endpoint of a broker node to access CMAK, and how to change the CMAK password.

Add broker node

If you need additional broker nodes while operating a cluster, you can add broker nodes. The number of added broker nodes can't be decreased. You can only add new broker nodes.

To add a broker node, follow these steps:

1. From the NAVER Cloud Platform console, click Services > Big Data & Analytics > Cloud Data Streaming Service in order.
2. Select the checkbox of the cluster to which you want to add a broker node, and then click Manage cluster > Add broker node.
3. When the add broker node window appears, enter the number of nodes you want to add.
   • If you select 1 for the number of nodes to add, then 1 broker node is added to the existing 3 nodes.
   • Up to 10 nodes can be added at once.
     
4. Click the [OK] button.
   • The broker node expansion starts, and the cluster's server status is displayed as Changing.
5. Check if the server status changes to Running.

Change node specifications

If you need to change the specifications of the manager or broker node during cluster operation, you can change the specifications of the node. You may not reduce the specifications from the currently used CPU and memory specifications; only upgrades to higher specifications are allowed.

To change the node specifications, follow these steps:

1. From the NAVER Cloud Platform console, click Services > Big Data & Analytics > Cloud Data Streaming Service in order.
2. After clicking the checkbox of the cluster to which you want to change the node specification, click Manage cluster > Change Node specifications.
3. When you see the node specification change window, select the node specification to change.
   • You can select the specifications of manager and broker nodes respectively.
   • You cannot select specifications lower than those currently in use.
4. Click the [OK] button.
   • The cluster's server status is displayed as Changing.
   • When the node specification change is complete, the cluster's server status is displayed as Running.

Change broker node public endpoint settings

You can add a dedicated public endpoint listener and advertised listener to the broker node's config file. Before changing the Broker Node Public Endpoint settings, you must complete the load balancer setting by referring to Load Balancer setting.

To change the Public Endpoint settings of a broker node, follow these steps:

1. From the NAVER Cloud Platform console, click Services > Big Data & Analytics > Cloud Data Streaming Service > Cluster in order.
2. Select the checkbox of the cluster of which you want to change the public endpoint settings, and then click Manage cluster > Change broker node public endpoint settings.
   • It displays the load balancer list that you can set as the public endpoint of the broker node.
3. Select load balancer.
   
   • You can select only Load Balancers using Network Type and Public IP.
   • You can't select Load Balance being used in another cluster.
4. Click the [OK] button.

CMAK connection

For more information, see Access CMAK.

Reset the CMAK connection password

If you have forgotten the CMAK access password, then you can change it.
To change the CMAK access password, follow these steps:

1. From the NAVER Cloud Platform console, click Services > Big Data & Analytics > Cloud Data Streaming Service > Cluster in order.
2. Click Manage cluster > Reset CMAK access password.
3. Enter the new password, and then click the [OK] button.
   • It may take several minutes for the application of the changed password.

Editing the CMAK connection domain settings

Before accessing CMAK, you need to activate the public domain that can access the CMAK server.
To change the CMAK access domain settings, follow these steps:

1. From the NAVER Cloud Platform console, click Services > Big Data & Analytics > Cloud Data Streaming Service > Cluster in order.
2. Click Manage cluster > Change CMAK access domain settings.
3. When a pop-up window appears, check the text for activation or deactivation, and click the [OK] button.
   
   • If the public domain is inactive, you can activate the public domain.
   • If the public domain is active, you can deactivate the public domain. External connections to CMAK will be blocked.

Restart cluster

If an issue occurs on the cluster status, you can restart CMAK, Kafka, and Zookeeper.

Restart all services

To restart Kafka, Zookeeper, and CMAK at once, follow these steps:

1. From the NAVER Cloud Platform console, click Services > Big Data & Analytics > Cloud Data Streaming Service > Cluster in order.
2. Click Restart cluster > Restart all services.

Restart Kafka/ Zookeeper

To restart Kafka and Zookeeper, follow these steps:

1. From the NAVER Cloud Platform console, click Services > Big Data & Analytics > Cloud Data Streaming Service > Cluster in order.
2. Click Restart cluster > Restart Kafka & Zookeeper.

Restart Kafka by node

To restart Kafka by node, follow these steps:

1. From the NAVER Cloud Platform console, click Services > Big Data & Analytics > Cloud Data Streaming Service > Cluster in order.
2. Click Restart cluster > Restart Kafka by node
3. Click the checkbox of the node to restart Kafka and click the [OK] button.
   
   • Kafka of the selected node is restarting.

Restart CMAK

To restart CMAK, follow these steps:

1. From the NAVER Cloud Platform console, click Services > Big Data & Analytics > Cloud Data Streaming Service > Cluster in order.
2. Click Restart cluster > Restart CMAK

Load balancer settings and ACG settings

The following describes how to create and set up a load balancer to set as a public endpoint of a broker node.

Create target group

To create a target group, follow these steps:

1. From the NAVER Cloud Platform console, click Services > Big Data & Analytics > Cloud Data Streaming Service in order.
2. From Cluster details, click the [View details] button of Broker node information.
3. When the broker node information window appears, check the information of Public Endpoint PlainText Listener Port.
   • Check the port corresponding to each broker node.
   • Check the information of the public endpoint TLS listener Port in case of using encrypted communication.
     
4. From the NAVER Cloud Platform console, click Services > Networking > Load Balancer > Target Group in order.
5. Click the Create target group button.
6. Enter the target group information, and then click the **[Next]**button.
   
   • Target group name: enter the target group's name
   • Target group type: select target group type and select VPC Server
   • VPC: select the same VPC as the VPC where the cluster was created
   • Protocol: select the TCP protocol
   • Port: enter one of the ports corresponding to the broker node as checked in Step 3
     • You need target groups created with the ports corresponding to their broker nodes.
     • If there are 3 broker nodes, then you'd need 3 target groups in total.
7. Enter information for the health check setting, and then click the [Next] button.
   
   • Protocol: select the TCP protocol
   • Port: enter the port you entered in the previous step
8. Select the target you want to apply to the target group, and then click the > button.
   
   • Select the broker node corresponding to the port entered in the previous step.
   • The corresponding ports to the broker nodes can be viewed by clicking the [View details] button of Broker node information.
9. Click the [Next] button.
10. After checking the information you configured, click the [Create target group] button.
11. Repeat the above process as many times as the number of broker nodes to create target groups.

Create load balancer

To create the load balancer, follow these steps:

1. From the NAVER Cloud Platform console, click Services > Networking > Load Balancer > Load Balancer in order.
2. Click the [Create load balancer] > [Create network load balancer], in order.
   • If you create another type of load balancer, it will not be exposed to Cloud Data Streaming Service.
3. Enter the load balancer creation information, and then click the [Next] button.
   
   • Load balancer name: enter the load balancer name
   • Network: select Public IP
   • Load balancing performance: select load balancer size depending on the load balancing performance
   • Target VPC: choose the same VPC as the one you selected when creating the Cloud Data Streaming Service Cluster
   • Select subnet: select the subnet dedicated for load balancer
4. In the set listener field, enter one of the ports that correspond to broker nodes, and then click the [Add] button.
   • For the remaining ports, add them using the add listener feature after completing the load balancer creation.
     
5. Click the [Next] button.
6. Select the target group which corresponds to the load balancer which you registered in the set listener step, and then click the [Next] button.
   
7. Check if the created information is correct, and then click the [Create load balancer] button.

Set target group listener

You need to complete the listener settings for remaining target groups.

1. From the NAVER Cloud Platform console, click Services > Networking > Load Balancer > Load Balancer in order.
2. Select the checkbox load balancer to change the listener settings for, and then click the [Change listener settings] button.
3. Click the [Add Listener] button
   
4. When the add listener page appears, enter the information.
   
   • Protocol: select TCP
   • Port: enter the port corresponding to the broker node
   • Target group: select the target group corresponding to the port
5. Click the [OK] button. A listener is added.
6. Add listeners for all target groups created in the Create Target Group step.

Set ACG

You need to set ACG rules for IPs so that the Broker nodes can be accessed from the load balancer and the outside.

To set ACG rules, follow these steps:

1. From the NAVER Cloud Platform console, click Services > Big Data & Analytics > Cloud Data Streaming Service > Cluster in order.
2. From the details area of the cluster you want to change the ACG settings for, click the image of Broker node ACG.
   • Move to set ACG page.
     
3. Select the broker node ACG from the ACG list and click the [Set ACG] button.
   • The set ACG rule window appears.
     
4. Create an ACG rule for the IP to access the load balancer and the broker node from the outside.
   
   • Protocol: select TCP
   • Access source: enter the IP range which needs access to the broker nodes
     • Enter the IP address range of the load balancer dedicated subnet to allow access from the load balancer
     • [myip]: you can check your public IP address and register the IP to the access source
   • Allowed port: enter the allowed ports. You can either enter a specific port of a range
5. After checking added rules, click the **[Apply]**button.
   • This rule applies to ACG.

Delete cluster

To delete clusters, follow these steps:

1. From the NAVER Cloud Platform console, click Services > Big Data & Analytics > Cloud Data Streaming Service in order.
2. In the Cluster menu, select the checkbox of the cluster to delete, and then click the [Delete] button.
3. When the Delete cluster window appears, click the [OK] button.