Using ZooKeeper

Prev Next

Available in VPC

Data Forest provides Apache ZooKeeper server. ZooKeeper is a distributed coordination system that helps manage tasks across applications. It includes zk-web, which allows users to view and manage znodes through the Web UI.

Check ZooKeeper app details

Once the app is created, you can view its details. If the Status in the app details is Stable, the app is running normally.

To view app details:

  1. In the VPC environment on the NAVER Cloud Platform console, navigate to i_menu > Services > Big Data & Analytics > Data Forest.
  2. Click Data Forest > Apps on the left.
  3. Select the account that owns the app.
  4. Click the app to view its details.
  5. Review the app details.
    df-zookeeper_2-1_updated_ko
    • Quick links: Endpoints exposed by the app components
      • zkweb-ui: Provides access to the Web UI, where you can configure ZooKeeper paths in the format "zkserver-0.app-name.service-account.cluster-domain:2181."
    • Component: The default values are the recommended resources.
      • zkweb: Runs zk-web.
      • zkserver: Runs Zookeeper.
Note
  • When creating a ZooKeeper app, the zkweb component is fixed at 1 container and the zkserver component is fixed at 3 containers under Configuration > Number of containers.

Example:
The following shows the zkweb UI access interface.
df-zookeeper_04_vpc_ko

After connecting successfully, you can view the znode details shown in the following.
df-zookeeper_10_vpc_ko

Integrate ZooKeeper with other apps

ZooKeeper can be integrated with apps such as Apache HBase, Apache Kafka, and OpenTSDB. To utilize apps such as HBase, Kafka, or OpenTSDB, you must create the ZooKeeper app first.

Check container's hostname

The container's hostname is assigned automatically when you create the app.
The number of containers for the zkserver component is set to 3 by default.
If a user named "test" uses an app named "zookeeper" to run ZOOKEEPER-3.4.13 with the zkserver component, the 3 containers executed will have the following hostnames:

  • zkserver-0.zookeeper.test.kr.df.naverncp.com
  • zkserver-1.zookeeper.test.kr.df.naverncp.com
  • zkserver-2.zookeeper.test.kr.df.naverncp.com

Configure the ZooKeeper app on the HBase app

To run the Apache HBase app, create the ZooKeeper app first.

Configure ZooKeeper for the Apache HBase app as "hbase.zookeeper.quorum" of the "hbase-site." Set it as the created ZooKeeper app's container. For more information, see Apache HBase Reference Guide.

Use ZooKeeper app with caution

Change the number of zkserver containers

As the address of an additional server must be included in the configuration of the currently running servers, it is difficult to change the number of zkserver containers without server downtime. For this reason, it is recommended to start the app with an appropriate number of containers from the beginning. Currently, the number of zkserver containers is fixed at 3. A feature that allows you to adjust the number of zkserver containers may be added in the future.

Note

If you have updated Configuration > Number of containers, find the "/conf/zoo.cfg" file under "files" and add or delete server X configurations according to the updated value. Additionally, update the env's DNS_REGISTRY_CHECK_HOSTS configuration to match the updated value.

Store ZooKeeper app data

  • The data (dataDir) of the ZooKeeper app created by the user is stored in the Data Forest's local file system. As long as the user does not delete (destroy) the app, the data is not removed.
  • If there are sufficient YARN queue resources, even if the user stops the ZooKeeper app and restarts it, the containers are re-launched on the same nodes, allowing access to the previous znode data.