Getting started

Available in VPC

Once you've reviewed the Cloud Hadoop specifications, and quickstart, you're ready to start using the service. Your first step is creating a Cloud Hadoop cluster. You can create and manage Cloud Hadoop clusters from the NAVER Cloud Platform console.
The following summarizes what you can learn from this guide:

• Preliminary task
• Create cluster
• Delete cluster

Preliminary task

1. Create Object Storage.
   Before creating a cluster you must have created an Object Storage bucket for storing and searching data. For more information, see Object Storage guides.

2. Create VPC and Subnet
   In the NAVER Cloud Platform console, navigate to Menu > All Services > Networking > VPC and create a VPC and Subnet. For more information, see VPC user guides. Regardless of the number of clusters, at least 1 VPC is required. You can have multiple clusters in the same VPC and use them. In a private VPC environment, you can create VPC only in the KR-2 Region.
   When creating Cloud Hadoop, you can create and use public subnets and private subnets according to the node's purpose. In the VPC environment, you can set edge nodes and master nodes as public subnets or private subnets and worker nodes only as private subnets. When creating Cloud Hadoop, the number of edge nodes is fixed at 1 and the number of master nodes is fixed at 2. In this guide, 1 private subnet and 2 public subnets were created.

3. Select a node type.
   Consider the expected usage and select a node type in advance.

Create cluster

To use NAVER Cloud Platform's Cloud Hadoop, you must create a cluster first.

To create a Cloud Hadoop cluster:

1. Access the NAVER Cloud Platform console.
2. In the top-right corner, click Region & Platform.
3. Select your region and VPC platform, and then click [Apply].
4. In the top-left corner, click Menu.
5. Navigate to All Services > Big Data & Analytics > Cloud Hadoop.
6. Click [Create cluster].
7. When the Create cluster page appears, proceed with the following steps in order:
   • 1. Set up a cluster
   • 2. Set storage and server
   • 3. Set authentication key
   • 4. Final confirmation

1. Set up a cluster

Specify the cluster settings information and then click [Next].

• Cluster version: Currently, Cloud Hadoop versions 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 2.0, 2.1, 2.2, and 2.3 are provided. For more information on cluster versions, see Cloud Hadoop release notes.
• Cluster type: There are currently four cluster types: Core Hadoop, Presto, HBase, and Spark. You can select a type with the components you need pre-installed. If you need to add necessary services, you can use the Add Service feature in Ambari, which is a cluster managing tool.
• Cluster add-on: In addition to the basic type, you can select additional components to install as options.
• Use catalog of Data Catalog: use Data Catalog's catalog to provide Cloud Hadoop Hive Metastore.
• Kerberos authentication configuration: Select if you want to configure a Secure Hadoop cluster using Kerberos. Realm is the authentication management domain. Kerberos Distribute Center (KDC) will be configured as the following set values:
  • Realm: KDC's Realm information KDC admin (can only use capital letters for Realm names)
  • KDC manager account password: KDC admin account’s password
• VPC: Select the VPC created in Preliminary task.
• Cluster admin account: Set the cluster account for accessing the management console of Ambari, Hue, and Zeppelin.
• Cluster admin account password: Enter the cluster admin account's password.
• Set ACG: Cloud Hadoop ACG is automatically created whenever you create a cluster. To set up network ACLs, you can select an automatically generated ACG and modify the rules. For more information about setting up ACGs, see the Firewall settings (ACG) guide.

2. Set storage and server

After specifying the storage and node server settings information, click [Next].

• Object Storage bucket: You can read and write data from the Object Storage bucket that you created in a preliminary task. When creating a cluster, select the Object Storage bucket created in a preliminary task. Locked buckets cannot be integrated with Cloud Hadoop. Keep this in mind when creating an Object Storage bucket.
• Bootstrap script: This feature is provided in the Cloud Hadoop 1.6 or higher. Bootstrap script feature is a function that runs the shell script uploaded to the Object Storage bucket integrated with Cloud Hadoop when creating Cloud Hadoop. To enable it, mark the check box and enter the file name of the uploaded shell script. The script will not be executed if the file does not exist in the Object Storage bucket or if an incorrect file name is entered. You can find bootstrap execution logs in the same path as the shell script uploaded to the Object Storage bucket.
• Support high availability: Cloud Hadoop basically provides redundancy for HDFS NameNode, YARN Resource Manager, Oozie Server, and HiveServer. Since this is the specification that is required as a minimum, it cannot be deselected.
• Edge node server type: Select a server type to use as a edge node. For the specifications of servers that can be used as edge nodes, see Supported server specifications by cluster node.
• Edge node subnet: Select the subnet to locate the edge node.
• Number of edge nodes: The number of edge nodes is fixed at 1.
• Master node subnet: Select the subnet to locate the master node.
• Master node server type: Select a server type to use as a master node. For the specifications of servers that can be used as master nodes, see Supported server specifications by cluster node.
• Number of master nodes: Since Cloud Hadoop provides high availability as the minimum specifications, the number of master nodes is fixed at 2.
• Master node storage type: Select a storage type. You can select between SSD and HDD. You cannot change the storage type after you create a cluster.
• Master node storage capacity: Select the storage capacity. You can select a minimum of 100 GB and up to 2000 GB (in 10 GB units). 4000 GB and 6000 GB can also be selected.
• Worker node subnet: Select the subnet to locate the worker node.
• Server type of worker nodes: Select a server type to use as a worker node. For the specifications of servers that can be used as operator nodes, see Supported server specifications by cluster node.
• Number of worker nodes: You can select 2-8 worker nodes. Worker nodes can be added or deleted even after the cluster is created.
• Worker node storage type: Select a storage type. You can select between SSD and HDD. You cannot change the storage type after you create a cluster.
• Worker node storage capacity: Select the storage capacity. You can select a minimum of 100 GB and up to 2000 GB (in 10 GB units). 4000 GB and 6000 GB can also be selected.
• Pricing plan: The pricing plan you selected when you created your account applies. For more pricing information, see Pricing information.

3. Set authentication key

Set the SSH authentication key required for accessing the node yourself.
Select an authentication key you have or create a new one and click [Next].

• To create a new authentication key, select Create new authentication key, enter the authentication key name, and click [Create and save authentication key].

4. Final confirmation

Check the details and click [Create].

Delete cluster

To delete a Cloud Hadoop cluster:

1. In the VPC environment of the NAVER Cloud Platform console, navigate to Menu > All Services > Big Data & Analytics > Cloud Hadoop.
2. Select the cluster to delete from the cluster list, and then click [Delete].
3. Enter the cluster name on the popup window to confirm deletion, and then click [Yes].

Delete Object Storage file or bucket

Select the file to delete from the Object Storage console and click [Edit] > Delete.
For more information on deleting Object Storage files or buckets, see Object Storage user guides.