Integrate Cloud DB for MySQL with external Hive Metastore (optional)

Available in VPC

This guide introduces how to integrate NAVER Cloud Platform’s Cloud DB for MySQL with Cloud Hadoop's External Hive Metastore.

Preparations

1. Please generate a Cloud Hadoop cluster.
   • For more information on generating Cloud Hadoop clusters, see Start Cloud Hadoop guide.
2. Please generate Cloud DB for MySQL.
   • Please generate a MySQL 5.7.x version as it can be integrated with the MySQL 5.7.x version.
   • For more information on generating Cloud DB for MySQL, see Start Cloud DB for MySQL Server Guide.
3. Please set up ACG.
   • In order to access the Cloud DB for MySQL service from Cloud Hadoop, please enter Cloud Hadoop ACG in the access source of Cloud DB for MySQL ACG and allow all communication ports (1-65535).
     

External Hive Metastore

Integrate external Hive Metastores

Hive Metastore server uses DB storage to manage metadata. The default settings are configured to use MySQL installed within Cloud Hadoop.
By using the external Hive Metastore integration method, Cloud DB for MySQL can be used by changing it to the metadata storage in the Hive Metastore server.
You need to check whether there is a Hive Job in progress as the Hive Metastore server is automatically restarted according to the Metastore storage replacement operation.
The replacement of metadata storage does not include the transfer of existing metadata.

How to change to the external Metastore is as follows:

1. From NAVER Cloud Platform console, click the Services > Big Data & Analytics > Cloud Hadoop menus sequentially.
2. After selecting a cluster to be integrated from the cluster list, click [Cluster Management] > External Hive Metastore Integration.
3. After entering the User_ID and password, click the [Test Connection] button.
   • There may be restrictions on usage depending on the password plugin settings of Cloud DB for MySQL.
     You must manage and integrate the Cloud DB for MySQL account in accordance with the Cloud Hadoop external Hive Metastore account policy.
4. After checking whether the test connection is normal, click the [Yes] button.
   • For a successful test connection, you need to perform the following items in advance.
   • After running the test connection, you can check the content of the console message to see whether it was successful.
     • Normal: connection test to Cloud DB for MySQL has been completed successfully.
     • ACG blocked: access to the DB server from Cloud Hadoop is blocked. Please revise the ACG settings in Cloud DB for MySQL.
     • Incorrect account ID and password: the account ID or password has been entered incorrectly. Please enter the correct account name and password.
     • Failure to comply with password restrictions: user password must contain a minimum of 8 letters and a maximum of 20 letters, and must contain at least 1 English uppercase and lowercase letter, special character, and number. Special characters such as ', ", `, |, /, &, +, or space are not allowed.
     • Account DDL not in possession: the account you entered cannot perform DDL commands. Please modify the DB permission with DDL.
     • If there is no DB generation procedure in Cloud DB for MySQL: database control is not smooth. Check whether the procedure sys.ncp_create_db is working properly.
     • In a case where Cloud DB for MySQL is already integrated: the requested DB service is currently integrated with Cloud Hadoop. Please select another DB service.
     • When the master node is down: the status of Cloud Hadoop’s master node is not normal. A server inspection is needed.
     • Unknown error: please make an inquiry to Customer Support.
     • In a case where Hive is deleted from the Customer's cluster: Hive is not installed in Cloud Hadoop. Please check the cluster.
5. Once the task of integrating external Hive Metastore starts, the Status field in the cluster list or detailed information becomes Being set up, and then displays again as Running once the task is completed.