--- title: "Cloud Hadoop overview" slug: "hadoop-vpc-1" tags: ["Cloud Hadoop", "VPC"] updated: 2026-10-08T07:04:06Z published: 2026-10-08T07:04:06Z canonical: "guide.ncloud-docs.com/hadoop-vpc-1" --- > ## Documentation Index > Fetch the complete documentation index at: https://guide.ncloud-docs.com/llms.txt > Use this file to discover all available pages before exploring further. # Cloud Hadoop overview Available in VPC Cloud Hadoop is a fully managed cloud analysis service where you can freely use open source-based frameworks such as Apache Hadoop, HBase, Spark, Hive, and Presto to process big data easily and quickly. Direct server access through a terminal is allowed, and you can directly manage it through the convenient cluster management feature provided through Ambari. You can easily configure the initial infrastructure with the Cloud Hadoop service on NAVER Cloud Platform. Stability, flexible scalability, and availability for services and tasks are secured by 2 master nodes provided including availability of node expansion/reduction at any time. In addition, you can analyze large-scale data with various frameworks and server types supported, and clusters can be controlled by management and monitoring through Web UI. ## Cloud Hadoop features - User convenience - Cloud Hadoop automatically supports cluster creation, reducing the burden on infrastructure management. - You can secure a system capable of analyzing anytime through the installation, configuration, and optimization processes of various open source frameworks. - Cost efficiency - It is an efficient service where you only pay as much as you used from the start to the end point of the cluster. - Cloud Hadoop saves large-scale data at a low cost by using NAVER Cloud Platform's Object Storage as its storage for data. - Flexible scalability and stability - You can easily reduce or increase the number of instances needed for analyzing data at the desired time. - 2 master nodes are provided for higher stability and availability of services and tasks. - Various frameworks supported - Hadoop: Framework that can distribute and process large-scale data sets across the whole computer clusters using a simple programming model - HBase: large-scale data storage that can be distributed and expanded - Spark: Integrated analysis engine for processing large-scale data - Hive: Data warehouse software for reading, inserting, and managing large-scale data sets in dispersed storage using SQL - Presto: Dispersed SQL query engine for big data - Web UI provided for management and monitoring - A UI is provided for managing the information and status of the Cloud Hadoop cluster. - Provision of root access permissions for clusters enables complete control over the clusters, as well as allows the setting values of the framework to be checked or edited. ## Cloud Hadoop user guide - [Cloud Hadoop overview](/docs/en/hadoop-vpc-1): Learn about the services, features, benefits, FAQs and find helpful resources. - [Cloud Hadoop quickstart](/docs/en/cloudhadoop-vpc-procedure): Follow a step-by-step guide through the entire quickstart. - [Cloud Hadoop prerequisites](/docs/en/cloudhadoop-vpc-prep): View specifications. - [Getting started with Cloud Hadoop](/docs/en/hadoop-vpc-2): Learn how to create Cloud Hadoop in the NAVER Cloud Platform console. - Using Cloud Hadoop: Learn how to access Cloud Hadoop. - Utilizing the Cloud Hadoop ecosystem: Learn how to utilize the applications provided by Cloud Hadoop. - Cloud Hadoop integrations: Learn how to integrate Cloud Hadoop with an external system. - [Cloud Hadoop resource management](/docs/en/cloudhadoop-vpc-resource): View Cloud Hadoop resource information. - [Cloud Hadoop permissions management](/docs/en/cloudhadoop-vpc-subaccount): Learn how to manage permissions and policies. - [Cloud Hadoop release notes](/docs/en/cloudhadoop-vpc-release): See documentation updates. ## Cloud Hadoop related resources Beyond the user guide, these resources provide additional context and support for Cloud Hadoop. Whether you're considering Cloud Hadoop or need in-depth information for development, marketing, and other purposes, these resources can help: - API guide: Instructions for developers. - [Cloud Hadoop (VPC) API guide](https://api.ncloud-docs.com/docs/en/analytics-vhadoop) - CLI guide: Instructions for developers. - [Cloud Hadoop (VPC) CLI guide](https://cli.ncloud-docs.com/docs/en/cli-vhadoop) - [Cloud Hadoop easy start guide](https://www.ncloud.com/guideCenter/guide/59): Basic usage of Cloud Hadoop. - [Sub Account user guide](https://guide.ncloud-docs.com/docs/en/subaccount-overview): Sub Account guide for those who need administrator accounts of various authority levels for managing Cloud Hadoop. - [Ncloud user environment guide](https://guide.ncloud-docs.com/docs/en/environment-environment-1-1): Explore VPC environments and supported services. - [Pricing plans, characteristics, detailed features](https://www.ncloud.com/product/analytics/cloudHadoop) - [Latest announcements](https://www.ncloud.com/intro/news): Stay informed about service updates and news. - [FAQs](https://www.ncloud.com/support/faq/prod?categoryCode2=ANALY&categoryCode3=SV_0086): Get answers to common Cloud Hadoop questions. - [Contact Support](https://www.ncloud.com/support/question): Get help if you can't find what you need in the user guide. ## FAQs. Q. Which cluster node types are provided by Cloud Hadoop? A. Cloud Hadoop clusters are sets of clusters or nodes, configured for distributed storage and analysis of data. Depending on the purpose of the node, there are 3 types of nodes in a cluster. - Edge node: Gateway node for external connections - Master node: Admin node for monitoring worker nodes. 2 master nodes are created with high availability support, and the number of master nodes cannot be changed - Worker node: Node that receives commands from master nodes and actually performs tasks such as data analysis. You can initially create 2 to 8 nodes. More nodes can be added/deleted dynamically afterward. Q. How is Cloud Hadoop service configured? A. Cloud Hadoop is a service for building and managing clusters easily and conveniently. You can create components such as Hadoop, HBase, Spark, and Presto and build and operate a system for processing large-scale data. You can install open source frameworks that can process large amounts of data such as Apache Hadoop, HBase, Hive, and Spark on the cluster. For the configuration of the Cloud Hadoop service, see the following configuration diagram (architecture): Q. `network error: connection timed out` occurs in the SSH access process in putty. A. When you have allowed ssh access (Port 22) in ACG but an error occurs in ssh access, ssh access (Port 22) is probably blocked in Network ACL (NACL). Allow ssh access (Port 22) in NACL. Q. What is the bandwidth of the NCP server? A. The basic bandwidth of the NCP server is around 1 Gbps (1 Gbit/sec). Q. In the process of reading data while using the NCP server, there is too much traffic overall. What should I do when there is too much network traffic use? A. - You can disperse data and traffic by adding several worker nodes. - You can save data in Object Storage by separating storage resources from computing resources, and read and save data of Object Storage by using the computing resources of Cloud Hadoop to reduce network traffic use. Q. In the Cloud Hadoop Ambari Metric service, what are the differences in features between the general operation status and the Maintenance mode operation status? A. The Maintenance mode feature provided by Ambari WebUI can be set by unit of service or host. - If you set the Maintenance mode, no notification can be sent. - If the Maintenance mode is set by unit of host (server), it is excluded from batch jobs such as service restart tasks when the batch jobs are conducted. Q. When running show tables in Hue, no View table list appears in Hive interpreter. A. When running `show tables`, only the general table list is exposed. You can run `show views` to check the View table list. Q. When I access Hive with an account that is not Hive and run a hive query, `Permission denied` error occurs. A. There are 2 solutions to this problem. - You can add the relevant account to Yarn Queue ACL. Log in to Ambari WebUI > select Yarn Queue Manager > select default (yarn queue) and add the relevant account to Users of Administer Queue and Users of Submit Applications. - If you use a hive account, you can use it without adding an account. Q. When I run `hadoop fsck /` and check the file system, an error occurs. A. fsck of hdfs can be run through an hdfs account. Log in to sshuser, convert the account into `sudo su - hdfs`, and then run it. Q. In the process of integrating Object Storage (S3) through Hive, a communication error occurs with S3. A. Check the Object Storage address for each Cloud Hadoop Region. Even if a server is within Public Subnet, if the master server has not received a public IP, you can communicate with only the private domain of Object Storage. Note The following are the domain addresses of Object Storage. Server within Public Subnet - Internet-based communication is available using kr.object.ncloudstorage.com, which is a public domain. - Private communication is available using kr.object.private.ncloudstorage.com, which is a private domain. Server within Private Subnet - Communication is available by default using kr.object.private.ncloudstorage.com, which is a private domain. - If you use NAT Gateway, you can communicate by using kr.object.ncloudstorage.com, which is a public domain. Q. I intend to perform data migration with the Object Storage bucket. Can I connect several Hadoop Clusters to a single Object Storage bucket? A. You cannot select an Object Storage bucket designated when creating Cloud Hadoop to create another Cloud Hadoop. To conduct migration, you can use the following method. 1. Create a new bucket on Object Storage and perform data upload. 2. When creating a new Cloud Hadoop, select the new bucket with the data uploaded. Q. If I want to delete the currently used Cloud Hadoop cluster but use the relevant data as they are, what should I do? A. You can use the data as they are even if you delete the Cloud Hadoop cluster through 2 methods. - When creating Cloud Hadoop, if you use [metastore](/docs/en/data-catalog-metastore) via [Data Catalog product](/docs/en/datacatalog-cloudhadoop-vpc), you can reuse the meta table of applications such as Hive/Trino/Impala as it is even if the cluster is deleted. - Save the data that needs to be analyzed in Object Storage, [integrate it with External table in Hive of Cloud Hadoop](/docs/en/hadoop-vpc-17), and then you can reuse it. Q. Do I have to select cluster add-ons (HBASE, Impala, Nifi, etc.) when creating the cluster or can I install them later to be able to use them? A. You do not have to select add-ons when creating a cluster. You can click **[Add service]** on Ambari Web UI to add and use the services later. Q. I cannot access Hive View from Apache Ambari. A. Ambari 2.7.0 and later versions do not support Hive View. If you want to use Hive View, you can access it via Hue. Q. If I use the Cloud Hadoop cluster 1.9 version, the Presto 0.240 version is included. Can I migrate Presto to the latest version? A. The Presto (Trino) version upgrade is not supported. For the Cloud Hadoop 2.0 version or higher, Trino 377, a higher version of Presto 0.240, is supported. For more information on the versions supported by Cloud Hadoop, see [Supported applications by cluster version](/docs/en/cloudhadoop-vpc-prep#%ED%81%B4%EB%9F%AC%EC%8A%A4%ED%84%B0%EB%B2%84%EC%A0%84%EB%B3%84%EC%A7%80%EC%9B%90%EC%95%A0%ED%94%8C%EB%A6%AC%EC%BC%80%EC%9D%B4%EC%85%98). Q. After the Ambari Infra Solr service has stopped, the service does not restart. A. Due to full GC caused by too much log data accumulated, the Infra Solr service might be stopped. - Infra Solr is a service to store logs, so you may not be able to start the service due to full GC based on the amount of log data stored for a long time. - If you cannot start the service, take the following actions. - Increase the Infra Solr heap size to start the service. You can adjust the heap size in **Ambari Web UI > Infra Solr > Configs**. - When the service starts normally, delete the log data stored before a certain period in `hadoop_logs` of Infra Solr. ``` # Examples of deleting data stored more than a month ago curl "http://{INFRA-SOLR-HOSTNAME}:8886/solr/hadoop_logs/update?commit=true" -H "Content-Type: text/xml" --data-binary "evtTime:[* TO NOW-1MONTHS]" ``` Q. When running the Hive query, the `System times on machines may be out of sync` error occurs. A. You need to synchronize system time and hardware time. Conduct the following tasks in all servers of Cloud Hadoop: 1. Check time - Check system time: `date` - Check hardware time: `hwclock` 2. Synchronize time - Apply hardware time to system time: `hwclock --hctosys` Q. When running the `ntpstat`, `unsynchronised` occurs. A. Synchronize Cloud Hadoop server time by referring to the [Checking time synchronization settings](/docs/en/compute-vpc-ntp-check). Q. Can I set the query log storage cycle for Trino? A. As Trino is an open source, it does not support the log storage cycle setting feature. Instead, you can manage the query history by using properties provided by Trino. - `query.max-history`: Sets the maximum number of queries that can be saved. - `query.min-expire-age`: Sets the minimum time required for history expiration. Trino's query history is saved in In-Memory, and its performance may be affected if you set the query.max-history value too high. Q. Can I save Trino query history as a file? A. As Trino is an open source, it does not support the feature of saving the query history as a file. Instead, you can use Trino API (`http://<TRONO_FQDN>:8285/ui/api/query`) to obtain the query history from the memory in JSON format and utilize it. Q. I would like to add a new account to Hive service. A. As Apache Hive uses the local account of the OS, you can create a new account in the cluster. Conduct the following tasks: 1. Create a new local account in all Cloud Hadoop servers - `useradd -u {uid} {new_user} -g hadoop` - It is recommended to set the same uid value across all servers. 2. Create a directory in HDFS for the new account - `hdfs dfs -mkdir /user/{new_user}` - `hdfs dfs -chown {new_user}:hadoop /user/{new_user}` Q. Where can I check the Impala port? A. You can check the Impala port from **Ambari Web UI > Impala > Configs > Advanced impala-port > Hive Server2 port**. The Impala port is set as 21050 by default. Q. What should I do to bring the SSL certificate of the edge node? A. You can bring the certificate of the edge node by copying it to the server you are currently working on by using the scp command. Or you can download the certificate from a web browser. You can bring the certificate by accessing the Application Web UI and clicking **Certificate Viewer > Details > Export**. Q. I cannot access the Application Web UI. A. Check if the access source and allowed port are set correctly in the ACG rule of the cluster. If you are using SSL VPN, be careful not to use 0.0.0.0/0 value as the destination address in Routing Table. For more information, see [Preliminary task](/docs/en/hadoop-vpc-7#%EC%82%AC%EC%A0%84-%EC%9E%91%EC%97%85) for accessing Web UI.