Using the Monitoring Console
- Print
- PDF
Using the Monitoring Console
- Print
- PDF
Article Summary
Share feedback
Thanks for sharing your feedback!
Available in VPC
Monitoring supports two types of dashboards for you to check various monitoring information on Cloud Hadoop performance and history. As Monitoring is a service included in NAVER Cloud Platform's Cloud Hadoop, it can be used without additional charges.
The types of dashboards provided by Monitoring are as follows.
- HADOOP Dashboard: monitoring information related to Cloud Hadoop in operation
- OS Dashboard: hardware and network information of each Cloud Hadoop server in operation
Through these two types of dashboards, you can check the information related to Cloud Hadoop from the last 2 months, as well as each server's hardware and network indicators. Each dashboard consists of graphic charts, and you can increase work efficiency by printing specific graphic charts or downloading them to your PC as files in various extensions.
Note
You can set for the user to be notified when monitoring results show that a certain indicator has exceeded its threshold or met specific conditions by having them recognized as events. For more details about using event and notification settings, refer to Monitoring Cloud Hadoop with Cloud Insight.
Getting started with Monitoring
- In order, click the Services > Big Data & Analytics > Cloud Hadoop menus on the NAVER Cloud Platform console.
- Click the [Create cluster] button, and then create a Cloud Hadoop cluster.
- Refer to the Getting started with Cloud Hadoop guide for more details on creating clusters.
- Click the Cloud Hadoop > Monitoring menus on the left.
- Click the clusters to be monitored from the Cloud Hadoop cluster list.
Monitoring page
The basics of using Monitoring are as follows.
- You can select the currently operating Cloud Hadoop clusters and the servers for each cluster from the area on the left.
- Click the cluster name to display the HADOOP Dashboard on the area on the right, and then click the server under the cluster name to display the OS Dashboard.
Check Monitoring Dashboards
The Dashboards provided by Monitoring consist of multiple graphic charts. Users can display and intuitively check only the information they want from the dashboard of each cluster. The following describes how to use dashboards.
HADOOP Dashboard
- By clicking the cluster you want from the Cloud Hadoop cluster list on the left, you can view the HADOOP Dashboard as shown on the right.
- The HADOOP Dashboard collects data every minute.
- Monitoring information are based on average values, and the frequency of queries varies, depending on the selected period type.
- The indicators you can check for each group are as follows.
| Group | Indicator | Unit | Explanation |
|---|---|---|---|
| Apps | apps_completed | num | number of applications submitted to YARN that have completed |
| apps_failed | num | number of applications submitted to YARN that have failed to complete | |
| apps_killed | num | number of applications submitted to YARN that have been killed | |
| apps_pending | num | number of applications submitted to YARN that are in a pending state | |
| apps_running | num | number of applications submitted to YARN that are running | |
| apps_submitted | num | number of applications submitted to YARN | |
| Blocks | corrupt_blocks | num | number of blocks that HDFS reports as corrupted |
| missing_blocks | num | number of blocks in which HDFS has no replicas | |
| pending_deletion_blocks | num | number of blocks marked for deletion | |
| pending_replication_blocks | num | status of block replication: blocks being replicated, age of replication requests, and unsuccessful replication requests | |
| under_replicated_blocks | num | number of blocks that need to be replicated one or more times | |
| Containers | allocated_container | num | number of resource containers allocated by the ResourceManager |
| pending_containers | num | number of containers in the queue that have not yet been allocated | |
| reserved_containers | num | number of containers reserved | |
| HDFS capacity(GB) | capacity_remaining_gb | GB | amount of remaining HDFS disk capacity |
| HDFS read/write(bytes) | hdfs_bytes_read | num | number of bytes read from HDFS |
| hdfs_bytes_written | num | number of bytes written to HDFS | |
| HDFS utilization(%) | hdfs_utilization | % | percentage of HDFS storage currently used |
| Memory(MB) | allocated_mb | MB | amount of memory allocated to the cluster |
| available_mb | MB | amount of memory available to be allocated | |
| reserved_mb | MB | amount of memory reserved | |
| total_mb | MB | total amount of memory in the cluster | |
| Nodes | num_live_data_nodes | num | number of data nodes that are receiving work from Hadoop |
| unhealthy_nodes | num | number of nodes available to MapReduce jobs marked in an UNHEALTHY state | |
| active_nodes | num | number of nodes presently running MapReduce tasks or jobs | |
| decommissioned_nodes | num | number of nodes allocated to MapReduce applications that have been marked in a DECOMMISSIONED state | |
| lost_nodes | num | number of nodes allocated to MapReduce that have been marked in a LOST state | |
| rebooted_nodes | num | number of nodes available to MapReduce that have been rebooted and marked in a REBOOTED state | |
| total_nodes | num | number of nodes presently available to MapReduce jobs | |
| V_cores | allocated_v_cores | num | number of core nodes working |
| pending_v_cores | num | number of core nodes waiting to be assigned | |
| Data transfers | total_load | num | total number of concurrent data transfers |
| YARN memory(%) | yarn_memory_available_percentage | % | percentage of remaining memory available to YARN (= available_mb / total_mb) |
- You can monitor the changes in the cluster's indicators in real time.
- This is how indicators change when the number of cluster data nodes is reduced.
- This is how indicators change when the number of cluster data nodes is reduced.
- Users can zoom in or out of graphs by placing the mouse cursor directly over the graph, as well as view indicators from the desired period on the dashboard by setting the search period.
- You can expand the chart extension by clicking on
, and you can download the graph as an expandable file. Choose a format to download your data.
, and you can download the graph as an expandable file. Choose a format to download your data.
OS Dashboard
- On the Monitoring page, select the server under the cluster instead of the cluster name. You can view the OS Dashboard.
- The OS Dashboard collects data every minute.
- Monitoring information are based on average values, and the frequency of queries varies, depending on the selected period type.
- You can check the master node, edge node, and data node that make up the Cloud Hadoop cluster, as well as check individual CPU usage, load average, memory, disk I/O, disk usage, and network I/O indicators.
Was this article helpful?