Hadoop monitoring with Cloud Insight

Available in VPC

NAVER Cloud Platform's Cloud Insight service allows you to monitor the performance and operational indicators of Hadoop and quickly identify and respond to failures.

Preparations

1. Create a Cloud Hadoop cluster.
   • For more information about creating a Cloud Hadoop cluster, see Getting started with Cloud Hadoop guide.
2. Subscribe to Cloud Insight.
   • For more information on subscribing to Cloud Insight, see Cloud Insight user guide

Configure dashboards

You can create dashboards and add widgets to monitor Cloud Hadoop from the Cloud Insight console screen.

To create a dashboard and add widgets in Cloud Insight, follow these steps:

1. In the VPC environment of the NAVER Cloud Platform console, navigate to > Services > Management & Governance > Cloud Insight (Monitoring) in order.
2. Click the [Create dashboard] button.
3. Enter a name and description for the dashboard and click [Create].
   
4. Click the [Add widget] button.
5. Enter a name for the widget, select a widget type, and click [Next].
   • This example uses the Time Series widget.
     
6. Enter the widget settings as shown below and click [Next].
   • On the [CPU] tab, select the CPU/used_rto and CPU/user_rto check boxes and click [Add selection].
     
   • Product Type: Cloud Hadoop(VPC)
   • Target: select All owned resources and select the cluster you want to monitor.
     When selecting Group, see Target Group settings
   • Metric: select All metrics, select the item to monitor, and click [Add selection].
     When selecting Template, see Rule Template settings
   • Setup data list: Dimension (properties), Interval (aggregation interval), and Aggregation (aggregation function) of the selected monitoring item
7. After reviewing the widget settings, click [Create].
   
   • The dashboard displays added widgets as illustrated below. You can use the added widget to monitor your Cloud Hadoop cluster.
     

Group and template settings

To make it easier to manage monitoring settings and widgets, you can group specific monitoring targets or save specific monitoring items (metrics) as templates.

Target group configuration

To create a target group and organize specific monitoring targets into groups, follow these steps:

1. In the VPC environment of the NAVER Cloud Platform console, navigate to > Services > Management & Governance > Cloud Insight (Monitoring) in order.
2. Click Configuration > Template in order.
3. Click the [Target Group] tab and click [Create target group].
4. Enter the group settings as shown below and click [Create].
   

• Product Type: Cloud Hadoop(VPC)
• Group name, Group description: enter a group name and description.
• Selectable monitoring targets: select all the monitoring targets to be included in the group and click .

Rule Template settings

To set up a rule template to save specific monitoring items as a template, follow these steps:

1. In the VPC environment of the NAVER Cloud Platform console, navigate to > Services > Management & Governance > Cloud Insight (Monitoring) in order.

2. Click Configuration > Template in order.

3. Click the [Rule Template] tab and click [Create rule template].

4. Enter the template settings as shown below and click [Next].
   

   • Product Type: Cloud Hadoop(VPC)
   • Template name, Description: enter a template name and description.
   • On each classification tab, locate and select the monitoring items (metrics) you want to include in the template.

5. Enter the monitoring conditions for each monitoring item as shown below and click [Save].
   

   • Dimension: properties of the monitoring item
   • Level: level at which the event occurs
   • Condition: condition under which the event occurs
   • Method: aggregation function of the monitoring item
   • Duration: duration
   Note

   The following example shows how to set an info level event to occur when the value of CPU/user_rto (cpu_idx: 1) in Cloud Hadoop (VPC) is 0 and lasts for 1 minute:
   

Set event

You can select a monitoring target and item, set monitoring conditions and notification actions to create an event, and check the status of the created event.

To set up an event, follow these steps:

1. In the VPC environment of the NAVER Cloud Platform console, navigate to > Services > Management & Governance > Cloud Insight (Monitoring) in order.
2. Click Configuration > Event Rule, in order.
3. Click [Create Event Rules].
4. Under Select a product to monitor, select Cloud Hadoop (VPC) and click [Next].
   
5. Select an individual monitoring target or monitoring group and click [Next].
   • To create a new group, see Target Group settings
     
6. Select an individual monitoring item or monitoring template and click [Next].
   • To create a new template, see Rule Template settings
     
7. In the [Send Notification Messages] tab, select a recipient group to be notified and click [Next].
   • To create a new notification recipient group, see Create recipient groups
     
8. After reviewing the event settings, click [Create].
   

Check event status

To check the status of an event you have created, follow these steps:

1. In the VPC environment of the NAVER Cloud Platform console, navigate to > Services > Management & Governance > Cloud Insight (Monitoring) in order.
2. Click Event. When an event occurs under an event rule, you can review the details of the event as shown below.
   

Create recipient groups

To create a notification recipient group to receive event notification messages and add recipients, follow these steps:

1. In the VPC environment of the NAVER Cloud Platform console, navigate to > Services > Management & Governance > Cloud Insight (Monitoring) in order.
2. Click Notification Recipient menu in order.
3. In the recipient group list, click , type the name of the group you want to create, and click .
   
4. In the Recipient Group list, click All Recipients.
5. Select the recipients you want to assign to the created group and click [Assign].
   • To add a new recipient, click [Add recipient] and refer to Cloud Insight user guide to add a recipient.
     
6. Enter the information of the notification recipient you want to add, complete the identity verification, and click [Register].

Cloud Hadoop Metric

For all clusters created, you can monitor the metrics listed below. Cloud Insight collects data for the metrics every 1 minute.