Using Kibana

Available in Classic

Kibana is an open-source interface for visualizing and exploring data stored in Elasticsearch clusters. You can use data to create various charts and manage clusters. This guide introduces the following features:

• Getting started with Kibana: How to access Kibana and an overview of the menu.
• Dev Tools: Introduction to the Console in the Dev Tools for easily calling APIs.
• Data management: How to store and retrieve data using APIs, create index patterns, and search data in the Discover menu.
• Manage snapshot: How to create snapshots using APIs, identify causes of snapshot creation failures, restore data from snapshots, and delete snapshots.

Getting started with Kibana

To use Kibana, you must first access Kibana. Log in to explore various menus in Kibana.

Access Kibana

When you create an Elasticsearch cluster in the Search Engine Service, Kibana is installed on the manager node and starts automatically. You can access Kibana either from the NAVER Cloud Platform console or through a web browser.

To access Kibana via a web browser:

1. In your web browser, connect to manager node public IP:80.
   • This accesses port 5601, which is forwarded to port 80.
2. In the following screen, enter the Kibana access information and click [Log in].
   
   • Username: Enter the Kibana access ID.
   • Password: Enter the Kibana access password.
   • In the first interface after login, click [Try our sample data] to use the sample data.

Kibana menu

The Kibana menu includes the following:

• Discover: Search stored data (see Search data).
• Visualize: Data visualization tool.
• Dashboard: View visualized data and search results at a glance.
• Timelion: Visualization tools for time-series data.
• Dev Tools: Manage data using APIs (see Dev Tools).
• Management: Manage index patterns, saved objects, and advanced Kibana settings.

Dev Tools

You can use the console in the Dev Tools menu to easily work with APIs and view request results. The Dev Tools interface includes the following:

Manage data

You can manage data using Kibana. From the console in the Dev Tools menu, you can store and view data through APIs. You can also create index patterns and search data stored in the cluster.

Store data

To store data using APIs:

1. In Kibana, click the Dev Tools menu.
2. In the console, call the API to store data.
   • The result is displayed on the right pane.
   • Example 1: Store data with the name and dept fields in the test index.

     PUT test/_doc/1
     {
         "name" : "asd",
         "dept" : "Document Management"
     }
     

   • Example 2: Create an index with 3 primary shards and 1 replica shard, with no data.

     PUT my_index
     {
     "settings": {
        "index": {
           "number_of_shards": 3,
           "number_of_replicas": 1
        }
     }
     }
     

View data

To view data stored in the cluster via the API:

1. In Kibana, click the Dev Tools menu.
2. In the console, call the API to view the data.
   • The results are displayed on the right pane.
   • Example: View data in text index.

     GET test/_search
     {
         "query" : {
             "match_all" : {}
         }
     }
     

Create index pattern

To use indices in Kibana, you must first create an index pattern. Using index patterns, you can view and manage data from multiple indices at once.

To create an index pattern:

1. In Kibana, click the Management menu.
2. Click the Index Patterns menu.
3. Click [Create index pattern].
4. Enter an index pattern in the Index pattern field to view it in Kibana.
   • Indices that match the index pattern are displayed.
5. Click [Next step].
6. Review the Configure settings, then click [Create index pattern].
   • If date or time fields are available in the index, select the appropriate field in the Time Filter field name.
   • Details of the created index pattern are displayed.

Search data

You can search data stored in the cluster from the Discover menu in Kibana.

To search data in the cluster:

1. In Kibana, click the Discover menu.
2. From the dropdown list, select the index pattern to view.
   • Data from the indices included in the selected index pattern is displayed.
     
3. Enter a search term (for example, name:asd) in the search field and press [Enter] to view the search results.
   • Select fields to display only the data you need.

Manage snapshot

In Kibana, you can create and delete snapshots using the console in the Dev Tools menu. If snapshot creation fails, you can identify the cause, or restore data using an existing snapshot.

Install Object Storage Plugin

To access Object Storage from the cluster, you must install the Object Storage Plugin on all nodes in the cluster. Follow these steps to install the plugin.

1. Connect to the manager node via SSH and run the following commands.(For more information on SSH access, see Connect to the manager node via SSH.)

echo "-Des.allow_insecure_settings=true" | sudo tee -a /etc/elasticsearch/jvm.options
sudo /usr/share/elasticsearch/bin/elasticsearch-plugin install repository-s3

2. Enter y to install the Object Storage Plugin.
3. Repeat steps 1–2 for all nodes in the cluster to install the Object Storage Plugin.
   • While connected to the manager node, run the command "ssh sshuser@Data Node IP Address" and type yes to connect to each data node via SSH.
     • In the Cluster menu, click [View details] to find the IP addresses of each data node.
   • While connected to the data node, run the logout command to return to the manager mode.
4. Restart the cluster to apply the installed plugin (see Restart cluster).

Create snapshot using Elasticsearch API

You can create a snapshot of a cluster by calling the Elasticsearch API.

To create a snapshot of a cluster:

1. In Kibana, click the Dev Tools menu.
2. In the console, run the following command to create a repository.

   PUT _snapshot/my-own-snapshot-repository
   {    
      "type" : "s3",  
      "settings": {        
            "region" : "kr",
            "endpoint" : "https://kr.object.ncloudstorage.com",
            "bucket": "name of the bucket to store snapshots",
            "path_style_access": "true",
            "access_key":"Access Key ID value",
            "secret_key":"Secret Key value"
         }
   }
   

   • my-own-snapshot-repository: Name the repository.
   • access_key, secret_key: API authentication key values. For more information on API authentication keys, see Ncloud API.
3. In the console, run the following command to call the snapshot creation API.

   POST _snapshot/my-own-snapshot-repository/my-own-snapshot-name
   

   • my-own-snapshot-repository: The repository name created in step 2.
   • my-own-snapshot-name: Name the snapshot.
   • Specify indices when calling the API as shown below to create snapshots only for the specified indices.

     PUT _snapshot/my-own-snapshot-repository/my-own-snapshot-name
     {
     "indices": "myindex, myindex2"
     }
     

   • The snapshot is stored at the specified path in Object Storage.

Identify cause of snapshot creation failure

To identify the cause of snapshot creation failures in Kibana:

1. In Kibana, click the Dev Tools menu.
2. In the console, run the GET _snapshot command to check the repository of a failed snapshot.
   • The list of repositories is displayed on the right pane.
3. Run the following command to view the detailed status of the snapshot.

   GET _snapshot/"snapshot repository"/"snapshot name"
   

Restore snapshot

You can use snapshots to restore cluster data or migrate data from one cluster to another.

To restore a snapshot:

1. In Kibana, click the Dev Tools menu.
2. In the console, run the following command to create a repository to restore.

   PUT _snapshot/restore-repository
   {  
   "type" : "s3",  
   "settings": {    
      "region" : "kr",    
      "endpoint" : "https://kr.object.ncloudstorage.com",
      "bucket": "estest",
      "path_style_access": "true",
      "access_key":"Access Key ID value",
      "secret_key":"Secret Key value"
   }
   }
   

   • bucket: Name of the bucket where the snapshot to restore is stored.

   • base_path: Folder name where the snapshot is located (cluster-name-cluster-number).

   • Run the following command to verify that the repository was created successfully.

     GET _snapshot
     

   • Run the following command to view the list of snapshots stored in the repository.

     GET _cat/snapshots/restore-repository
     

3. Run the following command to view the list of indices available for restoration in the snapshot.

   GET _snapshot/restore-repository/"snapshot name"
   

4. Run the following command to view the list of indices currently present in the cluster.
   • Indices cannot be restored if an index with the same name already exists in the current cluster.

   GET _cat/indices
   

5. Run the command to restore indices.
   • Restore a specific index
     • Example: Restore metricbeat-7.7.0-2021.06.23.

       POST _snapshot/restore-repository/test4/_restore
       {
       "indices": "metricbeat-7.7.0-2021.06.23",
       "ignore_unavailable": true
       }
       

   • Restore multiple indices using a wildcard (*)
     • Example: Restore all indices starting with metricbeat.

       POST _snapshot/restore-repository/test4/_restore
       {
       "indices": "metricbeat*",
       "ignore_unavailable": true
       }
       

   • Restore all indices except for specific ones.
     • Example: Restore all indices except kibana.

       POST _snapshot/restore-repository/test4/_restore
       {
       "indices": "-.kibana*",
       "ignore_unavailable": true
       }
       

   • Setting this to "indices": "*" will trigger a restore for all indices, but duplicate indices will cause the operation to fail.
6. Run the following command to check if the indices were restored successfully.
   • Immediately after restoration, the index status may temporarily be yellow, and the cluster status may show a warning.

   GET _cat/indices
   

Delete snapshot

1. In Kibana, click the Dev Tools menu.
2. In the console, run the following command to delete a snapshot.

   DELETE _snapshot/restore-repository/test4
   

   • You can run the following command to verify that the snapshot has been deleted.

     GET _cat/snapshots/restore-repository