Use Kibana

Available in Classic

Kibana is an open-source interface that visualizes and navigates data saved in an Elasticsearch cluster. You can use the data to create various charts and manage clusters. The following is what is described in Use Kibana.

• Start Kibana: How to access Kibana, menu guide
• Dev Tools: guide on console in Dev Tools menu for easy API calls
• Manage data: Guide on how to save and view data using API, create index patterns, and search data from the Discover menu
• Manage snapshot: guide on how to create a snapshot using API, check the cause of a snapshot creation failure, restore data using a snapshot, and delete a snapshot

Start Kibana

You need to access Kibana first to use Kibana. You can check Kibana's various menus after logging in.

Access to Kibana

When you create an Elasticsearch cluster in the search engine service, Kibana is automatically installed and runs on the manager node. You can access Kibana from the NAVER Cloud Platform console or web browser.

The following is how to access Kibana from a web browser.

1. Open a web browser and connect to the manager node public IP address:80.
   • Connect to port number 5601 that was forwarded to port number 80.
2. When the following window appears, enter Kibana access information and click the [Login] button.
   
   • Username: enter Kibana access ID
   • Password: enter Kibana access password
   • You can use sample data by clicking the [Try our sample data] button on the first screen that appears after logging in.

Kibana menu

The basic description of each Kibana menu is as follows:

• Discover: Search for saved data (refer to Search data)
• Visualize: Data visualization tool
• Dashboard: View visualized data and search results at a glance
• Timelion: Time series data visualization tool
• Dev Tools: Manage data using API (refer to Dev Tools)
• Management: manage index patterns, saved objects, and detailed Kibana settings

Dev Tools

You can easily use the API and get the results of your request using the Console in the Dev Tools menu. The basic description of the Dev Tools screen is as follows:

Manage data

You can use Kibana to manage data. You can save and view data through API in Console in the Dev Tools menu. You can also create index patterns and search data saved in clusters.

Data storage

The following describes how to save data through the API:

1. Click on the Dev Tools menu in Kibana.
2. Call the API from the Console to save the data.
   • The saved result appears on the right screen.
   • <Example 1> Save data with name and dept fields in index test

     PUT test/_doc/1
     {
         "name" : "asd",
         "dept" : "Document Management"
     }
     

   • <Example 2> Create an index with 3 primary shards and 1 replica shard without data

     PUT my_index
     {
     "settings": {
         "index": {
            "number_of_shards": 3,
            "number_of_replicas": 1
         }
     }
     }
     

View data

Here's how to view data saved in the cluster through the API.

1. Click on the Dev Tools menu in Kibana.
2. View the data by calling the API from the Console.
   • The search results appear on the right screen.
   • <example> View data in index test

     GET test/_search
     {
         "query" : {
             "match_all" : {}
         }
     }
     

Create index pattern

To use an index in Kibana, you must first create an index pattern. You can view and manage data from multiple indexes at once through index patterns.

The following describes how to create an index pattern:

1. Click the Management menu in Kibana.
2. Click on the Index Patterns menu.
3. Click on the [Create index pattern] button.
4. In Index pattern, enter an index pattern to check in Kibana.
   • The index that matches the specified index pattern is displayed.
5. Click on the [Next step] button.
6. Check the Configure settings and click on the [Create index pattern] button.
   • If you have date/time data, select the corresponding field in the Time Filter field name.
   • It displays information about the created index pattern.

Search data

You can search for data saved in a cluster from Kibana's Discover menu.

The following describes how to search for data in a cluster.

1. Click the Discover menu in Kibana.
2. Select the index pattern you want to view from the dropdown menu.
   • Data of the index included in the selected index pattern are displayed.
     
3. Enter a keyword (<example> name:asd) in the search window and click the [Enter] key to check the search results.
   • You can select a field to view only the information for the desired field.

Manage snapshot

You can create and delete a snapshot using the Console in Kibana's Dev Tools menu. If snapshot creation fails, you can determine the cause of failure, or you can restore data using the created snapshot.

Install Object Storage Plugin

You must install the Object Storage Plugin on all nodes of the cluster to access Object Storage in the cluster. Run the following procedure to install the plug-in.

1. After connecting to the manager node using SSH, run the following command (For more information about SSH connection, see Connect to manager node using SSH).

echo "-Des.allow_insecure_settings=true" | sudo tee -a /etc/elasticsearch/jvm.options
sudo /usr/share/elasticsearch/bin/elasticsearch-plugin install repository-s3

2. Enter y to install the Object Storage Plugin.
3. Repeat steps 1 and 2 for all nodes in the cluster to install the Object Storage Plugin.
   • While connected to the manager node, execute the ssh sshuser@data node IP address command and enter yes to connect to each data node via SSH.
     • You can check the IP address of the data node by clicking the [View details] button in the cluster details in the Cluster menu.
   • If you execute the logout command while connected to the data node, you will be returned to the manager node.
4. Restart the cluster for the installed plugins to take effect (refer to Restart cluster).

Create snapshots through Elasticsearch API

You can call the Elasticsearch API to create a snapshot of the cluster.

The following shows how to create a snapshot of a cluster.

1. Click on the Dev Tools menu in Kibana.

2. In the Console, execute the following command to create a repository.

   PUT _snapshot/my-own-snapshot-repository
   {    
      "type" : "s3",  
      "settings": {        
            "region" : "kr",
            "endpoint" : "https://kr.object.ncloudstorage.com",
            "bucket": "ame of the bucket to save the snapshot",
            "path_style_access": "true",
            "access_key":"Value of access key ID",
            "secret_key":"Value of secret key"
         }
   }
   

   • my-own-snapshot-repository: enter the desired repository name
   • access_key, secret_key: API authentication key value. You can create and check it in the My page > Manage account > Manage authentication keys menu of the NAVER Cloud platform portal.

3. In the Console, execute the following command to call the snapshot creation API.

   POST _snapshot/my-own-snapshot-repository/my-own-snapshot-name
   

   • my-own-snapshot-repository: repository name created in step 2
   • my-own-snapshot-name: enter the desired snapshot name
   • If you call the API by specifying the index as follows, you can create a snapshot only for the index you want.

     PUT _snapshot/my-own-snapshot-repository/my-own-snapshot-name
     {
     "indices": "myindex, myindex2"
     }
     

   • You can check the created snapshot in the specified path in the object storage.

Check the cause of the snapshot creation failure

How to check the cause of the snapshot creation failure in Kibana is as follows:

1. Click on the Dev Tools menu in Kibana.
2. After executing the GET _snapshot command in the Console, check the repository of the snapshot that failed to be created.
   • A list of repositories appears on the right screen.
3. Execute the following command to check the detailed status of the snapshot.

   GET _snapshot/"Repository of snapshot"/"Snapshot name"
   

Restore snapshot

You can use the snapshots to restore data in a cluster or move data from one cluster to another.

The following describes how to restore a snapshot.

1. Click on the Dev Tools menu in Kibana.

2. In the Console, execute the following command to create a repository to restore.

   PUT _snapshot/restore-repository
   {  
   "type" : "s3",  
   "settings": {    
      "region" : "kr",    
      "endpoint" : "https://kr.object.ncloudstorage.com",
      "bucket": "estest",
      "path_style_access": "true",
      "access_key":"Access Key ID 값",
      "secret_key":"Secret Key 값"
   }
   }
   

   • bucket: name of the bucket where the snapshot to be restored exists

   • base_path: name of the folder where the snapshot to be restored is located (cluster name- cluster number)

   • You can run the following command to check that the repository was created successfully.

     GET _snapshot
     

   • You can check the list of snapshots that exist in the repository by running the following command:

     GET _cat/snapshots/restore-repository
     

3. Execute the following command to check the list of indexes that can be restored and exist in the snapshot.

   GET _snapshot/restore-repository/"Snapshot name"
   

4. Execute the following command to check the list of indexes that currently exist in the cluster.

   • If an index with the same name from the index you want to restore exists in the cluster, then it can't be restored.

   GET _cat/indices
   

5. Execute the command to restore the desired index.

   • Restore for specific indexes
     • <example> Restore for metricbeat-7.7.0-2021.06.23

       POST _snapshot/restore-repository/test4/_restore
       {
       "indices": "metricbeat-7.7.0-2021.06.23",
       "ignore_unavailable": true
       }
       

   • Restore for multiple indexes using wildcards (*)
     • <example> Restore for all indexes starting with metricbeat

       POST _snapshot/restore-repository/test4/_restore
       {
       "indices": "metricbeat*",
       "ignore_unavailable": true
       }
       

   • Restore for all indexes except for specific indexes
     • <example> Restore for all indexes except kibana

       POST _snapshot/restore-repository/test4/_restore
       {
       "indices": "-.kibana*",
       "ignore_unavailable": true
       }
       

   • If set to "indices": "*", restore is attempted for all indexes, and the restore fails if duplicate indexes exist.

6. Run the following command to check if the index was restored successfully.

   • Immediately after restoration, the status of the index may be temporarily yellow, and the status of the cluster may be in the warning status.

   GET _cat/indices
   

Delete snapshot

1. Click on the Dev Tools menu in Kibana.

2. In the Console, execute the following command to delete a snapshot.

   DELETE _snapshot/restore-repository/test4
   

   • You can check that the snapshot has been deleted by running the following command:

     GET _cat/snapshots/restore-repository