Use Kibana/OpenSearch

Available in VPC

Kibana and OpenSearch Dashboards are open-source interfaces for visualizing and exploring data saved in Elasticsearch and OpenSearch clusters. You can use the data to create various charts and manage clusters. The Search Engine Service supports Kibana version 7.7.0 for Elasticsearch clusters and OpenSearch Dashboards version 1.3.3 for OpenSearch clusters. The following is what is described in Use Kibana/OpenSearch.

• Start Kibana: how to access Kibana, default screen guide
• Start OpenSearch Dashboards: how to access OpenSearch Dashboards, default screen guide
• Dev Tools: guide on console in Dev Tools menu for easy API calls
• Manage data: guide on how to save and view data using API, create index patterns, and search data from the Discover menu
• Manage snapshot: guide on how to create a snapshot using API, check the cause of a snapshot creation failure, restore data using a snapshot, and delete a snapshot

Start Kibana

You need to access Kibana first to use Kibana. You can check Kibana's various menus after logging in.

Access to Kibana

When you create an Elasticsearch cluster, Kibana is automatically installed and runs on the manager node.

The following describes how to access Kibana from outside.

1. Access load balancer IP address:80 from a web browser.
   • Connect to port number 5601 that was forwarded to port number 80.
2. When the following window appears, enter Kibana access information and click the [Login] button.
   
   • Username: enter Kibana access ID
   • Password: enter Kibana access password
   • You can use sample data by clicking the [Try our sample data] button on the first screen that appears after logging in.

Kibana screen

The basic description of the Kibana screen is as follows:

Field	Description
① Home, menu name	: click to go to the Home screen name of the menu you are currently checking
② Kibana menu	Recently viewed: Check recently viewed data Discover: search for saved data (refer to Search data) Visualize: visualize and add data to the dashboard Dashboard: view multiple visualization tools and search results at a glance Dev Tools: manage data using API (refer to Dev Tools) Management: manage index patterns, saved objects, and detailed Kibana settings
③ Kibana information	: click to check Kibana version, Kibana guide, and links to external resources : click to learn more about Kibana

Start OpenSearch Dashboards

To use OpenSearch Dashboards, you must first access OpenSearch Dashboards. You can check OpenSearch Dashboards' various menus after logging in.

Access OpenSearch Dashboards

When you create an OpenSearch cluster, OpenSearch Dashboards are automatically installed and run on the manager node.

The following describes how to access OpenSearch Dashboards from outside.

1. Access load balancer IP address:80 from a web browser.
   • Connect to port number 5601 that was forwarded to port number 80.
2. When the following window appears, enter OpenSearch Dashboards access information and click the [Log In] button.

• Username: Enter the OpenSearch Dashboards access ID
• Password: Enter the OpenSearch Dashboards access password
• You can use sample data by clicking the [Add data] button on the first screen that appears after logging in.

OpenSearch Dashboards screen

The basic description of the OpenSearch Dashboards screen is as follows:

Dev Tools

You can easily use the API and get the results of your request using the Console in the Dev Tools menu. The basic description of the Dev Tools screen is as follows:

Manage data

You can use Kibana and OpenSearch Dashboards to manage data. You can save and view data through API in Console in the Dev Tools menu. You can also create index patterns and search data saved in clusters.

Data storage

The following describes how to save data through the API:

1. Click the Dev Tools menu from Kibana and OpenSearch Dashboards.
2. Call the API from the Console to save the data.
   • The saved result appears on the right screen.
   • <Example 1> Save data with name and dept fields in index test

     PUT test/_doc/1
     {
         "name" : "asd",
         "dept" : "Document Management"
     }
     

   • <Example 2> Create an index with 3 primary shards and 1 replica shard without data

     PUT my_index
     {
     "settings": {
         "index": {
            "number_of_shards": 3,
            "number_of_replicas": 1
         }
     }
     }
     

View data

Here's how to view data saved in the cluster through the API.

1. Click the Dev Tools menu from Kibana and OpenSearch Dashboards.
2. View the data by calling the API from the Console.
   • The search results appear on the right screen.
   • <example> View data in index test

     GET test/_search
     {
         "query" : {
             "match_all" : {}
         }
     }
     

Create index pattern

You must first create an index pattern to use the index in Kibana and OpenSearch Dashboards. You can view and manage data from multiple indexes at once through index patterns.

The following describes how to create an index pattern:

1. In Kibana and OpenSearch Dashboards, click the Management or Stack Management menu.
2. Click on the Index Patterns menu.
3. Click on the [Create index pattern] button.
4. Enter an index pattern for verification in Kibana and OpenSearch Dashboards.
   • The index that matches the specified index pattern is displayed.
5. Click on the [Next step] button.
6. Check the Configure settings and click on the [Create index pattern] button.
   • If you have date/time data, select the corresponding field in the Time Filter field name or Time field.
   • It displays information about the created index pattern.

Search data

You can search for data saved in a cluster from Kibana and OpenSearch Dashboards' Discover menu.

The following describes how to search for data in a cluster.

1. Click the Discover menu from Kibana and OpenSearch Dashboards.
2. Select the index pattern you want to view from the dropdown menu.
   • Data of the index included in the selected index pattern are displayed.
     
3. Enter a keyword (<example> name:asd) in the search window and click the [Enter] key to check the search results.
   • You can select a field to view only the information for the desired field.

Manage snapshot

You can create and delete a snapshot using the Console in Kibana and OpenSearch Dashboards' Dev Tools menu. If snapshot creation fails, you can determine the cause of failure, or you can restore data using the created snapshot.

Create snapshots through Elasticsearch and OpenSearch API

If you want to rename the folder where you want to save the snapshot, or if snapshot creation fails due to a timeout in the console, you can create the snapshot directly through the Elasticsearch and OpenSearch API in Kibana and OpenSearch Dashboards.

• However, if you create a snapshot directly through the Elasticsearch and OpenSearch API, you cannot check the snapshot creation history from the console.

The following describes how to create a snapshot directly through the Elasticsearch and OpenSearch API:

1. Click the Dev Tools menu from Kibana and OpenSearch Dashboards.

2. In the Console, execute the following command to create a repository.

   PUT _snapshot/my-own-snapshot-repository
   {    
       "type" : "s3",
       "settings": {        
                   "region" : "kr",
                   "endpoint" : "https://kr.object.private.ncloudstorage.com", 
                   "bucket": "estest",  
                   "compress": true,
                   "base_path": "my-folder",  
                   "path_style_access": true
       }
   }
   

   • my-own-snapshot-repository: enter the desired repository name
   • base_path: enter the desired folder name
     • If you enter a folder name that is being used by another cluster, the snapshot file may not work properly.
   • region
     • For Korea region: kr
     • For Singapore region: sgn
   • endpoint
     • For Korea region: https://kr.object.private.ncloudstorage.com
     • For Singapore region: https://sg.object.private.ncloudstorage.com

3. In the Console, execute the following command to call the snapshot creation API.

   POST _snapshot/my-own-snapshot-repository/my-own-snapshot-name
   

   • my-own-snapshot-repository: enter the repository name created in step 2
   • my-own-snapshot-name: enter the desired snapshot name
   • If you call the API by specifying the index as follows, you can create a snapshot only for the index you want.

     PUT _snapshot/my-own-snapshot-repository/my-own-snapshot-name
     {
     "indices": "myindex, myindex2"
     }
     

   • You can check the created snapshot in the specified path in the object storage.

Check the cause of the snapshot creation failure

The following is how to determine the cause of a snapshot creation failure in Kibana and OpenSearch Dashboards:

1. Click the Dev Tools menu from Kibana and OpenSearch Dashboards.
2. After executing the GET _snapshot command in the Console, check the repository of the snapshot that failed to be created.
   • A list of repositories appears on the right screen.
3. Execute the following command to check the detailed status of the snapshot.

   GET _snapshot/"Repository of snapshot"/"Snapshot name"
   

Restore snapshot

You can use the snapshots to restore data in a cluster or move data from one cluster to another.

The following describes how to restore a snapshot.

1. Click the Dev Tools menu from Kibana and OpenSearch Dashboards.

2. In the Console, execute the following command to create a repository to restore.

   PUT _snapshot/restore-repository
   {  
   "type" : "s3",  
   "settings": {    
      "region" : "kr",    
      "endpoint" : "https://kr.object.private.ncloudstorage.com",
      "bucket": "estest",
      "compress": true,
      "base_path": "estest-1722213",
      "path_style_access": true
   }
   }
   

   • region

     • For Korea region: kr
     • For Singapore region: sgn

   • endpoint

     • For Korea region: https://kr.object.private.ncloudstorage.com
     • For Singapore region: https://sg.object.private.ncloudstorage.com

   • bucket: name of the bucket where the snapshot to be restored exists

   • base_path: name of the folder where the snapshot to be restored is located (cluster name- cluster number)

   • You can run the following command to check that the repository was created successfully.

     GET _snapshot
     

   • You can check the list of snapshots that exist in the repository by running the following command:

     GET _cat/snapshots/restore-repository
     

3. Execute the following command to check the list of indexes that can be restored and exist in the snapshot.

   GET _snapshot/restore-repository/"Snapshot name"
   

4. Execute the following command to check the list of indexes that currently exist in the cluster.

   • If an index with the same name from the index you want to restore exists in the cluster, then it can't be restored.

   GET _cat/indices
   

5. Execute the command to restore the desired index.

   • Restore for specific indexes
     • <example> Restore for metricbeat-7.7.0-2021.06.23

       POST _snapshot/restore-repository/test4/_restore
       {
       "indices": "metricbeat-7.7.0-2021.06.23",
       "ignore_unavailable": true
       }
       

   • Restore for multiple indexes using wildcards (*)
     • <example> Restore for all indexes starting with metricbeat

       POST _snapshot/restore-repository/test4/_restore
       {
       "indices": "metricbeat*",
       "ignore_unavailable": true
       }
       

   • Restore for all indexes except for specific indexes
     • <example> Restore for all indexes except kibana

       POST _snapshot/restore-repository/test4/_restore
       {
       "indices": "-.kibana*",
       "ignore_unavailable": true
       }
       

   • If set to "indices": "*", restore is attempted for all indexes, and the restore fails if duplicate indexes exist.

6. Run the following command to check if the index was restored successfully.

   • Immediately after restoration, the status of the index may be temporarily yellow, and the status of the cluster may be in the warning status.

   GET _cat/indices
   

Delete snapshot

1. Click the Dev Tools menu from Kibana and OpenSearch Dashboards.

2. In the Console, execute the following command to delete a snapshot.

   DELETE _snapshot/restore-repository/test4
   

   • You can check that the snapshot has been deleted by running the following command:

     GET _cat/snapshots/restore-repository