Add and manage data source

Available in VPC

This section describes how to add data sources for queries and manage registered data.

Add data source

Data sources for queries can be added through connections.
To add a data source, follow these steps:

1. In the VPC environment on the NAVER Cloud Platform console, navigate to > Services > Big Data & Analytics > Data Query.
2. Click the Query Editor menu.
3. Click [Add data source] at the top of the interface.
   • Alternatively, click [Manage data source], then click [Add data source] in the popup.
4. Enter the information for the data source you want to register.
   • Name: Enter up to 50 characters using English uppercase and lowercase letters, numbers, and underscores (_). The first character must be an English letter.
   • Source type: Select a single supported JDBC (various source types to be added later).
   • Connection: Select a connection created for the data you want to use from those registered in Data Catalog.
5. Click [Create].
   • The added data source is displayed in the Data source dropdown menu and in the list shown when you click the [Manage data source].
   • By selecting an item from the Data source drop-down menu, you can view the data imported through the connection in a tree format.
   • You are now ready to run queries using the database of the added data source. For more information on how to run queries, see Run and manage query.

Data source details

Connection and usage methods may vary depending on the data source type.

Data Catalog

• Data Query integrates schema information collected from the Data Catalog service for direct use in queries. Data Catalog is automatically added upon subscription, without a separate data source registration process.

Public Data

• The Data Query service provides public data sources that can be used for queries by default.
• Selecting public_data from the data source list displays it. The specific list of databases included in the source is subject to change.

JDBC

• The Data Query service supports JDBC connections, allowing you to easily connect to databases. This enables a variety of database operations, such as viewing, editing, and deleting data.
• Databases supported for connection with Data Query JDBC
  • MySQL (supports version compatibility with Cloud DB for MySQL)
  • MSSQL (supports version compatibility with Cloud DB for MSSQL)
  • PostgreSQL (supports version compatibility with Cloud DB for PostgreSQL)
  • MongoDB (supports version compatibility with Cloud DB for MongoDB)

Delete data source

You can stop integrations for the data sources that are no longer used for queries by deleting them.
To delete a data source, follow these steps:

1. In the VPC environment on the NAVER Cloud Platform console, navigate to > Services > Big Data & Analytics > Data Query.
2. Click the Query Editor menu.
3. Click [Manage data source] at the top of the interface.
4. In the data source popup, select the checkbox for the item you want to delete and click [Delete].
5. In the notification popup, click [OK].
   • Integrations are terminated and disappear from all data source lists.

Manage table

Registering a data source allows you to view the internal table fields in a tree format. At this time, you can easily manage tables by using the additional features provided for each table. The provided features are as follows:

• Preview table: Automatically enters a query in the query window to preview the table content.
• Create table DDL: Analyzes the table and automatically enters a DDL statement in the query window to create the table. Can be used for copying or editing tables.
• Delete table: Automatically enters a query statement in the query window to delete the table.
• View catalog: Go to the catalog in Data Catalog where the table is registered.

To run the additional features for table management, follow these steps:

1. In the VPC environment on the NAVER Cloud Platform console, navigate to > Services > Big Data & Analytics > Data Query.
2. Click the Query Editor menu.
3. Hover the cursor over the options menu next to the table you want in the data source tree component.
   • The table is classified by marks.
     
4. Select and click the desired feature.
   • Once you have selected the feature of automatically entering a query statement, you can run the query by clicking [Run].
   • If you have selected View catalog, the Table menu page of the Data Catalog service will be displayed in a new window.

Add table manually

You can register the data file to the table by running the table creation SQL statement in Data Query or using the table creation feature of Data Catalog. To add a table, follow these steps:

1. In the VPC environment on the NAVER Cloud Platform console, navigate to > Services > Big Data & Analytics > Data Query.
2. Click the Query Editor menu.
3. In the data source component, click the "+" button next to the search bar.
   • The "+" button is displayed as .

Create tables with SQL statements

CREATE TABLE

• Creates a new table by defining the schema directly and specifying the table properties and the data location.
• To create a table in Iceberg table format, see the CREATE TABLE(ICEBERG) syntax.

CREATE TABLE AS SELECT

• Creates a new table using existing tables and data.
• You can create a table with the same data and schema as the original table.
• Alternatively, you can create a table by selecting only the column you need through a SELECT syntax, or by simply transforming the data.
• To create a table in Iceberg table format, see the CREATE TABLE AS SELECT(ICEBERG) syntax.

CREATE VIEW

• Creates a new VIEW for the SELECT query you entered.
• You can create a complex SELECT query as a view and easily reference it later.

Create tables in Data Catalog

Specify the Object Storage path

You can register the data file to the table manually using Data Catalog's Scanner. Enter the information required for scanning and run Scanner.

• Database: Select a database for the table created by the scanner.
• Data type
  • Catalog Default: It is the default Hive Table type provided by Data Catalog.
  • Apache Iceberg: It is an open table format for the extensive analysis data set, which supports ACID transaction, schema evolution, and Time Travel query and helps work safely and simultaneously in Spark, Trino, and Hive.
• Path: Enter the path of the source data to scan.
  • Run a scan for sub-paths of the path you entered.
  • Click the [+Settings] to specify a detailed path for the bucket or sub-bucket.
• Scanning method
  • Create new scanner: Create a new scanner and run the scanner.
  • Select existing scanner: Run an existing scanner in Data Catalog with the same [data type], [database], and [path].
• Scan range: Specify the number of files to scan in Object Storage. Files are read in file name order.
  • Configurable from 1 to 100. The default is 10.
  • Scans the specified number of files for each leaf directory in the specified path.
  • Available only when [Data type] is set to Catalog Default.
• Pattern: Configure whether to include or exclude metadata collection for specific data.
  • Enter the pattern in Glob pattern format.
  • Exclude settings take precedence over Include settings.
  • Available only when [Data type] is set to Catalog Default.

Create tables in Data Catalog

You can create a table with a manually defined schema in the Data Catalog console, or by adjusting detailed scanner options.
For more information, see Create tables in Data Catalog.