Cloud DB Scanner usage examples

Prev Next

Available in VPC

Quickstart overview

Describes the process of scanning the schema of the source data you want and creating a table by creating a scanner targeting Cloud DB for MySQL.

Source data

The source data stored in Cloud DB for MySQL consists of air pollution measurement data, including various measurement information such as date and time, monitoring station, ozone concentration, and sulfur dioxide concentration.

Data examples

date area_code area_name measure_center_code measure_center_name fine_dust_per_hour fine_dust_per_day ultrafine_dust_per_day ozone_ppm nitrogen_dioxide_concentration_ppm carbon_monoxide_concentration_ppm sulfurous_acid_gas_concentration_ppm
202210302300 100 downtown 111123 junggu 69 59 49 0.013 0.064 0.8 0.004
202210302300 100 downtown 111121 junggu-2 82 59 56 0.008 0.074 0.8 0.003
202210302300 100 downtown 111131 yongsangu 68 58 64 0.028 0.037 0.7 0.003

Create and run scanner

Create connection 

# Basic information
Name: (atmosphere-mysql-connection)

# Source data
Data type: (Cloud DB for MySQL)
DB service: (DB service name of the user's Cloud DB for MySQL)
Database: (name of the database to be scanned in the DB service)

# Create account
User name: (DB account name to be used to access Cloud DB for MySQL)
Password: (DB password to be used to access Cloud DB for MySQL)
  • Create Account>User Name/Password: create a new DB account to access source data. The newly created account is granted read-only permission which can be limitedly accessed from Data Catalog only.
Note

If the user name entered is a name created on another connection or a user name that already exists, the user name already registered is used and the password entered is ignored. (The user name and password previously registered are not changed.)

Create scanner 

# Source data
Data type: (Cloud DB for MySQL)
Connection: (atmosphere-mysql-connection)
Path: (not entered)

# Execution option
Execution cycle: (on-demand)
Pattern: (not used)
  • Path: when entering a value, it is automatically processed as "corresponding value + %" Among the tables in the database for scanning, tables with names starting with the value are scanned.
  • Pattern: when specific patterns are included/excluded, only tables with the names of the patterns can be scanned. If patterns are not used, all tables in the selected database path are scanned.

If both path and pattern conditions are set, among the tables meeting the path and the pattern condition, tables that satisfy the pattern condition are scanned.

# Output data
Database: (default)
Prefix: (not entered)
When adding a schema: (update table definition)

  • prefix: a prefix is added to the output data table name to distinguish itself from the output data of other scans.

  • When adding a schema

    • Update table definition: when scanning, every detail related to the table change is updated.
    • Add only new columns: updates are performed when scanning only if columns are added to the table, and deleted columns are ignored.
    • Ignore: ignore all table changes when scanning.

Run scanner

  • With the scanner waiting to run, press the [Run] button to start scanning.
  • Scanners in the Start running state will change to the Waiting to run state as soon as the scan is complete.
  • You can check the results in the History tab and the schema in the Table menu.

Schema run result

Additional table Property information Schema
atmosphere-data connectionName,
mysqlCollation,
mysqlDataSize,
mysqlIndexSize,
mysqlIndexes,
mysqlRows,
mysqlTableSize,
scannerId,
scannerName, ...		 date (bigint ) date (bigint )
area_code (double)
area_name (string)
measure_center_code (double)
measure_center_name (string)
fine_dust_per_hour (double)
fine_dust_per_day (double)
ultrafine_dust_per_day (double)
ozone_ppm (double)
nitrogen_dioxide_concentration_ppm (double)
carbon_monoxide_concentration_ppm (double)
sulfurous_acid_gas_concentration_ppm (double)

Table location

Each scanned table has its own location information. The rules for creation are as follows:

Table name: TableName
Location: /DatabaseName.TableName
  • If the table with the same table name and location already exists after scanning, the scanned table will be updated, as it is recognized as the existing table.
  • Therefore, note that when scanning multiple DBMSs, an update will be performed if the database and table names are identical, even if the DBMSs are different. It is recommended to add a prefix to the table name.