- Print
- PDF
Table
- Print
- PDF
The latest service changes have not yet been reflected in this content. We will update the content as soon as possible. Please refer to the Korean version for information on the latest updates.
Available in VPC
A table is a metadata definition with details and schema of the data. You can create a table through the scanner or by defining your own schema. In the Table menu, you can create and manage tables and view collected metadata.
Table screen
The following is the basic description of the Table menu for Data Catalog.
Field | Description |
---|---|
① Menu name | Menu name currently being checked, number of tables being viewed |
② Basic features | Features displayed when initially entering the Table menu |
③ Post-creation features | Features activated after a table is created |
④ Search window | You can search by database name, table name, location, data format, and tag. You can also sort the order |
⑤ Table lists | List of tables being viewed, click to check detailed information |
⑥ Information tab | Check the relevant information upon clicking each tab |
Create table
You can create a table the way you want. You can create it as follows:
- Create table with manual schema definition: create tables by setting up your own database and schema
- Create table via scanner: automatically define schema via scanner to create table
Create tables with manual schema definition
You can create tables by setting up your own database and schema.
- Using Data Catalog as Cloud Hadoop's metastore, you can directly define and manage tables and table schemas. (will be supported)
- Even during ETL work that converts source/target data, you can directly define the schema for conversion. (will be supported)
The following describes how to generate tables with manual schema definition.
- From the NAVER Cloud Platform console, click the Services > Big Data & Analytics > Data Catalog menu, in order.
- Click the Table menu.
- Click the [Create table] button.
- Click Create tables with manual schema definition and click [Next].
- Enter basic information.
- Database: click the drop-down menu to select a database to connect the table
- Click the [Create database] button to create a database. (See Create database)
- Table name: enter a table name
- Location: where the data in the table exists
- Description: enter a table description
- Database: click the drop-down menu to select a database to connect the table
- Click the [Add] button and enter the schema information to add a custom schema.
- For more information on Data types, see Schema data type.
- Click the check box of the schema and click the [Delete] button to delete an added schema.
- If you do not add a custom schema, a schema with field name "default" will be added automatically.
- If you need to enter a partition key, click the Partition area and add the partition key.
- After clicking the [Add] button, enter the partition key name in the input box to add the partition key.
- You can select the partition by clicking its checkbox and then click the [Delete] button to delete the partition key.
- If necessary, click the Set tag area to add tags.
- After clicking the [Add] button, enter the tag information in the input box to add the tag.
- For more details on Tag type, see Tag data type.
- Click the check box of the tag to select it and click the [Delete] button to delete a tag.
- Click the [Load tag template] button to display the pop-up window for loading tag templates.
- Select and click a tag template and click the [Add] button to add a tag of the relevant tag template.
- For more details about tag templates, see Tag template.
- After clicking the [Add] button, enter the tag information in the input box to add the tag.
- Click the [Create] button.
Schema data type
The data types in the schema that can be defined manually and a description of each type are as follows:
Data type | Description |
---|---|
tinyint | Integer data (1 byte) |
smallint | Integer data (2 bytes) |
int | Integer data (4 bytes) |
bigint | Integer data (8 bytes) |
float | Floating decimal data (4 bytes) |
double | Floating decimal data (8 bytes) |
decimal | Fixed decimal data |
string | String data |
char | Fixed-length character type data |
varchar | Variable length character type data |
boolean | Data with true or false values |
binary | Binary data in char format |
timestamp | Date and time representation data, timestamp |
datetime | Date, time representative data (YYYY-MM-DD HH:MM:SS) |
date | Date representative data (YYYY-MM-DD) |
array | Collection of the data of the same type |
map | Data made of pairs of key and value |
struct | Various including various types of data and related schema |
uniontype | Type for storing various structure data types |
Examples of inputting detailed settings for each data type are as follows:
- <example> Detailed configuration of array type
ARRAY < STRUCT < place: STRING, start_year: INT > >
- <example> Detailed configuration of map type
MAP < STRING, ARRAY<STRING> >
- <example> Detailed configuration of struct type
STRUCT < place: STRING, start_year: INT >
- <example> Detailed configuration of uniontype type
UNIONTYPE < INT, DOUBLE, ARRAY<STRING>, STRUCT<a:INT,b:STRING> >
Create table via scanner
The following describes how to create a table by automatically defining the schema through the scanner.
- From the NAVER Cloud Platform console, click the Services > Big Data & Analytics > Data Catalog menu, in order.
- Click the Table menu.
- Click the [Create table] button.
- Click Create tables via scanner and click [Next].
- Move to the scanner creation screen.
- Tables are created automatically when you create and run the scanner.
- The table name is automatically set based on the name of the source data.
- For details on creating and running a scanner, see Scanner.
Search tables and check information
The following describes how to search for the created table and check the information.
- From the NAVER Cloud Platform console, click the Services > Big Data & Analytics > Data Catalog menu, in order.
- Click the Table menu.
- Enter the search conditions and click to search for the table.
- Click the table to check the information.
- Database: name of the database the table belongs to
- Table: table name
- Location: where the data in the table exists
- Data format: type of scanned data
- Created date: date when a table was first created
- Updated date: most recent date when editing a table’s information
- [Schema] tab: click to check the schema registered to the table
- For more information on data types, see Schema data type.
- [Partition] tab: click to check the partition registered to the table
- [Schema version] tab: click to view the schema version list, click a version to view the schema of that version
- [Tag] tab: click to check the tags registered in the table
- [Property data] tab: click to check property information of table and source data
- For more information on property keys, see Property data.
Property data
If you click the [Property data] tab in the table details area, you can check the property information of the table and source data. The following describes each information item.
Property key | Description |
---|---|
created_time | Mark unix time of table creation date and time |
last_modified_time | Mark unix time of table update date and time |
dataFormat | Format of data source |
delimiter | Delimiter if the source data is a CSV file |
dataType | Type of data source |
transient_lastDdlTime | Mark unix time of table DDL last change date |
scannerName | Scanner name of creating a table |
totalSize | Total amount of data scanned when the scan target is a directory |
numFiles | Total number of files scanned when the scan target is a directory |
isDirectory | TRUE if the scan target is a directory |
EXTERNAL | External storage |
scannerId | Scanner ID of creating a table |
connectionName | Connection name used to scan data |
inputFormat | Format for reading File into Object |
outputFormat | Format for writing File into Object |
serializationLib | Serializer and Deserializer Library |
clusterNo | Cluster number of the scanned Cloud data base product |
connectionId | Scanner connection ID that created a table |
mysqlCollation | Character sort setting of MySQL table |
mysqlDataSize | Data size of MySQL table |
mysqlIndexSize | Index size of MySQL table |
mysqlIndexes | Number of index of MySQL table |
mysqlRows | Number of saved row (record) of MySQL table |
mysqlTableSize | Total size of MySQL table |
mssqlCollation | Character sort setting of MSSQL table |
mssqlDataSize | Data size of MSSQL table |
mssqlIndexSize | Index size of MSSQL table |
mssqlIndexes | Number of index of MSSQL table |
mssqlRows | Number of saved row (record) of MSSQL table |
mssqlTableSize | Total size of MSSQL table |
postgresqlCollation | Character sort setting of PostgreSQL table |
postgresqlDataSize | Data size of PostgreSQL table |
postgresqlIndexSize | Index size of PostgreSQL table |
postgresqlIndexes | Number of index of PostgreSQL table |
postgresqlRows | Number of saved row (record) of PostgreSQL table |
postgresqlTableSize | Total size of PostgreSQL table |
mongodbAvgObjSize | Average document size of MongoDB collection |
mongodbFreeStorageSize | Size of available storage space in MongoDB database |
mongodbIndexSize | Index size of MongoDB collection |
mongodbIndexes | Number of index of MongoDB collection |
mongodbRowCount | Number of saved row (record) of MongoDB collection |
mongodbSize | Size of MongoDB database |
mongodbStorageSize | Storage size of MongoDB database |
mongodbTotalSize | Total size of MongoDB database |
Edit table
The following describes how to edit the information of the created table or to select the schema version.
The database included in the table name and the table cannot be edited.
- From the NAVER Cloud Platform console, click the Services > Big Data & Analytics > Data Catalog menu, in order.
- Click the Table menu.
- Click the table you want to edit and click the [Edit] button.
- Edit the table information on the table edit screen.
- You can edit the source data's location, table description, source data format, schema, and tags.
- Click the version drop-down menu in the schema area to select the schema version you want to edit.
- For further information of each item, see Create table.
- Click the [Save] button.
Delete table
The following describes how to delete the created table.
- If deleting a table, all meta information in the table are also deleted.
- If the deleted table is a Managed Table connected to Cloud Hadoop Hive, actual data may be deleted. (will be supported)
- A deleted table cannot be recovered.
- From the NAVER Cloud Platform console, click the Services > Big Data & Analytics > Data Catalog menu, in order.
- Click the Table menu.
- Click the table you want to delete and click the [Delete] button.
- When the notification pop-up window appears, check the cautions and click [Delete].