--- title: "Using iceberg" slug: "hadoop-vpc-using-iceberg" tags: ["Cloud Hadoop"] updated: 2026-09-17T09:17:45Z published: 2026-09-17T09:17:45Z canonical: "guide.ncloud-docs.com/hadoop-vpc-using-iceberg" stale: true --- > ## Documentation Index > Fetch the complete documentation index at: https://guide.ncloud-docs.com/llms.txt > Use this file to discover all available pages before exploring further. # Using iceberg Available in VPC [Iceberg](https://iceberg.apache.org/) is an open table type system for a vast analysis dataset that adds a table which uses a high-performance SQL table for Presto and Spark. ## Iceberg components Iceberg consists of 3 main components that form a hierarchy: the Iceberg Catalog, the Metadata Layer, and the Data Layer. ![chadoop-iceberg-1-1_ko](https://cdn.document360.io/6998976f-9d95-4df8-b847-d375892b92c2/Images/Documentation/chadoop-iceberg-1-1_en.png?sv=2026-02-06&spr=https&st=2026-11-17T06%3A09%3A30Z&se=2026-11-17T07%3A09%3A30Z&sr=b&sp=r&sig=e21Q%2Bw8pdP1BxZr5Y%2FeXU8b4KC9ZOEGDixB78MgHIXQ%3D) - **Iceberg Catalog Layer** - Used to identify the location to read or write data for a specified table. - Iceberg Catalog can find table metadata from the current point-in-time. - It can also find the metadata file that a query is looking for when the query is executed. - **Metadata Layer** - Consists of three components: a metadata file, a manifest list, and a manifest file. - Metadata files contain information about a table's schema, partition details, and snapshots, allowing you to quickly find the data required for queries. - **Data Layer** - The location where actual data files are stored, the data layer allows access to necessary data files using the metadata in the manifest file. ## Using iceberg To use iceberg, follow these steps: Note The following example is based on version 1.2.1: ### Test using hive shell Note - Iceberg is not globally registered in the default Hive classpath of Cloud Hadoop. When using Iceberg tables in Hive, you must specify the Iceberg runtime jar and libfb303 jar at Hive runtime. - To avoid affecting general Hive/Sqoop tasks, Iceberg-related jars are not globally registered in the cluster default settings. 1. Access Hive. ``` sudo su - hive ICEBERG_RUNTIME=/usr/nch/3.1.0.0-78/hive/lib/iceberg-hive-runtime-1.2.1.jar LIBFB303=/usr/nch/3.1.0.0-78/hive/lib/libfb303-0.9.3.jar [hive@nch023-ncl ~]$ hive \ --auxpath ${ICEBERG_RUNTIME}:${LIBFB303} \ --hiveconf hive.aux.jars.path=${ICEBERG_RUNTIME},${LIBFB303} Hive Session ID = cca75225-f55c-423b-b6c8-d8fb0 hive> set iceberg.engine.hive.enabled=true; hive> set iceberg.engine.hive.lock-enabled=false; hive> set hive.vectorized.execution.enabled=false; hive> set hive.execution.engine=mr; hive> set tez.mrreader.config.update.properties=hive.io.file.readcolumn.names,hive.io.file.readcolumn.ids; ``` 1. Create database. ``` hive> create database test; OK Time taken: 2.182 seconds ``` 1. Select database. ``` hive> use test; OK Time taken: 0.278 seconds Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases. ``` 1. Create table. ``` hive> CREATE EXTERNAL TABLE test_tbl (id int) STORED BY 'org.apache.iceberg.mr.hive.HiveIcebergStorageHandler' TBLPROPERTIES ( 'engine.hive.enabled'='true', 'engine.hive.lock-enabled'='false' ); OK Time taken: 2.796 seconds ``` 1. Use INSERT to insert data. ``` hive> INSERT INTO test_tbl values (1); Query ID = hive_20231012143056_a80b-fe72-472a-8773-4e7589 Total jobs = 1 Launching Job 1 out of 1 Number of reduce tasks is set to 0 since there's no reduce operator 23/10/12 14:30:57 INFO client.AHSProxy: Connecting to Application History server at nch023-ncl.nfra.io/10.168.142.23:10200 23/10/12 14:30:57 INFO client.AHSProxy: Connecting to Application History server at nch023-ncl.nfra.io/10.168.142.23:10200 Starting Job = job_1696850670798_0017, Tracking URL = http://nch2-ncl.nfra.io:8088/proxy/application_1696850670798_0017/ Kill Command = /usr/nch/3.1.0.0-78/hadoop/bin/mapred job -kill job_1696850670798_0017 Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 0 2023-10-12 14:31:07,818 Stage-2 map = 0%, reduce = 0% 2023-10-12 14:31:16,035 Stage-2 map = 100%, reduce = 0%, Cumulative CPU 5.33 sec MapReduce Total cumulative CPU time: 5 seconds 330 msec Ended Job = job_16968506_0017 MapReduce Jobs Launched: Stage-Stage-2: Map: 1 Cumulative CPU: 5.33 sec HDFS Read: 173742 HDFS Write: 2611 SUCCESS Total MapReduce CPU Time Spent: 5 seconds 330 msec OK Time taken: 22.507 seconds ``` 1. Use SELECT to confirm data. ``` hive> select * from test_tbl; OK 1 Time taken: 0.493 seconds, Fetched: 1 row(s) ``` 1. Check the table schema. ``` hive> show create table test_tbl; OK CREATE EXTERNAL TABLE `test_tbl`( `id` int COMMENT 'from deserializer') ROW FORMAT SERDE 'org.apache.iceberg.mr.hive.HiveIcebergSerDe' STORED BY 'org.apache.iceberg.mr.hive.HiveIcebergStorageHandler' LOCATION 'hdfs://test-test/warehouse/tablespace/managed/hive/test.db/test_tbl' TBLPROPERTIES ( 'bucketing_version'='2', 'current-schema'='{"type":"struct","schema-id":0,"fields":[{"id":1,"name":"id","required":false,"type":"int"}]}', 'current-snapshot-id'='128779159509', 'current-snapshot-summary'='{"added-data-files":"1","added-records":"1","added-files-size":"407","changed-partition-count":"1","total-records":"1","total-files-size":"407","total-data-files":"1","total-delete-files":"0","total-position-deletes":"0","total-equality-deletes":"0"}', 'current-snapshot-timestamp-ms'='1697088677165', 'engine.hive.enabled'='true', 'external.table.purge'='TRUE', 'last_modified_by'='hive', 'last_modified_time'='1697088657', 'metadata_location'='hdfs://test-test/warehouse/tablespace/managed/hive/test.db/test_tbl/metadata/00001-33b09b82-b9b9-4005-a804-3f7970fc23ec.metadata.json', 'previous_metadata_location'='hdfs://test-test/warehouse/tablespace/managed/hive/test.db/test_tbl/metadata/00000-5a7c11d1-b12b-45a-a75a8c975f85.metadata.json', 'snapshot-count'='1', 'table_type'='ICEBERG', 'transient_lastDdlTime'='1697088657', 'uuid'='95dffef0-97e6-4ca2-ae01-b5bfde8') Time taken: 0.315 seconds, Fetched: 25 row(s) ```