Using Impala

Available in VPC

Impala is a massively parallel processing engine that can analyze stored data in real time using interactive SQL. It can read data from various storage sources such as HBase and Kudu, as well as Hadoop HDFS, and be used in conjunction with Hive and Hue. It uses your existing Hive metastore without the need to use a dedicated metastore.
Impala appeared to solve the real-time query performance problem and multi-user support of Hive. While Hive and Pig process queries through the Map-Reduce framework, Impala has the fastest response speed because it uses its own distributed query engine.

Impala components

Impala service consists of three major components: Impala Daemon, StateStore, and Catalog Service.

Query execution process

To execute the query:

1. Perform a query using the Impala shell, ODBC, and so on in a specific impalad within the cluster.
2. Impalad retrieves the table schema from the Hive metastore, determines the suitability of the query statement, and collects data blocks and location information required for query execution from the HDFS name node.
3. Based on the recently updated Impala metadata, information necessary for query execution is propagated to all impalads in the cluster.
4. All impalads that receive the query and metadata read the data block to be processed from the local directory and process the query.
5. When tasks are completed in all impalads, the impalad that received the query from the user collects the results and delivers them to the user.

Using Impala

It describes how to use Impala.

Create cluster

From the NAVER Cloud Platform console, create the Cloud Hadoop cluster. For more information about creating clusters, see Create cluster.

Install Impala service from Ambari UI

You can install Impala using Add Service in the Ambari UI.

The options to enter during installation are as follows.
Advanced impala-env

Check Impala service from Ambari UI

You can check Impala services as follows: You can get start with the component and stop each of the service from this page.

• Summary: Checks hosts where components are installed.
• Configs: Changes configuration of the Impala service.
• Quick Links: Impala Statestore WEB-UI, Impala Catalog WEB-UI, Impala Server WEB-UI
  • Accessing these links requires tunneling. We recommend that you access through the Web UI link provided in the console. For more information, see Access Impala WEB UI.

Impala shell

Trino CLI provides a responsive shell to run queries. For more information about Impala shell, see Impala shell Documentation.

• Connect Impalad from Impala shell

  /usr/lib/impala_3_1_0_0_78/impala-shell/impala-shell -i <FQDN with hiveserver2 installed>
  
  Example: impala-shell -i m-001-cluster-name-***-hd.kr.ch.naverncp.com;
  

• Change Hive properties to create table
  After changing the value of Ambari UI > Hive > CONFIGS > ADVANCED > Advanced hive-site > hive.strict.managed.tables to false, save it and restart Hive.

• Create database and table

  CREATE database test;
  USE test;
  CREATE TABLE testTable
  (
  ID INT
  );
  

• Search after saving data

  INSERT INTO testTable(ID)
  VALUES (1);
  SELECT * FROM testTable;
  

Accessing Impala WEB UI

You can check the Impala editor in the Hue WEB UI.