Using Kudu

Available in VPC

Kudu is a column-based storage used in the Hadoop platform environment. Kudu is specifically designed for the Hadoop ecosystem, enabling Spark, MapReduce, and other Hadoop ecosystem projects to process and analyze data natively. Like a general DBMS, it provides a primary key, enabling millisecond-level random access, and at the same time, it is optimized for large-scale sequential reading, so it can bridge the gap between HBase and Parquet. In addition, since it supports both OLAP and OLTP queries, Kudu can simplify the structure of big data analysis systems.

Features of Kudu – Impala Integration

The API provided by Kudu is simple CRUD operations. You can enter desired filter conditions when retrieving data, but since it provides only a simple data retrieval function, you need a separate query processing engine to execute complex queries using GROUP BY or JOIN. You can usually use Impala with Kudu to handle complex queries.

Feature	Description
CREATE/ALTER/DROP TABLE	Impala supports operations to create, alter, and drop tables in Kudu. Impala enables flexible data processing and querying using the same Internal/External approach as other tables.
INSERT	Impala inserts data into Kudu using the same mechanism as HDFS and HBase.
UPDATE/DELETE	Impala supports row-level or batch UPDATE and DELETE SQL statements for data in Kudu tables. Complex joins can be specified in the FROM clause of a query.
Flexible Partitioning	Similar to the table partitioning concept in Hive, Kudu dynamically partitions tables into a predefined number of Tablets based on hash or range partitioning to evenly distribute write and query operations across the cluster.
Parallel Scan	For optimal performance on commodity hardware, the Kudu client used by Impala parallelizes scans across multiple Tablets.

Kudu architecture

There are four major components that make up the Kudu service: Table, Tablet, Tablet Server, and Master Server.

Component	Description
Table	Stores Kudu data and has a schema and Primary key. One table is divided into segments called tablets by the primary key and stored.
Tablet	A partition of a single table, similar to a partition in a relational database. One tablet is replicated to multiple tablet servers, and one of the replicas is elected as Leader Tablet through the Raft Consensus Algorithm. All replicas can provide data read services, and write operations are performed through agreement between Tablet Server groups.
Tablet Server	Stores Tablets and serves them to clients. Among the replicas, a certain tablet acts as a leader and the rest as a follower.
Master Server	The master server tracks and manages all Tablets, Tablet Servers, Catalog Tables, and other metadata information related to the cluster. At a certain point in time, a single master serves as the leader, and if a problem occurs with the current leader, a new master is registered using the Raft Consensus algorithm. It is also responsible for coordinating metadata operations to clients. Clients can access catalog tables through the master using the client API.

chadoop-31-001

Using Kudu

It describes how to use Kudu.

Install Kudu service from Ambari UI

You can install Kudu using Add service in the Ambari UI. The options to enter during installation are as follows.

Advanced kudu-master-env

kudu_master_hosts = KUDU_master_installation_FQDN
kudu_master_webserver_port = 8011

Advanced kudu-site

kudu_builtin_ntp_servers = 169.254.169.123,169.254.169.124 0.0.0.0

Advanced kudu-tserver-env

kudu_tserver_master_address = KUDU_tablet_installation_FQDN:7051
kudu_tserver_webserver_port = 8012

Custom kudu-tserver-env

kudu_tserver_hosts = KUDU_tablet_installation_FQDN

Check Kudu service in Ambari UI

You can check Kudu services as follows: You can get start with the component and stop each of the service from this page.

chadoop-31-002

Summary: Checks hosts where components are installed.
Configs: Changes configuration of the Kudu service.
Quick Links: Kudu Master WEB-UI, Kudu Tserver WEB-UI
- Accessing these links requires tunneling. We recommend that you access through the Web UI link provided in the console. For more information, see Access Web UI using tunneling.

Documentation Index

Using Kudu

Features of Kudu – Impala Integration

Kudu architecture

Using Kudu

Install Kudu service from Ambari UI

Check Kudu service in Ambari UI