Using Ranger

Available in VPC

This guide explains how to configure policies with Apache Ranger in Data Forest to share your own resources with other users. It can be used by connecting to Ranger UI.

Connect to Ranger

1. Connect to the Ranger UI. You can check the access address of the Ranger UI by clicking the [View connection information details] button in the Quick links section of any Data Forest app details.
2. Enter the username and password set upon the creation of Data Forest account in the Username and Password fields to log in.
   
3. You can view the components that are available for setting permissions in the Ranger home screen.
   • You can set ACL for HDFS, Hive. This configuration is done in units called a "Policy".
     

Se HDFS ACL

The following describes how to set HDFS.

1. From the Apache Ranger home screen, click HDFS > df_hadoop_koya or df_hadoop_tata.
   • df_hadoop_koya: Manage ACL for files under the koya namespace.
   • df_hadoop_tata: Manage ACL for files under the tata namespace.
2. Click the [Add New Policy] button.
   • Add permissions for directories and files (/user/example/foo, /user/example/bar) created by example user to example2.
     
3. Configure policy.
   • Policy Name: Set policy name. (A policy name already in use can't be used.)
   • Resource Path: Enter the HDFS path to set ACL.
     • Data Forest users can access the home directory hdfs:///user/$username path in HDFS, and can manage ACL only for the files under the home directory. (The paths such as hdfs:///tmp can't be used.)
     • Enter the path starting with /user/$username, without the hdfs:// scheme at the beginning.
     • Multiple paths can be entered.
     • The validity of the path is not checked, so make sure to enter it correctly.
     • Attention is required so that the path doesn't end with "/." (For example, if you enter the path as /user/example/foo/, the access can only be made to subdirectories but not to foo.)
   • Select Group: Specify the group to give permissions.
     • All users in the group will be given the permission.
     • All users belong to the users group in Data Forest . When creating accounts, they can't be designated to a different group.
   • Select User: Specify the user to give permissions.
   • Permission: Select the permissions to give. (permissions: Execute, Read, and Write)
     • For directories, you need the permission to Execute to get the file list (ls command) within the directory.
   • Delegate Admin
     • If the admin permission is delegated to a user/group by checking the Delegate Admin option, then the user/group can manage the policy. Take caution to not delegate the admin permission to many unspecified persons.
   • Add Validity Period: It can be set up to apply a policy during a specific period
     
4. Click the [Add] button. It takes about 5 minutes for a new policy to be created.
5. Check if the new policy has been created.
   

The structure of /user/example is as follows.
The permission is 'drwx------'. Adding policies does not change the permission itself.

$ hadoop fs -ls /user/example
Found 9 items
...
drwx------   - example services          0 2021-04-14 18:01 /user/example/bar
drwx------   - example services          0 2021-04-14 18:01 /user/example/foo
drwx------   - example services          0 2021-04-14 18:07 /user/example/foobar

If you try to access /user/example/foobar with the example2 account with the permission given by the policy, then you can see the access is unavailable because the permission is given only for foo and bar.

$ hadoop fs -ls /user/example/foobar
s: Permission denied: user=example2, access=READ_EXECUTE, inode="/user/example/foobar":example:services:drwx------

You can see that foo and bar, to which you have the permissions granted from the policy, can be accessed normally.

$ hadoop fs -ls /user/example/foo
Found 1 items
-rw-------   3 example services          0 2021-04-14 18:13 /user/example/foo/foo.txt
$ hadoop fs -ls /user/example/bar
Found 1 items
-rw-------   3 example services          0 2021-04-14 18:14 /user/example/bar/bar.txt

The permissions of Read, Write, and Execute are given, so the files can be copied as follows.

$ hadoop fs -copyToLocal /user/example/foo/foo.txt

Se Hive ACL

When Hive databases or tables are created, only the owners can access them. In order to allow other users to access, a permission should be granted through Ranger.

The following describes how to set Hive ACL.

1. Click Hive > df_hive from the Ranger home screen.
2. Click the [Add New Policy] button.
3. Enter the information.
   • The example of giving the permission for the table example__db_foo.bar_table of user example to example2
   • Policy Name: Set policy name. The policy name already in use can't be used.
   • Database: Enter the database to configure ACL. Multiple databases can be entered, and whether the database exists is not checked.
   • Table: Enter the table to configure ACL.
     • Whether the table exists is not checked.
     • All tables can be specified with *.
   • Hive Column: Data Forest does not support column-level ACLs. Enter *
   • Select Group: Specify the group to give permissions.
     • All users in the group will be given the permission.
     • All users in Date Forest belong to the users group. When creating accounts, separate group creation can't be designated.
   • Select User: Specify the user to give permissions.
   • Permission: Select the permissions to give
   • Delegate Admin: If the admin permission is delegated to a user/group by checking the Delegate Admin option, then the user/group can manage the policy. Take caution to not delegate the admin permission to many unspecified persons.
   • Add Validity Period: You can configure application period of a permission.
4. Click the [Add] button. It takes about 5 minutes for a new policy to be created.
5. Check if the new policy has been created.

Apply HDFS ACL

Since Hive table data is stored in HDFS, you must apply HDFS ACL as well as Hive ACL settings. If HDFS ACL is not configured, then read queries will fail.

HDFS ACLs to view table

Grant access permission to bar_table only. The permission is granted for all paths under bar_table (recursive).

Service: df_hadoop_koya
Resource Path: /user/example/warehouse/example__db_foo.db/bar_table   (recursive)
Select User: example2
Permissions: Read/Execute

HDFS ACLs to use database

It's a permission to use example__db_foo. Set it to non-recursive, as we're not going to grant the permission for all subordinate tables.

Service: df_hadoop_koya
Resource Path: /user/example/warehouse/example__db_foo.db   (non-recursive)
Select User: example2
Permissions: Read/Execute

You can send queries to example__db_foo.bar_table from the "example2" account.

Before setting permission

Permission error is returned.

0: jdbc:hive2://zk1.kr.df.naverncp.com:2181,z> select * from example__db_foo.bar_table;
Error: Error while compiling statement: FAILED: SemanticException Unable to fetch table bar_table. java.security.AccessControlException: Permission denied: user=example2, access=EXECUTE, inode="/user/example":example:services:drwx------

0: jdbc:hive2://zk1.kr.df.naverncp.com:2181,z> use example__db_foo;
Error: Error while compiling statement: FAILED: SemanticException MetaException(message:java.security.AccessControlException: Permission denied: user=example2, access=EXECUTE, inode="/user/example":example:services:drwx------

After setting permission

You can see that the normal access is granted after setting the permission.

0: jdbc:hive2://zk1.kr.df.naverncp.com:2181,z> select * from example__db_foo.bar_table;
INFO  : Compiling command(queryId=hive_20210415150318_9cd21ba3-9862-4310-9a53-69c5af0dd105): select * from example__db_foo.bar_table
INFO  : Semantic Analysis Completed (retrial = false)
INFO  : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:bar_table.id, type:bigint, comment:null)], properties:null)
INFO  : Completed compiling command(queryId=hive_20210415150318_9cd21ba3-9862-4310-9a53-69c5af0dd105); Time taken: 0.096 seconds
INFO  : Executing command(queryId=hive_20210415150318_9cd21ba3-9862-4310-9a53-69c5af0dd105): select * from example__db_foo.bar_table
INFO  : Completed executing command(queryId=hive_20210415150318_9cd21ba3-9862-4310-9a53-69c5af0dd105); Time taken: 0.001 seconds
INFO  : OK
+---------------+
| bar_table.id  |
+---------------+
+---------------+
No rows selected (0.122 seconds)
0: jdbc:hive2://zk1.kr.df.naverncp.com:2181,z> use example__db_foo;
INFO  : Compiling command(queryId=hive_20210415150322_ac4aa103-4ae3-483f-b388-43040cede0b5): use example__db_foo
INFO  : Semantic Analysis Completed (retrial = false)
INFO  : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
INFO  : Completed compiling command(queryId=hive_20210415150322_ac4aa103-4ae3-483f-b388-43040cede0b5); Time taken: 0.02 seconds
INFO  : Executing command(queryId=hive_20210415150322_ac4aa103-4ae3-483f-b388-43040cede0b5): use example__db_foo
INFO  : Starting task [Stage-0:DDL] in serial mode
INFO  : Completed executing command(queryId=hive_20210415150322_ac4aa103-4ae3-483f-b388-43040cede0b5); Time taken: 0.01 seconds
INFO  : OK
No rows affected (0.039 seconds)
0: jdbc:hive2://zk1.kr.df.naverncp.com:2181,z> select * from bar_table;
INFO  : Compiling command(queryId=hive_20210415150324_448ce1be-b913-4396-8232-92f76feb40e5): select * from bar_table
INFO  : Semantic Analysis Completed (retrial = false)
INFO  : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:bar_table.id, type:bigint, comment:null)], properties:null)
INFO  : Completed compiling command(queryId=hive_20210415150324_448ce1be-b913-4396-8232-92f76feb40e5); Time taken: 0.096 seconds
INFO  : Executing command(queryId=hive_20210415150324_448ce1be-b913-4396-8232-92f76feb40e5): select * from bar_table
INFO  : Completed executing command(queryId=hive_20210415150324_448ce1be-b913-4396-8232-92f76feb40e5); Time taken: 0.001 seconds
INFO  : OK
+---------------+
| bar_table.id  |
+---------------+
+---------------+