Manage Datafence

Prev Next

Available in VPC

You can configure and manage the Datafence environment (its shared storage) and boxes created within Datafence.

Configure Datafence environment

This section describes how to configure the Datafence server and its shared storage. The available configuration features are as follows:

Configure SSL VPN

To access the Datafence environment, you must use SSL VPN. Once SSL VPN is created in the VPC to which Datafence belongs, you can select and configure that specific SSL VPN.
To configure an SSL VPN:

  1. In the VPC environment on the NAVER Cloud Platform console, navigate to i_menu > Services > Big Data & Analytics > Datafence.
  2. Click the My Datafence menu.
  3. Select the tab of the Datafence to configure and click [Set SSL VPN].
  4. Select SSL VPN in the Set SSL VPN popup and click [Yes].
    • The selected SSL VPN is established as the Datafence access environment.

Edit Datafence environment

You can add, edit, and delete Datafence servers or change the settings of the NAS used as the shared data storage.
To edit the Datafence environment:

  1. In the VPC environment on the NAVER Cloud Platform console, navigate to i_menu > Services > Big Data & Analytics > Datafence.
  2. Click the My Datafence menu.
  3. Select the tab of the Datafence to configure and click [Edit Fence environment].
  4. Go to the Create Datafence environment (shared storage) step in the Edit Datafence environment popup, edit the information you want, and click [Next].
    • My Datafence server
      • Add server: Enter the server image, specifications, storage capacity, and quantity, then click [Add].
      • Edit server: Click an editable field in the registered server information to update it.
      • Delete server: Click [Delete] for the server you want to delete.
    • Save shared data
      • Add volume: Enter the capacity and quantity of shared data storage and click [Add].
      • Delete volume: Click [Delete] for the NAS volume you want to delete.
    • Memo: If needed, leave a memo of 50 characters or less.
  5. In the Set access information step, enter the password of the server's access account, and click [Next].
    • Enter a combination of 8 to 14 characters using English alphabets (both cases), numbers, and special characters. You cannot use quotation marks, backtick, ₩, /, &, $, and space among special characters.
    • If no server has been added, this step may be omitted.
  6. Review the edited details and click [Complete].
    • Datafence's configuration will be updated based on your input.

Reboot server

To reboot a Datafence server that manages analysis target data:

  1. In the VPC environment on the NAVER Cloud Platform console, navigate to i_menu > Services > Big Data & Analytics > Datafence.
  2. Click the My Datafence menu.
  3. Click the tab for the Datafence to configure, select the Datafence server to reboot, and then click [Reboot server].
  4. In the Reboot server popup, click [Yes].
    • The reboot of the selected server starts.

Change NAS capacity

To change the capacity of NAS used as the shared data storage:

  1. In the VPC environment on the NAVER Cloud Platform console, navigate to i_menu > Services > Big Data & Analytics > Datafence.
  2. Click the My Datafence menu.
  3. Click the tab for the Datafence to configure, select the NAS volume, and then click [Change NAS capacity].
  4. In the Edit NAS capacity popup, enter the new capacity and click [OK].
    • The NAS capacity is updated as entered.

Reset password

To change the password of the account used to access the Datafence server:

  1. In the VPC environment on the NAVER Cloud Platform console, navigate to i_menu > Services > Big Data & Analytics > Datafence.
  2. Click the My Datafence menu.
  3. Click the tab for the Datafence to configure, select the Datafence server, and then click [Reset password].
  4. In the Reset password popup, enter a new password.
    • Enter a combination of 8 to 14 characters using English alphabets (both cases), numbers, and special characters. You cannot use quotation marks, backtick, ₩, /, &, $, and space among special characters.
  5. Click [OK].

Configure external access ACG

A public IP assigned to the Datafence server allows the server access from outside sources. You can restrict access using the Access Control Group (ACG) and enhance security.
To access Datafence through a specific external server, configure ACG rules to allow that server's access. To configure ACG rules:

Caution

If you allow access from an outside source, that path can be exploited for a data breach. Exercise caution and enable external access only when strictly necessary.

  1. In the VPC environment on the NAVER Cloud Platform console, navigate to i_menu > Services > Big Data & Analytics > Datafence.
  2. Click the My Datafence menu.
  3. Select the tab of the Datafence to configure and click [Configure ACG rules regarding access to Fence server from outside sources].
  4. Read the checklist, indicate your agreement, and configure the access permission rules.
    • Add rule: Enter the protocol, source IP address, allowed port, and memo, then click [Add].
    • Delete rule: From the list of entered information, click [Delete].
  5. Once the settings are configured as desired, click [OK].

Configure ACG for internal communication

Mutual communication between the Datafence servers in Datafence and the servers in the box is blocked by default.
If you need the servers in the box to communicate its log information or other configuration values, configure ACG rules to allow internal communication.
To configure communication between the Datafence server and the box within Datafence:

  1. In the VPC environment on the NAVER Cloud Platform console, navigate to i_menu > Services > Big Data & Analytics > Datafence.
  2. Click the My Datafence menu.
  3. Select the tab of the Datafence to configure and click [Configure ACG rules for communication between Fence environment ↔ data box].
  4. Read the checklist, indicate your agreement, and configure the access permission rules.
    • Add rule: Enter the transfer direction, protocol, allowed port, and memo, and click [Add].
      • Regardless of the number of Boxes, the specified port is opened between the Datafence area and the Box component.
    • Delete rule: From the list of entered information, click [Delete].
  5. Once the settings are configured as desired, click [OK].

Create server image

To create an image of the Datafence server:

  1. In the VPC environment on the NAVER Cloud Platform console, navigate to i_menu > Services > Big Data & Analytics > Datafence.
  2. Click the My Datafence menu.
  3. Click the tab for the Datafence to configure, select the Datafence server, and then click [My server image].
  4. Review the details in the Create server image popup and click [Create].
    • The server image is created.

Configure box

This section describes how to configure the box included in the Datafence environment. The available configuration features are as follows:

Add box

To add a new box to the Datafence environment:

  1. In the VPC environment on the NAVER Cloud Platform console, navigate to i_menu > Services > Big Data & Analytics > Datafence.
  2. Click the My Datafence menu.
  3. Click the tab for the Datafence to configure, and in the data box component, click [Add box].
  4. Enter the information in the Create analysis environment step and click [Next] in the Add box popup.
    • Connect Server, Tensorflow Server, or Server: Enter the server image, specifications, storage capacity, and quantity, and then click [Add].
    • Hadoop Cluster: Enter the node-specific specifications, storage capacity, and quantity, and click [Add].
    • NAS: Enter the capacity and quantity, and click [Add].
    • Memo: If needed, leave a memo of 50 characters or less.
    Note

    The supported specifications and limits of each infrastructure are the same as the supported specifications of the Analysis box environment when creating Datafence.

  5. In the Set box access information step, enter the password for the server access account and click [Next].
    • Enter a combination of 8 to 14 characters using English alphabets (both cases), numbers, and special characters. You cannot use quotation marks, backtick, ₩, /, &, $, and space among special characters.
  6. Review your input and click [Complete].
    • The box is created.

Edit box

To edit the information of an existing box:

  1. In the VPC environment on the NAVER Cloud Platform console, navigate to i_menu > Services > Big Data & Analytics > Datafence.
  2. Click the My Datafence menu.
  3. Click the tab for the Datafence to configure, select the box, and then click [Edit box].
  4. In the Edit box popup, update the required information in the Create analysis environment step, and click [Next].
    • Connect Server/Tensorflow Server/Server
      • Add server: Enter the server image, specifications, storage capacity, and quantity, and then click [Add].
      • Edit server: Click an editable field in the registered server information to update it.
      • Delete server: Click [Delete] for the server you want to delete.
    • Hadoop Cluster
      • Add cluster: Enter the node-specific specifications, storage capacity, and quantity, and click [Add].
      • Edit cluster: Click editable items in the registered cluster information to update the settings.
      • Delete cluster: Click [Delete] for the cluster you want to delete.
    • NAS
      • Add volume: Enter the capacity and quantity, and click [Add].
      • Delete volume: Click [Delete] for the volume you want to delete.
    • Memo: If needed, leave a memo of 50 characters or less.
  5. In the Set box access information step, enter the password for the server access account and click [Next].
    • Enter a combination of 8 to 14 characters using English alphabets (both cases), numbers, and special characters. You cannot use quotation marks, backtick, ₩, /, &, $, and space among special characters.
    • If no server has been added, this step may be omitted.
  6. Review your input and click [Complete].
    • The box's configuration will be updated based on your input.

Delete box

To delete an unnecessary box in the Datafence environment:

  1. In the VPC environment on the NAVER Cloud Platform console, navigate to i_menu > Services > Big Data & Analytics > Datafence.
  2. Click the My Datafence menu.
  3. Click the tab for the Datafence to configure, select the box, and then click [Delete box].
  4. In the Delete popup, click [OK].
    • After deletion, the box is removed from the list.

Box details interface

You can access the Box details interface to configure or check whether external network access is blocked and whether public data access is enabled. You can also edit the properties of the infrastructure created in the box.

Click the tab for the Datafence in the My Datafence interface, then click [View] for the Box you want from the Box list.
The Box interface includes the following components:

datafence-manage-vpc_screen_ko

Component Description
① Status display component Check box attributes and operational status.
  • Network blocking status: Indicates whether external network communication is blocked. Settings can be updated.
  • Shared data access: Displays the number of shared data access requests. The access status can be updated.
  • Individual storage status: Shows the number of data transfers to individual storages in the box and the capacity in use.
  • Export review status: Shows the number of review requests to export analysis results and their statuses. Click [Export details] to access the Export Approval menu interface.
  • Creation date and time: Date and time when the box was created.
  • Status: Current status of the box.
  • Memo: A memo entered when creating the box. Click View to review the content. To edit the content, click [Edit].
② Infrastructure list component The component for the list of the infrastructure created in the box

Allow and block external communication

Configure a communication network to manage the data for analysis. A box is by default blocked from the Internet. However, if you need Internet access while configuring its analysis environment, you can temporarily allow external communications and block them afterward to enhance security.
To allow or block external communications:

Caution

If you allow external communication, its path can be exploited for a data breach. Exercise caution and restrict allowances for special circumstances.

  1. In the VPC environment on the NAVER Cloud Platform console, navigate to i_menu > Services > Big Data & Analytics > Datafence.
  2. Click the My Datafence menu.
  3. Click the Datafence tab to configure, and in the data box component, click [View] in the list.
  4. Find the Network blocking status in the upper section of the View box details interface and click the dropdown list button.
  5. Select whether to allow or block external communication.

Change the shared data access status

The data for analysis is mounted to each data group's box via the data-group-specific access feature.
The Datafence administrator can update the shared data's access status upon request from an analyst or to meet mid-task needs.
To change the shared data access status:

  1. In the VPC environment on the NAVER Cloud Platform console, navigate to i_menu > Services > Big Data & Analytics > Datafence.
  2. Click the My Datafence menu.
  3. Click the Datafence tab to configure, and in the data box component, click [View] in the list.
  4. Find the Shared data access status in the upper section of the View box details interface and click [Change].
  5. Configure access settings for each data group in the Change Shared Data Access Status popup.
  6. Click [OK].
    • The data group's access status and number of views are updated.

Reboot server

To reboot a selected Connect, TensorFlow, or Linux server in a box:

  1. In the VPC environment on the NAVER Cloud Platform console, navigate to i_menu > Services > Big Data & Analytics > Datafence.
  2. Click the My Datafence menu.
  3. Click the Datafence tab to configure, and in the data box component, click [View] in the list.
  4. In the View box details interface, select a server from the infrastructure list and click [Reboot server].
  5. In the Reboot server popup, click [OK].
    • The reboot of the selected server starts.

Change NAS capacity

To change the capacity of an individual NAS within a box:

  1. In the VPC environment on the NAVER Cloud Platform console, navigate to i_menu > Services > Big Data & Analytics > Datafence.
  2. Click the My Datafence menu.
  3. Click the Datafence tab to configure, and in the data box component, click [View] in the list.
  4. In the View box details interface, select a NAS from the infrastructure list and click [Change NAS capacity].
  5. In the Edit NAS capacity popup, enter the new capacity and click [OK].
    • The NAS capacity is updated as entered.

Reset password

To change the password of the account used to access each server and the Hadoop cluster in a box:

  1. In the VPC environment on the NAVER Cloud Platform console, navigate to i_menu > Services > Big Data & Analytics > Datafence.
  2. Click the My Datafence menu.
  3. Click the Datafence tab to configure, and in the data box component, click [View] in the list.
  4. In the View box details interface, select an infrastructure from the infrastructure list and click [Reset password].
  5. In the Reset password popup, enter a new password.
    • Enter a combination of 8 to 14 characters using English alphabets (both cases), numbers, and special characters. You cannot use quotation marks, backtick, ₩, /, &, $, and space among special characters.
  6. Click [OK].

Set ACG exception

For security, access from outside sources to the box is blocked by default.
To allow a specific external server access to the box, configure the ACG rules to make an exception. To configure ACG rules:

Caution

If you allow access from an outside source, that path can be exploited for a data breach. Exercise caution and enable external access only when strictly necessary.

  1. In the VPC environment on the NAVER Cloud Platform console, navigate to i_menu > Services > Big Data & Analytics > Datafence.
  2. Click the My Datafence menu.
  3. Click the Datafence tab to configure, and in the data box component, click [View] in the list.
  4. In the View box details interface, click [Configure ACG exception].
  5. Read the checklist, indicate your agreement, and configure the access permission rules.
    • Add rule: Enter the transfer direction, protocol, access source's IP address, allowed port, and memo, and click [Add].
    • Delete rule: From the list of entered information, click [Delete].
  6. Once the settings are configured as desired, click [OK].

Create server image

To create a server image in a box:

  1. In the VPC environment on the NAVER Cloud Platform console, navigate to i_menu > Services > Big Data & Analytics > Datafence.
  2. Click the My Datafence menu.
  3. Click the Datafence tab to configure, and in the data box component, click [View] in the list.
  4. In the View box details interface, select a server from the infrastructure list and click [My server image].
  5. Review the details in the Create server image popup and click [Create].
    • The server image is created.

Access box

The analyst needs to access the box via the received box information to analyze the target data. This guides on how to access the box and how to access the analysis environment within the box.

Note

SSL VPN connection is required before accessing the Box.

To connect SSL VPN:

  • Install SSL VPN Agent.
    • For more information on how to install the SSL VPN Agent, see Install SSL VPN Agent in the SSL VPN user guide (VPC).
  • Run the BIG-IP Edge Client.
    • For more information on how to access the BIG-IP Edge Client, see Access SSL VPN Agent in the SSL VPN user guide (VPC).
  • Enter the registered username and password and click the [Log on].
  • Enter the OTP code sent to your mobile phone or email account and click the [Log on].

Access Connect Server

To use Cloud Hadoop, TensorFlow, Linux, and Windows Server installed in the Box, you need to access Connect Server. To this end, this guide describes how to access Connect Server.

  1. Because the Connect Server is Windows-based, access it using Remote Desktop on your PC.
  2. After entering the Connect Server's IP, click [Connect] and enter your username and password.

Check data storage

All Connect Servers have the NAS mounted in the same folder, and mount location has rules as follows: You can check the NAS name in the View box details screen.

  • NAS used for requesting file import or export: C:\mount\NAS name.
  • Shared storage NAS: C:\mount\nasr\NAS name.

Cautions when using NAS in Connect / Windows Server

Caution

NAS is currently used with limits in the Connect Server and Windows Server. Thoroughly review the following items before use:

Mismatch of character encoding method

NAS provided by Datafence uses an NFS protocol usable for Linux Server.
The following issues can occur from the difference between the encoding method used by Linux and Windows Operating System (OS):

  • When the file name is created using Korean characters, the file name from each OS appears inaccurately.
    • When a file is created using Korean characters in NAS from Connect/Windows, the file name appears in an unknown format.
    • When a file is created using Korean characters in NAS from Linux, the file name appears in an unknown format in Connect/Windows.
  • When the file name is in Korean when importing files, the file name appears in an unknown format in Connect/Windows.
  • When a file is exported, the file name that appears in Linux is used as the standard. When the name of the created file is in Korean in Connect/Windows, it appears as an unknown format in Linux so export is impossible.
  • The new line difference between Connect/Windows and Linux can cause each file description to appear differently.

Actions usable for files and folders

Actions usable for file and folder of NAS in Connect/Windows Server are as follows:

File

GUI (file explorer) Command prompt
Create O (unable to specify name) O
Read O O
Copy O O
Move O O
Delete O O
Edit name X O
Edit file content O -

Folder

GUI (file explorer) Command prompt
Create O (unable to specify name) O
Copy O O
Move O X
Delete O O
Edit name X X

Use and access Cloud Hadoop

Configuring SSH for Hadoop access on Connect Server requires the following 2 steps:

  1. Convert authentication key
  2. Connect to cluster node

Convert authentication key

  1. After accessing Connect Server, run PuTTYgen.
  2. In Type of key to generate, select RSA and click [Load].
  3. In Users > Public > Desktop, select databox.pem and click Open.
  • To find a file in PEM format, select the option that allows you to see files of all types.
  1. Review the content in the Completion confirmation popup and click [OK].
  2. Click [Save private key] and save it in PPK format so that it can be used in PuTTY.
  • If PuTTYgen displays a warning message about saving a key without a password, click [Yes].
  • Save it using the same name as the previously generated authentication key. PuTTY automatically adds the .ppk file extension.

Access Hadoop node SSH

  1. After accessing Connect Server, run PuTTY.
  2. Select Session in the Category window and enter the following values in each field:
  • HostName(or IP address) : sshuser@HadoopIP
    • You can check Hadoop's IP in the View box details screen.
  • Port : 22
  1. In the Category window, select Connection > SSH > Auth > Credentials.
  2. In Private key file for authentication, click [Browse], select the PPK file converted from the PEM file, and then click [Open].
  3. Verify that the connection to Hadoop has been successfully established, as shown in the example below.

datafence-manage-ssh