AI Forest scenario
  • PDF

AI Forest scenario

  • PDF

It is available in a VPC environment.

The following shows the scenario where AI Forest is used to upload data to HDFS and execute jobs.

Create workspace

The following describes how to create a workspace.

  1. From the NAVER Cloud Platform console, click the Services > Big Data & Analytics > Data Forest menus, in that order.

  2. Click AI Forest > Workspace > [Create workspace] > [Advanced workspace].

  3. Enter account information.

    • Account name: df-test
    • Workspace name: ws
    • Workspace type: Singlebatch
  4. Select TensorFlow for the Docker image, and r2.3.1-py3 for the image version.
    df-af-quickstart_1-4_vpc_ko

    Note

    PyTorch is an open source machine learning library for Python programs. For more information, see PyTorch website.

  5. Select the default value for the GPU model name, number of GPU cores, and memory capacity.
    df-af-mnist_1-5_vpc_ko

  6. Enter the input items in the data settings area, and then click the [Add] button.

    • Input
      • Input Path: the path for input data to be copied from HDFS to the container Enter "/user/{username}/data_in."
      • Input Container Local Path: the path of the data saved after being copied to the container Enter "data_in."
    • Output
      • Output Path: the path for output data to be copied from the container to HDFS Enter "/user/{username}/data_out."
      • Output Container Local Path: the path of the output data saved in the container Enter "data_out."
        df-af-quickstart_1-6_vpc_ko.png
  7. Click the [Next] button. The workspace creation is completed.

Upload data to HDFS

If the user uses data sets for manual model learning, then the data set must be in the input path entered at the time of creating the workspace for Data Forest HDFS.

Upload the data to the HDFS path below. Uploaded data sets are copied under the Input Container Local Path entered at the time of creating the workspace.

  • hdfs://koya/user/{Data Forest account}/data_in
    df-qs_05_vpc_ko.png
Note

You can use the HUE app to upload and download files to and from HDFS. For more information, refer to Using HDFS.

Write code in workspace

The following describes how to write code in a workspace.

  1. From the NAVER Cloud Platform console, click the Services > Big Data & Analytics > Data Forest menus, in that order.

  2. Click AI Forest > Workspace Browser.

  3. When the upload window appears, drag the run.sh file to the upload window.

    • run.sh input example

      #!/bin/bash
      ls -l /data_in
      
  4. Click the [Start upload] button.

  5. Click the [OK] button when the upload is completed.

Submit GPU batch jobs

  1. From the NAVER Cloud Platform console, click the Services > Big Data & Analytics > Data Forest menus, in that order.
  2. Click AI Forest > Workspace Browser.
  3. Select an account, and then a workspace.
  4. Mark the checkbox of the run.sh file which is a shell script file, and then click the [Run] button.
  5. Enter ./run.sh in the Command field when the Run window appears, and then enter aiforest in App name.
    image.png
  6. Click the [Run] button.

Check execution logs

The following describes how to check execution logs.

  1. From the NAVER Cloud Platform console, click the Services > Big Data & Analytics > Data Forest > App menus, in that order.
  2. Click Quicklinks > AppMaster URL under the app's details.
    df-af-quickstart_log01_vpc_ko.png
  3. Click the Cluster > Application menu.
  4. Find the ID of the application with the name field that begins with "aiforest," and click it.
    df-af-quickstart_log02_vpc.png
  5. Click the application ID's Logs. You can see the log for the executed app.
    df-af-quickstart_log03_vpc.png

Was this article helpful?