Creating Ncloud TensorFlow Server (VPC, Public Subnet)

Available in VPC

1. About Ncloud TensorFlow Server

Ncloud TensorFlow Server provides typical deep learning and machine learning packages, including TensorFlow which is one of the most popular among data analysts, as installation types.

Most analysis jobs can be performed since data preprocessing, visualization, natural language processing, and machine learning and deep learning packages are provided. If necessary, you can easily install an additional package or upgrade an existing one by using the GUI or console command in Jupyter Notebook.

The web-based Jupyter Notebook allows you to perform jobs in the same analysis environment regardless of location, instantly check analysis results, and easily share them with others.

It also provides an administration script that enables you to easily manage Jupyter processes in a terminal environment.

For beginner analysts just starting their analysis jobs, machine learning (e.g., regression, classification, and clustering using scikit-learn) and deep learning regression and classification examples using Keras and TensorFlow, as well as data search and visualization, are provided.

2. FAQ before use

Q. What is Ncloud TensorFlow Server?

• Ncloud TensorFlow Server facilitates quick implementation of deep learning and machine learning by providing typical deep learning and machine learning packages, including TensorFlow, as well as the web-based Jupyter Notebook that can write code.
• Ncloud TensorFlow Server uses TensorFlow, which is an open source machine learning software library developed by the Google Brain team.

Q. What kinds of products are there?

• There are two products, ubuntu-16 and centOS-7, depending on the OS. (OS versions may differ.)
• The only difference lies in the OS, and the packages provided are the same. If you don't have a particular preference, then we recommend that you select the Ubuntu product, which is more commonly used.

Q. What types of packages are available, and can I use only the provided packages?

• The main packages include TensorFlow, Keras, and PyTorch.
• Since the package management tool called Anaconda is used, you can use the Conda binary to make additional installations of other packages.
• Please refer to the Manage package section of the Jupyter Notebook user guide for information about installing and managing packages.

Q. Can I use Java or other languages besides Python?

• Although TensorFlow provides APIs such as Java and Go, their stability isn't guaranteed. We recommend that you use Python.

Q. What server types are available?

• The Ncloud TensorFlow Server types provided by NAVER Cloud Platform are divided into Standard, High Memory, CPU Intensive, and GPU. The features of each server type are as follows.

1. Standard: The standard server of the NAVER Cloud Platform service that can be used for various IT businesses. It provides well-balanced server specifications and boasts of high availability and stability.
2. High Memory: A high-memory server of more than 64 GB that is suitable for operating applications that are easily affected by memory performance. Up to 10 High Memory servers can be created, and the server number limit can be adjusted by making a request to Support.
3. CPU Intensive: The Intel Xeon processor is equipped with TensorFlow that is optimally built for AVX512 Vector operations. It is suitable for high-performance deep learning workload processing. This CPU type only supports the Ubuntu 16.04 OS at this time.
4. GPU: The server is equipped with GPUs with V100 and T4 specifications. It is suitable for environments that process data quickly. Up to 5 servers can be created, and the server number limit can be adjusted by making a request to Support.

Q. Ncloud TensorFlow Server has been created. Where do I write the code?

• The TensorFlow code can be written in Jupyter Notebook. Add Port 18888 to ACG and connect to [public IP address:18888] on the web browser.
• There are data processing, visualization, and TensorFlow example files in the "samples" folder, so please refer to them if you are new to data analysis.

Q. How do I use TensorBoard?

• After adding Port 18889 to ACG, connect to [Public IP address:18889] in a web browser .
• After adding Port 18889 to ACG, connect to [Public IP address:18889] in a web browser (Refer to 4. Configure Jupyter connection environment after creating Ncloud TensorFlow Server).
• Since the tensorboard directory is located under the default Jupyter path (/home/ncp/workspace), you can specify this directory as the log directory in the TensorFlow code.

Q. I can't connect to Jupyter Notebook.

• To use Jupyter Notebook, you must request a public IP address and assign it to the server, as well as add ort 18888 to ACG. (Please refer to 4. Configure Jupyter connection environment after creating Ncloud TensorFlow Server)
• If you are unable to make access, even after checking the public IP address and ACG, then the reason may be that the Jupyter process has stopped. Execute the "jup restart" command in the terminal and restart the process. (Please refer to "Manage Jupyter process")

Q. (GPU only) When executing the TensorFlow code, the "out of memory" message is displayed and execution fails.

• Method 1: When executing multiple code simultaneously, the OOM issue may occur if the code executed first does not release any GPU memory. Shut down the code executed first and try again.
• Method 2: Following the instructions from the official TensorFlow document, add the code for limiting GPU memory as shown below.
• Map TensorFlow so that the process can see nearly all the memory of every GPU by default (assume that it is included in CUDA_VISIBLE_DEVICES). This reduces memory fragmentation and allows for more efficient use of GPU memory resources, which are relatively more valuable, in devices. Use the tf.config.experimental.set_visible_devices method to adjust the GPUs that can be accessed from TensorFlow.

gpus = tf.config.experimental.list_physical_devices('GPU')
if gpus:
  # Limit so that TensorFlow can only use the first GPU
  try:
    tf.config.experimental.set_visible_devices(gpus[0], 'GPU')
  except RuntimeError as e:
    # Accessible devices must be set when starting the program
    print(e)

• In some cases, it should be set so that the process is assigned to only a part of the available memory, or that memory can be used for the amount required by the process. TensorFlow offers two methods for this to be possible.

The first method involves calling tf.config.experimental.set_memory_growth and allowing an increase in memory. This assigns the amount of GPU memory required for assignment in runtime. A small amount of memory is assigned at first, and the GPU memory area assigned to the TensorFlow process is expanded when the program is executed and requires more GPU memory. Memory is not released since this worsens memory fragmentation. Enter the following code before a tensor or operation to allow the memory increase of a specific GPU.

gpus = tf.config.experimental.list_physical_devices('GPU')
if gpus:
  try:
    tf.config.experimental.set_memory_growth(gpus[0], True)
  except RuntimeError as e:
    # Memory increase must be set when starting the program.
    print(e)

3. Create Ncloud TensorFlow Server VM

Preliminary work 1. Create VPC

Create a VPC in Console > VPC. At least one VPC is required. You can create multiple Ncloud TensorFlow Server in one VPC.

Preliminary work 2. Create subnet (public)

You can configure it to allow access to Ncloud TensorFlow Server from outside a specific VPC, without restricting access to only inside a specific VPC environment. After creating a public subnet in the created VPC, assign a public IP to the Ncloud TensorFlow Server.

Create a subnet by setting the property value of Internet gateway exclusiveness to Y (Public).

Step 1. Connect to console

Connect to Console, and then click the Server menu.

• Click the [Create server] button to create a server.

Step 2. Configure the Ncloud TensorFlow Server VM

Select a VPC, subnet, storage type, server type, and pricing plan, and then enter a server name.

• VPC: Select the VPC created earlier. (E.g., vpc-tensorflow)

• Subnet: Select Public Subnet. (E.g., vpc-sub-pub-tensorflow)

• Storage type: Select a storage type to use as the boot storage.

  • Select SSD for services that require high-performance I/O and HDD for general services. However, you can only use SSD as additional storage if the boot storage is SSD.

• Server type: Select the server type you want.

  • Some server specifications may not be supported, depending on the boot disk type.
  • Select one server type from among Standard, High Memory, or GPU, depending on your purpose.

• Select pricing plan: You can select either the monthly or hourly pricing plan.

• Server name: Enter a server name.

  • This name is used to identify servers, so it must be unique.

• Network Interface: Configure the network interface. You can create up to 3 per server.

• Public IP: If you want to automatically allocate a public IP while creating a VM, select the [Assign a new public IP] button.

  • If you want to create a public IP when creating a server, then the subnet type must be public subnet and the number of server must be 1.

• [Next]: Click the Next button after completing the server configuration.

Step 3. Configure the authentication key

If you have an existing authentication key, select "Use an existing authentication key." If you don't have one, create a new authentication key according to the following procedure.

• ① Select Create a new authentication key.

• ② Enter an authentication key name.

• ③ Click the [Create and save authentication key] button to save the authentication key file to your local PC.

  • You received a new authentication key.
  • After saving it, please keep the authentication key in a safe place on your PC.
  • The authentication key is used to obtain an initial admin password.

• ④ Click the [Next] button.

Step 4. Set the firewall

You can set a firewall by using an existing ACG.

• You must select an ACG and select one of the existing ACGs that contain "VPC default ACG" that is provided by default when a VPC is created.

  • You can select up to 5 ACGs of your own.
  • You must select an ACG to activate the [Next] button so that you can proceed with the setting.

• Select an ACG you want to configure, and then click the [Next] button.

Step 5. Final confirmation

Do the final confirmation of the setting.

• Final confirmation: Make sure that the server image, server, authentication key, and ACG are set properly.

• [Create server]: After final confirmation, click the button to create a server.

  • This may take several minutes or longer to be completed.

Step 6. Check in the server list

Check the created server in the list.

• Wait until the server is created, the package is installed, and the server status becomes Running.

4. Configure Jupyter connection environment after creating Ncloud TensorFlow Server

Request the use of public IP address

If you want to connect to Jupyter Notebook via Internet Browser after creating a Ncloud TensorFlow Server, then you must select Public Subnet, create a VM, and connect to Jupyter using a public IP address. (You will be charged additional costs for using public IP addresses.)

Go to VPC > Server > Public IP and create a public IP.

• ① Click the [Request public IP] button to get a public IP assigned.

• ② From Select target server, select a Ncloud TensorFlow Server to assign the public IP address to.

ACG settings

Add an SSH "TCP/22" rule for terminal connections and a Jupyter Notebook (TCP/18888) rule to ACG, as well as TensorBoard (TCP/18889) if necessary.

If you have already added the rules when creating a server, then you do not need to set ACG in this step.

• ① Select ACG from the left menu.

• ② Select the ACG that you chose when creating the server, and then click [Set ACG] at the top.

• ③ In the ACG rule setting, enter a protocol, access source, and allowed port and create it.

  • Protocol: Select from TCP, UDP, and ICMP.
  • Access source: Enter an IP address or ACG name.
  • Allowed port: Specify a single port or a range of ports.
  • For a terminal connection, the rule for SSH "TCP/22" must be set in ACG of a server to be connected. The "TCP/18888" rule must be additionally set to use Jupyter Notebook, and "TCP/18889" rule must be set to use TensorBoard.
  • To add the rules, list a protocol, access source, and allowed port, and then click [Add].

• ④ When you're finished with the settings, click [Apply].

• ⑤ You can see the detailed information in the Details tab and the rules you have set up in the View rules tab.

Confirm the operation of Jupyter Notebook

You can access Jupyter Notebook after configuring the Public IP and ACG.

• The initial password for Jupyter Notebook is the server name of VM. For safe use, it is recommended to use it after changing the password.

  • For instructions on how to change your Jupyter Notebook password, see Change Jupyter password.

• The address to access Jupyter Notebook is http://{created-publicIP}:{18888}.

  • For instance, when the created public IP is 101.101.220.51, the access URL is http://101.101.220.51:18888.

5. Manage Jupyter Notebook

You need to connect to the Ncloud TensorFlow Server VM via a terminal program (such as PuTTY) and change the Jupyter password.

You can easily change your password and manage the Jupyter process through the provided administration script.

Get Admin Password

You need to know the admin password to connect to the Ncloud TensorFlow Server VM via a terminal program (such as PuTTY).

• ① Select Server from the left menu.

• ② Select the server, and Get admin password from [Manage servers].

• ③ Click [Drag files here or click here] to attach the authentication key file (.pem) registered in your PC when creating a server.

• ④ After attaching it, click the [Confirm password] button to get a password to use when first logging in to the server via a terminal program.

  • It is recommended to change the password to one that the admin can remember after connecting to the server.

How to connect to VM via a terminal

You can access the Ncloud TensorFlow Server VM with the admin password.

• ① Enter the public IP assigned to the Ncloud TensorFlow Server.

• ② Enter the admin password for the Ncloud TensorFlow Server VM.

• ① Run a terminal program.

• ② Request VM access by using the ssh command, enter the user information and password, and then log in.

  • Enter the command ssh root@${public-IP} for VM access. (For instance, if the public IP is 101.101.220.51, ssh root@101.101.220.51)
  • login as: root
  • Password: Enter the admin password confirmed above.

• ③ It is recommended to change the password of the root account to one you can easily remember after logging in.

  • Enter the "passwd root" command to enter a new password.

Change Jupyter password

The initial password used for Jupyter Notebook is set to the server name, so please change the password through the script provided to prevent unauthorized access.

• ① In the terminal, enter "jup passwd" to change the password.
  • Confirm the initial password: Same as the "server name" from Console > Server > Server List
  • A password must contain English letters, numbers, and special characters, and be at least 8 characters.
  • If you execute the script to change a password, then the Jupyter processes will be automatically restarted.

Manage Jupyter process

If you have changed configuration files and are required to restart the Jupyter process, you can stop or restart the process via the provided script.

• ① Stop Jupyter process: $ jup stop

• ② Start Jupyter process: $ jup start

• ③ Stop and start Jupyter process: $ jup restart

Manage TensorBoard process

If you start the TensorBoard process for the first time after creating a server or are required to restart the process, then you can stop or restart the process via the provided script.

• ① Stop TensorBoard process: $ jup tb-stop

• ② Start TensorBoard process: $ jup tb-start