Notebooks

Available in VPC

This guide describes the Notebooks interface. In Notebooks, you can create and delete Notebooks, view the list of active Notebooks, and monitor training processes. You can also view details and recent event history of active Notebooks.

Notebooks

The Notebooks interface includes the following components:

View Notebook list

The list of your Notebooks includes the following information:

• Notebook name: Notebook name set at creation.
• Creation date: Initial creation date and time.
• Status: Notebook status.
• Type: Runtime application selected when creating a Notebook.
  • JupyterLab
  • VisualStudio Code
• Image: Image selected when creating the Notebook.
• Actions: Access or change the status of the created Notebook.
  • CONNECT: Open the Notebook execution environment in a new window.
  • STOP: Stop using the Notebook.
  • START: Start using the Notebook.

Create a Notebook

To create a new Notebook:

1. Click [Add].
2. In the Create Notebook interface, enter a Notebook name.
   • Enter 3-30 characters using lowercase letters (a-z), numbers (0-9), and hyphens (-).
   • It must start with an English letter and end with an English letter or a number.
   • livy is a restricted keyword.
   • Must be unique across Tensorboards and existing Notebooks.
3. Select the program to run the Notebook.
   • JupyterLab
   • VisualStudio Code
4. Select an image. The images provided by ML expert Platform are as follows:
   • kubeflow-jupyter-*: Jupyter-lab based on the official Kubeflow image
   • kubeflow-codeserver-*: Codeserver based on the official Kubeflow image

5. Configure the CPU and memory specifications for the Notebook.

6. Configure the GPU instance specifications for the Notebook.
7. Configure the Workspace Volume. You can create a new volume for the Workspace Volume or use an existing one.

8. Configure the Data Volume. You can mount a separate Data Volume to the Notebook to import data or save task history. If a separate Data Volume is not required, you do not need to set one. You can create a new PVC for the Data Volume or use an existing PVC.

9. Select Configurations in Advanced Settings.

10. If needed, configure Miscellaneous Settings in Advanced Settings.
11. Click [Launch].

Delete a Notebook

To delete an active Notebook:

1. Select the Notebook to delete, then click [Delete].
2. When the Delete Notebook popup appears, enter the name of the Notebook to delete, then click [Delete].
3. Confirm that the selected Notebook has been removed from the Notebook list.

View Notebook details

You can view details for the selected Notebook. The information is organized into tabs.

Overview

The selected Notebook includes the following information:

• Volumes: Volumes mounted to the Notebook
• Shared memory enabled:
• Notebook creator: User who created the Notebook
• Configurations: Configuration information set when creating the Notebook
• Type: Program used to run the Notebook
• Minimum CPU: Minimum CPU value set at creation
• Maximum CPU: Maximum CPU value set at creation
• Minimum memory: Minimum memory value set at creation
• Maximum memory: Maximum memory value set at creation
• Image: Image selected when creating the Notebook.
• Environment: Notebook execution environment

Logs

You can view recent logs for the selected Notebook.

Events

You can view recent event history for the selected Notebook.

• Type: The type of event that occurred.
• Reason: The name of the event that occurred.
• Created At: Date and time when the event occurred.
• Message: Description of the event that occurred.

YAML

This tab displays the configuration of the selected Notebook in YAML format.

Create a custom image to use with Notebook

If you already have the image you want to use, you can skip the creation and upload process.

Create and upload an image

The code below is an example Dockerfile for building a custom image.
In particular, sudo may cause issues in the Notebook runtime environment when used with certain root permissions. Install it only when necessary.

FROM nvcr.io/nvidia/pytorch:24.05-py3

# Install kubectl so that kubectl commands can be used
RUN apt update && apt install curl -y && curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" && \
    install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl && \
    rm -f kubectl

# Install jupyterlab
RUN pip install notebook==6.0.2 jupyterlab==1.2.4 jupyterlab-server==1.0.6

# Install and configure sudo: Uncomment to install only if required
#RUN apt install -y sudo && \
#    echo "irteam ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/irteam && \
#    chmod 0440 /etc/sudoers.d/irteam

# Create a Notebook user
ENV NB_USER irteam
ENV NB_UID 500
RUN useradd -M -s /bin/bash -U -u ${NB_UID} ${NB_USER} \
    && mkdir -p /home/$NB_USER \
    && chown -R ${NB_USER}:${NB_USER} /home/$NB_USER

USER ${NB_USER}
WORKDIR /home/$NB_USER

# Define the Notebook startup script
CMD ["sh", "-c", "jupyter lab --notebook-dir=/home/$NB_USER --ip=0.0.0.0 --no-browser --allow-root --port=8888 --LabApp.token='' --LabApp.password='' --LabApp.allow_origin='*' --LabApp.base_url=${NB_PREFIX}"]

After creating the file above, you can build a new Docker image by running the following command. Specify the REPOSITORY and TAG values you want to use.
Also, have a Docker registry service ready to manage the image.

$ docker build -t REPOSITORY:TAG -f Dockerfile .

Once the image is successfully built, upload it to the Docker registry.

$ docker push REPOSITORY:TAG

Configure a custom image

Once your image is ready, expand Advanced Settings under the Image section during the Notebook creation step, and then select the Custom Image checkbox. In the Custom Image field, enter the image information in the REPOSITORY:TAG format. The Notebook then runs using the custom image.

Private registry

This section describes how to use a private registry that requires authentication.

If you run a Notebook using a custom image uploaded to a private registry, you may encounter the following error message:

ImagePullBackOff: Back-off pulling image "reg.image.com/xxx/yyy:z": ErrImagePull: failed to pull and unpack image "reg.image.com/xxx/yyy:z": failed to resolve reference "reg.image.com/xxx/yyy:z": unexpected status from HEAD request to reg.image.com/xxx/yyy:z: 403 Forbidden

This error occurs because the imagePullSecret required to access the private registry is not registered.
You can resolve this by using one of the following two methods.

First, create a Secret by running the command below. Enter the name of the Secret to create, registry address, username, and password.

$ kubectl create secret docker-registry {image-pull-secret-name} \
--docker-server='{my-registry}' \
--docker-username='{my-username}' \
--docker-password='****'

1. Use PodDefault

For this method, you define a PodDefault YAML file and deploy it as follows:

apiVersion: kubeflow.org/v1alpha1
kind: PodDefault
metadata:
  name: add-image-secret
  namespace:  p-{projectName}
spec:
  selector:
    matchLabels:
      add-image-secret: "true"
  desc: "add imagePullSecrets to pod"
  imagePullSecrets:
    - name: "{image-pull-secret-name}"

$ kubectl apply -f {yaml file name}

Once the PodDefault is successfully deployed, expand Advanced Settings during the Notebook creation step and select add imagePullSecrets to pod from the Configurations section. Then, proceed to create the Notebook.

2. Add imagePullSecret to the Notebook spec

For this method, you manually modify the specification of a Notebook that failed to start and add imagePullSecrets.

Modify the Notebook spec as shown below. In the imagePullSecrets field, enter the name of the Secret you created earlier, and then save your changes.
The Notebook changes to the Running state shortly and becomes available.

$ kubectl edit notebook {notebook-name}

apiVersion: kubeflow.org/v1
kind: Notebook
...
spec:
  template:
    spec:
      imagePullSecrets: # Add this
        - name: {image-pull-secret-name} # Add this
      containers:
        ...