Pipelines

Available in VPC.

A pipeline represents a workflow composed of components and the relationships between them, displayed as a graph. A pipeline includes the input parameters required to run the pipeline, as well as the input/output information for each component. A component is similar to a function. It takes input values, performs execution, and produces output values.

After creating your pipeline YAML file, you can upload and run it from the Pipelines interface or by using the Pipeline SDK.

An ML workflow typically includes tasks such as data preprocessing, data transformation, and model training. A component represents the code required to execute one step of such a workflow and is typically executed as a container.

mlxp_console_pipelines03_ko

Component	Description
Pipeline type	Create a new pipeline: Use this option to create a pipeline for the first time. Create a new pipeline version under an existing pepeline: Use this option to create a new version of an existing pipeline.
Pipeline Name	Enter the name of the pipeline. Because it must follow the Kubernetes naming pattern, only lowercase letters, numbers, -, and . are allowed.
Pipeline Description	Enter a description of the pipeline.
Pipeline Source	Upload file: Upload a pipeline YAML file that was compiled and generated by the SDK in your local environment. Import by URL: Download the pipeline YAML from a specific URL and create the pipeline from it.
Source Code	Enter information describing where and how the pipeline was created, such as the source location or code path. This typically refers to a Git repository URL or an internal code path within a Jupyter Notebook.

Caution

Uploading and normal use of pipeline YAML files generated with SDK 1.x are not supported.
ML expert Platform cannot use the default MinIO artifact store provided by Kubeflow Pipelines. You must use an artifact store that is isolated per namespace.

mlxp_console_pipelines01_ko

Component	Description
Pipeline name	The pipeline name.
Version name	The name of the pipeline version.
Description	A description of the pipeline.
Uploaded on	The time when the pipeline was uploaded.

mlxp_console_pipelines02_ko

Component	Description
Create Run	Configure and start a new run.
Upload version	Configure and upload a new version of the pipeline.
Create experiment	Configure and create a new experiment.
Graph tab	View the execution flow of the pipeline steps.
Pipeline Spec tab	View the contents of the pipeline specification YAML file.