PVC vs object store artifact storage for Kubeflow Pipelines with limited ops overhead
0 reputation · 26 Oct 2024, 01:50 UTC
Kubeflow is an ML platform for pipelines, notebooks, Katib and model serving on Kubernetes. Zero-downtime migration primitives for general applications are outside its scope.
The design question is for Kubeflow Pipelines artifact and metadata persistence in a small single-cluster setup with limited operational overhead. The documented approaches are per-namespace PersistentVolumeClaim storage for simplicity and cost, versus external object storage such as S3 or MinIO for durability and portability.
PVC storage reduces external dependencies but ties runs to the cluster and node, with manual backup and restore. Object store decouples artifacts from cluster lifecycle and enables sharing across namespaces and clusters, but adds credential management, network egress and operational overhead.
Artifact storage is typically a cluster-level decision that affects all namespaces, and changing it later requires reconfiguration with no built-in migration tooling. Behavior is version-sensitive across SDK and Pipelines releases.
Which storage mode preserves pipeline run artifacts after pod deletion in a single namespace under current Kubeflow Pipelines configuration? What operational changes are required to switch artifact storage later without built-in migration tooling? How do SDK v1 and v2 component models differ in artifact handling for the two storage options?