Breaking the Monolith: Moving to Modular Kubeflow Pipelines
Stop running monolithic ML scripts. Learn how to use Kubeflow Pipelines' Lightweight Python Components to build modular, cached, and scalable ML workflows.
ReadMeFeed / Community knowledge
Real questions. Useful conversations. Find the people who know your stack.
Stop running monolithic ML scripts. Learn how to use Kubeflow Pipelines' Lightweight Python Components to build modular, cached, and scalable ML workflows.
Use environment variables to authenticate the Kaggle CLI, verify access with a dataset list command, and download files in CI without exposing kaggle.json.
Stop the "it worked yesterday" problem in ML. Learn how to use Kaggle's immutable dataset versioning and CLI to pin data snapshots and prevent silent data drift in your pipelines.
Kubeflow Pipelines can skip expensive steps when inputs and images match, but caching behaves differently across SDK versions and can hide stale results.
Stop relying on implicit Keras exports. Learn how to use TensorFlow SavedModel signatures to create a stable contract between your training pipeline and production inference.
A practical pattern for wrapping ML scripts as typed KFP v2 components with pinned images, plus local compile checks and cluster verification steps.
Learn how to build modular, containerized ML workflows using the Kubeflow Pipelines SDK, focusing on component creation, DAG orchestration, and artifact management.
A concise architecture note for serving TensorFlow models with SavedModel and TensorFlow Serving: minimal design, trust boundaries, operational checks, failure modes and conditions that trigger a redesign.
Data scientists often copy‑paste preprocessing code, causing drift. This guide shows how to turn a CSV scaler into a reusable Kubeflow Pipeline component, compile it to Argo YAML, and run it on a cluster, ensuring reproducibility and CI/CD integration.