{ "schema": 1, "slug": "mlops", "title": "MLOps", "full_name": "MLOps: Machine Learning Operations", "category": "Data & AI", "tagline": "Take predictive ML models from notebook to reliable, monitored, governed production on Kubernetes and cloud.", "summary": "A hands-on 2026 roadmap for running predictive and deep-learning models in production. You will version data and features, track experiments and models in MLflow, automate continuous training, serve models on Kubernetes with KServe, and monitor drift, fairness and cost so models stay accurate long after launch.", "description": "MLOps applies DevOps, data engineering and software engineering discipline to the machine learning lifecycle: data and feature management, reproducible training, experiment tracking, a model registry, automated pipelines, deployment and monitoring. The goal is easy to state and hard to achieve: get models into production quickly, keep them accurate as the world changes, and be able to explain, reproduce and audit every model you ship.\n\nIn 2026 MLOps is mainstream infrastructure. Fraud scoring, forecasting, recommendations, pricing, credit risk and computer vision still deliver most of the measurable value from AI, and regulation such as the EU AI Act and frameworks such as the NIST AI RMF expect documented, monitored and governed models. The tooling has also consolidated around open standards: MLflow 3 for tracking and registry, Kubeflow and Airflow 3 for pipelines, KServe (a CNCF incubating project since 2025) for serving, and Kubernetes Dynamic Resource Allocation (GA in 1.34) for GPUs.\n\nThis path focuses on the classical and predictive ML lifecycle (tabular, time-series, vision and classic NLP models) operated on production infrastructure. If your main goal is running LLM, RAG or agent applications, continue with the LLMOps path, which builds on this foundation. MLOps also overlaps with DataOps (pipelines and data quality), Platform Engineering (self-service ML platforms) and SRE (SLOs for inference services).", "icon": "model_training", "accent": "#8B5CF6", "level": "Beginner → Advanced", "duration_weeks": 30, "weekly_hours": 10, "audience": [ "DevOps, cloud and platform engineers asked to deploy and run models built by data science teams", "Data scientists who want their models to reach production and stay healthy there", "Software and data engineers moving into ML engineering roles", "SREs and engineering managers responsible for the reliability, cost and governance of ML systems" ], "prerequisites": [ "Comfortable Python programming: functions, classes, virtual environments and package installs", "Working knowledge of Git and the Linux command line", "Basic statistics and an intuition for supervised learning (train/test split, overfitting, common metrics)", "Familiarity with Docker; Kubernetes basics help but are introduced where needed" ], "outcomes": [ "Build reproducible training workflows with versioned data, validated inputs and experiments tracked in MLflow", "Design a Feast feature pipeline that removes training/serving skew with point-in-time correct joins", "Automate continuous training with Kubeflow Pipelines or Airflow and gate model promotion with CI/CD checks", "Deploy models to Kubernetes with KServe using canary rollouts, autoscaling and the Open Inference Protocol", "Detect data and prediction drift with Evidently and wire alerts and retraining triggers into Prometheus and Grafana", "Schedule and share GPUs on Kubernetes with the NVIDIA GPU Operator, Dynamic Resource Allocation and Kueue", "Produce model cards, lineage and signed model artifacts that stand up to NIST AI RMF and EU AI Act style reviews", "Assess a team's MLOps maturity and plan a realistic path from manual notebooks to automated CI/CD for ML" ], "trends": [ "MLflow 3 made tracking and the registry GenAI-aware (LoggedModel, tracing, prompt registry), so one open platform now spans classical models and LLM applications.", "Kubernetes became the default ML substrate: KServe joined the CNCF as an incubating project in 2025, Dynamic Resource Allocation went GA in Kubernetes 1.34, and CNCF launched a Kubernetes AI Conformance program.", "Distributed training got simpler with Kubeflow Trainer v2 and the unified Kubeflow SDK, while Ray joined the PyTorch Foundation in October 2025.", "Cloud ML platforms and exams were reshaped in 2026: Vertex AI became Gemini Enterprise Agent Platform, Microsoft retired DP-100 in favour of the AI-300 MLOps Engineer exam, and AWS began rolling out MLA-C02.", "Model supply-chain security is now expected: signed model artifacts (OpenSSF model signing with Sigstore), CycloneDX ML-BOMs and safetensors instead of pickle files.", "The EU AI Act Digital Omnibus deal pushed stand-alone high-risk obligations to December 2027, but model documentation, monitoring and human oversight are already being built into ML platforms." ], "roles": [ { "title": "MLOps Engineer", "description": "Builds and runs the pipelines, registries, serving infrastructure and monitoring that move models from experiment to production, and keeps them reproducible and observable." }, { "title": "Machine Learning Engineer", "description": "Turns prototypes into production-grade training and inference code, optimises models for latency and cost, and owns model quality after launch." }, { "title": "ML Platform Engineer", "description": "Designs the internal ML platform on Kubernetes and cloud: GPU scheduling, Kubeflow and KServe, feature stores and self-service golden paths for data science teams." }, { "title": "Production-focused Data Scientist", "description": "Develops models with experiment tracking, tests and model cards from day one so they can be promoted through automated pipelines without rewrites." }, { "title": "AI Governance / Model Risk Engineer", "description": "Implements model inventory, lineage, fairness and explainability checks and the audit evidence that regulators and internal risk teams require." } ], "stages": [ { "title": "Foundations: Python, ML and the ML Lifecycle", "level": "Beginner", "duration_weeks": 4, "description": "Build the engineering and ML fundamentals every MLOps practitioner needs, and learn how the ML lifecycle differs from ordinary software delivery.", "milestone": "You can train, evaluate and package a scikit-learn model as a tested, containerised Python project and explain MLOps maturity levels 0 to 2.", "topics": [ { "title": "Python Engineering for ML", "description": "Write ML code as maintainable software: packages instead of notebooks, locked dependencies, type hints and unit tests. Every later stage (pipelines, serving, CI) assumes your training code is importable, testable and reproducible.", "est_hours": 12, "optional": false, "concepts": [ "project layout and pyproject.toml", "dependency locking with uv", "virtual environments", "type hints and linting", "unit tests with pytest", "configuration management", "structured logging" ], "tools": [ "Python", "pytest" ], "resources": [ { "title": "The Python Tutorial", "url": "https://docs.python.org/3/tutorial/", "type": "docs", "free": true }, { "title": "uv Documentation", "url": "https://docs.astral.sh/uv/", "type": "docs", "free": true }, { "title": "pytest Documentation", "url": "https://docs.pytest.org/", "type": "docs", "free": true } ] }, { "title": "Machine Learning Essentials for Operators", "description": "Learn enough ML to reason about what you operate: supervised learning, feature engineering, evaluation metrics, data leakage and the bias/variance trade-off. Engineers who understand metrics and failure modes design far better promotion gates and monitors.", "est_hours": 14, "optional": false, "concepts": [ "supervised vs unsupervised learning", "train/validation/test splits and cross-validation", "precision, recall, ROC-AUC and RMSE", "data leakage", "overfitting and regularisation", "scikit-learn pipelines", "PyTorch training loop basics" ], "tools": [ "scikit-learn", "PyTorch" ], "resources": [ { "title": "Google Machine Learning Crash Course", "url": "https://developers.google.com/machine-learning/crash-course", "type": "course", "free": true }, { "title": "scikit-learn User Guide", "url": "https://scikit-learn.org/stable/user_guide.html", "type": "docs", "free": true }, { "title": "PyTorch Tutorials", "url": "https://docs.pytorch.org/tutorials/", "type": "docs", "free": true } ] }, { "title": "Linux, Git and Containers for ML", "description": "ML workloads run as containers on Linux hosts, often with GPUs. Learn to build slim, reproducible images for training and inference, keep large data out of Git, and debug processes and resource limits on Linux.", "est_hours": 8, "optional": false, "concepts": [ "Dockerfiles for Python ML", "multi-stage builds and layer caching", "CUDA base images", "image size and security", "Git workflows and .gitignore for data", "Linux processes and resource limits" ], "tools": [ "Linux", "Git", "Docker" ], "resources": [ { "title": "Docker: Get Started", "url": "https://docs.docker.com/get-started/", "type": "docs", "free": true }, { "title": "Pro Git (free book)", "url": "https://git-scm.com/book/en/v2", "type": "book", "free": true }, { "title": "Docker Training and Certification Course — DevOpsSchool", "url": "https://www.devopsschool.com/courses/docker/", "type": "course", "free": false } ] }, { "title": "The ML Lifecycle and MLOps Maturity", "description": "Understand why ML systems accumulate hidden technical debt and how teams progress from manual notebooks (level 0) to automated pipelines (level 1) and full CI/CD for ML (level 2). This mental model frames every tool you learn next.", "est_hours": 6, "optional": false, "concepts": [ "ML lifecycle stages", "hidden technical debt in ML systems", "MLOps levels 0, 1 and 2", "continuous training", "roles: data scientist, ML engineer, platform team", "maturity assessment" ], "tools": [], "resources": [ { "title": "MLOps: Continuous delivery and automation pipelines in machine learning (Google Cloud)", "url": "https://docs.cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning", "type": "article", "free": true }, { "title": "Machine Learning Operations Maturity Model (Microsoft)", "url": "https://learn.microsoft.com/en-us/azure/architecture/ai-ml/guide/mlops-maturity-model", "type": "article", "free": true }, { "title": "MLOps Training and Certification Course — DevOpsSchool", "url": "https://www.devopsschool.com/courses/mlops/", "type": "course", "free": false } ] } ] }, { "title": "Data and Feature Management", "level": "Beginner", "duration_weeks": 4, "description": "Treat data as a versioned, validated product: version datasets, enforce data contracts, and serve consistent features to training and inference.", "milestone": "You can reproduce any training dataset by version, block bad data with automated checks, and serve the same features offline and online from Feast.", "topics": [ { "title": "Data Versioning and Lineage", "description": "A model is only reproducible if you can recreate the exact data it was trained on. Learn to version datasets and pipelines with DVC for project-scale work and lakeFS for data-lake scale (lakeFS took over stewardship of DVC in late 2025), and record lineage from raw data to model.", "est_hours": 8, "optional": false, "concepts": [ "dataset versioning", "DVC pipelines and remotes", "Git-like branches over object storage", "table formats (Delta Lake, Apache Iceberg)", "lineage and provenance", "reproducing a past training run" ], "tools": [ "DVC", "lakeFS" ], "resources": [ { "title": "DVC Documentation", "url": "https://doc.dvc.org/", "type": "docs", "free": true }, { "title": "lakeFS Documentation", "url": "https://docs.lakefs.io/", "type": "docs", "free": true } ] }, { "title": "Data Validation and Quality", "description": "Most production ML incidents start as data problems: schema changes, missing values or silently shifted distributions. Add automated data contracts and validation suites that fail pipelines before bad data reaches training or inference.", "est_hours": 8, "optional": false, "concepts": [ "schema and data contracts", "expectation suites", "null, range and uniqueness checks", "distribution checks", "validation as a pipeline gate", "data quality reporting" ], "tools": [ "Great Expectations" ], "resources": [ { "title": "Great Expectations (GX Core) Documentation", "url": "https://docs.greatexpectations.io/docs/home/", "type": "docs", "free": true }, { "title": "Pandera Documentation", "url": "https://pandera.readthedocs.io/", "type": "docs", "free": true } ] }, { "title": "Feature Engineering and Feature Stores", "description": "Feature stores give training and serving one definition of each feature, which removes training/serving skew. Learn Feast: feature views, offline and online stores, materialisation and point-in-time correct joins for leakage-free training sets.", "est_hours": 14, "optional": false, "concepts": [ "feature views and entities", "offline vs online stores", "point-in-time correct joins", "materialisation jobs", "training/serving skew", "feature freshness and TTL", "feature reuse and discovery" ], "tools": [ "Feast" ], "resources": [ { "title": "Feast Documentation", "url": "https://docs.feast.dev/", "type": "docs", "free": true }, { "title": "Feast Quickstart", "url": "https://docs.feast.dev/getting-started/quickstart", "type": "lab", "free": true } ] }, { "title": "Data Processing at Scale", "description": "When data outgrows a single machine, feature pipelines move to distributed engines. Learn the basics of Apache Spark and Ray Data and how lakehouse platforms such as Databricks combine them with governance and MLflow.", "est_hours": 10, "optional": true, "concepts": [ "DataFrames and lazy evaluation", "partitioning and shuffles", "batch feature pipelines", "Ray Data for ML preprocessing", "lakehouse architecture", "cost-aware cluster sizing" ], "tools": [ "Apache Spark", "Ray", "Databricks" ], "resources": [ { "title": "Apache Spark Documentation", "url": "https://spark.apache.org/docs/latest/", "type": "docs", "free": true }, { "title": "Ray Data Documentation", "url": "https://docs.ray.io/en/latest/data/data.html", "type": "docs", "free": true }, { "title": "Databricks Training and Certification Course — DevOpsSchool", "url": "https://www.devopsschool.com/courses/databricks/", "type": "course", "free": false } ] } ] }, { "title": "Experiment Tracking, Training and Model Registry", "level": "Intermediate", "duration_weeks": 4, "description": "Make every experiment traceable and every model a governed, versioned artifact, then scale training with tuning and distributed compute.", "milestone": "You can compare runs in MLflow, register a champion model with signature and lineage, and run a distributed hyperparameter search.", "topics": [ { "title": "Experiment Tracking with MLflow", "description": "Track parameters, metrics, artifacts and code versions for every run so results can be compared and reproduced. MLflow is the de facto open standard, and MLflow 3 adds the LoggedModel entity that links models to the runs, data and evaluations behind them.", "est_hours": 12, "optional": false, "concepts": [ "runs, experiments and autologging", "parameters, metrics and artifacts", "LoggedModel in MLflow 3", "tracking server and artifact store", "comparing runs", "dataset tracking" ], "tools": [ "MLflow" ], "resources": [ { "title": "MLflow for Traditional ML (official docs)", "url": "https://mlflow.org/docs/latest/ml/", "type": "docs", "free": true }, { "title": "MLflow Tracking", "url": "https://mlflow.org/docs/latest/ml/tracking/", "type": "docs", "free": true } ] }, { "title": "Model Registry, Packaging and Model Cards", "description": "A registry turns trained models into versioned, reviewable release candidates. Learn registry aliases (champion and challenger), model signatures, portable formats such as ONNX, and model cards that document intended use, data and limitations.", "est_hours": 10, "optional": false, "concepts": [ "registered models and versions", "aliases (champion, challenger)", "model signatures and input examples", "ONNX export", "model cards", "approval and promotion workflow" ], "tools": [ "MLflow", "ONNX" ], "resources": [ { "title": "MLflow Model Registry", "url": "https://mlflow.org/docs/latest/ml/model-registry/", "type": "docs", "free": true }, { "title": "ONNX Introduction", "url": "https://onnx.ai/onnx/intro/", "type": "docs", "free": true }, { "title": "Model Cards (Hugging Face Hub docs)", "url": "https://huggingface.co/docs/hub/model-cards", "type": "docs", "free": true } ] }, { "title": "Hyperparameter Tuning and Distributed Training", "description": "Scale training beyond a laptop: run efficient hyperparameter searches with Optuna, and distribute training with Ray Train or Kubeflow Trainer v2, which uses TrainJob and TrainingRuntime resources to run PyTorch jobs on Kubernetes.", "est_hours": 10, "optional": false, "concepts": [ "search spaces and pruning", "Bayesian optimisation", "data vs model parallelism", "Ray Train", "Kubeflow Trainer TrainJob", "checkpointing and fault tolerance" ], "tools": [ "Optuna", "Ray", "Kubeflow" ], "resources": [ { "title": "Optuna Documentation", "url": "https://optuna.readthedocs.io/", "type": "docs", "free": true }, { "title": "Ray Train Documentation", "url": "https://docs.ray.io/en/latest/train/train.html", "type": "docs", "free": true }, { "title": "Kubeflow Trainer", "url": "https://www.kubeflow.org/docs/components/trainer/", "type": "docs", "free": true } ] }, { "title": "Testing ML Systems", "description": "ML needs tests beyond unit tests: data tests, model performance thresholds, invariance and slice tests, and infrastructure tests for training and serving. Use the ML Test Score rubric to judge production readiness.", "est_hours": 8, "optional": false, "concepts": [ "unit tests for feature code", "data tests", "model quality thresholds", "slice-based and invariance tests", "reproducibility checks", "ML Test Score rubric" ], "tools": [ "pytest" ], "resources": [ { "title": "The ML Test Score: A Rubric for ML Production Readiness (Google Research)", "url": "https://research.google/pubs/the-ml-test-score-a-rubric-for-ml-production-readiness-and-technical-debt-reduction/", "type": "paper", "free": true }, { "title": "Rules of Machine Learning (Google)", "url": "https://developers.google.com/machine-learning/guides/rules-of-ml", "type": "article", "free": true } ] } ] }, { "title": "Pipelines, CI/CD and Continuous Training", "level": "Intermediate", "duration_weeks": 5, "description": "Automate the path from new data or code to a validated, registered model with pipeline orchestrators, CI/CD and GitOps.", "milestone": "A code or data change automatically triggers a pipeline that validates data, trains, evaluates against the champion and registers a challenger only when it wins.", "topics": [ { "title": "ML Pipelines with Kubeflow Pipelines", "description": "Kubeflow Pipelines (KFP v2) packages each step as a containerised component and runs reproducible DAGs on Kubernetes, with caching, artifacts and lineage. It is also the pipeline format used by several managed cloud ML services.", "est_hours": 16, "optional": false, "concepts": [ "components and pipelines (KFP v2 SDK)", "artifacts and metadata", "step caching", "parameterised and scheduled runs", "conditional promotion steps", "running on kind or a managed cluster" ], "tools": [ "Kubeflow", "Kubernetes" ], "resources": [ { "title": "Kubeflow Pipelines Overview", "url": "https://www.kubeflow.org/docs/components/pipelines/", "type": "docs", "free": true }, { "title": "Kubeflow Pipelines Getting Started", "url": "https://www.kubeflow.org/docs/components/pipelines/getting-started/", "type": "lab", "free": true } ] }, { "title": "Workflow Orchestration with Airflow, Metaflow and ZenML", "description": "Not every team runs Kubeflow. Apache Airflow 3 adds DAG versioning and asset-aware scheduling, Metaflow offers a data-scientist-friendly Python API, and ZenML provides a portable pipeline layer over many backends. Learn when each fits.", "est_hours": 12, "optional": false, "concepts": [ "DAGs and tasks", "Airflow 3 assets and data-aware scheduling", "DAG versioning", "Metaflow flows and steps", "ZenML stacks", "choosing an orchestrator" ], "tools": [ "Apache Airflow", "Metaflow", "ZenML" ], "resources": [ { "title": "Apache Airflow Documentation", "url": "https://airflow.apache.org/docs/apache-airflow/stable/index.html", "type": "docs", "free": true }, { "title": "Metaflow Documentation", "url": "https://docs.metaflow.org/", "type": "docs", "free": true }, { "title": "ZenML Documentation", "url": "https://docs.zenml.io/", "type": "docs", "free": true } ] }, { "title": "CI/CD and Continuous Training for ML", "description": "Apply CI/CD to three artifacts at once: code, data and models. Build pipelines that lint and test code, validate data, retrain, compare against the production model and promote through registry aliases with human approval where risk demands it.", "est_hours": 14, "optional": false, "concepts": [ "CI for training code", "continuous training triggers", "champion/challenger evaluation gates", "model promotion via registry aliases", "reports on pull requests", "environment promotion (dev, staging, prod)" ], "tools": [ "GitHub Actions", "MLflow" ], "resources": [ { "title": "Continuous Delivery for Machine Learning (martinfowler.com)", "url": "https://martinfowler.com/articles/cd4ml.html", "type": "article", "free": true }, { "title": "GitHub Actions Documentation", "url": "https://docs.github.com/en/actions", "type": "docs", "free": true }, { "title": "CML: Continuous Machine Learning", "url": "https://cml.dev/", "type": "docs", "free": true } ] }, { "title": "GitOps and Infrastructure as Code for ML Platforms", "description": "Manage the ML platform itself as code: provision clusters and storage with Terraform, run Kubernetes-native workflows with Argo Workflows, and deploy serving configuration declaratively with Argo CD.", "est_hours": 8, "optional": true, "concepts": [ "Terraform modules for ML infrastructure", "Argo Workflows for batch jobs", "GitOps with Argo CD", "environment overlays", "secrets for data access", "drift detection of platform config" ], "tools": [ "Terraform", "Argo Workflows", "Argo CD" ], "resources": [ { "title": "Argo Workflows Documentation", "url": "https://argo-workflows.readthedocs.io/en/latest/", "type": "docs", "free": true }, { "title": "Argo CD Documentation", "url": "https://argo-cd.readthedocs.io/", "type": "docs", "free": true }, { "title": "Argo Training and Certification Course — DevOpsSchool", "url": "https://www.devopsschool.com/courses/argo/", "type": "course", "free": false } ] } ] }, { "title": "Model Serving and Deployment", "level": "Advanced", "duration_weeks": 5, "description": "Serve models reliably in batch, online and streaming modes, on Kubernetes with KServe and high-performance runtimes, and release new versions safely.", "milestone": "You can run a model behind a KServe InferenceService with autoscaling, observe it, and shift traffic to a new version with an automated canary and rollback.", "topics": [ { "title": "Serving Patterns and Model Packaging", "description": "Choose the right pattern for the use case: batch scoring, online REST or gRPC inference, or streaming. Package models as services with FastAPI or BentoML and standardise request formats with the Open Inference Protocol.", "est_hours": 10, "optional": false, "concepts": [ "batch vs online vs streaming inference", "REST and gRPC endpoints", "Open Inference Protocol (V2)", "model packaging with BentoML", "pre- and post-processing", "latency budgets" ], "tools": [ "FastAPI", "BentoML" ], "resources": [ { "title": "BentoML Documentation", "url": "https://docs.bentoml.com/", "type": "docs", "free": true }, { "title": "FastAPI Documentation", "url": "https://fastapi.tiangolo.com/", "type": "docs", "free": true }, { "title": "Open Inference Protocol (KServe V2)", "url": "https://kserve.github.io/website/docs/concepts/architecture/data-plane/v2-protocol", "type": "docs", "free": true } ] }, { "title": "Model Serving on Kubernetes with KServe", "description": "KServe is the CNCF standard for serving predictive and generative models on Kubernetes. Learn InferenceService resources, model storage, request-based autoscaling with Knative including scale to zero, and transformers for feature lookup.", "est_hours": 16, "optional": false, "concepts": [ "InferenceService resources", "serving runtimes", "storage URIs and model caching", "Knative autoscaling and scale to zero", "transformers and feature lookup", "raw deployment vs serverless mode" ], "tools": [ "KServe", "Knative", "Kubernetes" ], "resources": [ { "title": "KServe Documentation", "url": "https://kserve.github.io/website/docs/intro", "type": "docs", "free": true }, { "title": "KServe Quickstart", "url": "https://kserve.github.io/website/docs/getting-started/quickstart-guide", "type": "lab", "free": true }, { "title": "Knative Training and Certification Course — DevOpsSchool", "url": "https://www.devopsschool.com/courses/knative/", "type": "course", "free": false } ] }, { "title": "High-Performance Inference Runtimes", "description": "Squeeze latency and cost with optimised runtimes: NVIDIA Dynamo-Triton (formerly Triton Inference Server) for multi-framework GPU serving with dynamic batching, ONNX Runtime for portable CPU and GPU acceleration, and Ray Serve for Python-native model composition.", "est_hours": 12, "optional": false, "concepts": [ "dynamic batching", "concurrent model execution", "model ensembles and composition", "ONNX Runtime execution providers", "CPU vs GPU serving economics", "load testing inference" ], "tools": [ "NVIDIA Dynamo-Triton", "ONNX Runtime", "Ray" ], "resources": [ { "title": "NVIDIA Triton Inference Server (Dynamo-Triton) User Guide", "url": "https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/index.html", "type": "docs", "free": true }, { "title": "ONNX Runtime Documentation", "url": "https://onnxruntime.ai/docs/", "type": "docs", "free": true }, { "title": "Ray Serve Documentation", "url": "https://docs.ray.io/en/latest/serve/index.html", "type": "docs", "free": true } ] }, { "title": "Progressive Delivery and Online Experiments", "description": "Release models like software: shadow deployments, canaries with automated analysis, blue-green switches and A/B tests that measure business impact. Combine KServe traffic splitting, Argo Rollouts and service-mesh routing.", "est_hours": 12, "optional": false, "concepts": [ "shadow (mirrored) traffic", "canary releases and automated rollback", "blue-green deployments", "A/B tests and guardrail metrics", "multi-armed bandits", "traffic splitting with a service mesh" ], "tools": [ "Argo Rollouts", "Istio", "KServe" ], "resources": [ { "title": "KServe Canary Rollout Example", "url": "https://kserve.github.io/website/docs/model-serving/predictive-inference/rollout-strategies/canary-example", "type": "lab", "free": true }, { "title": "Argo Rollouts Documentation", "url": "https://argo-rollouts.readthedocs.io/", "type": "docs", "free": true }, { "title": "Istio Training and Certification Course — DevOpsSchool", "url": "https://www.devopsschool.com/courses/istio/", "type": "course", "free": false } ] } ] }, { "title": "Monitoring, Drift and Responsible AI", "level": "Advanced", "duration_weeks": 4, "description": "Keep models healthy after launch: detect drift and quality decay, observe inference services, close the feedback loop and govern models responsibly.", "milestone": "You run dashboards and alerts for service health and model quality, retrain automatically on drift, and publish fairness and explainability evidence for each model version.", "topics": [ { "title": "Model Monitoring and Drift Detection", "description": "Models decay silently as data and behaviour change. Monitor input data drift, prediction drift and, when labels arrive, real quality metrics. Evidently provides open-source reports, test suites and metrics for tabular, text and LLM data.", "est_hours": 14, "optional": false, "concepts": [ "data drift vs concept drift", "prediction drift", "statistical tests (PSI, KS, Wasserstein)", "delayed ground truth", "segment and slice monitoring", "drift thresholds and alert fatigue" ], "tools": [ "Evidently" ], "resources": [ { "title": "Evidently Documentation", "url": "https://docs.evidentlyai.com/", "type": "docs", "free": true }, { "title": "Data Distribution Shifts and Monitoring (Chip Huyen)", "url": "https://huyenchip.com/2022/02/07/data-distribution-shifts-and-monitoring.html", "type": "article", "free": true }, { "title": "What is Data Drift in ML (Evidently guide)", "url": "https://www.evidentlyai.com/ml-in-production/data-drift", "type": "article", "free": true } ] }, { "title": "Operational Observability for Inference Services", "description": "An accurate model that times out is still an outage. Instrument inference services with OpenTelemetry, collect latency, throughput, error and GPU metrics in Prometheus, and define SLOs and dashboards in Grafana.", "est_hours": 10, "optional": false, "concepts": [ "golden signals for inference", "latency percentiles and SLOs", "OpenTelemetry traces and metrics", "GPU utilisation metrics", "custom model metrics", "alert routing" ], "tools": [ "Prometheus", "Grafana", "OpenTelemetry" ], "resources": [ { "title": "Prometheus Overview", "url": "https://prometheus.io/docs/introduction/overview/", "type": "docs", "free": true }, { "title": "OpenTelemetry for Python", "url": "https://opentelemetry.io/docs/languages/python/", "type": "docs", "free": true }, { "title": "Prometheus Training and Certification Course — DevOpsSchool", "url": "https://www.devopsschool.com/courses/prometheus/", "type": "course", "free": false } ] }, { "title": "Feedback Loops and Continuous Training", "description": "Connect monitoring back to training: capture predictions and outcomes, build labelling pipelines, and trigger retraining on schedules, drift or performance drops while guarding against feedback loops that bias future data.", "est_hours": 6, "optional": false, "concepts": [ "prediction and outcome logging", "label collection pipelines", "retraining triggers", "degenerate feedback loops", "backtesting before promotion", "model retirement" ], "tools": [], "resources": [ { "title": "AWS Well-Architected Machine Learning Lens", "url": "https://docs.aws.amazon.com/wellarchitected/latest/machine-learning-lens/machine-learning-lens.html", "type": "docs", "free": true } ] }, { "title": "Responsible AI and Model Governance", "description": "Fairness, explainability and documentation are now operational requirements. Measure group fairness with Fairlearn, explain predictions with SHAP, maintain a model inventory, and map controls to the NIST AI RMF and the EU AI Act.", "est_hours": 10, "optional": false, "concepts": [ "fairness metrics and mitigation", "global and local explanations (SHAP)", "model inventory and lineage", "NIST AI RMF functions", "EU AI Act risk tiers and timeline", "human oversight and audit trails" ], "tools": [ "Fairlearn", "SHAP" ], "resources": [ { "title": "NIST AI Risk Management Framework", "url": "https://www.nist.gov/itl/ai-risk-management-framework", "type": "docs", "free": true }, { "title": "EU AI Act: Regulatory Framework for AI (European Commission)", "url": "https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai", "type": "docs", "free": true }, { "title": "Fairlearn Documentation", "url": "https://fairlearn.org/", "type": "docs", "free": true } ] } ] }, { "title": "ML Platform Engineering at Scale", "level": "Advanced", "duration_weeks": 4, "description": "Run ML as a shared platform: schedule and share GPUs on Kubernetes, use managed cloud ML services wisely, secure the model supply chain and control cost.", "milestone": "You can operate a multi-team ML platform with GPU quotas, signed model artifacts and cost visibility, and justify build-versus-buy choices against managed services.", "topics": [ { "title": "GPU Scheduling on Kubernetes", "description": "GPUs are the scarcest, most expensive resource in ML. Install drivers and device plugins with the NVIDIA GPU Operator, request devices precisely with Dynamic Resource Allocation (GA in Kubernetes 1.34), share GPUs with MIG or time-slicing, and queue batch jobs fairly with Kueue.", "est_hours": 14, "optional": false, "concepts": [ "NVIDIA GPU Operator", "device plugins vs Dynamic Resource Allocation", "MIG and time-slicing", "Kueue quotas, cohorts and preemption", "gang scheduling", "GPU utilisation and bin-packing" ], "tools": [ "Kubernetes", "NVIDIA GPU Operator", "Kueue" ], "resources": [ { "title": "Dynamic Resource Allocation (Kubernetes docs)", "url": "https://kubernetes.io/docs/concepts/resource-management/dynamic-resource-allocation/", "type": "docs", "free": true }, { "title": "NVIDIA GPU Operator Documentation", "url": "https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/index.html", "type": "docs", "free": true }, { "title": "Kueue Documentation", "url": "https://kueue.sigs.k8s.io/docs/", "type": "docs", "free": true } ] }, { "title": "Managed ML Platforms", "description": "Know the managed alternatives and when they beat self-hosting: Amazon SageMaker AI, Google's Gemini Enterprise Agent Platform (formerly Vertex AI), Azure Machine Learning and Databricks. Compare pipelines, registries, endpoints, monitoring and lock-in.", "est_hours": 10, "optional": false, "concepts": [ "managed training and endpoints", "managed pipelines and registries", "feature store offerings", "build vs buy", "multi-cloud and portability", "identity and network isolation" ], "tools": [ "Amazon SageMaker AI", "Gemini Enterprise Agent Platform", "Azure Machine Learning", "Databricks" ], "resources": [ { "title": "What is Amazon SageMaker AI?", "url": "https://docs.aws.amazon.com/sagemaker/latest/dg/whatis.html", "type": "docs", "free": true }, { "title": "Gemini Enterprise Agent Platform Documentation", "url": "https://docs.cloud.google.com/gemini-enterprise-agent-platform", "type": "docs", "free": true }, { "title": "Azure Machine Learning Documentation", "url": "https://learn.microsoft.com/en-us/azure/machine-learning/", "type": "docs", "free": true } ] }, { "title": "ML Supply Chain Security", "description": "Models are executable artifacts: pickle files can run arbitrary code and public models can be tampered with. Prefer safetensors, sign models with the OpenSSF model-signing tooling built on Sigstore, and describe models and datasets in CycloneDX ML-BOMs.", "est_hours": 8, "optional": false, "concepts": [ "pickle deserialisation risks", "safetensors", "model signing and verification", "ML-BOM (CycloneDX)", "provenance for training data", "scanning containers and dependencies" ], "tools": [ "Sigstore" ], "resources": [ { "title": "OpenSSF Model Signing (model-transparency)", "url": "https://github.com/sigstore/model-transparency", "type": "docs", "free": true }, { "title": "CycloneDX Machine Learning BOM", "url": "https://cyclonedx.org/capabilities/mlbom/", "type": "docs", "free": true }, { "title": "safetensors Documentation", "url": "https://huggingface.co/docs/safetensors/index", "type": "docs", "free": true } ] }, { "title": "Cost and Capacity Management for ML", "description": "Training and inference can dominate a cloud bill. Allocate GPU and cluster cost to teams and models with OpenCost, use spot capacity for fault-tolerant training, right-size endpoints and track unit costs such as cost per thousand predictions.", "est_hours": 8, "optional": true, "concepts": [ "cost allocation by namespace and label", "spot and preemptible training", "endpoint right-sizing and autoscaling", "unit economics per prediction", "idle GPU detection", "FinOps for AI" ], "tools": [ "OpenCost" ], "resources": [ { "title": "OpenCost Documentation", "url": "https://opencost.io/docs/", "type": "docs", "free": true }, { "title": "FinOps Training and Certification Course — DevOpsSchool", "url": "https://www.devopsschool.com/courses/finops/", "type": "course", "free": false } ] } ] } ], "projects": [ { "title": "Reproducible Churn Model", "description": "Package a scikit-learn churn model as a Python project with DVC-versioned data, pytest tests, MLflow tracking and a Docker image. Deliver a README that lets anyone reproduce the exact metrics with one command.", "difficulty": "Beginner", "skills": [ "Python", "DVC", "MLflow", "Docker", "pytest" ] }, { "title": "Feature Store-Backed Fraud Scoring API", "description": "Define Feast feature views over a transactions dataset with an offline store and a Redis online store, train with point-in-time correct features, and serve real-time scores with BentoML or FastAPI reading the same online features.", "difficulty": "Intermediate", "skills": [ "Feast", "feature engineering", "BentoML", "Redis", "training/serving skew" ] }, { "title": "Continuous Training Pipeline", "description": "Build a Kubeflow Pipelines (or Airflow 3) pipeline that validates data with Great Expectations, trains, evaluates against the current champion and registers a challenger in MLflow only if it wins. Trigger it from GitHub Actions on new data or code.", "difficulty": "Intermediate", "skills": [ "Kubeflow Pipelines", "Great Expectations", "MLflow Model Registry", "GitHub Actions" ] }, { "title": "Canary Model Rollout on KServe", "description": "Deploy two model versions as a KServe InferenceService with canary traffic splitting, Prometheus metrics and Grafana dashboards, and automated rollback when latency or error-rate SLOs are breached.", "difficulty": "Advanced", "skills": [ "KServe", "Knative", "Prometheus", "Grafana", "progressive delivery" ] }, { "title": "Drift Monitoring and Retraining Loop", "description": "Compute data and prediction drift on logged production traffic with Evidently, expose drift scores to Prometheus, alert in Grafana and trigger the retraining pipeline automatically. Document thresholds, fairness checks and limitations in a model card.", "difficulty": "Advanced", "skills": [ "Evidently", "Prometheus", "continuous training", "Fairlearn", "model cards" ] }, { "title": "Multi-Team GPU ML Platform Blueprint", "description": "Provision a Kubernetes cluster with Terraform, install the NVIDIA GPU Operator, Kueue and KServe, define team quotas and GPU sharing, sign model artifacts with Sigstore, and publish a golden-path template that takes a data scientist from notebook to served model.", "difficulty": "Advanced", "skills": [ "Terraform", "Kubernetes", "NVIDIA GPU Operator", "Kueue", "Sigstore", "platform engineering" ] } ], "certifications": [ { "name": "AWS Certified Machine Learning Engineer – Associate", "provider": "Amazon Web Services", "url": "https://aws.amazon.com/certification/certified-machine-learning-engineer-associate/", "level": "Associate", "note": "AWS's main ML engineering certification since the ML Specialty exam retired in March 2026, covering data preparation, training, deployment and monitoring with SageMaker AI; the refreshed MLA-C02 version rolls out from September 2026." }, { "name": "Google Cloud Professional Machine Learning Engineer", "provider": "Google Cloud", "url": "https://cloud.google.com/learn/certification/machine-learning-engineer", "level": "Professional", "note": "Covers architecting, automating, serving and monitoring ML solutions on Google Cloud, now aligned to Gemini Enterprise Agent Platform (formerly Vertex AI)." }, { "name": "Microsoft Certified: Machine Learning Operations Engineer Associate (AI-300)", "provider": "Microsoft", "url": "https://learn.microsoft.com/en-us/credentials/certifications/operationalizing-machine-learning-and-generative-ai-solutions/", "level": "Associate", "note": "Replaced the retired DP-100 in 2026 and targets MLOps directly: MLOps infrastructure, model lifecycle and GenAIOps on Azure Machine Learning and Microsoft Foundry with GitHub Actions and Bicep." }, { "name": "Databricks Certified Machine Learning Professional", "provider": "Databricks", "url": "https://www.databricks.com/learn/certification/machine-learning-professional", "level": "Professional", "note": "Nearly half of the exam is ML Ops on Databricks (MLflow, feature engineering, model serving and monitoring), a strong fit for lakehouse-based teams." }, { "name": "Certified Kubernetes Administrator (CKA)", "provider": "CNCF / Linux Foundation", "url": "https://training.linuxfoundation.org/certification/certified-kubernetes-administrator-cka/", "level": "Professional", "note": "Hands-on Kubernetes administration underpins Kubeflow, KServe and GPU scheduling, so it is valuable for ML platform engineers." }, { "name": "MLOps Certified Professional (MLOCP)", "provider": "DevOpsSchool", "url": "https://www.devopsschool.com/certification/mlops-certified-professional.html", "level": "Professional", "note": "Instructor-led programme across the end-to-end MLOps toolchain (MLflow, Kubernetes, KServe, Databricks, observability) with hands-on projects and a scenario-based exam." }, { "name": "MLOps Foundation Certified (MLOps-FC)", "provider": "DevOpsSchool", "url": "https://www.devopsschool.com/certification/mlops-foundation-certification.html", "level": "Foundation", "note": "Entry-level DevOpsSchool credential on the ML lifecycle, MLflow, Kubeflow, feature stores, serving and monitoring, matching the first half of this path." } ], "tools_catalog": [ { "name": "Amazon SageMaker AI", "category": "Machine Learning & MLOps", "website": "https://aws.amazon.com/sagemaker/ai/", "ds_course_url": "https://www.devopsschool.com/courses/aws/" }, { "name": "Apache Airflow", "category": "Data Engineering", "website": "https://airflow.apache.org/", "ds_course_url": "" }, { "name": "Apache Spark", "category": "Data Engineering", "website": "https://spark.apache.org/", "ds_course_url": "https://www.devopsschool.com/courses/spark/" }, { "name": "Argo CD", "category": "GitOps", "website": "https://argo-cd.readthedocs.io/", "ds_course_url": "https://www.devopsschool.com/courses/argo/" }, { "name": "Argo Rollouts", "category": "CI/CD & Release", "website": "https://argoproj.github.io/rollouts/", "ds_course_url": "https://www.devopsschool.com/courses/argo/" }, { "name": "Argo Workflows", "category": "CI/CD & Release", "website": "https://argoproj.github.io/workflows/", "ds_course_url": "https://www.devopsschool.com/courses/argo/" }, { "name": "Azure Machine Learning", "category": "Machine Learning & MLOps", "website": "https://azure.microsoft.com/en-us/products/machine-learning", "ds_course_url": "https://www.devopsschool.com/courses/azure/" }, { "name": "BentoML", "category": "Machine Learning & MLOps", "website": "https://www.bentoml.com/", "ds_course_url": "" }, { "name": "Databricks", "category": "Data Engineering", "website": "https://www.databricks.com/", "ds_course_url": "https://www.devopsschool.com/courses/databricks/" }, { "name": "Docker", "category": "Containers & Kubernetes", "website": "https://www.docker.com/", "ds_course_url": "https://www.devopsschool.com/courses/docker/" }, { "name": "DVC", "category": "Machine Learning & MLOps", "website": "https://dvc.org/", "ds_course_url": "" }, { "name": "Evidently", "category": "Machine Learning & MLOps", "website": "https://www.evidentlyai.com/", "ds_course_url": "" }, { "name": "Fairlearn", "category": "Machine Learning & MLOps", "website": "https://fairlearn.org/", "ds_course_url": "" }, { "name": "FastAPI", "category": "Foundations & Languages", "website": "https://fastapi.tiangolo.com/", "ds_course_url": "" }, { "name": "Feast", "category": "Machine Learning & MLOps", "website": "https://feast.dev/", "ds_course_url": "" }, { "name": "Gemini Enterprise Agent Platform", "category": "Machine Learning & MLOps", "website": "https://cloud.google.com/products/gemini-enterprise-agent-platform", "ds_course_url": "https://www.devopsschool.com/courses/googlecloud/" }, { "name": "Git", "category": "Foundations & Languages", "website": "https://git-scm.com/", "ds_course_url": "https://www.devopsschool.com/courses/git/" }, { "name": "GitHub Actions", "category": "CI/CD & Release", "website": "https://github.com/features/actions", "ds_course_url": "https://www.devopsschool.com/courses/github/" }, { "name": "Grafana", "category": "Observability", "website": "https://grafana.com/", "ds_course_url": "https://www.devopsschool.com/courses/grafana/" }, { "name": "Great Expectations", "category": "Data Engineering", "website": "https://greatexpectations.io/", "ds_course_url": "" }, { "name": "Istio", "category": "Networking & Network Automation", "website": "https://istio.io/", "ds_course_url": "https://www.devopsschool.com/courses/istio/" }, { "name": "Knative", "category": "Containers & Kubernetes", "website": "https://knative.dev/", "ds_course_url": "https://www.devopsschool.com/courses/knative/" }, { "name": "KServe", "category": "Machine Learning & MLOps", "website": "https://kserve.github.io/website/", "ds_course_url": "" }, { "name": "Kubeflow", "category": "Machine Learning & MLOps", "website": "https://www.kubeflow.org/", "ds_course_url": "" }, { "name": "Kubernetes", "category": "Containers & Kubernetes", "website": "https://kubernetes.io/", "ds_course_url": "https://www.devopsschool.com/courses/kubernetes/" }, { "name": "Kueue", "category": "Containers & Kubernetes", "website": "https://kueue.sigs.k8s.io/", "ds_course_url": "" }, { "name": "lakeFS", "category": "Data Engineering", "website": "https://lakefs.io/", "ds_course_url": "" }, { "name": "Linux", "category": "Foundations & Languages", "website": "https://www.kernel.org/", "ds_course_url": "https://www.devopsschool.com/courses/linux/" }, { "name": "Metaflow", "category": "Machine Learning & MLOps", "website": "https://metaflow.org/", "ds_course_url": "" }, { "name": "MLflow", "category": "Machine Learning & MLOps", "website": "https://mlflow.org/", "ds_course_url": "https://www.devopsschool.com/courses/mlops/" }, { "name": "NVIDIA Dynamo-Triton", "category": "Machine Learning & MLOps", "website": "https://developer.nvidia.com/dynamo-triton", "ds_course_url": "" }, { "name": "NVIDIA GPU Operator", "category": "Machine Learning & MLOps", "website": "https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/index.html", "ds_course_url": "" }, { "name": "ONNX", "category": "Machine Learning & MLOps", "website": "https://onnx.ai/", "ds_course_url": "" }, { "name": "ONNX Runtime", "category": "Machine Learning & MLOps", "website": "https://onnxruntime.ai/", "ds_course_url": "" }, { "name": "OpenCost", "category": "FinOps & Sustainability", "website": "https://opencost.io/", "ds_course_url": "https://www.devopsschool.com/courses/finops/" }, { "name": "OpenTelemetry", "category": "Observability", "website": "https://opentelemetry.io/", "ds_course_url": "https://www.devopsschool.com/courses/opentelemetry/" }, { "name": "Optuna", "category": "Machine Learning & MLOps", "website": "https://optuna.org/", "ds_course_url": "" }, { "name": "Prometheus", "category": "Observability", "website": "https://prometheus.io/", "ds_course_url": "https://www.devopsschool.com/courses/prometheus/" }, { "name": "pytest", "category": "Foundations & Languages", "website": "https://docs.pytest.org/", "ds_course_url": "" }, { "name": "Python", "category": "Foundations & Languages", "website": "https://www.python.org/", "ds_course_url": "https://www.devopsschool.com/courses/python/" }, { "name": "PyTorch", "category": "Machine Learning & MLOps", "website": "https://pytorch.org/", "ds_course_url": "https://www.devopsschool.com/courses/deeplearning/" }, { "name": "Ray", "category": "Machine Learning & MLOps", "website": "https://www.ray.io/", "ds_course_url": "" }, { "name": "scikit-learn", "category": "Machine Learning & MLOps", "website": "https://scikit-learn.org/", "ds_course_url": "https://www.devopsschool.com/courses/ml/" }, { "name": "SHAP", "category": "Machine Learning & MLOps", "website": "https://shap.readthedocs.io/", "ds_course_url": "" }, { "name": "Sigstore", "category": "Security & Supply Chain", "website": "https://www.sigstore.dev/", "ds_course_url": "" }, { "name": "Terraform", "category": "Infrastructure as Code & Automation", "website": "https://developer.hashicorp.com/terraform", "ds_course_url": "https://www.devopsschool.com/courses/terraform/" }, { "name": "ZenML", "category": "Machine Learning & MLOps", "website": "https://www.zenml.io/", "ds_course_url": "" } ], "faqs": [ { "q": "What is the difference between MLOps and DevOps?", "a": "DevOps automates the delivery of code. MLOps must also version and validate data, track experiments, manage models as versioned artifacts and monitor statistical behaviour such as drift, because an ML system can degrade without any code change. MLOps reuses DevOps practices like CI/CD, infrastructure as code and observability, and adds continuous training and model governance on top." }, { "q": "Is MLOps still worth learning now that everyone is building with LLMs?", "a": "Yes. Most measurable AI value in fraud detection, forecasting, recommendations, pricing and risk still comes from predictive models, and LLMOps is built on the same foundations: registries, pipelines, serving, monitoring and governance. Learn MLOps first, then add LLM-specific skills with the LLMOps path." }, { "q": "Do I need to be a data scientist to become an MLOps engineer?", "a": "No. You need enough ML to understand metrics, data leakage and drift, but the job is mostly software, data and platform engineering. Many MLOps engineers come from DevOps, SRE or backend roles and partner with data scientists who own the modelling." }, { "q": "Which MLOps tools should I learn first?", "a": "Start with Python, Git, Docker and MLflow, because experiment tracking and a model registry pay off immediately. Then add a pipeline orchestrator (Kubeflow Pipelines or Airflow), a serving layer (KServe or BentoML) and a monitoring tool such as Evidently. Kubernetes skills become essential once you run shared platforms or GPUs." }, { "q": "How long does it take to learn MLOps?", "a": "At about 10 hours a week this path takes roughly 30 weeks. Engineers with strong DevOps backgrounds move faster through the infrastructure stages, while data scientists usually spend more time on Kubernetes, CI/CD and observability." }, { "q": "Which MLOps certification is best in 2026?", "a": "Choose the one that matches your cloud: AWS Certified Machine Learning Engineer – Associate, Google Professional Machine Learning Engineer, or Microsoft's AI-300 Machine Learning Operations Engineer Associate, which replaced DP-100 in 2026. Databricks Machine Learning Professional suits lakehouse teams and the CKA strengthens the platform side. A portfolio pipeline that trains, serves and monitors a model still matters more than any badge." }, { "q": "Do I need GPUs to practise MLOps?", "a": "Not for most of this path. Tabular models train on a laptop, and a local kind or minikube cluster is enough for Kubeflow Pipelines and KServe. Rent a cloud GPU by the hour only for the GPU scheduling and deep-learning serving exercises." } ], "related": [ "llmops", "dataops", "platform-engineering", "aiops" ], "key_tools": [ "MLflow", "Kubeflow", "KServe", "Ray", "Feast", "Evidently" ], "course_url": "https://www.devopsschool.com/courses/mlops/", "seo_title": "MLOps Learning Path 2026: Roadmap, Tools & Projects", "seo_description": "2026 MLOps roadmap: MLflow, Feast, Kubeflow, KServe, Evidently and GPU scheduling on Kubernetes, with hands-on projects and current certifications.", "url": "https://www.devopsschool.com/path/mlops/", "updated_at": "2026-10-26 08:58:45" }