ml-ops

name	ml-ops
description	Model deployment, versioning, monitoring, A/B testing, feature stores, and ML pipeline orchestration
layer	domain
category	ai-ml
triggers	["MLOps","model deployment","model monitoring","model versioning","A/B testing models","feature store","ML pipeline","model serving"]
inputs	[{"model":"Model type, framework (PyTorch, TensorFlow, ONNX, LLM)"},{"requirements":"Latency, throughput, availability, cost targets"},{"infrastructure":"Cloud provider, Kubernetes, serverless preferences"},{"workflow":"Training, evaluation, deployment, monitoring needs"}]
outputs	[{"deployment_architecture":"Model serving infrastructure design"},{"pipeline_config":"Training and deployment pipeline configuration"},{"monitoring_setup":"Model performance and data drift monitoring"},{"versioning_strategy":"Model and dataset version management"},{"ab_testing_plan":"Experiment design for model comparison"}]
linksTo	["docker","kubernetes","monitoring","cicd","logging"]
linkedFrom	["ai-agents","rag","plan"]
preferredNextSkills	["monitoring","docker"]
fallbackSkills	["cicd"]
riskLevel	medium
memoryReadPolicy	selective
memoryWritePolicy	none
sideEffects	["May create deployment configurations","May configure monitoring dashboards","Model deployments affect production traffic"]

ML Ops Skill

Purpose

Design and implement the operational infrastructure for machine learning systems: model versioning, reproducible training pipelines, deployment strategies, A/B testing, monitoring for data drift and model degradation, and feature stores. MLOps bridges the gap between training a model in a notebook and running it reliably in production.

Key Concepts

MLOps Lifecycle

DATA:
  Collection -> Cleaning -> Feature Engineering -> Feature Store
                                                       |
MODEL:                                                 v
  Experiment -> Train -> Evaluate -> Register -> Deploy -> Monitor
      ^                                                      |
      |                                                      v
      +<--- Retrain <--- Alert (drift/degradation detected) -+

Deployment Patterns

BATCH INFERENCE:
  Run predictions on a schedule (hourly, daily)
  Results stored in database/cache
  Good for: Recommendations, risk scoring, email personalization
  Latency: Minutes to hours (acceptable)

REAL-TIME INFERENCE:
  HTTP API endpoint, request-response
  Good for: Search ranking, fraud detection, chatbots
  Latency: <100ms (required)

STREAMING INFERENCE:
  Process events from a message queue
  Good for: Anomaly detection, real-time scoring
  Latency: Seconds (near real-time)

EDGE INFERENCE:
  Model runs on device (browser, mobile, IoT)
  Good for: Image classification, NLP on device
  Latency: <10ms (on-device)

LLM SERVING:
  Managed API (OpenAI, Anthropic) or self-hosted (vLLM, Ollama)
  Good for: Text generation, chat, code generation
  Latency: 1-30 seconds (token streaming)

Model Versioning

MODEL REGISTRY:
  model-name/
    v1.0.0/
      model.onnx (or model.pt, model weights)
      config.json (hyperparameters, architecture)
      metrics.json (evaluation results)
      requirements.txt (dependencies)
      README.md (training notes, known limitations)
    v1.1.0/
      ...
    v2.0.0/
      ...

VERSIONING SCHEME:
  MAJOR: Architecture change, different input/output schema
  MINOR: Retrained on new data, improved accuracy
  PATCH: Bug fix, configuration change

METADATA TO TRACK:
  - Training data version (hash or timestamp)
  - Hyperparameters used
  - Evaluation metrics (accuracy, F1, latency)
  - Training duration and cost
  - Git commit of training code
  - Feature set version

Patterns

Model Serving with FastAPI

# serve.py
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import onnxruntime  ort
 numpy  np

app = FastAPI(title=, version=)


session = ort.InferenceSession()

 ():
    features: []

 ():
    prediction: 
    confidence: 
    model_version: 


  ():
    input_array = np.array([request.features], dtype=np.float32)
    outputs = session.run(, {: input_array})

     PredictionResponse(
        prediction=(outputs[][]),
        confidence=(outputs[][]),
        model_version=,
    )


  ():
     {: , : }