ml-system-design-interview

name	ml-system-design-interview
license	Apache-2.0
description	Coaches end-to-end ML system design interviews covering inference pipelines, recommendation systems, RAG, feature stores, and monitoring. Use for L6+ design rounds, ML architecture whiteboarding, system design practice, serving tradeoff analysis. Activate on "ML system design", "ML interview", "recommendation system design", "RAG architecture", "feature store design", "model serving". NOT for coding interviews, behavioral questions, ML theory quizzes, or paper implementations.
allowed-tools	Read,Write,Edit
metadata	{"category":"Career & Interview","pairs-with":[{"skill":"interview-loop-strategist","reason":"Complementary skill"},{"skill":"senior-coding-interview","reason":"Complementary skill"},{"skill":"anthropic-technical-deep-dive","reason":"Complementary skill"},{"skill":"interview-simulator","reason":"Complementary skill"}],"tags":["interview","ml-system-design","machine-learning","architecture"]}
category	Career & Interview
tags	["ml-system-design","interview","machine-learning","architecture","preparation"]

ML System Design Interview

End-to-end ML pipeline design coaching for staff+ engineers. Covers the full arc from problem definition through production monitoring -- the scope expected at L6+ interviews at top-tier ML organizations.

Decision Points

Serving Architecture Selection

Query: What inference pattern should we use?

1. Input Predictability?
   ├─ Finite/enumerable → BATCH PREDICTION
   │  └─ Example: recommend videos for all users nightly
   └─ Infinite/dynamic → ONLINE INFERENCE
      └─ Go to #2

2. Latency Requirement?
   ├─ <100ms → ONLINE INFERENCE (cached features)
   │  └─ Example: search ranking, fraud detection
   ├─ 100ms-1s → NEAR-REAL-TIME
   │  └─ Example: feed ranking with fresh user signals
   └─ >1s → STREAMING COMPUTATION
       └─ Example: complex multi-modal analysis

3. Feature Freshness?
   ├─ Static features only → Precompute embeddings
   ├─ Daily updates acceptable → Feature store + cache
   └─ Real-time required → Online feature computation

Problem Scoping Strategy

Given a vague prompt like "design a recommendation system":

1. Business Objective Clarification
   ├─ Revenue optimization? → Focus on purchase conversion metrics
   ├─ Engagement optimization? → Focus on time-spent, retention metrics  
   └─ Exploration/discovery? → Focus on diversity, serendipity metrics

2. Scale Assessment
   ├─ <1M users → Simple collaborative filtering acceptable
   ├─ 1-100M users → Need distributed training, caching layer
   └─ >100M users → Multi-stage retrieval mandatory (retrieval→ranking)

3. Cold Start Criticality
   ├─ High new user/item churn → Content-based features essential
   ├─ Stable catalog → Collaborative filtering sufficient
   └─ Mixed → Hybrid approach with exploration bonus

Model Complexity Progression

Start simple, justify complexity upgrades:

1. Baseline Decision
   ├─ Tabular features → XGBoost/LightGBM
   ├─ Text/sequences → BERT/RoBERTa fine-tuned
   └─ Multi-modal → Pre-trained encoders + MLP

2. Production Upgrade Triggers
   ├─ Baseline hits accuracy ceiling → Deep architecture
   ├─ Cold start problems → Content-based + collaborative hybrid
   ├─ Real-time features critical → Online learning/streaming updates
   └─ Multi-objective conflicts → Multi-task architecture

3. Scaling Triggers  
   ├─ Inference latency >SLA → Model distillation/quantization
   ├─ Training time >acceptable → Distributed training
   └─ Serving cost >budget → Cheaper architecture or caching

Failure Modes

Cold Start Death Spiral

Symptoms: New users see irrelevant recommendations, bounce immediately, never generate engagement signals for learning. Root Cause: System relies entirely on collaborative filtering without content-based fallback. Detection Rule: If new user retention <50% of established user retention, you have cold start problems. : Implement content-based features (item metadata, user demographics), exploration bonus for new users, and onboarding questionnaire to bootstrap preferences.