ml-model-code-review

name	ml-model-code-review
enabled	true
description	Use when performing ml model code review — mL/AI code review template covering data leakage detection, model reproducibility, bias assessment, feature engineering validation, and experiment tracking. Provides a systematic review framework for machine learning pipelines, model training code, and inference service changes.
required_connections	[{"prefix":"github","label":"GitHub"}]
config_fields	[{"key":"repository","label":"Repository","required":true,"placeholder":"e.g., org/ml-pipeline"},{"key":"pr_number","label":"PR Number","required":true,"placeholder":"e.g., 1234"},{"key":"model_type","label":"Model Type","required":false,"placeholder":"e.g., classification, NLP, computer vision, recommendation"}]
features	["CODE_REVIEW"]

ML Model Code Review Skill

Review ML PR #{{ pr_number }} in {{ repository }} for {{ model_type }} model.

Workflow

Phase 1 — Data Leakage

DATA LEAKAGE CHECK
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[ ] Train/test split:
    [ ] Data split before any preprocessing
    [ ] No information from test set in training
    [ ] Temporal splits for time-series data
    [ ] Group-aware splits (same entity not in both sets)
[ ] Feature leakage:
    [ ] No target-derived features
    [ ] No future data used in features
    [ ] No features unavailable at inference time
    [ ] Label encoding fit only on training data
[ ] Preprocessing leakage:
    [ ] Scalers/normalizers fit only on training data
    [ ] Imputation values from training data only
    [ ] Feature selection based on training data only

Phase 2 — Reproducibility

REPRODUCIBILITY CHECK
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[ ] Random seeds:
    [ ] Random seeds set for all stochastic operations
    [ ] Seeds documented and versioned
    [ ] Deterministic mode enabled where possible
[ ] Experiment tracking:
    [ ] Hyperparameters logged (MLflow, W&B, etc.)
    [ ] Training metrics tracked
    [ ] Model artifacts versioned
    [ ] Data version recorded (DVC, dataset hash)
[ ] Environment:
    [ ] Dependencies pinned (requirements.txt / conda.yml)
    [ ] GPU/CUDA version documented
    [ ] Docker image versioned for training environment

Phase 3 — Bias and Fairness

BIAS ASSESSMENT
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[ ] Data bias:
    [ ] Training data representative of target population
    [ ] Class imbalance addressed (oversampling, weights)
    [ ] Protected attributes identified
    [ ] Demographic parity evaluated
[ ] Model bias:
    [ ] Performance metrics computed per subgroup
    [ ] Disparate impact ratio calculated
    [ ] Fairness constraints applied if needed
    [ ] Bias mitigation documented
[ ] Documentation:
    [ ] Model card created/updated
    [ ] Known limitations documented
    [ ] Intended use cases specified

Phase 4 — Model Quality

MODEL QUALITY
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[ ] Evaluation:
    [ ] Appropriate metrics for task (not just accuracy)
    [ ] Evaluation on held-out test set
    [ ] Cross-validation where appropriate
    [ ] Baseline comparison provided
[ ] Production readiness:
    [ ] Model size acceptable for deployment target
    [ ] Inference latency within SLA
    [ ] Model serialization format compatible
    [ ] Input validation at inference time
    [ ] Graceful handling of missing/malformed features
    [ ] A/B test plan for model rollout

Counter-Rationalizations

Shortcut	Counter	Why
"We can skip some steps for this case"	Adapt the workflow steps, don't skip them	Skipped steps are where incidents and oversights originate
"The user seems to already know what to do"	Complete all workflow phases with the user	The workflow catches blind spots that experience alone misses
"This is a minor case, full process is overkill"	Scale the process down, don't turn it off	Minor cases become major when unstructured; the process scales, not disappears
"I'll fill in the details later"	Complete each section before moving on	Deferred details are forgotten; real-time capture is more accurate
"The template output isn't necessary"	Always produce the structured output format	Structured output enables comparison, audit trails, and handoff to other teams

Output Format

Produce an ML review report with:

1. Data leakage findings (none / potential / confirmed)
2. Reproducibility assessment (reproducible / partially / not reproducible)
3. Bias and fairness evaluation
4. Model quality metrics vs baseline
5. Production readiness checklist status