experiment-tracking

name	experiment-tracking
version	2.0.0
sasmp_version	1.3.0
description	Master ML experiment tracking - MLflow, W&B, Neptune, versioning, reproducibility
bonded_agent	02-experiment-tracking
bond_type	PRIMARY_BOND
category	experimentation
difficulty	intermediate
estimated_hours	30
prerequisites	["mlops-basics"]
validation	{"pre_conditions":["Completed mlops-basics skill","Python environment setup"],"post_conditions":["Can set up experiment tracking","Can log parameters, metrics, artifacts","Can use model registry"],"parameter_schema":{"platform":{"type":"string","enum":["mlflow","wandb","neptune","comet"]}}}
observability	{"log_inputs":true,"log_outputs":true,"metrics":["experiments_created","runs_logged","models_registered"]}

Experiment Tracking Skill

Learn: Master ML experiment tracking for reproducibility and collaboration.

Skill Overview

Attribute	Value
Bonded Agent	02-experiment-tracking
Difficulty	Intermediate
Duration	30 hours
Prerequisites	mlops-basics

---

Learning Objectives

1. Set up experiment tracking infrastructure
2. Log parameters, metrics, and artifacts systematically
3. Compare experiments and identify best models
4. Use model registry for version management
5. Collaborate with team using shared tracking

---

Topics Covered

Module 1: Platform Setup (6 hours)

Platform Comparison:

Feature	MLflow	W&B	Neptune
Self-hosted	✅	❌	❌
Free tier	✅	✅	✅
Real-time	❌	✅	✅
Git integration	⚠️	✅	✅

Setup Exercises:

• Install MLflow and start local server
• Create W&B account and initialize project
• Compare UI/UX of both platforms

---

Module 2: Experiment Logging (10 hours)

What to Log:

# Complete logging example
with mlflow.start_run():
    # 1. Parameters (hyperparameters, configs)
    mlflow.log_params({
        "learning_rate": ,
        : ,
        : 
    })

    
     epoch  ():
        mlflow.log_metrics({
            : train_loss,
            : val_loss
        }, step=epoch)

    
    mlflow.log_artifact()
    mlflow.pytorch.log_model(model, )

    
    mlflow.set_tags({
        : ,
        : 
    })