Run any Skill in Manus with one click

reproducible-ml

Stars6

Forks0

UpdatedJune 6, 2026 at 02:16

Use when an ML result must be reproducible — fixing seeds, pinning environments, versioning data, and structuring projects so runs can be exactly recreated. Covers determinism gotchas across NumPy, PyTorch, and CUDA.

Installation

Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.

Run Skill in Manus

Source

param087

param087/agent-ml-skills

View GitHub Repository View Creator Repositories

Download

Run Skill in Manus

Related occupationsSOC

Based on SOC occupation classification

Software DevelopersComputer and Mathematical Occupations·SOC 15-1252

SKILL.md

readonly

name	reproducible-ml
description	Use when an ML result must be reproducible — fixing seeds, pinning environments, versioning data, and structuring projects so runs can be exactly recreated. Covers determinism gotchas across NumPy, PyTorch, and CUDA.

Reproducible ML

Overview

Reproducibility means: same code + same data + same config → same result. It is a prerequisite for trusting comparisons, debugging regressions, and shipping. Three pillars: seed everything, pin everything, version the data.

When to use

Results vary run-to-run.
Setting up a new project or research codebase.
Preparing work others must reproduce (papers, audits, reviews).

Pillar 1 — Seed everything

import os, random, numpy as np, torch

def seed_everything(seed: int = 42):
    os.environ["PYTHONHASHSEED"] = str(seed)
    random.seed(seed)
    np.random.seed(seed)
    torch.manual_seed(seed)
    torch.cuda.manual_seed_all(seed)
    torch.backends.cudnn.deterministic = True
    torch.backends.cudnn.benchmark = False  # disables nondeterministic autotuner

For DataLoaders, also set worker_init_fn and a generator so workers are deterministic.

Pillar 2 — Pin the environment

Pin exact versions: requirements.txt with ==, or uv.lock / poetry.lock / conda env export.
Record Python + CUDA + cuDNN versions in the run metadata.
Containerize (Docker) for cross-machine reproducibility.

Pillar 3 — Version the data

Hash datasets (sha256) and log the hash with every run.
Use DVC or dataset snapshots; never overwrite data.csv in place.
Treat data as immutable inputs keyed by version (data_v="2026-06-01").

Project layout that supports reproducibility

project/
├── data/            # raw (immutable) + processed, both versioned
├── src/             # importable code, no notebooks doing real work
├── configs/         # YAML/Hydra configs, one per experiment
├── scripts/         # entrypoints: train.py, evaluate.py
├── requirements.txt # or uv.lock / poetry.lock (pinned)
└── README.md        # exact commands to reproduce

Determinism gotchas

GPU reductions can be nondeterministic even with seeds — use torch.use_deterministic_algorithms(True) and set CUBLAS_WORKSPACE_CONFIG=:4096:8.
Parallel groupby/apply ordering can vary — sort before reducing.
set/dict ordering across processes — set PYTHONHASHSEED.
Non-pinned dependencies silently change behavior between installs.

Pitfalls

Seeding only NumPy but using PyTorch/CUDA RNGs.
"Works on my machine" — unpinned env + unversioned data.
Doing real computation in notebooks with hidden execution order — move logic to src/.

Hand-off

A repo where pip install -r requirements.txt && python scripts/train.py --config configs/exp.yaml recreates the exact result that experiment-tracking logged.

More from this repository

same repository

data-cleaning

param087/agent-ml-skills

Use when preparing raw data for modeling — handling missing values, duplicates, inconsistent types, outliers, and bad categorical values. Emphasizes fitting all imputation on train-only to avoid leakage.

2026-06-066

experiment-tracking

param087/agent-ml-skills

Use when running ML experiments that need to be compared, reproduced, or shared. Covers MLflow/Weights & Biases logging, what to track, run organization, and model registry basics.

2026-06-066

exploratory-data-analysis

param087/agent-ml-skills

Use when starting on a new dataset, before modeling, or when asked to "explore", "profile", "understand", or "summarize" data. Covers structured EDA, distributions, correlations, target leakage checks, and visualization.

2026-06-066

feature-engineering

param087/agent-ml-skills

Use when creating, encoding, scaling, or selecting features for ML models. Covers categorical encoding, numeric transforms, datetime/text/aggregation features, and leakage-safe target encoding.

2026-06-066

hyperparameter-tuning

param087/agent-ml-skills

Use when optimizing model hyperparameters. Covers search strategy (random vs Bayesian/Optuna), leakage-safe tuning inside CV, search-space design, early stopping, and budget management.

2026-06-066

imbalanced-data

param087/agent-ml-skills

Use when the target is rare (fraud, churn, disease, anomalies). Covers correct metrics, resampling (SMOTE/undersampling), class weights, threshold tuning, and avoiding the accuracy trap and resampling leakage.

2026-06-066

name	reproducible-ml
description	Use when an ML result must be reproducible — fixing seeds, pinning environments, versioning data, and structuring projects so runs can be exactly recreated. Covers determinism gotchas across NumPy, PyTorch, and CUDA.

Reproducible ML

Overview

When to use

Results vary run-to-run.
Setting up a new project or research codebase.
Preparing work others must reproduce (papers, audits, reviews).

Pillar 1 — Seed everything

import os, random, numpy as np, torch

def seed_everything(seed: int = 42):
    os.environ["PYTHONHASHSEED"] = str(seed)
    random.seed(seed)
    np.random.seed(seed)
    torch.manual_seed(seed)
    torch.cuda.manual_seed_all(seed)
    torch.backends.cudnn.deterministic = True
    torch.backends.cudnn.benchmark = False  # disables nondeterministic autotuner

For DataLoaders, also set worker_init_fn and a generator so workers are deterministic.

Pillar 2 — Pin the environment

Pin exact versions: requirements.txt with ==, or uv.lock / poetry.lock / conda env export.
Record Python + CUDA + cuDNN versions in the run metadata.
Containerize (Docker) for cross-machine reproducibility.

Pillar 3 — Version the data

Hash datasets (sha256) and log the hash with every run.
Use DVC or dataset snapshots; never overwrite data.csv in place.
Treat data as immutable inputs keyed by version (data_v="2026-06-01").

Project layout that supports reproducibility

project/
├── data/            # raw (immutable) + processed, both versioned
├── src/             # importable code, no notebooks doing real work
├── configs/         # YAML/Hydra configs, one per experiment
├── scripts/         # entrypoints: train.py, evaluate.py
├── requirements.txt # or uv.lock / poetry.lock (pinned)
└── README.md        # exact commands to reproduce

Determinism gotchas

GPU reductions can be nondeterministic even with seeds — use torch.use_deterministic_algorithms(True) and set CUBLAS_WORKSPACE_CONFIG=:4096:8.
Parallel groupby/apply ordering can vary — sort before reducing.
set/dict ordering across processes — set PYTHONHASHSEED.
Non-pinned dependencies silently change behavior between installs.

Pitfalls

Seeding only NumPy but using PyTorch/CUDA RNGs.
"Works on my machine" — unpinned env + unversioned data.
Doing real computation in notebooks with hidden execution order — move logic to src/.

Hand-off

A repo where pip install -r requirements.txt && python scripts/train.py --config configs/exp.yaml recreates the exact result that experiment-tracking logged.