// Autonomous time-budget experiment loop. Modify a training script, train for a fixed wall-clock budget, evaluate, record, repeat. Inspired by karpathy/autoresearch. Use for overnight architecture search, systematic hyperparameter sweeps, or any iterative model improvement workflow.
Comprehensive Claude Code knowledge base — plugins, hooks, skills, agents, MCP, channels, headless mode, permissions, settings, and all extensibility features. Use when building, configuring, debugging, or extending Claude Code.
Use when building a skill, creating a SKILL.md, packaging a workflow, making a slash command, or asked "how do I make a skill". Scaffolds the folder, generates SKILL.md from a template, validates against spec. Produces a complete ready-to-deploy skill folder: scripts, references, assets. Also use to review or improve an existing skill.
Statistical analysis, hypothesis testing, A/B testing, cohort analysis, segmentation, trend detection, business metrics, pre-delivery validation, and data visualization. Use when the user asks to "analyze this data", "run a statistical test", "compare groups", "find trends", "do A/B test analysis", "segment customers", "calculate KPIs", "validate this analysis", "check my work", "sanity check", "review my numbers", "make a chart", "create a dashboard", "plot the data", "visualize results", or mentions hypothesis testing, cohort analysis, business analytics, data validation, bar charts, line charts, heatmaps, scatter plots, or data storytelling.
Context engineering for building production LLM applications: context window management, degradation patterns, optimization strategies, memory system selection, multi-agent architecture, filesystem context patterns, and tool design principles. Use when building LLM apps, RAG pipelines, AI agents, multi-agent systems, or when designing memory, tool APIs, or context strategies for any language model application.
Explore, clean, and engineer datasets end-to-end: statistical profiling, distribution checks, missing value analysis, duplicate detection, outlier removal, type fixing, encoding, create features, encode categories, transform columns, add rolling windows, build interaction terms, and feature engineering. Supports pandas, polars, and PySpark. Use when the user wants to explore data, profile columns, understand a dataset, clean data, handle missing values, remove duplicates, fix data types, preprocess a dataset before modeling, create features, encode categories, transform columns, add rolling windows, build interaction terms, or do feature engineering.
Detect data drift, concept drift, and model performance degradation in production. Uses PSI, KS-test, and chi-squared for statistical drift, plus evidently and nannyml for automated reports. Use when monitoring a deployed model or comparing training vs production data distributions.
| name | autoexperiment |
| description | Autonomous time-budget experiment loop. Modify a training script, train for a fixed wall-clock budget, evaluate, record, repeat. Inspired by karpathy/autoresearch. Use for overnight architecture search, systematic hyperparameter sweeps, or any iterative model improvement workflow. |
| allowed-tools | Bash(uv run * scripts/time_budget_train.py *) Bash, Read, Write, Edit, Glob, Grep |
| argument-hint | path to train.py or description of experiment goal |
| model | opus |
| effort | max |
| disable-model-invocation | true |
| context | fork |
| agent | mlx:ml-engineer |
| compatibility | >=1.0 |
| metadata | {"category":"model-training","tags":["experiment-tracking","hyperparameter-search","autonomous","time-budget","iteration"],"phase":"train"} |
Run autonomous time-budget experiment loops. Each iteration modifies train.py,
trains for a fixed wall-clock budget, evaluates, records in results.tsv, and repeats.
results.tsv exists with a baseline (exp000) before iteratingEXPERIMENT.md with your goal, baseline, hypothesis, and constraints/mlx:autoexperiment path/to/train.pyEXPERIMENT.md for the current hypothesisresults.tsv for experiment historytrain.py with the single changetimeout $BUDGET uv run train.pyresults.tsv: KEEP / DISCARD / CRASHEXPERIMENT.md "Next to try" sectionSee references/EXPERIMENT.md.template for the hypothesis file format.
See scripts/time_budget_train.py for a complete training script template with all patterns.
total_nats / (math.log(2) * total_bytes) — vocab-independent metricif math.isnan(loss) or loss > 100: sys.exit(1)See references/autoexperiment-guide.md for full documentation.