xetrack

Guide users through rigorous ML/AI benchmarking experiments using xetrack. Use when users want to: (1) Compare ML models, hyperparameters, or architectures, (2) Benchmark LLM prompts, few-shot examples, or generation strategies, (3) Evaluate data processing pipelines or embeddings, (4) Set up reproducible experiments with caching and validation, (5) Debug existing benchmarks for data leaks or inconsistencies, (6) Analyze benchmark results with SQL/DuckDB. Helps design experiments end-to-start following single-execution principles.

Manage ML experiment versioning using Git + DVC + xetrack + DuckDB. Use this skill when the user wants to: version experiments with git tags and DVC, run sequential or parallel ML experiments with reproducibility, merge or rebase experiment results (SQLite databases, parquet data, models), set up DVC remotes and cache (including S3), manage model candidates and promotion, use git worktrees for parallel code/data experiments, track experiment metadata in SQLite with xetrack, retrieve models or data from past experiments, explore or compare historical experiments, or find what data was used. Triggers on: "version experiment", "tag experiment", "DVC setup", "merge results", "rebase data", "git worktree experiment", "parallel experiments", "model promotion", "candidates", "experiment tracking", "reproducibility", "dvc pipeline", "merge database", "data versioning", "retrieve model", "get model", "find experiment", "explore experiment", "compare experiments", "what data was used", "inspect results", "restore exper

API reference and usage patterns for xetrack — lightweight experiment tracking with SQLite/DuckDB. Use when writing code that uses xetrack, when unsure about Tracker or Reader API, when using the xt CLI, when working with caching/diskcache, or when tracking ML experiments. Triggers on "xetrack", "Tracker", "Reader", "tracker.track", "tracker.log", "xt cli", "xetrack cache", "experiment tracking", "track function", "track metrics".