ibm-workshop

Declare the pipeline from data source to predictor as a **skrub DataOps graph** (not as a bare `sklearn.Pipeline`). Every step is either a pure-Python function (stateless) attached via `.skb.apply_func`, or a sklearn-compatible estimator (stateful) attached via `.skb.apply`. Stops at the declared object — no fit, split, tuning, persistence, or evaluation. TRIGGER — any of: - Writing or editing code that declares any link in the chain *data source → predictor*: loaders, preprocessing, encoders / imputers / scalers, feature steps, composition objects (`Pipeline`, `ColumnTransformer`, skrub `tabular_pipeline`, `nn.Module`), or the final estimator. - A pure-Python data-processing function destined for the pipeline path (cleans / derives / reshapes) — whether wrapped via `FunctionTransformer`, `skrub.@deferred` / `skrub.var`, a custom `BaseEstimator` subclass, or just called in the training path before the estimator. - A step is added, removed, swapped, or reordered inside an existing pipeline de

Opinionated Python stack for data-science / ML work — one library per job, organized into tiers (mandatory / user choice / optional / transitive). SKILL.md is the index; per-library `references/<library>.md` files carry scope, "pick this when" / "pick something else when", and pairings. TRIGGER when (any of these): (1) **a library import fails** in this stack's domain — the answer is install, not substitute (see § "Missing dependency"); (2) **a library choice has to be made** — explicitly (the user asks "which library for X?") or implicitly (code is about to introduce a new dependency, or the project is being scaffolded and the tabular library hasn't been picked yet); (3) starting a new Python data-science / ML project; (4) the user or current code reaches for a substitute outside the stack (xgboost, lightgbm, black, isort, flake8, poetry, hatch), or reaches for `mlflow` to log params/metrics, or for `cross_val_score` + handwritten reporting — redirect: tracking → `skore` Project API, evaluation / reporting →

Methodology for evaluating a single sklearn-compatible learner (in particular, the `SkrubLearner` produced by `build-ml-pipeline`). Owns: which entry point to call (`skore.evaluate` first, the explicit report classes when needed), which cross-validator to pick from scikit-learn's catalogue, how to consume the structural metadata (`groups`, `times`, …) attached at build time via `.skb.mark_as_X(split_kwargs=...)`. Stops at "what does the report say". Defaults (metrics, plots) come from skore; only override on explicit user request. TRIGGER when: code calls `cross_val_score`, `cross_validate`, `classification_report`, or any handwritten metric print (`print(mean_squared_error(...))`); code calls `.skb.cross_validate(...)` (route through skore for richer output); user asks how to score, evaluate, or compare a single learner; user asks how to pick a cross-validator; user wants to see a report / metrics / diagnostic plots for a fitted learner. SKIP when: declaring the pipeline (use `build-ml-pipeline`); hyperparam

Decide where files live in an ML experimentation project: reusable code in `src/<pkg>/`, one `# %%` script per experiment in `experiments/`, design notes + index in `journal/`, reports in `reports/`, agent-only probes in `scratch/`. Owns the layout, the file-creation rules (one file per experiment, ask before editing), and the jupytext `# %%` script convention. Never imposes `data/` — the user owns that. TRIGGER — any of: - Starting a new ML project / scaffolding a workspace. - About to create the first experiment file in a project. - About to create `src/<pkg>/data.py` / `features.py` / `pipeline.py` / `evaluate.py` for the first time. - About to write a `.ipynb` for experimentation — redirect to a `# %%` script under `experiments/`. - User asks where something should live, how to organize the project, or how to set up the workspace. - About to add a new experiment iteration — decide new file vs edit existing (ask the user). SKIP when: the file is clearly part of an already-populated module (e.g., a

Look up the public API of a Python package against the *installed version* and cache what's worth keeping. Four shapes by question type: (0) cache hit under `scratch/api/<lib>/<version>/`; (1) `inspect.signature` + `pydoc.render_doc` for a symbol; (2) `dir` / `pkgutil.iter_modules` for a module surface; (3) WebSearch + WebFetch of versioned docs for narrative ("how", "which", "what does X return when Y"). Never write a symbol from training-data memory — recognition is not a lookup. TRIGGER — any of: - About to name a symbol (function / class / method / arg) in code. - User asks "what's the signature of X?", "what's in module Y?", "how do I call X?", "which of A/B should I use?". - User asks "what does X return when <condition>?" (Shape 3 — see decision table). - Another workflow skill (`build-ml-pipeline`, `evaluate-ml-pipeline`, `iterate-from-skore`, `smoke-test-ml-pipeline`) says "consult the API skill". - About to reach for a library's "obvious" pattern from memory. SKIP when: the signature is obvi