statspai-skill

# StatsPAI: Agent-Native Causal Inference & AER-Style Empirical Workflow StatsPAI is a validation-tiered Python package for causal inference and applied econometrics: one `import statspai as sp`, 1,100+ registered functions behind a self-describing API, and mature estimator result objects that commonly export to LaTeX / Word / Excel / BibTeX. This skill drives StatsPAI through the **canonical pipeline of an applied AER empirical paper**. Each step emits a paper-ready artifact (Table 1, event-study figure, Table 2 main results, robustness panel, replication stamp). - **Source**: https://github.com/brycewang-stanford/StatsPAI - **Install**: `pip install "statspai[fixest,plotting]"` (API surface re-validated against **statspai 1.19.0** — every `sp.*` reference, signature, and result-object attribute claim in this skill is checked by `validate_api_claims.py` in this folder). The bare `pip install statspai` is **not enough** for the default pipeline — see the dependency matrix below. - **Paper**: Wang & Rozelle (2026), *Journal of Open Source Software* 11(125), 10604, <https://doi.org/10.21105/joss.10604>; JSS materials in `Paper-JSS/README.md` and `docs/jss_source_audit_dossier.md` > **Install the right extras or the documented calls will raise `ImportError`.** Several core functions live behind optional dependency groups (verified from `pyproject.toml`): > > | You use… | Needs extra | Install | Symptom if missing | > |---|---|---|---| > | `sp.feols` / `sp.fepois` / `sp.feglm` (high-dim FE — the **default** for any `y ~ x \| fe` regression) | `fixest` (pyfixest) | `pip install "statspai[fixest]"` | `ImportError: pyfixest is required …` | > | Any figure (`sp.coefplot`, `sp.binscatter`, event-study/RD/SCM plots, `.plot()`) | `plotting` (matplotlib/seaborn) | `pip install "statspai[plotting]"` | `ImportError` on first plot | > | `sp.dragonnet` / `sp.tarnet` / `sp.cfrnet` / `sp.cevae` (neural causal) | `neural` (torch) | `pip install "statspai[neural]"` | `ImportError: PyTorch is required …` | > | `sp.causal_text.*` (text-as-treatment) | `text` (sentence-transformers) | `pip install "statspai[text]"` | `ImportError` on embed | > > A one-shot install covering the whole skill: `pip install "statspai[fixest,plotting,neural,text]"`. `sp.regtable` / `sp.collect` / Word+Excel+LaTeX export, `sp.regress`, IV, RD, DID (`callaway_santanna`), matching, DML, meta-learners, causal forest, BCF, TMLE, and the epi stack work on the base install. ## Verified skeleton (copy, then swap in your columns) This minimal pipeline **runs start-to-finish against statspai 1.19.0** (every call below was executed). It is the golden path — adapt column names / design, keep the call shapes and the unpack-then-save figure idiom. The full playbook (§−1 → §8) expands each step. ```python import numpy as np, pandas as pd, statspai as sp # df has: wage, training(0/1), worker_id, firm_id, year, first_treat_year, age, edu, tenure, ... # §1 Table 1 → Word/Excel/LaTeX mc = sp.mean_comparison(df, ["age","edu","tenure"], group="training", test="ttest", title="Table 1. Summary statistics") mc.to_word("tables/table1.docx"); mc.to_excel("tables/table1.xlsx") # §2 Estimand-first plan (freeze BEFORE estimating) q = sp.causal_question(treatment="training", outcome="wage", data=df, estimand="ATT", design="did", time_structure="panel", time="year", id="worker_id", covariates=["age","edu","tenure"]) plan = q.identify(); print(plan.summary()) # §3 Identification figure — from a CS/SA result (NOT event_study()); plotters return (fig, ax) cs = sp.callaway_santanna(df, y="wage", g="first_treat_year", t="year", i="worker_id", x=["age","edu"]) fig, ax = sp.enhanced_event_study_plot(cs, shade_pre=True); fig.savefig("figures/fig2a.png", dpi=300) # §4 Main table — mix sp.regress (no FE) + sp.feols (HDFE, needs statspai[fixest]) in ONE regtable M1 = sp.regress("wage ~ training", df, cluster="firm_id") M2 = sp.feols("wage ~ training + age + edu + tenure | industry + year", df, vcov={"CRV1":"firm_id"}) rt = sp.regtable(M1, M2, template="aer", coef_labels={"training":"Job training"}, model_labels=["(1) OLS","(2) FE"], stats=["N","R2","Cluster","FE"], title="Table 2. Effect of training on wages") rt.to_word("tables/table2.docx"); rt.to_excel("tables/table2.xlsx") open("tables/table2.tex","w").write(rt.to_latex()) # §5 Heterogeneity — per-row CATE at result.model_info["cate"] (there is NO .cate_estimates) ml = sp.metalearner(df, y="wage", treat="training", covariates=["age","edu","tenure"], learner="dr") fig, ax = sp.cate_plot(ml, kind="hist"); fig.savefig("figures/fig4.png", dpi=300) # §7 Robustness — Oster + E-value + honest-DID sensitivity figure sp.oster_bounds(data=df, y="wage", treat="training", controls=["age","edu","tenure"], r_max=1.3) sp.evalue(estimate=M2.params["training"], ci=tuple(M2.conf_int().loc["training"]), measure="RR") fig, ax = sp.sensitivity_plot(sp.honest_did(cs, method="smoothness"), original_estimate=cs.estimate, original_ci=cs.ci) fig.savefig("figures/fig6.png", dpi=300) # §8 One-file replication bundle (Word/Excel/LaTeX/Markdown from one source) c = sp.collect("Replication", template="aer") c.add_summary(df, vars=["wage","age","edu","tenure"], stats=["mean","sd","n"], title="Table 1") c.add_regression(M1, M2, model_labels=["(1)","(2)"], stats=["N","R2"], title="Table 2") for ext in ("docx","xlsx","tex","md"): c.save(f"replication/paper.{ext}") ``` > Epi (§A) and ML-causal (§B) reuse this exact scaffolding — only the §4 estimator stack changes (TMLE/g-formula/MR for epi; DML/meta-learner/causal-forest for ML), and every estimator still returns a result that drops into `sp.regtable` / `sp.collect`. ## Why for Agents 1. **Self-describing**: `sp.list_functions()` / `sp.describe_function(name)` / `sp.function_schema(name)` — registered symbols are discoverable without doc lookup. 2. **Structured results**: mature estimators return result objects with methods such as `.summary()`, `.plot()`, `.diagnostics`, `.to_latex()`, `.to_word()`, `.cite()` when supported. 3. **One import, full pipeline**: data contract → Table 1 → estimand-first DSL → identification graphs → main table → heterogeneity → mechanisms → robustness → replication package. 4. **Estimand-first**: `sp.causal_question(...).identify()` forces the "DID vs RD vs IV?" decision *before* estimation, with the identifying assumption written down — the way a referee expects to read it. ## SkillOpt-derived operating loop (read before the playbook) SkillOpt's useful lesson for this skill is procedural, not cosmetic: a skill is a bounded decision policy that should improve from rollout evidence while preserving verified behavior. Treat every StatsPAI request as a mini rollout: 1. **Route the mode first**: choose Default/AER, Mode A/epi, Mode B/ML-causal, or a narrow export-only path from the user's words. Do not run the full paper pipeline when the request is only "make Table 1" or "export this regression". 2. **Freeze the contract before estimating**: name `y`, treatment/exposure, unit/time ids, estimand, design, required artifacts, and install extras. If any field is missing, infer only when the column names make the choice obvious; otherwise produce a short blocking checklist instead of hallucinating columns. 3. **Start from the smallest verified call shape**: prefer the skeleton and the relevant section-specific snippet over ad hoc API guesses. For an unfamiliar function, call `sp.describe_function(name)` / `sp.function_schema(name)` before writing code. 4. **Widen one block at a time**: data contract → plan → diagnostic figure → main estimate → robustness/export. After each block, read warnings and object attributes before passing the result downstream. 5. **Gate the answer on artifacts, not intentions**: final responses should list the files produced, the identifying assumption, the estimator class, and any failed or skipped gate. Never claim "paper-ready" if Word/Excel/LaTeX exports or required diagnostics were not actually generated. 6. **Turn failures into bounded corrections**: if a call raises, fix the smallest wrong rule (signature, result type, optional extra, plot return shape) and continue from the last verified artifact. Do not rewrite the pipeline wholesale. ### SkillOpt-style execution gate (task-local card) Before generating or revising StatsPAI analysis code, compress the request into a task-local `best_skill` card: ```text best_skill: <mode + design + artifact target> train_signal: <current failure, user goal, or missing evidence> selection_split: <focal dataset/spec/output used to judge the candidate> heldout_gate: <checks the patch must pass beyond the focal example> accepted_patterns: <rules to reuse after validation> rejected_patterns: <failed shortcuts not to retry without new evidence> patch_scope: <one estimator/sample/export/robustness change> reject_if: <conditions that force rollback to the last passing spec> ``` 1. **Route card**: record the mode (`econ`, `epi`, or `ml-causal`), estimand, identification design, focal outcome/treatment, StatsPAI install extras, and required artifacts. 2. **Bounded edit**: change one decision at a time (sample rule, estimator, optional extra, plot return shape, export format, or robustness check). Prefer the smallest patch that can pass validation. 3. **Selection split discipline**: treat the user's immediate failure or requested artifact as the selection split. Reserve at least one alternate outcome, sample window, estimator family, or export target as the held-out gate. 4. **Held-out gate**: define checks before running code: row counts, key uniqueness, treatment support, missingness thresholds, expected table/figure files, and one non-focal robustness/specification that the change must not break. 5. **Reject buffer**: if a candidate spec fails the gate, log the failure, code diff, and gate output in `analysis_log.md`; revert to the last passing spec and do not retry the same unchecked pattern. 6. **Slow/meta update**: at the end of the task, write down `accepted_patterns` and `rejected_patterns` from the trajectory. Do not widen the canonical project template from a single passing run. 7. **Promote only after validation**: only turn a one-off fix into reusable project boilerplate after it passes the current data and at least one alternate outcome/sample/specification. ### Acceptance gates by request type | Request type | Minimum gates before final answer | |---|---| | Export-only / `outreg2` equivalent | At least one `RegtableResult` or `Collection` object is created; requested `.docx` / `.xlsx` / `.tex` paths are written or the exact missing optional dependency is reported | | AER DID / event study | `sp.causal_question(...).identify()` saved or printed; CS/SA result used for the event-study figure; numerical pre-trends checked separately with `sp.event_study(...)` or equivalent; Table 2 and at least one robustness/sensitivity artifact produced | | IV | First-stage F and instrument story reported before the 2SLS coefficient; no `\| fe` formula is passed to `sp.ivreg`; FE-IV needs explicit dummy construction or a stated limitation | | RD | McCrary/manipulation check plus RD plot are produced before the treatment-effect table; bandwidth/kernel sensitivity is in the robustness block | | Matching / weighting | Balance or love plot is produced before outcome estimation; weights are carried into the Table 1 / balance export when applicable | | Epi / target-trial | Target-trial protocol is written before modeling; positivity/overlap is checked; IPTW/g-formula/TMLE estimates are compared when data support them; E-value or equivalent sensitivity is reported | | ML causal / CATE | Train/holdout split and nuisance learners are explicit; per-row CATE source is valid (`model_info["cate"]` for meta-learners or `cf.effect(X)` for forests); policy/OPE claims use holdout data | | Stata/R migration | Use StatsPAI's self-description or translator surface first; preserve semantic notes for unsupported options instead of silently pretending full parity | ### Maintenance rule for future skill edits When improving this skill itself, follow a SkillOpt-style accept rule: propose a small add/delete/replace edit, then accept it only if it helps a concrete failure case and does not regress the verified skeleton, export cookbook, or Common Mistakes table. Use `EVALS.md` as the held-out gate set for future skill edits. Keep reusable fixes near the earliest section where an agent will need them; keep rare API traps in Common Mistakes. ## The AER-style empirical pipeline The skill mirrors the canonical sections of an applied AER / QJE / AEJ paper. Each step below is one paper section and one set of artifacts on disk. ``` Paper section Step StatsPAI moves ─────────────────────────── ───── ──────────────────────────────────────────────── Pre-Analysis Plan −1 sp.power.* + freeze IdentificationPlan to disk §1. Data 0 data_contract + sample-construction log (footnote 4) §1.1 Descriptives (Table 1) 1 sp.sumstats · sp.balance_table · sp.describe §2. Empirical Strategy 2 write equation + identifying assumption + sp.causal_question (LLM-DAG addendum) 2.5 sp.llm_dag_propose · validate · constrained §3. Identification graphics 3 event-study · first-stage F · McCrary · love plot §4. Main Results (Table 2) 4 progressive controls + FE (sp.regtable / sp.causal) §5. Heterogeneity (Table 3) 5 sp.subgroup_analysis · sp.continuous_did · CATE §6. Mechanisms 6 sp.mediation · sp.decompose §7. Robustness gauntlet 7 placebo · Oster · honest_did · E-value · 2-way / Conley SE · spec_curve §8. Replication package 8 .to_latex() · .plot() · reproducibility stamp ``` > **All code blocks below share one running example (`training → wage`, with `worker_id / firm_id / year / age / edu / tenure`) purely for readability.** Column names, `population`, `estimand`, and `design` values are **illustrative** — substitute the user's actual columns and research question. Only `sp.*` function names and argument *shapes* are normative. ## Three domain modes (default = AER econ; alternates = epi & ML-causal) The default playbook above is **AER-style applied econometrics** — the AEA convention: written-out estimating equation, identifying assumption table, design horse-race, full robustness gauntlet. The skill **also** ships two parallel sub-pipelines for the other two big causal-inference traditions, each reusing the same export stack (`sp.regtable / sp.collect / sp.paper_tables`) and result objects: | Mode | Reader convention | Identification stack | Reporting stack | Jump to | |---|---|---|---|---| | **Default — Applied Econ (AER / QJE / AEJ)** | "Show the equation + identifying assumption + design horse-race; controls visible; clustered SE" | DID / IV / RD / SCM / matching / `feols` HDFE | AER house-style multi-column `regtable` + 8-section paper layout | §−1 → §8 (entire playbook above) |