| name | jpam-data-analysis |
| description | Use when running and reporting the estimation for a Journal of Policy Analysis and Management (JPAM) manuscript — program-evaluation estimates plus the cost-benefit and distributional analysis JPAM expects, with robustness, heterogeneity, and honest uncertainty. Guides analysis norms; it does not replace the identification design. |
Data Analysis: Estimation, Cost-Benefit & Distribution (jpam-data-analysis)
JPAM analysis has two layers most field-journal papers skip: beyond the causal estimate, reviewers
expect attention to cost-benefit and distributional consequences — who gains, who pays, and is
it worth it? The estimate answers "does the policy work"; the cost-benefit and distributional work
answers "should we do it, and for whom." Both must be reported honestly, with uncertainty carried
through.
When to trigger
- Producing the main estimates and the robustness/heterogeneity suite
- Adding (or being asked to add) cost-benefit or distributional analysis
- A reviewer questioned standard errors, robustness, or the policy-relevance of the magnitude
- Translating an effect size into a decision-relevant quantity (per-dollar, per-recipient, MVPF)
Estimation norms
- Report effects in policy-relevant units — percentage points, dollars, per-recipient, per-dollar-
spent — not just standardized coefficients.
- Robustness as a coherent suite, not a coefficient dump: alternative specifications, samples,
bandwidths/estimators, and a placebo where the design allows. Show the result is not knife-edge.
- Theory-driven heterogeneity (from
jpam-theory-building), pre-specified where possible; report
which subgroup tests are primary and adjust for multiplicity.
- Honest uncertainty — confidence intervals, not just stars; discuss precision when a null is
policy-relevant ("we can rule out effects larger than X").
Cost-benefit & distributional analysis (JPAM premium)
- Cost-benefit: monetize benefits and costs on a common basis, state the discount rate and the
perspective (government budget vs. society), and run sensitivity to key assumptions. Where suitable,
report the Marginal Value of Public Funds (MVPF) or benefit-cost ratio.
- Distributional: show who gains and who bears the cost (by income, race, region, recipient vs.
taxpayer). A positive average effect with regressive incidence is a different policy story — say so.
- Fiscal externalities: account for downstream budget effects (e.g., reduced transfers, increased
tax revenue) where the literature does.
- Carry uncertainty through to the cost-benefit conclusion — do not present a point ratio as if the
estimate were certain.
Execution bridge (StatsPAI / Stata MCP)
Run the battery, don't just enumerate it. Full map:
execution-with-mcp. JPAM is policy analysis — program evaluation is the core; DiD/IV/RDD and the policy-relevant magnitude are decisive.
- Many outcomes / specifications:
romano_wolf (step-down FWER) or benjamini_hochberg.
- OVB sensitivity:
oster_delta / sensemakr.
- Inference:
wild_cluster_bootstrap (few clusters), twoway_cluster / conley.
- Re-fit off one handle:
audit_result(result_id) lists missing checks + the exact
suggest_function for each.
- Exhibits:
etable / did_summary_to_latex from the handle — no retyped numbers.
Decisive checks in the body, exhaustive battery in the appendix.
JF execution walkthrough.
Checklist
Anti-patterns
- Reporting only standardized effects a policymaker cannot act on
- A robustness "kitchen sink" that never states which checks address which threat
- Cost-benefit with a hidden discount rate or perspective, and no sensitivity analysis
- A flattering average effect that hides regressive distribution
- Presenting a benefit-cost ratio as certain when the underlying estimate has a wide CI
- Post hoc subgroup hunting presented as confirmatory
Calibration anchors (hedged)
- The cost-benefit and distributional layers are what most distinguish a JPAM analysis from a field-
economics paper — budget time for them, do not bolt them on at the end.
- An MVPF or benefit-cost ratio is only as credible as the estimate it rests on; report its sensitivity
to the effect-size CI and to the discount rate, not a single point.
- A precisely estimated null can be a publishable JPAM result if it rules out a policy-relevant effect
— frame it as "we can rule out effects larger than X," not "no effect."
Worked micro-example (illustrative)
An evaluation finds a job-training program raises quarterly earnings by $420 (95% CI $120–$720). The
JPAM analysis does not stop there: it converts this to a benefit-cost ratio (lifetime earnings gain
vs. per-participant cost) under a stated discount rate, runs sensitivity across the CI and discount
rate, shows the gain is concentrated among longer-tenured entrants (theory-driven heterogeneity), and
notes the program is net-positive to the government budget only above a take-up threshold. The policy
story is the package, not the $420. (Numbers illustrative.)
Output format
【Main estimate】effect in policy-relevant units (+ CI)
【Robustness】checks mapped to specific threats
【Heterogeneity】theory-driven subgroups + multiplicity handling
【Cost-benefit】perspective, discount rate, MVPF/BCR, sensitivity
【Distribution】who gains / who pays
【Next】jpam-tables-figures
Supplementary resources
