| name | jhe-robustness |
| description | Use when a Journal of Health Economics (JHE) result must be shown stable to specification, sample, inference, and mechanism threats before submission or in response to referees. Organizes robustness by identifying threat; it does not establish the design (jhe-identification) or build exhibits. |
Robustness Strategy (jhe-robustness)
When to trigger
- The headline estimate may be sensitive to specification, sample window, or functional form
- Inference is suspect: few clusters (states), serial correlation, or multiple outcomes/subgroups
- A referee will ask whether the result is a coverage/take-up artifact rather than the claimed effect
- The mechanism story is asserted but not separated from competing explanations
Robustness the JHE way: organize by threat, not by appendix
A wall of starred alternative specifications persuades no one. JHE referees want each robustness check mapped to a specific threat to the health-economics claim, with the point estimate's stability — not its significance — as the object. Build the robustness section as a threat-response ledger: name the threat a health economist would raise, run the check that addresses it, and report whether the magnitude moves. The threats that recur at JHE are selection, concurrent policy, measurement of health/utilization, and inference with few policy clusters.
Threat-to-check ledger
| Threat to the claim | Check that addresses it |
|---|
| Residual selection into insurance/treatment | bounds (Lee/Manski/Oster); selection-on-observables-to-unobservables (Oster δ) |
| Concurrent reform contaminates the policy variation | placebo on ineligible group/period; leave-one-reform-out; pre-period falsification |
| Staggered-timing bias | heterogeneity-robust estimator (CS / SA / dCDH); honest-DID sensitivity |
| Health/utilization mismeasurement (claims coding, self-report) | alternative outcome definitions; administrative vs. survey cross-check; coding-change robustness |
| Functional form / sample window | log vs. level, trimming outliers (skewed health spending!), alternative bandwidths, donut RD |
| Few-cluster inference (states) | wild-cluster bootstrap; randomization inference; correct clustering level |
| Multiple outcomes/subgroups | MHT adjustment (Romano–Wolf / sharpened q-values); pre-specify the primary outcome |
| Mechanism is one of several | horse-race the channels; show the competing story predicts a pattern you do not see |
Sequencing
- Lead with the threat the editor/referee will raise first — usually selection or concurrent policy at JHE.
- Report stability, not stars. Show the point estimate across checks in one figure or compact table; if it moves, say so and explain.
- Right-size the spending/skew problem. Health expenditure is heavily right-skewed and zero-inflated; defend the estimator (two-part, GLM, IHS) rather than defaulting to OLS on a log.
- Treat inference as a first-class robustness item, not a footnote — few-state clustering is a classic JHE referee catch.
- Pre-register the primary outcome where multiple health outcomes invite cherry-picking.
- Show, do not assert, stability. A single figure plotting the point estimate and CI across every check is worth more than a paragraph claiming robustness; a referee can read it in seconds.
Execution bridge (StatsPAI / Stata MCP)
Run the battery, don't just enumerate it. Full map:
execution-with-mcp. JHE is health economics — insurance/program reforms and selection; foreground DiD/IV/RDD and selection corrections.
- Many outcomes / specifications:
romano_wolf (step-down FWER) or benjamini_hochberg.
- OVB sensitivity:
oster_delta / sensemakr.
- Inference:
wild_cluster_bootstrap (few clusters), twoway_cluster / conley.
- Re-fit off one handle:
audit_result(result_id) lists missing checks + the exact
suggest_function for each.
- Exhibits:
etable / did_summary_to_latex from the handle — no retyped numbers.
Decisive checks in the body, exhaustive battery in the appendix.
JF execution walkthrough.
Checklist
Anti-patterns
- A leave-out or alternative-sample check run but never reconciled when the estimate moves
- An appendix of starred specifications with no map from check to threat
- OLS on log spending with no handling of zeros or skew
- Clustering below the policy level, then claiming significance
- Running every subgroup and reporting the significant ones with no MHT adjustment
- "Results are robust" with no figure showing the point estimate holding
- Dodging the selection threat with more controls instead of a bound or design fix
The skewed-spending decision, made explicitly
Health spending and utilization are the journal's signature dependent variables, and they are right-skewed, zero-inflated, and heavy-tailed — the estimator choice is itself a robustness question a referee will press. Make it a deliberate, defended choice rather than a default:
- Many zeros + continuous positive part → two-part model (probit/logit for any use × GLM for the positive amount); report both parts.
- Skew without excess zeros → GLM with a log link (often gamma), which avoids retransformation bias that plagues OLS-on-log.
- Want to keep zeros and interpret in levels → IHS or Poisson/PPML, stating the elasticity interpretation honestly.
- Whatever you pick, show the result is not an artifact of the functional form by reporting at least one alternative, and never present OLS-on-log as if retransformation were free.
Worked vignette (illustrative)
A provider-payment paper shows intensity rises after a fee change; a referee suspects it is patient selection, not a true behavioral response. The JHE fix: hold the threat ledger explicit — (a) an Oster δ shows selection on unobservables would need to be 2× the observables to overturn the result; (b) a placebo on a fee-unaffected service shows no jump; (c) the spending outcome is re-run with a two-part model given 30% zeros; (d) inference is wild-cluster bootstrapped over 41 providers. The point estimate holds across all four (say 6.2%, stable within ±0.8pp, illustrative). The mechanism — behavioral response, not selection — now survives.
Output format
【Journal】Journal of Health Economics
【Skill】jhe-robustness
【Primary threat】selection / concurrent-policy / staggered-bias / measurement / inference
【Threat→check ledger】[threat: check → estimate movement]
【Spending estimator】OLS / two-part / GLM / IHS — defended? [Y/N]
【Inference】clustering level + few-cluster correction
【MHT】adjusted across outcomes/subgroups? [Y/N]
【Verdict】estimate stable / moves (explained) / fragile
【Next skill】jhe-tables-figures
Handoff boundary
This skill stress-tests an already-identified estimate; it does not fix a broken design (that is jhe-identification) or present the results (that is jhe-tables-figures). If a robustness check reveals the estimate is not actually identified — it swings with the selection bound or fails the placebo — route back to jhe-identification rather than papering over it with more specifications. When the point estimate holds across the threat ledger, hand off to jhe-tables-figures to make the stability legible.
