causal-design

name	causal-design
description	Design or audit the identification strategy for an observational study. Use when the task concerns estimands, causal assumptions, threats to identification, or defensible research design rather than model implementation.
allowed-tools	Read, Write, Edit, Glob, Grep, AskUserQuestion, Task
argument-hint	[project-path or tex-file] [--mode design|audit]
agent-dependencies	["domain-reviewer"]
skill-dependencies	["data-analysis","experiment-design","proofread"]

Causal Design

Design and audit identification strategies for observational causal inference.

Output Path

Per rules/review-artefact-routing.md (auto-loads in research projects (path-scoped to paper-*/ and paper/)):

• Source slug: causal-design
• Write reports to: reviews/<scope>/causal-design/<YYYY-MM-DD-HHMM>.md inside the project, where <scope> is the paper slug (e.g. paper-philtech) for paper-level audits or _project for project-level reviews. Path is relative to the research project root, not the Task-Management repo.
• Never at project root (./CRITIC-REPORT.md-style filenames are forbidden — pre-rule layout).
• Idempotency: if today's timestamp exists, append a same-day descriptor to the path base ({date}-HHMM-revision.md, {date}-HHMM-r2.md, {date}-HHMM-pre-submission.md) — never overwrite.
• Index update: if reviews/INDEX.md exists, write a one-line entry under "Latest per source" pointing at the new file. Otherwise review-recap will rebuild the index next time it runs.
• Infrastructure repos (Task-Management, atlas-workspace, etc.): this section does not apply — the path-scoped rule won't load there.

Modes

Mode	What it does	Entry point
Design	Interview-driven strategy selection and memo production	"Design my causal strategy" / "What identification can I use?"
Audit	4-phase causal inference check on existing paper/scripts	"Check my identification" / "Audit my econometrics"

Default: Design. If the user points to an existing paper or estimation script, auto-select Audit mode.

When to Use

• Choosing an identification strategy for an observational study
• Stress-testing whether an existing strategy is credible
• Verifying that code implements the claimed identification design
• Mapping causal claims to their identifying assumptions

When NOT to Use

• Experimental design (RCTs, surveys, factorial) -- use experiment-design
• Running the analysis or generating results -- use data-analysis
• Literature search or citation gathering -- use an installed literature workflow or scholarly search
• Proofreading or compiling the paper -- use proofread, latex

Shared References

• Method probing questions: shared/method-probing-questions.md — ask before running any analysis (DiD, IV, RDD sections)
• Validation tiers: shared/validation-tiers.md — declare tier before designing strategy
• Escalation protocol: shared/escalation-protocol.md — escalate when identification is vague or unsound

---

Mode: Design

Phase 1: Interview

Before opening the interview, confirm the project's validation tier per shared/validation-tiers.md — Exploratory designs warrant lighter identification stress-testing than Publication-ready ones. Use shared/method-probing-questions.md as the interview backbone; the prompts below adapt those probes to causal identification specifically.

Conduct a structured interview to understand the research setting. Ask these questions (adapt to what the user has already shared):

1. Causal question: What causal effect are you trying to estimate? What is the treatment? What is the outcome?
2. Variation: What source of variation in treatment do you exploit? Is it natural, policy-driven, institutional?
3. Confounders: What are the main threats to identification? What unobservables worry you?
4. Data structure: Panel, cross-section, or repeated cross-section? What units and time periods?
5. Institutional context: Any thresholds, cutoffs, rollout dates, or instruments available?
6. Prior literature: What identification strategies have others used for similar questions?

Do not proceed until the causal question and data structure are clear.

Phase 2: Strategy Selection

Read references/design-decision-tree.md and walk through the decision tree with the user's answers:

• Match the research setting to the strongest available strategy
• If multiple strategies are viable, rank them by credibility and discuss trade-offs
• If the setting does not support any strong strategy, say so explicitly -- do not force a weak design

Phase 3: Strategy Memo

Write a strategy memo using references/strategy-memo-template.md. Save to docs/causal-strategy.md (or project-appropriate location).

The memo must specify:

1. Estimand -- the exact causal parameter being estimated, in formal notation
2. Identification strategy -- how variation is generated and why it is exogenous
3. Key assumptions -- each one stated, with a defence or test plan
4. Threats and mitigations -- what could go wrong and how to address it
5. Diagnostics plan -- which tests to run before trusting the estimates
6. Robustness checks -- pre-committed alternative specifications
7. Alternative strategies considered -- why they were rejected

This memo is what data-analysis Phase 3 checks for before allowing estimation. It locks the research design per the design-before-results rule.

Phase 4: Adversarial Review

The reviewer follows shared/escalation-protocol.md — when identification is vague or assumptions are hand-waved, the reviewer escalates rather than accommodating.

Delegate an adversarial review to the domain-reviewer agent. Read references/causal-audit-prompt.md and pass it as the prompt to the fresh-context sub-agent mechanism:

Launch the domain-reviewer agent with this prompt:
"You are reviewing a causal identification strategy memo. [Insert contents of causal-audit-prompt.md, customised with the specific strategy chosen]. The memo is at [path]. Focus exclusively on identification credibility."

The agent will produce a report at reviews/<scope>/domain-reviewer/<YYYY-MM-DD-HHMM>.md in the project, where <scope> is the paper slug or _project.

Phase 5: Iterate

Present the domain-reviewer's findings to the user. For each issue flagged:

• Discuss whether it is a genuine threat or can be addressed
• Update the strategy memo if the design changes
• If the strategy is fundamentally flawed, return to Phase 2

---

Mode: Audit

Phase 1: Extract Claims

Read the paper (.tex files) and/or estimation scripts to extract every causal claim:

• What effects does the paper claim to estimate?
• What language is used? ("causal", "effect of", "impact of", "leads to")
• Are claims hedged appropriately or overstated?

Produce a numbered list of claims with their locations (file:line).

Phase 2: Map Estimands to Identification

For each causal claim, determine:

Claim	Estimand	Strategy	Key Assumption	Stated?	Defended?
...	...	...	...	Yes/No	Yes/No

Flag any claim where:

• The estimand is undefined or vague
• The identification strategy is not stated
• Key assumptions are not listed or defended
• The strategy does not match the claim (e.g., claiming ATE but estimating LATE)

Phase 3: Assumption Diagnostics

For each identification strategy found, check whether the required diagnostics are present and passing:

DiD / Event Study:

• Pre-treatment parallel trends test (visual + formal)
• Staggered treatment handling (TWFE bias check, Callaway-Sant'Anna or Sun-Abraham if staggered)
• Anticipation effects check
• Treatment effect heterogeneity assessment

IV:

• First-stage F-statistic reported (> 10 for Stock-Yogo, > 104.7 for modern thresholds)
• Exclusion restriction argument (quality of narrative)
• Monotonicity discussion
• Over-identification test (if multiple instruments)
• Reduced form reported

RDD:

• McCrary density test (no bunching at cutoff)
• Bandwidth sensitivity (MSE-optimal + alternatives)
• Covariate balance at the cutoff
• Donut hole specification
• Placebo cutoffs

Synthetic Control:

• Pre-treatment fit quality (RMSPE)
• Donor pool selection justification
• Placebo tests (in-space, in-time)
• Leave-one-out robustness

Event Study:

• Pre-event coefficients jointly zero
• Dynamic treatment effects plotted
• Clean control group definition
• Anticipation effects addressed

Phase 4: Code-Design Alignment

If estimation code exists, verify it implements the claimed design:

• Does the regression specification match the paper's equations?
• Are standard errors computed correctly for the design? (clustered at the right level, heteroskedasticity-robust)
• Are the treatment and control groups defined as claimed?
• Are the diagnostics actually run, not just mentioned?
• Do robustness checks exist in code, or only in the text?

Audit Report

Produce an audit report at reviews/<scope>/causal-design/<YYYY-MM-DD-HHMM>.md (where <scope> is the paper slug or _project) with:

# Causal Audit Report

**Document:** [filename]
**Date:** YYYY-MM-DD
**Mode:** Audit

## Claims Inventory

[Numbered list of causal claims with locations]

## Estimand-Identification Map

[Table from Phase 2]

## Diagnostics Assessment

| Strategy | Diagnostic | Present? | Passing? | Notes |
|----------|-----------|----------|----------|-------|
| ... | ... | ... | ... | ... |

## Code-Design Alignment

[Phase 4 findings, or "N/A -- no code found"]

## Critical Issues

[List of issues that threaten identification credibility]

## Recommendations

[Ordered list of fixes, from most to least important]

---

Cross-References

Resource	When read
references/design-decision-tree.md	Design Phase 2 (strategy selection)
references/strategy-memo-template.md	Design Phase 3 (memo output)
references/causal-audit-prompt.md	Design Phase 4 (agent delegation prompt)
design-before-results rule	Both modes enforce this
domain-reviewer agent	Design Phase 4 (adversarial review)
data-analysis skill	Consumes the strategy memo
experiment-design skill	For experimental (not observational) designs
experiment-design/references/identification-strategies.md	Quick-reference for strategies (shared knowledge)