science-critique-approach

name	science-critique-approach
description	Critically review a causal DAG inquiry for missing confounders, identifiability issues, and structural problems. Use when the user wants to review a causal model, check confounders, or validate causal reasoning.

Critique a Causal Approach

Converted from Claude command /science:critique-approach.

Science Codex Command Preamble

Before executing any research command:

1. Resolve project profile: Read science.yaml and identify the project's profile. Use the canonical layout for that profile:

   • research → doc/, specs/, tasks/, knowledge/, papers/, models/, data/, code/
   • software → doc/, specs/, tasks/, knowledge/, plus native implementation roots such as src/ and tests/

2. Load role prompt: .ai/prompts/<role>.md if present, else references/role-prompts/<role>.md.

3. Load the science-research-methodology and science-scientific-writing Codex skills. If native skill loading is unavailable, use codex-skills/INDEX.md to map canonical Science skill names to generated skill files and source paths.

4. Read specs/research-question.md for project context when it exists.

5. Load project aspects: Read aspects from science.yaml (default: empty list). For each declared aspect, resolve the aspect file in this order:

   1. aspects/<name>/<name>.md — canonical Science aspects
   2. .ai/aspects/<name>.md — project-local aspect override or addition

   If neither path exists (the project declares an aspect that isn't shipped with Science and has no project-local definition), do not block: log a single line like aspect "<name>" declared in science.yaml but no definition found — proceeding without it and continue. Suggest the user either (a) drop the aspect from science.yaml, (b) author it under .ai/aspects/<name>.md, or (c) align the name with one shipped under aspects/.

   When executing command steps, incorporate the additional sections, guidance, and signal categories from loaded aspects. Aspect-contributed sections are whole sections inserted at the placement indicated in each aspect file.

6. Check for missing aspects: Scan for structural signals that suggest aspects the project could benefit from but hasn't declared:

   Signal	Suggests
   Files in specs/hypotheses/	hypothesis-testing
   Files in models/ (.dot, .json DAG files)	causal-modeling
   Workflow files, notebooks, or benchmark scripts in code/	computational-analysis
   Package manifests (pyproject.toml, package.json, Cargo.toml) at project root with project source code (not just tool dependencies)	software-development

   If a signal is detected and the corresponding aspect is not in the aspects list, briefly note it to the user before proceeding:

   "This project has [signal] but the [aspect] aspect isn't enabled. This would add [brief description of what the aspect contributes]. Want me to add it to science.yaml?"

   If the user agrees, add the aspect to science.yaml and load the aspect file before continuing. If they decline, proceed without it.

   Only check once per command invocation — do not re-prompt for the same aspect if the user has previously declined it in this session.

7. Resolve templates: When a command says "Read .ai/templates/<name>.md", check the project's .ai/templates/ directory first. If not found, read from templates/<name>.md. If neither exists, warn the user and proceed without a template — the command's Writing section provides sufficient structure.

8. Resolve science CLI invocation: When a command says to run science, prefer the project-local install path: uv run science <command>. This assumes the root pyproject.toml includes science as a dev dependency installed via uv add --dev --editable "$SCIENCE_TOOL_PATH" (the distribution is science; the entry point it installs is science). If that fails (no root pyproject.toml or science not in dependencies), fall back to: uv run --with <science-plugin-root>/science science <command>

Prerequisite: Read docs/proposition-and-evidence-model.md, docs/specs/2026-03-01-knowledge-graph-design.md, references/dag-two-axis-evidence-model.md, and docs/specs/2026-03-07-phase4b-causal-dag-design.md for ontology and causal modeling reference.

Overview

This command provides a critical, adversarial review of a causal inquiry. The agent operates as a discussant — looking for weaknesses, missing confounders, and threats to validity. This is not a rubber stamp; the goal is to surface problems before the researcher invests in analysis.

Tool invocation

All science commands below use this pattern:

uv run science <command>

For brevity, examples write just science <command> — always expand to uv run science <command> when executing. See command-preamble step 8 for fallback.

Rules

• MUST operate in discussant/critic role — be skeptical, not supportive
• MUST run structural validation first (inquiry validate)
• MUST challenge every causal edge for reverse causation and alternative explanations
• MUST check for missing confounders systematically
• MUST assess identifiability (can the target effect be estimated from observables?)
• MUST write review report to doc/inquiries/<slug>-critique.md
• MUST NOT dismiss concerns as "minor" or "unlikely" — surface all issues
• SHOULD reference references/dag-two-axis-evidence-model.md for pitfall patterns

Workflow

Step 1: Load the inquiry

The user provides an inquiry slug (e.g., science-critique-approach my-dag).

science inquiry show "<slug>" --format json
science inquiry validate "<slug>" --format json

Verify:

• The inquiry exists and is type causal
• Structural validation passes (or note failures)
• Treatment and outcome are set

Step 2: Graph-theoretic analysis

Export to pgmpy and analyze:

science inquiry export-pgmpy "<slug>" --output /tmp/dag_analysis.py

Read the generated script. Identify:

• Adjustment sets: What should be conditioned on to identify the causal effect?
• Testable implications: What conditional independencies does the DAG predict?
• Identifiability: Is the target effect (treatment → outcome) identifiable given observed variables?

If there are latent (unobserved) variables, note that the back-door criterion may not be satisfied.

Step 3: Challenge each causal edge

For every scic:causes edge in the DAG, ask:

1. Reverse causation: "Could B actually cause A instead of A causing B?"
2. Mediation: "Is there an unmeasured mediator M such that A → M → B? Does this matter for identification?"
3. Selection bias: "Could the study design or data collection process condition on a descendant of this edge?"
4. Temporal ordering: "Does A occur before B? Is there evidence for temporal precedence?"
5. Evidence quality: "What is the confidence and source for this edge? Is it well-established or speculative?"
6. Measurement path: "Is this edge supported by a direct observation, or by a proxy that should have measurement_model and proxy_directness called out explicitly?"

Step 4: Check for missing confounders

For every pair of variables (X, Y) connected by a causal edge:

• "What else could affect both X and Y?"
• "Are there environmental, genetic, or methodological factors that influence both?"
• "Would a domain expert immediately identify a missing common cause?"

For each identified missing confounder:

• Note it in the review
• Suggest adding it to the DAG
• Assess whether it's observable or latent

Also check whether supposedly independent support lines really collapse into one independence_group.

Step 5: Check for structural problems

• Collider bias: Is the user conditioning on (adjusting for) a collider? This creates spurious associations.
• M-bias: Is there a path where adjusting for a seemingly innocuous variable opens a non-causal path?
• Overadjustment: Is the user adjusting for a mediator on the causal path? This blocks the effect being estimated.
• Circular reasoning: Does evidence for edge A → B rely on the assumption that A → B?

Step 6: Assess overall validity

Rate each dimension:

Dimension	Assessment
Completeness	Are all plausible confounders included?
Identifiability	Can the target effect be estimated from observables?
Evidence quality	Are causal edges well-supported by literature/data?
Structural validity	No collider bias, M-bias, or overadjustment?
Temporal coherence	Does the causal ordering make temporal sense?
Sensitivity	How robust are conclusions to assumption violations?

Step 6b: Sensitivity analysis

For each key assumption or causal edge identified in steps 3-5, assess sensitivity:

1. What if this assumption is violated?

   • State the assumption explicitly
   • Describe how conclusions change if it's wrong
   • Rate impact: high (conclusions reverse) / moderate (conclusions weaken) / low (conclusions robust)

2. What if this relationship doesn't hold or is reversed?

   • For causal edges: what if A doesn't cause B, or B causes A?
   • For conceptual models: what if this link is spurious or mediated?

3. Unmeasured variables

   • For causal DAGs: for each critical path, what unmeasured confounder could explain the relationship?
   • For conceptual models: what hidden mediator or moderator could alter the relationship?

4. Robustness

   • What's the minimum effect size that would survive the identified threats?
   • How sensitive are conclusions to parameter choices?

5. Boundary conditions

   • Under what conditions does the model break down entirely?
   • Are there population, temporal, or contextual limits to applicability?

For mixed empirical / mechanistic arguments, ask whether the inquiry is silently promoting an empirical regularity into a mechanistic narrative without enough lower-layer support.

Include a sensitivity summary table in the review report:

Assumption	If Violated	Impact	Robustness
		high/moderate/low

Step 7: Write review report

Save to doc/inquiries/<slug>-critique.md:

# Causal DAG Critique: <label>

**Inquiry:** <slug>
**Treatment:** <treatment>
**Outcome:** <outcome>
**Reviewed:** <date>

## Structural Validation
<inquiry validate results>

## Identifiability Assessment
<adjustment sets, back-door criterion>

## Edge-by-Edge Review
<for each edge: evidence, challenges, confidence>

## Missing Confounders
<identified gaps>

## Structural Issues
<collider bias, M-bias, overadjustment>

## Sensitivity Analysis
<sensitivity summary table and key findings>

## Overall Assessment
<summary table with pass/warn/fail per dimension>

## Recommendations
<specific actionable items>

Update the inquiry status to critiqued.

Note: this status indicates the inquiry has been through critical review, NOT that it passed. The review report documents what was found.

Step 8: Present findings

Summarize the key findings to the user:

• Critical issues (must fix before analysis)
• Important concerns (should investigate)
• Minor notes (awareness items)
• Recommended next steps

If science health --format json surfaces proxy gaps, unsupported mechanistic narratives, or rival-model packet issues related to this inquiry, mention them explicitly in the critique summary.

Process Reflection

Reflect on the template and workflow used above.

If you have feedback (friction, gaps, suggestions, or things that worked well), report each item via:

science feedback add \
  --target "command:critique-approach" \
  --category <friction|gap|guidance|suggestion|positive> \
  --summary "<one-line summary>" \
  --detail "<optional prose>"

Guidelines:

• One entry per distinct issue (not one big dump)
• If the same issue has occurred before, the tool will detect it and increment recurrence automatically
• Skip if everything worked smoothly — no feedback is valid feedback
• For template-specific issues, use --target "template:<name>" instead