phase4-exit-check

name

phase4-exit-check

description

Exit rubric for Phase 4 (Ablation). Load after every variant in the ablation plan's execution set has a finalized result_{n}.md + analysis_{n}.md, and the final story-coherence-advisor Context-C checkpoint has produced a story_check_{k}.md. Returns PASS or NEEDS-REWORK with a routing-advisory table (proceed to Phase 5 / 6 vs. rollback target). Evaluates structural completeness of the ablation sweep's analyses and narrative coherence — NOT the quality of the ablations' verdicts, NOT the routing decision itself (that belongs to the Main Agent), NOT the paper framing (that is Phase 5 / 6). Does NOT re-diagnose any variant's analysis.

Phase 4 Exit Check

Read every phase_4_ablation/analysis/analysis_{n}.md in the directory, their matched phase_4_ablation/results/result_{n}.md files, the latest phase_4_ablation/plans/ablation_plan_v{n}.md, every phase_4_ablation/specs/ablation_spec_{handle}_v{r}.md referenced by an analysis, the full set of phase_3_main_experiment/story_checks/story_check_{k}.md (Phase 3 and Phase 4 share this directory), the confirmed phase_3_main_experiment/specs/framework_v{m}.md, and phase_0_ideation/ideas/IDEA.md. Apply the six criteria below in order. Each criterion is binary: pass or fail.

A single failing criterion means the overall result is NEEDS-REWORK.

This check is structural. It asks: has Phase 4 produced handoff-ready attribution evidence and a coherent narrative? It does not re- diagnose variants, re-dispute the Analyst's verdicts, re-grade the ablation plan, decide whether to proceed to Phase 5, or decide whether Phase 4 needs more variants. The routing signal — the aggregate rollback depth + the latest story_check verdict — is surfaced in the output for the Main Agent to act on, but is not itself a criterion.

---

Criterion 1 — Execution set is complete and every variant is finalized

Pass if the set of variants the Main Agent selected as the Phase 4 execution set (recorded in the session log at Sub-process 1 Step 2 of phase4-ablation) has, one-to-one, a set of launches matching each variant's Cell scope declared in the plan, and every launch's result_{n}.md is Status: COMPLETE or Status: FAILED — no RUNNING entries remain.

Fail if any selected variant has a missing launch (one or more of its Cell-scope cells has no result_{n}.md with the variant handle in its Slice field), OR any result_{n}.md is still RUNNING, OR there is a result_{n}.md whose Slice's variant handle does not match any variant in the execution set (stray runs — the Main Agent should resolve before exit).

To check: parse ablation_plan_v{n}.md §Variants for the execution set's Cell scope per handle; parse each result_{n}.md's §Slice for its (variant handle, cell) pair; cross-reference the product against the finalized set.

Note: variants that were explicitly paused per Sub-process 4 Step 4's option (a/b/c/d) during the sweep are NOT part of the execution set for this criterion — they were reclassified as deferred. The check is against the execution set as it stood when the sweep started PLUS any in-sweep additions, MINUS any in-sweep pauses. The session log is the authoritative record.

---

Criterion 2 — Every launch has a matched analysis

Pass if for every finalized phase_4_ablation/results/result_{n}.md there exists a phase_4_ablation/analysis/analysis_{n}.md with matching {n}.

Fail if any finalized result has no analysis, OR any analysis's {n} does not match a result.

---

Criterion 3 — Every analysis is structurally complete

Pass if every phase_4_ablation/analysis/analysis_{n}.md satisfies the same structural checks as phase3-exit-check Criterion 3 (in turn, phase2-exit-check Criteria 2, 3, 4, 5), with one Phase-4-specific addition:

• All required sections present (Header, Synthesis with four lines, Claim-Result Comparison, Findings or explicit "No findings", Context for the diagnosis, Evidence not available).
• Every Core Claim from IDEA.md appears as a row in the Claim- Result Comparison. Phase 4 analyses typically mark most Core Claim rows UNMEASURABLE (the variant does not defend Core Claims; that is Phase 3's job) and add one row for the variant's Ablation attribution prediction with Verdict SUPPORTED / NOT SUPPORTED / UNMEASURABLE.
• Every finding carries Observation, Evidence (with citation), Why-this-category, and Scope.
• Severity is consistent with the Ablation attribution prediction verdict — an attribution prediction marked NOT SUPPORTED has at least one finding naming the redirect (the story_check at Context C will reframe from there; this check only verifies the finding exists).

Fail if any single analysis fails any of those sub-checks. List which analysis and which sub-check failed.

---

Criterion 4 — At least one Phase 4 story check exists

Pass if at least one phase_3_main_experiment/story_checks/story_check_{k}.md has Invocation context = C (Phase 4 ablation integration), AND the latest story_check (story_check_{K}.md with the highest {K}) was written after the latest Phase 4 analysis (i.e., its Analyses read field references the latest phase_4_ablation/analysis/analysis_{N}.md). This proves the Story Advisor saw the final state of the ablation sweep.

Fail if no Context-C story_check exists, OR the latest story_check predates the latest Phase 4 analysis (stale attribution reading).

---

Criterion 5 — Latest story check verdict is COHERENT

Pass if the latest story_check_{K}.md has Verdict COHERENT.

Fail if Verdict is FRACTURED.

Unlike phase3-exit-check, Phase 4 does not allow FRACTURED-scoped-to-later-phase handoff. Phase 4 is the phase where attribution fractures get resolved — either by the ablation evidence sharpening a bullet, or by an Ablation Designer Case B revision adding a new variant, or by the Main Agent retiring a bullet and accepting narrower scope. A Phase 4 exit with FRACTURED means one of those resolutions did not happen and the paper would be written on a broken attribution story.

The remedy for a FRACTURED latest story_check at this gate:

• Method-ahead-of-story direction: the ablation evidence credited a component the story did not name, or the named component turned out not to be the driver. Route to Story Advisor Context C with a brief asking it to update bullets (Case B of story-coherence-advisor would be Context C with a revision scope). Resume Phase 4 as Case C with no new variants, just a fresh story_check, if the user agrees the bullet update is the fix.
• Story-ahead-of-method direction: the story promised an attribution the ablation cannot demonstrate. Either (i) route to Ablation Designer Case B to add a new variant that would close the evidence gap, or (ii) route to the Story Advisor to retire the over-claimed bullet. Main Agent picks.

The surfacing of these options is in §Output below, not in this criterion — Criterion 5 is the binary gate.

---

Criterion 6 — Every planned variant in the execution set has an attribution verdict

Pass if: for every variant in the execution set, the matched Phase 4 analyses collectively produce at least one Ablation attribution prediction row with a verdict (SUPPORTED, NOT SUPPORTED, or UNMEASURABLE). If a variant's Cell scope spans multiple cells, a verdict per cell is acceptable — as long as at least one exists per variant handle.

Fail if any variant in the execution set has analyses but no Ablation attribution prediction row in any of them. This means the Analyst was invoked but did not evaluate the ablation's falsifiable prediction — a Phase-4-specific structural gap.

This is the Phase-4 analogue of phase3-exit-check Criterion 6 (aggregate severity consistent with claim verdicts). The focus shift from Core Claim verdicts (Phase 3) to Ablation prediction verdicts (Phase 4) reflects the phase's purpose: defend attribution, not Core Claims.

A variant whose attribution prediction is UNMEASURABLE is passing for this criterion but is surfaced under the routing advisory below — UNMEASURABLE typically means the variant's code or data did not produce the metric needed to falsify the prediction, which is a Low-depth routing signal (fix the run, not the method).

---

Output

After evaluating all six criteria, output exactly this block:

## Phase 4 Exit Check

**Result: PASS / NEEDS-REWORK**

| Criterion | Result | Note |
|-----------|--------|------|
| 1. Execution set complete and finalized | PASS / FAIL | [counts: selected variants / finalized launches; list any running or stray] |
| 2. Every launch has matched analysis | PASS / FAIL | [list any result without analysis, or vice versa] |
| 3. Every analysis is structurally complete | PASS / FAIL | [list analyses that failed any sub-check; specify which sub-check] |
| 4. At least one Phase 4 (Context C) story check exists | PASS / FAIL | [latest story_check index and date; flag if stale] |
| 5. Latest story check verdict is COHERENT | PASS / FAIL | [latest Verdict; if FRACTURED, summarize the Most-important-reconciliation and its direction] |
| 6. Every variant has an attribution-prediction verdict | PASS / FAIL | [list any variant handle with no prediction verdict in its analyses] |

**Failing criteria:** [list by number, or "none — all criteria passed"]

**Aggregate rollback depth (max across all Phase 4 analyses):** None / Low / Medium / High / Critical

**Per-variant attribution snapshot:**

| Variant | Priority | Cells run | Attribution prediction verdict(s) | Notes |
|---------|----------|-----------|-----------------------------------|-------|
| {handle} | Critical / High / Standard / Optional | [list] | SUPPORTED / NOT SUPPORTED / UNMEASURABLE (per cell or collapsed) | [one line — e.g., "effect redirected to module X per analysis_7"] |

[One row per variant in the execution set.]

**Latest story check:** phase_3_main_experiment/story_checks/story_check_{K}.md
**Latest story check verdict:** COHERENT | FRACTURED
**Most important reconciliation (if FRACTURED):** [one line from story_check_{K}.md]

**Contribution bullets (from latest story_check):**

[List the 2–4 bullets verbatim from the latest story_check's
Contribution Bullets section. Each includes its handle,
falsifiable statement, attribution clause, and confirmation flag
(provisional / confirmed by ablation).]

**Routing signal for the Main Agent (advisory, NOT a decision):**

| Depth | Typical Main-Agent target |
|-------|---------------------------|
| None (and story COHERENT) | Proceed to Phase 5 (Review) / Phase 6 (Writing). Their skills are not yet built. |
| Low | `phase2-implementation`-style Case B scoped to `phase_4_ablation/code/{handle}/` for the failing variant's code bug. Resume `phase4-ablation` as Case C for that variant's affected launches. |
| Medium | Revisit the variant's spec via `phase4-ablation` Sub-process 2 (Architect ↔ Confirmer on `ablation_spec_{handle}_v{r+1}.md`). Resume as Case C. |
| High | Do **not** treat as Main-Agent routing — this is usually the ablation's intended signal (a bullet's attribution was wrong). The Story Advisor Context C output already captures the redirect; if Criterion 5 passed, this is a PASS even with High depth on some variants. If Criterion 5 failed on top of High depth, the Main Agent chooses between Ablation Designer Case B (new variant) or Story Advisor revision to retire / reframe the bullet. |
| Critical | **Phase 3 re-entry signal.** The variant's evidence contradicts Phase 3's Main Results, not just the attribution. Halt. User-confirmed routing only — options: (i) `phase3-main-experiment` Case C to rerun affected Phase 3 cells with a corrected spec, (ii) Refiner round on `IDEA.md` to reframe the Core Claim, (iii) Phase 0 re-ideation via `loop1-ideation-pipeline` Case C. Do not auto-route. |

**Narrative routing overlay:** If Criterion 5 failed (latest story
check is FRACTURED), the Main Agent must resolve the fracture
before Phase 5. The Story Advisor's "Most important reconciliation"
names the target — typically Ablation Designer Case B (new variant)
or Story Advisor revision (retire / reframe bullet). Aggregate
depth alone is not sufficient to proceed past this gate.

**Findings snapshot (for routing context):** [One line per analysis
with non-None rollback depth: "analysis_{n}.md: {variant handle}
{category} {severity}". Or "No non-None-depth analyses — ablation
sweep is clean."]

**Open known risks (from context across analyses):** [Union of
Accepted Additions and engaged Risk Flags carried forward across
the Phase 4 analyses. Or "None carried forward."]

**Deferred variants (not in execution set):** [Variants from the
plan with priority Standard or Optional that were not selected
into the execution set, or variants paused mid-sweep. One line
each: handle + priority + reason for deferral. These are NOT
blockers — surfaced so the Main Agent can decide whether to
escalate any to Loop 4 (review) or leave as future work.]

---

What this skill does NOT do

• Does not re-diagnose any variant. Each analysis's categorization and severity are authoritative. Criterion 3 checks structural completeness per variant; Criterion 6 checks the attribution-prediction verdict is present. Neither second- guesses the Analyst's categorization.
• Does not decide whether to add new variants. The plan's execution set + any in-sweep adjustments define the scope. Missing coverage of a contribution bullet is a plan-revision issue (Ablation Designer Case B), not an exit-check intervention.
• Does not auto-invoke Phase 5 / 6, Loop 3, or Phase 0. The Routing signal table is advisory. The Main Agent decides — and for High (with Criterion 5 failing) and Critical, user confirmation is required.
• Does not dispute the Story Advisor's Verdict. Criterion 5 reads the verdict structurally; if it reads FRACTURED, the remedy is to resolve the fracture upstream (Ablation Designer Case B or Story Advisor revision), not to override the Verdict here.
• Does not compare across Phase 4 plans. The plan of record is whatever the latest ablation_plan_v{n}.md declares; stale earlier-plan variants are not evaluated here unless they are still in the latest plan's execution set.
• Does not evaluate paper-writability. That is Phase 5 / 6's concern. The contribution bullets listed in §Output are for continuity, not approval.

---

Phase 4 → Phase 5 / 6 routing — the Main Agent's reading

Once this skill returns PASS with Aggregate rollback depth None (or High that Criterion 5 has already absorbed) and a COHERENT latest story check, the Main Agent's job (outside this skill) is to:

1. Write PHASE_4_SUMMARY.md at phase_4_ablation/PHASE_4_SUMMARY.md — one page: plan version, variants executed, per-variant attribution verdict table, final contribution bullets (verbatim from the latest story_check), accepted deviations, deferred variants.
2. Invoke phase5-review (not yet built at time of writing) with inputs: framework_v{m}.md, the latest story_check, Phase 3 + Phase 4 analyses, PHASE_3_SUMMARY.md, and PHASE_4_SUMMARY.md. Review should use the final contribution bullets as the review rubric's starting point.
3. Record the transition in _logs/decision_log.md — Phase 4 exit is a governance moment (attribution is locked).

For non-None depths that Criterion 5 did not absorb, the routing entry points are:

• Low: phase2-implementation Case B scoped to phase_4_ablation/code/{handle}/; resume phase4-ablation as Case C for the affected variant's launches.
• Medium: phase4-ablation Sub-process 2 (Architect ↔ Confirmer) on the variant's spec; resume as Case C.
• Critical: phase3-main-experiment Case C or deeper (Refiner / Phase 0), user-confirmed only.

For FRACTURED-verdict handoffs specifically: the resolution is almost always (a) Ablation Designer Case B for a new variant, or (b) a Story Advisor revision pass to retire / reframe the offending bullet. Both routes re-enter phase4-ablation as Case B (new plan revision) or remain in the current plan with a story-only reconciliation. The Main Agent picks; this skill only surfaces the fracture and its Most-important-reconciliation line.