loop2-exit-check

name

loop2-exit-check

description

Exit rubric for Loop 2 (stress testing). Load when the Main Agent needs to determine whether the current idea has survived adversarial scrutiny sufficiently to proceed to the Experimental Setup Module. Evaluates the latest Tester report against four criteria. Returns PASS or FAIL with specific failing criteria listed. Evaluates resilience to attack only — not development completeness (that was Loop 1's job).

Loop 2 Exit Check

Read the latest Tester report: find the highest-numbered tester_report_*.md in phase_0_ideation/stress_test/. Also read the current phase_0_ideation/ideas/IDEA.md for context, and the accompanying tester_literature_*.md with the same version number.

Apply the soundness pre-check first, then the four criteria below in order. Each criterion is binary: pass or fail. Do not score, weight, or partially credit them.

A single failing criterion means the overall result is FAIL.

---

Soundness pre-check — discount superficial paper-based attacks

The Tester is required to verify every paper cited as Evidence (Step 3.5 of the idea-pressure-tester agent). Before applying Criteria 1–4, audit the Tester report against the tester_literature_*.md verification blocks and discount any attack point whose basis is a paper that does not pass verification.

For each attack point with a paper cited as Evidence:

1. Locate the referenced paper's verification block in tester_literature_*.md.
2. Check the Verdict field.
3. Apply the rule:
   • verified-conflict: the attack stands as written.
   • superficial or insufficient-access, or the verification block is missing entirely: the paper-based portion of the Evidence is invalid. Re-read the attack's Evidence field: if it also includes a stand-alone logical argument or established empirical result, treat the attack as demoted one severity level (Fatal → Serious, Serious → Minor, Minor → discarded). If the only Evidence was the unverified paper, treat the attack as discarded for the purposes of Criteria 1–4.

Record in the output table (new row below Criterion 4, see Output section) which attacks were demoted or discarded, and why. If no attacks were demoted or discarded, state that explicitly.

Do not silently ignore unverified attacks. The discount record must be visible so the user and the Refiner can see which concerns were set aside and for what reason, and so the Tester is held to the verification standard in future rounds.

---

Criterion 1 — No fatal assumptions remain unaddressed

Pass if the Tester report contains zero attack points with severity "Fatal", OR every Fatal attack point from the current and all previous reports in this Loop 2 entry has been addressed in the current IDEA.md.

"Addressed" means one of:

• Resolved: the assumption has been removed, replaced, or the mechanism has been restructured so the attack no longer applies.
• Mitigated: the assumption is retained but IDEA.md now explicitly states the limitation and describes a concrete mitigation strategy or fallback.
• Reframed: the claim has been narrowed in scope so the fatal attack no longer targets the current formulation.

"Addressed" does NOT mean:

• Simply acknowledging the issue exists without action
• Moving the issue to "Open Questions" without mitigation
• Adding a vague qualifier ("this may not always hold")

To verify: for each Fatal attack point, check whether IDEA.md shows a substantive change responsive to that specific attack. If the Tester's re-test report explicitly confirms an attack is resolved, trust that assessment.

---

Criterion 2 — Novelty claim survives adversarial scrutiny

Pass if the Tester's adversarial search did not find a paper that fully preempts the idea (same problem + same or functionally equivalent mechanism), AND the differentiation from closest prior work stated in IDEA.md holds up against the Tester's novelty attacks.

Check by reading the Tester report's novelty-dimension attack points:

• If any novelty attack is rated Fatal and unaddressed → fail
• If the Tester found a directly preempting paper → fail (unless IDEA.md has since been revised to differentiate from it, verified by a re-test report)
• If novelty attacks are rated Serious but IDEA.md provides a specific, substantive differentiation → pass (the differentiation need not be airtight, but it must be specific and arguable, not hand-waving)

---

Criterion 3 — No internal contradictions

Pass if the Tester report contains no logical-consistency attack points rated Fatal or Serious that remain unaddressed.

Check specifically:

• Core claims do not contradict each other
• The proposed mechanism logically produces the claimed effect (the causal chain is stated and each link is plausible)
• The evaluation scope in "Problem Scope" is consistent with the breadth of the claims in "Core Claims" — claims are not broader than what the proposed evaluation could validate

---

Criterion 4 — Importance is justified

Pass if the Tester's importance-dimension attacks are either absent, rated Minor, or have been addressed.

"Addressed" means:

• The problem's significance is grounded in specific evidence (cited results, known limitations of existing methods, or demonstrated real-world impact)
• If the Tester raised that a simpler approach could achieve the same effect, IDEA.md explains why the simpler approach is insufficient (with specifics, not assertions)

Fail if a Serious or Fatal importance attack remains unaddressed — e.g., the problem is shown to be artificial, or a trivial baseline solves it.

---

Output

After evaluating all four criteria, output exactly this block:

## Loop 2 Exit Check

**Result: PASS / FAIL**

**Soundness pre-check:** [list attacks demoted or discarded due to
unverified paper evidence, with the reason — or "no attacks demoted or
discarded; all paper-based attacks had verified-conflict verdicts"]

| Criterion | Result | Note |
|-----------|--------|------|
| 1. No fatal assumptions unaddressed | PASS / FAIL | [what was found — apply to attacks AFTER soundness adjustments] |
| 2. Novelty survives scrutiny | PASS / FAIL | [what was found — apply to attacks AFTER soundness adjustments] |
| 3. No internal contradictions | PASS / FAIL | [what was found — apply to attacks AFTER soundness adjustments] |
| 4. Importance justified | PASS / FAIL | [what was found — apply to attacks AFTER soundness adjustments] |

**Failing criteria:** [list by number, or "none — all criteria passed"]

**Remaining Serious attacks:** [list any Serious-severity attacks (after
soundness adjustments) that persist but did not cause a criterion failure —
these are carried forward as known risks, not blockers]

Do not suggest how to fix failing criteria. That is the Refiner's responsibility (via Loop 1 retreat).