| name | loop2-exit-check |
| description | Exit rubric for Loop 2 (stress testing). Load when the Main Agent needs to determine whether the current idea has survived adversarial scrutiny sufficiently to proceed to the Experimental Setup Module. Evaluates the latest Tester report against four criteria. Returns PASS or FAIL with specific failing criteria listed. Evaluates resilience to attack only — not development completeness (that was Loop 1's job). |
Loop 2 Exit Check
Read the latest Tester report: find the highest-numbered tester_report_*.md
in phase_0_ideation/stress_test/. Also read the current
phase_0_ideation/ideas/IDEA.md for context, and the accompanying
tester_literature_*.md with the same version number.
Apply the soundness pre-check first, then the four criteria below in order.
Each criterion is binary: pass or fail. Do not score, weight, or partially
credit them.
A single failing criterion means the overall result is FAIL.
Soundness pre-check — discount superficial paper-based attacks
The Tester is required to verify every paper cited as Evidence (Step 3.5 of
the idea-pressure-tester agent). Before applying Criteria 1–4, audit the
Tester report against the tester_literature_*.md verification blocks and
discount any attack point whose basis is a paper that does not pass
verification.
For each attack point with a paper cited as Evidence:
- Locate the referenced paper's verification block in
tester_literature_*.md.
- Check the Verdict field.
- Apply the rule:
verified-conflict: the attack stands as written.superficial or insufficient-access, or the verification block is
missing entirely: the paper-based portion of the Evidence is invalid.
Re-read the attack's Evidence field: if it also includes a stand-alone
logical argument or established empirical result, treat the attack as
demoted one severity level (Fatal → Serious, Serious → Minor, Minor →
discarded). If the only Evidence was the unverified paper, treat the
attack as discarded for the purposes of Criteria 1–4.
Record in the output table (new row below Criterion 4, see Output section)
which attacks were demoted or discarded, and why. If no attacks were
demoted or discarded, state that explicitly.
Do not silently ignore unverified attacks. The discount record must be
visible so the user and the Refiner can see which concerns were set aside
and for what reason, and so the Tester is held to the verification standard
in future rounds.
Criterion 1 — No fatal assumptions remain unaddressed
Pass if the Tester report contains zero attack points with severity "Fatal",
OR every Fatal attack point from the current and all previous reports in this
Loop 2 entry has been addressed in the current IDEA.md.
"Addressed" means one of:
- Resolved: the assumption has been removed, replaced, or the mechanism
has been restructured so the attack no longer applies.
- Mitigated: the assumption is retained but IDEA.md now explicitly states
the limitation and describes a concrete mitigation strategy or fallback.
- Reframed: the claim has been narrowed in scope so the fatal attack no
longer targets the current formulation.
"Addressed" does NOT mean:
- Simply acknowledging the issue exists without action
- Moving the issue to "Open Questions" without mitigation
- Adding a vague qualifier ("this may not always hold")
To verify: for each Fatal attack point, check whether IDEA.md shows a
substantive change responsive to that specific attack. If the Tester's
re-test report explicitly confirms an attack is resolved, trust that assessment.
Criterion 2 — Novelty claim survives adversarial scrutiny
Pass if the Tester's adversarial search did not find a paper that fully
preempts the idea (same problem + same or functionally equivalent mechanism),
AND the differentiation from closest prior work stated in IDEA.md holds up
against the Tester's novelty attacks.
Check by reading the Tester report's novelty-dimension attack points:
- If any novelty attack is rated Fatal and unaddressed → fail
- If the Tester found a directly preempting paper → fail (unless IDEA.md has
since been revised to differentiate from it, verified by a re-test report)
- If novelty attacks are rated Serious but IDEA.md provides a specific,
substantive differentiation → pass (the differentiation need not be airtight,
but it must be specific and arguable, not hand-waving)
Criterion 3 — No internal contradictions
Pass if the Tester report contains no logical-consistency attack points
rated Fatal or Serious that remain unaddressed.
Check specifically:
- Core claims do not contradict each other
- The proposed mechanism logically produces the claimed effect (the causal
chain is stated and each link is plausible)
- The evaluation scope in "Problem Scope" is consistent with the breadth of
the claims in "Core Claims" — claims are not broader than what the proposed
evaluation could validate
Criterion 4 — Importance is justified
Pass if the Tester's importance-dimension attacks are either absent, rated
Minor, or have been addressed.
"Addressed" means:
- The problem's significance is grounded in specific evidence (cited results,
known limitations of existing methods, or demonstrated real-world impact)
- If the Tester raised that a simpler approach could achieve the same effect,
IDEA.md explains why the simpler approach is insufficient (with specifics,
not assertions)
Fail if a Serious or Fatal importance attack remains unaddressed — e.g.,
the problem is shown to be artificial, or a trivial baseline solves it.
Output
After evaluating all four criteria, output exactly this block:
## Loop 2 Exit Check
**Result: PASS / FAIL**
**Soundness pre-check:** [list attacks demoted or discarded due to
unverified paper evidence, with the reason — or "no attacks demoted or
discarded; all paper-based attacks had verified-conflict verdicts"]
| Criterion | Result | Note |
|-----------|--------|------|
| 1. No fatal assumptions unaddressed | PASS / FAIL | [what was found — apply to attacks AFTER soundness adjustments] |
| 2. Novelty survives scrutiny | PASS / FAIL | [what was found — apply to attacks AFTER soundness adjustments] |
| 3. No internal contradictions | PASS / FAIL | [what was found — apply to attacks AFTER soundness adjustments] |
| 4. Importance justified | PASS / FAIL | [what was found — apply to attacks AFTER soundness adjustments] |
**Failing criteria:** [list by number, or "none — all criteria passed"]
**Remaining Serious attacks:** [list any Serious-severity attacks (after
soundness adjustments) that persist but did not cause a criterion failure —
these are carried forward as known risks, not blockers]
Do not suggest how to fix failing criteria. That is the Refiner's responsibility
(via Loop 1 retreat).
