Use when the user wants to generate and literature-vet a pool of novel, testable research hypotheses for a question or domain. A multi-agent loop: a Generator proposes candidate hypotheses, a LiteratureScout grounds each in real retrieved literature (already known? closest prior work? what gap does it fill?), and a Judge scores them against a fixed rubric and keeps the strong, non-duplicate ones; rounds repeat — mutating toward the open gaps — until fresh rounds stop adding keepers. Not for sharpening or decomposing a research question (no grounding/scoring there), and not for grading an existing written proposal against the literature.
Use when the user wants to generate and literature-vet a pool of novel, testable research hypotheses for a question or domain. A multi-agent loop: a Generator proposes candidate hypotheses, a LiteratureScout grounds each in real retrieved literature (already known? closest prior work? what gap does it fill?), and a Judge scores them against a fixed rubric and keeps the strong, non-duplicate ones; rounds repeat — mutating toward the open gaps — until fresh rounds stop adding keepers. Not for sharpening or decomposing a research question (no grounding/scoring there), and not for grading an existing written proposal against the literature.
compatibility
Requires Python 3.9+
metadata
{"version":"0.1.0"}
Hypothesis Generation Loop
A multi-agent, literature-grounded generation loop. The artifact is a growing pool of
hypotheses; the feedback signal is the count of strong, distinct hypotheses that clear the bar —
where "strong" is decided against real retrieved literature, not assertion. Each round: generate →
ground → judge → keep → mutate toward the gaps, until the pool stops growing (saturation).
The discipline: a hypothesis enters the pool only if the literature says it is not already
established (novelty), prior work makes it plausible (grounding), and a feasible test exists.
Generating is not confirming — the output is a ranked set of strong candidates to test, each stated
with how to test it.
The cast (all in roles/):
roles/Generator.md — proposes a batch of candidate hypotheses aimed at the open gaps.
roles/LiteratureScout.md — grounds each candidate in real literature (novelty · support · gap);
emits litscout.json (validates schemas/litscout.schema.json).
roles/Judge.md — scores each against the fixed rubric and decides keep/kill/dedupe; emits
verdict.json (validates schemas/verdict.schema.json).
Spawn-or-degrade. On Claude Code, spawn Generator / LiteratureScout / Judge as real Agent
subagents each round; otherwise adopt each role inline in this context. You are the orchestrator.
When to use
Use when the user wants candidate hypotheses generated and vetted for a question or domain. Default:
run the full generate→ground→judge loop below until saturation. Escape hatch: if the user only wants a
single batch (no looping), run one round and report the kept hypotheses. Not for sharpening or
decomposing a question, and not for grading an existing written proposal.
Setup
Resolve bindings interactively. If loop.run.yaml exists in the working dir, load it, confirm the
values in one line, and skip to the loop. Otherwise: on Claude Code (the AskUserQuestion tool is
available) infer a likely value for each binding and present it as the recommended option; on other
hosts ask each as a quoted plain-text prompt. Then write loop.run.yaml (format:
examples/run.example.yaml) and confirm every value plus the live/degraded literature tier before
creating any other files.
binding
meaning
default
how to infer
<question>
the research question / domain, plus any scope (field, population, constraints)
—
ask the user
<gen_n>
candidate hypotheses the Generator proposes per round
6
—
<keep_threshold>
rubric score (0-100) a hypothesis must clear to enter the pool
65
—
<eval_scale>
LiteratureScout grounding depth (low/medium/high, see below)
medium
—
<sandbox_root>
where rounds, ledger, and lit cache live
./sandbox
—
<budget>
max rounds
6
—
<patience>
stop after this many rounds with no new kept hypothesis
2
—
<report>
final ranked hypothesis set
<sandbox_root>/hypotheses.md
—
Grounding depth dial (<eval_scale> caps per round — candidates examined deeply · queries each ·
papers read full-text):
preset
candidates deep
queries each
fulltext reads
low
2
1
0 (snippet/abstract only)
medium(recommended)
all
2
1
high
all
3
3
Literature toolchain. Paper search goes through the sibling literature-search skill — resolve
<lit_skill_dir> (it installs as a sibling, e.g. ~/.claude/skills/literature-search/),
<lit_py> = python3, and <lit> = <lit_skill_dir>/tools/lit_search.py (note the tools/
segment); append --cache-dir <sandbox_root>/literature/.cache after a subcommand to reuse the cache.
Confirm <lit> --help works at setup; if the skill is absent, tell the user and either install it
(copy the repo's loops/literature-search folder into ~/.claude/skills/) or degrade all retrieval to
WebSearch/WebFetch (tag that evidence source:"web"). The keyless S2 + arXiv core needs no setup; a
free S2_API_KEY makes snippet/cite reliable.
API key (optional, never block). The literature-search skill owns the key convention: run
<lit> keys --init, then have the user fill the printed keys.env themselves and never paste secrets
into chat. Re-run <lit> keys to record the tier in loop.run.yaml (literature_tiers, presence
only). A missing key just degrades to the keyless pool → WebSearch.
Initialise the sandbox once bindings are confirmed:
Start with an empty pool and gaps seeded from <question>; create no round files until the loop runs.
The loop
pool = the kept hypotheses (starts empty). gaps = open questions the LiteratureScout has surfaced
(starts empty; seed from <question>). dry = consecutive rounds with no new keep (starts 0). <N>
starts at 1.
Copy this checklist and tick items off:
Generate — run roles/Generator.md (spawn-or-degrade) with <question>, the current pool, gaps, and <gen_n>; it writes round<N>/candidates.json (<gen_n> specific, testable, plausibly-novel candidates aimed at the gaps, none duplicating the pool).
Ground — run roles/LiteratureScout.md (spawn-or-degrade) on candidates.json with <lit> and the <eval_scale> caps; it writes round<N>/litscout.json (validates schemas/litscout.schema.json) — per candidate: novelty + closest prior work, support, gap, testability, each citing real evidence.
Judge — as roles/Judge.md, apply the fixed rubric + evidence gate to litscout.json, checking each candidate against the pool for duplicates; write round<N>/verdict.json (validates schemas/verdict.schema.json): scores, total, keep, duplicate_of.
Update — add every keep:true non-duplicate to pool (with scores + grounding + how-to-test); add this round's gap points to gaps. If ≥1 new keep, dry = 0; else dry += 1.
Log one ledger row (see Ledger); N = N + 1.
Stop check — dry == <patience> (saturation) or N > <budget> → stop (see Stops).
Re-ground every round — novelty is judged from a fresh literature check each round, never carried
over, so "the literature already covers this" reliably kills a crowded idea. Every cited snippet comes
from a real retrieval that round; on {"error","fallback"} fall back to WebSearch/WebFetch — never
invent a paper.
schemas/litscout.schema.json gates the LiteratureScout output — a generic instance:
{"round":2,"evaluations":[{"hid":"r2h1","novelty_assessment":"novel","closest_prior_work":[{"claim":"X tested for facts, not skills","cites":["E1"]}],"support":[{"claim":"spacing aids motor consolidation","cites":["E2"]}],"gap":[{"claim":"long-term procedural retention untested at scale","cites":
schemas/verdict.schema.json gates the Judge output — a generic instance:
{"round":2,"verdicts":[{"hid":"r2h1","scores":{"novelty":4,"grounding":4,"testability":5,"specificity":4,"significance":4},"total":82.0,"keep":true,"gate_failures":[],"duplicate_of":null,"rationale":"Closest work shows X untested for skills -> novel; supported; clean RCT named."}
Ledger
<sandbox_root>/ledger.tsv, tab-separated, never commas in free text. Header:
Per-round candidates.json / litscout.json / verdict.json live in round<N>/. Report the best
state of the pool when stopping, not just the last round. Leave ledger.tsv, round*/, and
literature/ untracked.
Constraints
Never fabricate citations or snippets — every evidence entry is a real <lit>/WebFetch retrieval
from that round, verbatim; the evidence gate exists to catch fabrication.
Novelty is decided by the literature, not assertion — a hypothesis the search shows is already
established is killed, however appealing; "I think it's novel" with no closest-work search caps novelty.
Generate ≠ confirm — kept hypotheses are strong candidates to test, each stated with its test;
never report them as established findings.
The rubric is fixed and the keep-bar is stable across rounds, so saturation is a meaningful stop.
Reward distinct hypotheses, not volume — duplicates and rewordings are cut.
No installs — the sibling literature-search skill is stdlib-only; never print or commit API keys
(keys.env stays gitignored at the project root). The sandbox is self-contained — no ../ escapes.
Do not pause the loop to ask whether to continue; run until saturation or budget.
Stops
The loop stops on the first of:
Saturation — dry == <patience> consecutive rounds add no new kept hypothesis.
Budget — <budget> rounds reached.
End with the ranked hypothesis set (<report> path) — each hypothesis with its statement, novelty
assessment + closest prior work (cited), supporting evidence (cited), the gap it fills, and how to test
it — plus the pool-size trajectory from ledger.tsv and the strongest unexplored gaps, so the user
sees both the vetted hypotheses and where a deeper run would look next.
[
"E1"
]
}
]
,
"testability_note"
:
"RCT: spaced vs massed schedule, 1-month retention."
