メニュー
Automated pipeline for generating and refining multiple research hypotheses.
デフォルトでは、最初にソースを確認する Prompt が選択されています。直接コマンドに切り替えるか、ローカルコピーをダウンロードすることもできます。
インストールを決める前に、SKILL.md と SkillsMP に表示されている付属ファイルをお読みください。
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
直接コマンドでは確認用 Prompt が省略されます。実行前にソースを確認してください。
npx skills add https://github.com/zjunlp/Mechanist --skill hypothesis-batchコマンドは1行のまま表示されます。コピー前に横へスクロールして全体を確認してください。
ローカルで確認しますか？SkillsMP が現在取得できるファイルをダウンロードできます。
SOC 職業分類に基づく
SKILL.md を表示中
| name | hypothesis-batch |
| description | Automated pipeline for generating and refining multiple research hypotheses. |
| argument-hint | ["research-direction"] |
| allowed-tools | Bash(*), Read, Write, Edit, Grep, Glob, WebSearch, WebFetch, Agent, Skill, mcp__llm-chat__chat |
Orchestrate the claim stage for: $ARGUMENTS.
This skill chains sub-skills into a single automated, non-interactive pipeline:
/research-lit → /idea-creator → /novelty-check → /impact-check → /research-review → /research-refine-pipeline
(survey) (brainstorm) (verify novel) (verify it (critical feedback) (refine method + plan)
matters)
Batch shape: 30 in, 10 out. Phase 2 brainstorms ~30 candidate ideas across three angle-partitioned /idea-creator rounds. Novelty (Phase 3) is a hard gate; impact (Phase 3.5) and external review (Phase 4) then score every survivor, and Phase 4.25 makes the single cut to 10 — vetoing fatal design flaws outright, then ranking by impact first, reviewer score second, novelty last. From Phase 4.5 on, each of the 10 is worked independently and in parallel, and each gets its own directory.
Deliverables.
idea-stage/
RESEARCH_LIT.md # raw retrieval dump (audit-only)
LANDSCAPE.md # synthesized landscape
IDEA_REPORT.md # all ~30 ideas, ranked, with eliminations — the batch index
claims/
01_<name>/ # one directory per surviving idea, rank-ordered
FINAL_PROPOSAL.md
EXPERIMENT_PLAN.md
claim.json # ⭐ the deliverable a human reviewer reads
02_<name>/
…
10_<name>/
claim.json is the point of the whole run: a self-contained paper skeleton written for a human expert reviewer, who sees that file and nothing else. Everything above it is working material. Its specification is Phase 4.8 and is binding.
Invoke /research-lit to map the research landscape:
/research-lit "$ARGUMENTS"
What this does:
/mechanic-db-search) for relevant papersidea-stage/RESEARCH_LIT.md (raw retrieval dump, audit-only) and idea-stage/LANDSCAPE.md (synthesized landscape — Phase 2 reads this from disk)This phase loads strategy into context; it writes no artifact. It is a reference step — its job is to put the relevant strategy skills in context so Phase 2 (hypothesis direction) and Phase 4.5 (experiment plan) can apply them. Both skills are principles only: read them, do not execute their phases, and do not expect an output file from either.
/mechanism-behavior-discovery and read its SKILL.md in full (the standard for finding, sharpening, and validating a new behavioral phenomenon — the five bars Real / Non-obvious / Specific / Robust / Tractable, the discovery strategies, and the validation discipline). It decides what behavior is worth explaining. Log [behavior-strategy] discovery — loaded /mechanism-behavior-discovery./mechanism-explore via the Skill tool and read its SKILL.md in full; the strategy shapes how to explain the behavior (the hypothesis direction in Phase 2 and the experiment plan in Phase 4.5). It fixes the direction only — the concrete family is left unbound. Log [mechanism-strategy] discovery — loaded /mechanism-explore.Hold the loaded guidance in context for the rest of the run — do not copy it into any output file.
What the loaded strategy is used for (advisory — it never relaxes /idea-creator's novelty / feasibility / pilot filtering):
/mechanism-behavior-discovery), then frame candidate ideas as falsifiable mechanistic hypotheses (name the implicated internal object and the predicted causal relation) with at least one "boring" null (memorization / surface feature / shortcut), favoring ideas on a climbable ladder of evidence. The claim assumes the chosen phenomenon exists and attaches the mechanistic hypothesis — its actual existence is tested by the M0 gate when the plan is run, not asserted here.This phase writes idea-stage/IDEA_REPORT.md, the ranked candidate pool that Phases 3–3.5 cut down to 10.
Three rounds, not one call. /idea-creator generates 8–12 ideas per invocation — that count is fixed inside its own prompt template and is not a parameter, so a single call cannot produce 30. Run it three times and merge:
/idea-creator "$ARGUMENTS — angle: <round assignment>; do not regenerate or closely vary: <titles from earlier rounds>"
Assign each round a different slice of the search space, so the rounds diverge by construction rather than by luck:
| Round | Assignment |
|---|---|
| 1 | The landscape's structural gaps — one idea aimed at each gap LANDSCAPE.md names. |
| 2 | The mechanism directions from /mechanism-explore not yet used in round 1 (Location / Causal Intervention / Tuning & Editing / Formation Tracing / Unit Interpretation / Decision Auditing). |
| 3 | The phenomena classes underrepresented after rounds 1–2 — read the two merged rounds, name what kind of behavior is missing, and aim the third round there. |
Pass the titles already generated as an explicit exclusion in each later round. Round 3 is written after reading rounds 1–2, so its assignment is derived, not guessed.
What each round does:
idea-stage/LANDSCAPE.md from disk (written by Phase 1 / /research-lit), then brainstorm 8–12 concrete ideas via the external LLM reviewer (llm-chat).Merge into idea-stage/IDEA_REPORT.md. Concatenate the three rounds into one ## Ranked Ideas section, then deduplicate: two ideas are the same idea when they share a phenomenon and an internal object, whatever their titles say. Keep the better-stated one, record the merge. Target ~30 candidates; 24–36 is fine, and a merged pool below ~20 means the rounds collapsed onto each other — run a fourth round on whatever the dedupe revealed as over-covered.
Breadth is the point. The pool exists to be cut, so 30 near-duplicates of one framing is a failed Phase 2. Overshoot rather than undershoot: the pool must survive a novelty gate and a design veto and still leave 10.
Apply the Phase 1.75 strategy. Pass /mechanism-explore's framing into /idea-creator so each generated idea is stated as a falsifiable mechanistic hypothesis (internal object + predicted causal relation + at least one boring null), and bias ranking toward ideas with a climbable ladder of evidence. First apply /mechanism-behavior-discovery to surface and sharpen candidate phenomena from the landscape (state each crisply per the five bars and screen plausibility) — the claim assumes the chosen phenomenon exists and attaches a mechanistic hypothesis; the phenomenon's actual existence is tested by the M0 gate in the experiment stage, not claimed here. This shapes idea framing only — novelty, feasibility, and pilot filtering are unchanged.
Run a thorough novelty check on every surviving candidate — one /novelty-check per idea, dispatched in parallel:
/novelty-check "[idea N description]" # for each surviving idea
What this does:
Update idea-stage/IDEA_REPORT.md with deep novelty results. Eliminate any idea that turns out to be already published, and record for each survivor its three nearest works with the specific finding each established — Phase 4.8's Related Work field and checklist item 1 both need exactly this, and re-deriving it later wastes a second literature pass.
For each idea that survived novelty (Phase 3), run an impact check on the problem/behavior it studies — one /impact-check per idea, in parallel:
/impact-check "[idea N — behavior/problem + hypothesis]" # for each survivor
What this does: judges whether the studied problem/behavior is important — solves a real problem, is likely to be used/cited, could shift a field's direction, helps applications/industry/society/cross-disciplinary work, or reveals an important phenomenon even with a simple method. Outputs Impact: X/10 + PROCEED / PROCEED WITH CAUTION / DEPRIORITIZE.
Update idea-stage/IDEA_REPORT.md. Add an Impact: X/10 line to each surviving idea. No cut happens here — impact is one of three signals the Phase 4.25 selection combines.
Review every idea that survived novelty — not just a shortlist. One /research-review per idea, dispatched in parallel:
/research-review "[idea N with hypothesis + evidence]" # for each survivor
What this does:
Reviewing the full pool rather than a pre-selected 10 is deliberate: novelty and impact both judge whether the question is worth asking, and neither can tell whether the proposed design could actually answer it. That is the reviewer's lens, and it is worth having before the batch is fixed rather than after.
Update idea-stage/IDEA_REPORT.md with each idea's reviewer score and feedback. The feedback has two distinct downstream uses, and Phase 4.25 depends on telling them apart:
Classify each review as one or the other when recording it.
This is the only cut that produces the batch. It combines all three signals.
Step 1 — Apply the veto. Eliminate every idea whose Phase 4 review found a fatal design flaw, regardless of its impact score. An idea whose measurement cannot answer its own question does not become shippable by being important. Record it under ## Eliminated Ideas with reason vetoed at external review — <the flaw>.
Step 2 — Rank the rest. Order by impact first, reviewer score second, novelty third:
Step 3 — Take the top 10. Everything below the cut moves to ## Eliminated Ideas with reason below rank 10. This ranking fixes the folder numbering in Phase 4.5 — nothing downstream re-orders it.
Step 4 — Check the spread. If the top 10 collapse onto two or three phenomena, swap the lowest-ranked duplicates for the highest-ranked candidates that cover a distinct phenomenon, and note each swap. Ten skeletons on one phenomenon is a worse batch than ten on eight, even at slightly lower average score.
If fewer than 10 survive, run one top-up round: return to Phase 2, generate additional ideas aimed at the gaps the eliminations exposed, and put them through Phases 3 → 4.25. If the batch is still short after that round, ship what survived and state the shortfall plainly in IDEA_REPORT.md — never pad the batch with an idea that was already eliminated, never overturn a novelty verdict or a design veto to reach 10.
From here on the batch splits. Each of the 10 ideas is refined independently, in its own directory, and nothing is shared between them except the landscape. Run the 10 in parallel.
Step 0 — Create the directories. For each idea, in the Phase 4.25 rank order:
<name>: a snake_case slug, ≤ 6 words, naming the phenomenon — the same slug that becomes claim.json's Name field in Phase 4.8. Derive it once here and reuse it verbatim; the directory name and the Name field must not diverge.claims/<NN>_<name>/, where NN is the zero-padded rank (01…10).[claim] <NN>_<name> — "<idea title>".Every artifact from this phase onward is written inside that directory. Nothing goes to a shared refine-logs/.
Step 1 — Refine each idea. For each of the 10, invoke /research-refine-pipeline with that idea's title, description, evidence, and its Phase 4 reviewer feedback:
/research-refine-pipeline "[idea <NN>: title + description + evidence + reviewer feedback]"
and write its outputs to claims/<NN>_<name>/FINAL_PROPOSAL.md and claims/<NN>_<name>/EXPERIMENT_PLAN.md.
What /research-refine-pipeline does (per idea):
Freeze a Problem Anchor (the idea's problem statement) to prevent scope drift.
Iteratively refine the method / testing approach via external LLM review (up to 5 rounds, until score ≥ 9).
Generate a claim-driven experiment roadmap with ablations, budgets, and run order.
(Mechanism milestones — ladder of evidence, from the Phase 1.75 strategy): shape the roadmap to climb the ladder of evidence — a cheap correlational/attribution screen to localize candidates → a causal intervention (ablation / patching / steering) to confirm the survivors → matched-control + off-target + confound milestones for specificity — and have each intervention milestone record the expected sign, magnitude / dose-response, and its specificity control. Because the plan commits to a direction, not to a concrete mechanism submethod, any milestone field whose correct value depends on that submethod — typically n_pairs, intervention sites, the exact effect metric, and the per-run GPU-hours estimate — is necessarily provisional. Tag each such milestone with a method_sensitive: [<field>, ...] line (English machine field, written verbatim regardless of report language) marking the fields that must be re-bound once a submethod is picked.
(Phenomenon-validation gate M0): the claim only assumes the phenomenon exists; whether it is real is settled by running the plan, not by writing it. So the roadmap's first milestone is a hard gate that must carry the machine marker kind: phenomenon-validation (the title may be phrased / localized freely — the marker is what identifies the gate mechanically). With an explicit pass criterion:
All mechanism milestones (M1…Mn) declare depends_on: [M0], and the M0 block spells out the four-state verdict that governs them: established → run the mechanism milestones; conditional → restrict mechanism analysis to the conditions where the phenomenon holds and tag the claim conditional; not-established → stop and report a negative result; inconclusive (the M0 test itself is broken or underpowered) → fix and re-run M0. Never run mechanism work on an untested phenomenon.
Output: claims/<NN>_<name>/FINAL_PROPOSAL.md, claims/<NN>_<name>/EXPERIMENT_PLAN.md.
Failure isolation. If refinement fails for one idea, that failure stays local: record it in that directory and in IDEA_REPORT.md, and continue with the other nine. One bad claim never aborts the batch.
EXPERIMENT_PLAN.md shape. Each milestone block MAY declare these fields:
depends_on: [<milestone-name>, ...] — this milestone's runs must wait until every listed upstream milestone is complete before launching. Use for teacher → student chains, baseline → ablation chains, or any milestone that needs an upstream artifact (checkpoint, dataset, calibration result). The names refer to other milestone blocks in the same EXPERIMENT_PLAN.md.grid: { <param-name>: [<value>, <value>, ...], ... } — Cartesian-product expansion. Each combination produces one run with the params substituted via ${param-name} placeholders into the milestone's cmd: template. Use for multi-seed sweeps (e.g., grid: { seed: [42, 200, 201], n_hidden: [64, 128, 256], n_train_subset: [50000, 150000, 500000] } → 27 runs). When grid: is present, the milestone's cmd: is treated as a template (rather than a literal command) and id: may also be a template (e.g., s${seed}_N${n_hidden}_n${n_train_subset}).kind: phenomenon-validation — stable machine marker identifying the M0 gate, so it can be found by field rather than by title. The marker is an English machine field (per the Output-language protocol) and must be written verbatim regardless of the report language, leaving the title heading free to be phrased / localized. Every plan has exactly one kind: phenomenon-validation milestone (M0).method_sensitive: [<field>, ...] — per intervention milestone. Lists the fields whose correct value depends on the mechanism submethod (typically n_pairs, sites, metric, gpu_hours) and therefore must be re-bound before the milestone can run.In addition to the per-milestone fields above, EXPERIMENT_PLAN.md carries the top-metadata machine marker mechanism_strategy: (the strategic-direction block above).
Example milestone block:
### M2: Multi-seed main method
**Depends on**: M1 (baselines)
**Grid**:
seed: [42, 200, 201]
n_hidden: [64, 128, 256, 512]
n_train_subset: [50000, 150000, 500000]
**Cmd template**: `python run_method.py --seed ${seed} --n_hidden ${n_hidden} --n_train_subset ${n_train_subset} --backbone softmax --K 500 --L 96`
**Expected output (template)**: `results/main_N${n_hidden}_seed${seed}_n${n_train_subset}.json`
**Priority**: MUST-RUN
**Estimated GPU-hours per run**: 1.5h
This block expands to 36 runs. For a small milestone (≤ 5 runs, no depends_on, no grid), just list the runs explicitly instead.
claim.json — the reviewer-facing deliverableWrite one claim.json per directory: claims/<NN>_<name>/claim.json. This is the final result of the whole workflow — a paper skeleton good enough to submit to a top AI conference, written for a human expert reviewer who has claim.json and nothing else.
This stage plans; it does not run. Every model, dataset, condition, and measurement named in Experiments has to be reachable today rather than aspirational. But nothing is executed here, no GPU is touched, and anything phrased as a result is a fabrication.
Before writing, and again before shipping each file: first carefully consider the quality, novelty, and feasibility of the proposal you just created.
Feasibility. Ensure that the proposal does not require resources beyond what an academic lab could afford. In Experiments, ensure these are simple and feasible. Do not make things overly complicated. Novelty bought with an unrunnable plan is not novelty.
Clarity. Ensure the proposal is clear and concise, and the JSON is in the correct format. In Short Hypothesis, clarify the need for this specific direction, ensure this is the best setting to investigate this idea, and there are not obvious other simpler ways to answer the question. In Related Work, clearly clarify how the proposal distinguishes from the existing literature. In Experiments, be specific in exactly how you would test the hypothesis, and detail precise algorithmic changes. Include the evaluation metrics you would use. A threshold with no stated basis, an equivalence claim with no margin, and a prediction with no direction are all clarity failures, not stylistic ones.
Seven keys, exactly these names, exactly this order, every value a plain prose string. No extra keys, no nested objects, no arrays. Paragraph breaks inside a value are \n\n.
{
"Name": "...",
"Title": "...",
"Short Hypothesis": "...",
"Related Work": "...",
"Abstract": "...",
"Experiments": "...",
"Risk Factors and Limitations": "..."
}
The claim JSON should include the following fields:
Name is additionally bound to the directory: it is the same snake_case slug, ≤ 6 words, naming the phenomenon, that was derived in Phase 4.5 — the directory name and the Name field must not diverge. The field-by-field section below is this workflow's binding elaboration of the seven descriptions above; where it is more specific, it governs.
Short Hypothesis carries the whole claim in one paragraph. It names the triggering condition (what has to be true for the phenomenon to appear), the phenomenon (a falsifiable regularity in observable model behavior), the internal object held responsible (layer / head / neuron / SAE feature / direction / circuit), and H1…Hk — two to four sub-hypotheses, each with a stated direction. H1 is always the existence claim: does this happen at all. The last H is the one that pays: a causal intervention or an edit that moves the behavior, not another correlation.
Related Work names real, citable work — author and year — and for each says what result it established, then what this claim contradicts, extends past its stated scope, or looks at where it never looked. «They study X, we study Y» is a topical separation and does not count. It closes with at most one positioning sentence («the first to (i)… (ii)…»), and every clause in that sentence must survive checklist item 1.
Abstract reads like the abstract of a paper someone would actually open: the human-legible motivation first, then the controlled design, then the hypotheses in a sentence each, then what a positive result gives the field.
Experiments is the field a reviewer skims hardest. It is written as labelled paragraphs separated by \n\n, in this order:
Risk Factors and Limitations is numbered, and every entry is a threat that could actually land — the probe measures the wrong thing, the effect is a template artifact, the intervention has side effects, the ground truth is contestable — each paired with the specific mitigation already present in Experiments. A limitation with no mitigation is allowed only when it is genuinely out of scope, and must say so.
Experiments is organized as a ladder, cheapest and most falsifying first. Each rung becomes one numbered experiment, and a later rung is only worth running if the earlier ones hold.
| rung | what it establishes |
|---|---|
| 1 | the phenomenon happens at all, under its stated trigger |
| 2 | cheap correlational / attribution screening narrows the internal object down to candidates |
| 3 | intervening on a candidate moves the behavior — sign, magnitude, dose–response |
| 4 | matched controls, off-target checks, and confound checks rule out the mundane rivals |
| 5 (optional) | how far it holds — phrasing, position, model family, interpretability replication |
Experiment 1 is a gate with four outcomes, all spelled out in the text: established (the plan proceeds), conditional (it holds only under some conditions, and the later experiments are scoped to those), not established (the claim dies), inconclusive (the measurement itself failed and must be fixed and re-run, not adjudicated). Its pass criterion must cover reproducibility across paraphrase, seed, and decoding; controlled confounds; a sample size clear of noise; and the ruling out of trivial explanations. This is the same gate as EXPERIMENT_PLAN.md's M0, rewritten as prose.
Where a threshold is stated, say what it rests on — measured in a cited paper, or an estimate. Estimates are fine; estimates dressed as measurements are not.
The rungs themselves are planning vocabulary and never surface in claim.json. «Phenomenon-validation», «localization», «specificity», «M0», kind: phenomenon-validation, method_sensitive are this workflow's names for its own scaffolding, and a reviewer holding only claim.json has no way to decode them. Each experiment is headed instead by the hypothesis it tests and the finding it would produce — Experiment 1 (H1 — Half-life):, Experiment 4 (H4 — Activation edit): — and opens with a sentence saying what it does in the paper's own terms. The ladder should be legible from the order and content of the experiments, never from a label the reader has to look up.
Experiment 1's inconclusive outcome is not special to the gate. Every threshold splits the outcome space into three regions rather than two — results that clearly support the hypothesis, results that clearly refute it, and the band between them where the planned measurement cannot decide — and finite samples, measurement noise, and any equivalence margin guarantee that band exists for H2…Hk as well.
So each hypothesis states its own, in the experiment that tests it: where the band lies, in the threshold's own units (an effect between 0.5σ and 2σ of cross-seed noise, say, or a half-life whose interval spans zero); what a result there means, said plainly — Hi is not supported at the planned power, the effect being too small to separate from noise at this sample size, or the measurement too coarse to separate the mechanism from its rivals — and not «a weak trend consistent with Hi», not silence; and what follows, fixed before the data rather than after it: Hi reported as undecided, the pre-specified extension run if the plan budgets one (more items, more seeds, a finer measurement), and the experiments depending on Hi stopped, scoped, or run unchanged.
That belongs where the reviewer meets the hypothesis. Risk Factors and Limitations adds only what the experiment cannot: where the plan's own power estimate says the band is likely to be entered — a small expected effect, few available items, a high-variance intervention — say so there, with what it would cost to narrow it.
Every quantity that carries a number — a threshold, a margin, a sample size, an item count, a budget — is defined once and written identically everywhere it appears: same name, same value, same unit. Within claim.json, a bar introduced in Short Hypothesis is the bar the matching experiment's prediction enforces, the one Ablations and controls and Metrics report against, and the one any Risk Factors and Limitations entry mentioning it uses. Across the directory, claim.json, EXPERIMENT_PLAN.md, and FINAL_PROPOSAL.md describe a single study, so wherever the same quantity appears in more than one of them — n per condition, the sweep values, Experiment 1's pass criterion, the GPU-hour estimate — the values match.
When they disagree, decide which value is right and edit every file that carries it. claim.json is the plan rewritten in the reviewer's register: rewriting the prose is expected, rewriting the numbers is not, so never round or simplify a number on the way in, and never leave the plan at the old value after sharpening the claim. A method_sensitive field is provisional, which means one number awaiting re-binding — not a different number in each file.
This run writes far more than it ships. LANDSCAPE.md holds the survey; IDEA_REPORT.md holds the scores and the eliminations; FINAL_PROPOSAL.md and EXPERIMENT_PLAN.md hold the executor-facing plan. A reviewer sees claim.json and nothing else, and the whole difficulty of this phase lives in that gap.
Every piece of working material either dissolves into one of the seven fields or stays behind. It never gets a field of its own, and it never goes unplaced by default:
| what the run produces | where it lands in claim.json |
|---|---|
| the phenomenon and its triggering condition | Short Hypothesis, opening sentences |
| where the idea came from — the cross-domain result or human finding it transfers | Abstract, the opening motivation; a clause in Related Work when the source is itself citable |
| the structural gap in the landscape | Related Work, restated as what the literature has not done |
| the mechanistic conjecture — internal object, causal relation | Short Hypothesis, the H-list |
| the mundane rival accounts | Experiments, where each one's control is introduced; Risk Factors and Limitations for any left unhandled |
| the three nearest works (Phase 3 / checklist 1) | Related Work, roughly a clause each |
| the expert's prior (checklist 2) | Abstract or Related Work — whichever sentence explains why the result would surprise |
| the consumer on each branch (checklist 3) | Abstract, closing sentence |
| the breaking case (checklist 4) | Risk Factors and Limitations, as the stated scope bound |
| the ladder, its gates, controls, models, data | Experiments |
| novelty / impact / reviewer scores, rank, direction combinations, rung names, rejected ideas | nowhere — batch bookkeeping, stays in IDEA_REPORT.md |
The routing must not show. The table says where, not that a sentence must appear. It routes material already going into the paper; it is not a list of sentences to add. A row is discharged when its content is there — usually inside a sentence written for another reason, often as a subordinate clause, sometimes two rows at once. The test runs backwards: a reviewer must not be able to reconstruct this table from the prose. Nothing announces its own function — no «our contribution is novel because», no «the expert prior is», no «as a named consumer». If a row's content is already carried by a sentence that had to be written anyway, it is done; adding a dedicated sentence for it makes the paper worse, and a paper that reads like a completed form does not survive review.
Routing showing through: The expert prior here is that retraction fully erases the claim, so our result is non-obvious. If the claim is true, deployers of retrieval-augmented systems will audit their corrections; if false, they will not.
Absorbed: A retraction is normally assumed to do what it says: the claim is withdrawn, and what follows is computed without it. We find instead that the withdrawn claim keeps shaping downstream inferences long after the model has verbally accepted the correction — an assumption that every multi-turn and retrieval-augmented deployment currently relies on untested.
One register for the agent, another for the reviewer. Every row of the table is sourced from something written in a register that is not the paper's — a bulleted gap in LANDSCAPE.md, a scored entry in IDEA_REPORT.md, an executor-facing milestone in EXPERIMENT_PLAN.md covering the same experiments as Experiments. All of it was written to be processed, by a machine or by you later; none of it was written to be read. So carrying a sentence across means rewriting it in the paper's register: keep the numbers, the direction, and the controls; drop the operational scaffolding that matters only to whoever runs it; and restore what an executor never needed and a reviewer cannot do without — what the measurement is for, and what it would show.
Executor-facing:
M2: for l in [12,16,20] patch resid_post @ last_tok, α∈{0.5,1,2}, n=200/cond, log Δlogprob(P) vs C0; gate |Δ|>2σ_seed; fallback l=8 if OOM.Reviewer-facing: Experiment 3 (H3 — Causal confirmation): We patch the residual stream at layers 12, 16, and 20 at the final token of the retraction, sweeping the edit strength over three values on 200 items per condition, and measure the change in log-probability of the retracted claim relative to the no-mention baseline. If the localized component is causal rather than merely correlated, the effect should be negative, should grow monotonically with edit strength, and should exceed the cross-seed noise we measure in Experiment 1.
Nothing may point at a file the reviewer does not have. No «the gap identified in LANDSCAPE.md», no «see EXPERIMENT_PLAN.md», no «idea #3 in the batch», no bare «Gap 3», and no reference to the other nine claims — each claim.json stands entirely alone. Where such material earns a place in the paper it is restated in full: the gap becomes a sentence about what the literature has not done, the rejected alternative becomes a design choice with its reason. Fail: any noun phrase in claim.json whose referent lives only in another file.
Short Hypothesis, Related Work, and Abstract are continuous prose — no bullets, no sentence fragments, no telegraphic notation. They should be readable end to end by someone who has never seen this workflow.Experiments is structured at the label level and prose everywhere below it. The labels exist so a reviewer can find the model list without reading the whole field, but a label is a signpost, not a license to stop writing sentences. Write «We fit an exponential decay of the probe-minus-baseline difference against washout distance and report a per-model half-life in tokens; we predict a positive half-life for every model, longer for stronger initial assertions», not «fit: exp decay; x: washout; out: half-life (tok); pred: >0». Only item 7 is a list.claim.json, in the field where it is first used.TBD, no <…>, no «details to be determined». An unknown is either resolved or named in Risk Factors and Limitations as an open choice with its options.Write Experiments first — the ladder forces the claim to become concrete, and a claim that cannot be laddered is not ready to be written up. Then Short Hypothesis, Related Work, Abstract, Risk Factors and Limitations, and finally Title and Name; the title comes easiest once you know what the paper actually shows.
Run this on each claim.json before shipping it. Each item is a check that can fail on a fact, never a quality the author can assert. «This idea is non-obvious» is satisfied by typing it; «name the three nearest works and say which of their findings this contradicts» is not.
What these checks turn up is content, not scoring. Each finding has a field waiting for it in the routing table above and is written there as part of the paper's argument — never as a self-assessment of how the claim scores.
Is the idea new
Would it matter
Can it be settled
Short Hypothesis die with it? If the claim could survive by reinterpreting the outcome as a measurement problem, the gate is hung on the wrong object — re-hang it. (Inconclusive is the branch for a broken measurement; it must not become a hiding place for the claim.)Experiments. Any account with no control either gets one or moves into Risk Factors and Limitations as explicitly not ruled out. Listing a rival and then ignoring it is a fail.claim.json says how wide the band is, that a result inside it leaves Hi unsupported at the planned power, and what the dependent experiments do then. Fail: a prediction with only two outcomes; a band with no stated consequence; a non-significant result read as evidence for a null without an equivalence margin.claim.json and find the same quantity in EXPERIMENT_PLAN.md and FINAL_PROPOSAL.md, comparing value and unit — a percent silently becoming a proportion, tokens becoming layers. Fail: one quantity holding two values across the three files; a bar set in Short Hypothesis that a later experiment does not enforce at that value; a number in claim.json with no counterpart in the plan. Repair every file; never drop the number from claim.json instead.Every failure is repaired in place, not noted. If a failure cannot be repaired without changing the claim, switch the mechanism direction and rewrite. After two such restarts on the same idea, stop working it, mark it blocked in IDEA_REPORT.md with what is blocking it, and promote the highest-ranked eliminated idea into the vacated slot rather than shipping a weak skeleton.
Batch independence. The 10 claims are checked separately and must not converge: if two claim.json files end up with the same phenomenon, the same H-list, or the same Experiment 1, one of them was never a distinct idea — rewrite it onto its own phenomenon or replace it from the ranked list.
Read each draft as if it were the only file that exists: repair every referent that resolves only elsewhere, rewrite every sentence still addressed to the agent that will run it, and cut every sentence whose only job is to satisfy a row of the routing table. claim.json is always English, regardless of the language used in the other reports.
This is the end of the workflow. Once all ten claims/_/{FINAL_PROPOSAL.md, EXPERIMENT_PLAN.md, claim.json}` are on disk, stop. Do not implement, launch, or queue any experiment, and do not chain into another skill.
claim.json is the product; everything else is scaffolding. A run that produces a beautiful IDEA_REPORT.md and ten thin claim.json files has failed. Depth per claim beats polish on the index.claim.json, EXPERIMENT_PLAN.md, and FINAL_PROPOSAL.md; disagreement is repaired in every file, never smoothed over in the one a reviewer reads.