research-ops

# Research Ops Operator-initiated research workflows. For one-shot research questions, use `/research`. | Mode | Trigger | What it does | |------|---------|--------------| | `cycle` | dispatched by `/observe maintain` (research rotation), or `/research-ops cycle` by hand | The research-domain generate/execute lanes (discover/gap/plan/review/verify). NOT a standalone loop conductor. | | `compile` | `/research-ops compile <concept>` | Synthesize memos into unified article | | `diff` | `/research-ops diff <text or path>` | Extract what's NOT in training data | | `dispatch` | `/research-ops dispatch [depth]` | Parallel audit sweep | > **Independent research lanes.** Codex subprocesses can handle bounded lanes when > useful work remains for the parent. Match access and output permissions to the task; > verify uncertain transport before fan-out. See > [Codex subprocess dispatch](../model-guide/references/codex-subprocess-dispatch.md). --- # Mode: cycle (research generate/execute lanes) > **Not a loop conductor.** The single RSI-loop conductor is `/observe maintain` (one > `/loop 30m` window). This mode is the **research-domain worker** it dispatches when the > conductor's weekly research rotation is due — or run it by hand for a focused research push. > The earlier "run this on `/loop 15m`" framing was retired 2026-06-12 when the three > overlapping conductors (orchestrator, improve maintain, research-ops cycle) were merged into one. Two lanes split by the **verifier boundary** (constitution: verifier-conditioned autonomy): - **Generate (Dreamer)** — unattended idea-generation. Discover, gap-analyze, brainstorm, cross-model critique → write *buildable proposals* to `queue/` and *human-gated questions* to `decisions-pending/`. **Never executes.** Safe to run while you sleep because it only produces reversible drafts for a yes/no. This is the "wake up to N ideas, say yes/no" endgame. - **Execute (attended)** — plan, review, execute, verify the greenlit queue items. The partial verifier (you + cross-model + `verify_claim`) sits here, so this lane is **attended** — run it via `/loop` in an open session, or by hand. > **Live reference: hutter `OUTER-LOOP.md`.** hutter runs the *same* two-role shape (Grinder/Dreamer + git bus) and it's the proven, live instance (ledger updated daily). The one difference is the verifier: hutter has a **clean** bit-exact gate (regime 1), so its Grinder auto-ratchets unsupervised. Research domains are **partial-verifier** (regime 2) — synthesis/gap/discovery have no ground truth — so here the unattended lane *generates only*; execution stays attended. Do **not** port hutter's auto-ratchet, parking-lot, or pre-registered-ΔS machinery: those need a clean metric, and faking one is the vetoed `session_quality` trap. ## Scheduling primitives (corrected 2026-06-11) | Primitive | Reads LOCAL state? | Unattended? | Quota | Use for | |---|---|---|---|---| | `/loop [int] <cmd>` | Yes (current session) | No — needs session open | local, ~free | the **attended Execute lane**, and minimal-mode manual Generate | | **launchd plist** → `claude -p "/research-ops cycle"` | **Yes** | **Yes (while Mac awake)** | **zero** | the **unattended Generate lane for LOCAL projects** (hutter, etc.) | | `/schedule` (cloud Routine) | **NO — cloud checkout of ONE GitHub repo** | Yes (survives reboot) | counts vs ~15 routines/24 h, min 1 h interval | Generate lane ONLY when the project + skill + state are all committed to a GitHub repo and the work is repo-native | **`/schedule` is a CLOUD agent, not a local scheduler.** It clones one GitHub repo into a sandbox and cannot see local files, the local `queue/`, `ledger.db`, `gather-cycle-state.sh`, MCP state, or env. Its "prompt" is just `job_config.ccr.events[0].data.message.content` (a string — you can put `/research-ops cycle` there), but the skill and the project state must live *inside the cloned repo* for that to resolve. So: - **Local project with local state (hutter — no GitHub remote):** `/schedule` literally cannot reach it. Use **launchd** (zero-quota, local) or `/loop`. - **GitHub-resident project, skill committed into that repo, repo-native work:** `/schedule` fits. Default cloud model is `claude-sonnet-5` (bumped 2026-06-30 from `claude-sonnet-4-6` — same cheap-default-with-Opus-escalation design, newer model); request Opus if the generation warrants it. `/loop` is for babysitting an open session. The old "run all six phases unsupervised on `/loop 15m`" design is **retired** — it needed a babysat session *and* auto-executed unverified work (worst of both), which is why every project's CYCLE.md went stale. ## Live State !`bash ${CLAUDE_SKILL_DIR}/scripts/gather-cycle-state.sh "$(pwd)" 2>&1 | head -80` ## Which lane runs this fire One invocation of `/research-ops cycle` picks ONE lane from the Bus State above (first match wins): 1. **Greenlit items in `queue/` AND you are attended** (interactive session / `/loop`, not an unattended Routine) → **Lane B (Execute)**. Drain the queue. 2. **Otherwise** → **Lane A (Generate)**. Refill `queue/` + triage `decisions-pending/`. A `/schedule` Routine ALWAYS runs Lane A — it is unattended, so it must never execute (no clean verifier). Lane B only ever runs when a human is at the desk. If "NO STATE CHANGE" → one-line noop, stop. --- ## Lane A — Generate (Dreamer) The unattended generator. Runs as a **launchd job** (`claude -p "/research-ops cycle"`) for local projects, a `/schedule` cloud Routine only for GitHub-resident ones, or `/loop /research-ops cycle` in an open session (minimal/manual). Each fire: 1. `git pull`. Read `queue/` depth, `decisions-pending/`, the Live State improvement signals, and recent git log. 2. **Decide if there is work.** Queue healthy (≥8 unused proposals) AND no fresh improvement signals → write one `noop` line to `CYCLE.md` log, `git push` if changed, **stop**. Do not manufacture work. 3. **Discover + gap-analyze — lead with the divergent engine, not a raw search.** Run `/observe lever` for frontier-scanned step-changes (its `missing` mode when the surface is mature and you suspect unframed blind spots; `generators` when recent wins keep arriving off-trail), `/brainstorm` for within-frame candidates, the generators in [follow-up moves](../research/references/follow-up-moves.md) for new angles on stalled lines, and `search_preprints` + `/research dispatch` for the literature. **Science/research repos — standing SOTA sweep every fire:** pull the newest validated papers (`search_preprints` + `traverse_citations` since the last fire) AND newest tools/methods (`/trending-scout`, Exa) for the repo's domain, and propose upgrades that move it to the current frontier — genomics is the exemplar (always run the newest validated method, not the one that was SOTA when the pipeline was written). Compare every candidate against git log + research memos — skip anything already known. Read improvement signals from Live State; prioritize `STEER`. If discover returns empty, `/brainstorm` the project domain. SWE quality gaps belong to `/maintain` — don't duplicate. 4. **Write proposals to `queue/`.** Append ≤8 concrete, buildable, ONE-change proposals — each with: the change, files it touches, rationale tied to evidence, and how to verify it. These are *candidates*, not approvals. 5. **Human-gated items → `decisions-pending/`.** Anything in the "never autonomous" set (classification thresholds with clinical implications, validated clinical logic, new verification tooling, GOALS.md direction) goes here as a written question — never into `queue/`. For these, run `/critique model` cross-lab (Gemini 3.5 Flash + Opus, a *different* lab than the generator) and write recommendation + dissent + the open question. Do NOT greenlight. 6. `git commit + push` (`queue/`, `decisions-pending/`, `CYCLE.md` log). Keep output small. Never ask whether to continue. **WIP caps:** ≤8 unused queue proposals (skip discover above that); ≤3 undispositioned discoveries. **Turn budget:** stop searching at 70% of turns and synthesize. **Recitation:** recite key evidence before drawing conclusions. ### The Dreamer prompt For a **launchd / `claude -p`** local job, the prompt is just the string `/research-ops cycle` — the skill's lane logic (above) handles everything, so the prose block below is unnecessary. For a `/schedule` **cloud** Routine you must spell it out because the cloud agent starts with zero context and a different repo checkout; parameterize `{PROJECT}` / `{ROOT}` and ensure the skill + state are committed to the cloned repo: ``` <role>You are the Dreamer for {PROJECT}. You GENERATE buildable research/improvement proposals; you never execute them. Read-only on the project; write-only to queue/ and decisions-pending/. Discovery/synthesis here has no clean verifier, so you produce reversible drafts for a human yes/no — you never ship.</role> <on_each_fire> 1. cd {ROOT}; git pull. Read queue/ depth, decisions-pending/, recent git log (-15), and improvement signals (run scripts/gather-cycle-state.sh {ROOT}). 2. If queue ≥8 unused AND no fresh STEER/quality signal AND decisions-pending/ empty: write one noop line to CYCLE.md, git push if changed, STOP. Don't manufacture work. 3. Discover + gap-analyze: /observe lever for frontier step-changes (missing mode if the surface is mature, generators if wins keep arriving off-trail); /brainstorm within-frame; `operator-deck next` (the operator's usual thought steps, rotated per project: do the one whose trigger holds, then `operator-deck log`) for the angles he would push; search_preprints/Exa for new papers + tools. SCIENCE REPOS — standing SOTA sweep every fire: newest validated papers (search_preprints + traverse_citations since last fire) + newest tools (/trending-scout) → upgrade proposals (genomics exemplar: run the newest validated method, not the one SOTA when the pipeline was written). Skip anything in git log / research memos. Prioritize STEER. SWE quality → /maintain's lane, skip. 4. Append ≤8 buildable ONE-change proposals to queue/QUEUE_NNN.md — each: change, files, evidence-tied rationale, verification method. Candidates, not approvals. 5. "Never autonomous" items (clinical thresholds, validated logic, new verifier tooling, GOALS direction) → decisions-pending/ as a written question. Run /critique model cross-lab (Gemini 3.5 Flash + Opus) and write recommendation + dissent + open question. Do NOT greenlight. 6. git commit + push (queue/, decisions-pending/, CYCLE.md). Keep output small. You are bounded and unattended — do the work and STOP. Never ask whether to continue. </on_each_fire> ``` Cap fires ≤6/day (well under the ~15 routines/24 h ceiling). --- ## Lane B — Execute (attended) Run when you're at the desk (`/loop /research-ops cycle` in an open session, or by hand). This lane drains the queue the human greenlit. **Verify before execute** — never start a new execution with an unverified prior change. ### Phase priority (first match wins) 1. **Recent execution without verification** → verify. 2. **Greenlit items in `queue/`** → execute. The human greenlights by leaving a proposal in `queue/` (remove to block, reorder to prioritize). 3. **A queue proposal needs a plan/review before it's safe** → plan + review. ### Plan + Review **Plan:** Read the top greenlit proposal. Check `artifacts/failed-experiments/` for prior attempts on the same subsystem — if found, include "Previously tried: {summary}. Failed: {reason}" so you don't repeat it. Write files-to-change, what changes, verification method. **Review** — three steps: 1. **Blind first-pass:** read the plan WITHOUT re-reading the proposal's framing; form an independent assessment, THEN compare and note divergence (breaks confirmation bias — SycEval 58% fold rate without structural mitigation). 2. **Probe external claims inline:** HTTP-probe any URL / API / version the plan cites before trusting it. 3. **Cross-model review** (skip for trivial docstring/config tweaks): ```bash uv run python3 ~/Projects/skills/critique/scripts/model-review.py \ --context /tmp/cycle-plan.md --topic "research-cycle-{N}" \ --axes standard --project "$(pwd)" \ --question "Review for wrong assumptions, missing steps, anything that breaks existing functionality" ``` Route `--axes` by stakes (standard / deep / full). On failure, classify before retrying (auth→retry, rate-limit→other model, timeout→shrink context); never blind-retry the same model. ### Execute + Verify **Execute:** Note HEAD SHA. Implement. Commit. Reflect 1-3 lines (what was easier/harder, did assumptions hold). Mark the queue proposal consumed. **Verify** — two tracks: - **Track A — infra** (code/config/hooks/scripts): run tests, check file existence, compare to known-good. - **Track B — research output** (`research/*.md` produced/modified): extract 3-5 falsifiable claims (numbers, dates, named entities, causal assertions), call `verify_claim` on each, record verdicts. **FAIL** = any claim contradicted >0.7 confidence → revert flow. **WARN** = insufficient → `decisions-pending/`, don't block. **If verification fails:** archive before reverting (`git format-patch HEAD~1 -o artifacts/failed-experiments/` + JSON sidecar per `~/Projects/agent-infra/schemas/gap-fingerprint.json`), then `git revert HEAD` (not reset), mark the proposal `FAILED: {reason}` (skip 2 cycles), write to `decisions-pending/`. ## The bus = git files | File/dir | Dreamer (Lane A) | Execute (Lane B) / human | |---|---|---| | `queue/QUEUE_NNN.md` | appends buildable proposals | drains greenlit items; human blocks by removing | | `decisions-pending/*.md` | appends human-gated questions + cross-lab synthesis | human reads, greenlights into `queue/` | | `CYCLE.md` | one-line fire log + rolling state board | reads; human steers via notes | | `artifacts/failed-experiments/` | — | archived patches + fingerprints (prune >90 d) | Git is the durable, mergeable, auditable substrate (native-patterns). Separate `queue/` files avoid the merge conflicts a shared `CYCLE.md` queue caused. If Dreamer (cloud Routine) and Execute (local) are on different hosts, the Routine `git pull`s at fire start and `git push`es at end. `/maintain` owns the separate SWE quality lane (`MAINTAIN.md`) — this skill doesn't touch it. ## Rate-limit fallback Before a Lane-A fire, `CLAUDE_PROCS=$(pgrep claude | wc -l)`. If ≥6, route discover/gap-analyze through the shared dispatch helper instead of Claude subagents: ```bash uv run python3 ~/Projects/skills/scripts/llm-dispatch.py --profile cheap_tick \ --context /tmp/cycle-phase.md --prompt "[instruction]" --output /tmp/cycle-out.md \ --meta /tmp/cycle-meta.json --error-output /tmp/cycle-err.json ``` `model-review.py` already routes through llmx. Tag `[rate-limited]` in the CYCLE.md log. ## Minimal vs full - **Minimal (start here):** Lane B attended; Lane A run **manually** (`/loop /research-ops cycle` in an open session at the desk). Zero quota. - **Full:** Lane A unattended — **launchd** (`claude -p "/research-ops cycle"`) for local projects, or a `/schedule` cloud Routine for GitHub-resident ones — refilling `queue/` + triaging `decisions-pending/` (~daily). Promote minimal → full only once the queue demonstrably empties for lack of fresh ideas — not before (demand-gated, per OUTER-LOOP.md). ## Operating rules 1. **Never ask for input** in either lane. Lane A writes questions to `decisions-pending/`; Lane B surfaces them to the human at the desk. 2. **Lane A generates, never executes.** Lane B executes only greenlit items, only attended, verify-before-execute. 3. **Report in 1-3 lines:** phase, outcome, what's next. 4. **Idempotent:** check git log + queue before acting; don't redo completed work. --- # Mode: compile Synthesize scattered research memos into a unified concept article for **human consumption**. Reads across projects, extracts claims with provenance, surfaces contradictions, identifies gaps.