rumil-versus-generate

name	rumil-versus-generate
description	Run versus's generation scripts — completions and paraphrases — via versus/scripts/run_{completions,paraphrases}.py. Each model in the config routes to its provider (OpenRouter for non-claude ids today; claude-* may route direct to Anthropic in future iterations). Handles the uv --with dance and --config / path anchoring. Default scope is the canonical active essay set; pass --include-stale to widen. Use when the user wants to produce or top up versus completion/paraphrase rows. **Judgments are not in this skill** — the unified judge entry point lives in rumil-versus-judge.
allowed-tools	Bash, Read
argument-hint	--op completions|paraphrases [--model <id>...] [--essay <id>...] [--include-stale] [--limit N] [--current-only] [--dry-run]

rumil-versus-generate

Wraps the two versus generation scripts:

--op	Script	What it writes
completions	run_completions.py	rows in versus_texts — model continuations + a human baseline row per essay
paraphrases	run_paraphrases.py	rows in data/paraphrases.jsonl (paraphrase pipeline is currently dormant — kept for legacy invocation)

For judgments, use rumil-versus-judge. Its default (no --variant) runs the unified blind path that routes claude-* direct to Anthropic and everything else through OpenRouter, so there is no longer an OpenRouter-only judgment script.

When to use

Intent	This skill?
"run a flash / gpt-5.4 / gemini completion on essay X"	yes (single-shot path)
"regenerate paraphrases after a prompt bump"	yes
"run an orch-driven completion (TwoPhase / DraftAndEdit)"	no — use rumil-versus-complete
"top up judgments (any model)"	no — use rumil-versus-judge
"run a rumil:orch judge"	no — use rumil-versus-judge
"pre-flight check before a topup"	no — see status.py below

This skill fires single-shot completions: one LLM call per essay × prefix × model, no orchestrator, no workflow. For orchestrator-driven completions (a full rumil workflow per essay, budget-bounded research loop, dedup by config_hash) use rumil-versus-complete. Both write into versus_texts and are pairable in judging — single-shot rows carry the bare model id as source_id; orch rows carry orch:<workflow>:<model>:c<hash8>.

Before any run: check staleness

# run from rumil repo root
uv run python versus/scripts/status.py

Reports current / stale_prefix / stale_prompt / unknown_essay buckets for all three jsonl stores. Exit code 2 + STALE banner = existing rows reference OLD essay text or outdated prompts. Topups against stale rows silently extend that staleness — re-run paraphrases + completions first if the user isn't explicit. Same gate /versus uses in the UI.

Active-set default and --include-stale

Both scripts default to the canonical eval set: current schema_version and not in cfg.essays.exclude_ids — the same gate /versus applies in the UI (currently a 25-essay set).

Pass --include-stale to widen scope to every essay returned by the source fetchers (still minus exclude_ids). This pulls in off-feed or older-schema rows — useful for backfill or debugging, but those won't surface in /versus.

Default scope composes with --essay (AND): --essay forethought__broad-timelines means "that essay, only if it's also in the active set". With --include-stale, --essay is the only filter.

Invocation

Both scripts resolve paths relative to versus/ regardless of cwd. Run from the rumil repo root:

cd /Users/brian/code/rumil && uv run --with httpx --with pydantic --with pyyaml python versus/scripts/run_$OP.py $ARGUMENTS

The --with flags are versus's runtime deps (it's a separate pyproject, not installed in rumil's .venv). If you forget them the scripts fail with ModuleNotFoundError: httpx or similar.

Typical invocations

Flash completion across the active set (the canonical 25-essay set is the default):

--op completions --model google/gemini-3-flash-preview

Top up paraphrases (when prompt is bumped or new essays land):

--op paraphrases

Default configured models come from versus/config.yaml completion.models (flash + gpt-5.4-mini + claude variants). Each entry has a paraphrase: bool flag — paraphrases run only on the subset where paraphrase: true. The old top-level paraphrasing.models list was removed; paraphrasing now only carries enabled: bool.

Override with --model (repeatable).

Filter flags

• --essay <id> (repeatable) — restrict to specific essays.
• --include-stale — opt out of the active-set default; widen to every fetched essay (still minus exclude_ids).
• --limit N — cap number of calls.
• --dry-run — print the plan and exit.

What to surface

Both scripts print:

• [plan] N ... calls (concurrency=K) — count, use before confirming cost.
• [done i/N] <dedup_key> — one line per successful row.
• [skip] <key> — dedup hit, nothing written.
• [err ] <key>: <msg> — failure; other items still run.

Cost

OpenRouter pricing varies by model; rough per-call costs for this workflow's default max_tokens settings:

Model	Completion
google/gemini-3-flash-preview	~$0.002
openai/gpt-5.4-mini	~$0.01
openai/gpt-5.4	~$0.05

The 25-essay active set at flash = ~$0.15 for completions + ~$0.15 per judge for vs-human judgments. Cheap enough to run without confirming. GPT-5.4 at the same scope is ~$3 for completions + $3 per judge — confirm if > ~$10.

Always --dry-run first for anything beyond flash to see the exact plan count; it's free and takes a second.

Dedup & re-runs

Re-runs are free. Each row's key encodes the essay, prefix hash, sources, criterion, judge_model (with :p<hash>:v<N>:s<hash> tags), and order slot. Any change in those inputs produces a fresh row; identical inputs [skip]. So adding a model or criterion to the config and re-running fills gaps without clobbering.

Env

Both keys are needed depending on which models run:

• OPENROUTER_API_KEY — required for any non-claude model (gemini, gpt-5.4, etc.). Routed via OpenRouter.
• ANTHROPIC_API_KEY — required for any claude-* / anthropic/... model. Both run_completions.py and run_paraphrases.py apply the env cascade (versus/.env, then rumil/.env, then process env), matching what run_rumil_judgments.py does, so per-project .env files take precedence over the shell env.

Common gotchas

• Forgetting --with httpx --with pydantic --with pyyaml — the versus subproject isn't in rumil's lock, so a bare uv run python versus/scripts/... fails. The invocation command at the top is the canonical form.
• Stale rows in the UI after a schema_version bump — status.py catches this. The fix is to re-run --op paraphrases + --op completions to write fresh rows under the new keys; old rows stay as archaeology but don't show up in /versus.
• [skip-refusal] for several pairs at startup — that's the judge loader filtering out pairs built from earlier refusal rows in versus_texts. Not an error. Those pairs stay pending until the refusing model produces a non-refusal completion.

After a run

Suggest the user check /versus/results?essay=<id> (via the running rumil frontend) to see the new rows, or /versus/inspect?essay=<id> for pair-level breakdown with reasoning. Both pages filter by the same staleness gate status.py reports, so they should agree.