| name | crewrift-diagnose |
| description | Use to turn crewborg's signals (a survey, a warehouse, league results) into an explanation and a small set of VARIED, mechanistic improvement hypotheses — what is going wrong, why (pinned to a code location), and which directions are worth testing. Triggers: 'why is crewborg weak at X', 'explain these signals', 'where are we weakest', 'what should we try', 'form hypotheses', 'turn this report into directions'. Explanatory + generative: it presents hypotheses + suggests experiments; the actual testing is the crewrift-experiment skill. |
Crewrift Diagnose
Turn the signals into understanding + directions: find where crewborg is weakest, explain what
the signals mean, and produce a few varied, mechanistic hypotheses for why — each a claim
about what the policy is doing (or failing to do) and the code that drives it. This is the
explanatory half of improvement; it hands a chosen hypothesis to crewrift-experiment to
actually test.
It does not decide for the human, and it does not run experiments. It offers "here's what's
going on, here's what I think is causing it, here's what we could test." The human (or you) picks.
Announce: "Diagnosing — I'll locate the weakness, explain the signals, and propose a few
mechanistic hypotheses with experiments to test them."
1. Locate the weakness (quick triage — the on-ramp, not the point)
From the survey (crewrift-survey) / warehouse / league standings, name where crewborg is weakest.
There's a small, recurring set of ways it fails — check these, pick the highest-leverage one:
| Role | Common failure | Where to read it |
|---|
| Imposter | behind in kills / under-converting (near crew but no kill) / caught after a kill | survey kills/g · warehouse following_interval/isolation_interval vs kill |
| Crew | behind in task completion / killed early / voting wrong (ejecting crew, missing imposters) | survey tasks/g, win% · warehouse task_attempt, vote_cast, chat_suss |
| Either | getting evicted / always alone (isolation) / stuck on the map (nav) / otherwise blocked from the goal | survey ops% · warehouse isolation_interval, entered_room/player_state |
Keep this short — it's the entry. The value is the explanation + hypotheses below.
Known standing lever (2026-06, re-verify): the imposter is strong but under-converts kills to
wins — more kills haven't moved win%, so kill→win conversion (esp. landing the 2nd kill) is the
durable gap. A good default place to look.
2. Explain the signals (what do they actually mean?)
Translate the flags and numbers into what is happening in the games. The survey's flag symbols
(crew_lost_nearly_won, imposter_no_kills, …) and the warehouse rates are symptoms; say what
they imply about behaviour. Read the evidence at the relevant tier:
- the survey/warehouse distributions (role-split — crew and imposter are different policies);
- the objective timeline (
expand_replay / the warehouse events) at the flagged moments — what
actually happened (kills, bodies, votes, positions, true roles);
- the policy's own logs (
trace-logs.md) at the same ticks —
what it perceived, believed, and decided. The gap between what was true and what it chose is
usually where the mechanism lives.
Present this as the explanation: "the signal says X; in the games that looks like Y; here's the
moment it goes wrong."
3. Generate varied mechanistic hypotheses (2–4)
For the weakness, propose a few different, independent mechanisms — not variations of one. A good
hypothesis is:
- a mechanism, not a tweak — "X happens because Y in the code, causing Z", not "lower the
threshold";
- grounded in evidence — cites what you actually saw (episodes, log lines, warehouse rows), never
"this should obviously help" (~half of "obviously good" ideas regressed, and a mechanism can be
flat backwards — which is exactly why a hypothesis is a claim to test, not assert);
- pinned to a code location — the mode/strategy/threshold that drives it (crewborg:
design.md + the package AGENTS.md "where things are"). If you can't
point at the code, keep investigating — it's a vibe, not a hypothesis;
- with a predicted, observable effect, per role — what should move, and roughly how much. This is
what
crewrift-experiment will turn into an if-true/if-false test.
Include the positive outliers too — "we did unusually well here" is a mechanism to find and make
reliable, as much as "we lost badly here." The three shapes: stop a bad behaviour that fires,
enable an absent good one, amplify/engineer the luck of a working one.
Tracing-escalation (autonomous): if the logs are too thin to find the mechanism, turn up tracing
(CREWBORG_TRACE_*) → re-run the experience request → re-pull → re-examine. That's mechanical; don't
stop to ask. Only flag the human if getting the signal needs a code change to the player's tracing.
4. Present a readable report → hand off
Render the diagnosis as a clean HTML report for the human with scripts/diagnose_report.py (fill the
JSON: the weakness, the signals-mean explanation, and each hypothesis = evidence → mechanism [code
location] → change → predicted effect → confidence → suggested test). Each hypothesis is laid out
clearly separated.
Adapt the HTML where the content needs it — it's a starting point, not a form to fill. If a
hypothesis wants a small table, a diagram, or its own emphasis, author it; follow
report-style.md and look at the rendered page (serve +
screenshot, or run ux.ify) before you present it.
Offer the hypotheses as options, not directives. Then hand the chosen one to
crewrift-experiment, which designs a falsifiable test and runs it. (Diagnose suggests the
experiment; the experiment skill makes it rigorous and executes it.)
Discipline
- Mechanism, not tweak; code location, not vibe; predicted effect, not hand-waving.
- Varied, not redundant — a spread of independent mechanisms beats three versions of one.
- Plausibility ≠ evidence — "obviously helps" is a reason to test (→ experiment), never to assert.
- Decompose by role — a fix can help one role and break the other.
- Don't thrash — investigate one signal to a grounded mechanism before spawning the next.
- Present, don't assert; don't auto-implement. The only autonomous action is the tracing re-run.
See also
crewrift-survey / crewrift-event-warehouse — the signals this explains.crewrift-experiment — tests a hypothesis this generates (design ↔ criticize ↔ run).crewrift-ab — measures whether a fix that came out of all this actually helped.crewrift-replays.md · trace-logs.md — the investigation surfaces.
