| name | fix-department |
| description | Get a department or gem all the way to 100% conformance using per-cop validation and generated corpus reports, never manual README/docs edits. |
| allowed-tools | Bash(*), Read, Write, Edit, Grep, Glob, Task, TeamCreate, TaskCreate, TaskUpdate, TaskList, TaskGet, SendMessage |
Fix Department — Gem-Level 100% Conformance
This skill targets a specific department or gem (for example Naming,
rubocop-performance, rubocop-rspec, or rubocop-rails) and fixes ALL
diverging cops until it reaches 100% conformance.
docs/corpus.md is an autogenerated corpus report. The conformance/corpus
section in README.md is also autogenerated. Never edit those generated
sections directly. Other non-generated README.md content may still be edited
normally when appropriate. Treat the generated scorecard rows as read-only and
refresh them only through the CI corpus-oracle workflow/PR. Local dry-run
checks are fine, but never run a local generator/script that writes README.md
or docs/corpus.md.
Final stop condition:
- The target shows 100.0% / 0 FP / 0 FN in the generated corpus reports
(
README.md, docs/corpus.md, and related artifacts), never via manual edits.
- Linux CI/corpus oracle agrees with the local result.
Per-cop
check_cop.py --rerun results are intermediate count-based gates only.
They are necessary, but they are NOT sufficient to end /fix-department.
When the corpus oracle has concrete FP/FN examples for a cop, use
check_cop.py --rerun to verify before treating that cop as done locally.
Persistence
Default behavior for this skill is to keep iterating until the target is
actually done or truly blocked. Do not stop just because a batch was committed,
landed onto main, or pushed.
$fix-department --loop makes this explicit:
- Keep looping back to Phase 1 after each accepted cop fix, integration step,
or checkpoint.
- Do not ask non-blocking clarifying questions. Make the reasonable default
assumption and continue.
- Only stop for:
- target complete (
README.md / docs/corpus.md CI-owned scorecard at 100%)
- a real blocker or risky ambiguity that could damage unrelated work
- an explicit user interrupt, redirect, or wrap-up request
- If the user asks for an intermediate action such as commit / land / push,
treat that as a sub-step inside the loop, then resume the fix workflow.
Turn-Boundary Contract
Codex skill activation is turn-scoped, but /fix-department --loop should be
treated as a standing task across the thread.
- If thread history shows an active
/fix-department --loop run and the user
asks for status, commit, land, cherry-pick, rebase, push, or verification,
treat that request as a sub-step inside the same loop.
- After satisfying that sub-step, resume the interrupted phase or Phase 1 in
the same turn unless the user explicitly paused, stopped, or redirected the
run.
- Do not treat a successful commit, cherry-pick, rebase, or report as the end
of the loop by itself.
- If you truly cannot resume in the same turn, say that the loop is still
active and tell the user to run
$fix-continue --loop rather than silently
ending the workflow.
Worktree isolation depends on platform: On macOS (local dev) and Linux devcontainers,
use isolation: "worktree" for teammates. On ephemeral cloud VMs (not devcontainers), do NOT
use worktree isolation — teammates commit directly to the current branch. Cloud VM worktrees
get cleaned up on agent timeout, losing committed work.
To detect devcontainers: check if $USER is vscode and /.dockerenv exists.
Ephemeral cloud VMs typically run as root or a CI-specific user, not vscode.
Workflow
Branch Guard (cloud VMs only)
On ephemeral cloud VMs (not macOS, not devcontainers), never commit directly to
main. Before starting any work, check the environment and current branch:
uname -s
If you are on a cloud VM AND on main, create a feature branch immediately:
git checkout -b claude/fix-department-<target>-$(openssl rand -hex 3)
All commits go to this branch. Integration to main happens via PR (Phase 6).
On macOS or devcontainers, committing directly to main is acceptable.
Phase 0: Assess (you do this)
-
Always start with the scoreboard. Run the script and paste its full output verbatim
to the user (the table IS the primary output — do not summarize or skip it):
python3 .claude/skills/fix-department/scripts/gem_progress.py --summary
The script automatically detects cops fixed since the last corpus oracle run by
scanning git commit messages, and shows them as "Fixed (pending corpus confirmation)"
so the scoreboard reflects reality between corpus runs.
The script also prints a recommendation at the bottom.
-
Determine the target scope.
- If the user specified a core
rubocop department (for example Naming,
Layout, Lint, Metrics, Style), use the owning gem only for triage,
but treat the department row in README.md / docs/corpus.md as the
read-only scorecard.
- If the user specified a plugin gem (
rubocop-performance,
rubocop-rspec, rubocop-rails, etc.), the gem is both the triage scope
and the final scorecard.
-
If no gem/department was specified, after showing the table, let the user pick one.
-
Once a target is chosen (by user or from args), run the deep-dive:
python3 .claude/skills/fix-department/scripts/gem_progress.py --gem <gem-name>
Also inspect the current checked-in target row in README.md and
docs/corpus.md. Never patch the generated conformance section by hand.
That row, not the per-cop rerun output, is the lagging scorecard that must
eventually be refreshed to 100% by the CI corpus-oracle workflow/PR.
-
Bootstrap the environment before reducers/tests/check-cop:
git submodule update --init --recursive
- If
bench/corpus/vendor/bundle/ is missing for the active Ruby version, install it:
cd bench/corpus && BUNDLE_PATH=vendor/bundle bundle install
- Do not treat
vendor/corpus/ as mandatory bootstrap. The smoke test does not
need it, and cloud environments should not prefetch the full corpus checkout.
- To inspect corpus source files, fetch them directly from GitHub using repo info
from
bench/corpus/manifest.jsonl (no cloning needed):
gh api repos/OWNER/REPO/contents/PATH?ref=SHA --jq '.content' | base64 -d
- Missing vendor rubocop submodules are setup failures.
- Build isolation (required when multiple agents share the repo):
Multiple
/fix-department sessions may run in parallel on the same repo. Each
session MUST use a unique target directory and set NITROCOP_BIN to avoid
competing cargo build invocations:
export CARGO_TARGET_DIR="target-${target_name}"
cargo build --release
cp "${CARGO_TARGET_DIR}/release/nitrocop" "/tmp/nitrocop-${target_name}"
export NITROCOP_BIN="/tmp/nitrocop-${target_name}"
All Python scripts (check_cop.py, corpus_smoke_test.py,
reduce_mismatch.py) honor NITROCOP_BIN to find the binary. Copying to /tmp/
prevents other agents' builds from overwriting it. Rebuild and re-copy after code
changes (Phase 4 verification).
-
Show the user the target status and confirm it.
Phase 1: Plan Batch (you do this)
From the deep-dive output, select up to 4 cops for this batch. Priority order:
- FP-only cops (FP>0, FN=0) — pure false alarms, usually straightforward to fix
- Both FP+FN cops with highest FP — fix the FP side first
- FN-only cops (FP=0, FN>0) — missing detections, lower priority but needed for 100%
Skip Layout/ alignment cops unless they're the only ones remaining (complex multi-line state machines).
Establish ground truth first — before reading cop source or running reducers,
run these commands in parallel for each selected cop:
git log --oneline -5 -- src/cop/<dept>/<cop_name>.rs
python3 scripts/check_cop.py Department/CopName --verbose
Compare the latest cop commit date against the corpus oracle run date (shown in
check_cop.py output). If the oracle ran AFTER the latest fix, its FP/FN
numbers are the real remaining gap — not stale data from before the fix. This
tells you exactly what still needs fixing before you spend time reading source
or running reducers. Do NOT read the full cop source (which may have hundreds
of lines of investigation comments) until you know the current numbers and
which repos are diverging.
For each selected cop, investigate the FP/FN pattern:
python3 scripts/check_cop.py Department/CopName --examples --fp-only --limit 10
python3 scripts/check_cop.py Department/CopName --examples --fn-only --limit 10
check_cop.py prefers embedded snippets from corpus-results.json. If that
is insufficient, fetch full source files directly from GitHub using repo info
from bench/corpus/manifest.jsonl:
gh api repos/OWNER/REPO/contents/PATH?ref=SHA --jq '.content' | base64 -d
Synthetic-only cops (zero corpus activity): If check_cop.py shows no results, the cop
only has data in the synthetic corpus. Investigate using:
python3 -c "import json; d=json.loads(open('bench/synthetic/synthetic-results.json').read()); [print(c) for c in d['by_cop'] if c['cop']=='Department/CopName']"
The synthetic corpus files at bench/synthetic/project/ are handcrafted trigger patterns — read
the relevant source file to understand the expected behavior. Run python3 bench/synthetic/run_synthetic.py --verbose
to re-verify after fixing.
Run the delta reducer on up to 3 examples per cop (mix of FP and FN) to get minimal reproductions
(corpus cops only — not applicable to synthetic-only cops):
python3 scripts/reduce_mismatch.py Department/CopName repo_id filepath:line
python3 scripts/reduce_mismatch.py Department/CopName repo_id filepath:line --type fn
Pick examples from different repos when possible. The reduced files (typically 5–20 lines) go
to /tmp/nitrocop-reduce/ — read them and include them in the teammate prompt.
Summarize: cop name, FP/FN counts, minimal repro(s), root cause hypothesis.
When no example locations are available (check_cop.py shows counts but no
file paths, and no corpus repos cloned): still dispatch teammates, but tell them
upfront that examples are unavailable. For low-divergence cops (FP+FN ≤ 2), the
FP/FN may be a corpus artifact — tell teammates to compare nitrocop vs vendor
RuboCop source for edge cases, add any discovered test coverage, and document
findings. Do NOT expect teammates to web-search for specific repo files.
Phase 2: Dispatch (you do this)
-
Create a team with a unique name that includes the target:
TeamCreate(team_name="fix-<gem-or-department>", description="Bring <gem-name> to 100% conformance")
For example: fix-layout, fix-rubocop-performance, fix-naming.
Do NOT use the generic name fix-department — multiple agents running
/fix-department simultaneously will collide on the same team namespace,
causing teammates to be overwritten and messages to be misrouted.
-
Create tasks for each cop fix.
-
Spawn one teammate per cop using the Agent tool. Critical settings:
subagent_type: "general-purpose" — needs full edit/bash access
team_name: "fix-<gem-or-department>" — must match the name from step 1
mode: "bypassPermissions" — teammates need to run cargo test etc.
- macOS and Linux devcontainers: add
isolation: "worktree" so each teammate gets its own git worktree
- Ephemeral cloud VMs (not devcontainers): do NOT use
isolation: "worktree" — teammates commit directly
to the current branch (worktrees get cleaned up on timeout, losing work)
-
Each teammate prompt MUST include:
- The exact cop name (e.g.,
Performance/AncestorsInclude)
- The FP/FN counts and root cause hypothesis from your investigation
- The minimal repro(s) from the delta reducer — paste the reduced Ruby source directly
- Whether to focus on FP fixes, FN fixes, or both
- A reminder that parallel-agent activity often leaves unrelated local modifications; those files are off-limits
- The teammate workflow (Phase 3 below) — paste the full instructions
Phase 3: Teammate Workflow (paste this into each teammate's prompt)
You are fixing false positives/negatives in a single nitrocop cop to bring its gem
to 100% conformance. Follow the CLAUDE.md rules strictly.
**NEVER use git stash or git stash pop.** Commit directly to the current branch.
Parallel-agent activity is common. If you see unrelated modified files, do not edit/revert them.
## Step 0: Verify your working directory
Run `git rev-parse --show-toplevel` to confirm you're in the right repo.
If vendor RuboCop submodules are missing, run `git submodule update --init --recursive`.
If `bench/corpus/vendor/bundle/` is missing for the active Ruby version, install it with
`cd bench/corpus && BUNDLE_PATH=vendor/bundle bundle install`.
Do **not** clone the full `vendor/corpus/` checkout by default; only clone targeted repos
if the prompt/examples require local corpus source context.
## Steps
1. **Read the cop source** at `src/cop/<dept>/<cop_name>.rs`
Read the vendor RuboCop spec at `vendor/rubocop*/spec/rubocop/cop/<dept>/<cop_name>_spec.rb`
**Check for existing investigation comments** (marked with "Known false positives" or
"reverted") — these document previously attempted fixes that regressed on corpus
validation. Do NOT repeat the same approach. Either find a different root cause or
extend the prior approach to avoid its documented failure mode.
2. **Understand the FP/FN pattern** from the examples provided in your prompt.
If needed, fetch source files from GitHub using repo info from `bench/corpus/manifest.jsonl`:
`gh api repos/OWNER/REPO/contents/PATH?ref=SHA --jq '.content' | base64 -d`
**DO NOT run nitrocop or rubocop directly** — not on corpus repos, not on ad-hoc
files in `/tmp/`, not anywhere outside the test fixtures. Running `nitrocop` on
arbitrary paths fails ("No lockfile found") and wastes tokens. The ONLY ways to
verify cop behavior are:
- **Unit tests**: add patterns to `offense.rb` / `no_offense.rb` fixtures and run
`cargo test --lib -- <cop_name_snake>` — this is fast, reliable, and self-documenting.
- **Corpus validation**: `check_cop.py --rerun` for aggregate count regression on
cops you modified; `check_cop.py --verbose` for untouched cops when the latest
corpus oracle run is current.
- **Synthetic corpus**: `python3 bench/synthetic/run_synthetic.py --verbose` for
synthetic-only cops.
Never create test Ruby files outside the fixture directories.
**Bail-out rule:** If no example locations are available (check_cop.py shows
counts only, no file paths), no corpus repos are cloned locally, and the minimal
repro examples in the prompt are insufficient to identify a concrete bug: stop
investigating after analyzing the cop source + vendor RuboCop source. Compare the
two implementations for edge case differences. If you cannot identify a concrete
code bug within a reasonable effort, document your findings in the cop's doc comment
and report back that the FP/FN is likely a corpus artifact (stale cache, config
resolution, or line-number mismatch). Do NOT web-search for specific repo source
files or try to reconstruct corpus examples from GitHub — this rarely succeeds and
wastes significant time.
3. **Add test cases (TDD)**:
- For FP fixes: add the false-positive pattern to `tests/fixtures/cops/<dept>/<cop_name>/no_offense.rb`
- For FN fixes: add the missed detection to `tests/fixtures/cops/<dept>/<cop_name>/offense.rb`
- Run `cargo test --lib -- <cop_name_snake>` to verify the test FAILS
- Use debug mode (no `--release`) for fast TDD iteration (~8s first run, <1s incremental)
4. **Fix the cop implementation** in `src/cop/<dept>/<cop_name>.rs`
- Iterate with `cargo test --lib -- <cop_name_snake>` (debug mode) until tests pass
5. **Add investigation comment** to the cop's struct as a `///` doc comment. This is
**required** even if you fully fixed the cop — document what the original FP/FN were,
what you fixed, and any remaining gaps with root causes. Example:
```rust
/// ## Corpus investigation (YYYY-MM-DD)
///
/// Corpus oracle reported FP=X, FN=Y.
///
/// FP=X: Fixed by <description>. Commit <sha>.
/// FN=Y: <Fixed/remaining/expected behavior — with root cause>.
pub struct CopName;
-
Verify (pre-commit, release mode):
cargo test --release -p nitrocop --lib -- <cop_name_snake> — all tests pass
cargo fmt
cargo clippy --release -- -D warnings
- Important: Fixture tests bypass config (Include patterns, Enabled status).
A passing test does NOT guarantee the binary fires on real files. If fixing
an FN, also verify the binary detects the pattern — see AGENTS.md Key
Constraints for Prism block body shape gotchas (BeginNode vs StatementsNode,
itblock/numblock).
-
Commit your fix:
git add src/cop/<dept>/<cop_name>.rs tests/fixtures/cops/<dept>/<cop_name>/
git commit -m "Fix <Department/CopName> false positives/negatives
<one-line description of what was wrong>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>"
Stage only files for this cop fix. Do not include unrelated modified files.
-
Report back via SendMessage with:
- What the root cause was
- What you changed
- Whether tests pass
- The commit SHA
- If you could NOT fix it: explain why and whether it should be deferred
### Phase 4: Collect + Loop (you do this)
1. Wait for all teammates to report back.
2. For each completed fix:
- **Worktree mode (macOS / devcontainer):** note the worktree branch name from the Task result
and cherry-pick the commit into your working branch
- **No-worktree mode (ephemeral cloud VM):** commits are already on the current branch — no cherry-picking needed
3. **Rebuild the isolated binary** (must use the session's unique target dir):
```bash
cargo fmt
cargo clippy --release -- -D warnings
cargo test --release
# Rebuild to the session's isolated target dir and copy to /tmp
CARGO_TARGET_DIR="target-${target_name}" cargo build --release
cp "${CARGO_TARGET_DIR}/release/nitrocop" "/tmp/nitrocop-${target_name}"
export NITROCOP_BIN="/tmp/nitrocop-${target_name}"
python3 scripts/corpus_smoke_test.py --binary "$NITROCOP_BIN"
Run the corpus smoke test once per batch, not after every cop. It is the cheap
systemic guard for file discovery, config/plugin loading, directive handling,
and other cross-cop regressions that per-cop check_cop.py reruns will not catch.
It does not require vendor/corpus/.
-
Verify each fixed cop with the right acceptance gate (with NITROCOP_BIN set):
python3 scripts/check_cop.py Department/CopName --verbose --rerun
python3 bench/synthetic/run_synthetic.py --verbose
Corpus validation is the intermediate acceptance gate for corpus-backed
cops — unit tests passing is necessary but NOT sufficient. The goal is
PASS: aggregate offense count matches RuboCop for this cop. PASS: no new excess vs CI nitrocop baseline alone is not enough — the cop may still
have remaining mismatches. For synthetic-only cops, re-run the synthetic
benchmark and inspect that cop's entry in
bench/synthetic/synthetic-results.json; check_cop.py has no signal there.
Use --rerun only for cops changed in this batch; for untouched corpus cops,
use artifact mode (--verbose).
Do not stop here. Exact-match per-cop reruns can still leave the target
department/gem below 100% once the corpus reports are regenerated.
-
Handle regressions: When CI reports a regression, check CI logs first:
gh run view <run-id> --job <job-id> --log 2>&1 | grep -A 3 "FAIL:"
This immediately names the regressed repo(s) — do NOT re-run check_cop.py --rerun locally to find a regression that CI already identified.
If a fix increases FP count (even if unit tests pass), revert
the code change but add a detailed investigation comment to the cop source file
documenting what was tried, exactly where code changed, acceptance-gate numbers
before/after, why it regressed, and what a correct fix would need. Use:
-
Re-run the gem deep-dive to see updated progress:
python3 .claude/skills/fix-department/scripts/gem_progress.py --gem <gem-name>
Note: This still reads the original corpus data. Per-cop verification via
check_cop.py or run_synthetic.py gives the ground truth for fixed cops.
-
If the user explicitly asks for a corpus-report refresh, use the real
generation path rather than editing files by hand:
cargo run --release --bin bench_nitrocop -- conform
bench_nitrocop -- conform refreshes bench/conform.json and
bench/results.md only. It does NOT update the generated conformance section
in README.md or docs/corpus.md. When that generated content needs to
move, do not run local generators against those files. README.md and
docs/corpus.md are CI-owned artifacts and may only be updated by the CI
corpus-oracle workflow/PR. Local dry-run comparisons are allowed only if
they do not write those files. Never patch that generated content directly.
/fix-department is not done until the generated reports show the target at
100.0% / 0 FP / 0 FN.
-
After a CI corpus-report refresh, inspect the target row in README.md and
docs/corpus.md:
- If the row is still below 100%, go back to Phase 1 even if the modified
cops passed
check_cop.py --rerun.
- If the row is 100% locally but Linux CI/corpus oracle is not yet green or
disagrees, treat that as a parity bug and keep investigating. Do not
declare the department/gem complete yet.
- If local and Linux CI both show 100%, the target is done.
-
If diverging cops remain, go back to Phase 1 for the next batch.
-
For cops that teammates couldn't fix, decide whether to:
- Retry with more context in the next batch
- Defer with a documented reason
- Document ALL investigation outcomes as
/// comments on the cop's struct in its
source file. This is mandatory for EVERY cop in the batch — including:
- Cops that were fixed but have remaining FP/FN (document what was fixed and why
the remaining gaps exist)
- Cops that were investigated but need no code fix (e.g., FPs caused by encoding
differences, file-drop noise, config artifacts)
- Cops that were deferred (document why and what a fix would need)
- Cops with FN from
standard:disable handling or other config edge cases (these are still bugs to fix, not acceptable differences)
This prevents future investigators from repeating the same analysis. Do not
consider a cop "done" until it has an investigation comment, even if the fix itself
is already committed. Use:
Phase 5: Declare Done (you do this)
When the generated target row is at 100% and Linux CI agrees:
-
Run full verification:
cargo fmt
cargo clippy --release -- -D warnings
cargo test --release
python3 scripts/corpus_smoke_test.py --binary target/release/nitrocop
-
Report to the user:
- Department/gem name and total cops
- How many cops were fixed (with FP/FN reduction)
- How many cops were already perfect
- Any deferred cops with reasons
- Verification status:
local
fmt / clippy / test
per-cop check-cop / run_synthetic
corpus-report generation status
README.md / docs/corpus.md target row after CI refresh
Linux CI parity status
- Summary: ": 100% conformance (N cops, M fixed in this session)"
-
Do not declare success before the generated corpus reports and Linux CI are
both green. If CI confirmation is still pending, report that the target is
locally green but not complete yet.
Phase 6: Integrate Back to Main
Do not leave retained progress only in a feature branch.
-
Ensure all retained progress is committed:
- Accepted cop fixes: one commit per cop (preferred).
- Useful investigation artifacts retained in repo (for example, reverted-attempt notes): separate commit.
-
macOS / devcontainer (direct integration):
Integrate those commit(s) into main immediately (unless the user explicitly says not to).
git checkout main
git cherry-pick <sha1> [<sha2> ...]
Cloud VM (branch integration):
If you created a feature branch in the Branch Guard step, push it:
git push -u origin HEAD
-
Verify integration:
git log --oneline -n 10
git status --short --branch
-
Report exactly what was integrated (commit SHA(s) and short subjects, or PR URL).
-
If there is truly no repo-retained progress, explicitly report that no commit was made.
Arguments
/fix-department — show the scoreboard, recommend a gem, and ask which to target
/fix-department Naming — target the Naming department directly and keep going until its generated corpus-report rows hit 100%
/fix-department rubocop-performance — target rubocop-performance directly
/fix-department rubocop-rspec — target rubocop-rspec directly
/fix-department --input /path/to/corpus-results.json — use local corpus file
How to Choose the Next Gem
The scoreboard (gem_progress.py --summary) shows per-gem stats. Prioritize by:
- Zero uncovered cops — only gems where every cop has either corpus coverage
or synthetic coverage can claim true 100% conformance. The "Untest" column in the
scoreboard shows cops missing from both.
- Zero synthetic divergence — the "SynFP" and "SynFN" columns show FP/FN from
cops that only have synthetic coverage (no corpus activity). A gem showing
"corpus 100%, N syn diverge" in the Status column is NOT at true 100% — the
synthetic-only cops must also be fixed.
- Fewest diverging cops — less work to complete the gem. The "Dvrg" column shows this.
- Adoption value — rubocop-performance is the most commonly added plugin, so completing
it has more impact than rubocop-factory_bot, even if factory_bot is smaller.
- FP-free first — a gem with 0 FP but some FN is already safe to adopt (no false alarms).
Fix FNs later for completeness.
The scoreboard's Status column distinguishes "100% conformance" (all corpus AND synthetic
cops perfect) from "corpus 100%, N syn diverge" (corpus cops perfect but synthetic-only
cops have FP/FN). Always fix synthetic divergence before declaring a gem complete.