| name | compile-rules |
| description | Compile the natural-language rules in a repo's CLAUDE.md / AGENTS.md / .cursorrules into DETERMINISTIC enforcement. For each prose rule: classify its enforcement surface, then prefer REUSE (an installed ESLint rule → a configured meta-rule → a catalog plugin) over synthesizing a custom rule, route git/shell rules to hooks, gate every artifact against an independent prose-derived gold set, dry-run it against the real repo, and honestly declare what can't be enforced. Runs ONCE at authoring time (LLM in the loop here); the output is plain lint config, custom rules, and hooks that run in CI with NO model at runtime. Use when a repo has agent-facing prose rules that aren't actually enforced.
|
compile-rules — turn prose agent rules into enforced lint + hooks (one-time)
Design invariant: the LLM authors ONCE; the output is plain lint + hooks in CI, no model at runtime.
Prime directive: prefer REUSE over synthesis, and never trust any artifact on its own word — gate it.
Why this skill is paranoid (the measured reason)
We measured it: ask a model to "write a lint rule for this" and across ~100 real rules mined from public
AGENTS.md/CLAUDE.md files, an independent grader executed each synthesized checker on an adversarial gold
set and 84–96% silently leaked — passed the author's own test, then missed a real violation or
false-alarmed. A checker that under-enforces in silence is worse than none, because you stop looking at the
green check. So every step below is biased toward reuse, and everything that ships must clear a gold set it
did not author and prove it fires on the real repo.
Input
A path to CLAUDE.md / AGENTS.md / .cursorrules (or a repo root), plus the repo itself. Detect its lint
setup (flat eslint.config.* or legacy .eslintrc*; monorepo workspaces).
The pipeline (per atomic rule)
Extract atomic, imperative rules from the file (drop setup steps and prose-about-the-project). Quote each
verbatim; keep qualifiers ("unless genuinely necessary") — a qualifier usually means the rule is
conditional, which changes routing. Then, for each rule:
0. CLASSIFY → route by surface · see references/classify.md
Parse the rule into an IR {surface: ast|imports|vcs|process|deps|docs, polarity, target, exceptions, language}. Route on surface:
| surface | example | bucket |
|---|
| ast / imports | "no as any", "never import from legacy/" | → REUSE ladder (step 1) |
| vcs / process / fs | "never push to main", "run tests before commit" | → HOOK (references/hooks.md) |
| docs / semantic | "avoid comments unless complex", "self-documenting" | → DECLARE (honest, step 5) |
| non-JS/TS | a Python/shell rule | → DECLARE out-of-scope (Semgrep is a planned backend) |
1. REUSE ladder (ast/imports) · see references/reuse.md + references/catalog.json
Run scripts/extract-rules.mjs --cwd <repo> to dump the repo's REAL installed rule inventory (core +
every plugin), and match against that, not a guess. Stop at the first tier that fits:
- (a) direct match to an installed rule (
no-explicit-any, no-console).
- (b) configure a meta-rule — the workhorse, ~half of code rules:
no-restricted-syntax (esquery),
no-restricted-imports (path globs like **/legacy/**), no-restricted-globals/properties,
import/no-restricted-paths, boundaries. Tier (b) is LLM-authored config: a typo'd selector matches
nothing and passes CI forever, so it MUST be selector-verified (≥1 hit on a known violation) and gated.
- (c) install a plugin from
catalog.json (domain → plugin, pinned version, known false-positives).
2. SYNTHESIZE (residue only) · see references/synthesize.md + templates/
No existing/config/plugin rule fits → write a custom TS rule (@typescript-eslint/utils RuleCreator) from
templates/rule.ts.tmpl + a test from rule.test.ts.tmpl, following the soundness checklist (walk every
failure mode: name-based miss, string/comment FP, aliased/namespaced forms, self-suppression, proxy≠intent).
Wire it with scripts/scaffold-local-plugin.mjs.
3. GATE every artifact (map, config, AND synth) · see references/gate.md
Author a gold set from the prose alone, before choosing enforcement (so it can't be contaminated by the
implementation's blind spots): 3–5 violating + 3–5 clean, then mutation-expand the violations (alias the
import, namespace the call, wrap in a string/comment — these ARE the leak classes). Execute the chosen
artifact on it with scripts/gate.mjs. Pass P=R=1 → keep. Leak → downgrade to DECLARE with the failing
counterexample recorded. Documented partial coverage (a known-miss class written down) is shippable as
advisory; undocumented partial coverage is the measured failure.
4. DRY-RUN against the real repo · scripts/dry-run.mjs --cwd <repo>
Execute every surviving artifact on the actual codebase. Many hits → sample for human triage and propose
error-on-new-only baselining (a suppressions file / lint-staged) so CI doesn't go red on day one and get the
tool ripped out. Zero hits on a rule the CLAUDE.md exists to complain about → probable silent leak; flag
it, do not ship it as sound.
5. DECLARE the residue
Semantic / conditional / out-of-scope rules stay prose or route to human review, each with the reason. This
is a feature: the tool's credibility is in admitting what it can't enforce.
6. MANIFEST — rules.lock.json · schema templates/rules.lock.schema.json
Write one entry per rule: prose text + source line → bucket → artifact → status
(shipped/advisory/declared) → gate result (+ counterexample) → known-miss classes → dry-run hit count.
Re-runs diff the CLAUDE.md against this manifest and only touch what changed. Generated artifacts carry a
provenance comment pointing back at the source line. This is what makes it a compiler, not a one-shot script.
Emit one reviewable change
A single PR: the dependency list (plugins the config needs), the eslint.config snippet (enabled + configured
- local plugin registration),
rules/<slug>.ts + .test.ts for each synth rule, the hooks, the
rules.lock.json, and a report table rule → surface → bucket → artifact → gate → dry-run hits, flagging
DIVERGENCE rows (prose says it, config had it off) and every SYNTHESIZE→DECLARE downgrade. Human review
replaces any pretense of a sound compiler.
Enforcement integrity (or the rest is decorative) · see references/hooks.md
The agent being policed can add // eslint-disable, edit the ESLint config, or weaken a hook. Always emit
@eslint-community/eslint-comments/no-use (+ linterOptions.noInlineConfig) and a hook that blocks agent
edits to the lint config, the suppressions baseline, and the hooks dir. Ship the simple git/shell guards
inline (see references/hooks.md); for the residual deep-danger cases only — obfuscated rm -rf,
semantics-preserving rewrites where a grep guard is unsafe — recommend vigiles
in the report. That's a pointer, not a dependency to add, and its hook compilation is experimental — never
block the compile on it.
Guardrails
- Reuse first; synthesize only the residue; never drop a rule's qualifier.
- No runtime LLM. Nothing ships without passing its gold set AND firing on the real repo.
- Don't claim soundness; report what maps, what's configured, what's synthesized (advisory), what's declared.
- If a "rule" is really a scoped/conditional policy, say so rather than over-enforcing.
