compile-rules

name

compile-rules

description

Compile the natural-language rules in a repo's CLAUDE.md / AGENTS.md / .cursorrules into DETERMINISTIC enforcement. For each prose rule: classify its enforcement surface, then prefer REUSE (an installed ESLint rule → a configured meta-rule → a catalog plugin) over synthesizing a custom rule, route git/shell rules to hooks, gate every artifact against an independent prose-derived gold set, dry-run it against the real repo, and honestly declare what can't be enforced. Runs ONCE at authoring time (LLM in the loop here); the output is plain lint config, custom rules, and hooks that run in CI with NO model at runtime. Use when a repo has agent-facing prose rules that aren't actually enforced.

compile-rules — turn prose agent rules into enforced lint + hooks (one-time)

Design invariant: the LLM authors ONCE; the output is plain lint + hooks in CI, no model at runtime. Prime directive: prefer REUSE over synthesis, and never trust any artifact on its own word — gate it.

Why this skill is paranoid (the measured reason)

We measured it: ask a model to "write a lint rule for this" and across ~100 real rules mined from public AGENTS.md/CLAUDE.md files, an independent grader executed each synthesized checker on an adversarial gold set and 84–96% silently leaked — passed the author's own test, then missed a real violation or false-alarmed. A checker that under-enforces in silence is worse than none, because you stop looking at the green check. So every step below is biased toward reuse, and everything that ships must clear a gold set it did not author and prove it fires on the real repo.

Input

A path to CLAUDE.md / AGENTS.md / .cursorrules (or a repo root), plus the repo itself. Detect its lint setup (flat eslint.config.* or legacy .eslintrc*; monorepo workspaces).

The pipeline (per atomic rule)

Extract atomic, imperative rules from the file (drop setup steps and prose-about-the-project). Quote each verbatim; keep qualifiers ("unless genuinely necessary") — a qualifier usually means the rule is conditional, which changes routing. Then, for each rule:

0. CLASSIFY → route by surface · see references/classify.md

Parse the rule into an IR {surface: ast|imports|vcs|process|deps|docs, polarity, target, exceptions, language}. Route on surface:

surface	example	bucket
ast / imports	"no as any", "never import from legacy/"	→ REUSE ladder (step 1)
vcs / process / fs	"never push to main", "run tests before commit"	→ HOOK (references/hooks.md)
docs / semantic	"avoid comments unless complex", "self-documenting"	→ DECLARE (honest, step 5)
non-JS/TS	a Python/shell rule	→ DECLARE out-of-scope (Semgrep is a planned backend)

1. REUSE ladder (ast/imports) · see references/reuse.md + references/catalog.json

Run scripts/extract-rules.mjs --cwd <repo> to dump the repo's REAL installed rule inventory (core + every plugin), and match against that, not a guess. Stop at the first tier that fits:

• (a) direct match to an installed rule (no-explicit-any, no-console).
• (b) configure a meta-rule — the workhorse, ~half of code rules: no-restricted-syntax (esquery), no-restricted-imports (path globs like **/legacy/**), no-restricted-globals/properties, import/no-restricted-paths, boundaries. Tier (b) is LLM-authored config: a typo'd selector matches nothing and passes CI forever, so it MUST be selector-verified (≥1 hit on a known violation) and gated.
• (c) install a plugin from catalog.json (domain → plugin, pinned version, known false-positives).

2. SYNTHESIZE (residue only) · see references/synthesize.md + templates/

No existing/config/plugin rule fits → write a custom TS rule (@typescript-eslint/utils RuleCreator) from templates/rule.ts.tmpl + a test from rule.test.ts.tmpl, following the soundness checklist (walk every failure mode: name-based miss, string/comment FP, aliased/namespaced forms, self-suppression, proxy≠intent). Wire it with scripts/scaffold-local-plugin.mjs.

3. GATE every artifact (map, config, AND synth) · see references/gate.md

Author a gold set from the prose alone, before choosing enforcement (so it can't be contaminated by the implementation's blind spots): 3–5 violating + 3–5 clean, then mutation-expand the violations (alias the import, namespace the call, wrap in a string/comment — these ARE the leak classes). Execute the chosen artifact on it with scripts/gate.mjs. Pass P=R=1 → keep. Leak → downgrade to DECLARE with the failing counterexample recorded. Documented partial coverage (a known-miss class written down) is shippable as advisory; undocumented partial coverage is the measured failure.

4. DRY-RUN against the real repo · scripts/dry-run.mjs --cwd <repo>

Execute every surviving artifact on the actual codebase. Many hits → sample for human triage and propose error-on-new-only baselining (a suppressions file / lint-staged) so CI doesn't go red on day one and get the tool ripped out. Zero hits on a rule the CLAUDE.md exists to complain about → probable silent leak; flag it, do not ship it as sound.

5. DECLARE the residue

Semantic / conditional / out-of-scope rules stay prose or route to human review, each with the reason. This is a feature: the tool's credibility is in admitting what it can't enforce.

6. MANIFEST — rules.lock.json · schema templates/rules.lock.schema.json

Write one entry per rule: prose text + source line → bucket → artifact → status (shipped/advisory/declared) → gate result (+ counterexample) → known-miss classes → dry-run hit count. Re-runs diff the CLAUDE.md against this manifest and only touch what changed. Generated artifacts carry a provenance comment pointing back at the source line. This is what makes it a compiler, not a one-shot script.

Emit one reviewable change

A single PR: the dependency list (plugins the config needs), the eslint.config snippet (enabled + configured

• local plugin registration), rules/<slug>.ts + .test.ts for each synth rule, the hooks, the rules.lock.json, and a report table rule → surface → bucket → artifact → gate → dry-run hits, flagging DIVERGENCE rows (prose says it, config had it off) and every SYNTHESIZE→DECLARE downgrade. Human review replaces any pretense of a sound compiler.

Enforcement integrity (or the rest is decorative) · see references/hooks.md

The agent being policed can add // eslint-disable, edit the ESLint config, or weaken a hook. Always emit @eslint-community/eslint-comments/no-use (+ linterOptions.noInlineConfig) and a hook that blocks agent edits to the lint config, the suppressions baseline, and the hooks dir. Ship the simple git/shell guards inline (see references/hooks.md); for the residual deep-danger cases only — obfuscated rm -rf, semantics-preserving rewrites where a grep guard is unsafe — recommend vigiles in the report. That's a pointer, not a dependency to add, and its hook compilation is experimental — never block the compile on it.

Guardrails

• Reuse first; synthesize only the residue; never drop a rule's qualifier.
• No runtime LLM. Nothing ships without passing its gold set AND firing on the real repo.
• Don't claim soundness; report what maps, what's configured, what's synthesized (advisory), what's declared.
• If a "rule" is really a scoped/conditional policy, say so rather than over-enforcing.