architecture-fitness-functions

name	architecture-fitness-functions
description	Turn a codebase's prose architecture invariants (constitution/AGENTS.md hard rules, design-spec acceptance criteria, capability matrices) into deterministic, CI-enforced conformance tests — "architecture fitness functions" — so agents can't quietly break global invariants while every local test stays green. Covers the named invariant register, structural (build-time) vs behavioral (runtime) checks, the deterministic-not-LLM enforcement rule, grounding tests in real runtime code, and verifying drift is genuine before flagging it. Use when an agent-written or fast-moving codebase is drifting from its intended architecture, when you want to formalize "the architecture we actually want" and stop regressions in CI, or when reviewing whether a refactor preserved a system's global contracts.
author	skill-review

architecture-fitness-functions

Convert the architecture you want into machine-checked tests that fail the build when it drifts. The job: take the invariants that currently live as prose (an AGENTS.md "constitution", a design spec's acceptance criteria, a "capability matrix", roadmap rules) and compile them into deterministic checks wired into CI.

The failure mode this defends against

Local correctness does not guarantee global architectural alignment. An agent with a failing test in one module will "fix" it by satisfying that module's local contract while silently destroying a global invariant. The canonical example: a system distinguishes point-like data (a single moment) from range-like data (an interval T0→T1). An agent collapses a point into a degenerate interval so an existing overlap check passes — the test goes green, the local module is fine, but the cross-component invariant "point ≠ interval" is gone, and the next agent propagates the confusion. Every local test passing is exactly what makes this invisible. Fitness functions are the global check the local tests can't be.

Core rules

1. Deterministic code enforces — never an LLM. The agent (LLM) is the thing being governed; it is never the governor. An LLM is fuzzy and can be talked out of anything, so you cannot build a guardrail from one. Every "enforce" means plain code/config that passes or fails with no model in the loop. If the truth of the system lived in the agent, there'd be nothing to enforce.

2. Two walls — keep them separate; they're built from different materials.

   • Wall 1 — structural (build-time): "is the code still shaped the way we designed it?" Runs in CI on every PR. Examples:
     • "Core engine must not contain scanner vocabulary (scan_types)" → a test greps/AST-parses the core package and fails on a match.
     • "The adapter registry is exactly {mock, remote, local}" → import the tuple and assert it; if an agent adds an unapproved "extra", red build.
     • "The core layer must never import an infrastructure adapter" → an import-linter contract that fails on a forbidden dependency edge.
   • Wall 2 — behavioral (runtime): "does the system still behave per the contract?" Pin tables and matrices. Example: a single capability-matrix conformance test that asserts the whole per-mode security/capability table so no backend can start silently faking a lever. Don't lump them; Wall 1 catches structure drift, Wall 2 catches behavior drift.

3. A named invariant register is the single source of truth. Maintain one doc (e.g. docs/architecture/INVARIANTS.md) of INV-NNN entries: each invariant gets an id, a one-line statement, the source rule it encodes, and a pointer to the test that enforces it. This is the compiled spec — the single place "the architecture we want" is written down and linked to its checks.

4. Mark REGISTER-ONLY rows honestly — never make something look enforced that isn't. Invariants you've stated but not yet given a dedicated test are backlog. Tag them REGISTER-ONLY explicitly so nobody reads the register as a guarantee. An honest "this is a law we haven't wired up yet" beats a register that implies coverage it doesn't have. Same honesty rule for a matrix row whose mode isn't actually built (e.g. "intended 3 modes, 1 built") — record the gap as its own invariant rather than pretending parity.

Workflow

1. Map intended vs actual. Read the orienting docs first — AGENTS.md / constitution hard rules, the design spec's acceptance criteria, the capability matrix, the roadmap (it often already logs "drift that crept in"). Then map the source tree. The gap between the prose and the code is your register's seed list.
2. Ground every test in real runtime code — no fictional tests. Before writing a conformance test, read the actual module, its existing test conventions, and what's already pinned, so the test runs in the real suite and matches house style. Don't duplicate conformance checks the codebase already has scattered around — the value-add is the named register tying them together plus the missing structural/matrix tests.
3. Verify drift is genuine before crying wolf. When you're the one flagging "this is drift," accuracy matters. A vocabulary match (e.g. scan_type) can be legitimate in one layer and forbidden in another. Distinguish the roles before drawing the line: e.g. scanner as a workload the engine runs (forbidden) vs security validators as an output-governance gate that runs on agent-generated code before commit (legitimate). That is the same point-vs- interval distinction — an over-broad invariant that ignores the boundary will flag correct code. Open and read the suspect files before labeling them.
4. Write the checks, wire into CI. Add the tests, link each to its INV-NNN row, and run them in the PR pipeline so future drift is a red build, not a manual archaeology dig later.

Pitfalls

• Over-broad invariants flag legitimate code. An invariant that bans a word rather than a role will fail the build on correct usage in another layer. Name the boundary in the invariant statement.
• Inventing tests against code you didn't read. A conformance test that references a module/contract that doesn't exist as written is worse than none — it gives false confidence and breaks the suite. Ground first.
• Stale register/test residue after edits. Duplicated paragraphs or reason strings that name an offender a later carve-out removed will mislead the next reviewer. Re-read and re-run green after every edit to the register or tests.