| name | ai-agents-generation-and-release |
| description | Operate the ai-agents generation and release machinery, covering the seven build_all.py generators, generate_agents.py, sync_plugin_lib.py, the drift gates, the version-free plugin manifests, and the npm publish path. Use when you say `regenerate the mirrors`, `run the drift checks`, `why is the plugin version gate red`, `release the npm cli`. Do NOT use for environment setup (use `ai-agents-build-and-env`) or architecture rationale (use `ai-agents-architecture-contract`). |
| version | 1.0.0 |
| license | MIT |
ai-agents Generation and Release
Runbook for the build, generation, mirroring, versioning, and release machinery of this repo. Every command below was executed or read from source on 2026-07-02/03 and re-verified against the working tree on 2026-07-29; re-verify with the one-liners in Provenance before trusting a volatile number.
Jargon, defined once:
- Generator: a Python function or script that reads a canonical tree and writes a derived (generated) tree.
- Drift: the generated tree no longer matches what the generators would produce from the canonical tree.
- Canonical: the tree you are allowed to edit. Generated trees are outputs; you never hand-edit them.
- Semver: semantic version string MAJOR.MINOR.PATCH. The plugin manifests carry no version at all (ADR-092); the npm package still does.
Triggers
regenerate the mirrorsrun the drift checkswhy is the plugin version gate redrelease the npm cliwhy is the drift gate red
When NOT to Use This Skill
| You want | Use instead |
|---|
| Set up the dev environment, uv, hooks | ai-agents-build-and-env |
| WHY the seam is shaped this way, invariants | ai-agents-architecture-contract |
| Triage a red CI gate you do not understand | ai-agents-debugging-playbook |
| Change classification, commit caps, review gates | ai-agents-change-control |
| Escape hatches like the skip-drift-check marker | ai-agents-config-catalog |
Process
Phase 1: Know the Seam Before You Edit
The generation seam is ASYMMETRIC (ADR-072 is PROPOSED and refines this; the running system is what follows). Find your file in this table BEFORE editing. Editing a generated tree is the classic failure (see Anti-Patterns).
| Tree | Role | Canonical source | Regenerated by |
|---|
templates/agents/*.shared.md | CANONICAL for agents | itself | n/a |
src/vs-code-agents/ | generated | templates/agents | build/generate_agents.py |
src/copilot-cli/agents/ | generated | templates/agents | build/generate_agents.py |
docs/agent-catalog.md | generated | templates/agents | build_all.py (agent-catalog) |
.claude/ (rules, skills, hooks, commands, settings.json) | CANONICAL for everything else | itself | n/a (generators NEVER write here) |
src/copilot-cli/{skills,instructions,lib,hooks} | generated | .claude/ trees | build/scripts/build_all.py |
.github/instructions/ | generated | .claude/rules/ | build_all.py (rules) |
scripts/{hook_utilities,github_core,ai_review_common} | CANONICAL for shared Python | itself | n/a |
.claude/lib/ | mirrored copy (relative imports) | scripts/ packages | scripts/sync_plugin_lib.py |
src/claude/ | MANUAL hand-synced exception (ADR-036, Accepted) | edited by hand | no generator; semantic drift CI only |
Generator inventory inside build/scripts/build_all.py (list GENERATORS, build_all.py:435, order is load-bearing per the comment at build_all.py:429):
| # | Generator | Reads | Writes |
|---|
| 1 | agents | templates/agents/*.shared.md + templates/platforms/*.yaml | src/copilot-cli/agents/*.agent.md, src/vs-code-agents/*.agent.md |
| 2 | agent-catalog | templates/agents/*.shared.md | docs/agent-catalog.md |
| 3 | skills | .claude/skills/*/SKILL.md | src/copilot-cli/skills/ |
| 4 | commands | .claude/commands/*.md (top level, not CLAUDE.md) | src/copilot-cli/skills/ (command-bridge skills) |
| 5 | rules | .claude/rules/*.md | .github/instructions/*.instructions.md AND src/copilot-cli/instructions/ |
| 6 | lib | .claude/lib/ | src/copilot-cli/lib/ (must land before hooks) |
| 7 | hooks | .claude/settings.json + .claude/hooks/ | src/copilot-cli/hooks/ + src/copilot-cli/hooks/hooks.json |
Facts that prevent confusion:
build_all.py enforces a no-write invariant on .claude/ (REQ-003-010): if any generator writes there, the run exits 2 with REQ-003-010 VIOLATION. .claude/ is input only.- Generated-tree ownership is exactly
OWNED_PREFIXES = ("src/", ".github/instructions/", "docs/agent-catalog.md") (build_all.py:765). --check only flags staleness inside those prefixes.
- The hooks generator maps Stop, SubagentStop, PermissionRequest, and
PreCompact to their PascalCase compatibility names. Stop and SubagentStop
remain direct host registrations because their structured decisions require
host-level merging. A reported drop needs an identified source registration;
never assume it is normal.
- Every run overwrites the generation audit log in the gitignored build audit directory. Read it to see what each generator did.
Phase 2: Regenerate After Editing a Canonical Surface
Always commit the canonical edit and the regenerated output TOGETHER, in the same commit set, or the drift gates go red.
Interpreter note: build/generate_agents.py and build/scripts/build_all.py both import PyYAML through build/scripts/yaml_loader.py. Run them as uv run python <script> locally so the project venv supplies it. CI can invoke bare python3 because .github/actions/setup-code-env installs the locked dependencies system-wide first.
| You edited | Run | Then |
|---|
templates/agents/*.shared.md | uv run python build/generate_agents.py then uv run python build/scripts/build_all.py (refreshes docs/agent-catalog.md) | commit template + all regenerated files. If the same agent exists in src/claude/agents/, hand-apply the equivalent edit there (ADR-036 manual sync; semantic drift CI is the only net). |
.claude/skills/, .claude/commands/, .claude/rules/ | uv run python build/scripts/build_all.py | commit source + generated. No manifest edit (Phase 4) |
.claude/hooks/ or .claude/settings.json | uv run python build/scripts/build_all.py | same as above. The build-all-check pre-push job in lefthook.yml re-runs build_all.py --check at git push time and blocks if any generated output (including shims under src/copilot-cli/hooks/) drifts, so regenerate BEFORE pushing. |
scripts/hook_utilities/, scripts/github_core/, scripts/ai_review_common/ | python3 scripts/sync_plugin_lib.py (writes .claude/lib/) THEN uv run python build/scripts/build_all.py (writes src/copilot-cli/lib/) | BOTH are required; skipping either fails the Validate Generated Files CI. No manifest edit. |
src/claude/ (deliberate manual change) | nothing to regenerate | nothing to bump; the manifest carries no version (Phase 4) |
Useful flags, verified against source:
build/generate_agents.py: --validate (CI compare mode), --what-if (dry run, writes nothing), --templates-path, --output-root. Exit codes: 0 ok, 1 logic error or drift, 2 config error (docstring, generate_agents.py:13-16).build/scripts/build_all.py: --check (staleness gate; snapshots and restores owned trees), --clean, --audit-format json, --platform copilot-cli. Exit codes: 0 ok, 1 generator error, 2 config error or staleness, 3 audit blocklist violation (docstring, build_all.py:16-20).scripts/sync_plugin_lib.py: no flag syncs, --check is the CI dry run (exit 1 when out of sync). It also rewrites from scripts.x import to relative imports; do not "fix" those imports in .claude/lib/ by hand.
Phase 3: Run the Drift Gates Locally Before Pushing
Drift-gate matrix (all local commands verified runnable, all green on 2026-07-29):
| Gate | Catches | Local command | CI enforcement |
|---|
| Agent template drift | templates edited without regen (or vice versa) | uv run python build/generate_agents.py --validate | validate-generated-agents.yml, agent-drift-detection.yml |
| Full pipeline staleness | any canonical edit not mirrored to owned prefixes | uv run python build/scripts/build_all.py --check | validate-generated-agents.yml; named pre-push job in lefthook.yml |
| Lib mirror drift | scripts/ package edited without sync | python3 scripts/sync_plugin_lib.py --check | validate-generated-agents.yml |
| Install parity | plugin install layout broken | python3 scripts/validation/run_install_parity_ci.py | validate-generated-agents.yml |
| Manifest description parity | .claude vs src/copilot-cli plugin descriptions carry component counts | python3 build/scripts/check_plugin_manifest_parity.py | validate-generated-agents.yml, agent-drift-detection.yml (the version half was retired with ADR-092) |
| Plugin version field present | a manifest or marketplace entry carries version | pre-pr-validation job in lefthook.yml (scripts/validation/pre_pr.py) | validate-plugin-version-bump.yml |
| Semantic agent drift (src/claude) | hand-synced tree diverging in meaning | python3 build/scripts/detect_agent_drift.py | drift-detection.yml, weekly cron Monday 09:00 UTC (line 15); similarity threshold default 80 (detect_agent_drift.py:666-668), with a recorded-baseline floor so a clean checkout does not fail |
When a drift gate is red, the output shows the DIFFERENCE, not the DIRECTION. Ask "which side is canonical?" using the Phase 1 table before touching anything. The 2025-12-15 incident (retro: .agents/retrospective/2025-12-15-drift-detection-disaster.md) happened because an agent edited the SOURCE to match the GENERATED tree; the commit was reverted. Fix is always: edit canonical, rerun generator, commit both.
An emergency bypass marker for the drift gate exists but requires a reason and approval; see ai-agents-config-catalog. Route the decision through ai-agents-change-control.
Run the installed Copilot hook E2E in an isolated home. This executes the
marketplace layout and generated hook scripts without changing live plugin state:
E2E_HOME="$(mktemp -d)"
HOME="$E2E_HOME" COPILOT_HOME="$E2E_HOME/.copilot" \
copilot plugin marketplace add "$PWD"
HOME="$E2E_HOME" COPILOT_HOME="$E2E_HOME/.copilot" \
copilot plugin install project-toolkit@ai-agents
HOME="$E2E_HOME" COPILOT_HOME="$E2E_HOME/.copilot" \
RUN_INSTALLED_PLUGIN_HOOK_E2E=1 uv run pytest \
tests/e2e/test_installed_plugin_hook_e2e.py -q
Do not run this setup in the live Copilot home. Copilot CLI 1.0.72-1 was observed
removing the marketplace registration for copilot plugin uninstall project-toolkit while leaving the _direct/project-toolkit cache. The active
session then lost its hook scripts and denied matching tools until the
marketplace plugin was reinstalled.
Phase 4: Plugin Manifests Carry No Version
Three plugin manifests exist (verify:
find . -name plugin.json -path "*claude-plugin*"):
| Tree | Manifest | Plugin name |
|---|
.claude/ | .claude/.claude-plugin/plugin.json | project-toolkit (Claude) |
src/copilot-cli/ | src/copilot-cli/.claude-plugin/plugin.json | project-toolkit (Copilot) |
src/claude/ | src/claude/.claude-plugin/plugin.json | claude-agents |
The rule (docstring of build/scripts/validate_plugin_version_bump.py): none of
those manifests may carry a version field, and neither may an entry in either
marketplace.json. Claude Code then resolves freshness from the git commit SHA,
which moves on every merge. ADR-092 records the reversal of ADR-079's
hand-bumped rule after issue #4080 measured 14 of 22 conflicting PRs conflicting
on that one line.
Practical consequences:
- Any
.claude/ content edit regenerates src/copilot-cli/ too, but neither manifest changes: both are version-free, and check_plugin_manifest_parity.py now checks description component counts only.
src/claude/ is the same: no version to move..github/instructions/ and src/vs-code-agents/ carry no plugin.json at all.- Marketplace:
.claude-plugin/marketplace.json lists two plugins (claude-agents from ./src/claude, project-toolkit from ./.claude). The old marketplace count validator and its YAML config were retired, so treat description-count validation as currently unenforced and keep counts honest by hand.
Phase 5: npm Release Path
The npm surface is packages/ai-agents-cli (package
@rjmurillo/ai-agents; read package.json for the current version). It vendors
the Claude kit into consumer repos. Release procedure (source: RELEASING.md,
repo root):
- Bump
packages/ai-agents-cli/package.json version. Tag/version mismatch is a listed failure mode.
- Local sanity:
cd packages/ai-agents-cli && bun run build (runs bun build src/cli.ts --outdir dist --target node), bun test, tsc --noEmit via bun run typecheck. As of 2026-07-29 bun run typecheck exits 1 on this package with 38 errors (36 TS2591, 2 TS2503). @types/node 26.0.0 is installed, but the package tsconfig.json sets no types field, and TypeScript 7.0.2 no longer auto-includes it the way 5.9.3 does, so Node builtins are absent from the compilation. Issue #3865 tracks the fix: add "types": ["node"] and a CI job that runs typecheck and bun test. Before blaming your change for a red typecheck, check whether that fix landed: grep -q '"types"' packages/ai-agents-cli/tsconfig.json.
- Commit to main via normal PR flow (branch discipline still applies; see
ai-agents-change-control).
git tag vX.Y.Z then git push origin main --tags..github/workflows/publish.yml fires on v* tags: validates package metadata, publishes with OIDC provenance (id-token: write; NPM_TOKEN is fallback only). workflow_dispatch offers a dry-run input defaulting to true.- Verify: registry version, provenance badge on npmjs.com, clean-environment install (commands in RELEASING.md Verification section).
Rollback is roll-FORWARD: npm unpublish is restricted; fix, bump patch, retag (RELEASING.md Rollback procedures).
Anti-Patterns
| Anti-pattern | Why it burns you | Do instead |
|---|
Hand-editing src/copilot-cli/, src/vs-code-agents/, .github/instructions/, .claude/lib/ | Next regen silently overwrites your edit; ruff deliberately exempts generated Python (pyproject.toml [tool.ruff.lint.per-file-ignores] ignores all selected rule families under src/copilot-cli/{hooks,skills}/**/*.py), so lint will not even look at it | Edit the canonical tree (Phase 1 table), regenerate |
| Editing source to make a drift gate green | The 2025-12-15 disaster; drift output shows difference, not direction | Identify canonical side first, regenerate outward |
| Syncing lib with only one of the two steps | sync_plugin_lib.py feeds .claude/lib/; build_all.py feeds src/copilot-cli/lib/; missing either fails CI | Run both, in that order |
| Bumping the two project-toolkit manifests in lockstep to satisfy version parity | Version parity retired with the field (ADR-092); check_plugin_manifest_parity.py no longer compares versions, and the bump itself now fails the gate | Nothing to bump; leave both manifests version-free |
Re-adding a version to a manifest to clear a red gate | The gate fails on the field's presence, so the red is permanent | Delete the field; freshness already tracks the commit SHA |
Treating Dropped: N as normal without checking the source | Current source registrations all map to Copilot events; an unexplained drop means contract drift | Read the generation audit log, identify the source registration, and require an explicit eventDrop decision |
Running build_all.py with bare python3 in a fresh shell | PyYAML lives in the venv; import fails | uv run python build/scripts/build_all.py |
Running installed-plugin E2E against live ~/.copilot | Version skew and ambiguous uninstall mutate active hooks; a missing hook root can deny every matching tool | Set both HOME and COPILOT_HOME to one isolated directory and register the worktree marketplace there |
Verification
Run this checklist before pushing any change that touched a canonical or generated surface:
Provenance and Maintenance
Verified 2026-07-29 against the working tree (re-verification pass; the 2026-07-03 pass had rotted for build/scripts/build_all.py, build/generate_agents.py, pyproject.toml, scripts/sync_plugin_lib.py, .github/workflows/publish.yml, and .github/workflows/validate-generated-agents.yml). Volatile facts and how to re-check them:
| Fact | Source | Re-verify |
|---|
| 7 generators and their order | build/scripts/build_all.py:435-443 | grep -n -A9 "^GENERATORS" build/scripts/build_all.py |
| OWNED_PREFIXES trio | build/scripts/build_all.py:765 | grep -n "OWNED_PREFIXES" build/scripts/build_all.py |
| .claude/ no-write invariant | build/scripts/build_all.py:674 (rule), :1013 (snapshot), :1076-1081 (enforcement) | grep -n "REQ-003-010" build/scripts/build_all.py |
| build_all exit codes 0/1/2/3 | build/scripts/build_all.py:16-20 | sed -n '16,20p' build/scripts/build_all.py |
| generate_agents flags and exit codes | build/generate_agents.py:13-16,460-487 | uv run python build/generate_agents.py --help |
| sync pairs scripts to .claude/lib | scripts/sync_plugin_lib.py:27-31 | grep -n -A4 "SYNC_PAIRS" scripts/sync_plugin_lib.py |
| Plugin manifest locations, all version-free | the three plugin.json files | python3 build/scripts/validate_plugin_version_bump.py |
| Version-field prohibition and the ADR-092 reversal | build/scripts/validate_plugin_version_bump.py docstring | grep -n "WHY THE FIELD MUST BE ABSENT" build/scripts/validate_plugin_version_bump.py |
| Parity gate #2222 | build/scripts/check_plugin_manifest_parity.py:1-16 | python3 build/scripts/check_plugin_manifest_parity.py |
| Drift CI wiring | .github/workflows/validate-generated-agents.yml:165,174,212,225,239; agent-drift-detection.yml:146,156,159,171 | grep -n "uv run python" .github/workflows/validate-generated-agents.yml |
| Weekly semantic drift cron, threshold 80 | .github/workflows/drift-detection.yml:13-15; build/scripts/detect_agent_drift.py:666-668 | grep -n "cron" .github/workflows/drift-detection.yml |
| Git hook jobs, filters, and validators | lefthook.yml | uv run --frozen lefthook validate |
| Ruff exemption for generated Python | pyproject.toml [tool.ruff.lint.per-file-ignores] |
Maintenance: when a generator is added or removed from GENERATORS, when a fourth plugin.json appears, or if a marketplace count validator is reintroduced to replace the retired one, update Phase 1/4 tables and re-run every re-verify command above.
