ai-devkit

Run an evaluation harness over AI output to catch quality regressions when the model or harness changes. Reads golden tasks (input + a known expected outcome) from an evals/ directory, runs each against its named target (a skill, prompt, agent, or product LLM feature), scores the output against the expected oracle with a rubric/assert/exact grader, compares to a git-tracked baseline, and reports pass/fail per task plus a regression diff. A run fails on any drop vs baseline, not just on missing a threshold. Concrete, not aspirational: golden tasks, a baseline, acceptance thresholds, and a model/harness-stamped regression check. Does NOT grade against the implementation's own output (the oracle is the golden expected outcome). Trigger phrases: "run the evals", "eval the prompt", "did the model upgrade regress anything", "score against golden tasks", "set up an eval harness".

Execute atomic task files (T-NNN-<slug>.md) or a standalone plan.md with disciplined pre/per/post gates and write status back to frontmatter, closing the loop with /atomize so reconciliation works without manual edits. Detects the project's test runner from package.json / pyproject.toml / go.mod / pom.xml / Cargo.toml / composer.json. Respects depends_on between tasks, picks the next unblocked task on resume, and offers to capture lessons when a task is abandoned mid-flight. Modes: single task (T-NNN), whole initiative folder (docs/work/<NNN>-<slug>/), or a plan.md that was never atomized. Trigger phrases: "implement T-001", "implement this initiative", "execute the plan", "work on docs/work/NNN-slug", "resume implementation", "continue the initiative".

Author end-to-end test scenarios (optimistic + pessimistic paths) from a task, initiative, product-spec, or freeform description, and generate committed Playwright test files into tests/e2e/. Each scenario is a two-part artifact: a human-reviewable plan (tests/e2e/scenarios/<slug>.md, carrying depends_on links to docs/work/<NNN>-<slug>/ initiatives and T-NNN tasks) plus generated .spec.ts files — web via the browser, backend via Playwright's request fixture — tagged @optimistic / @pessimistic so /e2e-run can scope by path type. The plan is a mandatory human-review gate: no test code is written until the user confirms the paths. Optionally grounds web selectors against a live app via the Playwright MCP for honest role/test-id locators. Does NOT run the tests (that's /e2e-run) and never auto-heals. Trigger phrases: "write e2e scenarios", "create test scenarios", "scenario for T-NNN", "test the happy and edge paths", "author e2e tests".

Decompose a plan.md into atomic task files (T-NNN-<slug>.md) inside an initiative folder under docs/work/. Auto-detects mode: INITIAL (no T-*.md present — propose tasks from plan, write T-*.md + index.md) or RECONCILIATION (T-*.md exist — diff plan against current tasks, propose new/revised/obsoleted tasks; verify status:done claims against git log + file existence; never edit already-done tasks — write follow-up tasks instead). Trigger phrases: "atomize the plan", "decompose into tasks", "break down the plan", "create tasks from plan", "reconcile tasks", "the plan changed", "what tasks need updating".

Build an evidence-based map of an unfamiliar or large/legacy repository before you change it. Gathers cheap, deterministic CLI signals (git-history territory, a dependency graph, contributor history) OUTSIDE the model's context window, then has the agent interpret the evidence into one short, decision-oriented artifact — docs/architecture/repo-map.md (modules, entry points, dependency directions + cycles, coupling hotspots, sensitive areas / blast radius, who-to-ask, and explicit unknowns). Read-only: never modifies source. Detects the stack (JS/TS, Python) to pick the right dependency tool. Trigger phrases: "map this repo", "understand this codebase", "onboard me to this legacy project", "where is it risky to change", "/repo-map".

Per-decision research skill: gather facts and reasoning around a specific technology choice, vendor selection, or technical investigation, and write the findings to docs/analyzes/<slug>.md as a point-in-time research snapshot. Two modes (auto-selected via user choice): INTERVIEW — facilitator asks structured questions, user provides facts; INVESTIGATION — spawns research subagents (web + code) and synthesizes. Reads docs/reference/ for project- specific context — any file type (PDFs, DOCs, link collections, curated bibliographies), with sibling .md indexes for binaries. Closes with anti-bias cross-check (devil's advocate + pre-mortem). Use BEFORE making a significant technical choice, NOT for product-level discovery (use /discover for that). Trigger phrases: "should we adopt X", "research <topic>", "evaluate <option>", "vendor selection", "<lib A> vs <lib B>", "investigate <technology>", "what should I know about <topic>".

Initialize the /docs directory in this project — scaffold docs/{architecture,analyzes,reference,work}/ plus the tests/e2e/ home for /scenario, with README.md files if absent. Idempotent. For brownfield projects, also offers to run Claude Code's built-in /init to generate CLAUDE.md from codebase analysis.

Generate a CI/CD pipeline for a project: a GitHub Actions workflow that lints, tests, and enforces a coverage gate (default 80%), publishes coverage to an external dashboard (Codecov), and optionally adds an AI code-review step on PRs (kukuvaia on a self-hosted runner, or the official Claude Code Action). Detects the stack (Python/uv, Node/npm) to emit the right commands and reuses /implement's runner-detection. Writes nothing until you confirm the file plan. Idempotent. Trigger phrases: "set up CI", "add a CI/CD pipeline", "add GitHub Actions", "add a coverage gate", "add AI code review to CI", "/ci-setup".

Author AGENTS.md as the canonical single source of truth for project-specific agent rules, with a thin CLAUDE.md import shim and an optional .github/copilot-instructions.md shim — so Claude Code, Codex, Cursor, and Copilot all read the same file without duplication. Anchors every proposed rule in a real source (a docs/architecture/ decision, a docs/reference/ lessons.md entry, an incident, or a product-spec constraint) and refuses to add generic best-practice content already covered by ai-devkit's auto-active rules in ~/.claude/rules/. Enforces the "test of inclusion" up front and respects the ~200-line size budget plus U-shaped attention layout (critical rules at the top, references at the bottom). For area-specific rules, proposes a granular file (e.g. src/api/AGENTS.md) instead of polluting root. Trigger phrases: "set up AGENTS.md", "init agents file", "onboard the agent to this project", "create CLAUDE.md", "agent rules for this repo".

Facilitate a structured discovery conversation that turns an idea — greenfield or brownfield — into discover-notes.md, the input to /product-spec. Auto-detects context type from project markers in cwd (brownfield) or absence thereof (greenfield) and adapts all six discovery phases accordingly. For brownfield, automatically scans docs/product-spec.md, docs/architecture/, and docs/work/ for inheritable state (vision, persona, access control, business logic, FRs, NFRs) and switches to delta-only elicitation — systems evolve, they do not rebuild. Inheritance is written to a ## Inherited state section so each phase only asks what is changing. Use when the user is starting a new project from scratch OR shaping a meaningful change to an existing system (new module, significant feature, architectural improvement). Trigger phrases: "new project", "from scratch", "starting an app", "shape an idea", "brainstorm a product", "greenfield", "I have an idea", "existing project", "brownfield". Use BEFORE /product-spec, not in

Discover and run the end-to-end tests authored by /scenario, with the right tool per artifact type and scoped to what you ask for. Default runner is Playwright (web .spec.ts via the browser, API .api.spec.ts via the request fixture); dedicated API tools — Hurl, Schemathesis — are opt-in extensions for plain-text or OpenAPI-derived specs. Resolves scope from a scenario plan, an initiative folder, a T-NNN task (via depends_on), a path-type tag (@optimistic / @pessimistic), or a layer. Reports pass/fail per layer and per path type with trace artifacts. Strictly reports failures — never regenerates or auto-heals tests. Optionally appends a one-line result note to the linked T-NNN task. Trigger phrases: "run e2e", "run the scenarios", "e2e-run T-NNN", "run pessimistic paths", "execute the e2e tests".

Generate docs/product-spec.md from discover-notes.md (or raw notes) against the locked schema. Auto-routes to greenfield (10 sections) or brownfield (11 sections) template based on context_type in discover-notes.md or cwd auto-detection. Use when the user has discovery notes ready and wants a schema-conformant spec written to disk. Trigger phrases: "write the spec", "generate spec", "create the spec from notes", "turn notes into a spec", "spec from discover-notes". Use AFTER /discover, not in place of it.

Save the most recent plan (from Claude Code's /plan mode or pasted/file content) into a new initiative folder docs/work/NNN-<slug>/plan.md, then optionally chain into /atomize to decompose it into atomic task files. Auto-detects roadmap-shape plans (top-down sequencing with foundations + slices) and lands them at docs/roadmap.md instead — see references/roadmap-shape.md for the contract. Bridges Claude Code's built-in /plan mode (which only displays the plan) to the docs/work/ convention (where plans need to live as files to be consumed by /atomize). Auto-detects plan source: inline file arg → recent conversation context → ask user to paste. Trigger phrases: "save the plan", "save plan", "stash this plan", "create initiative folder", "atomize this plan", "the plan I just approved", "save the roadmap".

Automated Claude code review for a Bitbucket Cloud pull request, driven by the Bitbucket REST API. Fetches PR metadata + the unified diff over the API (no local clone required), runs a structured Claude review (correctness + reuse/simplification/efficiency, optional security pass), prints findings in chat, and — opt-in — posts them back to the PR as inline or summary comments. Auth is via API token (App Passwords are deprecated/disabled 2026-06-09). Bridges the gap that /code-review --comment only covers GitHub: this is the Bitbucket equivalent. Posting defaults to OFF so a run never silently writes to a shared PR. Trigger phrases: "review this Bitbucket PR", "bitbucket pull request review", "review PR <url>", "/bitbucket-review <url>".

Open a URL in a Playwright-driven browser (via the maister MCP server), wait for the page to settle, capture an accessibility snapshot + optional screenshot, surface console errors, and report what landed. Bridges the gap between WebFetch (server-rendered HTML only, no JS, no session) and pasting screenshots manually — works for pages behind a login wall (headed browser inherits an authenticated session) and for JavaScript-rendered SPAs that WebFetch can't read. Trigger phrases: "open this URL", "open in browser", "load page in Playwright", "screenshot this page", "what's on this page", "fetch rendered content".

Audit an existing agent-rules file (AGENTS.md, CLAUDE.md, .cursor/rules/*.mdc, .github/copilot-instructions.md, .github/instructions/*.md) across seven dimensions and emit a verdict per finding with concrete evidence. Goes beyond length / embedded code / language precision / redundancy / order (which 10x covers) by adding (6) cross-tool drift detection when multiple rules files exist for the same scope, and (7) dead-rule check that greps the codebase for the patterns each rule targets — if nothing matches, the rule may be obsolete. Cites the ai-devkit rule in ~/.claude/rules/ that already covers redundant content. Read-only by default; --fix applies safe rewrites (remove redundancy, split oversized rules, reorder by importance). Trigger phrases: "review AGENTS.md", "audit my CLAUDE.md", "check my rule file", "rule review", "is my AGENTS.md any good".