Menu
Run a graduated evaluation harness (0-100) scoring implementation quality across four layers: deterministic, architectural, LLM-as-Judge, and human-in-the-loop. In Codex, invoke with $adev:eval
Instalar com Codex ou Claude Copie este prompt, cole no Codex, Claude ou outro assistente e deixe que ele revise a página da skill e instale para você.
Baseado na classificação ocupacional SOC
Assess codebase readiness for agentic development. Scans the codebase using static file inspection and outputs maturity scores across 8 structural dimensions (raw mode) or 11 dimensions (adev mode). Use when you want to evaluate how well-suited a codebase is for AI-assisted development, identify improvement areas, or track progress over time. In Codex, invoke with $adev:assess
Explore a feature idea interactively and produce a Feature Charter. Validates against constitution and existing charters. Use before building any new feature or module, when the user mentions a new capability, or says 'let us build X'. In Codex, invoke with $adev:brainstorm
End-to-end build orchestrator. Chains review, plan, route, implement, and validate for one or more specs through a full lifecycle pipeline. Use when the user says 'build', 'end to end', 'full pipeline', 'build the spec', 'build the milestone', 'run the whole pipeline', or wants to execute multiple lifecycle steps in sequence without manual handoffs. In Codex, invoke with $adev:build
Proactively scan source code for dead exports, orphan files, unused dependencies, stale code, and duplicate logic. Produces severity-tiered reports from repomap artifacts. Use when the user says 'check code health', 'find dead code', 'unused exports', 'stale files', 'orphan files', 'unused dependencies', or wants to identify cleanup opportunities before refactoring. In Codex, invoke with $adev:codehealth
Context-aware systematic debugging. Checks ADRs for known issues, specs for expected behavior, and orientation for architecture context before investigating. Use when encountering any bug, test failure, or unexpected behavior. Trigger on 'fix this', 'fix the bug', 'something is broken', 'this test fails', 'unexpected error', 'why is this not working', 'this doesn't work', 'can you fix', 'there's an error', 'it's failing', or any request to fix, debug, or investigate a problem. In Codex, invoke with $adev:debug
Run a deployment pipeline defined in .context-index/deploy.yaml. Executes shell, manual, verify, gate, and ci-trigger steps in order with failure recovery and rollback guidance. Use when the user says 'deploy', 'run deploy', 'push to production', 'publish', or wants to execute a structured deployment. In Codex, invoke with $adev:deploy
| name | adev:eval |
| description | Run a graduated evaluation harness (0-100) scoring implementation quality across four layers: deterministic, architectural, LLM-as-Judge, and human-in-the-loop. In Codex, invoke with $adev:eval |
Score implementation quality across four evaluation layers, producing a graduated quality score (0-100) rather than binary pass/fail. Complements /adev:validate with nuanced quality assessment.
Announce at start: "I'm using the adev:eval skill to run the evaluation harness."
--spec <path>: evaluate the implementation of a specific spec (required)--layer <N>: run only a specific layer (1-4)--configure: interactive setup of eval configuration--rubric <path>: use a custom rubric for Layer 3--no-infra: skip infrastructure preflight checks (user-only — the agent must never set this flag).context-index/ must be initialized./adev:validate should have passed (eval builds on top of validation, not replaces it)..context-index/samples/ should have relevant golden samples..context-index/evals/config.yaml (generated by --configure).After verifying prerequisites, check whether the spec declares infra_requirements. If so, run the infrastructure preflight before proceeding to evaluation layers.
Layer-aware skip: If --layer 1 or --layer 2 is specified, skip the preflight step (no code execution against external systems in those layers).
--no-infra resolution: Read --no-infra flag from arguments. If not passed, check ADEV_NO_INFRA env var (only exact value 1 activates bypass). Read once at skill entry, convert to options.noInfra. The agent must never set --no-infra or ADEV_NO_INFRA autonomously — if preflight fails, report the failure and wait for user direction.
Plan path resolution: Glob for a .plan.md sibling adjacent to the spec (same directory, same base name). If absent, omit --plan.
Invocation: Run the preflight via the CLI:
adev preflight run --spec <specPath> [--plan <planPath>] [--timeout 10] [--no-infra]
Stdout is a single JSON object — the preflight report (the same shape runPreflight() returns). Exit codes: 0 on PASS or skipped, 2 on FAIL, 1 on argument errors.
If the report has passed === false (exit 2), display the formatted report and block:
Infrastructure Preflight: FAILED
<formatted report output>
Execution blocked. Options:
1. Fix the issues above and retry
2. Re-run with --no-infra to bypass (user decision only)
If report.passed === true and report.skipped === true, emit: "Infrastructure preflight skipped (--no-infra)."
If report.passed === true and report.skipped === false, proceed silently.
If lib/infra-preflight.mjs fails to import, block with: "Infrastructure preflight library could not be loaded: . Fix the library before proceeding."
Load Skill Extensions: Load any skill extension instructions before proceeding:
adev skill-ext load --skill eval
If the output is not __NONE__, incorporate it as additional standing instructions that apply to this skill's entire execution. Frame it as: "The following skill extension instructions apply to this invocation (source: installed domain extensions and/or project-level overrides)." If the output is __NONE__, continue normally.
Run all quality gates from governance/gates.yaml (or constitution fallback). This mirrors /adev:validate Check 1 but records detailed scores:
This layer is the floor. If Layer 1 scores 0, the total score cannot exceed 25 regardless of other layers.
Compare the implementation against golden samples and constitutional patterns:
Pattern consistency (0-10): For each file touched, check if it follows the pattern established by the matching golden sample. Measure: naming conventions, file structure, import organization, error handling style. Score based on deviation count.
Boundary compliance (0-5): Run governance/boundaries.yaml patterns. Score based on violations (0 = full score, each violation deducts 1 point).
Complexity metrics (0-5): Check file sizes, function lengths, nesting depth against constitution thresholds. Score based on compliance.
Test quality (0-5): Assess test files: do they test behavior (not implementation details)? Do they cover error cases? Are assertions meaningful? Score by checklist.
Dispatch a reviewer subagent with a rubric to score code quality on subjective dimensions that cannot be machine-checked:
Readability (0-5): Can a new developer understand this code without additional context? Are variable names descriptive? Is the control flow clear?
Maintainability (0-5): Is the code easy to modify? Are concerns separated? Are dependencies explicit? Would a change in one area cascade?
Spec fidelity (0-5): Does the implementation capture the spirit of the spec, not just the letter? Are edge cases handled gracefully?
Idiomatic usage (0-5): Does the code use the framework and language idiomatically? Does it follow community conventions for the declared platform?
Error handling (0-5): Are errors handled at the right level? Are error messages useful? Does the happy path degrade gracefully?
Dispatch the reviewer subagent (reasoning tier — read from model_tiers in .context-index/platform-context.yaml; fall back to the hardcoded default in .context-index/specs/cross-cutting/model-routing.md if unset, and log a one-time advisory). Prepend ultrathink as the first word of the subagent prompt to activate extended thinking. Provide:
The implementation diff (all files changed)
The Live Spec
The rubric (default or custom from --rubric)
Relevant golden samples for comparison
Instructions to score each dimension 0-5 with a one-sentence justification
A self-check instruction: "Before finalizing, verify every score has a justification grounded in the actual code, and no score is based on absence of information."
A return size constraint: "Keep your response under 1,500 tokens. Score each dimension concisely."
Overall Layer 3 score = sum of sub-scores (0-25)
Surface items that require human judgment. This layer does not auto-score but presents a checklist:
Present the checklist to the user. User marks each item as: PASS (5 points), ACCEPTABLE (3 points), NEEDS_WORK (0 points).
Total quality score = Layer 1 + Layer 2 + Layer 3 + Layer 4 (0-100)
| Score | Grade | Interpretation |
|---|---|---|
| 90-100 | A | Excellent. Exemplary implementation. |
| 75-89 | B | Good. Minor improvements possible. |
| 60-74 | C | Acceptable. Notable gaps in quality. |
| 40-59 | D | Below standard. Significant issues. |
| 0-39 | F | Failing. Major rework needed. |
Persona adaptation: The report written to disk always uses the full format below. The chat summary presented to the user should follow the active persona's output rules.
Write to .context-index/evals/<spec-slug>-eval.md:
# Evaluation Report: <Spec Title>
> **Date:** YYYY-MM-DD
> **Spec:** <path>
> **Overall Score:** N/100 (Grade: A-F)
## Layer 1: Deterministic Checks — N/25
- Tests: PASS/FAIL
- Lint: PASS/FAIL
- Typecheck: PASS/FAIL
## Layer 2: Architectural Conformance — N/25
- Pattern consistency: N/10
- Boundary compliance: N/5
- Complexity metrics: N/5
- Test quality: N/5
## Layer 3: LLM-as-a-Judge — N/25
- Readability: N/5 — <justification>
- Maintainability: N/5 — <justification>
- Spec fidelity: N/5 — <justification>
- Idiomatic usage: N/5 — <justification>
- Error handling: N/5 — <justification>
## Layer 4: HITL Checkpoints — N/25
- Business logic: PASS/ACCEPTABLE/NEEDS_WORK
- UX review: PASS/ACCEPTABLE/NEEDS_WORK/SKIPPED
- Security review: PASS/ACCEPTABLE/NEEDS_WORK/SKIPPED
- Performance: PASS/ACCEPTABLE/NEEDS_WORK/SKIPPED
## Trend
<comparison to previous evals for this module, if any>
--configure)Interactive setup that creates .context-index/evals/config.yaml:
# Eval harness configuration
layers:
deterministic: true # Layer 1 always on
architectural: true # Layer 2
llm_judge: true # Layer 3
hitl: false # Layer 4 (opt-in)
thresholds:
minimum_score: 60 # Below this triggers a warning
exemplary_score: 90 # Above this flags as golden sample candidate
rubric: default # Or path to custom rubric file
/adev:validate is the prerequisite (pass/fail gate). /adev:eval adds graduated scoring on top./adev:retro reads eval scores for quality trend analysis across sprints/adev:sample candidates can be identified from A-grade evaluations/adev:hygiene can include eval score trends in its drift report