Standardmäßig ist der Prompt ausgewählt, der zuerst die Quelle prüft. Sie können zu einem direkten Befehl wechseln oder eine lokale Kopie herunterladen.
Quelldateien prüfen
Lesen Sie SKILL.md und alle von SkillsMP angezeigten Begleitdateien, bevor Sie sich für eine Installation entscheiden.
Mit Codex oder Claude installieren Kopieren Sie diesen Prompt, fügen Sie ihn in Codex, Claude oder einen anderen Assistant ein und lassen Sie die Skill-Seite prüfen und installieren.
Ein direkter Befehl überspringt den Prüf-Prompt. Prüfen Sie die Quelle, bevor Sie ihn ausführen.
["Always evaluate all 5 dimensions before computing composite score","Cite specific evidence from the output being evaluated for each dimension score","Use the weighted composite (not simple average) for final verdict","Pair with verification-before-completion for pre-completion quality gates","Document evaluation verdicts in task metadata for traceability"]
error_handling
graceful
streaming
supported
source
builtin
trust_score
100
provenance_sha
cd4fd70eeb53b06c
Agent Evaluation
Overview
LLM-as-judge evaluation framework that scores AI-generated content on 5 dimensions using a 1-5 rubric. Agents evaluate outputs, compute a weighted composite score, and emit a structured verdict with evidence citations.
Core principle: Systematic quality verification before claiming completion. Agent-studio currently has no way to verify agent output quality — this skill fills that gap.
When to Use
Always:
Before marking a task complete (pair with verification-before-completion)
After a plan is generated (evaluate plan quality)
After code review outputs (evaluate review quality)
During reflection cycles (evaluate agent responses)
When comparing multiple agent outputs
Don't Use:
For binary pass/fail checks (use verification-before-completion instead)
For security audits (use security-architect skill)
For syntax/lint checking (use pnpm lint:fix)
The 5-Dimension Rubric
Every evaluation scores all 5 dimensions on a 1-5 scale:
Dimension
Weight
What It Measures
Accuracy
30%
Factual correctness; no hallucinations; claims are verifiable
Groundedness
25%
Claims are supported by citations, file references, or evidence from the codebase
Coherence
15%
Logical flow; internally consistent; no contradictions
Completeness
20%
All required aspects addressed; no critical gaps
Helpfulness
10%
Actionable; provides concrete next steps; reduces ambiguity
Scoring Scale (1-5)
Score
Meaning
5
Excellent — fully meets the dimension's criteria with no gaps
4
Good — meets criteria with minor gaps
3
Adequate — partially meets criteria; some gaps present
2
Poor — significant gaps or errors in this dimension
Evidence: Direct quote or file reference from the evaluated output
Rationale: Why this score was given (1-2 sentences)
Dimension 1: Accuracy
Checklist:
- [ ] Claims are factually correct (verify against codebase if possible)
- [ ] No hallucinated file paths, function names, or API calls
- [ ] Numbers and counts are accurate
- [ ] No contradictions with existing documentation
Dimension 2: Groundedness
Checklist:
- [ ] Claims cite specific files, line numbers, or task IDs
- [ ] Recommendations reference observable evidence
- [ ] No unsupported assertions ("this is probably X")
- [ ] Code examples use actual project patterns
Dimension 3: Coherence
Checklist:
- [ ] Logical flow from problem → analysis → recommendation
- [ ] No internal contradictions
- [ ] Terminology is consistent throughout
- [ ] Steps are in a rational order
Dimension 4: Completeness
Checklist:
- [ ] All required aspects of the task are addressed
- [ ] Edge cases are mentioned (if relevant)
- [ ] No critical gaps that would block action
- [ ] Follow-up steps are included
Dimension 5: Helpfulness
Checklist:
- [ ] Provides actionable next steps (not just observations)
- [ ] Concrete enough to act on without further clarification
- [ ] Reduces ambiguity rather than adding it
- [ ] Appropriate for the intended audience
// Load plan documentRead({ file_path: '.claude/context/plans/auth-design-plan-2026-02-21.md' });
// Evaluate against 5-dimension rubricSkill({ skill: 'agent-evaluation' });
// Provide the plan content as the output to evaluate
Evaluate Agent Response Before Completion
// Agent generates implementation summary// Before marking task complete, evaluate the summary qualitySkill({ skill: 'agent-evaluation' });
// If composite < 3.5, request improvements before TaskUpdate(completed)
Evaluate Code Review Output
// After code-reviewer runs, evaluate the review qualitySkill({ skill: 'agent-evaluation' });
// Ensures review is grounded in actual code evidence, not assertions
Batch Evaluation (comparing two outputs)
// Evaluate output A// Save verdict A// Evaluate output B// Save verdict B// Compare composites → choose higher scoring output
Integration with Verification-Before-Completion
The recommended quality gate pattern:
// Step 1: Do the work// Step 2: Evaluate with agent-evaluationSkill({ skill: 'agent-evaluation' });
// If verdict is POOR or FAILING → rework before proceeding// If verdict is ADEQUATE or better → proceed to verification// Step 3: Final gateSkill({ skill: 'verification-before-completion' });
// Step 4: Mark completeTaskUpdate({ taskId: 'X', status: 'completed' });
Iron Laws
NO COMPLETION CLAIM WITHOUT EVALUATION EVIDENCE — If composite score < 2.5 (POOR or FAILING), rework the output before marking any task complete.
ALWAYS score all 5 dimensions — never skip dimensions to save time; each dimension catches different failure modes (accuracy ≠ completeness ≠ groundedness).
ALWAYS cite specific evidence for every dimension score — "Evidence: [file:line or direct quote]" is mandatory, not optional. Assertions without grounding are invalid.
ALWAYS use the weighted composite — accuracy×0.30 + groundedness×0.25 + completeness×0.20 + coherence×0.15 + helpfulness×0.10. Never use simple average.
NEVER evaluate before the work is complete — evaluating incomplete outputs produces falsely low scores and wastes context budget.
Anti-Patterns
Anti-Pattern
Why It Fails
Correct Approach
Skipping dimensions to save time
Each dimension catches different failures
Always score all 5 dimensions
No evidence citation per dimension
Assertions without grounding are invalid
Quote specific text or file:line for every score
Using simple average for composite
Accuracy (30%) matters more than helpfulness (10%)
Use the weighted composite formula
Only checking EXCELLENT vs FAILING
ADEQUATE outputs need targeted improvements, not full rework
Use all 5 verdict tiers with appropriate action per tier
Evaluating before work is done
Incomplete outputs score falsely low
Evaluate completed outputs only
Treating evaluation as binary gate
Quality is a spectrum; binary pass/fail loses nuance
Use composite score + per-dimension breakdown together
Assigned Agents
This skill is used by:
qa — Primary: validates test outputs and QA reports before completion
