| name | ai-evaluation |
| department | oracle |
| description | Use when designing an evaluation framework for AI/LLM features. Covers golden dataset creation, automated scoring rubrics, hallucination detection, regression testing infrastructure, and production monitoring. Do not use for prompt design (use prompt-engineering) or RAG pipeline architecture (use rag-architecture). |
| version | 1 |
| triggers | ["eval","evaluation","hallucination","golden dataset","scoring","benchmark","regression","quality","accuracy"] |
AI Evaluation
Purpose
Design an evaluation framework for AI/LLM features, including golden dataset creation, automated scoring rubrics, hallucination detection, and regression testing infrastructure.
Scope Constraints
Reads feature specifications, evaluation requirements, and existing infrastructure details for framework design. Does not execute model inference, create production datasets, or access live model endpoints directly.
Inputs
- AI feature being evaluated (what it does, expected behavior)
- Input data examples and edge cases
- Quality requirements (accuracy thresholds, hallucination tolerance)
- Existing evaluation infrastructure (if any)
- Production monitoring requirements
Input Sanitization
No user-provided values are used in commands or file paths. All inputs are treated as read-only analysis targets.
Procedure
Progress Checklist
Step 1: Define Evaluation Dimensions
Identify what "good" means for this feature:
- Correctness: Does the output match the expected answer?
- Faithfulness: Does the output only use information from the provided context?
- Relevance: Does the output answer the actual question asked?
- Completeness: Does the output cover all aspects of the question?
- Format compliance: Does the output match the expected structure?
- Safety: Does the output avoid harmful, biased, or inappropriate content?
Step 2: Build Golden Dataset
Create a high-quality evaluation dataset:
- Size: Minimum 50 examples, ideally 200+ for statistical significance
- Distribution: Cover common cases (60%), edge cases (25%), adversarial cases (15%)
- Labeling: Each example has input, expected output, and scoring criteria
- Source: Real user queries (anonymized) + synthetically generated edge cases
- Versioning: Dataset is version-controlled alongside the code
Step 3: Design Automated Scoring
Create scoring rubrics that can run without human review:
- Exact match: For classification, extraction, or structured output (score: 0 or 1)
- Semantic similarity: Embedding-based comparison of generated vs expected (score: 0-1)
- LLM-as-judge: Use a stronger model to evaluate the output (score: 1-5 rubric)
- Rule-based checks: Required fields present, format valid, no PII leaked
- Composite score: Weighted combination of individual dimensions
Step 4: Design Hallucination Detection
Build specific checks for fabricated content:
- Reference validation: Every cited fact must trace back to a source document
- Entity verification: Named entities (people, dates, numbers) must appear in context
- Confidence calibration: When the model says "I'm not sure," is it actually uncertain?
- Contradiction detection: Does the output contradict the provided context?
- Fabrication patterns: Common hallucination patterns to flag (fake URLs, invented citations)
Step 5: Design Regression Testing
Build a CI/CD-compatible evaluation pipeline:
- Trigger: Run on prompt changes, model upgrades, or code changes affecting AI features
- Threshold enforcement: Fail the build if eval score drops below threshold
- Comparison reporting: Show score delta vs previous version, highlight regressions
- Fast vs full: Quick smoke test (20 examples) for every commit, full eval (200+) for releases
Step 6: Design Production Monitoring
Plan ongoing quality monitoring:
- Sampling: Evaluate X% of production requests against automated scoring
- Feedback loop: User thumbs-up/down, explicit corrections
- Drift detection: Score distribution shift over time (model degradation, data drift)
- Alerting: Score drops below threshold, hallucination rate spikes, latency increases
Compaction resilience: If context was lost during a long session, re-read the Inputs section to reconstruct what system is being analyzed, check the Progress Checklist for completed steps, then resume from the earliest incomplete step.
Output Format
# AI Evaluation Framework
## Evaluation Dimensions
| Dimension | Weight | Scoring Method | Threshold |
|-----------|--------|---------------|-----------|
| Correctness | 40% | LLM-as-judge (1-5) | ≥ 4.0 |
| Faithfulness | 30% | Reference validation | ≥ 95% |
| Relevance | 20% | Semantic similarity | ≥ 0.85 |
| Format compliance | 10% | Rule-based | 100% |
## Golden Dataset
**Size:** [N examples]
**Distribution:**
| Category | Count | Description |
|----------|-------|-------------|
| Common cases | N | [Description] |
| Edge cases | N | [Description] |
| Adversarial | N | [Description] |
**Storage:** [Location in repo]
**Versioning:** [Approach]
## Scoring Rubric
### Correctness (LLM-as-Judge)
| Score | Criteria |
|-------|---------|
| 5 | Perfect — matches expected output in all aspects |
| 4 | Good — minor differences that don't affect usefulness |
| 3 | Acceptable — correct core answer with some issues |
| 2 | Poor — partially correct but missing key information |
| 1 | Wrong — incorrect or misleading answer |
## Hallucination Detection
| Check | Method | Severity |
|-------|--------|----------|
| Reference validation | [Approach] | Critical |
| Entity verification | [Approach] | High |
| Contradiction detection | [Approach] | High |
## Regression Testing Pipeline
[Code change] → [Smoke test (20 examples)] → [Pass?] → [Merge]
[Release] → [Full eval (200+ examples)] → [Pass threshold?] → [Deploy]
**Threshold:** Composite score ≥ [X] to pass
**Reporting:** [Where results are published]
## Production Monitoring
| Metric | Sample Rate | Alert Threshold |
|--------|------------|-----------------|
| Composite score | 5% of requests | < [X] |
| Hallucination rate | 5% of requests | > [X%] |
| User satisfaction | All feedback | < [X] thumbs-up rate |
Handoff
- Hand off to prompt-engineering if evaluation results reveal prompt design deficiencies requiring structured redesign.
- Hand off to rag-architecture if evaluation findings indicate retrieval quality or chunking strategy issues.
Quality Checks
Evolution Notes
