validation-diagnostics

name	validation-diagnostics
description	Validation and diagnostic skill for HASTE. Compare planned vs implemented work, generate diagnostic reports, and feed misses back into skill refinement. Use when: 'validate implementation', 'compare to spec', 'diagnostic report', 'drift analysis', 'coverage check', 'implementation review'.
source	HASTE validation practices
domain	quality
level	intermediate
agents	["backend-validation","ui-validation","security-validation","orchestrator"]
created_date	2026-04-27
last_validated
validated_by
status	draft

Validation & Diagnostics

Overview

Structured process for comparing planned vs implemented work, generating diagnostic reports, and identifying gaps. Used by validation agents to provide concrete, evidence-based assessments.

Key Concepts

The Trust Problem

Agents will claim work is complete when it isn't. Validation must be:

• Observable — Based on test output, not agent claims
• Deterministic — Same input produces same verdict
• Evidence-based — Every finding references concrete code or test results
• Structured — Consistent format for easy human review

Patterns & Techniques

Planned vs Implemented Comparison

Step 1: Extract planned items From the spec, issue, or plan, extract a checklist of:

• Acceptance criteria
• Required endpoints/functions
• Expected model fields
• Required test coverage
• UI components specified

Step 2: Verify each item For each planned item, check:

• Does the code exist? (file search, grep)
• Does it match the specification? (read and compare)
• Is it tested? (find corresponding test)
• Does the test pass? (run and capture output)

Step 3: Generate drift report

## Drift Analysis: [Feature]

| Planned Item | Status | Evidence |
|-------------|--------|----------|
| [spec item] | ✅ Implemented | [file:line] |
| [spec item] | ⚠️ Partial | [what's missing] |
| [spec item] | ❌ Not found | [searched in...] |
| [unplanned] | ⚡ Scope creep | [file:line] |

Diagnostic Report Template

## Diagnostic Report: [Component/Feature]

### Summary
[1-2 sentence verdict]

### Test Results

[Actual test output — copy/paste, not paraphrased]

### Code Quality
| Metric | Result |
|--------|--------|
| Type hints present | ✅ / ❌ |
| Pydantic models used | ✅ / ❌ |
| Config class used (no hardcoded secrets) | ✅ / ❌ |
| Error handling present | ✅ / ❌ |
| Logger used (not print) | ✅ / ❌ |

### Findings
| # | Severity | Finding | Location | Recommendation |
|---|----------|---------|----------|----------------|
| 1 | [High/Med/Low] | [what] | [file:line] | [fix] |

### Coverage Gaps
[Code paths without tests]

### Verdict
✅ PASS | ⚠️ CONDITIONAL PASS | ❌ FAIL
[Explanation with evidence]