// Systematic validation methodology for ResearchPacks and Implementation Plans. Provides scoring rubrics and quality gates to ensure outputs meet standards before proceeding to next phase. Prevents garbage-in-garbage-out scenarios.
Systematic methodology for identifying, capturing, and documenting reusable patterns from implementations. Enables automatic learning and knowledge-core.md updates. Claude invokes this after successful implementations to preserve institutional knowledge.
Active context curation to fight context rot. Curates what goes into limited context window from constantly evolving information universe. 39% improvement, 84% token reduction.
Systematic approach for creating minimal-change, reversible implementation plans. Claude invokes this skill when transforming requirements/research into executable blueprints. Emphasizes simplicity, safety, and clear verification steps.
Systematic approach for gathering authoritative, version-accurate documentation. Claude invokes this skill when research is needed before implementation. Ensures truth over speed while achieving both.
| name | quality-validation |
| description | Systematic validation methodology for ResearchPacks and Implementation Plans. Provides scoring rubrics and quality gates to ensure outputs meet standards before proceeding to next phase. Prevents garbage-in-garbage-out scenarios. |
| auto_invoke | true |
| tags | ["validation","quality","verification","gates"] |
This skill provides systematic validation methodology to ensure ResearchPacks and Implementation Plans meet quality standards before proceeding to implementation.
Claude will automatically invoke this skill when:
Before scoring, detect research type to apply appropriate rubric:
Indicators:
Scoring: Use API Research Rubric (80+ pass threshold)
Indicators:
Scoring: Use Philosophy Research Rubric (70+ pass threshold)
Examples: Engineering philosophy, architectural patterns, best practices, methodology research
Indicators:
Scoring: Use Pattern Research Rubric (70+ pass threshold)
Why Different Thresholds?
Purpose: Ensure research is complete, accurate, and actionable before planning
Validation Rubric for API/Library Research (100 points total, 80+ pass threshold):
Passing Score: 80/100 or higher
Validation Process:
# Pseudo-code for validation logic
def validate_research_pack(research_pack):
score = 0
defects = []
# Completeness checks
if has_library_with_version(research_pack):
score += 10
else:
defects.append("CRITICAL: Library/version not identified")
api_count = count_documented_apis(research_pack)
if api_count >= 3:
score += 10
elif api_count > 0:
score += (api_count / 3) * 10
defects.append(f"MINOR: Only {api_count} APIs documented, need 3+")
else:
defects.append("CRITICAL: No APIs documented")
# ... (continue for all criteria)
return {
"score": score,
"grade": "PASS" if score >= 80 else "FAIL",
"defects": defects,
"recommendations": generate_recommendations(defects)
}
Output Format:
## 📊 ResearchPack Validation Report
**Overall Score**: [X]/100
**Grade**: [PASS ✅ / FAIL ❌]
### Breakdown
- Completeness: [X]/40
- Accuracy: [X]/30
- Citation: [X]/20
- Actionability: [X]/10
### Defects Found ([N])
#### CRITICAL (blocks implementation)
1. [Specific defect with example]
2. [Another defect]
#### MAJOR (should fix before proceeding)
1. [Defect]
#### MINOR (nice to have)
1. [Defect]
### Recommendations
**To reach passing score**:
1. [Specific action to take]
2. [Another action]
**If score >= 80**: ✅ **APPROVED** - Proceed to implementation-planner
**If score < 80**: ❌ **BLOCKED** - Fix critical/major defects and re-validate
Purpose: Ensure philosophy/pattern research is well-organized, sourced, and actionable
Validation Rubric for Philosophy Research (100 points total, 70+ pass threshold):
Passing Score: 70/100 or higher
Why Lower Threshold Than API Research?
Philosophy research is inherently more subjective and thematic. A well-organized thematic analysis with 7 patterns from 11 sources (like the Anthropic ResearchPack) deserves to pass even if it doesn't have "3+ API endpoints with exact signatures."
Philosophy research provides strategic value:
Example: Anthropic Engineering Philosophy ResearchPack
Would score:
Output Format:
## 📊 ResearchPack Validation Report (Philosophy Research)
**Overall Score**: [X]/100
**Grade**: [PASS ✅ / FAIL ❌]
**Research Type**: Philosophy/Pattern Research
### Breakdown
**Thematic Organization** ([X]/30):
- Clear themes: [Y/10] [✓/✗]
- Theme documentation: [Y/10] [✓/✗]
- Cross-synthesis: [Y/10] [✓/✗]
**Source Quality** ([X]/20):
- Official sources: [Y/10] [✓/✗]
- Multiple sources per theme: [Y/5] [✓/✗]
- Date/version info: [Y/5] [✓/✗]
**Actionable Insights** ([X]/30):
- Implementation checklist: [Y/15] [✓/✗]
- Patterns extracted: [Y/10] [✓/✗]
- Open questions: [Y/5] [✓/✗]
**Depth & Coverage** ([X]/20):
- Comprehensive coverage: [Y/10] [✓/✗]
- Sufficient detail: [Y/10] [✓/✗]
### Defects Found ([N])
#### CRITICAL (blocks implementation)
1. [Defect - if no themes identified, no patterns extracted, etc.]
#### MAJOR (should fix before proceeding)
1. [Defect - if only 1 source per theme, missing implementation checklist, etc.]
#### MINOR (nice to have)
1. [Defect - if some themes lack examples, could use more sources, etc.]
### Recommendations
**To reach passing score** (if < 70):
1. [Specific action to take]
2. [Another action]
**If score >= 70**: ✅ **APPROVED** - Proceed to implementation-planner
**If score < 70**: ❌ **BLOCKED** - Fix critical/major defects and re-validate
**Philosophy Research Note**: This research provides strategic guidance for implementation. Even if specific API details are needed later, the principles and patterns documented here are valuable for decision-making.
Purpose: Ensure plan is complete, safe, and executable before coding
Validation Rubric (100 points total):
Passing Score: 85/100 or higher (higher bar than research)
Validation Process:
def validate_implementation_plan(plan, research_pack):
score = 0
defects = []
# Completeness checks
if has_file_changes_list(plan):
score += 10
else:
defects.append("CRITICAL: No file changes specified")
steps = extract_steps(plan)
if all(step_has_verification(s) for s in steps):
score += 10
else:
missing = [s for s in steps if not step_has_verification(s)]
score += (len(steps) - len(missing)) / len(steps) * 10
defects.append(f"MAJOR: Steps {missing} lack verification")
# Safety checks
rollback = extract_rollback_plan(plan)
if has_exact_commands(rollback) and has_triggers(rollback):
score += 15
elif has_rollback_section(plan):
score += 8
defects.append("MAJOR: Rollback plan incomplete (missing commands or triggers)")
else:
defects.append("CRITICAL: No rollback plan")
# Alignment checks
apis_used = extract_apis_from_plan(plan)
research_apis = extract_apis_from_research(research_pack)
if all(api_matches_research(a, research_apis) for a in apis_used):
score += 10
else:
mismatches = find_api_mismatches(apis_used, research_apis)
defects.append(f"CRITICAL: APIs don't match ResearchPack: {mismatches}")
# ... (continue for all criteria)
return {
"score": score,
"grade": "PASS" if score >= 85 else "FAIL",
"defects": defects,
"recommendations": generate_recommendations(defects)
}
Output Format:
## 📊 Implementation Plan Validation Report
**Overall Score**: [X]/100
**Grade**: [PASS ✅ / FAIL ❌]
### Breakdown
- Completeness: [X]/35
- Safety: [X]/30
- Clarity: [X]/20
- Alignment: [X]/15
### Defects Found ([N])
#### CRITICAL (blocks implementation)
1. [Specific defect]
#### MAJOR (should fix)
1. [Defect]
#### MINOR (nice to have)
1. [Defect]
### API Alignment Check
✅ All APIs match ResearchPack
OR
❌ Mismatches found:
- Plan uses `foo(x, y)` but ResearchPack shows `foo(x: string, y?: number)`
### Recommendations
**To reach passing score**:
1. [Action]
**If score >= 85**: ✅ **APPROVED** - Proceed to code-implementer
**If score < 85**: ❌ **BLOCKED** - Fix defects and re-validate
Gates are MANDATORY checkpoints - cannot proceed to next phase without passing validation.
Trigger: @docs-researcher completes ResearchPack
Action: Validate ResearchPack
Decision:
- Score >= 80: ✅ Allow @implementation-planner to proceed
- Score < 80: ❌ Block, return to @docs-researcher with defect list
Trigger: @implementation-planner completes Implementation Plan
Action: Validate Implementation Plan + check alignment with ResearchPack
Decision:
- Score >= 85 AND APIs match: ✅ Allow @code-implementer to proceed
- Score < 85 OR APIs mismatch: ❌ Block, return to @implementation-planner with defect list
Trigger: @code-implementer reports completion
Action: Validate tests passed, build succeeded, no regressions
Decision:
- All checks pass: ✅ Mark complete
- Any check fails: ❌ Trigger self-correction loop (up to 3 attempts)
These validations should be automated via hooks (see hooks implementation):
{
"hooks": {
"PreToolUse": [
{
"matcher": "implementation-planner",
"command": "validate-research-pack.sh",
"action": "block_if_fails"
},
{
"matcher": "code-implementer",
"command": "validate-implementation-plan.sh",
"action": "block_if_fails"
}
]
}
}
Validation scripts return:
Hallucinated APIs:
❌ CRITICAL: API `redis.client.fetch()` not found in official docs
ResearchPack cites: redis.io/docs/clients/nodejs
Actual API: `client.get()` (verified at redis.io/docs/clients/nodejs#get)
FIX: Replace all instances of `fetch` with correct `get` API
Version mismatch:
❌ MAJOR: ResearchPack uses v3.x docs but project has v4.6.0
Example: v3 uses callbacks, v4 uses promises
FIX: Re-fetch docs for v4.6.0 specifically
Missing citations:
❌ MAJOR: 5 APIs listed without source URLs
APIs: set(), del(), ttl(), exists(), keys()
FIX: Add source URL for each (format: docs.com/path#section)
No rollback plan:
❌ CRITICAL: Rollback plan missing
FIX: Add section "## 🔄 Rollback Plan" with:
- Exact git commands to revert
- Configuration restoration steps
- Verification after rollback
- Triggers for when to rollback
Ambiguous steps:
❌ MAJOR: Step 3 says "Update the service" (too vague)
FIX: Specify:
- Which service? (path/to/ServiceName.ts)
- What update? (Add method X, modify method Y)
- How to verify? (run `npm test path/to/test.ts`)
API misalignment:
❌ CRITICAL: Plan uses `client.fetch(key)` but ResearchPack shows `client.get(key)`
FIX: Update plan to use correct API signature from ResearchPack
Hooks provide deterministic enforcement (always run, not LLM-dependent):
Research validation hook:
#!/bin/bash
# .claude/hooks/validate-research-pack.sh
RESEARCH_FILE="$1" # Path to ResearchPack file
# Check completeness
if ! grep -q "Target Library:" "$RESEARCH_FILE"; then
echo "❌ CRITICAL: Library not identified"
exit 1
fi
# Check API count
API_COUNT=$(grep -c "^###.*API" "$RESEARCH_FILE" || echo 0)
if [ "$API_COUNT" -lt 3 ]; then
echo "❌ MINOR: Only $API_COUNT APIs documented, need 3+"
# Don't block for this, just warn
fi
# Check citations
if ! grep -q "Source:" "$RESEARCH_FILE"; then
echo "❌ CRITICAL: No source citations found"
exit 1
fi
echo "✅ ResearchPack validation passed (score: [calculated]/100)"
exit 0
Plan validation hook (similar structure).
This skill ensures quality gates are objective, automated, and enforce the Research → Plan → Implement workflow deterministically.