review-os

name	review-os
description	Audit a generated OS repository for completeness, consistency, and architectural integrity against the 15 design principles and 23-item checklist.

/review-os - OS Architecture Audit

When to Use

• After /generate-os produces the OS repository
• When the designer wants to verify the OS is complete and consistent
• Before deploying the OS for real use

Inputs

• Required: A generated OS at output/[domain]-os/
• CRITICAL: reference/BLUEPRINT.md (master specification — the OS is audited against this)
• Auto-loaded: ALL domain-input/ files (as the spec to audit against)
• Auto-loaded: reference/architecture-patterns.md (the 15 principles to verify)

Process

Audit 1: Structural Completeness

Check that all expected files exist:

[ ] CLAUDE.md exists and is non-empty
[ ] DATA_CONTRACT.md exists and classifies all files
[ ] README.md exists
[ ] START-HERE.md exists with time estimates
[ ] .claude/skills/ has [N] skill folders (match skill-designs.md count)
[ ] Each skill folder has SKILL.md
[ ] sub-agents/ has [N] files (match sub-agent-designs.md count)
[ ] context-library/ has [N] template files (match identity-model.md count)
[ ] config/ has defaults.md and overrides.md
[ ] cowork-tasks/ has briefing files
[ ] setup/ has installation guide and checklist
[ ] insider-data/ has at minimum an index.md
[ ] batch/ exists (if batch-designs.md was provided)
[ ] modes/ exists (if localization-designs.md was provided)
[ ] backtesting/ exists (if backtesting-library.md was provided)
[ ] templates/ has artifact templates (if artifact-pipeline.md was provided)

Audit 2: Kernel Integrity

Read CLAUDE.md and verify:

[ ] Core philosophy present (3-5 rules)
[ ] Context awareness lists ALL context-library and config files
[ ] Data contract summary present with classification table and Golden Rule
[ ] Skills table lists ALL skills with correct /commands
[ ] Sub-agents table lists ALL sub-agents
[ ] Auto-behaviors section lists ALL trigger chains
[ ] Verification rules present and specific
[ ] Data acquisition rules present (if external data sources exist)
[ ] Empty context detection covers ALL context dependencies
[ ] Configuration precedence stated (overrides > defaults)
[ ] Writing style section present
[ ] Non-negotiable rule appears early and in bold
[ ] Getting started section present

Audit 3: Skill Consistency

For each SKILL.md:

[ ] Frontmatter has name and description
[ ] "When to Use" section present
[ ] Inputs list auto-loaded files that exist in context-library/
[ ] Process is step-by-step (not a vague description)
[ ] Output format is a concrete template
[ ] Quality checks have "good" AND "bad" examples
[ ] Empty context behavior defined for all required files
[ ] Auto-triggers match what CLAUDE.md declares (dual declaration check)
[ ] Configuration section reads overrides > defaults (if applicable)
[ ] Artifact Output section present (if skill produces non-text output)
[ ] Batch Mode Variant section present (if skill supports batch)
[ ] Data Source Dependencies section present (if skill uses external data)

Audit 4: Sub-Agent Adversarial Check

For each sub-agent:

[ ] Has a vivid persona (not generic "reviewer")
[ ] Triggers after a specific creation skill
[ ] Scoring dimensions DIFFER from the creation skill's process
[ ] "Good review" and "bad review" criteria defined
[ ] Output format includes specific scoring and actionable fixes
[ ] Artifact review capability present (if reviewing non-text output)

Critical check: For each sub-agent, answer: "What does this reviewer catch that the creation skill would miss?"

Audit 4.5: Data Contract Integrity

[ ] DATA_CONTRACT.md exists
[ ] Every persistent file in the OS is classified (user/system/hybrid)
[ ] The Golden Rule is specific and testable
[ ] No skill writes to a user-owned file without explicit user action
[ ] System-managed files are auto-updatable
[ ] Hybrid files have clear append/curate semantics
[ ] Classification matches actual skill behavior
[ ] config/overrides.md is classified as user-owned
[ ] config/defaults.md is classified as system-managed

Audit 5: Loop Integrity

For each feedback loop:

[ ] Data producer skill exists and writes to a real file
[ ] Data consumer skill exists and reads from the same file
[ ] Degradation behavior defined for 0, 1-2, 3-9, 10+ data points
[ ] Retro skill reads from all accumulating files
[ ] Briefing evolves as data accumulates

Audit 5.5: Artifact Pipeline Integrity

If artifact pipelines exist:

[ ] Each artifact type has a declared template in templates/
[ ] Design tokens are centralized (not scattered across skills)
[ ] Render pipeline is step-by-step and specific
[ ] Storage path and naming convention are declared
[ ] Verification criteria exist per artifact format
[ ] Fallback behavior defined for unavailable render tools
[ ] Naming convention is consistent across artifact types

Audit 5.6: Batch Architecture Integrity

If batch/ directory exists:

[ ] batch/batch-prompt.md is self-contained (no conversation dependency)
[ ] batch/batch-state.tsv has a defined schema
[ ] Status values are documented (pending, completed, failed, etc.)
[ ] Merge process is defined and idempotent
[ ] Lock file mechanism exists
[ ] Concurrency limits are specified
[ ] Resume/retry mechanisms exist
[ ] Workers do not depend on exclusive-access tools without constraints

Audit 5.7: Data Source Trust Levels

If external data sources exist:

[ ] Every source has an explicit trust level (1-3)
[ ] Level 3 sources have specific verification steps
[ ] Verification labels are standardized ([VERIFIED], [UNVERIFIED], [STALE])
[ ] Fallback chains exist for critical data paths
[ ] Staleness thresholds are defined per data type
[ ] Kernel's data acquisition rules section matches source definitions

Audit 5.8: Backtesting Library Integrity

If backtesting/ directory exists:

[ ] backtesting/library.md exists with case index and ordering scheme
[ ] Each case has a subdirectory in backtesting/cases/ with input.md and ground-truth.md
[ ] Input files contain ONLY the blank input (no answers, no ground truth content)
[ ] Ground truth files contain the user's actual delivered output
[ ] Physical isolation enforced: input.md and ground-truth.md are separate files in separate paths
[ ] No content embedding: metadata uses directional references only (filename, not inline content)
[ ] Cross-contamination validation exists: onboarding or ingest step checks for phrase overlap between input and ground truth
[ ] Cold-start test documented: can each backtestable skill produce output from input.md alone?
[ ] Library index declares ordering scheme (date, importance, or outcome)
[ ] At least one skill declares live reference behavior (reads ground truth during generation)
[ ] backtesting/results/ directory exists for system-generated comparison reports
[ ] DATA_CONTRACT.md classifies input.md and ground-truth.md as user-owned
[ ] DATA_CONTRACT.md classifies results/*.md as system-managed
[ ] Onboarding (START-HERE.md or setup/) includes backtesting library setup instructions
[ ] Minimum viable library size is stated (typically 3-5 cases)
[ ] Kernel includes backtesting library section with live reference rules
[ ] Background backtest infrastructure exists: backtest-prompt.md (self-contained worker prompt)
[ ] Background backtest infrastructure exists: backtest-runner.sh (orchestrator with lock file, dry-run, resume)
[ ] Background backtest infrastructure exists: backtest-state.tsv (per-case progress tracking)
[ ] Backtest runner includes pre-flight isolation validation before processing each case

Critical check: For each backtestable skill, verify that its SKILL.md includes a "Backtesting Integration" or "Live Reference" section that reads from the library during generation.

Audit 6: Blueprint Conformance (BLUEPRINT.md Sections 2-8)

Component Map (Section 2):

[ ] CLAUDE.md (Kernel)
[ ] DATA_CONTRACT.md (Ownership Boundaries)
[ ] context-library/ (State) — 4-7 files
[ ] config/ (Layered Configuration) — defaults + overrides
[ ] .claude/skills/ (Capabilities) — 8-18 skills
[ ] sub-agents/ (Quality Gates) — 2-4 reviewers
[ ] insider-data/ (Reference Knowledge)
[ ] cowork-tasks/ (Automation) — 2-3 part briefing
[ ] templates/ (Scaffolding + Artifact Blueprints)
[ ] setup/ (Onboarding)
[ ] [output-folder]/ (Logs + Artifacts)
[ ] batch/ (Worker Infrastructure) — if applicable
[ ] modes/ (Localized Modes) — if applicable
[ ] backtesting/ (Historical Calibration) — if applicable

Orchestration (Section 4):

[ ] Trigger chains documented for every creation skill (4a)
[ ] Connector map: every context file has readers/writers listed (4b)
[ ] At least 2 compounding loops fully wired (4c)
[ ] Dual declaration: every auto-behavior in BOTH kernel AND skill (4d)
[ ] Data contract enforced at runtime (4e)
[ ] Artifact pipelines wired (4f) — if applicable
[ ] Batch execution model documented (4g) — if applicable
[ ] Trust-layered acquisition documented (4h) — if applicable
[ ] Backtesting library wired (4i) — if applicable

Generation Checklist (Section 8) — all 23 items:

[ ] 1. One kernel file
[ ] 2. User state externalized
[ ] 3. Skills are stateless pure functions
[ ] 4. Every generation skill has adversarial reviewer
[ ] 5. One zero-setup entry point
[ ] 6. Auto-behaviors dual-declared
[ ] 7. Verification mandatory per skill
[ ] 8. Compounding loops exist
[ ] 9. Persona-conditional branching
[ ] 10. Graceful degradation
[ ] 11. Function-adaptive markers
[ ] 12. Scheduled automation
[ ] 13. Dependency table in kernel
[ ] 14. Connector map documented
[ ] 15. Trigger chains documented
[ ] 16. Two-system boundary enforced
[ ] 17. DATA_CONTRACT.md exists classifying every file
[ ] 18. Layered configuration (defaults + overrides)
[ ] 19. Artifact pipelines declared (if applicable)
[ ] 20. External data sources have trust levels (if applicable)
[ ] 21. Batch operations are resumable (if applicable)
[ ] 22. Orthogonal evaluation axes
[ ] 23. Backtesting library (if applicable) — cases, ordering, live reference, onboarding

15 Design Principles (Section 6):

[ ] P1: Separation of identity from capability
[ ] P2: Generation ≠ evaluation
[ ] P3: Declarative orchestration (dual declaration)
[ ] P4: State in flat files
[ ] P5: Compounding feedback loops
[ ] P6: Graceful degradation with quality gates
[ ] P7: Function-adaptive markers
[ ] P8: Zero-setup entry point
[ ] P9: Trust-layered data acquisition (if external data)
[ ] P10: Orthogonal evaluation axes
[ ] P11: Formal data contract
[ ] P12: Resumable idempotent operations (if batch)
[ ] P13: Artifact-as-deliverable (if non-text output)
[ ] P14: Layered configuration inheritance
[ ] P15: Historical backtesting library (if applicable)

Two-System Boundary Check:

[ ] No skill contains hardcoded user data
[ ] No context-library file contains skill logic
[ ] All context-library files ship as empty templates
[ ] Every skill reads user identity from context-library at runtime
[ ] domain-input/ content does NOT leak into the generated OS
[ ] No references to agent-os-factory anywhere in the OS
[ ] The OS has its own onboarding: START-HERE.md, setup/, empty-context messages

Audit 7: Cross-Reference Check

[ ] Every file path in CLAUDE.md points to a real file
[ ] Every "auto-loaded" reference in skills points to a real context file
[ ] Every sub-agent trigger references a real skill
[ ] Every auto-behavior's target skill exists
[ ] Briefing references correct file paths for output
[ ] DATA_CONTRACT.md references match actual files
[ ] Batch prompt references match actual context files

Output

# OS Architecture Audit - [Domain] OS

## Summary
- Files checked: [N]
- Issues found: [N] critical, [N] warnings
- Architectural principles: [N]/15 verified
- Checklist items: [N]/23 passed

## Critical Issues (must fix)
1. [Issue]: [where] — [what's wrong] — [how to fix]

## Warnings (should fix)
1. [Issue]: [where] — [what's wrong] — [how to fix]

## Verified
- [list of all checks that passed]

## Recommendation
[READY TO DEPLOY / FIX CRITICAL ISSUES FIRST / NEEDS SIGNIFICANT REWORK]

After the audit, offer to fix any issues found:

• Critical issues: fix automatically
• Warnings: describe the fix and ask if the designer wants it applied
• If READY TO DEPLOY: congratulate and suggest copying the output folder

Quality Checks

Good audit:

• Every check produces a specific pass/fail, not a vague assessment
• Critical issues have specific fix instructions
• Cross-reference check catches broken file paths
• Adversarial check for each sub-agent produces a clear answer
• Data contract verified against actual skill behavior
• Checklist items 17-23 verified (not just 1-16)

Bad audit:

• Rubber stamp ("everything looks good")
• Vague findings ("some skills could be more detailed")
• Missing cross-reference check
• Doesn't verify dual declaration of auto-behaviors
• Skips infrastructure audits (data contract, artifact, batch, trust levels, backtesting)