Menu
How to integrate scenario evaluation into verification reports.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Based on SOC occupation classification
| name | verification-with-scenarios |
| description | How to integrate scenario evaluation into verification reports. |
| for | verifier |
Every verification report includes a ## Scenario Evaluation section placed
after the spec-matching section.
For each scenario file found in scenarios/{project}/ (excluding _template.md
and satisfaction.md):
Failed scenarios are described with the same specificity as failed spec requirements — state exactly what the implementation does and what it should do.
When no scenarios exist for the project, include the section with:
No scenario holdouts found for this project.
Scenario evaluation contributes to but does not solely determine the satisfaction score. A strong spec match with poor scenario coverage is still a concern worth documenting in the report.
Evaluate factory infrastructure tasks directionally against
scenarios/meta/factory-itself.md: does this change move the factory closer to
or further from the meta-scenario? Document the reasoning. Do not apply pass/fail.
Complete reference for the factory CLI — all commands, arguments, options, and usage. Use when an agent needs to invoke factory commands, check correct syntax, or understand available options.
When and how to write a human-action-needed entry to memory/{agent}/needs.md. Agents write these entries when they hit blockers that only a human can resolve.
When you encounter a failure, correction, or new pattern, write a learning and optionally propose a skill.
How the factory workspace is organized and what each directory means.
What to examine during a reflection pass and how to write observations to memory/{agent}/needs.md.
When and how to write scenario holdouts for factory verification.
