메뉴
How to integrate scenario evaluation into verification reports.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
Complete reference for the factory CLI — all commands, arguments, options, and usage. Use when an agent needs to invoke factory commands, check correct syntax, or understand available options.
When and how to write a human-action-needed entry to memory/{agent}/needs.md. Agents write these entries when they hit blockers that only a human can resolve.
When you encounter a failure, correction, or new pattern, write a learning and optionally propose a skill.
How the factory workspace is organized and what each directory means.
What to examine during a reflection pass and how to write observations to memory/{agent}/needs.md.
When and how to write scenario holdouts for factory verification.
| name | verification-with-scenarios |
| description | How to integrate scenario evaluation into verification reports. |
| for | verifier |
Every verification report includes a ## Scenario Evaluation section placed
after the spec-matching section.
For each scenario file found in scenarios/{project}/ (excluding _template.md
and satisfaction.md):
Failed scenarios are described with the same specificity as failed spec requirements — state exactly what the implementation does and what it should do.
When no scenarios exist for the project, include the section with:
No scenario holdouts found for this project.
Scenario evaluation contributes to but does not solely determine the satisfaction score. A strong spec match with poor scenario coverage is still a concern worth documenting in the report.
Evaluate factory infrastructure tasks directionally against
scenarios/meta/factory-itself.md: does this change move the factory closer to
or further from the meta-scenario? Document the reasoning. Do not apply pass/fail.
