メニュー
Design a machine-checkable verifier for an agentic loop task. Use when defining definition-of-done, writing a new loop task, or when a loop fails because the success criterion is ambiguous.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
Scaffold a bring-your-own-repo loop task: find a failing verifier, write tasks/<id>.yaml, and run the loop in the user's codebase. Use when applying Loop Engineering outside atlas-oms or when the user says BYO or "my own repo."
Run a disciplined agentic loop on a verifiable task: goal → act → verify → correct → repeat until green or blocked. Use when executing a loop task with a machine-checkable verifier, or when the user asks to "run the loop."
Run prompt-only vs agent-loop vs background-agent comparison on the same task and log structured metrics. Use for workshop demos, A/B evidence, or cohort measurement.
| name | verifier-crafting |
| description | Design a machine-checkable verifier for an agentic loop task. Use when defining definition-of-done, writing a new loop task, or when a loop fails because the success criterion is ambiguous. |
The highest-leverage surface in Loop Engineering is the verifier. A loop is a prompt plus a signal the agent can iterate against.
| Property | Good | Bad |
|---|---|---|
| Output | Pass/fail exit code, unambiguous | Human judgment ("looks correct") |
| Scope | Tests the seeded pain point directly | Tests unrelated happy path only |
| Speed | Runs in seconds to low minutes | Full E2E suite per iteration |
| Stability | Deterministic on seeded state | Flaky or environment-dependent |
| Regression | Catches collateral breakage | Only checks the edited file |
Write one sentence: what is broken in the seeded state? The verifier must fail because of that pain, not because of a missing dependency.
| Signal type | When to use | Example |
|---|---|---|
| Unit / integration test | Logic bugs, race conditions, API contracts | pytest tests/order/test_status_transition.py |
| Type checker | Incorrect types, missing annotations | mypy src/orders/ |
| Linter | Style + some correctness gates | ruff check src/orders/ |
| Build | Compile errors, bundling | npm run build |
| Perf assertion | Regression on latency/throughput | pytest tests/perf/ --benchmark-only |
Prefer the narrowest verifier that still catches the root cause.
Run the verifier on the unmodified codebase. It must fail. If it passes on the broken state, the verifier does not encode the pain point — redesign.
Apply the reference solution (or known-good patch). The verifier must pass. If it still fails, the verifier or reference is wrong.
For rung 2+ tasks, add broader gates the agent must not break:
primary: pytest tests/order/test_status_transition.py
regression: pytest tests/order/ && mypy src/orders/
task_id: oms-status-race-001
verifier: pytest tests/order/test_status_transition.py
regression_gates: mypy src/orders/ && ruff check src/orders/
expected_iters: 3-5
common_fails: agent patches assertion without fixing race