Menu
Design a machine-checkable verifier for an agentic loop task. Use when defining definition-of-done, writing a new loop task, or when a loop fails because the success criterion is ambiguous.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Scaffold a bring-your-own-repo loop task: find a failing verifier, write tasks/<id>.yaml, and run the loop in the user's codebase. Use when applying Loop Engineering outside atlas-oms or when the user says BYO or "my own repo."
Run a disciplined agentic loop on a verifiable task: goal → act → verify → correct → repeat until green or blocked. Use when executing a loop task with a machine-checkable verifier, or when the user asks to "run the loop."
Run prompt-only vs agent-loop vs background-agent comparison on the same task and log structured metrics. Use for workshop demos, A/B evidence, or cohort measurement.
Based on SOC occupation classification
| name | verifier-crafting |
| description | Design a machine-checkable verifier for an agentic loop task. Use when defining definition-of-done, writing a new loop task, or when a loop fails because the success criterion is ambiguous. |
The highest-leverage surface in Loop Engineering is the verifier. A loop is a prompt plus a signal the agent can iterate against.
| Property | Good | Bad |
|---|---|---|
| Output | Pass/fail exit code, unambiguous | Human judgment ("looks correct") |
| Scope | Tests the seeded pain point directly | Tests unrelated happy path only |
| Speed | Runs in seconds to low minutes | Full E2E suite per iteration |
| Stability | Deterministic on seeded state | Flaky or environment-dependent |
| Regression | Catches collateral breakage | Only checks the edited file |
Write one sentence: what is broken in the seeded state? The verifier must fail because of that pain, not because of a missing dependency.
| Signal type | When to use | Example |
|---|---|---|
| Unit / integration test | Logic bugs, race conditions, API contracts | pytest tests/order/test_status_transition.py |
| Type checker | Incorrect types, missing annotations | mypy src/orders/ |
| Linter | Style + some correctness gates | ruff check src/orders/ |
| Build | Compile errors, bundling | npm run build |
| Perf assertion | Regression on latency/throughput | pytest tests/perf/ --benchmark-only |
Prefer the narrowest verifier that still catches the root cause.
Run the verifier on the unmodified codebase. It must fail. If it passes on the broken state, the verifier does not encode the pain point — redesign.
Apply the reference solution (or known-good patch). The verifier must pass. If it still fails, the verifier or reference is wrong.
For rung 2+ tasks, add broader gates the agent must not break:
primary: pytest tests/order/test_status_transition.py
regression: pytest tests/order/ && mypy src/orders/
task_id: oms-status-race-001
verifier: pytest tests/order/test_status_transition.py
regression_gates: mypy src/orders/ && ruff check src/orders/
expected_iters: 3-5
common_fails: agent patches assertion without fixing race