| name | phase-verify |
| description | Phase agent for the verify step of the 9-phase orchestrator pipeline (CTL-450).
NEW skill — has no canonical wrapper. Runs read-only adversarial verification
against the implement-phase diff: tsc, tests, lint, security scan, reward-hacking
scan, code review, test coverage, silent-failure hunt. Writes
${ORCH_DIR}/workers/<TICKET>/verify.json then emits phase.verify.complete.<ticket>.
Reads phase-implement.json as its prior-phase artifact. NEVER writes application
code — only test files allowed. Spawned via phase-agent-dispatch via slash
command — hence `user-invocable: true`.
|
| user-invocable | true |
| disable-model-invocation | false |
| allowed-tools | ["Read","Write","Grep","Glob","Task","Bash"] |
phase-verify
You are the verify phase agent. You run inside claude --bg and own a single
responsibility: read-only adversarial verification of the implement phase's diff,
producing ${ORCH_DIR}/workers/<TICKET>/verify.json with regression_risk,
findings, and tests_attempted fields. You then emit
phase.verify.complete.<ticket> and exit. Built on the [[_phase-agent-template]]
contract.
CRITICAL CONSTRAINT: NEVER write application code
You are a read-only verifier. The only files you may create or edit are:
- Test files (under
**/__tests__/, *.test.*, *.spec.*, test/**, tests/**)
- The
verify.json artifact in the worker directory
- Signal files via the standard emitter helper
Editing application code from this phase is a contract violation. If verification
surfaces a bug that requires code changes, you record the finding and let
[[phase-review]] (which IS allowed to write remediation commits) act on it.
Prelude
set -uo pipefail
: "${CATALYST_ORCHESTRATOR_DIR:?required (set by phase-agent-dispatch)}"
: "${CATALYST_ORCHESTRATOR_ID:?required}"
: "${CATALYST_PHASE:?required}"
: "${CATALYST_TICKET:?required}"
ORCH_DIR="$CATALYST_ORCHESTRATOR_DIR"
ORCH_ID="$CATALYST_ORCHESTRATOR_ID"
PHASE="$CATALYST_PHASE"
TICKET="$CATALYST_TICKET"
CHANNEL="orch-${ORCH_ID}"
SIGNAL_FILE="${ORCH_DIR}/workers/${TICKET}/phase-${PHASE}.json"
[[ -f "$SIGNAL_FILE" ]] || { echo "phase-${PHASE}: signal file missing" >&2; exit 1; }
PLUGIN_ROOT="${CLAUDE_PLUGIN_ROOT:-$(cd "$(dirname "$0")/../../.." && pwd)}"
COMMS="${PLUGIN_ROOT}/scripts/catalyst-comms"
[[ -x "$COMMS" ]] || COMMS="$(command -v catalyst-comms 2>/dev/null || true)"
if [[ -n "$COMMS" ]]; then
"$COMMS" join "$CHANNEL" --as "$TICKET" \
--capabilities "phase-${PHASE}: ${TICKET}" \
--orch "$ORCH_ID" --parent orchestrator --ttl 3600 >/dev/null 2>&1 || true
"$COMMS" send "$CHANNEL" "phase-verify started" --as "$TICKET" --type info \
--orch "$ORCH_ID" >/dev/null 2>&1 || true
fi
SESSION_SCRIPT="${PLUGIN_ROOT}/scripts/catalyst-session.sh"
if [[ -x "$SESSION_SCRIPT" ]]; then
CATALYST_SESSION_ID=$("$SESSION_SCRIPT" start \
--skill "phase-${PHASE}" \
--ticket "$TICKET" \
--workflow "${CATALYST_SESSION_ID:-}")
export CATALYST_SESSION_ID
fi
TS=$(date -u +%Y-%m-%dT%H:%M:%SZ)
TMP="${SIGNAL_FILE}.tmp.$$"
jq --arg ts "$TS" --arg sid "${CATALYST_SESSION_ID:-}" '
.status = "running"
| .updatedAt = $ts
| if $sid != "" then .catalystSessionId = $sid else . end
' "$SIGNAL_FILE" > "$TMP" \
&& mv "$TMP" "$SIGNAL_FILE"
IMPLEMENT_SIGNAL="${ORCH_DIR}/workers/${TICKET}/phase-implement.json"
if [[ ! -f "$IMPLEMENT_SIGNAL" ]]; then
echo "phase-verify: prior phase-implement.json missing" >&2
"${PLUGIN_ROOT}/scripts/phase-agent-emit-complete" \
--phase "$PHASE" --ticket "$TICKET" --status failed \
--reason "prior_artifact_missing:phase-implement.json"
exit 1
fi
/goal
/goal "I have written ${ORCH_DIR}/workers/${TICKET}/verify.json with the schema
{regression_risk:int, findings:[...], tests_attempted:int, gates:{...}} AND
I have NOT modified any application source files (only test files). I have
printed the path on stdout."
Work block
Run the same adversarial verification suite the current oneshot Phase 4 runs,
but record findings instead of attempting fixes.
1. Determine the base branch and diff
BASE_BRANCH=$(git remote show origin 2>/dev/null \
| grep "HEAD branch" | awk '{print $NF}')
BASE_BRANCH="${BASE_BRANCH:-main}"
DIFF_RANGE="origin/${BASE_BRANCH}...HEAD"
CTL-608: empty-branch backstop
Defense-in-depth for the live phase-implement empty-branch gate. That gate runs
in the phase-implement End block, but the execution-core reclaim-dead-work
path emits implement-complete on a worker's behalf without running that End
block — so an empty branch (0 commits ahead) can still reach verify. Counting
commits-ahead here means an empty branch emits phase.verify.failed instead of
running the full gate suite and advancing an empty branch to phase-pr. Reuses
BASE_BRANCH from step 1; uses only POSIX/zsh-safe git rev-list --count.
Fail-open (warn + continue) when the base is unresolvable, matching
phase-implement and the mirror block's _base branch unknown_ tolerance.
Uniquely-named fence so the e2e harness can extract+exercise it.
VERIFY_GATE_BASE=""
if git rev-parse --verify --quiet "origin/${BASE_BRANCH}" >/dev/null 2>&1; then
VERIFY_GATE_BASE="origin/${BASE_BRANCH}"
elif git rev-parse --verify --quiet "${BASE_BRANCH}" >/dev/null 2>&1; then
VERIFY_GATE_BASE="${BASE_BRANCH}"
fi
if [[ -n "${VERIFY_GATE_BASE}" ]]; then
VERIFY_AHEAD="$(git rev-list --count "${VERIFY_GATE_BASE}..HEAD" 2>/dev/null || echo 0)"
if [[ "${VERIFY_AHEAD:-0}" -le 0 ]]; then
echo "phase-verify: 0 commits ahead of ${VERIFY_GATE_BASE}; empty branch, nothing to verify (CTL-608)" >&2
"${PLUGIN_ROOT}/scripts/phase-agent-emit-complete" \
--phase "$PHASE" --ticket "$TICKET" --status failed \
--reason "empty_branch:0_commits_ahead_of_${VERIFY_GATE_BASE}"
[[ -n "$COMMS" && -x "$COMMS" ]] && "$COMMS" send "$CHANNEL" \
"phase-verify failed: empty branch (0 commits ahead of ${VERIFY_GATE_BASE})" \
--as "$TICKET" --type attention --orch "$ORCH_ID" >/dev/null 2>&1 || true
exit 1
fi
else
echo "phase-verify: could not resolve integration base (no origin/${BASE_BRANCH} or ${BASE_BRANCH}); skipping empty-branch gate (CTL-608)" >&2
fi
2. Run read-only gates
Run each gate; record pass/fail/skip into the in-memory results map. Do not stop
on first failure — verification is exhaustive.
| Gate | Tool | Skill / agent |
|---|
| Type check | tsc --noEmit (or project's typecheckCommand) | [[validate-type-safety]] |
| Reward-hacking scan | grep-based pattern check | [[scan-reward-hacking]] |
| Unit tests | project test command | [[validate-type-safety]] |
| Lint | project lint command | [[validate-type-safety]] |
| Security review | dependency + secret scan | /security-review (built-in) |
| Code review | style/guideline adherence | [[pr-review-toolkit:code-reviewer]] agent |
| Test coverage | per-file coverage on diff | [[pr-review-toolkit:pr-test-analyzer]] agent |
| Silent failures | unchecked try/catch + fallback hunting | [[pr-review-toolkit:silent-failure-hunter]] agent |
For each gate, run via Bash for the CLI ones and the Task tool for the agent
ones. Capture exit code + a one-line summary per gate.
3. Compute regression_risk (0–10)
Aggregate signal:
| Signal | Risk delta |
|---|
| Any required CLI gate failed (tsc/test/lint/security) | +3 each |
scan-reward-hacking flagged a HIGH-severity pattern | +3 |
code-reviewer flagged a structural issue | +2 |
pr-test-analyzer reports < 50% diff coverage | +2 |
silent-failure-hunter flagged unchecked catch / fallback | +2 |
Any agent surfaced a must-fix finding | +3 |
Clamp to [0, 10]. A regression_risk ≥ 5 means [[phase-review]] should create
remediation commits before the PR opens.
4. Optionally write test-only files
If pr-test-analyzer identifies an uncovered code path that has obvious tests, you
MAY add tests under **/__tests__/ or **/*.test.*. Track each file added in the
tests_attempted count. Do not edit application code under any circumstance;
silent-failure-hunter's findings go into findings, never into a fix.
5. Write the artifact
ARTIFACT="${ORCH_DIR}/workers/${TICKET}/verify.json"
jq -nc \
--argjson risk "$REGRESSION_RISK" \
--argjson findings "$FINDINGS_JSON" \
--argjson tests "$TESTS_ATTEMPTED" \
--argjson gates "$GATES_JSON" \
--arg ts "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
'{regression_risk: $risk, findings: $findings, tests_attempted: $tests,
gates: $gates, generatedAt: $ts}' > "${ARTIFACT}.tmp" \
&& mv "${ARTIFACT}.tmp" "$ARTIFACT"
Findings array shape — each entry:
{
"severity": "high|medium|low",
"kind": "type|test|lint|security|review|coverage|silent-failure|reward-hacking",
"file": "path/to/file.ts",
"line": 42,
"message": "Short human-readable description",
"recommendation": "What phase-review should do about this"
}
Gates object shape — keyed by gate name:
{
"typecheck": { "status": "pass|fail|skip", "exitCode": 0, "summary": "..." },
"tests": { "status": "pass", "exitCode": 0, "summary": "..." }
}
Inbox check (CTL-749)
After all gates have run, check for mid-flight context updates from the human:
- If
${ORCH_DIR}/workers/${TICKET}/inbox.jsonl exists and is non-empty, read it fully.
- Parse each JSONL line — entries have
kind: "comment" or kind: "description_changed".
- For each entry, decide:
- Absorb and continue: the update is additive context (clarification, extra constraints,
"also handle X") — fold it into your working context and continue. Post a brief reply comment
acknowledging the update (one sentence).
- Pause and replan: the update fundamentally changes scope or invalidates the current
approach — emit
failed with reason: "mid_flight_replan_needed" via
${PLUGIN_ROOT}/scripts/phase-agent-emit-complete and post the reason to Linear as a
comment before exiting.
- After reading, archive processed entries:
[[ -f "${ORCH_DIR}/workers/${TICKET}/inbox.jsonl" ]] && \
mv "${ORCH_DIR}/workers/${TICKET}/inbox.jsonl" \
"${ORCH_DIR}/workers/${TICKET}/inbox.processed-$(date +%s).jsonl" || true
- If no inbox file or it is empty, continue normally.
End block
TS=$(date -u +%Y-%m-%dT%H:%M:%SZ)
TMP="${SIGNAL_FILE}.tmp.$$"
jq --arg ts "$TS" --arg artifact "$ARTIFACT" \
'.updatedAt = $ts | .artifact = $artifact' \
"$SIGNAL_FILE" > "$TMP" && mv "$TMP" "$SIGNAL_FILE"
Mirror the phase output to Linear as a single comment (CTL-632). Renders
regression risk, per-gate pass/fail/skip, findings-by-severity, and the
full findings JSON inside a <details> block. Body is hard-truncated to
30,000 bytes (well under Linear's effective comment cap) with a marker.
Fail-open and idempotent via the per-phase marker file.
LINEAR_MIRROR_MARKER="${ORCH_DIR}/workers/${TICKET}/.linear-mirror-${PHASE}"
if [[ ! -e "${LINEAR_MIRROR_MARKER}" ]]; then
GATES_RENDERED="$(printf '%s' "${GATES_JSON}" | jq -r '
to_entries
| map("- **" + .key + "**: " + (.value.status // "unknown")
+ (if .value.summary then " — " + .value.summary else "" end))
| join("\n")' 2>/dev/null)"
FINDINGS_COUNT="$(printf '%s' "${FINDINGS_JSON}" | jq -r 'length' 2>/dev/null || echo 0)"
FINDINGS_BY_SEVERITY="$(printf '%s' "${FINDINGS_JSON}" | jq -r '
group_by(.severity)
| map("- " + (.[0].severity // "unknown") + ": " + (length|tostring))
| join("\n")' 2>/dev/null)"
FINDINGS_PRETTY="$(printf '%s' "${FINDINGS_JSON}" | jq -r '.' 2>/dev/null)"
MIRROR_BODY="$(cat <<EOF
**Phase Verify**
- **Regression risk**: ${REGRESSION_RISK} / 10
- **Tests attempted**: ${TESTS_ATTEMPTED}
- **Findings**: ${FINDINGS_COUNT}
**Gates**:
${GATES_RENDERED}
**Findings by severity**:
${FINDINGS_BY_SEVERITY:-_none_}
<details>
<summary>Full findings JSON</summary>
\`\`\`json
${FINDINGS_PRETTY}
\`\`\`
</details>
_Posted automatically by phase-verify (CTL-632)._
EOF
)"
MIRROR_FOOTER=""
if [[ -n "${PLUGIN_ROOT:-}" && -x "${PLUGIN_ROOT}/scripts/lib/phase-mirror-footer.sh" ]]; then
MIRROR_FOOTER="$("${PLUGIN_ROOT}/scripts/lib/phase-mirror-footer.sh" --orch-dir "${ORCH_DIR}" --ticket "${TICKET}" --phase "${PHASE}" 2>/dev/null || true)"
fi
[[ -n "${MIRROR_FOOTER}" ]] && MIRROR_BODY="${MIRROR_BODY}
${MIRROR_FOOTER}"
if [[ ${#MIRROR_BODY} -gt 30000 ]]; then
MIRROR_BODY="${MIRROR_BODY:0:30000}
_... (truncated)_"
fi
COMMENT_POST="${CATALYST_COMMENT_POST_HELPER:-${PLUGIN_ROOT}/scripts/lib/linear-comment-post.sh}"
if [[ ! -x "$COMMENT_POST" ]]; then COMMENT_POST="$(command -v linear-comment-post.sh 2>/dev/null || true)"; fi
if [[ -n "$COMMENT_POST" && -x "$COMMENT_POST" ]] && "$COMMENT_POST" "${TICKET}" "${MIRROR_BODY}" >/dev/null; then
: > "${LINEAR_MIRROR_MARKER}"
else
echo "phase-verify: linear-comment-post failed (continuing)" >&2
fi
fi
Capture friction (compound loop, CTL-789)
Before emitting completion, record this phase's friction to the shared per-ticket
friction log. This is the producer half of the compound-engineering loop:
[[ticket-compound]] later harvests thoughts/shared/friction/${TICKET}.md to
distill durable learnings. ${TICKET} is already resolved in the Prelude — do
not re-derive it.
Replace each <…> placeholder below with your real experience verifying this
ticket — 3–6 lines total, terse. "None." is a valid value for any bullet when
the phase was frictionless. The record header
(## <phase> · <TICKET> · <ISO-8601 timestamp>) is a cross-phase contract shared
by all five phase skills — keep it byte-identical; only the phase label differs.
This append is best-effort and off the critical path: it must NEVER fail the
phase or block the emit-complete below.
FRICTION_LOG="thoughts/shared/friction/${TICKET}.md"
mkdir -p "$(dirname "$FRICTION_LOG")"
[ -f "$FRICTION_LOG" ] || printf '# Friction log — %s\n' "${TICKET}" > "$FRICTION_LOG"
cat >> "$FRICTION_LOG" <<EOF
## verify · ${TICKET} · $(date +%Y-%m-%dT%H:%M:%S%z)
- **Backtracks / redone work:** <where you backtracked or redid work this phase — or "None.">
- **Missing / wrong / hard-to-find context:** <context that was absent, stale, or hard to locate — or "None.">
- **If I'd known:** <the ADR / guidance / past learning that would have saved this — the compounding signal — or "None.">
EOF
"${PLUGIN_ROOT}/scripts/phase-agent-emit-complete" \
--phase "$PHASE" --ticket "$TICKET" --status complete
if [[ -n "${ORCH_DIR:-}" && -f "${ORCH_DIR}/workers/${TICKET}/phase-${PHASE}.json" ]]; then
_SELF_BG=$(jq -r '.bg_job_id // empty' \
"${ORCH_DIR}/workers/${TICKET}/phase-${PHASE}.json" 2>/dev/null || true)
[[ -n "$_SELF_BG" ]] && claude stop "${_SELF_BG:0:8}" >/dev/null 2>&1 || true
fi
[[ -n "$COMMS" ]] && "$COMMS" done "$CHANNEL" --as "$TICKET" >/dev/null 2>&1 || true
Failure handling
A failure here means verification itself broke (e.g., a gate process crashed),
not that a gate failed — gate failures are recorded into findings and the phase
still emits complete.
"${PLUGIN_ROOT}/scripts/phase-agent-emit-complete" \
--phase "$PHASE" --ticket "$TICKET" --status failed \
--reason "<short reason>"
[[ -n "$COMMS" ]] && "$COMMS" send "$CHANNEL" \
"phase-verify failed: <reason>" --as "$TICKET" --type attention \
--orch "$ORCH_ID" >/dev/null 2>&1 || true
exit 1
Why this is a separate skill from validate-plan
[[validate-plan]] checks that a plan was executed against a known plan document.
phase-verify is adversarial — it doesn't read the plan; it reads the diff and
hunts for regressions. The orchestrator's pipeline may run both (validate-plan
inside implement, then verify on the resulting branch).
