verify

name	verify
description	Verify project behavior anchors — compilation, tests, and interaction checks after autonomous runs. NOT the Codex built-in /verify (which runs the app to observe a single change working) — this one walks the AGENTS.md "Features (Behavior Anchors)" list.

Clade for Codex

This workflow runs directly in Codex. Do not launch the claude CLI or delegate the workflow to Clade's MCP bridge.

Codex compatibility rules:

• Plugin skills are namespaced. Invoke this workflow explicitly as $clade:verify; a bare $name does not select the installed Clade plugin.
• Read the nearest AGENTS.md files for repository instructions. If a project has only CLAUDE.md, treat it as legacy project guidance and read it too.
• Store new Clade working state under .clade/ (or ~/.clade/ for personal state). Existing legacy Claude state may be read for migration, but do not create new vendor-specific state.
• A /skill-name reference means the corresponding Codex $clade:skill-name plugin skill, or the same workflow invoked naturally when explicit skill invocation is not available.
• Use Codex web, file, shell, image, and subagent capabilities when the source workflow names a vendor-specific tool. If a capability is unavailable, use the documented fallback instead of spawning another agent CLI.
• Paths such as <plugin-root>/... are relative to the installed Clade plugin containing this SKILL.md; resolve that root before invoking a helper.

Canonical Clade workflow

You are the Verify skill. You check that a project's key behaviors still work after code changes.

---

Step 1: Detect project type

Read AGENTS.md and look for ## Project Type section. Extract:

• Type: web-fullstack, api-only, cli, ml-pipeline, library, skill-system, toolkit
• Test command: if provided, this is the primary verification method
• Verify command: if provided, run this as a smoke test

If ## Project Type is missing, auto-detect by scanning the repo:

• package.json with next/react → web-fullstack
• requirements.txt with fastapi/flask → api-only
• setup.py/pyproject.toml with CLI entrypoints → cli
• Cargo.toml / go.mod → check for main package
• Fallback: unknown

Step 2: Read behavior anchors

Read AGENTS.md and look for ## Features (Behavior Anchors) section. Each line is a feature to verify:

- [feature-name]: [what should happen when user does X]

If no anchors found, note this gap and proceed with what's available.

Step 3: Run verification strategy

Execute checks based on project type. Run all applicable strategies:

Strategy: Test suite (all project types)

If a test command is specified in ## Project Type:

timeout 120 {test_command}

Record: pass/fail + count of passing/failing tests.

Context hygiene: if ~/.clade/scripts/quiet-run.sh is installed, wrap the command:

timeout 130 bash ~/.clade/scripts/quiet-run.sh {test_command}

Full output lands in .clade/logs/quiet-*.log, each line timestamped [HH:MM:SS]; only the verdict line + failure tail enters the transcript. The exit code is mirrored, so pass/fail detection is unchanged. Remember the printed full log: path — if the UI Interaction strategy runs later in this same /verify pass, its browser console output gets appended to this SAME file (see below), so a test failure and a JS console error land in one chronologically-ordered artifact.

If no test command but common test patterns exist:

• timeout 120 pytest / timeout 120 python -m pytest (Python)
• timeout 120 npm test / timeout 120 npx jest (Node)
• timeout 120 cargo test (Rust)
• timeout 120 go test ./... (Go)

Try the likely command. If it works, report results. If not, skip. If timeout fires: mark as ⚠ (test suite timed out — may have hanging test), do NOT retry.

Strategy: Compile/type check

• Python: python -m py_compile {main_files} or mypy if configured
• TypeScript: npx tsc --noEmit
• Rust: cargo check
• Go: go build ./...

Strategy: Verify command (smoke test)

If a verify command is specified in ## Project Type, run it.

Strategy: Behavior anchor check

For each anchor in ## Features:

1. Determine if the anchor is testable with available tools
2. If testable: run a quick check (e.g., CLI anchor → run the command with --help or sample input; API anchor → check the route exists; script → check it's executable and runs without error)
3. If NOT testable (requires browser, external service, credentials): mark as "unverifiable"

Anchor test examples:

• install.sh: copies files to ~/.clade/ → bash install.sh --dry-run or check the script is syntactically valid: bash -n install.sh
• slt: cycles statusline mode → bash slt --help or verify the script exists and is executable
• /commit: analyzes changes → verify the skill prompt file exists: test -f ~/.clade/skills/commit/prompt.md
• CLI tool → {tool} --help should exit 0
• API endpoint → check route is defined in source code (grep)

Strategy: UI Interaction (frontend only)

Conditions — ALL must be true:

1. Project type is web-fullstack (from AGENTS.md ## Project Type)
2. Playwright MCP tools are available — check your tool list for mcp__playwright__browser_navigate (Playwright MCP tools carry the mcp__playwright__ prefix). If absent, the browser MCP is not wired in — skip.

If conditions are not met, set INTERACTION_RESULT: skipped and move on.

Enable browser verification once per project with configs/scripts/setup-browser-verify.sh <project_dir> — it adds the Playwright MCP to .clade/mcp.json (which both worker spawns and /verify already load) and installs the Chromium binary. Below, browser_navigate/browser_snapshot/etc. are shorthand for the mcp__playwright__-prefixed tools.

Flow:

1. Run configs/scripts/ensure-dev-server.sh (no args needed — it reads AGENTS.md's Frontend: ... port NNNN line itself, defaulting to 3000). It is idempotent and flock-guarded (Thorsten Ball: shared discovery state, .clade/dev-server.json) — safe to call even when a concurrent worktree worker is also verifying, since only one of you will actually start it; the rest reuse the same server. Read its one-line output: PORT=<port> STATUS=reused|started|unreachable [PID=<pid>].

2. If STATUS=unreachable (exit code 1) → set INTERACTION_RESULT: partial, write "App unreachable at localhost:{port}" to .clade/playwright-issues.md, and move on. Do NOT block the verify. Do NOT retry startup yourself — the script already tried for 30s.

3. Otherwise (reused or started), connect to http://localhost:{port} via browser_navigate — the server is confirmed reachable at this point.

4. Take a browser_snapshot of the home page to get the accessibility tree.

5. Walk up to 5 pages (home + up to 4 linked pages):

   • For each page: browser_snapshot → identify interactive elements (buttons, forms, links, inputs)
   • Click/fill key interactive elements → check for errors, broken states, console errors
   • If a page requires authentication and no test credentials are available in AGENTS.md, mark as unverifiable — do NOT report login failure as a [BUG]
   • Take another snapshot after interactions to verify state changes

6. Call browser_console_messages once (after the page walk, not per-page — this is a summary read, not a live stream). If it returns any error/warning entries AND you remembered a quiet-run log path from the Test-suite strategy above, append them to that SAME file (Thorsten Ball: merged, time-correlatable log) — one line per message, timestamped and tagged so it sorts naturally alongside the test-run output.

   Never substitute message text directly into a shell command — a console message can originate from the page under test (including a malicious PR's own code), so it may contain $(...), backticks, or other shell metacharacters that would execute if pasted into a quoted command string. Instead, for each message: write it verbatim to a scratch file with the file-editing tools (not shell-interpreted, so no escaping is needed), then append using only paths/ fixed text in the shell command, never the message content itself:

   printf '[%s] [browser] ' "$(date +%H:%M:%S)" >> <remembered log path>
   cat <scratch file path> >> <remembered log path>
   printf '\n' >> <remembered log path>
   

   If no quiet-run log path exists (no test command / quiet-run not used), skip the append — the console findings still feed into .clade/playwright-issues.md below.

7. Evaluate:

   • Does navigation work? Are pages rendering content (not blank/error)?
   • Do interactive elements respond? Are forms submittable?
   • Any JS errors visible in the page? Any "undefined"/"null"/"NaN" rendering?
   • Is the UX intuitive? (layout makes sense, text is readable, actions are discoverable)

8. Write findings to .clade/playwright-issues.md (overwrite, do not append):

   • [BUG] tag for broken functionality (crashes, errors, broken flows, missing data)
   • [UX] tag for usability issues (confusing layout, missing feedback, accessibility gaps)
   • Include which page/element was affected

9. Set result:

   • INTERACTION_RESULT: pass — all flows work, no bugs found
   • INTERACTION_RESULT: partial — some flows unverifiable (app didn't start, pages unreachable)
   • INTERACTION_RESULT: fail — broken UI or unexpected errors found ([BUG] items exist)

Bounds: Max 2 minutes of interaction time. Max 5 pages. If time runs out, report what you found so far.

Strategy: Lint/format (optional, lightweight)

Only if project has linting configured (.eslintrc, ruff.toml, etc.):

{lint_command}

Report warnings but don't count as failures.

Step 4: Check VERIFY.md coverage (if present)

If VERIFY.md exists in the project root, read it and report coverage status.

This is a read-only step — do NOT fix anything here. /verify reports; /review fixes.

1. Count checkpoints by status: ✅ / ❌ / ⚠ / ⬜
2. Identify any ❌ checkpoints — these are confirmed regressions
3. Identify ⬜ checkpoints — these are coverage gaps (untested)

Impact on VERIFY_RESULT:

• Any ❌ checkpoint in VERIFY.md → VERIFY_RESULT = fail (confirmed regression)
• Only ⬜ checkpoints (no ❌) → VERIFY_RESULT = partial at most (gaps, not regressions)
• All ✅ or ⚠ → VERIFY.md does not degrade VERIFY_RESULT

If VERIFY.md does not exist: skip this step silently. Output VERIFY_COVERAGE: none in footer.

---

Step 5: Produce report

Write a human-readable summary, then the machine-parseable footer.

Summary format:

## Verify Report — {project_name}

### Test Suite
{pass/fail/skip details}

### Compile Check
{pass/fail details}

### Behavior Anchors
- [anchor-name]: PASS / FAIL (reason) / UNVERIFIABLE (reason)
- ...

### UI Interaction (frontend only)
{pass/partial/fail/skipped + details if applicable}

### VERIFY.md Coverage
{N ✅  N ❌  N ⚠  N ⬜ — or "not present"}
{list any ❌ checkpoint IDs and descriptions}

### Notes
{any observations, warnings, suggestions}

Structured issue checklist (.clade/verify-issues.md)

After producing the summary above, if ANY issues were found (failed anchors, test failures, compile errors, UI bugs, lint warnings), ALSO write a structured checklist to .clade/verify-issues.md.

Rules:

• Overwrite each run (not append) — old issues are stale
• Only create this file when there ARE issues. If everything passes, do NOT create it.
• One - [ ] checkbox per issue, one line each
• Use sections below — omit sections with no issues

Format:

## Failed Anchors
- [ ] anchor-name: brief description of failure

## Test Failures
- [ ] test_module::test_name: assertion error / brief reason

## Compile Errors
- [ ] file:line: error description

## UI Issues
- [ ] [BUG] page/element: what's broken
- [ ] [UX] page/element: usability concern

## Lint Warnings
- [ ] file:line: warning code + message

Copying from playwright-issues.md: If .clade/playwright-issues.md exists and has [BUG] or [UX] items, copy them into the UI Issues section above.

Annotation convention (for human reviewers): Users can annotate each checkbox to control what happens next:

• [fix] → auto-creates a fix task on next run
• [skip] → moved to .clade/skipped.md (won't be raised again)
• [wontfix] → moved to .clade/skipped.md with wontfix reason

Example: - [ ] [fix] slt: cycles to wrong mode after "off"

Unannotated items remain in the file for next review.

Footer (MUST be the last 5 lines — start.sh greps these):

VERIFY_RESULT: pass|partial|fail
FAILED_ANCHORS: anchor-name-1, anchor-name-2
UNVERIFIABLE: N
INTERACTION_RESULT: pass|partial|fail|skipped
VERIFY_COVERAGE: N_pass/N_total|none

Decision rules for VERIFY_RESULT:

• pass: all testable anchors pass, test suite passes (or no test suite), compile succeeds, no ❌ in VERIFY.md
• partial: some anchors are unverifiable (no test strategy, missing tools, insufficient coverage) BUT no testable anchor is regressing. Also used when: no test suite exists, no anchors defined, verify command not provided, or VERIFY.md has ⬜ gaps but no ❌ failures.
• fail: at least one testable anchor is now broken/regressing, OR test suite has new failures, OR compile errors introduced, OR VERIFY.md has ❌ checkpoints

FAILED_ANCHORS: comma-separated list of anchor names that FAIL (not unverifiable — only actual regressions). Use none if no failures. NEVER leave blank — blank line breaks grep in start.sh.

UNVERIFIABLE: count of anchors that could not be tested (integer). 0 if all anchors were testable.

VERIFY_COVERAGE: N_pass/N_total where N_pass = ✅ count, N_total = all checkpoints in VERIFY.md. Use none if VERIFY.md does not exist.

INTERACTION_RESULT: UI interaction test outcome.

• pass — all flows work, no bugs found
• partial — some flows unverifiable (app didn't start, page unreachable)
• fail — broken UI or unexpected errors found (.clade/playwright-issues.md has details)
• skipped — not a frontend project or no Playwright MCP available

Rules

• Run verification commands with the configured Codex permission policy context (the caller handles this)
• Never modify project code — this is a read-only verification skill
• Wrap ALL subprocess calls with timeout N: test suites timeout 120, curl/DB queries timeout 30, compile checks timeout 60. If timeout fires → mark ⚠, do NOT retry
• Fail-open on infrastructure errors (can't install deps, missing tools): mark as unverifiable, not fail
• When in doubt between partial and fail: if you CAN test it and it broke → fail. If you CAN'T test it → partial.

---

Completion Status

• ✅ DONE — task completed successfully
• ⚠ DONE_WITH_CONCERNS — completed but with caveats to note
• ❌ BLOCKED — cannot proceed; write details to .clade/blockers.md
• ❓ NEEDS_CONTEXT — missing information; use AskUserQuestion

3-strike rule: If the same approach fails 3 times, switch to BLOCKED — do not retry indefinitely.

Additional skill reference

Verify Skill

Post-iteration verification used by start.sh to check that behavior anchors still pass after autonomous work. Not user-invocable — called internally by the autonomous loop.

Delivery completion

If this workflow changes files or external state:

• Inspect the real final state before responding, including git status for a repository task.
• Never report DONE while task-owned changes are uncommitted. Use or continue $clade:delivery and create a repository-compliant checkpoint or preserve the work when committing is unavailable.
• When the user request or trusted repository policy makes publication, deployment, or live verification part of the task, do not silently downgrade the result to local-only work.
• If a required delivery transition lacks authority, credentials, a destination, or reachable external state, report BLOCKED or NEEDS_CONTEXT rather than appending a "not committed/pushed/deployed" caveat after DONE.