workflow-verify-before-filing

# Verify Before Filing > Operational scaffolding ships beside this file, not inside it: Phases 1–2 as > [`workflows/verify-before-filing.workflow.js`](workflows/verify-before-filing.workflow.js) > (agent prompts, schemas, gate logic), Phase 3 as the executable > [`scripts/file-wave.sh`](scripts/file-wave.sh). The rationale for both — why each > constant is what it is — plus the worked example is in [REFERENCE.md](REFERENCE.md). > This file is the decision layer. A backlog of upstream bug candidates — audit docs, "file this later" notes, workaround commits — is a list of **hypotheses dated to when they were observed**, not a filing queue. Upstream moved since: versions shipped, files restructured, other deployers reported the same thing, and some of your own diagnoses were wrong. Filing the backlog as-is produces duplicate and already-fixed reports — exactly the noise that makes maintainers stop reading your issues. **Verify every claim at upstream HEAD, dedup against the trackers (including your own earlier reports), and only file what survives.** Measured base rate (FVH → SIMPL-Open, 2026-06-11): of 24 accumulated candidates, **only 12 were real-and-current** — 7 claims were invalid on inspection, 3 were already fixed upstream, 1 was obsolete, 1 duplicated our own earlier report's by-catch. Half the backlog would have been noise. ## When to Use This Skill | Use this skill when... | Skip when... | |---|---| | Filing N accumulated candidates from docs/backlogs/git history | You just hit the bug minutes ago against current HEAD | | The observations are days-to-months old | Single trivially-checkable item — verify inline, then file | | Claims came from audit docs nobody re-checked | | | You've filed on this upstream before (self-dup risk) | | ## The Pipeline Phases 1–2 also ship as a bundled harness — see [Workflow harness (template)](#workflow-harness-template) below. ### Phase 0 — Consolidate a candidate manifest One JSON/table entry per candidate: id, the **claim** (precise, falsifiable), target upstream project, version observed, source refs (your commits/PRs that hold real error output), and known-filed prior reports to dedup against. Merge all sources first — audit docs, strategy docs, and git sweeps usually overlap. Shape: ```json { "id": "W2-13", "slug": "notification-smtp-ec-defaults", "claim": "Chart defaults SMTP to dev@simpl-europe.eu via ssl0.ovh.net (vendor dev infra) as live default values; should be placeholder/required.", "targets": ["group/subgroup/notification-service"], "observed_version": "2.1.1 (Apr 2026)", "sources": ["audit-doc item 6"], "evidence_prs": [1826] } ``` Keep prior-filed report URLs (with issue iids) in the same manifest so search agents can fetch their bodies. ### Phase 1 — Verify + dedup (two agents per candidate, parallel) **Verify agent** (read-only against upstream): fetch the implicated files at default-branch HEAD *and* the latest tag; quote the current content; return a verdict from a closed vocabulary: ``` still-present | partially-fixed | fixed-upstream | obsolete-version | claim-invalid | could-not-verify ``` plus `targetProject`, quoted `evidence`, `checkedRefs`, and `notes` (files moved, versions drifted, framing corrections). Hard rule: **agents are read-only upstream** — GET requests only; nothing writes until the filing phase. State that rule verbatim in every agent prompt. Both prompts, with the forge-tooling block and the schemas, are the `VERIFY_PROMPT` / `SEARCH_PROMPT` constants in [`workflows/verify-before-filing.workflow.js`](workflows/verify-before-filing.workflow.js) — adapt those rather than retyping them. **Gate precedence**: any duplicate kills the filing regardless of verdict; `could-not-verify` never files (record a human follow-up task instead). **Search agent**: tracker search (issues + MRs, all states, several phrasings including exact error strings) on the target project and group-wide — **plus fetch the full bodies of your own prior reports** and check overlap including their by-catch findings. Self-duplicates are the embarrassing kind. **Gate**: only `still-present`/`partially-fixed` with no duplicate proceeds. Everything else gets a recorded disposition — that record is a deliverable, not waste (see Phase 4). ### Phase 2 — Draft to a house template Per surviving candidate, one markdown file per issue: ```markdown # <symptom-first title — becomes the issue title> <!-- target: <project path> (stripped by the filing script) --> ## Summary <claim, with evidence as blob links PINNED to the verified refs (https://<forge>/<path>/-/blob/<ref>/<file>#L<n>) — not bare paths, not `main` if HEAD drifts> <real error signature mined from your own incident PRs/logs> ## Suggested fix <EXACTLY ONE recommended fix; alternatives get one trailing sentence; "happy to open the MR" only when trivial> --- Observed while <one-line deployment context>; verified against <refs> on <date>. ``` Never leak internal PR numbers or repo paths into the body — use them only to mine evidence. Then gate every draft through `agent-patterns-plugin:cold-read-gate` (isolated haiku maintainer cold-read; one revise round, re-gate only if the verdict was `needs-revision`). ### Phase 3 — Paced filing Issue-creation endpoints rate-limit aggressively (observed: a GitLab instance returning 429 after a **single** create). Never loop `gh`/`glab issue create` by hand — write the Phase 1+2 result array to a JSON file and run the script, which owns the pacing, the retries, and the URL manifest: ```bash bash "${CLAUDE_SKILL_DIR}/scripts/file-wave.sh" --results /abs/path/results.json --dry-run bash "${CLAUDE_SKILL_DIR}/scripts/file-wave.sh" --results /abs/path/results.json --forge glab --host <instance> ``` Run the real invocation with Bash `run_in_background: true` — at 70 s per create a wave outlives a foreground call. Dry-run first: it resolves every title and target project and creates nothing. The script files only `disposition: "file"` entries, exits 0 on an empty set, appends every URL **and** every failure to `filed-urls.txt`, and continues past a failure rather than aborting the batch. Forge dispatch is deterministic: `--forge gh|glab` > `$FILE_WAVE_FORGE` > a set `GITLAB_HOST` (implies `glab`) > `gh`. `--help` documents every flag; the pacing *rationale* and the invariants its tests pin are in [REFERENCE.md](REFERENCE.md). Cross-link related new issues afterwards (also paced). Created GitLab issues may surface as `/-/work_items/` URLs. ### Phase 4 — Bookkeeping (the dispositions are deliverables) - Annotate the **source docs** the candidates came from: filed URL, fixed-upstream (version), duplicate-of, obsolete, or claim-retracted — the audit trail keeps stale claims from being re-filed next quarter. - **Fixed-upstream discoveries usually imply local action**: a fork you can retire, a pin you can advance, a workaround you can delete. Record each as a follow-up task. - Post the disposition table to your tracking issue; close it if nothing known remains unfiled. ## Workflow harness (template) `workflows/verify-before-filing.workflow.js` ships beside this skill. **It is a TEMPLATE to adapt, not a script to run verbatim.** Read it, then rewrite it for the work in front of you. It covers Phases 1–2 only; Phase 3 is [`scripts/file-wave.sh`](scripts/file-wave.sh), whose input contract is the harness's return value. **Adapt freely:** the agent prompts and their forge-tooling block (the shipped one is GitLab), the wave width, the house draft template, the effort tiers, and the search phrasings. **Preserve across any adaptation:** (a) the loop bound comes from the candidate manifest passed in as `args`, never from a prose "for each" — including the ≤5 read wave, which paces reads the way `file-wave.sh` paces writes; (b) the closed verdict vocabulary and the gate `['still-present','partially-fixed'].includes(verdict) && duplicateFound === 'no'`, **in that precedence** — a duplicate kills the filing regardless of verdict, and `could-not-verify` never files; (c) two barriers — the intra-candidate `parallel([verify, search])`, because the gate reads *both*, and the batch-dedup pass, which compares survivors to **each other** rather than only to the tracker. Also structural: the cold-read agent is **never** the drafter (that independence is the gate), exactly one revise round, and `DRAFT_SCHEMA` carries the issue **body** — a workflow script has no filesystem, so dedup cannot merge on a path. **Agent budget:** 1 + 6 x candidates — up to six agents per candidate (verify, search, draft, cold-read, revise, re-cold-read) plus one batch dedup. The scale guard asks before every run, because the list comes from the caller at runtime. **Skip the harness when:** the manifest holds one or two candidates — that is a linear pass and the harness is pure overhead (the template aborts below three). A 24-candidate run is roughly 100–140 agents. The steps above remain the authoritative description of *what* each stage must produce; the harness only fixes *how* the work is split. Two clauses this template carries. The second is unconditional here — this skill's entire output is a forge mutation: > Never `Workflow({resumeFromRunId})` to retry a few failed worktree agents — a resume re-runs > agents that already succeeded and opens duplicate PRs (#1868). Re-dispatch the failed units > fresh and sequentially after checking > `gh pr list --head <branch> --state all --json number,state`. > Push, PR creation, and GitHub mutations happen **only** in the single sequential finalise > stage, never inside a fanned-out agent. Here that stage is Phase 3 > (`scripts/file-wave.sh`): every agent in the harness is read-only upstream, and the harness > returns data for the script to file. ## Verdict Vocabulary Notes | Verdict | Meaning | Typical doc annotation | |---|---|---| | `still-present` | Reproduced at HEAD + latest tag | filed URL | | `partially-fixed` | Upstream fixed some instances; file the remainder, cite their own fix as the pattern | filed URL (narrowed) | | `fixed-upstream` | Shipped in a release — note which | version + local follow-up | | `obsolete-version` | The affected line is superseded/retired | superseded note | | `claim-invalid` | The original diagnosis was wrong | retraction + what was actually true | | `could-not-verify` | Evidence unreachable | human follow-up task | `claim-invalid` is not failure — it's the workflow catching your own docs drifting from reality. Correct the doc in the same pass. ## Common Mistakes | Mistake | Correct approach | |---|---| | Filing the backlog as written ("the audit already verified it") | The audit verified it *then*; verify at HEAD *now* | | Dedup against the tracker but not your own issues | Your earlier reports' by-catch findings are duplicates too | | Quoting your old observed version in the issue | Quote HEAD/latest-tag content; cite the refs you checked | | Bulk-creating issues in a hand-written loop | 429 after the first create; run `scripts/file-wave.sh` (pacing + backoff + manifest) | | Discarding gated-out candidates silently | Dispositions update docs, retire forks, close tracking issues | | Letting verify agents have write access upstream | Read-only until the dedicated, paced filing step | ## Related - `agent-patterns-plugin:cold-read-gate` — the pre-publish legibility gate (Phase 2) - `agent-patterns-plugin:verify-before-plan` — same epistemics one level up: premises decay; check before acting on them - [`workflow-preflight`](../workflow-preflight/SKILL.md) — remote-state verification before implementation work, the in-repo sibling - User rule `verify-upstream-before-patching` (where present) — the single-item inline form of Phase 1