fs-failure-archaeology

# fs-failure-archaeology — the defect ledger Announce at start: "Consulting the failure archaeology ledger — checking whether this is a known, settled battle." This skill is the **single home** for the founder-skills repo's defect history. Before you debug anything in this repo, scan the at-a-glance table below. If your symptom matches an entry, read that entry and **stop re-investigating** — apply the recorded fix or respect the recorded accepted-risk. All relative paths below are from the repo root. **Terms used once, defined once:** | Term | Meaning | |---|---| | **Ledger entry** | One recorded defect: symptom, root cause, evidence, status. Numbered, never deleted. | | **Plugin cache** | `~/.claude/plugins/cache/founder-skills/` — the copy Claude Code actually loads after `/plugin install`. Mutable at runtime; NOT the repo. Environment-specific path. | | **PEP-668** | Python's "externally managed environment" marker: system `pip install` is blocked; you must use a venv. | | **Golden four** | The four house-style-conforming skills: market-validation, build-options, ecosystem-planning, fable-org-audit. See `fs-skill-style-guide`. | | **Status vocabulary** | `fixed` (fix committed), `fixed-uncommitted` (fix in working tree only), `open` (unfixed), `accepted-risk` (operator chose to live with it). | ## When NOT to use this skill - You want the **process** for fixing a defect (spec -> plan -> commits) → load `fs-change-control`. This skill records history; it never authorizes changes. - You need the **working test/venv commands** as a runbook → load `fs-toolchain-and-tests` (entries 2–3 below only explain *why* the README forms fail). - You're shipping a fix to the live master → load `fs-release-and-publish` for the pre-push gate. - You're writing or restyling a skill (entry 5's rewrite) → load `fs-skill-style-guide` + `fs-skill-authoring`. - You're editing the Pages site (entry 6) → load `fs-site-and-positioning`. - You want the PlatAtlas/market-map surface behind entry 8 → load `fs-platatlas-integration`. - You're doing the periodic drift sweep → load `fs-freshness-watch` (it re-runs this file's provenance commands, among others). --- ## The ledger at a glance (verified 2026-07-02) | # | Name | One-line symptom | Status (2026-07-02) | |---|---|---|---| | 1 | gitignore swallow | build-options crashes: `FileNotFoundError: matrix.template.html` | **fixed — committed 648d304, pushed 2026-07-02** | | 2 | PEP-668 README | `python3 -m pytest` → "No module named pytest" | open | | 3 | node directory form | `node --test tests/js/` → MODULE_NOT_FOUND; JS suite absent from README | open | | 4 | syntax-checked-only workflows | Two Workflow scripts never actually executed end-to-end | **proving runs executed 2026-07-02** (see campaign Phase 1; entry 9 fallout) | | 5 | prd/tasks import mismatch | Two skills in a foreign house style with an unstated CLI dependency | **fixed-uncommitted 2026-07-02** (rewritten to house style; agent-browser → "browser executor", 1 mention) | | 6 | site oversell drift | Pages site claims exceed what the skills do | **fixed-uncommitted 2026-07-02** (chips honest, card copy corrected; site-checks.sh green) | | 7 | add-skill.sh hazards | Validator can't catch missing assets; script `rm -rf`s a user dir | accepted-risk | | 8 | emit_atlas rename | 2026-06-11 rename could have broken external users; no telemetry | accepted-risk | | 9 | skeptic verdict drop | build-options workflow: stress-test verdicts silently defaulted to `survive` | fixed-uncommitted 2026-07-02 (found by the Phase-1 proving run) | --- ## Entry 1 — gitignore swallow: build-options shipped broken - **Symptom:** `/build-options` (and its pytest suite) failed with `FileNotFoundError` for `assets/matrix.template.html`. 2 of 9 tests failed. The plugin shipped this way from day one. - **Root cause:** `plugins/build-options/.gitignore` line 5 is `*.html` (added to ignore *generated* matrix HTML). It also swallowed the hand-written template, which was therefore **never committed**. `claude plugin validate` passed anyway — it checks manifest schema only, not asset presence (see entry 7). - **How it survived at all:** the sole copy lived in the mutable plugin cache (`~/.claude/plugins/cache/founder-skills/build-options/0.1.0/assets/matrix.template.html`, mtime 2026-06-24 00:15 — regenerated at runtime by a Claude session, while sibling assets carry the 2026-06-23 22:43 install mtime). A cache-only file is one `plugin uninstall` away from extinction. - **Evidence:** `.gitignore` `*.html` present since first commit `d0d3a40` (verified via `git show d91748c:plugins/build-options/.gitignore`); template absent from all history (`git log --oneline -- plugins/build-options/assets/matrix.template.html` was empty pre-rescue); pre-rescue pytest: 2 failed with FileNotFoundError. - **Rescue (operator-authorized, executed 2026-07-02):** cache copy copied into `plugins/build-options/assets/matrix.template.html`; `.gitignore` gained line 6 `!assets/matrix.template.html` (negation). Post-rescue: `9 passed` (re-run 2026-07-02). - **Status:** **fixed-uncommitted** as of 2026-07-02 — `git status` shows `M plugins/build-options/.gitignore` and untracked `plugins/build-options/assets/matrix.template.html`. The operator commits; assistants run no mutating git commands. If `git status` is clean and the template is tracked, promote this entry to `fixed`. - **Lesson:** any `.gitignore` wildcard in a plugin must carry `!`-negations for shipped assets, and "it works on my machine" can mean "the cache is hiding the loss." Structural rules live in `fs-plugin-anatomy`. ## Entry 2 — PEP-668: README test commands fail verbatim - **Symptom:** `README.md` lines 218–219 say `python3 -m pytest -q tests/`. On the maintainer host (Python 3.13.5) this prints `No module named pytest`, and `pip install pytest` is refused (PEP-668). - **Root cause:** README was written assuming a system pytest; the host Python is externally managed. - **Working form (re-verified 2026-07-02):** ```bash python3 -m venv ~/venvs/founder-skills && ~/venvs/founder-skills/bin/pip install pytest cd plugins/market-validation && ~/venvs/founder-skills/bin/python -m pytest -q tests/ # 6 passed cd plugins/build-options && ~/venvs/founder-skills/bin/python -m pytest -q tests/ # 9 passed ``` - **Status:** **open** — README still carries the failing form. Fixing it is a README change: route through `fs-change-control`. Full environment runbook: `fs-toolchain-and-tests`. ## Entry 3 — node:test directory form fails; JS suite undocumented - **Symptom:** `node --test tests/js/` → `MODULE_NOT_FOUND` (re-verified 2026-07-02 on Node v24.16.0). The top-level README's Develop/test section doesn't mention the JS suite at all. - **Root cause:** the tests live under `tests/js/*.test.mjs`; on this Node the directory form does not recurse to find them. Only the glob form works, and it's documented in exactly one place: `plugins/market-validation/SKILL.md:112`. - **Working form (quote the glob so the shell doesn't expand it):** ```bash cd plugins/market-validation && node --test 'tests/js/*.test.mjs' # 5 tests, all pass ``` - **Status:** **open** (README omission). The suite itself is green. ## Entry 4 — Workflow scripts are syntax-checked only - **Symptom:** none yet — that's the point. Two `Workflow({scriptPath})` scripts have **never been executed end-to-end in generalized form**: `plugins/market-validation/assets/research-workflow.js` (the args-path variant) and `plugins/build-options/assets/build-options-workflow.js` plus its prd handoff. - **Root cause:** they were generalized from private originals and validated by syntax check + the Function-wrapper JS harness (`tests/js/harness.mjs` stubs `agent`/`parallel`/`phase`/`log`/`args`), which is a simulation, not the real Workflow runtime. - **Evidence:** the skills say so themselves — `plugins/market-validation/SKILL.md:93` and `plugins/build-options/SKILL.md:63`: "syntax-checked only — its first real invocation is its proving run." - **Status:** **proving runs executed 2026-07-02** via the campaign (Phase 1, minimum-honest scale, real product topic): research-workflow.js passed every expected observation (4/4 angles, 24 curated, 24 survived, exact return shape, args threaded); build-options-workflow.js ran end-to-end and its proving run **caught entry 9** (silent skeptic-verdict drop) — which is what proving runs are for. Honesty labels graduate to "proven on 2026-07-02" via the promotion spec, never deleted (fence F5). ## Entry 5 — prd + tasks: off-style external imports - **Symptom:** `plugins/prd/SKILL.md` (200 lines) and `plugins/tasks/SKILL.md` (480 lines) read nothing like the golden four: different voice, ❌/✅ emoji, no `references/`/`assets/`, no model routing, no Announce convention, 2–6× the house length norm. - **Root cause:** imported wholesale from an external "compound engineering" methodology rather than authored in-house. - **Hidden dependency:** `plugins/tasks/SKILL.md:67` **mandates** the third-party `agent-browser` CLI (github.com/vercel-labs/agent-browser) for all browser acceptance criteria — an unstated install dependency nowhere in the install docs. - **Status:** **fixed-uncommitted 2026-07-02**: both rewritten in place to house style (prd 200→128 lines v0.2.0, tasks 480→150 lines v0.3.0), worked examples moved to `references/`, agent-browser genericized to "browser executor" (1 "such as" mention), handoff contract (`/tasks/prd-*.md` → `prd.json`, field names unchanged) verified by an independent acceptance agent — all 9 campaign Phase-2 criteria pass. tufte-viz's nonconforming `|` frontmatter remains the only never-copy holdout. Style source of truth: `fs-skill-style-guide`. ## Entry 6 — Pages site oversell drift - **Symptom:** the live site (`docs/index.html`, deploys on every master push) makes claims the repo itself forbids: - lines 227 and 230: "~0 tokens" cost chips for session-start/effort — README doctrine says "a few hundred tokens"; - line 243: fable-org-audit "runs itself weekly via /schedule..." — the skill only *documents* scheduling options; nothing runs itself. - **Root cause:** marketing copy drifted past the no-oversell rule during the 2026-06-11 site work. - **Status:** **fixed-uncommitted 2026-07-02** (operator un-gated Phase 3 in-session): chips now read "a few hundred tokens", the org-audit card describes weekly cadence as something you set up, count copy is "fifteen", and 3 maintainer-tooling cards were added — `grep -c "~0 tokens\|runs itself weekly" docs/index.html` → 0 and `site-checks.sh` is fully green. Sibling-site (claude-skills-site) coordination remains open: its copy still says "12 installable today". - **Lesson:** the repo's own doctrine (`fs-doctrine-and-honesty`) applies hardest to the most public surface; that's exactly where it slipped. ## Entry 7 — add-skill.sh hazards `scripts/add-skill.sh` (~160 lines) works, but three sharp edges are recorded so nobody is surprised (line numbers verified 2026-07-02): | Hazard | Evidence | Consequence | |---|---|---| | Validation is manifest-schema-only | lines 148–150 run `claude plugin validate`; it passed build-options while its core asset was missing (entry 1) | a green validate does NOT mean the plugin works | | Destructive symlink swap | line 142: `rm -rf "$SKILLS_DIR/$NAME"` when `~/.claude/skills/<name>` exists and is not a symlink | a real (non-symlink) personal skill directory of the same name is deleted | | Cross-lists into a second marketplace | line 22: `RRF_REPO="${RRF_MARKETPLACE:-$HOME/claude-code-plugins}"` (RobotRegistryFoundation) | on machines without that repo it skips with a notice — but on machines *with* it, running the script mutates a second repo | - **Status:** **accepted-risk** (no fix specced). Mitigations when running it: back up any same-named `~/.claude/skills/<name>` first; never trust `validate` as a functional test — run the suites (`fs-toolchain-and-tests`); check `~/claude-code-plugins` for unexpected diffs afterward. Full authoring workflow: `fs-skill-authoring`. ## Entry 8 — emit_atlas.py → emit_market_map.py rename - **Symptom (potential, never observed):** any external user invoking `emit_atlas.py` by path broke on 2026-06-11. - **Root cause:** deliberate vendor-neutral rename, commit `5f07aef` ("refactor: atlas emitter -> vendor-neutral market map emitter"): `assets/platatlas/` → `assets/market-map/`, `emit_atlas.py` → `emit_market_map.py`. - **Evidence:** the spec pre-declared the risk — `docs/superpowers/specs/2026-06-11-public-judgment-layer-design.md:88`: "the rename of `emit_atlas.py` could break external users mid-flight — acceptable, the marketplace install pulls fresh and nothing external imports the module by path." - **Status:** **accepted-risk**. Note the caveat: **no telemetry exists** to know whether anyone was actually affected — the "nothing external imports it" claim is an assumption, not a measurement. (The measured-not-claimed research direction in `fs-research-frontier` exists partly because of gaps like this.) - **Lesson:** renames of shipped executable assets are breaking changes on a live-publishing master; spec them, state the risk, and prefer a deprecation shim when cheap. ## Entry 9 — skeptic verdict drop: stress-test silently defaulted to "survive" - **Symptom:** the first real run of `build-options-workflow.js` (proving run, 2026-07-02) returned ALL THREE stress-tested options as `adversarial: {verdict: "survive", killerRisks: []}` — while the workflow journal showed the three skeptic agents had actually returned `wounded`, `wounded`, and `killed`. - **Root cause:** the refute prompt never told the skeptic the option's exact `id`; the schema made it invent an `optionId`; and the join (`advById[a.optionId] || {verdict:'survive', killerRisks:[]}`) silently replaced every mismatched echo with the default. A paraphrased id (e.g. `cloudflare-native-wedge` for `cf-native-soc2-evidence-wedge`) joined to nothing. - **Evidence:** run `wf_58e1fc39-7fb` journal: skeptic results carried optionIds `cloudflare-native-wedge` / `attestable-forensic-evidence-tier` / `attestable-evidence-rails-white-label-signing-api` vs actual option ids `cf-native-soc2-evidence-wedge` / `forensic-evidence-tier` / `attestable-evidence-rails-api`. The campaign skill had pre-documented this exact gotcha ("an empty killerRisks on a top-3 option is suspicious") — the proving run confirmed it as a live defect, not just a caution. - **Fix:** bind the verdict to the option **by construction** — the thunk's closure stamps `optionId: o.id` over the model's echo (`.then(v => v ? {...v, optionId: o.id} : v)`), plus the prompt now states the exact id. Trusting a model-echoed join key is the anti-pattern; the durable rule lives in `fs-anthropic-primitives` territory: join sub-agent outputs on keys YOU control.