experimental-architecture-stress-test

name

experimental-architecture-stress-test

description

Validate a feature's architecture before implementation by drafting it concretely, adversarially stress-testing it across green/yellow/red flows, reviewing it from multiple reviewer perspectives (end-user, developer-experience, architect, security, and — for cost surfaces — cost), and rendering a keep/revise/redesign verdict that clusters findings by root cause. Use when picking up a non-trivial feature or issue, when about to commit to an architecture, when asking 'is this the right design before we build it', or to avoid discovering a wrong architecture through post-merge bug review.

Experimental Architecture Stress-Test

Pressure-test a feature's architecture before writing code. The expensive failure mode is shipping an architecture that does not model the new concept's invariants, then discovering it one bug at a time in review (see Provenance). This workflow surfaces those invariants up front by running the design against its own worst flows and judging it from multiple reviewer perspectives (end-user, developer-experience, architect, security, and — for cost surfaces — cost).

This is the validation loop that sits after a draft architecture exists and before implementation begins. It composes experimental-architecture-spec (which produces the draft) rather than replacing it.

Core Rule

Make the architecture prove it handles every red flow, or name the structural reason it can't — and make it prove its assumptions are requirements, not inheritances. A pile of independent bugs and a wrong architecture look identical at first — the difference is whether the findings share a root cause. The load-bearing step is clustering: many findings tracing to one unmodeled assumption is a redesign signal, not N patches.

The review is an exploration, not a defense: not "how can we make this work" but are these the right primitives, are we overcomplicating, how can we simplify. Most design work is distilling the right shape of the problem being solved and mapping it to patterns or technologies that already exist for that shape. A review that only stresses the design as-given will elaborate mechanisms on a wrong shape forever — each round's findings get fixed with more machinery while the assumption that generated them survives. The Challenge phase exists to break that loop: trace every load-bearing assumption to an actual requirement, name the generic problem underneath it, and ask what boring existing solution already covers it.

The agent that drafts the architecture is not a trustworthy judge of it (the generation/evaluation separation). Stress and verdict run with an adversarial stance — the design is guilty until shown to handle each flow — and the perspective panel runs in contexts separate from the drafter.

When to use

• Picking up a non-trivial new feature or GitHub issue, from any entrypoint.
• A new subsystem, a cross-boundary feature, or one that introduces a new concept (a new data type, a new lifecycle, a new trust boundary, a new cost surface).
• Before committing to an architecture you'll be hard to walk back from.

Skip it for narrow bug fixes, mechanical migrations, isolated UI polish, and changes where the local pattern is already obvious. If a draft would be trivial, the stress-test is too.

The loop

1. Draft (one pass, composes experimental-architecture-spec). Produce a concrete, close-to-real-code architecture: public APIs, prod/test call graphs, seams, the invariants this design assumes, and the data each new concept carries. Skip if a usable draft already exists — point the workflow at it instead.

2. Challenge (adversarial to complexity — in both directions; runs concurrently with Stress). For each load-bearing assumption and new mechanism in the draft: trace it to an actual requirement (untraceable = inherited — flag it), distill the generic problem shape underneath it ("concurrent structured storage", "ordered event delivery", "scoped extensible configuration", …), map that shape to prior art — an existing technology, well-known pattern, or in-repo precedent — and recommend keep / simplify / replace / generalize. The two failure modes are duals:

   • Overcomplication — machinery tracing to no requirement, or re-solving a shape existing tech covers → simplify/replace.
   • Under-generalization — a fixed enum/allowlist/one-off where the product treats the same category as open and extensible elsewhere, the Nth bespoke instance of a shape the repo already assembles two other ways, or prior art solving the general problem while the draft hand-rolls a special case → generalize (name the registry/scope/seam; carry trust constraints as schema metadata, not hardcoding).

   A challenge that deletes machinery, or replaces a special case with a primitive the product already needs, is the most valuable finding the workflow can produce.

3. Stress (adversarial fan-out). Enumerate the feature's use cases as flows, each tagged:

   • green — the happy path.
   • yellow — expected-but-degraded: slow network, large input, partial data, reload mid-task, retries, empty/duplicate input.
   • red — adversarial/failure: removal or cancel mid-operation, concurrency on shared state, malicious or malformed input, crash mid-write, and inputs of a kind the design silently assumed away (e.g. a binary file where the path only models text).

   Run each flow against the draft and ask one question: does the architecture define correct, modeled behavior for this flow, or does it lean on something the design does not actually model? Every "leans on an unmodeled assumption" is a finding. (At deep rigor a completeness-critic round first hunts for the class of flow the enumeration missed, and unmodeled red findings are then adversarially re-verified — see Cost / rigor tier.)

A KEEP/REVISE verdict feeds implementation-workflow. A REDESIGN sends you back to step 1 with the named cause as the new constraint.

Running it

This skill ships a Workflow script (scripts/stress-test.workflow.js) and the reviewer personas it reads (under personas/).

• Preferred (multi-agent). Offer to launch it; on opt-in run Workflow({ scriptPath: "<skill-dir>/scripts/stress-test.workflow.js", args: { feature, anchors, draftPath? } }) (point scriptPath at wherever you installed the skill). The Workflow tool requires explicit user opt-in — surface the run and its rough agent count first, don't launch silently. The script resolves the personas via a PERSONA_DIR constant (default skills/experimental-architecture-stress-test/personas, relative to the repo root, or args.personaDir) — set it to your install path.
• Inline fallback. If the user declines the fan-out, run the five phases yourself: one draft, the assumption-challenge table (trace → shape → prior art → keep/simplify/replace), an enumerated flow table you stress one band at a time, the four standing persona reviews read from personas/ (add the cost lens only for a real cost surface), then the clustered verdict with explicit challenge resolutions.

args.feature is the intent/issue text, args.anchors the concrete entrypoints and files to ground the draft, args.draftPath an existing draft to skip step 1, and args.bands (e.g. ["red"]) restricts stressing to those bands — use it to cheaply re-validate a revised draft against just the failure band; the verdict then covers those bands only. args.rigor (alias args.cost) is the cost/rigor dial — see below. args.lenses overrides the panel's reviewer set for one run (e.g. ["architect","security","cost"] to pull the cost lens in for a feature with a real cost surface, without paying for the whole deep tier). (args reaches the script as a JSON value; the script also tolerates a JSON-string form.)

Two different cost surfaces

Do not conflate these — the skill serves both, in different places:

• The cost of the architecture under test is a review concern. Its one structural check — model-routing fit (does the design hardcode the top model / fail to route) — lives in the standing architecture-reviewer lens. The deeper token/spend/context profile is the cost-economics-reviewer persona, which is not a standing panel member: it's low-signal for most pre-implementation architectures at this stage, and runs only at deep rigor or when pulled in via args.lenses for a feature that genuinely introduces a cost surface.
• The cost of running the stress-test itself is a dial, not a perspective: args.rigor (see below). Nothing reviews it at runtime — spending tokens to police token spend would be self-defeating. The tier table in the script is the single place that governs it.

Cost / rigor tier (args.rigor, alias args.cost). One dial moves the workflow's own cost and rigor together — the two failure modes are symmetric: a top-tier session silently paying top-tier for 50 stress agents (over-spend), and a high-stakes design getting only a one-vote, red-band, cheap-model pass (under-rigor). Values (aliases low/medium/high):

• quick — a cheap sanity pass: red band only, Sonnet on every step including the judgment ones, three builder lenses (architect / security / developer-experience), single-vote stressing. A KEEP here is explicitly partial — the verdict is told which bands and lenses were scoped out.
• standard (default) — the historical routing: full fan-out, all bands, four standing lenses (adds end-user; cost is conditional); Sonnet for the per-flow stress volume center, Opus for enumeration and the panel, and only the three judgment-critical steps (draft-from-scratch, the assumption challenge, the root-cause verdict) inherit the session model. A top-tier session pays top-tier for exactly those three steps, not for the ~30–60 stress agents.
• deep — a structurally more thorough test for designs expensive to walk back. Adds the cost lens (five in total) and two mechanisms that raise rigor, not just spend: a completeness-critic enumeration round (a second pass that hunts for the class of flow the first pass silently assumed away — the exact failure this skill exists to catch), and multi-vote adversarial verification of every unmodeled red finding (independent refuters; a unanimous refutation overturns a false finding so it can't manufacture a bogus REDESIGN, a split vote leaves it standing but marked disputed). Stress runs on Opus here.

Cost. The script runs one stress agent per enumerated flow, so cost scales with the flow count: a thorough standard run on a real feature was ~70 agents / a few million tokens / ~15–20 min, and surfaced issues a whole review cycle would have. quick is a fraction of that; deep adds the critic round and up to verifyRedVotes agents per unmodeled red flow. Match the tier to the stakes; the inline fallback (one agent, all phases) is the floor below quick.

Output Shape

## Validated Architecture: <feature> (rigor: <quick|standard|deep>)

### Draft (or: draftPath)

<concrete architecture — APIs, call graphs, seams, assumed invariants, per-concept data>

### Assumption challenges

| assumption | traced requirement | problem shape | prior art | keep/simplify/replace |
| ---------- | ------------------ | ------------- | --------- | --------------------- |

### Flow matrix

| flow | band | modeled? | finding (unmodeled assumption) |
| ---- | ---- | -------- | ------------------------------ |

### Panel

- end-user / dev-experience / architect / security (+ cost, if run) — concerns + suggestions

### Root-cause clusters

- <cause> ← [findings…]

### Verdict: KEEP | REVISE | REDESIGN

- challenge resolutions (every challenge adopted or rejected, with reasons)
- if REVISE: the targeted fixes
- if REDESIGN: the structural cause + corrected direction

Working Rules

• Stress the original design intent, not a known bug list — the point is to surface issues the way they'd actually arise, before code.
• A flow with no clean, modeled answer is a finding even if "it'd probably be fine."
• Clustering is the deliverable, not the finding count. Always ask whether findings share a cause before recommending fixes.
• Keep the draft concrete enough that another agent could implement a KEEP verdict without reinventing architecture.
• Personas contribute suggestions, not vetoes; the verdict synthesizes them.

References

• personas/ — the reviewer personas the Panel phase reads (shipped with this skill).
• references/evals.md — routing and outcome evals for this workflow.
• If present in your skill set, composes with experimental-architecture-spec (produces the Phase 1 draft this loop validates), meta-loop-evaluator (the generation/evaluation separation and adversarial-stance posture), system-design-review (the architect lens this composes, not duplicates), and implementation-workflow (what runs after a KEEP/REVISE verdict).

Provenance

Motivated by a real feature (a file-upload substrate): a clean draft whose two unmodeled assumptions — "an upload is just a text artifact" and "uploads are ephemeral send-time state" — produced a cluster of binary-path and lifecycle-race bugs found one at a time in review, forcing a rewrite. This workflow exists to surface that class of structural error before code.

The Challenge phase was motivated by a workspaces/projects substrate: two consecutive REDESIGN rounds correctly found ever-finer mechanism gaps in a file-canonical design, but only a human stepping back asked "is this all coming from 'nodes should be markdown files'?" — the problem shape was concurrent structured storage with integrity, already solved by SQLite (and by an existing store in the same codebase). Stressing the design as-given could have elaborated mechanisms forever; the fix was retiring the assumption. Challenge makes that question a phase instead of a lucky interruption.

The generalize direction came from the same feature one revision later: the reviews let a hardcoded five-field "workspace config overlay" survive four rounds, in a product where every comparable category (tools, UI contributions, resources) is an extensible registry. The problem shape was scoped, schema-validated configuration with contributed types — VS Code's settings model — and again a human spotted it, not the review: the review only asked whether the design as-given was too much, never whether a closed list should be a seam. Challenging in both directions closes that hole.