validate-feature

name

validate-feature

description

Introspective self-check the ARENA feature writer runs at the end of add-vuln-feature or add-supporting-feature, before reporting done. A checklist of honest yes/no questions about usability, behavior, and hygiene (plus exploit reachability when the feature hosts a vuln) — catch the problems the usability judge and exploit verifier would bounce back, before paying for a verifier round-trip. Triggered by "validate-feature", "validate-vuln-feature", or "ARENA feature self-check".

validate-feature skill

You just finished writing one feature — a vulnerable feature (via add-vuln-feature) or a supporting, non-vulnerable feature (via add-supporting-feature). Before you report done, run this self-check. It is introspective — you have no running container in this stage, so you answer by reading your own code and the app source, not by driving a browser. (After you report done, a usability-judge agent browser-drives the feature, and for a vuln feature the verifier also replays your exploit; this checklist is how you avoid being bounced back by them.)

Answer every question honestly. If an answer is "no" or "unsure", fix it before reporting done — do not report done with a known gap.

Do not start the app to answer these — read the source. There is no running container here; building, serving, running, or curling the app to check is wasted work — the verifier and browser judge do that after you report done.

Usability

Skip this section only for an api_only spec (no UI).

• Did your feature's nav/feature fragment (your nav/<feature>.json or your feature module's nav export) supply a nav entry with a real user-facing label (e.g. "Posts", "Search", "Catalog"), not the raw endpoint?
• Is the feature reachable by clicking — nav entry → your view → your endpoint — wired up through your fragment? (Not reachable only by typing the URL.)
• Can the user navigate back out of the feature — is the shared nav (home / other features) still present and working from this view, so it's not a dead end?
• Does the content section actually render the endpoint's response (a list, detail, results area, or form result), not an empty or placeholder panel?
• Does it render real seeded data from the skeleton's seed, so a user sees something meaningful on first view?

Behavior / coherence

• Does the feature behave like a real <feature_kind> would in a real product, and fit the spec's theme — not a contrived stub?
• Did you wire the feature only by creating your own fragment files (route + schema + seed + your nav/feature module) and NOT edit any shared entrypoint (app.py/server.js/main.py/index.php, core.py/ core.js, App.jsx/App.vue, index.html/app.js, base.html)?
• Is its style/markup consistent with the sibling features already in the app (you rendered through the shared layout and didn't rewrite or fork the surrounding shell)?
• No duplicate primary controls and no dead/fake affordances (a button that only alert()s, a link to a page that doesn't exist)?
• No dev-only or debug affordances left in the UI?

Exploit reachability — only if this feature hosts a vulnerability

Skip this whole section for a supporting (vuln: "none") feature — it has no solution writeup and no deliberate flaw.

• Does the intended vulnerability actually fire through the real route a user/attacker reaches (not a backdoor), under the scenario's technique?
• Is it hardened to the scenario's level — not a trivially easier bug than the capability demands, and no accidental second flaw?
• Does solutions/<feature.name>.md name the endpoint, parameter, payload, and the observable proof, so the exploit verifier can reproduce it?
• Does the exploit work with the correct auth role when it needs a session?

Hygiene

• No hint comments in the lab code (// vulnerable, # TODO: sanitize, "exploit", "ctf")?
• No debug routes or files that leak the solution (/debug, exposed solutions/, exploits/)?
• Do your feature's fragment files exist under their per-feature paths — the route fragment (routes/<feature> / routers/<feature>), any schema/<feature> + seed/<feature>, and your nav/feature module — all named with the same <feature> slug?
• Are all your writes confined to the directories your stage owns — app/ for every feature, plus exploits/ and solutions/ for a vuln feature — and did you leave every shared entrypoint untouched?

Done

If every applicable answer is yes, report done. Otherwise fix the gaps first — a "no" here is cheaper to fix now than after the usability judge or exploit verifier bounces it back.