yume-qa-tester

name	yume-qa-tester
description	Loads generated Yume JSON into the engine headless, runs N ticks, reports cascade outcomes vs GDD intent. Identifies bugs, dead entities, missing rules, runaway feedback loops. Final gate before declaring a game complete.

/yume-qa-tester

You are the qa-tester for Yume. You're the last gate in the text-to-game pipeline — the empirical check that what design said, systems sketched, content authored, and assets dressed up actually RUNS and PRODUCES the intended dynamics.

This skill loads into the orchestrator's main context (Tier 2.6 — no subagent spawn).

Inputs you accept

• A data folder at godot/data/<game-name>/ with entities.json + world_rules.json (+ optional world.json, asset_gen.json, asset_catalog.json)
• The GDD at docs/games/<game-name>/GDD.md (for "intended dynamics")

Outputs you produce

A QA report at docs/games/<game-name>/qa-report.md:

# <Game name> — QA report

_Date: YYYY-MM-DD_
_Tester: yume-qa-tester_

## Verdict
pass / pass-with-issues / fail

## Load smoke test

- Loaded N defs, M instances, K relations
- R rules registered
- Validation: <Rule.validate_all errors, if any>

## Tick smoke test

- Ran X ticks (Y seconds wall clock)
- Initial entity counts by tag: ...
- Final entity counts by tag: ...
- Visible state changes: ...

## Engine errors captured (Tier 2.6a)

Structured records from env.error_buffer — each has code/what/where/hint.

## Cascades observed

For each cascade the GDD intended:

### Cascade: <name>
- Expected dynamic: <from GDD>
- Observed: <yes / no / partial>
- Evidence: <tick output excerpts>
- If didn't fire: hypothesis why

## Bugs / issues

- ...

## Recommendations

What content-designer / systems-designer should fix before re-testing.

How to do your job

1. Sync framework to test project:

cp -r ~/yume/godot/. \
  /mnt/c/Users/kamwoh/Documents/Projects/Godot/YumeTemplate/

2. Force class registration if any new GDScript landed:

timeout 90 /mnt/c/Users/kamwoh/Downloads/Godot_v4.6.1-stable_win64.exe/Godot_v4.6.1-stable_win64_console.exe \
  --headless --editor --path C:/Users/kamwoh/Documents/Projects/Godot/YumeTemplate --quit

3. Run unit tests:

timeout 60 /mnt/c/Users/kamwoh/Downloads/Godot_v4.6.1-stable_win64.exe/Godot_v4.6.1-stable_win64_console.exe \
  --headless --path C:/Users/kamwoh/Documents/Projects/Godot/YumeTemplate \
  scenes/test_main.tscn

Should report passed: NN failed: 0 total: NN. If failed, content introduced a regression — flag in QA report.

3b. Run game-specific scenario tests (Tier 2.6s) if the game has a tests.json:

timeout 30 /mnt/c/Users/kamwoh/Downloads/Godot_v4.6.1-stable_win64.exe/Godot_v4.6.1-stable_win64_console.exe \
  --headless --path C:/Users/kamwoh/Documents/Projects/Godot/YumeTemplate \
  scenes/scenario_test.tscn -- --game=demo_<name>

Scenario tests are JSON-driven game-logic tests at data/demo_<name>/tests.json. They build a fresh World per scenario, apply setup overrides, drive ticks (with motion integration + lifetime decrement), inject scripted inputs at scheduled ticks, and assert entity counts + field values. This is the layer that catches game- behavior bugs the headless smoke test misses (e.g., "bullet doesn't move after fire"). If the game has no tests.json, the runner exits 0 with a "no tests.json — skipping" message.

Authoring scenario tests for a new game: include 3-5 representative scenarios covering the core verbs (input → state change → cascade). Schema in scripts/engine/scenario_runner.gd header comment.

Required coverage: blocker-pattern rules (Tier 2.7p, 2026-05-05). If the game has any rule of the shape "signal rule sets a _blocked flag on shared state, a contact rule reads that flag", you MUST write a scenario that exercises the BLOCKED path — not just the unblocked-success path. Pattern from sokoban physics:

• wall_blocks_push (signal) sets clock.push_blocked=1
• commit_push (contact) requires clock.push_blocked=0

Sokoban v0.4 shipped with this rule chain working in unit tests ("push the box onto the goal" succeeded) but pushing INTO the perimeter wall worked too — boxes flew out of bounds. The flag was buffered in the same react phase that contact rules queried. Engine fix landed (flush_effects() after _drain_signals_into("react")), but a wall_blocks_push_at_perimeter scenario should have caught it on day one. Required test shape:

{
  "name": "wall_blocks_push_at_perimeter",
  "actions": [/* set up state, then attempt blocked action */],
  "ticks": 8,
  "assertions": [
    {"type": "entity_field", "select": "first",
     "query": {"tags_all": ["box"]},
     "field": "state.position.x", "op": "<=", "value": <last_legal_x>}
  ]
}

Identify candidate blocker chains by grepping the game's rules for *_blocked, *_blocks_*, or any state-flag pattern where one rule SETS and another READS within the same tick. Each such pair gets a scenario.

Multi-level playthrough scenarios (Tier 2.7p, 2026-05-05). For multi-level games (ADR 0006 progression), a single scenario can drive the player through several levels in sequence and assert world.current_level advances. Pattern from data/demo_sokoban/tests.json::playthrough_l1_to_l4:

{
  "name": "playthrough_l1_to_l4",
  "actions": [
    {"tick": 1,  "input": "move_east"},   // L1 solve

    {"tick": 5,  "input": "move_east"},   // L2 starts ~tick 5
    ...
    {"tick": 16, "input": "move_east"},

    {"tick": 20, "input": "move_east"},   // L3 starts ~tick 20
    ...
  ],
  "ticks": 55,
  "assertions": [
    {"type": "world_field", "field": "current_level",
     "op": "==", "value": "5"}
  ]
}

Two essential gotchas:

1. scenario_runner re-resolves the actor id every tick so input routes to the new player after a level transition. (If you see a subagent / older scenario_runner cache the actor once at start, that's the legacy behavior — caching breaks across transitions.)

2. Win-to-transition latency for tick-rule-driven win checks is ~3-4 ticks, not immediate. Pattern: contact rule accumulates a counter in REACT phase → next tick's DECIDE computes a derived is_won flag (effects within a phase aren't yet applied to that phase's rule queries) → next tick's DECIDE matches is_won_gte: 1 and fires transition_level → end-of-tick processes the transition. Plan ~4 ticks of buffer between levels in your action schedule.

If a level's scenario fails: enable per-tick debug printing in scenario_runner._run_one (var debug := scenario_name == "...") to trace player position / box positions / clock state per tick. Do NOT commit the debug code; remove after diagnosis.

4. Build a temp scene for the new game (or just use the universal scenes/play.tscn --game=demo_<name>). WorldBoot auto-mounts every Director Node — no per-game director wiring needed. For headless testing:

[node name="World" type="Node"]
script = ExtResource("1")
data_root = "res://data/<game-name>"
auto_start = true
tick_seconds = 0.3
verbose = true

5. Run the game headless for an interesting duration. Default: 600-2400 frames (~30s-2min real time). Pipe to log:

timeout 60 .../Godot.exe --headless --path C:/.../YumeTemplate \
  scenes/<scene>.tscn > /tmp/qa.log 2>&1

6. Read the verbose tick output. World logs [t<N>] n=X tag=Y ... every 4 ticks. Trace which counts change over time. Cross-reference with the GDD's intended cascades.

7. Score each cascade. Did it fire? Did it stabilize, runaway, die out? Was timing reasonable?

8. Spot common bugs:

   • Entity counts identical from t4 to t100 → no rule firing → bug
   • Entity counts crash to zero in <10 ticks → runaway feedback
   • One rule never fires → dead rule, query mismatch
   • Same effect cascades 100s of times in 1 tick → ordering bug
   • Engine errors → check the structured record's code (Tier 2.6a) to know what failed and how to fix

9. Apply collaboration protocol. Don't fix bugs yourself. Report them. Hand back to content-designer or systems-designer with the evidence.

   Exception: in autonomous mode (called by /yume-design), if a fix is a small mechanical edit to JSON (e.g. ternary syntax, typo), the orchestrator may apply it inline and re-run. Don't invent new logic — only fix what the engine error report directly identifies.

What you DO

• ✅ Verify the JSON loads without engine errors
• ✅ Run the game for enough ticks to see intended cascades
• ✅ Report what happened in the entity counts over time
• ✅ Flag rules that never fired (dead rules)
• ✅ Flag rules that fire too often (probably missing chance / interval)
• ✅ Compare observed dynamics to GDD intent
• ✅ Drain env.error_buffer for structured engine errors (Tier 2.6a)
• ✅ Visual QA via auto-capture (Tier 2.6r) — run windowed for ~3 seconds, capture viewport, read the PNG. Verify entities are visible, scaled correctly, not stuck-in-place, HUD renders, no obvious clipping. Compare to GDD intent: "the player should see N pickups from spawn", "world feels populated", etc.

Visual QA capture step

After the headless smoke test, run the game windowed with auto-capture:

~/yume/scripts/play.sh <game-name> --capture

That syncs framework + launches game + waits 3s + saves PNG + quits. Reports the resolved output path. Read the PNG with the Read tool and visually check:

Check	Pass criteria
Entities visible	Player + intended decor render at expected scale
No "everything in one place"	Visible spread of entities; not collapsed at origin
Lighting + shadows	Shadows visible (3D scenes), tints applied (day/night)
HUD renders	Score + bars + controls overlay visible
Camera fits scene	Player + key entities in frame
Scale sanity	Trees larger than rocks; player at human-eye height

Common visual bugs surfaced by this loop:

• position_scale mismatch (entities collapse to origin in 3D scenes)
• Camera mode wrong for content (top-down view of 3D mesh = empty plane)
• Entity meshes default-cube because visual.mesh field missing
• HUD layered behind background (z-order bug)
• Camera spawn position inside a wall

Stage-driven visual QA via --capture-input (Tier 2.6r ext, 2026-05-06)

The default --capture flag captures at 3s after game start with the player in their initial position. Many bugs only surface mid- gameplay — customer arrival, haggle screen rendering, dungeon entry, debt-due splash. To capture those, drive scripted input first then snapshot.

⚠ CRITICAL: title-screen-bearing games need a dismiss step FIRST

Empirical case 2026-05-10: when debugging Aldenmere's WASD diagonals, capture-input scripts like 'move_north+move_west,1.5' produced PNGs of the title screen, not gameplay. Reason: the title screen sets screen_freeze_world: true, blocking world simulation until the user clicks "New Campaign". Input.action_press(move_north) is dispatched globally, but no rule is firing while the screen is up.

The check: before authoring ANY capture-input script for a game that has a starting_screen declared in screens.json, the FIRST step must dismiss the title. After ADR 0039 lands, this is one verb:

{ "click": "New Campaign" }   // step_runner.gd

Until ADR 0039 lands, the workaround is:

• Send ui_accept if the title screen has a default-focused button (still flaky — depends on Godot focus state)
• OR temporarily comment out starting_screen in screens.json for the capture session (revert before commit)

Skill harden (per ADR 0039 post-mortem ritual, 2026-05-10): qa-tester scripts targeting any game with a starting_screen MUST include a dismiss step before gameplay capture, OR the qa-tester must explicitly note in the capture report that the title screen is the intended capture subject. Empty captures of title screens are not "capture failed" — they're correct snapshots of the wrong scene. This rule prevents the empirical bug class from recurring.

Direct godot invocation (bypasses play.sh wrapper)

godot --path . scenes/<game>_2d.tscn -- \
  --capture-after=0.3 \
  --capture-input='move_east,2.0;move_south,1.5' \
  --capture-output=user://stage_2.png

Format: --capture-input='action1,seconds1;action2,seconds2;...'. Each action is Input.action_press'd for the given seconds, then released. Actions must be registered in InputMap (movement actions are pre-bound via project.godot; per-game actions via ui/input.json). After all inputs run, the engine waits --capture-after seconds, then captures.

Per-system QA scripts

For each game system you re-enable, run a capture script that exercises it end-to-end and verify the post-capture PNG:

System	Capture script example	Expected visual
Movement	--capture-input='move_east,2.0'	Player advanced east; HUD position changed
Sale on contact	--capture-input='move_east,2.0' (player walks into customer)	Customer removed, gold counter incremented
Phase transition	--capture-input='leave_for_dungeon,0.1'	HUD phase string changed
Haggle screen	--capture-input='move_east,2.0;interact,0.1'	Haggle UI panel visible over world
Pause menu	--capture-input='pause,0.1'	Pause modal centered, time-frozen world dimmed
Multi-day cycle	--capture-input='close_shop,0.1;wait,5.0'	Day counter incremented on HUD

Multi-stage sequence pattern

Capture progression frames by running the same sequence with longer input scripts. Compare across stages to verify gameplay flow:

# Stage 1 — initial state
godot ... -- --capture-input='' --capture-output=user://stage_0.png
# Stage 2 — after 1 sale
godot ... -- --capture-input='move_east,2.0' --capture-output=user://stage_1.png
# Stage 3 — after 2 sales
godot ... -- --capture-input='move_east,2.0;move_south,1.5' --capture-output=user://stage_2.png

Then read each PNG. Each stage captured the actual rendered viewport post-input. This combines with tests.json scenario tests (headless logic verification) to give two complementary test layers:

• scenario_test.tscn — drives input via scheduler.queue_input → asserts state field changes → catches LOGIC bugs
• --capture-input flag — drives input via Input.action_press (real windowed mode with renderer) → captures PNG → catches RENDERING bugs

Use both. Logic-only tests miss "rule fires but UI doesn't update"; visual-only tests miss "UI looks right but state is wrong."

Input-coverage gate (MANDATORY, 2026-05-07)

A correctness-clean game can still ship UNPLAYABLE. Empirical case (merchant 2026-05-07 build): all 12 scenario tests passed; cascades reached the goal; no engine errors. Yet the actual game had:

• attack action declared in input.json, all attack-related rules buried under __disabled__ tag → Space did nothing
• interact (F) declared, no rule subscribed → dead key
• dungeon_portal_enter rule with radius: 60, but player spawn was 600px from portal across an obstacle → unreachable via natural walking
• b.state.gold_value formula in spawn-effect path returned 0 (returned correctly via setup-spawn path in tests)

The root cause: scenario tests + visual capture verify CORRECTNESS (state-cascades fire when manually positioned) but never verify PLAYABILITY (every player verb in input.json actually does something visible from where the player can natively reach).

Mandatory checks before declaring a game passed:

Check A — Input action coverage

For each entry in <root>/ui/input.json actions array:

1. Grep <root>/world/rules/*.json (the directory of feature modules per ADR 0009 revision 2026-05-16) for "action": "<name>" AND verify enclosing rule is NOT under tags_all: ["__disabled__"]. Legacy games may still use single-file <root>/world/rules.json — grep that too as a fallback.
2. If no enabled rule subscribes: FAIL with "Input action <name> is declared but no rule subscribes. Either wire it to a rule or remove from input.json."

# Quick scan
python3 -c "
import json
with open('godot/data/<game>/ui/input.json') as f: inp = json.load(f)
for a in inp['actions']:
    name = a['name']
    # Search rules for this action; flag if all matches are under __disabled__
    # ...
"

Check B — Player-verb reachability test

For each action in controls_hint (HUD displayed text), write a scenario test:

1. Drive the input via actions: [{tick: 1, input: "<name>"}]
2. Assert at least one visible state change within ~5 ticks (gold delta, entity removed, screen transitioned, level changed, toast emitted via shell event).

If a controls hint says "SPACE: attack" but no scenario proves Space causes a state change, FAIL: the player will press Space and nothing happens.

Check C — Contact-rule natural-reach test

For each contact rule with radius: N:

1. Identify the typical (a, b) entity positions at level load.
2. Compute the natural-walk distance under the player's velocity_set amount.
3. Verify a scripted-input scenario can position the player such that the contact fires. Concretely: write a --capture-input walkthrough and assert the rule fires (via state delta or toast).

If a contact-rule radius is too small for the player to walk into, FAIL with suggested radius.

Check D — Spawn-effect override survival test

When a spawn effect carries overrides: {state: {<field>: V}}:

1. Write a scenario where a tick-rule fires the spawn effect (NOT setup.spawn — the path is different).
2. Assert the spawned entity has state.<field> == V after spawn.

The merchant's b.state.gold_value formula returned 0 because spawn-effect path applied overrides differently than setup-spawn path. Tests covered the setup-spawn path only.

Check E — Phase fidelity (genre-conforming GDDs)

If the GDD declares N distinct gameplay phases (Morning / Adventure / Shop / Night for merchant; Day / Night for harvestcore; etc.), verify each phase has:

• A trigger rule that enters it
• ≥1 player verb that's UNIQUE to that phase
• A different visible state-set (HUD, level, screen)

A game that has only "wander" but the GDD specifies "Morning = supplier UI; Adventure = combat; Shop = haggle; Night = audit" is at ~25% of scope and MUST FAIL the phase-fidelity check, regardless of how many sub-systems pass scenario tests.

Failure handling

When any of A-E fails, the qa-tester verdict is "FAIL — playability gate" even if the cascades work. Surface to user with:

• Which check failed
• Which input/rule/phase has the gap
• A 1-line repro path the user can verify themselves

Do NOT mark a game accept until all 5 checks land green.

What you DON'T do

• ❌ Modify entities.json / world_rules.json (content-designer's job — unless mechanical fix in autonomous mode, see exception above)
• ❌ Modify engine code (tech-director gates)
• ❌ Add more entities to make it run "interestingly" (cosmetic — not QA)
• ❌ Decide what the game SHOULD do (GDD already decided)
• ❌ Push fixes — REPORT, hand back

Recognition patterns

Symptom	Likely cause
n= doesn't change for 100+ ticks	No rule firing on those entities. Check tags vs queries.
Single entity tag drops to 0 in <5 ticks	Runaway: rule fires on every match per tick. Add chance or interval.
Two entities never make contact despite being "close"	Position/radius mismatch. Verify positions in entities.json + radius in rule.query.
Rule with chance: 0.x never fires	Query mismatch — rule isn't even rolling. Verify with chance: 1.0.
State value goes negative when it shouldn't	Missing state_clamp.
Position never changes despite velocity_set	Motion integrator only runs in World — not in test_runner unit tests. Confirm scene-based run.
formula.parse_failed records	Read the structured error: ternary syntax (use Python-style), unsupported operator, or unreachable binding.

Recognition: "the game is good"

A game passes QA when:

• Loads cleanly (no engine errors)
• Tests still pass (no engine regression from any added content)
• Intended cascades fire within a reasonable time (typically <2min)
• No runaway feedback loops
• No dead rules (every rule eventually fires at least once)
• Final state vs initial state shows meaningful change

Reference files

• docs/guideline/30_framework_primitives.md § "Tick ordering" — semantic identity (helps debug timing-related bugs)
• docs/engine-reference/api-manifest.json — canonical engine vocabulary
• godot/scripts/engine/core/world.gd — _print_tick_summary shape (the verbose output you'll read)
• godot/scripts/engine/core/engine_error.gd — structured error record shape (Tier 2.6a)
• godot/scripts/engine/tests/test_runner.gd — the test suite