| name | investigating-game-balance |
| description | Use when a game "feels off" but you don't know where — bots dominate that shouldn't, prey never gets caught, scores don't match the design, players die invisibly, or one strategy beats every other. Open-ended diagnostic, not a targeted fix. |
investigating-game-balance
When to invoke
Symptoms that should trigger this:
- "rabbit-spam beats everyone, that's not the design"
- "no one ever catches the elephant"
- "bots score 30 in round 1 and 0 in every round after"
- "this strategy was supposed to be viable and it's not"
- a user reports a vibe — "feels too easy" / "feels broken" — without naming the cause
If the user has already named a specific tunable to change ("lower the moose gut prob"), skip this skill and use testing-game-balance-change instead.
Core principle
Balance bugs hide behind aggregate numbers. A "total captures" count can stay flat while the actual failure mode shifts. You diagnose by spreading the data across three axes simultaneously: bot type, prey kind, and time within the round. The pattern jumps out when you see all three at once.
The investigation workflow
- List the scenarios you need. Each major bot policy in self-play, plus 1-2 mixed rosters. Include the policy the user is complaining about. Don't include human players — automate-only.
- Pick a round length that lets failure modes appear. Short rounds (~60s) hide energy/economy bugs; long rounds (~5min) hide spawn-rate bugs. Run BOTH if you have the time.
- Use multiple seeds. Single-seed runs in stochastic games have huge variance — three captures vs zero might be noise, not a real delta. 3 seeds minimum for any conclusion.
- Capture three layers per round:
- Catches by kind (which prey actually got hunted)
- Score by bot (which policy is winning)
- End-of-round resource state per bot (energy / health / hand size / whatever the bot needs to function)
- Look for inconsistencies between layers, not at any one layer in isolation. The diagnostic patterns below come from cross-referencing.
Diagnostic patterns to look for
| Pattern across the data | Likely cause |
|---|
| Captures = 0 AND end-energies are 0-1 | Bots are starving, not failing to find prey |
| Captures = 0 AND end-energies are healthy | Bots can't see / can't reach / can't coordinate |
| One bot has score 0 every round | That bot's strategy isn't viable in this roster, OR it's broken |
| Score drops sharply round-over-round | State leaking between rounds (reset bug) |
| One prey kind is never caught | Spawn rate, flee mechanic, or coalition requirement is too tight |
| One prey kind is caught 10× more than expected | That prey is a free lunch — too high reward for the effort |
| Self-play succeeds but mixed-play fails | Cooperation depends on shared assumptions broken by other policies |
| Energies stay high but no captures | Bots resting too defensively — threshold too high |
What to write down
For each scenario, record a single line. Aim for legibility, not completeness:
roster=4×moose_hunter ticks=1800 rounds=3 seed=1
captures: Moose 5, Rabbit 4, Stag 1
scores : [30, 0, 30, 30] / [0, 0, 0, 0] / [21, 25, 21, 7]
end-energies: [44,30,25,50] / [41,40,32,29] / [37,36,32,40]
→ moose hunting works, round 2 was a wipe (no captures, energy still OK = bots couldn't find a moose)
The "→" line is the whole point. If you can't write one, you didn't look hard enough at the cross-references.
When to stop investigating and act
Stop when you can complete this sentence:
"The problem is X, and the smallest change I can make to test a fix is Y."
Then switch to testing-game-balance-change to validate Y.
If you can't finish that sentence, run more scenarios. Don't guess and tweak. A guess masquerading as a fix wastes the next investigation round.
Common mistakes
- Looking only at total captures. Total can stay flat while the failure mode changes from "bots starve" to "bots can't reach prey."
- Trusting a single seed. "It went from 6 to 5 captures, the change made things worse" — no, that's seed-1 noise. Need 3+ seeds.
- Running only short rounds. Energy / fuel / resource bugs are invisible at 60s.
- Forgetting mixed rosters. Self-play tests the policy in isolation; mixed reveals "this strategy fails when allies don't cooperate."
- Fixing the first thing you see. Often the visible symptom is downstream of the real bug. Investigate broadly before touching code.
A real example
In stag_hunt, a 60s self-play eval of elephant_hunter showed 1 capture/round and looked healthy. An 8-hour interactive session showed the same bots had 162 tramples vs 1 capture — they were starving inside long rounds, getting hit and unable to recover. Three layers told the story:
- captures: 1 (looked OK)
- score: max 29 (looked OK)
- end-energy: 0-1 across all 7 bots (smoking gun)
Without the end-energy snapshot, the score line "29" would have read as "fine" rather than "29 from one good moment, then died." Same lesson everywhere: capture the resource state, not just the result.
See also
testing-game-balance-change — once you know what to changedebugging-stuck-game-agents — when "captures = 0, energies = 0-1" appears- For stag_hunt specifically:
./balance_sweep.sh runs a fixed grid of scenarios with the right output shape.
