scout-top

name

scout-top

description

Scout a top player's PLAYSTYLE (not just their deck) to find concrete policy-tuning targets — per opponent-archetype W-L / bench (prize-liability) / attack tempo, plus decoded decisions where OUR current policy diverges from theirs. Use to tune a policy, find why we lose a matchup, or learn what top players do that we don't. Triggers — "トップの打ち方を分析", "方策のチューニング標的", "なぜミラーで負ける", "scout a top player", "where does our policy diverge".

Scout top playstyle → policy-tuning targets

The behavioral counterpart to /meta-watch (which finds what decks are played). This finds how top players play and where our policy differs — the front-end that produced the v009 discipline patch. Feeds /build-submit (test + ship the tweak).

Loop: /meta-watch (what) → /scout-top (how + our gap) → patch → /build-submit (test+ship).

READ FIRST (07-14): which method fits the situation

• Our agent is a tuned 3rd-party pilot (current: the LO pilot, v021+)? Decision-diff mimicry against top players has failed ~every time ("decision-match ≠ strength", exp042/048, bench-min NO-GO 07-14). The method that actually produced a shipped win (KO_OFF, +0.057 silver-band) is instrument + counterfactual, no top player needed: wrap the pilot's mode predicates with counters, FORCE them on/off on shared CRN seeds (workspace/exp054_upperband/probe_arch.py), and gate through the pilot-patch pipeline in /build-submit. Scout replays are for finding WHICH matchup to probe, not for copying decisions.
• Same-deck players above us on the ladder? Find them by scanning cached replays for the deck's signature cards, cross-ref the full LB — this measures the deck's real ceiling directly (07-14: 3 players with our exact 60/60 LO list sat IN silver, 923-983, killing the "deck ceiling" worry without any patch). Decision-diff vs them: workspace/exp054_upperband/diff_silver_lo.py (needs silver_lo_players.json-style game index). Treat divergences as PROBE CANDIDATES to counterfactual-test, never as rules to copy verbatim (their bench-minimization regressed when copied naively).
• Our own legacy chain (v014/turnbeam era)? The original steps below apply as written.

Steps

1. Ensure the top player's replays are cached (gitignored under references/raw/replays/). If not cached, download them:

   cd workspace/exp011_meta_watch
   uv run python top_meta.py <top_submission_id> <tag>     # -> top_<tag>/
   

   (Find a top sub_id via /meta-watch's LB + traversing episodes; or reuse a cached tag: diff_53858964, top_debauchery, top_tk, top_charmq, ...)

2. Behavioral gap, by opponent archetype — does the top player adapt, and where do WE diverge? Pass OUR CURRENT policy to compare the right baseline:

   cd workspace/exp018_adaptive
   uv run python analyze_adaptation.py <sub_id> <deck.json> <cache_tag> [policy.py]
   # policy.py exposes make_agent(deck): exp013_router/router_policy.py = v008 (default),
   #                                     exp018_adaptive/discipline_policy.py = v009
   

   Prints per opponent-archetype: their W-L, avg bench (prize-liability), attack/turn (tempo), and decision-match vs our policy. Read it as:

   • A matchup where they win but we'd match them least = our biggest gap there.
   • bench/tempo shifts vs an archetype = an opponent-adaptive trigger worth copying.
   • Uniformly low match across all matchups = a consistent style gap (e.g. they bench fewer / hold resources), not opponent-switching — fix it globally.

3. Decoded decisions (what cards/choices differ), for the concrete patch. Use policy_diff2 — it takes OUR CURRENT policy as the baseline. The old exp013_router/policy_diff.py hardcodes v008 (router) as the comparison policy; in exp042 this nearly produced a wrong conclusion ("discipline is missing" when it was already in the v010+ chain). Env flags select the chain version:

   cd workspace/exp042_benchdisc
   REVENGE_BONUS=50 [BENCH_DISC=1] uv run python policy_diff2.py \
       <sub_id> <deck.json> <max_eps> ../exp023_revenge/revenge_policy.py
   

   Lists TO_HAND search targets they fetch vs we fetch, and top divergence examples decoded to card names, by SelectContext. Read the sem-rate column, not the raw match rate: index-based comparison counts same-card-different-copy picks as mismatches (measurement artifact found in exp042). Interpret with the 6 lenses in references/knowledge/ptcg_strategy.md (prize / tempo / search / sequence / disruption / prize-liability). A low-match SelectContext is only actionable after checking the BASE choose(): e.g. TO_BENCH/SETUP_BENCH had NO scoring branch at all (always ), so the gap was structural, not a tuning constant.

Notes

• Needs the cabt engine (uv run) + cached replays. Behavioral signals from small caches (20-65 games) are directional, not significant — confirm with the n≥200 eval.
• Top players' edge is often CONSISTENCY/discipline (prize-liability, resource conservation), not opponent-archetype switching — see [[meta-and-leaderboard]].
• Reusable assets: analyze_adaptation.py (behavior by opponent), exp042_benchdisc/policy_diff2.py (decoded decision diff vs a SELECTABLE policy, with sem-rate), eval_mirror.py / eval_compare.py and exp035_turnbeam/eval_tb.py (noise-safe validation).
• A top player's cached replays are also BC / fine-tune material for the exp041 pretraining pipeline (Plan B2: Mogja J mirror games) — scouting output is dual-use, keep the decoded-decision caches.