perception-decoding

name	perception-decoding
description	Use for Crewrift-specific perception decoding recipes when optimizing a player.

Perception & state decoding — recipes (crewrift tier)

On-demand recipes (27). Trigger→action heuristics; pull the relevant one when its situation arises.

1. Decode the crewrift Sprite v1 scene from retained tables, labels and object-id ranges, not pixels; re-derive the vocabulary from game source

crewrift

Crewrift Sprite v1 is stateful and incremental with NO whole-frame message: maintain three retained tables -- Layers (u8 id), Sprites (u16 id with width/height/UTF-8 human-readable label/RGBA), Objects (u16 id with x/y/z/layer/sprite_id) -- and apply each server update (define/delete-object, define-sprite/layer, set-viewport, clear-objects). The first message is an init burst (clear-objects, define-layer 0, set-viewport 128x128, static sprite defs); thereafter each 24Hz tick carries only changed objects, so a bridge must TOLERATE partial startup frames (don't assume the map object or walkability sprite arrived). Reconstruct state from the triple (object-id range, sprite label, xy); only the walkability map and shadow/vision overlay need pixel decoding. Identify entities by sprite LABEL plus object-id RANGE (no CV): players 'player right/left' at ids 1000+joinOrder; bodies 'body ' at ids 2000+i (link a body to its player by COLOR, not id); task bubbles ids 3000+idx; the color vocabulary is the 16 PlayerColorNames. Object-id ranges are stable and disjoint, so the same label means different things across ranges: the voting candidate grid (~9300+) gives an authoritative per-color alive/dead census every meeting ('player ' alive vs 'body ' dead) and the role-reveal icon range (~9500+) shows an imposter its teammates. Read game time from the invisible 1x1 'tick ' marker sprite (id 5016) re-sent every frame, not a local message counter. These label strings and id bases ARE the perception contract but are NOT in sprite_v1.md and are version-sensitive -- when perception breaks after a version bump, suspect object-id ranges, sprite labels, or camera offsets drifted and re-derive them from coworld-crewrift src/crewrift/{sim,global}.nim (including the button-to-action contract in sim.nim applyInput, since gamepad buttons map to low-level attack/vent/move, not named actions). Over Sprite v1 the LABEL path is authoritative (notsus's spriteDetectionsReady), so ignore legacy CV/OCR/patch-hash machinery. _{sources: coworlds/coworld-crewrift/players/notsus/README.md, metta/agent-plugins/kitchensink/skills/ks.observe-protocol/SKILL.md, personal_labs/crewrift_lab/best_practices.md, personal_labs/crewrift_lab/crewrift/crewborg/AGENTS.md (+2)}

2. Crewrift Sprite-v1 coords are camera-relative; self is the implicit camera center -- compute self-position from constants and all motion in world coords

crewrift

Crewrift's Sprite v1 /player stream is structured scene data, not a framebuffer: the server streams object placements with exact coordinates plus sprites carrying text labels precisely so agents read state from structured data with NO computer vision or OCR. The only image decodes needed are two alpha masks (static walkability, dynamic LOS shadow); the whole-screen pixel sprite-decomposition approach was abandoned as too CPU-bound. The single MOST IMPORTANT fact (not in sprite_v1.md): object x/y are CAMERA-RELATIVE (0-128 screen range), not world space. Recover the camera from the world-map object (object id 1, sprite id 1, at (-cameraX,-cameraY)) so worldX = obj.x + cameraX. Your OWN avatar is NOT a scene object -- it is the implicit camera center; derive self world position by inverting the camera relation (cameraX = (player.x - SpriteDrawOffX + SpriteSize/2) - ScreenWidth/2; with SpriteSize=12, SpriteDrawOff (2,8), screen 128 -> self_world = (camera_x+60, camera_y+66), valid from tick 1; in Among-Them screen ~(60,66), cameraX=player.x-60, cameraY=player.y-66, full 128x128 window, no view radius) and read your own color/role/state from HUD sprite labels (imposter/ghost icon). Get this transform exact and confirm it round-trips before building nav -- every movement target depends on it. Do NOT compute velocity from on-screen self position -- it reads the same center forever and reports zero velocity even while sprinting, silently breaking anti-stuck jiggle and hunt/flee checks; compute velocity and all distances/commitment from WORLD/camera coordinates (camera delta). World coords are unavailable until the map object arrives (degrade gracefully on first ticks), and camera state resets on clear/delete of the map object -- handle that reset inside the decoder. Two pitfalls: a perception adapter that paints a 'visible' stub for every off-screen entity clamps their screen coords to the same edge pixel giving near-identical distances so the pick flips on noise -- treat any always-visible stub as a candidate-oscillation source; and sprite/draw coordinates and collision coordinates have DIFFERENT origins (player.x is the collision corner, sprite is centered, ~4px gap), so reuse the existing kill/report-range conversion before adding CollisionW/H/2 rather than reinventing it. Verify protocol facts against the notsus reference (protocols.nim, sim.nim, global.nim); rewrite any AGENTS.md/design section that implies CV for Crewrift. _{sources: bitworld/among_them/GRAPHICS_REPORT.md, bitworld/among_them/players/lively_lecun/ROADMAP.md, coworlds/coworld-crewrift/players/notsus/README.md, personal_labs/crewrift_lab/crewrift/crewborg/AGENTS.md (+10)}

3. Do all pixel perception in palette-index space, and know the ShadowMap collisions

crewrift

For a fixed 16-color indexed renderer (Among-Them/crewrift, indices 0-15), parse in PALETTE-INDEX space, not RGB, because tinting, shadows, and UI are all index transforms; index 255 is a transparent 'don't draw' sentinel that appears only in sprite source data, never in a received frame. Shadowed floor is recolored through ShadowMap[idx & 0xF], and ShadowMap collapses indices 1/13/14 down to 12 (void), so a shadowed dark-navy/light-blue tile is indistinguishable from out-of-bounds void; when scoring frame pixels against the map you MUST accept both the normal map color and ShadowMap[mapColor] (without this, localization fails in dark rooms like Electrical), and decide ambiguous tiles from the wall mask / shadow geometry, not raw color. _{sources: bitworld/among_them/GRAPHICS_REPORT.md, bitworld/among_them/players/how_to_make_a_bot.md, opencode:ses_2051bd225ffefNc9y2aOEEFjHW, opencode:ses_21eeac2baffegQWJK4QL6GuTEI}

4. Compute line-of-sight geometrically by mirroring the renderer's raycast, not from pixel color

crewrift

Shadow/out-of-LOS is a palette remapping (ShadowMap) that collapses several blues onto the void index, so you cannot read visibility from raw color; compute it geometrically from the wall mask and known camera origin. Among-Them uses an integer Bresenham raycast (steps = max(|dx|,|dy|)) from the player to each pixel: floor behind a wall is shadowed but walls themselves stay visible, living viewers are shadowed, ghosts see unshadowed, and there is NO circular view radius (LOS reaches the full screen edge bounded only by walls). Make the parser's entity-visibility test MIRROR the renderer exactly (entity drawn only if its center tile is unshadowed and on-screen) or you get phantom/missed entities. In Crewrift the per-player shadow overlay (object 13000, sprite 5010, transparent alpha = visible) gives this raycast result directly, resent on every camera move with no staleness, absent for ghosts and meetings. _{sources: bitworld/among_them/GRAPHICS_REPORT.md, personal_labs/crewrift_lab/crewrift/crewborg/design.md, opencode:ses_21eeac2baffegQWJK4QL6GuTEI}

5. Determine PHASE first via an ordered cheapest-most-discriminating cascade that enumerates every render path; read the renderer source for ground truth

crewrift · ⚠ session-derived, unverified

A pixel/sprite parser must determine game PHASE before interpreting anything else, then dispatch to a phase-specific reader -- only the play phase has real world vision, the rest are fixed-coordinate menu/grid layouts, so one universal layout parser is a dead end. In Among Them (pixel) use tiny-font OCR for phase labels (WAITING, IMPS, CREWMATE, SKIP, CREW WINS, IMPS WIN, DRAW, NO ONE); if none match but bottom-right shows digits with no icon grid it is Playing. In Crewrift Sprite-v1 there is no phase field: read phase from which interstitial TEXT sprites are present, and learn your own role mid-game from imposter/ghost HUD overlays. A single logical phase can have multiple RENDER PATHS (leader-in-chatroom vs non-leader-in-overworld), each drawing different text/colors, so phase-detection must enumerate every render path -- one phase appeared once as a round clock in color 2 (detected as generic 'playing') and once as a colored phase label, and a single rule missed half the cases. When two phases share gross visual features (both a double border plus black interior), a new phase gets swallowed by an existing branch, so place the minimal most-discriminating distinguisher BEFORE the generic fallback (specific phase-label reads at color 8 / color 1 at x=42 ahead of the round-clock 'PLAYING' check, since the round clock is present in many phases and shadows the specific signal), order cheapest-most-discriminating first, and document why each ordering exists. Phases that look structurally identical are often disambiguated by a single visual signature such as BORDER COLOR -- classify on the discriminating pixel (a neutral palette index for roster/summary panels vs the player's team color for the role-reveal panel) rather than OCR-ing/template-matching panel contents. To split one class into finer sub-views, classify by unique anchor signatures plus elimination (a 'YOU ARE' anchor for one panel, a 'ROUND' header for another; assign the remainder only by elimination, leave None when nothing confidently matches). Read the game server's RENDERING code, not just the player's perception code, for ground-truth pixel layout (a draw routine gave cell 16x18, max 8 columns, startX=(128-cols*16)/2, startY=42); HUD text-detection bugs hide as 'phase X never gets detected' with root cause a wrong (x,y) read position or wrong color filter (glyph OCR reads left-to-right and stops at the first non-matching glyph, so a wrong x or palette color silently returns an empty string). When restructuring a detector that returned early after parsing one case, move the early-return so classification runs first and field parsing stays gated to its panel. Ground heuristics in real recorded golden frame fixtures (128x128), not doc coordinates -- when doc and pixels disagree the pixels win -- and assert classification of all golden phase fixtures as a per-phase test gate (an aggregate accuracy number hides per-phase regressions). A phase the renderer produces but perception never detects is a latent player bug only cross-checking surfaces. _{sources: bitworld/among_them/GRAPHICS_REPORT.md, personal_labs/crewrift_lab/crewrift/crewborg/AGENTS.md, personal_labs/crewrift_lab/docs/crewrift-player.md, claude-code:fa645b7b-4fa0-4b55-992e-b273ee703391 (+9)}

6. Detect view/phase as an ordered priority cascade; gate interstitials by global black-pixel percentage and clean state on transitions

crewrift

Multi-phase games render radically different screens (lobby, role-reveal, play, voting, vote-result, game-over), so classify the PHASE first then dispatch to a phase-specific reader. Use an ORDERED disambiguating cascade (black-interior check, room-name prefix check) because views share pixel features (role-reveal vs info screen, chatroom vs global chat), and handle text VARIANTS explicitly (global-chat SHOUT variant, info-screen KNOWN variant, centered vs left-aligned game-over). Detect interstitials by GLOBAL black-pixel percentage >= 30% (the sim renders off-map gameplay space as MapVoidColor, not black; the older 'four corners black' test broke when chat/borders touched the corners) and gate them before any localization. On ENTERING an interstitial clear gameplay state and parse result/voting text; on LEAVING clear voting state, reseed localization from the remembered home, and clear stale path/goal/task-hold/velocity/jiggle because the sim teleports players home after voting. An agent can HALLUCINATE a view that never existed (a 'CHATROOM' view) -- validate the real view list against the reference implementation, not earlier agent assumptions, and scrap fabricated parsers entirely. _{sources: bitworld/among_them/players/SMART_BOT_GUIDE.md, bitworld/among_them/players/how_to_make_a_bot.md, opencode:ses_20b179f2fffeJe4vpbx2D4tWuR, opencode:ses_20c066070ffeBjXMnfQULai0Lw (+2)}

7. Detect the Persephone view from ordered fixed pixel probes, and give pending-entry its own enum

crewrift

Detect the current Persephone view from fixed pixel probes in a fixed ordered sequence (not a state field), because views share visual elements and order resolves ambiguity: check the double-border first (matching non-zero double border at (0,0) and (2,2) with black at (4,4) = role/roster/info reveal), then OCR header text at fixed offsets in specific palette colors ('R'...':' = playing, digit/digit = lobby, 'WHISP' = whisper, 'REVEAL' = reveal). Caveats: HUD titles are not always left-aligned -- the hostage-exchange screen renders its title centered at y=14, so detect it with a multi-x-offset scan; and the global-chat header suffix varies across versions (older 'CHAT', current 'SHOUT'), so accept both. Treat a waiting/pending-entry state as its own view enum even though it is mechanically the overworld with a special bottom bar, so the agent knows not to press buttons that would cancel the pending request, while still populating the still-extractable overworld spatial data. _{sources: players_checkouts/players/users/james/personal_cogs/persephone/GAME_AP, DESIGN_perception.md (Persephone perception module design)}

8. Read your own role/color from the fixed HUD corner sprite and center-spawn; make role/phase derivation STATEFUL and never latch a default on frame one

crewrift · negative result · ⚠ session-derived, unverified

Determine the viewer's role from the single corner HUD icon, not the player avatar: in Among-Them at screen (1,115) = SCREEN_HEIGHT - SPRITE_SIZE - 1, ghostIconSprite=ghost, killButtonSprite=living imposter (drawn shadowed when kill cooldown > 0), no icon=crewmate. Exploit deterministic game-start invariants for cheap self-identification: the server spawns the controlled player centered in the camera on init, so read your own color from the center sprite at game start. Cross-check every hardcoded HUD coordinate against the server render code AND a working reference bot (a comment claiming (109,110) 'must match' another bot was false; a wrong anchor makes the feature silently never fire). Phase and role derivation must be STATEFUL because downstream behavior modes branch on them and never activate if unset: a purely stateless derive_phase returns 'unknown' during ordinary play and sticks at RoleReveal, leaving crewmate Normal mode inert because it never sees Playing. Infer Playing once a reveal/meeting clears (or via the task counter on a mid-game join), and set self_role explicitly via the DEFAULT branch (derive 'crewmate' once in Playing with no imposter/ghost HUD marker present) -- omitting the alive/default branch means alive crewmates never get a role. Do NOT latch RoleCrewmate on frame one when no positive evidence is seen, because a broken detector or a not-yet-rendered HUD then permanently locks the wrong role; default only after N frames or derive role from the role-reveal interstitial classification, and reuse an existing debounce (e.g. the ghost-icon 2-frame threshold) rather than a 'not found -> assume X' branch. Enumerate every phase/role the deriver can output, test it transitions out of the initial/reveal state, and note role is not binary (a snapshot can carry role:'dead'). _{sources: bitworld/among_them/GRAPHICS_REPORT.md, bitworld/among_them/players/how_to_make_a_bot.md, personal_labs/crewrift_lab/crewrift/crewborg/AGENTS.md, claude-code:4430a077-5411-426b-a87a-75512ec4300f (+7)}

9. On the role-reveal screen, stamp durable facts and know the crewmate/imposter sprite asymmetry

crewrift

Parse the role-reveal interstitial text and stamp durable per-game facts: CREWMATE sets the crewmate role; IMPS marks self imposter AND records the teammate colors shown so they are never later treated as kill targets or suspects; reset ALL per-round state on CREW WINS / IMPS WIN. Critically, the same sprite-grid render means different things by role: the crewrift server draws ALL players for crewmates but only the imposter TEAM for imposters, so the role-reveal scan directly reveals teammates when you are the imposter but is a trap if you assume the sprite set is symmetric across roles. _{sources: bitworld/among_them/players/evidencebot_strategy.md, bitworld/among_them/players/how_to_make_a_bot.md, opencode:ses_204eb0641ffeIAD293yQHlSd61}

10. Localize the camera with a tiered cost ladder against a map cache, masking ALL dynamic pixels, with a tighter threshold for the local search; wander to lock

crewrift

Localize a screen-reading player's camera against a known static map by comparing the live frame, pre-hashed into 8x8 patch fingerprints, over a tiered cost ladder: (1) cheap sticky LOCAL search within ~8px of the last lock (~1ms) with a TIGHTER mismatch threshold; (2) patch-hash GLOBAL search where frame patches vote for the best (cameraX,cameraY) and top candidates are fully scored (~10-170ms, robust to occlusion/HUD overlay); (3) spiral full-frame fallback. Before scoring ANY candidate, mask out all dynamic pixels (centered local-player sprite, other crewmates, bodies, ghosts, task icons, radar pixels, kill-button icon, ghost status icon) or matching fails. Gate acceptance on errors<=maxErrors AND comparedPixels>=a minimum, and use a tighter threshold for the hinted local search than for brute-force: clean frames score ~5 mismatches but cluttered mid-game frames hit 50-90, and a slightly-drifted lock can still pass a loose fixed 100-mismatch gate, so a near-miss should be rejected not accepted as drift. A pixel-localizer needs fresh diverse map pixels to lock, so before lock drive the bot with raw cardinal-direction wandering (cycle up/right/down/left every ~36 ticks) rather than waypoint routing -- a stationary bot may see only a static crop that never matches uniquely; once locked but still bootstrapping, steer toward map centre to feed diverse pixels and move off the spawn edge. Fuse independent cues for a robust fix (e.g. the colored dot on the 20x20 minimap plus floor-grid dot alignment). Run sprite scans against the PREVIOUS frame's camera, then localize, and only re-scan against the new camera when the lock jumps beyond a teleport threshold (~32px). Keep landmark-based localization as the absolute anchor and use observed local-position displacement only as a correction layer when last-action timing is stale or no movement was observed -- do not replace landmark localization with displacement integration. _{sources: archive/cogames_playground/docs/plans/bop-it-localization-equip.md, bitworld/among_them/GRAPHICS_REPORT.md, bitworld/among_them/players/evidencebot_strategy.md, bitworld/among_them/players/how_to_make_a_bot.md (+5)}

11. Localize from the highest-signal source; treat minimap as a fog-immune fallback only

crewrift · ⚠ session-derived, unverified

Localize the agent's current position from the HIGHEST-signal observation source (direct camera/viewport, designed to be 1-2 tick and accurate) and NEVER estimate current position from a low-signal aggregate like the minimap; enforce 'current position is never estimated from minimap' on the perception side and treat coarse sources as best-guess fallback only. When fog-of-war limits direct view, the minimap is a PARTIALLY fog-immune source for persistent location estimates, but verify its limits in the renderer source first because coverage is COMPLEMENTARY, not full -- out-of-viewport players bypass the minimap shadow check entirely (the shadow buffer is only viewport-sized and the bounds check short-circuits before the shadow lookup). Minimap dots typically encode only COLOR, not identity, so identity must be inferred: when a known player is co-located with a dot, bind that player to the dot and track its movement to maintain identity across frames; model proximity (sharing a private channel) as a LIKELIHOOD of co-presence, not a guarantee. _{sources: opencode:ses_1fb148d00ffe5ob6k0yKT3aHke, opencode:ses_1fb48dd11fferLGXS1vWdHa0wS, opencode:ses_200bab964ffe5MjqDy3HjInDmF}

12. Match the Persephone 3x5 pixel font by filled/empty cells and outline color; re-verify glyph-identity claims

crewrift

Match a fixed pixel-font glyph by requiring target-color pixels to fall on the glyph's filled cells and non-target-color pixels on its empty cells, accepting at ~90% to absorb rendering artifacts. The 3x5 font supports only A-Z, 0-9, and :!?'.,-/*()<> and silently skips unsupported characters (#@%_+=[] and lowercase), so do not emit those in chat. Outline color is context-dependent -- Persephone renders outlines as color 0 (black) in HUD contexts (speech bubbles, pending-entry sprites, whisper header) rather than color 1, so a shape classifier needs an explicit outline_is_black flag to work in both overworld and HUD; reliable HUD detection (outline_is_black=True gave ~100% accuracy) can make a color-only id fallback unnecessary, but remove fallbacks only after verifying no regression on shared-color edge cases. NEGATIVE RESULT: do not trust a stale doc claim that distinct glyphs are pixel-identical -- the claim that S/5 and O/0 draw identically was wrong (the glyphs are distinct and OCR matches them unambiguously), so the parser's digit-normalization helpers (norm()) were demoted to no-ops; re-verify against the actual renderer bitmaps. _{sources: players_checkouts/players/users/james/personal_cogs/persephone/GAME_AP, players_checkouts/players/users/james/personal_cogs/persephone/TODO.md, DESIGN_perception.md (Persephone perception module design)}

13. Detect actors by anchored sprite matching with tint-only color voting, structural-pixel anchors, and shadow tolerance

crewrift · negative result · ⚠ session-derived, unverified

For detecting on-screen ACTORS, prefer template-anchored brute-force sprite matching (for every (x,y) anchor try the template with flips=[false,true], since sprites flipH when moving left and a one-sided matcher misses half the actors) over whole-frame palette-index histograms: histograms are fragile because UI/text pixels leak into counts (white menu text read as a 'white player'), and anchored matching needs no N-identical-frame stability gate (an interstitial can appear and vanish faster than the required streak). Every upstream Among Them bot (nottoodumb, evidencebot_v2, ivotewell, italkalot, mod_talks, modulabot) uses this identical scan, so adopt the convergent pattern. Among-Them sprites use wildcard tint pixels: color 3 (TintColor) is replaced by the player's color and color 9 (ShadeTintColor) by ShadowMap[color]; detect a player by matching the FIXED non-tinted structural pixels (outline, visor, backpack) then read the dominant tint-pixel color for identity. Let ONLY the template's tint/body pixels vote on color, never stable structural pixels (outline, visor, eyes), to sidestep palette collisions; and ensure each template has enough distinctive non-background stable pixels (~18 visor+eye) to anchor even when the body color equals the background. Distinguish chrome from identity by PIXEL COUNT, not presence: visor renders as palette 14 regardless of player but 14 is also a real body color, so a real tinted body (~40 px) vs visor (~10 px) needs a FIXED absolute margin (count > stableContribution + 20, ~half a body) not a proportional one that collapses at 8 sprites; calibrate any pixels-per-entity divisor to the true ~63 non-transparent pixels per sprite (an /100 estimate under-counted 2 as 1). Make matching shadow-tolerant by accepting each pixel if it equals the normal OR shadowed value (handles fully-lit, fully-shadowed, mixed sprites in one path), but when shadow_fill == shadow_outline the sprite collapses to a solid block -- restrict matching to normal colors then and return None honestly when genuinely ambiguous. Histogram-with-baseline is only for the narrower PRESENCE (not location) question, subtracting each sprite's known stable contribution. _{sources: bitworld/among_them/GRAPHICS_REPORT.md, opencode:ses_1f6f76557ffeCfmMTG7BPJwGRE, opencode:ses_1f7b6dc30ffeDmQtlUZI25HAjh, opencode:ses_20463f5e6ffeWi1ooj1agqH80a (+1)}

14. Decode identity from the unique (color, shape) pair, thread the new channel everywhere, and fuse complementary minimap/viewport coverage; report None honestly

crewrift · negative result

When an observable attribute has FEWER distinct values than the number of entities (8 colors but up to 24 players), identity from that attribute alone is ambiguous -- find a second independent attribute whose COMBINATION is unique (color x shape): lcm(8 colors, 12 shapes)=24 equals max player count, so if the product/LCM of value-space sizes is at least the max entity count and pairs are assigned by independent moduli, every (color, shape) pair decodes deterministically to one canonical index. Score shape templates against a 7x7 sprite with vectorized NumPy matching (matching/non-transparent pixels), accepting the best above a tunable threshold (default 0.70). When adding the new identity attribute, thread it through ALL downstream perception dataclasses in parallel with the existing one (add a shape field everywhere a color field exists: bubbles, chatroom occupants, exchange players, known players, message senders); missing one channel reintroduces ambiguity for that path. The minimap and viewport give complementary INVERSE coverage: the viewport gives full color+shape identity for nearby unobstructed players but cannot see far/behind obstacles, while the minimap reliably shows far-away players as color-only dots (they bypass the fog/shadow check since the 128x128 shadow buffer has no data for them) but hides nearby players behind walls. Minimap dots are color-only and up to 3 players share a color (index mod 8), so disambiguating a dot requires co-locating a viewport-identified player; the viewer's own dot is drawn last and overwrites others at its cell, so a closely-pursued target 'disappears' -- track lastSawTargetTick. Role indicators (pixel bars below sprites) reveal team and key-vs-grunt class, distinguishing the two roles within a key class only by dot position (center vs split), not color. Cap detected actors at the game's known maximum and reject scans that exceed it as noise (RoleRevealMaxDetectedColors = 3 discards any role-reveal scan returning more than 3 colors). A single-pixel sample is fragile (some shape templates have an outline/transparent center pixel) -- validate PER-TEMPLATE that any sampled pixel carries the expected signal and prefer scanning a region for the dominant signal. Make perception best-effort and honest: set fields to None on failure (fog, animation) and require agents to tolerate it -- when a player's shadow-fill color equals the shadow-outline color (blue and purple both map to palette 12), a fully-shadowed sprite's shape is genuinely unrecoverable, so report shape=None and let agents fall back to positional continuity. _{sources: players_checkouts/players/users/james/personal_cogs/persephone/GAME_AP, DESIGN_perception.md (Persephone perception module design), opencode:ses_1f6f76557ffeCfmMTG7BPJwGRE, opencode:ses_20463f5e6ffeWi1ooj1agqH80a (+2)}

15. Match task icons by bob phase and disambiguate the palette-9 / player-color collision

crewrift · negative result

Among-Them on-screen task icons (taskIconSprite) are palette index 9 and bob above the station on a 10-element cycle [0,0,-1,-1,-1,0,0,1,1,1] indexed by (tickCount//3) mod 10, so test all bob phases when matching at the 40 known task positions; off-screen tasks are a single yellow palette-8 pixel on the screen border (when showTaskArrows is on) -- reverse the player-to-pixel line for direction, matching radar within ~2px tolerance. NEGATIVE RESULTS: steering toward palette 10 chases map decoration, and a bare palette-9-cluster heuristic false-positives because blitSpriteOutlined tints players by replacing wildcard pixels with the player color, so a player whose color is 9 renders pixels indistinguishable from a task icon (and the walkability filter doesn't help, since players stand on walkable floor). Disambiguate real icons by multi-frame world-position stability (icons don't drift, players do), the +/-1px 3-tick bob, or sprite-template correlation -- never a global scan. _{sources: bitworld/among_them/GRAPHICS_REPORT.md, bitworld/among_them/players/lively_lecun/ROADMAP.md}

16. Resolve task ownership with authoritative signals and observable NEGATIVE evidence, latched against re-flagging

crewrift · negative result

Identify which signals are authoritative for ownership before trusting them: an on-screen task-icon match is authoritative (the server renders the icon only for YOUR tasks), an off-screen arrow/radar-dot is authoritative only if gated on a real radar match, and raw rectangle intersection (standing on a station) carries ZERO ownership info -- setting the arrow flag for every off-screen task or the active flag on any overlap destroys filtering. Infer assignment from graded evidence: a visible task icon is definite; a matched off-screen radar dot is probable (high-confidence single sighting); and accumulate negative evidence to mark a station resolved-not-mine after its icon is absent for several consecutive frames. Presence-only perception leaves most candidates unresolved (32 of 40 stations), so observing ABSENCE rules out the bulk -- model per-task state NotDoing -> Maybe(radar) -> Mandatory(icon visible) -> Completed, where seeing an icon always wakes a task and radar is weak evidence never proof. Crucially, only count absence you can observe: increment the task-icon miss counter ONLY while the icon area is fully on-screen with a small clear margin, never for off-screen/near-edge stations. The recurring bug is clearing a task by checking the standing RECTANGLE instead of the ICON area drawn ABOVE it; require the full icon rect visible with margin AND repeated absence for N frames. Exclude off-screen candidates with radar-ray geometry (cast a ray from the player through each screen-edge radar dot; if none intersects a task's padded icon box, exclude it this frame) but treat that exclusion as per-frame and reversible since movement changes alignment, unlike the durable resolved-not-mine flag. Sticky candidate flags cause loitering: DECOUPLE the clear condition from the set condition (don't let a stale/ambiguous radar dot veto a clean direct observation), add a per-target 'resolved' latch with exactly one override (a real icon re-rendering) reset each round, and only begin task-checking AFTER localization (which follows round-start assignment) so the latch doesn't fire before icons render. When matching a sprite to an expected object, compute the expected RAW screen position from the server's fixed render offsets plus ~2px tolerance and do not apply extra draw offsets the scan does not report. Reset all per-round belief state (slot states, checkout latches, resolved-not-mine flags, miss counters) on the role-reveal interstitial and skip belief updates entirely on interstitial frames. _{sources: bitworld/among_them/players/SMART_BOT_GUIDE.md, bitworld/among_them/players/evidencebot_strategy.md, bitworld/among_them/players/how_to_make_a_bot.md, personal_labs/crewrift_lab/crewrift/crewborg/design.md (+3)}

17. Drive the body-report reflex from position-based body identity, not a visible-body count

crewrift

Trigger the crewmate body-report reflex from position-based body identity (screen coords + camera offset), NOT a visible-body count: count-edge detection misses same-count corpse swaps (one body leaves view as another appears), whereas remembered-position unknown-body detection fires on a newly visible corpse even when the total count is stable. Match a visible body to a target within a generous radius (~30 world px) to absorb camera jitter and sprite-anchor offset, debounce single-frame misses by resetting the miss counter on any matching frame, and only give up after a sustained run of misses (~36 frames / 1.5s) indicating real despawn. Add a reflex cooldown (~96 ticks / 4s) plus known-body memory so the reflex does not re-fire on the same corpse while still allowing a different newly-seen body to trigger. _{sources: players_checkouts/players/users/james/personal_cogs/among_them/guided_, players_checkouts/players/users/james/personal_cogs/among_them/guided_, players/users/james/personal_cogs/among_them/guided_bot/coworld/README}

18. Drive the edge-triggered voting cursor through the parser's slot map, re-parsing every interstitial frame

crewrift

The voting UI is edge-triggered: it advances exactly one slot per fresh left/right keydown, so holding a direction moves only one step -- drive the cursor with a pulse-then-release state machine (hold a few ticks, emit a no-op release tick, recompute). Map a vote color target to its live slot through the perception parser's slot-to-color array, not slot==color, because the game may draw slots in live join order (fall back to slot==color only when no parser slot map exists). Re-run the voting parse on every interstitial frame, including after the phase is established, to keep cursor position and per-slot alive/dead state fresh; and when the parse fails and cursor index < 0, degrade gracefully to blind single-direction (right) navigation rather than stalling. _{sources: personal_cogs/among_them/guided_bot/MEETING_DESIGN.md}

19. Detect a real watched-task completion via the global HUD counter decrement plus a long dwell

crewrift

Derive a watched-task-completion (exculpatory) signal from the global remaining-tasks HUD counter: count a completion only when the counter decrements by exactly 1 WHILE exactly one visible living player ends a task dwell of >=56 ticks. Fake 'pretend' holds never decrement the global counter, so they cannot trigger the signal. _{sources: personal_labs/crewrift_lab/WORKING_CONTEXT.md}

20. Detect game events from genuinely-consecutive frame diffs with robust structural cues

crewrift · ⚠ session-derived, unverified

Detect events from consecutive-frame diffs, requiring the two compared frames to be genuinely consecutive and appending tape frames only on camera-ready frames (so a meeting's tick gap is self-protecting -- a detector cannot compare across it). Among-Them signals: not-interstitial -> interstitial = meeting or game-over; 0 -> >=1 bodies = body discovered; kill icon invisible -> visible = kill-cooldown ready; N -> N-1 visible crewmates = player went missing; ejection text appearing = ejection; chat naming a color = accusation. Prefer robust STRUCTURAL cues (region pixel counts, color-class ratios, blob detection) with wide threshold margins over exact-template or per-pixel comparisons, so cosmetic rendering changes don't break the agent; full OCR is overkill for most cues. Use physically-impossible-otherwise observations as cheap phase ground-truth: seeing seven players in one snapshot (the camera normally sees 1-2) reliably signals a meeting room, and a phase classifier that misses it skips all voting and casts zero votes despite seeing 7 players. A perception-driven player only knows the game ended if it detects the game-over screen (game_over is emitted solely on a PhaseGameOver transition in the rendered frame), so a slot whose connection was severed mid-task never emits game_over and ends on an ordinary task_started -- 'no game_over in the trace' reliably means the connection was cut before the game ended for that slot. _{sources: bitworld/among_them/players/SMART_BOT_GUIDE.md, bitworld/among_them/players/lively_lecun/ROADMAP.md, personal_labs/crewrift_lab/crewrift/crewborg/design.md, claude-code:5f9e8287-da92-4dff-bf39-2511bcaf0700}

21. Recover chat from pixels with a structured parser and color-pip speaker attribution

crewrift

When the game gives only the rendered framebuffer (all players see the OCR-readable voting screen), build a STRUCTURED frame parser (parseVoteFrame returning {colorIndex, lines} plus a derived 'who is being called sus' color) and feed concatenated text to the LLM, preferring it over ad-hoc pixel OCR once it exists. Attribute speakers by sampling the per-message color PIP at a fixed x-column immediately left of the chat text (VoteChatIconX=1), taking the dominant color with a 'prefer-above' tie-break, because naive nearest-pip mis-credits WRAPPED multi-line messages to the next speaker (wrapping is the edge case that breaks the obvious approach); return unknown/-1 on low confidence. This perception unlock has the widest downstream blast radius -- once speaker attribution worked, imposter-chat detection and vote-bandwagon correlation became feasible -- so prioritize it. The meeting/voting UI is the AUTHORITATIVE alive/dead census and vote record (Crewrift voting grid ids 9300+seq show 'player ' alive vs 'body ' dead, vote-result id 9600 names the ejected, 'NO ONE DIED' is a distinct sentinel); attribute votes via per-target vote dots, but the REPORTER needs a dedicated meeting-intro pass and must stay a -1 sentinel (not a fabricated value) until that pass exists. _{sources: bitworld/among_them/players/mod_talks/LLM_SPRINTS.md, bitworld/among_them/players/mod_talks/TODO.md, bitworld/among_them/players/modulabot/DESIGN.md, bitworld/among_them/players/modulabot/TODO.md (+4)}

22. Disambiguate overloaded Crewrift object-id bases by stream and value range; loop the whole packet and special-case skip-vote dots

crewrift · negative result · ⚠ session-derived, unverified

Crewrift object-id bases are overloaded across streams and value ranges, so assigning meaning by id-base alone misclassifies objects. (1) id base 7000 = player-name labels on the global/spectator stream (gated on showPlayerLabels) but = task arrows on /player, so confirm which stream the player reads first. (2) Vote dots split by range: normal vote dots are 10100 + target*16 + voter (bounded < 10100 + MAX_PLAYERS^2) while skip votes use a high base (observed 10400 + voter) sharing the same sprite, so decoding everything >=10100 as a normal dot yields bogus target ids -- split by range and represent skip as a distinct sentinel (e.g. target=-2). When decoding a binary websocket message, loop an offset across the WHOLE packet: one message can carry multiple concatenated sub-messages and reading only the first silently drops state. _{sources: claude-code:4430a077-5411-426b-a87a-75512ec4300f, codex:019e2886-d576-7770-be1b-6d92ca97761e, codex:019e66c3-638e-7e70-afdf-8f6df9299f6a, codex:019e7174-3079-7351-b8ea-b1bfcb917ff4}

23. Bake Crewrift static geometry from the server resource file at startup; track last-seen; treat task arrows as tri-state

crewrift · ⚠ session-derived, unverified

Bake the Crewrift static map (vent, emergency-button, room, task coordinates) from the server's resource file at startup, because those coordinates are NOT recoverable at runtime from the Sprite-v1 stream -- they live only in rendered map-sprite pixels and the walkability mask, and no message or HTTP endpoint serves them (server exposes only /healthz, /admin, /control/restart, /control/kick, replay/global client files, and the /player websocket). The resource format is line-based CSS-flavored (not brace-delimited): each rect block starts with a /* name / comment plus px-suffixed width/height/left(=x)/top(=y) and background: rgba(...); the parser flushes at the next / name / and commits only rects with name+x+y+w+h and w,h>0 (so a leading un-colored / vents */ bounding block is dropped). Classify at startup: 'task' rects -> task list in file order (matching the 3000/7000+idx stream index), 'ventN' rects -> grouped by trailing digit, named rects -> rooms, emergency button is a derived 28x34 rect centered on 'bridge' and clamped. Store the initial server packet's full contents (all task/event locations, room names) for later, since room names are needed to describe body locations during voting ('Body found around hydroponics'). Validate the vendored bake against the live walkability mask with a one-time fail-loud check the first time the alpha arrives (catches stale-bake mismatch); the walkability alpha is raw-snappy compressed, decode via cramjam. An entity's absence from a per-tick percept array (visible_players/visible_bodies) means 'not currently in my vision', NOT 'gone' -- track last-seen state and observability separately. Two per-incomplete-task finding signals exist: a 'task bubble' object (id 3000+idx, exact world position, on/near screen) and a 'task arrow' object (id 7000+idx, bearing only, off-screen, only if showTaskArrows is enabled); map the stream index to a world rect via the file-order task list. Detect whether arrows are enabled by OBSERVATION not assumption (the arrow sprite is always defined at init but arrow OBJECTS may be gated off): treat the flag as tri-state -- True on the first arrow object for a known off-screen task, False if several ticks pass with off-screen tasks and no arrow -- and when off, fall back to a room-by-room map sweep. When deriving evidence from positions, reuse this static map data (e.g. vent waypoints already in the nav graph) instead of adding a new perception pass. _{sources: coworlds/coworld-crewrift/players/notsus/README.md, personal_labs/crewrift_lab/crewrift/crewborg/AGENTS.md, personal_labs/crewrift_lab/crewrift/crewborg/design.md, claude-code:4430a077-5411-426b-a87a-75512ec4300f (+3)}

24. Decode the input/observation protocol from the canonical shared module, including sentinels, the clock, and set-vs-increment semantics

crewrift · ⚠ session-derived, unverified

Decode the input/observation protocol exactly from the canonical shared module before writing a policy, including reserved sentinel values, and do not infer bit layout from usage sites: Crewrift input is a bitfield (one bit per button) where mask 255 (0xFF) is reserved as a game-reset signal, so a bot emitting arbitrary masks can trigger reserved behavior. Watch for path drift -- authoritative protocol/constant definitions (input bits, screen dims) lived in a shared module outside the game dir and a Go port redefined the button bits independently, so treat the original shared module as source of truth and verify other ports have not drifted. Before driving the agent to any target screen, map the simulator state machine from source: the phase enum, every transition with its trigger (timer expiry vs player action vs server event vs player-count threshold), the tick rate, and each timer's value in ticks. Audit whether the player has any authoritative clock: a Crewrift policy's only time sense was a local scene.tick incremented per received frame, with nothing on the stream carrying a server tick/timestamp, so every tick-based threshold (vote budget, kill cooldown, flee hysteresis) silently drifts undetectably unless the stream is exactly one-message-per-tick -- correctness reduces to verifying there is no server-side frame coalescing/dropping, which you must check, not assume (note: the invisible 'tick ' marker sprite id 5016 does carry a server tick when present, so prefer it). Likewise read the engine's mutation/handler source for set-versus-increment semantics before baking economy/survival math: energy regen ADDS the solar value as a per-tick delta rather than SETTING energy to solar (16 energy + solar 3 -> 19, not 3), and a wrong assumption silently corrupts strategy. _{sources: claude-code:27984094-3b9e-4e67-92c5-da2a61ceb32a, claude-code:3837ef5e-3e9c-41bc-b290-df5865581698, claude-code:ebd6cd36-8956-4a1e-b039-841683ff7792}

25. Order crewrift events by (real tick, seq, key) and mark every estimated tick explicitly

crewrift

Sort crewrift events by the canonical key (ts, seq, key) where ts is the real game tick (the 0x01 frame tick id, a monotonic u32 with 0 = pre-spawn/lobby and gameplay starting at gameStartTick) and value.seq tie-breaks events sharing one tick (votes, multi-kill ticks). For two-party events like kills, put the actor (killer) in the player column and the other party in value.subject_player (victim), so 'how did slot N die' is action.kill rows where subject_player == N. Stdout lines are not individually tick-stamped, but during discussion/voting the map freezes while the replay advances as dense consecutive ticks (>=60) -- detect meetings that way and stamp vote/chat events inside [meeting.start, meeting.end], or at meeting.start ordered by seq if only the meeting number is known. The current reporter reverse-engineers ticks by regex-parsing game.stdout.log and interpolating between replay anchors, so most events carry tickEstimated:true and actor/action columns are null; always flag estimated ticks and never present one as real -- a true events.parquet would let the reporter set tickEstimated:false and populate actor/action. _{sources: coworld-source-repos.crewrift-upload/optimizers/instructions/mvp/repor, optimizers/docs/events-parquet-spec.md}

26. Parse crewrift meeting chat into (speaker,stance,target) triples, separate kills from ejections, and mirror the training transform at runtime

crewrift

Reduce templated crewrift meeting chat to (speaker, stance, target) triples: match color names with a per-game regex ordered longest-color-first (so 'pale blue' beats 'blue'), use keyword hints to classify accuse vs defend, ignore lines naming only the speaker, and drop unparseable lines rather than guessing. Distinguish kills from ejections by recording each kill's (tick, victim) so a later bare 'died' event with no matching kill is attributed to the open meeting's ejected slot rather than miscounted as a murder. At inference, the runtime suspicion scorer must mirror the training transform contract exactly -- dot the fit coefficients with features, add the intercept, apply a sigmoid, and apply the bin_spec boundary indicators (e.g. follow_death_samples bins [2,6], observed_samples bins [10,40]) so runtime binning matches how the model was trained. _{sources: personal_labs/crewrift_lab/crewborg/data/suspicion_weights.json, personal_labs/crewrift_lab/suspicion_lab/tools/features.py, personal_labs/crewrift_lab/suspicion_lab/tools/replay_parse.py}

27. Use crewrift's fixed slot-to-color palette and pin self color from the supplied slot

crewrift

Crewrift's slot-to-color palette is fixed and validated (0=red,1=orange,2=yellow,3=light blue,4=pink,5=lime,6=blue,7=pale blue,8=gray,9=white,10=dark brown,11=brown,12=dark teal,13=green,14=dark navy,15=black); cross-check stdout imposter colors against results.imposter[] (pink+pale blue -> slots 4,7) to confirm all identity namespaces agree. When Coworld supplies a slot (slot= in COWORLD_PLAYER_WS_URL or --slot), pin self color from that slot rather than reading it from noisy vote-screen markers, which makes meeting context and self-vote filtering unstable. _{sources: players_checkouts/players_2/players/crewrift/suspectra/README.md, optimizers/docs/events-parquet-spec.md}