| name | rerun-viewer-validation |
| description | Prove what the Rerun viewer rendered โ pixel evidence over logs. Use when a .rrd, blueprint, or Rerun rendering must be visually verified, when a timeline sweep or video of a recording is wanted, when an .rrd must be embedded in an HTML page, or when a gradio/WebViewer surface needs browser validation. |
| compatibility | Requires the rerun binary (with the viewer-mcp subcommand) and ffmpeg on PATH, plus a GPU or software rasterizer for headless rendering. The web branch additionally needs a browser automation tool (playwright or chrome-devtools) and network access. |
Rerun Viewer Validation
Prove what rendered. Logs, metadata, and rrd stats say what was sent; only pixels say what the viewer shows.
Decision tree โ pick by what you're validating
- Static render proof (does the .rrd load? does the blueprint lay out? do views render?) โ scripted ViewerClient. No MCP needed, deterministic, CI-friendly.
- Time or UI state (scrub to frame N, verify view X at time T, click/select entities, read panels) โ viewer MCP.
- Video / timeline sweep (watch an algorithm run) โ
scripts/rrd_to_video.py (this skill's helper). Never loop MCP screenshots for video.
- Web (embed an .rrd in an HTML page; validate a gradio-rerun app or WebViewer embed) โ read
references/web.md โ the iframe embed recipe (CORS, Chrome Local Network Access, tailscale) and the playwright validation recipes live there. The MCP structurally cannot reach a WASM viewer in a browser โ no gRPC server to dial.
Video vs embed (branches 3 vs 4, when both fit): the embedded rrd is the richer artifact โ fully inspectable, orbitable, scrubbable โ so prefer it when the recording is browser-sized. The WASM viewer holds the whole recording in memory, so check size first (ls -lh, rerun rrd stats) and gate embeds at a few hundred MB (hard ceiling ~1.5 GiB โ see references/web.md). Choose video when the recording is huge, when the audience only needs to watch (Slack, PR description), or when the data needs a visualizer the web viewer doesn't have (custom visualizers). Best of both, often: a trimmed/downsampled preview rrd for the embed plus a full-fidelity video.
Version rule for every branch: the viewer that validates must be โฅ the SDK that wrote the data โ Rerun has no forward compat, so an .rrd written by a newer SDK will not load in an older viewer (including the WASM viewer inside a gradio-rerun app pinned to an older release). Don't hardcode version numbers in prose or scripts; the environment's package pins (this skill's run_constraints) guarantee a capable viewer.
Headless vs headed
Default headless for both ViewerClient.spawn(headless=True) and rerun --headless:
- Renders real frames offscreen (1920ร1080 default) given a GPU or a software rasterizer (lavapipe). In a bare container with no Vulkan adapter it panics with "No graphics adapter was found" โ then fall back to the browser branch.
- No OS window โ immune to the occluded/minimized-window failure (MCP
screenshot times out if a headed window can't render, notably on macOS).
- Works over SSH/CI/no-
DISPLAY. A headed spawn without DISPLAY wedges silently (channel fills, rr.log blocks forever).
Go headed only when a human co-views: the user wants to watch you scrub, or wants the viewer left open afterwards. All tools work identically against either โ headed vs headless is user preference, not capability.
Lifecycle gotchas (both modes):
- The MCP never spawns a viewer. Always: spawn viewer โ
connect โ work.
ViewerClient.spawn resolves rerun from PATH โ stale global installs win. Always pass executable_path= pointing at the project env's rerun binary.detach_process defaults: headless โ attached (dies with your script / close()); headed โ detached (survives; only explicit close() kills it). Clean up detached viewers when done.
MCP: getting the tools
The server is rerun viewer-mcp (stdio); it dials a running viewer's gRPC ViewerControlService. In order of preference:
mcp__rerun__* tools already in your surface โ use them.- No tools, no restart possible โ drive the server over stdio yourself: newline-delimited JSON-RPC (
initialize โ notifications/initialized โ tools/call); reuse McpStdioClient from scripts/rrd_to_video.py.
- Register for future sessions:
claude mcp add rerun -- <env>/bin/rerun viewer-mcp (or codex mcp add โฆ). (The viewer-mcp subcommand exists since 0.34.)
- Delegating to a different agent CLI (e.g.
claude -p --mcp-config โฆ from a non-Claude harness) crosses a provider boundary โ confirm with the user first.
MCP: driving the viewer
17 tools: connect, disconnect, viewer_state, set_time, open_url (rerun-specific) + query_tree, get_node, screenshot, click, drag, hover, scroll, press_key, type_text, resize, wait_for, batch (egui UI, accessibility-tree based). Work observe โ act โ verify.
connect takes endpoint: "http://127.0.0.1:<port>" โ plain http, not the SDK's rerun+http://โฆ/proxy URL.open_url loads recordings: absolute file path (no file:// prefix), rerun:// dataset URI, or https URL.viewer_state first, always: recordings + per-timeline {timeline, type, min, max} + current time. Choose the timeline from this data, never by assumption.set_time: time is a sequence index for sequence timelines, nanoseconds for duration/timestamp timelines. play: true to run from there; default stays paused.screenshot always returns the PNG inline into context; save_path writes to disk in addition. Budget โค ~10 MCP screenshots per validation โ seeing evidence frames is the point; sweeping is the helper's job.- Prefer locators (
id from query_tree, role/label_contains) over raw pos; everything is in logical points (screenshot pixels at pixels_per_point: 1.0 align 1:1 with click coordinates).
batch chains act+observe (e.g. set_time โ screenshot) in one round trip.
ViewerClient: scripted static proof
import rerun as rr
from rerun.experimental import ViewerClient
with ViewerClient.spawn(
headless=True, port=9877, hide_welcome_screen=True,
executable_path="<env>/bin/rerun",
) as viewer:
rr.init("rrd_check", default_enabled=True, strict=True)
rr.connect_grpc(url=viewer.url)
rr.log_file_from_path("recording.rrd")
rr.get_global_data_recording().flush(timeout_sec=30.0)
import time; time.sleep(3.0)
viewer.save_screenshot("native-full.png")
Prefer save .rrd โ reload โ screenshot: it validates serialization, blueprint, and viewer loading in one pass. ViewerClient has no time-cursor setter โ the playhead lives in the MCP (set_time) only.
Per-view capture works only for views the viewer is currently rendering. The safe pattern is authoring the blueprint in-process: view = rrb.Spatial3DView(โฆ); rr.send_blueprint(view); viewer.save_screenshot(p, view_id=view.id) โ returns in milliseconds. The trap: a view_id the viewer can't resolve to a rendered view (an unknown uuid, or a saved-blueprint view right after replaying an .rrd) gets no reply and the blocking call hangs forever, with no diagnostic on 0.34.0. So always run view_id calls in a killable child process with a timeout, and for replayed recordings prefer cropping the full screenshot (view rectangles are deterministic for a fixed viewport). To enumerate a recording's saved views (their /view/<uuid> ids are the same namespace as view.id, but resolve only while rendered):
import rerun.experimental as rrx
r = rrx.RrdReader("recording.rrd")
for chunk in r.stream(store=r.blueprints()[0]).to_chunks():
if str(chunk.entity_path).startswith("/view/"):
print(chunk.entity_path, chunk.to_record_batch())
Video: timeline sweep to mp4
python scripts/rrd_to_video.py --rrd recording.rrd --out sweep.mp4 \
--rerun-bin <env>/bin/rerun [--timeline frame] [--frames 150] [--fps 15] [--collapse-panels]
Spawns a headless viewer, drives rerun viewer-mcp over stdio (set_time โ screenshot save_path per frame โ zero agent context), ffmpeg-encodes. 120 frames at 1080p โ 10 s: the per-frame cost is the settle wait plus a ~32 ms screenshot RPC, so --settle-ms is the speed/fidelity dial. Auto-picks the first non-log_time timeline; handles sequence and temporal timelines (--frames samples evenly across the range); stdlib-only โ needs just ffmpeg on PATH and the project env's rerun binary. The default --settle-ms 30 is enough for decoded video frames; raise to 100โ400 when overlay-heavy views (detections, segmentation) must fully stabilize per frame โ a mostly-duplicate sweep fails loudly with that advice (--allow-static overrides for genuinely static scenes). Verify 2โ3 sampled frames visually (Read start/middle/end PNGs with --keep-frames) before trusting the mp4.
Panel visibility
Collapse the blueprint/selection/time panels whenever the frame should be all content โ videos, embeds, clean screenshots:
- Live viewer, any recording: the top bar has one labeled toggle per panel; MCP
click with label_contains = "Blueprint panel toggle", "Time panel toggle", "Selection panel toggle". A fresh viewer starts with panels expanded, so one click each collapses; confirm via query_tree (the _streams_tree / _selection_panel panes disappear). The video helper does this for you: --collapse-panels.
- Recordings you author โ and therefore embeds, since panel state rides the saved blueprint:
rrb.Blueprint(<views>, collapse_panels=True), or per-panel rrb.BlueprintPanel(state="collapsed") / rrb.SelectionPanel(โฆ) / rrb.TimePanel(โฆ) with "collapsed" | "hidden" | "expanded". An .rrd re-saved this way opens chrome-free everywhere, including the WASM viewer iframe.
Evidence & checks
- Reports under
/tmp/rerun-viewer-validation/<timestamp>/: screenshots, notes.md recording Rerun version, command, .rrd path/size, chosen timeline + range, wait times, renderer string, pass/fail.
- Blank or wrong visuals โ inspect data before blaming blueprints:
rerun rrd verify|stats|print <file> (use the project env's binary).
- Keep viewport fixed; wait after load and after each time change. For encoded video streams, a moved playhead proves nothing about decode โ only nonblank, changing pixels do.
- Remote viewing (optional):
tailscale serve --https <port> --bg <report-dir>, pick an unused port, curl -k -I the URL to confirm reachability. Path mode is fine for plain HTML + screenshots; a report that embeds an .rrd needs the CORS proxy setup in references/web.md.
Docs
