capturing-product-media

name

capturing-product-media

description

Regenerate DorkOS product screenshots and video loops from the real UI, and manage the shot registry, human overrides, and version archives that feed the marketing site, docs, and changelogs. Use when marketing/docs media is stale after UI changes, a new feature or docs page needs a money shot, a person wants to override an automated capture, or a release calls for a fresh capture or archive — anything under apps/site/public/product/ or the apps/e2e/capture pipeline.

Capturing Product Media

DorkOS product media is a general system: one shot registry (apps/e2e/capture/shots.ts) feeding the marketing site, docs, and changelogs, from real UI rendering seeded demo data — never mockups, never doctored screenshots. Automated captures can be beaten by human overrides, and any published set can be frozen into a versioned archive. This skill is the map: the registry, the phases, overrides, archives, docs embeds, the honesty rules, and how to add a shot.

When To Use

• The UI changed and a screenshot or loop on the site or docs now looks stale or wrong.
• A new feature needs a "money shot" (hero still or animated loop) on /features, or a docs page needs an embedded <ProductShot>.
• A person wants to override an automated capture with hand-captured media.
• A release calls for a fresh capture or a frozen archive of the shots its notes embed.
• A media-guard test (features.test.ts or shots.test.ts) fails because an asset or registry entry is missing.

Commands

pnpm --filter @dorkos/e2e capture                  # record + process (the full refresh)
pnpm --filter @dorkos/e2e capture:record           # boot + drive + save raws to the library
pnpm --filter @dorkos/e2e capture:record --shards N # same, but record on N parallel isolated stacks
pnpm --filter @dorkos/e2e capture:process [run-id] # edit raws + apply overrides → product/ (default: latest)
pnpm --filter @dorkos/e2e capture:archive <label>  # freeze the current published set under archive/<label>/
pnpm --filter @dorkos/e2e test                     # unit tests: registry, partition, aspect guard, override discovery

--shards N (parallel record) splits the shots across N fully isolated stacks — each its own DORK_HOME (~/.dork-capture for shard 0, ~/.dork-capture-<i> after), server/Vite port pair (4344/4343 + i×10), and browser — recorded at the same time, then merged into one run. The process phase is unchanged: a serial run and a sharded run produce the same published asset set. Default is 1 (the serial path). Reach for it to shorten a full re-record when the box has spare cores; 2 shards is the reliable win, and returns fall off past that once per-shard boot+seed overhead dominates. --shards also works on capture (only the record phase parallelizes). Details: apps/e2e/capture/README.md (Parallel capture).

capture is fully reproducible. It runs both pipeline phases:

1. Record: wipes an isolated ~/.dork-capture home + an offline marketplace cache, boots a test-mode DorkOS server + Vite client on isolated ports (4344/4343, no real Claude/Codex/OpenCode credentials), seeds a deterministic fleet + tasks + sessions through real API/code paths, drives the real UI through every money state, and saves RAW recordings/screenshots into the media library. Shots flagged skipAuto (an override supplies them) are not driven.
2. Process: edits the raws (head-trim, end-seam crossfade, two-pass encode, poster extraction), applies human overrides on top, and writes apps/site/public/product/ + a v2 manifest.json, then tears down.

Before running, confirm ports 4344/4343 are free, and that the workspace packages are built (pnpm --filter "./packages/*" build) — a stale/missing @dorkos/shared (or other) dist stops the capture client from booting. The run takes several minutes (it builds server deps, boots, drives ~16 shots, and two-pass-encodes the loops).

The shot registry

apps/e2e/capture/shots.ts is the single source of truth. Each shot has an id, a kind (still or loop), a frame (desktop/mobile), and consumers (marketing / docs / changelog). The process phase writes the registry into manifest.json (shots); the site reads it (apps/site/.../marketing/lib/shots.ts) so the marketing ProductSurface union + LOOP_SURFACES and the docs <ProductShot> embeds stay consistent (guarded by shots.test.ts). Add a docs- or changelog-only shot by listing it with the right consumers — it never appears on /features unless tagged marketing. (kind is two-valued: every loop already ships a still, so there is no both.)

Human overrides

Drop hand-captured media in apps/e2e/capture/overrides/<shot-id>/ (still-light.png and/or loop-dark.{mp4,mov,webm,mkv}, optional override.json with reason/capturedBy/date/skipAuto) and it beats the automated capture. Overrides run through the same optimization path and are scaled to the shot's target dimensions; an aspect-ratio mismatch fails that shot loudly rather than cropping. Re-run capture:process to apply. Full workflow: apps/e2e/capture/overrides/README.md.

Archive (frozen versions)

capture:archive <label> snapshots the published set under archive/<label>/ (immutable). A docs/changelog embed of a past version points at /product/archive/<label>/…; current embeds use the live path. Archive only the shots a release's notes embed (--shots a,b,c) — archives are committed binaries, so keep them minimal.

Media Library and Editing Workflow

The pipeline behaves like an organized video editor: raws and deliverables never mix.

• Raws live in apps/e2e/capture/library/<run-id>/raw/ (gitignored; only library/README.md is committed) — untouched Playwright webms and unoptimized screenshots, next to a run.json recording provenance: capture settings, the app's git SHA, content hashes of config.ts + demo-scenarios.ts, and each loop's head-trim marker. A latest symlink points at the newest run.
• Processed deliverables live in apps/site/public/product/ (committed); manifest.json carries a runId field tying them to their source raws.
• Two-phase commands: capture:record (boot + drive + save raws), capture:process [run-id] (edit raws into the published set; defaults to latest), capture (both).
• Re-record vs re-process: UI changed, new surface, or seed-data change → capture:record then process. Editing/encode change only (trim, seam, bitrate, poster logic) → capture:process — no app boot, no re-shoot.
• Retention: the last 3 runs are kept; older runs are pruned automatically at the end of each record run (reported in the run output).

Architecture (files in apps/e2e/capture/)

• shots.ts — the shot registry (source of truth): ids, kinds, frames, consumers, target dimensions, the manifest snapshot projection, and partitionShots (round-robin shot assignment for parallel records; the session-list shots multi-session/mobile-sessions and agent-discovery are pinned to shard 0 via SHARD_0_PINNED_SHOTS).
• config.ts — ports, viewports, library/output/archive/overrides paths, and all deterministic demo data (fleet, tasks, pinned runs, sessions, prompt pool, canvas doc, discovery projects, marketplace registry). Everything a shot shows is pinned here; nothing depends on Date.now().
• boot.ts — builds server deps once (buildServerDeps) and spawns/tears down the test-mode server + Vite client on this shard's ports, with an isolated DORK_HOME and a directory boundary confined to the capture world. Process-group teardown (teardownAll) leaves no orphaned ports.
• seed.ts — pre-boot filesystem prep + post-boot API/DB seeding, all through real code paths.
• record.ts / process.ts / capture.ts / archive.ts — the entry points (record raws, process raws + overrides, both, freeze an archive). record.ts also orchestrates a --shards N parallel record: partition, spawn workers, merge.
• record-shard.ts — one parallel-record worker: preps its isolated filesystem, boots its own stack, and captures only its assigned shots into the shared run (spawned by record.ts, never run by hand).
• library.ts — the media library: run recorder (raw sink + run.json provenance), latest symlink, retention pruning, run loading.
• overrides.ts — human-override discovery, validation, and application (manual media beats the automated capture).
• lib.ts — shared Playwright plumbing: the theme init-script, the live-turn opener, the raw loop recorder with its head-trim marker, and the attemptShot skip guard.
• surfaces-desktop.ts / surfaces-mobile.ts — the per-surface drives (one drive* shared between a still and its loop; one shoot*/capture* that waits for the money state).
• optimize.ts — the editing stage: PNG recompression + aspect-validated scaling (sharp) and loop editing (bundled ffmpeg-static): head-trim, end-seam crossfade, two-pass VP9, poster extraction, then the v2 manifest writer.
• overrides/ — committed human-override sources. __tests__/ — unit tests for the pure logic.

Test-mode seam

Demo scenarios (paced streaming, tool approval, file-backed canvas, sub-agent fan-out) live in apps/server/src/services/runtimes/test-mode/demo-scenarios.ts, inside the test-mode runtime boundary, reachable only when DORKOS_TEST_RUNTIME=true and selectable only via POST /api/test/scenario. They emit standard stream events through the exact normalizer → projector → SSE path a production runtime uses, so the client renders real components against real stream data.

Manifest contract (v2)

The site consumes assets through ProductFrame (apps/site/.../marketing/ui/ProductFrame.tsx), which resolves files by convention: <id>-light.png (still) and, for loop shots, <id>-dark.webm + <id>-dark.png (the poster); loop-ness comes from the registry (shotHasLoop). Docs embed the same assets through <ProductShot id="…" />. manifest.json (schema v2) carries the registry (shots) and, per asset, its dimensions, size, (loops) duration, source (auto/manual), capturedAt, and either the source runId (auto) or override provenance (manual). Two guard tests pin it: features.test.ts (catalog media exists + framing) and shots.test.ts (registry ↔ LOOP_SURFACES consistency, manifest v2, and every docs <ProductShot> id resolves with its files present).

Honesty Rules (non-negotiable)

• Real UI + seeded data only. Every pixel is the actual app rendering data seeded through a real API/code path. No mock components, no DOM doctoring, no editing pixels in.
• Skip-and-report over faking. Each drive is error-isolated (attempt): if a surface can't reach its money state, it is skipped and logged — never faked. A clean run has zero skips; investigate any skip before shipping.
• Demo-claim gate. Only market what is genuinely working. If a surface or pillar is unverified, do not stage a shot that implies it works (see the GTM plan's demo-claim gate).

Staging Knobs

• Theme, flash-free. Playwright records from page creation, so a dark loop must be dark on frame one. seedThemeOnContext (lib.ts) registers a context.addInitScript that sets localStorage['dorkos-theme'] and the dark class on documentElement before any page script runs. Never set the theme after navigating — that puts a light boot frame on film.
• Wide right panel. The personality radar reads better with a generous right column: the drive seeds the exact JSON react-resizable-panels persists (WIDE_RIGHT_PANEL_LAYOUT in surfaces-desktop.ts). The canvas panel width is seeded server-side via the demo scenario's preferredWidth (demo-scenarios.ts) — bump it if the editor wraps too aggressively.
• Scan-boundary confinement (privacy). The onboarding discovery scan can auto-start before config resolves and sweep the server's directory boundary. That boundary is pinned to the capture world (DORKOS_BOUNDARY in boot.ts), and scanRoots points only at the seeded projects tree (PROJECTS_ROOT in config.ts). Never let discovery scan the operator's real home directory — the lesson from the privacy incident. Keep the boundary and scan roots inside ~/.dork-capture.
• Deterministic timestamps. Pinned run history uses fixed times (e.g. 2:47 AM) and a single realistic failure, so the ledger never churns between runs. Add new demo data to config.ts with pinned dates, never Date.now().

The Editing Stage (loops)

writeLoop (optimize.ts) edits every raw loop with ffmpeg-static:

1. Head-trim to the action. recordLoop records a marker into run.json: drives whose motion happens inside the drive (personality morphs, canvas typing) call mark() at the content start; drives that build a state and then hold default the trim to drive completion. ffmpeg cuts everything before the mark.
2. End-seam crossfade (~300ms). The clip opens clean — no blend on film — and its own first ~300ms of footage fades IN over the final ~300ms, so the literal last frame equals the literal first frame and the restart is invisible. Seam placement matters: a head-placed blend makes the video's opening read wrong (founder-reviewed lesson).
3. Two-pass VP9 targeting ~1.35MB (under the 1.5MB budget), no audio, normalized to the surface's dimensions — two passes because average-bitrate hits the target regardless of scene complexity.
4. Poster extracted from the loop itself — frame 0 by default (invisible poster→video handoff), or the last clean pre-seam frame for posterFrame: 'settled' shots (loops that open on streaming text or a loading skeleton; the poster doubles as the still that release notes and docs embed). No separate dark screenshots are taken.

Two hard-won ffmpeg rules live in this filter graph: the seam overlay MUST be windowed (enable='between(t,dur-cf,dur)' + eof_action=pass) — ffmpeg's default eof_action=repeat composites the overlaid segment's last frame over the ENTIRE clip, producing full-duration double-exposure ghosting; and the VFR Playwright source MUST be fps-normalized before frame-accurate trims/fades, or a 300ms window can hold a single barely-faded frame.

Deterministic and idempotent: same raw + markers → same webm + poster.

Adding A New Shot (end-to-end)

1. Register it. Add the shot to SHOTS in apps/e2e/capture/shots.ts with its id, kind, frame, and consumers. This is the source of truth; everything else follows.
2. Scenario (if needed). If the shot needs scripted stream activity, add a demo scenario in demo-scenarios.ts (inside the test-mode boundary) and select it via POST /api/test/scenario. Any seeded content must byte-match its config.ts counterpart when an autosave/hydration path compares it.
3. Capture fn. Add a drive* (shared by still and loop) + shoot*/capture* in surfaces-desktop.ts or surfaces-mobile.ts, wrapped in attemptShot('<id>', …). Wait for a real money-state selector before shooting. For a loop, add a LoopSpec to captureLoops; call mark() in the drive only if the motion is in-drive. (Or supply the shot entirely via an override + skipAuto — no drive needed.)
4. Manifest. It updates automatically — the drive records raw entries into run.json, the process phase publishes them, and writeManifest embeds the registry.
5. Consume it. Marketing: add the id to ProductSurface + (if a loop) LOOP_SURFACES in features.ts, then bind it to a feature's media. Docs: embed <ProductShot id="<id>" alt="…" /> in an .mdx. Changelog: reference https://dorkos.ai/product/<id>-….
6. Guards. features.test.ts and shots.test.ts enforce that the registry, catalog, docs embeds, and files all agree — run pnpm --filter @dorkos/site test.

Verify After A Run

• pnpm --filter @dorkos/site test — the media guard must pass (all referenced assets exist, framing matches).
• Spot-check a loop's first frame is dark: ffmpeg -i <surface>-dark.webm -frames:v 1 /tmp/f.png and confirm.
• Check per-loop sizes stay ≤1.5MB and the total budget is reasonable (manifest.json totalBytes).

Art Direction

• Desktop stills are 1280×800 @2x (text readable at rendered size); mobile is 390×844 @3x; loops match their surface's dimensions.
• What reads premium: an inhabited app (a real fleet, distinct session rows, green-with-one-failure run history), motion that tells a story (streaming tokens, morphing radar, pulsing concurrent sessions), and a settled, un-squished layout. Crop modes (top/bottom in ProductFrame) bias a frame toward the edge that holds content when the vertical center is empty.
• Keep loops short and looping cleanly; respect the size budget so the site stays fast.