figureout

!`node "${CLAUDE_SKILL_DIR}/scripts/project-context.mjs"` # figureout Turn a public video URL or a local video file into representative frames plus a timestamped transcript, then answer grounded questions about it. You run the bundled Node scripts, Read every printed frame so you actually see the footage, and cite MM:SS timestamps for every claim. You never publish and never upload the video itself. The injected line above prints a compact JSON snapshot of the current project: working directory, platform, whether `ffmpeg`, `ffprobe`, and `yt-dlp` resolve on PATH, the detected Whisper backend and whether a key is set, and any prior `figureout-output\` runs. Read it first; it tells you whether setup is ready before you invoke the pipeline. ## Setup preflight Before the first run, sense readiness. Run: ``` node "${CLAUDE_SKILL_DIR}/scripts/setup.mjs" --json ``` Branch on the `status` field: | `status` value | Meaning | Action you take | |----------------|---------|-----------------| | `ready` | Binaries present, and either captions-only use is fine or a Whisper key is set | Proceed to invocation. | | `needs_install` | One or more of `ffmpeg`, `ffprobe`, `yt-dlp` missing from PATH | Show the user the missing binaries and OS install hints from [setup](references/setup.md); do not invoke the pipeline until installed. | | `needs_key` | Binaries present but no Whisper key | Proceed for caption-backed videos. If the target has no captions and the user wants a transcript, use AskUserQuestion to offer: set a key, or continue frames-only with `--no-whisper`. | | `needs_install_and_key` | Binaries missing and no key | Resolve binaries first (see [setup](references/setup.md)), then handle the key as in `needs_key`. | For a silent gate in a script, use `--check`, which prints nothing and sets the exit code: `0` ready, `2` missing binaries, `3` missing key, `4` both. To scaffold the config and print OS-specific install hints, run `node "${CLAUDE_SKILL_DIR}/scripts/setup.mjs" install`. ## Invocation The core loop has three steps: 1. Run the orchestrator: ``` node "${CLAUDE_SKILL_DIR}/scripts/figureout.mjs" <video-url-or-path> [flags] ``` It prints a markdown report: a metadata header, a Frames section listing each extracted frame path with `t=MM:SS` and a reason, a fenced Transcript section, and a work-dir footer. 2. Read every frame path the report printed, in parallel, using the Read tool. This is mandatory: you must actually see the frames to describe the video. Reading them together lets you follow the footage in order. 3. Answer the user's question grounded in what you saw and read. Cite MM:SS timestamps for every visual or spoken claim, and quote transcript lines by their timestamp. When you are done, tell the user the work dir can be deleted. If the user asked a specific question, pass it with `--question` (or as trailing free text). Otherwise summarize what happens across the video, beginning to end. ## Flags Parse the invocation string directly. One positional, the rest double-dash: | Flag | Values | Default | |------|--------|---------| | positional | a public video URL or a local video file path | required | | `--question` | quoted string; the specific question to answer | summarize | | `--detail` | `transcript`, `efficient`, `balanced`, `token-burner` | `balanced` | | `--start` | seconds or `MM:SS`; start of a focus window | video start | | `--end` | seconds or `MM:SS`; end of a focus window | video end | | `--timestamps` | `"MM:SS,MM:SS"`; extract cue frames at these marks | none | | `--resolution` | integer pixel width for extracted frames | `512` | | `--no-dedup` | flag; keep near-duplicate frames | dedup on | | `--no-whisper` | flag; never upload audio, captions only | whisper allowed | | `--out-dir` | path for the working directory | temp dir | | `--feed-showoff` | flag; also write `figureout-analysis.json` for a showoff run | off | Detail modes cap the frame budget: `transcript` and `efficient` keep at most 50 frames, `balanced` at most 100, `token-burner` is uncapped. See [pipeline](references/pipeline.md) for the full budget and extraction mechanics. ## Focus mode and cue frames Two ways to zoom in on part of a video: - **Focus window** (`--start` / `--end`): restrict extraction and transcript to that span. The frame budget becomes denser inside a short window than across a whole video, so a 30-second window gets richer coverage than 30 seconds sampled from an hour. Budgets are in [pipeline](references/pipeline.md). - **Cue frames** (`--timestamps "MM:SS,MM:SS"`): pin exact frames at specific moments. Cue frames are extracted first and reserved against the cap, so they always survive deduplication and downsampling. Use them when the user points at a precise moment ("what is on screen at 2:14"). Cue marks outside an active focus window are dropped. Combine them: a focus window plus a couple of cue marks gives dense coverage of a region with guaranteed frames at the exact moments that matter. ## ShowOff bridge With `--feed-showoff`, the run also writes `figureout-analysis.json` into the working directory. A `showoff` run can point its evidence path at that file and reuse this analysis directly: the extracted frames are real on-disk JPEGs that become scored visual assets, and the transcript segments become sourced facts carrying their MM:SS timestamps. Any external service contacted during analysis is carried over in `external_services_used` so provenance survives the handoff. The exact schema and the consuming contract are in [showoff-bridge](references/showoff-bridge.md). ## Security and boundary State this precisely and do not overstate it: - figureout works with PUBLIC data only: a public video URL or a local file the user gave you. It fetches the video stream and any captions the host publishes. - The only network upload of user content is the extracted AUDIO to an approved Whisper provider (Groq or OpenAI). This happens ONLY when captions are absent, a key is set, and `--no-whisper` was not passed. The video itself is never uploaded, only the audio track re-encoded to a small mono MP3. - figureout never publishes, shares, or posts anything. It writes only to its working directory. - Do not describe the run as fully offline or imply nothing reaches the network. Caption fetches reach the video host, and Whisper transcription is a real network upload of extracted audio. State the precise boundary above instead. ## Platform note On Windows, use Windows-style paths with backslashes in every path and command. Do not use bash-only syntax (`&&`, `$()`, backtick subshells). Prefer the bundled Node scripts for all shell work; each one runs the same on win32, macOS, and Linux and hides child console windows. ## References | Reference | Content | |-----------|---------| | [pipeline](references/pipeline.md) | Frame budgets, perceptual dedup, detail modes, extraction strategies, focus mode | | [transcription](references/transcription.md) | Captions-first flow, Whisper fallback, chunking, providers | | [setup](references/setup.md) | Required binaries, the `.env` config, keys, preflight rubric and exit codes | | [showoff-bridge](references/showoff-bridge.md) | The `figureout-analysis.json` handoff contract | | [templates/analysis-report.md](templates/analysis-report.md) | The markdown report shape figureout.mjs prints | | `scripts/project-context.mjs` | Project state sensor (injected at the top of this file) | | `scripts/config.mjs` | Reads the `.env`, resolves detail mode, keys, and Whisper backend | | `scripts/setup.mjs` | Preflight and installer (`--check`, `--json`, `install`) | | `scripts/frames.mjs` | ffmpeg frame extraction, budgets, dedup, downsampling | | `scripts/transcribe.mjs` | VTT parsing and transcript formatting | | `scripts/whisper.mjs` | Audio extraction, chunking, and Whisper upload | | `scripts/figureout.mjs` | Orchestrator entry point invoked above |