storyboard-ocr

name	storyboard-ocr
description	OCR extracted storyboard frames, write a text manifest, and surface evidence-backed UI text from a local video.

Storyboard OCR

Use this after storyboard-extract when an agent needs machine-readable UI text from extracted frames instead of only images.

This skill owns:

• OCR over extracted storyboard frames
• per-frame text with confidence filtering
• per-line boxes and coarse regions when block/layout OCR is available
• storyboard.ocr.json
• an aggregated unique-text summary for later review prompts

Inputs

Provide either the storyboard directory or manifest path:

{
  "storyboardDir": "./output/storyboard",
  "minConfidence": 45
}

{
  "manifestPath": "./output/storyboard/storyboard.manifest.json",
  "minConfidence": 55
}

minConfidence defaults to 45 and must be between 0 and 100.

Outputs

The skill writes storyboard.ocr.json beside the storyboard manifest. The manifest includes per-frame OCR lines, semantic UI-like lines, confidence, regions, optional bounding boxes, frame quality, unique text summaries, and sampling metadata carried forward from extraction.

Sequencing Guidance

Run after storyboard-extract. Run before storyboard-transitions and storyboard-understand. If OCR quality is weak but the visual frames look important, manually inspect the images instead of relying only on text output.

Interpretation Notes

Use semanticLines for product/UI claims; raw lines may include subtitles, decorative text, or OCR noise. quality.status: reject means downstream text inference should be low-trust for that frame. Bottom-region long sentences are often classified as subtitle-like and suppressed from semantic summaries.

Abstention Rules

Do not treat OCR text as exact transcription unless confidence and frame quality are strong. Abstain from naming features, screens, or workflows if the supporting lines are low-confidence, suppressed, or only present in rejected frames.

Failure Modes

The input must point to an existing storyboard manifest and image files. Tesseract/OCR setup failures, missing frames, unreadable images, or an overly high minConfidence can produce sparse output. Sparse output is a signal to inspect the frame images, not a signal that the UI contains no text.

Repo-side runner:

node --import tsx scripts/harness/storyboard-ocr.ts

Installed-pack runner:

node ./.video-evaluator/agent/run-tool.mjs storyboard-ocr