video-prompt-ocr-extractor

name

video-prompt-ocr-extractor

description

Use this skill when the user wants to extract every on-screen prompt, image-generation prompt, UI prompt, caption block, or burned-in text prompt from a YouTube video or local video file, especially when transcript-based tools such as yt-transcript-summary missed visual overlays. This skill guides Codex to download/sample video frames, build contact sheets, crop prompt regions, transcribe visible text conservatively, reconcile against transcript or prior extraction files, and save a timestamped Markdown prompt inventory with uncertainty notes.

Video Prompt OCR Extractor

Purpose

Extract prompts that are visible in the video frames, not just spoken in captions. Treat transcript/video-summary extraction as a useful first pass, but not as the source of truth for text that is burned into the video.

Use this skill for requests like:

• "Extract all prompts shown in this video."
• "The transcript summary missed prompts shown on screen."
• "OCR the prompt overlays from this YouTube video."
• "Use this screenshot as an example and recover the rest of the prompts."

Core Rule

Do not rely on transcript text alone. For prompt overlays, verify against frames.

Required Reference

Before extracting on-screen prompts from a video, read and apply references/token-saving-mode.md. That reference is part of the skill contract: use a staged narrowing workflow, avoid inspecting every frame in detail, and keep evidence in contact sheets and prompt crops rather than pushing large visual payloads into the conversation.

Common failure modes of transcript-first extraction:

• Fast intro montages show prompts for less than a second.
• Prompts appear as UI overlays, lower thirds, or screenshots, not speech.
• Video models summarize representative examples instead of exhaustive text.
• On-screen text is repeated or partially obscured by picture-in-picture hosts.
• OCR quality changes by timestamp, crop, and seek accuracy.

Workflow

Apply the token-saving reference throughout this workflow. Start coarse, then narrow:

• Use transcript, description, chapters, or previous notes only to identify likely prompt-heavy regions.
• Build low-resolution contact sheets for those regions.
• Inspect the sheets to find candidate timestamps.
• Create high-resolution crops only for the prompt text regions.
• Transcribe from the crops and cite the evidence folders in the final artifact.

1. Establish Sources

From the working directory and user request, identify:

• YouTube URL or local video file.
• Existing transcript, summary, or prior prompt list.
• Attached screenshot examples that show the prompt style, position, font, and crop area.

If a transcript skill or video model is available, use it only as a candidate list. Mark it as incomplete until frame evidence is checked.

2. Download or Locate the Video

Prefer a local video file when one already exists. Otherwise download a manageable 720p copy:

yt-dlp -f "bv*[height<=720]+ba/b[height<=720]/best[height<=720]" --merge-output-format mp4 --write-info-json --no-playlist -o "source_video.%(ext)s" "<youtube-url>"

Verify duration and streams:

ffprobe -v error -show_entries format=duration -show_streams -of compact=p=0:nk=1 "source_video.mp4"

3. Build Frame Contact Sheets

Sample frames densely enough to catch fast overlays. Start with 1 frame per second over the full region likely to contain prompts. If there is a fast intro montage, scan it separately.

Use the bundled helper when useful:

python "$env:USERPROFILE\.codex\skills\video-prompt-ocr-extractor\scripts\video_prompt_scan.py" --video "source_video.mp4" --out-dir "video_prompt_scan" --start 00:00:00 --end 00:03:35 --fps 1

For comparison chapters, sample the likely prompt region:

python "$env:USERPROFILE\.codex\skills\video-prompt-ocr-extractor\scripts\video_prompt_scan.py" --video "source_video.mp4" --out-dir "video_prompt_scan_main" --start 00:03:35 --end 00:15:20 --fps 1

Inspect contact sheets visually. Identify every timestamp where a Prompt label or prompt-like UI block appears.

4. Create Enlarged Crops

Create evidence crops around prompt regions. Use the screenshot example to choose crop geometry. If fast seek lands between cuts, regenerate important crops with exact seek.

Example lower-third crop:

python "$env:USERPROFILE\.codex\skills\video-prompt-ocr-extractor\scripts\video_prompt_scan.py" --video "source_video.mp4" --out-dir "prompt_crops" --crop-times "00:00:27,00:00:33,00:03:54" --crop-rect "0:490:1280:230" --crop-scale 2 --exact-crop

--crop-rect uses x:y:w:h. For a 1280x720 video, common prompt overlays are often near:

• Intro full-width lower third: 0:490:1280:230
• Main comparison lower panel: 0:420:1280:300
• User-provided screenshot-like bottom band: 0:500:1280:220

Adjust after inspecting frames.

5. Transcribe Conservatively

For each prompt:

• Record timestamp.
• Transcribe the text as shown.
• Preserve quotes and explicit text-rendering instructions.
• Keep repeated prompts when used for separate examples, but mark them as repeated.
• Normalize only obvious encoding issues when the source text is clear, such as cafe vs mojibake from a prior transcript.
• If a word is ambiguous, use [uncertain: ...].

Do not invent missing words from the spoken narration. Use narration only to locate sections or disambiguate obvious references.

6. Reconcile Against Prior Extraction

Compare the visual list with existing prompt lists or transcript-derived markdown:

rg -n "Prompt|Generate|photorealistic|image|@Image" .

Write down what the prior method missed. Typical missed buckets:

• Intro montage prompts.
• Short edit commands.
• Repeated extraction prompt used on multiple products.
• Prompts partially covered by picture-in-picture video.
• Workflow-demo inputs like car.

7. Save the Artifact

Create a Markdown file in the user's working directory. Use this structure:

# All Image Generation Prompts Shown in the Video

Source video: ...
Extraction method: visual frame pass over downloaded video, contact sheets, and enlarged crops.
Evidence folders: ...

## Opening Montage Prompts
1. `[00:27]` ...

## Main Comparison Prompts
...

## Small Workflow Demo Input
- `[03:14-03:23]` car

## Notes and Uncertainty
...

If the user also asks for a process account, write a separate Markdown file beside the prompt inventory. Keep the account explicit about failed routes, tool availability, and why visual extraction was necessary.

Validation Checklist

Before finishing:

• Confirm the output file exists.
• Count numbered prompt entries mechanically.
• Search the artifact for known missing examples from screenshots or contact sheets.
• Mention any blocked route, such as expired API keys or missing OCR binaries.
• Keep generated evidence folders in the working directory unless the user asks for cleanup.