name: watch-youtube
description: Analyze a YouTube video by downloading its transcript, extracting semantically relevant frames using NLP, and compiling them into annotated storyboard grids optimized for Vision LLM token efficiency. After analysis, writes extracted knowledge to docs/wiki/ following wiki-schema rules. Use this skill whenever the user provides a YouTube URL and wants to analyze, summarize, or understand a video's visual content. Trigger on: "analyze this YouTube video", "understand what's shown in this video", "extract key frames", "summarize this tutorial", "what does this video show", "create a storyboard", "compress this video for LLM", "vision LLM video input".
watch-youtube: YouTube Video Storyboard for Vision LLMs
Turns a YouTube video into annotated JPEG storyboard grids, analyzes them with Vision LLM, and writes the extracted knowledge to docs/wiki/ using wiki-schema rules.
When the user invokes this skill
- Confirm the YouTube URL and desired output directory.
- Check that dependencies are installed (see Setup below).
- Run the pipeline via the
watch-youtube CLI.
- Read storyboards from
output/<video_id>/storyboard_page_*.jpg and analyze with Vision LLM.
- Show the user a structured analysis.
- Write the extracted knowledge to
docs/wiki/ following wiki-schema rules (see section below).
- Add a video record to
docs/wiki/Videos.md (see section below).
- Update
docs/wiki/Index.md.
Setup (run once per environment)
cd /Users/oguzcan/PycharmProjects/YouTube
source .venv/bin/activate
pip install -r requirements.txt
python -m spacy download en_core_web_sm
pip install -e .
brew install ffmpeg
Running the pipeline
watch-youtube "<YOUTUBE_URL>" \
--output-dir ./output \
--max-frames 20 \
--jpeg-quality 85 \
--verbose
Key options
| Flag | Default | Purpose |
|---|
--output-dir / -o | . | Where to save storyboard JPEGs |
--max-frames / -n | 30 | Cap on smart frames extracted |
--jpeg-quality / -q | 85 | JPEG quality 1–95 |
--silence-gap / -g | 5.0 | Seconds of silence that triggers a frame |
--groq-api-key / -k | env GROQ_API_KEY | Whisper fallback when no transcript |
--no-learn | off | Skip self-learning keyword store update |
--keep-temp | off | Keep raw frames and video file |
--verbose / -v | off | Show per-timestamp debug output |
Pipeline architecture
YouTube URL
│
▼
downloader.py Download transcript (VTT/SRT) + video MP4
│ Fallback: download audio → Groq Whisper API
▼
analyzer.py NLP timestamp extraction (spaCy + learned keyword store)
│ Rule A: keyword/deictic pattern matching
│ Rule B: silence gap detection (>= silence_gap seconds)
│ Post-run: TF-IDF self-learning updates data/keyword_store.json
▼
extractor.py ffmpeg frame extraction at smart timestamps only
│ Adaptive quality; skips black/corrupt frames
▼
compiler.py Pillow storyboard grid compilation
│ Adaptive cell resolution (720–1280px depending on grid density)
│ Transcript captions burned in below each frame
▼
output/<video_id>/storyboard_page_NNN.jpg (one or more JPEG grids, isolated per video)
│
▼
Vision LLM analysis Read storyboard images → structured summary
│
├──▶ docs/wiki/<title>.md Write topic pages (wiki-schema rules)
├──▶ docs/wiki/Videos.md Append video record (URL, date, pages)
└──▶ docs/wiki/Index.md Update index with new pages
Writing to docs/wiki (wiki-schema integration)
After analyzing the storyboard, always write a wiki entry to docs/wiki/ following these rules from wiki-schema:
Required frontmatter structure for each wiki page
**Özet:** [Max 3 sentences describing what was learned]
**Kütüphaneler/Teknolojiler:** [Key technologies/tools covered]
**Bağlantılar:** [[Related_Topic]], [[Another_Topic]]
Wiki file naming
- Use descriptive, PascalCase filenames:
OperatingSystemFundamentals.md, VirtualMemory.md
- After writing, update
[[docs/wiki/Index.md]] with a pointer to the new file
What to write in the wiki entry
- Main concepts explained in the video
- Key diagrams or visual structures (describe them in text)
- Relationships to other topics (with Obsidian-style
[[links]])
- Sponsor segments or tool mentions (tagged as such)
Example wiki entry
**Özet:** Bu video işletim sisteminin temel bileşenlerini anlatıyor: süreçler, kernel modu, sanal bellek ve dosya sistemi. CPU'nun tek bir hard-coded adresten boot ettiğini ve kernel'in Ring 0 ayrıcalığıyla çalıştığını gösteriyor.
**Kütüphaneler/Teknolojiler:** Linux Kernel, C, ffmpeg, spaCy
**Bağlantılar:** [[VirtualMemory]], [[FileSystem_Inodes]], [[DeviceDrivers]]
## İçerik
...
Always update Index.md
After writing a new wiki page, open docs/wiki/Index.md and add a link:
- [[NewPageTitle]] — one-line summary
Recording the video in docs/wiki/Videos.md
Always append a record to docs/wiki/Videos.md after each analysis:
### [Video Başlığı](https://www.youtube.com/watch?v=VIDEO_ID)
- **ID:** `VIDEO_ID`
- **Analiz tarihi:** YYYY-MM-DD
- **Süre:** ~X dakika
- **Transcript:** vtt / srt / whisper / synthetic
- **Kare sayısı:** N kare, M storyboard sayfası
- **Oluşturulan wiki sayfaları:** [[Sayfa1]], [[Sayfa2]]
- **Özet:** Tek cümlelik içerik özeti
This is the single source of truth for "which videos have been analyzed." Always check Videos.md before re-analyzing a URL that may already exist.
Adaptive resolution table
| Frames | Grid | Cell width |
|---|
| 1 | 1×1 | 1280px |
| 2 | 2×1 | 1024px |
| 3–4 | 2×2 | 960px |
| 5–6 | 3×2 | 800px |
| 7–9 | 3×3 | 720px |
| 10–12 | 4×3 | 640px |
| 13+ | 3×3 pages | 720px |
Troubleshooting
| Problem | Fix |
|---|
ffmpeg not found | brew install ffmpeg |
No module named spacy | pip install spacy && python -m spacy download en_core_web_sm |
Cannot access video | Video may be private, age-restricted, or region-blocked |
| No transcript found | Pass --groq-api-key or set GROQ_API_KEY env var |
| Module import error | Run pip install -e . from project root with venv active |
