video2blog

name	video2blog
version	1.0.0
description	Convert a long educational video (lecture, tutorial, conference talk) into a Chinese blog-style Markdown with sectioning and curated screenshots. Use when the user asks to 视频转博客, 讲课视频做笔记, video to blog, 提取视频截图, 把视频整理成博客, 把这个视频写成笔记, or wants to share a lecture video as a written document. The agent MUST actually look at candidate frames with the Read tool and pick ones that visually match the narrative -- no blind extraction at fixed timestamps. The skill enforces this with a hard 7-step workflow. Works with YouTube URLs (uses yt-dlp for video + auto-captions) or local video files.

video2blog

Convert a video into a curated, Chinese-language blog post with the right screenshots.

Core principle

Don't extract frames at fixed timestamps and insert them blindly. Frame timestamps rarely correlate with topic boundaries — a frame at minute 20 may have nothing to do with the section covering minute 20.

Do extract a dense candidate pool, then view each candidate in the time window and pick the ones that actually show the concept. This rule is the foundation of the entire workflow.

Input

• A YouTube URL (e.g. https://www.youtube.com/watch?v=II4giR4vOOo)
• OR a local video file path (.mp4 / .mkv / .webm / etc.)
• (Optional) a course / lecture title; default is to read it from the video metadata
• (Optional) language: zh-Hans (default) / en — controls transcript language + output language

Output

Recommended layout (v6, default) — separates user-facing deliverables from intermediates. Use this for any multi-asset project (e.g. converting a lecture series):

<project>/
├── README.md                          # course / series overview, links
├── docs/                              # 📖 user-facing deliverables
│   └── <basename>/                    # one folder per video
│       ├── <basename>.md              # the blog post (Chinese)
│       ├── <basename>.pdf             # rendered PDF (optional)
│       └── images_<basename>/         # 🔑 PER-LECTURE image subdir (NOT shared)
│           ├── cover.jpg
│           ├── s01.png ~ sNN.png
│           └── ...
└── build/                             # 🔧 intermediate (don't open)
    ├── videos/                        # <basename>.mp4 (1.5GB+ for a 9-lecture series)
    ├── subtitles/                     # <basename>.vtt + dedup.vtt
    ├── info/                          # <basename>.info.json
    ├── candidates/                    # candidates_<basename>/ (frame pool, can be deleted)
    ├── drafts/                        # <basename>_body_draft.md
    ├── progress/                      # batch*-progress.md (subagent logs)
    └── scripts/                       # any dedup / extraction Python helpers

Why the images_<basename>/ per-lecture subdir? A shared images/ dir causes cross-contamination when batching: the L02 subagent overwrote L01's images/ in the v5 MIT 6.S191 run because both wrote to the same path. The v6 fix is to scope every image dir to its <basename> (e.g. images_L01/, images_L02/). md2pdf's image pre-check (also v6) accepts both images/... and images_<id>/....

Why docs/ + build/? Separating user-facing deliverables from intermediate artifacts (videos, subtitles, info.json, drafts) keeps the project navigable. docs/ contains only what the reader needs; build/ holds everything else.

Single-video mode (no project wrapper): you may collapse docs/ and skip build/, but STILL scope images to images_<basename>/ to leave room for adding more videos later.

<basename>/
├── <basename>.md            # the blog post (Chinese)
├── <basename>.pdf           # optional rendered PDF
├── images_<basename>/       # per-lecture subdir (NEVER just `images/`)
│   ├── cover.jpg
│   └── s01.png ~ sNN.png
└── <basename>.mp4 / .srt / .info.json / candidates/   # intermediates

Step 0 — Preflight checklist

Before you start a run, verify these. If any required item fails, stop and tell the user — don't silently substitute or skip a step.

Prereqs (one-line verification)

Tool	Verify with	Required?	Notes
yt-dlp	yt-dlp --version	For YouTube URLs	Must be 2026.01+; older versions lack --remote-components and silently downgrade formats
ffmpeg	ffmpeg -version	Always	Needed for frame extract
node (or deno/bun)	node --version; cross-check with yt-dlp -v 2>&1 | grep "JS runtimes"	For YouTube URLs	yt-dlp 2026+ needs a JS runtime for YouTube extraction. The --js-runtimes node flag is REQUIRED if your runtime is node — yt-dlp only auto-detects deno by name, not node. If the flag is dropped and node is the only runtime, -v will show JS runtimes: none and YouTube format selection silently downgrades. Use node ≥ 18 (yt-dlp 2026.03+ requirement).
whisper	whisper --help	Only if no subs	Local fallback; optional unless the video has no auto-captions
Free disk	(OS-specific, e.g. df -h .)	Always	~1 GB free per 60-min video (mp4 720p ~600 MB + 120 candidate frames ~300 MB)

Installing missing Python tools (yt-dlp, whisper)

When a Python tool is missing, create a virtual environment inside the skill directory — do not install globally or via pip install --user.

cd <skill-directory>
python -m venv .venv
# Windows:
./.venv/Scripts/python.exe -m pip install yt-dlp openai-whisper
# macOS / Linux:
./.venv/bin/python -m pip install yt-dlp openai-whisper

Then invoke tools via their venv path, e.g. ./.venv/Scripts/python.exe -m yt_dlp ....

Expected timing per step (60-min video, 720p)

Step	Time	Notes
1. Download	1-3 min	Depends on network; slower if no JS runtime and formats downgrade
2. Transcript	<10 s	If using auto-captions; Whisper fallback is 5-15 min on CPU
3. Frame extract	30 s - 2 min	Scales with length × density (1/30s default)
4-7. Agent work	5-15 min	Dominated by Step 6 (viewing frames); agent judgment

Decision tree: which path?

B站 (bilibili.com / b23.tv) URL?
  ├── yes ──> Step 1 with 4 flags:
  │            --js-runtimes node
  │            --remote-components ejs:github
  │            --add-headers "User-Agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36\nReferer:https://www.bilibili.com/"
  │            --write-auto-subs
  │          (Note: B站 auto-subs are usually 中文 only; --write-auto-subs is best-effort.)
  │          If a premium format (1080P/720P) is needed, also add:
  │            --cookies-from-browser chrome
  │          If you still get 412/352 with cookies, see the lux fallback in the Multi-site downloaders section.
  │
  └── no ──> YouTube URL?
            ├── yes ──> Step 1 with all 3 flags:
            │            --js-runtimes node
            │            --remote-components ejs:github
            │            --write-auto-subs
            │
            └── no ──> local file path
                      ├── has subs (.srt/.vtt next to file)?
                      │     ├── yes ──> use them (skip Step 2 download)
                      │     └── no  ──> Step 2 with Whisper fallback
                      │
                      └── Step 3 onward is the same either way

🛑 STOP if the URL is from a different Chinese platform (抖音/快手/AcFun): skip yt-dlp and use lux per the Multi-site downloaders section. yt-dlp does not support extraction from these platforms.

B站-specific preflight (only if URL is bilibili.com or b23.tv)

• B站 anti-bot requires a browser User-Agent + Referer:https://www.bilibili.com/. The decision-tree command already passes these, but always test with yt-dlp -v --skip-download $URL first — if you get 412, the headers didn't take; if you get 352 on a channel page, plan for cookies (see Failure modes table).
• B站 auto-subs are almost always 中文 only; don't expect English subs even for English content.
• Premium formats (1080P/720P) require login cookies; the default format (480P/360P) is fine for blog use.

🛑 STOP conditions during preflight: any required prereq fails AND there's no viable fallback (e.g. ffmpeg missing for a local-only run) → stop and tell the user exactly which tool is missing.

Workflow

The agent follows Step 0 + Steps 1-7 in order. Do not skip the Look-At-Frames step (Step 6).

🛑 STOP at Step 1: if the user hasn't told you whether they want Chinese or English output, ask once. Don't default to Chinese without confirmation when the user prompt is in English.

Step 1 — Decide source & download

YouTube URL → use yt-dlp:

yt-dlp \
  --js-runtimes node \
  --remote-components ejs:github \
  --write-auto-subs \
  --sub-langs "zh-Hans,en,en-US,en-GB" \
  --sub-format "vtt/srt" \
  --write-info-json \
  -o "<basename>.%(ext)s" \
  -f "best[ext=mp4][height<=720]/best[ext=mp4]/best" \
  "$URL"

The --js-runtimes node flag is required for YouTube extraction on yt-dlp 2026+. Without it, you get WARNING: No supported JavaScript runtime could be found and may silently lose formats.

The --remote-components ejs:github flag is also required for higher-quality formats (720p+). Without it, yt-dlp silently downgrades to format 18 (360p). See the Failure modes table.

This gives you:

• <basename>.mp4 (720p preferred for size)
• <basename>.<lang>.vtt (auto-generated captions)
• <basename>.info.json (title, duration, uploader, etc.)

Local video → skip download; check that the file exists and is readable.

If yt-dlp is not installed, follow the venv install instructions in Step 0.

Step 2 — Pick & extract transcript

Prefer the VTT in the user's chosen language; if missing, fall back to any available language. Parse it to a simple (start_seconds, text) list.

If no subtitles exist at all, fall back to local Whisper:

whisper "<basename>.mp4" --language Chinese --model small --output_format srt

If Whisper is also unavailable, stop and tell the user: "No transcript available. Install openai-whisper into the skill's .venv (see Step 0) or provide a transcript file."

Step 3 — Extract dense candidate frames

mkdir -p "candidates_<basename>"
ffmpeg -i "<basename>.mp4" \
  -vf "fps=1/30,scale=640:-1" \
  -q:v 3 \
  "candidates_<basename>/p%03d.png"

Default: 1 frame per 30 seconds. For a 60-min video this gives ~120 frames. The cap is 200 frames; if the video is long (>100 min) and we'd exceed 200, downsample to 1/60s.

Pick the frame density by video type — the default is too sparse for fast-paced content:

Video type	Examples	Recommended fps	Why
Lecture / keynote (slow)	MIT OCW, conference talks, slide decks	1/60 (every 60s)	One visual concept per slide; sparse is enough
Tutorial / explainer (medium)	Most YouTube tutorials, online courses	1/30 (every 30s)	Default. 60 min → 120 frames
Animation / hand-drawn (fast)	3Blue1Brown, Kurzgesagt, Numberphile	1/15 (every 15s)	New visual every 5-10s; 30s intervals miss key frames

Heuristic: sample the first 30s of the video with ffmpeg -ss 0 -i input.mp4 -frames:v 1 -q:v 3 /tmp/sample.png and the first 2-min mark. If they're different scenes, it's fast-paced → use 1/15. If they look similar, it's slow → use 1/60 to save disk.

🛑 STOP if ffmpeg is missing: tell the user explicitly to install (MiKTeX users often have it via PATH; macOS: brew install ffmpeg; Linux: apt install ffmpeg).

Step 4 — Chunk the transcript by topic

Read the full transcript. Identify 5-10 topic boundaries. Each chunk should be ~5-10 minutes long, and the boundaries should be where the lecturer visibly moves to a new concept ("now let's talk about X", slide change, recap+transition).

Output: a list of chunks, each with (start, end, working_title).

# Pseudocode for what the agent does
chunks = [
  {"start": 0,   "end": 420,  "title": "导论：为什么 2026 还要学深度学习"},
  {"start": 420, "end": 1020, "title": "神经元与激活函数"},
  {"start": 1020,"end": 1620, "title": "反向传播与梯度下降"},
  # ...
]

Step 5 — Draft the body (Markdown only, no images yet)

For each chunk, write a Chinese blog-style section:

• Start with a 1-2 sentence hook grounded in the lecture
• 2-3 paragraphs of substance (key concepts, equations if any, examples)
• End with an insight callout in the ::: {.insight} fenced-div format (compatible with the md2pdf skill)

Keep each section ~200-400 Chinese characters. Don't write image placeholders yet — you'll pick frames in the next step.

Step 6 — Pick frames per section (the critical step)

This step cannot be skipped. Do not substitute with timestamp arithmetic.

For each section, do this:

1. List candidate frames in the section's time window. The 30s extraction means a 5-min section has ~10 candidates. Build a small table:

   Section 2 (神经元, 7:00-12:00):
     candidates/p014.png @ 7:00  (07:00 frame)
     candidates/p015.png @ 7:30
     ...
   

2. For each candidate, use the Read tool to view the image. The Read tool will show the multimodal LLM the image content. For each, write down:

   • What is visually on screen? (lecturer face / slide / whiteboard / code / chart / blank)
   • Does it actually illustrate a key concept in the section?
   • Rating: relevant / tangential / irrelevant

3. Pick 1-2 relevant images per section. Skip tangential/irrelevant frames entirely. If a section has no relevant candidates, write a note like <!-- no good frame found for this section --> and move on.

4. Copy the chosen images to images_<basename>/sNN.png where NN is the section number. Skip the rest. (NEVER images/ — per-lecture subdir is mandatory when batching, see Output section.)

If you are about to pick a frame without viewing it: stop, read the image, and re-evaluate.

Step 7 — Cover image + assembly

• YouTube source: download <video_id>_maxresdefault.jpg from https://i.ytimg.com/vi/<video_id>/maxresdefault.jpg. If 404 (some videos disable it), fall back to hqdefault.jpg and warn.
• Local source: extract frame at t=5s, scale to 1280x720, save as cover.jpg.

Then assemble the markdown:

# <title>

> 主讲：<lecturer> ｜ 时长：<mm:ss> ｜ 日期：<yyyy-mm-dd>
> 视频链接：<url>
> 配套资料：<course_homepage>

![封面](images_<basename>/cover.jpg)

这是 <course> 第 N 讲 <topic> 的精读笔记。<2-3 sentence hook>.

---

## 1. <section title>

<body text>

![<image caption>](images_<basename>/s01.png)

::: {.insight}
**Insight** — <key takeaway>
:::

---

## 2. ...

---

## 参考与跳转
- 课程主页：<url>
- 视频：<url>

Failure modes (with fix)

Symptom	Root cause	Fix
yt-dlp errors with Sign in to confirm you're not a bot	YouTube anti-bot	Switch to a Cobalt instance or use yt-dlp --cookies-from-browser chrome if user consents
ffmpeg not found	Missing binary	Tell user: install via apt/brew/winget; do not silently substitute
No auto-generated captions in user's language	Video has no auto-subs	Try Whisper local; if that fails, stop and report
Whisper is slow / OOM on 90-min video	Model too big	Use --model tiny or base; document the quality loss to user
candidates/ has 500+ frames and the disk is full	Long video + dense extraction	Re-run with -vf "fps=1/60" (every minute) and reduce scale to 480px
Agent picks frames without viewing them	Skipped Step 6.2	Re-run Step 6, viewing every candidate frame before choosing.
All candidates/ frames are "lecturer's face"	Camera was on the speaker the whole time	Pick the best lecturer-shot for the cover, document that no usable slides were found
Transcript is in a language the user didn't ask for	yt-dlp returned wrong sub track	Re-run with --sub-langs "zh-Hans" explicitly
WARNING: No supported JavaScript runtime could be found (or [debug] JS runtimes: none in -v output) — even though node --version works on this machine	yt-dlp 2026+ does NOT auto-detect node on PATH; it only auto-detects deno by binary name	(1) ALWAYS pass --js-runtimes node explicitly when targeting YouTube. The flag is required, not optional, for node. (2) If you only have deno, you can drop the flag — yt-dlp's deno detection works. (3) If you have neither, install Node.js LTS (yt-dlp 2026.03+ requires node ≥ 18).
yt-dlp returns an unexpectedly small format list with no error	Silent failure when JS runtime is missing — the extractor skips challenge-solving and gives up some formats	Same as above: install Node.js and re-run with --js-runtimes node
WARNING: [youtube] [jsc] Remote component challenge solver script ... was skipped then downloads format 18 (360p) instead of 720p	yt-dlp 2026+ has a second challenge layer (EJS remote components) beyond JS runtime. Without it, higher-quality formats are refused and yt-dlp silently falls back to legacy 360p	Add --remote-components ejs:github to the yt-dlp call. The flag auto-fetches the EJS script from GitHub on first run
VTT has 1000+ lines for a short video (e.g. 19 min video = 4000 lines)	YouTube auto-captions for some educational channels use 3-line rolling captions — each sentence appears 3 times in consecutive cues with growing tails	Run a rolling-caption dedup before chunking. Algorithm: keep only cues whose text doesn't start with the previous kept cue's first 30 chars. Reduces 4000→500 lines for typical cases
yt-dlp errors with HTTP Error 412: Precondition Failed on a B站 video page (https://www.bilibili.com/video/BV...)	B站 anti-bot rejects the default python-urllib User-Agent + missing Referer	Add --add-headers "User-Agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36\nReferer:https://www.bilibili.com/" to the yt-dlp call. The 412 goes away; you may then see "this video may be deleted or geo-restricted" (real geo-block) or a successful format list. Do NOT switch to lux for this — yt-dlp works fine with the right headers.
yt-dlp --flat-playlist https://space.bilibili.com/<uid>/video errors with Request is rejected by server (352) or Request is blocked by server (412) (and silently returns 0 items if you forgot -v)	B站's space page has a stricter anti-bot than the video page; UA+Referer alone isn't enough	(1) ALWAYS pass -v (or --dump-json) on B站 channel fetches — without it the run exits 0-looking with Downloading 0 items and the agent doesn't know it failed. (2) Use --cookies-from-browser chrome (or --cookies cookies.txt after exporting from the user's browser) — the space page needs a buvid3/b_nut cookie pair, not just headers. (3) If cookies are unavailable, fall back to scraping a single known BV ID and listing the user-provided video URLs manually.

Anti-patterns (do NOT do these)

1. ❌ Pick frames at fixed timestamps (e.g. 5/20/35/50 min) — frame timing does not correlate with topic boundaries.
2. ❌ Pick frames in numerical order — choose by content relevance, not sequence position.
3. ❌ Use maxresdefault.jpg for body images — it's the YouTube auto-picked thumbnail. Cover only.
4. ❌ Write the body without reading the transcript — the transcript is the ground truth for accurate summarization.
5. ❌ Insert images before viewing them — verify each candidate visually before placing it in the markdown.
6. ❌ Use lecturer-talking-head shots as content illustrations — if a section has no relevant frames, write <!-- no good frame found --> instead of padding with face shots.
7. ❌ Pick blank or slide-transition frames — these carry no information.
8. ❌ Skip or reduce ::: {.insight} callouts — insight blocks are the highest-value part of the output for most readers.
9. ❌ Use the default 1/30s frame density for animation or hand-drawn content — fast-paced visual content needs 1/15s or denser. See the pacing table in Step 3.
10. ❌ Read a 1000+ line VTT into chunking without dedup — YouTube auto-captions on educational content often have 3x rolling redundancy. Check line count first; if >1000 lines for a <30 min video, run a rolling-caption dedup before Step 4.

Multi-site downloader alternatives

yt-dlp is the default and covers ~1500+ sites (YouTube, Vimeo, Twitter/X, Bilibili, 抖音, etc.). For sites where yt-dlp lags (especially some Chinese video platforms), the community has built alternatives:

Tool	Stars (~)	Language	Strengths	Weaknesses	When to reach for it
yt-dlp	167K+	Python	Universal, well-maintained, huge format catalog	Some Chinese sites lag; needs JS runtime in 2026+	Default choice for 95% of cases
lux (iawia002/annie)	31K+	Go	Chinese-site king: B站/抖音/斗鱼/快手/爱奇艺/芒果TV/西瓜/AcFun	Smaller community; non-Chinese sites spotty	When yt-dlp returns a persistent 412/352 on a B站 URL after you've added browser UA + Referer + cookies, or when the user is targeting a smaller Chinese platform (抖音/快手/AcFun) that yt-dlp's extractor list lacks. NOT the first response to a 412 — try headers/cookies first.
you-get	57K+	Python	Wide historical coverage, simple CLI	Archived/unmaintained since 2022; security issues; many extractors stale	Don't use for new work; legacy fallback only
Cobalt (imputnet/cobalt)	8K+	Node/TS	Web-API style, can run as self-hosted service, no ads	Requires API setup; some sites block it	Bulk downloads, integration with other tools

If yt-dlp fails on a Chinese site (B站/抖音/快手 common case):

# Install lux (one-time)
# Option A: Go install
go install github.com/iawia002/lux@latest

# Option B: Download a release binary from
# https://github.com/iawia002/lux/releases

# Use it (cookies needed for some sites)
lux -c cookies.txt "https://www.bilibili.com/video/BVxxxxx"

Decision rule: always try yt-dlp first (it's the most reliable, has auto-subs, and gets the .info.json for free). Reach for lux only when yt-dlp returns a hard error on a known-good URL and the URL is on a Chinese video platform.

Templates & resources

Path	Purpose
scripts/extract.sh	One-shot: download (if URL) + extract candidates + write transcript. Use this to bootstrap.
test-prompts.json	The test prompts for darwin-skill to score this skill.
examples/neural-net-sample.md	Worked example: 3Blue1Brown "But what is a neural network?" (3-section blog draft with verified image picks). Use this as a template when running on a new video.

Quick start (TL;DR)

# 1. Bootstrap the artifacts
bash scripts/extract.sh "https://www.youtube.com/watch?v=II4giR4vOOo" lecture01

# 2. Then the agent (you) does Steps 4-7:
#    - chunk transcript
#    - draft body
#    - **view candidate frames, pick the relevant ones**
#    - assemble markdown
#    - (optional) build PDF via the md2pdf skill

What this skill does NOT do

• Won't auto-publish to a blog platform (e.g. Medium, 知乎) — out of scope
• Won't generate diagrams or illustrations that aren't in the video
• Won't transcribe music / lyrics
• Won't handle multi-camera videos (e.g. interview podcasts with two camera angles) — picks the main speaker
• Won't do scene-level editing (cuts, transitions) on the video itself

Iterating this skill

Like md2pdf, this skill is designed to be self-improved with darwin-skill. The dim8 (实测) score depends on running test prompts and verifying the agent actually viewed frames before picking — not just claimed it did.