doc-to-manim-video

name

doc-to-manim-video

description

Create narrated educational animation videos from knowledge documents and optional reference/model images using manimlib, edge-tts, and HyperFrames. Use when the user provides a Markdown/PDF/notes document and asks for a clear explainer animation, formula derivation video, scientific teaching video, voiceover, title/end cards, vector redraws of conceptual/model figures, dynamic visual explanations based on reference images, or validation that formulas and symbols inside ordinary text annotations render as real math instead of plain text.

Doc To Manim Video

Purpose

Turn a knowledge document into a polished explainer video: read the document, inspect any reference/model images supplied with it, design a storyboard, render mathematical and conceptual animation with manimlib, generate narration with edge-tts, assemble audio/video with HyperFrames, and verify that all formulas render as LaTeX.

Default output should include a title page, a clear derivation/story flow, pauses at key formulas, normal-speed natural voiceover, and an ending page.

When the source package includes conceptual figures, physical model diagrams, mechanism schematics, workflow diagrams, geometry cartoons, or "expected structure" images, treat them as visual references. Recreate their structure as clean Manim vector graphics whenever possible, then animate the components needed for the explanation. Do not simply insert screenshots of those model figures unless the user explicitly asks for a faithful image-based display.

Runtime

Use the user's project runtime when one is specified. Otherwise, run from the target project directory with manimgl, edge-tts or python -m edge_tts, ffmpeg, and npx hyperframes available on PATH.

cd <project-root>
manimgl <scene.py> <SceneName> -w -m --fps 24 --video_dir <hyperframes-project>/assets
npx hyperframes lint --verbose
npx hyperframes inspect --samples 8
npx hyperframes render --output <final-video.mp4> --fps 24 --quality standard --workers 1

Use edge-tts from the same shell/environment when available. If the command is unavailable, try python -m edge_tts. If the user provides a Conda/venv/runtime command, use that consistently for Python, Manim, audio generation, and validation.

Do not overwrite source documents. Put generated files under task-specific paths such as:

animations/<slug>_story.py
hyperframes/<slug>/assets/
hyperframes/<slug>/index.html
videos/<slug>.mp4

Language Settings

Default language is Chinese unless the user specifies another language.

Use normal speed and a natural female voice by default:

edge-tts --voice zh-CN-XiaoxiaoNeural --rate "+0%" --text "<narration>" --write-media <segment>.mp3

Recommended presets:

zh-CN: zh-CN-XiaoxiaoNeural, rate +0%
en-US: en-US-JennyNeural, rate +0%
ja-JP: ja-JP-NanamiNeural, rate +0%

If the user asks for a language setting, apply it to narration text, voice choice, index.html lang, and any visible title/end text. Only add subtitles when explicitly requested.

Workflow

1. Read the source document and extract the teaching arc: concepts, formulas, assumptions, and the order of derivation.
2. Inspect all supplied or discoverable reference images. Classify each one as:
   • conceptual/model schematic: redraw as Manim vector shapes and animate it;
   • workflow/architecture diagram: redraw as vector flow blocks, arrows, and labels;
   • data figure/photograph/microscopy image: use directly only when the real texture or measurement is essential, otherwise simplify into a vector abstraction;
   • logo/cover/photographic context: use sparingly as an image asset only if it improves orientation.
3. Write a compact storyboard with segment keys, narration text, target duration, formulas to show, visual references to redraw, and pause time after important equations.
4. Generate voiceover segment by segment with edge-tts; insert short silent pauses after key formulas so viewers can understand them.
5. Concatenate audio and write a timeline JSON with each segment start/end time.
6. Write the manimlib scene. Use one scene class, segment methods, and a hold(key) helper that waits until the audio timeline reaches each segment end.
7. Render Manim to an asset video, then re-encode it with dense keyframes for HyperFrames:

ffmpeg -y -i <manim-video.mp4> -c:v libx264 -preset medium -crf 18 -r 24 -g 24 -keyint_min 24 -pix_fmt yuv420p -movflags +faststart <manim-video-keyed.mp4>

8. Create a minimal HyperFrames composition that layers the keyed Manim video and the final voiceover audio. Do not add captions unless requested.
9. Run hyperframes lint, hyperframes inspect, render the final MP4, and verify with ffprobe that audio and video durations match.
10. Extract preview frames from title, vector-redrawn model diagrams, formula-heavy sections, the previously risky annotation sections, and the ending page. Inspect them visually before final delivery.

Reference Image To Vector Workflow

Use this whenever the user provides images as examples of the desired model structure, or when the source document/paper contains figures that are conceptually important.

1. Open and visually inspect the reference images before drawing. Identify the semantic parts, not just the pixels: regions, phases, particles, pores, interfaces, arrows, electric fields, charge signs, workflow blocks, inputs/outputs, labels, and hierarchy.
2. Decide which parts should move during the explanation. Prefer animating the mechanism, for example:
   • electric field arrows sweeping across a pore or particle;
   • ions or charges shifting inside an EDL;
   • highlights/rings tracing a solid-liquid boundary;
   • current paths bending through connected pores;
   • workflow blocks appearing step by step with arrows;
   • local quantities flowing into an effective parameter or final equation.
3. Redraw conceptual/model figures with Manim primitives such as Circle, Rectangle, RoundedRectangle, Polygon, Line, Arrow, VMobject, VGroup, Tex, and Text. Build small helper functions for reusable structures, e.g. vector_particle_edl(), vector_narrow_pore(), vector_framework(), or vector_microstructure().
4. Preserve the visual logic of the reference, but simplify for teaching. Remove tiny labels, dense legends, and publication clutter; replace them with larger Chinese labels, clean color coding, and a few animated callouts.
5. Do not use ImageMobject for conceptual/model diagrams in the final scene unless the user explicitly requests direct figure insertion. It is fine to prepare or inspect image crops as references during development, but the final animation should show the vector redraw.
6. If a real experimental image is essential, separate it from the mechanism diagram: show the real image briefly as context, then transition to a vector abstraction for explanation.
7. Label the relationship honestly, e.g. "按论文图 2 结构重绘" or "根据参考图抽象绘制", rather than implying the vector diagram is raw data.
8. When the user specifically says the current model is "not vivid", "not like the paper figure", "use the figure structure", or "draw it as vector", revise the scene by replacing pasted images or generic cartoons with vector redraws and dynamic mechanism animation.

Implementation pattern:

def vector_particle_edl(height=3.6):
    edl = Circle(radius=1.28).set_fill("#DDE8EE", 0.72).set_stroke(width=0)
    particle = Circle(radius=0.83).set_fill("#9DA4A6", 1).set_stroke("#3D464B", 2)
    field = Arrow(LEFT * 1.2, RIGHT * 1.2, color=GREEN)
    charges = VGroup(*[
        Text("+", font=FONT, font_size=22, color=INK).move_to(...)
        for ...
    ])
    return VGroup(edl, particle, field, charges).set_height(height)

Keep these helper functions deterministic and readable; a few handcrafted positions are better than a complicated image-tracing pipeline when the goal is explanation.

Manim Requirements

Use Tex(...) for every formula or symbol-bearing fragment. Never put formulas such as σ_dc / ω, sigma*, epsilon'', J_c, or ω inside plain Text(...) if they are meant to look mathematical.

Use helper functions like this:

def text(s, size=34, color=INK, weight=NORMAL, width=12.0):
    return fit(Text(s, font="PingFang SC", font_size=size, color=color, weight=weight), width)

def mathtex(s, size=48, color=INK, width=12.0, t2c=None):
    return fit(Tex(s, font_size=size, color=color, t2c=t2c or {}), width)

def inline_row(*parts, buff=0.12, width=12.0):
    return fit(VGroup(*parts).arrange(RIGHT, buff=buff), width)

For mixed Chinese and math annotations, split the sentence:

inline_row(
    mathtex(r"\omega", 32),
    text(" 越小，", 28),
    mathtex(r"\frac{\sigma_{\mathrm{dc}}}{\omega}", 34),
    text(" 越大", 28),
)

Before final render, search the animation source for likely plain-text formula mistakes:

rg 'σ|ε|ω|sigma|epsilon|_dc|\\*|J_c|J_d|/ ω|/ omega' animations/<slug>_story.py

Every hit in Text(...), text(...), or code(...) should be intentionally plain language or rewritten as mathtex(...)/inline_row(...).

If the user requested vector redraws of conceptual/model images, also search for accidental direct image use:

rg 'ImageMobject|assets/paper|fig01|fig02|fig03|screenshot|crop' animations/<slug>_story.py

Any remaining ImageMobject should be intentional, such as a real microscopy/photo/data image that must remain raster. Conceptual model diagrams should be represented by vector helper functions instead.

Video Structure

Always include:

• Title card: topic name plus a compact formula/symbol line.
• Concept setup: explain what problem the formula solves.
• Reference-to-vector concept scenes when the document includes important model images: redraw the model structure as vectors and animate the physical mechanism.
• Derivation scenes: one formula transformation per beat, with visual grouping and arrows.
• Interpretation scenes: translate real/imaginary parts or variables into physical meaning.
• Pause beats: hold key formulas for at least 1-3 seconds beyond narration when helpful.
• Ending card: summarize the final formula and the main physical takeaway.

Validation Checklist

Run and report:

npx hyperframes lint --verbose
npx hyperframes inspect --samples 8
ffprobe -v error -show_entries format=duration -show_streams -of json <final-video.mp4>

Also verify:

• Formulas are rendered with LaTeX, including inside ordinary note labels.
• Important conceptual/model reference images have been redrawn as vector diagrams unless direct raster use was explicitly requested or scientifically necessary.
• Vector model scenes show the mechanism dynamically, not only a static copy of the reference.
• No text overlaps or spills outside the 1280x720 frame.
• Title page and ending page are present.
• Voiceover is natural, normal speed, and aligned with visual pauses.
• Final MP4 path is under videos/ and shown to the user with an absolute Markdown media link.