qwen-mm-plugins-edu-agent

# Math and Science Tutorials Turn a math or science problem into a step-by-step Mandarin video tutorial. For an interactive page instead of a video, use the interactive workflow linked below. Teaching components — equations, diagrams, solution steps — are rendered as realistic objects: solid opaque panels (glass-look via borders + layered shadow, NO `backdrop-filter`, NO translucency), 3D metallic cards, animated SVG constructions, on themed backgrounds with animated gradient orbs and smooth ambient effects. The default theme is "Aurora Scholar" (blue wave texture + indigo/violet/cyan orbs); 4 alternative light themes are available (清雅湖蓝, 柔紫轻盈, 薄荷清新, 暖黄纸感) for visual variety across different problem types — see the Background Theme Catalog in design-system.md. ## Prerequisites (环境准备 — 开工前必查) This is a **skill-only** capability (no MCP server), so its runtime dependencies are **NOT** auto-installed by `uvx`. Verify ALL of the following before Step 0 — a missing one silently breaks a later step: | Dependency | Needed for | Install / check | |------------|-----------|-----------------| | **Node.js + npm/npx** | scaffold + render (`npx hyperframes`) | `node -v` (≥18) | | **hyperframes CLI** | `init` / `lint` / `validate` / `render` | pulled on demand via `npx hyperframes` (needs npm-registry access at scaffold time; the project then pins a version in `dist/package.json`) | | **Headless Chromium + OS libs** | `npx hyperframes render` (puppeteer) + post-render QA gates (`postcheck.py` / `precheck.py` drive headless Chrome) | the browser itself is auto-downloaded by puppeteer on first `npx hyperframes`; on **minimal Linux** you must also `apt install libnss3 libatk-bridge2.0-0 libgbm1 libasound2 libxkbcommon0 libgtk-3-0 fonts-noto-cjk` (else Chrome fails to launch, or CJK/formulas render as tofu boxes). Reuse a system Chrome via `export PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium`. | | **Python 3 + pip** | TTS script | `python3 -m pip --version` | | **`dashscope` `soundfile` `numpy` `requests` `pydantic`** | TTS synthesis, assembly and shared configuration | `python3 -m pip install dashscope soundfile numpy requests 'pydantic>=2.11,<3'` | | **ffmpeg** | loudness normalization (`loudnorm`) + frame extraction for self-check | `brew install ffmpeg` / `apt install ffmpeg` | | **`DASHSCOPE_API_KEY`** | Qwen-TTS (`qwen3-tts-flash`) | `export DASHSCOPE_API_KEY="sk-xxx"`, **or** put `DASHSCOPE_API_KEY=sk-xxx` in `~/.qwen-mm-plugins/config` for GUI-launched setups that don't inherit shell exports. `$EDU_SKILL_ROOT/scripts/generate_voice.py` reads it at runtime — **never `cat`/paste the key into the conversation.** | > **Network boundary:** `npx hyperframes init` and the TTS calls need internet. The *render* itself is air-gapped — that is why fonts / KaTeX / GSAP must be self-hosted into `dist/` (see Step 5 Prerequisites). DashScope TTS may rate-limit under high concurrency; if you hit `Throttling.RateQuota`, lower the thread-pool worker count and add backoff (see step-3). Before Step 0, resolve `EDU_SKILL_ROOT` to the **absolute directory containing this `SKILL.md`**. All shipped scripts and assets must be addressed through that root; they are not in the user's project. Shell tool calls do not necessarily share state, so set it in every command block that uses it (or substitute the resolved absolute path directly); never rely on an earlier shell invocation: ```bash EDU_SKILL_ROOT="<absolute directory containing qwen-mm-plugins-edu-agent/SKILL.md>" test -f "$EDU_SKILL_ROOT/scripts/precheck.py" || { echo "invalid EDU_SKILL_ROOT"; exit 1; } ``` ## Pipeline Overview | Step | Name | Artifact | Reference | |------|------|----------|-----------| | 0 | Image Input & Problem Extraction | `PROBLEM.md` | [step-0-image-input.md](references/step-0-image-input.md) | | 1 | Problem Analysis | `ANALYSIS.md` | [step-1-problem-analysis.md](references/step-1-problem-analysis.md) | | 2 | Teaching Script | `SCRIPT.md` | [step-2-teaching-script.md](references/step-2-teaching-script.md) | | 3 | Voice Generation | `narration.wav` + `transcript.json` + `captions.json` (sentence-by-sentence TTS, no Whisper) | [step-3-voice-generation.md](references/step-3-voice-generation.md) | | 4 | Storyboard | `STORYBOARD.md` | [step-4-storyboard.md](references/step-4-storyboard.md) | | 5 | Build Components | `compositions/*.html` | [step-5-build-components.md](references/step-5-build-components.md) | | 6 | Compose & Render | `index.html` + MP4 | [step-6-compose-render.md](references/step-6-compose-render.md) | ## Step 0: Image Input & Problem Extraction Read [references/step-0-image-input.md](references/step-0-image-input.md). Read all images from the `image_assets/` folder using the `Read` tool (Claude's multimodal vision directly interprets image content). Extract the complete problem text, convert all math expressions to LaTeX, and describe any figures or diagrams. If JSONL metadata is available (subject, sub_subject, question_type, stepwise_explanation), use it as context hints but treat the image as ground truth. <HARD-GATE> `PROBLEM.md` must exist with: complete problem text in Chinese, all math expressions in LaTeX, and figure descriptions (if applicable). All images in `image_assets/` must have been read. </HARD-GATE> ## Step 1: Problem Analysis Read [references/step-1-problem-analysis.md](references/step-1-problem-analysis.md). Parse the input math problem, classify its type, extract knowledge points, and produce a complete solution outline with numbered steps. When `PROBLEM.md` exists from Step 0, use it as the primary input source. <HARD-GATE> `ANALYSIS.md` must exist with: problem statement, type classification, solution steps, and final answer — all verified for correctness. </HARD-GATE> ## Step 2: Teaching Script Read [references/step-2-teaching-script.md](references/step-2-teaching-script.md). Write the Chinese narration script with scene markers. Apply math symbol pronunciation rules. Target pacing: 3.5-4.0 Chinese characters per second. <HARD-GATE> `SCRIPT.md` must exist with scene-separated narration text in Chinese. All math symbols converted to spoken Chinese. </HARD-GATE> ## Step 3: Voice Generation Read [references/step-3-voice-generation.md](references/step-3-voice-generation.md). TTS strategy: **DashScope Qwen-TTS via the official SDK** (`dashscope.MultiModalConversation`, model `qwen3-tts-flash`, HTTP — returns a WAV URL) — no self-hosted TTS node, no URL wiring. **Speed + accuracy:** synthesize every sentence **concurrently** through a thread pool (wall-clock ≈ the slowest single sentence, not the sum), then measure each returned clip's real duration for exact per-sentence timestamps. No Whisper dependency. Generate standard Mandarin TTS audio. Each sentence's exact duration is measured from its returned audio clip, producing both `narration.wav` and `transcript.json` in one pass with 100% accurate timestamps. No Whisper transcription needed. <HARD-GATE> `narration.wav`, `transcript.json`, and `captions.json` must exist. Audio must be loudness-normalized with `ffmpeg loudnorm` (EBU R128, -16 LUFS). Audio is generated with DashScope Qwen-TTS (`dashscope.MultiModalConversation`, model `qwen3-tts-flash`) — `DASHSCOPE_API_KEY` required. Timestamps are measured from TTS output (not estimated). Text in transcript/captions comes from the original script. Timestamps mapped to scene boundaries. </HARD-GATE> ## Step 4: Storyboard Read [references/step-4-storyboard.md](references/step-4-storyboard.md). Read [design-system.md](design-system.md) for visual tokens. Design per-scene visual layout, assign component templates from [math-components.md](references/math-components.md), and plan transitions. Check [assets/ASSET_CATALOG.md](assets/ASSET_CATALOG.md) to identify which pre-built visual components can be reused in each scene. **Select a background theme** from the Background Theme Catalog in design-system.md — set it in the Global Direction block (default: `aurora-scholar`). <HARD-GATE> `STORYBOARD.md` must exist with per-scene direction: component type, layout, animation choreography, and transition choice. </HARD-GATE> ## Step 5: Build Components Read the `hyperframes` skill — every composition authoring rule applies. Read the `gsap` skill for animation patterns. Read [references/step-5-build-components.md](references/step-5-build-components.md). Read [references/math-components.md](references/math-components.md) for component templates. Read [assets/ASSET_CATALOG.md](assets/ASSET_CATALOG.md) for pre-built visual components. **For geometry diagrams (几何图形)**, read [references/geometry-construction-guide.md](references/geometry-construction-guide.md) for coordinate computation patterns, angle arc construction, and complete worked examples (triangles, quadrilaterals, circles, rotations, reflections); also read [references/golden-example-geometry.md](references/golden-example-geometry.md) for a complete golden pipeline (PROBLEM → HTML compositions) showing the proven split layout, draw-on animation choreography, and narration-synced note blocks from a top-quality geometry proof video. **For circuit schematic diagrams (电路图)**, read [references/circuit-schematic-guide.md](references/circuit-schematic-guide.md) for physics rules and SVG symbol templates. **For any scene that must ANIMATE A PROCESS with real motion (生物过程/受力矢量/波动/滴定/运动演示 — things that split, move, get pulled, or a quantity that changes across stages), read [references/example-process-animation.md](references/example-process-animation.md)** — a top-scoring few-shot (洋葱有丝分裂) with two complete reference scenes and copy-me techniques: staged phase reveal, split-and-move, **connectors that track a moving object and shorten (纺锤丝/绳/矢量牵引)**, live count-up, and a stepped quantity-vs-stage chart (plus the no-360°-spin rule). **🧬 MANDATORY for ANY 染色体/细胞分裂题目 (有丝分裂/减数分裂/染色单体/着丝点/纺锤丝/移向两极): you MUST open and copy [references/examples/mitosis-anaphase.scene.html](references/examples/mitosis-anaphase.scene.html)** — the spindle fibers must be animated `<line>`s that TRACK the moving chromatids and SHORTEN (pull them to the poles), with a live 染色体数 count-up, and NO 360° spin on chromosomes. This is enforced at render time by `scripts/check_chromosome_example.py` (a chromosome cell-division scene without the fiber-shorten tween, or with a 360° spin, FAILS precheck). Scaffold the project with `npx hyperframes init dist --non-interactive --example blank` (the `--example blank` flag is required — hyperframes ≥0.7.77 rejects a bare `--non-interactive`). **The project root is the current workspace and the scaffold dir MUST be exactly `dist/` at that root — never nest it inside another folder (e.g. NOT `math-tutorial-output/dist`); the renderer and all gates assume `./dist`.** **Build ONE composition per scene listed in `STORYBOARD.md` — one special ray / one solution step / one concept per scene; NEVER cram several planned scenes into a single composition (that is the #1 cause of under-rendered, blank, or half-empty scenes and of run-to-run instability).** **Before writing custom HTML for any visual object**, check the [assets/](assets/) directory — it contains 83 pre-built K12 components (motion, optics, circuit, mechanics, fluid, chemistry, wave, indicators, math), including 6 circuit schematic symbols (`sch-battery`, `sch-ammeter`, `sch-voltmeter`, `sch-switch`, `sch-bulb`, `sch-resistor`). If a matching component exists, copy its CSS + HTML + JS hooks verbatim into the composition instead of building from scratch. <HARD-GATE> **>>> MANDATORY: run precheck and loop until it passes <<<** ```bash EDU_SKILL_ROOT="<absolute directory containing qwen-mm-plugins-edu-agent/SKILL.md>" python3 "$EDU_SKILL_ROOT/scripts/precheck.py" dist ``` This script auto-fixes LaTeX escaping, then runs all validation checks. **You MUST see `ALL CHECKS PASSED` in the output before proceeding to Step 6.** If any check shows `FAIL`: 1. Read the FAIL message — it prints the exact file, line, and fix instruction 2. Apply the fix to the offending file(s) 3. Re-run `python3 "$EDU_SKILL_ROOT/scripts/precheck.py" dist` 4. **Repeat steps 1-3 until the output contains `ALL CHECKS PASSED`** **Do NOT proceed to Step 6 with failing checks. Do NOT skip precheck. Do NOT treat precheck failures as warnings.** --- The precheck validates all of the following (each is also a standalone script in `scripts/`): - **Asset mirror** — GSAP/KaTeX/fonts mirrored into `compositions/` (sub-compositions resolve `./` relative to their own directory; without the mirror, GSAP/KaTeX/fonts silently fail → blank scenes) - **Composition format** — sub-compositions are HTML fragments, not full `<!doctype>` documents (full docs → blank panels) - **No CDN URLs** — no `cdn.jsdelivr.net` or external `src`/`href` (render sandbox is air-gapped → all external loads fail silently) - **KaTeX CJK** — no Chinese inside `katex.render()` or `data-tex` (KaTeX fonts lack CJK → tofu) - **CJK font on Chinese text (中文必须用中文字体)** — any SVG `<text>`/`<tspan>` or inline-styled element whose text contains Chinese MUST use a stack including `Noto Sans SC` (e.g. `font-family="Noto Sans SC, Inter, sans-serif"`); `Inter`/`sans-serif` alone has no CJK glyphs so Chinese renders as **NO GLYPH / 豆腐块**. Don't mix scripts in one `<text>` — split Latin (Inter) and Chinese (Noto Sans SC) into separate `<tspan>`/`<text>`. Gate: `scripts/check_cjk_font.py` - **KaTeX escaping** — LaTeX in JS strings double-escaped (`\\\\dfrac`, not `\\dfrac`; JS eats single `\\d` → "dfrac12" instead of fraction) - **Unrendered fractions** — no literal "dfrac"/"frac{" leaked into visible HTML text - **Caption size** — font-size 36–40px (not 64px giant subtitles) - **Caption pinned to bottom (字幕固定在视频下方)** — every caption bar lives ONLY in `index.html`'s root track and is styled `position:absolute; bottom:48px; left:50%; transform:translateX(-50%)`; NEVER use `top:`, and NEVER put a caption inside a scene composition (it would drift to the top/middle). Gate: `scripts/check_caption_position.py` - **Caption safe zone** — every scene reserves the bottom ~180px (content in the top ~900px) so the subtitle never covers content (字幕遮挡) - **Caption overflow** — the caption bar is width-bounded (`max-width:1600px`, never `white-space:nowrap`), uses `width:max-content` (so it uses the full width before wrapping instead of shrinking to the ~960px half-frame), and each cue is short (split long sentences into sequential one-line cues); the subtitle never runs off the frame edge (字幕超出边界). Gate: `scripts/check_caption_overflow.py` - **Caption always on top (字幕必须在最顶层)** — the caption bar has an unbeatable `z-index` (`2147483647`) so it is ALWAYS above every scene layer/panel and can never be covered (字幕被遮挡); scenes must keep their z-index small (<100). Gate: `scripts/check_caption_overflow.py` - **Scene layout** — `.scene-content` is a centering box that fills the frame (`position:absolute;inset:0;display:flex;align-items:center;justify-content:center`); filling alone (no flex-center trio) piles a single panel at the TOP with the bottom half empty (排版问题). Gate: `scripts/check_scene_layout.py` - **No oversize/overflow (排版过大)** — every element fits within 1920×1080 (content within the 1920×900 safe area with edge margins); no CSS `width`>1920px / `height`>1080px, keep `scale()` ≤ ~1.5; size by container (%/max-width/flex/grid), scale SVG via viewBox. Hero text ≤ ~96px is guidance (decorative watermarks may be larger). Gate: `scripts/check_scene_overflow.py`