smart-capcut-delivery

name

smart-capcut-delivery

description

Use this skill whenever the user wants to "拉片" (shot-by-shot analyze) a video, generate a director-friendly shot-breakdown report, extract keyframes from a video, transcribe its audio, detect scene cuts, write Seedance/Kling/Jimeng-style AIGC video reproduction prompts, or produce a polished HTML delivery page from a video file. Trigger this whenever the user mentions video shot analysis, scene detection, keyframe extraction, AIGC video prompt generation, or asks to turn a video into a structured analysis report (shots.json, frames, transcript, summary, HTML page). The skill handles the deterministic pipeline (probe → detect → extract → transcribe → align → build) via scripts; the visual analysis of each shot is performed by the Claude model running this conversation, using the extracted keyframes as multimodal input.

smart-capcut-delivery

Purpose

Turn a user-provided video into a director-friendly AI shot-analysis deliverable:

• frames/ — three keyframes per shot (start / mid / end)
• shots.json — single source of truth
• shots.csv — flat tabular export
• transcript.srt — Whisper transcript
• summary.md — high-level summary
• index.html — final delivery page
• run.log — pipeline log

The deliverable should feel like a polished video-analysis report, not an engineering dashboard.

Mandatory First Step

Before any processing, confirm with the user:

1. Input video path
2. Output directory

Do not process the video until the user confirms both. If the user gives a directory rather than a file, list candidate video files and ask them to choose.

Validation rules:

• Expand ~ to absolute paths.
• Accept extensions: .mp4, .mov, .mkv, .avi, .webm, .m4v.
• Create the output directory if needed; verify it is writable.
• Never modify the original video. The pipeline soft-links it as video.mp4 (falls back to copy on Windows without symlink permission).

Standard Workflow

1. Validate input and output paths.
2. Probe video metadata with ffprobe, falling back to OpenCV when needed.
3. Create output structure and run.log.
4. Run a dry-run first: only process the first 10 shots end-to-end.
5. Detect shots with PySceneDetect.
6. Extract three keyframes per shot: start, mid, end.
7. Transcribe audio to transcript.srt (Whisper).
8. Align transcript text to shots by time overlap.
9. Build a shots.json skeleton.
10. Visually analyze each shot. See "VLM Analysis" below — the default analyzer is Claude itself.
11. Generate Seedance-style AIGC reproduction prompts per shot.
12. Generate summary.md.
13. Render the delivery HTML.
14. Validate outputs.
15. Ask the user whether to open the HTML in a browser.

Dry-run Policy

Always run a first pass over 10 shots before full processing.

The dry-run must include keyframes, transcript alignment, shot analysis fields, the AI prompt field, the delivery HTML, and output validation — i.e. the same end-to-end shape as the full run.

Ask the user to approve analysis quality and UI direction before committing to the full video.

Data Contract

Use shots.json as the single source of truth. All CSV, summary, and HTML outputs are derived from it.

For the full schema, read references/schema.md.

VLM Analysis

For each shot, analyze only the three keyframes for that shot plus its transcript and a one-line summary of the previous shot. Do not load many shots into context at once — process one shot, write partial output, then release the images.

The default visual analyzer is the Claude model running this conversation. Read each shot's frames/shot_XXXX_start|mid|end.jpg directly as multimodal input, then output a JSON object matching the schema in references/vlm-analysis.md. Append each result as one line to analysis.jsonl, then merge with scripts/merge_vlm_results.py.

If the user prefers to run analysis with a third-party VL model (GPT-4o, Gemini, Qwen-VL, GLM-4V, etc.), or wants the analysis to run unattended via API, point them to scripts/analyze_shots_with_claude.py as a reference implementation (it uses the Anthropic API and can be adapted to any provider that accepts image inputs).

For VLM field definitions and confidence rules, read references/vlm-analysis.md.

AI Prompt Generation

Generate a Seedance 2.0-style prompt for each shot. It should be natural-language, reference-driven, and directly copyable.

For prompt rules and templates, read references/seedance-prompt.md.

HTML Delivery

The final page should keep visual sections UI-like while making the shot-language and content-narrative sections field-based and copy-friendly.

For page structure and UI rules, read references/html-delivery.md.

Script Strategy

Prefer stable scripts for deterministic steps:

• Video probing
• Shot detection
• Keyframe extraction
• Transcript alignment
• shots.json assembly
• AI prompt generation
• HTML rendering
• Output validation

See scripts/README.md for the script breakdown.

Quality Gates

Before final delivery, verify:

• shots.json parses successfully.
• Every shot has start/mid/end frame paths.
• Frame files actually exist on disk.
• video.mp4 exists or is a soft link.
• HTML uses relative paths.
• low_confidence and possible_missed_cut IDs are listed.
• run.log records each stage.

Final Response

When done, report:

• Output directory
• Total shots processed
• Low-confidence shot IDs
• Possible missed-cut shot IDs
• Whether dry-run or full-run was completed
• Whether HTML was opened