| name | carocut-media-visual |
| description | 视觉素材获取与生成。根据 resources.yaml 定义，通过库存图片/视频搜索（Pexels/Pixabay）或 AI 生图（Caro LLM）获取视觉素材。包含搜索策略、视觉隐喻搜索（counterpoint 声画关系）、构图感知搜索（framing 字段）、prompt 工程指南、素材质量评估标准。 |
Visual Assets
Retrieves stock images and generates custom images for video production. This step fulfills visual resource requirements defined in manifests/resources.yaml.
Decision Flow
digraph visual_decision {
"Resource from resources.yaml" [shape=doublecircle];
"source = existing?" [shape=diamond];
"Already in raws/" [shape=box];
"source = retrieve?" [shape=diamond];
"Search stock images" [shape=box];
"source = generate?" [shape=diamond];
"Generate with AI" [shape=box];
"Needs transparency?" [shape=diamond];
"Remove background" [shape=box];
"Save to raws/" [shape=box];
"Update resources.yaml" [shape=doublecircle];
"Resource from resources.yaml" -> "source = existing?";
"source = existing?" -> "Already in raws/" [label="yes"];
"source = existing?" -> "source = retrieve?" [label="no"];
"source = retrieve?" -> "Search stock images" [label="yes"];
"source = retrieve?" -> "source = generate?" [label="no"];
"source = generate?" -> "Generate with AI" [label="yes"];
"Search stock images" -> "Needs transparency?";
"Generate with AI" -> "Needs transparency?";
"Needs transparency?" -> "Remove background" [label="yes"];
"Needs transparency?" -> "Save to raws/" [label="no"];
"Remove background" -> "Save to raws/";
"Already in raws/" -> "Update resources.yaml";
"Save to raws/" -> "Update resources.yaml";
}
Resource Source Types
资源类型完整定义见 carocut-shared-schema skill。Media 只处理 visual 类型（image/video/sprite），不处理 data 和 components 类型。
| Source | When to Use |
|---|
existing | User-provided materials already in raws/ |
retrieve | Objective backgrounds, stock photos, stock videos (Pexels/Pixabay) |
generate | Custom characters, foreground elements with transparency |
Sprite Sheets (Recommended for Character Animation)
When the storyboard includes animated characters, mascots, or any element with multiple poses/actions, prefer generating sprite sheets over individual images. Sprite sheets provide:
- Consistent character appearance across all animation frames
- Smooth frame-by-frame animation in Remotion
- Efficient asset management (one file per character instead of many)
Video Resources (视频素材)
When the storyboard requires motion footage (product demos, b-roll, ambient shots), use type: video in resources.yaml:
- id: demo_recording
type: video
description: "产品演示录屏"
source: retrieve
search_keywords: ["product demo", "screen recording"]
duration_range: "5-15s"
Video Search Strategy
- Use Pexels for STOCK video retrieval when using tool
videos_search
- Search keywords follow the same English-keyword rules as image search
- Filter by duration range when specified in the resource definition
- Prefer HD (1080p) or higher resolution
Video Output
Video in resources.yaml
- id: vid_001
type: video
description: "Aerial city timelapse"
source: retrieve
search_keywords: ["aerial city", "timelapse", "urban"]
duration_range: "5-10s"
path: videos/retrieved/aerial_city_001.mp4
Image Retrieval (Stock Photos)
Search royalty-free images from Pexels and Pixabay.
Search Keywords Tips
| Need | Keywords |
|---|
| Video backgrounds | abstract, texture, gradient, bokeh, blur |
| Business | office, meeting, professional, workspace |
| Nature | mountain, forest, ocean, sunset, aerial |
| Technology | technology, computer, coding, digital |
| Minimal | minimal, white background, flat lay, clean |
Image Search Strategy
- Use English keywords for all searches
- Keep search result count low (3-5) to avoid context bloat
- Use specific keywords with style descriptors for better relevance
- When initial results are poor, expand with synonyms (e.g., "server room" -> "data center", "network infrastructure")
- Filter by orientation when the layout requires it (landscape for backgrounds, portrait for mobile)
- Filter by color when brand consistency matters
Visual Metaphor Search (视觉隐喻搜索)
When the storyboard specifies audio_visual_relation: counterpoint, the visual should not literally illustrate the narration. Instead, switch to metaphorical/associative search:
| Narrative Concept | Metaphor Search Keywords |
|---|
| 信息过载 | overflowing water, paper avalanche, cluttered desk |
| 快速增长 | sprouting plant timelapse, rising tide, rocket launch |
| 技术落后 | vintage technology, old computer, floppy disk |
| 复杂度 | tangled wires, maze, puzzle pieces |
| 突破/创新 | breaking through wall, sunrise, butterfly emerging |
| 竞争 | race track, chess game, climbing mountain |
| 连接/协作 | bridge, handshake, puzzle fitting together |
| 衰退/消失 | wilting flower, melting ice, fading light |
Rules:
- Only activate metaphor search when
audio_visual_relation: counterpoint
- For
sync or lead-visual, use literal/direct search as usual
- For
lead-audio, visuals can be more abstract/atmospheric but not necessarily metaphorical
- When using metaphor search, pick 2-3 keywords from the table and combine with style descriptors
- If the narrative concept is not in the table, derive a visual metaphor by analogy
Framing-Aware Search (构图感知搜索)
Adjust search keywords based on the storyboard's framing field to find images matching the intended shot composition:
| Framing | Append to Search Keywords |
|---|
ECU (Extreme Close-Up) | "close up", "detail", "macro", "texture" |
CU (Close-Up) | "close up", "detail", "macro", "texture" |
MCU (Medium Close-Up) | (no modification, default search) |
MS (Medium Shot) | (no modification, default search) |
LS (Long Shot) | "wide shot", "landscape", "panoramic", "aerial" |
ELS (Extreme Long Shot) | "wide shot", "landscape", "panoramic", "aerial" |
Rules:
- Append the framing keywords to the existing search query (do not replace)
- For
MCU and MS, no framing keywords are needed
- Combine with metaphor search when applicable (e.g., counterpoint + ECU → "close up texture tangled wires")
License Information
| Platform | License | Attribution | Commercial Use |
|---|
| Pexels | Pexels License | Appreciated but not required | Yes |
| Pixabay | CC0 / Pixabay License | Not required | Yes |
Image Generation (AI)
Generate custom images via OpenAI-compatible API.
Prompt Rules
- Write all prompts in English
- For subjects needing transparency: Include "on a solid white background" or "on a plain white background"
- For Chinese text in image: Use format
sign that reads "Chinese text here"
- Be specific: Describe composition, style, colors, lighting, perspective
Prompt Examples
Background image:
Abstract gradient background, soft blue to purple transition, subtle geometric patterns, clean modern aesthetic, suitable for tech presentation
Character with transparency:
A friendly 3D cartoon scientist character, female, wearing white lab coat and glasses, standing pose facing camera, on a plain white background, Pixar style rendering
Sprite sheet:
Grid: 8 columns x 4 rows.
Row 1: Idle - character breathing subtly
Row 2: Walking - 8-frame walk cycle from left
Row 3: Running - 8-frame run cycle
Row 4: Jumping - jump up and land sequence
Character: cute robot with round blue body, small antenna, expressive digital eyes.
Style: 3D cartoon, soft lighting, clean design.
Background Removal
Remove background from generated images to create transparent subjects. Works best when the original image has a clean, solid-color background.
Use Cases
- Characters/subjects generated on white background that need transparency
- Stock photos where only the subject is needed
- Foreground elements that will be composited over video backgrounds
Sprite Sheet Validation (MANDATORY)
After generating a sprite sheet, validation is performed automatically by the images_generate tool when using mode: sprite. You can also run validation independently with the images_validate_sprite tool.
Automatic Validation (Built-in to Generate)
When using images_generate with mode: sprite:
- Grid dimensions auto-detected from the prompt (patterns like "8 columns x 4 rows"), or explicitly via
cols/rows parameters
- Post-generation validation runs automatically:
- Checks image dimensions are evenly divisible by cols/rows
- Checks magenta chroma key coverage >= 15%
- Auto-retry on failure (up to
max_retries times, default 3):
- Each retry enhances the prompt with progressively stronger dimension constraints
- Adds explicit pixel resolution requirements (e.g., "output MUST be 4096x2048")
- Reports final validation result in JSON
Recommended usage:
images_generate(
prompt="Grid: 8 columns x 4 rows. ...",
output="/path/to/sprite.png",
mode="sprite",
cols=8, # explicit is more reliable than auto-inference
rows=4,
)
Independent Validation Tool
Use images_validate_sprite for post-hoc validation or re-checking:
images_validate_sprite(
image="/path/to/sprite.png",
cols=8,
rows=4,
fix_chroma=true, # optional: replace magenta with transparency
output="/path/to/sprite_transparent.png",
extract_frames="/path/to/frames/", # optional: extract individual frames
)
Returns JSON with check results:
{
"success": true,
"checks": [
{"check": "dimensions", "passed": true, "frame_size": "512x512"},
{"check": "aspect_ratio", "passed": true, "frame_aspect_ratio": 1.0},
{"check": "chroma_key", "passed": true, "magenta_coverage": 42.3}
]
}
Sprite Sheet Failure Handling
If sprite sheet generation fails after all retries:
- Log failure details (prompt, dimensions, error) in execution summary
- Do NOT silently skip — report the failure to orchestrator as
partial_failure
- Suggest alternatives to the user:
- Retry with simplified prompt (fewer cells, simpler subjects)
- Fall back to static images instead of sprite animation
- Provide alternative asset manually
- Continue processing remaining (non-failed) resources
Manual Fallback Validation
If the tools are unavailable, use Python/Pillow to verify:
from PIL import Image
img = Image.open("raws/images/generated/robot_sprite.png")
cols, rows = 8, 4
w, h = img.size
assert w % cols == 0, f"Width {w} not divisible by {cols} columns"
assert h % rows == 0, f"Height {h} not divisible by {rows} rows"
frame_w, frame_h = w // cols, h // rows
print(f"Sprite: {w}x{h}, Frame: {frame_w}x{frame_h}, Grid: {cols}x{rows}")
Validation Checklist
| Check | Expected | Auto-handled | Manual Action if Fails |
|---|
| Width divisible by columns | width % cols == 0 | Yes (auto-retry with enhanced prompt) | Regenerate with explicit resolution |
| Height divisible by rows | height % rows == 0 | Yes (auto-retry with enhanced prompt) | Regenerate with explicit resolution |
| Frame aspect ratio ~1:1 | ratio ≈ 1.0 ± 0.3 | Warning only | Adjust cols/rows ratio |
| Magenta coverage > 15% | Background is clean | Yes (auto-retry) | Regenerate with stronger magenta instruction |
| No magenta on character | Character pixels distinct | Not auto-checked | Add "avoid magenta/pink colors on character" to prompt |
| Frame-to-frame continuity | Smooth motion | Warning only | Regenerate with "smooth transition" emphasis |
Sprite Metadata in resources.yaml
After validation, record sprite metadata for downstream consumption:
- id: vis_sprite_001
type: sprite
description: "Robot character animation"
source: generate
path: images/generated/robot_sprite.png
sprite:
cols: 8
rows: 4
frameWidth: 512
frameHeight: 512
chromaKey: "#ff00ff"
animations:
idle: { row: 0, frameCount: 8 }
walk: { row: 1, frameCount: 8 }
run: { row: 2, frameCount: 8 }
jump: { row: 3, frameCount: 8 }
Output Structure
raws/
images/
existing/ # User-provided (unchanged)
logo.png
retrieved/ # From Pexels/Pixabay
server_room_001.jpg
abstract_bg_002.jpg
generated/ # From Caro LLM API
robot.png # With transparency
scientist.png # With transparency
background_tech.png # No transparency needed
robot_sprite.png # Sprite sheet (magenta bg)
videos/
retrieved/ # From Pexels Video API
product_demo_001.mp4
aerial_city_002.mp4
Workflow
- Read
manifests/resources.yaml to identify visual resources
- For each resource with
source: retrieve:
- Search stock images with appropriate keywords
- Download to
raws/images/retrieved/
- For each resource with
source: generate:
- If character/mascot with animation needs: use sprite mode (recommended)
- Craft appropriate prompt based on description
- Generate the image:
- Normal mode: Single generation, no validation
- Sprite mode: Use
images_generate with mode: sprite, cols, rows params. Built-in auto-validation and retry handles dimension/chroma issues.
- If needs transparency (non-sprite), remove background
- Sprite post-processing: After successful generation, optionally use
images_validate_sprite with fix_chroma=true to convert magenta to transparency
- Save to
raws/images/generated/
- Update
resources.yaml with actual file paths (include sprite metadata for sprite sheets)
- Report results to user
Incremental Mode
当 dispatch context 中 mode: "incremental" 时：
输入
- amendment.affected_resources 中指定的新增视觉资源 ID 列表
执行规则
- 读取 resources.yaml，仅定位指定的新增资源条目
- 按 source 类型（retrieve/generate）执行获取或生成
- 如需透明背景，执行背景移除
- 如为 sprite sheet，执行完整验证流程
- 更新 resources.yaml 中对应条目的 path 字段
- 返回处理摘要（获取/生成了哪些资源）
不执行
- 不重新扫描已有的视觉资源
- 不重新下载或重新生成已存在的素材
- 不修改未指定资源的 path 或元数据
Updating resources.yaml
After retrieving/generating images, update the resource entry with the actual path:
Before:
- id: vis_001
type: image
description: "Server room with blue lighting"
source: retrieve
After:
- id: vis_001
type: image
description: "Server room with blue lighting"
source: retrieve
path: images/retrieved/server_room_001.jpg
User Communication
Progress Report
视觉素材增强进行中...
处理中: vis_003 (server room background)
来源: Pexels
状态: 下载完成
Completion Report
视觉素材增强完成。
检索素材:
- 3 张图片 (Pexels)
- 2 张图片 (Pixabay)
生成素材:
- 2 张图片 (带透明背景)
- 1 张背景图
已更新: manifests/resources.yaml
输出目录: raws/images/
- retrieved/ (5 个文件)
- generated/ (3 个文件)
请确认素材后继续下一步。
Common Issues
| Issue | Cause | Solution |
|---|
| API key not found | Environment variable not set | Set PEXELS_API_KEY, PIXABAY_API_KEY, or CARO_LLM_API_KEY |
| Background removal messy | Original image has complex background | Use "on plain white background" in generation prompt |
| Search returns irrelevant results | Query too vague | Use specific English keywords, add style descriptors |
| Generated image style inconsistent | Prompt lacks detail | Specify art style, lighting, and composition clearly |
Notes
- Always use English for search queries and generation prompts
- Keep search result count low (3-5) to avoid context bloat
- For characters/subjects, always generate on white background then remove
- For animated characters, prefer sprite sheets -- they produce more consistent results than generating individual frames separately
- After generating sprite sheets, always validate frame splitting before proceeding
- Update resources.yaml with actual paths after processing
- All image paths in resources.yaml are relative to raws/
