video2dsprite

name	video2dsprite
description	Grok Build only. Turn a 2D character still into denser animation sprites via imagine_image base → imagine_video (6s/10s run-in-place) → ffmpeg frames → magenta chroma-key → dense sampled strips/grids/GIFs. Use when the user wants video-to-sprite, smoother run/walk cycles, or denser intermediate poses. Prefer generate2dsprite for crisp production pixel sheets. Triggers on "video to sprite", "imagine_video sprite", "dense walk cycle", "smooth run animation from video".
metadata	{"short-description":"Video→dense sprites (imagine_video + chroma postprocess)"}
user-invocable	false

Video2dsprite (Grok Build only)

Convert a base 2D character image into dense animation sprites using Grok Build's native video tools.

App-builder / Grok environment

Item	Value
Skill dir	.grok/skills/video2dsprite/
Scripts	python3 .grok/skills/video2dsprite/scripts/video2dsprite.py …
Video tools	imagine_video — animate one base asset_id (verify present)
Inspect	read_file on stills/frames; report paths for videos
Deps	ffmpeg + Pillow + numpy (preinstalled in app-builder image)
Output home	assets/sprites/video2dsprite/<name>/ under /workspace
Default	Prefer generate2dsprite for production heroes; this is the denser-motion path

base still → imagine_video (in-place motion) → extract frames → chroma key → sample/normalize → strip / grid / GIF

Platform gate (read first)

Runtime	Supported?
Grok Build (xAI)	Yes — requires an image generator + an image→video tool
Codex / Claude / other agents	No — they lack Grok video tools. Tell the user this skill is Grok Build only and offer $generate2dsprite instead

Gate on the capability, not the exact tool name. Image generation appears as imagine_image, or as generate_image / edit_image; video as imagine_video or generate_video. Use whichever pair your tool list has — the split tools take filesystem paths where this skill says asset_id, so the imagine_create_asset / imagine_view_media steps don't apply. Stop and explain only when you have no image→video tool at all. Do not fake motion with code-drawn frames.

This skill is an optional denser-motion path. It does not replace $generate2dsprite:

Use $generate2dsprite when…	Use $video2dsprite when…
Crisp pixel sheets, fixed grids, identity-critical heroes	User wants denser intermediate poses / smoother feeling loops
Attack/cast body sheets, prop packs, engine atlases	Experimenting with video-sourced run/walk/idle motion
Production default for most game sprites	User explicitly asks for video → frames → sprites

Video softens pixels, drifts identity, and leaves chroma fringes. Always QC; for production heroes, prefer $generate2dsprite unless the user wants the video look.

Parameters

Infer from the user request:

• subject: character / creature description, or path to existing still
• action: run | walk | idle | attack | custom motion phrase
• view: usually side (side-scroller). topdown is harder — warn and keep camera locked
• duration: 6 (default) or 10 seconds
• frame_counts: which denser sets to export, default 8,16,24,48
• cell_size: output sprite cell, default 128
• anchor: feet (default for side locomotion) | center
• bg: solid #FF00FF (required for chroma)
• name: output slug
• out_dir: working folder (default ./sprites/video2dsprite/<name>/ or project-relative)

Agent rules

1. Grok-only. Refuse on non-Grok runtimes with a short explanation + $generate2dsprite alternative.
2. Still → video, never text-to-video alone. Stage frame 1 as a clean still with imagine_image (from a prompt, or from a reference asset_id). Then call imagine_video with that still's asset_id.
3. In-place motion. Prompt for run/walk in place facing a fixed direction. No camera pan, no background scroll, no scene change. Subject stays roughly centered.
4. Solid magenta background on the base and preserved in the video prompt (#FF00FF / pure magenta). Required for flood-fill chroma.
5. Do not invent art with PIL/Canvas. Base art comes from imagine_image or a user/local still. Scripts only postprocess.
6. Do not put experimental outputs into the game unless the user asks to integrate.
7. Prefer one locomotion cycle for game use. Dense sample across a full 6s multi-cycle clip is fine for previews; for engine sheets, optionally re-sample a single cycle (12–16 frames) after visual QC.
8. Report absolute paths of video, cleaned frames, strips, and preview GIFs when done.

Workflow

1. Plan

Pick the smallest useful run:

• Side-view run/walk loop → this skill
• Multi-action hero kit → still use $generate2dsprite per action; only use video for locomotion if requested
• FX / projectile / prop packs → $generate2dsprite, not video

Create:

<out_dir>/
  base/
  video/
  frames-raw/
  frames-clean/
  sprite/          # default 8-frame set + denser x16/x24/x48
  prompt-used.txt
  pipeline-meta.json
  README.txt

2. Build the base still

Options:

• A. Existing sprite: read the image, composite onto solid #FF00FF if needed, then imagine_create_asset so Imagine can use it
• B. New character: imagine_image with solid magenta background, full body, side view, centered
• C. Match reference: imagine_image with the reference asset_id, moving it onto magenta and preserving identity

Base requirements:

• Full body visible, generous magenta margin
• Side view for run/walk (profile or 3/4 side), feet near bottom third
• Same art style as the rest of the project when a reference exists
• No text, UI, watermark, or second character

Download it with imagine_view_media and save as <out_dir>/base/<name>-base.png; keep the base asset_id for the video call.

Write the exact image prompt into prompt-used.txt.

3. Animate with imagine_video

Call Grok imagine_video:

• asset_ids: [<base still asset_id>] (not a filesystem path)
• duration: 6 (default) or 10
• resolution_name: 480p unless user asks 720p
• prompt: one short present-tense shot (see references/prompt-rules.md)

Mandatory motion constraints in the prompt:

• Subject runs/walks in place (treadmill style)
• Camera locked — no pan, zoom, or orbit
• Background stays flat solid magenta
• Identity, costume, palette stable for the whole shot
• Single continuous action only

Download the returned video asset with imagine_view_media and copy it to <out_dir>/video/<name>-<duration>s.mp4.

If video tools are unavailable, stop (platform gate).

4. Extract + chroma + sample (local script)

Run the processor (ffmpeg + Pillow + numpy):

python3 .grok/skills/video2dsprite/scripts/video2dsprite.py process \
  --video <out_dir>/video/<name>-6s.mp4 \
  --out-dir <out_dir> \
  --name <name> \
  --frame-counts 8,16,24,48 \
  --cell-size 128 \
  --body-height 100 \
  --foot-y 118 \
  --fps 0

Notes:

• --fps 0 = extract every decoded frame (use source fps)
• Magenta flood-fill from corners + despill
• Even sampling for each count in --frame-counts
• Feet-normalized cells, horizontal strip, grid, loop GIF per count

Optional: only re-sample denser sets from existing cleaned frames:

python3 .grok/skills/video2dsprite/scripts/video2dsprite.py sample \
  --clean-dir <out_dir>/frames-clean \
  --out-dir <out_dir> \
  --frame-counts 16,24,48 \
  --cell-size 128

5. QC

Visually check:

• Preview GIF loops without huge pops
• Magenta gone (no solid pink blocks); fringe acceptable or re-key
• Feet stay on a stable baseline (no hop from bad crop)
• Identity roughly stable (face/clothes not morphing every frame)
• Action is in-place (not sliding out of frame)
• For game use: pick one count (often 16 or 24) or cut one true cycle

If identity drifts hard or pixels are too soft, fall back to $generate2dsprite for production sheets and keep the video set as motion reference only.

6. Deliver

Report paths only (unless user asked to wire into a game):

• Video: video/*.mp4
• Dense sprites: sprite/x16|x24|x48/
• Strips / grids / GIFs: sprite/run-strip-N.png, run-grid-N.png, run-preview-N.gif
• Meta: pipeline-meta.json

Do not modify game code unless requested.

Defaults

• Duration: 6s
• Action: side run in place, facing right
• Export counts: 8, 16, 24, 48
• Cell: 128², body height ~100, feet at y≈118
• Background: #FF00FF
• Prefer single-asset imagine_video over multi-ref (compose multi-ref with imagine_image first if needed)

Tradeoffs (tell the user once)

Pros: denser intermediates → often feels smoother than 4–8 discrete gen poses.
Cons: softer pixels, identity drift, chroma fringe, multi-cycle 6s clips are not a single perfect loop, heavier assets.
Rule of thumb: 8→16→24 usually gains smoothness; 48 is often diminishing returns; 145 raw frames are for sampling, not all for runtime.

Resources

• references/prompt-rules.md — base still + video prompts
• references/pipeline.md — folder layout, ffmpeg, sampling strategy
• scripts/video2dsprite.py — extract, chroma, normalize, export

Relationship to other skills

• $generate2dsprite — primary sheet pipeline (Codex + Grok when image gen exists)
• $generate2dmap — maps; not used here
• $video2dsprite — Grok Build exclusive motion densification path
• $game-asset-core / $game-animation-frames — loop/flip-test/motion laws and engine-ready defaults; still use this skill’s scripts for sandbox execution (magenta base + chroma), not a freeform background color