image-to-cinematic-video

name

image-to-cinematic-video

description

Turn a prompt or an existing image into a polished multi-scene cinematic short video using Seedance for clip generation, uguu.se for file hosting, and FFmpeg for last-frame extraction and crossfade stitching. Supports two modes — PARALLEL (multiple scenes from the same reference image, concurrent, ~3 min total) and CHAIN (each scene starts on the last frame of the previous scene, sequential, ~3 min per scene, produces one continuous long take). Use this skill whenever the user wants to "make a video from an image", "animate this picture", "generate video scenes", "create a cinematic short", "stitch AI videos together", "make a long continuous video", "chain AI clips", or any workflow that goes from one reference image to a multi-clip stitched video. Also trigger when the user mentions Seedance, image-to-video, video stitching, crossfades between clips, frame chaining, last-frame continuity, or wants to turn a generated image into motion. Covers the full pipeline end-to-end — generate image with the chat-level `generate_image` tool (Gemini Nano Banana Pro by default) → pull the real bytes from `/api/images/:id` → upload to uguu → Seedance (with the relative-filename quirk handled) → copy output into the project folder → extract last frame → upload → chain into next scene → FFmpeg xfade stitch → upload final → write manifest.

Image → Cinematic Video Pipeline

A battle-tested, end-to-end workflow for turning one prompt (or one starting image) into a polished multi-scene stitched video. This skill only documents techniques that actually worked in production.

⚠ NON-NEGOTIABLE RULES

1. Use generate_image as the primary source-image generator. The image-ref system has been fixed — every {{IMAGE_REF:<id>}} returned by generate_image is now backed by real bytes available at GET http://localhost:3333/api/images/<id> (Bearer-auth with process.env.AGNT_AUTH_TOKEN). Use fetchImageRefBytes(ref, outPath) from save_generated_image.js to pull those bytes onto disk in one line. Default model: Gemini Nano Banana Pro (provider: 'gemini', model: 'nano-banana-pro-preview'). saveGeneratedImage() still exists as a direct-REST-API fallback when you want to bypass the chat tool entirely.

2. Every run gets its own project folder. Call createProjectDir(slug) at the top. It returns an absolute path under C:\Users\Studio\AppData\Roaming\AGNT\projects\<timestamp>_<slug>\ with scenes/ and frames/ subdirs pre-created. Every artifact (source image, each scene mp4, each last-frame jpg, final mp4, manifest.json) goes into that folder. One folder per run. No scattering files across the AGNT root.

3. Upload only to uguu.se. Catbox, Litterbox, 0x0.st, tmpfiles.org, transfer.sh, and file.io were all tested in this environment and failed (truncated files, 403s on GET, timeouts). uploadUguu() is the only host that reliably serves both small JPGs and large MP4s AND is reachable by the Seedance worker. Files live ~3 hours on uguu — fine for a run that takes ~15 min.

4. Seedance filename MUST be relative. The plugin prepends C:\Users\Studio\AppData\Roaming\AGNT\plugin-data\seedance\<userId>\ to whatever you pass. If you pass an absolute path, the resulting file lives at a double-prefixed path and ffmpeg will ENOENT. Always pass a plain filename like scifi_scene1.mp4, then immediately copy the output into your project folder with copySeedanceOutput().

5. CHAIN mode: extract and upload the last frame BETWEEN every scene. If you ever call scene N+1 with the original reference image as firstFrameUrl, you've broken the chain. Use extractAndUploadLastFrame(sceneMp4, projectDir, label) after every scene (except optionally the last), and pass its frameUrl as the next scene's firstFrameUrl.

6. Write a manifest.json at the end. Record mode, every prompt, every local path, every uguu URL, durations, costs, wall time. Keeps the run reproducible.

The files that do the work

scripts/
├── save_generated_image.js          ← fetchImageRefBytes()  ← PRIMARY
│                                      saveGeneratedImage()  ← fallback
│                                      createProjectDir(),
│                                      writeProjectManifest()
├── upload_uguu.js                   ← uploadUguu()  — the only upload host
├── copy_seedance_output.js          ← copySeedanceOutput()  — handle the
│                                      relative-filename quirk
├── extract_and_upload_last_frame.js ← extractAndUploadLastFrame(),
│                                      extractLastFrame()
└── stitch_xfade.js                  ← stitchXfade()  — the exact ffmpeg
                                       command that worked
references/
└── seedance-prompt-library.md       ← 20+ proven shot prompts

Mode choice

User's goal	Mode
Continuous walk-and-talk, long pan, evolving action	CHAIN
Trailer-style single-breath mini-film	CHAIN
Narrative >10s with continuity	CHAIN
3-shot montage / highlight reel / social post	PARALLEL
Different camera angles of the same subject	PARALLEL
Fastest turnaround	PARALLEL

The proven CHAIN pipeline (what actually worked)

All tool calls below are AGNT tool calls made in the conversation — not inside execute_javascript_code (which can't invoke AGNT tools). The code blocks are what goes INSIDE execute_javascript_code.

Step 1 — generate the source image (chat-level tool, Gemini Nano Banana Pro)

generate_image(
  prompt:       "A lone astronaut in a scuffed white spacesuit at the
                 threshold of a massive derelict alien spaceship
                 half-buried in orange desert dunes. Anamorphic lens
                 flare, golden-hour rim light, cyan glyphs on the hull,
                 35mm film grain, Denis Villeneuve.",
  provider:     "gemini",
  model:        "nano-banana-pro-preview",
  aspectRatio:  "16:9"
)

Capture the returned ref id, e.g. {{IMAGE_REF:img-toolu_01ABC...-0}} → img-toolu_01ABC...-0.

Step 2 — pull the bytes onto disk + upload to uguu

const path = require('path');
const {
  fetchImageRefBytes, createProjectDir,
} = require('C:/Users/Studio/.agnt/skills/image-to-cinematic-video/scripts/save_generated_image.js');
const { uploadUguu } = require('C:/Users/Studio/.agnt/skills/image-to-cinematic-video/scripts/upload_uguu.js');

const projectDir = createProjectDir('scifi_chain');
const srcImg = path.join(projectDir, 'source.jpg');

// One call: Bearer-auth GET /api/images/:id and write bytes to disk.
await fetchImageRefBytes('img-toolu_01ABC...-0', srcImg);

const imageUrl = await uploadUguu(srcImg);
require('fs').writeFileSync(path.join(projectDir, 'image_url.txt'), imageUrl);
console.log('PROJECT_DIR=' + projectDir);
console.log('IMAGE_URL=' + imageUrl);

Step 3 — Scene 1 (Seedance, AGNT tool call)

Pass a relative filename. Pass the uguu URL as firstFrameUrl.

seedance_api(
  prompt:        "Slow cinematic dolly-in on the astronaut as she raises
                  a gloved hand to the hull — ancient cyan glyphs flare
                  under her touch. Anamorphic lens flare, 35mm grain.",
  firstFrameUrl: "<imageUrl from step 2>",
  filename:      "scifi_scene1.mp4",              ← RELATIVE
  duration:      5,
  aspectRatio:   "16:9",
  resolution:    "720p"
)

Step 4 — copy scene 1 into project + extract/upload its last frame

const { copySeedanceOutput } = require('C:/Users/Studio/.agnt/skills/image-to-cinematic-video/scripts/copy_seedance_output.js');
const { extractAndUploadLastFrame } = require('C:/Users/Studio/.agnt/skills/image-to-cinematic-video/scripts/extract_and_upload_last_frame.js');

const scene1 = copySeedanceOutput('scifi_scene1.mp4',
                                  path.join(projectDir, 'scenes/scene1.mp4'));
const { frameUrl: frame1Url } =
  await extractAndUploadLastFrame(scene1, projectDir, 'scene1');
console.log('FRAME1_URL=' + frame1Url);

Step 5 — Scene 2 (Seedance, using frame1Url)

seedance_api(
  prompt:        "Continuous: the glyphs erupt into a cascade of runes,
                  a seam cracks open, warm white light pours out. Camera
                  pushes toward the widening doorway. 35mm grain.",
  firstFrameUrl: "<frame1Url>",
  filename:      "scifi_scene2.mp4",              ← RELATIVE
  duration:      5,
  aspectRatio:   "16:9",
  resolution:    "720p"
)

Step 6 — copy + last-frame for scene 2

const scene2 = copySeedanceOutput('scifi_scene2.mp4',
                                  path.join(projectDir, 'scenes/scene2.mp4'));
const { frameUrl: frame2Url } =
  await extractAndUploadLastFrame(scene2, projectDir, 'scene2');

Step 7 — Scene 3 (Seedance, using frame2Url)

seedance_api(
  prompt:        "Continuous: camera pushes through the doorway into a
                  cathedral-scale interior of crystalline pillars pulsing
                  with light. Astronaut silhouette walks forward.
                  Volumetric god-rays, 35mm grain.",
  firstFrameUrl: "<frame2Url>",
  filename:      "scifi_scene3.mp4",              ← RELATIVE
  duration:      5,
  aspectRatio:   "16:9",
  resolution:    "720p"
)

Step 8 — copy scene 3 + extract scene 3's last frame (thumbnail only — no upload needed)

const { extractLastFrame } = require('C:/Users/Studio/.agnt/skills/image-to-cinematic-video/scripts/extract_and_upload_last_frame.js');
const scene3 = copySeedanceOutput('scifi_scene3.mp4',
                                  path.join(projectDir, 'scenes/scene3.mp4'));
extractLastFrame(scene3, path.join(projectDir, 'frames/scene3_last.jpg'));

Step 9 — stitch + upload final + manifest

const { stitchXfade } = require('C:/Users/Studio/.agnt/skills/image-to-cinematic-video/scripts/stitch_xfade.js');
const { writeProjectManifest } = require('C:/Users/Studio/.agnt/skills/image-to-cinematic-video/scripts/save_generated_image.js');

const finalPath = path.join(projectDir, 'final.mp4');
const { durationSec, sizeBytes } = await stitchXfade({
  clips: [scene1, scene2, scene3],
  outputPath: finalPath,
  fadeDuration: 0.15,                // chain mode — short fade
  transition: 'fade',
  crf: 18,
});

const finalUrl = await uploadUguu(finalPath);

writeProjectManifest(projectDir, {
  generatedAt: new Date().toISOString(),
  mode: 'chain',
  sourceImage: { provider: 'gemini', model: 'nano-banana-pro-preview' },
  scenes: [
    { n: 1, path: scene1, lastFrameUrl: frame1Url },
    { n: 2, path: scene2, lastFrameUrl: frame2Url },
    { n: 3, path: scene3, lastFrameUrl: null },
  ],
  stitch: { fadeDuration: 0.15, transition: 'fade', crf: 18 },
  final: { path: finalPath, url: finalUrl, durationSec, sizeBytes },
  totals: { cost: 2.268, resolution: '720p', aspectRatio: '16:9' },
});

console.log('FINAL_URL=' + finalUrl);

Step 10 — reply to the user

<video src="<finalUrl>" controls autoplay loop muted playsinline
       style="width:100%;max-width:720px;border-radius:12px;
              box-shadow:0 8px 32px rgba(0,0,0,0.4);background:#000;"></video>

Plus a crisp summary (mode, duration, resolution, cost, wall time) and the absolute path to the project folder.

PARALLEL mode (lighter variant)

Same as above except:

• All N seedance calls go into ONE tool-call block with _executeAsync: true so they run concurrently.
• Every scene uses the SAME firstFrameUrl (the source image URL).
• No last-frame extraction between scenes.
• Use fadeDuration: 0.5 in stitchXfade (longer decorative fades make sense when seams visibly reset).

Seedance parameters (both modes)

Param	Value
aspectRatio	16:9 (landscape) or 9:16 (Reels/Shorts)
duration	5 seconds (sweet spot for cost and fidelity)
resolution	720p ($0.151/s = $0.756 per 5s clip)
firstFrameUrl	uguu URL — shared (parallel) or chained (chain)
filename	RELATIVE only, e.g. scifi_scene1.mp4

Prompt formula: [camera move] [subject] [action] [environment] [lighting] [style]

20+ proven prompts: references/seedance-prompt-library.md.

Image generation: which path to use

Situation	Path
Default — best quality, agentic flow	generate_image chat tool → fetchImageRefBytes()
You want a specific provider available in the chat tool	generate_image (set provider/model) → fetchImageRefBytes()
You're running headless inside execute_javascript_code only and don't want to round-trip through the chat tool	saveGeneratedImage() (Pollinations / Gemini REST / OpenAI REST)
You already have an image URL (user upload, etc.)	Skip generation, just uploadUguu() it

fetchImageRefBytes(ref, outputPath) does an authenticated GET /api/images/:ref against the local AGNT server, validates the response is binary image data (rejects HTML SPA fallbacks), and writes the bytes to disk. Returns { path, sizeBytes, contentType }.

Verified result from the canyon-runes production run

	Value
Source image	Gemini Nano Banana Pro via generate_image (776 KB JPG, 16:9)
Scenes	5 (chained)
Scene duration	5s each
Total after stitch	24.6s
Resolution	720p, 16:9
Total Seedance cost	$3.78
Final mp4	13.6 MB (libx264 CRF 18, 0.15s xfade)
Hosted at	https://d.uguu.se/GAjyPqKB.mp4

Project folder layout after a successful run:

<PROJECTS_ROOT>/<stamp>_<slug>/
├── source.jpg                 ← bytes from /api/images/:id
├── image_url.txt              ← uguu URL of source.jpg
├── scenes/
│   ├── scene1.mp4
│   ├── scene2.mp4
│   └── scene3.mp4
├── frames/
│   ├── scene1_last.jpg        ← → became scene 2's firstFrameUrl
│   ├── scene2_last.jpg        ← → became scene 3's firstFrameUrl
│   └── scene3_last.jpg        ← thumbnail / poster
├── final.mp4                  ← stitched output
├── final_url.txt              ← uguu URL of final.mp4
└── manifest.json              ← complete run record

Common failure modes (observed)

• ENOENT on seedance output → you passed an absolute filename. Use a relative one, then copySeedanceOutput().
• "Invalid base64 image url" from Seedance → the upload host you used isn't serving the file. Use uploadUguu().
• Stitched video is three disconnected cuts instead of one take → you forgot to swap firstFrameUrl between scenes. Use extractAndUploadLastFrame() after every scene except the last.
• Frame extraction gives a tiny/empty jpg → the video has no keyframe at exactly -0. The -sseof -0.1 offset in extract_and_upload_last_frame.js already handles this.
• fetchImageRefBytes() returns HTML instead of an image → the Authorization: Bearer ${AGNT_AUTH_TOKEN} header was dropped, so the server fell back to serving the SPA. The helper detects this (content-type sniff + magic-byte check) and throws — re-check the header and the ref id.