image-to-video-scroll-world

name

image-to-video-scroll-world

description

Build an immersive scroll-scrubbed "fly through the world" landing page for any industry or brand using generated keyframe art plus Magnific MCP and Seedance video. Create cohesive diorama or cinematic scenes, seamless frame-locked camera clips, encoded posters, SSIM-verified joins, mobile fallbacks, and a portable framework-agnostic scrub engine. Use for 3D worlds, Emons-style browse-through journeys, scroll cinematics, diorama landings, image-to-video hero sequences, or turning a business story into one continuous scrollable camera flight.

Image-to-video scroll world

Produce a landing page where scroll drives currentTime through pre-rendered video. Make the camera genuinely move; do not fake the effect by moving layered stills.

Generate this chain:

N anchor-locked scene stills
→ N camera legs or dives
→ optional N-1 connectors
→ scrub-friendly encodes + extracted posters
→ SSIM seam gate
→ portable scroll engine

Use OpenAI imagegen for stills and Magnific MCP for Seedance video. Preserve the upstream cth9191/scroll-world production method; replace only the Higgsfield generation layer. The bundled scrub engine, page template, and knockout utility are adapted from that MIT-licensed project at upstream commit ab91586599f784bc0b9d803a4c96ee25ffdf9eb6; retain LICENSE when redistributing.

The critical rule: make every seam frame-identical. For connectors, use the neighboring rendered videos' actual boundary frames, never the original stills.

0. Bootstrap

1. Confirm Magnific MCP exposes video planning, model catalog, balance, cost simulation, upload, generation, wait/get, and preview tools. Read references/magnific-mcp.md.
2. Confirm ffmpeg and ffprobe exist.
3. Confirm an image-generation skill is available for keyframes.
4. Inspect the target repository, brand rules, framework, routing, middleware, existing media, and page structure before editing.
5. Create a resumable work directory and asset ledger. Reuse completed assets; never repay for a successful generation.

1. Interview and scope

Ask only for information that cannot be inferred safely:

1. Ask openly what the world should be about and capture the business/product plus one-line pitch.
2. Import the brand from the supplied site or collect/propose a display name, tone, and 4–6 named hex colors.
3. Agree on art direction. Default to soft matte low-poly clay diorama, isometric miniature, warm light. Reuse the approved style preamble verbatim.
4. Choose a budget tier before proposing the journey:
   • Lean: 4 scenes, architecture A, about 4 stills + 4 video legs.
   • Standard: 5 scenes; A uses about 5 videos, B uses about 9 videos.
   • Showcase: 6–7 scenes, architecture B, previz when a cheaper compatible model exists, plus mobile options.
5. Propose ordered scenes sized to the budget. Give each an id, subject, focal point, eyebrow, title, one-sentence body, 0–3 proof tags, and optional CTA.
6. Ask for a mobile tier:
   • crop-safe: reuse desktop clips, no extra generation
   • mobile encodes: local 720p tight-GOP encodes, no extra generation
   • hero reframe: regenerate only 1–2 critical portrait scenes
   • portrait chain: build a separate 9:16 chain with its own seams and cost

Query current Seedance models and simulate the actual batch cost. Include a 20–30% reroll buffer. Obtain an explicit total credit/cost cap before paid generation. The historical Data Espresso run used 3 five-second 720p Seedance 2.0 Mini clips for 2,100 credits under a 3,500-credit cap; treat this only as history, never current pricing.

Read references/prompts.md before writing prompts.

2. Generate stills: anchor gate first

1. Generate one representative anchor still with imagegen.
2. Show it to the user and revise the shared style preamble until approved. Do not proceed past a rejected anchor.
3. Generate remaining stills using the approved anchor as a visual reference when the image tool supports it.
4. Review the batch for identical camera language, geometry, palette, lighting, materials, and subject treatment. Reroll only off-style scenes.
5. Keep full-resolution sources. Create web delivery variants later.

Compose focal subjects center-safe for object-fit: cover. Avoid generated text, logos, watermarks, and edge-critical details. Preserve stills as reduced-motion and no-video fallbacks.

3. Choose camera architecture

A. Continuous forward take

Use for grounded, realistic, architectural, product, industrial, or first-person walkthroughs.

Generate legs sequentially. Use scene 0's still as the first start frame. For each later leg, extract the previous leg's actual final frame and upload that as the next start frame. Do not force a wide end image; steer the next destination through the prompt. End every leg with a slow steady forward drift and begin the next by continuing it.

Allow expressive movement inside a leg—half-orbit, crane-up, lateral track, detail push, or rise-and-swoop—but never reverse velocity across a seam.

Cost: N video legs. Connectors: none. Use a small crossfade only to mask encoding variance.

B. Dive plus aerial connector

Use only for diorama, miniature, map-like, or god's-eye worlds where pulling out between scenes feels intentional.

Generate one dive per scene from its still. Extract the actual first and last frames from every rendered dive. Generate each connector with:

start = previous dive's actual last frame
end   = next dive's actual first frame

Cost: N dives + N-1 connectors. Permit a null connector on a low-priority seam when the user accepts a direct crossfade.

Do not use architecture B for a realistic walkthrough; the forward-dive to backward-pull-out reversal reads as a rewind.

4. Generate through Magnific MCP

Use one Seedance model for the entire final chain so motion character, grain, and color stay consistent. Require start-image support for architecture A and start+end keyframe support for architecture B connectors.

For every generation:

1. Call video_plan first unless the user explicitly requested one-shot generation.
2. Validate the plan's model against video_models_list; inspect supported durations, resolutions, aspect ratios, audio, and keyframes.
3. Upload local keyframes to Magnific and retain their creation identifiers or final asset URLs. Never pass a Magnific webUrl as a generation input.
4. Call simulate_cost with the exact intended video_generate arguments and verify the remaining approved cap and account balance.
5. Generate exactly one resumable leg/dive/connector record, disable sound effects, then show the intermediate creation to the user.
6. Wait only when the final asset URL is needed for chaining. Download the finished MP4 and update the ledger immediately.
7. Stop before exceeding the cap. Ask before switching provider/model if it materially changes style or cost.

For runs above four scenes, use a cheaper Seedance previz pass only when the current catalog provides a compatible frame-locking draft model. Assemble and review the full previz journey before spending on a final tier. If Seedance 2.0 Mini is the only suitable model, it can be the 720p final rather than inventing a nonexistent upgrade path.

Read references/pipeline.md for exact MCP payload shapes, upload/download flow, resumability, media encoding, posters, and SSIM checks.

5. Enforce the seamless chain

Extract frames from rendered source videos:

ffmpeg -v error -ss 0 -i dive-next.mp4 -frames:v 1 -q:v 2 next-first.png
ffmpeg -v error -sseof -0.15 -i dive-prev.mp4 -frames:v 1 -q:v 2 prev-last.png

Upload prev-last.png and next-first.png, then use them as the connector's Magnific start/end keyframes. Do not substitute the scene stills: the video renderer changes crop, geometry, and grade slightly, causing a visible pop.

After encoding, compare every adjacent boundary with SSIM:

• >= 0.90: pass
• 0.75–0.90: warn; inspect the crossfade
• < 0.75: fail; fix endpoints or regenerate

Replacing one clip can break both neighboring seams. Rerun the gate after every reroll.

6. Encode and derive assets

Encode the native output; never upscale a 720p source merely to label it 1080p.

• Desktop/master: H.264, yuv420p, CRF 20, GOP 8, fixed keyframes, no audio, faststart.
• Mobile: scale to 720-class, CRF 23, GOP 4, no audio, faststart.
• Poster: extract the encoded clip's first frame. Do not use the source still as its poster.
• Portrait: create a completely separate 9:16 chain when selected; do not mix aspect ratios mid-chain.

Run the bundled media helpers when useful:

python scripts/inspect_media.py <media files or directory>
python scripts/extract_seam_frames.py <clip.mp4> --output-dir <qa-dir>

7. Assemble the page

Copy or adapt references/scrub-engine.js. It is framework-agnostic and config-driven. Use references/index-template.html for a standalone example.

Configure ordered sections, clips, posters, optional mobile variants, connectors, copy, accents, pacing, linger, and CTA. Keep the generated world visually dominant and the UI chrome quiet.

The engine provides scroll-to-time scrubbing, blob loading for hosts without byte ranges, lazy loading, frame crossfades, route rail, reduced-motion handling, data-saver fallback, phone seek coalescing, iOS priming, Low Power Mode fallback, device-class media tiers, and safe-area handling.

Server-render a data-sw-seo copy mirror containing the real h1, scene headings, body copy, and CTA links. Hide it only after the engine mounts so crawlers and no-JS visitors retain the content.

Add ping-pong autoplay only after native scroll scrubbing works:

• play normalized progress 0 → 1, hold, then 1 → 0, hold
• do not physically move the viewport
• stop autoplay on real wheel/touch/page-scroll input
• capture autoplay and scroll baselines so manual handoff does not snap
• disable autoplay for reduced motion, data saver, deep links, and restored non-top scroll positions

Read references/react-next-patterns.md when adapting the engine into React or Next.js.

8. QA before shipping

1. Pass the SSIM gate for every asset seam.
2. Verify poster-to-video first paint has no crop or render jump.
3. Verify slow and fast forward/reverse scrubbing across every seam.
4. Confirm autoplay progresses forward and backward without resetting, and manual scroll handoff does not snap.
5. Confirm videos become seekable; use blob URLs when the host lacks byte-range behavior.
6. Emulate data saver and a rejected HTMLMediaElement.play(); both must fall back to stills without fetching/frozen video.
7. Verify phone, tablet, desktop, portrait, landscape, URL-bar collapse, safe areas, and no horizontal overflow.
8. Verify reduced motion loads stable stills and no decorative motion.
9. Verify direct hash navigation reaches its section without restarting hero autoplay.
10. Verify the console and network panel are clean and SEO copy exists in served HTML.

Read references/gotchas.md immediately when a generation, seam, seek, mobile, or fallback check fails.

Report the selected Magnific/Seedance model, architecture, generated files, actual spend, approved cap, mobile tier, seam scores, browser results, and fallbacks.

References

• references/magnific-mcp.md — provider-specific tool sequence and payload rules.
• references/prompts.md — intake fields, style preambles, camera grammar, and generation prompts.
• references/pipeline.md — resumable Magnific generation and local media pipeline.
• references/scrub-engine.js — upstream portable scrub engine.
• references/index-template.html — minimal standalone mount with SEO copy.
• references/knockout.py — optional border-connected background knockout.
• references/react-next-patterns.md — React/Next.js integration and autoplay handoff.
• references/gotchas.md — symptom-to-fix production guide.