create-lipsync-veed-fal

name	create-lipsync-veed-fal
description	Generate a talking-head video clip with VEED Fabric 1.0 via the fal.ai API. Same model as `create-lipsync-hedra` (Veed Fabric on Hedra) but uses fal's pay-per-call billing instead of Hedra credits. Inputs unchanged — character still + audio in, lipsynced MP4 out. Best for ad-grade lipsync where you want phoneme accuracy and don't need body motion.

create-lipsync-veed-fal

Purpose

The same Veed Fabric 1.0 model as create-lipsync-hedra, called through fal.ai's gateway instead of Hedra's. Use this when:

• You already have a FAL_API_KEY set up and don't want a Hedra subscription.
• You're doing a one-off batch and pay-per-call billing is cleaner than maintaining a Hedra credit balance.
• Your Hedra account ran out of credits mid-batch and you want to finish the remaining clips on fal.

Same output quality, same lipsync model, same input contract. Only the gateway + billing differ.

Pricing (as of 2026-05)

• 480p: $0.08 per second of generated video
• 720p: $0.15 per second of generated video

A 3-second 720p clip = $0.45. A 23-clip ad with ~60s total audio = ~$9. Compare to Hedra Creator plan ($30/mo) — fal wins for one-off batches, Hedra wins for ongoing use.

Pricing source: fal.ai/models/veed/fabric-1.0.

Inputs

Required:

• image_path — local path to character still (PNG or JPG, < 10 MB). Tight head-and-shoulders, neutral closed-mouth pose. Lipsync only works if the mouth starts in a neutral state — see the lessons in create-lipsync-hedra/SKILL.md for why.
• audio_path — local path to dialogue audio (MP3, WAV, M4A, or AAC, < 10 MB).
• output_path — local path where the resulting MP4 should be saved.

Optional:

• resolution — "480p" (cheaper, slightly lower quality) or "720p" (default, ad-grade).

Credentials:

• FAL_API_KEY (or FAL_KEY) in .env. Get one at fal.ai/dashboard/keys.

Preflight

test -n "$FAL_API_KEY" || test -n "$FAL_KEY" || { echo "Missing FAL_API_KEY / FAL_KEY in .env"; exit 1; }
python3 -c "import fal_client" || pip3 install fal_client
command -v ffprobe >/dev/null || { echo "ffprobe required"; exit 1; }

Workflow

The script scripts/generate.py handles upload → submit → poll → download.

python3 skills/atoms/lipsync/create-lipsync-veed-fal/scripts/generate.py \
  --image /path/to/character.png \
  --audio /path/to/line.mp3 \
  --output /path/to/clip.mp4 \
  --resolution 720p

What the script does:

1. Upload image to fal storage — fal_client.upload_file(image_path) returns a public https://v3.fal.media/... URL.
2. Upload audio to fal storage — same call, same return shape.
3. Submit generation — fal_client.subscribe("veed/fabric-1.0", {image_url, audio_url, resolution}). This is the synchronous pattern; it blocks until the result is ready (~1.5 min at 480p for 10s of audio, ~5 min at 720p).
4. Download the resulting video to output_path.
5. Verify with ffprobe that duration matches input audio within ±10%.
6. Write <output_path>.meta.json with {request_id, image_url, audio_url, duration_sec_video, duration_sec_audio, size_bytes, model: "veed/fabric-1.0", resolution} for cost tracking + debug.

The script uses fal_client.subscribe (sync) rather than the queue API because per-clip durations are 1-5 min — fine for one-shot calls. For very large batches (>50 clips) prefer fal_client.submit_async to parallelize without blocking threads.

Output

• <output_path> — MP4 talking-head clip with audio baked in.
• <output_path>.meta.json — request metadata + cost-tracking fields.

Quality Checks

• Output file > 100 KB and exists.
• ffprobe reports duration within ±10% of input audio.
• Output codec is h264 + AAC.
• meta.json includes request_id for retry / dispute.

Failure Modes

Symptom	Likely cause	Fix
401 Unauthorized	Bad / missing FAL_API_KEY	Verify env var; key starts with fal- or just a long hex string.
429 Too Many Requests	Account rate limit	fal has per-account RPS limits. Drop concurrency to 2-3.
Generation takes >10 min	fal queue is slow (peak hours)	Patient; or switch to fal's async submit API and poll yourself.
Lipsync drift mid-clip	Audio file has leading silence	Trim with ffmpeg -ss 0.1 before passing in.
Body waves / hands move unnaturally	Input still has loose clothing / wide framing	Crop tighter, head-and-shoulders only. Veed Fabric freezes the bust but not loose hands.
Mouth doesn't move	Audio asset upload failed silently	Check meta.json for valid audio_url and that it returns the audio bytes when fetched.

Limitations

1. Bust framing only. Veed Fabric animates head/face. The body is mostly frozen — body motion below shoulders is unstable.
2. 10 MB max per file. Both image and audio must be under 10 MB. For long audio (>3-4 minutes), split into multiple clips.
3. Resolution caps at 720p. For higher resolution, upscale in post (e.g. RealESRGAN) or use a different model.
4. No text_prompt parameter. Unlike Hedra, fal's veed/fabric-1.0 doesn't accept scene description — the model is pure image+audio.
5. Two resolution options only. 480p ($0.08/s) and 720p ($0.15/s). No 540p or 1080p.

When to use this vs create-lipsync-hedra

Use case	Recommended
One-off ad, no existing Hedra account	veed-fal
Recurring ad production, weekly batches	hedra (Creator plan amortizes well)
Hedra account out of credits mid-batch	veed-fal for the remaining clips
Need exact same look as previous batch (continuity)	Stick with whichever you started — outputs are model-identical but minor sampling variance exists between gateways
Need text_prompt for scene direction (rare for ad lipsync)	hedra (supports text_prompt)

References

• fal.ai/models/veed/fabric-1.0 — fal's model page
• Launching VEED Fabric 1.0 API — VEED's announcement
• VEED Fabric 1.0 on fal blog — fal's launch post
• Sibling skill: skills/atoms/lipsync/create-lipsync-hedra/SKILL.md — same model, different gateway
• QC: skills/atoms/review/review-lip-sync/SKILL.md — run on the output to verify lipsync