sume-quality-experiment

name	sume-quality-experiment
description	Design and run controlled Sume quality experiments with one-variable variants, reusable manifests, side-by-side artifact comparison, and rubric-based sanitized reports.

Sume Quality Experiment

Use this skill for controlled Avatar or Avatar Video comparisons.

Experiment Rules

1. Write a short local manifest before paid calls.
2. Change one variable per variant: quality tier, scene prompt, product image, avatar handle, script, duration, or aspect ratio.
3. Keep shared anchors identical across variants.
4. Run no-cost checks first: schema discovery, manifest validation, and local media inspection when outputs already exist.
5. Ask for explicit paid approval and a spend cap before paid generation.
6. Use idempotency keys and state files so reruns do not duplicate jobs.

Commands

sume tools schema avatar-videos.create --json
sume avatar-videos batch plan ./experiment.videos.json --output-file ./experiment.plan.json --json
sume avatar-videos batch create ./experiment.videos.json --state-file ./experiment.state.json --confirm-paid --json
sume avatar-videos batch watch ./experiment.videos.json --state-file ./experiment.state.json --json
sume avatar-videos batch result ./experiment.videos.json --state-file ./experiment.state.json --json

Use sume-media-qa after results exist if the user needs local files, still sheets, codec checks, or side-by-side comparison.

Rubric

Score or summarize:

• identity preservation;
• product fidelity;
• script/audio fit;
• scene continuity;
• artifact severity;
• metadata/readback usefulness;
• cost and elapsed time.

Not For

Do not use this skill for unrestricted exploration, raw provider endpoint tests, or one-off fastest iteration. Use sume-min-latency for speed and sume-spend-capped-dogfood for paid QA with a strict budget.