cosmos3-inference

# Cosmos3 Inference ## When to use this skill - Use when a user wants to generate images or videos with Cosmos3 - Use when a user asks about inference parameters, input formats, or parallelism - Use when a user wants to set up online serving (Ray Serve, Gradio) - Use when a user asks about prompt engineering or upsampling - For environment or import errors, hand off to **cosmos3-env-troubleshoot** ## Path convention All paths below are relative to the cosmos3 package root (`../../../` from this skill file). All `uv run` / `python` commands should also be run from there. ## Where to find answers | User question | Go to | | ----------------------------------------------------------------------- | --------------------------------------------------------------------------------------- | | How do I run inference? (single-GPU, multi-GPU) | `README.md` § Inference | | Which model should I use? (Nano vs Super, memory, shift) | `README.md` § Models | | Which modality? (t2i, t2v, i2v, examples) | `README.md` § Modalities | | What parallelism preset? (latency vs throughput) | `README.md` § Inference | | What input fields are available? (prompt, vision_path, num_frames, ...) | `docs/inference.md` § Sample Arguments | | What are the default parameter values? | `cosmos_framework/inference/defaults/<model_mode>/sample_args.json` (per-modality JSON) | | How do I use custom defaults? | `docs/inference.md` § Custom Defaults | | How do I override a parameter? (precedence) | `docs/faq.md` § How do I override a default parameter? | | What is the shift parameter? | `docs/faq.md` § What is the `shift` parameter? | | How many frames can I generate? (resolution caps) | `docs/faq.md` § How many frames can I generate? | | How do I start Ray Serve / Gradio / submit requests? | `docs/faq.md` § How do I run online inference with Ray? | | How do I upsample short prompts? | `docs/faq.md` § Prompt upsampling | | How do I use the low-level API? (examples/) | `examples/inference.py` (model API) / `examples/inference_pipeline.py` (pipeline API) | | All CLI flags | `uv run --all-extras --group=cu130 python -m cosmos_framework.scripts.inference --help` | ## Things not obvious from the docs - **Path resolution**: relative paths in input JSON files are resolved relative to the **JSON file's directory**, not the working directory. - **Seed**: always pass `--seed` for reproducible results. Without it, a random seed is used each time. - **Resume**: interrupted runs can be resumed by re-running the same command — existing outputs are skipped automatically. - **`--keep-going`**: continues processing remaining samples after a per-sample failure (e.g. guardrail rejection). Used in online serving by default. - **Unique names**: every sample in a run must have a unique `name` field, or the script will error. ## Related skills | Skill | When to use | | -------------------------------------- | ---------------------------------------------- | | `../cosmos3-setup/SKILL.md` | Installation and environment setup | | `../cosmos3-codebase-nav/SKILL.md` | Finding files, parameters, and configs in code | | `../cosmos3-env-troubleshoot/SKILL.md` | Debugging environment and runtime errors |