multimodal-serving

# Multimodal Serving Use this sub-skill when a task needs to load an align-anything model, choose a text/multimodal/omni Gradio CLI, prepare media/template inputs, or diagnose serving-time model and media failures. ## Start here 1. Read [`references/model-loading-and-cli.md`](references/model-loading-and-cli.md) to choose between `load_pretrained_models`, `AnyModel`, `AnyModelForScore`, `text_modal_cli`, `multi_modal_cli`, and `omni_modal_cli`. 2. Read [`references/media-and-templates.md`](references/media-and-templates.md) before constructing image, audio, video, or mixed omni messages. 3. Use [`scripts/check_model_loading.py`](scripts/check_model_loading.py) for an import-only, dry-run, or real model-loading smoke check. 4. Use [`scripts/run_cli_template.sh`](scripts/run_cli_template.sh) as the bundled serving launcher template instead of relying on source-tree shell snippets. 5. If loading, decoding, templating, or Gradio startup fails, use [`references/troubleshooting.md`](references/troubleshooting.md). ## Routing guidance Use this sub-skill for: - loading base or reward models through align-anything's auto-model registry; - setting device, dtype, cache, `trust_remote_code`, and optional modality/omni initialization flags; - launching the text, multimodal image/audio/video, or MiniCPM-O-style omni CLI; - translating uploaded media into the processor/model inputs expected by align-anything serving code; - explaining optional dependency and backend failures at serving time. Prefer another sub-skill when the task is training/alignment, reward-server deployment, evaluation-benchmark orchestration, or repo-level project setup rather than interactive inference. ## Boundaries and safety notes - Treat remote model code as executable code. Enable `trust_remote_code` only for model repositories you trust. - The bundled CLI launcher starts the package's Gradio CLIs, which currently request a shareable Gradio link. Use only in a trusted network/session unless you adapt the local launcher to disable sharing. - Large multimodal and omni models generally need CUDA/NPU-class memory. CPU import checks are useful, but they do not prove full generation throughput or memory fit. - Keep all local environment names, private install prefixes, and checkout-specific paths out of downstream reports and user-facing instructions.