add-recipe

name	add-recipe
description	Use when the user asks to add, update, validate, or publish a vLLM model recipe in this repo. Covers canonical Hugging Face identity, source/image requirements, checkpoint staging, hardware evidence, YAML-to-UI relationships, command validation, JSON generation, and staging the reviewed files.

Author and validate a vLLM recipe

Recipes are YAML files at models/<hf_org>/<hf_repo>.yaml. The path mirrors HuggingFace (huggingface.co/<hf_org>/<hf_repo>), and the site/API are generated at build time from these files + taxonomy.yaml + strategies/*.yaml.

Component contract: evidence → YAML → rendered surface

Do not author the YAML as isolated documentation. Each input controls a different consumer, and a plausible guide does not prove that the command builder shows the same thing.

HF repo + config.json ───────────────► model identity, architecture, context, variants
runtime source ref + distribution ──► Install block and reproducible prerequisites
hardware profile + strategy YAML ───► selectable topology and generated parallel flags
real serving evidence ──────────────► meta.hardware verified + tested guide section
recipe YAML ─► loader ─► command-synthesis ─► CommandBuilder / Verify / Bench
            └────────────────────────► static JSON API
guide markdown ──────────────────────► Guide accordion and explicit reference links

Component	Put it in	What consumes or displays it	Evidence required
Canonical recipe identity	file path + model.model_id	route, page title, header HF link, JSON path	exact HF repo; preserve org/repo case
Quantized or alternate checkpoint	variants.<key>.model_id, including variants.default.model_id when the preferred default differs from the family route	Variant pill, generated Serve/Verify/Bench model id, variant redirect	exact accessible checkpoint and its config
Alternate-checkpoint reference	guide Overview, serving section, and References	clickable Guide links	link the exact tested repo; the header HF link still points to the canonical recipe path
Runtime distribution	model.install and model.docker_image	Install tabs and Docker-wrapped command	only show an image/wheel that actually exists and contains support
Extra prerequisite	dependencies[]	Extra install block after Install and before Serve	additive only; it does not replace the Install tabs
Base runtime behavior	model.base_args/base_env	every generated command	required for every variant/hardware/strategy
Variant behavior	variants.*.extra_args/extra_env	selected variant command	required only for that checkpoint/precision
Hardware status	meta.hardware	recipe-wide verified caption or disabled hardware pill	verified only after an end-to-end run representative of the recipe; unsupported only after a confirmed current blocker; absent means no badge
Hardware-specific behavior	hardware_overrides	command/env after selecting that GPU generation	tested or documented runtime requirement; status and override are independent
Strategy availability	compatible_strategies	Strategy pills	architecture/topology support, not model popularity
Strategy customization	strategy_overrides	TP sizing or role-specific generated command	explicit topology requirement
Feature	features + opt_in_features	Feature pills and final command args	do not imply a feature was tested merely because base chat passed
Capacity hint	variants.*.vram_minimum_gb + taxonomy	VRAM warning and auto-fit TP hint	total-parameter sizing formula; not a hardware support gate
Narrative and claims	guide	Guide accordion only	separate tested facts from advertised/model-card capabilities
Agent/API output	generated public/<org>/<repo>.json	static consumers	generated by validation; never stage public/

Rendering consequences that are easy to miss

• The header's View on HuggingFace link is derived from the recipe file path, not the selected variant. If FP8/NVFP4/etc. is a different repo, link it explicitly in the Overview, its serving section, and References.
• meta.hardware is not variant-specific. A family recipe may keep the base HF path while variants.default.model_id selects a separately published preferred checkpoint; identify the tested variant explicitly in the guide so the recipe-wide badge is not mistaken for evidence about untested variants.
• variants.<key>.model_id changes the generated serve command and the Verify/Bench model id. A guide-only FP8 command does not update the interactive builder.
• dependencies[] is additive. For unreleased source support, place the complete build command in model.install.pip.command; if no image exists, set model.install.docker: false. Do not leave a generic Docker or nightly-wheel tab visible when it cannot run the model.
• nightly_required: true changes the default pip text and shows a nightly badge. It does not prove the nightly contains the needed model registry entry.
• meta.hardware.<id>: verified is a claim about the selected recipe path, variant, and working serving flow. Record the tested variant/context in the guide.
• The initial builder state should reproduce the successful baseline: default variant, default-on features, strategy_overrides.single_node_tp.tp, and base_args all affect it. Move untested features to opt_in_features and encode the tested TP/context rather than relying on auto-sizing or model-card limits.
• guide commands are not synthesized or checked against the builder. Preview both and reconcile model id, TP, flags, environment, and install method.

End-to-end steps

1. Resolve current sources before writing claims. Fetch the runtime repo and record the exact commit that contains model registration and required kernel/quantization fixes. Branch names move and similarly named branches may not contain the same code. Recheck the remote ref immediately before the final build. If support is unreleased, build from source rather than selecting an image that has not been published.

2. Confirm every HF id. Preserve the exact <org>/<repo> string. Resolve the canonical recipe repo and every alternate checkpoint independently. Verify access using the configured credential through the normal HF client; do not substitute a similarly named cached model or invent an alias.

3. Fetch model metadata. Run bash scripts/hf-info.sh <org>/<repo> or inspect the staged checkpoint's config.json / params.json. Extract:

   • architecture: moe if num_experts, num_local_experts, moe.num_experts, or a *MoE* architecture name is present. Otherwise dense.
   • the exact architectures[] / model_type; confirm the runtime resolves that architecture without a custom alias.
   • parameter_count: total params (e.g. "671B", "70B"). Use HF model card or the sum of shard sizes.
   • active_parameters: for MoE, the activated-per-token count (e.g. "37B" on DeepSeek-V3.2). For dense, equal to parameter_count.
   • context_length: max_position_embeddings from config.json (for VL models, from text_config.max_position_embeddings).
   • min_vllm_version: earliest stable support if known. For a source-only preview, also record the tested commit/version; a future target version alone is not enough.

4. Read the README — don't skip this. Run curl -sL "https://huggingface.co/<org>/<repo>/resolve/main/README.md" and scan the install / serve / usage sections in full. Configs are not enough; model authors put load-bearing requirements in prose. Mine the README for:

   • min_vllm_version / — phrases like "install vllm nightly", "requires nightly wheels", or an install snippet using mean + . A specific tag like "vLLM >= 0.12.0" sets that version. Don't default to when the README says otherwise.

YAML schema (top-level fields, in order)

meta:
  title: "..."                    # display name (e.g. "DeepSeek-V3.2")
  slug: "..."                     # lowercase-kebab (legacy, keep consistent with title)
  provider: "..."                 # human-readable org label (e.g. "DeepSeek")
  description: "..."              # one-sentence summary
  date_added: YYYY-MM-DD          # initial catalog addition; never change
  date_updated: YYYY-MM-DD        # latest material recipe change
  difficulty: beginner|intermediate|advanced
  tasks:                          # one or more of: text, multimodal, omni, embedding
    - text
  performance_headline: "..."     # optional pithy line for cards
  related_recipes: []             # optional list of "<org>/<repo>" ids
  # Optional. Tri-state:
  #   `verified`    — you've run this recipe on this GPU end-to-end (green ✓).
  #   `unsupported` — not yet runnable here today (compat gap, missing kernel,
  #                   upstream blocker). Pill disabled in UI with "Not yet
  #                   supported" tooltip. May flip later — revisit on updates.
  #   absent        — silent default, assumed to work. Don't mark "untested".
  hardware:
    h200: verified
    mi355x: verified
    # mi300x: unsupported    # e.g. when a required kernel/feature is missing


     
         
  
  
  
  
  
  
  
  
  
  
   
  
  
  
  
   
  
  
  
  
  
  
  
  
  
  
  
  
    
       
       
    
       
   
             
           
             
   []                   
   {}                    




    
     
                    
                      
                                  
                                  


                     
     
     [, , ]
  
     
     [, ]
                    
                 
     [, ]

                  
                    


                          
     
         
     
                              
        
     
     
     
     []  
     []
     {}

            
                   
                  
                  
   
                 
                   
                   
                       
  
  








  
     

               
      {  [],  {} }
   {  [],  {} }
         {  [],  {} }

               
  
                              
                                  
                                  
                                  
                                  
                                  
                                  
                                  
                                  
     []
     {}

                           
  
  
  
  
  
  
  
  
  
   [ ]

VRAM formula

vram_minimum_gb = ceil(params × bytes_per_param × 1.2) where params is the total parameter count (MoE includes inactive experts — they still live in VRAM).

Precision	Bytes/param
bf16, fp16	2
fp8, int8, awq, gptq (8-bit)	1
int4, nvfp4, fp4, mxfp4 (4-bit)	0.5

Example: a 70B BF16 model → 70 × 2 × 1.2 = 168 GB. Round up.

If the variant is model_id-overridden and the override is a different base model with its own param count (e.g. a distilled FP4 checkpoint), use the override's parameter count — verify it via HF.

Mixed-precision quants (NVFP4 / ModelOpt) — don't trust the bytes-per-param table. NVIDIA ModelOpt NVFP4 checkpoints are not uniformly 4-bit: only the MLP linears drop to NVFP4 (W4A16), while attention linears + KV cache stay FP8 and embeddings/norms stay higher precision. hf_quant_config.json shows quant_algo: MIXED_PRECISION in this case. The pure params × 0.5 × 1.2 formula then underestimates — e.g. nvidia/Qwen3.6-27B-NVFP4 is ~21.9 GB on disk, not the 13.5 GB the table implies, so 27B × 0.5 × 1.2 = 17 is wrong (the weights alone exceed it). For any mixed-precision checkpoint, size from the real weight footprint instead:

# total_size is in bytes → GB; then × 1.2 for KV/activation overhead
curl -sL "https://huggingface.co/<org>/<repo>/resolve/main/model.safetensors.index.json" \
  | python3 -c "import json,sys; print(round(json.load(sys.stdin)['metadata']['total_size']/1e9*1.2))"

So vram_minimum_gb = ceil(real_checkpoint_GB × 1.2) (Qwen3.6-27B-NVFP4 → ceil(21.9 × 1.2) = 27). The bytes-per-param table stays correct for uniform quants (plain int4/awq/gptq/fp8, and full-model FP4).

Naming and conventions

• Feature keys: prefer tool_calling, reasoning, spec_decoding. Don't use mtp — it's been renamed across the repo.
• Strategy list: MoE recipes usually support every serving strategy; dense recipes are limited to single_node_tp and multi_node_tp (TEP/DEP require MoE). KV offload is a separate axis and is NOT listed here — Off / Simple / both Mooncake modes are implicit on every non-omni recipe and COMPOSE with whatever serving strategy is selected (each Mooncake instance runs the strategy's exact command; parallelism never comes from the KV layer).
• KV Offload gating: fail-open. Only add kv_cache_strategy_hardware when a Mooncake mode is known NOT to work on a specific GPU — mark that strategy × GPU pair unsupported. Absence = assumed to work, same convention as meta.hardware.
• Variants: quantized variants reuse the base name (fp8, nvfp4, int4). If the quantized checkpoint is authored by someone else (e.g. nvidia/*-NVFP4), set model_id: inside the variant.
• Tasks: omni means served via vLLM-Omni (vllm serve <model> --omni). Add a top-level omni: block listing the task ids the recipe supports — bare strings for catalog defaults (tasks: [t2i]) or { id, model_id?, vram_minimum_gb?, description?, extra_args?, curl?, benchmark? } overrides when a task swaps the checkpoint (Wan2.2), needs per-task flags, or targets a non-LLM benchmark endpoint. benchmark is a static command override when the default vllm bench serve command is not applicable. Leave omni.serve_binary unset — vllm serve <model> --omni is correct for every recipe on vLLM 0.20.0+, where the vllm console-script delegates to vllm-omni on the --omni flag. No recipe currently overrides it; only a recipe pinned below 0.20.0 would need to. The catalog is src/lib/omni-tasks.js; do not add --omni to model.base_args (auto-injected).

Validation checklist

Before committing:

1. node scripts/build-recipes-api.mjs succeeds and the new recipe appears in the line count.
2. node -e "const d = require('./public/<hf_org>/<hf_repo>.json'); console.log(d.model.parameter_count, d.variants.default.vram_minimum_gb)" prints sensible values.
3. Inspect git diff --check and confirm the YAML top-level key order matches the schema.
4. If the recipe marks any KV-store mode unsupported under kv_cache_strategy_hardware, spot-check public/<hf_org>/<hf_repo>/hw/<gpu>/strategies/: no kv_store_* file may exist for an opted-out GPU (they are emitted for all other scalable hardware by default).
5. Preview the recipe route and select each authored hardware, variant, strategy, and feature. Verify the Install tabs, serve command, Verify model id, Bench model id, verified badge, and Guide links.
6. Compare the displayed command with the exact successful runtime command. Clearly label any advertised-but-untested context length or optional feature.

Commit

Stage only the new recipe (and providers.js if edited):

git add models/<hf_org>/<hf_repo>.yaml src/lib/providers.js
git commit -s -m "Add <hf_org>/<hf_repo> recipe"

Do not stage public/ (it's generated) or the design docs.