Use this skill when adding a new HuggingFace model architecture to mobius — including LLM, encoder-only, encoder-decoder, vision, audio, diffusion, or multimodal models. Covers the full workflow: config extraction, model class creation, registry registration, weight preprocessing, and testing. Also covers MoE and hybrid architectures.
Use this skill when choosing attention mask types, understanding ORT CUDA kernel dispatch, or optimizing attention performance. Covers bool mask vs float additive bias, Flash/MEA/unfused kernel selection, GQA dispatch rules, and nonpad_kv_seqlens for Flash eligibility.
Use when building/exporting ONNX models with the `mobius build` CLI (especially Phi-3 / Phi-3.5 or any model with `--execution-provider cuda` GQA fusion and/or `--static-cache`). Covers the current CLI syntax, the dtype flag values, the GQA-vs-static-cache interaction, how to verify fp16 GQA exports load in onnxruntime (the historical packed-QKV FLOAT32 load bug is fixed by the fp16 GQA fold-fix), and why fp16 GQA exports need VALUE-based weight checks (corr≈1.0 / norm), not just initializer count/dtype, to catch silently-zeroed packed-QKV weights.
Use this skill when adding or modifying a model that uses Mixture-of-Experts (MoE) layers. Covers gate variants (TopKGate, SparseMixerGate), MoELayer composition and expert routing, expert weight naming conventions for HuggingFace alignment, and preprocess_weights mappings for stacked expert tensors. Applicable to models like Mixtral, DeepSeek, and Qwen-MoE.
Add or modify multimodal (vision + language + audio) models in mobius. Use when wiring a VisionModel, projector, InputMixer, or VisionLanguageTask; handling image/audio token placeholders; choosing a projector variant; or splitting a model into 3-or-4 ONNX sub-models for ORT GenAI deployment.
Use this skill when adding or modifying a model's preprocess_weights method to align ONNX parameter names with HuggingFace weight names. Covers nn.ModuleList for Sequential patterns, wrapper modules for nesting, placeholder modules, non-consecutive indices, and which rename categories cannot be eliminated. Reduces or eliminates weight name renames by structuring nn.Module attributes to match HuggingFace naming conventions.
Debug and reduce CUDA Memcpy nodes in ONNX models built by mobius. Use when ORT warns about MemcpyFromHost/MemcpyToHost nodes added for CUDAExecutionProvider, when profiling shows excessive host-device transfers, or when CUDA graph capture fails due to Memcpy. Covers root-cause analysis, op-level attribution, and proven fix patterns for the most common offenders.
Debug wrong, garbled, or divergent output from multimodal ONNX models (vision-language, vision+audio, multi-encoder). Use when ORT GenAI multimodal output doesn't match HuggingFace, when building a new multimodal model and verifying component-by-component parity (vision encoder, speech encoder, embedding/projector, text decoder), when integration tests fail with large numerical differences, or when CUDA EP produces different results than CPU. Covers the 3-stage VL pipeline and 4-model multi-encoder pipeline isolation, 3D M-RoPE position IDs, CUDA EP gotchas, and systematic stage-by-stage comparison methodology.