mobius

Use this skill when adding a new HuggingFace model architecture to mobius — including LLM, encoder-only, encoder-decoder, vision, audio, diffusion, or multimodal models. Covers the full workflow: config extraction, model class creation, registry registration, weight preprocessing, and testing. Also covers MoE and hybrid architectures.

Use this skill when choosing attention mask types, understanding ORT CUDA kernel dispatch, or optimizing attention performance. Covers bool mask vs float additive bias, Flash/MEA/unfused kernel selection, GQA dispatch rules, and nonpad_kv_seqlens for Flash eligibility.

Use when building/exporting ONNX models with the `mobius build` CLI (especially Phi-3 / Phi-3.5 or any model with `--execution-provider cuda` GQA fusion and/or `--static-cache`). Covers the current CLI syntax, the dtype flag values, the GQA-vs-static-cache interaction, how to verify fp16 GQA exports load in onnxruntime (the historical packed-QKV FLOAT32 load bug is fixed by the fp16 GQA fold-fix), and why fp16 GQA exports need VALUE-based weight checks (corr≈1.0 / norm), not just initializer count/dtype, to catch silently-zeroed packed-QKV weights.

Use this skill when adding or modifying a model that uses Mixture-of-Experts (MoE) layers. Covers gate variants (TopKGate, SparseMixerGate), MoELayer composition and expert routing, expert weight naming conventions for HuggingFace alignment, and preprocess_weights mappings for stacked expert tensors. Applicable to models like Mixtral, DeepSeek, and Qwen-MoE.

Add or modify multimodal (vision + language + audio) models in mobius. Use when wiring a VisionModel, projector, InputMixer, or VisionLanguageTask; handling image/audio token placeholders; choosing a projector variant; or splitting a model into 3-or-4 ONNX sub-models for ORT GenAI deployment.

Use this skill when adding or modifying a model's preprocess_weights method to align ONNX parameter names with HuggingFace weight names. Covers nn.ModuleList for Sequential patterns, wrapper modules for nesting, placeholder modules, non-consecutive indices, and which rename categories cannot be eliminated. Reduces or eliminates weight name renames by structuring nn.Module attributes to match HuggingFace naming conventions.

Debug and reduce CUDA Memcpy nodes in ONNX models built by mobius. Use when ORT warns about MemcpyFromHost/MemcpyToHost nodes added for CUDAExecutionProvider, when profiling shows excessive host-device transfers, or when CUDA graph capture fails due to Memcpy. Covers root-cause analysis, op-level attribution, and proven fix patterns for the most common offenders.

Debug wrong, garbled, or divergent output from multimodal ONNX models (vision-language, vision+audio, multi-encoder). Use when ORT GenAI multimodal output doesn't match HuggingFace, when building a new multimodal model and verifying component-by-component parity (vision encoder, speech encoder, embedding/projector, text decoder), when integration tests fail with large numerical differences, or when CUDA EP produces different results than CPU. Covers the 3-stage VL pipeline and 4-model multi-encoder pipeline isolation, 3D M-RoPE position IDs, CUDA EP gotchas, and systematic stage-by-stage comparison methodology.

Use this skill when verifying that a new model is truly done and ready to merge. Provides a Definition-of-Done checklist covering all five test confidence levels (L1 graph build through L5 Foundry Local smoke-test), ORT GenAI runtime validation, Olive quantization compatibility, multi-dtype (f32/f16/bf16) and multi-EP (CPU/CUDA/DML) correctness, documentation requirements, and code review criteria.

Use this skill when exporting ONNX models with mobius and quantizing them with Olive for deployment. Covers the mobius CLI, EP options, INT4 quantization (Q4_K_M and NF4), HuggingFace upload structure, GPU-accelerated quantization, common issues, and testing quantized models.

Use this skill when profiling ONNX model performance on CUDA with OnnxRuntime. Covers ORT session profiling, reading JSON results, identifying compute and memory bottlenecks, debugging memcpy nodes, and measuring GenAI pipeline throughput (tok/s, TTFT).

Create or extend reusable ONNX building blocks in the mobius component library. Use when adding Attention, MLP, norm, RoPE, or embedding components; understanding parameter naming and nn.Module conventions; applying design principles (subclass over flags, model-agnostic); or wiring shared-weight / per-layer adapter patterns.

Use this skill when writing or modifying tests for mobius models and components. Covers the L1–L5 confidence system: unit tests (graph construction from tiny configs), integration tests (numerical parity with HuggingFace), golden tests (pre-computed reference comparison), and generation tests (multi-token output verification). Includes test commands, shared config infrastructure, testing utilities, tolerance guidelines, and common pitfalls.

Use this skill when building OnnxRuntime and onnxruntime-genai from source with CUDA support. Covers CUDA toolkit and cuDNN installation, ORT build flags, GenAI build linked to custom ORT, verification, and common build issues.

Use this skill when deploying mobius-exported ONNX models to Microsoft Foundry Local for local inference. Covers custom model registration via the cache directory, inference_model.json format, CLI and API usage, SDK limitations, and GenAI version compatibility.

Use this skill when generating genai_config.json or image_processor.json for onnxruntime-genai model exports, debugging ORT GenAI model loading errors, understanding the model type registry, or integrating ONNX models with the onnxruntime-genai runtime. Covers the full config format, the MultiModal pipeline architecture (vision/audio/embedding/decoder), and the ort-extensions image_processor.json format.

Use this skill when adding or modifying a diffusion or image-generation model in mobius. Covers UNet, VAE, DiT, Flux, SD3, ControlNet, adapters, and QwenImage architectures. Includes pipeline detection from diffusers configs, DiffusionTask and DiffusionModel classes, building blocks (timestep embeddings, cross-attention, downsampling/upsampling), and weight loading conventions for diffusers checkpoints.

Use this skill when managing parallel workstreams across multiple AI agents on a shared Git repository. Covers worktree isolation to prevent merge conflicts, commit coordination protocols, verification gates between dependent tasks, context management for stateless agents, and failure recovery patterns. Based on real experience coordinating 10–17 agents simultaneously.

Use this skill when creating ONNX rewrite rules that transform parts of an ONNX model graph, such as replacing standard ops with custom or fused ops. Covers the onnxscript.rewriter RewriteRuleClassBase API, pattern matching with op functions, check/rewrite method conventions, file organization under src/mobius/rewrite_rules/, and testing patterns for verifying rule correctness.