xybrid-init

name	xybrid-init
description	Generate model metadata for an ML model so it works with xybrid.

Generate a model_metadata.json for any ML model so it works with xybrid.

The user may provide a HuggingFace repo, a local directory, or nothing (in which case ask).

Input: $ARGUMENTS (optional — HuggingFace repo ID, URL, or local path)

---

Step 1: Determine Source

If $ARGUMENTS is provided, use it. Otherwise ask:

What model do you want to set up?

• Paste a HuggingFace repo (e.g. hexgrad/Kokoro-82M-v1.0-ONNX)
• Or a local directory path (e.g. ./my-model/)

Detect the source type:

• Contains huggingface.co/ or matches org/repo pattern → HuggingFace
• Is a local path that exists → Local directory
• Otherwise → ask the user to clarify

---

Step 2: Gather Context

If HuggingFace:

Fetch these three resources (use WebFetch for each):

1. Model card: https://huggingface.co/{repo}/raw/main/README.md
2. File listing: https://huggingface.co/api/models/{repo} (look at the siblings array for file names and sizes)
3. Config file (if exists): https://huggingface.co/{repo}/raw/main/config.json

Also check for these files (fetch if they exist in the file listing):

• tokenizer_config.json
• tokenizer.json
• generation_config.json

If Local directory:

1. List all files in the directory
2. Read any README.md, config.json, tokenizer_config.json if present
3. If Python is available and there's an .onnx file, inspect it:

   python3 -c "import onnx; m = onnx.load('MODEL.onnx'); print('Inputs:', [(i.name, [d.dim_value for d in i.type.tensor_type.shape.dim]) for i in m.graph.input]); print('Outputs:', [(o.name, [d.dim_value for d in o.type.tensor_type.shape.dim]) for o in m.graph.output])"
   

---

Step 3: Analyze and Generate

Using ALL the gathered context (model card, file list, config, ONNX inputs/outputs), generate a valid model_metadata.json.

Decision Tree

Use the model card description, file extensions, and config to determine the model type:

File-based detection:

• .gguf file → LLM (Gguf template)
• GGML Whisper weights (ggml-*.bin, or a .bin whose model card says whisper.cpp / GGML) → ASR (GgmlWhisper template) — the default ASR path
• .safetensors + whisper architecture → ASR (SafeTensors template, Candle runtime) — opt-in only, needs --features candle; prefer the GGML bundle above
• .onnx file → continue to task detection below
• .mlmodel / .mlpackage → CoreML template
• .tflite → TfLite template

Task detection (from model card + config):

• Model card mentions "text-to-speech", "TTS", "speech synthesis" → TTS
• Model card mentions "speech recognition", "ASR", "transcription" → ASR
• Model card mentions "embedding", "sentence transformer", "similarity" → Embedding
• Model card mentions "classification", "image", "vision", "ImageNet" → Vision
• Model card mentions "language model", "text generation", "chat", "instruct" → LLM
• Model card mentions "object detection", "YOLO", "segmentation" → Vision

Schema Reference

The model_metadata.json must conform to this exact schema:

{
  "model_id": "string (required) — kebab-case identifier",
  "version": "string (required) — model version",
  "description": "string (optional) — human-readable description",
  "execution_template": { "type": "...", ... },
  "preprocessing": [ ... ],
  "postprocessing": [ ... ],
  "files": [ "list of all required files" ],
  "metadata": { "task": "...", ... },
  "voices": { "... (TTS only)" }
}

Common optional metadata fields:

• tool_calling (boolean, LLMs only): advisory declaration that xybrid's local tool calling works end-to-end for this model — the template accepts a tools context AND the model emits a call format xybrid parses (currently LFM2-family pythonic and gemma-4-family call: notation). Declare true only for those verified families; omit when unknown (never infer from architecture); a model whose template renders tools but whose emissions xybrid cannot parse must NOT declare true — it would produce silent no-call responses.

Execution Templates

Choose ONE:

// ONNX model
{ "type": "Onnx", "model_file": "model.onnx" }

// GGML Whisper (whisper.cpp — the default ASR path, feature `asr-whispercpp`)
// `language`: omit or null to auto-detect. `audio_ctx`: 0 = no encoder
// truncation (the safe default — truncating is the biggest streaming speed
// lever but too much of it makes the decoder loop, so opt in per model after
// a quality check). `translate`: true translates to English instead of
// transcribing in the source language.
{ "type": "GgmlWhisper", "model_file": "model.bin", "language": "en", "audio_ctx": 0, "translate": false }

// SafeTensors (Candle runtime — Whisper only; opt-in `candle` feature, in no platform preset)
{ "type": "SafeTensors", "model_file": "model.safetensors", "architecture":      


       


     


     

Preprocessing Steps

Choose the appropriate chain based on task:

TTS (text-to-speech):

[{ "type": "Phonemize", "tokens_file": "tokens.txt", "backend": "MisakiDictionary", "add_padding": true, "normalize_text": true }]

Backends: MisakiDictionary (default, pure Rust), EspeakNG (multi-language, needs system install), CmuDictionary (legacy), OpenPhonemizer (hybrid dictionary + neural)

ASR (speech recognition) with ONNX:

[{ "type": "AudioDecode", "sample_rate": 16000, "channels": 1 }]

ASR with Whisper SafeTensors: empty [] (Candle handles internally)

Text embedding / NLP:

[{ "type": "Tokenize", "vocab_file": "tokenizer.json", "tokenizer_type": "WordPiece", "max_length": 512 }]

Tokenizer types: WordPiece (BERT), BPE (GPT), SentencePiece (T5)

Image classification / vision:

[
  { "type": "Resize", "width": 224, "height": 224 },
  { "type": "Normalize", "mean": [0.485, 0.456, 0.406], "std": [0.229, 0.224, 0.225] }
]

Use ImageNet normalization values unless model card specifies otherwise.

LLM (GGUF): empty [] (llama.cpp handles internally)

Postprocessing Steps

TTS:

[{ "type": "TTSAudioEncode", "sample_rate": 24000, "apply_postprocessing": true }]

ASR (CTC-based, e.g. Wav2Vec2):

[{ "type": "CTCDecode", "vocab_file": "vocab.json", "blank_index": 0 }]

ASR (Whisper SafeTensors): empty []

Text embedding:

[{ "type": "MeanPool", "dim": 1 }]

Image classification:

[{ "type": "Softmax", "dim": 1 }]

Or { "type": "Argmax" } if you just need the class index.

LLM: empty []

Voice Config (TTS only)

If the model has voice embeddings (e.g. voices.bin):

{
  "voices": {
    "format": "embedded",
    "file": "voices.bin",
    "loader": "binary_f32_256",
    "default": "voice_id",
    "selection_strategy": "TokenLength",
    "catalog": [
      { "id": "voice_id", "name": "Display Name", "index": 0, "gender": "female", "language": "en-US", "style": "neutral" }
    ]
  }
}

---

Step 4: Validate

Before presenting the result, verify:

1. All files in files array exist (in the HF repo or local directory)
2. model_file matches an actual file name
3. Preprocessing/postprocessing steps match the model task
4. Step types are valid — only use the types listed above
5. The JSON is valid — parseable, no trailing commas

---

Step 5: Real Examples for Reference

TTS (Kokoro 82M)

{
  "model_id": "kokoro-82m",
  "version": "1.0",
  "description": "Kokoro 82M - High-quality TTS with 24 voices",
  "execution_template": { "type": "Onnx", "model_file": "model.onnx" },
  "preprocessing": [{ "type": "Phonemize", "tokens_file": "tokens.txt", "backend": "MisakiDictionary", "add_padding": true, "normalize_text": true }],
  "postprocessing": [{ "type"      
      
              

LLM (Qwen 3.5 0.8B)

{
  "model_id": "qwen3.5-0.8b",
  "version": "1.0",
  "description": "Qwen 3.5 0.8B - Lightweight multilingual LLM",
  "execution_template": { "type": "Gguf", "model_file": "Qwen3.5-0.8B-Q4_K_M.gguf", "context_length": 4096 },
  "preprocessing": [],
  "postprocessing": [],
  "files": ["Qwen3.5-0.8B-Q4_K_M.gguf"],
  "metadata": { "task": "text-generation", "architecture": "qwen35", "backend"      

Text Embedding (all-MiniLM)

{
  "model_id": "all-minilm",
  "version": "L6-v2",
  "execution_template": { "type": "Onnx", "model_file": "model.onnx" },
  "preprocessing": [{ "type": "Tokenize", "vocab_file": "tokenizer.json", "tokenizer_type": "WordPiece", "max_length": 512 }],
  "postprocessing": [{ "type": "MeanPool", "dim": 1 }],
  "files":    
        

ASR (Whisper Tiny — SafeTensors)

{
  "model_id": "whisper-tiny",
  "version": "1.0",
  "description": "Whisper Tiny - Fast multilingual ASR (Candle runtime)",
  "execution_template": { "type": "SafeTensors", "model_file": "model.safetensors", "config_file": "config.json", "tokenizer_file": "tokenizer.json" },
  "preprocessing": [],
  "postprocessing": [],
  "files": ["model.safetensors", "config.json", "tokenizer.json", "melfilters.bytes"],
  "metadata":          

Image Classification (MNIST)

{
  "model_id": "mnist-digit-recognition",
  "version": "12",
  "description": "MNIST handwritten digit recognition",
  "execution_template": { "type": "Onnx", "model_file": "model.onnx" },
  "preprocessing": [
    { "type": "Reshape", "shape": [1, 1, 28, 28] },
    { "type": "Normalize", "mean": [0.0], "std":  
  
        
   
             

---

Step 6: Present and Save

Show the generated model_metadata.json to the user with a brief explanation of the key decisions made (e.g., "Using MisakiDictionary phonemizer because the model card says it's a Kokoro-based TTS model").

Then ask:

Save to {directory}/model_metadata.json?

If the user confirms, write the file.

---

Step 7: Download Model Files (if HuggingFace)

If the source is HuggingFace and the model files aren't local yet, offer to download:

Download model files (~{size})? This will fetch:

• model.onnx (150 MB)
• voices.bin (24 KB)
• ...

If the user confirms, download each file listed in the files array:

curl -L "https://huggingface.co/{repo}/resolve/main/{file}" -o "{directory}/{file}"

For files in subdirectories (e.g. misaki/us_gold.json), create the subdirectory first.

---

Step 8: Next Steps

After saving, print:

Your model is ready. Next steps:

# Test it works (use --input-audio <file>.wav for ASR models)
xybrid run --model {model_id} --input-text "test input"

# Or from Rust
cargo run --example your_test -p xybrid-core

# Or use /test-model to validate end-to-end

If /test-model is available (the user has xybrid cloned), suggest running it.