mblt-model-zoo

name	mblt-model-zoo
description	Use this skill when working in the Mobilint Model Zoo repository. It covers the repo layout, common edit patterns for vision model wrappers, transformers and MeloTTS extras, CLI entry points, and safe validation strategies when Mobilint NPU hardware or downloaded models may be unavailable.

Mobilint Model Zoo

When to Use This Skill

Use this skill for changes anywhere in this repository, especially when the task involves:

• Vision model wrappers, preprocessors, postprocessors, datasets, or evaluation helpers
• Hugging Face transformer integrations under mblt_model_zoo/hf_transformers
• MeloTTS integration under mblt_model_zoo/MeloTTS
• CLI entry points under mblt_model_zoo/cli
• Pytest or benchmark changes
• Repo docs that need to reflect the project's hardware-dependent workflows

First Reads

Start with the smallest set of files that anchor the task:

• pyproject.toml for dependencies, extras, scripts, Ruff settings, and Python versions
• README.md for the public package contract
• Area-specific docs such as tests/vision/TEST.md, tests/transformers/TEST.md, tests/MeloTTS/TEST.md, benchmark/vision/README.md, benchmark/transformers/README.md, or mblt_model_zoo/hf_transformers/README.md
• git status --short before editing so you do not overwrite unrelated user changes

Repo Map

• mblt_model_zoo/vision: Vision public API, model wrappers, preprocessing, postprocessing, datasets, evaluation, result objects, and task subpackages including face detection, OCR, and oriented bounding boxes
• mblt_model_zoo/compile: Installable compilation APIs. Vision compilation uses qbcompiler as an optional dependency loaded only when compilation starts.
• mblt_model_zoo/hf_transformers: Custom model, config, and proxy integrations for Transformers
• mblt_model_zoo/MeloTTS: MeloTTS runtime, API, CLI glue, and text normalization utilities
• mblt_model_zoo/cli: Installed CLI entry points. mblt-model-zoo is defined in pyproject.toml, and mblt-melotts-download is also exposed for MeloTTS setup
• tests: Pytest suites grouped by feature area
• benchmark/vision: Vision dataset organizers and benchmark scripts
• benchmark/transformers: TPS benchmark scripts and CLI examples for Hugging Face integrations

Working Conventions

Vision Model Work

Most vision model files follow the same pattern:

1. One or more ModelInfoSet enums define model_cfg, pre_cfg, and post_cfg.
2. An MBLT_Engine subclass exposes the public model constructor.
3. The model family is re-exported through the task package __init__.py, then through mblt_model_zoo/vision/__init__.py.

When editing these files:

• Keep OrderedDict-based config blocks unless the surrounding file already moved away from that pattern.
• Preserve public constructor arguments such as local_path, model_type, infer_mode, and product.
• Legacy compatibility wrappers still accept product, but the current YAML-backed registry ignores it. Use explicit model_cls, model_type, or model_path values when a task needs a non-default artifact.
• Be careful with changes that affect list_models() discovery. Exported classes must remain subclasses of MBLT_Engine.
• Keep mblt_model_zoo/vision/__init__.py lazy top-level compatibility exports aligned with task package __all__ lists. The current package still supports imports such as from mblt_model_zoo.vision import ResNet50, even though MBLT_Engine and task subpackages are the preferred surfaces for new code.

Shared Vision Pipeline Work

For preprocess or postprocess changes, check these modules first:

• mblt_model_zoo/vision/wrapper.py
• mblt_model_zoo/vision/utils/preprocess
• mblt_model_zoo/vision/utils/postprocess
• mblt_model_zoo/vision/utils/results.py
• mblt_model_zoo/vision/utils/types.py

Preserve behavior for the current dual-runtime contract unless the task explicitly changes it:

• Prefer model_path in new docs, examples, and tests. Treat mxq_path and onnx_path as compatibility aliases unless the task is specifically about those legacy names.
• mblt_model_zoo/vision/wrapper.py now auto-detects mxq versus onnx from model_path or file_cfg.model_path when framework is omitted.
• If an explicit framework conflicts with a local .mxq or .onnx suffix, preserve the current fail-fast ValueError behavior instead of silently switching runtimes.
• Preserve behavior for both local artifacts and Hugging Face downloads. For ONNX models, the wrapper may resolve a sibling .onnx file next to a local .mxq artifact or download the configured ONNX artifact from Hugging Face when needed.

Vision Dataset Work

Store validation dataset definitions in mblt_model_zoo/vision/datasets/*.yaml. Follow the Ultralytics-style path and val convention, and use repository-specific tasks and download metadata for validation routing and organizer sources. Resolve defaults through get_dataset_config_for_task() rather than duplicating paths or URLs. Preserve the organized layouts expected by Custom* datasets and evaluators; DOTAv1 original labels belong in labels/val_original so difficult-object filtering remains active. Keep dataset names and category-ID mappings in YAML, with existing utility label helpers reading from the registry.

Vision Compilation Work

Preserve the optional-dependency isolation around mblt_model_zoo.compile.vision:

• Keep qbcompiler imports lazy and inside the actual compilation path.
• Verify ordinary package imports, mblt_model_zoo.compile.vision imports, and non-compile CLI commands without qbcompiler installed or importable.
• Raise the concise qbcompiler installation error only when compile_vision_model() or mblt-model-zoo compile is invoked.
• Keep qbcompiler and the required ONNX Runtime package in the qbcompiler optional extra. Do not move them into base dependencies merely to simplify compilation imports.
• Preserve three mutually exclusive entry levels: use data_path for the original organized image dataset, subset_path for sampled images, and calib_data_path for preprocessed .npy tensors.
• Start at the supplied level. Skip organization and sampling for subset_path; skip organization, sampling, and preprocessing for calib_data_path, passing validated tensors directly to qbcompiler.
• Keep default downloaded and compiled model paths under ~/.mblt_model_zoo and default datasets under ~/.mblt_model_zoo/datasets; never derive those defaults from the checkout directory.
• Give every vision path that uses randomness an explicit seed default of 0, including packaged APIs, CLIs, benchmarks, and compatibility helpers. Preserve caller-provided seeds.
• Use tests/vision/test_compile_vision.py for compilation API, CLI, cleanup, dataset sampling, and missing-dependency regressions.

Transformers and MeloTTS Work

These areas have optional dependencies and heavier runtime assumptions.

• Install the matching extra before test execution.
• Prefer the commands documented in the corresponding tests/*/TEST.md.
• Narrow test scope with a subdirectory, file path, or -k filter before attempting a full suite.
• Expect some tests to require Mobilint hardware, model downloads, or extra data files.
• Remember the repo also defines optional extras for ONNX Runtime-backed vision work (onnxruntime, onnxruntime-gpu) and Qwen3-ASR (qwen-asr).
• tool.ruff.exclude currently excludes mblt_model_zoo/hf_transformers and mblt_model_zoo/MeloTTS, so preserve local style there instead of forcing broad cleanup.

Documentation Work

• Follow the Markdown rules in AGENTS.md.
• Keep docs operational and repo-specific.
• Keep vision docs aligned with the current public API: MBLT_Engine, list_models(), the shared --model-path CLI option, and framework auto-detect are the preferred surfaces for new guidance.
• benchmark/vision/README.md currently covers organizer flows for ImageNet, COCO, WiderFace, and DOTAv1. ImageNet, COCO, and DOTAv1 benchmark execution sections are complete; WiderFace benchmark execution is still pending.
• For rule or workflow documents, include YAML frontmatter with at least description and paths.

Validation

Use the smallest validation that meaningfully covers your change.

Base Environment

pip install -e . --group dev

Optional Extras

pip install -e ".[transformers]" --group dev
pip install -e ".[MeloTTS]" --group dev
pip install -e ".[onnxruntime]" --group dev
pip install -e ".[onnxruntime-gpu]" --group dev
pip install -e ".[qwen-asr]" --group dev
pip install -e ".[qbcompiler]" --group dev

Lint and Format Python

ruff check mblt_model_zoo tests benchmark
ruff format mblt_model_zoo tests benchmark
pre-commit run --files path/to/touched_file.py

Targeted Tests

pytest tests/vision/test_resnet50.py
pytest tests/transformers/text_generation/non_batch/test_qwen2.py -k "0.5B"
pytest tests/MeloTTS/test_melo.py -k "KR"
pytest tests/vision/test_cli_vision.py
pytest tests/vision/test_wrapper_download.py
pytest tests/vision/test_compile_vision.py

For the recent vision runtime and CLI path-resolution work, these targeted tests are especially relevant:

pytest tests/vision/test_cli_vision.py
pytest tests/vision/test_wrapper_download.py

Hardware-Aware Sweeps

Transformers tests expose shared NPU options through tests/conftest.py. Reuse existing flags such as:

--core-mode
--vision-core-mode
--text-core-mode
--encoder-core-mode
--decoder-core-mode

If the environment does not provide Mobilint NPU runtime support, stop at static checks or the narrowest safe test and call out the limitation explicitly.

The unified vision validation CLI currently supports:

• ImageNet for image classification models.
• COCO for object detection, instance segmentation, and pose estimation models.
• WiderFace for face detection models.
• DOTAv1 for oriented bounding box models.

Things to Avoid

• Do not rewrite large generated or vendor-like sections just to normalize style.
• Do not broaden test scope by default. Full suites are expensive and often hardware-bound.
• Do not revert unrelated work already present in the tree.
• Do not duplicate long command instructions across docs when an existing README or TEST.md already owns them.