| name | libreyolo-export-formats |
| description | Work on LibreYOLO's export subsystem: debug a failing export, fix an exported model that predicts differently from the .pt, extend a format's coverage to a new family/task, or add a new export format. Use when someone reports "export to X fails", "the ONNX/TensorRT model gives different results", asks which formats a family supports, or wants a new format wired. Covers the exporter/backend twin architecture, the metadata round-trip contract, per-format quirks and extras, the parity-testing method, and the support-claim bar. For a user just running export, see use-libreyolo; for Hailo specifically, libreyolo-export-hailo. |
LibreYOLO export formats (developer guide)
Export is two mirrored subsystems, and almost every export bug is a mismatch
between them:
- Exporters (
libreyolo/export/): BaseExporter ABC + one subclass per
format (OnnxExporter, TorchScriptExporter, TensorRTExporter,
OpenVINOExporter, NcnnExporter, TFLiteExporter, CoreMLExporter in
exporter.py, with format-specific helpers in sibling modules). They
serialize the graph and embed LibreYOLO metadata (family, size, task,
imgsz, class names, pose keypoint shape, ...).
- Backends (
libreyolo/backends/): BaseBackend + one subclass per
runtime. LibreYOLO("best.onnx") routes here; the backend reads the
embedded metadata, reproduces the family's preprocessing and
postprocessing (letterbox, NMS or DETR top-k, task-specific decode) in
numpy, and returns the same Results a .pt would.
The two REVIEW.md axioms that govern all of it: backends must behave like
models (same API, same Results, original-canvas coordinates) and
exported runtimes round-trip metadata (what the exporter writes, the
backend must read; a model loaded from an export must not need the user to
re-specify imgsz/task/classes).
The support tiers (do not overclaim)
Formats are not equally supported, and support is per (format, family,
task), not per format. ONNX is the release-gated supported backend; the
others are covered by e2e but several are marked experimental
(supported_backend vs experimental_backend markers in tests/e2e/).
libreyolo formats prints the format list with capabilities; the e2e files
(test_onnx.py, test_tensorrt.py, ...) are the truth for which families
each format actually covers. Before telling anyone "family X exports to Y",
find the (X, Y) case in those tests or run it yourself.
Per-format quirks (the ones that cost hours)
- ONNX: the Python API defaults
dynamic=True; downstream consumers
with static-shape compilers (TensorRT profiles aside, Hailo, some
runtimes) need dynamic=False. simplify=True runs onnxsim. nms=True
embeds NMS into the graph (changes the output contract; the backend knows,
external consumers must be told). Raw detect output for external
consumers is the [1, 84, 8400]-style head tensor; at least one external
C++ project depends on that exact contract, and tests/e2e/'s
sam3dbody contract test guards it: do not reshape it casually.
- TensorRT: version-pinned (
tensorrt-cu12==... in pyproject, matching
[[tool.uv.dependency-metadata]]); engines are GPU-arch-specific
artifacts, never redistributable; fp16 is where accuracy drift first
appears, compare against fp32 before blaming the model.
- TFLite: routes through onnx2tf (needs Python 3.12+ per the extra's
constraint) and includes graph surgery for RF-DETR (GridSample rewrite);
it is the most fragile chain, keep its extra versions synced with
pyproject comments.
- OpenVINO / ncnn: install their extras to test; ncnn goes through pnnx.
- CoreML: macOS-only end to end; e2e coverage lives in
test_coreml_roundtrip.py and skips elsewhere.
- Rectangular imgsz is only supported by the formats in
_RECTANGULAR_EXPORT_FORMATS (see exporter.py); everywhere else,
square-check errors are intentional.
- Some families need export-time wrappers or state snapshots (RT-DETR
wrapper, RF-DETR export-state snapshot/restore in
exporter.py); when an
export mutates model behavior, that pattern is the sanctioned fix.
Debugging "the exported model is wrong"
Work the pipeline in order; the bug class differs by stage:
- Export succeeded but predictions differ from
.pt: run both on the
same image and compare stage by stage. Preprocessing first (the backend
reimplements letterbox in numpy; a 1px offset here looks like "slightly
wrong boxes"), then raw tensor closeness (fp32 should be ~1e-5; fp16
looser), then postprocess (NMS thresholds, top-k count, class-map).
The parity assertions in tests/e2e/test_deterministic_inference.py and
the backend unit tests (tests/unit/test_backend_postprocess.py,
test_backend_validation_adapter.py) show the expected tolerances.
- Export itself fails: check the extra is installed
(
libreyolo checks reports export backends), then whether the op is
task-specific (pose/seg heads export differently; the exporter embeds
task metadata like keypoint shapes). Only then debug the converter
stack, and pin versions before patching code.
- Backend load fails on a file that exported fine: metadata round-trip
bug. Inspect what was embedded (for ONNX, the model's metadata_props)
and what
BaseBackend._read_metadata_* expects; fix the pair, and add
the missing key to both sides plus a unit test.
Extending coverage (family or task to an existing format)
- Make the model's forward export-clean for the format (no data-dependent
Python control flow on tensor values in the traced path).
- Teach the exporter any task metadata the backend will need (the pose
kpt_shape flow in exporter.py is the template).
- Teach the backend the family's decode: NMS-using YOLO-grid vs NMS-free
vs DETR top-k selection is centralized (
_is_nms_free_family,
_rfdetr_num_select in backends/base.py); route, don't fork.
- Add the e2e case to that format's test file with the right markers
(
libreyolo-write-e2e-tests), and a val-parity check if the family has
one (exported-backend val must agree with native val; there is adapter
machinery for exactly this).
Adding a whole new format
Copy the twin structure: exporter subclass (declare format_name, deps via
requires_*, write metadata), backend subclass (read metadata, reuse the
shared pre/postprocess helpers in BaseBackend, never fork them), a lazy
export in libreyolo/__init__.py, an optional-dependency extra in
pyproject.toml (never a core dep), libreyolo formats row, an
experimental_backend-marked e2e file, and docs. A new format starts
experimental and stays that way until family coverage and parity tests say
otherwise. Note the precedent that not everything becomes a format: when the
toolchain cannot be a pip dependency (Hailo's proprietary compiler), ship a
skill/doc for the two-stage flow instead of a format= target.
Related
skills/use-libreyolo/: the user-facing export + exported-inference API.skills/libreyolo-export-hailo/: the no-format precedent, done right.skills/libreyolo-write-e2e-tests/: markers for export coverage.docs/testing.md: which backends gate releases vs run experimentally.
