Skip to main content

fp16-trt-network

Implement or review FP16 and BF16 behavior in one family-owned, strongly typed TensorRT network without changing unrelated families.

Source facts

Repository
NVIDIA/TensorRT-Model-Connect
Last source activity
September 7, 2026 at 07:41
Detected SKILL.md language
English
Stars
276
Forks
73

Install options

The review-first prompt is selected by default. You can switch to a direct command or download a local copy.

Review the source files

Read SKILL.md and any companion files shown by SkillsMP before deciding whether to install.

File Explorer
2 files

Showing SKILL.md

SKILL.md
Source instructions · Read-only preview
name
fp16-trt-network
description
Implement or review FP16 and BF16 behavior in one family-owned, strongly typed TensorRT network without changing unrelated families.
# FP16 and BF16 TensorRT Networks Work in the owning `families/<family>/` implementation. Read its `model.py`, local graph helpers, manifests, and tests before changing dtype behavior. ## Preserve the typed graph - Create the network with `trt.NetworkDefinitionCreationFlag.STRONGLY_TYPED`. - Resolve `request.precision` explicitly and reject unsupported values. - Track checkpoint storage dtype, TensorRT tensor dtype, shape, and layout at every cast-sensitive boundary. - Create constants in an intentional storage dtype and use `network.add_cast(...)` when the runtime dtype differs. - Keep all inputs to an elementwise operation dtype-compatible, including epsilon, masks, scales, and affine parameters. - Preserve compact K/V projection and cache shapes for grouped or multi-query attention. Use FP32 only where the family implementation or measured reference evidence requires it, commonly reductions, normalization, probability arithmetic, or comparison-sensitive outputs. Cast back explicitly. Do not apply a blanket FP32 layer list or remove an established family boundary without evidence. Precision and quantization are separate request values. Change one at a time, and do not add a shared model helper merely because another family has similar code. Copy a small proven pattern into the owner when needed. ## Prove the change Build through the public Python build command with the exact checkpoint and precision. Inspect the resulting bundle, run focused family Python/C++ tests, then run the declared family E2E testcase against its existing oracle. Bundle size is only a signal. Qualification requires the requested graph path to execute and the unchanged family correctness contract to pass. Report any hardware or precision variant not run.
View on GitHub