Skip to main content

fp16-trt-network

Implement or review FP16 and BF16 behavior in one family-owned, strongly typed TensorRT network without changing unrelated families.

Informações da origem

Repositório
NVIDIA/TensorRT-Model-Connect
Última atividade na origem
7 de setembro de 2026 às 07:41
Idioma detectado do SKILL.md
inglês
Estrelas
273
Forks
71

Opções de instalação

Por padrão, está selecionado o prompt que primeiro revisa a origem. Você pode mudar para um comando direto ou baixar uma cópia local.

Revise os arquivos de origem

Leia o SKILL.md e os arquivos complementares exibidos pelo SkillsMP antes de decidir se vai instalar.

Explorador de arquivos
2 arquivos

Exibindo SKILL.md

SKILL.md
Instruções da origem · Visualização somente leitura
name
fp16-trt-network
description
Implement or review FP16 and BF16 behavior in one family-owned, strongly typed TensorRT network without changing unrelated families.
# FP16 and BF16 TensorRT Networks Work in the owning `families/<family>/` implementation. Read its `model.py`, local graph helpers, manifests, and tests before changing dtype behavior. ## Preserve the typed graph - Create the network with `trt.NetworkDefinitionCreationFlag.STRONGLY_TYPED`. - Resolve `request.precision` explicitly and reject unsupported values. - Track checkpoint storage dtype, TensorRT tensor dtype, shape, and layout at every cast-sensitive boundary. - Create constants in an intentional storage dtype and use `network.add_cast(...)` when the runtime dtype differs. - Keep all inputs to an elementwise operation dtype-compatible, including epsilon, masks, scales, and affine parameters. - Preserve compact K/V projection and cache shapes for grouped or multi-query attention. Use FP32 only where the family implementation or measured reference evidence requires it, commonly reductions, normalization, probability arithmetic, or comparison-sensitive outputs. Cast back explicitly. Do not apply a blanket FP32 layer list or remove an established family boundary without evidence. Precision and quantization are separate request values. Change one at a time, and do not add a shared model helper merely because another family has similar code. Copy a small proven pattern into the owner when needed. ## Prove the change Build through the public Python build command with the exact checkpoint and precision. Inspect the resulting bundle, run focused family Python/C++ tests, then run the declared family E2E testcase against its existing oracle. Bundle size is only a signal. Qualification requires the requested graph path to execute and the unchanged family correctness contract to pass. Report any hardware or precision variant not run.
Ver no GitHub