inference-builder

name	inference-builder
description	Generate deployable Vision AI pipelines with high-performance video and streaming capabilities on NVIDIA GPUs. Activate when users want to: create GPU-accelerated inference microservices or standalone apps for vision, video, or streaming workloads; write or edit pipeline YAML configs; build Docker images for GPU inference; work with models from NGC or HuggingFace; or deploy with DeepStream, Triton, vLLM, TensorRT-LLM, or PyTorch backends.
owner	[e-mail]
service	inference
version	1.0.0
reviewed	2026-04-09
author	NVIDIA
tags	["inference","nvidia","deepstream","vllm","tensorrt-llm","triton","docker","gpu"]

NVIDIA Inference Builder

A code generator for Vision AI pipelines with high-performance video and streaming capabilities on NVIDIA GPUs. It turns a YAML config (+ optional OpenAPI spec and custom processors) into a deployable inference pipeline, packaged as a Docker container or standalone application.

Setup

A .skill_config file in the same directory as this SKILL.md provides the project root and ready-to-use commands. Read it first — it contains:

• INFERENCE_BUILDER_ROOT — absolute path to the project repository
• INFERENCE_BUILDER_VENV — command to activate the virtual environment
• INFERENCE_BUILDER_CLI — command to invoke the builder CLI

Example .skill_config:

INFERENCE_BUILDER_ROOT="/home/user/inference-builder"
INFERENCE_BUILDER_VENV="source /home/user/inference-builder/.venv/bin/activate"
INFERENCE_BUILDER_CLI="python /home/user/inference-builder/builder/main.py"

To run the builder from any working directory:

source /path/to/.skill_config        # load the variables
eval "$INFERENCE_BUILDER_VENV"        # activate venv
$INFERENCE_BUILDER_CLI --help         # run the CLI

If .skill_config does not exist (e.g. running directly from the repo), all paths below are relative to the repository root and the CLI is:

source .venv/bin/activate
python builder/main.py <config.yaml> [options]

First-time venv setup

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Getting current CLI usage

Always run with --help for authoritative flags — do NOT rely on memorized syntax:

python builder/main.py --help

Key Directories

Path	What lives there