k8s-launch-kit-config

name	k8s-launch-kit-config
version	1.1.0
description	Use this skill when the user needs help understanding, creating, or editing a k8s-launch-kit (l8k) configuration file (l8k-config.yaml or cluster-config.yaml). Activate for: config file questions, parameter tuning, subnet configuration, NV-IPAM setup, DOCA driver settings, NIC configuration operator settings, changing MTU, VFs, resource names, or understanding what any config field does.
metadata	{"requires":{"skills":["k8s-launch-kit-shared"]}}

l8k: Configuration Files

PREREQUISITE: Read ../k8s-launch-kit-shared/SKILL.md for install paths, global flags, and output modes.

Understand, create, or edit l8k configuration files.

File Types

File	Source	Purpose
cluster-config.yaml	Generated by l8k discover	Hardware inventory (groups, NICs, rails, capabilities)
l8k-config.yaml	User-created or copied from cluster-config	Full config with both hardware + deployment settings

Usage

# Use a config file for generation
l8k generate --user-config my-config.yaml \
  --fabric ethernet --deployment-type sriov \
  --save-deployment-files ./output

# Combine user config with live discovery (hardware refreshed, settings kept)
l8k discover --user-config my-config.yaml \
  --kubeconfig ~/.kube/config \
  --save-cluster-config ./updated-config.yaml

Config Sections Quick Reference

Section	What It Controls
networkOperator	Operator namespace, version, image repository
docaDriver	OFED/DOCA driver image, version, blacklist settings
nvIpam	NV-IPAM IP pool ranges and subnet generation
sriov	VF count, resource prefix, MTU, link type
hostdev	Host device resource name
rdmaShared	RDMA shared device resource name
ipoib	IPoIB master interface, resource name
macvlan	MacVLAN master interface, mode
nicConfigurationOperator	NIC firmware template settings
spectrumX	OVS bridge config, multiplane mode, RDMA settings
profile	Profile selection criteria (fabric, deployment, multirail)
clusterConfig[]	Per-group hardware: NICs, nodes, capabilities, selectors

Each clusterConfig[] entry has these key fields:

• identifier — group name (used for NicNodePolicy naming). For groups with both machineType and gpuType resolved, this is the sanitised machine label (<machineType>-<gpuType>); otherwise a fallback group-N.
• machineType — server model (e.g. PowerEdge-XE9680); populated from nvidia.com/gpu.machine label or DMI fallback.
• gpuType — GPU SKU (e.g. NVIDIA-H200); populated from nvidia.com/gpu.product label or nvidia-smi fallback. Note: this field used to be called productType — the rename happened to disambiguate it from the server model. Old productType: keys in hand-authored configs must be renamed to gpuType:.
• capabilities.nodes.{sriov,rdma,ib} — what the underlying hardware supports.
• pfs[] — physical function list with PCI address, device ID, RDMA device, network interface, traffic class, rail, NUMA, GPU affinity.
• nodeSelector — Kubernetes node selector for this group. Source groups key on the machine label written by l8k discover: nvidia.kubernetes-launch-kit.machine: <machineType>-<gpuType>. Auto-merged groups (different machineTypes sharing a GPU type) key on nvidia.kubernetes-launch-kit.gpu: <gpuType> instead — discovery writes both labels onto every node, so the merged selector binds correctly across source machineTypes.
• workerNodes — explicit hostnames (populated by discovery).

For the full field-by-field reference with types, defaults, and descriptions, read references/config-schema.md.

Topology Presets

The presets/ directory contains pre-recorded topologies for known hardware combinations. A preset is a topology.yaml file with the following shape:

machineType: PowerEdge-XE9680   # required
gpuType: NVIDIA-H200            # required — both keys are matched as a pair
nicModel: BlueField-3 SuperNIC (ConnectX-7)
gpuInterconnect: NV18
numaNodes: 2

# Required if the preset will be used with `l8k generate --for`.
# Discovery-time overlay does not need this block.
capabilities:
  nodes:
    sriov: true
    rdma: true
    ib: false

pfs:
  - deviceID: a2dc
    pciAddress: 0000:1a:00.0
    traffic: east-west
    rail: 0
    numaNode: 0
    connectedGPU: GPU0
    gpuProximity: PIX

Lookup is exact-match on (machineType, gpuType). No any-GPU fallback — a preset that doesn't declare gpuType: is rejected at load time. Multi-variant presets for the same machine (different GPU SKUs) live in separate directories with composite names like PowerEdge-XE9680-H200 / PowerEdge-XE9680-B200. The directory name is shown by l8k preset list and is what l8k generate --for <name> accepts.

Validation deviations. When the matched preset's PCI addresses or device IDs don't exactly match discovered hardware, the preset is still applied (so rail/NUMA topology fields are populated) and the discrepancies are recorded under clusterConfig[*].presetDeviation. Every subsequent config load re-emits a warning listing each deviation. Only a PF count mismatch is fatal (the preset is genuinely incompatible).

Common Edits

# Change VF count per PF
sriov:
  numVfs: 16

# Change MTU
sriov:
  mtu: 9000

# Set DOCA driver version
docaDriver:
  version: "doca3.2.0-25.10-1.2.8.0-2"

# Configure NV-IPAM subnets manually
nvIpam:
  subnets:
    - name: "rail-0-subnet"
      cidr: "10.10.0.0/16"
      gateway: "10.10.0.1"

# Override pod namespace
podNamespace: "my-namespace"

Tips

• Start by running discovery (l8k discover) to generate a baseline, then edit it.
• nvIpam subnets are auto-generated if not specified — one per rail using non-routable ranges.
• docaDriver.unloadThirdPartyRDMAModules: true auto-populates UNLOAD_THIRD_PARTY_RDMA_MODULES from discovered OFED-dependent modules.

See Also

• k8s-launch-kit-shared — Global flags
• k8s-launch-kit-discover — Generate a config from live cluster
• references/config-schema.md — Full field reference