-
Prefer runtime sysfs manipulation over persistent modprobe configs for iGPU passthrough. Writing to /sys/bus/pci/drivers/vfio-pci/new_id and /sys/bus/pci/drivers/vfio-pci/bind is reversible without initramfs updates. Modprobe blacklists require update-initramfs and a reboot.
-
Single-GPU passthrough is supported for Intel iGPUs and discrete GPUs via Proxmox hookscripts. The hookscript pattern: pre-start unbinds the GPU from the native driver via sysfs and binds to vfio-pci; post-stop reverses the operation. The host runs headless (SSH/web only) while the VM has the GPU. NEVER attempt GPU passthrough on AMD APU iGPUs (Raven Ridge, etc.) — the GPU shares the SoC die and ANY unbind path (sysfs or modprobe -r) hangs the entire system. This is a hardware limitation, not fixable with hookscripts.
-
NEVER run modprobe -r amdgpu or modprobe -r i915 as GPU cleanup. ALWAYS use sysfs operations: unbind from vfio-pci, clear driver_override, PCI rescan. The rescan triggers the kernel to auto-bind the native driver. This is safe on ANY host regardless of GPU count.
-
GPU passthrough hookscript (/var/lib/vz/snippets/gpu-passthrough-hook.sh) manages the full lifecycle: discovers hostpci devices from VM config, stops GPU-consuming LXC containers, suspends conflicting VMs, binds/unbinds via sysfs, persists state in /run/gpu-passthrough/vm-<VMID>.state for post-stop recovery. The hookscript is generalized — works with any GPU vendor (Intel, AMD, NVIDIA) and any VM that uses --hostpci.
-
Previous bug: cleanup ran modprobe -r amdgpu on ALL hosts via E2E cleanup. On ai (single AMD GPU, USB ethernet), this caused a kernel panic. Fix: replaced all GPU modprobe -r with sysfs unbind + PCI rescan.
-
Previous bug: sysfs unbind of AMD Raven Ridge APU iGPU (1002:15dd) via hookscript pre-start hung the entire system. Unlike discrete GPUs, APU iGPUs share the SoC die with the CPU — even sysfs unbind triggers a GPU reset that freezes the NBIO, killing the entire system including USB ethernet (EHOSTUNREACH). This is the same class of failure as modprobe -r amdgpu but at the hardware level.
-
Previous bug: PCI rescan after vfio-pci unbind did NOT auto-bind the native driver when the module was already loaded. DRI devices (/dev/dri/renderD128) did not reappear. Fix: explicitly bind to the native driver after rescan (echo PCI_ADDR > /sys/bus/pci/drivers/i915/bind). The cleanup and hookscript post-stop both must do explicit rebinding, not rely on auto-binding.
-
Cleanup MUST match deployment scope. If the role uses sysfs-only binding (no modprobe configs), cleanup MUST NOT remove modprobe config files. Cleanup MUST also remove hookscript state files from /run/gpu-passthrough/ and the hookscript itself from /var/lib/vz/snippets/.
-
Display-exclusive hookscripts that stop DRI-sharing containers MUST NOT be attached during provisioning. Attaching during kiosk_lxc (Phase 2.5) causes the hookscript to fire when desktop_lxc starts in Phase 3, stopping all DRI containers (Kodi, Moonlight, Kiosk) before their configure plays run.
-
Pattern: deploy the hookscript FILE in the provisioning role, attach it to containers/VMs in a dedicated play AFTER all configure plays finish. This ensures all containers are configured before the hookscript can stop them.
-
Configure roles for DRI-sharing containers should include a defensive "ensure container is running" guard at the top (check pct status, start if stopped, wait for readiness). This handles re-runs where hookscripts may already be attached from a previous cycle.
-
Previous bug: kiosk_lxc deployed AND attached the display-exclusive hookscript during Phase 2.5. When desktop_lxc started in Phase 3, the hookscript's pre-start(400) stopped Kodi (301), Moonlight (302), and Kiosk (401). Configure Kodi then failed with "container '301' not running!" Fix: split hookscript deployment (provisioning) from attachment (post-configure play).
-
Verify assertions for DRI-sharing containers (Kodi, Kiosk, Moonlight) MUST skip systemctl is-active checks when the Desktop LXC is running and has grabbed the DRI render node. The display-exclusive hookscript manages this conflict. Check pct status {{ desktop_ct_id }} and gate the service-active assertion.
-
Previous bug: Kodi container was running but systemctl is-active kodi returned inactive during verify. Root cause: Desktop VM held the iGPU, DRI devices absent from container. Fix: added "'running' not in (_desktop_status.stdout | default(''))" condition to the Kodi service-active assertion.
-
Not all services run as systemd daemons. Moonlight-embedded is an on-demand streaming client binary (/usr/local/bin/moonlight), not a persistent service. Verify assertions for such services should check binary existence and config deployment, NOT systemctl is-active.
-
Previous bug: Moonlight verify assertion checked systemctl is-active moonlight but there IS no moonlight.service — moonlight-embedded is compiled from source as a CLI binary with no systemd unit file. The assertion always failed. Fix: changed to check binary existence and config file presence.
-
Every host MUST declare wol_capable (true/false) in host_vars. This tracks whether the host can be remotely recovered via Wake-on-LAN after a crash or shutdown.
-
USB ethernet adapters do NOT support WoL. The USB host controller powers down in S5 (standby) and lacks magic packet detection circuitry. Hosts connected exclusively via USB ethernet (e.g., ai) MUST have wol_capable: false.
-
NEVER run operations that could crash a non-WoL host from automation. This includes modprobe -r of the sole GPU driver, shutdown, poweroff, or any operation that could trigger a kernel panic. Non-WoL hosts require physical intervention to recover.
-
scripts/wol.sh MUST NOT include non-WoL hosts. Unit tests in tests/test_wol.py enforce this. The E2E verify playbook also asserts wol_capable is defined for every host and that non-WoL hosts don't appear in wol.sh.
-
Previous bug: ai was listed in wol.sh with its PCIe NIC MAC, but ai is connected via USB ethernet only. The PCIe NIC is not connected to the network, making WoL impossible.