GPU host prep

Experimental: GPU driver, runtime, and workload compatibility must be validated on the target hardware.

Fleet bootstrap does not install GPU drivers. Prepare every GPU node before FleetClusterJoin bootstrap, then enable the container runtime and device plugin after the cluster is ready.

Required sequence

1. Install a vendor-supported NVIDIA driver compatible with the running kernel.
2. Reboot and verify that the host driver is functional.
3. Bootstrap the cluster.
4. Run the GPU-cluster enable flow to install the NVIDIA container toolkit, configure the K3s runtime, and deploy the device plugin.
5. Run capability discovery and verify that Kubernetes reports GPU capacity.

The repository provides playbooks/flows/gpu-host-prep.yml for host preparation and playbooks/flows/gpu-cluster-enable.yml for post-ready runtime setup. That enable playbook, and cluster-gpu-enable.yml, import fleet_spoke_firewall with fleet_spoke_firewall_enabled: true and configure_firewalld: true. The role does not run unless fleet_spoke_firewall_enabled is true (it defaults to configure_firewalld, which defaults to false). It does not disable firewalld. When enabled it trusts wg-umkim and allows the hub overlay to reach K3s API TCP 6443 and 10250 and hostNetwork LLM inference TCP 8080. Review inventory inputs and Secure Boot requirements before use.

The deployed GPU inference chart is llama.cpp, using the CUDA llama-server image. Default llama-cpp-helm values use an NVIDIA runtime class and hostNetwork: true. Review host networking, resource requests, model cache storage, and node isolation before deploying inference.

Capability discovery uses the device plugin’s reported GPU capacity. A host driver alone is not evidence that the scheduler can place GPU workloads.

See fleet-llm and verification.