GPU Passthrough in Proxmox 7.2!
This is less of a blog post and more of a personal note, which I hope could help someone else out there as well. Keeping it shot and to-the-point in the hopes that this is easily searchable.
If you find any issues with the following, feel free to raise PR against this post.
Some of the specifications
Proxmox version: 7.2-11 Kernel version: 5.15.19-2-pve GPU: nvidia T600 4GB (PNY VCNT600-SB)
Now, let’s get started.
1. Passthrough modules to be loaded #
This will ensure that these modules are loaded, as these will help with the passthrough
/etc/modules, have these:
vfio vfio_iommu_type1 vfio_pci vfio_virqfd
2. Blacklist the existing drivers #
This will blacklist these drivers from being loaded.
blacklist nouveau blacklist nvidia blacklist nvidiafb
3. Allow unsafe interrupts (needed sometimes) #
options vfio_iommu_type1 allow_unsafe_interrupts=1
4. Add you GPU to the VFIO module config #
4.1 Now find IDs for your GPU #
- Find the main ID using
09:00.0 VGA compatible controller: NVIDIA Corporation Device 1fb1 (rev a1) ^^^^^^^ This is the Main GPU Subsystem: NVIDIA Corporation Device 1488 Kernel driver in use: vfio-pci Kernel modules: nvidiafb, nouveau 09:00.1 Audio device: NVIDIA Corporation Device 10fa (rev a1) ^^^^^^^ This is HDMI Audio Subsystem: NVIDIA Corporation Device 1488 Kernel driver in use: vfio-pci Kernel modules: snd_hda_intel
- Then list device-id and vendor-id using
09:00.0 0300: 10de:1fb1 (rev a1) ^^^^^^^^^ Subsystem: 10de:1488 Kernel driver in use: vfio-pci Kernel modules: nvidiafb, nouveau 09:00.1 0403: 10de:10fa (rev a1) ^^^^^^^^^ Subsystem: 10de:1488 Kernel driver in use: vfio-pci Kernel modules: snd_hda_intel
4.2 Add the device ID and vendor ID within vfio module’s config #
/etc/modprobe.d/vfio.conf, put these IDs
options vfio-pci ids=10de:1fb1,10de:10fa disable_vga=1
6. Set some more magic flags #
options kvm ignore_msrs=1
7. Disable GPU framebuffers during boot #
/etc/default/grub, ensure that we have
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on nomodeset video=vesafb:off video=efifb:off video=simplefb:off"
Update GRUB after updating the file above
update-grub && update-initramfs -u
8. Reboot #
shutdown -r 0
8. Normally attach the GPU to your choice of VM #
- Open Proxmox web UI
- Open “Hardware” configuration of your VM where you want to attach the GPU
- Click on “Add” on the top left, and select the GPU you want to pass through (the same one as the one in Step 4)
- Set it as Primary GPU (if that’s going to be the primary GPU) and save it.
- Boot the VM and install drivers (if needed by the guest OS)
I’ve had help from various sources in figuring this out and this happens to break during upgrades, but so far this particular configuration is working for me for the given version of proxmox with specified kernel and the specified GPU.