Broke system 3 times trying setup virt-manager for gaming and now gpu output is broken too...

I have over a decade of professional, on the job linux experience, and getting a vfio setup on the go took me about 40 hours of work & experimentation. It's not easy, that's for sure.

In summary, these were the major hurdles:

  • having multiple gpus which were different models
  • BIOS updates from the manufacturer because I couldn't make things work without some fixes from the manufacturer
  • building an OVMF firmware myself because the ubuntu provided one is garbage & virt-manager can't even make it work
  • Using Q35 for the emulated hardware and not x440i
  • having a 3 week stretch to get everything going where I'm not dependant on having my computer running, because it took 3 weeks to figure it out.
  • Proxmox got things working the easiest and they have some good guides (but performed poorly on my threadripper but shouldn't be a problem for ryzen systems)

Here's detail on hurdles and the list is long and doesn't list all of the problems I had to solve:

  • Updated bios. Asus had some serious bugs in the bios for my threadripper mobo when I first bought it and had to drop the project for over a year, waiting for them to fix the bios. A bios update in december 2019 made 90% of my problems go away.
  • A separate video card for the host and the guest VM which are two different models, and both supporting UEFI. I'm using a geforce 750 ti for the host and 1070ti for the guest.
  • I needed to have a separate computer available to use while I'm troubleshooting my main computer. Using my main computer was stressful because I was without a computer while sorting out issues.
  • My host gpu has to go in the first pci-e slot and the 2nd gpu goes in the other. The boot process grabs the first gpu and locks it for the host. Trying to pass through the host GPU is so complex I gave up.
  • My first successful attempt was using Ubuntu Server for the host. Cutting out the desktop environment meant I didn't have to worry about x grabbing any gpus and simplified the setup a bit. I had to use virt-install instead of virt-manager to set up the vm then editing the XML on the virsh console for further tweaks.
  • A working OVMF firmware was a nightmare to find. virt-manager on ubuntu doesn't actually use the OVMF firmware when you pick it in the UI and I had to resort to building the firmware myself & using it with nvram directives in the boot section of the vm config. Look at my post history for a docker-based build project which builds the ovmf firmware for your own use. If you can get docker going then run the makefile you'll get ovmf files that are rock solid. Google for "virt-install ovmf" for the params you need when creating the vm. If you use virt-manager you have to manually make it use the firware in the boot section.
/r/VFIO Thread Parent