Windows – GPU passthrough works with UEFI firmware but not Windows iso

gpu-passthroughkvmqemuuefiwindows

I am trying to set up a virtual machine with passthrough graphics. I am able to get the passthrough working for the UEFI shell, but not the official Windows installer. I can get the Windows installer to work, but only with emulated graphics

This boots the windows installer in a QEMU Window:

sudo qemu-system-x86_64 --enable-kvm \
-name TESTVM,process=TESTVM \
-cdrom /media/big-tank-8TB/OSISOS/Windows/WindowsOct2018.iso \
-smp 4 \
-cpu core2duo \
-m 4096 \
-vga qxl

This also boots the windows installer in a QEMU window (still no passthrough)

sudo qemu-system-x86_64 --enable-kvm \
-name TESTVM,process=TESTVM \
-cdrom /media/big-tank-8TB/OSISOS/Windows/WindowsOct2018.iso \
-smp 4 \
-cpu core2duo \
-m 4096 \
-device vfio-pci,host=43:00.0,multifunction=on \
-device vfio-pci,host=43:00.1 

But if I specify the paths to UEFI firmware, I get the Tiano slpash screen and then the UEFI shell both on the monitor attached to my passed-through video card and in a QEMU window.

sudo qemu-system-x86_64 --enable-kvm \
-name TESTVM,process=TESTVM \
-cdrom /media/big-tank-8TB/OSISOS/Windows/WindowsOct2018.iso \
-smp 4 \
-cpu core2duo \
-m 4096 \
-device vfio-pci,host=43:00.0,multifunction=on \
-device vfio-pci,host=43:00.1 \
-drive if=pflash,format=raw,readonly,file=/usr/share/OVMF/OVMF_CODE.fd \
-drive if=pflash,format=raw,file=/usr/share/OVMF/OVMF_VARS.fd 

Why is the passthrough working only in the absence of the UEFI files? Or, why is specifying the UEFI files preventing me from starting Windows?

Edit: Tried downloading a different version of Windows (April 2018 instead of the October one), same problem.

Edit: Tried purging and reinstalling OVMF, but no luck.

Edit: I can get to the boot manager by typing "exit" in the shell, but selecting the available DVD drive (and all other options) immediately falls back to the boot manager.

Edit: Ran this:

-name TESTVM,process=TESTVM \
-drive file=/media/big-tank-8TB/OSISOS/Windows/Win10_1803_English_x64.iso,index=1,media=cdrom  \
-drive file=/media/big-tank-8TB/OSISOS/Windows/virtio-win-0.1.160.iso,index=2,media=cdrom \
-smp 4 \
-cpu core2duo \
-m 4096 \
-device vfio-pci,host=43:00.0,multifunction=on \
-device vfio-pci,host=43:00.1 \
-drive if=pflash,format=raw,readonly,file=/usr/share/OVMF/OVMF_CODE.fd \
-drive if=pflash,format=raw,file=/usr/share/OVMF/OVMF_VARS.fd 

When I got too the uefi shell, I typed "exit" to get to the boot manager. In the boot manager, selecting the available DVD drive instantly fell back to the boot manager.

I then added another DVD drive by Boot Maintenance Manager > Boot Options > Add boot option, and then selected that in the Boot Manager menu.
the boot entry I picked.

This gave me a very brief "press any key to boot from CD". If I am fast enough, this boots into the Windows installer BUT ONLY IN THE QEMU WINDOW. The screen attached to the passed-through card was black with a simple cursor, as opposed to mirroring as with the UEFI/Boot Manager.

Edit: I am trying to pass through an NVIDIA GTX1070. Mobo is ASRock x399 Taichi, CPU is Threadripper 1950X. OS is Ubuntu Server with XFCE installed.

Edit: If I proceed with the installation, I still have Windows in the QEMU window and just a TianoCore splash screen on the passthrough card. If I go to the device manager, Windows sees the card, but it is stopped for some reason.
Error 43

Edit: I tried using these instructions to get rid of the code 43, to no avail. In order to try this, I used virt-install instead of qemu-system, and when doing this there is no TianoCore splash screen. But still code 43 when I get into Windows.

Edit: used dmesg to check for memory reservation errors as described here. Found none.
Edit: Also from the above link, used ROM parser and confirmed the presence of a "type 3 (EFI)"

Best Answer

You are on the correct track already. GPU Passthrough is not perfect, especially if it's an NVidia Card (Which you don't mention NVidia or AMD). Finish the setup on the Qemu Window. Make sure the Windows Machine is connected to Internet and let Windows Update install the graphics drivers for you. When you come back you should be greeted by a second monitor, if not, reboot. I usually then remove the spice/vnc console and only have the GPU monitor attached. Getting GPU Passthrough to work is all about trial and error.

Other things to try:

  • Install Windows without GPU passthrough, then attempt to passthrough GPU.
  • Install drivers via NVidia_drivers.exe
  • Install drivers via Windows Update
  • Bios vs UEFI
  • Q35 vs i440fx

Note: Code 43 is a known error w/ NVidia relating to NVidia drivers checking if they are running in a VM. NVidia sells cards specifically for running in a VM environment and attempts to block installation of drivers for consumer grade cards in a VM. You need to make sure to use the following in your domain.xml

<kvm>
 <hidden state='on'/>
</kvm>

See https://passthroughpo.st/apply-error-43-workaround/ and other resources for examples.

Here is a screenshot of my config:

QEMU GPU Passtrough Settings

Here is the "relevant" part of my domain.xml, I can share entire thing if you want, but it's got a bunch of unnecessary things.

  <os>
    <type arch='x86_64' machine='pc-i440fx-2.10'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/edk2/ovmf/OVMF_CODE.fd</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/Windows10_VARS.fd</nvram>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <kvm>
      <hidden state='on'/>
    </kvm>
    <vmport state='off'/>
  </features>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
Related Question