HOW-TO make dual-boot obsolete using kvm VGA passthrough

Questions about virtualization software
Forum rules
Before you post please read how to get help
powerhouse
Level 6
Level 6
Posts: 1086
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by powerhouse » Fri Apr 19, 2019 7:06 pm

Hello Tech-Junk1e:

Your 09:0.2 device is using the wrong driver, it should also be bound to vfio-pci.

Perhaps you didn't list it as an alias, see https://heiko-sieger.info/running-windo ... assthrough

Make sure you have 4 alias entries AND the PCI device identifiers listed in this file, for example:

options vfio-pci ids=10de:13c2,10de:0fbb

You should have 4 comma separated entries after the ids=...

Please also note that my tutorial is written for Windows 10, not Windows 7 !!!

Windows 7 doesn't support UEFI boot out of the box. You would have to create an updated version with UEFI boot.

The alternative is to use regular Seabios (legacy) boot, without the ovmf part. This legacy boot might work but can have disadvantages, which is why I don't describe it. See my links, in particular the link to the archlinux forum, for more on the Seabios legacy boot.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3, Noctua NH-D14 CPU cooler, GTX 970 + Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk 120GB + Samsung EVO 860 1TB SSD + 4 HDD, Corsair 500R, SeaSonic 660W Gold X PS https://heiko-sieger.info

Tech-Junk1e
Level 1
Level 1
Posts: 2
Joined: Fri Apr 19, 2019 1:04 pm

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by Tech-Junk1e » Fri Apr 19, 2019 9:19 pm

Unfortunately i have created all of the alias entries correctly, no matter what I do I cannot stop that particular driver from taking hold. at one point on a previous install I even tried blacklisting it (though not recently).

Code: Select all

root@TJs-WorkHorse:~# cat /sys/bus/pci/devices/0000:09:00.0/modalias
pci:v000010DEd00002182sv00003842sd00001263bc03sc00i00
root@TJs-WorkHorse:~# cat /sys/bus/pci/devices/0000:09:00.1/modalias
pci:v000010DEd00001AEBsv00003842sd00001263bc04sc03i00
root@TJs-WorkHorse:~# cat /sys/bus/pci/devices/0000:09:00.2/modalias
pci:v000010DEd00001AECsv00003842sd00001263bc0Csc03i30
root@TJs-WorkHorse:~# cat /sys/bus/pci/devices/0000:09:00.3/modalias
pci:v000010DEd00001AEDsv00003842sd00001263bc0Csc80i00

Code: Select all

alias pci:v000010DEd00002182sv00003842sd00001263bc03sc00i00 vfio-pci
alias pci:v000010DEd00001AEBsv00003842sd00001263bc04sc03i00 vfio-pci
alias pci:v000010DEd00001AECsv00003842sd00001263bc0Csc03i30 vfio-pci
alias pci:v000010DEd00001AEDsv00003842sd00001263bc0Csc80i00 vfio-pci
options vfio-pci ids=10de:2182,10de:1aeb,10de:1aec,10de:1aed
options vfio-pci disable_vga=1
I had forgotten about windows 7 not being UEFI compatible and will look into the alternative method you mentioned.

Also another question of inexperience, can I use my ISO in virtualbox and use qemu-utils to convert the .vdi file to a .img or would UEFI still get in the way?

EDIT: I have just tried to pci-stub the usb controller via both /etc/initramfs-tools/modules and etc/default/grub with no success. other users are encountering the same issue.

Code: Select all

root@TJs-WorkHorse:~# dmesg | grep pci-stub
[    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-4.15.0-47-generic root=/dev/mapper/mint--vg-root ro quiet splash pci-stub.ids=10de:1aec amd_iommu=on iommu=pt vt.handoff=1
[    0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-4.15.0-47-generic root=/dev/mapper/mint--vg-root ro quiet splash pci-stub.ids=10de:1aec amd_iommu=on iommu=pt vt.handoff=1
[    3.436830] pci-stub: add 10DE:1AEC sub=FFFFFFFF:FFFFFFFF cls=00000000/00000000
root@TJs-WorkHorse:~# lspci -kn | grep -A 2 09:00
09:00.0 0300: 10de:2182 (rev a1)
	Subsystem: 3842:1263
	Kernel driver in use: vfio-pci
--
09:00.1 0403: 10de:1aeb (rev a1)
	Subsystem: 3842:1263
	Kernel driver in use: vfio-pci
--
09:00.2 0c03: 10de:1aec (rev a1)
	Subsystem: 3842:1263
	Kernel driver in use: xhci_hcd
09:00.3 0c80: 10de:1aed (rev a1)
	Subsystem: 3842:1263
	Kernel driver in use: vfio-pci
here https://askubuntu.com/questions/719390/ ... assthrough
and here https://www.reddit.com/r/unRAID/comment ... his_erroe/

and I am still too green to understand the solution.

I also now have a UEFI enabled windows 7 ISO and a Win 10 ISO

powerhouse
Level 6
Level 6
Posts: 1086
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by powerhouse » Sat Apr 20, 2019 3:57 pm

Hello Tech-Junk1e,

My instructions work with UEFI.

It seems that the USB driver is very persistent. Have a look at the other options to bind the vfio-pci driver. One way is via grub file. Make sure whenever you make any changes in the /etc/initramfs you also run update-initramfs, similar to when you change anything in /etc/default/grub to run update-grub.

I'm quite sure that you did all that and I'm afraid I don't have a better answer, sorry.

Search for your graphics card and vfio-pci and see if anyone else solved the issue. In the worst case, sign up on the vfio list and post your dilemma. I hope I didn't miss a link in my references.

If you find a solution, please post it here! I will also try to keep an eye open but am very busy now.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3, Noctua NH-D14 CPU cooler, GTX 970 + Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk 120GB + Samsung EVO 860 1TB SSD + 4 HDD, Corsair 500R, SeaSonic 660W Gold X PS https://heiko-sieger.info

powerhouse
Level 6
Level 6
Posts: 1086
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by powerhouse » Sat Jun 08, 2019 6:29 am

powerhouse wrote:
Sat Apr 20, 2019 3:57 pm
Hello Tech-Junk1e,

My instructions work with UEFI.

It seems that the USB driver is very persistent. Have a look at the other options to bind the vfio-pci driver. One way is via grub file. Make sure whenever you make any changes in the /etc/initramfs you also run update-initramfs, similar to when you change anything in /etc/default/grub to run update-grub.

I'm quite sure that you did all that and I'm afraid I don't have a better answer, sorry.

Search for your graphics card and vfio-pci and see if anyone else solved the issue. In the worst case, sign up on the vfio list and post your dilemma. I hope I didn't miss a link in my references.

If you find a solution, please post it here! I will also try to keep an eye open but am very busy now.
I have updated my tutorial on my website and added a section on using the "driver override" feature of Linux that should solve the issue - see Using the “driver_override” feature in conjunction with the initial RAM disk (initramfs).
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3, Noctua NH-D14 CPU cooler, GTX 970 + Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk 120GB + Samsung EVO 860 1TB SSD + 4 HDD, Corsair 500R, SeaSonic 660W Gold X PS https://heiko-sieger.info

bash64
Level 3
Level 3
Posts: 108
Joined: Mon Mar 23, 2015 12:19 am

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by bash64 » Tue Jul 02, 2019 10:11 pm

Powerhouse,

I want to install the i915 vga arbiter patch.
I am running linux mint 19.1.
What is the quickest way to get a kernel installed that has this?
I am willing to compile, but so far internet instructions on doing this fail at the patch stage.
Thanks

powerhouse
Level 6
Level 6
Posts: 1086
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by powerhouse » Fri Jul 05, 2019 12:30 pm

bash64 wrote:
Tue Jul 02, 2019 10:11 pm
Powerhouse,

I want to install the i915 vga arbiter patch.
I am running linux mint 19.1.
What is the quickest way to get a kernel installed that has this?
I am willing to compile, but so far internet instructions on doing this fail at the patch stage.
Thanks
The question should be: Do you have to install the VGA arbiter patch?

This is from my tutorial:
If you plan to use the Intel IGD (integrated graphics device) for your Linux host, UEFI boot is the way to go. UEFI overcomes the VGA arbitration problem associated with the IGD and the use of the legacy Seabios.
If, for some reason, you cannot boot the VM using UEFI, and you want to use the Intel IDG for the host, you need to compile the i915 VGA arbiter patch into the kernel. Before you do, check the note below. For more on VGA arbitration, see here. For the i915 VGA arbiter patch, look here or under Part 15 – References.

Note: If your GPU does NOT support UEFI, there is still hope. You might be able to find a UEFI BIOS for your card at TechPowerUp Video BIOS Collection. A Youtube blogger calling himself Spaceinvader has produced a very helpful video on using a VBIOS.
See also here: https://heiko-sieger.info/running-windo ... ration_bug

If your "pass-through" graphics card supports UEFI, you don't need the patch. Please let me know if you use UEFI boot, as described in my tutorial.

Patching the kernel should be your last resource to solve the problem. Trust me, it's a pain in the neck.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3, Noctua NH-D14 CPU cooler, GTX 970 + Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk 120GB + Samsung EVO 860 1TB SSD + 4 HDD, Corsair 500R, SeaSonic 660W Gold X PS https://heiko-sieger.info

bash64
Level 3
Level 3
Posts: 108
Joined: Mon Mar 23, 2015 12:19 am

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by bash64 » Sun Aug 04, 2019 8:33 am

Hey Powerhouse.

I have built a new PC, full amd threadripper 2nd gen.
My video card is the Radeon Duo Pro R9 Fury X nano series.
It has two gpus.

Is it possible to hold a gpu for vfio in a card with two gpus?
So far no success.

Thanks

bash64
Level 3
Level 3
Posts: 108
Joined: Mon Mar 23, 2015 12:19 am

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by bash64 » Sun Aug 04, 2019 10:17 am

p.s.

I solved my need for vga arbitration.
It was as simple as switching from uefi mode to uefi with csm in my bios.
The hardware issue I had went away.
Basically, my vga 15 pin port was disabled when using nomodeset on boot.
No i915 driver...no vga port signal.
Switching to uefi with csm kept the vga port alive regardless and I was able to use my 23" led monitor with the vga port for linux.
I always thought it was a software issue.

powerhouse
Level 6
Level 6
Posts: 1086
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by powerhouse » Wed Aug 14, 2019 5:20 pm

bash64 wrote:
Sun Aug 04, 2019 10:17 am
p.s.

I solved my need for vga arbitration.
It was as simple as switching from uefi mode to uefi with csm in my bios.
The hardware issue I had went away.
Basically, my vga 15 pin port was disabled when using nomodeset on boot.
No i915 driver...no vga port signal.
Switching to uefi with csm kept the vga port alive regardless and I was able to use my 23" led monitor with the vga port for linux.
I always thought it was a software issue.
Sorry I didn't respond earlier - I was actually having a vacation :-)

I'm glad you managed to fix the issue. However, for the benefit of me and others, could you describe the problem that you fixed with it?

I understand that it has to do with the Radeon Duo Pro R9 Fury X nano, but I'm not exactly sure of what issue you ran into.

Thanks in advance!
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3, Noctua NH-D14 CPU cooler, GTX 970 + Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk 120GB + Samsung EVO 860 1TB SSD + 4 HDD, Corsair 500R, SeaSonic 660W Gold X PS https://heiko-sieger.info

bash64
Level 3
Level 3
Posts: 108
Joined: Mon Mar 23, 2015 12:19 am

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by bash64 » Wed Aug 28, 2019 8:39 pm

Sorry, I had two different posts for two different PCs.
The one was for the msi gt70 laptop I have used for 5 years.
Basically, the vga 15 pin out was disabled when doing qemu passthrough.
I was using the laptop screen for linux and the display port for qemu.
I wanted to have the laptop screen appear on my 23" monitor hooked up to the vga 15 pin port, but that port was strangely disabled.
It turned out that it was a bios issue.
In uefi mode the vga port is disabled.
When I switched to uefi+csm it worked. The laptop screen and the vga port mirrored each other.


As to the new PC with threadripper 2 cpu.
I managed to share an amd radeon with dual gpus out to qemu.
Both gpus have the same vid:pid numbers.
I simply listed them twice as vfio options as such:
options vfio-pci ids=1002:7300,1002:aae8,1002:7300

Both gpus and the 4k monitor audio both went over to windows just fine and gaming works.
Also discovered that windows 7 does NOT have a threadripper driver and will not.
Had to switch to using windows 10.

Thanks for responding.

I built this new desktop to get away from the issues of my aging laptop.
It was great figuring out how to make it work on the laptop despite it having nvidia optimus, but time to move on.
I now have a desktop with two video cards and qemu couldn't be any easier.
Running both OSs at the same time with out the limitations of the laptop.

I moved on to qemu 4.0 also and no issues. Seems backwards compatible with my exisiting script.

Question: Does qemu really require a bridge for tap0 to work?
When I have a bridge defined I cannot have wireless. If I turn it on it causes issues.

I'd really love if tap0 were independant and could get an ip on its own.

powerhouse
Level 6
Level 6
Posts: 1086
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by powerhouse » Thu Sep 19, 2019 2:13 pm

@bash64: Thanks for responding and sharing your experience - this is what makes the forum so valuable. (I knew I had seen you on the forum long ago and was a little confused.)

You do not need to define a bridge, and if you skip the network definition in the script, it should automatically configure a layer 3 (IP) network that works with WiFi.

The reason I advocate a bridge is performance and simplicity. But there are plenty of use cases where bridging doesn't work. If your PC is connected to the router via wireless, then bridging is a no go (well, there are ways to get bridging working over WiFi, but it's a pain to set up and I didn't succeed).

In truth, I haven't tested IP networking (layer 3) in a while and performance may well be good. On my bridged network, I get the following results when copying a 10GiB file:
HD (host, accessed via Samba) to SSD (on Windows 10 VM) - 46 sec, or 217 MByte/sec
SSD (Windows 10 VM on Samsung EVO 860) to SSD (host, via Samba, a Sandisk 120GB disk) - 20 sec, or 500 MByte/sec

By the way, creating the file on a HD using dd if=/dev/zero of=file.txt count=1024 bs=10485760 takes the same time (20 sec) as copying the file between SSD.

Copying the same 10GiB file from my Linux host to my server over Gigabit Ethernet takes 1:41 minutes, or 100 MByte/sec.

Perhaps you can use the above benchmarks for comparison.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3, Noctua NH-D14 CPU cooler, GTX 970 + Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk 120GB + Samsung EVO 860 1TB SSD + 4 HDD, Corsair 500R, SeaSonic 660W Gold X PS https://heiko-sieger.info

Post Reply

Return to “Virtualization”