HOW-TO make dual-boot obsolete using kvm VGA passthrough

[juju]

--snip--

I got the software out of personal curiosity, I was interested in how well my machine would perform compared to some of my friends scores and to hardware running nativley on Windows.

Before I switched my setup, I ran the Heaven Benchmark, with recommended settings from that overclock thread you linked. I had 54.4fps overall and a score of 1371 if I remember correctly. I ran that same benchmark also on my brothers native Windows 7 gaming machine he plays all day everyday (no job), his score was like 975 cant remember the fps average like 28 or 38, he has no problems playing games even new ones at a fairly competitive level. I don't think the guy cataloging the benchmarks realized how cool/usable your setup is when you posted your score, their is no slideshow with our setups I am amazed that this works so well, I even forget I am using a VM after playing games for awhile. Here is my User-benchmark: http://www.userbenchmark.com/UserRun/4829650 also I did on that machine. I had issues with Passmark being able to launch, I added the msrs which looks like it will fix it but I have not tried again. I moved my Windows 10 disk image to a non solid state last night before I reformatted so the benchmark speed for disk went down when I tested it again. Ill move it back to my SSD/ or get the Windows 7 setup.

I installed Linux Mint to test it out I also switched my Video card so the 1050 and 970 is now in the machine instead of the Firepro. I also switched my disk drives up I had initially I had my 500gb ssd for the guest and 1tb ssd for the host, this is reversed now. I think my cards always worked fine I may have just been confusing myself, I was expecting to see both NVIDIA cards in the Proprietary Driver dialog. But I just applied everything and rebooted and was all good! Linux mint looks like a great distro, I ran my default setup config and it updated it to my liking. I notice no problems with it in-fact it is better than my previous OS, it came with gufw and other pieces of software installed by default that I typically scramble to add

After I installed Linux mint, I modified my launcher script to point to my backed up windows 10 image file, I copied it over to a non SSD before formatting the disks, and I was surprised to see it just worked even after being moved around (super cool it was a ~450gb file). I ran benchmarks on it after setting up Linux Mint, but the run was worse just with the disk drive being a 3TB non-SSD. I'll get passmark and some more scores later and add them to your other thread, after I move the Windows 7 and 10 back to SSD.

I toiled with Windows 7 Installer for awhile last night but with not much success :0 I kept getting stuck at the Starting Windows dialog, uefi problems most likely. I started making a UEFI compatible usb installer but used a USB3 device and had issues with the speed when passing through. I will try again with USB2 and maybe using the Rufus installer like suggested by another user. I might as well use my Win10 vm to make it >:) I broke down and bought a ASUS pcie sound card, it should arrive tomorrow so I want to try using that instead of the USB device. I was using the USB device on my Linux machine to record audio streams in, so passing it through for games was not a long term solution.

If you have any suggestions on ways to tweak the performance of the VM further let me know, I would be interested in learning about any optimizations I could set on Windows or in qemu that would increase the performance. I did some of the basic things like High-performance power mode, and disabled the visual settings etc. Disable game dvr etc. One thing that is interesting to me is, my brothers user-benchmark (don't have it), showed he had 2% background cpu usage with Windows 7. Windows 10 I had some runs with 14-22% background usage, seems like better scores could be achievable on Windows 7 based on background usage alone. Or does that background rating notice whats happening with the other 4 cores I did not pass-through?

You mentioned you view your Guests disk files from your Host machine, if you have a link or explanation for this might be nice. I did not try mounting my img file maybe its just simple as that.

[HPMint]

4. USB mouse and keyboard: Remove the USB KVM switch and connect your mouse and keyboard directly to the passed through USB ports. How does that work? If that solves the problem, return your KVM switch to the shop for refund.

I had the exact same issue with my USB KVM switch. Over time, the mouse / keyboard would stop working. Connecting the keyboard/mouse directly to the USB solved the issue.

Many USB KVM switches are simply garbage. The best solution is probably Synergy. Another solution are multi-device keyboards and mice. I now use a Logitech K780 multi-device wireless keyboard and a Logitech MX Master wireless mouse (also multi-device), both pretty expensive, but they do their job.

That's a try I've done: keyboard+mouse use 3 USB ports. I've plugged them on the fresco controller and try to passthrough the controller... And I think it worked... (I should have write down all my tests and results). I think it worked, but as soon as I stop windows, mouse and keyboard were no more usable as the controller was no more visible for Linux.
I've just replay these tests:
- USB hub plugged: does not work
- kn&m plugged directly on the fresco card: it works fine... for USB (I don't know how long it will last). But boot was chaotic: Tianno screen for several seconds, then an error message "can't boot on CD/DVD", black screen a few seconds, then windows login screen. Tried winsat command and few hours later, it's not over! And shutdown is not possible, even with system_powerdown in qemu command line!
Next boot was fine. Winsat executes with errors like "ERROR: no adapter for device 0" but it progress.
WEI is 8.1. (and SSD is detected! )

So, it seems to work. Let's say its stable, there's no problem under Windows: how can I have back kb&m under Linux without going under the desk to plug them on another usb?...

I have Synergy... but windows must be my server: I can have some lag on my linux when I use Windows for video, but not while gaming on Windows.

Multi-device kb & mouse are not an option for me: I'm too much in love with mechanical KB

I've tried to activate XHCI hand-off in BIOS: no improvement. If the kb is plugged directly on the usb controller and the mouse on the KVM, no problem, I can play for hours. I'll try to plug a BT USB dongle directly on the card...

Passmark won't run Without "ignore_msrs=1", Windows 10 crash. With it, each time I run the test, I have the error 4. I've tried safemode, debugmode, nothing, passmark won't run

[powerhouse]

@HPMint: I agree - leave your sound setup, it works!

2. If you have configured your network using brctl or editing the /etc/network/interfaces file, there might be remnants of the configuration left that mess up a Network Manager configuration. When using Network Manager to configure the bridge, you should see the following interfaces:

Screenshot at 2017-09-02 13-58-52.png

When the VM is running, I get this Connection Information:

Screenshot at 2017-09-02 14-40-39.png

3. Drive configuration: Good that you found the solution - my drive configuration is very simple.

4. PCI passthrough: Once you pass through a PCI controller to the guest, when you shut down the guest the PCI controller does not automatically release and you need to manually (or via script) make it available to Linux. See the virsh command (can't remember the exact command).
It looks like you need to get rid of your KVM switch and USB hub. I forget to mention in my previous post that my USB hubs also had issues and would disconnect devices or simply not work. Now I'm free of this garbage and everything works.

You got too many errors, but try to restart Windows a few times and see if it goes away - sometimes Windows needs some time to install all the new drivers (or detect that a device is gone). It's called "plug and pray".

I hope very much that connecting you kb/m directly will solve the issues. You can use Synergy - a software solution - to use your kb/m in both Linux and Windows. Else, check out virsh.

Good luck!

(I'm going on a cruise for a week so won't be able to answer much.)

EDIT: For Passmark to work, you need the ignore_msrs... as written in the tutorial!

[powerhouse]

@juju: I noticed you use the standard "pc" (fx440) machine instead of "q35" - see my start script. In my case the "pc" (fx440) option always delivered bad performance. You cannot just change this option but would need to reinstall Windows. However, I have no idea which option is better for Ryzen.

Background CPU: It might be because it is a new installation of Windows and Windows is still indexing files etc. There are some Windows tweaking options that you can do - search the Internet as I don't recall them all. Regarding kvm/qemu I already mentioned everything in my tutorial. Have a look at my latest version here: https://heiko-sieger.info/running-windo ... ssthrough/.

Thanks for the benchmarks. Your GTX 970 bare metal performance is a little better than my GTX 970 VM performance - we are talking 2 fps better. You could run the Unigine Heaven benchmark in your Windows VM and compare the results. That would give you a good indication how well passthrough is working. The expected difference is a few %.

I look forward to see your Passmark results when running on SSD.

Windows 7: Make sure you got a Windows 7 install ISO that uses UEFI, else my how-to won't work.

[HPMint]

@HPMint: I agree - leave your sound setup, it works!

2. If you have configured your network using brctl or editing the /etc/network/interfaces file, there might be remnants of the configuration left that mess up a Network Manager configuration. When using Network Manager to configure the bridge, you should see the following interfaces:

Screenshot at 2017-09-02 13-58-52.png

When the VM is running, I get this Connection Information:

Screenshot at 2017-09-02 14-40-39.png

I followed the tuto and did not use brctl ou edit interfaces file.

4. PCI passthrough: Once you pass through a PCI controller to the guest, when you shut down the guest the PCI controller does not automatically release and you need to manually (or via script) make it available to Linux. See the virsh command (can't remember the exact command).
It looks like you need to get rid of your KVM switch and USB hub. I forget to mention in my previous post that my USB hubs also had issues and would disconnect devices or simply not work. Now I'm free of this garbage and everything works.
You got too many errors, but try to restart Windows a few times and see if it goes away - sometimes Windows needs some time to install all the new drivers (or detect that a device is gone). It's called "plug and pray".

I hope very much that connecting you kb/m directly will solve the issues. You can use Synergy - a software solution - to use your kb/m in both Linux and Windows. Else, check out virsh.

"plug and pray" ... I know this since W95 I've tried Linux (with slackware) in 96... Have an X config was a pain in the ass at this time... Need to fill manually Vertical & Horizontal Frequency of the monitor for each resolution (without error, otherwise, the screen can be broken!)...

Whatever... To reattach the USB controller, he line:

Code: Select all

virsh nodedev-reattach pci_0000_0c_00_0

seems to work fine!
The strange thing is that while the kb is plugged directly on the card, there's no problem with the mouse no matter how connected it is (KVM or hub)...
I think I'll stay with kb&m on the card, and add the hub with the kvm to plugged other devices (video camera) on it. Synergy is already install to use Linux while in Windows

With the installation of libvirt-bin (for virsh), a virbr0 bridge showed.

EDIT: For Passmark to work, you need the ignore_msrs... as written in the tutorial!

Yes, I did it: no more windows crash, but passmark won't start, but with a nice windows stating "Error 4".

[powerhouse]

@HPMint: About Passmark, unless you forgot to use the ignore_msgr option it might be because you use the PC or fx440 machine type.

See also here: https://www.passmark.com/forum/performa ... -qemu-bsod

[powerhouse]

@juju:

You mentioned you view your Guests disk files from your Host machine, if you have a link or explanation for this might be nice. I did not try mounting my img file maybe its just simple as that.

I use LVM formated drives and mount my volumes using kpartx.

If you use image files, for example qcow2, there are commands to mount the files on Linux. Just make sure the guest is not running, or mount it read only! Don't have a link now, but you'll find it when you search for image files or qcow.

[juju]

@juju: I noticed you use the standard "pc" (fx440) machine instead of "q35" - see my start script. In my case the "pc" (fx440) option always delivered bad performance. You cannot just change this option but would need to reinstall Windows. However, I have no idea which option is better for Ryzen.

Background CPU: It might be because it is a new installation of Windows and Windows is still indexing files etc. There are some Windows tweaking options that you can do - search the Internet as I don't recall them all. Regarding kvm/qemu I already mentioned everything in my tutorial. Have a look at my latest version here: https://heiko-sieger.info/running-windo ... ssthrough/.

Thanks for the benchmarks. Your GTX 970 bare metal performance is a little better than my GTX 970 VM performance - we are talking 2 fps better. You could run the Unigine Heaven benchmark in your Windows VM and compare the results. That would give you a good indication how well passthrough is working. The expected difference is a few %.

I look forward to see your Passmark results when running on SSD.

Windows 7: Make sure you got a Windows 7 install ISO that uses UEFI, else my how-to won't work.

The Heaven Benchmark was actually from the VM I built using your tutorial with the GTX 970 passthroughed. I did not get a chance to do one on a bare-metal Windows install that would have been a good comparison. I have not had one of those for a few years now.

I actually remember reading now that you got better performance using q35, I had switched it at some point because I had a Blackscreen issues. But honestly I think I was just not patient enough waiting for a reboot during my testing. I am going to install Windows 7 x64 Ultimate I'll use q35 and get some more benchmarks soonish, including Passmark, I will probably also do the same for Win 10 fresh on q35. I found a guy selling Win 7 ult x64 keys for 11 USD on ebay instant delivery, and I tried activating Win 10 with it and it worked also lol. It might be a couple days before I post something, I have been uninspired to write a program that I finally am finding some energy to do so.

[HPMint]

@HPMint: About Passmark, unless you forgot to use the ignore_msgr option it might be because you use the PC or fx440 machine type.

See also here: https://www.passmark.com/forum/performa ... -qemu-bsod

Nope: like you in the tuto, I use the Q35 machine type.

But I haven't tried to remove C:\ProgramData\PassMark before reinstall (one of thread reply). I'll try in a few days: adding the second graphic card was a difficult time for my case and some cable were not long enough. So no more virtu until new cables are received.

[juju]

@juju: I noticed you use the standard "pc" (fx440) machine instead of "q35" - see my start script. In my case the "pc" (fx440) option always delivered bad performance. You cannot just change this option but would need to reinstall Windows. However, I have no idea which option is better for Ryzen.

Background CPU: It might be because it is a new installation of Windows and Windows is still indexing files etc. There are some Windows tweaking options that you can do - search the Internet as I don't recall them all. Regarding kvm/qemu I already mentioned everything in my tutorial. Have a look at my latest version here: https://heiko-sieger.info/running-windo ... ssthrough/.

Thanks for the benchmarks. Your GTX 970 bare metal performance is a little better than my GTX 970 VM performance - we are talking 2 fps better. You could run the Unigine Heaven benchmark in your Windows VM and compare the results. That would give you a good indication how well passthrough is working. The expected difference is a few %.

I look forward to see your Passmark results when running on SSD.

Windows 7: Make sure you got a Windows 7 install ISO that uses UEFI, else my how-to won't work.

I tried to get Windows 7 Created again... I could not for the life of me create a UEFI bootable USB Drive with my Windows 7 ISO, I spent maybe more than 4 hours so far trying to do so. I downloaded my ISO from the official Microsoft subscription center, then used rufus in Windows and tried too create the thumb drive. Every-time I make the drive I get an error about explorer.exe or it fails to make edits or something to that effect. I tried on 2 different machines same issue. I read viewtopic.php?t=212692&start=260#p1296156 and other guides online but have had no success.

Synopsis of what I was doing, using MSFT Windows 7 Ultimate SP1 ISO, pass through a 8GB USB 2.0 thumb drive to my Windows 10 machine. Run Rufus using the settings settings suggested by bash64. Error near very end of running rufus aaah. I searched around for the error but did not find much, creator of project said to use process hacker and watch what has handles on file and kill those processes. I tried also using Linux to make it a few ways with Unetbootin and some other methods with no success either. I am wondering if my ISO is not "complete" or something I am running out of ideas, Ill run MD5 hash and check later. I tried to do this manually, format the stick as fat32 add bootable parameter with gparted, mount ISO copy contents to the disk that did not work either.

If you have any pointers it would be helpful at this point as long as they are not null I hate Windows so much I don't know where to begin lol. I'll send you a Windows 7 Ultimate 32/64 (Multi-Language) Key over PM for your efforts in creating your guide and providing me and others help when I get home. As far as I can tell the key can be used with either Win 7 or 10, I have used one of my Win 7 X64 Ultimate keys on the Windows 10 installer dialog and it was fine with it, I think it works because they offered the Free upgrades for awhile, therefore the older keys had to be valid as well.


Update: Great! I forgot I put my bootloader on my 1TB, I formatted it the other night to move my vm images around and now I am hosed. Might be a day or two before I am back up and running again. I'll pm that key 7 ASAP
Update 2: Back in action, Going to try and make a UEFI Win 7 installer again later today!

[juju]

Posting this reply from my Windows 7 x64 Ultimate SP1 KVM VGA Passthrough! What a very long and strange trip this has been

http://www.userbenchmark.com/UserRun/4934991

I managed to create a Win 7 VM from a USB Drive that was made with official win7 ult x64 sp1 ISO and rufus USB creator tool. I made this using my Windows 10 x64 passthrough

What worked was during the Windows 7 install my card was passthroughed, but I did not use the video card during the install process at all or even Windows 7 updates. I had vga std and nographic set the entire time until Windows updates put NVIDIA Driver on the machine, bingo!

After this point I could not get back into the VM...kind of expected though cause nothing has worked for like 2 weeks now.

I figured I would try again and install over the last install, this time I was using my Passthroughed card when trying to install instead of the vga std and nographic like last time. So I created a UEFI Bootable Win 7 ISO from the knowledge I learned over the past week, but I could not get the VM to boot to it at all. So I started fiddling with the boot to file options in the BIOS, I must have hit the right file in my Windows 7 Ult x64 Install that I did previously and my machine booted to it to my wild surprise. Since I had installed the NVIDIA Driver previously, I checked dxdiag and my GTX 970 shows up. Linux Mint Host is just fine while my Win 7 guest is running.

I need to take sometime to analyze what the rainbows actually made this work and save it off to the side but I think I have some clues. Windows Update is asking me to reboot lets hope I can get back to this point reliably

Cheers,
Juju

[HPMint]

This is the result of the passmark test:



The result seems low. But I give only 2 core & 2 threads from the 7700. No OC.
The 2D perf of the CG is pretty low too.

[powerhouse]

Just a short reply: I'm travelling a lot this month so won't be able to answer to posts in a timely fashion. I'll be back at the end of the month.

[Arp]

Hello,

I am trying to get this running, and I already fail at the first thing: Virtualization. I have an intel i7-7700 on an Asus Z270 board, both support virtualization. When I installed Linux, the intel VT-d option was disabled. I enabled it and made sure that Intel Virtualization is also enabled in the bios.

But still, dmesg will not show me "DMAR: Intel(R) Virtualization Technology for Directed I/O".

Do I need to reinstall because it was disabled while I was installing it? Or am I overlooking something?

Thx.

[HPMint]

But still, dmesg will not show me "DMAR: Intel(R) Virtualization Technology for Directed I/O".

Hello

Did you follow the 4th step of the first part?
You need to edit grub configuration and reboot.

[powerhouse]

But still, dmesg will not show me "DMAR: Intel(R) Virtualization Technology for Directed I/O".

Hello

Did you follow the 4th step of the first part?
You need to edit grub configuration and reboot.

After editing the grub file, make sure to run

Code: Select all

sudo update-grub

[powerhouse]

This is the result of the passmark test:



The result seems low. But I give only 2 core & 2 threads from the 7700. No OC.
The 2D perf of the CG is pretty low too.

That doesn't look too bad, particularly when you consider the no. of cores/threads you use for the VM.

My 2D performance is also low, but I can't see any problem with that.

[powerhouse]

Posting this reply from my Windows 7 x64 Ultimate SP1 KVM VGA Passthrough! What a very long and strange trip this has been

http://www.userbenchmark.com/UserRun/4934991

I managed to create a Win 7 VM from a USB Drive that was made with official win7 ult x64 sp1 ISO and rufus USB creator tool. I made this using my Windows 10 x64 passthrough

What worked was during the Windows 7 install my card was passthroughed, but I did not use the video card during the install process at all or even Windows 7 updates. I had vga std and nographic set the entire time until Windows updates put NVIDIA Driver on the machine, bingo!

After this point I could not get back into the VM...kind of expected though cause nothing has worked for like 2 weeks now.

I figured I would try again and install over the last install, this time I was using my Passthroughed card when trying to install instead of the vga std and nographic like last time. So I created a UEFI Bootable Win 7 ISO from the knowledge I learned over the past week, but I could not get the VM to boot to it at all. So I started fiddling with the boot to file options in the BIOS, I must have hit the right file in my Windows 7 Ult x64 Install that I did previously and my machine booted to it to my wild surprise. Since I had installed the NVIDIA Driver previously, I checked dxdiag and my GTX 970 shows up. Linux Mint Host is just fine while my Win 7 guest is running.

I need to take sometime to analyze what the rainbows actually made this work and save it off to the side but I think I have some clues. Windows Update is asking me to reboot lets hope I can get back to this point reliably

Cheers,
Juju

Congratulations! You actually made it. For the record, I gave up on Windows 7 and UEFI boot long ago.

[powerhouse]

About mounting qcow images under Linux, see http://ask.xmodulo.com/mount-qcow2-disk ... linux.html.

Make sure you DON'T run the VM at the same time!!!

[juju]

About mounting qcow images under Linux, see http://ask.xmodulo.com/mount-qcow2-disk ... linux.html.

Make sure you DON'T run the VM at the same time!!!

Roger! I made a mount script that has a check like in your VM Launcher script to see if it is not running then it mounts the disk drive. Mounting the drive works great I copied an ISO to my host that was downloaded to a Guest machine.

The short and sweet way to get Windows 7 working "easily" and this worked with both SP1 and non SP1 and does not require any uefi rufus media creation steps at all - I confirmed yesterday this works consistently and also found a workaround to have windows 7 boot without going into the bios and doing maintenance boot to file.

1) Make a VM with your Video Card passed through but use Spice and Cirrus instead of using the physical card for video during setup, mount the virtio driver disk from redhat, use whatever WIndows 7 ISO you have - q35 with ovmf etc same as the Win 10 setup
(I found it easy to use virt-manager to do the above, even though I don't use it ultimately in the end and never used it before, it default added the Spice and I switched it from qxl to cirrus - its pretty easy to pass through the video card and usb devices you need to setup the machine in virt-manager dialogs)

2) Install Windows 7 like normal to the Virtual disk, you will choke at "Starting Windows" if using qxl video mode is set - so make sure to use Cirrus (I could be mixing the two but I am almost certain you want to use cirrus), Update Windows 7 reboot a few times, Install your Nvidia Video Card Driver

3) Modify your Windows 10 qemu launcher script to point to the Windows 7 disk instead (Stop using Cirrus and just use your video card like you normally would), I literately just copy pasted your Windows 10 launcher renamed vm and pointed to new disk.

When I stop using virt-manager, and toss the qcow2 image I made into a copied launcher shell script identical to your windows 10 qemu launcher - when I run it it will not boot to the OS it gets stuck in the bios, this is fine.

4) How you get into the OS is you go into the BIOS and into the Boot maintenance manager, choose boot to file, choose the disk device that has Windows 7 installed on it (Not the network device), navigate to: C:\Windows\Boot\EFI\bootmgfw.efi - use this to boot. Now your Windows 7 Will start to boot, except the first time you do this when switching from using the Spice/cirrus to the real card - it will just sit at a black screen after it switches from the Starting Windows logo this is ok, I let this sit for like 2-3 minutes still black screen. I figure its actually working at this point but some bits are not set quite right at this point but on next boot it will be Ok. Close the VM down from the host ctrl+c - On the next boot of the VM, without changing anything. (Go into bios, boot to the efi like before). Now everything works perfectly, your NVIDIA GTX 970 or whatever card you use will be in use and its driver is in use. Windows 7 is working at this point! yay!

5) Getting around going into BIOS and doing maintenance boot to file every-time you want to Boot Windows 7

Here is where I did a bit of "hackery" haha - I learned enough about what the rufus tool does failing to get this working for so long. It was super simple, it copies the files from the ISO to a disk, makes a file called bootx64.efi copies it into a file location and makes the disk bootable - not that complicated.

So I decided to go ahead and copy the bootmgfw.efi file and paste it into the base: C:\Windows Directory similar to what rufus is doing. I copied it again in this location and renamed it bootx64.efi. So I have 2 efi files in the root windows location now, I think it being named bootx64.efi and being in the root location is enough for the BIOS to pick up on it automatically. I think you only need the bootx64 efi but I have both in the root WIndows 7 directory and it works great just boots right to disk every-time now,

Cheers,
Juju