HOW-TO make dual-boot obsolete using kvm VGA passthrough

Questions about virtualization software
Forum rules
Before you post read how to get help. Topics in this forum are automatically closed 6 months after creation.
Locked
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by powerhouse »

talos_d wrote:
powerhouse wrote:@talos_d: So you got Spice working. Have you also installed the Windows binaries? See https://www.spice-space.org/download.html.
Tried it,same result-there is graphic acceleration through the gpu but I am getting 1740 score in unigine heaven which is 40 percent slower than the regular passthrough,
Maybe I didnt set spice correctly,some of the set up for spice includes the lines

Code: Select all

-device virtio-serial -chardev spicevmc,id=vdagent,debug=0,name=vdagent \
 device virtserialport,chardev=vdagent,name=com.redhat.spice.0 \
but when I add them to the script,it causes the qxl window to be unusable,maybe they are meant for installation only.
You have seen the documentation here: https://www.spice-space.org/spice-user-manual.html.

According to that documentation the best option is -vga qxl perhaps with the options above, and installing the spice tools in Windows including the qxd driver. You may have to consider the way you invoke qemu, perhaps some options are only available after installing the drivers in Windows.

Another thing are the kernel modules mentioned here: https://wiki.archlinux.org/index.php/QEMU#qxl. You may want to check this out.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
garrettjk

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by garrettjk »

I currently get no display on my passthrough monitor when trying to run sudo ./windows10vm.sh
its contents are as follows:

#!/bin/bash

vmname="windows10vm"

if ps -A | grep -q $vmname; then
echo "$vmname is already running." &
exit 1

else

# use pulseaudio
export QEMU_AUDIO_DRV=pa
export QEMU_PA_SAMPLES=8192
export QEMU_AUDIO_TIMER_PERIOD=99
export QEMU_PA_SERVER=/run/user/1000/pulse/native

cp /usr/share/OVMF/OVMF_VARS.fd /tmp/my_vars.fd

qemu-system-x86_64 \
-name $vmname,process=$vmname \
-machine type=pc,accel=kvm \
-cpu host \
-smp 6,sockets=1,cores=3,threads=2\
-enable-kvm \
-m 2300 \
-mem-path /run/hugepages/kvm \
-mem-prealloc \
-balloon none \
-rtc clock=host,base=localtime \
-vga none \
-serial none \
-parallel none \
-soundhw hda \
-usb -usbdevice host04d9:fa52 -usbdevice host:04b4:5004 \
-device vfio-pci,host=07:00.0,multifunction=on \
-device vfio-pci,host=07:00.1 \
-drive if=pflash,format=raw,readonly,file=/usr/share/OVMF/OVMF_CODE.fd \
-drive if=pflash,format=raw,file=/tmp/my_vars.fd \
-boot order=dc \
-device virtio-scsi-pci,id=scsi \
-drive id=disk0,if=virtio,cache=none,format=raw,file=/media/libraries/win.img \
-drive file=/media/libraries/win10.iso,id=isocd,format=raw,if=none -device scsi-cd,drive=isocd \
-drive file=/media/libraries/virtio-win-0.1.126.iso,id=virtiocd,format=raw,if=none -device ide-cd,bus=ide.1,drive=virtiocd\
-netdev type=tap,id=net0,ifname=tap0,vhost=on \
-device virtio-net-pci,netdev=net0,mac=00:16:3e:00:01:01

exit 0
fi


My specs are:
Cpu - i5-6600k (IGPU for linux)
GPU - MSI R9 390 (passthrough)
Motherboard - Gigabyte GA-Z170-HD3P
Ram - 8G ddr4-2133


kvm-ok
INFO: /dev/kvm exists
KVM acceleration can be used

lsmod | grep kvm
kvm_intel 200704 0
kvm 593920 1 kvm_intel
irqbypass 16384 2 kvm,vfio_pci


lsmod | grep vfio
kvm_intel 200704 0
kvm 593920 1 kvm_intel
irqbypass 16384 2 kvm,vfio_pci


dmesg | grep vfio
[ 12.717423] vfio-pci 0000:07:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
[ 12.736184] vfio_pci: add [1002:67b1[ffff:ffff]] class 0x000000/00000000
[ 12.756189] vfio_pci: add [1002:aac8[ffff:ffff]] class 0x000000/00000000


VFIO - User Level meta-driver version: 0.3
QEMU version 2.5
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by powerhouse »

@talos_d: Thanks for the update. Hope you find a way to solve the issue.

Looking at the issue in a different way: What would happen if you pass through a USB port and reconnect your Wacom / mouse / keyboard via a small hub? You could start with the mouse at first to see if that solves anything.

As a next step, if that doesn't help, you could try to find a USB controller sitting on the PCI bus. List the PCI ports with:

Code: Select all

lspci
It might be worth trying, though it's not the same as what you want. Once you pass through a USB device, control passes to the guest. I wonder if Spice would show the mouse that is passed through.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by powerhouse »

garrettjk wrote:I currently get no display on my passthrough monitor when trying to run sudo ./windows10vm.sh
its contents are as follows:

#!/bin/bash

...
qemu-system-x86_64 \
...
-machine type=pc,accel=kvm \
...
-m 2300 \
...
-usb -usbdevice host04d9:fa52 -usbdevice host:04b4:5004 \
...
-drive file=/media/libraries/virtio-win-0.1.126.iso,id=virtiocd,format=raw,if=none -device ide-cd,bus=ide.1,drive=virtiocd\


My specs are:
Cpu - i5-6600k (IGPU for linux)
GPU - MSI R9 390 (passthrough)
Motherboard - Gigabyte GA-Z170-HD3P
Ram - 8G ddr4-2133
Looking at the script, there are some typos and other issues - see bold text:

1. machine type=pc -- unless you have a good reason, try q35. Note that if you installed Windows already, you will need to reinstall when changing the type.

2. -m 2300 -- this is very little memory for your Windows VM. The minimum should be -m 4G. Linux doesn't need so much, so your 8G is enough to get it running, but increase the Windows memory to 4G. Don't forget to increase your hugepages too!!!

3. -usb -usbdevice host04d9:fa52 -- should read -usb -usbdevice host:04d9:fa52 -- note the colon.

4. drive=virtiocd\ -- don't forget the space, this should read drive=virtiocd \

Please list your IOMMU groups, especially for the graphics card you want to pass through. If there are any problems, that is there are other devices inside the same IOMMU group with the graphics card, have a look at my new blog: https://heiko-sieger.info/iommu-groups- ... -consider/.

My guess: You run the script, the screen switches and you can't see the messages in the Linux terminal anymore. Is that correct? First fix your start script and run it again.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by powerhouse »

Because of the forum limitations (text length, no. of pictures, etc.), I'm hosting the tutorial on my own website here.

Inside the troubleshooting section, there is a correction:

BSOD when installing AMD Crimson drivers under Windows

Several users on the Redhat VFIO mailing list have reported problems with the installation of AMD Crimson drivers under Windows. This seems to affect a number of AMD graphics cards, as well as a number of different AMD Crimson driver releases. A workaround is described here: https://www.redhat.com/archives/vfio-us ... 00153.html

In this workaround the following line is added to the startup script, right above the definition of the graphics device:

Code: Select all

-device ioh3420,bus=pci,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1
Should the above configuration give you a “Bus ‘pci’ not found” error, change the line as follows:

Code: Select all

-device ioh3420,bus=pci.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1
Then change the graphics card passthrough options as follows:

Code: Select all

-device vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on \
-device vfio-pci,host=02:00.1,bus=root.1,addr=00.1 \
Replace the PCI IDs with the ones of your graphics card.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
swe3tdave

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by swe3tdave »

After days of trying to make this work on fedora, i removed it to install Linux Mint and tried it again with your tutorial... Same problem... :twisted:
Thanks to your troubleshooting section, i got the idea that my Windows 8 ISO might be the issue. So i tried with an Ubuntu ISO i still had and it booted fine... :roll: After that, a fresh ISO from microsoft and everything went smoothly! i even got my onboard Creative ZxR Sound Card working on Windows!! :D

i am sending sound from my Linux host to the Windows VM. This link was extremely usefull for that: https://www.tomjepp.co.uk/post/20150531 ... to-windows

Ahh, the nice sound of Victory! Thank You for the Tutorial, so far, its the best one i've seen yet.
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by powerhouse »

@swe3tdave: Thanks for your kind words. Your link to the audio tutorial will certainly be helpful to others and I will link to it in the tutorial.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
garrettjk

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by garrettjk »

powerhouse wrote:
Looking at the script, there are some typos and other issues - see bold text:

1. machine type=pc -- unless you have a good reason, try q35. Note that if you installed Windows already, you will need to reinstall when changing the type.

2. -m 2300 -- this is very little memory for your Windows VM. The minimum should be -m 4G. Linux doesn't need so much, so your 8G is enough to get it running, but increase the Windows memory to 4G. Don't forget to increase your hugepages too!!!

3. -usb -usbdevice host04d9:fa52 -- should read -usb -usbdevice host:04d9:fa52 -- note the colon.

4. drive=virtiocd\ -- don't forget the space, this should read drive=virtiocd \

Please list your IOMMU groups, especially for the graphics card you want to pass through. If there are any problems, that is there are other devices inside the same IOMMU group with the graphics card, have a look at my new blog: https://heiko-sieger.info/iommu-groups- ... -consider/.

My guess: You run the script, the screen switches and you can't see the messages in the Linux terminal anymore. Is that correct? First fix your start script and run it again.
I attempted to fix the typos that you suggested...
Now I am unable to view my iommu groups at all ( the iommu group folder is empty). I am also told that my kernel does not support huge pages when i try to edit those. I recently update my kernel and I believe that it may be the cause of the issue.
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by powerhouse »

garrettjk wrote:
powerhouse wrote:
Looking at the script, there are some typos and other issues - see bold text:

1. machine type=pc -- unless you have a good reason, try q35. Note that if you installed Windows already, you will need to reinstall when changing the type.

2. -m 2300 -- this is very little memory for your Windows VM. The minimum should be -m 4G. Linux doesn't need so much, so your 8G is enough to get it running, but increase the Windows memory to 4G. Don't forget to increase your hugepages too!!!

3. -usb -usbdevice host04d9:fa52 -- should read -usb -usbdevice host:04d9:fa52 -- note the colon.

4. drive=virtiocd\ -- don't forget the space, this should read drive=virtiocd \

Please list your IOMMU groups, especially for the graphics card you want to pass through. If there are any problems, that is there are other devices inside the same IOMMU group with the graphics card, have a look at my new blog: https://heiko-sieger.info/iommu-groups- ... -consider/.

My guess: You run the script, the screen switches and you can't see the messages in the Linux terminal anymore. Is that correct? First fix your start script and run it again.
I attempted to fix the typos that you suggested...
Now I am unable to view my iommu groups at all ( the iommu group folder is empty). I am also told that my kernel does not support huge pages when i try to edit those. I recently update my kernel and I believe that it may be the cause of the issue.
You may need to re-install hugepages. As to the IOMMU groups, make sure your /etc/default/grub file contains the option that enables IOMMU (see tutorial), and that you run

Code: Select all

update-grub
Reboot and your /sys/kernel/iommu_groups directory should be filled with folders. If that does not happen, check your BIOS settings (see my tutorial as well as the motherboard manual for more).

EDIT: If the above suggestions don't help, it might be better to do a clean installation of Linux and start from scratch. The start script changes you made have no bearing on the IOMMU groups. And the fact that hugepages aren't found may just indicate that you have made other alterations, or that something is wrong with either hardware or the Linux installation. You must stick to the tutorial - no shortcuts.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
wye141

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by wye141 »

I follow this thread a long time and i want to say that this is a great guide even if it doesnt work for me yet.

I tried to setup this for about ~9h now, and im stuck.

When im disabling "-vga none" everything seems to work fine, with this option enabled, the machine wont start, only the compatmonitor pops up.

Without "-vga none" it uses a MIcrosoft Standard Display Adaptor with 800x600 and average performance.
The Nvidia card is displayed beneath the shitty graphics adaptor in the device manager.

I tried to follow the guide step by step, maybe iam missing something, maybe you can see the wrong option/typo/error.

Here is my config:

Code: Select all

i7 3930k
16GB RAM
AMD RX550 (host)
Nvidia GTX1080 (WindowsVM)
I am using Windows 10 for the VM.

Here is my config:

Code: Select all

#!/bin/bash

vmname="GAMINGVM"

if ps -A | grep -q $vmname; then
   echo "$vmname is already running." &
   exit 1

else

# use pulseaudio
export QEMU_AUDIO_DRV=pa
export QEMU_PA_SAMPLES=8192
export QEMU_AUDIO_TIMER_PERIOD=99
export QEMU_PA_SERVER=/run/user/1000/pulse/native

cp /usr/share/OVMF/OVMF_VARS.fd /tmp/my_vars.fd

qemu-system-x86_64 \
  -name $vmname,process=$vmname \
  -machine type=pc,accel=kvm \
  -cpu host,kvm=off \
  -smp 8,sockets=1,cores=4,threads=2 \
  -enable-kvm \
  -m 11264 \
  -mem-path /dev/hugepages \
  -mem-prealloc \
  -balloon none \
  -rtc clock=host,base=localtime \
  -vga none \
  -serial none \
  -parallel none \
  -soundhw hda \
  -usb -usbdevice host:046d:c331 -usbdevice host:1038:1369 \
  -device vfio-pci,host=02:00.0,multifunction=on \
  -device vfio-pci,host=02:00.1 \
  -drive if=pflash,format=raw,readonly,file=/usr/share/OVMF/OVMF_CODE.fd \
  -drive if=pflash,format=raw,file=/tmp/my_vars.fd \
  -boot order=dc \
  -device virtio-scsi-pci,id=scsi \
  -drive id=disk0,if=virtio,cache=none,format=raw,file=/home/pscl/data/gamingvm.img \
  -drive file=/home/pscl/isos/windoofs.iso,id=isocd,format=raw,if=none -device scsi-cd,drive=isocd \
  -drive file=/home/pscl/isos/virtio.iso,id=virtiocd,format=raw,if=none -device ide-cd,bus=ide.1,drive=virtiocd \
  -netdev type=tap,id=net0,ifname=tap0,vhost=on \
  -device virtio-net-pci,netdev=net0,mac=E0:3F:49:6D:0F:F3

   exit 0
fi
/etc/default/grub

Code: Select all

GRUB_DEFAULT=0
GRUB_HIDDEN_TIMEOUT=0
GRUB_HIDDEN_TIMEOUT_QUIET=true
GRUB_TIMEOUT=10
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="modprobe.blacklist=nouveau quiet intel_iommu=on"
GRUB_CMDLINE_LINUX=""
Check for IOMMU

Code: Select all

dmesg | grep "Virtualization Technology for Directed I/O"
[    1.022297] DMAR: Intel(R) Virtualization Technology for Directed I/O
output of lspci | grep VGA

Code: Select all

01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Device 699f (rev c7)
02:00.0 VGA compatible controller: NVIDIA Corporation Device 1b80 (rev a1)
output of lspci -nn | grep 02:00.

Code: Select all

02:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:1b80] (rev a1)
02:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:10f0] (rev a1)
/etc/modprobe.d/local.conf

Code: Select all

options vfio-pci ids=10de:1b80,10de:10f0
/etc/initramfs-tools/modules

Code: Select all

options vfio-pci ids=10de:1b80,10de:10f0
iommu_groups

Code: Select all

/sys/kernel/iommu_groups/17/devices/0000:01:00.1
/sys/kernel/iommu_groups/17/devices/0000:01:00.0
/sys/kernel/iommu_groups/18/devices/0000:02:00.1
/sys/kernel/iommu_groups/18/devices/0000:02:00.0

Code: Select all

# List of modules that you want to include in your initramfs.
# They will be loaded at boot time in the order below.
#
# Syntax:  module_name [args ...]
#
# You must run update-initramfs(8) to effect this change.
#
# Examples:
#
# raid1
# sd_mod
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
vhost-net
kvm-ok

Code: Select all

INFO: /dev/kvm exists
KVM acceleration can be used
lsmod | grep kvm

Code: Select all

kvm_intel             200704  0
kvm                   593920  1 kvm_intel
irqbypass              16384  2 kvm,vfio_pci
lsmod | grep vfio

Code: Select all

vfio_pci               45056  0
vfio_virqfd            16384  1 vfio_pci
irqbypass              16384  2 kvm,vfio_pci
vfio_iommu_type1       24576  0
vfio                   32768  2 vfio_iommu_type1,vfio_pci
qemu-system-x86_64 –version

Code: Select all

QEMU emulator version 2.5.0 (Debian 1:2.5+dfsg-5ubuntu10.14), Copyright (c) 2003-2008 Fabrice Bellard
dmesg | grep vfio

Code: Select all

[    2.533225] vfio-pci 0000:02:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
[    2.552559] vfio_pci: add [10de:1b80[ffff:ffff]] class 0x000000/00000000
[    2.572569] vfio_pci: add [10de:10f0[ffff:ffff]] class 0x000000/00000000
[    3.929284] vfio-pci 0000:02:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
dmesg | grep VFIO

Code: Select all

[    2.527406] VFIO - User Level meta-driver version: 0.3
I hope that this information could make this easy for you.
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by powerhouse »

@wye141: You got nearly the same hardware I have and that is very good!

For completeness, please post the manufacturer and model of the motherboard. Check which BIOS software version the motherboard is running and if there are newer ones that might fix IOMMU issues - some manufacturers have had their issues, but in many cases they have been fixed.

The most obvious difference in your start script is the -machine type=pc option. You should change it to q35 for better performance. This option may or may not be the reason for your Windows VM not starting.

If you already installed Windows, changing the type to q35 or vice versa from q35 to pc will require you to reinstall your Windows VM.

When I install a VM I usually land in an UEFI command line screen and need to type "exit" to get to the UEFI menu. Make sure your Nvidia 1080 is connected to a screen and that you selected the proper input for that screen (via push-button or whatever).

You do not need the vga=on option.

Make sure your mouse and keyboard is passed through to the VM. It should not be working in Linux.

Let's start from here and see if it works.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
wye141

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by wye141 »

Thank you for the quick response!

The mainboard manufacturer is ASUS, its an Rampage IV Gene; https://www.asus.com/de/Motherboards/RAMPAGE_IV_GENE/
I enabled Intel VT-d through the BIOS, i checked again, if it is still enabled, it was.

I had installed windows with pc and then switched to q35, it worked, but with the microsoft standard grpahics adaptor, and not with the gtx1080.
I deleted the VM, because i thought the problem were the drivers inside, because it could not be started.

thank you, the problem is solved now, i connected the cable to the display and started the vm with -vga none and -nographics and q35, it worked.
i had a wrong idea about how this work.

The System is using the Microsoft Standard Adaptor, i will take a look at this, thanks for help!

The i7 3930k is still a very good cpu and i hope this one runs for the next two years, my biggest fear about this, is, that the mainboard will stop working, because they are really hard/expensive to get nowadays.

EDIT: Everything does work now, i could install the nvidia drivers and can use the full resolution of my Screen.

Iam using three displays, at the middle screen runs the windows vm with synergy as a server and at the left and right is the linux host displayed, the linux host runs as a client.
Synergy runs like a charm, i cant recognize any input lag, switching the mouse works good, even the clipboard is available at both systems.
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by powerhouse »

wye141 wrote:Thank you for the quick response!

The mainboard manufacturer is ASUS, its an Rampage IV Gene; https://www.asus.com/de/Motherboards/RAMPAGE_IV_GENE/
I enabled Intel VT-d through the BIOS, i checked again, if it is still enabled, it was.

I had installed windows with pc and then switched to q35, it worked, but with the microsoft standard grpahics adaptor, and not with the gtx1080.
I deleted the VM, because i thought the problem were the drivers inside, because it could not be started.

thank you, the problem is solved now, i connected the cable to the display and started the vm with -vga none and -nographics and q35, it worked.
i had a wrong idea about how this work.

The System is using the Microsoft Standard Adaptor, i will take a look at this, thanks for help!

The i7 3930k is still a very good cpu and i hope this one runs for the next two years, my biggest fear about this, is, that the mainboard will stop working, because they are really hard/expensive to get nowadays.

EDIT: Everything does work now, i could install the nvidia drivers and can use the full resolution of my Screen.

Iam using three displays, at the middle screen runs the windows vm with synergy as a server and at the left and right is the linux host displayed, the linux host runs as a client.
Synergy runs like a charm, i cant recognize any input lag, switching the mouse works good, even the clipboard is available at both systems.
Wow, good news! I'm glad everything is working so well. The 3930K is an excellent CPU. It may not have the fastest single thread performance, but when the software can make use of multiple cores, it kicks a..

You may want to install the CPU Frequency Scaling Monitor applet to your task bar. The VM doesn't automatically trigger turbo mode. With this applet to can switch to Performance mode and gain another ~20% performance boost.

It would be great if you could do a benchmark using http://www.userbenchmark.com/Software within Windows, then post the benchmark together with your hardware specs here: viewtopic.php?f=225&t=153482. Of course, Passmark results are also welcome (I use both).

Enjoy the new experience! Any feedback on what you run on Windows (games, etc.) and its performance are welcome.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
juju

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by juju »

Signed up to post a big thanks to powerhouse and all other linux mint users in this thread who contributed. I had tried to do a passthrough about 1.5 years ago on a different machine, but could not do ACS Override with my configuration. I built a new machine specs below, some parts missing.

I just got the kvm VGA passthrough working without an ACS Override patch. I had IOMMU grouping conflicts between both of my cards, but I updated my BIOS to the latest version and it split the PCI-E Video devices into their own IOMMU groupings. I read using the ACS Override patch is insecure, I doubt I would ever run into that issue but I am glad I got it to work without having to resort to using it. I had poor audio quality so I setup the Realtek ac97 audio drivers for the audio device and installed within the Windows 10 VM.

Machine
Host OS: Xubuntu 16.0.4
Guest OS: Windows 10 x64
CPU: AMD Ryzen 7 1800x - 12 Cores Guest - 4 Cores Host
Mobo: ASUS Crosshair VI Hero - 1501 BIOS Version
Host Card: AMD Firepro v7900
Guest: NVIDIA GTX 970 (Current Nvidia Driver)
RAM: 64GB (32GB per system right now)
Disks:
--Host: SSD - 500GB
--Guest: SSD - 1TB

I have a GTX 1050 TI SC I would like to get working but it was too difficult for me to find a way to assign a different driver to my 1050ti and the 970, whatever I put on one would go to the other. Either way for the time being the Firepro card I had kicking around did the trick. I am going to get back to messing around in the guest thanks again! Ill check back here at some point soon in-case anyone asks me any questions about the setup.
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by powerhouse »

juju wrote:Signed up to post a big thanks to powerhouse and all other linux mint users in this thread who contributed. I had tried to do a passthrough about 1.5 years ago on a different machine, but could not do ACS Override with my configuration. I built a new machine specs below, some parts missing.

I just got the kvm VGA passthrough working without an ACS Override patch. I had IOMMU grouping conflicts between both of my cards, but I updated my BIOS to the latest version and it split the PCI-E Video devices into their own IOMMU groupings. I read using the ACS Override patch is insecure, I doubt I would ever run into that issue but I am glad I got it to work without having to resort to using it. I had poor audio quality so I setup the Realtek ac97 audio drivers for the audio device and installed within the Windows 10 VM.

Machine
Host OS: Xubuntu 16.0.4
Guest OS: Windows 10 x64
CPU: AMD Ryzen 7 1800x - 12 Cores Guest - 4 Cores Host
Mobo: ASUS Crosshair VI Hero - 1501 BIOS Version
Host Card: AMD Firepro v7900
Guest: NVIDIA GTX 970 (Current Nvidia Driver)
RAM: 64GB (32GB per system right now)
Disks:
--Host: SSD - 500GB
--Guest: SSD - 1TB

I have a GTX 1050 TI SC I would like to get working but it was too difficult for me to find a way to assign a different driver to my 1050ti and the 970, whatever I put on one would go to the other. Either way for the time being the Firepro card I had kicking around did the trick. I am going to get back to messing around in the guest thanks again! Ill check back here at some point soon in-case anyone asks me any questions about the setup.
I'm glad you joined the club! It would be great if you could run some benchmark in the Windows VM so to get an idea of how it performs. See my post above.

Actually, you should be able to get your GTX 1050 working under Linux, and it shouldn't be that difficult either. You need to blacklist the nouveau driver (see tutorial). Also make sure that your GTX 970 is attached to the vfio-pci driver (use the initramfs method described in the tutorial) - since you already have VGA passthrough working, your settings should be fine.

Linux Mint comes with a convenient proprietary drivers manager. Check your GTX 1050 and select a proprietary driver. Make sure that your GTX 970 uses the Opensource (nouveau) driver. Install and reboot.

EDIT: Perhaps this may help for Xubuntu: https://sites.google.com/site/easylinux ... ng-drivers

Like you, I have two different Nvidia cards - one for the host (a Quadro 2000), one for the Windows VM (GTX 970). Once I discovered the trick with the proprietary driver selection, all was fine. See the troubleshooting section for more.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
juju

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by juju »

I purchased some Benchmarking Software (3DMark/PCMark) last night so I can start to run some tests with different configs/drivers, I will also review your post and get the windows scores ASAP (at work right now). I actually tried it the first night I got it working but hit an issue complaining about my realtime clock or something, I did not work through this yet.

I was a bit disheartened last night because my Video Game Rendering performance seemed to degrade to an unusable state over night magically o_o After a little troubleshooting I narrowed it down to the Windows 10 "Creator Update" that was automatically installed just before I turned my VM off the night before, it was the only thing that changed with the setup. Before the Windows 10 Creator Update I noticed no issues in any application I tested felt like using Native hardware. I decided to roll my Windows 10 build back to before the update and reinstalled my video drivers as they got removed during the rollback and my performance was back to normal. If I keep running into issues with Windows 10 x64, I might just go ahead and use Windows 7 x64 Ultimate.

I did the Passthrough specifically to play Native Windows games, I used a recent game "Elder Scrolls Online" as a way to benchmark my performance in the VM temporarily, I have almost everything cranked to Max, noticed no issues with FPS/performance after I had rolled back the OS version to before Creator Update. With the Creator Update version of Windows 10 it was basically a slideshow. Audio with the ac97 device I created did not work as well as I had liked, still lots of distortion. I found a USB Sound card in my room and just passed it through, now my Audio is Fantastic! I am looking at getting an ASUS or Soundblaster Soundcard to passthrough permanently.

I inadvertently did some Networking Benchmarking, I downloaded ~70GB worth of Games from Steam at an average of ~7mbps. I also get roughly about this speed when downloading games using native hardware so I was impressed it was not much slower.

Probably sometime this week or weekend, I will try and use my 1050, I might also try to use Linux Mint. I have no loyalty with distros, really I just prefer using Debian distros with xfce4 as I have my own theme I made. I know Linux Mint is debian so I will probably feel right at home. Thanks again for all the effort to make this thread/update it, I also checked out your Blog page! I have linked both to a few colleagues who have tried to do this also in the past years but failed.

Cheers,
juju
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by powerhouse »

@juju: Thanks for the reply. I hope you didn't buy the benchmarking software because of my request. Passmark can be had for a trial, and the userbenchmark I suggest now is free.

The good things about benchmarks is that it allows you to compare with similar configurations, and to discover problems in the configuration. With KVM there are a number of performance issues one may run into. The settings I've given in the how-to have worked well for me, and it seems with many other people too.

Networking speed: With a bridge set up you should get somewhere around 10Gbit between the host and the VM. This is a transfer speed of about 1.2GB/s - no SSD is fast enough for that. Of course, Internet speed is something different.

I'm using Win 10 with the creators update, but have tweaked the settings after installation. If you install Windows 7, make sure you can UEFI boot the Windows ISO.

Sound: It took me a while to get decent sound. The best way is to do what you did - pass through a sound card. I'm currently using the following in my script, which offers good sound without the crackling noise:

Code: Select all

export QEMU_AUDIO_DRV=alsa
export QEMU_ALSA_ADC_BUFFER_SIZE=1024 QEMU_ALSA_ADC_PERIOD_SIZE=256
export QEMU_ALSA_DAC_BUFFER_SIZE=1024 QEMU_ALSA_DAC_PERIOD_SIZE=256
export QEMU_AUDIO_DAC_FIXED_SETTINGS=1
export QEMU_AUDIO_DAC_FIXED_FREQ=44100 QEMU_AUDIO_DAC_FIXED_FMT=S16 QEMU_AUDIO_ADC_FIXED_FREQ=44100 QEMU_AUDIO_ADC_FIXED_FMT=S16
export QEMU_AUDIO_DAC_TRY_POLL=1 QEMU_AUDIO_ADC_TRY_POLL=1
export QEMU_AUDIO_TIMER_PERIOD=50
The qemu option then is:

Code: Select all

  -soundhw hda \
Obviously, your solution is better. Another option is to use a USB audio stick. There are some real good ones (audiophile quality). In that case I would pass through a USB controller.

I recently posted my Unigine Heaven benchmark (yet another free graphics benchmark) over at the overclock.net forum: http://www.overclock.net/t/1235557/offi ... t_26293511

It's not breathtaking, but absolutely comparable with other GTX970 benchmarks that were posted there (all others were overclocked, mine not).

You should be able to play Windows games on your VM and not notice any difference.

Don't switch to Linux Mint just because my how-to is written for it. You can replace the xed commands with gedit and it works the same in Ubuntu/Xubuntu. Stick with the distribution you feel most comfortable with. Look at my last post regarding how to get the 1050 work with your 970. I'm sure you'll make it work.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
HPMint

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by HPMint »

Hi everybody!

Thank you powerhouse for this How-to.

If win10 starts without problem, use it is impossible: without mouse and keyboard, it's ... difficult :mrgreen:

Some data about my hardware:
CPU: i7 7700K
MB: Asrock Z270 Taichi
RAM: 32Gb (16/16 win/linux)
GPU: 1080 Ti
Keyboard: Corsair K70
Mouse: Corsair M65

OS: Linux Mint 18.2
QEMU: 2.9 (I've build it myself).

I have a KVM USB 3.0 with 2 output ports and 1 input : I plugged an USB 3.0 hub, with keyboard and mouse plugged on it. The KVM is plugged on Intel USB 3.0 and ASMedia USB 3.0 on the MB.

This is the IOMMU groups:

Code: Select all

IOMMU Group 00 00:00.0 Host bridge [0600]: Intel Corporation Device [8086:591f] (rev 05)
IOMMU Group 01 00:01.0 PCI bridge [0604]: Intel Corporation Sky Lake PCIe Controller (x16) [8086:1901] (rev 05)
IOMMU Group 01 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:1b06] (rev a1)
IOMMU Group 01 01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:10ef] (rev a1)
IOMMU Group 02 00:02.0 VGA compatible controller [0300]: Intel Corporation Device [8086:5912] (rev 04)
IOMMU Group 03 00:14.0 USB controller [0c03]: Intel Corporation Device [8086:a2af]
IOMMU Group 03 00:14.2 Signal processing controller [1180]: Intel Corporation Device [8086:a2b1]
IOMMU Group 04 00:16.0 Communication controller [0780]: Intel Corporation Device [8086:a2ba]
IOMMU Group 05 00:17.0 SATA controller [0106]: Intel Corporation Device [8086:a282]
IOMMU Group 06 00:1b.0 PCI bridge [0604]: Intel Corporation Device [8086:a2e7] (rev f0)
IOMMU Group 07 00:1c.0 PCI bridge [0604]: Intel Corporation Device [8086:a290] (rev f0)
IOMMU Group 08 00:1c.2 PCI bridge [0604]: Intel Corporation Device [8086:a292] (rev f0)
IOMMU Group 09 00:1c.4 PCI bridge [0604]: Intel Corporation Device [8086:a294] (rev f0)
IOMMU Group 10 00:1d.0 PCI bridge [0604]: Intel Corporation Device [8086:a298] (rev f0)
IOMMU Group 11 00:1d.6 PCI bridge [0604]: Intel Corporation Device [8086:a29e] (rev f0)
IOMMU Group 12 00:1d.7 PCI bridge [0604]: Intel Corporation Device [8086:a29f] (rev f0)
IOMMU Group 13 00:1f.0 ISA bridge [0601]: Intel Corporation Device [8086:a2c5]
IOMMU Group 13 00:1f.2 Memory controller [0580]: Intel Corporation Device [8086:a2a1]
IOMMU Group 13 00:1f.3 Audio device [0403]: Intel Corporation Device [8086:a2f0]
IOMMU Group 13 00:1f.4 SMBus [0c05]: Intel Corporation Device [8086:a2a3]
IOMMU Group 14 00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (2) I219-V [8086:15b8]
IOMMU Group 15 03:00.0 USB controller [0c03]: ASMedia Technology Inc. Device [1b21:2142]
IOMMU Group 16 04:00.0 PCI bridge [0604]: ASMedia Technology Inc. Device [1b21:1184]
IOMMU Group 17 05:01.0 PCI bridge [0604]: ASMedia Technology Inc. Device [1b21:1184]
IOMMU Group 17 06:00.0 Network controller [0280]: Intel Corporation Wireless 3160 [8086:08b3] (rev 83)
IOMMU Group 18 05:03.0 PCI bridge [0604]: ASMedia Technology Inc. Device [1b21:1184]
IOMMU Group 18 07:00.0 Ethernet controller [0200]: Intel Corporation I211 Gigabit Network Connection [8086:1539] (rev 03)
IOMMU Group 19 05:05.0 PCI bridge [0604]: ASMedia Technology Inc. Device [1b21:1184]
IOMMU Group 19 08:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 02)
IOMMU Group 20 05:07.0 PCI bridge [0604]: ASMedia Technology Inc. Device [1b21:1184]
IOMMU Group 20 09:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 02)
IOMMU Group 21 0c:00.0 USB controller [0c03]: Fresco Logic FL1100 USB 3.0 Host Controller [1b73:1100] (rev 01)
The USB info by IOMMU:

Code: Select all

Bus 1 --> 0000:00:14.0 (IOMMU group 3)                           <=============== Intel Controller [8086:a2af]
Bus 001 Device 005: ID 8087:07dc Intel Corp. 
Bus 001 Device 003: ID 174c:2074 ASMedia Technology Inc. ASM1074 High-Speed hub
Bus 001 Device 010: ID 1b1c:0a03 Corsair 
Bus 001 Device 008: ID 1b1c:1b31 Corsair 
Bus 001 Device 015: ID 1b1c:1b38 Corsair                          <========================= Mouse or Keyboard
Bus 001 Device 013: ID 2109:2812 VIA Labs, Inc. VL812 Hub
[b]Bus 001 Device 014: ID 1b1c:1b2e Corsair                       <========================= Mouse or Keyboard
Bus 001 Device 012: ID 2109:2812 VIA Labs, Inc. VL812 Hub
Bus 001 Device 011: ID 2109:2812 VIA Labs, Inc. VL812 Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Bus 2 --> 0000:00:14.0 (IOMMU group 3)
Bus 002 Device 003: ID 174c:3074 ASMedia Technology Inc. ASM1074 SuperSpeed hub
Bus 002 Device 010: ID 11b0:6348 ATECH FLASH TECHNOLOGY 
Bus 002 Device 009: ID 2109:0812 VIA Labs, Inc. VL812 Hub
Bus 002 Device 008: ID 2109:0812 VIA Labs, Inc. VL812 Hub
Bus 002 Device 007: ID 2109:0812 VIA Labs, Inc. VL812 Hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub

Bus 3 --> 0000:03:00.0 (IOMMU group 15)                           <=============== ASMedia Controller [1b21:2142]
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Bus 4 --> 0000:03:00.0 (IOMMU group 15)
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub

Bus 5 --> 0000:0c:00.0 (IOMMU group 21)                           <=============== Fresco Controller [1b73:1100]
Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Bus 6 --> 0000:0c:00.0 (IOMMU group 21)
Bus 006 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
The grub line:

Code: Select all

GRUB_CMDLINE_LINUX_DEFAULT="modprobe.blacklist=nouveau quiet intel_iommu=on"
The modprobe.d/local.conf:

Code: Select all

options vfio-pci ids=10de:1b06,10de:10ef
The vfio-pci.cfg (for Ethernet & USB Controllers) :

Code: Select all

0000:00:1f.6
0000:03:00.0
The initramfs-tools/modules:

Code: Select all

vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
#vhost-net
I've commented vhost-net as I passthrough an ethernet controller.

And my script to start the VM:

Code: Select all

#!/bin/bash

vmname="windows10vm"

configfile=/etc/vfio-pci.cfg

vfiobind() {
   dev="$1"
        vendor=$(cat /sys/bus/pci/devices/$dev/vendor)
        device=$(cat /sys/bus/pci/devices/$dev/device)
        if [ -e /sys/bus/pci/devices/$dev/driver ]; then
                echo $dev > /sys/bus/pci/devices/$dev/driver/unbind
        fi
        echo $vendor $device > /sys/bus/pci/drivers/vfio-pci/new_id
}

if ps -A | grep -q $vmname; then
   echo "$vmname is already running." &
   exit 1

else

   cat $configfile | while read line;do
   echo $line | grep ^# >/dev/null 2>&1 && continue
      vfiobind $line
   done

cp /usr/share/OVMF/OVMF_VARS.fd /tmp/my_vars.fd

qemu-system-x86_64 \
  -name $vmname,process=$vmname \
  -machine type=q35,accel=kvm \
  -cpu host,kvm=off \
  -smp 4,sockets=1,cores=2,threads=2 \
  -m 16G \
  -mem-path /run/hugepages/kvm \
  -mem-prealloc \
  -balloon none \
  -rtc clock=host,base=localtime \
  -serial none \
  -nographic \
  -vga none \
  -parallel none \
  -device vfio-pci,host=01:00.0,multifunction=on \
  -device vfio-pci,host=01:00.1 \
  -device vfio-pci,host=00:1f.6 \
  -device vfio-pci,host=03:00.0 \
  -drive if=pflash,format=raw,readonly,file=/usr/share/OVMF/OVMF_CODE.fd \
  -drive if=pflash,format=raw,file=/tmp/my_vars.fd \
  -drive id=BootWin,if=none,format=raw,file=/dev/disk/by-id/ata-Crucial_CT480M500SSD1_xxxxxxx \
  -device virtio-blk-pci,drive=BootWin \
  -drive id=WinData,if=none,format=raw,file=/dev/disk/by-id/ata-SAMSUNG_HD204UI_yyyyyyy \
  -device virtio-blk-pci,drive=WinData \
  -drive id=Games1,if=none,format=raw,file=/dev/disk/by-id/ata-ST2000DM001-1ER164_zzzzzzz \
  -device virtio-blk-pci,drive=Games1 \
  -drive id=Games2,if=none,format=raw,file=/dev/disk/by-id/ata-ST2000DM001-1ER164_wwwwwww \
  -device virtio-blk-pci,drive=Games2 \
  -drive id=Divers,if=none,format=raw,file=/dev/disk/by-id/ata-ST31000524AS_vvvvvvvv \
  -device virtio-blk-pci,drive=Divers

And now, my problems:
- Sound: I was unable to have a good sound with hda (a lot of clicks), so I plugged the headset output of my windows screen into the line-in of Linux.
- Ethernet: I've manage to passthrough one of the 2 ethernet controller and it works. The bridge never worked. Don't know why...
- Disk: BootWin is a SSD... but Windows do not see it as is. I've tried to passthrough the SATA Controller. No success: Windows do not boot, got a black screen.
- And USB: a NIGHTMARE! I think I've tried all I can to have my keyboard and mouse, but each time, after some time, Windows loose the USB, keyboard and mouse goes offline with the sound as if I've unplug any USB device. With another mouse, (passed by -usb -usbdevice), if I deactivate/activate the usb controller in Windows, it comes back... to go down sometime later.
Sometimes it takes several hours before failure, sometimes, it takes a few minutes.
I've tried to switch to -usb -usbdevice host:1b1c:1b2e -usbdevice host:1b1c:1b38 (and removing the passthrough of USB Controller) but the VM do not even start or is super-slow and USB is not passed, it continues to work under Linux! I don't understand: if I plugged and pass another mouse/kb this way, it works fine...

All my search for the past week on the net and tries never work. I've installed the Fresco Controller to try with. It's worse: sometimes id is 0b:00.0 sometimes 0c:00.0, sometimes USB never even init...

I'm totally lost, I'm desperate! :cry: :cry:

Can someone help me? I can do any test you think of. If you want any log, data, please tell me where or how to find them.
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by powerhouse »

HPMint wrote: Some data about my hardware:
CPU: i7 7700K
MB: Asrock Z270 Taichi
RAM: 32Gb (16/16 win/linux)
GPU: 1080 Ti
Keyboard: Corsair K70
Mouse: Corsair M65

OS: Linux Mint 18.2
QEMU: 2.9 (I've build it myself).

I have a KVM USB 3.0 with 2 output ports and 1 input : I plugged an USB 3.0 hub, with keyboard and mouse plugged on it. The KVM is plugged on Intel USB 3.0 and ASMedia USB 3.0 on the MB.

...

And now, my problems:
- Sound: I was unable to have a good sound with hda (a lot of clicks), so I plugged the headset output of my windows screen into the line-in of Linux.
- Ethernet: I've manage to passthrough one of the 2 ethernet controller and it works. The bridge never worked. Don't know why...
- Disk: BootWin is a SSD... but Windows do not see it as is. I've tried to passthrough the SATA Controller. No success: Windows do not boot, got a black screen.
- And USB: a NIGHTMARE! I think I've tried all I can to have my keyboard and mouse, but each time, after some time, Windows loose the USB, keyboard and mouse goes offline with the sound as if I've unplug any USB device. With another mouse, (passed by -usb -usbdevice), if I deactivate/activate the usb controller in Windows, it comes back... to go down sometime later.
Sometimes it takes several hours before failure, sometimes, it takes a few minutes.
I've tried to switch to -usb -usbdevice host:1b1c:1b2e -usbdevice host:1b1c:1b38 (and removing the passthrough of USB Controller) but the VM do not even start or is super-slow and USB is not passed, it continues to work under Linux! I don't understand: if I plugged and pass another mouse/kb this way, it works fine...

All my search for the past week on the net and tries never work. I've installed the Fresco Controller to try with. It's worse: sometimes id is 0b:00.0 sometimes 0c:00.0, sometimes USB never even init...

I'm totally lost, I'm desperate! :cry: :cry:

Can someone help me? I can do any test you think of. If you want any log, data, please tell me where or how to find them.
1. For sound, try this:

Code: Select all

#use ALSA
export QEMU_AUDIO_DRV=alsa
export QEMU_ALSA_ADC_BUFFER_SIZE=1024 QEMU_ALSA_ADC_PERIOD_SIZE=256
export QEMU_ALSA_DAC_BUFFER_SIZE=1024 QEMU_ALSA_DAC_PERIOD_SIZE=256
export QEMU_AUDIO_DAC_FIXED_SETTINGS=1
export QEMU_AUDIO_DAC_FIXED_FREQ=44100 QEMU_AUDIO_DAC_FIXED_FMT=S16 QEMU_AUDIO_ADC_FIXED_FREQ=44100 QEMU_AUDIO_ADC_FIXED_FMT=S16
export QEMU_AUDIO_DAC_TRY_POLL=1 QEMU_AUDIO_ADC_TRY_POLL=1
export QEMU_AUDIO_TIMER_PERIOD=50
2. Bridge setup can be a pain in the neck. Instead of removing Network Manager, I'm now using it to configure the bridge. See Define a network bridge using Ubuntu’s / Linux Mint’s Network Manager application - sorry, but the forum limitations make it impossible to post tutorials with many pictures.
Your solution is perfectly OK.

3. BootWin SSD: In my tutorial I describe how to have Windows detect an SSD. Did you follow that? Or am I misunderstanding your post?

4. USB mouse and keyboard: Remove the USB KVM switch and connect your mouse and keyboard directly to the passed through USB ports. How does that work? If that solves the problem, return your KVM switch to the shop for refund.

I had the exact same issue with my USB KVM switch. Over time, the mouse / keyboard would stop working. Connecting the keyboard/mouse directly to the USB solved the issue.

Many USB KVM switches are simply garbage. The best solution is probably Synergy. Another solution are multi-device keyboards and mice. I now use a Logitech K780 multi-device wireless keyboard and a Logitech MX Master wireless mouse (also multi-device), both pretty expensive, but they do their job.

Some remarks and questions on your configuration:

a. I hate to see that a newer CPU such as the Intel 7700K is actually worse in terms of IOMMU and ACS compared to my old 3930K. Intel could have done better.
b. Why did you compile qemu 2.9? Was it needed to get VGA passthrough working? I run the standard qemu 2.5 that comes with Linux Mint and it works fine.
c. How is your WinBoot SSD configured? Is it an NTFS disk, a LVM disk, etc.? Did/does it hold your "bare metal" Windows installation?
If you pass through the WinBoot SSD's SATA controller, then WinBoot better be a native Windows disk with boot sector and everything. Else it won't work.
d. Did you try to configure the bridge using Network Manager, or did you use the bridge-utils tools?

Looks like you got a great gaming setup. Once the issues are sorted out, would you mind to post some benchmarks (see my previous 2-3 posts above)?
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
HPMint

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by HPMint »

Thank you for your interest in my problems.
powerhouse wrote: 1. For sound, try this:

Code: Select all

#use ALSA
export QEMU_AUDIO_DRV=alsa
export QEMU_ALSA_ADC_BUFFER_SIZE=1024 QEMU_ALSA_ADC_PERIOD_SIZE=256
export QEMU_ALSA_DAC_BUFFER_SIZE=1024 QEMU_ALSA_DAC_PERIOD_SIZE=256
export QEMU_AUDIO_DAC_FIXED_SETTINGS=1
export QEMU_AUDIO_DAC_FIXED_FREQ=44100 QEMU_AUDIO_DAC_FIXED_FMT=S16 QEMU_AUDIO_ADC_FIXED_FREQ=44100 QEMU_AUDIO_ADC_FIXED_FMT=S16
export QEMU_AUDIO_DAC_TRY_POLL=1 QEMU_AUDIO_ADC_TRY_POLL=1
export QEMU_AUDIO_TIMER_PERIOD=50
I keep that for later. My priority is the USB. Using the HDMI=>screen headset=>line-in=>real headset resolves two : while having the clicks, I've seen that video and sound desynchronize when playing youtube videos. Perhaps using your proposition would work. But I prefer to focus: one problem at a time. :)
powerhouse wrote: 2. Bridge setup can be a pain in the neck. Instead of removing Network Manager, I'm now using it to configure the bridge. See Define a network bridge using Ubuntu’s / Linux Mint’s Network Manager application - sorry, but the forum limitations make it impossible to post tutorials with many pictures.
Your solution is perfectly OK.
I've done what your how-to say: using Network manager to define the bridge. Windows was having an IP outside of my network (10.xx... instead of 192.168....), could not join the gateway even if force the values, etc. And passing through a controller was more simple if I need to unplug the windows from network and not Linux. By the way under windows, I have now 2 ethernet connections... I've read that qemu automatically create a network if no -netdev option is specified.
It's a detail in which I'll dig later.
powerhouse wrote: 3. BootWin SSD: In my tutorial I describe how to have Windows detect an SSD. Did you follow that? Or am I misunderstanding your post?
I was stuck some days on drives subject. I followed your tutorial and try to add my 5 HDD (1 SSD & 4 HDD) and a CD drive... I fall into the limitation of 4 drives max. Your tuto use the IDE bus built-in qemu, which is limited to 4 drives. But with the help of the links to post about drives optimizations, I've understand a lot and figured how to add SATA controllers with virtio-blk-pci (or virtio-scsi-pci).
So I use the lines:
-drive id=BootWin,if=none,format=raw,file=/dev/disk/by-id/ata-Crucial_CT480M500SSD1_xxxxxxx \
-device virtio-blk-pci,drive=BootWin \
to give the full disk for Windows boot (xxxxxxx are only the serial number, no part in it).
But now that you're asking... I know I've done the winsat formal, but I do not remember if it was before or after my conf with virtio-blk-pci...
I'll replay it... (see next paragraph).
powerhouse wrote: 4. USB mouse and keyboard: Remove the USB KVM switch and connect your mouse and keyboard directly to the passed through USB ports. How does that work? If that solves the problem, return your KVM switch to the shop for refund.

I had the exact same issue with my USB KVM switch. Over time, the mouse / keyboard would stop working. Connecting the keyboard/mouse directly to the USB solved the issue.

Many USB KVM switches are simply garbage. The best solution is probably Synergy. Another solution are multi-device keyboards and mice. I now use a Logitech K780 multi-device wireless keyboard and a Logitech MX Master wireless mouse (also multi-device), both pretty expensive, but they do their job.
That's a try I've done: keyboard+mouse use 3 USB ports. I've plugged them on the fresco controller and try to passthrough the controller... And I think it worked... (I should have write down all my tests and results). I think it worked, but as soon as I stop windows, mouse and keyboard were no more usable as the controller was no more visible for Linux.
I've just replay these tests:
- USB hub plugged: does not work
- kn&m plugged directly on the fresco card: it works fine... for USB (I don't know how long it will last). But boot was chaotic: Tianno screen for several seconds, then an error message "can't boot on CD/DVD", black screen a few seconds, then windows login screen. Tried winsat command and few hours later, it's not over! And shutdown is not possible, even with system_powerdown in qemu command line!
Next boot was fine. Winsat executes with errors like "ERROR: no adapter for device 0" but it progress.
WEI is 8.1. (and SSD is detected! :mrgreen: )

So, it seems to work. Let's say its stable, there's no problem under Windows: how can I have back kb&m under Linux without going under the desk to plug them on another usb?...

I have Synergy... but windows must be my server: I can have some lag on my linux when I use Windows for video, but not while gaming on Windows.

Multi-device kb & mouse are not an option for me: I'm too much in love with mechanical KB :oops:
powerhouse wrote: Some remarks and questions on your configuration:

a. I hate to see that a newer CPU such as the Intel 7700K is actually worse in terms of IOMMU and ACS compared to my old 3930K. Intel could have done better.
b. Why did you compile qemu 2.9? Was it needed to get VGA passthrough working? I run the standard qemu 2.5 that comes with Linux Mint and it works fine.
c. How is your WinBoot SSD configured? Is it an NTFS disk, a LVM disk, etc.? Did/does it hold your "bare metal" Windows installation?
If you pass through the WinBoot SSD's SATA controller, then WinBoot better be a native Windows disk with boot sector and everything. Else it won't work.
d. Did you try to configure the bridge using Network Manager, or did you use the bridge-utils tools?

Looks like you got a great gaming setup. Once the issues are sorted out, would you mind to post some benchmarks (see my previous 2-3 posts above)?
a. Before, I was having a i7-4970, and it was even worse!
b. With all my problems with USB, I've done some research and following google links I've come on the qemu project page. I've seen the 2.8, 2.9 and future 2.10, read the change log and found that 2.9 resolves USB 3 problems, which seems to be my case. So, let's build it 8) The first time, lsusb option was not active... It's nothing to say that it was worse than 2.5 :mrgreen:
c. WinBoot is GPT, entirely for Windows which automatically partitioned it at install. Just the virtio drivers installed during installation.
d. Network Manager as you describe.

And no problem for benchmarking my setup. Games are only a part of the use of this setup. Video are another one.
And I've a GTX 980 for Linux, as my goal is to play on Linux as long as games offer it for a part, and I' seen some problem with kodi when playing videos when Windows was running. I suppose that decoding video with the cpu while gaming on windows is a little to much.

Ah! And sorry if sometimes my english is weird, it's not my mother tongue, but I try to do my best and learn too. :roll:
Last edited by HPMint on Fri Sep 01, 2017 5:13 pm, edited 1 time in total.
Locked

Return to “Virtual Machines”