HOW-TO make dual-boot obsolete using kvm VGA passthrough

Questions about virtualization software
Forum rules
Before you post please read how to get help
powerhouse
Level 5
Level 5
Posts: 986
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Postby powerhouse » Tue Jul 11, 2017 3:25 am

diabolusss wrote:@powerhouse my startup script is:

Code: Select all

#!/bin/bash

vm_name=win10-mypc
vm_machine_type=q35
mem_size=4G
mem_pages=/dev/hugepages
pci_gpu=01:00.0
usb_mouse=15d9:0a4c
dir_os=/media/colt/_DATA/seabios.img
dir_bios=/usr/share/seabios/bios.bin
dir_cd_install=/media/colt/_DATA/win8.1_x86-x64.iso
dir_cd_drivers=/media/colt/_DATA/virtio-win-0.1.139.iso
boot_order=dc
net_card_mac=00:16:3e:00:01:01

if ps -A | grep -q $vm_name; then
   echo "$vm_name is already running." &
   exit 1
fi

qemu-system-x86_64 \
-enable-kvm \
-M $vm_machine_type \
-m $mem_size \
-cpu host,kvm=off \
-smp 6,sockets=1,cores=3,threads=2 \
-bios $dir_bios \
-serial none \
-parallel none \
-vga none \
-device vfio-pci,host=$pci_gpu,x-vga=on \
-rtc clock=host,base=localtime \
  -device virtio-scsi-pci,id=scsi \
  -drive id=disk0,if=virtio,cache=none,format=raw,file=$dir_os \
  -drive file=$dir_cd_install,id=isocd,format=raw,if=none -device scsi-cd,drive=isocd \
  -drive file=$dir_cd_drivers,id=virtiocd,format=raw,if=none -device ide-cd,bus=ide.1,drive=virtiocd \
  -boot order=$boot_order \
  -boot menu=on \
  -netdev type=tap,id=net0,ifname=tap0,vhost=on \
  -device virtio-net-pci,netdev=net0,mac=$net_card_mac

exit 0

Change

Code: Select all

-device vfio-pci,host=$pci_gpu,x-vga=on \

to

Code: Select all

-device vfio-pci,host=$pci_gpu,multifunction=on \
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Gigabyte GTX 970 + PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB + Samsung EVO 850 250GB SSD + 5 HDD, Corsair 500R case, SeaSonic 660W Gold X PS

powerhouse
Level 5
Level 5
Posts: 986
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Postby powerhouse » Tue Jul 11, 2017 7:13 am

If you are installing on an AMD Ryzen platform, you need to do the following (see also level1techs.com):

  1. Install the AGESA 1.0.0.6 BIOS update for your specific motherboard (see also http://www.anandtech.com/show/11447/amd-announces-ryzen-agesa-1006-update).
  2. Get the latest kernel. An easy way is using ukuu, see Ukuu – Simple Tool to Install the Latest Kernels in Ubuntu / Linux Mint.
  3. Get a version of Qemu > 2.6 by adding this repository:

    Code: Select all

    sudo add-apt-repository ppa:jacob/virtualisation

    and install / reinstall qemu.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Gigabyte GTX 970 + PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB + Samsung EVO 850 250GB SSD + 5 HDD, Corsair 500R case, SeaSonic 660W Gold X PS

diabolusss
Level 1
Level 1
Posts: 10
Joined: Sat Jun 24, 2017 7:24 am

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Postby diabolusss » Tue Jul 11, 2017 3:40 pm

@powerhouse
I changed x-vga to multifunction flag and new error appeared.

Qemu hangs on boot on message:

Code: Select all

Press F12 for boot menu


with error in linux console:

Code: Select all

dma: command 78 not supported
dma: command 78 not supported


No errors in dmesg.

Code: Select all

[ 1387.280307] vfio-pci 0000:01:00.0: enabling device (0400 -> 0403)
[ 1388.324490] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x1e@0x258
[ 1388.324509] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900

powerhouse
Level 5
Level 5
Posts: 986
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Postby powerhouse » Wed Jul 12, 2017 4:03 pm

diabolusss wrote:@powerhouse
I changed x-vga to multifunction flag and new error appeared.

Qemu hangs on boot on message:

Code: Select all

Press F12 for boot menu


with error in linux console:

Code: Select all

dma: command 78 not supported
dma: command 78 not supported


No errors in dmesg.

Code: Select all

[ 1387.280307] vfio-pci 0000:01:00.0: enabling device (0400 -> 0403)
[ 1388.324490] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x1e@0x258
[ 1388.324509] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900

EDIT: DON'T add the corresponding audio device:

Code: Select all

  -device vfio-pci,host=01:00.1 \

(I leave this in place so you see the change.)

You do not have an audio device on the graphics card.

Here some more suggestions, in the order I provide:

1. Remove the "multifunction..." option. Your device is not a multifunction device.
2. Add the following after -vga none \:

Code: Select all

-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \

3. Add/change the following:

Code: Select all

-device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,x-vga=on


The above is for Seabios. Sorry for my confusion on the x-vga option but it seems to be necessary here.

Again, if you can find a suitable UEFI BIOS for your graphics card, that would be the preferred option.

If the above doesn't work, you'll need to apply the VGA arbiter patch.
Last edited by powerhouse on Wed Jul 12, 2017 4:54 pm, edited 1 time in total.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Gigabyte GTX 970 + PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB + Samsung EVO 850 250GB SSD + 5 HDD, Corsair 500R case, SeaSonic 660W Gold X PS

diabolusss
Level 1
Level 1
Posts: 10
Joined: Sat Jun 24, 2017 7:24 am

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Postby diabolusss » Wed Jul 12, 2017 4:26 pm


Audio device is in another group. Do i really need it?

UPDATE. Ok, i'll try that.

Code: Select all

lspci -nn
00:00.0 Host bridge [0600]: Intel Corporation Sky Lake Host Bridge/DRAM Registers [8086:1910] (rev 07)
00:01.0 PCI bridge [0604]: Intel Corporation Sky Lake PCIe Controller (x16) [8086:1901] (rev 07)
00:02.0 VGA compatible controller [0300]: Intel Corporation Skylake Integrated Graphics [8086:191b] (rev 06)
00:04.0 Signal processing controller [1180]: Intel Corporation Skylake Processor Thermal Subsystem [8086:1903] (rev 07)
00:14.0 USB controller [0c03]: Intel Corporation Sunrise Point-H USB 3.0 xHCI Controller [8086:a12f] (rev 31)
00:14.2 Signal processing controller [1180]: Intel Corporation Sunrise Point-H Thermal subsystem [8086:a131] (rev 31)
00:16.0 Communication controller [0780]: Intel Corporation Sunrise Point-H CSME HECI #1 [8086:a13a] (rev 31)
00:17.0 RAID bus controller [0104]: Intel Corporation 82801 Mobile SATA Controller [RAID mode] [8086:282a] (rev 31)
00:1c.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI Express Root Port #1 [8086:a110] (rev f1)
00:1c.4 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI Express Root Port #5 [8086:a114] (rev f1)
00:1c.5 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI Express Root Port #6 [8086:a115] (rev f1)
00:1c.6 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI Express Root Port #7 [8086:a116] (rev f1)
00:1d.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI Express Root Port #9 [8086:a118] (rev f1)
00:1f.0 ISA bridge [0601]: Intel Corporation Sunrise Point-H LPC Controller [8086:a14e] (rev 31)
00:1f.2 Memory controller [0580]: Intel Corporation Sunrise Point-H PMC [8086:a121] (rev 31)
00:1f.3 Audio device [0403]: Intel Corporation Sunrise Point-H HD Audio [8086:a170] (rev 31)
00:1f.4 SMBus [0c05]: Intel Corporation Sunrise Point-H SMBus [8086:a123] (rev 31)
01:00.0 3D controller [0302]: NVIDIA Corporation GM204M [GeForce GTX 970M] [10de:13d8] (rev a1)
3b:00.0 Ethernet controller [0200]: Qualcomm Atheros Killer E2400 Gigabit Ethernet Controller [1969:e0a1] (rev 10)
3c:00.0 Network controller [0280]: Qualcomm Atheros QCA6174 802.11ac Wireless Network Adapter [168c:003e] (rev 32)
3d:00.0 Unassigned class [ff00]: Realtek Semiconductor Co., Ltd. RTS5227 PCI Express Card Reader [10ec:5227] (rev 01)
3e:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller [144d:a802] (rev 01)


Code: Select all

for a in /sys/kernel/iommu_groups/*; do find $a -type l; done
/sys/kernel/iommu_groups/0/devices/0000:00:00.0
/sys/kernel/iommu_groups/1/devices/0000:00:01.0
/sys/kernel/iommu_groups/1/devices/0000:01:00.0
/sys/kernel/iommu_groups/10/devices/0000:00:1c.6
/sys/kernel/iommu_groups/11/devices/0000:00:1d.0
/sys/kernel/iommu_groups/12/devices/0000:00:1f.3
/sys/kernel/iommu_groups/12/devices/0000:00:1f.4
/sys/kernel/iommu_groups/12/devices/0000:00:1f.2
/sys/kernel/iommu_groups/12/devices/0000:00:1f.0
/sys/kernel/iommu_groups/13/devices/0000:3b:00.0
/sys/kernel/iommu_groups/14/devices/0000:3c:00.0
/sys/kernel/iommu_groups/15/devices/0000:3d:00.0
/sys/kernel/iommu_groups/16/devices/0000:3e:00.0
/sys/kernel/iommu_groups/2/devices/0000:00:02.0
/sys/kernel/iommu_groups/3/devices/0000:00:04.0
/sys/kernel/iommu_groups/4/devices/0000:00:14.2
/sys/kernel/iommu_groups/4/devices/0000:00:14.0
/sys/kernel/iommu_groups/5/devices/0000:00:16.0
/sys/kernel/iommu_groups/6/devices/0000:00:17.0
/sys/kernel/iommu_groups/7/devices/0000:00:1c.0
/sys/kernel/iommu_groups/8/devices/0000:00:1c.4
/sys/kernel/iommu_groups/9/devices/0000:00:1c.5
Last edited by diabolusss on Thu Jul 13, 2017 3:39 am, edited 1 time in total.

powerhouse
Level 5
Level 5
Posts: 986
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Postby powerhouse » Wed Jul 12, 2017 4:56 pm

diabolusss wrote:
powerhouse wrote:You also need to pass through the corresponding audio device:

Code: Select all

  -device vfio-pci,host=01:00.1 \


Audio device is in another group. Do i really need it?

No - see my post above yours. (I should have deleted it.)
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Gigabyte GTX 970 + PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB + Samsung EVO 850 250GB SSD + 5 HDD, Corsair 500R case, SeaSonic 660W Gold X PS

powerhouse
Level 5
Level 5
Posts: 986
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Postby powerhouse » Wed Jul 12, 2017 5:04 pm

@diabolusss:

Add the following to your /etc/defaul/grub on the same line with iommu=on:

Code: Select all

intel_iommu=igfx_off


and

Code: Select all

sudo update-grub


Reboot to see if it makes a difference.

See also https://forum.level1techs.com/t/gta-v-on-linux-skylake-build-hardware-vm-passthrough/87440 for more on Skylake. (Note they use ArchLinux, which has patches integrated, unlike Linux Mint.)
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Gigabyte GTX 970 + PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB + Samsung EVO 850 250GB SSD + 5 HDD, Corsair 500R case, SeaSonic 660W Gold X PS

diabolusss
Level 1
Level 1
Posts: 10
Joined: Sat Jun 24, 2017 7:24 am

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Postby diabolusss » Thu Jul 13, 2017 3:52 pm

@powerhouse
I tried your suggestions, but none of them helped. I suppose there is a problem with my patched kernel, but i don't know how to check if vgaarb is actually working as i still receive error when x-vga option is enabled.

Code: Select all

qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,x-vga=on: vfio error: 0000:01:00.0: failed getting region info for VGA region index 8: Invalid argument
device does not support requested feature x-vga


I mention steps i used to build kernel here

Now i'm studying link you have provided about hvm with Skylake....

Code: Select all

dmesg
[    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-4.10.17--i915vgaarb-kernel root=UUID=85b070c7-1807-426e-8f2b-de59d732e051 ro modprobe.blacklist=nvidia quiet splash intel_iommu=on intel_iommu=igfx_off i915.enable_hd_vgaarb=1 vt.handoff=7
 [    0.000000] DMAR: IOMMU enabled
[    0.000000] DMAR: Disable GFX device mapping
[    0.049398] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.049398] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[    0.049398] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit.
[    0.049399] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the BIOS setting.
[    0.050778] DMAR-IR: Enabled IRQ remapping in xapic mode
[    0.302931] pci 0000:00:02.0: vgaarb: setting as boot VGA device
[    0.302932] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none
[    0.302935] pci 0000:00:02.0: vgaarb: bridge control possible
[    0.302935] vgaarb: loaded
[    0.842640] DMAR: No ATSR found
[    0.843168] DMAR: dmar1: Using Queued invalidation
[    0.843175] DMAR: Setting RMRR:
[    0.843207] DMAR: Setting identity map for device 0000:00:14.0 [0x2ee00000 - 0x2ee1ffff]
[    0.843212] DMAR: Prepare 0-16MiB unity mapping for LPC
[    0.843236] DMAR: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[    0.843240] DMAR: Intel(R) Virtualization Technology for Directed I/O
[    1.040209] i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io:owns=io+mem
[    1.063101] ACPI: Video Device [GFX0] (multi-head: yes  rom: no  post: no)
[    1.063380] input: Video Bus as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A08:00/LNXVIDEO:00/input/input6
[    1.063504] ACPI: Video Device [PEGP] (multi-head: no  rom: yes  post: no)
[    1.063540] input: Video Bus as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A08:00/device:12/LNXVIDEO:01/input/input7
[    1.063632] [drm] Initialized i915 1.6.0 20161121 for 0000:00:02.0 on minor 0
[    1.084784] i915 0000:00:02.0: fb0: inteldrmfb frame buffer device
[    2.000319] VFIO - User Level meta-driver version: 0.3
[    2.028495] vfio_pci: add [10de:13d8[ffff:ffff]] class 0x000000/00000000

powerhouse
Level 5
Level 5
Posts: 986
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Postby powerhouse » Sun Jul 16, 2017 7:48 am

Network configuration using the Network Manager

The Network Manager supplied by Ubuntu and Linux Mint to configure the network has made some progress and is now very usable to define a network bridge to connect our VM. Instead of using brctl and the /etc/network/interfaces configuration file, you can use the Network Manager to accomplish the same task.

Unfortunately the Linux Mint forum software puts up some obstacles (maximum 3 images), so I've decided to post the instructions on my own website (under construction). See link below:

Define a network bridge using Ubuntu’s / Linux Mint’s Network Manager application
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Gigabyte GTX 970 + PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB + Samsung EVO 850 250GB SSD + 5 HDD, Corsair 500R case, SeaSonic 660W Gold X PS

powerhouse
Level 5
Level 5
Posts: 986
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Postby powerhouse » Sun Jul 16, 2017 3:52 pm

diabolusss wrote:@powerhouse
I tried your suggestions, but none of them helped. I suppose there is a problem with my patched kernel, but i don't know how to check if vgaarb is actually working as i still receive error when x-vga option is enabled.

Code: Select all

qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,x-vga=on: vfio error: 0000:01:00.0: failed getting region info for VGA region index 8: Invalid argument
device does not support requested feature x-vga


I mention steps i used to build kernel here

Now i'm studying link you have provided about hvm with Skylake....

Code: Select all

dmesg
[    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-4.10.17--i915vgaarb-kernel root=UUID=85b070c7-1807-426e-8f2b-de59d732e051 ro modprobe.blacklist=nvidia quiet splash intel_iommu=on intel_iommu=igfx_off i915.enable_hd_vgaarb=1 vt.handoff=7
 [    0.000000] DMAR: IOMMU enabled
[    0.000000] DMAR: Disable GFX device mapping
[    0.049398] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.049398] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[    0.049398] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit.
[    0.049399] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the BIOS setting.
[    0.050778] DMAR-IR: Enabled IRQ remapping in xapic mode
[    0.302931] pci 0000:00:02.0: vgaarb: setting as boot VGA device
[    0.302932] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none
[    0.302935] pci 0000:00:02.0: vgaarb: bridge control possible
[    0.302935] vgaarb: loaded
[    0.842640] DMAR: No ATSR found
[    0.843168] DMAR: dmar1: Using Queued invalidation
[    0.843175] DMAR: Setting RMRR:
[    0.843207] DMAR: Setting identity map for device 0000:00:14.0 [0x2ee00000 - 0x2ee1ffff]
[    0.843212] DMAR: Prepare 0-16MiB unity mapping for LPC
[    0.843236] DMAR: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[    0.843240] DMAR: Intel(R) Virtualization Technology for Directed I/O
[    1.040209] i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io:owns=io+mem
[    1.063101] ACPI: Video Device [GFX0] (multi-head: yes  rom: no  post: no)
[    1.063380] input: Video Bus as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A08:00/LNXVIDEO:00/input/input6
[    1.063504] ACPI: Video Device [PEGP] (multi-head: no  rom: yes  post: no)
[    1.063540] input: Video Bus as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A08:00/device:12/LNXVIDEO:01/input/input7
[    1.063632] [drm] Initialized i915 1.6.0 20161121 for 0000:00:02.0 on minor 0
[    1.084784] i915 0000:00:02.0: fb0: inteldrmfb frame buffer device
[    2.000319] VFIO - User Level meta-driver version: 0.3
[    2.028495] vfio_pci: add [10de:13d8[ffff:ffff]] class 0x000000/00000000

It seems that your laptop uses Optimus and the solution to that - if any - is not an easy and straightforward one. Have a look at this thread that delves into the obstacles that Optimus throws in your way: https://www.reddit.com/r/VFIO/comments/5fx8jq/gpu_passthrough_incorrect_subsystem_in_windows/.

I was hoping that perhaps newer kernels could help solve this. But as you can see, this is no simple matter. I do not have any hands-on VGA passthrough experience with laptops, let alone Optimus boards.

Should you decide to continue with your trials, keep us posted here, whether you succeed or not.

EDIT: Boot your laptop and enter the BIOS setup. Look if you can change the graphics card to boot with - IGD or Nvidia. Select IGD, if it's not selected already.

See also https://www.redhat.com/archives/vfio-users/2017-April/msg00021.html

Again, I don't want you to get up your hopes. It just might be worth checking.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Gigabyte GTX 970 + PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB + Samsung EVO 850 250GB SSD + 5 HDD, Corsair 500R case, SeaSonic 660W Gold X PS

powerhouse
Level 5
Level 5
Posts: 986
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Postby powerhouse » Mon Jul 17, 2017 4:02 pm

@diabolusss: You may have already seen this, but just to make sure: https://bugs.launchpad.net/qemu/+bug/1680679.

On page 14 in this thread bash64 reports that he succeeded passing through an Nvidia GTX 880M card on an Optimus laptop, using Seabios. The thread above also contains his VM start script. You may compare it with yours and see if that can work. Again, as Alex replied to the bug report, it's a lucky fluke if it works:
I suspect the 'M' in GTX880M is the biggest contributor to the Code 43. The fact you can get it to work once per host boot on SeaBIOS is a fluke. If you can get custom ROMs, you might try playing with the vfio-pci x-pci-device-id option to masquerade as a discrete card, maybe that would avoid mobile code in the NVIDIA driver that would expect Optimus. Obviously do so at your own risk.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Gigabyte GTX 970 + PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB + Samsung EVO 850 250GB SSD + 5 HDD, Corsair 500R case, SeaSonic 660W Gold X PS

powerhouse
Level 5
Level 5
Posts: 986
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Postby powerhouse » Thu Jul 20, 2017 6:46 pm

I've reworked my how-to, made some updates, but because of the limitations here on this forum, I decided to post the updated version on my own website here: https://heiko-sieger.info/running-windows-10-on-linux-using-kvm-with-vga-passthrough/.
I will try to update the how-to here too.

My next project is to offer a how-to using the virt-manager GUI.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Gigabyte GTX 970 + PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB + Samsung EVO 850 250GB SSD + 5 HDD, Corsair 500R case, SeaSonic 660W Gold X PS

talos_d
Level 1
Level 1
Posts: 6
Joined: Thu May 11, 2017 1:09 pm

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Postby talos_d » Sat Jul 22, 2017 7:54 pm

Thank you for creating this tutorial,I am new to linux but was able to make it work with a low spec set up(cpu-g4560,gtx960 for passthrough and intel igfx for host).
My next goal is to make it work within in a windowed mode so I wont have to switch inputs physically on my monitor,I found this video- https://www.youtube.com/watch?v=d3HL9x0CEtY on youtube but I wasn't
able to figure out how he made it work.

as far as I can tell he used the vga qxl-sdl command to somehow make a virtual monitor and used mirroring monitors to mirror what is on the monitor output to the virtual monitor,he also used obs but I dont quite understand for what.
I you have any insight I would really appreciate it.
Thanks again!

powerhouse
Level 5
Level 5
Posts: 986
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Postby powerhouse » Sun Jul 23, 2017 11:43 pm

talos_d wrote:Thank you for creating this tutorial,I am new to linux but was able to make it work with a low spec set up(cpu-g4560,gtx960 for passthrough and intel igfx for host).
My next goal is to make it work within in a windowed mode so I wont have to switch inputs physically on my monitor,I found this video- https://www.youtube.com/watch?v=d3HL9x0CEtY on youtube but I wasn't
able to figure out how he made it work.

as far as I can tell he used the vga qxl-sdl command to somehow make a virtual monitor and used mirroring monitors to mirror what is on the monitor output to the virtual monitor,he also used obs but I dont quite understand for what.
I you have any insight I would really appreciate it.
Thanks again!

That is good to hear that it worked. Would you be willing to share your script?

I saw the video but don't understand Russian. It looks to me that he is running a laptop, but that is only a guess looking at the qemu command.

I have no experience with screen mirroring, sorry. Your guess is as good as mine. His use of acpitable is something I haven't explored yet.

Hope you find a way to mirror your screen.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Gigabyte GTX 970 + PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB + Samsung EVO 850 250GB SSD + 5 HDD, Corsair 500R case, SeaSonic 660W Gold X PS

powerhouse
Level 5
Level 5
Posts: 986
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Postby powerhouse » Sun Jul 23, 2017 11:50 pm

So I was playing with virt-manager following another tutorial to see how that works. Getting passthrough working is easy, but performance totally sucks!

I can see some of the benefits of virt-manager and libvirt in theory, but in practice it makes everything just a lot more complicated. Before judging it, I need to become more familiar. I will certainly give it another try or two.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Gigabyte GTX 970 + PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB + Samsung EVO 850 250GB SSD + 5 HDD, Corsair 500R case, SeaSonic 660W Gold X PS

talos_d
Level 1
Level 1
Posts: 6
Joined: Thu May 11, 2017 1:09 pm

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Postby talos_d » Mon Jul 24, 2017 10:20 pm

powerhouse wrote:
talos_d wrote:Thank you for creating this tutorial,I am new to linux but was able to make it work with a low spec set up(cpu-g4560,gtx960 for passthrough and intel igfx for host).
My next goal is to make it work within in a windowed mode so I wont have to switch inputs physically on my monitor,I found this video- https://www.youtube.com/watch?v=d3HL9x0CEtY on youtube but I wasn't
able to figure out how he made it work.

as far as I can tell he used the vga qxl-sdl command to somehow make a virtual monitor and used mirroring monitors to mirror what is on the monitor output to the virtual monitor,he also used obs but I dont quite understand for what.
I you have any insight I would really appreciate it.
Thanks again!

That is good to hear that it worked. Would you be willing to share your script?

I saw the video but don't understand Russian. It looks to me that he is running a laptop, but that is only a guess looking at the qemu command.

I have no experience with screen mirroring, sorry. Your guess is as good as mine. His use of acpitable is something I haven't explored yet.

Hope you find a way to mirror your screen.


The script is preety much exactly yours except I removed the last 4 lines after os installation:

Code: Select all

#!/bin/bash

#!/bin/bash
 
vmname="windows10vm"
 
if ps -A | grep -q $vmname; then
   echo "$vmname is already running." &
   exit 1
 
else
 
# use pulseaudio
export QEMU_AUDIO_DRV=pa
export QEMU_PA_SAMPLES=8192
export QEMU_AUDIO_TIMER_PERIOD=99
export QEMU_PA_SERVER=/run/user/1000/pulse/native
 
cp /usr/share/OVMF/OVMF_VARS.fd /tmp/my_vars.fd
 
qemu-system-x86_64 \
  -name $vmname,process=$vmname \
  -machine type=q35,accel=kvm \
  -cpu host,kvm=off \
  -smp 4,sockets=1,cores=2,threads=2 \
  -enable-kvm \
  -m 12G \
  -mem-path /dev/hugepages \
  -mem-prealloc \
  -balloon none \
  -rtc clock=host,base=localtime \
  -vga none \
  -nographic \
  -serial none \
  -parallel none \
  -soundhw hda \
  -usb -usbdevice host:056a:00b4 -usbdevice host:046d:c52e -usbdevice host:0781:556b \
  -device vfio-pci,host=01:00.0,multifunction=on \
  -device vfio-pci,host=01:00.1 \
  -drive if=pflash,format=raw,readonly,file=/usr/share/OVMF/OVMF_CODE.fd \
  -drive if=pflash,format=raw,file=/tmp/my_vars.fd \
  -boot order=dc \
  -device virtio-scsi-pci,id=scsi \
  -drive id=disk0,if=virtio,cache=none,format=raw,file=/media/user/win.img \
  -drive file=/home/user/ISOs/Win10_1703_English_x64.iso,id=isocd,format=raw,if=none -device scsi-cd,drive=isocd \
  -drive file=/home/user/Downloads/virtio-win-0.1.126.iso,id=virtiocd,format=raw,if=none -device ide-cd,bus=ide.1,drive=virtiocd \
 
  -netdev type=tap,id=net0,ifname=tap0,vhost=on \
  -device virtio-net-pci,netdev=net0,mac=00:16:3e:00:01:0 \
   
   exit 0
fi


Did a benchmark with 3dmark firestrike and there is very little overhead,photoshop and blender(which is my main reason for doing this) are working smoothly.
I also figured out what the russian dude did in my previous post and was able to run the vm in windowed mode with the passthroughed gpu...well kind of,its seems windows is a little laggy on the qxl fake screen.
If anyone is interested here is how you can make it work:after completing the gpu passthrough with the above script I replaced the lines-

Code: Select all

-vga none \
-nographic \


with

Code: Select all

-vga qxl -sdl \


This will load a fake monitor into your vm and qemu should start in windowed mode using the qxl adapter with the physical output of your monitor
being the second monitor/graphics adapter in the vm.
As you start the vm under device manager you will see two graphics adapters-the passthrouged card and microsoft basic adapter.
I updated the the latter with these drivers:http://people.redhat.com/vrozenfe/qxlwddm/ (version 0.11 or 0.12 worked for me) which
changed the microsoft basic adapter to Redhat qxl controller.
After that you need to set the monitor with the real card as your main monitor under display settings and choose extend monitors
under the same panel.I was able to run photoshop in the windowed display within linux with a decent performance but there seems to be a small mouse lag which the qxl
display has even running only by itself and is maybe caused by windows 10 not being fully compatible with the windows 8 qxl driver.

Mirroring displays issues:a better option than extending the screens would be to duplicate them using the project option located on the lower right in the taskbar panel
but that did not work for me for whatever reason.In the video I shared he solved it by downloading obs(a streaming program) and streamed a preview in fullscreen
of the real monitor with encoding disabled within the qxl window-this worked for me but with an unbearable lag(might be that my cpu is not good enough).

I might try to check and see how this works on windows 8,1 which the qxl driver version is designed for,it was really cool to see windows within linux and being able to
mouse grab go full screen and then exit without switching outputs,this is probably not a solution for gaming but its adequate for photoshop/programs that require light 3d acceleration.

powerhouse
Level 5
Level 5
Posts: 986
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Postby powerhouse » Tue Jul 25, 2017 12:08 am

@talos_d: Thanks sharing your setup and experience. By the way, what is your hardware?

I'm using kvm with VGA passthrough for pretty much the same purpose, except that I'm using Lightroom. Processing photos is the only reason I keep Windows around, else I would have dumped it long ago.

In my setup I use the GPU acceleration to improve Lightroom performance. I have a high-end monitor that allows me to write a 14-bit LUT directly to the screen using the DVI interface. This means when I start Windows, it uploads the LUT to the monitor, correcting the color output. When i do screen calibration, I connect my color calibration device to a passed through USB port. For all this to work the drivers and software within Windows need hardware access to the graphics card, USB, etc., so passthrough is essential.

In your setup you use the passed through graphics card for video acceleration, but mirror the output to a virtual screen (window) for convenience. The graphics performance should be nearly the same as bare metal, but the mirroring introduces some lag, whether lag in dispaying the frames , or when using the mouse. Your windowed VM runs on a Linux desktop, meaning that the communication is going through the network bridge you set up. You need to see if there is anything you can do to improve the network speed as this will have a direct impact on lag. The bridged network connection should perform pretty good, but I haven't invested in tweaking this further. In your use case this might be worth exploring.

A good graphics card test is Unigine Heaven - if that runs about as fast as on bare metal, you know you got it right.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Gigabyte GTX 970 + PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB + Samsung EVO 850 250GB SSD + 5 HDD, Corsair 500R case, SeaSonic 660W Gold X PS

powerhouse
Level 5
Level 5
Posts: 986
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Postby powerhouse » Tue Jul 25, 2017 11:25 am

One more note on the start script:

You can remove

Code: Select all

  -enable-kvm \

The above option has been replaced by

Code: Select all

-machine type=q35,accel=kvm \
in particular the "accel=kvm" part.

Defining the CD-ROM drives can also be simplified using:

Code: Select all

-drive file=/home/user/ISOs/Win10_1511_English_x64.iso,index=3,media=cdrom \
-drive file=/home/user/Downloads/virtio-win-0.1.140.iso,index=4,media=cdrom \


I'll try to fix that today in the tutorial.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Gigabyte GTX 970 + PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB + Samsung EVO 850 250GB SSD + 5 HDD, Corsair 500R case, SeaSonic 660W Gold X PS

talos_d
Level 1
Level 1
Posts: 6
Joined: Thu May 11, 2017 1:09 pm

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Postby talos_d » Tue Jul 25, 2017 9:26 pm

powerhouse wrote:@talos_d: Thanks sharing your setup and experience. By the way, what is your hardware?

I'm using kvm with VGA passthrough for pretty much the same purpose, except that I'm using Lightroom. Processing photos is the only reason I keep Windows around, else I would have dumped it long ago.

In my setup I use the GPU acceleration to improve Lightroom performance. I have a high-end monitor that allows me to write a 14-bit LUT directly to the screen using the DVI interface. This means when I start Windows, it uploads the LUT to the monitor, correcting the color output. When i do screen calibration, I connect my color calibration device to a passed through USB port. For all this to work the drivers and software within Windows need hardware access to the graphics card, USB, etc., so passthrough is essential.

In your setup you use the passed through graphics card for video acceleration, but mirror the output to a virtual screen (window) for convenience. The graphics performance should be nearly the same as bare metal, but the mirroring introduces some lag, whether lag in dispaying the frames , or when using the mouse. Your windowed VM runs on a Linux desktop, meaning that the communication is going through the network bridge you set up. You need to see if there is anything you can do to improve the network speed as this will have a direct impact on lag. The bridged network connection should perform pretty good, but I haven't invested in tweaking this further. In your use case this might be worth exploring.

A good graphics card test is Unigine Heaven - if that runs about as fast as on bare metal, you know you got it right.


My hardware is:

Desktop: Cinnamon 3.4.4 (Gtk 3.18.9-1ubuntu3.3) dm: lightdm Distro: Linux Mint 18.2 Sonya
Machine: Mobo: Gigabyte model: H270M-DS3H
CPU: Dual core Intel Pentium G4560
Memory:16 gb
Graphics: Card-1: Intel Device 5902
Card-2: NVIDIA GM206 [GeForce GTX 960]
Network: Card: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
Drives:
KINGSTON 120.0GB SSD
WESTREN DIGITAL 1TB

Thank you for for the advice on simplifying the script and speeding up the network bridge,I really have not looked into it yet.
I did the UNIGINE heaven benchmark and it turned out quite useful as I needed a benchmark that does
not run fullscreen(I had to keep the qxl fake monitor as my secondary in extended mode and that makes
running fullscreen applications not easy)the results show that the system does use my gtx 960 but there is
some performance drop-2940 score on the basic preset versus 2530 on the fake monitor.For me a 14 percent
difference is good enough to try and make this work(maybe with some optimization I can reduce the overhead)

Mouse lag:Well I figured out that the mouse Lag was definitely being caused by the qxl drivers-using the
0.16 drivers on this page http://depot.flexvdi.com/guest-tools/ made the mouse lag go away completely.
unfortunately it introduced a chaotic jumpy mouse behavior thats make the system close to unusable.
also while running unigine benchmark its clear that a subtle stutter persists.
its seems that windows 10 just does not like the qxl driver.Though on this page-
https://bugs.freedesktop.org/show_bug.cgi?id=91219#c26 the above mentioned driver
seemed to work for others without any buggy behavior so it might be something with my particular set up.

edit:it seems these drivers use spice which might be the root cause of the baggy/laggy behavior,
wonder if I can set it up without virt manager.

borg101
Level 3
Level 3
Posts: 112
Joined: Tue Oct 25, 2016 9:00 pm

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Postby borg101 » Wed Jul 26, 2017 10:49 pm

Is this possible to do with one gpu? Or could I use my mobo/cpu's gpu and my discreet gpu?
Mobo: ASRock model: Z97E-ITX | CPU: Quad core Intel Core i5-4690 | GPU: NVIDIA GM206 [GeForce GTX 960] | RAM: 16GB DDR3 | SSD/HDD: Total Size: 8251.6GB | OS: Mint 18.2 (Cinnamon)


Return to “Virtualization”