HOW-TO make dual-boot obsolete using kvm VGA passthrough

Questions about virtualization software
Forum rules
Before you post please read how to get help
bash64
Level 2
Level 2
Posts: 66
Joined: Mon Mar 23, 2015 12:19 am

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by bash64 » Sun Jan 21, 2018 3:40 pm

Hey,
I have lookingglass working fully.
Unfortunately, the ivshmem driver only works with windows 8(latest driver) and 10.
I am using windows 10, but w10 causes massive lag with pulseaudio.
I have windows7 and pulseaudio working perfectly so i am waiting for the windows 7 lookinglass host and driver.

You also have to switch your games to windowed-mode.
You cannot see them using lookingglass if they go full screen.

So, it does work.
If the game is in window mode you can also on the linux side enlarge to full screen mode.

What this means for me is that I use qemu to play my dx11 games but it requires an external monitor be hooked up to my laptop.
With lookingglass I no longer need the external monitor and do not have to take it with me to computer meetings.


just a few tips:
1) do not run the ivshmem-server like the website says to. you will be using ivshmem-plain instead. see below line.

2) you must use qemu 2.7.0 or higher with spice enabled and pulseaudio enabled
mint 18 uses qemu 2.5. this wont do.
after compiling my own qemu (2.11.50) and using the lines below I was FINALLY able to install the ivshmem driver they give you. but not until then.

here are the additional lines i use:
-object memory-backend-file,id=ivshmem,share=on,mem-path=/dev/shm/looking-glass,size=32M \
-device ivshmem-plain,memdev=ivshmem \
-spice port=5900,addr=127.0.0.1,disable-ticketing \

if you compile your own qemu (like i do) you need to tell ./configure some things
>./configure --enable-spice --audio-drv-list=pa
>make
>make install (i dont do this, i dont want to give my package manager heartburn, just give the full path to the compiled copy in your script).

if ./configure complains that you dont have the spice files installed then you need to install:
>sudo apt install libspice-protocol-dev libspice-server-dev


3) make sure you have the linux client downloaded and working. they changed it so redownload and compile it if need be. they now require libconfig-dev and libva-dev in addition to the existing dependencies.

4) for some reason i had to create my own /dev/shm/lookinglass file. if you have to here is how:
sudo touch /dev/shm/looking-glass
sudo chown {user}:kvm /dev/shm/looking-glass
sudo chmod 660 /dev/shm/looking-glass

replace user with the name of your login id, such as roger. mine would be chown roger:kvm /dev/shm/looking-glass

5) play video games using windowed mode.

driz
Level 1
Level 1
Posts: 22
Joined: Mon May 12, 2014 3:38 pm

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by driz » Mon Jan 29, 2018 9:21 pm

Howdy all, thought i'd post some horrible news. After the two win10 vms were running perfectly for over a year now, I was running updates on my other linux servers... some how i did pay attention and upgraded the distro of my kvm box. It pretty much ruined everything, now I get stuck at the windows logo during boot. I put minimal effort into troubleshooting, probably just going to rebuild from scratch and document the changes. As some of you may recall, for vm2 (the 4 yr old) I passed the full ssd through, while the 7yr old had a partition on an ssd passed through. I noticed gains in disk perf by passing the full disk, so i will ensure i do that for both vms this time. I'll try to post what happens following this rebuild.

Sucks that this happened, but that's what I get for not paying attention :(

side note: even if i try to boot from the win10 iso, after i "press any key to boot from disc" it freezes on the boot logo there too.. probably some driver passthru issue if I had to guess.

driz
Level 1
Level 1
Posts: 22
Joined: Mon May 12, 2014 3:38 pm

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by driz » Tue Jan 30, 2018 1:42 am

driz wrote:Howdy all, thought i'd post some horrible news. After the two win10 vms were running perfectly for over a year now, I was running updates on my other linux servers... some how i did pay attention and upgraded the distro of my kvm box. It pretty much ruined everything, now I get stuck at the windows logo during boot. I put minimal effort into troubleshooting, probably just going to rebuild from scratch and document the changes. As some of you may recall, for vm2 (the 4 yr old) I passed the full ssd through, while the 7yr old had a partition on an ssd passed through. I noticed gains in disk perf by passing the full disk, so i will ensure i do that for both vms this time. I'll try to post what happens following this rebuild.

Sucks that this happened, but that's what I get for not paying attention :(

side note: even if i try to boot from the win10 iso, after i "press any key to boot from disc" it freezes on the boot logo there too.. probably some driver passthru issue if I had to guess.
Alright, I just finished rebuilding the host and both guests successfully. To my surprise, the audio crackle that plagued my kids for a year is gone. I did make some changes, mostly because I didn't take details enough notes (even on my blog post about this very build...)

I haven't even written my new blog entry, so you guys get it first :D although, since I moved away from mint, maybe I shouldn't post here.

Media: http://releases.ubuntu.com/16.04/ubuntu ... so.torrent
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenial

I did not update the kernel which is still sitting at 4.4.0-87-generic and no passthrough issues.

I built both windows 10 guests concurrently, win10 install to 1st reboot, 2minutes 5 seconds. I did a direct passthrough of the ssd to each vm

Code: Select all

  -device virtio-scsi-pci,id=scsi \
   -drive file=/dev/sdd,id=disk0,if=none,format=raw,cache=none,aio=native \
  -device scsi-hd,drive=disk0 \
after initial boot, i installed netkvm drivers and let it use windows to update the AMD video drivers. I renamed the system and joined to my domain. I then shut down the system, removed the virtio and win10 install disk lines from my launch script and booted. Both systems blue screened, but after a restart booted fine. I believe this bsod is a known issue with the radeon drivers. Some of my notes indicated that it was alleviated if I used the actual radeon drivers, but my laziness and a reboot won out.

My WEI was 6.8 on both systems and i ran `winsat formal` concurrently on them.

As I mentioned before the crackle in my audio is completely gone! the kids will be so happy! My SSDs are still detected as thin provisioned drivers rather than SSD. I also went ahead and used the non-deprecated method of passing through usb devices. Just because during the epic failure, when qemu upgraded, the warning bothered me.

Code: Select all

-usb -device usb-host,vendorid=0x04f2,productid=0x0939 -device usb-host,vendorid=0x04f2,productid=0x0833 -device usb-host,vendorid=0x045e,productid=0x075d -device usb-host,hostbus=1,hostaddr=4 \
I'm still not using hugepages because I'm too lazy to do so. My ovmf was from the github repo and I used the latest virtio drivers ISO.

Here is my launch script:

Code: Select all

#!/bin/bash

configfile=/etc/vfio-pci1.cfg
vmname="kid1"

vfiobind() {
   dev="$1"
        vendor=$(cat /sys/bus/pci/devices/$dev/vendor)
        device=$(cat /sys/bus/pci/devices/$dev/device)
        if [ -e /sys/bus/pci/devices/$dev/driver ]; then
                echo $dev > /sys/bus/pci/devices/$dev/driver/unbind
        fi
        echo $vendor $device > /sys/bus/pci/drivers/vfio-pci/new_id

}


if ps -A | grep -q $vmname; then
   echo "$vmname is already running."
   exit 1

else

   cat $configfile | while read line;do
   echo $line | grep ^# >/dev/null 2>&1 && continue
      vfiobind $line
   done


if [ ! -f /tmp/system1.fd ]; then
  cp /srv/ovmf-x64/OVMF_VARS-pure-efi.fd /tmp/system1.fd
fi

/usr/bin/qemu-system-x86_64 \
  -name $vmname,process=$vmname \
  -serial none \
  -parallel none \
  -machine type=q35,accel=kvm \
  -cpu host,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time \
  -smp 4,sockets=1,cores=2,threads=2 \
  -enable-kvm \
  -m 6G \
  -mem-prealloc \
  -balloon none \
  -rtc clock=host,base=localtime \
  -vga none \
  -nographic \
  -usb -device usb-host,vendorid=0x040B,productid=0x2000 -device usb-host,vendorid=0x046d,productid=0xc01e -device usb-host,vendorid=0x046d,productid=0x0819 -device usb-host,hostbus=1,hostaddr=11 \
  -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
  -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,romfile=/srv/TV809MH.570 \
  -device vfio-pci,host=01:00.1,bus=pcie.0 \
  -drive if=pflash,format=raw,readonly,file=/srv/ovmf-x64/OVMF_CODE-pure-efi.fd \
  -drive if=pflash,format=raw,file=/tmp/system1.fd \
  -boot order=c \
  -object iothread,id=iothread0 \
  -device virtio-scsi-pci,id=scsi,iothread=iothread0 \
  -drive file=/dev/sdc1,id=disk0,if=none,format=raw,cache=directsync,aio=native \
  -device scsi-hd,drive=disk0,bootindex=1 \
  -netdev type=tap,id=net0,ifname=tap0,vhost=on \
  -device virtio-net-pci,netdev=net0,mac=52:54:00:ea:c8:8d

#  -mem-path /dev/hugepages1 \
#  -drive file=/srv/win10.iso,id=isocd,format=raw,if=none -device scsi-cd,drive=isocd,bootindex=1 \
#  -drive file=/srv/virtio-win.iso,id=virtiocd,format=raw,if=none -device ide-cd,bus=ide.1,drive=virtiocd \

exit 0
fi
my grub command line

Code: Select all

GRUB_CMDLINE_LINUX_DEFAULT="rd.driver.pre=vfio-pci quiet splash intel_iommu=on,igfx_off vfio_iommu_type1.allow_unsafe_interrupts=1 iommu=pt
I used to have kvm_intel.emulate_invalid_guest_state=0, but i couldn't remember why.. so now I don't.

I don't know if this will help anyone, but my kids game/homework on these guest vms daily (for over a year now) and the biggest complaint was the audio crackle. With that gone, a 6.8 score.. they're pretty happy. There's also enough resources left that I run my backup DNS (bind9) on the host.

If you wish to see my other configuration options, just look at my blog (link above) for the details. This install was the ubuntu LTS server edition, i ran update but did nothing to force a newer kernel than what came on it.

driz
Level 1
Level 1
Posts: 22
Joined: Mon May 12, 2014 3:38 pm

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by driz » Thu Feb 01, 2018 10:47 pm

I discovered today that the webcams don't work on the windows 10 vms :(

The errors showed this

Code: Select all

processing error - resetting ehci hc
emu-system-x86_64: libusb_release_interface: -4 [NO_DEVICE]
Attempt to fix #1

I've upgraded the kernel to 4.13.0-32-generic, but there was no change to the error
I'm not sure if this is typical, but I have noticed this

Code: Select all

[  156.431892] usb 1-4.2: reset low-speed USB device number 7 using xhci_hcd
[  157.219873] usb 1-4.1: reset low-speed USB device number 5 using xhci_hcd
[  157.883861] usb 1-4.2: reset low-speed USB device number 7 using xhci_hcd
[  158.507853] usb 1-4.2: reset low-speed USB device number 7 using xhci_hcd
[  158.883844] usb 1-4.3: reset high-speed USB device number 9 using xhci_hcd
[  159.551831] usb 1-4.1: reset low-speed USB device number 5 using xhci_hcd
[  160.163821] usb 1-4.1: reset low-speed USB device number 5 using xhci_hcd
[  160.535803] usb 1-4.4: reset full-speed USB device number 11 using xhci_hcd
[  162.227800] usb 1-4.4: reset full-speed USB device number 11 using xhci_hcd
[  162.455802] usb 1-4.4: reset full-speed USB device number 11 using xhci_hcd
[  182.975109] usb 1-3.3: reset low-speed USB device number 8 using xhci_hcd
[  183.531071] usb 1-3.4: reset low-speed USB device number 10 using xhci_hcd
[  183.919037] usb 1-3.2: reset high-speed USB device number 6 using xhci_hcd
[  184.111033] usb 1-3.1: reset full-speed USB device number 4 using xhci_hcd
[  184.526980] usb 1-3.2: reset high-speed USB device number 6 using xhci_hcd
[  184.766973] usb 1-3.2: reset high-speed USB device number 6 using xhci_hcd
[  185.118956] usb 1-3.1: reset full-speed USB device number 4 using xhci_hcd
[  185.486946] usb 1-3.4: reset low-speed USB device number 10 using xhci_hcd
[  185.946916] usb 1-3.1: reset full-speed USB device number 4 using xhci_hcd
[  186.446900] usb 1-3.4: reset low-speed USB device number 10 using xhci_hcd
[  187.398854] usb 1-3.3: reset low-speed USB device number 8 using xhci_hcd
[  188.106820] usb 1-3.3: reset low-speed USB device number 8 using xhci_hcd
What I thought was running perfectly is not. sad times. My next step, I suppose will be to try and upgrade qemu.. it's quite irritating that everything is working perfectly except the webcam...

Attempt to fix #2 Success
Using the PPA at https://launchpad.net/~jacob/+archive/u ... ter=xenial I upgraded my qemu from 2.5 to 2.6.2.

this resolved the issue, i was concerned the audio crackle would return, but it did not! Everything seems to be working perfectly now.

powerhouse
Level 6
Level 6
Posts: 1000
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by powerhouse » Wed Feb 07, 2018 2:10 pm

Hello bash64, thanks for sharing your hourglass solution. This seems like a great was to use a laptop without external screen. Much appreciate your input!
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Gigabyte GTX 970 + PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB + Samsung EVO 850 250GB SSD + 5 HDD, Corsair 500R case, SeaSonic 660W Gold X PS

powerhouse
Level 6
Level 6
Posts: 1000
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by powerhouse » Wed Feb 07, 2018 2:32 pm

@driz: I updated/upgraded a dozen times and almost never had an issue with it. The last time I remember having a problem was with Xen, that was probably 4-5 years ago. That said, I really feel for you.

Your setup is different from mine, for once you pass through drives, though I don't think that should cause a problem. If you updated the standard Linux Mint way, do you have any idea what might have caused the problem?

Linux Mint, for some inexplicable reason, sticks like glue to qemu 2.5. You can get a more up-to-date qemu version by adding the ppa found here: https://launchpad.net/~jacob/+archive/u ... ualisation

Code: Select all

sudo add-apt-repository ppa:jacob/virtualisation

Code: Select all

sudo apt-get update
As always with ppa's, be aware of the risks involved in adding external packages from an unknown source.

I'm currently running qemu 2.6.2.

I'm glad you found a solution to your audio problem. Here is my current script that works well with audio (I now use ALSA):

Code: Select all

#use ALSA
export QEMU_AUDIO_DRV=alsa
export QEMU_ALSA_ADC_BUFFER_SIZE=1024 QEMU_ALSA_ADC_PERIOD_SIZE=256
export QEMU_ALSA_DAC_BUFFER_SIZE=1024 QEMU_ALSA_DAC_PERIOD_SIZE=256
export QEMU_AUDIO_DAC_FIXED_SETTINGS=1
export QEMU_AUDIO_DAC_FIXED_FREQ=44100 QEMU_AUDIO_DAC_FIXED_FMT=S16 QEMU_AUDIO_ADC_FIXED_FREQ=44100 QEMU_AUDIO_ADC_FIXED_FMT=S16
export QEMU_AUDIO_DAC_TRY_POLL=1 QEMU_AUDIO_ADC_TRY_POLL=1
export QEMU_AUDIO_TIMER_PERIOD=50
Thanks for sharing your solution - really appreciated!

EDIT: I just saw you did the same using the ppa - it must be late and I'm tired...

EDIT 2: I'm using the latest kernel available in Linux Mint 18.3 (can't remember the release). Must be 4.13...
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Gigabyte GTX 970 + PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB + Samsung EVO 850 250GB SSD + 5 HDD, Corsair 500R case, SeaSonic 660W Gold X PS

vitriolicex
Level 1
Level 1
Posts: 1
Joined: Thu Feb 08, 2018 12:48 am
Location: Texas, USA

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by vitriolicex » Thu Feb 08, 2018 10:05 pm

Hello,
I'm having an issue with getting this to work on my computer - I get the output:

Code: Select all

collin@surveillance:~/vm$ sudo ./run.sh
qemu-system-x86_64: -device vfio-pci,host=03:00.0,multifunction=on: VFIO 0000:03:00.0 BAR 3 mmap unsupported. Performance may be slow
whenever I try to run my bash script. I get no output from the GPU passed through.

I am using an AMD Radeon R5 235X and an NVIDIA GTX 1060 3GB, the AMD as the "host" and the NVIDIA as the one being passed to the vm. I'm forced to use my AMD in the last PCI slot (05:00.0) because I plan to pass through the 1060 3GB as well as a 1060 6GB. Currently the 1060 3GB is in the PCI slot 03:00.0. I'm on Ubuntu (cinnamon would crash on linuxmint) - results from cat /proc/version:

Code: Select all

Linux version 4.13.0-32-generic (buildd@lgw01-amd64-004) (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.5)) #35~16.04.1-Ubuntu SMP Thu Jan 25 10:13:43 UTC 2018

First, to prevent the NVIDIA from being defaulted to render the host, I have been using this xorg.conf file in /etc/X11/xorg.conf:

Code: Select all

Section "Device"
        Identifier  "AMD GPU"
        Driver      "radeon"
        BusID       "PCI:5:0:0"
EndSection

Section "Monitor"
   Identifier  "ASUS MONITOR"
   HorizSync   30-94
   VertRefresh 48-85
EndSection

Section "Screen"
   Identifier  "SCREEN 1"
   Device      "AMD GPU"
   Monitor     "ASUS MONITOR"
   DefaultDepth    24
   SubSection "Display"
       Depth       24
       Modes       "1920x1200"
   EndSubSection
EndSection

Section "ServerLayout"
    Identifier  "Default Layout"
    Screen  0   "SCREEN 1"
EndSection
Second, I blacklisted nouveau so that the NVIDIA card can use pci-stub by adding this to /etc/modprobe.d/blacklist-nouveau.conf:

Code: Select all

blacklist nouveau
options nouveau modeset=0
I then followed your guide and noticed that my output for dmesg | grep vfio is different(the entries above [ 16.071934 ] were added when I tried running the script later):

Code: Select all

[   12.792120] vfio-pci 0000:03:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=io+mem
[   12.836044] vfio_pci: add [10de:1c02[ffff:ffff]] class 0x000000/00000000
[   12.864128] vfio_pci: add [10de:10f1[ffff:ffff]] class 0x000000/00000000
[   16.071934] vfio-pci 0000:03:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=io+mem
[   60.844184] vfio_ecap_init: 0000:03:00.0 hiding ecap 0x19@0x900
[   60.845603] vfio-pci 0000:03:00.0: BAR 3: can't reserve [mem 0xb0000000-0xb1ffffff 64bit pref]
[   63.553983] vfio-pci 0000:03:00.0: Invalid PCI ROM header signature: expecting 0xaa55, got 0xffff
[   63.553994] vfio-pci 0000:03:00.0: Invalid PCI ROM header signature: expecting 0xaa55, got 0xffff
[  881.640205] vfio_ecap_init: 0000:03:00.0 hiding ecap 0x19@0x900
[  881.641764] vfio-pci 0000:03:00.0: BAR 3: can't reserve [mem 0xb0000000-0xb1ffffff 64bit pref]
Here's the bash script I use:

Code: Select all

#!/bin/bash

vmname="windows10vm"

if ps -A | grep -q $vmname; then
   echo "$vmname is already running." &
   exit 1

else

# use pulseaudio
#export QEMU_AUDIO_DRV=pa
#export QEMU_PA_SAMPLES=8192
#export QEMU_AUDIO_TIMER_PERIOD=99
#export QEMU_PA_SERVER=/run/user/1000/pulse/native

#alsa/ac97 test 1/8/26 9:55am
export QEMU_AUDIO_DRV=alsa
export QEMU_ALSA_ADC_BUFFER_SIZE=1024 QEMU_ALSA_ADC_PERIOD_SIZE=256
export QEMU_ALSA_DAC_BUFFER_SIZE=1024 QEMU_ALSA_DAC_PERIOD_SIZE=256
export QEMU_AUDIO_DAC_FIXED_SETTINGS=1
export QEMU_AUDIO_DAC_FIXED_FREQ=44100 QEMU_AUDIO_DAC_FIXED_FMT=S16 QEMU_AUDIO_ADC_FIXED_FREQ=44100 QEMU_AUDIO_ADC_FIXED_FMT=S16
export QEMU_AUDIO_DAC_TRY_POLL=1 QEMU_AUDIO_ADC_TRY_POLL=1
export QEMU_AUDIO_TIMER_PERIOD=50

cp /usr/share/OVMF/OVMF_VARS.fd /tmp/my_vars.fd

qemu-system-x86_64 \
  -name $vmname,process=$vmname \
  -machine type=q35,accel=kvm \
  -cpu host,kvm=off \
  -smp 4,sockets=1,cores=2,threads=2 \
  -m 8G \
  -mem-prealloc \
  -balloon none \
  -rtc clock=host,base=localtime \
  -vga none \
  -serial none \
  -parallel none \
  -soundhw ac97 \
  -usb -usbdevice host:04f2:0939 -usbdevice host:04d9:a0cd -usbdevice host:28de:1142 \
  -device vfio-pci,host=03:00.0,multifunction=on \
  -device vfio-pci,host=03:00.1, \
  -drive if=pflash,format=raw,readonly,file=/usr/share/OVMF/OVMF_CODE.fd \
  -drive if=pflash,format=raw,file=/tmp/my_vars.fd \
  -boot order=dc \
  -drive id=disk0,if=virtio,cache=none,format=raw,file=/home/collin/vm/win.img \
  -drive file=/home/collin/vm/windows.iso,index=3,media=cdrom \
  -drive file=/home/collin/vm/virtio.iso,index=4,media=cdrom \
  -net user,net=10.0.0.0/8,host=10.0.0.1,hostfwd=tcp:127.0.0.1:2222-10.0.0.2:22 \
  -net nic 
   exit 0
fi
And for good measure, here's the Xorg.0.log:

Code: Select all

[    16.027] 
X.Org X Server 1.19.5
Release Date: 2017-10-12
[    16.028] X Protocol Version 11, Revision 0
[    16.028] Build Operating System: Linux 4.4.0-101-generic x86_64 Ubuntu
[    16.028] Current Operating System: Linux surveillance 4.13.0-32-generic #35~16.04.1-Ubuntu SMP Thu Jan 25 10:13:43 UTC 2018 x86_64
[    16.028] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-4.13.0-32-generic root=UUID=578c959d-72df-4509-b468-c5501bc7ca99 ro intel_iommu=on
[    16.028] Build Date: 24 November 2017  09:44:25AM
[    16.028] xorg-server 2:1.19.5-0ubuntu2~16.04.1 (For technical support please see http://www.ubuntu.com/support) 
[    16.028] Current version of pixman: 0.33.6
[    16.028] 	Before reporting problems, check http://wiki.x.org
	to make sure that you have the latest version.
[    16.028] Markers: (--) probed, (**) from config file, (==) default setting,
	(++) from command line, (!!) notice, (II) informational,
	(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
[    16.028] (==) Log file: "/var/log/Xorg.0.log", Time: Thu Feb  8 19:18:56 2018
[    16.028] (==) Using config file: "/etc/X11/xorg.conf"
[    16.028] (==) Using system config directory "/usr/share/X11/xorg.conf.d"
[    16.028] (==) ServerLayout "Default Layout"
[    16.028] (**) |-->Screen "SCREEN 1" (0)
[    16.028] (**) |   |-->Monitor "ASUS MONITOR"
[    16.028] (**) |   |-->Device "AMD GPU"
[    16.028] (==) Automatically adding devices
[    16.028] (==) Automatically enabling devices
[    16.028] (==) Automatically adding GPU devices
[    16.028] (==) Automatically binding GPU devices
[    16.028] (==) Max clients allowed: 256, resource mask: 0x1fffff
[    16.029] (WW) The directory "/usr/share/fonts/X11/cyrillic" does not exist.
[    16.029] 	Entry deleted from font path.
[    16.029] (WW) The directory "/usr/share/fonts/X11/100dpi/" does not exist.
[    16.029] 	Entry deleted from font path.
[    16.029] (WW) The directory "/usr/share/fonts/X11/75dpi/" does not exist.
[    16.029] 	Entry deleted from font path.
[    16.029] (WW) The directory "/usr/share/fonts/X11/100dpi" does not exist.
[    16.029] 	Entry deleted from font path.
[    16.029] (WW) The directory "/usr/share/fonts/X11/75dpi" does not exist.
[    16.029] 	Entry deleted from font path.
[    16.029] (==) FontPath set to:
	/usr/share/fonts/X11/misc,
	/usr/share/fonts/X11/Type1,
	built-ins
[    16.029] (==) ModulePath set to "/usr/lib/x86_64-linux-gnu/xorg/extra-modules,/usr/lib/xorg/extra-modules,/usr/lib/xorg/modules"
[    16.029] (II) The server relies on udev to provide the list of input devices.
	If no devices become available, reconfigure udev or disable AutoAddDevices.
[    16.029] (II) Loader magic: 0x55a100290e00
[    16.029] (II) Module ABI versions:
[    16.029] 	X.Org ANSI C Emulation: 0.4
[    16.029] 	X.Org Video Driver: 23.0
[    16.029] 	X.Org XInput driver : 24.1
[    16.029] 	X.Org Server Extension : 10.0
[    16.030] (++) using VT number 7

[    16.030] (II) systemd-logind: logind integration requires -keeptty and -keeptty was not provided, disabling logind integration
[    16.030] (II) xfree86: Adding drm device (/dev/dri/card0)
[    16.032] (--) PCI:*(0:3:0:0) 10de:1c02:1462:8c95 rev 161, Mem @ 0xfa000000/16777216, 0xa0000000/268435456, 0xb0000000/33554432, I/O @ 0x0000e000/128, BIOS @ 0x????????/131072
[    16.032] (--) PCI: (0:5:0:0) 1002:6771:1028:2120 rev 0, Mem @ 0xc0000000/268435456, 0xfb320000/131072, I/O @ 0x0000d000/256, BIOS @ 0x????????/131072
[    16.032] (II) LoadModule: "glx"
[    16.033] (II) Loading /usr/lib/xorg/modules/extensions/libglx.so
[    16.039] (II) Module glx: vendor="X.Org Foundation"
[    16.039] 	compiled for 1.19.5, module version = 1.0.0
[    16.039] 	ABI class: X.Org Server Extension, version 10.0
[    16.039] (II) LoadModule: "radeon"
[    16.039] (II) Loading /usr/lib/xorg/modules/drivers/radeon_drv.so
[    16.041] (II) Module radeon: vendor="X.Org Foundation"
[    16.041] 	compiled for 1.19.3, module version = 7.9.0
[    16.041] 	Module class: X.Org Video Driver
[    16.041] 	ABI class: X.Org Video Driver, version 23.0
[    16.041] (II) RADEON: Driver for ATI/AMD Radeon chipsets: [ cut down for character limit ]
[    16.071] (II) [KMS] Kernel modesetting enabled.
[    16.071] (**) RADEON(0): Depth 24, (--) framebuffer bpp 32
[    16.071] (II) RADEON(0): Pixel depth = 24 bits stored in 4 bytes (32 bpp pixmaps)
That's not the whole log, but I couldn't put much more because of the character limit.

Here's lspci -vnn:

Code: Select all

03:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:1c02] (rev a1) (prog-if 00 [VGA controller])
	Subsystem: Micro-Star International Co., Ltd. [MSI] Device [1462:8c95]
	Physical Slot: 6
	Flags: fast devsel, IRQ 27
	Memory at fa000000 (32-bit, non-prefetchable) [size=16M]
	Memory at a0000000 (64-bit, prefetchable) [size=256M]
	Memory at b0000000 (64-bit, prefetchable) [size=32M]
	I/O ports at e000 [size=128]
	Expansion ROM at 000c0000 [disabled] [size=128K]
	Capabilities: <access denied>
	Kernel driver in use: vfio-pci
	Kernel modules: nvidiafb, nouveau

03:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:10f1] (rev a1)
	Subsystem: Micro-Star International Co., Ltd. [MSI] Device [1462:8c95]
	Physical Slot: 6
	Flags: fast devsel, IRQ 55
	Memory at fb080000 (32-bit, non-prefetchable) [size=16K]
	Capabilities: <access denied>
	Kernel driver in use: vfio-pci
	Kernel modules: snd_hda_intel

05:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Caicos XTX [Radeon HD 8490 / R5 235X OEM] [1002:6771] (prog-if 00 [VGA controller])
	Subsystem: Dell Caicos XTX [Radeon HD 8490 / R5 235X OEM] [1028:2120]
	Physical Slot: 2-1
	Flags: bus master, fast devsel, latency 0, IRQ 50
	Memory at c0000000 (64-bit, prefetchable) [size=256M]
	Memory at fb320000 (64-bit, non-prefetchable) [size=128K]
	I/O ports at d000 [size=256]
	Expansion ROM at fb300000 [disabled] [size=128K]
	Capabilities: <access denied>
	Kernel driver in use: radeon
	Kernel modules: radeon

05:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Caicos HDMI Audio [Radeon HD 6400 Series] [1002:aa98]
	Subsystem: Dell Caicos HDMI Audio [Radeon HD 6400 Series] [1028:aa98]
	Physical Slot: 2-1
	Flags: bus master, fast devsel, latency 0, IRQ 54
	Memory at fb340000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: <access denied>
	Kernel driver in use: snd_hda_intel
	Kernel modules: snd_hda_intel
I have an Intel CPU and I know that this can work on the NVIDIA cards because I've used this guide before, having one NVIDIA as the host and the other as the guest GPU. Really, I just bought this AMD card to run linux while I use the windows VM with both NVIDIA card passed through.

If anyone needs more information, I can give it to them.
Thanks!

powerhouse
Level 6
Level 6
Posts: 1000
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by powerhouse » Sat Feb 17, 2018 6:03 am

@vitriolicex

First, fix the bash script:

Change

Code: Select all

 -device vfio-pci,host=03:00.1, \
to

Code: Select all

 -device vfio-pci,host=03:00.1 \
Second, you may need to add this to your grub file /etc/default/grub:

Code: Select all

GRUB_CMDLINE_LINUX="nomodeset vga=normal video=vesafb:off"
If the above doesn't help: You have two nearly identical GPUs, which may cause a problem. Try to remove one of them, and see if it works.

You mentioned that you use pci-stub? My tutorial uses vfio, so you need to check that you follow either this tutorial or another, but not both!

Your script is missing the part where I bind the graphics card:

Code: Select all

configfile=/etc/vfio-pci.cfg
vmname="win10vm"

vfiobind() {
	dev="$1"
        vendor=$(cat /sys/bus/pci/devices/$dev/vendor)
        device=$(cat /sys/bus/pci/devices/$dev/device)
        if [ -e /sys/bus/pci/devices/$dev/driver ]; then
                echo $dev > /sys/bus/pci/devices/$dev/driver/unbind
        fi
        echo $vendor $device > /sys/bus/pci/drivers/vfio-pci/new_id
   
}


if ps -A | grep -q $vmname; then
	zenity --info --window-icon=info --timeout=15 --text="$vmname is already running." &
	exit 1

else

#modprobe vfio-pci

	cat $configfile | while read line;do
	echo $line | grep ^# >/dev/null 2>&1 && continue
		vfiobind $line
	done
Of course, if you use pci-stub, you will have a different way of configuration. Make sure the vfio-pci configuration lines in the script are compatible.

See also https://wiki.archlinux.org/index.php/PC ... ng_the_GPU
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Gigabyte GTX 970 + PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB + Samsung EVO 850 250GB SSD + 5 HDD, Corsair 500R case, SeaSonic 660W Gold X PS

jockos
Level 1
Level 1
Posts: 1
Joined: Sun Feb 18, 2018 6:32 am

Re: HOW-TO make dual-boot obsolete using kvm VGA passthrough

Post by jockos » Sun Feb 18, 2018 6:49 am

Hi all!
I had a working configuration using this guide with one Nvidia card (1060) and the builtin Intel GPU (skylake). However, I recently got a new monitor which I wanted to use with Linux as well, so I added a second Nvidia GPU (1030), as the Intel one didn't output 4K.

Now I've done a fresh Mint (xfce) install (actually four already in trying to make it work), but I'm stuck..

I'm trying to go with this guide in order to not have to blacklist the Nvidia driver:
viewtopic.php?f=231&t=212692&start=40#p1174032

At boot I'm getting:
bash: /sbin/vfio-pci-override-vga.sh not found or something similar (it flashes by in a second).
And the /sys/bus/pci/devices/GPU AND GPUAUDIO/driver_override files contain null instead of vfio-pci as expected.

I'm quite new to Linux, so I really have no clue on how to fix this. I've tried running the /sbin/vfio-pci-override-vga.sh, which makes the driver_override files fill up correctly, but since it doesn't run at boot it's useless..



I've wasted about one day already trying to get the 1030 display 4k on HDMI, turns out the "High Speed" HDMI cable supplied with the monitor (AOC U3277PWQU) was not up to spec..
Noveau cannot detect this monitor on HDMI, so I'm using the Nvidia driver now. Will this be a problem as well?

Post Reply

Return to “Virtualization”