HOW-TO make dual-boot obsolete using kvm VGA passthrough

[ersatzhero]

Hello,

I would like to give a big thank you to powerhouse and everyone who helped to create this guide.
I love my setup with the Windows VM. It just works :>

But still, I got a networking problem when I have installed Docker following the official guide.
The VM can't reach the internet anymore after Docker has been installed.

I can't figure out what the problem is:
Is it the iptables rules installed by Docker or
is the problem the two added bridge networks or
something completely else?

I don't have any idea how to look for that, so I hope, someone can point me in the right direction.

Regards

[bash64]

hey powerhouse.

I need help switching to using a real drive.
I followed your link to using qemu images but there is nothing there about using real drives or LVM.
I bought a 512gb pny thumb drive,. it has read/write at 400/250mbsec.
I used pv to copy my existing image file to it. I can see the partitions on it,

I need the hard drive line for this to go into my bash script. it usually comes up as /dev/sdc.

p.s. i got lookingglass setup. as you know i have a laptop with nvidia optimus and i cannot run anything but seabios. well, lookingglass only has a w10 driver. i managed to convert window7/seabios to windows10/seabios using the microsoft mediacreationtool.exe.
it upgraded just fine and nvidia does not produce the code 43 error. seabios saves the day. but took over 8 hrs to finish. jeeeeesh. in the end the lookingglass driver wont install. wait for the official redhat release, i guess. w10/seabios is on the thumb. i just need help getting it booted. it already works, but slowly. the thumb should speed it up. i upgraded my laptop ram from 12gb to 28gb and gave qemu 20gb for gaming.

[cyberspectre]

Hi powerhouse, thanks for making this tutorial. If you don't mind, I'd like some guidance.

I just built a new system specifically for VGA passthrough. Specs are as follows:

Gigabyte GA-AX370-Gaming K7
AMD Ryzen 1600x
Nvidia GTX760FTW (for Linux)
Nvidia GTX970FTW (for passthrough)
32GB memory
120GB SSD
2TB HDD
Ubuntu 16.04 host

I got to part 7 successfully before noticing the link you posted to the tutorial for Ubuntu 16.04 using virt-manager, and now I'm wondering whether I should continue with this method or go the virt-manager route — and, if going with the second, how to continue after completing steps 1-6 of this guide.

The windows VM will be used for Adobe creative cloud and gaming. All of my files and games will be on the 2TB HDD, which I need to be accessible by both OSes. How do you suggest I proceed?

[odtech]

Hi powerhouse, thanks for making this tutorial. If you don't mind, I'd like some guidance.

I just built a new system specifically for VGA passthrough. Specs are as follows:

Gigabyte GA-AX370-Gaming K7
AMD Ryzen 1600x
Nvidia GTX760FTW (for Linux)
Nvidia GTX970FTW (for passthrough)
32GB memory
120GB SSD
2TB HDD
Ubuntu 16.04 host

I got to part 7 successfully before noticing the link you posted to the tutorial for Ubuntu 16.04 using virt-manager, and now I'm wondering whether I should continue with this method or go the virt-manager route — and, if going with the second, how to continue after completing steps 1-6 of this guide.

The windows VM will be used for Adobe creative cloud and gaming. All of my files and games will be on the 2TB HDD, which I need to be accessible by both OSes. How do you suggest I proceed?

Did you stop at step 7 because you ran into a issue or purely because you can't decide between qemu-kvm and virt-manager?

I've personally only used qemu-kvm so i can't help with pros and cons between the two but i image coming from windows the gui of virt-manager would be more friendly. The qemu console is however by no means difficult to use when you need to for examle add a usb thumb drive to the VM.

As for the drive i would reformat is to ext4 filesystem and have mint manage it then setup samba shares (windows shares) for the files you want to access from the windows guest.

With Ryzen just make sure you have one of the latest kernels installed for maximum compatibility. You can install kernel through the build in update manager in mint.

[powerhouse]

Hello,

Powerhouse,
Have you tried using looking glass yet?
You have to have a quadro professional card, but it changes qemu so that you do not need an external monitor for the gaming card.
You see your VM in a window with full gaming.

https://looking-glass.hostfission.com/

I will never buy a quadro so will never use this.

Hi bash64,

This is a very late reply but I'm literally traveling all the time now, except these few days.

I've never tried looking glass, but it sounds interesting. Not having the need for 2 graphics cards and perhaps screens (or the wiring) will definitely help the migration towards Linux with a gaming Windows VM when needed.

I do use a Quadro 2000 card, but currently in the host. My VM runs a Nvidia GTX 970.

It should work with the GTX 970 but then there is the issue with administrative tasks in Windows, or the login screen, both of which cannot be captured unless when using an Nvidia professional Quadro 2000 and higher card (Tesla, etc.). This is prohibitive for most users.

I would have liked to try Looking Glass with my Quadro, but this is my work station I use on a day to day basis. Until I have enough time and a good reason to tinker with my hardware, I won't touch the system.

Since you already compiled Looking Glass, you could give it a try with your system. The instructions are here: https://looking-glass.hostfission.com/node/9. In the qemu script you'll need to add something like:

Code: Select all

-device ivshmem-plain,memdev=ivshmem \
-object memory-backend-file,id=ivshmem,share=on,mem-path=/dev/shm/looking-glass,size=32M

Let us know if and how it works.

[powerhouse]

hey powerhouse.

I need help switching to using a real drive.
I followed your link to using qemu images but there is nothing there about using real drives or LVM.
I bought a 512gb pny thumb drive,. it has read/write at 400/250mbsec.
I used pv to copy my existing image file to it. I can see the partitions on it,

I need the hard drive line for this to go into my bash script. it usually comes up as /dev/sdc.

p.s. i got lookingglass setup. as you know i have a laptop with nvidia optimus and i cannot run anything but seabios. well, lookingglass only has a w10 driver. i managed to convert window7/seabios to windows10/seabios using the microsoft mediacreationtool.exe.
it upgraded just fine and nvidia does not produce the code 43 error. seabios saves the day. but took over 8 hrs to finish. jeeeeesh. in the end the lookingglass driver wont install. wait for the official redhat release, i guess. w10/seabios is on the thumb. i just need help getting it booted. it already works, but slowly. the thumb should speed it up. i upgraded my laptop ram from 12gb to 28gb and gave qemu 20gb for gaming.

Regarding Looking Glass, I just posted my answer above, having missed your post here.

Now to the drive configuration. Since you use a thumb drive, it is plugged into a USB port. You can try this line in the qemu script:

Code: Select all

-usb -usbdevice disk:/dev/sdc

where /dev/sdc is the thumb drive (make sure it is).

To test it, I used a Linux Mint thumb drive:

Code: Select all

sudo qemu-system-x86_64 -m 4G -machine type=q35,accel=kvm -cpu host,kvm=off -smp 4,sockets=1,cores=2,threads=2 -usb -usbdevice disk:/dev/sdc

Works like a charm.

LVM won't be that easy. Let me know if above option works.

[powerhouse]

Hello,

I would like to give a big thank you to powerhouse and everyone who helped to create this guide.
I love my setup with the Windows VM. It just works :>

But still, I got a networking problem when I have installed Docker following the official guide.
The VM can't reach the internet anymore after Docker has been installed.

I can't figure out what the problem is:
Is it the iptables rules installed by Docker or
is the problem the two added bridge networks or
something completely else?

I don't have any idea how to look for that, so I hope, someone can point me in the right direction.

Regards

Hello ersatzhero,

Great to hear that your Windows VM works!

I have no experience with Docker. However, I saw somewhere in the documentation you linked to that Docker uses IP forwarding, that is a routed network - see https://docs.docker.com/engine/installa ... stinstall/.

My tutorial uses a bridged network for communication between the VM and the host, as well as to the outside world.

My suggestion is this:

Remove the entire network configuration in the qemu script. Qemu will automatically use IP forwarding to connect the VM to the network. You may have to change the network configuration in Network Manager and remove the bridge.

Let me know if it helps.

Also, can you post a Passmark or Userbenchmark or your VM here: viewtopic.php?f=225&t=153482. The user benchmark is free and can be downloaded here: http://www.userbenchmark.com. This helps in troubleshooting and establishes comparable performance data.

[powerhouse]

Hi powerhouse, thanks for making this tutorial. If you don't mind, I'd like some guidance.

I just built a new system specifically for VGA passthrough. Specs are as follows:

Gigabyte GA-AX370-Gaming K7
AMD Ryzen 1600x
Nvidia GTX760FTW (for Linux)
Nvidia GTX970FTW (for passthrough)
32GB memory
120GB SSD
2TB HDD
Ubuntu 16.04 host

I got to part 7 successfully before noticing the link you posted to the tutorial for Ubuntu 16.04 using virt-manager, and now I'm wondering whether I should continue with this method or go the virt-manager route — and, if going with the second, how to continue after completing steps 1-6 of this guide.

The windows VM will be used for Adobe creative cloud and gaming. All of my files and games will be on the 2TB HDD, which I need to be accessible by both OSes. How do you suggest I proceed?

I hope odtech's reply got you in the right direction.

If you already made it to step 7, you may as well go through with the tutorial. I personally have had bad experiences with virt-manager, but that is very subjective. Many people have used virt-manager and been successful.

My issue with virt-manager is that it does about 90-95% of the job, and for the rest of it one has to edit the xml file. It's more work and less flexible, and you never know where to start looking when things don't work the way they should.

About using the HDD: It's best to use LVM to create a physical volume (PV), a volume group (VG) and a logical volume (LV). After creating the LV for Windows, DON'T format it under Linux! When you install the Windows VM, specify the LV as the installation disk (like /dev/VG/LV).

With LVM you always have the option to grow the volume by adding a new disk and making it available to the VG and LV. If you intend to use it for Adobe CC then 2TB is not much - trust me. My photos take up around 2-3TB alone, and I'm constantly adding more capacity.

Have a second drive and use it for backup. You can do it easily in LVM using mirroring.

From the host you can access the LVM volume using kpartx. See my post here: viewtopic.php?t=111783. I have written a small script to mount the Windows volume under Linux using kpartx. If you are interested, I can post it here.

[bash64]

hey powerhouse.

Now to the drive configuration. Since you use a thumb drive, it is plugged into a USB port. You can try this line in the qemu script:

Code: Select all

-usb -usbdevice disk:/dev/sdc

where /dev/sdc is the thumb drive (make sure it is).

To test it, I used a Linux Mint thumb drive:

Code: Select all

sudo qemu-system-x86_64 -m 4G -machine type=q35,accel=kvm -cpu host,kvm=off -smp 4,sockets=1,cores=2,threads=2 -usb -usbdevice disk:/dev/sdc

Works like a charm.

LVM won't be that easy. Let me know if above option works.

qemu-system-x86_64: -usbdevice disk:/dev/sdc: '-usbdevice' is deprecated, please use '-device usb-...' instead
WARNING: Image format was not specified for '/dev/sdc' and probing guessed raw.
Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
Specify the 'raw' format explicitly to remove the restrictions.

[cyberspectre]

Hi powerhouse, thanks for making this tutorial. If you don't mind, I'd like some guidance.

I just built a new system specifically for VGA passthrough. Specs are as follows:

Gigabyte GA-AX370-Gaming K7
AMD Ryzen 1600x
Nvidia GTX760FTW (for Linux)
Nvidia GTX970FTW (for passthrough)
32GB memory
120GB SSD
2TB HDD
Ubuntu 16.04 host

I got to part 7 successfully before noticing the link you posted to the tutorial for Ubuntu 16.04 using virt-manager, and now I'm wondering whether I should continue with this method or go the virt-manager route — and, if going with the second, how to continue after completing steps 1-6 of this guide.

The windows VM will be used for Adobe creative cloud and gaming. All of my files and games will be on the 2TB HDD, which I need to be accessible by both OSes. How do you suggest I proceed?

Did you stop at step 7 because you ran into a issue or purely because you can't decide between qemu-kvm and virt-manager?

I've personally only used qemu-kvm so i can't help with pros and cons between the two but i image coming from windows the gui of virt-manager would be more friendly. The qemu console is however by no means difficult to use when you need to for examle add a usb thumb drive to the VM.

As for the drive i would reformat is to ext4 filesystem and have mint manage it then setup samba shares (windows shares) for the files you want to access from the windows guest.

With Ryzen just make sure you have one of the latest kernels installed for maximum compatibility. You can install kernel through the build in update manager in mint.

Hi powerhouse, thanks for making this tutorial. If you don't mind, I'd like some guidance.

I just built a new system specifically for VGA passthrough. Specs are as follows:

Gigabyte GA-AX370-Gaming K7
AMD Ryzen 1600x
Nvidia GTX760FTW (for Linux)
Nvidia GTX970FTW (for passthrough)
32GB memory
120GB SSD
2TB HDD
Ubuntu 16.04 host

I got to part 7 successfully before noticing the link you posted to the tutorial for Ubuntu 16.04 using virt-manager, and now I'm wondering whether I should continue with this method or go the virt-manager route — and, if going with the second, how to continue after completing steps 1-6 of this guide.

The windows VM will be used for Adobe creative cloud and gaming. All of my files and games will be on the 2TB HDD, which I need to be accessible by both OSes. How do you suggest I proceed?

I hope odtech's reply got you in the right direction.

If you already made it to step 7, you may as well go through with the tutorial. I personally have had bad experiences with virt-manager, but that is very subjective. Many people have used virt-manager and been successful.

My issue with virt-manager is that it does about 90-95% of the job, and for the rest of it one has to edit the xml file. It's more work and less flexible, and you never know where to start looking when things don't work the way they should.

About using the HDD: It's best to use LVM to create a physical volume (PV), a volume group (VG) and a logical volume (LV). After creating the LV for Windows, DON'T format it under Linux! When you install the Windows VM, specify the LV as the installation disk (like /dev/VG/LV).

With LVM you always have the option to grow the volume by adding a new disk and making it available to the VG and LV. If you intend to use it for Adobe CC then 2TB is not much - trust me. My photos take up around 2-3TB alone, and I'm constantly adding more capacity.

Have a second drive and use it for backup. You can do it easily in LVM using mirroring.

From the host you can access the LVM volume using kpartx. See my post here: viewtopic.php?t=111783. I have written a small script to mount the Windows volume under Linux using kpartx. If you are interested, I can post it here.

Odtech, powerhouse, thanks for your replies! Mainly, the reasons I was debating whether to use this or virt-manager were / are:

- Memory usage; using hugepages, I can dedicate 24 GB of ram to Windows, but the memory isn't usable in Linux even when the VM isn't running. I'd like to be able to use all of the memory in Linux when the VM is off. Is there a way to do this?

- VNC; I'd like to be able to display the VM via VNC, which seems well integrated by virt-manager. I hear it's painful to set it up yourself.

- The HDD question. Setting up a samba share works for general purposes, but I doubt you could run games very well. But powerhouse has the answer, I think. Powerhouse, with kpartx, could I access the partition in Nautilus (as well as back it up continually with SpiderOak on Linux) while the Windows VM is running? In other words, seamless concurrent access?

[powerhouse]

qemu-system-x86_64: -usbdevice disk:/dev/sdc: '-usbdevice' is deprecated, please use '-device usb-...' instead
WARNING: Image format was not specified for '/dev/sdc' and probing guessed raw.
Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
Specify the 'raw' format explicitly to remove the restrictions.

Thanks for the feedback!

What version of qemu do you run, it's not 2.5?

EDIT: This is exactly the reason I sometimes hate KVM. The documentation is all over the place, they keep changing syntax by the hour, and try figure...

So what I dug out on the usb storage front is this:

https://github.com/qemu/qemu/blob/maste ... torage.txt

and

https://www.linux-kvm.org/page/USB

Of course the linux-kvm website is probably meant to be joke, with some of the info dating to 2017, whereas most of it was published in 2007 or so. The nice thing about this website is that it never says when something was posted, and usually forgets to mention which version of kvm the information relates to.

Perhaps I'm beating a dead horse. Am I?

Back to the issue - how to configure USB storage:

The best place to look for current information is actually on your hard drive. Go to

Code: Select all

/usr/share/doc/qemu-system-common

and open the file

Code: Select all

qdev-device-use.txt.gz

Somewhere in there you'll find this:

Code: Select all

-device usb-storage,drive=DRIVE-ID,removable=RMB

At least this is the documentation provided with qemu 2.5.

Here is perhaps a better, more up-to-date syntax for assigning USB thumb drives to a VM (taken from the Arch Linux forum):

Code: Select all

-device usb-ehci,id=usb,bus=pci.0,addr=0x4 -device usb-host,hostbus=3,hostaddr=2

The above would emulate a USB 2 controller (usb-ehci) and attach the USB stick to it.

Perhaps the best information can be obtained from the ArchLinux tutorial here: https://wiki.archlinux.org/index.php/QE ... USB_device

Using the -machine type=q35 option, the best way to go about is to create a USB 2 or USB 3 device and then attach the USB drive to it:
For USB 2:

Code: Select all

-device usb-ehci,id=ehci

For USB 3:

Code: Select all

-device nec-usb-xhci,id=xhci

Attach the USB drive:

Code: Select all

-device usb-host,bus=controller_id.0,vendorid=0xvendor_id,productid=0xproduct_id

I just hope I'm on the right track here.

[powerhouse]

...Odtech, powerhouse, thanks for your replies! Mainly, the reasons I was debating whether to use this or virt-manager were / are:

- Memory usage; using hugepages, I can dedicate 24 GB of ram to Windows, but the memory isn't usable in Linux even when the VM isn't running. I'd like to be able to use all of the memory in Linux when the VM is off. Is there a way to do this?

- VNC; I'd like to be able to display the VM via VNC, which seems well integrated by virt-manager. I hear it's painful to set it up yourself.

- The HDD question. Setting up a samba share works for general purposes, but I doubt you could run games very well. But powerhouse has the answer, I think. Powerhouse, with kpartx, could I access the partition in Nautilus (as well as back it up continually with SpiderOak on Linux) while the Windows VM is running? In other words, seamless concurrent access?

Memory: Thanks for the question. This let me to explore hugepages a little bit more.

My first suggestion: Install the linux-doc package. This will give you access to the kernel documentation and hugepages configuration. After installation, open the hugetlbpage.txt.gz file under:

Code: Select all

/usr/share/doc/linux-doc/vm

Here a quote:
/proc/sys/vm/nr_hugepages indicates the current number of "persistent" huge
pages in the kernel's huge page pool. "Persistent" huge pages will be
returned to the huge page pool when freed by a task. A user with root
privileges can dynamically allocate more or free some persistent huge pages
by increasing or decreasing the value of 'nr_hugepages'.

Here are the steps to change the number of persistent huge pages to 20:

Code: Select all

sudo -i
echo 20 > /proc/sys/vm/nr_hugepages

I configured my system to allocate 11000 huge pages (~24 GB). Here is the output when changing the allocated huge pages:

Code: Select all

root ~ # free -m
              total        used        free      shared  buff/cache   available
Mem:          32175       24336         946          97        6892        7247
Swap:          8187           0        8187
root ~ # echo 20 > /proc/sys/vm/nr_hugepages
root ~ # free -m
              total        used        free      shared  buff/cache   available
Mem:          32175        2364       22918          97        6892       29219
Swap:          8187           0        8187
root ~ # 

It works!

You should allocate the required number of huge pages at boot, as described in my tutorial. At the end of the boot process, you could write a simple script to where you reduce the number of huge pages, like shown here.

Then modify the VM startup script and allocate the required amount just before the qemu-... line. At the end of the script, make sure to reduce the number again. You might run into problems when your host - or other VMs - take up too much memory and you don't have enough to allocate to the VM. The document mentioned above will offer more information on that and how to go about.

VNC: This tutorial describes how to pass through a graphics card to the VM, so that you would be able to run Windows on its own screen (or switch your screen from host to guest and back, as I do). Access to the VM via VNC offers some convenience when you run (background-)processes in Windows, while working under the Linux host. Using virt-manager to accomplish the same will give you the same results, but will also allow you to switch back to the host and perhaps control the VM via VNC (I'm not sure now if that works).

I have been using RDP to control the Linux host from within Windows, in full-screen. It may not be the best solution, but it was simple and worked.

qemu has options to run a Spice client to control the VM. You might want to check that out - I haven't tried it.

HDD: Yes and no. kpartx allows you to mount and access partitions in a raw disk image. You can also mount file.img image files. In my case, I created a LVM raw partition - that is a partition that has not been formated - to hold my Windows installation. When installing Windows, Windows would see this LVM raw partition as an unformated drive, create some partitions, and format the main partition to NTFS. These partitions you can mount using kpartx.

However, you should never mount your Windows partition(s) for read-write when Windows is running. Doing so can create havoc with the file system, as Windows is totally unaware that some other system (the Linux host) is writing to its file system. You can safely mount the Windows partition for read-only!

I use kpartx in conjunction with LVM snapshots to backup Windows. However, it's a bad idea to backup the Windows VM while it is running. When Windows is running, it creates all sorts of temporary files etc. that can get in the way when you want to boot from an image you made of the VM. Snapshots allow you to freeze the current state, so while the system is being backed up, any changes to the running system will be written into a snapshot logical volume that will, upon completion and deletion of the snapshot, be merged with the VM partition.

SAMBA, on the other hand, is a network storage protocol. When you use Samba to access a running Windows VM from your host, Windows will be totally aware of that. So with Samba, two or more systems can access the same storage device and read and write (depending on the permissions you grant) at will.

I use Samba all the time to exchange files between my Windows VM and my Linux host. Since the virtual Ethernet bridge between the VM and the host has a bandwidth of around 10Gbit, the transfer speed is currently limited by the (slow) speed of my SSDs.

I assume that your Linux host won't need access to the games (apps and data) running on Windows, unless to back them up. For backup purposes it is best to close the VM and use Linux tools such as kpartx to mount the VM partitions under Linux and back them up to another (external) disk or system using Linux tools.

SpiderOak: I'm not familiar with SpiderOak, but I'm using both Dropbox and Google Drive for cloud backup and remote file synchronization. I suppose SpiderOak is similar. If you want to secure your Windows installation, just install SpiderOak inside the Windows VM and, once configured, it should do its magic. The same goes for installing SpiderOak on Linux. BUT, SpiderOak will not be able to see and backup your Windows VM unless you mount it using kpartx. I would not do that since you never know if the Linux version is capable of handling these things correctly.

To summarize the HDD solutions:
1. Use Samba to share files between Windows VM and Linux host (when both are running);
2. Use kpartx and a backup tool of your choice to make a local backup copy of your Windows VM disk image WHILE WINDOWS IS OFF.
3. Use SpiderOak or another commercial cloud backup solution to backup your Linux host and/or Windows VM to the cloud. Install and run SpiderOak under the respective OS. Make sure SpiderOak under Linux does not touch your Windows drive.

I hope that helps.

[bash64]

After many years of trying to achieve the perfect setup for playing DX11 games in linux I am finally there.
One of the main hurdles was overcoming audio stutter.
I traced down the issue to virtual hardfile lag. Not even using the redhat virtio drivers helped.
So, yesterday I got in a 1tb ssd, an intel 540s.
It fixed the issue.
Fallout 4 loads instantly and the audio never stutters.

The lag of emulating a hard drive is real.

I am using qemu 2.5 (which btw wont work with kernel 4.4. had to use 4.8 ).
I tried 2.11.50 which claims to have a whole new pulseaudio buffering system which fixes the audio stutter. It didn't.
Only removing the virtual hardfile emulation lag fixed it.
I must admit I was ready to give up for a long time, but hey, since nothing ever fully worked, what was there to lose?
I cannot recommend enough that everyone at least put their QEMU hardfile onto an SSD.

BTW: I did finally get windows 10 installed by upgrading windows7 using seabios to windows10 using the microsoft mediacreationtool.exe. it is quite slow without an ssd but i will be checking that.


For giggles here is my script to boot windows 7 (the nbd section does not work with windows 10 so i commented it out so as not to confuse anyone). lookingglass is disabled also. not done experimenting with that:


#!/bin/bash

#lookingglass...
#ivshmem-server -p /tmp/ivshmem.pid -S /tmp/ivshmem_socket -l 16M -n 8

cd /media/dad/QEMU-SSD/qemu-wd

#I have commented out nbd section for windows 10...made for windows 7 only
<<NBDSECTION
echo 'Creating nbd device nodes...'
sudo modprobe nbd max_part=63
sleep 2
echo 'Attach hardfile to device node nbd0...'
qemu-nbd -f raw -c /dev/nbd0 windows7.img
echo 'Create folder to mount hardfile to...'
mkdir -p /media/w7
echo 'Mount nbd0p1 to the folder...'
mount /dev/nbd0p1 /media/w7
echo 'Display the contents of the mounted folder...'
ls /media/w7/Boot/
echo 'Replace BOOTSTAT.DAT with backup copy...'
cp BOOTSTAT.DAT /media/w7/Boot/
cp BCD.LOG /media/w7/Boot/
cp BCD.LOG1 /media/w7/Boot/
cp BCD.LOG2 /media/w7/Boot/
echo 'Wait 5 seconds...'
sleep 5
echo 'Unmount and remove folder...'
umount /media/w7
echo 'Disconnect nbd device node...'
qemu-nbd -d /dev/nbd0
NBDSECTION

configfile=/media/dad/QEMU-SSD/qemu-wd/vfio-pci1.cfg

#vfiobind function....
vfiobind() {
dev="$1"
vendor=$(cat /sys/bus/pci/devices/$dev/vendor)
device=$(cat /sys/bus/pci/devices/$dev/device)
echo $dev $vendor $device
if [ -e /sys/bus/pci/devices/$dev/driver ]; then
echo $dev > /sys/bus/pci/devices/$dev/driver/unbind
fi
echo $vendor $device > /sys/bus/pci/drivers/vfio-pci/new_id
}

modprobe vfio-pci

cat $configfile | while read line;do
echo $line | grep ^# >/dev/null 2>&1 && continue
vfiobind $line
done

# use pulseaudio
# use pulseaudio
export QEMU_AUDIO_DRV=pa
export QEMU_PA_SAMPLES=8192
export QEMU_AUDIO_TIMER_PERIOD=99
export QEMU_PA_SERVER=/run/user/1000/pulse/native

#export QEMU_PA_BUFFER_SIZE_OUT=5000
#export QEMU_PA_BUFFER_SIZE_IN=5000
#export QEMU_PA_TLENGTH=5000
#export QEMU_PA_FRAGSIZE=5000

#use ALSA
#export QEMU_ALSA_DAC_BUFFER_SIZE=512
#export QEMU_ALSA_DAC_PERIOD_SIZE=170
#export QEMU_AUDIO_DRV=alsa

# display current time and expected boot time for Windows...
echo "Windows expected to reach desktop at ..."
boottime1=$(date -d "$starttime today + 6 minutes" +'%I:%M:%S')
boottime2=$(date -d "$starttime today + 7 minutes" +'%I:%M:%S')
echo "between $boottime1 and $boottime2..."

#launch windows 7 64bit using SEABIOS (non-uefi bios)....
#/home/dad/qemu/build/x86_64-softmmu/
qemu-system-x86_64 -machine type=q35,accel=kvm -cpu host,kvm=off \
-smp 8,sockets=1,cores=4,threads=2 \
-bios /usr/share/seabios/bios.bin \
-serial none \
-parallel none \
-m 20G \
-vga none \
-soundhw hda \
-mem-prealloc \
-balloon none \
-rtc clock=host,base=localtime \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-device virtio-scsi-pci,id=scsi \
-drive id=disk0,if=virtio,cache=none,format=raw,file=/media/dad/QEMU-SSD/qemu-wd/windows7.img \
-drive file=/media/dad/QEMU-SSD/qemu-wd/virtio-win-0.1.126.iso,id=isocd,format=raw,if=none -device scsi-cd,drive=isocd \
-netdev type=tap,id=net0,ifname=tap0 \
-device virtio-net-pci,netdev=net0,mac=00:16:3e:00:01:01 \
-usbdevice host:276d:1119 \
-boot menu=on \
-boot order=c

exit 0

[powerhouse]

Recently I had a closer look at the UserBenchmark website and noticed that this is actually very helpful in several ways:

* Determining the usability of a CPU or GPU
* Performance comparisons
* Drive options and their performance
* Configuration issues

I wrote a post on how I read the results - comments and critique are welcome: https://heiko-sieger.info/virtual-machi ... benchmark/.

[bash64]

Hey,
I have lookingglass working fully.
Unfortunately, the ivshmem driver only works with windows 8(latest driver) and 10.
I am using windows 10, but w10 causes massive lag with pulseaudio.
I have windows7 and pulseaudio working perfectly so i am waiting for the windows 7 lookinglass host and driver.

You also have to switch your games to windowed-mode.
You cannot see them using lookingglass if they go full screen.

So, it does work.
If the game is in window mode you can also on the linux side enlarge to full screen mode.

What this means for me is that I use qemu to play my dx11 games but it requires an external monitor be hooked up to my laptop.
With lookingglass I no longer need the external monitor and do not have to take it with me to computer meetings.


just a few tips:
1) do not run the ivshmem-server like the website says to. you will be using ivshmem-plain instead. see below line.

2) you must use qemu 2.7.0 or higher with spice enabled and pulseaudio enabled
mint 18 uses qemu 2.5. this wont do.
after compiling my own qemu (2.11.50) and using the lines below I was FINALLY able to install the ivshmem driver they give you. but not until then.

here are the additional lines i use:
-object memory-backend-file,id=ivshmem,share=on,mem-path=/dev/shm/looking-glass,size=32M \
-device ivshmem-plain,memdev=ivshmem \
-spice port=5900,addr=127.0.0.1,disable-ticketing \

if you compile your own qemu (like i do) you need to tell ./configure some things
>./configure --enable-spice --audio-drv-list=pa
>make
>make install (i dont do this, i dont want to give my package manager heartburn, just give the full path to the compiled copy in your script).

if ./configure complains that you dont have the spice files installed then you need to install:
>sudo apt install libspice-protocol-dev libspice-server-dev


3) make sure you have the linux client downloaded and working. they changed it so redownload and compile it if need be. they now require libconfig-dev and libva-dev in addition to the existing dependencies.

4) for some reason i had to create my own /dev/shm/lookinglass file. if you have to here is how:
sudo touch /dev/shm/looking-glass
sudo chown {user}:kvm /dev/shm/looking-glass
sudo chmod 660 /dev/shm/looking-glass

replace user with the name of your login id, such as roger. mine would be chown roger:kvm /dev/shm/looking-glass

5) play video games using windowed mode.

[driz]

Howdy all, thought i'd post some horrible news. After the two win10 vms were running perfectly for over a year now, I was running updates on my other linux servers... some how i did pay attention and upgraded the distro of my kvm box. It pretty much ruined everything, now I get stuck at the windows logo during boot. I put minimal effort into troubleshooting, probably just going to rebuild from scratch and document the changes. As some of you may recall, for vm2 (the 4 yr old) I passed the full ssd through, while the 7yr old had a partition on an ssd passed through. I noticed gains in disk perf by passing the full disk, so i will ensure i do that for both vms this time. I'll try to post what happens following this rebuild.

Sucks that this happened, but that's what I get for not paying attention

side note: even if i try to boot from the win10 iso, after i "press any key to boot from disc" it freezes on the boot logo there too.. probably some driver passthru issue if I had to guess.

[driz]

Howdy all, thought i'd post some horrible news. After the two win10 vms were running perfectly for over a year now, I was running updates on my other linux servers... some how i did pay attention and upgraded the distro of my kvm box. It pretty much ruined everything, now I get stuck at the windows logo during boot. I put minimal effort into troubleshooting, probably just going to rebuild from scratch and document the changes. As some of you may recall, for vm2 (the 4 yr old) I passed the full ssd through, while the 7yr old had a partition on an ssd passed through. I noticed gains in disk perf by passing the full disk, so i will ensure i do that for both vms this time. I'll try to post what happens following this rebuild.

Sucks that this happened, but that's what I get for not paying attention

side note: even if i try to boot from the win10 iso, after i "press any key to boot from disc" it freezes on the boot logo there too.. probably some driver passthru issue if I had to guess.

Alright, I just finished rebuilding the host and both guests successfully. To my surprise, the audio crackle that plagued my kids for a year is gone. I did make some changes, mostly because I didn't take details enough notes (even on my blog post about this very build...)

I haven't even written my new blog entry, so you guys get it first although, since I moved away from mint, maybe I shouldn't post here.

Media: http://releases.ubuntu.com/16.04/ubuntu ... so.torrent
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenial

I did not update the kernel which is still sitting at 4.4.0-87-generic and no passthrough issues.

I built both windows 10 guests concurrently, win10 install to 1st reboot, 2minutes 5 seconds. I did a direct passthrough of the ssd to each vm

Code: Select all

  -device virtio-scsi-pci,id=scsi \
   -drive file=/dev/sdd,id=disk0,if=none,format=raw,cache=none,aio=native \
  -device scsi-hd,drive=disk0 \

after initial boot, i installed netkvm drivers and let it use windows to update the AMD video drivers. I renamed the system and joined to my domain. I then shut down the system, removed the virtio and win10 install disk lines from my launch script and booted. Both systems blue screened, but after a restart booted fine. I believe this bsod is a known issue with the radeon drivers. Some of my notes indicated that it was alleviated if I used the actual radeon drivers, but my laziness and a reboot won out.

My WEI was 6.8 on both systems and i ran `winsat formal` concurrently on them.

As I mentioned before the crackle in my audio is completely gone! the kids will be so happy! My SSDs are still detected as thin provisioned drivers rather than SSD. I also went ahead and used the non-deprecated method of passing through usb devices. Just because during the epic failure, when qemu upgraded, the warning bothered me.

Code: Select all

-usb -device usb-host,vendorid=0x04f2,productid=0x0939 -device usb-host,vendorid=0x04f2,productid=0x0833 -device usb-host,vendorid=0x045e,productid=0x075d -device usb-host,hostbus=1,hostaddr=4 \

I'm still not using hugepages because I'm too lazy to do so. My ovmf was from the github repo and I used the latest virtio drivers ISO.

Here is my launch script:

Code: Select all

#!/bin/bash

configfile=/etc/vfio-pci1.cfg
vmname="kid1"

vfiobind() {
   dev="$1"
        vendor=$(cat /sys/bus/pci/devices/$dev/vendor)
        device=$(cat /sys/bus/pci/devices/$dev/device)
        if [ -e /sys/bus/pci/devices/$dev/driver ]; then
                echo $dev > /sys/bus/pci/devices/$dev/driver/unbind
        fi
        echo $vendor $device > /sys/bus/pci/drivers/vfio-pci/new_id

}


if ps -A | grep -q $vmname; then
   echo "$vmname is already running."
   exit 1

else

   cat $configfile | while read line;do
   echo $line | grep ^# >/dev/null 2>&1 && continue
      vfiobind $line
   done


if [ ! -f /tmp/system1.fd ]; then
  cp /srv/ovmf-x64/OVMF_VARS-pure-efi.fd /tmp/system1.fd
fi

/usr/bin/qemu-system-x86_64 \
  -name $vmname,process=$vmname \
  -serial none \
  -parallel none \
  -machine type=q35,accel=kvm \
  -cpu host,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time \
  -smp 4,sockets=1,cores=2,threads=2 \
  -enable-kvm \
  -m 6G \
  -mem-prealloc \
  -balloon none \
  -rtc clock=host,base=localtime \
  -vga none \
  -nographic \
  -usb -device usb-host,vendorid=0x040B,productid=0x2000 -device usb-host,vendorid=0x046d,productid=0xc01e -device usb-host,vendorid=0x046d,productid=0x0819 -device usb-host,hostbus=1,hostaddr=11 \
  -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
  -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,romfile=/srv/TV809MH.570 \
  -device vfio-pci,host=01:00.1,bus=pcie.0 \
  -drive if=pflash,format=raw,readonly,file=/srv/ovmf-x64/OVMF_CODE-pure-efi.fd \
  -drive if=pflash,format=raw,file=/tmp/system1.fd \
  -boot order=c \
  -object iothread,id=iothread0 \
  -device virtio-scsi-pci,id=scsi,iothread=iothread0 \
  -drive file=/dev/sdc1,id=disk0,if=none,format=raw,cache=directsync,aio=native \
  -device scsi-hd,drive=disk0,bootindex=1 \
  -netdev type=tap,id=net0,ifname=tap0,vhost=on \
  -device virtio-net-pci,netdev=net0,mac=52:54:00:ea:c8:8d

#  -mem-path /dev/hugepages1 \
#  -drive file=/srv/win10.iso,id=isocd,format=raw,if=none -device scsi-cd,drive=isocd,bootindex=1 \
#  -drive file=/srv/virtio-win.iso,id=virtiocd,format=raw,if=none -device ide-cd,bus=ide.1,drive=virtiocd \

exit 0
fi

my grub command line

Code: Select all

GRUB_CMDLINE_LINUX_DEFAULT="rd.driver.pre=vfio-pci quiet splash intel_iommu=on,igfx_off vfio_iommu_type1.allow_unsafe_interrupts=1 iommu=pt

I used to have kvm_intel.emulate_invalid_guest_state=0, but i couldn't remember why.. so now I don't.

I don't know if this will help anyone, but my kids game/homework on these guest vms daily (for over a year now) and the biggest complaint was the audio crackle. With that gone, a 6.8 score.. they're pretty happy. There's also enough resources left that I run my backup DNS (bind9) on the host.

If you wish to see my other configuration options, just look at my blog (link above) for the details. This install was the ubuntu LTS server edition, i ran update but did nothing to force a newer kernel than what came on it.

[driz]

I discovered today that the webcams don't work on the windows 10 vms

The errors showed this

Code: Select all

processing error - resetting ehci hc
emu-system-x86_64: libusb_release_interface: -4 [NO_DEVICE]

Attempt to fix #1

I've upgraded the kernel to 4.13.0-32-generic, but there was no change to the error
I'm not sure if this is typical, but I have noticed this

Code: Select all

[  156.431892] usb 1-4.2: reset low-speed USB device number 7 using xhci_hcd
[  157.219873] usb 1-4.1: reset low-speed USB device number 5 using xhci_hcd
[  157.883861] usb 1-4.2: reset low-speed USB device number 7 using xhci_hcd
[  158.507853] usb 1-4.2: reset low-speed USB device number 7 using xhci_hcd
[  158.883844] usb 1-4.3: reset high-speed USB device number 9 using xhci_hcd
[  159.551831] usb 1-4.1: reset low-speed USB device number 5 using xhci_hcd
[  160.163821] usb 1-4.1: reset low-speed USB device number 5 using xhci_hcd
[  160.535803] usb 1-4.4: reset full-speed USB device number 11 using xhci_hcd
[  162.227800] usb 1-4.4: reset full-speed USB device number 11 using xhci_hcd
[  162.455802] usb 1-4.4: reset full-speed USB device number 11 using xhci_hcd
[  182.975109] usb 1-3.3: reset low-speed USB device number 8 using xhci_hcd
[  183.531071] usb 1-3.4: reset low-speed USB device number 10 using xhci_hcd
[  183.919037] usb 1-3.2: reset high-speed USB device number 6 using xhci_hcd
[  184.111033] usb 1-3.1: reset full-speed USB device number 4 using xhci_hcd
[  184.526980] usb 1-3.2: reset high-speed USB device number 6 using xhci_hcd
[  184.766973] usb 1-3.2: reset high-speed USB device number 6 using xhci_hcd
[  185.118956] usb 1-3.1: reset full-speed USB device number 4 using xhci_hcd
[  185.486946] usb 1-3.4: reset low-speed USB device number 10 using xhci_hcd
[  185.946916] usb 1-3.1: reset full-speed USB device number 4 using xhci_hcd
[  186.446900] usb 1-3.4: reset low-speed USB device number 10 using xhci_hcd
[  187.398854] usb 1-3.3: reset low-speed USB device number 8 using xhci_hcd
[  188.106820] usb 1-3.3: reset low-speed USB device number 8 using xhci_hcd

What I thought was running perfectly is not. sad times. My next step, I suppose will be to try and upgrade qemu.. it's quite irritating that everything is working perfectly except the webcam...

Attempt to fix #2 Success
Using the PPA at https://launchpad.net/~jacob/+archive/u ... ter=xenial I upgraded my qemu from 2.5 to 2.6.2.

this resolved the issue, i was concerned the audio crackle would return, but it did not! Everything seems to be working perfectly now.

[powerhouse]

Hello bash64, thanks for sharing your hourglass solution. This seems like a great was to use a laptop without external screen. Much appreciate your input!

[powerhouse]

@driz: I updated/upgraded a dozen times and almost never had an issue with it. The last time I remember having a problem was with Xen, that was probably 4-5 years ago. That said, I really feel for you.

Your setup is different from mine, for once you pass through drives, though I don't think that should cause a problem. If you updated the standard Linux Mint way, do you have any idea what might have caused the problem?

Linux Mint, for some inexplicable reason, sticks like glue to qemu 2.5. You can get a more up-to-date qemu version by adding the ppa found here: https://launchpad.net/~jacob/+archive/u ... ualisation

Code: Select all

sudo add-apt-repository ppa:jacob/virtualisation

Code: Select all

sudo apt-get update

As always with ppa's, be aware of the risks involved in adding external packages from an unknown source.

I'm currently running qemu 2.6.2.

I'm glad you found a solution to your audio problem. Here is my current script that works well with audio (I now use ALSA):

Code: Select all

#use ALSA
export QEMU_AUDIO_DRV=alsa
export QEMU_ALSA_ADC_BUFFER_SIZE=1024 QEMU_ALSA_ADC_PERIOD_SIZE=256
export QEMU_ALSA_DAC_BUFFER_SIZE=1024 QEMU_ALSA_DAC_PERIOD_SIZE=256
export QEMU_AUDIO_DAC_FIXED_SETTINGS=1
export QEMU_AUDIO_DAC_FIXED_FREQ=44100 QEMU_AUDIO_DAC_FIXED_FMT=S16 QEMU_AUDIO_ADC_FIXED_FREQ=44100 QEMU_AUDIO_ADC_FIXED_FMT=S16
export QEMU_AUDIO_DAC_TRY_POLL=1 QEMU_AUDIO_ADC_TRY_POLL=1
export QEMU_AUDIO_TIMER_PERIOD=50

Thanks for sharing your solution - really appreciated!

EDIT: I just saw you did the same using the ppa - it must be late and I'm tired...

EDIT 2: I'm using the latest kernel available in Linux Mint 18.3 (can't remember the release). Must be 4.13...