HOW-TO make dual-boot obsolete using XEN VGA passthrough

Questions about virtualization software
Forum rules
Before you post read how to get help. Topics in this forum are automatically closed 6 months after creation.
powerhouse
Level 6
Level 6
Posts: 1141
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

@Boggler:

virt-manager: I've tried it in conjunction with the VGA passthrough how-to over on www.overclock.net where it works with Fedora. I found it easier to just write my own domU.cfg file, because that is all there is to creating a VM. It's clear text, easy to modify, and you know exactly what it means (or can check the meaning of each entry). I found virt-manager to be confusing. The good side of virt-manager is its integrated VNC viewer and the control panel for running VMs, with graphs and all that.

Marvell SATA controller: Oh I should have recalled that, I think you mentioned it earlier. I had exactly the same problem (my board also sports a Marvell SATA controller, one with the bug). I decided to not bother with it and disabled it in the BIOS. Then I installed a USB/SATA PCIe controller card - problem solved. I hope this bug fix (https://lkml.org/lkml/2014/5/22/685) will soon be released with a kernel update.

It's absolutely great that you made it work with KVM - it really doesn't matter much which hypervisor you use, as long as it works! As a side note, I managed - for the first time - get primary passthrough working with KVM and a AMD 7770 card.

Please post an update on the progress with sound.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
earlboy

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by earlboy »

Hello powerhouse, been a long time since I posted here. Looks like you delving into kvm as well. :)

Just a tip for those who are running two descrete video cards, e.g. nvdia for host and radeon for guest. It would be a lot easier if you compile the kernel and remove all drivers for the vid card you're going to passthrough. Like in my case, I removed all radeon stuff in the kernel, including the opensource ones, also making pci-stub (kvm) or xen-pcistub (xen) built into the kernel instead of a module made it a lot easier for me to do passthrough.

For kvm, the new vfio driver for passhtrough is supposed to be great but, if you have a multifunction device it may not be suitable, like in my case, using vfio, I can't pass my sata controller, since it was grouped with my lan card. I had to use the older pci assign method.

By the way I'm running kvm now, the advantage being :
- I don't have to use a usb sound card, kvm creates a virtual sound card, which can be directed back to the host which will output it to the speakers. Could not make work with spice, so I used vnc instead.
- I can install the proprietary nvidia drivers on the host. The nvidia drivers seems to have a conflict with the xen hypervisor which ended up with a blank screen.

Although I'm no longer using mint, I hope the tips I mentioned above may help.

cheers.
powerhouse
Level 6
Level 6
Posts: 1141
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

@earlboy: Thanks for the advice - this is indeed helpful.

Xen and Nvidia: The Nvidia driver identifies Xen and refuses to install/run. However, there is a command line option for the installer to disregard the Xen kernel (can't remember it, but Google might be your friend).

I haven't had any problems with assigning graphics cards to pciback when using the initramfs method (step 11 in the how-to), but there is always a potential that things don't work as expected. Compiling pciback or pcistub into the kernel and running the command from within grub makes sure that no graphics driver grabs the graphics card(s).

Hopefully someone will add a KVM how-to as this seems to be an alternative that is rapidly improving (except the documentation). Neither do I have the knowledge nor the time to do it at the moment.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
Chapapa

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by Chapapa »

hi im back again,

so i was trying to passthrough my usb device via xm usb-attach, but i ran into an error. maybe you know what this means:

first i did:
xm usb-hc-create win7g 2 2
xm usb-list-assignable-devices
7-1 : ID 045e:00d1 Microsoft Microsoft Optical Mouse with Tilt Wheel
7-2 : ID 04d9:1603 USB Keyboard
7-3.1 : ID 046d:c221 Logitech, Inc. G11/G15 Keyboard / Keyboard
7-3.4 : ID 046d:c222 G15 Keyboard G15 Keyboard
7-4 : ID 1532:0009 Tempest Habu Mouse

then this happened:
xm usb-attach win7g 0 1 7-1
Unexpected error: <class 'xen.util.vusb_util.UsbDeviceParseError'>

Please report to xen-devel@lists.xen.org
Traceback (most recent call last):
File "/usr/lib/xen-4.3/bin/xm", line 8, in <module>
main.main(sys.argv)
File "/usr/lib/xen-4.3/bin/../lib/python/xen/xm/main.py", line 3946, in main
_, rc = _run_cmd(cmd, cmd_name, args)
File "/usr/lib/xen-4.3/bin/../lib/python/xen/xm/main.py", line 3970, in _run_cmd
return True, cmd(args)
File "/usr/lib/xen-4.3/bin/../lib/python/xen/xm/main.py", line 3011, in xm_usb_attach
if vusb_util.bus_is_assigned(bus):
File "/usr/lib/xen-4.3/bin/../lib/python/xen/util/vusb_util.py", line 275, in bus_is_assigned
raise UsbDeviceParseError("Can't get assignment status: (%s)." % bus)
xen.util.vusb_util.UsbDeviceParseError: vusb: Error parsing USB device info: Can't get assignment status: (7-1).

regards

chapapa


edit:
i was trying to use my 00:14.0 and my 05:00.0 usb controllers (one pci bridge, one usb controller) and using ps2 for a keyboard.
added 00:14.0 to pciback.conf and as expected, all usb ports did not work when in linux. starting up dom1 with 05:00.0 works perfectly. starting up dom2 with 00:14.0 does not work.. it shows the the controller in the device manager, but my devices could not be installed.. this is frustrating... anyone know what to do?

dom2:
kernel = '/usr/lib/xen-default/boot/hvmloader'
builder='hvm'
memory = 3072
name = 'win7a'
vcpus=4
#pae=1
acpi=1
apic=1
on_xend_stop='shutdown'
vif = [ 'mac=00:16:3e:68:88:88,bridge=xenbr0' ]
disk = [ 'phy:/dev/mapper/win7-win7a,hda,w' , 'phy:/dev/mapper/xs-win7a,hdc,w' ]
device_model = '/usr/lib/xen-default/bin/qemu-dm'
boot='dc'
sdl=0
vnc=1
vncpasswd=''
stdvga=0
serial='pty'
tsc_mode=0
viridian=1
usb=1
usbdevice='tablet'
gfx_passthru=0
pci=[ '01:00.0', '01:00.1' , '00:1a.0' , '00:1d.0' ,'00:14.0' ]
localtime=1
pci_power_mgmt=1

dom1:

kernel = '/usr/lib/xen-default/boot/hvmloader'
builder='hvm'
memory = 3072
name = 'win7b'
vcpus=4
#pae=1
acpi=1
apic=1
on_xend_stop='shutdown'
vif = [ 'mac=00:16:3e:68:e1:03,bridge=xenbr0' ]
disk = [ 'phy:/dev/mapper/xs-win7b,hda,w' ]
device_model = '/usr/lib/xen-default/bin/qemu-dm'
boot='dc'
sdl=0
vnc=1
vncpasswd=''
stdvga=0
serial='pty'
tsc_mode=0
viridian=1
usb=1
usbdevice='tablet'
gfx_passthru=0
#HDMI und USB
pci=[ '02:00.0' , '02:00.1', '05:00.0' ]
#VGA
#pci=[ '01:00.0', '01:00.1' , ]
localtime=1
pci_power_mgmt=1

nano /etc/xen/win7b.cfg
# xm pci-list-assignable-devices
0000:02:00.0
0000:02:00.1
0000:01:00.0
0000:01:00.1
0000:00:14.0
0000:00:1a.0
0000:00:1b.0
0000:00:1d.0

note: i deletetd 05:00.0 from pciback.conf. it was working and i needed some devices to maneuver within dom2 to see if the controller were found etc.

for dom2 it tried all usb ports on the mainboard and front panel...
powerhouse
Level 6
Level 6
Posts: 1141
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

@Chapapa: There are some inconsistencies in your configuration files:

1. You said your removed 05:00.0 from your pciback.cfg file, and it indeed doesn't show up in xm pci-list-assignable-devices, but you list it in your dom1 config file!

2. I never tried xm usb-hc-create, only PCI passthrough, but you seem to be trying both at the same time. When you use PCI passthrough you do not need xm usb-hc-create etc.

3. Have a look here: http://wiki.xen.org/wiki/Xen_USB_Passthrough. You are trying to use PVUSB devices: "PVUSB uses a paravirtialized front-end/back-end interface, similar to the traditional Xen PV network and disk protocols. In order to use PVUSB, you need usbfront in your guest OS, and usbback in dom0 (or your USB driver domain). Unfortunately, kernel support for the PVUSB protocol isn't very good."
To use PVUSB you would also need a driver under Windows (the GPLPV drivers are said to have an early version of it).

A better method would be the emulated USB described in the same link above. If you need 2 USB devices, for example a keyboard and mouse, you need to hotplug them when the dom1/dom2 are running:
xm usb-add domain host:xx.yy
xm usb-del domain host:xx.yy

4. Have you tried passing through 00:1a.0 and 00:1d.0 to different domUs? This is what I'm doing. Here is my (shortened) lspci -k output:

Code: Select all

00:1a.0 USB controller: Intel Corporation C600/X79 series chipset USB2 Enhanced Host Controller #2 (rev 05)
	Subsystem: ASUSTeK Computer Inc. Device 84ef
	Kernel driver in use: pciback
00:1c.0 PCI bridge: Intel Corporation C600/X79 series chipset PCI Express Root Port 1 (rev b5)
	Kernel driver in use: pcieport
00:1c.1 PCI bridge: Intel Corporation C600/X79 series chipset PCI Express Root Port 2 (rev b5)
	Kernel driver in use: pcieport
00:1c.2 PCI bridge: Intel Corporation C600/X79 series chipset PCI Express Root Port 3 (rev b5)
	Kernel driver in use: pcieport
00:1c.3 PCI bridge: Intel Corporation C600/X79 series chipset PCI Express Root Port 4 (rev b5)
	Kernel driver in use: pcieport
00:1c.4 PCI bridge: Intel Corporation C600/X79 series chipset PCI Express Root Port 5 (rev b5)
	Kernel driver in use: pcieport
00:1d.0 USB controller: Intel Corporation C600/X79 series chipset USB2 Enhanced Host Controller #1 (rev 05)
	Subsystem: ASUSTeK Computer Inc. Device 84ef
	Kernel driver in use: ehci-pci
00:1a.0 is passed through to my Windows guest (using the pciback driver), while 00:1d.0 is used for Linux dom0.

Here is your lspci -k output (shortened):

Code: Select all

00:1a.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #2 (rev 05)
Subsystem: ASRock Incorporation Device 8c2d
Kernel driver in use: ehci-pci
00:1b.0 Audio device: Intel Corporation 8 Series/C220 Series Chipset High Definition Audio Controller (rev 05)
Subsystem: ASRock Incorporation Device c892
Kernel driver in use: pciback
00:1c.0 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #1 (rev d5)
Kernel driver in use: pcieport
00:1c.6 PCI bridge: Intel Corporation 82801 PCI Bridge (rev d5)
00:1d.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #1 (rev 05)
Subsystem: ASRock Incorporation Device 8c26
Kernel driver in use: ehci-pci
I don't see why this wouldn't work.

5. Don't use USB=1 in your domU config files when using PCI passthrough! USB=1 tells Xen to emulate the USB device in domU, whereas PCI passthrough provides the Windows domU with a real PCI device (hardware). These are two different methods. When using PCI passthrough you need to change your dom1 and dom2 config files to read:

Code: Select all

usb=0
6. To summarize the above, try the following:

dom0 (Linux) uses the 00:1d.0 USB controller
dom1 (Windows a) uses the 05:00.0 USB controller
dom2 (Windows b) uses the 00:1a.0 USB controller

Here the /etc/xen/pciback.cfg file:

Code: Select all

0000:02:00.0
0000:02:00.1
0000:01:00.0
0000:01:00.1
0000:00:1a.0
0000:00:1b.0
0000:05:00.0
Use 0000:00:14.0 as an alternative to 0000:05:00.0, or in addition to the above if needed. Try passing through as few PCI devices as possible (I would even remove the audio device 0000:00:1b.0 in the beginning and add it only when the rest works).

Use this dom1.cfg file:

Code: Select all

kernel = '/usr/lib/xen-default/boot/hvmloader'
builder='hvm'
memory = 3072
name = 'win7b'
vcpus=4
pae=1
acpi=1
apic=1
on_xend_stop='shutdown'
vif = [ 'mac=00:16:3e:68:e1:03,bridge=xenbr0' ]
disk = [ 'phy:/dev/mapper/xs-win7b,hda,w' ]
device_model = '/usr/lib/xen-default/bin/qemu-dm'
boot='dc'
sdl=0
vnc=1
vncpasswd=''
stdvga=0
serial='pty'
tsc_mode=0
viridian=1
usb=0
usbdevice='tablet'
gfx_passthru=0
#HDMI und USB
pci=[ '02:00.0' , '02:00.1', '05:00.0' ]
localtime=1
pci_power_mgmt=1
And this dom2.cfg file:

Code: Select all

kernel = '/usr/lib/xen-default/boot/hvmloader'
builder='hvm'
memory = 3072
name = 'win7a'
vcpus=4
pae=1
acpi=1
apic=1
on_xend_stop='shutdown'
vif = [ 'mac=00:16:3e:68:88:88,bridge=xenbr0' ]
disk = [ 'phy:/dev/mapper/win7-win7a,hda,w' , 'phy:/dev/mapper/xs-win7a,hdc,w' ]
device_model = '/usr/lib/xen-default/bin/qemu-dm'
boot='dc'
sdl=0
vnc=1
vncpasswd=''
stdvga=0
serial='pty'
tsc_mode=0
viridian=1
usb=0
usbdevice='tablet'
gfx_passthru=0
pci=[ '01:00.0', '01:00.1' , '00:1a.0' ]
localtime=1
pci_power_mgmt=1
Make sure you know where these USB ports are located on your motherboard / front panel / back panel.


If you use my startup script for the Windows domU (see here), you need to disable the pciback script in init.d using:

Code: Select all

sudo update-rc.d pciback disable
In this case remove the PCI IDs of your graphics cards from the /etc/xen/pciback.cfg file, which then contains only this:

Code: Select all

0000:00:1a.0
0000:00:1b.0
0000:05:00.0
To pass through your graphics cards, follow step 11 of my how-to. As root do:

Code: Select all

echo "xen-pciback passthrough=1 hide=(01:00.0)(01:00.1)" >> /etc/initramfs-tools/modules
echo "xen-pciback passthrough=1 hide=(02:00.0)(02:00.1)" >> /etc/initramfs-tools/modules
followed by:

Code: Select all

update-initramfs -k all -c
After that and a reboot, both graphics cards are attached to pciback (xm pci-list-assignable-devices should show). I assume you have an IGP graphics device for Linux, else you will get a blank screen and only have access via ssh/VNC or similar.
Using my script has the advantage that when you boot into Xen (before starting your domUs), all USB devices are still available under Linux dom0. This allows you to identify your USB ports and their corresponding PCI IDs (see step 8 in the how-to). For a detailed description on how to identify USB ports, see here.

Important: If the above doesn't work, you still have 2 options:
a) Use synergy - see step 8 under "Alternative method" in my how-to. This method does NOT require you to pass through any PCI USB devices!
b) Buy a USB expansion card (PCIe card), just make sure you can return it if it doesn't work. My card uses an ASMedia Technology Inc. ASM1042 chip that works well.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
earlboy

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by earlboy »

powerhouse, maybe you can help me out.

Doing vga pass through to a linux guest ( not windows ). All is well until I shutdown the guest and boot it again for the second time. The whole system hangs.
I've encountered this with windows as guest and the fix was to eject the video card upon shutdown, using a utility called deveject.

Is there an equivalent technique to eject the vga card in linux that you know of?

Doing an "echo 1 > /sys/devices/pci/.../reset" did not work.
powerhouse
Level 6
Level 6
Posts: 1141
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

earlboy wrote:powerhouse, maybe you can help me out.

Doing vga pass through to a linux guest ( not windows ). All is well until I shutdown the guest and boot it again for the second time. The whole system hangs.
I've encountered this with windows as guest and the fix was to eject the video card upon shutdown, using a utility called deveject.

Is there an equivalent technique to eject the vga card in linux that you know of?

Doing an "echo 1 > /sys/devices/pci/.../reset" did not work.
Sorry, I don't know. Never tried it and can't remember reading about it.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
Chapapa

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by Chapapa »

@powerhouse

thanks for your reply!!
yes i use xen 4.3, lm 16.
im not sure if your method works, i didnt change usb=0/1, so i will try that now.
But here is what i did:
i simply use 00:14.0 for dom1 and 00:05.0 for dom2.
here is what happens: 00:14.0 disables all mobo usb ports in linux ( i use a ps2 keyboard in linux^^). now in the domu it didnt work right away, but i had to install the usb drivers.
05:00.0 and 05:02.0 only work together. i think because they are behind the same bridge.


so basically this is how it looks like:

xxxxxxxx/ 00:1a:0
00:14:0 - 00:1d.0

if i wanted to pass through 00:1a.0 or 1d in a domu, it didnt work, because the main controller(00:14.0) is still in linux.

xxx/ 05:00.0
X
xxx\ 05:02.0
05:00 and 05:02 are behind the same pci bridge. so i have to assign them both to one domu. if i try different, i get an error: xxx needs to be coassigned with yyy. i tried altering the config file in xen, so that pci-strict-check would say no, the error did not come up, i could assign them to different domus, but only the 05.00.0 worked. 05:02.0 did not. it was shown in device manager, but no devices worked.

i encountered also a different problem. when i reboot my boot machine (or xm shutdown x) the vga card hangs. the usual procedure was: xm create x, vncviewer localhost:0 to see if it wants to go into repair mode(after a dirty reboot) and when the windows sign appears vncviewer would hang and my vga card kicks in and my monitor works. this does not happen after restarting now. instead of my vncviewer hanging and switching to my vga card, it boots right into windows. it happened before (i was fiddling with xm and xl and thought i destroyed something) , so i started from scratch. rebooting worked fine until today.now its happening again. and i didnt change anything (well nothing with my vga cards). do you have any idea what to do? i tried a startup script where i detach and attach the vga card in the startup and shutdown phase, but this didnt work.

for my linux machine i use my cpus gpu. but this doesnt work 100%. when i hide both 7850's leaving only the onboard graphic solution for linux, everytime i boot into linux it says "cinnamon crashed" and i have some kind of safe mode linux running. it didnt bother me, because i only need terminal, but maybe you know how to ressolve this issue.

so i have two plans now, first trying your method with usb=0, and otherwise using my method with ps2 for linux, 00:14.0 for dom1 and 05:00.0 for dom2. for convenience i still need to run a startup script to boot directly into dom1 or dom2 and then use ssh. i dont want to change my keyboard (from 00:14.0 to ps2 and back to start my vm) everytime i boot :D.

yeah i have used synergy since the beginning. but i want to have native usb support within my machines.

edit:

so i tried your method. where i leave 1d in linux. 14 is in the pciback.conf. and all pci ports on the mobo are within the vm :(.
powerhouse
Level 6
Level 6
Posts: 1141
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

@Chapapa:

If none of my suggestions works for you, you could share the problem with the Xen users mailing list. You need to sign up for it, but it's well worth the effort. I'm not sure but there may be a way to overcome the problem. The Xen users mailing list is watched by seasoned Xen users as well as Citrix support people, so it's the best place to look for help.

I have also had some success with LM 17, Xen 4.4 and the xl toolstack, although this success varies with the graphics card hardware. Not sure this can solve your USB issues, but it might be worth trying. (See link below for more information.)

Last not least you may try KVM. I have posted some notes on KVM and a link to a good tutorial (albeit for Arch Linux) - see http://forums.linuxmint.com/viewtopic.p ... 40#p878200.

Finally, you still have the option of getting a USB PCIe card.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
Crackerjackthe4th

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by Crackerjackthe4th »

Hey, powerhouse, excellent post.
It helped me a lot with Xen in general, on top of PCI passthrough.

I'm having a few issues, though. (I haven't seen this in this thread at all, either)
Long story short, I can't get VGA to work. I have all the compatible hardware, as far as I know, (intel i7-4770, Asus Z87i mobo, Radeon HD 7870, and I'm using a 2-way usb and HDMI kvm switch on one monitor) and entering [# xl pci-assignable-list] without a running VM returns all the devices I'm trying to passthrough, and entering it WITH a running VM returns nothing, so it *looks* like the devices are being passed through.
Using #lspci -vvv also shows that the board is owned by pciback, and I've blacklisted all related

Yet when I'm inside Windows 7's device manager, it just shows a little yellow triangle next to the GPU *as well as* the USB. Clicking on the "Radeon HD 7800 series VGA adapter" (side note: shouldn't it be reading the full device name?) only shows:

"This device canot find enough free resources that it can use. (code 12)"

By the way, this is even after installing the catalyst control center. Any further attempts to install drivers and Windows slaps me in the face with the whole "all drivers are up to date," even though the Catalyst Control Center says that there ISN'T a working driver.
So what are some common causes for this? I've been pulling my hair out over this for over a week. Maybe longer, I dunno. I'm not sure if this is important, but other guides have told me to *update windows as far as I can,* and then install the GPLPV drivers, all before even attempting VGA passthrough.

I'm not sure which logs I should even be looking at, so I'm not sure what exactly I'd need to show you to illustrate my situation.
If you can find the time to help a guy out, I'd be really grateful.
powerhouse
Level 6
Level 6
Posts: 1141
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

Crackerjackthe4th wrote:Hey, powerhouse, excellent post.
It helped me a lot with Xen in general, on top of PCI passthrough.

I'm having a few issues, though. (I haven't seen this in this thread at all, either)
Long story short, I can't get VGA to work. I have all the compatible hardware, as far as I know, (intel i7-4770, Asus Z87i mobo, Radeon HD 7870, and I'm using a 2-way usb and HDMI kvm switch on one monitor) and entering [# xl pci-assignable-list] without a running VM returns all the devices I'm trying to passthrough, and entering it WITH a running VM returns nothing, so it *looks* like the devices are being passed through.
Using #lspci -vvv also shows that the board is owned by pciback, and I've blacklisted all related

Yet when I'm inside Windows 7's device manager, it just shows a little yellow triangle next to the GPU *as well as* the USB. Clicking on the "Radeon HD 7800 series VGA adapter" (side note: shouldn't it be reading the full device name?) only shows:

"This device canot find enough free resources that it can use. (code 12)"

By the way, this is even after installing the catalyst control center. Any further attempts to install drivers and Windows slaps me in the face with the whole "all drivers are up to date," even though the Catalyst Control Center says that there ISN'T a working driver.
So what are some common causes for this? I've been pulling my hair out over this for over a week. Maybe longer, I dunno. I'm not sure if this is important, but other guides have told me to *update windows as far as I can,* and then install the GPLPV drivers, all before even attempting VGA passthrough.

I'm not sure which logs I should even be looking at, so I'm not sure what exactly I'd need to show you to illustrate my situation.
If you can find the time to help a guy out, I'd be really grateful.
Sorry for the late response.

In your domU config file you need to specify:

Code: Select all

device_model_version = "qemu-xen-traditional"
since you are using the xl tool stack.

See also here: http://lists.xenproject.org/archives/ht ... 00238.html

Hope it helps. If it doesn't, reduce the domU memory. If that doesn't help, use CPU pinning. Last not least, try the xm tool stack as described in this how-to.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
O_Shovah

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by O_Shovah »

Hello everybody,

First of all i wanted to thank powerhouse for the excellent guide to the xen passthrough system installation.

So i want to return at least the tiny bit i am abled to do, by sharing my working configuration components.
Please add those to the compatibility list.

As of 07.09.2014:

Xen 4.4 (latest updates)

Host system: Ubuntu 14.04 64 bit

Harddisk: Some seagate 500GB

CPU: Haswell Intel 4440 @ 3,1 GHz stock

Board: Asrock B-85M

All current BIOS versions properly make VT-d work:

1.70
2.10
2.20
2.40

RAM: 2x4GB Hyperion @ 1600 Mhz

Host system Graphics: Intel intergrated GPU (Ubuntu)

Guest system Graphics: AMD Fire Pro W7000 (Windows 7 Professional 64 bit) [Hardwarewise a 7870 with more memory]

I was struggeling about 3 months with massive stability issues of both systems and finally tracked it down to the ram.
Now it runs rock stable. I have testet several software sucessfully especially on the guest system:

Windows 7:
Solid edge ST 6
Siemens Unigraphics Nx
OpenCl suites

World of tanks
Warthunder
Crysis 3
powerhouse
Level 6
Level 6
Posts: 1141
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

Thanks O_Shovah, this is very helpful! I will update the list on overclock when I get to it.

I can feel your pain with the faulty memory modules and the 2 month search for the cause of the system instability. I had the same. This is why anybody buying or building a PC should run a memory test for at least 72 hours under full system/CPU load. The slightest error, even a single one, usually means that the memory is no good. My computer shop had to test several memory makes and brands until, after one month, they finally found one that works.

Thanks again for your contribution to the cause!
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
earlboy

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by earlboy »

Guys got a question for you.
How can you prevent a game from detecting that it is running in a VM?
I've been playing this online game and it runs great on KVM/XEN well until they installed an anti cheat system called xigncode3.
Now every time I try to run it, is says tha app cannot be run in a virtual machine.

Been googling for a long time now, and all that I found is the one for vmware, which is very different from xen/kvm.
I don't want to use vmware if I can help it, I've already got a stable system using kvm and I can swicth to xen anytime.
powerhouse
Level 6
Level 6
Posts: 1141
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

earlboy wrote:Guys got a question for you.
How can you prevent a game from detecting that it is running in a VM?
I've been playing this online game and it runs great on KVM/XEN well until they installed an anti cheat system called xigncode3.
Now every time I try to run it, is says tha app cannot be run in a virtual machine.

Been googling for a long time now, and all that I found is the one for vmware, which is very different from xen/kvm.
I don't want to use vmware if I can help it, I've already got a stable system using kvm and I can swicth to xen anytime.
I've been busy and away from the forum for some time, so excuse my late reply.

I suggest you download and run Passmark. Under KVM it usually fails to run all tests, unless with some specific option that I can't remember. You may want to search for KVM and passmark as I remember I posted on that, I believe in the Archlinux forum.

I never had an issue with Xen running Passmark. The reason I mention that is perhaps the game works under Xen (have you tried?), or by using some config option in KVM.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
powerhouse
Level 6
Level 6
Posts: 1141
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

Just a short update: I've installed Linux Mint 17.1 now and run Xen with the xl toolstack, together with my Nvidia Quadro 2000 for my Windows 7 domU. Everything seems to work fine so far.

I hope to update my how-to soon to reflect the minor changes needed to get Xen / VGA passthrough running on Linux Mint 17.1.

Cheers!
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
thehappygrub

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by thehappygrub »

wollymammoth wrote:Hey powerhouse, another happy customer :)

My setup:
mobo: Asrock z77 extreme4-m
RAM: 16GB (currently 2G for Dom0, rest DomU)
CPU: i7-3770
GPU: hd7950 for domU, integrated for dom0
OS: Mint16 x64 (mate), win7 x64
storage: 128GB SSD for both OSes, 3TB for data (samba share)


I did run into one hiccup trying to blacklist the 'radeon' driver from picking up my primary GPU, when setting:

Code: Select all

GRUB_CMDLINE_LINUX_DEFAULT="..."
- the 'nomodeset' option (by itself or in conjunction with 'radeon.blacklist=1') causes X to crash on boot dropping me to a non-graphical login ('nomodeset' was not there by default).
- 'radeon.blacklist=1' without 'nomodeset' seems to do nothing(?) and dom0 claims the hd7950
- adding 'blacklist radeon' to /etc/modprobe.d/blacklist.conf did nothing either...

I only achieved success following the first answer in https://ask.fedoraproject.org/en/questi ... on-module/
resulting in the mutant command line:

Code: Select all

GRUB_CMDLINE_LINUX_DEFAULT="radeon.blacklist=1 radeon.modeset=0 rd.driver.blacklist=radeon rd.blacklist=radeon quiet splash"
most of the options are likely not needed, but it works... so i'm not touching it :)


another (obvious in retrospect) point was to make sure to set the integrated/non-passed graphics adaptor as the default output device in the BIOS... :oops:

---------------------------

Also, I managed to get sound working without extra hardware using the jack audio system.
There is a small but barely noticable lag in the sound (completely un-optimised/fiddled-with settings), and quality is very good.

I have uploaded a how-to on what I did but its "waiting for a moderator" before its viewable...
#
Hi Wolly

I'am having issues with this board and pt-mem issues for the ati passthrough on bios version 1.8,could you provide the bios version you are using,and maybe the vt-d settings.
Did you have any vt-d parameters in the grub default config?.

Thanks in advance.
wollymammoth

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by wollymammoth »

thehappygrub wrote: Hi Wolly

I'am having issues with this board and pt-mem issues for the ati passthrough on bios version 1.8,could you provide the bios version you are using,and maybe the vt-d settings.
Did you have any vt-d parameters in the grub default config?.

Thanks in advance.
Hi thehappygrub,

I'm using "UEFI version: Z77 Extreme4-M P1.80" as reported on the UEFI main page,
I believe most UEFI settings were left at defaults and the only VT-d options are enable/disable:
in advanced->CPU config
- set "Intel Virtualisation Technology" to "enabled"
- ensure at the top of the page it reports "VT-x supported"
in advanced->North Bridge configuration
- set VT-d to "enabled"
- ensure at the top of the page it reports "VT-d supported"

my grub default config is exactly that in the guide with no extra options except for the mutant GRUB_CMDLINE_LINUX_DEFAULT

I am unsure what you mean by "pt-mem issues" and google didnt help me there either :?:
thehappygrub

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by thehappygrub »

Thanks and cheers for the reply.

I have had the amd systems running fine on the same image file ,and also the same system ( same m/b and xeon processor) before on win7 x64.
ubuntu posts here if you are looking for the A8 setup -http://ubuntuforums.org/showthread.php? ... st13000378.

The amd system were running fine on the win8 x64 also -fx8350 , with faster direct compute performance ( direct render /opengl) than
native windows with a r9-280x and quadro as the host screen,i did tweak the xen domain weights to 512 and latency to 0 in the configs.
Even had 3 screens in catalyst running,working on it ,playing on it etc,performance better and in some cases on 2% less than win native.

The current system iam trying to run win8-64 on is the old amd image i have been using very well for approx a year,but its become a pain
now i have setup the image on the intel system.

Appreciate the info it points to win8 on the intel or the build i have on the intel system.

BTW - the jack issue you are trying to conquer can be done by routing the audio to the usb/serial and via slip ,a very old method
in the days of old ,ie modems and the slip protocol.

See : http://en.wikipedia.org/wiki/Open_Sound_Control
thehappygrub

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by thehappygrub »

Asrock Z77-Extreme 4-M - Legacy kernel

I have converted from the ubuntu to the Linux mint protestant !,and experiencing the same issues with pci pass-through that I didn't have in kernel 3.10
and xen 4.2 with this exact hardware, was trying to ascertain whether I was using UEFI previously,does this really make that much difference with pass through?.
I have not found any reports of this making any difference with xen.

This machine works fine on 8.1 without passing through the vga or pci,only usb abd sound,the same image on the amd machine was passing the hd7950
through on the boot slot ! ,non-uefi.If i turn off Intels machine memory injection or adjacent line prefetch or caching ( bios settings) protection, the w8.1 and
w8 both fail to boot continually looping.

Has anyone here using pci or vga pass through on a legacy booting machine i.e. not UEFI?.

I have been experimenting with the advanced xl functions such as iomem maps and io ports,and
pci stubs,at present I have no pass through on the Asrock Z77-Extreme 4-M,usb and sound working fine.

Over a year ago I had this working on LVM and Xen with pass through,I have tried regressing to earlier kernels.

Some Logs () from a gfx-passthrough onto monitor 2 and display 2,this resulted in a hard lock-up of Dom0 & U.

The log indicates it is still looking for some sort of pci multiplexer that doesn't exist ,is this overhang from the older
Xen kernels ?,although 1b is the sound card ,is the pci control in xen caching /saving previous?,xenstore autosave
is off,although the xen forums are saying ignore this message,which rings true if I look at 00:14 ( usb) which
passes through successfully.

See ; /dev/xen/pci_iomul in log below
---------------------------------------------------------

/var/log/xen/qemu-dm-win8.log

pci_intx: intx=2
register_real_device: Real physical device 02:00.1 registered successfully!
IRQ type = INTx
dm-command: hot insert pass-through pci dev
register_real_device: Assigning real physical device 00:1b.0 ...
register_real_device: Disable MSI translation via per device option
register_real_device: Enable power management
pt_iomul_init: Error: pt_iomul_init can't open file /dev/xen/pci_iomul: No such file or directory: 0x0:0x1b.0x0
pt_register_regions: IO region registered (size=0x00004000 base_addr=0xf7310004)
pci_intx: intx=1
register_real_device: Real physical device 00:1b.0 registered successfuly!
IRQ type = INTx
dm-command: hot insert pass-through pci dev
register_real_device: Assigning real physical device 00:14.0 ...
register_real_device: Disable MSI translation via per device option
register_real_device: Enable power management
pt_iomul_init: Error: pt_iomul_init can't open file /dev/xen/pci_iomul: No such file or directory: 0x0:0x14.0x0
pt_register_regions: IO region registered (size=0x00010000 base_addr=0xf7300004)
pci_intx: intx=1
register_real_device: Real physical device 00:14.0 registered successfuly!
IRQ type = INTx
pt_iomem_map: e_phys=e0000000 maddr=d0000000 type=8 len=268435456 index=0 first_map=1
cirrus vga map change while on lfb mode
pt_iomem_map: e_phys=f3000000 maddr=f7200000 type=0 len=262144 index=2 first_map=1
pt_iomem_map: e_phys=f3060000 maddr=f7300000 type=0 len=65536 index=0 first_map=1
pt_iomem_map: e_phys=f3070000 maddr=f7260000 type=0 len=16384 index=0 first_map=1
pt_iomem_map: e_phys=f3074000 maddr=f7310000 type=0 len=16384 index=0 first_map=1
pt_ioport_map: e_phys=c100 pio_base=d000 len=256 index=4 first_map=1
mapping vram to f0000000 - f0400000
platform_fixed_ioport: changed ro/rw state of ROM memory area. now is rw state.
platform_fixed_ioport: changed ro/rw state of ROM memory area. now is ro state.
pt_pci_read_config: [00:04:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]


Grub Config & Guest Configs

GRUB_CMDLINE_XEN="intel_iommu=on availmem=11000M iommu=on iommu_inclusive_mapping=1 iommu=verbose iomem=relaxed
noirqbalance dom0_mem=4100M,max:4100M iommu_dump,debug loglvl=all guest_loglvl=all apic_verbosity=debug e820-verbose=true"

Win8.cfg is quite standard except for these ;

e820_host= 1
hap= 0
shadow_memory=256
nx=1
display= 2
monitor= 2
pci_permissive= 1
authority= 1
display= 2
monitor= 2
# hd-7950 io ranges +1
# ioports=[ 'd000-0xd0ff' ]
# hd-7950 ranges +1
# iomem=[ 'd0000000,10000000' , 'f7200000,40000' ,'d000,100', 'f7240000,20000','f7260000,4000' ]
# latency=0
# cpu_weight= "512"
memory = "4298"
maxmem= "4298"
# pci=[ '0000:02:00.0,msitranslate=1,power_mgmt=1,permissive=1', '0000:02:00.1,msitranslate=1,power_mgmt=1,permissive=1', '00:14.0' ]

Is compiling a custom kernel and compiling a custom xen kernel worth it?,with pci hard codes and bios inclusions etc ,bugger i havent had to get the kernel
pliers out since catching -bumblebee dev days :).

Any ideas anyone ? ,appreciate some help on this .

Regards
Locked

Return to “Virtual Machines”