HOW-TO make dual-boot obsolete using XEN VGA passthrough

Questions about virtualization software
Forum rules
Before you post read how to get help. Topics in this forum are automatically closed 6 months after creation.
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

Boggler wrote:@powerhouse

When choosing the normal boot option in grub, all the features seem to be recognized. svm is now in the list of supported features, AMD-Vi as well and IOMMU somehow:
...

Code: Select all

[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.13.11.2 root=/dev/mapper/vgmint-root ro iommu=1 radeon.blacklist=1 iommu=1 kopt=root=/dev/mapper/vgmint-root
[    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.13.11.2 root=/dev/mapper/vgmint-root ro iommu=1 radeon.blacklist=1 iommu=1 kopt=root=/dev/mapper/vgmint-root
[    0.000000] Please enable the IOMMU option in the BIOS setup
[    2.017670] AMD-Vi: Found IOMMU at 0000:00:00.2 cap 0x40
Your kernel command line looks a bit weird. Here is mine for comparison:

Code: Select all

Kernel command line: placeholder root=/dev/mapper/lm13-root ro nouveau.blacklist=1 quiet nomodeset
Check your /etc/default/grub file to see if there is something wrong. After any changes in that file, run

Code: Select all

sudo update-grub
To check the IOMMU support while running Xen, use:

Code: Select all

sudo xl dmesg
assuming you use the xl toolstack, else use sudo xm dmesg. The important part is here (this is my machine - yours will be different!):

Code: Select all

(XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
(XEN) Intel VT-d Snoop Control enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Shared EPT tables enabled.
(XEN) I/O virtualisation enabled
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

Just checked the qemu release notes for the latest qemu release 2.0:
Xen

PCI passthrough of devices with a ROM now works.
For more info see http://wiki.qemu.org/ChangeLog/2.0.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
FastRealm

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by FastRealm »

@Powerhouse

The KVM settings,

sudo qemu-system-x86_64 \
-bios /usr/share/qemu/bios.bin -vga none \
..........(etc etc)
-net nic,model=virtio,macaddr=00:16:3e:00:01:01 -net bridge,br=virbr0


How do you write a script or make a shortcut to run the command above?
Where do you save the script to?
I got everything working except this last part which i got no idea where to start.
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

FastRealm wrote:@Powerhouse

The KVM settings,

sudo qemu-system-x86_64 \
-bios /usr/share/qemu/bios.bin -vga none \
..........(etc etc)
-net nic,model=virtio,macaddr=00:16:3e:00:01:01 -net bridge,br=virbr0


How do you write a script or make a shortcut to run the command above?
Where do you save the script to?
I got everything working except this last part which i got no idea where to start.
I posted my script here: http://forums.linuxmint.com/viewtopic.p ... 40#p878200.

Just copy and paste it into an empty text file, make the necessary changes to reflect your hardware/preferences, save the file and make it executable. You can create a shortcut to the script as well.

Make sure you created/modified the necessary configuration files for PCI/VGA passthrough, such as the /etc/vfio-pci.cfg file. See https://bbs.archlinux.org/viewtopic.php?id=162768& for detailed instructions. You may not need to patch the kernel or qemu - I didn't.

If you got it working from the command line, pasting the command into my script and removing my qemu command should do the trick.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
Boggler

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by Boggler »

Hi again and sorry for not responding for so long :roll:
powerhouse wrote:Your kernel command line looks a bit weird.
I assume you mean that there are two iommu parameters. I read that somewhere (I don't even know in what kontext it was), probably it was a tutorial for kvm :D but it's not relevant anymore, see below.

I tried a few different graphics cards (as there were AMD-Vi: io_page_fault errors coming from the ID of my 1st graphics card) and ended up installing my system from scratch (I didn't have anything on it except the till then unsuccessful xen-attempt). I then tried patching and compiling a kernel and finally got v3.12.2 working (3.15.2 wouldn't work and 3.12.2 was reported working with my motherboard and a earlier version of the patch). Now I don't get any io_page_faults from my graphics card anymore, and with the parameter iommu=pt even the Marvell controllers work and don't produce io_page_fault errors. IOMMU still wouldn't work though (as to xl dmesg output).
I then google'd the appropriate lines and found that IOMMU was automatically disabled because of 'conflicting IO-APIC tables' (I actually saw a post from you on that topic while researching :D). With xen parameter iommu=no-amd-iommu-perdev-intremap that is also out of the way and IOMMU now works fine, as well as creating a xen domain with vga pasthrough.

It's a bit sad how close/easy the solution was, just a single parameter and IOMMU works...
At least I learned a few things (I'm fairly new to Linux).

My problem at the moment is that my UI is stuck after logging in, just like you wrote in Step 15. I blacklisted the radeon driver (1st GPU is still nvidia@nouveau; 2nd is ATI@radeon), which workes with wollymammoths kernel parameters (I checked, my GPU is running at pciback and the IDs are listed as assignable by xl). I also double-checked the to-be-assigned PCI IDs, they are correct.
I'll later reverse some of the steps to check at which point exactly I can't login outside of the tty anymore.
Have you got another idea what could be the problem here?

best regards
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

Boggler wrote:...My problem at the moment is that my UI is stuck after logging in, just like you wrote in Step 15. I blacklisted the radeon driver (1st GPU is still nvidia@nouveau; 2nd is ATI@radeon), which workes with wollymammoths kernel parameters (I checked, my GPU is running at pciback and the IDs are listed as assignable by xl). I also double-checked the to-be-assigned PCI IDs, they are correct.
I'll later reverse some of the steps to check at which point exactly I can't login outside of the tty anymore.
Have you got another idea what could be the problem here?

best regards
I'm glad you solved the IOMMU issue, I definitely need to update my how-to to include some possible kernel boot options when things go wrong.

Have a look here: http://forums.linuxmint.com/viewtopic.p ... 00#p846397:

1. Identify the PCI IDs of your domU GPU (the one you want to pass through). Turn of the PC and remove the domU GPU.

2. Reinstall Linux Mint 17 and Xen, together with the driver for your dom0 GPU. Note: If you want to run the Nvidia proprietary driver under dom0, you need to follow these instructions (modify the commands for Linux Mint/Ubuntu). Reboot to check if it works.

3. Follow my instructions in the how-to up until step 11, except that you don't change the toolstack to xm!

4. Add the "xen-pciback passthrough=1 hide=(02:00.0)(02:00.1)" using the PCI ID of your domU GPU to the /etc/initramfs-tools/modules file (see step 11 of the how-to) and run update-initramfs. Turn of the PC and install your domU graphics card.

5. Turn on the PC and hopefully you have a dom0 GUI and the secondary graphics card assigned to pciback, ready for passthrough.

6. Continue with the how-to, but change the Windows domU config file to resemble the following, making the changes to meet your requirements:

Code: Select all

builder='hvm'
memory = 8192
name = 'win7'
vcpus=6
pae=1
acpi=1
apic=1
on_xend_stop='shutdown'
vif = [ 'mac=00:16:3e:68:01:0e,type=vif,bridge=xenbr0' ]
disk = [ '/dev/mapper/guest-win7,raw,hda,rw' , '/home/user/ISOs/win7.iso,raw,hdd,devtype=cdrom' ]
device_model_version = 'qemu-xen-traditional'
boot='dc'
sdl=0
vnc=1
vncpasswd='password'
serial='pty'
tsc_mode=0
viridian=1
usb=0
usbdevice='tablet'
gfx_passthru=0
pci=[ '02:00.0', '02:00.1' ]
localtime=1
xen_platform_pci=1
pci_power_mgmt=1
The main difference to my original config file in the how-to (using the xm toolstack) is: device_model_version = 'qemu-xen-traditional'. The new qemu doesn't yet support PCI passthrough (at least not the one in the packages). Note: You could download the latest qemu 2 (see one of my posts above), compile it and give it a try - it should support PCI/VGA passthrough.

Hope it helps.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
Boggler

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by Boggler »

Thanks!
I tried it (reinstalling everything without the 2nd graphics card) but in the end, I got the same result.
The graphics card is assigned to the pciback driver, but I still can't get an UI working.
I added the pci-stub module to the kernel before compiling this time, and will now give kvm a try. A friend reccommended it but I wanted to try xen first because your guide is really detailed and thorough, unlike the kvm guides available.

Maybe I'll try xen again when I have a better 1st graphics card. The one I have now is just to get it working somehow and will be replaced as it doesn't support three displays and produces artifacts because of what I think is overheating VRAM :D

Thanks for all & best regards
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

Boggler wrote:Thanks!
I tried it (reinstalling everything without the 2nd graphics card) but in the end, I got the same result.
The graphics card is assigned to the pciback driver, but I still can't get an UI working.
I added the pci-stub module to the kernel before compiling this time, and will now give kvm a try. A friend reccommended it but I wanted to try xen first because your guide is really detailed and thorough, unlike the kvm guides available.

Maybe I'll try xen again when I have a better 1st graphics card. The one I have now is just to get it working somehow and will be replaced as it doesn't support three displays and produces artifacts because of what I think is overheating VRAM :D

Thanks for all & best regards
I'm not sure if compiling your own kernel is the best way to go. Have you tried the standard kernel shipped with Linux Mint or Ubuntu? I'm currently at LM16, but have tested LM17 as well. It might be a good idea to check the forum for help on graphics card / GUI issues. I'm afraid I can't be of much help on that. When you reinstalled Xen, did you also format your /home directory? If not, there might be some leftover desktop configuration files that could get in the way of loading the desktop. It's just a thought.

As to kvm, have a look a few posts above where I posted a link to a kvm how-to on the archlinux forum, together with my guest startup script for kvm. I've managed to get primary VGA passthrough working with kvm and a AMD 7770 card, but haven't been successful yet with my Nvidia Quadro 2000 (though I found a possible solution). Hopefully one day I will be able to share a kvm how-to for Linux Mint (right now I'm not really familiar with it).
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
Chapapa

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by Chapapa »

hello,

im pretty new to xen and linux but i have been working for some time on a setup and finally got my hardware.
my specs are:
cpu: i5-4670 onboard vga
mainboard: asrock z87 extreme 3
ram: 8gb corsair
ssd: 840 Evo 120GB
hdd: 2tb seagate
gpu1: hd 7850
gpu2: hd 7850
pci: usb 2.0 controller
pci: usb 2.0 controller

edit:
xen: 4.4
linuxmint 16

im currently running:
dom0: 2 cores, 2gb ram, onboard vga,
domu1: win7 64 bit, 4gb, gpu1,
domu2: win7 64bit, 2gb, gpu2

actually everything is running fine, except usb passthrough.. and i need some help.. i have been reading the last days to resolve the issue but to no luck.
here some details:
lspci output:
00:00.0 Host bridge: Intel Corporation 4th Gen Core Processor DRAM Controller (rev 06)
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller (rev 06)
00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x8 Controller (rev 06)
00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller (rev 06)
00:03.0 Audio device: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller (rev 06)
00:14.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB xHCI (rev 05)
00:16.0 Communication controller: Intel Corporation 8 Series/C220 Series Chipset Family MEI Controller #1 (rev 04)
00:19.0 Ethernet controller: Intel Corporation Ethernet Connection I217-V (rev 05)
00:1a.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #2 (rev 05)
00:1b.0 Audio device: Intel Corporation 8 Series/C220 Series Chipset High Definition Audio Controller (rev 05)
00:1c.0 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #1 (rev d5)
00:1c.6 PCI bridge: Intel Corporation 82801 PCI Bridge (rev d5)
00:1d.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #1 (rev 05)
00:1f.0 ISA bridge: Intel Corporation Z87 Express LPC Controller (rev 05)
00:1f.2 SATA controller: Intel Corporation 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode] (rev 05)
00:1f.3 SMBus: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller (rev 05)
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Pitcairn PRO [Radeon HD 7850]
01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series]
02:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Pitcairn PRO [Radeon HD 7850]
02:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series]
04:00.0 PCI bridge: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge (rev 04)
05:00.0 USB controller: NEC Corporation OHCI USB Controller (rev 41)
05:00.1 USB controller: NEC Corporation OHCI USB Controller (rev 41)
05:00.2 USB controller: NEC Corporation uPD72010x USB 2.0 Controller (rev 02)
05:02.0 USB controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 62)
05:02.1 USB controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 62)
05:02.2 USB controller: VIA Technologies, Inc. USB 2.0 (rev 65)

i have used the passthrough on following devices:
00:1a.0 usb
00:1b.0 hd audio
00:1d.0 usb
01:00.0 vga
01:00.1 audio
02:00.0 vga
02:00.1 audio
05:00.0 pci usb #1
05:00.1
05:00.2
05:02.0 pci usb #2
05:02.1
05:02.2

well i can see the controller, but the devices dont show up in domu but still in dom0. i even tried xm usb-add host:xxxx:yyyy. it only worked when the usb device was plugged in 05:02:0 and domu1 (with 05:00.0) was not running. <-- so there is a controller managing all pci slots? how can i manage to have 2 guest systems running with usb passthrough.

regards
chapapa
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

@Chapapa: 05:00.*, 05:02.*, and 00:1a.0 (or 00:1d.0 ...) seem to be different controllers. Can you assign each of them to a different dom?, i.e. dom0 (no passthrough), dom1 and dom2?

Please post your

Code: Select all

lspci -k
output.

Here some possible causes and solutions:

http://mediaserver8.blogspot.co.il/2014 ... rough.html.

Sometimes there are problems with PCI bridges. See also http://wiki.xen.org/wiki/Xen_PCI_Passth ... CI_devices and http://wiki.xen.org/wiki/VTd_HowTo, particularly the iommu options in grub.

I should be doing something about this (collect more info and expand the how-to), but can't find the time.

In the worst case, get a USB PCIe controller card (ASMedia chips are usually OK) and pass that one through. It shouldn't cost much and may save you a lot of time.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
Chapapa

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by Chapapa »

hi powerhouse,

well i cann assign 00:1a.0 or 00:1d.0 to different domu, but the devices dont show in the domu, still in dom0. i couldnt assign 05:00.0 and 05:02.0 at different domu (had to be co-assigned). i changed something in a config file (i will edit the specific file, cause i forgot the name). here is my lspci -k output:

00:00.0 Host bridge: Intel Corporation 4th Gen Core Processor DRAM Controller (rev 06)
Subsystem: ASRock Incorporation Device 0c00
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller (rev 06)
Kernel driver in use: pcieport
00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x8 Controller (rev 06)
Kernel driver in use: pcieport
00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller (rev 06)
Subsystem: ASRock Incorporation Device 0412
Kernel driver in use: i915
00:03.0 Audio device: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller (rev 06)
Subsystem: ASRock Incorporation Device 0c0c
Kernel driver in use: snd_hda_intel
00:14.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB xHCI (rev 05)
Subsystem: ASRock Incorporation Device 8c31
Kernel driver in use: xhci_hcd
00:16.0 Communication controller: Intel Corporation 8 Series/C220 Series Chipset Family MEI Controller #1 (rev 04)
Subsystem: ASRock Incorporation Device 8c3a
Kernel driver in use: mei_me
00:19.0 Ethernet controller: Intel Corporation Ethernet Connection I217-V (rev 05)
Subsystem: ASRock Incorporation Device 153b
Kernel driver in use: e1000e
00:1a.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #2 (rev 05)
Subsystem: ASRock Incorporation Device 8c2d
Kernel driver in use: ehci-pci
00:1b.0 Audio device: Intel Corporation 8 Series/C220 Series Chipset High Definition Audio Controller (rev 05)
Subsystem: ASRock Incorporation Device c892
Kernel driver in use: pciback
00:1c.0 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #1 (rev d5)
Kernel driver in use: pcieport
00:1c.6 PCI bridge: Intel Corporation 82801 PCI Bridge (rev d5)
00:1d.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #1 (rev 05)
Subsystem: ASRock Incorporation Device 8c26
Kernel driver in use: ehci-pci
00:1f.0 ISA bridge: Intel Corporation Z87 Express LPC Controller (rev 05)
Subsystem: ASRock Incorporation Device 8c44
Kernel driver in use: lpc_ich
00:1f.2 SATA controller: Intel Corporation 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode] (rev 05)
Subsystem: ASRock Incorporation Device 8c02
Kernel driver in use: ahci
00:1f.3 SMBus: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller (rev 05)
Subsystem: ASRock Incorporation Device 8c22
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Pitcairn PRO [Radeon HD 7850]
Subsystem: Hightech Information System Ltd. Device 201c
Kernel driver in use: pciback
01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series]
Subsystem: Hightech Information System Ltd. Device aab0
Kernel driver in use: pciback
02:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Pitcairn PRO [Radeon HD 7850]
Subsystem: Hightech Information System Ltd. Device 201c
Kernel driver in use: pciback
02:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series]
Subsystem: Hightech Information System Ltd. Device aab0
Kernel driver in use: pciback
04:00.0 PCI bridge: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge (rev 04)
05:00.0 USB controller: NEC Corporation OHCI USB Controller (rev 41)
Subsystem: NEC Corporation USB Controller
Kernel driver in use: pciback
05:00.1 USB controller: NEC Corporation OHCI USB Controller (rev 41)
Subsystem: NEC Corporation USB Controller
Kernel driver in use: ohci-pci
05:00.2 USB controller: NEC Corporation uPD72010x USB 2.0 Controller (rev 02)
Subsystem: NEC Corporation uPD72010x USB 2.0 Controller
Kernel driver in use: ehci-pci
05:02.0 USB controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 62)
Subsystem: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller
Kernel driver in use: pciback
05:02.1 USB controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 62)
Subsystem: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller
Kernel driver in use: uhci_hcd

thanks for the links.
kind regards

chapapa
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

@Chapapa:

Code: Select all

05:00.0 USB controller: NEC Corporation OHCI USB Controller (rev 41)
Subsystem: NEC Corporation USB Controller
Kernel driver in use: pciback
and

Code: Select all

05:02.0 USB controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 62)
Subsystem: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller
Kernel driver in use: pciback
are the only USB controllers assigned to pciback. Are you able to pass them to your domU?
Can you share your domU config files that go with that? Perhaps there is an entry missing.

Have you tried passing through the entire 05:00.* and/or 05:02.* ranges? See the links I posted in my previous reply.

Don't give up! I ran into similar issues. Sometimes reordering PCIe devices (cards) can do the trick (well, in my case at least). You should also scan your dmesg log file to see if there is some odd entry that may point to the problem, as well as the log files in /var/log/xen. Also, check

Code: Select all

sudo xm dmesg
(or sudo xl dmesg if you use xl).
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
Chapapa

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by Chapapa »

hi powerhouse,

i encountered another problem, my vga stops working after restarting domu. i use xm, but switched to xl shortly to test usb under another toolstack (so i think something is mixed up). i will delete everything now and restart the whole process. wish me luck. i will write as soon as everything is fresh and new.

kind regards

chapapa
Chapapa

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by Chapapa »

So i followed this guide and now the vgas are working like a charm. But again usb is giving me a headache.
i assume that there are only two usb controllers on my mainboard. 00:14.0 which ist the onboard usb and one pci controller which controlls both pci usb slots.
it doesnt make a difference when i change the onboard usb ports, it always shows the devices under 00:14:0.(i tried passing through 00:14.0, guess what.. no usb port was working in linux ^^).

Now this is what happens:
If i start dom1 with 05:00.0 passthrough it works. When i start dom2 simultaneously with 05:02.0 it gives me an error: "Using config file "/etc/xen/win7b.cfg".
Error: pci: 0000:05:00.0 must be co-assigned to the same guest with 0000:05:02.0". so again i assume there is a pci controller behind both slots. i tried "pci-passthrough-strict-check no" in /etc/xen/xend-config.sxp. the error didnt show up, i could see a controller in dom2 but the devices did not work.

i will give you some more details about my specs:

xen 4.3 not 4.4 like in a previous post.
usb-devices: 05:00.0 and 05:02.0 are in the pciback.conf so are not listed here
T: Bus=01 Lev=00 Prnt=00 Port=00 Cnt=00 Dev#= 1 Spd=480 MxCh= 2
D: Ver= 2.00 Cls=09(hub ) Sub=00 Prot=00 MxPS=64 #Cfgs= 1
P: Vendor=1d6b ProdID=0002 Rev=03.11
S: Manufacturer=Linux 3.11.0-12-generic ehci_hcd
S: Product=EHCI Host Controller
S: SerialNumber=0000:00:1a.0
C: #Ifs= 1 Cfg#= 1 Atr=e0 MxPwr=0mA
I: If#= 0 Alt= 0 #EPs= 1 Cls=09(hub ) Sub=00 Prot=00 Driver=hub

T: Bus=01 Lev=01 Prnt=01 Port=00 Cnt=01 Dev#= 2 Spd=480 MxCh= 6
D: Ver= 2.00 Cls=09(hub ) Sub=00 Prot=01 MxPS=64 #Cfgs= 1
P: Vendor=8087 ProdID=8008 Rev=00.05
C: #Ifs= 1 Cfg#= 1 Atr=e0 MxPwr=0mA
I: If#= 0 Alt= 0 #EPs= 1 Cls=09(hub ) Sub=00 Prot=00 Driver=hub

T: Bus=02 Lev=00 Prnt=00 Port=00 Cnt=00 Dev#= 1 Spd=480 MxCh= 2
D: Ver= 2.00 Cls=09(hub ) Sub=00 Prot=00 MxPS=64 #Cfgs= 1
P: Vendor=1d6b ProdID=0002 Rev=03.11
S: Manufacturer=Linux 3.11.0-12-generic ehci_hcd
S: Product=EHCI Host Controller
S: SerialNumber=0000:00:1d.0
C: #Ifs= 1 Cfg#= 1 Atr=e0 MxPwr=0mA
I: If#= 0 Alt= 0 #EPs= 1 Cls=09(hub ) Sub=00 Prot=00 Driver=hub

T: Bus=02 Lev=01 Prnt=01 Port=00 Cnt=01 Dev#= 2 Spd=480 MxCh= 8
D: Ver= 2.00 Cls=09(hub ) Sub=00 Prot=01 MxPS=64 #Cfgs= 1
P: Vendor=8087 ProdID=8000 Rev=00.05
C: #Ifs= 1 Cfg#= 1 Atr=e0 MxPwr=0mA
I: If#= 0 Alt= 0 #EPs= 1 Cls=09(hub ) Sub=00 Prot=00 Driver=hub

T: Bus=07 Lev=00 Prnt=00 Port=00 Cnt=00 Dev#= 1 Spd=480 MxCh=14
D: Ver= 2.00 Cls=09(hub ) Sub=00 Prot=01 MxPS=64 #Cfgs= 1
P: Vendor=1d6b ProdID=0002 Rev=03.11
S: Manufacturer=Linux 3.11.0-12-generic xhci_hcd
S: Product=xHCI Host Controller
S: SerialNumber=0000:00:14.0
C: #Ifs= 1 Cfg#= 1 Atr=e0 MxPwr=0mA
I: If#= 0 Alt= 0 #EPs= 1 Cls=09(hub ) Sub=00 Prot=00 Driver=hub

T: Bus=07 Lev=01 Prnt=01 Port=00 Cnt=01 Dev#= 15 Spd=12 MxCh= 0
D: Ver= 1.10 Cls=00(>ifc ) Sub=00 Prot=00 MxPS= 8 #Cfgs= 1
P: Vendor=1532 ProdID=0009 Rev=21.00
S: Manufacturer=Tempest
S: Product=Habu Mouse
C: #Ifs= 2 Cfg#= 1 Atr=a0 MxPwr=200mA
I: If#= 0 Alt= 0 #EPs= 1 Cls=03(HID ) Sub=00 Prot=02 Driver=usbhid
I: If#= 1 Alt= 0 #EPs= 1 Cls=03(HID ) Sub=01 Prot=01 Driver=usbhid

T: Bus=07 Lev=01 Prnt=01 Port=01 Cnt=02 Dev#= 16 Spd=12 MxCh= 4
D: Ver= 1.10 Cls=09(hub ) Sub=00 Prot=00 MxPS= 8 #Cfgs= 1
P: Vendor=046d ProdID=c223 Rev=01.03
S: Manufacturer=Logitech
S: Product=Logitech G15 Keyboard
C: #Ifs= 1 Cfg#= 1 Atr=80 MxPwr=100mA
I: If#= 0 Alt= 0 #EPs= 1 Cls=09(hub ) Sub=00 Prot=00 Driver=hub

T: Bus=07 Lev=02 Prnt=16 Port=00 Cnt=01 Dev#= 17 Spd=1.5 MxCh= 0
D: Ver= 2.00 Cls=00(>ifc ) Sub=00 Prot=00 MxPS= 8 #Cfgs= 1
P: Vendor=046d ProdID=c221 Rev=01.90
S: Product=Gaming Keyboard
C: #Ifs= 2 Cfg#= 1 Atr=a0 MxPwr=100mA
I: If#= 0 Alt= 0 #EPs= 1 Cls=03(HID ) Sub=01 Prot=01 Driver=usbhid
I: If#= 1 Alt= 0 #EPs= 1 Cls=03(HID ) Sub=00 Prot=00 Driver=usbhid

T: Bus=07 Lev=02 Prnt=16 Port=03 Cnt=02 Dev#= 18 Spd=12 MxCh= 0
D: Ver= 2.00 Cls=00(>ifc ) Sub=00 Prot=00 MxPS= 8 #Cfgs= 1
P: Vendor=046d ProdID=c222 Rev=01.03
S: Manufacturer=G15 Keyboard
S: Product=G15 Keyboard
C: #Ifs= 1 Cfg#= 1 Atr=80 MxPwr=100mA
I: If#= 0 Alt= 0 #EPs= 2 Cls=03(HID ) Sub=00 Prot=00 Driver=usbhid

T: Bus=07 Lev=01 Prnt=01 Port=04 Cnt=03 Dev#= 4 Spd=480 MxCh= 0
D: Ver= 2.00 Cls=00(>ifc ) Sub=00 Prot=00 MxPS=64 #Cfgs= 1
P: Vendor=1307 ProdID=0165 Rev=01.00
S: Manufacturer=USBest Technology
S: Product=USB Mass Storage Device
S: SerialNumber=00000000000004
C: #Ifs= 1 Cfg#= 1 Atr=80 MxPwr=98mA
I: If#= 0 Alt= 0 #EPs= 3 Cls=08(stor.) Sub=06 Prot=50 Driver=usb-storage

T: Bus=08 Lev=00 Prnt=00 Port=00 Cnt=00 Dev#= 1 Spd=5000 MxCh= 6
D: Ver= 3.00 Cls=09(hub ) Sub=00 Prot=03 MxPS= 9 #Cfgs= 1
P: Vendor=1d6b ProdID=0003 Rev=03.11
S: Manufacturer=Linux 3.11.0-12-generic xhci_hcd
S: Product=xHCI Host Controller
S: SerialNumber=0000:00:14.0
C: #Ifs= 1 Cfg#= 1 Atr=e0 MxPwr=0mA
I: If#= 0 Alt= 0 #EPs= 1 Cls=09(hub ) Sub=00 Prot=00 Driver=hub

lspci -k:
00:00.0 Host bridge: Intel Corporation 4th Gen Core Processor DRAM Controller (rev 06)
Subsystem: ASRock Incorporation Device 0c00
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller (rev 06)
Kernel driver in use: pcieport
00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x8 Controller (rev 06)
Kernel driver in use: pcieport
00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller (rev 06)
Subsystem: ASRock Incorporation Device 0412
Kernel driver in use: i915
00:03.0 Audio device: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller (rev 06)
Subsystem: ASRock Incorporation Device 0c0c
Kernel driver in use: snd_hda_intel
00:14.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB xHCI (rev 05)
Subsystem: ASRock Incorporation Device 8c31
Kernel driver in use: xhci_hcd
00:16.0 Communication controller: Intel Corporation 8 Series/C220 Series Chipset Family MEI Controller #1 (rev 04)
Subsystem: ASRock Incorporation Device 8c3a
Kernel driver in use: mei_me
00:19.0 Ethernet controller: Intel Corporation Ethernet Connection I217-V (rev 05)
Subsystem: ASRock Incorporation Device 153b
Kernel driver in use: e1000e
00:1a.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #2 (rev 05)
Subsystem: ASRock Incorporation Device 8c2d
Kernel driver in use: ehci-pci
00:1b.0 Audio device: Intel Corporation 8 Series/C220 Series Chipset High Definition Audio Controller (rev 05)
Subsystem: ASRock Incorporation Device c892
Kernel driver in use: pciback
00:1c.0 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #1 (rev d5)
Kernel driver in use: pcieport
00:1c.6 PCI bridge: Intel Corporation 82801 PCI Bridge (rev d5)
00:1d.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #1 (rev 05)
Subsystem: ASRock Incorporation Device 8c26
Kernel driver in use: ehci-pci
00:1f.0 ISA bridge: Intel Corporation Z87 Express LPC Controller (rev 05)
Subsystem: ASRock Incorporation Device 8c44
Kernel driver in use: lpc_ich
00:1f.2 SATA controller: Intel Corporation 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode] (rev 05)
Subsystem: ASRock Incorporation Device 8c02
Kernel driver in use: ahci
00:1f.3 SMBus: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller (rev 05)
Subsystem: ASRock Incorporation Device 8c22
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Pitcairn PRO [Radeon HD 7850]
Subsystem: Hightech Information System Ltd. Device 201c
Kernel driver in use: pciback
01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series]
Subsystem: Hightech Information System Ltd. Device aab0
Kernel driver in use: pciback
02:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Pitcairn PRO [Radeon HD 7850]
Subsystem: Hightech Information System Ltd. Device 201c
Kernel driver in use: pciback
02:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series]
Subsystem: Hightech Information System Ltd. Device aab0
Kernel driver in use: pciback
04:00.0 PCI bridge: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge (rev 04)
05:00.0 USB controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 62)
Subsystem: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller
Kernel driver in use: pciback
05:00.1 USB controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 62)
Subsystem: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller
Kernel driver in use: pciback
05:00.2 USB controller: VIA Technologies, Inc. USB 2.0 (rev 65)
Subsystem: VIA Technologies, Inc. USB 2.0 Controller
Kernel driver in use: pciback
05:02.0 USB controller: NEC Corporation OHCI USB Controller (rev 41)
Subsystem: NEC Corporation USB Controller
Kernel driver in use: pciback
05:02.1 USB controller: NEC Corporation OHCI USB Controller (rev 41)
Subsystem: NEC Corporation USB Controller
Kernel driver in use: pciback
05:02.2 USB controller: NEC Corporation uPD72010x USB 2.0 Controller (rev 02)
Subsystem: NEC Corporation uPD72010x USB 2.0 Controller
Kernel driver in use: pciback


pci-list-assignable-devices: --> as soon as i start dom1 all 05:00 and 05:02 devices disappear from this list.
0000:05:00.0
0000:05:00.1
0000:05:00.2
0000:05:02.0
0000:05:02.1
0000:05:02.2
0000:02:00.0
0000:02:00.1
0000:01:00.0
0000:01:00.1
0000:00:1b.0

dom1 config:
kernel = '/usr/lib/xen-default/boot/hvmloader'
builder='hvm'
memory = 4096
name = 'win7a'
vcpus=4
#pae=1
acpi=1
apic=1
on_xend_stop='shutdown'
vif = [ 'mac=00:16:3e:68:88:88,bridge=xenbr0' ]
disk = [ 'phy:/dev/mapper/win7-win7a,hda,w' , 'phy:/dev/mapper/xs-win7a,hdb,w'$
device_model = '/usr/lib/xen-default/bin/qemu-dm'
boot='cd'
sdl=0
vnc=1
vncpasswd=''
stdvga=0
serial='pty'
tsc_mode=0
viridian=1
usb=1
usbdevice='tablet'
gfx_passthru=0
pci=[ '01:00.0', '01:00.1' , '05:00.0' kernel = '/usr/lib/xen-default/boot/hvmloader'
builder='hvm'
memory = 4096
name = 'win7a'
vcpus=4
#pae=1
acpi=1
apic=1
on_xend_stop='shutdown'
vif = [ 'mac=00:16:3e:68:88:88,bridge=xenbr0' ]
disk = [ 'phy:/dev/mapper/win7-win7a,hda,w' , 'phy:/dev/mapper/xs-win7a,hdb,w'$
device_model = '/usr/lib/xen-default/bin/qemu-dm'
boot='cd'
sdl=0
vnc=1
vncpasswd=''
stdvga=0
serial='pty'
tsc_mode=0
viridian=1
usb=1
usbdevice='tablet'
gfx_passthru=0
pci=[ '01:00.0', '01:00.1' , '05:00.0' ]
localtime=1
pci_power_mgmt=1]
localtime=1
pci_power_mgmt=1


and finally
xm dmesg:

(XEN) Xen version 4.3.0 (Ubuntu 4.3.0-1ubuntu1.3) (stefan.bader@canonical.com) (gcc (Ubuntu/Linaro 4.8.1-10ubuntu8) 4.8.1) debug=n Wed Feb 19 15:33:00 UTC 2014
(XEN) Bootloader: GRUB 2.00-19ubuntu2.1
(XEN) Command line: placeholder iommu=1 dom0_mem=2048M
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN) Found 3 MBR signatures
(XEN) Found 3 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 000000000009d800 (usable)
(XEN) 000000000009d800 - 00000000000a0000 (reserved)
(XEN) 00000000000e0000 - 0000000000100000 (reserved)
(XEN) 0000000000100000 - 0000000087c8a000 (usable)
(XEN) 0000000087c8a000 - 0000000087c91000 (ACPI NVS)
(XEN) 0000000087c91000 - 0000000088884000 (usable)
(XEN) 0000000088884000 - 0000000088d83000 (reserved)
(XEN) 0000000088d83000 - 000000009bebd000 (usable)
(XEN) 000000009bebd000 - 000000009c0c3000 (reserved)
(XEN) 000000009c0c3000 - 000000009c101000 (usable)
(XEN) 000000009c101000 - 000000009c1aa000 (ACPI NVS)
(XEN) 000000009c1aa000 - 000000009cfff000 (reserved)
(XEN) 000000009cfff000 - 000000009d000000 (usable)
(XEN) 000000009f800000 - 00000000afa00000 (reserved)
(XEN) 00000000f8000000 - 00000000fc000000 (reserved)
(XEN) 00000000fec00000 - 00000000fec01000 (reserved)
(XEN) 00000000fed00000 - 00000000fed04000 (reserved)
(XEN) 00000000fed1c000 - 00000000fed20000 (reserved)
(XEN) 00000000fee00000 - 00000000fee01000 (reserved)
(XEN) 00000000ff000000 - 0000000100000000 (reserved)
(XEN) 0000000100000000 - 000000024f600000 (usable)
(XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA)
(XEN) ACPI: XSDT 9C185088, 0094 (r1 ALASKA A M I 1072009 AMI 10013)
(XEN) ACPI: FACP 9C18FC80, 010C (r5 ALASKA A M I 1072009 AMI 10013)
(XEN) ACPI: DSDT 9C1851B8, AAC3 (r2 ALASKA A M I 240 INTL 20091112)
(XEN) ACPI: FACS 9C1A8080, 0040
(XEN) ACPI: APIC 9C18FD90, 0072 (r3 ALASKA A M I 1072009 AMI 10013)
(XEN) ACPI: FPDT 9C18FE08, 0044 (r1 ALASKA A M I 1072009 AMI 10013)
(XEN) ACPI: ASF! 9C18FE50, 00A5 (r32 INTEL HCG 1 TFSM F4240)
(XEN) ACPI: SSDT 9C18FEF8, 0539 (r1 PmRef Cpu0Ist 3000 INTL 20051117)
(XEN) ACPI: SSDT 9C190438, 0AD8 (r1 PmRef CpuPm 3000 INTL 20051117)
(XEN) ACPI: SSDT 9C190F10, 01C7 (r1 PmRef LakeTiny 3000 INTL 20051117)
(XEN) ACPI: MCFG 9C1910D8, 003C (r1 ALASKA A M I 1072009 MSFT 97)
(XEN) ACPI: HPET 9C191118, 0038 (r1 ALASKA A M I 1072009 AMI. 5)
(XEN) ACPI: SSDT 9C191150, 036D (r1 SataRe SataTabl 1000 INTL 20091112)
(XEN) ACPI: SSDT 9C1914C0, 3493 (r1 SaSsdt SaSsdt 3000 INTL 20091112)
(XEN) ACPI: AAFT 9C194958, 02BB (r1 ALASKA OEMAAFT 1072009 MSFT 97)
(XEN) ACPI: DMAR 9C194C18, 00B8 (r1 INTEL HSW 1 INTL 1)
(XEN) ACPI: SSDT 9C194CD0, 0803 (r1 Intel_ IsctTabl 1000 INTL 20091112)
(XEN) System RAM: 7855MB (8044108kB)
(XEN) Domain heap initialised
(XEN) ACPI: 32/64X FACS address mismatch in FADT - 9c1a8080/0000000000000000, using 32
(XEN) Processor #0 7:12 APIC version 21
(XEN) Processor #2 7:12 APIC version 21
(XEN) Processor #4 7:12 APIC version 21
(XEN) Processor #6 7:12 APIC version 21
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
(XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
(XEN) Switched to APIC driver x2apic_cluster.
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 3399.066 MHz processor.
(XEN) Initing memory sharing.
(XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB.
(XEN) Intel VT-d iommu 1 supported page sizes: 4kB.
(XEN) Intel VT-d Snoop Control not enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Shared EPT tables not enabled.
(XEN) I/O virtualisation enabled
(XEN) - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using old ACK method
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 16 KiB.
(XEN) VMX: Supported advanced features:
(XEN) - APIC MMIO access virtualisation
(XEN) - APIC TPR shadow
(XEN) - Extended Page Tables (EPT)
(XEN) - Virtual-Processor Identifiers (VPID)
(XEN) - Virtual NMI
(XEN) - MSR direct-access bitmap
(XEN) - Unrestricted Guest
(XEN) - VMCS shadowing
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
(XEN) Brought up 4 CPUs
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Xen kernel: 64-bit, lsb, compat32
(XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x23f3000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 0000000240000000->0000000244000000 (491719 pages to be allocated)
(XEN) Init. ramdisk: 000000024b6c7000->000000024f5ffe00
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: ffffffff81000000->ffffffff823f3000
(XEN) Init. ramdisk: ffffffff823f3000->ffffffff8632be00
(XEN) Phys-Mach map: ffffffff8632c000->ffffffff8672c000
(XEN) Start info: ffffffff8672c000->ffffffff8672c4b4
(XEN) Page tables: ffffffff8672d000->ffffffff86764000
(XEN) Boot stack: ffffffff86764000->ffffffff86765000
(XEN) TOTAL: ffffffff80000000->ffffffff86800000
(XEN) ENTRY ADDRESS: ffffffff81d261e0
(XEN) Dom0 has maximum 4 VCPUs
(XEN) Scrubbing Free RAM: ........................................................done.
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 260kB init memory.
(XEN) traps.c:2503:d0 Domain attempted WRMSR 00000000000001fc from 0x000000000000005f to 0x000000000000005d.
(XEN) traps.c:2503:d0 Domain attempted WRMSR 00000000000001fc from 0x000000000000005f to 0x000000000000005d.
(XEN) traps.c:2503:d0 Domain attempted WRMSR 00000000000001fc from 0x000000000000005f to 0x000000000000005d.
(XEN) traps.c:2503:d0 Domain attempted WRMSR 00000000000001fc from 0x000000000000005f to 0x000000000000005d.


if there is other information you need just tell me :). my last chance to have two systems running with usb would be to use 00:14.0 and 05:00.0 on each dom0 leaving linux wihtout usb (using a ps2->usb adapter for a keyboard is possible right? otherwise i have to boot into dom1 and connect to linux via ssh.

the best way would obviously be to be able to use each pci usb card seperately, maybe there is something missing, or being able to use 00:1a.0 or 00:1d.0

hope this info helps !

kind regards

chapapa
Kryspy

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by Kryspy »

Can I keep an existing install of Mint and install XEN? Or do I have to start from scratch with a Mint XEN dom0 and then a Mint domU and Windows domU?

Kryspy
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

@Chapapa: You didn't post the second dom2 config file (and the first dom1 is mangled). Best include them inside "Code" format.

You should try also this combination: 0:1a.0, 0:1d.0, and 05:00.0 each for dom0, dom1, and dom2. It seems that 05:00.* and 05:02.* are behind the same PCI bridge. I can't remember whether or not there is a way to work around this, but search for IOMMU group. Or best, sign up for the xen user list and post a question - there are lots of competent people on there and it might be the fastest way to get a good answer. I'm sorry I can't be of more help on this - there are obviously areas I'm not really familiar with (but read on).

Last not least, some of your USB ports are not accessible via the existing USB ports on the back panel or front panel, but only via the USB headers on your board. You'd need to connect a USB panel with a cable to the header to use it. Try to see if you can assign separate USB controllers to each dom (0, 1, and 2), you should see via xm pci-list-assignable-devices if and when they are grabbed by dom1 or dom2 (don't forget to modify your domU config files AND your /etc/xen/pciback.conf file!!!). Once you have determined a combination that works, you might have to get one of those USB back panel plates to make the USB port/host accessible.

If the above doesn't help, and there is no combination of USB hosts (controllers) that can be separately assigned to dom0, dom1, and dom2, get a USB PCIe card (or a combined SATA / USB card). They aren't expensive.

Here is my guest startup script - you need to have the zenity package installed and then you need to change the name of the domX config file in the script (see beginning of file). With this script you don't need the pciback script mentioned in the how-to. The script also checks if the domX is already running, exiting gracefully with an info message.

Code: Select all

#!/bin/sh

domu="win7"
configfile=/etc/xen/pciback.conf

# The following bindback function is:
# License: GPLv2
# Author: Peter Maloney
#
# Script to bind devices to pciback (or pci-stub)
   
bindback() {
	device="$1"

	if [ ! -e "/sys/bus/pci/devices/$device" ]; then
		zenity --warning --window-icon=error --timeout=15 --text="ERROR: Device does not exist... cancelling" &
		return
	fi

#	with pciback, you do unbind, then new_slot, then bind

	if [ ! -e "/sys/bus/pci/devices/$device/driver" ]; then
		zenity --warning --window-icon=error --timeout=15 --text="    no driver to unbind" &
	else
		chmod +w "/sys/bus/pci/devices/${device}/driver/unbind"
		echo -n "$device" > "/sys/bus/pci/devices/$device/driver/unbind"
	fi

	chmod +w /sys/bus/pci/drivers/pciback/new_slot
	echo -n "$device" > /sys/bus/pci/drivers/pciback/new_slot

	chmod +w /sys/bus/pci/drivers/pciback/bind
	echo -n "$device" > /sys/bus/pci/drivers/pciback/bind
    
}



if xm list | grep -q $domu; then
	zenity --info --window-icon=info --timeout=15 --text="$domu is already running." &
	exit 1

else

	cat $configfile | while read line;do
	echo $line | grep ^# >/dev/null 2>&1 && continue
		bindback $line
	done

	xm create /etc/xen/$domu.cfg &> /dev/null
	zenity --info --window-icon=info --timeout=15 --text="Starting $domu, please wait." &
	exit 0

fi


So if I understand you correctly, you are now running LM16 with Xen 4.3 and the xm toolstack?
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

Kryspy wrote:Can I keep an existing install of Mint and install XEN? Or do I have to start from scratch with a Mint XEN dom0 and then a Mint domU and Windows domU?

Kryspy
Usually you can just install Xen onto an existing Linux Mint installation. Just make sure you have a complete backup of your system!

You may run into problems with for example the Nvidia proprietary driver who doesn't like Xen. As long as you use the open source driver under dom0, it should work.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
Boggler

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by Boggler »

powerhouse wrote:I'm not sure if compiling your own kernel is the best way to go. Have you tried the standard kernel shipped with Linux Mint or Ubuntu? I'm currently at LM16, but have tested LM17 as well. It might be a good idea to check the forum for help on graphics card / GUI issues. I'm afraid I can't be of much help on that. When you reinstalled Xen, did you also format your /home directory? If not, there might be some leftover desktop configuration files that could get in the way of loading the desktop. It's just a thought.
I did a little research but not much, as it wasn't very likely to find one with a similar issue. In most cases a black screen could be traced back to a faulty driver etc. but that's not the case here.
I'm pretty sure there is no problem with my kernel config, as I just copied the old config and used it. It could be though that the issue is originated by the differences between the generic kernel (3.13-something) and my compiled one (3.12.2). I tired first to compile the newest kernel, but the patch wouldn't work on that. The older version of the patch I found would work only on 3.12.2 though.

I formatted the complete LVM volume (I've got a boot partion on my SSD and the remaining space is assigned to LVM, which is then parted to swap, root and win7).
powerhouse wrote: As to kvm, have a look a few posts above where I posted a link to a kvm how-to on the archlinux forum, together with my guest startup script for kvm. I've managed to get primary VGA passthrough working with kvm and a AMD 7770 card, but haven't been successful yet with my Nvidia Quadro 2000 (though I found a possible solution). Hopefully one day I will be able to share a kvm how-to for Linux Mint (right now I'm not really familiar with it).
I looked a bit and found a guide on the arch forum, I assume we are speaking of the same. It is not as thorough as it is aimed at more experienced users, but I got it working pretty fast (1-2 hours). I couldn't get my peripherals to work though, even when I passed them through. I then got virt-manager and just configured everything in the GUI, where it worked OOTB, even with multiple monitors, networking and without the need to passthrough a set of peripherals. Only the sound isn't working (yet, hopefully), I already tried the options virt-manager gave me. Windows recognizes the output device and provides output, but on linux I get nothing of that. I read about someone with the same issue who just switched back to the commandline version where it works. So I'll look into whether I can edit the actual commands that are used to start QEMU. I gotta say the virt-manager is really neat with buttons and graphs and all :D
Do you use the virt-manager? If so, how did you get your sound to work?
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

Boggler wrote:I'm pretty sure there is no problem with my kernel config, as I just copied the old config and used it. It could be though that the issue is originated by the differences between the generic kernel (3.13-something) and my compiled one (3.12.2). I tired first to compile the newest kernel, but the patch wouldn't work on that. The older version of the patch I found would work only on 3.12.2 though.
Which patch are you referring to? As I already said, it might have been worth trying with the standard kernel.
Boggler wrote:So I'll look into whether I can edit the actual commands that are used to start QEMU. I gotta say the virt-manager is really neat with buttons and graphs and all :D
Do you use the virt-manager? If so, how did you get your sound to work?
There is, I believe, an export option in virt-manager that lets you export to a xml file.

I never managed to get anything passthrough-related working with virt-manager, so I would be very interested in getting more information on how you did it. Since I never found any human-readable configuration files for virt-manager, and since I couldn't find any good documentation on it, I soon marked it as useless and dumped it. Perhaps I didn't do it justice?

Sorry, I can't help with the sound configuration. See my how-to towards the end for some suggestions. I don't see why they shouldn't work with KVM.

I'm looking forward to see your KVM configuration steps.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
Boggler

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by Boggler »

powerhouse wrote:Which patch are you referring to? As I already said, it might have been worth trying with the standard kernel.
The patch to fix my marvell controllers. Without it and IOMMU enabled, I get loads of IO_PAGE_FAULT errors from the controllers and my graphics card, at most times I can't even boot completely because of that.
The latest version can be found here: https://github.com/awilliam/linux-vfio
The one that worked for me is this one, however: https://bugzilla.kernel.org/show_bug.cgi?id=42679#c30
There is, I believe, an export option in virt-manager that lets you export to a xml file.

I never managed to get anything passthrough-related working with virt-manager, so I would be very interested in getting more information on how you did it. Since I never found any human-readable configuration files for virt-manager, and since I couldn't find any good documentation on it, I soon marked it as useless and dumped it. Perhaps I didn't do it justice?
Maybe they didn't have the options for passthrough when you tried it last. I got to admit, it is not obvious where one can find the hardware options and I also needed someone to point it out for me :D

First you got to use the pci-stub driver or blacklist your normal graphics card driver and assign both PCI IDs to vfio-pci, like in the arch how-to.
After that it's pretty easy. You just start virt-manager, select new VM, go through the basic settings and install it normally. If it works, you shut it down and switch to the hardware settings (green button on the top left corner) and click on something like 'add hardware' on the bottom left. There you select passthrough PCI device and click on the PCI ID for your graphics card. You have to do that again for the HDMI audio controller on you graphics card, if you have one. Then just boot to Windows again and look in the device manager for the graphics card. If it is there, you got to install the AMD driver/catalyst center and after a reboot, you hopefully will get output from your graphics card. It didn't work for me at first, so I just removed all AMD stuff and installed it again (but it is very likely that I forgot to assign the card to vfio-pci after a reboot :D). After that you will now see the VM bios things and "windows is starting" on the emultated graphics card like before and everything after that comes as output from the real graphics card. You got to click into the virtual monitor to use your mouse and keyboard inside the VM.

With this setup it is now kind of ironic, because my Windows VM works just fine with the 7870 and my linux UI crashes constantly when I do anything, after I connected a second monitor. 3460x1440 is definitely too much for my passive Geforce 7600 with 256MB VRAM :D

I'll have a look at your sound configuration again and give it a try. The xml file could also be useful.
Locked

Return to “Virtual Machines”