Xen Error

Questions about virtualization software
Forum rules
Before you post please read how to get help
powerhouse
Level 6
Level 6
Posts: 1086
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: Xen Error

Post by powerhouse » Fri Apr 04, 2014 1:36 pm

Good news: It's working! Windows is still busy installing 140 updates, but I got the AMD 6450 working. Also installed the GPLPV drivers (disk, etc.) to see how this influences disk performance. I'll post some screen shots once I can reboot Windows.

To summarize:
LM13 with xm toolstack, a 3.2.0-23-generic kernel, and the oldest Xen 4.1.2 you can select from the repos works. The rest is described above and in the LM13 how-to.

Here the proof:
AMD Radeon 6450 VGA passthrough
AMD Radeon 6450 VGA passthrough
And here the WEI:
AMD Radeon 6450 WEI
AMD Radeon 6450 WEI
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3, Noctua NH-D14 CPU cooler, GTX 970 + Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk 120GB + Samsung EVO 860 1TB SSD + 4 HDD, Corsair 500R, SeaSonic 660W Gold X PS https://heiko-sieger.info

bonzi
Level 1
Level 1
Posts: 18
Joined: Wed Mar 19, 2014 1:49 am

Re: Xen Error

Post by bonzi » Fri Apr 04, 2014 10:20 pm

Hey powerhouse. I really appreciate your help, but unfortunately I just cannot get it to work despite doing everything exactly the same as you. I'm also on LM13, xm toolstack, kernel 3.2.0-23-generic and Xen 4.1.2. It just doesn't work, totally frustrating. Before I give up completely, what version of fglrx did you install? Do you think that could cause the problem? Maybe fglrx touches my 6450 and it is already italicized which causes this problem?

I think it might have something to do with my other hardware. I think I will have to wait on the quadro 2000 and hope that solves my problems.

powerhouse
Level 6
Level 6
Posts: 1086
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: Xen Error

Post by powerhouse » Sat Apr 05, 2014 3:17 am

bonzi wrote:Hey powerhouse. I really appreciate your help, but unfortunately I just cannot get it to work despite doing everything exactly the same as you. I'm also on LM13, xm toolstack, kernel 3.2.0-23-generic and Xen 4.1.2. It just doesn't work, totally frustrating. Before I give up completely, what version of fglrx did you install? Do you think that could cause the problem? Maybe fglrx touches my 6450 and it is already italicized which causes this problem?

I think it might have something to do with my other hardware. I think I will have to wait on the quadro 2000 and hope that solves my problems.
I don't think fglrx really matters for VGA passthrough. I installed fglrx-updates, for reference.

Hardware of course matters. IIRC you have PCI ID 03:00.0 for your AMD 6450. You could try to change the position of your 6450 and use a different PCI slot. One thing that comes to mind is that some boards use one PCI bridge to handle multiple PCIe slots, which might create problems. But before digging into that, let's make sure we got a comparable configuration. Here is my /etc/xen/win7.cfg:

Code: Select all

kernel = '/usr/lib/xen-default/boot/hvmloader'
builder='hvm'
memory = 4096
name = 'win7'
vcpus=6
#pae=1
acpi=1
apic=1
on_xend_stop='shutdown'
vif = [ 'mac=00:16:3e:68:e1:01,bridge=xenbr0' ]
disk = [ 'phy:/dev/mapper/external-win7ext,hda,w' , 'file:/media/lm13_data/ISOs/Win7.iso,hdc:cdrom,r' ]
device_model = '/usr/lib/xen-default/bin/qemu-dm'
boot='dc'
sdl=0
vnc=1
vncpasswd=''
stdvga=0
serial='pty'
tsc_mode=0
viridian=1
usb=1
usbdevice='tablet'
gfx_passthru=0
pci=[ '02:00.0', '02:00.1' ]
localtime=1
pci_power_mgmt=1
As I mentioned before, use the oldest kernel and Xen 4.1.2. By the way, I also installed a much newer kernel 3.8.0-38 and it works fine (EDIT: It did work fine, but now it gives error 22 - might be due to my installing some packages or a different fglrx driver). Just kernel 3.2.0-60 seems to be problematic. Need to do some more tests.
Last edited by powerhouse on Sun Apr 06, 2014 11:49 pm, edited 1 time in total.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3, Noctua NH-D14 CPU cooler, GTX 970 + Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk 120GB + Samsung EVO 860 1TB SSD + 4 HDD, Corsair 500R, SeaSonic 660W Gold X PS https://heiko-sieger.info

bonzi
Level 1
Level 1
Posts: 18
Joined: Wed Mar 19, 2014 1:49 am

Re: Xen Error

Post by bonzi » Sun Apr 06, 2014 3:20 am

powerhouse wrote: Hardware of course matters. IIRC you have PCI ID 03:00.0 for your AMD 6450. You could try to change the position of your 6450 and use a different PCI slot. One thing that comes to mind is that some boards use one PCI bridge to handle multiple PCIe slots, which might create problems.
I would try this but unfortunately I only have 2 full size pcie slots. So all I can do is swap the cards around which I have already done.
But before digging into that, let's make sure we got a comparable configuration. Here is my /etc/xen/win7.cfg:

Code: Select all

kernel = '/usr/lib/xen-default/boot/hvmloader'
builder='hvm'
memory = 4096
name = 'win7'
vcpus=6
#pae=1
acpi=1
apic=1
on_xend_stop='shutdown'
vif = [ 'mac=00:16:3e:68:e1:01,bridge=xenbr0' ]
disk = [ 'phy:/dev/mapper/external-win7ext,hda,w' , 'file:/media/lm13_data/ISOs/Win7.iso,hdc:cdrom,r' ]
device_model = '/usr/lib/xen-default/bin/qemu-dm'
boot='dc'
sdl=0
vnc=1
vncpasswd=''
stdvga=0
serial='pty'
tsc_mode=0
viridian=1
usb=1
usbdevice='tablet'
gfx_passthru=0
pci=[ '02:00.0', '02:00.1' ]
localtime=1
pci_power_mgmt=1
Yes, our configs are identical.
As I mentioned before, use the oldest kernel and Xen 4.1.2. By the way, I also installed a much newer kernel 3.8.0-38 and it works fine. Just kernel 3.2.0-60 seems to be problematic. Need to do some more tests.
I reinstalled LM13 from scratch as well as Xen 4.1.2 and made sure that I was on 3.2.0-23. Still no luck...still spits out the same old error 22.

powerhouse
Level 6
Level 6
Posts: 1086
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: Xen Error

Post by powerhouse » Sun Apr 06, 2014 3:37 pm

That's VERY odd. I'll try to add some ideas or things to check tomorrow.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3, Noctua NH-D14 CPU cooler, GTX 970 + Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk 120GB + Samsung EVO 860 1TB SSD + 4 HDD, Corsair 500R, SeaSonic 660W Gold X PS https://heiko-sieger.info

bonzi
Level 1
Level 1
Posts: 18
Joined: Wed Mar 19, 2014 1:49 am

Re: Xen Error

Post by bonzi » Sun Apr 06, 2014 10:12 pm

Thanks, I'll have my quadro 2000 tomorrow. So I will test that and post the results.

powerhouse
Level 6
Level 6
Posts: 1086
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: Xen Error

Post by powerhouse » Sun Apr 06, 2014 11:42 pm

bonzi wrote:Thanks, I'll have my quadro 2000 tomorrow. So I will test that and post the results.
The Nvidia Quadro should play much nicer. For reference, I have the PNY Quadro 2000.

About the AMD 6450, see this post: http://forums.linuxmint.com/viewtopic.p ... 82#p802456. It seems some earlier Xen 4.3 release, and perhaps an earlier 3.11 kernel release do work with this card.

I had varying success with kernel 3.8 and Xen 4.1.2. At first it worked, then when I tried again I would get error 22. My suspicion is that some security patch for the kernel broke VGA passthrough. Early 3.2 kernels are fine, but the 3.2...-60 definitely doesn't work. Xen also introduced a bug with security fixes (XSA), which is why Xen 4.1.3 gives the error 22 while Xen 4.1.2 - with the right kernel - works on my machine.

You should check that you downgraded/use release 4.1.2 for all Xen related packages (5 altogether). The dependency check doesn't work for this, so it's not enough to downgrade the hypervisor. Do a search for "xen" in Synaptic and note the installed Xen packages - downgrade all those that have 4.1.3 in the version. Other than that I don't have more ideas for now.

EDIT: It may also depend on the manufacturer of the AMD 6450 card. Not all cards are equal. Mine is a Sapphire HD 6450 - see http://www.sapphiretech.com/presentatio ... id=1&leg=0.
Different manufacturers have sometimes different BIOS implementations.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3, Noctua NH-D14 CPU cooler, GTX 970 + Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk 120GB + Samsung EVO 860 1TB SSD + 4 HDD, Corsair 500R, SeaSonic 660W Gold X PS https://heiko-sieger.info

bonzi
Level 1
Level 1
Posts: 18
Joined: Wed Mar 19, 2014 1:49 am

Re: Xen Error

Post by bonzi » Mon Apr 07, 2014 11:03 pm

Hi,

I wanted to report that the quadro 2000 is working perfectly using LM13, kernel 3.2.0-23 and Xen 4.1.2. I will try with a newer version at some point but it certainly looks like this was a problem with the amd cards not playing nice with Xen.

powerhouse
Level 6
Level 6
Posts: 1086
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: Xen Error

Post by powerhouse » Tue Apr 08, 2014 8:20 am

bonzi wrote:Hi,

I wanted to report that the quadro 2000 is working perfectly using LM13, kernel 3.2.0-23 and Xen 4.1.2. I will try with a newer version at some point but it certainly looks like this was a problem with the amd cards not playing nice with Xen.
That's good news! I agree that the AMD 6450 is quite a challenge for VGA passthrough, though in my case it worked with the kernel and Xen release you mention.

For best results with the Quadro 2000, try the latest LM 16 with the standard Xen 4.3 release and xl toolstack :) . Here is my win7.cfg file for the Quadro 2000:

Code: Select all

kernel = '/usr/lib/xen-default/boot/hvmloader'
builder='hvm'
memory = 8192
name = 'win7'
vcpus=6
pae=1
acpi=1
apic=1
on_xend_stop='shutdown'
vif = [ 'mac=00:16:3e:68:01:0e,type=vif,bridge=xenbr0' ]
disk = [ '/dev/mapper/guest-win7,raw,hda,rw' , '/home/user/ISOs/win7.iso,raw,hdd,devtype=cdrom' ]
device_model_version = 'qemu-xen-traditional'
boot='dc'
sdl=0
vnc=1
vncpasswd=''
serial='pty'
tsc_mode=0
viridian=1
usb=0
usbdevice='tablet'
gfx_passthru=0
pci=[ '02:00.0', '02:00.1' ]
localtime=1
xen_platform_pci=1
pci_power_mgmt=1
Adjust to your needs. In my case, using the xl toolstack versus the xm toolstack also reduced domU boot time by some 10 seconds. Windows now boots in ~20 seconds, using an SSD as media.

Important: When using a Nvidia Quadro card, you don't need to worry about shutting down the guest and rebooting it. There is no need for ejecting the graphics card upon shutdown, as is required by AMD cards when using the xl toolstack.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3, Noctua NH-D14 CPU cooler, GTX 970 + Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk 120GB + Samsung EVO 860 1TB SSD + 4 HDD, Corsair 500R, SeaSonic 660W Gold X PS https://heiko-sieger.info

powerhouse
Level 6
Level 6
Posts: 1086
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: Xen Error

Post by powerhouse » Tue Apr 08, 2014 9:12 am

@bonzi: I noticed that a couple of days ago I gave you some wrong advice regarding the win7.cfg changes for the xl toolstack. I fixed it in my reply, but here it is for the benefit of those following this thread. When running the xl toolstack instead of xm, the guest config file (e.g. win7.cfg) should have the following line instead of the device_model ... entry:

Code: Select all

device_model_version = 'qemu-xen-traditional'
The reason is that the new qemu-xen (qemu upstream) still doesn't support VGA passthrough. For an example of a complete guest config file, see my previous post.

Sorry for my confusion.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3, Noctua NH-D14 CPU cooler, GTX 970 + Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk 120GB + Samsung EVO 860 1TB SSD + 4 HDD, Corsair 500R, SeaSonic 660W Gold X PS https://heiko-sieger.info

bonzi
Level 1
Level 1
Posts: 18
Joined: Wed Mar 19, 2014 1:49 am

Re: Xen Error

Post by bonzi » Tue Apr 08, 2014 11:14 am

powerhouse wrote:
bonzi wrote:Hi,

I wanted to report that the quadro 2000 is working perfectly using LM13, kernel 3.2.0-23 and Xen 4.1.2. I will try with a newer version at some point but it certainly looks like this was a problem with the amd cards not playing nice with Xen.
That's good news! I agree that the AMD 6450 is quite a challenge for VGA passthrough, though in my case it worked with the kernel and Xen release you mention.

For best results with the Quadro 2000, try the latest LM 16 with the standard Xen 4.3 release and xl toolstack :) . Here is my win7.cfg file for the Quadro 2000:

Code: Select all

kernel = '/usr/lib/xen-default/boot/hvmloader'
builder='hvm'
memory = 8192
name = 'win7'
vcpus=6
pae=1
acpi=1
apic=1
on_xend_stop='shutdown'
vif = [ 'mac=00:16:3e:68:01:0e,type=vif,bridge=xenbr0' ]
disk = [ '/dev/mapper/guest-win7,raw,hda,rw' , '/home/user/ISOs/win7.iso,raw,hdd,devtype=cdrom' ]
device_model_version = 'qemu-xen-traditional'
boot='dc'
sdl=0
vnc=1
vncpasswd=''
serial='pty'
tsc_mode=0
viridian=1
usb=0
usbdevice='tablet'
gfx_passthru=0
pci=[ '02:00.0', '02:00.1' ]
localtime=1
xen_platform_pci=1
pci_power_mgmt=1
Adjust to your needs. In my case, using the xl toolstack versus the xm toolstack also reduced domU boot time by some 10 seconds. Windows now boots in ~20 seconds, using an SSD as media.

Important: When using a Nvidia Quadro card, you don't need to worry about shutting down the guest and rebooting it. There is no need for ejecting the graphics card upon shutdown, as is required by AMD cards when using the xl toolstack.
Thanks, I'm planning on going with LM17 with xl toolstack which will hopefully ship with Xen 4.4. Don't have a lot of time these days so I think I'll stay with LM13 for the next few weeks. Thanks again!

powerhouse
Level 6
Level 6
Posts: 1086
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: Xen Error

Post by powerhouse » Thu Apr 10, 2014 5:15 pm

I'm trying now Ubuntu 14.04 with Xen 4.4, qemu-xen (upstream), and xl:

So far I managed to install Windows 7 and the AMD driver, but after a guest reboot I only get a VNC screen. Device manager shows a yellow triangle and the properties window shows:
This device cannot find enough free resources that it can use

If you want to use this device, you will need to disable one of the other devices on this system.
I disabled the standard VGA (virtual VGA card) to see how that works. It doesn't - would have been too easy.

Next is qemu-xen-traditional:

Hurray, Windows boots and the screen switches to the AMD 6450 !

A few reboots further - everything seems to work. Except that I got a lousy WEI for Aero graphics. So I'm reinstalling the AMD driver now, this time with the CCC (default "Express" option in the installer). That didn't improve things. Here is the WEI:
ubuntu14.04_Xen4.4_AMD6450_WEI.PNG
Perhaps this low Aero performance has to do with my installing Windows using qemu-xen and then switching to qemu-xen-traditional ? I don't know.

The good news is that I can boot and shutdown the Windows 7 domU as often as I like.

Update: Just ran the Windows domU with 8GB of RAM (before I used 4GB). No issues with that. Also installed the GPLPV drivers (all of them), here the resulting WEI:
ubuntu14.04_Xen4.4_AMD6450_GPLPV_WEI.PNG
Slight performance improvement for the drive (a LVM formated external USB drive).
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3, Noctua NH-D14 CPU cooler, GTX 970 + Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk 120GB + Samsung EVO 860 1TB SSD + 4 HDD, Corsair 500R, SeaSonic 660W Gold X PS https://heiko-sieger.info

powerhouse
Level 6
Level 6
Posts: 1086
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: Xen Error

Post by powerhouse » Mon Apr 14, 2014 12:37 pm

Did some more tests today with Ubuntu 14.04:

1. Updated Ubuntu and packages - newest kernel is now 3.13.0-24.
2. Reinstalled Windows 7 with qemu-xen-traditional device model - had lots of issues installing the AMD driver and GPLPV.
3. Reinstalled Windows 7 with qemu-xen-traditional device model and pci-msi-translate and xen-platform-pci options enabled (=1) - install went OK but screen is flickering after installation of AMD drivers. Turned off both options. System a little more stable, but screen continues to flicker.
4. The AMD driver installation utility BSODs when doing an Express install - need to select Custom installation and deselect the AMD Catalayst Control Center (CCC)!

Here the WEI without GPLPV drivers:
WEI_no_GPLPV.PNG
Below the WEI with latest stable GPLPV drivers taken from here: http://www.ejbdigital.com.au/129/:
WEI_GPLPV_1_0_1089.PNG
The screen flicker is very annoying.

5. Deinstalled the AMD driver - Windows automatically chose an older version (8.850.0.0 from 4/19/2011). The flicker disappeared. Here the WEI:
WEI_GPLPV_1_0_1089_old_AMD_driver.PNG
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3, Noctua NH-D14 CPU cooler, GTX 970 + Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk 120GB + Samsung EVO 860 1TB SSD + 4 HDD, Corsair 500R, SeaSonic 660W Gold X PS https://heiko-sieger.info

Post Reply

Return to “Virtualization”