HOW-TO make dual-boot obsolete using XEN VGA passthrough

Questions about virtualization software
Forum rules
Before you post read how to get help. Topics in this forum are automatically closed 6 months after creation.
Locked
doggydogworld

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by doggydogworld »

Thanks for this post! I was stumped by the "Error 22" bug (Ubuntu 13.04 here), but then I downgraded every xen-related package to 4.2.1-0ubuntu3, which worked.

My setup:
Asrock A55MD2
AMD A8-5600K
ATi Radeon HD7770 (secondary passthrough to guest)
8GB RAM

This may be a longshot, but has anyone tried switching control of the PCI cards between pciback and the original Linux module, like fglrx, on the fly? In other words, I want fglrx to use the graphics card when the Win7 domU is powered off and then pass control to pciback right before starting the domU. When I try something like

Code: Select all

# echo "0000:01:00.0" > /sys/bus/pci/drivers/pciback/unbind
the system hangs.

It's not that the dom0 really needs the graphics card. I just don't want to have to buy a KVM switch, lol.
chrulri

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by chrulri »

doggydogworld wrote:This may be a longshot, but has anyone tried switching control of the PCI cards between pciback and the original Linux module, like fglrx, on the fly?
I accidentally did this before I blacklisted the radeon modules. The display server (Xorg) was causing the system hang (kernel panic) because it still tried to access my graphics card even though I unbound it from the driver (radeon).
I guess if you want to switch dynamically you would have to stop your display server first.
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

chrulri wrote:
doggydogworld wrote:This may be a longshot, but has anyone tried switching control of the PCI cards between pciback and the original Linux module, like fglrx, on the fly?
I accidentally did this before I blacklisted the radeon modules. The display server (Xorg) was causing the system hang (kernel panic) because it still tried to access my graphics card even though I unbound it from the driver (radeon).
I guess if you want to switch dynamically you would have to stop your display server first.
I'm afraid it is a little more complicated than this. From what I read, the graphics card needs to reset, which usually doesn't happen by switching drivers. Also, I'm not sure how many graphics cards support FLR or function level reset.

Switching control for other PCI devices is usually easy - just do:

Code: Select all

sudo virsh nodedev-reattach pci_0000_0a_00_0
where 0000_0a_00_0 is your PCI ID, just using _ instead of :

If all you are looking for is avoiding to buy a KVM, this might be enough to switch your keyboard and mouse. As for the screen, it may have 2 inputs and a button to switch between them.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
doggydogworld

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by doggydogworld »

powerhouse wrote:I'm afraid it is a little more complicated than this. From what I read, the graphics card needs to reset, which usually doesn't happen by switching drivers. Also, I'm not sure how many graphics cards support FLR or function level reset.
Ah, darn. From what I can tell, it seems that Quadros are pretty much the only cards known to support FLR. Thanks for the advice though!
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

doggydogworld wrote:
powerhouse wrote:I'm afraid it is a little more complicated than this. From what I read, the graphics card needs to reset, which usually doesn't happen by switching drivers. Also, I'm not sure how many graphics cards support FLR or function level reset.
Ah, darn. From what I can tell, it seems that Quadros are pretty much the only cards known to support FLR. Thanks for the advice though!
According to the lspci output of my Quadro 2000, it doesn't support FLR. I tried to reassign it to dom0 and the system became unstable, but it seems because of X. Anyway, it didn't work, so I gave up. All I read suggests that it's not easy to reclaim a graphics card, but I haven't researched nor tried all options.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
macester

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by macester »

Hi.

I have some issues, just bought a new asrock extreme 3 and a 4770 haswell.

I came from esxi but lacks support for the chipset so i though i try xen and googling arround and trying to find guides due to my limited linux expreiance i found your guide but have some issues, and I am not not sure what the differnce from the 13/14 installation is opposed to mint 15.

1, the on board 4200 graphics adapter isent supported on mint 14 (or ubuntu 12.10) so i guess i have to use mint 15?

2, read the notes on mint 15

but as a complete noob i dont full under stand :


$ grep -v ^# /etc/xen/pciback.conf
0000:01:00.0
0000:01:00.1
0000:05:01.0
0000:05:01.1
$ cat /etc/modprobe.d/xen-pciback.conf
# 01:00.0
blacklist radeon
# 05:01.0
blacklist snd-emu10k1
blacklist snd-emu10k1-synth
# 05:01.1
blacklist emu10k1-gp


should i just add (as the example):

01:00.0
05:01.0
05:01.1

in the: /etc/modprobe.d/xen-pciback.conf?

the way I understand it is that you add them to pciback but then hides some of them with xen-pciback?
and not sure on the change of toolstack either.

3, not as much xen but how would one go about formating sdb for lvm (following the example) to use for domU´s ?

3.1, instead would it yield better performace if sdb would be used as direct disk for say a win 8 installation?


So all in all the question im really asking is how would a noob like me do xen with pci passthrough on mint 15?


//Best Regards Marcus
chrulri

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by chrulri »

macester wrote: but as a complete noob i dont full under stand :


$ grep -v ^# /etc/xen/pciback.conf
0000:01:00.0
0000:01:00.1
0000:05:01.0
0000:05:01.1
$ cat /etc/modprobe.d/xen-pciback.conf
# 01:00.0
blacklist radeon
# 05:01.0
blacklist snd-emu10k1
blacklist snd-emu10k1-synth
# 05:01.1
blacklist emu10k1-gp


should i just add (as the example):

01:00.0
05:01.0
05:01.1

in the: /etc/modprobe.d/xen-pciback.conf?

the way I understand it is that you add them to pciback but then hides some of them with xen-pciback?
and not sure on the change of toolstack either.
The first file (/etc/xen/pciback.conf) is used by poweruser's script to attach the devices to xen pciback. Have a look at the original in http://forums.linuxmint.com/viewtopic.php?p=628768#p628768 at chapter 9 and 10.
The second file (/etc/modprobe.d/xen-pciback.conf) is used by modprobe to blacklist the kernel modules (drivers) I don't want to load and attach to devices as it may cause problems when they suddenly get unloaded by the first script.
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

@macester:

Onboard Intel graphics support: I don't know if LM 15 supports the 4200, but I guess chances are better as this is a new processor. An Internet search should get you an answer.

Your configuration files:
/etc/xen/pciback.conf: In this file you enter the PCI IDs of the devices you want to pass through to your Windows guest.
Important: Start with only the absolute necessary PCI IDs - the graphics card (usually two PCI IDs - in your case probably 01:00.0 and 01:00.1) and perhaps a USB controller if you need to pass it through. DON'T add any more devices like sound card etc.
Why? Because you want to install Windows with the graphics card passed thru giving you as little "headache" as possible. Each additional device can pause challenges. You can add them later when Windows is installed and VGA passthru is working.

/etc/modprobe.d/xen-pciback.conf: Here you enter the drivers that you don't want to load under Linux. The list here depends on the list above. You definitely list the driver(s) for the graphics card you want to pass thru. The snd-emu10k1... files are your drivers for the sound card you want to use in Windows, right? So at the beginning I would not list them here, nor the PCI ID of the sound card in the pciback.conf file above. Add them after you installed Windows and the graphics driver under Windows and your VGA passthrough is working properly.
Of course, you don't list the PCI IDs here, but the drivers. This file tells the Linux modprobe mechanism to NOT load the listed drivers during boot.


Toolstack: Linux Mint 15 uses the xl toolstack by default. After installation of LM 15 and Xen, I suggest changing it to the xm toolstack:

Code: Select all

sudo sed -i 's/TOOLSTACK=.*\+/TOOLSTACK="xm"/' /etc/default/xen
Important: There are issues with LM 15, also with the xm toolstack. See this post above by doggydogworld: [url]http://forums.linuxmint.com/viewtopic.php?f=42&t=112013&start=200#p747658[/url] to roll back Xen to a version where PCI passthrough works.

About formatting: When installing Linux Mint 15, you can use the LVM option of the installer - you don't need to follow the long separate how-to for installing LVM on LM13 or LM14.
You follow the how-to here and create the /boot (non-LVM i.e. regular ext2 partition), and / and /home LVM volumes formatted using ext4. For your Windows guest you create a LVM volume, but don't format it to anything!

In your /etc/xen/windows.cfg (your guest config file), you specify the unformatted LVM volume as your Windows drive. Just follow the how-to.

Hope you get it working!
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
chrulri

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by chrulri »

powerhouse wrote:Of course, you don't list the PCI IDs here, but the drivers. This file tells the Linux modprobe mechanism to NOT load the listed drivers during boot.
@macester: The lines containing the IDs have a # at the line start which means they're comments and irrelevant. Sorry, if that caused confusion
colinpickup

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by colinpickup »

Note that, from what people were saying, I was worried that I'd made a major mistake by updating to Linux Mint 15. I tried with xl and xm, xl gave a BSOD, xm gave the error 22. I thought I might need to reinstall to get my VM working again.

This is not the case, you just need to add the Quantal repos and force-version all of the xen packages to the quantal one, change the default toolstack to XM, and reboot into a 4.1 kernel.

Would it be an idea to, until the latest version is working for all, add a section to the guide on how to add the quantal repo to a new distro?
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

colinpickup wrote:Note that, from what people were saying, I was worried that I'd made a major mistake by updating to Linux Mint 15. I tried with xl and xm, xl gave a BSOD, xm gave the error 22. I thought I might need to reinstall to get my VM working again.

This is not the case, you just need to add the Quantal repos and force-version all of the xen packages to the quantal one, change the default toolstack to XM, and reboot into a 4.1 kernel.

Would it be an idea to, until the latest version is working for all, add a section to the guide on how to add the quantal repo to a new distro?
Thanks for the update and suggestion. I agree with you and plan to update the how-to, but I'm currently 2000 miles away from my Linux/Xen PC and will be able to try that only in September. Can you post the steps to downgrade Xen?
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

This weekend I did some hardware upgrade, replacing my AMD Radeon 6450 for a 7770, and adding an Asus Xonar Essence STX sound card (it was hard for me to buy an Asus product, but this sound card is as good as it gets for audiophiles, unless you spend some $$$$$ for a good external DAC). I should have bought a wiring cabinet too as the back of my PC looks like a patch panel in a data center.

After installing the cards, I booted and everything looked fine. But when I checked usb-devices there was an error with my ASMedia SATA/USB3 controller. I thought perhaps putting the ASMedia controller into a different slot might help, but the only way to do so was to exchange the two graphics cards - so I put my dom0 VGA card into the second PCIe 16 slot, and vice versa the domU card into the first, hoping to be able to configure it somehow. Next I rebooted into the BIOS to check the settings. I can't remember what I did exactly, but somehow I must have selected some default / preconfigured setup by accident and saved the settings (anyway I didn't find any VGA related settings). The PC booted showing the normal messages until it got stuck at a blank screen. I thought the graphics cards might be the issue (I use AMD for dom0 and Nvidia for domU) and rebooted into a regular LM14 (without Xen), where I got an Xorg error and a terminal screen. I couldn't find much under Google, whatever little I found I tried, such as some xorg.conf settings, but nothing worked.

Finally I gave up, deleted the xorg.conf file changes and returned the cards to how I installed them the first time. When booting into Xen I would get a black screen, but it booted fine into regular LM14. In Xen, before giving me the black screen, it displayed an HSM error. Sounds familiar, but I didn't pay much attention to it. I thought I had screwed up the Xorg settings so I booted a live LM14 USB stick and restored my Linux partitions from backup. It still didn't boot.

To end this long story, I finally remembered the HSM message and bothered to check the BIOS settings: Lo and behold VT-d was disabled - ouch :oops: ! After enabling VT-d Xen/LM14 booted happily into the dom0. The usb-device error with the ASMedia controller magically disappeared too (during the boot process I saw some "ASMedia ... PCIe Gen2" message telling that the BIOS adjusted the PCIe setting to match the card's capabilities).

So, everything works now. The Radeon 7770 is way better than the 6450 - no more tearing when scrolling, and HD video is smooth as it should be at 1900x1200 or so. Didn't even have to touch the driver, as the installed fglrx-updates driver package recognized the new hardware.

The only thing that keeps bothering me is that I can't install my primary (dom0/Linux) display card into the second PCIe-16 slot and the guest VGA into the first PCIe-16 slot. Any suggestions?


Note: The reasons I want to exchange the two is that the AMD 7770 is a two slot card and totally blocks one PCIe-1 slot, and the ASMedia controller in the next slot is half blocking its cooling fan. Next to my second PCIe-16 slot is an old PCI slot that I have no use for. My Nvidia Quadro 2000 takes only 1 slot and rearranging the cards would free 1 PCIe-1 slot and leave more space to the AMD 7770 graphics card for better cooling. I sometimes wonder who designs these PC motherboards :x .
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
lxq_ikrana

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by lxq_ikrana »

Hi,
I'm trying to setup passthrough to work on LM14 + Xen 4.1.3. Hardware is:
AMD FX-6350
Asus M5A97 R2.0
Radeon 7870 + 5450

I installed LM14 with LVM without issues. Installed Xen. Whenever I try to boot into Xen, it loads a couple of things, the last being 'loading initram' and then the system just reboots. I'm not sure what the problem is.

In the motherboard BIOS, there are two IOMMU options
IOMMU (Enabled/Disabled)
IOMMU Mode (64MB/Disabled)

I have no idea why there are two options. The motherboard manual on the Asus website shows just IOMMU as the option. I've tried to get it working with IOMMU Mode set to 64MB or Disabled (with the first setting enabled) but I get the same result. Gets to 'loading initram' and then just reboots without error messages.

Help?
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

lxq_ikrana wrote:Hi,
I'm trying to setup passthrough to work on LM14 + Xen 4.1.3. Hardware is:
AMD FX-6350
Asus M5A97 R2.0
Radeon 7870 + 5450

I installed LM14 with LVM without issues. Installed Xen. Whenever I try to boot into Xen, it loads a couple of things, the last being 'loading initram' and then the system just reboots. I'm not sure what the problem is.

In the motherboard BIOS, there are two IOMMU options
IOMMU (Enabled/Disabled)
IOMMU Mode (64MB/Disabled)

I have no idea why there are two options. The motherboard manual on the Asus website shows just IOMMU as the option. I've tried to get it working with IOMMU Mode set to 64MB or Disabled (with the first setting enabled) but I get the same result. Gets to 'loading initram' and then just reboots without error messages.

Help?
Boot into regular LM 14 (non-Xen) and go to /var/log, then check the xen logs as well as dmesg. Any hints? If you find any errors towards the end of the logs post them here.

Open a terminal and enter:

Code: Select all

grep -E "(vmx|svm)" --color=always /proc/cpuinfo
You should get a line with svm mentioned (vmx for Intel). If not something's wrong with the IOMMU setup - go into BIOS and try another setting. Also check the Asus website to see if they have BIOS upgrades that may address this issue (unfortunately Asus BIOS release information is often obscure and useless).

Have you already done some blacklisting, modified grub, etc.? If so, please post the changes you made. Or is this happening right after a vanilla installation of the Xen hypervisor?

Did you install the fglrx driver (proprietary AMD driver), or are you using the default radeon driver?

Edit: Can you post the relevant part of /etc/default/grub ?

Edit 2: Important!!! I just recalled that the grub instructions for iommu may be different for AMD CPUs. Check the Xen wiki for anything specific to AMD CPUs (there should be some links in my how-to). Sorry, I haven't used AMD CPUs for at least 7 years.
Another comment: If booting into a regular LM14 also fails, you probably forgot to install LVM2 in a change root environment (happened to me today, and it does hang on the initramfs message). In that case boot from USB stick and follow the instructions in my LVM how-to, from 7. to 12. here: [url]http://forums.linuxmint.com/viewtopic.php?f=42&t=108442#p610315[/url].
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
lxq_ikrana

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by lxq_ikrana »

Apologies for the late reply, I was away from the PC for a while.
powerhouse wrote: Boot into regular LM 14 (non-Xen) and go to /var/log, then check the xen logs as well as dmesg. Any hints? If you find any errors towards the end of the logs post them here.
The only line with the word error in dmesg is this:
[ 3.790431] EXT4-fs (dm-0): re-mounted. Opts: errors=remount-ro
powerhouse wrote: Open a terminal and enter:

Code: Select all

grep -E "(vmx|svm)" --color=always /proc/cpuinfo
You should get a line with svm mentioned (vmx for Intel). If not something's wrong with the IOMMU setup - go into BIOS and try another setting. Also check the Asus website to see if they have BIOS upgrades that may address this issue (unfortunately Asus BIOS release information is often obscure and useless).
The SVM line does show up, so hopefully that's not the problem. The Asus website is unhelpful :(
powerhouse wrote: Have you already done some blacklisting, modified grub, etc.? If so, please post the changes you made. Or is this happening right after a vanilla installation of the Xen hypervisor?

Did you install the fglrx driver (proprietary AMD driver), or are you using the default radeon driver?
No changes at all. Just installed xen and changed grub to show xen as a boot option.
powerhouse wrote: Edit: Can you post the relevant part of /etc/default/grub ?

Code: Select all

GRUB_DEFAULT=0
#GRUB_HIDDEN_TIMEOUT=0
#GRUB_HIDDEN_TIMEOUT_QUIET=true
GRUB_TIMEOUT=10
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
GRUB_CMDLINE_LINUX=""
GRUB_CMDLINE_XEN="iommu=1 dom0_mem=2048M"
powerhouse wrote: Edit 2: Important!!! I just recalled that the grub instructions for iommu may be different for AMD CPUs. Check the Xen wiki for anything specific to AMD CPUs (there should be some links in my how-to). Sorry, I haven't used AMD CPUs for at least 7 years.
I will check this and get back.
powerhouse wrote: Another comment: If booting into a regular LM14 also fails, you probably forgot to install LVM2 in a change root environment (happened to me today, and it does hang on the initramfs message). In that case boot from USB stick and follow the instructions in my LVM how-to, from 7. to 12. here: [url]http://forums.linuxmint.com/viewtopic.php?f=42&t=108442#p610315[/url].
Booting into regular LM14 works. I did have a problem of some sort with the first time I did it, but after a reinstall, at least that is working.

Thanks for the exhaustive reply. I'll check the xen website and update soon.

Edit: I seem to have Xen 4.1.3-3ubuntu1.7 installed. I'm not sure hwo I should downgrade these. In any case, I should still be able to at least boot into Xen?
lxq_ikrana

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by lxq_ikrana »

Okay, managed to get Xen to boot. It was a problem with the newer version. Downgraded to the version that works, and it boots now. Cinnamon is broken (no bottom panel/mint menu, keyboard doesn't work, no title bars for windows), but I can fix that later. I'll continue with the guide and come back if I have success/questions.

Many thanks!
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

lxq_ikrana wrote:Okay, managed to get Xen to boot. It was a problem with the newer version. Downgraded to the version that works, and it boots now. Cinnamon is broken (no bottom panel/mint menu, keyboard doesn't work, no title bars for windows), but I can fix that later. I'll continue with the guide and come back if I have success/questions.

Many thanks!
Good to hear you managed to boot into Xen. I just wrote a detailed reply but inadvertently deleted the entire text ( :oops: ).

I make it short: The problem is Asus !!! Your Asus M5A97 EVO R2.0 motherboard BIOS is faulty in that it has a broken IVRS ACPI table. That broken IVRS table in conjunction with a Xen security update XSA-36 causes IOMMU to not be enabled. This explains why you could boot into the older, pre-XSA-36 Xen version.

See here:
[url]http://lists.xen.org/archives/html/xen-announce/2013-02/msg00006.html[/url]
[url]http://www.gossamer-threads.com/lists/xen/users/283968?do=post_view_threaded#283968[/url]
specifically your board: [url]http://www.gossamer-threads.com/lists/xen/users/284083?do=post_view_threaded#284083[/url] - note that there is no fix, yet!

EDIT: Asus seems to be doing something about it and has beta BIOS versions for at least 2 boards available - see here: [url]http://xen.1045712.n5.nabble.com/Xen-IOMMU-disabled-due-to-IVRS-table-Blah-blah-blah-td5716461i20.html[/url]. Important: If it's not exactly your motherboard, don't try to use them! Best is to contact Asus.
I have included a note in my how-to.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
GizmoChicken

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by GizmoChicken »

powerhouse wrote:
lxq_ikrana wrote:Okay, managed to get Xen to boot. It was a problem with the newer version. Downgraded to the version that works, and it boots now. Cinnamon is broken (no bottom panel/mint menu, keyboard doesn't work, no title bars for windows), but I can fix that later. I'll continue with the guide and come back if I have success/questions.

Many thanks!
Good to hear you managed to boot into Xen. I just wrote a detailed reply but inadvertently deleted the entire text ( :oops: ).

I make it short: The problem is Asus !!! Your Asus M5A97 EVO R2.0 motherboard BIOS is faulty in that it has a broken IVRS ACPI table. That broken IVRS table in conjunction with a Xen security update XSA-36 causes IOMMU to not be enabled. This explains why you could boot into the older, pre-XSA-36 Xen version.

See here:
[url]http://lists.xen.org/archives/html/xen-announce/2013-02/msg00006.html[/url]
[url]http://www.gossamer-threads.com/lists/xen/users/283968?do=post_view_threaded#283968[/url]
specifically your board: [url]http://www.gossamer-threads.com/lists/xen/users/284083?do=post_view_threaded#284083[/url] - note that there is no fix, yet!

EDIT: Asus seems to be doing something about it and has beta BIOS versions for at least 2 boards available - see here: [url]http://xen.1045712.n5.nabble.com/Xen-IOMMU-disabled-due-to-IVRS-table-Blah-blah-blah-td5716461i20.html[/url]. Important: If it's not exactly your motherboard, don't try to use them! Best is to contact Asus.
I have included a note in my how-to.
Yep, the IVRS error affects most, if not all, ASUS motherboards that rely on the AMD 990FX and 970 chipsets. As you can see in the link to the thread in the Xen forums, ASUS issued a "test BIOS" to me for my ASUS M5A99FX PRO R2 (BIOS: M5A99FX-PRO-R20-ASUS-9903.CAP) and another "test BIOS" for a Sabertooth 990FX R2 (BIOS: SABERTOOTH-990FX-R20-ASUS-9901.CAP). Those "test BIOS" releases seemed to have cured the IVRS errors.

I don't know for certain what ASUS has planned, but I'm under the impression ASUS will at least make an effort to add the fix to future BIOS releases... perhaps not the next stable BIOS release, but eventually.

Anyone wanting a "test BIOS" for another ASUS motherboard that uses the AMD 990FX chipset or 970 chipset should consider contacting ASUS and providing them with enough information to get to the correct person. ASUS might issue a custom "test BIOS" for other motherboards upon request, but no promises.

Here's some information that may help to get to the right person at ASUS:
Name of my primary contact at ASUS: lijay_quan
Name of "test BIOS" issued for the ASUS M5A99FX PRO R2: M5A99FX-PRO-R20-ASUS-9903.CAP
Name of "test BIOS" issued for the ASUS Sabertooth 990FX R2: SABERTOOTH-990FX-R20-ASUS-9901.CAP

Hope the above helps.

Regards,
GizmoChicken

P.S. I'm also presently testing with KVM/VFIO. For those interested, here's a link to where I describe passthrough of a Windows 7 guest to my Nvidia GTX550Ti GPU using KVM/VFIO: https://bbs.archlinux.org/viewtopic.php ... 7#p1313007

Note that I haven't yet updated the description to reflect the fixed BIOS, and so, at present, the description still refers to a kernel-based IVRS override that I no longer use. That kernel-based IVRS override works with Kernel 3.11 in combination with KVM. I haven't test, and so I don't know for certain, but I doubt that the kernel-based IVRS override will work Xen. I'd love to hear that I'm wrong about this.
powerhouse
Level 6
Level 6
Posts: 1144
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

@GizmoChicken: Thanks for your response! I updated my how-to and put a link to both the Xen forum and your post here. I really hope Asus fixes this IOMMU issue in their next BIOS release. It would be also nice to see that they take Linux users more serious and invest a little in it (e.g. Linux support). I'm sure it will pay back many times.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
khagar

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by khagar »

Hi,


after upgrading ubuntu (dom0) from 13.04 to 13.10 my Windows domU didn't boot anymore (it just freezes at load; no error messages ). However, I switched back to the xm toolstack by running

Code: Select all

sudo sed -i 's/TOOLSTACK=.*\+/TOOLSTACK="xm"/' /etc/default/xen
and rebooting after that. Now everything works again :)
Locked

Return to “Virtual Machines”