HOW-TO make dual-boot obsolete using XEN VGA passthrough

[tigerdyr]

If I boot in Xen-4.3:

Code: Select all

mbalino@bolso /etc/xen $ cat /sys/hypervisor/properties/capabilities
xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 
mbalino@bolso /etc/xen $ egrep -c '(vmx|svm)' /proc/cpuinfo
0

When you are running in Xen, you are always running as a guest on top of a hypervisor, and as such you will not see the VT-d/VT-x features in the " already virtualized CPU". That is why if you boot into Linux without Xen, you will see those CPU features, but when you boot into Xen, you won't. That's the way it is by design.

[mbalino]

Relax buddy,
Let's go back to the basic.
1) You are using X79 chipset with Intel Core i7-4820K. Both VTd and VTx capable.
2) 2 graphic cards. ATI 7970 and ATI 7770. You want to passthrough ATI 7770.
3) Using Linux Mint Debian and not Linux Mint or any of the Ubuntu variant
4) I assume you got Xen4.3 installl with all PCI passthrough working.
5) Now can you boot into Xen 4.3 without problem?

Info
1. BIOS 4502 - 10/25/2013, Vt-d/x CAPABLE (grey color under CPU capabilities), Intel Virtualization : ENABLE (DISABLED)
2. I have the 7770 for DomU, but once I get everything running I will switch slots therefore the 7970 will be for DomU
3. Linux Current Version: Linux Mint Debian Testing Jessie (started from LMDE --> updated --> added repo Debiant Testing with priority 900 and updated again --> 1 Gb to update + CINNAMON (removed)
Conclusion: the system works really fast.
4. xen-4.3 installed, follow the steps and I ran the VM.
5. Boot in Xen 4.3 with no issue, the only thing is the FAIL message stating my CPU is nos capable
Other than that .... nothing that I recall

When I go to the VM, select add hardware --> PCI --> select 2nd vga --> Does not allow me Vt-d (IOMMU) error

[powerhouse]

Relax buddy,
Let's go back to the basic.
1) You are using X79 chipset with Intel Core i7-4820K. Both VTd and VTx capable.
2) 2 graphic cards. ATI 7970 and ATI 7770. You want to passthrough ATI 7770.
3) Using Linux Mint Debian and not Linux Mint or any of the Ubuntu variant
4) I assume you got Xen4.3 installl with all PCI passthrough working.
5) Now can you boot into Xen 4.3 without problem?

Info
1. BIOS 4502 - 10/25/2013, Vt-d/x CAPABLE (grey color under CPU capabilities), Intel Virtualization : ENABLE (DISABLED)
2. I have the 7770 for DomU, but once I get everything running I will switch slots therefore the 7970 will be for DomU
3. Linux Current Version: Linux Mint Debian Testing Jessie (started from LMDE --> updated --> added repo Debiant Testing with priority 900 and updated again --> 1 Gb to update + CINNAMON (removed)
Conclusion: the system works really fast.
4. xen-4.3 installed, follow the steps and I ran the VM.
5. Boot in Xen 4.3 with no issue, the only thing is the FAIL message stating my CPU is nos capable
Other than that .... nothing that I recall

When I go to the VM, select add hardware --> PCI --> select 2nd vga --> Does not allow me Vt-d (IOMMU) error

I have the same m/b and will check some things tomorrow. In the meantime please post:

1. The /etc/default/grub file
2. The /etc/xen/win7.cfg file (or whatever the name of your Windows domU config file)
3. The /etc/xen/pciback.cfg file

[mbalino]

I have the same m/b and will check some things tomorrow. In the meantime please post:

1. The /etc/default/grub file
2. The /etc/xen/win7.cfg file (or whatever the name of your Windows domU config file)
3. The /etc/xen/pciback.cfg file

Code: Select all

    
GRUB_DEFAULT="Xen 4.3-amd64"
# GRUB_HIDDEN_TIMEOUT=0
# GRUB_HIDDEN_TIMEOUT_QUIET=true
GRUB_TIMEOUT=10
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
GRUB_CMDLINE_LINUX=""
GRUB_CMDLINE_XEN="iommu=1 dom0_mem=2048M"

Code: Select all

 cat /etc/xen/win7.cfg
...
memory = 8196
kernel_override = "/usr/lib/xen-default/boot/hvmloader"
disk = ['/dev/mapper/os-Windows,raw,hda,rw']
device_model_override = '/usr/lib/xen-default/bin/qemu-dm'
tsc_mode = "default"

Code: Select all

mbalino@bolso ~ $ grep -v ^# /etc/xen/pciback.conf
0000:02:00.0
0000:02:00.1
mbalino@bolso ~ $ 

Code: Select all

mbalino@bolso ~ $ sudo xm pci-list-assignable-devices
0000:02:00.0
0000:02:00.1

[powerhouse]

@mbalino: I need the entire /etc/xen/win7.cfg

[powerhouse]

Relax buddy,
Let's go back to the basic.
1) You are using X79 chipset with Intel Core i7-4820K. Both VTd and VTx capable.
2) 2 graphic cards. ATI 7970 and ATI 7770. You want to passthrough ATI 7770.
3) Using Linux Mint Debian and not Linux Mint or any of the Ubuntu variant
4) I assume you got Xen4.3 installl with all PCI passthrough working.
5) Now can you boot into Xen 4.3 without problem?

Info
1. BIOS 4502 - 10/25/2013, Vt-d/x CAPABLE (grey color under CPU capabilities), Intel Virtualization : ENABLE (DISABLED)
2. I have the 7770 for DomU, but once I get everything running I will switch slots therefore the 7970 will be for DomU
3. Linux Current Version: Linux Mint Debian Testing Jessie (started from LMDE --> updated --> added repo Debiant Testing with priority 900 and updated again --> 1 Gb to update + CINNAMON (removed)
Conclusion: the system works really fast.
4. xen-4.3 installed, follow the steps and I ran the VM.
5. Boot in Xen 4.3 with no issue, the only thing is the FAIL message stating my CPU is nos capable
Other than that .... nothing that I recall

When I go to the VM, select add hardware --> PCI --> select 2nd vga --> Does not allow me Vt-d (IOMMU) error

1. EDIT 13.1.2014: The following is partly incorrect! VT-d has worked and is still working for the Asus Sabertooth X79 in conjunction with Sandybridge-E (SB-E) processors. The information on the vip.asus.com forum seems incorrect, at least I didn't find any issue with BIOS 2104, nor with the latest 4608 release.
Outdated/incorrect information (see above): Some time last year Asus introduced a BIOS bug that made VT-d unworkable, at least that what was reported on the Asus forum. See here: [url]http://vip.asus.com/forum/view.aspx?id=20120806185452699&board_id=1&model=SABERTOOTH+X79&page=1&SLanguage=en-us[/url].
Important: This bug was introduced more than a year ago. By now Asus has published several newer BIOS releases. However, it's quite possible that Asus never fixed the bug. If nothing helps, please contact Asus. Read the "hardware" section at the beginning with some more information on how and who to contact at Asus (GizmoChicken had some successful interaction with them and Asus actually fixed bugs in some of their AMD boards).
Note: I have the same board as you - the Asus Sabertooth X79 - but mine is older and I run it with an old BIOS that was not affected by the bug [EDIT: there is no bug regarding SB-E CPUs]. See also in my how-to.
Note 2: The fact that you can enable VT-d (or IOMMU) doesn't necessarily mean it's working. If you read the hardware section and follow the links to the Xen forum, you will find some detailed information on BIOS bugs, though the Sabertooth X79 most probably is a different bug. Again, I very much hope that Asus has fixed that by now, and you should first make sure that your configuration is OK.

2. This is not a good idea! You should insert your VGA cards into the proper slots and install the corresponding drivers. So if you want to use the 7770 for dom0 (Linux) and the 7970 for Windows, install the latter one in the second PCI-E slot.

3. Unless you have a really good reason to go with a "testing" release - don't. In fact, the closer you stick to this how-to, the better the chances that it will work. People here reported success with Linux Mint 13/14 but also 15 and 16 using various desktops. Xen 4.3 has a bug in the xl toolstack that wouldn't allow you to restart the domU (the first time you run Windows will be OK, but closing it and restarting it will probably freeze the PC). There is a work-around for this bug, but it requires some utility be installed in Windows and changing some shutdown script.

Note: The instructions in this how-to were written for the xm toolstack. The xl instructions and its domU config file are different.
I suggest using the xm toolstack for now, until the bug fix has been applied to Ubuntu / Linux Mint.

4. and 5. I need to see the entire win7.cfg file to comment on that. Also, make sure you followed all steps.

Here is what I would do now:

a) Find the PCI ID of one of you USB hosts (preferably a host/ports you are not using now). Then pass that through: add the PCI ID in the /etc/xen/pciback.cfg and in the /etc/xen/win7.cfg files.

b) Run

Code: Select all

sudo update-grub

just to make sure your /etc/default/grub file settings are being used.

c) Reboot and enter:

Code: Select all

sudo xm pci-list-assignable-devices

You should get the graphics card plus the PCI IDs of the USB host you are going to pass through.

d) Check which USB port(s) have been passed through by using a USB memory stick. Once you found the port, remove the stick.

e) Start the Windows VM. Once Windows is running insert the USB disk into the USB port you had passed through. If PCI passthrough is working Windows should recognize the stick, install a driver, and mount it.

Please report your results, particularly the last step.

P.S.: Don't forget sharing the full /etc/xen/win7.cfg (domU) config file. You should replace the MAC address(es) before posting.

[powerhouse]

Mmmhhh

I have already installed LM16 (6th time ...) I have no problem to do it one more time

Question: What is the benefits (real, for final user) to have LM16 vs LM14?
kernel features, packages updates, etc will not be available on LM14 since isnot a rolling release neither the latest one, am I right?

The only benefit I can see is that LM 16 and the newer kernel support hardware that wasn't around at earlier releases, so if you have the latest hardware then that might be reason. For all the rest there is very little if any reason to jump to LM 16. The LVM installer is nice, sure, but not a deal breaker. In fact, the best release right now is LM 13 because it's a long-term release that will give you years of support. I'm running LM 14 and can't complain about getting releases.

Another reason for having a new release is using external packages that require the latest release. I can't say I miss anything in LM 14 (or LM 13 for that).

By the way, sometimes newer releases introduce bugs or performance issues that weren't around before. People will remember the power management issues with Ubuntu and probably Linux Mint that appeared a few years ago and caused notebook batteries to be drained in no time. Another issue that still seems to be around (with LM 14) is the USB flash memory performance.

I thought that I can have sound in the VM without the needs for another sound card?

Before you spend money on another sound card, here is what I would check:

1. Can you connect your screen via HDMI? Your graphics card has a built-in sound chip which you can use via an HDMI interface.

2. If yes (HDMI), does you screen have built-in speakers? Or a headphone connector? In the first case Windows should take care of it and select the sound on your graphics card, as it won't find any other sound device (perhaps you'll first have to install the AMD graphics driver). That's the way it should work with VGA passthrough active.
In the second case you can use headphones or connect your stereo/active speakers to the headphone output.

3. If your screen has no connections and no built-in speakers and in case HDMI is not an option, you can get a cheap USB sound stick (I bought one for around $10) and connect the output of that stick to the input of your built-in sound. This way you can mix Linux and Windows sound. Of course, for better results you may want to invest in a real sound card. I have the Asus Xonar Essence and connected my stereo to it (without DAC, as I use the Xonar Essence). This card has also a headphone amplifier and is generally targeted to audiophiles. If you want an audiophile setup and bit-perfect sound with no resampling, let me know and I give you some links.
P.S.: My Asus Xonar runs under Linux, not Windows. But in either case it will work.

[mbalino]

You guys are right

I will format again and I will install KOLIVIA (LM15 - KDE) and start from the scratch.

My 2nd monitor -which will be conected to 2nd Vga (PT)- has speakers on it

Regards
Marcel

[FastRealm]

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough
by mbalino on Wed Dec 04, 2013 1:20 am

You guys are right

I will format again and I will install KOLIVIA (LM15 - KDE) and start from the scratch.

My 2nd monitor -which will be conected to 2nd Vga (PT)- has speakers on it

Regards
Marcel

- Don't use LM15 because xen 4.2 have issue with XL and XM if i remember correctly. You either use LM13/14(guaranty to work) or LM16(90%).
- So far LM16 with Xen 4.3 works fine except i cant install windows 8.1. I don't know if it is the Xen or my conf but the same conf on xen4.1 i can install win 8.1.
- I don't know if other encounter the same issue as me or not, if i use ATI 7xxx series i have problem with xen, but if i use an older card like ATI 4xxx series i got no issue.
If i use Intel sure work.

[mbalino]

Very well, PETRA will be again ...
I will do it tonight (installation)

What kind of problems did you find with Xen & ATI 7xxx?
Regards
Marcel

[powerhouse]

Very well, PETRA will be again ...
I will do it tonight (installation)

What kind of problems did you find with Xen & ATI 7xxx?
Regards
Marcel

I second fastrealm re LM 15 versus LMM 16. Use LM 16.

The AMD ATI 7xxx should work. There are many here that use it.

[powerhouse]

I finally found a little time to try Linux Mint 16 (installed on an external USB drive). I still haven't succeeded to get it running (the usual network-manager and ATI fglrx driver pitfalls), but hope to make it by tomorrow, if time permits. If all goes well I will revise my how-to to include Linux Mint 16. I also hope to be able to produce a mini-how-to, that is a greatly a shortened version.

[mbalino]

powerhouse

That work will be greatly appreciated since I could not reproduce my steps and get xen works again.

I did ALL my steps again but I can't get it work

I almost give up, but I am eagerly waiting for your new update!!

Regards
Marcel

[powerhouse]

Finally I made it! I installed Linux Mint 16 with Xen 4.3 and VGA passthrough. I have added the instructions in my original how-to.

Linux Mint 16 's version of network-manager is a major pain in the you know what. Everything went smooth except getting rid of that piece of nuisance. Eventually I found the right sequence and adding the DNS name server in the /etc/network/interfaces file helps.

I also took this opportunity to change the way the graphics card is bound to pciback, using the initramfs. This means that you could now dispose of the pciback script in /etc/init.d altogether, and use a small script to based on pciback and xm create ... to start the Windows guest. I left the pciback script in place as it is the simplest approach.

Do I recommend Linux Mint 16? Yes and no. It's better than Linux Mint 15, but I see no need to upgrade from Linux Mint 13 or 14 unless with hardware that requires the newer kernel. The network configuration in Linux Mint 16 is definitely more tricky than Linux Mint 13 or 14 (I didn't try 15).

Please report back if you find any "bugs" or problems.

[FastRealm]

In LM16 with Xen 4.3, Windows 8.1 doesn't work. It just hangs.
I use WICD to replace my network-manager but sadly WICD doesn't work on LM16 at the moment.
Other than that everything looks good, but still go back to LM14 because LM16 can't display Chinese and Japanese fonts.

[powerhouse]

In LM16 with Xen 4.3, Windows 8.1 doesn't work. It just hangs.
I use WICD to replace my network-manager but sadly WICD doesn't work on LM16 at the moment.
Other than that everything looks good, but still go back to LM14 because LM16 can't display Chinese and Japanese fonts.

I tried WICD and who knows what in the past - they are of no use here. The only success I ever had was getting rid of network-manager and doing the network configuration in the /etc/network/interfaces file. Another important file to look out for is /etc/resolv.conf. It should be a link and when you open the linked file you should see a line such as:

Code: Select all

nameserver 8.8.8.8

I'm currently on LM16 (booted from an external USB drive) to test it, but the way it looks now I see no reason to upgrade from LM14 which is what I run normally.

Thanks about the Windows 8.1 info - I will mention it in the how-to. By the way, did you follow my updated how-to to install LM16? Would like to get some feedback from others to make sure the steps I gave are indeed working (and that I haven't missed anything).

[powerhouse]

Just installed Wine and PlayOnLinux under LM16 and Xen 4.3. Both seem to work OK, though I had little luck getting Lightroom 4 to run (the installation went like a breeze using PlayOnLinux).

The point I want to make is that you can run Linux Mint on a hypervisor with Wine and some Windows applications, and run Windows as a guest. So, whenever you want to play a game or run an application that is not supported by Wine, you can still run your Windows as a guest and install it there. With more and more games becoming available under Linux or Steam you may be able to "move" them from the Windows VM to your Linux environment. The important part is that both work on the same machine, at the same time.

[FastRealm]

@powerhouse
I didn't follow the updated guide. I was still using the old guide before you updated it.
using the older method i need to install qemu, passthrough the vga and also usb using initramfs, if not the usb sometimes doesn't get passthrough and it will cause the xen to crash. And I also compile my own kernel with all the xen functions enable.
Other than that all work fine without issue.
Will try out the new method this weekend and feedback to see if you miss out anything.

[powerhouse]

@powerhouse
I didn't follow the updated guide. I was still using the old guide before you updated it.
using the older method i need to install qemu, passthrough the vga and also usb using initramfs, if not the usb sometimes doesn't get passthrough and it will cause the xen to crash. And I also compile my own kernel with all the xen functions enable.
Other than that all work fine without issue.
Will try out the new method this weekend and feedback to see if you miss out anything.

Thanks! Looking forward to your comments.

[sivel27]

Great guide powerhouse !

Sent from my Nexus 4 using Tapatalk