HOW-TO make dual-boot obsolete using XEN VGA passthrough

Write tutorials and howtos in here
There are more tutorials here http://community.linuxmint.com/tutorial/welcome
Forum rules
Do not start a support topic here please. Before you post please read this

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby mbalino on Thu Jan 16, 2014 7:28 pm

Powerhouse:
I would like to thank you for your amazing post, and all other users that replied and tried to help me.

It is unfortunate but I gave up and deleted LM16 and got back to W8.1PRO :cry:

I have a RHD7770 to use ... :oops: since I can not use with my HD6850 or my HD7970 for crossfire ..
so, will try again in the near future with my previous i5-2500k, but for now I GIVE UP ...

Thanks all of you again!
mbalino
Level 1
Level 1
 
Posts: 43
Joined: Sat Nov 23, 2013 11:02 pm

Linux Mint is funded by ads and donations.
 

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby powerhouse on Fri Jan 17, 2014 5:01 pm

mbalino wrote:Powerhouse:
I would like to thank you for your amazing post, and all other users that replied and tried to help me.

It is unfortunate but I gave up and deleted LM16 and got back to W8.1PRO :cry:

I have a RHD7770 to use ... :oops: since I can not use with my HD6850 or my HD7970 for crossfire ..
so, will try again in the near future with my previous i5-2500k, but for now I GIVE UP ...

Thanks all of you again!


Sorry to hear. By the way, don't try the i5-2500K - it won't work. The 2500K, like most "K" type CPUs, doesn't support VT-d.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Sapphire 7770 GPU, PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB SSD + various HD, Corsair 500R case, SeaSonic 660W Gold X PS
powerhouse
Level 5
Level 5
 
Posts: 576
Joined: Thu May 03, 2012 3:54 am

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby powerhouse on Wed Jan 22, 2014 3:18 pm

If you support a dedicated "virtualization forum" for Linux Mint, you can vote for it here: http://forums.linuxmint.com/viewtopic.php?f=29&t=112951
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Sapphire 7770 GPU, PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB SSD + various HD, Corsair 500R case, SeaSonic 660W Gold X PS
powerhouse
Level 5
Level 5
 
Posts: 576
Joined: Thu May 03, 2012 3:54 am

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby thelowendtheory on Fri Jan 24, 2014 2:00 pm

Hello, I followed this guide and get
Code: Select all
Error: Device model isn't ready for commands

When i use xm create, I read somewhere that the qemu log will tell me which device is not working, but I don't know where that is.
thelowendtheory
Level 1
Level 1
 
Posts: 1
Joined: Fri Jan 24, 2014 1:57 pm

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby powerhouse on Fri Jan 24, 2014 7:24 pm

thelowendtheory wrote:Hello, I followed this guide and get
Code: Select all
Error: Device model isn't ready for commands

When i use xm create, I read somewhere that the qemu log will tell me which device is not working, but I don't know where that is.


The reason for this error can be any of the three:

1. Configuration error in your /etc/xen/win.cfg file (whatever you call it) - check that the file contains:
Code: Select all
device_model = '/usr/lib/xen-default/bin/qemu-dm'

Check that qemu-dm in fact exists at that location - if not, see the next two points.
2. A missing link - see step 7 (note the different links for different releases).
3. Missing package - see step 6, if you run LM16 you need to install some more packages.

EDIT: You find the qemu-dm log files here:
Code: Select all
/var/log/xen
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Sapphire 7770 GPU, PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB SSD + various HD, Corsair 500R case, SeaSonic 660W Gold X PS
powerhouse
Level 5
Level 5
 
Posts: 576
Joined: Thu May 03, 2012 3:54 am

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby luminarius on Sun Jan 26, 2014 12:45 am

Would this be at all possible on a laptop? In other words, vga, pci passthrough exist on some given laptop?
luminarius
Level 1
Level 1
 
Posts: 1
Joined: Sun Jan 26, 2014 12:41 am

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby powerhouse on Mon Jan 27, 2014 1:54 am

luminarius wrote:Would this be at all possible on a laptop? In other words, vga, pci passthrough exist on some given laptop?

Yes, but the laptop needs to comply with the hardware requirements (e.g. VT-d) and possess 2 GPUs. However, it is also possible to use VGA passthrough with only one GPU, but you would need a second PC to connect to the laptop and setup the system.
In case you use only one GPU, you would need to boot into the guest system, without giving the Linux host any GPU (headless system). Access to the host would be through remote desktop. However, it would be much better and easier to have 2 GPUs (for example an IGPU and a discrete graphics chip).
For reference, see here: http://discussions.citrix.com/topic/338545-hotfix-xs62e005-gpu-passthrough-on-single-gpu-host/.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Sapphire 7770 GPU, PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB SSD + various HD, Corsair 500R case, SeaSonic 660W Gold X PS
powerhouse
Level 5
Level 5
 
Posts: 576
Joined: Thu May 03, 2012 3:54 am

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby powerhouse on Wed Feb 05, 2014 4:45 am

I have shortened my how-to and left only the LM 16 related instructions. For those who run Linux Mint 13 or 14, here is the old version (dated 05.02.2014 in the change log):

Xen VGA passthrough how-to for Linux Mint 13, 14, and 16 - release 2.1.2
Change log:
05.02.2014 - added link to alternative guide, shortened introduction, changed caption ("Hardware requirements")
27.01.2014 - removed outdated comments re Xen versions, added note re Fedora 20 guide
13.01.2014 - removed /etc/modules xen-pciback entry as we use the initramfs method, shortened text
11.01.2014 - fixed win7.cfg file formating errors (the " ' etc. quotes got messed up)
09.01.2014 - update on Asus Sabertooth X79 VT-d support
07.01.2014 - removed virt-manager as it doesn't work
06.01.2014 - added link to synergy website, enhanced "Preparation and hardware selection" section
29.12.2013 - new recommendation - use Linux Mint 16
20.12.2013 - VT-d not working with Ivybridge CPUs on Asus Rampage and Sabertooth X79 boards - new BIOS versions to be released soon
19.12.2013 - appeal to post Passmark / benchmark results here http://forums.linuxmint.com/viewtopic.php?f=225&t=153482
17.12.2013 - added info/link to identifying PCI ID of USB ports (controllers); added Xen release 4.1.5-0ubuntu0.12.10.1 as NOT working with VGA/PCI passthrough
10.12.2013 - warning about LM16 and Windows 8.1
09.12.2013 - added instructions for LM16; use initramfs to hide graphics card; major makeover of how-to
08.12.2013 - added references to LM16
22.11.2013 - link to successful Linux Mint 16 Cinnamon installation post
09.11.2013 - Asus BIOS update - link to post
26.10.2013 - added reference to Linux Mint 15 solution using xm toolstack
17.09.2013 - update about Asus test BIOS - link to post
13.09.2013 - revised note about faulty Asus BIOS - link to beta BIOS reference
13.09.2013 - added note about faulty BIOS in some Asus motherboards
21.07.2013 - added link to workaround for LM15 users, using the xm toolstack
16.07.2013 - added warning about Linux Mint 15 and unresolved bugs
11.07.2013 - PCI passthrough bug in Xen 4.1.3-3ubuntu1.7 - update
26.06.2013 - added link to post re xl toolstack and domU config file changes
20.06.2013 - adjust vcpus in win7.cfg file to more sensible values
14.06.2013 - Xen hypervisor 4.1.3-3ubuntu1.6 - PCI passthrough not working
02.06.2013 - link to Synaptic screen shots to force version - workaround for XSA46 bug
19.05.2013 - notes on Linux Mint 15: Xen 4.2 and xl toolstack, LVM option in installer; gfx_passthru=1 explanation
30.04.2013 - warning re xen hypervisor release 4.1.3-3ubuntu1.5 upgrade
29.04.2013 - added note on alternative method to load xen-pciback using the initramfs
18.04.2013 - added enable VT-d / IOMMU in BIOS step - renumbered steps
16.04.2013 - removed summary of steps; added info on AMD graphics cards
15.04.2013 - changed to consecutive numbering; commented out pae=1 option; added pci power management option and explanation; additional references
07.04.2013 - note on keyboard/mouse control using Xen with link to post; blacklisting radeon or nouveau driver
02.04.2013 - appeal to open "virtualization" forum on LM; AMD CPUs; Intel VT-d; Nvidia modding
17.02.2013 - link to latest GPLPV drivers and new instructions on using them; comment out "auto eth0" entry in /etc/network/interfaces
01.02.2013 - fixed ark.intel.com link to list VT-d enabled CPUs and added a link re VT-d M/B, added glossary
21.01.2013 - additional notes / corrections for Linux Mint 14 Mate
18.01.2013 - additional references and links; cleaning up the text
13.01.2013 - enabled STP in network config; added localtime=1 to domU config file
10.01.2013 - added note on LM 14; added usb-devices command for USB PCI ID identification
10.12.2012 - note about bridging and the network-manager package
12.11.2012 - minor changes/additions in part 4, particularly the Windows installation part
23.10.2012 - changed the title to make it more clear what it's all about: not having to dual-boot anymore and yet getting full graphics performance
18.09.2012 - additions in Part 2 and Part 3 (step 10)


Want a dedicated virtualization forum? Vote here:http://forums.linuxmint.com/viewtopic.php?f=29&t=112951

Abstract: This how-to is about running Microsoft Windows concurrently with Linux Mint on a Xen hypervisor, with FULL graphics acceleration for the Windows guest system.

There are special hardware requirements for this to work. Please make sure your hardware meets these requirements.

Running Windows as a guest in Linux is nothing new. But how about native graphics and near-native CPU performance? Here a little teaser - this is my Windows Experience Index when running Windows on top of Linux / Xen hypervisor, while Linux is fully functional at the same time:

Win7_VM_on_Xen.jpg

WEI using GPLPV block device driver and VGA graphics passthrough!


Introduction

For a long time I wanted to get rid of dual-booting Linux and Windows. But my photo editing software would only run on Windows (or Mac). So I started to look into virtualization. I needed true graphics acceleration from within Windows, as well as direct access to some USB ports, something VirtualBox cannot provide as of this writing!

Reading reports from users who had installed Windows as a virtual machine (VM) on top of a Xen hypervisor, with full graphics acceleration under Windows, I decided to go for it.

The following is for people who want to run Linux and Windows at the same time - without the need for dual-boot - with both Linux and Windows having native or near native graphics acceleration and performance.

Xen employs two related concepts that make it work: PCI passthrough and VGA passthrough.


Glossary

Here a few terms used in the how-to:
domU: This is a Virtual Machine or VM in Xen terminology. In our case it will be Windows.
dom0: This is a special Virtual Machine with administrative privileges. In our case this will be Linux Mint. All other VMs (or domUs in Xen talk) are started and controlled from the dom0.
PCI passthrough (or passthru): This is a technique which allows the dom0 to pass through a PCI device to the guest domU. The guest OS - in our case Windows - then uses its own hardware driver to access the device directly.
VGA passthrough: This is a specialized form of PCI passthrough for graphics cards / GPU. These devices are more complex and so is the interaction with them. Fortunately, using the right hardware together with Xen should take care of that. There are two forms of VGA passthrough:
VGA passthrough to GPU as primary adapter: The guest system (domU) boots and uses the passed through graphics card as its primary or only graphics adapter.
VGA passthrough to GPU as secondary adapter: The guest system first boots using the Qemu-dm virtual Cirrus adapter as the primary graphics adapter (this is where you see the BIOS screen and the Windows boot logo). After installation of the graphics driver under Windows and subsequent reboot of the domU, the system will switch to the secondary graphics adapter at some stage of it's boot process. This is the technique described here in this How-To.


Hardware requirements

First of all you need to check the hardware. This will ONLY work when the hardware requirements below are met, if not, forget this tutorial !!!

Hardware requirements:

    • CPU with VT-d support (Intel terminology) - see http://ark.intel.com/search/advanced/?s=t&VTD=true. When using an AMD CPU, it must support AMD-Vi, also known as IOMMU - the following chipsets support IOMMU: SR5690/SR5670 server chipsets, AMD 890FX, 990FX, 990X and 970 desktop chipsets. See http://wiki.xen.org/wiki/VTdHowTo for more information.
    • Motherboard and BIOS version with option to enable VT-d or IOMMU (beware, some vendors have buggy BIOS versions) - see http://wiki.xen.org/wiki/VTd_HowTo and http://www.overclock.net/t/1338063/vt-d-compatible-motherboards for compatible motherboards!
    • A dedicated graphics card for your Windows guest that supports Xen VGA passthrough - see http://wiki.xen.org/wiki/Xen_VGA_Passthrough_Tested_Adapters and http://www.overclock.net/t/1307834/xen-vga-passthrough-compatible-graphics-adapters for compatible graphics cards! In most cases, AMD will be a better choice, unless you got a "Multi-OS" Nvidia card - see http://www.nvidia.com/object/sli_multi_os.html.
    • Enough RAM - the more the better, though 8GB should do in most cases
    • Linux Mint 13 or later (or Ubuntu 12.04 or later), 64bit version - I recommend Linux Mint 16 !
    • Windows 7, Windows 8 (see note on Windows 8.1 below), or Windows XP

Check the above links! If you have a compatible CPU, motherboard (and BIOS), and graphics card you can skip to the how-to below.

Note: Some Asus motherboards - specifically those for AMD CPUs with AMD 990FX and 970 chipsets - have a faulty BIOS. This has been reported here http://www.gossamer-threads.com/lists/xen/users/283968?do=post_view_threaded#283968. A workaround is to install an older version of Xen using the "force" option in Synaptic. For more see here http://forums.linuxmint.com/viewtopic.php?f=42&t=112013&start=200#p764039.
EDIT: Asus has fixed it for at least two boards - see http://xen.1045712.n5.nabble.com/Xen-IOMMU-disabled-due-to-IVRS-table-Blah-blah-blah-td5716461i20.html and http://forums.linuxmint.com/viewtopic.php?f=42&t=112013&start=200#p764822 (thanks GizmoChicken!)

EDIT 2: Asus has released a new BIOS version v2005 that reportedly fixes the issues with some of its AMD CPU boards. For more details, see GizmoChicken's post here: http://forums.linuxmint.com/viewtopic.php?f=42&t=112013&start=220#p782159

Ivybridge CPUs on Asus Rampage and Sabertooth X79 boards: Asus Rampage and Sabertooth X79 boards currently don't support VT-d when used with the new generation Ivybridge CPUs. The good news is that Asus R&D is working on it and has released a beta BIOS to fix VT-d in one of their Rampage boards. A BIOS release is expected for the Asus Sabertooth X79 that is supposed to fix the issue. See here: http://rog.asus.com/forum/showthread.php?40309-No-VT-d-support-for-IVB-E-on-R4E-boards.

Here is my hardware list for reference:

- i7 3930K CPU with C2 stepping! (If you buy new, it should be with C2 stepping, but check for spec code = SR0KY!)
- Asus Sabertooth X79 motherboard with BIOS version 4608 (latest BIOS as of 09.01.2014)
- AMD 7770 graphics card as primary graphics adapter for Linux dom0 (before that I had an AMD 6450)
- Nvidia Quadro 2000 graphics card with "Multi-OS" feature (Nvidia terminology) as secondary graphics adapter for Windows domU - see below for more on that!
- USB KVM switch to connect keyboard/mouse to two different USB hosts and easily switch between them
- A graphics display with two DVI or HDMI ports and the ability to switch between the ports (e.g. "input" button)
- Enough RAM (I have 32GB) and disk space (I use a 120GB SSD and several 2TB hard drives, less will do)

Most Nvidia graphics cards are tricky to pass through to the guest OS (Windows domU), only the "Multi-OS" versions are designed to work out of the box. You could try to patch and rebuild the Linux kernel and Xen, and there are some instructions on how to do it, but this is way more demanding than getting my above hardware to work. I have failed with a Nvidia Quadro 600 card, though it may be possible. Most AMD (ATI) graphics cards are easier to get working with VGA passthrough. See http://wiki.xen.org/wiki/Xen_VGA_Passthrough_Tested_Adapters and http://wiki.xen.org/wiki/Xen_VGA_Passthrough. If in doubt, get one of the tested AMD graphics adapters that are listed under the above mentioned links!

For the very adventurous, there is a way to modify Nvidia graphics adapters to register as professional Nvidia cards and thus enable VGA passthrough. See here: http://www.eevblog.com/forum/projects/hacking-nvidia-cards-into-their-professional-counterparts/ and http://www.eevblog.com/forum/projects/hacking-nvidia-cards-into-their-professional-counterparts/msg207550/#msg207550. Warning: If you do modify your hardware, you will loose all manufacturer warranty and take a big chance of turning your expensive VGA card into a brick! I won't take any responsibility for that. You have been warned.

If you are planning to build a new PC, this is the time to check for compatible hardware!


How-to for Linux Mint 13 and 14 (for LM 16 see first post in this thread)

Note about Windows guests: This how-to has been written and tested with Windows 7. At least one user reported that this Linux Mint 16 / Xen 4.3 how-to doesn't work with a Windows 8.1 guest - see http://forums.linuxmint.com/viewtopic.php?f=42&t=112013&start=260#p794153. It may be an isolated incident - if you succeed with Windows 8.1, please report.


1. Install Linux Mint 13/14 or 16 64 bit with LVM / (root) and /home partitions

For Linux Mint 13 or 14, see my how-to here: http://forums.linuxmint.com/viewtopic.php?f=42&t=108442.

For Linux Mint 16, see my how-to here: http://forums.linuxmint.com/viewtopic.php?f=42&t=108442#p793203.

Recommendation: For those who follow this tutorial the first time, use Linux Mint 16 with Xen 4.3 and the "xm" tool stack. For those who already installed an earlier version - don't bother unless you need newer packages (though many have been back ported to LM13 which is a long term release).

Note: Xen 4.1, 4.2, and 4.3 have a bug when using the "xl" toolstack (the default toolstack from Xen 4.2 onwards) that causes issues when rebooting the Windows guest, or any other guest that uses PCI or VGA passthrough. Until that bug has been resolved in Ubuntu and Linux Mint, use the "xm" toolstack.


2. Enable VT-d / IOMMU in BIOS

Reboot your PC and enter your BIOS setup. Look for an IOMMU or VT-d option (check the manual of your motherboard) and enable it. VT-d is an essential prerequisite to make this work!

Note: Most motherboard BIOSes have VT-d disabled by default.


3. Install Xen hypervisor and utilities

After reboot into the installed Linux Mint system, update the packages using the "Update Manager". Then open a terminal window:

Linux Mint 13:

Code: Select all
sudo apt-get install xen-hypervisor-4.1-amd64=4.1.3-3ubuntu1 xen-utils-4.1=4.1.3-3ubuntu1 xen-utils-common=4.1.3-3ubuntu1 bridge-utils

Note: See below "Important note for Linux Mint 13 and 14".

Linux Mint 14:

Code: Select all
sudo apt-get install xen-system-amd64=4.1.3-3ubuntu1

Important note for Linux Mint 13 and 14: When updating the Xen hypervisor to version 4.1.3-3ubuntu1.5 or newer you will get an error when trying to load the Windows guest. Affected versions are 4.1.3-3ubuntu1.5, 4.1.3-3ubuntu1.6, 4.1.3-3ubuntu1.7, and 4.1.5-0ubuntu0.12.10.1. This is why the packages above are pinned to release 4.1.3-3ubuntu1. If you inadvertently upgrade the Xen-related packages, you can downgrade them using the "Force version..." option in the "Package" menu of Synaptic Package Manager (see screenshot below). You will need to downgrade the following packages:
Code: Select all
libxen-4.1 libxenstore3.0 xen-docs-4.1 xen-hypervisor-4.1-amd64 xen-system-amd64 xen-utils-4.1 xen-utils-common xenstore-utils

See http://forums.linuxmint.com/viewtopic.php?f=42&t=112013&p=725951#p725951.
Screenshot-Synaptic Package Manager.png




Linux Mint 16:

Code: Select all
sudo apt-get install xen-system-amd64

We want the "xm" toolstack, since the default "xl" toolstack has issues/bugs:
Code: Select all
sudo sed -i 's/TOOLSTACK=.*\+/TOOLSTACK="xm"/' /etc/default/xen



4. Add boot options to grub

Edit the /etc/default/grub file as root and add or change the following:
Code: Select all
GRUB_CMDLINE_XEN="iommu=1 dom0_mem=2048M"

The above Xen command line tells the hypervisor to use IOMMU (VT-d must be ENABLED in your BIOS) and gives the dom0 (Linux Mint) 2048 MB of RAM. Change the amount of RAM you give to your Linux Mint dom0 according to your needs.

While at it, comment out the GRUB_HIDDEN_TIMEOUT=0 and GRUB_HIDDEN_TIMEOUT_QUIET=true so that it looks like this:
Code: Select all
GRUB_DEFAULT="Xen 4.1-amd64"
# GRUB_HIDDEN_TIMEOUT=0
# GRUB_HIDDEN_TIMEOUT_QUIET=true
GRUB_TIMEOUT=10
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash nomodeset"
GRUB_CMDLINE_LINUX=""
GRUB_CMDLINE_XEN="iommu=1 dom0_mem=2048M"


Linux Mint 16: You need to change
Code: Select all
GRUB_DEFAULT="Xen 4.1-amd64"

to
Code: Select all
GRUB_DEFAULT="Xen 4.3-amd64"


Note: I changed my GRUB_DEFAULT to load Xen automatically at boot. With the GRUB_HIDDEN... options commented out you will have a grub menu to select your regular Linux kernel if things go bad, or to manually select either Xen or regular boot.

Open a terminal window and run
Code: Select all
sudo update-grub



5. Reboot into the Xen hypervisor

Select the appropriate grub menu option.


6. Adjust the network settings to use a bridge

The following terminal commands require root permission so we make it stick. Open a terminal window and enter:
Code: Select all
sudo -i


Linux Mint 16: We need to install some more packages:
Code: Select all
apt-get install qemu-common qemu-keymaps bridge-utils


Then for Linux Mint 13:
Code: Select all
/etc/init.d/network-manager stop


Linux Mint 14 and 16:
Code: Select all
stop network-manager

Stops the network manager so we can configure the bridge and enable it.

No matter which Linux Mint version you use, you need to decide if you want to use a dynamic IP address for your Linux Mint dom0, or a static address. I recommend using a static IP address (the second option below), as this will allow you to easily access your Linux dom0 from another PC in your network, or from within the Windows guest.

If you want to use DHCP to automatically assign an IP address etc. to your network card, replace the content of your /etc/network/interfaces file with this:
Code: Select all
auto lo
iface lo inet loopback

auto xenbr0
iface xenbr0 inet dhcp
    bridge_ports eth0

# auto eth0


For a static IP address, use this configuration in /etc/network/interfaces:
Code: Select all
auto lo
iface lo inet loopback

auto xenbr0

iface xenbr0 inet static
address 192.168.0.120  # IP of Linux Mint dom0
broadcast 192.168.0.255
netmask 255.255.255.0
gateway 192.168.0.1  # IP of router or modem
dns-nameservers 8.8.8.8  # IP(s) of DNS name server(s)
bridge_ports eth0
bridge_stp off    # disable Spanning Tree Protocol - optional
# bridge_waitport 0   # no delay before a port becomes available - optional
# bridge_fd 0    # no forwarding delay - optional

# There are no IP configurations for eth0, it’s all defined under xenbr0:
# auto eth0

Adopt the IP settings above to your requirements.

Enable networking:
Code: Select all
service networking restart

and check your network configuration:
Code: Select all
ifconfig


Note: If you do not find a xenbr0 entry, you may need to define a bridge using brctl. In the terminal window type:
Code: Select all
brctl addbr xenbr0

and check again with ifconfig.

Then test your network configuration - open Firefox and see if you have Internet access.

If all is fine, remove the "network-manager" package as this is a major pain in the neck (to be polite):
Code: Select all
apt-get remove network-manager



7. Create missing links

Linux Mint requires some fixes for the xen tools to work properly.

For Linux Mint 13 enter the following commands in the terminal window (as root):
Code: Select all
ln -s /usr/lib /usr/lib64
ln -s /usr/lib/xen-default /usr/lib/xen
ln -s /usr/share/qemu-linaro /usr/share/qemu


For Linux Mint 14 use these:
Code: Select all
ln -s /usr/lib /usr/lib64
ln -s /usr/lib/xen-4.1 /usr/lib/xen-default
ln -s /usr/share/qemu-linaro/qemu/ /usr/share/qemu

Thanks to Nesousx.

For Linux Mint 16 use these:
Code: Select all
ln -s /usr/lib /usr/lib64
ln -s /usr/lib/xen-4.3 /usr/lib/xen-default



8. Determine the PCI IDs of the devices you want to pass through to the domU (guest)

Enter the following command to list your graphics adapters' PCI IDs:
Code: Select all
lspci | grep VGA

You should get something like this:
    01:00.0 VGA compatible controller: Advanced Micro Devices [AMD] nee ATI Caicos [Radeon HD 6450]
    02:00.0 VGA compatible controller: NVIDIA Corporation GF106GL [Quadro 2000] (rev a1)
I have two graphics adapters:
    01:00.0 is the primary graphics adapter installed in the first PCIe 16 slot, in my case an AMD Radeon HD 6450 graphics card. This is the GPU currently used by Linux dom0.

    02:00.0 is the secondary graphics adapter installed in the second PCIe 16 slot, in my case a Nvidia Quadro 2000 card. This is the one we want to pass through to the guest system. This card MUST support VGA passthrough, or else this whole procedure will fail !!! Read the hardware requirements above at the beginning very carefully.

Make a note of the PCI ID you want to pass through. Most likely your graphics card will have a second PCI ID for the onboard audio controller. This is typically 02:00.1 (for the card with PCI ID 02:00.0). You need to pass that through as well!

To check, enter the following and scroll through the output:
Code: Select all
lspci | grep 02:00.

You will see something like this:
    02:00.0 VGA compatible controller: NVIDIA Corporation GF106GL [Quadro 2000] (rev a1)
    02:00.1 Audio device: NVIDIA Corporation GF106 High Definition Audio Controller (rev a1)

If you don't get the second line, there is no onboard audio controller you need to pass through, so use only the 02:00.0 PCI ID.

Note: You may get different PCI IDs altogether, so use the ones appropriate to your system.

Next are the USB devices (see also the "Alternative method" below):
Code: Select all
lspci | grep USB

The output on my machine is this:
    00:1a.0 USB controller: Intel Corporation X79 series chipset USB2 Enhanced Host Controller #2 (rev 05)
    00:1d.0 USB controller: Intel Corporation X79 series chipset USB2 Enhanced Host Controller #1 (rev 05)
    06:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller
    07:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller
    08:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller

I have 5 host controllers, and each of mine has 2 or more ports (IIRC).

I decided to pass through the second Intel USB controller for mouse/keyboard, since it's a USB-2 host controller which may give less troubles than USB-3. I also have a USB KVM switch that allows me to switch between two USB ports on the PC - one cable is connected to the passed-through USB port, the other to a USB-2 port that hasn't been passed through (for Linux). Here is my setup:
00.1a.0 is passed through to the domU (Windows 7) guest and one cable of the KVM is connected to this port (see below on how to determine that).
00.1d.0 is used with dom0 (Linux) and the second cable of the KVM is connected to that port.

To find out the PCI ID of the host controller(s) where your keyboard and mouse is currently connected, use:
Code: Select all
usb-devices

and scroll down to your mouse and keyboard entries. The PCI ID can be found on the "SerialNumber=" line of the corresponding Host Controller. In the example below my USB Multimedia Keyboard is connected to the EHCI Host Controller with the PCI ID 0000:00:1a.0:

    T: Bus=01 Lev=00 Prnt=00 Port=00 Cnt=00 Dev#= 1 Spd=480 MxCh= 2
    D: Ver= 2.00 Cls=09(hub ) Sub=00 Prot=00 MxPS=64 #Cfgs= 1
    P: Vendor=1d6b ProdID=0002 Rev=03.11
    S: Manufacturer=Linux 3.11.0-12-generic ehci_hcd
    S: Product=EHCI Host Controller
    S: SerialNumber=0000:00:1a.0
    C: #Ifs= 1 Cfg#= 1 Atr=e0 MxPwr=0mA
    I: If#= 0 Alt= 0 #EPs= 1 Cls=09(hub ) Sub=00 Prot=00 Driver=hub

    T: Bus=01 Lev=01 Prnt=01 Port=00 Cnt=01 Dev#= 2 Spd=480 MxCh= 6
    D: Ver= 2.00 Cls=09(hub ) Sub=00 Prot=01 MxPS=64 #Cfgs= 1
    P: Vendor=8087 ProdID=0024 Rev=00.00
    C: #Ifs= 1 Cfg#= 1 Atr=e0 MxPwr=0mA
    I: If#= 0 Alt= 0 #EPs= 1 Cls=09(hub ) Sub=00 Prot=00 Driver=hub

    T: Bus=01 Lev=02 Prnt=02 Port=04 Cnt=01 Dev#= 3 Spd=12 MxCh= 0
    D: Ver= 1.10 Cls=00(>ifc ) Sub=00 Prot=00 MxPS= 8 #Cfgs= 1
    P: Vendor=058f ProdID=9410 Rev=01.22
    S: Product=USB Multimedia Keyboard
    S: SerialNumber=USB Multimedia Keyboard
    C: #Ifs= 3 Cfg#= 1 Atr=a0 MxPwr=50mA
    I: If#= 0 Alt= 0 #EPs= 1 Cls=03(HID ) Sub=01 Prot=01 Driver=usbhid
    I: If#= 1 Alt= 0 #EPs= 1 Cls=03(HID ) Sub=00 Prot=00 Driver=usbhid
    I: If#= 2 Alt= 0 #EPs= 1 Cls=03(HID ) Sub=01 Prot=02 Driver=usbhid
    ...

:arrow: Tip: A detailed description on identifying USB ports and their corresponding PCI IDs can be found here: http://forums.linuxmint.com/viewtopic.php?f=42&t=112013&p=795878#p795878.

My KVM has two ports to connect both keyboard and mouse to it. If you don't use a KVM switch, it's best to have both keyboard and mouse on the same controller, so you may need to replug them and check again.

Alternative method: You can use "synergy" to share one mouse and keyboard between your Linux dom0 and your Windows guest, in which case you do not need to pass through USB hosts. See here: http://synergy-foss.org/. To install "synergy", enter:
Code: Select all
apt-get install synergy libcrypto++9 quicksynergy

For more information on using synergy, see http://forums.linuxmint.com/viewtopic.php?f=42&t=112013&start=80#p706770. Thanks gordon.cooke !


9. Create pciback.conf file

:arrow: If you do NOT pass through USB ports, skip to step 11.

Once you determined the PCI IDs for your graphics adapter and USB port(s) you want to pass through to Windows, create a config file named pciback.conf in /etc/xen with the PCI ID(s) of the USB host(s) you wish to pass through, for example:
Code: Select all
# This file contains the list of pci devices the pciback driver should bind
# to at system startup.
# Please provide the devices in long BDF notation.
# Example:
# 0000:08:00.0
# NO EMPTY LINES!
#
0000:00:1a.0

Note: Replace the PCI ID in this file with the one(s) you determined before! You must enter the long PCI ID notation, with a preceding 0000: for the domain! This configuration file is used by the pciback shell script below.


10. Install and activate pciback script

We are using the xen-pciback "hide" command to hide the second graphics card and bind it to xen-pciback (see above). However, if you wish to pass through USB hosts to your Windows guest, you can use the script below for convenience.

Save the following pciback script named pciback to /etc/init.d:
Code: Select all
#!/bin/sh
#
### BEGIN INIT INFO
# Provides: pciback
# Required-Start: $local_fs
# Required-Stop: $local_fs
# Should-Start:
# Should-Stop:
# X-Start-Before: xend
# X-Stop-After:
# Default-Start: 2 3 4 5
# Default-Stop:
### END INIT INFO

. /lib/lsb/init-functions
scriptname=/etc/init.d/pciback
configfile=/etc/xen/pciback.conf

unbind() {
# Unbind a PCI function from its driver as necessary
[ ! -e /sys/bus/pci/devices/$1/driver/unbind ] || \
echo -n $1 > /sys/bus/pci/devices/$1/driver/unbind
}

bind() {
# Add a new slot to the PCI Backend’s list
echo -n $1 > /sys/bus/pci/drivers/pciback/new_slot
# Now that the backend is watching for the slot, bind to it
echo -n $1 > /sys/bus/pci/drivers/pciback/bind
}

case $1 in

start)
cat $configfile | while read line;do
echo $line | grep ^# >/dev/null 2>&1 && continue
unbind $line
bind $line
done
;;

stop|status|restart|force-reload)
# As we don’t know which driver was bound before, there is not much we can do here
;;

*)
echo “Usage: $scriptname start” >&2
exit 3
;;

esac


Make the pciback shell script executable
Code: Select all
chmod +x /etc/init.d/pciback

and run
Code: Select all
update-rc.d pciback defaults 21 19

The number 21 means that it will run AFTER xend is started (xend has 20 in the init.d), and stopped before xend will stop.

Note: For some reason, the command insserv didn't work for me. You may try it instead of the above command. insserv doesn't need arguments since it should read the header and determine when to run the script.


11. Load xen-pciback and pass through graphics card

We use the initramfs method to pass through the graphics card:
Code: Select all
echo "xen-pciback passthrough=1 hide=(02:00.0)(02:00.1)" >> /etc/initramfs-tools/modules

Note: You have to change the PCI IDs to the ones you determined above!
The above command loads the xen-pciback module from initramfs and binds the graphics card to xen-pciback, before the graphics driver gets loaded.

Then we update the initramfs by issuing:
Code: Select all
update-initramfs -k all -c



12. Create the Windows 7 guest configuration file

Copy/paste the following configuration file into /etc/xen and name it win7.cfg (or whatever name you choose - see also note regarding Linux Mint 15 below):
Code: Select all
kernel = '/usr/lib/xen-default/boot/hvmloader'
builder='hvm'
# Memory reserved for Windows domU, in this case 4GB (adjust to your needs):
memory = 4096
# Name of the domU:
name = 'win7'
# Most CPUs have 4 cores / 8 threads (=8 vcpus). Check your CPU and change as needed!:
vcpus=6
# Only for 32 bit guests - don't use for 64 bit:
#pae=1
acpi=1
apic=1
on_xend_stop='shutdown'
# Here my virtual network interfaces – see /etc/network/interfaces below:
vif = [ 'mac=00:16:3e:68:e1:01,bridge=xenbr0' ]
# I assigned a static MAC address, else it will be changed each time Windows
# boots. The address should start with 00:16:3e., the rest is up to you.
#
# Specifying the disks and Windows ISO for installation, adjust to your needs:
disk = [ 'phy:/dev/mapper/guest-win7,hda,w' , 'file:/media/install/Win7.iso,hdc:cdrom,r' ]
# More disks can be added later using this same method. The path to the
# device or a file can be added. After the first comma is how the device will
# appear. “hda” is the first, “hdb” the second etc. “hda” will appear as IDE,
# sda will appear as SCSI or SATA. After the second comma “r” means read
# only and “w” is for write.
device_model = '/usr/lib/xen-default/bin/qemu-dm'
boot='dc'
# The above should be changed once Windows is installed: boot='c' to only
# show the Windows file system, else it may try to boot from the ISO image.
sdl=0
vnc=1
vncpasswd=''
stdvga=0
# nographic=1 #!!! only uncomment this if you are using win8 or are trying
# to get a Nvidia card to work. In my case – Nvidia Quadro 2000 - this was not
# needed.
serial='pty'
tsc_mode=0
viridian=1
#soundhw='all'
# This allows sharing the USB mouse/keyboard:
usb=1
usbdevice='tablet'
# gfx_passthru=0 is how it works for most AMD cards and my Quadro 2000 card.
# gfx_passthru=1 would pass through the graphics card as primary display adapter.
# You can change this later for iGPUs or nVidia if needed.
# Try it with 0 first!
gfx_passthru=0
#Enter the PCI IDs you wish to passthru:
pci=[ '02:00.0', '02:00.1' , '00:1a.0' ]
# The following lets Windows take the local time from the dom0:
localtime=1
#To turn on pci power management globally, use (see remarks under pci=... below):
pci_power_mgmt=1


Important: You must edit the file and adjust it to your requirements !!! It most likely won't work for your hardware/setup.

I have commented the configuration file to explain the different configuration options. Here some further explanations about the options you need to change/adopt for your hardware:

memory = # Enter the amount of memory (RAM) in Mbyte your guest will get. For example, if you have 8GB RAM, you could give the guest (Windows) a value of 4096 which is 4GB, leaving the rest for dom0 (Linux) and other Xen guests.

vcpus= # Enter the number of virtual CPUs your guest will get. I have a 6-core CPU with 12 threads and I decided to give Windows 10 of the 12 available threads (virtual CPUs). As long as I run only Linux, it will get all CPUs / VCPUs. Once I start Windows I give it 10 VCPUs, leaving 2 VCPUs for Linux. It's best to leave some CPU resources to the Linux dom0 since Linux still needs to handle I/O, networking, etc. In my case, having a 3930K CPU, 2 VCPUs for dom0 should be enough to handle that.

vif = [ 'vifname=win7,type=ioemu,mac=00:16:3e:68:e1:01,bridge=xenbr0' ] # This defines the networking part for the guest system. vifname=win7 gives a name to the interface (this feature is buggy, so I removed it). iommu means it uses the VT-d functions of the memory controller. mac=... assigns a static MAC address to the bridge so Windows won't complain about "illegal copy" etc. when the MAC changes for each boot. 00:16:3e:68: is reserved for Xen and you should use it too! Change the last two e1:01 to your liking. Just watch out that you don't have the same exact MAC addresses in your LAN or when running two guests simultaneously.

disk = [ 'phy:/dev/mapper/guest-win7,hda,w' , 'file:/media/install/Win7.iso,hdc:cdrom,r' ] # This is the disk configuration for the Windows guest, i.e. as Windows will see it. See below for creating the Windows partition on LVM. The 'file:/media...' option points to the Windows ISO image on your drive, reading it as CDROM.

boot='dc' # is the boot order for Windows. This entry will first boot from d: then c:. Once Windows is installed you need to edit this file and remove the "d" so it reads: boot='c'.

pci=[ '02:00.0', '02:00.1' , '00:1a.0' ] # These are the PCI IDs of the PCI devices we want to pass through (the ones we specified in the /etc/xen/pciback.conf file or in the /etc/initramfs-tools/modules file). If you have problems with the USB ports and power management, and the global "pci_power_mgmt=1" option doesn't work, try this:
pci=[ '02:00.0', '02:00.1' , '00:1a.0,power_mgmt=1' ]
See http://www.overclock.net/t/1205216/guide-create-a-gaming-virtual-machine/550#post_19572876.

The rest should be left as is for now. If, and only if your Windows doesn't boot or you don't get a screen or mouse/keyboard don't work, you may need to change the following, depending on the problem:

# nographic=1 # Uncomment if the Nvidia card doesn't work, or perhaps for Windows 8.

gfx_passthru=0 # Set to 1 may help with some Nvidia cards. Search the Internet for AMD cards and recommendations, though AMD cards should work with 0. Setting gfx_passthru=1 will pass through the graphics card as primary (and only) display adapter, without first booting the virtual Cirrus Logic adapter. In most cases this requires a patched Xen hypervisor, but in some cases it may work, depending on the graphics adapter and perhaps the Windows version in use.

usb=1 # Set to "0" if you pass through your keyboard/mouse (this may also be necessary after installing "Synergy")

usbdevice='tablet' # When usb=1 (see above), this option improves mouse tracking

Documentation on the above guest configuration options can be found here: http://xen.org/files/Support/XenConfigurationDetails.pdf


13. Create a logical volume to install Windows

To make this easier, I assume you are installing on a separate disk named /dev/sdb. Of course you can also use a partition.

Note: The disk (here sdb) MUST be flagged for LVM. You can use gparted for it. Don't format it!
Code: Select all
pvcreate /dev/sdb

Let's check:
Code: Select all
pvdisplay

This shows you the physical LVM volumes (PV). Your newly created PV should show up as empty.

Now we create a logical volume (LV) named "win7", size 100GB for Windows in volume group (VG) "guest":
Code: Select all
vgcreate guest /dev/sdb # assign the PV we created to volume group "guest"
lvcreate -L 100G -n win7 guest # create the logical volume named "win7", size 100G in VG "guest"


Let's check the results:
Code: Select all
lvdisplay

See if logical volume "win7" appears.


14. Copy the Windows installation DVD to disk

Make an installation folder for Windows:
Code: Select all
mkdir /media/install


Insert your Windows DVD into your DVD drive and enter:
Code: Select all
dd if=/dev/cdrom of=/media/install/Win7.iso

Note: You will need a Windows retail license to use Windows inside a virtual machine.


15. Reboot the computer into Xen

Important note: After reboot with the PCI devices being detached I had problems with the nouveau driver (Nvidia) for my secondary graphics card (the one to passthrough). My PC got stuck right after login. Blacklisting the nouveau driver worked. See http://community.linuxmint.com/tutorial/view/176 for how to blacklist the nouveau driver.
Comment: In my case the nouveau driver wasn't required, as my Nvidia card is only used by Windows.
EDIT: Similar problems are reported with AMD graphics cards, in which case the solution is to blacklist the radeon driver using the radeon.blacklist=1 grub entry like this:
Code: Select all
GRUB_CMDLINE_LINUX_DEFAULT="radeon.blacklist=1 quiet splash nomodeset"

However, if you choose to use 2 AMD or 2 Nvidia graphics cards, you would need the respective driver for your dom0. In that case the PCI devices should be detached before the driver starts. You may need to run update-rc.d pciback defaults 21 19 with different numbers, or find another way to run this script before the graphics drivers are loaded.

See also earlboy's post here: http://forums.linuxmint.com/viewtopic.php?f=42&t=112013&start=60#p705871.


16. Install vncviewer

We need a VNC viewer for the installation of Windows. In the terminal window:
Code: Select all
sudo apt-get install vncviewer



17. Verify that pciback works

First check that everything is setup correctly:
Code: Select all
sudo xm pci-list-assignable-devices

should give the list of detached devices that can be assigned (bound) to the Windows guest. My list looks like that:

    0000:02:00.0
    0000:02:00.1
    0000:00:1a.0

All devices I configured are ready for passthrough.

Note: If some PCI IDs are missing - namely the PCI IDs for the graphics card - check steps 9 to 11 to make sure you didn't miss anything (also verify the IDs). If all fails, try the method in this post: http://forums.linuxmint.com/viewtopic.php?f=42&t=112013&start=120#p711782. The method described in this post integrates the xen-pciback module into the initramfs so it can be loaded via a grub boot option, before the graphics driver grabs the card.

Code: Select all
sudo xm list

lists the current domains. At the moment only domain0 should be listed.

Note: For debugging purposes this command can come handy:
Code: Select all
sudo xm dmesg



18. Create the Windows domU using its configuration file

Code: Select all
sudo xm create /etc/xen/win7.cfg


and connect to it via VNC:
Code: Select all
sudo vncviewer localhost:0

Note: It may work without sudo.


19. Install Windows guest

Unless there is some configuration error (check your path to the /media/install/Win7.iso file or whatever you name it), the Windows installer should boot now and you should see the screen output in the console (enlarge it if necessary).

Install Windows as you normally would. At some point Windows will reboot the guest domU (not Linux).


20. Access Windows after automatic reboot

After Windows has rebooted, it should come back to the VNC console. If not, close VNCviewer and run again.

If Windows doesn't start and you get a qemu shell screen, it might have passed through your mouse/keyboard and/or your display adapter. Alternatively, you might need to change your /etc/xen/win7.cfg file and specify boot="c". Before you edit the win7.cfg file, shutdown the domU (via xm shutdown win7). Edit your win7.cfg file and run xm delete win7, followed by

a) sudo xm create /etc/xen/win7.cfg, and connect via VNC

Note: If, after your Windows guest reboots, you get a message like "parallel0 console" on your VNCviewer screen, see here: http://forums.linuxmint.com/viewtopic.php?f=42&t=112013&p=786927#p786784.


21. Install Windows graphics driver

After the second Windows 7 VM reboot, Windows will tell that "Setup is preparing your computer for first use...". Once booted, still in the VNC console screen, you'll need to install and enable the graphics adapter driver for your display card. Inside Windows under Device Manager, check for graphics adapters. It should list two graphics adapters, one is for the emulated Cirrus VGA adapter that you are currently using, the other one is the real graphics adapter (in my case it had a yellow triangle as the driver was missing).

Before going any further, check the following hardware requirements:

a) A separate screen connected to the secondary graphics adapter to be used by Windows; or a second cable connection to your current screen; or a KVM switch for keyboard, mouse, and optionally display. I use my KVM switch only for keyboard/mouse and connected my secondary graphics adapter via DVI cable to the second DVI port on my screen. Some screens will have VGA ports, or HDMI. Both should work, though I don't care much about VGA because of it's low quality compared to DVI or HDMI. The screen should have a button to select the input source, if using only one screen.
b) If you want to pass through your keyboard/mouse, the best choice is a USB KVM switch. Else you need to unplug and replug your keyboard/mouse to the appropriate USB ports.
c) You could also use your keyboard and mouse without passing the PCI device through (using the USB=1 and/or tablet=1 settings in the win7.cfg file). Frankly I have no experience with this.
Or you could use a second mouse/keyboard just for Windows when using PCI passthrough on the USB port.

Now install the driver for your graphics adapter. In my case, I went to the Nvidia support website and ran the Java script to determine the correct driver, downloaded and installed it. The more common option would be to let Windows find the suitable display driver.

Once the installation has finished:

* the screen should reset and switch to the graphics adapter. If at that point you get a black screen, switch your screen to the other display port. If you see the Windows screen - open a bottle of Champagne - you've done it.
* alternatively Windows may ask you to reboot. Do so and watch the boot process in the VNC screen. After loading the graphics driver, and before the login screen, Windows should switch the output to your installed graphics adapter and you should be greeted with a Windows screen on your display.

If you get a black screen when rebooting Windows, or at any point during installation, switch your display to the second input. Do you see a Windows screen - if yes, it's a reason to celebrate.


22. What if VGA passthrough doesn't work?

If you can't get your VGA adapter to pass through to Windows, there are some options you may want to check:

a) First make absolutely sure your graphics adapter is supported !!! See my lengthy introduction to hardware requirements. Unless you have a Nvidia "multi-OS" graphics adapter, AMD will be a better choice. Have a look at the following threads/blogs for more information on VGA passthrough and adapters:

http://www.overclock.net/t/1307834/xen-vga-passthrough-compatible-graphics-adapters

http://www.overclock.net/t/1205216/guide-create-a-gaming-virtual-machine - New updated instructions for Fedora 20

https://dl.dropboxusercontent.com/u/2259891/xen-hvm-vga-passthrough.pdf

http://gro.solexiv.de/2012/08/pci-passthrough-howto/

http://wiki.xen.org/wiki/Xen_VGA_Passthrough

http://wiki.xen.org/wiki/Xen_VGA_Passthrough_Tested_Adapters
This page seems a bit outdated, but probably the safest reference when purchasing a graphics card.

http://www.phoronix.com/scan.php?page=news_item&px=MTA3NjA - this is the complicated way to get Xen VGA passthrough with Nvidia cards (see link inside the text). This procedure should work with some Nvidia cards that are NOT "multi-OS".

http://www.davidgis.fr/blog/index.php?2011/12/07/860-xen-42unstable-patches-for-vga-pass-through - this is currently THE resource for instructions and patches when you want or need to compile the Xen hypervisor. This is specifically for Nvidia cards that don't have native support for VGA passthrough.


23. Change Windows boot order in guest config file

After Windows is installed, edit the /etc/xen/win7.cfg file and change the "boot=dc" option to boot="c". This is equivalent to ejecting the installation disk from the DVD drive. If the installation ISO is still required, but you want/need to change the boot order, specify boot="cd".


24. Booting the Windows guest

After everything is installed and VGA / PCI passthrough is working properly, you can use the following command to start your Windows domU:
Code: Select all
sudo xm create /etc/xen/win7.cfg



25. Install GPLPV drivers under Windows

Inside Windows, download and install the latest GPLPV drivers from here: http://www.meadowcourt.org/downloads/*.

As of this writing, the latest version is 0.11.0.372.

For my Windows 7 64-bit version I downloaded the gplpv_Vista2008x64_0.11.0.372.msi driver. Double-click on it to install under Windows. If the installer requests testsigning to be enabled, quit the installer, open a Windows admin shell and enter
Code: Select all
bcdedit.exe /set TESTSIGNING ON

Then reboot Windows and install the GPLPV drivers.

Important:When using the GPLPV network driver, you need to modify the win7.cfg and remove the "type=ioemu" option from the vif=... line.

* Note: The Univention website (http://wiki.univention.de/index.php?title=Installing-signed-GPLPV-drivers) that I had listed here before does NOT have these latest drivers!


26. Post Passmark results

Once your Windows VM with VGA passthrough is up and running, please post your Passmark results here: http://forums.linuxmint.com/viewtopic.php?f=225&t=153482.

Feel free to add other benchmark or performance results (WEI, etc.), particularly benchmarks using the same hardware with Windows installed on "bare metal". Or simply post your opinion about running Windows and Windows applications in a virtual machine.

Thanks !!!

END

Note: Once you successfully installed Windows on a Xen hypervisor, you may like the backup and restore scripts I've written. See http://forums.linuxmint.com/viewtopic.php?f=42&t=133089. Enjoy!
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Sapphire 7770 GPU, PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB SSD + various HD, Corsair 500R case, SeaSonic 660W Gold X PS
powerhouse
Level 5
Level 5
 
Posts: 576
Joined: Thu May 03, 2012 3:54 am

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby flobo71 on Wed Feb 12, 2014 11:05 am

Hi all,

it works with LM16 on my Asus Z87I-Pro (BIOS 0902) with i4570 and a radeon 7750 as second graphics card (iGPU is first).
Thanks for the great guide!

I blacklisted radeon in /etc/modprobe.d/blacklist.conf. The card would get initialized otherwise during boot.

As Windows 7 doesn't support the USB chipsets out of the box you will need to install USB drivers first before you
can attach mouse and keyboard to the VM. I used the ASMedia USB ports.

Performance seems slightly better with qemu-xen-traditional instead of qemu-dm. Whats the difference?

Also, the device manager shows missing driver for xen pci device #0. Any ideas how to fix this?
flobo71
Level 1
Level 1
 
Posts: 3
Joined: Wed Feb 12, 2014 10:19 am

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby sivel27 on Thu Feb 13, 2014 10:31 pm

heynow !!

i have a very silly question. it would seem as though im in a bit of a pickle.
i followed the old [lm13] guide, and everything was working great. all of the sudden,
when i issue the command to create the vm,it says its been created like normal, xm top indicates
the vm is running, and my kb and mouse lights up like always. the problem is i have no video output
on the card i passed through.

my main question is --- is it possible to look "into" the partition i installed windows to ? for the purpose of
backing up files from it ? as in using file manager to view the actual files on the lvm partition windows is installed too ?

thank in advance,
sivel
sivel27
Level 1
Level 1
 
Posts: 10
Joined: Sat May 23, 2009 12:54 pm

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby FastRealm on Fri Feb 14, 2014 1:32 am

Hi,

I want to ask is this do able or not.
Currently my LM Xen and Windows is connected to the internet and server using LAN.
Is it possible to connect LM and Win to server using LAN (Fix IP with NO internet connection) and use wifi dongle to access to internet?

And how is it possible to set virtual IP for LM Xen?

Thank you.
FastRealm
Level 1
Level 1
 
Posts: 18
Joined: Fri May 24, 2013 2:22 am

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby powerhouse on Fri Feb 14, 2014 6:58 am

sivel27 wrote:heynow !!

i have a very silly question. it would seem as though im in a bit of a pickle.
i followed the old [lm13] guide, and everything was working great. all of the sudden,
when i issue the command to create the vm,it says its been created like normal, xm top indicates
the vm is running, and my kb and mouse lights up like always. the problem is i have no video output
on the card i passed through.

my main question is --- is it possible to look "into" the partition i installed windows to ? for the purpose of
backing up files from it ? as in using file manager to view the actual files on the lvm partition windows is installed too ?

thank in advance,
sivel


1. It could be a very simple issue with the video cable - make sure it's connected properly and that your screen switches to the other input.

2. Occasionally it happens that - after a Windows update - Windows won't boot up into the login screen. It stays at a state where it uses the virtual VGA adapter. In this case issue the following:

Code: Select all
sudo xm vncviewer <domain>

specify the Windows 7 domain name (use "sudo xm list").

Windows may be waiting for user input.
Sometimes the problem is solved by shutting down the Windows guest and booting it again.

3. Now to your question: Yes, you can mount your Windows virtual partition! See here: http://forums.linuxmint.com/viewtopic.php?f=42&t=111783.
I suggest you mount it "read only".

P.S.: You can easily backup and restore your Windows VM. I believe I posted my backup and restore scripts here in this thread. If not, let me know and I post them. If you use them, make sure to change the source and targets to match your configuration/hardware.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Sapphire 7770 GPU, PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB SSD + various HD, Corsair 500R case, SeaSonic 660W Gold X PS
powerhouse
Level 5
Level 5
 
Posts: 576
Joined: Thu May 03, 2012 3:54 am

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby powerhouse on Fri Feb 14, 2014 7:21 am

FastRealm wrote:Hi,

I want to ask is this do able or not.
Currently my LM Xen and Windows is connected to the internet and server using LAN.
Is it possible to connect LM and Win to server using LAN (Fix IP with NO internet connection) and use wifi dongle to access to internet?

And how is it possible to set virtual IP for LM Xen?

Thank you.


I'm trying to understand what you want to accomplish. Here is what I understand:

1. Both dom0 and Windows in bridged mode, using static IP.
2. Network interfaces are eth0 and wlan0 (or whatever the Wifi dongle is listed under).

I haven't tried this yet but it looks like doable. You would have to use the Wifi dongle as your default network interface in your configuration. Try the following for a start (make sure to backup your old /etc/network/interfaces file):
Code: Select all
auto lo
iface lo inet loopback

auto xenbr0

iface xenbr0 inet static
address 192.168.0.120  # IP of Linux Mint dom0
broadcast 192.168.0.255
netmask 255.255.255.0
gateway 192.168.0.1  # IP of router or modem
dns-nameservers 8.8.8.8  # IP(s) of DNS name server(s)
bridge_ports eth0 wlan0 # specify the network interface of your Wifi dongle - use ifconfig to get the list of interfaces
bridge_stp off    # disable Spanning Tree Protocol - optional
# bridge_waitport 0   # no delay before a port becomes available - optional
# bridge_fd 0    # no forwarding delay - optional

# There are no IP configurations for eth0, it’s all defined under xenbr0:
# auto eth0


Use
Code: Select all
ifconfig
to determine the Wifi network interface.

Important: I haven't tried this configuration and do not have the means to test it now. In any case, the example above defines a layer 2 network (LAN) configuration without routing. That means that eth0 and wlan0 are bridged and traffic between these interfaces is forwarded transparently on layer 2. It means also that your server connected to eth0 (probably) will have direct LAN access to port wlan0 (or whatever your Wifi dongle will be named).

If things don't work, post the output of ifconfig here, as well as your interface configuration.

P.S.: All the above assumes that network manager is NOT installed. If it is, it might get more complicated.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Sapphire 7770 GPU, PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB SSD + various HD, Corsair 500R case, SeaSonic 660W Gold X PS
powerhouse
Level 5
Level 5
 
Posts: 576
Joined: Thu May 03, 2012 3:54 am

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby powerhouse on Fri Feb 14, 2014 8:16 am

flobo71 wrote:Hi all,

it works with LM16 on my Asus Z87I-Pro (BIOS 0902) with i4570 and a radeon 7750 as second graphics card (iGPU is first).
Thanks for the great guide!

I blacklisted radeon in /etc/modprobe.d/blacklist.conf. The card would get initialized otherwise during boot.

As Windows 7 doesn't support the USB chipsets out of the box you will need to install USB drivers first before you
can attach mouse and keyboard to the VM. I used the ASMedia USB ports.

Performance seems slightly better with qemu-xen-traditional instead of qemu-dm. Whats the difference?

Also, the device manager shows missing driver for xen pci device #0. Any ideas how to fix this?

Thanks flobo71 !

I have updated my how-to and added your recommendation under step 17 (checking pci-passthrough).

I'm very surprised that Windows 7 doesn't support your USB chipsets out-of-the-box. I never had an issue with my USB controllers, and I use both an Intel X79 and an ASMedia USB controller with Windows 7. I also have a Renesas USB controller, and almost all my USB ports are in use by either dom0 or Windows.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Sapphire 7770 GPU, PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB SSD + various HD, Corsair 500R case, SeaSonic 660W Gold X PS
powerhouse
Level 5
Level 5
 
Posts: 576
Joined: Thu May 03, 2012 3:54 am

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby FastRealm on Mon Feb 17, 2014 3:58 am

@Powerhouse

Have you tried the XL windows reboot script from ktz.me??
FastRealm
Level 1
Level 1
 
Posts: 18
Joined: Fri May 24, 2013 2:22 am

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby powerhouse on Wed Feb 19, 2014 2:30 pm

FastRealm wrote:@Powerhouse

Have you tried the XL windows reboot script from ktz.me??


No. Right now I fail to see the advantage of xl. Don't get me wrong - there may be advantages to xl versus xm.

If you know of any advantages to xl, please share.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Sapphire 7770 GPU, PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB SSD + various HD, Corsair 500R case, SeaSonic 660W Gold X PS
powerhouse
Level 5
Level 5
 
Posts: 576
Joined: Thu May 03, 2012 3:54 am

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby Achelous on Sun Feb 23, 2014 9:44 pm

Powerhouse,
First let me say that this tutorial is awesome. However, I am having trouble getting the answers to some questions I have even after scouring the internet.
  • Is it safe to use dom0 for general computing? If not, I should just run another copy of Mint in a VM?
  • I couldn't quite tell what mode we are running Win7 under. I know it's HVM, but is it FV, FV w/ PV, PVHVM, or PVH (which is supposed to be the best mode now)?
  • What is the best method of making VMs automatically run on startup?
Achelous
Level 1
Level 1
 
Posts: 1
Joined: Thu Feb 20, 2014 12:42 pm

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby wollymammoth on Mon Feb 24, 2014 6:39 am

Hey powerhouse, another happy customer :)

My setup:
mobo: Asrock z77 extreme4-m
RAM: 16GB (currently 2G for Dom0, rest DomU)
CPU: i7-3770
GPU: hd7950 for domU, integrated for dom0
OS: Mint16 x64 (mate), win7 x64
storage: 128GB SSD for both OSes, 3TB for data (samba share)


I did run into one hiccup trying to blacklist the 'radeon' driver from picking up my primary GPU, when setting:
Code: Select all
GRUB_CMDLINE_LINUX_DEFAULT="..."

- the 'nomodeset' option (by itself or in conjunction with 'radeon.blacklist=1') causes X to crash on boot dropping me to a non-graphical login ('nomodeset' was not there by default).
- 'radeon.blacklist=1' without 'nomodeset' seems to do nothing(?) and dom0 claims the hd7950
- adding 'blacklist radeon' to /etc/modprobe.d/blacklist.conf did nothing either...

I only achieved success following the first answer in https://ask.fedoraproject.org/en/question/28983/blacklist-disable-uninstall-or-destroy-radeon-module/
resulting in the mutant command line:

Code: Select all
GRUB_CMDLINE_LINUX_DEFAULT="radeon.blacklist=1 radeon.modeset=0 rd.driver.blacklist=radeon rd.blacklist=radeon quiet splash"


most of the options are likely not needed, but it works... so i'm not touching it :)


another (obvious in retrospect) point was to make sure to set the integrated/non-passed graphics adaptor as the default output device in the BIOS... :oops:

---------------------------

Also, I managed to get sound working without extra hardware using the jack audio system.
There is a small but barely noticable lag in the sound (completely un-optimised/fiddled-with settings), and quality is very good.

I have uploaded a how-to on what I did but its "waiting for a moderator" before its viewable...
wollymammoth
Level 1
Level 1
 
Posts: 4
Joined: Sat Feb 22, 2014 3:24 am

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby powerhouse on Mon Feb 24, 2014 8:21 am

Achelous wrote:Powerhouse,
First let me say that this tutorial is awesome. However, I am having trouble getting the answers to some questions I have even after scouring the internet.
  • Is it safe to use dom0 for general computing? If not, I should just run another copy of Mint in a VM?
  • I couldn't quite tell what mode we are running Win7 under. I know it's HVM, but is it FV, FV w/ PV, PVHVM, or PVH (which is supposed to be the best mode now)?
  • What is the best method of making VMs automatically run on startup?


Question 1: Depends on how you use your computer. I'm using dom0 for general computing for nearly one a half years now without any issues. My dom0 is like a regular Linux Mint desktop with all the Linux applications I need, including:
- LibreOffice
- Firefox, Thunderbird, etc.
- Handbrake - for DVD/Blu-Ray rip
- gmusicbrowser for bit-perfect audio playback
- Simple Scan
- various games
and tons of software and utilities.

Only when I need some Windows application will a boot the Windows VM. If I was using Windows on a regular/daily basis I would probably boot domU during system boot.

I was considering running two VMs, one for the Linux desktop, the other Windows. But that would just complicate matters. For example, it's much easier to pass through a graphics card to Windows than it is to a Linux HVM guest. I could go with a PV guest for Linux, but the options are limited. In the end, I chose the simple solution. If your computer is for desktop use, and you are the only user, I see no reason to complicate matters.

If your PC is used both as a desktop and as a local server, you can run the servers in dedicated VMs (headless, via ssh for example). There are plenty of dedicated server distros, for example NAS, mail servers, web servers, whatever you fancy. For some fancy home server ideas, see here http://thehomeserverblog.com/esxi-lab-specs/, which is based on the commercial ESXi, but you can do the same with Xen.

Only if you need to give access to your PC to people you don't trust, may it be worth the effort to run a headless dom0 with Linux and Windows VMs per need. Linux Mint is not a good choice for a headless installation, there are much better (lightweight) choices for that. Here is an example which I have been considering:
- PC with iGPU and two discrete graphics cards (together 3 GPUs) running a headless Linux server - see http://wiki.xen.org/wiki/Dom0_Kernels_for_Xen
- VMs:
* 2 remote seats (using HDMI extenders etc.) including screen, mouse and keyboard for my kids, each running either Linux or Windows (whatever they prefer)
* XBMC VM with HDMI wire to home theater / TV using the iGPU of the PC, remotely controlled via smart-phone and/or iPad
* NAS VM
* Whatever other server VMs I might need, for example webserver, ftp, etc.
* Linux PBX, for example http://www.asterisk.org/downloads/asterisknow
* In case I want to remotely connect - via Internet - to server(s) running on the PC, I'd install Shorewall.

Question 2: You got me :? . Windows runs as HVM, but I also installed the GPLPV drivers for disk i/o and network, which means that disk and network access are PV. The graphics card is passed through, so it's like bare metal to Windows (Windows uses it's own native drivers, or to be specific, the Windows drivers of the GPU vendor). All my USB ports used under Windows are passed through too.
To get some idea about performance, have a look here: http://forums.linuxmint.com/viewtopic.php?f=225&t=153482 (If you are running Xen with VGA passthrough and Windows guest, please post your own Passmark results in that forum !)

I'm extremely pleased with the performance of both my Linux Mint desktop and my Windows VM. I'm particular impressed by the Xen CPU scheduler, which does a terrific job. For example, my Windows VM gets 10 of 12 VCPUs since I run CPU intensive photo editing tasks in Windows. However, the scheduler actually only gives the domU what it asks for, the unused CPU resources stay with dom0. To benchmark both dom0 and domU (Windows) at the same time in a real-life scenario, I ripped a DVD with Handbrake under dom0 and simultaneously converted several hundred edited RAW photos to web-sized jpegs using Lightroom in the Windows VM. For whatever reason Lightroom cannot fully utilize 10 VCPUs, although it's capable of multi-threading. Here is my original post with the results: http://forums.linuxmint.com/viewtopic.php?f=42&t=112013#p631391.

In the meantime network throughput has also been solved. And I can't complain about disk I/O. I'm using the setup as described in this how-to, adapted to my hardware and my needs. As for disk I/O, I could as well pass through a SATA controller to Windows and let it handle disk I/O directly. But for now I think it's futile (see my benchmarks following the first link above). Having everything on LVM, including the Windows virtual partitions, makes it so much easier to backup/restore the data, or to grow the disk space when needed (which, in my case, happens once or twice a year). It also allows me to access my Windows partition from Linux dom0 (mounted as read-only, for safety). Of course, the drives can also be shared via Samba, which performs nicely in my setup.

In the end, I don't really care how it works (FV, FV w/ PV, PVHVM, or PVH to quote you), as long as it works well.

Question 3: I haven't given it a shot yet, but once you have a startup script (I think I posted one somewhere in this thread, near the beginning), you could either use Menu -> Control Center -> Startup Applications and add the script, or call it from here:
Code: Select all
/home/<user>/.config/autostart
For more on the second option, see http://xmodulo.com/2013/01/how-to-automatically-run-script-when-logging-into-ubuntu-desktop.html.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Sapphire 7770 GPU, PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB SSD + various HD, Corsair 500R case, SeaSonic 660W Gold X PS
powerhouse
Level 5
Level 5
 
Posts: 576
Joined: Thu May 03, 2012 3:54 am

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby powerhouse on Mon Feb 24, 2014 8:26 am

wollymammoth wrote:Hey powerhouse, another happy customer :)

My setup:
mobo: Asrock z77 extreme4-m
RAM: 16GB (currently 2G for Dom0, rest DomU)
CPU: i7-3770
GPU: hd7950 for domU, integrated for dom0
OS: Mint16 x64 (mate), win7 x64
storage: 128GB SSD for both OSes, 3TB for data (samba share)


I did run into one hiccup trying to blacklist the 'radeon' driver from picking up my primary GPU, when setting:
Code: Select all
GRUB_CMDLINE_LINUX_DEFAULT="..."

- the 'nomodeset' option (by itself or in conjunction with 'radeon.blacklist=1') causes X to crash on boot dropping me to a non-graphical login ('nomodeset' was not there by default).
- 'radeon.blacklist=1' without 'nomodeset' seems to do nothing(?) and dom0 claims the hd7950
- adding 'blacklist radeon' to /etc/modprobe.d/blacklist.conf did nothing either...

I only achieved success following the first answer in https://ask.fedoraproject.org/en/question/28983/blacklist-disable-uninstall-or-destroy-radeon-module/
resulting in the mutant command line:

Code: Select all
GRUB_CMDLINE_LINUX_DEFAULT="radeon.blacklist=1 radeon.modeset=0 rd.driver.blacklist=radeon rd.blacklist=radeon quiet splash"


most of the options are likely not needed, but it works... so i'm not touching it :)
...


Thanks for the feedback - this is very helpful !!! I need to look into it and will try to incorporate it in the how-to.

On another note, please post your Passmark results here: http://forums.linuxmint.com/viewtopic.php?f=225&t=153482
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Sapphire 7770 GPU, PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB SSD + various HD, Corsair 500R case, SeaSonic 660W Gold X PS
powerhouse
Level 5
Level 5
 
Posts: 576
Joined: Thu May 03, 2012 3:54 am

Linux Mint is funded by ads and donations.
 
PreviousNext

Return to Tutorials / Howtos

Who is online

Users browsing this forum: No registered users and 14 guests