HOW-TO make dual-boot obsolete using XEN VGA passthrough

Questions about virtualization software
Forum rules
Before you post read how to get help. Topics in this forum are automatically closed 6 months after creation.
Locuust

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by Locuust »

powerhouse wrote:1. Single card VGA passthrough: IIRC, in order to make do with one GPU, you would need to compile pciback into the kernel and add the pciback option to the grub file. The idea is to bind the graphics card to pciback before the graphics driver (dom0) gets loaded. But don't take my word on that, it's been a while since I played with it. See also http://www.overclock.net/t/1205216/guid ... al-machine for a great tutorial based on Fedora 16. It's a very long thread with tons of useful information.

In order to release a passed through PCI device, use:

Code: Select all

virsh nodedev-reattach pci_0000_0a_00_0
This reattaches the PCI device to dom0. It works well with USB controllers, but I haven't tested my graphics card yet. I have some doubts about reassigning graphics cards to different domUs, or back to dom0.
While I was considering compiling my own kernel once I had verified this works with my hardware (still waiting on shipments) I'd just assume not rock the boat too much with VGA passthrough. On the KVM side of the pond you have lockups and performance degradation being reported. I'll be happy enough to get it to work and I've already ordered my second video card. It's an interesting concept though. I like messing around with new stuff but generally with the eye on making it stable enough for everyday use.
2. AMD CPUs: You wrote a lot about AMD CPUs. I'm afraid I haven't got a clue about AMD CPUs, the last time I used one was some 7-8 years ago. A few years ago Intel integrated the memory controller into the CPU - I believe AMD uses another chip to host the memory controller, so it would also depend on the motherboard.
For both AMD and Intel, all parts must support IOMMU: CPU, motherboard (chipset), and BIOS, though in some cases the CPU doesn't play part in IOMMU (Intel Core 2 Duo, for example). What I mean is that you need to make sure that the combination of all three supports IOMMU.
lol That's funny I was just thinking to myself that I haven't messed around with desktop Intel processors since the days of the venerable 440BX chipset and Pentium II/III (released in 98). I'm probably one of the few people that bothered building a dual processor celeron machine that served faithfully for over a decade 24/7 (somebody really wanted a dual processor server but didn't want to pay for it lol) I did have some Xeon servers that I built and maintained a few years back so I'm not totally out of the loop. But on the desktop I haven't built a intel machine for about 15 years. I wouldn't call myself an AMD fanboi even if the first PC I built myself was a 386DX-40Mhz. Maybe I just have a soft spot for the underdog, but I can't deny that Intel has the performance lead more often than not.

As far as the memory controller goes it was actually AMD that first incorporated it with the Althon 64 in the early 2000's. For a short while that was one of the technical disadvantages that Intel had. The front side bus had to deal with memory requests as well as the I/O. To compound that problem the bandwidth available could hardly deal with the increasing data transfer rates of memory. Hypertransport also allowed for point to point links between processors which previously was handled by the FSB. The Althon 64 really was a design win for AMD that they didn't/weren't able to capitalize in the long run. Intel has certainly caught up and surpassed AMD since then.

I'll probably spend some time refreshing my memory on Intel tech today as I might upgrade my home server with it rather than continue to give it my AMD hand-me-downs.

Oh, if you referring to my speculation of using those APU's for VGA passthrough it was just a thought experiment. I do know you have to have support across the cpu/bios/MB and on paper they do. At least as far as the MB manual's bios settings indicate. IOMMU is a feature that is hard to get definitive info about from manufacturers. Until someone actually tries it you can't say for sure. In my searching for someone who has I noticed you've fielded a question about APU's over on phronix (man you have threads everywhere :P ) but it doesn't look like the guy has reported back on any success or failure. Though what he was describing was a bit different than what I had in mind. He wanted to passthrough his only GPU (from the CPU) to his DomU. I was talking about using the integrated GPU as a cheap option for the Dom0 in a dual card setup. The integrated GPU in AMD's processors are considerably better than their Intel counterparts so I can see it's application in a HTPC build. I think the problem with his idea is that is rather inconvenient and isn't computing in the living room supposed to be focused on ease of use? I wouldn't want to have to fiddle with things when I just want to kick back and relax.

As far as my VGA passthrough build goes I'm playing it safe and testing things as I go. My sister has a newer IPS monitor that fits the bill for my use of the HDMI audio use so I'll probably test the audio on hers before I buy my own. She's never used the audio jack and it'd be just my luck that the implementation is shoddy. I'll probably throw my second video card into my HTPC that's running Mint 14 to test for any driver issues. Well you get the idea.

One question did just come to mind though. The last time I tried messing with windows in a VM I was using XP and I've upgraded to Win 7 since then. Does Win7 give you a grace period on activation like XP did? I was just thinking it might be prudent to experiment with Xen on a separate drive. I've had my fill of resizing and moving partitions around on my last build. I'd like to confirm it works before I go toasting my dual boot. I really hope vga passthrough works for me as I really like the idea of switching to GPT and LVM completely. Though yesterday I started toying with the idea of trying the linux implementation of ZFS in conjunction with XEN but the suggested memory requirements and the added complexity has sufficiently deterred me. Maybe down the road when I upgrade my memory and I've ironed out Xen and VGA passthrough. Doing both at once sounds headache inducing.

Though my old habits from my days doing IT work might overcome reason... Raid... everywhere.... :twisted:
powerhouse
Level 6
Level 6
Posts: 1138
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

@Locuust: You got me - yes I have a few posts on a few forums. In a way I feel that Xen and VGA passthrough is not enough known, or often looked over.

Regarding your question about Windows VM: To run Windows in a VM, you would need a retail license (or better), not an OEM license. You can install Windows 7 without registering. In fact, you should register only after you install the GPLPV drivers within Windows, and after you have installed the graphics card driver for your passed-through card.

I would first install everything and run it a few days before registering it.

About system stability: My Xen installation is a production system, I use it daily for various scores including editing photos in the Windows VM. Uptime is important to me. By the way, the Amazon EC2 service is based on Xen, if I'm not mistaken. If you have a few PCs and a network, you could give hot migration of life VMs a try.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
Locuust

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by Locuust »

powerhouse wrote:@Locuust: You got me - yes I have a few posts on a few forums. In a way I feel that Xen and VGA passthrough is not enough known, or often looked over.
I hear ya ;)
Regarding your question about Windows VM: To run Windows in a VM, you would need a retail license (or better), not an OEM license. You can install Windows 7 without registering. In fact, you should register only after you install the GPLPV drivers within Windows, and after you have installed the graphics card driver for your passed-through card.
I hope you're wrong about the OEM thing. I remember reading that in your tutorial and immediately doing a search to confirm it. I found this page that says an OEM license is ok so long as it's installed *only* in a VM. I've ran OEM copies of Windows XP in VM's in the past. I suppose their stance could have changed in one of the new versions.

In hindsight though I guess it's a bit of a mute point as I'll have to re-activate with these hardware changes anyways. Not to mention I'd probably have to reload my standard windows install anyways. Tbh I held out on upgrading from XP for a long time. I know Windows 7 is supposed to be more forgiving about hardware upgrades but it's been my experience not to trust ms software in that regard. It's a good idea to refresh your install now and again anyways. I'm just being lazy :P
I would first install everything and run it a few days before registering it.
I assume you mean activate and not register. Registering software has always been optional. ;)
About system stability: My Xen installation is a production system, I use it daily for various scores including editing photos in the Windows VM. Uptime is important to me. By the way, the Amazon EC2 service is based on Xen, if I'm not mistaken. If you have a few PCs and a network, you could give hot migration of life VMs a try.
Yeah I have 4 PC's atm. Three of which are in the living room. I could probably integrate all three together into one Xen machine but from a security standpoint I hesitate to do so. I do plan on upgrading the fileserver that's running Ubuntu LTS to Xen so that I can test out some backend servers. In sort of convoluted way getting XEN with vga passthru running will help me stop procrastinating about my need to get up to speed on some software stacks and programming languages. It's been too long since I last played around with this stuff.

*edit* Speaking of the last of my hardware just got here. Gotta love when things arrive a day early. Not to mention a couple of the parts were ordered yesterday afternoon. Gotta love free next day shipping. Time to start tinkering.
Locuust

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by Locuust »

Update: So I got Mint 14 installed and loaded Xen. Booted into it and it appears I'm having some video driver problems? At least that's my guess as the only thing I have is a desktop with two icons. No other ui elements load. When I pull up nemo from the desktop it's missing the title bar. Suffice to say some parts of X failed to load. I tried switching to the proprietary drivers and the machine just black screens booting up. I'm really hoping I can get the amd driver to work because the open source drivers weren't doing much in the way of power saving (at least out of the box) and I was sitting here sweating while the two video cards ran nice and hot.

Everything boots fine outside of Xen. I've been looking around the xen wiki and all the pages that relate to video compatibility seem to be about passthrough. I'm probably not doing it right on my searches and missing a common issue.

Before that though I probably should figure out how to get the correct video card working for Dom0. Which might be tricky. As both cards are Radeons I can't blacklist so I'm guessing I need to do some magic in grub? I'll have to read up but I'm guessing I may need to compile the kernel with xen options? This page mentions compiling everything into the kernel allows you to do the pci back reservation from grub. So I guess that's probably the way to go? Sounds a bit like what you were talking about for single card VGA passthrough (without the need to swap it since I have a second card).

Still it won't mean much if I can't get the driver issue resolved. Back to reading.
powerhouse
Level 6
Level 6
Posts: 1138
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

Locuust wrote:Update: So I got Mint 14 installed and loaded Xen. Booted into it and it appears I'm having some video driver problems? At least that's my guess as the only thing I have is a desktop with two icons. No other ui elements load. When I pull up nemo from the desktop it's missing the title bar. Suffice to say some parts of X failed to load. I tried switching to the proprietary drivers and the machine just black screens booting up. I'm really hoping I can get the amd driver to work because the open source drivers weren't doing much in the way of power saving (at least out of the box) and I was sitting here sweating while the two video cards ran nice and hot.

Everything boots fine outside of Xen. I've been looking around the xen wiki and all the pages that relate to video compatibility seem to be about passthrough. I'm probably not doing it right on my searches and missing a common issue.

Before that though I probably should figure out how to get the correct video card working for Dom0. Which might be tricky. As both cards are Radeons I can't blacklist so I'm guessing I need to do some magic in grub? I'll have to read up but I'm guessing I may need to compile the kernel with xen options? This page mentions compiling everything into the kernel allows you to do the pci back reservation from grub. So I guess that's probably the way to go? Sounds a bit like what you were talking about for single card VGA passthrough (without the need to swap it since I have a second card).

Still it won't mean much if I can't get the driver issue resolved. Back to reading.
You are not trying UEFI (EFI) boot? In other words, I hope your motherboard and disk are using the MBR method. If so, try the following:

As a first step install ssh and enable remote ssh access from another PC on your LAN. This can be very helpful to fix VGA related problems.

Then do the following:

1. Boot Xen into safe mode (or otherwise get to the terminal).
2. Install the proprietary AMD driver.
3. Blacklist the opensource AMD driver (I believe blacklist is mentioned in the how-to, if not there are descriptions on how to blacklist "nouveau" for Nvidia - it's the same for AMD).

If I understand correctly, you got 2 graphics cards. Does your CPU also have a GPU? If yes, you may want to blacklist its driver too. If these steps fail and you don't find a way to make it work, please open a separate thread and post a link to it here.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
Locuust

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by Locuust »

powerhouse wrote:As a first step install ssh and enable remote ssh access from another PC on your LAN. This can be very helpful to fix VGA related problems.

Try the following:

1. Boot Xen into safe mode (or otherwise get to the terminal).
2. Install the proprietary AMD driver.
3. Blacklist the opensource AMD driver (I believe blacklist is mentioned in the how-to, if not there are descriptions on how to blacklist "nouveau" for Nvidia - it's the same for AMD).

If I understand correctly, you got 2 graphics cards. Does your CPU also have a GPU? If yes, you may want to blacklist its driver too. If these steps fail and you don't find a way to make it work, please open a separate thread and post a link to it here.
2 Radeon's (7870 & 7750), No GPU in the CPU.

The problem *is* the proprietary AMD driver. As soon as I loaded it and rebooted into Xen I lockup before I get to the login screen. I could at least login with the open source driver but that had it's own problems. I found some bug reports relating to a fglrx error I'm getting in kern.log but there was no solution. I was tempted to try the drivers from AMD's site but they seem to be only a minor version newer and this bug is a couple years old. My suspicion is that it's a particular hardware/software combination making it difficult for the devs to reproduce.

I was busy today so I didn't get to spend as much time as I like on this, but I'm working on a custom kernel in the mean time. If nothing else it will give me time to think about what to try next. ;)

*edit* Nope, not using EFI. Mint 14 sans Xen is fine.
earlboy

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by earlboy »

Update my xen win 7 vm is running like a champ.

Here a screen shot of xen in action. :D
Image

Hardware :
intel i7 3770
Azrock Z77 extreme 4
(2) Sapphire HD 5970 - used by the VM in crossfire

OS
LinuxMint KDE Edition - booting from UEFI (Dom0)
Win7 64 bit (DomU) - running 2 ati cards in crossfire

Edit : fixed image link as it's not working. (I'm an idiot when it comes to posting images in forums.)

P.S. My el-cheapo USB sound card conked out, so got no sound right no in the win vm.
earlboy

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by earlboy »

By the way here are some harware where I've tested xen.

MB :
Asrock Z77 Extrem4 - Works very well
Asrock Z77 Extrem6 - same as extreme 4 with more whistles and bangs included
Asrock H61M-VS R2.0 - this is a very cheap board but works well with xen

GPU ATI :
Sapphire HD 7970 - Catalyst Control Center can be installed. Can run in CrossFire
Sapphire 6570 - Catalyst Catalyst Control Center can be installed.
MSI (Forgot the model) - Crashes when CCC is installed, graphics driver will install, but sometimes it crashes. Have instability issues with games. Encountered problems with directx installation on my last reinstallation of xen using this card.

My friends xen rig which is using the H61M Mobo is also running like a champ, after we replaced the MSI Graphics card with the sapphire 6570.
powerhouse
Level 6
Level 6
Posts: 1138
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

earlboy wrote:By the way here are some harware where I've tested xen.

MB :
Asrock Z77 Extrem4 - Works very well
Asrock Z77 Extrem6 - same as extreme 4 with more whistles and bangs included
Asrock H61M-VS R2.0 - this is a very cheap board but works well with xen

GPU ATI :
Sapphire HD 7970 - Catalyst Control Center can be installed. Can run in CrossFire
Sapphire 6570 - Catalyst Catalyst Control Center can be installed.
MSI (Forgot the model) - Crashes when CCC is installed, graphics driver will install, but sometimes it crashes. Have instability issues with games. Encountered problems with directx installation on my last reinstallation of xen using this card.

My friends xen rig which is using the H61M Mobo is also running like a champ, after we replaced the MSI Graphics card with the sapphire 6570.
It's great you made it with KDE and crossfire.

Also thanks for sharing the hardware info - this is always helpful! Can you get the exact model name and perhaps VGA BIOS release of the MSI card that caused troubles? I would like to post these findings over at the overclock.net forum (see links in the how-to), as well as on the Xen wiki (if not posted already).
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
powerhouse
Level 6
Level 6
Posts: 1138
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

earlboy wrote:Just got it working now. Still using your guide with the following difference.

- Used linux mint 14 kde edition ( just want to try a different DE) :D
- There is a bug in the current xen kernel (the one ubuntu is using) wherein xen only gets 512MB under UEFI no matter what amount of ram you pass at boot time. This made dom0 DE very sluggish. Had to build 4.1.4 from source, which had the fix for it.
- I also had to ugrade my kernel to 3.8 since I was getting kernel panic when the pciback script gets executed.

I first tried xen 4.2.1 but it's using the xl toolkit instead of xm. Managed to get the vm running but could not get vga passthrough to work.

Windows 7 running full virt with vga passthrough. Will be testing with tera online again as soon as I finish my work.

As for the benefit, well there is a lot of discussion regarding the benefit of uefi floating around in the net.
For me, I did cause I thought it was cool. :lol:
Just for clarification: You had to build the kernel and Xen 4.1.4 from source? And you are using the xm toolstack?

Are you willing to share detailed information on how you got it working with UEFI, incl. kernel and Xen compilation steps? Perhaps in a separate how-to? I would then link to it from the first post here.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
powerhouse
Level 6
Level 6
Posts: 1138
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

Locuust wrote:
powerhouse wrote:As a first step install ssh and enable remote ssh access from another PC on your LAN. This can be very helpful to fix VGA related problems.

Try the following:

1. Boot Xen into safe mode (or otherwise get to the terminal).
2. Install the proprietary AMD driver.
3. Blacklist the opensource AMD driver (I believe blacklist is mentioned in the how-to, if not there are descriptions on how to blacklist "nouveau" for Nvidia - it's the same for AMD).

If I understand correctly, you got 2 graphics cards. Does your CPU also have a GPU? If yes, you may want to blacklist its driver too. If these steps fail and you don't find a way to make it work, please open a separate thread and post a link to it here.
2 Radeon's (7870 & 7750), No GPU in the CPU.

The problem *is* the proprietary AMD driver. As soon as I loaded it and rebooted into Xen I lockup before I get to the login screen. I could at least login with the open source driver but that had it's own problems. I found some bug reports relating to a fglrx error I'm getting in kern.log but there was no solution. I was tempted to try the drivers from AMD's site but they seem to be only a minor version newer and this bug is a couple years old. My suspicion is that it's a particular hardware/software combination making it difficult for the devs to reproduce.

I was busy today so I didn't get to spend as much time as I like on this, but I'm working on a custom kernel in the mean time. If nothing else it will give me time to think about what to try next. ;)

*edit* Nope, not using EFI. Mint 14 sans Xen is fine.
I strongly suggest opening a separate thread under the respective installation or graphics card section of the forum to get help from others. I'm no expert with video card driver issues, and I believe there are people here that could help you. But they won't be around in this how-to. Please post a link to your thread.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
powerhouse
Level 6
Level 6
Posts: 1138
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

For those interested in having an existing Windows installation run as a VM under Xen, here is a post at overclock.net that shares some info: http://www.overclock.net/t/1205216/guid ... t_19096913. This post also describes UEFI boot. See also further down the thread with more explanations.

The way I understand this to work is this (in very simple terms):

1. One SATA controller with its own disk is used by Linux / Xen.
2. Another SATA controller with a separate HDD carrying Windows is passed through to the Windows guest.
3. Duet and rEFInd is used to have Xen boot Windows from the separate passed through controller/HDD. With Duet, the guest first boots from an image file which contains the necessary stuff to access the SATA controller and disk.

Using this method allows one to keep an existing Windows installation on a separate disk, and boot it either using dual-boot or as a Xen guest via Linux.

I have not tried this (nor do I need it for myself), but I'm sure some who already have a working Windows installation may want to experiment with it - at your own risk. Make sure to completely backup your Windows disk first, to be safe.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
earlboy

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by earlboy »

powerhouse wrote:
earlboy wrote:Just got it working now. Still using your guide with the following difference.

- Used linux mint 14 kde edition ( just want to try a different DE) :D
- There is a bug in the current xen kernel (the one ubuntu is using) wherein xen only gets 512MB under UEFI no matter what amount of ram you pass at boot time. This made dom0 DE very sluggish. Had to build 4.1.4 from source, which had the fix for it.
- I also had to ugrade my kernel to 3.8 since I was getting kernel panic when the pciback script gets executed.

I first tried xen 4.2.1 but it's using the xl toolkit instead of xm. Managed to get the vm running but could not get vga passthrough to work.

Windows 7 running full virt with vga passthrough. Will be testing with tera online again as soon as I finish my work.

As for the benefit, well there is a lot of discussion regarding the benefit of uefi floating around in the net.
For me, I did cause I thought it was cool. :lol:
Just for clarification: You had to build the kernel and Xen 4.1.4 from source? And you are using the xm toolstack?

Are you willing to share detailed information on how you got it working with UEFI, incl. kernel and Xen compilation steps? Perhaps in a separate how-to? I would then link to it from the first post here.
Yes I'm using 4.1.4 for uefi booting, and still using xm toolstack.
I'm manage to make kernel 3.5.0-26 (just did an update using package)to work with xen 4.1.4.

Here's the stuff that I did differently from your guide.
- Updated my kernel to 3.5.0-26
- Compiled xen 4.1.4. Did not do the install.sh from the build. I just copied the xen kernel to the /boot directory and updated grub
- added the following parameters to xen "iommu=1 dom0_mem=min:2048M,max:4192M dom0_max_vcpus=4 dom0_vcpus_pin" so that I won't get kernel panic. "dom0_mem=<value>" caused kernel panic, as for the max_vpsu and vcpus_pin it may not be necessary.
xen seems to be sensitive about the dom0_mem parameter, tried different values, but "dom0_mem=min:2048M,max:4192M" did it for me.
- black listed the radeon driver from the linux kernel by adding "radeon.blacklist=1". This prevents kernel panic for happening when the pciback script gets executed. I've been getting random panics with the radeon driver when pciback tries to grab it.
- when installing the graphics card driver for ati in the vm, I install only the drivers first. Then do a couple of reboots then intall all the options in the catalyst intallation.
- I don't know if this just happens on my install, but everytime I install the ati gpu drivers on a fresh win7 vm, CCC won't start at boot, because it requires microsoft .net runtime. After installing it CCC should start up with no problems

The rest I followed your guid.

Cheers.
Locuust

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by Locuust »

powerhouse wrote:For those interested in having an existing Windows installation run as a VM under Xen, here is a post at overclock.net that shares some info: http://www.overclock.net/t/1205216/guid ... t_19096913. This post also describes UEFI boot. See also further down the thread with more explanations.
Interesting. Years ago I played around with running XP in virtualbox or dual booting at the same time. Virtualbox supports booting VMs from standard partitions. Iirc you had to keep reactivating windows because it saw the two (dualboot vs VM) as different hardware. Reading further it seems like this guy was after native disk access and not dual booting (even though it would be possible with this setup). Booting UEFI within a VM seems a bit overkill though but like he said you could use GPT that way (afaik that's the only way windows will support it and only under 64bit). Since he passed a raid controller and got it's bios running you could in theory attach an array to the VM. So you could surpass the 8-9 TB (forget the exact amount) limit on mbr/msdos partitions, or maybe he just wanted to have 32+ partitions? :P
earlboy wrote:Yes I'm using 4.1.4 for uefi booting, and still using xm toolstack.
I'm manage to make kernel 3.5.0-26 (just did an update using package)to work with xen 4.1.4.

Here's the stuff that I did differently from your guide.
- Updated my kernel to 3.5.0-26
- Compiled xen 4.1.4. Did not do the install.sh from the build. I just copied the xen kernel to the /boot directory and updated grub
- added the following parameters to xen "iommu=1 dom0_mem=min:2048M,max:4192M dom0_max_vcpus=4 dom0_vcpus_pin" so that I won't get kernel panic. "dom0_mem=<value>" caused kernel panic, as for the max_vpsu and vcpus_pin it may not be necessary.
xen seems to be sensitive about the dom0_mem parameter, tried different values, but "dom0_mem=min:2048M,max:4192M" did it for me.
- black listed the radeon driver from the linux kernel by adding "radeon.blacklist=1". This prevents kernel panic for happening when the pciback script gets executed. I've been getting random panics with the radeon driver when pciback tries to grab it.
- when installing the graphics card driver for ati in the vm, I install only the drivers first. Then do a couple of reboots then intall all the options in the catalyst intallation.
- I don't know if this just happens on my install, but everytime I install the ati gpu drivers on a fresh win7 vm, CCC won't start at boot, because it requires microsoft .net runtime. After installing it CCC should start up with no problems

The rest I followed your guid.

Cheers.
Grats man on getting it working. :)

Good to know about the xen parameters. I had read the memory min/max thing was a best practice due to the memory ballooning feature of Xen, and a lot of the xen wiki documents only show the Grub1 format. I wish I could blacklist the radeon driver but I need it for my Dom0. At least I don't have kernel panics at this point. Haven't touched pciback yet tho. Good to know about the CCC delayed install. In your earlier post I thought you meant the CCC in Mint was causing issues. Figures it'd be windows and that bloatware .NET. ;)
powerhouse
Level 6
Level 6
Posts: 1138
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

earlboy wrote:Yes I'm using 4.1.4 for uefi booting, and still using xm toolstack.
I'm manage to make kernel 3.5.0-26 (just did an update using package)to work with xen 4.1.4.

Here's the stuff that I did differently from your guide.
- Updated my kernel to 3.5.0-26
- Compiled xen 4.1.4. Did not do the install.sh from the build. I just copied the xen kernel to the /boot directory and updated grub
- added the following parameters to xen "iommu=1 dom0_mem=min:2048M,max:4192M dom0_max_vcpus=4 dom0_vcpus_pin" so that I won't get kernel panic. "dom0_mem=<value>" caused kernel panic, as for the max_vpsu and vcpus_pin it may not be necessary.
xen seems to be sensitive about the dom0_mem parameter, tried different values, but "dom0_mem=min:2048M,max:4192M" did it for me.
- black listed the radeon driver from the linux kernel by adding "radeon.blacklist=1". This prevents kernel panic for happening when the pciback script gets executed. I've been getting random panics with the radeon driver when pciback tries to grab it.
- when installing the graphics card driver for ati in the vm, I install only the drivers first. Then do a couple of reboots then intall all the options in the catalyst intallation.
- I don't know if this just happens on my install, but everytime I install the ati gpu drivers on a fresh win7 vm, CCC won't start at boot, because it requires microsoft .net runtime. After installing it CCC should start up with no problems

The rest I followed your guid.

Cheers.
Thanks for your summary! Very helpful.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
powerhouse
Level 6
Level 6
Posts: 1138
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

Locuust wrote:...I wish I could blacklist the radeon driver but I need it for my Dom0. At least I don't have kernel panics at this point. Haven't touched pciback yet tho. Good to know about the CCC delayed install. In your earlier post I thought you meant the CCC in Mint was causing issues. Figures it'd be windows and that bloatware .NET. ;)
I would still blacklist the radeon opensource driver, and then try to install the closed source gflrx driver and see where this is going. There used to be issues with the Nvidia nouveau opensource driver that would get in the way of the Nvidia proprietary driver and thus needed to be blacklisted. Again, for better support, post in the relevant sub-forum and place a link here for reference.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
earlboy

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by earlboy »

Locuust wrote:
Grats man on getting it working. :)

Good to know about the xen parameters. I had read the memory min/max thing was a best practice due to the memory ballooning feature of Xen, and a lot of the xen wiki documents only show the Grub1 format. I wish I could blacklist the radeon driver but I need it for my Dom0. At least I don't have kernel panics at this point. Haven't touched pciback yet tho. Good to know about the CCC delayed install. In your earlier post I thought you meant the CCC in Mint was causing issues. Figures it'd be windows and that bloatware .NET. ;)
Just got lucky with the hardware. I tried xen a few years back. I believe vga pass-through was available through patches, but I wasn't even able to get that far, becasue my hardware doesn't support vt-d.
I was a gigabyte fan then, but the problem with gigabyte is that, a lot of their boards never had the vt-d option enabled in their bios. I was also an nvidia fan (nvidia cards are faster in gaming IMHO).

gigabyte boards + nvidia cards = worst possible hardware combination for xen.

Used a lot of gigabyte boards and the only model I found that supported vt-d was the GA-EP35 board. Info about vt-d support was kinda confusing back then. And it is still confusing today, since a lot of customer support for mobo manufacturers will tell the customer that, vt-d is a feature in the processor not in the motherboard. Which it true, but the motherboard should still provide an option to enable it if the processor supports it. In the case of gigabyte, it does not.
Imagine my frustration when I bought a gigabyte X58A-UD7 rev. 1 with an i7 920 processor. It has all the whistles ang bangs, except for vt-d. :shock:
I'm now sticking with Asrock, and sapphire knowing that I will have less problems with them, when I install xen.
earlboy

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by earlboy »

powerhouse wrote:
Locuust wrote:...I wish I could blacklist the radeon driver but I need it for my Dom0. At least I don't have kernel panics at this point. Haven't touched pciback yet tho. Good to know about the CCC delayed install. In your earlier post I thought you meant the CCC in Mint was causing issues. Figures it'd be windows and that bloatware .NET. ;)
I would still blacklist the radeon opensource driver, and then try to install the closed source gflrx driver and see where this is going. There used to be issues with the Nvidia nouveau opensource driver that would get in the way of the Nvidia proprietary driver and thus needed to be blacklisted. Again, for better support, post in the relevant sub-forum and place a link here for reference.
When I was installing xen, I noticed some graphic glitches in my system too, even though I'm using the IGP. Nothing serious but still a glitch. It completely disappeared when I blacklisted the radeon driver in the kernel parameters. Blacklisting it in modprobe module did not work for me. Maybe because the radeon drivers still gets loaded but then gets unloaded when the modprobe module kicks in.
I think your best bet here is, like you mentioned in your previous post. Recompile the kernel with pciback built in, so that you can exclude one of your gpu for being grabbed by the radeon driver when the kernel loads during boot.

Hope this helps.
powerhouse
Level 6
Level 6
Posts: 1138
Joined: Thu May 03, 2012 3:54 am
Location: Israel
Contact:

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by powerhouse »

earlboy wrote:...gigabyte boards + nvidia cards = worst possible hardware combination for xen.

Used a lot of gigabyte boards and the only model I found that supported vt-d was the GA-EP35 board. Info about vt-d support was kinda confusing back then. And it is still confusing today, since a lot of customer support for mobo manufacturers will tell the customer that, vt-d is a feature in the processor not in the motherboard. Which it true, but the motherboard should still provide an option to enable it if the processor supports it. In the case of gigabyte, it does not.
Imagine my frustration when I bought a gigabyte X58A-UD7 rev. 1 with an i7 920 processor. It has all the whistles ang bangs, except for vt-d. :shock:
I'm now sticking with Asrock, and sapphire knowing that I will have less problems with them, when I install xen.
I would add:

ASUS boards + Nvidia cards = bad for Xen (and Linux)

I'm still trying to get an intelligent answer from Asus HQ on whether their newer BIOS releases support VT-d or not. Another Asus Sabertooth X79 user on the Asus forum reported a broken VT-d on a post 1203 BIOS release. Since Asus changed the way the BIOS is stored, there was no going back for that poor guy. Until today and many releases later, it is unclear of whether or not Asus fixed VT-d. Despite the user report (and confirmation by another user), Asus wrote to me that their new BIOS supports VT-d. But in the same email they also wrote that they don't "officially" support Linux. In other words, had I believed their statement and upgraded, I'd be screwed now. By the way, Asus BIOS release notes are real masterpieces of keeping users abreast. And if that is not enough, see what US customers can expect: http://www.overclock.net/t/1310484/asus ... fx-support.

From what I see on several forums and user reports, ASRock seems to be the best choice for motherboards. They had also some broken VT-d / BIOS issues, but they at least bothered to fix them. Their BIOS release notes, though sometimes a little confusing, at least provide some information.

I personally would prefer Nvidia graphics cards, if it wasn't for the lack of Xen support and VGA passthrough. I now have a quite expensive Nvidia Quadro card only for VGA passthrough (the "multi-OS" Nvidia cards, i.e. some of the professional Quadro etc. models, support VGA passthrough, but they are much more expensive than their GeForce / consumer counterparts). For someone who wants to use Windows for games I definitely recommend getting an AMD card.
Subjects of interest: Linux, vfio passthrough virtualization, photography
See my blog on virtualization, including tutorials: https://www.heiko-sieger.info/category/ ... alization/
gordon.cooke

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Post by gordon.cooke »

locuust wrote: Does Win7 give you a grace period on activation like XP did?
Yes it does. Im doing some testing using the Windows 7 iso download from Digital River. Wanted to be sure things work before I open the seal on the liscened copy I bought and couldn't return it anymore. Looks like it is *30* days before you have to enter the activation key.

Edit: Not sure why I saw 3 somewhere-- its 30 days. In windows you can right click on Computer, select properties and the bottom of the computer info screen will show the number of days left to activate.
Last edited by gordon.cooke on Mon Apr 08, 2013 11:52 pm, edited 1 time in total.
Locked

Return to “Virtual Machines”