HOW-TO make dual-boot obsolete using XEN VGA passthrough

Write tutorials and howtos in here
There are more tutorials here http://community.linuxmint.com/tutorial/welcome
Forum rules
Do not start a support topic here please. Before you post please read this

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby Locuust on Sun Apr 14, 2013 4:09 am

Well, with my crusade to get fglrx working canceled I've pressed on with the oss Radeon driver and I have everything but sound working in my DomU. I had planned on using an old PCI SB Live I found gathering dust but after inspecting my PCI/PCI-E slots I realized all my PCI slots were blocked by other cards. I'll probably just bite the bullet and get a USB sound card tomorrow. I have a kvm switch now so that makes things a bit easier. Still running dual monitors.

I got the PV drivers loaded in win7, but I'm not sure if I'm happy with the drive speed. The LV is sitting on an older sata drive (not flash) and I only got a 5.9 on the windows experience test (up from 5.1). I need to do some more meaningful tests though. All I know is when windows patched on shutdown (98 patches) it took over an hour to do. I'm not positive the PV drivers were working at that point though. All my other scores are maxed save for my processor that is like 7.3, but I only have 4 cores pinned to the DomU atm.

So far I've been stable. No problems I've seen with shutting down or rebooting the DomU. Though I haven't tested any 3D games so I can't say if performance suffers yet.

Once I've tested this for a few days I'll probably try KVM on my arch install just for "fun". :P

If I had more PCI-E slots in my machine I might consider getting a dedicated Nic and/or Controller to passthrough. But I'm worried about blocking the airflow inside my case (which could be bad when the weather here gets to the 110-115F range). Well it's time for bed.
Locuust
Level 3
Level 3
 
Posts: 156
Joined: Thu Oct 02, 2008 4:54 pm

Linux Mint is funded by ads and donations.
 

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby powerhouse on Sun Apr 14, 2013 4:47 pm

@Locuust:

Once you got everything set up I would be curious to see your Passmark results, particularly drive speed. You can download a trial version and run various tests in Windows. See my results somewhere earlier in this thread for comparison.

Passmark also compares your results with those of other "native" drives reported to them, so you immediately see how your drive performs.

KVM is starting to intrigue me too, but I don't have the time now nor any need for it. But in your case it might solve the fglrx driver issue. Then again, it has other issues (resetting the VGA card), but they may be solvable for you. If you do try, I can post some links to help solve the KVM issues.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Sapphire 7770 GPU, PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB SSD + various HD, Corsair 500R case, SeaSonic 660W Gold X PS
powerhouse
Level 5
Level 5
 
Posts: 646
Joined: Thu May 03, 2012 3:54 am

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby Locuust on Mon Apr 15, 2013 1:54 am

Well I got busy with other things irl today and never got a chance to run out and grab a USB soundcard. I did check walmart when I went to pick up some odds and ends. I figured I'd strike out but since I was there... They have usb wireless adapters and vanity usb drives but nothing in the way of sound save headphones and i-device paraphernalia. I should have just ordered it off newegg or something yesterday.

Passmark, eh? I haven't run a synthetic benchmark in ages and was checking what some review sites were using. Tom's Hardware uses IOMeter, but if I'm reading it right they're using the latest 'stable' release from 2007. oO I'll give passmark a look see. On the note of drive speed I really wish the Linux version of Zfs was more mature and under licensing compatible with the GPL. Meaning we'll probably never see it main lined into the Linux kernel. There is BtrsFs but that sounds like it's not production ready either. Too bad since I have a couple extra drives I could use here. I suppose I could do some basic software raid striping, but the lack of redundancy scares me a bit. If BSD was a little more desktop friendly I might have considered setting up Xen on it just because of Zfs. Wait.. I think Xen works on BSD... well, no matter.

Yeah that was my thought process on KVM too. There's a chance I could get it working and that (should) mean no problems with Fglrx since it wouldn't be running under a hypervisor directly. But there's so much uncertainty with KVM vga passthrough I decided to go with what I know works first. That talk about VFIO sounded promising but the patches for vga passthrough are in kernel 3.9. Here's hoping by the time 4.0 comes out we'll hear if it's working. After reading about VFIO I wondered if it might make nvidia a little more motivated in the desktop/soho server virtualization arena. As the idea is to provide PCI passthrough on any CPU architecture in KVM. Which means Nvidia might be interested in getting their video cards working with VFIO because it would port to ARM processors (which they seem plenty hot and bothered about). Read - Future vga passthrough for Nvidia cards? No, that can't be. Santa isn't real either. ;)

So, it seems network manager flips out as soon as I load my Win7 DomU. It starts cycling between declaring my connection down and requesting a new ip via DHCP. I knew that it wasn't compatible with bridges (tho somewhere I read that guys from Fedora are working on changing that), but I thought it would just ignore it since it was manually configured. I'll have to uninstall or disable it I suppose. Not sure if that will cause an issue with anything. My linux networking knowledge is a little rusty. I guess there was a point way back in 2007 were ubuntu had NM as a dependency for their desktop which meant you couldn't uninstall it. Doesn't seem to be a problem these days though.

Oh, and one other oddity. When I manually try to suspend my machine it's like it gets stuck halfway between modes and won't wake up. At first I thought it had something to do with my KVM switch (I hate acronym collisions), but then realized last night that I may have forgotten to shutdown my DomU. My theory is that because I don't have any of those 'On shutdown' entries in my DomU config file that my poor Dom0 got stuck waiting on win7 mid suspend. Need to verify if that's what's happening. Though I suppose I should get in the habit of checking (or install virt).

Update: No, scratch that. I just tried it again and I know DomU was down. My first idea is probably the correct one - my KVM. That's the only other thing that has changed.
Locuust
Level 3
Level 3
 
Posts: 156
Joined: Thu Oct 02, 2008 4:54 pm

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby powerhouse on Mon Apr 15, 2013 5:34 am

@Locuust: Regarding network manager, I believe I described how to disable it. In my case I simply ditched it. If your bridge setup is correct, you shouldn't need network manager. Somewhere earlier in this thread I posted some updates as to the network configuration. Perhaps it helps.

P.S.: I don't have wifi on my desktop PC. That would be the only use I could see for network manager.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Sapphire 7770 GPU, PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB SSD + various HD, Corsair 500R case, SeaSonic 660W Gold X PS
powerhouse
Level 5
Level 5
 
Posts: 646
Joined: Thu May 03, 2012 3:54 am

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby Locuust on Mon Apr 15, 2013 6:08 am

My bad. I was thinking that was only a temporary thing. Like stopping a service on a windows server doesn't actually keep it from loading at startup. You have to explicitly disable it from running at start up. Well, that's another reason to mess with Arch. Learning/refreshing my ailing memory. At any rate I knew NM's main advantage was auto management of wireless access of different kinds. I was just wondering if I had to setup some other service to at least setup the link at boot. But that doesn't sound to be the case. Guess I was over thinking things.

*edit* On another note do you have any problems with suspending Dom0? It occurred to me that something like this is probably low priority for a hypervisor typically used in a server stack. When I searched I found many articles dating back to 2009 about suspend issues. The latest - http://www.gossamer-threads.com/lists/xen/devel/270234 is from Feb of this year. Mind you this is for Xen 4.2 so it may have been just broken since 4.1. Well, at any rate I suppose I should have confirmed if I could suspend outside of Xen before I posted this, but my money is on it working. Brb.

Yup, works fine out of Xen. Should have realized that sooner. When I said the only thing that has changed is running Win7 DomU and my KVM switch I overlooked the elephant in the room - the fact that until now I've only spent brief periods in Xen to test things. Oops. Well, I can deal with no suspend. I was just worried that something else was borked. Now that I know the cause I can go about my business. Though, I seem to have forgotten something... @#$# I didn't order a soundcard. /sigh. Ah well.

*edit2* Got side tracked again and stumbled upon a phoronix article comparing Xen vs KVM from july of last year. I skimmed through the benchmarks and thought to myself it was a pretty close race (except for the disk benchmark at the beginning). I was surprised when their conclusion was KVM is clearly the winner... I couldn't help but feel like this was either a troll or just laziness/blatant bias. Benchmarks are really only meaningful if there's full disclosure how everything is setup for one. They didn't specify if the PV disk drivers were being used (it seems the PV network drivers were). I know personally there is some performance hit in Xen Disk I/O as I've experienced it, but at least be clear how you tested. This makes me tempted to run my own test. Some people pointed on the decision to run Xen in HVM in the comments, but I think it's fair to compare HVM to HVM. KVM doesn't have a PV mode after all. Oh, and it seemed Xen did really well in the floating point operations section. That seemed to be glossed over. Which makes one wonder... what use case were they even trying to benchmark? oO At any rate I got a chuckle out of it. I'm no Xen fanboi mind you (I've had my share of Xen headaches) and it would certainly intrigue me if my own testing showed disk I/O to be significantly faster under KVM. Otherwise I don't see enough disparities for me to switch. Of course maybe these numbers are more critical to big iron guys who are rolling out server farms. A performance difference of a few percent is way more meaningful at such a large scale. Yet, I have a feeling that they'd be more inclined to run their own benchmarks using their own workloads to answer the question - Xen or KVM? At least anyone worth their salt.
Locuust
Level 3
Level 3
 
Posts: 156
Joined: Thu Oct 02, 2008 4:54 pm

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby gordon.cooke on Mon Apr 15, 2013 11:15 pm

I can't take a screenshot inside windows yet (Im passing kb thorugh vnc so print screen not working... yet) Windows performance is:
Overall 4.7
  • Processor 6.6
  • RAM 5.5
  • Graphics 4.7
  • Game Graphics 6.1
  • HD 5.9
Im a little curious why the graphics is just 4.7, but the Game graphics for 3D etc is a 6.1?

i dontthink that is too shabby though. Im giving it two cores, with 2GB of RAM and 40GB raw LVM hard drive. Just need it for some basic office type tasks and maybe the occassional game (nothing hardcore).
------------------------------------------------------------------------
Mint 13 64bit Cinnamon
Asus U56E Laptop, Core i5-2410M, Intel graphics, 6GB RAM
Mint 9 64bit
Averatec 2573 Laptop, AMD Turion 64x2, ATI Graphics, Atheros wireless, 3GB RAM
User avatar
gordon.cooke
Level 3
Level 3
 
Posts: 151
Joined: Tue Dec 01, 2009 9:48 pm
Location: New York

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby Locuust on Mon Apr 15, 2013 11:17 pm

So I had been wondering what the "viridian=1" option meant in Xen DomU configs and did some digging. I guess this is a reference to the code name of MS's Hyper-V. More specifically the option allows Xen to notify windows vista or later guests that - 1) they are running in a VM, 2) certain PV options are available. From what I can determine from this page is that support PV support is rather limited. There is some talk in that thread that exposing Hyper-V "enlightened I/O" in the Xen backend drivers would mean out-of-the-box PV driver support for Vista and greater (if I understand it correctly). Unfortunately there's not exactly anyone volunteering to backward engineer MS's code into Xen for fear of the legalities. So in the end I think it's safe to say that we are going to be using James Harper's PV drivers for the foreseeable future.

It's ironic though that this conversation takes place roughly a month after MS submitted Hyper-V PV drivers into the Linux kernel after getting caught using GPL code in some Hyper-V drivers. Maybe what is being left unsaid here is that no one wants to stir up the hornet's nest after it's settled down. I remember hearing about this incident in passing but don't know of the details.

I'm sure this has been discussed in elsewhere and Powerhouse's config does have this option enabled, but I figured I'd just add this as a boot note to anyone following this thread. It will help with stability, but not I/O performance afaik.
Locuust
Level 3
Level 3
 
Posts: 156
Joined: Thu Oct 02, 2008 4:54 pm

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby Locuust on Tue Apr 16, 2013 12:02 am

gordon.cooke wrote:I can't take a screenshot inside windows yet (Im passing kb thorugh vnc so print screen not working... yet) Windows performance is:
Overall 4.7
  • Processor 6.6
  • RAM 5.5
  • Graphics 4.7
  • Game Graphics 6.1
  • HD 5.9
Im a little curious why the graphics is just 4.7, but the Game graphics for 3D etc is a 6.1?

i dontthink that is too shabby though. Im giving it two cores, with 2GB of RAM and 40GB raw LVM hard drive. Just need it for some basic office type tasks and maybe the occassional game (nothing hardcore).


I was at first surprised that your Processor and RAM scores are that much lower than mine considering you processor is an 8 core (I noticed in the other thread about Fglrx). But your only giving it 2 cores and 2GB of RAM. If you click on the "What do these numbers mean?" link on the Win7 performance screen it mentions that having <= 4 Gb of ram in 64bit automatically lowers the RAM score to 5.9. And I'm using 4 cores on my DomU. Is there a reason you set the RAM so low? Might consider loading 32bit windows if that's all you can afford to give it.

As for the graphics your guess is as good as mine. These sort of synthetic benchmarks are of questionable value anyhow. I'd check to see if you get playable frame rates in the games you play. If you do than rejoice at your dual-boot-less future ;)

Seems like we're in the same relative boat on hard drive performance ;) But again the meaning of these numbers are a bit subjective. What matters is if it performs to your expectations. Though they are of some value just to see if certain tweaks had a appreciable difference. Still, the fact that your RAM score is dictated by a certain amount of RAM makes me wonder just how arbitrary this windows experience thing is. What if not having signed controller drivers automatically demotes you to 5.9 unless your bypass some transfer rate threshold (powerhouse does have a flash drive)? I need to stop getting side tracked and run some real HD benchmarks.
Locuust
Level 3
Level 3
 
Posts: 156
Joined: Thu Oct 02, 2008 4:54 pm

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby powerhouse on Tue Apr 16, 2013 3:20 am

Locuust wrote:*edit* On another note do you have any problems with suspending Dom0?


I never tried suspending dom0 and made sure it won't suspend. To me it doesn't make sense to suspend dom0. How would dom0 know what's going on in a domU, even if it was possible to suspend? To me it's good enough that Xen applies the frequency scaling for the CPU and power states for PCI devices to keep power consumption at a minimum.

Locuust wrote:*edit2* Got side tracked again and stumbled upon a phoronix article comparing Xen vs KVM from july of last year. I skimmed through the benchmarks and thought to myself it was a pretty close race (except for the disk benchmark at the beginning). I was surprised when their conclusion was KVM is clearly the winner... I couldn't help but feel like this was either a troll or just laziness/blatant bias. Benchmarks are really only meaningful if there's full disclosure how everything is setup for one. They didn't specify if the PV disk drivers were being used (it seems the PV network drivers were). I know personally there is some performance hit in Xen Disk I/O as I've experienced it, but at least be clear how you tested. This makes me tempted to run my own test. Some people pointed on the decision to run Xen in HVM in the comments, but I think it's fair to compare HVM to HVM. KVM doesn't have a PV mode after all. Oh, and it seemed Xen did really well in the floating point operations section. That seemed to be glossed over. Which makes one wonder... what use case were they even trying to benchmark? oO At any rate I got a chuckle out of it. I'm no Xen fanboi mind you (I've had my share of Xen headaches) and it would certainly intrigue me if my own testing showed disk I/O to be significantly faster under KVM. Otherwise I don't see enough disparities for me to switch. Of course maybe these numbers are more critical to big iron guys who are rolling out server farms. A performance difference of a few percent is way more meaningful at such a large scale. Yet, I have a feeling that they'd be more inclined to run their own benchmarks using their own workloads to answer the question - Xen or KVM? At least anyone worth their salt.


The article you mentioned has been criticized by some - including me. You are probably right to question the bias of the person who's done the testing and wrote the article. In another Phoronix test they were able to show some problem with Xen, but when a Xen developer wrote to Phoronix and suggested to apply available patches to fix the problems, the Phoronix people (Michael?) didn't even bother to reply.

I disagree with comparing HVM with HVM (Xen versus KVM) when running Linux guests. When benchmarking different solutions, one needs to make sure to choose the best possible option. Under Xen this would probably be a PV guest. In any case, the article doesn't go into details as to how they tested Xen. They may have used PV-HVM drivers, but it sure doesn't look like they used a PV guest. So I really don't know what that test is good for, except for confusing people or promoting RedHat (KVM is largely sponsored by RedHat, and their commercial virtualization product for enterprise customers has KVM at its heart).

Now, if I remember correctly, Xen is used by Amazon EC2, Rackspace, and a host of other cloud service providers. I am quite sure they keep an eye on performance issues, as this means big money for them. Now, KVM is the newcomer and still has to proof itself, which may prevent the big guys from switching camps at this stage. What I am trying to say is that I don't know whether Xen or KVM performs better, and under which circumstances, but the two or three Phoronix benchmark comparisons I've seen seem to contribute more to the confusion.

On a practical side, I've read about significant performance differences between Xen and KVM by someone who tried both, who found KVM to be roughly 10x as fast as Xen for kernel compilations, yet others had performance issues with KVM (performance variations). To me that only shows that one needs to pay close attention to the Xen or KVM configuration, perhaps that Xen is a bit more finicky about it's setup, given its many options. I believe my Passmark results and the WEI I posted earlier in the thread show clearly what Xen is capable of. If you look at the Passmark I/O results for different storage types (SSD, striped LVM on 2 HDDs, regular LVM spanning on 2 HDDs), you will see that the results are very close to bare metal performance.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Sapphire 7770 GPU, PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB SSD + various HD, Corsair 500R case, SeaSonic 660W Gold X PS
powerhouse
Level 5
Level 5
 
Posts: 646
Joined: Thu May 03, 2012 3:54 am

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby powerhouse on Tue Apr 16, 2013 3:51 am

gordon.cooke wrote:I can't take a screenshot inside windows yet (Im passing kb thorugh vnc so print screen not working... yet) Windows performance is:
Overall 4.7
  • Processor 6.6
  • RAM 5.5
  • Graphics 4.7
  • Game Graphics 6.1
  • HD 5.9
Im a little curious why the graphics is just 4.7, but the Game graphics for 3D etc is a 6.1?

i dontthink that is too shabby though. Im giving it two cores, with 2GB of RAM and 40GB raw LVM hard drive. Just need it for some basic office type tasks and maybe the occassional game (nothing hardcore).


Thanks for sharing your WEI results. About 2 cores: do you mean 2 VCPUs or 2 cores = 4 VCPUs?

Your Windows guest resources are quite minimal, and 2GB RAM is borderline. About HD 5.9, you did install the latest GPLPV drivers? 5.9 would be OK for regular HDD, but very slow for SSD. The graphics results look a little strange. Don't know if more CPU or RAM would change them, but worth trying.

On the other hand, if all you need Windows for is to get some office tasks done, and the occasional game as you say, the configuration might just be right for you. Just for comparison, I run only 1 guest (Win 7 Pro 64bit) and do most stuff under Linux dom0. Yet my Windows guest gets most of the CPU (10 of 12 VCPUs) and RAM (24GB of 32GB) resources, my dom0 is limited to 6GB RAM. In practice dom0 will take all the CPU resources it needs when Windows isn't running, and all the CPU resources available when Windows is running. So, when Windows is running idle, my dom0 uses practically all 12 VCPUs. Xen has some scheduler that shares CPU time to dom0 and the guests, depending on demand. Only when my Windows guest runs a CPU demanding task will it claim the maximum CPU resources (5/6th of what the processor got).

There are 2 recommendations concerning performance:

1. Make sure dom0 gets enough CPU resources (for handling I/O, network, etc.) - in my case (6-core CPU = 12 VCPUs), 2 VCPUs are enough. In a 4-core / 8 VCPU setup 1-2 VCPUs should do. Some suggest using CPU pinning, but I haven't tried it.

2. It's been suggested to disable memory ballooning. With Linux Mint, 2GB of memory should be fine, but 4GB are definitely enough. I adjusted my /etc/fstab to have the temp file systems in RAM, which reduces write access to my SSD.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Sapphire 7770 GPU, PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB SSD + various HD, Corsair 500R case, SeaSonic 660W Gold X PS
powerhouse
Level 5
Level 5
 
Posts: 646
Joined: Thu May 03, 2012 3:54 am

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby Locuust on Tue Apr 16, 2013 4:08 am

powerhouse wrote:I never tried suspending dom0 and made sure it won't suspend. To me it doesn't make sense to suspend dom0. How would dom0 know what's going on in a domU, even if it was possible to suspend? To me it's good enough that Xen applies the frequency scaling for the CPU and power states for PCI devices to keep power consumption at a minimum.


Hmm.. For some reason I thought the config file had something like "on_suspend" but I was mistaken (you can suspend or save DomU's after all). I was in the habit of suspending my machine so it would be cooler/quieter at night (it's in my bedroom) and didn't want to close the running apps. Just being lazy again. Like I said I can live with shutting down my machine just need to get out of the habit. I'm honestly not as worried about the power consumption as I should be. But heat is not your friend in the desert. So forgive me for not adhering to what you think makes sense. ;)

The article you mentioned has been criticized by some - including me. You are probably right to question the bias of the person who's done the testing and wrote the article. In another Phoronix test they were able to show some problem with Xen, but when a Xen developer wrote to Phoronix and suggested to apply available patches to fix the problems, the Phoronix people (Michael?) didn't even bother to reply.


That doesn't strike me as proper journalism, but that's assuming Phoronix is ran by proper journalists or enthusiasts/hackers? Actually don't know the answer to that. That said I have noticed that the Xen developers are awfully quick to suggest patch X.Y.Z. In addition it seems like they have a hard time getting their patches applied to the kernel. I'm not sure if there's politics involved here or what, but I get the feeling that there's friction for reasons unbeknownst to me. As an end user I'm not to thrilled about having to patch things to get them working, but sometimes you have to get your hands dirty to things to work.

I disagree with comparing HVM with HVM (Xen versus KVM) when running Linux guests. When benchmarking different solutions, one needs to make sure to choose the best possible option. Under Xen this would probably be a PV guest. In any case, the article doesn't go into details as to how they tested Xen. They may have used PV-HVM drivers, but it sure doesn't look like they used a PV guest. So I really don't know what that test is good for, except for confusing people or promoting RedHat (KVM is largely sponsored by RedHat, and their commercial virtualization product for enterprise customers has KVM at its heart).


lol. Well KVM does seem to be the youngest child and Xen the red headed stepchild. In any case perhaps they should have tested Xen in both modes then? Again it's not comparing apples and apples, but then at least you'd be contrasting the performance differences of the modes. What I detest is when the benchmarks themselves favor a vendor/solution. For example, when binaries are compiled to take advantage of vendor specific features. Whenever I see a site comparing media encoding times I have to wonder if they are only taking advantage of Intel multimedia CPU features, etc. Well, putting aside it's accuracy, the test is meaningful to the topic of this discussion as Windows guests can't run in PV mode. :)

Now, if I remember correctly, Xen is used by Amazon EC2, Rackspace, and a host of other cloud service providers. I am quite sure they keep an eye on performance issues, as this means big money for them. Now, KVM is the newcomer and still has to proof itself, which may prevent the big guys from switching camps at this stage. What I am trying to say is that I don't know whether Xen or KVM performs better, and under which circumstances, but the two or three Phoronix benchmark comparisons I've seen seem to contribute more to the confusion.


Yeah, Opensuse actually includes Kernel versions for Amazon EC2. lol. I don't disagree with you. These guys keep an eye on the performance prize. I was suggesting that they wouldn't be the ones swayed by two or three Phoronix benchmarks. If they are.. well.. that certainly doesn't bode well for the future of the "cloud". God, I hate that marketing term. "Cloud" > /dev/null.

On a practical side, I've read about significant performance differences between Xen and KVM by someone who tried both, who found KVM to be roughly 10x as fast as Xen for kernel compilations, yet others had performance issues with KVM (performance variations). To me that only shows that one needs to pay close attention to the Xen or KVM configuration, perhaps that Xen is a bit more finicky about it's setup, given its many options. I believe my Passmark results and the WEI I posted earlier in the thread show clearly what Xen is capable of. If you look at the Passmark I/O results for different storage types (SSD, striped LVM on 2 HDDs, regular LVM spanning on 2 HDDs), you will see that the results are very close to bare metal performance.


Yup, stick the "your experience may vary" sticker on the whole thing and call it good. I guess this is a landmind I shouldn't have brought up.

At any rate I do plan on confirming just how close to bare metal I've been able to get.
Locuust
Level 3
Level 3
 
Posts: 156
Joined: Thu Oct 02, 2008 4:54 pm

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby Locuust on Tue Apr 16, 2013 4:36 am

powerhouse wrote:Thanks for sharing your WEI results. About 2 cores: do you mean 2 VCPUs or 2 cores = 4 VCPUs?

He's running an AMD processor so Num_VCPUs = Num_CPU/Cores.

In practice dom0 will take all the CPU resources it needs when Windows isn't running, and all the CPU resources available when Windows is running. So, when Windows is running idle, my dom0 uses practically all 12 VCPUs. Xen has some scheduler that shares CPU time to dom0 and the guests, depending on demand. Only when my Windows guest runs a CPU demanding task will it claim the maximum CPU resources (5/6th of what the processor got).


Afaik Dom0 will only use as many cores as vcpus if told to "pin" in grub. Otherwise the vcpus will roam across cores at the schedulers direction. You can also "pin" DomU's to specific cores as well. This is what I've done so that neither Dom0 or DomU interferes with each other. The down side being my Dom0 will never use my all my cores. I suppose it might be better to unpin Dom0 and keep Domu pinned. That way there will always be a couple cores completely free for Dom0 to use but it can still use all cores. There was also mention of a scheduling weight option for Dom0 in the xen wiki. Setting a higher weight for Dom0 would mean it's vcpu's could preempt DomU's. Probably more useful in a server where many DomU's with PV drivers are stressing the Dom0's backend drivers handling I/O.

Ballooning is automatically disabled when you specify a min and max memory amount for Dom0. Well, I guess technically if you set them to the same amounts. Otherwise it can "balloon" to the max amount after boot. The problem was something about Linux buffer/page allocation being determined at startup.
Locuust
Level 3
Level 3
 
Posts: 156
Joined: Thu Oct 02, 2008 4:54 pm

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby powerhouse on Wed Apr 17, 2013 5:16 pm

Locuust wrote:Hmm.. For some reason I thought the config file had something like "on_suspend" but I was mistaken (you can suspend or save DomU's after all). I was in the habit of suspending my machine so it would be cooler/quieter at night (it's in my bedroom) and didn't want to close the running apps. Just being lazy again. Like I said I can live with shutting down my machine just need to get out of the habit. I'm honestly not as worried about the power consumption as I should be. But heat is not your friend in the desert. So forgive me for not adhering to what you think makes sense. ;)


I can appreciate heat issues - live near the desert myself. The way I cope with it is using small dedicated devices like networked media tank for multimedia and an old notebook as a server - both have low power consumption and are quite robust to heat. My main PC is turned off when I'm not using it. SSD makes booting time a non-issue, though booting Xen and dom0 takes around 40 seconds, so does booting Windows domU. Booting a non-Xen Linux Mint takes perhaps 20-25 seconds.

There is some talk about S3 suspend in Xen 4.2.1, see http://www.gossamer-threads.com/lists/xen/devel/270234. But it looks like it's still experimental. You would also have to use the xl toolstack.

Locuust wrote:That doesn't strike me as proper journalism, but that's assuming Phoronix is ran by proper journalists or enthusiasts/hackers? Actually don't know the answer to that. That said I have noticed that the Xen developers are awfully quick to suggest patch X.Y.Z. In addition it seems like they have a hard time getting their patches applied to the kernel. I'm not sure if there's politics involved here or what, but I get the feeling that there's friction for reasons unbeknownst to me. As an end user I'm not to thrilled about having to patch things to get them working, but sometimes you have to get your hands dirty to things to work.


Let's not forget that Xen wasn't really invented for home use. It's comparable to VMware or other enterprise virtualization / cloud solutions. As to Phoronix, I don't know the motives, or if there are any. It probably boils down to practical limitations or perhaps lack of knowhow.

There are other studies/comparisons floating on the web, but some look rather marketing driven.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Sapphire 7770 GPU, PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB SSD + various HD, Corsair 500R case, SeaSonic 660W Gold X PS
powerhouse
Level 5
Level 5
 
Posts: 646
Joined: Thu May 03, 2012 3:54 am

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby powerhouse on Thu Apr 18, 2013 4:12 pm

I edited the how-to and changed the numbering to consecutive numbers in the hope that this improves readability.

Just today I noticed that I had forgotten an important step (though I hinted to it in the how-to): Enable VT-d / IOMMU in the motherboard BIOS !

Most if not all motherboards will have this option disabled by default. I just hope that those of you who tried the how-to noticed this and turned VT-d on.

As a side effect I had to renumber all the steps. Sorry for the confusion this may cause. One reason why I put release notes at the beginning.
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Sapphire 7770 GPU, PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB SSD + various HD, Corsair 500R case, SeaSonic 660W Gold X PS
powerhouse
Level 5
Level 5
 
Posts: 646
Joined: Thu May 03, 2012 3:54 am

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby Locuust on Fri Apr 19, 2013 4:55 pm

I'm in the middle of trying Xen vga passthrough on a different distro and I decided my Windows install was foobar so I decided to grab the drivers I had downloaded off the LV and do a fresh install. When I tried mounting my Win7 LV I got an error about it not appearing to be a proper ntfs drive. I was confused until I remembered that it appears as an actual disk to Win7 and it creates partitions inside the LV. Fortunately there's an easy way to map the partitions to /dev so you can mount them as normal.
kpartx -av /dev/mapper/<LV_NAME_HERE>

It should display the device names it's created (usually the LV's name with a 'p#' at the end)
Once you've mounted/unmounted the partition you need be sure to -
kpartx -dv /dev/mapper/<LV_NAME_HERE>


It works for raw image files too I believe though I have not tried it.

Incidently, I think I might have nuked my windows install for not. It appears Xen 4.2 and/or XL had decided to move my vnc session to another port. I'd connect via
vncviewer localhost:0

...and would get the normal windows splash screen as though it was actually loading. Nope. Just the after image you get when secondary vga passthrough kicks in and the emulated VGA gets turned off. I edited my configuration file and specified "vncdisplay=10" and ran
vncviewer localhost:10
that properly connected things and I was off to do my install. At some point I had commented out my pci passthroughs in my config file so it maybe with Xen 4.2 that you need to get through the install before passing (otherwise your VGA emulation gets borked).
Locuust
Level 3
Level 3
 
Posts: 156
Joined: Thu Oct 02, 2008 4:54 pm

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby powerhouse on Sat Apr 20, 2013 5:11 am

Re kpartx, see also: http://forums.linuxmint.com/viewtopic.php?f=42&t=111783

Thanks Locuust for bringing this up. Just be very careful not to mess up the Windows file system. kpartx gives you access to the "drives" of the Windows guest but Windows isn't aware of that (unlike with Samba). So it's best to use it while your Windows guest is shut down, or mount the Windows guest file system "read-only".
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Sapphire 7770 GPU, PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB SSD + various HD, Corsair 500R case, SeaSonic 660W Gold X PS
powerhouse
Level 5
Level 5
 
Posts: 646
Joined: Thu May 03, 2012 3:54 am

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby Locuust on Sat Apr 20, 2013 8:34 am

lol. Guess I'm late to the party. Anyhow, good point on not accessing it while the VM/guest is active. Not good to assume everyone knows.

I'm pricing what it would cost to build a second machine and do a decent upgrade on my server. Think I can keep it under a grand with the spare parts and cases I have laying around. I might not have had luck with VGA passthrough but it was educational at least. It's certainly convinced me I need to keep a Linux destop running for my day to day. And I'll still be playing around with Xen or KVM even if it's not on my desktop.

Well I suppose I actually *did* get VGA passthrough to work. I just couldn't get everything to work at once. ;)
Locuust
Level 3
Level 3
 
Posts: 156
Joined: Thu Oct 02, 2008 4:54 pm

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby mantisghost on Sat Apr 20, 2013 3:31 pm

Hello,

I think I am so so close but I have no idea what I do wrong. I did follow this awesome tutorial everything works untill I try to passthrought a radeon hd 4850 gpu (well making a bridge makes internet connection bugged and deinstaling packets mentiones before do no help but that wi will solve later). Anyway first my spec's are:

Linux Mint 14
cpu: i7 3770 (no k)
ram:32 gb
motherboard: asrock z77 extreme 4
gpu: radeon hd 4850

Now when I try to create virtual machine using cfg file I get kernel panic if im not mistaken.
It looks like this:
Image
Image

I did set up UEFI bios to use cpu integrated graphic card as default so radeon should be free to use also I did update kernel to 3.7.6 yet I still get screens of death. Im quite green when it comes to xen and linux but still I like the idea of vga passthrought. That said I would be gratefull for any help regarding this issue.
mantisghost
Level 1
Level 1
 
Posts: 8
Joined: Sat Apr 20, 2013 3:20 pm

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby powerhouse on Sat Apr 20, 2013 5:25 pm

mantisghost wrote:Hello,

I think I am so so close but I have no idea what I do wrong. I did follow this awesome tutorial everything works untill I try to passthrought a radeon hd 4850 gpu (well making a bridge makes internet connection bugged and deinstaling packets mentiones before do no help but that wi will solve later). Anyway first my spec's are:

Linux Mint 14
cpu: i7 3770 (no k)
ram:32 gb
motherboard: asrock z77 extreme 4
gpu: radeon hd 4850

Now when I try to create virtual machine using cfg file I get kernel panic if im not mistaken.
It looks like this:
Image
Image

I did set up UEFI bios to use cpu integrated graphic card as default so radeon should be free to use also I did update kernel to 3.7.6 yet I still get screens of death. Im quite green when it comes to xen and linux but still I like the idea of vga passthrought. That said I would be gratefull for any help regarding this issue.


I'm not sure about the Radeon HD 4850 (please check yourself) but the rest of the hardware looks good.

Start with step 5. below to list the assignable devices. Does your graphics card show?

1. Boot into the BIOS and check your BIOS release. Also make sure VT-d is enabled in the BIOS !!! Then go to the ASrock website and see the BIOS release notes. Make sure you got a BIOS release that supports VT-d. See also http://www.overclock.net/t/1338063/vt-d-compatible-motherboards.

2. Use the standard kernel that comes with LM 14. Mine is 3.5.0-27-generic. Update all packages using the standard repos.

3. Enter lsmod to list the loaded modules - "radeon" should NOT be listed. Blacklist the radeon driver - you don't need it at all! I think this might be the problem in your case.

4. Make sure you got the PCI IDs right. Check your pciback.conf file in /etc/xen. Only include the PCI IDs for the graphics card. Edit your win.cfg guest configuration file and compare the pci= line with whats in the pciback.conf file.

5. After you fixed everything, reboot into xen and check the assignable PCI devices:
Code: Select all
sudo xm pci-list-assignable-devices

It should list the PCI devices you defined in both the pciback.conf and your win.cfg file. Only when your graphics card is listed, try to boot the guest (sudo xm create...).

Please open a new thread and include your win.cfg file, as well as your /etc/default/grub file. Post a link to your thread here!
Asus Sabertooth X79, i7 3930K CPU, 8x4GB Kingston DDR3 RAM, Noctua NH-D14 CPU cooler, Sapphire 7770 GPU, PNY Quadro 2000 GPU, Asus Xonar Essence STX, Sandisk Extreme 120GB SSD + various HD, Corsair 500R case, SeaSonic 660W Gold X PS
powerhouse
Level 5
Level 5
 
Posts: 646
Joined: Thu May 03, 2012 3:54 am

Re: HOW-TO make dual-boot obsolete using XEN VGA passthrough

Postby earlboy on Sun Apr 21, 2013 11:04 am

mantisghost wrote:Hello,

I think I am so so close but I have no idea what I do wrong. I did follow this awesome tutorial everything works untill I try to passthrought a radeon hd 4850 gpu (well making a bridge makes internet connection bugged and deinstaling packets mentiones before do no help but that wi will solve later). Anyway first my spec's are:

Linux Mint 14
cpu: i7 3770 (no k)
ram:32 gb
motherboard: asrock z77 extreme 4
gpu: radeon hd 4850

Now when I try to create virtual machine using cfg file I get kernel panic if im not mistaken.
It looks like this:
Image
Image

I did set up UEFI bios to use cpu integrated graphic card as default so radeon should be free to use also I did update kernel to 3.7.6 yet I still get screens of death. Im quite green when it comes to xen and linux but still I like the idea of vga passthrought. That said I would be gratefull for any help regarding this issue.


Seen this error in my installation of xen. If you haven't added "radeon.blacklist=1", try adding it in the kernel parameters.
earlboy
Level 1
Level 1
 
Posts: 14
Joined: Fri Mar 08, 2013 4:35 am

Linux Mint is funded by ads and donations.
 
PreviousNext

Return to Tutorials / Howtos

Who is online

Users browsing this forum: No registered users and 4 guests