Recommendations for Virtulization Hardware

Recommendations for Virtulization Hardware

[ShadesOfGrey]

I'm looking to build a new personal computer.  I want it to function as 
a Linux desktop, provide network services for my home, and lastly, 
occasional Windows gaming.  From what I've gathered, virtualization 
using a Type 1 Hypervisor supporting PCI/VGA pass-through like KVM or 
Xen would be an attractive solution for my needs.  For reference, 
reading these threads on Ars Technica may be helpful to understand where 
I'm coming from, 
http://arstechnica.com/civis/viewtopic.php?f=6&t=1175674 and 
http://arstechnica.com/civis/viewtopic.php?f=11&t=1181867. But 
basically, I use Linux as my primary OS and would rather avoid dual 
booting or building two boxes just to play Windows games when I want to 
play Windows games.  I'm also intrigued by the concept of virtualization 
and would like to experiment with it as a solution for my case.

My problem is isolating which hardware to choose, specifically which 
combination of CPU, motherboard and video card.  Previously I had been 
relying on web searches to glean information from gaming and enthusiast 
web sites and tech specs from motherboard manufacturers.  After what I 
learned during my participation in the referenced threads at Ars 
Technica, I find myself back at square one.  Instead of trying to guess 
what hardware support KVM & Xen, and vice versa.  I'd like to know what 
hardware KVM & Xen users are actually using to run KVM & Xen? 
Particularly with consideration for 3D gaming and current generation 
hardware, BTW.

If there is need for further clarification, I'll answer any queries you 
might have.

Re: [Xen-users] Recommendations for Virtulization Hardware

[Nuno Magalhães]

Hi,

Just a Xen newbie myself, but from what i've gathered and fiddled, Xen
(P)VMs don't come with a graphics card. You'd have to remote to your
Windows HVM to play games. You can fiddle with PCI pass through for
some video cards and there's some VGA passtrough as well, but i don't
think running a recent game on a VM would provide good results (i'm
thinking FPSs here, not solitaire or simcity).

As for hardware i considered an ATI/Asus board with a Phenom II X6 and
16GB or DDR3 a while ago, plus box, PSU, no disks, around 500€... YMMV
and your needs as well.

Just my ill-informed 2c. HTH

Nuno

Re: Recommendations for Virtulization Hardware

[Alex Williamson]

On Thu, 2012-09-20 at 17:12 -0400, ShadesOfGrey wrote:
> I'm looking to build a new personal computer.  I want it to function as 
> a Linux desktop, provide network services for my home, and lastly, 
> occasional Windows gaming.  From what I've gathered, virtualization 
> using a Type 1 Hypervisor supporting PCI/VGA pass-through like KVM or 
> Xen would be an attractive solution for my needs.  For reference, 
> reading these threads on Ars Technica may be helpful to understand where 
> I'm coming from, 
> http://arstechnica.com/civis/viewtopic.php?f=6&t=1175674 and 
> http://arstechnica.com/civis/viewtopic.php?f=11&t=1181867. But 
> basically, I use Linux as my primary OS and would rather avoid dual 
> booting or building two boxes just to play Windows games when I want to 
> play Windows games.  I'm also intrigued by the concept of virtualization 
> and would like to experiment with it as a solution for my case.
> 
> My problem is isolating which hardware to choose, specifically which 
> combination of CPU, motherboard and video card.  Previously I had been 
> relying on web searches to glean information from gaming and enthusiast 
> web sites and tech specs from motherboard manufacturers.  After what I 
> learned during my participation in the referenced threads at Ars 
> Technica, I find myself back at square one.  Instead of trying to guess 
> what hardware support KVM & Xen, and vice versa.  I'd like to know what 
> hardware KVM & Xen users are actually using to run KVM & Xen? 
> Particularly with consideration for 3D gaming and current generation 
> hardware, BTW.
> 
> If there is need for further clarification, I'll answer any queries you 
> might have.

There have been a couple success reports of using AMD/ATI graphics cards
on Intel VT-d systems with KVM device assignment.  Nvidia cards have not
met with the same degree of success.  In both our cases, the graphics
device was assigned to a Windows guest as a secondary graphics head.
For me, Windows took over the assigned device as the primary graphics,
disabling the emulated graphics.  For my slow HD 5450, 3dMark seems to
get a similar score to what others see on bare metal.

Graphics device assignment on KVM is getting better, but should be
considered experimental at best.  Thanks,

Alex

Re: [Xen-users] Recommendations for Virtulization Hardware

[Andrew Bobulsky]

(CC'ing Casey on this, as I recommend his setup for an Intel-based solution)

Hello ShadesOfGrey,

Hehehe, talk about timing ;)

On Sep 20, 2012, at 5:15 PM, ShadesOfGrey <shades_of_grey@earthlink.net> wrote:

> I'm looking to build a new personal computer.  I want it to function as a Linux desktop, provide network services for my home, and lastly, occasional Windows gaming.  From what I've gathered, virtualization using a Type 1 Hypervisor supporting PCI/VGA pass-through like KVM or Xen would be an attractive solution for my needs.  For reference, reading these threads on Ars Technica may be helpful to understand where I'm coming from, http://arstechnica.com/civis/viewtopic.php?f=6&t=1175674 and http://arstechnica.com/civis/viewtopic.php?f=11&t=1181867. But basically, I use Linux as my primary OS and would rather avoid dual booting or building two boxes just to play Windows games when I want to play Windows games.  I'm also intrigued by the concept of virtualization and would like to experiment
  with it as a solution for my case.
>
> My problem is isolating which hardware to choose, specifically which combination of CPU, motherboard and video card.  Previously I had been relying on web searches to glean information from gaming and enthusiast web sites and tech specs from motherboard manufacturers.  After what I learned during my participation in the referenced threads at Ars Technica, I find myself back at square one.  Instead of trying to guess what hardware support KVM & Xen, and vice versa.  I'd like to know what hardware KVM & Xen users are actually using to run KVM & Xen? Particularly with consideration for 3D gaming and current generation hardware, BTW.
>
> If there is need for further clarification, I'll answer any queries you might have.
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xen.org
> http://lists.xen.org/xen-users

As one of the folks on the list who has done this---probably to the
most extreme degree---I can tell you it's good stuff.  It brings the
joys of datacenter consolidation to your gaming desktop, and also to
your wallet ;)

While my setup is now slightly dated, the 990FX chipset is still at
the top of the AMD offering, so you can shop around on CPUs, and buy a
cheaper "secondary" USB controller if you're not looking to cram in a
4-to-1 ratio.  I've never had much success passing through the onboard
USB from an AMD system, so I highly recommend picking up a little PCIe
x1 controller at the least.  That said, I'm convinced that highpoint
has one of the coincidentally-best products on the market for people
looking to do this, but I digress!

Take a look at, specifically, this post I made to the list some months
back, and I'll follow with some errata:
http://lists.xen.org/archives/html/xen-users/2012-05/msg00328.html

First, I've tested all of the hardware in the build that I
recommended, and indeed ended up building a four-headed unit. It works
like magic. Came in handy a few weeks ago when several of my friends
and I piled into a couple cars for a vacation where we wanted to play
games (yup, we're total nerds), but we couldn't fit four desktop cases
in addition to our stuff in the cars. :)

Second, by the time I got around to building it, the Antec One Hundred
wasn't available. Finding a case that supports 8 expansion slots is a
tough thing, but I found another similarly priced one, and it was a
dream to build.  I recommend it highly if you think you may want to
max out your slots and/or go deeper down the rabbit hole with
consolidated desktops:
http://www.newegg.com/Product/Product.aspx?Item=N82E16811112238

Aside from being a very solid case for the price point (good features
for screwless installation as well), to give you an idea of the size,
it is laid out in such a way that I could fit dual-GPU cards in it
(Radeon 5970s).  I ultimately had to remove the HDD mounts to pull it
off, but you shouldn't have that problem... Mostly because AMDs dual
GPU cards won't work for this, so don't buy one for this build.  It's
a problem with the PCIe switching hardware (well, the firmware
thereof, probably) that they use.  I'll save you the rambling, but
let's just say that it should work, but doesn't :(

Also, the case does look good! ;)

Finally, and this is unfortunate, for the AMD build, *I* recommend you
use ESXi. While Xen _does_ work with the hardware that I've listed,
I've never been able to get the VMs to work properly with the GPLPV
drivers, and these are crucial to performance.  I really, really want
to bring this project back up on Xen though, and will try again now
that 4.2 has gone RTM.  If you aren't buying anytime soon and would
like to hit me up in a few weeks, by all means drop me a line, and
I'll let you know if I've gotten around to it.


----------------


So, for the Intel route!

Casey DeLorme has, just this week, posted a fantastic set of detailed
videos and documentation on his setup, where he basically does exactly
what you're trying to accomplish.  You can find links to all of the
documentation, which I'm pretty sure covers his exact hardware, along
with videos of the installation process he used and a detailed,
written guide. Fine work if you ask me ;)

http://lists.xen.org/archives/html/xen-users/2012-09/msg00191.html

As far as his hardware goes, I'm not sure if it's the latest Intel
chips or not.  I've been eyeballing the i7-3770 myself (NOT the 3770K,
that one will not work, as [in my opinion] Intel has a pension for
artificially crippling their products for profit).  Haven't found a
board yet, but then again I started eyeballing hardware a day or two
ago.



Cheers,
Andrew Bobulsky

Re: Recommendations for Virtulization Hardware

[Casey DeLorme]

[-- Attachment #1.1: Type: text/plain, Size: 8173 bytes --]

Thanks for the CC Andrew,

I responded in private since I didn't want to spam the list.

Since we're on the subject of the latest hardware, my system was using a
Z68 Motherboard Chipset and Core i7 SandyBridge CPU (2600).  I just sold it
to a friend and upgraded to the latest.

Now running an IvyBridge 3770, Z77 Motherboard Chipset (ASRock Z77 Extreme
9), and similar components for the rest.  The Extreme 9 price is extremely
high, I wouldn't recommend it unless you have the money, their Z77 Extreme
6 is priced at the same as the Z68 Extreme4 Gen3, and has all the same
features plus more USB 3.0.  I would give that one a shake if you are
looking for newest components.  The hardest part of switching from the
Extreme4 Gen3 was remapping the USB ports to USB controllers by PCI BDF in
Linux, because I haven't found an easy way to do that using the systems
information yet.

As soon as I'm done putting together the details, I'll append the new
hardware information to the wiki.

As for changes, the new Z77 chipset has onboard USB 3.0 in exchange for
some of the formerly USB 2.0 ports.  In my performance demo video I showed
that USB 2.0 speeds are sub-par, but even SUB 2.0 devices can hit higher
rates when connected to a USB 3.0 port.  As far as the IvyBridge CPU,
onboard graphics are noticeably better in Dom0, and file decompression in
my Windows HVM appears to be faster.  I have not run any benchmark tests
yet.


I compiled the new Xen 4.2 stable, as tagged in the new 4.2 testing branch,
but I may try the older revision of Xen in my guide.  I am getting an error
on the first-launch attempt of any virtual machine with passed devices.
 When I try again it works, but it's 5 lines worth of errors which makes
the following success seem less joyful.

I tried kernel 3.5.4, and it appears to be suffering the same passthrough
bugs as 3.5.2, so I would avoid 3.5+ still.  I ended up with Kernel 3.4.11,
which is working.

One good bit of news was I kept my SSD, so I was able to just recreate the
HVM configuration and launch Windows without problems.  Only issues were
new drivers, and Windows asking to reactivate due to HW changes.


Hope this information helps,

~Casey

On Fri, Sep 21, 2012 at 12:30 PM, Andrew Bobulsky <rulerof@gmail.com> wrote:

> (CC'ing Casey on this, as I recommend his setup for an Intel-based
> solution)
>
> Hello ShadesOfGrey,
>
> Hehehe, talk about timing ;)
>
> On Sep 20, 2012, at 5:15 PM, ShadesOfGrey <shades_of_grey@earthlink.net>
> wrote:
>
> > I'm looking to build a new personal computer.  I want it to function as
> a Linux desktop, provide network services for my home, and lastly,
> occasional Windows gaming.  From what I've gathered, virtualization using a
> Type 1 Hypervisor supporting PCI/VGA pass-through like KVM or Xen would be
> an attractive solution for my needs.  For reference, reading these threads
> on Ars Technica may be helpful to understand where I'm coming from,
> http://arstechnica.com/civis/viewtopic.php?f=6&t=1175674 and
> http://arstechnica.com/civis/viewtopic.php?f=11&t=1181867. But basically,
> I use Linux as my primary OS and would rather avoid dual booting or
> building two boxes just to play Windows games when I want to play Windows
> games.  I'm also intrigued by the concept of virtualization and would like
> to experiment with it as a solution for my case.
> >
> > My problem is isolating which hardware to choose, specifically which
> combination of CPU, motherboard and video card.  Previously I had been
> relying on web searches to glean information from gaming and enthusiast web
> sites and tech specs from motherboard manufacturers.  After what I learned
> during my participation in the referenced threads at Ars Technica, I find
> myself back at square one.  Instead of trying to guess what hardware
> support KVM & Xen, and vice versa.  I'd like to know what hardware KVM &
> Xen users are actually using to run KVM & Xen? Particularly with
> consideration for 3D gaming and current generation hardware, BTW.
> >
> > If there is need for further clarification, I'll answer any queries you
> might have.
> >
> > _______________________________________________
> > Xen-users mailing list
> > Xen-users@lists.xen.org
> > http://lists.xen.org/xen-users
>
> As one of the folks on the list who has done this---probably to the
> most extreme degree---I can tell you it's good stuff.  It brings the
> joys of datacenter consolidation to your gaming desktop, and also to
> your wallet ;)
>
> While my setup is now slightly dated, the 990FX chipset is still at
> the top of the AMD offering, so you can shop around on CPUs, and buy a
> cheaper "secondary" USB controller if you're not looking to cram in a
> 4-to-1 ratio.  I've never had much success passing through the onboard
> USB from an AMD system, so I highly recommend picking up a little PCIe
> x1 controller at the least.  That said, I'm convinced that highpoint
> has one of the coincidentally-best products on the market for people
> looking to do this, but I digress!
>
> Take a look at, specifically, this post I made to the list some months
> back, and I'll follow with some errata:
> http://lists.xen.org/archives/html/xen-users/2012-05/msg00328.html
>
> First, I've tested all of the hardware in the build that I
> recommended, and indeed ended up building a four-headed unit. It works
> like magic. Came in handy a few weeks ago when several of my friends
> and I piled into a couple cars for a vacation where we wanted to play
> games (yup, we're total nerds), but we couldn't fit four desktop cases
> in addition to our stuff in the cars. :)
>
> Second, by the time I got around to building it, the Antec One Hundred
> wasn't available. Finding a case that supports 8 expansion slots is a
> tough thing, but I found another similarly priced one, and it was a
> dream to build.  I recommend it highly if you think you may want to
> max out your slots and/or go deeper down the rabbit hole with
> consolidated desktops:
> http://www.newegg.com/Product/Product.aspx?Item=N82E16811112238
>
> Aside from being a very solid case for the price point (good features
> for screwless installation as well), to give you an idea of the size,
> it is laid out in such a way that I could fit dual-GPU cards in it
> (Radeon 5970s).  I ultimately had to remove the HDD mounts to pull it
> off, but you shouldn't have that problem... Mostly because AMDs dual
> GPU cards won't work for this, so don't buy one for this build.  It's
> a problem with the PCIe switching hardware (well, the firmware
> thereof, probably) that they use.  I'll save you the rambling, but
> let's just say that it should work, but doesn't :(
>
> Also, the case does look good! ;)
>
> Finally, and this is unfortunate, for the AMD build, *I* recommend you
> use ESXi. While Xen _does_ work with the hardware that I've listed,
> I've never been able to get the VMs to work properly with the GPLPV
> drivers, and these are crucial to performance.  I really, really want
> to bring this project back up on Xen though, and will try again now
> that 4.2 has gone RTM.  If you aren't buying anytime soon and would
> like to hit me up in a few weeks, by all means drop me a line, and
> I'll let you know if I've gotten around to it.
>
>
> ----------------
>
>
> So, for the Intel route!
>
> Casey DeLorme has, just this week, posted a fantastic set of detailed
> videos and documentation on his setup, where he basically does exactly
> what you're trying to accomplish.  You can find links to all of the
> documentation, which I'm pretty sure covers his exact hardware, along
> with videos of the installation process he used and a detailed,
> written guide. Fine work if you ask me ;)
>
> http://lists.xen.org/archives/html/xen-users/2012-09/msg00191.html
>
> As far as his hardware goes, I'm not sure if it's the latest Intel
> chips or not.  I've been eyeballing the i7-3770 myself (NOT the 3770K,
> that one will not work, as [in my opinion] Intel has a pension for
> artificially crippling their products for profit).  Haven't found a
> board yet, but then again I started eyeballing hardware a day or two
> ago.
>
>
>
> Cheers,
> Andrew Bobulsky
>

[-- Attachment #1.2: Type: text/html, Size: 9904 bytes --]

[-- Attachment #2: Type: text/plain, Size: 125 bytes --]

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users