Supported #of CPUs/VMs per CPUs

Supported #of CPUs/VMs per CPUs

[whit]

Can find documented how many CPUS can Xen support and how many Virtual
Machines per CPU are allowed?  Can someone please supply this info? 

-- 
View this message in context: http://www.nabble.com/Supported--of-CPUs-VMs-per-CPUs-tf2278842.html#a6329630
Sent from the Xen - Dev forum at Nabble.com.

Re: Supported #of CPUs/VMs per CPUs

[Keir Fraser]

On 15/9/06 18:18, "whit" <whitson22@comcast.net> wrote:

> Can find documented how many CPUS can Xen support and how many Virtual
> Machines per CPU are allowed?  Can someone please supply this info?

Xen supports up to 32 CPUs on a 32-bit system, or 64 CPUs (by default) on a
64-bit system. Guests can have up to 32 virtual CPUs. The only limit to
number of domains is amount of hardware resource (mainly memory) that you
have to share.

 -- Keir

Re: Supported #of CPUs/VMs per CPUs

[whit]

Thanks for the reponse.  So if I understand your resposne, the following is
true.

1.  Maximum number of VMs is 32 per CPU
2.  Number of CPUs supported is 32 on 32-bit or 64 on 64bit

If above is true, how can domains/partitions can be unlimited.

Regards,

Teresa



Keir Fraser wrote:
> 
> On 15/9/06 18:18, "whit" <whitson22@comcast.net> wrote:
> 
>> Can find documented how many CPUS can Xen support and how many Virtual
>> Machines per CPU are allowed?  Can someone please supply this info?
> 
> Xen supports up to 32 CPUs on a 32-bit system, or 64 CPUs (by default) on
> a
> 64-bit system. Guests can have up to 32 virtual CPUs. The only limit to
> number of domains is amount of hardware resource (mainly memory) that you
> have to share.
> 
>  -- Keir
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 
> 

-- 
View this message in context: http://www.nabble.com/Supported--of-CPUs-VMs-per-CPUs-tf2278842.html#a6362415
Sent from the Xen - Dev mailing list archive at Nabble.com.

Re: Supported #of CPUs/VMs per CPUs

[Keir Fraser]

On 18/9/06 13:04, "whit" <whitson22@comcast.net> wrote:

> Thanks for the reponse.  So if I understand your resposne, the following is
> true.
> 
> 1.  Maximum number of VMs is 32 per CPU

No, I meant that for an SMP guest, the maximum number of virtual CPUs it can
appear to have is 16 (not 32 as I suggested in my first email).

> 2.  Number of CPUs supported is 32 on 32-bit or 64 on 64bit

Yes.

 -- Keir

RE: Supported #of CPUs/VMs per CPUs

[Puthiyaparambil, Aravindh]

> No, I meant that for an SMP guest, the maximum number of virtual CPUs
it
> can
> appear to have is 16 (not 32 as I suggested in my first email).
> 

I am a little confused here. Isn't MAX_VIRT_CPUS set at 32? I have been
able to bring up a 32-way x86_64 DomUs in the past. Has something
changed recently?

Aravindh Puthiyaparambil
Xen Development Team
Unisys

Re: Supported #of CPUs/VMs per CPUs

[Keir Fraser]

On 18/9/06 15:23, "Puthiyaparambil, Aravindh"
<aravindh.puthiyaparambil@unisys.com> wrote:

>> No, I meant that for an SMP guest, the maximum number of virtual CPUs
> it
>> can
>> appear to have is 16 (not 32 as I suggested in my first email).
>> 
> 
> I am a little confused here. Isn't MAX_VIRT_CPUS set at 32? I have been
> able to bring up a 32-way x86_64 DomUs in the past. Has something
> changed recently?

Erm, yes, looks like I was right first time after all! 32 VCPUs it is.

 -- Keir

Re: Supported #of CPUs/VMs per CPUs

[George Dunlap]

On 9/18/06, whit <whitson22@comcast.net> wrote:
> 1.  Maximum number of VMs is 32 per CPU
> If above is true, how can domains/partitions can be unlimited.

#1 is not true.  What Keir said was that the number of Virtual CPUs
per domain was 16*.

(* Amending to 16 from 32)

There is no maximum number of domains per cpu.  The maximum number of
domains *per machine* (regardless of the # of CPUs) depends on the
amount of resources available, mainly memory.

As an analogy, consider how many processes can you have per CPU in
Linux.  Well, it depends on how much memory you have, but generally a
whole lot.  (My non-loaded laptop has 141 right now.)  I've never
tried to max out the numer if idle domains in Xen, but in general, the
answer is quite a bit.

The other important answer to the question, "How many domains can I
have running on one CPU" is "One at a time."  If you have 50 domains
not doing anything, you'll probably be fine.  If you have 50 domains
all trying to do I/O- and cpu-intensive workloads on a 1-cpu system,
your performance will be awful.

OTOH, if you have 4 single-vcpu domains doing cpu-intensive stuff on a
4-cpu box, your performance will be just fine.   If you have 4
single-vcpu domains doing cpu-intensive stuff on a 1-cpu box, your
performance won't be that great.  If you have 8 single-vcpu domains
moderately loaded, active about 40-50% of the time, on a 4-cpu box,
you should have decent performance too.

Does that make sense?

 -George

Re: Supported #of CPUs/VMs per CPUs

[Florian Kirstein]

Hi,

> There is no maximum number of domains per cpu.  The maximum number of
> domains *per machine* (regardless of the # of CPUs) depends on the
> amount of resources available, mainly memory.
> I've never tried to max out the numer if idle domains in Xen, but
> in general, the answer is quite a bit.
I did try it out (see "maximum Number of DomUs" posting a few months
back) on a 16 GB Quad-Opteron, and was surprised hitting a limit
way before I used up all my memory, at about 107 DomUs. Xen has a
private heap of 16MB and each Domain needs some memory there, so
(possibly depending of what else is stored there) at about 100 DomUs
that's it.

Sure, not a real problem (there are few usefull szenarios in which
one would want 100 DomUs on one host), but worth mentioning IMHO...
Or did something change in this respect?

(:ul8er, r@y

RE: Supported #of CPUs/VMs per CPUs

[Petersson, Mats]

 

> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com 
> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of 
> Florian Kirstein
> Sent: 22 September 2006 01:34
> To: xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] Supported #of CPUs/VMs per CPUs
> 
> Hi,
> 
> > There is no maximum number of domains per cpu.  The maximum 
> number of
> > domains *per machine* (regardless of the # of CPUs) depends on the
> > amount of resources available, mainly memory.
> > I've never tried to max out the numer if idle domains in Xen, but
> > in general, the answer is quite a bit.
> I did try it out (see "maximum Number of DomUs" posting a few months
> back) on a 16 GB Quad-Opteron, and was surprised hitting a limit
> way before I used up all my memory, at about 107 DomUs. Xen has a
> private heap of 16MB and each Domain needs some memory there, so
> (possibly depending of what else is stored there) at about 100 DomUs
> that's it.

And for x86_64 there is no reason you can't change this limit, ether by
using the command-line argument, as I described in the post at that
point, or changing the parameter in the relevant header-file (also
described at the same time). There's really no reason why you can't
change this if you need to increase beyond this number of domains.

For 32-bit guests, it's a compromise, because the address space is
cramped with "only" 4GB of space, because the special heap needs to be
accessable by Xen at all times, thus mapped into all domains. This means
that the larger it is, the more memory space is occupied by the heap,
the less there is available for other things... I'm sure Keir or someone
will be able to give more details of why it is a bad thing to make it
much larger in 32-bit (including PAE, since PAE is only allowing the
entire system to have more than 4GB of memory, but at any given time,
the addressable space is 4GB, and the Xenheap must be mapped to ALL
guests). 

In .../xen/include/asm-x86/config.h:
XENHEAP_DEFAULT_MB    - Number of mbytes of heap. In 32-bit, this is 12
from DIRECTMAP_MBYTES, and 16 for x86_64.

Command line paramaeter:
xenheap_megabytes

--
Mats
> 
> Sure, not a real problem (there are few usefull szenarios in which
> one would want 100 DomUs on one host), but worth mentioning IMHO...
> Or did something change in this respect?

It's been like that for some time, I don't know how long tho'. There is,
as far as I understand, no STRICT limit for these things, it's just
about compromises between one benefit and another.

--
Mats
> 
> (:ul8er, r@y
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 
> 
> 

Re: Supported #of CPUs/VMs per CPUs

[George Dunlap]

100 virtual machines seems on one box seems like a lot to me. :-)

I beleve the Potemkin Virtual Honeyfarm project
(http://www.cs.ucsd.edu/~savage/papers/Sosp05.pdf) did some mods to
allow the numbers to go up into the multiple thousands.

 -George

On 9/22/06, Petersson, Mats <Mats.Petersson@amd.com> wrote:
>
>
> > -----Original Message-----
> > From: xen-devel-bounces@lists.xensource.com
> > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of
> > Florian Kirstein
> > Sent: 22 September 2006 01:34
> > To: xen-devel@lists.xensource.com
> > Subject: Re: [Xen-devel] Supported #of CPUs/VMs per CPUs
> >
> > Hi,
> >
> > > There is no maximum number of domains per cpu.  The maximum
> > number of
> > > domains *per machine* (regardless of the # of CPUs) depends on the
> > > amount of resources available, mainly memory.
> > > I've never tried to max out the numer if idle domains in Xen, but
> > > in general, the answer is quite a bit.
> > I did try it out (see "maximum Number of DomUs" posting a few months
> > back) on a 16 GB Quad-Opteron, and was surprised hitting a limit
> > way before I used up all my memory, at about 107 DomUs. Xen has a
> > private heap of 16MB and each Domain needs some memory there, so
> > (possibly depending of what else is stored there) at about 100 DomUs
> > that's it.
>
> And for x86_64 there is no reason you can't change this limit, ether by
> using the command-line argument, as I described in the post at that
> point, or changing the parameter in the relevant header-file (also
> described at the same time). There's really no reason why you can't
> change this if you need to increase beyond this number of domains.
>
> For 32-bit guests, it's a compromise, because the address space is
> cramped with "only" 4GB of space, because the special heap needs to be
> accessable by Xen at all times, thus mapped into all domains. This means
> that the larger it is, the more memory space is occupied by the heap,
> the less there is available for other things... I'm sure Keir or someone
> will be able to give more details of why it is a bad thing to make it
> much larger in 32-bit (including PAE, since PAE is only allowing the
> entire system to have more than 4GB of memory, but at any given time,
> the addressable space is 4GB, and the Xenheap must be mapped to ALL
> guests).
>
> In .../xen/include/asm-x86/config.h:
> XENHEAP_DEFAULT_MB    - Number of mbytes of heap. In 32-bit, this is 12
> from DIRECTMAP_MBYTES, and 16 for x86_64.
>
> Command line paramaeter:
> xenheap_megabytes
>
> --
> Mats
> >
> > Sure, not a real problem (there are few usefull szenarios in which
> > one would want 100 DomUs on one host), but worth mentioning IMHO...
> > Or did something change in this respect?
>
> It's been like that for some time, I don't know how long tho'. There is,
> as far as I understand, no STRICT limit for these things, it's just
> about compromises between one benefit and another.
>
> --
> Mats
> >
> > (:ul8er, r@y
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> >
> >
> >
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>

Re: Supported #of CPUs/VMs per CPUs

[Daniel P. Berrange]

On Mon, Sep 18, 2006 at 10:22:57AM +0100, Keir Fraser wrote:
> On 15/9/06 18:18, "whit" <whitson22@comcast.net> wrote:
> 
> > Can find documented how many CPUS can Xen support and how many Virtual
> > Machines per CPU are allowed?  Can someone please supply this info?
> 
> Xen supports up to 32 CPUs on a 32-bit system, or 64 CPUs (by default) on a
> 64-bit system. Guests can have up to 32 virtual CPUs. The only limit to
> number of domains is amount of hardware resource (mainly memory) that you
> have to share.

Are these hard limits, or just the kernel config defaults for Xen kernels ?
Any reason the guest VCPU count is limited to less than the physical number
on x86_64 ? (not that I expect much demand for 64 CPU guests)

Regards,
Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 

Re: Supported #of CPUs/VMs per CPUs

[Keir Fraser]

On 18/9/06 13:17, "Daniel P. Berrange" <berrange@redhat.com> wrote:

>> 
>> Xen supports up to 32 CPUs on a 32-bit system, or 64 CPUs (by default) on a
>> 64-bit system. Guests can have up to 32 virtual CPUs. The only limit to
>> number of domains is amount of hardware resource (mainly memory) that you
>> have to share.
> 
> Are these hard limits, or just the kernel config defaults for Xen kernels ?
> Any reason the guest VCPU count is limited to less than the physical number
> on x86_64 ? (not that I expect much demand for 64 CPU guests)

All are hard limits except number of supported CPUs on x86/64 (which is a
compile-time option).

Number of VCPUs per guest is limited by space in the shared_info page shared
between each guest and Xen. This limitation will go away in due course but
right now we have scalability at 4-8 VCPUs so in practise a hard limit of 16
is not the main issue.

 -- Keir