Maximum number of DomUs? (x86_64)

Maximum number of DomUs? (x86_64)

[Florian Kirstein]

Hallo,

I just tried to push a quad-opteron server with 16GB of memory to the limit
and I reached it a bit earlier than expected. It seems that no matter
how much memory I assign to each DomU, and wether i use loopback or
phy/lvm devices for the storage, or no vif/storage at all, I can't
crate more than 107 DomUs (yes, that's not a real problem, I'm just
curious :) After No. 107 that I get(*):
Using config file "vm107".
Started domain VM107
Using config file "vm108".
Error: (12, 'Cannot allocate memory')
Using config file "vm109".
Error: (12, 'Cannot allocate memory')

but of course there's still memory left in Domain-0 to balloon out:

Name                              ID Mem(MiB) VCPUs State  Time(s)
Domain-0                           0     9215     4 r-----   319.3
VM1                                1       64     1 ------     0.8
VM10                              10       64     1 -b----     0.7
VM100                            100       64     1 -b----     0.4

When I assign 128MB per DomU exactly the same effect:
Name                              ID Mem(MiB) VCPUs State  Time(s)
Domain-0                           0     2372     4 r-----  2196.3
VM1                              129      128     1 -b----     0.4
VM10                             138      128     1 -b----     0.4
VM100                            228      128     1 -b----     0.2

So I wonder, which structure could run out of memory there? Is this a known
limitation? Didn't find much about the maximum number of DomUs
in the list archives... The Dom0 is running a CentOS4 x86_64 and I don't
even need a DomU image to reach this limit, network and diskless DomUs
just running into an "no root filesystem" panic being "preserve"d after
the crash have the same effect.

(:ul8er, r@y
(*) yes, I know, I could have used a vmid parameter instead of creating
100s of configfiles :)

Re: Maximum number of DomUs? (x86_64)

[Keir Fraser]

On 9 Apr 2006, at 02:37, Florian Kirstein wrote:

> So I wonder, which structure could run out of memory there? Is this a 
> known
> limitation? Didn't find much about the maximum number of DomUs
> in the list archives... The Dom0 is running a CentOS4 x86_64 and I 
> don't
> even need a DomU image to reach this limit, network and diskless DomUs
> just running into an "no root filesystem" panic being "preserve"d after
> the crash have the same effect.

Xen's own private heap is fixed size (16MB) and that's used to allocate 
a certain amount of per-domain state. This restriction could be lifted 
on x86/64 but it's not particularly a priority right now.

  -- Keir