Re: Can't always start 32 bit domains after 64 bit domains

[Dario Faggioli <dario.faggioli@citrix.com>]

[-- Attachment #1.1: Type: text/plain, Size: 2662 bytes --]

On Tue, 2016-11-22 at 11:37 -0800, Sarah Newman wrote:
> On 11/22/2016 10:46 AM, Dario Faggioli wrote:
> > It's documented in here (although, the text can be improved a bit,
> > I think)... look for 'cpus' and 'cpus_soft' within the page: 
> > https://xenbits.xen.org/docs/unstable/man/xl.cfg.5.html
> > 
> > A more clear mention to using "all" can perhaps be added to the
> > wiki pages I listed in my other email.
> > 
> 
> Testing shows that memory allocation is still alternated between the
> two physical nodes, even after setting cpus='all' or removing
> cpus=... from the
> configuration file entirely.
> 
Not sure I'm getting. So, libxl default is to try to place a new domain
on a NUMA node (or on the smallest possible set of NUMA nodes). If you
don't specify neither cpus= nor cpus_soft= such default applies, and
you should _not_ get memory spread on more than one node (unless
placement fails for some reason, or the domain does not fit there).

If you specify cpus="node:0" or cpus_soft="node:0", libxl automatic
NUMA placement won't be execute, and you'll get memory only from NUMA
node 0.

If you specify cpus="all", libxl automatic NUMA placement won't be
executed, and you get memory from all the NUMA nodes, which AFAICT, is
what Xen does, if not instructed otherwise by the toolstack.

Are you seeing something different from this?

> If you're saying not specifying "cpus=..." will keep libxl from
> interfering with the Xens default allocation policy, then Xens
> default allocation
> policy no longer starts from the top of memory for 64 bit PV domains,
> at least for 4.6.3. Maybe it's starting from the top of memory per
> node.
> 
No, I'm saying that _not_ specifying cpus= will trigger libxl
interference. _Do_ specifying it, will either limit or disable it,
depending on how you specify it.

That's all I'm saying.

As far as I can remember, Xen's always been striping the memory among
all the available nodes, if not told otherwise. This has at least been
the case for all my test and experiments, on any NUMA box I've touched.

I have to admit I've not played much with 32 bit guests, though.

And let me add that, if we figure out what and where the issue is, and
come up with a sensible plan, I'm happy to think and help improving xl
and libxl NUMA placement to take these constraints into account.

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel