Re: 5x dom0 memory increase from Xen/Linux 3.4/2.6.18 to 4.1/3.0.0

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Wed, Jun 22, 2011 at 02:15:35PM +0100, Anthony Wright wrote:
> On 20/06/2011 13:45, Konrad Rzeszutek Wilk wrote:
> > On Fri, Jun 17, 2011 at 04:31:11PM +0100, Anthony Wright wrote:
> >> Lowering swiotlb helped, and got me down to 200M for dom0. What is the effect of reducing this value?
> > Less amount of bounce buffer. But you don't need the bounce buffer for PCI devices b/c you don't
> > have more than 4GB of physical memory in the machine.
> Do I only need bounce buffers if I have > 4GB of physical memory? In

Correct.
> this case should I allocate the 64M, or is it a sliding memory requirement?

It really is unknown. At some point the maintainer was thinking of adding
dynamic code so that it would increase as neccessary - but it never got
further than: "this would be neat". I would say try some small numbers
and when the machine is on full load it panics (or gives you a nasty
kernel message).

> >> I set CONFIG_XEN_MAX_DOMAIN_MEMORY down to 8, but that didn't seem to have any effect on dom0's memory requirement. What is this value? Does it only apply to a domU's memory usage?
> > It makes some internal datastructures (P2M) smaller. They are set up for 128GB or so machines initially.
> It sounds like this value applies to DomUs, does this config variables
> set the maximum amount of memory 128GB per DomU or across all DomUs,
> i.e. if I have 16 DomUs and a CONFIG_XEN_MAX_DOMAIN_MEMORY of 16, do
> they each have a maximum of 16GB, or do the get 1GB each?

I mistyped this. The internal datastructures just set the limits to what
they can go to. But they don't consume any real RAM - just virtual addresses
and during boot time that gets resolved (shrunk appropiately). So don't worry about it.

> >> I tried the memblock=debug options, and while I got lots of output, I could see very little on the subject of memory usage.
> > The numbers are what amount of memory is reserved. You can find out which are are is eating the most
> > by computing the difference.

> Maybe I'm misreading the output, but I couldn't see any numbers that
> look like memory being assigned. I've attached the dmesg output. Do I
> need to enable a CONFIG variable to get the output I need or am I
> missing something.

The memblock=debug should give you some idea of what is Reserved. The
Reserved includes memory that is allocated by boot-time services (P2M,
pagetables, NUMA) and by real reservations (for example ACPI space).
Using the 'memblock=debug' can give you an idea of what services are
reserving the most. Then we can narrow down who or what is eating the gobs
of memory.

see the 'Memory: ".. numbers. Also you might want to eliminate
the balloon usage space algother by doing two things:

Xen command line: dom0_mem=max:512MB

Linux command line: mem=512MB

That will effectivly remove any balloon space (so your Dom0 will _never_
grow up).