From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: 5x dom0 memory increase from Xen/Linux 3.4/2.6.18 to 4.1/3.0.0 Date: Wed, 22 Jun 2011 09:32:43 -0400 Message-ID: <20110622133243.GA8722@dumpdata.com> References: <20110616145611.GC6108@dumpdata.com> <25543076.8.1308324671857.JavaMail.root@zimbra.overnetdata.com> <20110620124521.GC2973@dumpdata.com> <4E01EAF7.9030900@overnetdata.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Content-Disposition: inline In-Reply-To: <4E01EAF7.9030900@overnetdata.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Anthony Wright Cc: xen-devel@lists.xensource.com List-Id: xen-devel@lists.xenproject.org On Wed, Jun 22, 2011 at 02:15:35PM +0100, Anthony Wright wrote: > On 20/06/2011 13:45, Konrad Rzeszutek Wilk wrote: > > On Fri, Jun 17, 2011 at 04:31:11PM +0100, Anthony Wright wrote: > >> Lowering swiotlb helped, and got me down to 200M for dom0. What is the effect of reducing this value? > > Less amount of bounce buffer. But you don't need the bounce buffer for PCI devices b/c you don't > > have more than 4GB of physical memory in the machine. > Do I only need bounce buffers if I have > 4GB of physical memory? In Correct. > this case should I allocate the 64M, or is it a sliding memory requirement? It really is unknown. At some point the maintainer was thinking of adding dynamic code so that it would increase as neccessary - but it never got further than: "this would be neat". I would say try some small numbers and when the machine is on full load it panics (or gives you a nasty kernel message). > >> I set CONFIG_XEN_MAX_DOMAIN_MEMORY down to 8, but that didn't seem to have any effect on dom0's memory requirement. What is this value? Does it only apply to a domU's memory usage? > > It makes some internal datastructures (P2M) smaller. They are set up for 128GB or so machines initially. > It sounds like this value applies to DomUs, does this config variables > set the maximum amount of memory 128GB per DomU or across all DomUs, > i.e. if I have 16 DomUs and a CONFIG_XEN_MAX_DOMAIN_MEMORY of 16, do > they each have a maximum of 16GB, or do the get 1GB each? I mistyped this. The internal datastructures just set the limits to what they can go to. But they don't consume any real RAM - just virtual addresses and during boot time that gets resolved (shrunk appropiately). So don't worry about it. > >> I tried the memblock=debug options, and while I got lots of output, I could see very little on the subject of memory usage. > > The numbers are what amount of memory is reserved. You can find out which are are is eating the most > > by computing the difference. > Maybe I'm misreading the output, but I couldn't see any numbers that > look like memory being assigned. I've attached the dmesg output. Do I > need to enable a CONFIG variable to get the output I need or am I > missing something. The memblock=debug should give you some idea of what is Reserved. The Reserved includes memory that is allocated by boot-time services (P2M, pagetables, NUMA) and by real reservations (for example ACPI space). Using the 'memblock=debug' can give you an idea of what services are reserving the most. Then we can narrow down who or what is eating the gobs of memory. see the 'Memory: ".. numbers. Also you might want to eliminate the balloon usage space algother by doing two things: Xen command line: dom0_mem=max:512MB Linux command line: mem=512MB That will effectivly remove any balloon space (so your Dom0 will _never_ grow up).