From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: memory size inside domU less than memory in domU config Date: Thu, 16 Jun 2011 15:25:15 -0400 Message-ID: <20110616192515.GA6100@dumpdata.com> References: <1308248350.28653.5.camel@mobile> <20110616184026.GA27795@dumpdata.com1308249917.28653.6.camel@mobile> <1308251938.28653.9.camel@mobile> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="X1bOJ3K7DJ5YkBrT" Return-path: Content-Disposition: inline In-Reply-To: <1308251938.28653.9.camel@mobile> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Vasiliy G Tolstov Cc: Dan Magenheimer , xen-devel@lists.xensource.com List-Id: xen-devel@lists.xenproject.org --X1bOJ3K7DJ5YkBrT Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Thu, Jun 16, 2011 at 11:18:58PM +0400, Vasiliy G Tolstov wrote: > On Thu, 2011-06-16 at 12:12 -0700, Dan Magenheimer wrote: > > I'm not sure, but I think Vasiliy's question is: > > > > Why/where is 78MB of memory being "wasted"? If one specifies > > 2048MB (4096MB) of memory for the guest, why doesn't the guest > > kernel report 2048MB (4096MB)? > > > > Using Vasiliy's /proc/meminfo output from below: > > 2048MB = 2097152KB; - 2016564KB = 80588KB = 78MB > > 4096MB = 4194304KB; - 4113716KB = 80588KB = 78MB > > > > Dan > > > > Thanks... for the memory! > > I really could use more / my throughput's on the floor > > The balloon is flat / my swap disk's fat / I've OOM's in store > > Overcommitted so much > > (with apologies to Bob Hope) > > =) Thanks Dan. May be this happening because i check dom0 support. I'm > try to rebuild kernel to use Daniel's memory hotplug and disable dom0 > support. Ah, that is what you meant. You are probably hitting a bug introduced by 24bdb0b62cc82120924762ae6bc85afc8c3f2b26. Can you try the attached patch please? It might not apply cleanly - if so you will have to manually fix it up. Or you can merge stable/bug.fixes from git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git and that will take care of that. --X1bOJ3K7DJ5YkBrT Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="xen_extra_mem_start.patch" commit acd049c6e99d2ad1195666195230f6881d1c1588 Author: Konrad Rzeszutek Wilk Date: Thu Jun 16 13:07:19 2011 -0400 xen/setup: Fix for incorrect xen_extra_mem_start. The earlier attempts (24bdb0b62cc82120924762ae6bc85afc8c3f2b26) at fixing this problem caused other problems to surface (PV guests with no PCI passthrough would have SWIOTLB turned on - which meant 64MB of precious contingous DMA32 memory being eaten up per guest). The problem was: "on xen we add an extra memory region at the end of the e820, and on this particular machine this extra memory region would start below 4g and cross over the 4g boundary: [0xfee01000-0x192655000) Unfortunately e820_end_of_low_ram_pfn does not expect an e820 layout like that so it returns 4g, therefore initial_memory_mapping will map [0 - 0x100000000), that is a memory range that includes some reserved memory regions." The memory range was the IOAPIC regions, and with the 1-1 mapping turned on, it would map them as RAM, not as MMIO regions. This caused the hypervisor to complain. Fortunately this is experienced only under the initial domain so we guard for it. Acked-by: Stefano Stabellini Signed-off-by: Konrad Rzeszutek Wilk diff --git a/arch/x86/xen/setup.c b/arch/x86/xen/setup.c index be1a464..60aeeb5 100644 --- a/arch/x86/xen/setup.c +++ b/arch/x86/xen/setup.c @@ -227,11 +227,7 @@ char * __init xen_memory_setup(void) memcpy(map_raw, map, sizeof(map)); e820.nr_map = 0; -#ifdef CONFIG_X86_32 xen_extra_mem_start = mem_end; -#else - xen_extra_mem_start = max((1ULL << 32), mem_end); -#endif for (i = 0; i < memmap.nr_entries; i++) { unsigned long long end; @@ -266,6 +262,12 @@ char * __init xen_memory_setup(void) if (map[i].size > 0) e820_add_region(map[i].addr, map[i].size, map[i].type); } + /* Align the balloon area so that max_low_pfn does not get set + * to be at the _end_ of the PCI gap at the far end (fee01000). + * Note that xen_extra_mem_start gets set in the loop above to be + * past the last E820 region. */ + if (xen_initial_domain() && (xen_extra_mem_start < (1ULL<<32))) + xen_extra_mem_start = (1ULL<<32); /* * In domU, the ISA region is normal, usable memory, but we --X1bOJ3K7DJ5YkBrT Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --X1bOJ3K7DJ5YkBrT--