From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jeremy Fitzhardinge Subject: Re: RE: Ballooning up Date: Mon, 13 Sep 2010 15:51:58 -0700 Message-ID: <4C8EAB0E.7040407@goop.org> References: <4C85F973.2030007@goop.org> <54eebb3a-f539-43be-8134-a969a4f671c4@default> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <54eebb3a-f539-43be-8134-a969a4f671c4@default> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Dan Magenheimer Cc: Stefano Stabellini , Xen-devel@lists.xensource.com, Daniel Kiper , Konrad Wilk List-Id: xen-devel@lists.xenproject.org On 09/13/2010 02:17 PM, Dan Magenheimer wrote: >> As a side-effect, it also works for dom0. If you set dom0_mem on the >> Xen command line, then nr_pages is limited to that value, but the >> kernel >> can still see the system's real E820 map, and therefore adds all the >> system's memory to its own balloon driver, potentially allowing dom0 to >> expand up to take all physical memory. >> >> However, this may caused bad side-effects if your system memory is much >> larger than your dom0_mem, especially if you use a 32-bit dom0. I may >> need to add a kernel command line option to limit the max initial >> balloon size to mitigate this... > I would call this dom0 functionality a bug. I think both Citrix > and Oracle use dom0_mem as a normal boot option for every > installation and, while I think both employ heuristics to choose > a larger dom0_mem for larger physical memory, I don't think it > grows large enough for, say, >256GB physical memory, to accommodate > the necessarily large number of page tables. > > So, I'd vote for NOT allowing dom0 to balloon up to physical > memory if dom0_mem is specified, and possibly a kernel command > line option that allows it to grow beyond. Or, possibly, no > option and never allow dom0 memory to grow beyond dom0_mem > unless (possibly) it grows with hot-plug. Yes, its a bit of a problem. The trouble is that the kernel can't really distinguish the two cases; either way, it sees a Xen-supplied xen_start_info->nr_pages as the amount of initial memory available, and an E820 table referring to more RAM beyond that. I guess there are three options: 1. add a "xen_maxmem" (or something) kernel parameter to override space specified in the E820 table 2. ignore E820 if its a privileged domain 3. only allow extra memory up to a certain ratio of the base memory (8x? 16x? 32x?) I think the third is probably the simplest and least hacky, as it directly addresses the underlying issue (and prevents domU mishaps as well). J