From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dan Magenheimer Subject: Re: can not use all available memory Date: Mon, 26 Nov 2012 12:37:29 -0800 (PST) Message-ID: References: <1353674039.16973.3.camel@zakaz.uk.xensource.com> <20121123124115.GW8912@reaktio.net> <1353674827.16973.4.camel@zakaz.uk.xensource.com> <20121123125503.GX8912@reaktio.net> <1353676553.16973.12.camel@zakaz.uk.xensource.com> <20121123132920.GY8912@reaktio.net> <1353677623.16973.13.camel@zakaz.uk.xensource.com> <1545900898.20121126192533@eikelenboom.it> <672d364a-6d34-4c89-b0fb-5af989877117@default> <20121126202411.GA8912@reaktio.net> Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <20121126202411.GA8912@reaktio.net> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: =?iso-8859-1?B?UGFzaSBL5HJra+RpbmVu?= Cc: Sander Eikelenboom , xen-devel@lists.xensource.com, Alexander Bienzeisler , Ian Campbell List-Id: xen-devel@lists.xenproject.org > From: Pasi K=E4rkk=E4inen [mailto:pasik@iki.fi] > Subject: Re: [Xen-devel] can not use all available memory > = > On Mon, Nov 26, 2012 at 12:20:40PM -0800, Dan Magenheimer wrote: > > > From: Sander Eikelenboom [mailto:linux@eikelenboom.it] > > > Subject: Re: [Xen-devel] can not use all available memory > > > > > > Monday, November 26, 2012, 5:58:28 PM, you wrote: > > > > > > >> From: Ian Campbell [mailto:Ian.Campbell@citrix.com] > > > >> Sent: Friday, November 23, 2012 6:34 AM > > > >> To: Pasi K=E4rkk=E4inen > > > >> Cc: xen-devel@lists.xensource.com; Alexander Bienzeisler > > > >> Subject: Re: [Xen-devel] can not use all available memory > > > >> > > > >> On Fri, 2012-11-23 at 13:29 +0000, Pasi K=E4rkk=E4inen wrote: > > > >> > On Fri, Nov 23, 2012 at 01:15:53PM +0000, Ian Campbell wrote: > > > >> > > On Fri, 2012-11-23 at 12:55 +0000, Pasi K=E4rkk=E4inen wrote: > > > >> > > > > > >> > > > > This has been discussed at length on the list before, plea= se check the > > > >> > > > > archives. > > > >> > > > > > > > >> > > > > > > >> > > > I tried googling quickly but I didn't find anything relevant= .. > > > >> > > > > > >> > > I found > > > >> > > http://web.archiveorange.com/archive/v/zCKz5T3PLvtyZDSPQc9i > > > >> > > in a matter of seconds, then: > > > >> > > http://lists.xen.org/archives/html/xen-devel/2011-11/msg01415.= html > > > >> > > http://lists.xen.org/archives/html/xen-users/2012-05/msg00139.= html > > > >> > > http://lists.xen.org/archives/html/xen-users/2012-05/msg00146.= html > > > >> > > > > > >> > > > To me this behaviour still seems wrong. What's the point of = autoballoon=3D1 trying to > > > >> > > > balloon down dom0 if the hypervisor already has enough free = memory for the VM ? > > > >> > > > > > > >> > > > In this case: > > > >> > > > - dom0_mem=3D2G > > > >> > > > - new VM to launch with size 16 GB. > > > >> > > > - Xen has 28 GB of free memory. > > > >> > > > > > > >> > > > So clearly there's no need to try to balloon down dom0.. > > > >> > > > > > >> > > Right, so don't set autoballoon then. > > > >> > > > > > >> > > > > >> > It's enabled as a default.. so many people hit this problem. > > > >> > > > > >> > > > > >> > > > not-yet-implemented check/feature in xl, or a bug? > > > >> > > > > > >> > > Neither, it is the intended behaviour of xl autoballoon, this = option > > > >> > > means exactly "take the required memory from dom0". > > > >> > > > > > >> > > > > >> > http://xenbits.xen.org/docs/4.2-testing/man/xl.conf.5.html > > > >> > > > > >> > "autoballoon=3DBOOLEAN > > > >> > > > > >> > If disabled then xl will not attempt to reduce the amount of > > > >> > memory assigned to domain 0 in order to create free memory when > > > >> > starting a new domain. You should set this if you use the dom0_m= em > > > >> > hypervisor command line to reduce the amount of memory given to = domain > > > >> > 0 by default. > > > >> > > > > >> > Default: 1" > > > >> > > > > >> > > > > >> > I think we should modify that to say "You should set autoballoon= =3D0 if you use the dom0_mem > > > >> hypervisor command line .." > > > >> > At least I understood that text in the opposite way.. > > > >> > > > >> Yes, there should be a s/set/unset/ in there I think. > > > >> > > > >> > Also: What happens if you have autoballoon=3D1 and you start som= e VMs, then stop the VMs, > > > >> > so most of the memory is now free in Xen.. > > > >> > > > >> xl balloons dom0 back up when it destroy domains with autoballoon= =3D1. > > > >> > > > >> > and then you try to start a big VM ? > > > >> > Aren't you going to hit the same problem as in this thread? > > > > > > > Hmmm... this behavior and default may make sense for the Citrix > > > > memory model (single machine, dom0 is "the user" so you want it > > > > to always hold most of physical RAM not used by guests). But > > > > probably not so for a more cloud-like memory model. > > > > > > > Is there any (easy) way to force autoballoon=3D0 if the hypervisor > > > > dom0_mem boot option is specified? Or is there some reasonably > > > > sane case I am missing where a user would want both dom0_mem > > > > and autoballoon=3D1? > > > > > > > Oracle VM always boots servers with dom0_mem=3D set so (if/when > > > > OVM switches to use xl), OVM will always set autoballoon off. > > > > So it's the large number of non-Citrix-non-Oracle Xen-as-a-service > > > > providers I am trying to help here. > > > > > > > > > > > > Hmm i was bitten by this 2 weeks ago, i found it a bit not intuitive = that: > > > - While the guest i was trying to start required less memory than was= freely available (according > to > > > xentop) outside of dom0 > > > - It would fail, because xl started to try to balloon dom0 down which= failed. > > > > Hi Sander -- > > > > I could be wrong (and I am confident someone will correct me if I am) b= ut > > I think this is because the Citrix memory model assumes there is an > > inference-driven policy engine for load-balancing memory across competi= ng > > virtual machines ("squeezed"). I suspect squeezed returns unallocated > > xen "free" memory to dom0. I forgot... it is called Dynamic Memory Control (DMC), not squeezed in the XenServer product. > > IMHO, such policy engines are good for demos and so salespeople can > > say "yes, this product supports memory overcommit" but Transcendent > > Memory goes quite a bit further (albeit not for guests with proprietary > > OS's). > = > Afaik XenServer/XCP and XenClient both use dom0_mem=3D option for Xen. Hmmm... looks like you are right. Dan