From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dan Magenheimer Subject: Re: Proposed new "memory capacity claim" hypercall/feature Date: Mon, 5 Nov 2012 14:58:52 -0800 (PST) Message-ID: <8bec0df2-cf4b-4dcf-bbb8-121357a29d52@default> References: <50939A1502000078000A5F61@nat28.tlf.novell.com> <7481128d-3f65-4cc3-ad96-1d4e9cd25094@default> <20121104203532.GA11377@ocelot.phlegethon.org> <26f55aab-7523-4681-9a61-a5e5740d43a9@default> <1352111399.25014.101.camel@hastur.hellion.org.uk> <5517445d-ed2e-4592-9a2b-98fc612f1ae8@default> <1352154288.7253.25.camel@hastur.hellion.org.uk> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <1352154288.7253.25.camel@hastur.hellion.org.uk> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Ian Campbell Cc: Olaf Hering , "Keir (Xen.org)" , Konrad Wilk , George Dunlap , Ian Jackson , "Tim (Xen.org)" , xen-devel@lists.xen.org, George Shuklin , DarioFaggioli , Jan Beulich , Kurt Hackel , Zhigang Wang List-Id: xen-devel@lists.xenproject.org > From: Ian Campbell [mailto:ian.campbell@citrix.com] > Subject: Re: Proposed new "memory capacity claim" hypercall/feature Hi Ian -- > On Mon, 2012-11-05 at 14:54 +0000, Dan Magenheimer wrote: > > > From: Ian Campbell [mailto:ian.campbell@citrix.com] > > > Sent: Monday, November 05, 2012 3:30 AM > > > To: Dan Magenheimer > > > Cc: Tim (Xen.org); Keir (Xen.org); Jan Beulich; Olaf Hering; George Dunlap; Ian Jackson; George > > > Shuklin; DarioFaggioli; xen-devel@lists.xen.org; Konrad Wilk; Kurt Hackel; Mukesh Rathor; Zhigang > Wang > > > Subject: Re: Proposed new "memory capacity claim" hypercall/feature > > > > > > On Mon, 2012-11-05 at 00:23 +0000, Dan Magenheimer wrote: > > > > There is no "free up enough memory on that host". Tmem doesn't start > > > > ballooning out enough memory to start the VM... the guests are > > > > responsible for doing the ballooning and it is _already done_. The > > > > machine either has sufficient free+freeable memory or it does not; > > > > > > How does one go about deciding which host in a multi thousand host > > > deployment to try the claim hypercall on? > > > > I don't get paid enough to solve that problem :-) > > > > VM placement (both for new domains and migration due to > > load-balancing and power-management) is dependent on a > > number of factors currently involving CPU utilization, > > SAN utilization, and LAN utilization, I think using > > historical trends on streams of sampled statistics. This > > is very non-deterministic as all of these factors may > > vary dramatically within a sampling interval. > > > > Adding free+freeable memory to this just adds one more > > such statistic. Actually two, as it is probably best to > > track free separately from freeable since a candidate > > host that has enough free memory should have preference > > over one with freeable memory. > > > > Sorry if that's not very satisfying but anything beyond that > > meager description is outside of my area of expertise. > > I guess I don't see how your proposed claim hypercall is useful if you > can't decide which machine you should call it on, whether it's 10s, 100s > or 1000s of hosts. Surely you aren't suggesting that the toolstack try > it on all (or even a subset) of them and see which sticks? > > By ignoring this part of the problem I think you are ignoring one of the > most important bits of the story, without which it is very hard to make > a useful and informed determination about the validity of the use cases > you are describing for the new call. I'm not ignoring it at all. One only needs to choose a machine and be prepared that the machine will (immediately) answer "sorry, won't fit". It's not necessary to choose the _optimal_ fit, only a probable one. Since failure is immediate, trying more than one machine (which should happen only rarely) is not particularly problematic, though I completely agree that trying _all_ of them might be. The existing OracleVM Manager already chooses domain launch candidates and load balancing candidates based on sampled CPU/SAN/LAN data, which is always stale but still sufficient as a rough estimate of the best machine to choose. Beyond that, I'm not particularly knowledgeable about the details and, even if I were, I'm not sure if the details are suitable for a public forum. But I can tell you that it has been shipping for over a year and here's some of what's published... look for DRS and DPM. http://www.oracle.com/us/technologies/virtualization/ovm3-whats-new-459313.pdf Dan