From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dan Magenheimer Subject: Re: Proposed new "memory capacity claim" hypercall/feature Date: Tue, 30 Oct 2012 09:33:16 -0700 (PDT) Message-ID: References: <234774e0-45d5-4a4e-99f1-4e4c3eaee615@default> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Keir Fraser , Jan Beulich Cc: Olaf Hering , Ian Campbell , Konrad Wilk , George Dunlap , George Shuklin , "Tim (Xen.org)" , xen-devel@lists.xen.org, Dario Faggioli , Kurt Hackel , Zhigang Wang , Ian Jackson List-Id: xen-devel@lists.xenproject.org > From: Keir Fraser [mailto:keir.xen@gmail.com] > Subject: Re: Proposed new "memory capacity claim" hypercall/feature > > On 30/10/2012 16:13, "Dan Magenheimer" wrote: > > >> Okay, so why is tmem incompatible with implementing claims in the toolstack? > > > > (Hmmm... maybe I could schedule the equivalent of a PhD qual exam > > for tmem with all the core Xen developers as examiners?) > > > > The short answer is tmem moves memory capacity around far too > > frequently to be managed by a userland toolstack, especially if > > the "controller" lives on a central "manager machine" in a > > data center (Oracle's model). The ebb and flow of memory supply > > and demand for each guest is instead managed entirely dynamically. > > I don't know. I agree that fine-grained memory management is the duty of the > hypervisor, but it seems to me that the toolstack should be able to handle > admission control. It knows how much memory each existing guest is allowed > to consume at max, > !!!!!!!!!!!how much memory the new guest requires!!!!!!!!!! > how much memory > the system has total... Isn't the decision then simple? A fundamental assumption of tmem is that _nobody_ knows how much memory a guest requires, not even the OS kernel running in the guest. If you have a toolstack that does know, please submit a paper to OSDI. ;-) If you have a toolstack that can do it for thousands of guests across hundreds of machines, please start up a company and allow me to invest. ;-) One way to think of tmem is as a huge co-feedback loop that estimates memory demand and deals effectively with the consequences of the (always wrong) estimate using very fine-grained adjustments AND mechanisms that allow maximum flexibility between guest memory demands while minimizing impact on the running guests. > Tmem should be fairly invisible to the toolstack, right? It can be invisible, as long as the toolstack doesn't either make the assumption that it controls every page allocated/freed by the hypervisor or make the assumption that a large allocation can be completed atomically. The first of those assumptions is what is generating all the controversy (George's worldview) and the second is the problem I am trying to solve with the "claim" hypercall/subop. And I'd like to solve it in a way that handles both tmem and non-tmem. Thanks, Dan