Re: Proposed new "memory capacity claim" hypercall/feature

[Dan Magenheimer <dan.magenheimer@oracle.com>]

> From: Keir Fraser [mailto:keir.xen@gmail.com]
> Subject: Re: Proposed new "memory capacity claim" hypercall/feature
> 
> On 30/10/2012 16:13, "Dan Magenheimer" <dan.magenheimer@oracle.com> wrote:
> 
> >> Okay, so why is tmem incompatible with implementing claims in the toolstack?
> >
> > (Hmmm... maybe I could schedule the equivalent of a PhD qual exam
> > for tmem with all the core Xen developers as examiners?)
> >
> > The short answer is tmem moves memory capacity around far too
> > frequently to be managed by a userland toolstack, especially if
> > the "controller" lives on a central "manager machine" in a
> > data center (Oracle's model).  The ebb and flow of memory supply
> > and demand for each guest is instead managed entirely dynamically.
> 
> I don't know. I agree that fine-grained memory management is the duty of the
> hypervisor, but it seems to me that the toolstack should be able to handle
> admission control. It knows how much memory each existing guest is allowed
> to consume at max,
>   !!!!!!!!!!!how much memory the new guest requires!!!!!!!!!!
> how much memory
> the system has total... Isn't the decision then simple?

A fundamental assumption of tmem is that _nobody_ knows how much memory
a guest requires, not even the OS kernel running in the guest.  If you
have a toolstack that does know, please submit a paper to OSDI. ;-)
If you have a toolstack that can do it for thousands of guests across
hundreds of machines, please start up a company and allow me to invest. ;-)

One way to think of tmem is as a huge co-feedback loop that estimates
memory demand and deals effectively with the consequences of the (always
wrong) estimate using very fine-grained adjustments AND mechanisms that
allow maximum flexibility between guest memory demands while minimizing
impact on the running guests.

> Tmem should be fairly invisible to the toolstack, right?

It can be invisible, as long as the toolstack doesn't either make
the assumption that it controls every page allocated/freed by the
hypervisor or make the assumption that a large allocation can be
completed atomically.  The first of those assumptions is what is
generating all the controversy (George's worldview) and the second
is the problem I am trying to solve with the "claim" hypercall/subop.
And I'd like to solve it in a way that handles both tmem and non-tmem.

Thanks,
Dan