Re: Proposed new "memory capacity claim" hypercall/feature

[Dan Magenheimer <dan.magenheimer@oracle.com>]

> From: Tim Deegan [mailto:tim@xen.org]
> Sent: Monday, October 29, 2012 4:36 PM
> To: Dan Magenheimer
> Cc: Keir (Xen.org); Jan Beulich; George Dunlap; Olaf Hering; Ian Campbell; Konrad Wilk; xen-
> devel@lists.xen.org; George Shuklin; Dario Faggioli; Kurt Hackel; Ian Jackson; Zhigang Wang; Mukesh
> Rathor
> Subject: Re: Proposed new "memory capacity claim" hypercall/feature
> 
> > The hypervisor must also enforce some semantics:  If an allocation
> > occurs such that a domain's tot_phys_pages would equal or exceed
> > d.tot_claimed_pages, then d.tot_claimed_pages becomes "unset".
> > This enforces the temporary nature of a claim:  Once a domain
> > fully "occupies" its claim, the claim silently expires.
> 
> Why does that happen?  If I understand you correctly, releasing the
> claim is something the toolstack should do once it knows it's no longer
> needed.

Hi Tim --

Thanks for the feedback!

I haven't thought this all the way through yet, but I think this
part of the design allows the toolstack to avoid monitoring the
domain until "total_phys_pages" reaches "total_claimed" pages,
which should make the implementation of claims in the toolstack
simpler, especially in many-server environments.
 
> > In the case of a dying domain, a XENMEM_release operation
> > is implied and must be executed by the hypervisor.
> >
> > Ideally, the quantity of unclaimed memory for each domain and
> > for the system should be query-able.  This may require additional
> > memory_op hypercalls.
> >
> > I'd very much appreciate feedback on this proposed design!
> 
> As I said, I'm not opposed to this, though even after reading through
> the other thread I'm not convinced that it's necessary (except in cases
> where guest-controlled operations are allowed to consume unbounded
> memory, which frankly gives me the heebie-jeebies).

A really detailed discussion of tmem would probably be good but,
yes, with tmem, guest-controlled* operations can and frequently will
absorb ALL physical RAM.  However, this is "freeable" (ephemeral)
memory used by the hypervisor on behalf of domains, not domain-owned
memory.

* "guest-controlled" I suspect is the heebie-jeebie word... in
  tmem, a better description might be "guest-controls-which-data-
  and-hypervisor-controls-how-many-pages"
 
> I think it needs a plan for handling restricted memory allocations.
> For example, some PV guests need their memory to come below a
> certain machine address, or entirely in superpages, and certain
> build-time allocations come from xenheap.  How would you handle that
> sort of thing?

Good point.  I think there's always been some uncertainty about
how to account for different zones and xenheap... are they part of the
domain's memory or not?  Deserves some more thought...  if
you can enumerate all such cases, that would be very helpful
(and probably valuable long-term documentation as well).

Thanks,
Dan