Re: domain creation vs querying free memory (xend and xl)

[Dan Magenheimer <dan.magenheimer@oracle.com>]

> From: Olaf Hering [mailto:olaf@aepfle.de]
> Subject: Re: [Xen-devel] domain creation vs querying free memory (xend and xl)
> 
> On Thu, Oct 04, Dan Magenheimer wrote:
> 
> > > From: Olaf Hering [mailto:olaf@aepfle.de]
> > > Subject: Re: [Xen-devel] domain creation vs querying free memory (xend and xl)
> > >
> > > On Mon, Oct 01, Dan Magenheimer wrote:
> > >
> >
> > Hi Olaf --
> >
> > Thanks for the reply.
> >
> > > domain. All of this needs math, not locking.
> > >  :
> > > As IanJ said, the memory handling code in libxl needs such a feature to
> > > do the math right. The proposed handling of
> > > sharing/paging/ballooning/PoD/tmem/... in libxl is just a small part of
> > > it.
> >
> > Unfortunately, as you observe in some of the cases earlier in your reply,
> > it is more than a math problem for libxl... it is a crystal ball problem.
> > If xl launches a domain D at time T and it takes N seconds before it has
> > completed asking the hypervisor for all of the memory M that D will require
> > to successfully launch, then xl must determine at time T the maximum memory
> > allocated across all running domains for the future time period between
> > T and T+N.  In other words, xl must predict the future.
> 
> I think xl can predict it, if it takes the target of all domains into
> account.  Certainly not down to a handful pages, it would be good enough
> to know if the calculated estimate of free memory is good for the new
> guest and its specific memory targets.

Well I don't know enough about the page-sharing implementation but
it's not hard with tmem to synthesize a workload where the amount of
free memory is half of RAM at time T and there is no RAM left at all at
time T+(N/2) and three quarters of RAM is free at time T+N.  That would
be very hard for xl to predict.  I expect that dramatic changes like
this might be harder with page-sharing but not impossible.
 
> > Clearly this is impossible especially when page-sharing is not communicating
> > its dynamic allocations (e.g. due to page-splitting) to libxl, and tmem
> > is not communicating allocations resulting from multiple domains
> > simultaenously making tmem hypercalls to libxl, and PoD is not communicating
> > its allocations to libxl, and in-guest-kernel selfballooning is not communicating
> > allocations to libxl.  Only the hypervisor is aware of every dynamic allocation
> > request.
> 
> The hypervisor can not predict the future either, and it has even less
> info about the individual targets of each domain.

The point is the hypervisor doesn't need to predict the future
and doesn't need to know the individual targets.  It just
acts on allocation requests and, with the proposed design,
on reservation requests.

> > Does that make sense?
> 
> It does, but:
> If xl reserves the memory in its own "virtual allocator", or if Xen gets
> such functionality, does not really matter, as long as its known how much
> exactly needs to be allocated. I think that part is missing.

I agree, though I think the only constraint is that the domain
must be capable of booting.  So if xl always requests a reservation
of "mem=", I would think that should always work.