Re: Proposed XENMEM_claim_pages hypercall: Analysis of problem and alternate solutions

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Mon, Jan 21, 2013 at 10:29:23AM +0000, Tim Deegan wrote:
> At 16:45 -0500 on 18 Jan (1358527542), Konrad Rzeszutek Wilk wrote:
> > On Thu, Jan 17, 2013 at 03:16:31PM +0000, Tim Deegan wrote:
> > > At 14:08 -0500 on 11 Jan (1357913294), Konrad Rzeszutek Wilk wrote:
> > > > But the solution to the hypercall failing are multiple - one is to 
> > > > try to "squeeze" all the guests to make space
> > > 
> > > AFAICT if the toolstack can squeeze guests up to make room then the
> > > reservation hypercall isn't necessary -- just use the squeezing
> > > mechanism to make sure that running VMs don't use up the memory you want
> > > for building new ones.
> > 
> > We might want to not do that until we have run out of options (this would
> > be a toolstack option to select the right choice). The other option is
> > to just launch the guest on another node.
> 
> Sure, I see that; but what I meant was: the reservation hypercall only
> makes any kind of sense if the toolstack can't squeeze the existing guests. 

OK. I am going to take the liberty here to assume that squeeze is setting
d->max_pages and kicking the guest to balloon down to some number.

> 
> If it can squeeze VMs, as part of that it must have some mechanism to
> stop them from immediately re-allocating all the memory as it frees it.
> So in the case where enough memory is already free, you just use that
> mechanism to protect it while you build the new VM.

Sure.
> 
> Or (since I get the impression that losing this allocation race is a
> rare event) you can take the optimistic route: after you've checked that
> enough memory is free, just start building the VM.  If you run out of
> memory part-way through, you can squeeze the other VMs back out so you can
> finish the job.

All of this revolves around 'squeezing' the existing guests from the
tool-stack side. And as such the options you enumerated are the right
way of fixing it. And also the ways Xapi are pretty good.

However that is not the problem we are trying to address. We do _not_ want
to squeeze the guest at all. We want to leave it up to the guest to
go and down as it sees fit. We just need to set the ceiling (at start
time, and this is d->max_pages), and let the guest increment/decrement
d->tot_pages as it sees fit. And while that is going on, still be able
to create new guests in parallel.