Re: [PATCH] RFC: initial libxl support for xenpaging

[George Dunlap <george.dunlap@citrix.com>]

On Mon, 2012-02-20 at 15:38 +0000, Olaf Hering wrote:
> On Mon, Feb 20, George Dunlap wrote:
> 
> > xl mem-set domain M
> >  xenpaging off: Set balloon target to M
> >  xenpaging on: Set paging target to M
> >  xenpaging delay: Set balloon target to M, and wait for actual memory
> > to reach M.  If it hasn't reached it by $paging_delay seconds, set
> > balloon target to M.
> 
> The tristate instead of a boolean is not really needed as well.
> 
> Right now a reduction of the balloon target depends on how fast the
> guests balloon driver can claim memory from the guest OS. This means it
> could take an infinite amount of time to reach the requested target.
> With paging the request to reduce the "footprint" can be reached as fast
> as xenpaging can page-out gfns (and page-in busy gfns).
> 
> So having a delay or not seems to depend on how mem-set is supposed to
> react, either now or at an (kind of) undefined time in the future. 
> 
> If "now" is not the desired mode then xenpaging can slowly work toward a
> new lower target by calling its evict_pages() with low numbers and and
> an adjustable delay between calls.

I don't think this is a good option.  Consider the following cases:
1. The guest is cooperative and the balloon driver is active, but it
will take 30s to reach the target.  
2. The guest is uncooperative, or the balloon driver is inactive /
broken / stuck / unable to get pages. 

In case 1, the paging driver will end up paging a bunch of pages out for
30s, only to page them back in again after the balloon driver reaches
its target.  Also, you're basically guaranteed to hit the double-paging
problem: the pager will find an old page and page it out; then in the
very near future, the guest OS will decide to page *that very page* out,
touch it (causing the pager to page it back in), then write it out to
disk itself -- causing at least 3 page writes per page paged out.  This
will unnecessarily slow the system down (and the balloon driver down)
and waste IO bandwidth.  The degree to which the pager messes things up
will depend on how fast you make it.  Fast => messes up a lot slow =>
doesn't mess much up.  So slow is good in this case.

In case 2, the time it takes to reach the target will depend entirely on
the rate at which you page things out.  Slow => takes a long time, fast
=> happens relatively quicly.  So slow is bad in this case.

So no matter what we choose, something is really bad.

Under the "delay" option, 1 will behave optimally -- no wasted I/O, no
double paging; and 2 will take a fixed amount of time before going
full-bore; so it is, I believe, better in both cases.

You don't have to implement the "delay" option if you don't want to; as
long as we expose the low-level controls, the administrator can do the
"fall-back" himself, and then someone (possibly me) can implement
something like that in the future (either before or after the 4.2
release).  

However, there should be a way that paging can be on but "xl mem-set"
will set the balloon target.  If we have "paging=manual", that could
mean, "I'll start the pager daemon, but you'll have to call the xl
mem-set-paging-target yourself."

 -George