RE: [PATCH] [linux-2.6.39.x for xen] tmem: self-ballooning and frontswap-selfshrinking

[Dan Magenheimer <dan.magenheimer@oracle.com>]

> > Is there any particular reason this (all) needs to be in the kernel at
> > all? Can a userspace daemon, using (possibly new) statistics exported
> > via /sys and /proc plus the existing balloon interfaces not implement
> > much the same thing? One nice advantage of doing that is that a
> > userspace daemon can more easily implement complex (or multiple)
> > algorithms, tune them etc. If there is a good reason for this to be in
> > kernel I think it should be expanded upon in the commit message.
> >
> > Ian.
> 
> I'm agree with Ian. Some time ago i write daemon that balloon up/down
> memory, most of the time it work's fine. But with specific software -
> for example mongodb and java aplications - algorithm needs changing and
> after some time i go to idea, that userspace can acts as ulatencyd -
> kernel provide interface to calculate and to up/down memory (in this
> case kernel need to provide frontswap shrink...) userspace daemon use
> statistics and interface can apply various memory calcalation patterns
> to various workloads and software used in server.

It certainly *can* be done.  I demonstrated that at Xen Summit 2008 
and the userland code has been in the Xen tools/misc tree
since Fall 2008.

I found some time ago that aggressive ballooning with widely
varying workload frequently caused OOMs that went away when
the selfballooning and frontswap-selfshrinking code was put in
the kernel, presumably making it more responsive.  The often fatal
nature of OOMs make it difficult to debug... it may be
possible to change the userspace code, for example, to sample
and adjust at a higher frequency... IIRC I just went with 
doing it in the kernel because it was what worked.

The real objective is for the kernel to be less "selfish" with
its memory.  Ideally, there should be a generic mechanism for
the kernel to "surrender" memory that it can live without...
and perhaps it plugs into the balloon driver, perhaps hotplug,
or perhaps something else.  However "memory it can live
without" is as vague as (and essentially the complement
of) "working set".

The use of "committed_as" is really just one estimator of
how "lean" the kernel could be and it happened to be exposed
in userspace.  It's also very aggressive, probably too aggressive
unless tmem is running.  I suspect there are better algorithms...
and those algorithms may not have all data exposed in userspace.

Dan