Re: Linux 2.6.29

Linus Torvalds
2009-03-27T00:54:51Z

On Thu, 26 Mar 2009, Linus Torvalds wrote:
>
> The only times tunables have worked for us is when they auto-tune.
>
> IOW, we don't have "use 35% of memory for buffer cache" tunables, we just
> dynamically auto-tune memory use. And no, we don't expect user space to
> run some "tuning program for their load" either.

IOW, what we could reasonably do is something along the lines of:

- start off with some reasonable value for max background dirty (per
block device) that defaults to something sane (quite possibly based on
simply memory size).

- assume that "foreground dirty" is just always 2* background dirty.

- if we hit the "max foreground dirty" during memory allocation, then we
shrink the background dirty value (logic: we never want to have to wait
synchronously)

- if we hit some maximum latency on writeback, shrink dirty aggressively
and based on how long the latency was (because at that point we have a
real _measure_ of how costly it is with that load).

- if we start doing background dirtying, but never hit the foreground
dirty even in dirty balancing (ie when a writer is actually _writing_,
as opposed to hitting it when allocating memory by a non-writer), then
slowly open up the window - we may be limiting too early.

.. add heuristics to taste. The point being, that if we do this based on
real loads, and based on hitting the real problems, then we might actually
be getting somewhere. In particular, if the filesystem sucks at writeout
(ie the limiter is not the _disk_, but the filesystem serialization), then
it should automatically also shrink the max dirty state.

The tunable then could become the maximum latency we accept or something
like that. Or the hysteresis limits/rules for the soft "grow" or "shrink"
events. At that point, maybe we could even find something that works for
most people.

Linus