Re: mmap dirty limits on 32 bit kernels (Was: [BUG] New Kernel Bugs)

Linus Torvalds
2007-11-15T21:26:35Z

On Thu, 15 Nov 2007, Linus Torvalds wrote:
>
> The problem with HIGHMEM is that it causes various metadata (dentries,
> inodes, page struct tables etc) to eat up memory "prime real estate" under
> the same kind of conditions that also dirty a lot of memory. So the reason
> we disallow HIGHMEM from dirty limits is only *partly* the per-device or
> mapping DMA limits, and to a large degree the fact that non-highmem memory
> is special in general, and it is usually the non-highmem areas that are
> constrained - and need to be protected.

Final note on this (promise):

I'd really be very interested to hear if the patch I *do* think makes
sense (ie the removal of the old "unmapped_ratio" logic) actually already
solves most of Bron's problems.

It may well be that that unmapped_ratio logic effectively undid the system
configuration changes that Bron has done. It doesn't matter if Bron has

>From our sysctl.conf:
# This should help reduce flushing on Cache::FastMmap files
vm.dirty_background_ratio = 50
vm.dirty_expire_centisecs = 9000
vm.dirty_ratio = 80
vm.dirty_writeback_centisecs = 3000

if it turns out that the "unmapped_ratio" logic turns the 80% back down to
5%.

It may well be that 80% of the non-highmem memory is plenty good enough!

Sure, older kernels allowed even more of memory to be dirty (since they
didn't count dirty mappings at all), but we may have a case where the fact
that we discount the HIGHMEM stuff isn't the major problem in itself, and
that the dirty_ratio sysctl should be ok - but just gets screwed over by
that unmapped_ratio logic.

So Bron, if you can test that patch, I'd love to hear if it matters. It
may not make any difference (maybe you don't actually trigger the
unmapped_ratio logic at all), but I think it has the potential for being
totally broken for you.

People that don't change the dirty_ratio from the default values would
generally never care, because the default dirty-ratio is *already* so low
that even if the unmapped_ratio logic triggers, it won't much matter!

Linus