Re: [PATCH][1/2] adjust dirty threshold for lowmem-only mappings

[Andrea Arcangeli <andrea@suse.de>]

On Sat, Dec 25, 2004 at 12:59:10PM -0500, Rik van Riel wrote:
> 4) any memory that could be affected by the swap token (process
>    text, data, stack, ...) is allocated with __GFP_HIGHMEM, so
>    that all lives in the highmem zone with 2.5GB free
> 5) since dd is not being paged out at all, and can dirty memory
>    without limit, the VM gets backed into a corner and will
>    trigger an OOM kill - even though most of lowmem is simply
>    dirty page cache

This shouldn't happen of course, and it's a bit hard to see how can it
work fine for 23 hours and break at the 24th hour since it's quite a
repetitive algorithm. (sure it could be a race or the algorithm being
very fragile, but I can't reproduce problems here)

Plus doing cp /dev/zero . should be even worse since it also fills up
the highmem.

Are you sure cron isn't spawning something big?

Anyway my point is that swap-token is _proven_ to trigger suprious oom
kills, so if you could just reproduce once with Con's patch applied and
default sysctl value, then you would provide the proof it's unrelated.

I agree with your reasoning, I think you're right, but I'd like to be
sure we're not missing something. There are definitely other reports
where the ignore-token patch wasn't enough and Con's patch fixed it.

I also recommend you to keep vmstat in the background, in my experience
swap token was filling all swap with freeable swapcache (but it wasn't
freeable due the referenced ++ that swap-token does), and then the oom
killer was invoked despite all that freeable swapcache.

So on a computer that had plenty of lowmem and highmem free, in seconds
it would run out of memory with all swap allocated.

I agree dd shouldn't be enough, but the 1 day variable may be just some
big cron task that we didn't put into the equation.

So I still would like to see a `vmstat 1` before/after the killing, and
to hear the confirmation that Con's patch doesn't help.

The only thing I can imagine being wrong with `cp /dev/zero /dev/sd?`
while working fine on `cp /dev/zero .`, are the write throttling levels
that might be taking highmem into account while they really cannot take
highmem into account, I mean nr_free_buffer_pages must be used by the
write throttling and not nr_free_pages, but I'd be surprised if this
wasn't correct. You may want to check this bit just in case. If this is
correct then doing cp /dev/zero . should fail too, no? I for sure can't
reproduce here, and by your same arguments about the highmem levels, it
shouldn't matter how much ram I have (I've 1G). The less ram I have, the
worse it should behave.

Without more data and without being able to reproduce I can't be more
helpful than this.

Thanks.