Re: [PATCH][1/2] adjust dirty threshold for lowmem-only mappings

Rik van Riel
2004-12-25T17:59:34Z

On Sat, 25 Dec 2004, Andrea Arcangeli wrote:
> On Fri, Dec 24, 2004 at 05:12:32PM -0500, Rik van Riel wrote:
>> The process 'dd', and all the other processes, live in
>> the highmem zone, which has 2.5GB of memory free. Now
>> tell me again why you think the swap token has any
>> relevance to those 950MB of pagecache that is filling
>> up lowmem ?
>
> If 2.5G of ram is really free, then how can the oom killer be invoked in
> the first place? If that happens it means you're under a lowmem
> shortage, something you apparently ruled out when you said
> lowmem_reserve couldn't help your workload.

Let me explain a 3rd time:

1) run dd if=/dev/zero of=/dev/hdaN on a system with 4GB RAM

2) the pagecache mapping for /dev/hdaN can only come from
lowmem, of which we have roughly 900MB

3) the dirty_limit is 40% of 4GB, or roughly 1.6GB - the dd
from (1) will not throttle itself at all, but will just
fill up lowmem without limitation

4) any memory that could be affected by the swap token (process
text, data, stack, ...) is allocated with __GFP_HIGHMEM, so
that all lives in the highmem zone with 2.5GB free

5) since dd is not being paged out at all, and can dirty memory
without limit, the VM gets backed into a corner and will
trigger an OOM kill - even though most of lowmem is simply
dirty page cache

6) an unpatched 2.6.10-rc kernel will OOM kill in minutes on
a test system here

6) Andrew's total_pages patch, marcelo's vm-writeout-throttle patch
and my two patches improve the situation a lot, and the OOM kill
takes a day or so to be triggered

If you have any more questions as to why the bug happens, don't
hesitate to ask and I'll explain you why this problem happens.

--
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." - Brian W. Kernighan