Re: [PATCH] strict VM overcommit accounting for 2.4.32/2.4.33-pre1

[Willy Tarreau <willy@w.ods.org>]

On Sat, Dec 31, 2005 at 05:02:20PM +0300, Al Boldi wrote:
> Willy Tarreau wrote:
(...)
> > As shown in my previous mail, it allows malloc() to return NULL. I've
> > also successfully verified that it allows mmap() to fail if there is
> > not enough memory. I disabled swap, and set the overcommit_ratio to 95
> > and could not kill the system. Above this, it becomes tricky. At 97, I
> > see the last malloc() calls take a very long time, and at 98, the
> > system still hangs. But 95% without swap seems stable here.
> 
> Thanks, for confirming this!  And I agree that this patch and 2.6 offer an 
> important and necessary workaround to inhibit OOM-killer, but it's no more 
> than a workaround.
> 
> And so the question remains:  Why should overcommit come into play at all 
> when dealing with physical RAM only?

It is very important when you have many processes wasting lots of unused
memory. For instance, when firefox allocates 10 MB and uses only 3, then
the remaining 7 MB can be used by another process. But if finally firefox
tries to use them, and there is no memory left, then chances are that some
processes will be killed. I believe the same problem happens after a fork()
because data gets COWed but I'm not certain about this.

I'd bet that using a heavy GUI under X with no swap an overcommit_ratio
set around 95%, you could get occasionnal malloc() failures. But once
again, I may be wrong.

> Shouldn't it be possible to disable overcommit completely, thus giving kswapd 
> a break from running wild trying to find something to swap/page, which is 
> the reason why the system gets unstable going over 95% in your example.

I think it's going unstable above 95% because there are lots of other
areas consuming memory. I don't know for example if dentry caches,
network buffers, various hash tables, etc... are accounted for.

> Thanks!
> 
> --
> Al

Willy