Suspected bug in shrink_caches()

Suspected bug in shrink_caches()

[Sasha Pachev]

Rik:

I am quite sure there is still a bug in shrink_caches(), at least there was 
one in 2.4.17-rc2. I have not tried later releases, but there is nothing in 
the changelog about the fixes anywhere near that area of the code, so I have 
to assume the problem is still there.

Here is how one can expose the bug:

 - disable kswapd and turn off all swap for simplicity
 - run two applications concurrently, one cache intenstive ( eg find / or 
simply create/initialize/read a large file), and the other RAM-intensive - 
allocate a large block and initialize it

When we get to the point where free memory starts running low, even though we 
may have something like 100 MB of cache, shrink_caches() fails to free up 
enough memory, which triggers the evil oom killer. Obviously, in the above 
situation the correct behaviour is to go on cache diet before considering the 
murders.

-- 
MySQL Development Team
For technical support contracts, visit https://order.mysql.com/
   __  ___     ___ ____  __ 
  /  |/  /_ __/ __/ __ \/ /   Sasha Pachev <sasha@mysql.com>
 / /|_/ / // /\ \/ /_/ / /__  MySQL AB, http://www.mysql.com/
/_/  /_/\_, /___/\___\_\___/  Provo, Utah, USA
       <___/                  

Re: Suspected bug in shrink_caches()

[José Luis Domingo López]

On Wednesday, 02 January 2002, at 22:38:40 -0700,
Sasha Pachev wrote:

> I am quite sure there is still a bug in shrink_caches(), at least there was 
> one in 2.4.17-rc2. I have not tried later releases, but there is nothing in 
> the changelog about the fixes anywhere near that area of the code, so I have 
> to assume the problem is still there.
> [...]
> When we get to the point where free memory starts running low, even though we 
> may have something like 100 MB of cache, shrink_caches() fails to free up 
> enough memory, which triggers the evil oom killer. Obviously, in the above 
> situation the correct behaviour is to go on cache diet before considering the 
> murders.
> 
I can confirm this behaviour in my own machine. 128 MB of RAM, swap
space turned off, machine running KDE 2.2.x, Konqueror, rxvt's, gkrellm,
xmms and some daemons, and mighty Mozilla :). At this point there are
still a couple of MB free, a couple buffered and aproximately 40 MB
in caches.

Trying to load OpenOffice641 the kernel feels it's OOM time, and kills
something, usually Mozilla (as it should). However, "watch cat
/proc/meminfo" reports about 30 MB in caches when OOM kicks in. Any
attempt to "tune" parameters under "/proc/sys/vm/" changes nothing.
Attempted changes include:
echo "0 0" > /proc/sys/vm/pagetable_cache

All this is with plain 2.4.17, no patches applied. Another thing I have
observed is caches growing "too" much, but this is more of a subjective
feeling than anything else.

Maybe unrelated, maybe not, is the fact that in 2.4.17 I still see short
"freezes" in interactive response after doing intensive disk access
(untarring something, finishing some dd'ing, rebuilding Debian package
database, etc.). Both on medium-end hardware (128 MB RAM, PIII 600) and
low-end hardware (64 MB RAM, P166). Vanilla 2.4.17 on both, with ext2 as
the only filesystem used in mounted partitions.

-- 
José Luis Domingo López
Linux Registered User #189436     Debian Linux Woody (P166 64 MB RAM)
 
jdomingo AT internautas DOT   org  => Spam at your own risk

Re: Suspected bug in shrink_caches()

[Andrew Morton]

José Luis Domingo López wrote:
> 
> On Wednesday, 02 January 2002, at 22:38:40 -0700,
> Sasha Pachev wrote:
> 
> > I am quite sure there is still a bug in shrink_caches(), at least there was
> > one in 2.4.17-rc2. I have not tried later releases, but there is nothing in
> > the changelog about the fixes anywhere near that area of the code, so I have
> > to assume the problem is still there.
> > [...]
> > When we get to the point where free memory starts running low, even though we
> > may have something like 100 MB of cache, shrink_caches() fails to free up
> > enough memory, which triggers the evil oom killer. Obviously, in the above
> > situation the correct behaviour is to go on cache diet before considering the
> > murders.
> >
> I can confirm this behaviour in my own machine. 128 MB of RAM, swap
> space turned off, machine running KDE 2.2.x, Konqueror, rxvt's, gkrellm,
> xmms and some daemons, and mighty Mozilla :). At this point there are
> still a couple of MB free, a couple buffered and aproximately 40 MB
> in caches.

The current VM gets really unhappy when you push it hard if there's no
swap available.

> ...
> 
> Maybe unrelated, maybe not, is the fact that in 2.4.17 I still see short
> "freezes" in interactive response after doing intensive disk access
> (untarring something, finishing some dd'ing, rebuilding Debian package
> database, etc.). Both on medium-end hardware (128 MB RAM, PIII 600) and
> low-end hardware (64 MB RAM, P166). Vanilla 2.4.17 on both, with ext2 as
> the only filesystem used in mounted partitions.

Please test
http://www.zip.com.au/~akpm/linux/2.4/2.4.18-pre1/read-latency2.patch

-