Swapping in 2.6.10 and 2.6.11.11 on a desktop system

Swapping in 2.6.10 and 2.6.11.11 on a desktop system

[Alexander Gretencord]

Hi,

[1] With swappiness = 60 I either get swap hell (2.6.10) or the oom killer 
kicks in (2.6.11.11)

[2] I upgraded to 2.6.11.11 from 2.6.8.1 yesterday and tried to compile 
something. After some time I come back and the compile has aborted because 
the oom killer killed the compiler process. There is no additional use of 
swap space (although some applications that were also running could have been 
swapped out). There was a similar bugreport with this behaviour some time ago 
for 2.6.11.8 but that one included a swappiness value of 0, i got 60.

Then I tried 2.6.10. The oom bugs were gone, the compile finished over night. 
About an hour ago I started a very RAM consuming application and at 2/3 RAM 
usage I get swap hell. The system constantly swaps, yet ram usage _and_ swap 
usage stay about the same. Even setting swappiness to 0 from an ssh login 
(switching to a console does not work) did not help, only killing the 
consuming application.

I also applied the ck hard-swappiness Patch to my 2.6.10 and tested a bit 
further. Depending on the swappiness I get hellish swapping behaviour at 
different levels of RAM usage and even managed to get the oom killer to step 
in (at swappiness=20). I had about 250MB of RAM used and boom -> oom killer 
shoots down some bigger java processes. At swappiness=80 I get massive 
swapping with 250MB of RAM usage. When reaching that level, the fs cache goes 
up to 300MB while the applications that need the ram are left with about 120 
megs. Rest was buffers.

I'm really out of ideas. 2.6.8.1 with swappiness=0 is by far the best I've 
managed to get by now. With that I can use my system until RAM and swap are 
completely full. No swap hell and no oom killer yet. Maybe I just have to 
search the lkml a bit more to find something usable. If someone else already 
has, please tell me (and tell me why it's not in the mainline kernel 
already :))

I have 512MB of RAM and another 512MB of swap.

Please cc me in replies as I'm not on the list.


Alex

Re: Swapping in 2.6.10 and 2.6.11.11 on a desktop system

[Con Kolivas]

[-- Attachment #1: Type: text/plain, Size: 825 bytes --]

On Wed, 15 Jun 2005 00:53, Alexander Gretencord wrote:
> Hi,
>
> [1] With swappiness = 60 I either get swap hell (2.6.10) or the oom killer
> kicks in (2.6.11.11)
>
> [2] I upgraded to 2.6.11.11 from 2.6.8.1 yesterday and tried to compile
> something. After some time I come back and the compile has aborted because
> the oom killer killed the compiler process. There is no additional use of
> swap space (although some applications that were also running could have
> been swapped out). There was a similar bugreport with this behaviour some
> time ago for 2.6.11.8 but that one included a swappiness value of 0, i got
> 60.

> I have 512MB of RAM and another 512MB of swap.

Try the mapped watermark patch from -ck on 2.6.11*
http://ck.kolivas.org/patches/2.6/2.6.11/2.6.11-ck10/patches/mapped_watermark3.diff

Cheers,
Con

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Swapping in 2.6.10 and 2.6.11.11 on a desktop system

[Alexander Gretencord]

On Tuesday 14 June 2005 18:42, Con Kolivas wrote:
> Try the mapped watermark patch from -ck on 2.6.11*

Unfortunately this patch does not help either. The patch buys me time but then 
I get swapping at the 300MB mark. 2.6.8.1 with swappiness=0 swaps later than 
this...

These are 'free -m' statistics a minute after reaching the 300MB mark:

             total       used       free     shared    buffers     cached
Mem:           503        498          5          0         36        376
-/+ buffers/cache:         85        418
Swap:          494        227        266

Why does the kernel think that I need 400MB of disk cache when I start some 
memory hungry apps? Can't we have a hard limit on disk cache, like: "Don't 
use more than 100MB of disk cache".

The problem seems to be that instead of using a big disk cache when theres 
plenty of ram and reducing disk cache when applications need the ram, the 
disk cache shrinks until a magic watermark and then grows and grows until 
theres no ram left for the applications. At least thats the behaviour I am 
seeing.


Alex

P.S.: Please cc me as I'm not on the list.

Re: Swapping in 2.6.10 and 2.6.11.11 on a desktop system

[Con Kolivas]

[-- Attachment #1: Type: text/plain, Size: 594 bytes --]

On Wed, 15 Jun 2005 23:44, Alexander Gretencord wrote:
> On Tuesday 14 June 2005 18:42, Con Kolivas wrote:
> > Try the mapped watermark patch from -ck on 2.6.11*
>
> Unfortunately this patch does not help either. The patch buys me time but
> then I get swapping at the 300MB mark. 2.6.8.1 with swappiness=0 swaps
> later than this...

Try tweaking it with this:
echo 100 > /proc/sys/vm/mapped

If this tries so hard to avoid swap that you get an out-of-memory condition 
you may also have to disable the hard maplimit with this:
echo 0 > /proc/sys/vm/hardmaplimit

Cheers,
Con

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Swapping in 2.6.10 and 2.6.11.11 on a desktop system

[Alexander Gretencord]

On Thursday 16 June 2005 06:16, you wrote:
> echo 100 > /proc/sys/vm/mapped

Doesn't work either.

What I do to test this is the following:

Start X with KDE, start a Konqueror instance, a firefox, eclipse and vmware. 
This increases memory load to about 250MB. Then I start os inside the vmware 
virtual machine. This increases memory allocation because the guest operating 
system has to exist somewhere and generates IOs while the virtual disk is 
accessed. With mapped=100 it takes longer until the system begins to swap but 
once it has begun swapping I get this:

             total       used       free     shared    buffers     cached
Mem:           503        498          5          0         10        412
-/+ buffers/cache:         75        428
Swap:          494        230        264

It does not matter which patch I am using, once the magical "begin 
swapping"-mark is reached, real ram usage drops to a bare minimum and cache 
usage goes up. The IO cache is probably not even used as there are virtually 
no reusable parts. 

The problem is the kernel thinking in the wrong direction (freeing ram for 
disk cache). Maybe that's ok for a streaming server which uses 10MB of RAM 
for code and internal data and is constantly accessing the same 400MB of 
streaming video data. Or a webserver serving a big amount of static pages 
etc. but it is not ok for my desktop workload where I have about 400MB of 
data belonging to applications and IOs are not repeating very often (disk 
cache efficiency is probably very low).

Any idea why the kernel behaves like this and how I can get expected 
behaviour? Expected behaviour for a desktop would be:

Use as much disk cache as there is free RAM. Once applications start using all 
the ram, only keep a certain percentage/bare minimum of disk cache. What is 
absolutely undesirable is a growing and growing disk cache when there is not 
enough ram to keep applications and/or their data in ram.

> If this tries so hard to avoid swap that you get an out-of-memory condition
> you may also have to disable the hard maplimit with this:
> echo 0 > /proc/sys/vm/hardmaplimit

Yes I get oom conditions with mapped=100 but they are not my real problem, the 
problem is the disk cache usage pattern. I it would help, I still have some 
dmesg output from the oom killer.

Any ideas? Am I just doing something wrong? I don't use any 
special /proc-settings other than the swappiness/mapped test values.


Alex