Can't make use of swap memory in 2.6.7-bk19

Can't make use of swap memory in 2.6.7-bk19

[Peter Osterlund]

I created a test program that allocates a 300MB buffer and writes to
all bytes sequentially. On my computer, which has 256MB RAM and 512MB
swap, the program gets OOM killed after dirtying about 140-180MB, and
the kernel reports:

Out of Memory: Killed process 3421 (memalloc2).

I ran "vmstat 1" during the test:

procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id wa
 1  0  89844 130624    676  22036  107  296   635   329 1214   452  4  3 63 30
 0  0  89844 130608    692  22036    0    0     0    72 1164   504  3  1 96  0
 0  0  89844 130608    692  22036    0    0     0     0 1067   256  0  0 100  0
 0  0  89844 130608    692  22036    0    0     0     0 1066   248  0  0 100  0
 2  1  99864   2576     80  10016    0    0     8     0 1065  1052 42 41 15  2
[~10s delay here, then all lines below are displayed very quickly]
 0 12 249332   2216    108   5304  164 19440  5280 19440 3572   846  1 24  0 75
 0 11 249332   1992    124   7028  152  372  2620   372 1167   221  0  2  0 98
 0 16  95972 145548    196   8668  216  508  1908   540 1165   248  0  3  0 97
 0 12  95972 143160    228  11124  120    0  2616     0 1184   249  0  1  0 99
 0  8  95972 141040    240  13136  208    0  2236     0 1167   263  0  1  0 99
 0  4  95972 139448    248  14464  252    0  1588     0 1168   248  1  2  0 97
 0  0  95924 138412    260  15400  184    0  1080     8 1187   259  2  0 11 87

"free" directly afterwards:

             total       used       free     shared    buffers     cached
Mem:        255460     119304     136156          0        332      17892
-/+ buffers/cache:     101080     154380
Swap:       530136      89916     440220

I tried setting swappiness to 0, 50, 100, tried changing
overcommit_memory, added an extra 300MB swap file, but I was still not
able to make the test program succeed.

I see this problem using 2.6.7-bk19. I haven't tried other 2.6
kernels, but 2.4.27-rc2 doesn't show the problem.

Here is the test program:

#include <stdio.h>
#include <stdlib.h>

#define MB (1024 * 1024)
#define SIZE (300 * MB)

int main()
{
    int i;
    char *ptr;

    ptr = malloc(SIZE);
    for (i = 0; i < SIZE; i++) {
	ptr[i] = i;
	if ((i % MB) == 0)
	    printf("%d\n", i / MB);
    }
    return 0;
}

-- 
Peter Osterlund - petero2@telia.com
http://w1.894.telia.com/~u89404340

Re: Can't make use of swap memory in 2.6.7-bk19

[William Lee Irwin III]

On Thu, Jul 08, 2004 at 03:36:10AM +0200, Peter Osterlund wrote:
> I created a test program that allocates a 300MB buffer and writes to
> all bytes sequentially. On my computer, which has 256MB RAM and 512MB
> swap, the program gets OOM killed after dirtying about 140-180MB, and
> the kernel reports:
> Out of Memory: Killed process 3421 (memalloc2).
> I ran "vmstat 1" during the test:

akpm, shall I revisit GFP_WIRED?


-- wli

Re: Can't make use of swap memory in 2.6.7-bk19

[Nick Piggin]

[-- Attachment #1: Type: text/plain, Size: 333 bytes --]

Peter Osterlund wrote:
> I created a test program that allocates a 300MB buffer and writes to
> all bytes sequentially. On my computer, which has 256MB RAM and 512MB
> swap, the program gets OOM killed after dirtying about 140-180MB, and
> the kernel reports:
> 

Someone hand me a paper bag... Peter, can you give this patch a try?

[-- Attachment #2: vm-allocfail-fix.patch --]
[-- Type: text/x-patch, Size: 812 bytes --]




---

 linux-2.6-npiggin/mm/vmscan.c |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

diff -puN mm/vmscan.c~vm-allocfail-fix mm/vmscan.c
--- linux-2.6/mm/vmscan.c~vm-allocfail-fix	2004-07-08 12:10:29.000000000 +1000
+++ linux-2.6-npiggin/mm/vmscan.c	2004-07-08 12:12:33.000000000 +1000
@@ -917,12 +917,12 @@ int try_to_free_pages(struct zone **zone
 			sc.nr_reclaimed += reclaim_state->reclaimed_slab;
 			reclaim_state->reclaimed_slab = 0;
 		}
-		if (sc.nr_reclaimed >= SWAP_CLUSTER_MAX) {
+		total_scanned += sc.nr_scanned;
+		total_reclaimed += sc.nr_reclaimed;
+		if (total_reclaimed >= SWAP_CLUSTER_MAX) {
 			ret = 1;
 			goto out;
 		}
-		total_scanned += sc.nr_scanned;
-		total_reclaimed += sc.nr_reclaimed;
 
 		/*
 		 * Try to write back as many pages as we just scanned.  This

_

Re: Can't make use of swap memory in 2.6.7-bk19

[William Lee Irwin III]

Peter Osterlund wrote:
>> I created a test program that allocates a 300MB buffer and writes to
>> all bytes sequentially. On my computer, which has 256MB RAM and 512MB
>> swap, the program gets OOM killed after dirtying about 140-180MB, and
>> the kernel reports:

On Thu, Jul 08, 2004 at 12:14:16PM +1000, Nick Piggin wrote:
> Someone hand me a paper bag... Peter, can you give this patch a try?

Heh, one goes in while I'm not looking, and look what happens.


-- wli

Re: Can't make use of swap memory in 2.6.7-bk19

[Peter Osterlund]

William Lee Irwin III <wli@holomorphy.com> writes:

> Peter Osterlund wrote:
> >> I created a test program that allocates a 300MB buffer and writes to
> >> all bytes sequentially. On my computer, which has 256MB RAM and 512MB
> >> swap, the program gets OOM killed after dirtying about 140-180MB, and
> >> the kernel reports:
> 
> On Thu, Jul 08, 2004 at 12:14:16PM +1000, Nick Piggin wrote:
> > Someone hand me a paper bag... Peter, can you give this patch a try?
> 
> Heh, one goes in while I'm not looking, and look what happens.

Actually, the failure is caused by this change:

http://linux.bkbits.net:8080/linux-2.5/cset@40db004cKFYB35xMHcRXNijl81BLag?nav=index.html|ChangeSet@-3w

It only fails when /proc/sys/vm/laptop_mode is 1.

-- 
Peter Osterlund - petero2@telia.com
http://w1.894.telia.com/~u89404340

Re: Can't make use of swap memory in 2.6.7-bk19

[William Lee Irwin III]

William Lee Irwin III <wli@holomorphy.com> writes:
>> Heh, one goes in while I'm not looking, and look what happens.

On Thu, Jul 08, 2004 at 02:59:11PM +0200, Peter Osterlund wrote:
> Actually, the failure is caused by this change:
> http://linux.bkbits.net:8080/linux-2.5/cset@40db004cKFYB35xMHcRXNijl81BLag?nav=index.html|ChangeSet@-3w
> It only fails when /proc/sys/vm/laptop_mode is 1.

Oh, then I'm stuck in the GFP_WIRED quagmire after all. I guess since
fixing it involves adding lines I'm in deep trouble.


-- wli

Re: Can't make use of swap memory in 2.6.7-bk19

[Nick Piggin]

William Lee Irwin III wrote:
> William Lee Irwin III <wli@holomorphy.com> writes:
> 
>>>Heh, one goes in while I'm not looking, and look what happens.
> 
> 
> On Thu, Jul 08, 2004 at 02:59:11PM +0200, Peter Osterlund wrote:
> 
>>Actually, the failure is caused by this change:
>>http://linux.bkbits.net:8080/linux-2.5/cset@40db004cKFYB35xMHcRXNijl81BLag?nav=index.html|ChangeSet@-3w
>>It only fails when /proc/sys/vm/laptop_mode is 1.
> 
> 
> Oh, then I'm stuck in the GFP_WIRED quagmire after all. I guess since
> fixing it involves adding lines I'm in deep trouble.
> 

Or just see if you can tighten up the conditions for OOM to
start with?

Re: Can't make use of swap memory in 2.6.7-bk19

[William Lee Irwin III]

William Lee Irwin III wrote:
>> Oh, then I'm stuck in the GFP_WIRED quagmire after all. I guess since
>> fixing it involves adding lines I'm in deep trouble.

On Fri, Jul 09, 2004 at 10:57:01AM +1000, Nick Piggin wrote:
> Or just see if you can tighten up the conditions for OOM to
> start with?

You must not have seen the patches. the thread starts with Message-id:
<0406231407.HbLbJbXaHbKbWa5aJb1a4aKb0a3aKb1a0a2aMbMbYa3aLbMb3aJbWaJbXaMbLb1a342@holomorphy.com>

They added a flag indicating wiredness or no to the gfp_mask, which was
then propagated down the call chain and eventually passed as an argument
to out_of_memory(). In turn, out_of_memory() used the flag to determine
whether the nr_swap_pages > 0 check was relevant. i.e. they refined the
OOM conditions based on the wiredness of the failing allocation. What
probably got the stuff permavetoed was the stats reporting I did along
with it that would have been trivial to drop while retaining the needed
functional change. The patch was motivated by the nr_swap_pages > 0
check deadlocking. The __GFP_WIRED business was done to discriminate
the obvious deadlocking scenario from the false OOM mentioned here.

-- wli

Re: Can't make use of swap memory in 2.6.7-bk19

[Nick Piggin]

William Lee Irwin III wrote:
> William Lee Irwin III wrote:
> 
>>>Oh, then I'm stuck in the GFP_WIRED quagmire after all. I guess since
>>>fixing it involves adding lines I'm in deep trouble.
> 
> 
> On Fri, Jul 09, 2004 at 10:57:01AM +1000, Nick Piggin wrote:
> 
>>Or just see if you can tighten up the conditions for OOM to
>>start with?
> 
> 
> You must not have seen the patches. the thread starts with Message-id:
> <0406231407.HbLbJbXaHbKbWa5aJb1a4aKb0a3aKb1a0a2aMbMbYa3aLbMb3aJbWaJbXaMbLb1a342@holomorphy.com>
> 
> They added a flag indicating wiredness or no to the gfp_mask, which was
> then propagated down the call chain and eventually passed as an argument
> to out_of_memory(). In turn, out_of_memory() used the flag to determine
> whether the nr_swap_pages > 0 check was relevant. i.e. they refined the
> OOM conditions based on the wiredness of the failing allocation. What
> probably got the stuff permavetoed was the stats reporting I did along
> with it that would have been trivial to drop while retaining the needed
> functional change. The patch was motivated by the nr_swap_pages > 0
> check deadlocking. The __GFP_WIRED business was done to discriminate
> the obvious deadlocking scenario from the false OOM mentioned here.
> 

No, I did see those patches. I'm not saying they're not worth
persuing; on the contrary, they look quite interesting. However,
it might worthwhile looking at more basic things first, for this
problem anyway.

Re: Can't make use of swap memory in 2.6.7-bk19

[William Lee Irwin III]

William Lee Irwin III wrote:
>> They added a flag indicating wiredness or no to the gfp_mask, which was
>> then propagated down the call chain and eventually passed as an argument
>> to out_of_memory(). In turn, out_of_memory() used the flag to determine
>> whether the nr_swap_pages > 0 check was relevant. i.e. they refined the
>> OOM conditions based on the wiredness of the failing allocation. What
>> probably got the stuff permavetoed was the stats reporting I did along
>> with it that would have been trivial to drop while retaining the needed
>> functional change. The patch was motivated by the nr_swap_pages > 0
>> check deadlocking. The __GFP_WIRED business was done to discriminate
>> the obvious deadlocking scenario from the false OOM mentioned here.

On Fri, Jul 09, 2004 at 12:06:54PM +1000, Nick Piggin wrote:
> No, I did see those patches. I'm not saying they're not worth
> persuing; on the contrary, they look quite interesting. However,
> it might worthwhile looking at more basic things first, for this
> problem anyway.

Enumerate those more basic things.


-- wli

Re: Can't make use of swap memory in 2.6.7-bk19

[Andrew Morton]

William Lee Irwin III <wli@holomorphy.com> wrote:
>
> William Lee Irwin III wrote:
> >> They added a flag indicating wiredness or no to the gfp_mask, which was
> >> then propagated down the call chain and eventually passed as an argument
> >> to out_of_memory(). In turn, out_of_memory() used the flag to determine
> >> whether the nr_swap_pages > 0 check was relevant. i.e. they refined the
> >> OOM conditions based on the wiredness of the failing allocation. What
> >> probably got the stuff permavetoed was the stats reporting I did along
> >> with it that would have been trivial to drop while retaining the needed
> >> functional change. The patch was motivated by the nr_swap_pages > 0
> >> check deadlocking. The __GFP_WIRED business was done to discriminate
> >> the obvious deadlocking scenario from the false OOM mentioned here.
> 
> On Fri, Jul 09, 2004 at 12:06:54PM +1000, Nick Piggin wrote:
> > No, I did see those patches. I'm not saying they're not worth
> > persuing; on the contrary, they look quite interesting. However,
> > it might worthwhile looking at more basic things first, for this
> > problem anyway.
> 
> Enumerate those more basic things.
> 

1: work out why it's prematurely calling out_of_memory() when laptop_mode=1.

Re: Can't make use of swap memory in 2.6.7-bk19

[William Lee Irwin III]

William Lee Irwin III <wli@holomorphy.com> wrote:
>> Enumerate those more basic things.

On Thu, Jul 08, 2004 at 07:12:54PM -0700, Andrew Morton wrote:
> 1: work out why it's prematurely calling out_of_memory() when laptop_mode=1.

The obvious difference in writeback policy.

I've apparently touched on policy, and paid for that mistake with an
overpoweringly Sterculian whiff of penguins. Now backing away slowly...


-- wli

Re: Can't make use of swap memory in 2.6.7-bk19

[Andrew Morton]

William Lee Irwin III <wli@holomorphy.com> wrote:
>
> William Lee Irwin III <wli@holomorphy.com> wrote:
> >> Enumerate those more basic things.
> 
> On Thu, Jul 08, 2004 at 07:12:54PM -0700, Andrew Morton wrote:
> > 1: work out why it's prematurely calling out_of_memory() when laptop_mode=1.
> 
> The obvious difference in writeback policy.

The writeback code isn't in the picture with this workload - there's no
dirty pagecache around.

> I've apparently touched on policy, and paid for that mistake with an
> overpoweringly Sterculian whiff of penguins. Now backing away slowly...

Not sure what that means.

The problem is trivial to reproduce.  Killing these lines in shrink_list():

			if (laptop_mode && !sc->may_writepage)
				goto keep_locked;

makes it go away.  Something's out of whack in there, and the removal of
the free swapspace test exposed some prior problem.  I'll poke at it some
more.

Re: Can't make use of swap memory in 2.6.7-bk19

[Nick Piggin]

William Lee Irwin III wrote:

> 
> Enumerate those more basic things.
> 

Why does it fail when laptop_mode is set, and not otherwise?
Proably by the time laptop mode decides to start writing something,
the scanner is ready to call it quits and go OOM. Maybe laptop
mode needs to fire up the disk and do another pass before OOM.

Re: Can't make use of swap memory in 2.6.7-bk19

[Peter Osterlund]

Nick Piggin <nickpiggin@yahoo.com.au> writes:

> Peter Osterlund wrote:
> > I created a test program that allocates a 300MB buffer and writes to
> > all bytes sequentially. On my computer, which has 256MB RAM and 512MB
> > swap, the program gets OOM killed after dirtying about 140-180MB, and
> > the kernel reports:
> >
> 
> Someone hand me a paper bag... Peter, can you give this patch a try?

Doesn't help. My test program still fails in the same way.

-- 
Peter Osterlund - petero2@telia.com
http://w1.894.telia.com/~u89404340

Re: Can't make use of swap memory in 2.6.7-bk19

[Andrew Morton]

Peter Osterlund <petero2@telia.com> wrote:
>
> Nick Piggin <nickpiggin@yahoo.com.au> writes:
> 
> > Peter Osterlund wrote:
> > > I created a test program that allocates a 300MB buffer and writes to
> > > all bytes sequentially. On my computer, which has 256MB RAM and 512MB
> > > swap, the program gets OOM killed after dirtying about 140-180MB, and
> > > the kernel reports:
> > >
> > 
> > Someone hand me a paper bag... Peter, can you give this patch a try?
> 
> Doesn't help. My test program still fails in the same way.
> 

Something odd is happening - I've run that testcase in various shapes and
forms a huge number of times.

What filesystems are in use?  Is there anything unusual about the setup? 
Do earlier 2.6 kernels exhibit the same problem?  Is something wrong with
the disk system?

Re: Can't make use of swap memory in 2.6.7-bk19

[Nick Piggin]

Andrew Morton wrote:
> Peter Osterlund <petero2@telia.com> wrote:

>>Doesn't help. My test program still fails in the same way.
>>
> 
> 
> Something odd is happening - I've run that testcase in various shapes and
> forms a huge number of times.
> 
> What filesystems are in use?  Is there anything unusual about the setup? 
> Do earlier 2.6 kernels exhibit the same problem?  Is something wrong with
> the disk system?
> 

Also, have you changed /proc/sys/vm/swappiness?

Re: Can't make use of swap memory in 2.6.7-bk19

[Peter Osterlund]

On Thu, 8 Jul 2004, Nick Piggin wrote:

> Andrew Morton wrote:
> > Peter Osterlund <petero2@telia.com> wrote:
>
> >>Doesn't help. My test program still fails in the same way.
> >
> > Something odd is happening - I've run that testcase in various shapes and
> > forms a huge number of times.
> >
> > What filesystems are in use?  Is there anything unusual about the setup?
> > Do earlier 2.6 kernels exhibit the same problem?  Is something wrong with
> > the disk system?

I use only ext3, the disc system is working fine AFAIK. 2.6.7-bk10 has the
same problem, 2.6.7-bk2 completes the test case, but it takes 50-70s, and
the system is completely frozen until the program finishes.

> Also, have you changed /proc/sys/vm/swappiness?

swappiness is set to 60.

However, I realized that I had set /proc/sys/vm/laptop_mode to 1. If I set
it to 0, 2.6.7-bk10 starts to work.

-- 
Peter Osterlund - petero2@telia.com
http://w1.894.telia.com/~u89404340

Re: Can't make use of swap memory in 2.6.7-bk19

[William Lee Irwin III]

On Thu, Jul 08, 2004 at 11:30:45AM +0200, Peter Osterlund wrote:
> swappiness is set to 60.
> However, I realized that I had set /proc/sys/vm/laptop_mode to 1. If I set
> it to 0, 2.6.7-bk10 starts to work.

Probably not what will get merged, but does the following brutal hack
do anything for you?


Index: laptop-2.6.8-rc1/mm/vmscan.c
===================================================================
--- laptop-2.6.8-rc1.orig/mm/vmscan.c	2004-07-11 10:33:55.000000000 -0700
+++ laptop-2.6.8-rc1/mm/vmscan.c	2004-07-13 22:18:04.193959968 -0700
@@ -902,7 +902,7 @@
 	sc.may_writepage = 0;
 
 	inc_page_state(allocstall);
-
+retry:
 	for (i = 0; zones[i] != 0; i++)
 		zones[i]->temp_priority = DEF_PRIORITY;
 
@@ -940,8 +940,14 @@
 		if (sc.nr_scanned && priority < DEF_PRIORITY - 2)
 			blk_congestion_wait(WRITE, HZ/10);
 	}
-	if ((gfp_mask & __GFP_FS) && !(gfp_mask & __GFP_NORETRY))
-		out_of_memory();
+	if ((gfp_mask & __GFP_FS) && !(gfp_mask & __GFP_NORETRY)) {
+		if (!laptop_mode || sc.may_writepage)
+			out_of_memory();
+		else {
+			sc.may_writepage = 1;
+			goto retry;
+		}
+	}
 out:
 	for (i = 0; zones[i] != 0; i++)
 		zones[i]->prev_priority = zones[i]->temp_priority;

Re: Can't make use of swap memory in 2.6.7-bk19

[Peter Osterlund]

William Lee Irwin III <wli@holomorphy.com> writes:

> On Thu, Jul 08, 2004 at 11:30:45AM +0200, Peter Osterlund wrote:
> > swappiness is set to 60.
> > However, I realized that I had set /proc/sys/vm/laptop_mode to 1. If I set
> > it to 0, 2.6.7-bk10 starts to work.
> 
> Probably not what will get merged, but does the following brutal hack
> do anything for you?

Doesn't help. I added some printk's to your patch and got this:

try_to_free_pages: ---
try_to_free_pages: lap:1 totscan:128 dirty:4 unstable:0
try_to_free_pages: lap:1 totscan:256 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:512 dirty:0 unstable:0
try_to_free_pages: ---
try_to_free_pages: lap:1 totscan:120 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:248 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:504 dirty:0 unstable:0
try_to_free_pages: ---
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:256 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:511 dirty:0 unstable:0
try_to_free_pages: ---
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:256 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:508 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:959 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:952 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:960 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:956 dirty:0 unstable:0
try_to_free_pages: ---
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:256 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:488 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:1715 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:628 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:1852 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:1856 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:1848 dirty:0 unstable:0
try_to_free_pages: ---
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: ---
try_to_free_pages: lap:1 totscan:73 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:137 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:265 dirty:0 unstable:0
try_to_free_pages: ---
try_to_free_pages: setting may_writepage=1
try_to_free_pages: oom
try_to_free_pages: ---
try_to_free_pages: setting may_writepage=1
try_to_free_pages: oom
try_to_free_pages: ---
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: oom
try_to_free_pages: ---
try_to_free_pages: setting may_writepage=1
try_to_free_pages: oom
try_to_free_pages: ---
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: oom
try_to_free_pages: ---
try_to_free_pages: setting may_writepage=1
try_to_free_pages: ---
try_to_free_pages: setting may_writepage=1
try_to_free_pages: oom
try_to_free_pages: ---
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:128 dirty:0 unstable:0
try_to_free_pages: oom
try_to_free_pages: ---
try_to_free_pages: lap:1 totscan:64 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:64 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:64 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:64 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:64 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:64 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:64 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:64 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:64 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:64 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:64 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:64 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:64 dirty:0 unstable:0
try_to_free_pages: oom
try_to_free_pages: ---
try_to_free_pages: setting may_writepage=1
try_to_free_pages: ---
try_to_free_pages: setting may_writepage=1
try_to_free_pages: oom
try_to_free_pages: lap:1 totscan:265 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:265 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:265 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:265 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:265 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:265 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:265 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:265 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:265 dirty:0 unstable:0
try_to_free_pages: oom
try_to_free_pages: lap:1 totscan:256 dirty:0 unstable:0
try_to_free_pages: lap:1 totscan:576 dirty:0 unstable:0
try_to_free_pages: ---
try_to_free_pages: lap:1 totscan:182 dirty:0 unstable:0
try_to_free_pages: ---
try_to_free_pages: setting may_writepage=1
try_to_free_pages: oom
Out of Memory: Killed process 2603 (memalloc2).
try_to_free_pages: ---
try_to_free_pages: setting may_writepage=1
try_to_free_pages: oom

---

 linux-petero/mm/vmscan.c |   31 ++++++++++++++++++++++++++++---
 1 files changed, 28 insertions(+), 3 deletions(-)

diff -puN mm/vmscan.c~laptop-swap-fix mm/vmscan.c
--- linux/mm/vmscan.c~laptop-swap-fix	2004-07-14 10:30:02.000000000 +0200
+++ linux-petero/mm/vmscan.c	2004-07-14 12:22:53.000000000 +0200
@@ -897,12 +897,13 @@ int try_to_free_pages(struct zone **zone
 	struct reclaim_state *reclaim_state = current->reclaim_state;
 	struct scan_control sc;
 	int i;
+	int first = 1;
 
 	sc.gfp_mask = gfp_mask;
 	sc.may_writepage = 0;
 
 	inc_page_state(allocstall);
-
+retry:
 	for (i = 0; zones[i] != 0; i++)
 		zones[i]->temp_priority = DEF_PRIORITY;
 
@@ -932,6 +933,14 @@ int try_to_free_pages(struct zone **zone
 		 * writeout.  So in laptop mode, write out the whole world.
 		 */
 		if (total_scanned > SWAP_CLUSTER_MAX + SWAP_CLUSTER_MAX/2) {
+			int dirty = read_page_state(nr_dirty);
+			int unstable = read_page_state(nr_unstable);
+			if (first) {
+				printk("try_to_free_pages: ---\n");
+				first = 0;
+			}
+			printk("try_to_free_pages: lap:%d totscan:%d dirty:%d unstable:%d\n",
+			       laptop_mode, total_scanned, dirty, unstable);
 			wakeup_bdflush(laptop_mode ? 0 : total_scanned);
 			sc.may_writepage = 1;
 		}
@@ -940,8 +949,24 @@ int try_to_free_pages(struct zone **zone
 		if (sc.nr_scanned && priority < DEF_PRIORITY - 2)
 			blk_congestion_wait(WRITE, HZ/10);
 	}
-	if ((gfp_mask & __GFP_FS) && !(gfp_mask & __GFP_NORETRY))
-		out_of_memory();
+	if ((gfp_mask & __GFP_FS) && !(gfp_mask & __GFP_NORETRY)) {
+		if (!laptop_mode || sc.may_writepage) {
+			if (first) {
+				printk("try_to_free_pages: ---\n");
+				first = 0;
+			}
+			printk("try_to_free_pages: oom\n");
+			out_of_memory();
+		} else {
+			if (first) {
+				printk("try_to_free_pages: ---\n");
+				first = 0;
+			}
+			printk("try_to_free_pages: setting may_writepage=1\n");
+			sc.may_writepage = 1;
+			goto retry;
+		}
+	}
 out:
 	for (i = 0; zones[i] != 0; i++)
 		zones[i]->prev_priority = zones[i]->temp_priority;
_

-- 
Peter Osterlund - petero2@telia.com
http://w1.894.telia.com/~u89404340

Re: Can't make use of swap memory in 2.6.7-bk19

[William Lee Irwin III]

William Lee Irwin III <wli@holomorphy.com> writes:
>> Probably not what will get merged, but does the following brutal hack
>> do anything for you?

On Wed, Jul 14, 2004 at 12:39:18PM +0200, Peter Osterlund wrote:
> Doesn't help. I added some printk's to your patch and got this:

Brilliant, about zero dirty. Okay, I'm desperate. Could you try running
this and see what it spews when the OOM kill happens?


-- wli

Index: for_akpm-2.6.8-rc1/mm/oom_kill.c
===================================================================
--- for_akpm-2.6.8-rc1.orig/mm/oom_kill.c	2004-07-11 10:34:39.000000000 -0700
+++ for_akpm-2.6.8-rc1/mm/oom_kill.c	2004-07-14 03:54:23.990164640 -0700
@@ -220,7 +220,7 @@
 /**
  * out_of_memory - is the system out of memory?
  */
-void out_of_memory(void)
+void out_of_memory(unsigned gfp_mask, int order)
 {
 	/*
 	 * oom_lock protects out_of_memory()'s static variables.
@@ -273,6 +273,10 @@
 
 	/* oom_kill() sleeps */
 	spin_unlock(&oom_lock);
+	printk("Out of memory: pid %d, comm %-13.13s, gfp 0x%u, order %d\n",
+		current->pid, current->comm, gfp_mask, order);
+	show_free_areas();
+	dump_stack();
 	oom_kill();
 	spin_lock(&oom_lock);
 
Index: for_akpm-2.6.8-rc1/mm/vmscan.c
===================================================================
--- for_akpm-2.6.8-rc1.orig/mm/vmscan.c	2004-07-13 22:18:04.193959000 -0700
+++ for_akpm-2.6.8-rc1/mm/vmscan.c	2004-07-14 03:53:31.422156192 -0700
@@ -942,7 +942,7 @@
 	}
 	if ((gfp_mask & __GFP_FS) && !(gfp_mask & __GFP_NORETRY)) {
 		if (!laptop_mode || sc.may_writepage)
-			out_of_memory();
+			out_of_memory(gfp_mask, order);
 		else {
 			sc.may_writepage = 1;
 			goto retry;
Index: for_akpm-2.6.8-rc1/include/linux/swap.h
===================================================================
--- for_akpm-2.6.8-rc1.orig/include/linux/swap.h	2004-07-13 22:57:05.104087536 -0700
+++ for_akpm-2.6.8-rc1/include/linux/swap.h	2004-07-14 03:53:09.308517976 -0700
@@ -148,7 +148,7 @@
 #define vm_swap_full() (nr_swap_pages*2 < total_swap_pages)
 
 /* linux/mm/oom_kill.c */
-extern void out_of_memory(void);
+void out_of_memory(unsigned, int);
 
 /* linux/mm/memory.c */
 extern void swapin_readahead(swp_entry_t, unsigned long, struct vm_area_struct *);

Re: Can't make use of swap memory in 2.6.7-bk19

[Peter Osterlund]

William Lee Irwin III <wli@holomorphy.com> writes:

> William Lee Irwin III <wli@holomorphy.com> writes:
> >> Probably not what will get merged, but does the following brutal hack
> >> do anything for you?
> 
> On Wed, Jul 14, 2004 at 12:39:18PM +0200, Peter Osterlund wrote:
> > Doesn't help. I added some printk's to your patch and got this:
> 
> Brilliant, about zero dirty. Okay, I'm desperate. Could you try running
> this and see what it spews when the OOM kill happens?

Out of memory: pid 2655, comm xterm        , gfp 0x466, order 0
DMA per-cpu:
cpu 0 hot: low 2, high 6, batch 1
cpu 0 cold: low 0, high 2, batch 1
Normal per-cpu:
cpu 0 hot: low 28, high 84, batch 14
cpu 0 cold: low 0, high 28, batch 14
HighMem per-cpu: empty

Free pages:        1820kB (0kB HighMem)
Active:754 inactive:56177 dirty:0 writeback:3427 unstable:0 free:455 slab:2880 mapped:890 pagetables:384
DMA free:884kB min:28kB low:56kB high:84kB active:24kB inactive:11124kB present:16384kB
protections[]: 14 252 252
Normal free:936kB min:476kB low:952kB high:1428kB active:2992kB inactive:213584kB present:245248kB
protections[]: 0 238 238
HighMem free:0kB min:128kB low:256kB high:384kB active:0kB inactive:0kB present:0kB
protections[]: 0 0 0
DMA: 1*4kB 0*8kB 1*16kB 1*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 884kB
Normal: 0*4kB 1*8kB 0*16kB 1*32kB 0*64kB 1*128kB 1*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 936kB
HighMem: empty
Swap cache: add 57227, delete 1309, find 331/331, race 0+0
 [<c0131d69>] out_of_memory+0xb2/0x105
 [<c013a0bd>] try_to_free_pages+0x143/0x190
 [<c0132bae>] __alloc_pages+0x1c3/0x347
 [<c013595b>] do_page_cache_readahead+0x13b/0x197
 [<c012fb50>] filemap_nopage+0x2d8/0x371
 [<c013cfe9>] do_no_page+0xb7/0x30f
 [<c013d431>] handle_mm_fault+0xd6/0x171
 [<c0111076>] do_page_fault+0x346/0x548
 [<c01d5868>] __copy_to_user_ll+0x48/0x6c
 [<c015c7c3>] sys_select+0x228/0x4b0
 [<c0110d30>] do_page_fault+0x0/0x548
 [<c01040a1>] error_code+0x2d/0x38
Out of Memory: Killed process 2666 (memalloc2).

-- 
Peter Osterlund - petero2@telia.com
http://w1.894.telia.com/~u89404340

Re: Can't make use of swap memory in 2.6.7-bk19

[William Lee Irwin III]

On Wed, Jul 14, 2004 at 02:55:57PM +0200, Peter Osterlund wrote:
> Out of memory: pid 2655, comm xterm        , gfp 0x466, order 0
[...]
>  [<c0131d69>] out_of_memory+0xb2/0x105
>  [<c013a0bd>] try_to_free_pages+0x143/0x190
>  [<c0132bae>] __alloc_pages+0x1c3/0x347
>  [<c013595b>] do_page_cache_readahead+0x13b/0x197
>  [<c012fb50>] filemap_nopage+0x2d8/0x371
>  [<c013cfe9>] do_no_page+0xb7/0x30f
>  [<c013d431>] handle_mm_fault+0xd6/0x171
>  [<c0111076>] do_page_fault+0x346/0x548
>  [<c01d5868>] __copy_to_user_ll+0x48/0x6c
>  [<c015c7c3>] sys_select+0x228/0x4b0
>  [<c0110d30>] do_page_fault+0x0/0x548
>  [<c01040a1>] error_code+0x2d/0x38
> Out of Memory: Killed process 2666 (memalloc2).

$ printf "%lx\n" $(( 0x100 | 0x80 | 0x40 | 0x10 | 0x2 ))
1d2
$ egrep '(0x100|0x80|0x40|0x10) |0x02' /mnt/dm0/laptop-2.6.8-rc1/include/linux/gfp.h                               
#define __GFP_HIGHMEM   0x02
#define __GFP_WAIT      0x10    /* Can wait and reschedule? */
#define __GFP_IO        0x40    /* Can start physical IO? */
#define __GFP_FS        0x80    /* Can call down to low-level FS? */
#define __GFP_COLD      0x100   /* Cache-cold page required */

Hmm, I wonder why we didn't just fail the allocation. Maybe we should
check (gfp_mask & __GFP_NOFAIL) instead of !(gfp_mask & __GFP_NORETRY);
everything else should be allowed to fail, except things loop if
without __GFP_NORETRY, as presumably only __GFP_NOFAIL allocations are
ones that supposedly can't handle failures in-context.

The only difference laptop_mode should have is dirty memory handling,
but you don't have any dirty memory. Maybe swapcache is fooling things.
Most notably, add_to_swap() sets the page dirty...

Something is very wrong here... could you try this?


- wli

Index: oom-2.6.8-rc1/mm/vmscan.c
===================================================================
--- oom-2.6.8-rc1.orig/mm/vmscan.c	2004-07-14 06:17:13.876343912 -0700
+++ oom-2.6.8-rc1/mm/vmscan.c	2004-07-14 06:22:15.986416200 -0700
@@ -417,7 +417,8 @@
 				goto keep_locked;
 			if (!may_enter_fs)
 				goto keep_locked;
-			if (laptop_mode && !sc->may_writepage)
+			if (laptop_mode && !sc->may_writepage &&
+							!PageSwapCache(page))
 				goto keep_locked;
 
 			/* Page is dirty, try to write it out here */

Re: Can't make use of swap memory in 2.6.7-bk19

[Peter Osterlund]

William Lee Irwin III <wli@holomorphy.com> writes:

> The only difference laptop_mode should have is dirty memory handling,
> but you don't have any dirty memory. Maybe swapcache is fooling things.
> Most notably, add_to_swap() sets the page dirty...
> 
> Something is very wrong here... could you try this?
...
> Index: oom-2.6.8-rc1/mm/vmscan.c
> ===================================================================
> --- oom-2.6.8-rc1.orig/mm/vmscan.c	2004-07-14 06:17:13.876343912 -0700
> +++ oom-2.6.8-rc1/mm/vmscan.c	2004-07-14 06:22:15.986416200 -0700
> @@ -417,7 +417,8 @@
>  				goto keep_locked;
>  			if (!may_enter_fs)
>  				goto keep_locked;
> -			if (laptop_mode && !sc->may_writepage)
> +			if (laptop_mode && !sc->may_writepage &&
> +							!PageSwapCache(page))
>  				goto keep_locked;
>  
>  			/* Page is dirty, try to write it out here */

This patch fixes my problem. No more bogus OOMs in laptop mode, and
the test program runs approximately equally fast in laptop mode as it
does in "normal" mode.

Thanks.

-- 
Peter Osterlund - petero2@telia.com
http://w1.894.telia.com/~u89404340