Commenting out out_of_memory() function in __alloc_pages()

Commenting out out_of_memory() function in __alloc_pages()

[Abu M. Muttalib]

[-- Attachment #1: Type: text/plain, Size: 794 bytes --]

Hi,

I am getting the Out of memory.

To circumvent the problem, I have commented the call to "out_of_memory(),
and replaced "goto restart" with "goto nopage".

At "nopage:" lable I have added a call to "schedule()" and then "return
NULL" after "schedule()".

I tried the modified kernel with a test application, the test application is
mallocing memory in a loop. Unlike as expected the process gets killed. On
second run of the same application I am getting the page allocation failure
as expected but subsequently the system hangs.

I am attaching the test application and the log herewith.

I am getting this exception with kernel 2.6.13. With kernel
2.4.19-rmka7-pxa1 there was no problem.

Why its so? What can I do to alleviate the OOM problem?

Thanks in anticipation and regards,
Abu.

[-- Attachment #2: mail_oom_test_6.TXT --]
[-- Type: text/plain, Size: 11528 bytes --]

sh-3.00# ./test1\a

OOM Test: Counter = 0
....
OOM Test: Counter = 6635
, OOM: Out of Memory would have been called....<4>test1: page allocation failure. order:0, mode:0xd2
\rMem-info:
\rDMA per-cpu:
\rcpu 0 hot: low 2, high 6, batch 1 used:2
\rcpu 0 cold: low 0, high 2, batch 1 used:1
\rNormal per-cpu: empty
\rHighMem per-cpu: empty
\rFree pages:         588kB (0kB HighMem)
\rActive:2178 inactive:230 dirty:0 writeback:0 unstable:0 free:147 slab:367 mapped:2178 pagetables:43
\rDMA free:588kB min:512kB low:640kB high:768kB active:8712kB inactive:920kB present:16384kB pages_scanned:4851 all_unreclaimable? no
\rlowmem_reserve[]: 0 0 0
\rNormal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
\rlowmem_reserve[]: 0 0 0
\rHighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
\rlowmem_reserve[]: 0 0 0
\rDMA: 9*4kB 3*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 1*512kB 0*1024kB = 588kB
\rNormal: empty
\rHighMem: empty
\rSwap cache: add 0, delete 0, find 0/0, race 0+0
\rFree swap  = 0kB
\rTotal swap = 0kB
\rFree swap:            0kB
\r4096 pages of RAM
\r218 free pages
\r625 reserved pages
\r367 slab pages
\r0 pages shared
\r0 pages swap cached
\rVM: killing process test1
\rKilled
sh-3.00# ./test1

OOM Test: Counter = 0
....
OOM Test: Counter = 6416
, OOM: Out of Memory would have been called....<4>test1: page allocation failure. order:0, mode:0x201d0
\rMem-info:
\rDMA per-cpu:
\rcpu 0 hot: low 2, high 6, batch 1 used:2
\rcpu 0 cold: low 0, high 2, batch 1 used:1
\rNormal per-cpu: empty
\rHighMem per-cpu: empty
\rFree pages:         672kB (0kB HighMem)
\rActive:2187 inactive:170 dirty:0 writeback:0 unstable:0 free:168 slab:365 mapped:2125 pagetables:43
\rDMA free:672kB min:512kB low:640kB high:768kB active:8748kB inactive:680kB present:16384kB pages_scanned:2939 all_unreclaimable? no
\rlowmem_reserve[]: 0 0 0
\rNormal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
\rlowmem_reserve[]: 0 0 0
\rHighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
\rlowmem_reserve[]: 0 0 0
\rDMA: 28*4kB 6*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 1*512kB 0*1024kB = 672kB
\rNormal: empty
\rHighMem: empty
\rSwap cache: add 0, delete 0, find 0/0, race 0+0
\rFree swap  = 0kB
\rTotal swap = 0kB
\rFree swap:            0kB
\r4096 pages of RAM
\r239 free pages
\r625 reserved pages
\r365 slab pages
\r13 pages shared
\r0 pages swap cached
\rError -5 while decompressing!
\rc01e42f4(2388)->c0754000(4096)
\rError -3 while decompressing!
\rc01e4c48(2531)->c0caf000(4096)
\rError -3 while decompressing!
\rc01e562b(2643)->c0914000(4096)
\r, OOM: Out of Memory would have been called....<4>test1: page allocation failure. order:0, mode:0x201d2
\rMem-info:
\rDMA per-cpu:
\rcpu 0 hot: low 2, high 6, batch 1 used:3
\rcpu 0 cold: low 0, high 2, batch 1 used:1
\rNormal per-cpu: empty
\rHighMem per-cpu: empty
\rFree pages:         600kB (0kB HighMem)
\rActive:2267 inactive:113 dirty:0 writeback:0 unstable:0 free:150 slab:365 mapped:2125 pagetables:43
\rDMA free:600kB min:512kB low:640kB high:768kB active:9068kB inactive:452kB present:16384kB pages_scanned:222 all_unreclaimable? no
\rlowmem_reserve[]: 0 0 0
\rNormal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
\rlowmem_reserve[]: 0 0 0
\rHighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
\rlowmem_reserve[]: 0 0 0
\rDMA: 22*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 1*512kB 0*1024kB = 600kB
\rNormal: empty
\rHighMem: empty
\rSwap cache: add 0, delete 0, find 0/0, race 0+0
\rFree swap  = 0kB
\rTotal swap = 0kB
\rFree swap:            0kB
\r4096 pages of RAM
\r222 free pages
\r625 reserved pages
\r365 slab pages
\r13 pages shared
\r0 pages swap cached
\r, OOM: Out of Memory would have been called....<4>init: page allocation failure. order:0, mode:0x201d0
\rMem-info:
\rDMA per-cpu:
\rcpu 0 hot: low 2, high 6, batch 1 used:3
\rcpu 0 cold: low 0, high 2, batch 1 used:1
\rNormal per-cpu: empty
\rHighMem per-cpu: empty
\rFree pages:         696kB (0kB HighMem)
\rActive:2220 inactive:136 dirty:0 writeback:0 unstable:0 free:174 slab:365 mapped:2125 pagetables:43
\rDMA free:696kB min:512kB low:640kB high:768kB active:8880kB inactive:544kB present:16384kB pages_scanned:6729 all_unreclaimable? no
\rlowmem_reserve[]: 0 0 0
\rNormal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
\rlowmem_reserve[]: 0 0 0
\rHighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
\rlowmem_reserve[]: 0 0 0
\rDMA: 44*4kB 1*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 1*512kB 0*1024kB = 696kB
\rNormal: empty
\rHighMem: empty
\rSwap cache: add 0, delete 0, find 0/0, race 0+0
\rFree swap  = 0kB
\rTotal swap = 0kB
\rFree swap:            0kB
\r4096 pages of RAM
\r246 free pages
\r625 reserved pages
\r365 slab pages
\r3 pages shared
\r0 pages swap cached
\rError -5 while decompressing!
\rc01e7949(2386)->c040d000(4096)
\rError -3 while decompressing!
\rc01e829b(2297)->c05e9000(4096)
\r, OOM: Out of Memory would have been called....<4>init: page allocation failure. order:0, mode:0x201d2
\rMem-info:
\rDMA per-cpu:
\rcpu 0 hot: low 2, high 6, batch 1 used:4
\rcpu 0 cold: low 0, high 2, batch 1 used:1
\rNormal per-cpu: empty
\rHighMem per-cpu: empty
\rFree pages:         600kB (0kB HighMem)
\rActive:2189 inactive:166 dirty:0 writeback:0 unstable:0 free:150 slab:364 mapped:2125 pagetables:43
\rDMA free:600kB min:512kB low:640kB high:768kB active:8756kB inactive:664kB present:16384kB pages_scanned:7675 all_unreclaimable? no
\rlowmem_reserve[]: 0 0 0
\rNormal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
\rlowmem_reserve[]: 0 0 0
\rHighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
\rlowmem_reserve[]: 0 0 0
\rDMA: 20*4kB 1*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 1*512kB 0*1024kB = 600kB
\rNormal: empty
\rHighMem: empty
\rSwap cache: add 0, delete 0, find 0/0, race 0+0
\rFree swap  = 0kB
\rTotal swap = 0kB
\rFree swap:            0kB
\r4096 pages of RAM
\r223 free pages
\r625 reserved pages
\r364 slab pages
\r1 pages shared
\r0 pages swap cached
\r, OOM: Out of Memory would have been called....<4>test1: page allocation failure. order:0, mode:0x201d0
\rMem-info:
\rDMA per-cpu:
\rcpu 0 hot: low 2, high 6, batch 1 used:4
\rcpu 0 cold: low 0, high 2, batch 1 used:1
\rNormal per-cpu: empty
\rHighMem per-cpu: empty
\rFree pages:         600kB (0kB HighMem)
\rActive:2291 inactive:64 dirty:0 writeback:0 unstable:0 free:150 slab:364 mapped:2125 pagetables:43
\rDMA free:600kB min:512kB low:640kB high:768kB active:9164kB inactive:256kB present:16384kB pages_scanned:10106 all_unreclaimable? yes
\rlowmem_reserve[]: 0 0 0
\rNormal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
\rlowmem_reserve[]: 0 0 0
\rHighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
\rlowmem_reserve[]: 0 0 0
\rDMA: 20*4kB 1*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 1*512kB 0*1024kB = 600kB
\rNormal: empty
\rHighMem: empty
\rSwap cache: add 0, delete 0, find 0/0, race 0+0
\rFree swap  = 0kB
\rTotal swap = 0kB
\rFree swap:            0kB
\r4096 pages of RAM
\r223 free pages
\r625 reserved pages
\r364 slab pages
\r2 pages shared
\r0 pages swap cached
\r, OOM: Out of Memory would have been called....<4>init: page allocation failure. order:0, mode:0x201d2
\rMem-info:
\rDMA per-cpu:
\rcpu 0 hot: low 2, high 6, batch 1 used:4
\rcpu 0 cold: low 0, high 2, batch 1 used:0
\rNormal per-cpu: empty
\rHighMem per-cpu: empty
\rFree pages:         512kB (0kB HighMem)
\rActive:2296 inactive:125 dirty:0 writeback:0 unstable:0 free:128 slab:364 mapped:2127 pagetables:43
\rDMA free:512kB min:512kB low:640kB high:768kB active:9184kB inactive:500kB present:16384kB pages_scanned:10106 all_unreclaimable? yes
\rlowmem_reserve[]: 0 0 0
\rNormal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
\rlowmem_reserve[]: 0 0 0
\rHighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
\rlowmem_reserve[]: 0 0 0
\rDMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 1*512kB 0*1024kB = 512kB
\rNormal: empty
\rHighMem: empty
\rSwap cache: add 0, delete 0, find 0/0, race 0+0
\rFree swap  = 0kB
\rTotal swap = 0kB
\rFree swap:            0kB
\r4096 pages of RAM
\r200 free pages
\r625 reserved pages
\r364 slab pages
\r34 pages shared
\r0 pages swap cached
\r, OOM: Out of Memory would have been called....<4>init: page allocation failure. order:0, mode:0x201d2
\rMem-info:
\rDMA per-cpu:
\rcpu 0 hot: low 2, high 6, batch 1 used:4
\rcpu 0 cold: low 0, high 2, batch 1 used:0
\rNormal per-cpu: empty
\rHighMem per-cpu: empty
\rFree pages:         512kB (0kB HighMem)
\rActive:2296 inactive:125 dirty:0 writeback:0 unstable:0 free:128 slab:364 mapped:2127 pagetables:43
\rDMA free:512kB min:512kB low:640kB high:768kB active:9184kB inactive:500kB present:16384kB pages_scanned:10106 all_unreclaimable? yes
\rlowmem_reserve[]: 0 0 0
\rNormal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
\rlowmem_reserve[]: 0 0 0
\rHighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
\rlowmem_reserve[]: 0 0 0
\rDMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 1*512kB 0*1024kB = 512kB
\rNormal: empty
\rHighMem: empty
\rSwap cache: add 0, delete 0, find 0/0, race 0+0
\rFree swap  = 0kB
\rTotal swap = 0kB
\rFree swap:            0kB
\r4096 pages of RAM
\r200 free pages
\r625 reserved pages
\r364 slab pages
\r34 pages shared
\r0 pages swap cached
\r, OOM: Out of Memory would have been called....<4>test1: page allocation failure. order:0, mode:0x201d2
\rMem-info:
\rDMA per-cpu:
\rcpu 0 hot: low 2, high 6, batch 1 used:4
\rcpu 0 cold: low 0, high 2, batch 1 used:0
\rNormal per-cpu: empty
\rHighMem per-cpu: empty
\rFree pages:         512kB (0kB HighMem)
\rActive:2296 inactive:125 dirty:0 writeback:0 unstable:0 free:128 slab:364 mapped:2133 pagetables:43
\rDMA free:512kB min:512kB low:640kB high:768kB active:9184kB inactive:500kB present:16384kB pages_scanned:10106 all_unreclaimable? yes
\rlowmem_reserve[]: 0 0 0
\rNormal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
\rlowmem_reserve[]: 0 0 0
\rHighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
\rlowmem_reserve[]: 0 0 0
\rDMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 1*512kB 0*1024kB = 512kB
\rNormal: empty
\rHighMem: empty
\rSwap cache: add 0, delete 0, find 0/0, race 0+0
\rFree swap  = 0kB
\rTotal swap = 0kB
\rFree swap:            0kB
\r4096 pages of RAM
\r200 free pages
\r625 reserved pages
\r364 slab pages
\r40 pages shared
\r0 pages swap cached
\r, OOM: Out of Memory would have been called....VM: killing process test1






















[-- Attachment #3: test1.c --]
[-- Type: application/octet-stream, Size: 319 bytes --]

#include<stdio.h>
#include<string.h>

main()
{
	char* buff;
	int count;
	
	count=0;
	while(1)
	{
		printf("\nOOM Test: Counter = %d", count);
		buff = (char*) malloc(1024);
	//	memset(buff,'\0',1024);
		count++;
			
		if (buff==NULL)
		{
			printf("\nOOM Test: Memory allocation error");
		}
	}
}

Re: Commenting out out_of_memory() function in __alloc_pages()

[Robin Holt]

I am not sure about x86, but on ia64, you would be very hard pressed
for this application to actually run you out of memory.  With the
memset commented out, you would be allocating vmas, etc, but you
would not be actually putting pages behind those virtual addresses.

Thanks,
Robin

---------------------------  test1.c  ----------------------------------

#include<stdio.h>
#include<string.h>

main()
{
	char* buff;
	int count;
	
	count=0;
	while(1)
	{
		printf("\nOOM Test: Counter = %d", count);
		buff = (char*) malloc(1024);
	//	memset(buff,'\0',1024);
		count++;
			
		if (buff==NULL)
		{
			printf("\nOOM Test: Memory allocation error");
		}
	}
}
-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

Re: Commenting out out_of_memory() function in __alloc_pages()

[Chase Venters]

On Fri, 7 Jul 2006, Abu M. Muttalib wrote:

> Hi,
>
> I am getting the Out of memory.
>
> To circumvent the problem, I have commented the call to "out_of_memory(),
> and replaced "goto restart" with "goto nopage".
>
> At "nopage:" lable I have added a call to "schedule()" and then "return
> NULL" after "schedule()".

I wouldn't recommend gutting the oom killer...

> I tried the modified kernel with a test application, the test application is
> mallocing memory in a loop. Unlike as expected the process gets killed. On
> second run of the same application I am getting the page allocation failure
> as expected but subsequently the system hangs.
>
> I am attaching the test application and the log herewith.
>
> I am getting this exception with kernel 2.6.13. With kernel
> 2.4.19-rmka7-pxa1 there was no problem.
>
> Why its so? What can I do to alleviate the OOM problem?

First you should know what is causing them. Is an application leaking 
memory, or is the kernel leaking memory? "ps" can help you answer the 
first question, while "watch cat /proc/meminfo" can help you answer the 
second.

If kernel memory usage seems to be rising steadily over time, report it as 
a bug. Otherwise, fix the broken application.

The reason for the "OOM killer" is because Linux does "VM overcommit". 
Please read "Documentation/vm/overcommit-accounting" for more information, 
including what you'll need if you want to disable "VM overcommit" to 
hopefully stop the OOM killer from coming around.

(When using VM overcommit, the OOM killer is very necessary for a healthy 
system... sometimes the kernel _needs_ memory, and you can't tell it NO. 
In those cases, the OOM killer is invoked to find something to 
sacrifice...)

> Thanks in anticipation and regards,
> Abu.
>

Thanks,
Chase
-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

Script to check a process

[Diego Alencar Alves de Lima]

Hello,

I have a system that I monitor remotely but I have a problem: I need to have 3 
processes running on it all the time (actually, it's the same process running 
3 times) and I can't be watching the system 24/7.

How can I make a script that will check to see if a certain process is 
running? The process name is "nspluginviewer" and there should be 3 of it 
running (each one called from a konqueror process).  Then it would reboot the 
computer if it didn't detect the three running.

Thanks
-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

Re: Script to check a process

[Frederik Deweerdt]

On Sat, Jul 08, 2006 at 01:09:48AM -0300, Diego Alencar Alves de Lima wrote:
> How can I make a script that will check to see if a certain process is 
> running? The process name is "nspluginviewer" and there should be 3 of it 
> running (each one called from a konqueror process).  Then it would reboot the 
> computer if it didn't detect the three running.
> 
There are many ways to do it, one could be:

while [ $(pidof <command> | awk '{print NF;}') -eq <nr occurences> ]; do sleep 1; done ; shutdown -r now

Regards,
Frederik
-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

RE: Commenting out out_of_memory() function in __alloc_pages()

[Abu M. Muttalib]

The funniest part is that with memset commented out_of_memory observed,
contrary to my expectation.

I don't know why. It shouldn't have. I am running the application on an ARM
target.

Regards,
Abu.

-----Original Message-----
From: Robin Holt [mailto:holt@sgi.com]
Sent: Friday, July 07, 2006 3:25 PM
To: Abu M. Muttalib
Cc: kernelnewbies@nl.linux.org; linux-newbie@vger.kernel.org;
linux-kernel@vger.kernel.org; linux-mm
Subject: Re: Commenting out out_of_memory() function in __alloc_pages()


I am not sure about x86, but on ia64, you would be very hard pressed
for this application to actually run you out of memory.  With the
memset commented out, you would be allocating vmas, etc, but you
would not be actually putting pages behind those virtual addresses.

Thanks,
Robin

---------------------------  test1.c  ----------------------------------

#include<stdio.h>
#include<string.h>

main()
{
	char* buff;
	int count;

	count=0;
	while(1)
	{
		printf("\nOOM Test: Counter = %d", count);
		buff = (char*) malloc(1024);
	//	memset(buff,'\0',1024);
		count++;

		if (buff==NULL)
		{
			printf("\nOOM Test: Memory allocation error");
		}
	}
}

-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

RE: Commenting out out_of_memory() function in __alloc_pages()

[Abu M. Muttalib]

I am not sure about x86, but on ia64, you would be very hard pressed
for this application to actually run you out of memory.  With the
memset commented out, you would be allocating vmas, etc, but you
would not be actually putting pages behind those virtual addresses.

---------------------------  test1.c  ----------------------------------

#include<stdio.h>
#include<string.h>

main()
{
	char* buff;
	int count;

	count=0;
	while(1)
	{
		printf("\nOOM Test: Counter = %d", count);
		buff = (char*) malloc(1024);
	//	memset(buff,'\0',1024);
		count++;

		if (buff==NULL)
		{
			printf("\nOOM Test: Memory allocation error");
		}
	}
}

---------------------------  test1.c  ----------------------------------

>The funniest part is that with memset commented out_of_memory observed,
contrary to my expectation.
>
>I don't know why. It shouldn't have. I am running the application on an ARM
target.

>Regards,
>Abu.

I fail to understand that why the OS doesn't return NULL as per man pages of
malloc. It insteat results in OOM.

~Abu.

RE: Commenting out out_of_memory() function in __alloc_pages()

[Abu M. Muttalib]

Where the function alloc_pages is defined? A silly question, but I am not
able to find out.

Please help.

Thanks and regards,
~Abu.

-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

RE: Commenting out out_of_memory() function in __alloc_pages()

[Chase Venters]

On Tue, 11 Jul 2006, Abu M. Muttalib wrote:

>
> I fail to understand that why the OS doesn't return NULL as per man pages of
> malloc. It insteat results in OOM.

Well, my "malloc" manpage describes the Linux behavior under "Bugs", 
though it gives the overcommit practice a harsh and unfair treatment. Let 
me give you an example of why the OS behaves in this way.

Say I've got an Apache web server that is going to fork() 10 children. 
Under traditional fork() semantics, you need 10 copies of all of the 
memory holding stuff like configuration structures, etc. There are two 
reasons why we might not want 10 copies:

1. Some of those pages of data won't change. So why have 10 copies that 
you're going to have to constantly move in and out of cache?

2. Why waste that memory in the first place?

Now, if we were just worried about #1, we could "reserve" room for 9 
copies and still share the single copy (in a CoW scheme). But the act of 
reserving the room would probably just slow fork() down needlessly (when 
fork() is one of the most common and possibly expensive system calls).

Now apps get overcommitted memory too because they do things like ask for 
a ton of memory and then not use it, or use it gradually... in either 
case, Linux (by default) gambles that it can make better choices.

And it turns out that in 999 out of 1000 cases, it can.

If you want strict malloc(), you can use the sysctl to turn off 
overcommit. It's even appropriate to do so for certain applications.

> ~Abu.
>

Thanks,
Chase
-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

Re: Commenting out out_of_memory() function in __alloc_pages()

[Mouhammad Tayseer Alquoatli]

[-- Attachment #1: Type: text/plain, Size: 232 bytes --]

On 7/11/06, Abu M. Muttalib <abum@aftek.com> wrote:
>
> Where the function alloc_pages is defined? A silly question, but I am not
> able to find out.


Hi
it's defined in /include/linux/gfp.h


>
>


-- 
Mouhammad Tayseer Alquoatli

[-- Attachment #2: Type: text/html, Size: 690 bytes --]

RE: Commenting out out_of_memory() function in __alloc_pages()

[Abu M. Muttalib]

Hi,

In the same context I want to know whether kernel 2.6.13 broken?

I got oom, and as per the documentation and code of out_of_memory, all the
processes, which share the same memory address as the process chosen to be
killed, i.e. threads, will be killed. But in my case its not the case
always.

At times the process chosen, "Angelia", is killed will all its threads but
at other times, only some of the threads get killed and the print from oom
killer (function out_of_memory) comes repeatedly but no process is killed.
Why its so?

Anticipation and regards,
Abu.

----------------------------------------------------------------------------
----------------------------------------------------------------
oom-killer: gfp_mask=0x201d0, order=0
Mem-info:
DMA per-cpu:
cpu 0 hot: low 2, high 6, batch 1 used:2
cpu 0 cold: low 0, high 2, batch 1 used:1
Normal per-cpu: empty
HighMem per-cpu: empty
Free pages:         712kB (0kB HighMem)
Active:1359 inactive:122 dirty:0 writeback:0 unstable:0 free:178 slab:469
mapped:1244 pagetables:452
DMA free:712kB min:512kB low:640kB high:768kB active:5436kB inactive:488kB
present:16384kB pages_scanned:5261 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB
pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB
present:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
DMA: 48*4kB 7*8kB 1*16kB 0*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB =
712kB
Normal: empty
HighMem: empty
Swap cache: add 0, delete 0, find 0/0, race 0+0
Free swap  = 0kB
Total swap = 0kB
Free swap:            0kB
4096 pages of RAM
285 free pages
631 reserved pages
469 slab pages
950 pages shared
0 pages swap cached
Out of Memory: Killed process 1137 (Angelia).
oom-killer: gfp_mask=0x201d2, order=0
Mem-info:
DMA per-cpu:
cpu 0 hot: low 2, high 6, batch 1 used:2
cpu 0 cold: low 0, high 2, batch 1 used:1
Normal per-cpu: empty
HighMem per-cpu: empty
Free pages:         712kB (0kB HighMem)
Active:1308 inactive:173 dirty:0 writeback:0 unstable:0 free:178 slab:469
mapped:1244 pagetables:452
DMA free:712kB min:512kB low:640kB high:768kB active:5232kB inactive:692kB
present:16384kB pages_scanned:7971 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0
Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB
pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB
present:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
DMA: 48*4kB 7*8kB 1*16kB 0*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB =
712kB
Normal: empty
HighMem: empty
Swap cache: add 0, delete 0, find 0/0, race 0+0
Free swap  = 0kB
Total swap = 0kB
Free swap:            0kB
4096 pages of RAM
285 free pages
631 reserved pages
469 slab pages
950 pages shared
0 pages swap cached
oom-killer: gfp_mask=0x201d2, order=0
Mem-info:
DMA per-cpu:
cpu 0 hot: low 2, high 6, batch 1 used:2
cpu 0 cold: low 0, high 2, batch 1 used:1
Normal per-cpu: empty
HighMem per-cpu: empty
Free pages:         712kB (0kB HighMem)
Active:1308 inactive:173 dirty:0 writeback:0 unstable:0 free:178 slab:469
mapped:1244 pagetables:452
DMA free:712kB min:512kB low:640kB high:768kB active:5232kB inactive:692kB
present:16384kB pages_scanned:7971 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0
Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB
pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB
present:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
DMA: 48*4kB 7*8kB 1*16kB 0*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB =
712kB
Normal: empty
HighMem: empty
Swap cache: add 0, delete 0, find 0/0, race 0+0
Free swap  = 0kB
Total swap = 0kB
Free swap:            0kB
4096 pages of RAM
285 free pages
631 reserved pages
469 slab pages
950 pages shared
0 pages swap cached
oom-killer: gfp_mask=0x201d2, order=0
Mem-info:
DMA per-cpu:
cpu 0 hot: low 2, high 6, batch 1 used:2
cpu 0 cold: low 0, high 2, batch 1 used:0
Normal per-cpu: empty
HighMem per-cpu: empty
Free pages:         512kB (0kB HighMem)
Active:1314 inactive:232 dirty:0 writeback:0 unstable:0 free:128 slab:470
mapped:1250 pagetables:452
DMA free:512kB min:512kB low:640kB high:768kB active:5256kB inactive:928kB
present:16384kB pages_scanned:7971 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0
Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB
pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB
present:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
DMA: 0*4kB 6*8kB 1*16kB 0*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB =
512kB
Normal: empty
HighMem: empty
Swap cache: add 0, delete 0, find 0/0, race 0+0
Free swap  = 0kB
Total swap = 0kB
Free swap:            0kB
4096 pages of RAM
234 free pages
631 reserved pages
470 slab pages
989 pages shared
0 pages swap cached
oom-killer: gfp_mask=0x201d2, order=0
Mem-info:
DMA per-cpu:
cpu 0 hot: low 2, high 6, batch 1 used:2
cpu 0 cold: low 0, high 2, batch 1 used:0
Normal per-cpu: empty
HighMem per-cpu: empty
Free pages:         512kB (0kB HighMem)
Active:1314 inactive:232 dirty:0 writeback:0 unstable:0 free:128 slab:470
mapped:1250 pagetables:452
DMA free:512kB min:512kB low:640kB high:768kB active:5256kB inactive:928kB
present:16384kB pages_scanned:7971 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0
Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB
pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB
present:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
DMA: 0*4kB 6*8kB 1*16kB 0*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB =
512kB
Normal: empty
HighMem: empty
Swap cache: add 0, delete 0, find 0/0, race 0+0
Free swap  = 0kB
Total swap = 0kB
Free swap:            0kB
4096 pages of RAM
234 free pages
631 reserved pages
470 slab pages
989 pages shared
0 pages swap cached
oom-killer: gfp_mask=0x201d2, order=0
Mem-info:
DMA per-cpu:
cpu 0 hot: low 2, high 6, batch 1 used:2
cpu 0 cold: low 0, high 2, batch 1 used:0
Normal per-cpu: empty
HighMem per-cpu: empty
Free pages:         512kB (0kB HighMem)
Active:1314 inactive:232 dirty:0 writeback:0 unstable:0 free:128 slab:470
mapped:1250 pagetables:452
DMA free:512kB min:512kB low:640kB high:768kB active:5256kB inactive:928kB
present:16384kB pages_scanned:7971 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0
Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB
pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB
present:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
DMA: 0*4kB 6*8kB 1*16kB 0*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB =
512kB
Normal: empty
HighMem: empty
Swap cache: add 0, delete 0, find 0/0, race 0+0
Free swap  = 0kB
Total swap = 0kB
Free swap:            0kB
4096 pages of RAM
234 free pages
631 reserved pages
470 slab pages
989 pages shared
0 pages swap cached
oom-killer: gfp_mask=0x201d2, order=0
Mem-info:
DMA per-cpu:
cpu 0 hot: low 2, high 6, batch 1 used:2
cpu 0 cold: low 0, high 2, batch 1 used:0
Normal per-cpu: empty
HighMem per-cpu: empty
Free pages:         512kB (0kB HighMem)
Active:1314 inactive:232 dirty:0 writeback:0 unstable:0 free:128 slab:470
mapped:1250 pagetables:452
DMA free:512kB min:512kB low:640kB high:768kB active:5256kB inactive:928kB
present:16384kB pages_scanned:7971 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0
Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB
pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB
present:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
DMA: 0*4kB 6*8kB 1*16kB 0*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB =
512kB
Normal: empty
HighMem: empty
Swap cache: add 0, delete 0, find 0/0, race 0+0
Free swap  = 0kB
Total swap = 0kB
Free swap:            0kB
4096 pages of RAM
234 free pages
631 reserved pages
470 slab pages
989 pages shared
0 pages swap cached
oom-killer: gfp_mask=0x201d2, order=0
Mem-info:
DMA per-cpu:
cpu 0 hot: low 2, high 6, batch 1 used:2
cpu 0 cold: low 0, high 2, batch 1 used:0
Normal per-cpu: empty
HighMem per-cpu: empty
Free pages:         512kB (0kB HighMem)
Active:1314 inactive:232 dirty:0 writeback:0 unstable:0 free:128 slab:470
mapped:1250 pagetables:452
DMA free:512kB min:512kB low:640kB high:768kB active:5256kB inactive:928kB
present:16384kB pages_scanned:7971 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0
Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB
pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB
present:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
DMA: 0*4kB 6*8kB 1*16kB 0*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB =
512kB
Normal: empty
HighMem: empty
Swap cache: add 0, delete 0, find 0/0, race 0+0
Free swap  = 0kB
Total swap = 0kB
Free swap:            0kB
4096 pages of RAM
234 free pages
631 reserved pages
470 slab pages
989 pages shared
0 pages swap cached
oom-killer: gfp_mask=0x201d2, order=0
Mem-info:
DMA per-cpu:
cpu 0 hot: low 2, high 6, batch 1 used:2
cpu 0 cold: low 0, high 2, batch 1 used:0
Normal per-cpu: empty
HighMem per-cpu: empty
Free pages:         512kB (0kB HighMem)
Active:1347 inactive:203 dirty:0 writeback:0 unstable:0 free:128 slab:470
mapped:1250 pagetables:452
DMA free:512kB min:512kB low:640kB high:768kB active:5388kB inactive:812kB
present:16384kB pages_scanned:8037 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0
Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB
pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB
present:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
DMA: 0*4kB 6*8kB 1*16kB 0*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB =
512kB
Normal: empty
HighMem: empty
Swap cache: add 0, delete 0, find 0/0, race 0+0
Free swap  = 0kB
Total swap = 0kB
Free swap:            0kB
4096 pages of RAM
234 free pages
631 reserved pages
470 slab pages
985 pages shared
0 pages swap cached
oom-killer: gfp_mask=0x201d2, order=0
Mem-info:
DMA per-cpu:
cpu 0 hot: low 2, high 6, batch 1 used:2
cpu 0 cold: low 0, high 2, batch 1 used:0
Normal per-cpu: empty
HighMem per-cpu: empty
Free pages:         512kB (0kB HighMem)
Active:1347 inactive:203 dirty:0 writeback:0 unstable:0 free:128 slab:470
mapped:1250 pagetables:452
DMA free:512kB min:512kB low:640kB high:768kB active:5388kB inactive:812kB
present:16384kB pages_scanned:8037 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0
Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB
pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB
present:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
DMA: 0*4kB 6*8kB 1*16kB 0*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB =
512kB
Normal: empty
HighMem: empty
Swap cache: add 0, delete 0, find 0/0, race 0+0
Free swap  = 0kB
Total swap = 0kB
Free swap:            0kB
4096 pages of RAM
234 free pages
631 reserved pages
470 slab pages
985 pages shared
0 pages swap cached
----------------------------------------------------------------------------
----------------------------------------------------------------