Memory management problem?

Memory management problem?

[Chris Morrow]

I'm running into a problem where the linux kernel is running
out of memory and starts killing processes in an attempt to
recover. The environment is as follows.

A CDB89712 based system with 16 Meg. sdram and 16 meg CFI flash.
Linux-2.4.19-rmk1 with linux-2.4.19-pre10-shared-zlib patch.

First, on another system I create a number of files with random
contents and calculate their md5sums as follows;

#!/bin/sh

rm randFile*

numFiles=40
blockSize=1024
blocksPerFile=100

proc createFiles () {
	numFiles=$1
	blockSize=$2
	blockPerFile=$3
	fileSize=$(($blocksPerFile * $blockSize))
	while [ $numFiles -gt 0 ]; do
     		echo "creating file $numFiles"
     		dd if=/dev/urandom of=randFile.$numFiles \
			bs=$blockSize count=$blocksPerFile
     		let $((numFiles--))
		done
}

createFiles 40 1024 100

tar cf rand.tar randFile*
md5sum randFile* > rand-md5sum

Next, run the following on the test machine.

#!/bin/sh

while true; do
	date
	tar -xvf /home/yotta/crm/junk/jffs2test/rand.tar
	/home/yotta/crm/junk/jffs2test/verify.sh
done


After about 3 iterations, the kernel will start killing
processes.

There are some "deflateInit failed" warning messages during
the run.

Keeping the total size of the files to about 4 meg but changing
the size of the individual files, I have seen the memory
problem.

Anybody got any ideas?

-- 
Chris Morrow	YottaYotta Inc. email: cmorrow@yottayotta.com
phone: (780) 989 6814 web:	http:  //www.yottayotta.com

Re: Memory management problem?

[David Woodhouse]

cmorrow@YottaYotta.com said:
> A CDB89712 based system with 16 Meg. sdram and 16 meg CFI flash.
> Linux-2.4.19-rmk1 with linux-2.4.19-pre10-shared-zlib patch. 

cmorrow@YottaYotta.com said:
>  There are some "deflateInit failed" warning messages during
> the run.

You shouldn't see that if you're actually using the shared zlib patch.


--
dwmw2

Re: Memory management problem?

[Piotr Trojanek]

there are two issues about out of memory:
1) you get malloc() with null
2) kernel decides to kill something

1: this should be checked in program code
2: look in /usr/src/linux/mm/oom.c

On Tue, Oct 01, 2002 at 04:09:47PM -0600, Chris Morrow wrote:
> 
> I'm running into a problem where the linux kernel is running
> out of memory and starts killing processes in an attempt to
> recover. The environment is as follows.
> 
> A CDB89712 based system with 16 Meg. sdram and 16 meg CFI flash.
> Linux-2.4.19-rmk1 with linux-2.4.19-pre10-shared-zlib patch.
> 
> First, on another system I create a number of files with random
> contents and calculate their md5sums as follows;
> 
> #!/bin/sh
> 
> rm randFile*
> 
> numFiles=40
> blockSize=1024
> blocksPerFile=100
> 
> proc createFiles () {
> 	numFiles=$1
> 	blockSize=$2
> 	blockPerFile=$3
> 	fileSize=$(($blocksPerFile * $blockSize))
> 	while [ $numFiles -gt 0 ]; do
>     		echo "creating file $numFiles"
>     		dd if=/dev/urandom of=randFile.$numFiles \
> 			bs=$blockSize count=$blocksPerFile
>     		let $((numFiles--))
> 		done
> }
> 
> createFiles 40 1024 100
> 
> tar cf rand.tar randFile*
> md5sum randFile* > rand-md5sum
> 
> Next, run the following on the test machine.
> 
> #!/bin/sh
> 
> while true; do
> 	date
> 	tar -xvf /home/yotta/crm/junk/jffs2test/rand.tar
> 	/home/yotta/crm/junk/jffs2test/verify.sh
> done
> 
> 
> After about 3 iterations, the kernel will start killing
> processes.
> 
> There are some "deflateInit failed" warning messages during
> the run.
> 
> Keeping the total size of the files to about 4 meg but changing
> the size of the individual files, I have seen the memory
> problem.
> 
> Anybody got any ideas?
> 
> -- 
> Chris Morrow	YottaYotta Inc. email: cmorrow@yottayotta.com
> phone: (780) 989 6814 web:	http:  //www.yottayotta.com
> 
> 
> 
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/

-- 
Z powazaniem,
Piotr Trojanek
Zespol Informatyki
ELPROMA Elektronika