Memory Problem

Memory Problem

[Philipp Steinkrueger]

Hello,


i have a problem with memory management and i coulndt find a solution
doing web-research, so perhaps someone on the list can point me to a
document or give me another answer.

first, this is my system configuration:

linux kernel 2.4.19 with high-memory enabled because i put 2 GB Ram.
128 MB Swap Partition and a 250 MB Swap File.


The Problem appears with the mysql database server. here is the error
message:

Can't create a new thread (errno 11). If you are not out of available
memory, you
can consult the manual for a possible OS-dependent bug

and this is what cat /proc/meminfo shows while the system runs regulary:

MemTotal:      2069560 kB
MemFree:         24576 kB
MemShared:           0 kB
Buffers:         80808 kB
Cached:        1271288 kB
SwapCached:       1924 kB
Active:         737160 kB
Inactive:       861112 kB
HighTotal:     1179584 kB
HighFree:         2044 kB
LowTotal:       889976 kB
LowFree:         22532 kB
SwapTotal:      384476 kB
SwapFree:       379248 kB


Memfree is at 24 MB but i caught a meminfo when it was about 4 or 6 MB.

i am worried about the high "Cached" value. Could someone tell me if the
cached
content is to be considered as "reserved" and will not be used by programms
requesting
more memory ?

my general questions are:

1) how can i control the memory ? where can i tell the system how much
memory should be
used to cache stuff ? perhaps i can solve the problem this way.
2) what else does the kernel do when a programm spawns a new thread ? if
memory is
not the problem, what else could go wrong when creating a thread ?


If you need something else, like a memstat output, please let me know.


Thank you very much,
Philipp

Memory Problem

[Philipp Steinkrueger]

Hello,


i have a problem with memory management and i coulndt find a solution
doing web-research, so perhaps someone on the list can point me to a
document or give me another answer.

first, this is my system configuration:

linux kernel 2.4.19 with high-memory enabled because i put 2 GB Ram.
128 MB Swap Partition and a 250 MB Swap File.


The Problem appears with the mysql database server. here is the error
message:

Can't create a new thread (errno 11). If you are not out of available
memory, you
can consult the manual for a possible OS-dependent bug

and this is what cat /proc/meminfo shows while the system runs regulary:

MemTotal:      2069560 kB
MemFree:         24576 kB
MemShared:           0 kB
Buffers:         80808 kB
Cached:        1271288 kB
SwapCached:       1924 kB
Active:         737160 kB
Inactive:       861112 kB
HighTotal:     1179584 kB
HighFree:         2044 kB
LowTotal:       889976 kB
LowFree:         22532 kB
SwapTotal:      384476 kB
SwapFree:       379248 kB


Memfree is at 24 MB but i caught a meminfo when it was about 4 or 6 MB.

i am worried about the high "Cached" value. Could someone tell me if the
cached
content is to be considered as "reserved" and will not be used by programms
requesting
more memory ?

my general questions are:

1) how can i control the memory ? where can i tell the system how much
memory should be
used to cache stuff ? perhaps i can solve the problem this way.
2) what else does the kernel do when a programm spawns a new thread ? if
memory is
not the problem, what else could go wrong when creating a thread ?


If you need something else, like a memstat output, please let me know.


Thank you very much,
Philipp

Re: Memory Problem

[Rik van Riel]

On Mon, 7 Oct 2002, Philipp Steinkrueger wrote:

> The Problem appears with the mysql database server. here is the error
> message:
>
> Can't create a new thread (errno 11). If you are not out of available
> memory, you can consult the manual for a possible OS-dependent bug

> 2) what else does the kernel do when a programm spawns a new thread ? if
> memory is not the problem, what else could go wrong when creating a
> thread ?

There are two limits you could be running into:

1) you run into the per-user thread limit or the system-wide
   thread limit

2) memory fragmentation, there is no area of 2 contiguous free pages

regards,

Rik
-- 
Bravely reimplemented by the knights who say "NIH".

http://www.surriel.com/		http://distro.conectiva.com/

Spamtraps of the month:  september@surriel.com trac@trac.org

Re: Memory Problem

[Rik van Riel]

On Mon, 7 Oct 2002, Philipp Steinkrueger wrote:

> The Problem appears with the mysql database server. here is the error
> message:
>
> Can't create a new thread (errno 11). If you are not out of available
> memory, you can consult the manual for a possible OS-dependent bug

> 2) what else does the kernel do when a programm spawns a new thread ? if
> memory is not the problem, what else could go wrong when creating a
> thread ?

There are two limits you could be running into:

1) you run into the per-user thread limit or the system-wide
   thread limit

2) memory fragmentation, there is no area of 2 contiguous free pages

regards,

Rik
-- 
Bravely reimplemented by the knights who say "NIH".

http://www.surriel.com/		http://distro.conectiva.com/

Spamtraps of the month:  september@surriel.com trac@trac.org

Re: Memory Problem

[Dave Hansen]

Rik van Riel wrote:
> On Mon, 7 Oct 2002, Philipp Steinkrueger wrote:
> 
> 
>>The Problem appears with the mysql database server. here is the error
>>message:
>>
>>Can't create a new thread (errno 11). If you are not out of available
>>memory, you can consult the manual for a possible OS-dependent bug
> 
> 
>>2) what else does the kernel do when a programm spawns a new thread ? if
>>memory is not the problem, what else could go wrong when creating a
>>thread ?
> 
> 2) memory fragmentation, there is no area of 2 contiguous free pages

If you're running current 2.5, you can check /proc/buddyinfo to detect 
fragmentation.  If there are any non-zero numbers in columns after the 
first one in ZONE_NORMAL or ZONE_DMA, you should be all right

-- 
Dave Hansen
haveblue@us.ibm.com

Re: Memory Problem

[Glynn Clements]

Philipp Steinkrueger wrote:

> i have a problem with memory management and i coulndt find a solution
> doing web-research, so perhaps someone on the list can point me to a
> document or give me another answer.
> 
> first, this is my system configuration:
> 
> linux kernel 2.4.19 with high-memory enabled because i put 2 GB Ram.
> 128 MB Swap Partition and a 250 MB Swap File.
> 
> 
> The Problem appears with the mysql database server. here is the error
> message:
> 
> Can't create a new thread (errno 11). If you are not out of available
> memory, you
> can consult the manual for a possible OS-dependent bug
> 
> and this is what cat /proc/meminfo shows while the system runs regulary:
> 
> MemTotal:      2069560 kB
> MemFree:         24576 kB
> MemShared:           0 kB
> Buffers:         80808 kB
> Cached:        1271288 kB
> SwapCached:       1924 kB
> Active:         737160 kB
> Inactive:       861112 kB
> HighTotal:     1179584 kB
> HighFree:         2044 kB
> LowTotal:       889976 kB
> LowFree:         22532 kB
> SwapTotal:      384476 kB
> SwapFree:       379248 kB
> 
> 
> Memfree is at 24 MB but i caught a meminfo when it was about 4 or 6 MB.
> 
> i am worried about the high "Cached" value. Could someone tell me if the
> cached
> content is to be considered as "reserved" and will not be used by programms
> requesting
> more memory ?

No, the OS will use the "cached" memory if an application wants it. 
"free" memory basically means "wasted", i.e. it isn't used for
anything.

IOW, you aren't even close to running out of memory; you have around
1.2Gb available.

-- 
Glynn Clements <glynn.clements@virgin.net>

memory problem

[최영일]

I'm trying to boot with ppc-kernel(2.4.21-pre5)
I'm trying to boot to the platform for our team to develop.
I add some driver to the kernel.
Unhappily, the size of my driver is over 3M bytes.
So, My total kernel size is over 5M(5140376 byte).
Then, My system(440 cpu) can not boot
Just kernel(no my driver or with small size my driver) can boot.
I don't know reason.
 does Linux have limit to kernel size?
Please, let me know why to happen?

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Memory problem

[Filip Sneppe]

On Thu, 2003-07-03 at 11:53, Cilliè Burger wrote:
> Hi Everyone
> 
> I was wondering if anyone has a solution to this problem.
> 
> I have a the following box that sits between our router and switch:
> 
> Pentium 200, 64 Mbyte RAM, Linux version 2.4.18-3 
> (bhcompile@stripples.devel.redhat.com)
> (gcc version 2.96 20000731 (Red Hat Linux 7.3 2.96-110)), iptables v1.2.5
> 
> I almost never reboot this box, but lately I have noticed a dramatic 
> increase in memory consumption.
> 
> I start out on bootup with about 40 MB or so free and in a weeks time 
> its down to about 800KB.
> When iptables is restarted and the rules flushed and reloaded I reclaim 
> about 6024 KB, which then gradually
> decreases back to about a meg in a 16 hour period.
> 
...
> 
> Why does iptables consume so much memory ?
> Why does iptables appear to loose so much memory ? When regarding this 
> question, consider the following:
> 
> On reboot and before loading of rules there is about 40 MB free ram. 
> After loading the rules, and about two weeks uptime
> there is about 800KB of free memory. After flushing the rules, theres 
> only 6024 KB free.
> Is there a slight possibility that this may be due to a memory leak of 
> some sort ?
> 
> Thanks in advance for your help. Keep up the good work Netfilter .
> 

Hi Cilliè,

I understand your concerns about memory consumption, but there is
no information in your mail showing that the memory used by the
firewall is in fact used by connection tracking or any other netfilter
kernel structures. 

In fact, many Linux admins will tell you that any Linux box that has
free memory after system boot will end up using all available memory
after a little while: that memory is simply used for buffering and
caching filesystem operations.

So in order to get an idea about your box' memory consumption, send us
the output of:

cat /proc/meminfo
cat /proc/slabinfo
wc -l /proc/net/ip_conntrack

Regards,
Filip

Memory problem

[Cilliè Burger]

Hi Everyone

I was wondering if anyone has a solution to this problem.

I have a the following box that sits between our router and switch:

Pentium 200, 64 Mbyte RAM, Linux version 2.4.18-3 
(bhcompile@stripples.devel.redhat.com)
(gcc version 2.96 20000731 (Red Hat Linux 7.3 2.96-110)), iptables v1.2.5

I almost never reboot this box, but lately I have noticed a dramatic 
increase in memory consumption.

I start out on bootup with about 40 MB or so free and in a weeks time 
its down to about 800KB.
When iptables is restarted and the rules flushed and reloaded I reclaim 
about 6024 KB, which then gradually
decreases back to about a meg in a 16 hour period.

I run about 400 rules on this box and ipt_conntrack_max is set at 4096.


I do want to add more memory to the box, but i have this strange feeling 
that it will just consume all of that aswell until
it reaches some kind of lower limit on allowable free memory.

Unfortunately I am not sure of how to count the number of simultaneous 
connection
but since we run a few mail and web-servers and also a few busy dns servers.
I estimate that there are about 300 connections per second.

My questions, if anyone has payed attention thus far :)

Why does iptables consume so much memory ?
Why does iptables appear to loose so much memory ? When regarding this 
question, consider the following:

On reboot and before loading of rules there is about 40 MB free ram. 
After loading the rules, and about two weeks uptime
there is about 800KB of free memory. After flushing the rules, theres 
only 6024 KB free.
Is there a slight possibility that this may be due to a memory leak of 
some sort ?

Thanks in advance for your help. Keep up the good work Netfilter .

Regards,

Cilliiè Burger
SA-DOMAIN Internet Services

Re: Memory problem

[Filip Sneppe]

Hi Cilliè,

On Thu, 2003-07-03 at 17:04, Cilliè Burger wrote:

> Here are the details you requested.
> 
>  cat /proc/meminfo
>          total:    used:    free:  shared: buffers:  cached:
> Mem:  63598592 62529536  1069056        0 23306240 19935232
> Swap: 200237056  3911680 196325376
> MemTotal:        62108 kB
> MemFree:          1044 kB
> MemShared:           0 kB
> Buffers:         22760 kB
> Cached:          19192 kB
> SwapCached:        276 kB
> Active:          26476 kB
> Inact_dirty:     14428 kB
> Inact_clean:      2428 kB
> Inact_target:     8664 kB
> HighTotal:           0 kB
> HighFree:            0 kB
> LowTotal:        62108 kB
> LowFree:          1044 kB
> SwapTotal:      195544 kB
> SwapFree:       191724 kB
> Committed_AS:     6052 kB
> 
> ------------------------------------------------------------
> 
> 
>  cat /proc/slabinfo
> slabinfo - version: 1.1
> kmem_cache            61     70    112    2    2    1
> ip_conntrack        1193   1628    352  127  148    1
...
> inode_cache        17328  19976    496 2497 2497    1
> dentry_cache       17522  23275    112  665  665    1
...
> buffer_head         9797  12040     96  272  301    1
...

> 
> wc -l /proc/net/ip_conntrack
>    1107 /proc/net/ip_conntrack
> 

See, ip_conntrack is only using 1193 * 352 bytes per connection, 
and the number of connections tracked is quite reasonable.

You can also see that a fair amount of RAM is simply used
for caching of filesytem operations (see cached: entry
of meminfo output and the inode_cache and dentry_cache
in slabinfo output).

So I wouldn't worry about any memory leaks in netfilter
connection tracking. The numbers you provided look quite normal.

Regards,
Filip

RE: Memory problem

[Daniel Chemko]

Speaking of missing memory, I am having a fun time trying to find where I am loosing memory. It seems like a progressive leak, but I am not sure that it isn't just some internal caching mechanism.. What you all think?

#cat /proc/meminfo
        total:    used:    free:  shared: buffers:  cached:
Mem:  195235840 190230528  5005312        0  1622016 16596992
Swap: 3224289280  8986624 3215302656
MemTotal:       190660 kB
MemFree:          4888 kB
MemShared:           0 kB
Buffers:          1584 kB
Cached:          12944 kB
SwapCached:       3264 kB
Active:          11844 kB
ActiveAnon:       3812 kB
ActiveCache:      8032 kB
Inact_dirty:         0 kB
Inact_laundry:    3756 kB
Inact_clean:      2508 kB
Inact_target:     3620 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:       190660 kB
LowFree:          4888 kB
SwapTotal:     3148720 kB
SwapFree:      3139944 kB

# cat /proc/slabinfo 
slabinfo - version: 1.1
kmem_cache            66     70    108    2    2    1
ip_fib_hash           55    112     32    1    1    1
urb_priv               1     58     64    1    1    1
journal_head         111   5775     48    2   75    1
revoke_table           2    250     12    1    1    1
revoke_record          0    224     32    0    2    1
clip_arp_cache         0      0    128    0    0    1
ip_conntrack         480   1740    384   86  174    1
ip_mrt_cache           0      0    128    0    0    1
tcp_tw_bucket         52    780    128    2   26    1
tcp_bind_bucket       14    112     32    1    1    1
tcp_open_request       0     30    128    0    1    1
inet_peer_cache       23   7830     64    1  135    1
ip_dst_cache         367   6975    256   25  465    1
arp_cache             32    270    128    2    9    1
blkdev_requests     1472   1500    128   50   50    1
dnotify_cache          0      0     20    0    0    1
file_lock_cache        3     41     92    1    1    1
fasync_cache           0      0     16    0    0    1
uid_cache              4    112     32    1    1    1
skbuff_head_cache    230    465    256   25   31    1
sock                  92    117   1280   32   39    1
sigqueue               0     29    132    0    1    1
kiobuf                 0      0     64    0    0    1
cdev_cache            10   2494     64    1   43    1
bdev_cache             5     58     64    1    1    1
mnt_cache             13     58     64    1    1    1
inode_cache          543    679    512   97   97    1
dentry_cache         514   1350    128   45   45    1
dquot                  0      0    128    0    0    1
filp                1270   1290    128   43   43    1
names_cache            0     21   4096    0   21    1
buffer_head         1532  38048     92   95  928    1
mm_struct          34802  34845    256 2322 2323    1
vm_area_struct      1476   6000    128   67  200    1
fs_cache              27    174     64    1    3    1
files_cache           27    119    512    8   17    1
signal_cache          41    174     64    2    3    1
sighand_cache         33    132   1408    5   12    4
task_struct            0      0   1792    0    0    1
pte_chain            527   9570    128   23  319    1
size-131072(DMA)       0      0 131072    0    0   32
size-131072            0      0 131072    0    0   32
size-65536(DMA)        0      0  65536    0    0   16
size-65536             1      3  65536    1    3   16
size-32768(DMA)        0      0  32768    0    0    8
size-32768             0      4  32768    0    4    8
size-16384(DMA)        0      0  16384    0    0    4
size-16384             0      1  16384    0    1    4
size-8192(DMA)         0      0   8192    0    0    2
size-8192              7     24   8192    7   24    2
size-4096(DMA)         0      0   4096    0    0    1
size-4096             54     92   4096   54   92    1
size-2048(DMA)         0      0   2048    0    0    1
size-2048            153    442   2048   77  221    1
size-1024(DMA)         0      0   1024    0    0    1
size-1024             51     76   1024   14   19    1
size-512(DMA)          0      0    512    0    0    1
size-512              56    160    512    8   20    1
size-256(DMA)          0      0    256    0    0    1
size-256              21    945    256    2   63    1
size-128(DMA)          1     30    128    1    1    1
size-128            1155   1230    128   40   41    1
size-64(DMA)           0      0    128    0    0    1
size-64              223    630    128    9   21    1
size-32(DMA)          18     58     64    1    1    1
size-32              292    754     64    8   13    1

-----Original Message-----
From: Filip Sneppe [mailto:filip.sneppe@cronos.be] 
Sent: Thursday, July 03, 2003 7:42 AM
To: Cilliè Burger
Cc: netfilter@lists.netfilter.org
Subject: Re: Memory problem

Hi Cilliè,

On Thu, 2003-07-03 at 17:04, Cilliè Burger wrote:

> Here are the details you requested.
> 
>  cat /proc/meminfo
>          total:    used:    free:  shared: buffers:  cached:
> Mem:  63598592 62529536  1069056        0 23306240 19935232
> Swap: 200237056  3911680 196325376
> MemTotal:        62108 kB
> MemFree:          1044 kB
> MemShared:           0 kB
> Buffers:         22760 kB
> Cached:          19192 kB
> SwapCached:        276 kB
> Active:          26476 kB
> Inact_dirty:     14428 kB
> Inact_clean:      2428 kB
> Inact_target:     8664 kB
> HighTotal:           0 kB
> HighFree:            0 kB
> LowTotal:        62108 kB
> LowFree:          1044 kB
> SwapTotal:      195544 kB
> SwapFree:       191724 kB
> Committed_AS:     6052 kB
> 
> ------------------------------------------------------------
> 
> 
>  cat /proc/slabinfo
> slabinfo - version: 1.1
> kmem_cache            61     70    112    2    2    1
> ip_conntrack        1193   1628    352  127  148    1
...
> inode_cache        17328  19976    496 2497 2497    1
> dentry_cache       17522  23275    112  665  665    1
...
> buffer_head         9797  12040     96  272  301    1
...

> 
> wc -l /proc/net/ip_conntrack
>    1107 /proc/net/ip_conntrack
> 

See, ip_conntrack is only using 1193 * 352 bytes per connection, 
and the number of connections tracked is quite reasonable.

You can also see that a fair amount of RAM is simply used
for caching of filesytem operations (see cached: entry
of meminfo output and the inode_cache and dentry_cache
in slabinfo output).

So I wouldn't worry about any memory leaks in netfilter
connection tracking. The numbers you provided look quite normal.

Regards,
Filip

Memory problem

[Pankaj]

Hi,

I doing some performance test on my application. Platform is

Red Hat Enterprise Linux ES release 3 (Taroon)
Kernel 2.4.21-4.ELsmp on an i686

I have done some load test for my application and afterwards closed my
application. Now
no application is running on my machine. Following is output of my
/proc/meminfo

[root@bplinux89 rsinsp]# cat /proc/meminfo
        total:    used:    free:  shared: buffers:  cached:
Mem:  4224958464 2294509568 1930448896        0 200970240 1705332736
Swap: 2146787328        0 2146787328
MemTotal:      4125936 kB
MemFree:       1885204 kB
MemShared:           0 kB
Buffers:        196260 kB
Cached:        1665364 kB
SwapCached:          0 kB
Active:         473520 kB
ActiveAnon:      40996 kB
ActiveCache:    432524 kB
Inact_dirty:    118716 kB
Inact_laundry: 1222752 kB
Inact_clean:     86976 kB
Inact_target:   380392 kB
HighTotal:     3276716 kB
HighFree:      1689640 kB
LowTotal:       849220 kB
LowFree:        195564 kB
SwapTotal:     2096472 kB
SwapFree:      2096472 kB
HugePages_Total:     0
HugePages_Free:      0
Hugepagesize:     2048 kB


I am really worried about high Inact_laundry and cached memory size. Is it
normal?

Regards,
Pankaj.


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

re: memory problem

[Ron Michael Khu]

what method did u use to allocate memory??

static:
 char array[1024*1024] ;

dynamic
 char *array = (char*)calloc(  1024*1024, sizeof(char) );

and why r u allocating such huge chunks for an array?



Ankit Jain wrote:

>hi
>
>well i am fixed up in a new problem
>
>i am using a array of size 1024*1024
>
>it gives segmentation fault
>
>it works with 512*512
>
>my menory size is 128 mb and 512 swap
>
>i have enough memory space and using gcc 3.2 v
>
>may be i need to increase the memory size
>
>how to do that?
>
>ankit
>
>________________________________________________________________________
>Yahoo! Messenger - Communicate instantly..."Ping" 
>your friends today! Download Messenger Now 
>http://uk.messenger.yahoo.com/download/index.html
>-
>To unsubscribe from this list: send the line "unsubscribe linux-c-programming" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>
>  
>

MEMORY PROBLEM

[ppclinux]

Hello,
I have a Freescale-PPC8270 ( Former Motorola ) based board with 256Megs of
RAM.

I am using the Montavista Vista Kernel 2.4.20.

When the system is handling larger files ~100M when NFS mounted and ~20M
when RAMDISKed the kernel crashes since things are being overwritten in
memory.

The crash happens when handling/creating large files by measn of e.g.
* dd
* gunzip
* sftp download
etc. it seems to be a general memory handling problem.

When looking at the memory though there are plenty of free space, or there
should be plenty of free space but for some reason things are
overwritten...

The Kernel reports actually 256M of memory. Top also shows that there is o
r should be more than enough free memory for the operations I am trying to
pull :-))

Can you give me suggestions where to start digging into this problem?

Best Regards // Matias

Re: MEMORY PROBLEM

[Jerry Van Baren]

ppclinux@sundmangroup.com wrote:
> Hello,
> I have a Freescale-PPC8270 ( Former Motorola ) based board with 256Megs of
> RAM.
> 
> I am using the Montavista Vista Kernel 2.4.20.
> 
> When the system is handling larger files ~100M when NFS mounted and ~20M
> when RAMDISKed the kernel crashes since things are being overwritten in
> memory.
> 
> The crash happens when handling/creating large files by measn of e.g.
> * dd
> * gunzip
> * sftp download
> etc. it seems to be a general memory handling problem.
> 
> When looking at the memory though there are plenty of free space, or there
> should be plenty of free space but for some reason things are
> overwritten...
> 
> The Kernel reports actually 256M of memory. Top also shows that there is o
> r should be more than enough free memory for the operations I am trying to
> pull :-))
> 
> Can you give me suggestions where to start digging into this problem?
> 
> Best Regards // Matias

Your SDRAM initialization is suspect
   http://www.denx.de/twiki/bin/view/DULG/LinuxCrashesRandomly
   http://www.denx.de/twiki/bin/view/DULG/SDRAM
   http://www.denx.de/twiki/bin/view/DULG/UBootCrashAfterRelocation

gvb

Re: MEMORY PROBLEM

[ppclinux]

> Your SDRAM initialization is suspect

Thx for that. Those words made me go through the SDRAM
initialization once again and BINGO the PSDMR register had
an address line setting wrong.

The thing is that we have boards with 128M and 256M and the
PSDMR register settings was the same for both. Other involved
registers were modified by the book.

THX!

Cheers // Matias


> ppclinux@sundmangroup.com wrote:
>> Hello,
>> I have a Freescale-PPC8270 ( Former Motorola ) based board with 256Megs
>> of
>> RAM.
>>
>> I am using the Montavista Vista Kernel 2.4.20.
>>
>> When the system is handling larger files ~100M when NFS mounted and ~20M
>> when RAMDISKed the kernel crashes since things are being overwritten in
>> memory.
>>
>> The crash happens when handling/creating large files by measn of e.g.
>> * dd
>> * gunzip
>> * sftp download
>> etc. it seems to be a general memory handling problem.
>>
>> When looking at the memory though there are plenty of free space, or
>> there
>> should be plenty of free space but for some reason things are
>> overwritten...
>>
>> The Kernel reports actually 256M of memory. Top also shows that there is
>> o
>> r should be more than enough free memory for the operations I am trying
>> to
>> pull :-))
>>
>> Can you give me suggestions where to start digging into this problem?
>>
>> Best Regards // Matias
>
> Your SDRAM initialization is suspect
>    http://www.denx.de/twiki/bin/view/DULG/LinuxCrashesRandomly
>    http://www.denx.de/twiki/bin/view/DULG/SDRAM
>    http://www.denx.de/twiki/bin/view/DULG/UBootCrashAfterRelocation
>
> gvb
> _______________________________________________
> Linuxppc-embedded mailing list
> Linuxppc-embedded@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-embedded
>

Memory problem

[Froggy / Froggy Corp.]

Hi everyone,

I just see a memory problem : i have 1Gb install on my computer and only
275Mb are available when i take a look with "free".
I use kernel 2.6.10, the bios tell me that i really have 1Gb (2*512
DDR), the kernel have 4Gb option enable, so i dont understand.

If someone have an idea.

Thx in advance for any help,
Regards,

Here is some information :
-------------------------------------------------------------------------
Odyssee:~# free
             total       used       free     shared    buffers    
cached
Mem:        275896      51932     223964          0       2976     
23952
-/+ buffers/cache:      25004     250892
Swap:      8433976          0    8433976
-------------------------------------------------------------------------
Odyssee:~# cat /proc/meminfo 
MemTotal:       275896 kB
MemFree:        223516 kB
Buffers:          3124 kB
Cached:          24008 kB
SwapCached:          0 kB
Active:          20620 kB
Inactive:        14740 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:       275896 kB
LowFree:        223516 kB
SwapTotal:     8433976 kB
MemTotal:       275896 kB
MemFree:        223516 kB
Buffers:          3124 kB
Cached:          24008 kB
SwapCached:          0 kB
Active:          20620 kB
Inactive:        14740 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:       275896 kB
LowFree:        223516 kB
SwapTotal:     8433976 kB
SwapFree:      8433976 kB
Dirty:               4 kB
Writeback:           0 kB
Mapped:          15944 kB
Slab:            10680 kB
CommitLimit:   8571924 kB
Committed_AS:   517448 kB
PageTables:        732 kB
VmallocTotal:   745464 kB
VmallocUsed:      4484 kB
VmallocChunk:   740696 kB
-------------------------------------------------------------------------
Linux version 2.6.10 (root@Odyssee) (gcc version 3.3.5 (Debian
1:3.3.5-12)) #3 S
MP Sun Apr 24 16:00:50 CEST 2005
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009d800 (usable)
 BIOS-e820: 000000000009d800 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 0000000011340400 (usable)
 BIOS-e820: 0000000011340400 - 0000000011343400 (ACPI NVS)
 BIOS-e820: 0000000011343400 - 0000000011350400 (ACPI data)
 BIOS-e820: 00000000fec00000 - 0000000100000000 (reserved)
user-defined physical RAM map:
 user: 0000000000000000 - 000000000009d800 (usable)
 user: 000000000009d800 - 00000000000a0000 (reserved)
 user: 00000000000f0000 - 0000000000100000 (reserved)
 user: 0000000000100000 - 0000000011340400 (usable)
 user: 0000000011340400 - 0000000011343400 (ACPI NVS)
 user: 0000000011343400 - 0000000011350400 (ACPI data)
 user: 00000000fec00000 - 0000000100000000 (reserved)
0MB HIGHMEM available.
275MB LOWMEM available.
found SMP MP-table at 000f5650
On node 0 totalpages: 70464
  DMA zone: 4096 pages, LIFO batch:1
  Normal zone: 66368 pages, LIFO batch:16
  HighMem zone: 0 pages, LIFO batch:1
DMI 2.2 present.
ACPI: RSDP (v000 IntelR                                ) @ 0x000f7110
ACPI: RSDT (v001 IntelR AWRDACPI 0x42302e31 AWRD 0x00000000) @
0x3fff3000
ACPI: FADT (v001 IntelR AWRDACPI 0x42302e31 AWRD 0x00000000) @
0x3fff3040
ACPI: MADT (v001 IntelR AWRDACPI 0x42302e31 AWRD 0x00000000) @
0x3fff7280
ACPI: DSDT (v001 INTELR AWRDACPI 0x00001000 MSFT 0x0100000e) @
0x00000000
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
Processor #0 15:3 APIC version 20
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
Processor #1 15:3 APIC version 20
ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
ACPI: IOAPIC (id[0x03] address[0xfec10000] gsi_base[24])
IOAPIC[1]: apic_id 3, version 32, address 0xfec10000, GSI 24-47
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ9 used by override.
Enabling APIC mode:  Flat.  Using 2 I/O APICs
Using ACPI (MADT) for SMP configuration information
Built 1 zonelists
Kernel command line: auto BOOT_IMAGE=linux ro root=901 mem=1024M
mapped APIC to ffffd000 (fee00000)
mapped IOAPIC to ffffc000 (fec00000)
mapped IOAPIC to ffffb000 (fec10000)
Initializing CPU#0
CPU 0 irqstacks, hard=c03c1000 soft=c03bf000
PID hash table entries: 2048 (order: 11, 32768 bytes)
Detected 2800.832 MHz processor.
Using tsc for high-res timesource
Console: colour VGA+ 80x25
Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
Memory: 275576k/281856k available (1828k kernel code, 5752k reserved,
769k data,
 188k init, 0k highmem)
Checking if this processor honours the WP bit even in supervisor mode...
Ok.
Calibrating delay loop... 5537.79 BogoMIPS (lpj=2768896)
Mount-cache hash table entries: 512 (order: 0, 4096 bytes)
CPU: After generic identify, caps: bfebfbff 00000000 00000000 00000000
CPU: After vendor identify, caps:  bfebfbff 00000000 00000000 00000000
monitor/mwait feature present.
using mwait in idle threads.
CPU: Trace cache: 12K uops, L1 D cache: 16K
CPU: L2 cache: 1024K
CPU: Physical Processor ID: 0
CPU: After all inits, caps:        bfebfbff 00000000 00000000 00000080
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#0.
CPU0: Intel P4/Xeon Extended MCE MSRs (12) available
CPU0: Thermal monitoring enabled
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Checking 'hlt' instruction... OK.
CPU0: Intel(R) Pentium(R) 4 CPU 2.80GHz stepping 03
per-CPU timeslice cutoff: 2926.38 usecs.
task migration cache decay timeout: 3 msecs.
Booting processor 1/1 eip 3000
CPU 1 irqstacks, hard=c03c2000 soft=c03c0000
Initializing CPU#1
Calibrating delay loop... 5586.94 BogoMIPS (lpj=2793472)
CPU: After generic identify, caps: bfebfbff 00000000 00000000 00000000
CPU: After vendor identify, caps:  bfebfbff 00000000 00000000 00000000
monitor/mwait feature present.
CPU: Trace cache: 12K uops, L1 D cache: 16K
CPU: L2 cache: 1024K
CPU: Physical Processor ID: 0
CPU: After all inits, caps:        bfebfbff 00000000 00000000 00000080
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#1.
CPU1: Intel P4/Xeon Extended MCE MSRs (12) available
CPU1: Thermal monitoring enabled
CPU1: Intel(R) Pentium(R) 4 CPU 2.80GHz stepping 03
Total of 2 processors activated (11124.73 BogoMIPS).
ENABLING IO-APIC IRQs
..TIMER: vector=0x31 pin1=2 pin2=-1
checking TSC synchronization across 2 CPUs: passed.
Brought up 2 CPUs
CPU0:
 domain 0: span 3
  groups: 1 2
  domain 1: span 3
   groups: 3
CPU1:
 domain 0: span 3
  groups: 2 1
  domain 1: span 3
   groups: 3
NET: Registered protocol family 16
PCI: PCI BIOS revision 2.10 entry at 0xface0, last bus=3
PCI: Using configuration type 1
mtrr: v2.0 (20020519)
ACPI: Subsystem revision 20041105
ACPI: Interpreter enabled
ACPI: Using IOAPIC for interrupt routing
ACPI: PCI Root Bridge [PCI0] (00:00)
PCI: Probing PCI hardware (bus 00)
PCI: Ignoring BAR0-3 of IDE controller 0000:00:1f.1
PCI: Transparent bridge - 0000:00:1e.0
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.HUB0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.HRB_._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 *5 7 9 10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 7 *9 10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 7 9 10 11 *12 14 15)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 7 9 10 11 *12 14 15)
ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 *5 7 9 10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 7 9 10 11 *12 14 15)
ACPI: PCI Interrupt Link [LNK0] (IRQs 3 4 5 7 9 10 11 12 14 15) *0,
disabled.
ACPI: PCI Interrupt Link [LNK1] (IRQs 3 4 5 7 *9 10 11 12 14 15)
SCSI subsystem initialized
PCI: Using ACPI for IRQ routing
** PCI interrupts are no longer routed automatically.  If this
** causes a device to stop working, it is probably because the
** driver failed to call pci_enable_device().  As a temporary
** workaround, the "pci=routeirq" argument restores the old
** behavior.  If this argument makes the device work again,
** please email the output of "lspci" to bjorn.helgaas@hp.com
** so I can fix the driver.
audit: initializing netlink socket (disabled)
audit(1114353108.896:0): initialized
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
Initializing Cryptographic API
serio: i8042 AUX port at 0x60,0x64 irq 12
serio: i8042 KBD port at 0x60,0x64 irq 1
Serial: 8250/16550 driver $Revision: 1.90 $ 8 ports, IRQ sharing
disabled
ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
ttyS2 at I/O 0x3e8 (irq = 4) is a 16550A
ttyS3 at I/O 0x2e8 (irq = 3) is a 16550A
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered
Intel(R) PRO/1000 Network Driver - version 5.5.4-k2
Copyright (c) 1999-2004 Intel Corporation.
ACPI: PCI interrupt 0000:01:01.0[A] -> GSI 18 (level, low) -> IRQ 18
PCI: Setting latency timer of device 0000:01:01.0 to 64
e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
ACPI: PCI interrupt 0000:03:02.0[A] -> GSI 17 (level, low) -> IRQ 17
e1000: eth1: e1000_probe: Intel(R) PRO/1000 Network Connection
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with
idebus=xx
Probing IDE interface ide0...
Probing IDE interface ide1...
ide1: Wait for ready failed before probe !
hdd: FX600S, ATAPI CD/DVD-ROM drive
ide2: I/O resource 0x3EE-0x3EE not free.
ide2: ports already in use, skipping probe
Probing IDE interface ide3...
ide3: Wait for ready failed before probe !
Probing IDE interface ide4...
ide4: Wait for ready failed before probe !
Probing IDE interface ide5...
ide5: Wait for ready failed before probe !
elevator: using anticipatory as default io scheduler
ide1 at 0x170-0x177,0x376 on irq 15
libata version 1.10 loaded.
sata_sil version 0.8
ACPI: PCI interrupt 0000:03:03.0[A] -> GSI 20 (level, low) -> IRQ 20
ata1: SATA max UDMA/100 cmd 0xD2002080 ctl 0xD200208A bmdma 0xD2002000
irq 20
ata2: SATA max UDMA/100 cmd 0xD20020C0 ctl 0xD20020CA bmdma 0xD2002008
irq 20
ata3: SATA max UDMA/100 cmd 0xD2002280 ctl 0xD200228A bmdma 0xD2002200
irq 20
ata4: SATA max UDMA/100 cmd 0xD20022C0 ctl 0xD20022CA bmdma 0xD2002208
irq 20
ata1: no device found (phy stat 00000000)
scsi0 : sata_sil
ata2: no device found (phy stat 00000000)
scsi1 : sata_sil
ata3: dev 0 cfg 49:2f00 82:74eb 83:7fea 84:4023 85:74e8 86:3c02 87:4023
88:203f
ata3: dev 0 ATA, max UDMA/100, 321672960 sectors: lba48
ata3: dev 0 configured for UDMA/100
scsi2 : sata_sil
ata4: no device found (phy stat 00000000)
scsi3 : sata_sil
  Vendor: ATA       Model: HDS722516VLSA80   Rev: V34O
  Type:   Direct-Access                      ANSI SCSI revision: 05
SCSI device sda: 321672960 512-byte hdwr sectors (164697 MB)
SCSI device sda: drive cache: write back
SCSI device sda: 321672960 512-byte hdwr sectors (164697 MB)
SCSI device sda: drive cache: write back
 sda: sda1 sda2 sda3 sda4 < sda5 sda6 sda7 >
Attached scsi disk sda at scsi2, channel 0, id 0, lun 0
Attached scsi generic sg0 at scsi2, channel 0, id 0, lun 0,  type 0
mice: PS/2 mouse device common for all mice
input: AT Translated Set 2 keyboard on isa0060/serio0
md: raid1 personality registered as nr 3
md: md driver 0.90.1 MAX_MD_DEVS=256, MD_SB_DISKS=27
NET: Registered protocol family 2
IP: routing cache hash table of 2048 buckets, 16Kbytes
TCP: Hash tables configured (established 32768 bind 32768)
arp_tables: (C) 2002 David S. Miller
NET: Registered protocol family 1
NET: Registered protocol family 17
Starting balanced_irq
ACPI wakeup devices: 
PCI0 CSAD HUB0 UAR1 UAR2 USB0 USB1 USBE MODM 
ACPI: (supports S0 S1 S4 S5)
md: Autodetecting RAID arrays.
md: autorun ...
md: considering sda7 ...
md:  adding sda7 ...
md: sda6 has different UUID to sda7
md: sda5 has different UUID to sda7
md: sda3 has different UUID to sda7
md: sda2 has different UUID to sda7
md: sda1 has different UUID to sda7
md: created md5
md: bind<sda7>
md: running: <sda7>
raid1: raid set md5 active with 1 out of 2 mirrors
md: considering sda6 ...
md:  adding sda6 ...
md: sda5 has different UUID to sda6
md: sda3 has different UUID to sda6
md: sda2 has different UUID to sda6
md: sda1 has different UUID to sda6
md: created md4
md: bind<sda6>
md: running: <sda6>
raid1: raid set md4 active with 1 out of 2 mirrors
md: considering sda5 ...
md:  adding sda5 ...
md: sda3 has different UUID to sda5
md: sda2 has different UUID to sda5
md: sda1 has different UUID to sda5
md: created md3
md: bind<sda5>
md: running: <sda5>
raid1: raid set md3 active with 1 out of 2 mirrors
md: considering sda3 ...
md:  adding sda3 ...
md: sda2 has different UUID to sda3
md: sda1 has different UUID to sda3
md: created md2
md: bind<sda3>
md: running: <sda3>
raid1: raid set md2 active with 1 out of 2 mirrors
md: considering sda2 ...
md:  adding sda2 ...
md: sda1 has different UUID to sda2
md: created md1
md: bind<sda2>
md: running: <sda2>
raid1: raid set md1 active with 1 out of 2 mirrors
md: considering sda1 ...
md:  adding sda1 ...
md: created md0
md: bind<sda1>
md: running: <sda1>
raid1: raid set md0 active with 1 out of 2 mirrors
md: ... autorun DONE.
kjournald starting.  Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
VFS: Mounted root (ext3 filesystem) readonly.
Freeing unused kernel memory: 188k freed
Adding 8433976k swap on /dev/md5.  Priority:-1 extents:1
EXT3 FS on md1, internal journal
kjournald starting.  Commit interval 5 seconds
EXT3 FS on md2, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on md3, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
e1000: eth0: e1000_watchdog: NIC Link is Up 100 Mbps Full Duplex

------------------------------------------------------------

Re: Memory problem

[Erik Mouw]

On Sun, Apr 24, 2005 at 04:40:50PM +0200, Froggy / Froggy Corp. wrote:
> I just see a memory problem : i have 1Gb install on my computer and only
> 275Mb are available when i take a look with "free".
> I use kernel 2.6.10, the bios tell me that i really have 1Gb (2*512
> DDR), the kernel have 4Gb option enable, so i dont understand.
> 
> If someone have an idea.
> 
> Thx in advance for any help,
> Regards,
> 
> Here is some information :

[snip]

> -------------------------------------------------------------------------
> Linux version 2.6.10 (root@Odyssee) (gcc version 3.3.5 (Debian
> 1:3.3.5-12)) #3 S
> MP Sun Apr 24 16:00:50 CEST 2005
> BIOS-provided physical RAM map:
>  BIOS-e820: 0000000000000000 - 000000000009d800 (usable)
>  BIOS-e820: 000000000009d800 - 00000000000a0000 (reserved)
>  BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
>  BIOS-e820: 0000000000100000 - 0000000011340400 (usable)
>  BIOS-e820: 0000000011340400 - 0000000011343400 (ACPI NVS)
>  BIOS-e820: 0000000011343400 - 0000000011350400 (ACPI data)
>  BIOS-e820: 00000000fec00000 - 0000000100000000 (reserved)

The kernel takes the amount of memory the BIOS tells it to use. If you
count all memory in the "usable" sections of the RAM map, it all adds
up to 275 MB.

  erik@arthur:~> bc -l
  ibase=16
  (((11340400 - 100000) + (9D800)) / 400 / 400
  274.8662109375

Might be a BIOS bug, or the BIOS RAM tester decided only 275 MB is
useable. Try upgrading the BIOS, or reset the CMOS.


Erik

-- 
+-- Erik Mouw -- www.harddisk-recovery.com -- +31 70 370 12 90 --
| Lab address: Delftechpark 26, 2628 XH, Delft, The Netherlands

Re: Memory problem

[Froggy / Froggy Corp.]

Thx, in fact i just solve my problem before reading your email.
Problem came from bios, but i dont understand where, because all "memory
hole" were disable ...

Thx again.

Erik Mouw wrote:

> Might be a BIOS bug, or the BIOS RAM tester decided only 275 MB is
> useable. Try upgrading the BIOS, or reset the CMOS.
> 
> Erik
> 
> --
> +-- Erik Mouw -- www.harddisk-recovery.com -- +31 70 370 12 90 --
> | Lab address: Delftechpark 26, 2628 XH, Delft, The Netherlands