* Severe Linux 2.4 kernel memory leakage
@ 2001-11-25 14:49 Chris Chabot
2001-11-25 15:03 ` Florian Weimer
` (3 more replies)
0 siblings, 4 replies; 19+ messages in thread
From: Chris Chabot @ 2001-11-25 14:49 UTC (permalink / raw)
To: linux-kernel
Hi, I have a firewall / file server box which is displaying (severe)
memory leakage, presumably by the kernel.
The box has ran Redhat 7.1 and 7.2, with plain vanilla linux kernels
2.4.9 upto 2.4.15, in all situations the same problem appeared.
The problem is that when the box boots up, it uses about 60Mb of memory.
However after only 1 1/2 days, the memory usage is already around 430Mb
(!!). (this is ofcource used - buffers - cache, as displayed by 'free').
When i do a ps aux, and add the 'resident' memory usage of the
applications, the memory usage should be around 70-80Mb (a bit higher
then @ boot time since bind uses more memory for caching). Yet 'free'
happely tells me:
total used free shared buffers
cached
Mem: 1029752 1019188 10564 0 130888
456000
-/+ buffers/cache: 432300 597452
Swap: 2104464 996 2103468
When the box keeps on running for about a month, the memory usage gets
so high that it turns into a swap-crazy, low-memory and slow server ;-/
(it does free up cache memory, and swaps stuff out, however the 'leaked'
memory only grows and is never re-claimed).
The box runs dhcpd, bind, fetchmail (cron), pppd (to adsl modem), smb,
nfs, xinetd (imapd mostly) and sshd.
Also it has a (custom) iptables firewall script, and a simple ip route
hack to allow 'outbound interface == inbound interface' (using ipmark
based routing) for my cable modem & adsl modem. Also it has a 310Gb raid
0 array on 4 IDE disks.
Since this box has ran several versions of different kernels, redhat
distro's, and various firewall scripts. I tend to believe this is a more
'structural' problem within the linux kernel.
The box firewalls for both my cable modem and my adsl modem, and has 3
network cards (1 direct to cable, one direct to adsl, one to local
network).
The hardware on the box is : Asus p2b-ds, 2x p3-600, 1Gb (ECC) ram, 3
network cards (1x Intel EtherExpressPro, 2x 3c905 tx), Internal adaptect
29xx u2w scsi, internal intel IDE, 2x Seagate Cheetah (u2w) 18 Gb disks
(/ and /var), 4x 80 Gb Maxtor IDE disks (raid 0 array) and a NVidia TNT2
card. This hardware
The kernel is compiled with all network- and scsi card and raid0 drivers
build in, and nfs + iptables as modules. The machine currently uses ext3
(also build in), however this problem was also present before i
converted the raid0 volume to ext3, so i do not suspect it to cause this
problem. The kernel is also set for HIGHMEM (4gb) to use the last Mb's
of the 1Gb of ram (else 127Mb isnt detected).
If there is any additional information i can provide, please feel free
to ask! Also please CC me in the replies, since i am not subscribed to
the linux-kernel list.
I do not know which component (iptables / route hack / raid0 / network
cards / highmem) cause this problem. I run several of these components
on other servers, without the same problems.. However in this
combination, the kernel seems very leaky ;-/ Any and all sugestions or
help is greatly apreciated.
Additional info on the box:
My Routes script (to allow cable and adsl to use the same outbound
interface as inbound, to prevent invalid routing over the default gw):
#!/bin/bash
echo 1 > /proc/sys/net/ipv4/route/flush
echo "Removing old rules"
ip rule del from 24.132.33.179 table a2000 &>/dev/null
ip rule del from 213.84.192.197 table xs4all &>/dev/null
ip route del table a2000 &>/dev/null
ip route del table xs4all &>/dev/null
echo "Setting routing"
ip rule add from 24.132.33.179 table a2000 prio 20
ip rule add from 213.84.192.197 table xs4all prio 30
ip route add 0/0 table a2000 dev eth0 prio 20
ip route add 0/0 table xs4all dev ppp0 prio 30
free (this is after 1 1/2 day):
total used free shared buffers
cached
Mem: 1029752 1018528 11224 0 131608
454556
-/+ buffers/cache: 432364 597388
Swap: 2104464 996 2103468
ps aux:
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME
COMMAND
root 1 0.0 0.0 1416 476 ? S Nov24 0:04 init
[3]
root 2 0.0 0.0 0 0 ? SW Nov24 0:00
[keventd]
root 3 0.0 0.0 0 0 ? SWN Nov24 0:00
[ksoftirqd_CPU0]
root 4 0.0 0.0 0 0 ? SWN Nov24 0:00
[ksoftirqd_CPU1]
root 5 0.0 0.0 0 0 ? SW Nov24 0:13
[kswapd]
root 6 0.0 0.0 0 0 ? SW Nov24 0:00
[bdflush]
root 7 0.0 0.0 0 0 ? SW Nov24 0:03
[kupdated]
root 8 0.0 0.0 0 0 ? SW Nov24 0:00
[scsi_eh_0]
root 9 0.0 0.0 0 0 ? SW< Nov24 0:00
[mdrecoveryd]
root 10 0.0 0.0 0 0 ? SW Nov24 0:01
[kjournald]
root 145 0.0 0.0 0 0 ? SW Nov24 0:00
[kjournald]
root 146 0.0 0.0 0 0 ? SW Nov24 0:01
[kjournald]
root 147 0.0 0.0 0 0 ? SW Nov24 0:18
[kjournald]
root 719 0.0 0.0 1476 604 ? S Nov24 0:00 syslogd
-m 0 -r
root 724 0.0 0.0 1404 476 ? S Nov24 0:00 klogd -2
-x
bin 744 0.0 0.0 1660 764 ? S Nov24 0:00
portmap
root 801 0.0 0.0 1792 568 ? S Nov24 0:00
rpc.rquotad
root 806 0.0 0.0 1620 716 ? S Nov24 0:00
rpc.mountd
root 811 0.0 0.0 0 0 ? SW Nov24 0:20
[nfsd]
root 812 0.0 0.0 0 0 ? SW Nov24 0:20
[nfsd]
root 813 0.0 0.0 0 0 ? SW Nov24 0:20
[nfsd]
root 814 0.0 0.0 0 0 ? SW Nov24 0:19
[nfsd]
root 815 0.0 0.0 0 0 ? SW Nov24 0:20
[nfsd]
root 816 0.0 0.0 0 0 ? SW Nov24 0:19
[nfsd]
root 817 0.0 0.0 0 0 ? SW Nov24 0:20
[nfsd]
root 818 0.0 0.0 0 0 ? SW Nov24 0:20
[nfsd]
root 819 0.0 0.0 0 0 ? SW Nov24 0:00
[lockd]
root 820 0.0 0.0 0 0 ? SW Nov24 0:00
[rpciod]
root 892 0.0 0.0 1920 896 ? S Nov24 0:00
/usr/sbin/pppd ca
root 911 0.0 0.1 2680 1084 ? S Nov24 0:02
/usr/sbin/sshd
root 931 0.0 0.1 2312 1032 ? S Nov24 0:00 xinetd
-stayalive
root 951 0.0 0.0 1796 648 ? S Nov24 0:00
/usr/sbin/dhcpd
named 1009 0.0 0.4 15328 4364 ? S Nov24 0:00 named -u
named
named 1011 0.0 0.4 15328 4364 ? S Nov24 0:01 named -u
named
named 1012 0.0 0.4 15328 4364 ? S Nov24 0:07 named -u
named
named 1013 0.0 0.4 15328 4364 ? S Nov24 0:06 named -u
named
named 1014 0.0 0.4 15328 4364 ? S Nov24 0:04 named -u
named
named 1015 0.0 0.4 15328 4364 ? S Nov24 0:01 named -u
named
root 1033 0.9 0.0 1456 528 ? S Nov24 29:38
/usr/sbin/pptp pp
root 1043 0.0 0.1 5684 1384 ? S Nov24 0:10 sendmail:
accepti
root 1062 0.0 0.0 1648 676 ? S Nov24 0:00
crond
root 1541 0.0 0.0 1388 380 tty1 S Nov24 0:00
/sbin/mingetty tt
root 1542 0.0 0.0 1388 380 tty2 S Nov24 0:00
/sbin/mingetty tt
root 1545 0.0 0.0 1448 560 ? S Nov24 0:00
/usr/sbin/pptp pp
root 7003 0.0 0.1 3260 1132 ? S 05:00 0:00 smbd
-D
root 7008 0.0 0.1 2448 1128 ? S 05:00 0:00 nmbd
-D
root 7609 0.0 0.1 3732 2016 ? S 14:39 0:00
/usr/sbin/sshd
root 7611 0.0 0.1 2612 1448 pts/1 S 14:39 0:00
-bash
root 13333 0.0 0.0 2656 752 pts/1 R 14:57 0:00 ps aux
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Severe Linux 2.4 kernel memory leakage
2001-11-25 14:49 Severe Linux 2.4 kernel memory leakage Chris Chabot
@ 2001-11-25 15:03 ` Florian Weimer
2001-11-25 15:30 ` Chris Chabot
2001-11-25 15:10 ` James Morris
` (2 subsequent siblings)
3 siblings, 1 reply; 19+ messages in thread
From: Florian Weimer @ 2001-11-25 15:03 UTC (permalink / raw)
To: linux-kernel
Chris Chabot <chabotc@reviewboard.com> writes:
> When the box keeps on running for about a month,
Which kernels have you run for about a month, and which ones showed
this extreme behavior? Obviously not 2.4.15...
The amount of available memory decreasing is quite normal, due to the
growing cache.
--
Florian Weimer Florian.Weimer@RUS.Uni-Stuttgart.DE
University of Stuttgart http://cert.uni-stuttgart.de/
RUS-CERT +49-711-685-5973/fax +49-711-685-5898
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Severe Linux 2.4 kernel memory leakage
2001-11-25 14:49 Severe Linux 2.4 kernel memory leakage Chris Chabot
2001-11-25 15:03 ` Florian Weimer
@ 2001-11-25 15:10 ` James Morris
2001-11-25 15:19 ` Chris Chabot
2001-11-25 15:27 ` Peter T. Breuer
2001-11-26 6:49 ` Mike Galbraith
3 siblings, 1 reply; 19+ messages in thread
From: James Morris @ 2001-11-25 15:10 UTC (permalink / raw)
To: Chris Chabot; +Cc: linux-kernel
On 25 Nov 2001, Chris Chabot wrote:
> Also it has a (custom) iptables firewall script
Are you using ipchains emulation?
- James
--
James Morris
<jmorris@intercode.com.au>
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Severe Linux 2.4 kernel memory leakage
2001-11-25 15:10 ` James Morris
@ 2001-11-25 15:19 ` Chris Chabot
0 siblings, 0 replies; 19+ messages in thread
From: Chris Chabot @ 2001-11-25 15:19 UTC (permalink / raw)
To: James Morris; +Cc: linux-kernel
[-- Attachment #1: Type: text/plain, Size: 1225 bytes --]
nope, just plain netfilter/iptables. Specificly (lsmod output) :
ipt_TOS 880 8 (autoclean)
ipt_MASQUERADE 1312 4 (autoclean)
ipt_state 576 7 (autoclean)
ipt_REJECT 2816 7 (autoclean)
ipt_LOG 3408 24 (autoclean)
ipt_limit 1008 26 (autoclean)
ip_nat_ftp 3184 0 (unused)
ip_conntrack_ftp 3536 0 [ip_nat_ftp]
iptable_mangle 1712 0 (autoclean) (unused)
iptable_nat 14448 1 (autoclean) [ipt_MASQUERADE
ip_nat_ftp]
ip_conntrack 15056 5 (autoclean) [ipt_MASQUERADE ipt_state
ip_nat_ftp ip_conntrack_ftp iptable_nat]
iptable_filter 1680 0 (autoclean) (unused)
ip_tables 11392 11 [ipt_TOS ipt_MASQUERADE ipt_state
ipt_REJECT ipt_LOG ipt_limit iptable_mangle iptable_nat iptable_filter]
I've also attached the output of 'iptables -L -n' so u can get an idea
of what its running.
-- Chris
On Sun, 2001-11-25 at 16:10, James Morris wrote:
> On 25 Nov 2001, Chris Chabot wrote:
>
> > Also it has a (custom) iptables firewall script
>
> Are you using ipchains emulation?
>
>
> - James
> --
> James Morris
> <jmorris@intercode.com.au>
>
[-- Attachment #2: iptables-output.txt --]
[-- Type: text/plain, Size: 25423 bytes --]
Chain INPUT (policy DROP)
target prot opt source destination
CHECKFLAGS tcp -- 0.0.0.0/0 0.0.0.0/0
INETIN all -- 0.0.0.0/0 0.0.0.0/0
CHECKFLAGS tcp -- 0.0.0.0/0 0.0.0.0/0
INETIN all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 192.168.0.0/24 0.0.0.0/0
ACCEPT all -- 10.0.0.0/24 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
UDPACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:67
UDPACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:67
Chain FORWARD (policy DROP)
target prot opt source destination
ACCEPT all -- 192.168.0.0/24 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 192.168.0.0/24
ACCEPT all -- 10.0.0.0/24 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 10.0.0.0/24
TCPACCEPT tcp -- 0.0.0.0/0 192.168.0.10 tcp dpt:80
TCPACCEPT tcp -- 0.0.0.0/0 192.168.0.10 tcp dpt:80
TCPACCEPT tcp -- 0.0.0.0/0 192.168.0.10 tcp dpt:443
TCPACCEPT tcp -- 0.0.0.0/0 192.168.0.10 tcp dpt:443
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
INETOUT all -- 0.0.0.0/0 0.0.0.0/0
INETOUT all -- 0.0.0.0/0 0.0.0.0/0
Chain CHECKFLAGS (2 references)
target prot opt source destination
LOG tcp -- 0.0.0.0/0 0.0.0.0/0 tcp flags:0x3F/0x29 limit: avg 1/sec burst 5 LOG flags 0 level 6 prefix `Posible NMAP-XMAS:'
DROP tcp -- 0.0.0.0/0 0.0.0.0/0 tcp flags:0x3F/0x29
LOG tcp -- 0.0.0.0/0 0.0.0.0/0 tcp flags:0x06/0x06 limit: avg 1/sec burst 5 LOG flags 0 level 6 prefix `Posible SYN/RST:'
DROP tcp -- 0.0.0.0/0 0.0.0.0/0 tcp flags:0x06/0x06
LOG tcp -- 0.0.0.0/0 0.0.0.0/0 tcp flags:0x03/0x03 limit: avg 1/sec burst 5 LOG flags 0 level 6 prefix `Posible SYN/FIN:'
DROP tcp -- 0.0.0.0/0 0.0.0.0/0 tcp flags:0x03/0x03
LOG tcp -- 0.0.0.0/0 0.0.0.0/0 tcp option=64 limit: avg 1/sec burst 5 LOG flags 0 level 6 prefix `Bogus TCP FLAG 64'
DROP tcp -- 0.0.0.0/0 0.0.0.0/0 tcp option=64
LOG tcp -- 0.0.0.0/0 0.0.0.0/0 tcp option=128 limit: avg 1/sec burst 5 LOG flags 0 level 6 prefix `Bogus TCP FLAG 128'
DROP tcp -- 0.0.0.0/0 0.0.0.0/0 tcp option=128
LOG tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:31337 limit: avg 1/sec burst 5 LOG flags 0 level 6 prefix `Back Orifice:'
DROP tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:31337
LOG tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:20034 limit: avg 1/sec burst 5 LOG flags 0 level 6 prefix `Back Orifice:'
DROP tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:20034
LOG tcp -- 0.0.0.0/0 0.0.0.0/0 tcp spt:27665 limit: avg 1/sec burst 5 LOG flags 0 level 6 prefix `Trinoo:'
DROP tcp -- 0.0.0.0/0 0.0.0.0/0 tcp spt:27665
LOG udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:27665 limit: avg 1/sec burst 5 LOG flags 0 level 6 prefix `Trinoo:'
DROP udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:27665
INETIN all -- 0.0.0.0/0 0.0.0.0/0
Chain INETIN (3 references)
target prot opt source destination
LDROP all -- 10.0.0.0/8 0.0.0.0/0
LDROP all -- 172.16.0.0/12 0.0.0.0/0
LDROP all -- 192.168.0.0/16 0.0.0.0/0
LDROP all -- 224.0.0.0/4 0.0.0.0/0
LDROP all -- 240.0.0.0/5 0.0.0.0/0
LDROP all -- 10.0.0.0/8 0.0.0.0/0
LDROP all -- 172.16.0.0/12 0.0.0.0/0
LDROP all -- 192.168.0.0/16 0.0.0.0/0
LDROP all -- 224.0.0.0/4 0.0.0.0/0
LDROP all -- 240.0.0.0/5 0.0.0.0/0
LDROP all -- 0.0.0.0/8 0.0.0.0/0
LDROP all -- 1.0.0.0/8 0.0.0.0/0
LDROP all -- 2.0.0.0/8 0.0.0.0/0
LDROP all -- 5.0.0.0/8 0.0.0.0/0
LDROP all -- 7.0.0.0/8 0.0.0.0/0
LDROP all -- 23.0.0.0/8 0.0.0.0/0
LDROP all -- 27.0.0.0/8 0.0.0.0/0
LDROP all -- 31.0.0.0/8 0.0.0.0/0
LDROP all -- 36.0.0.0/8 0.0.0.0/0
LDROP all -- 37.0.0.0/8 0.0.0.0/0
LDROP all -- 39.0.0.0/8 0.0.0.0/0
LDROP all -- 41.0.0.0/8 0.0.0.0/0
LDROP all -- 42.0.0.0/8 0.0.0.0/0
LDROP all -- 58.0.0.0/8 0.0.0.0/0
LDROP all -- 59.0.0.0/8 0.0.0.0/0
LDROP all -- 60.0.0.0/8 0.0.0.0/0
LDROP all -- 67.0.0.0/8 0.0.0.0/0
LDROP all -- 68.0.0.0/8 0.0.0.0/0
LDROP all -- 69.0.0.0/8 0.0.0.0/0
LDROP all -- 70.0.0.0/8 0.0.0.0/0
LDROP all -- 71.0.0.0/8 0.0.0.0/0
LDROP all -- 72.0.0.0/8 0.0.0.0/0
LDROP all -- 73.0.0.0/8 0.0.0.0/0
LDROP all -- 74.0.0.0/8 0.0.0.0/0
LDROP all -- 75.0.0.0/8 0.0.0.0/0
LDROP all -- 76.0.0.0/8 0.0.0.0/0
LDROP all -- 77.0.0.0/8 0.0.0.0/0
LDROP all -- 78.0.0.0/8 0.0.0.0/0
LDROP all -- 79.0.0.0/8 0.0.0.0/0
LDROP all -- 82.0.0.0/8 0.0.0.0/0
LDROP all -- 83.0.0.0/8 0.0.0.0/0
LDROP all -- 84.0.0.0/8 0.0.0.0/0
LDROP all -- 85.0.0.0/8 0.0.0.0/0
LDROP all -- 86.0.0.0/8 0.0.0.0/0
LDROP all -- 87.0.0.0/8 0.0.0.0/0
LDROP all -- 88.0.0.0/8 0.0.0.0/0
LDROP all -- 89.0.0.0/8 0.0.0.0/0
LDROP all -- 90.0.0.0/8 0.0.0.0/0
LDROP all -- 91.0.0.0/8 0.0.0.0/0
LDROP all -- 92.0.0.0/8 0.0.0.0/0
LDROP all -- 93.0.0.0/8 0.0.0.0/0
LDROP all -- 94.0.0.0/8 0.0.0.0/0
LDROP all -- 95.0.0.0/8 0.0.0.0/0
LDROP all -- 96.0.0.0/8 0.0.0.0/0
LDROP all -- 97.0.0.0/8 0.0.0.0/0
LDROP all -- 98.0.0.0/8 0.0.0.0/0
LDROP all -- 99.0.0.0/8 0.0.0.0/0
LDROP all -- 100.0.0.0/8 0.0.0.0/0
LDROP all -- 101.0.0.0/8 0.0.0.0/0
LDROP all -- 102.0.0.0/8 0.0.0.0/0
LDROP all -- 103.0.0.0/8 0.0.0.0/0
LDROP all -- 104.0.0.0/8 0.0.0.0/0
LDROP all -- 105.0.0.0/8 0.0.0.0/0
LDROP all -- 106.0.0.0/8 0.0.0.0/0
LDROP all -- 107.0.0.0/8 0.0.0.0/0
LDROP all -- 108.0.0.0/8 0.0.0.0/0
LDROP all -- 109.0.0.0/8 0.0.0.0/0
LDROP all -- 110.0.0.0/8 0.0.0.0/0
LDROP all -- 111.0.0.0/8 0.0.0.0/0
LDROP all -- 112.0.0.0/8 0.0.0.0/0
LDROP all -- 113.0.0.0/8 0.0.0.0/0
LDROP all -- 114.0.0.0/8 0.0.0.0/0
LDROP all -- 115.0.0.0/8 0.0.0.0/0
LDROP all -- 116.0.0.0/8 0.0.0.0/0
LDROP all -- 117.0.0.0/8 0.0.0.0/0
LDROP all -- 118.0.0.0/8 0.0.0.0/0
LDROP all -- 119.0.0.0/8 0.0.0.0/0
LDROP all -- 120.0.0.0/8 0.0.0.0/0
LDROP all -- 121.0.0.0/8 0.0.0.0/0
LDROP all -- 122.0.0.0/8 0.0.0.0/0
LDROP all -- 123.0.0.0/8 0.0.0.0/0
LDROP all -- 124.0.0.0/8 0.0.0.0/0
LDROP all -- 125.0.0.0/8 0.0.0.0/0
LDROP all -- 126.0.0.0/8 0.0.0.0/0
LDROP all -- 127.0.0.0/8 0.0.0.0/0
LDROP all -- 197.0.0.0/8 0.0.0.0/0
LDROP all -- 201.0.0.0/8 0.0.0.0/0
LDROP all -- 219.0.0.0/8 0.0.0.0/0
LDROP all -- 220.0.0.0/8 0.0.0.0/0
LDROP all -- 221.0.0.0/8 0.0.0.0/0
LDROP all -- 222.0.0.0/8 0.0.0.0/0
LDROP all -- 223.0.0.0/8 0.0.0.0/0
LDROP all -- 240.0.0.0/8 0.0.0.0/0
LDROP all -- 241.0.0.0/8 0.0.0.0/0
LDROP all -- 242.0.0.0/8 0.0.0.0/0
LDROP all -- 243.0.0.0/8 0.0.0.0/0
LDROP all -- 244.0.0.0/8 0.0.0.0/0
LDROP all -- 245.0.0.0/8 0.0.0.0/0
LDROP all -- 246.0.0.0/8 0.0.0.0/0
LDROP all -- 247.0.0.0/8 0.0.0.0/0
LDROP all -- 248.0.0.0/8 0.0.0.0/0
LDROP all -- 249.0.0.0/8 0.0.0.0/0
LDROP all -- 250.0.0.0/8 0.0.0.0/0
LDROP all -- 251.0.0.0/8 0.0.0.0/0
LDROP all -- 252.0.0.0/8 0.0.0.0/0
LDROP all -- 253.0.0.0/8 0.0.0.0/0
LDROP all -- 254.0.0.0/8 0.0.0.0/0
TREJECT all -- 0.0.0.0/0 0.0.0.0/0 state INVALID
UDPACCEPT udp -- 194.109.137.130 0.0.0.0/0 udp dpt:53
TCPACCEPT tcp -- 194.109.137.130 0.0.0.0/0 tcp dpt:53
UDPACCEPT udp -- 194.109.137.130 0.0.0.0/0 udp dpt:53
TCPACCEPT tcp -- 194.109.137.130 0.0.0.0/0 tcp dpt:53
UDPACCEPT udp -- 213.84.192.197 0.0.0.0/0 udp dpt:53
TCPACCEPT tcp -- 213.84.192.197 0.0.0.0/0 tcp dpt:53
UDPACCEPT udp -- 213.84.192.197 0.0.0.0/0 udp dpt:53
TCPACCEPT tcp -- 213.84.192.197 0.0.0.0/0 tcp dpt:53
UDPACCEPT udp -- 63.117.39.22 0.0.0.0/0 udp dpt:53
TCPACCEPT tcp -- 63.117.39.22 0.0.0.0/0 tcp dpt:53
UDPACCEPT udp -- 63.117.39.22 0.0.0.0/0 udp dpt:53
TCPACCEPT tcp -- 63.117.39.22 0.0.0.0/0 tcp dpt:53
ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0 icmp type 8 limit: avg 1/sec burst 3
ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0 icmp !type 8
TCPACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:21
TCPACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:22
TCPACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:25
TCPACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:80
TCPACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:67
TCPACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:68
TCPACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:69
UDPACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:53
UDPACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:67
UDPACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:68
UDPACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:69
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 state ESTABLISHED
TCPACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpts:1024:65535 state RELATED
UDPACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpts:1024:65535 state RELATED
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 state ESTABLISHED
TCPACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpts:1024:65535 state RELATED
UDPACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpts:1024:65535 state RELATED
TREJECT all -- 0.0.0.0/0 0.0.0.0/0
Chain INETOUT (2 references)
target prot opt source destination
LDROP all -- 0.0.0.0/0 10.0.0.0/8
LDROP all -- 0.0.0.0/0 172.16.0.0/12
LDROP all -- 0.0.0.0/0 192.168.0.0/16
LDROP all -- 0.0.0.0/0 224.0.0.0/4
LDROP all -- 0.0.0.0/0 240.0.0.0/5
LDROP all -- 0.0.0.0/0 10.0.0.0/8
LDROP all -- 0.0.0.0/0 172.16.0.0/12
LDROP all -- 0.0.0.0/0 192.168.0.0/16
LDROP all -- 0.0.0.0/0 224.0.0.0/4
LDROP all -- 0.0.0.0/0 240.0.0.0/5
LDROP all -- 0.0.0.0/0 0.0.0.0/8
LDROP all -- 0.0.0.0/0 1.0.0.0/8
LDROP all -- 0.0.0.0/0 2.0.0.0/8
LDROP all -- 0.0.0.0/0 5.0.0.0/8
LDROP all -- 0.0.0.0/0 7.0.0.0/8
LDROP all -- 0.0.0.0/0 23.0.0.0/8
LDROP all -- 0.0.0.0/0 27.0.0.0/8
LDROP all -- 0.0.0.0/0 31.0.0.0/8
LDROP all -- 0.0.0.0/0 36.0.0.0/8
LDROP all -- 0.0.0.0/0 37.0.0.0/8
LDROP all -- 0.0.0.0/0 39.0.0.0/8
LDROP all -- 0.0.0.0/0 41.0.0.0/8
LDROP all -- 0.0.0.0/0 42.0.0.0/8
LDROP all -- 0.0.0.0/0 58.0.0.0/8
LDROP all -- 0.0.0.0/0 59.0.0.0/8
LDROP all -- 0.0.0.0/0 60.0.0.0/8
LDROP all -- 0.0.0.0/0 67.0.0.0/8
LDROP all -- 0.0.0.0/0 68.0.0.0/8
LDROP all -- 0.0.0.0/0 69.0.0.0/8
LDROP all -- 0.0.0.0/0 70.0.0.0/8
LDROP all -- 0.0.0.0/0 71.0.0.0/8
LDROP all -- 0.0.0.0/0 72.0.0.0/8
LDROP all -- 0.0.0.0/0 73.0.0.0/8
LDROP all -- 0.0.0.0/0 74.0.0.0/8
LDROP all -- 0.0.0.0/0 75.0.0.0/8
LDROP all -- 0.0.0.0/0 76.0.0.0/8
LDROP all -- 0.0.0.0/0 77.0.0.0/8
LDROP all -- 0.0.0.0/0 78.0.0.0/8
LDROP all -- 0.0.0.0/0 79.0.0.0/8
LDROP all -- 0.0.0.0/0 82.0.0.0/8
LDROP all -- 0.0.0.0/0 83.0.0.0/8
LDROP all -- 0.0.0.0/0 84.0.0.0/8
LDROP all -- 0.0.0.0/0 85.0.0.0/8
LDROP all -- 0.0.0.0/0 86.0.0.0/8
LDROP all -- 0.0.0.0/0 87.0.0.0/8
LDROP all -- 0.0.0.0/0 88.0.0.0/8
LDROP all -- 0.0.0.0/0 89.0.0.0/8
LDROP all -- 0.0.0.0/0 90.0.0.0/8
LDROP all -- 0.0.0.0/0 91.0.0.0/8
LDROP all -- 0.0.0.0/0 92.0.0.0/8
LDROP all -- 0.0.0.0/0 93.0.0.0/8
LDROP all -- 0.0.0.0/0 94.0.0.0/8
LDROP all -- 0.0.0.0/0 95.0.0.0/8
LDROP all -- 0.0.0.0/0 96.0.0.0/8
LDROP all -- 0.0.0.0/0 97.0.0.0/8
LDROP all -- 0.0.0.0/0 98.0.0.0/8
LDROP all -- 0.0.0.0/0 99.0.0.0/8
LDROP all -- 0.0.0.0/0 100.0.0.0/8
LDROP all -- 0.0.0.0/0 101.0.0.0/8
LDROP all -- 0.0.0.0/0 102.0.0.0/8
LDROP all -- 0.0.0.0/0 103.0.0.0/8
LDROP all -- 0.0.0.0/0 104.0.0.0/8
LDROP all -- 0.0.0.0/0 105.0.0.0/8
LDROP all -- 0.0.0.0/0 106.0.0.0/8
LDROP all -- 0.0.0.0/0 107.0.0.0/8
LDROP all -- 0.0.0.0/0 108.0.0.0/8
LDROP all -- 0.0.0.0/0 109.0.0.0/8
LDROP all -- 0.0.0.0/0 110.0.0.0/8
LDROP all -- 0.0.0.0/0 111.0.0.0/8
LDROP all -- 0.0.0.0/0 112.0.0.0/8
LDROP all -- 0.0.0.0/0 113.0.0.0/8
LDROP all -- 0.0.0.0/0 114.0.0.0/8
LDROP all -- 0.0.0.0/0 115.0.0.0/8
LDROP all -- 0.0.0.0/0 116.0.0.0/8
LDROP all -- 0.0.0.0/0 117.0.0.0/8
LDROP all -- 0.0.0.0/0 118.0.0.0/8
LDROP all -- 0.0.0.0/0 119.0.0.0/8
LDROP all -- 0.0.0.0/0 120.0.0.0/8
LDROP all -- 0.0.0.0/0 121.0.0.0/8
LDROP all -- 0.0.0.0/0 122.0.0.0/8
LDROP all -- 0.0.0.0/0 123.0.0.0/8
LDROP all -- 0.0.0.0/0 124.0.0.0/8
LDROP all -- 0.0.0.0/0 125.0.0.0/8
LDROP all -- 0.0.0.0/0 126.0.0.0/8
LDROP all -- 0.0.0.0/0 127.0.0.0/8
LDROP all -- 0.0.0.0/0 197.0.0.0/8
LDROP all -- 0.0.0.0/0 201.0.0.0/8
LDROP all -- 0.0.0.0/0 219.0.0.0/8
LDROP all -- 0.0.0.0/0 220.0.0.0/8
LDROP all -- 0.0.0.0/0 221.0.0.0/8
LDROP all -- 0.0.0.0/0 222.0.0.0/8
LDROP all -- 0.0.0.0/0 223.0.0.0/8
LDROP all -- 0.0.0.0/0 240.0.0.0/8
LDROP all -- 0.0.0.0/0 241.0.0.0/8
LDROP all -- 0.0.0.0/0 242.0.0.0/8
LDROP all -- 0.0.0.0/0 243.0.0.0/8
LDROP all -- 0.0.0.0/0 244.0.0.0/8
LDROP all -- 0.0.0.0/0 245.0.0.0/8
LDROP all -- 0.0.0.0/0 246.0.0.0/8
LDROP all -- 0.0.0.0/0 247.0.0.0/8
LDROP all -- 0.0.0.0/0 248.0.0.0/8
LDROP all -- 0.0.0.0/0 249.0.0.0/8
LDROP all -- 0.0.0.0/0 250.0.0.0/8
LDROP all -- 0.0.0.0/0 251.0.0.0/8
LDROP all -- 0.0.0.0/0 252.0.0.0/8
LDROP all -- 0.0.0.0/0 253.0.0.0/8
LDROP all -- 0.0.0.0/0 254.0.0.0/8
UDPACCEPT udp -- 0.0.0.0/0 194.109.137.130 udp spt:53
TCPACCEPT tcp -- 0.0.0.0/0 194.109.137.130 tcp spt:53 flags:!0x16/0x02
UDPACCEPT udp -- 0.0.0.0/0 194.109.137.130 udp spt:53
TCPACCEPT tcp -- 0.0.0.0/0 194.109.137.130 tcp spt:53 flags:!0x16/0x02
UDPACCEPT udp -- 0.0.0.0/0 213.84.192.197 udp spt:53
TCPACCEPT tcp -- 0.0.0.0/0 213.84.192.197 tcp spt:53 flags:!0x16/0x02
UDPACCEPT udp -- 0.0.0.0/0 213.84.192.197 udp spt:53
TCPACCEPT tcp -- 0.0.0.0/0 213.84.192.197 tcp spt:53 flags:!0x16/0x02
UDPACCEPT udp -- 0.0.0.0/0 63.117.39.22 udp spt:53
TCPACCEPT tcp -- 0.0.0.0/0 63.117.39.22 tcp spt:53 flags:!0x16/0x02
UDPACCEPT udp -- 0.0.0.0/0 63.117.39.22 udp spt:53
TCPACCEPT tcp -- 0.0.0.0/0 63.117.39.22 tcp spt:53 flags:!0x16/0x02
TCPACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp spt:21
TCPACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp spt:22
TCPACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp spt:25
TCPACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp spt:80
TCPACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp spt:67
TCPACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp spt:68
TCPACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp spt:69
UDPACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp spt:53
UDPACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp spt:67
UDPACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp spt:68
UDPACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp spt:69
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
Chain LDROP (214 references)
target prot opt source destination
LOG tcp -- 0.0.0.0/0 0.0.0.0/0 limit: avg 1/sec burst 5 LOG flags 0 level 6 prefix `TCP Dropped '
LOG udp -- 0.0.0.0/0 0.0.0.0/0 limit: avg 1/sec burst 5 LOG flags 0 level 6 prefix `UDP Dropped '
LOG icmp -- 0.0.0.0/0 0.0.0.0/0 limit: avg 1/sec burst 5 LOG flags 0 level 6 prefix `ICMP Dropped '
LOG all -f 0.0.0.0/0 0.0.0.0/0 limit: avg 1/sec burst 5 LOG flags 0 level 4 prefix `FRAGMENT Dropped '
DROP all -- 0.0.0.0/0 0.0.0.0/0
Chain LREJECT (0 references)
target prot opt source destination
LOG tcp -- 0.0.0.0/0 0.0.0.0/0 limit: avg 1/sec burst 5 LOG flags 0 level 6 prefix `TCP Rejected '
LOG udp -- 0.0.0.0/0 0.0.0.0/0 limit: avg 1/sec burst 5 LOG flags 0 level 6 prefix `UDP Rejected '
LOG icmp -- 0.0.0.0/0 0.0.0.0/0 limit: avg 1/sec burst 5 LOG flags 0 level 6 prefix `ICMP Dropped '
LOG all -f 0.0.0.0/0 0.0.0.0/0 limit: avg 1/sec burst 5 LOG flags 0 level 4 prefix `FRAGMENT Rejected '
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
Chain LTREJECT (0 references)
target prot opt source destination
LOG tcp -- 0.0.0.0/0 0.0.0.0/0 limit: avg 1/sec burst 5 LOG flags 0 level 6 prefix `TCP Rejected '
LOG udp -- 0.0.0.0/0 0.0.0.0/0 limit: avg 1/sec burst 5 LOG flags 0 level 6 prefix `UDP Rejected '
LOG icmp -- 0.0.0.0/0 0.0.0.0/0 limit: avg 1/sec burst 5 LOG flags 0 level 6 prefix `ICMP Dropped '
LOG all -f 0.0.0.0/0 0.0.0.0/0 limit: avg 1/sec burst 5 LOG flags 0 level 4 prefix `FRAGMENT Rejected '
REJECT tcp -- 0.0.0.0/0 0.0.0.0/0 reject-with tcp-reset
REJECT udp -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
DROP icmp -- 0.0.0.0/0 0.0.0.0/0
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
Chain TCPACCEPT (32 references)
target prot opt source destination
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp flags:0x16/0x02 limit: avg 40/sec burst 5
LOG tcp -- 0.0.0.0/0 0.0.0.0/0 tcp flags:0x16/0x02 limit: avg 1/sec burst 5 LOG flags 0 level 4 prefix `Possible SynFlood '
TREJECT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp flags:0x16/0x02
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp flags:!0x16/0x02
LOG all -- 0.0.0.0/0 0.0.0.0/0 limit: avg 1/sec burst 5 LOG flags 0 level 4 prefix `Mismatch in TCPACCEPT '
TREJECT all -- 0.0.0.0/0 0.0.0.0/0
Chain TREJECT (5 references)
target prot opt source destination
REJECT tcp -- 0.0.0.0/0 0.0.0.0/0 reject-with tcp-reset
REJECT udp -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
DROP icmp -- 0.0.0.0/0 0.0.0.0/0
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
Chain UDPACCEPT (24 references)
target prot opt source destination
ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0
LOG all -- 0.0.0.0/0 0.0.0.0/0 limit: avg 1/sec burst 5 LOG flags 0 level 4 prefix `Mismatch on UDPACCEPT '
TREJECT all -- 0.0.0.0/0 0.0.0.0/0
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Severe Linux 2.4 kernel memory leakage
2001-11-25 14:49 Severe Linux 2.4 kernel memory leakage Chris Chabot
2001-11-25 15:03 ` Florian Weimer
2001-11-25 15:10 ` James Morris
@ 2001-11-25 15:27 ` Peter T. Breuer
2001-11-25 15:39 ` Chris Chabot
2001-11-25 17:03 ` Andi Kleen
2001-11-26 6:49 ` Mike Galbraith
3 siblings, 2 replies; 19+ messages in thread
From: Peter T. Breuer @ 2001-11-25 15:27 UTC (permalink / raw)
To: Chris Chabot; +Cc: linux-kernel
"A month of sundays ago Chris Chabot wrote:"
> The box has ran Redhat 7.1 and 7.2, with plain vanilla linux kernels
> 2.4.9 upto 2.4.15, in all situations the same problem appeared.
>
> The problem is that when the box boots up, it uses about 60Mb of memory.
> However after only 1 1/2 days, the memory usage is already around 430Mb
> (!!). (this is ofcource used - buffers - cache, as displayed by 'free').
I also have this problem. Unknown circumstances provoke it. Kernel
2.4.9 to 2.4.13. When it occurs I lose about 30MB a day.
Dual 500MHz i686, 4 scsi disks (adaptec) under raid5 and raid0
with 2 intelpro's and 1 IDE disk (and xfs and lvm).
Right now I'm on 2.4.9 and it's NOT happening. Doing nothing different
to any other day.
> When the box keeps on running for about a month, the memory usage gets
> so high that it turns into a swap-crazy, low-memory and slow server ;-/
> (it does free up cache memory, and swaps stuff out, however the 'leaked'
> memory only grows and is never re-claimed).
Same.
> The box runs dhcpd, bind, fetchmail (cron), pppd (to adsl modem), smb,
> nfs, xinetd (imapd mostly) and sshd.
Only thing in common with me is nfs. Running X 4.1. glibc 2.1.
> based routing) for my cable modem & adsl modem. Also it has a 310Gb raid
> 0 array on 4 IDE disks.
Could be.
> The hardware on the box is : Asus p2b-ds, 2x p3-600, 1Gb (ECC) ram, 3
My mobo is whatever came from dell, and you also are running 2xP3. My
ram is also ECC but there's only 128MB of it.
> network cards (1x Intel EtherExpressPro, 2x 3c905 tx), Internal adaptect
I have 2 network cards, both EEPRO.
> 29xx u2w scsi, internal intel IDE, 2x Seagate Cheetah (u2w) 18 Gb disks
Yep, I have internal adaptec too. Aic7xxx running ultra 160 at 20MHz
on terminated cable.
Adaptec AIC7xxx driver version: 6.2.1
aic7892: Ultra160 Wide Channel A, SCSI Id=7, 32/255 SCBs
4 WD disks:
Host: scsi0 Channel: 00 Id: 00 Lun: 00
Vendor: WDIGTL Model: WDE9100 ULTRA2 Rev: 1.21
Type: Direct-Access ANSI SCSI revision: 02
Host: scsi0 Channel: 00 Id: 01 Lun: 00
Vendor: WDIGTL Model: WDE9100 ULTRA2 Rev: 1.21
Type: Direct-Access ANSI SCSI revision: 02
Host: scsi0 Channel: 00 Id: 02 Lun: 00
Vendor: WDIGTL Model: WDE9100 ULTRA2 Rev: 1.21
Type: Direct-Access ANSI SCSI revision: 02
Host: scsi0 Channel: 00 Id: 03 Lun: 00
Vendor: WDIGTL Model: WDE9100 ULTRA2 Rev: 1.21
Type: Direct-Access ANSI SCSI revision: 02
> (/ and /var), 4x 80 Gb Maxtor IDE disks (raid 0 array) and a NVidia TNT2
> card. This hardware
Umm .. I think I run ati rage, external card, though there is one on
the mobo.
(--) PCI:*(0:16:0) ATI Mach64 GU rev 154, Mem @ 0xf5000000/24,
0xfe201000/12, I/O @ 0xd400/8
(--) PCI: (1:0:0) ATI Mach64 GW rev 122, Mem @ 0xfc000000/24,
0xfbfff000/12, I/O @ 0xec00/8
> The kernel is compiled with all network- and scsi card and raid0 drivers
> build in, and nfs + iptables as modules. The machine currently uses ext3
I have it all compiled OUT. Including iptables, which I don't use.
> (also build in), however this problem was also present before i
> converted the raid0 volume to ext3, so i do not suspect it to cause this
I am using xfs on top of lvm on top of raid5.
> problem. The kernel is also set for HIGHMEM (4gb) to use the last Mb's
> of the 1Gb of ram (else 127Mb isnt detected).
Mine isn't. Normal setup.
> I do not know which component (iptables / route hack / raid0 / network
> cards / highmem) cause this problem. I run several of these components
Looks from this as though it might be raid5 or 0 + adaptec scsi + SMP.
Peter
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Severe Linux 2.4 kernel memory leakage
2001-11-25 15:03 ` Florian Weimer
@ 2001-11-25 15:30 ` Chris Chabot
2001-11-25 15:41 ` Phil Sorber
2001-11-25 15:55 ` Mr. Shannon Aldinger
0 siblings, 2 replies; 19+ messages in thread
From: Chris Chabot @ 2001-11-25 15:30 UTC (permalink / raw)
To: Florian Weimer; +Cc: linux-kernel
The kernel i ran for about a month was kernel 2.4.11.
Ofcource i am aware that the memory usage grows as more memory is used
for buffers/cache. (specialy since its also a large file server).
However if you check my 'free' output, and the ps aux output you will
notice that the 430Mb used is with the cache and buffer usage already
subtracted from the 'total usage' (else usage is just below 1 gig).
Of 430Mb, (counting ps aux res values), just below 80 Mb is used by the
applications. the rest is just 'missing'.
So the current memory division is about (sources: application = added ps
aux output, buffer/cache/free = 'free' command, sysv shm from 'ipcs')
Applications: 80Mb
Buffers: 127Mb
Cache: 460Mb
Sysv shm: 0
Free: 9.5Mb
memory total 1Gb
Unaccounted +/- 360Mb
ps, yes i did check /dev/shm, and 'ipcs' and no memory is used as sysv
shared memory
-- Chris
On Sun, 2001-11-25 at 16:03, Florian Weimer wrote:
> Chris Chabot <chabotc@reviewboard.com> writes:
>
> > When the box keeps on running for about a month,
>
> Which kernels have you run for about a month, and which ones showed
> this extreme behavior? Obviously not 2.4.15...
>
> The amount of available memory decreasing is quite normal, due to the
> growing cache.
>
> --
> Florian Weimer Florian.Weimer@RUS.Uni-Stuttgart.DE
> University of Stuttgart http://cert.uni-stuttgart.de/
> RUS-CERT +49-711-685-5973/fax +49-711-685-5898
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Severe Linux 2.4 kernel memory leakage
2001-11-25 15:27 ` Peter T. Breuer
@ 2001-11-25 15:39 ` Chris Chabot
2001-11-25 17:03 ` Andi Kleen
1 sibling, 0 replies; 19+ messages in thread
From: Chris Chabot @ 2001-11-25 15:39 UTC (permalink / raw)
To: ptb; +Cc: linux-kernel
Hi, It's almost nice to hear i am not the only person with this problem
;-)
After reading your email, the common factors seem to be:
- Dual P3 Setup (diff mobo, same intel chipset?)
- Software Raid
- Mixed IDE / SCSI
- Internal Adaptec AHA-29xx
(i have the 80mb, you the 160mb version)
- Multiple network cards
- Intel EtherExpress Pro 10/100
- ECC checking ram
Since you don't use weird routes, nor iptables, i think it's posible to
assume these do not cause the problem. Also since you do not have the
problem under 2.4.9 (ditto for me if i remeber correctly), it is safe to
assume the bug was introduced in kernel 2.4.10 or up.
ps, the reason why i imidiatly switched to 2.4.11 (and up) and not have
a lot of experiance with 2.4.9 is because my dell servers with adaptect
hardware raid are a LOT (> 100%) faster under those newer kernels.
Somehow a block layer change in that kernel speeded up the dells a lot..
and since i love consitent kernel versions accross all my machines, i
upgraded my own boxes as well.
-- Chris
On Sun, 2001-11-25 at 16:27, Peter T. Breuer wrote:
> "A month of sundays ago Chris Chabot wrote:"
> > The box has ran Redhat 7.1 and 7.2, with plain vanilla linux kernels
> > 2.4.9 upto 2.4.15, in all situations the same problem appeared.
> >
> > The problem is that when the box boots up, it uses about 60Mb of memory.
> > However after only 1 1/2 days, the memory usage is already around 430Mb
> > (!!). (this is ofcource used - buffers - cache, as displayed by 'free').
>
> I also have this problem. Unknown circumstances provoke it. Kernel
> 2.4.9 to 2.4.13. When it occurs I lose about 30MB a day.
>
> Dual 500MHz i686, 4 scsi disks (adaptec) under raid5 and raid0
> with 2 intelpro's and 1 IDE disk (and xfs and lvm).
>
> Right now I'm on 2.4.9 and it's NOT happening. Doing nothing different
> to any other day.
>
> > When the box keeps on running for about a month, the memory usage gets
> > so high that it turns into a swap-crazy, low-memory and slow server ;-/
> > (it does free up cache memory, and swaps stuff out, however the 'leaked'
> > memory only grows and is never re-claimed).
>
> Same.
>
> > The box runs dhcpd, bind, fetchmail (cron), pppd (to adsl modem), smb,
> > nfs, xinetd (imapd mostly) and sshd.
>
> Only thing in common with me is nfs. Running X 4.1. glibc 2.1.
>
> > based routing) for my cable modem & adsl modem. Also it has a 310Gb raid
> > 0 array on 4 IDE disks.
>
> Could be.
>
> > The hardware on the box is : Asus p2b-ds, 2x p3-600, 1Gb (ECC) ram, 3
>
> My mobo is whatever came from dell, and you also are running 2xP3. My
> ram is also ECC but there's only 128MB of it.
>
> > network cards (1x Intel EtherExpressPro, 2x 3c905 tx), Internal adaptect
>
> I have 2 network cards, both EEPRO.
>
> > 29xx u2w scsi, internal intel IDE, 2x Seagate Cheetah (u2w) 18 Gb disks
>
> Yep, I have internal adaptec too. Aic7xxx running ultra 160 at 20MHz
> on terminated cable.
>
> Adaptec AIC7xxx driver version: 6.2.1
> aic7892: Ultra160 Wide Channel A, SCSI Id=7, 32/255 SCBs
>
> 4 WD disks:
>
> Host: scsi0 Channel: 00 Id: 00 Lun: 00
> Vendor: WDIGTL Model: WDE9100 ULTRA2 Rev: 1.21
> Type: Direct-Access ANSI SCSI revision: 02
> Host: scsi0 Channel: 00 Id: 01 Lun: 00
> Vendor: WDIGTL Model: WDE9100 ULTRA2 Rev: 1.21
> Type: Direct-Access ANSI SCSI revision: 02
> Host: scsi0 Channel: 00 Id: 02 Lun: 00
> Vendor: WDIGTL Model: WDE9100 ULTRA2 Rev: 1.21
> Type: Direct-Access ANSI SCSI revision: 02
> Host: scsi0 Channel: 00 Id: 03 Lun: 00
> Vendor: WDIGTL Model: WDE9100 ULTRA2 Rev: 1.21
> Type: Direct-Access ANSI SCSI revision: 02
>
> > (/ and /var), 4x 80 Gb Maxtor IDE disks (raid 0 array) and a NVidia TNT2
> > card. This hardware
>
> Umm .. I think I run ati rage, external card, though there is one on
> the mobo.
>
> (--) PCI:*(0:16:0) ATI Mach64 GU rev 154, Mem @ 0xf5000000/24,
> 0xfe201000/12, I/O @ 0xd400/8
> (--) PCI: (1:0:0) ATI Mach64 GW rev 122, Mem @ 0xfc000000/24,
> 0xfbfff000/12, I/O @ 0xec00/8
>
> > The kernel is compiled with all network- and scsi card and raid0 drivers
> > build in, and nfs + iptables as modules. The machine currently uses ext3
>
> I have it all compiled OUT. Including iptables, which I don't use.
>
> > (also build in), however this problem was also present before i
> > converted the raid0 volume to ext3, so i do not suspect it to cause this
>
> I am using xfs on top of lvm on top of raid5.
>
> > problem. The kernel is also set for HIGHMEM (4gb) to use the last Mb's
> > of the 1Gb of ram (else 127Mb isnt detected).
>
> Mine isn't. Normal setup.
>
> > I do not know which component (iptables / route hack / raid0 / network
> > cards / highmem) cause this problem. I run several of these components
>
> Looks from this as though it might be raid5 or 0 + adaptec scsi + SMP.
>
> Peter
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Severe Linux 2.4 kernel memory leakage
2001-11-25 15:30 ` Chris Chabot
@ 2001-11-25 15:41 ` Phil Sorber
2001-11-25 15:51 ` Chris Chabot
2001-11-25 15:53 ` François Cami
2001-11-25 15:55 ` Mr. Shannon Aldinger
1 sibling, 2 replies; 19+ messages in thread
From: Phil Sorber @ 2001-11-25 15:41 UTC (permalink / raw)
To: Chris Chabot; +Cc: Florian Weimer, linux-kernel@vger.kernel.org
[-- Attachment #1: Type: text/plain, Size: 304 bytes --]
On Sun, 2001-11-25 at 10:30, Chris Chabot wrote:
> The kernel i ran for about a month was kernel 2.4.11.
>
wasn't kernel 2.4.11 labeled "dontuse"?
that had a serious bug in it.
>
--
Phil Sorber
AIM: PSUdaemon
IRC: irc.openprojects.net #psulug PSUdaemon
GnuPG: keyserver - pgp.mit.edu
[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Severe Linux 2.4 kernel memory leakage
2001-11-25 15:41 ` Phil Sorber
@ 2001-11-25 15:51 ` Chris Chabot
2001-11-25 17:17 ` Phil Sorber
2001-11-25 15:53 ` François Cami
1 sibling, 1 reply; 19+ messages in thread
From: Chris Chabot @ 2001-11-25 15:51 UTC (permalink / raw)
To: Phil Sorber; +Cc: Florian Weimer, linux-kernel@vger.kernel.org
Hi Phil, I think you are right. When i look @ /boot (good way of seeing,
which & when) it tells me that my upgrade schedule was:
Aug 22 2001 bzImage-2.4.9
Sep 27 07:17 bzImage-2.4.10
Oct 12 04:27 bzImage-2.4.11
Oct 12 16:58 bzImage-2.4.12
Nov 8 17:26 bzImage-2.4.13
Nov 10 10:41 bzImage-2.4.14
Nov 24 10:46 bzImage-2.4.15
So apearantly 2.4.11 was dont use (2.4.12 followed later the same day in
my upgrade cycle). Then for almost a month no upgrades while running
2.4.12, and from there on folowing the kernel upgrade cycle again.
-- Chris
On Sun, 2001-11-25 at 16:41, Phil Sorber wrote:
> On Sun, 2001-11-25 at 10:30, Chris Chabot wrote:
> > The kernel i ran for about a month was kernel 2.4.11.
> >
>
> wasn't kernel 2.4.11 labeled "dontuse"?
>
> that had a serious bug in it.
>
> >
> --
> Phil Sorber
> AIM: PSUdaemon
> IRC: irc.openprojects.net #psulug PSUdaemon
> GnuPG: keyserver - pgp.mit.edu
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Severe Linux 2.4 kernel memory leakage
2001-11-25 15:41 ` Phil Sorber
2001-11-25 15:51 ` Chris Chabot
@ 2001-11-25 15:53 ` François Cami
1 sibling, 0 replies; 19+ messages in thread
From: François Cami @ 2001-11-25 15:53 UTC (permalink / raw)
To: Phil Sorber; +Cc: Chris Chabot, Florian Weimer, linux-kernel@vger.kernel.org
Phil Sorber wrote:
> On Sun, 2001-11-25 at 10:30, Chris Chabot wrote:
>
>>The kernel i ran for about a month was kernel 2.4.11.
>>
>>
>
> wasn't kernel 2.4.11 labeled "dontuse"?
>
> that had a serious bug in it.
>
>
I'm wondering why 2.4.15-greased-turkey isn't labelled
2.4.15-bad-turkey, or in other words 2.4.15-dontuse
François
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Severe Linux 2.4 kernel memory leakage
2001-11-25 15:30 ` Chris Chabot
2001-11-25 15:41 ` Phil Sorber
@ 2001-11-25 15:55 ` Mr. Shannon Aldinger
2001-11-25 16:14 ` Florian Weimer
2001-11-26 15:11 ` Christoph Rohland
1 sibling, 2 replies; 19+ messages in thread
From: Mr. Shannon Aldinger @ 2001-11-25 15:55 UTC (permalink / raw)
To: linux-kernel
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 25 Nov 2001, Chris Chabot wrote:
> Of 430Mb, (counting ps aux res values), just below 80 Mb is used by the
> applications. the rest is just 'missing'.
>
Are you using tmpfs, that had problems in the earlier 2.4.x's IIRC.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: Made with pgp4pine 1.76
iEYEARECAAYFAjwBFHwACgkQwtU6L/A4vVDmzgCeITZ6/njcztWClfPfthOGTnfE
io8An0l2BPZIyJGhhXijFfYoTl/OsTyL
=Q2bB
-----END PGP SIGNATURE-----
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Severe Linux 2.4 kernel memory leakage
2001-11-25 15:55 ` Mr. Shannon Aldinger
@ 2001-11-25 16:14 ` Florian Weimer
2001-11-26 15:11 ` Christoph Rohland
1 sibling, 0 replies; 19+ messages in thread
From: Florian Weimer @ 2001-11-25 16:14 UTC (permalink / raw)
To: linux-kernel
"Mr. Shannon Aldinger" <god@yinyang.hjsoft.com> writes:
> Are you using tmpfs, that had problems in the earlier 2.4.x's IIRC.
I've seen tmpfs problems with 2.4.13+xfs, BTW: As soon as /tmp grows
so large that something has to be swapped out, the machine essentially
locks. Known problem?
(I'm going to debug this some day and provide more details, but
currently, I'm busy setting up a new machine, and this one can't be
used for such testing any longer.)
--
Florian Weimer Florian.Weimer@RUS.Uni-Stuttgart.DE
University of Stuttgart http://cert.uni-stuttgart.de/
RUS-CERT +49-711-685-5973/fax +49-711-685-5898
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Severe Linux 2.4 kernel memory leakage
2001-11-25 15:27 ` Peter T. Breuer
2001-11-25 15:39 ` Chris Chabot
@ 2001-11-25 17:03 ` Andi Kleen
2001-11-26 20:15 ` Bill Davidsen
1 sibling, 1 reply; 19+ messages in thread
From: Andi Kleen @ 2001-11-25 17:03 UTC (permalink / raw)
To: ptb; +Cc: linux-kernel
"Peter T. Breuer" <ptb@it.uc3m.es> writes:
> "A month of sundays ago Chris Chabot wrote:"
> > The box has ran Redhat 7.1 and 7.2, with plain vanilla linux kernels
> > 2.4.9 upto 2.4.15, in all situations the same problem appeared.
> >
> > The problem is that when the box boots up, it uses about 60Mb of memory.
> > However after only 1 1/2 days, the memory usage is already around 430Mb
> > (!!). (this is ofcource used - buffers - cache, as displayed by 'free').
>
> I also have this problem. Unknown circumstances provoke it. Kernel
> 2.4.9 to 2.4.13. When it occurs I lose about 30MB a day.
Compare snapshots of /proc/slabinfo before and after.
It may be completely harmless; e.g. a slab cache. free is unfortunately
quite misleading with newer kernels; it doesn't give information about
many important caches (e.g. not about the slab caches)
-Andi
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Severe Linux 2.4 kernel memory leakage
2001-11-25 15:51 ` Chris Chabot
@ 2001-11-25 17:17 ` Phil Sorber
2001-11-25 18:44 ` Chris Chabot
0 siblings, 1 reply; 19+ messages in thread
From: Phil Sorber @ 2001-11-25 17:17 UTC (permalink / raw)
To: Chris Chabot; +Cc: Florian Weimer, linux-kernel@vger.kernel.org
[-- Attachment #1: Type: text/plain, Size: 351 bytes --]
On Sun, 2001-11-25 at 10:51, Chris Chabot wrote:
> Nov 24 10:46 bzImage-2.4.15
are you running this now? cause it has a major bug too :) i am running
it, but i patched. just a heads up if you didn't see this on the list
already...
--
Phil Sorber
AIM: PSUdaemon
IRC: irc.openprojects.net #psulug PSUdaemon
GnuPG: keyserver - pgp.mit.edu
[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Severe Linux 2.4 kernel memory leakage
2001-11-25 17:17 ` Phil Sorber
@ 2001-11-25 18:44 ` Chris Chabot
0 siblings, 0 replies; 19+ messages in thread
From: Chris Chabot @ 2001-11-25 18:44 UTC (permalink / raw)
To: Phil Sorber; +Cc: Florian Weimer, linux-kernel@vger.kernel.org
Thanks for the headsup, yea i saw the news on ... well just about every
linux news site, and add a few ;-) Figured i could wait for 2.4.16 since
i hope it will be released before i have to unmount anything ;-)
-- Chris
On Sun, 2001-11-25 at 18:17, Phil Sorber wrote:
> On Sun, 2001-11-25 at 10:51, Chris Chabot wrote:
> > Nov 24 10:46 bzImage-2.4.15
>
> are you running this now? cause it has a major bug too :) i am running
> it, but i patched. just a heads up if you didn't see this on the list
> already...
>
> --
> Phil Sorber
> AIM: PSUdaemon
> IRC: irc.openprojects.net #psulug PSUdaemon
> GnuPG: keyserver - pgp.mit.edu
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Severe Linux 2.4 kernel memory leakage
2001-11-25 14:49 Severe Linux 2.4 kernel memory leakage Chris Chabot
` (2 preceding siblings ...)
2001-11-25 15:27 ` Peter T. Breuer
@ 2001-11-26 6:49 ` Mike Galbraith
2001-11-26 9:50 ` Chris Chabot
3 siblings, 1 reply; 19+ messages in thread
From: Mike Galbraith @ 2001-11-26 6:49 UTC (permalink / raw)
To: Chris Chabot; +Cc: linux-kernel
On 25 Nov 2001, Chris Chabot wrote:
> Hi, I have a firewall / file server box which is displaying (severe)
> memory leakage, presumably by the kernel.
>
> The box has ran Redhat 7.1 and 7.2, with plain vanilla linux kernels
> 2.4.9 upto 2.4.15, in all situations the same problem appeared.
With 2.4.9 as well? I have an IKD patch for 2.4.7 which I could
update to 2.4.9 fairly quickly if you'd like to try memleak on the
thing. It might even go in fairly cleanly as is.
-Mike
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Severe Linux 2.4 kernel memory leakage
2001-11-26 6:49 ` Mike Galbraith
@ 2001-11-26 9:50 ` Chris Chabot
0 siblings, 0 replies; 19+ messages in thread
From: Chris Chabot @ 2001-11-26 9:50 UTC (permalink / raw)
To: Mike Galbraith; +Cc: linux-kernel
After i recieved an email from Peter T. who had the same problem, but
_not_ under 2.4.9 i re-checked, and indeed, the problems dont appear in
kernel versions =< 2.4.9. So in either 2.4.10 or 2.4.11 the memory
leakage was 'introduced'.
My current preminition is that it could be the software raid layer thats
causing the leakage, but it also could be a combination of factors. (see
prev email to Peter T / lkml about the common factors in the 2
situations). On the other hand, i'm willing to try anything ;-)
-- Chris
On Mon, 2001-11-26 at 07:49, Mike Galbraith wrote:
> On 25 Nov 2001, Chris Chabot wrote:
>
> > Hi, I have a firewall / file server box which is displaying (severe)
> > memory leakage, presumably by the kernel.
> >
> > The box has ran Redhat 7.1 and 7.2, with plain vanilla linux kernels
> > 2.4.9 upto 2.4.15, in all situations the same problem appeared.
>
> With 2.4.9 as well? I have an IKD patch for 2.4.7 which I could
> update to 2.4.9 fairly quickly if you'd like to try memleak on the
> thing. It might even go in fairly cleanly as is.
>
> -Mike
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Severe Linux 2.4 kernel memory leakage
2001-11-25 15:55 ` Mr. Shannon Aldinger
2001-11-25 16:14 ` Florian Weimer
@ 2001-11-26 15:11 ` Christoph Rohland
1 sibling, 0 replies; 19+ messages in thread
From: Christoph Rohland @ 2001-11-26 15:11 UTC (permalink / raw)
To: god; +Cc: linux-kernel
Hi Shannon,
On Sun, 25 Nov 2001, Shannon Aldinger wrote:
> Are you using tmpfs, that had problems in the earlier 2.4.x's IIRC.
tmpfs always shows up as cached (stock) or shared (in -ac)
Greetings
Christoph
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Severe Linux 2.4 kernel memory leakage
2001-11-25 17:03 ` Andi Kleen
@ 2001-11-26 20:15 ` Bill Davidsen
0 siblings, 0 replies; 19+ messages in thread
From: Bill Davidsen @ 2001-11-26 20:15 UTC (permalink / raw)
To: Andi Kleen; +Cc: Linux Kernel Mailing List
On Sun, 25 Nov 2001, Andi Kleen wrote:
> "Peter T. Breuer" <ptb@it.uc3m.es> writes:
>
> > "A month of sundays ago Chris Chabot wrote:"
> > > The box has ran Redhat 7.1 and 7.2, with plain vanilla linux kernels
> > > 2.4.9 upto 2.4.15, in all situations the same problem appeared.
> > >
> > > The problem is that when the box boots up, it uses about 60Mb of memory.
> > > However after only 1 1/2 days, the memory usage is already around 430Mb
> > > (!!). (this is ofcource used - buffers - cache, as displayed by 'free').
> >
> > I also have this problem. Unknown circumstances provoke it. Kernel
> > 2.4.9 to 2.4.13. When it occurs I lose about 30MB a day.
>
> Compare snapshots of /proc/slabinfo before and after.
This may be useful, but I've never seen anything like that magnitude of
usage, either on dns servers (some of mine are up ~150 days), or usenet
servers (several about to hit the 497 day problem). It will be
insteresting to see what's reported, though.
> It may be completely harmless; e.g. a slab cache. free is unfortunately
> quite misleading with newer kernels; it doesn't give information about
> many important caches (e.g. not about the slab caches)
--
bill davidsen <davidsen@tmr.com>
CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.
^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2001-11-26 20:22 UTC | newest]
Thread overview: 19+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2001-11-25 14:49 Severe Linux 2.4 kernel memory leakage Chris Chabot
2001-11-25 15:03 ` Florian Weimer
2001-11-25 15:30 ` Chris Chabot
2001-11-25 15:41 ` Phil Sorber
2001-11-25 15:51 ` Chris Chabot
2001-11-25 17:17 ` Phil Sorber
2001-11-25 18:44 ` Chris Chabot
2001-11-25 15:53 ` François Cami
2001-11-25 15:55 ` Mr. Shannon Aldinger
2001-11-25 16:14 ` Florian Weimer
2001-11-26 15:11 ` Christoph Rohland
2001-11-25 15:10 ` James Morris
2001-11-25 15:19 ` Chris Chabot
2001-11-25 15:27 ` Peter T. Breuer
2001-11-25 15:39 ` Chris Chabot
2001-11-25 17:03 ` Andi Kleen
2001-11-26 20:15 ` Bill Davidsen
2001-11-26 6:49 ` Mike Galbraith
2001-11-26 9:50 ` Chris Chabot
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox