* mdadm memory leak?
@ 2005-07-05 4:24 David Kowis
2005-07-05 4:30 ` David Kowis
2005-07-05 4:57 ` Neil Brown
0 siblings, 2 replies; 20+ messages in thread
From: David Kowis @ 2005-07-05 4:24 UTC (permalink / raw)
To: linux-raid; +Cc: Eric Sandall
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
I'm not entirely sure if this is mdadm's fault, but I cannot find anything else that would cause the problem, since mdadm is the only new
thing and I'm pretty sure it's not 2.6.11.12's fault. Anyways, on to my issue:
I'm running samba, apache2, mysql, postgresql, and a few other things. I've got an Athlon-XP 1700+ with 768Mb RAM. Right after startup I've
got about 600Mb of free memory, and as time progresses, and I use samba for things (playing an MP3,) my amount of free memory declines
rather rapidly. It hovers around 8Mb of free ram, with no swap usage. The computer has bogged down bad enough that oom-killer has had to
kill just about everything. ps and top don't show anything eating up all my memory. I'm very impressed with mdadm and I'd like to keep using
it, but i'd also like to have a bit of free memory on my computer. I'm using an XFS file system on a 200Gb mirrored RAID array, two drives,
on seperate IDE channels (seperate cables.)
Thanks for your time,
- --
David Kowis
ISO Team Lead - www.sourcemage.org
SourceMage GNU/Linux
One login to rule them all, one login to find them. One login to bring them all, and in the web bind them.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (MingW32)
iD8DBQFCygt+tgErhgxHMHsRAqr+AJ9BtFcFXGKJtE7Ec5mZ25rRn7SD0ACePwzo
C86x7ogIqvFbtQRsgUZihwg=
=gX4c
-----END PGP SIGNATURE-----
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: mdadm memory leak?
2005-07-05 4:24 mdadm memory leak? David Kowis
@ 2005-07-05 4:30 ` David Kowis
2005-07-05 4:59 ` Guy
2005-07-05 4:57 ` Neil Brown
1 sibling, 1 reply; 20+ messages in thread
From: David Kowis @ 2005-07-05 4:30 UTC (permalink / raw)
To: linux-raid; +Cc: Eric Sandall
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
whoops, I was mistaken, and a fool for not checking, but I don't use XFS, it's reiserfs on the 200gb array.
Sorry about the second mail.
David Kowis wrote:
> I'm not entirely sure if this is mdadm's fault, but I cannot find anything else that would cause the problem, since mdadm is the only new
> thing and I'm pretty sure it's not 2.6.11.12's fault. Anyways, on to my issue:
> I'm running samba, apache2, mysql, postgresql, and a few other things. I've got an Athlon-XP 1700+ with 768Mb RAM. Right after startup I've
> got about 600Mb of free memory, and as time progresses, and I use samba for things (playing an MP3,) my amount of free memory declines
> rather rapidly. It hovers around 8Mb of free ram, with no swap usage. The computer has bogged down bad enough that oom-killer has had to
> kill just about everything. ps and top don't show anything eating up all my memory. I'm very impressed with mdadm and I'd like to keep using
> it, but i'd also like to have a bit of free memory on my computer. I'm using an XFS file system on a 200Gb mirrored RAID array, two drives,
> on seperate IDE channels (seperate cables.)
> Thanks for your time,
- --
David Kowis
ISO Team Lead - www.sourcemage.org
SourceMage GNU/Linux
One login to rule them all, one login to find them. One login to bring them all, and in the web bind them.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (MingW32)
iD8DBQFCygzvtgErhgxHMHsRAp5+AJsHDK40HwrDu+G4KT4d7zotJTXV0QCeLf3E
wqSc7wKPBHuTPOyruAuJOKY=
=yXjm
-----END PGP SIGNATURE-----
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: mdadm memory leak?
2005-07-05 4:24 mdadm memory leak? David Kowis
2005-07-05 4:30 ` David Kowis
@ 2005-07-05 4:57 ` Neil Brown
2005-07-05 15:36 ` Eric Sandall
` (2 more replies)
1 sibling, 3 replies; 20+ messages in thread
From: Neil Brown @ 2005-07-05 4:57 UTC (permalink / raw)
To: David Kowis; +Cc: linux-raid, Eric Sandall
On Monday July 4, dkowis@shlrm.org wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> I'm not entirely sure if this is mdadm's fault, but I cannot find anything else that would cause the problem, since mdadm is the only new
> thing and I'm pretty sure it's not 2.6.11.12's fault. Anyways, on to my issue:
> I'm running samba, apache2, mysql, postgresql, and a few other things. I've got an Athlon-XP 1700+ with 768Mb RAM. Right after startup I've
> got about 600Mb of free memory, and as time progresses, and I use samba for things (playing an MP3,) my amount of free memory declines
> rather rapidly. It hovers around 8Mb of free ram, with no swap usage. The computer has bogged down bad enough that oom-killer has had to
> kill just about everything. ps and top don't show anything eating up all my memory. I'm very impressed with mdadm and I'd like to keep using
> it, but i'd also like to have a bit of free memory on my computer. I'm using an XFS file system on a 200Gb mirrored RAID array, two drives,
> on seperate IDE channels (seperate cables.)
> Thanks for your time,
Hmmm.
There is an md related memory leak in 2.6.12, but I don't think it is
there in 2.6.11.anything.
If 'ps' doesn't show anything, the next place to look is
/proc/slabinfo (which 'slabtop' might display for you).
What does 'cat /proc/slabinfo' show?
NeilBrown
^ permalink raw reply [flat|nested] 20+ messages in thread
* RE: mdadm memory leak?
2005-07-05 4:30 ` David Kowis
@ 2005-07-05 4:59 ` Guy
2005-07-05 15:49 ` Eric Sandall
2005-07-05 15:56 ` David Kowis
0 siblings, 2 replies; 20+ messages in thread
From: Guy @ 2005-07-05 4:59 UTC (permalink / raw)
To: 'David Kowis', linux-raid; +Cc: 'Eric Sandall'
Run "ipcs" to see if you have shared memory usage that seems wrong, or
grows.
# ipcs -m
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
I have none, but I don't have a database. Databases I have used on other
systems tend to use a lot of shared memory.
Also, if you were really out of memory, you would have active swapping.
It is normal for the buffer cache to use most "unused" memory, so it would
seem like you have almost none free, but the buffer cache will give up
memory when needed. I think you have another problem, not memory related.
Also, you can stop mdadm from running. The system will still work, just not
monitor the arrays. If you really think it is mdadm related, kill it. Or
use "/etc/init.d/mdmonitor stop". At least as a test.
Run top, look at this line:
Mem: 515296k av, 508128k used, 7168k free, 0k shrd, 128412k buff
I think buff (128412k) is the amount of "unused" memory. But not sure. I
never had a memory issue with Linux, so have not researched this. But I
have on other Unixes.
Guy
> -----Original Message-----
> From: linux-raid-owner@vger.kernel.org [mailto:linux-raid-
> owner@vger.kernel.org] On Behalf Of David Kowis
> Sent: Tuesday, July 05, 2005 12:31 AM
> To: linux-raid@vger.kernel.org
> Cc: Eric Sandall
> Subject: Re: mdadm memory leak?
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> whoops, I was mistaken, and a fool for not checking, but I don't use XFS,
> it's reiserfs on the 200gb array.
> Sorry about the second mail.
>
> David Kowis wrote:
> > I'm not entirely sure if this is mdadm's fault, but I cannot find
> anything else that would cause the problem, since mdadm is the only new
> > thing and I'm pretty sure it's not 2.6.11.12's fault. Anyways, on to my
> issue:
> > I'm running samba, apache2, mysql, postgresql, and a few other things.
> I've got an Athlon-XP 1700+ with 768Mb RAM. Right after startup I've
> > got about 600Mb of free memory, and as time progresses, and I use samba
> for things (playing an MP3,) my amount of free memory declines
> > rather rapidly. It hovers around 8Mb of free ram, with no swap usage.
> The computer has bogged down bad enough that oom-killer has had to
> > kill just about everything. ps and top don't show anything eating up all
> my memory. I'm very impressed with mdadm and I'd like to keep using
> > it, but i'd also like to have a bit of free memory on my computer. I'm
> using an XFS file system on a 200Gb mirrored RAID array, two drives,
> > on seperate IDE channels (seperate cables.)
> > Thanks for your time,
>
> - --
> David Kowis
>
> ISO Team Lead - www.sourcemage.org
> SourceMage GNU/Linux
>
> One login to rule them all, one login to find them. One login to bring
> them all, and in the web bind them.
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.1 (MingW32)
>
> iD8DBQFCygzvtgErhgxHMHsRAp5+AJsHDK40HwrDu+G4KT4d7zotJTXV0QCeLf3E
> wqSc7wKPBHuTPOyruAuJOKY=
> =yXjm
> -----END PGP SIGNATURE-----
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: mdadm memory leak?
2005-07-05 4:57 ` Neil Brown
@ 2005-07-05 15:36 ` Eric Sandall
2005-07-05 21:08 ` Neil Brown
2005-07-05 15:52 ` David Kowis
2005-07-05 21:41 ` David Kowis
2 siblings, 1 reply; 20+ messages in thread
From: Eric Sandall @ 2005-07-05 15:36 UTC (permalink / raw)
To: Neil Brown; +Cc: David Kowis, linux-raid, Eric Sandall
On Tue, 5 Jul 2005, Neil Brown wrote:
> On Monday July 4, dkowis@shlrm.org wrote:
>> I'm not entirely sure if this is mdadm's fault, but I cannot find anything else that would cause the problem, since mdadm is the only new
>> thing and I'm pretty sure it's not 2.6.11.12's fault. Anyways, on to my issue:
>> I'm running samba, apache2, mysql, postgresql, and a few other things. I've got an Athlon-XP 1700+ with 768Mb RAM. Right after startup I've
>> got about 600Mb of free memory, and as time progresses, and I use samba for things (playing an MP3,) my amount of free memory declines
>> rather rapidly. It hovers around 8Mb of free ram, with no swap usage. The computer has bogged down bad enough that oom-killer has had to
>> kill just about everything. ps and top don't show anything eating up all my memory. I'm very impressed with mdadm and I'd like to keep using
>> it, but i'd also like to have a bit of free memory on my computer. I'm using an XFS file system on a 200Gb mirrored RAID array, two drives,
>> on seperate IDE channels (seperate cables.)
>> Thanks for your time,
>
> Hmmm.
> There is an md related memory leak in 2.6.12, but I don't think it is
> there in 2.6.11.anything.
I also have this problem, but am using 2.6.10-as7 (also reiserfs on
the raid-1 array). This machine has 1Gb RAM, but has roughly 2-40Mb
free at any time after running for a day (I can leave the machine up
as long as I want, but all processes run slow as they now have to do a
lot of swapping). The kernel was compiled with gcc 3.4.4 and glibc
2.3.5 with NPTL (if that matters).
> If 'ps' doesn't show anything, the next place to look is
> /proc/slabinfo (which 'slabtop' might display for you).
Active / Total Objects (% used) : 12080357 / 12132001 (99.6%)
Active / Total Slabs (% used) : 176776 / 176776 (100.0%)
Active / Total Caches (% used) : 66 / 101 (65.3%)
Active / Total Size (% used) : 668099.80K / 671784.20K (99.5%)
Minimum / Average / Maximum Object : 0.01K / 0.05K / 128.00K
OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
6013634 6013300 99% 0.02K 26609 226 106436K biovec-1
6013306 6013297 99% 0.09K 146666 41 586664K bio
35625 16267 45% 0.05K 475 75 1900K buffer_head
17446 6289 36% 0.06K 286 61 1144K size-64
10829 3488 32% 0.03K 91 119 364K size-32
8613 5222 60% 0.13K 297 29 1188K dentry_cache
7490 6923 92% 0.27K 535 14 2140K radix_tree_node
6486 3436 52% 0.08K 138 47 552K vm_area_struct
3195 2092 65% 0.25K 213 15 852K ip_dst_cache
2573 1888 73% 0.12K 83 31 332K size-128
2442 890 36% 0.01K 6 407 24K anon_vma
1425 886 62% 0.16K 57 25 228K filp
1419 1317 92% 0.35K 129 11 516K
reiser_inode_cache
798 651 81% 0.28K 57 14 228K inode_cache
420 240 57% 0.19K 21 20 84K
skbuff_head_cache
420 365 86% 0.37K 42 10 168K
shmem_inode_cache
359 359 100% 4.00K 359 1 1436K size-4096
338 262 77% 0.29K 26 13 104K
proc_inode_cache
305 256 83% 0.06K 5 61 20K biovec-4
260 256 98% 0.19K 13 20 52K biovec-16
260 256 98% 0.75K 52 5 208K biovec-64
260 256 98% 1.50K 52 5 416K biovec-128
256 256 100% 3.00K 128 2 1024K biovec-(256)
244 35 14% 0.06K 4 61 16K inet_peer_cache
238 56 23% 0.03K 2 119 8K fs_cache
226 24 10% 0.02K 1 226 4K tcp_bind_bucket
226 13 5% 0.02K 1 226 4K ip_fib_alias
216 189 87% 0.50K 27 8 108K size-512
205 77 37% 0.09K 5 41 20K key_jar
196 181 92% 2.00K 98 2 392K size-2048
187 76 40% 0.34K 17 11 68K ip_conntrack
183 21 11% 0.06K 3 61 12K as_arq
...
> What does 'cat /proc/slabinfo' show?
slabinfo - version: 2.1
# name <active_objs> <num_objs> <objsize> <objperslab> <pagesperslab> : tunables <batchcount> <limit> <sharedfactor> : slabdata <active_slabs> <num_slabs> <sharedavail>
ip_conntrack_expect 0 0 128 31 1 : tunables 120 60 0 : slabdata 0 0 0
ip_conntrack 40 187 352 11 1 : tunables 54 27 0 : slabdata 17 17 0
ip_fib_alias 13 226 16 226 1 : tunables 120 60 0 : slabdata 1 1 0
ip_fib_hash 13 119 32 119 1 : tunables 120 60 0 : slabdata 1 1 0
rpc_buffers 8 8 2048 2 1 : tunables 24 12 0 : slabdata 4 4 0
rpc_tasks 8 25 160 25 1 : tunables 120 60 0 : slabdata 1 1 0
rpc_inode_cache 0 0 384 10 1 : tunables 54 27 0 : slabdata 0 0 0
unix_sock 17 20 384 10 1 : tunables 54 27 0 : slabdata 2 2 0
tcp_tw_bucket 0 0 96 41 1 : tunables 120 60 0 : slabdata 0 0 0
tcp_bind_bucket 24 226 16 226 1 : tunables 120 60 0 : slabdata 1 1 0
tcp_open_request 0 0 64 61 1 : tunables 120 60 0 : slabdata 0 0 0
inet_peer_cache 35 244 64 61 1 : tunables 120 60 0 : slabdata 4 4 0
secpath_cache 0 0 128 31 1 : tunables 120 60 0 : slabdata 0 0 0
xfrm_dst_cache 0 0 256 15 1 : tunables 120 60 0 : slabdata 0 0 0
ip_dst_cache 2085 3195 256 15 1 : tunables 120 60 0 : slabdata 213 213 0
arp_cache 8 31 128 31 1 : tunables 120 60 0 : slabdata 1 1 0
raw_sock 3 8 480 8 1 : tunables 54 27 0 : slabdata 1 1 0
udp_sock 20 24 480 8 1 : tunables 54 27 0 : slabdata 3 3 0
tcp_sock 26 40 1024 4 1 : tunables 54 27 0 : slabdata 10 10 0
flow_cache 0 0 96 41 1 : tunables 120 60 0 : slabdata 0 0 0
dm_tio 0 0 16 226 1 : tunables 120 60 0 : slabdata 0 0 0
dm_io 0 0 16 226 1 : tunables 120 60 0 : slabdata 0 0 0
cfq_ioc_pool 0 0 24 156 1 : tunables 120 60 0 : slabdata 0 0 0
cfq_pool 0 0 104 38 1 : tunables 120 60 0 : slabdata 0 0 0
crq_pool 0 0 56 70 1 : tunables 120 60 0 : slabdata 0 0 0
deadline_drq 0 0 52 75 1 : tunables 120 60 0 : slabdata 0 0 0
as_arq 26 61 64 61 1 : tunables 120 60 0 : slabdata 1 1 0
reiser_inode_cache 1300 1419 360 11 1 : tunables 54 27 0 : slabdata 129 129 0
dnotify_cache 0 0 20 185 1 : tunables 120 60 0 : slabdata 0 0 0
eventpoll_pwq 0 0 36 107 1 : tunables 120 60 0 : slabdata 0 0 0
eventpoll_epi 0 0 96 41 1 : tunables 120 60 0 : slabdata 0 0 0
kioctx 0 0 160 25 1 : tunables 120 60 0 : slabdata 0 0 0
kiocb 0 0 128 31 1 : tunables 120 60 0 : slabdata 0 0 0
fasync_cache 0 0 16 226 1 : tunables 120 60 0 : slabdata 0 0 0
shmem_inode_cache 380 420 376 10 1 : tunables 54 27 0 : slabdata 42 42 0
posix_timers_cache 0 0 96 41 1 : tunables 120 60 0 : slabdata 0 0 0
uid_cache 8 61 64 61 1 : tunables 120 60 0 : slabdata 1 1 0
blkdev_ioc 64 156 24 156 1 : tunables 120 60 0 : slabdata 1 1 0
blkdev_queue 7 11 352 11 1 : tunables 54 27 0 : slabdata 1 1 0
blkdev_requests 18 27 148 27 1 : tunables 120 60 0 : slabdata 1 1 0
biovec-(256) 256 256 3072 2 2 : tunables 24 12 0 : slabdata 128 128 0
biovec-128 256 260 1536 5 2 : tunables 24 12 0 : slabdata 52 52 0
biovec-64 256 260 768 5 1 : tunables 54 27 0 : slabdata 52 52 0
biovec-16 256 260 192 20 1 : tunables 120 60 0 : slabdata 13 13 0
biovec-4 258 305 64 61 1 : tunables 120 60 0 : slabdata 5 5 0
biovec-1 6013296 6013634 16 226 1 : tunables 120 60 0 : slabdata 26609 26609 0
bio 6013291 6013306 96 41 1 : tunables 120 60 0 : slabdata 146666 146666 0
file_lock_cache 30 45 88 45 1 : tunables 120 60 0 : slabdata 1 1 0
sock_inode_cache 72 144 320 12 1 : tunables 54 27 0 : slabdata 12 12 0
skbuff_head_cache 240 420 192 20 1 : tunables 120 60 0 : slabdata 21 21 0
sock 4 12 320 12 1 : tunables 54 27 0 : slabdata 1 1 0
key_jar 78 205 96 41 1 : tunables 120 60 0 : slabdata 5 5 0
proc_inode_cache 262 338 300 13 1 : tunables 54 27 0 : slabdata 26 26 0
sigqueue 5 27 148 27 1 : tunables 120 60 0 : slabdata 1 1 0
radix_tree_node 6895 7490 276 14 1 : tunables 54 27 0 : slabdata 535 535 0
bdev_cache 7 10 384 10 1 : tunables 54 27 0 : slabdata 1 1 0
mnt_cache 23 41 96 41 1 : tunables 120 60 0 : slabdata 1 1 0
inode_cache 652 798 284 14 1 : tunables 54 27 0 : slabdata 57 57 0
dentry_cache 5162 8613 136 29 1 : tunables 120 60 0 : slabdata 297 297 0
filp 886 1425 160 25 1 : tunables 120 60 0 : slabdata 57 57 0
names_cache 17 17 4096 1 1 : tunables 24 12 0 : slabdata 17 17 0
idr_layer_cache 64 87 136 29 1 : tunables 120 60 0 : slabdata 3 3 0
buffer_head 16142 35625 52 75 1 : tunables 120 60 0 : slabdata 475 475 0
mm_struct 83 84 544 7 1 : tunables 54 27 0 : slabdata 12 12 0
vm_area_struct 3436 6486 84 47 1 : tunables 120 60 0 : slabdata 138 138 0
fs_cache 57 238 32 119 1 : tunables 120 60 0 : slabdata 2 2 0
files_cache 56 117 416 9 1 : tunables 54 27 0 : slabdata 13 13 0
signal_cache 73 135 256 15 1 : tunables 120 60 0 : slabdata 9 9 0
sighand_cache 70 87 1312 3 1 : tunables 24 12 0 : slabdata 29 29 0
task_struct 92 96 1264 3 1 : tunables 24 12 0 : slabdata 32 32 0
anon_vma 890 2442 8 407 1 : tunables 120 60 0 : slabdata 6 6 0
pgd 57 57 4096 1 1 : tunables 24 12 0 : slabdata 57 57 0
size-131072(DMA) 0 0 131072 1 32 : tunables 8 4 0 : slabdata 0 0 0
size-131072 0 0 131072 1 32 : tunables 8 4 0 : slabdata 0 0 0
size-65536(DMA) 0 0 65536 1 16 : tunables 8 4 0 : slabdata 0 0 0
size-65536 0 0 65536 1 16 : tunables 8 4 0 : slabdata 0 0 0
size-32768(DMA) 0 0 32768 1 8 : tunables 8 4 0 : slabdata 0 0 0
size-32768 1 1 32768 1 8 : tunables 8 4 0 : slabdata 1 1 0
size-16384(DMA) 0 0 16384 1 4 : tunables 8 4 0 : slabdata 0 0 0
size-16384 11 11 16384 1 4 : tunables 8 4 0 : slabdata 11 11 0
size-8192(DMA) 0 0 8192 1 2 : tunables 8 4 0 : slabdata 0 0 0
size-8192 1 1 8192 1 2 : tunables 8 4 0 : slabdata 1 1 0
size-4096(DMA) 0 0 4096 1 1 : tunables 24 12 0 : slabdata 0 0 0
size-4096 355 355 4096 1 1 : tunables 24 12 0 : slabdata 355 355 0
size-2048(DMA) 0 0 2048 2 1 : tunables 24 12 0 : slabdata 0 0 0
size-2048 186 190 2048 2 1 : tunables 24 12 0 : slabdata 95 95 0
size-1024(DMA) 0 0 1024 4 1 : tunables 54 27 0 : slabdata 0 0 0
size-1024 148 148 1024 4 1 : tunables 54 27 0 : slabdata 37 37 0
size-512(DMA) 0 0 512 8 1 : tunables 54 27 0 : slabdata 0 0 0
size-512 189 216 512 8 1 : tunables 54 27 0 : slabdata 27 27 0
size-256(DMA) 0 0 256 15 1 : tunables 120 60 0 : slabdata 0 0 0
size-256 96 105 256 15 1 : tunables 120 60 0 : slabdata 7 7 0
size-192(DMA) 0 0 192 20 1 : tunables 120 60 0 : slabdata 0 0 0
size-192 100 100 192 20 1 : tunables 120 60 0 : slabdata 5 5 0
size-128(DMA) 0 0 128 31 1 : tunables 120 60 0 : slabdata 0 0 0
size-128 1948 2573 128 31 1 : tunables 120 60 0 : slabdata 83 83 0
size-64(DMA) 0 0 64 61 1 : tunables 120 60 0 : slabdata 0 0 0
size-64 6289 17446 64 61 1 : tunables 120 60 0 : slabdata 286 286 0
size-32(DMA) 0 0 32 119 1 : tunables 120 60 0 : slabdata 0 0 0
size-32 3488 10829 32 119 1 : tunables 120 60 0 : slabdata 91 91 0
kmem_cache 124 124 128 31 1 : tunables 120 60 0 : slabdata 4 4 0
-sandalle
--
Eric Sandall | Source Mage GNU/Linux Developer
eric@sandall.us | http://www.sourcemage.org/
http://eric.sandall.us/ | SysAdmin @ Inst. Shock Physics @ WSU
http://counter.li.org/ #196285 | http://www.shock.wsu.edu/
^ permalink raw reply [flat|nested] 20+ messages in thread
* RE: mdadm memory leak?
2005-07-05 4:59 ` Guy
@ 2005-07-05 15:49 ` Eric Sandall
2005-07-05 15:56 ` David Kowis
1 sibling, 0 replies; 20+ messages in thread
From: Eric Sandall @ 2005-07-05 15:49 UTC (permalink / raw)
To: Guy; +Cc: 'David Kowis', linux-raid, 'Eric Sandall'
On Tue, 5 Jul 2005, Guy wrote:
> Run "ipcs" to see if you have shared memory usage that seems wrong, or
> grows.
>
> # ipcs -m
>
> ------ Shared Memory Segments --------
> key shmid owner perms bytes nattch status
$ ipcs -m
------ Shared Memory Segments --------
key shmid owner perms bytes nattch
status
0x73657469 0 nobody 666 131224 1
(probably from my mysql service)
> I have none, but I don't have a database. Databases I have used on other
> systems tend to use a lot of shared memory.
>
> Also, if you were really out of memory, you would have active swapping.
>
> It is normal for the buffer cache to use most "unused" memory, so it would
> seem like you have almost none free, but the buffer cache will give up
> memory when needed. I think you have another problem, not memory related.
> Also, you can stop mdadm from running. The system will still work, just not
> monitor the arrays. If you really think it is mdadm related, kill it. Or
> use "/etc/init.d/mdmonitor stop". At least as a test.
I am not running the mdadm monitor process (not sure about David Kowis).
> Run top, look at this line:
> Mem: 515296k av, 508128k used, 7168k free, 0k shrd, 128412k buff
Mem: 905948k total, 844084k used, 61864k free, 57648k buffers
(I guess my range is 2Mb-60Mb, not -40Mb as I reported earlier.
Granted I'm running less right now to try and ease the load...).
> I think buff (128412k) is the amount of "unused" memory. But not sure. I
> never had a memory issue with Linux, so have not researched this. But I
> have on other Unixes.
Me either, until recently. I don't recall when this started happening,
but after talking with David Kowis it seems that it happened when we
both went from using raidtools to mdadm for our RAID-1 arrays.
<snip>
-sandalle
--
Eric Sandall | Source Mage GNU/Linux Developer
eric@sandall.us | http://www.sourcemage.org/
http://eric.sandall.us/ | SysAdmin @ Inst. Shock Physics @ WSU
http://counter.li.org/ #196285 | http://www.shock.wsu.edu/
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: mdadm memory leak?
2005-07-05 4:57 ` Neil Brown
2005-07-05 15:36 ` Eric Sandall
@ 2005-07-05 15:52 ` David Kowis
2005-07-05 21:23 ` Neil Brown
2005-07-05 21:41 ` David Kowis
2 siblings, 1 reply; 20+ messages in thread
From: David Kowis @ 2005-07-05 15:52 UTC (permalink / raw)
To: Neil Brown; +Cc: linux-raid, Eric Sandall
[-- Attachment #1.1.1: Type: text/plain, Size: 4163 bytes --]
Quoting Neil Brown <neilb@cse.unsw.edu.au>:
> Hmmm.
> There is an md related memory leak in 2.6.12, but I don't think it is
> there in 2.6.11.anything.
>
> If 'ps' doesn't show anything, the next place to look is
> /proc/slabinfo (which 'slabtop' might display for you).
Slabtop:
Active / Total Objects (% used) : 217562 / 225483 (96.5%)
Active / Total Slabs (% used) : 3972 / 3972 (100.0%)
Active / Total Caches (% used) : 78 / 139 (56.1%)
Active / Total Size (% used) : 14328.78K / 15891.08K (90.2%)
Minimum / Average / Maximum Object : 0.01K / 0.07K / 128.00K
OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
152098 151909 99% 0.02K 673 226 2692K fasync_cache
24867 24846 99% 0.05K 307 81 1228K buffer_head
12432 8306 66% 0.27K 888 14 3552K radix_tree_node
7308 6876 94% 0.13K 252 29 1008K dentry_cache
6303 5885 93% 0.36K 573 11 2292K reiser_inode_cache
3690 3684 99% 0.09K 82 45 328K vm_area_struct
2675 2645 98% 0.04K 25 107 100K sysfs_dir_cache
2405 2360 98% 0.29K 185 13 740K inode_cache
2142 2082 97% 0.03K 18 119 72K size-32
2074 1503 72% 0.06K 34 61 136K size-64
1891 1842 97% 0.12K 61 31 244K size-128
814 523 64% 0.01K 2 407 8K anon_vma
750 749 99% 0.16K 30 25 120K filp
452 300 66% 0.02K 2 226 8K biovec-1
305 281 92% 0.06K 5 61 20K bio
305 256 83% 0.06K 5 61 20K biovec-4
260 256 98% 0.19K 13 20 52K biovec-16
260 256 98% 0.75K 52 5 208K biovec-64
260 256 98% 1.50K 52 5 416K biovec-128
256 256 100% 3.00K 128 2 1024K biovec-(256)
247 242 97% 0.30K 19 13 76K proc_inode_cache
226 9 3% 0.02K 1 226 4K ip_fib_alias
226 16 7% 0.02K 1 226 4K tcp_bind_bucket
200 71 35% 0.19K 10 20 40K skbuff_head_cache
184 176 95% 0.50K 23 8 92K size-512
156 53 33% 0.02K 1 156 4K blkdev_ioc
155 155 100% 0.12K 5 31 20K kmem_cache
124 119 95% 1.00K 31 4 124K size-1024
120 120 100% 0.25K 8 15 32K size-256
119 59 49% 0.03K 1 119 4K fs_cache
119 9 7% 0.03K 1 119 4K ip_fib_hash
119 8 6% 0.03K 1 119 4K fib6_nodes
108 108 100% 4.00K 108 1 432K size-4096
99 89 89% 1.25K 33 3 132K task_struct
92 92 100% 2.00K 46 2 184K size-2048
90 80 88% 0.25K 6 15 24K signal_cache
89 89 100% 8.00K 89 1 712K size-8192
88 80 90% 0.34K 8 11 32K sock_inode_cache
87 77 88% 1.28K 29 3 116K sighand_cache
87 86 98% 0.13K 3 29 12K idr_layer_cache
80 63 78% 0.19K 4 20 16K size-192
72 59 81% 0.41K 8 9 32K files_cache
70 59 84% 0.56K 10 7 40K mm_struct
65 24 36% 0.06K 1 65 4K as_arq
62 32 51% 0.12K 2 31 8K sgpool-8
61 7 11% 0.06K 1 61 4K uid_cache
61 1 1% 0.06K 1 61 4K inet_peer_cache
59 59 100% 4.00K 59 1 236K pgd
>
> What does 'cat /proc/slabinfo' show?
I've attached my /proc/slabinfo.
Thanks :)
David
--
One login to rule them all, one login to find them. One login to bring
them all,
and in the web bind them.
----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.
[-- Attachment #1.1.2: /proc/slabinfo --]
[-- Type: application/octet-stream, Size: 15083 bytes --]
slabinfo - version: 2.1
# name <active_objs> <num_objs> <objsize> <objperslab> <pagesperslab> : tunables <limit> <batchcount> <sharedfactor> : slabdata <active_slabs> <num_slabs> <sharedavail>
rpc_buffers 8 8 2048 2 1 : tunables 24 12 0 : slabdata 4 4 0
rpc_tasks 8 20 192 20 1 : tunables 120 60 0 : slabdata 1 1 0
rpc_inode_cache 0 0 416 9 1 : tunables 54 27 0 : slabdata 0 0 0
fib6_nodes 8 119 32 119 1 : tunables 120 60 0 : slabdata 1 1 0
ip6_dst_cache 7 18 224 18 1 : tunables 120 60 0 : slabdata 1 1 0
ndisc_cache 2 25 160 25 1 : tunables 120 60 0 : slabdata 1 1 0
rawv6_sock 3 6 640 6 1 : tunables 54 27 0 : slabdata 1 1 0
udpv6_sock 1 6 608 6 1 : tunables 54 27 0 : slabdata 1 1 0
tcpv6_sock 7 7 1120 7 2 : tunables 24 12 0 : slabdata 1 1 0
unix_sock 33 40 384 10 1 : tunables 54 27 0 : slabdata 4 4 0
ip_conntrack_expect 0 0 84 47 1 : tunables 120 60 0 : slabdata 0 0 0
ip_conntrack 13 19 212 19 1 : tunables 120 60 0 : slabdata 1 1 0
tcp_tw_bucket 2 31 128 31 1 : tunables 120 60 0 : slabdata 1 1 0
tcp_bind_bucket 17 226 16 226 1 : tunables 120 60 0 : slabdata 1 1 0
tcp_open_request 0 0 96 41 1 : tunables 120 60 0 : slabdata 0 0 0
inet_peer_cache 1 61 64 61 1 : tunables 120 60 0 : slabdata 1 1 0
ip_fib_alias 9 226 16 226 1 : tunables 120 60 0 : slabdata 1 1 0
ip_fib_hash 9 119 32 119 1 : tunables 120 60 0 : slabdata 1 1 0
ip_dst_cache 11 30 256 15 1 : tunables 120 60 0 : slabdata 2 2 0
arp_cache 3 31 128 31 1 : tunables 120 60 0 : slabdata 1 1 0
raw_sock 2 8 480 8 1 : tunables 54 27 0 : slabdata 1 1 0
udp_sock 16 16 480 8 1 : tunables 54 27 0 : slabdata 2 2 0
tcp_sock 12 12 1024 4 1 : tunables 54 27 0 : slabdata 3 3 0
flow_cache 0 0 96 41 1 : tunables 120 60 0 : slabdata 0 0 0
uhci_urb_priv 0 0 44 88 1 : tunables 120 60 0 : slabdata 0 0 0
cfq_ioc_pool 0 0 24 156 1 : tunables 120 60 0 : slabdata 0 0 0
cfq_pool 0 0 104 38 1 : tunables 120 60 0 : slabdata 0 0 0
crq_pool 0 0 52 75 1 : tunables 120 60 0 : slabdata 0 0 0
deadline_drq 0 0 48 81 1 : tunables 120 60 0 : slabdata 0 0 0
as_arq 24 130 60 65 1 : tunables 120 60 0 : slabdata 2 2 0
mqueue_inode_cache 1 8 480 8 1 : tunables 54 27 0 : slabdata 1 1 0
xfs_chashlist 0 0 20 185 1 : tunables 120 60 0 : slabdata 0 0 0
xfs_ili 0 0 140 28 1 : tunables 120 60 0 : slabdata 0 0 0
xfs_ifork 0 0 56 70 1 : tunables 120 60 0 : slabdata 0 0 0
xfs_efi_item 0 0 260 15 1 : tunables 54 27 0 : slabdata 0 0 0
xfs_efd_item 0 0 260 15 1 : tunables 54 27 0 : slabdata 0 0 0
xfs_buf_item 0 0 148 27 1 : tunables 120 60 0 : slabdata 0 0 0
xfs_dabuf 0 0 16 226 1 : tunables 120 60 0 : slabdata 0 0 0
xfs_da_state 0 0 336 12 1 : tunables 54 27 0 : slabdata 0 0 0
xfs_trans 0 0 592 13 2 : tunables 54 27 0 : slabdata 0 0 0
xfs_inode 0 0 352 11 1 : tunables 54 27 0 : slabdata 0 0 0
xfs_btree_cur 0 0 132 30 1 : tunables 120 60 0 : slabdata 0 0 0
xfs_bmap_free_item 0 0 12 290 1 : tunables 120 60 0 : slabdata 0 0 0
xfs_buf_t 0 0 224 18 1 : tunables 120 60 0 : slabdata 0 0 0
linvfs_icache 0 0 320 12 1 : tunables 54 27 0 : slabdata 0 0 0
udf_inode_cache 0 0 348 11 1 : tunables 54 27 0 : slabdata 0 0 0
nfs_write_data 36 36 448 9 1 : tunables 54 27 0 : slabdata 4 4 0
nfs_read_data 32 36 448 9 1 : tunables 54 27 0 : slabdata 4 4 0
nfs_inode_cache 0 0 544 7 1 : tunables 54 27 0 : slabdata 0 0 0
nfs_page 0 0 64 61 1 : tunables 120 60 0 : slabdata 0 0 0
isofs_inode_cache 0 0 320 12 1 : tunables 54 27 0 : slabdata 0 0 0
fat_inode_cache 0 0 348 11 1 : tunables 54 27 0 : slabdata 0 0 0
fat_cache 0 0 20 185 1 : tunables 120 60 0 : slabdata 0 0 0
ext2_inode_cache 0 0 400 10 1 : tunables 54 27 0 : slabdata 0 0 0
journal_handle 0 0 20 185 1 : tunables 120 60 0 : slabdata 0 0 0
journal_head 0 0 48 81 1 : tunables 120 60 0 : slabdata 0 0 0
revoke_table 0 0 12 290 1 : tunables 120 60 0 : slabdata 0 0 0
revoke_record 0 0 16 226 1 : tunables 120 60 0 : slabdata 0 0 0
ext3_inode_cache 0 0 472 8 1 : tunables 54 27 0 : slabdata 0 0 0
ext3_xattr 0 0 44 88 1 : tunables 120 60 0 : slabdata 0 0 0
reiser_inode_cache 5617 6303 368 11 1 : tunables 54 27 0 : slabdata 573 573 0
dnotify_cache 0 0 20 185 1 : tunables 120 60 0 : slabdata 0 0 0
eventpoll_pwq 0 0 36 107 1 : tunables 120 60 0 : slabdata 0 0 0
eventpoll_epi 0 0 96 41 1 : tunables 120 60 0 : slabdata 0 0 0
kioctx 0 0 160 25 1 : tunables 120 60 0 : slabdata 0 0 0
kiocb 0 0 128 31 1 : tunables 120 60 0 : slabdata 0 0 0
fasync_cache 151909 152098 16 226 1 : tunables 120 60 0 : slabdata 673 673 0
shmem_inode_cache 7 10 384 10 1 : tunables 54 27 0 : slabdata 1 1 0
posix_timers_cache 0 0 96 41 1 : tunables 120 60 0 : slabdata 0 0 0
uid_cache 7 61 64 61 1 : tunables 120 60 0 : slabdata 1 1 0
sgpool-128 32 32 2048 2 1 : tunables 24 12 0 : slabdata 16 16 0
sgpool-64 32 32 1024 4 1 : tunables 54 27 0 : slabdata 8 8 0
sgpool-32 32 32 512 8 1 : tunables 54 27 0 : slabdata 4 4 0
sgpool-16 32 45 256 15 1 : tunables 120 60 0 : slabdata 3 3 0
sgpool-8 32 62 128 31 1 : tunables 120 60 0 : slabdata 2 2 0
blkdev_ioc 68 156 24 156 1 : tunables 120 60 0 : slabdata 1 1 0
blkdev_queue 14 22 360 11 1 : tunables 54 27 0 : slabdata 2 2 0
blkdev_requests 24 56 140 28 1 : tunables 120 60 0 : slabdata 2 2 0
biovec-(256) 256 256 3072 2 2 : tunables 24 12 0 : slabdata 128 128 0
biovec-128 256 260 1536 5 2 : tunables 24 12 0 : slabdata 52 52 0
biovec-64 256 260 768 5 1 : tunables 54 27 0 : slabdata 52 52 0
biovec-16 256 260 192 20 1 : tunables 120 60 0 : slabdata 13 13 0
biovec-4 256 305 64 61 1 : tunables 120 60 0 : slabdata 5 5 0
biovec-1 258 452 16 226 1 : tunables 120 60 0 : slabdata 2 2 0
bio 261 305 64 61 1 : tunables 120 60 0 : slabdata 5 5 0
file_lock_cache 12 45 88 45 1 : tunables 120 60 0 : slabdata 1 1 0
sock_inode_cache 81 88 352 11 1 : tunables 54 27 0 : slabdata 8 8 0
skbuff_head_cache 72 200 192 20 1 : tunables 120 60 0 : slabdata 10 10 0
sock 5 12 320 12 1 : tunables 54 27 0 : slabdata 1 1 0
proc_inode_cache 247 247 308 13 1 : tunables 54 27 0 : slabdata 19 19 0
sigqueue 27 27 148 27 1 : tunables 120 60 0 : slabdata 1 1 0
radix_tree_node 8272 12432 276 14 1 : tunables 54 27 0 : slabdata 888 888 0
bdev_cache 10 18 416 9 1 : tunables 54 27 0 : slabdata 2 2 0
sysfs_dir_cache 2645 2675 36 107 1 : tunables 120 60 0 : slabdata 25 25 0
mnt_cache 20 41 96 41 1 : tunables 120 60 0 : slabdata 1 1 0
inode_cache 2360 2405 292 13 1 : tunables 54 27 0 : slabdata 185 185 0
dentry_cache 6443 7308 136 29 1 : tunables 120 60 0 : slabdata 252 252 0
filp 750 750 160 25 1 : tunables 120 60 0 : slabdata 30 30 0
names_cache 1 1 4096 1 1 : tunables 24 12 0 : slabdata 1 1 0
idr_layer_cache 86 87 136 29 1 : tunables 120 60 0 : slabdata 3 3 0
buffer_head 24543 24543 48 81 1 : tunables 120 60 0 : slabdata 303 303 0
mm_struct 70 70 576 7 1 : tunables 54 27 0 : slabdata 10 10 0
vm_area_struct 3645 3645 88 45 1 : tunables 120 60 0 : slabdata 81 81 0
fs_cache 74 119 32 119 1 : tunables 120 60 0 : slabdata 1 1 0
files_cache 72 72 416 9 1 : tunables 54 27 0 : slabdata 8 8 0
signal_cache 90 90 256 15 1 : tunables 120 60 0 : slabdata 6 6 0
sighand_cache 87 87 1312 3 1 : tunables 24 12 0 : slabdata 29 29 0
task_struct 99 99 1280 3 1 : tunables 24 12 0 : slabdata 33 33 0
anon_vma 573 814 8 407 1 : tunables 120 60 0 : slabdata 2 2 0
pgd 60 60 4096 1 1 : tunables 24 12 0 : slabdata 60 60 0
size-131072(DMA) 0 0 131072 1 32 : tunables 8 4 0 : slabdata 0 0 0
size-131072 0 0 131072 1 32 : tunables 8 4 0 : slabdata 0 0 0
size-65536(DMA) 0 0 65536 1 16 : tunables 8 4 0 : slabdata 0 0 0
size-65536 0 0 65536 1 16 : tunables 8 4 0 : slabdata 0 0 0
size-32768(DMA) 0 0 32768 1 8 : tunables 8 4 0 : slabdata 0 0 0
size-32768 0 0 32768 1 8 : tunables 8 4 0 : slabdata 0 0 0
size-16384(DMA) 0 0 16384 1 4 : tunables 8 4 0 : slabdata 0 0 0
size-16384 0 0 16384 1 4 : tunables 8 4 0 : slabdata 0 0 0
size-8192(DMA) 0 0 8192 1 2 : tunables 8 4 0 : slabdata 0 0 0
size-8192 89 89 8192 1 2 : tunables 8 4 0 : slabdata 89 89 0
size-4096(DMA) 0 0 4096 1 1 : tunables 24 12 0 : slabdata 0 0 0
size-4096 109 109 4096 1 1 : tunables 24 12 0 : slabdata 109 109 0
size-2048(DMA) 0 0 2048 2 1 : tunables 24 12 0 : slabdata 0 0 0
size-2048 95 96 2048 2 1 : tunables 24 12 0 : slabdata 48 48 0
size-1024(DMA) 0 0 1024 4 1 : tunables 54 27 0 : slabdata 0 0 0
size-1024 118 124 1024 4 1 : tunables 54 27 0 : slabdata 31 31 0
size-512(DMA) 0 0 512 8 1 : tunables 54 27 0 : slabdata 0 0 0
size-512 184 184 512 8 1 : tunables 54 27 0 : slabdata 23 23 0
size-256(DMA) 0 0 256 15 1 : tunables 120 60 0 : slabdata 0 0 0
size-256 120 120 256 15 1 : tunables 120 60 0 : slabdata 8 8 0
size-192(DMA) 0 0 192 20 1 : tunables 120 60 0 : slabdata 0 0 0
size-192 79 80 192 20 1 : tunables 120 60 0 : slabdata 4 4 0
size-128(DMA) 0 0 128 31 1 : tunables 120 60 0 : slabdata 0 0 0
size-128 1812 1891 128 31 1 : tunables 120 60 0 : slabdata 61 61 0
size-64(DMA) 0 0 64 61 1 : tunables 120 60 0 : slabdata 0 0 0
size-64 1501 2074 64 61 1 : tunables 120 60 0 : slabdata 34 34 0
size-32(DMA) 0 0 32 119 1 : tunables 120 60 0 : slabdata 0 0 0
size-32 2082 2142 32 119 1 : tunables 120 60 0 : slabdata 18 18 0
kmem_cache 155 155 128 31 1 : tunables 120 60 0 : slabdata 5 5 0
[-- Attachment #1.2: PGP Digital Signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
[-- Attachment #2: PGP Public Key --]
[-- Type: application/pgp-keys, Size: 1793 bytes --]
^ permalink raw reply [flat|nested] 20+ messages in thread
* RE: mdadm memory leak?
2005-07-05 4:59 ` Guy
2005-07-05 15:49 ` Eric Sandall
@ 2005-07-05 15:56 ` David Kowis
1 sibling, 0 replies; 20+ messages in thread
From: David Kowis @ 2005-07-05 15:56 UTC (permalink / raw)
To: Guy; +Cc: linux-raid, 'Eric Sandall'
[-- Attachment #1.1: Type: text/plain, Size: 1895 bytes --]
Quoting Guy <bugzilla@watkins-home.com>:
> Run "ipcs" to see if you have shared memory usage that seems wrong, or
> grows.
>
> # ipcs -m
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x00000000 65536 root 600 33554432 11 dest
0x0052e2c1 98305 postgres 600 10330112 11
> I have none, but I don't have a database. Databases I have used on other
> systems tend to use a lot of shared memory.
>
> Also, if you were really out of memory, you would have active swapping.
This has happened before. The computer was thrashing so badly, all I could do
was hit ctrl-alt-del and hope it eventually shut down cleanly. Which it did
about an hour later.
>
> It is normal for the buffer cache to use most "unused" memory, so it would
> seem like you have almost none free, but the buffer cache will give up
> memory when needed. I think you have another problem, not memory related.
> Also, you can stop mdadm from running. The system will still work, just not
> monitor the arrays. If you really think it is mdadm related, kill it. Or
> use "/etc/init.d/mdmonitor stop". At least as a test.
I think mdadm isn't actually running, so it might be the device mapper stuff
itself. ps -eaf | grep mdadm shows nothing
>
> Run top, look at this line:
> Mem: 515296k av, 508128k used, 7168k free, 0k shrd, 128412k buff
From my top:
Mem: 773984k total, 765556k used, 8428k free, 65812k buffers
Swap: 2755136k total, 0k used, 2755136k free, 526632k cached
>
> I think buff (128412k) is the amount of "unused" memory. But not sure. I
> never had a memory issue with Linux, so have not researched this. But I
> have on other Unixes.
>
> Guy
>
----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.
[-- Attachment #1.2: PGP Digital Signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
[-- Attachment #2: PGP Public Key --]
[-- Type: application/pgp-keys, Size: 1793 bytes --]
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: mdadm memory leak?
2005-07-05 15:36 ` Eric Sandall
@ 2005-07-05 21:08 ` Neil Brown
2005-07-05 22:04 ` Eric Sandall
0 siblings, 1 reply; 20+ messages in thread
From: Neil Brown @ 2005-07-05 21:08 UTC (permalink / raw)
To: Eric Sandall; +Cc: David Kowis, linux-raid
On Tuesday July 5, eric@sandall.us wrote:
> On Tue, 5 Jul 2005, Neil Brown wrote:
> > On Monday July 4, dkowis@shlrm.org wrote:
> >> I'm not entirely sure if this is mdadm's fault, but I cannot find anything else that would cause the problem, since mdadm is the only new
> >> thing and I'm pretty sure it's not 2.6.11.12's fault. Anyways, on to my issue:
> >> I'm running samba, apache2, mysql, postgresql, and a few other things. I've got an Athlon-XP 1700+ with 768Mb RAM. Right after startup I've
> >> got about 600Mb of free memory, and as time progresses, and I use samba for things (playing an MP3,) my amount of free memory declines
> >> rather rapidly. It hovers around 8Mb of free ram, with no swap usage. The computer has bogged down bad enough that oom-killer has had to
> >> kill just about everything. ps and top don't show anything eating up all my memory. I'm very impressed with mdadm and I'd like to keep using
> >> it, but i'd also like to have a bit of free memory on my computer. I'm using an XFS file system on a 200Gb mirrored RAID array, two drives,
> >> on seperate IDE channels (seperate cables.)
> >> Thanks for your time,
> >
> > Hmmm.
> > There is an md related memory leak in 2.6.12, but I don't think it is
> > there in 2.6.11.anything.
>
> I also have this problem, but am using 2.6.10-as7 (also reiserfs on
> the raid-1 array). This machine has 1Gb RAM, but has roughly 2-40Mb
> free at any time after running for a day (I can leave the machine up
> as long as I want, but all processes run slow as they now have to do a
> lot of swapping). The kernel was compiled with gcc 3.4.4 and glibc
> 2.3.5 with NPTL (if that matters).
>
> > If 'ps' doesn't show anything, the next place to look is
> > /proc/slabinfo (which 'slabtop' might display for you).
>
> Active / Total Objects (% used) : 12080357 / 12132001 (99.6%)
> Active / Total Slabs (% used) : 176776 / 176776 (100.0%)
> Active / Total Caches (% used) : 66 / 101 (65.3%)
> Active / Total Size (% used) : 668099.80K / 671784.20K (99.5%)
> Minimum / Average / Maximum Object : 0.01K / 0.05K / 128.00K
>
> OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
> 6013634 6013300 99% 0.02K 26609 226 106436K biovec-1
> 6013306 6013297 99% 0.09K 146666 41 586664K bio
These two lines point to the problem - it is a leak of 'bio's.
This patch fixes it.
### Diffstat output
./drivers/md/md.c | 1 +
1 files changed, 1 insertion(+)
diff ./drivers/md/md.c~current~ ./drivers/md/md.c
--- ./drivers/md/md.c~current~ 2005-06-30 11:07:38.000000000 +1000
+++ ./drivers/md/md.c 2005-06-28 13:02:04.000000000 +1000
@@ -338,6 +338,7 @@ static int super_written(struct bio *bio
if (atomic_dec_and_test(&rdev->mddev->pending_writes))
wake_up(&rdev->mddev->sb_wait);
+ bio_put(bio);
return 0;
}
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: mdadm memory leak?
2005-07-05 15:52 ` David Kowis
@ 2005-07-05 21:23 ` Neil Brown
2005-07-05 21:50 ` David Kowis
2005-07-08 22:04 ` David Kowis
0 siblings, 2 replies; 20+ messages in thread
From: Neil Brown @ 2005-07-05 21:23 UTC (permalink / raw)
To: David Kowis; +Cc: linux-raid, Eric Sandall
On Tuesday July 5, dkowis@shlrm.org wrote:
> Quoting Neil Brown <neilb@cse.unsw.edu.au>:
>
> > Hmmm.
> > There is an md related memory leak in 2.6.12, but I don't think it is
> > there in 2.6.11.anything.
> >
> > If 'ps' doesn't show anything, the next place to look is
> > /proc/slabinfo (which 'slabtop' might display for you).
> Slabtop:
> Active / Total Objects (% used) : 217562 / 225483 (96.5%)
> Active / Total Slabs (% used) : 3972 / 3972 (100.0%)
> Active / Total Caches (% used) : 78 / 139 (56.1%)
> Active / Total Size (% used) : 14328.78K / 15891.08K (90.2%)
> Minimum / Average / Maximum Object : 0.01K / 0.07K / 128.00K
>
> OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
> 152098 151909 99% 0.02K 673 226 2692K fasync_cache
> 24867 24846 99% 0.05K 307 81 1228K buffer_head
> 12432 8306 66% 0.27K 888 14 3552K radix_tree_node
> 7308 6876 94% 0.13K 252 29 1008K dentry_cache
> 6303 5885 93% 0.36K 573 11 2292K reiser_inode_cache
So you have about 16 megabytes used by the slab cache, none of the big
users 'md' related.
16M doesn't sound like a big deal, so I suspect this isn't the source
of the leak.
From a separate Email I see:
> # ipcs -m
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x00000000 65536 root 600 33554432 11 dest
0x0052e2c1 98305 postgres 600 10330112 11
that you have 43M is shared-memory, which is more that the slab is
using but still barely 6% of your total memory.
> Mem: 773984k total, 765556k used, 8428k free, 65812k buffers
> Swap: 2755136k total, 0k used, 2755136k free, 526632k cached
The fact that swap isn't being touched at all suggests that you aren't
currently running low on memory.
The fact the free is low doesn't directly indicate a problem. Linux
uses free memory to cache files. It will discard then from the cache
if it needs more memory.
The fact that the OOM killer is hiting obviously is a problem. Maybe
you need to report this on linux-kernel was an OOM problem.
NeilBrown
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: mdadm memory leak?
2005-07-05 4:57 ` Neil Brown
2005-07-05 15:36 ` Eric Sandall
2005-07-05 15:52 ` David Kowis
@ 2005-07-05 21:41 ` David Kowis
2 siblings, 0 replies; 20+ messages in thread
From: David Kowis @ 2005-07-05 21:41 UTC (permalink / raw)
To: Neil Brown; +Cc: linux-raid, Eric Sandall
[-- Attachment #1: Type: text/plain, Size: 4400 bytes --]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Quoting Neil Brown <neilb@cse.unsw.edu.au>:
> Hmmm.
> There is an md related memory leak in 2.6.12, but I don't think it is
> there in 2.6.11.anything.
>
> If 'ps' doesn't show anything, the next place to look is
> /proc/slabinfo (which 'slabtop' might display for you).
Slabtop:
Active / Total Objects (% used) : 217562 / 225483 (96.5%)
Active / Total Slabs (% used) : 3972 / 3972 (100.0%)
Active / Total Caches (% used) : 78 / 139 (56.1%)
Active / Total Size (% used) : 14328.78K / 15891.08K (90.2%)
Minimum / Average / Maximum Object : 0.01K / 0.07K / 128.00K
OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
152098 151909 99% 0.02K 673 226 2692K fasync_cache
24867 24846 99% 0.05K 307 81 1228K buffer_head
12432 8306 66% 0.27K 888 14 3552K radix_tree_node
7308 6876 94% 0.13K 252 29 1008K dentry_cache
6303 5885 93% 0.36K 573 11 2292K reiser_inode_cache
3690 3684 99% 0.09K 82 45 328K vm_area_struct
2675 2645 98% 0.04K 25 107 100K sysfs_dir_cache
2405 2360 98% 0.29K 185 13 740K inode_cache
2142 2082 97% 0.03K 18 119 72K size-32
2074 1503 72% 0.06K 34 61 136K size-64
1891 1842 97% 0.12K 61 31 244K size-128
814 523 64% 0.01K 2 407 8K anon_vma
750 749 99% 0.16K 30 25 120K filp
452 300 66% 0.02K 2 226 8K biovec-1
305 281 92% 0.06K 5 61 20K bio
305 256 83% 0.06K 5 61 20K biovec-4
260 256 98% 0.19K 13 20 52K biovec-16
260 256 98% 0.75K 52 5 208K biovec-64
260 256 98% 1.50K 52 5 416K biovec-128
256 256 100% 3.00K 128 2 1024K biovec-(256)
247 242 97% 0.30K 19 13 76K proc_inode_cache
226 9 3% 0.02K 1 226 4K ip_fib_alias
226 16 7% 0.02K 1 226 4K tcp_bind_bucket
200 71 35% 0.19K 10 20 40K skbuff_head_cache
184 176 95% 0.50K 23 8 92K size-512
156 53 33% 0.02K 1 156 4K blkdev_ioc
155 155 100% 0.12K 5 31 20K kmem_cache
124 119 95% 1.00K 31 4 124K size-1024
120 120 100% 0.25K 8 15 32K size-256
119 59 49% 0.03K 1 119 4K fs_cache
119 9 7% 0.03K 1 119 4K ip_fib_hash
119 8 6% 0.03K 1 119 4K fib6_nodes
108 108 100% 4.00K 108 1 432K size-4096
99 89 89% 1.25K 33 3 132K task_struct
92 92 100% 2.00K 46 2 184K size-2048
90 80 88% 0.25K 6 15 24K signal_cache
89 89 100% 8.00K 89 1 712K size-8192
88 80 90% 0.34K 8 11 32K sock_inode_cache
87 77 88% 1.28K 29 3 116K sighand_cache
87 86 98% 0.13K 3 29 12K idr_layer_cache
80 63 78% 0.19K 4 20 16K size-192
72 59 81% 0.41K 8 9 32K files_cache
70 59 84% 0.56K 10 7 40K mm_struct
65 24 36% 0.06K 1 65 4K as_arq
62 32 51% 0.12K 2 31 8K sgpool-8
61 7 11% 0.06K 1 61 4K uid_cache
61 1 1% 0.06K 1 61 4K inet_peer_cache
59 59 100% 4.00K 59 1 236K pgd
>
> What does 'cat /proc/slabinfo' show?
I've attached my /proc/slabinfo.
Thanks :)
David
- --
One login to rule them all, one login to find them. One login to bring
them all,
and in the web bind them.
- ----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (MingW32)
iD8DBQFCyv5ytgErhgxHMHsRAr8QAKCHHKplEpHR+fF8bDoit7cNpWGkrgCaA83d
a2iF8PJ5QGfMZ+BRLLMEUlw=
=9Bqi
-----END PGP SIGNATURE-----
[-- Attachment #2: slabinfo.txt --]
[-- Type: text/plain, Size: 15083 bytes --]
slabinfo - version: 2.1
# name <active_objs> <num_objs> <objsize> <objperslab> <pagesperslab> : tunables <limit> <batchcount> <sharedfactor> : slabdata <active_slabs> <num_slabs> <sharedavail>
rpc_buffers 8 8 2048 2 1 : tunables 24 12 0 : slabdata 4 4 0
rpc_tasks 8 20 192 20 1 : tunables 120 60 0 : slabdata 1 1 0
rpc_inode_cache 0 0 416 9 1 : tunables 54 27 0 : slabdata 0 0 0
fib6_nodes 8 119 32 119 1 : tunables 120 60 0 : slabdata 1 1 0
ip6_dst_cache 7 18 224 18 1 : tunables 120 60 0 : slabdata 1 1 0
ndisc_cache 2 25 160 25 1 : tunables 120 60 0 : slabdata 1 1 0
rawv6_sock 3 6 640 6 1 : tunables 54 27 0 : slabdata 1 1 0
udpv6_sock 1 6 608 6 1 : tunables 54 27 0 : slabdata 1 1 0
tcpv6_sock 7 7 1120 7 2 : tunables 24 12 0 : slabdata 1 1 0
unix_sock 33 40 384 10 1 : tunables 54 27 0 : slabdata 4 4 0
ip_conntrack_expect 0 0 84 47 1 : tunables 120 60 0 : slabdata 0 0 0
ip_conntrack 13 19 212 19 1 : tunables 120 60 0 : slabdata 1 1 0
tcp_tw_bucket 2 31 128 31 1 : tunables 120 60 0 : slabdata 1 1 0
tcp_bind_bucket 17 226 16 226 1 : tunables 120 60 0 : slabdata 1 1 0
tcp_open_request 0 0 96 41 1 : tunables 120 60 0 : slabdata 0 0 0
inet_peer_cache 1 61 64 61 1 : tunables 120 60 0 : slabdata 1 1 0
ip_fib_alias 9 226 16 226 1 : tunables 120 60 0 : slabdata 1 1 0
ip_fib_hash 9 119 32 119 1 : tunables 120 60 0 : slabdata 1 1 0
ip_dst_cache 11 30 256 15 1 : tunables 120 60 0 : slabdata 2 2 0
arp_cache 3 31 128 31 1 : tunables 120 60 0 : slabdata 1 1 0
raw_sock 2 8 480 8 1 : tunables 54 27 0 : slabdata 1 1 0
udp_sock 16 16 480 8 1 : tunables 54 27 0 : slabdata 2 2 0
tcp_sock 12 12 1024 4 1 : tunables 54 27 0 : slabdata 3 3 0
flow_cache 0 0 96 41 1 : tunables 120 60 0 : slabdata 0 0 0
uhci_urb_priv 0 0 44 88 1 : tunables 120 60 0 : slabdata 0 0 0
cfq_ioc_pool 0 0 24 156 1 : tunables 120 60 0 : slabdata 0 0 0
cfq_pool 0 0 104 38 1 : tunables 120 60 0 : slabdata 0 0 0
crq_pool 0 0 52 75 1 : tunables 120 60 0 : slabdata 0 0 0
deadline_drq 0 0 48 81 1 : tunables 120 60 0 : slabdata 0 0 0
as_arq 24 130 60 65 1 : tunables 120 60 0 : slabdata 2 2 0
mqueue_inode_cache 1 8 480 8 1 : tunables 54 27 0 : slabdata 1 1 0
xfs_chashlist 0 0 20 185 1 : tunables 120 60 0 : slabdata 0 0 0
xfs_ili 0 0 140 28 1 : tunables 120 60 0 : slabdata 0 0 0
xfs_ifork 0 0 56 70 1 : tunables 120 60 0 : slabdata 0 0 0
xfs_efi_item 0 0 260 15 1 : tunables 54 27 0 : slabdata 0 0 0
xfs_efd_item 0 0 260 15 1 : tunables 54 27 0 : slabdata 0 0 0
xfs_buf_item 0 0 148 27 1 : tunables 120 60 0 : slabdata 0 0 0
xfs_dabuf 0 0 16 226 1 : tunables 120 60 0 : slabdata 0 0 0
xfs_da_state 0 0 336 12 1 : tunables 54 27 0 : slabdata 0 0 0
xfs_trans 0 0 592 13 2 : tunables 54 27 0 : slabdata 0 0 0
xfs_inode 0 0 352 11 1 : tunables 54 27 0 : slabdata 0 0 0
xfs_btree_cur 0 0 132 30 1 : tunables 120 60 0 : slabdata 0 0 0
xfs_bmap_free_item 0 0 12 290 1 : tunables 120 60 0 : slabdata 0 0 0
xfs_buf_t 0 0 224 18 1 : tunables 120 60 0 : slabdata 0 0 0
linvfs_icache 0 0 320 12 1 : tunables 54 27 0 : slabdata 0 0 0
udf_inode_cache 0 0 348 11 1 : tunables 54 27 0 : slabdata 0 0 0
nfs_write_data 36 36 448 9 1 : tunables 54 27 0 : slabdata 4 4 0
nfs_read_data 32 36 448 9 1 : tunables 54 27 0 : slabdata 4 4 0
nfs_inode_cache 0 0 544 7 1 : tunables 54 27 0 : slabdata 0 0 0
nfs_page 0 0 64 61 1 : tunables 120 60 0 : slabdata 0 0 0
isofs_inode_cache 0 0 320 12 1 : tunables 54 27 0 : slabdata 0 0 0
fat_inode_cache 0 0 348 11 1 : tunables 54 27 0 : slabdata 0 0 0
fat_cache 0 0 20 185 1 : tunables 120 60 0 : slabdata 0 0 0
ext2_inode_cache 0 0 400 10 1 : tunables 54 27 0 : slabdata 0 0 0
journal_handle 0 0 20 185 1 : tunables 120 60 0 : slabdata 0 0 0
journal_head 0 0 48 81 1 : tunables 120 60 0 : slabdata 0 0 0
revoke_table 0 0 12 290 1 : tunables 120 60 0 : slabdata 0 0 0
revoke_record 0 0 16 226 1 : tunables 120 60 0 : slabdata 0 0 0
ext3_inode_cache 0 0 472 8 1 : tunables 54 27 0 : slabdata 0 0 0
ext3_xattr 0 0 44 88 1 : tunables 120 60 0 : slabdata 0 0 0
reiser_inode_cache 5617 6303 368 11 1 : tunables 54 27 0 : slabdata 573 573 0
dnotify_cache 0 0 20 185 1 : tunables 120 60 0 : slabdata 0 0 0
eventpoll_pwq 0 0 36 107 1 : tunables 120 60 0 : slabdata 0 0 0
eventpoll_epi 0 0 96 41 1 : tunables 120 60 0 : slabdata 0 0 0
kioctx 0 0 160 25 1 : tunables 120 60 0 : slabdata 0 0 0
kiocb 0 0 128 31 1 : tunables 120 60 0 : slabdata 0 0 0
fasync_cache 151909 152098 16 226 1 : tunables 120 60 0 : slabdata 673 673 0
shmem_inode_cache 7 10 384 10 1 : tunables 54 27 0 : slabdata 1 1 0
posix_timers_cache 0 0 96 41 1 : tunables 120 60 0 : slabdata 0 0 0
uid_cache 7 61 64 61 1 : tunables 120 60 0 : slabdata 1 1 0
sgpool-128 32 32 2048 2 1 : tunables 24 12 0 : slabdata 16 16 0
sgpool-64 32 32 1024 4 1 : tunables 54 27 0 : slabdata 8 8 0
sgpool-32 32 32 512 8 1 : tunables 54 27 0 : slabdata 4 4 0
sgpool-16 32 45 256 15 1 : tunables 120 60 0 : slabdata 3 3 0
sgpool-8 32 62 128 31 1 : tunables 120 60 0 : slabdata 2 2 0
blkdev_ioc 68 156 24 156 1 : tunables 120 60 0 : slabdata 1 1 0
blkdev_queue 14 22 360 11 1 : tunables 54 27 0 : slabdata 2 2 0
blkdev_requests 24 56 140 28 1 : tunables 120 60 0 : slabdata 2 2 0
biovec-(256) 256 256 3072 2 2 : tunables 24 12 0 : slabdata 128 128 0
biovec-128 256 260 1536 5 2 : tunables 24 12 0 : slabdata 52 52 0
biovec-64 256 260 768 5 1 : tunables 54 27 0 : slabdata 52 52 0
biovec-16 256 260 192 20 1 : tunables 120 60 0 : slabdata 13 13 0
biovec-4 256 305 64 61 1 : tunables 120 60 0 : slabdata 5 5 0
biovec-1 258 452 16 226 1 : tunables 120 60 0 : slabdata 2 2 0
bio 261 305 64 61 1 : tunables 120 60 0 : slabdata 5 5 0
file_lock_cache 12 45 88 45 1 : tunables 120 60 0 : slabdata 1 1 0
sock_inode_cache 81 88 352 11 1 : tunables 54 27 0 : slabdata 8 8 0
skbuff_head_cache 72 200 192 20 1 : tunables 120 60 0 : slabdata 10 10 0
sock 5 12 320 12 1 : tunables 54 27 0 : slabdata 1 1 0
proc_inode_cache 247 247 308 13 1 : tunables 54 27 0 : slabdata 19 19 0
sigqueue 27 27 148 27 1 : tunables 120 60 0 : slabdata 1 1 0
radix_tree_node 8272 12432 276 14 1 : tunables 54 27 0 : slabdata 888 888 0
bdev_cache 10 18 416 9 1 : tunables 54 27 0 : slabdata 2 2 0
sysfs_dir_cache 2645 2675 36 107 1 : tunables 120 60 0 : slabdata 25 25 0
mnt_cache 20 41 96 41 1 : tunables 120 60 0 : slabdata 1 1 0
inode_cache 2360 2405 292 13 1 : tunables 54 27 0 : slabdata 185 185 0
dentry_cache 6443 7308 136 29 1 : tunables 120 60 0 : slabdata 252 252 0
filp 750 750 160 25 1 : tunables 120 60 0 : slabdata 30 30 0
names_cache 1 1 4096 1 1 : tunables 24 12 0 : slabdata 1 1 0
idr_layer_cache 86 87 136 29 1 : tunables 120 60 0 : slabdata 3 3 0
buffer_head 24543 24543 48 81 1 : tunables 120 60 0 : slabdata 303 303 0
mm_struct 70 70 576 7 1 : tunables 54 27 0 : slabdata 10 10 0
vm_area_struct 3645 3645 88 45 1 : tunables 120 60 0 : slabdata 81 81 0
fs_cache 74 119 32 119 1 : tunables 120 60 0 : slabdata 1 1 0
files_cache 72 72 416 9 1 : tunables 54 27 0 : slabdata 8 8 0
signal_cache 90 90 256 15 1 : tunables 120 60 0 : slabdata 6 6 0
sighand_cache 87 87 1312 3 1 : tunables 24 12 0 : slabdata 29 29 0
task_struct 99 99 1280 3 1 : tunables 24 12 0 : slabdata 33 33 0
anon_vma 573 814 8 407 1 : tunables 120 60 0 : slabdata 2 2 0
pgd 60 60 4096 1 1 : tunables 24 12 0 : slabdata 60 60 0
size-131072(DMA) 0 0 131072 1 32 : tunables 8 4 0 : slabdata 0 0 0
size-131072 0 0 131072 1 32 : tunables 8 4 0 : slabdata 0 0 0
size-65536(DMA) 0 0 65536 1 16 : tunables 8 4 0 : slabdata 0 0 0
size-65536 0 0 65536 1 16 : tunables 8 4 0 : slabdata 0 0 0
size-32768(DMA) 0 0 32768 1 8 : tunables 8 4 0 : slabdata 0 0 0
size-32768 0 0 32768 1 8 : tunables 8 4 0 : slabdata 0 0 0
size-16384(DMA) 0 0 16384 1 4 : tunables 8 4 0 : slabdata 0 0 0
size-16384 0 0 16384 1 4 : tunables 8 4 0 : slabdata 0 0 0
size-8192(DMA) 0 0 8192 1 2 : tunables 8 4 0 : slabdata 0 0 0
size-8192 89 89 8192 1 2 : tunables 8 4 0 : slabdata 89 89 0
size-4096(DMA) 0 0 4096 1 1 : tunables 24 12 0 : slabdata 0 0 0
size-4096 109 109 4096 1 1 : tunables 24 12 0 : slabdata 109 109 0
size-2048(DMA) 0 0 2048 2 1 : tunables 24 12 0 : slabdata 0 0 0
size-2048 95 96 2048 2 1 : tunables 24 12 0 : slabdata 48 48 0
size-1024(DMA) 0 0 1024 4 1 : tunables 54 27 0 : slabdata 0 0 0
size-1024 118 124 1024 4 1 : tunables 54 27 0 : slabdata 31 31 0
size-512(DMA) 0 0 512 8 1 : tunables 54 27 0 : slabdata 0 0 0
size-512 184 184 512 8 1 : tunables 54 27 0 : slabdata 23 23 0
size-256(DMA) 0 0 256 15 1 : tunables 120 60 0 : slabdata 0 0 0
size-256 120 120 256 15 1 : tunables 120 60 0 : slabdata 8 8 0
size-192(DMA) 0 0 192 20 1 : tunables 120 60 0 : slabdata 0 0 0
size-192 79 80 192 20 1 : tunables 120 60 0 : slabdata 4 4 0
size-128(DMA) 0 0 128 31 1 : tunables 120 60 0 : slabdata 0 0 0
size-128 1812 1891 128 31 1 : tunables 120 60 0 : slabdata 61 61 0
size-64(DMA) 0 0 64 61 1 : tunables 120 60 0 : slabdata 0 0 0
size-64 1501 2074 64 61 1 : tunables 120 60 0 : slabdata 34 34 0
size-32(DMA) 0 0 32 119 1 : tunables 120 60 0 : slabdata 0 0 0
size-32 2082 2142 32 119 1 : tunables 120 60 0 : slabdata 18 18 0
kmem_cache 155 155 128 31 1 : tunables 120 60 0 : slabdata 5 5 0
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: mdadm memory leak?
2005-07-05 21:23 ` Neil Brown
@ 2005-07-05 21:50 ` David Kowis
2005-07-08 22:04 ` David Kowis
1 sibling, 0 replies; 20+ messages in thread
From: David Kowis @ 2005-07-05 21:50 UTC (permalink / raw)
To: Neil Brown; +Cc: linux-raid, Eric Sandall
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
I resent this email with the slabinfo attachment on it, because I didn't think it sent at all. I got a message back telling me that
slabinfo.com is not a valid attachment. Apparently if you do not put an extension on the file, it's .com .
perhaps the slabinfo.txt will have more info :) if not, well then thanks for the time :)
Neil Brown wrote:
> On Tuesday July 5, dkowis@shlrm.org wrote:
>
>>Quoting Neil Brown <neilb@cse.unsw.edu.au>:
>>
>>
>>>Hmmm.
>>>There is an md related memory leak in 2.6.12, but I don't think it is
>>>there in 2.6.11.anything.
>>>
>>>If 'ps' doesn't show anything, the next place to look is
>>>/proc/slabinfo (which 'slabtop' might display for you).
>>
>>Slabtop:
>>Active / Total Objects (% used) : 217562 / 225483 (96.5%)
>>Active / Total Slabs (% used) : 3972 / 3972 (100.0%)
>>Active / Total Caches (% used) : 78 / 139 (56.1%)
>>Active / Total Size (% used) : 14328.78K / 15891.08K (90.2%)
>>Minimum / Average / Maximum Object : 0.01K / 0.07K / 128.00K
>>
>> OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
>>152098 151909 99% 0.02K 673 226 2692K fasync_cache
>>24867 24846 99% 0.05K 307 81 1228K buffer_head
>>12432 8306 66% 0.27K 888 14 3552K radix_tree_node
>> 7308 6876 94% 0.13K 252 29 1008K dentry_cache
>> 6303 5885 93% 0.36K 573 11 2292K reiser_inode_cache
>
>
> So you have about 16 megabytes used by the slab cache, none of the big
> users 'md' related.
> 16M doesn't sound like a big deal, so I suspect this isn't the source
> of the leak.
> From a separate Email I see:
>
>># ipcs -m
>
> ------ Shared Memory Segments --------
> key shmid owner perms bytes nattch status
> 0x00000000 65536 root 600 33554432 11 dest
> 0x0052e2c1 98305 postgres 600 10330112 11
>
> that you have 43M is shared-memory, which is more that the slab is
> using but still barely 6% of your total memory.
>
>
>>Mem: 773984k total, 765556k used, 8428k free, 65812k buffers
>>Swap: 2755136k total, 0k used, 2755136k free, 526632k cached
>
>
> The fact that swap isn't being touched at all suggests that you aren't
> currently running low on memory.
> The fact the free is low doesn't directly indicate a problem. Linux
> uses free memory to cache files. It will discard then from the cache
> if it needs more memory.
> The fact that the OOM killer is hiting obviously is a problem. Maybe
> you need to report this on linux-kernel was an OOM problem.
>
> NeilBrown
>
>
>
> !DSPAM:42caff8778281883178545!
>
- --
David Kowis
ISO Team Lead - www.sourcemage.org
SourceMage GNU/Linux
One login to rule them all, one login to find them. One login to bring them all, and in the web bind them.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (MingW32)
iD8DBQFCywCotgErhgxHMHsRAggzAJ9/K3WJ1gNrx3aLOdB9JDeDQDtnwACeKZan
qTjRYIgqvOEsvWMSfMD9vIQ=
=a7r7
-----END PGP SIGNATURE-----
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: mdadm memory leak?
2005-07-05 21:08 ` Neil Brown
@ 2005-07-05 22:04 ` Eric Sandall
2005-07-06 1:30 ` Neil Brown
0 siblings, 1 reply; 20+ messages in thread
From: Eric Sandall @ 2005-07-05 22:04 UTC (permalink / raw)
To: Neil Brown; +Cc: Eric Sandall, David Kowis, linux-raid
On Wed, 6 Jul 2005, Neil Brown wrote:
> On Tuesday July 5, eric@sandall.us wrote:
<snip>
>> Active / Total Objects (% used) : 12080357 / 12132001 (99.6%)
>> Active / Total Slabs (% used) : 176776 / 176776 (100.0%)
>> Active / Total Caches (% used) : 66 / 101 (65.3%)
>> Active / Total Size (% used) : 668099.80K / 671784.20K (99.5%)
>> Minimum / Average / Maximum Object : 0.01K / 0.05K / 128.00K
>>
>> OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
>> 6013634 6013300 99% 0.02K 26609 226 106436K biovec-1
>> 6013306 6013297 99% 0.09K 146666 41 586664K bio
>
> These two lines point to the problem - it is a leak of 'bio's.
>
> This patch fixes it.
>
> ### Diffstat output
> ./drivers/md/md.c | 1 +
> 1 files changed, 1 insertion(+)
>
> diff ./drivers/md/md.c~current~ ./drivers/md/md.c
> --- ./drivers/md/md.c~current~ 2005-06-30 11:07:38.000000000 +1000
> +++ ./drivers/md/md.c 2005-06-28 13:02:04.000000000 +1000
> @@ -338,6 +338,7 @@ static int super_written(struct bio *bio
>
> if (atomic_dec_and_test(&rdev->mddev->pending_writes))
> wake_up(&rdev->mddev->sb_wait);
> + bio_put(bio);
> return 0;
> }
Which kernel is that against? It doesn't apply against my 2.6.10-as7
nor David Kowis' 2.6.12.1 and 2.6.11.12 kernel source trees
(I don't even have the super_written function the patch is referring
to)?
-sandalle
--
Eric Sandall | Source Mage GNU/Linux Developer
eric@sandall.us | http://www.sourcemage.org/
http://eric.sandall.us/ | SysAdmin @ Inst. Shock Physics @ WSU
http://counter.li.org/ #196285 | http://www.shock.wsu.edu/
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: mdadm memory leak?
2005-07-05 22:04 ` Eric Sandall
@ 2005-07-06 1:30 ` Neil Brown
2005-07-09 20:11 ` Eric Sandall
2005-07-17 4:52 ` Eric Sandall
0 siblings, 2 replies; 20+ messages in thread
From: Neil Brown @ 2005-07-06 1:30 UTC (permalink / raw)
To: Eric Sandall; +Cc: David Kowis, linux-raid
On Tuesday July 5, eric@sandall.us wrote:
> On Wed, 6 Jul 2005, Neil Brown wrote:
> > On Tuesday July 5, eric@sandall.us wrote:
> <snip>
> >> Active / Total Objects (% used) : 12080357 / 12132001 (99.6%)
> >> Active / Total Slabs (% used) : 176776 / 176776 (100.0%)
> >> Active / Total Caches (% used) : 66 / 101 (65.3%)
> >> Active / Total Size (% used) : 668099.80K / 671784.20K (99.5%)
> >> Minimum / Average / Maximum Object : 0.01K / 0.05K / 128.00K
> >>
> >> OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
> >> 6013634 6013300 99% 0.02K 26609 226 106436K biovec-1
> >> 6013306 6013297 99% 0.09K 146666 41 586664K bio
> >
> > These two lines point to the problem - it is a leak of 'bio's.
> >
> > This patch fixes it.
> >
> > ### Diffstat output
> > ./drivers/md/md.c | 1 +
> > 1 files changed, 1 insertion(+)
> >
> > diff ./drivers/md/md.c~current~ ./drivers/md/md.c
> > --- ./drivers/md/md.c~current~ 2005-06-30 11:07:38.000000000 +1000
> > +++ ./drivers/md/md.c 2005-06-28 13:02:04.000000000 +1000
> > @@ -338,6 +338,7 @@ static int super_written(struct bio *bio
> >
> > if (atomic_dec_and_test(&rdev->mddev->pending_writes))
> > wake_up(&rdev->mddev->sb_wait);
> > + bio_put(bio);
> > return 0;
> > }
>
> Which kernel is that against? It doesn't apply against my 2.6.10-as7
> nor David Kowis' 2.6.12.1 and 2.6.11.12 kernel source trees
> (I don't even have the super_written function the patch is referring
> to)?
It was against ... 2.6.12-mm1 or similar I think. Looks like the bug
didn't get to main-line until just before the fix, which it good.
2.6.10-as7 seems to have a different though related bug.
It contains the patch:
http://www.acm.cs.rpi.edu/~dilinger/patches/2.6.10/as7/linux-2.6.10-as7/051-md_sync_page_io_max_vecs.patch
which contains:
diff -Nru a/drivers/md/md.c b/drivers/md/md.c
--- a/drivers/md/md.c 2005-01-22 00:16:02 -08:00
+++ b/drivers/md/md.c 2005-01-22 00:16:02 -08:00
@@ -332,29 +332,26 @@
static int sync_page_io(struct block_device *bdev, sector_t sector, int size,
struct page *page, int rw)
{
- struct bio bio;
- struct bio_vec vec;
+ struct bio *bio = bio_alloc(GFP_KERNEL, 1);
struct completion event;
+ int ret;
+
+ bio_get(bio);
....
+ bio_put(bio);
+ return ret;
}
bio_alloc sets the refcount to 1.
bio_get increments it to 2.
bio_put sets it back to 1. But it never reaches zero.
You want to get rid of that bio_get near the top of sync_page_io.
NeilBrown
^ permalink raw reply [flat|nested] 20+ messages in thread
* RE: mdadm memory leak?
@ 2005-07-06 16:09 Bailey, Scott
0 siblings, 0 replies; 20+ messages in thread
From: Bailey, Scott @ 2005-07-06 16:09 UTC (permalink / raw)
To: Neil Brown, Eric Sandall; +Cc: David Kowis, linux-raid
>diff -Nru a/drivers/md/md.c b/drivers/md/md.c
>--- a/drivers/md/md.c 2005-01-22 00:16:02 -08:00
>+++ b/drivers/md/md.c 2005-01-22 00:16:02 -08:00
>@@ -332,29 +332,26 @@
> static int sync_page_io(struct block_device *bdev, sector_t sector,
int size,
> struct page *page, int rw)
> {
>- struct bio bio;
>- struct bio_vec vec;
>+ struct bio *bio = bio_alloc(GFP_KERNEL, 1);
> struct completion event;
>+ int ret;
>+
>+ bio_get(bio);
>....
>+ bio_put(bio);
>+ return ret;
> }
>
>bio_alloc sets the refcount to 1.
>bio_get increments it to 2.
>bio_put sets it back to 1. But it never reaches zero.
>
>You want to get rid of that bio_get near the top of sync_page_io.
I had been wondering why my Alphaserver running Debian kernel 2.6.10-5
had been OOMing every week or so. :-) When I started reading this
thread, nearly a week since the last reboot, I found I had more than 3GB
of memory tied up in bio and biovec-1 slab. (!)
With the offending "bio_get(bio);" removed as described above, the
system is looking much better. With unpatched kernel, I rebooted system
and started synchronization of 9GB RAID-1 drive - at completion, the bio
slab had grown to something like 15,000+ entries. With patched kernel, I
rebooted and am doing another RAID-1 rebuild now - with that about 50%
complete, there are only 992 bio entries. (And only half of those are in
use - this never went below 99% before.)
Many thanks,
Scott Bailey
scott.bailey@eds.com
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: mdadm memory leak?
2005-07-05 21:23 ` Neil Brown
2005-07-05 21:50 ` David Kowis
@ 2005-07-08 22:04 ` David Kowis
2005-07-08 23:15 ` Tyler
1 sibling, 1 reply; 20+ messages in thread
From: David Kowis @ 2005-07-08 22:04 UTC (permalink / raw)
To: Neil Brown; +Cc: linux-raid, Eric Sandall
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Neil Brown wrote:
>> OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
>>152098 151909 99% 0.02K 673 226 2692K fasync_cache
>>24867 24846 99% 0.05K 307 81 1228K buffer_head
>>12432 8306 66% 0.27K 888 14 3552K radix_tree_node
>> 7308 6876 94% 0.13K 252 29 1008K dentry_cache
>> 6303 5885 93% 0.36K 573 11 2292K reiser_inode_cache
>
>
> So you have about 16 megabytes used by the slab cache, none of the big
> users 'md' related.
> 16M doesn't sound like a big deal, so I suspect this isn't the source
> of the leak.
> From a separate Email I see:
>
>>Mem: 773984k total, 765556k used, 8428k free, 65812k buffers
>>Swap: 2755136k total, 0k used, 2755136k free, 526632k cached
>
> The fact that swap isn't being touched at all suggests that you aren't
> currently running low on memory.
> The fact the free is low doesn't directly indicate a problem. Linux
> uses free memory to cache files. It will discard then from the cache
> if it needs more memory.
> The fact that the OOM killer is hiting obviously is a problem. Maybe
> you need to report this on linux-kernel was an OOM problem.
>
I've let my computer run for a while longer, and it's eaten more memory than before. I'll paste the relevant parts. I'm not sure this is a
kernel thing, since I didn't have this problem before setting up a mirrored RAID array using mdadm.
from slabtop:
Active / Total Objects (% used) : 80755 / 86856 (93.0%)
Active / Total Slabs (% used) : 2974 / 2975 (100.0%)
Active / Total Caches (% used) : 76 / 140 (54.3%)
Active / Total Size (% used) : 11445.72K / 12330.93K (92.8%)
Minimum / Average / Maximum Object : 0.01K / 0.14K / 128.00K
OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
44469 44441 99% 0.05K 549 81 2196K buffer_head
8946 8192 91% 0.27K 639 14 2556K radix_tree_node
6960 4720 67% 0.13K 240 29 960K dentry_cache
3510 3510 100% 0.09K 78 45 312K vm_area_struct
3179 2522 79% 0.36K 289 11 1156K reiser_inode_cache
3050 2477 81% 0.06K 50 61 200K size-64
2782 2713 97% 0.04K 26 107 104K sysfs_dir_cache
2405 2378 98% 0.29K 185 13 740K inode_cache
2142 2142 100% 0.03K 18 119 72K size-32
1860 1817 97% 0.12K 60 31 240K size-128
875 875 100% 0.16K 35 25 140K filp
from free:
total used free shared buffers cached
Mem: 773900 765416 8484 0 75680 450004
- -/+ buffers/cache: 239732 534168
Swap: 2755136 118568 2636568
Something's digesting memory and not giving it back.
I'm now running 2.6.12.1 instead of 2.6.11.12 in the hopes that something was fixed, but no.
Thanks,
- --
David Kowis
ISO Team Lead - www.sourcemage.org
SourceMage GNU/Linux
One login to rule them all, one login to find them. One login to bring them all, and in the web bind them.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (MingW32)
iD8DBQFCzvhftgErhgxHMHsRAthSAJ0bSkP2xuhIeXYcuXx0P46ENdom2ACgmhB1
J0A5DfZu5vxwlKT2Dar55J8=
=pmX1
-----END PGP SIGNATURE-----
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: mdadm memory leak?
2005-07-08 22:04 ` David Kowis
@ 2005-07-08 23:15 ` Tyler
2005-07-09 4:20 ` David Kowis
0 siblings, 1 reply; 20+ messages in thread
From: Tyler @ 2005-07-08 23:15 UTC (permalink / raw)
To: David Kowis; +Cc: linux-raid
David Kowis wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>
>
>Neil Brown wrote:
>
>
>>> OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
>>>152098 151909 99% 0.02K 673 226 2692K fasync_cache
>>>24867 24846 99% 0.05K 307 81 1228K buffer_head
>>>12432 8306 66% 0.27K 888 14 3552K radix_tree_node
>>> 7308 6876 94% 0.13K 252 29 1008K dentry_cache
>>> 6303 5885 93% 0.36K 573 11 2292K reiser_inode_cache
>>>
>>>
>>So you have about 16 megabytes used by the slab cache, none of the big
>>users 'md' related.
>>16M doesn't sound like a big deal, so I suspect this isn't the source
>>of the leak.
>> From a separate Email I see:
>>
>>
>>
>>>Mem: 773984k total, 765556k used, 8428k free, 65812k buffers
>>>Swap: 2755136k total, 0k used, 2755136k free, 526632k cached
>>>
>>>
>>The fact that swap isn't being touched at all suggests that you aren't
>>currently running low on memory.
>>The fact the free is low doesn't directly indicate a problem. Linux
>>uses free memory to cache files. It will discard then from the cache
>>if it needs more memory.
>>The fact that the OOM killer is hiting obviously is a problem. Maybe
>>you need to report this on linux-kernel was an OOM problem.
>>
>>
>>
>I've let my computer run for a while longer, and it's eaten more memory than before. I'll paste the relevant parts. I'm not sure this is a
>kernel thing, since I didn't have this problem before setting up a mirrored RAID array using mdadm.
>
>
>
David, unless you have mdadm running in monitor mode, its not "actively"
in memory.. the MD driver is in the kernel, and in memory. Mdadm is
simply a tool, just as fsck.ext2 or such.. its not active until you run
it, and leaves memory when it finishes its one task. (Someone correct
me if I'm wrong please....).
Tyler.
>from slabtop:
> Active / Total Objects (% used) : 80755 / 86856 (93.0%)
> Active / Total Slabs (% used) : 2974 / 2975 (100.0%)
> Active / Total Caches (% used) : 76 / 140 (54.3%)
> Active / Total Size (% used) : 11445.72K / 12330.93K (92.8%)
> Minimum / Average / Maximum Object : 0.01K / 0.14K / 128.00K
>
> OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
> 44469 44441 99% 0.05K 549 81 2196K buffer_head
> 8946 8192 91% 0.27K 639 14 2556K radix_tree_node
> 6960 4720 67% 0.13K 240 29 960K dentry_cache
> 3510 3510 100% 0.09K 78 45 312K vm_area_struct
> 3179 2522 79% 0.36K 289 11 1156K reiser_inode_cache
> 3050 2477 81% 0.06K 50 61 200K size-64
> 2782 2713 97% 0.04K 26 107 104K sysfs_dir_cache
> 2405 2378 98% 0.29K 185 13 740K inode_cache
> 2142 2142 100% 0.03K 18 119 72K size-32
> 1860 1817 97% 0.12K 60 31 240K size-128
> 875 875 100% 0.16K 35 25 140K filp
>
>from free:
> total used free shared buffers cached
>Mem: 773900 765416 8484 0 75680 450004
>- -/+ buffers/cache: 239732 534168
>Swap: 2755136 118568 2636568
>
>
>Something's digesting memory and not giving it back.
>I'm now running 2.6.12.1 instead of 2.6.11.12 in the hopes that something was fixed, but no.
>
>Thanks,
>- --
>David Kowis
>
>ISO Team Lead - www.sourcemage.org
>SourceMage GNU/Linux
>
>One login to rule them all, one login to find them. One login to bring them all, and in the web bind them.
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v1.4.1 (MingW32)
>
>iD8DBQFCzvhftgErhgxHMHsRAthSAJ0bSkP2xuhIeXYcuXx0P46ENdom2ACgmhB1
>J0A5DfZu5vxwlKT2Dar55J8=
>=pmX1
>-----END PGP SIGNATURE-----
>-
>To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
>
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: mdadm memory leak?
2005-07-08 23:15 ` Tyler
@ 2005-07-09 4:20 ` David Kowis
0 siblings, 0 replies; 20+ messages in thread
From: David Kowis @ 2005-07-09 4:20 UTC (permalink / raw)
To: Tyler; +Cc: linux-raid
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
>> David, unless you have mdadm running in monitor mode, its not "actively"
>> in memory.. the MD driver is in the kernel, and in memory. Mdadm is
>> simply a tool, just as fsck.ext2 or such.. its not active until you run
>> it, and leaves memory when it finishes its one task. (Someone correct
>> me if I'm wrong please....).
>
>> Tyler.
Well, perhaps you can point me in the right direction. I've got lots of slab usage by
47061 47030 99% 0.05K 581 81 2324K buffer_head
and I'm just trying to find out what's eating the ram on my computer. It's got to be somehow connected to the md device that I'm using.
because this doesn't exist on any of my other computers that have a 2.6 kernel on them. Only the one with the mdadm created RAID array. I'm
not saying that it's mdadm, but perhaps some aspect of the md "system".
thanks for your time,
- --
David Kowis
ISO Team Lead - www.sourcemage.org
SourceMage GNU/Linux
One login to rule them all, one login to find them. One login to bring them all, and in the web bind them.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (MingW32)
iD8DBQFCz1CAtgErhgxHMHsRAmmXAJ4r0GXFKlTIKm++VW4aT+jklCvr2gCdEcXg
1lkWtJZ8AXE+M3kKTadiNdM=
=HVOK
-----END PGP SIGNATURE-----
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: mdadm memory leak?
2005-07-06 1:30 ` Neil Brown
@ 2005-07-09 20:11 ` Eric Sandall
2005-07-17 4:52 ` Eric Sandall
1 sibling, 0 replies; 20+ messages in thread
From: Eric Sandall @ 2005-07-09 20:11 UTC (permalink / raw)
To: Neil Brown; +Cc: Eric Sandall, David Kowis, linux-raid
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Wed, 6 Jul 2005, Neil Brown wrote:
> On Tuesday July 5, eric@sandall.us wrote:
>> Which kernel is that against? It doesn't apply against my 2.6.10-as7
>> nor David Kowis' 2.6.12.1 and 2.6.11.12 kernel source trees
>> (I don't even have the super_written function the patch is referring
>> to)?
>
> It was against ... 2.6.12-mm1 or similar I think. Looks like the bug
> didn't get to main-line until just before the fix, which it good.
>
> 2.6.10-as7 seems to have a different though related bug.
> It contains the patch:
> http://www.acm.cs.rpi.edu/~dilinger/patches/2.6.10/as7/linux-2.6.10-as7/051-md_sync_page_io_max_vecs.patch
>
> which contains:
> diff -Nru a/drivers/md/md.c b/drivers/md/md.c
> --- a/drivers/md/md.c 2005-01-22 00:16:02 -08:00
> +++ b/drivers/md/md.c 2005-01-22 00:16:02 -08:00
> @@ -332,29 +332,26 @@
> static int sync_page_io(struct block_device *bdev, sector_t sector, int size,
> struct page *page, int rw)
> {
> - struct bio bio;
> - struct bio_vec vec;
> + struct bio *bio = bio_alloc(GFP_KERNEL, 1);
> struct completion event;
> + int ret;
> +
> + bio_get(bio);
> ....
> + bio_put(bio);
> + return ret;
> }
>
> bio_alloc sets the refcount to 1.
> bio_get increments it to 2.
> bio_put sets it back to 1. But it never reaches zero.
>
> You want to get rid of that bio_get near the top of sync_page_io.
I've patched[0] my kernel with that and recompiled it, but haven't
rebooted into it yet as I have some processes running that I don't
want to kill until they've finished.
Should be good to go tomorrow, then I'll report back, thanks. :)
- -sandalle
[0] Removing only the bio_get(bio);, not applying the entire patch.
- --
Eric Sandall | Source Mage GNU/Linux Developer
eric@sandall.us | http://www.sourcemage.org/
http://eric.sandall.us/ | SysAdmin @ Inst. Shock Physics @ WSU
http://counter.li.org/ #196285 | http://www.shock.wsu.edu/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
iD8DBQFC0C9hHXt9dKjv3WERAqgFAJ9UsWg7gtm2aUv/rLtPnjsBrV6A7QCgww1F
r7s0Th0J5LMLXIkfdyv5G/8=
=tMC0
-----END PGP SIGNATURE-----
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: mdadm memory leak?
2005-07-06 1:30 ` Neil Brown
2005-07-09 20:11 ` Eric Sandall
@ 2005-07-17 4:52 ` Eric Sandall
1 sibling, 0 replies; 20+ messages in thread
From: Eric Sandall @ 2005-07-17 4:52 UTC (permalink / raw)
To: Neil Brown; +Cc: Eric Sandall, David Kowis, linux-raid
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Wed, 6 Jul 2005, Neil Brown wrote:
> It was against ... 2.6.12-mm1 or similar I think. Looks like the bug
> didn't get to main-line until just before the fix, which it good.
>
> 2.6.10-as7 seems to have a different though related bug.
> It contains the patch:
> http://www.acm.cs.rpi.edu/~dilinger/patches/2.6.10/as7/linux-2.6.10-as7/051-md_sync_page_io_max_vecs.patch
>
> which contains:
> diff -Nru a/drivers/md/md.c b/drivers/md/md.c
> --- a/drivers/md/md.c 2005-01-22 00:16:02 -08:00
> +++ b/drivers/md/md.c 2005-01-22 00:16:02 -08:00
> @@ -332,29 +332,26 @@
> static int sync_page_io(struct block_device *bdev, sector_t sector, int size,
> struct page *page, int rw)
> {
> - struct bio bio;
> - struct bio_vec vec;
> + struct bio *bio = bio_alloc(GFP_KERNEL, 1);
> struct completion event;
> + int ret;
> +
> + bio_get(bio);
> ....
> + bio_put(bio);
> + return ret;
> }
>
> bio_alloc sets the refcount to 1.
> bio_get increments it to 2.
> bio_put sets it back to 1. But it never reaches zero.
>
> You want to get rid of that bio_get near the top of sync_page_io.
Applying this fix I still 'seem' to run out of memory with `top`, but
bio and biovec are no longer insane:
$ slabtop
Active / Total Objects (% used) : 77209 / 156871 (49.2%)
Active / Total Slabs (% used) : 5203 / 5211 (99.8%)
Active / Total Caches (% used) : 66 / 101 (65.3%)
Active / Total Size (% used) : 11391.00K / 20616.61K (55.3%)
Minimum / Average / Maximum Object : 0.01K / 0.13K / 128.00K
65400 38946 59% 0.05K 872 75 3488K buffer_head
31476 7412 23% 0.06K 516 61 2064K size-64
14546 9797 67% 0.27K 1039 14 4156K radix_tree_node
13717 4024 29% 0.13K 473 29 1892K dentry_cache
11363 2691 23% 0.35K 1033 11 4132K
reiser_inode_cache
4522 3005 66% 0.03K 38 119 152K size-32
3713 2605 70% 0.08K 79 47 316K vm_area_struct
1953 1953 100% 0.12K 63 31 252K size-128
1221 650 53% 0.01K 3 407 12K anon_vma
925 710 76% 0.16K 37 25 148K filp
784 645 82% 0.28K 56 14 224K inode_cache
546 216 39% 0.29K 42 13 168K
proc_inode_cache
460 280 60% 0.19K 23 20 92K
skbuff_head_cache
452 263 58% 0.02K 2 226 8K biovec-1
410 302 73% 0.09K 10 41 40K bio
350 330 94% 0.37K 35 10 140K
shmem_inode_cache
305 256 83% 0.06K 5 61 20K biovec-4
280 257 91% 0.75K 56 5 224K biovec-64
260 257 98% 0.19K 13 20 52K biovec-16
260 256 98% 1.50K 52 5 416K biovec-128
256 256 100% 3.00K 128 2 1024K biovec-(256)
254 187 73% 2.00K 127 2 508K size-2048
226 24 10% 0.02K 1 226 4K tcp_bind_bucket
...
Thanks for the fix!
(tested on a 2.6.10-as7 kernel)
- -sandalle
- --
Eric Sandall | Source Mage GNU/Linux Developer
eric@sandall.us | http://www.sourcemage.org/
http://eric.sandall.us/ | SysAdmin @ Inst. Shock Physics @ WSU
http://counter.li.org/ #196285 | http://www.shock.wsu.edu/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
iD8DBQFC2eQrHXt9dKjv3WERAiS/AJ9lvx+pUiDcpLe1HR5D669iQERSzACaA3UN
tvg0ENq/c4qx/z8G5d27nWk=
=82kA
-----END PGP SIGNATURE-----
^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2005-07-17 4:52 UTC | newest]
Thread overview: 20+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2005-07-05 4:24 mdadm memory leak? David Kowis
2005-07-05 4:30 ` David Kowis
2005-07-05 4:59 ` Guy
2005-07-05 15:49 ` Eric Sandall
2005-07-05 15:56 ` David Kowis
2005-07-05 4:57 ` Neil Brown
2005-07-05 15:36 ` Eric Sandall
2005-07-05 21:08 ` Neil Brown
2005-07-05 22:04 ` Eric Sandall
2005-07-06 1:30 ` Neil Brown
2005-07-09 20:11 ` Eric Sandall
2005-07-17 4:52 ` Eric Sandall
2005-07-05 15:52 ` David Kowis
2005-07-05 21:23 ` Neil Brown
2005-07-05 21:50 ` David Kowis
2005-07-08 22:04 ` David Kowis
2005-07-08 23:15 ` Tyler
2005-07-09 4:20 ` David Kowis
2005-07-05 21:41 ` David Kowis
-- strict thread matches above, loose matches on Subject: below --
2005-07-06 16:09 Bailey, Scott
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).