public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
* Re: rsync out of memory 2.6.8-rc2
@ 2004-07-19  9:19 Klaus Dittrich
  2004-07-19 11:20 ` dentry cache leak? " bert hubert
  0 siblings, 1 reply; 23+ messages in thread
From: Klaus Dittrich @ 2004-07-19  9:19 UTC (permalink / raw)
  To: linux mailing-list; +Cc: gilbertd, nickpiggin, kladit

Nick your mail got lost here, sorry.

I did rsync -av xeon2:/disc1 . on the target system.
xeon2 is the system that gets out of memory.
/disc1 is 60GB and contains several thousand files.
I think this is a common backup situation.

I found out I could trigger the memory outage using du -s /disc1 too.

-- 
Klaus

^ permalink raw reply	[flat|nested] 23+ messages in thread
* Re: dentry cache leak? Re: rsync out of memory 2.6.8-rc2
@ 2004-07-26 15:06 Klaus Dittrich
  2004-07-26 19:37 ` Andrew Morton
  2004-07-29 21:07 ` Andrew Morton
  0 siblings, 2 replies; 23+ messages in thread
From: Klaus Dittrich @ 2004-07-26 15:06 UTC (permalink / raw)
  To: linux mailing-list; +Cc: kladit

>Can you narrow the onset of the problem down to any particular kernel
>snapshot?

Did it and here is the answer.

kernel-2.6.7 and bk's up to 2.6.7-bk7 survived a du -s,
kernels starting with 2.6.7-bk8 did not.


Compiler gcc-3.4.1 
System SMP 2 Xeon-CPU's
Hardware Tyan 2665 Mobo 
/disc1 is a software raid-1, mirror of two 60GB IDE disks.

To be sure to have clean conditions, I recompiled 2.6.7 and
some subsequent -bk kernels.

The configuration of each kernel is of a copy of .config from 
2.6.7 and innovations during make oldconfig were answered using
the defaults.

Some observations while du -s is running:

With the 'good' kernels there is a freeze of about a second 
when memory usage (xosview) has grown up to about 1.4 GByte.
Not using irqbalance results in very high system load of CPU0.

Gathered data follows below. Please cc me.

--
Klaus

linux-2.6.7 (good)
===========
cat /proc/sys/fs/dentry-state
7015    4538    45      0       0       0
du -s /disc1/
42119916        /disc1/
cat /proc/sys/fs/dentry-state
719959  690331  45      0       0       0

cat /proc/vmstat
nr_dirty 6
nr_writeback 0
nr_unstable 0
nr_page_table_pages 417
nr_mapped 36574
nr_slab 219486
pgpgin 822287
pgpgout 125952
pswpin 0
pswpout 0
pgalloc_high 218904
pgalloc_normal 590088
pgalloc_dma 3534
pgfree 967065
pgactivate 182625
pgdeactivate 156250
pgfault 216065
pgmajfault 2694
pgrefill_high 0
pgrefill_normal 154846
pgrefill_dma 1404
pgsteal_high 0
pgsteal_normal 81532
pgsteal_dma 483
pgscan_kswapd_high 0
pgscan_kswapd_normal 130065
pgscan_kswapd_dma 721
pgscan_direct_high 0
pgscan_direct_normal 179566
pgscan_direct_dma 2425
pginodesteal 12160
slabs_scanned 2817088
kswapd_steal 51850
kswapd_inodesteal 1888
pageoutrun 1083
allocstall 2857
pgrotated 8111

linux-2.6.7-bk6 (good)
===============
du -s /disc1/
42120684        /disc1/
cat /proc/sys/fs/dentry-state
720795  691157  45      0       0       0

cat /proc/vmstat
nr_dirty 11
nr_writeback 0
nr_unstable 0
nr_page_table_pages 459
nr_mapped 42928
nr_slab 219592
pgpgin 848899
pgpgout 135604
pswpin 0
pswpout 0
pgalloc_high 256670
pgalloc_normal 619555
pgalloc_dma 3210
pgfree 1023837
pgactivate 185293
pgdeactivate 157150
pgfault 264830
pgmajfault 3024
pgrefill_high 0
pgrefill_normal 156851
pgrefill_dma 299
pgsteal_high 0
pgsteal_normal 82775
pgsteal_dma 397
pgscan_kswapd_high 0
pgscan_kswapd_normal 126659
pgscan_kswapd_dma 457
pgscan_direct_high 0
pgscan_direct_normal 147678
pgscan_direct_dma 365
pginodesteal 6491
slabs_scanned 2800364
kswapd_steal 55007
kswapd_inodesteal 8269
pageoutrun 1147
allocstall 2225
pgrotated 8831

linux-2.6.7-bk7 (good)
===============
cat /proc/sys/fs/dentry-state
6990    4522    45      0       0       0
du -s /disc1
..
some snapshoots of /proc/sys/fs/dentry-state
761894  723814  45      0       0       0
760283  722353  45      0       0       0
759821  721937  45      0       0       0
759872  722127  45      0       0       0
759742  722079  45      0       0       0
759666  722090  45      0       0       0
757161  719829  45      0       0       0
755236  718111  45      0       0       0
755639  718584  45      0       0       0
714917  683972  45      0       0       0
716095  685185  45      0       0       0
715329  684496  45      0       0       0
..
42121188      /disc1

cat /proc/vmstat
nr_dirty 7
nr_writeback 0
nr_unstable 0
nr_page_table_pages 469
nr_mapped 43769
nr_slab 219515
pgpgin 854375
pgpgout 136916
pswpin 0
pswpout 0
pgalloc_high 265027
pgalloc_normal 621919
pgalloc_dma 3790
pgfree 1031340
pgactivate 196742
pgdeactivate 169483
pgfault 288603
pgmajfault 3068
pgrefill_high 0
pgrefill_normal 169001
pgrefill_dma 482
pgsteal_high 0
pgsteal_normal 83976
pgsteal_dma 490
pgscan_kswapd_high 0
pgscan_kswapd_normal 134412
pgscan_kswapd_dma 738
pgscan_direct_high 0
pgscan_direct_normal 175212
pgscan_direct_dma 1310
pginodesteal 3476
slabs_scanned 2826482
kswapd_steal 55611
kswapd_inodesteal 6862
pageoutrun 1214
allocstall 2734
pgrotated 8730

linux-2.6.7-bk8 (bad)
===============

cat /proc/sys/fs/dentry-state
7357    4821    45      0       0       0
du -s /disc1/
After some processes wer killed ..
cat /proc/sys/fs/dentry-state
1090719 1035089 45      0       0       0

cat /proc/vmstat
nr_dirty 26
nr_writeback 0
nr_unstable 0
nr_page_table_pages 365
nr_mapped 29495
nr_slab 219834
pgpgin 594519
pgpgout 80416
pswpin 0
pswpout 0
pgalloc_high 199307
pgalloc_normal 489087
pgalloc_dma 3711
pgfree 868142
pgactivate 94618
pgdeactivate 65790
pgfault 227492
pgmajfault 3065
pgrefill_high 0
pgrefill_normal 65694
pgrefill_dma 96
pgsteal_high 0
pgsteal_normal 46724
pgsteal_dma 337
pgscan_kswapd_high 0
pgscan_kswapd_normal 49351
pgscan_kswapd_dma 331
pgscan_direct_high 0
pgscan_direct_normal 46908
pgscan_direct_dma 41
pginodesteal 2743
slabs_scanned 1186575
kswapd_steal 39703
kswapd_inodesteal 6800
pageoutrun 323
allocstall 1088
pgrotated 1382

^ permalink raw reply	[flat|nested] 23+ messages in thread
* Re: dentry cache leak? Re: rsync out of memory 2.6.8-rc2
@ 2004-08-01  9:47 Klaus Dittrich
  0 siblings, 0 replies; 23+ messages in thread
From: Klaus Dittrich @ 2004-08-01  9:47 UTC (permalink / raw)
  To: linux mailing-list

I applied your patch to 2.6.8-rc-bk10 and the system survied 
three concurrent du -s started each started with a delay.

Conspicuous is that memory usage (xosview) does not go above 1.5GB now.
Here is a snapshoot of cat/proc/vmstat taken at a time when memory usage
has stagnated.

--
Klaus

nr_dirty 1294
nr_writeback 0
nr_unstable 0
nr_page_table_pages 425
nr_mapped 38811
nr_slab 211923
pgpgin 1157363
pgpgout 150328
pswpin 0
pswpout 0
pgalloc_high 295581
pgalloc_normal 716183
pgalloc_dma 5455
pgfree 1150071
pgactivate 115758
pgdeactivate 74788
pgfault 263245
pgmajfault 1441
pgrefill_high 0
pgrefill_normal 74461
pgrefill_dma 327
pgsteal_high 0
pgsteal_normal 97886
pgsteal_dma 527
pgscan_kswapd_high 0
pgscan_kswapd_normal 92301
pgscan_kswapd_dma 490
pgscan_direct_high 0
pgscan_direct_normal 22869
pgscan_direct_dma 180
pginodesteal 22206
slabs_scanned 6414979
kswapd_steal 79035
kswapd_inodesteal 32158
pageoutrun 535
allocstall 562
pgrotated 5286

^ permalink raw reply	[flat|nested] 23+ messages in thread

end of thread, other threads:[~2004-08-03  6:59 UTC | newest]

Thread overview: 23+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2004-07-19  9:19 rsync out of memory 2.6.8-rc2 Klaus Dittrich
2004-07-19 11:20 ` dentry cache leak? " bert hubert
     [not found]   ` <20040719113228.GA15295@outpost.ds9a.nl>
2004-07-19 12:56     ` Klaus Dittrich
2004-07-26  1:06       ` Andrew Morton
  -- strict thread matches above, loose matches on Subject: below --
2004-07-26 15:06 Klaus Dittrich
2004-07-26 19:37 ` Andrew Morton
2004-07-26 20:02   ` Klaus Dittrich
2004-07-26 20:38     ` Andrew Morton
2004-07-26 21:39       ` Klaus Dittrich
2004-07-26 22:14         ` Jan-Frode Myklebust
2004-07-27 20:43           ` Klaus Dittrich
2004-07-27 21:05             ` Klaus Dittrich
2004-07-28  1:21               ` Nick Piggin
2004-07-28 15:06                 ` Klaus Dittrich
2004-07-29  3:31                   ` Nick Piggin
2004-07-29 21:07 ` Andrew Morton
2004-07-30 16:30   ` Marcelo Tosatti
2004-07-30 19:47     ` Andrew Morton
2004-07-31  2:01       ` Marcelo Tosatti
2004-07-31 17:23       ` Linus Torvalds
2004-07-31 21:39         ` Andrew Morton
2004-08-03  6:55           ` Nick Piggin
2004-08-01  9:47 Klaus Dittrich

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox