Re: [Ext2-devel] Re: OOM problems on 2.6.12-rc1 with many fsx tests

[Mingming Cao <cmm@us.ibm.com>]

On Sat, 2005-03-26 at 16:23 -0800, Mingming Cao wrote:
> On Fri, 2005-03-25 at 14:11 -0800, Badari Pulavarty wrote:
> > On Fri, 2005-03-25 at 13:56, Andrew Morton wrote:
> > > Mingming Cao <cmm@us.ibm.com> wrote:
> > > >
> > > > I run into OOM problem again on 2.6.12-rc1. I run some(20) fsx tests on
> > > > 2.6.12-rc1 kernel(and 2.6.11-mm4) on ext3 filesystem, after about 10
> > > > hours the system hit OOM, and OOM keep killing processes one by one. I
> > > > could reproduce this problem very constantly on a 2 way PIII 700MHZ with
> > > > 512MB RAM. Also the problem could be reproduced on running the same test
> > > > on reiser fs.
> > > > 
> > > > The fsx command is:
> > > > 
> > > > ./fsx -c 10 -n -r 4096 -w 4096 /mnt/test/foo1 &
> > > 
> > > I was able to reproduce this on ext3.  Seven instances of the above leaked
> > > 10-15MB over 10 hours.  All of it permanently stuck on the LRU.
> > > 
> > > I'll continue to poke at it - see what kernel it started with, which
> > > filesystems it affects, whether it happens on UP&&!PREEMPT, etc.  Not a
> > > quick process.
> > 
> > I reproduced *similar* issue with 2.6.11. The reason I say similar, is
> > there is no OOM kill, but very low free memory and machine doesn't
> > respond at all. (I booted my machine with 256M memory and ran 20 copies
> > of fsx on ext3).
> > 
> > 
> 
> Yes, I re-run the same test on 2.6.11 for 24 hours, like Badari see on
> his machine, my machine did not go to OOM on 2.6.11,still alive, but
> memory is very low(only 5M free). Killed all fsx and umount the ext3
> filesystem did not bring back much memory. I will going to rerun the
> tests without the mapped read/write to see what happen.
> 
> 

Run fsx tests without mapped IO on 2.6.11 seems fine.  Here is
the /proc/meminfo after 18 hours run:

# cat /proc/meminfo
MemTotal:       510464 kB
MemFree:          6004 kB
Buffers:        179420 kB
Cached:           9144 kB
SwapCached:          0 kB
Active:         313236 kB
Inactive:       171380 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:       510464 kB
LowFree:          6004 kB
SwapTotal:     1052216 kB
SwapFree:      1052216 kB
Dirty:            2100 kB
Writeback:           0 kB
Mapped:          24884 kB
Slab:            14788 kB
CommitLimit:   1307448 kB
Committed_AS:    78032 kB
PageTables:        720 kB
VmallocTotal:   516024 kB
VmallocUsed:      1672 kB
VmallocChunk:   514352 kB

elm3b92:~ # killall -9 fsx
elm3b92:~ # cat /proc/meminfo
MemTotal:       510464 kB
MemFree:         21332 kB
Buffers:        179668 kB
Cached:           8828 kB
SwapCached:          0 kB
Active:         298748 kB
Inactive:       171152 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:       510464 kB
LowFree:         21332 kB
SwapTotal:     1052216 kB
SwapFree:      1052216 kB
Dirty:            1140 kB
Writeback:           0 kB
Mapped:          11648 kB
Slab:            14632 kB
CommitLimit:   1307448 kB
Committed_AS:    59800 kB
PageTables:        492 kB
VmallocTotal:   516024 kB
VmallocUsed:      1672 kB
VmallocChunk:   514352 kB

elm3b92:~ # umount /mnt/ext3
elm3b92:~ # cat /proc/meminfo
MemTotal:       510464 kB
MemFree:        181636 kB
Buffers:         22092 kB
Cached:           6740 kB
SwapCached:          0 kB
Active:         151284 kB
Inactive:       158948 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:       510464 kB
LowFree:        181636 kB
SwapTotal:     1052216 kB
SwapFree:      1052216 kB
Dirty:               0 kB
Writeback:           0 kB
Mapped:          11656 kB
Slab:            14052 kB
CommitLimit:   1307448 kB
Committed_AS:    59800 kB
PageTables:        492 kB
VmallocTotal:   516024 kB
VmallocUsed:      1672 kB
VmallocChunk:   514352 kB