Re: [Ext2-devel] Re: OOM problems on 2.6.12-rc1 with many fsx tests

Mingming Cao
2005-03-27T19:26:14Z

On Sat, 2005-03-26 at 16:23 -0800, Mingming Cao wrote:
> On Fri, 2005-03-25 at 14:11 -0800, Badari Pulavarty wrote:
> > On Fri, 2005-03-25 at 13:56, Andrew Morton wrote:
> > > Mingming Cao <cmm@us.ibm.com> wrote:
> > > >
> > > > I run into OOM problem again on 2.6.12-rc1. I run some(20) fsx tests on
> > > > 2.6.12-rc1 kernel(and 2.6.11-mm4) on ext3 filesystem, after about 10
> > > > hours the system hit OOM, and OOM keep killing processes one by one. I
> > > > could reproduce this problem very constantly on a 2 way PIII 700MHZ with
> > > > 512MB RAM. Also the problem could be reproduced on running the same test
> > > > on reiser fs.
> > > >
> > > > The fsx command is:
> > > >
> > > > ./fsx -c 10 -n -r 4096 -w 4096 /mnt/test/foo1 &
> > >
> > > I was able to reproduce this on ext3. Seven instances of the above leaked
> > > 10-15MB over 10 hours. All of it permanently stuck on the LRU.
> > >
> > > I'll continue to poke at it - see what kernel it started with, which
> > > filesystems it affects, whether it happens on UP&&!PREEMPT, etc. Not a
> > > quick process.
> >
> > I reproduced *similar* issue with 2.6.11. The reason I say similar, is
> > there is no OOM kill, but very low free memory and machine doesn't
> > respond at all. (I booted my machine with 256M memory and ran 20 copies
> > of fsx on ext3).
> >
> >
>
> Yes, I re-run the same test on 2.6.11 for 24 hours, like Badari see on
> his machine, my machine did not go to OOM on 2.6.11,still alive, but
> memory is very low(only 5M free). Killed all fsx and umount the ext3
> filesystem did not bring back much memory. I will going to rerun the
> tests without the mapped read/write to see what happen.
>
>

Run fsx tests without mapped IO on 2.6.11 seems fine. Here is
the /proc/meminfo after 18 hours run:

# cat /proc/meminfo
MemTotal: 510464 kB
MemFree: 6004 kB
Buffers: 179420 kB
Cached: 9144 kB
SwapCached: 0 kB
Active: 313236 kB
Inactive: 171380 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 510464 kB
LowFree: 6004 kB
SwapTotal: 1052216 kB
SwapFree: 1052216 kB
Dirty: 2100 kB
Writeback: 0 kB
Mapped: 24884 kB
Slab: 14788 kB
CommitLimit: 1307448 kB
Committed_AS: 78032 kB
PageTables: 720 kB
VmallocTotal: 516024 kB
VmallocUsed: 1672 kB
VmallocChunk: 514352 kB

elm3b92:~ # killall -9 fsx
elm3b92:~ # cat /proc/meminfo
MemTotal: 510464 kB
MemFree: 21332 kB
Buffers: 179668 kB
Cached: 8828 kB
SwapCached: 0 kB
Active: 298748 kB
Inactive: 171152 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 510464 kB
LowFree: 21332 kB
SwapTotal: 1052216 kB
SwapFree: 1052216 kB
Dirty: 1140 kB
Writeback: 0 kB
Mapped: 11648 kB
Slab: 14632 kB
CommitLimit: 1307448 kB
Committed_AS: 59800 kB
PageTables: 492 kB
VmallocTotal: 516024 kB
VmallocUsed: 1672 kB
VmallocChunk: 514352 kB

elm3b92:~ # umount /mnt/ext3
elm3b92:~ # cat /proc/meminfo
MemTotal: 510464 kB
MemFree: 181636 kB
Buffers: 22092 kB
Cached: 6740 kB
SwapCached: 0 kB
Active: 151284 kB
Inactive: 158948 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 510464 kB
LowFree: 181636 kB
SwapTotal: 1052216 kB
SwapFree: 1052216 kB
Dirty: 0 kB
Writeback: 0 kB
Mapped: 11656 kB
Slab: 14052 kB
CommitLimit: 1307448 kB
Committed_AS: 59800 kB
PageTables: 492 kB
VmallocTotal: 516024 kB
VmallocUsed: 1672 kB
VmallocChunk: 514352 kB