Re: Recursive chmod/chown OOM kills box with 32MB RAM

[Bill Davidsen <davidsen@tmr.com>]

Denis Vlasenko wrote:
> On Monday 30 January 2006 15:22, Chris Mason wrote:
> 
>>On Monday 30 January 2006 01:11, Hans Reiser wrote:
>>
>>>Chris, would Denis Vlasenko wrote:
>>>
>>>>[CCing namesys]
>>>>
>>>>Narrowed it down to 100% reproducible case:
>>>>
>>>>	chown -Rc 0:<n> .
>>>>
>>>>in a top directory of tree containing ~21938 files
>>>>on reiser3 partition:
>>>>
>>>>	/dev/sdc3 on /.3 type reiserfs (rw,noatime)
>>>>
>>>>causes oom kill storm. "ls -lR", "find ." etc work fine.
>>>>
>>>>I suspected that it is a leak in winbindd libnss module,
>>>>but chown does not seem to grow larger in top, and also
>>>>running it under softlimit -m 400000 still causes oom kills
>>>>while chown's RSS stays below 4MB.
>>
>>In order for the journaled filesystems to make sure the FS is consistent after 
>>a crash, we need to keep some blocks in memory until other blocks have been 
>>written.  These blocks are pinned, and can't be freed until a certain amount 
>>of io is done.
>>
>>In the case of reiserfs, it might pin as much as the size of the journal at 
>>any time.  The default journal is 32MB, which is much too large for a system 
>>with only 32MB of ram.
>>
>>You can shrink the log of an existing filesystem.  The minimum size is 513 
>>blocks, you might try 1024 as a good starting poing.
>>
>>reiserfstune -s 1024 /dev/xxxx
>>
>>The filesystem must be unmounted first.
> 
> 
> Will try this and report the result.
> 
> Please consider printing a big fat warning at mount time if total RAM
> on the system is close to sum of RAM space required for all currently
> mounted reiserfs partitions...

I would think that rather than warn about the problem that it would be 
better to teach the filesystem not to use all of memory, and to stop and 
wait for i/o to finish rather than pin so many pages that the system 
becomes unusable. For definitions of usable which include not killing 
random small processes because the filesystem code isn't playing nicely.

Would some global count of reiser_pinned_pages be possible as a way to 
track the problem?

-- 
    -bill davidsen (davidsen@tmr.com)
"The secret to procrastination is to put things off until the
  last possible moment - but no longer"  -me