Re: ~500 megs cached yet 2.6.5 goes into swap hell

["Martin J. Bligh" <mbligh@aracnet.com>]

>>  I suspect rsync is taking two passes across the source files for its
>>  checksumming thing.  If so, this will defeat the pagecache use-once logic. 
>>  The kernel sees the second touch of the pages and assumes that there will
>>  be a third touch.
> 
> OK, a bit of fiddling does indicate that if a file is present on both
> client and server, and is modified on the client, the rsync client will
> indeed touch the pagecache pages twice.  Does this describe the files which
> you're copying at all?
> 
> One thing you could do is to run `watch -n1 cat /proc/meminfo'.  Cause lots
> of memory to be freed up then do the copy.  Monitor the size of the active
> and inactive lists.  If the active list is growing then we know that rsync
> is touching pages twice.
> 
> That would be an unfortunate special-case.

Personally, I think that the use-twice logic is a bit of a hack that mostly
works. If we moved to a method where we kept an eye on which pages are 
associated with which address_space (for mapped pages) or which process
(for anonymous pages) we'd have a much better shot at stopping any one
process / file from monopolizing the whole of system memory. 

We'd also be able to favour memory for files that are still open over ones 
that have been closed, and recognize linear access scan patterns per file,
and reclaim more agressively from the overscanned areas, and favour higher
prio tasks over lower prio ones (including, but not limited to interactive).

Global LRU (even with the tweaks it has in Linux) doesn't seem optimal.

M.