Re: 2.6.14 kswapd eating too much CPU

[Marcelo Tosatti <marcelo.tosatti@cyclades.com>]

Hi Jan,

> 	The task that probably triggers this problem is a cron job
> doing full-text indexing of mailing list archive, so it accesses lots
> of small files, and then recreates the inverted index, which is one big
> file. So maybe inode cache shrinking or something may be the problem there.
> However, the cron job does an incremental reindexing only, so I think it
> reads less than 100 files per each run.
> : 
> : Maybe you should also try profile/oprofile during the kswapd peeks?
> : 
> 	Do you have any details on it? I can of course RTFdocs of oprofile,
> but should I try to catch something special?

It does seem to scan SLABs intensively:

pgscan_kswapd_high 0
pgscan_kswapd_normal 940269891
pgscan_kswapd_dma 0
pgscan_direct_high 0
pgscan_direct_normal 13837131
pgscan_direct_dma 0
pginodesteal 11216563
slabs_scanned 160160350534400
kswapd_steal 909876526
kswapd_inodesteal 305039060
pageoutrun 30139677
allocstall 4067783

If you take the amont of scanned slabs and divide by the sum of
direct/kswapd pagescans:

160160350534400 / (940269891+13837131) = 167864

Which means that for each page scanned about 168000 slab entries are
scanned. Does not look very good.

Other than the profiling can you please also try Bharata's
shrinkable slab cache statistics patch?

http://lkml.org/lkml/2005/10/26/1