Re: Free memory never fully used, swapping

[Mel Gorman <mel@csn.ul.ie>]

On Tue, Nov 23, 2010 at 10:43:29PM -0800, Simon Kirby wrote:
> On Tue, Nov 23, 2010 at 10:04:03AM +0000, Mel Gorman wrote:
> 
> > On Mon, Nov 22, 2010 at 03:44:19PM -0800, Andrew Morton wrote:
> > > On Mon, 15 Nov 2010 11:52:46 -0800
> > > Simon Kirby <sim@hostway.ca> wrote:
> > > 
> > > > I noticed that CONFIG_NUMA seems to enable some more complicated
> > > > reclaiming bits and figured it might help since most stock kernels seem
> > > > to ship with it now.  This seems to have helped, but it may just be
> > > > wishful thinking.  We still see this happening, though maybe to a lesser
> > > > degree.  (The following observations are with CONFIG_NUMA enabled.)
> > > > 
> > 
> > Hi,
> > 
> > As this is a NUMA machine, what is the value of
> > /proc/sys/vm/zone_reclaim_mode ? When enabled, this reclaims memory
> > local to the node in preference to using remote nodes. For certain
> > workloads this performs better but for users that expect all of memory
> > to be used, it has surprising results.
> > 
> > If set to 1, try testing with it set to 0 and see if it makes a
> > difference. Thanks
> 
> Hi Mel,
> 
> It is set to 0.  It's an Intel EM64T...I only enabled CONFIG_NUMA since
> it seemed to enable some more complicated handling, and I figured it
> might help, but it didn't seem to.  It's also required for
> CONFIG_COMPACTION, but that is still marked experimental.
> 

I'm surprised a little that you are bringing compaction up because unless
there are high-order involved, it wouldn't make a difference. Is there
a constant source of high-order allocations in the system e.g. a network
card configured to use jumbo frames? A possible consequence of that is that
reclaim is kicking in early to free order-[2-4] pages that would prevent 100%
of memory being used.

-- 
Mel Gorman
Part-time Phd Student                          Linux Technology Center
University of Limerick                         IBM Dublin Software Lab