Re: [PATCH]: Free pages from local pcp lists under tight memory conditions

[Andrew Morton <akpm@osdl.org>]

Rohit Seth <rohit.seth@intel.com> wrote:
>
> On Wed, 2005-11-23 at 11:30 -0800, Christoph Lameter wrote:
> > On Tue, 22 Nov 2005, Rohit Seth wrote:
> > 
> > > [PATCH]: This patch free pages (pcp->batch from each list at a time) from
> > > local pcp lists when a higher order allocation request is not able to 
> > > get serviced from global free_list.
> > 
> > Ummm.. One controversial idea: How about removing the complete pcp 
> > subsystem? Last time we disabled pcps we saw that the effect 
> > that it had was within noise ratio on AIM7. The lru lock taken without 
> > pcp is in the local zone and thus rarely contended.
> 
> Oh please stop.
> 
> This per_cpu_pagelist is one great logic that has got added in
> allocator.  Besides providing pages without the need to acquire the zone
> lock, it is one single main reason the coloring effect is drastically
> reduced in 2.6 (over 2.4) based kernels.
> 

hm.  Before it was merged in 2.5.x, the feature was very marginal from a
performance POV in my testing on 4-way.

I was able to demonstrate a large (~60%?) speedup in one microbenckmark
which consisted of four processes writing 16k to a file and truncating it
back to zero again.  That gain came from the cache warmth effect, which is
the other benefit which these cpu-local pages are supposed to provide.

I don't think Martin was able to demonstrate much benefit from the lock
contention reduction on 16-way NUMAQ either.

So I dithered for months and it was a marginal merge, so it's appropriate
to justify the continued presence of the code.

We didn't measure for any coloring effects though.  In fact, I didn't know
that this feature actually provided any benefit in that area.