Re: [RFC PATCH 00/19] Cleanup and optimise the page allocator V2

[Lin Ming <ming.m.lin@intel.com>]

We tested this v2 patch series with 2.6.29-rc6 on different machines.

		4P qual-core	2P qual-core	2P qual-core HT
		tigerton	stockley	Nehalem
		------------------------------------------------
tbench		+3%		+2%		0%
oltp		-2%		0%		0%
aim7		0%		0%		0%
specjbb2005	+3%		0%		0%
hackbench	0%		0%		0%	

netperf:
TCP-S-112k	0%		-1%		0%
TCP-S-64k	0%		-1%		+1%
TCP-RR-1	0%		0%		+1%
UDP-U-4k	-2%		0%		-2%
UDP-U-1k	+3%		0%		0%
UDP-RR-1	0%		0%		0%
UDP-RR-512	-1%		0%		+1%

Lin Ming

On Tue, 2009-02-24 at 20:16 +0800, Mel Gorman wrote:
> Still a work in progress but enough has changed that I want to show what
> it current looks like. Performance is still improved a little but there are
> some large outstanding pieces of fruit
> 
> 1. Improving free_pcppages_bulk() does a lot of looping, maybe could be better
> 2. gfp_zone() is still using a cache line for data. I wasn't able to translate
>    Kamezawa-sans suggestion into usable code
> 
> The following two items should be picked up in a second or third pass at
> improving the page allocator
> 
> 1. Working out if knowing whether pages are cold/hot on free is worth it or
>    not
> 2. Precalculating zonelists for cpusets (Andi described how it could be done,
>    it's straight-forward, just will take time but it doesn't affect the
>    majority of users)
> 
> Changes since V1
>   o Remove the ifdef CONFIG_CPUSETS from inside get_page_from_freelist()
>   o Use non-lock bit operations for clearing the mlock flag
>   o Factor out alloc_flags calculation so it is only done once (Peter)
>   o Make gfp.h a bit prettier and clear-cut (Peter)
>   o Instead of deleting a debugging check, replace page_count() in the
>     free path with a version that does not check for compound pages (Nick)
>   o Drop the alteration for hot/cold page freeing until we know if it
>     helps or not
> 
> The complexity of the page allocator has been increasing for some time
> and it has now reached the point where the SLUB allocator is doing strange
> tricks to avoid the page allocator. This is obviously bad as it may encourage
> other subsystems to try avoiding the page allocator as well.
> 
> This series of patches is intended to reduce the cost of the page
> allocator by doing the following.
> 
> Patches 1-3 iron out the entry paths slightly and remove stupid sanity
> checks from the fast path.
> 
> Patch 4 uses a lookup table instead of a number of branches to decide what
> zones are usable given the GFP flags.
> 
> Patch 5 tidies up some flags
> 
> Patch 6 avoids repeated checks of the zonelist
> 
> Patch 7 breaks the allocator up into a fast and slow path where the fast
> path later becomes one long inlined function.
> 
> Patches 8-12 avoids calculating the same things repeatedly and instead
> calculates them once.
> 
> Patches 13-14 inline parts of the allocator fast path
> 
> Patch 15 avoids calling get_pageblock_migratetype() potentially twice on
> every page free
> 
> Patch 16 reduces the number of times interrupts are disabled by reworking
> what free_page_mlock() does and not using locked versions of bit operations.
> 
> Patch 17 avoids using the zonelist cache on non-NUMA machines
> 
> Patch 18 simplifies some debugging checks made during alloc and free.
> 
> Patch 19 avoids a list search in the allocator fast path.
> 
> Running all of these through a profiler shows me the cost of page allocation
> and freeing is reduced by a nice amount without drastically altering how the
> allocator actually works. Excluding the cost of zeroing pages, the cost of
> allocation is reduced by 25% and the cost of freeing by 12%.  Again excluding
> zeroing a page, much of the remaining cost is due to counters, debugging
> checks and interrupt disabling.  Of course when a page has to be zeroed,
> the dominant cost of a page allocation is zeroing it.
> 
> These patches reduce the text size of the kernel by 180 bytes on the one
> x86-64 machine I checked.
> 
> Range of results (positive is good) on 7 machines that completed tests.
> 
> o Kernbench elapsed time	-0.04	to	0.79%
> o Kernbench system time		0 	to	3.74%
> o tbench			-2.85%  to	5.52%
> o Hackbench-sockets		all differences within  noise
> o Hackbench-pipes		-2.98%  to	9.11%
> o Sysbench			-0.04%  to	5.50%
> 
> With hackbench-pipes, only 2 machines out of 7 showed results outside of
> the noise. In almost all cases the strandard deviation between runs of
> hackbench-pipes was reduced with the patches.
> 
> I still haven't run a page-allocator micro-benchmark to see what sort of
> figures that gives.
> 
>  arch/ia64/hp/common/sba_iommu.c   |    2 
>  arch/ia64/kernel/mca.c            |    3 
>  arch/ia64/kernel/uncached.c       |    3 
>  arch/ia64/sn/pci/pci_dma.c        |    3 
>  arch/powerpc/platforms/cell/ras.c |    2 
>  arch/x86/kvm/vmx.c                |    2 
>  drivers/misc/sgi-gru/grufile.c    |    2 
>  drivers/misc/sgi-xp/xpc_uv.c      |    2 
>  include/linux/cpuset.h            |    2 
>  include/linux/gfp.h               |   62 +--
>  include/linux/mm.h                |    1 
>  include/linux/mmzone.h            |    8 
>  init/main.c                       |    1 
>  kernel/profile.c                  |    8 
>  mm/filemap.c                      |    2 
>  mm/hugetlb.c                      |    4 
>  mm/internal.h                     |   11 
>  mm/mempolicy.c                    |    2 
>  mm/migrate.c                      |    2 
>  mm/page_alloc.c                   |  642 +++++++++++++++++++++++++-------------
>  mm/slab.c                         |    4 
>  mm/slob.c                         |    4 
>  mm/vmalloc.c                      |    1 
>  23 files changed, 490 insertions(+), 283 deletions(-)