Re: [patch] mm: Convert global dirty_exceeded flag to per-node node_dirty_exceeded

[Ravikiran G Thirumalai <kiran@scalex86.org>]

On Mon, Jan 16, 2006 at 06:13:23PM -0800, Andrew Morton wrote:
> Ravikiran G Thirumalai <kiran@scalex86.org> wrote:
> >
> > Convert global dirty_exceeded flag to per-node node_dirty_exceeded.
> > 
> > dirty_exceeded ping pongs between nodes in order to force all cpus in
> > the system to increase the frequency of calls to balance_dirty_pages.
> > 
> > Currently dirty_exceeded is used by balance_dirty_pages_ratelimited to
> > force all CPUs in the system call balance_dirty_pages often, in order to
> > reduce the amount of dirty pages in the entire system (based on
> > dirty_thresh and one CPU exceeding thee ratelimits).  As dirty_exceeded
> > is a global variable, it will ping-pong between nodes of a NUMA system
> > which is not good.
> 
> Did you not test this obvious little optimisation?

We ran the test we encountered this problem on with your patch.
At first it looked like it did not help.  But later we found that there was
false sharing on this variable.  So padding dirty_exceeded to cacheline
along with your patch helps.

Thanks,
Kiran


Avoid false sharing on dirty_exceeded.

Signed-off-by: Ravikiran Thirumalai <kiran@scalex86.org>

Index: build-2.6.15/mm/page-writeback.c
===================================================================
--- build-2.6.15.orig/mm/page-writeback.c	2006-01-17 12:46:00.000000000 -0800
+++ build-2.6.15/mm/page-writeback.c	2006-01-17 16:21:47.000000000 -0800
@@ -46,7 +46,7 @@
 static long ratelimit_pages = 32;
 
 static long total_pages;	/* The total number of pages in the machine. */
-static int dirty_exceeded;	/* Dirty mem may be over limit */
+static int dirty_exceeded __cacheline_aligned_in_smp;	/* Dirty mem may be over limit */
 
 /*
  * When balance_dirty_pages decides that the caller needs to perform some