Re: [PATCH 004 of 4] knfsd: Provide sunrpc pool_mode module option

[Neil Brown <neilb@suse.de>]

On Monday March 5, akpm@linux-foundation.org wrote:
> On Tue, 6 Mar 2007 13:15:20 +1100 NeilBrown <neilb@suse.de> wrote:
> 
> > Provide a module param "pool_mode" for sunrpc.ko which allows a
> > sysadmin to choose the mode for mapping NFS thread service pools
> > to CPUs.  Values are:
> > 
> > auto	    choose a mapping mode heuristically
> > global	    (default, same as the pre-2.6.19 code) a single global pool
> > percpu	    one pool per CPU
> > pernode	    one pool per NUMA node
> > 
> > Note that since 2.6.19 the hardcoded behaviour has been "auto",
> > this patch makes the default "global".
> > 
> > The pool mode can be changed after boot/modprobe using /sys, if the
> > NFS and lockd services have been shut down.  A useful side effect of
> > this change is to fix a small memory leak when unloading the module.
> 
> Mutter.  Is this really suitable and needed for 2.6.21 at this stage in
> its life?

Something is definitely needed.  Currently on a 4-way SMP machine,
nfsd might only use 1 CPU (depends a bit on irq routing I think).

If that patch is too big, maybe this one?

NeilBrown

--------------------------------
Avoid using nfsd process pools on SMP machines.

From: Neil Brown <neilb@suse.de>

process-pools have real benefits for NUMA, but on SMP
machines they only work if network interface interrupts
go to all CPUs (via round-robin or multiple nics).  This is
not always the case, so disable the pools in this case until
a better solution is developped.

Signed-off-by: Neil Brown <neilb@suse.de>

### Diffstat output
 ./net/sunrpc/svc.c |    6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff .prev/net/sunrpc/svc.c ./net/sunrpc/svc.c
--- .prev/net/sunrpc/svc.c	2007-03-06 16:07:19.000000000 +1100
+++ ./net/sunrpc/svc.c	2007-03-06 16:08:53.000000000 +1100
@@ -79,7 +79,11 @@ svc_pool_map_choose_mode(void)
 		 * x86_64 kernel on Xeons.  In this case we
 		 * want to divide the pools on cpu boundaries.
 		 */
-		return SVC_POOL_PERCPU;
+		/* actually, unless your IRQs round-robin nicely,
+		 * this turns out to be really bad, so just
+		 * go GLOBAL for now until a better fix can be developped
+		 */
+		return SVC_POOL_GLOBAL;
 	}
 
 	/* default: one global pool */