[PATCH 1/3] VM: add kzalloc_node inline

[PATCH 1/3] VM: add kzalloc_node inline

[Jeff Layton]

...to get zeroed out memory from a particular NUMA node

Signed-off-by: Jeff Layton <jlayton@redhat.com>
---

 include/linux/slab.h |   11 +++++++++++
 1 files changed, 11 insertions(+), 0 deletions(-)

diff --git a/include/linux/slab.h b/include/linux/slab.h
index 805ed4b..c2ad350 100644
--- a/include/linux/slab.h
+++ b/include/linux/slab.h
@@ -276,6 +276,17 @@ static inline void *kzalloc(size_t size, gfp_t flags)
 	return kmalloc(size, flags | __GFP_ZERO);
 }
 
+/**
+ * kzalloc_node - allocate zeroed memory from a particular memory node.
+ * @size: how many bytes of memory are required.
+ * @flags: the type of memory to allocate (see kmalloc).
+ * @node: memory node from which to allocate
+ */
+static inline void *kzalloc_node(size_t size, gfp_t flags, int node)
+{
+	return kmalloc_node(size, flags | __GFP_ZERO, node);
+}
+
 #ifdef CONFIG_SLABINFO
 extern const struct seq_operations slabinfo_op;
 ssize_t slabinfo_write(struct file *, const char __user *, size_t, loff_t *);

[PATCH 2/3] sunrpc: have pooled services make NUMA-friendly allocations

[Jeff Layton]

Currently, svc_prepare_thread allocates memory using plain kmalloc()
and alloc_page() calls, even for threads that are destined to run on
different CPUs or NUMA nodes than the current one. Add a function to
translate a poolid into a NUMA node, and have svc_prepare_thread and
svc_init_buffer allocate memory on those nodes instead.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
---

 net/sunrpc/svc.c |   43 +++++++++++++++++++++++++++++++++++++------
 1 files changed, 37 insertions(+), 6 deletions(-)

diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c
index 01c7e31..a61e7aa 100644
--- a/net/sunrpc/svc.c
+++ b/net/sunrpc/svc.c
@@ -332,6 +332,31 @@ svc_pool_map_set_cpumask(unsigned int pidx, cpumask_t *oldmask)
 }
 
 /*
+ * for a given poolid, return the closest NUMA node to its threads
+ */
+static unsigned int
+svc_pool_to_node(unsigned int pidx)
+{
+	struct svc_pool_map *m = &svc_pool_map;
+	unsigned int poolnode = m->pool_to[pidx];
+
+	/*
+	 * The caller checks for sv_nrpools > 1, which
+	 * implies that we've been initialized.
+	 */
+	BUG_ON(m->count == 0);
+
+	switch (m->mode) {
+	case SVC_POOL_PERNODE:
+		return poolnode;
+	case SVC_POOL_PERCPU:
+		return cpu_to_node(poolnode);
+	}
+
+	return numa_node_id();
+}
+
+/*
  * Use the mapping mode to choose a pool for a given CPU.
  * Used when enqueueing an incoming RPC.  Always returns
  * a non-NULL pool pointer.
@@ -507,7 +532,7 @@ EXPORT_SYMBOL(svc_destroy);
  * We allocate pages and place them in rq_argpages.
  */
 static int
-svc_init_buffer(struct svc_rqst *rqstp, unsigned int size)
+svc_init_buffer(struct svc_rqst *rqstp, unsigned int size, unsigned int node)
 {
 	unsigned int pages, arghi;
 
@@ -517,7 +542,7 @@ svc_init_buffer(struct svc_rqst *rqstp, unsigned int size)
 	arghi = 0;
 	BUG_ON(pages > RPCSVC_MAXPAGES);
 	while (pages) {
-		struct page *p = alloc_page(GFP_KERNEL);
+		struct page *p = alloc_pages_node(node, GFP_KERNEL, 0);
 		if (!p)
 			break;
 		rqstp->rq_pages[arghi++] = p;
@@ -543,8 +568,14 @@ struct svc_rqst *
 svc_prepare_thread(struct svc_serv *serv, struct svc_pool *pool)
 {
 	struct svc_rqst	*rqstp;
+	unsigned int	node;
+
+	if (serv->sv_nrpools > 1)
+		node = svc_pool_to_node(pool->sp_id);
+	else
+		node = numa_node_id();
 
-	rqstp = kzalloc(sizeof(*rqstp), GFP_KERNEL);
+	rqstp = kzalloc_node(sizeof(*rqstp), GFP_KERNEL, node);
 	if (!rqstp)
 		goto out_enomem;
 
@@ -558,15 +589,15 @@ svc_prepare_thread(struct svc_serv *serv, struct svc_pool *pool)
 	rqstp->rq_server = serv;
 	rqstp->rq_pool = pool;
 
-	rqstp->rq_argp = kmalloc(serv->sv_xdrsize, GFP_KERNEL);
+	rqstp->rq_argp = kmalloc_node(serv->sv_xdrsize, GFP_KERNEL, node);
 	if (!rqstp->rq_argp)
 		goto out_thread;
 
-	rqstp->rq_resp = kmalloc(serv->sv_xdrsize, GFP_KERNEL);
+	rqstp->rq_resp = kmalloc_node(serv->sv_xdrsize, GFP_KERNEL, node);
 	if (!rqstp->rq_resp)
 		goto out_thread;
 
-	if (!svc_init_buffer(rqstp, serv->sv_max_mesg))
+	if (!svc_init_buffer(rqstp, serv->sv_max_mesg, node))
 		goto out_thread;
 
 	return rqstp;

[PATCH 3/3] sunrpc: change default for pooled services to SVC_POOL_AUTO and make settings persistent

[Jeff Layton]

The current default for pooled services is SVC_POOL_GLOBAL, which just
does allocations in whatever pool the current process is running, and
doesn't restrict which CPUs the tasks will run.

Change the default to SVC_POOL_AUTO so that large-scale machines will
automatically choose an allocation and cpu masking scheme that's better
suited to their architecture. Also, change it so that when the pool
refcount goes to zero, that we do not reset the allocation scheme to the
compile-time default. The setting should stick once it's made.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
---

 net/sunrpc/svc.c |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)

diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c
index a61e7aa..7238c37 100644
--- a/net/sunrpc/svc.c
+++ b/net/sunrpc/svc.c
@@ -39,7 +39,7 @@ enum {
 	SVC_POOL_PERCPU,	/* one pool per cpu */
 	SVC_POOL_PERNODE	/* one pool per numa node */
 };
-#define SVC_POOL_DEFAULT	SVC_POOL_GLOBAL
+#define SVC_POOL_DEFAULT	SVC_POOL_AUTO
 
 /*
  * Structure for mapping cpus to pools and vice versa.
@@ -280,7 +280,6 @@ svc_pool_map_put(void)
 	mutex_lock(&svc_pool_map_mutex);
 
 	if (!--m->count) {
-		m->mode = SVC_POOL_DEFAULT;
 		kfree(m->to_pool);
 		kfree(m->pool_to);
 		m->npools = 0;

Re: [PATCH 2/3] sunrpc: have pooled services make NUMA-friendly allocations

[Jeff Layton]

On Tue, 03 Jun 2008 07:18:02 -0400
Jeff Layton <jlayton@redhat.com> wrote:

> Currently, svc_prepare_thread allocates memory using plain kmalloc()
> and alloc_page() calls, even for threads that are destined to run on
> different CPUs or NUMA nodes than the current one. Add a function to
> translate a poolid into a NUMA node, and have svc_prepare_thread and
> svc_init_buffer allocate memory on those nodes instead.
> 
> Signed-off-by: Jeff Layton <jlayton@redhat.com>
> ---


--------[snip]--------
>  
> -	rqstp = kzalloc(sizeof(*rqstp), GFP_KERNEL);
> +	rqstp = kzalloc_node(sizeof(*rqstp), GFP_KERNEL, node);

Bruce,
  It looks like AKPM has taken the kzalloc_node patch into -mm. I'd
like to have you take this set into your tree at some point, but don't
want you to have to carry that VM patch too. Would you be amenable to me
changing the above to something like:

	/* FIXME: change to kzalloc_node when/if it makes it to mainline */
	rqstp = kmalloc_node(sizeof(*rqstp), GFP_KERNEL | __GFP_ZERO, node);

...and then we can make the FIXME change when mainline has the new inline?

Thanks,
-- 
Jeff Layton <jlayton@redhat.com>

Re: [PATCH 2/3] sunrpc: have pooled services make NUMA-friendly allocations

[J. Bruce Fields]

On Fri, Jun 06, 2008 at 11:21:45AM -0400, Jeff Layton wrote:
> On Tue, 03 Jun 2008 07:18:02 -0400
> Jeff Layton <jlayton@redhat.com> wrote:
> 
> > Currently, svc_prepare_thread allocates memory using plain kmalloc()
> > and alloc_page() calls, even for threads that are destined to run on
> > different CPUs or NUMA nodes than the current one. Add a function to
> > translate a poolid into a NUMA node, and have svc_prepare_thread and
> > svc_init_buffer allocate memory on those nodes instead.
> > 
> > Signed-off-by: Jeff Layton <jlayton@redhat.com>
> > ---
> 
> 
> --------[snip]--------
> >  
> > -	rqstp = kzalloc(sizeof(*rqstp), GFP_KERNEL);
> > +	rqstp = kzalloc_node(sizeof(*rqstp), GFP_KERNEL, node);
> 
> Bruce,
>   It looks like AKPM has taken the kzalloc_node patch into -mm. I'd
> like to have you take this set into your tree at some point, but don't
> want you to have to carry that VM patch too. Would you be amenable to me
> changing the above to something like:
> 
> 	/* FIXME: change to kzalloc_node when/if it makes it to mainline */
> 	rqstp = kmalloc_node(sizeof(*rqstp), GFP_KERNEL | __GFP_ZERO, node);
> 
> ...and then we can make the FIXME change when mainline has the new inline?

I'd rather just take a copy of the patch.  Perhaps you see a problem I
don't--but if it's really identical to the patch that'll go into
upstream, then they'll merge trivially and there shouldn't be a problem.

--b.