From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752459AbXJ1Dfn (ORCPT ); Sat, 27 Oct 2007 23:35:43 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1751881AbXJ1DdK (ORCPT ); Sat, 27 Oct 2007 23:33:10 -0400 Received: from netops-testserver-4-out.sgi.com ([192.48.171.29]:37604 "EHLO relay.sgi.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751240AbXJ1DdA (ORCPT ); Sat, 27 Oct 2007 23:33:00 -0400 Message-Id: <20071028033259.992768446@sgi.com> References: <20071028033156.022983073@sgi.com> User-Agent: quilt/0.46-1 Date: Sat, 27 Oct 2007 20:32:03 -0700 From: Christoph Lameter To: Matthew Wilcox Cc: akpm@linux-foundation.org Cc: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org Cc: Pekka Enberg Subject: [patch 07/10] SLUB: Avoid referencing kmem_cache structure in __slab_alloc Content-Disposition: inline; filename=slub_objects_in_per_cpu Sender: linux-kernel-owner@vger.kernel.org X-Mailing-List: linux-kernel@vger.kernel.org There is the need to use the objects per slab in the first part of __slab_alloc() which is still pretty hot. Copy the number of objects per slab into the kmem_cache_cpu structure. That way we can get the value from a cache line that we already need to touch. This brings the kmem_cache_cpu structure up to 4 even words. There is no increase in the size of kmem_cache_cpu since the size of object is rounded to the next word. Signed-off-by: Christoph Lameter --- include/linux/slub_def.h | 1 + mm/slub.c | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) Index: linux-2.6/include/linux/slub_def.h =================================================================== --- linux-2.6.orig/include/linux/slub_def.h 2007-10-26 19:09:16.000000000 -0700 +++ linux-2.6/include/linux/slub_def.h 2007-10-27 07:55:12.000000000 -0700 @@ -17,6 +17,7 @@ struct kmem_cache_cpu { int node; unsigned int offset; unsigned int objsize; + unsigned int objects; }; struct kmem_cache_node { Index: linux-2.6/mm/slub.c =================================================================== --- linux-2.6.orig/mm/slub.c 2007-10-27 07:52:12.000000000 -0700 +++ linux-2.6/mm/slub.c 2007-10-27 07:55:12.000000000 -0700 @@ -1512,7 +1512,7 @@ load_freelist: object = c->page->freelist; c->freelist = object[c->offset]; - c->page->inuse = s->objects; + c->page->inuse = c->objects; c->page->freelist = c->page->end; c->node = page_to_nid(c->page); unlock_out: @@ -1896,6 +1896,7 @@ static void init_kmem_cache_cpu(struct k c->node = 0; c->offset = s->offset / sizeof(void *); c->objsize = s->objsize; + c->objects = s->objects; } static void init_kmem_cache_node(struct kmem_cache_node *n) --