From: Christoph Lameter <cl@linux.com>
To: Pekka Enberg <penberg@cs.helsinki.fi>
Cc: David Rientjes <rientjes@google.com>,
Andi Kleen <andi@firstfloor.org>,
tj@kernel.org, Metathronius Galabant <m.galabant@googlemail.com>,
Matt Mackall <mpm@selenic.com>,
Eric Dumazet <eric.dumazet@gmail.com>,
Adrian Drzewiecki <z@drze.net>,
linux-mm@kvack.org
Subject: [slub rfc1 11/12] slub: Remove kmem_cache_cpu dependency from acquire slab
Date: Fri, 02 Sep 2011 15:47:08 -0500 [thread overview]
Message-ID: <20110902204745.539298711@linux.com> (raw)
In-Reply-To: 20110902204657.105194589@linux.com
[-- Attachment #1: remove_kmem_cache_cpu_dependency_from_acquire_slab --]
[-- Type: text/plain, Size: 2363 bytes --]
Instead of putting the freepointer into the kmem_cache_cpu structure put it
into the page struct reusing the lru.next field.
Also convert the manual warning into a WARN_ON.
Signed-off-by: Christoph Lameter <cl@linux.com>
---
mm/slub.c | 41 +++++++++++++++--------------------------
1 file changed, 15 insertions(+), 26 deletions(-)
Index: linux-2.6/mm/slub.c
===================================================================
--- linux-2.6.orig/mm/slub.c 2011-09-02 10:12:25.981176437 -0500
+++ linux-2.6/mm/slub.c 2011-09-02 10:12:30.881176403 -0500
@@ -1569,37 +1569,25 @@ static inline int acquire_slab(struct km
* The old freelist is the list of objects for the
* per cpu allocation list.
*/
- do {
- freelist = page->freelist;
- counters = page->counters;
- new.counters = counters;
- new.inuse = page->objects;
+ freelist = page->freelist;
+ counters = page->counters;
+ new.counters = counters;
+ new.inuse = page->objects;
- VM_BUG_ON(new.frozen);
- new.frozen = 1;
+ VM_BUG_ON(new.frozen);
+ new.frozen = 1;
- } while (!__cmpxchg_double_slab(s, page,
+ if (!__cmpxchg_double_slab(s, page,
freelist, counters,
NULL, new.counters,
- "lock and freeze"));
-
- remove_partial(n, page);
+ "acquire_slab"))
- if (freelist) {
- /* Populate the per cpu freelist */
- this_cpu_write(s->cpu_slab->freelist, freelist);
- this_cpu_write(s->cpu_slab->page, page);
- return 1;
- } else {
- /*
- * Slab page came from the wrong list. No object to allocate
- * from. Put it onto the correct list and continue partial
- * scan.
- */
- printk(KERN_ERR "SLUB: %s : Page without available objects on"
- " partial list\n", s->name);
return 0;
- }
+
+ remove_partial(n, page);
+ WARN_ON(!freelist);
+ page->lru.next = freelist;
+ return 1;
}
/*
@@ -2133,7 +2121,8 @@ new_slab:
page = get_partial(s, gfpflags, node);
if (page) {
stat(s, ALLOC_FROM_PARTIAL);
- freelist = c->freelist;
+ freelist = page->lru.next;
+ c->page = page;
if (kmem_cache_debug(s))
goto debug;
goto load_freelist;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2011-09-02 20:47 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-09-02 20:46 [slub rfc1 00/12] slub: RFC lockless allocation paths V1 Christoph Lameter
2011-09-02 20:46 ` [slub rfc1 01/12] slub: free slabs without holding locks (V2) Christoph Lameter
2011-09-02 20:46 ` [slub rfc1 02/12] slub: Remove useless statements in __slab_alloc Christoph Lameter
2011-09-02 20:47 ` [slub rfc1 03/12] slub: Get rid of the node field Christoph Lameter
2011-09-02 20:47 ` [slub rfc1 04/12] slub: Separate out kmem_cache_cpu processing from deactivate_slab Christoph Lameter
2011-09-02 20:47 ` [slub rfc1 05/12] slub: Extract get_freelist from __slab_alloc Christoph Lameter
2011-09-02 20:47 ` [slub rfc1 06/12] slub: Use freelist instead of "object" in __slab_alloc Christoph Lameter
2011-09-02 20:47 ` [slub rfc1 07/12] slub: pass page to node_match() instead of kmem_cache_cpu structure Christoph Lameter
2011-09-02 20:47 ` [slub rfc1 08/12] slub: enable use of deactivate_slab with interrupts on Christoph Lameter
2011-09-02 20:47 ` [slub rfc1 09/12] slub: Run deactivate_slab with interrupts enabled Christoph Lameter
2011-09-02 20:47 ` [slub rfc1 10/12] slub: Enable use of get_partial " Christoph Lameter
2011-09-02 20:47 ` Christoph Lameter [this message]
2011-09-02 20:47 ` [slub rfc1 12/12] slub: Drop page field from kmem_cache_cpu Christoph Lameter
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20110902204745.539298711@linux.com \
--to=cl@linux.com \
--cc=andi@firstfloor.org \
--cc=eric.dumazet@gmail.com \
--cc=linux-mm@kvack.org \
--cc=m.galabant@googlemail.com \
--cc=mpm@selenic.com \
--cc=penberg@cs.helsinki.fi \
--cc=rientjes@google.com \
--cc=tj@kernel.org \
--cc=z@drze.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.