From: Christoph Lameter <cl@linux.com>
To: Pekka Enberg <penberg@cs.helsinki.fi>
Cc: David Rientjes <rientjes@google.com>,
Andi Kleen <andi@firstfloor.org>,
tj@kernel.org, Metathronius Galabant <m.galabant@googlemail.com>,
Matt Mackall <mpm@selenic.com>,
Eric Dumazet <eric.dumazet@gmail.com>,
Adrian Drzewiecki <z@drze.net>,
linux-mm@kvack.org
Subject: [slub rfc1 05/12] slub: Extract get_freelist from __slab_alloc
Date: Fri, 02 Sep 2011 15:47:02 -0500 [thread overview]
Message-ID: <20110902204742.029988881@linux.com> (raw)
In-Reply-To: 20110902204657.105194589@linux.com
[-- Attachment #1: extract_get_freelist --]
[-- Type: text/plain, Size: 2797 bytes --]
get_freelist retrieves free objects from the page freelist (put there by remote
frees) or deactivates a slab page if no more objects are available.
Signed-off-by: Christoph Lameter <cl@linux.com>
---
mm/slub.c | 57 ++++++++++++++++++++++++++++++++-------------------------
1 file changed, 32 insertions(+), 25 deletions(-)
Index: linux-2.6/mm/slub.c
===================================================================
--- linux-2.6.orig/mm/slub.c 2011-09-02 08:20:25.911219458 -0500
+++ linux-2.6/mm/slub.c 2011-09-02 08:20:32.491219417 -0500
@@ -2024,6 +2024,37 @@ slab_out_of_memory(struct kmem_cache *s,
}
/*
+ * Check the page->freelist of a page and either transfer the freelist to the per cpu freelist
+ * or deactivate the page.
+ *
+ * The page is still frozen if the return value is not NULL.
+ *
+ * If this function returns NULL then the page has been unfrozen.
+ */
+static inline void *get_freelist(struct kmem_cache *s, struct page *page)
+{
+ struct page new;
+ unsigned long counters;
+ void *freelist;
+
+ do {
+ freelist = page->freelist;
+ counters = page->counters;
+ new.counters = counters;
+ VM_BUG_ON(!new.frozen);
+
+ new.inuse = page->objects;
+ new.frozen = freelist != NULL;
+
+ } while (!cmpxchg_double_slab(s, page,
+ freelist, counters,
+ NULL, new.counters,
+ "get_freelist"));
+
+ return freelist;
+}
+
+/*
* Slow path. The lockless freelist is empty or we need to perform
* debugging duties.
*
@@ -2047,8 +2078,6 @@ static void *__slab_alloc(struct kmem_ca
void **object;
struct page *page;
unsigned long flags;
- struct page new;
- unsigned long counters;
local_irq_save(flags);
#ifdef CONFIG_PREEMPT
@@ -2074,29 +2103,7 @@ static void *__slab_alloc(struct kmem_ca
stat(s, ALLOC_SLOWPATH);
- do {
- object = page->freelist;
- counters = page->counters;
- new.counters = counters;
- VM_BUG_ON(!new.frozen);
-
- /*
- * If there is no object left then we use this loop to
- * deactivate the slab which is simple since no objects
- * are left in the slab and therefore we do not need to
- * put the page back onto the partial list.
- *
- * If there are objects left then we retrieve them
- * and use them to refill the per cpu queue.
- */
-
- new.inuse = page->objects;
- new.frozen = object != NULL;
-
- } while (!__cmpxchg_double_slab(s, page,
- object, counters,
- NULL, new.counters,
- "__slab_alloc"));
+ object = get_freelist(s, page);
if (unlikely(!object)) {
c->page = NULL;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2011-09-02 20:47 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-09-02 20:46 [slub rfc1 00/12] slub: RFC lockless allocation paths V1 Christoph Lameter
2011-09-02 20:46 ` [slub rfc1 01/12] slub: free slabs without holding locks (V2) Christoph Lameter
2011-09-02 20:46 ` [slub rfc1 02/12] slub: Remove useless statements in __slab_alloc Christoph Lameter
2011-09-02 20:47 ` [slub rfc1 03/12] slub: Get rid of the node field Christoph Lameter
2011-09-02 20:47 ` [slub rfc1 04/12] slub: Separate out kmem_cache_cpu processing from deactivate_slab Christoph Lameter
2011-09-02 20:47 ` Christoph Lameter [this message]
2011-09-02 20:47 ` [slub rfc1 06/12] slub: Use freelist instead of "object" in __slab_alloc Christoph Lameter
2011-09-02 20:47 ` [slub rfc1 07/12] slub: pass page to node_match() instead of kmem_cache_cpu structure Christoph Lameter
2011-09-02 20:47 ` [slub rfc1 08/12] slub: enable use of deactivate_slab with interrupts on Christoph Lameter
2011-09-02 20:47 ` [slub rfc1 09/12] slub: Run deactivate_slab with interrupts enabled Christoph Lameter
2011-09-02 20:47 ` [slub rfc1 10/12] slub: Enable use of get_partial " Christoph Lameter
2011-09-02 20:47 ` [slub rfc1 11/12] slub: Remove kmem_cache_cpu dependency from acquire slab Christoph Lameter
2011-09-02 20:47 ` [slub rfc1 12/12] slub: Drop page field from kmem_cache_cpu Christoph Lameter
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20110902204742.029988881@linux.com \
--to=cl@linux.com \
--cc=andi@firstfloor.org \
--cc=eric.dumazet@gmail.com \
--cc=linux-mm@kvack.org \
--cc=m.galabant@googlemail.com \
--cc=mpm@selenic.com \
--cc=penberg@cs.helsinki.fi \
--cc=rientjes@google.com \
--cc=tj@kernel.org \
--cc=z@drze.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).