[PATCH] slab: document SLAB_DESTROY_BY_RCU

[PATCH] slab: document SLAB_DESTROY_BY_RCU

[Peter Zijlstra]

Subject: slab: document SLAB_DESTROY_BY_RCU
From: Peter Zijlstra <a.p.zijlstra@chello.nl>
Date: Wed, 02 Apr 2008 14:13:42 +0200

Explain this SLAB_DESTROY_BY_RCU thing...

Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Acked-by: Jens Axboe <jens.axboe@oracle.com>
Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 include/linux/slab.h |   26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

Index: linux-2.6/include/linux/slab.h
===================================================================
--- linux-2.6.orig/include/linux/slab.h
+++ linux-2.6/include/linux/slab.h
@@ -23,6 +23,32 @@
 #define SLAB_CACHE_DMA		0x00004000UL	/* Use GFP_DMA memory */
 #define SLAB_STORE_USER		0x00010000UL	/* DEBUG: Store the last owner for bug hunting */
 #define SLAB_PANIC		0x00040000UL	/* Panic if kmem_cache_create() fails */
+/*
+ * SLAB_DESTROY_BY_RCU - **WARNING** READ THIS!
+ *
+ * This delays freeing the SLAB page by a grace period, it does _NOT_
+ * delay object freeing. This means that if you do kmem_cache_free()
+ * that memory location is free to be reused at any time. Thus it may
+ * be possible to see another object there in the same RCU grace period.
+ *
+ * This feature only ensures the memory location backing the object
+ * stays valid, the trick to using this is relying on an independent
+ * object validation pass. Something like:
+ *
+ *  rcu_read_lock()
+ * again:
+ *  obj = lockless_lookup(key);
+ *  if (obj) {
+ *    if (!try_get_ref(obj)) // might fail for free objects
+ *      goto again;
+ *
+ *    if (obj->key != key) { // not the object we expected
+ *      put_ref(obj);
+ *      goto again;
+ *    }
+ *  }
+ *  rcu_read_unlock();
+ */
 #define SLAB_DESTROY_BY_RCU	0x00080000UL	/* Defer freeing slabs to RCU */
 #define SLAB_MEM_SPREAD		0x00100000UL	/* Spread some memory over cpuset */
 #define SLAB_TRACE		0x00200000UL	/* Trace allocations and frees */

Re: [PATCH] slab: document SLAB_DESTROY_BY_RCU

[Pekka Enberg]

On Thu, Nov 13, 2008 at 4:28 PM, Peter Zijlstra <peterz@infradead.org> wrote:
> Subject: slab: document SLAB_DESTROY_BY_RCU
> From: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Date: Wed, 02 Apr 2008 14:13:42 +0200
>
> Explain this SLAB_DESTROY_BY_RCU thing...
>
> Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Acked-by: Jens Axboe <jens.axboe@oracle.com>
> Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>

Acked-by: Pekka Enberg <penberg@cs.helsinki.fi>

> ---
>  include/linux/slab.h |   26 ++++++++++++++++++++++++++
>  1 file changed, 26 insertions(+)
>
> Index: linux-2.6/include/linux/slab.h
> ===================================================================
> --- linux-2.6.orig/include/linux/slab.h
> +++ linux-2.6/include/linux/slab.h
> @@ -23,6 +23,32 @@
>  #define SLAB_CACHE_DMA         0x00004000UL    /* Use GFP_DMA memory */
>  #define SLAB_STORE_USER                0x00010000UL    /* DEBUG: Store the last owner for bug hunting */
>  #define SLAB_PANIC             0x00040000UL    /* Panic if kmem_cache_create() fails */
> +/*
> + * SLAB_DESTROY_BY_RCU - **WARNING** READ THIS!
> + *
> + * This delays freeing the SLAB page by a grace period, it does _NOT_
> + * delay object freeing. This means that if you do kmem_cache_free()
> + * that memory location is free to be reused at any time. Thus it may
> + * be possible to see another object there in the same RCU grace period.
> + *
> + * This feature only ensures the memory location backing the object
> + * stays valid, the trick to using this is relying on an independent
> + * object validation pass. Something like:
> + *
> + *  rcu_read_lock()
> + * again:
> + *  obj = lockless_lookup(key);
> + *  if (obj) {
> + *    if (!try_get_ref(obj)) // might fail for free objects
> + *      goto again;
> + *
> + *    if (obj->key != key) { // not the object we expected
> + *      put_ref(obj);
> + *      goto again;
> + *    }
> + *  }
> + *  rcu_read_unlock();
> + */
>  #define SLAB_DESTROY_BY_RCU    0x00080000UL    /* Defer freeing slabs to RCU */
>  #define SLAB_MEM_SPREAD                0x00100000UL    /* Spread some memory over cpuset */
>  #define SLAB_TRACE             0x00200000UL    /* Trace allocations and frees */
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: [PATCH] slab: document SLAB_DESTROY_BY_RCU

[Hugh Dickins]

On Thu, 13 Nov 2008, Peter Zijlstra wrote:
> Subject: slab: document SLAB_DESTROY_BY_RCU

Yes, that's good, thanks Peter.  But I still think it's even
better with at least a pointer to the original description in
mm/slab.c: I've added one or two lines to the comment below.


From: Peter Zijlstra <a.p.zijlstra@chello.nl>

Explain this SLAB_DESTROY_BY_RCU thing...

Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Acked-by: Jens Axboe <jens.axboe@oracle.com>
Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Signed-off-by: Hugh Dickins <hugh@veritas.com>
---
 include/linux/slab.h |   28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

Index: linux-2.6/include/linux/slab.h
===================================================================
--- linux-2.6.orig/include/linux/slab.h
+++ linux-2.6/include/linux/slab.h
@@ -23,6 +23,34 @@
 #define SLAB_CACHE_DMA		0x00004000UL	/* Use GFP_DMA memory */
 #define SLAB_STORE_USER		0x00010000UL	/* DEBUG: Store the last owner for bug hunting */
 #define SLAB_PANIC		0x00040000UL	/* Panic if kmem_cache_create() fails */
+/*
+ * SLAB_DESTROY_BY_RCU - **WARNING** READ THIS!
+ *
+ * This delays freeing the SLAB page by a grace period, it does _NOT_
+ * delay object freeing. This means that if you do kmem_cache_free()
+ * that memory location is free to be reused at any time. Thus it may
+ * be possible to see another object there in the same RCU grace period.
+ *
+ * This feature only ensures the memory location backing the object
+ * stays valid, the trick to using this is relying on an independent
+ * object validation pass. Something like:
+ *
+ *  rcu_read_lock()
+ * again:
+ *  obj = lockless_lookup(key);
+ *  if (obj) {
+ *    if (!try_get_ref(obj)) // might fail for free objects
+ *      goto again;
+ *
+ *    if (obj->key != key) { // not the object we expected
+ *      put_ref(obj);
+ *      goto again;
+ *    }
+ *  }
+ *  rcu_read_unlock();
+ *
+ * See also the comment on struct slab_rcu in mm/slab.c.
+ */
 #define SLAB_DESTROY_BY_RCU	0x00080000UL	/* Defer freeing slabs to RCU */
 #define SLAB_MEM_SPREAD		0x00100000UL	/* Spread some memory over cpuset */
 #define SLAB_TRACE		0x00200000UL	/* Trace allocations and frees */

Re: [PATCH] slab: document SLAB_DESTROY_BY_RCU

[Peter Zijlstra]

On Thu, 2008-11-13 at 16:33 +0000, Hugh Dickins wrote:
> On Thu, 13 Nov 2008, Peter Zijlstra wrote:
> > Subject: slab: document SLAB_DESTROY_BY_RCU
> 
> Yes, that's good, thanks Peter.  But I still think it's even
> better with at least a pointer to the original description in
> mm/slab.c: I've added one or two lines to the comment below.

Ah, yes, I think to remember we had a similar discussion back in april.

Thanks!

Re: [PATCH] slab: document SLAB_DESTROY_BY_RCU

[Christoph Lameter]

On Thu, 13 Nov 2008, Pekka Enberg wrote:

> Acked-by: Pekka Enberg <penberg@cs.helsinki.fi>

Acked-by: Christoph Lameter <cl@linux-foundation.org

(hope Pekka is going to sign off on this and merge it instead of acking)

Re: [PATCH] slab: document SLAB_DESTROY_BY_RCU

[Pekka Enberg]

On Thu, 13 Nov 2008, Pekka Enberg wrote:
> > Acked-by: Pekka Enberg <penberg@cs.helsinki.fi>

On Thu, 2008-11-13 at 11:25 -0600, Christoph Lameter wrote:
> Acked-by: Christoph Lameter <cl@linux-foundation.org
> 
> (hope Pekka is going to sign off on this and merge it instead of
> acking)

Sure, I can merge it but I assumed there was some reason it was sent to
Andrew and not me. Peter?

		Pekka

Re: [PATCH] slab: document SLAB_DESTROY_BY_RCU

[Paul E. McKenney]

On Thu, Nov 13, 2008 at 03:28:29PM +0100, Peter Zijlstra wrote:
> Subject: slab: document SLAB_DESTROY_BY_RCU
> From: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Date: Wed, 02 Apr 2008 14:13:42 +0200
> 
> Explain this SLAB_DESTROY_BY_RCU thing...

With Hugh's suggested change,

Re-acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>

> Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Acked-by: Jens Axboe <jens.axboe@oracle.com>
> Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> ---
>  include/linux/slab.h |   26 ++++++++++++++++++++++++++
>  1 file changed, 26 insertions(+)
> 
> Index: linux-2.6/include/linux/slab.h
> ===================================================================
> --- linux-2.6.orig/include/linux/slab.h
> +++ linux-2.6/include/linux/slab.h
> @@ -23,6 +23,32 @@
>  #define SLAB_CACHE_DMA		0x00004000UL	/* Use GFP_DMA memory */
>  #define SLAB_STORE_USER		0x00010000UL	/* DEBUG: Store the last owner for bug hunting */
>  #define SLAB_PANIC		0x00040000UL	/* Panic if kmem_cache_create() fails */
> +/*
> + * SLAB_DESTROY_BY_RCU - **WARNING** READ THIS!
> + *
> + * This delays freeing the SLAB page by a grace period, it does _NOT_
> + * delay object freeing. This means that if you do kmem_cache_free()
> + * that memory location is free to be reused at any time. Thus it may
> + * be possible to see another object there in the same RCU grace period.
> + *
> + * This feature only ensures the memory location backing the object
> + * stays valid, the trick to using this is relying on an independent
> + * object validation pass. Something like:
> + *
> + *  rcu_read_lock()
> + * again:
> + *  obj = lockless_lookup(key);
> + *  if (obj) {
> + *    if (!try_get_ref(obj)) // might fail for free objects
> + *      goto again;
> + *
> + *    if (obj->key != key) { // not the object we expected
> + *      put_ref(obj);
> + *      goto again;
> + *    }
> + *  }
> + *  rcu_read_unlock();
> + */
>  #define SLAB_DESTROY_BY_RCU	0x00080000UL	/* Defer freeing slabs to RCU */
>  #define SLAB_MEM_SPREAD		0x00100000UL	/* Spread some memory over cpuset */
>  #define SLAB_TRACE		0x00200000UL	/* Trace allocations and frees */
> 

Re: [PATCH] slab: document SLAB_DESTROY_BY_RCU

[Peter Zijlstra]

On Thu, 2008-11-13 at 19:31 +0200, Pekka Enberg wrote:
> On Thu, 13 Nov 2008, Pekka Enberg wrote:
> > > Acked-by: Pekka Enberg <penberg@cs.helsinki.fi>
> 
> On Thu, 2008-11-13 at 11:25 -0600, Christoph Lameter wrote:
> > Acked-by: Christoph Lameter <cl@linux-foundation.org
> > 
> > (hope Pekka is going to sign off on this and merge it instead of
> > acking)
> 
> Sure, I can merge it but I assumed there was some reason it was sent to
> Andrew and not me. Peter?

I'm just having one of those days,..

Please merge ;-)

Re: [PATCH] slab: document SLAB_DESTROY_BY_RCU

[Pekka Enberg]

Hugh Dickins wrote:
> On Thu, 13 Nov 2008, Peter Zijlstra wrote:
>> Subject: slab: document SLAB_DESTROY_BY_RCU
> 
> Yes, that's good, thanks Peter.  But I still think it's even
> better with at least a pointer to the original description in
> mm/slab.c: I've added one or two lines to the comment below.

Applied, thanks!