From: Eric Dumazet <eric.dumazet@gmail.com>
To: Christoph Lameter <cl@linux-foundation.org>
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
Pekka Enberg <penberg@cs.helsinki.fi>,
Zdenek Kabelac <zdenek.kabelac@gmail.com>,
Patrick McHardy <kaber@trash.net>, Robin Holt <holt@sgi.com>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
Jesper Dangaard Brouer <hawk@comx.dk>,
Linux Netdev List <netdev@vger.kernel.org>,
Netfilter Developers <netfilter-devel@vger.kernel.org>
Subject: Re: [PATCH] slub: fix slab_pad_check()
Date: Fri, 04 Sep 2009 00:08:42 +0200 [thread overview]
Message-ID: <4AA03E6A.7070800@gmail.com> (raw)
In-Reply-To: <alpine.DEB.1.10.0909031736340.24199@V090114053VZO-1>
Christoph Lameter a écrit :
> On Thu, 3 Sep 2009, Paul E. McKenney wrote:
>
>> 2. CPU 0 discovers that the slab cache can now be destroyed.
>>
>> It determines that there are no users, and has guaranteed
>> that there will be no future users. So it knows that it
>> can safely do kmem_cache_destroy().
>>
>> 3. In absence of rcu_barrier(), kmem_cache_destroy() would
>> immediately tear down the slab data structures.
>
> Of course. This has been discussed before.
>
> You need to ensure that no objects are in use before destroying a slab. In
> case of DESTROY_BY_RCU you must ensure that there are no potential
> readers. So use a suitable rcu barrier or something else like a
> synchronize_rcu...
>
>>> But going through the RCU period is pointless since no user of the cache
>>> remains.
>> Which is irrelevant. The outstanding RCU callback was posted by the
>> slab cache itself, -not- by the user of the slab cache.
>
> There will be no rcu callbacks generated at kmem_cache_destroy with the
> patch I posted.
>
>>> The dismantling does not need RCU since there are no operations on the
>>> objects in progress. So simply switch DESTROY_BY_RCU off for close.
>> Unless I am missing something, this patch re-introduces the bug that
>> the rcu_barrier() was added to prevent. So, in absence of a better
>> explanation of what I am missing:
>
> The "fix" was ill advised. Slab users must ensure that no objects are in
> use before destroying a slab. Only the slab users know how the objects
> are being used. The slab allocator itself cannot know how to ensure that
> there are no pending references. Putting a rcu_barrier in there creates an
> inconsistency in the operation of kmem_cache_destroy() and an expectation
> of functionality that the function cannot provide.
>
Problem is not _objects_ Christoph, but _slabs_, and your patch is not working.
Its true that when User calls kmem_cache_destroy(), all _objects_ were previously freed.
This is mandatory, with or withou SLAB_DESTROY_BY_RCU thing
Problem is that slub has some internal state, including some to-be-freed _slabs_,
that User have no control at all on it.
User cannot "know" slabs are freed, inuse, or whatever state in cache or call_rcu queues.
Face it, SLAB_DESTROY_BY_RCU is internal affair (to slub/slab/... allocators)
We absolutely need a rcu_barrier() somewhere, believe it or not. You can argue that it should
be done *before*, but it gives no speedup, only potential bugs.
Only case User should do its rcu_barrier() itself is if it knows some call_rcu() are pending
and are delaying _objects_ freeing (typical !SLAB_DESTROY_RCU usage in RCU algos).
I dont even understand why you care so much about kmem_cache_destroy(SLAB_DESTROY_BY_RCU),
given that almost nobody use it. We took almost one month to find out what the bug was in first
place...
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
WARNING: multiple messages have this Message-ID (diff)
From: Eric Dumazet <eric.dumazet@gmail.com>
To: Christoph Lameter <cl@linux-foundation.org>
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
Pekka Enberg <penberg@cs.helsinki.fi>,
Zdenek Kabelac <zdenek.kabelac@gmail.com>,
Patrick McHardy <kaber@trash.net>, Robin Holt <holt@sgi.com>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
Jesper Dangaard Brouer <hawk@comx.dk>,
Linux Netdev List <netdev@vger.kernel.org>,
Netfilter Developers <netfilter-devel@vger.kernel.org>
Subject: Re: [PATCH] slub: fix slab_pad_check()
Date: Fri, 04 Sep 2009 00:08:42 +0200 [thread overview]
Message-ID: <4AA03E6A.7070800@gmail.com> (raw)
In-Reply-To: <alpine.DEB.1.10.0909031736340.24199@V090114053VZO-1>
Christoph Lameter a écrit :
> On Thu, 3 Sep 2009, Paul E. McKenney wrote:
>
>> 2. CPU 0 discovers that the slab cache can now be destroyed.
>>
>> It determines that there are no users, and has guaranteed
>> that there will be no future users. So it knows that it
>> can safely do kmem_cache_destroy().
>>
>> 3. In absence of rcu_barrier(), kmem_cache_destroy() would
>> immediately tear down the slab data structures.
>
> Of course. This has been discussed before.
>
> You need to ensure that no objects are in use before destroying a slab. In
> case of DESTROY_BY_RCU you must ensure that there are no potential
> readers. So use a suitable rcu barrier or something else like a
> synchronize_rcu...
>
>>> But going through the RCU period is pointless since no user of the cache
>>> remains.
>> Which is irrelevant. The outstanding RCU callback was posted by the
>> slab cache itself, -not- by the user of the slab cache.
>
> There will be no rcu callbacks generated at kmem_cache_destroy with the
> patch I posted.
>
>>> The dismantling does not need RCU since there are no operations on the
>>> objects in progress. So simply switch DESTROY_BY_RCU off for close.
>> Unless I am missing something, this patch re-introduces the bug that
>> the rcu_barrier() was added to prevent. So, in absence of a better
>> explanation of what I am missing:
>
> The "fix" was ill advised. Slab users must ensure that no objects are in
> use before destroying a slab. Only the slab users know how the objects
> are being used. The slab allocator itself cannot know how to ensure that
> there are no pending references. Putting a rcu_barrier in there creates an
> inconsistency in the operation of kmem_cache_destroy() and an expectation
> of functionality that the function cannot provide.
>
Problem is not _objects_ Christoph, but _slabs_, and your patch is not working.
Its true that when User calls kmem_cache_destroy(), all _objects_ were previously freed.
This is mandatory, with or withou SLAB_DESTROY_BY_RCU thing
Problem is that slub has some internal state, including some to-be-freed _slabs_,
that User have no control at all on it.
User cannot "know" slabs are freed, inuse, or whatever state in cache or call_rcu queues.
Face it, SLAB_DESTROY_BY_RCU is internal affair (to slub/slab/... allocators)
We absolutely need a rcu_barrier() somewhere, believe it or not. You can argue that it should
be done *before*, but it gives no speedup, only potential bugs.
Only case User should do its rcu_barrier() itself is if it knows some call_rcu() are pending
and are delaying _objects_ freeing (typical !SLAB_DESTROY_RCU usage in RCU algos).
I dont even understand why you care so much about kmem_cache_destroy(SLAB_DESTROY_BY_RCU),
given that almost nobody use it. We took almost one month to find out what the bug was in first
place...
next prev parent reply other threads:[~2009-09-03 22:09 UTC|newest]
Thread overview: 71+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-08-11 12:42 System freeze on reboot - general protection fault Zdenek Kabelac
2009-08-11 14:34 ` Christoph Lameter
2009-08-11 14:52 ` Zdenek Kabelac
2009-08-11 15:03 ` Christoph Lameter
2009-08-11 15:32 ` Zdenek Kabelac
2009-08-11 15:48 ` Robin Holt
2009-08-11 21:10 ` Zdenek Kabelac
2009-08-12 22:16 ` Zdenek Kabelac
2009-08-12 22:21 ` Christoph Lameter
2009-08-13 17:09 ` Zdenek Kabelac
2009-08-14 9:33 ` Zdenek Kabelac
2009-08-16 9:16 ` Eric Dumazet
2009-08-17 14:03 ` Patrick McHardy
2009-09-02 21:45 ` Zdenek Kabelac
2009-09-02 22:17 ` Eric Dumazet
2009-09-02 22:31 ` Zdenek Kabelac
2009-09-03 1:04 ` [PATCH] slub: fix slab_pad_check() and SLAB_DESTROY_BY_RCU Eric Dumazet
2009-09-03 6:31 ` Pekka Enberg
2009-09-03 6:31 ` Pekka Enberg
2009-09-03 7:38 ` Eric Dumazet
2009-09-03 7:38 ` Eric Dumazet
2009-09-03 7:51 ` Pekka Enberg
2009-09-03 17:50 ` Christoph Lameter
2009-09-03 14:05 ` Pekka Enberg
2009-09-03 14:18 ` [PATCH] slub: Fix kmem_cache_destroy() with SLAB_DESTROY_BY_RCU Eric Dumazet
2009-09-03 19:48 ` Pekka Enberg
2009-09-03 19:56 ` Eric Dumazet
2009-09-03 19:56 ` Eric Dumazet
2009-09-03 17:45 ` [PATCH] slub: fix slab_pad_check() and SLAB_DESTROY_BY_RCU Christoph Lameter
2009-09-03 14:08 ` [PATCH] slub: fix slab_pad_check() Eric Dumazet
2009-09-03 18:38 ` Christoph Lameter
2009-09-03 15:01 ` Paul E. McKenney
2009-09-03 15:02 ` Eric Dumazet
2009-09-03 19:24 ` Christoph Lameter
2009-09-03 17:44 ` Paul E. McKenney
2009-09-03 22:43 ` Christoph Lameter
2009-09-03 22:03 ` Paul E. McKenney
2009-09-04 15:33 ` Christoph Lameter
2009-09-03 22:08 ` Eric Dumazet [this message]
2009-09-03 22:08 ` Eric Dumazet
2009-09-03 22:17 ` Eric Dumazet
2009-09-04 15:39 ` Christoph Lameter
2009-09-04 20:42 ` Paul E. McKenney
2009-09-04 20:42 ` Paul E. McKenney
2009-09-04 15:38 ` Christoph Lameter
2009-09-03 17:59 ` Eric Dumazet
2009-09-03 17:59 ` Eric Dumazet
2009-09-03 19:00 ` Pekka Enberg
2009-09-03 22:44 ` Christoph Lameter
2009-09-03 23:17 ` Paul E. McKenney
2009-09-04 15:42 ` Christoph Lameter
2009-09-04 20:43 ` Paul E. McKenney
2009-09-08 19:57 ` Christoph Lameter
2009-09-08 22:20 ` Paul E. McKenney
2009-09-08 22:41 ` Christoph Lameter
2009-09-08 22:59 ` Paul E. McKenney
2009-09-09 14:04 ` Christoph Lameter
2009-09-09 14:42 ` Paul E. McKenney
2009-09-09 14:53 ` Christoph Lameter
2009-09-09 15:09 ` Paul E. McKenney
2009-09-03 19:34 ` Pekka Enberg
2009-09-03 15:00 ` [PATCH] slub: fix slab_pad_check() and SLAB_DESTROY_BY_RCU Paul E. McKenney
2009-09-03 13:42 ` Paul E. McKenney
2009-09-03 13:42 ` Paul E. McKenney
2009-09-03 13:28 ` Zdenek Kabelac
2009-09-03 13:46 ` Eric Dumazet
2009-09-03 13:46 ` Eric Dumazet
2009-09-03 14:35 ` Zdenek Kabelac
2009-09-03 14:35 ` Zdenek Kabelac
2009-09-03 18:17 ` System freeze on reboot - general protection fault Paul E. McKenney
2009-09-03 18:17 ` Paul E. McKenney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4AA03E6A.7070800@gmail.com \
--to=eric.dumazet@gmail.com \
--cc=cl@linux-foundation.org \
--cc=hawk@comx.dk \
--cc=holt@sgi.com \
--cc=kaber@trash.net \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=netfilter-devel@vger.kernel.org \
--cc=paulmck@linux.vnet.ibm.com \
--cc=penberg@cs.helsinki.fi \
--cc=zdenek.kabelac@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.