Re: [RFC, PATCH] fs: push rcu_barrier() from deactivate_locked_super() to filesystems

Kirill A. Shutemov
2012-06-08T21:59:40Z

[-- Attachment #1: Type: text/plain, Size: 2528 bytes --]

On Fri, Jun 08, 2012 at 02:43:58PM -0700, Linus Torvalds wrote:
> On Fri, Jun 8, 2012 at 2:28 PM, Kirill A. Shutemov
> <kirill.shutemov@linux.intel.com> wrote:
> > From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
> >
> > There's no reason to call rcu_barrier() on every deactivate_locked_super().
> > We only need to make sure that all delayed rcu free inodes are flushed
> > before we destroy related cache.
> >
> > Removing rcu_barrier() from deactivate_locked_super() affects some
> > fas paths. E.g. on my machine exit_group() of a last process in IPC
> > namespace takes 0.07538s. rcu_barrier() takes 0.05188s of that time.
>
> I think we should just delete it.
>
> kmem_cache_destroy() (at least for SLUB) already has:
>
> if (s->flags & SLAB_DESTROY_BY_RCU)
> rcu_barrier();
>
> in it. But I think it's too late - it gets called *after* we do
> kmem_cache_close(), and I get the feeling that we should do it before.
>
> Shouldn't that be sufficient? And if other slab allocators don't have
> this, we should add it to them too.
>
> Hmm?

When I tried SLAB_DESTROY_BY_RCU I've got problem:

[ 36.687999] Pid: 3455, comm: rmmod Not tainted 3.5.0-rc1-00130-g48d212a-dirty #40
[ 36.688001] Call Trace:
[ 36.688012] [<ffffffff8113367a>] slab_err+0xaa/0xd0
[ 36.688020] [<ffffffff8113515a>] ? __kmalloc+0x10a/0x110
[ 36.688026] [<ffffffff8113647d>] kmem_cache_destroy+0x1dd/0x420
[ 36.688056] [<ffffffffa00f0f25>] btrfs_destroy_cachep+0x15/0x60 [btrfs]
[ 36.688076] [<ffffffffa013cac3>] exit_btrfs_fs+0x9/0x3a [btrfs]
[ 36.688083] [<ffffffff810c324e>] sys_delete_module+0x16e/0x2f0
[ 36.688090] [<ffffffff8128cf29>] ? lockdep_sys_exit_thunk+0x35/0x67
[ 36.688097] [<ffffffff8161eba6>] system_call_fastpath+0x1a/0x1f
[ 36.688111] Pid: 3455, comm: rmmod Not tainted 3.5.0-rc1-00130-g48d212a-dirty #40
[ 36.688114] Call Trace:
[ 36.688119] [<ffffffff811365ee>] kmem_cache_destroy+0x34e/0x420
[ 36.688143] [<ffffffffa00f0f25>] btrfs_destroy_cachep+0x15/0x60 [btrfs]
[ 36.688162] [<ffffffffa013cac3>] exit_btrfs_fs+0x9/0x3a [btrfs]
[ 36.688168] [<ffffffff810c324e>] sys_delete_module+0x16e/0x2f0
[ 36.688174] [<ffffffff8128cf29>] ? lockdep_sys_exit_thunk+0x35/0x67
[ 36.688179] [<ffffffff8161eba6>] system_call_fastpath+0x1a/0x1f

IIUC, moving rcu_barrier() up should help, but I can't say that I fully
understand SLAB_DESTROY_BY_RCU semantics.

--
Kirill A. Shutemov

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]