Re: [PATCH nf] netfilter: invoke synchronize_rcu after set the _hook_ to NULL

[Pablo Neira Ayuso <pablo@netfilter.org>]

On Fri, Mar 24, 2017 at 09:01:17PM +0800, Liping Zhang wrote:
> Hi Pablo,
> 
> 2017-03-24 20:17 GMT+08:00 Pablo Neira Ayuso <pablo@netfilter.org>:
> >> --- a/net/netfilter/nfnetlink_cttimeout.c
> >> +++ b/net/netfilter/nfnetlink_cttimeout.c
> >> @@ -646,8 +646,8 @@ static void __exit cttimeout_exit(void)
> >>  #ifdef CONFIG_NF_CONNTRACK_TIMEOUT
> >>       RCU_INIT_POINTER(nf_ct_timeout_find_get_hook, NULL);
> >>       RCU_INIT_POINTER(nf_ct_timeout_put_hook, NULL);
> >> +     synchronize_rcu();
> >>  #endif /* CONFIG_NF_CONNTRACK_TIMEOUT */
> >> -     rcu_barrier();
> >
> > cttimeout relies on kfree_rcu().
> >
> > Are you sure we don't need this?
> >
> > According to:
> >
> > https://www.kernel.org/doc/Documentation/RCU/rcubarrier.txt
> >
> > "We could try placing a synchronize_rcu() in the module-exit code path,
> > but this is not sufficient."
> >
> > Then:
> >
> > "Please note that rcu_barrier() does -not- imply synchronize_rcu(), in
> > particular, if there are no RCU callbacks queued anywhere,
> > rcu_barrier() is within its rights to return immediately, without
> > waiting for a grace period to elapse."
> 
> This is because we use kfree_rcu to free the cttimeout objects. So I think
> rcu_barrier() is not needed anymore.
> 
> Quoted from https://lwn.net/Articles/433493/ :
> "And kfree_rcu() is also help for unloadable modules, kfree_rcu() does not
> queue any function which belong to the module, so a rcu_barrier() can
> be avoid when module exit."
> 
> Also from commit 9ab1544eb419 ("rcu: introduce kfree_rcu()"):
> "Many rcu callbacks functions just call kfree() on the base structure.
> These functions are trivial, but their size adds up, and furthermore
> when they are used in a kernel module, that module must invoke the
> high-latency rcu_barrier() function at module-unload time."

Right, thanks for explaining.

I think we can get this smaller: it should be possible to avoid this
synchronize_rcu() call from nf_conntrack_{register,unregister}_notifier().
These two are called from ctnetlink netns path, this patch already
adds a synchronize_rcu() spot to ctnetlink module removal which is
where the event callback can vanish.

You can just add a comment there so we don't forget about this, eg.

@@ -290,6 +290,7 @@  void nf_conntrack_unregister_notifier(struct net
*net,
        BUG_ON(notify != new);
        RCU_INIT_POINTER(net->ct.nf_conntrack_event_cb, NULL);
        mutex_unlock(&nf_ct_ecache_mutex);
+       /* synchronize_rcu() is called from ctnetlink. */
 }

What do you think?