Re: [PATCH] net: add a synchronize_net() in netdev_rx_handler_unregister()

[Eric Dumazet <eric.dumazet@gmail.com>]

On Fri, 2013-03-29 at 17:12 +0100, Jiri Pirko wrote:
> Fri, Mar 29, 2013 at 04:38:15PM CET, eric.dumazet@gmail.com wrote:
> >On Fri, 2013-03-29 at 16:11 +0100, Ivan Vecera wrote:
> >
> >> Erik, why doesn't help the write barrier between the assignments. It 
> >> should guarantee their orders... or not?
> >> 
> >
> >Its not enough, I wont explain here why as RCU is quite well documented
> >in Documentation/RCU
> 
> Can you point me exact paragraph? I'm unable to find that :(
> 

You need a bit of RCU history to understand the issue

rcu_assign_pointer(dev->rx_handler, NULL) is certainly not needing a
barrier _before_ setting rx_handler to NULL.

Old kernels had this rcu_assign_pointer() implementation since
commit d99c4f6b13b3149bc83703ab1493beaeaaaf8a2d 
(Remove rcu_assign_pointer() penalty for NULL pointers)

#define rcu_assign_pointer(p, v) \
	({ \
		if (!__builtin_constant_p(v) || \
		    ((v) != NULL)) \
			smp_wmb(); \
		(p) = (v); \
	})

Note that wmb() was _not_ done if v was NULL


Because of various sparse issues, commit
d322f45ceed525daa9401154590bbae3222cfefb
(rcu: Make rcu_assign_pointer() unconditionally insert a memory barrier)
removed the conditional, because RCU_INIT_POINTER() was available.

In the rx_handler/rx_handler_data, we use two pointers protected by RCU,
but we want to only test rx_hander being NULL, and avoid testing
rx_handler_data.

Nothing in RCU guarantees that two different pointers have any order.

Here is what could happen

CPU0                                      CPU1

handler = rcu_dereference(dev->rx_handler)
if (handler) {
   handler(dev, ...);

                                          dev->rx_handler = NULL;
                                          smp_wmb(); // OR NOT
                                          dev->rx_handler_data = NULL;
                                          smp_wmb(); // OR NOT
 handler(dev)
   priv_data = rcu_dereference(dev->rx_handler_data);
   x = priv_data->some_field;   // CRASH because priv_data is NULL