Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to allow consolidation

[Peter Zijlstra <peterz@infradead.org>]

On Wed, Oct 07, 2015 at 07:33:25AM -0700, Paul E. McKenney wrote:
> > I'm sure you know what that means, but I've no clue ;-) That is, I
> > wouldn't know where to start looking in the RCU implementation to verify
> > the barrier is either needed or sufficient. Unless you mean _everywhere_
> > :-)
> 
> Pretty much everywhere.
> 
> Let's take the usual RCU removal pattern as an example:
> 
> 	void f1(struct foo *p)
> 	{
> 		list_del_rcu(p);
> 		synchronize_rcu_expedited();
> 		kfree(p);
> 	}
> 
> 	void f2(void)
> 	{
> 		struct foo *p;
> 
> 		list_for_each_entry_rcu(p, &my_head, next)
> 			do_something_with(p);
> 	}
> 
> So the synchronize_rcu_expedited() acts as an extremely heavyweight
> memory barrier that pairs with the rcu_dereference() inside of
> list_for_each_entry_rcu().  Easy enough, right?
> 
> But what exactly within synchronize_rcu_expedited() provides the
> ordering?  The answer is a web of lock-based critical sections and
> explicit memory barriers, with the one you called out as needing
> a comment being one of them.

Right, but seeing there's possible implementations of sync_rcu(_exp)*()
that do not have the whole rcu_node tree like thing, there's more to
this particular barrier than the semantics of sync_rcu().

Some implementation choice requires this barrier upgrade -- and in
another email I suggest its the whole tree thing, we need to firmly
establish the state of one level before propagating the state up etc.

Now I'm not entirely sure this is fully correct, but its the best I
could come up.