Re: [PATCH tip/core/rcu 13/15] rcu: Introduce for_each_rcu_flavor() and use it

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Fri, Jun 15, 2012 at 04:52:40PM -0700, Josh Triplett wrote:
> On Fri, Jun 15, 2012 at 02:06:08PM -0700, Paul E. McKenney wrote:
> > From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> > 
> > The arrival of TREE_PREEMPT_RCU some years back included some ugly
> > code involving either #ifdef or #ifdef'ed wrapper functions to iterate
> > over all non-SRCU flavors of RCU.  This commit therefore introduces
> > a for_each_rcu_flavor() iterator over the rcu_state structures for each
> > flavor of RCU to clean up a bit of the ugliness.
> > 
> > Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> 
> Great cleanup!
> 
> A few comments below, though.
> 
> >  kernel/rcutree.c        |   53 +++++++++++++---------
> >  kernel/rcutree.h        |   12 ++---
> >  kernel/rcutree_plugin.h |  116 -----------------------------------------------
> >  3 files changed, 36 insertions(+), 145 deletions(-)
> 
> Awesome diffstat.

;-)

> > diff --git a/kernel/rcutree.c b/kernel/rcutree.c
> > index bd4e41c..75ad92a 100644
> > --- a/kernel/rcutree.c
> > +++ b/kernel/rcutree.c
> > @@ -84,6 +84,7 @@ struct rcu_state rcu_bh_state = RCU_STATE_INITIALIZER(rcu_bh, call_rcu_bh);
> >  DEFINE_PER_CPU(struct rcu_data, rcu_bh_data);
> >  
> >  static struct rcu_state *rcu_state;
> > +LIST_HEAD(rcu_struct_flavors);
> 
> Does any means exist to turn this into a constant array known at compile
> time rather than a runtime linked list?  Having this as a compile-time
> constant may allow the compiler to unroll for_each_rcu_flavor and
> potentially inline the calls inside it.

I could do that, but none of the traversals is anywhere near performance
critical, and all the ways I can think of to do this are uglier than
the list.

> > @@ -2539,9 +2548,10 @@ rcu_init_percpu_data(int cpu, struct rcu_state *rsp, int preemptible)
> >  
> >  static void __cpuinit rcu_prepare_cpu(int cpu)
> >  {
> > -	rcu_init_percpu_data(cpu, &rcu_sched_state, 0);
> > -	rcu_init_percpu_data(cpu, &rcu_bh_state, 0);
> > -	rcu_preempt_init_percpu_data(cpu);
> > +	struct rcu_state *rsp;
> > +
> > +	for_each_rcu_flavor(rsp)
> > +		rcu_init_percpu_data(cpu, rsp, 0);
> 
> This results in passing 0 as the "preemptible" parameter of
> rcu_init_percpu_data, which seems wrong if the preemptible parameter has
> any meaning at all. :)

Good catch!  Hmmm...  Probably best to move this to the rcu_state
initialization.

> > @@ -2577,18 +2588,15 @@ static int __cpuinit rcu_cpu_notify(struct notifier_block *self,
> >  		 * touch any data without introducing corruption. We send the
> >  		 * dying CPU's callbacks to an arbitrarily chosen online CPU.
> >  		 */
> > -		rcu_cleanup_dying_cpu(&rcu_bh_state);
> > -		rcu_cleanup_dying_cpu(&rcu_sched_state);
> > -		rcu_preempt_cleanup_dying_cpu();
> > -		rcu_cleanup_after_idle(cpu);
> > +		for_each_rcu_flavor(rsp)
> > +			rcu_cleanup_dying_cpu(rsp);
> 
> Why did rcu_cleanup_after_idle go away here?

Because I fat-fingered it.  Thank you very much for spotting this.  It
would have been nasty to find otherwise.

							Thanx, Paul