Re: [PATCH tip/core/rcu 13/15] rcu: Introduce for_each_rcu_flavor() and use it

[Josh Triplett <josh@joshtriplett.org>]

On Fri, Jun 15, 2012 at 02:06:08PM -0700, Paul E. McKenney wrote:
> From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> 
> The arrival of TREE_PREEMPT_RCU some years back included some ugly
> code involving either #ifdef or #ifdef'ed wrapper functions to iterate
> over all non-SRCU flavors of RCU.  This commit therefore introduces
> a for_each_rcu_flavor() iterator over the rcu_state structures for each
> flavor of RCU to clean up a bit of the ugliness.
> 
> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>

Great cleanup!

A few comments below, though.

>  kernel/rcutree.c        |   53 +++++++++++++---------
>  kernel/rcutree.h        |   12 ++---
>  kernel/rcutree_plugin.h |  116 -----------------------------------------------
>  3 files changed, 36 insertions(+), 145 deletions(-)

Awesome diffstat.

> diff --git a/kernel/rcutree.c b/kernel/rcutree.c
> index bd4e41c..75ad92a 100644
> --- a/kernel/rcutree.c
> +++ b/kernel/rcutree.c
> @@ -84,6 +84,7 @@ struct rcu_state rcu_bh_state = RCU_STATE_INITIALIZER(rcu_bh, call_rcu_bh);
>  DEFINE_PER_CPU(struct rcu_data, rcu_bh_data);
>  
>  static struct rcu_state *rcu_state;
> +LIST_HEAD(rcu_struct_flavors);

Does any means exist to turn this into a constant array known at compile
time rather than a runtime linked list?  Having this as a compile-time
constant may allow the compiler to unroll for_each_rcu_flavor and
potentially inline the calls inside it.

> @@ -2539,9 +2548,10 @@ rcu_init_percpu_data(int cpu, struct rcu_state *rsp, int preemptible)
>  
>  static void __cpuinit rcu_prepare_cpu(int cpu)
>  {
> -	rcu_init_percpu_data(cpu, &rcu_sched_state, 0);
> -	rcu_init_percpu_data(cpu, &rcu_bh_state, 0);
> -	rcu_preempt_init_percpu_data(cpu);
> +	struct rcu_state *rsp;
> +
> +	for_each_rcu_flavor(rsp)
> +		rcu_init_percpu_data(cpu, rsp, 0);

This results in passing 0 as the "preemptible" parameter of
rcu_init_percpu_data, which seems wrong if the preemptible parameter has
any meaning at all. :)

> @@ -2577,18 +2588,15 @@ static int __cpuinit rcu_cpu_notify(struct notifier_block *self,
>  		 * touch any data without introducing corruption. We send the
>  		 * dying CPU's callbacks to an arbitrarily chosen online CPU.
>  		 */
> -		rcu_cleanup_dying_cpu(&rcu_bh_state);
> -		rcu_cleanup_dying_cpu(&rcu_sched_state);
> -		rcu_preempt_cleanup_dying_cpu();
> -		rcu_cleanup_after_idle(cpu);
> +		for_each_rcu_flavor(rsp)
> +			rcu_cleanup_dying_cpu(rsp);

Why did rcu_cleanup_after_idle go away here?

- Josh Triplett