[PATCH rcu 0/9] Miscellaneous fixes for v5.18

[PATCH rcu 0/9] Miscellaneous fixes for v5.18

[Paul E. McKenney]

Hello!

This series provides miscellaneous fixes:

1.	MAINTAINERS:  Add Frederic and Neeraj to their RCU files.

2.	Fix description of kvfree_rcu(), courtesy of "Uladzislau Rezki
	(Sony)".

3.	Add mutex for rcu boost kthread spawning and affinity setting,
	courtesy of David Woodhouse.

4.	Inline __call_rcu() into call_rcu().

5.	Record work creation stack trace with interrupts enabled,
	courtesy of Zqiang.

6.	Mark writes to the rcu_segcblist structure's ->flags field.

7.	Uninline multi-use function: finish_rcuwait(), courtesy of
	Ingo Molnar.

8.	Remove __read_mostly annotations from rcu_scheduler_active
	externs, courtesy of Ingo Molnar.

9.	Replace cpumask_weight with cpumask_empty where appropriate,
	courtesy of Yury Norov.

						Thanx, Paul

------------------------------------------------------------------------

 b/MAINTAINERS                |    2 
 b/include/linux/rcupdate.h   |    2 
 b/include/linux/rcutree.h    |    2 
 b/include/linux/rcuwait.h    |    6 --
 b/kernel/rcu/rcu_segcblist.h |    4 -
 b/kernel/rcu/tree.c          |    1 
 b/kernel/rcu/tree.h          |    3 +
 b/kernel/rcu/tree_nocb.h     |    4 -
 b/kernel/rcu/tree_plugin.h   |   10 +++-
 b/kernel/rcu/update.c        |    7 +++
 include/linux/rcupdate.h     |    2 
 kernel/rcu/tree.c            |   93 +++++++++++++++++++------------------------
 kernel/rcu/tree_plugin.h     |    2 
 13 files changed, 73 insertions(+), 65 deletions(-)

[PATCH rcu 1/9] MAINTAINERS: Add Frederic and Neeraj to their RCU files

[Paul E. McKenney]

Adding Frederic as an RCU maintainer for kernel/rcu/tree_nocb.h given his
work with offloading and de-offloading callbacks from CPUs.  Also adding
Neeraj for kernel/rcu/tasks.h given his focused work on RCU Tasks Trace.
As in I am reasonably certain that each understands the full contents
of the corresponding file.

Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
Cc: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
 MAINTAINERS | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index ea3e6c9143848..5ad49de532b1a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -16298,6 +16298,8 @@ F:	tools/testing/selftests/resctrl/
 
 READ-COPY UPDATE (RCU)
 M:	"Paul E. McKenney" <paulmck@kernel.org>
+M:	Frederic Weisbecker <frederic@kernel.org> (kernel/rcu/tree_nocb.h)
+M:	Neeraj Upadhyay <quic_neeraju@quicinc.com> (kernel/rcu/tasks.h)
 M:	Josh Triplett <josh@joshtriplett.org>
 R:	Steven Rostedt <rostedt@goodmis.org>
 R:	Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
-- 
2.31.1.189.g2e36527f23

Re: [PATCH rcu 1/9] MAINTAINERS: Add Frederic and Neeraj to their RCU files

[Frederic Weisbecker]

On Fri, Feb 04, 2022 at 03:07:57PM -0800, Paul E. McKenney wrote:
> Adding Frederic as an RCU maintainer for kernel/rcu/tree_nocb.h given his
> work with offloading and de-offloading callbacks from CPUs.  Also adding
> Neeraj for kernel/rcu/tasks.h given his focused work on RCU Tasks Trace.
> As in I am reasonably certain that each understands the full contents
> of the corresponding file.
> 
> Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
> Cc: Frederic Weisbecker <frederic@kernel.org>
> Signed-off-by: Paul E. McKenney <paulmck@kernel.org>

Acked-by: Frederic Weisbecker <frederic@kernel.org>

Thanks! :)

Re: [PATCH rcu 1/9] MAINTAINERS: Add Frederic and Neeraj to their RCU files

[Paul E. McKenney]

On Fri, Feb 11, 2022 at 04:15:29PM +0100, Frederic Weisbecker wrote:
> On Fri, Feb 04, 2022 at 03:07:57PM -0800, Paul E. McKenney wrote:
> > Adding Frederic as an RCU maintainer for kernel/rcu/tree_nocb.h given his
> > work with offloading and de-offloading callbacks from CPUs.  Also adding
> > Neeraj for kernel/rcu/tasks.h given his focused work on RCU Tasks Trace.
> > As in I am reasonably certain that each understands the full contents
> > of the corresponding file.
> > 
> > Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
> > Cc: Frederic Weisbecker <frederic@kernel.org>
> > Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
> 
> Acked-by: Frederic Weisbecker <frederic@kernel.org>
> 
> Thanks! :)

Thank -you- for all the help with RCU!

I will apply on my next rebase.

							Thanx, Paul

[PATCH rcu 2/9] rcu: Fix description of kvfree_rcu()

[Paul E. McKenney]

From: "Uladzislau Rezki (Sony)" <urezki@gmail.com>

The kvfree_rcu() header comment's description of the "ptr" parameter
is unclear, therefore rephrase it to make it clear that it is a pointer
to the memory to eventually be passed to kvfree().

Reported-by: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Uladzislau Rezki (Sony) <urezki@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
 include/linux/rcupdate.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h
index 88b42eb464068..9d7df8d36af07 100644
--- a/include/linux/rcupdate.h
+++ b/include/linux/rcupdate.h
@@ -924,7 +924,7 @@ static inline notrace void rcu_read_unlock_sched_notrace(void)
  *
  *     kvfree_rcu(ptr);
  *
- * where @ptr is a pointer to kvfree().
+ * where @ptr is the pointer to be freed by kvfree().
  *
  * Please note, head-less way of freeing is permitted to
  * use from a context that has to follow might_sleep()
-- 
2.31.1.189.g2e36527f23

[PATCH rcu 3/9] rcu: Add mutex for rcu boost kthread spawning and affinity setting

[Paul E. McKenney]

From: David Woodhouse <dwmw@amazon.co.uk>

As we handle parallel CPU bringup, we will need to take care to avoid
spawning multiple boost threads, or race conditions when setting their
affinity. Spotted by Paul McKenney.

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
 kernel/rcu/tree.c        |  1 +
 kernel/rcu/tree.h        |  3 +++
 kernel/rcu/tree_plugin.h | 10 ++++++++--
 3 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index a4c25a6283b0b..d1d1a8c51223b 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -4570,6 +4570,7 @@ static void __init rcu_init_one(void)
 			init_waitqueue_head(&rnp->exp_wq[2]);
 			init_waitqueue_head(&rnp->exp_wq[3]);
 			spin_lock_init(&rnp->exp_lock);
+			mutex_init(&rnp->boost_kthread_mutex);
 		}
 	}
 
diff --git a/kernel/rcu/tree.h b/kernel/rcu/tree.h
index 486fc901bd085..3b8b60de07c38 100644
--- a/kernel/rcu/tree.h
+++ b/kernel/rcu/tree.h
@@ -110,6 +110,9 @@ struct rcu_node {
 				/*  side effect, not as a lock. */
 	unsigned long boost_time;
 				/* When to start boosting (jiffies). */
+	struct mutex boost_kthread_mutex;
+				/* Exclusion for thread spawning and affinity */
+				/*  manipulation. */
 	struct task_struct *boost_kthread_task;
 				/* kthread that takes care of priority */
 				/*  boosting for this rcu_node structure. */
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index c5b45c2f68a15..07845dcd33c5e 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -1172,15 +1172,16 @@ static void rcu_spawn_one_boost_kthread(struct rcu_node *rnp)
 	struct sched_param sp;
 	struct task_struct *t;
 
+	mutex_lock(&rnp->boost_kthread_mutex);
 	if (rnp->boost_kthread_task || !rcu_scheduler_fully_active)
-		return;
+		goto out;
 
 	rcu_state.boost = 1;
 
 	t = kthread_create(rcu_boost_kthread, (void *)rnp,
 			   "rcub/%d", rnp_index);
 	if (WARN_ON_ONCE(IS_ERR(t)))
-		return;
+		goto out;
 
 	raw_spin_lock_irqsave_rcu_node(rnp, flags);
 	rnp->boost_kthread_task = t;
@@ -1188,6 +1189,9 @@ static void rcu_spawn_one_boost_kthread(struct rcu_node *rnp)
 	sp.sched_priority = kthread_prio;
 	sched_setscheduler_nocheck(t, SCHED_FIFO, &sp);
 	wake_up_process(t); /* get to TASK_INTERRUPTIBLE quickly. */
+
+ out:
+	mutex_unlock(&rnp->boost_kthread_mutex);
 }
 
 /*
@@ -1210,6 +1214,7 @@ static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp, int outgoingcpu)
 		return;
 	if (!zalloc_cpumask_var(&cm, GFP_KERNEL))
 		return;
+	mutex_lock(&rnp->boost_kthread_mutex);
 	for_each_leaf_node_possible_cpu(rnp, cpu)
 		if ((mask & leaf_node_cpu_bit(rnp, cpu)) &&
 		    cpu != outgoingcpu)
@@ -1218,6 +1223,7 @@ static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp, int outgoingcpu)
 	if (cpumask_weight(cm) == 0)
 		cpumask_copy(cm, housekeeping_cpumask(HK_FLAG_RCU));
 	set_cpus_allowed_ptr(t, cm);
+	mutex_unlock(&rnp->boost_kthread_mutex);
 	free_cpumask_var(cm);
 }
 
-- 
2.31.1.189.g2e36527f23

Re: [PATCH rcu 3/9] rcu: Add mutex for rcu boost kthread spawning and affinity setting

[Frederic Weisbecker]

On Fri, Feb 04, 2022 at 03:07:59PM -0800, Paul E. McKenney wrote:
> From: David Woodhouse <dwmw@amazon.co.uk>
> 
> As we handle parallel CPU bringup, we will need to take care to avoid
> spawning multiple boost threads, or race conditions when setting their
> affinity. Spotted by Paul McKenney.
> 
> Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
> Signed-off-by: Paul E. McKenney <paulmck@kernel.org>

Reviewed-by: Frederic Weisbecker <frederic@kernel.org>

Speaking of, we have:

rcu_init()
   for_each_online_cpu(cpu) // should be boot CPU only at this stage ?
       rcutree_prepare_cpu(cpu)
           rcu_spawn_one_boost_kthread(cpu)


early_initcall()
    rcu_spawn_gp_kthread()
        rcu_spawn_boost_kthreads()
	    rcu_for_each_leaf_node(rnp)
	        rcu_rnp_online_cpus(rnp) // as above, only boot CPU at this stage.
                    rcu_spawn_one_boost_kthread(cpu)

cpu_up()
    rcutree_prepare_cpu(cpu)
        rcu_spawn_one_boost_kthread(cpu)


My guess is that we could remove rcu_spawn_boost_kthreads() and simplify
rcu_init(). Something like this (untested yet):

diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 86eec6a0f1a1..da8ac2b6f8cc 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -4526,7 +4526,6 @@ static int __init rcu_spawn_gp_kthread(void)
 	raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
 	wake_up_process(t);
 	rcu_spawn_nocb_kthreads();
-	rcu_spawn_boost_kthreads();
 	rcu_spawn_core_kthreads();
 	return 0;
 }
@@ -4813,7 +4812,7 @@ static void __init kfree_rcu_batch_init(void)
 
 void __init rcu_init(void)
 {
-	int cpu;
+	int cpu = smp_processor_id();
 
 	rcu_early_boot_tests();
 
@@ -4833,11 +4832,10 @@ void __init rcu_init(void)
 	 * or the scheduler are operational.
 	 */
 	pm_notifier(rcu_pm_notify, 0);
-	for_each_online_cpu(cpu) {
-		rcutree_prepare_cpu(cpu);
-		rcu_cpu_starting(cpu);
-		rcutree_online_cpu(cpu);
-	}
+
+	rcutree_prepare_cpu(cpu);
+	rcu_cpu_starting(cpu);
+	rcutree_online_cpu(cpu);
 
 	/* Create workqueue for Tree SRCU and for expedited GPs. */
 	rcu_gp_wq = alloc_workqueue("rcu_gp", WQ_MEM_RECLAIM, 0);
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index 6082dd23408f..90925a589774 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -1226,18 +1226,6 @@ static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp, int outgoingcpu)
 	free_cpumask_var(cm);
 }
 
-/*
- * Spawn boost kthreads -- called as soon as the scheduler is running.
- */
-static void __init rcu_spawn_boost_kthreads(void)
-{
-	struct rcu_node *rnp;
-
-	rcu_for_each_leaf_node(rnp)
-		if (rcu_rnp_online_cpus(rnp))
-			rcu_spawn_one_boost_kthread(rnp);
-}
-
 #else /* #ifdef CONFIG_RCU_BOOST */
 
 static void rcu_initiate_boost(struct rcu_node *rnp, unsigned long flags)
@@ -1263,10 +1251,6 @@ static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp, int outgoingcpu)
 {
 }
 
-static void __init rcu_spawn_boost_kthreads(void)
-{
-}
-
 #endif /* #else #ifdef CONFIG_RCU_BOOST */
 
 /*

Re: [PATCH rcu 3/9] rcu: Add mutex for rcu boost kthread spawning and affinity setting

[Frederic Weisbecker]

On Fri, Feb 11, 2022 at 03:57:57PM +0100, Frederic Weisbecker wrote:
> On Fri, Feb 04, 2022 at 03:07:59PM -0800, Paul E. McKenney wrote:
> > From: David Woodhouse <dwmw@amazon.co.uk>
> > 
> > As we handle parallel CPU bringup, we will need to take care to avoid
> > spawning multiple boost threads, or race conditions when setting their
> > affinity. Spotted by Paul McKenney.
> > 
> > Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
> > Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
> 
> Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
> 
> Speaking of, we have:
> 
> rcu_init()
>    for_each_online_cpu(cpu) // should be boot CPU only at this stage ?
>        rcutree_prepare_cpu(cpu)
>            rcu_spawn_one_boost_kthread(cpu)
> 
> 
> early_initcall()
>     rcu_spawn_gp_kthread()
>         rcu_spawn_boost_kthreads()
> 	    rcu_for_each_leaf_node(rnp)
> 	        rcu_rnp_online_cpus(rnp) // as above, only boot CPU at this stage.
>                     rcu_spawn_one_boost_kthread(cpu)
> 
> cpu_up()
>     rcutree_prepare_cpu(cpu)
>         rcu_spawn_one_boost_kthread(cpu)
> 
> 
> My guess is that we could remove rcu_spawn_boost_kthreads() and simplify
> rcu_init(). Something like this (untested yet):
> 
> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
> index 86eec6a0f1a1..da8ac2b6f8cc 100644
> --- a/kernel/rcu/tree.c
> +++ b/kernel/rcu/tree.c
> @@ -4526,7 +4526,6 @@ static int __init rcu_spawn_gp_kthread(void)
>  	raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
>  	wake_up_process(t);
>  	rcu_spawn_nocb_kthreads();
> -	rcu_spawn_boost_kthreads();
>  	rcu_spawn_core_kthreads();
>  	return 0;
>  }
> @@ -4813,7 +4812,7 @@ static void __init kfree_rcu_batch_init(void)
>  
>  void __init rcu_init(void)
>  {
> -	int cpu;
> +	int cpu = smp_processor_id();
>  
>  	rcu_early_boot_tests();
>  
> @@ -4833,11 +4832,10 @@ void __init rcu_init(void)
>  	 * or the scheduler are operational.
>  	 */
>  	pm_notifier(rcu_pm_notify, 0);
> -	for_each_online_cpu(cpu) {
> -		rcutree_prepare_cpu(cpu);
> -		rcu_cpu_starting(cpu);
> -		rcutree_online_cpu(cpu);
> -	}
> +
> +	rcutree_prepare_cpu(cpu);
> +	rcu_cpu_starting(cpu);
> +	rcutree_online_cpu(cpu);
>  
>  	/* Create workqueue for Tree SRCU and for expedited GPs. */
>  	rcu_gp_wq = alloc_workqueue("rcu_gp", WQ_MEM_RECLAIM, 0);
> diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
> index 6082dd23408f..90925a589774 100644
> --- a/kernel/rcu/tree_plugin.h
> +++ b/kernel/rcu/tree_plugin.h
> @@ -1226,18 +1226,6 @@ static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp, int outgoingcpu)
>  	free_cpumask_var(cm);
>  }
>  
> -/*
> - * Spawn boost kthreads -- called as soon as the scheduler is running.
> - */
> -static void __init rcu_spawn_boost_kthreads(void)
> -{
> -	struct rcu_node *rnp;
> -
> -	rcu_for_each_leaf_node(rnp)
> -		if (rcu_rnp_online_cpus(rnp))
> -			rcu_spawn_one_boost_kthread(rnp);
> -}
> -
>  #else /* #ifdef CONFIG_RCU_BOOST */
>  
>  static void rcu_initiate_boost(struct rcu_node *rnp, unsigned long flags)
> @@ -1263,10 +1251,6 @@ static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp, int outgoingcpu)
>  {
>  }
>  
> -static void __init rcu_spawn_boost_kthreads(void)
> -{
> -}
> -
>  #endif /* #else #ifdef CONFIG_RCU_BOOST */
>  
>  /*

nocb kthread creation is similar but it depends on the gp kthread.
So we can't rely on rcu_init() -> rcu_prepare_cpu() and we must keep
the early_initcall() -> rcu_spawn_gp_kthread().

That would become (untested again):

diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index da8ac2b6f8cc..9284625a9a50 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -4525,7 +4525,7 @@ static int __init rcu_spawn_gp_kthread(void)
 	smp_store_release(&rcu_state.gp_kthread, t);  /* ^^^ */
 	raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
 	wake_up_process(t);
-	rcu_spawn_nocb_kthreads();
+	rcu_spawn_cpu_nocb_kthread(smp_processor_id());
 	rcu_spawn_core_kthreads();
 	return 0;
 }
diff --git a/kernel/rcu/tree_nocb.h b/kernel/rcu/tree_nocb.h
index 636d0546a4e9..711f6eb7f7e1 100644
--- a/kernel/rcu/tree_nocb.h
+++ b/kernel/rcu/tree_nocb.h
@@ -1277,22 +1277,6 @@ static void rcu_spawn_cpu_nocb_kthread(int cpu)
 	WRITE_ONCE(rdp->nocb_gp_kthread, rdp_gp->nocb_gp_kthread);
 }
 
-/*
- * Once the scheduler is running, spawn rcuo kthreads for all online
- * no-CBs CPUs.  This assumes that the early_initcall()s happen before
- * non-boot CPUs come online -- if this changes, we will need to add
- * some mutual exclusion.
- */
-static void __init rcu_spawn_nocb_kthreads(void)
-{
-	int cpu;
-
-	if (rcu_nocb_is_setup) {
-		for_each_online_cpu(cpu)
-			rcu_spawn_cpu_nocb_kthread(cpu);
-	}
-}
-
 /* How many CB CPU IDs per GP kthread?  Default of -1 for sqrt(nr_cpu_ids). */
 static int rcu_nocb_gp_stride = -1;
 module_param(rcu_nocb_gp_stride, int, 0444);
@@ -1549,10 +1533,6 @@ static void rcu_spawn_cpu_nocb_kthread(int cpu)
 {
 }
 
-static void __init rcu_spawn_nocb_kthreads(void)
-{
-}
-
 static void show_rcu_nocb_state(struct rcu_data *rdp)
 {
 }

Re: [PATCH rcu 3/9] rcu: Add mutex for rcu boost kthread spawning and affinity setting

[Paul E. McKenney]

On Fri, Feb 11, 2022 at 04:11:48PM +0100, Frederic Weisbecker wrote:
> On Fri, Feb 11, 2022 at 03:57:57PM +0100, Frederic Weisbecker wrote:
> > On Fri, Feb 04, 2022 at 03:07:59PM -0800, Paul E. McKenney wrote:
> > > From: David Woodhouse <dwmw@amazon.co.uk>
> > > 
> > > As we handle parallel CPU bringup, we will need to take care to avoid
> > > spawning multiple boost threads, or race conditions when setting their
> > > affinity. Spotted by Paul McKenney.
> > > 
> > > Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
> > > Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
> > 
> > Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
> > 
> > Speaking of, we have:
> > 
> > rcu_init()
> >    for_each_online_cpu(cpu) // should be boot CPU only at this stage ?
> >        rcutree_prepare_cpu(cpu)
> >            rcu_spawn_one_boost_kthread(cpu)
> > 
> > 
> > early_initcall()
> >     rcu_spawn_gp_kthread()
> >         rcu_spawn_boost_kthreads()
> > 	    rcu_for_each_leaf_node(rnp)
> > 	        rcu_rnp_online_cpus(rnp) // as above, only boot CPU at this stage.
> >                     rcu_spawn_one_boost_kthread(cpu)
> > 
> > cpu_up()
> >     rcutree_prepare_cpu(cpu)
> >         rcu_spawn_one_boost_kthread(cpu)
> > 
> > 
> > My guess is that we could remove rcu_spawn_boost_kthreads() and simplify
> > rcu_init(). Something like this (untested yet):

If you also add a WARN_ON() for more than one CPU being online at
rcu_init() time, I am good with this approach.  The main danger that
the pre-rcu_init() portion of the boot time becomes the long straw in
the eternal quest to make systems boot faster, but it is not hard to
put it back.  Hence the WARN_ON() to make it clear that adjustment
is needed.  ;-)

							Thanx, Paul

> > diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
> > index 86eec6a0f1a1..da8ac2b6f8cc 100644
> > --- a/kernel/rcu/tree.c
> > +++ b/kernel/rcu/tree.c
> > @@ -4526,7 +4526,6 @@ static int __init rcu_spawn_gp_kthread(void)
> >  	raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
> >  	wake_up_process(t);
> >  	rcu_spawn_nocb_kthreads();
> > -	rcu_spawn_boost_kthreads();
> >  	rcu_spawn_core_kthreads();
> >  	return 0;
> >  }
> > @@ -4813,7 +4812,7 @@ static void __init kfree_rcu_batch_init(void)
> >  
> >  void __init rcu_init(void)
> >  {
> > -	int cpu;
> > +	int cpu = smp_processor_id();
> >  
> >  	rcu_early_boot_tests();
> >  
> > @@ -4833,11 +4832,10 @@ void __init rcu_init(void)
> >  	 * or the scheduler are operational.
> >  	 */
> >  	pm_notifier(rcu_pm_notify, 0);
> > -	for_each_online_cpu(cpu) {
> > -		rcutree_prepare_cpu(cpu);
> > -		rcu_cpu_starting(cpu);
> > -		rcutree_online_cpu(cpu);
> > -	}
> > +
> > +	rcutree_prepare_cpu(cpu);
> > +	rcu_cpu_starting(cpu);
> > +	rcutree_online_cpu(cpu);
> >  
> >  	/* Create workqueue for Tree SRCU and for expedited GPs. */
> >  	rcu_gp_wq = alloc_workqueue("rcu_gp", WQ_MEM_RECLAIM, 0);
> > diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
> > index 6082dd23408f..90925a589774 100644
> > --- a/kernel/rcu/tree_plugin.h
> > +++ b/kernel/rcu/tree_plugin.h
> > @@ -1226,18 +1226,6 @@ static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp, int outgoingcpu)
> >  	free_cpumask_var(cm);
> >  }
> >  
> > -/*
> > - * Spawn boost kthreads -- called as soon as the scheduler is running.
> > - */
> > -static void __init rcu_spawn_boost_kthreads(void)
> > -{
> > -	struct rcu_node *rnp;
> > -
> > -	rcu_for_each_leaf_node(rnp)
> > -		if (rcu_rnp_online_cpus(rnp))
> > -			rcu_spawn_one_boost_kthread(rnp);
> > -}
> > -
> >  #else /* #ifdef CONFIG_RCU_BOOST */
> >  
> >  static void rcu_initiate_boost(struct rcu_node *rnp, unsigned long flags)
> > @@ -1263,10 +1251,6 @@ static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp, int outgoingcpu)
> >  {
> >  }
> >  
> > -static void __init rcu_spawn_boost_kthreads(void)
> > -{
> > -}
> > -
> >  #endif /* #else #ifdef CONFIG_RCU_BOOST */
> >  
> >  /*
> 
> nocb kthread creation is similar but it depends on the gp kthread.
> So we can't rely on rcu_init() -> rcu_prepare_cpu() and we must keep
> the early_initcall() -> rcu_spawn_gp_kthread().
> 
> That would become (untested again):
> 
> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
> index da8ac2b6f8cc..9284625a9a50 100644
> --- a/kernel/rcu/tree.c
> +++ b/kernel/rcu/tree.c
> @@ -4525,7 +4525,7 @@ static int __init rcu_spawn_gp_kthread(void)
>  	smp_store_release(&rcu_state.gp_kthread, t);  /* ^^^ */
>  	raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
>  	wake_up_process(t);
> -	rcu_spawn_nocb_kthreads();
> +	rcu_spawn_cpu_nocb_kthread(smp_processor_id());
>  	rcu_spawn_core_kthreads();
>  	return 0;
>  }
> diff --git a/kernel/rcu/tree_nocb.h b/kernel/rcu/tree_nocb.h
> index 636d0546a4e9..711f6eb7f7e1 100644
> --- a/kernel/rcu/tree_nocb.h
> +++ b/kernel/rcu/tree_nocb.h
> @@ -1277,22 +1277,6 @@ static void rcu_spawn_cpu_nocb_kthread(int cpu)
>  	WRITE_ONCE(rdp->nocb_gp_kthread, rdp_gp->nocb_gp_kthread);
>  }
>  
> -/*
> - * Once the scheduler is running, spawn rcuo kthreads for all online
> - * no-CBs CPUs.  This assumes that the early_initcall()s happen before
> - * non-boot CPUs come online -- if this changes, we will need to add
> - * some mutual exclusion.
> - */
> -static void __init rcu_spawn_nocb_kthreads(void)
> -{
> -	int cpu;
> -
> -	if (rcu_nocb_is_setup) {
> -		for_each_online_cpu(cpu)
> -			rcu_spawn_cpu_nocb_kthread(cpu);
> -	}
> -}
> -
>  /* How many CB CPU IDs per GP kthread?  Default of -1 for sqrt(nr_cpu_ids). */
>  static int rcu_nocb_gp_stride = -1;
>  module_param(rcu_nocb_gp_stride, int, 0444);
> @@ -1549,10 +1533,6 @@ static void rcu_spawn_cpu_nocb_kthread(int cpu)
>  {
>  }
>  
> -static void __init rcu_spawn_nocb_kthreads(void)
> -{
> -}
> -
>  static void show_rcu_nocb_state(struct rcu_data *rdp)
>  {
>  }

Re: [PATCH rcu 3/9] rcu: Add mutex for rcu boost kthread spawning and affinity setting

[Frederic Weisbecker]

On Fri, Feb 11, 2022 at 07:42:03AM -0800, Paul E. McKenney wrote:
> On Fri, Feb 11, 2022 at 04:11:48PM +0100, Frederic Weisbecker wrote:
> > On Fri, Feb 11, 2022 at 03:57:57PM +0100, Frederic Weisbecker wrote:
> > > On Fri, Feb 04, 2022 at 03:07:59PM -0800, Paul E. McKenney wrote:
> > > > From: David Woodhouse <dwmw@amazon.co.uk>
> > > > 
> > > > As we handle parallel CPU bringup, we will need to take care to avoid
> > > > spawning multiple boost threads, or race conditions when setting their
> > > > affinity. Spotted by Paul McKenney.
> > > > 
> > > > Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
> > > > Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
> > > 
> > > Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
> > > 
> > > Speaking of, we have:
> > > 
> > > rcu_init()
> > >    for_each_online_cpu(cpu) // should be boot CPU only at this stage ?
> > >        rcutree_prepare_cpu(cpu)
> > >            rcu_spawn_one_boost_kthread(cpu)
> > > 
> > > 
> > > early_initcall()
> > >     rcu_spawn_gp_kthread()
> > >         rcu_spawn_boost_kthreads()
> > > 	    rcu_for_each_leaf_node(rnp)
> > > 	        rcu_rnp_online_cpus(rnp) // as above, only boot CPU at this stage.
> > >                     rcu_spawn_one_boost_kthread(cpu)
> > > 
> > > cpu_up()
> > >     rcutree_prepare_cpu(cpu)
> > >         rcu_spawn_one_boost_kthread(cpu)
> > > 
> > > 
> > > My guess is that we could remove rcu_spawn_boost_kthreads() and simplify
> > > rcu_init(). Something like this (untested yet):
> 
> If you also add a WARN_ON() for more than one CPU being online at
> rcu_init() time, I am good with this approach.  The main danger that
> the pre-rcu_init() portion of the boot time becomes the long straw in
> the eternal quest to make systems boot faster, but it is not hard to
> put it back.  Hence the WARN_ON() to make it clear that adjustment
> is needed.  ;-)
> 
> 							Thanx, Paul

Make sense. On my TODO list then.

Thanks!

[PATCH rcu 4/9] rcu: Inline __call_rcu() into call_rcu()

[Paul E. McKenney]

Because __call_rcu() is invoked only by call_rcu(), this commit inlines
the former into the latter.

Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
 kernel/rcu/tree.c | 91 ++++++++++++++++++++++-------------------------
 1 file changed, 42 insertions(+), 49 deletions(-)

diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index d1d1a8c51223b..f1bb7ccc00847 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -2995,9 +2995,47 @@ static void check_cb_ovld(struct rcu_data *rdp)
 	raw_spin_unlock_rcu_node(rnp);
 }
 
-/* Helper function for call_rcu() and friends.  */
-static void
-__call_rcu(struct rcu_head *head, rcu_callback_t func)
+/**
+ * call_rcu() - Queue an RCU callback for invocation after a grace period.
+ * @head: structure to be used for queueing the RCU updates.
+ * @func: actual callback function to be invoked after the grace period
+ *
+ * The callback function will be invoked some time after a full grace
+ * period elapses, in other words after all pre-existing RCU read-side
+ * critical sections have completed.  However, the callback function
+ * might well execute concurrently with RCU read-side critical sections
+ * that started after call_rcu() was invoked.
+ *
+ * RCU read-side critical sections are delimited by rcu_read_lock()
+ * and rcu_read_unlock(), and may be nested.  In addition, but only in
+ * v5.0 and later, regions of code across which interrupts, preemption,
+ * or softirqs have been disabled also serve as RCU read-side critical
+ * sections.  This includes hardware interrupt handlers, softirq handlers,
+ * and NMI handlers.
+ *
+ * Note that all CPUs must agree that the grace period extended beyond
+ * all pre-existing RCU read-side critical section.  On systems with more
+ * than one CPU, this means that when "func()" is invoked, each CPU is
+ * guaranteed to have executed a full memory barrier since the end of its
+ * last RCU read-side critical section whose beginning preceded the call
+ * to call_rcu().  It also means that each CPU executing an RCU read-side
+ * critical section that continues beyond the start of "func()" must have
+ * executed a memory barrier after the call_rcu() but before the beginning
+ * of that RCU read-side critical section.  Note that these guarantees
+ * include CPUs that are offline, idle, or executing in user mode, as
+ * well as CPUs that are executing in the kernel.
+ *
+ * Furthermore, if CPU A invoked call_rcu() and CPU B invoked the
+ * resulting RCU callback function "func()", then both CPU A and CPU B are
+ * guaranteed to execute a full memory barrier during the time interval
+ * between the call to call_rcu() and the invocation of "func()" -- even
+ * if CPU A and CPU B are the same CPU (but again only if the system has
+ * more than one CPU).
+ *
+ * Implementation of these memory-ordering guarantees is described here:
+ * Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst.
+ */
+void call_rcu(struct rcu_head *head, rcu_callback_t func)
 {
 	static atomic_t doublefrees;
 	unsigned long flags;
@@ -3011,7 +3049,7 @@ __call_rcu(struct rcu_head *head, rcu_callback_t func)
 		/*
 		 * Probable double call_rcu(), so leak the callback.
 		 * Use rcu:rcu_callback trace event to find the previous
-		 * time callback was passed to __call_rcu().
+		 * time callback was passed to call_rcu().
 		 */
 		if (atomic_inc_return(&doublefrees) < 4) {
 			pr_err("%s(): Double-freed CB %p->%pS()!!!  ", __func__, head, head->func);
@@ -3060,51 +3098,6 @@ __call_rcu(struct rcu_head *head, rcu_callback_t func)
 		local_irq_restore(flags);
 	}
 }
-
-/**
- * call_rcu() - Queue an RCU callback for invocation after a grace period.
- * @head: structure to be used for queueing the RCU updates.
- * @func: actual callback function to be invoked after the grace period
- *
- * The callback function will be invoked some time after a full grace
- * period elapses, in other words after all pre-existing RCU read-side
- * critical sections have completed.  However, the callback function
- * might well execute concurrently with RCU read-side critical sections
- * that started after call_rcu() was invoked.
- *
- * RCU read-side critical sections are delimited by rcu_read_lock()
- * and rcu_read_unlock(), and may be nested.  In addition, but only in
- * v5.0 and later, regions of code across which interrupts, preemption,
- * or softirqs have been disabled also serve as RCU read-side critical
- * sections.  This includes hardware interrupt handlers, softirq handlers,
- * and NMI handlers.
- *
- * Note that all CPUs must agree that the grace period extended beyond
- * all pre-existing RCU read-side critical section.  On systems with more
- * than one CPU, this means that when "func()" is invoked, each CPU is
- * guaranteed to have executed a full memory barrier since the end of its
- * last RCU read-side critical section whose beginning preceded the call
- * to call_rcu().  It also means that each CPU executing an RCU read-side
- * critical section that continues beyond the start of "func()" must have
- * executed a memory barrier after the call_rcu() but before the beginning
- * of that RCU read-side critical section.  Note that these guarantees
- * include CPUs that are offline, idle, or executing in user mode, as
- * well as CPUs that are executing in the kernel.
- *
- * Furthermore, if CPU A invoked call_rcu() and CPU B invoked the
- * resulting RCU callback function "func()", then both CPU A and CPU B are
- * guaranteed to execute a full memory barrier during the time interval
- * between the call to call_rcu() and the invocation of "func()" -- even
- * if CPU A and CPU B are the same CPU (but again only if the system has
- * more than one CPU).
- *
- * Implementation of these memory-ordering guarantees is described here:
- * Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst.
- */
-void call_rcu(struct rcu_head *head, rcu_callback_t func)
-{
-	__call_rcu(head, func);
-}
 EXPORT_SYMBOL_GPL(call_rcu);
 
 
-- 
2.31.1.189.g2e36527f23

[PATCH rcu 5/9] kasan: Record work creation stack trace with interrupts enabled

[Paul E. McKenney]

From: Zqiang <qiang1.zhang@intel.com>

Recording the work creation stack trace for KASAN reports in
call_rcu() is expensive, due to unwinding the stack, but also
due to acquiring depot_lock inside stackdepot (which may be contended).
Because calling kasan_record_aux_stack_noalloc() does not require
interrupts to already be disabled, this may unnecessarily extend
the time with interrupts disabled.

Therefore, move calling kasan_record_aux_stack() before the section
with interrupts disabled.

Acked-by: Marco Elver <elver@google.com>
Signed-off-by: Zqiang <qiang1.zhang@intel.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
 kernel/rcu/tree.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index f1bb7ccc00847..ca8d7dd026eeb 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -3060,8 +3060,8 @@ void call_rcu(struct rcu_head *head, rcu_callback_t func)
 	}
 	head->func = func;
 	head->next = NULL;
-	local_irq_save(flags);
 	kasan_record_aux_stack_noalloc(head);
+	local_irq_save(flags);
 	rdp = this_cpu_ptr(&rcu_data);
 
 	/* Add the callback to our list. */
-- 
2.31.1.189.g2e36527f23

[PATCH rcu 6/9] rcu: Mark writes to the rcu_segcblist structure's ->flags field

[Paul E. McKenney]

KCSAN reports data races between the rcu_segcblist_clear_flags() and
rcu_segcblist_set_flags() functions, though misreporting the latter
as a call to rcu_segcblist_is_enabled() from call_rcu().  This commit
converts the updates of this field to WRITE_ONCE(), relying on the
resulting unmarked reads to continue to detect buggy concurrent writes
to this field.

Reported-by: Zhouyi Zhou <zhouzhouyi@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Cc: Frederic Weisbecker <frederic@kernel.org>
---
 kernel/rcu/rcu_segcblist.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/rcu/rcu_segcblist.h b/kernel/rcu/rcu_segcblist.h
index e373fbe44da5e..431cee212467d 100644
--- a/kernel/rcu/rcu_segcblist.h
+++ b/kernel/rcu/rcu_segcblist.h
@@ -56,13 +56,13 @@ static inline long rcu_segcblist_n_cbs(struct rcu_segcblist *rsclp)
 static inline void rcu_segcblist_set_flags(struct rcu_segcblist *rsclp,
 					   int flags)
 {
-	rsclp->flags |= flags;
+	WRITE_ONCE(rsclp->flags, rsclp->flags | flags);
 }
 
 static inline void rcu_segcblist_clear_flags(struct rcu_segcblist *rsclp,
 					     int flags)
 {
-	rsclp->flags &= ~flags;
+	WRITE_ONCE(rsclp->flags, rsclp->flags & ~flags);
 }
 
 static inline bool rcu_segcblist_test_flags(struct rcu_segcblist *rsclp,
-- 
2.31.1.189.g2e36527f23

[PATCH rcu 7/9] rcu: Uninline multi-use function: finish_rcuwait()

[Paul E. McKenney]

From: Ingo Molnar <mingo@kernel.org>

This is a rarely used function, so uninlining its 3 instructions
is probably a win or a wash - but the main motivation is to
make <linux/rcuwait.h> independent of task_struct details.

Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
 include/linux/rcuwait.h | 6 +-----
 kernel/rcu/update.c     | 7 +++++++
 2 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/include/linux/rcuwait.h b/include/linux/rcuwait.h
index 61c56cca95c43..8052d34da7826 100644
--- a/include/linux/rcuwait.h
+++ b/include/linux/rcuwait.h
@@ -47,11 +47,7 @@ static inline void prepare_to_rcuwait(struct rcuwait *w)
 	rcu_assign_pointer(w->task, current);
 }
 
-static inline void finish_rcuwait(struct rcuwait *w)
-{
-        rcu_assign_pointer(w->task, NULL);
-	__set_current_state(TASK_RUNNING);
-}
+extern void finish_rcuwait(struct rcuwait *w);
 
 #define rcuwait_wait_event(w, condition, state)				\
 ({									\
diff --git a/kernel/rcu/update.c b/kernel/rcu/update.c
index 156892c22bb56..df6197f0d5ce8 100644
--- a/kernel/rcu/update.c
+++ b/kernel/rcu/update.c
@@ -407,6 +407,13 @@ void __wait_rcu_gp(bool checktiny, int n, call_rcu_func_t *crcu_array,
 }
 EXPORT_SYMBOL_GPL(__wait_rcu_gp);
 
+void finish_rcuwait(struct rcuwait *w)
+{
+        rcu_assign_pointer(w->task, NULL);
+	__set_current_state(TASK_RUNNING);
+}
+EXPORT_SYMBOL_GPL(finish_rcuwait);
+
 #ifdef CONFIG_DEBUG_OBJECTS_RCU_HEAD
 void init_rcu_head(struct rcu_head *head)
 {
-- 
2.31.1.189.g2e36527f23

[PATCH rcu 8/9] rcu: Remove __read_mostly annotations from rcu_scheduler_active externs

[Paul E. McKenney]

From: Ingo Molnar <mingo@kernel.org>

Remove the __read_mostly attributes from the rcu_scheduler_active
extern declarations, because these attributes are ignored for
prototypes and we'd have to include the full <linux/cache.h> header
to gain this functionally pointless attribute defined.

Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
 include/linux/rcupdate.h | 2 +-
 include/linux/rcutree.h  | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h
index 9d7df8d36af07..e7c39c200e2b0 100644
--- a/include/linux/rcupdate.h
+++ b/include/linux/rcupdate.h
@@ -84,7 +84,7 @@ static inline int rcu_preempt_depth(void)
 
 /* Internal to kernel */
 void rcu_init(void);
-extern int rcu_scheduler_active __read_mostly;
+extern int rcu_scheduler_active;
 void rcu_sched_clock_irq(int user);
 void rcu_report_dead(unsigned int cpu);
 void rcutree_migrate_callbacks(int cpu);
diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h
index 53209d6694001..76665db179fa1 100644
--- a/include/linux/rcutree.h
+++ b/include/linux/rcutree.h
@@ -62,7 +62,7 @@ static inline void rcu_irq_exit_check_preempt(void) { }
 void exit_rcu(void);
 
 void rcu_scheduler_starting(void);
-extern int rcu_scheduler_active __read_mostly;
+extern int rcu_scheduler_active;
 void rcu_end_inkernel_boot(void);
 bool rcu_inkernel_boot_has_ended(void);
 bool rcu_is_watching(void);
-- 
2.31.1.189.g2e36527f23

[PATCH rcu 9/9] rcu: Replace cpumask_weight with cpumask_empty where appropriate

[Paul E. McKenney]

From: Yury Norov <yury.norov@gmail.com>

In some places, RCU code calls cpumask_weight() to check if any bit of a
given cpumask is set. We can do it more efficiently with cpumask_empty()
because cpumask_empty() stops traversing the cpumask as soon as it finds
first set bit, while cpumask_weight() counts all bits unconditionally.

Signed-off-by: Yury Norov <yury.norov@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
 kernel/rcu/tree_nocb.h   | 4 ++--
 kernel/rcu/tree_plugin.h | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/kernel/rcu/tree_nocb.h b/kernel/rcu/tree_nocb.h
index eeafb546a7a09..f83c7b1d61103 100644
--- a/kernel/rcu/tree_nocb.h
+++ b/kernel/rcu/tree_nocb.h
@@ -1169,7 +1169,7 @@ void __init rcu_init_nohz(void)
 	struct rcu_data *rdp;
 
 #if defined(CONFIG_NO_HZ_FULL)
-	if (tick_nohz_full_running && cpumask_weight(tick_nohz_full_mask))
+	if (tick_nohz_full_running && !cpumask_empty(tick_nohz_full_mask))
 		need_rcu_nocb_mask = true;
 #endif /* #if defined(CONFIG_NO_HZ_FULL) */
 
@@ -1348,7 +1348,7 @@ static void __init rcu_organize_nocb_kthreads(void)
  */
 void rcu_bind_current_to_nocb(void)
 {
-	if (cpumask_available(rcu_nocb_mask) && cpumask_weight(rcu_nocb_mask))
+	if (cpumask_available(rcu_nocb_mask) && !cpumask_empty(rcu_nocb_mask))
 		WARN_ON(sched_setaffinity(current->pid, rcu_nocb_mask));
 }
 EXPORT_SYMBOL_GPL(rcu_bind_current_to_nocb);
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index 07845dcd33c5e..efd0c87d2ffae 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -1220,7 +1220,7 @@ static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp, int outgoingcpu)
 		    cpu != outgoingcpu)
 			cpumask_set_cpu(cpu, cm);
 	cpumask_and(cm, cm, housekeeping_cpumask(HK_FLAG_RCU));
-	if (cpumask_weight(cm) == 0)
+	if (cpumask_empty(cm))
 		cpumask_copy(cm, housekeeping_cpumask(HK_FLAG_RCU));
 	set_cpus_allowed_ptr(t, cm);
 	mutex_unlock(&rnp->boost_kthread_mutex);
-- 
2.31.1.189.g2e36527f23

Re: [PATCH rcu 9/9] rcu: Replace cpumask_weight with cpumask_empty where appropriate

[Frederic Weisbecker]

On Fri, Feb 04, 2022 at 03:08:05PM -0800, Paul E. McKenney wrote:
> From: Yury Norov <yury.norov@gmail.com>
> 
> In some places, RCU code calls cpumask_weight() to check if any bit of a
> given cpumask is set. We can do it more efficiently with cpumask_empty()
> because cpumask_empty() stops traversing the cpumask as soon as it finds
> first set bit, while cpumask_weight() counts all bits unconditionally.
> 
> Signed-off-by: Yury Norov <yury.norov@gmail.com>
> Signed-off-by: Paul E. McKenney <paulmck@kernel.org>

Acked-by: Frederic Weisbecker <frederic@kernel.org>