From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: linux-kernel@vger.kernel.org
Cc: mingo@elte.hu, laijs@cn.fujitsu.com, dipankar@in.ibm.com,
akpm@linux-foundation.org, mathieu.desnoyers@polymtl.ca,
josht@linux.vnet.ibm.com, dvhltc@us.ibm.com, niv@us.ibm.com,
tglx@linutronix.de, peterz@infradead.org, rostedt@goodmis.org,
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Subject: [PATCH tip/core/rcu 3/3] Initialize multi-level RCU grace periods holding locks
Date: Tue, 8 Sep 2009 15:54:37 -0700 [thread overview]
Message-ID: <12524504773190-git-send-email-> (raw)
In-Reply-To: <20090908225349.GA19524@linux.vnet.ibm.com>
From: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Prior implementations initialized the root and any internal nodes without
holding locks, then initialized the leaves holding locks. This is a
false economy, as the leaf nodes will usually greatly outnumber the root
and internal nodes. Acquiring locks on all nodes is conceptually much
simpler as well.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
kernel/rcutree.c | 41 ++++++++++++-----------------------------
1 files changed, 12 insertions(+), 29 deletions(-)
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index 03866bd..bca0aba 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -640,41 +640,24 @@ rcu_start_gp(struct rcu_state *rsp, unsigned long flags)
spin_lock(&rsp->onofflock); /* irqs already disabled. */
/*
- * Set the quiescent-state-needed bits in all the non-leaf RCU
- * nodes for all currently online CPUs. This operation relies
- * on the layout of the hierarchy within the rsp->node[] array.
- * Note that other CPUs will access only the leaves of the
- * hierarchy, which still indicate that no grace period is in
- * progress. In addition, we have excluded CPU-hotplug operations.
- *
- * We therefore do not need to hold any locks. Any required
- * memory barriers will be supplied by the locks guarding the
- * leaf rcu_nodes in the hierarchy.
- */
-
- rnp_end = rsp->level[NUM_RCU_LVLS - 1];
- for (rnp_cur = &rsp->node[0]; rnp_cur < rnp_end; rnp_cur++) {
- rnp_cur->qsmask = rnp_cur->qsmaskinit;
- rnp->gpnum = rsp->gpnum;
- }
-
- /*
- * Now set up the leaf nodes. Here we must be careful. First,
- * we need to hold the lock in order to exclude other CPUs, which
- * might be contending for the leaf nodes' locks. Second, as
- * soon as we initialize a given leaf node, its CPUs might run
- * up the rest of the hierarchy. We must therefore acquire locks
- * for each node that we touch during this stage. (But we still
- * are excluding CPU-hotplug operations.)
+ * Set the quiescent-state-needed bits in all the rcu_node
+ * structures for all currently online CPUs in breadth-first
+ * order, starting from the root rcu_node structure. This
+ * operation relies on the layout of the hierarchy within the
+ * rsp->node[] array. Note that other CPUs will access only
+ * the leaves of the hierarchy, which still indicate that no
+ * grace period is in progress, at least until the corresponding
+ * leaf node has been initialized. In addition, we have excluded
+ * CPU-hotplug operations.
*
* Note that the grace period cannot complete until we finish
* the initialization process, as there will be at least one
* qsmask bit set in the root node until that time, namely the
- * one corresponding to this CPU.
+ * one corresponding to this CPU, due to the fact that we have
+ * irqs disabled.
*/
rnp_end = &rsp->node[NUM_RCU_NODES];
- rnp_cur = rsp->level[NUM_RCU_LVLS - 1];
- for (; rnp_cur < rnp_end; rnp_cur++) {
+ for (rnp_cur = &rsp->node[0]; rnp_cur < rnp_end; rnp_cur++) {
spin_lock(&rnp_cur->lock); /* irqs already disabled. */
rnp_cur->qsmask = rnp_cur->qsmaskinit;
rnp->gpnum = rsp->gpnum;
--
1.5.2.5
next prev parent reply other threads:[~2009-09-08 22:54 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-09-08 22:53 [PATCH tip/core/rcu 0/3] increase rcutorture intensity and fix a couple resulting bugs Paul E. McKenney
2009-09-08 22:54 ` [PATCH tip/core/rcu 1/3] rcutorture: Occasionally delay readers enough to make RCU force_quiescent_state Paul E. McKenney
2009-09-12 7:45 ` Ingo Molnar
2009-09-12 14:36 ` Paul E. McKenney
2009-09-12 7:55 ` [tip:core/urgent] " tip-bot for Josh Triplett
2009-09-17 22:10 ` tip-bot for Josh Triplett
2009-09-08 22:54 ` [PATCH tip/core/rcu 2/3] Need to update rnp->gpnum if preemptable RCU is to be reliable Paul E. McKenney
2009-09-12 7:55 ` [tip:core/urgent] rcu: " tip-bot for Paul E. McKenney
2009-09-17 22:10 ` tip-bot for Paul E. McKenney
2009-09-08 22:54 ` Paul E. McKenney [this message]
2009-09-12 7:55 ` [tip:core/urgent] rcu: Initialize multi-level RCU grace periods holding locks tip-bot for Paul E. McKenney
2009-09-17 22:10 ` tip-bot for Paul E. McKenney
2009-09-08 23:37 ` [PATCH tip/core/rcu 0/3] increase rcutorture intensity and fix a couple resulting bugs Steven Rostedt
2009-09-09 17:43 ` Valdis.Kletnieks
2009-09-09 18:23 ` Paul E. McKenney
2009-09-12 7:46 ` Ingo Molnar
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=12524504773190-git-send-email- \
--to=paulmck@linux.vnet.ibm.com \
--cc=akpm@linux-foundation.org \
--cc=dipankar@in.ibm.com \
--cc=dvhltc@us.ibm.com \
--cc=josht@linux.vnet.ibm.com \
--cc=laijs@cn.fujitsu.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mathieu.desnoyers@polymtl.ca \
--cc=mingo@elte.hu \
--cc=niv@us.ibm.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox