From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: linux-kernel@vger.kernel.org
Cc: mingo@elte.hu, laijs@cn.fujitsu.com, dipankar@in.ibm.com,
josht@linux.vnet.ibm.com, akpm@linux-foundation.org,
mathieu.desnoyers@polymtl.ca, dvhltc@us.ibm.com, niv@us.ibm.com,
tglx@linutronix.de, peterz@infradead.org, rostedt@goodmis.org,
hugh.dickins@tiscali.co.uk, benh@kernel.crashing.org,
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Subject: [PATCH -tip/core/rcu 1/6] Split hierarchical RCU initialization into boot-time and CPU-online pieces
Date: Sat, 15 Aug 2009 09:53:46 -0700 [thread overview]
Message-ID: <1250355231152-git-send-email-> (raw)
In-Reply-To: <20090815165153.GA8886@linux.vnet.ibm.com>
This patch divides the rcutree initialization into boot-time and
hotplug-time components, so that the tree data structures are guaranteed
to be fully linked at boot time regardless of what might happen in CPU
hotplug operations. This makes RCU more resilient against CPU hotplug
misbehavior (and vice versa), but more importantly, does a better job
of compartmentalizing the code.
Reported-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
LKML-Reference: http://lkml.org/lkml/2009/8/2/127
---
kernel/rcutree.c | 53 +++++++++++++++++++++++++++++++++--------------------
1 files changed, 33 insertions(+), 20 deletions(-)
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index 7717b95..f3e4327 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -1325,22 +1325,40 @@ int rcu_needs_cpu(int cpu)
}
/*
- * Initialize a CPU's per-CPU RCU data. We take this "scorched earth"
- * approach so that we don't have to worry about how long the CPU has
- * been gone, or whether it ever was online previously. We do trust the
- * ->mynode field, as it is constant for a given struct rcu_data and
- * initialized during early boot.
- *
- * Note that only one online or offline event can be happening at a given
- * time. Note also that we can accept some slop in the rsp->completed
- * access due to the fact that this CPU cannot possibly have any RCU
- * callbacks in flight yet.
+ * Do boot-time initialization of a CPU's per-CPU RCU data.
+ */
+static void __init
+rcu_boot_init_percpu_data(int cpu, struct rcu_state *rsp)
+{
+ unsigned long flags;
+ int i;
+ struct rcu_data *rdp = rsp->rda[cpu];
+ struct rcu_node *rnp = rcu_get_root(rsp);
+
+ /* Set up local state, ensuring consistent view of global state. */
+ spin_lock_irqsave(&rnp->lock, flags);
+ rdp->grpmask = 1UL << (cpu - rdp->mynode->grplo);
+ rdp->nxtlist = NULL;
+ for (i = 0; i < RCU_NEXT_SIZE; i++)
+ rdp->nxttail[i] = &rdp->nxtlist;
+ rdp->qlen = 0;
+#ifdef CONFIG_NO_HZ
+ rdp->dynticks = &per_cpu(rcu_dynticks, cpu);
+#endif /* #ifdef CONFIG_NO_HZ */
+ rdp->cpu = cpu;
+ spin_unlock_irqrestore(&rnp->lock, flags);
+}
+
+/*
+ * Initialize a CPU's per-CPU RCU data. Note that only one online or
+ * offline event can be happening at a given time. Note also that we
+ * can accept some slop in the rsp->completed access due to the fact
+ * that this CPU cannot possibly have any RCU callbacks in flight yet.
*/
static void __cpuinit
rcu_init_percpu_data(int cpu, struct rcu_state *rsp)
{
unsigned long flags;
- int i;
long lastcomp;
unsigned long mask;
struct rcu_data *rdp = rsp->rda[cpu];
@@ -1355,16 +1373,7 @@ rcu_init_percpu_data(int cpu, struct rcu_state *rsp)
rdp->qs_pending = 1; /* so set up to respond to current GP. */
rdp->beenonline = 1; /* We have now been online. */
rdp->passed_quiesc_completed = lastcomp - 1;
- rdp->grpmask = 1UL << (cpu - rdp->mynode->grplo);
- rdp->nxtlist = NULL;
- for (i = 0; i < RCU_NEXT_SIZE; i++)
- rdp->nxttail[i] = &rdp->nxtlist;
- rdp->qlen = 0;
rdp->blimit = blimit;
-#ifdef CONFIG_NO_HZ
- rdp->dynticks = &per_cpu(rcu_dynticks, cpu);
-#endif /* #ifdef CONFIG_NO_HZ */
- rdp->cpu = cpu;
spin_unlock(&rnp->lock); /* irqs remain disabled. */
/*
@@ -1539,8 +1548,12 @@ void __init __rcu_init(void)
#endif /* #ifdef CONFIG_RCU_CPU_STALL_DETECTOR */
rcu_init_one(&rcu_state);
RCU_DATA_PTR_INIT(&rcu_state, rcu_data);
+ for_each_possible_cpu(i)
+ rcu_boot_init_percpu_data(i, &rcu_state);
rcu_init_one(&rcu_bh_state);
RCU_DATA_PTR_INIT(&rcu_bh_state, rcu_bh_data);
+ for_each_possible_cpu(i)
+ rcu_boot_init_percpu_data(i, &rcu_bh_state);
for_each_online_cpu(i)
rcu_cpu_notify(&rcu_nb, CPU_UP_PREPARE, (void *)(long)i);
--
1.5.2.5
next prev parent reply other threads:[~2009-08-15 16:53 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-08-15 16:51 [PATCH -tip/core/rcu 1/6] Cleanups and fixes for RCU in face of heavy CPU-hotplug stress Paul E. McKenney
2009-08-15 16:53 ` Paul E. McKenney [this message]
2009-08-15 17:07 ` [tip:core/rcu] rcu: Split hierarchical RCU initialization into boot-time and CPU-online pieces tip-bot for Paul E. McKenney
2009-08-15 16:53 ` [PATCH -tip/core/rcu 2/6] Introduce cpu_notifier() to handle !HOTPLUG_CPU case Paul E. McKenney
2009-08-15 17:07 ` [tip:core/rcu] cpu hotplug: " tip-bot for Paul E. McKenney
2009-08-17 17:21 ` [PATCH -tip/core/rcu 2/6] " Josh Triplett
2009-08-17 18:28 ` Paul E. McKenney
2009-08-15 16:53 ` [PATCH -tip/core/rcu 3/6] Simplify RCU CPU-hotplug notification Paul E. McKenney
2009-08-15 17:07 ` [tip:core/rcu] rcu: " tip-bot for Paul E. McKenney
2009-08-20 4:02 ` [PATCH -tip/core/rcu 3/6] " Lai Jiangshan
2009-08-20 4:21 ` Paul E. McKenney
2009-08-15 16:53 ` [PATCH -tip/core/rcu 4/6] Make preemptable RCU scan all CPUs when summing RCU counters Paul E. McKenney
2009-08-15 17:07 ` [tip:core/rcu] rcu: " tip-bot for Paul E. McKenney
2009-08-15 16:53 ` [PATCH -tip/core/rcu 5/6] Make rcupreempt_trace.c look at offline CPUs Paul E. McKenney
2009-08-15 17:07 ` [tip:core/rcu] rcu: " tip-bot for Paul E. McKenney
2009-08-15 16:53 ` [PATCH -tip/core/rcu 6/6] Fix typo in rcu_irq_exit() comment header Paul E. McKenney
2009-08-15 17:00 ` Ingo Molnar
2009-08-15 17:10 ` Paul E. McKenney
2009-08-15 17:11 ` Ingo Molnar
2009-08-15 17:08 ` [tip:core/rcu] rcu: " tip-bot for Josh Triplett
2009-08-17 18:24 ` [PATCH -tip/core/rcu 1/6] Cleanups and fixes for RCU in face of heavy CPU-hotplug stress Josh Triplett
2009-08-17 19:20 ` Paul E. McKenney
2009-08-18 15:26 ` Ingo Molnar
2009-08-18 20:07 ` Paul E. McKenney
2009-08-19 6:06 ` Paul E. McKenney
2009-08-19 11:59 ` Ingo Molnar
2009-08-19 12:09 ` [tip:core/rcu] rcu: Delay rcu_barrier() wait until beginning of next CPU-hotunplug operation tip-bot for Paul E. McKenney
2009-08-19 15:24 ` [PATCH -tip/core/rcu 1/6] Cleanups and fixes for RCU in face of heavy CPU-hotplug stress Mathieu Desnoyers
2009-08-19 16:38 ` Paul E. McKenney
2009-08-19 18:10 ` Mathieu Desnoyers
2009-08-19 18:31 ` Paul E. McKenney
2009-08-20 14:03 ` Mathieu Desnoyers
2009-08-21 14:17 ` Ingo Molnar
2009-08-21 14:29 ` Steven Rostedt
2009-08-21 14:44 ` Ingo Molnar
2009-08-21 15:00 ` Mathieu Desnoyers
2009-08-21 15:37 ` Paul E. McKenney
2009-08-21 14:58 ` Mathieu Desnoyers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1250355231152-git-send-email- \
--to=paulmck@linux.vnet.ibm.com \
--cc=akpm@linux-foundation.org \
--cc=benh@kernel.crashing.org \
--cc=dipankar@in.ibm.com \
--cc=dvhltc@us.ibm.com \
--cc=hugh.dickins@tiscali.co.uk \
--cc=josht@linux.vnet.ibm.com \
--cc=laijs@cn.fujitsu.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mathieu.desnoyers@polymtl.ca \
--cc=mingo@elte.hu \
--cc=niv@us.ibm.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox