From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: linux-kernel@vger.kernel.org
Cc: mingo@elte.hu, laijs@cn.fujitsu.com, dipankar@in.ibm.com,
akpm@linux-foundation.org, mathieu.desnoyers@polymtl.ca,
josh@joshtriplett.org, dvhltc@us.ibm.com, niv@us.ibm.com,
tglx@linutronix.de, peterz@infradead.org, rostedt@goodmis.org,
Valdis.Kletnieks@vt.edu, dhowells@redhat.com,
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Subject: [PATCH tip/core/rcu 03/11] rcu: prohibit starting new grace periods while forcing quiescent states
Date: Mon, 4 Jan 2010 15:09:02 -0800 [thread overview]
Message-ID: <126264655052-git-send-email-> (raw)
In-Reply-To: <20100104230850.GA22111@linux.vnet.ibm.com>
From: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reduce the number and variety of race conditions by prohibiting the
start of a new grace period while force_quiescent_state() is active.
A new fqs_active flag in the rcu_state structure is used to trace
whether or not force_quiescent_state() is active, and this new flag
is tested by rcu_start_gp(). If the CPU that closed out the last
grace period needs another grace period, this new grace period may
be delayed up to one scheduling-clock tick, but it will eventually
get started.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
kernel/rcutree.c | 31 +++++++++++++++++--------------
kernel/rcutree.h | 2 ++
2 files changed, 19 insertions(+), 14 deletions(-)
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index d42ad30..41688ff 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -659,7 +659,7 @@ rcu_start_gp(struct rcu_state *rsp, unsigned long flags)
struct rcu_data *rdp = rsp->rda[smp_processor_id()];
struct rcu_node *rnp = rcu_get_root(rsp);
- if (!cpu_needs_another_gp(rsp, rdp)) {
+ if (!cpu_needs_another_gp(rsp, rdp) || rsp->fqs_active) {
if (rnp->completed == rsp->completed) {
spin_unlock_irqrestore(&rnp->lock, flags);
return;
@@ -1195,6 +1195,7 @@ static void force_quiescent_state(struct rcu_state *rsp, int relaxed)
struct rcu_node *rnp = rcu_get_root(rsp);
u8 signaled;
u8 forcenow;
+ u8 gpdone;
if (!rcu_gp_in_progress(rsp))
return; /* No grace period in progress, nothing to force. */
@@ -1206,15 +1207,16 @@ static void force_quiescent_state(struct rcu_state *rsp, int relaxed)
(long)(rsp->jiffies_force_qs - jiffies) >= 0)
goto unlock_fqs_ret; /* no emergency and done recently. */
rsp->n_force_qs++;
- spin_lock(&rnp->lock);
+ spin_lock(&rnp->lock); /* irqs already disabled */
lastcomp = rsp->gpnum - 1;
signaled = rsp->signaled;
rsp->jiffies_force_qs = jiffies + RCU_JIFFIES_TILL_FORCE_QS;
if(!rcu_gp_in_progress(rsp)) {
rsp->n_force_qs_ngp++;
- spin_unlock(&rnp->lock);
+ spin_unlock(&rnp->lock); /* irqs remain disabled */
goto unlock_fqs_ret; /* no GP in progress, time updated. */
}
+ rsp->fqs_active = 1;
switch (signaled) {
case RCU_GP_IDLE:
case RCU_GP_INIT:
@@ -1223,15 +1225,16 @@ static void force_quiescent_state(struct rcu_state *rsp, int relaxed)
case RCU_SAVE_DYNTICK:
- spin_unlock(&rnp->lock);
+ spin_unlock(&rnp->lock); /* irqs remain disabled */
if (RCU_SIGNAL_INIT != RCU_SAVE_DYNTICK)
break; /* So gcc recognizes the dead code. */
/* Record dyntick-idle state. */
- if (rcu_process_dyntick(rsp, lastcomp,
- dyntick_save_progress_counter))
- goto unlock_fqs_ret;
- spin_lock(&rnp->lock);
+ gpdone = rcu_process_dyntick(rsp, lastcomp,
+ dyntick_save_progress_counter);
+ spin_lock(&rnp->lock); /* irqs already disabled */
+ if (gpdone)
+ break;
/* fall into next case. */
case RCU_SAVE_COMPLETED:
@@ -1252,17 +1255,17 @@ static void force_quiescent_state(struct rcu_state *rsp, int relaxed)
case RCU_FORCE_QS:
/* Check dyntick-idle state, send IPI to laggarts. */
- spin_unlock(&rnp->lock);
- if (rcu_process_dyntick(rsp, rsp->completed_fqs,
- rcu_implicit_dynticks_qs))
- goto unlock_fqs_ret;
+ spin_unlock(&rnp->lock); /* irqs remain disabled */
+ gpdone = rcu_process_dyntick(rsp, rsp->completed_fqs,
+ rcu_implicit_dynticks_qs);
/* Leave state in case more forcing is required. */
- spin_lock(&rnp->lock);
+ spin_lock(&rnp->lock); /* irqs already disabled */
break;
}
- spin_unlock(&rnp->lock);
+ rsp->fqs_active = 0;
+ spin_unlock(&rnp->lock); /* irqs remain disabled */
unlock_fqs_ret:
spin_unlock_irqrestore(&rsp->fqslock, flags);
}
diff --git a/kernel/rcutree.h b/kernel/rcutree.h
index d2a0046..dc386a7 100644
--- a/kernel/rcutree.h
+++ b/kernel/rcutree.h
@@ -277,6 +277,8 @@ struct rcu_state {
u8 signaled ____cacheline_internodealigned_in_smp;
/* Force QS state. */
+ u8 fqs_active; /* force_quiescent_state() */
+ /* is running. */
long gpnum; /* Current gp number. */
long completed; /* # of last completed gp. */
--
1.5.2.5
next prev parent reply other threads:[~2010-01-04 23:09 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-01-04 23:08 [PATCH tip/core/rcu 0/11] rcu: suppress GP start to simplify force_quiescent_state() Paul E. McKenney
2010-01-04 23:09 ` [PATCH tip/core/rcu 01/11] rcu: adjust force_quiescent_state() locking, step 1 Paul E. McKenney
2010-01-13 10:24 ` [tip:core/rcu] rcu: Adjust " tip-bot for Paul E. McKenney
2010-01-04 23:09 ` [PATCH tip/core/rcu 02/11] rcu: adjust force_quiescent_state() locking, step 2 Paul E. McKenney
2010-01-13 10:25 ` [tip:core/rcu] rcu: Adjust " tip-bot for Paul E. McKenney
2010-01-04 23:09 ` Paul E. McKenney [this message]
2010-01-13 10:25 ` [tip:core/rcu] rcu: Prohibit starting new grace periods while forcing quiescent states tip-bot for Paul E. McKenney
2010-01-04 23:09 ` [PATCH tip/core/rcu 04/11] rcu: eliminate local variable signaled from force_quiescent_state() Paul E. McKenney
2010-01-13 10:25 ` [tip:core/rcu] rcu: Eliminate " tip-bot for Paul E. McKenney
2010-01-04 23:09 ` [PATCH tip/core/rcu 05/11] rcu: eliminate local variable lastcomp " Paul E. McKenney
2010-01-13 10:25 ` [tip:core/rcu] rcu: Eliminate " tip-bot for Paul E. McKenney
2010-01-04 23:09 ` [PATCH tip/core/rcu 06/11] rcu: eliminate second argument of rcu_process_dyntick() Paul E. McKenney
2010-01-13 10:26 ` [tip:core/rcu] rcu: Eliminate " tip-bot for Paul E. McKenney
2010-01-04 23:09 ` [PATCH tip/core/rcu 07/11] rcu: eliminate rcu_process_dyntick() return value Paul E. McKenney
2010-01-13 10:26 ` [tip:core/rcu] rcu: Eliminate " tip-bot for Paul E. McKenney
2010-01-04 23:09 ` [PATCH tip/core/rcu 08/11] rcu: remove leg of force_quiescent_state() switch statement Paul E. McKenney
2010-01-13 10:26 ` [tip:core/rcu] rcu: Remove " tip-bot for Paul E. McKenney
2010-01-04 23:09 ` [PATCH tip/core/rcu 09/11] rcu: remove redundant grace-period check Paul E. McKenney
2010-01-13 10:26 ` [tip:core/rcu] rcu: Remove " tip-bot for Paul E. McKenney
2010-01-04 23:09 ` [PATCH tip/core/rcu 10/11] rcu: make force_quiescent_state() start grace period if needed Paul E. McKenney
2010-01-13 10:27 ` [tip:core/rcu] rcu: Make " tip-bot for Paul E. McKenney
2010-01-04 23:09 ` [PATCH tip/core/rcu 11/11] rcu: add force_quiescent_state() testing to rcutorture Paul E. McKenney
2010-01-13 10:27 ` [tip:core/rcu] rcu: Add " tip-bot for Paul E. McKenney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=126264655052-git-send-email- \
--to=paulmck@linux.vnet.ibm.com \
--cc=Valdis.Kletnieks@vt.edu \
--cc=akpm@linux-foundation.org \
--cc=dhowells@redhat.com \
--cc=dipankar@in.ibm.com \
--cc=dvhltc@us.ibm.com \
--cc=josh@joshtriplett.org \
--cc=laijs@cn.fujitsu.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mathieu.desnoyers@polymtl.ca \
--cc=mingo@elte.hu \
--cc=niv@us.ibm.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.