From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: linux-kernel@vger.kernel.org
Cc: mingo@elte.hu, laijs@cn.fujitsu.com, dipankar@in.ibm.com,
akpm@linux-foundation.org, mathieu.desnoyers@polymtl.ca,
josh@joshtriplett.org, dvhltc@us.ibm.com, niv@us.ibm.com,
tglx@linutronix.de, peterz@infradead.org, rostedt@goodmis.org,
Valdis.Kletnieks@vt.edu, dhowells@redhat.com,
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Subject: [PATCH RFC tip/core/rcu 03/18] rcu: prohibit starting new grace periods while forcing quiescent states
Date: Tue, 15 Dec 2009 15:02:26 -0800 [thread overview]
Message-ID: <12609181611516-git-send-email-> (raw)
In-Reply-To: <20091215230213.GA9093@linux.vnet.ibm.com>
From: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Proposed for 2.6.34, not for inclusion.
Reduce the number and variety of race conditions by prohibiting the
start of a new grace period while force_quiescent_state() is active.
A new fqs_active flag in the rcu_state structure is used to trace
whether or not force_quiescent_state() is active, and this new flag
is tested by rcu_start_gp(). If the CPU that closed out the last
grace period needs another grace period, this new grace period may
be delayed up to one scheduling-clock tick, but it will eventually
get started.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
kernel/rcutree.c | 31 +++++++++++++++++--------------
kernel/rcutree.h | 2 ++
2 files changed, 19 insertions(+), 14 deletions(-)
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index d42ad30..41688ff 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -659,7 +659,7 @@ rcu_start_gp(struct rcu_state *rsp, unsigned long flags)
struct rcu_data *rdp = rsp->rda[smp_processor_id()];
struct rcu_node *rnp = rcu_get_root(rsp);
- if (!cpu_needs_another_gp(rsp, rdp)) {
+ if (!cpu_needs_another_gp(rsp, rdp) || rsp->fqs_active) {
if (rnp->completed == rsp->completed) {
spin_unlock_irqrestore(&rnp->lock, flags);
return;
@@ -1195,6 +1195,7 @@ static void force_quiescent_state(struct rcu_state *rsp, int relaxed)
struct rcu_node *rnp = rcu_get_root(rsp);
u8 signaled;
u8 forcenow;
+ u8 gpdone;
if (!rcu_gp_in_progress(rsp))
return; /* No grace period in progress, nothing to force. */
@@ -1206,15 +1207,16 @@ static void force_quiescent_state(struct rcu_state *rsp, int relaxed)
(long)(rsp->jiffies_force_qs - jiffies) >= 0)
goto unlock_fqs_ret; /* no emergency and done recently. */
rsp->n_force_qs++;
- spin_lock(&rnp->lock);
+ spin_lock(&rnp->lock); /* irqs already disabled */
lastcomp = rsp->gpnum - 1;
signaled = rsp->signaled;
rsp->jiffies_force_qs = jiffies + RCU_JIFFIES_TILL_FORCE_QS;
if(!rcu_gp_in_progress(rsp)) {
rsp->n_force_qs_ngp++;
- spin_unlock(&rnp->lock);
+ spin_unlock(&rnp->lock); /* irqs remain disabled */
goto unlock_fqs_ret; /* no GP in progress, time updated. */
}
+ rsp->fqs_active = 1;
switch (signaled) {
case RCU_GP_IDLE:
case RCU_GP_INIT:
@@ -1223,15 +1225,16 @@ static void force_quiescent_state(struct rcu_state *rsp, int relaxed)
case RCU_SAVE_DYNTICK:
- spin_unlock(&rnp->lock);
+ spin_unlock(&rnp->lock); /* irqs remain disabled */
if (RCU_SIGNAL_INIT != RCU_SAVE_DYNTICK)
break; /* So gcc recognizes the dead code. */
/* Record dyntick-idle state. */
- if (rcu_process_dyntick(rsp, lastcomp,
- dyntick_save_progress_counter))
- goto unlock_fqs_ret;
- spin_lock(&rnp->lock);
+ gpdone = rcu_process_dyntick(rsp, lastcomp,
+ dyntick_save_progress_counter);
+ spin_lock(&rnp->lock); /* irqs already disabled */
+ if (gpdone)
+ break;
/* fall into next case. */
case RCU_SAVE_COMPLETED:
@@ -1252,17 +1255,17 @@ static void force_quiescent_state(struct rcu_state *rsp, int relaxed)
case RCU_FORCE_QS:
/* Check dyntick-idle state, send IPI to laggarts. */
- spin_unlock(&rnp->lock);
- if (rcu_process_dyntick(rsp, rsp->completed_fqs,
- rcu_implicit_dynticks_qs))
- goto unlock_fqs_ret;
+ spin_unlock(&rnp->lock); /* irqs remain disabled */
+ gpdone = rcu_process_dyntick(rsp, rsp->completed_fqs,
+ rcu_implicit_dynticks_qs);
/* Leave state in case more forcing is required. */
- spin_lock(&rnp->lock);
+ spin_lock(&rnp->lock); /* irqs already disabled */
break;
}
- spin_unlock(&rnp->lock);
+ rsp->fqs_active = 0;
+ spin_unlock(&rnp->lock); /* irqs remain disabled */
unlock_fqs_ret:
spin_unlock_irqrestore(&rsp->fqslock, flags);
}
diff --git a/kernel/rcutree.h b/kernel/rcutree.h
index d2a0046..dc386a7 100644
--- a/kernel/rcutree.h
+++ b/kernel/rcutree.h
@@ -277,6 +277,8 @@ struct rcu_state {
u8 signaled ____cacheline_internodealigned_in_smp;
/* Force QS state. */
+ u8 fqs_active; /* force_quiescent_state() */
+ /* is running. */
long gpnum; /* Current gp number. */
long completed; /* # of last completed gp. */
--
1.5.2.5
next prev parent reply other threads:[~2009-12-15 23:03 UTC|newest]
Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-12-15 23:02 [PATCH RFC tip/core/rcu 0/18] rcu: simplify race conditions, add checking Paul E. McKenney
2009-12-15 23:02 ` [PATCH RFC tip/core/rcu 01/18] rcu: adjust force_quiescent_state() locking, step 1 Paul E. McKenney
2009-12-15 23:02 ` [PATCH RFC tip/core/rcu 02/18] rcu: adjust force_quiescent_state() locking, step 2 Paul E. McKenney
2009-12-15 23:02 ` Paul E. McKenney [this message]
2009-12-15 23:02 ` [PATCH RFC tip/core/rcu 04/18] rcu: eliminate local variable signaled from force_quiescent_state() Paul E. McKenney
2009-12-15 23:02 ` [PATCH RFC tip/core/rcu 05/18] rcu: eliminate local variable lastcomp " Paul E. McKenney
2009-12-15 23:02 ` [PATCH RFC tip/core/rcu 06/18] rcu: eliminate second argument of rcu_process_dyntick() Paul E. McKenney
2009-12-15 23:02 ` [PATCH RFC tip/core/rcu 07/18] rcu: eliminate rcu_process_dyntick() return value Paul E. McKenney
2009-12-15 23:02 ` [PATCH RFC tip/core/rcu 08/18] rcu: remove leg of force_quiescent_state() switch statement Paul E. McKenney
2009-12-15 23:02 ` [PATCH RFC tip/core/rcu 09/18] rcu: remove redundant grace-period check Paul E. McKenney
2009-12-15 23:02 ` [PATCH RFC tip/core/rcu 10/18] rcu: make force_quiescent_state() start grace period if needed Paul E. McKenney
2009-12-15 23:02 ` [PATCH RFC tip/core/rcu 11/18] rcu: add force_quiescent_state() testing to rcutorture Paul E. McKenney
2009-12-15 23:02 ` [PATCH RFC tip/core/rcu 12/18] rcu: make MAINTAINERS file match new RCU reality Paul E. McKenney
2009-12-16 0:53 ` Josh Triplett
2009-12-15 23:02 ` [PATCH RFC tip/core/rcu 13/18] rcu: add debug check for too many rcu_read_unlock() Paul E. McKenney
2009-12-15 23:02 ` [PATCH RFC tip/core/rcu 14/18] rcu: lockdep check for exiting to user space as RCU reader Paul E. McKenney
2009-12-16 10:24 ` Peter Zijlstra
2009-12-16 15:11 ` Paul E. McKenney
2009-12-15 23:02 ` [PATCH RFC tip/core/rcu 15/18] rcu: give different levels of the rcu_node hierarchy distinct lockdep names Paul E. McKenney
2009-12-16 0:59 ` Josh Triplett
2009-12-16 1:59 ` Paul E. McKenney
2009-12-16 10:26 ` Peter Zijlstra
2009-12-16 10:33 ` Peter Zijlstra
2009-12-16 15:13 ` Paul E. McKenney
2009-12-15 23:02 ` [PATCH RFC tip/core/rcu 16/18] rcu: make lockdep aware of SRCU read-side critical sections Paul E. McKenney
2009-12-15 23:02 ` [PATCH RFC tip/core/rcu 17/18] rcu: Provide different lockdep classes for each flavor of RCU Paul E. McKenney
2009-12-15 23:02 ` [PATCH RFC tip/core/rcu 18/18] rcu: add primitives to check for RCU read-side critical sections Paul E. McKenney
2009-12-16 1:04 ` Josh Triplett
2009-12-16 2:08 ` Paul E. McKenney
2009-12-16 10:31 ` Peter Zijlstra
2009-12-16 15:18 ` Paul E. McKenney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=12609181611516-git-send-email- \
--to=paulmck@linux.vnet.ibm.com \
--cc=Valdis.Kletnieks@vt.edu \
--cc=akpm@linux-foundation.org \
--cc=dhowells@redhat.com \
--cc=dipankar@in.ibm.com \
--cc=dvhltc@us.ibm.com \
--cc=josh@joshtriplett.org \
--cc=laijs@cn.fujitsu.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mathieu.desnoyers@polymtl.ca \
--cc=mingo@elte.hu \
--cc=niv@us.ibm.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.