From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753318AbbGAU7W (ORCPT ); Wed, 1 Jul 2015 16:59:22 -0400 Received: from e31.co.us.ibm.com ([32.97.110.149]:54870 "EHLO e31.co.us.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752069AbbGAU7O (ORCPT ); Wed, 1 Jul 2015 16:59:14 -0400 X-Helo: d03dlp02.boulder.ibm.com X-MailFrom: paulmck@linux.vnet.ibm.com X-RcptTo: linux-kernel@vger.kernel.org Date: Wed, 1 Jul 2015 13:59:06 -0700 From: "Paul E. McKenney" To: Peter Zijlstra Cc: linux-kernel@vger.kernel.org, mingo@kernel.org, laijs@cn.fujitsu.com, dipankar@in.ibm.com, akpm@linux-foundation.org, mathieu.desnoyers@efficios.com, josh@joshtriplett.org, tglx@linutronix.de, rostedt@goodmis.org, dhowells@redhat.com, edumazet@google.com, dvhart@linux.intel.com, fweisbec@gmail.com, oleg@redhat.com, bobby.prani@gmail.com Subject: Re: [PATCH RFC tip/core/rcu 2/5] rcu: Short-circuit normal GPs via expedited GPs Message-ID: <20150701205906.GQ3717@linux.vnet.ibm.com> Reply-To: paulmck@linux.vnet.ibm.com References: <20150630214805.GA7795@linux.vnet.ibm.com> <1435700910-9104-1-git-send-email-paulmck@linux.vnet.ibm.com> <1435700910-9104-2-git-send-email-paulmck@linux.vnet.ibm.com> <20150701100352.GO19282@twins.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150701100352.GO19282@twins.programming.kicks-ass.net> User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 15070120-8236-0000-0000-00000CD7A783 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Jul 01, 2015 at 12:03:52PM +0200, Peter Zijlstra wrote: > On Tue, Jun 30, 2015 at 02:48:27PM -0700, Paul E. McKenney wrote: > > @@ -2121,17 +2137,24 @@ static int __noreturn rcu_gp_kthread(void *arg) > > TPS("fqswait")); > > rsp->gp_state = RCU_GP_WAIT_FQS; > > ret = wait_event_interruptible_timeout(rsp->gp_wq, > > - ((gf = READ_ONCE(rsp->gp_flags)) & > > - RCU_GP_FLAG_FQS) || > > - (!READ_ONCE(rnp->qsmask) && > > - !rcu_preempt_blocked_readers_cgp(rnp)), > > - j); > > + ((gf = READ_ONCE(rsp->gp_flags)) & > > + RCU_GP_FLAG_FQS) || > > + (!READ_ONCE(rnp->qsmask) && > > + !rcu_preempt_blocked_readers_cgp(rnp)) || > > + rcu_exp_gp_seq_done(rsp->exp_rsp, > > + rsp->gp_exp_snap), > > + j); > > How about using a helper function there? > > static inline bool rcu_gp_done(rsp, rnp) > { > /* Forced Quiescent State complete */ > if (READ_ONCE(rsp->gp_flags) & RCU_GP_FLAG_FQS) > return true; > > /* QS not masked and not blocked by preempted readers */ > if (!READ_ONCE(rnp->qsmask) && > !rcu_preempt_blocked_readers_cgp(rnp)) > return true; > > /* Expedited Grace Period completed */ > if (rcu_exp_gp_seq_done(rsp)) > return true; > > return false; > } > > ret = wait_event_interruptible_timeout(rsp->gp_wq, > rcu_gp_done(rsp, rnp), j); Fair point, and applies to the original as well, give or take comments and naming. Please see below for the corresponding patch. Thoughts? Thanx, Paul ------------------------------------------------------------------------ commit a8bb9abbf3690e507d61efe8144cec091ce5fb02 Author: Paul E. McKenney Date: Wed Jul 1 13:50:28 2015 -0700 rcu: Pull out wait_event*() condition into helper function The condition for the wait_event_interruptible_timeout() that waits to do the next force-quiescent-state scan is a bit ornate: ((gf = READ_ONCE(rsp->gp_flags)) & RCU_GP_FLAG_FQS) || (!READ_ONCE(rnp->qsmask) && !rcu_preempt_blocked_readers_cgp(rnp)) This commit therefore pulls this condition out into a helper function and comments its component conditions. Reported-by: Peter Zijlstra Signed-off-by: Paul E. McKenney diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 4992bfd360b6..2afb8e8c5134 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -1904,6 +1904,26 @@ static int rcu_gp_init(struct rcu_state *rsp) } /* + * Helper function for wait_event_interruptible_timeout() wakeup + * at force-quiescent-state time. + */ +static bool rcu_gp_fqs_check_wake(struct rcu_state *rsp, int *gfp) +{ + struct rcu_node *rnp = rcu_get_root(rsp); + + /* Someone like call_rcu() requested a force-quiescent-state scan. */ + *gfp = READ_ONCE(rsp->gp_flags); + if (*gfp & RCU_GP_FLAG_FQS) + return true; + + /* The current grace period has completed. */ + if (!READ_ONCE(rnp->qsmask) && !rcu_preempt_blocked_readers_cgp(rnp)) + return true; + + return false; +} + +/* * Do one round of quiescent-state forcing. */ static int rcu_gp_fqs(struct rcu_state *rsp, int fqs_state_in) @@ -2067,11 +2087,7 @@ static int __noreturn rcu_gp_kthread(void *arg) TPS("fqswait")); rsp->gp_state = RCU_GP_WAIT_FQS; ret = wait_event_interruptible_timeout(rsp->gp_wq, - ((gf = READ_ONCE(rsp->gp_flags)) & - RCU_GP_FLAG_FQS) || - (!READ_ONCE(rnp->qsmask) && - !rcu_preempt_blocked_readers_cgp(rnp)), - j); + rcu_gp_fqs_check_wake(rsp, &gf), j); rsp->gp_state = RCU_GP_DONE_FQS; /* Locking provides needed memory barriers. */ /* If grace period done, leave loop. */