From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754451AbbJGPPZ (ORCPT ); Wed, 7 Oct 2015 11:15:25 -0400 Received: from e37.co.us.ibm.com ([32.97.110.158]:44801 "EHLO e37.co.us.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751864AbbJGPPY (ORCPT ); Wed, 7 Oct 2015 11:15:24 -0400 X-IBM-Helo: d03dlp02.boulder.ibm.com X-IBM-MailFrom: paulmck@linux.vnet.ibm.com X-IBM-RcptTo: linux-kernel@vger.kernel.org Date: Wed, 7 Oct 2015 08:15:19 -0700 From: "Paul E. McKenney" To: Peter Zijlstra Cc: Mathieu Desnoyers , linux-kernel@vger.kernel.org, Ingo Molnar , Lai Jiangshan , dipankar@in.ibm.com, Andrew Morton , josh@joshtriplett.org, Thomas Gleixner , rostedt , dhowells@redhat.com, edumazet@google.com, dvhart@linux.intel.com, fweisbec@gmail.com, oleg@redhat.com, bobby prani Subject: Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to allow consolidation Message-ID: <20151007151519.GG3910@linux.vnet.ibm.com> Reply-To: paulmck@linux.vnet.ibm.com References: <20151006162907.GA12020@linux.vnet.ibm.com> <1444148977-14108-1-git-send-email-paulmck@linux.vnet.ibm.com> <1444148977-14108-2-git-send-email-paulmck@linux.vnet.ibm.com> <20151006202937.GX3604@twins.programming.kicks-ass.net> <20151006205850.GW3910@linux.vnet.ibm.com> <20151007075114.GW2881@worktop.programming.kicks-ass.net> <941567906.21756.1444207325021.JavaMail.zimbra@efficios.com> <20151007110120.GE17308@twins.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151007110120.GE17308@twins.programming.kicks-ass.net> User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 15100715-0025-0000-0000-00001DB41C8A Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Oct 07, 2015 at 01:01:20PM +0200, Peter Zijlstra wrote: > On Wed, Oct 07, 2015 at 08:42:05AM +0000, Mathieu Desnoyers wrote: > > ----- On Oct 7, 2015, at 3:51 AM, Peter Zijlstra peterz@infradead.org wrote: > > > > > On Tue, Oct 06, 2015 at 01:58:50PM -0700, Paul E. McKenney wrote: > > >> On Tue, Oct 06, 2015 at 10:29:37PM +0200, Peter Zijlstra wrote: > > >> > On Tue, Oct 06, 2015 at 09:29:21AM -0700, Paul E. McKenney wrote: > > >> > > +static void __maybe_unused rcu_report_exp_rnp(struct rcu_state *rsp, > > >> > > + struct rcu_node *rnp, bool wake) > > >> > > +{ > > >> > > + unsigned long flags; > > >> > > + unsigned long mask; > > >> > > + > > >> > > + raw_spin_lock_irqsave(&rnp->lock, flags); > > >> > > > >> > Normally we require a comment with barriers, explaining the order and > > >> > the pairing etc.. :-) > > >> > > > >> > > + smp_mb__after_unlock_lock(); > > >> > > >> Hmmmm... That is not good. > > >> > > >> Worse yet, I am missing comments on most of the pre-existing barriers > > >> of this form. > > > > > > Yes I noticed.. :/ > > > > > >> The purpose is to enforce the heavy-weight grace-period memory-ordering > > >> guarantees documented in the synchronize_sched() header comment and > > >> elsewhere. > > > > > >> They pair with anything you might use to check for violation > > >> of these guarantees, or, simiarly, any ordering that you might use when > > >> relying on these guarantees. > > > > > > I'm sure you know what that means, but I've no clue ;-) That is, I > > > wouldn't know where to start looking in the RCU implementation to verify > > > the barrier is either needed or sufficient. Unless you mean _everywhere_ > > > :-) > > > > One example is the new membarrier system call. It relies on synchronize_sched() > > to enforce this: > > That again doesn't explain which UNLOCKs with non-matching lock values > it pairs with and what particular ordering is important here. > > I'm fully well aware of what sync_sched() guarantees and how one can use > it, that is not the issue, what I'm saying is that a generic description > of sync_sched() doesn't help in figuring out WTH that barrier is for and > which other code I should also inspect. Unfortunately, the answer is "pretty much all of it". :-( The enforced ordering relies on pretty much every acquisition/release of an rcu_node structure's ->lock and all the dyntick-idle stuff, plus some explicit barriers and a few smp_load_acquire()s and smp_store_release()s. Thanx, Paul