From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933110AbcFBSnY (ORCPT ); Thu, 2 Jun 2016 14:43:24 -0400 Received: from e18.ny.us.ibm.com ([129.33.205.208]:51271 "EHLO e18.ny.us.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932339AbcFBSnX (ORCPT ); Thu, 2 Jun 2016 14:43:23 -0400 X-IBM-Helo: d01dlp03.pok.ibm.com X-IBM-MailFrom: paulmck@linux.vnet.ibm.com X-IBM-RcptTo: linux-kernel@vger.kernel.org Date: Thu, 2 Jun 2016 06:10:23 -0700 From: "Paul E. McKenney" To: Mark Rutland Cc: catalin.marinas@arm.com, dennis.chen@arm.com, jiangshanlai@gmail.com, josh@joshtriplett.org, linux-kernel@vger.kernel.org, mathieu.desnoyers@efficios.com, rostedt@goodmis.org, steve.capper@arm.com, will.deacon@arm.com Subject: Re: [PATCHv3] rcu: tree: correctly handle sparse possible cpus Message-ID: <20160602131023.GF5231@linux.vnet.ibm.com> Reply-To: paulmck@linux.vnet.ibm.com References: <1464000179-8808-1-git-send-email-mark.rutland@arm.com> <20160523161333.GC3825@linux.vnet.ibm.com> <20160531110314.GA4254@leverpostej> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160531110314.GA4254@leverpostej> User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 16060218-0045-0000-0000-0000045B5C98 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, May 31, 2016 at 12:03:15PM +0100, Mark Rutland wrote: > On Mon, May 23, 2016 at 09:13:33AM -0700, Paul E. McKenney wrote: > > On Mon, May 23, 2016 at 11:42:59AM +0100, Mark Rutland wrote: > > > In many cases in the RCU tree code, we iterate over the set of cpus for > > > a leaf node described by rcu_node::grplo and rcu_node::grphi, checking > > > per-cpu data for each cpu in this range. However, if the set of possible > > > cpus is sparse, some cpus described in this range are not possible, and > > > thus no per-cpu region will have been allocated (or initialised) for > > > them by the generic percpu code. > > > > > > Erroneous accesses to a per-cpu area for these !possible cpus may fault > > > or may hit other data depending on the addressed generated when the > > > erroneous per cpu offset is applied. In practice, both cases have been > > > observed on arm64 hardware (the former being silent, but detectable with > > > additional patches). > > > > > > To avoid issues resulting from this, we must iterate over the set of > > > *possible* cpus for a given leaf node. This patch add a new helper, > > > for_each_leaf_node_possible_cpu, to enable this. As iteration is often > > > intertwined with rcu_node local bitmask manipulation, a new > > > leaf_node_cpu_bit helper is added to make this simpler and more > > > consistent. The RCU tree code is made to use both of these where > > > appropriate. > > > > Thank you, Mark, queued for review and further testing. > > > > Thanx, Paul > > Thanks Paul. > > Unfortunately, it seems that in my haste to spin v3, I missed your suggested > logic to handle the !cpu_possible(rnp->grplo) case [1]. Sorry about that, > evidently I was not being sufficiently thorough. > > Would you be happy to fold that in, as per the diff below? Otherwise I can send > that as a fixup patch, or a respin the whole thing as v4, per your preference. > > I've given the below a spin on arm64 atop of -rcu/dev, with and without > RCU_BOOST and/or RCU_TRACE, and I've build-tested likewise for x86. I've > devised and tested the !cpu_possible(rnp->grplo) case by messing with the arm64 > SMP code and the RCU tree fanout. So far everything seems happy: no build > issues, UBSAN splats, or other runtime failures. > > So fingers crossed, that's the last issue with this patch blatted... Please do respin the patch -- cleaner history and easier for people to dig through. Thanx, Paul > Thanks, > Mark. > > [1] lkml.kernel.org/r/20160517190106.GJ3528@linux.vnet.ibm.com > > ---->8---- > diff --git a/kernel/rcu/tree.h b/kernel/rcu/tree.h > index dc0b7da..f714f87 100644 > --- a/kernel/rcu/tree.h > +++ b/kernel/rcu/tree.h > @@ -291,7 +291,7 @@ struct rcu_node { > * Iterate over all possible CPUs in a leaf RCU node. > */ > #define for_each_leaf_node_possible_cpu(rnp, cpu) \ > - for ((cpu) = rnp->grplo; \ > + for ((cpu) = cpumask_next(rnp->grplo - 1, cpu_possible_mask); \ > cpu <= rnp->grphi; \ > cpu = cpumask_next((cpu), cpu_possible_mask)) > >