From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.3 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS,USER_AGENT_MUTT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4271CC43142 for ; Tue, 26 Jun 2018 18:17:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0887926ED3 for ; Tue, 26 Jun 2018 18:17:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0887926ED3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.vnet.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934038AbeFZSRn (ORCPT ); Tue, 26 Jun 2018 14:17:43 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:39192 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752427AbeFZSRj (ORCPT ); Tue, 26 Jun 2018 14:17:39 -0400 Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w5QIEaXS164672 for ; Tue, 26 Jun 2018 14:17:38 -0400 Received: from e15.ny.us.ibm.com (e15.ny.us.ibm.com [129.33.205.205]) by mx0b-001b2d01.pphosted.com with ESMTP id 2juqm2896b-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 26 Jun 2018 14:17:38 -0400 Received: from localhost by e15.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 26 Jun 2018 14:17:38 -0400 Received: from b01cxnp22033.gho.pok.ibm.com (9.57.198.23) by e15.ny.us.ibm.com (146.89.104.202) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 26 Jun 2018 14:17:33 -0400 Received: from b01ledav003.gho.pok.ibm.com (b01ledav003.gho.pok.ibm.com [9.57.199.108]) by b01cxnp22033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w5QIHWJb6947256 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Tue, 26 Jun 2018 18:17:32 GMT Received: from b01ledav003.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AAC60B2067; Tue, 26 Jun 2018 14:17:25 -0400 (EDT) Received: from b01ledav003.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8E579B2066; Tue, 26 Jun 2018 14:17:25 -0400 (EDT) Received: from paulmck-ThinkPad-W541 (unknown [9.70.82.159]) by b01ledav003.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 26 Jun 2018 14:17:25 -0400 (EDT) Received: by paulmck-ThinkPad-W541 (Postfix, from userid 1000) id 8637E16C15BE; Tue, 26 Jun 2018 11:19:37 -0700 (PDT) Date: Tue, 26 Jun 2018 11:19:37 -0700 From: "Paul E. McKenney" To: Peter Zijlstra Cc: linux-kernel@vger.kernel.org, mingo@kernel.org, jiangshanlai@gmail.com, dipankar@in.ibm.com, akpm@linux-foundation.org, mathieu.desnoyers@efficios.com, josh@joshtriplett.org, tglx@linutronix.de, rostedt@goodmis.org, dhowells@redhat.com, edumazet@google.com, fweisbec@gmail.com, oleg@redhat.com, joel@joelfernandes.org Subject: Re: [PATCH tip/core/rcu 13/22] rcu: Fix grace-period hangs due to race with CPU offline Reply-To: paulmck@linux.vnet.ibm.com References: <20180626002052.GA24146@linux.vnet.ibm.com> <20180626171048.2181-13-paulmck@linux.vnet.ibm.com> <20180626174424.GK2494@hirez.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20180626174424.GK2494@hirez.programming.kicks-ass.net> User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-GCONF: 00 x-cbid: 18062618-0068-0000-0000-0000030E87A5 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00009259; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000266; SDB=6.01052691; UDB=6.00539682; IPR=6.00830615; MB=3.00021866; MTD=3.00000008; XFM=3.00000015; UTC=2018-06-26 18:17:36 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18062618-0069-0000-0000-000044D1543D Message-Id: <20180626181937.GG3593@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2018-06-26_09:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1806210000 definitions=main-1806260204 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Jun 26, 2018 at 07:44:24PM +0200, Peter Zijlstra wrote: > On Tue, Jun 26, 2018 at 10:10:39AM -0700, Paul E. McKenney wrote: > > diff --git a/kernel/rcu/tree.h b/kernel/rcu/tree.h > > index 3def94fc9c74..6683da6e4ecc 100644 > > --- a/kernel/rcu/tree.h > > +++ b/kernel/rcu/tree.h > > @@ -363,6 +363,10 @@ struct rcu_state { > > const char *name; /* Name of structure. */ > > char abbr; /* Abbreviated name. */ > > struct list_head flavors; /* List of RCU flavors. */ > > + > > + spinlock_t ofl_lock ____cacheline_internodealigned_in_smp; > > Are you really sure you didn't mean to use ____cacheline_aligned_in_smp > ? This internode crap gives you full page alignment under certain rare > configs. When I get done consolidating, there will only be one rcu_state structure in the kernel. On the other hand, the choice of ____cacheline_internodealigned_in_smp was made a very long time ago, so this would not be a bad time to discuss the pros and cons of a change. There are six more of these in kernel/rcu/tree.h, and three of them are in rcu_node, two of them in rcu_data, and another in rcu_state. The ones in rcu_node and especially in rcu_data (which is per-CPU) would be quite a bit more painful from a memory-size viewpoint than the two in rcu_state. The initial reason for ____cacheline_internodealigned_in_smp was that some of the fields can be accessed by random CPUs, while others are used more locally, give or take our usual contention over the handling of CPU numbers. ;-) Thanx, Paul