From mboxrd@z Thu Jan 1 00:00:00 1970 From: paulmck@linux.vnet.ibm.com (Paul E. McKenney) Date: Tue, 12 Dec 2017 11:36:02 -0800 Subject: [PATCH] ARM: CPU hotplug: Delegate complete() to surviving CPU In-Reply-To: <20171212173759.GE10595@n2100.armlinux.org.uk> References: <20171212172059.GA11875@linux.vnet.ibm.com> <20171212173759.GE10595@n2100.armlinux.org.uk> Message-ID: <20171212193602.GG7829@linux.vnet.ibm.com> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org On Tue, Dec 12, 2017 at 05:37:59PM +0000, Russell King - ARM Linux wrote: > On Tue, Dec 12, 2017 at 09:20:59AM -0800, Paul E. McKenney wrote: > > The ARM implementation of arch_cpu_idle_dead() invokes complete(), but > > does so after RCU has stopped watching the outgoing CPU, which results > > in lockdep complaints because complete() invokes functions containing RCU > > readers. This patch therefore uses Thomas Gleixner's trick of delegating > > the complete() call to a surviving CPU via smp_call_function_single(). > > > > Reported-by: Peng Fan > > Reported-by: Russell King - ARM Linux > > Signed-off-by: Paul E. McKenney > > Tested-by: Tested-by: Fabio Estevam > > Cc: Russell King > > Cc: Ingo Molnar > > Cc: "Peter Zijlstra (Intel)" > > Cc: Michal Hocko > > Cc: Thomas Gleixner > > Cc: > > As I just described in response to Fabio's testing, this doesn't solve > anything if CONFIG_BL_SWITCHER is enabled. We could lose the unlock of > a spinlock in the GIC code for sending the IPI. As I already said > previously in our discussion (but I guess you just don't believe me): Sorry, Russell, but most days I don't even believe myself. So it is nothing personal, just one of the occupational hazards of being me. > "2. there's some optional locking in the GIC driver that cause problems > for the cpu dying path. > > The concensus last time around was that the IPI solution is a non- > starter, so the seven year proven-reliable solution (disregarding the > RCU warning) persists because I don't think anyone came up with a > better solution." > > Using smp_call_function_single() invokes the IPI paths. OK, another approach is to have the dying CPU simply set an in-memory flag, which a surviving CPU polls for. There are of course any number of ways of doing the polling loop. So what bad thing happens when you use that approach? Thanx, Paul From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752358AbdLLTgJ (ORCPT ); Tue, 12 Dec 2017 14:36:09 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:56556 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752209AbdLLTgI (ORCPT ); Tue, 12 Dec 2017 14:36:08 -0500 Date: Tue, 12 Dec 2017 11:36:02 -0800 From: "Paul E. McKenney" To: Russell King - ARM Linux Cc: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Peng Fan , Fabio Estevam , Ingo Molnar , "Peter Zijlstra (Intel)" , Michal Hocko , Thomas Gleixner Subject: Re: [PATCH] ARM: CPU hotplug: Delegate complete() to surviving CPU Reply-To: paulmck@linux.vnet.ibm.com References: <20171212172059.GA11875@linux.vnet.ibm.com> <20171212173759.GE10595@n2100.armlinux.org.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20171212173759.GE10595@n2100.armlinux.org.uk> User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-GCONF: 00 x-cbid: 17121219-0024-0000-0000-0000030054C7 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00008194; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000244; SDB=6.00959264; UDB=6.00485119; IPR=6.00739286; BA=6.00005738; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00018506; XFM=3.00000015; UTC=2017-12-12 19:36:05 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17121219-0025-0000-0000-0000464BE397 Message-Id: <20171212193602.GG7829@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-12-12_10:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1712120278 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Dec 12, 2017 at 05:37:59PM +0000, Russell King - ARM Linux wrote: > On Tue, Dec 12, 2017 at 09:20:59AM -0800, Paul E. McKenney wrote: > > The ARM implementation of arch_cpu_idle_dead() invokes complete(), but > > does so after RCU has stopped watching the outgoing CPU, which results > > in lockdep complaints because complete() invokes functions containing RCU > > readers. This patch therefore uses Thomas Gleixner's trick of delegating > > the complete() call to a surviving CPU via smp_call_function_single(). > > > > Reported-by: Peng Fan > > Reported-by: Russell King - ARM Linux > > Signed-off-by: Paul E. McKenney > > Tested-by: Tested-by: Fabio Estevam > > Cc: Russell King > > Cc: Ingo Molnar > > Cc: "Peter Zijlstra (Intel)" > > Cc: Michal Hocko > > Cc: Thomas Gleixner > > Cc: > > As I just described in response to Fabio's testing, this doesn't solve > anything if CONFIG_BL_SWITCHER is enabled. We could lose the unlock of > a spinlock in the GIC code for sending the IPI. As I already said > previously in our discussion (but I guess you just don't believe me): Sorry, Russell, but most days I don't even believe myself. So it is nothing personal, just one of the occupational hazards of being me. > "2. there's some optional locking in the GIC driver that cause problems > for the cpu dying path. > > The concensus last time around was that the IPI solution is a non- > starter, so the seven year proven-reliable solution (disregarding the > RCU warning) persists because I don't think anyone came up with a > better solution." > > Using smp_call_function_single() invokes the IPI paths. OK, another approach is to have the dying CPU simply set an in-memory flag, which a surviving CPU polls for. There are of course any number of ways of doing the polling loop. So what bad thing happens when you use that approach? Thanx, Paul