From mboxrd@z Thu Jan 1 00:00:00 1970 From: paulmck@linux.vnet.ibm.com (Paul E. McKenney) Date: Tue, 12 Dec 2017 11:31:37 -0800 Subject: [PATCH] ARM: CPU hotplug: Delegate complete() to surviving CPU In-Reply-To: <20171212174046.gn4t3qeck3bq7scw@tarshish> References: <20171212172059.GA11875@linux.vnet.ibm.com> <20171212174046.gn4t3qeck3bq7scw@tarshish> Message-ID: <20171212193137.GF7829@linux.vnet.ibm.com> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org On Tue, Dec 12, 2017 at 07:40:46PM +0200, Baruch Siach wrote: > Hi Paul, > > On Tue, Dec 12, 2017 at 09:20:59AM -0800, Paul E. McKenney wrote: > > The ARM implementation of arch_cpu_idle_dead() invokes complete(), but > > does so after RCU has stopped watching the outgoing CPU, which results > > in lockdep complaints because complete() invokes functions containing RCU > > readers. This patch therefore uses Thomas Gleixner's trick of delegating > > the complete() call to a surviving CPU via smp_call_function_single(). > > > > Reported-by: Peng Fan > > Reported-by: Russell King - ARM Linux > > Signed-off-by: Paul E. McKenney > > Tested-by: Tested-by: Fabio Estevam > > Fabio reported only once, though he might have tested twice. Actually, he did test twice. Apparently this patch has problems with CONFIG_BL_SWITCHER=y kernels. But yes, I guess I did get a bit carried away with the Tested-by's, didn't I? ;-) Thanx, Paul > baruch > > > Cc: Russell King > > Cc: Ingo Molnar > > Cc: "Peter Zijlstra (Intel)" > > Cc: Michal Hocko > > Cc: Thomas Gleixner > > Cc: > > > > diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c > > index b4fbf00ee4ad..75f85e20aafa 100644 > > --- a/arch/arm/kernel/smp.c > > +++ b/arch/arm/kernel/smp.c > > @@ -267,6 +267,14 @@ void __cpu_die(unsigned int cpu) > > } > > > > /* > > + * Invoke complete() on behalf of the outgoing CPU. > > + */ > > +static void arch_cpu_idle_dead_complete(void *arg) > > +{ > > + complete(&cpu_died); > > +} > > + > > +/* > > * Called from the idle thread for the CPU which has been shutdown. > > * > > * Note that we disable IRQs here, but do not re-enable them > > @@ -293,9 +301,11 @@ void arch_cpu_idle_dead(void) > > /* > > * Tell __cpu_die() that this CPU is now safe to dispose of. Once > > * this returns, power and/or clocks can be removed at any point > > - * from this CPU and its cache by platform_cpu_kill(). > > + * from this CPU and its cache by platform_cpu_kill(). We cannot > > + * call complete() this late, so we delegate it to an online CPU. > > */ > > - complete(&cpu_died); > > + smp_call_function_single(cpumask_first(cpu_online_mask), > > + arch_cpu_idle_dead_complete, NULL, 0); > > > > /* > > * Ensure that the cache lines associated with that completion are > > -- > http://baruch.siach.name/blog/ ~. .~ Tk Open Systems > =}------------------------------------------------ooO--U--Ooo------------{= > - baruch at tkos.co.il - tel: +972.52.368.4656, http://www.tkos.co.il - > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752346AbdLLTbn (ORCPT ); Tue, 12 Dec 2017 14:31:43 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:50834 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751803AbdLLTbl (ORCPT ); Tue, 12 Dec 2017 14:31:41 -0500 Date: Tue, 12 Dec 2017 11:31:37 -0800 From: "Paul E. McKenney" To: Baruch Siach Cc: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Michal Hocko , "Peter Zijlstra (Intel)" , Russell King - ARM Linux , Fabio Estevam , Peng Fan , Thomas Gleixner , Ingo Molnar Subject: Re: [PATCH] ARM: CPU hotplug: Delegate complete() to surviving CPU Reply-To: paulmck@linux.vnet.ibm.com References: <20171212172059.GA11875@linux.vnet.ibm.com> <20171212174046.gn4t3qeck3bq7scw@tarshish> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20171212174046.gn4t3qeck3bq7scw@tarshish> User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-GCONF: 00 x-cbid: 17121219-2213-0000-0000-000002495C19 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00008194; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000244; SDB=6.00959263; UDB=6.00485119; IPR=6.00739285; BA=6.00005738; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00018506; XFM=3.00000015; UTC=2017-12-12 19:31:40 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17121219-2214-0000-0000-0000586C6D36 Message-Id: <20171212193137.GF7829@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-12-12_10:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1712120278 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Dec 12, 2017 at 07:40:46PM +0200, Baruch Siach wrote: > Hi Paul, > > On Tue, Dec 12, 2017 at 09:20:59AM -0800, Paul E. McKenney wrote: > > The ARM implementation of arch_cpu_idle_dead() invokes complete(), but > > does so after RCU has stopped watching the outgoing CPU, which results > > in lockdep complaints because complete() invokes functions containing RCU > > readers. This patch therefore uses Thomas Gleixner's trick of delegating > > the complete() call to a surviving CPU via smp_call_function_single(). > > > > Reported-by: Peng Fan > > Reported-by: Russell King - ARM Linux > > Signed-off-by: Paul E. McKenney > > Tested-by: Tested-by: Fabio Estevam > > Fabio reported only once, though he might have tested twice. Actually, he did test twice. Apparently this patch has problems with CONFIG_BL_SWITCHER=y kernels. But yes, I guess I did get a bit carried away with the Tested-by's, didn't I? ;-) Thanx, Paul > baruch > > > Cc: Russell King > > Cc: Ingo Molnar > > Cc: "Peter Zijlstra (Intel)" > > Cc: Michal Hocko > > Cc: Thomas Gleixner > > Cc: > > > > diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c > > index b4fbf00ee4ad..75f85e20aafa 100644 > > --- a/arch/arm/kernel/smp.c > > +++ b/arch/arm/kernel/smp.c > > @@ -267,6 +267,14 @@ void __cpu_die(unsigned int cpu) > > } > > > > /* > > + * Invoke complete() on behalf of the outgoing CPU. > > + */ > > +static void arch_cpu_idle_dead_complete(void *arg) > > +{ > > + complete(&cpu_died); > > +} > > + > > +/* > > * Called from the idle thread for the CPU which has been shutdown. > > * > > * Note that we disable IRQs here, but do not re-enable them > > @@ -293,9 +301,11 @@ void arch_cpu_idle_dead(void) > > /* > > * Tell __cpu_die() that this CPU is now safe to dispose of. Once > > * this returns, power and/or clocks can be removed at any point > > - * from this CPU and its cache by platform_cpu_kill(). > > + * from this CPU and its cache by platform_cpu_kill(). We cannot > > + * call complete() this late, so we delegate it to an online CPU. > > */ > > - complete(&cpu_died); > > + smp_call_function_single(cpumask_first(cpu_online_mask), > > + arch_cpu_idle_dead_complete, NULL, 0); > > > > /* > > * Ensure that the cache lines associated with that completion are > > -- > http://baruch.siach.name/blog/ ~. .~ Tk Open Systems > =}------------------------------------------------ooO--U--Ooo------------{= > - baruch@tkos.co.il - tel: +972.52.368.4656, http://www.tkos.co.il - >