From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752357AbdIVSnU (ORCPT ); Fri, 22 Sep 2017 14:43:20 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:53942 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752220AbdIVSnS (ORCPT ); Fri, 22 Sep 2017 14:43:18 -0400 Date: Fri, 22 Sep 2017 11:43:14 -0700 From: "Paul E. McKenney" To: Sebastian Andrzej Siewior Cc: Josh Triplett , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , linux-kernel@vger.kernel.org Subject: Re: [PATCH 1/3] srcu: use cpu_online() instead custom check Reply-To: paulmck@linux.vnet.ibm.com References: <20170922152806.22860-1-bigeasy@linutronix.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170922152806.22860-1-bigeasy@linutronix.de> User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-GCONF: 00 x-cbid: 17092218-0056-0000-0000-000003CD79ED X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00007779; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000231; SDB=6.00920829; UDB=6.00462736; IPR=6.00701050; BA=6.00005601; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00017249; XFM=3.00000015; UTC=2017-09-22 18:43:15 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17092218-0057-0000-0000-000008047FEF Message-Id: <20170922184314.GS3521@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-09-22_07:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1707230000 definitions=main-1709220260 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Sep 22, 2017 at 05:28:04PM +0200, Sebastian Andrzej Siewior wrote: > The current check via srcu_online is slightly racy because after looking > at srcu_online there could be an interrupt that interrupted us long > enough until the CPU we checked against went offline. But in that case, wouldn't the interrupt block the synchronize_sched() later in the offline sequence? More to the point, are you actually seeing this failure, or is this a theoretical bug? > An alternative would be to hold the hotplug rwsem (so the CPUs don't > change their state) and then check based on cpu_online() if we queue it > on a specific CPU or not. queue_work_on() itself can handle if something > is enqueued on an offline CPU but a timer which is enqueued on an offline > CPU won't fire until the CPU is back online. > > I am not sure if the removal in rcu_init() is okay or not. I assume that > SRCU won't enqueue a work item before SRCU is up and ready. Another alternative would be to disable preemption across the check and the call to queue_delayed_work_on(). Yet another alternative would be to have an SRCU-specific per-CPU lock that is acquired across the setting and clearing of srcu_online, and also across the check and the call to queue_delayed_work_on(). This last would be more consistent with a desire to remove the synchronize_sched() from the offline sequence. Or am I missing something here? Thanx, Paul > Signed-off-by: Sebastian Andrzej Siewior > --- > kernel/rcu/srcutree.c | 22 ++++------------------ > kernel/rcu/tree.c | 6 ------ > 2 files changed, 4 insertions(+), 24 deletions(-) > > diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c > index 729a8706751d..d190af0e56f8 100644 > --- a/kernel/rcu/srcutree.c > +++ b/kernel/rcu/srcutree.c > @@ -36,6 +36,7 @@ > #include > #include > #include > +#include > > #include "rcu.h" > #include "rcu_segcblist.h" > @@ -424,21 +425,6 @@ static void srcu_gp_start(struct srcu_struct *sp) > WARN_ON_ONCE(state != SRCU_STATE_SCAN1); > } > > -/* > - * Track online CPUs to guide callback workqueue placement. > - */ > -DEFINE_PER_CPU(bool, srcu_online); > - > -void srcu_online_cpu(unsigned int cpu) > -{ > - WRITE_ONCE(per_cpu(srcu_online, cpu), true); > -} > - > -void srcu_offline_cpu(unsigned int cpu) > -{ > - WRITE_ONCE(per_cpu(srcu_online, cpu), false); > -} > - > /* > * Place the workqueue handler on the specified CPU if online, otherwise > * just run it whereever. This is useful for placing workqueue handlers > @@ -450,12 +436,12 @@ static bool srcu_queue_delayed_work_on(int cpu, struct workqueue_struct *wq, > { > bool ret; > > - preempt_disable(); > - if (READ_ONCE(per_cpu(srcu_online, cpu))) > + cpus_read_lock(); > + if (cpu_online(cpu)) > ret = queue_delayed_work_on(cpu, wq, dwork, delay); > else > ret = queue_delayed_work(wq, dwork, delay); > - preempt_enable(); > + cpus_read_unlock(); > return ret; > } > > diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c > index 1250e4bd4b85..a3cb562955c9 100644 > --- a/kernel/rcu/tree.c > +++ b/kernel/rcu/tree.c > @@ -3729,8 +3729,6 @@ int rcutree_online_cpu(unsigned int cpu) > { > sync_sched_exp_online_cleanup(cpu); > rcutree_affinity_setting(cpu, -1); > - if (IS_ENABLED(CONFIG_TREE_SRCU)) > - srcu_online_cpu(cpu); > return 0; > } > > @@ -3741,8 +3739,6 @@ int rcutree_online_cpu(unsigned int cpu) > int rcutree_offline_cpu(unsigned int cpu) > { > rcutree_affinity_setting(cpu, cpu); > - if (IS_ENABLED(CONFIG_TREE_SRCU)) > - srcu_offline_cpu(cpu); > return 0; > } > > @@ -4188,8 +4184,6 @@ void __init rcu_init(void) > for_each_online_cpu(cpu) { > rcutree_prepare_cpu(cpu); > rcu_cpu_starting(cpu); > - if (IS_ENABLED(CONFIG_TREE_SRCU)) > - srcu_online_cpu(cpu); > } > } > > -- > 2.14.1 >