From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751960AbdG0OhG (ORCPT ); Thu, 27 Jul 2017 10:37:06 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:47441 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751668AbdG0OhF (ORCPT ); Thu, 27 Jul 2017 10:37:05 -0400 Date: Thu, 27 Jul 2017 07:36:58 -0700 From: "Paul E. McKenney" To: Boqun Feng Cc: Peter Zijlstra , linux-kernel@vger.kernel.org, mingo@kernel.org, jiangshanlai@gmail.com, dipankar@in.ibm.com, akpm@linux-foundation.org, mathieu.desnoyers@efficios.com, josh@joshtriplett.org, tglx@linutronix.de, rostedt@goodmis.org, dhowells@redhat.com, edumazet@google.com, fweisbec@gmail.com, oleg@redhat.com, will.deacon@arm.com Subject: Re: [PATCH tip/core/rcu 4/5] sys_membarrier: Add expedited option Reply-To: paulmck@linux.vnet.ibm.com References: <20170725211926.GA3730@linux.vnet.ibm.com> <20170725215510.GD28975@worktop> <20170725235936.GC3730@linux.vnet.ibm.com> <20170726074128.ybb3e4flnjkrpi74@hirez.programming.kicks-ass.net> <20170726154110.GN3730@linux.vnet.ibm.com> <20170727083003.ivb2fr47vepa2g6t@hirez.programming.kicks-ass.net> <20170727130816.GP3730@linux.vnet.ibm.com> <20170727135551.tg7cduw22krjzbvt@tardis> <20170727141633.GT3730@linux.vnet.ibm.com> <20170727142955.bb2bdgbjcm745xwo@tardis> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170727142955.bb2bdgbjcm745xwo@tardis> User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-GCONF: 00 x-cbid: 17072714-0056-0000-0000-000003AD8D3A X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00007435; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000214; SDB=6.00893669; UDB=6.00446800; IPR=6.00673830; BA=6.00005495; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00016408; XFM=3.00000015; UTC=2017-07-27 14:37:01 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17072714-0057-0000-0000-000007E3AD06 Message-Id: <20170727143658.GW3730@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-07-27_07:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1706020000 definitions=main-1707270229 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Jul 27, 2017 at 10:29:55PM +0800, Boqun Feng wrote: > On Thu, Jul 27, 2017 at 07:16:33AM -0700, Paul E. McKenney wrote: > > On Thu, Jul 27, 2017 at 09:55:51PM +0800, Boqun Feng wrote: > > > Hi Paul, > > > > > > I have a side question out of curiosity: > > > > > > How does synchronize_sched() work properly for sys_membarrier()? > > > > > > sys_membarrier() requires every other CPU does a smp_mb() before it > > > returns, and I know synchronize_sched() will wait until all CPUs running > > > a kernel thread do a context-switch, which has a smp_mb(). However, I > > > believe sched flavor RCU treat CPU running a user thread as a quiesent > > > state, so synchronize_sched() could return without that CPU does a > > > context switch. > > > > > > So why could we use synchronize_sched() for sys_membarrier()? > > > > > > In particular, could the following happens? > > > > > > CPU 0: CPU 1: > > > ========================= ========================== > > > > > > {read Y}(reordered) <------------------------------+ > > > store Y; | > > > read X; --------------------------------------+ | > > > sys_membarrier(): | | > > > synchronize_sched(); update_process_times(user): //user == true | | > > > rcu_check_callbacks(usr): | | > > > if (user || ..) { | | > > > rcu_sched_qs() | | > > > ... | | > > > | | > > > > The reporting of the quiescent state will acquire the leaf rcu_node > > structure's lock, with an smp_mb__after_unlock_lock(), which will > > one way or another be a full memory barrier. So the reorderings > > cannot happen. > > > > Unless I am missing something subtle. ;-) > > > > Well, smp_mb__after_unlock_lock() in ARM64 is a no-op, and ARM64's lock > doesn't provide a smp_mb(). > > So my point is more like: synchronize_sched() happens to be a > sys_membarrier() because of some implementation detail, and if some day > we come up with a much cheaper way to implement sched flavor > RCU(hopefully!), synchronize_sched() may be not good for the job. So at > least, we'd better document this somewhere? Last I heard, ARM's unlock/lock acted as a full barrier. Will? Please see the synchronize_sched() comment header for the documentation you are asking for. And the "Memory-Barrier Guarantees" section of Documentation/RCU/Design/Requirements/Requirements.html. Thanx, Paul > Regards, > Boqun > > > Thanx, Paul > > > > > | | > > > read Y; --------------------------------------+----+ > > > store X; | > > > {read X}(reordered) <-------------------------+ > > > > > > I assume the timer interrupt handler, which interrupts a user space and > > > reports a quiesent state for sched flavor RCU, may not have a smp_mb() > > > in some code path. > > > > > > I may miss something subtle, but it just not very obvious how > > > synchronize_sched() will guarantee a remote CPU running in userspace to > > > do a smp_mb() before it returns, this is at least not in RCU > > > requirements, right? > > > > > > Regards, > > > Boqun > > > >