From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752473AbdJFTS3 (ORCPT ); Fri, 6 Oct 2017 15:18:29 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:41916 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751639AbdJFTS1 (ORCPT ); Fri, 6 Oct 2017 15:18:27 -0400 Date: Fri, 6 Oct 2017 12:18:22 -0700 From: "Paul E. McKenney" To: Peter Zijlstra Cc: linux-kernel@vger.kernel.org, mingo@kernel.org, jiangshanlai@gmail.com, dipankar@in.ibm.com, akpm@linux-foundation.org, mathieu.desnoyers@efficios.com, josh@joshtriplett.org, tglx@linutronix.de, rostedt@goodmis.org, dhowells@redhat.com, edumazet@google.com, fweisbec@gmail.com, oleg@redhat.com Subject: Re: [PATCH tip/core/rcu 1/9] rcu: Provide GP ordering in face of migrations and delays Reply-To: paulmck@linux.vnet.ibm.com References: <20171004212915.GA10089@linux.vnet.ibm.com> <1507152575-11055-1-git-send-email-paulmck@linux.vnet.ibm.com> <20171005094114.yoqkhndfwidczfqj@hirez.programming.kicks-ass.net> <20171005145513.GO3521@linux.vnet.ibm.com> <20171005153913.5wanmnfmi6i3byv7@hirez.programming.kicks-ass.net> <20171005161909.GS3521@linux.vnet.ibm.com> <20171005162514.moaa24mn2pkto5fv@hirez.programming.kicks-ass.net> <20171005182204.GT3521@linux.vnet.ibm.com> <20171006090723.qbcea5xnwpngrcxi@hirez.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20171006090723.qbcea5xnwpngrcxi@hirez.programming.kicks-ass.net> User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-GCONF: 00 x-cbid: 17100619-0048-0000-0000-000001F11523 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00007851; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000235; SDB=6.00927387; UDB=6.00466642; IPR=6.00707646; BA=6.00005623; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00017424; XFM=3.00000015; UTC=2017-10-06 19:18:25 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17100619-0049-0000-0000-000042C97D5D Message-Id: <20171006191822.GI3521@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-10-06_04:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=2 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1707230000 definitions=main-1710060268 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Oct 06, 2017 at 11:07:23AM +0200, Peter Zijlstra wrote: > On Thu, Oct 05, 2017 at 11:22:04AM -0700, Paul E. McKenney wrote: > > Hmmm... Here is what I was worried about: > > > > C C-PaulEMcKenney-W+RWC4+2017-10-05 > > > > { > > } > > > > P0(int *a, int *x) > > { > > WRITE_ONCE(*a, 1); > > smp_mb(); /* Lock acquisition for rcu_node ->lock. */ > > WRITE_ONCE(*x, 1); > > } > > > > P1(int *x, int *y, spinlock_t *l) > > { > > r3 = READ_ONCE(*x); > > smp_mb(); /* Lock acquisition for rcu_node ->lock. */ > > spin_lock(l); /* Locking in complete(). */ > > WRITE_ONCE(*y, 1); > > spin_unlock(l); > > } > > > > P2(int *y, int *b, spinlock_t *l) > > { > > spin_lock(l); /* Locking in wait_for_completion. */ > > r4 = READ_ONCE(*y); > > spin_unlock(l); > > r1 = READ_ONCE(*b); > > } > > > > P3(int *b, int *a) > > { > > WRITE_ONCE(*b, 1); > > smp_mb(); > > r2 = READ_ONCE(*a); > > } > > > > exists (1:r3=1 /\ 2:r4=1 /\ 2:r1=0 /\ 3:r2=0) > > /me goes and install this herd thing again.. I'm sure I had it running > _somewhere_.. A well. > > C C-PaulEMcKenney-W+RWC4+2017-10-05 > > { > } > > P0(int *a, int *x) > { > WRITE_ONCE(*a, 1); > smp_mb(); /* Lock acquisition for rcu_node ->lock. */ > WRITE_ONCE(*x, 1); > } > > P1(int *x, int *y) > { > r3 = READ_ONCE(*x); > smp_mb(); /* Lock acquisition for rcu_node ->lock. */ > smp_store_release(y, 1); > } > > P2(int *y, int *b) > { > r4 = smp_load_acquire(y); > r1 = READ_ONCE(*b); > } > > P3(int *b, int *a) > { > WRITE_ONCE(*b, 1); > smp_mb(); > r2 = READ_ONCE(*a); > } > > exists (1:r3=1 /\ 2:r4=1 /\ 2:r1=0 /\ 3:r2=0) > > > Is what I was thinking of, I think that is the minimal ordering > complete()/wait_for_completion() need to provide. OK, I will bite... What do the smp_store_release() and the smp_load_acquire() correspond to? I see just plain locking in wait_for_completion() and complete(). > (also, that r# numbering confuses the hell out of me, its not related to > P nor to the variables) Yeah, it is random, sorry!!! > Test C-PaulEMcKenney-W+RWC4+2017-10-05 Allowed > States 15 > 1:r3=0; 2:r1=0; 2:r4=0; 3:r2=0; > 1:r3=0; 2:r1=0; 2:r4=0; 3:r2=1; > 1:r3=0; 2:r1=0; 2:r4=1; 3:r2=0; > 1:r3=0; 2:r1=0; 2:r4=1; 3:r2=1; > 1:r3=0; 2:r1=1; 2:r4=0; 3:r2=0; > 1:r3=0; 2:r1=1; 2:r4=0; 3:r2=1; > 1:r3=0; 2:r1=1; 2:r4=1; 3:r2=0; > 1:r3=0; 2:r1=1; 2:r4=1; 3:r2=1; > 1:r3=1; 2:r1=0; 2:r4=0; 3:r2=0; > 1:r3=1; 2:r1=0; 2:r4=0; 3:r2=1; > 1:r3=1; 2:r1=0; 2:r4=1; 3:r2=1; > 1:r3=1; 2:r1=1; 2:r4=0; 3:r2=0; > 1:r3=1; 2:r1=1; 2:r4=0; 3:r2=1; > 1:r3=1; 2:r1=1; 2:r4=1; 3:r2=0; > 1:r3=1; 2:r1=1; 2:r4=1; 3:r2=1; > No > Witnesses > Positive: 0 Negative: 15 > Condition exists (1:r3=1 /\ 2:r4=1 /\ 2:r1=0 /\ 3:r2=0) > Observation C-PaulEMcKenney-W+RWC4+2017-10-05 Never 0 15 > Time C-PaulEMcKenney-W+RWC4+2017-10-05 0.04 > Hash=f7f8ad6eab33e90718a394bcb021557d But yes, looking closer, this corresponds to the rule of thumb about non-rf relations and full memory barriers. We have two non-rf relations (P2->P3 and P3->P0), so we need two full barriers, one each between the non-rf relations. So I dropped that patch yesterday. The main thing I was missing was that there is no ordering-free fastpath in wait_for_completion() and complete(): Each unconditionally acquires the lock. So the smp_mb() that I was trying to add doesn't need to be there. Thanx, Paul