From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Paul E. McKenney" Subject: Re: [v3,11/41] mips: reuse asm-generic/barrier.h Date: Mon, 25 Jan 2016 22:03:22 -0800 Message-ID: <20160126060322.GJ4503@linux.vnet.ibm.com> References: <5697F6D2.60409@imgtec.com> <20160114203430.GC3818@linux.vnet.ibm.com> <56980C91.1010403@imgtec.com> <20160114212913.GF3818@linux.vnet.ibm.com> <20160115085554.GF3421@worktop> <20160115091348.GA27936@worktop> <20160115174612.GV3818@linux.vnet.ibm.com> <20160115212714.GM3421@worktop> <20160115215853.GC3818@linux.vnet.ibm.com> <20160125164242.GF22927@arm.com> Reply-To: paulmck@linux.vnet.ibm.com Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Content-Disposition: inline In-Reply-To: <20160125164242.GF22927@arm.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: virtualization-bounces@lists.linux-foundation.org Errors-To: virtualization-bounces@lists.linux-foundation.org To: Will Deacon Cc: linux-mips@linux-mips.org, linux-ia64@vger.kernel.org, "Michael S. Tsirkin" , Peter Zijlstra , virtualization@lists.linux-foundation.org, "H. Peter Anvin" , sparclinux@vger.kernel.org, Ingo Molnar , linux-arch@vger.kernel.org, linux-s390@vger.kernel.org, Russell King - ARM Linux , user-mode-linux-devel@lists.sourceforge.net, linux-sh@vger.kernel.org, Michael Ellerman , x86@kernel.org, xen-devel@lists.xenproject.org, Ingo Molnar , linux-xtensa@linux-xtensa.org, james.hogan@imgtec.com, Arnd Bergmann , Stefano Stabellini , adi-buildroot-devel@lists.sourceforge.net, Leonid Yegoshin , ddaney.cavm@gmail.com, Thomas Gleixner , linux-metag@vger.kernel. List-Id: linux-arch.vger.kernel.org On Mon, Jan 25, 2016 at 04:42:43PM +0000, Will Deacon wrote: > On Fri, Jan 15, 2016 at 01:58:53PM -0800, Paul E. McKenney wrote: > > On Fri, Jan 15, 2016 at 10:27:14PM +0100, Peter Zijlstra wrote: > > > On Fri, Jan 15, 2016 at 09:46:12AM -0800, Paul E. McKenney wrote: > > > > On Fri, Jan 15, 2016 at 10:13:48AM +0100, Peter Zijlstra wrote: > > > > > > > > And the stuff we're confused about is how best to express the difference > > > > > and guarantees of these two forms of transitivity and how exactly they > > > > > interact. > > > > > > > > Hoping my memory-barrier.txt patch helps here... > > > > > > Yes, that seems a good start. But yesterday you raised the 'fun' point > > > of two globally ordered sequences connected by a single local link. > > > > The conclusion that I am slowly coming to is that litmus tests should > > not be thought of as linear chains, but rather as cycles. If you think > > of it as a cycle, then it doesn't matter where the local link is, just > > how many of them and how they are connected. > > Do you have some examples of this? I'm struggling to make it work in my > mind, or are you talking specifically in the context of the kernel > memory model? Now that you mention it, maybe it would be best to keep the transitive and non-transitive separate for the time being anyway. Just because it might be possible to deal with does not necessarily mean that we should be encouraging it. ;-) > > But I will admit that there are some rather strange litmus tests that > > challenge this cycle-centric view, for example, the one shown below. > > It turns out that herd and ppcmem disagree on the outcome. (The Power > > architects side with ppcmem.) > > > > > And I think I'm still confused on LWSYNC (in the smp_wmb case) when one > > > of the stores looses a conflict, and if that scenario matters. If it > > > does, we should inspect the same case for other barriers. > > > > Indeed. I am still working on how these should be described. My > > current thought is to be quite conservative on what ordering is > > actually respected, however, the current task is formalizing how > > RCU plays with the rest of the memory model. > > > > Thanx, Paul > > > > ------------------------------------------------------------------------ > > > > PPC Overlapping Group-B sets version 4 > > "" > > (* When the Group-B sets from two different barriers involve instructions in > > the same thread, within that thread one set must contain the other. > > > > P0 P1 P2 > > Rx=1 Wy=1 Wz=2 > > dep. lwsync lwsync > > Ry=0 Wz=1 Wx=1 > > Rz=1 > > > > assert(!(z=2)) > > > > Forbidden by ppcmem, allowed by herd. > > *) > > { > > 0:r1=x; 0:r2=y; 0:r3=z; > > 1:r1=x; 1:r2=y; 1:r3=z; 1:r4=1; > > 2:r1=x; 2:r2=y; 2:r3=z; 2:r4=1; 2:r5=2; > > } > > P0 | P1 | P2 ; > > lwz r6,0(r1) | stw r4,0(r2) | stw r5,0(r3) ; > > xor r7,r6,r6 | lwsync | lwsync ; > > lwzx r7,r7,r2 | stw r4,0(r3) | stw r4,0(r1) ; > > lwz r8,0(r3) | | ; > > > > exists > > (z=2 /\ 0:r6=1 /\ 0:r7=0 /\ 0:r8=1) > > That really hurts. Assuming that the "assert(!(z=2))" is actually there > to constrain the coherence order of z to be {0->1->2}, then I think that > this test is forbidden on arm using dmb instead of lwsync. That said, I > also don't think the Rz=1 in P0 changes anything. What about the smp_wmb() variant of dmb that orders only stores? > The double negatives don't help here! (it is forbidden to guarantee that > z is not always 2). Yes, this is a weird one, and I don't know of any use of it. Thanx, Paul From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from e34.co.us.ibm.com ([32.97.110.152]:37761 "EHLO e34.co.us.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754486AbcAZGIq (ORCPT ); Tue, 26 Jan 2016 01:08:46 -0500 Received: from localhost by e34.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 25 Jan 2016 23:08:46 -0700 Date: Mon, 25 Jan 2016 22:03:22 -0800 From: "Paul E. McKenney" Subject: Re: [v3,11/41] mips: reuse asm-generic/barrier.h Message-ID: <20160126060322.GJ4503@linux.vnet.ibm.com> Reply-To: paulmck@linux.vnet.ibm.com References: <5697F6D2.60409@imgtec.com> <20160114203430.GC3818@linux.vnet.ibm.com> <56980C91.1010403@imgtec.com> <20160114212913.GF3818@linux.vnet.ibm.com> <20160115085554.GF3421@worktop> <20160115091348.GA27936@worktop> <20160115174612.GV3818@linux.vnet.ibm.com> <20160115212714.GM3421@worktop> <20160115215853.GC3818@linux.vnet.ibm.com> <20160125164242.GF22927@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160125164242.GF22927@arm.com> Sender: linux-arch-owner@vger.kernel.org List-ID: To: Will Deacon Cc: Peter Zijlstra , Leonid Yegoshin , "Michael S. Tsirkin" , linux-kernel@vger.kernel.org, Arnd Bergmann , linux-arch@vger.kernel.org, Andrew Cooper , Russell King - ARM Linux , virtualization@lists.linux-foundation.org, Stefano Stabellini , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , Joe Perches , David Miller , linux-ia64@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-metag@vger.kernel.org, linux-mips@linux-mips.org, x86@kernel.org, user-mode-linux-devel@lists.sourceforge.net, adi-buildroot-devel@lists.sourceforge.net, linux-sh@vger.kernel.org, linux-xtensa@linux-xtensa.org, xen-devel@lists.xenproject.org, Ralf Baechle , Ingo Molnar , ddaney.cavm@gmail.com, james.hogan@imgtec.com, Michael Ellerman Message-ID: <20160126060322.9UnvYLOgWXRVKdkuH-bBDnNyzyohdBTiwNRIV1DISCc@z> On Mon, Jan 25, 2016 at 04:42:43PM +0000, Will Deacon wrote: > On Fri, Jan 15, 2016 at 01:58:53PM -0800, Paul E. McKenney wrote: > > On Fri, Jan 15, 2016 at 10:27:14PM +0100, Peter Zijlstra wrote: > > > On Fri, Jan 15, 2016 at 09:46:12AM -0800, Paul E. McKenney wrote: > > > > On Fri, Jan 15, 2016 at 10:13:48AM +0100, Peter Zijlstra wrote: > > > > > > > > And the stuff we're confused about is how best to express the difference > > > > > and guarantees of these two forms of transitivity and how exactly they > > > > > interact. > > > > > > > > Hoping my memory-barrier.txt patch helps here... > > > > > > Yes, that seems a good start. But yesterday you raised the 'fun' point > > > of two globally ordered sequences connected by a single local link. > > > > The conclusion that I am slowly coming to is that litmus tests should > > not be thought of as linear chains, but rather as cycles. If you think > > of it as a cycle, then it doesn't matter where the local link is, just > > how many of them and how they are connected. > > Do you have some examples of this? I'm struggling to make it work in my > mind, or are you talking specifically in the context of the kernel > memory model? Now that you mention it, maybe it would be best to keep the transitive and non-transitive separate for the time being anyway. Just because it might be possible to deal with does not necessarily mean that we should be encouraging it. ;-) > > But I will admit that there are some rather strange litmus tests that > > challenge this cycle-centric view, for example, the one shown below. > > It turns out that herd and ppcmem disagree on the outcome. (The Power > > architects side with ppcmem.) > > > > > And I think I'm still confused on LWSYNC (in the smp_wmb case) when one > > > of the stores looses a conflict, and if that scenario matters. If it > > > does, we should inspect the same case for other barriers. > > > > Indeed. I am still working on how these should be described. My > > current thought is to be quite conservative on what ordering is > > actually respected, however, the current task is formalizing how > > RCU plays with the rest of the memory model. > > > > Thanx, Paul > > > > ------------------------------------------------------------------------ > > > > PPC Overlapping Group-B sets version 4 > > "" > > (* When the Group-B sets from two different barriers involve instructions in > > the same thread, within that thread one set must contain the other. > > > > P0 P1 P2 > > Rx=1 Wy=1 Wz=2 > > dep. lwsync lwsync > > Ry=0 Wz=1 Wx=1 > > Rz=1 > > > > assert(!(z=2)) > > > > Forbidden by ppcmem, allowed by herd. > > *) > > { > > 0:r1=x; 0:r2=y; 0:r3=z; > > 1:r1=x; 1:r2=y; 1:r3=z; 1:r4=1; > > 2:r1=x; 2:r2=y; 2:r3=z; 2:r4=1; 2:r5=2; > > } > > P0 | P1 | P2 ; > > lwz r6,0(r1) | stw r4,0(r2) | stw r5,0(r3) ; > > xor r7,r6,r6 | lwsync | lwsync ; > > lwzx r7,r7,r2 | stw r4,0(r3) | stw r4,0(r1) ; > > lwz r8,0(r3) | | ; > > > > exists > > (z=2 /\ 0:r6=1 /\ 0:r7=0 /\ 0:r8=1) > > That really hurts. Assuming that the "assert(!(z=2))" is actually there > to constrain the coherence order of z to be {0->1->2}, then I think that > this test is forbidden on arm using dmb instead of lwsync. That said, I > also don't think the Rz=1 in P0 changes anything. What about the smp_wmb() variant of dmb that orders only stores? > The double negatives don't help here! (it is forbidden to guarantee that > z is not always 2). Yes, this is a weird one, and I don't know of any use of it. Thanx, Paul