Re: LSE atomic op ordering is weaker than intended?

From: Will Deacon <will@kernel.org>
To: Hector Martin <marcan@marcan.st>
Cc: Linux ARM <linux-arm-kernel@lists.infradead.org>,
	Marc Zyngier <maz@kernel.org>,
	Mark Rutland <mark.rutland@arm.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Boqun Feng <boqun.feng@gmail.com>,
	Catalin Marinas <catalin.marinas@arm.com>
Subject: Re: LSE atomic op ordering is weaker than intended?
Date: Wed, 3 Mar 2021 21:38:16 +0000	[thread overview]
Message-ID: <20210303213815.GC20055@willie-the-truck> (raw)
In-Reply-To: <0d194019-deb9-8d9b-823c-4226f2d2dce2@marcan.st>

On Thu, Mar 04, 2021 at 04:37:31AM +0900, Hector Martin wrote:
> On 04/03/2021 03.40, Will Deacon wrote:
> > > // ...stuff that needs to be ordered prior to the atomic here
> > > ret = atomic_fetch_or_release(flags...)
> > > if (condition on ret and unrelated stuff) {
> > > 	writel(reg_send, ...) // includes pre-barrier
> > 
> > Looks you have a control dependency here, so I think that can be
> > writel_relaxed() [with a comment!].
> 
> Wouldn't the control dependency be strictly on the *read* portion of the
> atomic, while I'm looking to order on the write part?

Ah, sorry, I didn't realise you were ordering against the write. I'll wait
for the v3, but just one thing that might help:

> That is, that I can create two ordered sequences on two PEs:
> 
> (normal access 1; mmio write 1)
> (mmio write 2; normal access 2)
> 
> Such that I can assert that at least one of the following orderings are
> true:
> 
> (normal access 1; normal access 2)
> (as observed by the PEs)
> 
> or:
> 
> (mmio write 2; mmio write 1)
> (as observed by the MMIO device)

One thing to bear in mind here is that the MMIO device cannot "observe"
anything in the architectural sense because it is a slave interface. In
order to observe a memory access, you must emit a read or a write
transaction, and it's this notion of observation which the shareability
domains are built around.

So for this example, we can talk about the CPUs (in the inner-shareable
domain) observing the MMIO writes and inner-shareable barriers are
sufficient for that. The device mapping of the MMIO registers will then
ensure that they arrive at the endpoint in that order too.

> But never this combination:
> 
> (normal access 2; normal access 1)
> (mmio write 1; mmio write 2)
> 
> > But hack something together, and I'll look at your v3 to see what's going
> > on.
> 
> Please do, hopefully the full code makes more sense.

Hopefully, as I don't grok how this deals with spurious interrupts if it
only does MMIO writes.

Will

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel