Re: [PATCH] arm64: io: Relax implicit barriers in default I/O accessors

From: Catalin Marinas <catalin.marinas@arm.com>
To: Will Deacon <will@kernel.org>
Cc: Will Deacon <will.deacon@arm.com>, linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH] arm64: io: Relax implicit barriers in default I/O accessors
Date: Mon, 5 Aug 2019 11:39:05 +0100	[thread overview]
Message-ID: <20190805103905.GC59981@iMac.local> (raw)
In-Reply-To: <20190729170518.14271-1-will@kernel.org>

On Mon, Jul 29, 2019 at 06:05:18PM +0100, Will Deacon wrote:
> As a concrete example, consider the following:
> 
> 	memcpy(dma_buffer, data, bufsz);
> 	writel(DMA_START, dev->ctrl_reg);
> 
> A DMB ST instruction between the final write to the DMA buffer and the
> write to the control register will ensure that the writes to the DMA
> buffer are observed before the write to the control register by all
> observers. Put another way, if an observer can see the write to the
> control register, it can also see the writes to memory.

I think one of the counter arguments here were that a device does not
"observe" the write to the control register as that's not a master
access (by the device). Do you mean that if another CPU (not the device)
can observe the writel(), it would have also observed the write to the
DMA buffer (assuming the DMB)? Since the device is also an observer of
the DMA buffer accesses, the multi-copy atomicity ensures that the
device is also seeing the buffer updates following a DMB.

If I got this right, I'm fine with the patch ;).

> This has always
> been the case and is not sufficient to provide the ordering required by
> Linux, since there is no guarantee that the master interface of the
> DMA-capable device has observed either of the accesses. However, in an
> other-multi-copy atomic world, we can infer two things:
> 
>   1. A write arriving at an endpoint shared between multiple CPUs is
>      visible to all CPUs
> 
>   2. A write that is visible to all CPUs is also visible to all other
>      observers in the shareability domain
> 
> Pieced together, this allows us to use DMB OSHST for our default I/O
> write accessors and DMB OSHLD for our default I/O read accessors (the
> outer-shareability is for handling non-cacheable mappings) for shared
> devices. Memory-mapped, DMA-capable peripherals that are private to a
> CPU (i.e. inaccessible to other CPUs) still require the DSB, however
> these are few and far between and typically require special treatment
> anyway which is outside of the scope of the portable driver API (e.g.
> GIC, page-table walker, SPE profiler).

I think there is another class of devices which are not CPU private
(USB, network). The buffer here is on-chip and the CPU can't do much
other than issuing a DSB (and even this may not be sufficient). The
multi-copy atomicity rule would work between CPUs here but not
necessarily for the device. Not sure they rely on the barrier in
writel(), I guess we can wait and fix them with the mandatory barriers
afterwards. In the meantime:

Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel