Memory-mapped I/O barriers, state of affairs?

Memory-mapped I/O barriers, state of affairs?

[Leon Woestenberg]

Hello,

searching for good info on device IO racing against host memory writes
I came upon this 2006 article:

http://lwn.net/Articles/198988/

"there is no guaranteed ordering between writes to regular memory and
writes to a memory-mapped I/O range. "

What is the current solution for that particular problem, i.e. how
should I make sure host memory writes are committed before I have an
external DMA device act on it?

Thanks,
-- 
Leon Woestenberg.

Re: Memory-mapped I/O barriers, state of affairs?

[Andreas Bombe]

On Mon, Mar 22, 2010 at 08:18:49PM +0100, Leon Woestenberg wrote:
> What is the current solution for that particular problem, i.e. how
> should I make sure host memory writes are committed before I have an
> external DMA device act on it?

The dma_sync_* functions, if you reuse DMA buffers without unmapping,
take care of that.  Otherwise the process of mapping them handles it.

Documentation/DMA-API.txt and Documentation/PCI/PCI-DMA-mapping.txt
document those functions.

Re: Memory-mapped I/O barriers, state of affairs?

[Leon Woestenberg]

Hello Andreas, all,

On Mon, Mar 22, 2010 at 9:16 PM, Andreas Bombe <aeb@debian.org> wrote:
> On Mon, Mar 22, 2010 at 08:18:49PM +0100, Leon Woestenberg wrote:
>> What is the current solution for that particular problem, i.e. how
>> should I make sure host memory writes are committed before I have an
>> external DMA device act on it?
>
> The dma_sync_* functions, if you reuse DMA buffers without unmapping,
> take care of that.  Otherwise the process of mapping them handles it.
>
The mapping makes the memory cache-coherent. I already use that.

But does that mean that this coherency is guaranteed to occur before I
start MMIO access to a device, i.e. is there a guaranteed ordering
between the writes?

Regards,
-- 
Leon

Re: Memory-mapped I/O barriers, state of affairs?

[Robert Hancock]

On 03/22/2010 04:12 PM, Leon Woestenberg wrote:
> Hello Andreas, all,
>
> On Mon, Mar 22, 2010 at 9:16 PM, Andreas Bombe<aeb@debian.org>  wrote:
>> On Mon, Mar 22, 2010 at 08:18:49PM +0100, Leon Woestenberg wrote:
>>> What is the current solution for that particular problem, i.e. how
>>> should I make sure host memory writes are committed before I have an
>>> external DMA device act on it?
>>
>> The dma_sync_* functions, if you reuse DMA buffers without unmapping,
>> take care of that.  Otherwise the process of mapping them handles it.
>>
> The mapping makes the memory cache-coherent. I already use that.
>
> But does that mean that this coherency is guaranteed to occur before I
> start MMIO access to a device, i.e. is there a guaranteed ordering
> between the writes?

Well, for a streaming mapping, the device may not see the write to the 
memory at all until it's synced or unmapped (for one thing, if the 
kernel is using swiotlb to map the memory, it's an entirely different 
piece of memory and it needs to copy the data to where the device can 
actually see it).

For a coherent mapping, however, though I can't say for certain what the 
behavior is "supposed" to be, many drivers seem to depend on writes to 
these regions being ordered with respect to regular memory and would 
break if a particular platform didn't obey that.