Re: [RFC PATCH 0/3] iommu: Add range flush operation

[Russell King - ARM Linux <linux-lFZ/pmaqli7XmaaqVzeoHQ@public.gmane.org>]

On Tue, Sep 29, 2015 at 05:27:12PM +0100, Robin Murphy wrote:
> Eh, swings and roundabouts. An argument denoting whether the flush is being
> called on the map or unmap path would be fine,

Sorry, that statement is wrong.  It's not about whether you flush before
or after the DMA operation.  I'm afraid I'm probably going to tell you
how to suck eggs here, because I don't think you quite "get it" with
non-dma-coherent modern CPUs.

Modern CPUs prefetch data into their caches, and they also randomly write
back data from their caches to memory.  When performing a DMA operation
from device to memory, you need to do two things with CPU caches which
aren't coherent:

1. Before starting the DMA operation, you need to walk over the memory to
   be mapped, ensuring that any dirty cache lines are written back.  This
   is to prevent dirty cache lines overwriting data that has already been
   DMA'd from the device.

2. After the DMA operation has completed, you need to walk over the
   memory again, invalidating any cache lines which may have been
   speculatively loaded from that memory while DMA was running.  These
   cache lines may have been loaded prior to the DMA operation placing
   the new data into memory.

So, it's not a before-or-after, you have to always perform write-back
cache maintanence prior to any DMA operation, and then invalidate cache
maintanence after the DMA operation has completed for any mapping which
the DMA may have written to (which means device-to-memory and
bidirectional mappings.)

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.