Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Dave Martin <Dave.Martin-5wv7dgnIgG8@public.gmane.org>]

On Thu, May 01, 2014 at 11:02:14PM +0900, Cho KyongHo wrote:
> On Tue, 29 Apr 2014 23:00:29 +0200, Arnd Bergmann wrote:
> > On Tuesday 29 April 2014 13:07:54 Grant Grundler wrote:
> > > On Tue, Apr 29, 2014 at 11:16 AM, Dave Martin <Dave.Martin-5wv7dgnIgG8@public.gmane.org> wrote:
> > > ...
> > > > An IOMMU is really a specialised bridge
> > > 
> > > Is a GART a bridge?
> > > 
> > > IOMMUs can provide three basic functions:
> > > 1) remap address space to reach phys mem ranges that the device is
> > > otherwise not capable of accessing (classic 32-bit DMA to reach 64-bit
> > > Phys address)
> > > 
> > > 2) implement scatter-gather (page level granularity) so the device
> > > doesn't have to
> > > 
> > > 3) provide some level of system protection against "rogue" DMA by
> > > forcing everything through a DMA mapping interface and faulting when
> > > encountering unmapped DMA transactions.
> > 
> > [ 4) provide isolation between multiple contexts, typically for purposes
> >      of virtualization]
> > 
> > > I summarize IOMMUs as: "participate in the routing of MMIO
> > > transactions in the system fabric."
> > > In this sense, IOMMUs are sort of a bridge. Defining what kind of
> > > routing they can do (coalesce transactions? remapping MMIO domains?)
> > > and which address ranges they route would describe most of that
> > > functionality.
> > > 
> > > This "remapping" of MMIO transaction is also usually asymmetric.
> > > Meaning routing of "downstream" transactions *might* be completely
> > > different than the routing + remapping of transactions heading
> > > upstream. DMA mappings services are designed to handle only the
> > > transactions generated (aka "mastered") by a downstream device.
> > 
> > For the purposes of the DT binding, we have a 'ranges' property
> > that defines the downstream translation (CPU-to-MMIO) and a
> > 'dma-ranges' property for the opposite address translation
> > (device-to-memory).
> > 
> > > >, so it may be cleaner to describe
> > > > an IOMMU using a real bus node in the DT, if we also define a way to make
> > > > master/slave linkages explicit where it matters.
> > > 
> > > "where it matters" is a bit vague.  Is the goal to just enable DMA
> > > mapping services to "do the right thing" for a device that can
> > > generate DMA?
> > 
> > Yes. It's very complicated unfortunately, because we have to be
> > able to deal with arbitrary combinations of a lot of oddball cases
> > that can show up in random SoCs:
> > 
> > - device can only do DMA to a limited address range
> > - DMA is noncoherent and needs manual cache management
> > - DMA address is at an offset from physical address
> > - some devices have an IOMMU
> > - some IOMMUs are shared between devices
> > - some devices with IOMMU can have multiple simultaneous contexts
> > - a device may access some memory directly and some other memory through IOMMU
> 
> Do we need to consider this case?
> I don't think a device can have different contexts at the same time.
> If there such device is in a system, its driver must handle it correctly
> with different devices descriptors for the different contexts, for example.
> I mean, if a device has two DMA ports that are in different contexts,
> they can be treated as different devices which are handed by a driver.

GPUs will definitely be capable of acting on behalf of multiple contexts
at the same time, in a dynamic fashion.  This doesn't necessarily mean
that it masters through different ports or onto different buses though.

Sketching out how we would describe this in DT doesn't imply that we
need Linux to support it.  It's more about asking: if we have to support
this in the future, how badly will it screw up the current framework?

> I worry that abstracting everything we can think may make the problem harder.

That's always a risk, though pain today may be worth it if is saves a
larger amount of pain in the future.

As I say on the other branch of this thread, I'll follow up with
something a bit more concrete to illustrate the kind of thing I mean.

Cheers
---Dave