RE: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

Stuart Yoder
2014-06-16T18:53:52Z

> -----Original Message-----
> From: Will Deacon [mailto:will.deacon-5wv7dgnIgG8@public.gmane.org]
> Sent: Monday, June 16, 2014 12:04 PM
> To: Yoder Stuart-B08248
> Cc: Sethi Varun-B16395; Thierry Reding; Mark Rutland;
> devicetree-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; linux-samsung-soc-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Pawel
> Moll; Arnd Bergmann; Ian Campbell; Grant Grundler; Stephen Warren; linux-
> kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Marc Zyngier; Linux IOMMU; Rob Herring; Kumar
> Gala; linux-tegra-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Cho KyongHo; Dave P Martin; linux-arm-
> kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org
> Subject: Re: [PATCH v2] devicetree: Add generic IOMMU device tree
> bindings
>
> Hi Stuart,
>
> On Mon, Jun 16, 2014 at 05:56:32PM +0100, Stuart Yoder wrote:
> > > Do you have use-cases where you really need to change these mappings
> > > dynamically?
> >
> > Yes. In the case of a PCI bus-- you may not know in advance how many
> > PCI devices there are until you probe the bus. We have another FSL
> > proprietary bus we call the "fsl-mc" bus that is similar.
>
> For that case, though, you could still describe an algorithmic
> transformation from RequesterID to StreamID which corresponds to a fixed
> mapping.
>
> > Another thing to consider-- starting with SMMUv2, as you know, there
> > is a new distributed architecture with multiple TBUs and a centralized
> > TCU that walks the SMMU page tables. So instead of sprinkling multiple
> > SMMUs all over an SoC you now have the option a 1 central TCU and
> sprinkling
> > multiple TBUs around. However, this means that the stream ID
> namespace
> > is now global and can be pretty limited. In the SMMU implementation we
> > have there are only 64 stream ID total for our Soc. But we have many
> more
> > masters than that.
> >
> > So we look at stream IDs as really corresponding to an 'isolation
> context'
> > and not to a bus master. An isolation context is the domain you are
> > trying to isolate with the SMMU. Devices that all belong to the same
> > 'isolation context' can share the same stream ID, since they share
> > the same domain and page tables.
>
> Ok, this is more compelling.
>
> > So, perhaps by default some/most SMMU masters may have a default stream
> ID
> > of 0x0 that is used by the host...and that could be represented
> > statically in the device tree.
> >
> > But, we absolutely will need to dynamically set new stream IDs
> > into masters when a new IOMMU 'domain' is created and devices
> > are added to it. All the devices in a domain will share
> > the same stream ID.
> >
> > So whatever we do, let's please have an architecture flexible enough
> > to allow for this.
>
> What is the software interface to the logic that assigns the StreamIDs?
> Is
> it part of the SMMU, or a separate device (or set of devices)?

For us at the hardware level there are a few different ways that the streamIDs
can be set. It is not part of the SMMU. In the cases where there is simply
1 device to 1 streamID (e.g. USB controller) there is an SoC register where
you just program the stream ID. In the case of PCI, our PCI controller
has a RequesterID-to-streamID table that you set via some PCI controller
registers.

The way we generally thought it would work was something like
this:
-u-boot/bootloader makes any static streamID allocation if needed,
sets a default streamID (e.g. 0x0) in device and expresses
that in the device tree
-device tree would express relationship between devices
(including bus controllers) and the SMMU through mmu-masters
property
-u-boot would express the range of unused (or used) streamIDs via a new
device tree property so the kernel SMMU driver knows what streamIDs are
free
-in the SMMU driver a different vendor specific 'add_device' callback
could be used to handle our special cases where we need to set/change
the stream ID for devices added to a domain

Thanks,
Stuart