[PATCH] documentation: iommu: add description of ARM System MMU binding

[PATCH] documentation: iommu: add description of ARM System MMU binding

[Will Deacon]

This patch adds a description of the device tree binding for the ARM
System MMU architecture.

Cc: Rob Herring <robherring2@gmail.com>
Cc: Andreas Herrmann <andreas.herrmann@calxeda.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
---

Hello,

The driver for this is still a WIP. Both Andreas and myself have prototype
code, but we're planning to merge that together to get something more
general. Deciding on the binding is a good first step.

All comments welcome,

Will

 .../devicetree/bindings/iommu/arm,smmu.txt         | 61 ++++++++++++++++++++++
 1 file changed, 61 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/iommu/arm,smmu.txt

diff --git a/Documentation/devicetree/bindings/iommu/arm,smmu.txt b/Documentation/devicetree/bindings/iommu/arm,smmu.txt
new file mode 100644
index 0000000..938325f
--- /dev/null
+++ b/Documentation/devicetree/bindings/iommu/arm,smmu.txt
@@ -0,0 +1,61 @@
+* ARM System MMU Architecture Implementation
+
+ARM SoCs may contain an implementation of the ARM System Memory
+Management Unit Architecture, which can be used to provide 1 or 2 stages
+of address translation to bus masters external to the CPU.
+
+The SMMU may also raise interrupts in response to various fault
+conditions.
+
+** System MMU required properties:
+
+- compatible    : Should be one of "arm,smmu-v1" or "arm,smmu-v2"
+                  depending on the version of the architecture
+                  implemented.
+
+- reg           : Base address and size of the SMMU.
+
+- #global-interrupts : The number of global interrupts exposed by the
+                       device.
+
+- interrupts    : Interrupt list, with the first #global-irqs entries
+                  corresponding to the global interrupts and any
+                  following entries corresponding to context interrupts,
+                  specified in order of their indexing by the SMMU.
+
+- mmu-masters   : A list of phandles to device nodes representing bus
+                  masters for which the SMMU can provide a translation.
+
+- stream-ids    : A list of 16-bit values corresponding to the StreamIDs
+                  for the devices listed in the mmu-masters property.
+                  This list must be same length as mmu-masters, so
+                  masters with multiple stream-ids will have multiple
+                  entries in mmu-masters.
+
+** System MMU optional properties:
+
+- smmu-parent   : When multiple SMMUs are chained together, this
+                  property can be used to provide a phandle to the
+                  parent SMMU (that is the next SMMU on the path going
+                  from the mmu-masters towards memory) node for this
+                  SMMU.
+
+Example:
+
+        smmu {
+                compatible = "arm,smmu-v1";
+                reg = <0xba5e0000 0x10000>;
+                #global-interrupts = <2>;
+                interrupts = <0 32 4>,
+                             <0 33 4>,
+                             <0 34 4>, /* This is the first context interrupt */
+                             <0 35 4>,
+                             <0 36 4>,
+                             <0 37 4>;
+                mmu-masters = <&dma0>,
+                              <&dma0>,
+                              <&dma1>;
+                stream-ids  = <0xd01d>,
+                              <0xd01e>,
+                              <0xd11c>;
+        };
-- 
1.8.0

[PATCH] documentation: iommu: add description of ARM System MMU binding

[Rob Herring]

On 04/04/2013 11:50 AM, Will Deacon wrote:
> This patch adds a description of the device tree binding for the ARM
> System MMU architecture.
> 
> Cc: Rob Herring <robherring2@gmail.com>
> Cc: Andreas Herrmann <andreas.herrmann@calxeda.com>
> Signed-off-by: Will Deacon <will.deacon@arm.com>
> ---
> 
> Hello,
> 
> The driver for this is still a WIP. Both Andreas and myself have prototype
> code, but we're planning to merge that together to get something more
> general. Deciding on the binding is a good first step.

Thanks for getting this out.

> All comments welcome,
> 
> Will
> 
>  .../devicetree/bindings/iommu/arm,smmu.txt         | 61 ++++++++++++++++++++++
>  1 file changed, 61 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/iommu/arm,smmu.txt
> 
> diff --git a/Documentation/devicetree/bindings/iommu/arm,smmu.txt b/Documentation/devicetree/bindings/iommu/arm,smmu.txt
> new file mode 100644
> index 0000000..938325f
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/iommu/arm,smmu.txt
> @@ -0,0 +1,61 @@
> +* ARM System MMU Architecture Implementation
> +
> +ARM SoCs may contain an implementation of the ARM System Memory
> +Management Unit Architecture, which can be used to provide 1 or 2 stages
> +of address translation to bus masters external to the CPU.
> +
> +The SMMU may also raise interrupts in response to various fault
> +conditions.
> +
> +** System MMU required properties:
> +
> +- compatible    : Should be one of "arm,smmu-v1" or "arm,smmu-v2"
> +                  depending on the version of the architecture
> +                  implemented.

We can keep these, but we should have specific models like arm,smmu-400,
etc. as well.

> +
> +- reg           : Base address and size of the SMMU.
> +
> +- #global-interrupts : The number of global interrupts exposed by the
> +                       device.
> +
> +- interrupts    : Interrupt list, with the first #global-irqs entries
> +                  corresponding to the global interrupts and any
> +                  following entries corresponding to context interrupts,
> +                  specified in order of their indexing by the SMMU.
> +
> +- mmu-masters   : A list of phandles to device nodes representing bus
> +                  masters for which the SMMU can provide a translation.
> +
> +- stream-ids    : A list of 16-bit values corresponding to the StreamIDs
> +                  for the devices listed in the mmu-masters property.
> +                  This list must be same length as mmu-masters, so
> +                  masters with multiple stream-ids will have multiple
> +                  entries in mmu-masters.

Your example below is actually 32-bit values in the DTB. You can
annotate them to actually be 16-bit if you want. But I would just leave
them as 32-bit.

> +
> +** System MMU optional properties:
> +
> +- smmu-parent   : When multiple SMMUs are chained together, this
> +                  property can be used to provide a phandle to the
> +                  parent SMMU (that is the next SMMU on the path going
> +                  from the mmu-masters towards memory) node for this
> +                  SMMU.

Does the SMMU need to know if it is coherent or not?

Rob

> +
> +Example:
> +
> +        smmu {
> +                compatible = "arm,smmu-v1";
> +                reg = <0xba5e0000 0x10000>;
> +                #global-interrupts = <2>;
> +                interrupts = <0 32 4>,
> +                             <0 33 4>,
> +                             <0 34 4>, /* This is the first context interrupt */
> +                             <0 35 4>,
> +                             <0 36 4>,
> +                             <0 37 4>;
> +                mmu-masters = <&dma0>,
> +                              <&dma0>,
> +                              <&dma1>;
> +                stream-ids  = <0xd01d>,
> +                              <0xd01e>,
> +                              <0xd11c>;
> +        };
> 

[PATCH] documentation: iommu: add description of ARM System MMU binding

[Will Deacon]

Hi Rob,

On Fri, Apr 05, 2013 at 05:43:06PM +0100, Rob Herring wrote:
> On 04/04/2013 11:50 AM, Will Deacon wrote:
> > This patch adds a description of the device tree binding for the ARM
> > System MMU architecture.
> > 
> > Cc: Rob Herring <robherring2@gmail.com>
> > Cc: Andreas Herrmann <andreas.herrmann@calxeda.com>
> > Signed-off-by: Will Deacon <will.deacon@arm.com>
> > ---
> > 
> > Hello,
> > 
> > The driver for this is still a WIP. Both Andreas and myself have prototype
> > code, but we're planning to merge that together to get something more
> > general. Deciding on the binding is a good first step.
> 
> Thanks for getting this out.

No problem.

> > All comments welcome,
> > 
> > Will
> > 
> >  .../devicetree/bindings/iommu/arm,smmu.txt         | 61 ++++++++++++++++++++++
> >  1 file changed, 61 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/iommu/arm,smmu.txt
> > 
> > diff --git a/Documentation/devicetree/bindings/iommu/arm,smmu.txt b/Documentation/devicetree/bindings/iommu/arm,smmu.txt
> > new file mode 100644
> > index 0000000..938325f
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/iommu/arm,smmu.txt
> > @@ -0,0 +1,61 @@
> > +* ARM System MMU Architecture Implementation
> > +
> > +ARM SoCs may contain an implementation of the ARM System Memory
> > +Management Unit Architecture, which can be used to provide 1 or 2 stages
> > +of address translation to bus masters external to the CPU.
> > +
> > +The SMMU may also raise interrupts in response to various fault
> > +conditions.
> > +
> > +** System MMU required properties:
> > +
> > +- compatible    : Should be one of "arm,smmu-v1" or "arm,smmu-v2"
> > +                  depending on the version of the architecture
> > +                  implemented.
> 
> We can keep these, but we should have specific models like arm,smmu-400,
> etc. as well.

Ok, if distinctions need to be between MMU-400 and a v1 implementation, then
we can add those strings later.

> > +
> > +- reg           : Base address and size of the SMMU.
> > +
> > +- #global-interrupts : The number of global interrupts exposed by the
> > +                       device.
> > +
> > +- interrupts    : Interrupt list, with the first #global-irqs entries
> > +                  corresponding to the global interrupts and any
> > +                  following entries corresponding to context interrupts,
> > +                  specified in order of their indexing by the SMMU.
> > +
> > +- mmu-masters   : A list of phandles to device nodes representing bus
> > +                  masters for which the SMMU can provide a translation.
> > +
> > +- stream-ids    : A list of 16-bit values corresponding to the StreamIDs
> > +                  for the devices listed in the mmu-masters property.
> > +                  This list must be same length as mmu-masters, so
> > +                  masters with multiple stream-ids will have multiple
> > +                  entries in mmu-masters.
> 
> Your example below is actually 32-bit values in the DTB. You can
> annotate them to actually be 16-bit if you want. But I would just leave
> them as 32-bit.

I'm also parsing them as u32 in my driver, so yes, u32 it is!

> > +
> > +** System MMU optional properties:
> > +
> > +- smmu-parent   : When multiple SMMUs are chained together, this
> > +                  property can be used to provide a phandle to the
> > +                  parent SMMU (that is the next SMMU on the path going
> > +                  from the mmu-masters towards memory) node for this
> > +                  SMMU.
> 
> Does the SMMU need to know if it is coherent or not?

You mean with respect to table walks? That's actually probable from the
device (along with a whole bunch of other parameters) using SMMU_IDR0.

I'll fix up the other comments and send out a v2 next week.

Cheers for the review,

Will

[PATCH] documentation: iommu: add description of ARM System MMU binding

[Rob Herring]

On 04/05/2013 11:57 AM, Will Deacon wrote:
> Hi Rob,
> 
> On Fri, Apr 05, 2013 at 05:43:06PM +0100, Rob Herring wrote:

[...]

>>> +- compatible    : Should be one of "arm,smmu-v1" or "arm,smmu-v2"
>>> +                  depending on the version of the architecture
>>> +                  implemented.
>>
>> We can keep these, but we should have specific models like arm,smmu-400,
>> etc. as well.
> 
> Ok, if distinctions need to be between MMU-400 and a v1 implementation, then
> we can add those strings later.

No, you want to have specific values in the dtb's up front. If there is
something needed later in the kernel, you don't want to require a dtb
update then. Adding future parts later is fine, but we already know what
blocks we have.

Rob

[PATCH] documentation: iommu: add description of ARM System MMU binding

[Will Deacon]

On Fri, Apr 05, 2013 at 07:25:26PM +0100, Rob Herring wrote:
> On 04/05/2013 11:57 AM, Will Deacon wrote:
> > Hi Rob,
> > 
> > On Fri, Apr 05, 2013 at 05:43:06PM +0100, Rob Herring wrote:
> 
> [...]
> 
> >>> +- compatible    : Should be one of "arm,smmu-v1" or "arm,smmu-v2"
> >>> +                  depending on the version of the architecture
> >>> +                  implemented.
> >>
> >> We can keep these, but we should have specific models like arm,smmu-400,
> >> etc. as well.
> > 
> > Ok, if distinctions need to be between MMU-400 and a v1 implementation, then
> > we can add those strings later.
> 
> No, you want to have specific values in the dtb's up front. If there is
> something needed later in the kernel, you don't want to require a dtb
> update then. Adding future parts later is fine, but we already know what
> blocks we have.

Yes, you're right. I'll add those for v2.

Cheers,

Will

[PATCH] documentation: iommu: add description of ARM System MMU binding

[Olav Haugan]

On 4/4/2013 9:50 AM, Will Deacon wrote:
> This patch adds a description of the device tree binding for the ARM
> System MMU architecture.
> 
> Cc: Rob Herring <robherring2@gmail.com>
> Cc: Andreas Herrmann <andreas.herrmann@calxeda.com>
> Signed-off-by: Will Deacon <will.deacon@arm.com>
> ---
> 
> Hello,
> 
> The driver for this is still a WIP. Both Andreas and myself have prototype
> code, but we're planning to merge that together to get something more
> general. Deciding on the binding is a good first step.
> 
> All comments welcome,
> 
> Will
> 
>  .../devicetree/bindings/iommu/arm,smmu.txt         | 61 ++++++++++++++++++++++
>  1 file changed, 61 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/iommu/arm,smmu.txt
> 
> diff --git a/Documentation/devicetree/bindings/iommu/arm,smmu.txt b/Documentation/devicetree/bindings/iommu/arm,smmu.txt
> new file mode 100644
> index 0000000..938325f
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/iommu/arm,smmu.txt
> @@ -0,0 +1,61 @@
> +* ARM System MMU Architecture Implementation
> +
> +ARM SoCs may contain an implementation of the ARM System Memory
> +Management Unit Architecture, which can be used to provide 1 or 2 stages
> +of address translation to bus masters external to the CPU.
> +
> +The SMMU may also raise interrupts in response to various fault
> +conditions.
> +
> +** System MMU required properties:
> +
> +- compatible    : Should be one of "arm,smmu-v1" or "arm,smmu-v2"
> +                  depending on the version of the architecture
> +                  implemented.
> +
> +- reg           : Base address and size of the SMMU.
> +
> +- #global-interrupts : The number of global interrupts exposed by the
> +                       device.
> +
> +- interrupts    : Interrupt list, with the first #global-irqs entries
> +                  corresponding to the global interrupts and any
> +                  following entries corresponding to context interrupts,
> +                  specified in order of their indexing by the SMMU.
> +
> +- mmu-masters   : A list of phandles to device nodes representing bus
> +                  masters for which the SMMU can provide a translation.

I am not sure I understand the use of mmu-masters. I would imagine that
the bus masters themselves would have phandles to their respective SMMUs
that provides the translation.

> +
> +- stream-ids    : A list of 16-bit values corresponding to the StreamIDs
> +                  for the devices listed in the mmu-masters property.
> +                  This list must be same length as mmu-masters, so
> +                  masters with multiple stream-ids will have multiple
> +                  entries in mmu-masters.
> +

Why are the stream IDs (SIDs) coupled with the bus masters in this way?
You are probably following a different pattern than we do. Our SMMU
driver programs the SMMU SIDs and does not really know or care which
mmu-master it belongs to.

> +** System MMU optional properties:
> +
> +- smmu-parent   : When multiple SMMUs are chained together, this
> +                  property can be used to provide a phandle to the
> +                  parent SMMU (that is the next SMMU on the path going
> +                  from the mmu-masters towards memory) node for this
> +                  SMMU.
> +
> +Example:
> +
> +        smmu {
> +                compatible = "arm,smmu-v1";
> +                reg = <0xba5e0000 0x10000>;
> +                #global-interrupts = <2>;
> +                interrupts = <0 32 4>,
> +                             <0 33 4>,
> +                             <0 34 4>, /* This is the first context interrupt */
> +                             <0 35 4>,
> +                             <0 36 4>,
> +                             <0 37 4>;
> +                mmu-masters = <&dma0>,
> +                              <&dma0>,
> +                              <&dma1>;
> +                stream-ids  = <0xd01d>,
> +                              <0xd01e>,
> +                              <0xd11c>;
> +        };
> 

Hi Will,

Please see my comments above. I would like to work with you on defining
the bindings for System MMU. We have had System MMU DT bindings for some
time and I'd like to share what we are doing in hope that we can merge
something that works for all of us.

We use some of the same properties but we have a lot more. We also model
the context banks as children of each SMMU in an object-oriented way.

Here is our current binding for SMMUv1:

* Qualcomm MSM IOMMU v1

Required properties:
- compatible : one of:
	- "qcom,msm-smmu-v1"
- reg : offset and length of the register set for the device. Optional
	offset and length for clock register for additional clock that
	needs to be turned on for access to this IOMMU.
- reg-names: "iommu_base", "clk_base" (optional)
- label: name of this IOMMU instance.

Optional properties:
- qcom,iommu-secure-id : Secure identifier for the IOMMU block
- vdd-supply : vdd-supply: phandle to GDSC regulator controlling this IOMMU.
- qcom,alt-vdd-supply : Alternative regulator needed to access IOMMU
  configuration registers.
- interrupts : should contain the performance monitor overflow interrupt
number.
- qcom,iommu-enable-halt : Enable halt of the IOMMU before programming
certain	registers
- qcom,iommu-pmu-ngroups: Number of Performance Monitor Unit (PMU) groups.
- qcom,iommu-pmu-ncounters: Number of PMU counters per group.
- qcom,iommu-pmu-event-classes: List of event classes supported.
- qcom,needs-alt-core-clk : boolean to enable the secondary core clock
for access to the IOMMU configuration registers
- qcom,iommu-bfb-regs : An array of unsigned 32-bit integers
corresponding to BFB register addresses that need to be configured for
performance tuning purposes. If this property is present, the
qcom,iommu-bfb-data must also be present. Register addresses are
specified as an offset from the base of the IOMMU hardware block. This
property may be omitted if no BFB register configuration needs to be
done for a particular IOMMU hardware instance. The registers specified
by this property shall fall within the IOMMU implementation-defined
register region.
- qcom,iommu-bfb-data : An array of unsigned 32-bit integers
representing the values to be programmed into the corresponding
registers given by the qcom,iommu-bfb-regs property. If this property is
present, the qcom,iommu-bfb-regs property shall also be present, and the
lengths of both properties shall be the same.

- List of sub nodes, one for each of the translation context banks
supported.
  Each sub node has the following required properties:

  - reg : offset and length of the register set for the context bank.
  - interrupts : should contain the context bank interrupt.
  - qcom,iommu-ctx-sids : List of stream identifiers associated with
this translation context.
  - label : Name of the context bank

Optional sub-node properties:
   - qcom,secure-context : boolean indicating that a context is secure
 and programmed by the secure environment.

Example:

	qcom,iommu at fda64000 {
		compatible = "qcom,msm-smmu-v1";
		reg = <0xfda64000 0x10000>;
		reg-names = "iommu_base";
		vdd-supply = <&gdsc_iommu>;
		qcom,iommu-bfb-regs = <0x204c 0x2050>;
		qcom,iommu-bfb-data = <0xffff 0xffce>;
		label = "iommu_0";
		qcom,iommu-pmu-ngroups = <1>;
		qcom,iommu-pmu-ncounters = <8>;
		qcom,iommu-pmu-event-classes = <0x00,
						0x01>;

		qcom,iommu-ctx at fda6c000 {
			reg = <0xfda6c000 0x1000>;
			interrupts = <0 70 0>;
			qcom,iommu-ctx-sids = <0 2>;
			label = "ctx_0";
		};
		qcom,iommu-ctx at fda6d000 {
			reg = <0xfda6d000 0x1000>;
			interrupts = <0 70 0>;
			qcom,iommu-ctx-sids = <1>;
			label = "ctx_1";
		};
	};

Please let me know your thoughts on this.

Thanks,

Olav Haugan

-- 
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

[PATCH] documentation: iommu: add description of ARM System MMU binding

[Will Deacon]

Hi Olav,

On Fri, Apr 05, 2013 at 09:44:49PM +0100, Olav Haugan wrote:
> On 4/4/2013 9:50 AM, Will Deacon wrote:
> > diff --git a/Documentation/devicetree/bindings/iommu/arm,smmu.txt b/Documentation/devicetree/bindings/iommu/arm,smmu.txt
> > new file mode 100644
> > index 0000000..938325f
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/iommu/arm,smmu.txt
> > @@ -0,0 +1,61 @@
> > +* ARM System MMU Architecture Implementation
> > +
> > +ARM SoCs may contain an implementation of the ARM System Memory
> > +Management Unit Architecture, which can be used to provide 1 or 2 stages
> > +of address translation to bus masters external to the CPU.
> > +
> > +The SMMU may also raise interrupts in response to various fault
> > +conditions.
> > +
> > +** System MMU required properties:
> > +
> > +- compatible    : Should be one of "arm,smmu-v1" or "arm,smmu-v2"
> > +                  depending on the version of the architecture
> > +                  implemented.
> > +
> > +- reg           : Base address and size of the SMMU.
> > +
> > +- #global-interrupts : The number of global interrupts exposed by the
> > +                       device.
> > +
> > +- interrupts    : Interrupt list, with the first #global-irqs entries
> > +                  corresponding to the global interrupts and any
> > +                  following entries corresponding to context interrupts,
> > +                  specified in order of their indexing by the SMMU.
> > +
> > +- mmu-masters   : A list of phandles to device nodes representing bus
> > +                  masters for which the SMMU can provide a translation.
> 
> I am not sure I understand the use of mmu-masters. I would imagine that
> the bus masters themselves would have phandles to their respective SMMUs
> that provides the translation.

The problem with that is when you have chained SMMUs. E.g., a separate SMMU
for stage 1 and stage 2, where the StreamIDs are not the same going into the
second SMMU on the chain. The set of masters and StreamIDs coming into a
system MMU is really a property of that system MMU, and is determined at
integration time.

> > +
> > +- stream-ids    : A list of 16-bit values corresponding to the StreamIDs
> > +                  for the devices listed in the mmu-masters property.
> > +                  This list must be same length as mmu-masters, so
> > +                  masters with multiple stream-ids will have multiple
> > +                  entries in mmu-masters.
> > +
> 
> Why are the stream IDs (SIDs) coupled with the bus masters in this way?
> You are probably following a different pattern than we do. Our SMMU
> driver programs the SMMU SIDs and does not really know or care which
> mmu-master it belongs to.

Generally, the StreamIDs are fixed in hardware (as a function of various AXI
bits -- see the SMMU integration guide) and cannot be set by software.
Furthermore, when the StreamIDs have an implicit effect on IOMMU domain
configuration, since stream-matching may not be always be able to identify a
master uniquely.

> Please see my comments above. I would like to work with you on defining
> the bindings for System MMU. We have had System MMU DT bindings for some
> time and I'd like to share what we are doing in hope that we can merge
> something that works for all of us.
> 
> We use some of the same properties but we have a lot more. We also model
> the context banks as children of each SMMU in an object-oriented way.

Hmm, what you have looks really... complicated. Why do you need so much stuff?

> Here is our current binding for SMMUv1:
> 
> * Qualcomm MSM IOMMU v1
> 
> Required properties:
> - compatible : one of:
> 	- "qcom,msm-smmu-v1"
> - reg : offset and length of the register set for the device. Optional
> 	offset and length for clock register for additional clock that
> 	needs to be turned on for access to this IOMMU.
> - reg-names: "iommu_base", "clk_base" (optional)

Since I'm just working on a software model, I've not gone near clocks.
However, clocks are fairly well understood so we could easily add those
later if they're needed.

> - label: name of this IOMMU instance.

What?

> Optional properties:
> - qcom,iommu-secure-id : Secure identifier for the IOMMU block
> - vdd-supply : vdd-supply: phandle to GDSC regulator controlling this IOMMU.
> - qcom,alt-vdd-supply : Alternative regulator needed to access IOMMU
>   configuration registers.
> - interrupts : should contain the performance monitor overflow interrupt
> number.
> - qcom,iommu-enable-halt : Enable halt of the IOMMU before programming
> certain	registers
> - qcom,iommu-pmu-ngroups: Number of Performance Monitor Unit (PMU) groups.
> - qcom,iommu-pmu-ncounters: Number of PMU counters per group.
> - qcom,iommu-pmu-event-classes: List of event classes supported.
> - qcom,needs-alt-core-clk : boolean to enable the secondary core clock
> for access to the IOMMU configuration registers
> - qcom,iommu-bfb-regs : An array of unsigned 32-bit integers
> corresponding to BFB register addresses that need to be configured for
> performance tuning purposes. If this property is present, the
> qcom,iommu-bfb-data must also be present. Register addresses are
> specified as an offset from the base of the IOMMU hardware block. This
> property may be omitted if no BFB register configuration needs to be
> done for a particular IOMMU hardware instance. The registers specified
> by this property shall fall within the IOMMU implementation-defined
> register region.
> - qcom,iommu-bfb-data : An array of unsigned 32-bit integers
> representing the values to be programmed into the corresponding
> registers given by the qcom,iommu-bfb-regs property. If this property is
> present, the qcom,iommu-bfb-regs property shall also be present, and the
> lengths of both properties shall be the same.

This really looks specific to your implementation/integration and I can't
see that we'd want this in the binding to be honest. It would be good to
have PMU support in the future, but I've not thought about that part yet.

> - List of sub nodes, one for each of the translation context banks
> supported.
>   Each sub node has the following required properties:
> 
>   - reg : offset and length of the register set for the context bank.

Why do you need this? The structure of the context banks is well defined by
the SMMU architecture.

>   - interrupts : should contain the context bank interrupt.
>   - qcom,iommu-ctx-sids : List of stream identifiers associated with
> this translation context.

StreamIDs aren't statically associated with a translation context. Why do
you put them here? Also, how do this interact with stream matching?

>   - label : Name of the context bank

Again: what?

> Optional sub-node properties:
>    - qcom,secure-context : boolean indicating that a context is secure
>  and programmed by the secure environment.

Why does Linux care about this?

> Please let me know your thoughts on this.

I'd really rather start off small, and describe precisely what we need to
get architecturally-compliant SMMUs off the ground. Then, as the code grows
to use more features (PM, PMU, ...) then we can extend the binding.
Otherwise, we paint ourselves into a corner with the binding before we've
developed any driver code.

Will

[PATCH] documentation: iommu: add description of ARM System MMU binding

[Olav Haugan]

Hi Will,

On 4/8/2013 2:25 AM, Will Deacon wrote:
> Hi Olav,
> 
> On Fri, Apr 05, 2013 at 09:44:49PM +0100, Olav Haugan wrote:
>> On 4/4/2013 9:50 AM, Will Deacon wrote:
>>> diff --git a/Documentation/devicetree/bindings/iommu/arm,smmu.txt b/Documentation/devicetree/bindings/iommu/arm,smmu.txt
>>> new file mode 100644
>>> index 0000000..938325f
>>> --- /dev/null
>>> +++ b/Documentation/devicetree/bindings/iommu/arm,smmu.txt
>>> @@ -0,0 +1,61 @@
>>> +* ARM System MMU Architecture Implementation
>>> +
>>> +ARM SoCs may contain an implementation of the ARM System Memory
>>> +Management Unit Architecture, which can be used to provide 1 or 2 stages
>>> +of address translation to bus masters external to the CPU.
>>> +
>>> +The SMMU may also raise interrupts in response to various fault
>>> +conditions.
>>> +
>>> +** System MMU required properties:
>>> +
>>> +- compatible    : Should be one of "arm,smmu-v1" or "arm,smmu-v2"
>>> +                  depending on the version of the architecture
>>> +                  implemented.
>>> +
>>> +- reg           : Base address and size of the SMMU.
>>> +
>>> +- #global-interrupts : The number of global interrupts exposed by the
>>> +                       device.
>>> +
>>> +- interrupts    : Interrupt list, with the first #global-irqs entries
>>> +                  corresponding to the global interrupts and any
>>> +                  following entries corresponding to context interrupts,
>>> +                  specified in order of their indexing by the SMMU.
>>> +
>>> +- mmu-masters   : A list of phandles to device nodes representing bus
>>> +                  masters for which the SMMU can provide a translation.
>>
>> I am not sure I understand the use of mmu-masters. I would imagine that
>> the bus masters themselves would have phandles to their respective SMMUs
>> that provides the translation.
> 
> The problem with that is when you have chained SMMUs. E.g., a separate SMMU
> for stage 1 and stage 2, where the StreamIDs are not the same going into the
> second SMMU on the chain. The set of masters and StreamIDs coming into a
> system MMU is really a property of that system MMU, and is determined at
> integration time.
> 
>>> +
>>> +- stream-ids    : A list of 16-bit values corresponding to the StreamIDs
>>> +                  for the devices listed in the mmu-masters property.
>>> +                  This list must be same length as mmu-masters, so
>>> +                  masters with multiple stream-ids will have multiple
>>> +                  entries in mmu-masters.
>>> +
>>
>> Why are the stream IDs (SIDs) coupled with the bus masters in this way?
>> You are probably following a different pattern than we do. Our SMMU
>> driver programs the SMMU SIDs and does not really know or care which
>> mmu-master it belongs to.
> 
> Generally, the StreamIDs are fixed in hardware (as a function of various AXI
> bits -- see the SMMU integration guide) and cannot be set by software.
> Furthermore, when the StreamIDs have an implicit effect on IOMMU domain
> configuration, since stream-matching may not be always be able to identify a
> master uniquely.

Let me clarify what I mean by "our SMMU driver programs the SMMU SIDs".
What I meant to say is that our SMMU driver programs the SID into the
Stream Match Register (SMR) to route the transactions to the correct
context bank based on the SID.

>> Please see my comments above. I would like to work with you on defining
>> the bindings for System MMU. We have had System MMU DT bindings for some
>> time and I'd like to share what we are doing in hope that we can merge
>> something that works for all of us.
>>
>> We use some of the same properties but we have a lot more. We also model
>> the context banks as children of each SMMU in an object-oriented way.
> 
> Hmm, what you have looks really... complicated. Why do you need so much stuff?

Yes, some of the stuff is implementation specific.

>> Here is our current binding for SMMUv1:
>>
>> * Qualcomm MSM IOMMU v1
>>
>> Required properties:
>> - compatible : one of:
>> 	- "qcom,msm-smmu-v1"
>> - reg : offset and length of the register set for the device. Optional
>> 	offset and length for clock register for additional clock that
>> 	needs to be turned on for access to this IOMMU.
>> - reg-names: "iommu_base", "clk_base" (optional)
> 
> Since I'm just working on a software model, I've not gone near clocks.
> However, clocks are fairly well understood so we could easily add those
> later if they're needed.
> 
>> - label: name of this IOMMU instance.
> 
> What?

This is mostly used for debugging allowing us to log a name that is more
user friendly instead of something like "fda64000.qcom,iommu" as the
device name. We also use this to create debugfs directory with a user
friendly name.

> 
>> Optional properties:
>> - qcom,iommu-secure-id : Secure identifier for the IOMMU block
>> - vdd-supply : vdd-supply: phandle to GDSC regulator controlling this IOMMU.
>> - qcom,alt-vdd-supply : Alternative regulator needed to access IOMMU
>>   configuration registers.
>> - interrupts : should contain the performance monitor overflow interrupt
>> number.
>> - qcom,iommu-enable-halt : Enable halt of the IOMMU before programming
>> certain	registers
>> - qcom,iommu-pmu-ngroups: Number of Performance Monitor Unit (PMU) groups.
>> - qcom,iommu-pmu-ncounters: Number of PMU counters per group.
>> - qcom,iommu-pmu-event-classes: List of event classes supported.
>> - qcom,needs-alt-core-clk : boolean to enable the secondary core clock
>> for access to the IOMMU configuration registers
>> - qcom,iommu-bfb-regs : An array of unsigned 32-bit integers
>> corresponding to BFB register addresses that need to be configured for
>> performance tuning purposes. If this property is present, the
>> qcom,iommu-bfb-data must also be present. Register addresses are
>> specified as an offset from the base of the IOMMU hardware block. This
>> property may be omitted if no BFB register configuration needs to be
>> done for a particular IOMMU hardware instance. The registers specified
>> by this property shall fall within the IOMMU implementation-defined
>> register region.
>> - qcom,iommu-bfb-data : An array of unsigned 32-bit integers
>> representing the values to be programmed into the corresponding
>> registers given by the qcom,iommu-bfb-regs property. If this property is
>> present, the qcom,iommu-bfb-regs property shall also be present, and the
>> lengths of both properties shall be the same.
> 
> This really looks specific to your implementation/integration and I can't
> see that we'd want this in the binding to be honest. It would be good to
> have PMU support in the future, but I've not thought about that part yet.

Yes, some of this is specific to our implementation. However, the
properties related to regulators and PMU are not specific I believe. How
do you do handle regulators?

>> - List of sub nodes, one for each of the translation context banks
>> supported.
>>   Each sub node has the following required properties:
>>
>>   - reg : offset and length of the register set for the context bank.
> 
> Why do you need this? The structure of the context banks is well defined by
> the SMMU architecture.

Since we specify the context banks as children of the IOMMU we specify
the address of each node. I believe you should be specifying the "reg"
property for the nodes when you do this.

>>   - interrupts : should contain the context bank interrupt.
>>   - qcom,iommu-ctx-sids : List of stream identifiers associated with
>> this translation context.
> 
> StreamIDs aren't statically associated with a translation context. Why do
> you put them here? Also, how do this interact with stream matching?

We put the StreamIDs in the context bank to tell our driver that these
StreamIDs should be programmed into the Stream Match Register (SMR) to
route the transactions bearing these StreamIDs to this context bank. We
don't really change the SID2CB mapping during run-time. It is set during
compile time.

>>   - label : Name of the context bank
> 
> Again: what?

Same idea as the label in the IOMMU parent node. We also use this to
allow clients to find context bank devices to attach to based on the name.

>> Optional sub-node properties:
>>    - qcom,secure-context : boolean indicating that a context is secure
>>  and programmed by the secure environment.
> 
> Why does Linux care about this?

Linux cares about this because if we are doing content protection using
one of the context banks then the context bank is programmed by the
secure environment and not accessible/programmable by the HLOS.

> 
>> Please let me know your thoughts on this.
> 
> I'd really rather start off small, and describe precisely what we need to
> get architecturally-compliant SMMUs off the ground. Then, as the code grows
> to use more features (PM, PMU, ...) then we can extend the binding.
> Otherwise, we paint ourselves into a corner with the binding before we've
> developed any driver code.

Sure, I did not mean to add implementation specific stuff to upstream
bindings. Does it makes sense to you thought the way we modeled context
banks as child nodes?

Thanks,

Olav Haugan

-- 
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

[PATCH] documentation: iommu: add description of ARM System MMU binding

[Will Deacon]

On Mon, Apr 08, 2013 at 06:03:54PM +0100, Olav Haugan wrote:
> Hi Will,

Hello,

> > Generally, the StreamIDs are fixed in hardware (as a function of various AXI
> > bits -- see the SMMU integration guide) and cannot be set by software.
> > Furthermore, when the StreamIDs have an implicit effect on IOMMU domain
> > configuration, since stream-matching may not be always be able to identify a
> > master uniquely.
> 
> Let me clarify what I mean by "our SMMU driver programs the SMMU SIDs".
> What I meant to say is that our SMMU driver programs the SID into the
> Stream Match Register (SMR) to route the transactions to the correct
> context bank based on the SID.

Ok, but then how does that interact with the IOMMU API in Linux? e.g. if a
client asks for an iova <-> pa mapping on a device, then you need to lookup
the StreamIDs for that device, no?

> > This really looks specific to your implementation/integration and I can't
> > see that we'd want this in the binding to be honest. It would be good to
> > have PMU support in the future, but I've not thought about that part yet.
> 
> Yes, some of this is specific to our implementation. However, the
> properties related to regulators and PMU are not specific I believe. How
> do you do handle regulators?

As I said, I'm just using a software model more the moment, so no clocks or
regulators. We can add these later, since Linux has good support for things
like that already.

> >> - List of sub nodes, one for each of the translation context banks
> >> supported.
> >>   Each sub node has the following required properties:
> >>
> >>   - reg : offset and length of the register set for the context bank.
> > 
> > Why do you need this? The structure of the context banks is well defined by
> > the SMMU architecture.
> 
> Since we specify the context banks as children of the IOMMU we specify
> the address of each node. I believe you should be specifying the "reg"
> property for the nodes when you do this.

It's completely redundant information though!

> >>   - interrupts : should contain the context bank interrupt.
> >>   - qcom,iommu-ctx-sids : List of stream identifiers associated with
> >> this translation context.
> > 
> > StreamIDs aren't statically associated with a translation context. Why do
> > you put them here? Also, how do this interact with stream matching?
> 
> We put the StreamIDs in the context bank to tell our driver that these
> StreamIDs should be programmed into the Stream Match Register (SMR) to
> route the transactions bearing these StreamIDs to this context bank. We
> don't really change the SID2CB mapping during run-time. It is set during
> compile time.

Again, I'm struggling to see how this interfaces to the more general-purpose
IOMMU API.

> >> Optional sub-node properties:
> >>    - qcom,secure-context : boolean indicating that a context is secure
> >>  and programmed by the secure environment.
> > 
> > Why does Linux care about this?
> 
> Linux cares about this because if we are doing content protection using
> one of the context banks then the context bank is programmed by the
> secure environment and not accessible/programmable by the HLOS.

Why can't the secure world just hide those context banks by using
SMMU_SCR1.NSNUMCBO?

> > I'd really rather start off small, and describe precisely what we need to
> > get architecturally-compliant SMMUs off the ground. Then, as the code grows
> > to use more features (PM, PMU, ...) then we can extend the binding.
> > Otherwise, we paint ourselves into a corner with the binding before we've
> > developed any driver code.
> 
> Sure, I did not mean to add implementation specific stuff to upstream
> bindings. Does it makes sense to you thought the way we modeled context
> banks as child nodes?

Not really, no. It seems to configure everything up-front, which makes
dynamic use of the IOMMU difficult afaict. It may suit your use-case (as you
say, you even set up the mappings at compile-time) but it's not especially
general.

Will

[PATCH] documentation: iommu: add description of ARM System MMU binding

[Olav Haugan]

Hi again Will,

On 4/10/2013 10:37 AM, Will Deacon wrote:
> On Mon, Apr 08, 2013 at 06:03:54PM +0100, Olav Haugan wrote:
> 
>>> Generally, the StreamIDs are fixed in hardware (as a function of
>>> various AXI bits -- see the SMMU integration guide) and cannot be
>>> set by software. Furthermore, when the StreamIDs have an implicit
>>> effect on IOMMU domain configuration, since stream-matching may
>>> not be always be able to identify a master uniquely.
>> 
>> Let me clarify what I mean by "our SMMU driver programs the SMMU
>> SIDs". What I meant to say is that our SMMU driver programs the SID
>> into the Stream Match Register (SMR) to route the transactions to
>> the correct context bank based on the SID.
> 
> Ok, but then how does that interact with the IOMMU API in Linux? e.g.
> if a client asks for an iova <-> pa mapping on a device, then you
> need to lookup the StreamIDs for that device, no?

Can you explain this use case a bit more. I am not clear what you mean
by a "client"? Are you referring to a client in Linux or are you
referring to the client being the core behind an IOMMU (such as a DMA
engine or a video core)?

I am not sure how StreamIDs relates to the IOMMU API in Linux :-)
I think it helps if I explain my use case to you so you understand how
we use the IOMMU and maybe you can explain your use case and how your
driver works so that we can get on the same page. BTW, our ARMv1 IOMMU
driver can be found @ [1]

Our use case:

1) During boot IOMMU HW is probed, devices for each IOMMU and for each
context bank are created.
2) Video session starts.
3) Linux Video Driver (LVD) creates one or more domain
(iommu_domain_alloc) and calls iommu_attach_device() for each context
bank it is going to use.
4) LVD then calls map as needed for the buffers that the video core
needs to access.
5) LVD shares IOVA with video core and kicks off video core decoding.
6) Video session ends.
7) LVD unmaps
8) LVD detaches.

The video driver selects context bank to attach to based on the use case
and does not know about StreamIDs really.

BTW, when you call iommu_attach_device() what "device" do you pass as
argument? Do you pass the IOMMU as a device? How do you specify which
Context Bank to attach to?

>>> StreamIDs aren't statically associated with a translation
>>> context. Why do you put them here? Also, how do this interact
>>> with stream matching?
>> 
>> We put the StreamIDs in the context bank to tell our driver that
>> these StreamIDs should be programmed into the Stream Match Register
>> (SMR) to route the transactions bearing these StreamIDs to this
>> context bank. We don't really change the SID2CB mapping during
>> run-time. It is set during compile time.
> 
> Again, I'm struggling to see how this interfaces to the more
> general-purpose IOMMU API.

What general-purpose IOMMU API deals with StreamIDs?

>>>> Optional sub-node properties: - qcom,secure-context : boolean
>>>> indicating that a context is secure and programmed by the
>>>> secure environment.
>>> 
>>> Why does Linux care about this?
>> 
>> Linux cares about this because if we are doing content protection
>> using one of the context banks then the context bank is programmed
>> by the secure environment and not accessible/programmable by the
>> HLOS.
> 
> Why can't the secure world just hide those context banks by using 
> SMMU_SCR1.NSNUMCBO?

Yes, that is a possibility for context banks. If you have 1 or more
"secure" context banks it also means that Linux does not have access
(read: can't have) to the global register space and this must be
communicated somehow (hence the "secure-id" IOMMU node property. The
secure-context property might be redundant. However, there might be use
cases were Linux would want to know about the secure context bank. One
such use case could be to register for page fault from the secure
context bank so that Linux can log a useful page fault message to the
kernel log.

[1]
https://www.codeaurora.org/gitweb/quic/la/?p=kernel/msm.git;a=blob;f=drivers/iommu/msm_iommu-v2.c;h=e3aa30c6e31bf55ae5eea3800a8ad910545c5edb;hb=bd3e933fcaca9ffdb45ddc0ce341c45e1ee14091

Thanks,

Olav Haugan

-- 
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

[PATCH] documentation: iommu: add description of ARM System MMU binding

[Will Deacon]

On Sat, Apr 13, 2013 at 10:02:09PM +0100, Olav Haugan wrote:
> Hi again Will,

Hello Olav,

> On 4/10/2013 10:37 AM, Will Deacon wrote:
> > Ok, but then how does that interact with the IOMMU API in Linux? e.g.
> > if a client asks for an iova <-> pa mapping on a device, then you
> > need to lookup the StreamIDs for that device, no?
> 
> Can you explain this use case a bit more. I am not clear what you mean
> by a "client"? Are you referring to a client in Linux or are you
> referring to the client being the core behind an IOMMU (such as a DMA
> engine or a video core)?

Sure. By `client', I mean any use of the IOMMU API. It could be something
like your LVD example later on, it could be KVM or it could be indirectly
via dma-mapping. Basically, any user of <linux/iommu.h>.

> I am not sure how StreamIDs relates to the IOMMU API in Linux :-)

That's the thing; they don't, at least not explicitly. The StreamIDs must
all be handled inside the driver and it's actually what the driver would
deal with, rather than devices.

For example, if I create an IOMMU domain containing a DMA controller, that
means that I have associated a set of StreamIDs (i.e. those configured to
identify the DMA controller's master ports) with that domain. That may have
further consequences on which other devices can be put in the same domain
(stream-matching may require some IDs to sit in the same address space/domain)
and also effectively parameterises the domain, by pointing those StreamIDs
at the same context bank.

> I think it helps if I explain my use case to you so you understand how
> we use the IOMMU and maybe you can explain your use case and how your
> driver works so that we can get on the same page. BTW, our ARMv1 IOMMU
> driver can be found @ [1]
> 
> Our use case:
> 
> 1) During boot IOMMU HW is probed, devices for each IOMMU and for each
> context bank are created.

So you create a device for each context bank...

> 2) Video session starts.
> 3) Linux Video Driver (LVD) creates one or more domain
> (iommu_domain_alloc) and calls iommu_attach_device() for each context
> bank it is going to use.

...and attach the context bank to the domain dynamically?

If so, doesn't that strongly tie your video driver to the SMMU?

> 4) LVD then calls map as needed for the buffers that the video core
> needs to access.
> 5) LVD shares IOVA with video core and kicks off video core decoding.
> 6) Video session ends.
> 7) LVD unmaps
> 8) LVD detaches.
> 
> The video driver selects context bank to attach to based on the use case
> and does not know about StreamIDs really.
> 
> BTW, when you call iommu_attach_device() what "device" do you pass as
> argument? Do you pass the IOMMU as a device? How do you specify which
> Context Bank to attach to?

This seems to be where we differ. My anticipated use-case is that a domain
is allocated, which defines an empty address space. Masters are then
attached to the domain by passing their struct device, so this may
correspond to a DMA controller or a video core, in your case. The context
bank is allocated dynamically when the first device is added to the
domain, and then it is subsequently shared between all the masters in that
domain.

I think this also matches what dma-mapping.c expects.

> > Why can't the secure world just hide those context banks by using 
> > SMMU_SCR1.NSNUMCBO?
> 
> Yes, that is a possibility for context banks. If you have 1 or more
> "secure" context banks it also means that Linux does not have access
> (read: can't have) to the global register space and this must be
> communicated somehow (hence the "secure-id" IOMMU node property. The
> secure-context property might be redundant. However, there might be use
> cases were Linux would want to know about the secure context bank. One
> such use case could be to register for page fault from the secure
> context bank so that Linux can log a useful page fault message to the
> kernel log.

Perhaps, but I don't think it's worth complicating the binding based on a
potential nice-to-have fault message.

Will

[PATCH] documentation: iommu: add description of ARM System MMU binding

[Olav Haugan]

On 4/15/2013 6:13 AM, Will Deacon wrote:
> On Sat, Apr 13, 2013 at 10:02:09PM +0100, Olav Haugan wrote:
>> On 4/10/2013 10:37 AM, Will Deacon wrote:
>>> Ok, but then how does that interact with the IOMMU API in Linux? e.g.
>>> if a client asks for an iova <-> pa mapping on a device, then you
>>> need to lookup the StreamIDs for that device, no?
>>
>> Can you explain this use case a bit more. I am not clear what you mean
>> by a "client"? Are you referring to a client in Linux or are you
>> referring to the client being the core behind an IOMMU (such as a DMA
>> engine or a video core)?
> 
> Sure. By `client', I mean any use of the IOMMU API. It could be something
> like your LVD example later on, it could be KVM or it could be indirectly
> via dma-mapping. Basically, any user of <linux/iommu.h>.

Ok, so if I do a VA2PA on a domain (calling iommu_iova_to_phys()) you
don't need any information about StreamID (no need to look up the
StreamID). You would just walk the page table of the domain and find the
address (or use ATS on the IOMMU if supported by the HW).

>> I am not sure how StreamIDs relates to the IOMMU API in Linux :-)
> 
> That's the thing; they don't, at least not explicitly. The StreamIDs must
> all be handled inside the driver and it's actually what the driver would
> deal with, rather than devices.
> 
> For example, if I create an IOMMU domain containing a DMA controller, that
> means that I have associated a set of StreamIDs (i.e. those configured to
> identify the DMA controller's master ports) with that domain. That may have
> further consequences on which other devices can be put in the same domain
> (stream-matching may require some IDs to sit in the same address space/domain)
> and also effectively parameterises the domain, by pointing those StreamIDs
> at the same context bank.

Yes, this is clear to me.

>> I think it helps if I explain my use case to you so you understand how
>> we use the IOMMU and maybe you can explain your use case and how your
>> driver works so that we can get on the same page. BTW, our ARMv1 IOMMU
>> driver can be found @ [1]
>>
>> Our use case:
>>
>> 1) During boot IOMMU HW is probed, devices for each IOMMU and for each
>> context bank are created.
> 
> So you create a device for each context bank...
> 
>> 2) Video session starts.
>> 3) Linux Video Driver (LVD) creates one or more domain
>> (iommu_domain_alloc) and calls iommu_attach_device() for each context
>> bank it is going to use.
> 
> ...and attach the context bank to the domain dynamically?

Yes

> If so, doesn't that strongly tie your video driver to the SMMU?

Isn't this more or less what you are doing in DT where you associate
specific devices with an IOMMU (mmu-masters)?

>> The video driver selects context bank to attach to based on the use case
>> and does not know about StreamIDs really.
>>
>> BTW, when you call iommu_attach_device() what "device" do you pass as
>> argument? Do you pass the IOMMU as a device? How do you specify which
>> Context Bank to attach to?
> 
> This seems to be where we differ. My anticipated use-case is that a domain
> is allocated, which defines an empty address space. Masters are then
> attached to the domain by passing their struct device, so this may
> correspond to a DMA controller or a video core, in your case. The context
> bank is allocated dynamically when the first device is added to the
> domain, and then it is subsequently shared between all the masters in that
> domain.

I feel that there are some limitation with this. Maybe you can address
the following:

1) What if you want two context banks in one domain (shared page table)?
2) What if a master (device) needs to use 2 or more context banks?

>>> Why can't the secure world just hide those context banks by using 
>>> SMMU_SCR1.NSNUMCBO?
>>
>> Yes, that is a possibility for context banks. If you have 1 or more
>> "secure" context banks it also means that Linux does not have access
>> (read: can't have) to the global register space and this must be
>> communicated somehow (hence the "secure-id" IOMMU node property. The
>> secure-context property might be redundant. However, there might be use
>> cases were Linux would want to know about the secure context bank. One
>> such use case could be to register for page fault from the secure
>> context bank so that Linux can log a useful page fault message to the
>> kernel log.
> 
> Perhaps, but I don't think it's worth complicating the binding based on a
> potential nice-to-have fault message.
> 

However, global register space (including Stream Match Registers [SMR])
needs to be programmed by the secure environment for security or by
hyper-visor if you are in such an environment.

Thanks,

Olav Haugan

-- 
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

[PATCH] documentation: iommu: add description of ARM System MMU binding

[Will Deacon]

Hi Olav,

On Tue, Apr 16, 2013 at 07:18:42PM +0100, Olav Haugan wrote:
> On 4/15/2013 6:13 AM, Will Deacon wrote:
> > If so, doesn't that strongly tie your video driver to the SMMU?
> 
> Isn't this more or less what you are doing in DT where you associate
> specific devices with an IOMMU (mmu-masters)?

No. The device-tree describes the *hardware*, as per usual. The StreamIDs
are fixed properties of the SoC and we can't change them from Linux, so we
describe all of the StreamIDs upstream of each SMMU so that we can program
the thing. There's no way to generic way to discover them.

> > This seems to be where we differ. My anticipated use-case is that a domain
> > is allocated, which defines an empty address space. Masters are then
> > attached to the domain by passing their struct device, so this may
> > correspond to a DMA controller or a video core, in your case. The context
> > bank is allocated dynamically when the first device is added to the
> > domain, and then it is subsequently shared between all the masters in that
> > domain.
> 
> I feel that there are some limitation with this. Maybe you can address
> the following:
> 
> 1) What if you want two context banks in one domain (shared page table)?

Why would a page table shared between devices require multiple context
banks? Multiple SMRs and S2CRs, sure, but they would ultimately point at the
same context bank (and hence same address space).

> 2) What if a master (device) needs to use 2 or more context banks?

If a master needs to be in two address spaces at once, then it will need to
attach it's StreamIDs to different domains. You can't place a single
StreamID in two address spaces (this is an architectural constraint).

> > Perhaps, but I don't think it's worth complicating the binding based on a
> > potential nice-to-have fault message.
> > 
> 
> However, global register space (including Stream Match Registers [SMR])
> needs to be programmed by the secure environment for security or by
> hyper-visor if you are in such an environment.

Of course, but that's not Linux (and KVM would likely have some additional
code if it wanted to do device passthrough, for example).

Will

[PATCH] documentation: iommu: add description of ARM System MMU binding

[Olav Haugan]

Hi Will,

On 4/18/2013 12:01 PM, Will Deacon wrote:
> On Tue, Apr 16, 2013 at 07:18:42PM +0100, Olav Haugan wrote:
>> On 4/15/2013 6:13 AM, Will Deacon wrote:
>>> If so, doesn't that strongly tie your video driver to the SMMU?
>>
>> Isn't this more or less what you are doing in DT where you associate
>> specific devices with an IOMMU (mmu-masters)?
> 
> No. The device-tree describes the *hardware*, as per usual. The StreamIDs
> are fixed properties of the SoC and we can't change them from Linux, so we
> describe all of the StreamIDs upstream of each SMMU so that we can program
> the thing. There's no way to generic way to discover them.

I meant that you are putting phandles to bus masters in the device tree
that use/need the SMMU for VA2PA translation. Doesn't this put a
dependency on the bus master devices? How will this work? Lets say that
we have a device during bootup that needs to use the SMMU (such as a
display processor [DP]). Don't you have cyclic dependency? The SMMU
device needs the DP device (phandle) but the DP device needs the SMMU to
initialize its device driver?

>>> This seems to be where we differ. My anticipated use-case is that a domain
>>> is allocated, which defines an empty address space. Masters are then
>>> attached to the domain by passing their struct device, so this may
>>> correspond to a DMA controller or a video core, in your case. The context
>>> bank is allocated dynamically when the first device is added to the
>>> domain, and then it is subsequently shared between all the masters in that
>>> domain.
>>
>> I feel that there are some limitation with this. Maybe you can address
>> the following:
>>
>> 1) What if you want two context banks in one domain (shared page table)?
> 
> Why would a page table shared between devices require multiple context
> banks? Multiple SMRs and S2CRs, sure, but they would ultimately point at the
> same context bank (and hence same address space).

What if you want to have 2 different VMID's being generated? Also, what
about TLB management? If I have two context banks I can invalidate only
entries associated with 1 of the context banks (if VMID differ for the
two context banks).

>> 2) What if a master (device) needs to use 2 or more context banks?
> 
> If a master needs to be in two address spaces at once, then it will need to
> attach it's StreamIDs to different domains. You can't place a single
> StreamID in two address spaces (this is an architectural constraint).

Yes, you would have a separate domain. I am just wondering how I would
model this in DT using the bindings that you are proposing? How does it
work? The bindings specify bus masters to StreamIDs. So if I call attach
with the master device you will allocate a context bank and program the
StreamIDs as specified in the DT. So now if I want to have another
context bank associated with the same master device what do I do? I call
into attach with a new domain but with the same master device but the
master device is already attached to a context/domain.

Thanks,

Olav Haugan

-- 
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

[PATCH] documentation: iommu: add description of ARM System MMU binding

[Will Deacon]

On Tue, Apr 23, 2013 at 11:54:53PM +0100, Olav Haugan wrote:
> Hi Will,

Hello again,

> On 4/18/2013 12:01 PM, Will Deacon wrote:
> > No. The device-tree describes the *hardware*, as per usual. The StreamIDs
> > are fixed properties of the SoC and we can't change them from Linux, so we
> > describe all of the StreamIDs upstream of each SMMU so that we can program
> > the thing. There's no way to generic way to discover them.
> 
> I meant that you are putting phandles to bus masters in the device tree
> that use/need the SMMU for VA2PA translation. Doesn't this put a
> dependency on the bus master devices? How will this work? Lets say that
> we have a device during bootup that needs to use the SMMU (such as a
> display processor [DP]). Don't you have cyclic dependency? The SMMU
> device needs the DP device (phandle) but the DP device needs the SMMU to
> initialize its device driver?

That's a problem you have with the driver model anyway. The SMMU must be
registered on the same bus as the device before it can be used and the
devices must be added to that bus before they can be attached to a domain.

Getting phandles has no dependencies on anything -- the only dependency is
that the device is added to the bus on which the SMMU sits, just like every
other IOMMU driver.

> > Why would a page table shared between devices require multiple context
> > banks? Multiple SMRs and S2CRs, sure, but they would ultimately point at the
> > same context bank (and hence same address space).
> 
> What if you want to have 2 different VMID's being generated? Also, what
> about TLB management? If I have two context banks I can invalidate only
> entries associated with 1 of the context banks (if VMID differ for the
> two context banks).

Huh? We allocate one VMID per domain. If you want more VMIDs, use more
domains. TLB invalidation is per-domain, so there's no issue there.

> > If a master needs to be in two address spaces at once, then it will need to
> > attach it's StreamIDs to different domains. You can't place a single
> > StreamID in two address spaces (this is an architectural constraint).
> 
> Yes, you would have a separate domain. I am just wondering how I would
> model this in DT using the bindings that you are proposing? How does it
> work? The bindings specify bus masters to StreamIDs. So if I call attach
> with the master device you will allocate a context bank and program the
> StreamIDs as specified in the DT. So now if I want to have another
> context bank associated with the same master device what do I do? I call
> into attach with a new domain but with the same master device but the
> master device is already attached to a context/domain.

Why would you want to place a StreamID into two domains? That doesn't make
any sense and isn't even supported by the architecture (things like
conflicting SMR entries may not even be reported).

Will

[PATCH] documentation: iommu: add description of ARM System MMU binding

[Olav Haugan]

Hi Will,

On 4/24/2013 2:55 AM, Will Deacon wrote:
> On Tue, Apr 23, 2013 at 11:54:53PM +0100, Olav Haugan wrote:
>> On 4/18/2013 12:01 PM, Will Deacon wrote:
> Getting phandles has no dependencies on anything -- the only dependency is
> that the device is added to the bus on which the SMMU sits, just like every
> other IOMMU driver.
> 
>>> Why would a page table shared between devices require multiple context
>>> banks? Multiple SMRs and S2CRs, sure, but they would ultimately point at the
>>> same context bank (and hence same address space).
>>
>> What if you want to have 2 different VMID's being generated? Also, what
>> about TLB management? If I have two context banks I can invalidate only
>> entries associated with 1 of the context banks (if VMID differ for the
>> two context banks).
> 
> Huh? We allocate one VMID per domain. If you want more VMIDs, use more
> domains. TLB invalidation is per-domain, so there's no issue there.

If we allocate a new domain you would have another address space (page
table), no? I was just trying to figure out if your binding proposal
would allow for 2 context banks in one domain which it does not seem to
support.

>>> If a master needs to be in two address spaces at once, then it will need to
>>> attach it's StreamIDs to different domains. You can't place a single
>>> StreamID in two address spaces (this is an architectural constraint).
>>
>> Yes, you would have a separate domain. I am just wondering how I would
>> model this in DT using the bindings that you are proposing? How does it
>> work? The bindings specify bus masters to StreamIDs. So if I call attach
>> with the master device you will allocate a context bank and program the
>> StreamIDs as specified in the DT. So now if I want to have another
>> context bank associated with the same master device what do I do? I call
>> into attach with a new domain but with the same master device but the
>> master device is already attached to a context/domain.
> 
> Why would you want to place a StreamID into two domains? That doesn't make
> any sense and isn't even supported by the architecture (things like
> conflicting SMR entries may not even be reported).

I think you misunderstood me. I am talking about having for example 1
master with two (2) context banks so that StreamID "1" goes to CB0 and
StreamID "2" goes to CB1. This means that the master device is attached
to two different domains. How do I model this with the the device tree
bindings?



Olav Haugan

-- 
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

[PATCH] documentation: iommu: add description of ARM System MMU binding

[Andreas Herrmann]

On Tue, May 07, 2013 at 04:26:02PM -0400, Olav Haugan wrote:
> Hi Will,
> 
> On 4/24/2013 2:55 AM, Will Deacon wrote:
> > On Tue, Apr 23, 2013 at 11:54:53PM +0100, Olav Haugan wrote:
> >> On 4/18/2013 12:01 PM, Will Deacon wrote:
> > Getting phandles has no dependencies on anything -- the only dependency is
> > that the device is added to the bus on which the SMMU sits, just like every
> > other IOMMU driver.
> > 
> >>> Why would a page table shared between devices require multiple context
> >>> banks? Multiple SMRs and S2CRs, sure, but they would ultimately point at the
> >>> same context bank (and hence same address space).
> >>
> >> What if you want to have 2 different VMID's being generated? Also, what
> >> about TLB management? If I have two context banks I can invalidate only
> >> entries associated with 1 of the context banks (if VMID differ for the
> >> two context banks).
> > 
> > Huh? We allocate one VMID per domain. If you want more VMIDs, use more
> > domains. TLB invalidation is per-domain, so there's no issue there.
> 
> If we allocate a new domain you would have another address space (page
> table), no? I was just trying to figure out if your binding proposal
> would allow for 2 context banks in one domain which it does not seem to
> support.
> 
> >>> If a master needs to be in two address spaces at once, then it will need to
> >>> attach it's StreamIDs to different domains. You can't place a single
> >>> StreamID in two address spaces (this is an architectural constraint).
> >>
> >> Yes, you would have a separate domain. I am just wondering how I would
> >> model this in DT using the bindings that you are proposing? How does it
> >> work? The bindings specify bus masters to StreamIDs. So if I call attach
> >> with the master device you will allocate a context bank and program the
> >> StreamIDs as specified in the DT. So now if I want to have another
> >> context bank associated with the same master device what do I do? I call
> >> into attach with a new domain but with the same master device but the
> >> master device is already attached to a context/domain.
> > 
> > Why would you want to place a StreamID into two domains? That doesn't make
> > any sense and isn't even supported by the architecture (things like
> > conflicting SMR entries may not even be reported).
> 
> I think you misunderstood me. I am talking about having for example 1
> master with two (2) context banks so that StreamID "1" goes to CB0 and
> StreamID "2" goes to CB1. This means that the master device is attached
> to two different domains. How do I model this with the the device tree
> bindings?

Hi Olav,

I think with the proposed device tree binding you can't model this.

Do we have such use cases already? Or what future use cases would
require this?

I can imagine of multiple contexts per device for stuff similar to
what PASID in PCI Express are used for.

But in such a case you probably want to have some configurable bits in
the StreamID (that should be set by an SMMU driver). And the DT
binding should contain the number of contexts that a master device can
support.


Andreas

[PATCH] documentation: iommu: add description of ARM System MMU binding

[Will Deacon]

On Mon, May 13, 2013 at 10:07:24AM +0100, Andreas Herrmann wrote:
> On Tue, May 07, 2013 at 04:26:02PM -0400, Olav Haugan wrote:
> > I think you misunderstood me. I am talking about having for example 1
> > master with two (2) context banks so that StreamID "1" goes to CB0 and
> > StreamID "2" goes to CB1. This means that the master device is attached
> > to two different domains. How do I model this with the the device tree
> > bindings?
> 
> Hi Olav,
> 
> I think with the proposed device tree binding you can't model this.
> 
> Do we have such use cases already? Or what future use cases would
> require this?
> 
> I can imagine of multiple contexts per device for stuff similar to
> what PASID in PCI Express are used for.
> 
> But in such a case you probably want to have some configurable bits in
> the StreamID (that should be set by an SMMU driver). And the DT
> binding should contain the number of contexts that a master device can
> support.

See my reply to your other thread, but I don't think this should be solved
in the SMMU driver. I really think that the device/domain configuration
should be driven by the master device drivers, rather than the device-tree.

Will

[PATCH] documentation: iommu: add description of ARM System MMU binding

[Olav Haugan]

Hi Andreas,

On 5/13/2013 2:07 AM, Andreas Herrmann wrote:
> On Tue, May 07, 2013 at 04:26:02PM -0400, Olav Haugan wrote:
>>
>>>>> If a master needs to be in two address spaces at once, then it will need to
>>>>> attach it's StreamIDs to different domains. You can't place a single
>>>>> StreamID in two address spaces (this is an architectural constraint).
>>>>
>>>> Yes, you would have a separate domain. I am just wondering how I would
>>>> model this in DT using the bindings that you are proposing? How does it
>>>> work? The bindings specify bus masters to StreamIDs. So if I call attach
>>>> with the master device you will allocate a context bank and program the
>>>> StreamIDs as specified in the DT. So now if I want to have another
>>>> context bank associated with the same master device what do I do? I call
>>>> into attach with a new domain but with the same master device but the
>>>> master device is already attached to a context/domain.
>>>
>>> Why would you want to place a StreamID into two domains? That doesn't make
>>> any sense and isn't even supported by the architecture (things like
>>> conflicting SMR entries may not even be reported).
>>
>> I think you misunderstood me. I am talking about having for example 1
>> master with two (2) context banks so that StreamID "1" goes to CB0 and
>> StreamID "2" goes to CB1. This means that the master device is attached
>> to two different domains. How do I model this with the the device tree
>> bindings?
> 
> Hi Olav,
> 
> I think with the proposed device tree binding you can't model this.
> 
> Do we have such use cases already? Or what future use cases would
> require this?
> 
> I can imagine of multiple contexts per device for stuff similar to
> what PASID in PCI Express are used for.
> 
> But in such a case you probably want to have some configurable bits in
> the StreamID (that should be set by an SMMU driver). And the DT
> binding should contain the number of contexts that a master device can
> support.
> 

Sorry for the late reply.

The only real use case at this time for a master having more than 1 CB
is for content protection I believe. I understand that this driver does
not currently support content protection or anything related to security.

Thanks,

Olav Haugan

-- 
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation