[PATCH v2 0/3] Enable PCIe ATS for devicetree boot

[PATCH v2 0/3] Enable PCIe ATS for devicetree boot

[Jean-Philippe Brucker]

Before enabling Address Translation Support (ATS) in endpoints, the OS
needs to confirm that the Root Complex supports it. Obtain this
information from the firmware description since there is no architected
method. ACPI provides a bit via IORT tables, so add the devicetree
equivalent.

Since v1 [1] I added the review and ack tags, thanks all. This should be
ready to go via the IOMMU tree.

[1] https://lore.kernel.org/all/20240429113938.192706-2-jean-philippe@linaro.org/

Jean-Philippe Brucker (3):
  dt-bindings: PCI: generic: Add ats-supported property
  iommu/of: Support ats-supported device-tree property
  arm64: dts: fvp: Enable PCIe ATS for Base RevC FVP

 .../devicetree/bindings/pci/host-generic-pci.yaml        | 6 ++++++
 drivers/iommu/of_iommu.c                                 | 9 +++++++++
 arch/arm64/boot/dts/arm/fvp-base-revc.dts                | 1 +
 3 files changed, 16 insertions(+)

-- 
2.45.2

[PATCH v2 1/3] dt-bindings: PCI: generic: Add ats-supported property

[Jean-Philippe Brucker]

Add a way for firmware to tell the OS that ATS is supported by the PCI
root complex. An endpoint with ATS enabled may send Translation Requests
and Translated Memory Requests, which look just like Normal Memory
Requests with a non-zero AT field. So a root controller that ignores the
AT field may simply forward the request to the IOMMU as a Normal Memory
Request, which could end badly. In any case, the endpoint will be
unusable.

The ats-supported property allows the OS to only enable ATS in endpoints
if the root controller can handle ATS requests. Only add the property to
pcie-host-ecam-generic for the moment. For non-generic root controllers,
availability of ATS can be inferred from the compatible string.

Reviewed-by: Rob Herring <robh@kernel.org>
Reviewed-by: Liviu Dudau <liviu.dudau@arm.com>
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
 Documentation/devicetree/bindings/pci/host-generic-pci.yaml | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/Documentation/devicetree/bindings/pci/host-generic-pci.yaml b/Documentation/devicetree/bindings/pci/host-generic-pci.yaml
index 3484e0b4b412e..bcfbaf5582cc9 100644
--- a/Documentation/devicetree/bindings/pci/host-generic-pci.yaml
+++ b/Documentation/devicetree/bindings/pci/host-generic-pci.yaml
@@ -110,6 +110,12 @@ properties:
   iommu-map-mask: true
   msi-parent: true
 
+  ats-supported:
+    description:
+      Indicates that a PCIe host controller supports ATS, and can handle Memory
+      Requests with Address Type (AT).
+    type: boolean
+
 required:
   - compatible
   - reg
-- 
2.45.2

[PATCH v2 2/3] iommu/of: Support ats-supported device-tree property

[Jean-Philippe Brucker]

Device-tree declares whether a PCI root-complex supports ATS by setting
the "ats-supported" property. Copy this flag into device fwspec to let
IOMMU drivers quickly check if they can enable ATS for a device.

Tested-by: Ketan Patil <ketanp@nvidia.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Liviu Dudau <liviu.dudau@arm.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
 drivers/iommu/of_iommu.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/drivers/iommu/of_iommu.c b/drivers/iommu/of_iommu.c
index 3afe0b48a48db..082b94c2b3291 100644
--- a/drivers/iommu/of_iommu.c
+++ b/drivers/iommu/of_iommu.c
@@ -105,6 +105,14 @@ static int of_iommu_configure_device(struct device_node *master_np,
 		      of_iommu_configure_dev(master_np, dev);
 }
 
+static void of_pci_check_device_ats(struct device *dev, struct device_node *np)
+{
+	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
+
+	if (fwspec && of_property_read_bool(np, "ats-supported"))
+		fwspec->flags |= IOMMU_FWSPEC_PCI_RC_ATS;
+}
+
 /*
  * Returns:
  *  0 on success, an iommu was configured
@@ -147,6 +155,7 @@ int of_iommu_configure(struct device *dev, struct device_node *master_np,
 		pci_request_acs();
 		err = pci_for_each_dma_alias(to_pci_dev(dev),
 					     of_pci_iommu_init, &info);
+		of_pci_check_device_ats(dev, master_np);
 	} else {
 		err = of_iommu_configure_device(master_np, dev, id);
 	}
-- 
2.45.2

[PATCH v2 3/3] arm64: dts: fvp: Enable PCIe ATS for Base RevC FVP

[Jean-Philippe Brucker]

Declare that the host controller supports ATS, so the OS can enable it
for ATS-capable PCIe endpoints.

Acked-by: Sudeep Holla <sudeep.holla@arm.com>
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
 arch/arm64/boot/dts/arm/fvp-base-revc.dts | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm64/boot/dts/arm/fvp-base-revc.dts b/arch/arm64/boot/dts/arm/fvp-base-revc.dts
index 60472d65a3557..85f1c15cc65d0 100644
--- a/arch/arm64/boot/dts/arm/fvp-base-revc.dts
+++ b/arch/arm64/boot/dts/arm/fvp-base-revc.dts
@@ -243,6 +243,7 @@ pci: pci@40000000 {
 		iommu-map = <0x0 &smmu 0x0 0x10000>;
 
 		dma-coherent;
+		ats-supported;
 	};
 
 	smmu: iommu@2b400000 {
-- 
2.45.2

Re: [PATCH v2 0/3] Enable PCIe ATS for devicetree boot

[Jean-Philippe Brucker]

Hi Joerg,

On Fri, Jun 07, 2024 at 11:54:13AM +0100, Jean-Philippe Brucker wrote:
> Before enabling Address Translation Support (ATS) in endpoints, the OS
> needs to confirm that the Root Complex supports it. Obtain this
> information from the firmware description since there is no architected
> method. ACPI provides a bit via IORT tables, so add the devicetree
> equivalent.
> 
> Since v1 [1] I added the review and ack tags, thanks all. This should be
> ready to go via the IOMMU tree.

This series enables ATS for devicetree boot, and is needed on an Nvidia
system: https://lore.kernel.org/linux-arm-kernel/ZeJP6CwrZ2FSbTYm@Asurada-Nvidia/

Would you mind picking it up for v6.11?

Thanks,
Jean

> 
> [1] https://lore.kernel.org/all/20240429113938.192706-2-jean-philippe@linaro.org/
> 
> Jean-Philippe Brucker (3):
>   dt-bindings: PCI: generic: Add ats-supported property
>   iommu/of: Support ats-supported device-tree property
>   arm64: dts: fvp: Enable PCIe ATS for Base RevC FVP
> 
>  .../devicetree/bindings/pci/host-generic-pci.yaml        | 6 ++++++
>  drivers/iommu/of_iommu.c                                 | 9 +++++++++
>  arch/arm64/boot/dts/arm/fvp-base-revc.dts                | 1 +
>  3 files changed, 16 insertions(+)
> 
> -- 
> 2.45.2
> 

Re: [PATCH v2 0/3] Enable PCIe ATS for devicetree boot

[Will Deacon]

On Mon, Jul 01, 2024 at 11:24:00AM +0100, Jean-Philippe Brucker wrote:
> On Fri, Jun 07, 2024 at 11:54:13AM +0100, Jean-Philippe Brucker wrote:
> > Before enabling Address Translation Support (ATS) in endpoints, the OS
> > needs to confirm that the Root Complex supports it. Obtain this
> > information from the firmware description since there is no architected
> > method. ACPI provides a bit via IORT tables, so add the devicetree
> > equivalent.
> > 
> > Since v1 [1] I added the review and ack tags, thanks all. This should be
> > ready to go via the IOMMU tree.
> 
> This series enables ATS for devicetree boot, and is needed on an Nvidia
> system: https://lore.kernel.org/linux-arm-kernel/ZeJP6CwrZ2FSbTYm@Asurada-Nvidia/
> 
> Would you mind picking it up for v6.11?

I'll take a look.

Will

Re: [PATCH v2 0/3] Enable PCIe ATS for devicetree boot

[Jean-Philippe Brucker]

On Mon, Jul 01, 2024 at 12:57:23PM +0100, Will Deacon wrote:
> On Mon, Jul 01, 2024 at 11:24:00AM +0100, Jean-Philippe Brucker wrote:
> > On Fri, Jun 07, 2024 at 11:54:13AM +0100, Jean-Philippe Brucker wrote:
> > > Before enabling Address Translation Support (ATS) in endpoints, the OS
> > > needs to confirm that the Root Complex supports it. Obtain this
> > > information from the firmware description since there is no architected
> > > method. ACPI provides a bit via IORT tables, so add the devicetree
> > > equivalent.
> > > 
> > > Since v1 [1] I added the review and ack tags, thanks all. This should be
> > > ready to go via the IOMMU tree.
> > 
> > This series enables ATS for devicetree boot, and is needed on an Nvidia
> > system: https://lore.kernel.org/linux-arm-kernel/ZeJP6CwrZ2FSbTYm@Asurada-Nvidia/
> > 
> > Would you mind picking it up for v6.11?
> 
> I'll take a look.

Thanks Will, coming back from holidays I hadn't seen Joerg's announce from
Friday before sending this

Thanks,
Jean

Re: [PATCH v2 0/3] Enable PCIe ATS for devicetree boot

[Will Deacon]

On Fri, 07 Jun 2024 11:54:13 +0100, Jean-Philippe Brucker wrote:
> Before enabling Address Translation Support (ATS) in endpoints, the OS
> needs to confirm that the Root Complex supports it. Obtain this
> information from the firmware description since there is no architected
> method. ACPI provides a bit via IORT tables, so add the devicetree
> equivalent.
> 
> Since v1 [1] I added the review and ack tags, thanks all. This should be
> ready to go via the IOMMU tree.
> 
> [...]

Applied to iommu (pci/ats), thanks!

[1/3] dt-bindings: PCI: generic: Add ats-supported property
      https://git.kernel.org/iommu/c/40929e8e5449
[2/3] iommu/of: Support ats-supported device-tree property
      https://git.kernel.org/iommu/c/86e02a88bedc
[3/3] arm64: dts: fvp: Enable PCIe ATS for Base RevC FVP
      https://git.kernel.org/iommu/c/6bac3388889c

Cheers,
-- 
Will

https://fixes.arm64.dev
https://next.arm64.dev
https://will.arm64.dev

Re: [PATCH v2 1/3] dt-bindings: PCI: generic: Add ats-supported property

[Krzysztof Wilczyński]

Hello,

> Add a way for firmware to tell the OS that ATS is supported by the PCI
> root complex. An endpoint with ATS enabled may send Translation Requests
> and Translated Memory Requests, which look just like Normal Memory
> Requests with a non-zero AT field. So a root controller that ignores the
> AT field may simply forward the request to the IOMMU as a Normal Memory
> Request, which could end badly. In any case, the endpoint will be
> unusable.
> 
> The ats-supported property allows the OS to only enable ATS in endpoints
> if the root controller can handle ATS requests. Only add the property to
> pcie-host-ecam-generic for the moment. For non-generic root controllers,
> availability of ATS can be inferred from the compatible string.

Applied to dt-bindings, thank you!

[1/1] dt-bindings: PCI: generic: Add ats-supported property
      https://git.kernel.org/pci/pci/c/631b2e7318d45

	Krzysztof

Re: [PATCH v2 1/3] dt-bindings: PCI: generic: Add ats-supported property

[Will Deacon]

Hi Krzysztof,

On Sat, Jul 06, 2024 at 12:24:31PM +0900, Krzysztof Wilczyński wrote:
> > Add a way for firmware to tell the OS that ATS is supported by the PCI
> > root complex. An endpoint with ATS enabled may send Translation Requests
> > and Translated Memory Requests, which look just like Normal Memory
> > Requests with a non-zero AT field. So a root controller that ignores the
> > AT field may simply forward the request to the IOMMU as a Normal Memory
> > Request, which could end badly. In any case, the endpoint will be
> > unusable.
> > 
> > The ats-supported property allows the OS to only enable ATS in endpoints
> > if the root controller can handle ATS requests. Only add the property to
> > pcie-host-ecam-generic for the moment. For non-generic root controllers,
> > availability of ATS can be inferred from the compatible string.
> 
> Applied to dt-bindings, thank you!
> 
> [1/1] dt-bindings: PCI: generic: Add ats-supported property
>       https://git.kernel.org/pci/pci/c/631b2e7318d45

FWIW: I already picked this up in the IOMMU tree last week. If you need
it to avoid a conflict, then you can pull from the pci/ats branch:

https://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux.git/commit/?h=pci/ats&id=40929e8e5449a18bc98baf7a907dd6674bd60049

Cheers,

Will