[PATCH V10 00/12] IOMMU probe deferral support

[PATCH V10 00/12] IOMMU probe deferral support

[Sricharan R]

This series calls the dma ops configuration for the devices
at a generic place so that it works for all busses.
The dma_configure_ops for a device is now called during
the device_attach callback just before the probe of the
bus/driver is called. Similarly dma_deconfigure is called during
device/driver_detach path.

pci_bus_add_devices    (platform/amba)(_device_create/driver_register)
       |                         |
pci_bus_add_device     (device_add/driver_register)
       |                         |
device_attach           device_initial_probe
       |                         |
__device_attach_driver    __device_attach_driver
       |
driver_probe_device
       |
really_probe
       |
dma_configure

Similarly on the device/driver_unregister path __device_release_driver is
called which inturn calls dma_deconfigure.

Rebased the series against mainline 4.11-rc5. Applies and builds cleanly
against mainline and linux-next, iommu-next.
  
* Tested with platform and pci devices for probe deferral
  and reprobe on arm64 based platform.

Previous post of this series [8]. 

Please note that, i have kept the tested/acked tags intact from V8
because V9/10 were for more fixes that was added, so the original
tags that was given for the functional testing remains the same.

 [V10]
     * Rebased on top of 4.11-rc5.
 
     * Fixed coherent_dma_mask 64bit overflow issue [8]
       for OF. The fix for OF was added as a separate
       patch#6, since the issue is true even without probe deferral,
       but gets reproduced with the probe deferral series.
       Added Lorenzo's ACPI fix for coherent_dma_mask overflow
       and the fix for dma_configure getting called more than
       once for the same device.

     * Also fixed an build issue caught by kbuild robot for
       m68k arch. The issue was dma_(de)configure was not
       getting defined for !CONFIG_HAS_DMA, so fixed that as well.

 [V9]
     * Rebased on top of 4.11-rc1.

     * Merged Robin's fixes for legacy binding issue,
       pci devices with no iommu-map property and deferencing
       of_iommu_table after init.
     
 [V8]
     * Picked up all the acks and tested tags from Marek and
       Hanjun for DT and ACPI patches respectively, since
       no functional changes was done.

     * Addressed Minor comments Sinan and Bjorn.

     * Added Robin's fix for fixing the deferencing NULL for
       of_iommu_table after init in patch #2.

     * Rebased it on top of linux-next

 [V7]
     * Updated the subject and commit log for patch #6 as per
       comments from Lorenzo. No functional changes.

 [V6]
     * Fixed a bug in dma_configure function pointed out by
       Robin.
     * Reordered the patches as per comments from Robin and
       Lorenzo.
     * Added Tags.

 [V5]
     * Reworked the pci configuration code hanging outside and
       pushed it to dma_configure as in PATCH#5,6,7.
       Also added a couple of patches that Lorenzo provided for
       correcting the Probe deferring mechanism in case of
       ACPI devices from here [5].

 [V4]
     * Took the reworked patches [2] from Robin's branch and
       rebased on top of Lorenzo's ACPI IORT ARM support series [3].

     * Added the patches for moving the dma ops configuration of
       acpi based devices to probe time as well.
 [V3]
     * Removed the patch to split dma_masks/dma_ops configuration
       separately based on review comments that both masks and ops are
       required only during the device probe time.

     * Reworked the series based on Generic DT bindings series.

     * Added call to iommu's remove_device in the cleanup path for arm and
       arm64.

     * Removed the notifier trick in arm64 to handle early device
       registration.

     * Added reset of dma_ops in cleanup path for arm based on comments.

     * Fixed the pci_iommu_configure path and tested with PCI device as
       well.
 
     * Fixed a bug to return the correct iommu_ops from patch 7 [4] in
       last post.

     * Fixed few other cosmetic comments.
  
 [V2]
     * Updated the Initial post to call dma_configure/deconfigure from
       generic code
 
     * Added iommu add_device callback from of_iommu_configure path

 [V1]
     * Initial post from Laurent Pinchart [1]

[1] http://lists.linuxfoundation.org/pipermail/iommu/2015-May/013016.html
[2] http://www.linux-arm.org/git?p=linux-rm.git;a=shortlog;h=refs/heads/iommu/defer
[3] https://lkml.org/lkml/2016/11/21/141
[4] https://www.mail-archive.com/iommu@xxxxxxxxxxxxxxxxxxxxxxxxxx/msg13940.html
[5] git://git.kernel.org/pub/scm/linux/kernel/git/lpieralisi/linux.git iommu/probe-deferral
[6] http://www.spinics.net/lists/linux-pci/msg57992.html
[7] https://www.spinics.net/lists/arm-kernel/msg556209.html
[8] http://patchwork.ozlabs.org/patch/743898/

Laurent Pinchart (3):
  of: dma: Move range size workaround to of_dma_get_range()
  of: dma: Make of_dma_deconfigure() public
  iommu: of: Handle IOMMU lookup failure with deferred probing or error

Lorenzo Pieralisi (2):
  ACPI/IORT: Add function to check SMMUs drivers presence
  ACPI/IORT: Remove linker section for IORT entries probing

Robin Murphy (3):
  iommu/of: Refactor of_iommu_configure() for error handling
  iommu/of: Prepare for deferred IOMMU configuration
  iommu/arm-smmu: Clean up early-probing workarounds

Sricharan R (4):
  of: device: Fix overflow of coherent_dma_mask
  of/acpi: Configure dma operations at probe time for platform/amba/pci
    bus devices
  drivers: acpi: Handle IOMMU lookup failure with deferred probing or
    error
  arm64: dma-mapping: Remove the notifier trick to handle early setting
    of dma_ops

 arch/arm64/mm/dma-mapping.c       | 142 +++++---------------------------------
 drivers/acpi/arm64/iort.c         |  48 ++++++++++++-
 drivers/acpi/glue.c               |   5 --
 drivers/acpi/scan.c               |  11 ++-
 drivers/base/dd.c                 |   9 +++
 drivers/base/dma-mapping.c        |  41 +++++++++++
 drivers/iommu/arm-smmu-v3.c       |  46 +-----------
 drivers/iommu/arm-smmu.c          | 110 +++++++++++++----------------
 drivers/iommu/of_iommu.c          | 126 ++++++++++++++++++++++++---------
 drivers/of/address.c              |  20 +++++-
 drivers/of/device.c               |  36 +++++-----
 drivers/of/platform.c             |  10 +--
 drivers/pci/probe.c               |  28 --------
 include/acpi/acpi_bus.h           |   2 +-
 include/asm-generic/vmlinux.lds.h |   1 -
 include/linux/acpi.h              |   7 +-
 include/linux/acpi_iort.h         |   3 -
 include/linux/dma-mapping.h       |  12 ++++
 include/linux/of_device.h         |  10 ++-
 19 files changed, 329 insertions(+), 338 deletions(-)

-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[PATCH V10 01/12] iommu/of: Refactor of_iommu_configure() for error handling

[Sricharan R]

From: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>

In preparation for some upcoming cleverness, rework the control flow in
of_iommu_configure() to minimise duplication and improve the propogation
of errors. It's also as good a time as any to switch over from the
now-just-a-compatibility-wrapper of_iommu_get_ops() to using the generic
IOMMU instance interface directly.

Tested-by: Marek Szyprowski <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
Signed-off-by: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>
---
 drivers/iommu/of_iommu.c | 83 +++++++++++++++++++++++++++++++-----------------
 1 file changed, 53 insertions(+), 30 deletions(-)

diff --git a/drivers/iommu/of_iommu.c b/drivers/iommu/of_iommu.c
index 2683e9f..8f4e599 100644
--- a/drivers/iommu/of_iommu.c
+++ b/drivers/iommu/of_iommu.c
@@ -96,6 +96,28 @@ int of_get_dma_window(struct device_node *dn, const char *prefix, int index,
 }
 EXPORT_SYMBOL_GPL(of_get_dma_window);
 
+static const struct iommu_ops
+*of_iommu_xlate(struct device *dev, struct of_phandle_args *iommu_spec)
+{
+	const struct iommu_ops *ops;
+	struct fwnode_handle *fwnode = &iommu_spec->np->fwnode;
+	int err;
+
+	ops = iommu_ops_from_fwnode(fwnode);
+	if (!ops || !ops->of_xlate)
+		return NULL;
+
+	err = iommu_fwspec_init(dev, &iommu_spec->np->fwnode, ops);
+	if (err)
+		return ERR_PTR(err);
+
+	err = ops->of_xlate(dev, iommu_spec);
+	if (err)
+		return ERR_PTR(err);
+
+	return ops;
+}
+
 static int __get_pci_rid(struct pci_dev *pdev, u16 alias, void *data)
 {
 	struct of_phandle_args *iommu_spec = data;
@@ -105,10 +127,11 @@ static int __get_pci_rid(struct pci_dev *pdev, u16 alias, void *data)
 }
 
 static const struct iommu_ops
-*of_pci_iommu_configure(struct pci_dev *pdev, struct device_node *bridge_np)
+*of_pci_iommu_init(struct pci_dev *pdev, struct device_node *bridge_np)
 {
 	const struct iommu_ops *ops;
 	struct of_phandle_args iommu_spec;
+	int err;
 
 	/*
 	 * Start by tracing the RID alias down the PCI topology as
@@ -123,56 +146,56 @@ static int __get_pci_rid(struct pci_dev *pdev, u16 alias, void *data)
 	 * bus into the system beyond, and which IOMMU it ends up at.
 	 */
 	iommu_spec.np = NULL;
-	if (of_pci_map_rid(bridge_np, iommu_spec.args[0], "iommu-map",
-			   "iommu-map-mask", &iommu_spec.np, iommu_spec.args))
-		return NULL;
+	err = of_pci_map_rid(bridge_np, iommu_spec.args[0], "iommu-map",
+			     "iommu-map-mask", &iommu_spec.np,
+			     iommu_spec.args);
+	if (err)
+		return err == -ENODEV ? NULL : ERR_PTR(err);
 
-	ops = iommu_ops_from_fwnode(&iommu_spec.np->fwnode);
-	if (!ops || !ops->of_xlate ||
-	    iommu_fwspec_init(&pdev->dev, &iommu_spec.np->fwnode, ops) ||
-	    ops->of_xlate(&pdev->dev, &iommu_spec))
-		ops = NULL;
+	ops = of_iommu_xlate(&pdev->dev, &iommu_spec);
 
 	of_node_put(iommu_spec.np);
 	return ops;
 }
 
-const struct iommu_ops *of_iommu_configure(struct device *dev,
-					   struct device_node *master_np)
+static const struct iommu_ops
+*of_platform_iommu_init(struct device *dev, struct device_node *np)
 {
 	struct of_phandle_args iommu_spec;
-	struct device_node *np;
 	const struct iommu_ops *ops = NULL;
 	int idx = 0;
 
-	if (dev_is_pci(dev))
-		return of_pci_iommu_configure(to_pci_dev(dev), master_np);
-
 	/*
 	 * We don't currently walk up the tree looking for a parent IOMMU.
 	 * See the `Notes:' section of
 	 * Documentation/devicetree/bindings/iommu/iommu.txt
 	 */
-	while (!of_parse_phandle_with_args(master_np, "iommus",
-					   "#iommu-cells", idx,
-					   &iommu_spec)) {
-		np = iommu_spec.np;
-		ops = iommu_ops_from_fwnode(&np->fwnode);
-
-		if (!ops || !ops->of_xlate ||
-		    iommu_fwspec_init(dev, &np->fwnode, ops) ||
-		    ops->of_xlate(dev, &iommu_spec))
-			goto err_put_node;
-
-		of_node_put(np);
+	while (!of_parse_phandle_with_args(np, "iommus", "#iommu-cells",
+					   idx, &iommu_spec)) {
+		ops = of_iommu_xlate(dev, &iommu_spec);
+		of_node_put(iommu_spec.np);
 		idx++;
+		if (IS_ERR_OR_NULL(ops))
+			break;
 	}
 
 	return ops;
+}
+
+const struct iommu_ops *of_iommu_configure(struct device *dev,
+					   struct device_node *master_np)
+{
+	const struct iommu_ops *ops;
+
+	if (!master_np)
+		return NULL;
+
+	if (dev_is_pci(dev))
+		ops = of_pci_iommu_init(to_pci_dev(dev), master_np);
+	else
+		ops = of_platform_iommu_init(dev, master_np);
 
-err_put_node:
-	of_node_put(np);
-	return NULL;
+	return IS_ERR(ops) ? NULL : ops;
 }
 
 static int __init of_iommu_init(void)
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[PATCH V10 02/12] iommu/of: Prepare for deferred IOMMU configuration

[Sricharan R]

From: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>

IOMMU configuration represents unchanging properties of the hardware,
and as such should only need happen once in a device's lifetime, but
the necessary interaction with the IOMMU device and driver complicates
exactly when that point should be.

Since the only reasonable tool available for handling the inter-device
dependency is probe deferral, we need to prepare of_iommu_configure()
to run later than it is currently called (i.e. at driver probe rather
than device creation), to handle being retried, and to tell whether a
not-yet present IOMMU should be waited for or skipped (by virtue of
having declared a built-in driver or not).

Tested-by: Marek Szyprowski <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
Signed-off-by: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>
---
 drivers/iommu/of_iommu.c | 43 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 42 insertions(+), 1 deletion(-)

diff --git a/drivers/iommu/of_iommu.c b/drivers/iommu/of_iommu.c
index 8f4e599..c8be889 100644
--- a/drivers/iommu/of_iommu.c
+++ b/drivers/iommu/of_iommu.c
@@ -96,6 +96,19 @@ int of_get_dma_window(struct device_node *dn, const char *prefix, int index,
 }
 EXPORT_SYMBOL_GPL(of_get_dma_window);
 
+static bool of_iommu_driver_present(struct device_node *np)
+{
+	/*
+	 * If the IOMMU still isn't ready by the time we reach init, assume
+	 * it never will be. We don't want to defer indefinitely, nor attempt
+	 * to dereference __iommu_of_table after it's been freed.
+	 */
+	if (system_state > SYSTEM_BOOTING)
+		return false;
+
+	return of_match_node(&__iommu_of_table, np);
+}
+
 static const struct iommu_ops
 *of_iommu_xlate(struct device *dev, struct of_phandle_args *iommu_spec)
 {
@@ -104,12 +117,20 @@ int of_get_dma_window(struct device_node *dn, const char *prefix, int index,
 	int err;
 
 	ops = iommu_ops_from_fwnode(fwnode);
-	if (!ops || !ops->of_xlate)
+	if ((ops && !ops->of_xlate) ||
+	    (!ops && !of_iommu_driver_present(iommu_spec->np)))
 		return NULL;
 
 	err = iommu_fwspec_init(dev, &iommu_spec->np->fwnode, ops);
 	if (err)
 		return ERR_PTR(err);
+	/*
+	 * The otherwise-empty fwspec handily serves to indicate the specific
+	 * IOMMU device we're waiting for, which will be useful if we ever get
+	 * a proper probe-ordering dependency mechanism in future.
+	 */
+	if (!ops)
+		return ERR_PTR(-EPROBE_DEFER);
 
 	err = ops->of_xlate(dev, iommu_spec);
 	if (err)
@@ -186,14 +207,34 @@ const struct iommu_ops *of_iommu_configure(struct device *dev,
 					   struct device_node *master_np)
 {
 	const struct iommu_ops *ops;
+	struct iommu_fwspec *fwspec = dev->iommu_fwspec;
 
 	if (!master_np)
 		return NULL;
 
+	if (fwspec) {
+		if (fwspec->ops)
+			return fwspec->ops;
+
+		/* In the deferred case, start again from scratch */
+		iommu_fwspec_free(dev);
+	}
+
 	if (dev_is_pci(dev))
 		ops = of_pci_iommu_init(to_pci_dev(dev), master_np);
 	else
 		ops = of_platform_iommu_init(dev, master_np);
+	/*
+	 * If we have reason to believe the IOMMU driver missed the initial
+	 * add_device callback for dev, replay it to get things in order.
+	 */
+	if (!IS_ERR_OR_NULL(ops) && ops->add_device &&
+	    dev->bus && !dev->iommu_group) {
+		int err = ops->add_device(dev);
+
+		if (err)
+			ops = ERR_PTR(err);
+	}
 
 	return IS_ERR(ops) ? NULL : ops;
 }
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[PATCH V10 03/12] of: dma: Move range size workaround to of_dma_get_range()

[Sricharan R]

From: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>

Invalid dma-ranges values should be worked around when retrieving the
DMA range in of_dma_get_range(), not by all callers of the function.
This isn't much of a problem now that we have a single caller, but that
situation will change when moving DMA configuration to device probe
time.

Tested-by: Marek Szyprowski <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>
---
 drivers/of/address.c | 20 ++++++++++++++++++--
 drivers/of/device.c  | 15 ---------------
 2 files changed, 18 insertions(+), 17 deletions(-)

diff --git a/drivers/of/address.c b/drivers/of/address.c
index 02b2903..6aeb816 100644
--- a/drivers/of/address.c
+++ b/drivers/of/address.c
@@ -819,8 +819,8 @@ void __iomem *of_io_request_and_map(struct device_node *np, int index,
  *	CPU addr (phys_addr_t)	: pna cells
  *	size			: nsize cells
  *
- * It returns -ENODEV if "dma-ranges" property was not found
- * for this device in DT.
+ * Return 0 on success, -ENODEV if the "dma-ranges" property was not found for
+ * this device in DT, or -EINVAL if the CPU address or size is invalid.
  */
 int of_dma_get_range(struct device_node *np, u64 *dma_addr, u64 *paddr, u64 *size)
 {
@@ -880,6 +880,22 @@ int of_dma_get_range(struct device_node *np, u64 *dma_addr, u64 *paddr, u64 *siz
 	*dma_addr = dmaaddr;
 
 	*size = of_read_number(ranges + naddr + pna, nsize);
+	/*
+	 * DT nodes sometimes incorrectly set the size as a mask. Work around
+	 * those incorrect DT by computing the size as mask + 1.
+	 */
+	if (*size & 1) {
+		pr_warn("%s: size 0x%llx for dma-range in node(%s) set as mask\n",
+			__func__, *size, np->full_name);
+		*size = *size + 1;
+	}
+
+	if (!*size) {
+		pr_err("%s: invalid size zero for dma-range in node(%s)\n",
+		       __func__, np->full_name);
+		ret = -EINVAL;
+		goto out;
+	}
 
 	pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n",
 		 *dma_addr, *paddr, *size);
diff --git a/drivers/of/device.c b/drivers/of/device.c
index b1e6beb..09dedd0 100644
--- a/drivers/of/device.c
+++ b/drivers/of/device.c
@@ -110,21 +110,6 @@ void of_dma_configure(struct device *dev, struct device_node *np)
 		size = dev->coherent_dma_mask + 1;
 	} else {
 		offset = PFN_DOWN(paddr - dma_addr);
-
-		/*
-		 * Add a work around to treat the size as mask + 1 in case
-		 * it is defined in DT as a mask.
-		 */
-		if (size & 1) {
-			dev_warn(dev, "Invalid size 0x%llx for dma-range\n",
-				 size);
-			size = size + 1;
-		}
-
-		if (!size) {
-			dev_err(dev, "Adjusted size 0x%llx invalid\n", size);
-			return;
-		}
 		dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
 	}
 
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

Re: [PATCH V10 03/12] of: dma: Move range size workaround to of_dma_get_range()

[Robin Murphy]

On 04/04/17 11:18, Sricharan R wrote:
> From: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>
> 
> Invalid dma-ranges values should be worked around when retrieving the
> DMA range in of_dma_get_range(), not by all callers of the function.
> This isn't much of a problem now that we have a single caller, but that
> situation will change when moving DMA configuration to device probe
> time.

Just for completeness:

Reviewed-by: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>

> Tested-by: Marek Szyprowski <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
> Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>
> ---
>  drivers/of/address.c | 20 ++++++++++++++++++--
>  drivers/of/device.c  | 15 ---------------
>  2 files changed, 18 insertions(+), 17 deletions(-)
> 
> diff --git a/drivers/of/address.c b/drivers/of/address.c
> index 02b2903..6aeb816 100644
> --- a/drivers/of/address.c
> +++ b/drivers/of/address.c
> @@ -819,8 +819,8 @@ void __iomem *of_io_request_and_map(struct device_node *np, int index,
>   *	CPU addr (phys_addr_t)	: pna cells
>   *	size			: nsize cells
>   *
> - * It returns -ENODEV if "dma-ranges" property was not found
> - * for this device in DT.
> + * Return 0 on success, -ENODEV if the "dma-ranges" property was not found for
> + * this device in DT, or -EINVAL if the CPU address or size is invalid.
>   */
>  int of_dma_get_range(struct device_node *np, u64 *dma_addr, u64 *paddr, u64 *size)
>  {
> @@ -880,6 +880,22 @@ int of_dma_get_range(struct device_node *np, u64 *dma_addr, u64 *paddr, u64 *siz
>  	*dma_addr = dmaaddr;
>  
>  	*size = of_read_number(ranges + naddr + pna, nsize);
> +	/*
> +	 * DT nodes sometimes incorrectly set the size as a mask. Work around
> +	 * those incorrect DT by computing the size as mask + 1.
> +	 */
> +	if (*size & 1) {
> +		pr_warn("%s: size 0x%llx for dma-range in node(%s) set as mask\n",
> +			__func__, *size, np->full_name);
> +		*size = *size + 1;
> +	}
> +
> +	if (!*size) {
> +		pr_err("%s: invalid size zero for dma-range in node(%s)\n",
> +		       __func__, np->full_name);
> +		ret = -EINVAL;
> +		goto out;
> +	}
>  
>  	pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n",
>  		 *dma_addr, *paddr, *size);
> diff --git a/drivers/of/device.c b/drivers/of/device.c
> index b1e6beb..09dedd0 100644
> --- a/drivers/of/device.c
> +++ b/drivers/of/device.c
> @@ -110,21 +110,6 @@ void of_dma_configure(struct device *dev, struct device_node *np)
>  		size = dev->coherent_dma_mask + 1;
>  	} else {
>  		offset = PFN_DOWN(paddr - dma_addr);
> -
> -		/*
> -		 * Add a work around to treat the size as mask + 1 in case
> -		 * it is defined in DT as a mask.
> -		 */
> -		if (size & 1) {
> -			dev_warn(dev, "Invalid size 0x%llx for dma-range\n",
> -				 size);
> -			size = size + 1;
> -		}
> -
> -		if (!size) {
> -			dev_err(dev, "Adjusted size 0x%llx invalid\n", size);
> -			return;
> -		}
>  		dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
>  	}
>  
> 

Re: [PATCH V10 03/12] of: dma: Move range size workaround to of_dma_get_range()

[Frank Rowand]

On 04/04/17 03:18, Sricharan R wrote:
> From: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
> 
> Invalid dma-ranges values should be worked around when retrieving the
> DMA range in of_dma_get_range(), not by all callers of the function.
> This isn't much of a problem now that we have a single caller, but that
> situation will change when moving DMA configuration to device probe
> time.

Please drop this patch from the series.  I just now sent Rob an
alternative that reflects other changes that have occurred since
this patch series was first created many, many moons ago
(https://lkml.org/lkml/2017/4/6/53).

Thanks,

Frank

> 
> Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
> Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
> ---
>  drivers/of/address.c | 20 ++++++++++++++++++--
>  drivers/of/device.c  | 15 ---------------
>  2 files changed, 18 insertions(+), 17 deletions(-)
> 
> diff --git a/drivers/of/address.c b/drivers/of/address.c
> index 02b2903..6aeb816 100644
> --- a/drivers/of/address.c
> +++ b/drivers/of/address.c
> @@ -819,8 +819,8 @@ void __iomem *of_io_request_and_map(struct device_node *np, int index,
>   *	CPU addr (phys_addr_t)	: pna cells
>   *	size			: nsize cells
>   *
> - * It returns -ENODEV if "dma-ranges" property was not found
> - * for this device in DT.
> + * Return 0 on success, -ENODEV if the "dma-ranges" property was not found for
> + * this device in DT, or -EINVAL if the CPU address or size is invalid.
>   */
>  int of_dma_get_range(struct device_node *np, u64 *dma_addr, u64 *paddr, u64 *size)
>  {
> @@ -880,6 +880,22 @@ int of_dma_get_range(struct device_node *np, u64 *dma_addr, u64 *paddr, u64 *siz
>  	*dma_addr = dmaaddr;
>  
>  	*size = of_read_number(ranges + naddr + pna, nsize);
> +	/*
> +	 * DT nodes sometimes incorrectly set the size as a mask. Work around
> +	 * those incorrect DT by computing the size as mask + 1.
> +	 */
> +	if (*size & 1) {
> +		pr_warn("%s: size 0x%llx for dma-range in node(%s) set as mask\n",
> +			__func__, *size, np->full_name);
> +		*size = *size + 1;
> +	}
> +
> +	if (!*size) {
> +		pr_err("%s: invalid size zero for dma-range in node(%s)\n",
> +		       __func__, np->full_name);
> +		ret = -EINVAL;
> +		goto out;
> +	}
>  
>  	pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n",
>  		 *dma_addr, *paddr, *size);
> diff --git a/drivers/of/device.c b/drivers/of/device.c
> index b1e6beb..09dedd0 100644
> --- a/drivers/of/device.c
> +++ b/drivers/of/device.c
> @@ -110,21 +110,6 @@ void of_dma_configure(struct device *dev, struct device_node *np)
>  		size = dev->coherent_dma_mask + 1;
>  	} else {
>  		offset = PFN_DOWN(paddr - dma_addr);
> -
> -		/*
> -		 * Add a work around to treat the size as mask + 1 in case
> -		 * it is defined in DT as a mask.
> -		 */
> -		if (size & 1) {
> -			dev_warn(dev, "Invalid size 0x%llx for dma-range\n",
> -				 size);
> -			size = size + 1;
> -		}
> -
> -		if (!size) {
> -			dev_err(dev, "Adjusted size 0x%llx invalid\n", size);
> -			return;
> -		}
>  		dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
>  	}
>  
> 

Re: [PATCH V10 03/12] of: dma: Move range size workaround to of_dma_get_range()

[Sricharan R]

Hi Frank,

On 4/6/2017 11:54 AM, Frank Rowand wrote:
> On 04/04/17 03:18, Sricharan R wrote:
>> From: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>
>>
>> Invalid dma-ranges values should be worked around when retrieving the
>> DMA range in of_dma_get_range(), not by all callers of the function.
>> This isn't much of a problem now that we have a single caller, but that
>> situation will change when moving DMA configuration to device probe
>> time.
>
> Please drop this patch from the series.  I just now sent Rob an
> alternative that reflects other changes that have occurred since
> this patch series was first created many, many moons ago
> (https://lkml.org/lkml/2017/4/6/53).

ok, would drop this.

Regards,
  Sricharan

-- 
"QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

Re: [PATCH V10 03/12] of: dma: Move range size workaround to of_dma_get_range()

[Robin Murphy]

Hi Frank,

On 06/04/17 07:24, Frank Rowand wrote:
> On 04/04/17 03:18, Sricharan R wrote:
>> From: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>
>>
>> Invalid dma-ranges values should be worked around when retrieving the
>> DMA range in of_dma_get_range(), not by all callers of the function.
>> This isn't much of a problem now that we have a single caller, but that
>> situation will change when moving DMA configuration to device probe
>> time.
> 
> Please drop this patch from the series.  I just now sent Rob an
> alternative that reflects other changes that have occurred since
> this patch series was first created many, many moons ago
> (https://lkml.org/lkml/2017/4/6/53).

Fine by me too - if we can get rid of that dubious fixup altogether, all
the better. Feel free to transfer my Reviewed-by to your patch if you like.

Thanks,
Robin.

> Thanks,
> 
> Frank
> 
>>
>> Tested-by: Marek Szyprowski <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
>> Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>
>> ---
>>  drivers/of/address.c | 20 ++++++++++++++++++--
>>  drivers/of/device.c  | 15 ---------------
>>  2 files changed, 18 insertions(+), 17 deletions(-)
>>
>> diff --git a/drivers/of/address.c b/drivers/of/address.c
>> index 02b2903..6aeb816 100644
>> --- a/drivers/of/address.c
>> +++ b/drivers/of/address.c
>> @@ -819,8 +819,8 @@ void __iomem *of_io_request_and_map(struct device_node *np, int index,
>>   *	CPU addr (phys_addr_t)	: pna cells
>>   *	size			: nsize cells
>>   *
>> - * It returns -ENODEV if "dma-ranges" property was not found
>> - * for this device in DT.
>> + * Return 0 on success, -ENODEV if the "dma-ranges" property was not found for
>> + * this device in DT, or -EINVAL if the CPU address or size is invalid.
>>   */
>>  int of_dma_get_range(struct device_node *np, u64 *dma_addr, u64 *paddr, u64 *size)
>>  {
>> @@ -880,6 +880,22 @@ int of_dma_get_range(struct device_node *np, u64 *dma_addr, u64 *paddr, u64 *siz
>>  	*dma_addr = dmaaddr;
>>  
>>  	*size = of_read_number(ranges + naddr + pna, nsize);
>> +	/*
>> +	 * DT nodes sometimes incorrectly set the size as a mask. Work around
>> +	 * those incorrect DT by computing the size as mask + 1.
>> +	 */
>> +	if (*size & 1) {
>> +		pr_warn("%s: size 0x%llx for dma-range in node(%s) set as mask\n",
>> +			__func__, *size, np->full_name);
>> +		*size = *size + 1;
>> +	}
>> +
>> +	if (!*size) {
>> +		pr_err("%s: invalid size zero for dma-range in node(%s)\n",
>> +		       __func__, np->full_name);
>> +		ret = -EINVAL;
>> +		goto out;
>> +	}
>>  
>>  	pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n",
>>  		 *dma_addr, *paddr, *size);
>> diff --git a/drivers/of/device.c b/drivers/of/device.c
>> index b1e6beb..09dedd0 100644
>> --- a/drivers/of/device.c
>> +++ b/drivers/of/device.c
>> @@ -110,21 +110,6 @@ void of_dma_configure(struct device *dev, struct device_node *np)
>>  		size = dev->coherent_dma_mask + 1;
>>  	} else {
>>  		offset = PFN_DOWN(paddr - dma_addr);
>> -
>> -		/*
>> -		 * Add a work around to treat the size as mask + 1 in case
>> -		 * it is defined in DT as a mask.
>> -		 */
>> -		if (size & 1) {
>> -			dev_warn(dev, "Invalid size 0x%llx for dma-range\n",
>> -				 size);
>> -			size = size + 1;
>> -		}
>> -
>> -		if (!size) {
>> -			dev_err(dev, "Adjusted size 0x%llx invalid\n", size);
>> -			return;
>> -		}
>>  		dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
>>  	}
>>  
>>

[PATCH V10 04/12] of: dma: Make of_dma_deconfigure() public

[Sricharan R]

From: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>

As part of moving DMA initializing to probe time the
of_dma_deconfigure() function will need to be called from different
source files. Make it public and move it to drivers/of/device.c where
the of_dma_configure() function is.

Tested-by: Marek Szyprowski <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>
---
 drivers/of/device.c       | 12 ++++++++++++
 drivers/of/platform.c     |  5 -----
 include/linux/of_device.h |  3 +++
 3 files changed, 15 insertions(+), 5 deletions(-)

diff --git a/drivers/of/device.c b/drivers/of/device.c
index 09dedd0..c17c19d 100644
--- a/drivers/of/device.c
+++ b/drivers/of/device.c
@@ -136,6 +136,18 @@ void of_dma_configure(struct device *dev, struct device_node *np)
 }
 EXPORT_SYMBOL_GPL(of_dma_configure);
 
+/**
+ * of_dma_deconfigure - Clean up DMA configuration
+ * @dev:	Device for which to clean up DMA configuration
+ *
+ * Clean up all configuration performed by of_dma_configure_ops() and free all
+ * resources that have been allocated.
+ */
+void of_dma_deconfigure(struct device *dev)
+{
+	arch_teardown_dma_ops(dev);
+}
+
 int of_device_register(struct platform_device *pdev)
 {
 	device_initialize(&pdev->dev);
diff --git a/drivers/of/platform.c b/drivers/of/platform.c
index 5dfcc96..5344db5 100644
--- a/drivers/of/platform.c
+++ b/drivers/of/platform.c
@@ -158,11 +158,6 @@ struct platform_device *of_device_alloc(struct device_node *np,
 }
 EXPORT_SYMBOL(of_device_alloc);
 
-static void of_dma_deconfigure(struct device *dev)
-{
-	arch_teardown_dma_ops(dev);
-}
-
 /**
  * of_platform_device_create_pdata - Alloc, initialize and register an of_device
  * @np: pointer to node to create device for
diff --git a/include/linux/of_device.h b/include/linux/of_device.h
index c12dace..af98455 100644
--- a/include/linux/of_device.h
+++ b/include/linux/of_device.h
@@ -56,6 +56,7 @@ static inline struct device_node *of_cpu_device_node_get(int cpu)
 }
 
 void of_dma_configure(struct device *dev, struct device_node *np);
+void of_dma_deconfigure(struct device *dev);
 #else /* CONFIG_OF */
 
 static inline int of_driver_match_device(struct device *dev,
@@ -105,6 +106,8 @@ static inline struct device_node *of_cpu_device_node_get(int cpu)
 }
 static inline void of_dma_configure(struct device *dev, struct device_node *np)
 {}
+static inline void of_dma_deconfigure(struct device *dev)
+{}
 #endif /* CONFIG_OF */
 
 #endif /* _LINUX_OF_DEVICE_H */
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

Re: [PATCH V10 04/12] of: dma: Make of_dma_deconfigure() public

[Robin Murphy]

On 04/04/17 11:18, Sricharan R wrote:
> From: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>
> 
> As part of moving DMA initializing to probe time the
> of_dma_deconfigure() function will need to be called from different
> source files. Make it public and move it to drivers/of/device.c where
> the of_dma_configure() function is.

Reviewed-by: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>

> Tested-by: Marek Szyprowski <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
> Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>
> ---
>  drivers/of/device.c       | 12 ++++++++++++
>  drivers/of/platform.c     |  5 -----
>  include/linux/of_device.h |  3 +++
>  3 files changed, 15 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/of/device.c b/drivers/of/device.c
> index 09dedd0..c17c19d 100644
> --- a/drivers/of/device.c
> +++ b/drivers/of/device.c
> @@ -136,6 +136,18 @@ void of_dma_configure(struct device *dev, struct device_node *np)
>  }
>  EXPORT_SYMBOL_GPL(of_dma_configure);
>  
> +/**
> + * of_dma_deconfigure - Clean up DMA configuration
> + * @dev:	Device for which to clean up DMA configuration
> + *
> + * Clean up all configuration performed by of_dma_configure_ops() and free all
> + * resources that have been allocated.
> + */
> +void of_dma_deconfigure(struct device *dev)
> +{
> +	arch_teardown_dma_ops(dev);
> +}
> +
>  int of_device_register(struct platform_device *pdev)
>  {
>  	device_initialize(&pdev->dev);
> diff --git a/drivers/of/platform.c b/drivers/of/platform.c
> index 5dfcc96..5344db5 100644
> --- a/drivers/of/platform.c
> +++ b/drivers/of/platform.c
> @@ -158,11 +158,6 @@ struct platform_device *of_device_alloc(struct device_node *np,
>  }
>  EXPORT_SYMBOL(of_device_alloc);
>  
> -static void of_dma_deconfigure(struct device *dev)
> -{
> -	arch_teardown_dma_ops(dev);
> -}
> -
>  /**
>   * of_platform_device_create_pdata - Alloc, initialize and register an of_device
>   * @np: pointer to node to create device for
> diff --git a/include/linux/of_device.h b/include/linux/of_device.h
> index c12dace..af98455 100644
> --- a/include/linux/of_device.h
> +++ b/include/linux/of_device.h
> @@ -56,6 +56,7 @@ static inline struct device_node *of_cpu_device_node_get(int cpu)
>  }
>  
>  void of_dma_configure(struct device *dev, struct device_node *np);
> +void of_dma_deconfigure(struct device *dev);
>  #else /* CONFIG_OF */
>  
>  static inline int of_driver_match_device(struct device *dev,
> @@ -105,6 +106,8 @@ static inline struct device_node *of_cpu_device_node_get(int cpu)
>  }
>  static inline void of_dma_configure(struct device *dev, struct device_node *np)
>  {}
> +static inline void of_dma_deconfigure(struct device *dev)
> +{}
>  #endif /* CONFIG_OF */
>  
>  #endif /* _LINUX_OF_DEVICE_H */
> 

[PATCH V10 05/12] ACPI/IORT: Add function to check SMMUs drivers presence

[Sricharan R]

From: Lorenzo Pieralisi <lorenzo.pieralisi-5wv7dgnIgG8@public.gmane.org>

The IOMMU probe deferral implementation requires a mechanism to detect
if drivers for SMMU components are built-in in the kernel to detect
whether IOMMU configuration for a given device should be deferred (ie
SMMU drivers present but still not probed) or not (drivers not present).

Add a simple function to IORT to detect if SMMU drivers for SMMU
components managed by IORT are built-in in the kernel.

Tested-by: Hanjun Guo <hanjun.guo-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi-5wv7dgnIgG8@public.gmane.org>
Cc: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>
Cc: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
---
 drivers/acpi/arm64/iort.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
index 4a5bb96..3dd9ec3 100644
--- a/drivers/acpi/arm64/iort.c
+++ b/drivers/acpi/arm64/iort.c
@@ -523,6 +523,19 @@ static int arm_smmu_iort_xlate(struct device *dev, u32 streamid,
 	return ret;
 }
 
+static inline bool iort_iommu_driver_enabled(u8 type)
+{
+	switch (type) {
+	case ACPI_IORT_NODE_SMMU_V3:
+		return IS_BUILTIN(CONFIG_ARM_SMMU_V3);
+	case ACPI_IORT_NODE_SMMU:
+		return IS_BUILTIN(CONFIG_ARM_SMMU);
+	default:
+		pr_warn("IORT node type %u does not describe an SMMU\n", type);
+		return false;
+	}
+}
+
 static const struct iommu_ops *iort_iommu_xlate(struct device *dev,
 					struct acpi_iort_node *node,
 					u32 streamid)
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

Re: [PATCH V10 05/12] ACPI/IORT: Add function to check SMMUs drivers presence

[Robin Murphy]

On 04/04/17 11:18, Sricharan R wrote:
> From: Lorenzo Pieralisi <lorenzo.pieralisi-5wv7dgnIgG8@public.gmane.org>
> 
> The IOMMU probe deferral implementation requires a mechanism to detect
> if drivers for SMMU components are built-in in the kernel to detect
> whether IOMMU configuration for a given device should be deferred (ie
> SMMU drivers present but still not probed) or not (drivers not present).
> 
> Add a simple function to IORT to detect if SMMU drivers for SMMU
> components managed by IORT are built-in in the kernel.

Ah, if only DT could be this neat and tidy :D

Reviewed-by: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>

> Tested-by: Hanjun Guo <hanjun.guo-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi-5wv7dgnIgG8@public.gmane.org>
> Cc: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>
> Cc: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
> ---
>  drivers/acpi/arm64/iort.c | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
> 
> diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> index 4a5bb96..3dd9ec3 100644
> --- a/drivers/acpi/arm64/iort.c
> +++ b/drivers/acpi/arm64/iort.c
> @@ -523,6 +523,19 @@ static int arm_smmu_iort_xlate(struct device *dev, u32 streamid,
>  	return ret;
>  }
>  
> +static inline bool iort_iommu_driver_enabled(u8 type)
> +{
> +	switch (type) {
> +	case ACPI_IORT_NODE_SMMU_V3:
> +		return IS_BUILTIN(CONFIG_ARM_SMMU_V3);
> +	case ACPI_IORT_NODE_SMMU:
> +		return IS_BUILTIN(CONFIG_ARM_SMMU);
> +	default:
> +		pr_warn("IORT node type %u does not describe an SMMU\n", type);
> +		return false;
> +	}
> +}
> +
>  static const struct iommu_ops *iort_iommu_xlate(struct device *dev,
>  					struct acpi_iort_node *node,
>  					u32 streamid)
> 

[PATCH V10 06/12] of: device: Fix overflow of coherent_dma_mask

[Sricharan R]

Size of the dma-range is calculated as coherent_dma_mask + 1
and passed to arch_setup_dma_ops further. It overflows when
the coherent_dma_mask is set for full 64 bits 0xFFFFFFFFFFFFFFFF,
resulting in size getting passed as 0 wrongly. Fix this by
passsing in max(mask, mask + 1). Note that in this case
when the mask is set to full 64bits, we will be passing the mask
itself to arch_setup_dma_ops instead of the size. The real fix
for this should be to make arch_setup_dma_ops receive the
mask and handle it, to be done in the future.

Signed-off-by: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
---
 drivers/of/device.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/of/device.c b/drivers/of/device.c
index c17c19d..c2ae6bb 100644
--- a/drivers/of/device.c
+++ b/drivers/of/device.c
@@ -107,7 +107,7 @@ void of_dma_configure(struct device *dev, struct device_node *np)
 	ret = of_dma_get_range(np, &dma_addr, &paddr, &size);
 	if (ret < 0) {
 		dma_addr = offset = 0;
-		size = dev->coherent_dma_mask + 1;
+		size = max(dev->coherent_dma_mask, dev->coherent_dma_mask + 1);
 	} else {
 		offset = PFN_DOWN(paddr - dma_addr);
 		dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

Re: [PATCH V10 06/12] of: device: Fix overflow of coherent_dma_mask

[Robin Murphy]

On 04/04/17 11:18, Sricharan R wrote:
> Size of the dma-range is calculated as coherent_dma_mask + 1
> and passed to arch_setup_dma_ops further. It overflows when
> the coherent_dma_mask is set for full 64 bits 0xFFFFFFFFFFFFFFFF,
> resulting in size getting passed as 0 wrongly. Fix this by
> passsing in max(mask, mask + 1). Note that in this case
> when the mask is set to full 64bits, we will be passing the mask
> itself to arch_setup_dma_ops instead of the size. The real fix
> for this should be to make arch_setup_dma_ops receive the
> mask and handle it, to be done in the future.

Agreed; in the meantime, this fix is neat and robust enough.

Reviewed-by: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>

> Signed-off-by: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
> ---
>  drivers/of/device.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/of/device.c b/drivers/of/device.c
> index c17c19d..c2ae6bb 100644
> --- a/drivers/of/device.c
> +++ b/drivers/of/device.c
> @@ -107,7 +107,7 @@ void of_dma_configure(struct device *dev, struct device_node *np)
>  	ret = of_dma_get_range(np, &dma_addr, &paddr, &size);
>  	if (ret < 0) {
>  		dma_addr = offset = 0;
> -		size = dev->coherent_dma_mask + 1;
> +		size = max(dev->coherent_dma_mask, dev->coherent_dma_mask + 1);
>  	} else {
>  		offset = PFN_DOWN(paddr - dma_addr);
>  		dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
> 

Re: [PATCH V10 06/12] of: device: Fix overflow of coherent_dma_mask

[Frank Rowand]

On 04/04/17 03:18, Sricharan R wrote:
> Size of the dma-range is calculated as coherent_dma_mask + 1
> and passed to arch_setup_dma_ops further. It overflows when
> the coherent_dma_mask is set for full 64 bits 0xFFFFFFFFFFFFFFFF,
> resulting in size getting passed as 0 wrongly. Fix this by
> passsing in max(mask, mask + 1). Note that in this case
> when the mask is set to full 64bits, we will be passing the mask
> itself to arch_setup_dma_ops instead of the size. The real fix
> for this should be to make arch_setup_dma_ops receive the
> mask and handle it, to be done in the future.
> 
> Signed-off-by: Sricharan R <sricharan@codeaurora.org>
> ---
>  drivers/of/device.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/of/device.c b/drivers/of/device.c
> index c17c19d..c2ae6bb 100644
> --- a/drivers/of/device.c
> +++ b/drivers/of/device.c
> @@ -107,7 +107,7 @@ void of_dma_configure(struct device *dev, struct device_node *np)
>  	ret = of_dma_get_range(np, &dma_addr, &paddr, &size);
>  	if (ret < 0) {
>  		dma_addr = offset = 0;
> -		size = dev->coherent_dma_mask + 1;
> +		size = max(dev->coherent_dma_mask, dev->coherent_dma_mask + 1);
>  	} else {
>  		offset = PFN_DOWN(paddr - dma_addr);
>  		dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
> 

NACK.

Passing an invalid size to arch_setup_dma_ops() is only part of the problem.
size is also used in of_dma_configure() before calling arch_setup_dma_ops():

        dev->coherent_dma_mask = min(dev->coherent_dma_mask,
                                     DMA_BIT_MASK(ilog2(dma_addr + size)));
        *dev->dma_mask = min((*dev->dma_mask),
                             DMA_BIT_MASK(ilog2(dma_addr + size)));

which would be incorrect for size == 0xffffffffffffffffULL when
dma_addr != 0.  So the proposed fix really is not papering over
the base problem very well.

I agree that the proper solution involves passing a mask instead
of a size to arch_setup_dma_ops().

-Frank

Re: [PATCH V10 06/12] of: device: Fix overflow of coherent_dma_mask

[Robin Murphy]

On 06/04/17 08:01, Frank Rowand wrote:
> On 04/04/17 03:18, Sricharan R wrote:
>> Size of the dma-range is calculated as coherent_dma_mask + 1
>> and passed to arch_setup_dma_ops further. It overflows when
>> the coherent_dma_mask is set for full 64 bits 0xFFFFFFFFFFFFFFFF,
>> resulting in size getting passed as 0 wrongly. Fix this by
>> passsing in max(mask, mask + 1). Note that in this case
>> when the mask is set to full 64bits, we will be passing the mask
>> itself to arch_setup_dma_ops instead of the size. The real fix
>> for this should be to make arch_setup_dma_ops receive the
>> mask and handle it, to be done in the future.
>>
>> Signed-off-by: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
>> ---
>>  drivers/of/device.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/drivers/of/device.c b/drivers/of/device.c
>> index c17c19d..c2ae6bb 100644
>> --- a/drivers/of/device.c
>> +++ b/drivers/of/device.c
>> @@ -107,7 +107,7 @@ void of_dma_configure(struct device *dev, struct device_node *np)
>>  	ret = of_dma_get_range(np, &dma_addr, &paddr, &size);
>>  	if (ret < 0) {
>>  		dma_addr = offset = 0;
>> -		size = dev->coherent_dma_mask + 1;
>> +		size = max(dev->coherent_dma_mask, dev->coherent_dma_mask + 1);
>>  	} else {
>>  		offset = PFN_DOWN(paddr - dma_addr);
>>  		dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
>>
> 
> NACK.
> 
> Passing an invalid size to arch_setup_dma_ops() is only part of the problem.
> size is also used in of_dma_configure() before calling arch_setup_dma_ops():
> 
>         dev->coherent_dma_mask = min(dev->coherent_dma_mask,
>                                      DMA_BIT_MASK(ilog2(dma_addr + size)));
>         *dev->dma_mask = min((*dev->dma_mask),
>                              DMA_BIT_MASK(ilog2(dma_addr + size)));
> 
> which would be incorrect for size == 0xffffffffffffffffULL when
> dma_addr != 0.  So the proposed fix really is not papering over
> the base problem very well.

I'm not sure I agree there. Granted, there exist many more problematic
aspects than are dealt with here (I've got more patches cooking to sort
out some of the other issues we have with dma-ranges), but considering
size specifically:

- It is not possible to explicitly specify a range with a size of 2^64
in DT. If someone does specify a size of 0, they've done a silly thing
and should not be surprised that it ends badly.

- It *is* perfectly legitimate for bus code (or a previous device
driver, once we start coming here at probe time) to have set a device's
DMA mask to 0xffffffffffffffffULL. If this code then blindly overflows
and infers an invalid size of 0 from that, breaking things in the
process, that is this code's fault alone. It just so happens that
nothing managed to trigger the latent problem until patch #7 here shakes
up the callsites.

Yes, wacky impossible base + size combinations in DT were a theoretical
problem before, and remain a theoretical problem, but also fall into the
"how did you ever expect this to work?" category. There's certainly
plenty more we can do to improve the DT parsing/validation, but that
still doesn't apply to this path where the information is *not* coming
from the DT at all.

> I agree that the proper solution involves passing a mask instead
> of a size to arch_setup_dma_ops().

Having started writing that patch too, I can tell you it's a big bugger
touching multiple architectures and fixing up various drivers doing
stupid things, hence why I'm happy with this point fix being the lesser
of two evils in terms of not holding up this mostly-orthogonal series.

Robin.

> 
> -Frank
> 

Re: [PATCH V10 06/12] of: device: Fix overflow of coherent_dma_mask

[Rob Herring]

On Thu, Apr 6, 2017 at 5:24 AM, Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org> wrote:
> On 06/04/17 08:01, Frank Rowand wrote:
>> On 04/04/17 03:18, Sricharan R wrote:
>>> Size of the dma-range is calculated as coherent_dma_mask + 1
>>> and passed to arch_setup_dma_ops further. It overflows when
>>> the coherent_dma_mask is set for full 64 bits 0xFFFFFFFFFFFFFFFF,
>>> resulting in size getting passed as 0 wrongly. Fix this by
>>> passsing in max(mask, mask + 1). Note that in this case
>>> when the mask is set to full 64bits, we will be passing the mask
>>> itself to arch_setup_dma_ops instead of the size. The real fix
>>> for this should be to make arch_setup_dma_ops receive the
>>> mask and handle it, to be done in the future.
>>>
>>> Signed-off-by: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
>>> ---
>>>  drivers/of/device.c | 2 +-
>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/of/device.c b/drivers/of/device.c
>>> index c17c19d..c2ae6bb 100644
>>> --- a/drivers/of/device.c
>>> +++ b/drivers/of/device.c
>>> @@ -107,7 +107,7 @@ void of_dma_configure(struct device *dev, struct device_node *np)
>>>      ret = of_dma_get_range(np, &dma_addr, &paddr, &size);
>>>      if (ret < 0) {
>>>              dma_addr = offset = 0;
>>> -            size = dev->coherent_dma_mask + 1;
>>> +            size = max(dev->coherent_dma_mask, dev->coherent_dma_mask + 1);
>>>      } else {
>>>              offset = PFN_DOWN(paddr - dma_addr);
>>>              dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
>>>
>>
>> NACK.
>>
>> Passing an invalid size to arch_setup_dma_ops() is only part of the problem.
>> size is also used in of_dma_configure() before calling arch_setup_dma_ops():
>>
>>         dev->coherent_dma_mask = min(dev->coherent_dma_mask,
>>                                      DMA_BIT_MASK(ilog2(dma_addr + size)));
>>         *dev->dma_mask = min((*dev->dma_mask),
>>                              DMA_BIT_MASK(ilog2(dma_addr + size)));
>>
>> which would be incorrect for size == 0xffffffffffffffffULL when
>> dma_addr != 0.  So the proposed fix really is not papering over
>> the base problem very well.
>
> I'm not sure I agree there. Granted, there exist many more problematic
> aspects than are dealt with here (I've got more patches cooking to sort
> out some of the other issues we have with dma-ranges), but considering
> size specifically:
>
> - It is not possible to explicitly specify a range with a size of 2^64
> in DT. If someone does specify a size of 0, they've done a silly thing
> and should not be surprised that it ends badly.

And because of this, we allow ~0 (both 32 and 64 bit) in DT dma-ranges
and fix these up as 2^32 and 2^64 sizes.

Rob

Re: [PATCH V10 06/12] of: device: Fix overflow of coherent_dma_mask

[Robin Murphy]

On 06/04/17 14:56, Rob Herring wrote:
> On Thu, Apr 6, 2017 at 5:24 AM, Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org> wrote:
>> On 06/04/17 08:01, Frank Rowand wrote:
>>> On 04/04/17 03:18, Sricharan R wrote:
>>>> Size of the dma-range is calculated as coherent_dma_mask + 1
>>>> and passed to arch_setup_dma_ops further. It overflows when
>>>> the coherent_dma_mask is set for full 64 bits 0xFFFFFFFFFFFFFFFF,
>>>> resulting in size getting passed as 0 wrongly. Fix this by
>>>> passsing in max(mask, mask + 1). Note that in this case
>>>> when the mask is set to full 64bits, we will be passing the mask
>>>> itself to arch_setup_dma_ops instead of the size. The real fix
>>>> for this should be to make arch_setup_dma_ops receive the
>>>> mask and handle it, to be done in the future.
>>>>
>>>> Signed-off-by: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
>>>> ---
>>>>  drivers/of/device.c | 2 +-
>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/drivers/of/device.c b/drivers/of/device.c
>>>> index c17c19d..c2ae6bb 100644
>>>> --- a/drivers/of/device.c
>>>> +++ b/drivers/of/device.c
>>>> @@ -107,7 +107,7 @@ void of_dma_configure(struct device *dev, struct device_node *np)
>>>>      ret = of_dma_get_range(np, &dma_addr, &paddr, &size);
>>>>      if (ret < 0) {
>>>>              dma_addr = offset = 0;
>>>> -            size = dev->coherent_dma_mask + 1;
>>>> +            size = max(dev->coherent_dma_mask, dev->coherent_dma_mask + 1);
>>>>      } else {
>>>>              offset = PFN_DOWN(paddr - dma_addr);
>>>>              dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
>>>>
>>>
>>> NACK.
>>>
>>> Passing an invalid size to arch_setup_dma_ops() is only part of the problem.
>>> size is also used in of_dma_configure() before calling arch_setup_dma_ops():
>>>
>>>         dev->coherent_dma_mask = min(dev->coherent_dma_mask,
>>>                                      DMA_BIT_MASK(ilog2(dma_addr + size)));
>>>         *dev->dma_mask = min((*dev->dma_mask),
>>>                              DMA_BIT_MASK(ilog2(dma_addr + size)));
>>>
>>> which would be incorrect for size == 0xffffffffffffffffULL when
>>> dma_addr != 0.  So the proposed fix really is not papering over
>>> the base problem very well.
>>
>> I'm not sure I agree there. Granted, there exist many more problematic
>> aspects than are dealt with here (I've got more patches cooking to sort
>> out some of the other issues we have with dma-ranges), but considering
>> size specifically:
>>
>> - It is not possible to explicitly specify a range with a size of 2^64
>> in DT. If someone does specify a size of 0, they've done a silly thing
>> and should not be surprised that it ends badly.
> 
> And because of this, we allow ~0 (both 32 and 64 bit) in DT dma-ranges
> and fix these up as 2^32 and 2^64 sizes.

...which is what Frank's patch gets rid of.

Robin.

> 
> Rob
> 

Re: [PATCH V10 06/12] of: device: Fix overflow of coherent_dma_mask

[Frank Rowand]

On 04/06/17 03:24, Robin Murphy wrote:
> On 06/04/17 08:01, Frank Rowand wrote:
>> On 04/04/17 03:18, Sricharan R wrote:
>>> Size of the dma-range is calculated as coherent_dma_mask + 1
>>> and passed to arch_setup_dma_ops further. It overflows when
>>> the coherent_dma_mask is set for full 64 bits 0xFFFFFFFFFFFFFFFF,
>>> resulting in size getting passed as 0 wrongly. Fix this by
>>> passsing in max(mask, mask + 1). Note that in this case
>>> when the mask is set to full 64bits, we will be passing the mask
>>> itself to arch_setup_dma_ops instead of the size. The real fix
>>> for this should be to make arch_setup_dma_ops receive the
>>> mask and handle it, to be done in the future.
>>>
>>> Signed-off-by: Sricharan R <sricharan@codeaurora.org>
>>> ---
>>>  drivers/of/device.c | 2 +-
>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/of/device.c b/drivers/of/device.c
>>> index c17c19d..c2ae6bb 100644
>>> --- a/drivers/of/device.c
>>> +++ b/drivers/of/device.c
>>> @@ -107,7 +107,7 @@ void of_dma_configure(struct device *dev, struct device_node *np)
>>>  	ret = of_dma_get_range(np, &dma_addr, &paddr, &size);
>>>  	if (ret < 0) {
>>>  		dma_addr = offset = 0;
>>> -		size = dev->coherent_dma_mask + 1;
>>> +		size = max(dev->coherent_dma_mask, dev->coherent_dma_mask + 1);
>>>  	} else {
>>>  		offset = PFN_DOWN(paddr - dma_addr);
>>>  		dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
>>>
>>
>> NACK.
>>
>> Passing an invalid size to arch_setup_dma_ops() is only part of the problem.
>> size is also used in of_dma_configure() before calling arch_setup_dma_ops():
>>
>>         dev->coherent_dma_mask = min(dev->coherent_dma_mask,
>>                                      DMA_BIT_MASK(ilog2(dma_addr + size)));
>>         *dev->dma_mask = min((*dev->dma_mask),
>>                              DMA_BIT_MASK(ilog2(dma_addr + size)));
>>
>> which would be incorrect for size == 0xffffffffffffffffULL when
>> dma_addr != 0.  So the proposed fix really is not papering over
>> the base problem very well.
> 
> I'm not sure I agree there. Granted, there exist many more problematic
> aspects than are dealt with here (I've got more patches cooking to sort
> out some of the other issues we have with dma-ranges), but considering
> size specifically:
> 
> - It is not possible to explicitly specify a range with a size of 2^64
> in DT. If someone does specify a size of 0, they've done a silly thing
> and should not be surprised that it ends badly.
> 
> - It *is* perfectly legitimate for bus code (or a previous device
> driver, once we start coming here at probe time) to have set a device's
> DMA mask to 0xffffffffffffffffULL. If this code then blindly overflows
> and infers an invalid size of 0 from that, breaking things in the
> process, that is this code's fault alone. It just so happens that
> nothing managed to trigger the latent problem until patch #7 here shakes
> up the callsites.

The existing code that uses size does not appear capable of dealing with
the case of DMA mask of 0xffffffffffffffffULL since 2^64 does not fit
into size.

The code affected by the DMA mask is not within my area of knowledge, so
take the following with a grain of salt.  If a DMA mask of
0xffffffffffffffffULL is provided, would the code still work without error
(though with reduced capability) if the mask was changed to
0xefffffffffffffffULL?  I would guess that the location to do so would
be where dev->coherent_dma_mask is set, or some other location that
is not of_dma_configure().  This would just be a temporary workaround.


> Yes, wacky impossible base + size combinations in DT were a theoretical
> problem before, and remain a theoretical problem, but also fall into the
> "how did you ever expect this to work?" category. There's certainly
> plenty more we can do to improve the DT parsing/validation, but that
> still doesn't apply to this path where the information is *not* coming
> from the DT at all.
> 
>> I agree that the proper solution involves passing a mask instead
>> of a size to arch_setup_dma_ops().
> 
> Having started writing that patch too, I can tell you it's a big bugger
> touching multiple architectures and fixing up various drivers doing
> stupid things, hence why I'm happy with this point fix being the lesser
> of two evils in terms of not holding up this mostly-orthogonal series.
> 
> Robin.
> 
>>
>> -Frank
>>
> 
> 

Re: [PATCH V10 06/12] of: device: Fix overflow of coherent_dma_mask

[Sricharan R]

Hi Frank,

On 4/6/2017 12:31 PM, Frank Rowand wrote:
> On 04/04/17 03:18, Sricharan R wrote:
>> Size of the dma-range is calculated as coherent_dma_mask + 1
>> and passed to arch_setup_dma_ops further. It overflows when
>> the coherent_dma_mask is set for full 64 bits 0xFFFFFFFFFFFFFFFF,
>> resulting in size getting passed as 0 wrongly. Fix this by
>> passsing in max(mask, mask + 1). Note that in this case
>> when the mask is set to full 64bits, we will be passing the mask
>> itself to arch_setup_dma_ops instead of the size. The real fix
>> for this should be to make arch_setup_dma_ops receive the
>> mask and handle it, to be done in the future.
>>
>> Signed-off-by: Sricharan R <sricharan@codeaurora.org>
>> ---
>>  drivers/of/device.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/drivers/of/device.c b/drivers/of/device.c
>> index c17c19d..c2ae6bb 100644
>> --- a/drivers/of/device.c
>> +++ b/drivers/of/device.c
>> @@ -107,7 +107,7 @@ void of_dma_configure(struct device *dev, struct device_node *np)
>>  	ret = of_dma_get_range(np, &dma_addr, &paddr, &size);
>>  	if (ret < 0) {
>>  		dma_addr = offset = 0;
>> -		size = dev->coherent_dma_mask + 1;
>> +		size = max(dev->coherent_dma_mask, dev->coherent_dma_mask + 1);
>>  	} else {
>>  		offset = PFN_DOWN(paddr - dma_addr);
>>  		dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
>>
>
> NACK.
>
> Passing an invalid size to arch_setup_dma_ops() is only part of the problem.
> size is also used in of_dma_configure() before calling arch_setup_dma_ops():
>
>         dev->coherent_dma_mask = min(dev->coherent_dma_mask,
>                                      DMA_BIT_MASK(ilog2(dma_addr + size)));
>         *dev->dma_mask = min((*dev->dma_mask),
>                              DMA_BIT_MASK(ilog2(dma_addr + size)));
>
> which would be incorrect for size == 0xffffffffffffffffULL when
> dma_addr != 0.  So the proposed fix really is not papering over
> the base problem very well.
>

Ok, but with your fix for of_dma_get_range and the above fix,
dma_addr will be '0' when size = 0xffffffffffffffffULL,
but DMA_BIT_MASK(ilog2(dma_addr + size)) would be wrong though,
making coherent_dma_mask to be smaller 0x7fffffffffffffffULL.

Regards,
  Sricharan

> I agree that the proper solution involves passing a mask instead
> of a size to arch_setup_dma_ops().
>

-- 
"QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

Re: [PATCH V10 06/12] of: device: Fix overflow of coherent_dma_mask

[Frank Rowand]

On 04/06/17 04:01, Sricharan R wrote:
> Hi Frank,
> 
> On 4/6/2017 12:31 PM, Frank Rowand wrote:
>> On 04/04/17 03:18, Sricharan R wrote:
>>> Size of the dma-range is calculated as coherent_dma_mask + 1
>>> and passed to arch_setup_dma_ops further. It overflows when
>>> the coherent_dma_mask is set for full 64 bits 0xFFFFFFFFFFFFFFFF,
>>> resulting in size getting passed as 0 wrongly. Fix this by
>>> passsing in max(mask, mask + 1). Note that in this case
>>> when the mask is set to full 64bits, we will be passing the mask
>>> itself to arch_setup_dma_ops instead of the size. The real fix
>>> for this should be to make arch_setup_dma_ops receive the
>>> mask and handle it, to be done in the future.
>>>
>>> Signed-off-by: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
>>> ---
>>>  drivers/of/device.c | 2 +-
>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/of/device.c b/drivers/of/device.c
>>> index c17c19d..c2ae6bb 100644
>>> --- a/drivers/of/device.c
>>> +++ b/drivers/of/device.c
>>> @@ -107,7 +107,7 @@ void of_dma_configure(struct device *dev, struct device_node *np)
>>>      ret = of_dma_get_range(np, &dma_addr, &paddr, &size);
>>>      if (ret < 0) {
>>>          dma_addr = offset = 0;
>>> -        size = dev->coherent_dma_mask + 1;
>>> +        size = max(dev->coherent_dma_mask, dev->coherent_dma_mask + 1);
>>>      } else {
>>>          offset = PFN_DOWN(paddr - dma_addr);
>>>          dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
>>>
>>
>> NACK.
>>
>> Passing an invalid size to arch_setup_dma_ops() is only part of the problem.
>> size is also used in of_dma_configure() before calling arch_setup_dma_ops():
>>
>>         dev->coherent_dma_mask = min(dev->coherent_dma_mask,
>>                                      DMA_BIT_MASK(ilog2(dma_addr + size)));
>>         *dev->dma_mask = min((*dev->dma_mask),
>>                              DMA_BIT_MASK(ilog2(dma_addr + size)));
>>
>> which would be incorrect for size == 0xffffffffffffffffULL when
>> dma_addr != 0.  So the proposed fix really is not papering over
>> the base problem very well.
>>
> 
> Ok, but with your fix for of_dma_get_range and the above fix,
> dma_addr will be '0' when size = 0xffffffffffffffffULL,
> but DMA_BIT_MASK(ilog2(dma_addr + size)) would be wrong though,
> making coherent_dma_mask to be smaller 0x7fffffffffffffffULL.

Yes, that was my point.  Setting size to 0x7fffffffffffffffULL
affects several places.  Another potential location (based only
on the function header comment, not from reading the code) is
iommu_dma_init_domain().  The header comment says:

    * @base and @size should be exact multiples of IOMMU page granularity to
    * avoid rounding surprises.

I have not read enough context to really understand of_dma_configure(), but
it seems there is yet another issue in how the error return case from
of_dma_get_range() is handled (with the existing code, as well as if
my patch gets accepted).  An error return value can mean _either_
there is no dma-ranges property _or_ "an other problem occurred".  Should
the "an other problem occurred" case be handled by defaulting size to
a value based on dev->coherent_dma_mask (the current case) or should the
attempt to set up the DMA configuration just fail?

> 
> Regards,
>  Sricharan
> 
>> I agree that the proper solution involves passing a mask instead
>> of a size to arch_setup_dma_ops().
>>
> 

Re: [PATCH V10 06/12] of: device: Fix overflow of coherent_dma_mask

[Sricharan R]

Hi Frank,

On 4/7/2017 1:04 AM, Frank Rowand wrote:
> On 04/06/17 04:01, Sricharan R wrote:
>> Hi Frank,
>>
>> On 4/6/2017 12:31 PM, Frank Rowand wrote:
>>> On 04/04/17 03:18, Sricharan R wrote:
>>>> Size of the dma-range is calculated as coherent_dma_mask + 1
>>>> and passed to arch_setup_dma_ops further. It overflows when
>>>> the coherent_dma_mask is set for full 64 bits 0xFFFFFFFFFFFFFFFF,
>>>> resulting in size getting passed as 0 wrongly. Fix this by
>>>> passsing in max(mask, mask + 1). Note that in this case
>>>> when the mask is set to full 64bits, we will be passing the mask
>>>> itself to arch_setup_dma_ops instead of the size. The real fix
>>>> for this should be to make arch_setup_dma_ops receive the
>>>> mask and handle it, to be done in the future.
>>>>
>>>> Signed-off-by: Sricharan R <sricharan@codeaurora.org>
>>>> ---
>>>>  drivers/of/device.c | 2 +-
>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/drivers/of/device.c b/drivers/of/device.c
>>>> index c17c19d..c2ae6bb 100644
>>>> --- a/drivers/of/device.c
>>>> +++ b/drivers/of/device.c
>>>> @@ -107,7 +107,7 @@ void of_dma_configure(struct device *dev, struct device_node *np)
>>>>      ret = of_dma_get_range(np, &dma_addr, &paddr, &size);
>>>>      if (ret < 0) {
>>>>          dma_addr = offset = 0;
>>>> -        size = dev->coherent_dma_mask + 1;
>>>> +        size = max(dev->coherent_dma_mask, dev->coherent_dma_mask + 1);
>>>>      } else {
>>>>          offset = PFN_DOWN(paddr - dma_addr);
>>>>          dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
>>>>
>>>
>>> NACK.
>>>
>>> Passing an invalid size to arch_setup_dma_ops() is only part of the problem.
>>> size is also used in of_dma_configure() before calling arch_setup_dma_ops():
>>>
>>>         dev->coherent_dma_mask = min(dev->coherent_dma_mask,
>>>                                      DMA_BIT_MASK(ilog2(dma_addr + size)));
>>>         *dev->dma_mask = min((*dev->dma_mask),
>>>                              DMA_BIT_MASK(ilog2(dma_addr + size)));
>>>
>>> which would be incorrect for size == 0xffffffffffffffffULL when
>>> dma_addr != 0.  So the proposed fix really is not papering over
>>> the base problem very well.
>>>
>>
>> Ok, but with your fix for of_dma_get_range and the above fix,
>> dma_addr will be '0' when size = 0xffffffffffffffffULL,
>> but DMA_BIT_MASK(ilog2(dma_addr + size)) would be wrong though,
>> making coherent_dma_mask to be smaller 0x7fffffffffffffffULL.
>
> Yes, that was my point.  Setting size to 0x7fffffffffffffffULL
> affects several places.  Another potential location (based only
> on the function header comment, not from reading the code) is
> iommu_dma_init_domain().  The header comment says:
>
>     * @base and @size should be exact multiples of IOMMU page granularity to
>     * avoid rounding surprises.
>

ok, this is the same problem that should get solved when arch_setup_dma_ops
is prepared to take mask instead of size. It would still work as said above
with a smaller mask than specified.

> I have not read enough context to really understand of_dma_configure(), but
> it seems there is yet another issue in how the error return case from
> of_dma_get_range() is handled (with the existing code, as well as if
> my patch gets accepted).  An error return value can mean _either_
> there is no dma-ranges property _or_ "an other problem occurred".  Should
> the "an other problem occurred" case be handled by defaulting size to
> a value based on dev->coherent_dma_mask (the current case) or should the
> attempt to set up the DMA configuration just fail?

The handling of return error value looks like separate item, but looks
like its correct with what is there now. (ie) when of_dma_get_range fails
either because 'dma-ranges property' populated in DT (or) because of some
erroneous DT setting, better to set the mask which the driver has specified
and ignore DT. So the above patch just corrects a mistake in that path.

Regards,
  Sricharan

-- 
"QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

Re: [PATCH V10 06/12] of: device: Fix overflow of coherent_dma_mask

[Robin Murphy]

On 06/04/17 20:34, Frank Rowand wrote:
> On 04/06/17 04:01, Sricharan R wrote:
>> Hi Frank,
>>
>> On 4/6/2017 12:31 PM, Frank Rowand wrote:
>>> On 04/04/17 03:18, Sricharan R wrote:
>>>> Size of the dma-range is calculated as coherent_dma_mask + 1
>>>> and passed to arch_setup_dma_ops further. It overflows when
>>>> the coherent_dma_mask is set for full 64 bits 0xFFFFFFFFFFFFFFFF,
>>>> resulting in size getting passed as 0 wrongly. Fix this by
>>>> passsing in max(mask, mask + 1). Note that in this case
>>>> when the mask is set to full 64bits, we will be passing the mask
>>>> itself to arch_setup_dma_ops instead of the size. The real fix
>>>> for this should be to make arch_setup_dma_ops receive the
>>>> mask and handle it, to be done in the future.
>>>>
>>>> Signed-off-by: Sricharan R <sricharan@codeaurora.org>
>>>> ---
>>>>  drivers/of/device.c | 2 +-
>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/drivers/of/device.c b/drivers/of/device.c
>>>> index c17c19d..c2ae6bb 100644
>>>> --- a/drivers/of/device.c
>>>> +++ b/drivers/of/device.c
>>>> @@ -107,7 +107,7 @@ void of_dma_configure(struct device *dev, struct device_node *np)
>>>>      ret = of_dma_get_range(np, &dma_addr, &paddr, &size);
>>>>      if (ret < 0) {
>>>>          dma_addr = offset = 0;
>>>> -        size = dev->coherent_dma_mask + 1;
>>>> +        size = max(dev->coherent_dma_mask, dev->coherent_dma_mask + 1);
>>>>      } else {
>>>>          offset = PFN_DOWN(paddr - dma_addr);
>>>>          dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
>>>>
>>>
>>> NACK.
>>>
>>> Passing an invalid size to arch_setup_dma_ops() is only part of the problem.
>>> size is also used in of_dma_configure() before calling arch_setup_dma_ops():
>>>
>>>         dev->coherent_dma_mask = min(dev->coherent_dma_mask,
>>>                                      DMA_BIT_MASK(ilog2(dma_addr + size)));
>>>         *dev->dma_mask = min((*dev->dma_mask),
>>>                              DMA_BIT_MASK(ilog2(dma_addr + size)));
>>>
>>> which would be incorrect for size == 0xffffffffffffffffULL when
>>> dma_addr != 0.  So the proposed fix really is not papering over
>>> the base problem very well.
>>>
>>
>> Ok, but with your fix for of_dma_get_range and the above fix,
>> dma_addr will be '0' when size = 0xffffffffffffffffULL,
>> but DMA_BIT_MASK(ilog2(dma_addr + size)) would be wrong though,
>> making coherent_dma_mask to be smaller 0x7fffffffffffffffULL.
> 
> Yes, that was my point.  Setting size to 0x7fffffffffffffffULL
> affects several places.  Another potential location (based only
> on the function header comment, not from reading the code) is
> iommu_dma_init_domain().  The header comment says:
> 
>     * @base and @size should be exact multiples of IOMMU page granularity to
>     * avoid rounding surprises.

That is really only referring to the fact that some of the work done
therein involves truncation to PFNs, so anyone passing in non-exact
values expecting them to round a particular way may get things off by a
page one way or the other. It's not going to have much practical
significance for real devices (in particular since size is used more as
a sanity check than any kind of actual limit there).

> I have not read enough context to really understand of_dma_configure(), but
> it seems there is yet another issue in how the error return case from
> of_dma_get_range() is handled (with the existing code, as well as if
> my patch gets accepted).  An error return value can mean _either_
> there is no dma-ranges property _or_ "an other problem occurred".  Should
> the "an other problem occurred" case be handled by defaulting size to
> a value based on dev->coherent_dma_mask (the current case) or should the
> attempt to set up the DMA configuration just fail?

There is indeed a lot wrong with of_dma_configure() and
arch_setup_dma_ops(), but fixing those is beyond the scope of this
series. This is just working around a latent bug in the one specific
case where a value is *not* derived from DT. Any DT which worked before
still works; any DT which made of_dma_configure() go wrong before still
makes of_dma_configure() go wrong exactly the same.

Whilst it's not ideal, since a DMA mask basically represents the maximum
size of address that that particular device can be given, I can't see it
making any practical difference for a full 64-bit DMA mask to be trimmed
down to 63 bits upon re-probing - no system is likely to have that many
physical address bits anyway, and I don't think any IOMMUs support that
large an IOVA space either, so as long as it's still big enough to cover
"everything", it'll be OK.

Of course, whether DMA_BIT_MASK(ilog2(dma_addr + size)) is the right
thing to do in the first place is yet another matter, as there are
plenty of cases where it results in something which can't reach the
given range at all, but again, this isn't the place. Much as I'm keen to
get the behaviour of of_dma_configure() sorted out properly, it doesn't
seem reasonable that that should suddenly block this
almost-entirely-orthogonal series that various other work has been
waiting on for some time now. The WIP patch I have for
arch_setup_dma_ops() already touches 3 architectures and 4 other
subsystems...

Robin.

> 
>>
>> Regards,
>>  Sricharan
>>
>>> I agree that the proper solution involves passing a mask instead
>>> of a size to arch_setup_dma_ops().
>>>
>>
> 

Re: [PATCH V10 06/12] of: device: Fix overflow of coherent_dma_mask

[Frank Rowand]

On 04/07/17 07:46, Robin Murphy wrote:
> On 06/04/17 20:34, Frank Rowand wrote:
>> On 04/06/17 04:01, Sricharan R wrote:
>>> Hi Frank,
>>>
>>> On 4/6/2017 12:31 PM, Frank Rowand wrote:
>>>> On 04/04/17 03:18, Sricharan R wrote:
>>>>> Size of the dma-range is calculated as coherent_dma_mask + 1
>>>>> and passed to arch_setup_dma_ops further. It overflows when
>>>>> the coherent_dma_mask is set for full 64 bits 0xFFFFFFFFFFFFFFFF,
>>>>> resulting in size getting passed as 0 wrongly. Fix this by
>>>>> passsing in max(mask, mask + 1). Note that in this case
>>>>> when the mask is set to full 64bits, we will be passing the mask
>>>>> itself to arch_setup_dma_ops instead of the size. The real fix
>>>>> for this should be to make arch_setup_dma_ops receive the
>>>>> mask and handle it, to be done in the future.
>>>>>
>>>>> Signed-off-by: Sricharan R <sricharan@codeaurora.org>
>>>>> ---
>>>>>  drivers/of/device.c | 2 +-
>>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>
>>>>> diff --git a/drivers/of/device.c b/drivers/of/device.c
>>>>> index c17c19d..c2ae6bb 100644
>>>>> --- a/drivers/of/device.c
>>>>> +++ b/drivers/of/device.c
>>>>> @@ -107,7 +107,7 @@ void of_dma_configure(struct device *dev, struct device_node *np)
>>>>>      ret = of_dma_get_range(np, &dma_addr, &paddr, &size);
>>>>>      if (ret < 0) {
>>>>>          dma_addr = offset = 0;
>>>>> -        size = dev->coherent_dma_mask + 1;
>>>>> +        size = max(dev->coherent_dma_mask, dev->coherent_dma_mask + 1);
>>>>>      } else {
>>>>>          offset = PFN_DOWN(paddr - dma_addr);
>>>>>          dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
>>>>>
>>>>
>>>> NACK.
>>>>
>>>> Passing an invalid size to arch_setup_dma_ops() is only part of the problem.
>>>> size is also used in of_dma_configure() before calling arch_setup_dma_ops():
>>>>
>>>>         dev->coherent_dma_mask = min(dev->coherent_dma_mask,
>>>>                                      DMA_BIT_MASK(ilog2(dma_addr + size)));
>>>>         *dev->dma_mask = min((*dev->dma_mask),
>>>>                              DMA_BIT_MASK(ilog2(dma_addr + size)));
>>>>
>>>> which would be incorrect for size == 0xffffffffffffffffULL when
>>>> dma_addr != 0.  So the proposed fix really is not papering over
>>>> the base problem very well.
>>>>
>>>
>>> Ok, but with your fix for of_dma_get_range and the above fix,
>>> dma_addr will be '0' when size = 0xffffffffffffffffULL,
>>> but DMA_BIT_MASK(ilog2(dma_addr + size)) would be wrong though,
>>> making coherent_dma_mask to be smaller 0x7fffffffffffffffULL.
>>
>> Yes, that was my point.  Setting size to 0x7fffffffffffffffULL
>> affects several places.  Another potential location (based only
>> on the function header comment, not from reading the code) is
>> iommu_dma_init_domain().  The header comment says:
>>
>>     * @base and @size should be exact multiples of IOMMU page granularity to
>>     * avoid rounding surprises.
> 
> That is really only referring to the fact that some of the work done
> therein involves truncation to PFNs, so anyone passing in non-exact
> values expecting them to round a particular way may get things off by a
> page one way or the other. It's not going to have much practical
> significance for real devices (in particular since size is used more as
> a sanity check than any kind of actual limit there).
> 
>> I have not read enough context to really understand of_dma_configure(), but
>> it seems there is yet another issue in how the error return case from
>> of_dma_get_range() is handled (with the existing code, as well as if
>> my patch gets accepted).  An error return value can mean _either_
>> there is no dma-ranges property _or_ "an other problem occurred".  Should
>> the "an other problem occurred" case be handled by defaulting size to
>> a value based on dev->coherent_dma_mask (the current case) or should the
>> attempt to set up the DMA configuration just fail?
> 
> There is indeed a lot wrong with of_dma_configure() and
> arch_setup_dma_ops(), but fixing those is beyond the scope of this
> series. This is just working around a latent bug in the one specific
> case where a value is *not* derived from DT. Any DT which worked before
> still works; any DT which made of_dma_configure() go wrong before still
> makes of_dma_configure() go wrong exactly the same.
> 
> Whilst it's not ideal, since a DMA mask basically represents the maximum
> size of address that that particular device can be given, I can't see it
> making any practical difference for a full 64-bit DMA mask to be trimmed
> down to 63 bits upon re-probing - no system is likely to have that many
> physical address bits anyway, and I don't think any IOMMUs support that
> large an IOVA space either, so as long as it's still big enough to cover
> "everything", it'll be OK.
> 
> Of course, whether DMA_BIT_MASK(ilog2(dma_addr + size)) is the right
> thing to do in the first place is yet another matter, as there are
> plenty of cases where it results in something which can't reach the
> given range at all, but again, this isn't the place. Much as I'm keen to
> get the behaviour of of_dma_configure() sorted out properly, it doesn't
> seem reasonable that that should suddenly block this
> almost-entirely-orthogonal series that various other work has been
> waiting on for some time now. The WIP patch I have for
> arch_setup_dma_ops() already touches 3 architectures and 4 other
> subsystems...

In a reply to my original NACK email, I just now retracted the NACK,
but with a requested change for readability.

I buy your analysis and argument here.  The patch will improve things
a little, but it will be good to revisit of_dma_configure() in the
future to further clean things up.

-Frank

> 
> Robin.
> 
>>
>>>
>>> Regards,
>>>  Sricharan
>>>
>>>> I agree that the proper solution involves passing a mask instead
>>>> of a size to arch_setup_dma_ops().
>>>>
>>>
>>
> 
> 

Re: [PATCH V10 06/12] of: device: Fix overflow of coherent_dma_mask

[Robin Murphy]

On 08/04/17 00:13, Frank Rowand wrote:
> On 04/07/17 07:46, Robin Murphy wrote:
>> On 06/04/17 20:34, Frank Rowand wrote:
>>> On 04/06/17 04:01, Sricharan R wrote:
>>>> Hi Frank,
>>>>
>>>> On 4/6/2017 12:31 PM, Frank Rowand wrote:
>>>>> On 04/04/17 03:18, Sricharan R wrote:
>>>>>> Size of the dma-range is calculated as coherent_dma_mask + 1
>>>>>> and passed to arch_setup_dma_ops further. It overflows when
>>>>>> the coherent_dma_mask is set for full 64 bits 0xFFFFFFFFFFFFFFFF,
>>>>>> resulting in size getting passed as 0 wrongly. Fix this by
>>>>>> passsing in max(mask, mask + 1). Note that in this case
>>>>>> when the mask is set to full 64bits, we will be passing the mask
>>>>>> itself to arch_setup_dma_ops instead of the size. The real fix
>>>>>> for this should be to make arch_setup_dma_ops receive the
>>>>>> mask and handle it, to be done in the future.
>>>>>>
>>>>>> Signed-off-by: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
>>>>>> ---
>>>>>>  drivers/of/device.c | 2 +-
>>>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>>
>>>>>> diff --git a/drivers/of/device.c b/drivers/of/device.c
>>>>>> index c17c19d..c2ae6bb 100644
>>>>>> --- a/drivers/of/device.c
>>>>>> +++ b/drivers/of/device.c
>>>>>> @@ -107,7 +107,7 @@ void of_dma_configure(struct device *dev, struct device_node *np)
>>>>>>      ret = of_dma_get_range(np, &dma_addr, &paddr, &size);
>>>>>>      if (ret < 0) {
>>>>>>          dma_addr = offset = 0;
>>>>>> -        size = dev->coherent_dma_mask + 1;
>>>>>> +        size = max(dev->coherent_dma_mask, dev->coherent_dma_mask + 1);
>>>>>>      } else {
>>>>>>          offset = PFN_DOWN(paddr - dma_addr);
>>>>>>          dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
>>>>>>
>>>>>
>>>>> NACK.
>>>>>
>>>>> Passing an invalid size to arch_setup_dma_ops() is only part of the problem.
>>>>> size is also used in of_dma_configure() before calling arch_setup_dma_ops():
>>>>>
>>>>>         dev->coherent_dma_mask = min(dev->coherent_dma_mask,
>>>>>                                      DMA_BIT_MASK(ilog2(dma_addr + size)));
>>>>>         *dev->dma_mask = min((*dev->dma_mask),
>>>>>                              DMA_BIT_MASK(ilog2(dma_addr + size)));
>>>>>
>>>>> which would be incorrect for size == 0xffffffffffffffffULL when
>>>>> dma_addr != 0.  So the proposed fix really is not papering over
>>>>> the base problem very well.
>>>>>
>>>>
>>>> Ok, but with your fix for of_dma_get_range and the above fix,
>>>> dma_addr will be '0' when size = 0xffffffffffffffffULL,
>>>> but DMA_BIT_MASK(ilog2(dma_addr + size)) would be wrong though,
>>>> making coherent_dma_mask to be smaller 0x7fffffffffffffffULL.
>>>
>>> Yes, that was my point.  Setting size to 0x7fffffffffffffffULL
>>> affects several places.  Another potential location (based only
>>> on the function header comment, not from reading the code) is
>>> iommu_dma_init_domain().  The header comment says:
>>>
>>>     * @base and @size should be exact multiples of IOMMU page granularity to
>>>     * avoid rounding surprises.
>>
>> That is really only referring to the fact that some of the work done
>> therein involves truncation to PFNs, so anyone passing in non-exact
>> values expecting them to round a particular way may get things off by a
>> page one way or the other. It's not going to have much practical
>> significance for real devices (in particular since size is used more as
>> a sanity check than any kind of actual limit there).
>>
>>> I have not read enough context to really understand of_dma_configure(), but
>>> it seems there is yet another issue in how the error return case from
>>> of_dma_get_range() is handled (with the existing code, as well as if
>>> my patch gets accepted).  An error return value can mean _either_
>>> there is no dma-ranges property _or_ "an other problem occurred".  Should
>>> the "an other problem occurred" case be handled by defaulting size to
>>> a value based on dev->coherent_dma_mask (the current case) or should the
>>> attempt to set up the DMA configuration just fail?
>>
>> There is indeed a lot wrong with of_dma_configure() and
>> arch_setup_dma_ops(), but fixing those is beyond the scope of this
>> series. This is just working around a latent bug in the one specific
>> case where a value is *not* derived from DT. Any DT which worked before
>> still works; any DT which made of_dma_configure() go wrong before still
>> makes of_dma_configure() go wrong exactly the same.
>>
>> Whilst it's not ideal, since a DMA mask basically represents the maximum
>> size of address that that particular device can be given, I can't see it
>> making any practical difference for a full 64-bit DMA mask to be trimmed
>> down to 63 bits upon re-probing - no system is likely to have that many
>> physical address bits anyway, and I don't think any IOMMUs support that
>> large an IOVA space either, so as long as it's still big enough to cover
>> "everything", it'll be OK.
>>
>> Of course, whether DMA_BIT_MASK(ilog2(dma_addr + size)) is the right
>> thing to do in the first place is yet another matter, as there are
>> plenty of cases where it results in something which can't reach the
>> given range at all, but again, this isn't the place. Much as I'm keen to
>> get the behaviour of of_dma_configure() sorted out properly, it doesn't
>> seem reasonable that that should suddenly block this
>> almost-entirely-orthogonal series that various other work has been
>> waiting on for some time now. The WIP patch I have for
>> arch_setup_dma_ops() already touches 3 architectures and 4 other
>> subsystems...
> 
> In a reply to my original NACK email, I just now retracted the NACK,
> but with a requested change for readability.
> 
> I buy your analysis and argument here.  The patch will improve things
> a little, but it will be good to revisit of_dma_configure() in the
> future to further clean things up.

Thanks! As mentioned, I have some patches brewing in this area which
should hopefully help somewhat; I'll try to get them posted this week.

Robin.

> 
> -Frank
> 
>>
>> Robin.
>>
>>>
>>>>
>>>> Regards,
>>>>  Sricharan
>>>>
>>>>> I agree that the proper solution involves passing a mask instead
>>>>> of a size to arch_setup_dma_ops().
>>>>>
>>>>
>>>
>>
>>
> 

Re: [PATCH V10 06/12] of: device: Fix overflow of coherent_dma_mask

[Frank Rowand]

On 04/06/17 00:01, Frank Rowand wrote:
> On 04/04/17 03:18, Sricharan R wrote:
>> Size of the dma-range is calculated as coherent_dma_mask + 1
>> and passed to arch_setup_dma_ops further. It overflows when
>> the coherent_dma_mask is set for full 64 bits 0xFFFFFFFFFFFFFFFF,
>> resulting in size getting passed as 0 wrongly. Fix this by
>> passsing in max(mask, mask + 1). Note that in this case
>> when the mask is set to full 64bits, we will be passing the mask
>> itself to arch_setup_dma_ops instead of the size. The real fix
>> for this should be to make arch_setup_dma_ops receive the
>> mask and handle it, to be done in the future.
>>
>> Signed-off-by: Sricharan R <sricharan@codeaurora.org>
>> ---
>>  drivers/of/device.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/drivers/of/device.c b/drivers/of/device.c
>> index c17c19d..c2ae6bb 100644
>> --- a/drivers/of/device.c
>> +++ b/drivers/of/device.c
>> @@ -107,7 +107,7 @@ void of_dma_configure(struct device *dev, struct device_node *np)
>>  	ret = of_dma_get_range(np, &dma_addr, &paddr, &size);
>>  	if (ret < 0) {
>>  		dma_addr = offset = 0;
>> -		size = dev->coherent_dma_mask + 1;
>> +		size = max(dev->coherent_dma_mask, dev->coherent_dma_mask + 1);

NACK withdrawn below.

However, I would prefer a change to this line for readability. Using max() results
in the correct result, but obscures the reason behind the algorithm, where the
intent is to avoid an overflow.  How about something like:

	size = (dev->coherent_dma_mask == 0xffffffffffffffffULL)
		? 0xffffffffffffffffULL : dev->coherent_dma_mask + 1;


>>  	} else {
>>  		offset = PFN_DOWN(paddr - dma_addr);
>>  		dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
>>
> 
> NACK.
> 
> Passing an invalid size to arch_setup_dma_ops() is only part of the problem.
> size is also used in of_dma_configure() before calling arch_setup_dma_ops():
> 
>         dev->coherent_dma_mask = min(dev->coherent_dma_mask,
>                                      DMA_BIT_MASK(ilog2(dma_addr + size)));
>         *dev->dma_mask = min((*dev->dma_mask),
>                              DMA_BIT_MASK(ilog2(dma_addr + size)));
> 
> which would be incorrect for size == 0xffffffffffffffffULL when
> dma_addr != 0.  So the proposed fix really is not papering over

  ^^^^^^^^^^^^^  This is the flaw in my objection.  When in the
(ret < 0) path, dma_addr is set to zero.  So my worry about dma_addr != 0
is baseless.

I withdraw my NACK because my analysis was flawed.

-Frank

> the base problem very well.
> 
> I agree that the proper solution involves passing a mask instead
> of a size to arch_setup_dma_ops().
> 
> -Frank
> 

[PATCH V10 07/12] of/acpi: Configure dma operations at probe time for platform/amba/pci bus devices

[Sricharan R]

Configuring DMA ops at probe time will allow deferring device probe when
the IOMMU isn't available yet. The dma_configure for the device is
now called from the generic device_attach callback just before the
bus/driver probe is called. This way, configuring the DMA ops for the
device would be called at the same place for all bus_types, hence the
deferred probing mechanism should work for all buses as well.

pci_bus_add_devices    (platform/amba)(_device_create/driver_register)
       |                         |
pci_bus_add_device     (device_add/driver_register)
       |                         |
device_attach           device_initial_probe
       |                         |
__device_attach_driver    __device_attach_driver
       |
driver_probe_device
       |
really_probe
       |
dma_configure

Similarly on the device/driver_unregister path __device_release_driver is
called which inturn calls dma_deconfigure.

This patch changes the dma ops configuration to probe time for
both OF and ACPI based platform/amba/pci bus devices.

Tested-by: Marek Szyprowski <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
Tested-by: Hanjun Guo <hanjun.guo-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
Acked-by: Bjorn Helgaas <bhelgaas-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> (drivers/pci part)
Acked-by: Rafael J. Wysocki <rafael.j.wysocki-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
---

 [V10] Added dummy dma_(de)configure functions in case
       of !CONFIG_HAS_DMA to avoid build breaks.

 drivers/acpi/glue.c         |  5 -----
 drivers/base/dd.c           |  9 +++++++++
 drivers/base/dma-mapping.c  | 40 ++++++++++++++++++++++++++++++++++++++++
 drivers/of/platform.c       |  5 +----
 drivers/pci/probe.c         | 28 ----------------------------
 include/linux/dma-mapping.h | 12 ++++++++++++
 6 files changed, 62 insertions(+), 37 deletions(-)

diff --git a/drivers/acpi/glue.c b/drivers/acpi/glue.c
index fb19e1c..c05f241 100644
--- a/drivers/acpi/glue.c
+++ b/drivers/acpi/glue.c
@@ -176,7 +176,6 @@ int acpi_bind_one(struct device *dev, struct acpi_device *acpi_dev)
 	struct list_head *physnode_list;
 	unsigned int node_id;
 	int retval = -EINVAL;
-	enum dev_dma_attr attr;
 
 	if (has_acpi_companion(dev)) {
 		if (acpi_dev) {
@@ -233,10 +232,6 @@ int acpi_bind_one(struct device *dev, struct acpi_device *acpi_dev)
 	if (!has_acpi_companion(dev))
 		ACPI_COMPANION_SET(dev, acpi_dev);
 
-	attr = acpi_get_dma_attr(acpi_dev);
-	if (attr != DEV_DMA_NOT_SUPPORTED)
-		acpi_dma_configure(dev, attr);
-
 	acpi_physnode_link_name(physical_node_name, node_id);
 	retval = sysfs_create_link(&acpi_dev->dev.kobj, &dev->kobj,
 				   physical_node_name);
diff --git a/drivers/base/dd.c b/drivers/base/dd.c
index a1fbf55..4882f06 100644
--- a/drivers/base/dd.c
+++ b/drivers/base/dd.c
@@ -19,6 +19,7 @@
 
 #include <linux/device.h>
 #include <linux/delay.h>
+#include <linux/dma-mapping.h>
 #include <linux/module.h>
 #include <linux/kthread.h>
 #include <linux/wait.h>
@@ -356,6 +357,10 @@ static int really_probe(struct device *dev, struct device_driver *drv)
 	if (ret)
 		goto pinctrl_bind_failed;
 
+	ret = dma_configure(dev);
+	if (ret)
+		goto dma_failed;
+
 	if (driver_sysfs_add(dev)) {
 		printk(KERN_ERR "%s: driver_sysfs_add(%s) failed\n",
 			__func__, dev_name(dev));
@@ -417,6 +422,8 @@ static int really_probe(struct device *dev, struct device_driver *drv)
 	goto done;
 
 probe_failed:
+	dma_deconfigure(dev);
+dma_failed:
 	if (dev->bus)
 		blocking_notifier_call_chain(&dev->bus->p->bus_notifier,
 					     BUS_NOTIFY_DRIVER_NOT_BOUND, dev);
@@ -826,6 +833,8 @@ static void __device_release_driver(struct device *dev, struct device *parent)
 			drv->remove(dev);
 
 		device_links_driver_cleanup(dev);
+		dma_deconfigure(dev);
+
 		devres_release_all(dev);
 		dev->driver = NULL;
 		dev_set_drvdata(dev, NULL);
diff --git a/drivers/base/dma-mapping.c b/drivers/base/dma-mapping.c
index efd71cf..449b948 100644
--- a/drivers/base/dma-mapping.c
+++ b/drivers/base/dma-mapping.c
@@ -7,9 +7,11 @@
  * This file is released under the GPLv2.
  */
 
+#include <linux/acpi.h>
 #include <linux/dma-mapping.h>
 #include <linux/export.h>
 #include <linux/gfp.h>
+#include <linux/of_device.h>
 #include <linux/slab.h>
 #include <linux/vmalloc.h>
 
@@ -341,3 +343,41 @@ void dma_common_free_remap(void *cpu_addr, size_t size, unsigned long vm_flags)
 	vunmap(cpu_addr);
 }
 #endif
+
+/*
+ * Common configuration to enable DMA API use for a device
+ */
+#include <linux/pci.h>
+
+int dma_configure(struct device *dev)
+{
+	struct device *bridge = NULL, *dma_dev = dev;
+	enum dev_dma_attr attr;
+
+	if (dev_is_pci(dev)) {
+		bridge = pci_get_host_bridge_device(to_pci_dev(dev));
+		dma_dev = bridge;
+		if (IS_ENABLED(CONFIG_OF) && dma_dev->parent &&
+		    dma_dev->parent->of_node)
+			dma_dev = dma_dev->parent;
+	}
+
+	if (dma_dev->of_node) {
+		of_dma_configure(dev, dma_dev->of_node);
+	} else if (has_acpi_companion(dma_dev)) {
+		attr = acpi_get_dma_attr(to_acpi_device_node(dma_dev->fwnode));
+		if (attr != DEV_DMA_NOT_SUPPORTED)
+			acpi_dma_configure(dev, attr);
+	}
+
+	if (bridge)
+		pci_put_host_bridge_device(bridge);
+
+	return 0;
+}
+
+void dma_deconfigure(struct device *dev)
+{
+	of_dma_deconfigure(dev);
+	acpi_dma_deconfigure(dev);
+}
diff --git a/drivers/of/platform.c b/drivers/of/platform.c
index 5344db5..2aa4ebb 100644
--- a/drivers/of/platform.c
+++ b/drivers/of/platform.c
@@ -22,6 +22,7 @@
 #include <linux/slab.h>
 #include <linux/of_address.h>
 #include <linux/of_device.h>
+#include <linux/of_iommu.h>
 #include <linux/of_irq.h>
 #include <linux/of_platform.h>
 #include <linux/platform_device.h>
@@ -186,11 +187,9 @@ static struct platform_device *of_platform_device_create_pdata(
 
 	dev->dev.bus = &platform_bus_type;
 	dev->dev.platform_data = platform_data;
-	of_dma_configure(&dev->dev, dev->dev.of_node);
 	of_msi_configure(&dev->dev, dev->dev.of_node);
 
 	if (of_device_add(dev) != 0) {
-		of_dma_deconfigure(&dev->dev);
 		platform_device_put(dev);
 		goto err_clear_flag;
 	}
@@ -248,7 +247,6 @@ static struct amba_device *of_amba_device_create(struct device_node *node,
 		dev_set_name(&dev->dev, "%s", bus_id);
 	else
 		of_device_make_bus_id(&dev->dev);
-	of_dma_configure(&dev->dev, dev->dev.of_node);
 
 	/* Allow the HW Peripheral ID to be overridden */
 	prop = of_get_property(node, "arm,primecell-periphid", NULL);
@@ -542,7 +540,6 @@ static int of_platform_device_destroy(struct device *dev, void *data)
 		amba_device_unregister(to_amba_device(dev));
 #endif
 
-	of_dma_deconfigure(dev);
 	of_node_clear_flag(dev->of_node, OF_POPULATED);
 	of_node_clear_flag(dev->of_node, OF_POPULATED_BUS);
 	return 0;
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index dfc9a27..5a8dd43 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -1893,33 +1893,6 @@ static void pci_set_msi_domain(struct pci_dev *dev)
 	dev_set_msi_domain(&dev->dev, d);
 }
 
-/**
- * pci_dma_configure - Setup DMA configuration
- * @dev: ptr to pci_dev struct of the PCI device
- *
- * Function to update PCI devices's DMA configuration using the same
- * info from the OF node or ACPI node of host bridge's parent (if any).
- */
-static void pci_dma_configure(struct pci_dev *dev)
-{
-	struct device *bridge = pci_get_host_bridge_device(dev);
-
-	if (IS_ENABLED(CONFIG_OF) &&
-		bridge->parent && bridge->parent->of_node) {
-			of_dma_configure(&dev->dev, bridge->parent->of_node);
-	} else if (has_acpi_companion(bridge)) {
-		struct acpi_device *adev = to_acpi_device_node(bridge->fwnode);
-		enum dev_dma_attr attr = acpi_get_dma_attr(adev);
-
-		if (attr == DEV_DMA_NOT_SUPPORTED)
-			dev_warn(&dev->dev, "DMA not supported.\n");
-		else
-			acpi_dma_configure(&dev->dev, attr);
-	}
-
-	pci_put_host_bridge_device(bridge);
-}
-
 void pci_device_add(struct pci_dev *dev, struct pci_bus *bus)
 {
 	int ret;
@@ -1933,7 +1906,6 @@ void pci_device_add(struct pci_dev *dev, struct pci_bus *bus)
 	dev->dev.dma_mask = &dev->dma_mask;
 	dev->dev.dma_parms = &dev->dma_parms;
 	dev->dev.coherent_dma_mask = 0xffffffffull;
-	pci_dma_configure(dev);
 
 	pci_set_dma_max_seg_size(dev, 65536);
 	pci_set_dma_seg_boundary(dev, 0xffffffff);
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 0977317..4f3eece 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -728,6 +728,18 @@ void *dma_mark_declared_memory_occupied(struct device *dev,
 }
 #endif /* CONFIG_HAVE_GENERIC_DMA_COHERENT */
 
+#ifdef CONFIG_HAS_DMA
+int dma_configure(struct device *dev);
+void dma_deconfigure(struct device *dev);
+#else
+static inline int dma_configure(struct device *dev)
+{
+	return 0;
+}
+
+static inline void dma_deconfigure(struct device *dev) {}
+#endif
+
 /*
  * Managed DMA API
  */
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

Re: [PATCH V10 07/12] of/acpi: Configure dma operations at probe time for platform/amba/pci bus devices

[Robin Murphy]

On 04/04/17 11:18, Sricharan R wrote:
> Configuring DMA ops at probe time will allow deferring device probe when
> the IOMMU isn't available yet. The dma_configure for the device is
> now called from the generic device_attach callback just before the
> bus/driver probe is called. This way, configuring the DMA ops for the
> device would be called at the same place for all bus_types, hence the
> deferred probing mechanism should work for all buses as well.
> 
> pci_bus_add_devices    (platform/amba)(_device_create/driver_register)
>        |                         |
> pci_bus_add_device     (device_add/driver_register)
>        |                         |
> device_attach           device_initial_probe
>        |                         |
> __device_attach_driver    __device_attach_driver
>        |
> driver_probe_device
>        |
> really_probe
>        |
> dma_configure
> 
> Similarly on the device/driver_unregister path __device_release_driver is
> called which inturn calls dma_deconfigure.
> 
> This patch changes the dma ops configuration to probe time for
> both OF and ACPI based platform/amba/pci bus devices.

Reviewed-by: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>

It's possible we could subsume {of,acpi}_dma_deconfigure() into
dma_deconfigure() entirely in future if it becomes clear that neither of
them will ever need to do anything firmware-specific, but I think for
now it's probably safer to keep the current symmetry - calling
arch_teardown_dma_ops() twice is benign (and even if it weren't, I'd say
it really should be!)

Robin.

> Tested-by: Marek Szyprowski <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
> Tested-by: Hanjun Guo <hanjun.guo-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
> Acked-by: Bjorn Helgaas <bhelgaas-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> (drivers/pci part)
> Acked-by: Rafael J. Wysocki <rafael.j.wysocki-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
> Signed-off-by: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
> ---
> 
>  [V10] Added dummy dma_(de)configure functions in case
>        of !CONFIG_HAS_DMA to avoid build breaks.
> 
>  drivers/acpi/glue.c         |  5 -----
>  drivers/base/dd.c           |  9 +++++++++
>  drivers/base/dma-mapping.c  | 40 ++++++++++++++++++++++++++++++++++++++++
>  drivers/of/platform.c       |  5 +----
>  drivers/pci/probe.c         | 28 ----------------------------
>  include/linux/dma-mapping.h | 12 ++++++++++++
>  6 files changed, 62 insertions(+), 37 deletions(-)
> 
> diff --git a/drivers/acpi/glue.c b/drivers/acpi/glue.c
> index fb19e1c..c05f241 100644
> --- a/drivers/acpi/glue.c
> +++ b/drivers/acpi/glue.c
> @@ -176,7 +176,6 @@ int acpi_bind_one(struct device *dev, struct acpi_device *acpi_dev)
>  	struct list_head *physnode_list;
>  	unsigned int node_id;
>  	int retval = -EINVAL;
> -	enum dev_dma_attr attr;
>  
>  	if (has_acpi_companion(dev)) {
>  		if (acpi_dev) {
> @@ -233,10 +232,6 @@ int acpi_bind_one(struct device *dev, struct acpi_device *acpi_dev)
>  	if (!has_acpi_companion(dev))
>  		ACPI_COMPANION_SET(dev, acpi_dev);
>  
> -	attr = acpi_get_dma_attr(acpi_dev);
> -	if (attr != DEV_DMA_NOT_SUPPORTED)
> -		acpi_dma_configure(dev, attr);
> -
>  	acpi_physnode_link_name(physical_node_name, node_id);
>  	retval = sysfs_create_link(&acpi_dev->dev.kobj, &dev->kobj,
>  				   physical_node_name);
> diff --git a/drivers/base/dd.c b/drivers/base/dd.c
> index a1fbf55..4882f06 100644
> --- a/drivers/base/dd.c
> +++ b/drivers/base/dd.c
> @@ -19,6 +19,7 @@
>  
>  #include <linux/device.h>
>  #include <linux/delay.h>
> +#include <linux/dma-mapping.h>
>  #include <linux/module.h>
>  #include <linux/kthread.h>
>  #include <linux/wait.h>
> @@ -356,6 +357,10 @@ static int really_probe(struct device *dev, struct device_driver *drv)
>  	if (ret)
>  		goto pinctrl_bind_failed;
>  
> +	ret = dma_configure(dev);
> +	if (ret)
> +		goto dma_failed;
> +
>  	if (driver_sysfs_add(dev)) {
>  		printk(KERN_ERR "%s: driver_sysfs_add(%s) failed\n",
>  			__func__, dev_name(dev));
> @@ -417,6 +422,8 @@ static int really_probe(struct device *dev, struct device_driver *drv)
>  	goto done;
>  
>  probe_failed:
> +	dma_deconfigure(dev);
> +dma_failed:
>  	if (dev->bus)
>  		blocking_notifier_call_chain(&dev->bus->p->bus_notifier,
>  					     BUS_NOTIFY_DRIVER_NOT_BOUND, dev);
> @@ -826,6 +833,8 @@ static void __device_release_driver(struct device *dev, struct device *parent)
>  			drv->remove(dev);
>  
>  		device_links_driver_cleanup(dev);
> +		dma_deconfigure(dev);
> +
>  		devres_release_all(dev);
>  		dev->driver = NULL;
>  		dev_set_drvdata(dev, NULL);
> diff --git a/drivers/base/dma-mapping.c b/drivers/base/dma-mapping.c
> index efd71cf..449b948 100644
> --- a/drivers/base/dma-mapping.c
> +++ b/drivers/base/dma-mapping.c
> @@ -7,9 +7,11 @@
>   * This file is released under the GPLv2.
>   */
>  
> +#include <linux/acpi.h>
>  #include <linux/dma-mapping.h>
>  #include <linux/export.h>
>  #include <linux/gfp.h>
> +#include <linux/of_device.h>
>  #include <linux/slab.h>
>  #include <linux/vmalloc.h>
>  
> @@ -341,3 +343,41 @@ void dma_common_free_remap(void *cpu_addr, size_t size, unsigned long vm_flags)
>  	vunmap(cpu_addr);
>  }
>  #endif
> +
> +/*
> + * Common configuration to enable DMA API use for a device
> + */
> +#include <linux/pci.h>
> +
> +int dma_configure(struct device *dev)
> +{
> +	struct device *bridge = NULL, *dma_dev = dev;
> +	enum dev_dma_attr attr;
> +
> +	if (dev_is_pci(dev)) {
> +		bridge = pci_get_host_bridge_device(to_pci_dev(dev));
> +		dma_dev = bridge;
> +		if (IS_ENABLED(CONFIG_OF) && dma_dev->parent &&
> +		    dma_dev->parent->of_node)
> +			dma_dev = dma_dev->parent;
> +	}
> +
> +	if (dma_dev->of_node) {
> +		of_dma_configure(dev, dma_dev->of_node);
> +	} else if (has_acpi_companion(dma_dev)) {
> +		attr = acpi_get_dma_attr(to_acpi_device_node(dma_dev->fwnode));
> +		if (attr != DEV_DMA_NOT_SUPPORTED)
> +			acpi_dma_configure(dev, attr);
> +	}
> +
> +	if (bridge)
> +		pci_put_host_bridge_device(bridge);
> +
> +	return 0;
> +}
> +
> +void dma_deconfigure(struct device *dev)
> +{
> +	of_dma_deconfigure(dev);
> +	acpi_dma_deconfigure(dev);
> +}
> diff --git a/drivers/of/platform.c b/drivers/of/platform.c
> index 5344db5..2aa4ebb 100644
> --- a/drivers/of/platform.c
> +++ b/drivers/of/platform.c
> @@ -22,6 +22,7 @@
>  #include <linux/slab.h>
>  #include <linux/of_address.h>
>  #include <linux/of_device.h>
> +#include <linux/of_iommu.h>
>  #include <linux/of_irq.h>
>  #include <linux/of_platform.h>
>  #include <linux/platform_device.h>
> @@ -186,11 +187,9 @@ static struct platform_device *of_platform_device_create_pdata(
>  
>  	dev->dev.bus = &platform_bus_type;
>  	dev->dev.platform_data = platform_data;
> -	of_dma_configure(&dev->dev, dev->dev.of_node);
>  	of_msi_configure(&dev->dev, dev->dev.of_node);
>  
>  	if (of_device_add(dev) != 0) {
> -		of_dma_deconfigure(&dev->dev);
>  		platform_device_put(dev);
>  		goto err_clear_flag;
>  	}
> @@ -248,7 +247,6 @@ static struct amba_device *of_amba_device_create(struct device_node *node,
>  		dev_set_name(&dev->dev, "%s", bus_id);
>  	else
>  		of_device_make_bus_id(&dev->dev);
> -	of_dma_configure(&dev->dev, dev->dev.of_node);
>  
>  	/* Allow the HW Peripheral ID to be overridden */
>  	prop = of_get_property(node, "arm,primecell-periphid", NULL);
> @@ -542,7 +540,6 @@ static int of_platform_device_destroy(struct device *dev, void *data)
>  		amba_device_unregister(to_amba_device(dev));
>  #endif
>  
> -	of_dma_deconfigure(dev);
>  	of_node_clear_flag(dev->of_node, OF_POPULATED);
>  	of_node_clear_flag(dev->of_node, OF_POPULATED_BUS);
>  	return 0;
> diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
> index dfc9a27..5a8dd43 100644
> --- a/drivers/pci/probe.c
> +++ b/drivers/pci/probe.c
> @@ -1893,33 +1893,6 @@ static void pci_set_msi_domain(struct pci_dev *dev)
>  	dev_set_msi_domain(&dev->dev, d);
>  }
>  
> -/**
> - * pci_dma_configure - Setup DMA configuration
> - * @dev: ptr to pci_dev struct of the PCI device
> - *
> - * Function to update PCI devices's DMA configuration using the same
> - * info from the OF node or ACPI node of host bridge's parent (if any).
> - */
> -static void pci_dma_configure(struct pci_dev *dev)
> -{
> -	struct device *bridge = pci_get_host_bridge_device(dev);
> -
> -	if (IS_ENABLED(CONFIG_OF) &&
> -		bridge->parent && bridge->parent->of_node) {
> -			of_dma_configure(&dev->dev, bridge->parent->of_node);
> -	} else if (has_acpi_companion(bridge)) {
> -		struct acpi_device *adev = to_acpi_device_node(bridge->fwnode);
> -		enum dev_dma_attr attr = acpi_get_dma_attr(adev);
> -
> -		if (attr == DEV_DMA_NOT_SUPPORTED)
> -			dev_warn(&dev->dev, "DMA not supported.\n");
> -		else
> -			acpi_dma_configure(&dev->dev, attr);
> -	}
> -
> -	pci_put_host_bridge_device(bridge);
> -}
> -
>  void pci_device_add(struct pci_dev *dev, struct pci_bus *bus)
>  {
>  	int ret;
> @@ -1933,7 +1906,6 @@ void pci_device_add(struct pci_dev *dev, struct pci_bus *bus)
>  	dev->dev.dma_mask = &dev->dma_mask;
>  	dev->dev.dma_parms = &dev->dma_parms;
>  	dev->dev.coherent_dma_mask = 0xffffffffull;
> -	pci_dma_configure(dev);
>  
>  	pci_set_dma_max_seg_size(dev, 65536);
>  	pci_set_dma_seg_boundary(dev, 0xffffffff);
> diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
> index 0977317..4f3eece 100644
> --- a/include/linux/dma-mapping.h
> +++ b/include/linux/dma-mapping.h
> @@ -728,6 +728,18 @@ void *dma_mark_declared_memory_occupied(struct device *dev,
>  }
>  #endif /* CONFIG_HAVE_GENERIC_DMA_COHERENT */
>  
> +#ifdef CONFIG_HAS_DMA
> +int dma_configure(struct device *dev);
> +void dma_deconfigure(struct device *dev);
> +#else
> +static inline int dma_configure(struct device *dev)
> +{
> +	return 0;
> +}
> +
> +static inline void dma_deconfigure(struct device *dev) {}
> +#endif
> +
>  /*
>   * Managed DMA API
>   */
> 

Re: [PATCH V10 07/12] of/acpi: Configure dma operations at probe time for platform/amba/pci bus devices

[Sricharan R]

Hi Robin,

On 4/4/2017 5:47 PM, Robin Murphy wrote:
> On 04/04/17 11:18, Sricharan R wrote:
>> Configuring DMA ops at probe time will allow deferring device probe when
>> the IOMMU isn't available yet. The dma_configure for the device is
>> now called from the generic device_attach callback just before the
>> bus/driver probe is called. This way, configuring the DMA ops for the
>> device would be called at the same place for all bus_types, hence the
>> deferred probing mechanism should work for all buses as well.
>>
>> pci_bus_add_devices    (platform/amba)(_device_create/driver_register)
>>        |                         |
>> pci_bus_add_device     (device_add/driver_register)
>>        |                         |
>> device_attach           device_initial_probe
>>        |                         |
>> __device_attach_driver    __device_attach_driver
>>        |
>> driver_probe_device
>>        |
>> really_probe
>>        |
>> dma_configure
>>
>> Similarly on the device/driver_unregister path __device_release_driver is
>> called which inturn calls dma_deconfigure.
>>
>> This patch changes the dma ops configuration to probe time for
>> both OF and ACPI based platform/amba/pci bus devices.
>
> Reviewed-by: Robin Murphy <robin.murphy@arm.com>
>
> It's possible we could subsume {of,acpi}_dma_deconfigure() into
> dma_deconfigure() entirely in future if it becomes clear that neither of
> them will ever need to do anything firmware-specific, but I think for
> now it's probably safer to keep the current symmetry - calling
> arch_teardown_dma_ops() twice is benign (and even if it weren't, I'd say
> it really should be!)
>

Ya, calling it twice should cause no harm. If nothing firmware specific
gets added this could simply be put in deconfigure itself in future.
Thanks for all the reviews !!

Regards,
  Sricharan



> Robin.
>
>> Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
>> Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
>> Acked-by: Bjorn Helgaas <bhelgaas@google.com> (drivers/pci part)
>> Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
>> Signed-off-by: Sricharan R <sricharan@codeaurora.org>
>> ---
>>
>>  [V10] Added dummy dma_(de)configure functions in case
>>        of !CONFIG_HAS_DMA to avoid build breaks.
>>
>>  drivers/acpi/glue.c         |  5 -----
>>  drivers/base/dd.c           |  9 +++++++++
>>  drivers/base/dma-mapping.c  | 40 ++++++++++++++++++++++++++++++++++++++++
>>  drivers/of/platform.c       |  5 +----
>>  drivers/pci/probe.c         | 28 ----------------------------
>>  include/linux/dma-mapping.h | 12 ++++++++++++
>>  6 files changed, 62 insertions(+), 37 deletions(-)
>>
>> diff --git a/drivers/acpi/glue.c b/drivers/acpi/glue.c
>> index fb19e1c..c05f241 100644
>> --- a/drivers/acpi/glue.c
>> +++ b/drivers/acpi/glue.c
>> @@ -176,7 +176,6 @@ int acpi_bind_one(struct device *dev, struct acpi_device *acpi_dev)
>>  	struct list_head *physnode_list;
>>  	unsigned int node_id;
>>  	int retval = -EINVAL;
>> -	enum dev_dma_attr attr;
>>
>>  	if (has_acpi_companion(dev)) {
>>  		if (acpi_dev) {
>> @@ -233,10 +232,6 @@ int acpi_bind_one(struct device *dev, struct acpi_device *acpi_dev)
>>  	if (!has_acpi_companion(dev))
>>  		ACPI_COMPANION_SET(dev, acpi_dev);
>>
>> -	attr = acpi_get_dma_attr(acpi_dev);
>> -	if (attr != DEV_DMA_NOT_SUPPORTED)
>> -		acpi_dma_configure(dev, attr);
>> -
>>  	acpi_physnode_link_name(physical_node_name, node_id);
>>  	retval = sysfs_create_link(&acpi_dev->dev.kobj, &dev->kobj,
>>  				   physical_node_name);
>> diff --git a/drivers/base/dd.c b/drivers/base/dd.c
>> index a1fbf55..4882f06 100644
>> --- a/drivers/base/dd.c
>> +++ b/drivers/base/dd.c
>> @@ -19,6 +19,7 @@
>>
>>  #include <linux/device.h>
>>  #include <linux/delay.h>
>> +#include <linux/dma-mapping.h>
>>  #include <linux/module.h>
>>  #include <linux/kthread.h>
>>  #include <linux/wait.h>
>> @@ -356,6 +357,10 @@ static int really_probe(struct device *dev, struct device_driver *drv)
>>  	if (ret)
>>  		goto pinctrl_bind_failed;
>>
>> +	ret = dma_configure(dev);
>> +	if (ret)
>> +		goto dma_failed;
>> +
>>  	if (driver_sysfs_add(dev)) {
>>  		printk(KERN_ERR "%s: driver_sysfs_add(%s) failed\n",
>>  			__func__, dev_name(dev));
>> @@ -417,6 +422,8 @@ static int really_probe(struct device *dev, struct device_driver *drv)
>>  	goto done;
>>
>>  probe_failed:
>> +	dma_deconfigure(dev);
>> +dma_failed:
>>  	if (dev->bus)
>>  		blocking_notifier_call_chain(&dev->bus->p->bus_notifier,
>>  					     BUS_NOTIFY_DRIVER_NOT_BOUND, dev);
>> @@ -826,6 +833,8 @@ static void __device_release_driver(struct device *dev, struct device *parent)
>>  			drv->remove(dev);
>>
>>  		device_links_driver_cleanup(dev);
>> +		dma_deconfigure(dev);
>> +
>>  		devres_release_all(dev);
>>  		dev->driver = NULL;
>>  		dev_set_drvdata(dev, NULL);
>> diff --git a/drivers/base/dma-mapping.c b/drivers/base/dma-mapping.c
>> index efd71cf..449b948 100644
>> --- a/drivers/base/dma-mapping.c
>> +++ b/drivers/base/dma-mapping.c
>> @@ -7,9 +7,11 @@
>>   * This file is released under the GPLv2.
>>   */
>>
>> +#include <linux/acpi.h>
>>  #include <linux/dma-mapping.h>
>>  #include <linux/export.h>
>>  #include <linux/gfp.h>
>> +#include <linux/of_device.h>
>>  #include <linux/slab.h>
>>  #include <linux/vmalloc.h>
>>
>> @@ -341,3 +343,41 @@ void dma_common_free_remap(void *cpu_addr, size_t size, unsigned long vm_flags)
>>  	vunmap(cpu_addr);
>>  }
>>  #endif
>> +
>> +/*
>> + * Common configuration to enable DMA API use for a device
>> + */
>> +#include <linux/pci.h>
>> +
>> +int dma_configure(struct device *dev)
>> +{
>> +	struct device *bridge = NULL, *dma_dev = dev;
>> +	enum dev_dma_attr attr;
>> +
>> +	if (dev_is_pci(dev)) {
>> +		bridge = pci_get_host_bridge_device(to_pci_dev(dev));
>> +		dma_dev = bridge;
>> +		if (IS_ENABLED(CONFIG_OF) && dma_dev->parent &&
>> +		    dma_dev->parent->of_node)
>> +			dma_dev = dma_dev->parent;
>> +	}
>> +
>> +	if (dma_dev->of_node) {
>> +		of_dma_configure(dev, dma_dev->of_node);
>> +	} else if (has_acpi_companion(dma_dev)) {
>> +		attr = acpi_get_dma_attr(to_acpi_device_node(dma_dev->fwnode));
>> +		if (attr != DEV_DMA_NOT_SUPPORTED)
>> +			acpi_dma_configure(dev, attr);
>> +	}
>> +
>> +	if (bridge)
>> +		pci_put_host_bridge_device(bridge);
>> +
>> +	return 0;
>> +}
>> +
>> +void dma_deconfigure(struct device *dev)
>> +{
>> +	of_dma_deconfigure(dev);
>> +	acpi_dma_deconfigure(dev);
>> +}
>> diff --git a/drivers/of/platform.c b/drivers/of/platform.c
>> index 5344db5..2aa4ebb 100644
>> --- a/drivers/of/platform.c
>> +++ b/drivers/of/platform.c
>> @@ -22,6 +22,7 @@
>>  #include <linux/slab.h>
>>  #include <linux/of_address.h>
>>  #include <linux/of_device.h>
>> +#include <linux/of_iommu.h>
>>  #include <linux/of_irq.h>
>>  #include <linux/of_platform.h>
>>  #include <linux/platform_device.h>
>> @@ -186,11 +187,9 @@ static struct platform_device *of_platform_device_create_pdata(
>>
>>  	dev->dev.bus = &platform_bus_type;
>>  	dev->dev.platform_data = platform_data;
>> -	of_dma_configure(&dev->dev, dev->dev.of_node);
>>  	of_msi_configure(&dev->dev, dev->dev.of_node);
>>
>>  	if (of_device_add(dev) != 0) {
>> -		of_dma_deconfigure(&dev->dev);
>>  		platform_device_put(dev);
>>  		goto err_clear_flag;
>>  	}
>> @@ -248,7 +247,6 @@ static struct amba_device *of_amba_device_create(struct device_node *node,
>>  		dev_set_name(&dev->dev, "%s", bus_id);
>>  	else
>>  		of_device_make_bus_id(&dev->dev);
>> -	of_dma_configure(&dev->dev, dev->dev.of_node);
>>
>>  	/* Allow the HW Peripheral ID to be overridden */
>>  	prop = of_get_property(node, "arm,primecell-periphid", NULL);
>> @@ -542,7 +540,6 @@ static int of_platform_device_destroy(struct device *dev, void *data)
>>  		amba_device_unregister(to_amba_device(dev));
>>  #endif
>>
>> -	of_dma_deconfigure(dev);
>>  	of_node_clear_flag(dev->of_node, OF_POPULATED);
>>  	of_node_clear_flag(dev->of_node, OF_POPULATED_BUS);
>>  	return 0;
>> diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
>> index dfc9a27..5a8dd43 100644
>> --- a/drivers/pci/probe.c
>> +++ b/drivers/pci/probe.c
>> @@ -1893,33 +1893,6 @@ static void pci_set_msi_domain(struct pci_dev *dev)
>>  	dev_set_msi_domain(&dev->dev, d);
>>  }
>>
>> -/**
>> - * pci_dma_configure - Setup DMA configuration
>> - * @dev: ptr to pci_dev struct of the PCI device
>> - *
>> - * Function to update PCI devices's DMA configuration using the same
>> - * info from the OF node or ACPI node of host bridge's parent (if any).
>> - */
>> -static void pci_dma_configure(struct pci_dev *dev)
>> -{
>> -	struct device *bridge = pci_get_host_bridge_device(dev);
>> -
>> -	if (IS_ENABLED(CONFIG_OF) &&
>> -		bridge->parent && bridge->parent->of_node) {
>> -			of_dma_configure(&dev->dev, bridge->parent->of_node);
>> -	} else if (has_acpi_companion(bridge)) {
>> -		struct acpi_device *adev = to_acpi_device_node(bridge->fwnode);
>> -		enum dev_dma_attr attr = acpi_get_dma_attr(adev);
>> -
>> -		if (attr == DEV_DMA_NOT_SUPPORTED)
>> -			dev_warn(&dev->dev, "DMA not supported.\n");
>> -		else
>> -			acpi_dma_configure(&dev->dev, attr);
>> -	}
>> -
>> -	pci_put_host_bridge_device(bridge);
>> -}
>> -
>>  void pci_device_add(struct pci_dev *dev, struct pci_bus *bus)
>>  {
>>  	int ret;
>> @@ -1933,7 +1906,6 @@ void pci_device_add(struct pci_dev *dev, struct pci_bus *bus)
>>  	dev->dev.dma_mask = &dev->dma_mask;
>>  	dev->dev.dma_parms = &dev->dma_parms;
>>  	dev->dev.coherent_dma_mask = 0xffffffffull;
>> -	pci_dma_configure(dev);
>>
>>  	pci_set_dma_max_seg_size(dev, 65536);
>>  	pci_set_dma_seg_boundary(dev, 0xffffffff);
>> diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
>> index 0977317..4f3eece 100644
>> --- a/include/linux/dma-mapping.h
>> +++ b/include/linux/dma-mapping.h
>> @@ -728,6 +728,18 @@ void *dma_mark_declared_memory_occupied(struct device *dev,
>>  }
>>  #endif /* CONFIG_HAVE_GENERIC_DMA_COHERENT */
>>
>> +#ifdef CONFIG_HAS_DMA
>> +int dma_configure(struct device *dev);
>> +void dma_deconfigure(struct device *dev);
>> +#else
>> +static inline int dma_configure(struct device *dev)
>> +{
>> +	return 0;
>> +}
>> +
>> +static inline void dma_deconfigure(struct device *dev) {}
>> +#endif
>> +
>>  /*
>>   * Managed DMA API
>>   */
>>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

-- 
"QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[PATCH V10 08/12] iommu: of: Handle IOMMU lookup failure with deferred probing or error

[Sricharan R]

From: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>

Failures to look up an IOMMU when parsing the DT iommus property need to
be handled separately from the .of_xlate() failures to support deferred
probing.

The lack of a registered IOMMU can be caused by the lack of a driver for
the IOMMU, the IOMMU device probe not having been performed yet, having
been deferred, or having failed.

The first case occurs when the device tree describes the bus master and
IOMMU topology correctly but no device driver exists for the IOMMU yet
or the device driver has not been compiled in. Return NULL, the caller
will configure the device without an IOMMU.

The second and third cases are handled by deferring the probe of the bus
master device which will eventually get reprobed after the IOMMU.

The last case is currently handled by deferring the probe of the bus
master device as well. A mechanism to either configure the bus master
device without an IOMMU or to fail the bus master device probe depending
on whether the IOMMU is optional or mandatory would be a good
enhancement.

Tested-by: Marek Szyprowski <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
Signed-off-by: Laurent Pichart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>
Signed-off-by: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
---
 drivers/base/dma-mapping.c | 5 +++--
 drivers/iommu/of_iommu.c   | 4 ++--
 drivers/of/device.c        | 7 ++++++-
 include/linux/of_device.h  | 9 ++++++---
 4 files changed, 17 insertions(+), 8 deletions(-)

diff --git a/drivers/base/dma-mapping.c b/drivers/base/dma-mapping.c
index 449b948..82bd45c 100644
--- a/drivers/base/dma-mapping.c
+++ b/drivers/base/dma-mapping.c
@@ -353,6 +353,7 @@ int dma_configure(struct device *dev)
 {
 	struct device *bridge = NULL, *dma_dev = dev;
 	enum dev_dma_attr attr;
+	int ret = 0;
 
 	if (dev_is_pci(dev)) {
 		bridge = pci_get_host_bridge_device(to_pci_dev(dev));
@@ -363,7 +364,7 @@ int dma_configure(struct device *dev)
 	}
 
 	if (dma_dev->of_node) {
-		of_dma_configure(dev, dma_dev->of_node);
+		ret = of_dma_configure(dev, dma_dev->of_node);
 	} else if (has_acpi_companion(dma_dev)) {
 		attr = acpi_get_dma_attr(to_acpi_device_node(dma_dev->fwnode));
 		if (attr != DEV_DMA_NOT_SUPPORTED)
@@ -373,7 +374,7 @@ int dma_configure(struct device *dev)
 	if (bridge)
 		pci_put_host_bridge_device(bridge);
 
-	return 0;
+	return ret;
 }
 
 void dma_deconfigure(struct device *dev)
diff --git a/drivers/iommu/of_iommu.c b/drivers/iommu/of_iommu.c
index c8be889..9f44ee8 100644
--- a/drivers/iommu/of_iommu.c
+++ b/drivers/iommu/of_iommu.c
@@ -236,7 +236,7 @@ const struct iommu_ops *of_iommu_configure(struct device *dev,
 			ops = ERR_PTR(err);
 	}
 
-	return IS_ERR(ops) ? NULL : ops;
+	return ops;
 }
 
 static int __init of_iommu_init(void)
@@ -247,7 +247,7 @@ static int __init of_iommu_init(void)
 	for_each_matching_node_and_match(np, matches, &match) {
 		const of_iommu_init_fn init_fn = match->data;
 
-		if (init_fn(np))
+		if (init_fn && init_fn(np))
 			pr_err("Failed to initialise IOMMU %s\n",
 				of_node_full_name(np));
 	}
diff --git a/drivers/of/device.c b/drivers/of/device.c
index c2ae6bb..bea8aec 100644
--- a/drivers/of/device.c
+++ b/drivers/of/device.c
@@ -82,7 +82,7 @@ int of_device_add(struct platform_device *ofdev)
  * can use a platform bus notifier and handle BUS_NOTIFY_ADD_DEVICE events
  * to fix up DMA configuration.
  */
-void of_dma_configure(struct device *dev, struct device_node *np)
+int of_dma_configure(struct device *dev, struct device_node *np)
 {
 	u64 dma_addr, paddr, size;
 	int ret;
@@ -129,10 +129,15 @@ void of_dma_configure(struct device *dev, struct device_node *np)
 		coherent ? " " : " not ");
 
 	iommu = of_iommu_configure(dev, np);
+	if (IS_ERR(iommu))
+		return PTR_ERR(iommu);
+
 	dev_dbg(dev, "device is%sbehind an iommu\n",
 		iommu ? " " : " not ");
 
 	arch_setup_dma_ops(dev, dma_addr, size, iommu, coherent);
+
+	return 0;
 }
 EXPORT_SYMBOL_GPL(of_dma_configure);
 
diff --git a/include/linux/of_device.h b/include/linux/of_device.h
index af98455..2cacdd8 100644
--- a/include/linux/of_device.h
+++ b/include/linux/of_device.h
@@ -55,7 +55,7 @@ static inline struct device_node *of_cpu_device_node_get(int cpu)
 	return of_node_get(cpu_dev->of_node);
 }
 
-void of_dma_configure(struct device *dev, struct device_node *np);
+int of_dma_configure(struct device *dev, struct device_node *np);
 void of_dma_deconfigure(struct device *dev);
 #else /* CONFIG_OF */
 
@@ -104,8 +104,11 @@ static inline struct device_node *of_cpu_device_node_get(int cpu)
 {
 	return NULL;
 }
-static inline void of_dma_configure(struct device *dev, struct device_node *np)
-{}
+
+static inline int of_dma_configure(struct device *dev, struct device_node *np)
+{
+	return 0;
+}
 static inline void of_dma_deconfigure(struct device *dev)
 {}
 #endif /* CONFIG_OF */
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

Re: [PATCH V10 08/12] iommu: of: Handle IOMMU lookup failure with deferred probing or error

[Robin Murphy]

On 04/04/17 11:18, Sricharan R wrote:
> From: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
> 
> Failures to look up an IOMMU when parsing the DT iommus property need to
> be handled separately from the .of_xlate() failures to support deferred
> probing.
> 
> The lack of a registered IOMMU can be caused by the lack of a driver for
> the IOMMU, the IOMMU device probe not having been performed yet, having
> been deferred, or having failed.
> 
> The first case occurs when the device tree describes the bus master and
> IOMMU topology correctly but no device driver exists for the IOMMU yet
> or the device driver has not been compiled in. Return NULL, the caller
> will configure the device without an IOMMU.
> 
> The second and third cases are handled by deferring the probe of the bus
> master device which will eventually get reprobed after the IOMMU.
> 
> The last case is currently handled by deferring the probe of the bus
> master device as well. A mechanism to either configure the bus master
> device without an IOMMU or to fail the bus master device probe depending
> on whether the IOMMU is optional or mandatory would be a good
> enhancement.

Reviewed-by: Robin Murphy <robin.murphy@arm.com>

> Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
> Signed-off-by: Laurent Pichart <laurent.pinchart+renesas@ideasonboard.com>
> Signed-off-by: Sricharan R <sricharan@codeaurora.org>
> ---
>  drivers/base/dma-mapping.c | 5 +++--
>  drivers/iommu/of_iommu.c   | 4 ++--
>  drivers/of/device.c        | 7 ++++++-
>  include/linux/of_device.h  | 9 ++++++---
>  4 files changed, 17 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/base/dma-mapping.c b/drivers/base/dma-mapping.c
> index 449b948..82bd45c 100644
> --- a/drivers/base/dma-mapping.c
> +++ b/drivers/base/dma-mapping.c
> @@ -353,6 +353,7 @@ int dma_configure(struct device *dev)
>  {
>  	struct device *bridge = NULL, *dma_dev = dev;
>  	enum dev_dma_attr attr;
> +	int ret = 0;
>  
>  	if (dev_is_pci(dev)) {
>  		bridge = pci_get_host_bridge_device(to_pci_dev(dev));
> @@ -363,7 +364,7 @@ int dma_configure(struct device *dev)
>  	}
>  
>  	if (dma_dev->of_node) {
> -		of_dma_configure(dev, dma_dev->of_node);
> +		ret = of_dma_configure(dev, dma_dev->of_node);
>  	} else if (has_acpi_companion(dma_dev)) {
>  		attr = acpi_get_dma_attr(to_acpi_device_node(dma_dev->fwnode));
>  		if (attr != DEV_DMA_NOT_SUPPORTED)
> @@ -373,7 +374,7 @@ int dma_configure(struct device *dev)
>  	if (bridge)
>  		pci_put_host_bridge_device(bridge);
>  
> -	return 0;
> +	return ret;
>  }
>  
>  void dma_deconfigure(struct device *dev)
> diff --git a/drivers/iommu/of_iommu.c b/drivers/iommu/of_iommu.c
> index c8be889..9f44ee8 100644
> --- a/drivers/iommu/of_iommu.c
> +++ b/drivers/iommu/of_iommu.c
> @@ -236,7 +236,7 @@ const struct iommu_ops *of_iommu_configure(struct device *dev,
>  			ops = ERR_PTR(err);
>  	}
>  
> -	return IS_ERR(ops) ? NULL : ops;
> +	return ops;
>  }
>  
>  static int __init of_iommu_init(void)
> @@ -247,7 +247,7 @@ static int __init of_iommu_init(void)
>  	for_each_matching_node_and_match(np, matches, &match) {
>  		const of_iommu_init_fn init_fn = match->data;
>  
> -		if (init_fn(np))
> +		if (init_fn && init_fn(np))
>  			pr_err("Failed to initialise IOMMU %s\n",
>  				of_node_full_name(np));
>  	}
> diff --git a/drivers/of/device.c b/drivers/of/device.c
> index c2ae6bb..bea8aec 100644
> --- a/drivers/of/device.c
> +++ b/drivers/of/device.c
> @@ -82,7 +82,7 @@ int of_device_add(struct platform_device *ofdev)
>   * can use a platform bus notifier and handle BUS_NOTIFY_ADD_DEVICE events
>   * to fix up DMA configuration.
>   */
> -void of_dma_configure(struct device *dev, struct device_node *np)
> +int of_dma_configure(struct device *dev, struct device_node *np)
>  {
>  	u64 dma_addr, paddr, size;
>  	int ret;
> @@ -129,10 +129,15 @@ void of_dma_configure(struct device *dev, struct device_node *np)
>  		coherent ? " " : " not ");
>  
>  	iommu = of_iommu_configure(dev, np);
> +	if (IS_ERR(iommu))
> +		return PTR_ERR(iommu);
> +
>  	dev_dbg(dev, "device is%sbehind an iommu\n",
>  		iommu ? " " : " not ");
>  
>  	arch_setup_dma_ops(dev, dma_addr, size, iommu, coherent);
> +
> +	return 0;
>  }
>  EXPORT_SYMBOL_GPL(of_dma_configure);
>  
> diff --git a/include/linux/of_device.h b/include/linux/of_device.h
> index af98455..2cacdd8 100644
> --- a/include/linux/of_device.h
> +++ b/include/linux/of_device.h
> @@ -55,7 +55,7 @@ static inline struct device_node *of_cpu_device_node_get(int cpu)
>  	return of_node_get(cpu_dev->of_node);
>  }
>  
> -void of_dma_configure(struct device *dev, struct device_node *np);
> +int of_dma_configure(struct device *dev, struct device_node *np);
>  void of_dma_deconfigure(struct device *dev);
>  #else /* CONFIG_OF */
>  
> @@ -104,8 +104,11 @@ static inline struct device_node *of_cpu_device_node_get(int cpu)
>  {
>  	return NULL;
>  }
> -static inline void of_dma_configure(struct device *dev, struct device_node *np)
> -{}
> +
> +static inline int of_dma_configure(struct device *dev, struct device_node *np)
> +{
> +	return 0;
> +}
>  static inline void of_dma_deconfigure(struct device *dev)
>  {}
>  #endif /* CONFIG_OF */
> 

[PATCH V10 09/12] drivers: acpi: Handle IOMMU lookup failure with deferred probing or error

[Sricharan R]

This is an equivalent to the DT's handling of the iommu master's probe
with deferred probing when the corrsponding iommu is not probed yet.
The lack of a registered IOMMU can be caused by the lack of a driver for
the IOMMU, the IOMMU device probe not having been performed yet, having
been deferred, or having failed.

The first case occurs when the firmware describes the bus master and
IOMMU topology correctly but no device driver exists for the IOMMU yet
or the device driver has not been compiled in. Return NULL, the caller
will configure the device without an IOMMU.

The second and third cases are handled by deferring the probe of the bus
master device which will eventually get reprobed after the IOMMU.

The last case is currently handled by deferring the probe of the bus
master device as well. A mechanism to either configure the bus master
device without an IOMMU or to fail the bus master device probe depending
on whether the IOMMU is optional or mandatory would be a good
enhancement.

Tested-by: Hanjun Guo <hanjun.guo-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
[Lorenzo: Added fixes for dma_coherent_mask overflow, acpi_dma_configure
          called multiple times for same device]
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi-5wv7dgnIgG8@public.gmane.org>
Signed-off-by: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
---

 [V10] Added Lorenzo's ACPI fix for coherent_dma_mask overflow
       and the fix for dma_configure getting called more than
       once for the same device.

 drivers/acpi/arm64/iort.c  | 33 ++++++++++++++++++++++++++++++++-
 drivers/acpi/scan.c        | 11 ++++++++---
 drivers/base/dma-mapping.c |  2 +-
 include/acpi/acpi_bus.h    |  2 +-
 include/linux/acpi.h       |  7 +++++--
 5 files changed, 47 insertions(+), 8 deletions(-)

diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
index 3dd9ec3..e323ece 100644
--- a/drivers/acpi/arm64/iort.c
+++ b/drivers/acpi/arm64/iort.c
@@ -543,6 +543,14 @@ static const struct iommu_ops *iort_iommu_xlate(struct device *dev,
 	const struct iommu_ops *ops = NULL;
 	int ret = -ENODEV;
 	struct fwnode_handle *iort_fwnode;
+	struct iommu_fwspec *fwspec = dev->iommu_fwspec;
+
+	/*
+	 * If we already translated the fwspec there
+	 * is nothing left to do, return the iommu_ops.
+	 */
+	if (fwspec && fwspec->ops)
+		return fwspec->ops;
 
 	if (node) {
 		iort_fwnode = iort_get_fwnode(node);
@@ -550,8 +558,17 @@ static const struct iommu_ops *iort_iommu_xlate(struct device *dev,
 			return NULL;
 
 		ops = iommu_ops_from_fwnode(iort_fwnode);
+		/*
+		 * If the ops look-up fails, this means that either
+		 * the SMMU drivers have not been probed yet or that
+		 * the SMMU drivers are not built in the kernel;
+		 * Depending on whether the SMMU drivers are built-in
+		 * in the kernel or not, defer the IOMMU configuration
+		 * or just abort it.
+		 */
 		if (!ops)
-			return NULL;
+			return iort_iommu_driver_enabled(node->type) ?
+			       ERR_PTR(-EPROBE_DEFER) : NULL;
 
 		ret = arm_smmu_iort_xlate(dev, streamid, iort_fwnode, ops);
 	}
@@ -625,12 +642,26 @@ const struct iommu_ops *iort_iommu_configure(struct device *dev)
 
 		while (parent) {
 			ops = iort_iommu_xlate(dev, parent, streamid);
+			if (IS_ERR_OR_NULL(ops))
+				return ops;
 
 			parent = iort_node_get_id(node, &streamid,
 						  IORT_IOMMU_TYPE, i++);
 		}
 	}
 
+	/*
+	 * If we have reason to believe the IOMMU driver missed the initial
+	 * add_device callback for dev, replay it to get things in order.
+	 */
+	if (!IS_ERR_OR_NULL(ops) && ops->add_device &&
+	    dev->bus && !dev->iommu_group) {
+		int err = ops->add_device(dev);
+
+		if (err)
+			ops = ERR_PTR(err);
+	}
+
 	return ops;
 }
 
diff --git a/drivers/acpi/scan.c b/drivers/acpi/scan.c
index 1926918..2a513cc 100644
--- a/drivers/acpi/scan.c
+++ b/drivers/acpi/scan.c
@@ -1373,20 +1373,25 @@ enum dev_dma_attr acpi_get_dma_attr(struct acpi_device *adev)
  * @dev: The pointer to the device
  * @attr: device dma attributes
  */
-void acpi_dma_configure(struct device *dev, enum dev_dma_attr attr)
+int acpi_dma_configure(struct device *dev, enum dev_dma_attr attr)
 {
 	const struct iommu_ops *iommu;
+	u64 size;
 
 	iort_set_dma_mask(dev);
 
 	iommu = iort_iommu_configure(dev);
+	if (IS_ERR(iommu))
+		return PTR_ERR(iommu);
 
+	size = max(dev->coherent_dma_mask, dev->coherent_dma_mask + 1);
 	/*
 	 * Assume dma valid range starts at 0 and covers the whole
 	 * coherent_dma_mask.
 	 */
-	arch_setup_dma_ops(dev, 0, dev->coherent_dma_mask + 1, iommu,
-			   attr == DEV_DMA_COHERENT);
+	arch_setup_dma_ops(dev, 0, size, iommu, attr == DEV_DMA_COHERENT);
+
+	return 0;
 }
 EXPORT_SYMBOL_GPL(acpi_dma_configure);
 
diff --git a/drivers/base/dma-mapping.c b/drivers/base/dma-mapping.c
index 82bd45c..755a2b5 100644
--- a/drivers/base/dma-mapping.c
+++ b/drivers/base/dma-mapping.c
@@ -368,7 +368,7 @@ int dma_configure(struct device *dev)
 	} else if (has_acpi_companion(dma_dev)) {
 		attr = acpi_get_dma_attr(to_acpi_device_node(dma_dev->fwnode));
 		if (attr != DEV_DMA_NOT_SUPPORTED)
-			acpi_dma_configure(dev, attr);
+			ret = acpi_dma_configure(dev, attr);
 	}
 
 	if (bridge)
diff --git a/include/acpi/acpi_bus.h b/include/acpi/acpi_bus.h
index ef0ae8a..2a9a5de 100644
--- a/include/acpi/acpi_bus.h
+++ b/include/acpi/acpi_bus.h
@@ -575,7 +575,7 @@ struct acpi_pci_root {
 
 bool acpi_dma_supported(struct acpi_device *adev);
 enum dev_dma_attr acpi_get_dma_attr(struct acpi_device *adev);
-void acpi_dma_configure(struct device *dev, enum dev_dma_attr attr);
+int acpi_dma_configure(struct device *dev, enum dev_dma_attr attr);
 void acpi_dma_deconfigure(struct device *dev);
 
 struct acpi_device *acpi_find_child_device(struct acpi_device *parent,
diff --git a/include/linux/acpi.h b/include/linux/acpi.h
index 9b05886..79d06ef6 100644
--- a/include/linux/acpi.h
+++ b/include/linux/acpi.h
@@ -762,8 +762,11 @@ static inline enum dev_dma_attr acpi_get_dma_attr(struct acpi_device *adev)
 	return DEV_DMA_NOT_SUPPORTED;
 }
 
-static inline void acpi_dma_configure(struct device *dev,
-				      enum dev_dma_attr attr) { }
+static inline int acpi_dma_configure(struct device *dev,
+				     enum dev_dma_attr attr)
+{
+	return 0;
+}
 
 static inline void acpi_dma_deconfigure(struct device *dev) { }
 
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

Re: [PATCH V10 09/12] drivers: acpi: Handle IOMMU lookup failure with deferred probing or error

[Robin Murphy]

On 04/04/17 11:18, Sricharan R wrote:
> This is an equivalent to the DT's handling of the iommu master's probe
> with deferred probing when the corrsponding iommu is not probed yet.
> The lack of a registered IOMMU can be caused by the lack of a driver for
> the IOMMU, the IOMMU device probe not having been performed yet, having
> been deferred, or having failed.
> 
> The first case occurs when the firmware describes the bus master and
> IOMMU topology correctly but no device driver exists for the IOMMU yet
> or the device driver has not been compiled in. Return NULL, the caller
> will configure the device without an IOMMU.
> 
> The second and third cases are handled by deferring the probe of the bus
> master device which will eventually get reprobed after the IOMMU.
> 
> The last case is currently handled by deferring the probe of the bus
> master device as well. A mechanism to either configure the bus master
> device without an IOMMU or to fail the bus master device probe depending
> on whether the IOMMU is optional or mandatory would be a good
> enhancement.

I think we really have caught everything now...

Reviewed-by: Robin Murphy <robin.murphy@arm.com>

> Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
> [Lorenzo: Added fixes for dma_coherent_mask overflow, acpi_dma_configure
>           called multiple times for same device]
> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> Signed-off-by: Sricharan R <sricharan@codeaurora.org>
> ---
> 
>  [V10] Added Lorenzo's ACPI fix for coherent_dma_mask overflow
>        and the fix for dma_configure getting called more than
>        once for the same device.
> 
>  drivers/acpi/arm64/iort.c  | 33 ++++++++++++++++++++++++++++++++-
>  drivers/acpi/scan.c        | 11 ++++++++---
>  drivers/base/dma-mapping.c |  2 +-
>  include/acpi/acpi_bus.h    |  2 +-
>  include/linux/acpi.h       |  7 +++++--
>  5 files changed, 47 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> index 3dd9ec3..e323ece 100644
> --- a/drivers/acpi/arm64/iort.c
> +++ b/drivers/acpi/arm64/iort.c
> @@ -543,6 +543,14 @@ static const struct iommu_ops *iort_iommu_xlate(struct device *dev,
>  	const struct iommu_ops *ops = NULL;
>  	int ret = -ENODEV;
>  	struct fwnode_handle *iort_fwnode;
> +	struct iommu_fwspec *fwspec = dev->iommu_fwspec;
> +
> +	/*
> +	 * If we already translated the fwspec there
> +	 * is nothing left to do, return the iommu_ops.
> +	 */
> +	if (fwspec && fwspec->ops)
> +		return fwspec->ops;
>  
>  	if (node) {
>  		iort_fwnode = iort_get_fwnode(node);
> @@ -550,8 +558,17 @@ static const struct iommu_ops *iort_iommu_xlate(struct device *dev,
>  			return NULL;
>  
>  		ops = iommu_ops_from_fwnode(iort_fwnode);
> +		/*
> +		 * If the ops look-up fails, this means that either
> +		 * the SMMU drivers have not been probed yet or that
> +		 * the SMMU drivers are not built in the kernel;
> +		 * Depending on whether the SMMU drivers are built-in
> +		 * in the kernel or not, defer the IOMMU configuration
> +		 * or just abort it.
> +		 */
>  		if (!ops)
> -			return NULL;
> +			return iort_iommu_driver_enabled(node->type) ?
> +			       ERR_PTR(-EPROBE_DEFER) : NULL;
>  
>  		ret = arm_smmu_iort_xlate(dev, streamid, iort_fwnode, ops);
>  	}
> @@ -625,12 +642,26 @@ const struct iommu_ops *iort_iommu_configure(struct device *dev)
>  
>  		while (parent) {
>  			ops = iort_iommu_xlate(dev, parent, streamid);
> +			if (IS_ERR_OR_NULL(ops))
> +				return ops;
>  
>  			parent = iort_node_get_id(node, &streamid,
>  						  IORT_IOMMU_TYPE, i++);
>  		}
>  	}
>  
> +	/*
> +	 * If we have reason to believe the IOMMU driver missed the initial
> +	 * add_device callback for dev, replay it to get things in order.
> +	 */
> +	if (!IS_ERR_OR_NULL(ops) && ops->add_device &&
> +	    dev->bus && !dev->iommu_group) {
> +		int err = ops->add_device(dev);
> +
> +		if (err)
> +			ops = ERR_PTR(err);
> +	}
> +
>  	return ops;
>  }
>  
> diff --git a/drivers/acpi/scan.c b/drivers/acpi/scan.c
> index 1926918..2a513cc 100644
> --- a/drivers/acpi/scan.c
> +++ b/drivers/acpi/scan.c
> @@ -1373,20 +1373,25 @@ enum dev_dma_attr acpi_get_dma_attr(struct acpi_device *adev)
>   * @dev: The pointer to the device
>   * @attr: device dma attributes
>   */
> -void acpi_dma_configure(struct device *dev, enum dev_dma_attr attr)
> +int acpi_dma_configure(struct device *dev, enum dev_dma_attr attr)
>  {
>  	const struct iommu_ops *iommu;
> +	u64 size;
>  
>  	iort_set_dma_mask(dev);
>  
>  	iommu = iort_iommu_configure(dev);
> +	if (IS_ERR(iommu))
> +		return PTR_ERR(iommu);
>  
> +	size = max(dev->coherent_dma_mask, dev->coherent_dma_mask + 1);
>  	/*
>  	 * Assume dma valid range starts at 0 and covers the whole
>  	 * coherent_dma_mask.
>  	 */
> -	arch_setup_dma_ops(dev, 0, dev->coherent_dma_mask + 1, iommu,
> -			   attr == DEV_DMA_COHERENT);
> +	arch_setup_dma_ops(dev, 0, size, iommu, attr == DEV_DMA_COHERENT);
> +
> +	return 0;
>  }
>  EXPORT_SYMBOL_GPL(acpi_dma_configure);
>  
> diff --git a/drivers/base/dma-mapping.c b/drivers/base/dma-mapping.c
> index 82bd45c..755a2b5 100644
> --- a/drivers/base/dma-mapping.c
> +++ b/drivers/base/dma-mapping.c
> @@ -368,7 +368,7 @@ int dma_configure(struct device *dev)
>  	} else if (has_acpi_companion(dma_dev)) {
>  		attr = acpi_get_dma_attr(to_acpi_device_node(dma_dev->fwnode));
>  		if (attr != DEV_DMA_NOT_SUPPORTED)
> -			acpi_dma_configure(dev, attr);
> +			ret = acpi_dma_configure(dev, attr);
>  	}
>  
>  	if (bridge)
> diff --git a/include/acpi/acpi_bus.h b/include/acpi/acpi_bus.h
> index ef0ae8a..2a9a5de 100644
> --- a/include/acpi/acpi_bus.h
> +++ b/include/acpi/acpi_bus.h
> @@ -575,7 +575,7 @@ struct acpi_pci_root {
>  
>  bool acpi_dma_supported(struct acpi_device *adev);
>  enum dev_dma_attr acpi_get_dma_attr(struct acpi_device *adev);
> -void acpi_dma_configure(struct device *dev, enum dev_dma_attr attr);
> +int acpi_dma_configure(struct device *dev, enum dev_dma_attr attr);
>  void acpi_dma_deconfigure(struct device *dev);
>  
>  struct acpi_device *acpi_find_child_device(struct acpi_device *parent,
> diff --git a/include/linux/acpi.h b/include/linux/acpi.h
> index 9b05886..79d06ef6 100644
> --- a/include/linux/acpi.h
> +++ b/include/linux/acpi.h
> @@ -762,8 +762,11 @@ static inline enum dev_dma_attr acpi_get_dma_attr(struct acpi_device *adev)
>  	return DEV_DMA_NOT_SUPPORTED;
>  }
>  
> -static inline void acpi_dma_configure(struct device *dev,
> -				      enum dev_dma_attr attr) { }
> +static inline int acpi_dma_configure(struct device *dev,
> +				     enum dev_dma_attr attr)
> +{
> +	return 0;
> +}
>  
>  static inline void acpi_dma_deconfigure(struct device *dev) { }
>  
> 

[PATCH V10 10/12] arm64: dma-mapping: Remove the notifier trick to handle early setting of dma_ops

[Sricharan R]

With arch_setup_dma_ops now being called late during device's probe after
the device's iommu is probed, the notifier trick required to handle the
early setup of dma_ops before the iommu group gets created is not
required. So removing the notifier's here.

Tested-by: Marek Szyprowski <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
Tested-by: Hanjun Guo <hanjun.guo-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
Acked-by: Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>
Signed-off-by: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
[rm: clean up even more]
Signed-off-by: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>
---
 arch/arm64/mm/dma-mapping.c | 142 ++++++--------------------------------------
 1 file changed, 18 insertions(+), 124 deletions(-)

diff --git a/arch/arm64/mm/dma-mapping.c b/arch/arm64/mm/dma-mapping.c
index 81cdb2e..b465759 100644
--- a/arch/arm64/mm/dma-mapping.c
+++ b/arch/arm64/mm/dma-mapping.c
@@ -813,34 +813,26 @@ static void __iommu_unmap_sg_attrs(struct device *dev,
 	.mapping_error = iommu_dma_mapping_error,
 };
 
-/*
- * TODO: Right now __iommu_setup_dma_ops() gets called too early to do
- * everything it needs to - the device is only partially created and the
- * IOMMU driver hasn't seen it yet, so it can't have a group. Thus we
- * need this delayed attachment dance. Once IOMMU probe ordering is sorted
- * to move the arch_setup_dma_ops() call later, all the notifier bits below
- * become unnecessary, and will go away.
- */
-struct iommu_dma_notifier_data {
-	struct list_head list;
-	struct device *dev;
-	const struct iommu_ops *ops;
-	u64 dma_base;
-	u64 size;
-};
-static LIST_HEAD(iommu_dma_masters);
-static DEFINE_MUTEX(iommu_dma_notifier_lock);
+static int __init __iommu_dma_init(void)
+{
+	return iommu_dma_init();
+}
+arch_initcall(__iommu_dma_init);
 
-static bool do_iommu_attach(struct device *dev, const struct iommu_ops *ops,
-			   u64 dma_base, u64 size)
+static void __iommu_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
+				  const struct iommu_ops *ops)
 {
-	struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
+	struct iommu_domain *domain;
+
+	if (!ops)
+		return;
 
 	/*
-	 * If the IOMMU driver has the DMA domain support that we require,
-	 * then the IOMMU core will have already configured a group for this
-	 * device, and allocated the default domain for that group.
+	 * The IOMMU core code allocates the default DMA domain, which the
+	 * underlying IOMMU driver needs to support via the dma-iommu layer.
 	 */
+	domain = iommu_get_domain_for_dev(dev);
+
 	if (!domain)
 		goto out_err;
 
@@ -851,109 +843,11 @@ static bool do_iommu_attach(struct device *dev, const struct iommu_ops *ops,
 		dev->dma_ops = &iommu_dma_ops;
 	}
 
-	return true;
+	return;
+
 out_err:
-	pr_warn("Failed to set up IOMMU for device %s; retaining platform DMA ops\n",
+	 pr_warn("Failed to set up IOMMU for device %s; retaining platform DMA ops\n",
 		 dev_name(dev));
-	return false;
-}
-
-static void queue_iommu_attach(struct device *dev, const struct iommu_ops *ops,
-			      u64 dma_base, u64 size)
-{
-	struct iommu_dma_notifier_data *iommudata;
-
-	iommudata = kzalloc(sizeof(*iommudata), GFP_KERNEL);
-	if (!iommudata)
-		return;
-
-	iommudata->dev = dev;
-	iommudata->ops = ops;
-	iommudata->dma_base = dma_base;
-	iommudata->size = size;
-
-	mutex_lock(&iommu_dma_notifier_lock);
-	list_add(&iommudata->list, &iommu_dma_masters);
-	mutex_unlock(&iommu_dma_notifier_lock);
-}
-
-static int __iommu_attach_notifier(struct notifier_block *nb,
-				   unsigned long action, void *data)
-{
-	struct iommu_dma_notifier_data *master, *tmp;
-
-	if (action != BUS_NOTIFY_BIND_DRIVER)
-		return 0;
-
-	mutex_lock(&iommu_dma_notifier_lock);
-	list_for_each_entry_safe(master, tmp, &iommu_dma_masters, list) {
-		if (data == master->dev && do_iommu_attach(master->dev,
-				master->ops, master->dma_base, master->size)) {
-			list_del(&master->list);
-			kfree(master);
-			break;
-		}
-	}
-	mutex_unlock(&iommu_dma_notifier_lock);
-	return 0;
-}
-
-static int __init register_iommu_dma_ops_notifier(struct bus_type *bus)
-{
-	struct notifier_block *nb = kzalloc(sizeof(*nb), GFP_KERNEL);
-	int ret;
-
-	if (!nb)
-		return -ENOMEM;
-
-	nb->notifier_call = __iommu_attach_notifier;
-
-	ret = bus_register_notifier(bus, nb);
-	if (ret) {
-		pr_warn("Failed to register DMA domain notifier; IOMMU DMA ops unavailable on bus '%s'\n",
-			bus->name);
-		kfree(nb);
-	}
-	return ret;
-}
-
-static int __init __iommu_dma_init(void)
-{
-	int ret;
-
-	ret = iommu_dma_init();
-	if (!ret)
-		ret = register_iommu_dma_ops_notifier(&platform_bus_type);
-	if (!ret)
-		ret = register_iommu_dma_ops_notifier(&amba_bustype);
-#ifdef CONFIG_PCI
-	if (!ret)
-		ret = register_iommu_dma_ops_notifier(&pci_bus_type);
-#endif
-	return ret;
-}
-arch_initcall(__iommu_dma_init);
-
-static void __iommu_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
-				  const struct iommu_ops *ops)
-{
-	struct iommu_group *group;
-
-	if (!ops)
-		return;
-	/*
-	 * TODO: As a concession to the future, we're ready to handle being
-	 * called both early and late (i.e. after bus_add_device). Once all
-	 * the platform bus code is reworked to call us late and the notifier
-	 * junk above goes away, move the body of do_iommu_attach here.
-	 */
-	group = iommu_group_get(dev);
-	if (group) {
-		do_iommu_attach(dev, ops, dma_base, size);
-		iommu_group_put(group);
-	} else {
-		queue_iommu_attach(dev, ops, dma_base, size);
-	}
 }
 
 void arch_teardown_dma_ops(struct device *dev)
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[PATCH V10 11/12] iommu/arm-smmu: Clean up early-probing workarounds

[Sricharan R]

From: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>

Now that the appropriate ordering is enforced via probe-deferral of
masters in core code, rip it all out and bask in the simplicity.

Tested-by: Hanjun Guo <hanjun.guo-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
Acked-by: Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>
Signed-off-by: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>
[Sricharan: Rebased on top of ACPI IORT SMMU series]
Signed-off-by: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
---
 drivers/iommu/arm-smmu-v3.c |  46 +-----------------
 drivers/iommu/arm-smmu.c    | 110 +++++++++++++++++++-------------------------
 2 files changed, 49 insertions(+), 107 deletions(-)

diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index 591bb96..4362139 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -2761,51 +2761,9 @@ static int arm_smmu_device_remove(struct platform_device *pdev)
 	.probe	= arm_smmu_device_probe,
 	.remove	= arm_smmu_device_remove,
 };
+module_platform_driver(arm_smmu_driver);
 
-static int __init arm_smmu_init(void)
-{
-	static bool registered;
-	int ret = 0;
-
-	if (!registered) {
-		ret = platform_driver_register(&arm_smmu_driver);
-		registered = !ret;
-	}
-	return ret;
-}
-
-static void __exit arm_smmu_exit(void)
-{
-	return platform_driver_unregister(&arm_smmu_driver);
-}
-
-subsys_initcall(arm_smmu_init);
-module_exit(arm_smmu_exit);
-
-static int __init arm_smmu_of_init(struct device_node *np)
-{
-	int ret = arm_smmu_init();
-
-	if (ret)
-		return ret;
-
-	if (!of_platform_device_create(np, NULL, platform_bus_type.dev_root))
-		return -ENODEV;
-
-	return 0;
-}
-IOMMU_OF_DECLARE(arm_smmuv3, "arm,smmu-v3", arm_smmu_of_init);
-
-#ifdef CONFIG_ACPI
-static int __init acpi_smmu_v3_init(struct acpi_table_header *table)
-{
-	if (iort_node_match(ACPI_IORT_NODE_SMMU_V3))
-		return arm_smmu_init();
-
-	return 0;
-}
-IORT_ACPI_DECLARE(arm_smmu_v3, ACPI_SIG_IORT, acpi_smmu_v3_init);
-#endif
+IOMMU_OF_DECLARE(arm_smmuv3, "arm,smmu-v3", NULL);
 
 MODULE_DESCRIPTION("IOMMU API for ARM architected SMMUv3 implementations");
 MODULE_AUTHOR("Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>");
diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c
index b493c99..792fe7d 100644
--- a/drivers/iommu/arm-smmu.c
+++ b/drivers/iommu/arm-smmu.c
@@ -2075,6 +2075,23 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev,
 	return 0;
 }
 
+static void arm_smmu_bus_init(void)
+{
+	/* Oh, for a proper bus abstraction */
+	if (!iommu_present(&platform_bus_type))
+		bus_set_iommu(&platform_bus_type, &arm_smmu_ops);
+#ifdef CONFIG_ARM_AMBA
+	if (!iommu_present(&amba_bustype))
+		bus_set_iommu(&amba_bustype, &arm_smmu_ops);
+#endif
+#ifdef CONFIG_PCI
+	if (!iommu_present(&pci_bus_type)) {
+		pci_request_acs();
+		bus_set_iommu(&pci_bus_type, &arm_smmu_ops);
+	}
+#endif
+}
+
 static int arm_smmu_device_probe(struct platform_device *pdev)
 {
 	struct resource *res;
@@ -2180,21 +2197,30 @@ static int arm_smmu_device_probe(struct platform_device *pdev)
 	arm_smmu_device_reset(smmu);
 	arm_smmu_test_smr_masks(smmu);
 
-	/* Oh, for a proper bus abstraction */
-	if (!iommu_present(&platform_bus_type))
-		bus_set_iommu(&platform_bus_type, &arm_smmu_ops);
-#ifdef CONFIG_ARM_AMBA
-	if (!iommu_present(&amba_bustype))
-		bus_set_iommu(&amba_bustype, &arm_smmu_ops);
-#endif
-#ifdef CONFIG_PCI
-	if (!iommu_present(&pci_bus_type)) {
-		pci_request_acs();
-		bus_set_iommu(&pci_bus_type, &arm_smmu_ops);
-	}
-#endif
+	/*
+	 * For ACPI and generic DT bindings, an SMMU will be probed before
+	 * any device which might need it, so we want the bus ops in place
+	 * ready to handle default domain setup as soon as any SMMU exists.
+	 */
+	if (!using_legacy_binding)
+		arm_smmu_bus_init();
+
+	return 0;
+}
+
+/*
+ * With the legacy DT binding in play, though, we have no guarantees about
+ * probe order, but then we're also not doing default domains, so we can
+ * delay setting bus ops until we're sure every possible SMMU is ready,
+ * and that way ensure that no add_device() calls get missed.
+ */
+static int arm_smmu_legacy_bus_init(void)
+{
+	if (using_legacy_binding)
+		arm_smmu_bus_init();
 	return 0;
 }
+device_initcall_sync(arm_smmu_legacy_bus_init);
 
 static int arm_smmu_device_remove(struct platform_device *pdev)
 {
@@ -2219,56 +2245,14 @@ static int arm_smmu_device_remove(struct platform_device *pdev)
 	.probe	= arm_smmu_device_probe,
 	.remove	= arm_smmu_device_remove,
 };
-
-static int __init arm_smmu_init(void)
-{
-	static bool registered;
-	int ret = 0;
-
-	if (!registered) {
-		ret = platform_driver_register(&arm_smmu_driver);
-		registered = !ret;
-	}
-	return ret;
-}
-
-static void __exit arm_smmu_exit(void)
-{
-	return platform_driver_unregister(&arm_smmu_driver);
-}
-
-subsys_initcall(arm_smmu_init);
-module_exit(arm_smmu_exit);
-
-static int __init arm_smmu_of_init(struct device_node *np)
-{
-	int ret = arm_smmu_init();
-
-	if (ret)
-		return ret;
-
-	if (!of_platform_device_create(np, NULL, platform_bus_type.dev_root))
-		return -ENODEV;
-
-	return 0;
-}
-IOMMU_OF_DECLARE(arm_smmuv1, "arm,smmu-v1", arm_smmu_of_init);
-IOMMU_OF_DECLARE(arm_smmuv2, "arm,smmu-v2", arm_smmu_of_init);
-IOMMU_OF_DECLARE(arm_mmu400, "arm,mmu-400", arm_smmu_of_init);
-IOMMU_OF_DECLARE(arm_mmu401, "arm,mmu-401", arm_smmu_of_init);
-IOMMU_OF_DECLARE(arm_mmu500, "arm,mmu-500", arm_smmu_of_init);
-IOMMU_OF_DECLARE(cavium_smmuv2, "cavium,smmu-v2", arm_smmu_of_init);
-
-#ifdef CONFIG_ACPI
-static int __init arm_smmu_acpi_init(struct acpi_table_header *table)
-{
-	if (iort_node_match(ACPI_IORT_NODE_SMMU))
-		return arm_smmu_init();
-
-	return 0;
-}
-IORT_ACPI_DECLARE(arm_smmu, ACPI_SIG_IORT, arm_smmu_acpi_init);
-#endif
+module_platform_driver(arm_smmu_driver);
+
+IOMMU_OF_DECLARE(arm_smmuv1, "arm,smmu-v1", NULL);
+IOMMU_OF_DECLARE(arm_smmuv2, "arm,smmu-v2", NULL);
+IOMMU_OF_DECLARE(arm_mmu400, "arm,mmu-400", NULL);
+IOMMU_OF_DECLARE(arm_mmu401, "arm,mmu-401", NULL);
+IOMMU_OF_DECLARE(arm_mmu500, "arm,mmu-500", NULL);
+IOMMU_OF_DECLARE(cavium_smmuv2, "cavium,smmu-v2", NULL);
 
 MODULE_DESCRIPTION("IOMMU API for ARM architected SMMU implementations");
 MODULE_AUTHOR("Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>");
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[PATCH V10 12/12] ACPI/IORT: Remove linker section for IORT entries probing

[Sricharan R]

From: Lorenzo Pieralisi <lorenzo.pieralisi-5wv7dgnIgG8@public.gmane.org>

The IORT linker section introduced by commit 34ceea275f62
("ACPI/IORT: Introduce linker section for IORT entries probing")
was needed to make sure SMMU drivers are registered (and therefore
probed) in the kernel before devices using the SMMU have a chance
to probe in turn.

Through the introduction of deferred IOMMU configuration the linker
section based IORT probing infrastructure is not needed any longer, in
that device/SMMU probe dependencies are managed through the probe
deferral mechanism, making the IORT linker section infrastructure
unused, so that it can be removed.

Remove the unused IORT linker section probing infrastructure
from the kernel to complete the ACPI IORT IOMMU configure probe
deferral mechanism implementation.

Tested-by: Hanjun Guo <hanjun.guo-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi-5wv7dgnIgG8@public.gmane.org>
Cc: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>
Cc: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
---
 drivers/acpi/arm64/iort.c         | 2 --
 include/asm-generic/vmlinux.lds.h | 1 -
 include/linux/acpi_iort.h         | 3 ---
 3 files changed, 6 deletions(-)

diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
index e323ece..e7b1940 100644
--- a/drivers/acpi/arm64/iort.c
+++ b/drivers/acpi/arm64/iort.c
@@ -1000,6 +1000,4 @@ void __init acpi_iort_init(void)
 	}
 
 	iort_init_platform_devices();
-
-	acpi_probe_device_table(iort);
 }
diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index 0968d13..9faa26c 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -566,7 +566,6 @@
 	IRQCHIP_OF_MATCH_TABLE()					\
 	ACPI_PROBE_TABLE(irqchip)					\
 	ACPI_PROBE_TABLE(clksrc)					\
-	ACPI_PROBE_TABLE(iort)						\
 	EARLYCON_TABLE()
 
 #define INIT_TEXT							\
diff --git a/include/linux/acpi_iort.h b/include/linux/acpi_iort.h
index 77e0809..f167e1d04 100644
--- a/include/linux/acpi_iort.h
+++ b/include/linux/acpi_iort.h
@@ -52,7 +52,4 @@ const struct iommu_ops *iort_iommu_configure(struct device *dev)
 { return NULL; }
 #endif
 
-#define IORT_ACPI_DECLARE(name, table_id, fn)		\
-	ACPI_DECLARE_PROBE_ENTRY(iort, name, table_id, 0, NULL, 0, fn)
-
 #endif /* __ACPI_IORT_H__ */
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

Re: [PATCH V10 12/12] ACPI/IORT: Remove linker section for IORT entries probing

[Robin Murphy]

On 04/04/17 11:18, Sricharan R wrote:
> From: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> 
> The IORT linker section introduced by commit 34ceea275f62
> ("ACPI/IORT: Introduce linker section for IORT entries probing")
> was needed to make sure SMMU drivers are registered (and therefore
> probed) in the kernel before devices using the SMMU have a chance
> to probe in turn.
> 
> Through the introduction of deferred IOMMU configuration the linker
> section based IORT probing infrastructure is not needed any longer, in
> that device/SMMU probe dependencies are managed through the probe
> deferral mechanism, making the IORT linker section infrastructure
> unused, so that it can be removed.
> 
> Remove the unused IORT linker section probing infrastructure
> from the kernel to complete the ACPI IORT IOMMU configure probe
> deferral mechanism implementation.

Reviewed-by: Robin Murphy <robin.murphy@arm.com>

> Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> Cc: Robin Murphy <robin.murphy@arm.com>
> Cc: Sricharan R <sricharan@codeaurora.org>
> ---
>  drivers/acpi/arm64/iort.c         | 2 --
>  include/asm-generic/vmlinux.lds.h | 1 -
>  include/linux/acpi_iort.h         | 3 ---
>  3 files changed, 6 deletions(-)
> 
> diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> index e323ece..e7b1940 100644
> --- a/drivers/acpi/arm64/iort.c
> +++ b/drivers/acpi/arm64/iort.c
> @@ -1000,6 +1000,4 @@ void __init acpi_iort_init(void)
>  	}
>  
>  	iort_init_platform_devices();
> -
> -	acpi_probe_device_table(iort);
>  }
> diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
> index 0968d13..9faa26c 100644
> --- a/include/asm-generic/vmlinux.lds.h
> +++ b/include/asm-generic/vmlinux.lds.h
> @@ -566,7 +566,6 @@
>  	IRQCHIP_OF_MATCH_TABLE()					\
>  	ACPI_PROBE_TABLE(irqchip)					\
>  	ACPI_PROBE_TABLE(clksrc)					\
> -	ACPI_PROBE_TABLE(iort)						\
>  	EARLYCON_TABLE()
>  
>  #define INIT_TEXT							\
> diff --git a/include/linux/acpi_iort.h b/include/linux/acpi_iort.h
> index 77e0809..f167e1d04 100644
> --- a/include/linux/acpi_iort.h
> +++ b/include/linux/acpi_iort.h
> @@ -52,7 +52,4 @@ const struct iommu_ops *iort_iommu_configure(struct device *dev)
>  { return NULL; }
>  #endif
>  
> -#define IORT_ACPI_DECLARE(name, table_id, fn)		\
> -	ACPI_DECLARE_PROBE_ENTRY(iort, name, table_id, 0, NULL, 0, fn)
> -
>  #endif /* __ACPI_IORT_H__ */
> 

Re: [PATCH V10 00/12] IOMMU probe deferral support

[Robin Murphy]

On 04/04/17 11:18, Sricharan R wrote:
> This series calls the dma ops configuration for the devices
> at a generic place so that it works for all busses.
> The dma_configure_ops for a device is now called during
> the device_attach callback just before the probe of the
> bus/driver is called. Similarly dma_deconfigure is called during
> device/driver_detach path.
> 
> pci_bus_add_devices    (platform/amba)(_device_create/driver_register)
>        |                         |
> pci_bus_add_device     (device_add/driver_register)
>        |                         |
> device_attach           device_initial_probe
>        |                         |
> __device_attach_driver    __device_attach_driver
>        |
> driver_probe_device
>        |
> really_probe
>        |
> dma_configure
> 
> Similarly on the device/driver_unregister path __device_release_driver is
> called which inturn calls dma_deconfigure.
> 
> Rebased the series against mainline 4.11-rc5. Applies and builds cleanly
> against mainline and linux-next, iommu-next.
>   
> * Tested with platform and pci devices for probe deferral
>   and reprobe on arm64 based platform.

Sricharan, thanks for keeping this going - I really think we're there now :)

Joerg, I realise that at -rc5 time is getting on a bit already, but even
the non-vintage parts of the series are pretty mature now so it would be
nice to at least give it a spin in -next. If you don't quite share my
confidence for landing it in 4.12, please consider it for early next
cycle to get a full workout.

Thanks,
Robin.

> Previous post of this series [8]. 
> 
> Please note that, i have kept the tested/acked tags intact from V8
> because V9/10 were for more fixes that was added, so the original
> tags that was given for the functional testing remains the same.
> 
>  [V10]
>      * Rebased on top of 4.11-rc5.
>  
>      * Fixed coherent_dma_mask 64bit overflow issue [8]
>        for OF. The fix for OF was added as a separate
>        patch#6, since the issue is true even without probe deferral,
>        but gets reproduced with the probe deferral series.
>        Added Lorenzo's ACPI fix for coherent_dma_mask overflow
>        and the fix for dma_configure getting called more than
>        once for the same device.
> 
>      * Also fixed an build issue caught by kbuild robot for
>        m68k arch. The issue was dma_(de)configure was not
>        getting defined for !CONFIG_HAS_DMA, so fixed that as well.
> 
>  [V9]
>      * Rebased on top of 4.11-rc1.
> 
>      * Merged Robin's fixes for legacy binding issue,
>        pci devices with no iommu-map property and deferencing
>        of_iommu_table after init.
>      
>  [V8]
>      * Picked up all the acks and tested tags from Marek and
>        Hanjun for DT and ACPI patches respectively, since
>        no functional changes was done.
> 
>      * Addressed Minor comments Sinan and Bjorn.
> 
>      * Added Robin's fix for fixing the deferencing NULL for
>        of_iommu_table after init in patch #2.
> 
>      * Rebased it on top of linux-next
> 
>  [V7]
>      * Updated the subject and commit log for patch #6 as per
>        comments from Lorenzo. No functional changes.
> 
>  [V6]
>      * Fixed a bug in dma_configure function pointed out by
>        Robin.
>      * Reordered the patches as per comments from Robin and
>        Lorenzo.
>      * Added Tags.
> 
>  [V5]
>      * Reworked the pci configuration code hanging outside and
>        pushed it to dma_configure as in PATCH#5,6,7.
>        Also added a couple of patches that Lorenzo provided for
>        correcting the Probe deferring mechanism in case of
>        ACPI devices from here [5].
> 
>  [V4]
>      * Took the reworked patches [2] from Robin's branch and
>        rebased on top of Lorenzo's ACPI IORT ARM support series [3].
> 
>      * Added the patches for moving the dma ops configuration of
>        acpi based devices to probe time as well.
>  [V3]
>      * Removed the patch to split dma_masks/dma_ops configuration
>        separately based on review comments that both masks and ops are
>        required only during the device probe time.
> 
>      * Reworked the series based on Generic DT bindings series.
> 
>      * Added call to iommu's remove_device in the cleanup path for arm and
>        arm64.
> 
>      * Removed the notifier trick in arm64 to handle early device
>        registration.
> 
>      * Added reset of dma_ops in cleanup path for arm based on comments.
> 
>      * Fixed the pci_iommu_configure path and tested with PCI device as
>        well.
>  
>      * Fixed a bug to return the correct iommu_ops from patch 7 [4] in
>        last post.
> 
>      * Fixed few other cosmetic comments.
>   
>  [V2]
>      * Updated the Initial post to call dma_configure/deconfigure from
>        generic code
>  
>      * Added iommu add_device callback from of_iommu_configure path
> 
>  [V1]
>      * Initial post from Laurent Pinchart [1]
> 
> [1] http://lists.linuxfoundation.org/pipermail/iommu/2015-May/013016.html
> [2] http://www.linux-arm.org/git?p=linux-rm.git;a=shortlog;h=refs/heads/iommu/defer
> [3] https://lkml.org/lkml/2016/11/21/141
> [4] https://www.mail-archive.com/iommu@xxxxxxxxxxxxxxxxxxxxxxxxxx/msg13940.html
> [5] git://git.kernel.org/pub/scm/linux/kernel/git/lpieralisi/linux.git iommu/probe-deferral
> [6] http://www.spinics.net/lists/linux-pci/msg57992.html
> [7] https://www.spinics.net/lists/arm-kernel/msg556209.html
> [8] http://patchwork.ozlabs.org/patch/743898/
> 
> Laurent Pinchart (3):
>   of: dma: Move range size workaround to of_dma_get_range()
>   of: dma: Make of_dma_deconfigure() public
>   iommu: of: Handle IOMMU lookup failure with deferred probing or error
> 
> Lorenzo Pieralisi (2):
>   ACPI/IORT: Add function to check SMMUs drivers presence
>   ACPI/IORT: Remove linker section for IORT entries probing
> 
> Robin Murphy (3):
>   iommu/of: Refactor of_iommu_configure() for error handling
>   iommu/of: Prepare for deferred IOMMU configuration
>   iommu/arm-smmu: Clean up early-probing workarounds
> 
> Sricharan R (4):
>   of: device: Fix overflow of coherent_dma_mask
>   of/acpi: Configure dma operations at probe time for platform/amba/pci
>     bus devices
>   drivers: acpi: Handle IOMMU lookup failure with deferred probing or
>     error
>   arm64: dma-mapping: Remove the notifier trick to handle early setting
>     of dma_ops
> 
>  arch/arm64/mm/dma-mapping.c       | 142 +++++---------------------------------
>  drivers/acpi/arm64/iort.c         |  48 ++++++++++++-
>  drivers/acpi/glue.c               |   5 --
>  drivers/acpi/scan.c               |  11 ++-
>  drivers/base/dd.c                 |   9 +++
>  drivers/base/dma-mapping.c        |  41 +++++++++++
>  drivers/iommu/arm-smmu-v3.c       |  46 +-----------
>  drivers/iommu/arm-smmu.c          | 110 +++++++++++++----------------
>  drivers/iommu/of_iommu.c          | 126 ++++++++++++++++++++++++---------
>  drivers/of/address.c              |  20 +++++-
>  drivers/of/device.c               |  36 +++++-----
>  drivers/of/platform.c             |  10 +--
>  drivers/pci/probe.c               |  28 --------
>  include/acpi/acpi_bus.h           |   2 +-
>  include/asm-generic/vmlinux.lds.h |   1 -
>  include/linux/acpi.h              |   7 +-
>  include/linux/acpi_iort.h         |   3 -
>  include/linux/dma-mapping.h       |  12 ++++
>  include/linux/of_device.h         |  10 ++-
>  19 files changed, 329 insertions(+), 338 deletions(-)
> 

Re: [PATCH V10 00/12] IOMMU probe deferral support

[Lorenzo Pieralisi]

On Tue, Apr 04, 2017 at 01:49:29PM +0100, Robin Murphy wrote:
> On 04/04/17 11:18, Sricharan R wrote:
> > This series calls the dma ops configuration for the devices
> > at a generic place so that it works for all busses.
> > The dma_configure_ops for a device is now called during
> > the device_attach callback just before the probe of the
> > bus/driver is called. Similarly dma_deconfigure is called during
> > device/driver_detach path.
> > 
> > pci_bus_add_devices    (platform/amba)(_device_create/driver_register)
> >        |                         |
> > pci_bus_add_device     (device_add/driver_register)
> >        |                         |
> > device_attach           device_initial_probe
> >        |                         |
> > __device_attach_driver    __device_attach_driver
> >        |
> > driver_probe_device
> >        |
> > really_probe
> >        |
> > dma_configure
> > 
> > Similarly on the device/driver_unregister path __device_release_driver is
> > called which inturn calls dma_deconfigure.
> > 
> > Rebased the series against mainline 4.11-rc5. Applies and builds cleanly
> > against mainline and linux-next, iommu-next.
> >   
> > * Tested with platform and pci devices for probe deferral
> >   and reprobe on arm64 based platform.
> 
> Sricharan, thanks for keeping this going - I really think we're there now :)

FYI, I re-tested this series with ACPI and tried a merge with IORT
patches queued for 4.12 (via arm64) and there does not seem to be any
merge conflicts so it should be ready to go.

Thanks,
Lorenzo

> Joerg, I realise that at -rc5 time is getting on a bit already, but even
> the non-vintage parts of the series are pretty mature now so it would be
> nice to at least give it a spin in -next. If you don't quite share my
> confidence for landing it in 4.12, please consider it for early next
> cycle to get a full workout.
> 
> Thanks,
> Robin.
> 
> > Previous post of this series [8]. 
> > 
> > Please note that, i have kept the tested/acked tags intact from V8
> > because V9/10 were for more fixes that was added, so the original
> > tags that was given for the functional testing remains the same.
> > 
> >  [V10]
> >      * Rebased on top of 4.11-rc5.
> >  
> >      * Fixed coherent_dma_mask 64bit overflow issue [8]
> >        for OF. The fix for OF was added as a separate
> >        patch#6, since the issue is true even without probe deferral,
> >        but gets reproduced with the probe deferral series.
> >        Added Lorenzo's ACPI fix for coherent_dma_mask overflow
> >        and the fix for dma_configure getting called more than
> >        once for the same device.
> > 
> >      * Also fixed an build issue caught by kbuild robot for
> >        m68k arch. The issue was dma_(de)configure was not
> >        getting defined for !CONFIG_HAS_DMA, so fixed that as well.
> > 
> >  [V9]
> >      * Rebased on top of 4.11-rc1.
> > 
> >      * Merged Robin's fixes for legacy binding issue,
> >        pci devices with no iommu-map property and deferencing
> >        of_iommu_table after init.
> >      
> >  [V8]
> >      * Picked up all the acks and tested tags from Marek and
> >        Hanjun for DT and ACPI patches respectively, since
> >        no functional changes was done.
> > 
> >      * Addressed Minor comments Sinan and Bjorn.
> > 
> >      * Added Robin's fix for fixing the deferencing NULL for
> >        of_iommu_table after init in patch #2.
> > 
> >      * Rebased it on top of linux-next
> > 
> >  [V7]
> >      * Updated the subject and commit log for patch #6 as per
> >        comments from Lorenzo. No functional changes.
> > 
> >  [V6]
> >      * Fixed a bug in dma_configure function pointed out by
> >        Robin.
> >      * Reordered the patches as per comments from Robin and
> >        Lorenzo.
> >      * Added Tags.
> > 
> >  [V5]
> >      * Reworked the pci configuration code hanging outside and
> >        pushed it to dma_configure as in PATCH#5,6,7.
> >        Also added a couple of patches that Lorenzo provided for
> >        correcting the Probe deferring mechanism in case of
> >        ACPI devices from here [5].
> > 
> >  [V4]
> >      * Took the reworked patches [2] from Robin's branch and
> >        rebased on top of Lorenzo's ACPI IORT ARM support series [3].
> > 
> >      * Added the patches for moving the dma ops configuration of
> >        acpi based devices to probe time as well.
> >  [V3]
> >      * Removed the patch to split dma_masks/dma_ops configuration
> >        separately based on review comments that both masks and ops are
> >        required only during the device probe time.
> > 
> >      * Reworked the series based on Generic DT bindings series.
> > 
> >      * Added call to iommu's remove_device in the cleanup path for arm and
> >        arm64.
> > 
> >      * Removed the notifier trick in arm64 to handle early device
> >        registration.
> > 
> >      * Added reset of dma_ops in cleanup path for arm based on comments.
> > 
> >      * Fixed the pci_iommu_configure path and tested with PCI device as
> >        well.
> >  
> >      * Fixed a bug to return the correct iommu_ops from patch 7 [4] in
> >        last post.
> > 
> >      * Fixed few other cosmetic comments.
> >   
> >  [V2]
> >      * Updated the Initial post to call dma_configure/deconfigure from
> >        generic code
> >  
> >      * Added iommu add_device callback from of_iommu_configure path
> > 
> >  [V1]
> >      * Initial post from Laurent Pinchart [1]
> > 
> > [1] http://lists.linuxfoundation.org/pipermail/iommu/2015-May/013016.html
> > [2] http://www.linux-arm.org/git?p=linux-rm.git;a=shortlog;h=refs/heads/iommu/defer
> > [3] https://lkml.org/lkml/2016/11/21/141
> > [4] https://www.mail-archive.com/iommu@xxxxxxxxxxxxxxxxxxxxxxxxxx/msg13940.html
> > [5] git://git.kernel.org/pub/scm/linux/kernel/git/lpieralisi/linux.git iommu/probe-deferral
> > [6] http://www.spinics.net/lists/linux-pci/msg57992.html
> > [7] https://www.spinics.net/lists/arm-kernel/msg556209.html
> > [8] http://patchwork.ozlabs.org/patch/743898/
> > 
> > Laurent Pinchart (3):
> >   of: dma: Move range size workaround to of_dma_get_range()
> >   of: dma: Make of_dma_deconfigure() public
> >   iommu: of: Handle IOMMU lookup failure with deferred probing or error
> > 
> > Lorenzo Pieralisi (2):
> >   ACPI/IORT: Add function to check SMMUs drivers presence
> >   ACPI/IORT: Remove linker section for IORT entries probing
> > 
> > Robin Murphy (3):
> >   iommu/of: Refactor of_iommu_configure() for error handling
> >   iommu/of: Prepare for deferred IOMMU configuration
> >   iommu/arm-smmu: Clean up early-probing workarounds
> > 
> > Sricharan R (4):
> >   of: device: Fix overflow of coherent_dma_mask
> >   of/acpi: Configure dma operations at probe time for platform/amba/pci
> >     bus devices
> >   drivers: acpi: Handle IOMMU lookup failure with deferred probing or
> >     error
> >   arm64: dma-mapping: Remove the notifier trick to handle early setting
> >     of dma_ops
> > 
> >  arch/arm64/mm/dma-mapping.c       | 142 +++++---------------------------------
> >  drivers/acpi/arm64/iort.c         |  48 ++++++++++++-
> >  drivers/acpi/glue.c               |   5 --
> >  drivers/acpi/scan.c               |  11 ++-
> >  drivers/base/dd.c                 |   9 +++
> >  drivers/base/dma-mapping.c        |  41 +++++++++++
> >  drivers/iommu/arm-smmu-v3.c       |  46 +-----------
> >  drivers/iommu/arm-smmu.c          | 110 +++++++++++++----------------
> >  drivers/iommu/of_iommu.c          | 126 ++++++++++++++++++++++++---------
> >  drivers/of/address.c              |  20 +++++-
> >  drivers/of/device.c               |  36 +++++-----
> >  drivers/of/platform.c             |  10 +--
> >  drivers/pci/probe.c               |  28 --------
> >  include/acpi/acpi_bus.h           |   2 +-
> >  include/asm-generic/vmlinux.lds.h |   1 -
> >  include/linux/acpi.h              |   7 +-
> >  include/linux/acpi_iort.h         |   3 -
> >  include/linux/dma-mapping.h       |  12 ++++
> >  include/linux/of_device.h         |  10 ++-
> >  19 files changed, 329 insertions(+), 338 deletions(-)
> > 
> 

Re: [PATCH V10 00/12] IOMMU probe deferral support

[Rob Herring]

On Tue, Apr 4, 2017 at 5:18 AM, Sricharan R <sricharan@codeaurora.org> wrote:
> This series calls the dma ops configuration for the devices
> at a generic place so that it works for all busses.
> The dma_configure_ops for a device is now called during
> the device_attach callback just before the probe of the
> bus/driver is called. Similarly dma_deconfigure is called during
> device/driver_detach path.

For patches 3, 4, 6, 7, 8:

Acked-by: Rob Herring <robh@kernel.org>

Re: [PATCH V10 00/12] IOMMU probe deferral support

[Frank Rowand]

On 04/04/17 18:23, Rob Herring wrote:
> On Tue, Apr 4, 2017 at 5:18 AM, Sricharan R <sricharan@codeaurora.org> wrote:
>> This series calls the dma ops configuration for the devices
>> at a generic place so that it works for all busses.
>> The dma_configure_ops for a device is now called during
>> the device_attach callback just before the probe of the
>> bus/driver is called. Similarly dma_deconfigure is called during
>> device/driver_detach path.
> 
> For patches 3, 4, 6, 7, 8:
> 
> Acked-by: Rob Herring <robh@kernel.org>
> .
> 

And a few hours later, I had the opposite opinion on a couple of
patches, so there is some ongoing discussion working this out.

-Frank