[PATCH RESEND v2 0/8] Cache-coherent DMA access using UIO

[PATCH RESEND v2 0/8] Cache-coherent DMA access using UIO

[Anup Patel]

The goal of this patchset is to improve UIO framework and UIO dmem
driver to allow cache-coherent DMA accesses from user-space.

This patchset is based on two previous patchsets:
1) [PATCH v5 0/6] UIO driver for APM X-Gene QMTM
(Refer, http://www.spinics.net/lists/devicetree/msg58244.html)
2) [PATCH 0/4] Fix and extend uio_dmem_genirq
(Refer, https://lkml.org/lkml/2016/5/17/141)

We have adopted only patch0-3 of patchset1 which was abandoned
long time back. We have taken care of last few unaddressed comments
on these patches.

The patchset2 is quite recent has been adopted entirely. We have
taken care review comments on these patches too.

This patchset is based on v4.7-rc7 tag and it is available in uio-v2
branch of https://github.com/Broadcom/arm64-linux.git

Changes since v1:
 - Make patch5 as first patch1 and mark it for linux-stable
 - Check return value of dma_set_coherent_mask()
 - Get UIO dmem dynamic region details as module parameter instead
   of getting it from DT node

Ankit Jindal (3):
  uio: code style cleanup
  uio: Add new UIO_MEM_PHYS_CACHE type for mem regions
  Documentation: Update documentation for UIO_MEM_PHYS_CACHE and
    UIO_MEM_DEVICE

Anup Patel (2):
  uio: Add new UIO_MEM_DEVICE type for mem regions
  uio: Use new memtypes in uio_dmem_genirq

Jan Viktorin (3):
  uio: fix dmem_region_start computation
  uio: UIO_IRQ_NONE is a valid option for uioinfo->irq
  uio: bind uio_dmem_genirq via OF

 Documentation/DocBook/uio-howto.tmpl |   6 +-
 drivers/uio/uio.c                    |  32 ++++--
 drivers/uio/uio_dmem_genirq.c        | 190 +++++++++++++++++++++++++++--------
 include/linux/uio_driver.h           |  10 +-
 4 files changed, 182 insertions(+), 56 deletions(-)

-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH RESEND v2 1/8] uio: fix dmem_region_start computation

[Anup Patel]

From: Jan Viktorin <viktorin@rehivetech.com>

The variable i contains a total number of resources (including
IORESOURCE_IRQ). However, we want the dmem_region_start to point
after the last resource of type IORESOURCE_MEM. The original behaviour
leads (very likely) to skipping several UIO mapping regions and makes
them useless. Fix this by computing dmem_region_start from the uiomem
which points to the last used UIO mapping.

Fixes: 0a0c3b5a24bd ("Add new uio device for dynamic memory allocation")

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
Signed-off-by: Anup Patel <anup.patel@broadcom.com>
Cc: linux-stable <stable@vger.kernel.org>
---
 drivers/uio/uio_dmem_genirq.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/uio/uio_dmem_genirq.c b/drivers/uio/uio_dmem_genirq.c
index 915facb..e1134a4 100644
--- a/drivers/uio/uio_dmem_genirq.c
+++ b/drivers/uio/uio_dmem_genirq.c
@@ -229,7 +229,7 @@ static int uio_dmem_genirq_probe(struct platform_device *pdev)
 		++uiomem;
 	}
 
-	priv->dmem_region_start = i;
+	priv->dmem_region_start = uiomem - &uioinfo->mem[0];
 	priv->num_dmem_regions = pdata->num_dynamic_regions;
 
 	for (i = 0; i < pdata->num_dynamic_regions; ++i) {
-- 
1.9.1

[PATCH RESEND v2 2/8] uio: code style cleanup

[Anup Patel]

From: Ankit Jindal <thatsjindal@gmail.com>

This patch fixes the indentation of switch-case block in uio driver.

Signed-off-by: Ankit Jindal <thatsjindal@gmail.com>
Signed-off-by: Anup Patel <anup.patel@broadcom.com>
---
 drivers/uio/uio.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/drivers/uio/uio.c b/drivers/uio/uio.c
index fba021f..f2729b7 100644
--- a/drivers/uio/uio.c
+++ b/drivers/uio/uio.c
@@ -703,13 +703,13 @@ static int uio_mmap(struct file *filep, struct vm_area_struct *vma)
 	}
 
 	switch (idev->info->mem[mi].memtype) {
-		case UIO_MEM_PHYS:
-			return uio_mmap_physical(vma);
-		case UIO_MEM_LOGICAL:
-		case UIO_MEM_VIRTUAL:
-			return uio_mmap_logical(vma);
-		default:
-			return -EINVAL;
+	case UIO_MEM_PHYS:
+		return uio_mmap_physical(vma);
+	case UIO_MEM_LOGICAL:
+	case UIO_MEM_VIRTUAL:
+		return uio_mmap_logical(vma);
+	default:
+		return -EINVAL;
 	}
 }
 
-- 
1.9.1

[PATCH RESEND v2 3/8] uio: Add new UIO_MEM_PHYS_CACHE type for mem regions

[Anup Patel]

From: Ankit Jindal <thatsjindal@gmail.com>

Currently, three types of mem regions are supported: UIO_MEM_PHYS,
UIO_MEM_LOGICAL and UIO_MEM_VIRTUAL. Among these UIO_MEM_PHYS helps
UIO driver export physcial memory to user space as non-cacheable
user memory. Typcially memory-mapped registers of a device are exported
to user space as UIO_MEM_PHYS type mem region. The UIO_MEM_PHYS type
is not efficient if dma-capable devices are capable of maintaining
coherency with CPU caches.

This patch adds new type UIO_MEM_PHYS_CACHE for mem regions to enable
cacheable access to physical memory from user space.

Signed-off-by: Ankit Jindal <thatsjindal@gmail.com>
Signed-off-by: Anup Patel <anup.patel@broadcom.com>
---
 drivers/uio/uio.c          | 16 +++++++++++++---
 include/linux/uio_driver.h |  9 +++++----
 2 files changed, 18 insertions(+), 7 deletions(-)

diff --git a/drivers/uio/uio.c b/drivers/uio/uio.c
index f2729b7..0e53076 100644
--- a/drivers/uio/uio.c
+++ b/drivers/uio/uio.c
@@ -641,7 +641,7 @@ static const struct vm_operations_struct uio_physical_vm_ops = {
 #endif
 };
 
-static int uio_mmap_physical(struct vm_area_struct *vma)
+static int uio_mmap_physical(struct vm_area_struct *vma, int memtype)
 {
 	struct uio_device *idev = vma->vm_private_data;
 	int mi = uio_find_mem_index(vma);
@@ -656,7 +656,16 @@ static int uio_mmap_physical(struct vm_area_struct *vma)
 		return -EINVAL;
 
 	vma->vm_ops = &uio_physical_vm_ops;
-	vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
+	switch (memtype) {
+	case UIO_MEM_PHYS:
+		vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
+		break;
+	case UIO_MEM_PHYS_CACHE:
+		/* Do nothing. */
+		break;
+	default:
+		return -EINVAL;
+	}
 
 	/*
 	 * We cannot use the vm_iomap_memory() helper here,
@@ -704,7 +713,8 @@ static int uio_mmap(struct file *filep, struct vm_area_struct *vma)
 
 	switch (idev->info->mem[mi].memtype) {
 	case UIO_MEM_PHYS:
-		return uio_mmap_physical(vma);
+	case UIO_MEM_PHYS_CACHE:
+		return uio_mmap_physical(vma, idev->info->mem[mi].memtype);
 	case UIO_MEM_LOGICAL:
 	case UIO_MEM_VIRTUAL:
 		return uio_mmap_logical(vma);
diff --git a/include/linux/uio_driver.h b/include/linux/uio_driver.h
index 32c0e83..31359aee 100644
--- a/include/linux/uio_driver.h
+++ b/include/linux/uio_driver.h
@@ -124,10 +124,11 @@ extern void uio_event_notify(struct uio_info *info);
 #define UIO_IRQ_NONE	0
 
 /* defines for uio_mem->memtype */
-#define UIO_MEM_NONE	0
-#define UIO_MEM_PHYS	1
-#define UIO_MEM_LOGICAL	2
-#define UIO_MEM_VIRTUAL 3
+#define UIO_MEM_NONE		0
+#define UIO_MEM_PHYS		1
+#define UIO_MEM_LOGICAL		2
+#define UIO_MEM_VIRTUAL		3
+#define UIO_MEM_PHYS_CACHE	4
 
 /* defines for uio_port->porttype */
 #define UIO_PORT_NONE	0
-- 
1.9.1

[PATCH RESEND v2 4/8] uio: Add new UIO_MEM_DEVICE type for mem regions

[Anup Patel]

On ARM64, the MMU supports special memory attributes for device
memory/registers. Due to this we have pgprot_device() provided
by asm/pgtable.h of arch/arm64.

On architectures that do not have special MMU attribute for device
memory/registers, the asm-generic/pgtable.h maps pgprot_device()
to pgprot_noncached().

This patch introduces a new UIO mem region type UIO_MEM_DEVICE to
represent device registers/memory. The UIO device drivers should
prefer this new UIO mem region type for memory mapped device registers.

Signed-off-by: Anup Patel <anup.patel@broadcom.com>
---
 drivers/uio/uio.c          | 4 ++++
 include/linux/uio_driver.h | 1 +
 2 files changed, 5 insertions(+)

diff --git a/drivers/uio/uio.c b/drivers/uio/uio.c
index 0e53076..a00990c 100644
--- a/drivers/uio/uio.c
+++ b/drivers/uio/uio.c
@@ -663,6 +663,9 @@ static int uio_mmap_physical(struct vm_area_struct *vma, int memtype)
 	case UIO_MEM_PHYS_CACHE:
 		/* Do nothing. */
 		break;
+	case UIO_MEM_DEVICE:
+		vma->vm_page_prot = pgprot_device(vma->vm_page_prot);
+		break;
 	default:
 		return -EINVAL;
 	}
@@ -714,6 +717,7 @@ static int uio_mmap(struct file *filep, struct vm_area_struct *vma)
 	switch (idev->info->mem[mi].memtype) {
 	case UIO_MEM_PHYS:
 	case UIO_MEM_PHYS_CACHE:
+	case UIO_MEM_DEVICE:
 		return uio_mmap_physical(vma, idev->info->mem[mi].memtype);
 	case UIO_MEM_LOGICAL:
 	case UIO_MEM_VIRTUAL:
diff --git a/include/linux/uio_driver.h b/include/linux/uio_driver.h
index 31359aee..7349f95 100644
--- a/include/linux/uio_driver.h
+++ b/include/linux/uio_driver.h
@@ -129,6 +129,7 @@ extern void uio_event_notify(struct uio_info *info);
 #define UIO_MEM_LOGICAL		2
 #define UIO_MEM_VIRTUAL		3
 #define UIO_MEM_PHYS_CACHE	4
+#define UIO_MEM_DEVICE		5
 
 /* defines for uio_port->porttype */
 #define UIO_PORT_NONE	0
-- 
1.9.1

[PATCH RESEND v2 5/8] Documentation: Update documentation for UIO_MEM_PHYS_CACHE and UIO_MEM_DEVICE

[Anup Patel]

From: Ankit Jindal <thatsjindal@gmail.com>

This patch updates UIO documentation for new mem region
types UIO_MEM_PHYS_CACHE and UIO_MEM_DEVICE.

Signed-off-by: Ankit Jindal <thatsjindal@gmail.com>
Signed-off-by: Anup Patel <anup.patel@broadcom.com>
---
 Documentation/DocBook/uio-howto.tmpl | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/Documentation/DocBook/uio-howto.tmpl b/Documentation/DocBook/uio-howto.tmpl
index cd0e452..de9dafe 100644
--- a/Documentation/DocBook/uio-howto.tmpl
+++ b/Documentation/DocBook/uio-howto.tmpl
@@ -529,8 +529,10 @@ the memory region, it will show up in the corresponding sysfs node.
 <varname>int memtype</varname>: Required if the mapping is used. Set this to
 <varname>UIO_MEM_PHYS</varname> if you you have physical memory on your
 card to be mapped. Use <varname>UIO_MEM_LOGICAL</varname> for logical
-memory (e.g. allocated with <function>kmalloc()</function>). There's also
-<varname>UIO_MEM_VIRTUAL</varname> for virtual memory.
+memory (e.g. allocated with <function>kmalloc()</function>). There are also
+<varname>UIO_MEM_VIRTUAL</varname> for virtual memory,
+<varname>UIO_MEM_PHYS_CACHE</varname> for cacheable physical memory and,
+<varname>UIO_MEM_DEVICE</varname> for memory mapped device registers.
 </para></listitem>
 
 <listitem><para>
-- 
1.9.1

[PATCH RESEND v2 6/8] uio: UIO_IRQ_NONE is a valid option for uioinfo->irq

[Anup Patel]

From: Jan Viktorin <viktorin@rehivetech.com>

We can simplify handling of platform_get_irq into one place as it is
acceptable to see UIO_IRQ_NONE instead of a valid IRQ number. Some
devices don't have or don't need any interrupt to be handled. The
same change has been already done for uio_pdrv_genirq.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 drivers/uio/uio_dmem_genirq.c | 17 ++++++-----------
 1 file changed, 6 insertions(+), 11 deletions(-)

diff --git a/drivers/uio/uio_dmem_genirq.c b/drivers/uio/uio_dmem_genirq.c
index e1134a4..a4d6d81 100644
--- a/drivers/uio/uio_dmem_genirq.c
+++ b/drivers/uio/uio_dmem_genirq.c
@@ -154,8 +154,6 @@ static int uio_dmem_genirq_probe(struct platform_device *pdev)
 	int i;
 
 	if (pdev->dev.of_node) {
-		int irq;
-
 		/* alloc uioinfo for one device */
 		uioinfo = kzalloc(sizeof(*uioinfo), GFP_KERNEL);
 		if (!uioinfo) {
@@ -165,13 +163,6 @@ static int uio_dmem_genirq_probe(struct platform_device *pdev)
 		}
 		uioinfo->name = pdev->dev.of_node->name;
 		uioinfo->version = "devicetree";
-
-		/* Multiple IRQs are not supported */
-		irq = platform_get_irq(pdev, 0);
-		if (irq == -ENXIO)
-			uioinfo->irq = UIO_IRQ_NONE;
-		else
-			uioinfo->irq = irq;
 	}
 
 	if (!uioinfo || !uioinfo->name || !uioinfo->version) {
@@ -200,14 +191,18 @@ static int uio_dmem_genirq_probe(struct platform_device *pdev)
 	priv->pdev = pdev;
 	mutex_init(&priv->alloc_lock);
 
+	/* Multiple IRQs are not supported */
 	if (!uioinfo->irq) {
 		ret = platform_get_irq(pdev, 0);
-		if (ret < 0) {
+		uioinfo->irq = ret;
+		if (ret == -ENXIO && pdev->dev.of_node)
+			uioinfo->irq = UIO_IRQ_NONE;
+		else if (ret < 0) {
 			dev_err(&pdev->dev, "failed to get IRQ\n");
 			goto bad1;
 		}
-		uioinfo->irq = ret;
 	}
+
 	uiomem = &uioinfo->mem[0];
 
 	for (i = 0; i < pdev->num_resources; ++i) {
-- 
1.9.1

[PATCH RESEND v2 7/8] uio: bind uio_dmem_genirq via OF

[Anup Patel]

From: Jan Viktorin <viktorin@rehivetech.com>

The uio_dmem_genirq works in a similar ways as uio_pdrv_genirq now.

It accepts the of_id module parameter to specify UIO compatible
string as module parameter. There are few other module parameters
to specify number bits in DMA mask and sizes dynamic regions.

Following are the newly added module parameters:
1) of_id: The UIO compatible string to be used for DT probing
2) dmem_dma_bits: This is the number of DMA mask bits for UIO device.
3) dmem_sizes: This is a formatted string specifying sizes of dynamic
required by some of the UIO devices.

Below are few examples of formatted string for dmem_sizes:
a) UIO dmem device with 3 dynamic regions:
uio_dmem_genirq.dmem_sizes=abc:4K,16K,4M
b) Two UIO dmem devices with different number of dynamic regions:
uio_dmem_genirq.dmem_sizes=abc:4K,16K,4M;xyz:8K

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
Signed-off-by: Anup Patel <anup.patel@broadcom.com>
---
 drivers/uio/uio_dmem_genirq.c | 162 +++++++++++++++++++++++++++++++++++-------
 1 file changed, 135 insertions(+), 27 deletions(-)

diff --git a/drivers/uio/uio_dmem_genirq.c b/drivers/uio/uio_dmem_genirq.c
index a4d6d81..1db2559 100644
--- a/drivers/uio/uio_dmem_genirq.c
+++ b/drivers/uio/uio_dmem_genirq.c
@@ -12,6 +12,27 @@
  * the Free Software Foundation.
  */
 
+/*
+ * For DeviceTree based platform devices, the compatible string, the DMA mask
+ * bits and sizes of dynamic regions are derived from kernel command-line.
+ *
+ * The format for specifying dynamic region sizes in kernel command line
+ * is as follows:
+ *
+ * uio_dmem_genirq.dmem_sizes := <uio_dmem_sizes_def>[;<uio_dmem_sizes_def>]
+ * <uio_dmem_sizes_def>       := <uio_name>:<size>[,<size>]
+ * <uio_name>                 := name as shown in /sys/class/uio/uioX/name
+ * <size>                     := standard linux memsize
+ *
+ * Examples:
+ *
+ * 1) UIO dmem device with 3 dynamic regions:
+ * uio_dmem_genirq.dmem_sizes=abc:4K,16K,4M
+ *
+ * 2) Two UIO dmem devices with different number of dynamic regions:
+ * uio_dmem_genirq.dmem_sizes=abc:4K,16K,4M;xyz:8K
+ */
+
 #include <linux/platform_device.h>
 #include <linux/uio_driver.h>
 #include <linux/spinlock.h>
@@ -144,46 +165,134 @@ static int uio_dmem_genirq_irqcontrol(struct uio_info *dev_info, s32 irq_on)
 	return 0;
 }
 
+static unsigned int uio_dmem_dma_bits = 32;
+static char uio_dmem_sizes[256];
+
+static int uio_dmem_genirq_alloc_platdata(struct platform_device *pdev)
+{
+	int ret, name_len;
+	u32 regions = 0;
+	char *s, *name, *sz;
+	char *sizes_str = NULL, *sizes_endstr = NULL;
+	struct uio_dmem_genirq_pdata pdata;
+	unsigned long long sizes[MAX_UIO_MAPS];
+
+	memset(&pdata, 0, sizeof(pdata));
+
+	/* Set DMA coherent mask */
+	if (uio_dmem_dma_bits > 64)
+		uio_dmem_dma_bits = 64;
+	ret = dma_set_coherent_mask(&pdev->dev,
+				    DMA_BIT_MASK(uio_dmem_dma_bits));
+	if (ret) {
+		dev_err(&pdev->dev, "Unable to dma_set_coherent_mask\n");
+		return ret;
+	}
+
+	/* Find-out start and end of sizes list */
+	s = uio_dmem_sizes;
+	while (*s != '\0') {
+		name = s;
+		s = strchr(s, ':');
+		if (!s)
+			break;
+		name_len = s - name;
+		s++;
+		if (*s == '\0')
+			break;
+		sz = s;
+		s = strchr(s, ';');
+		if (!s)
+			s = &uio_dmem_sizes[strlen(uio_dmem_sizes)];
+		if (strncmp(name, pdev->dev.of_node->name, name_len) == 0) {
+			sizes_str = sz;
+			sizes_endstr = s;
+			break;
+		}
+		s++;
+	}
+
+	/* Parse dynamic regions from sizes list */
+	regions = 0;
+	sizes[0] = 0;
+	while (sizes_str && sizes_endstr &&
+	       (sizes_str < sizes_endstr) &&
+	       (*sizes_str != '\0') &&
+	       (*sizes_str != ';') &&
+	       (regions < MAX_UIO_MAPS)) {
+		if (*sizes_str == ',') {
+			sizes_str++;
+			continue;
+		}
+		sizes[regions] = memparse(sizes_str, &sizes_str);
+		if (sizes[regions])
+			regions++;
+	}
+
+	/* Populate platform data */
+	pdata.num_dynamic_regions = regions;
+	pdata.dynamic_region_sizes = devm_kzalloc(&pdev->dev,
+			sizeof(*pdata.dynamic_region_sizes) *
+			pdata.num_dynamic_regions, GFP_KERNEL);
+	if (!pdata.dynamic_region_sizes) {
+		dev_err(&pdev->dev, "Unable to alloc dynamic_region_sizes\n");
+		ret = -ENOMEM;
+		return ret;
+	}
+
+	while (regions--)
+		pdata.dynamic_region_sizes[regions] = sizes[regions];
+
+	pdata.uioinfo.name = pdev->dev.of_node->name;
+	pdata.uioinfo.version = "devicetree";
+
+	return platform_device_add_data(pdev, &pdata, sizeof(pdata));
+}
+
 static int uio_dmem_genirq_probe(struct platform_device *pdev)
 {
-	struct uio_dmem_genirq_pdata *pdata = dev_get_platdata(&pdev->dev);
-	struct uio_info *uioinfo = &pdata->uioinfo;
-	struct uio_dmem_genirq_platdata *priv;
-	struct uio_mem *uiomem;
-	int ret = -EINVAL;
 	int i;
+	int ret = -EINVAL;
+	struct uio_mem *uiomem;
+	struct uio_info *uioinfo;
+	struct uio_dmem_genirq_pdata *pdata;
+	struct uio_dmem_genirq_platdata *priv;
 
 	if (pdev->dev.of_node) {
-		/* alloc uioinfo for one device */
-		uioinfo = kzalloc(sizeof(*uioinfo), GFP_KERNEL);
-		if (!uioinfo) {
-			ret = -ENOMEM;
-			dev_err(&pdev->dev, "unable to kmalloc\n");
+		ret = uio_dmem_genirq_alloc_platdata(pdev);
+		if (ret)
 			goto bad2;
-		}
-		uioinfo->name = pdev->dev.of_node->name;
-		uioinfo->version = "devicetree";
 	}
 
+	pdata = dev_get_platdata(&pdev->dev);
+	uioinfo = &pdata->uioinfo;
+
 	if (!uioinfo || !uioinfo->name || !uioinfo->version) {
 		dev_err(&pdev->dev, "missing platform_data\n");
-		goto bad0;
+		goto bad2;
 	}
 
 	if (uioinfo->handler || uioinfo->irqcontrol ||
 	    uioinfo->irq_flags & IRQF_SHARED) {
 		dev_err(&pdev->dev, "interrupt configuration error\n");
-		goto bad0;
+		goto bad2;
 	}
 
 	priv = kzalloc(sizeof(*priv), GFP_KERNEL);
 	if (!priv) {
 		ret = -ENOMEM;
 		dev_err(&pdev->dev, "unable to kmalloc\n");
-		goto bad0;
+		goto bad2;
 	}
 
-	dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32));
+	if (!pdev->dev.of_node) {
+		ret = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32));
+		if (ret) {
+			dev_err(&pdev->dev,
+				"unable to dma_set_coherent_mask\n");
+			goto bad1;
+		}
+	}
 
 	priv->uioinfo = uioinfo;
 	spin_lock_init(&priv->lock);
@@ -278,10 +387,6 @@ static int uio_dmem_genirq_probe(struct platform_device *pdev)
 	return 0;
  bad1:
 	kfree(priv);
- bad0:
-	/* kfree uioinfo for OF */
-	if (pdev->dev.of_node)
-		kfree(uioinfo);
  bad2:
 	return ret;
 }
@@ -296,10 +401,6 @@ static int uio_dmem_genirq_remove(struct platform_device *pdev)
 	priv->uioinfo->handler = NULL;
 	priv->uioinfo->irqcontrol = NULL;
 
-	/* kfree uioinfo for OF */
-	if (pdev->dev.of_node)
-		kfree(priv->uioinfo);
-
 	kfree(priv);
 	return 0;
 }
@@ -327,10 +428,17 @@ static const struct dev_pm_ops uio_dmem_genirq_dev_pm_ops = {
 };
 
 #ifdef CONFIG_OF
-static const struct of_device_id uio_of_genirq_match[] = {
-	{ /* empty for now */ },
+static struct of_device_id uio_of_genirq_match[] = {
+	{ /* This is filled with module_parm */ },
+	{ /* end of list */ },
 };
 MODULE_DEVICE_TABLE(of, uio_of_genirq_match);
+module_param_string(of_id, uio_of_genirq_match[0].compatible, 128, 0);
+MODULE_PARM_DESC(of_id, "Openfirmware id of the device to be handled by uio");
+module_param_named(dmem_dma_bits, uio_dmem_dma_bits, uint, 0);
+MODULE_PARM_DESC(dmem_dma_bits, "Number of bits in DMA mask");
+module_param_string(dmem_sizes, uio_dmem_sizes, 256, 0);
+MODULE_PARM_DESC(dmem_sizes, "Comma separated dynamic region sizes");
 #endif
 
 static struct platform_driver uio_dmem_genirq = {
-- 
1.9.1

[PATCH RESEND v2 8/8] uio: Use new memtypes in uio_dmem_genirq

[Anup Patel]

This patch extends uio_dmem_genirq driver to use recently
added memtypes as follows:
1. Use UIO_MEM_DEVICE memtype for MEM resources
2. Use UIO_MEM_PHYS_CACHE memtype for dynamic regions
when UIO DT node is marked as DMA coherent.

Signed-off-by: Anup Patel <anup.patel@broadcom.com>
---
 drivers/uio/uio_dmem_genirq.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/drivers/uio/uio_dmem_genirq.c b/drivers/uio/uio_dmem_genirq.c
index 1db2559..ce36afd 100644
--- a/drivers/uio/uio_dmem_genirq.c
+++ b/drivers/uio/uio_dmem_genirq.c
@@ -327,7 +327,7 @@ static int uio_dmem_genirq_probe(struct platform_device *pdev)
 			break;
 		}
 
-		uiomem->memtype = UIO_MEM_PHYS;
+		uiomem->memtype = UIO_MEM_DEVICE;
 		uiomem->addr = r->start;
 		uiomem->size = resource_size(r);
 		++uiomem;
@@ -343,7 +343,12 @@ static int uio_dmem_genirq_probe(struct platform_device *pdev)
 					" dynamic and fixed memory regions.\n");
 			break;
 		}
-		uiomem->memtype = UIO_MEM_PHYS;
+
+		if (pdev->dev.of_node &&
+		    of_dma_is_coherent(pdev->dev.of_node))
+			uiomem->memtype = UIO_MEM_PHYS_CACHE;
+		else
+			uiomem->memtype = UIO_MEM_PHYS;
 		uiomem->addr = DMEM_MAP_ERROR;
 		uiomem->size = pdata->dynamic_region_sizes[i];
 		++uiomem;
-- 
1.9.1

Re: [PATCH RESEND v2 0/8] Cache-coherent DMA access using UIO

[Arnd Bergmann]

On Monday, August 8, 2016 11:22:29 AM CEST Anup Patel wrote:
> The goal of this patchset is to improve UIO framework and UIO dmem
> driver to allow cache-coherent DMA accesses from user-space.
> 
> This patchset is based on two previous patchsets:
> 1) [PATCH v5 0/6] UIO driver for APM X-Gene QMTM
> (Refer, http://www.spinics.net/lists/devicetree/msg58244.html)
> 2) [PATCH 0/4] Fix and extend uio_dmem_genirq
> (Refer, https://lkml.org/lkml/2016/5/17/141)
> 
> We have adopted only patch0-3 of patchset1 which was abandoned
> long time back. We have taken care of last few unaddressed comments
> on these patches.
> 
> The patchset2 is quite recent has been adopted entirely. We have
> taken care review comments on these patches too.
> 
> This patchset is based on v4.7-rc7 tag and it is available in uio-v2
> branch of https://github.com/Broadcom/arm64-linux.git


UIO devices are generally meant to be things that do not
perform DMA and that don't screw up the rest of the system
when misused. A device that is able to access any physical
memory doesn't belong into this category. The way that
uio_dmem_genirq.c gets around this is by requiring the device
to be created by some code that sets up a separate IOMMU
domain first, but the DT probing here doesn't do that.
Note that IOMMU domains typically use 32-bit addressing,
so the entire "dma_mask from property" dance isn't even
required.

Also, this seems to duplicate a lot of the work that
went into "vfio". Can you explain why we need another way
of doing the same thing here?

	Arnd

Re: [PATCH RESEND v2 0/8] Cache-coherent DMA access using UIO

[Anup Patel]

Hi Arnd,

On Wed, Aug 10, 2016 at 9:25 PM, Arnd Bergmann <arnd-r2nGTMty4D4@public.gmane.org> wrote:
> On Monday, August 8, 2016 11:22:29 AM CEST Anup Patel wrote:
>> The goal of this patchset is to improve UIO framework and UIO dmem
>> driver to allow cache-coherent DMA accesses from user-space.
>>
>> This patchset is based on two previous patchsets:
>> 1) [PATCH v5 0/6] UIO driver for APM X-Gene QMTM
>> (Refer, http://www.spinics.net/lists/devicetree/msg58244.html)
>> 2) [PATCH 0/4] Fix and extend uio_dmem_genirq
>> (Refer, https://lkml.org/lkml/2016/5/17/141)
>>
>> We have adopted only patch0-3 of patchset1 which was abandoned
>> long time back. We have taken care of last few unaddressed comments
>> on these patches.
>>
>> The patchset2 is quite recent has been adopted entirely. We have
>> taken care review comments on these patches too.
>>
>> This patchset is based on v4.7-rc7 tag and it is available in uio-v2
>> branch of https://github.com/Broadcom/arm64-linux.git
>
>
> UIO devices are generally meant to be things that do not
> perform DMA and that don't screw up the rest of the system
> when misused. A device that is able to access any physical
> memory doesn't belong into this category. The way that
> uio_dmem_genirq.c gets around this is by requiring the device
> to be created by some code that sets up a separate IOMMU
> domain first, but the DT probing here doesn't do that.
> Note that IOMMU domains typically use 32-bit addressing,
> so the entire "dma_mask from property" dance isn't even
> required.

IMHO, UIO devices are meant for things that are not behind
any IOMMU hardware.

Yes, any mis-programming in user space using UIO can
potentially screw-up the rest of the system but this is
generally a known/assumed fact for people who are using UIO.

>
> Also, this seems to duplicate a lot of the work that
> went into "vfio". Can you explain why we need another way
> of doing the same thing here?

We can only use "vfio" for devices that are behind some
kind of IOMMU (Right??). For devices not having IOMMU
support will have to use UIO for user space access.

Particularly, there are lot of FPGA-based solutions and legacy
hardware which do not have IOMMU support (devices on
FPGA or specific devices).

In our use case, we have some FPGA-based device which
does not have IOMMU support and we are accessing this
FPGA-based device from user-space.

This patchset only tries to extend "uio" and "uio_dmem_genirq".
There is no intention of duplicating what has been already
done for "vfio".

I do agree that "vfio" should eventually become defacto method
of accessing devices in user space but that requires devices to
always have IOMMU support.

Regards,
Anup
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH RESEND v2 0/8] Cache-coherent DMA access using UIO

[Alex Williamson]

On Thu, 11 Aug 2016 09:30:19 +0530
Anup Patel <anup.patel@broadcom.com> wrote:

> Hi Arnd,
> 
> On Wed, Aug 10, 2016 at 9:25 PM, Arnd Bergmann <arnd@arndb.de> wrote:
> > On Monday, August 8, 2016 11:22:29 AM CEST Anup Patel wrote:  
> >> The goal of this patchset is to improve UIO framework and UIO dmem
> >> driver to allow cache-coherent DMA accesses from user-space.
> >>
> >> This patchset is based on two previous patchsets:
> >> 1) [PATCH v5 0/6] UIO driver for APM X-Gene QMTM
> >> (Refer, http://www.spinics.net/lists/devicetree/msg58244.html)
> >> 2) [PATCH 0/4] Fix and extend uio_dmem_genirq
> >> (Refer, https://lkml.org/lkml/2016/5/17/141)
> >>
> >> We have adopted only patch0-3 of patchset1 which was abandoned
> >> long time back. We have taken care of last few unaddressed comments
> >> on these patches.
> >>
> >> The patchset2 is quite recent has been adopted entirely. We have
> >> taken care review comments on these patches too.
> >>
> >> This patchset is based on v4.7-rc7 tag and it is available in uio-v2
> >> branch of https://github.com/Broadcom/arm64-linux.git  
> >
> >
> > UIO devices are generally meant to be things that do not
> > perform DMA and that don't screw up the rest of the system
> > when misused. A device that is able to access any physical
> > memory doesn't belong into this category. The way that
> > uio_dmem_genirq.c gets around this is by requiring the device
> > to be created by some code that sets up a separate IOMMU
> > domain first, but the DT probing here doesn't do that.
> > Note that IOMMU domains typically use 32-bit addressing,
> > so the entire "dma_mask from property" dance isn't even
> > required.  
> 
> IMHO, UIO devices are meant for things that are not behind
> any IOMMU hardware.
> 
> Yes, any mis-programming in user space using UIO can
> potentially screw-up the rest of the system but this is
> generally a known/assumed fact for people who are using UIO.
> 
> >
> > Also, this seems to duplicate a lot of the work that
> > went into "vfio". Can you explain why we need another way
> > of doing the same thing here?  
> 
> We can only use "vfio" for devices that are behind some
> kind of IOMMU (Right??). For devices not having IOMMU
> support will have to use UIO for user space access.
> 
> Particularly, there are lot of FPGA-based solutions and legacy
> hardware which do not have IOMMU support (devices on
> FPGA or specific devices).
> 
> In our use case, we have some FPGA-based device which
> does not have IOMMU support and we are accessing this
> FPGA-based device from user-space.
> 
> This patchset only tries to extend "uio" and "uio_dmem_genirq".
> There is no intention of duplicating what has been already
> done for "vfio".
> 
> I do agree that "vfio" should eventually become defacto method
> of accessing devices in user space but that requires devices to
> always have IOMMU support.

A vfio no-iommu mode exists since v4.5:

http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=03a76b60f8ba27974e2d252bc555d2c103420e15

Thanks,
Alex

Re: [PATCH RESEND v2 0/8] Cache-coherent DMA access using UIO

[Anup Patel]

Hi Alex,

On Thu, Aug 11, 2016 at 9:46 AM, Alex Williamson
<alex.williamson@redhat.com> wrote:
> On Thu, 11 Aug 2016 09:30:19 +0530
> Anup Patel <anup.patel@broadcom.com> wrote:
>
>> Hi Arnd,
>>
>> On Wed, Aug 10, 2016 at 9:25 PM, Arnd Bergmann <arnd@arndb.de> wrote:
>> > On Monday, August 8, 2016 11:22:29 AM CEST Anup Patel wrote:
>> >> The goal of this patchset is to improve UIO framework and UIO dmem
>> >> driver to allow cache-coherent DMA accesses from user-space.
>> >>
>> >> This patchset is based on two previous patchsets:
>> >> 1) [PATCH v5 0/6] UIO driver for APM X-Gene QMTM
>> >> (Refer, http://www.spinics.net/lists/devicetree/msg58244.html)
>> >> 2) [PATCH 0/4] Fix and extend uio_dmem_genirq
>> >> (Refer, https://lkml.org/lkml/2016/5/17/141)
>> >>
>> >> We have adopted only patch0-3 of patchset1 which was abandoned
>> >> long time back. We have taken care of last few unaddressed comments
>> >> on these patches.
>> >>
>> >> The patchset2 is quite recent has been adopted entirely. We have
>> >> taken care review comments on these patches too.
>> >>
>> >> This patchset is based on v4.7-rc7 tag and it is available in uio-v2
>> >> branch of https://github.com/Broadcom/arm64-linux.git
>> >
>> >
>> > UIO devices are generally meant to be things that do not
>> > perform DMA and that don't screw up the rest of the system
>> > when misused. A device that is able to access any physical
>> > memory doesn't belong into this category. The way that
>> > uio_dmem_genirq.c gets around this is by requiring the device
>> > to be created by some code that sets up a separate IOMMU
>> > domain first, but the DT probing here doesn't do that.
>> > Note that IOMMU domains typically use 32-bit addressing,
>> > so the entire "dma_mask from property" dance isn't even
>> > required.
>>
>> IMHO, UIO devices are meant for things that are not behind
>> any IOMMU hardware.
>>
>> Yes, any mis-programming in user space using UIO can
>> potentially screw-up the rest of the system but this is
>> generally a known/assumed fact for people who are using UIO.
>>
>> >
>> > Also, this seems to duplicate a lot of the work that
>> > went into "vfio". Can you explain why we need another way
>> > of doing the same thing here?
>>
>> We can only use "vfio" for devices that are behind some
>> kind of IOMMU (Right??). For devices not having IOMMU
>> support will have to use UIO for user space access.
>>
>> Particularly, there are lot of FPGA-based solutions and legacy
>> hardware which do not have IOMMU support (devices on
>> FPGA or specific devices).
>>
>> In our use case, we have some FPGA-based device which
>> does not have IOMMU support and we are accessing this
>> FPGA-based device from user-space.
>>
>> This patchset only tries to extend "uio" and "uio_dmem_genirq".
>> There is no intention of duplicating what has been already
>> done for "vfio".
>>
>> I do agree that "vfio" should eventually become defacto method
>> of accessing devices in user space but that requires devices to
>> always have IOMMU support.
>
> A vfio no-iommu mode exists since v4.5:
>
> http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=03a76b60f8ba27974e2d252bc555d2c103420e15

It's good that we have "No-IOMMU" mode in VFIO now but I see
that it's for vfio_pci only. Has this been extended for vfio_platform?

We would certainly like to move to VFIO if "No-IOMMU" mode
is available for vfio_platform devices because in-our use case
devices are memory-mapped FPGA devices.

Meanwhile, I would like to have this patchset for benefit of
people who already using UIO.

Regards,
Anup

Re: [PATCH RESEND v2 0/8] Cache-coherent DMA access using UIO

[Alex Williamson]

On Thu, 11 Aug 2016 10:06:19 +0530
Anup Patel <anup.patel@broadcom.com> wrote:

> Hi Alex,
> 
> On Thu, Aug 11, 2016 at 9:46 AM, Alex Williamson
> <alex.williamson@redhat.com> wrote:
> > On Thu, 11 Aug 2016 09:30:19 +0530
> > Anup Patel <anup.patel@broadcom.com> wrote:
> >  
> >> Hi Arnd,
> >>
> >> On Wed, Aug 10, 2016 at 9:25 PM, Arnd Bergmann <arnd@arndb.de> wrote:  
> >> > On Monday, August 8, 2016 11:22:29 AM CEST Anup Patel wrote:  
> >> >> The goal of this patchset is to improve UIO framework and UIO dmem
> >> >> driver to allow cache-coherent DMA accesses from user-space.
> >> >>
> >> >> This patchset is based on two previous patchsets:
> >> >> 1) [PATCH v5 0/6] UIO driver for APM X-Gene QMTM
> >> >> (Refer, http://www.spinics.net/lists/devicetree/msg58244.html)
> >> >> 2) [PATCH 0/4] Fix and extend uio_dmem_genirq
> >> >> (Refer, https://lkml.org/lkml/2016/5/17/141)
> >> >>
> >> >> We have adopted only patch0-3 of patchset1 which was abandoned
> >> >> long time back. We have taken care of last few unaddressed comments
> >> >> on these patches.
> >> >>
> >> >> The patchset2 is quite recent has been adopted entirely. We have
> >> >> taken care review comments on these patches too.
> >> >>
> >> >> This patchset is based on v4.7-rc7 tag and it is available in uio-v2
> >> >> branch of https://github.com/Broadcom/arm64-linux.git  
> >> >
> >> >
> >> > UIO devices are generally meant to be things that do not
> >> > perform DMA and that don't screw up the rest of the system
> >> > when misused. A device that is able to access any physical
> >> > memory doesn't belong into this category. The way that
> >> > uio_dmem_genirq.c gets around this is by requiring the device
> >> > to be created by some code that sets up a separate IOMMU
> >> > domain first, but the DT probing here doesn't do that.
> >> > Note that IOMMU domains typically use 32-bit addressing,
> >> > so the entire "dma_mask from property" dance isn't even
> >> > required.  
> >>
> >> IMHO, UIO devices are meant for things that are not behind
> >> any IOMMU hardware.
> >>
> >> Yes, any mis-programming in user space using UIO can
> >> potentially screw-up the rest of the system but this is
> >> generally a known/assumed fact for people who are using UIO.
> >>  
> >> >
> >> > Also, this seems to duplicate a lot of the work that
> >> > went into "vfio". Can you explain why we need another way
> >> > of doing the same thing here?  
> >>
> >> We can only use "vfio" for devices that are behind some
> >> kind of IOMMU (Right??). For devices not having IOMMU
> >> support will have to use UIO for user space access.
> >>
> >> Particularly, there are lot of FPGA-based solutions and legacy
> >> hardware which do not have IOMMU support (devices on
> >> FPGA or specific devices).
> >>
> >> In our use case, we have some FPGA-based device which
> >> does not have IOMMU support and we are accessing this
> >> FPGA-based device from user-space.
> >>
> >> This patchset only tries to extend "uio" and "uio_dmem_genirq".
> >> There is no intention of duplicating what has been already
> >> done for "vfio".
> >>
> >> I do agree that "vfio" should eventually become defacto method
> >> of accessing devices in user space but that requires devices to
> >> always have IOMMU support.  
> >
> > A vfio no-iommu mode exists since v4.5:
> >
> > http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=03a76b60f8ba27974e2d252bc555d2c103420e15  
> 
> It's good that we have "No-IOMMU" mode in VFIO now but I see
> that it's for vfio_pci only. Has this been extended for vfio_platform?

Yes, it went in during the most recent merge window for v4.8:

9698cbf vfio: platform: support No-IOMMU mode
 
> We would certainly like to move to VFIO if "No-IOMMU" mode
> is available for vfio_platform devices because in-our use case
> devices are memory-mapped FPGA devices.
> 
> Meanwhile, I would like to have this patchset for benefit of
> people who already using UIO.
> 
> Regards,
> Anup