[PATCH v2 0/4] Fix kernel panic in dma-coherent allocations

[PATCH v2 0/4] Fix kernel panic in dma-coherent allocations

[Brian Starkey]

Hi,

These patches implement a MEMREMAP_WC flag for memremap(), which can
be used to obtain writecombine mappings. This is then used for setting
up dma_coherent_mem regions which use the DMA_MEMORY_MAP flag.

The motivation is to fix an alignment fault on arm64, and the
suggestion to implement MEMREMAP_WC for this case was made at [1].
That particular issue is handled in patch 4, which makes sure that the
appropriate memset function is used when zeroing allocations mapped as
IO memory.

Best Regards,
Brian

[1] http://lists.infradead.org/pipermail/linux-arm-kernel/2015-December/390857.html

Changes since v1:
* Added preparatory patch removing flag modifications in memremap()
  (Suggested by Andrew Morton)
* Rebase onto linux-next

Brian Starkey (4):
  memremap: don't modify flags
  memremap: add MEMREMAP_WC flag.
  drivers: dma-coherent: use MEMREMAP_WC for DMA_MEMORY_MAP
  drivers: dma-coherent: use memset_io for DMA_MEMORY_IO mappings

 drivers/base/dma-coherent.c |   25 ++++++++++++++++++++-----
 include/linux/io.h          |    1 +
 kernel/memremap.c           |   24 +++++++++++++++---------
 3 files changed, 36 insertions(+), 14 deletions(-)

-- 
1.7.9.5

[PATCH v2 1/4] memremap: don't modify flags

[Brian Starkey]

Don't modify the flags input argument to memremap(). MEMREMAP_WB is
already a special case so we can check for it directly instead of
clearing flag bits in each mapper.

Signed-off-by: Brian Starkey <brian.starkey@arm.com>
---
 kernel/memremap.c |   14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/kernel/memremap.c b/kernel/memremap.c
index 59c55af..92adb19 100644
--- a/kernel/memremap.c
+++ b/kernel/memremap.c
@@ -64,6 +64,9 @@ void *memremap(resource_size_t offset, size_t size, unsigned long flags)
 				       IORESOURCE_SYSTEM_RAM, IORES_DESC_NONE);
 	void *addr = NULL;
 
+	if (!flags)
+		return NULL;
+
 	if (is_ram == REGION_MIXED) {
 		WARN_ONCE(1, "memremap attempted on mixed range %pa size: %#lx\n",
 				&offset, (unsigned long) size);
@@ -72,7 +75,6 @@ void *memremap(resource_size_t offset, size_t size, unsigned long flags)
 
 	/* Try all mapping types requested until one returns non-NULL */
 	if (flags & MEMREMAP_WB) {
-		flags &= ~MEMREMAP_WB;
 		/*
 		 * MEMREMAP_WB is special in that it can be satisifed
 		 * from the direct map.  Some archs depend on the
@@ -86,21 +88,19 @@ void *memremap(resource_size_t offset, size_t size, unsigned long flags)
 	}
 
 	/*
-	 * If we don't have a mapping yet and more request flags are
-	 * pending then we will be attempting to establish a new virtual
+	 * If we don't have a mapping yet and other request flags are
+	 * present then we will be attempting to establish a new virtual
 	 * address mapping.  Enforce that this mapping is not aliasing
 	 * System RAM.
 	 */
-	if (!addr && is_ram == REGION_INTERSECTS && flags) {
+	if (!addr && is_ram == REGION_INTERSECTS && flags != MEMREMAP_WB) {
 		WARN_ONCE(1, "memremap attempted on ram %pa size: %#lx\n",
 				&offset, (unsigned long) size);
 		return NULL;
 	}
 
-	if (!addr && (flags & MEMREMAP_WT)) {
-		flags &= ~MEMREMAP_WT;
+	if (!addr && (flags & MEMREMAP_WT))
 		addr = ioremap_wt(offset, size);
-	}
 
 	return addr;
 }
-- 
1.7.9.5

[PATCH v2 2/4] memremap: add MEMREMAP_WC flag.

[Brian Starkey]

Add a flag to memremap() for writecombine mappings. Mappings satisfied
by this flag will not be cached, however writes may be delayed or
combined into more efficient bursts. This is most suitable for
buffers written sequentially by the CPU for use by other DMA devices.

Signed-off-by: Brian Starkey <brian.starkey@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
---
 include/linux/io.h |    1 +
 kernel/memremap.c  |   13 +++++++++++--
 2 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/include/linux/io.h b/include/linux/io.h
index 32403b5..e2c8419 100644
--- a/include/linux/io.h
+++ b/include/linux/io.h
@@ -135,6 +135,7 @@ enum {
 	/* See memremap() kernel-doc for usage description... */
 	MEMREMAP_WB = 1 << 0,
 	MEMREMAP_WT = 1 << 1,
+	MEMREMAP_WC = 1 << 2,
 };
 
 void *memremap(resource_size_t offset, size_t size, unsigned long flags);
diff --git a/kernel/memremap.c b/kernel/memremap.c
index 92adb19..03ce593 100644
--- a/kernel/memremap.c
+++ b/kernel/memremap.c
@@ -41,11 +41,13 @@ static void *try_ram_remap(resource_size_t offset, size_t size)
  * memremap() - remap an iomem_resource as cacheable memory
  * @offset: iomem resource start address
  * @size: size of remap
- * @flags: either MEMREMAP_WB or MEMREMAP_WT
+ * @flags: any of MEMREMAP_WB, MEMREMAP_WT and MEMREMAP_WC
  *
  * memremap() is "ioremap" for cases where it is known that the resource
  * being mapped does not have i/o side effects and the __iomem
- * annotation is not applicable.
+ * annotation is not applicable. In the case of multiple flags, the different
+ * mapping types will be attempted in the order listed below until one of
+ * them succeeds.
  *
  * MEMREMAP_WB - matches the default mapping for System RAM on
  * the architecture.  This is usually a read-allocate write-back cache.
@@ -57,6 +59,10 @@ static void *try_ram_remap(resource_size_t offset, size_t size)
  * cache or are written through to memory and never exist in a
  * cache-dirty state with respect to program visibility.  Attempts to
  * map System RAM with this mapping type will fail.
+ *
+ * MEMREMAP_WC - establish a writecombine mapping, whereby writes may
+ * be coalesced together (e.g. in the CPU's write buffers), but is otherwise
+ * uncached. Attempts to map System RAM with this mapping type will fail.
  */
 void *memremap(resource_size_t offset, size_t size, unsigned long flags)
 {
@@ -102,6 +108,9 @@ void *memremap(resource_size_t offset, size_t size, unsigned long flags)
 	if (!addr && (flags & MEMREMAP_WT))
 		addr = ioremap_wt(offset, size);
 
+	if (!addr && (flags & MEMREMAP_WC))
+		addr = ioremap_wc(offset, size);
+
 	return addr;
 }
 EXPORT_SYMBOL(memremap);
-- 
1.7.9.5

[PATCH v2 3/4] drivers: dma-coherent: use MEMREMAP_WC for DMA_MEMORY_MAP

[Brian Starkey]

When the DMA_MEMORY_MAP flag is used, memory which can be accessed
directly should be returned, so use memremap(..., MEMREMAP_WC) to
provide a writecombine mapping.

Signed-off-by: Brian Starkey <brian.starkey@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
---
 drivers/base/dma-coherent.c |   20 ++++++++++++++++----
 1 file changed, 16 insertions(+), 4 deletions(-)

diff --git a/drivers/base/dma-coherent.c b/drivers/base/dma-coherent.c
index 87b8083..25bb398 100644
--- a/drivers/base/dma-coherent.c
+++ b/drivers/base/dma-coherent.c
@@ -2,6 +2,7 @@
  * Coherent per-device memory handling.
  * Borrowed from i386
  */
+#include <linux/io.h>
 #include <linux/slab.h>
 #include <linux/kernel.h>
 #include <linux/module.h>
@@ -31,7 +32,10 @@ static bool dma_init_coherent_memory(
 	if (!size)
 		goto out;
 
-	mem_base = ioremap(phys_addr, size);
+	if (flags & DMA_MEMORY_MAP)
+		mem_base = memremap(phys_addr, size, MEMREMAP_WC);
+	else
+		mem_base = ioremap(phys_addr, size);
 	if (!mem_base)
 		goto out;
 
@@ -54,8 +58,12 @@ static bool dma_init_coherent_memory(
 
 out:
 	kfree(dma_mem);
-	if (mem_base)
-		iounmap(mem_base);
+	if (mem_base) {
+		if (flags & DMA_MEMORY_MAP)
+			memunmap(mem_base);
+		else
+			iounmap(mem_base);
+	}
 	return false;
 }
 
@@ -63,7 +71,11 @@ static void dma_release_coherent_memory(struct dma_coherent_mem *mem)
 {
 	if (!mem)
 		return;
-	iounmap(mem->virt_base);
+
+	if (mem->flags & DMA_MEMORY_MAP)
+		memunmap(mem->virt_base);
+	else
+		iounmap(mem->virt_base);
 	kfree(mem->bitmap);
 	kfree(mem);
 }
-- 
1.7.9.5

[PATCH v2 4/4] drivers: dma-coherent: use memset_io for DMA_MEMORY_IO mappings

[Brian Starkey]

Use memset_io() for DMA_MEMORY_IO mappings which are mapped as I/O
memory, and regular memset() for DMA_MEMORY_MAP mappings.

This fixes the below alignment fault on arm64 for DMA_MEMORY_IO
mappings, where memset() uses the DC ZVA instruction which is
invalid on device memory.

   Unhandled fault: alignment fault (0x96000061) at 0xffffff8000380000
   Internal error: : 96000061 [#1] PREEMPT SMP
   Modules linked in: hdlcd(+) clk_scpi
   CPU: 4 PID: 1355 Comm: systemd-udevd Not tainted 4.4.0-rc1+ #5
   Hardware name: ARM Juno development board (r0) (DT)
   task: ffffffc9763eee00 ti: ffffffc9758c4000 task.ti: ffffffc9758c4000
   PC is at __efistub_memset+0x1ac/0x200
   LR is at dma_alloc_from_coherent+0xb0/0x120
   pc : [<ffffffc00030ff2c>] lr : [<ffffffc00042a918>] pstate: 400001c5
   sp : ffffffc9758c79a0
   x29: ffffffc9758c79a0 x28: ffffffc000635cd0
   x27: 0000000000000124 x26: ffffffc000119ef4
   x25: 0000000000010000 x24: 0000000000000140
   x23: ffffffc07e9ac3a8 x22: ffffffc9758c7a58
   x21: ffffffc9758c7a68 x20: 0000000000000004
   x19: ffffffc07e9ac380 x18: 0000000000000001
   x17: 0000007fae1bbba8 x16: ffffffc0001b2d1c
   x15: ffffffffffffffff x14: 0ffffffffffffffe
   x13: 0000000000000010 x12: ffffff800837ffff
   x11: ffffff800837ffff x10: 0000000040000000
   x9 : 0000000000000000 x8 : ffffff8000380000
   x7 : 0000000000000000 x6 : 000000000000003f
   x5 : 0000000000000040 x4 : 0000000000000000
   x3 : 0000000000000004 x2 : 000000000000ffc0
   x1 : 0000000000000000 x0 : ffffff8000380000

Signed-off-by: Brian Starkey <brian.starkey@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
---
 drivers/base/dma-coherent.c |    5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/base/dma-coherent.c b/drivers/base/dma-coherent.c
index 25bb398..bdf28f7 100644
--- a/drivers/base/dma-coherent.c
+++ b/drivers/base/dma-coherent.c
@@ -187,7 +187,10 @@ int dma_alloc_from_coherent(struct device *dev, ssize_t size,
 	 */
 	*dma_handle = mem->device_base + (pageno << PAGE_SHIFT);
 	*ret = mem->virt_base + (pageno << PAGE_SHIFT);
-	memset(*ret, 0, size);
+	if (mem->flags & DMA_MEMORY_MAP)
+		memset(*ret, 0, size);
+	else
+		memset_io(*ret, 0, size);
 	spin_unlock_irqrestore(&mem->spinlock, flags);
 
 	return 1;
-- 
1.7.9.5

[PATCH v2 4/4] drivers: dma-coherent: use memset_io for DMA_MEMORY_IO mappings

[Andrew Morton]

On Mon, 29 Feb 2016 16:09:24 +0000 Brian Starkey <brian.starkey@arm.com> wrote:

> Use memset_io() for DMA_MEMORY_IO mappings which are mapped as I/O
> memory, and regular memset() for DMA_MEMORY_MAP mappings.
> 
> This fixes the below alignment fault on arm64 for DMA_MEMORY_IO
> mappings, where memset() uses the DC ZVA instruction which is
> invalid on device memory.

What's the urgency of this fix?  "Hair on fire needed in stable asap"
or "Nice to have in there for 4.6" or what?

[PATCH v2 4/4] drivers: dma-coherent: use memset_io for DMA_MEMORY_IO mappings

[Brian Starkey]

Hi Andrew,

On Mon, Feb 29, 2016 at 03:17:49PM -0800, Andrew Morton wrote:
>On Mon, 29 Feb 2016 16:09:24 +0000 Brian Starkey <brian.starkey@arm.com> wrote:
>
>> Use memset_io() for DMA_MEMORY_IO mappings which are mapped as I/O
>> memory, and regular memset() for DMA_MEMORY_MAP mappings.
>>
>> This fixes the below alignment fault on arm64 for DMA_MEMORY_IO
>> mappings, where memset() uses the DC ZVA instruction which is
>> invalid on device memory.
>
>What's the urgency of this fix?  "Hair on fire needed in stable asap"
>or "Nice to have in there for 4.6" or what?
>

No-one else is complaining so probably not the former. If it could
make 4.6 though that would be grand.

As for stable, anything before 3.15 doesn't hit the fault (on arm64)
because memset() is different, so probably not needed there.
For anything after that this patch in isolation isn't a full fix,
because dma_init_coherent_memory() will still use the wrong mapping
function for DMA_MEMORY_MAP. In that case, I think it needs to be the
whole series or nothing. I don't have a strong opinion either way, but
perhaps someone else does.

Thanks
Brian