Linux MIPS Architecture development
 help / color / mirror / Atom feed
* [PATCH v3 0/7] dma-mapping: DMA_ATTR_NO_KERNEL_MAPPING related fixes
       [not found] <CGME20261119151527eucas1p16c69a0580b2caabe66f952572362021f@eucas1p1.samsung.com>
@ 2026-11-19 15:15 ` Marek Szyprowski
  2026-11-19 15:15   ` [PATCH v3 1/8] xtensa: write back dirty lines in arch_dma_prep_coherent() Marek Szyprowski
                     ` (8 more replies)
  0 siblings, 9 replies; 12+ messages in thread
From: Marek Szyprowski @ 2026-11-19 15:15 UTC (permalink / raw)
  To: linux-mips, linuxppc-dev, iommu, linux-kernel
  Cc: Marek Szyprowski, Robin Murphy, Christoph Hellwig,
	Thomas Bogendoerfer, Madhavan Srinivasan, Michael Ellerman,
	Nicholas Piggin, Christophe Leroy (CS GROUP),
	Ritesh Harjani (IBM), Shrikanth Hegde, Chris Zankel, Max Filippov

Hi All,

arch_dma_prep_coherent() is called by the generic DMA code on pages
which may be located in highmem: dma_direct_alloc() forces the remap
path for a highmem page returned from CMA and calls this hook on it
before creating the uncached mapping. However, the hook has never been
documented and some implementations assume that the page has a kernel
linear mapping. This issue affects MIPS, xtensa and PowerPC. It gets
fixed by patches #2-#5.

xtensa also only invalidates the cache, which discards the zeroes
written by the dma_direct_alloc(). Patch #1 switches it to a
writeback+invalidate.

Patch #6 documents that arch_dma_prep_coherent() must write back dirty
lines and must accept highmem pages. With that in place, patch #7 drops
the PageHighMem() check in dma_direct_alloc_no_mapping(), which so far
skipped cache maintenance for highmem CMA pages allocated with
DMA_ATTR_NO_KERNEL_MAPPING.

Patch #8 makes dma_direct_alloc_no_mapping() always return zeroed
memory, like all other dma_alloc_* paths. Passing __GFP_ZERO is not
enough there, because CMA ignores it.

The analysis and the patchset was prepared with the help of an LLM. The
code compiles fine, but it has not been runtime tested on any of the
affected architectures yet. Testing on real hardware would be
appreciated. I've reviewed the resulting changes.

I plan to merge this via my dma-mapping tree, I can provide a tag if
needed for merging it also to arch code.

Best regards
Marek Szyprowski, PhD
Samsung R&D Institute Poland


Changelog:
v3:
- added patch improving lowmem handling in __dma_sync_page() (PowerPC
  arch) to avoid performance regression in latter patches
- added fixes tag to "powerpc/mm: handle highmem pages in
  arch_dma_prep_coherent()" patch
- collected tags

v2: https://lore.kernel.org/all/20261110094830.2081847-1-m.szyprowski@samsung.com/
- fixed minor issues in comments and patch descriptions pointed by sashiko
- dropped RFC annotation

v1: https://lore.kernel.org/all/20261106225628.215319-1-m.szyprowski@samsung.com/


Patch summary:

Marek Szyprowski (8):
  xtensa: write back dirty lines in arch_dma_prep_coherent()
  xtensa: handle highmem pages in arch_dma_prep_coherent()
  MIPS: mm: handle highmem pages in arch_dma_prep_coherent()
  powerpc/mm: skip kmap for lowmem pages in __dma_sync_page()
  powerpc/mm: handle highmem pages in arch_dma_prep_coherent()
  dma-mapping: document that arch_dma_prep_coherent() must handle
    highmem
  dma-direct: do not skip cache maintenance for highmem pages in
    no-mapping allocations
  dma-direct: zero memory allocated with DMA_ATTR_NO_KERNEL_MAPPING

 arch/mips/mm/dma-noncoherent.c    | 10 +++++-----
 arch/powerpc/mm/dma-noncoherent.c | 16 +++++++++-------
 arch/xtensa/kernel/pci-dma.c      | 12 +++++++++++-
 include/linux/dma-map-ops.h       | 11 +++++++++++
 kernel/dma/direct.c               |  9 +++++++--
 5 files changed, 43 insertions(+), 15 deletions(-)

-- 
2.43.0


^ permalink raw reply	[flat|nested] 12+ messages in thread

* [PATCH v3 1/8] xtensa: write back dirty lines in arch_dma_prep_coherent()
  2026-11-19 15:15 ` [PATCH v3 0/7] dma-mapping: DMA_ATTR_NO_KERNEL_MAPPING related fixes Marek Szyprowski
@ 2026-11-19 15:15   ` Marek Szyprowski
  2026-11-19 15:15   ` [PATCH v3 2/8] xtensa: handle highmem pages " Marek Szyprowski
                     ` (7 subsequent siblings)
  8 siblings, 0 replies; 12+ messages in thread
From: Marek Szyprowski @ 2026-11-19 15:15 UTC (permalink / raw)
  To: linux-mips, linuxppc-dev, iommu, linux-kernel
  Cc: Marek Szyprowski, Robin Murphy, Christoph Hellwig,
	Thomas Bogendoerfer, Madhavan Srinivasan, Michael Ellerman,
	Nicholas Piggin, Christophe Leroy (CS GROUP),
	Ritesh Harjani (IBM), Shrikanth Hegde, Chris Zankel, Max Filippov

arch_dma_prep_coherent() is expected to remove any dirty cache lines
covering the buffer, so that they are neither lost nor evicted later on
top of data written through the uncached alias or by the device.
xtensa implements it with __invalidate_dcache_range(), which uses the
DHI instruction and discards dirty lines without writing them back.

This loses data written through the cached kernel mapping right before
the call. dma_direct_alloc() zeroes the buffer with memset() on the
cached linear mapping and only then calls arch_dma_prep_coherent() and
switches to the KSEG bypass alias via arch_dma_set_uncached(). On cores
with a write-back data cache the zeroes still sitting in the cache are
dropped, and dma_alloc_coherent() returns a buffer with stale content.

Use __flush_invalidate_dcache_range() (DHWBI) instead, which writes the
dirty lines back before invalidating them. On cores with a write-through
data cache it maps to __invalidate_dcache_range(), so nothing changes
there.

Fixes: 0f665b9e2a6d ("xtensa: use the generic uncached segment support")
Reviewed-by: Max Filippov <jcmvbkbc@gmail.com>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
---
 arch/xtensa/kernel/pci-dma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/xtensa/kernel/pci-dma.c b/arch/xtensa/kernel/pci-dma.c
index 94955caa4488..e1c00ec37eed 100644
--- a/arch/xtensa/kernel/pci-dma.c
+++ b/arch/xtensa/kernel/pci-dma.c
@@ -82,7 +82,7 @@ void arch_sync_dma_for_device(phys_addr_t paddr, size_t size,
 
 void arch_dma_prep_coherent(struct page *page, size_t size)
 {
-	__invalidate_dcache_range((unsigned long)page_address(page), size);
+	__flush_invalidate_dcache_range((unsigned long)page_address(page), size);
 }
 
 /*
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v3 2/8] xtensa: handle highmem pages in arch_dma_prep_coherent()
  2026-11-19 15:15 ` [PATCH v3 0/7] dma-mapping: DMA_ATTR_NO_KERNEL_MAPPING related fixes Marek Szyprowski
  2026-11-19 15:15   ` [PATCH v3 1/8] xtensa: write back dirty lines in arch_dma_prep_coherent() Marek Szyprowski
@ 2026-11-19 15:15   ` Marek Szyprowski
  2026-11-19 15:15   ` [PATCH v3 3/8] MIPS: mm: " Marek Szyprowski
                     ` (6 subsequent siblings)
  8 siblings, 0 replies; 12+ messages in thread
From: Marek Szyprowski @ 2026-11-19 15:15 UTC (permalink / raw)
  To: linux-mips, linuxppc-dev, iommu, linux-kernel
  Cc: Marek Szyprowski, Robin Murphy, Christoph Hellwig,
	Thomas Bogendoerfer, Madhavan Srinivasan, Michael Ellerman,
	Nicholas Piggin, Christophe Leroy (CS GROUP),
	Ritesh Harjani (IBM), Shrikanth Hegde, Chris Zankel, Max Filippov

arch_dma_prep_coherent() performs the cache maintenance on the address
returned by page_address(). xtensa with CONFIG_HIGHMEM uses
HASHED_PAGE_VIRTUAL, so page_address() returns NULL for a highmem page
that is not kmapped, and the cache operation then works on the virtual
range starting at 0 instead of the buffer, leaving any dirty cache lines
of the buffer in place.

Highmem pages do reach this function: dma_alloc_contiguous() can return
a highmem CMA page, which dma_direct_alloc() handles by forcing the remap
path and calling arch_dma_prep_coherent() on that page before creating
the uncached mapping.

Fix this by reusing do_cache_op(), which the arch_sync_dma_for_*() hooks
already use and which maps highmem pages one page at a time with
kmap_atomic(). For lowmem pages it does a single cache operation over
the whole range, so the behaviour is unchanged there. A small wrapper
is needed because __flush_invalidate_dcache_range() is a macro on cores
with a write-through data cache.

Fixes: 0f665b9e2a6d ("xtensa: use the generic uncached segment support")
Reviewed-by: Max Filippov <jcmvbkbc@gmail.com>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
---
 arch/xtensa/kernel/pci-dma.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/arch/xtensa/kernel/pci-dma.c b/arch/xtensa/kernel/pci-dma.c
index e1c00ec37eed..41467a1ba068 100644
--- a/arch/xtensa/kernel/pci-dma.c
+++ b/arch/xtensa/kernel/pci-dma.c
@@ -80,9 +80,19 @@ void arch_sync_dma_for_device(phys_addr_t paddr, size_t size,
 	}
 }
 
+/*
+ * __flush_invalidate_dcache_range() may be a macro on cores with a
+ * write-through data cache, so it cannot be passed to do_cache_op().
+ */
+static void flush_invalidate_dcache_range(unsigned long start,
+					  unsigned long size)
+{
+	__flush_invalidate_dcache_range(start, size);
+}
+
 void arch_dma_prep_coherent(struct page *page, size_t size)
 {
-	__flush_invalidate_dcache_range((unsigned long)page_address(page), size);
+	do_cache_op(page_to_phys(page), size, flush_invalidate_dcache_range);
 }
 
 /*
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v3 3/8] MIPS: mm: handle highmem pages in arch_dma_prep_coherent()
  2026-11-19 15:15 ` [PATCH v3 0/7] dma-mapping: DMA_ATTR_NO_KERNEL_MAPPING related fixes Marek Szyprowski
  2026-11-19 15:15   ` [PATCH v3 1/8] xtensa: write back dirty lines in arch_dma_prep_coherent() Marek Szyprowski
  2026-11-19 15:15   ` [PATCH v3 2/8] xtensa: handle highmem pages " Marek Szyprowski
@ 2026-11-19 15:15   ` Marek Szyprowski
  2026-11-19 15:15   ` [PATCH v3 4/8] powerpc/mm: skip kmap for lowmem pages in __dma_sync_page() Marek Szyprowski
                     ` (5 subsequent siblings)
  8 siblings, 0 replies; 12+ messages in thread
From: Marek Szyprowski @ 2026-11-19 15:15 UTC (permalink / raw)
  To: linux-mips, linuxppc-dev, iommu, linux-kernel
  Cc: Marek Szyprowski, Robin Murphy, Christoph Hellwig,
	Thomas Bogendoerfer, Madhavan Srinivasan, Michael Ellerman,
	Nicholas Piggin, Christophe Leroy (CS GROUP),
	Ritesh Harjani (IBM), Shrikanth Hegde, Chris Zankel, Max Filippov

arch_dma_prep_coherent() performs the cache maintenance on the address
returned by page_address(). On 32-bit MIPS with CONFIG_HIGHMEM the kernel
uses HASHED_PAGE_VIRTUAL, so page_address() returns NULL for a highmem
page that is not kmapped, and dma_cache_wback_inv() then operates on
the virtual range starting at 0 instead of the buffer, leaving any dirty
cache lines of the buffer in place.

Highmem pages do reach this function: dma_alloc_contiguous() can return
a highmem CMA page, which dma_direct_alloc() handles by forcing the remap
path and calling arch_dma_prep_coherent() on that page before creating
the uncached mapping. Since commit f5ff79fddf0e ("dma-mapping: remove
CONFIG_DMA_REMAP") this path is available on all MMU architectures,
including MIPS, which previously rejected highmem CMA pages.

Fix this by reusing dma_sync_phys(), which already handles highmem pages
by mapping them one page at a time with kmap_atomic(). For lowmem pages
it does a single dma_cache_wback_inv() over the whole range, so the
behaviour is unchanged there.

Fixes: f5ff79fddf0e ("dma-mapping: remove CONFIG_DMA_REMAP")
Acked-by: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
---
 arch/mips/mm/dma-noncoherent.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/arch/mips/mm/dma-noncoherent.c b/arch/mips/mm/dma-noncoherent.c
index 30ef3e247eb7..d8e5bc85bb3b 100644
--- a/arch/mips/mm/dma-noncoherent.c
+++ b/arch/mips/mm/dma-noncoherent.c
@@ -44,11 +44,6 @@ static inline bool cpu_needs_post_dma_flush(void)
 	}
 }
 
-void arch_dma_prep_coherent(struct page *page, size_t size)
-{
-	dma_cache_wback_inv((unsigned long)page_address(page), size);
-}
-
 void *arch_dma_set_uncached(void *addr, size_t size)
 {
 	return (void *)(__pa(addr) + UNCAC_BASE);
@@ -127,6 +122,11 @@ void arch_sync_dma_for_device(phys_addr_t paddr, size_t size,
 	dma_sync_phys(paddr, size, dir, true);
 }
 
+void arch_dma_prep_coherent(struct page *page, size_t size)
+{
+	dma_sync_phys(page_to_phys(page), size, DMA_BIDIRECTIONAL, true);
+}
+
 #ifdef CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU
 void arch_sync_dma_for_cpu(phys_addr_t paddr, size_t size,
 		enum dma_data_direction dir)
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v3 4/8] powerpc/mm: skip kmap for lowmem pages in __dma_sync_page()
  2026-11-19 15:15 ` [PATCH v3 0/7] dma-mapping: DMA_ATTR_NO_KERNEL_MAPPING related fixes Marek Szyprowski
                     ` (2 preceding siblings ...)
  2026-11-19 15:15   ` [PATCH v3 3/8] MIPS: mm: " Marek Szyprowski
@ 2026-11-19 15:15   ` Marek Szyprowski
  2026-11-19 17:13     ` Ritesh Harjani
  2026-11-19 15:15   ` [PATCH v3 5/8] powerpc/mm: handle highmem pages in arch_dma_prep_coherent() Marek Szyprowski
                     ` (4 subsequent siblings)
  8 siblings, 1 reply; 12+ messages in thread
From: Marek Szyprowski @ 2026-11-19 15:15 UTC (permalink / raw)
  To: linux-mips, linuxppc-dev, iommu, linux-kernel
  Cc: Marek Szyprowski, Robin Murphy, Christoph Hellwig,
	Thomas Bogendoerfer, Madhavan Srinivasan, Michael Ellerman,
	Nicholas Piggin, Christophe Leroy (CS GROUP),
	Ritesh Harjani (IBM), Shrikanth Hegde, Chris Zankel, Max Filippov

With CONFIG_HIGHMEM enabled, __dma_sync_page() unconditionally goes
through __dma_sync_page_highmem(), which disables interrupts and
kmaps the buffer one page at a time. This adds needless per-page
overhead to every streaming DMA sync on such kernels.

Use the per-page kmap path only when the buffer reaches into highmem
and do a single __dma_sync() on the linear mapping otherwise.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
---
 arch/powerpc/mm/dma-noncoherent.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/arch/powerpc/mm/dma-noncoherent.c b/arch/powerpc/mm/dma-noncoherent.c
index 30260b5d146d..3ce7dfc213d9 100644
--- a/arch/powerpc/mm/dma-noncoherent.c
+++ b/arch/powerpc/mm/dma-noncoherent.c
@@ -96,12 +96,16 @@ static void __dma_sync_page(phys_addr_t paddr, size_t size, int dir)
 	struct page *page = pfn_to_page(paddr >> PAGE_SHIFT);
 	unsigned offset = paddr & ~PAGE_MASK;
 
+	if (!size)
+		return;
+
 #ifdef CONFIG_HIGHMEM
-	__dma_sync_page_highmem(page, offset, size, dir);
-#else
-	unsigned long start = (unsigned long)page_address(page) + offset;
-	__dma_sync((void *)start, size, dir);
+	if (PageHighMem(pfn_to_page(PHYS_PFN(paddr + size - 1)))) {
+		__dma_sync_page_highmem(page, offset, size, dir);
+		return;
+	}
 #endif
+	__dma_sync(page_address(page) + offset, size, dir);
 }
 
 void arch_sync_dma_for_device(phys_addr_t paddr, size_t size,
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v3 5/8] powerpc/mm: handle highmem pages in arch_dma_prep_coherent()
  2026-11-19 15:15 ` [PATCH v3 0/7] dma-mapping: DMA_ATTR_NO_KERNEL_MAPPING related fixes Marek Szyprowski
                     ` (3 preceding siblings ...)
  2026-11-19 15:15   ` [PATCH v3 4/8] powerpc/mm: skip kmap for lowmem pages in __dma_sync_page() Marek Szyprowski
@ 2026-11-19 15:15   ` Marek Szyprowski
  2026-11-19 17:15     ` Ritesh Harjani
  2026-11-19 15:15   ` [PATCH v3 6/8] dma-mapping: document that arch_dma_prep_coherent() must handle highmem Marek Szyprowski
                     ` (3 subsequent siblings)
  8 siblings, 1 reply; 12+ messages in thread
From: Marek Szyprowski @ 2026-11-19 15:15 UTC (permalink / raw)
  To: linux-mips, linuxppc-dev, iommu, linux-kernel
  Cc: Marek Szyprowski, Robin Murphy, Christoph Hellwig,
	Thomas Bogendoerfer, Madhavan Srinivasan, Michael Ellerman,
	Nicholas Piggin, Christophe Leroy (CS GROUP),
	Ritesh Harjani (IBM), Shrikanth Hegde, Chris Zankel, Max Filippov

arch_dma_prep_coherent() performs the cache maintenance on the address
returned by page_address(). On 32-bit powerpc with CONFIG_HIGHMEM that
is NULL for a highmem page which is not kmapped, so the flush would
operate on the wrong virtual range and leave the dirty lines of the
buffer in place.

The generic DMA code may pass highmem pages to this hook:
dma_direct_alloc() handles a highmem page returned by
dma_alloc_contiguous() by forcing the remap path, which calls
arch_dma_prep_coherent() on that page before creating the uncached
mapping. This cannot happen on powerpc today only because it does not
select HAVE_DMA_CONTIGUOUS, so dma-direct never gets a highmem page
there, but nothing in the hook's contract rules it out.

Use __dma_sync_page() with DMA_BIDIRECTIONAL, which the
arch_sync_dma_for_*() hooks already use and which kmaps highmem pages
one at a time. Without CONFIG_HIGHMEM it does the same
flush_dcache_range() on the linear mapping as before.

Fixes: f2902a2fb40c ("powerpc: use the generic dma coherent remap allocator")
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
---
 arch/powerpc/mm/dma-noncoherent.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/arch/powerpc/mm/dma-noncoherent.c b/arch/powerpc/mm/dma-noncoherent.c
index 3ce7dfc213d9..af2a42e47016 100644
--- a/arch/powerpc/mm/dma-noncoherent.c
+++ b/arch/powerpc/mm/dma-noncoherent.c
@@ -122,7 +122,5 @@ void arch_sync_dma_for_cpu(phys_addr_t paddr, size_t size,
 
 void arch_dma_prep_coherent(struct page *page, size_t size)
 {
-	unsigned long kaddr = (unsigned long)page_address(page);
-
-	flush_dcache_range(kaddr, kaddr + size);
+	__dma_sync_page(page_to_phys(page), size, DMA_BIDIRECTIONAL);
 }
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v3 6/8] dma-mapping: document that arch_dma_prep_coherent() must handle highmem
  2026-11-19 15:15 ` [PATCH v3 0/7] dma-mapping: DMA_ATTR_NO_KERNEL_MAPPING related fixes Marek Szyprowski
                     ` (4 preceding siblings ...)
  2026-11-19 15:15   ` [PATCH v3 5/8] powerpc/mm: handle highmem pages in arch_dma_prep_coherent() Marek Szyprowski
@ 2026-11-19 15:15   ` Marek Szyprowski
  2026-11-19 15:15   ` [PATCH v3 7/8] dma-direct: do not skip cache maintenance for highmem pages in no-mapping allocations Marek Szyprowski
                     ` (2 subsequent siblings)
  8 siblings, 0 replies; 12+ messages in thread
From: Marek Szyprowski @ 2026-11-19 15:15 UTC (permalink / raw)
  To: linux-mips, linuxppc-dev, iommu, linux-kernel
  Cc: Marek Szyprowski, Robin Murphy, Christoph Hellwig,
	Thomas Bogendoerfer, Madhavan Srinivasan, Michael Ellerman,
	Nicholas Piggin, Christophe Leroy (CS GROUP),
	Ritesh Harjani (IBM), Shrikanth Hegde, Chris Zankel, Max Filippov

arch_dma_prep_coherent() has no documentation, and several
implementations assumed that the page passed to it always has a kernel
linear mapping and used page_address() on it. That is not the case:
dma_direct_alloc() and atomic_pool_expand() can get a highmem page from
CMA and pass it to arch_dma_prep_coherent() before remapping it, and
dma-iommu allocates the pages for non-contiguous buffers with
__GFP_HIGHMEM.

Add a kernel-doc comment describing what the hook has to do and stating
explicitly that it must accept highmem pages without a permanent kernel
mapping.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
---
 include/linux/dma-map-ops.h | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h
index afaf2c12189b..ee4ac12bf2ad 100644
--- a/include/linux/dma-map-ops.h
+++ b/include/linux/dma-map-ops.h
@@ -371,6 +371,17 @@ static inline void arch_sync_dma_for_cpu_all(void)
 }
 #endif /* CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL */
 
+/**
+ * arch_dma_prep_coherent - prepare memory for use as a coherent DMA buffer
+ * @page: first page of the physically contiguous buffer
+ * @size: size of the buffer in bytes
+ *
+ * Called before the buffer is accessed through an uncached alias or handed
+ * to a non-coherent device. The implementation must write back any dirty
+ * data cache lines covering the buffer, so no dirty line can be evicted
+ * later on top of data written through the uncached alias or by the device.
+ * Must accept highmem pages without a permanent kernel mapping.
+ */
 #ifdef CONFIG_ARCH_HAS_DMA_PREP_COHERENT
 void arch_dma_prep_coherent(struct page *page, size_t size);
 #else
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v3 7/8] dma-direct: do not skip cache maintenance for highmem pages in no-mapping allocations
  2026-11-19 15:15 ` [PATCH v3 0/7] dma-mapping: DMA_ATTR_NO_KERNEL_MAPPING related fixes Marek Szyprowski
                     ` (5 preceding siblings ...)
  2026-11-19 15:15   ` [PATCH v3 6/8] dma-mapping: document that arch_dma_prep_coherent() must handle highmem Marek Szyprowski
@ 2026-11-19 15:15   ` Marek Szyprowski
  2026-11-19 15:15   ` [PATCH v3 8/8] dma-direct: zero memory allocated with DMA_ATTR_NO_KERNEL_MAPPING Marek Szyprowski
  2026-11-19 18:15   ` [PATCH v3 0/7] dma-mapping: DMA_ATTR_NO_KERNEL_MAPPING related fixes Marek Szyprowski
  8 siblings, 0 replies; 12+ messages in thread
From: Marek Szyprowski @ 2026-11-19 15:15 UTC (permalink / raw)
  To: linux-mips, linuxppc-dev, iommu, linux-kernel
  Cc: Marek Szyprowski, Robin Murphy, Christoph Hellwig,
	Thomas Bogendoerfer, Madhavan Srinivasan, Michael Ellerman,
	Nicholas Piggin, Christophe Leroy (CS GROUP),
	Ritesh Harjani (IBM), Shrikanth Hegde, Chris Zankel, Max Filippov

dma_direct_alloc_no_mapping() calls arch_dma_prep_coherent() only for
lowmem pages. The check was added when the DMA_ATTR_NO_KERNEL_MAPPING
handling was moved to dma-direct, because some implementations of the
hook used page_address() and could not handle highmem pages.

As a result, a buffer backed by a highmem CMA page is handed to the
device without removing the dirty cache lines covering it, which may
still be present from the previous user of the page. These lines can be
evicted later and corrupt data written by a non-coherent device.

All implementations of arch_dma_prep_coherent() now handle highmem
pages, and the hook is documented to do so; dma_direct_alloc() already
calls it on highmem pages in its remap path. Drop the check so that
highmem pages get the same cache maintenance as lowmem pages.

Fixes: d98849aff879 ("dma-direct: handle DMA_ATTR_NO_KERNEL_MAPPING in common code")
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
---
 kernel/dma/direct.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
index 25e9a86835fd..fea0dd36dd25 100644
--- a/kernel/dma/direct.c
+++ b/kernel/dma/direct.c
@@ -207,8 +207,7 @@ static void *dma_direct_alloc_no_mapping(struct device *dev, size_t size,
 		return NULL;
 
 	/* remove any dirty cache lines on the kernel alias */
-	if (!PageHighMem(page))
-		arch_dma_prep_coherent(page, size);
+	arch_dma_prep_coherent(page, size);
 	/*
 	 * return the page pointer as the opaque cookie.
 	 * Never used for unencrypted allocation
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v3 8/8] dma-direct: zero memory allocated with DMA_ATTR_NO_KERNEL_MAPPING
  2026-11-19 15:15 ` [PATCH v3 0/7] dma-mapping: DMA_ATTR_NO_KERNEL_MAPPING related fixes Marek Szyprowski
                     ` (6 preceding siblings ...)
  2026-11-19 15:15   ` [PATCH v3 7/8] dma-direct: do not skip cache maintenance for highmem pages in no-mapping allocations Marek Szyprowski
@ 2026-11-19 15:15   ` Marek Szyprowski
  2026-11-19 18:15   ` [PATCH v3 0/7] dma-mapping: DMA_ATTR_NO_KERNEL_MAPPING related fixes Marek Szyprowski
  8 siblings, 0 replies; 12+ messages in thread
From: Marek Szyprowski @ 2026-11-19 15:15 UTC (permalink / raw)
  To: linux-mips, linuxppc-dev, iommu, linux-kernel
  Cc: Marek Szyprowski, Robin Murphy, Christoph Hellwig,
	Thomas Bogendoerfer, Madhavan Srinivasan, Michael Ellerman,
	Nicholas Piggin, Christophe Leroy (CS GROUP),
	Ritesh Harjani (IBM), Shrikanth Hegde, Chris Zankel, Max Filippov

dma_direct_alloc_no_mapping() explicitly clears __GFP_ZERO before
allocating the pages, so buffers allocated with
DMA_ATTR_NO_KERNEL_MAPPING are handed out with stale contents of the
previously freed pages. Such buffers are typically passed to devices or
mapped to userspace, which may expose old kernel data.

The mask comes from the regular allocation path, which always zeroes the
buffer manually through its kernel mapping once it is set up. The
DMA_ATTR_NO_KERNEL_MAPPING path returns early with only a page cookie,
so it never reaches that memset(). This has been the case since the
attribute was first supported in the generic remap code in commit
e440e26a0251 ("dma-remap: support DMA_ATTR_NO_KERNEL_MAPPING") and was
carried over when that handling was lifted into dma-direct.

All other dma_alloc_* paths always return zeroed memory, so do the same
here. Passing __GFP_ZERO to the page allocator is not sufficient, as
__dma_direct_alloc_pages() first tries dma_alloc_contiguous(), and CMA
ignores __GFP_ZERO.

Closes: https://sashiko.dev/#/bug/linux-c4421083-e44e-4373-a2c2-45649ea49c0b
Fixes: d98849aff879 ("dma-direct: handle DMA_ATTR_NO_KERNEL_MAPPING in common code")
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
---
 kernel/dma/direct.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
index fea0dd36dd25..4bfb3e76cd76 100644
--- a/kernel/dma/direct.c
+++ b/kernel/dma/direct.c
@@ -6,6 +6,7 @@
  */
 #include <linux/memblock.h> /* for max_pfn */
 #include <linux/export.h>
+#include <linux/highmem.h>
 #include <linux/mm.h>
 #include <linux/dma-map-ops.h>
 #include <linux/scatterlist.h>
@@ -201,11 +202,16 @@ static void *dma_direct_alloc_no_mapping(struct device *dev, size_t size,
 		dma_addr_t *dma_handle, gfp_t gfp)
 {
 	struct page *page;
+	unsigned long i;
 
 	page = __dma_direct_alloc_pages(dev, size, gfp & ~__GFP_ZERO, true);
 	if (!page)
 		return NULL;
 
+	/* we always manually zero the allocated memory */
+	for (i = 0; i < size >> PAGE_SHIFT; i++)
+		clear_highpage(page + i);
+
 	/* remove any dirty cache lines on the kernel alias */
 	arch_dma_prep_coherent(page, size);
 	/*
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* Re: [PATCH v3 4/8] powerpc/mm: skip kmap for lowmem pages in __dma_sync_page()
  2026-11-19 15:15   ` [PATCH v3 4/8] powerpc/mm: skip kmap for lowmem pages in __dma_sync_page() Marek Szyprowski
@ 2026-11-19 17:13     ` Ritesh Harjani
  0 siblings, 0 replies; 12+ messages in thread
From: Ritesh Harjani @ 2026-11-19 17:13 UTC (permalink / raw)
  To: Marek Szyprowski, linux-mips, linuxppc-dev, iommu, linux-kernel
  Cc: Marek Szyprowski, Robin Murphy, Christoph Hellwig,
	Thomas Bogendoerfer, Madhavan Srinivasan, Michael Ellerman,
	Nicholas Piggin, Christophe Leroy (CS GROUP), Shrikanth Hegde,
	Chris Zankel, Max Filippov

Marek Szyprowski <m.szyprowski@samsung.com> writes:

> With CONFIG_HIGHMEM enabled, __dma_sync_page() unconditionally goes
> through __dma_sync_page_highmem(), which disables interrupts and
> kmaps the buffer one page at a time. This adds needless per-page
> overhead to every streaming DMA sync on such kernels.
>
> Use the per-page kmap path only when the buffer reaches into highmem
> and do a single __dma_sync() on the linear mapping otherwise.
>
> Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
> ---
>  arch/powerpc/mm/dma-noncoherent.c | 12 ++++++++----
>  1 file changed, 8 insertions(+), 4 deletions(-)

Thanks for taking care of this. LGTM.

Feel free to add:
Reviewed-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH v3 5/8] powerpc/mm: handle highmem pages in arch_dma_prep_coherent()
  2026-11-19 15:15   ` [PATCH v3 5/8] powerpc/mm: handle highmem pages in arch_dma_prep_coherent() Marek Szyprowski
@ 2026-11-19 17:15     ` Ritesh Harjani
  0 siblings, 0 replies; 12+ messages in thread
From: Ritesh Harjani @ 2026-11-19 17:15 UTC (permalink / raw)
  To: Marek Szyprowski, linux-mips, linuxppc-dev, iommu, linux-kernel
  Cc: Marek Szyprowski, Robin Murphy, Christoph Hellwig,
	Thomas Bogendoerfer, Madhavan Srinivasan, Michael Ellerman,
	Nicholas Piggin, Christophe Leroy (CS GROUP), Shrikanth Hegde,
	Chris Zankel, Max Filippov

Marek Szyprowski <m.szyprowski@samsung.com> writes:

> arch_dma_prep_coherent() performs the cache maintenance on the address
> returned by page_address(). On 32-bit powerpc with CONFIG_HIGHMEM that
> is NULL for a highmem page which is not kmapped, so the flush would
> operate on the wrong virtual range and leave the dirty lines of the
> buffer in place.
>
> The generic DMA code may pass highmem pages to this hook:
> dma_direct_alloc() handles a highmem page returned by
> dma_alloc_contiguous() by forcing the remap path, which calls
> arch_dma_prep_coherent() on that page before creating the uncached
> mapping. This cannot happen on powerpc today only because it does not
> select HAVE_DMA_CONTIGUOUS, so dma-direct never gets a highmem page
> there, but nothing in the hook's contract rules it out.

That is correct.

>
> Use __dma_sync_page() with DMA_BIDIRECTIONAL, which the
> arch_sync_dma_for_*() hooks already use and which kmaps highmem pages
> one at a time. Without CONFIG_HIGHMEM it does the same
> flush_dcache_range() on the linear mapping as before.
>
> Fixes: f2902a2fb40c ("powerpc: use the generic dma coherent remap allocator")

I don't think we need a Fixes tag here for the reason you quoted above.
I’m afraid that adding a Fixes tag might cause this patch to be
backported without the previous patch, which would result in extra work
for lowmem pages on highmem-enabled kernels

I think we should drop the "Fixes" tag here.

> Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
> ---
>  arch/powerpc/mm/dma-noncoherent.c | 4 +---
>  1 file changed, 1 insertion(+), 3 deletions(-)

Otherwise, LGTM. Feel free to add:
Reviewed-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH v3 0/7] dma-mapping: DMA_ATTR_NO_KERNEL_MAPPING related fixes
  2026-11-19 15:15 ` [PATCH v3 0/7] dma-mapping: DMA_ATTR_NO_KERNEL_MAPPING related fixes Marek Szyprowski
                     ` (7 preceding siblings ...)
  2026-11-19 15:15   ` [PATCH v3 8/8] dma-direct: zero memory allocated with DMA_ATTR_NO_KERNEL_MAPPING Marek Szyprowski
@ 2026-11-19 18:15   ` Marek Szyprowski
  8 siblings, 0 replies; 12+ messages in thread
From: Marek Szyprowski @ 2026-11-19 18:15 UTC (permalink / raw)
  To: linux-mips, linuxppc-dev, iommu, linux-kernel
  Cc: Robin Murphy, Christoph Hellwig, Thomas Bogendoerfer,
	Madhavan Srinivasan, Michael Ellerman, Nicholas Piggin,
	Christophe Leroy (CS GROUP), Ritesh Harjani (IBM),
	Shrikanth Hegde, Chris Zankel, Max Filippov

On 19.11.2026 16:15, Marek Szyprowski wrote:
> arch_dma_prep_coherent() is called by the generic DMA code on pages
> which may be located in highmem: dma_direct_alloc() forces the remap
> path for a highmem page returned from CMA and calls this hook on it
> before creating the uncached mapping. However, the hook has never been
> documented and some implementations assume that the page has a kernel
> linear mapping. This issue affects MIPS, xtensa and PowerPC. It gets
> fixed by patches #2-#5.
>
> xtensa also only invalidates the cache, which discards the zeroes
> written by the dma_direct_alloc(). Patch #1 switches it to a
> writeback+invalidate.
>
> Patch #6 documents that arch_dma_prep_coherent() must write back dirty
> lines and must accept highmem pages. With that in place, patch #7 drops
> the PageHighMem() check in dma_direct_alloc_no_mapping(), which so far
> skipped cache maintenance for highmem CMA pages allocated with
> DMA_ATTR_NO_KERNEL_MAPPING.
>
> Patch #8 makes dma_direct_alloc_no_mapping() always return zeroed
> memory, like all other dma_alloc_* paths. Passing __GFP_ZERO is not
> enough there, because CMA ignores it.
>
> The analysis and the patchset was prepared with the help of an LLM. The
> code compiles fine, but it has not been runtime tested on any of the
> affected architectures yet. Testing on real hardware would be
> appreciated. I've reviewed the resulting changes.
>
> I plan to merge this via my dma-mapping tree, I can provide a tag if
> needed for merging it also to arch code.
>
> Best regards
> Marek Szyprowski, PhD
> Samsung R&D Institute Poland


Merged into dma-mapping-for-next, with Fixes tag dropped in patch #5
as requested by Ritesh. Here is a stable tag if one needs to merge
this patchset also to arch tree:

git://git.kernel.org/pub/scm/linux/kernel/git/mszyprowski/linux.git tags/no_kernel_mapping_fixes

Best regards
-- 
Marek Szyprowski, PhD
Samsung R&D Institute Poland


^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2026-11-19 18:15 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
     [not found] <CGME20261119151527eucas1p16c69a0580b2caabe66f952572362021f@eucas1p1.samsung.com>
2026-11-19 15:15 ` [PATCH v3 0/7] dma-mapping: DMA_ATTR_NO_KERNEL_MAPPING related fixes Marek Szyprowski
2026-11-19 15:15   ` [PATCH v3 1/8] xtensa: write back dirty lines in arch_dma_prep_coherent() Marek Szyprowski
2026-11-19 15:15   ` [PATCH v3 2/8] xtensa: handle highmem pages " Marek Szyprowski
2026-11-19 15:15   ` [PATCH v3 3/8] MIPS: mm: " Marek Szyprowski
2026-11-19 15:15   ` [PATCH v3 4/8] powerpc/mm: skip kmap for lowmem pages in __dma_sync_page() Marek Szyprowski
2026-11-19 17:13     ` Ritesh Harjani
2026-11-19 15:15   ` [PATCH v3 5/8] powerpc/mm: handle highmem pages in arch_dma_prep_coherent() Marek Szyprowski
2026-11-19 17:15     ` Ritesh Harjani
2026-11-19 15:15   ` [PATCH v3 6/8] dma-mapping: document that arch_dma_prep_coherent() must handle highmem Marek Szyprowski
2026-11-19 15:15   ` [PATCH v3 7/8] dma-direct: do not skip cache maintenance for highmem pages in no-mapping allocations Marek Szyprowski
2026-11-19 15:15   ` [PATCH v3 8/8] dma-direct: zero memory allocated with DMA_ATTR_NO_KERNEL_MAPPING Marek Szyprowski
2026-11-19 18:15   ` [PATCH v3 0/7] dma-mapping: DMA_ATTR_NO_KERNEL_MAPPING related fixes Marek Szyprowski

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox