* [PATCH v2 0/2] mm/huge_memory: flush dirty file PMDs before removing rmap
@ 2026-11-16 15:06 David Hildenbrand (Arm)
2026-11-16 15:06 ` [PATCH v2 1/2] mm/mmu_gather: delay rmap removal on UP as well David Hildenbrand (Arm)
2026-11-16 15:06 ` [PATCH v2 2/2] mm/huge_memory: flush dirty file PMDs before removing rmap David Hildenbrand (Arm)
0 siblings, 2 replies; 10+ messages in thread
From: David Hildenbrand (Arm) @ 2026-11-16 15:06 UTC (permalink / raw)
To: Will Deacon, Aneesh Kumar K.V, Andrew Morton, Nick Piggin,
Peter Zijlstra, Arnd Bergmann, Johannes Weiner, Hugh Dickins,
Zi Yan, Baolin Wang, Liam R. Howlett, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Lance Yang, Usama Arif, Kiryl Shutsemau,
Lorenzo Stoakes, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Jan Kara, Ross Zwisler
Cc: linux-arch, linux-mm, linux-kernel, David Hildenbrand (Arm),
stable, Chris Mason
Details in #2. Now includes a fix for a preexisting issue on UP in #1.
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
---
Changes in v2:
- "mm/mmu_gather: delay rmap removal on UP as well"
- Added
- "mm/huge_memory: flush dirty file PMDs before removing rmap"
- Adjust comment in zap_huge_pmd_folio() to talk about stale TLB
entries, not mentioning other CPUs
- Adjust patch description
- Pick up RBs
- Use Closes instead of Link for report
- Link to v1: https://patch.msgid.link/20261113-dirty_file_pmd-v1-1-8c11f2a2e65f@kernel.org
---
David Hildenbrand (Arm) (2):
mm/mmu_gather: delay rmap removal on UP as well
mm/huge_memory: flush dirty file PMDs before removing rmap
include/asm-generic/tlb.h | 16 ++++------------
mm/huge_memory.c | 39 +++++++++++++++++++++++++++++++--------
mm/memory.c | 2 +-
mm/mmu_gather.c | 4 ++--
4 files changed, 38 insertions(+), 23 deletions(-)
---
base-commit: 25f5c92e1ac06496679ea229e5c825aa88968efd
change-id: 20261113-dirty_file_pmd-90e63640a593
Best regards,
--
David Hildenbrand (Arm) <david@kernel.org>
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v2 1/2] mm/mmu_gather: delay rmap removal on UP as well
2026-11-16 15:06 [PATCH v2 0/2] mm/huge_memory: flush dirty file PMDs before removing rmap David Hildenbrand (Arm)
@ 2026-11-16 15:06 ` David Hildenbrand (Arm)
2026-11-16 15:16 ` Lorenzo Stoakes (ARM)
` (2 more replies)
2026-11-16 15:06 ` [PATCH v2 2/2] mm/huge_memory: flush dirty file PMDs before removing rmap David Hildenbrand (Arm)
1 sibling, 3 replies; 10+ messages in thread
From: David Hildenbrand (Arm) @ 2026-11-16 15:06 UTC (permalink / raw)
To: Will Deacon, Aneesh Kumar K.V, Andrew Morton, Nick Piggin,
Peter Zijlstra, Arnd Bergmann, Johannes Weiner, Hugh Dickins,
Zi Yan, Baolin Wang, Liam R. Howlett, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Lance Yang, Usama Arif, Kiryl Shutsemau,
Lorenzo Stoakes, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Jan Kara, Ross Zwisler
Cc: linux-arch, linux-mm, linux-kernel, David Hildenbrand (Arm),
stable
Commit 5df397dec7c4 ("mm: delay page_remove_rmap() until after the TLB
has been flushed") introduced delayed rmap removal of dirty pages until
after the TLB flush, so writeback cannot clean a page that is still
writable through a stale TLB entry.
It skips this on UP, because on UP there are no remote TLBs (single CPU)
and because the PTL prevents preemption.
The PTL prevents preemption while removing the rmap, but the local TLB
flush is usually only performed after dropping the PTL.
Consequently, after dropping the PTL we could reschedule to another
thread of the same mm, which does not flush the TLB. A kworker could
then run into the original race with writeback: it cleans the folio,
but the other thread can still modify it through the stale TLB entries,
and these writes get lost.
Fix it by simply not special-casing UP. Now, only architectures that
select MMU_GATHER_NO_GATHER (s390) will skip delay_rmap handling.
Fixes: 5df397dec7c4 ("mm: delay page_remove_rmap() until after the TLB has been flushed")
Cc: stable@vger.kernel.org
Reported-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Link: https://lore.kernel.org/r/avrxOt_GyXLXNqtR@gremlin
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
---
include/asm-generic/tlb.h | 6 +-----
mm/mmu_gather.c | 2 --
2 files changed, 1 insertion(+), 7 deletions(-)
diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h
index 4bba02fd5d543..cb087315b4e72 100644
--- a/include/asm-generic/tlb.h
+++ b/include/asm-generic/tlb.h
@@ -249,23 +249,19 @@ extern bool __tlb_remove_page_size(struct mmu_gather *tlb, struct page *page, in
bool __tlb_remove_folio_pages(struct mmu_gather *tlb, struct page *page,
unsigned int nr_pages, bool delay_rmap);
-#ifdef CONFIG_SMP
/*
* This both sets 'delayed_rmap', and returns true. It would be an inline
* function, except we define it before the 'struct mmu_gather'.
*/
#define tlb_delay_rmap(tlb) (((tlb)->delayed_rmap = 1), true)
extern void tlb_flush_rmaps(struct mmu_gather *tlb, struct vm_area_struct *vma);
-#endif
#endif
/*
* We have a no-op version of the rmap removal that doesn't
* delay anything. That is used on S390, which flushes remote
- * TLBs synchronously, and on UP, which doesn't have any
- * remote TLBs to flush and is not preemptible due to this
- * all happening under the page table lock.
+ * TLBs synchronously.
*/
#ifndef tlb_delay_rmap
#define tlb_delay_rmap(tlb) (false)
diff --git a/mm/mmu_gather.c b/mm/mmu_gather.c
index e98397ac577c4..c022aeb7c84d0 100644
--- a/mm/mmu_gather.c
+++ b/mm/mmu_gather.c
@@ -49,7 +49,6 @@ static bool tlb_next_batch(struct mmu_gather *tlb)
return true;
}
-#ifdef CONFIG_SMP
static void tlb_flush_rmap_batch(struct mmu_gather_batch *batch, struct vm_area_struct *vma)
{
struct encoded_page **pages = batch->encoded_pages;
@@ -91,7 +90,6 @@ void tlb_flush_rmaps(struct mmu_gather *tlb, struct vm_area_struct *vma)
tlb_flush_rmap_batch(tlb->active, vma);
tlb->delayed_rmap = 0;
}
-#endif
/*
* We might end up freeing a lot of pages. Reschedule on a regular
--
2.56.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v2 2/2] mm/huge_memory: flush dirty file PMDs before removing rmap
2026-11-16 15:06 [PATCH v2 0/2] mm/huge_memory: flush dirty file PMDs before removing rmap David Hildenbrand (Arm)
2026-11-16 15:06 ` [PATCH v2 1/2] mm/mmu_gather: delay rmap removal on UP as well David Hildenbrand (Arm)
@ 2026-11-16 15:06 ` David Hildenbrand (Arm)
2026-11-16 15:19 ` Lorenzo Stoakes (ARM)
` (2 more replies)
1 sibling, 3 replies; 10+ messages in thread
From: David Hildenbrand (Arm) @ 2026-11-16 15:06 UTC (permalink / raw)
To: Will Deacon, Aneesh Kumar K.V, Andrew Morton, Nick Piggin,
Peter Zijlstra, Arnd Bergmann, Johannes Weiner, Hugh Dickins,
Zi Yan, Baolin Wang, Liam R. Howlett, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Lance Yang, Usama Arif, Kiryl Shutsemau,
Lorenzo Stoakes, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Jan Kara, Ross Zwisler
Cc: linux-arch, linux-mm, linux-kernel, David Hildenbrand (Arm),
stable, Chris Mason
Unmapping a file PMD can lose concurrent stores when it races with
writeback. zap_huge_pmd() removes rmap and drops the PMD lock before the
gather flushes the TLB. Writeback can then clean the folio while another
thread can still write through a stale writable TLB entry. Those stores
do not fault and dirty the folio again. Filesystem DAX has the same
problem with its dirty tags.
Commit 5df397dec7c4 ("mm: delay page_remove_rmap() until after the TLB
has been flushed") fixed the same race for PTEs by adding delayed rmap
removal to the mmu_gather infrastructure. The PMD path still removes
rmap before flushing.
Flush the TLB under the PMD lock before removing rmap for dirty file
PMDs. Skip the early flush at address-space teardown, when no users
remain, and for mappings without writeback, including shmem and device
DAX.
Use tlb_can_delay_rmap() to share the PTE path's architecture checks.
On s390, clearing the entry already handles TLB synchronization, and
__tlb_remove_page_size() immediately drops the mapping reference, so
rmap must be removed first. Make the predicate side-effect-free and set
the gather's delayed_rmap flag when an entry is actually queued for
delayed rmap removal.
Queue the folio before the early flush, while the gather still has a
valid range. Keep batching references and drain a full batch only after
dropping the PMD lock. The early flush resets the gather's TLB state,
so draining the batch does not flush that range again.
Although writing to files through mmap is generally discouraged,
unmapping such mappings may become slower: each dirty file PMD now
requires a TLB flush under the PMD lock. Future work should properly
extend the delayed-rmap machinery to PMDs and hold the PMD spinlock
across multiple entries, so their TLB invalidations and rmap removals
can be batched.
Fixes: 4b4bb46d00b3 ("dax: clear dirty entry tags on cache flush")
Cc: stable@vger.kernel.org
Reported-by: Kiryl Shutsemau <kas@kernel.org>
Closes: https://lore.kernel.org/linux-mm/aoWw_iKEDLjKL2jr@thinkstation/
Reported-by: Chris Mason <mason@kernel.org>
Link: https://git.kernel.org/linus/5df397dec7c4c08c23bd14f162f1228836faa4ce
Suggested-by: Lance Yang <lance.yang@linux.dev>
Suggested-by: Usama Arif <usama.arif@linux.dev>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
---
include/asm-generic/tlb.h | 10 +++-------
mm/huge_memory.c | 39 +++++++++++++++++++++++++++++++--------
mm/memory.c | 2 +-
mm/mmu_gather.c | 2 ++
4 files changed, 37 insertions(+), 16 deletions(-)
diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h
index cb087315b4e72..b318e3e9e66a8 100644
--- a/include/asm-generic/tlb.h
+++ b/include/asm-generic/tlb.h
@@ -249,11 +249,7 @@ extern bool __tlb_remove_page_size(struct mmu_gather *tlb, struct page *page, in
bool __tlb_remove_folio_pages(struct mmu_gather *tlb, struct page *page,
unsigned int nr_pages, bool delay_rmap);
-/*
- * This both sets 'delayed_rmap', and returns true. It would be an inline
- * function, except we define it before the 'struct mmu_gather'.
- */
-#define tlb_delay_rmap(tlb) (((tlb)->delayed_rmap = 1), true)
+#define tlb_can_delay_rmap() (true)
extern void tlb_flush_rmaps(struct mmu_gather *tlb, struct vm_area_struct *vma);
#endif
@@ -263,8 +259,8 @@ extern void tlb_flush_rmaps(struct mmu_gather *tlb, struct vm_area_struct *vma);
* delay anything. That is used on S390, which flushes remote
* TLBs synchronously.
*/
-#ifndef tlb_delay_rmap
-#define tlb_delay_rmap(tlb) (false)
+#ifndef tlb_can_delay_rmap
+#define tlb_can_delay_rmap() (false)
static inline void tlb_flush_rmaps(struct mmu_gather *tlb, struct vm_area_struct *vma) { }
#endif
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 7791872f14af7..79d400ea9553d 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2508,10 +2508,12 @@ static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd)
mm_dec_nr_ptes(mm);
}
-static void zap_huge_pmd_folio(struct mm_struct *mm, struct vm_area_struct *vma,
- pmd_t pmdval, struct folio *folio, bool is_present)
+static void zap_huge_pmd_folio(struct mmu_gather *tlb, struct vm_area_struct *vma,
+ pmd_t pmdval, struct folio *folio, bool is_present, bool *force_flush)
{
const bool is_device_private = pmd_is_device_private_entry(pmdval);
+ struct mm_struct *mm = tlb->mm;
+ bool delay_rmap = false;
if (folio_test_anon(folio)) {
add_mm_counter(mm, MM_ANONPAGES, -HPAGE_PMD_NR);
@@ -2519,23 +2521,42 @@ static void zap_huge_pmd_folio(struct mm_struct *mm, struct vm_area_struct *vma,
add_mm_counter(mm, mm_counter_file(folio),
-HPAGE_PMD_NR);
- if (is_present && pmd_dirty(pmdval))
+ if (is_present && pmd_dirty(pmdval)) {
folio_mark_dirty(folio);
+ /*
+ * Remove rmap after the TLB flush if stale TLB entries
+ * could still be used to write to a mapping that
+ * supports writeback.
+ */
+ if (!tlb->fullmm && vma->vm_file &&
+ mapping_can_writeback(vma->vm_file->f_mapping))
+ delay_rmap = tlb_can_delay_rmap();
+ }
if (is_present && pmd_young(pmdval) &&
likely(vma_has_recency(vma)))
folio_mark_accessed(folio);
}
+ if (!is_present && !is_device_private)
+ return;
+
/*
* Present and device private folios are rmappable. Remove the
* rmap after transferring young+dirty to the folio.
*/
- if (is_present || is_device_private)
+ if (!delay_rmap)
folio_remove_rmap_pmd(folio, &folio->page, vma);
- /* Device private folios are pinned. */
+ /* Device private folios are pinned and don't need a TLB flush. */
if (is_device_private)
folio_put(folio);
+ else if (__tlb_remove_page_size(tlb, &folio->page, HPAGE_PMD_SIZE))
+ *force_flush = true;
+
+ if (delay_rmap) {
+ tlb_flush_mmu_tlbonly(tlb);
+ folio_remove_rmap_pmd(folio, &folio->page, vma);
+ }
}
static struct folio *normal_or_softleaf_folio_pmd(struct vm_area_struct *vma,
@@ -2573,6 +2594,7 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,
struct mm_struct *mm = tlb->mm;
struct folio *folio = NULL;
bool is_present = false;
+ bool force_flush = false;
bool has_deposit;
spinlock_t *ptl;
pmd_t orig_pmd;
@@ -2597,13 +2619,14 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,
folio = normal_or_softleaf_folio_pmd(vma, addr, orig_pmd, is_present);
has_deposit = vma_has_deposited_pgtable(vma);
if (folio)
- zap_huge_pmd_folio(mm, vma, orig_pmd, folio, is_present);
+ zap_huge_pmd_folio(tlb, vma, orig_pmd, folio, is_present,
+ &force_flush);
if (has_deposit)
zap_deposited_table(mm, pmd);
spin_unlock(ptl);
- if (is_present && folio)
- tlb_remove_page_size(tlb, &folio->page, HPAGE_PMD_SIZE);
+ if (force_flush)
+ tlb_flush_mmu(tlb);
return true;
}
diff --git a/mm/memory.c b/mm/memory.c
index 7f019afc35ca0..cd954afb146f4 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -1757,7 +1757,7 @@ static __always_inline void zap_present_folio_ptes(struct mmu_gather *tlb,
ptent = get_and_clear_full_ptes(mm, addr, pte, nr, tlb->fullmm);
if (pte_dirty(ptent)) {
folio_mark_dirty(folio);
- if (tlb_delay_rmap(tlb)) {
+ if (tlb_can_delay_rmap()) {
delay_rmap = true;
*force_flush = true;
}
diff --git a/mm/mmu_gather.c b/mm/mmu_gather.c
index c022aeb7c84d0..fc547be33a919 100644
--- a/mm/mmu_gather.c
+++ b/mm/mmu_gather.c
@@ -169,6 +169,8 @@ static bool __tlb_remove_folio_pages_size(struct mmu_gather *tlb,
struct mmu_gather_batch *batch;
VM_BUG_ON(!tlb->end);
+ if (delay_rmap)
+ tlb->delayed_rmap = 1;
#ifdef CONFIG_MMU_GATHER_PAGE_SIZE
VM_WARN_ON(tlb->page_size != page_size);
--
2.56.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH v2 1/2] mm/mmu_gather: delay rmap removal on UP as well
2026-11-16 15:06 ` [PATCH v2 1/2] mm/mmu_gather: delay rmap removal on UP as well David Hildenbrand (Arm)
@ 2026-11-16 15:16 ` Lorenzo Stoakes (ARM)
2026-11-16 15:18 ` David Hildenbrand (Arm)
2026-11-16 15:41 ` Zi Yan
2026-11-16 15:57 ` Kiryl Shutsemau (Meta)
2 siblings, 1 reply; 10+ messages in thread
From: Lorenzo Stoakes (ARM) @ 2026-11-16 15:16 UTC (permalink / raw)
To: David Hildenbrand (Arm)
Cc: Will Deacon, Aneesh Kumar K.V, Andrew Morton, Nick Piggin,
Peter Zijlstra, Arnd Bergmann, Johannes Weiner, Hugh Dickins,
Zi Yan, Baolin Wang, Liam R. Howlett, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Lance Yang, Usama Arif, Kiryl Shutsemau,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Jan Kara, Ross Zwisler, linux-arch, linux-mm, linux-kernel,
stable
On Mon, Nov 16, 2026 at 04:06:58PM +0100, David Hildenbrand (Arm) wrote:
> Commit 5df397dec7c4 ("mm: delay page_remove_rmap() until after the TLB
> has been flushed") introduced delayed rmap removal of dirty pages until
> after the TLB flush, so writeback cannot clean a page that is still
> writable through a stale TLB entry.
>
> It skips this on UP, because on UP there are no remote TLBs (single CPU)
> and because the PTL prevents preemption.
>
> The PTL prevents preemption while removing the rmap, but the local TLB
> flush is usually only performed after dropping the PTL.
>
> Consequently, after dropping the PTL we could reschedule to another
> thread of the same mm, which does not flush the TLB. A kworker could
> then run into the original race with writeback: it cleans the folio,
> but the other thread can still modify it through the stale TLB entries,
> and these writes get lost.
>
> Fix it by simply not special-casing UP. Now, only architectures that
> select MMU_GATHER_NO_GATHER (s390) will skip delay_rmap handling.
>
> Fixes: 5df397dec7c4 ("mm: delay page_remove_rmap() until after the TLB has been flushed")
> Cc: stable@vger.kernel.org
> Reported-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
> Link: https://lore.kernel.org/r/avrxOt_GyXLXNqtR@gremlin
Nitty, but I think better as Closes? :P
> Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
LGTM so:
Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
> ---
> include/asm-generic/tlb.h | 6 +-----
> mm/mmu_gather.c | 2 --
> 2 files changed, 1 insertion(+), 7 deletions(-)
>
> diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h
> index 4bba02fd5d543..cb087315b4e72 100644
> --- a/include/asm-generic/tlb.h
> +++ b/include/asm-generic/tlb.h
> @@ -249,23 +249,19 @@ extern bool __tlb_remove_page_size(struct mmu_gather *tlb, struct page *page, in
> bool __tlb_remove_folio_pages(struct mmu_gather *tlb, struct page *page,
> unsigned int nr_pages, bool delay_rmap);
>
> -#ifdef CONFIG_SMP
> /*
> * This both sets 'delayed_rmap', and returns true. It would be an inline
> * function, except we define it before the 'struct mmu_gather'.
> */
> #define tlb_delay_rmap(tlb) (((tlb)->delayed_rmap = 1), true)
> extern void tlb_flush_rmaps(struct mmu_gather *tlb, struct vm_area_struct *vma);
> -#endif
>
> #endif
>
> /*
> * We have a no-op version of the rmap removal that doesn't
> * delay anything. That is used on S390, which flushes remote
> - * TLBs synchronously, and on UP, which doesn't have any
> - * remote TLBs to flush and is not preemptible due to this
> - * all happening under the page table lock.
> + * TLBs synchronously.
> */
> #ifndef tlb_delay_rmap
> #define tlb_delay_rmap(tlb) (false)
> diff --git a/mm/mmu_gather.c b/mm/mmu_gather.c
> index e98397ac577c4..c022aeb7c84d0 100644
> --- a/mm/mmu_gather.c
> +++ b/mm/mmu_gather.c
> @@ -49,7 +49,6 @@ static bool tlb_next_batch(struct mmu_gather *tlb)
> return true;
> }
>
> -#ifdef CONFIG_SMP
> static void tlb_flush_rmap_batch(struct mmu_gather_batch *batch, struct vm_area_struct *vma)
> {
> struct encoded_page **pages = batch->encoded_pages;
> @@ -91,7 +90,6 @@ void tlb_flush_rmaps(struct mmu_gather *tlb, struct vm_area_struct *vma)
> tlb_flush_rmap_batch(tlb->active, vma);
> tlb->delayed_rmap = 0;
> }
> -#endif
>
> /*
> * We might end up freeing a lot of pages. Reschedule on a regular
>
> --
> 2.56.1
>
--
Cheers, Lorenzo
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2 1/2] mm/mmu_gather: delay rmap removal on UP as well
2026-11-16 15:16 ` Lorenzo Stoakes (ARM)
@ 2026-11-16 15:18 ` David Hildenbrand (Arm)
0 siblings, 0 replies; 10+ messages in thread
From: David Hildenbrand (Arm) @ 2026-11-16 15:18 UTC (permalink / raw)
To: Lorenzo Stoakes (ARM)
Cc: Will Deacon, Aneesh Kumar K.V, Andrew Morton, Nick Piggin,
Peter Zijlstra, Arnd Bergmann, Johannes Weiner, Hugh Dickins,
Zi Yan, Baolin Wang, Liam R. Howlett, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Lance Yang, Usama Arif, Kiryl Shutsemau,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Jan Kara, Ross Zwisler, linux-arch, linux-mm, linux-kernel,
stable
On 11/16/26 16:16, Lorenzo Stoakes (ARM) wrote:
> On Mon, Nov 16, 2026 at 04:06:58PM +0100, David Hildenbrand (Arm) wrote:
>> Commit 5df397dec7c4 ("mm: delay page_remove_rmap() until after the TLB
>> has been flushed") introduced delayed rmap removal of dirty pages until
>> after the TLB flush, so writeback cannot clean a page that is still
>> writable through a stale TLB entry.
>>
>> It skips this on UP, because on UP there are no remote TLBs (single CPU)
>> and because the PTL prevents preemption.
>>
>> The PTL prevents preemption while removing the rmap, but the local TLB
>> flush is usually only performed after dropping the PTL.
>>
>> Consequently, after dropping the PTL we could reschedule to another
>> thread of the same mm, which does not flush the TLB. A kworker could
>> then run into the original race with writeback: it cleans the folio,
>> but the other thread can still modify it through the stale TLB entries,
>> and these writes get lost.
>>
>> Fix it by simply not special-casing UP. Now, only architectures that
>> select MMU_GATHER_NO_GATHER (s390) will skip delay_rmap handling.
>>
>> Fixes: 5df397dec7c4 ("mm: delay page_remove_rmap() until after the TLB has been flushed")
>> Cc: stable@vger.kernel.org
>> Reported-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
>> Link: https://lore.kernel.org/r/avrxOt_GyXLXNqtR@gremlin
>
> Nitty, but I think better as Closes? :P
Ack, fixed it on #2 but forget it here ... why does checkpatch not complain? Weird.
Will change when picking :)
--
Cheers,
David
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2 2/2] mm/huge_memory: flush dirty file PMDs before removing rmap
2026-11-16 15:06 ` [PATCH v2 2/2] mm/huge_memory: flush dirty file PMDs before removing rmap David Hildenbrand (Arm)
@ 2026-11-16 15:19 ` Lorenzo Stoakes (ARM)
2026-11-16 16:10 ` Kiryl Shutsemau (Meta)
2026-11-16 16:45 ` Zi Yan
2 siblings, 0 replies; 10+ messages in thread
From: Lorenzo Stoakes (ARM) @ 2026-11-16 15:19 UTC (permalink / raw)
To: David Hildenbrand (Arm)
Cc: Will Deacon, Aneesh Kumar K.V, Andrew Morton, Nick Piggin,
Peter Zijlstra, Arnd Bergmann, Johannes Weiner, Hugh Dickins,
Zi Yan, Baolin Wang, Liam R. Howlett, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Lance Yang, Usama Arif, Kiryl Shutsemau,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Jan Kara, Ross Zwisler, linux-arch, linux-mm, linux-kernel,
stable, Chris Mason
On Mon, Nov 16, 2026 at 04:06:59PM +0100, David Hildenbrand (Arm) wrote:
> Unmapping a file PMD can lose concurrent stores when it races with
> writeback. zap_huge_pmd() removes rmap and drops the PMD lock before the
> gather flushes the TLB. Writeback can then clean the folio while another
> thread can still write through a stale writable TLB entry. Those stores
> do not fault and dirty the folio again. Filesystem DAX has the same
> problem with its dirty tags.
>
> Commit 5df397dec7c4 ("mm: delay page_remove_rmap() until after the TLB
> has been flushed") fixed the same race for PTEs by adding delayed rmap
> removal to the mmu_gather infrastructure. The PMD path still removes
> rmap before flushing.
>
> Flush the TLB under the PMD lock before removing rmap for dirty file
> PMDs. Skip the early flush at address-space teardown, when no users
> remain, and for mappings without writeback, including shmem and device
> DAX.
>
> Use tlb_can_delay_rmap() to share the PTE path's architecture checks.
> On s390, clearing the entry already handles TLB synchronization, and
> __tlb_remove_page_size() immediately drops the mapping reference, so
> rmap must be removed first. Make the predicate side-effect-free and set
> the gather's delayed_rmap flag when an entry is actually queued for
> delayed rmap removal.
>
> Queue the folio before the early flush, while the gather still has a
> valid range. Keep batching references and drain a full batch only after
> dropping the PMD lock. The early flush resets the gather's TLB state,
> so draining the batch does not flush that range again.
>
> Although writing to files through mmap is generally discouraged,
> unmapping such mappings may become slower: each dirty file PMD now
> requires a TLB flush under the PMD lock. Future work should properly
> extend the delayed-rmap machinery to PMDs and hold the PMD spinlock
> across multiple entries, so their TLB invalidations and rmap removals
> can be batched.
>
> Fixes: 4b4bb46d00b3 ("dax: clear dirty entry tags on cache flush")
> Cc: stable@vger.kernel.org
> Reported-by: Kiryl Shutsemau <kas@kernel.org>
> Closes: https://lore.kernel.org/linux-mm/aoWw_iKEDLjKL2jr@thinkstation/
> Reported-by: Chris Mason <mason@kernel.org>
> Link: https://git.kernel.org/linus/5df397dec7c4c08c23bd14f162f1228836faa4ce
> Suggested-by: Lance Yang <lance.yang@linux.dev>
> Suggested-by: Usama Arif <usama.arif@linux.dev>
> Reviewed-by: Lance Yang <lance.yang@linux.dev>
> Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
> Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
LGTM so:
Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
> ---
> include/asm-generic/tlb.h | 10 +++-------
> mm/huge_memory.c | 39 +++++++++++++++++++++++++++++++--------
> mm/memory.c | 2 +-
> mm/mmu_gather.c | 2 ++
> 4 files changed, 37 insertions(+), 16 deletions(-)
>
> diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h
> index cb087315b4e72..b318e3e9e66a8 100644
> --- a/include/asm-generic/tlb.h
> +++ b/include/asm-generic/tlb.h
> @@ -249,11 +249,7 @@ extern bool __tlb_remove_page_size(struct mmu_gather *tlb, struct page *page, in
> bool __tlb_remove_folio_pages(struct mmu_gather *tlb, struct page *page,
> unsigned int nr_pages, bool delay_rmap);
>
> -/*
> - * This both sets 'delayed_rmap', and returns true. It would be an inline
> - * function, except we define it before the 'struct mmu_gather'.
> - */
> -#define tlb_delay_rmap(tlb) (((tlb)->delayed_rmap = 1), true)
> +#define tlb_can_delay_rmap() (true)
> extern void tlb_flush_rmaps(struct mmu_gather *tlb, struct vm_area_struct *vma);
>
> #endif
> @@ -263,8 +259,8 @@ extern void tlb_flush_rmaps(struct mmu_gather *tlb, struct vm_area_struct *vma);
> * delay anything. That is used on S390, which flushes remote
> * TLBs synchronously.
> */
> -#ifndef tlb_delay_rmap
> -#define tlb_delay_rmap(tlb) (false)
> +#ifndef tlb_can_delay_rmap
> +#define tlb_can_delay_rmap() (false)
> static inline void tlb_flush_rmaps(struct mmu_gather *tlb, struct vm_area_struct *vma) { }
> #endif
>
> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index 7791872f14af7..79d400ea9553d 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -2508,10 +2508,12 @@ static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd)
> mm_dec_nr_ptes(mm);
> }
>
> -static void zap_huge_pmd_folio(struct mm_struct *mm, struct vm_area_struct *vma,
> - pmd_t pmdval, struct folio *folio, bool is_present)
> +static void zap_huge_pmd_folio(struct mmu_gather *tlb, struct vm_area_struct *vma,
> + pmd_t pmdval, struct folio *folio, bool is_present, bool *force_flush)
> {
> const bool is_device_private = pmd_is_device_private_entry(pmdval);
> + struct mm_struct *mm = tlb->mm;
> + bool delay_rmap = false;
>
> if (folio_test_anon(folio)) {
> add_mm_counter(mm, MM_ANONPAGES, -HPAGE_PMD_NR);
> @@ -2519,23 +2521,42 @@ static void zap_huge_pmd_folio(struct mm_struct *mm, struct vm_area_struct *vma,
> add_mm_counter(mm, mm_counter_file(folio),
> -HPAGE_PMD_NR);
>
> - if (is_present && pmd_dirty(pmdval))
> + if (is_present && pmd_dirty(pmdval)) {
> folio_mark_dirty(folio);
> + /*
> + * Remove rmap after the TLB flush if stale TLB entries
> + * could still be used to write to a mapping that
> + * supports writeback.
> + */
> + if (!tlb->fullmm && vma->vm_file &&
> + mapping_can_writeback(vma->vm_file->f_mapping))
> + delay_rmap = tlb_can_delay_rmap();
> + }
> if (is_present && pmd_young(pmdval) &&
> likely(vma_has_recency(vma)))
> folio_mark_accessed(folio);
> }
>
> + if (!is_present && !is_device_private)
> + return;
> +
> /*
> * Present and device private folios are rmappable. Remove the
> * rmap after transferring young+dirty to the folio.
> */
> - if (is_present || is_device_private)
> + if (!delay_rmap)
> folio_remove_rmap_pmd(folio, &folio->page, vma);
>
> - /* Device private folios are pinned. */
> + /* Device private folios are pinned and don't need a TLB flush. */
> if (is_device_private)
> folio_put(folio);
> + else if (__tlb_remove_page_size(tlb, &folio->page, HPAGE_PMD_SIZE))
> + *force_flush = true;
> +
> + if (delay_rmap) {
> + tlb_flush_mmu_tlbonly(tlb);
> + folio_remove_rmap_pmd(folio, &folio->page, vma);
> + }
> }
>
> static struct folio *normal_or_softleaf_folio_pmd(struct vm_area_struct *vma,
> @@ -2573,6 +2594,7 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,
> struct mm_struct *mm = tlb->mm;
> struct folio *folio = NULL;
> bool is_present = false;
> + bool force_flush = false;
> bool has_deposit;
> spinlock_t *ptl;
> pmd_t orig_pmd;
> @@ -2597,13 +2619,14 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,
> folio = normal_or_softleaf_folio_pmd(vma, addr, orig_pmd, is_present);
> has_deposit = vma_has_deposited_pgtable(vma);
> if (folio)
> - zap_huge_pmd_folio(mm, vma, orig_pmd, folio, is_present);
> + zap_huge_pmd_folio(tlb, vma, orig_pmd, folio, is_present,
> + &force_flush);
> if (has_deposit)
> zap_deposited_table(mm, pmd);
>
> spin_unlock(ptl);
> - if (is_present && folio)
> - tlb_remove_page_size(tlb, &folio->page, HPAGE_PMD_SIZE);
> + if (force_flush)
> + tlb_flush_mmu(tlb);
> return true;
> }
>
> diff --git a/mm/memory.c b/mm/memory.c
> index 7f019afc35ca0..cd954afb146f4 100644
> --- a/mm/memory.c
> +++ b/mm/memory.c
> @@ -1757,7 +1757,7 @@ static __always_inline void zap_present_folio_ptes(struct mmu_gather *tlb,
> ptent = get_and_clear_full_ptes(mm, addr, pte, nr, tlb->fullmm);
> if (pte_dirty(ptent)) {
> folio_mark_dirty(folio);
> - if (tlb_delay_rmap(tlb)) {
> + if (tlb_can_delay_rmap()) {
> delay_rmap = true;
> *force_flush = true;
> }
> diff --git a/mm/mmu_gather.c b/mm/mmu_gather.c
> index c022aeb7c84d0..fc547be33a919 100644
> --- a/mm/mmu_gather.c
> +++ b/mm/mmu_gather.c
> @@ -169,6 +169,8 @@ static bool __tlb_remove_folio_pages_size(struct mmu_gather *tlb,
> struct mmu_gather_batch *batch;
>
> VM_BUG_ON(!tlb->end);
> + if (delay_rmap)
> + tlb->delayed_rmap = 1;
>
> #ifdef CONFIG_MMU_GATHER_PAGE_SIZE
> VM_WARN_ON(tlb->page_size != page_size);
>
> --
> 2.56.1
>
--
Cheers, Lorenzo
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2 1/2] mm/mmu_gather: delay rmap removal on UP as well
2026-11-16 15:06 ` [PATCH v2 1/2] mm/mmu_gather: delay rmap removal on UP as well David Hildenbrand (Arm)
2026-11-16 15:16 ` Lorenzo Stoakes (ARM)
@ 2026-11-16 15:41 ` Zi Yan
2026-11-16 15:57 ` Kiryl Shutsemau (Meta)
2 siblings, 0 replies; 10+ messages in thread
From: Zi Yan @ 2026-11-16 15:41 UTC (permalink / raw)
To: David Hildenbrand (Arm)
Cc: Will Deacon, Aneesh Kumar K.V, Andrew Morton, Nick Piggin,
Peter Zijlstra, Arnd Bergmann, Johannes Weiner, Hugh Dickins,
Baolin Wang, Liam R. Howlett, Nico Pache, Ryan Roberts, Dev Jain,
Barry Song, Lance Yang, Usama Arif, Kiryl Shutsemau,
Lorenzo Stoakes, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Jan Kara, Ross Zwisler,
linux-arch, linux-mm, linux-kernel, stable
On 16 Nov 2026, at 10:06, David Hildenbrand (Arm) wrote:
> Commit 5df397dec7c4 ("mm: delay page_remove_rmap() until after the TLB
> has been flushed") introduced delayed rmap removal of dirty pages until
> after the TLB flush, so writeback cannot clean a page that is still
> writable through a stale TLB entry.
>
> It skips this on UP, because on UP there are no remote TLBs (single CPU)
> and because the PTL prevents preemption.
>
> The PTL prevents preemption while removing the rmap, but the local TLB
> flush is usually only performed after dropping the PTL.
>
> Consequently, after dropping the PTL we could reschedule to another
> thread of the same mm, which does not flush the TLB. A kworker could
> then run into the original race with writeback: it cleans the folio,
> but the other thread can still modify it through the stale TLB entries,
> and these writes get lost.
>
> Fix it by simply not special-casing UP. Now, only architectures that
> select MMU_GATHER_NO_GATHER (s390) will skip delay_rmap handling.
>
> Fixes: 5df397dec7c4 ("mm: delay page_remove_rmap() until after the TLB has been flushed")
> Cc: stable@vger.kernel.org
> Reported-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
> Link: https://lore.kernel.org/r/avrxOt_GyXLXNqtR@gremlin
> Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
> ---
> include/asm-generic/tlb.h | 6 +-----
> mm/mmu_gather.c | 2 --
> 2 files changed, 1 insertion(+), 7 deletions(-)
>
Makes sense to me.
Acked-by: Zi Yan <ziy@nvidia.com>
Best Regards,
Yan, Zi
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2 1/2] mm/mmu_gather: delay rmap removal on UP as well
2026-11-16 15:06 ` [PATCH v2 1/2] mm/mmu_gather: delay rmap removal on UP as well David Hildenbrand (Arm)
2026-11-16 15:16 ` Lorenzo Stoakes (ARM)
2026-11-16 15:41 ` Zi Yan
@ 2026-11-16 15:57 ` Kiryl Shutsemau (Meta)
2 siblings, 0 replies; 10+ messages in thread
From: Kiryl Shutsemau (Meta) @ 2026-11-16 15:57 UTC (permalink / raw)
To: David Hildenbrand (Arm)
Cc: Will Deacon, Aneesh Kumar K.V, Andrew Morton, Nick Piggin,
Peter Zijlstra, Arnd Bergmann, Johannes Weiner, Hugh Dickins,
Zi Yan, Baolin Wang, Liam R. Howlett, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Lance Yang, Usama Arif, Lorenzo Stoakes,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Jan Kara, Ross Zwisler, linux-arch, linux-mm, linux-kernel,
stable
On Mon, Nov 16, 2026 at 04:06:58PM +0100, David Hildenbrand (Arm) wrote:
> Commit 5df397dec7c4 ("mm: delay page_remove_rmap() until after the TLB
> has been flushed") introduced delayed rmap removal of dirty pages until
> after the TLB flush, so writeback cannot clean a page that is still
> writable through a stale TLB entry.
>
> It skips this on UP, because on UP there are no remote TLBs (single CPU)
> and because the PTL prevents preemption.
>
> The PTL prevents preemption while removing the rmap, but the local TLB
> flush is usually only performed after dropping the PTL.
>
> Consequently, after dropping the PTL we could reschedule to another
> thread of the same mm, which does not flush the TLB. A kworker could
> then run into the original race with writeback: it cleans the folio,
> but the other thread can still modify it through the stale TLB entries,
> and these writes get lost.
>
> Fix it by simply not special-casing UP. Now, only architectures that
> select MMU_GATHER_NO_GATHER (s390) will skip delay_rmap handling.
>
> Fixes: 5df397dec7c4 ("mm: delay page_remove_rmap() until after the TLB has been flushed")
> Cc: stable@vger.kernel.org
> Reported-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
> Link: https://lore.kernel.org/r/avrxOt_GyXLXNqtR@gremlin
> Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Acked-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
--
Kiryl Shutsemau / Kirill A. Shutemov
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2 2/2] mm/huge_memory: flush dirty file PMDs before removing rmap
2026-11-16 15:06 ` [PATCH v2 2/2] mm/huge_memory: flush dirty file PMDs before removing rmap David Hildenbrand (Arm)
2026-11-16 15:19 ` Lorenzo Stoakes (ARM)
@ 2026-11-16 16:10 ` Kiryl Shutsemau (Meta)
2026-11-16 16:45 ` Zi Yan
2 siblings, 0 replies; 10+ messages in thread
From: Kiryl Shutsemau (Meta) @ 2026-11-16 16:10 UTC (permalink / raw)
To: David Hildenbrand (Arm), Jani Nikula, Joonas Lahtinen,
Tvrtko Ursulin, Andi Shyti
Cc: Will Deacon, Aneesh Kumar K.V, Andrew Morton, Nick Piggin,
Peter Zijlstra, Arnd Bergmann, Johannes Weiner, Hugh Dickins,
Zi Yan, Baolin Wang, Liam R. Howlett, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Lance Yang, Usama Arif, Lorenzo Stoakes,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Jan Kara, Ross Zwisler, linux-arch, linux-mm, linux-kernel,
stable, Chris Mason
On Mon, Nov 16, 2026 at 04:06:59PM +0100, David Hildenbrand (Arm) wrote:
> Unmapping a file PMD can lose concurrent stores when it races with
> writeback. zap_huge_pmd() removes rmap and drops the PMD lock before the
> gather flushes the TLB. Writeback can then clean the folio while another
> thread can still write through a stale writable TLB entry. Those stores
> do not fault and dirty the folio again. Filesystem DAX has the same
> problem with its dirty tags.
>
> Commit 5df397dec7c4 ("mm: delay page_remove_rmap() until after the TLB
> has been flushed") fixed the same race for PTEs by adding delayed rmap
> removal to the mmu_gather infrastructure. The PMD path still removes
> rmap before flushing.
>
> Flush the TLB under the PMD lock before removing rmap for dirty file
> PMDs. Skip the early flush at address-space teardown, when no users
> remain, and for mappings without writeback, including shmem and device
> DAX.
The patch looks good and you can use
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
But my LLM machinery flagged that __shmem_writeback() might need
fixing. Normal shmem doesn't have writeback, but i915 does its own, and
it is subject to the race.
+i915 folks.
Possible fix:
-- >8 --
From: "Kiryl Shutsemau (Meta)" <kas@kernel.org>
Subject: [PATCH] drm/i915/gem: Skip writeback of shmem folios with extra
references
__shmem_writeback() writes out every dirty folio that is not mapped. A
folio stops being mapped when munmap() removes its rmap, but the TLB
flush comes later. Until then another thread can still store to it
through a stale TLB entry:
CPU0 (munmap) CPU1 (same mm) CPU2 (i915 shrinker)
zap_huge_pmd()
folio_remove_rmap_pmd()
__shmem_writeback()
clear dirty
!folio_mapped()
shmem_write_folio()
store via stale TLB
tlb_finish_mmu()
The store leaves the folio clean in the swap cache. Once reclaim frees
it, the object reads back from swap without that store.
The legacy mmap ioctl maps the shmem file directly, and i915 mounts it
with huge=within_size on some platforms, so the folio can be PMD-mapped.
The core mm keeps the rmap until after the flush for PTEs, but not for
shmem PMDs: shmem has no writeback, and pageout() refuses a folio with
an extra reference.
Do the same here. The unmapping side holds its reference until the TLB
is flushed, so a folio with references beyond the page cache and
writeback_iter() is left to reclaim, as a mapped one already is.
Fixes: 2d6692e642e7 ("drm/i915: Start writeback from the shrinker")
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
---
drivers/gpu/drm/i915/gem/i915_gem_shmem.c | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c
index bf9e7480f0f5..748772c358ac 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c
@@ -316,9 +316,15 @@ void __shmem_writeback(size_t size, struct address_space *mapping)
* leaving only CPU mmapings around) and add those folios to the LRU
* instead of invoking writeback so they are aged and paged out
* as normal.
+ *
+ * The same goes for a folio with references beyond the page cache
+ * and writeback_iter(). A folio that was just unmapped can still be
+ * written through a stale TLB entry until the unmapping side flushes
+ * the TLB and drops its reference. pageout() checks the same.
*/
while ((folio = writeback_iter(mapping, &wbc, folio, &error))) {
- if (folio_mapped(folio))
+ if (folio_mapped(folio) ||
+ folio_ref_count(folio) != 1 + folio_nr_pages(folio))
folio_redirty_for_writepage(&wbc, folio);
else
error = shmem_write_folio(folio);
--
Kiryl Shutsemau / Kirill A. Shutemov
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH v2 2/2] mm/huge_memory: flush dirty file PMDs before removing rmap
2026-11-16 15:06 ` [PATCH v2 2/2] mm/huge_memory: flush dirty file PMDs before removing rmap David Hildenbrand (Arm)
2026-11-16 15:19 ` Lorenzo Stoakes (ARM)
2026-11-16 16:10 ` Kiryl Shutsemau (Meta)
@ 2026-11-16 16:45 ` Zi Yan
2 siblings, 0 replies; 10+ messages in thread
From: Zi Yan @ 2026-11-16 16:45 UTC (permalink / raw)
To: David Hildenbrand (Arm)
Cc: Will Deacon, Aneesh Kumar K.V, Andrew Morton, Nick Piggin,
Peter Zijlstra, Arnd Bergmann, Johannes Weiner, Hugh Dickins,
Baolin Wang, Liam R. Howlett, Nico Pache, Ryan Roberts, Dev Jain,
Barry Song, Lance Yang, Usama Arif, Kiryl Shutsemau,
Lorenzo Stoakes, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Jan Kara, Ross Zwisler,
linux-arch, linux-mm, linux-kernel, stable, Chris Mason
On 16 Nov 2026, at 10:06, David Hildenbrand (Arm) wrote:
> Unmapping a file PMD can lose concurrent stores when it races with
> writeback. zap_huge_pmd() removes rmap and drops the PMD lock before the
> gather flushes the TLB. Writeback can then clean the folio while another
> thread can still write through a stale writable TLB entry. Those stores
> do not fault and dirty the folio again. Filesystem DAX has the same
> problem with its dirty tags.
>
> Commit 5df397dec7c4 ("mm: delay page_remove_rmap() until after the TLB
> has been flushed") fixed the same race for PTEs by adding delayed rmap
> removal to the mmu_gather infrastructure. The PMD path still removes
> rmap before flushing.
>
> Flush the TLB under the PMD lock before removing rmap for dirty file
> PMDs. Skip the early flush at address-space teardown, when no users
> remain, and for mappings without writeback, including shmem and device
> DAX.
>
> Use tlb_can_delay_rmap() to share the PTE path's architecture checks.
> On s390, clearing the entry already handles TLB synchronization, and
> __tlb_remove_page_size() immediately drops the mapping reference, so
> rmap must be removed first. Make the predicate side-effect-free and set
> the gather's delayed_rmap flag when an entry is actually queued for
> delayed rmap removal.
>
> Queue the folio before the early flush, while the gather still has a
> valid range. Keep batching references and drain a full batch only after
> dropping the PMD lock. The early flush resets the gather's TLB state,
> so draining the batch does not flush that range again.
>
> Although writing to files through mmap is generally discouraged,
> unmapping such mappings may become slower: each dirty file PMD now
> requires a TLB flush under the PMD lock. Future work should properly
> extend the delayed-rmap machinery to PMDs and hold the PMD spinlock
> across multiple entries, so their TLB invalidations and rmap removals
> can be batched.
>
> Fixes: 4b4bb46d00b3 ("dax: clear dirty entry tags on cache flush")
> Cc: stable@vger.kernel.org
> Reported-by: Kiryl Shutsemau <kas@kernel.org>
> Closes: https://lore.kernel.org/linux-mm/aoWw_iKEDLjKL2jr@thinkstation/
> Reported-by: Chris Mason <mason@kernel.org>
> Link: https://git.kernel.org/linus/5df397dec7c4c08c23bd14f162f1228836faa4ce
> Suggested-by: Lance Yang <lance.yang@linux.dev>
> Suggested-by: Usama Arif <usama.arif@linux.dev>
> Reviewed-by: Lance Yang <lance.yang@linux.dev>
> Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
> Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
> ---
> include/asm-generic/tlb.h | 10 +++-------
> mm/huge_memory.c | 39 +++++++++++++++++++++++++++++++--------
> mm/memory.c | 2 +-
> mm/mmu_gather.c | 2 ++
> 4 files changed, 37 insertions(+), 16 deletions(-)
>
The fix looks good to me.
Reviewed-by: Zi Yan <ziy@nvidia.com>
Best Regards,
Yan, Zi
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2026-11-16 16:45 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-16 15:06 [PATCH v2 0/2] mm/huge_memory: flush dirty file PMDs before removing rmap David Hildenbrand (Arm)
2026-11-16 15:06 ` [PATCH v2 1/2] mm/mmu_gather: delay rmap removal on UP as well David Hildenbrand (Arm)
2026-11-16 15:16 ` Lorenzo Stoakes (ARM)
2026-11-16 15:18 ` David Hildenbrand (Arm)
2026-11-16 15:41 ` Zi Yan
2026-11-16 15:57 ` Kiryl Shutsemau (Meta)
2026-11-16 15:06 ` [PATCH v2 2/2] mm/huge_memory: flush dirty file PMDs before removing rmap David Hildenbrand (Arm)
2026-11-16 15:19 ` Lorenzo Stoakes (ARM)
2026-11-16 16:10 ` Kiryl Shutsemau (Meta)
2026-11-16 16:45 ` Zi Yan
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox