[PATCH] mm/huge_memory: transfer the pmd dirty bit to the folio on zap

[PATCH] mm/huge_memory: transfer the pmd dirty bit to the folio on zap

[Usama Arif]

zap_huge_pmd_folio() propagates the pmd young bit to the folio for the
file case, but not the dirty bit.  The pte path does propagate it, in
zap_present_folio_ptes() and so does the pmd split path, in
__split_huge_pmd_locked().

For most file mappings the omission is harmless, because writing to a
shared file mapping goes through page_mkwrite(), which dirties the
folio.  tmpfs is different: it has no page_mkwrite(), and
vma_wants_writenotify() is false for it, so a *read* fault on a
MAP_SHARED tmpfs mapping installs a writable pmd via do_read_fault().
do_read_fault() does not call fault_dirty_shared_page(), so subsequent
stores through that mapping set only the hardware dirty bit in the pmd
and never call folio_mark_dirty().  A shmem folio allocated by a fault
is marked uptodate but not dirty (see the clear: block in
shmem_get_folio_gfp()), so PG_dirty is never set at all.

Unmapping such a folio - munmap(), or exit_mmap() when the process dies
- then loses the only record that it was written, because zap_huge_pmd()
drops the pmd without transferring the dirty bit.  Reclaim afterwards
sees a clean shmem folio: the whole swap-out block in
shrink_folio_list() is inside "if (folio_test_dirty(folio))", so
pageout() is skipped and the folio falls into __remove_mapping().
There, folio_is_file_lru() is false for a swapbacked folio, so no shadow
entry is created and __filemap_remove_folio(folio, NULL) simply empties
the i_pages slot.  The data is freed without ever being written to swap,
and the next fault on that index returns a freshly zeroed folio.

This is silent data loss for any process that keeps state in a
MAP_SHARED tmpfs segment across an unmap - for example a cache handed
from one process generation to the next through /dev/shm.  It requires
the folio to be PMD-mapped, so it only shows up once shmem THP is
enabled (which is what we did in Meta fleet and started noticing crashes);
with THP off the pte path transfers the dirty bit correctly.
It also only becomes visible when swap is enabled, because with no swap
device shmem folios (which are on the anon LRU) are not scanned by
reclaim at all, so the clean folio is never dropped.

Reproduced on x86_64 with a tmpfs mounted huge=within_size: read-fault a
2MB-backed region, write a known pattern through the resulting mapping,
munmap, force reclaim of the cgroup, then re-map and read back.  Without
this patch the region reads back as zeros and vmstat shows zswpout 0 -
the data was discarded rather than swapped.  With this patch the region
reads back correctly and the pages are swapped out as expected.  With
huge=never, or when the first touch is a write, the test passes either
way.

Fixes: 800d8c63b2e9 ("shmem: add huge pages support")
Cc: <stable@vger.kernel.org>
Signed-off-by: Usama Arif <usama.arif@linux.dev>
---
 mm/huge_memory.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index ced400f72d43a..afbb5974bd225 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2449,6 +2449,8 @@ static void zap_huge_pmd_folio(struct mm_struct *mm, struct vm_area_struct *vma,
 		add_mm_counter(mm, mm_counter_file(folio),
 			       -HPAGE_PMD_NR);
 
+		if (is_present && pmd_dirty(pmdval))
+			folio_mark_dirty(folio);
 		if (is_present && pmd_young(pmdval) &&
 		    likely(vma_has_recency(vma)))
 			folio_mark_accessed(folio);
-- 
2.53.0-Meta

Re: [PATCH] mm/huge_memory: transfer the pmd dirty bit to the folio on zap

[David Hildenbrand (Arm)]

On 8/19/26 12:12, Usama Arif wrote:
> zap_huge_pmd_folio() propagates the pmd young bit to the folio for the
> file case, but not the dirty bit.  The pte path does propagate it, in
> zap_present_folio_ptes() and so does the pmd split path, in
> __split_huge_pmd_locked().
> 
> For most file mappings the omission is harmless, because writing to a
> shared file mapping goes through page_mkwrite(), which dirties the
> folio.  tmpfs is different: it has no page_mkwrite(), and
> vma_wants_writenotify() is false for it, so a *read* fault on a
> MAP_SHARED tmpfs mapping installs a writable pmd via do_read_fault().
> do_read_fault() does not call fault_dirty_shared_page(), so subsequent
> stores through that mapping set only the hardware dirty bit in the pmd
> and never call folio_mark_dirty().  A shmem folio allocated by a fault
> is marked uptodate but not dirty (see the clear: block in
> shmem_get_folio_gfp()), so PG_dirty is never set at all.
> 
> Unmapping such a folio - munmap(), or exit_mmap() when the process dies
> - then loses the only record that it was written, because zap_huge_pmd()
> drops the pmd without transferring the dirty bit.  Reclaim afterwards
> sees a clean shmem folio: the whole swap-out block in
> shrink_folio_list() is inside "if (folio_test_dirty(folio))", so
> pageout() is skipped and the folio falls into __remove_mapping().
> There, folio_is_file_lru() is false for a swapbacked folio, so no shadow
> entry is created and __filemap_remove_folio(folio, NULL) simply empties
> the i_pages slot.  The data is freed without ever being written to swap,
> and the next fault on that index returns a freshly zeroed folio.
> 
> This is silent data loss for any process that keeps state in a
> MAP_SHARED tmpfs segment across an unmap - for example a cache handed
> from one process generation to the next through /dev/shm.  It requires
> the folio to be PMD-mapped, so it only shows up once shmem THP is
> enabled (which is what we did in Meta fleet and started noticing crashes);
> with THP off the pte path transfers the dirty bit correctly.
> It also only becomes visible when swap is enabled, because with no swap
> device shmem folios (which are on the anon LRU) are not scanned by
> reclaim at all, so the clean folio is never dropped.
> 
> Reproduced on x86_64 with a tmpfs mounted huge=within_size: read-fault a
> 2MB-backed region, write a known pattern through the resulting mapping,
> munmap, force reclaim of the cgroup, then re-map and read back.  Without
> this patch the region reads back as zeros and vmstat shows zswpout 0 -
> the data was discarded rather than swapped.  With this patch the region
> reads back correctly and the pages are swapped out as expected.  With
> huge=never, or when the first touch is a write, the test passes either
> way.
> 
> Fixes: 800d8c63b2e9 ("shmem: add huge pages support")
> Cc: <stable@vger.kernel.org>
> Signed-off-by: Usama Arif <usama.arif@linux.dev>
> ---
>  mm/huge_memory.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index ced400f72d43a..afbb5974bd225 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -2449,6 +2449,8 @@ static void zap_huge_pmd_folio(struct mm_struct *mm, struct vm_area_struct *vma,
>  		add_mm_counter(mm, mm_counter_file(folio),
>  			       -HPAGE_PMD_NR);
>  
> +		if (is_present && pmd_dirty(pmdval))
> +			folio_mark_dirty(folio);

We don't need that for anon folios, though. So best to just resemble
zap_present_folio_ptes() logic and do it only for !anon folios?

-- 
Cheers,

David

Re: [PATCH] mm/huge_memory: transfer the pmd dirty bit to the folio on zap

[David Hildenbrand (Arm)]

On 8/19/26 16:13, David Hildenbrand (Arm) wrote:
> On 8/19/26 12:12, Usama Arif wrote:
>> zap_huge_pmd_folio() propagates the pmd young bit to the folio for the
>> file case, but not the dirty bit.  The pte path does propagate it, in
>> zap_present_folio_ptes() and so does the pmd split path, in
>> __split_huge_pmd_locked().
>>
>> For most file mappings the omission is harmless, because writing to a
>> shared file mapping goes through page_mkwrite(), which dirties the
>> folio.  tmpfs is different: it has no page_mkwrite(), and
>> vma_wants_writenotify() is false for it, so a *read* fault on a
>> MAP_SHARED tmpfs mapping installs a writable pmd via do_read_fault().
>> do_read_fault() does not call fault_dirty_shared_page(), so subsequent
>> stores through that mapping set only the hardware dirty bit in the pmd
>> and never call folio_mark_dirty().  A shmem folio allocated by a fault
>> is marked uptodate but not dirty (see the clear: block in
>> shmem_get_folio_gfp()), so PG_dirty is never set at all.
>>
>> Unmapping such a folio - munmap(), or exit_mmap() when the process dies
>> - then loses the only record that it was written, because zap_huge_pmd()
>> drops the pmd without transferring the dirty bit.  Reclaim afterwards
>> sees a clean shmem folio: the whole swap-out block in
>> shrink_folio_list() is inside "if (folio_test_dirty(folio))", so
>> pageout() is skipped and the folio falls into __remove_mapping().
>> There, folio_is_file_lru() is false for a swapbacked folio, so no shadow
>> entry is created and __filemap_remove_folio(folio, NULL) simply empties
>> the i_pages slot.  The data is freed without ever being written to swap,
>> and the next fault on that index returns a freshly zeroed folio.
>>
>> This is silent data loss for any process that keeps state in a
>> MAP_SHARED tmpfs segment across an unmap - for example a cache handed
>> from one process generation to the next through /dev/shm.  It requires
>> the folio to be PMD-mapped, so it only shows up once shmem THP is
>> enabled (which is what we did in Meta fleet and started noticing crashes);
>> with THP off the pte path transfers the dirty bit correctly.
>> It also only becomes visible when swap is enabled, because with no swap
>> device shmem folios (which are on the anon LRU) are not scanned by
>> reclaim at all, so the clean folio is never dropped.
>>
>> Reproduced on x86_64 with a tmpfs mounted huge=within_size: read-fault a
>> 2MB-backed region, write a known pattern through the resulting mapping,
>> munmap, force reclaim of the cgroup, then re-map and read back.  Without
>> this patch the region reads back as zeros and vmstat shows zswpout 0 -
>> the data was discarded rather than swapped.  With this patch the region
>> reads back correctly and the pages are swapped out as expected.  With
>> huge=never, or when the first touch is a write, the test passes either
>> way.
>>
>> Fixes: 800d8c63b2e9 ("shmem: add huge pages support")
>> Cc: <stable@vger.kernel.org>
>> Signed-off-by: Usama Arif <usama.arif@linux.dev>
>> ---
>>  mm/huge_memory.c | 2 ++
>>  1 file changed, 2 insertions(+)
>>
>> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
>> index ced400f72d43a..afbb5974bd225 100644
>> --- a/mm/huge_memory.c
>> +++ b/mm/huge_memory.c
>> @@ -2449,6 +2449,8 @@ static void zap_huge_pmd_folio(struct mm_struct *mm, struct vm_area_struct *vma,
>>  		add_mm_counter(mm, mm_counter_file(folio),
>>  			       -HPAGE_PMD_NR);
>>  
>> +		if (is_present && pmd_dirty(pmdval))
>> +			folio_mark_dirty(folio);
> 
> We don't need that for anon folios, though. So best to just resemble
> zap_present_folio_ptes() logic and do it only for !anon folios?
> 

I'm stupid, that check is not visible in the diff above :)

Thanks!

Acked-by: David Hildenbrand (Arm) <david@kernel.org>

-- 
Cheers,

David

Re: [PATCH] mm/huge_memory: transfer the pmd dirty bit to the folio on zap

[Kiryl Shutsemau]

On Wed, Aug 19, 2026 at 03:12:22AM -0700, Usama Arif wrote:
> zap_huge_pmd_folio() propagates the pmd young bit to the folio for the
> file case, but not the dirty bit.  The pte path does propagate it, in
> zap_present_folio_ptes() and so does the pmd split path, in
> __split_huge_pmd_locked().
> 
> For most file mappings the omission is harmless, because writing to a
> shared file mapping goes through page_mkwrite(), which dirties the
> folio.  tmpfs is different: it has no page_mkwrite(), and
> vma_wants_writenotify() is false for it, so a *read* fault on a
> MAP_SHARED tmpfs mapping installs a writable pmd via do_read_fault().
> do_read_fault() does not call fault_dirty_shared_page(), so subsequent
> stores through that mapping set only the hardware dirty bit in the pmd
> and never call folio_mark_dirty().
>
> A shmem folio allocated by a fault
> is marked uptodate but not dirty (see the clear: block in
> shmem_get_folio_gfp()), so PG_dirty is never set at all.
> 
> Unmapping such a folio - munmap(), or exit_mmap() when the process dies
> - then loses the only record that it was written, because zap_huge_pmd()
> drops the pmd without transferring the dirty bit.  Reclaim afterwards
> sees a clean shmem folio: the whole swap-out block in
> shrink_folio_list() is inside "if (folio_test_dirty(folio))", so
> pageout() is skipped and the folio falls into __remove_mapping().
> There, folio_is_file_lru() is false for a swapbacked folio, so no shadow
> entry is created and __filemap_remove_folio(folio, NULL) simply empties
> the i_pages slot.  The data is freed without ever being written to swap,
> and the next fault on that index returns a freshly zeroed folio.
> 
> This is silent data loss for any process that keeps state in a
> MAP_SHARED tmpfs segment across an unmap - for example a cache handed
> from one process generation to the next through /dev/shm.  It requires
> the folio to be PMD-mapped, so it only shows up once shmem THP is
> enabled (which is what we did in Meta fleet and started noticing crashes);
> with THP off the pte path transfers the dirty bit correctly.
> It also only becomes visible when swap is enabled, because with no swap
> device shmem folios (which are on the anon LRU) are not scanned by
> reclaim at all, so the clean folio is never dropped.
> 
> Reproduced on x86_64 with a tmpfs mounted huge=within_size: read-fault a
> 2MB-backed region, write a known pattern through the resulting mapping,
> munmap, force reclaim of the cgroup, then re-map and read back.  Without
> this patch the region reads back as zeros and vmstat shows zswpout 0 -
> the data was discarded rather than swapped.  With this patch the region
> reads back correctly and the pages are swapped out as expected.  With
> huge=never, or when the first touch is a write, the test passes either
> way.

+Hugh.

Oopsie.

I'm confused why it took a decade to discover the bug...
Maybe read ahead of write for shmem is too rare, I donno.

> 
> Fixes: 800d8c63b2e9 ("shmem: add huge pages support")

This would be more precise: b5072380eb61 ("thp: support file pages in zap_huge_pmd()")

Reviewed-by: Kiryl Shutsemau <kas@kernel.org>

> Cc: <stable@vger.kernel.org>
> Signed-off-by: Usama Arif <usama.arif@linux.dev>
> ---
>  mm/huge_memory.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index ced400f72d43a..afbb5974bd225 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -2449,6 +2449,8 @@ static void zap_huge_pmd_folio(struct mm_struct *mm, struct vm_area_struct *vma,
>  		add_mm_counter(mm, mm_counter_file(folio),
>  			       -HPAGE_PMD_NR);
>  
> +		if (is_present && pmd_dirty(pmdval))
> +			folio_mark_dirty(folio);

Unrelated to your patch, but noticed while looking at it: we drop the rmap
here under the pmd lock, while the TLB flush is deferred to
tlb_finish_mmu(). The pte path handles this with
tlb_delay_rmap()/force_flush (5df397dec7c4), but there's no pmd equivalent:
tlb_flush_rmap_batch() only knows folio_remove_rmap_ptes(), and
zap_huge_pmd() uses tlb_remove_page_size(), which takes no delay_rmap.

Doesn't matter for shmem, but xfs & friends do get PMD-order folios, and
do_set_pmd() makes the pmd dirty+writable once page_mkwrite() has run. So
folio_mkclean() can clean the folio while another CPU still stores through a
stale TLB entry -- silently lost write, no PG_dirty left behind.

I think we need to fix this too.

Wanna give it a try?

>  		if (is_present && pmd_young(pmdval) &&
>  		    likely(vma_has_recency(vma)))
>  			folio_mark_accessed(folio);
> -- 
> 2.53.0-Meta
> 

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: [PATCH] mm/huge_memory: transfer the pmd dirty bit to the folio on zap

[Lance Yang]

On Wed, Aug 19, 2026 at 03:31:40PM +0100, Kiryl Shutsemau wrote:
>On Wed, Aug 19, 2026 at 03:12:22AM -0700, Usama Arif wrote:
[...]
>> ---
>>  mm/huge_memory.c | 2 ++
>>  1 file changed, 2 insertions(+)
>> 
>> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
>> index ced400f72d43a..afbb5974bd225 100644
>> --- a/mm/huge_memory.c
>> +++ b/mm/huge_memory.c
>> @@ -2449,6 +2449,8 @@ static void zap_huge_pmd_folio(struct mm_struct *mm, struct vm_area_struct *vma,
>>  		add_mm_counter(mm, mm_counter_file(folio),
>>  			       -HPAGE_PMD_NR);
>>  
>> +		if (is_present && pmd_dirty(pmdval))
>> +			folio_mark_dirty(folio);
>
>Unrelated to your patch, but noticed while looking at it: we drop the rmap
>here under the pmd lock, while the TLB flush is deferred to
>tlb_finish_mmu(). The pte path handles this with
>tlb_delay_rmap()/force_flush (5df397dec7c4), but there's no pmd equivalent:
>tlb_flush_rmap_batch() only knows folio_remove_rmap_ptes(), and
>zap_huge_pmd() uses tlb_remove_page_size(), which takes no delay_rmap.

Well spotted!

>Doesn't matter for shmem, but xfs & friends do get PMD-order folios, and

Right. pageout() cannot pass its refcount check while PMD mapping still
holds an extra folio ref, and mmu_gather drops that ref only after TLB
flush.

>do_set_pmd() makes the pmd dirty+writable once page_mkwrite() has run. So
>folio_mkclean() can clean the folio while another CPU still stores through a
>stale TLB entry -- silently lost write, no PG_dirty left behind.

Yep. Writeback can run folio_mkclean() while that ref is still held,
though, and with rmap already gone it misses the PMD ...

>I think we need to fix this too.

+1

>Wanna give it a try?

zap_huge_pmd() only handles one PMD under PTL anyway ... how about just
flushing before folio_remove_rmap_pmd()?

---8<---
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index afbb5974bd22..6fb34924ef66 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2531,6 +2531,14 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,
 	is_present = pmd_present(orig_pmd);
 	folio = normal_or_softleaf_folio_pmd(vma, addr, orig_pmd, is_present);
 	has_deposit = has_deposited_pgtable(vma, orig_pmd, folio);
+	/*
+	 * folio_mkclean() relies on the rmap to find writable mappings.
+	 * Flush stale TLB entries before removing it below.
+	 */
+	if (folio && is_present && !folio_test_anon(folio) &&
+	    pmd_dirty(orig_pmd))
+		tlb_flush_mmu_tlbonly(tlb);
+
 	if (folio)
 		zap_huge_pmd_folio(mm, vma, orig_pmd, folio, is_present);
 	if (has_deposit)
---

Cheers, Lance

>
>>  		if (is_present && pmd_young(pmdval) &&
>>  		    likely(vma_has_recency(vma)))
>>  			folio_mark_accessed(folio);
>> -- 
>> 2.53.0-Meta
>> 
>
>-- 
>  Kiryl Shutsemau / Kirill A. Shutemov
>

Re: [PATCH] mm/huge_memory: transfer the pmd dirty bit to the folio on zap

[Usama Arif]

On 19/08/2026 17:17, Lance Yang wrote:
> 
> On Wed, Aug 19, 2026 at 03:31:40PM +0100, Kiryl Shutsemau wrote:
>> On Wed, Aug 19, 2026 at 03:12:22AM -0700, Usama Arif wrote:
> [...]
>>> ---
>>>  mm/huge_memory.c | 2 ++
>>>  1 file changed, 2 insertions(+)
>>>
>>> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
>>> index ced400f72d43a..afbb5974bd225 100644
>>> --- a/mm/huge_memory.c
>>> +++ b/mm/huge_memory.c
>>> @@ -2449,6 +2449,8 @@ static void zap_huge_pmd_folio(struct mm_struct *mm, struct vm_area_struct *vma,
>>>  		add_mm_counter(mm, mm_counter_file(folio),
>>>  			       -HPAGE_PMD_NR);
>>>  
>>> +		if (is_present && pmd_dirty(pmdval))
>>> +			folio_mark_dirty(folio);
>>
>> Unrelated to your patch, but noticed while looking at it: we drop the rmap
>> here under the pmd lock, while the TLB flush is deferred to
>> tlb_finish_mmu(). The pte path handles this with
>> tlb_delay_rmap()/force_flush (5df397dec7c4), but there's no pmd equivalent:
>> tlb_flush_rmap_batch() only knows folio_remove_rmap_ptes(), and
>> zap_huge_pmd() uses tlb_remove_page_size(), which takes no delay_rmap.
> 
> Well spotted!
> 
>> Doesn't matter for shmem, but xfs & friends do get PMD-order folios, and
> 
> Right. pageout() cannot pass its refcount check while PMD mapping still
> holds an extra folio ref, and mmu_gather drops that ref only after TLB
> flush.
> 
>> do_set_pmd() makes the pmd dirty+writable once page_mkwrite() has run. So
>> folio_mkclean() can clean the folio while another CPU still stores through a
>> stale TLB entry -- silently lost write, no PG_dirty left behind.
> 
> Yep. Writeback can run folio_mkclean() while that ref is still held,
> though, and with rmap already gone it misses the PMD ...
> 
>> I think we need to fix this too.
> 
> +1
> 
>> Wanna give it a try?
> 
> zap_huge_pmd() only handles one PMD under PTL anyway ... how about just
> flushing before folio_remove_rmap_pmd()?
> 
> ---8<---
> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index afbb5974bd22..6fb34924ef66 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -2531,6 +2531,14 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,
>  	is_present = pmd_present(orig_pmd);
>  	folio = normal_or_softleaf_folio_pmd(vma, addr, orig_pmd, is_present);
>  	has_deposit = has_deposited_pgtable(vma, orig_pmd, folio);
> +	/*
> +	 * folio_mkclean() relies on the rmap to find writable mappings.
> +	 * Flush stale TLB entries before removing it below.
> +	 */
> +	if (folio && is_present && !folio_test_anon(folio) &&
> +	    pmd_dirty(orig_pmd))
> +		tlb_flush_mmu_tlbonly(tlb);
> +
>  	if (folio)
>  		zap_huge_pmd_folio(mm, vma, orig_pmd, folio, is_present);
>  	if (has_deposit)
> ---


I am currently at below to reduce tlb flushes, but still WIP


diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h
index bdcc2778ac64f..60bdd6287b5a9 100644
--- a/include/asm-generic/tlb.h
+++ b/include/asm-generic/tlb.h
@@ -301,6 +301,12 @@ bool __tlb_remove_folio_pages(struct mmu_gather *tlb, struct page *page,
  * function, except we define it before the 'struct mmu_gather'.
  */
 #define tlb_delay_rmap(tlb) (((tlb)->delayed_rmap = 1), true)
+/*
+ * Like tlb_delay_rmap() but without the side effect, for callers that must
+ * flush rather than delay: can another CPU still reach this mapping through a
+ * stale TLB entry once its rmap entry is gone? Not during fullmm teardown.
+ */
+#define tlb_rmap_needs_flush(tlb) (!(tlb)->fullmm)
 extern void tlb_flush_rmaps(struct mmu_gather *tlb, struct vm_area_struct *vma);
 #endif

@@ -315,6 +321,7 @@ extern void tlb_flush_rmaps(struct mmu_gather *tlb, struct vm_area_struct *vma);
  */
 #ifndef tlb_delay_rmap
 #define tlb_delay_rmap(tlb) (false)
+#define tlb_rmap_needs_flush(tlb) (false)
 static inline void tlb_flush_rmaps(struct mmu_gather *tlb, struct vm_area_struct *vma) { }
 #endif

diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index afbb5974bd225..76d8d5cf92ee0 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2493,6 +2493,33 @@ static bool has_deposited_pgtable(struct vm_area_struct *vma, pmd_t pmdval,
        return folio && folio_test_anon(folio);
 }


+static bool pmd_zap_needs_tlb_flush(struct mmu_gather *tlb, pmd_t pmdval,
+                                   struct folio *folio, bool is_present)
+{
+       struct address_space *mapping;
+
+       if (!is_present || !pmd_dirty(pmdval) || folio_test_anon(folio))
+               return false;
+       if (!tlb_rmap_needs_flush(tlb))
+               return false;
+
+       mapping = folio_mapping(folio);
+       return mapping && mapping_can_writeback(mapping);
+}
+
 /**
  * zap_huge_pmd - Zap a huge THP which is of PMD size.
  * @tlb: The MMU gather TLB state associated with the operation.
@@ -2531,8 +2558,15 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,
        is_present = pmd_present(orig_pmd);
        folio = normal_or_softleaf_folio_pmd(vma, addr, orig_pmd, is_present);
        has_deposit = has_deposited_pgtable(vma, orig_pmd, folio);
-       if (folio)
+       if (folio) {
+               /* Flush before zap_huge_pmd_folio() drops the rmap entry. */
+               if (pmd_zap_needs_tlb_flush(tlb, orig_pmd, folio, is_present)) {
+                       tlb_flush_mmu_tlbonly(tlb);
+                       /* Re-arm: tlb_remove_page_size() needs tlb->end set. */
+                       tlb_remove_pmd_tlb_entry(tlb, pmd, addr);
+               }
                zap_huge_pmd_folio(mm, vma, orig_pmd, folio, is_present);
+       }
        if (has_deposit)
                zap_deposited_table(mm, pmd);

> 
> Cheers, Lance
> 
>>
>>>  		if (is_present && pmd_young(pmdval) &&
>>>  		    likely(vma_has_recency(vma)))
>>>  			folio_mark_accessed(folio);
>>> -- 
>>> 2.53.0-Meta
>>>
>>
>> -- 
>>  Kiryl Shutsemau / Kirill A. Shutemov
>>

Re: [PATCH] mm/huge_memory: transfer the pmd dirty bit to the folio on zap

[Pedro Falcato]

On Wed, Aug 19, 2026 at 03:31:40PM +0100, Kiryl Shutsemau wrote:
> On Wed, Aug 19, 2026 at 03:12:22AM -0700, Usama Arif wrote:
> > zap_huge_pmd_folio() propagates the pmd young bit to the folio for the
> > file case, but not the dirty bit.  The pte path does propagate it, in
> > zap_present_folio_ptes() and so does the pmd split path, in
> > __split_huge_pmd_locked().
> > 
> > For most file mappings the omission is harmless, because writing to a
> > shared file mapping goes through page_mkwrite(), which dirties the
> > folio.  tmpfs is different: it has no page_mkwrite(), and
> > vma_wants_writenotify() is false for it, so a *read* fault on a
> > MAP_SHARED tmpfs mapping installs a writable pmd via do_read_fault().
> > do_read_fault() does not call fault_dirty_shared_page(), so subsequent
> > stores through that mapping set only the hardware dirty bit in the pmd
> > and never call folio_mark_dirty().
> >
> > A shmem folio allocated by a fault
> > is marked uptodate but not dirty (see the clear: block in
> > shmem_get_folio_gfp()), so PG_dirty is never set at all.
> > 
> > Unmapping such a folio - munmap(), or exit_mmap() when the process dies
> > - then loses the only record that it was written, because zap_huge_pmd()
> > drops the pmd without transferring the dirty bit.  Reclaim afterwards
> > sees a clean shmem folio: the whole swap-out block in
> > shrink_folio_list() is inside "if (folio_test_dirty(folio))", so
> > pageout() is skipped and the folio falls into __remove_mapping().
> > There, folio_is_file_lru() is false for a swapbacked folio, so no shadow
> > entry is created and __filemap_remove_folio(folio, NULL) simply empties
> > the i_pages slot.  The data is freed without ever being written to swap,
> > and the next fault on that index returns a freshly zeroed folio.
> > 
> > This is silent data loss for any process that keeps state in a
> > MAP_SHARED tmpfs segment across an unmap - for example a cache handed
> > from one process generation to the next through /dev/shm.  It requires
> > the folio to be PMD-mapped, so it only shows up once shmem THP is
> > enabled (which is what we did in Meta fleet and started noticing crashes);
> > with THP off the pte path transfers the dirty bit correctly.
> > It also only becomes visible when swap is enabled, because with no swap
> > device shmem folios (which are on the anon LRU) are not scanned by
> > reclaim at all, so the clean folio is never dropped.
> > 
> > Reproduced on x86_64 with a tmpfs mounted huge=within_size: read-fault a
> > 2MB-backed region, write a known pattern through the resulting mapping,
> > munmap, force reclaim of the cgroup, then re-map and read back.  Without
> > this patch the region reads back as zeros and vmstat shows zswpout 0 -
> > the data was discarded rather than swapped.  With this patch the region
> > reads back correctly and the pages are swapped out as expected.  With
> > huge=never, or when the first touch is a write, the test passes either
> > way.
> 
> +Hugh.
> 
> Oopsie.
> 
> I'm confused why it took a decade to discover the bug...
> Maybe read ahead of write for shmem is too rare, I donno.
> 
> > 
> > Fixes: 800d8c63b2e9 ("shmem: add huge pages support")
> 
> This would be more precise: b5072380eb61 ("thp: support file pages in zap_huge_pmd()")
> 
> Reviewed-by: Kiryl Shutsemau <kas@kernel.org>
> 
> > Cc: <stable@vger.kernel.org>
> > Signed-off-by: Usama Arif <usama.arif@linux.dev>
> > ---
> >  mm/huge_memory.c | 2 ++
> >  1 file changed, 2 insertions(+)
> > 
> > diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> > index ced400f72d43a..afbb5974bd225 100644
> > --- a/mm/huge_memory.c
> > +++ b/mm/huge_memory.c
> > @@ -2449,6 +2449,8 @@ static void zap_huge_pmd_folio(struct mm_struct *mm, struct vm_area_struct *vma,
> >  		add_mm_counter(mm, mm_counter_file(folio),
> >  			       -HPAGE_PMD_NR);
> >  
> > +		if (is_present && pmd_dirty(pmdval))
> > +			folio_mark_dirty(folio);
> 
> Unrelated to your patch, but noticed while looking at it: we drop the rmap
> here under the pmd lock, while the TLB flush is deferred to
> tlb_finish_mmu(). The pte path handles this with
> tlb_delay_rmap()/force_flush (5df397dec7c4), but there's no pmd equivalent:
> tlb_flush_rmap_batch() only knows folio_remove_rmap_ptes(), and
> zap_huge_pmd() uses tlb_remove_page_size(), which takes no delay_rmap.
> 
> Doesn't matter for shmem, but xfs & friends do get PMD-order folios, and
> do_set_pmd() makes the pmd dirty+writable once page_mkwrite() has run. So
> folio_mkclean() can clean the folio while another CPU still stores through a
> stale TLB entry -- silently lost write, no PG_dirty left behind.

Where do you see page_mkwrite being called in the same path as do_set_pmd()?
Per my understanding of the code, this Should Not Happen, and it really Should
Not Happen for many, many reasons (write amplification being the main one).

Namely, see the comment in wp_huge_pmd():
	/* COW or write-notify handled on pte level: split pmd. */

if file huge pages get mapped writable, that's a bug.
 

-- 
Pedro

Re: [PATCH] mm/huge_memory: transfer the pmd dirty bit to the folio on zap

[Hugh Dickins]

On Wed, 19 Aug 2026, Kiryl Shutsemau wrote:
> On Wed, Aug 19, 2026 at 03:12:22AM -0700, Usama Arif wrote:
> > zap_huge_pmd_folio() propagates the pmd young bit to the folio for the
> > file case, but not the dirty bit.  The pte path does propagate it, in
> > zap_present_folio_ptes() and so does the pmd split path, in
> > __split_huge_pmd_locked().
> > 
> > For most file mappings the omission is harmless, because writing to a
> > shared file mapping goes through page_mkwrite(), which dirties the
> > folio.  tmpfs is different: it has no page_mkwrite(), and
> > vma_wants_writenotify() is false for it, so a *read* fault on a
> > MAP_SHARED tmpfs mapping installs a writable pmd via do_read_fault().
> > do_read_fault() does not call fault_dirty_shared_page(), so subsequent
> > stores through that mapping set only the hardware dirty bit in the pmd
> > and never call folio_mark_dirty().
> >
> > A shmem folio allocated by a fault
> > is marked uptodate but not dirty (see the clear: block in
> > shmem_get_folio_gfp()), so PG_dirty is never set at all.
> > 
> > Unmapping such a folio - munmap(), or exit_mmap() when the process dies
> > - then loses the only record that it was written, because zap_huge_pmd()
> > drops the pmd without transferring the dirty bit.  Reclaim afterwards
> > sees a clean shmem folio: the whole swap-out block in
> > shrink_folio_list() is inside "if (folio_test_dirty(folio))", so
> > pageout() is skipped and the folio falls into __remove_mapping().
> > There, folio_is_file_lru() is false for a swapbacked folio, so no shadow
> > entry is created and __filemap_remove_folio(folio, NULL) simply empties
> > the i_pages slot.  The data is freed without ever being written to swap,
> > and the next fault on that index returns a freshly zeroed folio.
> > 
> > This is silent data loss for any process that keeps state in a
> > MAP_SHARED tmpfs segment across an unmap - for example a cache handed
> > from one process generation to the next through /dev/shm.  It requires
> > the folio to be PMD-mapped, so it only shows up once shmem THP is
> > enabled (which is what we did in Meta fleet and started noticing crashes);
> > with THP off the pte path transfers the dirty bit correctly.
> > It also only becomes visible when swap is enabled, because with no swap
> > device shmem folios (which are on the anon LRU) are not scanned by
> > reclaim at all, so the clean folio is never dropped.
> > 
> > Reproduced on x86_64 with a tmpfs mounted huge=within_size: read-fault a
> > 2MB-backed region, write a known pattern through the resulting mapping,
> > munmap, force reclaim of the cgroup, then re-map and read back.  Without
> > this patch the region reads back as zeros and vmstat shows zswpout 0 -
> > the data was discarded rather than swapped.  With this patch the region
> > reads back correctly and the pages are swapped out as expected.  With
> > huge=never, or when the first touch is a write, the test passes either
> > way.
> 
> +Hugh.
> 
> Oopsie.

How ghastly! Thanks for finding and fixing, Usama.

> 
> I'm confused why it took a decade to discover the bug...
> Maybe read ahead of write for shmem is too rare, I donno.

It isn't entirely clear from the report, but this is all about modifying
a 2MiB+ *hole* in a shmem file through an mmap thereof, with first fault
a read fault not a write fault.  I suppose only a few proceed in that way
(though truncating an empty file to some size and then mmap'ing that size
is very normal).

Or everybody who tried to report this bug, wrote their report into a
2MiB hole in a shmem file through an mmap thereof.

Other than holes, all shmem folios are dirty throughout (and re-marked
dirty as soon as brought back from swap): so for most, it doesn't matter
what the pmd says.

This raised a dim memory, took a while to locate what I was remembering:
e1f1b1572e8d ("mm/huge_memory.c: fix data loss when splitting a file pmd")
from 2018.

Not quite the same; but what a pity that one didn't prompt any of us
to look further and find what Usama now has.

> 
> > 
> > Fixes: 800d8c63b2e9 ("shmem: add huge pages support")
> 
> This would be more precise: b5072380eb61 ("thp: support file pages in zap_huge_pmd()")
> 
> Reviewed-by: Kiryl Shutsemau <kas@kernel.org>
> 
> > Cc: <stable@vger.kernel.org>
> > Signed-off-by: Usama Arif <usama.arif@linux.dev>

Acked-by: Hugh Dickins <hughd@google.com>

Re: [PATCH] mm/huge_memory: transfer the pmd dirty bit to the folio on zap

[Lance Yang]

On Wed, Aug 19, 2026 at 03:12:22AM -0700, Usama Arif wrote:
>zap_huge_pmd_folio() propagates the pmd young bit to the folio for the
>file case, but not the dirty bit.  The pte path does propagate it, in
>zap_present_folio_ptes() and so does the pmd split path, in
>__split_huge_pmd_locked().
>
>For most file mappings the omission is harmless, because writing to a
>shared file mapping goes through page_mkwrite(), which dirties the
>folio.  tmpfs is different: it has no page_mkwrite(), and
>vma_wants_writenotify() is false for it, so a *read* fault on a
>MAP_SHARED tmpfs mapping installs a writable pmd via do_read_fault().
>do_read_fault() does not call fault_dirty_shared_page(), so subsequent
>stores through that mapping set only the hardware dirty bit in the pmd
>and never call folio_mark_dirty().  A shmem folio allocated by a fault
>is marked uptodate but not dirty (see the clear: block in
>shmem_get_folio_gfp()), so PG_dirty is never set at all.
>
>Unmapping such a folio - munmap(), or exit_mmap() when the process dies
>- then loses the only record that it was written, because zap_huge_pmd()
>drops the pmd without transferring the dirty bit.  Reclaim afterwards
>sees a clean shmem folio: the whole swap-out block in
>shrink_folio_list() is inside "if (folio_test_dirty(folio))", so
>pageout() is skipped and the folio falls into __remove_mapping().
>There, folio_is_file_lru() is false for a swapbacked folio, so no shadow
>entry is created and __filemap_remove_folio(folio, NULL) simply empties
>the i_pages slot.  The data is freed without ever being written to swap,
>and the next fault on that index returns a freshly zeroed folio.
>
>This is silent data loss for any process that keeps state in a
>MAP_SHARED tmpfs segment across an unmap - for example a cache handed
>from one process generation to the next through /dev/shm.  It requires
>the folio to be PMD-mapped, so it only shows up once shmem THP is
>enabled (which is what we did in Meta fleet and started noticing crashes);
>with THP off the pte path transfers the dirty bit correctly.
>It also only becomes visible when swap is enabled, because with no swap
>device shmem folios (which are on the anon LRU) are not scanned by
>reclaim at all, so the clean folio is never dropped.
>
>Reproduced on x86_64 with a tmpfs mounted huge=within_size: read-fault a
>2MB-backed region, write a known pattern through the resulting mapping,
>munmap, force reclaim of the cgroup, then re-map and read back.  Without
>this patch the region reads back as zeros and vmstat shows zswpout 0 -
>the data was discarded rather than swapped.  With this patch the region
>reads back correctly and the pages are swapped out as expected.  With
>huge=never, or when the first touch is a write, the test passes either
>way.
>
>Fixes: 800d8c63b2e9 ("shmem: add huge pages support")
>Cc: <stable@vger.kernel.org>
>Signed-off-by: Usama Arif <usama.arif@linux.dev>
>---

Good catch!

Tested-by: Lance Yang <lance.yang@linux.dev>

Re: [PATCH] mm/huge_memory: transfer the pmd dirty bit to the folio on zap

[Zi Yan]

On Wed Aug 19, 2026 at 6:12 AM EDT, Usama Arif wrote:
> zap_huge_pmd_folio() propagates the pmd young bit to the folio for the
> file case, but not the dirty bit.  The pte path does propagate it, in
> zap_present_folio_ptes() and so does the pmd split path, in
> __split_huge_pmd_locked().
>
> For most file mappings the omission is harmless, because writing to a
> shared file mapping goes through page_mkwrite(), which dirties the
> folio.  tmpfs is different: it has no page_mkwrite(), and
> vma_wants_writenotify() is false for it, so a *read* fault on a
> MAP_SHARED tmpfs mapping installs a writable pmd via do_read_fault().
> do_read_fault() does not call fault_dirty_shared_page(), so subsequent
> stores through that mapping set only the hardware dirty bit in the pmd
> and never call folio_mark_dirty().  A shmem folio allocated by a fault
> is marked uptodate but not dirty (see the clear: block in
> shmem_get_folio_gfp()), so PG_dirty is never set at all.
>
> Unmapping such a folio - munmap(), or exit_mmap() when the process dies
> - then loses the only record that it was written, because zap_huge_pmd()
> drops the pmd without transferring the dirty bit.  Reclaim afterwards
> sees a clean shmem folio: the whole swap-out block in
> shrink_folio_list() is inside "if (folio_test_dirty(folio))", so
> pageout() is skipped and the folio falls into __remove_mapping().
> There, folio_is_file_lru() is false for a swapbacked folio, so no shadow
> entry is created and __filemap_remove_folio(folio, NULL) simply empties
> the i_pages slot.  The data is freed without ever being written to swap,
> and the next fault on that index returns a freshly zeroed folio.
>
> This is silent data loss for any process that keeps state in a
> MAP_SHARED tmpfs segment across an unmap - for example a cache handed
> from one process generation to the next through /dev/shm.  It requires
> the folio to be PMD-mapped, so it only shows up once shmem THP is
> enabled (which is what we did in Meta fleet and started noticing crashes);
> with THP off the pte path transfers the dirty bit correctly.
> It also only becomes visible when swap is enabled, because with no swap
> device shmem folios (which are on the anon LRU) are not scanned by
> reclaim at all, so the clean folio is never dropped.
>
> Reproduced on x86_64 with a tmpfs mounted huge=within_size: read-fault a
> 2MB-backed region, write a known pattern through the resulting mapping,
> munmap, force reclaim of the cgroup, then re-map and read back.  Without
> this patch the region reads back as zeros and vmstat shows zswpout 0 -
> the data was discarded rather than swapped.  With this patch the region
> reads back correctly and the pages are swapped out as expected.  With
> huge=never, or when the first touch is a write, the test passes either
> way.
>
> Fixes: 800d8c63b2e9 ("shmem: add huge pages support")
> Cc: <stable@vger.kernel.org>
> Signed-off-by: Usama Arif <usama.arif@linux.dev>
> ---
>  mm/huge_memory.c | 2 ++
>  1 file changed, 2 insertions(+)
>
Make sense. Thanks.

Reviewed-by: Zi Yan <ziy@nvidia.com>

-- 
Best Regards,
Yan, Zi

Re: [PATCH] mm/huge_memory: transfer the pmd dirty bit to the folio on zap

[Lorenzo Stoakes (ARM)]

On Wed, Aug 19, 2026 at 03:12:22AM -0700, Usama Arif wrote:
> zap_huge_pmd_folio() propagates the pmd young bit to the folio for the
> file case, but not the dirty bit.  The pte path does propagate it, in
> zap_present_folio_ptes() and so does the pmd split path, in
> __split_huge_pmd_locked().
>
> For most file mappings the omission is harmless, because writing to a
> shared file mapping goes through page_mkwrite(), which dirties the
> folio.  tmpfs is different: it has no page_mkwrite(), and
> vma_wants_writenotify() is false for it, so a *read* fault on a
> MAP_SHARED tmpfs mapping installs a writable pmd via do_read_fault().
> do_read_fault() does not call fault_dirty_shared_page(), so subsequent
> stores through that mapping set only the hardware dirty bit in the pmd
> and never call folio_mark_dirty().  A shmem folio allocated by a fault
> is marked uptodate but not dirty (see the clear: block in
> shmem_get_folio_gfp()), so PG_dirty is never set at all.
>
> Unmapping such a folio - munmap(), or exit_mmap() when the process dies
> - then loses the only record that it was written, because zap_huge_pmd()
> drops the pmd without transferring the dirty bit.  Reclaim afterwards
> sees a clean shmem folio: the whole swap-out block in
> shrink_folio_list() is inside "if (folio_test_dirty(folio))", so
> pageout() is skipped and the folio falls into __remove_mapping().
> There, folio_is_file_lru() is false for a swapbacked folio, so no shadow
> entry is created and __filemap_remove_folio(folio, NULL) simply empties
> the i_pages slot.  The data is freed without ever being written to swap,
> and the next fault on that index returns a freshly zeroed folio.
>
> This is silent data loss for any process that keeps state in a
> MAP_SHARED tmpfs segment across an unmap - for example a cache handed
> from one process generation to the next through /dev/shm.  It requires
> the folio to be PMD-mapped, so it only shows up once shmem THP is
> enabled (which is what we did in Meta fleet and started noticing crashes);
> with THP off the pte path transfers the dirty bit correctly.
> It also only becomes visible when swap is enabled, because with no swap
> device shmem folios (which are on the anon LRU) are not scanned by
> reclaim at all, so the clean folio is never dropped.

Good lord.

>
> Reproduced on x86_64 with a tmpfs mounted huge=within_size: read-fault a
> 2MB-backed region, write a known pattern through the resulting mapping,
> munmap, force reclaim of the cgroup, then re-map and read back.  Without
> this patch the region reads back as zeros and vmstat shows zswpout 0 -
> the data was discarded rather than swapped.  With this patch the region
> reads back correctly and the pages are swapped out as expected.  With
> huge=never, or when the first touch is a write, the test passes either
> way.
>
> Fixes: 800d8c63b2e9 ("shmem: add huge pages support")
> Cc: <stable@vger.kernel.org>
> Signed-off-by: Usama Arif <usama.arif@linux.dev>

Thanks for this! LGTM so:

Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>

> ---
>  mm/huge_memory.c | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index ced400f72d43a..afbb5974bd225 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -2449,6 +2449,8 @@ static void zap_huge_pmd_folio(struct mm_struct *mm, struct vm_area_struct *vma,
>  		add_mm_counter(mm, mm_counter_file(folio),
>  			       -HPAGE_PMD_NR);
>
> +		if (is_present && pmd_dirty(pmdval))
> +			folio_mark_dirty(folio);
>  		if (is_present && pmd_young(pmdval) &&
>  		    likely(vma_has_recency(vma)))
>  			folio_mark_accessed(folio);
> --
> 2.53.0-Meta
>

--
Cheers, Lorenzo