Linux-mm Archive on lore.kernel.org
 help / color / mirror / Atom feed
From: "David Hildenbrand (Arm)" <david@kernel.org>
To: "Lorenzo Stoakes" <ljs@kernel.org>,
	"Liam R. Howlett" <liam@infradead.org>,
	"Vlastimil Babka" <vbabka@kernel.org>,
	"Borislav Petkov (AMD)" <bp@alien8.de>,
	"Cenk Yılmaz" <cenk@deepfinresearch.com>,
	"Zhang Peng" <zippermonkey@icloud.com>,
	"Mike Rapoport" <rppt@kernel.org>,
	"Suren Baghdasaryan" <surenb@google.com>,
	"Michal Hocko" <mhocko@suse.com>,
	"Andrew Morton" <akpm@linux-foundation.org>,
	"Zi Yan" <ziy@nvidia.com>,
	"Matthew Brost" <matthew.brost@intel.com>,
	"Joshua Hahn" <joshua.hahnjy@gmail.com>,
	"Byungchul Park" <byungchul@sk.com>,
	"Gregory Price" <gourry@gourry.net>,
	"Ying Huang" <ying.huang@linux.alibaba.com>,
	"Alistair Popple" <apopple@nvidia.com>,
	"Chris Li" <chrisl@kernel.org>,
	"Kairui Song" <kasong@tencent.com>,
	"Kemeng Shi" <shikemeng@huaweicloud.com>,
	"Nhat Pham" <nphamcs@gmail.com>,
	"Baoquan He" <baoquan.he@linux.dev>,
	"Barry Song" <baohua@kernel.org>,
	"Youngjun Park" <youngjun.park@lge.com>,
	"Mel Gorman" <mgorman@suse.de>,
	"Sasha Levin" <sasha.levin@oracle.com>,
	"Rik van Riel" <riel@redhat.com>,
	"Ingo Molnar" <mingo@kernel.org>, "Peter Xu" <peterx@redhat.com>,
	"Hugh Dickins" <hughd@google.com>,
	"Axel Rasmussen" <axelrasmussen@google.com>,
	"Johannes Weiner" <hannes@cmpxchg.org>,
	"Michal Hocko" <mhocko@kernel.org>,
	"Qi Zheng" <qi.zheng@linux.dev>,
	"Shakeel Butt" <shakeel.butt@linux.dev>,
	"Yuanchu Xie" <yuanchu@google.com>, "Wei Xu" <weixugc@google.com>,
	"Baolin Wang" <baolin.wang@linux.alibaba.com>,
	"Andrea Arcangeli" <aarcange@redhat.com>,
	"Thomas Gleixner" <tglx@kernel.org>
Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org,
	 stable@vger.kernel.org,
	"David Hildenbrand (Arm)" <david@kernel.org>
Subject: [PATCH 3/4] RFC: mm/rmap: don't defer TLB flushes when unmapping lazyfree folios
Date: Wed, 18 Nov 2026 10:20:38 +0100	[thread overview]
Message-ID: <20261118-file_folio_flush-v1-3-e9e739ed14e0@kernel.org> (raw)
In-Reply-To: <20261118-file_folio_flush-v1-0-e9e739ed14e0@kernel.org>

When reclaim discards a lazyfree folio, it clears the PTE but delays the
TLB flush. Until flushed, mapping a new folio into the now-none PTE
can leave some CPUs still reading from the old folio while others
already write to the new folio.

This is hard to trigger in practice and would mostly look like a weird
use-after-free bug in an allocator that uses MADV_FREE, but it can be
reproduced reliably with three threads pinned to different CPUs:

(1) Thread A populates the pages, MADV_FREE's them and then
    MADV_PAGEOUT's them.
(2) Thread B reads all pages to keep TLB entries cached and verifies the
    content of pages published by thread C.
(3) Thread C waits until mincore() indicates that a page is no longer
    present, writes a new value to it, and publishes it to thread B.

Thread B can then read the old value after the new folio was mapped and
the content updated.

One way to handle it would be adding conditional flushes to
do_anonymous_page(), migrate_vma_insert_page(), UFFDIO_COPY and
UFFDIO_MOVE ... essentially anywhere we install new anonymous folios,
which would mean that these places flush quite frequently on concurrent
reclaim.

To perform targeted flushes, we'd have to keep some PTE markers behind,
which is also rather nasty and not an easy fix.

So let's not defer the TLB flush for lazyfree folios, which includes
folios in VM_DROPPABLE mappings.

To avoid flushing if the folio was redirtied or has additional
references, perform all non-deferred TLB flushes only once we know that
we won't restore the PTEs, right before removing the rmap.

The downside is that we no longer batch TLB flushes when discarding
lazyfree folios, which harms performance, but is the right thing to do
for now.

Fixes: 854e9ed09ded ("mm: support madvise(MADV_FREE)")
Cc: stable@vger.kernel.org
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
---
 mm/rmap.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/mm/rmap.c b/mm/rmap.c
index 09c33b9ec5850..3935eed23a361 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -2228,6 +2228,7 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,
 	struct mmu_notifier_range range;
 	enum ttu_flags flags = (enum ttu_flags)(long)arg;
 	unsigned long nr_pages = 1, end_addr;
+	bool flush;
 	unsigned long pfn;
 	int ptes = 0;
 
@@ -2253,6 +2254,7 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,
 
 	while (page_vma_mapped_walk(&pvmw)) {
 		nr_pages = 1;
+		flush = false;
 
 		/*
 		 * If the folio is in an mlock()d vma, we must not swap it out.
@@ -2339,11 +2341,18 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,
 			 * architecture must guarantee that a clear->dirty
 			 * transition on a cached TLB entry is written through
 			 * and traps if the PTE is unmapped.
+			 *
+			 * If we don't defer the flush, we flush only once we
+			 * know that we won't restore the PTEs below. We never
+			 * defer the flush for lazyfree folios: if we discard
+			 * them, the PTE will be none, and a page fault could
+			 * map a new folio while stale TLB entries still exist.
 			 */
-			if (should_defer_flush(mm, flags))
+			if (!folio_test_lazyfree(folio) &&
+			    should_defer_flush(mm, flags))
 				set_tlb_ubc_flush_pending(mm, pteval, address, end_addr);
 			else
-				flush_tlb_range(vma, address, end_addr);
+				flush = true;
 			if (pte_dirty(pteval))
 				folio_mark_dirty(folio);
 		} else {
@@ -2402,6 +2411,8 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,
 			add_mm_counter(mm, mm_counter_file(folio), -nr_pages);
 		}
 finish_unmap:
+		if (flush)
+			flush_tlb_range(vma, address, end_addr);
 		folio_remove_rmap_ptes(folio, page, nr_pages, vma);
 		if (vma_test_any_mask(vma, VMA_LOCKED_MASK))
 			mlock_drain_local();

-- 
2.56.1



  parent reply	other threads:[~2026-11-18  9:21 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-18  9:20 [PATCH 0/4] mm: fixes for stale TLB entries due to delayed/batched TLB flushes David Hildenbrand (Arm)
2026-11-18  9:20 ` [PATCH 1/4] mm: flush stale TLB entries before installing a CoW'ed folio David Hildenbrand (Arm)
2026-11-18  9:20 ` [PATCH 2/4] mm/vmscan: flush deferred writable TLB entries before unlocking the folio David Hildenbrand (Arm)
2026-11-18  9:20 ` David Hildenbrand (Arm) [this message]
2026-11-18  9:20 ` [PATCH 4/4] RFC: mm/vmscan: unlock unmapped folios only after the deferred TLB flush David Hildenbrand (Arm)

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261118-file_folio_flush-v1-3-e9e739ed14e0@kernel.org \
    --to=david@kernel.org \
    --cc=aarcange@redhat.com \
    --cc=akpm@linux-foundation.org \
    --cc=apopple@nvidia.com \
    --cc=axelrasmussen@google.com \
    --cc=baohua@kernel.org \
    --cc=baolin.wang@linux.alibaba.com \
    --cc=baoquan.he@linux.dev \
    --cc=bp@alien8.de \
    --cc=byungchul@sk.com \
    --cc=cenk@deepfinresearch.com \
    --cc=chrisl@kernel.org \
    --cc=gourry@gourry.net \
    --cc=hannes@cmpxchg.org \
    --cc=hughd@google.com \
    --cc=joshua.hahnjy@gmail.com \
    --cc=kasong@tencent.com \
    --cc=liam@infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=ljs@kernel.org \
    --cc=matthew.brost@intel.com \
    --cc=mgorman@suse.de \
    --cc=mhocko@kernel.org \
    --cc=mhocko@suse.com \
    --cc=mingo@kernel.org \
    --cc=nphamcs@gmail.com \
    --cc=peterx@redhat.com \
    --cc=qi.zheng@linux.dev \
    --cc=riel@redhat.com \
    --cc=rppt@kernel.org \
    --cc=sasha.levin@oracle.com \
    --cc=shakeel.butt@linux.dev \
    --cc=shikemeng@huaweicloud.com \
    --cc=stable@vger.kernel.org \
    --cc=surenb@google.com \
    --cc=tglx@kernel.org \
    --cc=vbabka@kernel.org \
    --cc=weixugc@google.com \
    --cc=ying.huang@linux.alibaba.com \
    --cc=youngjun.park@lge.com \
    --cc=yuanchu@google.com \
    --cc=zippermonkey@icloud.com \
    --cc=ziy@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox