[PATCH for 7.4-rc2+ 0/7] Re-enable discarding lazyfree THPs at PMD level

[PATCH for 7.4-rc2+ 0/7] Re-enable discarding lazyfree THPs at PMD level

[Zi Yan]

For maintainers, Patch 6 requires first 5 patches to work correctly, so it
might be better to get ack from arch maintainers for first 5 patches and
merge the series via mm tree. The patch disabling lazyfree THP discard at
PMD level has been picked up [2] and would take some time to test, so this
patchset probably can be considered after the merge window closes. I am
sending it out for early feedback.

Motivation
===

A bug was found in the PMD-level discard of lazyfree THPs [1]. The
proposed fix keeps the PMD invalidated with pmdp_invalidate() instead of
clearing it, but that does not work on every architecture:

1. riscv, LoongArch and s390 let GUP-fast follow an invalidated PMD;
2. powerpc/book3s64 page table check counts an invalidated PMD as mapped;
3. sparc64's thp_pte_count becomes unbalanced around pmdp_invalidate().

So the PMD-level discard was disabled instead [2]. This series fixes the
arch code and re-enables it.

The discard decides from the folio refcount, so GUP-fast must not take a
reference through an invalidated PMD, and arch accounting must stay
balanced when an invalidated PMD is set again or cleared.

Details
===

Patches 1-5 fix the arch code, patch 6 from Kyle keeps the PMD invalidated
instead of cleared until the discard is committed, and patch 7 enables the
PMD-level discard again:

1. powerpc/book3s64: page table check counts an invalidated PMD as a live
   mapping, so setting it again or clearing it decrements the map count a
   second time and hits BUG_ON(). This is already reachable via MADV_COLD
   and mprotect() with CONFIG_PAGE_TABLE_CHECK.

2. riscv: an invalidated PMD still passes pmd_present() and the generic
   pmd_access_permitted(), so GUP-fast follows it. Add a
   pmd_access_permitted() that requires _PAGE_PRESENT.

3. LoongArch: same problem and fix as riscv.

4. s390: _SEGMENT_ENTRY_INVALID also marks an old PMD, so GUP-fast follows
   an invalidated PMD. Always clear _SEGMENT_ENTRY_READ in
   pmdp_invalidate(), which makes it pmd_protnone(), so GUP-fast rejects
   it.

5. sparc64: pmdp_invalidate() decrements thp_pte_count assuming a THP
   split follows, so setting the PMD again loses a count. Count only
   valid huge PMDs in __set_pmd_acct() instead, which keeps invalidate,
   restore, clear and split balanced.

6. mm/huge_memory: use pmdp_invalidate() in the discard, so munmap() and
   mremap() wait on the PMD lock instead of skipping a none PMD.

7. mm/rmap: discard lazyfree THPs at PMD level again.

Note
===

x86, arm64 and powerpc already make GUP-fast reject an invalidated PMD,
and sparc64 has no GUP-fast. 32-bit MIPS would not reject it, but it
cannot have THP: _PAGE_HUGE makes PFN_PTE_SHIFT exceed PAGE_SHIFT, so the
BUILD_BUG_ON() in arch_mm_preinit() fires, which is why Ingenic dropped
huge pages in commit 69b686ddd579 ("MIPS: ingenic: Remove
CPU_SUPPORTS_HUGEPAGES").

Link: https://lore.kernel.org/all/20261009165214.40212-2-kylebot@openai.com/ [1]
Link: https://lore.kernel.org/all/20261012-lazyfree-stable-fix-v2-1-6387d6e30063@nvidia.com/ [2]

Signed-off-by: Zi Yan <ziy@nvidia.com>
---
Kyle Zeng (1):
      mm/huge_memory: avoid transient none PMDs during lazyfree reclaim

Zi Yan (6):
      powerpc/book3s64/mm: don't count invalidated PMDs in page table check
      riscv: mm: don't let GUP-fast follow invalidated THP PMDs
      LoongArch: mm: don't let GUP-fast follow invalidated THP PMDs
      s390/mm: always clear _SEGMENT_ENTRY_READ in pmdp_invalidate()
      sparc64: mm: fix thp_pte_count accounting around pmdp_invalidate()
      mm/rmap: re-enable discarding lazyfree THPs at PMD level

 arch/loongarch/include/asm/pgtable.h         |  6 ++++++
 arch/powerpc/include/asm/book3s/64/pgtable.h |  3 ++-
 arch/riscv/include/asm/pgtable.h             |  6 ++++++
 arch/s390/include/asm/pgtable.h              |  8 ++++++--
 arch/sparc/mm/tlb.c                          | 26 ++++++++++++--------------
 mm/huge_memory.c                             |  5 +++--
 mm/rmap.c                                    | 11 +++++++++++
 7 files changed, 46 insertions(+), 19 deletions(-)
---
base-commit: 71a9b024444eba184335b67867897714e4da143c
change-id: 20261007-anon_vma_uaf_fix-e7accd1d2ee1

Best regards,
--  
Yan, Zi


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

[PATCH for 7.4-rc2+ 2/7] riscv: mm: don't let GUP-fast follow invalidated THP PMDs

[Zi Yan]

pmdp_invalidate() leaves a PMD whose pmd_present() is still true, so the
generic pmd_access_permitted() returns true for an invalidated PMD. Then
GUP-fast can grab a reference on the folio and, as the PMD value doesn't
change until the caller finishes, the recheck after taking the reference
passes too.

Code that invalidates a PMD and then decides, from the folio refcount,
whether it may drop the mapping relies on GUP-fast backing off from an
invalidated PMD, like it does from a cleared one. x86, arm64 and powerpc
already reject invalidated PMDs in pmd_access_permitted().

Provide a riscv pmd_access_permitted() that requires _PAGE_PRESENT, so
invalidated (and PROT_NONE) PMDs are not followed by GUP-fast.

Assisted-by: LLM
Signed-off-by: Zi Yan <ziy@nvidia.com>
To: Paul Walmsley <pjw@kernel.org>
To: Palmer Dabbelt <palmer@dabbelt.com>
To: Albert Ou <aou@eecs.berkeley.edu>
Cc: Alexandre Ghiti <alex@ghiti.fr>
Cc: Jason Gunthorpe <jgg@nvidia.com>
Cc: John Hubbard <jhubbard@nvidia.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: linux-riscv@lists.infradead.org
Cc: linux-kernel@vger.kernel.org
---
 arch/riscv/include/asm/pgtable.h | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index d48f90140841e..66aac56631f0c 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -816,6 +816,12 @@ static inline int pmd_write(pmd_t pmd)
 	return pte_write(pmd_pte(pmd));
 }
 
+#define pmd_access_permitted pmd_access_permitted
+static inline bool pmd_access_permitted(pmd_t pmd, bool write)
+{
+	return (pmd_val(pmd) & _PAGE_PRESENT) && (!write || pmd_write(pmd));
+}
+
 #define pud_write pud_write
 static inline int pud_write(pud_t pud)
 {

-- 
2.53.0


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH for 7.4-rc2+ 2/7] riscv: mm: don't let GUP-fast follow invalidated THP PMDs

[Zi Yan]

On Thu Oct 15, 2026 at 2:13 PM EDT, Zi Yan wrote:
> pmdp_invalidate() leaves a PMD whose pmd_present() is still true, so the
> generic pmd_access_permitted() returns true for an invalidated PMD. Then
> GUP-fast can grab a reference on the folio and, as the PMD value doesn't
> change until the caller finishes, the recheck after taking the reference
> passes too.
>
> Code that invalidates a PMD and then decides, from the folio refcount,
> whether it may drop the mapping relies on GUP-fast backing off from an
> invalidated PMD, like it does from a cleared one. x86, arm64 and powerpc
> already reject invalidated PMDs in pmd_access_permitted().
>
> Provide a riscv pmd_access_permitted() that requires _PAGE_PRESENT, so
> invalidated (and PROT_NONE) PMDs are not followed by GUP-fast.
>
> Assisted-by: LLM
> Signed-off-by: Zi Yan <ziy@nvidia.com>
> To: Paul Walmsley <pjw@kernel.org>
> To: Palmer Dabbelt <palmer@dabbelt.com>
> To: Albert Ou <aou@eecs.berkeley.edu>
> Cc: Alexandre Ghiti <alex@ghiti.fr>
> Cc: Jason Gunthorpe <jgg@nvidia.com>
> Cc: John Hubbard <jhubbard@nvidia.com>
> Cc: Peter Xu <peterx@redhat.com>
> Cc: linux-riscv@lists.infradead.org
> Cc: linux-kernel@vger.kernel.org
> ---
>  arch/riscv/include/asm/pgtable.h | 6 ++++++
>  1 file changed, 6 insertions(+)
>
> diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
> index d48f90140841e..66aac56631f0c 100644
> --- a/arch/riscv/include/asm/pgtable.h
> +++ b/arch/riscv/include/asm/pgtable.h
> @@ -816,6 +816,12 @@ static inline int pmd_write(pmd_t pmd)
>  	return pte_write(pmd_pte(pmd));
>  }
>  
> +#define pmd_access_permitted pmd_access_permitted
> +static inline bool pmd_access_permitted(pmd_t pmd, bool write)
> +{
> +	return (pmd_val(pmd) & _PAGE_PRESENT) && (!write || pmd_write(pmd));
> +}
> +

Sashiko asked if this function allows GUP-fast to follow execute-only
pages so that the page can be read.

Answer:

yes, and the new patch is at[1] and fixes both issues.

[1] https://lore.kernel.org/all/20261015-riscv-gup-read-fix-v1-1-8cf7f2504029@nvidia.com/

-- 
Best Regards,
Yan, Zi


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv