[PATCH 0/2] mm: stop calling pmd_folio() on special PMDs

[PATCH 0/2] mm: stop calling pmd_folio() on special PMDs

[Gregory Price]

Andrew: the first patch in this series will annoyingly conflict with the
zone device fixes (see Closes tag) where Sashiko found these. Ordering
on backporting is annoying here, not sure of the best approach.

===

Two page table walkers resolve the folio behind a PMD with pmd_folio(),
which is only valid for a PMD mapping a refcounted struct page:

  madvise_cold_or_pageout_pte_range()   mm/madvise.c
  queue_folios_pmd()                    mm/mempolicy.c

vmf_insert_pfn_pmd() installs special PMDs holding a raw pfn that need not
have a memmap entry at all.  Both walkers can reach one and fault on the
first folio field read.  The PTE halves of both already use
vm_normal_folio(); these two patches make the PMD halves match.

The four callers of vmf_insert_pfn_pmd(), and which walker each reaches:

  drivers/vfio/pci/vfio_pci_core.c        VM_PFNMAP     mempolicy
  drivers/gpu/drm/drm_gem_shmem_helper.c  VM_PFNMAP     mempolicy
  drivers/gpu/drm/panthor/panthor_gem.c   VM_PFNMAP     mempolicy
  drivers/hv/mshv_vtl_main.c              VM_MIXEDMAP   both

can_madv_lru_vma() rejects VM_PFNMAP, so only mshv_vtl_low reaches the
madvise walker, and that needs CAP_SYS_ADMIN.  queue_pages_walk_ops
supplies its own ->test_walk, so walk_page_test()'s generic VM_PFNMAP skip
never runs and vfio-pci is reachable by any process holding the device fd.
Hence the different stable tags.

One behaviour change: mbind(MPOL_MF_STRICT) over a PMD mapped VM_PFNMAP
region now returns 0 rather than -EIO.  The PTE loop already returned 0
there.  drm_gem_shmem and panthor are where this is observable, since they
PMD map pages that do have a memmap entry and so never faulted.

Reproducer
==========

No hardware needed.  An out of tree module stands in for the drivers above:
three misc devices, each with a ->huge_fault calling vmf_insert_pfn_pmd(),
plus VM_HUGEPAGE so the fault path takes the PMD branch.  

  /dev/pmdspec_mixed    VM_MIXEDMAP, pfn at the 1 TiB mark, no memmap
  /dev/pmdspec_pfnmap   VM_PFNMAP,   pfn at the 1 TiB mark, no memmap
  /dev/pmdspec_real     VM_PFNMAP,   real alloc_pages(PMD_ORDER) on node 0

Userspace maps the device into a PMD aligned window, reads one byte to
fault the PMD in, checks a module parameter to confirm it went in, then
issues the operation.  

  vng --run <bzImage> --user root --memory 4G --verbose \
      --append "numa=fake=2" \
      --exec "insmod pmdspec.ko && ./pmdspec_test <subtest>"

numa=fake=2 gives a node 1 to bind to; the module allocates its real page
on node 0, which is what makes queue_folio_required() true.

  subtest        operation                              parent   series
  --------------------------------------------------------------------
  madv_cold      madvise(MADV_COLD)                     oops     ret=0
  madv_pageout   madvise(MADV_PAGEOUT)                  oops     ret=0
  mbind_mixed    mbind(MPOL_BIND, n1, MPOL_MF_MOVE)     oops     ret=0
  mbind_pfnmap   mbind(MPOL_BIND, n1, MPOL_MF_STRICT)   oops     ret=0
  mbind_real     mbind(MPOL_BIND, n1, MPOL_MF_STRICT)   -EIO     ret=0

Two things the table shows that are easy to miss in the code:

  - mbind_mixed passes only MPOL_MF_MOVE.  MPOL_MF_STRICT is not needed for
    a VM_MIXEDMAP vma: walk_page_test() only skips VM_PFNMAP, and
    vma_migratable() is true for VM_MIXEDMAP.

  - mbind_real demonstrates the user visible change (-EIO -> 0)

Reported-by: sashiko-bot <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260817220810.1175596-1-gourry%40gourry.net
Assisted-by: Claude:claude-opus-5

Gregory Price (2):
  mm/mempolicy: use vm_normal_folio_pmd() in queue_folios_pmd()
  mm/madvise: use vm_normal_folio_pmd() in cold/pageout PMD range

 mm/madvise.c   |  4 +++-
 mm/mempolicy.c | 15 +++++++++------
 2 files changed, 12 insertions(+), 7 deletions(-)

-- 
2.55.0

[PATCH 1/2] mm/mempolicy: use vm_normal_folio_pmd() in queue_folios_pmd()

[Gregory Price]

mmap a VM_PFNMAP region whose ->huge_fault installs a PMD through
vmf_insert_pfn_pmd() - a vfio-pci MMIO BAR does this - then

	mbind(p, len, MPOL_BIND, &mask, maxnode, MPOL_MF_STRICT);

With a stand-in module for the driver:

  BUG: unable to handle page fault for address: fffff96dc0000008
  RIP: 0010:queue_folios_pte_range+0xaf/0x440
   walk_pgd_range+0x52b/0xaf0
   __walk_page_range+0x6a/0x1d0
   walk_page_range_mm_unsafe+0x193/0x230
   queue_pages_range+0x64/0xa0
   do_mbind+0x25e/0x640

queue_folios_pmd(), inlined above, calls pmd_folio() on that PMD.  The pfn
is raw MMIO with no memmap entry, so the folio lands in unpopulated
vmemmap.  Neither guard stops the walk:

  walk_page_test()         skips VM_PFNMAP, but queue_pages_walk_ops
                           supplies ->test_walk, so it never runs
  queue_pages_test_walk()  honours vma_migratable(), but only while
                           MPOL_MF_STRICT is clear

A VM_MIXEDMAP vma needs neither flag, being vma_migratable(), so plain
mbind(MPOL_MF_MOVE) reaches this too - and there the bad folio carries on
into migrate_folio_add() and folio_isolate_lru().  mshv_vtl_low is such a
mapping.

Use vm_normal_folio_pmd() and skip on NULL, as the PTE loop in
queue_folios_pte_range() already does with vm_normal_folio().  The huge
zero PMD moves ahead of the lookup, since vm_normal_folio_pmd() returns
NULL for it and its ACTION_CONTINUE would be lost.

mbind(MPOL_MF_STRICT) over a PMD mapped VM_PFNMAP region now returns 0
rather than -EIO.  The PTE loop already returned 0 there.

Fixes: 3c8e44c9b369 ("mm: mark special bits for huge pfn mappings when inject")
Reported-by: sashiko-bot <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260817220810.1175596-1-gourry%40gourry.net
Cc: stable@vger.kernel.org
Assisted-by: Claude:claude-opus-5
Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/mempolicy.c | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index f1aba551f9f1..85803c845965 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -650,7 +650,8 @@ static inline bool queue_folio_required(struct folio *folio,
 	return node_isset(nid, *qp->nmask) == !(flags & MPOL_MF_INVERT);
 }
 
-static void queue_folios_pmd(pmd_t *pmd, struct mm_walk *walk)
+static void queue_folios_pmd(pmd_t *pmd, unsigned long addr,
+			     struct mm_walk *walk)
 {
 	struct folio *folio;
 	struct queue_pages *qp = walk->private;
@@ -661,13 +662,15 @@ static void queue_folios_pmd(pmd_t *pmd, struct mm_walk *walk)
 			qp->nr_failed++;
 		return;
 	}
-	folio = pmd_folio(pmdval);
-	if (folio_is_zone_device(folio))
-		return;
-	if (is_huge_zero_folio(folio)) {
+	if (is_huge_zero_pmd(pmdval)) {
 		walk->action = ACTION_CONTINUE;
 		return;
 	}
+	folio = vm_normal_folio_pmd(walk->vma, addr, pmdval);
+	if (!folio)
+		return;
+	if (folio_is_zone_device(folio))
+		return;
 	if (!queue_folio_required(folio, qp))
 		return;
 	if (!(qp->flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) ||
@@ -700,7 +703,7 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned long addr,
 
 	ptl = pmd_trans_huge_lock(pmd, vma);
 	if (ptl) {
-		queue_folios_pmd(pmd, walk);
+		queue_folios_pmd(pmd, addr, walk);
 		spin_unlock(ptl);
 		goto out;
 	}
-- 
2.55.0

Re: [PATCH 1/2] mm/mempolicy: use vm_normal_folio_pmd() in queue_folios_pmd()

[David Hildenbrand (Arm)]

On 8/21/26 16:49, Gregory Price wrote:
> mmap a VM_PFNMAP region whose ->huge_fault installs a PMD through
> vmf_insert_pfn_pmd() - a vfio-pci MMIO BAR does this - then
> 
> 	mbind(p, len, MPOL_BIND, &mask, maxnode, MPOL_MF_STRICT);
> 
> With a stand-in module for the driver:
> 
>   BUG: unable to handle page fault for address: fffff96dc0000008
>   RIP: 0010:queue_folios_pte_range+0xaf/0x440
>    walk_pgd_range+0x52b/0xaf0
>    __walk_page_range+0x6a/0x1d0
>    walk_page_range_mm_unsafe+0x193/0x230
>    queue_pages_range+0x64/0xa0
>    do_mbind+0x25e/0x640
> 
> queue_folios_pmd(), inlined above, calls pmd_folio() on that PMD.  The pfn
> is raw MMIO with no memmap entry, so the folio lands in unpopulated
> vmemmap.  Neither guard stops the walk:
> 
>   walk_page_test()         skips VM_PFNMAP, but queue_pages_walk_ops
>                            supplies ->test_walk, so it never runs
>   queue_pages_test_walk()  honours vma_migratable(), but only while
>                            MPOL_MF_STRICT is clear
> 
> A VM_MIXEDMAP vma needs neither flag, being vma_migratable(), so plain
> mbind(MPOL_MF_MOVE) reaches this too - and there the bad folio carries on
> into migrate_folio_add() and folio_isolate_lru().  mshv_vtl_low is such a
> mapping.
> 
> Use vm_normal_folio_pmd() and skip on NULL, as the PTE loop in
> queue_folios_pte_range() already does with vm_normal_folio().  The huge
> zero PMD moves ahead of the lookup, since vm_normal_folio_pmd() returns
> NULL for it and its ACTION_CONTINUE would be lost.
> 
> mbind(MPOL_MF_STRICT) over a PMD mapped VM_PFNMAP region now returns 0
> rather than -EIO.  The PTE loop already returned 0 there.
> 
> Fixes: 3c8e44c9b369 ("mm: mark special bits for huge pfn mappings when inject")
> Reported-by: sashiko-bot <sashiko-bot@kernel.org>
> Closes: https://sashiko.dev/#/patchset/20260817220810.1175596-1-gourry%40gourry.net
> Cc: stable@vger.kernel.org
> Assisted-by: Claude:claude-opus-5
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
> ---
>  mm/mempolicy.c | 15 +++++++++------
>  1 file changed, 9 insertions(+), 6 deletions(-)
> 
> diff --git a/mm/mempolicy.c b/mm/mempolicy.c
> index f1aba551f9f1..85803c845965 100644
> --- a/mm/mempolicy.c
> +++ b/mm/mempolicy.c
> @@ -650,7 +650,8 @@ static inline bool queue_folio_required(struct folio *folio,
>  	return node_isset(nid, *qp->nmask) == !(flags & MPOL_MF_INVERT);
>  }
>  
> -static void queue_folios_pmd(pmd_t *pmd, struct mm_walk *walk)
> +static void queue_folios_pmd(pmd_t *pmd, unsigned long addr,
> +			     struct mm_walk *walk)
>  {
>  	struct folio *folio;
>  	struct queue_pages *qp = walk->private;
> @@ -661,13 +662,15 @@ static void queue_folios_pmd(pmd_t *pmd, struct mm_walk *walk)
>  			qp->nr_failed++;
>  		return;
>  	}
> -	folio = pmd_folio(pmdval);
> -	if (folio_is_zone_device(folio))
> -		return;
> -	if (is_huge_zero_folio(folio)) {
> +	if (is_huge_zero_pmd(pmdval)) {
>  		walk->action = ACTION_CONTINUE;
>  		return;
>  	}
> +	folio = vm_normal_folio_pmd(walk->vma, addr, pmdval);
> +	if (!folio)
> +		return


I think this is best written as:

folio = vm_normal_folio_pmd(walk->vma, addr, pmdval);
if (!folio) {
	if (is_huge_zero_pmd(pmdval)) {
		walk->action = ACTION_CONTINUE;
		return;
	}
	return;
}
That way we minimize checks on the expected path.

Acked-by: David Hildenbrand (Arm) <david@kernel.org>

-- 
Cheers,

David

Re: [PATCH 1/2] mm/mempolicy: use vm_normal_folio_pmd() in queue_folios_pmd()

[Gregory Price]

On Thu, Aug 27, 2026 at 07:05:21PM +0200, David Hildenbrand (Arm) wrote:
> >  	}
> > +	folio = vm_normal_folio_pmd(walk->vma, addr, pmdval);
> > +	if (!folio)
> > +		return
> 
> 
> I think this is best written as:
> 
> folio = vm_normal_folio_pmd(walk->vma, addr, pmdval);
> if (!folio) {
> 	if (is_huge_zero_pmd(pmdval)) {
> 		walk->action = ACTION_CONTINUE;
> 		return;
> 	}
> 	return;
> }
> That way we minimize checks on the expected path.
> 

Sure, i'll spin a v2

> Acked-by: David Hildenbrand (Arm) <david@kernel.org>
> 

Thanks!

~Gregory

[PATCH 2/2] mm/madvise: use vm_normal_folio_pmd() in cold/pageout PMD range

[Gregory Price]

mmap a VM_MIXEDMAP region whose ->huge_fault installs a PMD through
vmf_insert_pfn_pmd() - mshv_vtl_low does this, and needs CAP_SYS_ADMIN to
open - then

	madvise(p, PMD_SIZE, MADV_PAGEOUT);

With a stand-in module for the driver:

  BUG: unable to handle page fault for address: fffff587c0000008
  RIP: 0010:madvise_cold_or_pageout_pte_range+0x410/0x9b0
   walk_pgd_range+0x52b/0xaf0
   __walk_page_range+0x6a/0x1d0
   walk_page_range_vma_unsafe+0x8e/0x120
   madvise_pageout+0xb2/0x180
   madvise_vma_behavior+0x46b/0xa90
   do_madvise+0x108/0x190
   __x64_sys_madvise+0x26/0x30

Nothing validates the pfn on the way in:

  can_madv_lru_vma()     rejects VM_PFNMAP, but not VM_MIXEDMAP
  can_fault()            *pfn = vmf->pgoff & ~(mask >> PAGE_SHIFT);
  vmf_insert_pfn_pmd()   no pfn_valid() check
  pmd_folio()            pfn_to_page() -> unpopulated vmemmap

Even with a valid pfn the path is wrong.  The mapping carries no rmap, so
folio_maybe_mapped_shared() sees mapcount 0, and the walker goes on to
folio_deactivate(), or folio_isolate_lru() plus reclaim_pages(), against a
folio this mapping does not own.

Use vm_normal_folio_pmd() and skip on NULL, as the PTE half of this same
walker already does with vm_normal_folio().  The huge zero PMD is already
handled further up by is_huge_zero_pmd().

Fixes: 3c8e44c9b369 ("mm: mark special bits for huge pfn mappings when inject")
Reported-by: sashiko-bot <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260817220810.1175596-1-gourry%40gourry.net
Cc: <stable@vger.kernel.org> # v6.19+
Assisted-by: Claude:claude-opus-5
Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/madvise.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/mm/madvise.c b/mm/madvise.c
index ffd6a68320a8..ab362bc482a5 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -393,7 +393,9 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,
 			goto huge_unlock;
 		}
 
-		folio = pmd_folio(orig_pmd);
+		folio = vm_normal_folio_pmd(vma, addr, orig_pmd);
+		if (!folio)
+			goto huge_unlock;
 
 		if (folio_is_zone_device(folio))
 			goto huge_unlock;
-- 
2.55.0

Re: [PATCH 2/2] mm/madvise: use vm_normal_folio_pmd() in cold/pageout PMD range

[David Hildenbrand (Arm)]

On 8/21/26 16:49, Gregory Price wrote:
> mmap a VM_MIXEDMAP region whose ->huge_fault installs a PMD through
> vmf_insert_pfn_pmd() - mshv_vtl_low does this, and needs CAP_SYS_ADMIN to
> open - then
> 
> 	madvise(p, PMD_SIZE, MADV_PAGEOUT);
> 
> With a stand-in module for the driver:
> 
>   BUG: unable to handle page fault for address: fffff587c0000008
>   RIP: 0010:madvise_cold_or_pageout_pte_range+0x410/0x9b0
>    walk_pgd_range+0x52b/0xaf0
>    __walk_page_range+0x6a/0x1d0
>    walk_page_range_vma_unsafe+0x8e/0x120
>    madvise_pageout+0xb2/0x180
>    madvise_vma_behavior+0x46b/0xa90
>    do_madvise+0x108/0x190
>    __x64_sys_madvise+0x26/0x30
> 
> Nothing validates the pfn on the way in:
> 
>   can_madv_lru_vma()     rejects VM_PFNMAP, but not VM_MIXEDMAP
>   can_fault()            *pfn = vmf->pgoff & ~(mask >> PAGE_SHIFT);
>   vmf_insert_pfn_pmd()   no pfn_valid() check
>   pmd_folio()            pfn_to_page() -> unpopulated vmemmap
> 
> Even with a valid pfn the path is wrong.  The mapping carries no rmap, so
> folio_maybe_mapped_shared() sees mapcount 0, and the walker goes on to
> folio_deactivate(), or folio_isolate_lru() plus reclaim_pages(), against a
> folio this mapping does not own.
> 
> Use vm_normal_folio_pmd() and skip on NULL, as the PTE half of this same
> walker already does with vm_normal_folio().  The huge zero PMD is already
> handled further up by is_huge_zero_pmd().

The last part doesn't make sense, see below.

> 
> Fixes: 3c8e44c9b369 ("mm: mark special bits for huge pfn mappings when inject")
> Reported-by: sashiko-bot <sashiko-bot@kernel.org>
> Closes: https://sashiko.dev/#/patchset/20260817220810.1175596-1-gourry%40gourry.net
> Cc: <stable@vger.kernel.org> # v6.19+
> Assisted-by: Claude:claude-opus-5
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
> ---
>  mm/madvise.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/mm/madvise.c b/mm/madvise.c
> index ffd6a68320a8..ab362bc482a5 100644
> --- a/mm/madvise.c
> +++ b/mm/madvise.c
> @@ -393,7 +393,9 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,
>  			goto huge_unlock;
>  		}
>  
> -		folio = pmd_folio(orig_pmd);
> +		folio = vm_normal_folio_pmd(vma, addr, orig_pmd);
> +		if (!folio)
> +			goto huge_unlock;
I think you can now drop the is_huge_zero_pmd() check, because
!vm_normal_folio_pmd() includes the huge zero folio.

-- 
Cheers,

David

Re: [PATCH 2/2] mm/madvise: use vm_normal_folio_pmd() in cold/pageout PMD range

[Gregory Price]

On Thu, Aug 27, 2026 at 07:07:38PM +0200, David Hildenbrand (Arm) wrote:
> On 8/21/26 16:49, Gregory Price wrote:
> > 
> > Fixes: 3c8e44c9b369 ("mm: mark special bits for huge pfn mappings when inject")
> > Reported-by: sashiko-bot <sashiko-bot@kernel.org>
> > Closes: https://sashiko.dev/#/patchset/20260817220810.1175596-1-gourry%40gourry.net
> > Cc: <stable@vger.kernel.org> # v6.19+
> > Assisted-by: Claude:claude-opus-5
> > Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
> > ---
> >  mm/madvise.c | 4 +++-
> >  1 file changed, 3 insertions(+), 1 deletion(-)
> > 
> > diff --git a/mm/madvise.c b/mm/madvise.c
> > index ffd6a68320a8..ab362bc482a5 100644
> > --- a/mm/madvise.c
> > +++ b/mm/madvise.c
> > @@ -393,7 +393,9 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,
> >  			goto huge_unlock;
> >  		}
> >  
> > -		folio = pmd_folio(orig_pmd);
> > +		folio = vm_normal_folio_pmd(vma, addr, orig_pmd);
> > +		if (!folio)
> > +			goto huge_unlock;
> I think you can now drop the is_huge_zero_pmd() check, because
> !vm_normal_folio_pmd() includes the huge zero folio.
> 

I was err'ing on the side of not removing existing checks, but I can
take a look.  Trivilially - I think you are right.

~Gregory