[PATCH 0/3] mm: handle device-private PMDs in walk callbacks

[PATCH 0/3] mm: handle device-private PMDs in walk callbacks

[Usama Arif]

Since commit 368076f52ebe ("mm/huge_memory: add device-private THP support
to PMD operations") a PMD may hold a device-private swap entry whenever
an HMM-based GPU driver migrates an anonymous THP folio to device memory
via migrate_vma_pages().

pmd_trans_huge_lock() succeeds for such PMDs (pmd_is_huge() returns true
for any non-present, non-none huge PMD), so several MM walk callbacks
that used to assume present THP or migration entry are now reachable with
a device-private PMD. The results range from a VM_BUG_ON() firing on debug
kernels, to an oops on a bogus vmemmap dereference, to silently isolating
an unrelated live folio from LRU in the aliasing case.

The first 2 fixes were reported as pre-existing issues by sashiko in my
PMD swap entry series [1]. Hopefully sashiko won't point these out
in the next PMD swap entry series :)

[1] https://sashiko.dev/#/patchset/20260703173903.3789516-1-usama.arif%40linux.dev?part=6
 
Usama Arif (3):
  mm/mempolicy: skip device-private PMDs when queueing folios
  mm/madvise: skip device-private PMDs in cold and pageout walks
  mm/huge_memory: skip device-private PMDs in madvise_free_huge_pmd

 mm/huge_memory.c | 3 +++
 mm/madvise.c     | 3 +++
 mm/mempolicy.c   | 2 ++
 3 files changed, 8 insertions(+)

-- 
2.53.0-Meta

[PATCH 1/3] mm/mempolicy: skip device-private PMDs when queueing folios

[Usama Arif]

queue_folios_pmd() is called under pmd_trans_huge_lock(), whose
pmd_is_huge() check returns true for any non-present, non-none huge
PMD - including a device-private swap entry. Passing such a PMD to
pmd_folio() extracts garbage bits as a PFN and returns a bogus folio
pointer.

Potential trigger: an HMM-based GPU driver migrates an anonymous THP
folio to device memory via migrate_vma_pages(), leaving a device-private
PMD. Userspace then calls mbind(), migrate_pages() or
set_mempolicy_home_node() on that range.

Skip device-private PMDs, matching how queue_folios_pte_range() skips
device-private PTE entries by checking !pte_present().

Reported-by: sashiko-bot <sashiko-bot@kernel.org>
Link: https://sashiko.dev/#/patchset/20260703173903.3789516-1-usama.arif%40linux.dev?part=6
Fixes: 368076f52ebe ("mm/huge_memory: add device-private THP support to PMD operations")
Signed-off-by: Usama Arif <usama.arif@linux.dev>
---
 mm/mempolicy.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 914f81863db5..eda817539c77 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -659,6 +659,8 @@ static void queue_folios_pmd(pmd_t *pmd, struct mm_walk *walk)
 		qp->nr_failed++;
 		return;
 	}
+	if (unlikely(pmd_is_device_private_entry(*pmd)))
+		return;
 	folio = pmd_folio(*pmd);
 	if (is_huge_zero_folio(folio)) {
 		walk->action = ACTION_CONTINUE;
-- 
2.53.0-Meta

[PATCH 2/3] mm/madvise: skip device-private PMDs in cold and pageout walks

[Usama Arif]

madvise_cold_or_pageout_pte_range() takes pmd_trans_huge_lock(), whose
pmd_is_huge() check returns true for a device-private PMD. The subsequent
!pmd_present() branch has a VM_BUG_ON() asserting migration is the only
allowed non-present case; a device-private PMD trips it.

Potential trigger: an HMM-based GPU driver races with
madvise(MADV_COLD)/MADV_PAGEOUT: pmd_trans_huge(*pmd) reads true, then
migrate_vma_pages() flips the PMD to a device-private entry before the
PMD lock is acquired.

Skip device-private PMDs after taking the lock, before the !pmd_present()
check.

Reported-by: sashiko-bot <sashiko-bot@kernel.org>
Link: https://sashiko.dev/#/patchset/20260703173903.3789516-1-usama.arif%40linux.dev?part=6
Fixes: 368076f52ebe ("mm/huge_memory: add device-private THP support to PMD operations")
Signed-off-by: Usama Arif <usama.arif@linux.dev>
---
 mm/madvise.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/mm/madvise.c b/mm/madvise.c
index 9292f60b19aa..870be398c6f3 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -384,6 +384,9 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,
 			return 0;
 
 		orig_pmd = *pmd;
+		if (pmd_is_device_private_entry(orig_pmd))
+			goto huge_unlock;
+
 		if (is_huge_zero_pmd(orig_pmd))
 			goto huge_unlock;
 
-- 
2.53.0-Meta

[PATCH 3/3] mm/huge_memory: skip device-private PMDs in madvise_free_huge_pmd

[Usama Arif]

madvise_free_pte_range() checks pmd_trans_huge(*pmd) unlocked, then
madvise_free_huge_pmd() takes pmd_trans_huge_lock().
pmd_is_huge() returns true for a device-private PMD, so orig_pmd can
be device-private and hit the VM_BUG_ON() on the !pmd_present() branch.

Potential trigger: an HMM-based GPU driver races with madvise(MADV_FREE):
migrate_vma_pages() flips the PMD to a device-private entry between the
caller's pmd_trans_huge() check and the callee's pmd_trans_huge_lock().

Skip device-private PMDs after taking the lock, before the
!pmd_present() check.

Fixes: 368076f52ebe ("mm/huge_memory: add device-private THP support to PMD operations")
Signed-off-by: Usama Arif <usama.arif@linux.dev>
---
 mm/huge_memory.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index c0892cc533a9..cfce9f31b30e 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2296,6 +2296,9 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,
 	if (is_huge_zero_pmd(orig_pmd))
 		goto out;
 
+	if (pmd_is_device_private_entry(orig_pmd))
+		goto out;
+
 	if (unlikely(!pmd_present(orig_pmd))) {
 		VM_BUG_ON(thp_migration_supported() &&
 				  !pmd_is_migration_entry(orig_pmd));
-- 
2.53.0-Meta

Re: [PATCH 0/3] mm: handle device-private PMDs in walk callbacks

[Lorenzo Stoakes]

+cc Balbir

Thanks for the series!

I think Balbir should be cc'd no?

On Tue, Jul 07, 2026 at 06:45:06AM -0700, Usama Arif wrote:
> Since commit 368076f52ebe ("mm/huge_memory: add device-private THP support
> to PMD operations") a PMD may hold a device-private swap entry whenever
> an HMM-based GPU driver migrates an anonymous THP folio to device memory
> via migrate_vma_pages().
>
> pmd_trans_huge_lock() succeeds for such PMDs (pmd_is_huge() returns true
> for any non-present, non-none huge PMD), so several MM walk callbacks
> that used to assume present THP or migration entry are now reachable with
> a device-private PMD. The results range from a VM_BUG_ON() firing on debug
> kernels, to an oops on a bogus vmemmap dereference, to silently isolating
> an unrelated live folio from LRU in the aliasing case.
>
> The first 2 fixes were reported as pre-existing issues by sashiko in my
> PMD swap entry series [1]. Hopefully sashiko won't point these out
> in the next PMD swap entry series :)

Yeah I don't love these 'existing problem' reports (distracting from other work,
add workload, really it feels like that should be a passive reporting mode for
sashiko not arbitrarily added to other series), but sending it as a separate
series is indeed the right way :)

These of course should never block an unrelated series.

>
> [1] https://sashiko.dev/#/patchset/20260703173903.3789516-1-usama.arif%40linux.dev?part=6

Hmm we've had a whole host of issues with device-private PMDs recently.

Balbir - would it be possible to audit all of the code paths and proactively see
if there's anything else that could hit problems here?

>
> Usama Arif (3):
>   mm/mempolicy: skip device-private PMDs when queueing folios
>   mm/madvise: skip device-private PMDs in cold and pageout walks
>   mm/huge_memory: skip device-private PMDs in madvise_free_huge_pmd
>
>  mm/huge_memory.c | 3 +++
>  mm/madvise.c     | 3 +++
>  mm/mempolicy.c   | 2 ++
>  3 files changed, 8 insertions(+)
>
> --
> 2.53.0-Meta
>

Cheers, Lorenzo

Re: [PATCH 0/3] mm: handle device-private PMDs in walk callbacks

[Usama Arif]

On 07/07/2026 15:01, Lorenzo Stoakes wrote:
> +cc Balbir
> 
> Thanks for the series!
> 
> I think Balbir should be cc'd no?
> 

Ah yes, sorry about that! I just sent the patches to the list I got from
get_maintainers.pl.

I think maybe we should modify get_maintainers.pl so that if there is
a "Fixes" in commit message, than the author of that commit is also
in the output from that script?

> On Tue, Jul 07, 2026 at 06:45:06AM -0700, Usama Arif wrote:
>> Since commit 368076f52ebe ("mm/huge_memory: add device-private THP support
>> to PMD operations") a PMD may hold a device-private swap entry whenever
>> an HMM-based GPU driver migrates an anonymous THP folio to device memory
>> via migrate_vma_pages().
>>
>> pmd_trans_huge_lock() succeeds for such PMDs (pmd_is_huge() returns true
>> for any non-present, non-none huge PMD), so several MM walk callbacks
>> that used to assume present THP or migration entry are now reachable with
>> a device-private PMD. The results range from a VM_BUG_ON() firing on debug
>> kernels, to an oops on a bogus vmemmap dereference, to silently isolating
>> an unrelated live folio from LRU in the aliasing case.
>>
>> The first 2 fixes were reported as pre-existing issues by sashiko in my
>> PMD swap entry series [1]. Hopefully sashiko won't point these out
>> in the next PMD swap entry series :)
> 
> Yeah I don't love these 'existing problem' reports (distracting from other work,
> add workload, really it feels like that should be a passive reporting mode for
> sashiko not arbitrarily added to other series), but sending it as a separate
> series is indeed the right way :)
> 
> These of course should never block an unrelated series.
> 

Yes agreed!

>>
>> [1] https://sashiko.dev/#/patchset/20260703173903.3789516-1-usama.arif%40linux.dev?part=6
> 
> Hmm we've had a whole host of issues with device-private PMDs recently.
> 
> Balbir - would it be possible to audit all of the code paths and proactively see
> if there's anything else that could hit problems here?
> 
>>
>> Usama Arif (3):
>>   mm/mempolicy: skip device-private PMDs when queueing folios
>>   mm/madvise: skip device-private PMDs in cold and pageout walks
>>   mm/huge_memory: skip device-private PMDs in madvise_free_huge_pmd
>>
>>  mm/huge_memory.c | 3 +++
>>  mm/madvise.c     | 3 +++
>>  mm/mempolicy.c   | 2 ++
>>  3 files changed, 8 insertions(+)
>>
>> --
>> 2.53.0-Meta
>>
> 
> Cheers, Lorenzo

Re: [PATCH 0/3] mm: handle device-private PMDs in walk callbacks

[Zi Yan]

+Alistair as well.

On 7 Jul 2026, at 10:01, Lorenzo Stoakes wrote:

> +cc Balbir
>
> Thanks for the series!
>
> I think Balbir should be cc'd no?
>
> On Tue, Jul 07, 2026 at 06:45:06AM -0700, Usama Arif wrote:
>> Since commit 368076f52ebe ("mm/huge_memory: add device-private THP support
>> to PMD operations") a PMD may hold a device-private swap entry whenever
>> an HMM-based GPU driver migrates an anonymous THP folio to device memory
>> via migrate_vma_pages().
>>
>> pmd_trans_huge_lock() succeeds for such PMDs (pmd_is_huge() returns true
>> for any non-present, non-none huge PMD), so several MM walk callbacks
>> that used to assume present THP or migration entry are now reachable with
>> a device-private PMD. The results range from a VM_BUG_ON() firing on debug
>> kernels, to an oops on a bogus vmemmap dereference, to silently isolating
>> an unrelated live folio from LRU in the aliasing case.
>>
>> The first 2 fixes were reported as pre-existing issues by sashiko in my
>> PMD swap entry series [1]. Hopefully sashiko won't point these out
>> in the next PMD swap entry series :)
>
> Yeah I don't love these 'existing problem' reports (distracting from other work,
> add workload, really it feels like that should be a passive reporting mode for
> sashiko not arbitrarily added to other series), but sending it as a separate
> series is indeed the right way :)
>
> These of course should never block an unrelated series.
>
>>
>> [1] https://sashiko.dev/#/patchset/20260703173903.3789516-1-usama.arif%40linux.dev?part=6
>
> Hmm we've had a whole host of issues with device-private PMDs recently.
>
> Balbir - would it be possible to audit all of the code paths and proactively see
> if there's anything else that could hit problems here?
>
>>
>> Usama Arif (3):
>>   mm/mempolicy: skip device-private PMDs when queueing folios
>>   mm/madvise: skip device-private PMDs in cold and pageout walks
>>   mm/huge_memory: skip device-private PMDs in madvise_free_huge_pmd
>>
>>  mm/huge_memory.c | 3 +++
>>  mm/madvise.c     | 3 +++
>>  mm/mempolicy.c   | 2 ++
>>  3 files changed, 8 insertions(+)
>>
>> --
>> 2.53.0-Meta
>>
>
> Cheers, Lorenzo


Best Regards,
Yan, Zi

Re: [PATCH 0/3] mm: handle device-private PMDs in walk callbacks

[Joshua Hahn]

On Tue,  7 Jul 2026 06:45:06 -0700 Usama Arif <usama.arif@linux.dev> wrote:

> Since commit 368076f52ebe ("mm/huge_memory: add device-private THP support
> to PMD operations") a PMD may hold a device-private swap entry whenever
> an HMM-based GPU driver migrates an anonymous THP folio to device memory
> via migrate_vma_pages().
> 
> pmd_trans_huge_lock() succeeds for such PMDs (pmd_is_huge() returns true
> for any non-present, non-none huge PMD), so several MM walk callbacks
> that used to assume present THP or migration entry are now reachable with
> a device-private PMD. The results range from a VM_BUG_ON() firing on debug
> kernels, to an oops on a bogus vmemmap dereference, to silently isolating
> an unrelated live folio from LRU in the aliasing case.
> 
> The first 2 fixes were reported as pre-existing issues by sashiko in my
> PMD swap entry series [1]. Hopefully sashiko won't point these out
> in the next PMD swap entry series :)

Hello Usama,

Thank you for the fixes! I think all of them look good and they are fixes
so please feel free to add my

Reviewed-by: Joshua Hahn <joshua.hahnjy@gmail.com>

One thing I wanted to maybe consider is for 1/3, whether it could be
good to gate by !pmd_present() like the sites in 2/3 and 3/3, maybe
that will be more robust in the future if there are other cases we
should protect against (and the same as queue_folios_pte_range, which
uses the !pte_present() check too).

For 2/3 and 3/3 we already have the !pmd_present() check, would it be
worth considering just changing the VM_BUG_ON condition to include
!pmd_is_device_private_entry()?

Just wanted to toss my 2c. Thanks again for the fixes Usama!
Joshua

> [1] https://sashiko.dev/#/patchset/20260703173903.3789516-1-usama.arif%40linux.dev?part=6
>  
> Usama Arif (3):
>   mm/mempolicy: skip device-private PMDs when queueing folios
>   mm/madvise: skip device-private PMDs in cold and pageout walks
>   mm/huge_memory: skip device-private PMDs in madvise_free_huge_pmd
> 
>  mm/huge_memory.c | 3 +++
>  mm/madvise.c     | 3 +++
>  mm/mempolicy.c   | 2 ++
>  3 files changed, 8 insertions(+)
> 
> -- 
> 2.53.0-Meta

Re: [PATCH 1/3] mm/mempolicy: skip device-private PMDs when queueing folios

[Zi Yan]

On Tue Jul 7, 2026 at 9:45 AM EDT, Usama Arif wrote:
> queue_folios_pmd() is called under pmd_trans_huge_lock(), whose
> pmd_is_huge() check returns true for any non-present, non-none huge
> PMD - including a device-private swap entry. Passing such a PMD to
> pmd_folio() extracts garbage bits as a PFN and returns a bogus folio
> pointer.
>
> Potential trigger: an HMM-based GPU driver migrates an anonymous THP
> folio to device memory via migrate_vma_pages(), leaving a device-private
> PMD. Userspace then calls mbind(), migrate_pages() or
> set_mempolicy_home_node() on that range.
>
> Skip device-private PMDs, matching how queue_folios_pte_range() skips
> device-private PTE entries by checking !pte_present().
>
> Reported-by: sashiko-bot <sashiko-bot@kernel.org>
> Link: https://sashiko.dev/#/patchset/20260703173903.3789516-1-usama.arif%40linux.dev?part=6
> Fixes: 368076f52ebe ("mm/huge_memory: add device-private THP support to PMD operations")
> Signed-off-by: Usama Arif <usama.arif@linux.dev>
> ---
>  mm/mempolicy.c | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/mm/mempolicy.c b/mm/mempolicy.c
> index 914f81863db5..eda817539c77 100644
> --- a/mm/mempolicy.c
> +++ b/mm/mempolicy.c
> @@ -659,6 +659,8 @@ static void queue_folios_pmd(pmd_t *pmd, struct mm_walk *walk)
>  		qp->nr_failed++;
>  		return;
>  	}
> +	if (unlikely(pmd_is_device_private_entry(*pmd)))
> +		return;

Should it also increase qp->nr_failed like pmd migration entry above?
It also applies to your "if (unlikely(pmd_is_swap_entry(*pmd)))" in the
PMD-level swap patchset.

In addition, since pmd_is_huge() returns true for all non-present
non-none pmds, other softleaf entries should also be excluded here? Oh,
because there is no PMD level hwpoison or pmd marker.

-- 
Best Regards,
Yan, Zi

Re: [PATCH 2/3] mm/madvise: skip device-private PMDs in cold and pageout walks

[Zi Yan]

On Tue Jul 7, 2026 at 9:45 AM EDT, Usama Arif wrote:
> madvise_cold_or_pageout_pte_range() takes pmd_trans_huge_lock(), whose
> pmd_is_huge() check returns true for a device-private PMD. The subsequent
> !pmd_present() branch has a VM_BUG_ON() asserting migration is the only
> allowed non-present case; a device-private PMD trips it.
>
> Potential trigger: an HMM-based GPU driver races with
> madvise(MADV_COLD)/MADV_PAGEOUT: pmd_trans_huge(*pmd) reads true, then
> migrate_vma_pages() flips the PMD to a device-private entry before the
> PMD lock is acquired.
>
> Skip device-private PMDs after taking the lock, before the !pmd_present()
> check.
>
> Reported-by: sashiko-bot <sashiko-bot@kernel.org>
> Link: https://sashiko.dev/#/patchset/20260703173903.3789516-1-usama.arif%40linux.dev?part=6
> Fixes: 368076f52ebe ("mm/huge_memory: add device-private THP support to PMD operations")
> Signed-off-by: Usama Arif <usama.arif@linux.dev>
> ---
>  mm/madvise.c | 3 +++
>  1 file changed, 3 insertions(+)
>

LGTM.

Reviewed-by: Zi Yan <ziy@nvidia.com>

Since it could trigger VM_BUG_ON(), probably cc stable?


-- 
Best Regards,
Yan, Zi

Re: [PATCH 3/3] mm/huge_memory: skip device-private PMDs in madvise_free_huge_pmd

[Zi Yan]

On Tue Jul 7, 2026 at 9:45 AM EDT, Usama Arif wrote:
> madvise_free_pte_range() checks pmd_trans_huge(*pmd) unlocked, then
> madvise_free_huge_pmd() takes pmd_trans_huge_lock().
> pmd_is_huge() returns true for a device-private PMD, so orig_pmd can
> be device-private and hit the VM_BUG_ON() on the !pmd_present() branch.
>
> Potential trigger: an HMM-based GPU driver races with madvise(MADV_FREE):
> migrate_vma_pages() flips the PMD to a device-private entry between the
> caller's pmd_trans_huge() check and the callee's pmd_trans_huge_lock().
>
> Skip device-private PMDs after taking the lock, before the
> !pmd_present() check.
>
> Fixes: 368076f52ebe ("mm/huge_memory: add device-private THP support to PMD operations")
> Signed-off-by: Usama Arif <usama.arif@linux.dev>
> ---
>  mm/huge_memory.c | 3 +++
>  1 file changed, 3 insertions(+)
>
LGTM. cc stable as well.

Reviewed-by: Zi Yan <ziy@nvidia.com>

-- 
Best Regards,
Yan, Zi

Re: [PATCH 0/3] mm: handle device-private PMDs in walk callbacks

[Balbir Singh]

On Tue, Jul 07, 2026 at 03:01:56PM +0100, Lorenzo Stoakes wrote:
> +cc Balbir
> 
> Thanks for the series!
> 
> I think Balbir should be cc'd no?
>

Thanks, my linux-mm monitoring has been largely skimming through the
list. Explict cc's are always appreciated.

> On Tue, Jul 07, 2026 at 06:45:06AM -0700, Usama Arif wrote:
> > Since commit 368076f52ebe ("mm/huge_memory: add device-private THP support
> > to PMD operations") a PMD may hold a device-private swap entry whenever
> > an HMM-based GPU driver migrates an anonymous THP folio to device memory
> > via migrate_vma_pages().
> >
> > pmd_trans_huge_lock() succeeds for such PMDs (pmd_is_huge() returns true
> > for any non-present, non-none huge PMD), so several MM walk callbacks
> > that used to assume present THP or migration entry are now reachable with
> > a device-private PMD. The results range from a VM_BUG_ON() firing on debug
> > kernels, to an oops on a bogus vmemmap dereference, to silently isolating
> > an unrelated live folio from LRU in the aliasing case.
> >
> > The first 2 fixes were reported as pre-existing issues by sashiko in my
> > PMD swap entry series [1]. Hopefully sashiko won't point these out
> > in the next PMD swap entry series :)
> 
> Yeah I don't love these 'existing problem' reports (distracting from other work,
> add workload, really it feels like that should be a passive reporting mode for
> sashiko not arbitrarily added to other series), but sending it as a separate
> series is indeed the right way :)
> 
> These of course should never block an unrelated series.
> 
> >
> > [1] https://sashiko.dev/#/patchset/20260703173903.3789516-1-usama.arif%40linux.dev?part=6
> 
> Hmm we've had a whole host of issues with device-private PMDs recently.
> 
> Balbir - would it be possible to audit all of the code paths and proactively see
> if there's anything else that could hit problems here?
> 

Let me run through this and audit them. My usage of device private PMD
did not expose them, but I can see a bunch of reports in the link above



> >
> > Usama Arif (3):
> >   mm/mempolicy: skip device-private PMDs when queueing folios
> >   mm/madvise: skip device-private PMDs in cold and pageout walks
> >   mm/huge_memory: skip device-private PMDs in madvise_free_huge_pmd
> >
> >  mm/huge_memory.c | 3 +++
> >  mm/madvise.c     | 3 +++
> >  mm/mempolicy.c   | 2 ++
> >  3 files changed, 8 insertions(+)
> >
> > --
> > 2.53.0-Meta
> >
>

Thanks Lorenzo!

Balbir

Re: [PATCH 3/3] mm/huge_memory: skip device-private PMDs in madvise_free_huge_pmd

[Balbir Singh]

On Tue, Jul 07, 2026 at 06:45:09AM -0700, Usama Arif wrote:
> madvise_free_pte_range() checks pmd_trans_huge(*pmd) unlocked, then
> madvise_free_huge_pmd() takes pmd_trans_huge_lock().
> pmd_is_huge() returns true for a device-private PMD, so orig_pmd can
> be device-private and hit the VM_BUG_ON() on the !pmd_present() branch.
> 
> Potential trigger: an HMM-based GPU driver races with madvise(MADV_FREE):
> migrate_vma_pages() flips the PMD to a device-private entry between the
> caller's pmd_trans_huge() check and the callee's pmd_trans_huge_lock().
> 
> Skip device-private PMDs after taking the lock, before the
> !pmd_present() check.
> 
> Fixes: 368076f52ebe ("mm/huge_memory: add device-private THP support to PMD operations")
> Signed-off-by: Usama Arif <usama.arif@linux.dev>
> ---
>  mm/huge_memory.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index c0892cc533a9..cfce9f31b30e 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -2296,6 +2296,9 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,
>  	if (is_huge_zero_pmd(orig_pmd))
>  		goto out;
>  
> +	if (pmd_is_device_private_entry(orig_pmd))
> +		goto out;
> +
>  	if (unlikely(!pmd_present(orig_pmd))) {
>  		VM_BUG_ON(thp_migration_supported() &&
>  				  !pmd_is_migration_entry(orig_pmd));
>

The change makes sense, I spent some time thinking if we should make
madvise pmd free work for THP PMD entries, but that should be a separate
patchset, it is much more complex and involved.

Reviewed-by: Balbir Singh <balbirs@nvidia.com>

Re: [PATCH 2/3] mm/madvise: skip device-private PMDs in cold and pageout walks

[Balbir Singh]

On Tue, Jul 07, 2026 at 06:45:08AM -0700, Usama Arif wrote:
> madvise_cold_or_pageout_pte_range() takes pmd_trans_huge_lock(), whose
> pmd_is_huge() check returns true for a device-private PMD. The subsequent
> !pmd_present() branch has a VM_BUG_ON() asserting migration is the only
> allowed non-present case; a device-private PMD trips it.
> 
> Potential trigger: an HMM-based GPU driver races with
> madvise(MADV_COLD)/MADV_PAGEOUT: pmd_trans_huge(*pmd) reads true, then
> migrate_vma_pages() flips the PMD to a device-private entry before the
> PMD lock is acquired.
> 
> Skip device-private PMDs after taking the lock, before the !pmd_present()
> check.
> 
> Reported-by: sashiko-bot <sashiko-bot@kernel.org>
> Link: https://sashiko.dev/#/patchset/20260703173903.3789516-1-usama.arif%40linux.dev?part=6
> Fixes: 368076f52ebe ("mm/huge_memory: add device-private THP support to PMD operations")
> Signed-off-by: Usama Arif <usama.arif@linux.dev>
> ---
>  mm/madvise.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/mm/madvise.c b/mm/madvise.c
> index 9292f60b19aa..870be398c6f3 100644
> --- a/mm/madvise.c
> +++ b/mm/madvise.c
> @@ -384,6 +384,9 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,
>  			return 0;
>  
>  		orig_pmd = *pmd;
> +		if (pmd_is_device_private_entry(orig_pmd))
> +			goto huge_unlock;
> +
>  		if (is_huge_zero_pmd(orig_pmd))
>  			goto huge_unlock;
>  
>

Reviewed-by: Balbir Singh <balbirs@nvidia.com>

Re: [PATCH 1/3] mm/mempolicy: skip device-private PMDs when queueing folios

[Balbir Singh]

On Tue, Jul 07, 2026 at 06:45:07AM -0700, Usama Arif wrote:
> queue_folios_pmd() is called under pmd_trans_huge_lock(), whose
> pmd_is_huge() check returns true for any non-present, non-none huge
> PMD - including a device-private swap entry. Passing such a PMD to
> pmd_folio() extracts garbage bits as a PFN and returns a bogus folio
> pointer.
> 
> Potential trigger: an HMM-based GPU driver migrates an anonymous THP
> folio to device memory via migrate_vma_pages(), leaving a device-private
> PMD. Userspace then calls mbind(), migrate_pages() or
> set_mempolicy_home_node() on that range.
> 
> Skip device-private PMDs, matching how queue_folios_pte_range() skips
> device-private PTE entries by checking !pte_present().
>
> Reported-by: sashiko-bot <sashiko-bot@kernel.org>
> Link: https://sashiko.dev/#/patchset/20260703173903.3789516-1-usama.arif%40linux.dev?part=6
> Fixes: 368076f52ebe ("mm/huge_memory: add device-private THP support to PMD operations")
> Signed-off-by: Usama Arif <usama.arif@linux.dev>
> ---
>  mm/mempolicy.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/mm/mempolicy.c b/mm/mempolicy.c
> index 914f81863db5..eda817539c77 100644
> --- a/mm/mempolicy.c
> +++ b/mm/mempolicy.c
> @@ -659,6 +659,8 @@ static void queue_folios_pmd(pmd_t *pmd, struct mm_walk *walk)
>  		qp->nr_failed++;
>  		return;
>  	}
> +	if (unlikely(pmd_is_device_private_entry(*pmd)))
> +		return;

The changelog lead me to believe that this is skipped via
!pmd_present()? Do we need to skip just device private entries Shouldn't
we set qp->nr_failed++ similar to what we do for migration entries or do
we silently ignore them?

>  	folio = pmd_folio(*pmd);
>  	if (is_huge_zero_folio(folio)) {
>  		walk->action = ACTION_CONTINUE;
> 

Thanks,
Balbir

Re: [PATCH 0/3] mm: handle device-private PMDs in walk callbacks

[Lorenzo Stoakes]

On Tue, Jul 07, 2026 at 03:12:11PM +0100, Usama Arif wrote:
> I think maybe we should modify get_maintainers.pl so that if there is
> a "Fixes" in commit message, than the author of that commit is also
> in the output from that script?
>

I thought it did that already, but it's forbidden perl magic that I dare not
look at so who knows :P

Wasn't nagging per se about that (for once :P), often people who'd be good to cc-
get missed anyway even doing all the right things, so just wanted to make sure
Balbir was aware, also so we can make sure there's no other device private PMD
stuff lurking! :)

Cheers, Lorenzo

Re: [PATCH 0/3] mm: handle device-private PMDs in walk callbacks

[Lorenzo Stoakes]

On Wed, Jul 08, 2026 at 08:57:51AM +1000, Balbir Singh wrote:
> On Tue, Jul 07, 2026 at 03:01:56PM +0100, Lorenzo Stoakes wrote:
> > +cc Balbir
> >
> > Thanks for the series!
> >
> > I think Balbir should be cc'd no?
> >
>
> Thanks, my linux-mm monitoring has been largely skimming through the
> list. Explict cc's are always appreciated.

Honestly I don't know how you do it, I gave up on linux-mm a long time ago and
pretty much rely on cc- now (which isn't always perfect ofc... :)

>
> > On Tue, Jul 07, 2026 at 06:45:06AM -0700, Usama Arif wrote:
> > > Since commit 368076f52ebe ("mm/huge_memory: add device-private THP support
> > > to PMD operations") a PMD may hold a device-private swap entry whenever
> > > an HMM-based GPU driver migrates an anonymous THP folio to device memory
> > > via migrate_vma_pages().
> > >
> > > pmd_trans_huge_lock() succeeds for such PMDs (pmd_is_huge() returns true
> > > for any non-present, non-none huge PMD), so several MM walk callbacks
> > > that used to assume present THP or migration entry are now reachable with
> > > a device-private PMD. The results range from a VM_BUG_ON() firing on debug
> > > kernels, to an oops on a bogus vmemmap dereference, to silently isolating
> > > an unrelated live folio from LRU in the aliasing case.
> > >
> > > The first 2 fixes were reported as pre-existing issues by sashiko in my
> > > PMD swap entry series [1]. Hopefully sashiko won't point these out
> > > in the next PMD swap entry series :)
> >
> > Yeah I don't love these 'existing problem' reports (distracting from other work,
> > add workload, really it feels like that should be a passive reporting mode for
> > sashiko not arbitrarily added to other series), but sending it as a separate
> > series is indeed the right way :)
> >
> > These of course should never block an unrelated series.
> >
> > >
> > > [1] https://sashiko.dev/#/patchset/20260703173903.3789516-1-usama.arif%40linux.dev?part=6
> >
> > Hmm we've had a whole host of issues with device-private PMDs recently.
> >
> > Balbir - would it be possible to audit all of the code paths and proactively see
> > if there's anything else that could hit problems here?
> >
>
> Let me run through this and audit them. My usage of device private PMD
> did not expose them, but I can see a bunch of reports in the link above

Great thanks :)

>
> > >
> > > Usama Arif (3):
> > >   mm/mempolicy: skip device-private PMDs when queueing folios
> > >   mm/madvise: skip device-private PMDs in cold and pageout walks
> > >   mm/huge_memory: skip device-private PMDs in madvise_free_huge_pmd
> > >
> > >  mm/huge_memory.c | 3 +++
> > >  mm/madvise.c     | 3 +++
> > >  mm/mempolicy.c   | 2 ++
> > >  3 files changed, 8 insertions(+)
> > >
> > > --
> > > 2.53.0-Meta
> > >
> >
>
> Thanks Lorenzo!

You're welcome! :)

>
> Balbir

Cheers, Lorenzo

Re: [PATCH 0/3] mm: handle device-private PMDs in walk callbacks

[Balbir Singh]

On Wed, Jul 08, 2026 at 09:23:36AM +0100, Lorenzo Stoakes wrote:
> On Tue, Jul 07, 2026 at 03:12:11PM +0100, Usama Arif wrote:
> > I think maybe we should modify get_maintainers.pl so that if there is
> > a "Fixes" in commit message, than the author of that commit is also
> > in the output from that script?
> >
> 
> I thought it did that already, but it's forbidden perl magic that I dare not
> look at so who knows :P
> 
> Wasn't nagging per se about that (for once :P), often people who'd be good to cc-
> get missed anyway even doing all the right things, so just wanted to make sure
> Balbir was aware, also so we can make sure there's no other device private PMD
> stuff lurking! :)
>

Yes, thanks for brining it to my attention! I have started looking at
the set of changes again to see if

1. I can improve test coverage (mempolicy tests for example) and madvise
2. If there is lurking code that has not been tested at my end or
covered.

It's a slow burn, but I do appreciate a Cc, I skim through linux-mm, but
the volume may make me miss topics of interest due to TZ and other
factors depending on when I check email.

> Cheers, Lorenzo

Cheers, Balbir

Re: [PATCH 1/3] mm/mempolicy: skip device-private PMDs when queueing folios

[Usama Arif]

On 07/07/2026 21:33, Zi Yan wrote:
> On Tue Jul 7, 2026 at 9:45 AM EDT, Usama Arif wrote:
>> queue_folios_pmd() is called under pmd_trans_huge_lock(), whose
>> pmd_is_huge() check returns true for any non-present, non-none huge
>> PMD - including a device-private swap entry. Passing such a PMD to
>> pmd_folio() extracts garbage bits as a PFN and returns a bogus folio
>> pointer.
>>
>> Potential trigger: an HMM-based GPU driver migrates an anonymous THP
>> folio to device memory via migrate_vma_pages(), leaving a device-private
>> PMD. Userspace then calls mbind(), migrate_pages() or
>> set_mempolicy_home_node() on that range.
>>
>> Skip device-private PMDs, matching how queue_folios_pte_range() skips
>> device-private PTE entries by checking !pte_present().
>>
>> Reported-by: sashiko-bot <sashiko-bot@kernel.org>
>> Link: https://sashiko.dev/#/patchset/20260703173903.3789516-1-usama.arif%40linux.dev?part=6
>> Fixes: 368076f52ebe ("mm/huge_memory: add device-private THP support to PMD operations")
>> Signed-off-by: Usama Arif <usama.arif@linux.dev>
>> ---
>>  mm/mempolicy.c | 2 ++
>>  1 file changed, 2 insertions(+)
>>
>> diff --git a/mm/mempolicy.c b/mm/mempolicy.c
>> index 914f81863db5..eda817539c77 100644
>> --- a/mm/mempolicy.c
>> +++ b/mm/mempolicy.c
>> @@ -659,6 +659,8 @@ static void queue_folios_pmd(pmd_t *pmd, struct mm_walk *walk)
>>  		qp->nr_failed++;
>>  		return;
>>  	}
>> +	if (unlikely(pmd_is_device_private_entry(*pmd)))
>> +		return;
> 
> Should it also increase qp->nr_failed like pmd migration entry above?
> It also applies to your "if (unlikely(pmd_is_swap_entry(*pmd)))" in the
> PMD-level swap patchset.

The PTE path only increments nr_failed for migration entries, other non-present
entries, including device-private and swap, are skipped. I kept things aligned
with the PTE path.

> In addition, since pmd_is_huge() returns true for all non-present
> non-none pmds, other softleaf entries should also be excluded here? Oh,
> because there is no PMD level hwpoison or pmd marker.
> 

I think what Joshua recommended in [1] makes sense, I will change the PMD code
to gate on !pmd_present() instead. That keeps migration entry accounting, skips
device-private now, and will also avoid calling pmd_folio() on any future non-present
PMD softleaf such as PMD swap.

[1] https://lore.kernel.org/all/20260707191917.3213033-1-joshua.hahnjy@gmail.com/

Re: [PATCH 0/3] mm: handle device-private PMDs in walk callbacks

[Usama Arif]

On 07/07/2026 20:19, Joshua Hahn wrote:
> On Tue,  7 Jul 2026 06:45:06 -0700 Usama Arif <usama.arif@linux.dev> wrote:
> 
>> Since commit 368076f52ebe ("mm/huge_memory: add device-private THP support
>> to PMD operations") a PMD may hold a device-private swap entry whenever
>> an HMM-based GPU driver migrates an anonymous THP folio to device memory
>> via migrate_vma_pages().
>>
>> pmd_trans_huge_lock() succeeds for such PMDs (pmd_is_huge() returns true
>> for any non-present, non-none huge PMD), so several MM walk callbacks
>> that used to assume present THP or migration entry are now reachable with
>> a device-private PMD. The results range from a VM_BUG_ON() firing on debug
>> kernels, to an oops on a bogus vmemmap dereference, to silently isolating
>> an unrelated live folio from LRU in the aliasing case.
>>
>> The first 2 fixes were reported as pre-existing issues by sashiko in my
>> PMD swap entry series [1]. Hopefully sashiko won't point these out
>> in the next PMD swap entry series :)
> 
> Hello Usama,
> 
> Thank you for the fixes! I think all of them look good and they are fixes
> so please feel free to add my
> 
> Reviewed-by: Joshua Hahn <joshua.hahnjy@gmail.com>
> 
> One thing I wanted to maybe consider is for 1/3, whether it could be
> good to gate by !pmd_present() like the sites in 2/3 and 3/3, maybe
> that will be more robust in the future if there are other cases we
> should protect against (and the same as queue_folios_pte_range, which
> uses the !pte_present() check too).
> 
> For 2/3 and 3/3 we already have the !pmd_present() check, would it be
> worth considering just changing the VM_BUG_ON condition to include
> !pmd_is_device_private_entry()?
> 
> Just wanted to toss my 2c. Thanks again for the fixes Usama!
> Joshua
> 
Thanks for the reviews Joshua, I think they all made sense.
I have sent v2 [1] following your suggestions.

[1] https://lore.kernel.org/all/20260708122040.861335-1-usama.arif@linux.dev/

Re: [PATCH 1/3] mm/mempolicy: skip device-private PMDs when queueing folios

[Zi Yan]

On 8 Jul 2026, at 6:54, Usama Arif wrote:

> On 07/07/2026 21:33, Zi Yan wrote:
>> On Tue Jul 7, 2026 at 9:45 AM EDT, Usama Arif wrote:
>>> queue_folios_pmd() is called under pmd_trans_huge_lock(), whose
>>> pmd_is_huge() check returns true for any non-present, non-none huge
>>> PMD - including a device-private swap entry. Passing such a PMD to
>>> pmd_folio() extracts garbage bits as a PFN and returns a bogus folio
>>> pointer.
>>>
>>> Potential trigger: an HMM-based GPU driver migrates an anonymous THP
>>> folio to device memory via migrate_vma_pages(), leaving a device-private
>>> PMD. Userspace then calls mbind(), migrate_pages() or
>>> set_mempolicy_home_node() on that range.
>>>
>>> Skip device-private PMDs, matching how queue_folios_pte_range() skips
>>> device-private PTE entries by checking !pte_present().
>>>
>>> Reported-by: sashiko-bot <sashiko-bot@kernel.org>
>>> Link: https://sashiko.dev/#/patchset/20260703173903.3789516-1-usama.arif%40linux.dev?part=6
>>> Fixes: 368076f52ebe ("mm/huge_memory: add device-private THP support to PMD operations")
>>> Signed-off-by: Usama Arif <usama.arif@linux.dev>
>>> ---
>>>  mm/mempolicy.c | 2 ++
>>>  1 file changed, 2 insertions(+)
>>>
>>> diff --git a/mm/mempolicy.c b/mm/mempolicy.c
>>> index 914f81863db5..eda817539c77 100644
>>> --- a/mm/mempolicy.c
>>> +++ b/mm/mempolicy.c
>>> @@ -659,6 +659,8 @@ static void queue_folios_pmd(pmd_t *pmd, struct mm_walk *walk)
>>>  		qp->nr_failed++;
>>>  		return;
>>>  	}
>>> +	if (unlikely(pmd_is_device_private_entry(*pmd)))
>>> +		return;
>>
>> Should it also increase qp->nr_failed like pmd migration entry above?
>> It also applies to your "if (unlikely(pmd_is_swap_entry(*pmd)))" in the
>> PMD-level swap patchset.
>
> The PTE path only increments nr_failed for migration entries, other non-present
> entries, including device-private and swap, are skipped. I kept things aligned
> with the PTE path.
>
>> In addition, since pmd_is_huge() returns true for all non-present
>> non-none pmds, other softleaf entries should also be excluded here? Oh,
>> because there is no PMD level hwpoison or pmd marker.
>>
>
> I think what Joshua recommended in [1] makes sense, I will change the PMD code
> to gate on !pmd_present() instead. That keeps migration entry accounting, skips
> device-private now, and will also avoid calling pmd_folio() on any future non-present
> PMD softleaf such as PMD swap.
>
> [1] https://lore.kernel.org/all/20260707191917.3213033-1-joshua.hahnjy@gmail.com/

Makes sense. Thank you for the explanation.

Best Regards,
Yan, Zi