[PATCH] mm/hmm: skip NUMA balancing on protnone faults

[PATCH] mm/hmm: skip NUMA balancing on protnone faults

[Matthew Auld]

When automatic NUMA balancing marks a PTE or PMD as protnone
(PAGE_NONE), hmm_pte_need_fault() requests a fault to restore access
permissions, including _PAGE_RW when write access is needed.

Calling handle_mm_fault() from hmm_do_fault() currently triggers NUMA
balancing in do_numa_page() and do_huge_pmd_numa_page(). When
hmm_range_fault() runs from a device driver or kworker context (such as
a GPU SVM or userptr rebind worker), folios get migrated toward the CPU
node executing the remote fault.

Also, each folio migration invokes try_to_migrate_one() ->
mmu_notifier_invalidate_range_start(), which invalidates the
caller's mmu_interval_notifier while holding mmap_lock. That causes
hmm_range_fault() to abort with -EBUSY after every 2 MiB PMD and exhaust
its retry timeout on large buffers, which has been observed with large
imported Xe userptr buffers.

Add FAULT_FLAG_NO_NUMA and set it in hmm_do_fault() so protnone entries
restore PTE/PMD permissions in-place without performing NUMA balancing,
matching GUP's default !FOLL_HONOR_NUMA_FAULT behaviour.

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: Mika Penttilä <mpenttil@redhat.com>
Cc: Leon Romanovsky <leon@kernel.org>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
---
 include/linux/mm.h       | 3 ++-
 include/linux/mm_types.h | 3 +++
 mm/hmm.c                 | 2 +-
 mm/huge_memory.c         | 3 +++
 mm/memory.c              | 3 +++
 5 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/include/linux/mm.h b/include/linux/mm.h
index 77b5d450fe45..4b6ad3c0b591 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -717,7 +717,8 @@ static inline bool fault_flag_allow_retry_first(enum fault_flag flags)
 	{ FAULT_FLAG_REMOTE,		"REMOTE" }, \
 	{ FAULT_FLAG_INSTRUCTION,	"INSTRUCTION" }, \
 	{ FAULT_FLAG_INTERRUPTIBLE,	"INTERRUPTIBLE" }, \
-	{ FAULT_FLAG_VMA_LOCK,		"VMA_LOCK" }
+	{ FAULT_FLAG_VMA_LOCK,		"VMA_LOCK" }, \
+	{ FAULT_FLAG_NO_NUMA,		"NO_NUMA" }
 
 /*
  * vm_fault is filled by the pagefault handler and passed to the vma's
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 1ed35d258d2e..2ea71c1bb170 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -1819,6 +1819,8 @@ enum tlb_flush_reason {
  * @FAULT_FLAG_ORIG_PTE_VALID: whether the fault has vmf->orig_pte cached.
  *                        We should only access orig_pte if this flag set.
  * @FAULT_FLAG_VMA_LOCK: The fault is handled under VMA lock.
+ * @FAULT_FLAG_NO_NUMA: Do not perform NUMA balancing migration or NUMA
+ *                      task access accounting on protnone faults.
  *
  * About @FAULT_FLAG_ALLOW_RETRY and @FAULT_FLAG_TRIED: we can specify
  * whether we would allow page faults to retry by specifying these two
@@ -1857,6 +1859,7 @@ enum fault_flag {
 	FAULT_FLAG_UNSHARE =		1 << 10,
 	FAULT_FLAG_ORIG_PTE_VALID =	1 << 11,
 	FAULT_FLAG_VMA_LOCK =		1 << 12,
+	FAULT_FLAG_NO_NUMA =		1 << 13,
 };
 
 typedef unsigned int __bitwise zap_flags_t;
diff --git a/mm/hmm.c b/mm/hmm.c
index 2f1e98c6b644..62cdeb071781 100644
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@ -648,7 +648,7 @@ static int hmm_do_fault(struct mm_struct *mm,
 	unsigned long addr = hmm_vma_walk->last;
 	unsigned long end = hmm_vma_walk->end;
 	unsigned int required_fault = hmm_vma_walk->required_fault;
-	unsigned int fault_flags = FAULT_FLAG_REMOTE;
+	unsigned int fault_flags = FAULT_FLAG_REMOTE | FAULT_FLAG_NO_NUMA;
 	struct vm_area_struct *vma;
 
 	if (hmm_vma_walk->locked)
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 0d589be136d2..ea380bf45df0 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2361,6 +2361,9 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf)
 	if (!folio)
 		goto out_map;
 
+	if (vmf->flags & FAULT_FLAG_NO_NUMA)
+		goto out_map;
+
 	nid = folio_nid(folio);
 
 	target_nid = numa_migrate_check(folio, vmf, haddr, &flags, writable,
diff --git a/mm/memory.c b/mm/memory.c
index fe72e66c9c2e..53c819a6e49a 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -6404,6 +6404,9 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf)
 	if (!folio || folio_is_zone_device(folio))
 		goto out_map;
 
+	if (vmf->flags & FAULT_FLAG_NO_NUMA)
+		goto out_map;
+
 	nid = folio_nid(folio);
 	nr_pages = folio_nr_pages(folio);
 

---
base-commit: 4318887a3efa4c53d6f083ccfd7105b809a3ac65
change-id: 20261112-numa-3649f4bf57c9

Re: [PATCH] mm/hmm: skip NUMA balancing on protnone faults

[David Hildenbrand (Arm)]

On 11/13/26 17:25, Matthew Auld wrote:
> When automatic NUMA balancing marks a PTE or PMD as protnone
> (PAGE_NONE), hmm_pte_need_fault() requests a fault to restore access
> permissions, including _PAGE_RW when write access is needed.
> 
> Calling handle_mm_fault() from hmm_do_fault() currently triggers NUMA
> balancing in do_numa_page() and do_huge_pmd_numa_page(). When
> hmm_range_fault() runs from a device driver or kworker context (such as
> a GPU SVM or userptr rebind worker), folios get migrated toward the CPU
> node executing the remote fault.
> 
> Also, each folio migration invokes try_to_migrate_one() ->
> mmu_notifier_invalidate_range_start(), which invalidates the
> caller's mmu_interval_notifier while holding mmap_lock. That causes
> hmm_range_fault() to abort with -EBUSY after every 2 MiB PMD and exhaust
> its retry timeout on large buffers, which has been observed with large
> imported Xe userptr buffers.
> 
> Add FAULT_FLAG_NO_NUMA and set it in hmm_do_fault() so protnone entries
> restore PTE/PMD permissions in-place without performing NUMA balancing,
> matching GUP's default !FOLL_HONOR_NUMA_FAULT behaviour.

Ehm, that's not how it works.

GUP will not trigger a "special" fault and will try not triggering a fault and
not restore permissions.

It will try continuing, leaving the protnone pte as it is.

See gup_can_follow_protnone().

For read-only access that should always work.

For write-access we always trigger an ordinary fault and do NUMA migration etc.


So FAULT_FLAG_NO_NUMA is doing something completely different and I don't agree
with it.


-- 
Cheers,

David

Re: [PATCH] mm/hmm: skip NUMA balancing on protnone faults

[Thomas Hellström]

Hi, David

On Fri, 2026-11-13 at 19:27 +0100, David Hildenbrand (Arm) wrote:
> On 11/13/26 17:25, Matthew Auld wrote:
> > When automatic NUMA balancing marks a PTE or PMD as protnone
> > (PAGE_NONE), hmm_pte_need_fault() requests a fault to restore
> > access
> > permissions, including _PAGE_RW when write access is needed.
> > 
> > Calling handle_mm_fault() from hmm_do_fault() currently triggers
> > NUMA
> > balancing in do_numa_page() and do_huge_pmd_numa_page(). When
> > hmm_range_fault() runs from a device driver or kworker context
> > (such as
> > a GPU SVM or userptr rebind worker), folios get migrated toward the
> > CPU
> > node executing the remote fault.
> > 
> > Also, each folio migration invokes try_to_migrate_one() ->
> > mmu_notifier_invalidate_range_start(), which invalidates the
> > caller's mmu_interval_notifier while holding mmap_lock. That causes
> > hmm_range_fault() to abort with -EBUSY after every 2 MiB PMD and
> > exhaust
> > its retry timeout on large buffers, which has been observed with
> > large
> > imported Xe userptr buffers.
> > 
> > Add FAULT_FLAG_NO_NUMA and set it in hmm_do_fault() so protnone
> > entries
> > restore PTE/PMD permissions in-place without performing NUMA
> > balancing,
> > matching GUP's default !FOLL_HONOR_NUMA_FAULT behaviour.
> 
> Ehm, that's not how it works.
> 
> GUP will not trigger a "special" fault and will try not triggering a
> fault and
> not restore permissions.
> 
> It will try continuing, leaving the protnone pte as it is.
> 
> See gup_can_follow_protnone().
> 
> For read-only access that should always work.
> 
> For write-access we always trigger an ordinary fault and do NUMA
> migration etc.
> 
> 
> So FAULT_FLAG_NO_NUMA is doing something completely different and I
> don't agree
> with it.
> 

There are two major problems with the current behaviour.

1) The work item executing hmm_range_fault() might have no affinity at
all to the GPU about to access the memory.

2) If migration happens, then hmm_range_fault() will stop and retry
leading to excessive execution time.

While perhaps Matt's commit message doesn't correctly describe the GUP
behaviour, do you have any alternative suggestion than to avoid
migration in this case? I think it's a reasonable idea given the
problems with the current code?

Thanks,
Thomas
 

Re: [PATCH] mm/hmm: skip NUMA balancing on protnone faults

[David Hildenbrand (Arm)]

On 11/13/26 21:34, Thomas Hellström wrote:
> Hi, David
> 
> On Fri, 2026-11-13 at 19:27 +0100, David Hildenbrand (Arm) wrote:
>> On 11/13/26 17:25, Matthew Auld wrote:
>>> When automatic NUMA balancing marks a PTE or PMD as protnone
>>> (PAGE_NONE), hmm_pte_need_fault() requests a fault to restore
>>> access
>>> permissions, including _PAGE_RW when write access is needed.
>>>
>>> Calling handle_mm_fault() from hmm_do_fault() currently triggers
>>> NUMA
>>> balancing in do_numa_page() and do_huge_pmd_numa_page(). When
>>> hmm_range_fault() runs from a device driver or kworker context
>>> (such as
>>> a GPU SVM or userptr rebind worker), folios get migrated toward the
>>> CPU
>>> node executing the remote fault.
>>>
>>> Also, each folio migration invokes try_to_migrate_one() ->
>>> mmu_notifier_invalidate_range_start(), which invalidates the
>>> caller's mmu_interval_notifier while holding mmap_lock. That causes
>>> hmm_range_fault() to abort with -EBUSY after every 2 MiB PMD and
>>> exhaust
>>> its retry timeout on large buffers, which has been observed with
>>> large
>>> imported Xe userptr buffers.
>>>
>>> Add FAULT_FLAG_NO_NUMA and set it in hmm_do_fault() so protnone
>>> entries
>>> restore PTE/PMD permissions in-place without performing NUMA
>>> balancing,
>>> matching GUP's default !FOLL_HONOR_NUMA_FAULT behaviour.
>>
>> Ehm, that's not how it works.
>>
>> GUP will not trigger a "special" fault and will try not triggering a
>> fault and
>> not restore permissions.
>>
>> It will try continuing, leaving the protnone pte as it is.
>>
>> See gup_can_follow_protnone().
>>
>> For read-only access that should always work.
>>
>> For write-access we always trigger an ordinary fault and do NUMA
>> migration etc.
>>
>>
>> So FAULT_FLAG_NO_NUMA is doing something completely different and I
>> don't agree
>> with it.
>>
> 
> There are two major problems with the current behaviour.
> 
> 1) The work item executing hmm_range_fault() might have no affinity at
> all to the GPU about to access the memory.
> 
> 2) If migration happens, then hmm_range_fault() will stop and retry
> leading to excessive execution time.
> 
> While perhaps Matt's commit message doesn't correctly describe the GUP
> behaviour, do you have any alternative suggestion than to avoid
> migration in this case? I think it's a reasonable idea given the
> problems with the current code?

Adding fault reasons that hack around NUMA hinting such that a actual CPU
accessing it later will not trigger ... NUMA hinting?

I might be missing the "reasonable" in here :P

I recall that John (cc) kept saying "GPUs and NUMA balancing is inherently
incompatible, you have to disable NUMA balancing".

You could likely implement something similar to GUP, where you would allow for
bypassing protnone for reading.

That wouldn't give you access to the writing side. There might be ways to sort
out some of the writing (similar to FOLL_FORCE handling in GUP), but I'd rather
not go down that path. KVM also doesn't seem to need that.

-- 
Cheers,

David

Re: [PATCH] mm/hmm: skip NUMA balancing on protnone faults

[Gregory Price (Meta)]

On 2026-11-13 21:43 +0100, David Hildenbrand (Arm) wrote:
> On 11/13/26 21:34, Thomas Hellström wrote:
> > 
> > There are two major problems with the current behaviour.
> > 
> > 1) The work item executing hmm_range_fault() might have no affinity at
> > all to the GPU about to access the memory.
> > 
> > 2) If migration happens, then hmm_range_fault() will stop and retry
> > leading to excessive execution time.
> > 
> > While perhaps Matt's commit message doesn't correctly describe the GUP
> > behaviour, do you have any alternative suggestion than to avoid
> > migration in this case? I think it's a reasonable idea given the
> > problems with the current code?
> 
> Adding fault reasons that hack around NUMA hinting such that a actual CPU
> accessing it later will not trigger ... NUMA hinting?
> 
> I might be missing the "reasonable" in here :P
> 

Ehhh... I think the hint faulting code is already very tenuous, so yeah,
lets maybe not try to go down that route.

> I recall that John (cc) kept saying "GPUs and NUMA balancing is inherently
> incompatible, you have to disable NUMA balancing".
> 

If this is a NUMA-hotplugged GPU, then this is resolved by a private
node, because this:

@@ -6404,6 +6404,9 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf)
        if (!folio || folio_is_zone_device(folio))


changes to

        if (!folio || folio_is_common_memory(folio))

And all folios on the GPU node are never subject to NUMA Balancing

so balancing can stay enabled while the GPU memory is not subject to
PROT_NONE spew.

~Gregory

-- 
~Gregory

Re: [PATCH] mm/hmm: skip NUMA balancing on protnone faults

[Thomas Hellström]

On Fri, 2026-11-13 at 21:43 +0100, David Hildenbrand (Arm) wrote:
> On 11/13/26 21:34, Thomas Hellström wrote:
> > Hi, David
> > 
> > On Fri, 2026-11-13 at 19:27 +0100, David Hildenbrand (Arm) wrote:
> > > On 11/13/26 17:25, Matthew Auld wrote:
> > > > When automatic NUMA balancing marks a PTE or PMD as protnone
> > > > (PAGE_NONE), hmm_pte_need_fault() requests a fault to restore
> > > > access
> > > > permissions, including _PAGE_RW when write access is needed.
> > > > 
> > > > Calling handle_mm_fault() from hmm_do_fault() currently
> > > > triggers
> > > > NUMA
> > > > balancing in do_numa_page() and do_huge_pmd_numa_page(). When
> > > > hmm_range_fault() runs from a device driver or kworker context
> > > > (such as
> > > > a GPU SVM or userptr rebind worker), folios get migrated toward
> > > > the
> > > > CPU
> > > > node executing the remote fault.
> > > > 
> > > > Also, each folio migration invokes try_to_migrate_one() ->
> > > > mmu_notifier_invalidate_range_start(), which invalidates the
> > > > caller's mmu_interval_notifier while holding mmap_lock. That
> > > > causes
> > > > hmm_range_fault() to abort with -EBUSY after every 2 MiB PMD
> > > > and
> > > > exhaust
> > > > its retry timeout on large buffers, which has been observed
> > > > with
> > > > large
> > > > imported Xe userptr buffers.
> > > > 
> > > > Add FAULT_FLAG_NO_NUMA and set it in hmm_do_fault() so protnone
> > > > entries
> > > > restore PTE/PMD permissions in-place without performing NUMA
> > > > balancing,
> > > > matching GUP's default !FOLL_HONOR_NUMA_FAULT behaviour.
> > > 
> > > Ehm, that's not how it works.
> > > 
> > > GUP will not trigger a "special" fault and will try not
> > > triggering a
> > > fault and
> > > not restore permissions.
> > > 
> > > It will try continuing, leaving the protnone pte as it is.
> > > 
> > > See gup_can_follow_protnone().
> > > 
> > > For read-only access that should always work.
> > > 
> > > For write-access we always trigger an ordinary fault and do NUMA
> > > migration etc.
> > > 
> > > 
> > > So FAULT_FLAG_NO_NUMA is doing something completely different and
> > > I
> > > don't agree
> > > with it.
> > > 
> > 
> > There are two major problems with the current behaviour.
> > 
> > 1) The work item executing hmm_range_fault() might have no affinity
> > at
> > all to the GPU about to access the memory.
> > 
> > 2) If migration happens, then hmm_range_fault() will stop and retry
> > leading to excessive execution time.
> > 
> > While perhaps Matt's commit message doesn't correctly describe the
> > GUP
> > behaviour, do you have any alternative suggestion than to avoid
> > migration in this case? I think it's a reasonable idea given the
> > problems with the current code?
> 
> Adding fault reasons that hack around NUMA hinting such that a actual
> CPU
> accessing it later will not trigger ... NUMA hinting?
> 
> I might be missing the "reasonable" in here :P

Wouldn't the net effect be that content stays put instead of being
migrated? In both cases a later CPU access will not trigger NUMA
hinting except that the proposed change doesn't send content to an
irrelevant node?

> 
> I recall that John (cc) kept saying "GPUs and NUMA balancing is
> inherently
> incompatible, you have to disable NUMA balancing".

Perhaps we could force hmm_range_fault() to run on the node closest to
the GPU. Then at least the migration might be beneficial.

> 
> You could likely implement something similar to GUP, where you would
> allow for
> bypassing protnone for reading.

We could ofc look at that. Might help in some cases.

Thanks,
Thomas


> 
> That wouldn't give you access to the writing side. There might be
> ways to sort
> out some of the writing (similar to FOLL_FORCE handling in GUP), but
> I'd rather
> not go down that path. KVM also doesn't seem to need that.

Re: [PATCH] mm/hmm: skip NUMA balancing on protnone faults

[David Hildenbrand (Arm)]

On 11/13/26 21:56, Thomas Hellström wrote:
> On Fri, 2026-11-13 at 21:43 +0100, David Hildenbrand (Arm) wrote:
>> On 11/13/26 21:34, Thomas Hellström wrote:
>>> Hi, David
>>>
>>>
>>> There are two major problems with the current behaviour.
>>>
>>> 1) The work item executing hmm_range_fault() might have no affinity
>>> at
>>> all to the GPU about to access the memory.
>>>
>>> 2) If migration happens, then hmm_range_fault() will stop and retry
>>> leading to excessive execution time.
>>>
>>> While perhaps Matt's commit message doesn't correctly describe the
>>> GUP
>>> behaviour, do you have any alternative suggestion than to avoid
>>> migration in this case? I think it's a reasonable idea given the
>>> problems with the current code?
>>
>> Adding fault reasons that hack around NUMA hinting such that a actual
>> CPU
>> accessing it later will not trigger ... NUMA hinting?
>>
>> I might be missing the "reasonable" in here :P
> 
> Wouldn't the net effect be that content stays put instead of being
> migrated? In both cases a later CPU access will not trigger NUMA
> hinting except that the proposed change doesn't send content to an
> irrelevant node?

My point is that you are essentially disabling NUMA hinting because the CPU will
no longer trigger NUMA hinting faults.

But then, just disable it entirely. Maybe John knows what to do.

> 
>>
>> I recall that John (cc) kept saying "GPUs and NUMA balancing is
>> inherently
>> incompatible, you have to disable NUMA balancing".
> 
> Perhaps we could force hmm_range_fault() to run on the node closest to
> the GPU. Then at least the migration might be beneficial.

That's also true.

> 
>>
>> You could likely implement something similar to GUP, where you would
>> allow for
>> bypassing protnone for reading.
> 
> We could ofc look at that. Might help in some cases.
Would be interesting if that already solves most of your problem. Apparently for
KVM it did, but not sure why the write side was never that important.

-- 
Cheers,

David