Re: [PATCH] mm: mempolicy: fix automatic numa balancing for shmem

Re: [PATCH] mm: mempolicy: fix automatic numa balancing for shmem

[Sean Christopherson]

On Thu, Jul 23, 2026, Yan Zhao wrote:
> On Mon, Jun 29, 2026 at 12:33:37PM -0400, Johannes Weiner wrote:
> > Neha reports that mapped shmem aren't considered for NUMA balancing,
> > noting convergence problems and bandwidth bottlenecking for cachelib
> > based workloads on tiered memory systems.
> > 
> > Looking at the code and going through the git history, this doesn't
> > actually seem intentional:
> > 
> > Commit fc3147245d19 ("mm: numa: Limit NUMA scanning to migrate-on-fault
> > VMAs") added a vma_policy_mof() gate to task_numa_work() so VMAs whose
> > policy lacks MPOL_F_MOF are skipped from NUMA balancing scans. The
> > motivation was a real usecase: Oracle was pinning shared segments with
> > mbind(MPOL_BIND) so trapping faults was both expensive and pointless.
> > 
> > The handling of NULL from vm_ops->get_policy, however, treated "user
> > explicitly opted out" the same as "user never specified anything." For
> > VMAs whose shared policy is absent - the common case for shmem - the
> > scan was disabled too.
> > 
> > This issue is old. It probably hurts less in conventional NUMA. But it's
> > very noticable on tiered systems, where entire tmpfs workingsets can get
> > stuck on lower-bandwidth memory.
> > 
> > Fix this by having vma_policy_mof() use __get_vma_policy() directly, and
> > thereby handle the fallback to task policy (-> preferred_node_policy()
> > has MPOL_F_MOF per default). Every other consumer of vm_ops->get_policy
> > already handles it this way, the scan-eligibility check was the outlier.
> > 
> > This preserves Mel's intended fix: don't scan stuff the user explicitly
> > pinned. But allow default policy vmas to participate in balancing.
> Hi,
> 
> This patch introduces a performance regression of a KVM stress test, which I
> addressed in the KVM selftest itself (see the analysis in the patch log).
> Could you share your thoughts on whether the userspace fix is the appropriate
> approach?

Yikes.  This could have meaningful "real world" impact on VMs backed with shmem,
not just on KVM's convoluted stress test.  NUMA balancing generally performs
poorly for VMs due to the higher costs of VM-Exits versus page faults, and due
to inefficiencies in the mmu_notifier interface (KVM does a full TLB shootdown
of the affected VM on every MMU_NOTIFY_PROTECTION_VMA event).

My stance is that using NUMA balancing with KVM guests is a terrible idea, and
that anyone that insists on using such a setup gets to suffer the consequences.
But in this case, IIUC, this change will "silently" enable NUMA balancing for
shmem-based KVM setups where it was previously disabled (albeit unintentionally).

I'm not fundamentally opposed to the change, but I do worry that downstream KVM
users could be in for a nasty surprise.

Re: [PATCH] mm: mempolicy: fix automatic numa balancing for shmem

[Johannes Weiner]

On Thu, Jul 23, 2026 at 06:51:26AM -0700, Sean Christopherson wrote:
> On Thu, Jul 23, 2026, Yan Zhao wrote:
> > On Mon, Jun 29, 2026 at 12:33:37PM -0400, Johannes Weiner wrote:
> > > Neha reports that mapped shmem aren't considered for NUMA balancing,
> > > noting convergence problems and bandwidth bottlenecking for cachelib
> > > based workloads on tiered memory systems.
> > > 
> > > Looking at the code and going through the git history, this doesn't
> > > actually seem intentional:
> > > 
> > > Commit fc3147245d19 ("mm: numa: Limit NUMA scanning to migrate-on-fault
> > > VMAs") added a vma_policy_mof() gate to task_numa_work() so VMAs whose
> > > policy lacks MPOL_F_MOF are skipped from NUMA balancing scans. The
> > > motivation was a real usecase: Oracle was pinning shared segments with
> > > mbind(MPOL_BIND) so trapping faults was both expensive and pointless.
> > > 
> > > The handling of NULL from vm_ops->get_policy, however, treated "user
> > > explicitly opted out" the same as "user never specified anything." For
> > > VMAs whose shared policy is absent - the common case for shmem - the
> > > scan was disabled too.
> > > 
> > > This issue is old. It probably hurts less in conventional NUMA. But it's
> > > very noticable on tiered systems, where entire tmpfs workingsets can get
> > > stuck on lower-bandwidth memory.
> > > 
> > > Fix this by having vma_policy_mof() use __get_vma_policy() directly, and
> > > thereby handle the fallback to task policy (-> preferred_node_policy()
> > > has MPOL_F_MOF per default). Every other consumer of vm_ops->get_policy
> > > already handles it this way, the scan-eligibility check was the outlier.
> > > 
> > > This preserves Mel's intended fix: don't scan stuff the user explicitly
> > > pinned. But allow default policy vmas to participate in balancing.
> > Hi,
> > 
> > This patch introduces a performance regression of a KVM stress test, which I
> > addressed in the KVM selftest itself (see the analysis in the patch log).
> > Could you share your thoughts on whether the userspace fix is the appropriate
> > approach?
> 
> Yikes.  This could have meaningful "real world" impact on VMs backed with shmem,
> not just on KVM's convoluted stress test.  NUMA balancing generally performs
> poorly for VMs due to the higher costs of VM-Exits versus page faults, and due
> to inefficiencies in the mmu_notifier interface (KVM does a full TLB shootdown
> of the affected VM on every MMU_NOTIFY_PROTECTION_VMA event).
> 
> My stance is that using NUMA balancing with KVM guests is a terrible idea, and
> that anyone that insists on using such a setup gets to suffer the consequences.

I agree. Surely we cannot leave shmem balancing broken due to
that. It's also not clear to me how many people even enable numa
balancing.

> But in this case, IIUC, this change will "silently" enable NUMA balancing for
> shmem-based KVM setups where it was previously disabled (albeit unintentionally).
> 
> I'm not fundamentally opposed to the change, but I do worry that downstream KVM
> users could be in for a nasty surprise.

This should be very visible in early kernel validation after an
upgrade. VM hosts tend to not do much else, with VM memory dominating
the host. You'd expect a significant uptick in numa_pte_updates,
numa_hint_faults, and a change in per-node nr_shmem stats. And the
patch subject makes it trivial to find in a commit delta scan.

Re: [PATCH] mm: mempolicy: fix automatic numa balancing for shmem

[Gregory Price]

On Thu, Jul 23, 2026 at 10:54:06AM -0400, Johannes Weiner wrote:
> > 
> > My stance is that using NUMA balancing with KVM guests is a terrible idea, and
> > that anyone that insists on using such a setup gets to suffer the consequences.
> 
> I agree. Surely we cannot leave shmem balancing broken due to
> that. It's also not clear to me how many people even enable numa
> balancing.
> 

I lean towards Johannes' stance here that consistency is better, and if
KVM has a known-poor interaction with numa balancing, maybe KVM should
simply set a default policy (when none are present) to avoid this.

That said, this is painful because MPOL_F_MOF / MPOL_F_MORON are already
a silently inherited policy on boot (added during init).  This will be
confusing for users.

This is not documented anywhere.  Maybe this change should come with a
docs addition that says the default global policy is ACTUALLY:

   MPOL_LOCAL + (MPOL_F_MOF | MPOL_F_MORON)

Instead of just MPOL_LOCAL and letting that silent implementation
detail bite people.

~Gregory

Re: [PATCH] mm: mempolicy: fix automatic numa balancing for shmem

[Johannes Weiner]

On Thu, Jul 23, 2026 at 11:58:27AM -0400, Gregory Price wrote:
> On Thu, Jul 23, 2026 at 10:54:06AM -0400, Johannes Weiner wrote:
> > > 
> > > My stance is that using NUMA balancing with KVM guests is a terrible idea, and
> > > that anyone that insists on using such a setup gets to suffer the consequences.
> > 
> > I agree. Surely we cannot leave shmem balancing broken due to
> > that. It's also not clear to me how many people even enable numa
> > balancing.
> > 
> 
> I lean towards Johannes' stance here that consistency is better, and if
> KVM has a known-poor interaction with numa balancing, maybe KVM should
> simply set a default policy (when none are present) to avoid this.
> 
> That said, this is painful because MPOL_F_MOF / MPOL_F_MORON are already
> a silently inherited policy on boot (added during init).  This will be
> confusing for users.
> 
> This is not documented anywhere.  Maybe this change should come with a
> docs addition that says the default global policy is ACTUALLY:
> 
>    MPOL_LOCAL + (MPOL_F_MOF | MPOL_F_MORON)
> 
> Instead of just MPOL_LOCAL and letting that silent implementation
> detail bite people.

Hm, where do you see "Instead of just MPOL_LOCAL"?

Documentation/filesystems/tmpfs.rst says it defaults to "default" and
points me to set_mempolicy(2). That in turn in general makes little
mention of numa balancing and doesn't mention MPOL_F_MOF/MPOL_F_MORON.
In fact these are documented as internal flags.

The numa_balancing entry in
Documentation/admin-guide/sysctl/kernel.rst however DOES say plainly
that enabling this feature will sample and migrate mapped process
memory to where it's used, and that this has a performance penalty.

It seems there is a general lack of documented interactions between
user-requested policies and the system-wide numa-balancing behavior.