damon.lists.linux.dev archive mirror
 help / color / mirror / Atom feed
* [PATCH 00/60] Private NUMA nodes
@ 2026-11-20 18:08 Gregory Price via B4 Relay
  2026-11-20 18:08 ` [PATCH 01/60] nodemask: add node-state iterators Gregory Price via B4 Relay
                   ` (60 more replies)
  0 siblings, 61 replies; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:08 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

This is v6 of the Private NUMA node series. The series falls into four
parts, each building on the one before:

   1-18  mm: introduce N_MEMORY_COMMON
  19-26  restrict folio-walking MM services to common memory
  27-44  N_MEMORY_PRIVATE, kept out of cpusets, mempolicy, accounting
         and the page allocator's ordinary paths
  45-61  private node features, memory hotplug of private memory,
         dax/kmem private mode, selftests and documentation

Patches 1-26 change no behavior on any system and can be reviewed and
merged on their own. Without private nodes, every path in patches 27-61
behaves as before.

The full stack is also available at:
https://github.com/gourryinverse/linux/tree/scratch/gourry/managed_nodes/private_nodes_v6

Motivation
==========

Some memory should be visible as a NUMA node without becoming part of
the general allocation pool: HBM, coherent GPU or accelerator memory,
or device memory a driver hands out itself.

Today such memory is either ordinary System RAM, where any allocation
can fall back onto it and every MM service manages it, or ZONE_DEVICE,
where the buddy allocator and the MM services are not available at all.

Private nodes sit in between. Their memory is managed by the buddy
allocator, but ordinary allocations never land on it, and generic MM
services leave it alone unless the node opts back in.

Model
=====

  N_MEMORY = N_MEMORY_COMMON + N_MEMORY_PRIVATE

  - N_MEMORY keeps meaning "this node has memory".
  - N_MEMORY_COMMON means ordinary allocations and kernel services
    may use it.
  - Allocation isolation comes from the zonelists: a private node is
    on no fallback zonelist and has an empty __GFP_THISNODE zonelist.
    It is reached only through its own private zonelist.
  - Services that walk existing folios check folio_is_common_memory()
    where they already skip ZONE_DEVICE.
  - A private node can opt into compaction, reclaim and userspace
    placement through per-node features. The services are written in
    terms of their own node state (N_MEMORY_RECLAIM, ...), so services
    don't need to be "aware" of anything but themselves.

The node state use / filtering model goes as such:
  1) If you're walking the normal zonelists, you're already filtered
  2) If you're walking nodelists to touch memory, filter on
     N_MEMORY_COMMON by default (replacing N_MEMORY).
  3) If you're walking structures to operate on folios / pages,
     check folio_is_common_memory() (replacing zone_device checks)
  4) If the service allows private memory interactions, use the
     N_MEMORY_<FEATURE> bit corresponding to the service.

     e.g.: reclaim checks N_MEMORY_FEATURE when walking N_MEMORY
     or when walking the private memory zonelist.

N_MEMORY_COMMON (patches 1-18)
==============================

N_MEMORY has two meanings today:
  1) this node has memory
  2) ordinary allocations and MM services can use this node's memory

The two are the same today. They stop being the same once a NUMA node
with memory isn't part of the general allocation pool.

N_MEMORY_COMMON takes the second meaning, and the N_MEMORY users that
mean it are converted. N_MEMORY continues reporting memory topology.

The rule for choosing between them:

  N_MEMORY         the node has memory: topology, per-node metadata and
                   statistics, cpuset.mems.
  N_MEMORY_COMMON  ordinary allocations or a kernel service may use the
                   node's memory: zonelists, allocation and migration
                   targets, per-node service threads, and sizing or
                   accounting meant for ordinary allocations.

Code that walks the existing zonelist needs no check. The zonelists
are built from N_MEMORY_COMMON, so they already hold only memory the
allocation may use.

Unless a system actually has a private node, N_MEMORY and
N_MEMORY_COMMON are the same set, so behavior should be unchanged.

  1-3   node-state helpers, N_MEMORY_COMMON itself, and the
        `has_common_memory` sysfs ABI.
  4     zonelists: the fallback lists hold common nodes, and a node's
        __GFP_THISNODE list is built only for a common node
  5-11  allocation and migration targets
  12-13 kswapd and kcompactd
  14-18 sizing and accounting meant for ordinary allocations

has_common_memory lists the same nodes as has_memory until memory
outside N_MEMORY_COMMON exists. It is added now so that userspace, such
as libnuma, can start choosing allocation targets from it, and so that
the meaning is fixed before anything makes the two lists differ.

Folio-walking MM services (patches 19-26)
=========================================

Patches 1-18 decide where memory comes from: zonelists, allocation
targets and per-node services follow N_MEMORY_COMMON. Some MM services
instead reach folios that already exist, wherever they are: through
page tables (KSM, khugepaged, NUMA hinting, madvise, mbind() and
migrate_pages(), move_pages()) or through physical addresses (DAMON).

These services skip ZONE_DEVICE folios with folio_is_zone_device() and
page_is_zone_device().

Patch 19 generalizes those checks into folio_is_common_memory() and
page_is_common_memory(), which reject ZONE_DEVICE folios and folios
on a node outside N_MEMORY_COMMON.

The rule: a service that walks folios checks folio_is_common_memory()
where it would otherwise check folio_is_zone_device().

Rather than add new hooks to page-table and pfn walkers, this reuses the
places that already opt ZONE_DEVICE memory out.

  19    folio_is_common_memory() and page_is_common_memory()
  20-26 ZONE_DEVICE checks in folio-walking services become
        page/folio_is_common_memory(). One component per patch.

N_MEMORY_PRIVATE (patches 27-44)
================================

Patches 27-44 add the memory that makes N_MEMORY and N_MEMORY_COMMON
differ. A private node has online System RAM that ordinary allocations
do not use and MM services leave alone, so neither an allocation that
names no node nor a node-local kernel allocation can land on it.

Most of these patches add the explicit opt-outs that the N_MEMORY ->
N_MEMORY_COMMON conversion did not cover, largely concentrated in
cpuset, mempolicy and migration.

In the page allocator, each node gets a private zonelist holding its
own zones only if it is private. MM code reaches private memory only
through that zonelist (ALLOC_ZONELIST_PRIVATE plus a nodemask).

A private request whose node becomes ineligible relaxes onto common
memory, and exhausting private memory fails the allocation (SIGBUS on
fault) instead of invoking the OOM killer (this can change in the
future with additional work).

Nothing in patches 27-44 creates private memory. Memory hotplug of
private memory, with dax/kmem as its first user, comes in patches
56-57, after the feature bits that let a private node opt into
reclaim, compaction and userspace placement.

  27-28 N_MEMORY_PRIVATE, exported as has_private_memory.
        A per-node feature mask on the pgdat.
  29-30 accounting: MemTotal/MemFree/other limits ignore private mem.
        the auto-movable online policy ignores private memory
  31-35 cpusets and mempolicy: private nodes are inherited only on
        request, never rebound onto, never spread over, and never
        migrate_pages() destinations
  36    explicit reclaim and compaction requests refuse private nodes
  37-44 page allocator:
        37-38 the reserves and watermarks, and contiguous allocation,
              skip/refuse private memory
        39-40 the private zonelist and its selection by MM callers
        41    a private request whose node becomes ineligible relaxes
              onto common memory to ensure forward progress
        42    exhausting private memory fails with SIGBUS, like an
              exhausted HugeTLB pool
        43    an alloc_flags-aware folio allocator (Brendan Jackman)
        44    a bind policy whose nodes became private falls back to
              common memory

Private node features and providers (patches 45-61)
===================================================

Patches 1-44 keep every generic MM service off private memory.

Some private memory wants a few of them back: HBM or coherent GPU
memory that userspace manages with mbind() and migrate_pages() needs
reclaim and compaction, while staying out of incidental allocation
(no fallback onto it, only explicit placement).

Patches 45-55 add three opt-in feature bits and corresponding node
states:

  N_MEMORY_COMPACTION   kcompactd, proactive and explicit compaction
  N_MEMORY_RECLAIM      kswapd, memcg and explicit reclaim
  N_MEMORY_MEMPOLICY    mbind() and explicit migration may target it

The states cover what the kernel does on its own initiative. Direct
reclaim and compaction serve an allocation within its own zonelist, so
an allocation from a private node may reclaim and compact it either way.

RECLAIM requires COMPACTION, and MEMPOLICY requires both. Userspace
control (mbind, migration) generates fragmentation, and as a result
private allocations would fail unpredictably without reclaim/compaction.

Relcaim and compaction are required by mempolicy because otherwise
order-N allocations cause unpredictable out of memory conditions due
to fragmentation.

Demotion and NUMA balancing support are deferred to a later series
but can be trivially support to allow demotions from HBM to DRAM
as a way of cheap tiering.

Every common node carries all three, so the existing N_MEMORY_COMMON
users that mean "reclaim" or "compaction" move to the new states.

Service support:

  45-46 N_MEMORY_COMPACTION, and compaction switched to it
  47-48 N_MEMORY_RECLAIM, and reclaim switched to it
  49    madvise reclaim advice only for reclaim-capable memory
  50    memcg reclaim also visits private nodes that permit reclaim
  51    N_MEMORY_MEMPOLICY
  52    explicit migration acts on N_MEMORY_MEMPOLICY memory
  53    N_MEMORY_MEMPOLICY memory counts as general-purpose.
        MemAvailable (free and file pages) and dirty limits count
	only common memory.
  54    MPOL_F_PRIVATE_NODES: mbind() flag that binds a mapping to
        one private node.
  55    migrate_pages() support for N_MEMORY_MEMPOLICY nodes

Creating a private node

  56    memory hotplug of private memory, with a feature mask
  57    dax/kmem private mode

Testing and Documentation:

  58-60 selftests: private kmem devices and services, memory policy
        bindings, migration and exhaustion, and cpuset interaction
  61    Documentation/mm/numa_private_nodes.rst

MPOL_F_PRIVATE_NODES is new uapi. Without it, private nodes in an
mbind() mask are ignored, similar to nodes outside the cpuset.

libnuma and numactl need a matching change to reach private nodes.

There is no sysfs list of the nodes that accept private bindings.
userspace can probe with MPOL_F_PRIVATE_NODES.  Otherwise - we can
discuss how service support should be discoverable to userspace.

Memory hotplug is the only source of private memory right now.

All System RAM on a node shares one configuration: hotplug with a
different one fails with -EBUSY.

Costs and known limits
======================

  - ZONELIST_PRIVATE is a third zonelist per pgdat; with NODES_SHIFT=10
    that is about 80KB per possible node, though a private zonelist
    holds at most one node's zones.

    local-only zonelists can be space-optimized if this matters.

  - Free page reporting reports free private pages like any others.

  - Tasks are OOM-scored by all of their resident memory,
    private pages included.

Changes since v5 ("Private Memory NUMA Nodes")
==============================================

  - Reordered so the N_MEMORY audit (patches 1-26) comes first and can
    be reviewed (and merged) on its own if desired.
  - Capabilities became memory features with node states, so a service
    can be written in terms of itself: reclaim walks N_MEMORY_RECLAIM,
    compaction N_MEMORY_COMPACTION.
  - The capability set is pared down to COMPACTION, RECLAIM and
    MEMPOLICY for the first run.  DEMOTION, NUMA_BALANCING, and
    compressed RAM support are deferred.
  - The private zonelist is now local-only. A fallback variant can be
    added if a use case needs one.
  - cpuset and OOM semantics are worked out:
      - cpuset.mems governs private nodes
      - private-only request inherits the parent's common nodes.
  - Private bindings are an explicit uapi flag, MPOL_F_PRIVATE_NODES,
    rather than inferred from the nodemask.
  - Exhausting private placement fails the fault with SIGBUS, as an
    exhausted HugeTLB pool does, instead of invoking the OOM killer.
    OOM support can be added later, but this keeps initial support
    reasonable simple.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
Gregory Price (Meta) (60):
      nodemask: add node-state iterators
      mm: publish and clear node memory states via helpers
      mm: introduce the common-memory node state
      mm/page_alloc: build zonelists from common memory
      mm/mempolicy: keep interleave and tmpfs mount policies on common memory
      mm/memory_hotplug: migrate offlined pages to common memory
      mm/memory-tiers: restrict demotion topology to common memory
      mm/migrate: reject move_pages() destinations outside common memory
      mm/damon: reject migration destinations outside common memory
      sched/numa: restrict NUMA migration targets to common memory
      fork: check cached stack locality only on common memory nodes
      mm/vmscan: restrict reclaim to common memory
      mm/compaction: restrict compaction to common memory
      mm: base hash distribution on common memory nodes
      mm/page_alloc: count common memory nodes for kswapd recovery
      mm/oom_kill: compare OOM constraints against common memory
      PM: hibernate: size the image from common memory
      mm/khugepaged: base the recommended min_free_kbytes on common memory
      mm: add folio_is_common_memory() and page_is_common_memory()
      mm/ksm: restrict merging to common memory
      mm/khugepaged: restrict collapse to common memory
      mm/mempolicy: don't NUMA-hint folios outside common memory
      mm/damon: restrict DAMON-native folio operations to common memory
      mm/madvise: restrict reclaim advice to common memory
      mm/mempolicy: queue only common memory for mbind() and migrate_pages()
      mm/migrate: leave pages outside common memory in move_pages()
      mm: add the private-memory node state
      mm: select memory features per node
      mm: keep private memory out of the system memory totals
      mm/memory_hotplug: keep private memory out of the auto-movable ratio
      cgroup/cpuset: inherit common memory when a nodemask is private only
      cgroup/cpuset: keep spread page cache off private nodes
      cgroup/cpuset: ignore private nodes when matching OOM victims
      mm/mempolicy: don't rebind policies onto private nodes
      mm/mempolicy: keep private nodes out of migrate_pages() destinations
      mm: refuse explicit reclaim and compaction of private nodes
      mm/page_alloc: keep private memory out of reclaim reserves
      mm/page_alloc: refuse contiguous allocation from private memory
      mm/page_alloc: add a private-node zonelist
      mm/page_alloc: let MM callers select the local private zonelist
      mm/page_alloc: relax a private request whose node becomes ineligible
      mm: fail private placement on exhaustion instead of OOM killing
      mm/page_alloc: fall back when a bind policy's nodes become private
      mm: add N_MEMORY_COMPACTION node state
      mm/compaction: restrict compaction to N_MEMORY_COMPACTION
      mm: add N_MEMORY_RECLAIM node state
      mm/vmscan: restrict reclaim to N_MEMORY_RECLAIM instead of N_MEMORY_COMMON
      mm/madvise: restrict reclaim advice to reclaim-capable memory
      mm/vmscan: reclaim private nodes for memcg
      mm: add N_MEMORY_MEMPOLICY node state
      mm: restrict explicit migration to N_MEMORY_MEMPOLICY
      mm: count memory with userspace placement as general purpose
      mm/mempolicy: add MPOL_F_PRIVATE_NODES for private node bindings
      mm/migrate: route explicit NUMA migration to private nodes
      mm/memory_hotplug: online and offline private memory nodes
      dax/kmem: add a private memory mode
      selftests/dax: test private kmem devices and services
      selftests/dax: test memory policy on private nodes
      selftests/dax: test cpuset interaction with private nodes
      Documentation/mm: describe private memory nodes

 Documentation/ABI/stable/sysfs-devices-node        |  27 +
 Documentation/ABI/testing/sysfs-bus-dax            |  19 +
 Documentation/admin-guide/cgroup-v1/cpusets.rst    |   7 +
 Documentation/admin-guide/cgroup-v2.rst            |  13 +-
 .../admin-guide/mm/numa_memory_policy.rst          |  22 +
 Documentation/mm/index.rst                         |   1 +
 Documentation/mm/numa_private_nodes.rst            | 223 ++++++++
 Documentation/mm/physical_memory.rst               |  25 +-
 drivers/base/node.c                                |   9 +-
 drivers/dax/kmem.c                                 |  55 +-
 include/linux/memory_hotplug.h                     |   2 +-
 include/linux/mmzone.h                             |   6 +-
 include/linux/nodemask.h                           |  74 ++-
 include/linux/sched.h                              |   2 +
 include/linux/swap.h                               |   4 +-
 include/uapi/linux/mempolicy.h                     |   4 +-
 kernel/cgroup/cpuset.c                             |  85 ++-
 kernel/fork.c                                      |   2 +-
 kernel/power/snapshot.c                            |  15 +-
 kernel/sched/fair.c                                |   2 +-
 mm/compaction.c                                    |  10 +-
 mm/damon/core.c                                    |   5 +-
 mm/damon/ops-common.c                              |   7 +-
 mm/damon/vaddr.c                                   |   2 +-
 mm/huge_memory.c                                   |   2 +-
 mm/internal.h                                      |  71 +++
 mm/khugepaged.c                                    |  20 +-
 mm/ksm.c                                           |   8 +-
 mm/madvise.c                                       |   6 +-
 mm/memory-tiers.c                                  |  23 +-
 mm/memory.c                                        |   5 +
 mm/memory_hotplug.c                                | 120 ++--
 mm/mempolicy.c                                     | 165 ++++--
 mm/migrate.c                                       |  15 +-
 mm/mm_init.c                                       |  20 +-
 mm/mmap.c                                          |   6 +-
 mm/nommu.c                                         |   4 +-
 mm/oom_kill.c                                      |   7 +-
 mm/page-writeback.c                                |   7 +-
 mm/page_alloc.c                                    | 199 ++++++-
 mm/page_alloc.h                                    |  22 +-
 mm/show_mem.c                                      |  49 +-
 mm/vmscan.c                                        |  58 +-
 tools/testing/selftests/dax/.gitignore             |   3 +
 tools/testing/selftests/dax/Makefile               |  10 +
 tools/testing/selftests/dax/config                 |   6 +
 tools/testing/selftests/dax/kmem_private.sh        |  94 ++++
 tools/testing/selftests/dax/kmem_private_cpuset.sh |  70 +++
 tools/testing/selftests/dax/kmem_private_policy.sh |  58 ++
 .../selftests/dax/kmem_private_policy_tool.c       | 620 +++++++++++++++++++++
 tools/testing/selftests/dax/kmem_private_rebind.sh | 301 ++++++++++
 .../selftests/dax/kmem_private_rebind_tool.c       | 495 ++++++++++++++++
 .../testing/selftests/dax/kmem_private_services.sh |  97 ++++
 tools/testing/selftests/dax/private_node_common.sh | 268 +++++++++
 tools/testing/selftests/dax/settings               |   2 +-
 55 files changed, 3249 insertions(+), 203 deletions(-)
---
base-commit: 15f114dac9a7bde1796f0e54e22726fd31cfa8a5
change-id: 20261120-private_nodes_v6-64d04596e126

Best regards,
-- 
~Gregory



^ permalink raw reply	[flat|nested] 124+ messages in thread

* [PATCH 01/60] nodemask: add node-state iterators
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
@ 2026-11-20 18:08 ` Gregory Price via B4 Relay
  2026-11-20 18:20   ` sashiko-bot
  2026-11-20 18:08 ` [PATCH 02/60] mm: publish and clear node memory states via helpers Gregory Price via B4 Relay
                   ` (59 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:08 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Add first_node_state() and next_node_state(), so callers can walk any
node state the way first_memory_node and next_memory_node walk N_MEMORY.

Add for_each_node_mask_state(), which iterates a nodemask intersected
with a node state using the bitmap intersection helpers, without a
temporary nodemask.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 include/linux/nodemask.h | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/include/linux/nodemask.h b/include/linux/nodemask.h
index 607373bd83ee..90bdcffa42ea 100644
--- a/include/linux/nodemask.h
+++ b/include/linux/nodemask.h
@@ -411,6 +411,11 @@ enum node_states {
 
 extern nodemask_t node_states[NR_NODE_STATES];
 
+/* Iterate the nodes in a mask that also belong to the specified state. */
+#define for_each_node_mask_state(node, mask, state)			\
+	for_each_and_bit(node, nodes_addr(mask),			\
+			 nodes_addr(node_states[state]), MAX_NUMNODES)
+
 #if MAX_NUMNODES > 1
 static __always_inline int node_state(int node, enum node_states state)
 {
@@ -435,6 +440,12 @@ static __always_inline int num_node_state(enum node_states state)
 #define for_each_node_state(__node, __state) \
 	for_each_node_mask((__node), node_states[__state])
 
+#define first_node_state(state)	first_node(node_states[state])
+static __always_inline unsigned int next_node_state(int nid, enum node_states state)
+{
+	return next_node(nid, node_states[state]);
+}
+
 #define first_online_node	first_node(node_states[N_ONLINE])
 #define first_memory_node	first_node(node_states[N_MEMORY])
 static __always_inline unsigned int next_online_node(int nid)
@@ -481,6 +492,12 @@ static __always_inline int num_node_state(enum node_states state)
 #define for_each_node_state(node, __state) \
 	for ( (node) = 0; (node) == 0; (node) = 1)
 
+#define first_node_state(state)	0
+static __always_inline unsigned int next_node_state(int nid, enum node_states state)
+{
+	return MAX_NUMNODES;
+}
+
 #define first_online_node	0
 #define first_memory_node	0
 #define next_online_node(nid)	(MAX_NUMNODES)

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 02/60] mm: publish and clear node memory states via helpers
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
  2026-11-20 18:08 ` [PATCH 01/60] nodemask: add node-state iterators Gregory Price via B4 Relay
@ 2026-11-20 18:08 ` Gregory Price via B4 Relay
  2026-11-20 18:20   ` sashiko-bot
  2026-11-20 18:08 ` [PATCH 03/60] mm: introduce the common-memory node state Gregory Price via B4 Relay
                   ` (58 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:08 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

N_MEMORY is set in free_area_init() and online_pages(), and cleared in
offline_pages().

Centralize those updates in node_set/clear_memory_state(), so states
that subdivide N_MEMORY can be published together with it.

Behavior is unchanged.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/internal.h       |  3 +++
 mm/memory_hotplug.c |  4 ++--
 mm/mm_init.c        |  8 ++++----
 mm/page_alloc.c     | 10 ++++++++++
 4 files changed, 19 insertions(+), 6 deletions(-)

diff --git a/mm/internal.h b/mm/internal.h
index de4593b3e3b6..c102e0f498bf 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -62,6 +62,9 @@ void lru_add_drain_cpu_zone(struct zone *zone);
 void folio_deactivate(struct folio *folio);
 void folio_mark_lazyfree(struct folio *folio);
 
+void node_set_memory_state(int nid);
+void node_clear_memory_state(int nid);
+
 /* mm/vmscan.c */
 unsigned long zone_reclaimable_pages(struct zone *zone);
 unsigned long try_to_free_pages(struct zonelist *zonelist, int order,
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index e5024f934932..e5cb770e4b09 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -1197,7 +1197,7 @@ int online_pages(unsigned long pfn, unsigned long nr_pages,
 	adjust_present_page_count(pfn_to_page(pfn), group, nr_pages);
 
 	if (node_arg.nid >= 0)
-		node_set_state(nid, N_MEMORY);
+		node_set_memory_state(nid);
 	/*
 	 * Check whether we are adding normal memory to the node for the first
 	 * time.
@@ -2165,7 +2165,7 @@ int offline_pages(unsigned long start_pfn, unsigned long nr_pages,
 	 * list. Otherwise this node would still appear in the fallback lists.
 	 */
 	if (node_arg.nid >= 0)
-		node_clear_state(node, N_MEMORY);
+		node_clear_memory_state(node);
 	if (!populated_zone(zone)) {
 		zone_pcp_reset(zone);
 		build_all_zonelists(NULL);
diff --git a/mm/mm_init.c b/mm/mm_init.c
index a9d848c92678..46fe9a274a7a 100644
--- a/mm/mm_init.c
+++ b/mm/mm_init.c
@@ -1763,11 +1763,13 @@ static void __init free_area_init_node(int nid)
 	lru_gen_init_pgdat(pgdat);
 }
 
-/* Any regular or high memory on that node? */
+/* Publish the memory states of a node that has present pages. */
 static void __init check_for_memory(pg_data_t *pgdat)
 {
 	enum zone_type zone_type;
 
+	node_set_memory_state(pgdat->node_id);
+
 	for (zone_type = 0; zone_type <= ZONE_MOVABLE - 1; zone_type++) {
 		struct zone *zone = &pgdat->node_zones[zone_type];
 		if (populated_zone(zone)) {
@@ -1924,10 +1926,8 @@ static void __init free_area_init(void)
 		 *memory-less node. The pgdat will get fully initialized by
 		 *hotadd_init_pgdat() when memory is hotplugged into this node.
 		 */
-		if (pgdat->node_present_pages) {
-			node_set_state(nid, N_MEMORY);
+		if (pgdat->node_present_pages)
 			check_for_memory(pgdat);
-		}
 	}
 
 	calc_nr_kernel_pages();
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index aa1ed599c7c6..88d507f93b1a 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -208,6 +208,16 @@ nodemask_t node_states[NR_NODE_STATES] __read_mostly = {
 };
 EXPORT_SYMBOL(node_states);
 
+void __meminit node_set_memory_state(int nid)
+{
+	node_set_state(nid, N_MEMORY);
+}
+
+void __meminit node_clear_memory_state(int nid)
+{
+	node_clear_state(nid, N_MEMORY);
+}
+
 gfp_t gfp_allowed_mask __read_mostly = GFP_BOOT_MASK;
 
 #ifdef CONFIG_HUGETLB_PAGE_SIZE_VARIABLE

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 03/60] mm: introduce the common-memory node state
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
  2026-11-20 18:08 ` [PATCH 01/60] nodemask: add node-state iterators Gregory Price via B4 Relay
  2026-11-20 18:08 ` [PATCH 02/60] mm: publish and clear node memory states via helpers Gregory Price via B4 Relay
@ 2026-11-20 18:08 ` Gregory Price via B4 Relay
  2026-11-20 18:22   ` sashiko-bot
  2026-11-20 18:08 ` [PATCH 04/60] mm/page_alloc: build zonelists from common memory Gregory Price via B4 Relay
                   ` (57 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:08 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

N_MEMORY is overloaded with two meanings:
  1) a node has memory.
  2) a node's memory is reachable via ordinary allocation or
     an arbitrary MM service may use that memory.

Allow N_MEMORY to mean #1, and introduce N_MEMORY_COMMON to mean #2.

N_MEMORY_COMMON nodes contain "common memory" capable of being managed
arbitrarily by the kernel MM based solely on zone membership. Following
commits convert relevant N_MEMORY users to N_MEMORY_COMMON.

Expose the state via `has_common_memory` so userspace can identify
the memory available to ordinary allocations and common MM services.

Add has_memory to the docs while we're here.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 Documentation/ABI/stable/sysfs-devices-node | 17 +++++++++++++++++
 Documentation/mm/physical_memory.rst        |  8 +++++++-
 drivers/base/node.c                         |  2 ++
 include/linux/nodemask.h                    |  9 ++++++++-
 mm/page_alloc.c                             |  3 +++
 5 files changed, 37 insertions(+), 2 deletions(-)

diff --git a/Documentation/ABI/stable/sysfs-devices-node b/Documentation/ABI/stable/sysfs-devices-node
index 2d0e023f22a7..0a2cf3088233 100644
--- a/Documentation/ABI/stable/sysfs-devices-node
+++ b/Documentation/ABI/stable/sysfs-devices-node
@@ -10,6 +10,15 @@ Contact:	Linux Memory Management list <linux-mm@kvack.org>
 Description:
 		Nodes that are online.
 
+What:		/sys/devices/system/node/has_memory
+Date:		December 2012
+Contact:	Linux Memory Management list <linux-mm@kvack.org>
+Description:
+		Nodes that have memory.
+
+		This memory may or may not be reachable via standard
+		allocations, see has_common_memory for details.
+
 What:		/sys/devices/system/node/has_normal_memory
 Date:		October 2002
 Contact:	Linux Memory Management list <linux-mm@kvack.org>
@@ -29,6 +38,14 @@ Description:
 		Nodes that have regular or high memory.
 		Depends on CONFIG_HIGHMEM.
 
+What:		/sys/devices/system/node/has_common_memory
+Date:		November 2026
+Contact:	Linux Memory Management list <linux-mm@kvack.org>
+Description:
+		Nodes whose memory ordinary allocations may use without
+		requesting the node explicitly, and that kernel memory
+		management services manage.
+
 What:		/sys/devices/system/node/nodeX
 Date:		October 2002
 Contact:	Linux Memory Management list <linux-mm@kvack.org>
diff --git a/Documentation/mm/physical_memory.rst b/Documentation/mm/physical_memory.rst
index 2e67e8b23a99..86a7d6080535 100644
--- a/Documentation/mm/physical_memory.rst
+++ b/Documentation/mm/physical_memory.rst
@@ -168,7 +168,12 @@ nodes with particular properties as defined by ``enum node_states``:
   The node has regular or high memory. When ``CONFIG_HIGHMEM`` is disabled
   aliased to ``N_NORMAL_MEMORY``.
 ``N_MEMORY``
-  The node has memory(regular, high, movable)
+  The node has online memory (regular, high, movable).  This is a topology and
+  accounting state. Presence in this mask does not mean ordinary allocations
+  can use the node's memory. See ``N_MEMORY_COMMON``.
+``N_MEMORY_COMMON``
+  Ordinary allocations and memory management services may use the node's
+  memory.
 ``N_CPU``
   The node has one or more CPUs
 ``N_GENERIC_INITIATOR``
@@ -184,6 +189,7 @@ For example, for node 2 with normal memory and CPUs, bit 2 will be set in ::
   node_states[N_NORMAL_MEMORY]
   node_states[N_HIGH_MEMORY]
   node_states[N_MEMORY]
+  node_states[N_MEMORY_COMMON]
   node_states[N_CPU]
 
 For various operations possible with nodemasks please refer to
diff --git a/drivers/base/node.c b/drivers/base/node.c
index dcdc8626627a..cc6696d1390f 100644
--- a/drivers/base/node.c
+++ b/drivers/base/node.c
@@ -965,6 +965,7 @@ static struct node_attr node_state_attr[] = {
 	[N_HIGH_MEMORY] = _NODE_ATTR(has_high_memory, N_HIGH_MEMORY),
 #endif
 	[N_MEMORY] = _NODE_ATTR(has_memory, N_MEMORY),
+	[N_MEMORY_COMMON] = _NODE_ATTR(has_common_memory, N_MEMORY_COMMON),
 	[N_CPU] = _NODE_ATTR(has_cpu, N_CPU),
 	[N_GENERIC_INITIATOR] = _NODE_ATTR(has_generic_initiator,
 					   N_GENERIC_INITIATOR),
@@ -978,6 +979,7 @@ static struct attribute *node_state_attrs[] = {
 	&node_state_attr[N_HIGH_MEMORY].attr.attr,
 #endif
 	&node_state_attr[N_MEMORY].attr.attr,
+	&node_state_attr[N_MEMORY_COMMON].attr.attr,
 	&node_state_attr[N_CPU].attr.attr,
 	&node_state_attr[N_GENERIC_INITIATOR].attr.attr,
 	NULL
diff --git a/include/linux/nodemask.h b/include/linux/nodemask.h
index 90bdcffa42ea..c3b984105bd1 100644
--- a/include/linux/nodemask.h
+++ b/include/linux/nodemask.h
@@ -398,7 +398,14 @@ enum node_states {
 #else
 	N_HIGH_MEMORY = N_NORMAL_MEMORY,
 #endif
-	N_MEMORY,		/* The node has memory(regular, high, movable) */
+	/*
+	 * N_MEMORY indicates online memory exists on the node - it is used
+	 * to manage per-node meta-data and topology decisions.
+	 * N_MEMORY_COMMON indicates whether the node can satisfy arbitrary
+	 * allocations and whether a service should manage that node's folios.
+	 */
+	N_MEMORY,		/* the node has memory */
+	N_MEMORY_COMMON,	/* common pool (fallback zonelist) */
 	N_CPU,		/* The node has one or more cpus */
 	N_GENERIC_INITIATOR,	/* The node has one or more Generic Initiators */
 	NR_NODE_STATES
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 88d507f93b1a..ddb04b72a160 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -203,6 +203,7 @@ nodemask_t node_states[NR_NODE_STATES] __read_mostly = {
 	[N_HIGH_MEMORY] = { { [0] = 1UL } },
 #endif
 	[N_MEMORY] = { { [0] = 1UL } },
+	[N_MEMORY_COMMON] = { { [0] = 1UL } },
 	[N_CPU] = { { [0] = 1UL } },
 #endif	/* NUMA */
 };
@@ -211,10 +212,12 @@ EXPORT_SYMBOL(node_states);
 void __meminit node_set_memory_state(int nid)
 {
 	node_set_state(nid, N_MEMORY);
+	node_set_state(nid, N_MEMORY_COMMON);
 }
 
 void __meminit node_clear_memory_state(int nid)
 {
+	node_clear_state(nid, N_MEMORY_COMMON);
 	node_clear_state(nid, N_MEMORY);
 }
 

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 04/60] mm/page_alloc: build zonelists from common memory
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (2 preceding siblings ...)
  2026-11-20 18:08 ` [PATCH 03/60] mm: introduce the common-memory node state Gregory Price via B4 Relay
@ 2026-11-20 18:08 ` Gregory Price via B4 Relay
  2026-11-20 18:27   ` sashiko-bot
  2026-11-20 18:08 ` [PATCH 05/60] mm/mempolicy: keep interleave and tmpfs mount policies on " Gregory Price via B4 Relay
                   ` (56 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:08 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Zonelists decide which nodes an allocation can reach.

The fallback lists hold every node in N_MEMORY, and a node's
__GFP_THISNODE list holds its own zones whatever the node's state.

Ordinary allocations should reach only common (general purpose) memory.

Build the fallback zonelists from N_MEMORY_COMMON, and __GFP_THISNODE
zonelists only for a common node.

The build_local_zonelist() refactor lets us reuse the code to build
a local-only zonelist for nodes outside the N_MEMORY_COMMON set later.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/page_alloc.c | 23 ++++++++++++-----------
 1 file changed, 12 insertions(+), 11 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index ddb04b72a160..0448d04e048a 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -5947,17 +5947,14 @@ int find_next_best_node_in(int node, nodemask_t *used_node_mask,
 }
 
 
-/*
- * Build __GFP_THISNODE zonelists
- */
-static void build_thisnode_zonelists(pg_data_t *pgdat)
+/* Build a local-only zonelist when this node is eligible. */
+static void build_local_zonelist(pg_data_t *pgdat, int zlidx, bool eligible)
 {
-	struct zoneref *zonerefs;
-	int nr_zones;
+	struct zoneref *zonerefs =
+		pgdat->node_zonelists[zlidx]._zonerefs;
 
-	zonerefs = pgdat->node_zonelists[ZONELIST_NOFALLBACK]._zonerefs;
-	nr_zones = build_zonerefs_node(pgdat, zonerefs);
-	zonerefs += nr_zones;
+	if (eligible)
+		zonerefs += build_zonerefs_node(pgdat, zonerefs);
 	zonerefs->zone = NULL;
 	zonerefs->zone_idx = 0;
 }
@@ -6001,8 +5998,12 @@ static void build_node_zonelist(pg_data_t *pgdat, const nodemask_t *candidates,
 
 static void build_zonelists(pg_data_t *pgdat)
 {
-	build_node_zonelist(pgdat, &node_states[N_MEMORY], ZONELIST_FALLBACK);
-	build_thisnode_zonelists(pgdat);
+	int nid = pgdat->node_id;
+
+	build_node_zonelist(pgdat, &node_states[N_MEMORY_COMMON],
+			    ZONELIST_FALLBACK);
+	build_local_zonelist(pgdat, ZONELIST_NOFALLBACK,
+			     node_state(nid, N_MEMORY_COMMON));
 }
 
 #ifdef CONFIG_HAVE_MEMORYLESS_NODES

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 05/60] mm/mempolicy: keep interleave and tmpfs mount policies on common memory
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (3 preceding siblings ...)
  2026-11-20 18:08 ` [PATCH 04/60] mm/page_alloc: build zonelists from common memory Gregory Price via B4 Relay
@ 2026-11-20 18:08 ` Gregory Price via B4 Relay
  2026-11-20 18:22   ` sashiko-bot
  2026-11-20 18:08 ` [PATCH 06/60] mm/memory_hotplug: migrate offlined pages to " Gregory Price via B4 Relay
                   ` (55 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:08 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Use N_MEMORY_COMMON for boot mempolicy calculations and their per-node
sysfs entries (weighted interleave weights).

Restrict the node set mpol_set_nodemask() applies to set_mempolicy()
and mbind() masks to N_MEMORY_COMMON nodes.

A tmpfs mpol= nodelist must name only N_MEMORY_COMMON nodes, and an
interleave policy without a nodelist defaults to them.

Later when an N_MEMORY node is omitted from the N_MEMORY_COMMON set,
they will be stripped from mempolicy nodemasks as they're not eligible
(similar to memoryless nodes today).

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/mempolicy.c | 30 +++++++++++++++---------------
 1 file changed, 15 insertions(+), 15 deletions(-)

diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 6f800fb961ae..806577e1d48b 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -213,10 +213,10 @@ static void reduce_interleave_weights(unsigned int *bw, u8 *new_iw)
 	int nid;
 
 	/*
-	 * N_MEMORY can change while wi_state_lock is held. Remember the
-	 * nodes contributing to sum_bw so all passes use the same set.
+	 * N_MEMORY_COMMON can change while wi_state_lock is held. Remember
+	 * the nodes contributing to sum_bw so all passes use the same set.
 	 */
-	for_each_node_state(nid, N_MEMORY) {
+	for_each_node_state(nid, N_MEMORY_COMMON) {
 		node_set(nid, nodes);
 		sum_bw += bw[nid];
 	}
@@ -439,9 +439,9 @@ static int mpol_set_nodemask(struct mempolicy *pol,
 	if (!pol || pol->mode == MPOL_LOCAL)
 		return 0;
 
-	/* Check N_MEMORY */
+	/* Check N_MEMORY_COMMON */
 	nodes_and(nsc->mask1,
-		  cpuset_current_mems_allowed, node_states[N_MEMORY]);
+		  cpuset_current_mems_allowed, node_states[N_MEMORY_COMMON]);
 
 	VM_BUG_ON(!nodes);
 
@@ -2104,7 +2104,7 @@ bool apply_policy_zone(struct mempolicy *policy, enum zone_type zone)
 	 * if policy->nodes has movable memory only,
 	 * we apply policy when gfp_zone(gfp) = ZONE_MOVABLE only.
 	 *
-	 * policy->nodes is intersect with node_states[N_MEMORY].
+	 * policy->nodes is intersect with node_states[N_MEMORY_COMMON].
 	 * so if the following test fails, it implies
 	 * policy->nodes has movable memory only.
 	 */
@@ -3398,11 +3398,11 @@ void __init numa_policy_init(void)
 
 	/*
 	 * Set interleaving policy for system init. Interleaving is only
-	 * enabled across suitably sized nodes (default is >= 16MB), or
-	 * fall back to the largest node if they're all smaller.
+	 * enabled across suitably sized common memory nodes (default is
+	 * >= 16MB), or fall back to the largest node if they're all smaller.
 	 */
 	nodes_clear(interleave_nodes);
-	for_each_node_state(nid, N_MEMORY) {
+	for_each_node_state(nid, N_MEMORY_COMMON) {
 		unsigned long total_pages = node_present_pages(nid);
 
 		/* Preserve the largest node */
@@ -3474,7 +3474,7 @@ int mpol_parse_str(char *str, struct mempolicy **mpol)
 		*nodelist++ = '\0';
 		if (nodelist_parse(nodelist, nodes))
 			goto out;
-		if (!nodes_subset(nodes, node_states[N_MEMORY]))
+		if (!nodes_subset(nodes, node_states[N_MEMORY_COMMON]))
 			goto out;
 	} else
 		nodes_clear(nodes);
@@ -3506,7 +3506,7 @@ int mpol_parse_str(char *str, struct mempolicy **mpol)
 		 * Default to online nodes with memory if no nodelist
 		 */
 		if (!nodelist)
-			nodes = node_states[N_MEMORY];
+			nodes = node_states[N_MEMORY_COMMON];
 		break;
 	case MPOL_LOCAL:
 		/*
@@ -3901,6 +3901,9 @@ static int wi_node_notifier(struct notifier_block *nb,
 
 	switch (action) {
 	case NODE_ADDED_FIRST_MEMORY:
+		/* Weighted interleave spans only common memory. */
+		if (!node_state(nid, N_MEMORY_COMMON))
+			break;
 		err = sysfs_wi_node_add(nid);
 		if (err)
 			pr_err("failed to add sysfs for node%d during hotplug: %d\n",
@@ -3932,10 +3935,7 @@ static int __init add_weighted_interleave_group(struct kobject *mempolicy_kobj)
 	if (err)
 		goto err_put_kobj;
 
-	for_each_online_node(nid) {
-		if (!node_state(nid, N_MEMORY))
-			continue;
-
+	for_each_node_state(nid, N_MEMORY_COMMON) {
 		err = sysfs_wi_node_add(nid);
 		if (err) {
 			pr_err("failed to add sysfs for node%d during init: %d\n",

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 06/60] mm/memory_hotplug: migrate offlined pages to common memory
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (4 preceding siblings ...)
  2026-11-20 18:08 ` [PATCH 05/60] mm/mempolicy: keep interleave and tmpfs mount policies on " Gregory Price via B4 Relay
@ 2026-11-20 18:08 ` Gregory Price via B4 Relay
  2026-11-20 18:20   ` sashiko-bot
  2026-11-20 18:08 ` [PATCH 07/60] mm/memory-tiers: restrict demotion topology " Gregory Price via B4 Relay
                   ` (54 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:08 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

do_migrate_range() / alloc_migration_target() is only capable of
allocating memory on N_MEMORY_COMMON nodes. The destination nodemask
presently starts with N_MEMORY and removes the source-node.

In the current code, if the destination mask becomes empty, the source
node is restored to allow migration during partial offlining.

Consider this configuration:
  Node0 = N_MEMORY_COMMON
  Node1 = N_MEMORY, not N_MEMORY_COMMON

Node 1 keeps the destination mask non-empty, despite being unreachable.
This leaves migration with no usable destination because the source
node is not an eligible allocation target.

Restrict the hot-unplug migration target-list to N_MEMORY_COMMON so it
reflects the nodes the allocator can reach.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/memory_hotplug.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index e5cb770e4b09..1d3d1792074c 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -1926,7 +1926,7 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn)
 		folio_put(folio);
 	}
 	if (!list_empty(&source)) {
-		nodemask_t nmask = node_states[N_MEMORY];
+		nodemask_t nmask = node_states[N_MEMORY_COMMON];
 		struct migration_target_control mtc = {
 			.nmask = &nmask,
 			.gfp_mask = GFP_KERNEL | __GFP_MOVABLE | __GFP_RETRY_MAYFAIL,
@@ -1941,9 +1941,9 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn)
 		mtc.nid = folio_nid(list_first_entry(&source, struct folio, lru));
 
 		/*
-		 * try to allocate from a different node but reuse this node
-		 * if there are no other online nodes to be used (e.g. we are
-		 * offlining a part of the only existing node)
+		 * try to allocate from a different common node but reuse this
+		 * node if there are no other common nodes to be used (e.g. we
+		 * are offlining a part of the only common node)
 		 */
 		node_clear(mtc.nid, nmask);
 		if (nodes_empty(nmask))

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 07/60] mm/memory-tiers: restrict demotion topology to common memory
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (5 preceding siblings ...)
  2026-11-20 18:08 ` [PATCH 06/60] mm/memory_hotplug: migrate offlined pages to " Gregory Price via B4 Relay
@ 2026-11-20 18:08 ` Gregory Price via B4 Relay
  2026-11-20 18:21   ` sashiko-bot
  2026-11-20 18:08 ` [PATCH 08/60] mm/migrate: reject move_pages() destinations outside " Gregory Price via B4 Relay
                   ` (53 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:08 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Memory outside N_MEMORY_COMMON is outside the ordinary allocation pool
and cannot be a generic demotion source or destination.

Build demotion paths and the default DRAM performance reference from
N_MEMORY_COMMON rather than every node that has memory, and do not
assign other nodes to a memory tier through the generic hotplug path.

Removing a node's memory already resets its demotion targets, so a node
that unplugs and returns outside N_MEMORY_COMMON keeps none.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/memory-tiers.c | 23 ++++++++++++-----------
 1 file changed, 12 insertions(+), 11 deletions(-)

diff --git a/mm/memory-tiers.c b/mm/memory-tiers.c
index 7fc56a8c6651..97d5ecefd89f 100644
--- a/mm/memory-tiers.c
+++ b/mm/memory-tiers.c
@@ -370,7 +370,8 @@ int next_demotion_node(int node, const nodemask_t *allowed_mask)
 	 * closest demotion target.
 	 */
 	nodes_complement(mask, *allowed_mask);
-	return find_next_best_node_in(node, &mask, &node_states[N_MEMORY]);
+	return find_next_best_node_in(node, &mask,
+				      &node_states[N_MEMORY_COMMON]);
 }
 
 static void disable_all_demotion_targets(void)
@@ -405,7 +406,7 @@ static void dump_demotion_targets(void)
 {
 	int node;
 
-	for_each_node_state(node, N_MEMORY) {
+	for_each_node_state(node, N_MEMORY_COMMON) {
 		struct memory_tier *memtier = __node_get_memory_tier(node);
 		nodemask_t preferred = node_demotion[node].preferred;
 
@@ -441,7 +442,7 @@ static void establish_demotion_targets(void)
 
 	disable_all_demotion_targets();
 
-	for_each_node_state(node, N_MEMORY) {
+	for_each_node_state(node, N_MEMORY_COMMON) {
 		best_distance = -1;
 		nd = &node_demotion[node];
 
@@ -459,7 +460,7 @@ static void establish_demotion_targets(void)
 		 * nodelist to skip list so that we find the best node from the
 		 * memtier nodelist.
 		 */
-		nodes_andnot(tier_nodes, node_states[N_MEMORY], tier_nodes);
+		nodes_andnot(tier_nodes, node_states[N_MEMORY_COMMON], tier_nodes);
 
 		/*
 		 * Find all the nodes in the memory tier node list of same best distance.
@@ -468,7 +469,7 @@ static void establish_demotion_targets(void)
 		 */
 		do {
 			target = find_next_best_node_in(node, &tier_nodes,
-							&node_states[N_MEMORY]);
+							&node_states[N_MEMORY_COMMON]);
 			if (target == NUMA_NO_NODE)
 				break;
 
@@ -507,7 +508,7 @@ static void establish_demotion_targets(void)
 	 * allocation to a set of nodes that is closer the above selected
 	 * preferred node.
 	 */
-	lower_tier = node_states[N_MEMORY];
+	lower_tier = node_states[N_MEMORY_COMMON];
 	list_for_each_entry(memtier, &memory_tiers, list) {
 		/*
 		 * Keep removing current tier from lower_tier nodes,
@@ -554,7 +555,7 @@ static struct memory_tier *set_node_memory_tier(int node)
 
 	lockdep_assert_held_once(&memory_tier_lock);
 
-	if (!node_state(node, N_MEMORY))
+	if (!node_state(node, N_MEMORY_COMMON))
 		return ERR_PTR(-EINVAL);
 
 	mt_calc_adistance(node, &adist);
@@ -706,8 +707,8 @@ static int __init memory_tier_late_init(void)
 	get_online_mems();
 	guard(mutex)(&memory_tier_lock);
 
-	/* Assign each uninitialized N_MEMORY node to a memory tier. */
-	for_each_node_state(nid, N_MEMORY) {
+	/* Assign each uninitialized common memory node to a memory tier. */
+	for_each_node_state(nid, N_MEMORY_COMMON) {
 		/*
 		 * Some device drivers may have initialized
 		 * memory tiers, potentially bringing memory nodes
@@ -929,8 +930,8 @@ static int __init memory_tier_init(void)
 	if (IS_ERR(default_dram_type))
 		panic("%s() failed to allocate default DRAM tier\n", __func__);
 
-	/* Record nodes with memory and CPU to set default DRAM performance. */
-	nodes_and(default_dram_nodes, node_states[N_MEMORY],
+	/* Record common-memory nodes with CPUs for default DRAM performance. */
+	nodes_and(default_dram_nodes, node_states[N_MEMORY_COMMON],
 		  node_states[N_CPU]);
 
 	hotplug_node_notifier(memtier_hotplug_callback, MEMTIER_HOTPLUG_PRI);

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 08/60] mm/migrate: reject move_pages() destinations outside common memory
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (6 preceding siblings ...)
  2026-11-20 18:08 ` [PATCH 07/60] mm/memory-tiers: restrict demotion topology " Gregory Price via B4 Relay
@ 2026-11-20 18:08 ` Gregory Price via B4 Relay
  2026-11-20 18:19   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 09/60] mm/damon: reject migration " Gregory Price via B4 Relay
                   ` (52 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:08 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

move_pages(2) places user memory on a node the caller picks. As with
memory policies, the destination should be general purpose memory.

Reject a destination outside N_MEMORY_COMMON with -ENODEV, exactly
the same as a node without memory is rejected today.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/migrate.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/migrate.c b/mm/migrate.c
index 65869b8acec5..208788b130fa 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -2411,7 +2411,7 @@ static int do_pages_move(struct mm_struct *mm, nodemask_t task_nodes,
 		err = -ENODEV;
 		if (node < 0 || node >= MAX_NUMNODES)
 			goto out_flush;
-		if (!node_state(node, N_MEMORY))
+		if (!node_state(node, N_MEMORY_COMMON))
 			goto out_flush;
 
 		err = -EACCES;

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 09/60] mm/damon: reject migration destinations outside common memory
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (7 preceding siblings ...)
  2026-11-20 18:08 ` [PATCH 08/60] mm/migrate: reject move_pages() destinations outside " Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:18   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 10/60] sched/numa: restrict NUMA migration targets to " Gregory Price via B4 Relay
                   ` (51 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

DAMOS migration actions move folios to a node the user picks. As with
move_pages(2), the destination should be general purpose memory.

Reject a destination outside N_MEMORY_COMMON, exactly the same as a
node without memory is rejected today.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/damon/ops-common.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/damon/ops-common.c b/mm/damon/ops-common.c
index 2216a0023154..15cabc79613b 100644
--- a/mm/damon/ops-common.c
+++ b/mm/damon/ops-common.c
@@ -498,7 +498,7 @@ unsigned long damon_migrate_pages(struct list_head *folio_list, int target_nid)
 		return nr_migrated;
 
 	if (target_nid < 0 || target_nid >= MAX_NUMNODES ||
-			!node_state(target_nid, N_MEMORY)) {
+			!node_state(target_nid, N_MEMORY_COMMON)) {
 		damon_putback_folio_list(folio_list);
 		return nr_migrated;
 	}

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 10/60] sched/numa: restrict NUMA migration targets to common memory
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (8 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 09/60] mm/damon: reject migration " Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:21   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 11/60] fork: check cached stack locality only on common memory nodes Gregory Price via B4 Relay
                   ` (50 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

should_numa_migrate_memory() refuses to migrate folios towards a
memoryless node, since the migration can only fail; see
commit 3fb43636876d ("sched/numa, mm: do not try to migrate memory to
memoryless nodes").

A node outside N_MEMORY_COMMON can have CPUs but has an empty
__GFP_THISNODE zonelist, so a NUMA hinting fault from one of its CPUs
would isolate the folio, fail to allocate the destination and put the
folio back.

Check N_MEMORY_COMMON instead.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 kernel/sched/fair.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 0c8af1cac2d5..18db9bd25db2 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -2876,7 +2876,7 @@ bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio,
 	/*
 	 * Cannot migrate to memoryless nodes.
 	 */
-	if (!node_state(dst_nid, N_MEMORY))
+	if (!node_state(dst_nid, N_MEMORY_COMMON))
 		return false;
 
 	/*

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 11/60] fork: check cached stack locality only on common memory nodes
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (9 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 10/60] sched/numa: restrict NUMA migration targets to " Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:20   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 12/60] mm/vmscan: restrict reclaim to common memory Gregory Price via B4 Relay
                   ` (49 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

try_release_thread_stack_to_cache() caches a freed stack only if all of
its pages are on the local node, unless the local node has no memory.

A CPU on a node outside N_MEMORY_COMMON gets its stacks from other
nodes, as a CPU on a memoryless node does, so the locality check would
keep it from ever caching a stack.

Skip the check when the local node has no common memory.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 kernel/fork.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/fork.c b/kernel/fork.c
index 5efceb8d58a7..28c58f2765f7 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -254,7 +254,7 @@ static bool try_release_thread_stack_to_cache(struct vm_struct *vm_area)
 	 */
 	scoped_guard(preempt) {
 		nid = numa_node_id();
-		if (node_state(nid, N_MEMORY)) {
+		if (node_state(nid, N_MEMORY_COMMON)) {
 			for (i = 0; i < vm_area->nr_pages; i++) {
 				struct page *page = vm_area->pages[i];
 				if (page_to_nid(page) != nid)

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 12/60] mm/vmscan: restrict reclaim to common memory
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (10 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 11/60] fork: check cached stack locality only on common memory nodes Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:25   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 13/60] mm/compaction: restrict compaction " Gregory Price via B4 Relay
                   ` (48 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Reclaim serves the ordinary allocation domain.

Restrict it to N_MEMORY_COMMON:
  - start kswapd only on common nodes, at boot or hotplug.
  - walk common nodes when MGLRU's debugfs interface iterates or
    validates nodes, using first_node_state() and next_node_state() in
    place of first_memory_node and next_memory_node.

Direct reclaim serves an allocation from its own zonelist, so it is left
alone.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/memory_hotplug.c |  3 ++-
 mm/vmscan.c         | 12 ++++++------
 2 files changed, 8 insertions(+), 7 deletions(-)

diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 1d3d1792074c..a599fcad65d3 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -1222,7 +1222,8 @@ int online_pages(unsigned long pfn, unsigned long nr_pages,
 	/* reinitialise watermarks and update pcp limits */
 	init_per_zone_wmark_min();
 
-	kswapd_run(nid);
+	if (node_state(nid, N_MEMORY_COMMON))
+		kswapd_run(nid);
 	kcompactd_run(nid);
 
 	if (node_arg.nid >= 0)
diff --git a/mm/vmscan.c b/mm/vmscan.c
index bc7579d47f9b..5b53e0391d80 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -5729,7 +5729,7 @@ static void *lru_gen_seq_start(struct seq_file *m, loff_t *pos)
 	do {
 		int nid;
 
-		for_each_node_state(nid, N_MEMORY) {
+		for_each_node_state(nid, N_MEMORY_COMMON) {
 			if (!nr_to_skip--)
 				return mem_cgroup_lruvec(memcg, NODE_DATA(nid));
 		}
@@ -5754,13 +5754,13 @@ static void *lru_gen_seq_next(struct seq_file *m, void *v, loff_t *pos)
 
 	++*pos;
 
-	nid = next_memory_node(nid);
+	nid = next_node_state(nid, N_MEMORY_COMMON);
 	if (nid == MAX_NUMNODES) {
 		memcg = mem_cgroup_iter(NULL, memcg, NULL);
 		if (!memcg)
 			return NULL;
 
-		nid = first_memory_node;
+		nid = first_node_state(N_MEMORY_COMMON);
 	}
 
 	return mem_cgroup_lruvec(memcg, NODE_DATA(nid));
@@ -5832,7 +5832,7 @@ static int lru_gen_seq_show(struct seq_file *m, void *v)
 	DEFINE_MAX_SEQ(lruvec);
 	DEFINE_MIN_SEQ(lruvec);
 
-	if (nid == first_memory_node) {
+	if (nid == first_node_state(N_MEMORY_COMMON)) {
 		const char *path = memcg ? m->private : "";
 
 #ifdef CONFIG_MEMCG
@@ -5935,7 +5935,7 @@ static int run_cmd(char cmd, u64 memcg_id, int nid, unsigned long seq,
 	int err = -EINVAL;
 	struct mem_cgroup *memcg = NULL;
 
-	if (nid < 0 || nid >= MAX_NUMNODES || !node_state(nid, N_MEMORY))
+	if (nid < 0 || nid >= MAX_NUMNODES || !node_state(nid, N_MEMORY_COMMON))
 		return -EINVAL;
 
 	if (!mem_cgroup_disabled()) {
@@ -7907,7 +7907,7 @@ static int __init kswapd_init(void)
 {
 	int nid;
 
-	for_each_node_state(nid, N_MEMORY)
+	for_each_node_state(nid, N_MEMORY_COMMON)
  		kswapd_run(nid);
 	register_sysctl_init("vm", vmscan_sysctl_table);
 	return 0;

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 13/60] mm/compaction: restrict compaction to common memory
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (11 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 12/60] mm/vmscan: restrict reclaim to common memory Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:20   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 14/60] mm: base hash distribution on common memory nodes Gregory Price via B4 Relay
                   ` (47 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Compaction serves the ordinary allocation domain.

Restrict it to N_MEMORY_COMMON:
  - start kcompactd only on common nodes, at boot and hotplug.
  - compact only common nodes for /proc/sys/vm/compact_memory,
    and trigger proactive compaction only on common nodes.

compact_memory and the proactiveness trigger walked every online node
before. Memoryless nodes have nothing to compact and no kcompactd, so
leaving them out changes nothing.

Direct compaction serves an allocation from its own zonelist, so it is
left alone.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/compaction.c     | 6 +++---
 mm/memory_hotplug.c | 3 ++-
 2 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/mm/compaction.c b/mm/compaction.c
index 426563acf7ac..879f372a891e 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -2973,7 +2973,7 @@ static int compact_nodes(void)
 	/* Flush pending updates to the LRU lists */
 	lru_add_drain_all();
 
-	for_each_online_node(nid) {
+	for_each_node_state(nid, N_MEMORY_COMMON) {
 		ret = compact_node(NODE_DATA(nid), false);
 		if (ret)
 			return ret;
@@ -2992,7 +2992,7 @@ static int compaction_proactiveness_sysctl_handler(const struct ctl_table *table
 		return rc;
 
 	if (write && sysctl_compaction_proactiveness) {
-		for_each_online_node(nid) {
+		for_each_node_state(nid, N_MEMORY_COMMON) {
 			pg_data_t *pgdat = NODE_DATA(nid);
 
 			if (pgdat->proactive_compact_trigger)
@@ -3366,7 +3366,7 @@ static int __init kcompactd_init(void)
 {
 	int nid;
 
-	for_each_node_state(nid, N_MEMORY)
+	for_each_node_state(nid, N_MEMORY_COMMON)
 		kcompactd_run(nid);
 	register_sysctl_init("vm", vm_compaction);
 	return 0;
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index a599fcad65d3..6b6873cfbc5a 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -1224,7 +1224,8 @@ int online_pages(unsigned long pfn, unsigned long nr_pages,
 
 	if (node_state(nid, N_MEMORY_COMMON))
 		kswapd_run(nid);
-	kcompactd_run(nid);
+	if (node_state(nid, N_MEMORY_COMMON))
+		kcompactd_run(nid);
 
 	if (node_arg.nid >= 0)
 		/* First memory added successfully. Notify consumers. */

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 14/60] mm: base hash distribution on common memory nodes
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (12 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 13/60] mm/compaction: restrict compaction " Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:19   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 15/60] mm/page_alloc: count common memory nodes for kswapd recovery Gregory Price via B4 Relay
                   ` (46 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

hashdist is useful when large kernel hash tables can be allocated across
multiple NUMA nodes. The tables come from ordinary allocations, which
only N_MEMORY_COMMON nodes serve.

Count N_MEMORY_COMMON nodes when applying the single-node optimization.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/mm_init.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/mm_init.c b/mm/mm_init.c
index 46fe9a274a7a..f21b3dbcc92a 100644
--- a/mm/mm_init.c
+++ b/mm/mm_init.c
@@ -685,7 +685,7 @@ __setup("hashdist=", set_hashdist);
 
 static inline void fixup_hashdist(void)
 {
-	if (num_node_state(N_MEMORY) == 1)
+	if (num_node_state(N_MEMORY_COMMON) == 1)
 		hashdist = false;
 }
 #else

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 15/60] mm/page_alloc: count common memory nodes for kswapd recovery
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (13 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 14/60] mm: base hash distribution on common memory nodes Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:20   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 16/60] mm/oom_kill: compare OOM constraints against common memory Gregory Price via B4 Relay
                   ` (45 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

When a zone recovers above its high watermark, free_frozen_page_commit()
clears a hopeless kswapd state if more than one memory node exists,
since allocations could then have fallen back to another node without
triggering direct reclaim here.

Only common memory provide fallbacks, so count N_MEMORY_COMMON nodes.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/page_alloc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 0448d04e048a..1ec7d8000792 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -2958,7 +2958,7 @@ static bool free_frozen_page_commit(struct zone *zone,
 		 * kswapd work again by resetting kswapd_failures.
 		 */
 		if (kswapd_test_hopeless(pgdat) &&
-		    num_node_state(N_MEMORY) > 1)
+		    num_node_state(N_MEMORY_COMMON) > 1)
 			kswapd_clear_hopeless(pgdat, KSWAPD_CLEAR_HOPELESS_PCP);
 	}
 	return ret;

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 16/60] mm/oom_kill: compare OOM constraints against common memory
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (14 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 15/60] mm/page_alloc: count common memory nodes for kswapd recovery Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:21   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 17/60] PM: hibernate: size the image from " Gregory Price via B4 Relay
                   ` (44 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Ordinary allocations reach only common-memory zonelists. Compare policy
and cpuset constraints against N_MEMORY_COMMON, so nodes outside it
neither make an otherwise global policy appear constrained nor inflate
the constrained capacity.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/oom_kill.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/mm/oom_kill.c b/mm/oom_kill.c
index 5d48bd862c27..1ec22bdad1fd 100644
--- a/mm/oom_kill.c
+++ b/mm/oom_kill.c
@@ -274,9 +274,9 @@ static enum oom_constraint constrained_alloc(struct oom_control *oc)
 	 * get_page_from_freelist().
 	 */
 	if (oc->nodemask &&
-	    !nodes_subset(node_states[N_MEMORY], *oc->nodemask)) {
+	    !nodes_subset(node_states[N_MEMORY_COMMON], *oc->nodemask)) {
 		oc->totalpages = total_swap_pages;
-		for_each_node_mask(nid, *oc->nodemask)
+		for_each_node_mask_state(nid, *oc->nodemask, N_MEMORY_COMMON)
 			oc->totalpages += node_present_pages(nid);
 		return CONSTRAINT_MEMORY_POLICY;
 	}
@@ -289,7 +289,8 @@ static enum oom_constraint constrained_alloc(struct oom_control *oc)
 
 	if (cpuset_limited) {
 		oc->totalpages = total_swap_pages;
-		for_each_node_mask(nid, cpuset_current_mems_allowed)
+		for_each_node_mask_state(nid, cpuset_current_mems_allowed,
+					 N_MEMORY_COMMON)
 			oc->totalpages += node_present_pages(nid);
 		return CONSTRAINT_CPUSET;
 	}

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 17/60] PM: hibernate: size the image from common memory
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (15 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 16/60] mm/oom_kill: compare OOM constraints against common memory Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:35   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 18/60] mm/khugepaged: base the recommended min_free_kbytes on " Gregory Price via B4 Relay
                   ` (43 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

hibernate_preallocate_memory(), enough_free_mem() and
count_free_highmem_pages() count the free pages of every populated zone
as room for the hibernation image, but the pages used to store the image
can only come from common memory.

Count free pages only in zones on N_MEMORY_COMMON nodes.

The image metadata estimate is a separate sum.  The bitmaps track the
page frames, and every zone's frames are tracked (regardless of node),
so that sum still needs to be calculated for all populated zones.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 kernel/power/snapshot.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/kernel/power/snapshot.c b/kernel/power/snapshot.c
index d5dba0e50b2e..912804bee361 100644
--- a/kernel/power/snapshot.c
+++ b/kernel/power/snapshot.c
@@ -1291,16 +1291,19 @@ static void mark_free_pages(struct zone *zone)
 /**
  * count_free_highmem_pages - Compute the total number of free highmem pages.
  *
- * The returned number is system-wide.
+ * Only common memory counts, since image pages come from common memory.
  */
 static unsigned int count_free_highmem_pages(void)
 {
 	struct zone *zone;
 	unsigned int cnt = 0;
 
-	for_each_populated_zone(zone)
+	for_each_populated_zone(zone) {
+		if (!node_state(zone_to_nid(zone), N_MEMORY_COMMON))
+			continue;
 		if (is_highmem(zone))
 			cnt += zone_page_state(zone, NR_FREE_PAGES);
+	}
 
 	return cnt;
 }
@@ -1868,6 +1871,9 @@ int hibernate_preallocate_memory(void)
 	size = 0;
 	for_each_populated_zone(zone) {
 		size += snapshot_additional_pages(zone);
+		/* Image pages come from ordinary allocations. */
+		if (!node_state(zone_to_nid(zone), N_MEMORY_COMMON))
+			continue;
 		if (is_highmem(zone))
 			highmem += zone_page_state(zone, NR_FREE_PAGES);
 		else
@@ -2011,9 +2017,12 @@ static int enough_free_mem(unsigned int nr_pages, unsigned int nr_highmem)
 	struct zone *zone;
 	unsigned int free = alloc_normal;
 
-	for_each_populated_zone(zone)
+	for_each_populated_zone(zone) {
+		if (!node_state(zone_to_nid(zone), N_MEMORY_COMMON))
+			continue;
 		if (!is_highmem(zone))
 			free += zone_page_state(zone, NR_FREE_PAGES);
+	}
 
 	nr_pages += count_pages_for_highmem(nr_highmem);
 	pr_debug("Normal pages needed: %u + %u, available pages: %u\n",

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 18/60] mm/khugepaged: base the recommended min_free_kbytes on common memory
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (16 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 17/60] PM: hibernate: size the image from " Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:20   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 19/60] mm: add folio_is_common_memory() and page_is_common_memory() Gregory Price via B4 Relay
                   ` (42 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

set_recommended_min_free_kbytes() reserves two pageblocks per populated
zone to help THP allocations avoid fragmentation. The reserve serves
ordinary allocations, which use only common memory, so count only the
zones on N_MEMORY_COMMON nodes.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/khugepaged.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index 2dc4245814df..a930d99f89da 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -3151,6 +3151,9 @@ void set_recommended_min_free_kbytes(void)
 	}
 
 	for_each_populated_zone(zone) {
+		if (!node_state(zone_to_nid(zone), N_MEMORY_COMMON))
+			continue;
+
 		/*
 		 * We don't need to worry about fragmentation of
 		 * ZONE_MOVABLE since it only has movable pages.

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 19/60] mm: add folio_is_common_memory() and page_is_common_memory()
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (17 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 18/60] mm/khugepaged: base the recommended min_free_kbytes on " Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:20   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 20/60] mm/ksm: restrict merging to common memory Gregory Price via B4 Relay
                   ` (41 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

N_MEMORY_COMMON decides where ordinary allocations and node-scoped
services go, but MM services that walk folios reach them through page
tables, physical addresses or LRU lists, whatever node they are on.

Add folio_is_common_memory() and page_is_common_memory() for those
services. ZONE_DEVICE folios are never common memory, so the helpers
can replace the folio_is_zone_device() and page_is_zone_device() checks
those services already make. Following commits convert them.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/internal.h | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/mm/internal.h b/mm/internal.h
index c102e0f498bf..cd0562f0b68f 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -65,6 +65,21 @@ void folio_mark_lazyfree(struct folio *folio);
 void node_set_memory_state(int nid);
 void node_clear_memory_state(int nid);
 
+/*
+ * Common memory is available to generic MM services. ZONE_DEVICE folios do
+ * not qualify even when their NUMA node is common.
+ */
+static inline bool folio_is_common_memory(struct folio *folio)
+{
+	return !folio_is_zone_device(folio) &&
+	       node_state(folio_nid(folio), N_MEMORY_COMMON);
+}
+
+static inline bool page_is_common_memory(struct page *page)
+{
+	return folio_is_common_memory(page_folio(page));
+}
+
 /* mm/vmscan.c */
 unsigned long zone_reclaimable_pages(struct zone *zone);
 unsigned long try_to_free_pages(struct zonelist *zonelist, int order,

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 20/60] mm/ksm: restrict merging to common memory
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (18 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 19/60] mm: add folio_is_common_memory() and page_is_common_memory() Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:20   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 21/60] mm/khugepaged: restrict collapse " Gregory Price via B4 Relay
                   ` (40 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

KSM is an opportunistic MM service and should leave memory outside
the ordinary memory pool alone - as it already does for ZONE_DEVICE
folios where its scan picks up candidate pages.

Use folio_is_common_memory() at those same sites, which still rejects
ZONE_DEVICE folios.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/ksm.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/mm/ksm.c b/mm/ksm.c
index 623a2cdb3924..25b2c2f94486 100644
--- a/mm/ksm.c
+++ b/mm/ksm.c
@@ -831,7 +831,7 @@ static struct page *get_mergeable_page(struct ksm_rmap_item *rmap_item)
 
 	folio = folio_walk_start(&fw, vma, addr, 0);
 	if (folio) {
-		if (!folio_is_zone_device(folio) &&
+		if (folio_is_common_memory(folio) &&
 		    folio_test_anon(folio)) {
 			folio_get(folio);
 			page = fw.page;
@@ -2571,7 +2571,8 @@ static int ksm_next_page_pmd_entry(pmd_t *pmdp, unsigned long addr, unsigned lon
 				goto not_found_unlock;
 			folio = page_folio(page);
 
-			if (folio_is_zone_device(folio) || !folio_test_anon(folio))
+			if (!folio_is_common_memory(folio) ||
+			    !folio_test_anon(folio))
 				goto not_found_unlock;
 
 			page += ((addr & (PMD_SIZE - 1)) >> PAGE_SHIFT);
@@ -2595,7 +2596,8 @@ static int ksm_next_page_pmd_entry(pmd_t *pmdp, unsigned long addr, unsigned lon
 			continue;
 		folio = page_folio(page);
 
-		if (folio_is_zone_device(folio) || !folio_test_anon(folio))
+		if (!folio_is_common_memory(folio) ||
+		    !folio_test_anon(folio))
 			continue;
 		goto found_unlock;
 	}

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 21/60] mm/khugepaged: restrict collapse to common memory
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (19 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 20/60] mm/ksm: restrict merging to common memory Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:21   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 22/60] mm/mempolicy: don't NUMA-hint folios outside " Gregory Price via B4 Relay
                   ` (39 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Collapse is an opportunistic MM service and should operate only on
common memory. The anonymous scan and isolation paths already filter
zone device for the same reason.

Replace the zone_device check with page_is_common_memory().

collapse_scan_file() gains a folio_is_common_memory() check. Page-cache
folios are never ZONE_DEVICE, so there it only filters memory outside
N_MEMORY_COMMON.

collapse_file() rechecks once each folio is locked, since a folio can be
added or migrated after collapse_scan_file() looked at it.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/khugepaged.c | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index a930d99f89da..b1c6bb95a936 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -628,7 +628,7 @@ static enum scan_result __collapse_huge_page_isolate(struct vm_area_struct *vma,
 			goto out;
 		}
 		page = vm_normal_page(vma, addr, pteval);
-		if (unlikely(!page) || unlikely(is_zone_device_page(page))) {
+		if (unlikely(!page) || unlikely(!page_is_common_memory(page))) {
 			result = SCAN_PAGE_NULL;
 			goto out;
 		}
@@ -1673,7 +1673,7 @@ static enum scan_result collapse_scan_anon(struct vm_area_struct *vma,
 		}
 
 		page = vm_normal_page(vma, addr, pteval);
-		if (unlikely(!page) || unlikely(is_zone_device_page(page))) {
+		if (unlikely(!page) || unlikely(!page_is_common_memory(page))) {
 			result = SCAN_PAGE_NULL;
 			goto out_unmap;
 		}
@@ -2397,6 +2397,12 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
 			goto out_unlock;
 		}
 
+		/* A folio may have been added or migrated since the scan. */
+		if (unlikely(!folio_is_common_memory(folio))) {
+			result = SCAN_PAGE_NULL;
+			goto out_unlock;
+		}
+
 		if (!is_shmem && (folio_test_dirty(folio) ||
 				  folio_test_writeback(folio))) {
 			/*
@@ -2737,6 +2743,13 @@ static enum scan_result collapse_scan_file(struct mm_struct *mm,
 			continue;
 		}
 
+		if (unlikely(!folio_is_common_memory(folio))) {
+			result = SCAN_PAGE_NULL;
+			failed_pfn = folio_pfn(folio);
+			folio_put(folio);
+			break;
+		}
+
 		if (is_pmd_order(folio_order(folio))) {
 			result = SCAN_PTE_MAPPED_HUGEPAGE;
 			failed_pfn = folio_pfn(folio);

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 22/60] mm/mempolicy: don't NUMA-hint folios outside common memory
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (20 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 21/60] mm/khugepaged: restrict collapse " Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:21   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 23/60] mm/damon: restrict DAMON-native folio operations to " Gregory Price via B4 Relay
                   ` (38 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

The NUMA balancing scanner installs PROT_NONE only on folios that
balancing may migrate.

folio_can_map_prot_numa() already skips ZONE_DEVICE folios. Skip folios
outside the common memory pool (N_MEMORY_COMMON) as well. Do not inject
hint faults on those folios or migrate them.

Convert the zone_device check to a common memory check (which includes
the zone device check explicitly).

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/mempolicy.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 806577e1d48b..07d577aca78c 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -863,7 +863,7 @@ bool folio_can_map_prot_numa(struct folio *folio, struct vm_area_struct *vma,
 {
 	int nid;
 
-	if (!folio || folio_is_zone_device(folio) || folio_test_ksm(folio))
+	if (!folio || !folio_is_common_memory(folio) || folio_test_ksm(folio))
 		return false;
 
 	/* Also skip shared copy-on-write folios */

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 23/60] mm/damon: restrict DAMON-native folio operations to common memory
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (21 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 22/60] mm/mempolicy: don't NUMA-hint folios outside " Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:21   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 24/60] mm/madvise: restrict reclaim advice " Gregory Price via B4 Relay
                   ` (37 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

DAMON acquires folios to age them, check their accesses, and apply LRU
operations and migration. Treat these as general-purpose MM services
and restrict them to N_MEMORY_COMMON.

Memory outside N_MEMORY_COMMON can still hold ordinary LRU folios, so
it would pass the checks that keep ZONE_DEVICE memory out of these
operations.

Treat these folios like ZONE_DEVICE memory.

DAMON cannot acquire such a folio, so it never ages one and virtual
address monitoring reports it as accessed, and DAMOS cannot isolate
one for migration.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/damon/ops-common.c | 5 +++--
 mm/damon/vaddr.c      | 2 +-
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/mm/damon/ops-common.c b/mm/damon/ops-common.c
index 15cabc79613b..57201a0c5e3a 100644
--- a/mm/damon/ops-common.c
+++ b/mm/damon/ops-common.c
@@ -17,8 +17,9 @@
 
 static bool damon_folio_acceptable(struct folio *folio, bool monitor)
 {
-	return folio_test_lru(folio) ||
-		(monitor && folio_test_hugetlb(folio));
+	return folio_is_common_memory(folio) &&
+		(folio_test_lru(folio) ||
+		 (monitor && folio_test_hugetlb(folio)));
 }
 
 /*
diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c
index 1e65e7651cd9..338876559140 100644
--- a/mm/damon/vaddr.c
+++ b/mm/damon/vaddr.c
@@ -830,7 +830,7 @@ static void damos_va_migrate_dests_add(struct folio *folio,
 	if (folio_nid(folio) == nid)
 		return;
 
-	if (!folio_isolate_lru(folio))
+	if (!folio_is_common_memory(folio) || !folio_isolate_lru(folio))
 		return;
 	node_stat_add_folio(folio, NR_ISOLATED_ANON +
 			folio_is_file_lru(folio));

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 24/60] mm/madvise: restrict reclaim advice to common memory
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (22 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 23/60] mm/damon: restrict DAMON-native folio operations to " Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:19   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 25/60] mm/mempolicy: queue only common memory for mbind() and migrate_pages() Gregory Price via B4 Relay
                   ` (36 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

MADV_COLD and MADV_PAGEOUT age or reclaim the folios a caller names,
and MADV_FREE marks them lazyfree, whatever node they are on.

Restrict them to common memory: convert their ZONE_DEVICE checks,
including the one in madvise_free_huge_pmd(), to
folio_is_common_memory().

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/huge_memory.c | 2 +-
 mm/madvise.c     | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 75e8641bf9ff..cae28513b121 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2452,7 +2452,7 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,
 
 	folio = pmd_folio(orig_pmd);
 
-	if (folio_is_zone_device(folio))
+	if (!folio_is_common_memory(folio))
 		goto out;
 
 	/*
diff --git a/mm/madvise.c b/mm/madvise.c
index 2d54cb9863ea..0f1648526f15 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -404,7 +404,7 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,
 		if (!folio)
 			goto huge_unlock;
 
-		if (folio_is_zone_device(folio))
+		if (!folio_is_common_memory(folio))
 			goto huge_unlock;
 
 		/* Do not interfere with other mappings of this folio */
@@ -485,7 +485,7 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,
 			continue;
 
 		folio = vm_normal_folio(vma, addr, ptent);
-		if (!folio || folio_is_zone_device(folio))
+		if (!folio || !folio_is_common_memory(folio))
 			continue;
 
 		/*
@@ -715,7 +715,7 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr,
 		}
 
 		folio = vm_normal_folio(vma, addr, ptent);
-		if (!folio || folio_is_zone_device(folio))
+		if (!folio || !folio_is_common_memory(folio))
 			continue;
 
 		/*

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 25/60] mm/mempolicy: queue only common memory for mbind() and migrate_pages()
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (23 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 24/60] mm/madvise: restrict reclaim advice " Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:25   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 26/60] mm/migrate: leave pages outside common memory in move_pages() Gregory Price via B4 Relay
                   ` (35 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

mbind(MPOL_MF_MOVE) and migrate_pages() queue folios from a task's
mappings for migration.

Queue only folios on common memory: convert the ZONE_DEVICE checks to
folio_is_common_memory(). As with ZONE_DEVICE folios, the skipped folios
do not count as failures, so MPOL_MF_STRICT does not report them.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/mempolicy.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 07d577aca78c..e208018499d6 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -687,7 +687,7 @@ static void queue_folios_pmd(pmd_t *pmd, unsigned long addr,
 		return;
 	}
 	folio = vm_normal_folio_pmd(walk->vma, addr, pmdval);
-	if (!folio || folio_is_zone_device(folio))
+	if (!folio || !folio_is_common_memory(folio))
 		return;
 	if (!queue_folio_required(folio, qp))
 		return;
@@ -745,7 +745,7 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned long addr,
 			continue;
 		}
 		folio = vm_normal_folio(vma, addr, ptent);
-		if (!folio || folio_is_zone_device(folio))
+		if (!folio || !folio_is_common_memory(folio))
 			continue;
 		if (folio_test_large(folio) && max_nr != 1)
 			nr = folio_pte_batch(folio, pte, ptent, max_nr);

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 26/60] mm/migrate: leave pages outside common memory in move_pages()
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (24 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 25/60] mm/mempolicy: queue only common memory for mbind() and migrate_pages() Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:21   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 27/60] mm: add the private-memory node state Gregory Price via B4 Relay
                   ` (34 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

move_pages(2) moves the pages a caller names, and reports the node of
each page in status queries.

Leave pages outside common memory where they are: refuse to move them,
similar to ZONE_DEVICE pages. Status queries move nothing, so they
still report the node of each page.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/migrate.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/migrate.c b/mm/migrate.c
index 208788b130fa..ac583afa1bc5 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -2287,7 +2287,7 @@ static int __add_folio_for_migration(struct folio *folio, int node,
 	if (is_zero_folio(folio) || is_huge_zero_folio(folio))
 		return -EFAULT;
 
-	if (folio_is_zone_device(folio))
+	if (!folio_is_common_memory(folio))
 		return -ENOENT;
 
 	if (folio_nid(folio) == node)

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 27/60] mm: add the private-memory node state
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (25 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 26/60] mm/migrate: leave pages outside common memory in move_pages() Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:25   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 28/60] mm: select memory features per node Gregory Price via B4 Relay
                   ` (33 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

N_MEMORY_COMMON describes memory that ordinary allocations and MM
services may use. Some memory is online but is not part of that pool.

Add N_MEMORY_PRIVATE for nodes with such memory.

N_MEMORY_COMMON and N_MEMORY_PRIVATE partition N_MEMORY, so a caller can
drop private nodes from a mask without also dropping memoryless nodes.

Expose the state via `has_private_memory`.

Without CONFIG_NUMA, node 0 holds all memory and none of it is private.
We have to adjust the node_state check to return false for this node,
previously when CONFIG_NUMA was disabled it would always return true.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 Documentation/ABI/stable/sysfs-devices-node | 10 ++++++++++
 Documentation/mm/physical_memory.rst        |  3 +++
 drivers/base/node.c                         |  2 ++
 include/linux/nodemask.h                    | 12 ++++++++----
 mm/page_alloc.c                             |  1 +
 5 files changed, 24 insertions(+), 4 deletions(-)

diff --git a/Documentation/ABI/stable/sysfs-devices-node b/Documentation/ABI/stable/sysfs-devices-node
index 0a2cf3088233..70ab780c6ddf 100644
--- a/Documentation/ABI/stable/sysfs-devices-node
+++ b/Documentation/ABI/stable/sysfs-devices-node
@@ -46,6 +46,16 @@ Description:
 		requesting the node explicitly, and that kernel memory
 		management services manage.
 
+What:		/sys/devices/system/node/has_private_memory
+Date:		November 2026
+Contact:	Linux Memory Management list <linux-mm@kvack.org>
+Description:
+		Nodelist of the memory nodes absent from has_common_memory.
+		Together the two lists partition has_memory.
+
+		Ordinary allocations do not use memory on a node listed here.
+		See Documentation/mm/numa_private_nodes.rst.
+
 What:		/sys/devices/system/node/nodeX
 Date:		October 2002
 Contact:	Linux Memory Management list <linux-mm@kvack.org>
diff --git a/Documentation/mm/physical_memory.rst b/Documentation/mm/physical_memory.rst
index 86a7d6080535..ddba7c890418 100644
--- a/Documentation/mm/physical_memory.rst
+++ b/Documentation/mm/physical_memory.rst
@@ -174,6 +174,9 @@ nodes with particular properties as defined by ``enum node_states``:
 ``N_MEMORY_COMMON``
   Ordinary allocations and memory management services may use the node's
   memory.
+``N_MEMORY_PRIVATE``
+  The node has memory but is not on the fallback zonelists. Together with
+  ``N_MEMORY_COMMON`` it partitions ``N_MEMORY``.
 ``N_CPU``
   The node has one or more CPUs
 ``N_GENERIC_INITIATOR``
diff --git a/drivers/base/node.c b/drivers/base/node.c
index cc6696d1390f..56a1741a6a45 100644
--- a/drivers/base/node.c
+++ b/drivers/base/node.c
@@ -966,6 +966,7 @@ static struct node_attr node_state_attr[] = {
 #endif
 	[N_MEMORY] = _NODE_ATTR(has_memory, N_MEMORY),
 	[N_MEMORY_COMMON] = _NODE_ATTR(has_common_memory, N_MEMORY_COMMON),
+	[N_MEMORY_PRIVATE] = _NODE_ATTR(has_private_memory, N_MEMORY_PRIVATE),
 	[N_CPU] = _NODE_ATTR(has_cpu, N_CPU),
 	[N_GENERIC_INITIATOR] = _NODE_ATTR(has_generic_initiator,
 					   N_GENERIC_INITIATOR),
@@ -980,6 +981,7 @@ static struct attribute *node_state_attrs[] = {
 #endif
 	&node_state_attr[N_MEMORY].attr.attr,
 	&node_state_attr[N_MEMORY_COMMON].attr.attr,
+	&node_state_attr[N_MEMORY_PRIVATE].attr.attr,
 	&node_state_attr[N_CPU].attr.attr,
 	&node_state_attr[N_GENERIC_INITIATOR].attr.attr,
 	NULL
diff --git a/include/linux/nodemask.h b/include/linux/nodemask.h
index c3b984105bd1..5a9722d57701 100644
--- a/include/linux/nodemask.h
+++ b/include/linux/nodemask.h
@@ -403,9 +403,12 @@ enum node_states {
 	 * to manage per-node meta-data and topology decisions.
 	 * N_MEMORY_COMMON indicates whether the node can satisfy arbitrary
 	 * allocations and whether a service should manage that node's folios.
+	 * N_MEMORY_PRIVATE holds the other memory nodes: together the two
+	 * partition N_MEMORY.
 	 */
 	N_MEMORY,		/* the node has memory */
 	N_MEMORY_COMMON,	/* common pool (fallback zonelist) */
+	N_MEMORY_PRIVATE,	/* memory outside the common pool */
 	N_CPU,		/* The node has one or more cpus */
 	N_GENERIC_INITIATOR,	/* The node has one or more Generic Initiators */
 	NR_NODE_STATES
@@ -478,9 +481,10 @@ static __always_inline void node_set_offline(int nid)
 
 #else
 
+/* Node 0 holds all memory, and none of it is private. */
 static __always_inline int node_state(int node, enum node_states state)
 {
-	return node == 0;
+	return node == 0 && state != N_MEMORY_PRIVATE;
 }
 
 static __always_inline void node_set_state(int node, enum node_states state)
@@ -493,13 +497,13 @@ static __always_inline void node_clear_state(int node, enum node_states state)
 
 static __always_inline int num_node_state(enum node_states state)
 {
-	return 1;
+	return state != N_MEMORY_PRIVATE;
 }
 
 #define for_each_node_state(node, __state) \
-	for ( (node) = 0; (node) == 0; (node) = 1)
+	for ((node) = 0; (node) == 0 && (__state) != N_MEMORY_PRIVATE; (node) = 1)
 
-#define first_node_state(state)	0
+#define first_node_state(state)	((state) == N_MEMORY_PRIVATE ? MAX_NUMNODES : 0)
 static __always_inline unsigned int next_node_state(int nid, enum node_states state)
 {
 	return MAX_NUMNODES;
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 1ec7d8000792..50336eb58edf 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -217,6 +217,7 @@ void __meminit node_set_memory_state(int nid)
 
 void __meminit node_clear_memory_state(int nid)
 {
+	node_clear_state(nid, N_MEMORY_PRIVATE);
 	node_clear_state(nid, N_MEMORY_COMMON);
 	node_clear_state(nid, N_MEMORY);
 }

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 28/60] mm: select memory features per node
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (26 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 27/60] mm: add the private-memory node state Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:24   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 29/60] mm: keep private memory out of the system memory totals Gregory Price via B4 Relay
                   ` (32 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Whether a node's memory is common or private is a property of the
memory, so record it per node.

Add a per-node memory feature mask (pgdat->memory_features) with a
single bit for common memory. Node states are published from the node's
features at boot and when memory is onlined. A node without the common
bit joins N_MEMORY_PRIVATE.

Ordinary System RAM carries every feature. Without NUMA, node 0 holds
all memory, so a mask without common memory is invalid.

Every node keeps the default mask, so all memory remains common.

Suggested-by: Dan J Williams <djbw@kernel.org>
Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 include/linux/mmzone.h   |  1 +
 include/linux/nodemask.h | 12 ++++++++++++
 mm/internal.h            |  2 +-
 mm/memory_hotplug.c      |  2 +-
 mm/mm_init.c             |  7 ++++++-
 mm/page_alloc.c          |  9 +++++++--
 6 files changed, 28 insertions(+), 5 deletions(-)

diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 6a2ad0cff795..d6c1a43c0bc8 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -1612,6 +1612,7 @@ typedef struct pglist_data {
 #ifdef CONFIG_MEMORY_FAILURE
 	struct memory_failure_stats mf_stats;
 #endif
+	unsigned long memory_features;	/* NODE_MEMORY_FEAT_* mask */
 } pg_data_t;
 
 #define node_present_pages(nid)	(NODE_DATA(nid)->node_present_pages)
diff --git a/include/linux/nodemask.h b/include/linux/nodemask.h
index 5a9722d57701..02908ef23a62 100644
--- a/include/linux/nodemask.h
+++ b/include/linux/nodemask.h
@@ -542,6 +542,18 @@ static __always_inline int node_random(const nodemask_t *maskp)
 #define for_each_online_node(node) for_each_node_state(node, N_ONLINE)
 #define for_each_node_with_cpus(node)	for_each_node_state(node, N_CPU)
 
+/* Private nodes omit NODE_MEMORY_FEAT_COMMON. */
+#define NODE_MEMORY_FEAT_COMMON		BIT(0) /* add to fallback zonelist */
+#define NODE_MEMORY_FEAT_ALL		NODE_MEMORY_FEAT_COMMON
+
+/* Private memory needs NUMA: node 0 holds all memory without it. */
+static inline bool node_memory_features_valid(unsigned long features)
+{
+	if (features & ~NODE_MEMORY_FEAT_ALL)
+		return false;
+	return IS_ENABLED(CONFIG_NUMA) || (features & NODE_MEMORY_FEAT_COMMON);
+}
+
 /*
  * For nodemask scratch area.
  * NODEMASK_ALLOC(type, name) allocates an object with a specified type and
diff --git a/mm/internal.h b/mm/internal.h
index cd0562f0b68f..e89dd1153662 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -62,7 +62,7 @@ void lru_add_drain_cpu_zone(struct zone *zone);
 void folio_deactivate(struct folio *folio);
 void folio_mark_lazyfree(struct folio *folio);
 
-void node_set_memory_state(int nid);
+void node_set_memory_state(int nid, unsigned long features);
 void node_clear_memory_state(int nid);
 
 /*
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 6b6873cfbc5a..b4d94021d46e 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -1197,7 +1197,7 @@ int online_pages(unsigned long pfn, unsigned long nr_pages,
 	adjust_present_page_count(pfn_to_page(pfn), group, nr_pages);
 
 	if (node_arg.nid >= 0)
-		node_set_memory_state(nid);
+		node_set_memory_state(nid, NODE_DATA(nid)->memory_features);
 	/*
 	 * Check whether we are adding normal memory to the node for the first
 	 * time.
diff --git a/mm/mm_init.c b/mm/mm_init.c
index f21b3dbcc92a..99a45fd04ac5 100644
--- a/mm/mm_init.c
+++ b/mm/mm_init.c
@@ -1585,6 +1585,9 @@ int __ref free_area_init_core_hotplug(struct pglist_data *pgdat)
 	pgdat->node_start_pfn = 0;
 	pgdat->node_present_pages = 0;
 
+	/* memory is common unless marked private */
+	pgdat->memory_features = NODE_MEMORY_FEAT_ALL;
+
 	/*
 	 * Hot-unplug can leave per-cpu vmstat deltas unfolded (folders skip
 	 * offline nodes) - reconcile this at online. Foreign access to counters
@@ -1743,6 +1746,8 @@ static void __init free_area_init_node(int nid)
 
 	pgdat->node_id = nid;
 	pgdat->node_start_pfn = start_pfn;
+	/* memory is common unless marked private */
+	pgdat->memory_features = NODE_MEMORY_FEAT_ALL;
 
 	if (start_pfn != end_pfn) {
 		pr_info("Initmem setup node %d [mem %#018Lx-%#018Lx]\n", nid,
@@ -1768,7 +1773,7 @@ static void __init check_for_memory(pg_data_t *pgdat)
 {
 	enum zone_type zone_type;
 
-	node_set_memory_state(pgdat->node_id);
+	node_set_memory_state(pgdat->node_id, pgdat->memory_features);
 
 	for (zone_type = 0; zone_type <= ZONE_MOVABLE - 1; zone_type++) {
 		struct zone *zone = &pgdat->node_zones[zone_type];
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 50336eb58edf..3a7c12fec906 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -209,10 +209,15 @@ nodemask_t node_states[NR_NODE_STATES] __read_mostly = {
 };
 EXPORT_SYMBOL(node_states);
 
-void __meminit node_set_memory_state(int nid)
+void __meminit node_set_memory_state(int nid, unsigned long features)
 {
+	WARN_ON_ONCE(!node_memory_features_valid(features));
+
 	node_set_state(nid, N_MEMORY);
-	node_set_state(nid, N_MEMORY_COMMON);
+	if (features & NODE_MEMORY_FEAT_COMMON)
+		node_set_state(nid, N_MEMORY_COMMON);
+	else
+		node_set_state(nid, N_MEMORY_PRIVATE);
 }
 
 void __meminit node_clear_memory_state(int nid)

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 29/60] mm: keep private memory out of the system memory totals
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (27 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 28/60] mm: select memory features per node Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:40   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 30/60] mm/memory_hotplug: keep private memory out of the auto-movable ratio Gregory Price via B4 Relay
                   ` (31 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

totalram_pages() (MemTotal) counts every managed page, and MemFree
counts every free page, including private memory.

Many limits and defaults are sized from totalram_pages():
  - the overcommit limit
  - OOM capacity
  - tmpfs
  - zswap defaults
  - others

Private memory is not usable by ordinary allocations, so it should
not inflate these counts.

Count only general purpose memory in totalram_pages() and
nr_free_pages(). For now general purpose memory is common memory.

The two must agree because some callers (files_maxfiles_init()) use
their difference as used memory. A node's features cannot change while
it has memory, so counting each node by its features keeps the total
exact.

Several readers use NR_FREE_PAGES directly with the intent of getting
the general purpose memory count:
 - overcommit reserve defaults
 - DAMON's free-memory metrics
 - dirty limits
 - MemAvailable

Switch them to nr_free_pages() as well.

The vmstat counters and per-node meminfo still count private memory.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 include/linux/swap.h |  4 ++--
 mm/damon/core.c      |  5 +++--
 mm/internal.h        |  7 +++++++
 mm/mmap.c            |  6 +++---
 mm/nommu.c           |  4 ++--
 mm/page-writeback.c  |  2 +-
 mm/page_alloc.c      |  3 ++-
 mm/show_mem.c        | 20 ++++++++++++++++++--
 8 files changed, 38 insertions(+), 13 deletions(-)

diff --git a/include/linux/swap.h b/include/linux/swap.h
index 4f686709d71f..fe7eb13385a0 100644
--- a/include/linux/swap.h
+++ b/include/linux/swap.h
@@ -308,8 +308,8 @@ static inline swp_entry_t folio_page_swap_entry(const struct folio *folio,
 /* linux/mm/page_alloc.c */
 extern unsigned long totalreserve_pages;
 
-/* Definition of global_zone_page_state not available yet */
-#define nr_free_pages() global_zone_page_state(NR_FREE_PAGES)
+/* linux/mm/show_mem.c */
+unsigned long nr_free_pages(void);
 
 /* linux/mm/folio.c */
 void folio_add_lru(struct folio *folio);
diff --git a/mm/damon/core.c b/mm/damon/core.c
index 986404b547eb..3eec5a17b5e4 100644
--- a/mm/damon/core.c
+++ b/mm/damon/core.c
@@ -10,6 +10,7 @@
 #include <linux/kthread.h>
 #include <linux/memcontrol.h>
 #include <linux/mm.h>
+#include <linux/swap.h>
 #include <linux/pid.h>
 #include <linux/psi.h>
 #include <linux/sched.h>
@@ -3229,7 +3230,7 @@ static unsigned int damos_hugepage_mem_bp(void)
 	unsigned long thp, total_pages, free_pages;
 
 	total_pages = totalram_pages();
-	free_pages = global_zone_page_state(NR_FREE_PAGES);
+	free_pages = nr_free_pages();
 
 	if (total_pages <= free_pages)
 		return 10000;
@@ -3867,7 +3868,7 @@ static int damos_get_wmark_metric_value(enum damos_wmark_metric metric,
 {
 	switch (metric) {
 	case DAMOS_WMARK_FREE_MEM_RATE:
-		*metric_value = global_zone_page_state(NR_FREE_PAGES) * 1000 /
+		*metric_value = nr_free_pages() * 1000 /
 		       totalram_pages();
 		return 0;
 	default:
diff --git a/mm/internal.h b/mm/internal.h
index e89dd1153662..9c162ded5305 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -779,6 +779,13 @@ void set_recommended_min_free_kbytes(void);
 
 extern char * const zone_names[MAX_NR_ZONES];
 
+/* true if the node is capable of servicing general system allocations */
+static inline bool node_memory_is_general_purpose(int nid)
+{
+	return READ_ONCE(NODE_DATA(nid)->memory_features) &
+	       NODE_MEMORY_FEAT_COMMON;
+}
+
 extern int min_free_kbytes;
 extern int defrag_mode;
 
diff --git a/mm/mmap.c b/mm/mmap.c
index 7618e72766a5..091d67f82221 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -1567,7 +1567,7 @@ static int init_user_reserve(void)
 {
 	unsigned long free_kbytes;
 
-	free_kbytes = K(global_zone_page_state(NR_FREE_PAGES));
+	free_kbytes = K(nr_free_pages());
 
 	sysctl_user_reserve_kbytes = min(free_kbytes / 32, SZ_128K);
 	return 0;
@@ -1588,7 +1588,7 @@ static int init_admin_reserve(void)
 {
 	unsigned long free_kbytes;
 
-	free_kbytes = K(global_zone_page_state(NR_FREE_PAGES));
+	free_kbytes = K(nr_free_pages());
 
 	sysctl_admin_reserve_kbytes = min(free_kbytes / 32, SZ_8K);
 	return 0;
@@ -1632,7 +1632,7 @@ static int reserve_mem_notifier(struct notifier_block *nb,
 
 		break;
 	case MEM_OFFLINE:
-		free_kbytes = K(global_zone_page_state(NR_FREE_PAGES));
+		free_kbytes = K(nr_free_pages());
 
 		if (sysctl_user_reserve_kbytes > free_kbytes) {
 			init_user_reserve();
diff --git a/mm/nommu.c b/mm/nommu.c
index df99495c8902..a6f70c5d9ec8 100644
--- a/mm/nommu.c
+++ b/mm/nommu.c
@@ -1858,7 +1858,7 @@ static int __meminit init_user_reserve(void)
 {
 	unsigned long free_kbytes;
 
-	free_kbytes = K(global_zone_page_state(NR_FREE_PAGES));
+	free_kbytes = K(nr_free_pages());
 
 	sysctl_user_reserve_kbytes = min(free_kbytes / 32, 1UL << 17);
 	return 0;
@@ -1879,7 +1879,7 @@ static int __meminit init_admin_reserve(void)
 {
 	unsigned long free_kbytes;
 
-	free_kbytes = K(global_zone_page_state(NR_FREE_PAGES));
+	free_kbytes = K(nr_free_pages());
 
 	sysctl_admin_reserve_kbytes = min(free_kbytes / 32, 1UL << 13);
 	return 0;
diff --git a/mm/page-writeback.c b/mm/page-writeback.c
index 5939f781fc54..00293a64fcea 100644
--- a/mm/page-writeback.c
+++ b/mm/page-writeback.c
@@ -317,7 +317,7 @@ static unsigned long global_dirtyable_memory(void)
 {
 	unsigned long x;
 
-	x = global_zone_page_state(NR_FREE_PAGES);
+	x = nr_free_pages();
 	/*
 	 * Pages reserved for the kernel should not be considered
 	 * dirtyable, to prevent a situation where reclaim has to
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 3a7c12fec906..0a5176a6e9db 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -6513,7 +6513,8 @@ static void setup_per_zone_lowmem_reserve(void);
 void adjust_managed_page_count(struct page *page, long count)
 {
 	atomic_long_add(count, &page_zone(page)->managed_pages);
-	totalram_pages_add(count);
+	if (node_memory_is_general_purpose(page_to_nid(page)))
+		totalram_pages_add(count);
 	setup_per_zone_lowmem_reserve();
 }
 EXPORT_SYMBOL(adjust_managed_page_count);
diff --git a/mm/show_mem.c b/mm/show_mem.c
index b938cbcd774a..b90fb5a0016b 100644
--- a/mm/show_mem.c
+++ b/mm/show_mem.c
@@ -45,7 +45,7 @@ long si_mem_available(void)
 	 * Estimate the amount of memory available for userspace allocations,
 	 * without causing swapping or OOM.
 	 */
-	available = global_zone_page_state(NR_FREE_PAGES) - totalreserve_pages;
+	available = nr_free_pages() - totalreserve_pages;
 
 	/*
 	 * Not all the page cache can be freed, otherwise the system will
@@ -73,11 +73,27 @@ long si_mem_available(void)
 }
 EXPORT_SYMBOL_GPL(si_mem_available);
 
+/* Free pages of general-purpose memory, the memory totalram_pages() counts. */
+unsigned long nr_free_pages(void)
+{
+	unsigned long nr = global_zone_page_state(NR_FREE_PAGES);
+	int nid;
+
+	if (!IS_ENABLED(CONFIG_NUMA))
+		return nr;
+
+	for_each_node_state(nid, N_MEMORY_PRIVATE) {
+		if (!node_memory_is_general_purpose(nid))
+			nr -= min(nr, sum_zone_node_page_state(nid, NR_FREE_PAGES));
+	}
+	return nr;
+}
+
 void si_meminfo(struct sysinfo *val)
 {
 	val->totalram = totalram_pages();
 	val->sharedram = global_node_page_state(NR_SHMEM);
-	val->freeram = global_zone_page_state(NR_FREE_PAGES);
+	val->freeram = nr_free_pages();
 	val->bufferram = nr_blockdev_pages();
 	val->totalhigh = totalhigh_pages();
 	val->freehigh = nr_free_highpages();

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 30/60] mm/memory_hotplug: keep private memory out of the auto-movable ratio
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (28 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 29/60] mm: keep private memory out of the system memory totals Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:33   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 31/60] cgroup/cpuset: inherit common memory when a nodemask is private only Gregory Price via B4 Relay
                   ` (30 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

The auto-movable online policy keeps ZONE_MOVABLE within
auto_movable_ratio of the early kernel memory, since kernel memory
holds the metadata movable memory needs. It counts all populated zones.

Private memory in ZONE_NORMAL is counted as kernel memory, but no
kernel allocation can use it. The policy can then online more
ZONE_MOVABLE than kernel memory can support.

Leave zones and dynamic memory groups on private nodes out of the
statistics the ratio is computed from, as ZONE_DEVICE memory already is.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/memory_hotplug.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index b4d94021d46e..d7ce60c6815f 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -782,6 +782,10 @@ struct auto_movable_stats {
 static void auto_movable_stats_account_zone(struct auto_movable_stats *stats,
 					    struct zone *zone)
 {
+	/* Private memory is outside the movable:kernel balance. */
+	if (node_state(zone_to_nid(zone), N_MEMORY_PRIVATE))
+		return;
+
 	if (zone_idx(zone) == ZONE_MOVABLE) {
 		stats->movable_pages += zone->present_pages;
 	} else {
@@ -808,6 +812,10 @@ static int auto_movable_stats_account_group(struct memory_group *group,
 	struct auto_movable_group_stats *stats = arg;
 	long pages;
 
+	/* private memory is outside the movable:kernel balance. */
+	if (node_state(group->nid, N_MEMORY_PRIVATE))
+		return 0;
+
 	/*
 	 * We don't support modifying the config while the auto-movable online
 	 * policy is already enabled. Just avoid the division by zero below.

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 31/60] cgroup/cpuset: inherit common memory when a nodemask is private only
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (29 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 30/60] mm/memory_hotplug: keep private memory out of the auto-movable ratio Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:43   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 32/60] cgroup/cpuset: keep spread page cache off private nodes Gregory Price via B4 Relay
                   ` (29 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

cpuset.mems governs every N_MEMORY node, but ordinary allocations
can use only N_MEMORY_COMMON memory.

An effective mask without N_MEMORY_COMMON nodes creates an unsatisfiable
condition for ordinary system allocations, such as page tables. This
can cause an unsatisfiable OOM or an infinite fault retry loop.

Treat a cpuset.mems mask without a common memory node as a condition
to inherit the parent's common memory nodes.

The inheritance behavior differs subtly between v1 and v2:

operation                             v1              v2
 write {}, no tasks                 remain empty    inherit all
 write {}, populated                -ENOSPC         inherit all
 write {private}                    -ENOSPC         inherit common
 write {outside parent}             -EACCES         inherit common
 write {common, private}             normal intersection in both modes
 task attach w/ {private}           -ENOSPC         use effective mask
 {common, private}, unplug common   clear & move    keep & add common

In the v2 {private} case, the cpuset inherits only the parent's common
nodes and gains no access to unrelated private nodes. A request naming
no node the parent allows does the same. Systems without private nodes
are entirely unchanged by this commit.

An actual empty request still inherits the parent's full effective mask.

Example: (C=common, P=private)
  parent.cpuset.mems = {C0, C1, P2, P3}
  child.cpuset.mems  = {C0, P2}

  /* User sets private-only mask */
  echo {P2} > child.cpuset.mems
  child.cpuset.mems  = P2
  child.cpuset.effective_mems = {C0, C1, P2}

      or

  /* Hot-unplug removes C0 */
  hotunplug {C0}
  child.cpuset.mems  = {C0, P2}
  child.cpuset.effective_mems = {C1, P2}

      or

  /* User clears the nodemask entirely */
  echo {} > child.cpuset.mems
  child.cpuset.effective_mems => {C0, C1, P2, P3}

Private nodes are thus non-inheritable unless the nodemask is empty.

Rejecting a private-only nodemask in v2 would be confusing (-EINVAL
on a valid node) and inconsistent with hotplug, which can still leave
a cpuset with only private memory. Resolve every case the same way:
effective_mems inherits only the common nodes.

For v1, a write rejects a mask without common memory with -ENOSPC,
even with no tasks attached: v1 has no effective mask to add common
nodes to, so tasks attached later would have no common memory. Memory
hotplug that leaves a mask with only private memory moves its tasks to
an ancestor.

Automatic memory migration uses only common-memory destinations.

A task's mems_allowed comes from guarantee_online_mems(), which walks
up the hierarchy while a cpuset has no memory. Walk up while it has no
common memory instead, so a task never gets a private-only mask in the
window between memory going offline and the hotplug update of the
effective masks.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 kernel/cgroup/cpuset.c | 58 +++++++++++++++++++++++++++++++++++++++-----------
 1 file changed, 46 insertions(+), 12 deletions(-)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index db32d330cfbc..6802ca13c46b 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -534,22 +534,24 @@ static void guarantee_active_cpus(struct task_struct *tsk,
 
 /*
  * Return in *pmask the portion of a cpusets's mems_allowed that
- * are online, with memory.  If none are online with memory, walk
- * up the cpuset hierarchy until we find one that does have some
- * online mems.  The top cpuset always has some mems online. With v2,
+ * are online, with memory.  If none of them is common memory, walk
+ * up the cpuset hierarchy until we find one that does have some.
+ * The top cpuset always has some common memory online. With v2,
  * effective_mems should always contain online memory nodes except
  * during the transition period where a memory node hotunplug operation
  * is in progress.
  *
- * One way or another, we guarantee to return some non-empty subset
- * of node_states[N_MEMORY].
+ * One way or another, we guarantee to return some subset of
+ * node_states[N_MEMORY] that includes common memory.
  *
  * Call with callback_lock or cpuset_mutex held.
  */
 static void guarantee_online_mems(struct cpuset *cs, nodemask_t *pmask)
 {
-	while (!nodes_and(*pmask, cs->effective_mems, node_states[N_MEMORY]))
+	while (!nodes_intersects(cs->effective_mems,
+				 node_states[N_MEMORY_COMMON]))
 		cs = parent_cs(cs);
+	nodes_and(*pmask, cs->effective_mems, node_states[N_MEMORY]);
 }
 
 /**
@@ -1684,22 +1686,40 @@ static void compute_effective_cpumask(struct cpumask *new_cpus,
 		cpumask_copy(new_cpus, parent->effective_cpus);
 }
 
+/* @mems names private memory nodes but no common memory node. */
+static bool mems_private_only(const nodemask_t *mems)
+{
+	return nodes_intersects(*mems, node_states[N_MEMORY_PRIVATE]) &&
+	       !nodes_intersects(*mems, node_states[N_MEMORY_COMMON]);
+}
+
 /**
  * compute_effective_nodemask - Compute the effective nodemask of the cpuset
  * @new_mems: the temp variable for the new effective_mems mask
  * @cs: the cpuset the need to recompute the new effective_mems mask
  * @parent: the parent cpuset
  *
- * For v2, the parent's effective_mems is inherited if nodemask is empty.
+ * A nodemask whose memory is all private cannot serve an ordinary allocation.
+ * In v2, a non-empty request retains the requested nodes granted by the parent
+ * and inherits the parent's nodes other than private ones.
+ * In v1, cpuset_hotplug_update_tasks() empties such a mask, so the tasks move
+ * to an ancestor.
  */
 static void compute_effective_nodemask(nodemask_t *new_mems,
 				       struct cpuset *cs, struct cpuset *parent)
 {
-	bool has_mems;
+	nodemask_t inherited;
 
-	has_mems = nodes_and(*new_mems, cs->mems_allowed, parent->effective_mems);
-	if (!has_mems && is_in_v2_mode())
+	nodes_and(*new_mems, cs->mems_allowed, parent->effective_mems);
+	if (!is_in_v2_mode())
+		return;
+	if (nodes_empty(cs->mems_allowed)) {
 		nodes_copy(*new_mems, parent->effective_mems);
+	} else if (nodes_empty(*new_mems) || mems_private_only(new_mems)) {
+		nodes_andnot(inherited, parent->effective_mems,
+			     node_states[N_MEMORY_PRIVATE]);
+		nodes_or(*new_mems, *new_mems, inherited);
+	}
 }
 
 /*
@@ -2667,6 +2687,8 @@ static void cpuset_migrate_mm_workfn(struct work_struct *work)
 	struct cpuset_migrate_mm_work *mwork =
 		container_of(work, struct cpuset_migrate_mm_work, work);
 
+	/* Private nodes are never migration destinations. */
+	nodes_andnot(mwork->to, mwork->to, node_states[N_MEMORY_PRIVATE]);
 	/* on a wq worker, no need to worry about %current's mems_allowed */
 	do_migrate_pages(mwork->mm, &mwork->from, &mwork->to, MPOL_MF_MOVE_ALL);
 	mmput(mwork->mm);
@@ -2900,6 +2922,14 @@ static int update_nodemask(struct cpuset *cs, struct cpuset *trialcs,
 			  top_cpuset.mems_allowed))
 		return -EINVAL;
 
+	/*
+	 * In v1, if a mask has no common memory nodes, it's treated as empty
+	 * because there is no effective-mems inherited fallback. As a result,
+	 * we treat this state the same as-if there was no memory at all.
+	 */
+	if (!is_in_v2_mode() && mems_private_only(&trialcs->mems_allowed))
+		return -ENOSPC;
+
 	/* No change? nothing to do */
 	if (nodes_equal(cs->mems_allowed, trialcs->mems_allowed))
 		return 0;
@@ -3090,7 +3120,7 @@ static int update_prstate(struct cpuset *cs, int new_prs)
 
 /*
  * Check to see if a cpuset can accept a new task
- * For v1, cpus_allowed and mems_allowed can't be empty.
+ * For v1, cpus_allowed can't be empty and mems_allowed must have common memory.
  * For v2, effective_cpus can't be empty.
  * Note that in v1, effective_cpus = cpus_allowed.
  *
@@ -3103,7 +3133,8 @@ static int cpuset_can_attach_check(struct cpuset *cs, struct cpuset *oldcs,
 	bool cpus_updated, mems_updated;
 
 	if (cpumask_empty(cs->effective_cpus) ||
-	   (!is_in_v2_mode() && nodes_empty(cs->mems_allowed)))
+	   (!is_in_v2_mode() && (nodes_empty(cs->mems_allowed) ||
+				 mems_private_only(&cs->mems_allowed))))
 		return -ENOSPC;
 
 	if (!oldcs)
@@ -3988,6 +4019,9 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs, struct tmpmasks *tmp)
 	parent = parent_cs(cs);
 	compute_effective_cpumask(&new_cpus, cs, parent);
 	compute_effective_nodemask(&new_mems, cs, parent);
+	/* A legacy cpuset left with only private memory moves its tasks. */
+	if (!is_in_v2_mode() && mems_private_only(&new_mems))
+		nodes_clear(new_mems);
 
 	if (!tmp || !cs->partition_root_state)
 		goto update_tasks;

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 32/60] cgroup/cpuset: keep spread page cache off private nodes
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (30 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 31/60] cgroup/cpuset: inherit common memory when a nodemask is private only Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:31   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 33/60] cgroup/cpuset: ignore private nodes when matching OOM victims Gregory Price via B4 Relay
                   ` (28 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

With memory_spread_page set, cpuset_mem_spread_node() rotates the
preferred node for page cache over the task's mems_allowed.

mems_allowed can include private nodes. A private preferred node will
fall back to the nearest common node, which takes an extra share.

Rotate only over the allowed nodes outside N_MEMORY_PRIVATE. A task's
mems_allowed always holds common memory, so the loop always terminates.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 kernel/cgroup/cpuset.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 6802ca13c46b..8401b69ee3da 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -4600,7 +4600,14 @@ void cpuset_nodes_allowed(struct cgroup *cgroup, nodemask_t *mask)
  */
 static int cpuset_spread_node(int *rotor)
 {
-	return *rotor = next_node_in(*rotor, current->mems_allowed);
+	int nid = *rotor;
+
+	/* Spread page cache only over memory that ordinary allocations use. */
+	do {
+		nid = next_node_in(nid, current->mems_allowed);
+	} while (nid < MAX_NUMNODES && node_state(nid, N_MEMORY_PRIVATE));
+
+	return *rotor = nid;
 }
 
 /**

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 33/60] cgroup/cpuset: ignore private nodes when matching OOM victims
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (31 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 32/60] cgroup/cpuset: keep spread page cache off private nodes Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:24   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 34/60] mm/mempolicy: don't rebind policies onto private nodes Gregory Price via B4 Relay
                   ` (27 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

The OOM killer only considers a task an eligible victim if its
mems_allowed intersects the allocating task's mems_allowed. The
assumption is that a shared node means shared ordinary memory.

Private nodes break that assumption. If two cpusets partition common
memory but both allow a private node, an OOM in one could kill tasks in
the other and free nothing the allocating task can use.

Initially, private memory allocations will never invoke the OOM killer,
so only common memory pressure reaches this check.

Leave private nodes out of the intersection.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 kernel/cgroup/cpuset.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 8401b69ee3da..e20f9ccb8ab7 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -4628,15 +4628,19 @@ int cpuset_mem_spread_node(void)
  * @tsk2: pointer to task_struct of some other task.
  *
  * Description: Return true if @tsk1's mems_allowed intersects the
- * mems_allowed of @tsk2.  Used by the OOM killer to determine if
- * one of the task's memory usage might impact the memory available
- * to the other.
+ * mems_allowed of @tsk2 outside private memory nodes.  Used by the OOM
+ * killer to determine if one of the task's memory usage might impact the
+ * memory available to the other.
  **/
 
 int cpuset_mems_allowed_intersects(const struct task_struct *tsk1,
 				   const struct task_struct *tsk2)
 {
-	return nodes_intersects(tsk1->mems_allowed, tsk2->mems_allowed);
+	nodemask_t shared;
+
+	nodes_and(shared, tsk1->mems_allowed, tsk2->mems_allowed);
+	nodes_andnot(shared, shared, node_states[N_MEMORY_PRIVATE]);
+	return !nodes_empty(shared);
 }
 
 /**

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 34/60] mm/mempolicy: don't rebind policies onto private nodes
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (32 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 33/60] cgroup/cpuset: ignore private nodes when matching OOM victims Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:35   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 35/60] mm/mempolicy: keep private nodes out of migrate_pages() destinations Gregory Price via B4 Relay
                   ` (26 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

A cpuset can allow private nodes, and rebinding a policy to a new cpuset
mask can move its nodes onto them:
  - a static policy reapplies the raw user mask.
  - relative and remapped policies follow node positions in the cpuset
    mask.

Ordinary policies cannot use private nodes.

Rebind policies over the cpuset's nodes other than private ones, and
record the remap base the same way, so adding or removing a private node
moves no policy.

Cpusets keep common memory in every effective mask, so a mask with no
node left is a bug. Warn, and keep the whole mask rather than leave the
policy empty.

On systems without a private node, rebinding is unchanged.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/mempolicy.c | 42 ++++++++++++++++++++++++++++++------------
 1 file changed, 30 insertions(+), 12 deletions(-)

diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index e208018499d6..81ccc481cea1 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -418,6 +418,18 @@ static int mpol_new_preferred(struct mempolicy *pol, const nodemask_t *nodes)
 	return 0;
 }
 
+/*
+ * Policies are rebound over the cpuset's nodes other than private ones. A node
+ * removed since the remap base was recorded keeps its position.
+ */
+static void mpol_rebindable_mems(nodemask_t *dst, const nodemask_t *nodes)
+{
+	nodes_andnot(*dst, *nodes, node_states[N_MEMORY_PRIVATE]);
+	/* Cpusets keep common memory in every effective mask. */
+	if (WARN_ON_ONCE(nodes_empty(*dst)))
+		*dst = *nodes;
+}
+
 /*
  * mpol_set_nodemask is called after mpol_new() to set up the nodemask, if
  * any, for the new policy.  mpol_new() has already validated the nodes
@@ -453,7 +465,8 @@ static int mpol_set_nodemask(struct mempolicy *pol,
 	if (mpol_store_user_nodemask(pol))
 		pol->w.user_nodemask = *nodes;
 	else
-		pol->w.cpuset_mems_allowed = cpuset_current_mems_allowed;
+		mpol_rebindable_mems(&pol->w.cpuset_mems_allowed,
+				     &cpuset_current_mems_allowed);
 
 	ret = mpol_ops[pol->mode].create(pol, &nsc->mask2);
 	return ret;
@@ -528,20 +541,22 @@ static void mpol_rebind_default(struct mempolicy *pol, const nodemask_t *nodes)
 
 static void mpol_rebind_nodemask(struct mempolicy *pol, const nodemask_t *nodes)
 {
-	nodemask_t tmp;
+	nodemask_t tmp, allowed;
+
+	mpol_rebindable_mems(&allowed, nodes);
 
-	if (pol->flags & MPOL_F_STATIC_NODES)
-		nodes_and(tmp, pol->w.user_nodemask, *nodes);
-	else if (pol->flags & MPOL_F_RELATIVE_NODES)
-		mpol_relative_nodemask(&tmp, &pol->w.user_nodemask, nodes);
-	else {
+	if (pol->flags & MPOL_F_STATIC_NODES) {
+		nodes_and(tmp, pol->w.user_nodemask, allowed);
+	} else if (pol->flags & MPOL_F_RELATIVE_NODES) {
+		mpol_relative_nodemask(&tmp, &pol->w.user_nodemask, &allowed);
+	} else {
 		nodes_remap(tmp, pol->nodes, pol->w.cpuset_mems_allowed,
-								*nodes);
-		pol->w.cpuset_mems_allowed = *nodes;
+			    allowed);
+		pol->w.cpuset_mems_allowed = allowed;
 	}
 
 	if (nodes_empty(tmp))
-		tmp = *nodes;
+		tmp = allowed;
 
 	pol->nodes = tmp;
 }
@@ -549,7 +564,7 @@ static void mpol_rebind_nodemask(struct mempolicy *pol, const nodemask_t *nodes)
 static void mpol_rebind_preferred(struct mempolicy *pol,
 						const nodemask_t *nodes)
 {
-	pol->w.cpuset_mems_allowed = *nodes;
+	mpol_rebindable_mems(&pol->w.cpuset_mems_allowed, nodes);
 }
 
 /*
@@ -561,10 +576,13 @@ static void mpol_rebind_preferred(struct mempolicy *pol,
  */
 static void mpol_rebind_policy(struct mempolicy *pol, const nodemask_t *newmask)
 {
+	nodemask_t mems;
+
 	if (!pol || pol->mode == MPOL_LOCAL)
 		return;
+	mpol_rebindable_mems(&mems, newmask);
 	if (!mpol_store_user_nodemask(pol) &&
-	    nodes_equal(pol->w.cpuset_mems_allowed, *newmask))
+	    nodes_equal(pol->w.cpuset_mems_allowed, mems))
 		return;
 
 	mpol_ops[pol->mode].rebind(pol, newmask);

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 35/60] mm/mempolicy: keep private nodes out of migrate_pages() destinations
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (33 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 34/60] mm/mempolicy: don't rebind policies onto private nodes Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:25   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 36/60] mm: refuse explicit reclaim and compaction of private nodes Gregory Price via B4 Relay
                   ` (25 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

migrate_pages(2) intersects its destination nodes with the caller's
cpuset, which can include private nodes.

A private destination gets mapped to some source node, and every page
bound for it attempts a __GFP_THISNODE allocation. A private node is not
on its own __GFP_THISNODE zonelist, so the call fails with -ENOMEM and
logs an allocation failure.

Leave private nodes out of the destination mask, as move_pages(2)
already rejects them. A mask that names only private nodes fails with
-EINVAL.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/mempolicy.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 81ccc481cea1..2bfe8b2b6287 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -1958,6 +1958,8 @@ static int kernel_migrate_pages(pid_t pid, unsigned long maxnode,
 	}
 
 	task_nodes = cpuset_mems_allowed(current);
+	/* Private memory is not a migration destination. */
+	nodes_andnot(task_nodes, task_nodes, node_states[N_MEMORY_PRIVATE]);
 	if (!nodes_and(*new, *new, task_nodes))
 		goto out_put;
 

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 36/60] mm: refuse explicit reclaim and compaction of private nodes
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (34 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 35/60] mm/mempolicy: keep private nodes out of migrate_pages() destinations Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:24   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 37/60] mm/page_alloc: keep private memory out of reclaim reserves Gregory Price via B4 Relay
                   ` (24 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

kswapd and kcompactd do not run on private nodes, but nodeN/reclaim and
nodeN/compact name a node directly. Reject them for a private node with
-EOPNOTSUPP.

Memoryless nodes keep accepting the requests.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/compaction.c | 3 +++
 mm/vmscan.c     | 3 +++
 2 files changed, 6 insertions(+)

diff --git a/mm/compaction.c b/mm/compaction.c
index 879f372a891e..283d4d4e78c3 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -3038,6 +3038,9 @@ static ssize_t compact_store(struct device *dev,
 	int nid = dev->id;
 
 	if (nid >= 0 && nid < nr_node_ids && node_online(nid)) {
+		if (node_state(nid, N_MEMORY_PRIVATE))
+			return -EOPNOTSUPP;
+
 		/* Flush pending updates to the LRU lists */
 		lru_add_drain_all();
 
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 5b53e0391d80..3e934135e805 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -8268,6 +8268,9 @@ static ssize_t reclaim_store(struct device *dev,
 {
 	int ret, nid = dev->id;
 
+	if (node_state(nid, N_MEMORY_PRIVATE))
+		return -EOPNOTSUPP;
+
 	ret = user_proactive_reclaim((char *)buf, NULL, NODE_DATA(nid));
 	return ret ? ret : count;
 }

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 37/60] mm/page_alloc: keep private memory out of reclaim reserves
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (35 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 36/60] mm: refuse explicit reclaim and compaction of private nodes Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:31   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 38/60] mm/page_alloc: refuse contiguous allocation from private memory Gregory Price via B4 Relay
                   ` (23 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Private memory cannot serve general system allocations, so it should not
contribute to reserves.

Exclude private capacity from the lowmem denominator and the global
reserve total, so the pages_min budget and the system reserve stay on
common memory. A private node still records its own reserve, which
its per-node dirty limit subtracts.

Zero private zones' watermarks. No kswapd runs for a private node to
restore them, and the pages below the minimum would be unreachable.
With no high watermark, a migratetype fallback never boosts them
either.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/page_alloc.c | 23 ++++++++++++++++++++---
 1 file changed, 20 insertions(+), 3 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 0a5176a6e9db..de1a13a91ef4 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -6610,6 +6610,8 @@ static void calculate_totalreserve_pages(void)
 	enum zone_type i, j;
 
 	for_each_online_pgdat(pgdat) {
+		/* Only common memory holds the system reserve. */
+		bool common = node_state(pgdat->node_id, N_MEMORY_COMMON);
 
 		pgdat->totalreserve_pages = 0;
 
@@ -6640,7 +6642,8 @@ static void calculate_totalreserve_pages(void)
 
 			pgdat->totalreserve_pages += max;
 
-			reserve_pages += max;
+			if (common)
+				reserve_pages += max;
 		}
 	}
 	totalreserve_pages = reserve_pages;
@@ -6705,9 +6708,13 @@ static void __setup_per_zone_wmarks(void)
 	struct zone *zone;
 	unsigned long flags;
 
-	/* Calculate total number of !ZONE_HIGHMEM and !ZONE_MOVABLE pages */
+	/*
+	 * Calculate total number of !ZONE_HIGHMEM and !ZONE_MOVABLE pages.
+	 * Distribute the global pages_min budget only over common memory.
+	 */
 	for_each_zone(zone) {
-		if (!is_highmem(zone) && zone_idx(zone) != ZONE_MOVABLE)
+		if (!is_highmem(zone) && zone_idx(zone) != ZONE_MOVABLE &&
+		    !node_state(zone_to_nid(zone), N_MEMORY_PRIVATE))
 			lowmem_pages += zone_managed_pages(zone);
 	}
 
@@ -6715,6 +6722,16 @@ static void __setup_per_zone_wmarks(void)
 		u64 tmp;
 
 		spin_lock_irqsave(&zone->lock, flags);
+		/* No kswapd runs for a private node to restore watermarks. */
+		if (node_state(zone_to_nid(zone), N_MEMORY_PRIVATE)) {
+			zone->_watermark[WMARK_MIN] = 0;
+			zone->_watermark[WMARK_LOW] = 0;
+			zone->_watermark[WMARK_HIGH] = 0;
+			zone->_watermark[WMARK_PROMO] = 0;
+			zone->watermark_boost = 0;
+			spin_unlock_irqrestore(&zone->lock, flags);
+			continue;
+		}
 		tmp = (u64)pages_min * zone_managed_pages(zone);
 		tmp = div64_ul(tmp, lowmem_pages);
 		if (is_highmem(zone) || zone_idx(zone) == ZONE_MOVABLE) {

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 38/60] mm/page_alloc: refuse contiguous allocation from private memory
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (36 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 37/60] mm/page_alloc: keep private memory out of reclaim reserves Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:24   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 39/60] mm/page_alloc: add a private-node zonelist Gregory Price via B4 Relay
                   ` (22 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Exact-PFN contiguous allocation bypasses the zonelists and can invoke
compaction and reclaim.

Reject a range on a private node with -EPERM before isolating any pages,
and document the error.

If a user of private ranges shows up, alloc_contig_range() can grow an
ACR flag or an exported api for private allocations.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/page_alloc.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index de1a13a91ef4..08e57f809a03 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -7379,7 +7379,8 @@ static void __free_contig_frozen_range(unsigned long pfn, unsigned long nr_pages
  * free_frozen_pages() also could be used to free compound frozen pages
  * directly.
  *
- * Return: zero on success or negative error code.
+ * Return: zero on success or negative error code
+ *         -EPERM if the range is on a private memory node.
  */
 int alloc_contig_frozen_range_noprof(unsigned long start, unsigned long end,
 		acr_flags_t alloc_flags, gfp_t gfp_mask)
@@ -7410,6 +7411,9 @@ int alloc_contig_frozen_range_noprof(unsigned long start, unsigned long end,
 	if (WARN_ON_ONCE((gfp_mask & __GFP_COMP) && order > MAX_FOLIO_ORDER))
 		return -EINVAL;
 
+	if (node_state(zone_to_nid(cc.zone), N_MEMORY_PRIVATE))
+		return -EPERM;
+
 	gfp_mask = current_gfp_context(gfp_mask);
 	if (__alloc_contig_verify_gfp_mask(gfp_mask, (gfp_t *)&cc.gfp_mask))
 		return -EINVAL;

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 39/60] mm/page_alloc: add a private-node zonelist
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (37 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 38/60] mm/page_alloc: refuse contiguous allocation from private memory Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:31   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 40/60] mm/page_alloc: let MM callers select the local private zonelist Gregory Price via B4 Relay
                   ` (21 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

A private node carries N_MEMORY without N_MEMORY_COMMON, so the FALLBACK
and NOFALLBACK zonelists already exclude it. That leaves no zonelist
that can reach it.

Add ZONELIST_PRIVATE, which holds a node's local zones only if the node
is private. Common nodes have an empty private zonelist.

Like the existing zonelists, it is sized for every zone of every node.
That adds a zoneref (16 bytes on 64-bit) per possible zone, MAX_NUMNODES
* MAX_NR_ZONES, to each pgdat. A smaller local-only zonelist type can be
a follow-up if this matters.

When the zonelist becomes selectable, __GFP_THISNODE will still suppress
OOM, while the page allocator selects the private zonelist instead of
NOFALLBACK.

If a user grows requiring fallback behavior to COMMON nodes when the
private node is out of memory, we can consider adding a fallback
variant - but for now private node allocations use a local-only
zonelist to limit complexity.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 include/linux/mmzone.h | 5 +++--
 mm/mm_init.c           | 5 +++--
 mm/page_alloc.c        | 2 ++
 3 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index d6c1a43c0bc8..05f212b56c9d 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -1402,10 +1402,11 @@ enum {
 	ZONELIST_FALLBACK,	/* zonelist with fallback */
 #ifdef CONFIG_NUMA
 	/*
-	 * The NUMA zonelists are doubled because we need zonelists that
-	 * restrict the allocations to a single node for __GFP_THISNODE.
+	 * NUMA nodes also have local-only zonelists that restrict allocations
+	 * to a single node, for __GFP_THISNODE and for private-node access.
 	 */
 	ZONELIST_NOFALLBACK,	/* zonelist without fallback (__GFP_THISNODE) */
+	ZONELIST_PRIVATE,	/* local private-node access, no fallback */
 #endif
 	MAX_ZONELISTS
 };
diff --git a/mm/mm_init.c b/mm/mm_init.c
index 99a45fd04ac5..a6ff49043b5c 100644
--- a/mm/mm_init.c
+++ b/mm/mm_init.c
@@ -99,7 +99,8 @@ void __init mminit_verify_zonelist(void)
 
 			/* Print information about the zonelist */
 			printk(KERN_DEBUG "mminit::zonelist %s %d:%s = ",
-				listid > 0 ? "thisnode" : "general", nid,
+				listid == 0 ? "general" :
+				listid == 1 ? "thisnode" : "private", nid,
 				zone->name);
 
 			/* Iterate the zonelist */
@@ -2706,7 +2707,7 @@ void __init mm_core_init(void)
 	init_zero_page_pfn();
 
 	/* Initializations relying on SMP setup */
-	BUILD_BUG_ON(MAX_ZONELISTS > 2);
+	BUILD_BUG_ON(MAX_ZONELISTS > 3);
 	build_all_zonelists(NULL);
 	page_alloc_init_cpuhp();
 	alloc_tag_sec_init();
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 08e57f809a03..0bd21916cea1 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -6010,6 +6010,8 @@ static void build_zonelists(pg_data_t *pgdat)
 			    ZONELIST_FALLBACK);
 	build_local_zonelist(pgdat, ZONELIST_NOFALLBACK,
 			     node_state(nid, N_MEMORY_COMMON));
+	build_local_zonelist(pgdat, ZONELIST_PRIVATE,
+			     node_state(nid, N_MEMORY_PRIVATE));
 }
 
 #ifdef CONFIG_HAVE_MEMORYLESS_NODES

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 40/60] mm/page_alloc: let MM callers select the local private zonelist
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (38 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 39/60] mm/page_alloc: add a private-node zonelist Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:46   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 41/60] mm/page_alloc: relax a private request whose node becomes ineligible Gregory Price via B4 Relay
                   ` (20 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Add ALLOC_ZONELIST_PRIVATE, which selects the local private zonelist
independently of __GFP_THISNODE.

__GFP_THISNODE still governs OOM behavior separately.

Allocations are otherwise still filtered by the task's cpuset, this
creates some quirks.

Reserve allocations normally drop the nodemask and the cpuset check.
A private request keeps both, since private memory has no reserves
to grant. A non-blocking __GFP_HIGH request still skips the cpuset
check, as it does for ordinary allocations.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/page_alloc.c | 21 +++++++++++++++------
 mm/page_alloc.h | 15 +++++++++++++++
 2 files changed, 30 insertions(+), 6 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 0bd21916cea1..a219057931c3 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -4926,16 +4926,24 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
 		goto got_pg;
 
 	reserve_flags = __gfp_pfmemalloc_flags(gfp_mask);
-	if (reserve_flags)
+	if (reserve_flags) {
+		unsigned int keep = ALLOC_KSWAPD;
+
+		/* The cpuset still grants a private request's node. */
+		if (ac->alloc_flags & ALLOC_ZONELIST_PRIVATE)
+			keep |= ALLOC_CPUSET;
 		alloc_flags = alloc_flags_cma(gfp_mask) | reserve_flags |
-				ac->alloc_flags | (alloc_flags & ALLOC_KSWAPD);
+				ac->alloc_flags | (alloc_flags & keep);
+	}
 
 	/*
 	 * Reset the nodemask and zonelist iterators if memory policies can be
 	 * ignored. These allocations are high priority and system rather than
-	 * user oriented.
+	 * user oriented. A private request keeps its node: private memory has
+	 * no reserves to grant.
 	 */
-	if (!(alloc_flags & ALLOC_CPUSET) || reserve_flags) {
+	if ((!(alloc_flags & ALLOC_CPUSET) || reserve_flags) &&
+	    !(ac->alloc_flags & ALLOC_ZONELIST_PRIVATE)) {
 		ac->nodemask = NULL;
 		ac->preferred_zoneref = first_zones_zonelist(ac->zonelist,
 					ac->highest_zoneidx, ac->nodemask);
@@ -5164,7 +5172,7 @@ static inline bool prepare_alloc_pages(gfp_t gfp_mask, unsigned int order,
 		unsigned int *alloc_flags)
 {
 	ac->highest_zoneidx = gfp_zone(gfp_mask);
-	ac->zonelist = node_zonelist(preferred_nid, gfp_mask);
+	ac->zonelist = select_zonelist(preferred_nid, gfp_mask, *alloc_flags);
 	ac->nodemask = nodemask;
 	ac->migratetype = gfp_migratetype(gfp_mask);
 
@@ -5477,7 +5485,8 @@ struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order,
 	unsigned int zonelist_iter_cookie;
 
 	/* Other flags could be supported later if needed. */
-	if (WARN_ON(alloc_flags & ~(ALLOC_NOLOCK | ALLOC_NO_CODETAG)))
+	if (WARN_ON(alloc_flags & ~(ALLOC_NOLOCK | ALLOC_NO_CODETAG |
+				    ALLOC_ZONELIST_PRIVATE)))
 		return NULL;
 
 	if (!alloc_order_allowed(gfp, order, alloc_flags))
diff --git a/mm/page_alloc.h b/mm/page_alloc.h
index 917653ae14a3..c0a0e6abe4cd 100644
--- a/mm/page_alloc.h
+++ b/mm/page_alloc.h
@@ -56,6 +56,11 @@
  * alloc_tag_sub_check().
  */
 #define ALLOC_NO_CODETAG       0x1000
+#ifdef CONFIG_NUMA
+#define ALLOC_ZONELIST_PRIVATE 0x2000	/* use the local private zonelist */
+#else
+#define ALLOC_ZONELIST_PRIVATE 0x0
+#endif
 
 /* Flags that allow allocations below the min watermark. */
 #define ALLOC_RESERVES (ALLOC_MIN_RESERVE|ALLOC_HIGHATOMIC|ALLOC_OOM)
@@ -100,6 +105,16 @@ struct alloc_context {
 	unsigned int reclaim_compact_order;
 };
 
+static inline struct zonelist *
+select_zonelist(int nid, gfp_t gfp, unsigned int alloc_flags)
+{
+#ifdef CONFIG_NUMA
+	if (alloc_flags & ALLOC_ZONELIST_PRIVATE)
+		return &NODE_DATA(nid)->node_zonelists[ZONELIST_PRIVATE];
+#endif
+	return node_zonelist(nid, gfp);
+}
+
 /*
  * This function returns the order of a free page in the buddy system. In
  * general, page_zone(page)->lock must be held by the caller to prevent the

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 41/60] mm/page_alloc: relax a private request whose node becomes ineligible
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (39 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 40/60] mm/page_alloc: let MM callers select the local private zonelist Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:32   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 42/60] mm: fail private placement on exhaustion instead of OOM killing Gregory Price via B4 Relay
                   ` (19 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

A private request allocates only from its node's private zonelist. If
the cpuset revokes the node, or memory hotplug removes it, the request
has nowhere left to allocate, and a faulting task retries forever.

An ordinary bind policy does not hit this. A cpuset change rebinds it,
and check_retry_cpuset() drops a nodemask that no longer intersects the
cpuset. A private binding is never rebound, and dropping its nodemask
does not help, since the private zonelist still holds only its node.

Add relax_private_alloc() as the last of the slowpath retry checks.
When no node in a private request's zonelist is in its nodemask and
allowed by the cpuset, it drops ALLOC_ZONELIST_PRIVATE and the nodemask
and switches to the common zonelist of the preferred node, which
alloc_context now records. It runs after check_retry_cpuset() and
check_retry_zonelist(), so a concurrent update retries with the original
constraints instead.

A node counts in any zone type, so a GFP zone restriction or an
exhausted node never relaxes a request. A __GFP_THISNODE private request
is never relaxed. check_retry_cpuset() leaves a private request's
nodemask alone.

The alloc_context comment now lists nodemask and alloc_flags among the
fields the slowpath may change.

Without private nodes, the slowpath retry checks are unchanged.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/page_alloc.c | 60 +++++++++++++++++++++++++++++++++++++++++++++++++++++----
 mm/page_alloc.h |  7 ++++---
 2 files changed, 60 insertions(+), 7 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index a219057931c3..63cc8729c41a 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -4078,6 +4078,35 @@ __alloc_pages_cpuset_fallback(gfp_t gfp_mask, unsigned int order,
 	return page;
 }
 
+/* A request on the private zonelist. */
+static bool alloc_is_private(const struct alloc_context *ac)
+{
+	return ac->alloc_flags & ALLOC_ZONELIST_PRIVATE;
+}
+
+/*
+ * Whether a private request can still allocate: some node in its zonelist is
+ * in its nodemask and allowed by the cpuset. Any zone type counts, so neither a
+ * GFP zone restriction nor an exhausted node makes the request unsatisfiable.
+ */
+static bool alloc_private_reachable(gfp_t gfp, const struct alloc_context *ac)
+{
+	struct zoneref *z;
+	struct zone *zone;
+
+	/* A strict local private request keeps its allocation domain. */
+	if (gfp & __GFP_THISNODE)
+		return true;
+
+	for_each_zone_zonelist_nodemask(zone, z, ac->zonelist, MAX_NR_ZONES - 1,
+					ac->nodemask) {
+		if (!cpusets_enabled() ||
+		    node_isset(zone_to_nid(zone), cpuset_current_mems_allowed))
+			return true;
+	}
+	return false;
+}
+
 static inline struct page *
 __alloc_pages_may_oom(gfp_t gfp_mask, unsigned int order,
 	const struct alloc_context *ac, unsigned long *did_some_progress)
@@ -4753,8 +4782,10 @@ check_retry_cpuset(int cpuset_mems_cookie, struct alloc_context *ac)
 	 * from MPOL_BIND mempolicy (whose documented semantics is to be ignored
 	 * when it does not intersect with the cpuset restrictions) or the
 	 * caller can deal with a violated nodemask.
+	 *
+	 * relax_private_alloc() handles a private request instead.
 	 */
-	if (cpusets_enabled() && ac->nodemask &&
+	if (cpusets_enabled() && ac->nodemask && !alloc_is_private(ac) &&
 			!cpuset_nodemask_valid_mems_allowed(ac->nodemask)) {
 		ac->nodemask = NULL;
 		return true;
@@ -4773,6 +4804,23 @@ check_retry_cpuset(int cpuset_mems_cookie, struct alloc_context *ac)
 	return false;
 }
 
+/*
+ * Relax a private request that can no longer allocate, because the cpuset
+ * revoked its node or memory hotplug removed it. Drop the nodemask and allocate
+ * from the common zonelist of the preferred node. Called after
+ * check_retry_cpuset() and check_retry_zonelist() found no concurrent update.
+ */
+static bool relax_private_alloc(gfp_t gfp, struct alloc_context *ac)
+{
+	if (!alloc_is_private(ac) || alloc_private_reachable(gfp, ac))
+		return false;
+
+	ac->nodemask = NULL;
+	ac->alloc_flags &= ~ALLOC_ZONELIST_PRIVATE;
+	ac->zonelist = node_zonelist(ac->preferred_nid, gfp);
+	return true;
+}
+
 static void check_alloc_stall_warn(gfp_t gfp_mask, const nodemask_t *nodemask,
 				unsigned int order, unsigned long alloc_start_time)
 {
@@ -5060,7 +5108,8 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
 	 * unless it can execute just once.
 	 */
 	if (check_retry_cpuset(cpuset_mems_cookie, ac) ||
-	    check_retry_zonelist(zonelist_iter_cookie))
+	    check_retry_zonelist(zonelist_iter_cookie) ||
+	    relax_private_alloc(gfp_mask, ac))
 		goto restart;
 
 	if (should_reclaim_retry(gfp_mask, order, ac, alloc_flags,
@@ -5095,7 +5144,8 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
 	 * a unnecessary OOM kill.
 	 */
 	if (check_retry_cpuset(cpuset_mems_cookie, ac) ||
-	    check_retry_zonelist(zonelist_iter_cookie))
+	    check_retry_zonelist(zonelist_iter_cookie) ||
+	    relax_private_alloc(gfp_mask, ac))
 		goto restart;
 
 	/* Reclaim has failed us, start killing things */
@@ -5121,7 +5171,8 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
 	 * a unnecessary OOM kill.
 	 */
 	if (check_retry_cpuset(cpuset_mems_cookie, ac) ||
-	    check_retry_zonelist(zonelist_iter_cookie))
+	    check_retry_zonelist(zonelist_iter_cookie) ||
+	    relax_private_alloc(gfp_mask, ac))
 		goto restart;
 
 	/*
@@ -5172,6 +5223,7 @@ static inline bool prepare_alloc_pages(gfp_t gfp_mask, unsigned int order,
 		unsigned int *alloc_flags)
 {
 	ac->highest_zoneidx = gfp_zone(gfp_mask);
+	ac->preferred_nid = preferred_nid;
 	ac->zonelist = select_zonelist(preferred_nid, gfp_mask, *alloc_flags);
 	ac->nodemask = nodemask;
 	ac->migratetype = gfp_migratetype(gfp_mask);
diff --git a/mm/page_alloc.h b/mm/page_alloc.h
index c0a0e6abe4cd..3dd73ce641b5 100644
--- a/mm/page_alloc.h
+++ b/mm/page_alloc.h
@@ -73,11 +73,11 @@
  * between functions involved in allocations, including the alloc_pages*
  * family of functions.
  *
- * nodemask, migratetype and highest_zoneidx are initialized only once in
+ * migratetype, preferred_nid and highest_zoneidx are initialized only once in
  * __alloc_pages() and then never change.
  *
- * zonelist, preferred_zone and highest_zoneidx are set first in
- * __alloc_pages() for the fast path, and might be later changed
+ * zonelist, nodemask, preferred_zone, highest_zoneidx and alloc_flags are set
+ * first in __alloc_pages() for the fast path, and might be later changed
  * in __alloc_pages_slowpath(). All other functions pass the whole structure
  * by a const pointer.
  */
@@ -85,6 +85,7 @@ struct alloc_context {
 	struct zonelist *zonelist;
 	const nodemask_t *nodemask;
 	struct zoneref *preferred_zoneref;
+	int preferred_nid;
 	int migratetype;
 
 	/*

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 42/60] mm: fail private placement on exhaustion instead of OOM killing
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (40 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 41/60] mm/page_alloc: relax a private request whose node becomes ineligible Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:40   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 43/60] mm/page_alloc: fall back when a bind policy's nodes become private Gregory Price via B4 Relay
                   ` (18 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Private memory belongs to a device or specialized service and is a fixed
pool, like the HugeTLB pool. The OOM killer has no good answer for it:
the task that faults the last page may not be the one that filled the
node, a remote accessor or another mapper of a shared object would be
blamed, and killing a task does not free shared private pages.

When reclaim cannot satisfy an allocation from a private zonelist, fail
it instead of OOMing.
  - __alloc_pages_may_oom() does not select a victim for a private
    request, and marks the task.
  - handle_mm_fault() turns VM_FAULT_OOM from such a failure into
    VM_FAULT_SIGBUS. GUP and remote access see -EFAULT, and other
    callers see the allocation failure, e.g. -ENOMEM from write(2).

A request that became ineligible since the last retry check is not
exhausted. Report progress so the slowpath retries and relaxes it.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 include/linux/sched.h |  2 ++
 mm/memory.c           |  5 +++++
 mm/page_alloc.c       | 14 ++++++++++++++
 3 files changed, 21 insertions(+)

diff --git a/include/linux/sched.h b/include/linux/sched.h
index e4f6323d7855..8a2e37475e5d 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -1038,6 +1038,8 @@ struct task_struct {
 	/* whether the LRU algorithm may apply to this access */
 	unsigned			in_lru_fault:1;
 #endif
+	/* A private memory allocation failed; a fault reports SIGBUS. */
+	unsigned			private_alloc_failed:1;
 #ifdef CONFIG_COMPAT_BRK
 	unsigned			brk_randomized:1;
 #endif
diff --git a/mm/memory.c b/mm/memory.c
index 282266dd5e5d..4a8a08df28ff 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -7003,6 +7003,7 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address,
 	}
 
 	is_droppable = !!(vma->vm_flags & VM_DROPPABLE);
+	current->private_alloc_failed = 0;
 
 	/*
 	 * Enable the memcg OOM handling for faults triggered in user
@@ -7034,6 +7035,10 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address,
 	if (is_droppable && (flags & FAULT_FLAG_USER))
 		ret &= ~VM_FAULT_OOM;
 
+	/* Exhausted private memory fails like an empty HugeTLB pool. */
+	if (unlikely(current->private_alloc_failed) && (ret & VM_FAULT_OOM))
+		ret = (ret & ~VM_FAULT_OOM) | VM_FAULT_SIGBUS;
+
 	if (flags & FAULT_FLAG_USER) {
 		mem_cgroup_exit_user_fault();
 		/*
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 63cc8729c41a..ea48991ef589 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -4176,6 +4176,20 @@ __alloc_pages_may_oom(gfp_t gfp_mask, unsigned int order,
 	 * failures more gracefully we should just bail out here.
 	 */
 
+	/*
+	 * Private memory is a fixed pool, exhausting it fails the allocation
+	 * rather than selecting an OOM victim. Otherwise, if there is some
+	 * indication of progress because of a revoked request, just report
+	 * some progress so the slowpath retries and relaxes it.
+	 */
+	if (ac->alloc_flags & ALLOC_ZONELIST_PRIVATE) {
+		if (!alloc_private_reachable(gfp_mask, ac))
+			*did_some_progress = 1;
+		else
+			current->private_alloc_failed = 1;
+		goto out;
+	}
+
 	/* Exhausted what can be done so it's blame time */
 	if (out_of_memory(&oc) ||
 	    WARN_ON_ONCE_GFP(gfp_mask & __GFP_NOFAIL, gfp_mask)) {

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 43/60] mm/page_alloc: fall back when a bind policy's nodes become private
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (41 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 42/60] mm: fail private placement on exhaustion instead of OOM killing Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:34   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 44/60] mm: add N_MEMORY_COMPACTION node state Gregory Price via B4 Relay
                   ` (17 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Memory hotplug can make a node private after a bind policy naming it
was created. A static bind policy, a shared memory policy, or any bind
policy without cpusets is not rebound when the node's memory is
removed, so it still names the node when the node returns as private
memory.

The common zonelists hold no private node, so a policy naming only such
nodes leaves the allocation no zone to use. The nodemask still
intersects the cpuset, so check_retry_cpuset() keeps it, and a fault
through the policy is retried forever.

Let relax_private_alloc() treat a nodemask that names a private node
like a private request: once no node in the zonelist is in the nodemask
and allowed by the cpuset, drop the nodemask and allocate from common
memory. Only a request on the private zonelist keeps __GFP_THISNODE
strict.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/page_alloc.c | 31 +++++++++++++++++++------------
 1 file changed, 19 insertions(+), 12 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index ea48991ef589..b7339bc634d5 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -4078,16 +4078,20 @@ __alloc_pages_cpuset_fallback(gfp_t gfp_mask, unsigned int order,
 	return page;
 }
 
-/* A request on the private zonelist. */
+/* A private request, or a nodemask that names a private node. */
 static bool alloc_is_private(const struct alloc_context *ac)
 {
-	return ac->alloc_flags & ALLOC_ZONELIST_PRIVATE;
+	if (ac->alloc_flags & ALLOC_ZONELIST_PRIVATE)
+		return true;
+	return ac->nodemask &&
+	       nodes_intersects(*ac->nodemask, node_states[N_MEMORY_PRIVATE]);
 }
 
 /*
- * Whether a private request can still allocate: some node in its zonelist is
- * in its nodemask and allowed by the cpuset. Any zone type counts, so neither a
- * GFP zone restriction nor an exhausted node makes the request unsatisfiable.
+ * Whether a request that names private memory can still allocate: some node in
+ * its zonelist is in its nodemask and allowed by the cpuset. Any zone type
+ * counts, so neither a GFP zone restriction nor an exhausted node makes the
+ * request unsatisfiable.
  */
 static bool alloc_private_reachable(gfp_t gfp, const struct alloc_context *ac)
 {
@@ -4095,7 +4099,7 @@ static bool alloc_private_reachable(gfp_t gfp, const struct alloc_context *ac)
 	struct zone *zone;
 
 	/* A strict local private request keeps its allocation domain. */
-	if (gfp & __GFP_THISNODE)
+	if ((gfp & __GFP_THISNODE) && (ac->alloc_flags & ALLOC_ZONELIST_PRIVATE))
 		return true;
 
 	for_each_zone_zonelist_nodemask(zone, z, ac->zonelist, MAX_NR_ZONES - 1,
@@ -4819,10 +4823,11 @@ check_retry_cpuset(int cpuset_mems_cookie, struct alloc_context *ac)
 }
 
 /*
- * Relax a private request that can no longer allocate, because the cpuset
- * revoked its node or memory hotplug removed it. Drop the nodemask and allocate
- * from the common zonelist of the preferred node. Called after
- * check_retry_cpuset() and check_retry_zonelist() found no concurrent update.
+ * Relax a request that names private memory but can no longer allocate,
+ * because the cpuset revoked its node or memory hotplug removed it or made it
+ * private. Drop the nodemask and allocate from the common zonelist of the
+ * preferred node. Called after check_retry_cpuset() and check_retry_zonelist()
+ * found no concurrent update.
  */
 static bool relax_private_alloc(gfp_t gfp, struct alloc_context *ac)
 {
@@ -4830,8 +4835,10 @@ static bool relax_private_alloc(gfp_t gfp, struct alloc_context *ac)
 		return false;
 
 	ac->nodemask = NULL;
-	ac->alloc_flags &= ~ALLOC_ZONELIST_PRIVATE;
-	ac->zonelist = node_zonelist(ac->preferred_nid, gfp);
+	if (ac->alloc_flags & ALLOC_ZONELIST_PRIVATE) {
+		ac->alloc_flags &= ~ALLOC_ZONELIST_PRIVATE;
+		ac->zonelist = node_zonelist(ac->preferred_nid, gfp);
+	}
 	return true;
 }
 

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 44/60] mm: add N_MEMORY_COMPACTION node state
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (42 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 43/60] mm/page_alloc: fall back when a bind policy's nodes become private Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:24   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 45/60] mm/compaction: restrict compaction to N_MEMORY_COMPACTION Gregory Price via B4 Relay
                   ` (16 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Add N_MEMORY_COMPACTION for the nodes compaction may operate on, and
NODE_MEMORY_FEAT_COMPACTION for memory providers to select it.

Common memory carries every feature, so a mask with
NODE_MEMORY_FEAT_COMMON must now include compaction.

The state is internal and has no sysfs file. Internal states follow
N_GENERIC_INITIATOR, so check sysfs coverage only for the states up to
it.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 Documentation/mm/physical_memory.rst |  7 +++++++
 drivers/base/node.c                  |  5 +++--
 include/linux/nodemask.h             | 20 ++++++++++++++++----
 mm/page_alloc.c                      |  4 ++++
 4 files changed, 30 insertions(+), 6 deletions(-)

diff --git a/Documentation/mm/physical_memory.rst b/Documentation/mm/physical_memory.rst
index ddba7c890418..5d0a69f6c23c 100644
--- a/Documentation/mm/physical_memory.rst
+++ b/Documentation/mm/physical_memory.rst
@@ -182,6 +182,12 @@ nodes with particular properties as defined by ``enum node_states``:
 ``N_GENERIC_INITIATOR``
   The node has one or more Generic Initiators
 
+The following states are internal and not exported through sysfs:
+
+``N_MEMORY_COMPACTION``
+  The kernel compacts the node on its own initiative (kcompactd, proactive
+  and explicit compaction).
+
 For each node that has a property described above, the bit corresponding to the
 node ID in the ``node_states[<property>]`` bitmask is set.
 
@@ -194,6 +200,7 @@ For example, for node 2 with normal memory and CPUs, bit 2 will be set in ::
   node_states[N_MEMORY]
   node_states[N_MEMORY_COMMON]
   node_states[N_CPU]
+  node_states[N_MEMORY_COMPACTION]
 
 For various operations possible with nodemasks please refer to
 ``include/linux/nodemask.h``.
diff --git a/drivers/base/node.c b/drivers/base/node.c
index 56a1741a6a45..19f93cda90be 100644
--- a/drivers/base/node.c
+++ b/drivers/base/node.c
@@ -1000,8 +1000,9 @@ void __init node_dev_init(void)
 {
 	int ret, i;
 
- 	BUILD_BUG_ON(ARRAY_SIZE(node_state_attr) != NR_NODE_STATES);
- 	BUILD_BUG_ON(ARRAY_SIZE(node_state_attrs)-1 != NR_NODE_STATES);
+	/* Node states after N_GENERIC_INITIATOR have no sysfs file. */
+	BUILD_BUG_ON(ARRAY_SIZE(node_state_attr) != N_GENERIC_INITIATOR + 1);
+	BUILD_BUG_ON(ARRAY_SIZE(node_state_attrs)-1 != N_GENERIC_INITIATOR + 1);
 
 	ret = subsys_system_register(&node_subsys, cpu_root_attr_groups);
 	if (ret)
diff --git a/include/linux/nodemask.h b/include/linux/nodemask.h
index 02908ef23a62..2757acffff2b 100644
--- a/include/linux/nodemask.h
+++ b/include/linux/nodemask.h
@@ -411,6 +411,8 @@ enum node_states {
 	N_MEMORY_PRIVATE,	/* memory outside the common pool */
 	N_CPU,		/* The node has one or more cpus */
 	N_GENERIC_INITIATOR,	/* The node has one or more Generic Initiators */
+	/* Internal states, not exported through sysfs */
+	N_MEMORY_COMPACTION,	/* mm compaction may operate on the node */
 	NR_NODE_STATES
 };
 
@@ -542,16 +544,26 @@ static __always_inline int node_random(const nodemask_t *maskp)
 #define for_each_online_node(node) for_each_node_state(node, N_ONLINE)
 #define for_each_node_with_cpus(node)	for_each_node_state(node, N_CPU)
 
-/* Private nodes omit NODE_MEMORY_FEAT_COMMON. */
+/*
+ * Private nodes omit NODE_MEMORY_FEAT_COMMON. Their providers opt into
+ * individual MM services with the remaining bits.
+ */
 #define NODE_MEMORY_FEAT_COMMON		BIT(0) /* add to fallback zonelist */
-#define NODE_MEMORY_FEAT_ALL		NODE_MEMORY_FEAT_COMMON
+#define NODE_MEMORY_FEAT_COMPACTION	BIT(1) /* mm compaction */
+#define NODE_MEMORY_FEAT_ALL		(NODE_MEMORY_FEAT_COMMON | \
+					 NODE_MEMORY_FEAT_COMPACTION)
 
-/* Private memory needs NUMA: node 0 holds all memory without it. */
+/*
+ * Common memory carries every feature. Private memory needs NUMA: node 0
+ * holds all memory without it.
+ */
 static inline bool node_memory_features_valid(unsigned long features)
 {
 	if (features & ~NODE_MEMORY_FEAT_ALL)
 		return false;
-	return IS_ENABLED(CONFIG_NUMA) || (features & NODE_MEMORY_FEAT_COMMON);
+	if (features & NODE_MEMORY_FEAT_COMMON)
+		return features == NODE_MEMORY_FEAT_ALL;
+	return IS_ENABLED(CONFIG_NUMA);
 }
 
 /*
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index b7339bc634d5..335ce73e645e 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -204,6 +204,7 @@ nodemask_t node_states[NR_NODE_STATES] __read_mostly = {
 #endif
 	[N_MEMORY] = { { [0] = 1UL } },
 	[N_MEMORY_COMMON] = { { [0] = 1UL } },
+	[N_MEMORY_COMPACTION] = { { [0] = 1UL } },
 	[N_CPU] = { { [0] = 1UL } },
 #endif	/* NUMA */
 };
@@ -218,10 +219,13 @@ void __meminit node_set_memory_state(int nid, unsigned long features)
 		node_set_state(nid, N_MEMORY_COMMON);
 	else
 		node_set_state(nid, N_MEMORY_PRIVATE);
+	if (features & NODE_MEMORY_FEAT_COMPACTION)
+		node_set_state(nid, N_MEMORY_COMPACTION);
 }
 
 void __meminit node_clear_memory_state(int nid)
 {
+	node_clear_state(nid, N_MEMORY_COMPACTION);
 	node_clear_state(nid, N_MEMORY_PRIVATE);
 	node_clear_state(nid, N_MEMORY_COMMON);
 	node_clear_state(nid, N_MEMORY);

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 45/60] mm/compaction: restrict compaction to N_MEMORY_COMPACTION
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (43 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 44/60] mm: add N_MEMORY_COMPACTION node state Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:30   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 46/60] mm: add N_MEMORY_RECLAIM node state Gregory Price via B4 Relay
                   ` (15 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Restrict compaction to N_MEMORY_COMPACTION instead of common memory:
  - compact N_MEMORY_COMPACTION nodes for /proc/sys/vm/compact_memory.
  - start kcompactd and trigger proactive compaction on those nodes.

Direct compaction is unchanged: an allocation compacts the zones on its
zonelist, and a private node's zones are only on its private zonelist.

nodeN/compact fails with -EOPNOTSUPP for private nodes without the
state.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/compaction.c     | 9 +++++----
 mm/memory_hotplug.c | 2 +-
 2 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/mm/compaction.c b/mm/compaction.c
index 283d4d4e78c3..6835ddb91f29 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -2973,7 +2973,7 @@ static int compact_nodes(void)
 	/* Flush pending updates to the LRU lists */
 	lru_add_drain_all();
 
-	for_each_node_state(nid, N_MEMORY_COMMON) {
+	for_each_node_state(nid, N_MEMORY_COMPACTION) {
 		ret = compact_node(NODE_DATA(nid), false);
 		if (ret)
 			return ret;
@@ -2992,7 +2992,7 @@ static int compaction_proactiveness_sysctl_handler(const struct ctl_table *table
 		return rc;
 
 	if (write && sysctl_compaction_proactiveness) {
-		for_each_node_state(nid, N_MEMORY_COMMON) {
+		for_each_node_state(nid, N_MEMORY_COMPACTION) {
 			pg_data_t *pgdat = NODE_DATA(nid);
 
 			if (pgdat->proactive_compact_trigger)
@@ -3038,7 +3038,8 @@ static ssize_t compact_store(struct device *dev,
 	int nid = dev->id;
 
 	if (nid >= 0 && nid < nr_node_ids && node_online(nid)) {
-		if (node_state(nid, N_MEMORY_PRIVATE))
+		if (node_state(nid, N_MEMORY) &&
+		    !node_state(nid, N_MEMORY_COMPACTION))
 			return -EOPNOTSUPP;
 
 		/* Flush pending updates to the LRU lists */
@@ -3369,7 +3370,7 @@ static int __init kcompactd_init(void)
 {
 	int nid;
 
-	for_each_node_state(nid, N_MEMORY_COMMON)
+	for_each_node_state(nid, N_MEMORY_COMPACTION)
 		kcompactd_run(nid);
 	register_sysctl_init("vm", vm_compaction);
 	return 0;
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index d7ce60c6815f..1cb10c88954e 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -1232,7 +1232,7 @@ int online_pages(unsigned long pfn, unsigned long nr_pages,
 
 	if (node_state(nid, N_MEMORY_COMMON))
 		kswapd_run(nid);
-	if (node_state(nid, N_MEMORY_COMMON))
+	if (node_state(nid, N_MEMORY_COMPACTION))
 		kcompactd_run(nid);
 
 	if (node_arg.nid >= 0)

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 46/60] mm: add N_MEMORY_RECLAIM node state
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (44 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 45/60] mm/compaction: restrict compaction to N_MEMORY_COMPACTION Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:25   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 47/60] mm/vmscan: restrict reclaim to N_MEMORY_RECLAIM instead of N_MEMORY_COMMON Gregory Price via B4 Relay
                   ` (14 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Add N_MEMORY_RECLAIM as an internal state for the nodes page reclaim
may operate on, and NODE_MEMORY_FEAT_RECLAIM for memory providers to
select it.

Reclaim frees order-0 pages and leaves high-order requests to
compaction, so a private node that permits reclaim must permit
compaction too.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 Documentation/mm/physical_memory.rst |  4 ++++
 include/linux/nodemask.h             | 11 +++++++++--
 mm/page_alloc.c                      |  4 ++++
 3 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/Documentation/mm/physical_memory.rst b/Documentation/mm/physical_memory.rst
index 5d0a69f6c23c..1bafe8c085a3 100644
--- a/Documentation/mm/physical_memory.rst
+++ b/Documentation/mm/physical_memory.rst
@@ -187,6 +187,9 @@ The following states are internal and not exported through sysfs:
 ``N_MEMORY_COMPACTION``
   The kernel compacts the node on its own initiative (kcompactd, proactive
   and explicit compaction).
+``N_MEMORY_RECLAIM``
+  The kernel reclaims the node on its own initiative (kswapd, memcg and
+  explicit reclaim).
 
 For each node that has a property described above, the bit corresponding to the
 node ID in the ``node_states[<property>]`` bitmask is set.
@@ -201,6 +204,7 @@ For example, for node 2 with normal memory and CPUs, bit 2 will be set in ::
   node_states[N_MEMORY_COMMON]
   node_states[N_CPU]
   node_states[N_MEMORY_COMPACTION]
+  node_states[N_MEMORY_RECLAIM]
 
 For various operations possible with nodemasks please refer to
 ``include/linux/nodemask.h``.
diff --git a/include/linux/nodemask.h b/include/linux/nodemask.h
index 2757acffff2b..be5edbee2b22 100644
--- a/include/linux/nodemask.h
+++ b/include/linux/nodemask.h
@@ -413,6 +413,7 @@ enum node_states {
 	N_GENERIC_INITIATOR,	/* The node has one or more Generic Initiators */
 	/* Internal states, not exported through sysfs */
 	N_MEMORY_COMPACTION,	/* mm compaction may operate on the node */
+	N_MEMORY_RECLAIM,	/* mm reclaim may operate on the node */
 	NR_NODE_STATES
 };
 
@@ -550,12 +551,15 @@ static __always_inline int node_random(const nodemask_t *maskp)
  */
 #define NODE_MEMORY_FEAT_COMMON		BIT(0) /* add to fallback zonelist */
 #define NODE_MEMORY_FEAT_COMPACTION	BIT(1) /* mm compaction */
+#define NODE_MEMORY_FEAT_RECLAIM	BIT(2) /* mm reclaim */
 #define NODE_MEMORY_FEAT_ALL		(NODE_MEMORY_FEAT_COMMON | \
-					 NODE_MEMORY_FEAT_COMPACTION)
+					 NODE_MEMORY_FEAT_COMPACTION | \
+					 NODE_MEMORY_FEAT_RECLAIM)
 
 /*
  * Common memory carries every feature. Private memory needs NUMA: node 0
- * holds all memory without it.
+ * holds all memory without it. Reclaim leaves high-order requests to
+ * compaction, so it requires compaction.
  */
 static inline bool node_memory_features_valid(unsigned long features)
 {
@@ -563,6 +567,9 @@ static inline bool node_memory_features_valid(unsigned long features)
 		return false;
 	if (features & NODE_MEMORY_FEAT_COMMON)
 		return features == NODE_MEMORY_FEAT_ALL;
+	if ((features & NODE_MEMORY_FEAT_RECLAIM) &&
+	    !(features & NODE_MEMORY_FEAT_COMPACTION))
+		return false;
 	return IS_ENABLED(CONFIG_NUMA);
 }
 
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 335ce73e645e..864da908a59b 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -205,6 +205,7 @@ nodemask_t node_states[NR_NODE_STATES] __read_mostly = {
 	[N_MEMORY] = { { [0] = 1UL } },
 	[N_MEMORY_COMMON] = { { [0] = 1UL } },
 	[N_MEMORY_COMPACTION] = { { [0] = 1UL } },
+	[N_MEMORY_RECLAIM] = { { [0] = 1UL } },
 	[N_CPU] = { { [0] = 1UL } },
 #endif	/* NUMA */
 };
@@ -221,10 +222,13 @@ void __meminit node_set_memory_state(int nid, unsigned long features)
 		node_set_state(nid, N_MEMORY_PRIVATE);
 	if (features & NODE_MEMORY_FEAT_COMPACTION)
 		node_set_state(nid, N_MEMORY_COMPACTION);
+	if (features & NODE_MEMORY_FEAT_RECLAIM)
+		node_set_state(nid, N_MEMORY_RECLAIM);
 }
 
 void __meminit node_clear_memory_state(int nid)
 {
+	node_clear_state(nid, N_MEMORY_RECLAIM);
 	node_clear_state(nid, N_MEMORY_COMPACTION);
 	node_clear_state(nid, N_MEMORY_PRIVATE);
 	node_clear_state(nid, N_MEMORY_COMMON);

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 47/60] mm/vmscan: restrict reclaim to N_MEMORY_RECLAIM instead of N_MEMORY_COMMON
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (45 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 46/60] mm: add N_MEMORY_RECLAIM node state Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:38   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 48/60] mm/madvise: restrict reclaim advice to reclaim-capable memory Gregory Price via B4 Relay
                   ` (13 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

kswapd and the reclaim interfaces currently serve only common memory. A
private node that opts into reclaim should get them too.

Restrict reclaim to N_MEMORY_RECLAIM:
  - start kswapd only on N_MEMORY_RECLAIM nodes.
  - walk N_MEMORY_RECLAIM nodes in MGLRU's debugfs interface.
  - reject nodeN/reclaim for nodes without N_MEMORY_RECLAIM.

Direct reclaim is unchanged: an allocation reclaims the zones on its
zonelist, and a private node's zones are only on its private zonelist.

Private nodes that permit reclaim get watermarks again (the capped
minimum that highmem and movable zones get) and are boosted, since
their kswapd restores both.

Private nodes without the state keep zero watermarks.

The hopeless-kswapd recovery on page free exists because allocations
that fall back to another node skip the direct reclaim that would clear
the state. Only common memory offers that fallback, so apply the
recovery only to common nodes now that private nodes run kswapd.

Leave private zones out of si_mem_available()'s low watermark sum,
which bounds the common page cache and slab that must stay.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/memory_hotplug.c |  2 +-
 mm/page_alloc.c     | 11 +++++++----
 mm/show_mem.c       |  7 +++++--
 mm/vmscan.c         | 14 +++++++-------
 4 files changed, 20 insertions(+), 14 deletions(-)

diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 1cb10c88954e..8146f77919d5 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -1230,7 +1230,7 @@ int online_pages(unsigned long pfn, unsigned long nr_pages,
 	/* reinitialise watermarks and update pcp limits */
 	init_per_zone_wmark_min();
 
-	if (node_state(nid, N_MEMORY_COMMON))
+	if (node_state(nid, N_MEMORY_RECLAIM))
 		kswapd_run(nid);
 	if (node_state(nid, N_MEMORY_COMPACTION))
 		kcompactd_run(nid);
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 864da908a59b..cc70e1b45322 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -2972,6 +2972,7 @@ static bool free_frozen_page_commit(struct zone *zone,
 		 * kswapd work again by resetting kswapd_failures.
 		 */
 		if (kswapd_test_hopeless(pgdat) &&
+		    node_state(pgdat->node_id, N_MEMORY_COMMON) &&
 		    num_node_state(N_MEMORY_COMMON) > 1)
 			kswapd_clear_hopeless(pgdat, KSWAPD_CLEAR_HOPELESS_PCP);
 	}
@@ -6814,8 +6815,8 @@ static void __setup_per_zone_wmarks(void)
 		u64 tmp;
 
 		spin_lock_irqsave(&zone->lock, flags);
-		/* No kswapd runs for a private node to restore watermarks. */
-		if (node_state(zone_to_nid(zone), N_MEMORY_PRIVATE)) {
+		/* Nodes that reclaim does not serve carry no watermarks. */
+		if (!node_state(zone_to_nid(zone), N_MEMORY_RECLAIM)) {
 			zone->_watermark[WMARK_MIN] = 0;
 			zone->_watermark[WMARK_LOW] = 0;
 			zone->_watermark[WMARK_HIGH] = 0;
@@ -6826,11 +6827,13 @@ static void __setup_per_zone_wmarks(void)
 		}
 		tmp = (u64)pages_min * zone_managed_pages(zone);
 		tmp = div64_ul(tmp, lowmem_pages);
-		if (is_highmem(zone) || zone_idx(zone) == ZONE_MOVABLE) {
+		if (is_highmem(zone) || zone_idx(zone) == ZONE_MOVABLE ||
+		    node_state(zone_to_nid(zone), N_MEMORY_PRIVATE)) {
 			/*
 			 * __GFP_HIGH and PF_MEMALLOC allocations usually don't
 			 * need highmem and movable zones pages, so cap pages_min
-			 * to a small  value here.
+			 * to a small  value here. Private memory holds no system
+			 * reserve, only this small minimum.
 			 *
 			 * The WMARK_HIGH-WMARK_LOW and (WMARK_LOW-WMARK_MIN)
 			 * deltas control async page reclaim, and so should
diff --git a/mm/show_mem.c b/mm/show_mem.c
index b90fb5a0016b..24dd33ba783b 100644
--- a/mm/show_mem.c
+++ b/mm/show_mem.c
@@ -38,8 +38,11 @@ long si_mem_available(void)
 	unsigned long reclaimable;
 	struct zone *zone;
 
-	for_each_zone(zone)
-		wmark_low += low_wmark_pages(zone);
+	/* Only common zones bound the page cache and slab that must stay. */
+	for_each_zone(zone) {
+		if (!node_state(zone_to_nid(zone), N_MEMORY_PRIVATE))
+			wmark_low += low_wmark_pages(zone);
+	}
 
 	/*
 	 * Estimate the amount of memory available for userspace allocations,
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 3e934135e805..03d0919d11be 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -5729,7 +5729,7 @@ static void *lru_gen_seq_start(struct seq_file *m, loff_t *pos)
 	do {
 		int nid;
 
-		for_each_node_state(nid, N_MEMORY_COMMON) {
+		for_each_node_state(nid, N_MEMORY_RECLAIM) {
 			if (!nr_to_skip--)
 				return mem_cgroup_lruvec(memcg, NODE_DATA(nid));
 		}
@@ -5754,13 +5754,13 @@ static void *lru_gen_seq_next(struct seq_file *m, void *v, loff_t *pos)
 
 	++*pos;
 
-	nid = next_node_state(nid, N_MEMORY_COMMON);
+	nid = next_node_state(nid, N_MEMORY_RECLAIM);
 	if (nid == MAX_NUMNODES) {
 		memcg = mem_cgroup_iter(NULL, memcg, NULL);
 		if (!memcg)
 			return NULL;
 
-		nid = first_node_state(N_MEMORY_COMMON);
+		nid = first_node_state(N_MEMORY_RECLAIM);
 	}
 
 	return mem_cgroup_lruvec(memcg, NODE_DATA(nid));
@@ -5832,7 +5832,7 @@ static int lru_gen_seq_show(struct seq_file *m, void *v)
 	DEFINE_MAX_SEQ(lruvec);
 	DEFINE_MIN_SEQ(lruvec);
 
-	if (nid == first_node_state(N_MEMORY_COMMON)) {
+	if (nid == first_node_state(N_MEMORY_RECLAIM)) {
 		const char *path = memcg ? m->private : "";
 
 #ifdef CONFIG_MEMCG
@@ -5935,7 +5935,7 @@ static int run_cmd(char cmd, u64 memcg_id, int nid, unsigned long seq,
 	int err = -EINVAL;
 	struct mem_cgroup *memcg = NULL;
 
-	if (nid < 0 || nid >= MAX_NUMNODES || !node_state(nid, N_MEMORY_COMMON))
+	if (nid < 0 || nid >= MAX_NUMNODES || !node_state(nid, N_MEMORY_RECLAIM))
 		return -EINVAL;
 
 	if (!mem_cgroup_disabled()) {
@@ -7907,7 +7907,7 @@ static int __init kswapd_init(void)
 {
 	int nid;
 
-	for_each_node_state(nid, N_MEMORY_COMMON)
+	for_each_node_state(nid, N_MEMORY_RECLAIM)
  		kswapd_run(nid);
 	register_sysctl_init("vm", vmscan_sysctl_table);
 	return 0;
@@ -8268,7 +8268,7 @@ static ssize_t reclaim_store(struct device *dev,
 {
 	int ret, nid = dev->id;
 
-	if (node_state(nid, N_MEMORY_PRIVATE))
+	if (node_state(nid, N_MEMORY) && !node_state(nid, N_MEMORY_RECLAIM))
 		return -EOPNOTSUPP;
 
 	ret = user_proactive_reclaim((char *)buf, NULL, NODE_DATA(nid));

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 48/60] mm/madvise: restrict reclaim advice to reclaim-capable memory
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (46 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 47/60] mm/vmscan: restrict reclaim to N_MEMORY_RECLAIM instead of N_MEMORY_COMMON Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:29   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 49/60] mm/vmscan: reclaim private nodes for memcg Gregory Price via B4 Relay
                   ` (12 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

MADV_COLD, MADV_PAGEOUT and MADV_FREE reclaim or age folios directly,
bypassing the node filter reclaim uses, so they act only on common
memory.

Add folio_allows_reclaim(), which requires the folio's node to permit
reclaim, and use it instead for those advices, including their
huge-page paths.

No behavior change to common nodes.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/huge_memory.c | 2 +-
 mm/internal.h    | 7 +++++++
 mm/madvise.c     | 6 +++---
 3 files changed, 11 insertions(+), 4 deletions(-)

diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index cae28513b121..00c72b91ed30 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2452,7 +2452,7 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,
 
 	folio = pmd_folio(orig_pmd);
 
-	if (!folio_is_common_memory(folio))
+	if (!folio_allows_reclaim(folio))
 		goto out;
 
 	/*
diff --git a/mm/internal.h b/mm/internal.h
index 9c162ded5305..fa1ae221564e 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -80,6 +80,13 @@ static inline bool page_is_common_memory(struct page *page)
 	return folio_is_common_memory(page_folio(page));
 }
 
+/* May reclaim act on this folio's memory? */
+static inline bool folio_allows_reclaim(struct folio *folio)
+{
+	return !folio_is_zone_device(folio) &&
+	       node_state(folio_nid(folio), N_MEMORY_RECLAIM);
+}
+
 /* mm/vmscan.c */
 unsigned long zone_reclaimable_pages(struct zone *zone);
 unsigned long try_to_free_pages(struct zonelist *zonelist, int order,
diff --git a/mm/madvise.c b/mm/madvise.c
index 0f1648526f15..a5fa3b948a20 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -404,7 +404,7 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,
 		if (!folio)
 			goto huge_unlock;
 
-		if (!folio_is_common_memory(folio))
+		if (!folio_allows_reclaim(folio))
 			goto huge_unlock;
 
 		/* Do not interfere with other mappings of this folio */
@@ -485,7 +485,7 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,
 			continue;
 
 		folio = vm_normal_folio(vma, addr, ptent);
-		if (!folio || !folio_is_common_memory(folio))
+		if (!folio || !folio_allows_reclaim(folio))
 			continue;
 
 		/*
@@ -715,7 +715,7 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr,
 		}
 
 		folio = vm_normal_folio(vma, addr, ptent);
-		if (!folio || !folio_is_common_memory(folio))
+		if (!folio || !folio_allows_reclaim(folio))
 			continue;
 
 		/*

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 49/60] mm/vmscan: reclaim private nodes for memcg
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (47 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 48/60] mm/madvise: restrict reclaim advice to reclaim-capable memory Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:31   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 50/60] mm: add N_MEMORY_MEMPOLICY node state Gregory Price via B4 Relay
                   ` (11 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Memcg reclaim does not serve one allocation, so it should cover every
node that permits reclaim. It walks the current node's fallback
zonelist, which holds no private node, so a private node that permits
reclaim is never scanned for a cgroup over its limit.

After the zonelist, visit the nodes that permit reclaim and that the
zonelist did not cover, and give them the same refault snapshot and
congestion reset. The zonelist keeps setting the order, which matters
because memcg reclaim stops once it has freed enough.

Direct reclaim for an allocation still walks only that allocation's
zonelist. A private allocation's zonelist covers its own node, and
reclaiming private memory cannot satisfy a common allocation.

Without private nodes the zonelist covers every node that permits
reclaim, so behavior is unchanged.

Another way to do this would be to implement a reclaim zonelist that
only reclaim uses, but this implementation does not carry the static
per-node size overhead that adding another zonelist does.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/vmscan.c | 43 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 42 insertions(+), 1 deletion(-)

diff --git a/mm/vmscan.c b/mm/vmscan.c
index 03d0919d11be..49ec85ed00e0 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -6636,6 +6636,21 @@ static void consider_reclaim_throttle(pg_data_t *pgdat, struct scan_control *sc)
 		reclaim_throttle(pgdat, VMSCAN_THROTTLE_NOPROGRESS);
 }
 
+/* Whether shrink_zones() reaches @nid through @zonelist. */
+static bool zonelist_reaches_node(struct zonelist *zonelist, int nid,
+				  struct scan_control *sc)
+{
+	struct zoneref *z;
+	struct zone *zone;
+
+	for_each_zone_zonelist_nodemask(zone, z, zonelist, sc->reclaim_idx,
+					sc->nodemask) {
+		if (zone_to_nid(zone) == nid)
+			return true;
+	}
+	return false;
+}
+
 /*
  * This is the direct reclaim path, for page-allocating processes.  We only
  * try to reclaim pages from zones which will satisfy the caller's allocation
@@ -6651,6 +6666,7 @@ static void shrink_zones(struct zonelist *zonelist, struct scan_control *sc)
 	gfp_t orig_mask;
 	pg_data_t *last_pgdat = NULL;
 	pg_data_t *first_pgdat = NULL;
+	int nid;
 
 	/*
 	 * If the number of buffer_heads in the machine exceeds the maximum
@@ -6706,6 +6722,14 @@ static void shrink_zones(struct zonelist *zonelist, struct scan_control *sc)
 		shrink_node(zone->zone_pgdat, sc);
 	}
 
+	/* A cgroup's memory may be on nodes that are not on the zonelist. */
+	if (cgroup_reclaim(sc)) {
+		for_each_node_state(nid, N_MEMORY_RECLAIM) {
+			if (!zonelist_reaches_node(zonelist, nid, sc))
+				shrink_node(NODE_DATA(nid), sc);
+		}
+	}
+
 	if (first_pgdat)
 		consider_reclaim_throttle(first_pgdat, sc);
 
@@ -6754,6 +6778,7 @@ static unsigned long do_try_to_free_pages(struct zonelist *zonelist,
 	pg_data_t *last_pgdat;
 	struct zoneref *z;
 	struct zone *zone;
+	int nid;
 retry:
 	delayacct_freepages_start();
 
@@ -6792,6 +6817,21 @@ static unsigned long do_try_to_free_pages(struct zonelist *zonelist,
 		}
 	}
 
+	/*
+	 * Also cover the nodes shrink_zones() reclaimed beyond the zonelist.
+	 * Finishing a zonelist node again is harmless.
+	 */
+	if (cgroup_reclaim(sc)) {
+		for_each_node_state(nid, N_MEMORY_RECLAIM) {
+			struct lruvec *lruvec;
+
+			snapshot_refaults(sc->target_mem_cgroup, NODE_DATA(nid));
+			lruvec = mem_cgroup_lruvec(sc->target_mem_cgroup,
+						   NODE_DATA(nid));
+			clear_bit(LRUVEC_CGROUP_CONGESTED, &lruvec->flags);
+		}
+	}
+
 	delayacct_freepages_end();
 
 	if (sc->nr_reclaimed)
@@ -7038,7 +7078,8 @@ unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *memcg,
 	/*
 	 * Traverse the ZONELIST_FALLBACK zonelist of the current node to put
 	 * equal pressure on all the nodes. This is based on the assumption that
-	 * the reclaim does not bail out early.
+	 * the reclaim does not bail out early. shrink_zones() also visits the
+	 * nodes that permit reclaim and are not on the zonelist.
 	 */
 	struct zonelist *zonelist = node_zonelist(numa_node_id(), sc.gfp_mask);
 

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 50/60] mm: add N_MEMORY_MEMPOLICY node state
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (48 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 49/60] mm/vmscan: reclaim private nodes for memcg Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:27   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 51/60] mm: restrict explicit migration to N_MEMORY_MEMPOLICY Gregory Price via B4 Relay
                   ` (10 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Add N_MEMORY_MEMPOLICY as an internal state for the nodes that memory
policy interfaces and explicit migration may target, and
NODE_MEMORY_FEAT_MEMPOLICY for memory providers to select it.

Userspace placement lets ordinary processes use the node like any other
memory. That assumes the node is reclaimed and compacted, so the feature
requires both.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 Documentation/mm/physical_memory.rst |  3 +++
 include/linux/nodemask.h             | 11 +++++++++--
 mm/page_alloc.c                      |  4 ++++
 3 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/Documentation/mm/physical_memory.rst b/Documentation/mm/physical_memory.rst
index 1bafe8c085a3..75752c3a5694 100644
--- a/Documentation/mm/physical_memory.rst
+++ b/Documentation/mm/physical_memory.rst
@@ -190,6 +190,8 @@ The following states are internal and not exported through sysfs:
 ``N_MEMORY_RECLAIM``
   The kernel reclaims the node on its own initiative (kswapd, memcg and
   explicit reclaim).
+``N_MEMORY_MEMPOLICY``
+  Memory-policy interfaces and explicit migration may target the node.
 
 For each node that has a property described above, the bit corresponding to the
 node ID in the ``node_states[<property>]`` bitmask is set.
@@ -205,6 +207,7 @@ For example, for node 2 with normal memory and CPUs, bit 2 will be set in ::
   node_states[N_CPU]
   node_states[N_MEMORY_COMPACTION]
   node_states[N_MEMORY_RECLAIM]
+  node_states[N_MEMORY_MEMPOLICY]
 
 For various operations possible with nodemasks please refer to
 ``include/linux/nodemask.h``.
diff --git a/include/linux/nodemask.h b/include/linux/nodemask.h
index be5edbee2b22..c7a1800d9171 100644
--- a/include/linux/nodemask.h
+++ b/include/linux/nodemask.h
@@ -414,6 +414,7 @@ enum node_states {
 	/* Internal states, not exported through sysfs */
 	N_MEMORY_COMPACTION,	/* mm compaction may operate on the node */
 	N_MEMORY_RECLAIM,	/* mm reclaim may operate on the node */
+	N_MEMORY_MEMPOLICY,	/* memory policy may target the node */
 	NR_NODE_STATES
 };
 
@@ -552,14 +553,17 @@ static __always_inline int node_random(const nodemask_t *maskp)
 #define NODE_MEMORY_FEAT_COMMON		BIT(0) /* add to fallback zonelist */
 #define NODE_MEMORY_FEAT_COMPACTION	BIT(1) /* mm compaction */
 #define NODE_MEMORY_FEAT_RECLAIM	BIT(2) /* mm reclaim */
+#define NODE_MEMORY_FEAT_MEMPOLICY	BIT(3) /* userspace NUMA controls */
 #define NODE_MEMORY_FEAT_ALL		(NODE_MEMORY_FEAT_COMMON | \
 					 NODE_MEMORY_FEAT_COMPACTION | \
-					 NODE_MEMORY_FEAT_RECLAIM)
+					 NODE_MEMORY_FEAT_RECLAIM | \
+					 NODE_MEMORY_FEAT_MEMPOLICY)
 
 /*
  * Common memory carries every feature. Private memory needs NUMA: node 0
  * holds all memory without it. Reclaim leaves high-order requests to
- * compaction, so it requires compaction.
+ * compaction, so it requires compaction. Userspace placement assumes the
+ * node is reclaimed and compacted like any other memory.
  */
 static inline bool node_memory_features_valid(unsigned long features)
 {
@@ -570,6 +574,9 @@ static inline bool node_memory_features_valid(unsigned long features)
 	if ((features & NODE_MEMORY_FEAT_RECLAIM) &&
 	    !(features & NODE_MEMORY_FEAT_COMPACTION))
 		return false;
+	if ((features & NODE_MEMORY_FEAT_MEMPOLICY) &&
+	    !(features & NODE_MEMORY_FEAT_RECLAIM))
+		return false;
 	return IS_ENABLED(CONFIG_NUMA);
 }
 
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index cc70e1b45322..eff9c3d8251f 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -206,6 +206,7 @@ nodemask_t node_states[NR_NODE_STATES] __read_mostly = {
 	[N_MEMORY_COMMON] = { { [0] = 1UL } },
 	[N_MEMORY_COMPACTION] = { { [0] = 1UL } },
 	[N_MEMORY_RECLAIM] = { { [0] = 1UL } },
+	[N_MEMORY_MEMPOLICY] = { { [0] = 1UL } },
 	[N_CPU] = { { [0] = 1UL } },
 #endif	/* NUMA */
 };
@@ -224,10 +225,13 @@ void __meminit node_set_memory_state(int nid, unsigned long features)
 		node_set_state(nid, N_MEMORY_COMPACTION);
 	if (features & NODE_MEMORY_FEAT_RECLAIM)
 		node_set_state(nid, N_MEMORY_RECLAIM);
+	if (features & NODE_MEMORY_FEAT_MEMPOLICY)
+		node_set_state(nid, N_MEMORY_MEMPOLICY);
 }
 
 void __meminit node_clear_memory_state(int nid)
 {
+	node_clear_state(nid, N_MEMORY_MEMPOLICY);
 	node_clear_state(nid, N_MEMORY_RECLAIM);
 	node_clear_state(nid, N_MEMORY_COMPACTION);
 	node_clear_state(nid, N_MEMORY_PRIVATE);

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 51/60] mm: restrict explicit migration to N_MEMORY_MEMPOLICY
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (49 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 50/60] mm: add N_MEMORY_MEMPOLICY node state Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:38   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 52/60] mm: count memory with userspace placement as general purpose Gregory Price via B4 Relay
                   ` (9 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

mbind(MPOL_MF_MOVE), migrate_pages() and move_pages() act only on folios
on common memory. N_MEMORY_MEMPOLICY nodes are the nodes memory policy
and explicit migration may target, so let explicit migration also act on
folios already on them.

Add folio_allows_mempolicy(), which requires the folio's node to permit
memory policy, and use it instead.

Memory on other private nodes is still left alone:
  - move_pages() refuses to migrate it.
  - mbind(MPOL_MF_MOVE) and migrate_pages() skip its folios.

Cpuset migration can now find pages on these nodes. Evacuate them only
from nodes the cpuset no longer allows. Private nodes are still never
destinations for this kind of migration.

Migration keeps its best-effort behavior and can affect shared backing
pages seen by other mappers.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 kernel/cgroup/cpuset.c | 8 +++++++-
 mm/internal.h          | 7 +++++++
 mm/mempolicy.c         | 4 ++--
 mm/migrate.c           | 2 +-
 4 files changed, 17 insertions(+), 4 deletions(-)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index e20f9ccb8ab7..7d4b922a1914 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -2686,8 +2686,14 @@ static void cpuset_migrate_mm_workfn(struct work_struct *work)
 {
 	struct cpuset_migrate_mm_work *mwork =
 		container_of(work, struct cpuset_migrate_mm_work, work);
+	nodemask_t retained;
 
-	/* Private nodes are never migration destinations. */
+	/*
+	 * Private nodes are never migration destinations. Evacuate private
+	 * memory only from nodes the cpuset no longer allows.
+	 */
+	nodes_and(retained, mwork->to, node_states[N_MEMORY_PRIVATE]);
+	nodes_andnot(mwork->from, mwork->from, retained);
 	nodes_andnot(mwork->to, mwork->to, node_states[N_MEMORY_PRIVATE]);
 	/* on a wq worker, no need to worry about %current's mems_allowed */
 	do_migrate_pages(mwork->mm, &mwork->from, &mwork->to, MPOL_MF_MOVE_ALL);
diff --git a/mm/internal.h b/mm/internal.h
index fa1ae221564e..4bac570bb29f 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -87,6 +87,13 @@ static inline bool folio_allows_reclaim(struct folio *folio)
 	       node_state(folio_nid(folio), N_MEMORY_RECLAIM);
 }
 
+/* May userspace NUMA operations act on this folio's memory? */
+static inline bool folio_allows_mempolicy(struct folio *folio)
+{
+	return !folio_is_zone_device(folio) &&
+	       node_state(folio_nid(folio), N_MEMORY_MEMPOLICY);
+}
+
 /* mm/vmscan.c */
 unsigned long zone_reclaimable_pages(struct zone *zone);
 unsigned long try_to_free_pages(struct zonelist *zonelist, int order,
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 2bfe8b2b6287..9d2e56017ebe 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -705,7 +705,7 @@ static void queue_folios_pmd(pmd_t *pmd, unsigned long addr,
 		return;
 	}
 	folio = vm_normal_folio_pmd(walk->vma, addr, pmdval);
-	if (!folio || !folio_is_common_memory(folio))
+	if (!folio || !folio_allows_mempolicy(folio))
 		return;
 	if (!queue_folio_required(folio, qp))
 		return;
@@ -763,7 +763,7 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned long addr,
 			continue;
 		}
 		folio = vm_normal_folio(vma, addr, ptent);
-		if (!folio || !folio_is_common_memory(folio))
+		if (!folio || !folio_allows_mempolicy(folio))
 			continue;
 		if (folio_test_large(folio) && max_nr != 1)
 			nr = folio_pte_batch(folio, pte, ptent, max_nr);
diff --git a/mm/migrate.c b/mm/migrate.c
index ac583afa1bc5..6791834f14fe 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -2287,7 +2287,7 @@ static int __add_folio_for_migration(struct folio *folio, int node,
 	if (is_zero_folio(folio) || is_huge_zero_folio(folio))
 		return -EFAULT;
 
-	if (!folio_is_common_memory(folio))
+	if (!folio_allows_mempolicy(folio))
 		return -ENOENT;
 
 	if (folio_nid(folio) == node)

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 52/60] mm: count memory with userspace placement as general purpose
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (50 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 51/60] mm: restrict explicit migration to N_MEMORY_MEMPOLICY Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:33   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 53/60] mm/mempolicy: add MPOL_F_PRIVATE_NODES for private node bindings Gregory Price via B4 Relay
                   ` (8 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Userspace places ordinary data on N_MEMORY_MEMPOLICY nodes, and neither
the commit charge nor a task's RSS can tell nodes apart. Treat their
memory as general purpose: totalram_pages() and MemFree count it, and so
do the limits sized from them (overcommit, OOM capacity).

Other defaults sized from totalram_pages(), such as the zswap pool and
tmpfs sizes, also see the larger total, as they already do for memory
the kernel cannot allocate from in ZONE_MOVABLE.

Free memory on these nodes serves only private bindings and migration,
so counting it in MemAvailable or the global dirtyable memory would
overstate both. Add common_free_pages() and use it there.

MemAvailable also assumes about half of the file LRU can be reclaimed.
Reclaiming file pages on these nodes frees only private memory, so
count only the file LRU on common nodes, via common_file_pages().

File pages that reach a private node through migration stay covered
by the dirty limits' file LRU term.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/internal.h       | 26 ++++++++++++++++++++++++--
 mm/page-writeback.c |  7 ++++++-
 mm/show_mem.c       | 24 ++++++++++++++++++++----
 3 files changed, 50 insertions(+), 7 deletions(-)

diff --git a/mm/internal.h b/mm/internal.h
index 4bac570bb29f..c83c100a5253 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -793,11 +793,33 @@ void set_recommended_min_free_kbytes(void);
 
 extern char * const zone_names[MAX_NR_ZONES];
 
-/* true if the node is capable of servicing general system allocations */
+/*
+ * true if the node is capable of servicing general system allocations or
+ * userspace directed allocations (mempolicy, migration)
+ */
 static inline bool node_memory_is_general_purpose(int nid)
 {
 	return READ_ONCE(NODE_DATA(nid)->memory_features) &
-	       NODE_MEMORY_FEAT_COMMON;
+	       NODE_MEMORY_FEAT_MEMPOLICY;
+}
+
+unsigned long common_free_pages(void);
+
+/* File LRU pages on common nodes, whose reclaim frees common memory. */
+static inline unsigned long common_file_pages(void)
+{
+	unsigned long nr = global_node_page_state(NR_ACTIVE_FILE) +
+			   global_node_page_state(NR_INACTIVE_FILE);
+	int nid;
+
+	if (!IS_ENABLED(CONFIG_NUMA))
+		return nr;
+
+	for_each_node_state(nid, N_MEMORY_PRIVATE) {
+		nr -= min(nr, node_page_state(NODE_DATA(nid), NR_ACTIVE_FILE) +
+			      node_page_state(NODE_DATA(nid), NR_INACTIVE_FILE));
+	}
+	return nr;
 }
 
 extern int min_free_kbytes;
diff --git a/mm/page-writeback.c b/mm/page-writeback.c
index 00293a64fcea..6181266bf159 100644
--- a/mm/page-writeback.c
+++ b/mm/page-writeback.c
@@ -317,7 +317,12 @@ static unsigned long global_dirtyable_memory(void)
 {
 	unsigned long x;
 
-	x = nr_free_pages();
+	/*
+	 * Free private memory serves only private bindings and migration.
+	 * File pages that reach private memory are counted with the file
+	 * LRU below.
+	 */
+	x = common_free_pages();
 	/*
 	 * Pages reserved for the kernel should not be considered
 	 * dirtyable, to prevent a situation where reclaim has to
diff --git a/mm/show_mem.c b/mm/show_mem.c
index 24dd33ba783b..de4769e06a53 100644
--- a/mm/show_mem.c
+++ b/mm/show_mem.c
@@ -46,17 +46,18 @@ long si_mem_available(void)
 
 	/*
 	 * Estimate the amount of memory available for userspace allocations,
-	 * without causing swapping or OOM.
+	 * without causing swapping or OOM. Only free common memory serves
+	 * ordinary allocations.
 	 */
-	available = nr_free_pages() - totalreserve_pages;
+	available = common_free_pages() - totalreserve_pages;
 
 	/*
 	 * Not all the page cache can be freed, otherwise the system will
 	 * start swapping or thrashing. Assume at least half of the page
 	 * cache, or the low watermark worth of cache, needs to stay.
 	 */
-	pagecache = global_node_page_state(NR_ACTIVE_FILE) +
-		global_node_page_state(NR_INACTIVE_FILE);
+	/* Reclaiming file pages on private nodes frees only private memory. */
+	pagecache = common_file_pages();
 	pagecache -= min(pagecache / 2, wmark_low);
 	available += pagecache;
 
@@ -92,6 +93,21 @@ unsigned long nr_free_pages(void)
 	return nr;
 }
 
+/* Free pages on common nodes, the free memory ordinary allocations can use. */
+unsigned long common_free_pages(void)
+{
+	unsigned long nr = global_zone_page_state(NR_FREE_PAGES);
+	int nid;
+
+	if (!IS_ENABLED(CONFIG_NUMA))
+		return nr;
+
+	for_each_node_state(nid, N_MEMORY_PRIVATE) {
+		nr -= min(nr, sum_zone_node_page_state(nid, NR_FREE_PAGES));
+	}
+	return nr;
+}
+
 void si_meminfo(struct sysinfo *val)
 {
 	val->totalram = totalram_pages();

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 53/60] mm/mempolicy: add MPOL_F_PRIVATE_NODES for private node bindings
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (51 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 52/60] mm: count memory with userspace placement as general purpose Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:44   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 54/60] mm/migrate: route explicit NUMA migration to private nodes Gregory Price via B4 Relay
                   ` (7 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Private bindings behave differently from ordinary ones: they are strict,
never rebound, and fail with SIGBUS instead of invoking the OOM killer
when their node is exhausted. Callers should know they are opting into
that behavior explicitly.

Add MPOL_F_PRIVATE_NODES, an mbind() mode flag that binds a mapping to
one private node in N_MEMORY_MEMPOLICY. Without the flag, private nodes
in a mask are ignored - the same as nodes outside the cpuset.

Older kernels reject the flag, so userspace can probe for it.

A private binding must name exactly one node. The private zonelist
covers only that node, and a memoryless node left in the mask would
widen the binding once it gains memory.

The flag selects a rebind mode like MPOL_F_STATIC_NODES and
MPOL_F_RELATIVE_NODES, so it excludes both: a private binding is never
rebound. policy_nodemask() already ignores a binding with no node in the
current cpuset, so revoking access suspends the binding to common
memory, and restoring access resumes it.

The flag also excludes MPOL_F_NUMA_BALANCING, since NUMA balancing does
not act on private memory.  This can be loosened in the future.

Reject private bindings on HugeTLB mappings. Pool reservations are made
at mmap() time against global counts, and pool growth and dequeue use
the common zonelists, so the binding could never be satisfied. This
can also be loosened in the future if a use case grows HugeTLB support.

set_mempolicy() rejects the flag, because a task policy also steers the
task's kernel allocations.

Policies other than MPOL_BIND only apply to common memory nodes.
This can also be loosened later if a use case appears, but no compelling
use case has been proposed in 2 years of discussions.  This minimizes
the overall complexity of the base HBM/GPU use case considerably.

Use policy_nodemask() for the existing cpuset and zone checks, then
select the local private zonelist only when the effective target is
eligible private memory.

set_mempolicy_home_node() rejects a home node other than the bound
private node, since there is no fallback behavior in either direction,
an unmatched home node is nonsensical.

numa_maps reports private bindings as "private".

Document the flag in the admin guide.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 .../admin-guide/mm/numa_memory_policy.rst          | 22 +++++++
 include/uapi/linux/mempolicy.h                     |  4 +-
 mm/mempolicy.c                                     | 76 +++++++++++++++++++---
 3 files changed, 92 insertions(+), 10 deletions(-)

diff --git a/Documentation/admin-guide/mm/numa_memory_policy.rst b/Documentation/admin-guide/mm/numa_memory_policy.rst
index 90ab26e805a9..f2dc59cd6acc 100644
--- a/Documentation/admin-guide/mm/numa_memory_policy.rst
+++ b/Documentation/admin-guide/mm/numa_memory_policy.rst
@@ -341,6 +341,28 @@ MPOL_F_RELATIVE_NODES
 	MPOL_PREFERRED policies that were created with an empty nodemask
 	(local allocation).
 
+MPOL_F_PRIVATE_NODES
+	This flag binds a VMA to a single private memory node (see
+	Documentation/mm/numa_private_nodes.rst).  Without it, private
+	nodes in a policy's nodemask are ignored, as nodes outside the
+	task's cpuset are.
+
+	Only mbind() with MPOL_BIND accepts the flag, and its nodemask
+	must name exactly one private node that permits userspace
+	placement.
+	A private binding is never rebound when the task's allowed nodes
+	change.  While its node is ineligible (outside the task's cpuset,
+	or no longer private memory that permits userspace placement),
+	the binding is suspended and allocations use common memory.  It
+	resumes when the node becomes eligible again.  If the node returns
+	as common memory, the binding acts as an ordinary binding to that
+	node.
+	get_mempolicy() reports the original nodemask.
+
+	MPOL_F_PRIVATE_NODES cannot be combined with
+	MPOL_F_STATIC_NODES, MPOL_F_RELATIVE_NODES or
+	MPOL_F_NUMA_BALANCING, and cannot be used on HugeTLB mappings.
+
 Memory Policy Reference Counting
 ================================
 
diff --git a/include/uapi/linux/mempolicy.h b/include/uapi/linux/mempolicy.h
index 7f6fc9599693..4dcdf8f68803 100644
--- a/include/uapi/linux/mempolicy.h
+++ b/include/uapi/linux/mempolicy.h
@@ -31,13 +31,15 @@ enum mempolicy_mode {
 #define MPOL_F_STATIC_NODES	(1 << 15)
 #define MPOL_F_RELATIVE_NODES	(1 << 14)
 #define MPOL_F_NUMA_BALANCING	(1 << 13) /* Optimize with NUMA balancing if possible */
+#define MPOL_F_PRIVATE_NODES	(1 << 12) /* mbind() to a private memory node */
 
 /*
  * MPOL_MODE_FLAGS is the union of all possible optional mode flags passed to
  * either set_mempolicy() or mbind().
  */
 #define MPOL_MODE_FLAGS							\
-	(MPOL_F_STATIC_NODES | MPOL_F_RELATIVE_NODES | MPOL_F_NUMA_BALANCING)
+	(MPOL_F_STATIC_NODES | MPOL_F_RELATIVE_NODES | MPOL_F_NUMA_BALANCING | \
+	 MPOL_F_PRIVATE_NODES)
 
 /* Whether the nodemask is specified by users */
 #define MPOL_USER_NODEMASK_FLAGS (MPOL_F_STATIC_NODES | MPOL_F_RELATIVE_NODES)
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 9d2e56017ebe..cb5419cb7ee8 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -451,9 +451,13 @@ static int mpol_set_nodemask(struct mempolicy *pol,
 	if (!pol || pol->mode == MPOL_LOCAL)
 		return 0;
 
-	/* Check N_MEMORY_COMMON */
-	nodes_and(nsc->mask1,
-		  cpuset_current_mems_allowed, node_states[N_MEMORY_COMMON]);
+	/* Private bindings name a private node; others use common memory. */
+	if (pol->flags & MPOL_F_PRIVATE_NODES)
+		nodes_and(nsc->mask1, node_states[N_MEMORY_MEMPOLICY],
+			  node_states[N_MEMORY_PRIVATE]);
+	else
+		nsc->mask1 = node_states[N_MEMORY_COMMON];
+	nodes_and(nsc->mask1, nsc->mask1, cpuset_current_mems_allowed);
 
 	VM_BUG_ON(!nodes);
 
@@ -488,6 +492,13 @@ static struct mempolicy *mpol_new(unsigned short mode, unsigned short flags,
 	}
 	VM_BUG_ON(!nodes);
 
+	/*
+	 * The private zonelist covers a single node, so a private bind must name
+	 * exactly one node; a memoryless bit would later widen the binding.
+	 */
+	if ((flags & MPOL_F_PRIVATE_NODES) && nodes_weight(*nodes) != 1)
+		return ERR_PTR(-EINVAL);
+
 	/*
 	 * MPOL_PREFERRED cannot be used with MPOL_F_STATIC_NODES or
 	 * MPOL_F_RELATIVE_NODES if the nodemask is empty (local allocation).
@@ -580,6 +591,9 @@ static void mpol_rebind_policy(struct mempolicy *pol, const nodemask_t *newmask)
 
 	if (!pol || pol->mode == MPOL_LOCAL)
 		return;
+	/* Allocation checks a private binding against the current cpuset. */
+	if (pol->flags & MPOL_F_PRIVATE_NODES)
+		return;
 	mpol_rebindable_mems(&mems, newmask);
 	if (!mpol_store_user_nodemask(pol) &&
 	    nodes_equal(pol->w.cpuset_mems_allowed, mems))
@@ -1587,6 +1601,21 @@ static long do_mbind(unsigned long start, unsigned long len,
 	if (err)
 		goto mpol_out;
 
+	/* Private nodes cannot hold HugeTLB pages; this is unsupported. */
+	if (new && (new->flags & MPOL_F_PRIVATE_NODES)) {
+		vma_iter_init(&vmi, mm, start);
+		for_each_vma_range(vmi, vma, end) {
+			if (vma_is_hugetlb(vma)) {
+				err = -EINVAL;
+				break;
+			}
+		}
+		if (err) {
+			mmap_write_unlock(mm);
+			goto mpol_out;
+		}
+	}
+
 	/*
 	 * Lock the VMAs before scanning for pages to migrate,
 	 * to ensure we don't miss a concurrently inserted page.
@@ -1771,6 +1800,11 @@ static inline int sanitize_mpol_flags(int *mode, unsigned short *flags)
 		return -EINVAL;
 	if ((*flags & MPOL_F_STATIC_NODES) && (*flags & MPOL_F_RELATIVE_NODES))
 		return -EINVAL;
+	/* A private binding is never rebound or balanced. */
+	if ((*flags & MPOL_F_PRIVATE_NODES) &&
+	    (*mode != MPOL_BIND ||
+	     (*flags & (MPOL_USER_NODEMASK_FLAGS | MPOL_F_NUMA_BALANCING))))
+		return -EINVAL;
 	if (*flags & MPOL_F_NUMA_BALANCING) {
 		if (*mode == MPOL_BIND || *mode == MPOL_PREFERRED_MANY)
 			*flags |= (MPOL_F_MOF | MPOL_F_MORON);
@@ -1851,6 +1885,11 @@ SYSCALL_DEFINE4(set_mempolicy_home_node, unsigned long, start, unsigned long, le
 			err = -EOPNOTSUPP;
 			break;
 		}
+		if ((old->flags & MPOL_F_PRIVATE_NODES) &&
+		    !node_isset(home_node, old->nodes)) {
+			err = -EINVAL;
+			break;
+		}
 		new = mpol_dup(old);
 		if (IS_ERR(new)) {
 			err = PTR_ERR(new);
@@ -1888,6 +1927,10 @@ static long kernel_set_mempolicy(int mode, const unsigned long __user *nmask,
 	if (err)
 		return err;
 
+	/* A task policy also steers the task's kernel allocations. */
+	if (mode_flags & MPOL_F_PRIVATE_NODES)
+		return -EINVAL;
+
 	err = get_nodes(&nodes, nmask, maxnode);
 	if (err)
 		return err;
@@ -2124,7 +2167,8 @@ bool apply_policy_zone(struct mempolicy *policy, enum zone_type zone)
 	 * if policy->nodes has movable memory only,
 	 * we apply policy when gfp_zone(gfp) = ZONE_MOVABLE only.
 	 *
-	 * policy->nodes is intersect with node_states[N_MEMORY_COMMON].
+	 * policy->nodes is intersect with node_states[N_MEMORY_COMMON], or with
+	 * the private N_MEMORY_MEMPOLICY nodes for MPOL_F_PRIVATE_NODES.
 	 * so if the following test fails, it implies
 	 * policy->nodes has movable memory only.
 	 */
@@ -2495,10 +2539,20 @@ static struct page *alloc_pages_preferred_many(gfp_t gfp, unsigned int order,
 static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order,
 		struct mempolicy *pol, pgoff_t ilx, int nid)
 {
+	unsigned int alloc_flags = ALLOC_DEFAULT;
 	nodemask_t *nodemask;
 	struct page *page;
 
 	nodemask = policy_nodemask(gfp, pol, ilx, &nid);
+	if (nodemask && (pol->flags & MPOL_F_PRIVATE_NODES)) {
+		int private_nid = first_node(*nodemask);
+
+		if (node_state(private_nid, N_MEMORY_MEMPOLICY) &&
+		    node_state(private_nid, N_MEMORY_PRIVATE)) {
+			nid = private_nid;
+			alloc_flags = ALLOC_ZONELIST_PRIVATE;
+		}
+	}
 
 	if (pol->mode == MPOL_PREFERRED_MANY)
 		return alloc_pages_preferred_many(gfp, order, nid, nodemask);
@@ -2520,24 +2574,26 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order,
 		    pol->mode != MPOL_WEIGHTED_INTERLEAVE &&
 		    (!nodemask || node_isset(nid, *nodemask))) {
 			/*
-			 * First, try to allocate THP only on local node, but
+			 * First, try to allocate THP only on the selected node, but
 			 * don't reclaim unnecessarily, just compact.
 			 */
 			page = __alloc_frozen_pages_noprof(
 				gfp | __GFP_THISNODE | __GFP_NORETRY, order,
-				nid, NULL, ALLOC_DEFAULT);
+				nid, (alloc_flags & ALLOC_ZONELIST_PRIVATE) ?
+					 nodemask : NULL, alloc_flags);
 			if (page || !(gfp & __GFP_DIRECT_RECLAIM))
 				return page;
 			/*
 			 * If hugepage allocations are configured to always
 			 * synchronous compact or the vma has been madvised
 			 * to prefer hugepage backing, retry allowing remote
-			 * memory with both reclaim and compact as well.
+			 * memory with both reclaim and compact as well. A
+			 * private binding stays on its local zonelist.
 			 */
 		}
 	}
 
-	page = __alloc_frozen_pages_noprof(gfp, order, nid, nodemask, ALLOC_DEFAULT);
+	page = __alloc_frozen_pages_noprof(gfp, order, nid, nodemask, alloc_flags);
 
 	if (unlikely(pol->mode == MPOL_INTERLEAVE ||
 		     pol->mode == MPOL_WEIGHTED_INTERLEAVE) && page) {
@@ -3651,12 +3707,14 @@ void mpol_to_str(char *buffer, int maxlen, struct mempolicy *pol)
 		p += snprintf(p, buffer + maxlen - p, "=");
 
 		/*
-		 * Static and relative are mutually exclusive.
+		 * Static, relative and private are mutually exclusive.
 		 */
 		if (flags & MPOL_F_STATIC_NODES)
 			p += snprintf(p, buffer + maxlen - p, "static");
 		else if (flags & MPOL_F_RELATIVE_NODES)
 			p += snprintf(p, buffer + maxlen - p, "relative");
+		else if (flags & MPOL_F_PRIVATE_NODES)
+			p += snprintf(p, buffer + maxlen - p, "private");
 
 		if (flags & MPOL_F_NUMA_BALANCING) {
 			if (!is_power_of_2(flags & MPOL_MODE_FLAGS))

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 54/60] mm/migrate: route explicit NUMA migration to private nodes
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (52 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 53/60] mm/mempolicy: add MPOL_F_PRIVATE_NODES for private node bindings Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:42   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 55/60] mm/memory_hotplug: online and offline private memory nodes Gregory Price via B4 Relay
                   ` (6 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

move_pages(2) and migrate_pages(2) allocate the destination folio by
node id with __GFP_THISNODE. A private node is not on its own
__GFP_THISNODE (ZONELIST_NOFALLBACK) zonelist, so the allocation never
lands on it.

When the destination is a private node in N_MEMORY_MEMPOLICY, select the
node's private zonelist and a single-node mask in the syscall path, and
pass them to alloc_migration_target() via migration_target_control.

move_pages(2) now accepts such a destination rather than only common
memory, and still refuses other private nodes with -ENODEV, so
do_move_pages_to_node() only needs to check N_MEMORY_PRIVATE.
migrate_pages(2) keeps such nodes in its destination mask, and drops
other private nodes as it drops nodes outside the caller's cpuset.

The other users of alloc_migration_target() are unchanged.

HugeTLB folios still cannot move to a private node, since private nodes
hold no HugeTLB pages.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 mm/internal.h  |  1 +
 mm/mempolicy.c | 19 +++++++++++++++----
 mm/migrate.c   | 13 ++++++++++++-
 3 files changed, 28 insertions(+), 5 deletions(-)

diff --git a/mm/internal.h b/mm/internal.h
index c83c100a5253..646efc2eacd1 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -1309,6 +1309,7 @@ struct migration_target_control {
 	int nid;		/* preferred node id */
 	nodemask_t *nmask;
 	gfp_t gfp_mask;
+	unsigned int alloc_flags;	/* ALLOC_ZONELIST_PRIVATE or 0 */
 	enum migrate_reason reason;
 };
 
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index cb5419cb7ee8..a7cbb05ac86c 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -1340,7 +1340,7 @@ static bool migrate_folio_add(struct folio *folio, struct list_head *foliolist,
 static long migrate_to_node(struct mm_struct *mm, int source, int dest,
 			    int flags)
 {
-	nodemask_t nmask;
+	nodemask_t nmask, dest_mask;
 	struct vm_area_struct *vma;
 	LIST_HEAD(pagelist);
 	long nr_failed;
@@ -1351,6 +1351,14 @@ static long migrate_to_node(struct mm_struct *mm, int source, int dest,
 		.reason = MR_SYSCALL,
 	};
 
+	/* Private placement is reachable only through the private zonelist. */
+	if (node_state(dest, N_MEMORY_MEMPOLICY) &&
+	    node_state(dest, N_MEMORY_PRIVATE)) {
+		init_nodemask_of_node(&dest_mask, dest);
+		mtc.nmask = &dest_mask;
+		mtc.alloc_flags = ALLOC_ZONELIST_PRIVATE;
+	}
+
 	nodes_clear(nmask);
 	node_set(source, nmask);
 
@@ -1951,7 +1959,7 @@ static int kernel_migrate_pages(pid_t pid, unsigned long maxnode,
 	struct mm_struct *mm = NULL;
 	struct task_struct *task;
 	nodemask_t task_nodes;
-	int err;
+	int err, nid;
 	nodemask_t *old;
 	nodemask_t *new;
 	NODEMASK_SCRATCH(scratch);
@@ -2001,8 +2009,11 @@ static int kernel_migrate_pages(pid_t pid, unsigned long maxnode,
 	}
 
 	task_nodes = cpuset_mems_allowed(current);
-	/* Private memory is not a migration destination. */
-	nodes_andnot(task_nodes, task_nodes, node_states[N_MEMORY_PRIVATE]);
+	/* Private nodes are destinations only if they permit userspace placement. */
+	for_each_node_state(nid, N_MEMORY_PRIVATE) {
+		if (!node_state(nid, N_MEMORY_MEMPOLICY))
+			node_clear(nid, task_nodes);
+	}
 	if (!nodes_and(*new, *new, task_nodes))
 		goto out_put;
 
diff --git a/mm/migrate.c b/mm/migrate.c
index 6791834f14fe..eda961d43df0 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -2250,6 +2250,9 @@ struct folio *alloc_migration_target(struct folio *src, unsigned long private)
 	if (is_highmem_idx(zidx) || zidx == ZONE_MOVABLE)
 		gfp_mask |= __GFP_HIGHMEM;
 
+	if (mtc->alloc_flags)
+		return __folio_alloc_flags(gfp_mask, order, nid, mtc->nmask,
+					   mtc->alloc_flags);
 	return __folio_alloc(gfp_mask, order, nid, mtc->nmask);
 }
 
@@ -2267,6 +2270,7 @@ static int store_status(int __user *status, int start, int value, int nr)
 
 static int do_move_pages_to_node(struct list_head *pagelist, int node)
 {
+	nodemask_t nmask;
 	int err;
 	struct migration_target_control mtc = {
 		.nid = node,
@@ -2274,6 +2278,13 @@ static int do_move_pages_to_node(struct list_head *pagelist, int node)
 		.reason = MR_SYSCALL,
 	};
 
+	/* A private node is reachable only through its private zonelist. */
+	if (node_state(node, N_MEMORY_PRIVATE)) {
+		init_nodemask_of_node(&nmask, node);
+		mtc.nmask = &nmask;
+		mtc.alloc_flags = ALLOC_ZONELIST_PRIVATE;
+	}
+
 	err = migrate_pages(pagelist, alloc_migration_target, NULL,
 		(unsigned long)&mtc, MIGRATE_SYNC, MR_SYSCALL, NULL);
 	if (err)
@@ -2411,7 +2422,7 @@ static int do_pages_move(struct mm_struct *mm, nodemask_t task_nodes,
 		err = -ENODEV;
 		if (node < 0 || node >= MAX_NUMNODES)
 			goto out_flush;
-		if (!node_state(node, N_MEMORY_COMMON))
+		if (!node_state(node, N_MEMORY_MEMPOLICY))
 			goto out_flush;
 
 		err = -EACCES;

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 55/60] mm/memory_hotplug: online and offline private memory nodes
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (53 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 54/60] mm/migrate: route explicit NUMA migration to private nodes Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:34   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 56/60] dax/kmem: add a private memory mode Gregory Price via B4 Relay
                   ` (5 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Allow a driver to hotplug memory onto a private node at runtime.

Pass a feature mask through driver-managed hotplug, and reject masks
that node_memory_features_valid() refuses.

Memory joins a node only with the same mask. The node's features hold
while any System RAM block remains (including offline blocks), and are
reset after the last one is removed, even when ZONE_DEVICE memory keeps
the node alive.

dax/kmem and the other hotplug callers pass NODE_MEMORY_FEAT_ALL, which
registers the node as N_MEMORY_COMMON.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 drivers/dax/kmem.c             |  2 +-
 include/linux/memory_hotplug.h |  2 +-
 mm/internal.h                  |  9 ++++
 mm/memory_hotplug.c            | 94 +++++++++++++++++++++++++++++++-----------
 4 files changed, 80 insertions(+), 27 deletions(-)

diff --git a/drivers/dax/kmem.c b/drivers/dax/kmem.c
index a48d699cf344..5ec653bc88f0 100644
--- a/drivers/dax/kmem.c
+++ b/drivers/dax/kmem.c
@@ -126,7 +126,7 @@ static int dax_kmem_do_hotplug(struct dev_dax *dev_dax,
 		 */
 		rc = __add_memory_driver_managed(data->mgid, range.start,
 				range_len(&range), kmem_name, mhp_flags,
-				online_type);
+				online_type, NODE_MEMORY_FEAT_ALL);
 
 		if (rc) {
 			dev_warn(dev, "mapping%d: %#llx-%#llx memory add failed\n",
diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h
index ec71debc0673..ae6046510958 100644
--- a/include/linux/memory_hotplug.h
+++ b/include/linux/memory_hotplug.h
@@ -304,7 +304,7 @@ extern int add_memory_resource(int nid, struct resource *resource,
 			       mhp_t mhp_flags);
 int __add_memory_driver_managed(int nid, u64 start, u64 size,
 				const char *resource_name, mhp_t mhp_flags,
-				enum mmop online_type);
+				enum mmop online_type, unsigned long features);
 extern int add_memory_driver_managed(int nid, u64 start, u64 size,
 				     const char *resource_name,
 				     mhp_t mhp_flags);
diff --git a/mm/internal.h b/mm/internal.h
index 646efc2eacd1..0eefa0df1350 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -65,6 +65,15 @@ void folio_mark_lazyfree(struct folio *folio);
 void node_set_memory_state(int nid, unsigned long features);
 void node_clear_memory_state(int nid);
 
+#ifdef CONFIG_MEMORY_HOTPLUG
+bool can_claim_node_memory(int nid, unsigned long features);
+#else
+static inline bool can_claim_node_memory(int nid, unsigned long features)
+{
+	return false;
+}
+#endif
+
 /*
  * Common memory is available to generic MM services. ZONE_DEVICE folios do
  * not qualify even when their NUMA node is common.
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 8146f77919d5..37e64c146b46 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -1494,6 +1494,43 @@ static int create_altmaps_and_memory_blocks(int nid, struct memory_group *group,
 	return ret;
 }
 
+static int check_no_memblock_for_node_cb(struct memory_block *mem, void *arg)
+{
+	int nid = *(int *)arg;
+
+	/*
+	 * If a memory block belongs to multiple nodes, the stored nid is not
+	 * reliable. However, such blocks are always online (e.g., cannot get
+	 * offlined) and, therefore, are still spanned by the node.
+	 */
+	return mem->nid == nid ? -EEXIST : 0;
+}
+
+/* Caller must hold the memory hotplug lock for these checks. */
+static bool node_has_memory_blocks(int nid)
+{
+	return for_each_memory_block(&nid, check_no_memblock_for_node_cb);
+}
+
+static bool node_has_system_ram(int nid)
+{
+	/* Mixed-node memory blocks are always online and covered by N_MEMORY. */
+	return node_state(nid, N_MEMORY) || node_has_memory_blocks(nid);
+}
+
+/*
+ * Whether memory with @features may join @nid. Offline System RAM blocks keep
+ * the node's features until the last block is removed.
+ */
+bool can_claim_node_memory(int nid, unsigned long features)
+{
+	pg_data_t *pgdat = NODE_DATA(nid);
+
+	if (pgdat->memory_features == features)
+		return true;
+	return !node_has_system_ram(nid);
+}
+
 /*
  * NOTE: The caller must call lock_device_hotplug() to serialize hotplug
  * and online/offline operations (triggered e.g. by sysfs).
@@ -1501,11 +1538,12 @@ static int create_altmaps_and_memory_blocks(int nid, struct memory_group *group,
  * we are OK calling __meminit stuff here - we have CONFIG_MEMORY_HOTPLUG
  */
 static int __add_memory_resource(int nid, struct resource *res, mhp_t mhp_flags,
-				 enum mmop online_type)
+				 enum mmop online_type, unsigned long features)
 {
 	struct mhp_params params = { .pgprot = pgprot_mhp(PAGE_KERNEL) };
 	enum memblock_flags memblock_flags = MEMBLOCK_NONE;
 	struct memory_group *group = NULL;
+	unsigned long old_features;
 	u64 start, size;
 	bool new_node = false;
 	int ret;
@@ -1528,6 +1566,8 @@ static int __add_memory_resource(int nid, struct resource *res, mhp_t mhp_flags,
 		WARN(1, "node %d was absent from the node_possible_map\n", nid);
 		return -EINVAL;
 	}
+	if (!node_memory_features_valid(features))
+		return -EINVAL;
 
 	mem_hotplug_begin();
 
@@ -1552,6 +1592,14 @@ static int __add_memory_resource(int nid, struct resource *res, mhp_t mhp_flags,
 		new_node = true;
 	}
 
+	old_features = NODE_DATA(nid)->memory_features;
+	if (!can_claim_node_memory(nid, features)) {
+		ret = -EBUSY;
+		goto error;
+	}
+	/* Set the node's features before onlining its memory. */
+	WRITE_ONCE(NODE_DATA(nid)->memory_features, features);
+
 	/*
 	 * Self hosted memmap array
 	 */
@@ -1597,6 +1645,8 @@ static int __add_memory_resource(int nid, struct resource *res, mhp_t mhp_flags,
 
 	return ret;
 error:
+	/* Restore the previous features while holding the lock. */
+	WRITE_ONCE(NODE_DATA(nid)->memory_features, old_features);
 	if (new_node) {
 		node_set_offline(nid);
 		unregister_node(nid);
@@ -1612,7 +1662,8 @@ static int __add_memory_resource(int nid, struct resource *res, mhp_t mhp_flags,
 int add_memory_resource(int nid, struct resource *res, mhp_t mhp_flags)
 {
 	return __add_memory_resource(nid, res, mhp_flags,
-				     mhp_get_default_online_type());
+				     mhp_get_default_online_type(),
+				     NODE_MEMORY_FEAT_ALL);
 }
 
 /* requires device_hotplug_lock, see __add_memory_resource() */
@@ -1651,6 +1702,9 @@ EXPORT_SYMBOL_GPL(add_memory);
  * @resource_name: Resource name in format "System RAM ($DRIVER)"
  * @mhp_flags: Memory hotplug flags
  * @online_type: Auto-Online behavior (offline, online, kernel, movable)
+ * @features: NODE_MEMORY_FEAT_* services the memory opts into;
+ *            %NODE_MEMORY_FEAT_ALL adds it as ordinary common system RAM,
+ *            and a mask without %NODE_MEMORY_FEAT_COMMON as private memory.
  *
  * Add special, driver-managed memory to the system as system RAM. Such
  * memory is not exposed via the raw firmware-provided memmap as system
@@ -1677,7 +1731,7 @@ EXPORT_SYMBOL_GPL(add_memory);
  */
 int __add_memory_driver_managed(int nid, u64 start, u64 size,
 		const char *resource_name, mhp_t mhp_flags,
-		enum mmop online_type)
+		enum mmop online_type, unsigned long features)
 {
 	struct resource *res;
 	int rc;
@@ -1698,7 +1752,7 @@ int __add_memory_driver_managed(int nid, u64 start, u64 size,
 		goto out_unlock;
 	}
 
-	rc = __add_memory_resource(nid, res, mhp_flags, online_type);
+	rc = __add_memory_resource(nid, res, mhp_flags, online_type, features);
 	if (rc < 0)
 		release_memory_resource(res);
 
@@ -1727,8 +1781,8 @@ int add_memory_driver_managed(int nid, u64 start, u64 size,
 			      const char *resource_name, mhp_t mhp_flags)
 {
 	return __add_memory_driver_managed(nid, start, size, resource_name,
-			mhp_flags,
-			mhp_get_default_online_type());
+			mhp_flags, mhp_get_default_online_type(),
+			NODE_MEMORY_FEAT_ALL);
 }
 EXPORT_SYMBOL_GPL(add_memory_driver_managed);
 
@@ -1789,18 +1843,6 @@ bool mhp_range_allowed(u64 start, u64 size, bool need_mapping)
 
 #ifdef CONFIG_MEMORY_HOTREMOVE
 
-static int check_no_memblock_for_node_cb(struct memory_block *mem, void *arg)
-{
-	int nid = *(int *)arg;
-
-	/*
-	 * If a memory block belongs to multiple nodes, the stored nid is not
-	 * reliable. However, such blocks are always online (e.g., cannot get
-	 * offlined) and, therefore, are still spanned by the node.
-	 */
-	return mem->nid == nid ? -EEXIST : 0;
-}
-
 /* Caller must hold the memory hotplug lock for this check. */
 static bool node_is_memoryless(int nid)
 {
@@ -1811,12 +1853,9 @@ static bool node_is_memoryless(int nid)
 	 */
 	if (node_spanned_pages(nid))
 		return false;
-	/*
-	 * Offline memory blocks may not be spanned by the node yet, but they
-	 * link to it in sysfs and can be onlined later, so the node is not
-	 * memoryless while any remain.
-	 */
-	return !for_each_memory_block(&nid, check_no_memblock_for_node_cb);
+
+	/* Offline memory blocks keep the node alive so they can be onlined. */
+	return !node_has_memory_blocks(nid);
 }
 
 /*
@@ -2347,8 +2386,13 @@ static int try_remove_memory(u64 start, u64 size)
 
 	release_mem_region_adjustable(start, size);
 
-	if (nid != NUMA_NO_NODE)
+	if (nid != NUMA_NO_NODE) {
+		/* No System RAM left: reset the node's features. */
+		if (!node_has_system_ram(nid))
+			WRITE_ONCE(NODE_DATA(nid)->memory_features,
+				   NODE_MEMORY_FEAT_ALL);
 		try_offline_node(nid);
+	}
 
 	mem_hotplug_done();
 	return 0;

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 56/60] dax/kmem: add a private memory mode
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (54 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 55/60] mm/memory_hotplug: online and offline private memory nodes Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:31   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 57/60] selftests/dax: test private kmem devices and services Gregory Price via B4 Relay
                   ` (4 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Add a `private` boolean sysfs attribute, defaulting to false and
changeable only while memory is unplugged.

When true, dax/kmem onlines the node as private memory with
NODE_MEMORY_FEAT_MEMPOLICY, NODE_MEMORY_FEAT_RECLAIM and
NODE_MEMORY_FEAT_COMPACTION.

Userspace targets private capacity via single-node MPOL_F_PRIVATE_NODES
bindings on anonymous and shmem-backed mappings. Reclaim and compaction
manage the memory, and the system does not otherwise use it.

The attribute is hidden without CONFIG_NUMA or CONFIG_MEMORY_HOTREMOVE,
since selecting private mode requires unplugging the memory first, and
the default behavior of dax/kmem probe creates the memory blocks (an
unfortunately backward compatibility wart).

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 Documentation/ABI/testing/sysfs-bus-dax | 19 ++++++++++++
 drivers/dax/kmem.c                      | 55 +++++++++++++++++++++++++++++++--
 2 files changed, 72 insertions(+), 2 deletions(-)

diff --git a/Documentation/ABI/testing/sysfs-bus-dax b/Documentation/ABI/testing/sysfs-bus-dax
index 60703fffa99e..d024cf8e5ae0 100644
--- a/Documentation/ABI/testing/sysfs-bus-dax
+++ b/Documentation/ABI/testing/sysfs-bus-dax
@@ -175,3 +175,22 @@ Description:
 		A device can only be onlined from the "unplugged" state, so a
 		device must be returned to "unplugged" before it can be onlined
 		into a different state.
+
+What:		/sys/bus/dax/devices/daxX.Y/private
+Date:		October, 2026
+Contact:	nvdimm@lists.linux.dev
+Description:
+		(RW) Select private memory when a device is bound to kmem.
+		Accepts boolean values and defaults to 0. The value can only
+		change while state is "unplugged", before memory blocks exist.
+		The attribute is hidden without CONFIG_NUMA or
+		CONFIG_MEMORY_HOTREMOVE.
+
+		With private=1, the next online enables userspace NUMA
+		placement, together with the reclaim and compaction it
+		requires. The memory is excluded from ordinary allocations
+		and from other memory management services.
+
+		Userspace can bind anonymous and shmem-backed mappings to it
+		with a single-node mbind() policy using
+		(MPOL_BIND | MPOL_F_PRIVATE_NODES).
diff --git a/drivers/dax/kmem.c b/drivers/dax/kmem.c
index 5ec653bc88f0..2954f22ececf 100644
--- a/drivers/dax/kmem.c
+++ b/drivers/dax/kmem.c
@@ -46,6 +46,7 @@ struct dax_kmem_data {
 	const char *res_name;
 	int mgid;
 	int state;
+	bool private;
 	struct mutex lock; /* protects hotplug state transitions */
 	struct resource *res[];
 };
@@ -89,9 +90,15 @@ static int dax_kmem_do_hotplug(struct dev_dax *dev_dax,
 			       int online_type)
 {
 	struct device *dev = &dev_dax->dev;
+	unsigned long features;
 	int i, rc, added = 0;
 	mhp_t mhp_flags;
 
+	features = data->private ? NODE_MEMORY_FEAT_MEMPOLICY |
+				   NODE_MEMORY_FEAT_RECLAIM |
+				   NODE_MEMORY_FEAT_COMPACTION :
+				   NODE_MEMORY_FEAT_ALL;
+
 	if (dax_kmem_state_is_online(data->state))
 		return -EINVAL;
 
@@ -126,7 +133,7 @@ static int dax_kmem_do_hotplug(struct dev_dax *dev_dax,
 		 */
 		rc = __add_memory_driver_managed(data->mgid, range.start,
 				range_len(&range), kmem_name, mhp_flags,
-				online_type, NODE_MEMORY_FEAT_ALL);
+				online_type, features);
 
 		if (rc) {
 			dev_warn(dev, "mapping%d: %#llx-%#llx memory add failed\n",
@@ -380,6 +387,33 @@ static ssize_t state_store(struct device *dev, struct device_attribute *attr,
 	return len;
 }
 
+static ssize_t private_show(struct device *dev,
+			    struct device_attribute *attr, char *buf)
+{
+	struct dax_kmem_data *data = dev_get_drvdata(dev);
+
+	guard(mutex)(&data->lock);
+	return sysfs_emit(buf, "%d\n", data->private);
+}
+
+static ssize_t private_store(struct device *dev, struct device_attribute *attr,
+			     const char *buf, size_t len)
+{
+	struct dax_kmem_data *data = dev_get_drvdata(dev);
+	bool private;
+	int rc;
+
+	rc = kstrtobool(buf, &private);
+	if (rc)
+		return rc;
+
+	guard(mutex)(&data->lock);
+	if (data->private != private && data->state != DAX_KMEM_UNPLUGGED)
+		return -EBUSY;
+	data->private = private;
+	return len;
+}
+
 static int dev_dax_kmem_probe(struct dev_dax *dev_dax)
 {
 	struct device *dev = &dev_dax->dev;
@@ -560,12 +594,29 @@ static void dev_dax_kmem_remove(struct dev_dax *dev_dax)
 #endif /* CONFIG_MEMORY_HOTREMOVE */
 
 static DEVICE_ATTR_RW(state);
+static DEVICE_ATTR_RW(private);
 
 static struct attribute *dev_dax_kmem_attrs[] = {
 	&dev_attr_state.attr,
+	&dev_attr_private.attr,
 	NULL,
 };
-ATTRIBUTE_GROUPS(dev_dax_kmem);
+
+static umode_t dev_dax_kmem_visible(struct kobject *kobj,
+				    struct attribute *attr, int n)
+{
+	/* Selecting private memory needs an unplug, which needs hot-remove. */
+	if (attr == &dev_attr_private.attr &&
+	    (!IS_ENABLED(CONFIG_NUMA) || !IS_ENABLED(CONFIG_MEMORY_HOTREMOVE)))
+		return 0;
+	return attr->mode;
+}
+
+static const struct attribute_group dev_dax_kmem_group = {
+	.attrs = dev_dax_kmem_attrs,
+	.is_visible = dev_dax_kmem_visible,
+};
+__ATTRIBUTE_GROUPS(dev_dax_kmem);
 
 static struct dax_device_driver device_dax_kmem_driver = {
 	.probe = dev_dax_kmem_probe,

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 57/60] selftests/dax: test private kmem devices and services
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (55 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 56/60] dax/kmem: add a private memory mode Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:42   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 58/60] selftests/dax: test memory policy on private nodes Gregory Price via B4 Relay
                   ` (3 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Test dax/kmem private mode on a device-dax instance:
  - the private attribute across memory hotplug: private memory joins
    has_private_memory, the mode changes only while unplugged, and
    online common memory cannot become private.
  - services on the private node: kswapd and kcompactd run,
    nodeN/reclaim and nodeN/compact are accepted, and HugeTLB pool
    growth stays off the node.

The tests reconfigure a device, so they run only when DAX_KMEM_TEST_DEV
names one (daxX.Y), or is "auto" to pick a device whose memory is not
online. The device's original configuration is restored on exit.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 tools/testing/selftests/dax/Makefile               |   3 +
 tools/testing/selftests/dax/config                 |   2 +
 tools/testing/selftests/dax/kmem_private.sh        |  94 ++++++++++
 .../testing/selftests/dax/kmem_private_services.sh |  97 ++++++++++
 tools/testing/selftests/dax/private_node_common.sh | 208 +++++++++++++++++++++
 5 files changed, 404 insertions(+)

diff --git a/tools/testing/selftests/dax/Makefile b/tools/testing/selftests/dax/Makefile
index 25a4f3d73a5b..55d2ba0de8d9 100644
--- a/tools/testing/selftests/dax/Makefile
+++ b/tools/testing/selftests/dax/Makefile
@@ -2,5 +2,8 @@
 all:
 
 TEST_PROGS := dax-kmem-hotplug.sh
+TEST_PROGS += kmem_private.sh
+TEST_PROGS += kmem_private_services.sh
+TEST_FILES := private_node_common.sh
 
 include ../lib.mk
diff --git a/tools/testing/selftests/dax/config b/tools/testing/selftests/dax/config
index 4c9aaeb6ceb4..42853ed9e99f 100644
--- a/tools/testing/selftests/dax/config
+++ b/tools/testing/selftests/dax/config
@@ -2,3 +2,5 @@ CONFIG_DEV_DAX=m
 CONFIG_DEV_DAX_KMEM=m
 CONFIG_MEMORY_HOTPLUG=y
 CONFIG_MEMORY_HOTREMOVE=y
+CONFIG_NUMA=y
+CONFIG_COMPACTION=y
diff --git a/tools/testing/selftests/dax/kmem_private.sh b/tools/testing/selftests/dax/kmem_private.sh
new file mode 100755
index 000000000000..ceba05e69e14
--- /dev/null
+++ b/tools/testing/selftests/dax/kmem_private.sh
@@ -0,0 +1,94 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# Exercise the kmem "private" attribute across memory hotplug: when it may
+# change, and whether onlined memory joins has_private_memory or
+# has_common_memory.
+#
+# Needs root and DAX_KMEM_TEST_DEV=<daxX.Y|auto>, see private_node_common.sh.
+
+# shellcheck disable=SC1091
+DIR="$(dirname "$(readlink -f "$0")")"
+. "$DIR"/../kselftest/ktap_helpers.sh
+. "$DIR"/private_node_common.sh
+
+trap pn_unbind_kmem_private EXIT
+pn_setup
+
+if ! echo unplugged > "$D/state" 2>/dev/null ||
+   pn_node_in "$PN" has_memory; then
+	ktap_exit_fail_msg "private memory cannot be unplugged"
+fi
+
+ktap_set_plan 8
+
+name="private mode can be selected while unplugged"
+if echo 0 > "$D/private" 2>/dev/null && echo 1 > "$D/private" 2>/dev/null &&
+   [ "$(cat "$D/private")" = 1 ]; then
+	ktap_test_pass "$name"
+else
+	ktap_test_fail "$name"
+fi
+
+name="invalid private setting is rejected"
+if ! echo invalid > "$D/private" 2>/dev/null &&
+   [ "$(cat "$D/private")" = 1 ]; then
+	ktap_test_pass "$name"
+else
+	ktap_test_fail "$name"
+fi
+
+name="private online publishes private memory"
+if echo online_movable > "$D/state" 2>/dev/null &&
+   pn_node_is_private "$PN" && ! pn_node_in "$PN" has_common_memory; then
+	ktap_test_pass "$name"
+else
+	ktap_test_fail "$name"
+fi
+
+# Writing the current value again is accepted.
+name="private mode cannot change while memory blocks exist"
+if ! echo 0 > "$D/private" 2>/dev/null &&
+   [ "$(cat "$D/private")" = 1 ] && echo 1 > "$D/private" 2>/dev/null; then
+	ktap_test_pass "$name"
+else
+	ktap_test_fail "$name"
+fi
+
+name="unplug removes the private node from memory lists"
+if echo unplugged > "$D/state" 2>/dev/null &&
+   ! pn_node_in "$PN" has_memory &&
+   ! pn_node_in "$PN" has_private_memory; then
+	ktap_test_pass "$name"
+else
+	ktap_test_fail "$name"
+fi
+
+name="common mode preserves normal kmem behavior"
+if echo 0 > "$D/private" 2>/dev/null &&
+   echo online_movable > "$D/state" 2>/dev/null &&
+   pn_node_in "$PN" has_common_memory; then
+	ktap_test_pass "$name"
+else
+	ktap_test_fail "$name"
+fi
+
+name="online common memory cannot be reclassified as private"
+if ! echo 1 > "$D/private" 2>/dev/null &&
+   [ "$(cat "$D/private")" = 0 ]; then
+	ktap_test_pass "$name"
+else
+	ktap_test_fail "$name"
+fi
+
+name="private mode can be restored through an unplug cycle"
+if echo unplugged > "$D/state" 2>/dev/null &&
+   echo 1 > "$D/private" 2>/dev/null &&
+   echo online_movable > "$D/state" 2>/dev/null &&
+   pn_node_is_private "$PN"; then
+	ktap_test_pass "$name"
+else
+	ktap_test_fail "$name"
+fi
+
+ktap_finished
diff --git a/tools/testing/selftests/dax/kmem_private_services.sh b/tools/testing/selftests/dax/kmem_private_services.sh
new file mode 100755
index 000000000000..df5d17fdb947
--- /dev/null
+++ b/tools/testing/selftests/dax/kmem_private_services.sh
@@ -0,0 +1,97 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# Check which MM services a private kmem node gets: no HugeTLB pool pages,
+# but kswapd, kcompactd and per-node compaction and reclaim.
+#
+# Needs root and DAX_KMEM_TEST_DEV=<daxX.Y|auto>, see private_node_common.sh.
+
+# shellcheck disable=SC1091
+DIR="$(dirname "$(readlink -f "$0")")"
+. "$DIR"/../kselftest/ktap_helpers.sh
+. "$DIR"/private_node_common.sh
+
+trap pn_unbind_kmem_private EXIT
+pn_setup
+
+ktap_set_plan 5
+
+# Use the default size: a gigantic page may not fit the device at all.
+hsize=$(awk '/^Hugepagesize:/ { print $2 }' /proc/meminfo)
+pool="$NODE_BASE/node$PN/hugepages/hugepages-${hsize}kB/nr_hugepages"
+global=/sys/kernel/mm/hugepages/hugepages-${hsize}kB/nr_hugepages
+name_node="per-node HugeTLB pool growth skips private memory"
+name_global="HugeTLB pool growth falls back to other nodes"
+if [ -z "$hsize" ] || [ ! -w "$pool" ] || [ ! -w "$global" ]; then
+	ktap_print_msg "HugeTLB pools unavailable"
+	ktap_test_skip "$name_node"
+	ktap_test_skip "$name_global"
+else
+	before=$(cat "$pool")
+	echo $((before + 8)) > "$pool" 2>/dev/null
+	after=$(cat "$pool")
+	echo "$before" > "$pool" 2>/dev/null
+	if [ "$after" = "$before" ]; then
+		ktap_test_pass "$name_node"
+	else
+		ktap_print_msg "private-node pool grew $before->$after"
+		ktap_test_fail "$name_node"
+	fi
+
+	gbefore=$(cat "$global")
+	echo $((gbefore + 64)) > "$global" 2>/dev/null
+	gafter=$(cat "$global")
+	after=$(cat "$pool")
+	echo "$gbefore" > "$global" 2>/dev/null
+	if [ "$gafter" -le "$gbefore" ]; then
+		ktap_print_msg "HugeTLB pool could not grow on any node"
+		ktap_test_skip "$name_global"
+	elif [ "$after" = "$before" ]; then
+		ktap_test_pass "$name_global"
+	else
+		ktap_print_msg "private-node pool grew $before->$after"
+		ktap_test_fail "$name_global"
+	fi
+fi
+
+compact="$NODE_BASE/node$PN/compact"
+name="kswapd and kcompactd run for the private node"
+if ! pgrep -x "kswapd$PN" >/dev/null; then
+	ktap_print_msg "kswapd$PN is not running"
+	ktap_test_fail "$name"
+elif [ ! -e "$compact" ]; then
+	ktap_print_msg "kcompactd unavailable without compaction"
+	ktap_test_skip "$name"
+elif pgrep -x "kcompactd$PN" >/dev/null; then
+	ktap_test_pass "$name"
+else
+	ktap_print_msg "kcompactd$PN is not running"
+	ktap_test_fail "$name"
+fi
+
+name="private-node compaction is accepted"
+if [ ! -e "$compact" ]; then
+	ktap_print_msg "per-node compaction unavailable"
+	ktap_test_skip "$name"
+elif echo 1 > "$compact" 2>/dev/null; then
+	ktap_test_pass "$name"
+else
+	ktap_test_fail "$name"
+fi
+
+reclaim="$NODE_BASE/node$PN/reclaim"
+name="private-node reclaim is accepted"
+if [ ! -e "$reclaim" ]; then
+	ktap_print_msg "per-node reclaim unavailable"
+	ktap_test_skip "$name"
+elif err=$(export LC_ALL=C; echo 1M 2>&1 > "$reclaim"); then
+	ktap_test_pass "$name"
+elif [[ $err == *"Resource temporarily unavailable"* ]]; then
+	# An idle node may have nothing to reclaim.
+	ktap_test_pass "$name"
+else
+	ktap_print_msg "$err"
+	ktap_test_fail "$name"
+fi
+
+ktap_finished
diff --git a/tools/testing/selftests/dax/private_node_common.sh b/tools/testing/selftests/dax/private_node_common.sh
new file mode 100644
index 000000000000..57c5231e768b
--- /dev/null
+++ b/tools/testing/selftests/dax/private_node_common.sh
@@ -0,0 +1,208 @@
+# SPDX-License-Identifier: GPL-2.0
+# shellcheck shell=bash disable=SC2034
+# Helpers shared by the kmem_private*.sh selftests, sourced after
+# ../kselftest/ktap_helpers.sh.
+#
+# DAX_KMEM_TEST_DEV=<daxX.Y|auto> opts into reconfiguring a device-dax
+# instance as private kmem memory. "auto" only uses devices whose memory is
+# not online and that no process has open or mapped. A named device is used
+# whatever its state. Either is restored on exit.
+#
+# pn_setup() sets D to the sysfs directory of that device and PN to its node.
+
+DAX_BASE=/sys/bus/dax/devices
+NODE_BASE=/sys/devices/system/node
+
+# Whether nodelist $1 (e.g. "0-2,4") contains node $2.
+pn_nodelist_has()
+{
+	local nid=$2 token low high
+
+	for token in ${1//,/ }; do
+		low=${token%-*}
+		high=${token#*-}
+		if [ "$nid" -ge "$low" ] 2>/dev/null &&
+		   [ "$nid" -le "$high" ] 2>/dev/null; then
+			return 0
+		fi
+	done
+	return 1
+}
+
+# Whether node $1 is in the node state list $2, e.g. has_memory.
+pn_node_in()
+{
+	[ -r "$NODE_BASE/$2" ] &&
+		pn_nodelist_has "$(cat "$NODE_BASE/$2")" "$1"
+}
+
+pn_node_is_private()
+{
+	pn_node_in "$1" has_private_memory
+}
+
+# Whether any process has device node $1 open or mapped.
+pn_dev_in_use()
+{
+	local fd
+
+	for fd in /proc/[0-9]*/fd/*; do
+		[ "$(readlink "$fd" 2>/dev/null)" = "$1" ] && return 0
+	done
+	grep -qs " $1\$" /proc/[0-9]*/maps
+}
+
+# Record the original configuration of device $1, bound to driver $2, for
+# pn_unbind_kmem_private().
+pn_record_kmem()
+{
+	# Binding the recorded device again keeps its original configuration.
+	[ "${PN_DEV:-}" != "$1" ] || return 0
+	PN_DEV=$1
+	PN_ORIG_DRIVER=$2
+	PN_ORIG_STATE=$(cat "$1/state" 2>/dev/null)
+	PN_ORIG_PRIVATE=$(cat "$1/private" 2>/dev/null)
+	PN_NEW_ID=
+}
+
+# Move device $1 from driver $2 to kmem and online it as private memory on
+# node $3.
+pn_online_private()
+{
+	local dev=$1 driver=$2 nid=$3 name
+
+	name=$(basename "$dev")
+	if [ "$driver" = kmem ]; then
+		echo unplugged > "$dev/state" || return 1
+	elif [ -n "$driver" ]; then
+		echo "$name" > "/sys/bus/dax/drivers/$driver/unbind" || return 1
+	fi
+	# A private node holds the memory of this device only.
+	if pn_node_in "$nid" has_memory; then
+		return 1
+	fi
+	if [ "$driver" != kmem ]; then
+		echo "$name" > /sys/bus/dax/drivers/kmem/new_id || return 1
+		PN_NEW_ID=1
+	fi
+	echo unplugged > "$dev/state" &&
+		echo 1 > "$dev/private" &&
+		echo online_movable > "$dev/state" &&
+		pn_node_is_private "$nid"
+}
+
+# Online device $1 as private kmem memory, or return 1 if it is unusable.
+pn_bind_kmem_private()
+{
+	local dev=$1 name driver nid
+
+	name=$(basename "$dev")
+	nid=$(cat "$dev/target_node" 2>/dev/null) || return 1
+	driver=$(basename "$(readlink "$dev/driver" 2>/dev/null)" 2>/dev/null)
+	if [ "$driver" = kmem ] &&
+	   [ "$(cat "$dev/state" 2>/dev/null)" != unplugged ]; then
+		# Leave online kmem memory alone unless it is ours or was named.
+		if [ "$DAX_KMEM_TEST_DEV" = auto ] &&
+		   [ "${PN_DEV:-}" != "$dev" ]; then
+			return 1
+		fi
+		if [ "$(cat "$dev/private" 2>/dev/null)" = 1 ] &&
+		   pn_node_is_private "$nid"; then
+			pn_record_kmem "$dev" "$driver"
+			return 0
+		fi
+	elif pn_node_in "$nid" has_memory; then
+		return 1
+	elif [ "$DAX_KMEM_TEST_DEV" = auto ] && pn_dev_in_use "/dev/$name"; then
+		return 1
+	fi
+
+	pn_record_kmem "$dev" "$driver"
+	if ! pn_online_private "$dev" "$driver" "$nid" 2>/dev/null; then
+		pn_unbind_kmem_private
+		return 1
+	fi
+}
+
+# Restore the device recorded by pn_bind_kmem_private(), if any.
+pn_unbind_kmem_private()
+{
+	local dev=${PN_DEV:-} name
+
+	[ -n "$dev" ] || return 0
+	PN_DEV=
+	name=$(basename "$dev")
+	if [ "$(basename "$(readlink "$dev/driver" 2>/dev/null)")" = kmem ]; then
+		echo unplugged > "$dev/state" 2>/dev/null || return 1
+		if [ -n "$PN_ORIG_PRIVATE" ]; then
+			echo "$PN_ORIG_PRIVATE" > "$dev/private" 2>/dev/null
+		fi
+	fi
+	if [ "$PN_ORIG_DRIVER" = kmem ]; then
+		case "$PN_ORIG_STATE" in
+		unplugged) ;;
+		offline)
+			# kmem cannot restore blocks onlined behind its back.
+			ktap_print_msg "$name was offline and is left unplugged"
+			;;
+		*) echo "$PN_ORIG_STATE" > "$dev/state" 2>/dev/null ;;
+		esac
+		return 0
+	fi
+	if [ -e "$dev/driver" ]; then
+		echo "$name" > /sys/bus/dax/drivers/kmem/unbind 2>/dev/null
+	fi
+	if [ -n "$PN_NEW_ID" ]; then
+		echo "$name" > /sys/bus/dax/drivers/kmem/remove_id 2>/dev/null
+	fi
+	if [ -n "$PN_ORIG_DRIVER" ]; then
+		echo "$name" > "/sys/bus/dax/drivers/$PN_ORIG_DRIVER/bind" \
+			2>/dev/null
+	fi
+	return 0
+}
+
+pn_skip()
+{
+	ktap_skip_all "$1"
+	exit "$KSFT_SKIP"
+}
+
+# An explicitly selected device must provision, auto may find none.
+pn_skip_or_fail()
+{
+	if [ "$DAX_KMEM_TEST_DEV" = auto ]; then
+		pn_skip "$1"
+	fi
+	ktap_exit_fail_msg "$1"
+}
+
+# Print the KTAP header, check the prerequisites and bind a private kmem
+# device, setting D and PN.
+pn_setup()
+{
+	local dev nid
+
+	ktap_print_header
+	[ "$(id -u)" = 0 ] || pn_skip "must be run as root"
+	[ -n "${DAX_KMEM_TEST_DEV:-}" ] ||
+		pn_skip "set DAX_KMEM_TEST_DEV=<daxX.Y|auto> to select a test device"
+	[ -e "$NODE_BASE/has_private_memory" ] ||
+		pn_skip "kernel lacks private memory nodes"
+	modprobe -q kmem 2>/dev/null
+	[ -d /sys/bus/dax/drivers/kmem ] || pn_skip "dax kmem driver unavailable"
+
+	for dev in "$DAX_BASE"/dax*; do
+		if [ "$DAX_KMEM_TEST_DEV" != auto ] &&
+		   [ "$(basename "$dev")" != "$DAX_KMEM_TEST_DEV" ]; then
+			continue
+		fi
+		nid=$(cat "$dev/target_node" 2>/dev/null) || continue
+		[ "$nid" -ge 0 ] 2>/dev/null || continue
+		pn_bind_kmem_private "$dev" || continue
+		D=$dev
+		PN=$nid
+		return 0
+	done
+	pn_skip_or_fail "no private kmem device"
+}

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 58/60] selftests/dax: test memory policy on private nodes
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (56 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 57/60] selftests/dax: test private kmem devices and services Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:35   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 59/60] selftests/dax: test cpuset interaction with " Gregory Price via B4 Relay
                   ` (2 subsequent siblings)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Test the userspace interfaces for private memory with
kmem_private_policy_tool, which reports KTAP results:
  - MPOL_F_PRIVATE_NODES: mbind() binds one private node and
    get_mempolicy() reports the flag, while set_mempolicy(), other
    modes and flags, other masks and HugeTLB mappings are rejected.
  - unflagged, relative and static masks naming the private node use
    only common memory.
  - move_pages() and migrate_pages() move pages to and from the node.
  - reclaim: MADV_PAGEOUT, and node and memcg reclaim splitting
    partially mapped THPs.
  - exhausting private memory fails the fault with SIGBUS and leaves
    another task on the node alone. The test forks the faulting tasks.

Raise the dax selftest timeout to 300 seconds; the policy test faults in
node-sized buffers.

Run with DAX_KMEM_TEST_DEV=<daxX.Y|auto>.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 tools/testing/selftests/dax/.gitignore             |   2 +
 tools/testing/selftests/dax/Makefile               |   4 +
 tools/testing/selftests/dax/config                 |   3 +
 tools/testing/selftests/dax/kmem_private_policy.sh |  58 ++
 .../selftests/dax/kmem_private_policy_tool.c       | 620 +++++++++++++++++++++
 tools/testing/selftests/dax/private_node_common.sh |  28 +
 tools/testing/selftests/dax/settings               |   2 +-
 7 files changed, 716 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/dax/.gitignore b/tools/testing/selftests/dax/.gitignore
new file mode 100644
index 000000000000..fce2950a4e05
--- /dev/null
+++ b/tools/testing/selftests/dax/.gitignore
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0-only
+kmem_private_policy_tool
diff --git a/tools/testing/selftests/dax/Makefile b/tools/testing/selftests/dax/Makefile
index 55d2ba0de8d9..26473b640a4c 100644
--- a/tools/testing/selftests/dax/Makefile
+++ b/tools/testing/selftests/dax/Makefile
@@ -4,6 +4,10 @@ all:
 TEST_PROGS := dax-kmem-hotplug.sh
 TEST_PROGS += kmem_private.sh
 TEST_PROGS += kmem_private_services.sh
+TEST_PROGS += kmem_private_policy.sh
 TEST_FILES := private_node_common.sh
+TEST_GEN_FILES := kmem_private_policy_tool
+
+CFLAGS += -Wall $(KHDR_INCLUDES)
 
 include ../lib.mk
diff --git a/tools/testing/selftests/dax/config b/tools/testing/selftests/dax/config
index 42853ed9e99f..5436c4a195c8 100644
--- a/tools/testing/selftests/dax/config
+++ b/tools/testing/selftests/dax/config
@@ -4,3 +4,6 @@ CONFIG_MEMORY_HOTPLUG=y
 CONFIG_MEMORY_HOTREMOVE=y
 CONFIG_NUMA=y
 CONFIG_COMPACTION=y
+CONFIG_MIGRATION=y
+CONFIG_MEMCG=y
+CONFIG_TRANSPARENT_HUGEPAGE=y
diff --git a/tools/testing/selftests/dax/kmem_private_policy.sh b/tools/testing/selftests/dax/kmem_private_policy.sh
new file mode 100755
index 000000000000..98727927ea25
--- /dev/null
+++ b/tools/testing/selftests/dax/kmem_private_policy.sh
@@ -0,0 +1,58 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# Check memory policy, migration, reclaim and exhaustion on private kmem
+# memory. The checks live in kmem_private_policy_tool, which reports the
+# KTAP plan and results.
+#
+# Needs root, DAX_KMEM_TEST_DEV=<daxX.Y|auto> (see private_node_common.sh)
+# and one common memory node. A memoryless node, swap and a cgroup-v2 memory
+# controller enable a few more checks.
+
+# shellcheck disable=SC1091
+DIR="$(dirname "$(readlink -f "$0")")"
+. "$DIR"/../kselftest/ktap_helpers.sh
+. "$DIR"/private_node_common.sh
+
+TOOL="$DIR/kmem_private_policy_tool"
+CG="$CGROOT/kmem-private-policy-$$"
+
+# shellcheck disable=SC2317 # invoked by the EXIT trap
+cleanup()
+{
+	[ -d "$CG" ] && rmdir "$CG" 2>/dev/null
+	if [ -n "${MEMCG_ENABLED:-}" ]; then
+		echo -memory > "$CGROOT/cgroup.subtree_control" 2>/dev/null
+	fi
+	pn_unbind_kmem_private
+}
+trap cleanup EXIT
+
+pn_setup
+[ -x "$TOOL" ] || pn_skip "kmem_private_policy_tool not built"
+pn_find_common_nodes 1
+pn_require_mask_nodes "$PN" "${COMMON[0]}"
+
+# A memoryless node lets the tool check that a private binding names one node.
+MEMLESS=
+possible=$(cat "$NODE_BASE/possible")
+for ((nid = 0; nid < $(getconf LONG_BIT); nid++)); do
+	pn_nodelist_has "$possible" "$nid" || continue
+	pn_node_in "$nid" has_memory && continue
+	MEMLESS=$nid
+	break
+done
+
+# memcg reclaim needs the tool in a non-root memory cgroup.
+CG_PROCS=/dev/null
+if [ "$(stat -fc %T "$CGROOT" 2>/dev/null)" = cgroup2fs ] &&
+   grep -qw memory "$CGROOT/cgroup.controllers" 2>/dev/null; then
+	if ! grep -qw memory "$CGROOT/cgroup.subtree_control" &&
+	   echo +memory > "$CGROOT/cgroup.subtree_control" 2>/dev/null; then
+		MEMCG_ENABLED=1
+	fi
+	mkdir "$CG" 2>/dev/null && CG_PROCS="$CG/cgroup.procs"
+fi
+
+# An empty MEMLESS omits the optional argument.
+sh -c "echo \$\$ > '$CG_PROCS'; exec '$TOOL' '$PN' '${COMMON[0]}' $MEMLESS"
diff --git a/tools/testing/selftests/dax/kmem_private_policy_tool.c b/tools/testing/selftests/dax/kmem_private_policy_tool.c
new file mode 100644
index 000000000000..0769436b6068
--- /dev/null
+++ b/tools/testing/selftests/dax/kmem_private_policy_tool.c
@@ -0,0 +1,620 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Memory policy, migration, reclaim and exhaustion on a private memory node,
+ * reported as KTAP results. kmem_private_policy.sh prints the KTAP header,
+ * provisions the node and runs:
+ *
+ *   kmem_private_policy_tool <private-nid> <common-nid> [memoryless-nid]
+ */
+#define _GNU_SOURCE
+#include <errno.h>
+#include <linux/mempolicy.h>
+#include <poll.h>
+#include <signal.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/mman.h>
+#include <sys/syscall.h>
+#include <sys/wait.h>
+#include <time.h>
+#include <unistd.h>
+
+#include "../kselftest.h"
+
+#ifndef MLOCK_ONFAULT
+#define MLOCK_ONFAULT 1
+#endif
+
+#ifndef MPOL_F_PRIVATE_NODES
+#define MPOL_F_PRIVATE_NODES (1 << 12)
+#endif
+
+#define NR_PAGES 64
+#define PATTERN 0xab
+#define NR_THPS 8
+/* The kernel reads maxnode - 1 bits. */
+#define MAXNODE (8 * sizeof(unsigned long) + 1)
+
+enum { FAIL, PASS, SKIP };
+
+static long page_size;
+static size_t huge_len;
+
+static void report(int result, const char *name)
+{
+	if (result == SKIP)
+		ksft_test_result_skip("%s\n", name);
+	else
+		ksft_test_result(result == PASS, "%s\n", name);
+}
+
+static void *anon_map(size_t len)
+{
+	void *ptr = mmap(NULL, len, PROT_READ | PROT_WRITE,
+			 MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+
+	return ptr == MAP_FAILED ? NULL : ptr;
+}
+
+/* How many of the @nr pages at @addr are on @nid, by a move_pages() query. */
+static long pages_on_node(void *addr, long nr, int nid)
+{
+	void **pages = calloc(nr, sizeof(*pages));
+	int *status = calloc(nr, sizeof(*status));
+	long i, count = -1;
+
+	if (pages && status) {
+		for (i = 0; i < nr; i++)
+			pages[i] = addr + i * page_size;
+		if (!syscall(SYS_move_pages, 0, nr, pages, NULL, status, 0UL)) {
+			count = 0;
+			for (i = 0; i < nr; i++)
+				count += status[i] == nid;
+		}
+	}
+	free(pages);
+	free(status);
+	return count;
+}
+
+static long mbind_mask(void *addr, size_t len, int mode, unsigned long mask)
+{
+	return syscall(SYS_mbind, addr, len, mode, &mask, MAXNODE, 0UL);
+}
+
+static long bind_private(void *addr, size_t len, int nid)
+{
+	return mbind_mask(addr, len, MPOL_BIND | MPOL_F_PRIVATE_NODES,
+			  1UL << nid);
+}
+
+static int rejected(long rc)
+{
+	return rc == -1 && errno == EINVAL;
+}
+
+static long read_long(const char *path)
+{
+	FILE *file = fopen(path, "r");
+	long val = -1;
+
+	if (file) {
+		if (fscanf(file, "%ld", &val) != 1)
+			val = -1;
+		fclose(file);
+	}
+	return val;
+}
+
+/* A field of /sys/devices/system/node/node<nid>/meminfo, in kB. */
+static long node_meminfo(int nid, const char *field)
+{
+	char path[64], line[256], key[64];
+	FILE *file;
+	long val = -1;
+
+	snprintf(path, sizeof(path), "/sys/devices/system/node/node%d/meminfo",
+		 nid);
+	snprintf(key, sizeof(key), " %s:", field);
+	file = fopen(path, "r");
+	if (!file)
+		return -1;
+	while (fgets(line, sizeof(line), file)) {
+		char *pos = strstr(line, key);
+
+		if (pos) {
+			val = atol(pos + strlen(key));
+			break;
+		}
+	}
+	fclose(file);
+	return val;
+}
+
+static int thp_disabled(void)
+{
+	char mode[128] = "";
+	FILE *file = fopen("/sys/kernel/mm/transparent_hugepage/enabled", "r");
+
+	if (!file)
+		return 1;
+	if (!fgets(mode, sizeof(mode), file))
+		mode[0] = '\0';
+	fclose(file);
+	return strstr(mode, "[never]") != NULL;
+}
+
+/*
+ * Reclaim through @path splits partially unmapped THPs on a private node,
+ * returning the unmapped subpages.
+ */
+static int thp_split_reclaimed(int nid, const char *path)
+{
+	size_t len = NR_THPS * huge_len;
+	char stat[256], *map, *buf;
+	long before, partial;
+	FILE *file;
+	int i;
+
+	if (!huge_len)
+		return SKIP;
+	snprintf(stat, sizeof(stat),
+		 "/sys/kernel/mm/transparent_hugepage/hugepages-%zukB/stats/nr_anon_partially_mapped",
+		 huge_len >> 10);
+	before = read_long(stat);
+	if (before < 0)
+		return SKIP;
+	map = anon_map(len + huge_len);
+	if (!map)
+		return FAIL;
+	buf = (char *)(((unsigned long)map + huge_len - 1) & ~(huge_len - 1));
+	if (madvise(buf, len, MADV_HUGEPAGE) || bind_private(buf, len, nid)) {
+		munmap(map, len + huge_len);
+		return SKIP;
+	}
+	memset(buf, PATTERN, len);
+	if (pages_on_node(buf, len / page_size, nid) !=
+	    (long)(len / page_size)) {
+		munmap(map, len + huge_len);
+		return FAIL;
+	}
+	for (i = 0; i < NR_THPS; i++)
+		madvise(buf + i * huge_len + huge_len / 2, huge_len / 2,
+			MADV_DONTNEED);
+
+	/* With THP enabled, a private binding must get its THPs. */
+	partial = read_long(stat) - before;
+	if (partial < NR_THPS) {
+		munmap(map, len + huge_len);
+		return thp_disabled() ? SKIP : FAIL;
+	}
+	file = fopen(path, "w");
+	if (file) {
+		fputs("64M", file);
+		fclose(file);
+	}
+	partial = read_long(stat) - before;
+	munmap(map, len + huge_len);
+	return partial <= NR_THPS / 4 ? PASS : FAIL;
+}
+
+static int swap_active(void)
+{
+	char line[256];
+	FILE *file = fopen("/proc/swaps", "r");
+	int lines = 0;
+
+	if (!file)
+		return 0;
+	while (fgets(line, sizeof(line), file))
+		lines++;
+	fclose(file);
+	return lines > 1;
+}
+
+/* Reclaim advice pages out private memory, given active swap. */
+static int pageout_acts(void *addr, size_t len, int nid)
+{
+	if (!swap_active())
+		return SKIP;
+	if (bind_private(addr, len, nid))
+		return FAIL;
+	memset(addr, PATTERN, len);
+	if (pages_on_node(addr, NR_PAGES, nid) != NR_PAGES ||
+	    madvise(addr, len, MADV_PAGEOUT))
+		return FAIL;
+	/* Reclaim may skip a few pages, for example ones still in a batch. */
+	return pages_on_node(addr, NR_PAGES, nid) <= NR_PAGES / 4 ? PASS : FAIL;
+}
+
+/* Find memory.reclaim of the non-root cgroup this task runs in. */
+static int memcg_reclaim_path(char *path, size_t size)
+{
+	/* Short enough that the memory.reclaim path always fits @path. */
+	char line[2048], *cgroup = NULL;
+	FILE *file = fopen("/proc/self/cgroup", "r");
+
+	if (!file)
+		return 0;
+	while (fgets(line, sizeof(line), file)) {
+		if (!strncmp(line, "0::/", 4)) {
+			line[strcspn(line, "\n")] = 0;
+			cgroup = line + 3;
+			break;
+		}
+	}
+	fclose(file);
+	if (!cgroup || !strcmp(cgroup, "/"))
+		return 0;
+	snprintf(path, size, "/sys/fs/cgroup%s/memory.reclaim", cgroup);
+	return !access(path, W_OK);
+}
+
+/*
+ * A relative or static policy whose mask names the private node must still
+ * place memory on common nodes, and keep the user's mask.
+ */
+static int common_policy(void *addr, size_t len, int nid, int requested_mode,
+			 unsigned long mask, int task_policy)
+{
+	unsigned long reported = 0;
+	int mode = 0;
+	long rc;
+
+	if (task_policy) {
+		if (syscall(SYS_mbind, addr, len, MPOL_DEFAULT, NULL, 0UL, 0UL))
+			return FAIL;
+		rc = syscall(SYS_set_mempolicy, requested_mode, &mask, MAXNODE);
+	} else {
+		rc = mbind_mask(addr, len, requested_mode, mask);
+	}
+	if (rc)
+		return FAIL;
+
+	rc = syscall(SYS_get_mempolicy, &mode, &reported, MAXNODE,
+		     task_policy ? NULL : addr, task_policy ? 0 : MPOL_F_ADDR);
+	if (task_policy)
+		syscall(SYS_set_mempolicy, MPOL_DEFAULT, NULL, 0UL);
+	if (rc || mode != requested_mode || reported != mask ||
+	    madvise(addr, len, MADV_DONTNEED))
+		return FAIL;
+	memset(addr, PATTERN, len);
+	return pages_on_node(addr, NR_PAGES, nid) == 0 ? PASS : FAIL;
+}
+
+/* The flag binds exactly one private node and excludes the other modes. */
+static int private_flag_validated(void *addr, size_t len,
+				  unsigned long private_mask,
+				  unsigned long common_mask)
+{
+	const struct {
+		int mode;
+		unsigned long mask;
+	} cases[] = {
+		{ MPOL_BIND | MPOL_F_PRIVATE_NODES | MPOL_F_STATIC_NODES,
+		  private_mask },
+		{ MPOL_BIND | MPOL_F_PRIVATE_NODES | MPOL_F_RELATIVE_NODES,
+		  private_mask },
+		{ MPOL_BIND | MPOL_F_PRIVATE_NODES | MPOL_F_NUMA_BALANCING,
+		  private_mask },
+		{ MPOL_PREFERRED | MPOL_F_PRIVATE_NODES, private_mask },
+		{ MPOL_BIND | MPOL_F_PRIVATE_NODES, common_mask },
+		{ MPOL_BIND | MPOL_F_PRIVATE_NODES, private_mask | common_mask },
+	};
+	size_t i;
+
+	for (i = 0; i < ARRAY_SIZE(cases); i++) {
+		errno = 0;
+		if (!rejected(mbind_mask(addr, len, cases[i].mode,
+					 cases[i].mask)))
+			return FAIL;
+	}
+	return PASS;
+}
+
+/* Run @fn in a child, so its effects on this process do not leak. */
+static int in_child(int (*fn)(int), int nid)
+{
+	int status;
+	pid_t pid = fork();
+
+	if (pid < 0)
+		return FAIL;
+	if (!pid)
+		_exit(fn(nid));
+	if (waitpid(pid, &status, 0) != pid || !WIFEXITED(status))
+		return FAIL;
+	return WEXITSTATUS(status);
+}
+
+/* migrate_pages(2) moves a task's pages onto the private node. */
+static int migrate_to_private(int nid)
+{
+	unsigned long old_nodes, new_nodes = 1UL << nid;
+	char *addr = anon_map(NR_PAGES * page_size);
+	void *page;
+	int status;
+
+	if (!addr)
+		return FAIL;
+	memset(addr, PATTERN, NR_PAGES * page_size);
+	page = addr;
+	if (syscall(SYS_move_pages, 0, 1UL, &page, NULL, &status, 0UL) ||
+	    status < 0 || status == nid)
+		return FAIL;
+	old_nodes = 1UL << status;
+	if (syscall(SYS_migrate_pages, 0, MAXNODE, &old_nodes, &new_nodes) < 0)
+		return FAIL;
+	return pages_on_node(addr, NR_PAGES, nid) >= NR_PAGES * 3 / 4 ?
+	       PASS : FAIL;
+}
+
+static double now(void)
+{
+	struct timespec ts;
+
+	clock_gettime(CLOCK_MONOTONIC, &ts);
+	return ts.tv_sec + ts.tv_nsec / 1e9;
+}
+
+/*
+ * Fork a child that faults @kb of locked memory bound to the private node,
+ * so that reclaim cannot make room, then writes to @ready and sleeps.
+ */
+static pid_t start_faulter(int nid, long kb, int ready)
+{
+	size_t len = (size_t)kb << 10, off;
+	char *addr;
+	pid_t pid;
+
+	pid = fork();
+	if (pid)
+		return pid;
+	addr = mmap(NULL, len, PROT_READ | PROT_WRITE,
+		    MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
+	if (addr == MAP_FAILED || bind_private(addr, len, nid) ||
+	    madvise(addr, len, MADV_NOHUGEPAGE) ||
+	    syscall(SYS_mlock2, addr, len, MLOCK_ONFAULT))
+		_exit(2);
+	for (off = 0; off < len; off += page_size)
+		addr[off] = 1;
+	if (write(ready, "", 1) != 1)
+		_exit(2);
+	pause();
+	_exit(0);
+}
+
+/* Wait up to @secs for a faulter to write to @ready. */
+static int faulter_ready(int ready, int secs)
+{
+	struct pollfd pfd = { .fd = ready, .events = POLLIN };
+	char byte;
+
+	return poll(&pfd, 1, secs * 1000) == 1 && read(ready, &byte, 1) == 1;
+}
+
+/* Wait up to @secs for @pid to exit and return its status, or -1. */
+static int wait_exit(pid_t pid, double secs)
+{
+	double end = now() + secs;
+	int status;
+
+	do {
+		if (waitpid(pid, &status, WNOHANG) == pid)
+			return status;
+		usleep(100000);
+	} while (now() < end);
+	return -1;
+}
+
+static int alive(pid_t pid)
+{
+	return waitpid(pid, NULL, WNOHANG) == 0;
+}
+
+static void stop(pid_t pid)
+{
+	kill(pid, SIGKILL);
+	waitpid(pid, NULL, 0);
+}
+
+/*
+ * Exhausting private memory fails the fault with SIGBUS, as an exhausted
+ * HugeTLB pool does, and leaves other tasks on the node alone.
+ */
+static void private_exhaustion(int nid)
+{
+	const char *name = "exhausting private memory fails the fault with SIGBUS";
+	long free_kb = node_meminfo(nid, "MemFree");
+	long total_kb = node_meminfo(nid, "MemTotal");
+	int pipefd[2], status = -1, result = FAIL;
+	pid_t holder, faulter;
+
+	/* Strict overcommit refuses the oversized mapping before it faults. */
+	if (read_long("/proc/sys/vm/overcommit_memory") == 2) {
+		report(SKIP, name);
+		return;
+	}
+	if (free_kb <= 0 || total_kb <= 0 || pipe(pipefd)) {
+		report(FAIL, name);
+		return;
+	}
+
+	/* Another task holding memory on the node must be left alone. */
+	holder = start_faulter(nid, free_kb * 6 / 10, pipefd[1]);
+	if (holder > 0 && faulter_ready(pipefd[0], 60)) {
+		faulter = start_faulter(nid, total_kb * 2, pipefd[1]);
+		if (faulter > 0) {
+			status = wait_exit(faulter, 60);
+			if (status == -1)
+				stop(faulter);
+		}
+		if (status != -1 && WIFSIGNALED(status) &&
+		    WTERMSIG(status) == SIGBUS && alive(holder))
+			result = PASS;
+	}
+	report(result, name);
+
+	if (holder > 0)
+		stop(holder);
+	close(pipefd[0]);
+	close(pipefd[1]);
+}
+
+int main(int argc, char **argv)
+{
+	unsigned long private_mask, common_mask, mixed_mask, allowed = 0;
+	void *ordinary, *bound, *probe, *mixed, *paged, *huge;
+	int nid, common, memless = -1, mode, status, result;
+	char path[4096];
+	void *pages[1];
+	int nodes[1];
+	long len, rc;
+
+	if (argc != 3 && argc != 4)
+		ksft_exit_fail_msg("usage: %s <private-nid> <common-nid> [memoryless-nid]\n",
+				   argv[0]);
+	nid = atoi(argv[1]);
+	common = atoi(argv[2]);
+	if (argc == 4)
+		memless = atoi(argv[3]);
+
+	page_size = sysconf(_SC_PAGESIZE);
+	rc = read_long("/sys/kernel/mm/transparent_hugepage/hpage_pmd_size");
+	huge_len = rc > 0 ? rc : 0;
+	len = NR_PAGES * page_size;
+	private_mask = 1UL << nid;
+	common_mask = 1UL << common;
+	mixed_mask = private_mask | common_mask;
+
+	ordinary = anon_map(len);
+	bound = anon_map(len);
+	probe = anon_map(len);
+	mixed = anon_map(len);
+	paged = anon_map(len);
+	if (!ordinary || !bound || !probe || !mixed || !paged)
+		ksft_exit_fail_msg("mmap failed\n");
+
+	ksft_set_plan(20);
+
+	memset(ordinary, PATTERN, len);
+	report(pages_on_node(ordinary, NR_PAGES, nid) == 0 ? PASS : FAIL,
+	       "ordinary allocation stays off private memory");
+
+	rc = syscall(SYS_get_mempolicy, &mode, &allowed, MAXNODE, NULL,
+		     MPOL_F_MEMS_ALLOWED);
+	report(!rc && (allowed & private_mask) ? PASS : FAIL,
+	       "mems_allowed reports the private node");
+
+	errno = 0;
+	rc = syscall(SYS_set_mempolicy, MPOL_BIND | MPOL_F_PRIVATE_NODES,
+		     &private_mask, MAXNODE);
+	report(rejected(rc) ? PASS : FAIL,
+	       "set_mempolicy() rejects MPOL_F_PRIVATE_NODES");
+	syscall(SYS_set_mempolicy, MPOL_DEFAULT, NULL, 0UL);
+
+	rc = bind_private(bound, len, nid);
+	memset(bound, PATTERN, len);
+	report(!rc && pages_on_node(bound, NR_PAGES, nid) == NR_PAGES &&
+	       pages_on_node(bound, NR_PAGES, common) == 0 ? PASS : FAIL,
+	       "a private binding places memory on the private node");
+
+	mode = 0;
+	rc = syscall(SYS_get_mempolicy, &mode, NULL, 0UL, bound, MPOL_F_ADDR);
+	report(!rc && mode == (MPOL_BIND | MPOL_F_PRIVATE_NODES) ? PASS : FAIL,
+	       "get_mempolicy() reports MPOL_F_PRIVATE_NODES");
+
+	/* A mixed mask is an ordinary binding narrowed to common memory. */
+	rc = mbind_mask(mixed, len, MPOL_BIND, mixed_mask);
+	memset(mixed, PATTERN, len);
+	report(!rc && pages_on_node(mixed, NR_PAGES, common) == NR_PAGES ?
+	       PASS : FAIL,
+	       "an unflagged mixed mask uses only common memory");
+
+	/* Without the flag, a private-only mask has no usable node. */
+	errno = 0;
+	report(rejected(mbind_mask(probe, len, MPOL_BIND, private_mask)) ?
+	       PASS : FAIL, "an unflagged private-only mask is rejected");
+
+	report(private_flag_validated(probe, len, private_mask, common_mask),
+	       "MPOL_F_PRIVATE_NODES rejects other modes, flags and masks");
+
+	/* Adding a memoryless node to a private binding must fail. */
+	if (memless < 0) {
+		result = SKIP;
+	} else {
+		errno = 0;
+		result = rejected(mbind_mask(probe, len,
+					     MPOL_BIND | MPOL_F_PRIVATE_NODES,
+					     private_mask | (1UL << memless))) ?
+			 PASS : FAIL;
+	}
+	report(result, "a private binding names exactly one node");
+
+	/* Private nodes cannot hold HugeTLB pages. */
+	huge = huge_len ? mmap(NULL, huge_len, PROT_READ | PROT_WRITE,
+			       MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB |
+			       MAP_NORESERVE, -1, 0) : MAP_FAILED;
+	if (huge == MAP_FAILED) {
+		result = SKIP;
+	} else {
+		errno = 0;
+		result = rejected(bind_private(huge, huge_len, nid)) ?
+			 PASS : FAIL;
+		munmap(huge, huge_len);
+	}
+	report(result, "a private binding on HugeTLB memory is rejected");
+
+	/* Relative bits are positions in the common-node set, not node IDs. */
+	report(common_policy(probe, len, nid, MPOL_BIND | MPOL_F_RELATIVE_NODES,
+			     private_mask, 1),
+	       "a relative task policy stays on common memory");
+	report(common_policy(probe, len, nid, MPOL_BIND | MPOL_F_RELATIVE_NODES,
+			     private_mask, 0),
+	       "a relative binding stays on common memory");
+	/* Filtering the effective policy must not alter its user mask. */
+	report(common_policy(probe, len, nid, MPOL_BIND | MPOL_F_STATIC_NODES,
+			     mixed_mask, 1),
+	       "a static task policy keeps its mask and uses common memory");
+
+	pages[0] = ordinary;
+	nodes[0] = nid;
+	rc = syscall(SYS_move_pages, 0, 1UL, pages, nodes, &status,
+		     MPOL_MF_MOVE);
+	report(!rc && pages_on_node(ordinary, NR_PAGES, nid) == 1 ? PASS : FAIL,
+	       "move_pages() moves a page to the private node");
+
+	pages[0] = bound;
+	nodes[0] = common;
+	rc = syscall(SYS_move_pages, 0, 1UL, pages, nodes, &status,
+		     MPOL_MF_MOVE);
+	report(!rc && pages_on_node(bound, NR_PAGES, common) == 1 ? PASS : FAIL,
+	       "move_pages() moves a private page to common memory");
+
+	report(in_child(migrate_to_private, nid),
+	       "migrate_pages() moves pages to the private node");
+
+	report(pageout_acts(paged, len, nid),
+	       "MADV_PAGEOUT pages out private memory");
+
+	snprintf(path, sizeof(path), "/sys/devices/system/node/node%d/reclaim",
+		 nid);
+	report(thp_split_reclaimed(nid, path),
+	       "node reclaim splits partially mapped private THPs");
+
+	/* memcg reclaim may page out the mappings above, so it runs late. */
+	report(memcg_reclaim_path(path, sizeof(path)) ?
+	       thp_split_reclaimed(nid, path) : SKIP,
+	       "memcg reclaim splits partially mapped private THPs");
+
+	munmap(paged, len);
+	munmap(probe, len);
+	munmap(mixed, len);
+	munmap(bound, len);
+	munmap(ordinary, len);
+
+	private_exhaustion(nid);
+
+	ksft_finished();
+}
diff --git a/tools/testing/selftests/dax/private_node_common.sh b/tools/testing/selftests/dax/private_node_common.sh
index 57c5231e768b..cd03d9489822 100644
--- a/tools/testing/selftests/dax/private_node_common.sh
+++ b/tools/testing/selftests/dax/private_node_common.sh
@@ -12,6 +12,7 @@
 
 DAX_BASE=/sys/bus/dax/devices
 NODE_BASE=/sys/devices/system/node
+CGROOT=/sys/fs/cgroup
 
 # Whether nodelist $1 (e.g. "0-2,4") contains node $2.
 pn_nodelist_has()
@@ -206,3 +207,30 @@ pn_setup()
 	done
 	pn_skip_or_fail "no private kmem device"
 }
+
+# Set COMMON to the first $1 nodes with common memory.
+pn_find_common_nodes()
+{
+	local node nid
+
+	COMMON=()
+	for node in "$NODE_BASE"/node[0-9]*; do
+		nid=${node##*node}
+		pn_node_in "$nid" has_common_memory || continue
+		COMMON+=("$nid")
+		[ "${#COMMON[@]}" = "$1" ] && return 0
+	done
+	pn_skip "requires $1 common memory node(s)"
+}
+
+# The helper tools pass node masks as a single unsigned long.
+pn_require_mask_nodes()
+{
+	local nid bits
+
+	bits=$(getconf LONG_BIT)
+	for nid in "$@"; do
+		[ "$nid" -lt "$bits" ] ||
+			pn_skip "node $nid does not fit the helper node mask"
+	done
+}
diff --git a/tools/testing/selftests/dax/settings b/tools/testing/selftests/dax/settings
index ba4d85f74cd6..694d70710ff0 100644
--- a/tools/testing/selftests/dax/settings
+++ b/tools/testing/selftests/dax/settings
@@ -1 +1 @@
-timeout=90
+timeout=300

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 59/60] selftests/dax: test cpuset interaction with private nodes
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (57 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 58/60] selftests/dax: test memory policy on private nodes Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:35   ` sashiko-bot
  2026-11-20 18:09 ` [PATCH 60/60] Documentation/mm: describe private memory nodes Gregory Price via B4 Relay
  2026-11-20 18:59 ` [PATCH 00/60] Private NUMA nodes Gregory Price (Meta)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Test private nodes in cgroup-v2 cpusets with kmem_private_rebind_tool:
  - cpuset.mems may name a private node, a private-only request
    inherits the parent's common nodes, and ordinary allocations in the
    cpuset stay on common memory.
  - policies across cpuset.mems changes and task attachment: interleave
    and positional rebinds never reach private memory, and a private
    binding keeps its node while it is allowed, evacuates to common
    memory when it is revoked, and returns when it is restored.
  - policies across hotplug: a private binding falls back while its
    node is offline or common and resumes once the node is private
    again, and a common binding falls back when its node becomes
    private.

Run with DAX_KMEM_TEST_DEV=<daxX.Y|auto>. The rebind tests need two
common memory nodes.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 tools/testing/selftests/dax/.gitignore             |   1 +
 tools/testing/selftests/dax/Makefile               |   3 +
 tools/testing/selftests/dax/config                 |   1 +
 tools/testing/selftests/dax/kmem_private_cpuset.sh |  70 +++
 tools/testing/selftests/dax/kmem_private_rebind.sh | 301 +++++++++++++
 .../selftests/dax/kmem_private_rebind_tool.c       | 495 +++++++++++++++++++++
 tools/testing/selftests/dax/private_node_common.sh |  32 ++
 7 files changed, 903 insertions(+)

diff --git a/tools/testing/selftests/dax/.gitignore b/tools/testing/selftests/dax/.gitignore
index fce2950a4e05..bb9c006203d0 100644
--- a/tools/testing/selftests/dax/.gitignore
+++ b/tools/testing/selftests/dax/.gitignore
@@ -1,2 +1,3 @@
 # SPDX-License-Identifier: GPL-2.0-only
 kmem_private_policy_tool
+kmem_private_rebind_tool
diff --git a/tools/testing/selftests/dax/Makefile b/tools/testing/selftests/dax/Makefile
index 26473b640a4c..de1327f3a7a7 100644
--- a/tools/testing/selftests/dax/Makefile
+++ b/tools/testing/selftests/dax/Makefile
@@ -5,8 +5,11 @@ TEST_PROGS := dax-kmem-hotplug.sh
 TEST_PROGS += kmem_private.sh
 TEST_PROGS += kmem_private_services.sh
 TEST_PROGS += kmem_private_policy.sh
+TEST_PROGS += kmem_private_cpuset.sh
+TEST_PROGS += kmem_private_rebind.sh
 TEST_FILES := private_node_common.sh
 TEST_GEN_FILES := kmem_private_policy_tool
+TEST_GEN_FILES += kmem_private_rebind_tool
 
 CFLAGS += -Wall $(KHDR_INCLUDES)
 
diff --git a/tools/testing/selftests/dax/config b/tools/testing/selftests/dax/config
index 5436c4a195c8..54ef671020c5 100644
--- a/tools/testing/selftests/dax/config
+++ b/tools/testing/selftests/dax/config
@@ -7,3 +7,4 @@ CONFIG_COMPACTION=y
 CONFIG_MIGRATION=y
 CONFIG_MEMCG=y
 CONFIG_TRANSPARENT_HUGEPAGE=y
+CONFIG_CPUSETS=y
diff --git a/tools/testing/selftests/dax/kmem_private_cpuset.sh b/tools/testing/selftests/dax/kmem_private_cpuset.sh
new file mode 100755
index 000000000000..2ae64b973f01
--- /dev/null
+++ b/tools/testing/selftests/dax/kmem_private_cpuset.sh
@@ -0,0 +1,70 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# Check that cgroup-v2 cpuset.mems may name a private kmem node without
+# exposing it to ordinary allocations.
+#
+# Needs root, DAX_KMEM_TEST_DEV=<daxX.Y|auto> (see private_node_common.sh),
+# one common memory node and the cgroup-v2 cpuset controller.
+
+# shellcheck disable=SC1091
+DIR="$(dirname "$(readlink -f "$0")")"
+. "$DIR"/../kselftest/ktap_helpers.sh
+. "$DIR"/private_node_common.sh
+
+TOOL="$DIR/kmem_private_rebind_tool"
+CG="$CGROOT/kmem-private-cpuset-$$"
+
+# shellcheck disable=SC2317 # invoked by the EXIT trap
+cleanup()
+{
+	pn_cpuset_cleanup "$CG"
+	pn_unbind_kmem_private
+}
+trap cleanup EXIT
+
+pn_setup
+[ -x "$TOOL" ] || pn_skip "kmem_private_rebind_tool not built"
+pn_find_common_nodes 1
+pn_require_mask_nodes "$PN" "${COMMON[0]}"
+pn_cpuset_cgroup "$CG"
+
+ktap_set_plan 4
+
+name="mixed common/private cpuset.mems is accepted"
+if echo "${COMMON[0]},$PN" > "$CG/cpuset.mems" 2>/dev/null &&
+   pn_nodelist_has "$(cat "$CG/cpuset.mems.effective")" "$PN"; then
+	ktap_test_pass "$name"
+else
+	ktap_test_fail "$name"
+fi
+
+name="a private-only request keeps the node and inherits common memory"
+if echo "$PN" > "$CG/cpuset.mems" 2>/dev/null &&
+   pn_nodelist_has "$(cat "$CG/cpuset.mems")" "$PN" &&
+   pn_nodelist_has "$(cat "$CG/cpuset.mems.effective")" "$PN" &&
+   pn_nodelist_has "$(cat "$CG/cpuset.mems.effective")" "${COMMON[0]}"; then
+	ktap_test_pass "$name"
+else
+	ktap_test_fail "$name"
+fi
+
+name="ordinary allocation in the cpuset stays on common memory"
+out=$(timeout 60 sh -c "echo \$\$ > '$CG/cgroup.procs'; exec '$TOOL' \
+	ordinary '$PN' '${COMMON[0]}' '${COMMON[0]}'")
+if [ "$out" = "private=0" ]; then
+	ktap_test_pass "$name"
+else
+	ktap_print_msg "$out"
+	ktap_test_fail "$name"
+fi
+
+name="removing the private node revokes it"
+if echo "${COMMON[0]}" > "$CG/cpuset.mems" 2>/dev/null &&
+   ! pn_nodelist_has "$(cat "$CG/cpuset.mems.effective")" "$PN"; then
+	ktap_test_pass "$name"
+else
+	ktap_test_fail "$name"
+fi
+
+ktap_finished
diff --git a/tools/testing/selftests/dax/kmem_private_rebind.sh b/tools/testing/selftests/dax/kmem_private_rebind.sh
new file mode 100755
index 000000000000..5d39be04f179
--- /dev/null
+++ b/tools/testing/selftests/dax/kmem_private_rebind.sh
@@ -0,0 +1,301 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# Check memory policies on private kmem memory while a cgroup-v2 cpuset is
+# reshaped, tasks move between cpusets, and the node is hotplugged. Each case
+# starts kmem_private_rebind_tool, changes the cpuset or the node, and sends
+# the tool SIGUSR1 to check where its memory went.
+#
+# Needs root, DAX_KMEM_TEST_DEV=<daxX.Y|auto> (see private_node_common.sh),
+# two common memory nodes and the cgroup-v2 cpuset controller.
+
+# shellcheck disable=SC1091
+DIR="$(dirname "$(readlink -f "$0")")"
+. "$DIR"/../kselftest/ktap_helpers.sh
+. "$DIR"/private_node_common.sh
+
+TOOL="$DIR/kmem_private_rebind_tool"
+CG="$CGROOT/kmem-private-rebind-$$"
+CGCOMMON="$CGROOT/kmem-private-rebind-common-$$"
+OUT=$(mktemp)
+
+# CHILD is the timeout wrapper and TARGET the tool, which may be its child.
+stop_tool()
+{
+	[ -n "${TARGET:-}" ] && kill "$TARGET" 2>/dev/null
+	[ -n "${CHILD:-}" ] && kill "$CHILD" 2>/dev/null
+	[ -n "${CHILD:-}" ] && wait "$CHILD" 2>/dev/null
+	TARGET=
+	CHILD=
+}
+
+# shellcheck disable=SC2317 # invoked by the EXIT trap
+cleanup()
+{
+	stop_tool
+	rm -f "$OUT"
+	pn_cpuset_cleanup "$CG" "$CGCOMMON"
+	pn_unbind_kmem_private
+}
+trap cleanup EXIT
+
+wait_output()
+{
+	local pattern=$1 i
+
+	for ((i = 0; i < 100; i++)); do
+		grep -q "$pattern" "$OUT" && return 0
+		kill -0 "$CHILD" 2>/dev/null || break
+		sleep 0.1
+	done
+	return 1
+}
+
+# Start the tool in mode $1, inside cgroup $2 if given, and wait until it is
+# ready for the first SIGUSR1.
+start_tool()
+{
+	local mode=$1 cg=${2:-}
+
+	: > "$OUT"
+	if [ -n "$cg" ]; then
+		timeout 60 sh -ec "echo \$\$ > '$cg/cgroup.procs'; exec '$TOOL' \
+			'$mode' '$PN' '${COMMON[0]}' '${COMMON[1]}'" > "$OUT" 2>&1 &
+	else
+		timeout 60 "$TOOL" "$mode" "$PN" "${COMMON[0]}" "${COMMON[1]}" \
+			> "$OUT" 2>&1 &
+	fi
+	CHILD=$!
+	wait_output '^ready pid=' || return 1
+	TARGET=$(sed -n 's/^ready pid=//p' "$OUT")
+}
+
+# Signal the tool to check stage $1 and report its result as test $2.
+# Returns 1 if the tool did not answer.
+report_stage()
+{
+	local stage=$1 name=$2 line
+
+	if ! kill -USR1 "$TARGET" || ! wait_output "^stage=$stage "; then
+		ktap_print_msg "no answer from the tool"
+		ktap_test_fail "$name"
+		return 1
+	fi
+	line=$(grep "^stage=$stage " "$OUT")
+	ktap_print_msg "$line"
+	if grep -q 'ok=1$' <<<"$line"; then
+		ktap_test_pass "$name"
+	else
+		ktap_test_fail "$name"
+	fi
+}
+
+# Run task policy mode $1 in the cpuset across one cpuset.mems change. The
+# tool's result is the last line of $OUT.
+run_task_policy()
+{
+	echo "${COMMON[0]},$PN" > "$CG/cpuset.mems"
+	if ! start_tool "$1" "$CG" ||
+	   ! echo "${COMMON[1]},$PN" > "$CG/cpuset.mems" ||
+	   ! kill -USR1 "$TARGET" || ! wait "$CHILD"; then
+		stop_tool
+		echo "tool failed" >> "$OUT"
+		return 1
+	fi
+	CHILD=
+	TARGET=
+}
+
+# Allow or revoke the private node at stage $2, by attaching the tool to
+# another cpuset or by writing mask $3 to cpuset.mems, for action $1.
+transition_cpuset()
+{
+	local action=$1 stage=$2 mask=$3
+
+	case "$action:$stage" in
+	attach:1) echo "$TARGET" > "$CGCOMMON/cgroup.procs" ;;
+	attach:*) echo "$TARGET" > "$CG/cgroup.procs" ;;
+	*) echo "$mask" > "$CG/cpuset.mems" ;;
+	esac
+}
+
+# A private binding in tool mode $1 while its cpuset allows, revokes and
+# allows the private node again, by action reshape or attach.
+run_private_transitions()
+{
+	local mode=$1 label=$2 action=${3:-reshape}
+	local stage mask title cg=
+	local titles=("keeps private pages and placement while the node stays allowed"
+		"evacuates private pages and uses common placement when access is revoked"
+		"restores the original private request when access returns")
+
+	echo "${COMMON[0]},$PN" > "$CG/cpuset.mems"
+	if [ "$action" = attach ]; then
+		echo "${COMMON[1]},$PN" > "$CG/cpuset.mems"
+	fi
+	# Shared-policy parents and task-attachment tools start at the root.
+	if [ "$mode" = private-anon ] && [ "$action" = reshape ]; then
+		cg=$CG
+	fi
+	if ! start_tool "$mode" "$cg"; then
+		ktap_print_msg "tool setup failed"
+		for title in "${titles[@]}"; do
+			ktap_test_fail "$label $title"
+		done
+		stop_tool
+		return
+	fi
+	if [ "$mode" = private-shared ] && [ "$action" = reshape ]; then
+		echo "$TARGET" > "$CG/cgroup.procs"
+	fi
+	for stage in 0 1 2; do
+		mask=${COMMON[1]}
+		[ "$stage" = 1 ] || mask+=",$PN"
+		if ! transition_cpuset "$action" "$stage" "$mask"; then
+			ktap_print_msg "cannot change the cpuset"
+			ktap_test_fail "$label ${titles[$stage]}"
+			continue
+		fi
+		report_stage "$stage" "$label ${titles[$stage]}"
+	done
+	wait "$CHILD"
+	CHILD=
+	TARGET=
+}
+
+# Take the private node through stage $1: offline, common, offline, private.
+retype_node()
+{
+	case "$1" in
+	0|2) echo unplugged > "$D/state" ;;
+	1) echo 0 > "$D/private" && echo online_movable > "$D/state" ;;
+	3) echo 1 > "$D/private" && echo online_movable > "$D/state" ;;
+	esac
+}
+
+# A private binding in tool mode $1 while its node is hotplugged and retyped.
+run_private_retype()
+{
+	local mode=$1 label=$2
+	local stage name ready=false
+	local titles=("falls back while the private node is offline"
+		"binds the original node when it returns as common memory"
+		"falls back again after draining the common node"
+		"restores private placement when the node returns as private memory")
+
+	echo "${COMMON[1]},$PN" > "$CG/cpuset.mems"
+	start_tool "$mode" "$CG" && ready=true
+	for stage in 0 1 2 3; do
+		name="$label ${titles[$stage]}"
+		if [ "$ready" != true ]; then
+			ktap_print_msg "tool is not running"
+			ktap_test_fail "$name"
+		elif ! retype_node "$stage"; then
+			ktap_print_msg "cannot hotplug $(basename "$D")"
+			ktap_test_fail "$name"
+			ready=false
+		elif ! report_stage "$stage" "$name"; then
+			ready=false
+		fi
+	done
+	stop_tool
+	pn_bind_kmem_private "$D"
+}
+
+# A static binding in tool mode $1 of a common node that becomes private.
+run_common_retype()
+{
+	local mode=$1 name=$2 line
+
+	if ! echo unplugged > "$D/state" || ! echo 0 > "$D/private" ||
+	   ! echo online_movable > "$D/state"; then
+		ktap_print_msg "cannot provision common kmem memory"
+		ktap_test_fail "$name"
+		return
+	fi
+	echo "${COMMON[0]},$PN" > "$CG/cpuset.mems"
+	if ! start_tool "$mode" "$CG"; then
+		ktap_print_msg "tool setup failed"
+		ktap_test_fail "$name"
+		stop_tool
+		return
+	fi
+	if ! echo unplugged > "$D/state" || ! echo 1 > "$D/private" ||
+	   ! echo online_movable > "$D/state" ||
+	   ! echo "${COMMON[1]},$PN" > "$CG/cpuset.mems"; then
+		ktap_print_msg "cannot retype kmem memory"
+		ktap_test_fail "$name"
+		stop_tool
+		return
+	fi
+	kill -USR1 "$TARGET"
+	if wait "$CHILD"; then
+		line=$(tail -1 "$OUT")
+		ktap_print_msg "$line"
+		if grep -q 'ok=1$' <<<"$line"; then
+			ktap_test_pass "$name"
+		else
+			ktap_test_fail "$name"
+		fi
+	else
+		ktap_print_msg "tool failed or timed out"
+		ktap_test_fail "$name"
+	fi
+	CHILD=
+	TARGET=
+}
+
+pn_setup
+[ -x "$TOOL" ] || pn_skip "kmem_private_rebind_tool not built"
+pn_find_common_nodes 2
+pn_require_mask_nodes "$PN" "${COMMON[@]}"
+pn_cpuset_cgroup "$CG"
+pn_cpuset_cgroup "$CGCOMMON"
+echo "${COMMON[0]},$PN" > "$CG/cpuset.mems" ||
+	ktap_exit_fail_msg "cannot set cpuset.mems of $CG"
+echo "${COMMON[1]}" > "$CGCOMMON/cpuset.mems" ||
+	ktap_exit_fail_msg "cannot set cpuset.mems of $CGCOMMON"
+
+ktap_set_plan 25
+
+run_task_policy interleave
+line=$(tail -1 "$OUT")
+ktap_print_msg "$line"
+name="interleave policy cannot reacquire private memory on rebind"
+if grep -q '^private=0 ' <<<"$line"; then
+	ktap_test_pass "$name"
+else
+	ktap_test_fail "$name"
+fi
+
+run_task_policy bind
+line=$(tail -1 "$OUT")
+ktap_print_msg "$line"
+name="positional cpuset migration excludes private memory"
+if grep -q '^private=0 ' <<<"$line"; then
+	ktap_test_pass "$name"
+else
+	ktap_test_fail "$name"
+fi
+name="removed common-node pages migrate to common fallback"
+if grep -q ' new_common=64$' <<<"$line"; then
+	ktap_test_pass "$name"
+else
+	ktap_test_fail "$name"
+fi
+
+run_private_transitions private-anon "anonymous binding"
+run_private_transitions private-shared \
+	"shared binding across distinct mapper cpusets"
+run_private_transitions private-anon \
+	"anonymous binding across task attachment" attach
+run_private_transitions private-shared \
+	"shared binding across task attachment" attach
+run_private_retype retype-anon "anonymous private binding"
+run_private_retype retype-shared "shared private binding"
+run_common_retype static \
+	"shared common binding falls back after its node becomes private"
+run_common_retype static-mixed \
+	"shared common binding requires an allowed common target"
+
+ktap_finished
diff --git a/tools/testing/selftests/dax/kmem_private_rebind_tool.c b/tools/testing/selftests/dax/kmem_private_rebind_tool.c
new file mode 100644
index 000000000000..3d88d1e61c2f
--- /dev/null
+++ b/tools/testing/selftests/dax/kmem_private_rebind_tool.c
@@ -0,0 +1,495 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Helper for kmem_private_cpuset.sh and kmem_private_rebind.sh. Most modes
+ * place memory under a policy, print "ready pid=<pid>" and wait for SIGUSR1
+ * while the script changes the cpuset or the node, then report where the
+ * memory went. Modes with several stages wait for one SIGUSR1 per stage and
+ * print one "stage=<n> ... ok=<0|1>" line each.
+ *
+ * Modes that check their own results exit 1 when a check fails. All modes
+ * exit 2 on usage or setup errors.
+ */
+#define _GNU_SOURCE
+#include <linux/mempolicy.h>
+#include <signal.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/mman.h>
+#include <sys/syscall.h>
+#include <sys/wait.h>
+#include <unistd.h>
+
+#ifndef MPOL_F_PRIVATE_NODES
+#define MPOL_F_PRIVATE_NODES (1 << 12)
+#endif
+
+#define NR_PAGES 64
+/* The kernel reads maxnode - 1 bits. */
+#define MAXNODE (8 * sizeof(unsigned long) + 1)
+
+enum mode {
+	ORDINARY,
+	INTERLEAVE,
+	BIND,
+	PRIVATE_ANON,
+	PRIVATE_SHARED,
+	RETYPE_ANON,
+	RETYPE_SHARED,
+	STATIC,
+	STATIC_MIXED,
+	NR_MODES,
+};
+
+static const struct {
+	const char *name;
+	const char *help;
+} modes[NR_MODES] = {
+	[ORDINARY] = { "ordinary", "allocate without a policy" },
+	[INTERLEAVE] = { "interleave",
+			 "task interleave over <private> and <common1>" },
+	[BIND] = { "bind", "task bind to <common1>" },
+	[PRIVATE_ANON] = { "private-anon",
+			   "private binding of anonymous memory" },
+	[PRIVATE_SHARED] = { "private-shared",
+			     "private binding of shared memory, two mappers" },
+	[RETYPE_ANON] = { "retype-anon",
+			  "private-anon across node hotplug" },
+	[RETYPE_SHARED] = { "retype-shared",
+			    "private-shared across node hotplug" },
+	[STATIC] = { "static", "static binding of shared memory to <private>" },
+	[STATIC_MIXED] = { "static-mixed",
+			   "static binding to <private> and <common1>" },
+};
+
+static void print_usage(const char *prog)
+{
+	int i;
+
+	fprintf(stderr, "usage: %s <mode> <private> <common1> <common2>\n",
+		prog);
+	for (i = 0; i < NR_MODES; i++)
+		fprintf(stderr, "  %-15s %s\n", modes[i].name, modes[i].help);
+	exit(2);
+}
+
+/* Node masks are a single unsigned long. */
+static bool node_fits(int nid)
+{
+	return nid >= 0 && nid < (int)(8 * sizeof(unsigned long));
+}
+
+static void *map_anon(long len)
+{
+	return mmap(NULL, len, PROT_READ | PROT_WRITE,
+		    MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+}
+
+static void *map_shared(int *fd, long len)
+{
+	*fd = memfd_create("private-rebind", 0);
+	if (*fd < 0 || ftruncate(*fd, len))
+		return MAP_FAILED;
+	return mmap(NULL, len, PROT_READ | PROT_WRITE, MAP_SHARED, *fd, 0);
+}
+
+static void announce_ready(void)
+{
+	printf("ready pid=%d\n", getpid());
+	fflush(stdout);
+}
+
+static long count_node(void *addr, long page_size, int nid)
+{
+	void *pages[NR_PAGES];
+	int status[NR_PAGES];
+	long i, count = 0;
+
+	for (i = 0; i < NR_PAGES; i++)
+		pages[i] = addr + i * page_size;
+	if (syscall(SYS_move_pages, 0, NR_PAGES, pages, NULL, status, 0UL))
+		return -1;
+	for (i = 0; i < NR_PAGES; i++)
+		count += status[i] == nid;
+	return count;
+}
+
+static bool range_matches(void *ptr, long page_size, unsigned char value)
+{
+	long i;
+
+	for (i = 0; i < NR_PAGES; i++)
+		if (*(unsigned char *)(ptr + i * page_size) != value)
+			return false;
+	return true;
+}
+
+static int bind_mapping(void *ptr, long len, unsigned long mask, int flags)
+{
+	return syscall(SYS_mbind, ptr, len, MPOL_BIND | flags,
+		       &mask, MAXNODE, 0UL);
+}
+
+static bool policy_matches(void *ptr, unsigned long mask, int flags)
+{
+	unsigned long actual = 0;
+	int mode;
+
+	return !syscall(SYS_get_mempolicy, &mode, &actual, MAXNODE,
+			ptr, MPOL_F_ADDR) &&
+		mode == (MPOL_BIND | flags) && actual == mask;
+}
+
+/* Report how many pages are on the private node. */
+static int ordinary(long page_size, int priv)
+{
+	long len = NR_PAGES * page_size;
+	void *ptr = map_anon(len);
+
+	if (ptr == MAP_FAILED)
+		return 2;
+	memset(ptr, 0xab, len);
+	printf("private=%ld\n", count_node(ptr, page_size, priv));
+	return 0;
+}
+
+static void report_placement(void *ptr, long page_size, int priv,
+			     int common1, int common2)
+{
+	printf("private=%ld old_common=%ld new_common=%ld\n",
+	       count_node(ptr, page_size, priv),
+	       count_node(ptr, page_size, common1),
+	       count_node(ptr, page_size, common2));
+}
+
+/* Interleave over the private node and allocate after the cpuset change. */
+static int task_interleave(long page_size, int priv, int common1,
+			   int common2, const sigset_t *waitset)
+{
+	unsigned long mask = (1UL << common1) | (1UL << priv);
+	long len = NR_PAGES * page_size;
+	void *ptr;
+	int sig;
+
+	if (syscall(SYS_set_mempolicy, MPOL_INTERLEAVE, &mask, MAXNODE))
+		return 2;
+	announce_ready();
+	sigwait(waitset, &sig);
+	ptr = map_anon(len);
+	if (ptr == MAP_FAILED)
+		return 2;
+	memset(ptr, 0xcd, len);
+	report_placement(ptr, page_size, priv, common1, common2);
+	munmap(ptr, len);
+	return 0;
+}
+
+/* Bind to the first common node and allocate before the cpuset change. */
+static int task_bind(long page_size, int priv, int common1, int common2,
+		     const sigset_t *waitset)
+{
+	unsigned long mask = 1UL << common1;
+	long len = NR_PAGES * page_size;
+	void *ptr;
+	int sig;
+
+	if (syscall(SYS_set_mempolicy, MPOL_BIND, &mask, MAXNODE))
+		return 2;
+	ptr = map_anon(len);
+	if (ptr == MAP_FAILED)
+		return 2;
+	memset(ptr, 0xab, len);
+	announce_ready();
+	sigwait(waitset, &sig);
+	report_placement(ptr, page_size, priv, common1, common2);
+	munmap(ptr, len);
+	return 0;
+}
+
+struct shared_result {
+	long private_pages;
+	long shared_pages;
+};
+
+/* Keep one mapper outside the cpuset that is being changed. */
+static int shared_mapper(void *ptr, long len, long page_size, int priv,
+			 int common, int requests, int replies, pid_t child)
+{
+	struct shared_result result;
+	int stage, status;
+	bool contents;
+	void *range;
+
+	while (read(requests, &stage, sizeof(stage)) == sizeof(stage)) {
+		range = ptr + (stage + 4) * len;
+		memset(range, 0xef, len);
+		result.private_pages = count_node(range, page_size, priv);
+
+		/* Existing shared pages are not reallocated for another mapper. */
+		range = ptr + (stage + 1) * len;
+		contents = range_matches(range, page_size, 0xcd);
+		result.shared_pages = count_node(range, page_size,
+						 stage == 1 ? common : priv);
+		if (!contents)
+			result.shared_pages = -1;
+		if (write(replies, &result, sizeof(result)) != sizeof(result))
+			break;
+	}
+	close(requests);
+	close(replies);
+	if (waitpid(child, &status, 0) != child || !WIFEXITED(status))
+		return 1;
+	return WEXITSTATUS(status);
+}
+
+/*
+ * A private binding across three cpuset changes that allow, revoke and allow
+ * the node again. Range 0 of the mapping is populated up front and range
+ * 1 + stage at each stage. For shared memory the parent stays outside the
+ * cpuset as a second mapper and populates range 4 + stage.
+ */
+static int private_rebind(bool shared, long page_size, int priv, int common,
+			  const sigset_t *waitset)
+{
+	unsigned long mask = 1UL << priv;
+	long len = NR_PAGES * page_size;
+	long total = (shared ? 7 : 4) * len;
+	int fd = -1, requests[2], replies[2];
+	int stage, sig, failed = 0;
+	struct shared_result other;
+	long private_pages, common_pages, existing, existing_common;
+	bool policy, contents, ok;
+	pid_t child;
+	void *ptr, *range;
+
+	if (shared)
+		ptr = map_shared(&fd, total);
+	else
+		ptr = map_anon(total);
+	if (ptr == MAP_FAILED ||
+	    bind_mapping(ptr, total, mask, MPOL_F_PRIVATE_NODES))
+		return 2;
+	/* Keep later ranges absent until their respective cpuset transition. */
+	if (madvise(ptr, total, MADV_NOHUGEPAGE))
+		return 2;
+	memset(ptr, 0xab, len);
+	if (count_node(ptr, page_size, priv) != NR_PAGES)
+		return 1;
+
+	if (shared) {
+		if (pipe(requests) || pipe(replies))
+			return 2;
+		child = fork();
+		if (child < 0)
+			return 2;
+		if (child) {
+			close(requests[1]);
+			close(replies[0]);
+			failed = shared_mapper(ptr, len, page_size, priv, common,
+					       requests[0], replies[1], child);
+			goto out;
+		}
+		close(requests[0]);
+		close(replies[1]);
+		/* Populate the child's PTEs before cpuset-requested migration. */
+		if (!range_matches(ptr, page_size, 0xab))
+			return 2;
+	}
+
+	announce_ready();
+	for (stage = 0; stage < 3; stage++) {
+		if (sigwait(waitset, &sig))
+			return 2;
+		range = ptr + (stage + 1) * len;
+		memset(range, 0xcd, len);
+		private_pages = count_node(range, page_size, priv);
+		common_pages = count_node(range, page_size, common);
+		/* File-backed PTEs can be populated lazily after fork. */
+		contents = range_matches(ptr, page_size, 0xab);
+		existing = count_node(ptr, page_size, priv);
+		existing_common = count_node(ptr, page_size, common);
+		policy = policy_matches(ptr, mask, MPOL_F_PRIVATE_NODES);
+		ok = policy && contents;
+		/* Old pages leave the private node only once it is revoked. */
+		if (stage == 0)
+			ok &= existing == NR_PAGES && existing_common == 0;
+		else
+			ok &= existing == 0 && existing_common == NR_PAGES;
+		if (stage == 1)
+			ok &= common_pages == NR_PAGES && private_pages == 0;
+		else
+			ok &= private_pages == NR_PAGES && common_pages == 0;
+		other.private_pages = -1;
+		other.shared_pages = -1;
+		if (shared) {
+			if (write(requests[1], &stage, sizeof(stage)) != sizeof(stage) ||
+			    read(replies[0], &other, sizeof(other)) != sizeof(other))
+				return 2;
+			ok &= other.private_pages == NR_PAGES &&
+				other.shared_pages == NR_PAGES;
+		}
+		printf("stage=%d private=%ld common=%ld existing_private=%ld ",
+		       stage, private_pages, common_pages, existing);
+		printf("existing_common=%ld ", existing_common);
+		printf("policy=%d other_private=%ld shared_pages=%ld ok=%d\n",
+		       policy, other.private_pages, other.shared_pages, ok);
+		fflush(stdout);
+		failed |= !ok;
+	}
+	if (shared) {
+		close(requests[1]);
+		close(replies[0]);
+	}
+out:
+	munmap(ptr, total);
+	if (fd >= 0)
+		close(fd);
+	return failed;
+}
+
+/*
+ * A private binding while the node goes offline, returns as common memory,
+ * goes offline again and returns as private memory. Range @stage of the
+ * mapping is populated at each stage.
+ */
+static int private_retype(bool shared, long page_size, int target, int common,
+			  const sigset_t *waitset)
+{
+	unsigned long mask = 1UL << target;
+	long len = NR_PAGES * page_size;
+	long private_pages, common_pages;
+	int fd = -1, stage, sig, i, failed = 0;
+	bool policy, contents, ok;
+	void *ptr, *range;
+
+	if (shared)
+		ptr = map_shared(&fd, 4 * len);
+	else
+		ptr = map_anon(4 * len);
+	if (ptr == MAP_FAILED ||
+	    bind_mapping(ptr, 4 * len, mask, MPOL_F_PRIVATE_NODES) ||
+	    madvise(ptr, 4 * len, MADV_NOHUGEPAGE))
+		return 2;
+
+	/* Keep the policy alive without preventing the initial hot-unplug. */
+	announce_ready();
+	for (stage = 0; stage < 4; stage++) {
+		if (sigwait(waitset, &sig))
+			return 2;
+		range = ptr + stage * len;
+		memset(range, 0xa0 + stage, len);
+		private_pages = count_node(range, page_size, target);
+		common_pages = count_node(range, page_size, common);
+		policy = policy_matches(ptr, mask, MPOL_F_PRIVATE_NODES);
+		contents = true;
+		for (i = 0; i <= stage; i++)
+			contents &= range_matches(ptr + i * len, page_size,
+						  0xa0 + i);
+		ok = policy && contents;
+		if (stage & 1)
+			ok &= private_pages == NR_PAGES && common_pages == 0;
+		else
+			ok &= private_pages == 0 && common_pages == NR_PAGES;
+		printf("stage=%d target=%ld common=%ld policy=%d contents=%d ok=%d\n",
+		       stage, private_pages, common_pages, policy, contents, ok);
+		fflush(stdout);
+		failed |= !ok;
+	}
+	munmap(ptr, 4 * len);
+	if (fd >= 0)
+		close(fd);
+	return failed;
+}
+
+/*
+ * A static binding of shared memory to a common node, optionally with a
+ * second common node, that the script turns private before a second mapper
+ * faults the pages in.
+ */
+static int common_retype(bool mixed, long page_size, int target,
+			 int old_common, int new_common,
+			 const sigset_t *waitset)
+{
+	unsigned long mask = 1UL << target;
+	long len = NR_PAGES * page_size;
+	long private_pages, common_pages;
+	int fd, sig;
+	bool policy, ok;
+	void *ptr, *other;
+
+	if (mixed)
+		mask |= 1UL << old_common;
+	ptr = map_shared(&fd, len);
+	if (ptr == MAP_FAILED ||
+	    bind_mapping(ptr, len, mask, MPOL_F_STATIC_NODES))
+		return 2;
+	other = mmap(NULL, len, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
+	if (other == MAP_FAILED)
+		return 2;
+	/* No resident pages: the target can be removed and retyped. */
+	announce_ready();
+	if (sigwait(waitset, &sig))
+		return 2;
+	memset(other, 0xcd, len);
+	private_pages = count_node(other, page_size, target);
+	common_pages = count_node(other, page_size, new_common);
+	policy = policy_matches(other, mask, MPOL_F_STATIC_NODES);
+	ok = private_pages == 0 && common_pages == NR_PAGES && policy;
+	printf("private=%ld common=%ld policy=%d ok=%d\n",
+	       private_pages, common_pages, policy, ok);
+	munmap(other, len);
+	munmap(ptr, len);
+	close(fd);
+	return !ok;
+}
+
+int main(int argc, char **argv)
+{
+	long page_size = sysconf(_SC_PAGESIZE);
+	int priv, common1, common2, mode;
+	sigset_t waitset;
+
+	if (argc != 5)
+		print_usage(argv[0]);
+	for (mode = 0; mode < NR_MODES; mode++)
+		if (!strcmp(argv[1], modes[mode].name))
+			break;
+	if (mode == NR_MODES)
+		print_usage(argv[0]);
+	priv = atoi(argv[2]);
+	common1 = atoi(argv[3]);
+	common2 = atoi(argv[4]);
+	if (!node_fits(priv) || !node_fits(common1) || !node_fits(common2)) {
+		fprintf(stderr, "node ids must be 0 to %zu\n",
+			8 * sizeof(unsigned long) - 1);
+		return 2;
+	}
+
+	sigemptyset(&waitset);
+	sigaddset(&waitset, SIGUSR1);
+	sigprocmask(SIG_BLOCK, &waitset, NULL);
+
+	switch (mode) {
+	case ORDINARY:
+		return ordinary(page_size, priv);
+	case INTERLEAVE:
+		return task_interleave(page_size, priv, common1, common2,
+				       &waitset);
+	case BIND:
+		return task_bind(page_size, priv, common1, common2, &waitset);
+	case PRIVATE_ANON:
+	case PRIVATE_SHARED:
+		return private_rebind(mode == PRIVATE_SHARED, page_size, priv,
+				      common2, &waitset);
+	case RETYPE_ANON:
+	case RETYPE_SHARED:
+		return private_retype(mode == RETYPE_SHARED, page_size, priv,
+				      common2, &waitset);
+	case STATIC:
+	case STATIC_MIXED:
+		return common_retype(mode == STATIC_MIXED, page_size, priv,
+				     common1, common2, &waitset);
+	}
+	return 2;
+}
diff --git a/tools/testing/selftests/dax/private_node_common.sh b/tools/testing/selftests/dax/private_node_common.sh
index cd03d9489822..5c58f3195d22 100644
--- a/tools/testing/selftests/dax/private_node_common.sh
+++ b/tools/testing/selftests/dax/private_node_common.sh
@@ -234,3 +234,35 @@ pn_require_mask_nodes()
 			pn_skip "node $nid does not fit the helper node mask"
 	done
 }
+
+# Create cgroup $1 with the cpuset controller enabled and all CPUs.
+pn_cpuset_cgroup()
+{
+	[ "$(stat -fc %T "$CGROOT" 2>/dev/null)" = cgroup2fs ] ||
+		pn_skip "requires cgroup v2"
+	grep -qw cpuset "$CGROOT/cgroup.controllers" 2>/dev/null ||
+		pn_skip "cpuset controller unavailable"
+	if ! grep -qw cpuset "$CGROOT/cgroup.subtree_control"; then
+		echo +cpuset > "$CGROOT/cgroup.subtree_control" 2>/dev/null ||
+			pn_skip "cannot enable cpuset controller"
+		PN_CPUSET_ENABLED=1
+	fi
+	if ! mkdir "$1" ||
+	   ! cat "$CGROOT/cpuset.cpus.effective" > "$1/cpuset.cpus"; then
+		ktap_exit_fail_msg "cannot create $1"
+	fi
+}
+
+# Remove the cgroups $@ made by pn_cpuset_cgroup(), and the cpuset
+# controller if it enabled it.
+pn_cpuset_cleanup()
+{
+	local cg
+
+	for cg in "$@"; do
+		[ -d "$cg" ] && rmdir "$cg" 2>/dev/null
+	done
+	[ -n "${PN_CPUSET_ENABLED:-}" ] || return 0
+	PN_CPUSET_ENABLED=
+	echo -cpuset > "$CGROOT/cgroup.subtree_control" 2>/dev/null
+}

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* [PATCH 60/60] Documentation/mm: describe private memory nodes
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (58 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 59/60] selftests/dax: test cpuset interaction with " Gregory Price via B4 Relay
@ 2026-11-20 18:09 ` Gregory Price via B4 Relay
  2026-11-20 18:31   ` sashiko-bot
  2026-11-20 18:59 ` [PATCH 00/60] Private NUMA nodes Gregory Price (Meta)
  60 siblings, 1 reply; 124+ messages in thread
From: Gregory Price via B4 Relay @ 2026-11-20 18:09 UTC (permalink / raw)
  To: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan
  Cc: linux-kernel, linux-mm, linux-cxl, linux-doc, driver-core, damon,
	linux-pm, linux-fsdevel, cgroups, nvdimm, linux-kselftest,
	Gregory Price (Meta)

From: "Gregory Price (Meta)" <gourry@gourry.net>

Add Documentation/mm/numa_private_nodes.rst for administrators,
covering:
  - creating a private node with dax kmem private mode.
  - binding memory to it with MPOL_F_PRIVATE_NODES, and explicit
    migration.
  - controlling access with cpusets.
  - what happens when the node becomes unavailable or full.
  - shared memory placement.
  - which memory management services apply, and how memory counters
    account private memory.

In both cgroup guides, document that cpuset memory migration never
targets a private node and moves pages off one only when the cpuset no
longer allows it. Document how a private-only cpuset.mems gains the
parent's common memory nodes in v2, and is rejected in v1.

Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---
 Documentation/admin-guide/cgroup-v1/cpusets.rst |   7 +
 Documentation/admin-guide/cgroup-v2.rst         |  13 +-
 Documentation/mm/index.rst                      |   1 +
 Documentation/mm/numa_private_nodes.rst         | 223 ++++++++++++++++++++++++
 4 files changed, 243 insertions(+), 1 deletion(-)

diff --git a/Documentation/admin-guide/cgroup-v1/cpusets.rst b/Documentation/admin-guide/cgroup-v1/cpusets.rst
index fb3fb6fa88cb..642b6f568bcc 100644
--- a/Documentation/admin-guide/cgroup-v1/cpusets.rst
+++ b/Documentation/admin-guide/cgroup-v1/cpusets.rst
@@ -653,6 +653,13 @@ the cpuset is preserved during these migration operations if possible.
 For example if the page was on the second valid node of the prior cpuset
 then the page will be placed on the second valid node of the new cpuset.
 
+Private memory nodes (see Documentation/mm/numa_private_nodes.rst) are
+never a migration destination. With 'cpuset.memory_migrate' set, pages
+on a private node migrate only when the cpuset no longer allows that
+node. Listing a private node in 'cpuset.mems' permits explicit private
+bindings to it, but tasks do not otherwise allocate from it. Writing a
+'cpuset.mems' that names only private nodes fails with -ENOSPC.
+
 Also if 'cpuset.memory_migrate' is set true, then if that cpuset's
 'cpuset.mems' file is modified, pages allocated to tasks in that
 cpuset, that were on nodes in the previous setting of 'cpuset.mems',
diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
index 9e3c67b02ae2..8b24c2648040 100644
--- a/Documentation/admin-guide/cgroup-v2.rst
+++ b/Documentation/admin-guide/cgroup-v2.rst
@@ -2626,6 +2626,12 @@ Cpuset Interface Files
 	tasks within the cgroup to be migrated to the designated nodes if
 	they are currently using memory outside of the designated nodes.
 
+	Private memory nodes (see Documentation/mm/numa_private_nodes.rst)
+	are never a migration destination.  Pages on a private node are
+	migrated only when the node is removed from "cpuset.mems.effective".
+	Listing a private node in "cpuset.mems" permits explicit private
+	bindings to it, but tasks do not otherwise allocate from it.
+
 	There is a cost for this memory migration.  The migration
 	may not be complete and some memory pages may be left behind.
 	So it is recommended that "cpuset.mems" should be set properly
@@ -2652,7 +2658,12 @@ Cpuset Interface Files
 	parent cgroup that will be available to be used by this cgroup.
 	Otherwise, it should be a subset of "cpuset.mems" unless none of
 	the memory nodes listed in "cpuset.mems" can be granted.  In this
-	case, it will be treated just like an empty "cpuset.mems".
+	case, it will be treated just like an empty "cpuset.mems", except
+	that the parent's private memory nodes are left out.
+
+	If the granted nodes are all private memory nodes, the parent's
+	other memory nodes are added, so that tasks always have memory for
+	ordinary allocations.
 
 	Its value will be affected by memory nodes hotplug events.
 
diff --git a/Documentation/mm/index.rst b/Documentation/mm/index.rst
index e9ae5cd82163..ecb83e1bdbf6 100644
--- a/Documentation/mm/index.rst
+++ b/Documentation/mm/index.rst
@@ -66,6 +66,7 @@ documentation, or deleted if it has served its purpose.
    mmu_notifier
    multigen_lru
    numa
+   numa_private_nodes
    overcommit-accounting
    page_migration
    page_frags
diff --git a/Documentation/mm/numa_private_nodes.rst b/Documentation/mm/numa_private_nodes.rst
new file mode 100644
index 000000000000..2fedf7534446
--- /dev/null
+++ b/Documentation/mm/numa_private_nodes.rst
@@ -0,0 +1,223 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+====================
+Private memory nodes
+====================
+
+A private memory node is a NUMA node whose memory the kernel does not use for
+ordinary allocations. Applications place memory there only by asking for that
+node explicitly. This keeps specialized memory, such as HBM or coherent device
+memory, visible as a NUMA node without making it a general fallback for every
+allocation. Memory that is not private is called common memory.
+
+The kernel manages a private node's memory with the usual page allocator. The
+driver that provides the memory decides which memory management services the
+node also gets. The dax kmem private mode described below enables reclaim,
+compaction and explicit placement by applications.
+
+``/sys/devices/system/node/has_private_memory`` lists the private nodes, and
+``has_common_memory`` lists the others. Together they make up ``has_memory``.
+
+Creating a private node
+=======================
+
+Private memory is added through memory hotplug and requires ``CONFIG_NUMA``.
+With ``CONFIG_DEV_DAX_KMEM`` and ``CONFIG_MEMORY_HOTREMOVE``, a device-dax
+instance bound to ``kmem`` can be onlined as private memory. Select private mode
+while the device's memory is unplugged::
+
+    echo unplugged > /sys/bus/dax/devices/daxX.Y/state
+    echo 1 > /sys/bus/dax/devices/daxX.Y/private
+    echo online_movable > /sys/bus/dax/devices/daxX.Y/state
+
+The ``private`` attribute defaults to 0, which keeps ordinary kmem behavior.
+Its value can change only while the device is unplugged. Unbinding kmem
+discards the setting. Bind the device with memory auto-onlining disabled, so
+that its memory is not onlined as common memory first.
+
+All System RAM on a node shares one configuration. Adding memory with a
+different configuration to a node that already has System RAM fails with
+``-EBUSY``. To change the configuration, remove every System RAM block on the
+node first. Offlining the blocks is not enough, since offline blocks keep their
+configuration so they can be onlined again.
+
+A private node may have CPUs. Like the CPUs of a memoryless node, they get
+ordinary memory from the nearest common nodes.
+
+Using private memory
+====================
+
+An application binds an anonymous or shmem-backed mapping to one private node
+with ``mbind(2)``, passing ``MPOL_BIND | MPOL_F_PRIVATE_NODES``, before it
+faults the pages in::
+
+    buf = mmap(NULL, length, PROT_READ | PROT_WRITE,
+               MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+    /* mask contains exactly the chosen private node */
+    mbind(buf, length, MPOL_BIND | MPOL_F_PRIVATE_NODES, mask, maxnode, 0);
+    memset(buf, 0, length);
+
+``mbind(2)`` rejects a private binding with ``-EINVAL`` if:
+
+- the mask names more than one node, or a node that is not private memory
+  with userspace placement enabled.
+- the mode is not ``MPOL_BIND``, or the flag is combined with
+  ``MPOL_F_STATIC_NODES``, ``MPOL_F_RELATIVE_NODES`` or
+  ``MPOL_F_NUMA_BALANCING``.
+- the mapping is a HugeTLB mapping. Private nodes hold no HugeTLB pages.
+- the node is not in the caller's cpuset.
+
+Kernels without private node support reject the flag with ``-EINVAL``, so
+applications can probe for it. ``get_mempolicy(2)`` reports the flag.
+
+Without ``MPOL_F_PRIVATE_NODES``, private nodes in a ``set_mempolicy(2)`` or
+``mbind(2)`` mask are ignored, as nodes outside the cpuset are. A mask that
+names only private nodes is therefore rejected with ``-EINVAL``.
+``set_mempolicy(2)`` does not accept the flag, because a task policy also
+applies to the kernel's allocations on behalf of the task. Preferred,
+interleave and preferred-many policies, tmpfs ``mpol=`` mount options, HugeTLB
+pools and automatic NUMA balancing use only common memory.
+
+``move_pages(2)`` and ``migrate_pages(2)`` can move pages to a private node
+with userspace placement enabled, subject to their usual permission checks.
+They take no flag. ``move_pages(2)`` rejects other private nodes with
+``-ENODEV``, and ``migrate_pages(2)`` ignores them, as it ignores nodes outside
+the caller's cpuset.
+
+Page cache for regular files is never allocated from private memory.
+Copy-on-write pages of a private file mapping follow a binding like anonymous
+memory, and explicit migration can move existing file pages to a private node.
+A private binding on a shmem-backed mapping allocates its pages from the
+private node, and reclaim swaps them out like other shmem pages.
+
+guest_memfd memory cannot be moved, so a binding places it on a private node
+only if the node has memory onlined as ``online`` or ``online_kernel``.
+
+Controlling access
+==================
+
+``cpuset.mems`` includes private nodes by default, so any task can bind to a
+private node and use up its capacity. Use ``cpuset.mems`` to limit which tasks
+may use a private node. Listing a private node in ``cpuset.mems`` permits
+private bindings to it. Tasks do not otherwise allocate from it.
+
+In cgroup v2, a ``cpuset.mems`` that names only private nodes is accepted, and
+the parent's common nodes are added to ``cpuset.mems.effective`` so that tasks
+always have memory for ordinary allocations. The parent's other private nodes
+are not added. An empty ``cpuset.mems`` inherits the parent's effective nodes,
+private nodes included.
+
+In cgroup v1, writing a ``cpuset.mems`` that names only private nodes, or
+attaching a task to such a cpuset, fails with ``-ENOSPC``. When memory hotplug
+leaves a v1 cpuset with only private memory, its tasks move to an ancestor.
+
+Cpuset memory migration never moves pages to a private node. It moves pages
+off a private node only when the cpuset no longer allows that node.
+
+When a node becomes unavailable
+===============================
+
+A private binding applies while its node is online private memory with
+userspace placement enabled and the task's cpuset allows it. While it applies,
+allocations use only that node and never fall back to common memory.
+
+Otherwise the binding is suspended and allocations use the task's common
+memory. The binding is never rebound to another node. It applies again when
+its node becomes available again. If the node comes back as common memory, the
+binding acts like an ordinary binding to that node. ``get_mempolicy(2)`` keeps
+reporting the original binding throughout.
+
+==============================  ==================  =========================
+Event                           New allocations     Existing private pages
+==============================  ==================  =========================
+cpuset removes the node         common memory       migrated to common memory
+cpuset allows the node again    the private node    not migrated back
+node is being hot-removed       the private node    migrated to common memory
+node is removed                 common memory       none remain
+node returns as private memory  the private node
+node returns as common memory   that common node
+==============================  ==================  =========================
+
+While a node is being hot-removed, its free memory is no longer available, so
+faults on mappings bound to it can fail with ``SIGBUS`` until removal finishes.
+
+Allocations that cannot use ``ZONE_MOVABLE``, such as guest_memfd memory, ignore
+a binding to a private node whose memory is all movable and use common memory.
+Long-term pins migrate pages out of ``ZONE_MOVABLE`` as usual.
+
+An ordinary policy that is not rebound, such as a static binding or a shared
+memory policy, can name a node that later comes back as private memory. If the
+task may use no common node in such a policy, the policy is ignored and the
+allocation uses common memory.
+
+Exhaustion and OOM
+==================
+
+A private node is a fixed pool, like a HugeTLB pool. When the node is full and
+reclaim cannot free enough memory on it, an allocation for a private binding
+fails without invoking the OOM killer and without falling back to common
+memory. A page fault then gets ``SIGBUS``, as with an exhausted HugeTLB pool.
+Other callers see an error, such as ``-ENOMEM`` from ``write(2)`` to a shmem
+file or ``-EFAULT`` from a write through ``/proc/pid/mem`` or
+``process_vm_writev(2)``. Allocations succeed again once memory on the node is
+freed.
+
+Private pages are charged to memory cgroups like other memory. Reaching a
+memory cgroup limit causes reclaim and OOM handling as usual, and memcg reclaim
+also reclaims private nodes that permit reclaim.
+
+Tasks whose cpusets overlap only on private nodes are not chosen as OOM victims
+for each other.
+
+Shared memory
+=============
+
+A private binding on a shmem-backed mapping, such as memfd, tmpfs or shared
+anonymous memory, applies to the file's offset range, not to each task that
+maps it. Whether a new page is allocated on the private node depends on the
+cpuset of the task that faults it in. A task whose cpuset excludes the node
+allocates the page on common memory, and every task that maps the file then
+uses that page. A file can therefore hold both private and common pages.
+
+Shared pages keep using private memory until they are freed. Unmapping the
+file does not free them while the file still exists or another task keeps it
+open. Truncate the file or punch holes in it to return the memory.
+
+Memory management services
+==========================
+
+kswapd and kcompactd run on a private node only if its driver enables reclaim
+and compaction, as the dax kmem private mode does. Writing to ``nodeN/reclaim``
+or ``nodeN/compact`` fails with ``-EOPNOTSUPP`` for a private node without the
+service. ``MADV_COLD``, ``MADV_PAGEOUT`` and ``MADV_FREE`` act only on pages on
+nodes that permit reclaim.
+
+Private nodes are excluded from KSM merging, THP collapse, automatic NUMA
+balancing, DAMON actions, demotion between memory tiers and HugeTLB pool
+growth. A KSM page that is explicitly migrated to a private node remains
+shareable while it is mapped.
+
+Memory hot-remove and soft offline migrate pages on a private node to common
+memory. Pinned or otherwise unmovable pages can still prevent removal.
+
+Accounting
+==========
+
+``MemTotal`` and ``MemFree`` in ``/proc/meminfo`` include private memory with
+userspace placement enabled, such as dax kmem private memory, since it holds
+ordinary application data. They leave out other private memory. The overcommit
+limit (``CommitLimit``) and other limits sized from total memory follow
+``MemTotal``.
+
+``MemAvailable`` and the dirty page limits count free memory on common nodes
+only, since free private memory serves only private bindings. ``MemTotal``
+minus ``MemAvailable`` therefore includes free private memory.
+
+Per-node statistics, including ``nodeN/meminfo``, ``/proc/vmstat`` and
+``memory.numa_stat``, count all memory, private memory included.
+
+Limitations
+===========
+
+- A task's OOM score includes its resident private pages.
+- Free page reporting reports free private pages like other free pages.

-- 
2.56.1



^ permalink raw reply related	[flat|nested] 124+ messages in thread

* Re: [PATCH 09/60] mm/damon: reject migration destinations outside common memory
  2026-11-20 18:09 ` [PATCH 09/60] mm/damon: reject migration " Gregory Price via B4 Relay
@ 2026-11-20 18:18   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:18 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> DAMOS migration actions move folios to a node the user picks. As with
> move_pages(2), the destination should be general purpose memory.
> 
> Reject a destination outside N_MEMORY_COMMON, exactly the same as a
> node without memory is rejected today.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=9


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 14/60] mm: base hash distribution on common memory nodes
  2026-11-20 18:09 ` [PATCH 14/60] mm: base hash distribution on common memory nodes Gregory Price via B4 Relay
@ 2026-11-20 18:19   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:19 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> hashdist is useful when large kernel hash tables can be allocated across
> multiple NUMA nodes. The tables come from ordinary allocations, which
> only N_MEMORY_COMMON nodes serve.
> 
> Count N_MEMORY_COMMON nodes when applying the single-node optimization.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=14


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 08/60] mm/migrate: reject move_pages() destinations outside common memory
  2026-11-20 18:08 ` [PATCH 08/60] mm/migrate: reject move_pages() destinations outside " Gregory Price via B4 Relay
@ 2026-11-20 18:19   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:19 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> move_pages(2) places user memory on a node the caller picks. As with
> memory policies, the destination should be general purpose memory.
> 
> Reject a destination outside N_MEMORY_COMMON with -ENODEV, exactly
> the same as a node without memory is rejected today.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=8


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 24/60] mm/madvise: restrict reclaim advice to common memory
  2026-11-20 18:09 ` [PATCH 24/60] mm/madvise: restrict reclaim advice " Gregory Price via B4 Relay
@ 2026-11-20 18:19   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:19 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> MADV_COLD and MADV_PAGEOUT age or reclaim the folios a caller names,
> and MADV_FREE marks them lazyfree, whatever node they are on.
> 
> Restrict them to common memory: convert their ZONE_DEVICE checks,
> including the one in madvise_free_huge_pmd(), to
> folio_is_common_memory().
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=24


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 19/60] mm: add folio_is_common_memory() and page_is_common_memory()
  2026-11-20 18:09 ` [PATCH 19/60] mm: add folio_is_common_memory() and page_is_common_memory() Gregory Price via B4 Relay
@ 2026-11-20 18:20   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:20 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> N_MEMORY_COMMON decides where ordinary allocations and node-scoped
> services go, but MM services that walk folios reach them through page
> tables, physical addresses or LRU lists, whatever node they are on.
> 
> Add folio_is_common_memory() and page_is_common_memory() for those
> services. ZONE_DEVICE folios are never common memory, so the helpers
> can replace the folio_is_zone_device() and page_is_zone_device() checks
> those services already make. Following commits convert them.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=19


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 02/60] mm: publish and clear node memory states via helpers
  2026-11-20 18:08 ` [PATCH 02/60] mm: publish and clear node memory states via helpers Gregory Price via B4 Relay
@ 2026-11-20 18:20   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:20 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> N_MEMORY is set in free_area_init() and online_pages(), and cleared in
> offline_pages().
> 
> Centralize those updates in node_set/clear_memory_state(), so states
> that subdivide N_MEMORY can be published together with it.
> 
> Behavior is unchanged.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=2


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 11/60] fork: check cached stack locality only on common memory nodes
  2026-11-20 18:09 ` [PATCH 11/60] fork: check cached stack locality only on common memory nodes Gregory Price via B4 Relay
@ 2026-11-20 18:20   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:20 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> try_release_thread_stack_to_cache() caches a freed stack only if all of
> its pages are on the local node, unless the local node has no memory.
> 
> A CPU on a node outside N_MEMORY_COMMON gets its stacks from other
> nodes, as a CPU on a memoryless node does, so the locality check would
> keep it from ever caching a stack.
> 
> Skip the check when the local node has no common memory.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=11


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 20/60] mm/ksm: restrict merging to common memory
  2026-11-20 18:09 ` [PATCH 20/60] mm/ksm: restrict merging to common memory Gregory Price via B4 Relay
@ 2026-11-20 18:20   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:20 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> KSM is an opportunistic MM service and should leave memory outside
> the ordinary memory pool alone - as it already does for ZONE_DEVICE
> folios where its scan picks up candidate pages.
> 
> Use folio_is_common_memory() at those same sites, which still rejects
> ZONE_DEVICE folios.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=20


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 15/60] mm/page_alloc: count common memory nodes for kswapd recovery
  2026-11-20 18:09 ` [PATCH 15/60] mm/page_alloc: count common memory nodes for kswapd recovery Gregory Price via B4 Relay
@ 2026-11-20 18:20   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:20 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> When a zone recovers above its high watermark, free_frozen_page_commit()
> clears a hopeless kswapd state if more than one memory node exists,
> since allocations could then have fallen back to another node without
> triggering direct reclaim here.
> 
> Only common memory provide fallbacks, so count N_MEMORY_COMMON nodes.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=15


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 13/60] mm/compaction: restrict compaction to common memory
  2026-11-20 18:09 ` [PATCH 13/60] mm/compaction: restrict compaction " Gregory Price via B4 Relay
@ 2026-11-20 18:20   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:20 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> Compaction serves the ordinary allocation domain.
> 
> Restrict it to N_MEMORY_COMMON:
>   - start kcompactd only on common nodes, at boot and hotplug.
>   - compact only common nodes for /proc/sys/vm/compact_memory,
>     and trigger proactive compaction only on common nodes.
> 
> compact_memory and the proactiveness trigger walked every online node
> before. Memoryless nodes have nothing to compact and no kcompactd, so
> leaving them out changes nothing.
> 
> Direct compaction serves an allocation from its own zonelist, so it is
> left alone.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=13


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 18/60] mm/khugepaged: base the recommended min_free_kbytes on common memory
  2026-11-20 18:09 ` [PATCH 18/60] mm/khugepaged: base the recommended min_free_kbytes on " Gregory Price via B4 Relay
@ 2026-11-20 18:20   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:20 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> set_recommended_min_free_kbytes() reserves two pageblocks per populated
> zone to help THP allocations avoid fragmentation. The reserve serves
> ordinary allocations, which use only common memory, so count only the
> zones on N_MEMORY_COMMON nodes.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=18


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 06/60] mm/memory_hotplug: migrate offlined pages to common memory
  2026-11-20 18:08 ` [PATCH 06/60] mm/memory_hotplug: migrate offlined pages to " Gregory Price via B4 Relay
@ 2026-11-20 18:20   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:20 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> do_migrate_range() / alloc_migration_target() is only capable of
> allocating memory on N_MEMORY_COMMON nodes. The destination nodemask
> presently starts with N_MEMORY and removes the source-node.
> 
> In the current code, if the destination mask becomes empty, the source
> node is restored to allow migration during partial offlining.
> 
> Consider this configuration:
>   Node0 = N_MEMORY_COMMON
>   Node1 = N_MEMORY, not N_MEMORY_COMMON
> 
> Node 1 keeps the destination mask non-empty, despite being unreachable.
> This leaves migration with no usable destination because the source
> node is not an eligible allocation target.
> 
> Restrict the hot-unplug migration target-list to N_MEMORY_COMMON so it
> reflects the nodes the allocator can reach.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=6


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 01/60] nodemask: add node-state iterators
  2026-11-20 18:08 ` [PATCH 01/60] nodemask: add node-state iterators Gregory Price via B4 Relay
@ 2026-11-20 18:20   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:20 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> Add first_node_state() and next_node_state(), so callers can walk any
> node state the way first_memory_node and next_memory_node walk N_MEMORY.
> 
> Add for_each_node_mask_state(), which iterates a nodemask intersected
> with a node state using the bitmap intersection helpers, without a
> temporary nodemask.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=1


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 21/60] mm/khugepaged: restrict collapse to common memory
  2026-11-20 18:09 ` [PATCH 21/60] mm/khugepaged: restrict collapse " Gregory Price via B4 Relay
@ 2026-11-20 18:21   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:21 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> Collapse is an opportunistic MM service and should operate only on
> common memory. The anonymous scan and isolation paths already filter
> zone device for the same reason.
> 
> Replace the zone_device check with page_is_common_memory().
> 
> collapse_scan_file() gains a folio_is_common_memory() check. Page-cache
> folios are never ZONE_DEVICE, so there it only filters memory outside
> N_MEMORY_COMMON.
> 
> collapse_file() rechecks once each folio is locked, since a folio can be
> added or migrated after collapse_scan_file() looked at it.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=21


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 16/60] mm/oom_kill: compare OOM constraints against common memory
  2026-11-20 18:09 ` [PATCH 16/60] mm/oom_kill: compare OOM constraints against common memory Gregory Price via B4 Relay
@ 2026-11-20 18:21   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:21 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> Ordinary allocations reach only common-memory zonelists. Compare policy
> and cpuset constraints against N_MEMORY_COMMON, so nodes outside it
> neither make an otherwise global policy appear constrained nor inflate
> the constrained capacity.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=16


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 23/60] mm/damon: restrict DAMON-native folio operations to common memory
  2026-11-20 18:09 ` [PATCH 23/60] mm/damon: restrict DAMON-native folio operations to " Gregory Price via B4 Relay
@ 2026-11-20 18:21   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:21 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> DAMON acquires folios to age them, check their accesses, and apply LRU
> operations and migration. Treat these as general-purpose MM services
> and restrict them to N_MEMORY_COMMON.
> 
> Memory outside N_MEMORY_COMMON can still hold ordinary LRU folios, so
> it would pass the checks that keep ZONE_DEVICE memory out of these
> operations.
> 
> Treat these folios like ZONE_DEVICE memory.
> 
> DAMON cannot acquire such a folio, so it never ages one and virtual
> address monitoring reports it as accessed, and DAMOS cannot isolate
> one for migration.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=23


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 07/60] mm/memory-tiers: restrict demotion topology to common memory
  2026-11-20 18:08 ` [PATCH 07/60] mm/memory-tiers: restrict demotion topology " Gregory Price via B4 Relay
@ 2026-11-20 18:21   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:21 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> Memory outside N_MEMORY_COMMON is outside the ordinary allocation pool
> and cannot be a generic demotion source or destination.
> 
> Build demotion paths and the default DRAM performance reference from
> N_MEMORY_COMMON rather than every node that has memory, and do not
> assign other nodes to a memory tier through the generic hotplug path.
> 
> Removing a node's memory already resets its demotion targets, so a node
> that unplugs and returns outside N_MEMORY_COMMON keeps none.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=7


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 26/60] mm/migrate: leave pages outside common memory in move_pages()
  2026-11-20 18:09 ` [PATCH 26/60] mm/migrate: leave pages outside common memory in move_pages() Gregory Price via B4 Relay
@ 2026-11-20 18:21   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:21 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> move_pages(2) moves the pages a caller names, and reports the node of
> each page in status queries.
> 
> Leave pages outside common memory where they are: refuse to move them,
> similar to ZONE_DEVICE pages. Status queries move nothing, so they
> still report the node of each page.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=26


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 22/60] mm/mempolicy: don't NUMA-hint folios outside common memory
  2026-11-20 18:09 ` [PATCH 22/60] mm/mempolicy: don't NUMA-hint folios outside " Gregory Price via B4 Relay
@ 2026-11-20 18:21   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:21 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> The NUMA balancing scanner installs PROT_NONE only on folios that
> balancing may migrate.
> 
> folio_can_map_prot_numa() already skips ZONE_DEVICE folios. Skip folios
> outside the common memory pool (N_MEMORY_COMMON) as well. Do not inject
> hint faults on those folios or migrate them.
> 
> Convert the zone_device check to a common memory check (which includes
> the zone device check explicitly).
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=22


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 10/60] sched/numa: restrict NUMA migration targets to common memory
  2026-11-20 18:09 ` [PATCH 10/60] sched/numa: restrict NUMA migration targets to " Gregory Price via B4 Relay
@ 2026-11-20 18:21   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:21 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> should_numa_migrate_memory() refuses to migrate folios towards a
> memoryless node, since the migration can only fail; see
> commit 3fb43636876d ("sched/numa, mm: do not try to migrate memory to
> memoryless nodes").
> 
> A node outside N_MEMORY_COMMON can have CPUs but has an empty
> __GFP_THISNODE zonelist, so a NUMA hinting fault from one of its CPUs
> would isolate the folio, fail to allocate the destination and put the
> folio back.
> 
> Check N_MEMORY_COMMON instead.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=10


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 03/60] mm: introduce the common-memory node state
  2026-11-20 18:08 ` [PATCH 03/60] mm: introduce the common-memory node state Gregory Price via B4 Relay
@ 2026-11-20 18:22   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:22 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> N_MEMORY is overloaded with two meanings:
>   1) a node has memory.
>   2) a node's memory is reachable via ordinary allocation or
>      an arbitrary MM service may use that memory.
> 
> Allow N_MEMORY to mean #1, and introduce N_MEMORY_COMMON to mean #2.
> 
> N_MEMORY_COMMON nodes contain "common memory" capable of being managed
> arbitrarily by the kernel MM based solely on zone membership. Following
> commits convert relevant N_MEMORY users to N_MEMORY_COMMON.
> 
> Expose the state via `has_common_memory` so userspace can identify
> the memory available to ordinary allocations and common MM services.
> 
> Add has_memory to the docs while we're here.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=3


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 05/60] mm/mempolicy: keep interleave and tmpfs mount policies on common memory
  2026-11-20 18:08 ` [PATCH 05/60] mm/mempolicy: keep interleave and tmpfs mount policies on " Gregory Price via B4 Relay
@ 2026-11-20 18:22   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:22 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> Use N_MEMORY_COMMON for boot mempolicy calculations and their per-node
> sysfs entries (weighted interleave weights).
> 
> Restrict the node set mpol_set_nodemask() applies to set_mempolicy()
> and mbind() masks to N_MEMORY_COMMON nodes.
> 
> A tmpfs mpol= nodelist must name only N_MEMORY_COMMON nodes, and an
> interleave policy without a nodelist defaults to them.
> 
> Later when an N_MEMORY node is omitted from the N_MEMORY_COMMON set,
> they will be stripped from mempolicy nodemasks as they're not eligible
> (similar to memoryless nodes today).
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=5


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 38/60] mm/page_alloc: refuse contiguous allocation from private memory
  2026-11-20 18:09 ` [PATCH 38/60] mm/page_alloc: refuse contiguous allocation from private memory Gregory Price via B4 Relay
@ 2026-11-20 18:24   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:24 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> Exact-PFN contiguous allocation bypasses the zonelists and can invoke
> compaction and reclaim.
> 
> Reject a range on a private node with -EPERM before isolating any pages,
> and document the error.
> 
> If a user of private ranges shows up, alloc_contig_range() can grow an
> ACR flag or an exported api for private allocations.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=38


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 33/60] cgroup/cpuset: ignore private nodes when matching OOM victims
  2026-11-20 18:09 ` [PATCH 33/60] cgroup/cpuset: ignore private nodes when matching OOM victims Gregory Price via B4 Relay
@ 2026-11-20 18:24   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:24 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> The OOM killer only considers a task an eligible victim if its
> mems_allowed intersects the allocating task's mems_allowed. The
> assumption is that a shared node means shared ordinary memory.
> 
> Private nodes break that assumption. If two cpusets partition common
> memory but both allow a private node, an OOM in one could kill tasks in
> the other and free nothing the allocating task can use.
> 
> Initially, private memory allocations will never invoke the OOM killer,
> so only common memory pressure reaches this check.
> 
> Leave private nodes out of the intersection.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=33


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 28/60] mm: select memory features per node
  2026-11-20 18:09 ` [PATCH 28/60] mm: select memory features per node Gregory Price via B4 Relay
@ 2026-11-20 18:24   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:24 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> Whether a node's memory is common or private is a property of the
> memory, so record it per node.
> 
> Add a per-node memory feature mask (pgdat->memory_features) with a
> single bit for common memory. Node states are published from the node's
> features at boot and when memory is onlined. A node without the common
> bit joins N_MEMORY_PRIVATE.
> 
> Ordinary System RAM carries every feature. Without NUMA, node 0 holds
> all memory, so a mask without common memory is invalid.
> 
> Every node keeps the default mask, so all memory remains common.
> 
> Suggested-by: Dan J Williams <djbw@kernel.org>
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=28


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 36/60] mm: refuse explicit reclaim and compaction of private nodes
  2026-11-20 18:09 ` [PATCH 36/60] mm: refuse explicit reclaim and compaction of private nodes Gregory Price via B4 Relay
@ 2026-11-20 18:24   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:24 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> kswapd and kcompactd do not run on private nodes, but nodeN/reclaim and
> nodeN/compact name a node directly. Reject them for a private node with
> -EOPNOTSUPP.
> 
> Memoryless nodes keep accepting the requests.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=36


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 44/60] mm: add N_MEMORY_COMPACTION node state
  2026-11-20 18:09 ` [PATCH 44/60] mm: add N_MEMORY_COMPACTION node state Gregory Price via B4 Relay
@ 2026-11-20 18:24   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:24 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> Add N_MEMORY_COMPACTION for the nodes compaction may operate on, and
> NODE_MEMORY_FEAT_COMPACTION for memory providers to select it.
> 
> Common memory carries every feature, so a mask with
> NODE_MEMORY_FEAT_COMMON must now include compaction.
> 
> The state is internal and has no sysfs file. Internal states follow
> N_GENERIC_INITIATOR, so check sysfs coverage only for the states up to
> it.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=44


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 46/60] mm: add N_MEMORY_RECLAIM node state
  2026-11-20 18:09 ` [PATCH 46/60] mm: add N_MEMORY_RECLAIM node state Gregory Price via B4 Relay
@ 2026-11-20 18:25   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:25 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> Add N_MEMORY_RECLAIM as an internal state for the nodes page reclaim
> may operate on, and NODE_MEMORY_FEAT_RECLAIM for memory providers to
> select it.
> 
> Reclaim frees order-0 pages and leaves high-order requests to
> compaction, so a private node that permits reclaim must permit
> compaction too.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=46


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 25/60] mm/mempolicy: queue only common memory for mbind() and migrate_pages()
  2026-11-20 18:09 ` [PATCH 25/60] mm/mempolicy: queue only common memory for mbind() and migrate_pages() Gregory Price via B4 Relay
@ 2026-11-20 18:25   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:25 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> mbind(MPOL_MF_MOVE) and migrate_pages() queue folios from a task's
> mappings for migration.
> 
> Queue only folios on common memory: convert the ZONE_DEVICE checks to
> folio_is_common_memory(). As with ZONE_DEVICE folios, the skipped folios
> do not count as failures, so MPOL_MF_STRICT does not report them.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=25


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 27/60] mm: add the private-memory node state
  2026-11-20 18:09 ` [PATCH 27/60] mm: add the private-memory node state Gregory Price via B4 Relay
@ 2026-11-20 18:25   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:25 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> N_MEMORY_COMMON describes memory that ordinary allocations and MM
> services may use. Some memory is online but is not part of that pool.
> 
> Add N_MEMORY_PRIVATE for nodes with such memory.
> 
> N_MEMORY_COMMON and N_MEMORY_PRIVATE partition N_MEMORY, so a caller can
> drop private nodes from a mask without also dropping memoryless nodes.
> 
> Expose the state via `has_private_memory`.
> 
> Without CONFIG_NUMA, node 0 holds all memory and none of it is private.
> We have to adjust the node_state check to return false for this node,
> previously when CONFIG_NUMA was disabled it would always return true.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=27


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 35/60] mm/mempolicy: keep private nodes out of migrate_pages() destinations
  2026-11-20 18:09 ` [PATCH 35/60] mm/mempolicy: keep private nodes out of migrate_pages() destinations Gregory Price via B4 Relay
@ 2026-11-20 18:25   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:25 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> migrate_pages(2) intersects its destination nodes with the caller's
> cpuset, which can include private nodes.
> 
> A private destination gets mapped to some source node, and every page
> bound for it attempts a __GFP_THISNODE allocation. A private node is not
> on its own __GFP_THISNODE zonelist, so the call fails with -ENOMEM and
> logs an allocation failure.
> 
> Leave private nodes out of the destination mask, as move_pages(2)
> already rejects them. A mask that names only private nodes fails with
> -EINVAL.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=35


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 12/60] mm/vmscan: restrict reclaim to common memory
  2026-11-20 18:09 ` [PATCH 12/60] mm/vmscan: restrict reclaim to common memory Gregory Price via B4 Relay
@ 2026-11-20 18:25   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:25 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> Reclaim serves the ordinary allocation domain.
> 
> Restrict it to N_MEMORY_COMMON:
>   - start kswapd only on common nodes, at boot or hotplug.
>   - walk common nodes when MGLRU's debugfs interface iterates or
>     validates nodes, using first_node_state() and next_node_state() in
>     place of first_memory_node and next_memory_node.
> 
> Direct reclaim serves an allocation from its own zonelist, so it is left
> alone.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=12


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 04/60] mm/page_alloc: build zonelists from common memory
  2026-11-20 18:08 ` [PATCH 04/60] mm/page_alloc: build zonelists from common memory Gregory Price via B4 Relay
@ 2026-11-20 18:27   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:27 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> Zonelists decide which nodes an allocation can reach.
> 
> The fallback lists hold every node in N_MEMORY, and a node's
> __GFP_THISNODE list holds its own zones whatever the node's state.
> 
> Ordinary allocations should reach only common (general purpose) memory.
> 
> Build the fallback zonelists from N_MEMORY_COMMON, and __GFP_THISNODE
> zonelists only for a common node.
> 
> The build_local_zonelist() refactor lets us reuse the code to build
> a local-only zonelist for nodes outside the N_MEMORY_COMMON set later.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=4


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 50/60] mm: add N_MEMORY_MEMPOLICY node state
  2026-11-20 18:09 ` [PATCH 50/60] mm: add N_MEMORY_MEMPOLICY node state Gregory Price via B4 Relay
@ 2026-11-20 18:27   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:27 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> Add N_MEMORY_MEMPOLICY as an internal state for the nodes that memory
> policy interfaces and explicit migration may target, and
> NODE_MEMORY_FEAT_MEMPOLICY for memory providers to select it.
> 
> Userspace placement lets ordinary processes use the node like any other
> memory. That assumes the node is reclaimed and compacted, so the feature
> requires both.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=50


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 48/60] mm/madvise: restrict reclaim advice to reclaim-capable memory
  2026-11-20 18:09 ` [PATCH 48/60] mm/madvise: restrict reclaim advice to reclaim-capable memory Gregory Price via B4 Relay
@ 2026-11-20 18:29   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:29 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> MADV_COLD, MADV_PAGEOUT and MADV_FREE reclaim or age folios directly,
> bypassing the node filter reclaim uses, so they act only on common
> memory.
> 
> Add folio_allows_reclaim(), which requires the folio's node to permit
> reclaim, and use it instead for those advices, including their
> huge-page paths.
> 
> No behavior change to common nodes.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=48


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 45/60] mm/compaction: restrict compaction to N_MEMORY_COMPACTION
  2026-11-20 18:09 ` [PATCH 45/60] mm/compaction: restrict compaction to N_MEMORY_COMPACTION Gregory Price via B4 Relay
@ 2026-11-20 18:30   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:30 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> Restrict compaction to N_MEMORY_COMPACTION instead of common memory:
>   - compact N_MEMORY_COMPACTION nodes for /proc/sys/vm/compact_memory.
>   - start kcompactd and trigger proactive compaction on those nodes.
> 
> Direct compaction is unchanged: an allocation compacts the zones on its
> zonelist, and a private node's zones are only on its private zonelist.
> 
> nodeN/compact fails with -EOPNOTSUPP for private nodes without the
> state.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=45


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 60/60] Documentation/mm: describe private memory nodes
  2026-11-20 18:09 ` [PATCH 60/60] Documentation/mm: describe private memory nodes Gregory Price via B4 Relay
@ 2026-11-20 18:31   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:31 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> Add Documentation/mm/numa_private_nodes.rst for administrators,
> covering:
>   - creating a private node with dax kmem private mode.
>   - binding memory to it with MPOL_F_PRIVATE_NODES, and explicit
>     migration.
>   - controlling access with cpusets.
>   - what happens when the node becomes unavailable or full.
>   - shared memory placement.
>   - which memory management services apply, and how memory counters
>     account private memory.
> 
> In both cgroup guides, document that cpuset memory migration never
> targets a private node and moves pages off one only when the cpuset no
> longer allows it. Document how a private-only cpuset.mems gains the
> parent's common memory nodes in v2, and is rejected in v1.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=60


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 37/60] mm/page_alloc: keep private memory out of reclaim reserves
  2026-11-20 18:09 ` [PATCH 37/60] mm/page_alloc: keep private memory out of reclaim reserves Gregory Price via B4 Relay
@ 2026-11-20 18:31   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:31 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> Private memory cannot serve general system allocations, so it should not
> contribute to reserves.
> 
> Exclude private capacity from the lowmem denominator and the global
> reserve total, so the pages_min budget and the system reserve stay on
> common memory. A private node still records its own reserve, which
> its per-node dirty limit subtracts.
> 
> Zero private zones' watermarks. No kswapd runs for a private node to
> restore them, and the pages below the minimum would be unreachable.
> With no high watermark, a migratetype fallback never boosts them
> either.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=37


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 39/60] mm/page_alloc: add a private-node zonelist
  2026-11-20 18:09 ` [PATCH 39/60] mm/page_alloc: add a private-node zonelist Gregory Price via B4 Relay
@ 2026-11-20 18:31   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:31 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> A private node carries N_MEMORY without N_MEMORY_COMMON, so the FALLBACK
> and NOFALLBACK zonelists already exclude it. That leaves no zonelist
> that can reach it.
> 
> Add ZONELIST_PRIVATE, which holds a node's local zones only if the node
> is private. Common nodes have an empty private zonelist.
> 
> Like the existing zonelists, it is sized for every zone of every node.
> That adds a zoneref (16 bytes on 64-bit) per possible zone, MAX_NUMNODES
> * MAX_NR_ZONES, to each pgdat. A smaller local-only zonelist type can be
> a follow-up if this matters.
> 
> When the zonelist becomes selectable, __GFP_THISNODE will still suppress
> OOM, while the page allocator selects the private zonelist instead of
> NOFALLBACK.
> 
> If a user grows requiring fallback behavior to COMMON nodes when the
> private node is out of memory, we can consider adding a fallback
> variant - but for now private node allocations use a local-only
> zonelist to limit complexity.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=39


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 56/60] dax/kmem: add a private memory mode
  2026-11-20 18:09 ` [PATCH 56/60] dax/kmem: add a private memory mode Gregory Price via B4 Relay
@ 2026-11-20 18:31   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:31 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> Add a `private` boolean sysfs attribute, defaulting to false and
> changeable only while memory is unplugged.
> 
> When true, dax/kmem onlines the node as private memory with
> NODE_MEMORY_FEAT_MEMPOLICY, NODE_MEMORY_FEAT_RECLAIM and
> NODE_MEMORY_FEAT_COMPACTION.
> 
> Userspace targets private capacity via single-node MPOL_F_PRIVATE_NODES
> bindings on anonymous and shmem-backed mappings. Reclaim and compaction
> manage the memory, and the system does not otherwise use it.
> 
> The attribute is hidden without CONFIG_NUMA or CONFIG_MEMORY_HOTREMOVE,
> since selecting private mode requires unplugging the memory first, and
> the default behavior of dax/kmem probe creates the memory blocks (an
> unfortunately backward compatibility wart).
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=56


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 32/60] cgroup/cpuset: keep spread page cache off private nodes
  2026-11-20 18:09 ` [PATCH 32/60] cgroup/cpuset: keep spread page cache off private nodes Gregory Price via B4 Relay
@ 2026-11-20 18:31   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:31 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> With memory_spread_page set, cpuset_mem_spread_node() rotates the
> preferred node for page cache over the task's mems_allowed.
> 
> mems_allowed can include private nodes. A private preferred node will
> fall back to the nearest common node, which takes an extra share.
> 
> Rotate only over the allowed nodes outside N_MEMORY_PRIVATE. A task's
> mems_allowed always holds common memory, so the loop always terminates.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=32


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 49/60] mm/vmscan: reclaim private nodes for memcg
  2026-11-20 18:09 ` [PATCH 49/60] mm/vmscan: reclaim private nodes for memcg Gregory Price via B4 Relay
@ 2026-11-20 18:31   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:31 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> Memcg reclaim does not serve one allocation, so it should cover every
> node that permits reclaim. It walks the current node's fallback
> zonelist, which holds no private node, so a private node that permits
> reclaim is never scanned for a cgroup over its limit.
> 
> After the zonelist, visit the nodes that permit reclaim and that the
> zonelist did not cover, and give them the same refault snapshot and
> congestion reset. The zonelist keeps setting the order, which matters
> because memcg reclaim stops once it has freed enough.
> 
> Direct reclaim for an allocation still walks only that allocation's
> zonelist. A private allocation's zonelist covers its own node, and
> reclaiming private memory cannot satisfy a common allocation.
> 
> Without private nodes the zonelist covers every node that permits
> reclaim, so behavior is unchanged.
> 
> Another way to do this would be to implement a reclaim zonelist that
> only reclaim uses, but this implementation does not carry the static
> per-node size overhead that adding another zonelist does.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=49


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 41/60] mm/page_alloc: relax a private request whose node becomes ineligible
  2026-11-20 18:09 ` [PATCH 41/60] mm/page_alloc: relax a private request whose node becomes ineligible Gregory Price via B4 Relay
@ 2026-11-20 18:32   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:32 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> A private request allocates only from its node's private zonelist. If
> the cpuset revokes the node, or memory hotplug removes it, the request
> has nowhere left to allocate, and a faulting task retries forever.
> 
> An ordinary bind policy does not hit this. A cpuset change rebinds it,
> and check_retry_cpuset() drops a nodemask that no longer intersects the
> cpuset. A private binding is never rebound, and dropping its nodemask
> does not help, since the private zonelist still holds only its node.
> 
> Add relax_private_alloc() as the last of the slowpath retry checks.
> When no node in a private request's zonelist is in its nodemask and
> allowed by the cpuset, it drops ALLOC_ZONELIST_PRIVATE and the nodemask
> and switches to the common zonelist of the preferred node, which
> alloc_context now records. It runs after check_retry_cpuset() and
> check_retry_zonelist(), so a concurrent update retries with the original
> constraints instead.
> 
> A node counts in any zone type, so a GFP zone restriction or an
> exhausted node never relaxes a request. A __GFP_THISNODE private request
> is never relaxed. check_retry_cpuset() leaves a private request's
> nodemask alone.
> 
> The alloc_context comment now lists nodemask and alloc_flags among the
> fields the slowpath may change.
> 
> Without private nodes, the slowpath retry checks are unchanged.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=41


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 30/60] mm/memory_hotplug: keep private memory out of the auto-movable ratio
  2026-11-20 18:09 ` [PATCH 30/60] mm/memory_hotplug: keep private memory out of the auto-movable ratio Gregory Price via B4 Relay
@ 2026-11-20 18:33   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:33 UTC (permalink / raw)
  To: Gregory Price (Meta), Yury Norov, Rasmus Villemoes,
	David Hildenbrand, Liam R. Howlett, Vlastimil Babka,
	Mike Rapoport, Michal Hocko, Andrew Morton, Oscar Salvador,
	Johannes Weiner, Suren Baghdasaryan, Brendan Jackman, Zi Yan,
	Jonathan Corbet, Shuah Khan, Randy Dunlap, Greg Kroah-Hartman,
	Rafael J. Wysocki, Danilo Krummrich, Johan Hovold, Joshua Hahn,
	Rakie Kim, Ying Huang, Matthew Brost, Byungchul Park,
	Alistair Popple, SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli,
	Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
	Mel Gorman, Valentin Schneider, K Prateek Nayak, Kees Cook,
	Kairui Song, Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen,
	Yuanchu Xie, Wei Xu, Baoquan He, Baolin Wang, Michal Hocko,
	David Rientjes, Pavel Machek, Len Brown, Nico Pache, Ryan Roberts,
	Dev Jain, Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin,
	Chengming Zhou, Lorenzo Stoakes, Pedro Falcato, Jann Horn,
	Chris Li, Kemeng Shi, Nhat Pham, Youngjun Park,
	Matthew Wilcox (Oracle), Jan Kara, Waiman Long, Ridong Chen,
	Tejun Heo, Michal Koutný, Dan Williams, Vishal Verma,
	Dave Jiang, Alison Schofield, Shuah Khan
  Cc: linux-mm, linux-cxl, damon, nvdimm

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] mm/memory_hotplug: broken auto-movable accounting on private
  nodes
--

commit 6d04c084f583deb362f04abc899f0521e026dbfc
Author: Gregory Price (Meta) <gourry@gourry.net>

mm/memory_hotplug: keep private memory out of the auto-movable ratio

This commit skips nodes with N_MEMORY_PRIVATE set in
auto_movable_stats_account_zone() and auto_movable_stats_account_group()
when computing statistics for the auto-movable online policy.

> diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
> index b4d94021d46ef..d7ce60c6815f8 100644
> --- a/mm/memory_hotplug.c
> +++ b/mm/memory_hotplug.c
> @@ -782,6 +782,10 @@ struct auto_movable_stats {
>  static void auto_movable_stats_account_zone(struct auto_movable_stats *stats,
>  					    struct zone *zone)
>  {
> +	/* Private memory is outside the movable:kernel balance. */
> +	if (node_state(zone_to_nid(zone), N_MEMORY_PRIVATE))
> +		return;
> +
>  	if (zone_idx(zone) == ZONE_MOVABLE) {
>  		stats->movable_pages += zone->present_pages;
>  	} else {
> @@ -808,6 +812,10 @@ static int auto_movable_stats_account_group(struct memory_group *group,
>  	struct auto_movable_group_stats *stats = arg;
>  	long pages;
>  
> +	/* private memory is outside the movable:kernel balance. */
> +	if (node_state(group->nid, N_MEMORY_PRIVATE))
> +		return 0;
> +
>  	/*
>  	 * We don't support modifying the config while the auto-movable online
>  	 * policy is already enabled. Just avoid the division by zero below.

[Severity: Medium]
Does skipping N_MEMORY_PRIVATE here without updating
auto_movable_can_online_movable() and auto_movable_zone_for_pfn() break
auto-movable accounting when onlining memory on private nodes?

In auto_movable_can_online_movable(), walk_dynamic_memory_groups() excludes
the target group being onlined, so auto_movable_stats_account_group() only
runs for other groups.  For the target group itself,
auto_movable_can_online_movable() relies on
auto_movable_stats_account_zone() having counted its present_movable_pages
in stats.movable_pages, and then directly adds group->present_kernel_pages:

mm/memory_hotplug.c:auto_movable_can_online_movable() {
	...
	walk_dynamic_memory_groups(nid, auto_movable_stats_account_group,
				   group, &group_stats);
	if (kernel_early_pages <= group_stats.req_kernel_early_pages)
		return false;
	kernel_early_pages -= group_stats.req_kernel_early_pages;
	movable_pages -= group_stats.movable_pages;

	if (group && group->is_dynamic)
		kernel_early_pages += group->present_kernel_pages;
	...
}

Because auto_movable_stats_account_zone() now skips N_MEMORY_PRIVATE nodes,
a dynamic memory group on a private node still adds its
present_kernel_pages to kernel_early_pages in the NUMA_NO_NODE check while
its present_movable_pages are omitted from movable_pages.

In addition, auto_movable_zone_for_pfn() computes
nr_pages = max_pages - online_pages assuming online_pages were already
counted in stats.movable_pages by auto_movable_stats_account_zone(), which
no longer holds once the first block on a private node is onlined and sets
N_MEMORY_PRIVATE.  When auto_movable_numa_aware is disabled, skipping
ZONE_MOVABLE on private nodes also allows unbounded ZONE_MOVABLE memory to
be onlined on private nodes without counting against the movable budget,
whereas when auto_movable_numa_aware is enabled, the per-node
auto_movable_can_online_movable(nid, ...) check sees kernel_early_pages == 0
and group_stats.req_kernel_early_pages == 0 and always returns false.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=30

^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 52/60] mm: count memory with userspace placement as general purpose
  2026-11-20 18:09 ` [PATCH 52/60] mm: count memory with userspace placement as general purpose Gregory Price via B4 Relay
@ 2026-11-20 18:33   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:33 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> Userspace places ordinary data on N_MEMORY_MEMPOLICY nodes, and neither
> the commit charge nor a task's RSS can tell nodes apart. Treat their
> memory as general purpose: totalram_pages() and MemFree count it, and so
> do the limits sized from them (overcommit, OOM capacity).
> 
> Other defaults sized from totalram_pages(), such as the zswap pool and
> tmpfs sizes, also see the larger total, as they already do for memory
> the kernel cannot allocate from in ZONE_MOVABLE.
> 
> Free memory on these nodes serves only private bindings and migration,
> so counting it in MemAvailable or the global dirtyable memory would
> overstate both. Add common_free_pages() and use it there.
> 
> MemAvailable also assumes about half of the file LRU can be reclaimed.
> Reclaiming file pages on these nodes frees only private memory, so
> count only the file LRU on common nodes, via common_file_pages().
> 
> File pages that reach a private node through migration stay covered
> by the dirty limits' file LRU term.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=52


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 55/60] mm/memory_hotplug: online and offline private memory nodes
  2026-11-20 18:09 ` [PATCH 55/60] mm/memory_hotplug: online and offline private memory nodes Gregory Price via B4 Relay
@ 2026-11-20 18:34   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:34 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> Allow a driver to hotplug memory onto a private node at runtime.
> 
> Pass a feature mask through driver-managed hotplug, and reject masks
> that node_memory_features_valid() refuses.
> 
> Memory joins a node only with the same mask. The node's features hold
> while any System RAM block remains (including offline blocks), and are
> reset after the last one is removed, even when ZONE_DEVICE memory keeps
> the node alive.
> 
> dax/kmem and the other hotplug callers pass NODE_MEMORY_FEAT_ALL, which
> registers the node as N_MEMORY_COMMON.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=55


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 43/60] mm/page_alloc: fall back when a bind policy's nodes become private
  2026-11-20 18:09 ` [PATCH 43/60] mm/page_alloc: fall back when a bind policy's nodes become private Gregory Price via B4 Relay
@ 2026-11-20 18:34   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:34 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> Memory hotplug can make a node private after a bind policy naming it
> was created. A static bind policy, a shared memory policy, or any bind
> policy without cpusets is not rebound when the node's memory is
> removed, so it still names the node when the node returns as private
> memory.
> 
> The common zonelists hold no private node, so a policy naming only such
> nodes leaves the allocation no zone to use. The nodemask still
> intersects the cpuset, so check_retry_cpuset() keeps it, and a fault
> through the policy is retried forever.
> 
> Let relax_private_alloc() treat a nodemask that names a private node
> like a private request: once no node in the zonelist is in the nodemask
> and allowed by the cpuset, drop the nodemask and allocate from common
> memory. Only a request on the private zonelist keeps __GFP_THISNODE
> strict.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=43


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 59/60] selftests/dax: test cpuset interaction with private nodes
  2026-11-20 18:09 ` [PATCH 59/60] selftests/dax: test cpuset interaction with " Gregory Price via B4 Relay
@ 2026-11-20 18:35   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:35 UTC (permalink / raw)
  To: Gregory Price (Meta), Yury Norov, Rasmus Villemoes,
	David Hildenbrand, Liam R. Howlett, Vlastimil Babka,
	Mike Rapoport, Michal Hocko, Andrew Morton, Oscar Salvador,
	Johannes Weiner, Suren Baghdasaryan, Brendan Jackman, Zi Yan,
	Jonathan Corbet, Shuah Khan, Randy Dunlap, Greg Kroah-Hartman,
	Rafael J. Wysocki, Danilo Krummrich, Johan Hovold, Joshua Hahn,
	Rakie Kim, Ying Huang, Matthew Brost, Byungchul Park,
	Alistair Popple, SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli,
	Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
	Mel Gorman, Valentin Schneider, K Prateek Nayak, Kees Cook,
	Kairui Song, Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen,
	Yuanchu Xie, Wei Xu, Baoquan He, Baolin Wang, Michal Hocko,
	David Rientjes, Pavel Machek, Len Brown, Nico Pache, Ryan Roberts,
	Dev Jain, Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin,
	Chengming Zhou, Lorenzo Stoakes, Pedro Falcato, Jann Horn,
	Chris Li, Kemeng Shi, Nhat Pham, Youngjun Park,
	Matthew Wilcox (Oracle), Jan Kara, Waiman Long, Ridong Chen,
	Tejun Heo, Michal Koutný, Dan Williams, Vishal Verma,
	Dave Jiang, Alison Schofield, Shuah Khan
  Cc: linux-mm, linux-cxl, damon, nvdimm

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Low] selftests/dax: stage desync and wait hang in
  run_private_transitions()
--

commit 86268e0cbec65176c860c84e90fc221fb5d0e80c
Author: Gregory Price (Meta) <gourry@gourry.net>

selftests/dax: test cpuset interaction with private nodes

This commit adds selftests and a C helper tool to verify cgroup-v2 cpuset
interactions with private kmem nodes across cpuset.mems changes, task
attachment, and node hotplug/retyping.

> diff --git a/tools/testing/selftests/dax/kmem_private_rebind.sh b/tools/testing/selftests/dax/kmem_private_rebind.sh
> new file mode 100755
> index 0000000000000..5d39be04f1795
> --- /dev/null
> +++ b/tools/testing/selftests/dax/kmem_private_rebind.sh
> @@ -0,0 +1,301 @@

[ ... ]

> +run_private_transitions()
> +{
> +	local mode=$1 label=$2 action=${3:-reshape}
> +	local stage mask title cg=
> +	local titles=("keeps private pages and placement while the node stays allowed"
> +		"evacuates private pages and uses common placement when access is revoked"
> +		"restores the original private request when access returns")
> +
> +	echo "${COMMON[0]},$PN" > "$CG/cpuset.mems"
> +	if [ "$action" = attach ]; then
> +		echo "${COMMON[1]},$PN" > "$CG/cpuset.mems"
> +	fi
> +	# Shared-policy parents and task-attachment tools start at the root.
> +	if [ "$mode" = private-anon ] && [ "$action" = reshape ]; then
> +		cg=$CG
> +	fi
> +	if ! start_tool "$mode" "$cg"; then
> +		ktap_print_msg "tool setup failed"
> +		for title in "${titles[@]}"; do
> +			ktap_test_fail "$label $title"
> +		done
> +		stop_tool
> +		return
> +	fi
> +	if [ "$mode" = private-shared ] && [ "$action" = reshape ]; then
> +		echo "$TARGET" > "$CG/cgroup.procs"
> +	fi
> +	for stage in 0 1 2; do
> +		mask=${COMMON[1]}
> +		[ "$stage" = 1 ] || mask+=",$PN"
> +		if ! transition_cpuset "$action" "$stage" "$mask"; then
> +			ktap_print_msg "cannot change the cpuset"
> +			ktap_test_fail "$label ${titles[$stage]}"
> +			continue
> +		fi
> +		report_stage "$stage" "$label ${titles[$stage]}"
> +	done
> +	wait "$CHILD"
> +	CHILD=
> +	TARGET=
> +}

[Severity: Low]
Does continuing on a transition_cpuset() failure desynchronize the stage
loop from kmem_private_rebind_tool and leave wait "$CHILD" blocked until
the 60-second timeout expires?

In private_rebind(), the helper tool loops for stage = 0, 1, 2 and blocks
in sigwait() for one SIGUSR1 per stage.  Skipping report_stage() with
continue omits sending SIGUSR1 for the failed stage, so subsequent
iterations wait for a stage number the tool has not reached yet, and after
the loop the tool remains blocked in sigwait() because it received fewer
than three signals.  Handling this like run_private_retype() by tracking
tool readiness and calling stop_tool at the end would avoid the hang.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=59

^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 58/60] selftests/dax: test memory policy on private nodes
  2026-11-20 18:09 ` [PATCH 58/60] selftests/dax: test memory policy on private nodes Gregory Price via B4 Relay
@ 2026-11-20 18:35   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:35 UTC (permalink / raw)
  To: Gregory Price (Meta), Yury Norov, Rasmus Villemoes,
	David Hildenbrand, Liam R. Howlett, Vlastimil Babka,
	Mike Rapoport, Michal Hocko, Andrew Morton, Oscar Salvador,
	Johannes Weiner, Suren Baghdasaryan, Brendan Jackman, Zi Yan,
	Jonathan Corbet, Shuah Khan, Randy Dunlap, Greg Kroah-Hartman,
	Rafael J. Wysocki, Danilo Krummrich, Johan Hovold, Joshua Hahn,
	Rakie Kim, Ying Huang, Matthew Brost, Byungchul Park,
	Alistair Popple, SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli,
	Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
	Mel Gorman, Valentin Schneider, K Prateek Nayak, Kees Cook,
	Kairui Song, Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen,
	Yuanchu Xie, Wei Xu, Baoquan He, Baolin Wang, Michal Hocko,
	David Rientjes, Pavel Machek, Len Brown, Nico Pache, Ryan Roberts,
	Dev Jain, Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin,
	Chengming Zhou, Lorenzo Stoakes, Pedro Falcato, Jann Horn,
	Chris Li, Kemeng Shi, Nhat Pham, Youngjun Park,
	Matthew Wilcox (Oracle), Jan Kara, Waiman Long, Ridong Chen,
	Tejun Heo, Michal Koutný, Dan Williams, Vishal Verma,
	Dave Jiang, Alison Schofield, Shuah Khan
  Cc: linux-mm, linux-cxl, damon, nvdimm

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] selftests/dax: common_policy() resets task policy before
  faulting pages
--

commit 2c509a362d5823231bce734e3d872af08303fbb5
Author: Gregory Price (Meta) <gourry@gourry.net>

selftests/dax: test memory policy on private nodes

This commit adds a selftest tool and wrapper script to verify memory policy,
migration, reclaim, and exhaustion behavior on private memory nodes.

> diff --git a/tools/testing/selftests/dax/kmem_private_policy_tool.c b/tools/testing/selftests/dax/kmem_private_policy_tool.c
> new file mode 100644
> index 0000000000000..0769436b6068e
> --- /dev/null
> +++ b/tools/testing/selftests/dax/kmem_private_policy_tool.c

[ ... ]

> +/*
> + * A relative or static policy whose mask names the private node must still
> + * place memory on common nodes, and keep the user's mask.
> + */
> +static int common_policy(void *addr, size_t len, int nid, int requested_mode,
> +			 unsigned long mask, int task_policy)
> +{
> +	unsigned long reported = 0;
> +	int mode = 0;
> +	long rc;
> +
> +	if (task_policy) {
> +		if (syscall(SYS_mbind, addr, len, MPOL_DEFAULT, NULL, 0UL, 0UL))
> +			return FAIL;
> +		rc = syscall(SYS_set_mempolicy, requested_mode, &mask, MAXNODE);
> +	} else {
> +		rc = mbind_mask(addr, len, requested_mode, mask);
> +	}
> +	if (rc)
> +		return FAIL;
> +
> +	rc = syscall(SYS_get_mempolicy, &mode, &reported, MAXNODE,
> +		     task_policy ? NULL : addr, task_policy ? 0 : MPOL_F_ADDR);
> +	if (task_policy)
> +		syscall(SYS_set_mempolicy, MPOL_DEFAULT, NULL, 0UL);
> +	if (rc || mode != requested_mode || reported != mask ||
> +	    madvise(addr, len, MADV_DONTNEED))
> +		return FAIL;
> +	memset(addr, PATTERN, len);
> +	return pages_on_node(addr, NR_PAGES, nid) == 0 ? PASS : FAIL;
> +}

[Severity: Medium]
Does resetting the task policy to MPOL_DEFAULT before madvise(MADV_DONTNEED)
and memset() prevent the task_policy cases from testing page placement under
requested_mode?  When task_policy is set, both the VMA policy and the task
policy are back to MPOL_DEFAULT by the time memset() faults the pages in, so
the relative and static task policy tests always allocate under MPOL_DEFAULT.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=58

^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 34/60] mm/mempolicy: don't rebind policies onto private nodes
  2026-11-20 18:09 ` [PATCH 34/60] mm/mempolicy: don't rebind policies onto private nodes Gregory Price via B4 Relay
@ 2026-11-20 18:35   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:35 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> A cpuset can allow private nodes, and rebinding a policy to a new cpuset
> mask can move its nodes onto them:
>   - a static policy reapplies the raw user mask.
>   - relative and remapped policies follow node positions in the cpuset
>     mask.
> 
> Ordinary policies cannot use private nodes.
> 
> Rebind policies over the cpuset's nodes other than private ones, and
> record the remap base the same way, so adding or removing a private node
> moves no policy.
> 
> Cpusets keep common memory in every effective mask, so a mask with no
> node left is a bug. Warn, and keep the whole mask rather than leave the
> policy empty.
> 
> On systems without a private node, rebinding is unchanged.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=34


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 17/60] PM: hibernate: size the image from common memory
  2026-11-20 18:09 ` [PATCH 17/60] PM: hibernate: size the image from " Gregory Price via B4 Relay
@ 2026-11-20 18:35   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:35 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> hibernate_preallocate_memory(), enough_free_mem() and
> count_free_highmem_pages() count the free pages of every populated zone
> as room for the hibernation image, but the pages used to store the image
> can only come from common memory.
> 
> Count free pages only in zones on N_MEMORY_COMMON nodes.
> 
> The image metadata estimate is a separate sum.  The bitmaps track the
> page frames, and every zone's frames are tracked (regardless of node),
> so that sum still needs to be calculated for all populated zones.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=17


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 47/60] mm/vmscan: restrict reclaim to N_MEMORY_RECLAIM instead of N_MEMORY_COMMON
  2026-11-20 18:09 ` [PATCH 47/60] mm/vmscan: restrict reclaim to N_MEMORY_RECLAIM instead of N_MEMORY_COMMON Gregory Price via B4 Relay
@ 2026-11-20 18:38   ` sashiko-bot
  2026-11-20 18:41     ` Gregory Price (Meta)
  0 siblings, 1 reply; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:38 UTC (permalink / raw)
  To: Gregory Price (Meta), Yury Norov, Rasmus Villemoes,
	David Hildenbrand, Liam R. Howlett, Vlastimil Babka,
	Mike Rapoport, Michal Hocko, Andrew Morton, Oscar Salvador,
	Johannes Weiner, Suren Baghdasaryan, Brendan Jackman, Zi Yan,
	Jonathan Corbet, Shuah Khan, Randy Dunlap, Greg Kroah-Hartman,
	Rafael J. Wysocki, Danilo Krummrich, Johan Hovold, Joshua Hahn,
	Rakie Kim, Ying Huang, Matthew Brost, Byungchul Park,
	Alistair Popple, SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli,
	Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
	Mel Gorman, Valentin Schneider, K Prateek Nayak, Kees Cook,
	Kairui Song, Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen,
	Yuanchu Xie, Wei Xu, Baoquan He, Baolin Wang, Michal Hocko,
	David Rientjes, Pavel Machek, Len Brown, Nico Pache, Ryan Roberts,
	Dev Jain, Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin,
	Chengming Zhou, Lorenzo Stoakes, Pedro Falcato, Jann Horn,
	Chris Li, Kemeng Shi, Nhat Pham, Youngjun Park,
	Matthew Wilcox (Oracle), Jan Kara, Waiman Long, Ridong Chen,
	Tejun Heo, Michal Koutný, Dan Williams, Vishal Verma,
	Dave Jiang, Alison Schofield, Shuah Khan
  Cc: linux-mm, linux-cxl, damon, nvdimm

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [High] mm/vmscan: lru_gen_age_node() triggers global OOM kill from
  private kswapd
--

commit 5d53d0a1dd37bac1ef5e22f989ac592e04dac809
Author: Gregory Price (Meta) <gourry@gourry.net>

mm/vmscan: restrict reclaim to N_MEMORY_RECLAIM instead of N_MEMORY_COMMON

This commit updates kswapd startup, zone watermark setup, MGLRU debugfs
iteration, and proactive reclaim to check N_MEMORY_RECLAIM instead of
N_MEMORY_COMMON so private nodes that opt into reclaim are served by
reclaim and kswapd.

[ ... ]

> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index 3e934135e8052..03d0919d11bee 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c

[ ... ]

> @@ -7907,7 +7907,7 @@ static int __init kswapd_init(void)
>  {
>  	int nid;
>  
> -	for_each_node_state(nid, N_MEMORY_COMMON)
> +	for_each_node_state(nid, N_MEMORY_RECLAIM)
>   		kswapd_run(nid);
>  	register_sysctl_init("vm", vmscan_sysctl_table);
>  	return 0;

[Severity: High]
Can starting kswapd on a private N_MEMORY_RECLAIM node trigger a global
OOM kill when MGLRU min_ttl_ms is set?

When kswapd runs balance_pgdat() -> kswapd_age_node() ->
lru_gen_age_node() and every generation on the private node is younger
than lru_gen_min_ttl (or all lruvecs on the node are too small or below
memory.min), lru_gen_age_node() invokes out_of_memory() with an
unconstrained oom_control:

mm/vmscan.c:lru_gen_age_node() {
	...
	if (!reclaimable && mutex_trylock(&oom_lock)) {
		struct oom_control oc = {
			.gfp_mask = sc->gfp_mask,
		};

		out_of_memory(&oc);

		mutex_unlock(&oom_lock);
	}
}

Because oc.zonelist and oc.nodemask are NULL, out_of_memory() selects the
highest-scoring process across the entire system, even if that process
only uses common memory and holds no pages on the private node.  Should
the out_of_memory() block in lru_gen_age_node() be restricted to
N_MEMORY_COMMON nodes?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=47

^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 51/60] mm: restrict explicit migration to N_MEMORY_MEMPOLICY
  2026-11-20 18:09 ` [PATCH 51/60] mm: restrict explicit migration to N_MEMORY_MEMPOLICY Gregory Price via B4 Relay
@ 2026-11-20 18:38   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:38 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> mbind(MPOL_MF_MOVE), migrate_pages() and move_pages() act only on folios
> on common memory. N_MEMORY_MEMPOLICY nodes are the nodes memory policy
> and explicit migration may target, so let explicit migration also act on
> folios already on them.
> 
> Add folio_allows_mempolicy(), which requires the folio's node to permit
> memory policy, and use it instead.
> 
> Memory on other private nodes is still left alone:
>   - move_pages() refuses to migrate it.
>   - mbind(MPOL_MF_MOVE) and migrate_pages() skip its folios.
> 
> Cpuset migration can now find pages on these nodes. Evacuate them only
> from nodes the cpuset no longer allows. Private nodes are still never
> destinations for this kind of migration.
> 
> Migration keeps its best-effort behavior and can affect shared backing
> pages seen by other mappers.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=51


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 29/60] mm: keep private memory out of the system memory totals
  2026-11-20 18:09 ` [PATCH 29/60] mm: keep private memory out of the system memory totals Gregory Price via B4 Relay
@ 2026-11-20 18:40   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:40 UTC (permalink / raw)
  To: Gregory Price (Meta); +Cc: damon

> From: "Gregory Price (Meta)" <gourry@gourry.net>
> 
> totalram_pages() (MemTotal) counts every managed page, and MemFree
> counts every free page, including private memory.
> 
> Many limits and defaults are sized from totalram_pages():
>   - the overcommit limit
>   - OOM capacity
>   - tmpfs
>   - zswap defaults
>   - others
> 
> Private memory is not usable by ordinary allocations, so it should
> not inflate these counts.
> 
> [ ... ]
> Switch them to nr_free_pages() as well.
> 
> The vmstat counters and per-node meminfo still count private memory.
> 
> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=29


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 42/60] mm: fail private placement on exhaustion instead of OOM killing
  2026-11-20 18:09 ` [PATCH 42/60] mm: fail private placement on exhaustion instead of OOM killing Gregory Price via B4 Relay
@ 2026-11-20 18:40   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:40 UTC (permalink / raw)
  To: Gregory Price (Meta), Yury Norov, Rasmus Villemoes,
	David Hildenbrand, Liam R. Howlett, Vlastimil Babka,
	Mike Rapoport, Michal Hocko, Andrew Morton, Oscar Salvador,
	Johannes Weiner, Suren Baghdasaryan, Brendan Jackman, Zi Yan,
	Jonathan Corbet, Shuah Khan, Randy Dunlap, Greg Kroah-Hartman,
	Rafael J. Wysocki, Danilo Krummrich, Johan Hovold, Joshua Hahn,
	Rakie Kim, Ying Huang, Matthew Brost, Byungchul Park,
	Alistair Popple, SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli,
	Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
	Mel Gorman, Valentin Schneider, K Prateek Nayak, Kees Cook,
	Kairui Song, Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen,
	Yuanchu Xie, Wei Xu, Baoquan He, Baolin Wang, Michal Hocko,
	David Rientjes, Pavel Machek, Len Brown, Nico Pache, Ryan Roberts,
	Dev Jain, Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin,
	Chengming Zhou, Lorenzo Stoakes, Pedro Falcato, Jann Horn,
	Chris Li, Kemeng Shi, Nhat Pham, Youngjun Park,
	Matthew Wilcox (Oracle), Jan Kara, Waiman Long, Ridong Chen,
	Tejun Heo, Michal Koutný, Dan Williams, Vishal Verma,
	Dave Jiang, Alison Schofield, Shuah Khan
  Cc: linux-mm, linux-cxl, damon, nvdimm

Thank you for your contribution! Sashiko AI review found 3 potential
issues to consider:
- [High] mm: stale current->private_alloc_failed turns non-private OOM
  into SIGBUS
- [High] mm: private alloc failures bypassing __alloc_pages_may_oom()
  leak VM_FAULT_OOM
- [High] mm: VM_FAULT_SIGBUS on private alloc failure breaks
  FAULT_FLAG_UNSHARE callers
--

commit 7e8b136eb40856d46304f949553ac1db87e76cc8
Author: Gregory Price (Meta) <gourry@gourry.net>

mm: fail private placement on exhaustion instead of OOM killing

This commit skips the OOM killer when an allocation from a private
zonelist fails, marking the task with current->private_alloc_failed so
handle_mm_fault() converts VM_FAULT_OOM into VM_FAULT_SIGBUS.

> diff --git a/mm/memory.c b/mm/memory.c
> index 282266dd5e5df..4a8a08df28ff9 100644
> --- a/mm/memory.c
> +++ b/mm/memory.c
> @@ -7003,6 +7003,7 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address,
>  	}
>  
>  	is_droppable = !!(vma->vm_flags & VM_DROPPABLE);
> +	current->private_alloc_failed = 0;
>  
>  	/*
>  	 * Enable the memcg OOM handling for faults triggered in user
> @@ -7034,6 +7035,10 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address,
>  	if (is_droppable && (flags & FAULT_FLAG_USER))
>  		ret &= ~VM_FAULT_OOM;
>  
> +	/* Exhausted private memory fails like an empty HugeTLB pool. */
> +	if (unlikely(current->private_alloc_failed) && (ret & VM_FAULT_OOM))
> +		ret = (ret & ~VM_FAULT_OOM) | VM_FAULT_SIGBUS;

[Severity: High]
Does converting VM_FAULT_OOM to VM_FAULT_SIGBUS unconditionally break
FAULT_FLAG_UNSHARE callers?

Callers passing FAULT_FLAG_UNSHARE | FAULT_FLAG_REMOTE expect allocation
failures in wp_page_copy() to return VM_FAULT_OOM and treat
VM_FAULT_SIGBUS as concurrent truncation having unmapped the page:

arch/s390/mm/gmap_helpers.c:gmap_helper_unshare_zeropages() {
	...
		fault = handle_mm_fault(vma, addr,
					FAULT_FLAG_UNSHARE | FAULT_FLAG_REMOTE,
					NULL);
		if (fault & VM_FAULT_OOM)
			return -ENOMEM;
		/*
		 * Even if handle_mm_fault() returns VM_FAULT_SIGBUS, we did not
		 * fail to break COW and must retry, because we experienced
		 * concurrent truncation and the page mapping was modified.
		 */
		goto retry;
}

Because the private allocation failure leaves the shared zeropage mapped
at addr, gmap_helper_unshare_zeropages() finds the same zeropage on every
walk and loops forever.  In break_ksm(), VM_FAULT_SIGBUS terminates the
loop and returns 0 instead of -ENOMEM, allowing __ksm_del_vma(),
ksm_madvise(), and unmerge_and_remove_all_rmap_items() to clear
VM_MERGEABLE or tear down KSM rmap items while KSM pages are still mapped.

> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 63cc8729c41ad..ea48991ef5898 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -4176,6 +4176,20 @@ __alloc_pages_may_oom(gfp_t gfp_mask, unsigned int order,
>  	 * failures more gracefully we should just bail out here.
>  	 */
>  
> +	/*
> +	 * Private memory is a fixed pool, exhausting it fails the allocation
> +	 * rather than selecting an OOM victim. Otherwise, if there is some
> +	 * indication of progress because of a revoked request, just report
> +	 * some progress so the slowpath retries and relaxes it.
> +	 */
> +	if (ac->alloc_flags & ALLOC_ZONELIST_PRIVATE) {
> +		if (!alloc_private_reachable(gfp_mask, ac))
> +			*did_some_progress = 1;
> +		else
> +			current->private_alloc_failed = 1;
> +		goto out;
> +	}

[Severity: High]
Can current->private_alloc_failed stay set after a recoverable private
allocation failure and turn a later non-private VM_FAULT_OOM in the same
fault into VM_FAULT_SIGBUS?

handle_mm_fault() clears current->private_alloc_failed only at entry.  In
alloc_anon_folio(), trying an mTHP order in 1..PAGE_ALLOC_COSTLY_ORDER
with GFP_TRANSHUGE (when VM_HUGEPAGE is set) on a fragmented private node
reaches __alloc_pages_may_oom() and sets current->private_alloc_failed = 1
before falling back to order-0 folio_prealloc(), which succeeds.
Speculative swapin readahead in swap_vma_readahead() and slowpath restarts
via relax_private_alloc() at nopage: also leave
current->private_alloc_failed set after the fault continues.

If a later step in the same fault (such as mem_cgroup_charge() on the
order-0 folio or a common-memory allocation after relaxation) returns
VM_FAULT_OOM, handle_mm_fault() consumes the stale flag, reports
VM_FAULT_SIGBUS instead of OOM, and calls mem_cgroup_oom_synchronize(false)
to cancel memcg OOM handling.

[Severity: High]
What happens when a private allocation in a page fault fails without
reaching this ALLOC_ZONELIST_PRIVATE check?

Private nodes are commonly onlined only in ZONE_MOVABLE.  When a fault
cannot use ZONE_MOVABLE (for example, during FOLL_LONGTERM GUP where
PF_MEMALLOC_PIN causes current_gfp_context() to strip __GFP_MOVABLE),
first_zones_zonelist() in __alloc_pages_slowpath() finds no usable zone in
ZONELIST_PRIVATE and jumps directly to nopage:.  At nopage:,
relax_private_alloc() sees alloc_private_reachable() == true (since it
scans up to MAX_NR_ZONES - 1) and does not relax the allocation, so
__alloc_pages_slowpath() returns NULL with current->private_alloc_failed
still 0.  The early goto out branches earlier in __alloc_pages_may_oom()
(such as PF_DUMPCORE, order > PAGE_ALLOC_COSTLY_ORDER,
__GFP_RETRY_MAYFAIL | __GFP_THISNODE, or highest_zoneidx < ZONE_NORMAL)
also bypass this block.

When that happens, handle_mm_fault() returns raw VM_FAULT_OOM without
having invoked out_of_memory(): GUP receives -ENOMEM instead of -EFAULT,
and a user page fault enters pagefault_out_of_memory() and retries the
faulting instruction in an infinite fault loop.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=42

^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 47/60] mm/vmscan: restrict reclaim to N_MEMORY_RECLAIM instead of N_MEMORY_COMMON
  2026-11-20 18:38   ` sashiko-bot
@ 2026-11-20 18:41     ` Gregory Price (Meta)
  0 siblings, 0 replies; 124+ messages in thread
From: Gregory Price (Meta) @ 2026-11-20 18:41 UTC (permalink / raw)
  To: sashiko-reviews
  Cc: Gregory Price (Meta), Yury Norov, Rasmus Villemoes,
	David Hildenbrand, Liam R. Howlett, Vlastimil Babka,
	Mike Rapoport, Michal Hocko, Andrew Morton, Oscar Salvador,
	Johannes Weiner, Suren Baghdasaryan, Brendan Jackman, Zi Yan,
	Jonathan Corbet, Shuah Khan, Randy Dunlap, Greg Kroah-Hartman,
	Rafael J. Wysocki, Danilo Krummrich, Johan Hovold, Joshua Hahn,
	Rakie Kim, Ying Huang, Matthew Brost, Byungchul Park,
	Alistair Popple, SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli,
	Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
	Mel Gorman, Valentin Schneider, K Prateek Nayak, Kees Cook,
	Kairui Song, Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen,
	Yuanchu Xie, Wei Xu, Baoquan He, Baolin Wang, Michal Hocko,
	David Rientjes, Pavel Machek, Len Brown, Nico Pache, Ryan Roberts,
	Dev Jain, Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin,
	Chengming Zhou, Lorenzo Stoakes, Pedro Falcato, Jann Horn,
	Chris Li, Kemeng Shi, Nhat Pham, Youngjun Park,
	Matthew Wilcox (Oracle), Jan Kara, Waiman Long, Ridong Chen,
	Tejun Heo, Michal Koutný, Dan Williams, Vishal Verma,
	Dave Jiang, Alison Schofield, Shuah Khan, linux-mm, linux-cxl,
	damon, nvdimm

On 2026-11-20 18:38 +0000, sashiko-bot@kernel.org wrote:
> > @@ -7907,7 +7907,7 @@ static int __init kswapd_init(void)
> >  {
> >  	int nid;
> >  
> > -	for_each_node_state(nid, N_MEMORY_COMMON)
> > +	for_each_node_state(nid, N_MEMORY_RECLAIM)
> >   		kswapd_run(nid);
> >  	register_sysctl_init("vm", vmscan_sysctl_table);
> >  	return 0;
> 
> [Severity: High]
> Can starting kswapd on a private N_MEMORY_RECLAIM node trigger a global
> OOM kill when MGLRU min_ttl_ms is set?
> 
> When kswapd runs balance_pgdat() -> kswapd_age_node() ->
> lru_gen_age_node() and every generation on the private node is younger
> than lru_gen_min_ttl (or all lruvecs on the node are too small or below
> memory.min), lru_gen_age_node() invokes out_of_memory() with an
> unconstrained oom_control:
> 
> mm/vmscan.c:lru_gen_age_node() {
> 	...
> 	if (!reclaimable && mutex_trylock(&oom_lock)) {
> 		struct oom_control oc = {
> 			.gfp_mask = sc->gfp_mask,
> 		};
> 
> 		out_of_memory(&oc);
> 
> 		mutex_unlock(&oom_lock);
> 	}
> }
> 
> Because oc.zonelist and oc.nodemask are NULL, out_of_memory() selects the
> highest-scoring process across the entire system, even if that process
> only uses common memory and holds no pages on the private node.  Should
> the out_of_memory() block in lru_gen_age_node() be restricted to
> N_MEMORY_COMMON nodes?
>

I haven't been able to generate this condition, but I can adjust the oom
control here accordingly.  This should be trivial to adjust.

-- 
~Gregory


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 57/60] selftests/dax: test private kmem devices and services
  2026-11-20 18:09 ` [PATCH 57/60] selftests/dax: test private kmem devices and services Gregory Price via B4 Relay
@ 2026-11-20 18:42   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:42 UTC (permalink / raw)
  To: Gregory Price (Meta), Yury Norov, Rasmus Villemoes,
	David Hildenbrand, Liam R. Howlett, Vlastimil Babka,
	Mike Rapoport, Michal Hocko, Andrew Morton, Oscar Salvador,
	Johannes Weiner, Suren Baghdasaryan, Brendan Jackman, Zi Yan,
	Jonathan Corbet, Shuah Khan, Randy Dunlap, Greg Kroah-Hartman,
	Rafael J. Wysocki, Danilo Krummrich, Johan Hovold, Joshua Hahn,
	Rakie Kim, Ying Huang, Matthew Brost, Byungchul Park,
	Alistair Popple, SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli,
	Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
	Mel Gorman, Valentin Schneider, K Prateek Nayak, Kees Cook,
	Kairui Song, Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen,
	Yuanchu Xie, Wei Xu, Baoquan He, Baolin Wang, Michal Hocko,
	David Rientjes, Pavel Machek, Len Brown, Nico Pache, Ryan Roberts,
	Dev Jain, Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin,
	Chengming Zhou, Lorenzo Stoakes, Pedro Falcato, Jann Horn,
	Chris Li, Kemeng Shi, Nhat Pham, Youngjun Park,
	Matthew Wilcox (Oracle), Jan Kara, Waiman Long, Ridong Chen,
	Tejun Heo, Michal Koutný, Dan Williams, Vishal Verma,
	Dave Jiang, Alison Schofield, Shuah Khan
  Cc: linux-mm, linux-cxl, damon, nvdimm

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] selftests/dax: kmem bind in pn_online_private() without
  offline policy
--

commit 92dda1f7387efcfeae9e63aee96816a2f4e187f1
Author: Gregory Price (Meta) <gourry@gourry.net>

selftests/dax: test private kmem devices and services

This commit adds kselftests exercising the dax/kmem private sysfs
attribute across memory hotplug and verifying which MM services run on a
private kmem node.

> diff --git a/tools/testing/selftests/dax/private_node_common.sh b/tools/testing/selftests/dax/private_node_common.sh
> new file mode 100644
> index 0000000000000..57c5231e768b8
> --- /dev/null
> +++ b/tools/testing/selftests/dax/private_node_common.sh
> @@ -0,0 +1,208 @@
[ ... ]
> +# Move device $1 from driver $2 to kmem and online it as private memory on
> +# node $3.
> +pn_online_private()
> +{
> +	local dev=$1 driver=$2 nid=$3 name
> +
> +	name=$(basename "$dev")
> +	if [ "$driver" = kmem ]; then
> +		echo unplugged > "$dev/state" || return 1
> +	elif [ -n "$driver" ]; then
> +		echo "$name" > "/sys/bus/dax/drivers/$driver/unbind" || return 1
> +	fi
> +	# A private node holds the memory of this device only.
> +	if pn_node_in "$nid" has_memory; then
> +		return 1
> +	fi
> +	if [ "$driver" != kmem ]; then
> +		echo "$name" > /sys/bus/dax/drivers/kmem/new_id || return 1
> +		PN_NEW_ID=1
> +	fi
> +	echo unplugged > "$dev/state" &&
> +		echo 1 > "$dev/private" &&
> +		echo online_movable > "$dev/state" &&
> +		pn_node_is_private "$nid"
> +}

[Severity: Medium]
Can binding a non-kmem device via /sys/bus/dax/drivers/kmem/new_id wedge
the device as online common memory when
/sys/devices/system/memory/auto_online_blocks is set to online or
online_kernel?

dev_dax_kmem_probe() hotplugs the device at bind time with private=false
and online_type = mhp_get_default_online_type().  If auto_online_blocks
onlines the blocks into a common kernel zone during probe, unmovable
kernel allocations can cause "echo unplugged > $dev/state" to fail with
-EBUSY before private=1 can be set.  When pn_unbind_kmem_private() runs
on that failure, the unplug attempt fails at the same point after PN_DEV
is cleared, returning early without removing $name via
/sys/bus/dax/drivers/kmem/remove_id or rebinding the original driver.

Should pn_online_private() temporarily set
/sys/devices/system/memory/auto_online_blocks to offline around the
new_id bind, the way dax-kmem-hotplug.sh does?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=57

^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 54/60] mm/migrate: route explicit NUMA migration to private nodes
  2026-11-20 18:09 ` [PATCH 54/60] mm/migrate: route explicit NUMA migration to private nodes Gregory Price via B4 Relay
@ 2026-11-20 18:42   ` sashiko-bot
  2026-11-20 18:56     ` Gregory Price (Meta)
  0 siblings, 1 reply; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:42 UTC (permalink / raw)
  To: Gregory Price (Meta), Yury Norov, Rasmus Villemoes,
	David Hildenbrand, Liam R. Howlett, Vlastimil Babka,
	Mike Rapoport, Michal Hocko, Andrew Morton, Oscar Salvador,
	Johannes Weiner, Suren Baghdasaryan, Brendan Jackman, Zi Yan,
	Jonathan Corbet, Shuah Khan, Randy Dunlap, Greg Kroah-Hartman,
	Rafael J. Wysocki, Danilo Krummrich, Johan Hovold, Joshua Hahn,
	Rakie Kim, Ying Huang, Matthew Brost, Byungchul Park,
	Alistair Popple, SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli,
	Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
	Mel Gorman, Valentin Schneider, K Prateek Nayak, Kees Cook,
	Kairui Song, Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen,
	Yuanchu Xie, Wei Xu, Baoquan He, Baolin Wang, Michal Hocko,
	David Rientjes, Pavel Machek, Len Brown, Nico Pache, Ryan Roberts,
	Dev Jain, Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin,
	Chengming Zhou, Lorenzo Stoakes, Pedro Falcato, Jann Horn,
	Chris Li, Kemeng Shi, Nhat Pham, Youngjun Park,
	Matthew Wilcox (Oracle), Jan Kara, Waiman Long, Ridong Chen,
	Tejun Heo, Michal Koutný, Dan Williams, Vishal Verma,
	Dave Jiang, Alison Schofield, Shuah Khan
  Cc: linux-mm, linux-cxl, damon, nvdimm

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [High] mm/migrate: call to undeclared __folio_alloc_flags() in
  alloc_migration_target()
--

commit ca3cf1708bab40e7c2c443ad00cbc882e2163cee
Author: Gregory Price (Meta) <gourry@gourry.net>

mm/migrate: route explicit NUMA migration to private nodes

This commit routes move_pages(2) and migrate_pages(2) allocations for
N_MEMORY_MEMPOLICY private destination nodes through the node's private
zonelist by passing ALLOC_ZONELIST_PRIVATE via migration_target_control.

> diff --git a/mm/migrate.c b/mm/migrate.c
> index 6791834f14fe1..eda961d43df0e 100644
> --- a/mm/migrate.c
> +++ b/mm/migrate.c
> @@ -2250,6 +2250,9 @@ struct folio *alloc_migration_target(struct folio *src, unsigned long private)
>  	if (is_highmem_idx(zidx) || zidx == ZONE_MOVABLE)
>  		gfp_mask |= __GFP_HIGHMEM;
>  
> +	if (mtc->alloc_flags)
> +		return __folio_alloc_flags(gfp_mask, order, nid, mtc->nmask,
> +					   mtc->alloc_flags);
>  	return __folio_alloc(gfp_mask, order, nid, mtc->nmask);
>  }

[Severity: High]
Where is __folio_alloc_flags() declared?  Neither include/linux/gfp.h nor
mm/page_alloc.c defines __folio_alloc_flags() or
__folio_alloc_flags_noprof(), so building mm/migrate.c with
CONFIG_MIGRATION=y fails with an implicit function declaration error.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=54

^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 31/60] cgroup/cpuset: inherit common memory when a nodemask is private only
  2026-11-20 18:09 ` [PATCH 31/60] cgroup/cpuset: inherit common memory when a nodemask is private only Gregory Price via B4 Relay
@ 2026-11-20 18:43   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:43 UTC (permalink / raw)
  To: Gregory Price (Meta), Yury Norov, Rasmus Villemoes,
	David Hildenbrand, Liam R. Howlett, Vlastimil Babka,
	Mike Rapoport, Michal Hocko, Andrew Morton, Oscar Salvador,
	Johannes Weiner, Suren Baghdasaryan, Brendan Jackman, Zi Yan,
	Jonathan Corbet, Shuah Khan, Randy Dunlap, Greg Kroah-Hartman,
	Rafael J. Wysocki, Danilo Krummrich, Johan Hovold, Joshua Hahn,
	Rakie Kim, Ying Huang, Matthew Brost, Byungchul Park,
	Alistair Popple, SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli,
	Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
	Mel Gorman, Valentin Schneider, K Prateek Nayak, Kees Cook,
	Kairui Song, Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen,
	Yuanchu Xie, Wei Xu, Baoquan He, Baolin Wang, Michal Hocko,
	David Rientjes, Pavel Machek, Len Brown, Nico Pache, Ryan Roberts,
	Dev Jain, Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin,
	Chengming Zhou, Lorenzo Stoakes, Pedro Falcato, Jann Horn,
	Chris Li, Kemeng Shi, Nhat Pham, Youngjun Park,
	Matthew Wilcox (Oracle), Jan Kara, Waiman Long, Ridong Chen,
	Tejun Heo, Michal Koutný, Dan Williams, Vishal Verma,
	Dave Jiang, Alison Schofield, Shuah Khan
  Cc: linux-mm, linux-cxl, damon, nvdimm

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] cgroup/cpuset: private nodes in mwork->from skew
  do_migrate_pages() remap
--

commit a812b8d4e921d8df891074bca9a3aa4014ffe723
Author: Gregory Price (Meta) <gourry@gourry.net>

cgroup/cpuset: inherit common memory when a nodemask is private only

This commit updates cpuset nodemask calculation and validation so that a
cpuset.mems mask containing only private memory nodes inherits common
memory nodes from its parent in v2 or is rejected in v1, and excludes
private memory nodes from automatic page migration destinations.

> diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
> index db32d330cfbc2..6802ca13c46b7 100644
> --- a/kernel/cgroup/cpuset.c
> +++ b/kernel/cgroup/cpuset.c

[ ... ]

> @@ -2667,6 +2687,8 @@ static void cpuset_migrate_mm_workfn(struct work_struct *work)
>  	struct cpuset_migrate_mm_work *mwork =
>  		container_of(work, struct cpuset_migrate_mm_work, work);
>  
> +	/* Private nodes are never migration destinations. */
> +	nodes_andnot(mwork->to, mwork->to, node_states[N_MEMORY_PRIVATE]);
>  	/* on a wq worker, no need to worry about %current's mems_allowed */
>  	do_migrate_pages(mwork->mm, &mwork->from, &mwork->to, MPOL_MF_MOVE_ALL);
>  	mmput(mwork->mm);

[Severity: Medium]
Does stripping N_MEMORY_PRIVATE only from mwork->to while leaving private
nodes in mwork->from skew the node remapping in do_migrate_pages()?

mm/mempolicy.c:do_migrate_pages() {
    ...
			if ((nodes_weight(*from) != nodes_weight(*to)) &&
						(node_isset(s, *to)))
				continue;

			d = node_remap(s, *from, *to);
    ...
}

Both mwork->from (from cs->old_mems_allowed) and mwork->to come from
guarantee_online_mems(), which includes all N_MEMORY nodes.  For example,
with common nodes C0 and C1 and private node P2, if effective_mems changes
from {C1, P2} to {C0, C1, P2} (or {C0, C1}), mwork->from is {C1, P2} and
mwork->to becomes {C0, C1}.  Because both masks have weight 2,
do_migrate_pages() does not skip C1 even though it is still set in
mwork->to, and node_remap() maps ordinal 0 of mwork->from (C1) to ordinal
0 of mwork->to (C0), migrating pages off C1 onto C0 even though C1
remained allowed.

Also, because cpuset_migrate_mm() checks nodes_equal(*from, *to) before
private nodes are stripped, adding or removing only a private node (e.g.
{C0} -> {C0, P2}) still queues migration work and runs
do_migrate_pages() -> lru_cache_disable() when the common node masks are
unchanged.  Later in this series, commit "mm: restrict explicit migration
to N_MEMORY_MEMPOLICY" strips retained private nodes (mwork->to &
node_states[N_MEMORY_PRIVATE]) from mwork->from, but private nodes present
only in mwork->from remain set and still skew nodes_weight(*from) and
node_remap() when common nodes change at the same time.  Filtering
N_MEMORY_PRIVATE from both *from and *to in cpuset_migrate_mm() before the
nodes_equal() check would avoid both issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=31

^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 53/60] mm/mempolicy: add MPOL_F_PRIVATE_NODES for private node bindings
  2026-11-20 18:09 ` [PATCH 53/60] mm/mempolicy: add MPOL_F_PRIVATE_NODES for private node bindings Gregory Price via B4 Relay
@ 2026-11-20 18:44   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:44 UTC (permalink / raw)
  To: Gregory Price (Meta), Yury Norov, Rasmus Villemoes,
	David Hildenbrand, Liam R. Howlett, Vlastimil Babka,
	Mike Rapoport, Michal Hocko, Andrew Morton, Oscar Salvador,
	Johannes Weiner, Suren Baghdasaryan, Brendan Jackman, Zi Yan,
	Jonathan Corbet, Shuah Khan, Randy Dunlap, Greg Kroah-Hartman,
	Rafael J. Wysocki, Danilo Krummrich, Johan Hovold, Joshua Hahn,
	Rakie Kim, Ying Huang, Matthew Brost, Byungchul Park,
	Alistair Popple, SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli,
	Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
	Mel Gorman, Valentin Schneider, K Prateek Nayak, Kees Cook,
	Kairui Song, Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen,
	Yuanchu Xie, Wei Xu, Baoquan He, Baolin Wang, Michal Hocko,
	David Rientjes, Pavel Machek, Len Brown, Nico Pache, Ryan Roberts,
	Dev Jain, Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin,
	Chengming Zhou, Lorenzo Stoakes, Pedro Falcato, Jann Horn,
	Chris Li, Kemeng Shi, Nhat Pham, Youngjun Park,
	Matthew Wilcox (Oracle), Jan Kara, Waiman Long, Ridong Chen,
	Tejun Heo, Michal Koutný, Dan Williams, Vishal Verma,
	Dave Jiang, Alison Schofield, Shuah Khan
  Cc: linux-mm, linux-cxl, damon, nvdimm

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [High] mm/mempolicy: incomplete suspension of private bindings in
  alloc_pages_mpol()
--

commit ca3f7c695f126ea25e293faadb9871a86c3c7899
Author: Gregory Price (Meta) <gourry@gourry.net>

mm/mempolicy: add MPOL_F_PRIVATE_NODES for private node bindings

This commit adds the MPOL_F_PRIVATE_NODES mode flag for mbind() to explicitly
bind a VMA with MPOL_BIND to a single private memory node in
N_MEMORY_MEMPOLICY.

> diff --git a/mm/mempolicy.c b/mm/mempolicy.c
> index 9d2e56017ebe5..cb5419cb7ee84 100644
> --- a/mm/mempolicy.c
> +++ b/mm/mempolicy.c
[ ... ]
> @@ -2495,10 +2539,20 @@ static struct page *alloc_pages_preferred_many(gfp_t gfp, unsigned int order,
>  static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order,
>  		struct mempolicy *pol, pgoff_t ilx, int nid)
>  {
> +	unsigned int alloc_flags = ALLOC_DEFAULT;
>  	nodemask_t *nodemask;
>  	struct page *page;
>  
>  	nodemask = policy_nodemask(gfp, pol, ilx, &nid);
> +	if (nodemask && (pol->flags & MPOL_F_PRIVATE_NODES)) {
> +		int private_nid = first_node(*nodemask);
> +
> +		if (node_state(private_nid, N_MEMORY_MEMPOLICY) &&
> +		    node_state(private_nid, N_MEMORY_PRIVATE)) {
> +			nid = private_nid;
> +			alloc_flags = ALLOC_ZONELIST_PRIVATE;
> +		}
> +	}
>  
>  	if (pol->mode == MPOL_PREFERRED_MANY)
>  		return alloc_pages_preferred_many(gfp, order, nid, nodemask);
> @@ -2520,24 +2574,26 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order,
>  		    pol->mode != MPOL_WEIGHTED_INTERLEAVE &&
>  		    (!nodemask || node_isset(nid, *nodemask))) {
>  			/*
> -			 * First, try to allocate THP only on local node, but
> +			 * First, try to allocate THP only on the selected node, but
>  			 * don't reclaim unnecessarily, just compact.
>  			 */
>  			page = __alloc_frozen_pages_noprof(
>  				gfp | __GFP_THISNODE | __GFP_NORETRY, order,
> -				nid, NULL, ALLOC_DEFAULT);
> +				nid, (alloc_flags & ALLOC_ZONELIST_PRIVATE) ?
> +					 nodemask : NULL, alloc_flags);
>  			if (page || !(gfp & __GFP_DIRECT_RECLAIM))
>  				return page;
>  			/*
>  			 * If hugepage allocations are configured to always
>  			 * synchronous compact or the vma has been madvised
>  			 * to prefer hugepage backing, retry allowing remote
> -			 * memory with both reclaim and compact as well.
> +			 * memory with both reclaim and compact as well. A
> +			 * private binding stays on its local zonelist.
>  			 */
>  		}
>  	}
>  
> -	page = __alloc_frozen_pages_noprof(gfp, order, nid, nodemask, ALLOC_DEFAULT);
> +	page = __alloc_frozen_pages_noprof(gfp, order, nid, nodemask, alloc_flags);

[Severity: High]
Does alloc_pages_mpol() fail to suspend a private binding to common memory
when private_nid becomes ineligible without returning as N_MEMORY_COMMON?

When private_nid is not in both N_MEMORY_MEMPOLICY and N_MEMORY_PRIVATE,
alloc_pages_mpol() leaves nodemask set to &pol->nodes with alloc_flags =
ALLOC_DEFAULT instead of clearing nodemask to NULL when
!node_state(private_nid, N_MEMORY_COMMON).

If private_nid is offlined, node_clear_memory_state() clears N_MEMORY_PRIVATE
and N_HIGH_MEMORY, so apply_policy_zone() still accepts ZONE_MOVABLE faults.
When CONFIG_CPUSETS=n (or before cpuset hotplug work updates
current->mems_allowed), policy_nodemask() still returns &pol->nodes. Passing
nodemask = &pol->nodes with ALLOC_DEFAULT finds no zones in ZONELIST_FALLBACK,
and in __alloc_pages_slowpath() alloc_is_private() returns false because
ALLOC_ZONELIST_PRIVATE is not set and N_MEMORY_PRIVATE was already cleared:

mm/page_alloc.c:alloc_is_private() {
	if (ac->alloc_flags & ALLOC_ZONELIST_PRIVATE)
		return true;
	return ac->nodemask &&
	       nodes_intersects(*ac->nodemask, node_states[N_MEMORY_PRIVATE]);
}

As a result, relax_private_alloc() does not relax ac->nodemask, the fault
fails without setting current->private_alloc_failed, and handle_mm_fault()
returns VM_FAULT_OOM and invokes the global OOM killer instead of falling
back to common memory. Likewise, if private_nid stays online in
N_MEMORY_PRIVATE without N_MEMORY_MEMPOLICY, leaving nodemask = &pol->nodes
with ALLOC_DEFAULT misses the fastpath on every fault and forces every
allocation through __alloc_pages_slowpath() and relax_private_alloc().

In addition, if set_mempolicy_home_node() was used to set pol->home_node to
private_nid, policy_nodemask() unconditionally sets *nid = pol->home_node
even when the private binding is suspended:

mm/mempolicy.c:policy_nodemask() {
    ...
	case MPOL_BIND:
		if (apply_policy_zone(pol, gfp_zone(current_gfp_context(gfp))) &&
		    cpuset_nodemask_valid_mems_allowed(&pol->nodes))
			nodemask = &pol->nodes;
		if (pol->home_node != NUMA_NO_NODE)
			*nid = pol->home_node;
		break;
    ...
}

On the PMD THP branch in alloc_pages_mpol(), __alloc_frozen_pages_noprof() is
then called with __GFP_THISNODE, nid = private_nid, and ALLOC_DEFAULT, which
selects private_nid's empty ZONELIST_NOFALLBACK and causes
GFP_TRANSHUGE_LIGHT allocations (!(gfp & __GFP_DIRECT_RECLAIM)) to return
NULL instead of allocating from common memory.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=53

^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 40/60] mm/page_alloc: let MM callers select the local private zonelist
  2026-11-20 18:09 ` [PATCH 40/60] mm/page_alloc: let MM callers select the local private zonelist Gregory Price via B4 Relay
@ 2026-11-20 18:46   ` sashiko-bot
  0 siblings, 0 replies; 124+ messages in thread
From: sashiko-bot @ 2026-11-20 18:46 UTC (permalink / raw)
  To: Gregory Price (Meta), Yury Norov, Rasmus Villemoes,
	David Hildenbrand, Liam R. Howlett, Vlastimil Babka,
	Mike Rapoport, Michal Hocko, Andrew Morton, Oscar Salvador,
	Johannes Weiner, Suren Baghdasaryan, Brendan Jackman, Zi Yan,
	Jonathan Corbet, Shuah Khan, Randy Dunlap, Greg Kroah-Hartman,
	Rafael J. Wysocki, Danilo Krummrich, Johan Hovold, Joshua Hahn,
	Rakie Kim, Ying Huang, Matthew Brost, Byungchul Park,
	Alistair Popple, SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli,
	Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
	Mel Gorman, Valentin Schneider, K Prateek Nayak, Kees Cook,
	Kairui Song, Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen,
	Yuanchu Xie, Wei Xu, Baoquan He, Baolin Wang, Michal Hocko,
	David Rientjes, Pavel Machek, Len Brown, Nico Pache, Ryan Roberts,
	Dev Jain, Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin,
	Chengming Zhou, Lorenzo Stoakes, Pedro Falcato, Jann Horn,
	Chris Li, Kemeng Shi, Nhat Pham, Youngjun Park,
	Matthew Wilcox (Oracle), Jan Kara, Waiman Long, Ridong Chen,
	Tejun Heo, Michal Koutný, Dan Williams, Vishal Verma,
	Dave Jiang, Alison Schofield, Shuah Khan
  Cc: linux-mm, linux-cxl, damon, nvdimm

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] mm/page_alloc: skipped reserve retry in
  __alloc_pages_slowpath()
--

commit c7c69e6797446cf3c44127e33fd21dc6c24deefe
Author: Gregory Price (Meta) <gourry@gourry.net>

mm/page_alloc: let MM callers select the local private zonelist

This commit adds ALLOC_ZONELIST_PRIVATE to allow internal MM callers to
select the local private zonelist independently of __GFP_THISNODE, while
retaining the nodemask and cpuset filtering during reserve allocations.

> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 0bd21916cea1b..a219057931c3d 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -4926,16 +4926,24 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
>  		goto got_pg;
>  
>  	reserve_flags = __gfp_pfmemalloc_flags(gfp_mask);
> -	if (reserve_flags)
> +	if (reserve_flags) {
> +		unsigned int keep = ALLOC_KSWAPD;
> +
> +		/* The cpuset still grants a private request's node. */
> +		if (ac->alloc_flags & ALLOC_ZONELIST_PRIVATE)
> +			keep |= ALLOC_CPUSET;
>  		alloc_flags = alloc_flags_cma(gfp_mask) | reserve_flags |
> -				ac->alloc_flags | (alloc_flags & ALLOC_KSWAPD);
> +				ac->alloc_flags | (alloc_flags & keep);
> +	}
>  
>  	/*
>  	 * Reset the nodemask and zonelist iterators if memory policies can be
>  	 * ignored. These allocations are high priority and system rather than
> -	 * user oriented.
> +	 * user oriented. A private request keeps its node: private memory has
> +	 * no reserves to grant.
>  	 */
> -	if (!(alloc_flags & ALLOC_CPUSET) || reserve_flags) {
> +	if ((!(alloc_flags & ALLOC_CPUSET) || reserve_flags) &&
> +	    !(ac->alloc_flags & ALLOC_ZONELIST_PRIVATE)) {
>  		ac->nodemask = NULL;
>  		ac->preferred_zoneref = first_zones_zonelist(ac->zonelist,
>  					ac->highest_zoneidx, ac->nodemask);

[Severity: Medium]
Does gating this entire block on !(ac->alloc_flags & ALLOC_ZONELIST_PRIVATE)
skip the can_retry_reserves retry after reserve_flags updates alloc_flags?

When reserve_flags is non-zero (for example, when current has PF_MEMALLOC
set or is an OOM victim), the preceding block updates alloc_flags with
reserve_flags and clears ALLOC_NOFRAGMENT.  Skipping the block below also
skips the can_retry_reserves goto retry, so PF_MEMALLOC and
!can_direct_reclaim callers jump straight to nopage without ever calling
get_page_from_freelist() with the updated alloc_flags (and
!can_direct_reclaim callers also miss the defrag_mode ALLOC_NOFRAGMENT
retry because ALLOC_NOFRAGMENT was already cleared):

mm/page_alloc.c:__alloc_pages_slowpath() {
	...
		if (can_retry_reserves) {
			can_retry_reserves = false;
			goto retry;
		}
	}

	/* Caller is not willing to reclaim, we can't balance anything */
	if (!can_direct_reclaim) {
		if (defrag_mode && (alloc_flags & ALLOC_NOFRAGMENT) &&
		    !(gfp_mask & __GFP_NORETRY) &&
		    (gfp_mask & __GFP_KSWAPD_RECLAIM)) {
			alloc_flags &= ~ALLOC_NOFRAGMENT;
			goto retry;
		}
		goto nopage;
	}

	/* Avoid recursion of direct reclaim */
	if (current->flags & PF_MEMALLOC)
		goto nopage;
	...
}

Only the ac->nodemask and ac->preferred_zoneref reset should be skipped for
ALLOC_ZONELIST_PRIVATE when reserve_flags adjusted alloc_flags, while still
performing the can_retry_reserves retry.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261120-private_nodes_v6-v1-0-74f5f9a115c2@gourry.net?part=40

^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 54/60] mm/migrate: route explicit NUMA migration to private nodes
  2026-11-20 18:42   ` sashiko-bot
@ 2026-11-20 18:56     ` Gregory Price (Meta)
  0 siblings, 0 replies; 124+ messages in thread
From: Gregory Price (Meta) @ 2026-11-20 18:56 UTC (permalink / raw)
  To: sashiko-reviews
  Cc: Gregory Price (Meta), Yury Norov, Rasmus Villemoes,
	David Hildenbrand, Liam R. Howlett, Vlastimil Babka,
	Mike Rapoport, Michal Hocko, Andrew Morton, Oscar Salvador,
	Johannes Weiner, Suren Baghdasaryan, Brendan Jackman, Zi Yan,
	Jonathan Corbet, Shuah Khan, Randy Dunlap, Greg Kroah-Hartman,
	Rafael J. Wysocki, Danilo Krummrich, Johan Hovold, Joshua Hahn,
	Rakie Kim, Ying Huang, Matthew Brost, Byungchul Park,
	Alistair Popple, SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli,
	Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
	Mel Gorman, Valentin Schneider, K Prateek Nayak, Kees Cook,
	Kairui Song, Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen,
	Yuanchu Xie, Wei Xu, Baoquan He, Baolin Wang, Michal Hocko,
	David Rientjes, Pavel Machek, Len Brown, Nico Pache, Ryan Roberts,
	Dev Jain, Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin,
	Chengming Zhou, Lorenzo Stoakes, Pedro Falcato, Jann Horn,
	Chris Li, Kemeng Shi, Nhat Pham, Youngjun Park,
	Matthew Wilcox (Oracle), Jan Kara, Waiman Long, Ridong Chen,
	Tejun Heo, Michal Koutný, Dan Williams, Vishal Verma,
	Dave Jiang, Alison Schofield, Shuah Khan, linux-mm, linux-cxl,
	damon, nvdimm

On 2026-11-20 18:42 +0000, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 1 potential
> issue to consider:
> - [High] mm/migrate: call to undeclared __folio_alloc_flags() in
>   alloc_migration_target()
> --
> 
> commit ca3cf1708bab40e7c2c443ad00cbc882e2163cee
> Author: Gregory Price (Meta) <gourry@gourry.net>
> 
> mm/migrate: route explicit NUMA migration to private nodes
> 
> This commit routes move_pages(2) and migrate_pages(2) allocations for
> N_MEMORY_MEMPOLICY private destination nodes through the node's private
> zonelist by passing ALLOC_ZONELIST_PRIVATE via migration_target_control.
> 
> > diff --git a/mm/migrate.c b/mm/migrate.c
> > index 6791834f14fe1..eda961d43df0e 100644
> > --- a/mm/migrate.c
> > +++ b/mm/migrate.c
> > @@ -2250,6 +2250,9 @@ struct folio *alloc_migration_target(struct folio *src, unsigned long private)
> >  	if (is_highmem_idx(zidx) || zidx == ZONE_MOVABLE)
> >  		gfp_mask |= __GFP_HIGHMEM;
> >  
> > +	if (mtc->alloc_flags)
> > +		return __folio_alloc_flags(gfp_mask, order, nid, mtc->nmask,
> > +					   mtc->alloc_flags);
> >  	return __folio_alloc(gfp_mask, order, nid, mtc->nmask);
> >  }
> 
> [Severity: High]
> Where is __folio_alloc_flags() declared?  Neither include/linux/gfp.h nor
> mm/page_alloc.c defines __folio_alloc_flags() or
> __folio_alloc_flags_noprof(), so building mm/migrate.c with
> CONFIG_MIGRATION=y fails with an implicit function declaration error.
>

b4 dropped a commit on me when i made the prep branch because it
it was tagged Author: Brendan Jackman and I filtered on myself.

this commit should have immediately preceded this one:

https://lore.kernel.org/linux-mm/20260923211041.3127588-4-gourry@gourry.net/

the branch in the cover letter has everything

I will respin this again, but i'll address the sashiko reports and
let this sit for a bit.  The first 2 chunks are still largely
reviewable.

-- 
~Gregory


^ permalink raw reply	[flat|nested] 124+ messages in thread

* Re: [PATCH 00/60] Private NUMA nodes
  2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
                   ` (59 preceding siblings ...)
  2026-11-20 18:09 ` [PATCH 60/60] Documentation/mm: describe private memory nodes Gregory Price via B4 Relay
@ 2026-11-20 18:59 ` Gregory Price (Meta)
  60 siblings, 0 replies; 124+ messages in thread
From: Gregory Price (Meta) @ 2026-11-20 18:59 UTC (permalink / raw)
  To: Gregory Price (Meta)
  Cc: Yury Norov, Rasmus Villemoes, David Hildenbrand, Liam R. Howlett,
	Vlastimil Babka, Mike Rapoport, Michal Hocko, Andrew Morton,
	Oscar Salvador, Johannes Weiner, Suren Baghdasaryan,
	Brendan Jackman, Zi Yan, Jonathan Corbet, Shuah Khan,
	Randy Dunlap, Greg Kroah-Hartman, Rafael J. Wysocki,
	Danilo Krummrich, Johan Hovold, Joshua Hahn, Rakie Kim,
	Ying Huang, Matthew Brost, Byungchul Park, Alistair Popple,
	SJ Park, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
	Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
	Valentin Schneider, K Prateek Nayak, Kees Cook, Kairui Song,
	Barry Song, Qi Zheng, Shakeel Butt, Axel Rasmussen, Yuanchu Xie,
	Wei Xu, Baoquan He, Baolin Wang, Michal Hocko, David Rientjes,
	Pavel Machek, Len Brown, Nico Pache, Ryan Roberts, Dev Jain,
	Lance Yang, Usama Arif, Kiryl Shutsemau, Xu Xin, Chengming Zhou,
	Lorenzo Stoakes, Pedro Falcato, Jann Horn, Chris Li, Kemeng Shi,
	Nhat Pham, Youngjun Park, Matthew Wilcox (Oracle), Jan Kara,
	Waiman Long, Ridong Chen, Tejun Heo, Michal Koutný,
	Dan Williams, Vishal Verma, Dave Jiang, Alison Schofield,
	Shuah Khan, linux-kernel, linux-mm, linux-cxl, linux-doc,
	driver-core, damon, linux-pm, linux-fsdevel, cgroups, nvdimm,
	linux-kselftest

On 2026-11-20 13:08 -0500, Gregory Price (Meta) wrote:
> This is v6 of the Private NUMA node series. The series falls into four
> parts, each building on the one before:
> 
>    1-18  mm: introduce N_MEMORY_COMMON
>   19-26  restrict folio-walking MM services to common memory
>   27-44  N_MEMORY_PRIVATE, kept out of cpusets, mempolicy, accounting
>          and the page allocator's ordinary paths
>   45-61  private node features, memory hotplug of private memory,
>          dax/kmem private mode, selftests and documentation
> 
> Patches 1-26 change no behavior on any system and can be reviewed and
> merged on their own. Without private nodes, every path in patches 27-61
> behaves as before.
> 
> The full stack is also available at:
> https://github.com/gourryinverse/linux/tree/scratch/gourry/managed_nodes/private_nodes_v6
>

Apologies - my b4 prep dropped a commit from the stack because I made
the mistake of filtering on author (myself) and it stripped out this
commit:

https://lore.kernel.org/linux-mm/20260923211041.3127588-4-gourry@gourry.net/

The commit is one the branch listed above, and should be otherwise
indepedent of the series (can be pulled in ahead of everything else).

Sorry for the noise / compilation error, won't happen again.

-- 
~Gregory


^ permalink raw reply	[flat|nested] 124+ messages in thread

end of thread, other threads:[~2026-11-20 18:59 UTC | newest]

Thread overview: 124+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-20 18:08 [PATCH 00/60] Private NUMA nodes Gregory Price via B4 Relay
2026-11-20 18:08 ` [PATCH 01/60] nodemask: add node-state iterators Gregory Price via B4 Relay
2026-11-20 18:20   ` sashiko-bot
2026-11-20 18:08 ` [PATCH 02/60] mm: publish and clear node memory states via helpers Gregory Price via B4 Relay
2026-11-20 18:20   ` sashiko-bot
2026-11-20 18:08 ` [PATCH 03/60] mm: introduce the common-memory node state Gregory Price via B4 Relay
2026-11-20 18:22   ` sashiko-bot
2026-11-20 18:08 ` [PATCH 04/60] mm/page_alloc: build zonelists from common memory Gregory Price via B4 Relay
2026-11-20 18:27   ` sashiko-bot
2026-11-20 18:08 ` [PATCH 05/60] mm/mempolicy: keep interleave and tmpfs mount policies on " Gregory Price via B4 Relay
2026-11-20 18:22   ` sashiko-bot
2026-11-20 18:08 ` [PATCH 06/60] mm/memory_hotplug: migrate offlined pages to " Gregory Price via B4 Relay
2026-11-20 18:20   ` sashiko-bot
2026-11-20 18:08 ` [PATCH 07/60] mm/memory-tiers: restrict demotion topology " Gregory Price via B4 Relay
2026-11-20 18:21   ` sashiko-bot
2026-11-20 18:08 ` [PATCH 08/60] mm/migrate: reject move_pages() destinations outside " Gregory Price via B4 Relay
2026-11-20 18:19   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 09/60] mm/damon: reject migration " Gregory Price via B4 Relay
2026-11-20 18:18   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 10/60] sched/numa: restrict NUMA migration targets to " Gregory Price via B4 Relay
2026-11-20 18:21   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 11/60] fork: check cached stack locality only on common memory nodes Gregory Price via B4 Relay
2026-11-20 18:20   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 12/60] mm/vmscan: restrict reclaim to common memory Gregory Price via B4 Relay
2026-11-20 18:25   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 13/60] mm/compaction: restrict compaction " Gregory Price via B4 Relay
2026-11-20 18:20   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 14/60] mm: base hash distribution on common memory nodes Gregory Price via B4 Relay
2026-11-20 18:19   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 15/60] mm/page_alloc: count common memory nodes for kswapd recovery Gregory Price via B4 Relay
2026-11-20 18:20   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 16/60] mm/oom_kill: compare OOM constraints against common memory Gregory Price via B4 Relay
2026-11-20 18:21   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 17/60] PM: hibernate: size the image from " Gregory Price via B4 Relay
2026-11-20 18:35   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 18/60] mm/khugepaged: base the recommended min_free_kbytes on " Gregory Price via B4 Relay
2026-11-20 18:20   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 19/60] mm: add folio_is_common_memory() and page_is_common_memory() Gregory Price via B4 Relay
2026-11-20 18:20   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 20/60] mm/ksm: restrict merging to common memory Gregory Price via B4 Relay
2026-11-20 18:20   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 21/60] mm/khugepaged: restrict collapse " Gregory Price via B4 Relay
2026-11-20 18:21   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 22/60] mm/mempolicy: don't NUMA-hint folios outside " Gregory Price via B4 Relay
2026-11-20 18:21   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 23/60] mm/damon: restrict DAMON-native folio operations to " Gregory Price via B4 Relay
2026-11-20 18:21   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 24/60] mm/madvise: restrict reclaim advice " Gregory Price via B4 Relay
2026-11-20 18:19   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 25/60] mm/mempolicy: queue only common memory for mbind() and migrate_pages() Gregory Price via B4 Relay
2026-11-20 18:25   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 26/60] mm/migrate: leave pages outside common memory in move_pages() Gregory Price via B4 Relay
2026-11-20 18:21   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 27/60] mm: add the private-memory node state Gregory Price via B4 Relay
2026-11-20 18:25   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 28/60] mm: select memory features per node Gregory Price via B4 Relay
2026-11-20 18:24   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 29/60] mm: keep private memory out of the system memory totals Gregory Price via B4 Relay
2026-11-20 18:40   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 30/60] mm/memory_hotplug: keep private memory out of the auto-movable ratio Gregory Price via B4 Relay
2026-11-20 18:33   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 31/60] cgroup/cpuset: inherit common memory when a nodemask is private only Gregory Price via B4 Relay
2026-11-20 18:43   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 32/60] cgroup/cpuset: keep spread page cache off private nodes Gregory Price via B4 Relay
2026-11-20 18:31   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 33/60] cgroup/cpuset: ignore private nodes when matching OOM victims Gregory Price via B4 Relay
2026-11-20 18:24   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 34/60] mm/mempolicy: don't rebind policies onto private nodes Gregory Price via B4 Relay
2026-11-20 18:35   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 35/60] mm/mempolicy: keep private nodes out of migrate_pages() destinations Gregory Price via B4 Relay
2026-11-20 18:25   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 36/60] mm: refuse explicit reclaim and compaction of private nodes Gregory Price via B4 Relay
2026-11-20 18:24   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 37/60] mm/page_alloc: keep private memory out of reclaim reserves Gregory Price via B4 Relay
2026-11-20 18:31   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 38/60] mm/page_alloc: refuse contiguous allocation from private memory Gregory Price via B4 Relay
2026-11-20 18:24   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 39/60] mm/page_alloc: add a private-node zonelist Gregory Price via B4 Relay
2026-11-20 18:31   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 40/60] mm/page_alloc: let MM callers select the local private zonelist Gregory Price via B4 Relay
2026-11-20 18:46   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 41/60] mm/page_alloc: relax a private request whose node becomes ineligible Gregory Price via B4 Relay
2026-11-20 18:32   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 42/60] mm: fail private placement on exhaustion instead of OOM killing Gregory Price via B4 Relay
2026-11-20 18:40   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 43/60] mm/page_alloc: fall back when a bind policy's nodes become private Gregory Price via B4 Relay
2026-11-20 18:34   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 44/60] mm: add N_MEMORY_COMPACTION node state Gregory Price via B4 Relay
2026-11-20 18:24   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 45/60] mm/compaction: restrict compaction to N_MEMORY_COMPACTION Gregory Price via B4 Relay
2026-11-20 18:30   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 46/60] mm: add N_MEMORY_RECLAIM node state Gregory Price via B4 Relay
2026-11-20 18:25   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 47/60] mm/vmscan: restrict reclaim to N_MEMORY_RECLAIM instead of N_MEMORY_COMMON Gregory Price via B4 Relay
2026-11-20 18:38   ` sashiko-bot
2026-11-20 18:41     ` Gregory Price (Meta)
2026-11-20 18:09 ` [PATCH 48/60] mm/madvise: restrict reclaim advice to reclaim-capable memory Gregory Price via B4 Relay
2026-11-20 18:29   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 49/60] mm/vmscan: reclaim private nodes for memcg Gregory Price via B4 Relay
2026-11-20 18:31   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 50/60] mm: add N_MEMORY_MEMPOLICY node state Gregory Price via B4 Relay
2026-11-20 18:27   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 51/60] mm: restrict explicit migration to N_MEMORY_MEMPOLICY Gregory Price via B4 Relay
2026-11-20 18:38   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 52/60] mm: count memory with userspace placement as general purpose Gregory Price via B4 Relay
2026-11-20 18:33   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 53/60] mm/mempolicy: add MPOL_F_PRIVATE_NODES for private node bindings Gregory Price via B4 Relay
2026-11-20 18:44   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 54/60] mm/migrate: route explicit NUMA migration to private nodes Gregory Price via B4 Relay
2026-11-20 18:42   ` sashiko-bot
2026-11-20 18:56     ` Gregory Price (Meta)
2026-11-20 18:09 ` [PATCH 55/60] mm/memory_hotplug: online and offline private memory nodes Gregory Price via B4 Relay
2026-11-20 18:34   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 56/60] dax/kmem: add a private memory mode Gregory Price via B4 Relay
2026-11-20 18:31   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 57/60] selftests/dax: test private kmem devices and services Gregory Price via B4 Relay
2026-11-20 18:42   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 58/60] selftests/dax: test memory policy on private nodes Gregory Price via B4 Relay
2026-11-20 18:35   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 59/60] selftests/dax: test cpuset interaction with " Gregory Price via B4 Relay
2026-11-20 18:35   ` sashiko-bot
2026-11-20 18:09 ` [PATCH 60/60] Documentation/mm: describe private memory nodes Gregory Price via B4 Relay
2026-11-20 18:31   ` sashiko-bot
2026-11-20 18:59 ` [PATCH 00/60] Private NUMA nodes Gregory Price (Meta)

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).