[PATCH v3 0/7] mm/page_owner: add bpf_iter and kfuncs

[PATCH v3 0/7] mm/page_owner: add bpf_iter and kfuncs

[Zhen Ni]

Hi all,

Introduction
============

page_owner records who allocated every page and is a core memory-debug
facility, but it currently reaches userspace as the debugfs flat dump:
slicing allocations by pid, comm, NUMA node, cgroup or stack, etc.,
means streaming and parsing the entire file in userspace, which is
expensive on large hosts and hard to query live.

Exposing the same walk to eBPF lets a program filter and aggregate in
the kernel and surface only the records it cares about. This series
adds that access as a seq_file bpf_iter target and an open-coded
iterator.

Implementation
==============

The page walk is factored into page_owner_next_eligible(), shared by
the read path and the iterators so eligibility stays identical. Two
kfuncs expose information that a BPF program cannot decode from the
snapshot alone: per-page memcg info including the full cgroup path,
and the stack trace behind a stack_depot handle. Each record is
snapshotted under the page_ext RCU lock, so consumers read a
consistent copy.

Testing
=======

The selftest exercises the seq_file and open-coded iterators, and
validates the stack_snprint and get_memcg_info kfuncs.

Changes in v3
=============
- Add a mem-hotplug read lock across the page scan.
- Drop the live struct page from the scan cursor and the bpf_iter context.
- Check the page_owner_inited static key in the iterator.
- Without scan->page, start at min_low_pfn - 1 and let next() pre-increment.
- Gate the kfuncs with KF_PERFMON.
- Enable BPF_ITER_RESCHED for the bpf_iter target.
- Check ENODATA/EOPNOTSUPP from <errno.h>.

Changes in v2
=============
- Route the zero-handle case through ext_put_continue.
- Drop the scan->pfn++.
- Fill page/po/pfn only when v is non-NULL.
- Implement page_owner_seq_stop().
- bpf_page_owner_stack_snprint() now takes the scan cursor.
- Buffer args renamed to buf__uninit/buf__sz.
- Removes the v1 get_nid() kfunc. The node is now stored in scan->nid.
- bpf_page_owner_get_memcg_info() now resolves the page with
  pfn_to_online_page() for an offline or removed pfn.
- Buffer/info args renamed to info__uninit/buf__uninit/buf__sz.
- Drop the get_nid kfunc coverage.
- Enable CONFIG_PAGE_OWNER in the bpf selftests config.

v2: https://lore.kernel.org/linux-mm/20261013122125.843451-1-zhen.ni@easystack.cn/
v1: https://lore.kernel.org/linux-mm/20261009112308.240769-1-zhen.ni@easystack.cn/


Zhen Ni (7):
  mm/page_owner: extract page_owner_next_eligible() from read loop
  mm/page_owner: protect page access with mem-hotplug read lock
  mm/page_owner: add bpf_iter target "page_owner"
  mm/page_owner: add open-coded page_owner iterator
  mm/page_owner: record the NUMA node in the scan cursor
  mm/page_owner: add bpf_page_owner_get_memcg_info() kfunc
  selftests/bpf: add tests for the page_owner bpf_iter target

 kernel/bpf/helpers.c                          |  10 +
 mm/Makefile                                   |   3 +
 mm/page_owner.c                               | 252 ++++++++++++------
 mm/page_owner.h                               |  60 +++++
 mm/page_owner_iter.c                          | 232 ++++++++++++++++
 tools/testing/selftests/bpf/config            |   2 +
 .../bpf/prog_tests/page_owner_iter.c          | 193 ++++++++++++++
 .../selftests/bpf/progs/bpf_iter_page_owner.c | 157 +++++++++++
 8 files changed, 826 insertions(+), 83 deletions(-)
 create mode 100644 mm/page_owner.h
 create mode 100644 mm/page_owner_iter.c
 create mode 100644 tools/testing/selftests/bpf/prog_tests/page_owner_iter.c
 create mode 100644 tools/testing/selftests/bpf/progs/bpf_iter_page_owner.c

-- 
2.20.1

[PATCH v3 1/7] mm/page_owner: extract page_owner_next_eligible() from read loop

[Zhen Ni]

Move the page scanning logic out of read_page_owner() into a dedicated
helper, in preparation for using it in the page_owner bpf_iter target.

The helper advances a struct page_owner_scan cursor to the next page
eligible for page_owner output and returns true on a hit. On a hit it
takes a snapshot of the page_owner record inside the page_ext RCU
window.

No change to the output is intended. handle is read from the snapshot,
taken before the non-zero check, so the validated and reported handle
are the same value.

Signed-off-by: Zhen Ni <zhen.ni@easystack.cn>
---
Changes in v3:
- Drop the page field in struct page_owner_scan.

Changes in v2:
- Route the zero-handle case through ext_put_continue.

v1: https://lore.kernel.org/linux-mm/20261009112308.240769-2-zhen.ni@easystack.cn/
v2: https://lore.kernel.org/linux-mm/20261013122125.843451-2-zhen.ni@easystack.cn/
---
 mm/page_owner.c | 101 ++++++++++++++++++++++++++++++------------------
 1 file changed, 64 insertions(+), 37 deletions(-)

diff --git a/mm/page_owner.c b/mm/page_owner.c
index fbbda7ba914b..92a92cdf74ec 100644
--- a/mm/page_owner.c
+++ b/mm/page_owner.c
@@ -36,6 +36,15 @@ struct page_owner {
 	pid_t free_tgid;
 };
 
+/*
+ * Cursor and per-page result of page_owner_next_eligible().
+ */
+struct page_owner_scan {
+	/* resume point */
+	unsigned long pfn;
+	struct page_owner po_snap;
+};
+
 struct stack {
 	struct stack_record *stack_record;
 	struct stack *next;
@@ -725,37 +734,24 @@ void __dump_page_owner(const struct page *page)
 	page_ext_put(page_ext);
 }
 
-static ssize_t
-read_page_owner(struct file *file, char __user *buf, size_t count, loff_t *ppos)
+/*
+ * Advance @scan to the next page eligible for page_owner output.
+ * On a hit, fill scan->pfn/po_snap and return true; the caller
+ * must advance scan->pfn past the hit before calling again to resume
+ * the scan.
+ */
+static bool page_owner_next_eligible(struct page_owner_scan *scan)
 {
-	unsigned long pfn;
-	struct page *page;
-	struct page_ext *page_ext;
-	struct page_owner *page_owner;
-	depot_stack_handle_t handle;
-	struct page_owner_filter_state *state = file->private_data;
+	unsigned long pfn = scan->pfn;
 
-	if (!static_branch_unlikely(&page_owner_inited))
-		return -EINVAL;
-
-	page = NULL;
-	if (*ppos == 0)
-		pfn = min_low_pfn;
-	else
-		pfn = *ppos;
 	/* Find a valid PFN or the start of a MAX_ORDER_NR_PAGES area */
 	while (!pfn_valid(pfn) && (pfn & (MAX_ORDER_NR_PAGES - 1)) != 0)
 		pfn++;
 
-	/* Find an allocated page */
 	for (; pfn < max_pfn; pfn++) {
-		/*
-		 * This temporary page_owner is required so
-		 * that we can avoid the context switches while holding
-		 * the rcu lock and copying the page owner information to
-		 * user through copy_to_user() or GFP_KERNEL allocations.
-		 */
-		struct page_owner page_owner_tmp;
+		struct page_owner *page_owner;
+		struct page_ext *page_ext;
+		struct page *page;
 
 		/*
 		 * If the new page is in a new MAX_ORDER_NR_PAGES area,
@@ -798,13 +794,45 @@ read_page_owner(struct file *file, char __user *buf, size_t count, loff_t *ppos)
 			goto ext_put_continue;
 
 		/*
-		 * Access to page_ext->handle isn't synchronous so we should
-		 * be careful to access it.
+		 * Take a snapshot of the page_owner record inside the RCU
+		 * window, then validate it, so that print_page_owner()
+		 * reads the checked handle from the same snapshot.
 		 */
-		handle = READ_ONCE(page_owner->handle);
-		if (!handle)
+		scan->pfn = pfn;
+		scan->po_snap = *page_owner;
+
+		if (!scan->po_snap.handle)
 			goto ext_put_continue;
 
+		page_ext_put(page_ext);
+		return true;
+
+ext_put_continue:
+		page_ext_put(page_ext);
+		cond_resched();
+	}
+
+	return false;
+}
+
+static ssize_t
+read_page_owner(struct file *file, char __user *buf, size_t count, loff_t *ppos)
+{
+	struct page_owner_scan scan;
+	struct page_owner_filter_state *state = file->private_data;
+
+	if (!static_branch_unlikely(&page_owner_inited))
+		return -EINVAL;
+
+	if (*ppos == 0)
+		scan.pfn = min_low_pfn;
+	else
+		scan.pfn = *ppos;
+
+	/* Find an allocated page */
+	while (page_owner_next_eligible(&scan)) {
+		struct page *page = pfn_to_page(scan.pfn);
+
 		if (state->nid_filter_enabled) {
 			int nid;
 			memdesc_flags_t page_flags = READ_ONCE(page->flags);
@@ -814,21 +842,20 @@ read_page_owner(struct file *file, char __user *buf, size_t count, loff_t *ppos)
 			 * VM_BUG_ON when accessing poisoned pages.
 			 */
 			if (page_flags.f == PAGE_POISON_PATTERN)
-				goto ext_put_continue;
+				goto skip_continue;
 			nid = memdesc_nid(&page_flags);
 			if (!node_isset(nid, state->nid_filter))
-				goto ext_put_continue;
+				goto skip_continue;
 		}
 
 		/* Record the next PFN to read in the file offset */
-		*ppos = pfn + 1;
+		*ppos = scan.pfn + 1;
 
-		page_owner_tmp = *page_owner;
-		page_ext_put(page_ext);
-		return print_page_owner(buf, count, pfn, page,
-				&page_owner_tmp, handle, state);
-ext_put_continue:
-		page_ext_put(page_ext);
+		return print_page_owner(buf, count, scan.pfn, page,
+				&scan.po_snap, scan.po_snap.handle, state);
+
+skip_continue:
+		scan.pfn++;
 		cond_resched();
 	}
 
-- 
2.20.1

[PATCH v3 2/7] mm/page_owner: protect page access with mem-hotplug read lock

[Zhen Ni]

The page_owner scan walks PFNs locklessly and dereferences the resolved
struct page in the scan itself and in print_page_owner(). With
CONFIG_MEMORY_HOTREMOVE, a concurrent remove_memory() can tear down the
section's vmemmap, so a dereference after a stale validity check may
fault.

Take a mem-hotplug read lock across each page's check-then-deref. In the
scan, gate every PFN with pfn_to_online_page() under the reader and drop
it before cond_resched(), so hotplug can still interleave between pages;
in print_page_owner(), re-resolve the page under a reader covering just
the print of one record.

The reader is cheap: get_online_mems()/put_online_mems() are just
per-CPU counter increments/decrements when no hotplug is active, so
they contend only with an actual add/remove. Taking the lock per-page
bounds the writer's wait to a single page's check-and-deref, keeping
hot-remove stalls brief while all accesses stay protected.

This makes the traversal crash-free. A recycled-but-present page can
still yield stale data -- the lockless scan is best-effort and
unchanged by this patch.

Signed-off-by: Zhen Ni <zhen.ni@easystack.cn>
---
Changes in v3:
- New patch. Add the mem-hotplug protect.
---
 mm/page_owner.c | 35 ++++++++++++++++++++++++++++++-----
 1 file changed, 30 insertions(+), 5 deletions(-)

diff --git a/mm/page_owner.c b/mm/page_owner.c
index 92a92cdf74ec..8e72a85c663c 100644
--- a/mm/page_owner.c
+++ b/mm/page_owner.c
@@ -762,13 +762,22 @@ static bool page_owner_next_eligible(struct page_owner_scan *scan)
 			continue;
 		}
 
-		page = pfn_to_page(pfn);
+		/*
+		 * Hold a mem-hotplug reader across the page derefs so the
+		 * vmemmap isn't torn down mid-scan; released before
+		 * cond_resched().
+		 */
+		get_online_mems();
+		page = pfn_to_online_page(pfn);
+		if (!page)
+			goto put_online_continue;
+
 		if (skip_buddy_pages(&pfn, page))
-			continue;
+			goto put_online_continue;
 
 		page_ext = page_ext_get(page);
 		if (unlikely(!page_ext))
-			continue;
+			goto put_online_continue;
 
 		/*
 		 * Some pages could be missed by concurrent allocation or free,
@@ -805,10 +814,13 @@ static bool page_owner_next_eligible(struct page_owner_scan *scan)
 			goto ext_put_continue;
 
 		page_ext_put(page_ext);
+		put_online_mems();
 		return true;
 
 ext_put_continue:
 		page_ext_put(page_ext);
+put_online_continue:
+		put_online_mems();
 		cond_resched();
 	}
 
@@ -831,7 +843,17 @@ read_page_owner(struct file *file, char __user *buf, size_t count, loff_t *ppos)
 
 	/* Find an allocated page */
 	while (page_owner_next_eligible(&scan)) {
-		struct page *page = pfn_to_page(scan.pfn);
+		struct page *page;
+		ssize_t ret;
+
+		/*
+		 * The scan is lockless. Re-resolve the page under a mem-hotplug
+		 * read-side lock so that we never access a hot-removed section.
+		 */
+		get_online_mems();
+		page = pfn_to_online_page(scan.pfn);
+		if (!page)
+			goto skip_continue;
 
 		if (state->nid_filter_enabled) {
 			int nid;
@@ -851,10 +873,13 @@ read_page_owner(struct file *file, char __user *buf, size_t count, loff_t *ppos)
 		/* Record the next PFN to read in the file offset */
 		*ppos = scan.pfn + 1;
 
-		return print_page_owner(buf, count, scan.pfn, page,
+		ret = print_page_owner(buf, count, scan.pfn, page,
 				&scan.po_snap, scan.po_snap.handle, state);
+		put_online_mems();
+		return ret;
 
 skip_continue:
+		put_online_mems();
 		scan.pfn++;
 		cond_resched();
 	}
-- 
2.20.1

[PATCH v3 3/7] mm/page_owner: add bpf_iter target "page_owner"

[Zhen Ni]

Register a bpf_iter target "page_owner" that walks the pages eligible
for page_owner output, with the same eligibility rules as the
/sys/kernel/debug/page_owner read path.

For each eligible page the attached BPF program receives the pfn and the
page_owner record, and can freely filter and format output.

The po_snap record in the scan cursor is taken inside the page_ext
RCU window and handed to the BPF program as a copy, so no RCU window
is needed at consumption time. Torn reads remain possible if the
record is concurrently updated while it is copied, but this is the
same best-effort behavior as the read path has always had.

The target is registered from pageowner_init() when both
CONFIG_PAGE_OWNER and CONFIG_BPF_SYSCALL are enabled; a registration
failure only disables the bpf_iter target.

Userspace tools such as bpftrace can attach to this target like any
other bpf_iter. For example, to count pages owned by init (pid 1):

    bpftrace -e 'iter:page_owner / ctx->po->pid == 1 /
                 { @ = sum(1 << ctx->po->order); }'

Signed-off-by: Zhen Ni <zhen.ni@easystack.cn>
---
Changes in v3:
- Drop the page pointer from the BPF context.
- Enable BPF_ITER_RESCHED.
- Fix the bpftrace example in the commit message.

Changes in v2:
- Drop the scan->pfn++.
- Fill page/po/pfn only when v is non-NULL.
- Implement page_owner_seq_stop().

v2: https://lore.kernel.org/linux-mm/20261013122125.843451-3-zhen.ni@easystack.cn/
v1: https://lore.kernel.org/linux-mm/20261009112308.240769-3-zhen.ni@easystack.cn/
---
 mm/Makefile          |   3 ++
 mm/page_owner.c      |  32 +++---------
 mm/page_owner.h      |  47 +++++++++++++++++
 mm/page_owner_iter.c | 118 +++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 175 insertions(+), 25 deletions(-)
 create mode 100644 mm/page_owner.h
 create mode 100644 mm/page_owner_iter.c

diff --git a/mm/Makefile b/mm/Makefile
index e7245cb88c66..d7307b5a964c 100644
--- a/mm/Makefile
+++ b/mm/Makefile
@@ -115,6 +115,9 @@ obj-$(CONFIG_DEBUG_KMEMLEAK) += kmemleak.o
 obj-$(CONFIG_DEBUG_RODATA_TEST) += rodata_test.o
 obj-$(CONFIG_DEBUG_VM_PGTABLE) += debug_vm_pgtable.o
 obj-$(CONFIG_PAGE_OWNER) += page_owner.o
+ifdef CONFIG_BPF_SYSCALL
+obj-$(CONFIG_PAGE_OWNER) += page_owner_iter.o
+endif
 obj-$(CONFIG_MEMORY_ISOLATION) += page_isolation.o
 obj-$(CONFIG_ZSMALLOC)	+= zsmalloc.o
 obj-$(CONFIG_GENERIC_EARLY_IOREMAP) += early_ioremap.o
diff --git a/mm/page_owner.c b/mm/page_owner.c
index 8e72a85c663c..ece3b69a2579 100644
--- a/mm/page_owner.c
+++ b/mm/page_owner.c
@@ -14,6 +14,7 @@
 #include <linux/sched/clock.h>
 
 #include "page_alloc.h"
+#include "page_owner.h"
 
 /*
  * TODO: teach PAGE_OWNER_STACK_DEPTH (__dump_page_owner and save_stack)
@@ -21,30 +22,6 @@
  */
 #define PAGE_OWNER_STACK_DEPTH (16)
 
-struct page_owner {
-	unsigned short order;
-	short last_migrate_reason;
-	gfp_t gfp_mask;
-	depot_stack_handle_t handle;
-	depot_stack_handle_t free_handle;
-	u64 ts_nsec;
-	u64 free_ts_nsec;
-	char comm[TASK_COMM_LEN];
-	pid_t pid;
-	pid_t tgid;
-	pid_t free_pid;
-	pid_t free_tgid;
-};
-
-/*
- * Cursor and per-page result of page_owner_next_eligible().
- */
-struct page_owner_scan {
-	/* resume point */
-	unsigned long pfn;
-	struct page_owner po_snap;
-};
-
 struct stack {
 	struct stack_record *stack_record;
 	struct stack *next;
@@ -740,7 +717,7 @@ void __dump_page_owner(const struct page *page)
  * must advance scan->pfn past the hit before calling again to resume
  * the scan.
  */
-static bool page_owner_next_eligible(struct page_owner_scan *scan)
+bool page_owner_next_eligible(struct page_owner_scan *scan)
 {
 	unsigned long pfn = scan->pfn;
 
@@ -1221,6 +1198,11 @@ static int __init pageowner_init(void)
 			    &stack_fops);
 	debugfs_create_file("count_threshold", 0600, dir, NULL,
 			    &threshold_fops);
+
+#ifdef CONFIG_BPF_SYSCALL
+	if (page_owner_iter_register())
+		pr_warn("page_owner: failed to register bpf_iter target\n");
+#endif
 	return 0;
 }
 late_initcall(pageowner_init)
diff --git a/mm/page_owner.h b/mm/page_owner.h
new file mode 100644
index 000000000000..0aa7695bec82
--- /dev/null
+++ b/mm/page_owner.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * mm/ internal page_owner and page_owner_iter declarations
+ */
+
+#ifndef __MM_PAGE_OWNER_H
+#define __MM_PAGE_OWNER_H
+
+#include <linux/page_ext.h>
+#include <linux/sched.h>
+#include <linux/stackdepot.h>
+
+/*
+ * mm/page_owner.c
+ */
+struct page_owner {
+	unsigned short order;
+	short last_migrate_reason;
+	gfp_t gfp_mask;
+	depot_stack_handle_t handle;
+	depot_stack_handle_t free_handle;
+	u64 ts_nsec;
+	u64 free_ts_nsec;
+	char comm[TASK_COMM_LEN];
+	pid_t pid;
+	pid_t tgid;
+	pid_t free_pid;
+	pid_t free_tgid;
+};
+
+/*
+ * Cursor and per-page result of page_owner_next_eligible().
+ */
+struct page_owner_scan {
+	/* resume point */
+	unsigned long pfn;
+	struct page_owner po_snap;
+};
+
+bool page_owner_next_eligible(struct page_owner_scan *scan);
+
+/*
+ * mm/page_owner_iter.c
+ */
+int page_owner_iter_register(void);
+
+#endif /* __MM_PAGE_OWNER_H */
diff --git a/mm/page_owner_iter.c b/mm/page_owner_iter.c
new file mode 100644
index 000000000000..529b2c4a589a
--- /dev/null
+++ b/mm/page_owner_iter.c
@@ -0,0 +1,118 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/bpf.h>
+#include <linux/btf_ids.h>
+#include <linux/memblock.h>
+#include <linux/mm.h>
+#include <linux/seq_file.h>
+
+#include "page_owner.h"
+
+struct bpf_iter__page_owner {
+	__bpf_md_ptr(struct bpf_iter_meta *, meta);
+	__bpf_md_ptr(struct page_owner *, po);
+	u64 pfn;
+};
+
+DEFINE_BPF_ITER_FUNC(page_owner, struct bpf_iter_meta *meta,
+		     struct page_owner *po, u64 pfn)
+
+static void *page_owner_seq_start(struct seq_file *seq, loff_t *posp)
+{
+	struct page_owner_scan *scan = seq->private;
+
+	if (*posp == 0)
+		scan->pfn = min_low_pfn;
+
+	if (!page_owner_next_eligible(scan))
+		return NULL;
+	return &scan->po_snap;
+}
+
+static void *page_owner_seq_next(struct seq_file *seq, void *v, loff_t *posp)
+{
+	struct page_owner_scan *scan = seq->private;
+
+	++*posp;
+	scan->pfn++;
+	if (!page_owner_next_eligible(scan))
+		return NULL;
+
+	return &scan->po_snap;
+}
+
+static int page_owner_prog_seq_show(struct bpf_prog *prog,
+				    struct bpf_iter_meta *meta, void *v)
+{
+	struct page_owner_scan *scan = meta->seq->private;
+	struct bpf_iter__page_owner ctx = {};
+
+	ctx.meta = meta;
+	if (v) {
+		/* @po: hit-time snapshot, no RCU window needed. */
+		ctx.po = &scan->po_snap;
+		ctx.pfn = scan->pfn;
+	}
+
+	return bpf_iter_run_prog(prog, &ctx);
+}
+
+static int page_owner_seq_show(struct seq_file *seq, void *v)
+{
+	struct bpf_iter_meta meta;
+	struct bpf_prog *prog;
+
+	meta.seq = seq;
+	prog = bpf_iter_get_info(&meta, false);
+	if (!prog)
+		return 0;
+
+	return page_owner_prog_seq_show(prog, &meta, v);
+}
+
+static void page_owner_seq_stop(struct seq_file *seq, void *v)
+{
+	if (!v) {
+		struct bpf_iter_meta meta = { .seq = seq };
+		struct bpf_prog *prog;
+
+		prog = bpf_iter_get_info(&meta, true);
+		if (prog)
+			page_owner_prog_seq_show(prog, &meta, NULL);
+	}
+}
+
+static const struct seq_operations page_owner_iter_seq_ops = {
+	.start	= page_owner_seq_start,
+	.next	= page_owner_seq_next,
+	.stop	= page_owner_seq_stop,
+	.show	= page_owner_seq_show,
+};
+
+static const struct bpf_iter_seq_info page_owner_iter_seq_info = {
+	.seq_ops		= &page_owner_iter_seq_ops,
+	.init_seq_private	= NULL,
+	.fini_seq_private	= NULL,
+	.seq_priv_size		= sizeof(struct page_owner_scan),
+};
+
+static struct bpf_iter_reg page_owner_iter_reg_info = {
+	.target			= "page_owner",
+	.ctx_arg_info_size	= 1,
+	.ctx_arg_info		= {
+		{ offsetof(struct bpf_iter__page_owner, po),
+		  PTR_TO_BTF_ID_OR_NULL },
+	},
+	.seq_info		= &page_owner_iter_seq_info,
+	.feature		= BPF_ITER_RESCHED,
+};
+
+BTF_ID_LIST(page_owner_btf_ids)
+BTF_ID(struct, page_owner)
+
+int page_owner_iter_register(void)
+{
+	page_owner_iter_reg_info.ctx_arg_info[0].btf_id =
+		page_owner_btf_ids[0];
+
+	return bpf_iter_reg_target(&page_owner_iter_reg_info);
+}
-- 
2.20.1

[PATCH v3 4/7] mm/page_owner: add open-coded page_owner iterator

[Zhen Ni]

Add an open-coded bpf_iter over the pages eligible for page_owner
output, reusing page_owner_next_eligible().

A sleepable BPF program drives it with
bpf_iter_page_owner_new()/next()/destroy(); each next() returns the
page_owner_scan. po_snap is copied inside the page_ext RCU window, so
the program needs no RCU section when it reads it.

Also add bpf_page_owner_stack_snprint(), which renders the stack trace
of the current page into a caller-provided buffer.

Signed-off-by: Zhen Ni <zhen.ni@easystack.cn>
---
Changes in v3:
- Gate the kfuncs with KF_PERFMON.
- Check the page_owner_inited static key in the iterator.
- Without scan->page, start at min_low_pfn - 1 and let next() pre-increment.
- bpf_page_owner_stack_snprint() rejects a NULL or zero-size buffer.

Changes in v2:
- bpf_page_owner_stack_snprint() now takes the scan cursor.
- Buffer args renamed to buf__uninit/buf__sz.

v2: https://lore.kernel.org/linux-mm/20261013122125.843451-4-zhen.ni@easystack.cn/
v1: https://lore.kernel.org/linux-mm/20261009112308.240769-4-zhen.ni@easystack.cn/
---
 kernel/bpf/helpers.c |  9 +++++++
 mm/page_owner.h      |  3 +++
 mm/page_owner_iter.c | 64 ++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 76 insertions(+)

diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c
index 712dca5a2c5b..0ba836f77957 100644
--- a/kernel/bpf/helpers.c
+++ b/kernel/bpf/helpers.c
@@ -4946,6 +4946,15 @@ BTF_ID_FLAGS(func, bpf_iter_dmabuf_new, KF_ITER_NEW | KF_SLEEPABLE)
 BTF_ID_FLAGS(func, bpf_iter_dmabuf_next, KF_ITER_NEXT | KF_RET_NULL | KF_SLEEPABLE)
 BTF_ID_FLAGS(func, bpf_iter_dmabuf_destroy, KF_ITER_DESTROY | KF_SLEEPABLE)
 #endif
+#ifdef CONFIG_PAGE_OWNER
+BTF_ID_FLAGS(func, bpf_iter_page_owner_new,
+	     KF_ITER_NEW | KF_SLEEPABLE | KF_PERFMON)
+BTF_ID_FLAGS(func, bpf_iter_page_owner_next,
+	     KF_ITER_NEXT | KF_RET_NULL | KF_SLEEPABLE | KF_PERFMON)
+BTF_ID_FLAGS(func, bpf_iter_page_owner_destroy,
+	     KF_ITER_DESTROY | KF_SLEEPABLE | KF_PERFMON)
+BTF_ID_FLAGS(func, bpf_page_owner_stack_snprint, KF_SLEEPABLE | KF_PERFMON)
+#endif
 BTF_ID_FLAGS(func, __bpf_trap)
 BTF_ID_FLAGS(func, bpf_strcmp, KF_PERFMON);
 BTF_ID_FLAGS(func, bpf_strcasecmp, KF_PERFMON);
diff --git a/mm/page_owner.h b/mm/page_owner.h
index 0aa7695bec82..4ab513f17fc9 100644
--- a/mm/page_owner.h
+++ b/mm/page_owner.h
@@ -6,6 +6,7 @@
 #ifndef __MM_PAGE_OWNER_H
 #define __MM_PAGE_OWNER_H
 
+#include <linux/jump_label.h>
 #include <linux/page_ext.h>
 #include <linux/sched.h>
 #include <linux/stackdepot.h>
@@ -13,6 +14,8 @@
 /*
  * mm/page_owner.c
  */
+DECLARE_STATIC_KEY_FALSE(page_owner_inited);
+
 struct page_owner {
 	unsigned short order;
 	short last_migrate_reason;
diff --git a/mm/page_owner_iter.c b/mm/page_owner_iter.c
index 529b2c4a589a..4b9741a2de53 100644
--- a/mm/page_owner_iter.c
+++ b/mm/page_owner_iter.c
@@ -4,6 +4,7 @@
 #include <linux/memblock.h>
 #include <linux/mm.h>
 #include <linux/seq_file.h>
+#include <linux/stackdepot.h>
 
 #include "page_owner.h"
 
@@ -116,3 +117,66 @@ int page_owner_iter_register(void)
 
 	return bpf_iter_reg_target(&page_owner_iter_reg_info);
 }
+
+/* opaque to BPF programs */
+struct bpf_iter_page_owner {
+	__u64 __opaque[16];
+} __attribute__((aligned(8)));
+
+struct bpf_iter_page_owner_kern {
+	struct page_owner_scan scan;
+} __attribute__((aligned(8)));
+
+__bpf_kfunc_start_defs();
+
+__bpf_kfunc int bpf_iter_page_owner_new(struct bpf_iter_page_owner *it)
+{
+	struct bpf_iter_page_owner_kern *kit = (void *)it;
+
+	BUILD_BUG_ON(sizeof(*kit) > sizeof(*it));
+	BUILD_BUG_ON(__alignof__(*kit) != __alignof__(*it));
+
+	kit->scan.pfn = min_low_pfn - 1;
+	return 0;
+}
+
+__bpf_kfunc struct page_owner_scan *
+bpf_iter_page_owner_next(struct bpf_iter_page_owner *it)
+{
+	struct bpf_iter_page_owner_kern *kit = (void *)it;
+
+	if (!static_branch_unlikely(&page_owner_inited))
+		return NULL;
+
+	cond_resched();
+
+	kit->scan.pfn++;
+	if (!page_owner_next_eligible(&kit->scan))
+		return NULL;
+
+	return &kit->scan;
+}
+
+__bpf_kfunc void bpf_iter_page_owner_destroy(struct bpf_iter_page_owner *it)
+{
+}
+
+__bpf_kfunc int bpf_page_owner_stack_snprint(struct page_owner_scan *scan,
+					     char *buf__uninit, u32 buf__sz)
+{
+	int len = 0;
+
+	if (!buf__uninit || !buf__sz)
+		return 0;
+
+	if (static_branch_unlikely(&page_owner_inited))
+		len = stack_depot_snprint(scan->po_snap.handle, buf__uninit,
+					  buf__sz, 0);
+
+	if (!len)
+		buf__uninit[0] = '\0';
+
+	return len;
+}
+
+__bpf_kfunc_end_defs();
-- 
2.20.1

[PATCH v3 5/7] mm/page_owner: record the NUMA node in the scan cursor

[Zhen Ni]

Compute the page's NUMA node in page_owner_next_eligible() while the
page is still valid, and store it in scan->nid.

The debugfs nid filter reads scan->nid instead of recomputing it, and a
BPF program iterating the cursor reads the node straight from scan->nid.

Resolve nid at scan time to avoid dereferencing a possibly-unmapped page.
Keep it in the transient scan cursor rather than struct page_owner, so
nid does not add per-page memory overhead.

Signed-off-by: Zhen Ni <zhen.ni@easystack.cn>
---
Changes in v3:
- Add @scan->nid to the comment describing what is filled.

Changes in v2:
- Removes the v1 get_nid() kfunc. The node is now stored in scan->nid.

v2: https://lore.kernel.org/linux-mm/20261013122125.843451-5-zhen.ni@easystack.cn/
v1: https://lore.kernel.org/linux-mm/20261009112308.240769-5-zhen.ni@easystack.cn/
---
 mm/page_owner.c      | 29 +++++++++++++++++------------
 mm/page_owner.h      |  1 +
 mm/page_owner_iter.c |  4 +++-
 3 files changed, 21 insertions(+), 13 deletions(-)

diff --git a/mm/page_owner.c b/mm/page_owner.c
index ece3b69a2579..8bc15bcdc24e 100644
--- a/mm/page_owner.c
+++ b/mm/page_owner.c
@@ -711,9 +711,22 @@ void __dump_page_owner(const struct page *page)
 	page_ext_put(page_ext);
 }
 
+/*
+ * NUMA node of @page, bypassing PF_POISONED_CHECK() in page_to_nid() to
+ * avoid VM_BUG_ON on poisoned pages; -1 for poisoned pages.
+ */
+static int page_owner_get_nid(struct page *page)
+{
+	memdesc_flags_t page_flags = READ_ONCE(page->flags);
+
+	if (page_flags.f == PAGE_POISON_PATTERN)
+		return -1;
+	return memdesc_nid(&page_flags);
+}
+
 /*
  * Advance @scan to the next page eligible for page_owner output.
- * On a hit, fill scan->pfn/po_snap and return true; the caller
+ * On a hit, fill scan->pfn/po_snap/nid and return true; the caller
  * must advance scan->pfn past the hit before calling again to resume
  * the scan.
  */
@@ -790,6 +803,7 @@ bool page_owner_next_eligible(struct page_owner_scan *scan)
 		if (!scan->po_snap.handle)
 			goto ext_put_continue;
 
+		scan->nid = page_owner_get_nid(page);
 		page_ext_put(page_ext);
 		put_online_mems();
 		return true;
@@ -833,17 +847,8 @@ read_page_owner(struct file *file, char __user *buf, size_t count, loff_t *ppos)
 			goto skip_continue;
 
 		if (state->nid_filter_enabled) {
-			int nid;
-			memdesc_flags_t page_flags = READ_ONCE(page->flags);
-
-			/*
-			 * Bypass PF_POISONED_CHECK() in page_to_nid() to avoid
-			 * VM_BUG_ON when accessing poisoned pages.
-			 */
-			if (page_flags.f == PAGE_POISON_PATTERN)
-				goto skip_continue;
-			nid = memdesc_nid(&page_flags);
-			if (!node_isset(nid, state->nid_filter))
+			if (scan.nid < 0 ||
+			    !node_isset(scan.nid, state->nid_filter))
 				goto skip_continue;
 		}
 
diff --git a/mm/page_owner.h b/mm/page_owner.h
index 4ab513f17fc9..934bc9f84d6f 100644
--- a/mm/page_owner.h
+++ b/mm/page_owner.h
@@ -38,6 +38,7 @@ struct page_owner_scan {
 	/* resume point */
 	unsigned long pfn;
 	struct page_owner po_snap;
+	int nid;
 };
 
 bool page_owner_next_eligible(struct page_owner_scan *scan);
diff --git a/mm/page_owner_iter.c b/mm/page_owner_iter.c
index 4b9741a2de53..b7e0ce1dfa80 100644
--- a/mm/page_owner_iter.c
+++ b/mm/page_owner_iter.c
@@ -12,10 +12,11 @@ struct bpf_iter__page_owner {
 	__bpf_md_ptr(struct bpf_iter_meta *, meta);
 	__bpf_md_ptr(struct page_owner *, po);
 	u64 pfn;
+	int nid;
 };
 
 DEFINE_BPF_ITER_FUNC(page_owner, struct bpf_iter_meta *meta,
-		     struct page_owner *po, u64 pfn)
+		     struct page_owner *po, u64 pfn, int nid)
 
 static void *page_owner_seq_start(struct seq_file *seq, loff_t *posp)
 {
@@ -52,6 +53,7 @@ static int page_owner_prog_seq_show(struct bpf_prog *prog,
 		/* @po: hit-time snapshot, no RCU window needed. */
 		ctx.po = &scan->po_snap;
 		ctx.pfn = scan->pfn;
+		ctx.nid = scan->nid;
 	}
 
 	return bpf_iter_run_prog(prog, &ctx);
-- 
2.20.1

[PATCH v3 6/7] mm/page_owner: add bpf_page_owner_get_memcg_info() kfunc

[Zhen Ni]

Expose per-page memcg information to BPF programs, so that a bpf_iter
over page_owner can aggregate and filter allocations by memcg.
Attributing an allocation to a specific cgroup needs its full cgroup
path, not just the memcg name.

Split memcg info collection out of print_page_owner_memcg() into
get_page_memcg_info(), which fills struct memcg_info and, when given a
caller-provided buffer, writes the full cgroup path via cgroup_path().
print_page_owner_memcg() now consumes the struct; this introduces no
functional change to the debugfs output.

The helper takes a single READ_ONCE snapshot within one RCU read-side
critical section, so the flags and cgroup path reflect a consistent
view of the page. cgroup_path() returns the path relative to the cgroup
hierarchy root, so memcg attribution works on both cgroup v1 and v2.

Because scan->pfn is selected by a lockless scan, the kfunc re-resolves
it with pfn_to_online_page() under get_online_mems() to avoid accessing
a hot-removed section. If the PFN is offline or removed, the kfunc
returns -ENODATA instead of dereferencing memcg_data.

Signed-off-by: Zhen Ni <zhen.ni@easystack.cn>
---
Changes in v3:
- Check the page_owner_inited static key.
- Resolve the page with pfn_to_online_page() under get_online_mems().

Changes in v2:
- bpf_page_owner_get_memcg_info() now resolves the page with
  pfn_to_online_page() for an offline or removed pfn.
- Buffer/info args renamed to info__uninit/buf__uninit/buf__sz.

v2: https://lore.kernel.org/linux-mm/20261013122125.843451-6-zhen.ni@easystack.cn/
v1: https://lore.kernel.org/linux-mm/20261009112308.240769-6-zhen.ni@easystack.cn/
---
 kernel/bpf/helpers.c |  1 +
 mm/page_owner.c      | 85 ++++++++++++++++++++++++++++++++++----------
 mm/page_owner.h      |  9 +++++
 mm/page_owner_iter.c | 48 +++++++++++++++++++++++++
 4 files changed, 124 insertions(+), 19 deletions(-)

diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c
index 0ba836f77957..bba9eeb076c3 100644
--- a/kernel/bpf/helpers.c
+++ b/kernel/bpf/helpers.c
@@ -4954,6 +4954,7 @@ BTF_ID_FLAGS(func, bpf_iter_page_owner_next,
 BTF_ID_FLAGS(func, bpf_iter_page_owner_destroy,
 	     KF_ITER_DESTROY | KF_SLEEPABLE | KF_PERFMON)
 BTF_ID_FLAGS(func, bpf_page_owner_stack_snprint, KF_SLEEPABLE | KF_PERFMON)
+BTF_ID_FLAGS(func, bpf_page_owner_get_memcg_info, KF_SLEEPABLE | KF_PERFMON)
 #endif
 BTF_ID_FLAGS(func, __bpf_trap)
 BTF_ID_FLAGS(func, bpf_strcmp, KF_PERFMON);
diff --git a/mm/page_owner.c b/mm/page_owner.c
index 8bc15bcdc24e..7470b5d002be 100644
--- a/mm/page_owner.c
+++ b/mm/page_owner.c
@@ -538,16 +538,22 @@ void pagetypeinfo_showmixedcount_print(struct seq_file *m,
 
 #ifdef CONFIG_MEMCG
 /*
- * Looking for memcg information and print it out
+ * Collect memcg information of @page into @info; when @buf and a
+ * nonzero @size are given, also copy the full cgroup path of the
+ * charged memcg into @buf.
+ *
+ * Return: 0 if @page is charged to an attributable memcg; -ENODATA
+ * otherwise.
  */
-static inline int print_page_owner_memcg(char *kbuf, size_t count, int ret,
-					 struct page *page)
+int get_page_memcg_info(struct page *page, struct memcg_info *info,
+			char *buf, int size)
 {
 	unsigned long memcg_data;
 	struct obj_cgroup *objcg;
 	struct mem_cgroup *memcg;
-	bool online;
-	char name[80];
+	int ret = -ENODATA;
+
+	*info = (struct memcg_info){};
 
 	rcu_read_lock();
 	memcg_data = READ_ONCE(page->memcg_data);
@@ -555,31 +561,68 @@ static inline int print_page_owner_memcg(char *kbuf, size_t count, int ret,
 		goto out_unlock;
 
 	if (memcg_data & MEMCG_DATA_OBJEXTS) {
-		ret += scnprintf(kbuf + ret, count - ret,
-				"Slab cache page\n");
+		info->is_slab = true;
 		goto out_unlock;
 	}
 
 	objcg = (void *)(memcg_data & ~OBJEXTS_FLAGS_MASK);
-	memcg = objcg ? obj_cgroup_memcg(objcg) : NULL;
+	if (!objcg)
+		goto out_unlock;
+
+	memcg = obj_cgroup_memcg(objcg);
 	if (!memcg)
 		goto out_unlock;
 
-	online = css_is_online(&memcg->css);
-	cgroup_name(memcg->css.cgroup, name, sizeof(name));
-	ret += scnprintf(kbuf + ret, count - ret,
-			"Charged %sto %smemcg %s\n",
-			(memcg_data & MEMCG_DATA_KMEM) ? "(via objcg) " : "",
-			online ? "" : "offline ",
-			name);
+	info->is_kmem = (memcg_data & MEMCG_DATA_KMEM) != 0;
+	info->is_online = css_is_online(&memcg->css);
+	cgroup_name(memcg->css.cgroup, info->name, sizeof(info->name));
+
+	if (buf && size)
+		cgroup_path(memcg->css.cgroup, buf, size);
+	ret = 0;
 out_unlock:
 	rcu_read_unlock();
 
+	if (ret < 0 && buf && size)
+		buf[0] = '\0';
+
+	return ret;
+}
+
+/*
+ * Print memcg information from memcg_info
+ */
+static inline int print_page_owner_memcg(char *kbuf, size_t count, int ret,
+					 const struct memcg_info *info)
+{
+	if (!info)
+		return ret;
+
+	if (info->is_slab)
+		ret += scnprintf(kbuf + ret, count - ret,
+				"Slab cache page\n");
+
+	if (info->name[0])
+		ret += scnprintf(kbuf + ret, count - ret,
+				"Charged %sto %smemcg %s\n",
+				info->is_kmem ? "(via objcg) " : "",
+				info->is_online ? "" : "offline ",
+				info->name);
+
 	return ret;
 }
 #else
+int get_page_memcg_info(struct page *page, struct memcg_info *info,
+			char *buf, int size)
+{
+	*info = (struct memcg_info){};
+	if (buf && size)
+		buf[0] = '\0';
+	return -EOPNOTSUPP;
+}
+
 static inline int print_page_owner_memcg(char *kbuf, size_t count, int ret,
-					 struct page *page)
+					 const struct memcg_info *info)
 {
 	return ret;
 }
@@ -589,7 +632,8 @@ static ssize_t
 print_page_owner(char __user *buf, size_t count, unsigned long pfn,
 		struct page *page, struct page_owner *page_owner,
 		depot_stack_handle_t handle,
-		struct page_owner_filter_state *state)
+		struct page_owner_filter_state *state,
+		const struct memcg_info *memcg_info)
 {
 	int ret, pageblock_mt, page_mt;
 	char *kbuf;
@@ -639,7 +683,7 @@ print_page_owner(char __user *buf, size_t count, unsigned long pfn,
 			migrate_reason_names[page_owner->last_migrate_reason]);
 	}
 
-	ret = print_page_owner_memcg(kbuf, count, ret, page);
+	ret = print_page_owner_memcg(kbuf, count, ret, memcg_info);
 
 	ret += snprintf(kbuf + ret, count - ret, "\n");
 	if (ret >= count)
@@ -822,6 +866,7 @@ static ssize_t
 read_page_owner(struct file *file, char __user *buf, size_t count, loff_t *ppos)
 {
 	struct page_owner_scan scan;
+	struct memcg_info memcg_info;
 	struct page_owner_filter_state *state = file->private_data;
 
 	if (!static_branch_unlikely(&page_owner_inited))
@@ -855,8 +900,10 @@ read_page_owner(struct file *file, char __user *buf, size_t count, loff_t *ppos)
 		/* Record the next PFN to read in the file offset */
 		*ppos = scan.pfn + 1;
 
+		get_page_memcg_info(page, &memcg_info, NULL, 0);
 		ret = print_page_owner(buf, count, scan.pfn, page,
-				&scan.po_snap, scan.po_snap.handle, state);
+				&scan.po_snap, scan.po_snap.handle, state,
+				&memcg_info);
 		put_online_mems();
 		return ret;
 
diff --git a/mm/page_owner.h b/mm/page_owner.h
index 934bc9f84d6f..65598415af73 100644
--- a/mm/page_owner.h
+++ b/mm/page_owner.h
@@ -41,7 +41,16 @@ struct page_owner_scan {
 	int nid;
 };
 
+struct memcg_info {
+	char name[80];
+	bool is_slab;
+	bool is_kmem;
+	bool is_online;
+};
+
 bool page_owner_next_eligible(struct page_owner_scan *scan);
+int get_page_memcg_info(struct page *page, struct memcg_info *info,
+			char *buf, int size);
 
 /*
  * mm/page_owner_iter.c
diff --git a/mm/page_owner_iter.c b/mm/page_owner_iter.c
index b7e0ce1dfa80..a649e1207e24 100644
--- a/mm/page_owner_iter.c
+++ b/mm/page_owner_iter.c
@@ -181,4 +181,52 @@ __bpf_kfunc int bpf_page_owner_stack_snprint(struct page_owner_scan *scan,
 	return len;
 }
 
+/**
+ * bpf_page_owner_get_memcg_info - Report the memcg charged for the current page
+ * @scan: page_owner scan handle.
+ * @info__uninit: output buffer for the memcg_info.
+ * @buf__uninit: optional output buffer for the full cgroup path.
+ * @buf__sz: size of @buf__uninit.
+ *
+ * Fills @info__uninit with the memcg charged for the page backing
+ * @scan->pfn.  If @buf__uninit and @buf__sz are provided, the full
+ * cgroup path is also copied out, so a BPF program can attribute a
+ * page_owner allocation to a specific cgroup.
+ *
+ * @scan->pfn is resolved to a struct page under get_online_mems(),
+ * so the page cannot be accessed after its memory section is offlined
+ * or removed.
+ *
+ * Returns:
+ * * 0 on success
+ * * -ENODATA if the page is not online or not charged to a memcg
+ * * -EOPNOTSUPP if built without CONFIG_MEMCG
+ */
+__bpf_kfunc int bpf_page_owner_get_memcg_info(struct page_owner_scan *scan,
+					      struct memcg_info *info__uninit,
+					      char *buf__uninit, u32 buf__sz)
+{
+	struct page *page;
+	int ret = -ENODATA;
+
+	if (!static_branch_unlikely(&page_owner_inited))
+		goto nodata;
+
+	get_online_mems();
+	page = pfn_to_online_page(scan->pfn);
+	if (!page) {
+		put_online_mems();
+		goto nodata;
+	}
+	ret = get_page_memcg_info(page, info__uninit, buf__uninit, buf__sz);
+	put_online_mems();
+	return ret;
+
+nodata:
+	*info__uninit = (struct memcg_info){};
+	if (buf__uninit && buf__sz)
+		buf__uninit[0] = '\0';
+	return ret;
+}
+
 __bpf_kfunc_end_defs();
-- 
2.20.1

[PATCH v3 7/7] selftests/bpf: add tests for the page_owner bpf_iter target

[Zhen Ni]

Add a sleepable BPF program and a userspace runner covering the seq_file
and open-coded page_owner iterators. The test checks:

  - seq_file walk, unfiltered and pid-filtered: records read back from
    the iterator fd match the lines the program printed.
  - open-coded iterator parity with the seq path.
  - the kfuncs' contracts: a non-empty stack trace, and memcg
    classification.

CONFIG_PAGE_OWNER is enabled in the bpf selftests config so the
program's page_owner BTF types are available. Skipped when debugfs
page_owner is unavailable.

Signed-off-by: Zhen Ni <zhen.ni@easystack.cn>
---
Changes in v3:
- Check the ENODATA/EOPNOTSUPP instead of local macros.

Changes in v2:
- Drop the get_nid kfunc coverage.
- Enable CONFIG_PAGE_OWNER in the bpf selftests config.

v2: https://lore.kernel.org/linux-mm/20261013122125.843451-7-zhen.ni@easystack.cn/
v1: https://lore.kernel.org/linux-mm/20261009112308.240769-7-zhen.ni@easystack.cn/
---
 tools/testing/selftests/bpf/config            |   2 +
 .../bpf/prog_tests/page_owner_iter.c          | 193 ++++++++++++++++++
 .../selftests/bpf/progs/bpf_iter_page_owner.c | 157 ++++++++++++++
 3 files changed, 352 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/page_owner_iter.c
 create mode 100644 tools/testing/selftests/bpf/progs/bpf_iter_page_owner.c

diff --git a/tools/testing/selftests/bpf/config b/tools/testing/selftests/bpf/config
index ea7044f30adc..731bc3d79881 100644
--- a/tools/testing/selftests/bpf/config
+++ b/tools/testing/selftests/bpf/config
@@ -136,3 +136,5 @@ CONFIG_SMC_HS_CTRL_BPF=y
 CONFIG_DIBS=y
 CONFIG_DIBS_LO=y
 CONFIG_PM_WAKELOCKS=y
+CONFIG_DEBUG_KERNEL=y
+CONFIG_PAGE_OWNER=y
diff --git a/tools/testing/selftests/bpf/prog_tests/page_owner_iter.c b/tools/testing/selftests/bpf/prog_tests/page_owner_iter.c
new file mode 100644
index 000000000000..3684ca308935
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/page_owner_iter.c
@@ -0,0 +1,193 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 */
+
+#include <test_progs.h>
+#include "bpf_iter_page_owner.skel.h"
+
+#define DEBUGFS_PAGE_OWNER	"/sys/kernel/debug/page_owner"
+
+static bool page_owner_available(void)
+{
+	return access(DEBUGFS_PAGE_OWNER, R_OK) == 0;
+}
+
+/* Count link_fd output lines; -1 on error. */
+static long iter_record_count(int link_fd)
+{
+	char *line = NULL;
+	size_t len = 0;
+	long count = 0;
+	int iter_fd;
+	FILE *f;
+
+	iter_fd = bpf_iter_create(link_fd);
+	if (!ASSERT_GE(iter_fd, 0, "bpf_iter_create"))
+		return -1;
+
+	f = fdopen(iter_fd, "r");
+	if (!ASSERT_OK_PTR(f, "fdopen")) {
+		close(iter_fd);
+		return -1;
+	}
+
+	for (;;) {
+		if (getline(&line, &len, f) > 0) {
+			count++;
+			continue;
+		}
+		if (!ferror(f))
+			break;		/* clean EOF */
+		/*
+		 * bpf_seq_read() returns -EAGAIN when a chunk scanned.
+		 * Not an error, just retry the read.
+		 */
+		if (errno == EAGAIN) {
+			clearerr(f);
+			continue;
+		}
+		ASSERT_OK(errno, "iter_read");
+		count = -1;
+		break;
+	}
+
+	free(line);
+	fclose(f);
+	return count;
+}
+
+/* Validate bpf_page_owner_get_memcg_info() results from the open-coded run. */
+static void check_memcg_skel(struct bpf_iter_page_owner *skel)
+{
+	fprintf(stderr,
+		"memcg: seen=%u charged=%u online=%u nodata=%u notsup=%u bad=%u first='%s' name='%s' flags=%u\n",
+		skel->bss->memcg_seen, skel->bss->memcg_charged,
+		skel->bss->memcg_online, skel->bss->memcg_nodata,
+		skel->bss->memcg_notsup, skel->bss->memcg_bad,
+		skel->bss->memcg_first_path, skel->bss->memcg_first_name,
+		skel->bss->memcg_first_flags);
+
+	if (!ASSERT_GT(skel->bss->memcg_seen, 0, "memcg_seen_gt_0"))
+		return;
+
+	/* all -EOPNOTSUPP: CONFIG_MEMCG is off */
+	if (skel->bss->memcg_notsup == skel->bss->memcg_seen) {
+		ASSERT_EQ(skel->bss->memcg_bad, 0, "memcg_stub_invariants");
+		return;
+	}
+
+	if (!ASSERT_EQ(skel->bss->memcg_bad, 0, "memcg_kfunc_invariants"))
+		return;
+
+	/* Every call is either charged or nodata */
+	if (!ASSERT_EQ(skel->bss->memcg_charged + skel->bss->memcg_nodata,
+		       skel->bss->memcg_seen, "memcg_ret_classified"))
+		return;
+
+	if (!skel->bss->memcg_first_saved)
+		return;
+
+	if (!ASSERT_EQ(skel->bss->memcg_first_path[0], '/',
+		       "memcg_first_path_abs"))
+		return;
+
+	if (skel->bss->memcg_first_name[0])
+		ASSERT_OK_PTR(strstr(skel->bss->memcg_first_path,
+				    skel->bss->memcg_first_name),
+			      "memcg_name_in_path");
+}
+
+void serial_test_page_owner_iter(void)
+{
+	struct bpf_iter_page_owner *skel = NULL;
+	LIBBPF_OPTS(bpf_test_run_opts, opts);
+	long iter_count, unfiltered_count, oc_diff, pid;
+	unsigned int printed_before;
+	int link_fd, prog_fd;
+
+	if (!page_owner_available()) {
+		printf("%s:SKIP:page_owner debugfs unavailable "
+		       "(CONFIG_PAGE_OWNER + page_owner=on)\n", __func__);
+		test__skip();
+		return;
+	}
+
+	skel = bpf_iter_page_owner__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "bpf_iter_page_owner__open_and_load"))
+		return;
+
+	/* --- seq file iterator: unfiltered run --- */
+	skel->bss->filter_en = 0;
+
+	if (!ASSERT_OK(bpf_iter_page_owner__attach(skel), "skel_attach"))
+		goto destroy;
+
+	link_fd = bpf_link__fd(skel->links.dump_page_owner);
+	iter_count = iter_record_count(link_fd);
+	if (iter_count < 0)
+		goto destroy;
+	unfiltered_count = iter_count;
+
+	fprintf(stderr, "iter_count=%ld printed_lines=%u total_seen=%u\n",
+		iter_count, skel->bss->printed_lines, skel->bss->total_seen);
+
+	ASSERT_GT(iter_count, 0, "unfiltered_lines_gt_0");
+
+	ASSERT_EQ((long)skel->bss->printed_lines, iter_count,
+		  "lines_match_printed");
+
+	/* unfiltered: every visited page is printed, so the two must match */
+	ASSERT_EQ(skel->bss->total_seen, skel->bss->printed_lines,
+		  "unfiltered_all_printed");
+
+	/* --- seq file iterator: pid-filtered run --- */
+	pid = getpid();
+	skel->bss->filter_en = 1;
+	skel->bss->filter_pid = (pid_t)pid;
+
+	/* printed_lines is cumulative across runs: compare the delta */
+	printed_before = skel->bss->printed_lines;
+	iter_count = iter_record_count(link_fd);
+	if (iter_count < 0)
+		goto destroy;
+
+	fprintf(stderr,
+		"pid=%ld iter_count=%ld printed_delta=%u\n",
+		pid, iter_count, skel->bss->printed_lines - printed_before);
+
+	ASSERT_GT(iter_count, 0, "filtered_lines_gt_0");
+	ASSERT_EQ(skel->bss->printed_lines - printed_before, iter_count,
+		  "filtered_printed_match");
+
+	/* --- open-coded iterator parity --- */
+	prog_fd = bpf_program__fd(skel->progs.scan_open_coded);
+
+	ASSERT_OK(bpf_prog_test_run_opts(prog_fd, &opts), "prog_test_run");
+	fprintf(stderr,
+		"open_coded_seen=%u nid_seen=%u first_nid=%d "
+		"stack_bytes=%u stack=[\n%s]\n",
+		skel->bss->open_coded_seen, skel->bss->nid_seen,
+		skel->data->first_nid, skel->bss->stack_bytes,
+		skel->bss->stack_text);
+
+	/* same eligibility as the seq path: counts must be close */
+	ASSERT_GT(skel->bss->open_coded_seen, 0, "oc_seen_gt_0");
+	oc_diff = (long)skel->bss->open_coded_seen - unfiltered_count;
+	if (oc_diff < 0)
+		oc_diff = -oc_diff;
+	ASSERT_LE(oc_diff, unfiltered_count / 100 + 64,
+		  "oc_seen_close_to_iter");
+
+	/* the nid kfunc returned a valid node for every page */
+	ASSERT_EQ(skel->bss->nid_seen, skel->bss->open_coded_seen,
+		  "nid_valid_all");
+	ASSERT_GE(skel->data->first_nid, 0, "first_nid_valid");
+
+	ASSERT_GT(skel->bss->stack_bytes, 0, "stack_snprint_ok");
+	ASSERT_OK_PTR(strstr(skel->bss->stack_text, "+0x"), "stack_frames");
+	ASSERT_OK_PTR(strchr(skel->bss->stack_text, '\n'), "stack_lines");
+
+	check_memcg_skel(skel);
+
+destroy:
+	bpf_iter_page_owner__destroy(skel);
+}
diff --git a/tools/testing/selftests/bpf/progs/bpf_iter_page_owner.c b/tools/testing/selftests/bpf/progs/bpf_iter_page_owner.c
new file mode 100644
index 000000000000..1cd99956aff3
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/bpf_iter_page_owner.c
@@ -0,0 +1,157 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 */
+
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include <errno.h>
+
+char _license[] SEC("license") = "GPL";
+
+__u32 filter_en;
+__u32 filter_pid;
+__u32 total_seen;
+__u32 printed_lines;
+
+__u32 open_coded_seen;
+__u32 nid_seen;
+__s32 first_nid = -1;
+char stack_text[2048];
+__u32 stack_bytes;
+
+__u32 memcg_seen;
+__u32 memcg_charged;
+__u32 memcg_online;
+__u32 memcg_nodata;
+__u32 memcg_notsup;
+__u32 memcg_bad;
+__u32 memcg_first_saved;
+__u32 memcg_first_flags;	/* bit0 is_online, bit1 is_kmem */
+__u32 memcg_first_path_len;
+char memcg_pathbuf[4096];
+char memcg_first_path[4096];
+char memcg_first_name[80];
+
+#define MEMCG_F_ONLINE	(1 << 0)
+#define MEMCG_F_OBJCG	(1 << 1)
+
+extern int bpf_iter_page_owner_new(struct bpf_iter_page_owner *it) __ksym;
+extern struct page_owner_scan *
+bpf_iter_page_owner_next(struct bpf_iter_page_owner *it) __ksym;
+extern void bpf_iter_page_owner_destroy(struct bpf_iter_page_owner *it) __ksym;
+extern int bpf_page_owner_stack_snprint(struct page_owner_scan *scan,
+					char *buf__uninit, u32 buf__sz) __ksym;
+extern int bpf_page_owner_get_memcg_info(struct page_owner_scan *scan,
+					 struct memcg_info *info__uninit,
+					 char *buf__uninit, u32 buf__sz) __ksym;
+
+static __u32 copy_str(char *dst, const char *src, const __u32 max)
+{
+	__u32 i = 0;
+
+	while (i + 1 < max && src[i]) {
+		dst[i] = src[i];
+		i++;
+	}
+	dst[i] = '\0';
+	return i;
+}
+
+/*
+ * Exercise bpf_page_owner_get_memcg_info() until 32 pages charged to an
+ * ONLINE memcg have been seen, checking its per-return contract:
+ *  - 0:          buf holds an absolute cgroup path;
+ *  - -ENODATA:   buf was NUL-terminated;
+ *  - -EOPNOTSUPP: counted (CONFIG_MEMCG is off);
+ *
+ * The first online charged sample is saved for userspace to print and
+ * cross-check (name vs basename of path).
+ */
+static void check_memcg(struct page_owner_scan *scan)
+{
+	static struct memcg_info info;
+	int err;
+
+	err = bpf_page_owner_get_memcg_info(scan, &info, memcg_pathbuf,
+					    sizeof(memcg_pathbuf));
+	memcg_seen++;
+
+	if (err == 0) {
+		memcg_charged++;
+
+		if (memcg_pathbuf[0] != '/') {
+			memcg_bad++;
+			return;
+		}
+
+		if (!info.is_online)
+			return;
+		memcg_online++;
+
+		/* keep the first online sample for userspace */
+		if (!memcg_first_saved) {
+			memcg_first_path_len = copy_str(memcg_first_path,
+							memcg_pathbuf,
+							sizeof(memcg_first_path));
+			copy_str(memcg_first_name, info.name,
+				 sizeof(memcg_first_name));
+			memcg_first_flags = MEMCG_F_ONLINE |
+					    (info.is_kmem ? MEMCG_F_OBJCG : 0);
+			memcg_first_saved = 1;
+		}
+	} else if (err == -ENODATA) {
+		memcg_nodata++;
+		if (memcg_pathbuf[0] != '\0')
+			memcg_bad++;
+	} else if (err == -EOPNOTSUPP) {
+		memcg_notsup++;
+		if (memcg_pathbuf[0] != '\0')
+			memcg_bad++;
+	} else {
+		memcg_bad++;
+	}
+}
+
+SEC("syscall")
+int scan_open_coded(const void *ctx)
+{
+	struct bpf_iter_page_owner it;
+	struct page_owner_scan *scan;
+
+	bpf_iter_page_owner_new(&it);
+
+	while ((scan = bpf_iter_page_owner_next(&it))) {
+		open_coded_seen++;
+		if (scan->nid >= 0)
+			nid_seen++;
+		if (memcg_online < 32)
+			check_memcg(scan);
+		/* grab the stack of the 100th page for userspace to check */
+		if (open_coded_seen == 100) {
+			first_nid = scan->nid;
+			if (scan->po_snap.handle)
+				stack_bytes = bpf_page_owner_stack_snprint(
+					scan, stack_text, sizeof(stack_text));
+		}
+	}
+	bpf_iter_page_owner_destroy(&it);
+	return 0;
+}
+
+SEC("iter/page_owner")
+int dump_page_owner(struct bpf_iter__page_owner *ctx)
+{
+	struct page_owner *po = ctx->po;
+
+	if (!po)
+		return 0;
+
+	total_seen++;
+
+	if (filter_en && po->pid != (pid_t)filter_pid)
+		return 0;
+
+	BPF_SEQ_PRINTF(ctx->meta->seq, "pfn %llu pid %d comm %s\n",
+		       ctx->pfn, po->pid, po->comm);
+	printed_lines++;
+	return 0;
+}
-- 
2.20.1

Re: [PATCH v3 7/7] selftests/bpf: add tests for the page_owner bpf_iter target

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] selftests/bpf: ASSERT_OK_PTR() on strstr() and strchr() passes on NULL
--

commit c911a68fd56ba0ae206e2cf17e238a8881f03bd2
Author: Zhen Ni <zhen.ni@easystack.cn>

selftests/bpf: add tests for the page_owner bpf_iter target

This commit adds a sleepable BPF program and a userspace test runner to
exercise the seq_file and open-coded page_owner BPF iterators and their
associated kfuncs. It also enables CONFIG_PAGE_OWNER in the BPF selftests
configuration.

> diff --git a/tools/testing/selftests/bpf/prog_tests/page_owner_iter.c b/tools/testing/selftests/bpf/prog_tests/page_owner_iter.c
> new file mode 100644
> index 0000000000000..3684ca3089355
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/prog_tests/page_owner_iter.c
[ ... ]
> +/* Validate bpf_page_owner_get_memcg_info() results from the open-coded run. */
> +static void check_memcg_skel(struct bpf_iter_page_owner *skel)
> +{
[ ... ]
> +	if (skel->bss->memcg_first_name[0])
> +		ASSERT_OK_PTR(strstr(skel->bss->memcg_first_path,
> +				    skel->bss->memcg_first_name),
> +			      "memcg_name_in_path");
> +}
> +
> +void serial_test_page_owner_iter(void)
> +{
[ ... ]
> +	ASSERT_GT(skel->bss->stack_bytes, 0, "stack_snprint_ok");
> +	ASSERT_OK_PTR(strstr(skel->bss->stack_text, "+0x"), "stack_frames");
> +	ASSERT_OK_PTR(strchr(skel->bss->stack_text, '\n'), "stack_lines");

[Severity: Medium]
Will these ASSERT_OK_PTR() checks in serial_test_page_owner_iter() and
check_memcg_skel() in tools/testing/selftests/bpf/prog_tests/page_owner_iter.c
actually catch a NULL return from strstr() or strchr()?

In tools/testing/selftests/bpf/test_progs.h, ASSERT_OK_PTR() checks the
pointer via libbpf_get_error():

    #define ASSERT_OK_PTR(ptr, name) ({                                 \
            static int duration = 0;                                    \
            const void *___res = (ptr);                                 \
            int ___err = libbpf_get_error(___res);                      \
            bool ___ok = ___err == 0;                                   \
            CHECK(!___ok, (name), "unexpected error: %d\n", ___err);    \
            ___ok;                                                      \
    })

And in tools/lib/bpf/libbpf.c, libbpf_get_error() returns -errno when
given NULL:

    long libbpf_get_error(const void *ptr)
    {
            if (!IS_ERR_OR_NULL(ptr))
                    return 0;

            if (IS_ERR(ptr))
                    errno = -PTR_ERR(ptr);

            return -errno;
    }

Because strstr() and strchr() do not set errno when returning NULL on a
non-match, if errno is 0 after bpf_prog_test_run_opts() succeeds,
libbpf_get_error(NULL) returns 0 and ASSERT_OK_PTR() silently passes even
when "+0x", '\n', or memcg_first_name is missing.

Should these checks use ASSERT_NEQ(..., NULL, ...) instead?

> +
> +	check_memcg_skel(skel);
> +
> +destroy:
> +	bpf_iter_page_owner__destroy(skel);
> +}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261021112437.1799213-1-zhen.ni@easystack.cn?part=7

Re: [PATCH v3 3/7] mm/page_owner: add bpf_iter target "page_owner"

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] mm/page_owner: rescan and post-stop show() in page_owner_seq_start() at EOF
--

commit a6ba125169596a18417440d179f83a946227810e
Author: Zhen Ni <zhen.ni@easystack.cn>

mm/page_owner: add bpf_iter target "page_owner"

This commit adds a "page_owner" bpf_iter target that iterates over pages
eligible for page_owner output using page_owner_next_eligible() and passes
the PFN and a snapshot of struct page_owner to the attached BPF program.

> diff --git a/mm/page_owner_iter.c b/mm/page_owner_iter.c
> new file mode 100644
> index 0000000000000..529b2c4a589a9
> --- /dev/null
> +++ b/mm/page_owner_iter.c
> @@ -0,0 +1,118 @@

[ ... ]

> +static void *page_owner_seq_start(struct seq_file *seq, loff_t *posp)
> +{
> +	struct page_owner_scan *scan = seq->private;
> +
> +	if (*posp == 0)
> +		scan->pfn = min_low_pfn;
> +
> +	if (!page_owner_next_eligible(scan))
> +		return NULL;
> +	return &scan->po_snap;
> +}
> +
> +static void *page_owner_seq_next(struct seq_file *seq, void *v, loff_t *posp)
> +{
> +	struct page_owner_scan *scan = seq->private;
> +
> +	++*posp;
> +	scan->pfn++;
> +	if (!page_owner_next_eligible(scan))
> +		return NULL;
> +
> +	return &scan->po_snap;
> +}

[Severity: Medium]
Can this rescan PFNs and invoke page_owner_seq_show() again after the final
page_owner_seq_stop() callback has already run?

In mm/page_owner.c:page_owner_next_eligible(), scan->pfn is copied into a
local variable and only written back inside the loop before checking
scan->po_snap.handle:

mm/page_owner.c:page_owner_next_eligible() {
	unsigned long pfn = scan->pfn;
	...
	for (; pfn < max_pfn; pfn++) {
		...
		scan->pfn = pfn;
		scan->po_snap = *page_owner;

		if (!scan->po_snap.handle)
			goto ext_put_continue;
		...
		return true;
		...
	}

	return false;
}

When page_owner_seq_next() advances scan->pfn past the last eligible page
and calls page_owner_next_eligible(scan), the loop reaches max_pfn and
returns false without updating scan->pfn to max_pfn. As a result, scan->pfn
is left at the PFN right after the last hit (or at an intermediate PFN where
po_snap.handle was zero).

In kernel/bpf/bpf_iter.c:bpf_seq_read(), when page_owner_seq_next() returns
NULL, bpf_seq_read() calls page_owner_seq_stop(seq, NULL) (running the BPF
program with ctx.po == NULL), marks iter_priv->done_stop = true, and flushes
any buffered output in seq->buf to userspace with a positive return value.

When userspace issues the next read() call to observe EOF, bpf_seq_read()
calls page_owner_seq_start(), which calls page_owner_next_eligible(scan) and
rescans from the stale scan->pfn up to max_pfn. If any page in that range
was allocated between the two read() calls, page_owner_next_eligible(scan)
returns true and page_owner_seq_show() invokes the BPF program again with
ctx.po != NULL after the terminal ctx.po == NULL stop callback has already
run.

Similarly, if the initial scan finds no eligible pages (*posp remains 0) and
the BPF program writes output during page_owner_seq_stop(seq, NULL), the
follow-up read() enters page_owner_seq_start() with *posp == 0, resets
scan->pfn = min_low_pfn, and rescans the entire PFN range.

Should page_owner_next_eligible() update scan->pfn = pfn when the loop
terminates at max_pfn (and only store scan->pfn = pfn on a hit after checking
scan->po_snap.handle), and should page_owner_seq_start() also ensure *posp is
advanced so a follow-up read() after an empty scan does not reset scan->pfn
to min_low_pfn?

[ ... ]

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261021112437.1799213-1-zhen.ni@easystack.cn?part=3

Re: [PATCH v3 4/7] mm/page_owner: add open-coded page_owner iterator

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Critical] mm/page_owner: forged handle and stack UAF in bpf_page_owner_stack_snprint()
- [High] mm/page_owner: bpf_iter_page_owner_next() violates sticky-NULL contract
- [High] mm/page_owner: buf__uninit in bpf_page_owner_stack_snprint() poisons stack
--

commit c92f39f3d2ecdefdc5191d503168d482f4fadf55
Author: Zhen Ni <zhen.ni@easystack.cn>

mm/page_owner: add open-coded page_owner iterator

This commit introduces an open-coded BPF iterator for page_owner using
bpf_iter_page_owner_new(), bpf_iter_page_owner_next(), and
bpf_iter_page_owner_destroy(), along with the bpf_page_owner_stack_snprint()
kfunc to format the current page's allocation stack trace into a buffer.

> diff --git a/mm/page_owner_iter.c b/mm/page_owner_iter.c
> index 529b2c4a589a9..4b9741a2de53c 100644
> --- a/mm/page_owner_iter.c
> +++ b/mm/page_owner_iter.c
[ ... ]
> @@ -116,3 +117,66 @@ int page_owner_iter_register(void)
[ ... ]
> +__bpf_kfunc struct page_owner_scan *
> +bpf_iter_page_owner_next(struct bpf_iter_page_owner *it)
> +{
> +	struct bpf_iter_page_owner_kern *kit = (void *)it;
> +
> +	if (!static_branch_unlikely(&page_owner_inited))
> +		return NULL;
> +
> +	cond_resched();
> +
> +	kit->scan.pfn++;
> +	if (!page_owner_next_eligible(&kit->scan))
> +		return NULL;

[Severity: High]
In kernel/bpf/verifier.c:process_iter_next_call(), once a KF_ITER_NEXT kfunc
returns NULL and transitions the iterator to BPF_ITER_STATE_DRAINED, the
verifier assumes any subsequent call on the same iterator also returns NULL
and marks BPF_REG_0 as constant scalar 0:

	cur_iter->iter.state = BPF_ITER_STATE_DRAINED;
	__mark_reg_const_zero(env, &cur_fr->regs[BPF_REG_0]);

However, mm/page_owner.c:page_owner_next_eligible() iterates using a local
pfn variable and only updates scan->pfn when an allocated page is reached:

	unsigned long pfn = scan->pfn;
	...
	for (; pfn < max_pfn; pfn++) {
		...
		scan->pfn = pfn;
		scan->po_snap = *page_owner;

When the loop reaches max_pfn without finding an eligible page, it returns
false while leaving kit->scan.pfn at the previously matched PFN plus one (or
min_low_pfn if no page matched).

If a BPF program calls bpf_iter_page_owner_next() again after it has returned
NULL, kit->scan.pfn++ advances from that earlier PFN and rescans up to
max_pfn. Because the scan is lockless and calls cond_resched(), a concurrent
allocation in that tail PFN range (or max_pfn increasing via memory hotplug)
can cause bpf_iter_page_owner_next() to return a non-NULL &kit->scan pointer
at runtime when the verifier tracks R0 as scalar 0.

Should bpf_iter_page_owner_next() latch the drained state (or set
kit->scan.pfn to max_pfn on exhaustion and check it before incrementing) so
that subsequent calls cannot rescan or return non-NULL?

> +
> +	return &kit->scan;
> +}
> +
> +__bpf_kfunc void bpf_iter_page_owner_destroy(struct bpf_iter_page_owner *it)
> +{
> +}
> +
> +__bpf_kfunc int bpf_page_owner_stack_snprint(struct page_owner_scan *scan,
> +					     char *buf__uninit, u32 buf__sz)

[Severity: High]
In mm/page_owner_iter.c:bpf_page_owner_stack_snprint(), the buffer argument
is annotated as buf__uninit paired with buf__sz.

In kernel/bpf/verifier.c:check_kfunc_args(), the __uninit suffix is only
handled for KF_ARG_PTR_TO_DYNPTR, whereas a KF_ARG_PTR_TO_MEM and
KF_ARG_MEM_SIZE pair is validated with BPF_READ | BPF_WRITE and raw_mode set
to false:

	ret = check_mem_size_reg(env, buff_reg, size_reg, buff_argno, argno,
				 BPF_READ | BPF_WRITE, true, meta, &failure);

At the same time, the static stack liveness pass in
record_arg_access() -> bpf_kfunc_stack_access_bytes() checks
is_kfunc_arg_uninit() and returns -size for buf__uninit:

	if (is_kfunc_arg_uninit(btf, &args[arg]))
		return -size;

In kernel/bpf/liveness.c:record_stack_access_off(), a negative access_bytes
marks all covered stack slots as must_write (dead prior to the call), even
though bpf_page_owner_stack_snprint() only writes up to len + 1 bytes. At any
state-pruning checkpoint before the call (such as inside a
bpf_iter_page_owner_next() loop), __clean_func_state() marks those dead stack
slots with STACK_POISON, causing check_mem_size_reg() ->
check_stack_range_initialized() to reject the BPF program with -EACCES.

Could buf__uninit be renamed to buf (or changed to an interface where the
verifier and stack liveness pass agree on initialization semantics) so that
passing a stack buffer in an iterator loop is not rejected?

> +{
> +	int len = 0;
> +
> +	if (!buf__uninit || !buf__sz)
> +		return 0;
> +
> +	if (static_branch_unlikely(&page_owner_inited))
> +		len = stack_depot_snprint(scan->po_snap.handle, buf__uninit,
> +					  buf__sz, 0);

[Severity: Critical]
In mm/page_owner_iter.c:bpf_page_owner_stack_snprint(), scan->po_snap.handle
is passed directly to stack_depot_snprint() -> stack_depot_fetch() ->
depot_fetch_stack(). In lib/stackdepot.c:depot_fetch_stack(), the handle is
assumed to be a valid stackdepot handle:

	if (pool_index >= pools_num_cached) {
		WARN(1, "pool index %d out of bounds (%d) for stack id %08x\n",
		     pool_index, pools_num_cached, handle);
		return NULL;
	}
	...
	stack = pool + offset;
	if (WARN_ON(!refcount_read(&stack->count)))
		return NULL;

An arbitrary handle value can trigger WARN() on an out-of-range pool_index or
zero refcount, or point stack at an arbitrary offset inside
stack_pools[pool_index] where a forged stack->size causes
stack_trace_snprint() to read out of bounds past the end of the pool.

There are two ways a BPF program can supply an arbitrary scan->po_snap.handle
and corrupt or retain stack memory:

1. Because struct page_owner_scan and struct page_owner in mm/page_owner.h
contain only scalar fields and char comm[TASK_COMM_LEN],
__btf_type_is_scalar_struct() returns true. In
kernel/bpf/verifier.c:check_kfunc_args(), when a non-BTF pointer such as
PTR_TO_STACK or PTR_TO_MAP_VALUE is passed as struct page_owner_scan *scan,
the verifier falls back to fixed-size memory and accepts arbitrary
BPF-controlled bytes:

	if (!__btf_type_is_scalar_struct(env, meta->btf, ref_t, 0)) {
		...
		return -EINVAL;
	}
	kf_arg_type = KF_ARG_PTR_TO_MEM | MEM_FIXED_SIZE;
	fallthrough;

2. In mm/page_owner_iter.c:bpf_iter_page_owner_next(), the returned
&kit->scan pointer aliases the on-stack struct bpf_iter_page_owner slots as
PTR_TO_BTF_ID | PTR_TRUSTED. In kernel/bpf/verifier.c:check_kfunc_call(),
regs[BPF_REG_0].parent_id = meta.ref_obj.id is only set in the
!__btf_type_is_struct(ptr_type) branch, not in the PTR_TO_BTF_ID branch:

		if (meta.ref_obj.id) {
			err = validate_ref_obj(env, &meta.ref_obj);
			if (err)
				return err;
			regs[BPF_REG_0].parent_id = meta.ref_obj.id;
		}

		if (is_kfunc_rcu_protected(&meta))
			regs[BPF_REG_0].type |= MEM_RCU;
	} else {
		enum bpf_reg_type type = PTR_TO_BTF_ID;

Because parent_id remains 0 on the returned scan pointer, calling
bpf_iter_page_owner_destroy() marks the iterator stack slots STACK_INVALID
without invalidating registers holding scan. A BPF program can then reuse
those stack slots (spilling pointers or writing arbitrary scalars) and read
them back through scan or pass the overwritten scan->po_snap.handle to
bpf_page_owner_stack_snprint().

Can this be avoided by keeping the stackdepot handle internal to the opaque
struct bpf_iter_page_owner state (having bpf_page_owner_stack_snprint() take
struct bpf_iter_page_owner *it) and tying the lifetime of the pointer
returned by bpf_iter_page_owner_next() to the iterator's ref_obj_id?

[ ... ]

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261021112437.1799213-1-zhen.ni@easystack.cn?part=4

Re: [PATCH v3 6/7] mm/page_owner: add bpf_page_owner_get_memcg_info() kfunc

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [High] mm/page_owner: tail page _nr_pages dereferenced as objcg in get_page_memcg_info()
- [Medium] mm/page_owner: __uninit in bpf_page_owner_get_memcg_info() poisons stack
- [Medium] mm/page_owner: missing __nullable on buf in bpf_page_owner_get_memcg_info()
--

commit 6c73498afe7c643ebd28b236a8d852daad168a14
Author: Zhen Ni <zhen.ni@easystack.cn>

mm/page_owner: add bpf_page_owner_get_memcg_info() kfunc

This commit factors per-page memcg lookup out of print_page_owner_memcg()
into get_page_memcg_info() and adds the bpf_page_owner_get_memcg_info()
kfunc so BPF page_owner iterators can retrieve memcg metadata and the full
cgroup path.

[ ... ]

> diff --git a/mm/page_owner.c b/mm/page_owner.c
> index 8bc15bcdc24e9..7470b5d002be4 100644
> --- a/mm/page_owner.c
> +++ b/mm/page_owner.c
> @@ -538,16 +538,22 @@ void pagetypeinfo_showmixedcount_print(struct seq_file *m,
>  
>  #ifdef CONFIG_MEMCG
>  /*
> - * Looking for memcg information and print it out
> + * Collect memcg information of @page into @info; when @buf and a
> + * nonzero @size are given, also copy the full cgroup path of the
> + * charged memcg into @buf.
> + *
> + * Return: 0 if @page is charged to an attributable memcg; -ENODATA
> + * otherwise.
>   */
> -static inline int print_page_owner_memcg(char *kbuf, size_t count, int ret,
> -					 struct page *page)
> +int get_page_memcg_info(struct page *page, struct memcg_info *info,
> +			char *buf, int size)
>  {
>  	unsigned long memcg_data;
>  	struct obj_cgroup *objcg;
>  	struct mem_cgroup *memcg;
> -	bool online;
> -	char name[80];
> +	int ret = -ENODATA;
> +
> +	*info = (struct memcg_info){};
>  
>  	rcu_read_lock();
>  	memcg_data = READ_ONCE(page->memcg_data);
> @@ -555,31 +561,68 @@ static inline int print_page_owner_memcg(char *kbuf, size_t count, int ret,
>  		goto out_unlock;
>  
>  	if (memcg_data & MEMCG_DATA_OBJEXTS) {
> -		ret += scnprintf(kbuf + ret, count - ret,
> -				"Slab cache page\n");
> +		info->is_slab = true;
>  		goto out_unlock;
>  	}
>  
>  	objcg = (void *)(memcg_data & ~OBJEXTS_FLAGS_MASK);
> -	memcg = objcg ? obj_cgroup_memcg(objcg) : NULL;
> +	if (!objcg)
> +		goto out_unlock;
> +
> +	memcg = obj_cgroup_memcg(objcg);

[Severity: High]
Can get_page_memcg_info() in mm/page_owner.c dereference folio->_nr_pages as
a struct obj_cgroup pointer when inspecting tail page 1 of a compound folio?

In struct folio (include/linux/mm_types.h), the first tail page (__page_1)
places _nr_pages at the same offset as struct page::memcg_data:

include/linux/mm_types.h:struct folio {
    ...
			atomic_t _mapcount_1;
			atomic_t _refcount_1;
#ifdef NR_PAGES_IN_LARGE_FOLIO
			unsigned int _nr_pages;
#endif
		};
		struct page __page_1;
    ...
}

For a compound folio with order >= 3, _nr_pages is 1U << order (for example,
0x8 or 0x200), so its low bits matching OBJEXTS_FLAGS_MASK are zero.

Because bpf_page_owner_get_memcg_info() in mm/page_owner_iter.c calls
get_page_memcg_info() on scan->pfn without stabilizing the page reference or
folio state (and struct page_owner_scan is a scalar-only struct that a BPF
program can also populate on the stack or in a map), page can refer to tail
page 1 of a compound folio.

In get_page_memcg_info(), page->memcg_data is read before checking
PageTail(page):

	memcg_data = READ_ONCE(page->memcg_data);
	if (!memcg_data || PageTail(page))
		goto out_unlock;

If tail page 1 is concurrently freed via __free_pages_prepare() ->
free_tail_page_prepare() -> clear_compound_head() after READ_ONCE() reads
_nr_pages into memcg_data, or if the folio is concurrently split to order 0
in mm/huge_memory.c:__split_folio_to_order() where clear_compound_head()
clears PageTail(new_head) before new_folio->memcg_data overwrites _nr_pages:

mm/huge_memory.c:__split_folio_to_order() {
    ...
		clear_compound_head(new_head);
		if (new_order) {
			prep_compound_page(new_head, new_order);
			folio_set_large_rmappable(new_folio);
		}
    ...
#ifdef CONFIG_MEMCG
		new_folio->memcg_data = folio->memcg_data;
#endif
}

wouldn't PageTail(page) evaluate to false while memcg_data holds 1UL << order,
causing obj_cgroup_memcg(objcg) to dereference that page count as a pointer?

[ ... ]

> diff --git a/mm/page_owner_iter.c b/mm/page_owner_iter.c
> index b7e0ce1dfa80a..a649e1207e246 100644
> --- a/mm/page_owner_iter.c
> +++ b/mm/page_owner_iter.c
> @@ -181,4 +181,52 @@ __bpf_kfunc int bpf_page_owner_stack_snprint(struct page_owner_scan *scan,
>  	return len;
>  }
>  
> +/**
> + * bpf_page_owner_get_memcg_info - Report the memcg charged for the current page
> + * @scan: page_owner scan handle.
> + * @info__uninit: output buffer for the memcg_info.
> + * @buf__uninit: optional output buffer for the full cgroup path.
> + * @buf__sz: size of @buf__uninit.
> + *
> + * Fills @info__uninit with the memcg charged for the page backing
> + * @scan->pfn.  If @buf__uninit and @buf__sz are provided, the full
> + * cgroup path is also copied out, so a BPF program can attribute a
> + * page_owner allocation to a specific cgroup.
> + *
> + * @scan->pfn is resolved to a struct page under get_online_mems(),
> + * so the page cannot be accessed after its memory section is offlined
> + * or removed.
> + *
> + * Returns:
> + * * 0 on success
> + * * -ENODATA if the page is not online or not charged to a memcg
> + * * -EOPNOTSUPP if built without CONFIG_MEMCG
> + */
> +__bpf_kfunc int bpf_page_owner_get_memcg_info(struct page_owner_scan *scan,
> +					      struct memcg_info *info__uninit,
> +					      char *buf__uninit, u32 buf__sz)

[Severity: Medium]
Will the __uninit suffix on info__uninit and buf__uninit in
bpf_page_owner_get_memcg_info() (mm/page_owner_iter.c) cause the BPF verifier
to reject valid programs that pass stack buffers across a pruning checkpoint?

In kernel/bpf/verifier.c:bpf_kfunc_stack_access_bytes(), any kfunc argument
matching is_kfunc_arg_uninit() returns a negative size:

	if (is_kfunc_arg_uninit(btf, &args[arg]))
		return -size;
	return size;

That negative size tells record_stack_access_off() in kernel/bpf/liveness.c
that the kfunc call overwrites the stack slots without reading them, so static
stack liveness analysis marks those slots dead prior to the call. When
clean_verifier_state() -> __clean_func_state() in kernel/bpf/states.c runs at
an earlier pruning checkpoint (such as the second iteration of
bpf_iter_page_owner_next(), or a branch/call after stack allocation), it
poisons those dead stack slots with STACK_POISON:

	for (j = start; j < end; j++)
		st->stack[i].slot_type[j] = STACK_POISON;

However, check_kfunc_args() in kernel/bpf/verifier.c only honors
is_kfunc_arg_uninit() for KF_ARG_PTR_TO_DYNPTR. For KF_ARG_PTR_TO_MEM
(info__uninit) and KF_ARG_MEM_SIZE (buf__uninit), check_kfunc_args() validates
the buffers with BPF_READ | BPF_WRITE:

	ret = check_mem_reg(env, reg, argno, type_size, BPF_READ | BPF_WRITE,
			    meta, &known_memory);
    ...
	ret = check_mem_size_reg(env, buff_reg, size_reg, buff_argno, argno,
				 BPF_READ | BPF_WRITE, true, meta, &failure);

When check_stack_range_initialized() in kernel/bpf/verifier.c performs the
BPF_READ check with allow_poison == false, it hits *stype == STACK_POISON and
fails verification with -EACCES ("slot poisoned by dead code elimination").

> +{
> +	struct page *page;
> +	int ret = -ENODATA;
> +
> +	if (!static_branch_unlikely(&page_owner_inited))
> +		goto nodata;
> +
> +	get_online_mems();
> +	page = pfn_to_online_page(scan->pfn);
> +	if (!page) {
> +		put_online_mems();
> +		goto nodata;
> +	}
> +	ret = get_page_memcg_info(page, info__uninit, buf__uninit, buf__sz);
> +	put_online_mems();
> +	return ret;
> +
> +nodata:
> +	*info__uninit = (struct memcg_info){};
> +	if (buf__uninit && buf__sz)
> +		buf__uninit[0] = '\0';

[Severity: Medium]
Should the optional cgroup path buffer parameter in
bpf_page_owner_get_memcg_info() (mm/page_owner_iter.c) use a __nullable
suffix so BPF programs can pass NULL when they only need struct memcg_info?

Both bpf_page_owner_get_memcg_info() and get_page_memcg_info() check whether
buf is NULL, and the kernel-doc describes buf__uninit as an optional output
buffer. However, in kernel/bpf/verifier.c:get_kfunc_arg_type(), PTR_MAYBE_NULL
is only added when is_kfunc_arg_nullable() matches the "__nullable" (or
"__arena") suffix:

static bool is_kfunc_arg_nullable(const struct btf *btf, const struct btf_param *arg)
{
	return btf_param_match_suffix(btf, arg, "__nullable") ||
	       btf_param_match_suffix(btf, arg, "__arena");
}

Without a __nullable suffix on the buffer argument, check_kfunc_args() in
kernel/bpf/verifier.c rejects a BPF program that passes NULL (or a
PTR_MAYBE_NULL register) for arg#3 with -EACCES ("Possibly NULL pointer passed
to trusted arg#3").

> +	return ret;
> +}
> +
>  __bpf_kfunc_end_defs();

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261021112437.1799213-1-zhen.ni@easystack.cn?part=6

Re: [PATCH v3 1/7] mm/page_owner: extract page_owner_next_eligible() from read loop

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 4369 bytes --]

> mm/page_owner: extract page_owner_next_eligible() from read loop

TL;DR: The nid filter's page->flags read now happens outside the page_ext RCU
window, so it may fault on an unmapped vmemmap during memory hot-remove; later
patches fix it, so it is only a bisect window.

> diff --git a/mm/page_owner.c b/mm/page_owner.c
> index fbbda7ba914b..92a92cdf74ec 100644
> --- a/mm/page_owner.c
> +++ b/mm/page_owner.c

[ ... ]

> @@ -798,13 +794,45 @@ read_page_owner(struct file *file, char __user *buf, size_t count, loff_t *ppos)
>  			goto ext_put_continue;
>
> -		/*
> -		 * Access to page_ext->handle isn't synchronous so we should
> -		 * be careful to access it.
> +		/*
> +		 * Take a snapshot of the page_owner record inside the RCU
> +		 * window, then validate it, so that print_page_owner()
> +		 * reads the checked handle from the same snapshot.
>  		 */
> -		handle = READ_ONCE(page_owner->handle);
> -		if (!handle)
> +		scan->pfn = pfn;
> +		scan->po_snap = *page_owner;
> +
> +		if (!scan->po_snap.handle)
>  			goto ext_put_continue;
>
> +		page_ext_put(page_ext);
> +		return true;
> +
> +ext_put_continue:
> +		page_ext_put(page_ext);
> +		cond_resched();
> +	}
> +
> +	return false;
> +}

[ ... ]

> +	/* Find an allocated page */
> +	while (page_owner_next_eligible(&scan)) {
> +		struct page *page = pfn_to_page(scan.pfn);
> +
>  		if (state->nid_filter_enabled) {
>  			int nid;
>  			memdesc_flags_t page_flags = READ_ONCE(page->flags);

Can this READ_ONCE(page->flags) fault on an unmapped vmemmap?

In the parent commit the nid check ran between page_ext_get() and
page_ext_put(), so it was inside rcu_read_lock():

    page_ext = page_ext_get(page);   /* rcu_read_lock() */
    ...
    if (state->nid_filter_enabled) {
        memdesc_flags_t page_flags = READ_ONCE(page->flags);
        ...
            goto ext_put_continue;
    }
    ...
    ext_put_continue:
        page_ext_put(page_ext);      /* rcu_read_unlock() */

Now page_owner_next_eligible() calls page_ext_put() before it returns true,
and read_page_owner() reads page->flags with no RCU section, no
get_online_mems(), and no page reference held.

The RCU section mattered for memory hot-remove:

    offline_pages()
      memory_notify(MEM_OFFLINE)
        page_ext_callback()
          offline_page_ext()
            synchronize_rcu()

try_remove_memory() requires the blocks to be offline before
arch_remove_memory() -> ... -> section_deactivate() ->
depopulate_section_memmap() unmaps the vmemmap.  While a page_ext_get()
reader was active, that section's memmap could not be torn down.

After this patch, an offline plus remove can complete between page_ext_put()
and the READ_ONCE(page->flags), because synchronize_rcu() no longer waits
for this reader.  This needs CONFIG_PAGE_OWNER with page_owner=on,
CONFIG_MEMORY_HOTREMOVE, a nid filter set through page_owner_write(), and a
concurrent hot-remove of movable memory.

The commit message says "No change to the output is intended", but it does
not mention that the nid filter's struct page access moves out of the page_ext
RCU window.

For context, the parent already read struct page without a pin in
skip_buddy_pages() (PageBuddy() before page_ext_get()) and in
print_page_owner() (page->flags, pageblock flags and page->memcg_data after
page_ext_put()), so this widens an existing race rather than adding a new
class of problem.  It does turn one protected access into an unprotected
one, though.

This is fixed later in the series, so it is only a bisect window issue:

- 915f86c7862f ("mm/page_owner: protect page access with mem-hotplug read
  lock") re-resolves the page in read_page_owner() with pfn_to_online_page()
  under get_online_mems() before the nid check.

- 2cf991a93cbd ("mm/page_owner: record the NUMA node in the scan cursor")
  moves the nid computation back into page_owner_next_eligible(), "to avoid
  dereferencing a possibly-unmapped page".

Could the nid be read in page_owner_next_eligible() before page_ext_put(),
for example by storing it in the cursor?  Alternatively, could the
mem-hotplug protection patch be reordered ahead of this one?

[ ... ]


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/39593164040

Re: [PATCH v3 2/7] mm/page_owner: protect page access with mem-hotplug read lock

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 4819 bytes --]

> mm/page_owner: protect page access with mem-hotplug read lock

TL;DR: read_page_owner() holds the mem-hotplug read lock across
copy_to_user(); a page fault there may deadlock against mmap_lock (lock-order
inversion with memremap_pages callers) and userspace can stall hot-remove
indefinitely.

> diff --git a/mm/page_owner.c b/mm/page_owner.c
> --- a/mm/page_owner.c
> +++ b/mm/page_owner.c

[ ... ]

> @@ -831,7 +843,17 @@ read_page_owner(struct file *file, char __user *buf, size_t count, loff_t *ppos)
>
>  	/* Find an allocated page */
>  	while (page_owner_next_eligible(&scan)) {
> -		struct page *page = pfn_to_page(scan.pfn);
> +		struct page *page;
> +		ssize_t ret;
> +
> +		/*
> +		 * The scan is lockless. Re-resolve the page under a mem-hotplug
> +		 * read-side lock so that we never access a hot-removed section.
> +		 */
> +		get_online_mems();
> +		page = pfn_to_online_page(scan.pfn);
> +		if (!page)
> +			goto skip_continue;

[ ... ]

> @@ -851,10 +873,13 @@ read_page_owner(struct file *file, char __user *buf, size_t count, loff_t *ppos)
>  		/* Record the next PFN to read in the file offset */
>  		*ppos = scan.pfn + 1;
>
> -		return print_page_owner(buf, count, scan.pfn, page,
> +		ret = print_page_owner(buf, count, scan.pfn, page,
>  				&scan.po_snap, scan.po_snap.handle, state);
> +		put_online_mems();
> +		return ret;
                ^^^^

Can this deadlock against mmap_lock?  read_page_owner() now holds the
mem_hotplug_lock reader across the whole print_page_owner() call, and that
includes the copy_to_user() of the formatted record:

print_page_owner()
    ...
    if (copy_to_user(buf, kbuf, ret))
            ret = -EFAULT;

Once kbuf is formatted, nothing in that copy touches struct page, so the
reader isn't needed there.  A user page fault on buf taken inside this
region takes current->mm->mmap_lock for read (on x86 a fault from
kernel-mode uaccess always does, see the 'goto lock_mmap' path in
arch/x86/mm/fault.c for !FAULT_FLAG_USER), and __might_fault() records that
dependency for lockdep.  That creates the lock order:

mem_hotplug_lock(R) -> mmap_lock(R)

The opposite order already exists in the tree.  Code that hotplugs device
memory while holding mmap_read_lock goes through memremap_pages() ->
pagemap_range() -> mem_hotplug_begin(), which does cpus_read_lock() and then
percpu_down_write(&mem_hotplug_lock):

nouveau_svmm_bind()
    mmap_read_lock(mm)
    nouveau_dmem_migrate_vma()
      nouveau_dmem_migrate_chunk()
        nouveau_dmem_migrate_copy_one()
          nouveau_dmem_page_alloc_locked()
            nouveau_dmem_chunk_alloc()
              memremap_pages()

lib/test_hmm.c:dmirror_migrate_to_device()
    mmap_read_lock(mm)
    dmirror_migrate_alloc_and_copy()
      dmirror_devmem_alloc_page()
        dmirror_allocate_chunk()
          memremap_pages()

With three threads sharing one mm, could this sequence occur?

T1 read(page_owner):
    get_online_mems(), then copy_to_user() faults on buf
T2 SVM bind ioctl:
    holds mmap_read_lock(mm), blocks in percpu_down_write(&mem_hotplug_lock)
    waiting for T1
T3 mmap()/munmap():
    queues on mmap_write_lock(mm) behind T2
T1 fault:
    mmap_read_lock(mm) blocks behind T3

None of these waits has a timeout.  While T2 waits it also holds
cpus_read_lock(), so CPU hotplug writers and every later get_online_mems()
caller stall too.

Even without that cycle, the reader can now be held for as long as
userspace wants.  A buf backed by userfaultfd or by a slow or remote file
mapping could stall memory hot-remove for an unbounded time.  Does that
match the commit message claim:

    "Taking the lock per-page bounds the writer's wait to a single page's
    check-and-deref, keeping hot-remove stalls brief while all accesses stay
    protected."

For comparison, ptdump_walk_pgd() already takes get_online_mems() followed
by mmap_write_lock(current->mm).  This new instance sits on a much more
commonly used debug path, and here the lock isn't needed around
copy_to_user() at all.

Would it work to format the record under the reader (or snapshot the
page-derived fields: pageblock migratetype, page->flags, memcg info), call
put_online_mems(), and only then call copy_to_user()?  The kmalloc() in
print_page_owner() could also move outside the locked region.

The last commit in the series (5b123f2cd336 "selftests/bpf: add tests for
the page_owner bpf_iter target") still calls print_page_owner() under the
reader, so the series does not address this later.

>
>  skip_continue:
> +		put_online_mems();
>  		scan.pfn++;
>  		cond_resched();
>  	}


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/39593164040

Re: [PATCH v3 0/7] mm/page_owner: add bpf_iter and kfuncs

[Andrew Morton]

On Wed, 21 Oct 2026 19:24:30 +0800 Zhen Ni <zhen.ni@easystack.cn> wrote:

> page_owner records who allocated every page and is a core memory-debug
> facility, but it currently reaches userspace as the debugfs flat dump:
> slicing allocations by pid, comm, NUMA node, cgroup or stack, etc.,
> means streaming and parsing the entire file in userspace, which is
> expensive on large hosts and hard to query live.

Confused.  Your series 8570b65220e8 ("mm/page_owner: add print_mode
filter") addressed this significantly?

Re: [PATCH v3 0/7] mm/page_owner: add bpf_iter and kfuncs

[Alexei Starovoitov]

On Wed, Oct 21, 2026 at 04:24 AM Zhen Ni <zhen.ni@easystack.cn> wrote:
> Exposing the same walk to eBPF lets a program filter and aggregate in
> the kernel and surface only the records it cares about. This series
> adds that access as a seq_file bpf_iter target and an open-coded
> iterator.

This is overkill. Two iterators, two kfuncs that print strings,
a rewrite of read_page_owner(), a new file and a new header.

All that the prog needs from the kernel is the page_owner record
of a pfn. That's one kfunc. The walk, the eligibility rules and
the filters belong in the bpf prog:

  bpf_for(i, 0, ctx->nr) {
          pfn = ctx->start + i;
          if (bpf_page_owner_read(pfn, &po) ||
              pfn & ((1ul << po.order) - 1))
                  continue;
          if (po.pid != pid || bpf_page_cgroup_ino(pfn) != cgrp)
                  continue;
          /* add 1 << po.order to a hash map keyed by po.handle */
  }


Something like this:

diff --git a/mm/bpf_memcontrol.c b/mm/bpf_memcontrol.c
index d8f579c..3adb716 100644
--- a/mm/bpf_memcontrol.c
+++ b/mm/bpf_memcontrol.c
@@ -161,6 +161,20 @@ __bpf_kfunc void bpf_mem_cgroup_flush_stats(struct mem_cgroup *memcg)
 	mem_cgroup_flush_stats(memcg);
 }

+/**
+ * bpf_page_cgroup_ino - Get the cgroup a page is charged to
+ * @pfn: the page
+ *
+ * Return: What /proc/kpagecgroup has for @pfn: the inode number of the closest
+ * online ancestor of the memory cgroup, 0 if the page is not charged.
+ */
+__bpf_kfunc u64 bpf_page_cgroup_ino(unsigned long pfn)
+{
+	struct page *page = pfn_to_online_page(pfn);
+
+	return page ? page_cgroup_ino(page) : 0;
+}
+
 /**
  * bpf_proactive_reclaim - proactively reclaim memory from a memory cgroup
  * @memcg: the target memory cgroup to reclaim from.
@@ -218,6 +232,7 @@ BTF_ID_FLAGS(func, bpf_mem_cgroup_memory_events)
 BTF_ID_FLAGS(func, bpf_mem_cgroup_usage)
 BTF_ID_FLAGS(func, bpf_mem_cgroup_page_state)
 BTF_ID_FLAGS(func, bpf_mem_cgroup_flush_stats, KF_SLEEPABLE)
+BTF_ID_FLAGS(func, bpf_page_cgroup_ino, KF_PERFMON)

 BTF_KFUNCS_END(bpf_memcontrol_kfuncs)

diff --git a/mm/page_owner.c b/mm/page_owner.c
index fbbda7b..a26d7fe 100644
--- a/mm/page_owner.c
+++ b/mm/page_owner.c
@@ -12,6 +12,7 @@
 #include <linux/seq_file.h>
 #include <linux/memcontrol.h>
 #include <linux/sched/clock.h>
+#include <linux/btf.h>

 #include "page_alloc.h"

@@ -1144,6 +1145,44 @@ static int threshold_set(void *data, u64 val)

 DEFINE_SIMPLE_ATTRIBUTE(threshold_fops, &threshold_get, &threshold_set, "%llu\n");

+__bpf_kfunc_start_defs();
+
+/**
+ * bpf_page_owner_read - Copy the page_owner record of an allocated page
+ * @pfn: the page
+ * @po__uninit: where to copy the record to
+ *
+ * All pages of a high-order allocation have the same record.
+ *
+ * Return: 0 on success, -ENOENT if @pfn is not online or not allocated.
+ */
+__bpf_kfunc int bpf_page_owner_read(unsigned long pfn, struct page_owner *po__uninit)
+{
+	struct page *page = pfn_to_online_page(pfn);
+	struct page_ext *page_ext = page ? page_ext_get(page) : NULL;
+
+	if (!page_ext || !test_bit(PAGE_EXT_OWNER_ALLOCATED, &page_ext->flags)) {
+		page_ext_put(page_ext);
+		memset(po__uninit, 0, sizeof(*po__uninit));
+		return -ENOENT;
+	}
+
+	*po__uninit = *get_page_owner(page_ext);
+	page_ext_put(page_ext);
+	return 0;
+}
+
+__bpf_kfunc_end_defs();
+
+BTF_KFUNCS_START(page_owner_kfuncs)
+BTF_ID_FLAGS(func, bpf_page_owner_read, KF_PERFMON)
+BTF_KFUNCS_END(page_owner_kfuncs)
+
+static const struct btf_kfunc_id_set page_owner_kfunc_set = {
+	.owner	= THIS_MODULE,
+	.set	= &page_owner_kfuncs,
+};
+
 static int __init pageowner_init(void)
 {
 	struct dentry *dir;
@@ -1169,6 +1208,6 @@ static int __init pageowner_init(void)
 			    &stack_fops);
 	debugfs_create_file("count_threshold", 0600, dir, NULL,
 			    &threshold_fops);
-	return 0;
+	return register_btf_kfunc_id_set(BPF_PROG_TYPE_UNSPEC, &page_owner_kfunc_set);
 }
 late_initcall(pageowner_init)

pw-bot: cr

Re: [PATCH v3 0/7] mm/page_owner: add bpf_iter and kfuncs

[zhen.ni]

在 2026/10/22 02:52, Andrew Morton 写道:
> On Wed, 21 Oct 2026 19:24:30 +0800 Zhen Ni <zhen.ni@easystack.cn> wrote:
> 
>> page_owner records who allocated every page and is a core memory-debug
>> facility, but it currently reaches userspace as the debugfs flat dump:
>> slicing allocations by pid, comm, NUMA node, cgroup or stack, etc.,
>> means streaming and parsing the entire file in userspace, which is
>> expensive on large hosts and hard to query live.
> 
> Confused.  Your series 8570b65220e8 ("mm/page_owner: add print_mode
> filter") addressed this significantly?
> 
> 

Thank you. I'd like to share some background on how this new approach
came about.

After series 8570b65220e8 ("mm/page_owner: add print_mode filter"), I
attempted to further extend the filter capabilities so that userspace
could more selectively capture page owner results. This effort led to
the series [PATCH v3 0/8] mm/page_owner: Add PID/TGID/COMM and cgroup
filtering.

During that discussion, we gradually reached a consensus that "a BPF
iter" might be a better solution to this problem. The main concerns
with continuing to extend the kernel-side filter approach were:

  1. As filter capabilities grow, the kernel side needs to handle
  increasingly complex parameter parsing and various combinations of
  different filter attributes, which is not conducive to maintainability.

  2. More fundamentally, kernel-side filters cannot provide truly
  flexible filtering because the filtering logic is hardcoded in the
  kernel.

At that point, I did a POC to verify the BPF iter direction:

  
https://lore.kernel.org/linux-mm/99a94068-ab37-47c0-878c-7c1ada5204b2@easystack.cn/

It seems there is broad agreement on this direction. This new series is
the implementation of that approach.


Thanks,
Zhen

Re: [PATCH v3 0/7] mm/page_owner: add bpf_iter and kfuncs

[zhen.ni]

在 2026/10/22 06:00, Alexei Starovoitov 写道:
> On Wed, Oct 21, 2026 at 04:24 AM Zhen Ni <zhen.ni@easystack.cn> wrote:
>> Exposing the same walk to eBPF lets a program filter and aggregate in
>> the kernel and surface only the records it cares about. This series
>> adds that access as a seq_file bpf_iter target and an open-coded
>> iterator.
> 
> This is overkill. Two iterators, two kfuncs that print strings,
> a rewrite of read_page_owner(), a new file and a new header.
> 

Thanks for the detailed review and the sketch. I agree with the
principle that a thin data kfunc would be ideal if it were sufficient,
but I’d like to explain why the current design ended up this way.

On the two iterator surfaces:
They aren’t duplicate implementations. Both are thin shells over the
single page_owner_next_eligible() extracted from read_page_owner(), so
the eligibility rules exist exactly once. There’s no code redundancy;
the two iters only differ in access patterns while sharing the same
underlying iteration logic. This shape is also what Documentation/bpf/
bpf_iterators.rst prescribes:

When adding a new BPF iterator program, it is expected that similar
functionality will be added as open-coded iterator for maximum
flexibility.
It's also expected that iteration logic and code will be maximally 
shared and reused between two iterator API surfaces.

On the two “print strings” kfuncs:
These are things a prog genuinely cannot do on its own. The record only
carries an opaque stack-depot handle, and resolving it is not reachable
from BPF. Likewise, the cgroup path is built by cgroup_path() walking
the kernfs parent chain — a prog has no way to format it. Given the 512-
byte BPF stack limit, the caller-provides-buffer pattern (as in
bpf_d_path() and bpf_snprintf_btf()) is the standard shape for this kind
of kfunc. I agree they could have been mentioned earlier rather than
left to the diff.

On “a rewrite of read_page_owner()”:
This is really an extraction of common functions, not a rewrite. Aside
from the fix for mem-hotplug locking (which addresses a pre-existing bug
in the debugfs interface), there are no substantial logic changes. It’s
not a large-scale refactoring.

On the new file and header:
This is for code clarity — to distinguish the traditional interface from
the BPF interface and make maintenance easier.

> All that the prog needs from the kernel is the page_owner record
> of a pfn. That's one kfunc. The walk, the eligibility rules and
> the filters belong in the bpf prog:
> 
>    bpf_for(i, 0, ctx->nr) {
>            pfn = ctx->start + i;
>            if (bpf_page_owner_read(pfn, &po) ||
>                pfn & ((1ul << po.order) - 1))
>                    continue;
>            if (po.pid != pid || bpf_page_cgroup_ino(pfn) != cgrp)
>                    continue;
>            /* add 1 << po.order to a hash map keyed by po.handle */
>    }
> 

On why the prog can't actually execute it:

1. What range to scan. Where do ctx->start and ctx->nr come from? No
in-tree iterator context provides them, and a prog cannot derive them:
min_low_pfn..max_pfn, memblock holes, and offline sections are all
kernel state. The loop bounds alone require either another kfunc or a
kernel-side walk — which is exactly what the iter target already is.

2. What makes a page eligible. The alignment test in the sketch is the
only part the kernel agrees is expressible in a prog. Everything before
it reads state a prog doesn’t have a handle to: pfn_valid() over
MAX_ORDER_NR_PAGES areas, pfn_to_online_page() under get_online_mems(),
skip_buddy_pages(), the PAGE_EXT_OWNER / PAGE_EXT_OWNER_ALLOCATED flags
in page_ext, and taking the record snapshot inside the RCU window so the
checked value and the reported value are the same copy.

3. The walk is deliberately lockless, so exposing struct page to the
prog is not safe. The pfn validity check and the page_ext dereference
must happen under the same kernel-controlled RCU window. Otherwise the
prog could observe offlined pages or stale page_ext data, breaking
consistency between the check and the snapshot. Keeping the walk in the
kernel guarantees that the page_owner record is read atomically with
respect to mem-hotplug and RCU protection.

4. One record doesn’t give the prog the useful data. The record holds
only opaque handle/free_handle tokens; stack-depot resolution exists
solely in the kernel. The memcg is neither in the record nor trivially
reachable — the path is built by cgroup_path() walking the kernfs parent
chain. The NUMA node comes from page_owner_get_nid(), which bypasses
page_to_nid()’s poison check. And so on.


Thanks again for pushing on this. I’m still concerned that the sketch
may not be workable in practice, and that it would also lose some useful
capabilities. I hope the above clarifies the reasoning. Happy to discuss
further.



Thanks,
Zhen

Re: [PATCH v3 0/7] mm/page_owner: add bpf_iter and kfuncs

[Alexei Starovoitov]

On Fri, Oct 23, 2026 at 07:51 AM zhen.ni <zhen.ni@easystack.cn> wrote:

tldr: it's still nack

> underlying iteration logic. This shape is also what Documentation/bpf/
> bpf_iterators.rst prescribes:
>
> When adding a new BPF iterator program, it is expected that similar
> functionality will be added as open-coded iterator for maximum
> flexibility.

It says what to do when an iterator is added. It doesn't say that
one is needed.
Iterators are for kernel data structures that a prog cannot walk
on its own: tasks, cgroups, kmem_caches.
pfn is a number. bpf_for() is the iterator.

> 1. What range to scan. Where do ctx->start and ctx->nr come from? No
> in-tree iterator context provides them, and a prog cannot derive them:
> min_low_pfn..max_pfn, memblock holes, and offline sections are all
> kernel state. The loop bounds alone require either another kfunc or a
> kernel-side walk — which is exactly what the iter target already is.

From user space. It's a syscall prog, like scan_open_coded() in your
patch 7. Its ctx is what user space passes in ctx_in.
/proc/zoneinfo has start_pfn and spanned for every zone.
In holes and offline sections pfn_to_online_page() returns NULL
and bpf_page_owner_read() returns -ENOENT.

> 2. What makes a page eligible. The alignment test in the sketch is the
> only part the kernel agrees is expressible in a prog. Everything before
> it reads state a prog doesn’t have a handle to: pfn_valid() over
> MAX_ORDER_NR_PAGES areas, pfn_to_online_page() under get_online_mems(),
> skip_buddy_pages(), the PAGE_EXT_OWNER / PAGE_EXT_OWNER_ALLOCATED flags
> in page_ext, and taking the record snapshot inside the RCU window so the
> checked value and the reported value are the same copy.

bpf_page_owner_read() in the diff does all of that:
pfn_to_online_page(), page_ext_get(), PAGE_EXT_OWNER_ALLOCATED check
and the copy of the record before page_ext_put().
PAGE_EXT_OWNER is set together with PAGE_EXT_OWNER_ALLOCATED and it's
never cleared. No need to check both.
__reset_page_owner() clears PAGE_EXT_OWNER_ALLOCATED, so
skip_buddy_pages() and MAX_ORDER_NR_PAGES steps only make the walk
faster. They don't change what is eligible.

> 3. The walk is deliberately lockless, so exposing struct page to the
> prog is not safe. The pfn validity check and the page_ext dereference
> must happen under the same kernel-controlled RCU window. Otherwise the

The prog doesn't see struct page or page_ext. It passes a pfn and gets
a copy of the record. The lookup, the check and the copy are in one RCU
section inside the kfunc. offline_page_ext() does synchronize_rcu()
before freeing page_ext.
page_ext_from_phys() does the same lookup from any context.

> 4. One record doesn’t give the prog the useful data. The record holds
> only opaque handle/free_handle tokens; stack-depot resolution exists
> solely in the kernel. The memcg is neither in the record nor trivially
> reachable — the path is built by cgroup_path() walking the kernfs parent
> chain. The NUMA node comes from page_owner_get_nid(), which bypasses
> page_to_nid()’s poison check. And so on.

The handle is the key to filter and aggregate by.
page_owner_stacks/show_stacks_handles maps handles to stacks.
Same for memcg. bpf_page_cgroup_ino() in the diff returns the id.
st_ino of the cgroup directory is that id.

Re: [PATCH v3 0/7] mm/page_owner: add bpf_iter and kfuncs

[zhen.ni]

在 2026/10/23 23:58, Alexei Starovoitov 写道:
> On Fri, Oct 23, 2026 at 07:51 AM zhen.ni <zhen.ni@easystack.cn> wrote:
> 
> tldr: it's still nack
> 
>> underlying iteration logic. This shape is also what Documentation/bpf/
>> bpf_iterators.rst prescribes:
>>
>> When adding a new BPF iterator program, it is expected that similar
>> functionality will be added as open-coded iterator for maximum
>> flexibility.
> 
> It says what to do when an iterator is added. It doesn't say that
> one is needed.
> Iterators are for kernel data structures that a prog cannot walk
> on its own: tasks, cgroups, kmem_caches.
> pfn is a number. bpf_for() is the iterator.

I agree the doc is prescribing what to do when an iterator is added, not
that every case must have one. But my point is that having both
iterators coexist is the convention, not overkill.

Page owner traversal looks like iterating over PFNs, but PFN is just a
number; what is actually being traversed is page_owner records, buddy
blocks, online/ offline sections, memblock holes—kernel state that a
prog cannot see on its own. So it is not a pure numeric-range walk.

> 
>> 1. What range to scan. Where do ctx->start and ctx->nr come from? No
>> in-tree iterator context provides them, and a prog cannot derive them:
>> min_low_pfn..max_pfn, memblock holes, and offline sections are all
>> kernel state. The loop bounds alone require either another kfunc or a
>> kernel-side walk — which is exactly what the iter target already is.
> 
>  From user space. It's a syscall prog, like scan_open_coded() in your
> patch 7. Its ctx is what user space passes in ctx_in.
> /proc/zoneinfo has start_pfn and spanned for every zone.
> In holes and offline sections pfn_to_online_page() returns NULL
> and bpf_page_owner_read() returns -ENOENT.
> 

I agree the range can be obtained. But being able to obtain a range is
not the same as that range being meaningful to the user. Page owner
users typically care about a process, a cgroup's leaks, or the global
distribution; PFN is just a physical address index, and users almost
never know in advance which zone or which PFN range to scan. Letting the
prog specify a PFN range raises the bar for writing it while the
practical use cases remain limited. This is not a feasibility issue—it
is a question of abstraction level.

>> 2. What makes a page eligible. The alignment test in the sketch is the
>> only part the kernel agrees is expressible in a prog. Everything before
>> it reads state a prog doesn’t have a handle to: pfn_valid() over
>> MAX_ORDER_NR_PAGES areas, pfn_to_online_page() under get_online_mems(),
>> skip_buddy_pages(), the PAGE_EXT_OWNER / PAGE_EXT_OWNER_ALLOCATED flags
>> in page_ext, and taking the record snapshot inside the RCU window so the
>> checked value and the reported value are the same copy.
> 
> bpf_page_owner_read() in the diff does all of that:
> pfn_to_online_page(), page_ext_get(), PAGE_EXT_OWNER_ALLOCATED check
> and the copy of the record before page_ext_put().
> PAGE_EXT_OWNER is set together with PAGE_EXT_OWNER_ALLOCATED and it's
> never cleared. No need to check both.
> __reset_page_owner() clears PAGE_EXT_OWNER_ALLOCATED, so
> skip_buddy_pages() and MAX_ORDER_NR_PAGES steps only make the walk
> faster. They don't change what is eligible.

I agree on eligibility. But I'd like to expand on "only faster": what
those steps change is the scale of the walk, and under SPARSEMEM that
scale is structural, not a constant factor. On a 256G, 8-NUMA machine I
have, node0 is:

crash> struct pglist_data ffff0106ffffe580 | grep node_spanned_pages
   node_spanned_pages = 17235968,
crash> struct pglist_data ffff0106ffffe580 | grep node_present_pages
   node_present_pages = 523024,

For this one node alone, about 97% of spanned is holes. With
bpf_for(pfn, start, end) plus a bpf_page_owner_read() per iteration,
that is at least 16,712,944 kfunc calls, and bpf_for has no scheduling
point inside, and all of them are wasted work. The gap grows with memory
size, and I'm concerned "only faster" understates the runtime cost on
real machines.

One more structural issue: bpf_for is capped at BPF_MAX_LOOPS = 8 * 1024
* 1024, and this is a runtime check—the verifier will not reject it up
front. The spanned of this node is already 17.23M, so a single bpf_for
cannot cover it; you have to split it into N chunks and poll, with the
chunking state kept in a prog map, i.e. a loop nested in a loop.

And cur/end are stored as BPF_W (32-bit), while the PFN upper bound
tracks the physical address width: on x86-64 with 46-bit PA, max_pfn can
reach 2^34, and 2^40 with LA57; on arm64, 2^36 for 48-bit PA and 2^40
for 52-bit PA. A 32-bit PFN range will overflow here.

Solving all of this requires yet another, more complex kfunc—and it
looks increasingly like bpf iter. So why not just use the iter?

> 
>> 3. The walk is deliberately lockless, so exposing struct page to the
>> prog is not safe. The pfn validity check and the page_ext dereference
>> must happen under the same kernel-controlled RCU window. Otherwise the
> 
> The prog doesn't see struct page or page_ext. It passes a pfn and gets
> a copy of the record. The lookup, the check and the copy are in one RCU
> section inside the kfunc. offline_page_ext() does synchronize_rcu()
> before freeing page_ext.
> page_ext_from_phys() does the same lookup from any context.

I have no doubt about feasibility.

> 
>> 4. One record doesn’t give the prog the useful data. The record holds
>> only opaque handle/free_handle tokens; stack-depot resolution exists
>> solely in the kernel. The memcg is neither in the record nor trivially
>> reachable — the path is built by cgroup_path() walking the kernfs parent
>> chain. The NUMA node comes from page_owner_get_nid(), which bypasses
>> page_to_nid()’s poison check. And so on.
> 
> The handle is the key to filter and aggregate by.
> page_owner_stacks/show_stacks_handles maps handles to stacks.
> Same for memcg. bpf_page_cgroup_ino() in the diff returns the id.
> st_ino of the cgroup directory is that id.
> 
> 

Similarly, I have no doubt about feasibility. I do have some concerns
about data consistency and performance. It also increases prog
complexity.

In summary, while I don't dispute the feasibility of the bpf_for +
bpf_page_owner_read approach and fully agree that kernel-side simplicity
is paramount, I wonder if we should also weigh the complexity on the
program side. If using this kfunc imposes a steep learning curve, I'm
concerned it might limit adoption. If a feature lacks practical utility,
it becomes difficult to justify the effort.

Thanks for the detailed explanations and for walking me through
these points.



Thanks,
Zhen

Re: [PATCH v3 0/7] mm/page_owner: add bpf_iter and kfuncs

[Alexei Starovoitov]

On Sat, Oct 24, 2026 at 10:19 AM zhen.ni <zhen.ni@easystack.cn> wrote:
> For this one node alone, about 97% of spanned is holes. With
> bpf_for(pfn, start, end) plus a bpf_page_owner_read() per iteration,
> that is at least 16,712,944 kfunc calls, and bpf_for has no scheduling
> point inside, and all of them are wasted work. The gap grows with memory

Right. spanned includes the holes. So don't pass them.
/sys/devices/system/memory/ has memoryN only for present blocks.
/sys/devices/system/node/nodeN/ links to the blocks of that node.
/proc/iomem has "System RAM" ranges.
523024 pages in a 32G node looks like 64K pages.
Then the whole 256G box is 4M pfns.
When a hole is passed anyway the call ends at !online_section() in
pfn_to_online_page(). /proc/kpageflags does that plus put_user()
for every pfn, holes included.
Syscall progs run under migrate_disable() only.

> front. The spanned of this node is already 17.23M, so a single bpf_for
> cannot cover it; you have to split it into N chunks and poll, with the
> chunking state kept in a prog map, i.e. a loop nested in a loop.

The loop over chunks is in user space. One bpf_prog_test_run_opts()
per range with start and nr in ctx_in. No map, no nested loop.

> for 52-bit PA. A 32-bit PFN range will overflow here.

The snippet was:
  bpf_for(i, 0, ctx->nr) {
          pfn = ctx->start + i;
i is an offset within the chunk. start and pfn are 64-bit.

> Solving all of this requires yet another, more complex kfunc—and it
> looks increasingly like bpf iter. So why not just use the iter?

Nothing of the above needs kernel code.
v3 is +474 -83 lines on the kernel side. The diff I sent is +55 -1
and it doesn't touch the debugfs path.

Re: [PATCH v3 0/7] mm/page_owner: add bpf_iter and kfuncs

[zhen.ni]

在 2026/10/25 03:08, Alexei Starovoitov 写道:
> On Sat, Oct 24, 2026 at 10:19 AM zhen.ni <zhen.ni@easystack.cn> wrote:
>> For this one node alone, about 97% of spanned is holes. With
>> bpf_for(pfn, start, end) plus a bpf_page_owner_read() per iteration,
>> that is at least 16,712,944 kfunc calls, and bpf_for has no scheduling
>> point inside, and all of them are wasted work. The gap grows with memory
> 
> Right. spanned includes the holes. So don't pass them.
> /sys/devices/system/memory/ has memoryN only for present blocks.
> /sys/devices/system/node/nodeN/ links to the blocks of that node.
> /proc/iomem has "System RAM" ranges.

Thanks for taking the time to review this.

 From /sys/devices/system/memory/ you get a discrete list of memory
blocks. The prog (or userspace) still has to walk those blocks, handle
boundaries, and track progress. That is traversal logic that an iterator
already absorbs into a single next() call. Pushing it out just moves the
complexity; it doesn't eliminate it.

Therefore, this only demonstrates feasibility, not superiority.

> 523024 pages in a 32G node looks like 64K pages.
> Then the whole 256G box is 4M pfns.
> When a hole is passed anyway the call ends at !online_section() in
> pfn_to_online_page(). /proc/kpageflags does that plus put_user()
> for every pfn, holes included.

True, the kernel safely returns an error for offline PFNs. But paying a
kfunc call just to learn that a PFN is ineligible is exactly the waste
the iterator eliminates — the kernel-side cursor skips holes and
ineligible pages before the prog ever sees them.

> Syscall progs run under migrate_disable() only.

Agreed, migrate_disable() pins the task to one CPU without disabling
preemption on PREEMPT_RT. But being preemptible doesn't mean the loop is
free — an 8M-iteration tight loop still consumes CPU, delays RCU grace
periods, and piles up softirq work. Skipping ineligible pages up front
reduces that pressure.

> 
>> front. The spanned of this node is already 17.23M, so a single bpf_for
>> cannot cover it; you have to split it into N chunks and poll, with the
>> chunking state kept in a prog map, i.e. a loop nested in a loop.
> 
> The loop over chunks is in user space. One bpf_prog_test_run_opts()
> per range with start and nr in ctx_in. No map, no nested loop.
> 

The outer loop being in userspace doesn't remove the chunking burden —
it just relocates it. Userspace still has to parse sysfs, construct
ranges, issue one syscall per chunk, read the map, accumulate results
into a global aggregate, clear the map, and handle partial failures. And
in the snippet, i is an offset within the chunk while start and pfn are
64-bit — the chunking arithmetic itself is non-trivial. An iterator
absorbs all of this into next().

>> for 52-bit PA. A 32-bit PFN range will overflow here.
> 
> The snippet was:
>    bpf_for(i, 0, ctx->nr) {
>            pfn = ctx->start + i;
> i is an offset within the chunk. start and pfn are 64-bit.
> 
>> Solving all of this requires yet another, more complex kfunc—and it
>> looks increasingly like bpf iter. So why not just use the iter?
> 
> Nothing of the above needs kernel code.

Feasibility isn't in question. The question is whether we want every
prog writer to reinvent this traversal plumbing, or let the kernel do
what iterators are designed for.

> v3 is +474 -83 lines on the kernel side. The diff I sent is +55 -1
> and it doesn't touch the debugfs path.
> 

That diff includes my improvements to the page_owner infrastructure, so
the line count doesn't really establish that bpf_for is better than
iter. The iter approach only adds an exit on the debugfs path — thin
shells over the single page_owner_next_eligible() extracted from
read_page_owner(). It changes no logic on the debugfs path (aside from
the mem-hotplug locking fix, which addresses a pre-existing bug in the
debugfs interface).

And the fact that the bpf_for approach doesn't touch the debugfs path
is, if anything, a disadvantage: it means two sets of eligibility rules
would have to be maintained going forward.

With the iter approach, any future change only needs to be made in
page_owner_next_eligible().



Thanks,
Zhen

Re: [PATCH v3 0/7] mm/page_owner: add bpf_iter and kfuncs

[Alexei Starovoitov]

On Sat, Oct 24, 2026 at 07:16 PM zhen.ni <zhen.ni@easystack.cn> wrote:
> Feasibility isn't in question. The question is whether we want every
> prog writer to reinvent this traversal plumbing, or let the kernel do
> what iterators are designed for.

What the prog and user space can do on their own stays out of
the kernel. The reason for the nack is the same as before.

I'm done with this thread and all of your future patches.
It's auto NACK for all of them. Pls carry it forward,
so no one in the kernel community wastes time on this anymore.

pw-bot: cr