Re: [LTP] hugemmap10: Account for memory policy in mmap counter checks

Re: [LTP] hugemmap10: Account for memory policy in mmap counter checks

[linuxtestproject.agent]

Hi Huaisheng,

On Tue Sep 29 15:24:10 2026 +0800, Huaisheng Ye <yehuaisheng@open-hieco.net> wrote:
> hugemmap10: Account for memory policy in mmap counter checks

Verdict - Reviewed

---
Note:

The agent can sometimes produce false positives although often its
findings are genuine. If you find issues with the review, please
comment this email or ignore the suggestions.

Regards,
LTP AI Reviewer

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH v3 0/3] hugemmap10: Account for memory policy in counter checks

[Huaisheng Ye]

The hugetlb reservation fix allocates surplus hugepages when global free
pages are outside nodes allowed by the current cpuset and MPOL_BIND policy.
This series updates hugemmap10's global counter expectations to match that
behavior.

Patch 1 fixes mappings that create their reservation at mmap() time. Patches
2 and 3 add and validate a MAP_NORESERVE private-mapping mode, which lets the
test exercise fault-time reservations without booting an older kernel.

Changes in v3:
- Rebase onto 113594f53 ("hugemmap10: Treat fd 0 as valid in cleanup").
- Add hugemmap10 -n to runtest/hugetlb so the default suite exercises the
  private fault-time reservation path.
- Remove the redundant private_noreserve variable and use
  private_noreserve_opt directly to detect -n.
- Define MAP_TYPE in lapi/mmap.h when libc headers lack it, and include
  that header in hugemmap10.c.
- Collect Reviewed-by in 1/3 and 3/3.

v2: https://lore.kernel.org/all/20260929072412.20979-1-yehuaisheng@open-hieco.net/

Huaisheng Ye (3):
  hugemmap10: Account for memory policy in mmap counter checks
  hugemmap10: Add private MAP_NORESERVE test mode
  hugemmap10: Account for memory policy in private fault checks

 include/lapi/mmap.h                           |   4 +
 runtest/hugetlb                               |   1 +
 .../kernel/mem/hugetlb/hugemmap/hugemmap10.c  | 155 ++++++++++++++++--
 3 files changed, 145 insertions(+), 15 deletions(-)

-- 
2.52.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH v3 1/3] hugemmap10: Account for memory policy in mmap counter checks

[Huaisheng Ye]

Expected surplus accounting for mappings that reserve at mmap() time only
considers the global free and reserved hugepage counters. But sometimes,
for instance with MPOL_BIND, globally free hugepages can reside outside
nodes from which the task can allocate or plans to allocate. Hugetlb
then creates surplus pages on allowed nodes, while the test expects
no counter change.

For example, reproduce this on an eight-node system running 7.3.0-rc1:

    $ numactl --membind=0-3 ./hugemmap10
    tst_hugepage.c:84: TINFO: 3 hugepage(s) reserved
    ...
    hugemmap10.c:386: TINFO: Base pool size: 0
    hugemmap10.c:313: TINFO: Clean...
    hugemmap10.c:364: TINFO: OK
    hugemmap10.c:313: TINFO: Untouched, shared...
    hugemmap10.c:364: TINFO: OK
    hugemmap10.c:313: TINFO: Untouched, private...
    hugemmap10.c:364: TINFO: OK
    hugemmap10.c:313: TINFO: Touched, shared...
    hugemmap10.c:364: TINFO: OK
    hugemmap10.c:313: TINFO: Touched, private...
    hugemmap10.c:364: TINFO: OK
    hugemmap10.c:386: TINFO: Base pool size: 1
    hugemmap10.c:313: TINFO: Clean...
    hugemmap10.c:319: TFAIL: While doing mmap shared with no touch: Bad HugePages_Total: expected 1, actual 2
    hugemmap10.c:319: TFAIL: While doing mmap shared with no touch: Bad HugePages_Free: expected 1, actual 2
    hugemmap10.c:319: TFAIL: While doing mmap shared with no touch: Bad HugePages_Surp: expected 0, actual 1

    Summary:
    passed   0
    failed   3
    broken   0
    skipped  0
    warnings 0

    $ numactl --membind=4-7 ./hugemmap10
    ...
    hugemmap10.c:364: TINFO: OK
    hugemmap10.c:429: TPASS: Hugepages Counters works as expected.

    Summary:
    passed   1
    failed   0
    broken   0
    skipped  0
    warnings 0

If global free hugepages are on nodes 4-7 while the memory policy restricts
hugemmap10 to nodes 0-3, the counter discrepancy is observed.

Read the task policy and cpuset mask, then sum free hugepages on the
nodes available to the mapping. Calculate the expected surplus as the
maximum of the global and allowed-node shortfalls. Match hugetlb's fallback
to the cpuset when an MPOL_BIND mask no longer intersects it.

This fixes counter expectations for MAP_SHARED and MAP_PRIVATE mappings
that create reservations at mmap() time.

This complements the hugetlb reservation fix:
https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git/commit/?id=21f514ad520459fbfa58ccdf171bb68148c2f5aa

Signed-off-by: Huaisheng Ye <yehuaisheng@open-hieco.net>
Reviewed-by: Li Wang <li.wang@linux.dev>
---
 .../kernel/mem/hugetlb/hugemmap/hugemmap10.c  | 107 +++++++++++++++++-
 1 file changed, 103 insertions(+), 4 deletions(-)

diff --git a/testcases/kernel/mem/hugetlb/hugemmap/hugemmap10.c b/testcases/kernel/mem/hugetlb/hugemmap/hugemmap10.c
index fc530dda4..1f8d00226 100644
--- a/testcases/kernel/mem/hugetlb/hugemmap/hugemmap10.c
+++ b/testcases/kernel/mem/hugetlb/hugemmap/hugemmap10.c
@@ -13,6 +13,9 @@
  */
 
 #define _GNU_SOURCE
+#include <errno.h>
+#include <linux/mempolicy.h>
+#include <stdlib.h>
 #include <unistd.h>
 #include <stdio.h>
 #include <sys/mount.h>
@@ -21,8 +24,10 @@
 #include <sys/types.h>
 
 #include "hugetlb.h"
+#include "lapi/syscalls.h"
 
 #define MNTPOINT "hugetlbfs/"
+#define ULONG_BITS (sizeof(unsigned long) * CHAR_BIT)
 
 static long hpage_size;
 static int private_resv;
@@ -48,6 +53,98 @@ static void read_meminfo_huge(long *total, long *free, long *resv, long *surp)
 	*surp = SAFE_READ_MEMINFO(MEMINFO_HPAGE_SURP);
 }
 
+static int node_isset(unsigned long node, const unsigned long *nodemask)
+{
+	return !!(nodemask[node / ULONG_BITS] & (1UL << (node % ULONG_BITS)));
+}
+
+static int nodemasks_intersect(const unsigned long *left,
+			       const unsigned long *right, unsigned long maxnode)
+{
+	unsigned long node;
+
+	for (node = 0; node < maxnode; node++) {
+		if (node_isset(node, left) && node_isset(node, right))
+			return 1;
+	}
+
+	return 0;
+}
+
+static unsigned long surplus_pages_needed(int hpages, unsigned long allowed_free)
+{
+	unsigned long global_shortfall = 0;
+	unsigned long allowed_shortfall = 0;
+
+	if (hpages + prev_resv > prev_free)
+		global_shortfall = hpages - prev_free + prev_resv;
+
+	if ((unsigned long)hpages > allowed_free)
+		allowed_shortfall = hpages - allowed_free;
+
+	return MAX(global_shortfall, allowed_shortfall);
+}
+
+static int mapping_creates_reservation(int flags)
+{
+	return (flags & MAP_SHARED) || private_resv;
+}
+
+static unsigned long read_allowed_free_hugepages(void)
+{
+	char path[PATH_MAX];
+	size_t mask_size = getpagesize();
+	unsigned long allowed_free = 0;
+	unsigned long maxnode = mask_size * CHAR_BIT;
+	unsigned long *policy_nodemask;
+	unsigned long *cpuset_nodemask;
+	unsigned long node, node_free;
+	int mode, use_bind;
+
+	policy_nodemask = SAFE_CALLOC(1, mask_size);
+	cpuset_nodemask = SAFE_CALLOC(1, mask_size);
+
+	if (syscall(__NR_get_mempolicy, &mode, policy_nodemask,
+		    maxnode, NULL, 0)) {
+		if (errno == ENOSYS) {
+			allowed_free = prev_free;
+			goto out;
+		}
+		tst_brk(TBROK | TERRNO, "get_mempolicy() failed");
+	}
+
+	if (syscall(__NR_get_mempolicy, NULL, cpuset_nodemask,
+		    maxnode, NULL, MPOL_F_MEMS_ALLOWED))
+		tst_brk(TBROK | TERRNO,
+			"get_mempolicy(MPOL_F_MEMS_ALLOWED) failed");
+
+	mode &= ~MPOL_MODE_FLAGS;
+	/* Hugetlb ignores a bind policy that no longer overlaps the cpuset. */
+	use_bind = mode == MPOL_BIND &&
+		nodemasks_intersect(policy_nodemask, cpuset_nodemask, maxnode);
+	for (node = 0; node < maxnode; node++) {
+		if (!node_isset(node, cpuset_nodemask))
+			continue;
+		if (use_bind && !node_isset(node, policy_nodemask))
+			continue;
+
+		snprintf(path, sizeof(path),
+			 "/sys/devices/system/node/node%lu/hugepages/"
+			 "hugepages-%ldkB/free_hugepages",
+			 node, hpage_size / 1024);
+		if (access(path, R_OK))
+			continue;
+
+		SAFE_FILE_SCANF(path, "%lu", &node_free);
+		allowed_free += node_free;
+	}
+
+out:
+	free(policy_nodemask);
+	free(cpuset_nodemask);
+	return allowed_free;
+}
+
 static int kernel_has_private_reservations(void)
 {
 	int fd;
@@ -178,10 +275,14 @@ out:
 
 static int map_(int s, int hpages, int flags, char *desc, int line)
 {
+	unsigned long allowed_free = 0;
 	long et, ef, er, es;
+	int creates_reservation = mapping_creates_reservation(flags);
 
 	map_fd[s] = tst_creat_unlinked(MNTPOINT, 0, 0600);
 	map_size[s] = hpages * hpage_size;
+	if (creates_reservation)
+		allowed_free = read_allowed_free_hugepages();
 	map_addr[s] = SAFE_MMAP(NULL, map_size[s], PROT_READ|PROT_WRITE, flags,
 				map_fd[s], 0);
 	touched[s] = 0;
@@ -201,11 +302,9 @@ static int map_(int s, int hpages, int flags, char *desc, int line)
 	 * mappings behave like MAP_SHARED at mmap time.  Otherwise,
 	 * no counter updates will occur.
 	 */
-	if ((flags & MAP_SHARED) || private_resv) {
-		unsigned long shortfall = 0;
+	if (creates_reservation) {
+		unsigned long shortfall = surplus_pages_needed(hpages, allowed_free);
 
-		if (hpages + prev_resv > prev_free)
-			shortfall = hpages - prev_free + prev_resv;
 		et += shortfall;
 		ef += shortfall;
 		er += hpages;
-- 
2.52.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH v3 2/3] hugemmap10: Add private MAP_NORESERVE test mode

[Huaisheng Ye]

Add -n to apply MAP_NORESERVE only to private mappings. Shared
mappings retain their original flags and reservation behavior.

Use the private mapping flags for both the reservation probe and test
mappings. This makes private_resv and its TDEBUG report reflect whether
the mappings in use create a hugetlb reservation at mmap() time.

The option lets the test exercise the fault-time private reservation
path used by upstream kernels without booting an older kernel. Add
hugemmap10 -n to runtest/hugetlb so the default suite covers that path.

Use private_noreserve_opt directly as the option presence flag. Define
MAP_TYPE in lapi/mmap.h for libc implementations that lack it.

Signed-off-by: Huaisheng Ye <yehuaisheng@open-hieco.net>
---
 include/lapi/mmap.h                           |  4 +++
 runtest/hugetlb                               |  1 +
 .../kernel/mem/hugetlb/hugemmap/hugemmap10.c  | 27 ++++++++++++++++---
 3 files changed, 29 insertions(+), 3 deletions(-)

diff --git a/include/lapi/mmap.h b/include/lapi/mmap.h
index f20eb3bd5..12085ae27 100644
--- a/include/lapi/mmap.h
+++ b/include/lapi/mmap.h
@@ -14,6 +14,10 @@
 # define MAP_SHARED_VALIDATE 0x03
 #endif
 
+#ifndef MAP_TYPE
+# define MAP_TYPE 0x0f
+#endif
+
 #ifndef MAP_HUGETLB
 # define MAP_HUGETLB 0x40000
 #endif
diff --git a/runtest/hugetlb b/runtest/hugetlb
index 621c9718a..f5ffa7292 100644
--- a/runtest/hugetlb
+++ b/runtest/hugetlb
@@ -13,6 +13,7 @@ hugemmap07 hugemmap07
 hugemmap08 hugemmap08
 hugemmap09 hugemmap09
 hugemmap10 hugemmap10
+hugemmap10 hugemmap10 -n
 hugemmap11 hugemmap11
 hugemmap12 hugemmap12
 hugemmap13 hugemmap13
diff --git a/testcases/kernel/mem/hugetlb/hugemmap/hugemmap10.c b/testcases/kernel/mem/hugetlb/hugemmap/hugemmap10.c
index 1f8d00226..43aca50e1 100644
--- a/testcases/kernel/mem/hugetlb/hugemmap/hugemmap10.c
+++ b/testcases/kernel/mem/hugetlb/hugemmap/hugemmap10.c
@@ -24,6 +24,7 @@
 #include <sys/types.h>
 
 #include "hugetlb.h"
+#include "lapi/mmap.h"
 #include "lapi/syscalls.h"
 
 #define MNTPOINT "hugetlbfs/"
@@ -31,6 +32,7 @@
 
 static long hpage_size;
 static int private_resv;
+static char *private_noreserve_opt;
 
 #define NR_SLOTS	2
 #define SL_SETUP	0
@@ -90,6 +92,16 @@ static int mapping_creates_reservation(int flags)
 	return (flags & MAP_SHARED) || private_resv;
 }
 
+/* Exercise the fault-time reservation path used by older kernels. */
+static int private_mmap_flags(int flags)
+{
+	if (private_noreserve_opt &&
+	    (flags & MAP_TYPE) == MAP_PRIVATE)
+		return flags | MAP_NORESERVE;
+
+	return flags;
+}
+
 static unsigned long read_allowed_free_hugepages(void)
 {
 	char path[PATH_MAX];
@@ -155,7 +167,8 @@ static int kernel_has_private_reservations(void)
 	read_meminfo_huge(&t, &f, &r, &s);
 	fd = tst_creat_unlinked(MNTPOINT, 0, 0600);
 
-	p = SAFE_MMAP(NULL, hpage_size, PROT_READ|PROT_WRITE, MAP_PRIVATE, fd, 0);
+	p = SAFE_MMAP(NULL, hpage_size, PROT_READ|PROT_WRITE,
+		      private_mmap_flags(MAP_PRIVATE), fd, 0);
 
 	read_meminfo_huge(&nt, &nf, &nr, &ns);
 
@@ -283,8 +296,9 @@ static int map_(int s, int hpages, int flags, char *desc, int line)
 	map_size[s] = hpages * hpage_size;
 	if (creates_reservation)
 		allowed_free = read_allowed_free_hugepages();
-	map_addr[s] = SAFE_MMAP(NULL, map_size[s], PROT_READ|PROT_WRITE, flags,
-				map_fd[s], 0);
+	map_addr[s] = SAFE_MMAP(NULL, map_size[s], PROT_READ|PROT_WRITE,
+			private_mmap_flags(flags),
+			map_fd[s], 0);
 	touched[s] = 0;
 
 	et = prev_total;
@@ -534,6 +548,8 @@ static void setup(void)
 	hpage_size = SAFE_READ_MEMINFO(MEMINFO_HPAGE_SIZE)*1024;
 	SAFE_FILE_PRINTF(PATH_VM_OVERCOMMIT_HPAGES, "%lu", tst_hugepages);
 	private_resv = kernel_has_private_reservations();
+	tst_res(TDEBUG, "MAP_PRIVATE mmap() %s a hugetlb reservation",
+		private_resv ? "creates" : "does not create");
 }
 
 static void cleanup(void)
@@ -543,6 +559,11 @@ static void cleanup(void)
 
 static struct tst_test test = {
 	.needs_root = 1,
+	.options = (struct tst_option[]) {
+		{"n", &private_noreserve_opt,
+		 "Map private VMAs with MAP_NORESERVE"},
+		{}
+	},
 	.mntpoint = MNTPOINT,
 	.needs_hugetlbfs = 1,
 	.save_restore = (const struct tst_path_val[]) {
-- 
2.52.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH v3 3/3] hugemmap10: Account for memory policy in private fault checks

[Huaisheng Ye]

Kernels without private mmap-time reservations or with flags MAP_NORESERVE
create the reservation when the private VMA first faults. With MPOL_BIND,
globally free hugepages can be outside the allowed nodes, so this fault
can create surplus pages even when the global counters appear sufficient.

    $ numactl --cpunodebind=0 --membind=2 ./hugemmap10 -n
    tst_hugepage.c:84: TINFO: 3 hugepage(s) reserved
    tst_tmpdir.c:308: TINFO: Using /tmp/LTP_hugdeSTgL as tmpdir (xfs filesystem)
    tst_test.c:1218: TINFO: Mounting none to /tmp/LTP_hugdeSTgL/hugetlbfs fstyp=hugetlbfs flags=0
    tst_test.c:2048: TINFO: LTP version: 20260529-237-g132c8b0a8
    tst_test.c:2051: TINFO: Tested kernel: 7.3.0-rc1-test-32+ #33 SMP PREEMPT Mon Sep  7 11:50:02 CST 2026 x86_64
    tst_kconfig.c:90: TINFO: Parsing kernel config '/proc/config.gz'
    tst_test.c:1876: TINFO: Overall timeout per run is 0h 00m 30s
    hugemmap10.c:498: TINFO: Base pool size: 0
    hugemmap10.c:425: TINFO: Clean...
    hugemmap10.c:476: TINFO: OK
    hugemmap10.c:425: TINFO: Untouched, shared...
    hugemmap10.c:476: TINFO: OK
    hugemmap10.c:425: TINFO: Untouched, private...
    hugemmap10.c:476: TINFO: OK
    hugemmap10.c:425: TINFO: Touched, shared...
    hugemmap10.c:476: TINFO: OK
    hugemmap10.c:425: TINFO: Touched, private...
    hugemmap10.c:476: TINFO: OK
    hugemmap10.c:498: TINFO: Base pool size: 1
    hugemmap10.c:425: TINFO: Clean...
    hugemmap10.c:448: TFAIL: While touching the addr after mmap private: Bad HugePages_Total: expected 1, actual 2
    hugemmap10.c:448: TFAIL: While touching the addr after mmap private: Bad HugePages_Free: expected 0, actual 1
    hugemmap10.c:448: TFAIL: While touching the addr after mmap private: Bad HugePages_Surp: expected 0, actual 1

    Summary:
    passed   0
    failed   3
    broken   0
    skipped  0
    warnings 0

Get free hugepages on allowed nodes before touching an unreserved
private mapping and use the maximum of the global and allowed-node
shortfalls to calculate the expected counters.

Signed-off-by: Huaisheng Ye <yehuaisheng@open-hieco.net>
Reviewed-by: Li Wang <li.wang@linux.dev>
---
 .../kernel/mem/hugetlb/hugemmap/hugemmap10.c  | 21 ++++++++++++-------
 1 file changed, 13 insertions(+), 8 deletions(-)

diff --git a/testcases/kernel/mem/hugetlb/hugemmap/hugemmap10.c b/testcases/kernel/mem/hugetlb/hugemmap/hugemmap10.c
index 43aca50e1..b29c55a4f 100644
--- a/testcases/kernel/mem/hugetlb/hugemmap/hugemmap10.c
+++ b/testcases/kernel/mem/hugetlb/hugemmap/hugemmap10.c
@@ -363,7 +363,7 @@ static int unmap_(int s, int hpages, int flags, char *desc, int line)
 	 * reservation.  If those pages were not touched, then they will
 	 * not have been freed by the code above.  Free them here.
 	 */
-	if ((flags & MAP_SHARED) || private_resv) {
+	if (mapping_creates_reservation(flags)) {
 		int unused_surplus = MIN(hpages - touched[s], es);
 
 		et -= unused_surplus;
@@ -378,10 +378,16 @@ static int unmap_(int s, int hpages, int flags, char *desc, int line)
 
 static int touch_(int s, int hpages, int flags, char *desc, int line)
 {
+	unsigned long allowed_free = 0;
 	long et, ef, er, es;
+	int creates_reservation = mapping_creates_reservation(flags);
 	int nr;
 	char *c;
 
+	/* An unreserved private mapping creates its reservation on fault. */
+	if (!creates_reservation)
+		allowed_free = read_allowed_free_hugepages();
+
 	for (c = map_addr[s], nr = hpages;
 			hpages && c < map_addr[s] + map_size[s];
 			c += hpage_size, nr--)
@@ -403,19 +409,18 @@ static int touch_(int s, int hpages, int flags, char *desc, int line)
 	 * which could elevate total, free, and surplus counts.  resv is
 	 * unchanged but free must be decreased.
 	 */
-	if (flags & MAP_SHARED || private_resv) {
+	if (creates_reservation) {
 		et = prev_total;
 		ef = prev_free - hpages;
 		er = prev_resv - hpages;
 		es = prev_surp;
 	} else {
-		if (hpages + prev_resv > prev_free)
-			et = prev_total + (hpages - prev_free + prev_resv);
-		else
-			et = prev_total;
+		unsigned long shortfall = surplus_pages_needed(hpages, allowed_free);
+
+		et = prev_total + shortfall;
 		er = prev_resv;
-		es = prev_surp + et - prev_total;
-		ef = prev_free - hpages + et - prev_total;
+		es = prev_surp + shortfall;
+		ef = prev_free - hpages + shortfall;
 	}
 	return verify_counters(line, desc, et, ef, er, es);
 }
-- 
2.52.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] hugemmap10: Account for memory policy in mmap counter checks

[linuxtestproject.agent]

Hi Huaisheng,

On 2026-10-10, Huaisheng Ye wrote:
> hugemmap10: Account for memory policy in mmap counter checks

--- [PATCH 1/3] ---

> +	mode &= ~MPOL_MODE_FLAGS;
> +	/* Hugetlb ignores a bind policy that no longer overlaps the cpuset. */
> +	use_bind = mode == MPOL_BIND &&
> +		nodemasks_intersect(policy_nodemask, cpuset_nodemask, maxnode);

Medium: get_mempolicy() returns the original user mask for
MPOL_F_RELATIVE_NODES, not physical node IDs. With cpuset {1,2}, relative
bind {0}, and one free hugepage on node 2 only, this falls back to the
whole cpuset and predicts zero surplus although the effective bind node
1 needs one surplus page. Preserve the mode flags and resolve the
relative mask against allowed memory nodes, including folding, before
intersecting it or applying the fallback.

--- [PATCH 2/3] ---

>  hugemmap10 hugemmap10
> +hugemmap10 hugemmap10 -n

Medium: both variants use the same runtest identifier, which Kirk uses
for session restore and reboot recovery. If the default invocation
completes before an interruption, the pending -n invocation is also
considered completed and skipped. Give it a distinct identifier, for
example "hugemmap10_noreserve hugemmap10 -n".

--- [PATCH 3/3] ---

> +	/* An unreserved private mapping creates its reservation on fault. */

Low: MAP_NORESERVE faults allocate unreserved hugepages; they do not
create a hugetlb mapping reservation. hugetlb_reserve_pages() returns
before creating the private reservation map, and HugePages_Rsvd stays
unchanged, as this branch's er = prev_resv already expects. Describe this
as fault-time allocation in the comment and commit message.

Verdict - Needs revision

---
Note:

The agent can sometimes produce false positives although often its
findings are genuine. If you find issues with the review, please
comment this email or ignore the suggestions.

Regards,
LTP AI Reviewer

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp