All of lore.kernel.org
 help / color / mirror / Atom feed
* [LTP] [PATCH v2 0/3] hugemmap10: Account for memory policy in counter checks
@ 2026-09-29  7:24 Huaisheng Ye
  2026-09-29  7:24 ` [LTP] [PATCH v2 1/3] hugemmap10: Account for memory policy in mmap " Huaisheng Ye
                   ` (2 more replies)
  0 siblings, 3 replies; 9+ messages in thread
From: Huaisheng Ye @ 2026-09-29  7:24 UTC (permalink / raw)
  To: li.wang, chrubis, pvorel; +Cc: ltp

The hugetlb reservation fix allocates surplus hugepages when global free
pages are outside nodes allowed by the current cpuset and MPOL_BIND policy.
This series updates hugemmap10's global counter expectations to match that
behavior.

Patch 1 fixes mappings that create their reservation at mmap() time. Patches
2 and 3 add and validate a MAP_NORESERVE private-mapping mode, which lets the
test exercise fault-time reservations without booting an older kernel.

Changes in v2:
- Expand the v1 change[1] from mmap-time to private fault-time accounting
  patches, with the MAP_NORESERVE test mode between them.
- Match hugetlb's MPOL_BIND behavior when its policy mask no longer overlaps
  the current cpuset: fall back to the cpuset mask instead of treating the
  allowed set as empty.
- Normalize nodemask bit tests to avoid truncating bits in the upper half of
  an unsigned long on 64-bit systems.
- Add -n to map only private VMAs with MAP_NORESERVE. The reservation probe
  uses the same flags, so the private_resv debug report reflects the actual
  private mapping behavior. MAP_SHARED mappings are unchanged.
- Account for the allowed-node shortfall when an unreserved private mapping
  first faults in touch_(), and share the reservation predicate with unmap_().

[1] https://lore.kernel.org/ltp/20260914032359.9956-1-yehuaisheng@open-hieco.net/

Huaisheng Ye (3):
  hugemmap10: Account for memory policy in mmap counter checks
  hugemmap10: Add private MAP_NORESERVE test mode
  hugemmap10: Account for memory policy in private fault checks

 .../kernel/mem/hugetlb/hugemmap/hugemmap10.c  | 156 ++++++++++++++++--
 1 file changed, 141 insertions(+), 15 deletions(-)

-- 
2.52.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 9+ messages in thread
* [LTP] [PATCH v3 1/3] hugemmap10: Account for memory policy in mmap counter checks
@ 2026-10-10  9:51 Huaisheng Ye
  2026-10-10 15:50 ` [LTP] " linuxtestproject.agent
  0 siblings, 1 reply; 9+ messages in thread
From: Huaisheng Ye @ 2026-10-10  9:51 UTC (permalink / raw)
  To: li.wang, chrubis, pvorel; +Cc: ltp

Expected surplus accounting for mappings that reserve at mmap() time only
considers the global free and reserved hugepage counters. But sometimes,
for instance with MPOL_BIND, globally free hugepages can reside outside
nodes from which the task can allocate or plans to allocate. Hugetlb
then creates surplus pages on allowed nodes, while the test expects
no counter change.

For example, reproduce this on an eight-node system running 7.3.0-rc1:

    $ numactl --membind=0-3 ./hugemmap10
    tst_hugepage.c:84: TINFO: 3 hugepage(s) reserved
    ...
    hugemmap10.c:386: TINFO: Base pool size: 0
    hugemmap10.c:313: TINFO: Clean...
    hugemmap10.c:364: TINFO: OK
    hugemmap10.c:313: TINFO: Untouched, shared...
    hugemmap10.c:364: TINFO: OK
    hugemmap10.c:313: TINFO: Untouched, private...
    hugemmap10.c:364: TINFO: OK
    hugemmap10.c:313: TINFO: Touched, shared...
    hugemmap10.c:364: TINFO: OK
    hugemmap10.c:313: TINFO: Touched, private...
    hugemmap10.c:364: TINFO: OK
    hugemmap10.c:386: TINFO: Base pool size: 1
    hugemmap10.c:313: TINFO: Clean...
    hugemmap10.c:319: TFAIL: While doing mmap shared with no touch: Bad HugePages_Total: expected 1, actual 2
    hugemmap10.c:319: TFAIL: While doing mmap shared with no touch: Bad HugePages_Free: expected 1, actual 2
    hugemmap10.c:319: TFAIL: While doing mmap shared with no touch: Bad HugePages_Surp: expected 0, actual 1

    Summary:
    passed   0
    failed   3
    broken   0
    skipped  0
    warnings 0

    $ numactl --membind=4-7 ./hugemmap10
    ...
    hugemmap10.c:364: TINFO: OK
    hugemmap10.c:429: TPASS: Hugepages Counters works as expected.

    Summary:
    passed   1
    failed   0
    broken   0
    skipped  0
    warnings 0

If global free hugepages are on nodes 4-7 while the memory policy restricts
hugemmap10 to nodes 0-3, the counter discrepancy is observed.

Read the task policy and cpuset mask, then sum free hugepages on the
nodes available to the mapping. Calculate the expected surplus as the
maximum of the global and allowed-node shortfalls. Match hugetlb's fallback
to the cpuset when an MPOL_BIND mask no longer intersects it.

This fixes counter expectations for MAP_SHARED and MAP_PRIVATE mappings
that create reservations at mmap() time.

This complements the hugetlb reservation fix:
https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git/commit/?id=21f514ad520459fbfa58ccdf171bb68148c2f5aa

Signed-off-by: Huaisheng Ye <yehuaisheng@open-hieco.net>
Reviewed-by: Li Wang <li.wang@linux.dev>
---
 .../kernel/mem/hugetlb/hugemmap/hugemmap10.c  | 107 +++++++++++++++++-
 1 file changed, 103 insertions(+), 4 deletions(-)

diff --git a/testcases/kernel/mem/hugetlb/hugemmap/hugemmap10.c b/testcases/kernel/mem/hugetlb/hugemmap/hugemmap10.c
index fc530dda4..1f8d00226 100644
--- a/testcases/kernel/mem/hugetlb/hugemmap/hugemmap10.c
+++ b/testcases/kernel/mem/hugetlb/hugemmap/hugemmap10.c
@@ -13,6 +13,9 @@
  */
 
 #define _GNU_SOURCE
+#include <errno.h>
+#include <linux/mempolicy.h>
+#include <stdlib.h>
 #include <unistd.h>
 #include <stdio.h>
 #include <sys/mount.h>
@@ -21,8 +24,10 @@
 #include <sys/types.h>
 
 #include "hugetlb.h"
+#include "lapi/syscalls.h"
 
 #define MNTPOINT "hugetlbfs/"
+#define ULONG_BITS (sizeof(unsigned long) * CHAR_BIT)
 
 static long hpage_size;
 static int private_resv;
@@ -48,6 +53,98 @@ static void read_meminfo_huge(long *total, long *free, long *resv, long *surp)
 	*surp = SAFE_READ_MEMINFO(MEMINFO_HPAGE_SURP);
 }
 
+static int node_isset(unsigned long node, const unsigned long *nodemask)
+{
+	return !!(nodemask[node / ULONG_BITS] & (1UL << (node % ULONG_BITS)));
+}
+
+static int nodemasks_intersect(const unsigned long *left,
+			       const unsigned long *right, unsigned long maxnode)
+{
+	unsigned long node;
+
+	for (node = 0; node < maxnode; node++) {
+		if (node_isset(node, left) && node_isset(node, right))
+			return 1;
+	}
+
+	return 0;
+}
+
+static unsigned long surplus_pages_needed(int hpages, unsigned long allowed_free)
+{
+	unsigned long global_shortfall = 0;
+	unsigned long allowed_shortfall = 0;
+
+	if (hpages + prev_resv > prev_free)
+		global_shortfall = hpages - prev_free + prev_resv;
+
+	if ((unsigned long)hpages > allowed_free)
+		allowed_shortfall = hpages - allowed_free;
+
+	return MAX(global_shortfall, allowed_shortfall);
+}
+
+static int mapping_creates_reservation(int flags)
+{
+	return (flags & MAP_SHARED) || private_resv;
+}
+
+static unsigned long read_allowed_free_hugepages(void)
+{
+	char path[PATH_MAX];
+	size_t mask_size = getpagesize();
+	unsigned long allowed_free = 0;
+	unsigned long maxnode = mask_size * CHAR_BIT;
+	unsigned long *policy_nodemask;
+	unsigned long *cpuset_nodemask;
+	unsigned long node, node_free;
+	int mode, use_bind;
+
+	policy_nodemask = SAFE_CALLOC(1, mask_size);
+	cpuset_nodemask = SAFE_CALLOC(1, mask_size);
+
+	if (syscall(__NR_get_mempolicy, &mode, policy_nodemask,
+		    maxnode, NULL, 0)) {
+		if (errno == ENOSYS) {
+			allowed_free = prev_free;
+			goto out;
+		}
+		tst_brk(TBROK | TERRNO, "get_mempolicy() failed");
+	}
+
+	if (syscall(__NR_get_mempolicy, NULL, cpuset_nodemask,
+		    maxnode, NULL, MPOL_F_MEMS_ALLOWED))
+		tst_brk(TBROK | TERRNO,
+			"get_mempolicy(MPOL_F_MEMS_ALLOWED) failed");
+
+	mode &= ~MPOL_MODE_FLAGS;
+	/* Hugetlb ignores a bind policy that no longer overlaps the cpuset. */
+	use_bind = mode == MPOL_BIND &&
+		nodemasks_intersect(policy_nodemask, cpuset_nodemask, maxnode);
+	for (node = 0; node < maxnode; node++) {
+		if (!node_isset(node, cpuset_nodemask))
+			continue;
+		if (use_bind && !node_isset(node, policy_nodemask))
+			continue;
+
+		snprintf(path, sizeof(path),
+			 "/sys/devices/system/node/node%lu/hugepages/"
+			 "hugepages-%ldkB/free_hugepages",
+			 node, hpage_size / 1024);
+		if (access(path, R_OK))
+			continue;
+
+		SAFE_FILE_SCANF(path, "%lu", &node_free);
+		allowed_free += node_free;
+	}
+
+out:
+	free(policy_nodemask);
+	free(cpuset_nodemask);
+	return allowed_free;
+}
+
 static int kernel_has_private_reservations(void)
 {
 	int fd;
@@ -178,10 +275,14 @@ out:
 
 static int map_(int s, int hpages, int flags, char *desc, int line)
 {
+	unsigned long allowed_free = 0;
 	long et, ef, er, es;
+	int creates_reservation = mapping_creates_reservation(flags);
 
 	map_fd[s] = tst_creat_unlinked(MNTPOINT, 0, 0600);
 	map_size[s] = hpages * hpage_size;
+	if (creates_reservation)
+		allowed_free = read_allowed_free_hugepages();
 	map_addr[s] = SAFE_MMAP(NULL, map_size[s], PROT_READ|PROT_WRITE, flags,
 				map_fd[s], 0);
 	touched[s] = 0;
@@ -201,11 +302,9 @@ static int map_(int s, int hpages, int flags, char *desc, int line)
 	 * mappings behave like MAP_SHARED at mmap time.  Otherwise,
 	 * no counter updates will occur.
 	 */
-	if ((flags & MAP_SHARED) || private_resv) {
-		unsigned long shortfall = 0;
+	if (creates_reservation) {
+		unsigned long shortfall = surplus_pages_needed(hpages, allowed_free);
 
-		if (hpages + prev_resv > prev_free)
-			shortfall = hpages - prev_free + prev_resv;
 		et += shortfall;
 		ef += shortfall;
 		er += hpages;
-- 
2.52.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 9+ messages in thread
end of thread, other threads:[~2026-10-10 15:50 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-09-29  7:24 [LTP] [PATCH v2 0/3] hugemmap10: Account for memory policy in counter checks Huaisheng Ye
2026-09-29  7:24 ` [LTP] [PATCH v2 1/3] hugemmap10: Account for memory policy in mmap " Huaisheng Ye
2026-09-29 11:18   ` [LTP] " linuxtestproject.agent
2026-10-08  8:26   ` [LTP] [PATCH v2 1/3] " Li Wang
2026-09-29  7:24 ` [LTP] [PATCH v2 2/3] hugemmap10: Add private MAP_NORESERVE test mode Huaisheng Ye
2026-10-08  9:09   ` Li Wang
2026-09-29  7:24 ` [LTP] [PATCH v2 3/3] hugemmap10: Account for memory policy in private fault checks Huaisheng Ye
2026-10-08  9:40   ` Li Wang
  -- strict thread matches above, loose matches on Subject: below --
2026-10-10  9:51 [LTP] [PATCH v3 1/3] hugemmap10: Account for memory policy in mmap counter checks Huaisheng Ye
2026-10-10 15:50 ` [LTP] " linuxtestproject.agent

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.