[PATCH v2] mm/slab: reject unsupported kmalloc() sizes

[Zi Yan <ziy@nvidia.com>]

kmalloc() is used to allocate physically contiguous memory for kernel
allocations. For requests larger than KMALLOC_MAX_CACHE_SIZE, kmalloc()
uses the page allocator and can only support up to KMALLOC_MAX_SIZE. For
request sizes bigger than KMALLOC_MAX_SIZE, the page allocator can emit a
WARN because kmalloc() allocates an order greater than MAX_PAGE_ORDER. To
make the KMALLOC_MAX_SIZE limit more explicit instead of relying on the
page allocator check, add a KMALLOC_MAX_SIZE check inside kmalloc() to emit
a WARN and return NULL for requested sizes bigger than KMALLOC_MAX_SIZE.
Like the WARN in the page allocator and kvmalloc(), allow __GFP_NOWARN to
suppress the WARN.

Suggested-by: "Vlastimil Babka (SUSE)" <vbabka@kernel.org>
Signed-off-by: Zi Yan <ziy@nvidia.com>
---
Instead of passing requests greater than KMALLOC_MAX_SIZE to the page
allocator, reject them early inside kmalloc().
---
Changes in v2:
1. Rejected kmalloc() request sizes bigger than KMALLOC_MAX_SIZE by
   emitting a WARN and returning NULL inside kmalloc(). The WARN can be
   suppressed by __GFP_NOWARN. So the page allocator will no longer see
   excessively large requests from kmalloc().
- Link to v1: https://patch.msgid.link/20260817-limit_kmalloc_size-v1-1-5bef487701cc@nvidia.com
---
 mm/slub.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/mm/slub.c b/mm/slub.c
index f9b56cb439e70..23e57a8fe55b4 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -5342,7 +5342,12 @@ static void *___kmalloc_large_node(size_t size, gfp_t flags, int node)
 {
 	struct page *page;
 	void *ptr = NULL;
-	unsigned int order = get_order(size);
+	unsigned int order;
+
+	if (WARN_ON_ONCE_GFP(size > KMALLOC_MAX_SIZE, flags))
+		return NULL;
+
+	order = get_order(size);
 
 	if (unlikely(flags & GFP_SLAB_BUG_MASK))
 		flags = kmalloc_fix_flags(flags);

---
base-commit: cee9395acd8043be0644b25c34bfa86623f2b935
change-id: 20260817-limit_kmalloc_size-3a4a2c73beac

Best regards,
--  
Yan, Zi