From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from out-171.mta0.migadu.com (out-171.mta0.migadu.com [91.218.175.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2B5E431E83C for ; Mon, 25 May 2026 08:23:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.171 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779697429; cv=none; b=QcH0ct9I2BIhz3TZHTkcMBoDYe7hLvCQS4sHGuyCO/sP6sTEOKReBRlje2eL3EPRR9XeXcL600iHrWNmkEpR+1/0SeOLPTah/AEsJxHrStaMzxeJcpS7CKf8RkdrmhRpn6gOmOrfBV75q6k/MqKiB22M9q6shVswNLwQnl6ly5g= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779697429; c=relaxed/simple; bh=HTp1fwACvvffUwhKUT3mlvi95hBFQwg4O94OwI+/CQ0=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=fJ02wtxzssfqNgdi3eeSuQ2G5W3CWRGjxk9/aDid8HIMAilymeqUz+UZR8nxswJPm4UWPQGHsAhiAKDx1LPSmE8IwYbLZRlvy/+/lPo1U7dWNq18lXbGX1LcUys3L8NPg4CQ4W601mT1jU7nzzptSBkoWN6q4S7PI3pqC6mat+I= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=wcq/tjVp; arc=none smtp.client-ip=91.218.175.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="wcq/tjVp" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1779697425; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=HFN1D7/+D81myf4e4MmmURImtWjMJ87+3r+yuGuUTwM=; b=wcq/tjVpC7qyKukhUCzKmt24dmcGpCU2eHtWHhmC9lHTPX4W/9rVBHOZSbrsWmaV79yxie h+1iAoj1jkZ7Yrnr9TS+kLhgn+0TY9K1+F3/zFXf2nT6R0zqAebAOEcspRqoCjwYEPrRWk AmTsrax+o15N9nz65XiAr0V38z/JzWM= From: Hao Li To: vbabka@kernel.org, harry@kernel.org, akpm@linux-foundation.org Cc: cl@gentwo.org, rientjes@google.com, roman.gushchin@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Hao Li Subject: [PATCH] mm/slub: allocate sheaves on local memory nodes Date: Mon, 25 May 2026 16:13:47 +0800 Message-ID: <20260525082312.16012-1-hao.li@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT Sheaves are per-CPU allocator metadata and their object arrays are accessed from the local fast paths. Allocate them with a NUMA node hint instead of using plain kzalloc(). While no measurable performance improvement was observed, this approach is theoretically correct. During bootstrap we allocate sheaves for all possible CPUs before every possible CPU has an initialized cpu_to_mem() value, so compute the memory node from local_memory_node(cpu_to_node(cpu)) just like what __build_all_zonelists does. Signed-off-by: Hao Li --- This patch might conflict with Shengming's patch. Let's review it first, and if it looks good, I'll rebase afterwards. Thanks. mm/slub.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index 180973a4a3d2..ff1b1e932719 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -2757,7 +2757,7 @@ static inline void *setup_object(struct kmem_cache *s, void *object) } static struct slab_sheaf *__alloc_empty_sheaf(struct kmem_cache *s, gfp_t gfp, - unsigned int capacity) + unsigned int capacity, int node) { struct slab_sheaf *sheaf; size_t sheaf_size; @@ -2776,7 +2776,7 @@ static struct slab_sheaf *__alloc_empty_sheaf(struct kmem_cache *s, gfp_t gfp, gfp |= __GFP_NO_OBJ_EXT; sheaf_size = struct_size(sheaf, objects, capacity); - sheaf = kzalloc(sheaf_size, gfp); + sheaf = kzalloc_node(sheaf_size, gfp, node); if (unlikely(!sheaf)) return NULL; @@ -2791,7 +2791,7 @@ static struct slab_sheaf *__alloc_empty_sheaf(struct kmem_cache *s, gfp_t gfp, static inline struct slab_sheaf *alloc_empty_sheaf(struct kmem_cache *s, gfp_t gfp) { - return __alloc_empty_sheaf(s, gfp, s->sheaf_capacity); + return __alloc_empty_sheaf(s, gfp, s->sheaf_capacity, numa_mem_id()); } static void free_empty_sheaf(struct kmem_cache *s, struct slab_sheaf *sheaf) @@ -8413,10 +8413,17 @@ static void __init bootstrap_cache_sheaves(struct kmem_cache *s) for_each_possible_cpu(cpu) { struct slub_percpu_sheaves *pcs; + int mem_node; pcs = per_cpu_ptr(s->cpu_sheaves, cpu); - pcs->main = __alloc_empty_sheaf(s, GFP_KERNEL, capacity); + /* + * Cannot use cpu_to_mem() here because it's only initialized + * for online CPUs at this point (see __build_all_zonelists), + * while we need to allocate sheaves for all possible CPUs. + */ + mem_node = local_memory_node(cpu_to_node(cpu)); + pcs->main = __alloc_empty_sheaf(s, GFP_KERNEL, capacity, mem_node); if (!pcs->main) { failed = true; -- 2.54.0