From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from out-181.mta1.migadu.com (out-181.mta1.migadu.com [95.215.58.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 36FBB3D16FD for ; Thu, 18 Jun 2026 10:10:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.181 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781777423; cv=none; b=ZPXKAYr0ALvDAxjeL1WDVjnSACUcDus1IoPdvP4mynoYlc6oFQgiY5kkeQFPAOwTHfm0w3OAE9ZfpSmsbuYvkXfKkgoykCT6W0OrtdsB51r+DnDkKAgyXdFxDdGM1t4AYdgU7FN7CH6d9yud6L5x9mFZJy7UeC3m/Blit9TL4xY= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781777423; c=relaxed/simple; bh=TFW94jN7w6RlXSw0Hg1h5biKeoFl6+/U26J4ruZGZ2k=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=XkoXZVreeErwZMurK/UShyFRMrnvnzrvkT2MvA1osQ/vm154HeUWzipUV1rERcXck8WItUMr+PsbV7O7I0wS/aeSw3a+EvTCdXJjD7r+kwHxC6eNnNH/dZ7GykX0SweNgzf0FlPCI6RI2jMPVk/z5dltZKd3PMQQGPDaeTpB2g8= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=JYtY25+h; arc=none smtp.client-ip=95.215.58.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="JYtY25+h" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1781777416; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=l+HD+s8JWgfwWWOKKHpxeLeLnAP3UnMEfrdGWQ0SBAo=; b=JYtY25+hFIyt/k7UgYp+cbAS50ZBAiQWvZAI5iGFiNLn3NbSYmZ918QHKw0Y90jnhceWOG 5FuccA6vHyheCbfuBqi81src8GikA+9gi8LJA6LAAGbgTUAT200FHI7cD1uVtMZjDKrlK8 hMSWXalEasZbkpM/bDlmzUwPOfyYvs4= From: Hao Li To: vbabka@kernel.org, harry@kernel.org Cc: akpm@linux-foundation.org, cl@gentwo.org, rientjes@google.com, roman.gushchin@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Hao Li Subject: [PATCH] mm/slub: deduplicate NUMA policy calculation in allocation paths Date: Thu, 18 Jun 2026 18:08:52 +0800 Message-ID: <20260618100913.346636-1-hao.li@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT Currently, alloc_from_pcs() and __slab_alloc_node() both calculate the NUMA policy independently. Since they are called consecutively in paths like __kmalloc_nolock_noprof() and slab_alloc_node(), this leads to redundant computations. Introduce a helper function to resolve the NUMA policy once, eliminating the duplicated code and reducing execution overhead. Signed-off-by: Hao Li --- mm/slub.c | 72 ++++++++++++++++++++++--------------------------------- 1 file changed, 29 insertions(+), 43 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index 62e9cd46916f..45e9f379b7da 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -4523,32 +4523,36 @@ static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, return object; } +static __always_inline int apply_numa_policy(int node) +{ +#ifdef CONFIG_NUMA + if (static_branch_unlikely(&strict_numa) && + node == NUMA_NO_NODE) { + + struct mempolicy *mpol = current->mempolicy; + + if (mpol) { + /* + * Special BIND rule support. If the local node + * is in permitted set then do not redirect + * to a particular node. + * Otherwise we apply the memory policy to get + * the node we need to allocate on. + */ + if (mpol->mode != MPOL_BIND || + !node_isset(numa_mem_id(), mpol->nodes)) + node = mempolicy_slab_node(); + } + } +#endif + return node; +} + static void *__slab_alloc_node(struct kmem_cache *s, gfp_t gfpflags, int node, const struct slab_alloc_context *ac) { void *object; -#ifdef CONFIG_NUMA - if (static_branch_unlikely(&strict_numa) && - node == NUMA_NO_NODE) { - - struct mempolicy *mpol = current->mempolicy; - - if (mpol) { - /* - * Special BIND rule support. If the local node - * is in permitted set then do not redirect - * to a particular node. - * Otherwise we apply the memory policy to get - * the node we need to allocate on. - */ - if (mpol->mode != MPOL_BIND || - !node_isset(numa_mem_id(), mpol->nodes)) - node = mempolicy_slab_node(); - } - } -#endif - object = ___slab_alloc(s, gfpflags, node, ac); return object; @@ -4756,28 +4760,6 @@ void *alloc_from_pcs(struct kmem_cache *s, gfp_t gfp, unsigned int alloc_flags, bool node_requested; void *object; -#ifdef CONFIG_NUMA - if (static_branch_unlikely(&strict_numa) && - node == NUMA_NO_NODE) { - - struct mempolicy *mpol = current->mempolicy; - - if (mpol) { - /* - * Special BIND rule support. If the local node - * is in permitted set then do not redirect - * to a particular node. - * Otherwise we apply the memory policy to get - * the node we need to allocate on. - */ - if (mpol->mode != MPOL_BIND || - !node_isset(numa_mem_id(), mpol->nodes)) - - node = mempolicy_slab_node(); - } - } -#endif - node_requested = IS_ENABLED(CONFIG_NUMA) && node != NUMA_NO_NODE; /* @@ -4927,6 +4909,8 @@ static __fastpath_inline void *slab_alloc_node(struct kmem_cache *s, if (unlikely(object)) goto out; + node = apply_numa_policy(node); + object = alloc_from_pcs(s, gfpflags, ac->alloc_flags, node); if (unlikely(!object)) @@ -5430,6 +5414,8 @@ static void *__kmalloc_nolock_noprof(DECL_TOKEN_PARAMS(size, token), gfp_t gfp_f */ return NULL; + node = apply_numa_policy(node); + ret = alloc_from_pcs(s, gfp_flags, ac->alloc_flags, node); if (ret) goto success; -- 2.54.0