From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C93D83F9292 for ; Fri, 15 May 2026 16:24:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778862281; cv=none; b=rFlg3LJLkF+wSr2BV/kc2PKZoCaOIKqoUmnCnf0/h7E7GGXgyt/SFzmU+LlDpxcNTSorqujvwdXTNgP7uvIcgBG0H+JrMe19cZo2VRN6JHpOwAJpax4vmua4ai9N43YKzj34gOjdAsL2k8pig/0xRkV7vj4HwDRrT42iWXiImvM= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778862281; c=relaxed/simple; bh=EYPxp3BS503vPYeqgCva7KZMCxYvgQlPQAcZTIN2fQA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=dRQ+kddAM4ridlDK74kUOl0SG6pcrQ/HVzVk1Aj4IcFRBphDdoMKc/uQ07z2oKpBQ8qGj/Adli2p2H/qUl7ZoiZbIe0dQRjGQZE59fkMzxQu4ZNI+yV9ivRdRkyVMmiQ4mz4XT7CGIoU0cWCIYEAMOTnLgC9GVcF0jghTXefS6M= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=NHj4PGGj; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="NHj4PGGj" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B05EFC2BCB3; Fri, 15 May 2026 16:24:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1778862281; bh=EYPxp3BS503vPYeqgCva7KZMCxYvgQlPQAcZTIN2fQA=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=NHj4PGGjWVqj4heBlOwpnCF8Lu2CxX7rp0GRoOUwVffhmaROBnA665SPrxppjDVyh S8tr9O1BkG1+pHvszW1CKnFNhQZFb70OkMgKicuWRsnEd/Egz24qMPdCku9T87mAcr iNGRZsUQJOhO/PuutCm9BSfBdJe8+Qo5Ek+VwMXMxpesqmVyGwZin7jvVJ3wRH1s/B bTtyL1I5hMwY+O8k9Vuk6JvZliEOre0/CnWysb1XkngTVhZHwmFSFA8gqDaS4qGlKg KLQNhFUCaZj0d5d9vWusw4b+jsMG8jP4flA+Nwg9+xP6ZzLcfpFAXF0Eqm0TOYyNdt 7FOGiwx/9iG+A== From: "Harry Yoo (Oracle)" Date: Sat, 16 May 2026 01:24:25 +0900 Subject: [PATCH RFC 1/8] mm/slab: do not store cache pointer in struct slab_sheaf Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20260516-sheaves-tuning-v1-1-221aa3e1d829@kernel.org> References: <20260516-sheaves-tuning-v1-0-221aa3e1d829@kernel.org> In-Reply-To: <20260516-sheaves-tuning-v1-0-221aa3e1d829@kernel.org> To: Vlastimil Babka , Andrew Morton , Hao Li , Christoph Lameter , David Rientjes , Roman Gushchin Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Suren Baghdasaryan , "Liam R. Howlett" X-Mailer: b4 0.16-dev The `cache` field of struct slab_sheaf is only read on the slow path when freeing an RCU sheaf. Storing it in every sheaf is an overkill. Drop the field. In rcu_free_sheaf() and rcu_free_sheaf_nobarn(), fetch the kmem_cache pointer via virt_to_slab(sheaf->objects[0])->slab_cache instead. As sheaf is only attached to pcs->rcu_free once it holds at least one object, the lookup is safe. Add a WARN_ON_ONCE() in case an empty sheaf ever reaches the RCU free path. In that case, the cache is unknown, so free_empty_sheaf() now tolerates a NULL cache argument. However, the case is never expected to trigger. While at it, remove the stale comment in init_percpu_sheaves(). Signed-off-by: Harry Yoo (Oracle) --- mm/slub.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index 5ef54d546bc2..75281eb802de 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -422,7 +422,6 @@ struct slab_sheaf { bool pfmemalloc; }; }; - struct kmem_cache *cache; unsigned int size; int node; /* only used for rcu_sheaf */ void *objects[]; @@ -2781,8 +2780,6 @@ static struct slab_sheaf *__alloc_empty_sheaf(struct kmem_cache *s, gfp_t gfp, if (unlikely(!sheaf)) return NULL; - sheaf->cache = s; - stat(s, SHEAF_ALLOC); return sheaf; @@ -2802,13 +2799,14 @@ static void free_empty_sheaf(struct kmem_cache *s, struct slab_sheaf *sheaf) * warning, therefore replace NULL with CODETAG_EMPTY to indicate * that the extension for this sheaf is expected to be NULL. */ - if (s->flags & SLAB_KMALLOC) + if (s && (s->flags & SLAB_KMALLOC)) mark_obj_codetag_empty(sheaf); VM_WARN_ON_ONCE(sheaf->size > 0); kfree(sheaf); - stat(s, SHEAF_FREE); + if (s) + stat(s, SHEAF_FREE); } static unsigned int @@ -2968,12 +2966,15 @@ static void rcu_free_sheaf_nobarn(struct rcu_head *head) struct kmem_cache *s; sheaf = container_of(head, struct slab_sheaf, rcu_head); - s = sheaf->cache; + if (WARN_ON_ONCE(!sheaf->size)) { + free_empty_sheaf(NULL, sheaf); + return; + } + s = virt_to_slab(sheaf->objects[0])->slab_cache; __rcu_free_sheaf_prepare(s, sheaf); sheaf_flush_unused(s, sheaf); - free_empty_sheaf(s, sheaf); } @@ -5019,7 +5020,6 @@ kmem_cache_prefill_sheaf(struct kmem_cache *s, gfp_t gfp, unsigned int size) return NULL; stat(s, SHEAF_PREFILL_OVERSIZE); - sheaf->cache = s; sheaf->capacity = size; /* @@ -5873,8 +5873,12 @@ static void rcu_free_sheaf(struct rcu_head *head) struct kmem_cache *s; sheaf = container_of(head, struct slab_sheaf, rcu_head); + if (WARN_ON_ONCE(!sheaf->size)) { + free_empty_sheaf(NULL, sheaf); + return; + } - s = sheaf->cache; + s = virt_to_slab(sheaf->objects[0])->slab_cache; /* * This may remove some objects due to slab_free_hook() returning false, @@ -7616,10 +7620,6 @@ static int init_percpu_sheaves(struct kmem_cache *s) * It's also safe to share the single static bootstrap_sheaf * with zero-sized objects array as it's never modified. * - * Bootstrap_sheaf also has NULL pointer to kmem_cache so we - * recognize it and not attempt to free it when destroying the - * cache. - * * We keep bootstrap_sheaf for kmem_cache and kmem_cache_node, * caches with debug enabled, and all caches with SLUB_TINY. * For kmalloc caches it's used temporarily during the initial -- 2.43.0