Re: [PATCH v3 06/21] slab: introduce percpu sheaves bootstrap

Vlastimil Babka
2026-01-19T09:34:09Z

On 1/17/26 03:11, Suren Baghdasaryan wrote:
> On Fri, Jan 16, 2026 at 2:40 PM Vlastimil Babka <vbabka@suse.cz> wrote:
>> Thus sharing the single bootstrap sheaf like this for multiple caches
>> and cpus is safe.
>>
>> Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
>> ---
>> mm/slub.c | 119 ++++++++++++++++++++++++++++++++++++++++++--------------------
>> 1 file changed, 81 insertions(+), 38 deletions(-)
>>
>> diff --git a/mm/slub.c b/mm/slub.c
>> index edf341c87e20..706cb6398f05 100644
>> --- a/mm/slub.c
>> +++ b/mm/slub.c
>> @@ -501,6 +501,18 @@ struct kmem_cache_node {
>> struct node_barn *barn;
>> };
>>
>> +/*
>> + * Every cache has !NULL s->cpu_sheaves but they may point to the
>> + * bootstrap_sheaf temporarily during init, or permanently for the boot caches
>> + * and caches with debugging enabled, or all caches with CONFIG_SLUB_TINY. This
>> + * helper distinguishes whether cache has real non-bootstrap sheaves.
>> + */
>> +static inline bool cache_has_sheaves(struct kmem_cache *s)
>> +{
>> + /* Test CONFIG_SLUB_TINY for code elimination purposes */
>> + return !IS_ENABLED(CONFIG_SLUB_TINY) && s->sheaf_capacity;
>> +}
>> +
>> static inline struct kmem_cache_node *get_node(struct kmem_cache *s, int node)
>> {
>> return s->node[node];
>> @@ -2855,6 +2867,10 @@ static void pcs_destroy(struct kmem_cache *s)
>> if (!pcs->main)
>> continue;
>>
>> + /* bootstrap or debug caches, it's the bootstrap_sheaf */
>> + if (!pcs->main->cache)
>> + continue;
>
> I wonder why we can't simply check cache_has_sheaves(s) at the
> beginning and skip the loop altogether.
> I realize that __kmem_cache_release()->pcs_destroy() is called in the
> failure path of do_kmem_cache_create() and s->cpu_sheaves might be
> partially initialized if alloc_empty_sheaf() fails somewhere in the
> middle of the loop inside init_percpu_sheaves(). But for that,
> s->sheaf_capacity should still be non-zero, so checking
> cache_has_sheaves() at the beginning of pcs_destroy() should still
> work, no?

I think it should, will do.

> BTW, I see one last check for s->cpu_sheaves that you didn't replace
> with cache_has_sheaves() inside __kmem_cache_release(). I think that's
> because it's also in the failure path of do_kmem_cache_create() and
> it's possible that s->sheaf_capacity > 0 while s->cpu_sheaves == NULL
> (if alloc_percpu(struct slub_percpu_sheaves) fails). It might be
> helpful to add a comment inside __kmem_cache_release() to explain why
> cache_has_sheaves() can't be used there.

The reason is rather what Harry said. I'll move the check to pcs_destroy()
and add comment there.

diff --git a/mm/slub.c b/mm/slub.c
index 706cb6398f05..6b19aa518a1a 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -2858,19 +2858,26 @@ static void pcs_destroy(struct kmem_cache *s)
{
int cpu;

+ /*
+ * We may be unwinding cache creation that failed before or during the
+ * allocation of this.
+ */
+ if (!s->cpu_sheaves)
+ return;
+
+ /* pcs->main can only point to the bootstrap sheaf, nothing to free */
+ if (!cache_has_sheaves(s))
+ goto free_pcs;
+
for_each_possible_cpu(cpu) {
struct slub_percpu_sheaves *pcs;

pcs = per_cpu_ptr(s->cpu_sheaves, cpu);

- /* can happen when unwinding failed create */
+ /* This can happen when unwinding failed cache creation. */
if (!pcs->main)
continue;

- /* bootstrap or debug caches, it's the bootstrap_sheaf */
- if (!pcs->main->cache)
- continue;
-
/*
* We have already passed __kmem_cache_shutdown() so everything
* was flushed and there should be no objects allocated from
@@ -2889,6 +2896,7 @@ static void pcs_destroy(struct kmem_cache *s)
}
}

+free_pcs:
free_percpu(s->cpu_sheaves);
s->cpu_sheaves = NULL;
}
@@ -5379,6 +5387,9 @@ kmem_cache_prefill_sheaf(struct kmem_cache *s, gfp_t gfp, unsigned int size)
struct slab_sheaf *sheaf = NULL;
struct node_barn *barn;

+ if (unlikely(!size))
+ return NULL;
+
if (unlikely(size > s->sheaf_capacity)) {

sheaf = kzalloc(struct_size(sheaf, objects, size), gfp);
@@ -7833,8 +7844,7 @@ static void free_kmem_cache_nodes(struct kmem_cache *s)
void __kmem_cache_release(struct kmem_cache *s)
{
cache_random_seq_destroy(s);
- if (s->cpu_sheaves)
- pcs_destroy(s);
+ pcs_destroy(s);
#ifdef CONFIG_PREEMPT_RT
if (s->cpu_slab)
lockdep_unregister_key(&s->lock_key);