Linux-mm Archive on lore.kernel.org
 help / color / mirror / Atom feed
From: "Vlastimil Babka (SUSE)" <vbabka@kernel.org>
To: Breno Leitao <leitao@debian.org>, Harry Yoo <harry@kernel.org>,
	Hao Li <hao.li@linux.dev>, Christoph Lameter <cl@gentwo.org>,
	David Rientjes <rientjes@google.com>,
	Roman Gushchin <roman.gushchin@linux.dev>,
	Andrew Morton <akpm@linux-foundation.org>
Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org,
	kernel-team@meta.com, Catalin Marinas <catalin.marinas@arm.com>
Subject: Re: [PATCH v2] slab: allow sheaves for SLAB_NOLEAKTRACE caches
Date: Mon, 16 Nov 2026 11:47:09 +0100	[thread overview]
Message-ID: <43dde2e9-24a7-4e09-ba07-d1166963e16b@kernel.org> (raw)
In-Reply-To: <20261113-kmemleak_sheaves-v2-1-8c557c66cec2@debian.org>

On 11/13/26 15:36, Breno Leitao wrote:
> This is a patch that makes kmemleak cheaper to run, by letting
> kmemleak's own metadata cache use a _new_ sheaves type. The flag only
> makes that possible without recursion.
> 
> The cost today:
>   * every allocation kmemleak tracks needs one kmemleak_object from
>     object_cache. An RCU callback frees it later.
> 
> Why that's slow:
>   * object_cache has no sheaves, so each of those allocations and frees
>     goes to the shared slab lists under n->list_lock.
>      - On meta's fleet kmemleak kernels, list_lock is the second most
>        contended lock, after kmemleak_lock.
>      - About 88% of that wait is RCU callbacks freeing kmemleak objects.
> 
> Why sheaves aren't allowed:
>   * a sheaf is a kmalloc object. Tracking it makes kmemleak allocate
>     another kmemleak_object, which needs a sheaf, and so on.
> 
> When I reported the recursion in [1], I suggested keeping
> SLAB_NOLEAKTRACE caches away from sheaves, and commit e47c897a2949
> ("slab: add sheaves to most caches") did that. It stopped the
> recursion, but it took the sheaves away from the whole cache, although
> only the kmemleak tracking of the sheaf itself recursed.
> 
> Commit 17c38c88294d ("slab: remove cpu (partial) slabs usage from
> allocation paths"), from the same series, then left these caches with
> no per-CPU caching at all.
> 
> In hindsight, skipping kmemleak only for the
> sheaf itself, as the new SLAB_ALLOC_NO_LEAKTRACE does, would have been
> the better fix.
> 
> The sheaf is allocated with SLAB_ALLOC_NO_RECURSE as well, like kmalloc
> sheaves. Without it, a backing kmalloc cache that needs a new sheaf of
> its own allocates it with SLAB_ALLOC_NO_LEAKTRACE too. kmemleak then
> does not track that sheaf, and can report objects queued in it by
> kfree_rcu() as leaks.
> 
> SLAB_ALLOC_NO_RECURSE also skips allocation tagging, so mark the codetag
> of these sheaves empty on free, as is done for kmalloc sheaves.
> 
> Link: https://lore.kernel.org/all/aW_NK2NXVgtuzCVH@gmail.com/ [1]
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
> Signed-off-by: Breno Leitao <leitao@debian.org>

Applied to slab/for-next, thanks!

> ---
> Changes in v2:
> - Fix sashiko findings:
>   - Set SLAB_ALLOC_NO_RECURSE along with SLAB_ALLOC_NO_LEAKTRACE, so the
>     flag stays off the kmalloc cache's own sheaf allocation, per Catalin
>     Marinas.
>   - Mark the codetag of SLAB_NOLEAKTRACE sheaves empty on free, per
>     Catalin Marinas.
> - Link to v1: https://patch.msgid.link/20261112-kmemleak_sheaves-v1-1-e40e3d73b253@debian.org
> ---
>  mm/slab.h |  1 +
>  mm/slub.c | 22 +++++++++++++---------
>  2 files changed, 14 insertions(+), 9 deletions(-)
> 
> diff --git a/mm/slab.h b/mm/slab.h
> index e94ac63d10dc03..7b20f10aa46a2e 100644
> --- a/mm/slab.h
> +++ b/mm/slab.h
> @@ -23,6 +23,7 @@
>  #define SLAB_ALLOC_NEW_SLAB	0x02 /* a flag for alloc_slab_obj_exts() */
>  #define SLAB_ALLOC_NO_RECURSE	0x04 /* prevent kmalloc() recursion */
>  #define SLAB_ALLOC_NO_OBJ_EXT	0x08 /* prevent obj_exts array allocation */
> +#define SLAB_ALLOC_NO_LEAKTRACE	0x10 /* skip kmemleak tracking */
>  
>  #define SLAB_FREE_DEFAULT	0x00 /* no flags */
>  #define SLAB_FREE_NOLOCK	0x01 /* spinning not allowed */
> diff --git a/mm/slub.c b/mm/slub.c
> index 8369853ddca721..b927cd684ec656 100644
> --- a/mm/slub.c
> +++ b/mm/slub.c
> @@ -2850,6 +2850,14 @@ static struct slab_sheaf *__alloc_empty_sheaf(struct kmem_cache *s, gfp_t gfp,
>  	if (s->flags & SLAB_KMALLOC)
>  		alloc_flags |= SLAB_ALLOC_NO_RECURSE;
>  
> +	/*
> +	 * kmemleak allocates its metadata from SLAB_NOLEAKTRACE caches, so
> +	 * tracking their sheaves would recurse into kmemleak. NO_RECURSE stops
> +	 * NO_LEAKTRACE from reaching the kmalloc cache's own sheaf allocation.
> +	 */
> +	if (s->flags & SLAB_NOLEAKTRACE)
> +		alloc_flags |= SLAB_ALLOC_NO_LEAKTRACE | SLAB_ALLOC_NO_RECURSE;
> +
>  	sheaf_size = struct_size(sheaf, objects, capacity);
>  	sheaf = kmalloc_flags(sheaf_size, gfp | __GFP_ZERO, alloc_flags, NUMA_NO_NODE);
>  
> @@ -2883,7 +2891,7 @@ static void __free_empty_sheaf(struct kmem_cache *s, struct slab_sheaf *sheaf,
>  	 * warning, therefore replace NULL with CODETAG_EMPTY to indicate
>  	 * that the extension for this sheaf is expected to be NULL.
>  	 */
> -	if (s->flags & SLAB_KMALLOC)
> +	if (s->flags & (SLAB_KMALLOC | SLAB_NOLEAKTRACE))
>  		mark_obj_codetag_empty(sheaf);
>  
>  	VM_WARN_ON_ONCE(sheaf->size > 0);
> @@ -4727,7 +4735,8 @@ bool slab_post_alloc_hook(struct kmem_cache *s, gfp_t flags, size_t size,
>  		if (init && p[i] && !is_kfence_address(p[i]))
>  			memset(p[i], 0, zero_size);
>  
> -		if (alloc_flags_allow_spinning(ac->alloc_flags))
> +		if (alloc_flags_allow_spinning(ac->alloc_flags) &&
> +		    !(ac->alloc_flags & SLAB_ALLOC_NO_LEAKTRACE))
>  			kmemleak_alloc_recursive(p[i], s->object_size, 1,
>  						 s->flags, init_flags);
>  		kmsan_slab_alloc(s, p[i], init_flags);
> @@ -7987,13 +7996,8 @@ static unsigned int calculate_sheaf_capacity(struct kmem_cache *s,
>  	if (IS_ENABLED(CONFIG_SLUB_TINY) || s->flags & SLAB_DEBUG_FLAGS)
>  		return 0;
>  
> -	/*
> -	 * Bootstrap caches can't have sheaves for now (SLAB_NO_SHEAVES).
> -	 * SLAB_NOLEAKTRACE caches (e.g., kmemleak's object_cache) must not
> -	 * have sheaves to avoid recursion when sheaf allocation triggers
> -	 * kmemleak tracking.
> -	 */
> -	if (s->flags & (SLAB_NO_SHEAVES | SLAB_NOLEAKTRACE))
> +	/* Bootstrap caches can't have sheaves for now (SLAB_NO_SHEAVES). */
> +	if (s->flags & SLAB_NO_SHEAVES)
>  		return 0;
>  
>  	/*
> 
> ---
> base-commit: 630059a5a632377b9b7b487baccde92b7ae1dd47
> change-id: 20261110-kmemleak_sheaves-b3531591ad21
> 
> Best regards,
> --  
> Breno Leitao <leitao@debian.org>
> 



      parent reply	other threads:[~2026-11-16 10:47 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-13 14:36 [PATCH v2] slab: allow sheaves for SLAB_NOLEAKTRACE caches Breno Leitao
2026-11-13 18:47 ` Catalin Marinas
2026-11-16 10:47 ` Vlastimil Babka (SUSE) [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=43dde2e9-24a7-4e09-ba07-d1166963e16b@kernel.org \
    --to=vbabka@kernel.org \
    --cc=akpm@linux-foundation.org \
    --cc=catalin.marinas@arm.com \
    --cc=cl@gentwo.org \
    --cc=hao.li@linux.dev \
    --cc=harry@kernel.org \
    --cc=kernel-team@meta.com \
    --cc=leitao@debian.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=rientjes@google.com \
    --cc=roman.gushchin@linux.dev \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox