Re: [RFC PATCH 2/2] mm/slub: introduce slab parking to reduce list_lock contention

[Pedro Falcato <pfalcato@suse.de>]

On Mon, Sep 07, 2026 at 03:38:22PM +0200, Vlastimil Babka (SUSE) wrote:
> On 8/24/26 14:25, Hao Li wrote:
> > Introduce a mechanism called parking to mitigate lock contention in the
> > free slowpath.
> 
> Interesting!
> 
> > In the free slowpath, when __slab_free() transitions a full slab into a
> > partial/empty slab through a free operation, it must acquire the list
> > lock to add these newly freed partial/empty slabs to the partial list.
> > 
> > Why must partial and empty slabs converted from full slabs be added to
> > the partial list? Because only by doing so can the sheaf refill or alloc
> > slowpath see these partial slabs and allocate from them. Therefore, the
> > list insertion must be performed, which requires acquiring the lock and
> > leads to heavy lock contention under high concurrency.
> > 
> > Analysis of profiling data from the will-it-scale mmap1 benchmark shows
> > that full -> partial transitions account for a large proportion, second
> > only to partial -> partial.
> > 
> > With extra instrumentation added to __slab_free(), the following data
> > was collected for the maple_node cache (in counts):
> > 
> >   partial->partial    843017414
> >   full->partial       550719384
> >   partial->empty      17459564
> >   full->empty         2
> 
> That's a lot inded. I'd be careful if it's some specific aspect of the test,
> i.e. lots of parallel allocations followed by lots of frees, that wouldn't
> be that common in realistic workloads. But worth looking into at least.
> If it's this kind of pathologic behavior, then I expect changing sheaf size
> as suggested by Pedro wouldn't help much.

mmap1[1] is a very simple test. It just mmaps and munmaps. I assume what we're
observing here is the churn in maple tree nodes as we allocate/free/split/etc.

I suspect it's also possible that bulk RCU freeing is causing this. It might
1) be substantially delayed and 2) dumping a lot of freed maple tree node
objects that overflow the pcs.

[1] https://github.com/heatd/will-it-scale/blob/master/tests/mmap1.c

> 
> > Since the fundamental purpose of __slab_free() is to make newly freed
> > partial/empty slabs visible to the sheaf refill or alloc slowpath, these
> > slabs can be temporarily stored in a staging area in a lockless manner
> > instead of making the free slowpath contend for the lock. The sheaf
> 
> However the lockless manipulation is still going to contend on the
> llist_head. But perhaps it's limited enough in both users and lenght of
> operations to make a difference.
> 
> > refill or alloc slowpath then checks this staging area first when
> > allocating objects. This achieves the goal of making these slabs visible
> > to the sheaf refill or alloc slowpath while allowing the free slowpath
> > to operate locklessly. This process is called "parking".
> 
> I'm gonna pull a David Hildenbrand trick here and question the name :)
> It seems to me it's an (extension of the) partial list, but lockless.
> Parking would suggest to me that it's put somewhere aside not to be used, or
> something.
> 
> > Parking occurs in only one case: when __slab_free() encounters a full ->
> > partial/empty transition and the trylock fails. In this case,
> > __slab_free() attaches the slab to an llist locklessly, instead of
> > waiting for the lock unnecessarily.
> 
> It could be interesting to also see if skipping the trylock completely
> (another cache contending operation) helps even more. Also whether moving
> the llist_node to a different cache line than list_lock (and fields
> protected by it) helps even more, or not.
> 
> > Conversely, the process of moving these parked slabs from the llist back
> > to the partial list is called "unpark". Unpark occurs in four cases:
> > 
> > 1. Sheaf refill or alloc slowpath: This is the core case. The sheaf
> >    refill or alloc slowpath must see the parked slabs, so the first
> >    thing done after acquiring the lock in the sheaf refill or alloc
> >    slowpath is unpark.
> > 2. Cache shrinking: shrinking also needs to see slabs in the parked
> >    state.
> > 3. Cache destruction: kmem_cache_destroy() must also see parked slabs,
> >    which is obvious, otherwise memory would leak.
> > 4. delayed_work (see corner case b below)
> > 
> > Why is this scheme correct? Because paths entering the sheaf refill or
> > alloc slowpath can see both slabs on the partial list and slabs on the
> > parked llist, while allocation paths that do not enter the sheaf refill
> > or alloc slowpath would not check the partial list in the first place
> > and naturally do not need to care about parked slabs. Therefore, whether
> > an allocation takes the sheaf refill or the alloc slowpath or not, slabs
> > on the parked llist and slabs on the partial list make no difference to
> > the allocator. This visibility equivalence is the core of the scheme.
> > This analysis also shows that the scheme does not affect the utilization
> > of partial slabs or lead to increased fragmentation.
> > 
> > Corner cases to handle:
> > a. Parked slabs may become completely empty. Therefore, unpark must also
> >    check min_partial and free excess empty slabs instead of adding them
> >    back to the partial list.
> > 
> > b. In rare cases, the system may go idle immediately after slabs are
> >    parked, and the sheaf refill or alloc slowpath may never run
> >    again. These parked slabs would then remain in the llist until the
> >    next sheaf refill or alloc slowpath performs an unpark. To
> >    solve this problem, add a delayed_work named unpark_work to add
> >    parked slabs back to the partial list when no other path unparks
> >    them.
> 
> OK, but is this a problem that needs the delayed work? If the slabs are
> still partial, they would just sit on the partial list rather than on the
> llist, but it would cause no extra bloat?
> It could be a problem only if free slab(s) got stuck on the llist.
> 
> So I'd try to avoid the delayed work as it's quite a red flag. Periodic
> flushing of alien arrays used to be a very unpopular part of SLAB
> implementation. I think there might be two ways:
> 
> 1) submit the delayed work only when transitioning partial->empty slab on
> the llist. Would likely require flagging slabs that are on the llist.
> Hopefully this will limit the submissions to negligible amounts.
> 
> 2) remove the delayed work completely, instead __slab_free() would perform
> the "unpark" immediately when detecting partial->empty slab transition on
> the llist. Would need flagging the slabs as well.

3) Don't do any of this and keep a small(ish?) barn locally, for each CPU (welcome
back per-CPU partial slabs!). I'll play around with this idea and see if I can
get interesting results...

I also have other vague, handwavy ideas... Hmm...

But I really, really suspect that deferring work and playing around with
trylocks is really just working around allocator deficiencies.

-- 
Pedro