Re: [PATCH v1] mm: move LRU size stat updates out of lru_lock in folio batches

[Shakeel Butt <shakeel.butt@linux.dev>]

On Thu, Oct 22, 2026 at 01:03:13PM -0700, Joanne Koong wrote:
> folio_batch_move_lru() and folios_put_refs() take the lruvec's lru_lock
> once per batch, but they update the node and memcg lruvec stats for
> every folio in the batch while holding the lock. Only the LRU lists and
> the memcg lru_zone_size need lru_lock. Updating the stats while holding
> the lru_lock is unnecessary, and they make up a large part of the
> critical section, as seen in [1]. 

Can you please add a tldr from [1] here? Too much text in that link.

> With many threads faulting and freeing
> memory in a single memcg, about half of all cycles are spent waiting for
> lru_lock, and the hold time of the lru_lock directly limits throughput.

Do you have perf profile showing this? Mainly I want to know where in stats
update we are spending the CPU most.

I wonder if we can find a lower bound here by removing the stats update
altogether in this code path (just for experimentation) and measure the
throughput we get.

> 
> Accumulate the stat changes per LRU list while holding the lock instead,
> and only apply them after dropping the lock (while still inside
> the RCU read section that keeps the lruvec's memcg alive). This moves
> the stat updates out of the lru_lock critical section and turns the
> per-folio updates into at most one update per LRU list per batch.
> 
> The batch is attached to the lruvec while lru_lock is held so that the
> LRU add/del helpers don't need to pass it down. To keep struct lruvec
> from growing, cost_lock is moved into the hole after lru_lock, which
> leaves room for the batch pointer in the same cacheline as lru_lock.
> 
> The stats can now lag behind the LRU lists by one batch, which is well
> within the per-cpu batching they already have.
> 
> On a 2-socket Skylake (80 CPUs, 2 NUMA nodes), with threads pinned to
> CPUs repeatedly faulting in and freeing memory in a single memcg,
> average pages per second over 7 freshly created memcgs per run, 2 boots
> each:
> 
>                      before    after
>   shmem, 20 threads  4.74M     5.47M   (+15%)
>   shmem, 80 threads  7.94M    10.33M   (+30%)
>   anon,  20 threads  3.04M     3.35M   (+10%)
>   anon,  80 threads  3.23M     3.41M   (+5%)
> 
> Link: https://lore.kernel.org/linux-fsdevel/CAJnrk1ZezmmzrYdf8KP6re_a76izMS=qGPeC8ZZXiteHyZZC-Q@mail.gmail.com/ [1]
> Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
> ---
> An alternative that avoids adding the lru_size_batch pointer to the lruvec
> struct is to pass the batch down explicitly, but the plumbing for this is a
> bit ugly (and doesn't cover mglru paths). The code looks like [2]. Happy to go
> with this if that is preferred.

Do you expect there might be opportunity to batch the stats other than LRU stats
you batched in this patch? If there is then I think we should at least explore
if a more general stat batching can be done.

This code looks good though.