Re: [PATCH v4 0/4] Introduce Per-CPU Work helpers (was QPW)

[Leonardo Bras <leobras.c@gmail.com>]

On Mon, Jul 13, 2026 at 10:07:23AM +0200, Sebastian Andrzej Siewior wrote:
> On 2026-07-12 17:32:49 [-0300], Leonardo Bras wrote:
> > > > The idea:
> > > > Currently with PREEMPT_RT=y, local_locks() become per-cpu spinlocks.
> > 
> > Hi Sebastian, thank you for reviewing!
> > (Sorry for the delay)
> > 
> > > It does not become a _spin_lock because it does not spin. It sleeps.
> > 
> > Right, it's a per-cpu mutex. 
> > My point is that it's a full lock, and we could use it instead of doing the 
> > whole scheduling thing, since we are already paying the 'atomic overhead' 
> > to get the lock here.
> 
> The whole lock is a spinlock_t. There is also raw_spinlock_t and
> bit_spin_lock(). All three are considered spinlocks.
> 
> > > > In this case, instead of scheduling work on a remote cpu, it should
> > > > be safe to grab that remote cpu's per-cpu spinlock and run the required
> > > > work locally. That major cost, which is un/locking in every local function,
> > > > already happens in PREEMPT_RT.
> > > 
> > > We did have this before but only in the RT tree. It was a bit messy from
> > > the naming because it started with local_ but then it was a remote CPU.
> > 
> > Had the same naming issue here. This idea was initially a expansion to 
> > local_lock() mechanism, about the same way you were planning in the past.
> > 
> > > The main issue was the different code path which led to a few deadlocks
> > > back then.
> > > By the time local_lock_t went upstream, the cross-CPU locking was
> > > removed. As far as I remember, the cross-CPU user which did schedule
> > > work on a remote CPU and annoyed NOHZ folks were replaced.
> > 
> > I understand this could be a big issue if used in a generic way.
> > 
> > What I am proposing here a mechanism that standardizes those 
> > local_lock()+IPI strategies based on how they are done today, so we are 
> > only explected to get 'remote-cpu' pwlocks in the 'IPI replacement' 
> > operations.
> > 
> > The idea is pwlock_local* in every local function, and pwlock*(,cpu) in 
> > operations that can be remote. 
> > 
> > Maybe being used in a more constrained way, it has less chance of being an 
> > issue. Also, the whole idea is to improve CPU isolation numbers by 
> > reducing IPIs, so maybe NOHZ people will be happier with that :)
> 
> I get that part. The local_lock_t part is cheap on !RT and becomes a
> full lock on RT. While the lock details change the overall expectation
> remain the same. With this change it is possible to acquire the lock
> cross-CPU but this depends on the config/ setup. This might not be easy
> in terms of testing and maintenance.
> The more potential users you have, the better it might become in terms
> of a selling argument. If you have just (say) two users it might be
> simpler to address just those.
> 

IIRC there are multiple users of this mechanism around the kernel. I 
remember picking the first three (swap, slub and memcontrol) as examples of 
how to use pwlocks (QPW at the time). With the concept proven, I could then 
proceed to work with other potential users to replace it. (We ended up 
dropping memcontrol in the process)

The main idea is to replace as many potential users as possible to reduce 
as much as possible the amount of IPIs in isolated cpus, and allow Linux 
to run workloads which require much lower latency.

This idea came as a general solution to a bunch of latency violations  
Marcelo and I were coming across.

Thanks!
Leo