Re: [REGRESSION] osnoise: "eventpoll: Replace rwlock with spinlock" causes ~50µs noise spikes on isolated PREEMPT_RT cores

Sebastian Andrzej Siewior
2026-03-26T14:31:53Z

On 2026-03-26 16:00:57 [+0200], Ionut Nechita (Wind River) wrote:
> Summary across all isolated cores (32 CPUs):
>
> With spinlock With rwlock (reverted)
> MAX noise (ns): 44,343 - 51,869 0 - 10
> IRQ count/sample: ~6,650 - 6,870 3 - 7
> Thread noise/sample: ~5,700 - 5,940 0 - 1
> CPU availability: 94.5% - 95.3% ~100%

is there some load or just idle with osnoise?

> My understanding of the root cause: the original rwlock allowed
> ep_poll_callback() (producer side, running from IRQ context on any CPU)
> to use read_lock, which does not cause cross-CPU contention on isolated
> cores when no local epoll activity exists. With the spinlock conversion,
> on PREEMPT_RT spinlock_t becomes an rt_mutex. This means that even if
> the isolated core is not involved in any epoll activity, the lock's
> cacheline bouncing and potential PI-boosted wakeups from housekeeping
> CPUs can inject noise into the isolated cores via IPI or cache
> invalidation traffic.

With the read_lock() you can acquire the lock with multiple readers.
Each read will increment the "reader counter" so there is cache line
activity. If a isolated CPU does not participate, it does not
participate. With the change to spinlock_t there can be only one user at
a time. So the other have to wait and again, isolated core which don't
participate are not affected.

> The commit message acknowledges the throughput regression but argues
> real workloads won't notice. However, for RT/latency-sensitive
> deployments with CPU isolation, the impact is severe and measurable
> even with zero local epoll usage.
>
> I believe this needs either:
> a) A revert of the backport for stable RT trees, or

I highly doubt since it affected RT loads.

> b) A fix that avoids the spinlock contention path for isolated CPUs
>
> I can provide the full osnoise trace data if needed.

So the question is why are the isolated core affected if they don't
participate is epoll.

> Tested on:
> Linux system-0 6.12.78-vanilla-{0,1} SMP PREEMPT_RT x86_64
> Linux system-0 6.12.57-vanilla-{0,1} SMP PREEMPT_RT x86_64
>
> Thanks,
> Ionut.

Sebastian