Re: RT wakeup latency due to unbound workqueue execution (runtime mitigation?)

[Sebastian Andrzej Siewior <bigeasy@linutronix.de>]

On 2026-02-02 13:42:07 [+0200], vigashini kesavan wrote:
> Hi,
> 
> I am investigating occasional cyclictest wakeup latency spikes on a
> PREEMPT_RT system (Linux 6.1.158-rt) and have traced it to the
> execution of unbound workqueue items (flush_memcg_stats_dwork,
> wb_workfn) on the measurement CPU.
> 
> These are observed under default Boot-time parameters, without any
> modification to parameters such as isolcpus, nohz_full parameters or
> workqueue cpumask. The intention is to analyze latency behaviour under
> default boot-time configuration, relying only on runtime mechanisms
> such as cgroup configuration or task level CPU affinity.
> Manual pinning of kworker threads through task affinity was tried
> without success.

You can't pin an unbound worker to a CPU, this just does not work long
term. If you want this work to be invoked only a specific CPU you could
do replace the queue_delayed_work() invokcation with something like
	queue_delayed_work_on(1, system_wq, &stats_flush_dwork, FLUSH_TIME);
to have it invoked always on CPU1.

> Question:
> Is there a supported or recommended runtime mechanism to prevent
> unbound workqueue execution on a specific CPU. Or to defer this class
> of work away from any CPU? (without any boot-time workqueue cpumasks)

The hack above would be one way. But since you asked for recommended:
There is /sys/devices/virtual/workqueue/cpumask which contains the CPUs
for "unbound" workqueues. This has an effect on all of them.

> Also, is there any option to attribute the unbound workqueue execution
> to any kernel subsystems or runtime activities that could queue this
> work. To better understand any potential userspace triggers for this.

That folder
	/sys/devices/virtual/workqueue/

contains all the options you have. Some workqueues are dedicated and
have a name and allow changes. Other do not. The one you named, does
not use a specific workqueue but a generic one.

> Any guidance would be appreciated.
> 
> Thanks,
> Vigashini Kesavan

Sebastian