Re: [PATCH v2 3/3] softirq: Use a dedicated thread for timer wakeups on PREEMPT_RT.

[Frederic Weisbecker <frederic@kernel.org>]

Le Thu, Oct 24, 2024 at 04:55:51PM +0200, Sebastian Andrzej Siewior a écrit :
> A timer/ hrtimer softirq is raised in-IRQ context. With threaded
> interrupts enabled or on PREEMPT_RT this leads to waking the ksoftirqd
> for the processing of the softirq. ksoftirqd runs as SCHED_OTHER which
> means it will compete with other tasks for CPU ressources.
> This can introduce long delays for timer processing on heavy loaded
> systems and is not desired.
> 
> Split the TIMER_SOFTIRQ and HRTIMER_SOFTIRQ processing into a dedicated
> timers thread and let it run at the lowest SCHED_FIFO priority.
> Wake-ups for RT tasks happen from hardirq context so only timer_list timers
> and hrtimers for "regular" tasks are processed here. The higher priority
> ensures that wakeups are performed before scheduling SCHED_OTHER tasks.
> 
> Using a dedicated variable to store the pending softirq bits values
> ensure that the timer are not accidentally picked up by ksoftirqd and
> other threaded interrupts.
> It shouldn't be picked up by ksoftirqd since it runs at lower priority.
> However if ksoftirqd is already running while a timer fires, then
> ksoftird will be PI-boosted due to the BH-lock to ktimer's priority.
> Ideally we try to avoid having ksoftirqd running.
> 
> The timer thread can pick up pending softirqs from ksoftirqd but only
> if the softirq load is high. It is not be desired that the picked up
> softirqs are processed at SCHED_FIFO priority under high softirq load
> but this can already happen by a PI-boost by a force-threaded interrupt.
> 
> [ frederic@kernel.org: rcutorture.c fixes, storm fix by introduction of
>   local_timers_pending() for tick_nohz_next_event() ]
> 
> [ junxiao.chang@intel.com: Ensure ktimersd gets woken up even if a
>   softirq is currently served. ]
> 
> Reviewed-by: Paul E. McKenney <paulmck@kernel.org> [rcutorture]
> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>

Reviewed-by: Frederic Weisbecker <frederic@kernel.org>

Just a few nits:


> ---
>  include/linux/interrupt.h | 44 +++++++++++++++++++++++++
>  kernel/rcu/rcutorture.c   |  6 ++++
>  kernel/softirq.c          | 69 ++++++++++++++++++++++++++++++++++++++-
>  kernel/time/hrtimer.c     |  4 +--
>  kernel/time/tick-sched.c  |  2 +-
>  kernel/time/timer.c       |  2 +-
>  6 files changed, 122 insertions(+), 5 deletions(-)
> 
> diff --git a/include/linux/interrupt.h b/include/linux/interrupt.h
> index 457151f9f263d..9637af78087f3 100644
> --- a/include/linux/interrupt.h
> +++ b/include/linux/interrupt.h
> @@ -616,6 +616,50 @@ extern void __raise_softirq_irqoff(unsigned int nr);
>  extern void raise_softirq_irqoff(unsigned int nr);
>  extern void raise_softirq(unsigned int nr);
>  
> +/*
> + * Handle timers in a dedicated thread at a low SCHED_FIFO priority instead in
> + * ksoftirqd as to be prefred over SCHED_NORMAL tasks.
> + */

This doesn't parse. How about, inspired by your changelog:

"""
Wake-ups for RT tasks happen from hardirq context so only timer_list timers
and hrtimers for SCHED_OTHER tasks are processed from softirq. As they are
raised from hardirq, their processing would normally happen from ksoftirqd
which runs as SCHED_OTHER and compete with other tasks. Moving timers softirqs
to a low-prio SCHED_FIFO kthread instead ensures that wakeups from timers are
performed before scheduling the target SCHED_OTHER tasks.
"""

Thanks.