All of lore.kernel.org
 help / color / mirror / Atom feed
* [REGRESSION] 6.12.y, 6.18.y: fair server runs SCHED_OTHER ahead of SCHED_FIFO on an idle CPU (fixed in 6.19)
@ 2026-11-20 15:09 Florent Carli
  2026-11-20 17:09 ` Juri Lelli
  0 siblings, 1 reply; 10+ messages in thread
From: Florent Carli @ 2026-11-20 15:09 UTC (permalink / raw)
  To: stable; +Cc: Peter Zijlstra, Juri Lelli, Vincent Guittot, linux-rt-users

Hello,

As I understand it, the fair server is deferred: it should run
SCHED_OTHER tasks ahead of RT tasks only when RT tasks have kept the
CPU busy for most of a period (about 950 ms out of 1 s), so that fair
tasks do not starve.

On 6.12.y and 6.18.y, I see it run SCHED_OTHER tasks ahead of RT tasks
on a CPU where nothing starves. My setup is a PREEMPT_RT system (Debian
trixie, 6.12.117 rt) with a time critical SCHED_FIFO 98 thread on an
isolated CPU. That CPU is idle most of the time. The thread wakes up
every 300 us and runs for a few us. From time to time, a SCHED_OTHER
task that wakes up on that CPU runs ahead of the FIFO thread, for as
long as it wants to run: a container health check (bash) delayed the
FIFO thread by 3.4 ms this way.

6.19.y and later do not show this.

My reading of the 6.12.y code: about one second after the last fair
activity on the CPU, the zero-laxity timer fires and replenishes the
server with dl_defer_running = 1. Idle time then drains its runtime,
and dl_server_update_idle_time() clears dl_defer_running after about
50 ms of idle. A fair task that wakes up inside those ~50 ms runs under
the server, ahead of any FIFO task, until it sleeps.

Reproducer, as root, on an isolated CPU with nothing else on it:

  CPU=22
  python3 - "$CPU" <<'EOF' &
  import os, sys, time
  os.sched_setaffinity(0, {int(sys.argv[1])})
  while True:
      time.sleep(1.02)
      end = time.monotonic() + 0.003
      while time.monotonic() < end:
          pass
  EOF
  cyclictest -m -p 98 -a $CPU --mainaffinity=0 -t 1 -i 200 -D 30 -q

--mainaffinity matters: cyclictest's main thread is SCHED_OTHER, and
when it stays on $CPU its activity every ~60 ms keeps pushing the
server's period out and hides the problem. Any other fair activity on
the CPU more often than every ~950 ms does the same.

Results, sleep between 1.00 and 1.04 s, 3 ms spin. Max FIFO latency,
and wake-ups later than 500 us:

  bare metal, 6.12.117 Debian rt         3001 us, 10 per 20 s
  (10 ms spin instead of 3 ms)           9986 us
  KVM guest on dedicated host CPUs:
  6.12.117 Debian rt                     3000 us, 10 per 20 s
  6.12.118, vanilla PREEMPT_RT           2900 us, 8 in 270 s
  6.12.118 + v6.12-rt queue              3000 us, 14-15 per 30 s
  6.18.15 Debian rt                      3000 us, 10 per 20 s
  6.19.14, 7.3.4 Debian rt               < 25 us, 0
  all of the above with
  fair_server/cpuN/runtime = 0           < 25 us, 0

The lazy preemption patches of the -rt queue make the window much
easier to hit, but vanilla 6.12.118 is affected too.

The fix went into 6.19:

  e636ffb9e31b ("sched/deadline: Fix dl_server time accounting")
  f5a538c07df2 ("sched/deadline: Fix dl_server stop condition")
  1e0a2ba7afb1 ("sched: Provide idle_rq() helper")
  ca1e8eede4fc ("sched/deadline: Fix server stopping with runnable tasks")

None of them carries a stable tag, and only 1e0a2ba7afb1 applies
cleanly to 6.12.y or 6.18.y.

I have a backport of the four to 6.12.118, with small adaptations
(rq->donor does not exist in 6.12, so rq->curr is used). With it, the
reproducer shows no late wake-up, both on vanilla 6.12.118 and with the
-rt queue, and the starvation guard still works: under a SCHED_FIFO
busy loop, a SCHED_OTHER task on the same CPU still gets 4.7 to 4.9%
of it, as without the backport (0% with the server runtime at 0).

Would you take this series for 6.18.y and 6.12.y? I can send the 6.12.y
backport as patches and prepare the 6.18.y one, or leave it to the
authors if they prefer.

Until then, writing 0 to /sys/kernel/debug/sched/fair_server/cpuN/runtime
on the isolated CPUs avoids the problem.

Thanks,
Florent.

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2026-11-21 15:27 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-20 15:09 [REGRESSION] 6.12.y, 6.18.y: fair server runs SCHED_OTHER ahead of SCHED_FIFO on an idle CPU (fixed in 6.19) Florent Carli
2026-11-20 17:09 ` Juri Lelli
2026-11-20 18:19   ` Florent Carli
2026-11-21 15:27     ` Sasha Levin
2026-11-20 18:19   ` [PATCH 6.12.y 0/4] sched/deadline: dl_server fixes from 6.19 Florent Carli
2026-11-20 18:19     ` [PATCH 6.12.y 1/4] sched/deadline: Fix dl_server time accounting Florent Carli
2026-11-20 18:19     ` [PATCH 6.12.y 2/4] sched/deadline: Fix dl_server stop condition Florent Carli
2026-11-20 18:19     ` [PATCH 6.12.y 3/4] sched: Provide idle_rq() helper Florent Carli
2026-11-20 18:19     ` [PATCH 6.12.y 4/4] sched/deadline: Fix server stopping with runnable tasks Florent Carli
2026-11-21 15:27     ` [PATCH 6.12.y 0/4] sched/deadline: dl_server fixes from 6.19 Sasha Levin

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.