[PATCH 1/1] sched/psi: skip irqtime accounting when no new irq time has elapsed

[Usama Arif <usama.arif@linux.dev>]

psi_account_irqtime() reads irq_time_read() into a per-rq cumulative
counter and only bails out when the delta vs. the previously accounted
amount is negative.  A delta of exactly zero is treated as "do the
work": psi_write_begin() is taken, cpu_clock(cpu) is read (which on
x86 ends up in native_sched_clock() / rdtsc) and the cgroup ancestor
chain is walked to add zero to every group's PSI_IRQ_FULL bucket.

The zero-delta case is common in practice -- it fires every time a
context switch crosses a PSI group boundary on a CPU that hasn't
serviced an interrupt between the two switches.

Measured on a 176-thread AMD EPYC 9D64 server running a compute
intensive production workload, instrumented with bpftrace over a 30s
window (irq_time_read() read directly from the per-CPU cpu_irqtime so
that delta == 0 and delta < 0 could be separated):

  @total              17,229,311    (100.0%)
  @ret_curr_swapper    7,864,195    ( 45.6%)  curr->pid == 0
  @ret_samegrp           323,299    (  1.9%)  same cgroup as prev
  @reached_delta       9,041,817    ( 52.5%)
    @delta_positive    6,358,192    ( 36.9%)  real work
    @delta_zero        2,683,625    ( 15.6%)  work wasted (this patch)
    @delta_negative          (0)    (  0.0%)  monotonic clock

So 15.6 % of all psi_account_irqtime() calls - and 29.7 % of the
calls that get past the early returns - hit the delta == 0 case;
delta < 0 did not occur once in the 30 s window.  Under the current
code each of those ~89 k calls per second performs the full seqcount
write + cpu_clock() read + cgroup-chain walk just to add 0 to every
group's PSI_IRQ_FULL counter.

Extend the early-return to also cover delta == 0.  rq->psi_irq_time
does not need updating in that case (it would store the same value
back) and no PSI bucket would change.  The existing behaviour for
delta > 0 is untouched.

Signed-off-by: Usama Arif <usama.arif@linux.dev>
---
 kernel/sched/psi.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/sched/psi.c b/kernel/sched/psi.c
index d9c9d9480a45..848955f8893d 100644
--- a/kernel/sched/psi.c
+++ b/kernel/sched/psi.c
@@ -1023,7 +1023,7 @@ void psi_account_irqtime(struct rq *rq, struct task_struct *curr, struct task_st
 
 	irq = irq_time_read(cpu);
 	delta = (s64)(irq - rq->psi_irq_time);
-	if (delta < 0)
+	if (delta <= 0)
 		return;
 	rq->psi_irq_time = irq;
 
-- 
2.53.0-Meta