public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
From: Madadi Vineeth Reddy <vineethr@linux.ibm.com>
To: Mel Gorman <mgorman@techsingularity.net>,
	Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@redhat.com>,
	Juri Lelli <juri.lelli@redhat.com>,
	Dietmar Eggemann <dietmar.eggemann@arm.com>,
	Valentin Schneider <vschneid@redhat.com>,
	Chris Mason <clm@meta.com>,
	linux-kernel@vger.kernel.org,
	Madadi Vineeth Reddy <vineethr@linux.ibm.com>
Subject: Re: [REGRESSION] [PATCH 0/2 v5] Reintroduce NEXT_BUDDY for EEVDF
Date: Thu, 8 Jan 2026 15:31:52 +0530	[thread overview]
Message-ID: <ec3ea66f-3a0d-4b5a-ab36-ce778f159b5b@linux.ibm.com> (raw)
In-Reply-To: <20251112122521.1331238-1-mgorman@techsingularity.net>

On 12/11/25 17:55, Mel Gorman wrote:
> Changes since v4
> o Splitout decisions into separate functions			(peterz)
> o Flow clarity							(peterz)
> 
> Changes since v3
> o Place new code near first consumer				(peterz)
> o Separate between PREEMPT_SHORT and NEXT_BUDDY			(peterz)
> o Naming and code flow clarity					(peterz)
> o Restore slice protection					(peterz)
> 
> Changes since v2
> o Review feedback applied from Prateek
> 
> I've been chasing down a number of schedule issues recently like many
> others and found they were broadly grouped as
> 
> 1. Failure to boost CPU frequency with powersave/ondemand governors
> 2. Processors entering idle states that are too deep
> 3. Differences in wakeup latencies for wakeup-intensive workloads
> 
> Adding topology into account means that there is a lot of machine-specific
> behaviour which may explain why some discussions recently have reproduction
> problems. Nevertheless, the removal of LAST_BUDDY and NEXT_BUDDY being
> disabled has an impact on wakeup latencies.
> 
> This series enables NEXT_BUDDY and may select a wakee if it's eligible to
> run even though other unrelated tasks may have an earlier deadline.
> 
> Mel Gorman (2):
>   sched/fair: Enable scheduler feature NEXT_BUDDY
>   sched/fair: Reimplement NEXT_BUDDY to align with EEVDF goals
> 
>  kernel/sched/fair.c     | 152 ++++++++++++++++++++++++++++++++++------
>  kernel/sched/features.h |   2 +-
>  2 files changed, 131 insertions(+), 23 deletions(-)
> 

Hi Mel, Peter,

During internal testing, I noticed approximately 7% regression in a real-world workload
called DayTrader.

Git bisect pointed to this patch:
"sched/fair: Reimplement NEXT_BUDDY to align with EEVDF goals"

Before this patch was merged, I reported a regression in v4 with schbench and stress-ng.
From that discussion: 
https://lore.kernel.org/all/ddfde793-ad6e-4517-96b8-662dcb78acc8@linux.ibm.com/#t

```
So with frequent wakeups, queued tasks (even with earlier deadlines) may be
unfairly delayed. I understand that this would fade away quickly as the
woken up task that got to run due to buddy preference would accumulate negative
lag and would not be eligible to run again, but the starvation could be higher if
wakeups are very high.

To test this, I ran schbench (many message and worker threads) together with
stress-ng (CPU-bound), and observed stress-ng's bogo-ops throughput dropped by
around 64%.

This shows a significant regression for CPU-bound tasks under heavy wakeup loads.
```

I understand that stress-ng bogo-ops is not a reliable metric. However, the problem
appears to be real, as DayTrader also shows regression with this patch.

To check if WF_SYNC related change is the issue, I tried to decrease threshold by 
`echo 50000 > /sys/kernel/debug/sched/migration_cost_ns` so that waker could preempt
quickly in WF_SYNC case. This helped but I understand that it changes a lot of code paths
that use migration_cost_ns. So, when I decreased only threshold in this patch, the
performance didn't improve.

So, I think the problem is in making the tasks that are having earlier deadline to wait in
presence of frequent wakeups is hurting CPU intensive workloads.

Any thoughts/ideas?

Meanwhile, I will also spend time to workaround this patch and see if the performance
could be improved.

Thanks,
Vineeth


      parent reply	other threads:[~2026-01-08 10:02 UTC|newest]

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-11-12 12:25 [PATCH 0/2 v5] Reintroduce NEXT_BUDDY for EEVDF Mel Gorman
2025-11-12 12:25 ` [PATCH 1/2] sched/fair: Enable scheduler feature NEXT_BUDDY Mel Gorman
2025-11-14 12:19   ` [tip: sched/core] " tip-bot2 for Mel Gorman
2025-11-17 16:23   ` tip-bot2 for Mel Gorman
     [not found] ` <20251112122521.1331238-3-mgorman@techsingularity.net>
2025-11-12 14:48   ` [PATCH 2/2] sched/fair: Reimplement NEXT_BUDDY to align with EEVDF goals Peter Zijlstra
2025-11-13  8:26     ` Madadi Vineeth Reddy
2025-11-13  9:04     ` Mel Gorman
2025-11-14 12:13       ` Peter Zijlstra
2025-11-14 12:19   ` [tip: sched/core] " tip-bot2 for Mel Gorman
2025-11-17 16:23   ` tip-bot2 for Mel Gorman
2025-12-22 10:57     ` [REGRESSION] " Ryan Roberts
2026-01-02 12:38       ` Ryan Roberts
2026-01-02 15:52         ` Dietmar Eggemann
2026-01-05 11:45           ` Ryan Roberts
2026-01-05 14:38             ` Shrikanth Hegde
2026-01-05 16:33               ` Ryan Roberts
2026-01-07 15:30             ` Dietmar Eggemann
2026-01-08  8:50               ` Mel Gorman
2026-01-08 13:15                 ` Ryan Roberts
2026-01-09 10:15                   ` Ryan Roberts
2026-01-12  7:47                     ` Peter Zijlstra
2026-01-12  8:52                       ` Ryan Roberts
2026-01-12  9:57                         ` Peter Zijlstra
2026-01-12 10:27                           ` Ryan Roberts
2026-01-13  6:31                         ` K Prateek Nayak
2026-01-15 10:16                     ` Mel Gorman
2026-01-08 10:01 ` Madadi Vineeth Reddy [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ec3ea66f-3a0d-4b5a-ab36-ce778f159b5b@linux.ibm.com \
    --to=vineethr@linux.ibm.com \
    --cc=clm@meta.com \
    --cc=dietmar.eggemann@arm.com \
    --cc=juri.lelli@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mgorman@techsingularity.net \
    --cc=mingo@redhat.com \
    --cc=peterz@infradead.org \
    --cc=vschneid@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox