[PATCH v4] sched/fair: Preserve wake-affine CPU for non-SMT reciprocal sync wakeups

[PATCH v4] sched/fair: Preserve wake-affine CPU for non-SMT reciprocal sync wakeups

[Shubhang Kaushik (Ampere)]

For WF_SYNC wakeups, wake_affine() may select the waker CPU, but the CFS
wakeup path still passes that target to select_idle_sibling(). The idle
CPU search can then move the wakee away from the wake-affine target.

Pipe-style ping-pong workloads expose this because the wakee is handed
back and forth between two tasks. In that case, moving the wakee to
another idle CPU can cost more than preserving the wake-affine waker CPU.

Use the existing last_wakee and wake_wide() state to identify narrow
reciprocal WF_SYNC wakeups:

  A wakes B
  B wakes A
  A wakes B
  ...

Handle only this narrow reciprocal case on non-SMT systems. Once the
wake-affine path has selected or kept the waker CPU, preserve that target
when the waker rq has no other runnable fair task. Return the waker CPU
before select_idle_sibling() so the idle CPU search does not move this
handoff away from the wake-affine target.

This does not define a generic WF_SYNC placement rule. Generic WF_SYNC
wakeups continue through the existing wake_affine() and
select_idle_sibling() behavior. SMT systems also continue through
select_idle_sibling(), where idle sibling/core placement can be handled
with SMT topology visible.

On asymmetric-capacity systems, still require the wakee to fit on the
waker CPU.

Signed-off-by: Shubhang Kaushik (Ampere) <sh@gentwo.org>
---
Tested on 80-core non-SMT Ampere Altra, tip:sched/core baseline.

perf bench sched pipe -l 1000000, 20 runs:
default:
  3.985 -> 3.187 usec/op mean,   about 20.0% improvement
  4.026 -> 3.181 usec/op median, about 21.0% improvement

taskset -c 78,79:
  3.851 -> 3.144 usec/op mean,   about 18.4% improvement
  3.804 -> 3.140 usec/op median, about 17.4% improvement

taskset -c 79:
  3.055 -> 3.113 usec/op mean,   about 1.9% slower
  3.045 -> 3.109 usec/op median, about 2.1% slower

Hackbench process/thread pipe cases with 1/2/4/8 groups were within
noise, with mean deltas from -1.8% to +3.7% over 10 runs.

Schbench normal mode at 8/40/80/240 workers and schbench pipe mode at
1/2/4/8 workers showed no material regression.

Baseline: tip/sched/core at 5186ef36909c
---
Changes in v4:
  - Preserve the waker CPU only after the wake-affine path selected or
    kept it.
  - Clarify that WF_SYNC remains a hint, not a generic placement rule.
  - Leave SMT systems on the existing select_idle_sibling() path.
  - Refresh testing on tip:sched/core.

Link to v3: https://lore.kernel.org/r/20260727-b4-sched-sync-wakeup-v3-1-90cf481dbd85@gentwo.org

Changes in v3:
  - Limit the direct waker-CPU preference to !sched_smt_active(); SMT
    systems continue through the existing wake_affine() and
    select_idle_sibling() path.
  - Drop the redundant affinity check; want_affine already verifies the
    waker CPU is allowed.
  - Use a plain p->last_wakee read instead of READ_ONCE().
  - Rebase and refresh testing on v7.2-rc5.

Link to v2: https://lore.kernel.org/r/20260722-b4-sched-sync-wakeup-v2-1-f1164560b24b@gentwo.org

Changes in v2:
  - Move the reciprocal handoff preference under the existing
    SD_WAKE_AFFINE domain check.
  - Drop futex from the changelog motivation.
  - Refresh perf bench sched pipe results after rebasing.

Link to v1: https://lore.kernel.org/r/20260721-b4-sched-sync-wakeup-v1-1-dc94f184e27f@gentwo.org
---
 kernel/sched/fair.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index d78467ec6ee1343050fcc2794dafb38ade3599e5..e377dc43460a691315d895e5cde5208987791ce8 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -8794,6 +8794,26 @@ static inline bool asym_fits_cpu(unsigned long util,
 	return true;
 }
 
+/*
+ * For reciprocal WF_SYNC handoffs, prefer the waker CPU when it has no
+ * other runnable fair task.
+ */
+static bool prefer_sync_pair_cpu(struct task_struct *p, int cpu)
+{
+	struct rq *rq = cpu_rq(cpu);
+
+	if ((rq->nr_running - cfs_h_nr_delayed(rq)) != 1)
+		return false;
+
+	if (sched_asym_cpucap_active()) {
+		sync_entity_load_avg(&p->se);
+		if (!task_fits_cpu(p, cpu))
+			return false;
+	}
+
+	return true;
+}
+
 /*
  * Try and locate an idle core/thread in the LLC cache domain.
  */
@@ -9582,6 +9602,12 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int wake_flags)
 			if (cpu != prev_cpu)
 				new_cpu = wake_affine(tmp, p, cpu, prev_cpu, sync);
 
+			if (sync && !sched_smt_active() &&
+			    new_cpu == cpu &&
+			    p->last_wakee == current &&
+			    prefer_sync_pair_cpu(p, cpu))
+				return cpu;
+
 			sd = NULL; /* Prefer wake_affine over balance flags */
 			break;
 		}

---
base-commit: 075b74841bd0065a3bda3440873c747938e69b68
change-id: 20260721-b4-sched-sync-wakeup-04d40cbeb1da

Best regards,
-- 
Shubhang Kaushik (Ampere) <sh@gentwo.org>

Re: [PATCH v4] sched/fair: Preserve wake-affine CPU for non-SMT reciprocal sync wakeups

[Shrikanth Hegde]

Hi Shubhang.

Please give time for discussion/reply for the people looking at
your patches. Even before I could respond to your v3, you have sent v4.
And your v4 doesn't addresses the concerns raised in v3.

On 8/4/26 7:15 AM, Shubhang Kaushik (Ampere) wrote:
> For WF_SYNC wakeups, wake_affine() may select the waker CPU, but the CFS
> wakeup path still passes that target to select_idle_sibling(). The idle
> CPU search can then move the wakee away from the wake-affine target.
> 
> Pipe-style ping-pong workloads expose this because the wakee is handed
> back and forth between two tasks. In that case, moving the wakee to
> another idle CPU can cost more than preserving the wake-affine waker CPU.
> 
> Use the existing last_wakee and wake_wide() state to identify narrow
> reciprocal WF_SYNC wakeups:
> 
>    A wakes B
>    B wakes A
>    A wakes B
>    ...
> 
> Handle only this narrow reciprocal case on non-SMT systems. Once the
> wake-affine path has selected or kept the waker CPU, preserve that target
> when the waker rq has no other runnable fair task. Return the waker CPU
> before select_idle_sibling() so the idle CPU search does not move this
> handoff away from the wake-affine target.
> 

Why on non-SMT? Why the same problem cannot happen in SMT systems?


> This does not define a generic WF_SYNC placement rule. Generic WF_SYNC
> wakeups continue through the existing wake_affine() and
> select_idle_sibling() behavior. SMT systems also continue through
> select_idle_sibling(), where idle sibling/core placement can be handled
> with SMT topology visible.
> 
> On asymmetric-capacity systems, still require the wakee to fit on the
> waker CPU.
> 
> Signed-off-by: Shubhang Kaushik (Ampere) <sh@gentwo.org>
> ---
> Tested on 80-core non-SMT Ampere Altra, tip:sched/core baseline.

Where is your LLC? Does it has multiple cores and currently
you end up choosing an idle core?
Your numbers below pretty much tell the same story.

> 
> perf bench sched pipe -l 1000000, 20 runs:

IIUC, sched pipe doesn't do any work apart from ping-pong.

> default:
>    3.985 -> 3.187 usec/op mean,   about 20.0% improvement
>    4.026 -> 3.181 usec/op median, about 21.0% improvement
> 
> taskset -c 78,79:
>    3.851 -> 3.144 usec/op mean,   about 18.4% improvement
>    3.804 -> 3.140 usec/op median, about 17.4% improvement
> 
> taskset -c 79:
>    3.055 -> 3.113 usec/op mean,   about 1.9% slower
>    3.045 -> 3.109 usec/op median, about 2.1% slower
> 

Which means you get the best result when it runs on same CPU.
The rest of the changes likely enforce that behavior. Then same issue is
prevalent in SMT world too.

> Hackbench process/thread pipe cases with 1/2/4/8 groups were within
> noise, with mean deltas from -1.8% to +3.7% over 10 runs.
> 
> Schbench normal mode at 8/40/80/240 workers and schbench pipe mode at
> 1/2/4/8 workers showed no material regression.
> 
> Baseline: tip/sched/core at 5186ef36909c

As I said in v3, before we add bells/whistles to sync path, i want
to know what is expected of sync behavior today.
And that should be documented in Documentation/scheduler/

Be it,
- current way of hint only and scheduler can still choose an idle core/idle cpu etc.
- Should it be enforcing it to waker cpu if waker cpu has only one task.
- Whatever the policy maybe.

Current api usage is tricky to use and effect is visible in real life workloads.
The case I mentioned in v3 of networking code using sync api leads to strange
results due to sync mechanism.
- It depends whether waker/wakee are running on same node.
- Result of wake_wide.
In other end, user sees inconsistent latency/throughput.

We can keep on adding minor changes to sync api path,
but one benchmark will benefit and one will suffer.
Having the behavior documented is a good start.

Peter, Ingo, Vincent, Mel, Prateek,
What do you guys think?

Re: [PATCH v4] sched/fair: Preserve wake-affine CPU for non-SMT reciprocal sync wakeups

[K Prateek Nayak]

Hello Shrikanth,

On 8/4/2026 10:10 AM, Shrikanth Hegde wrote:
> As I said in v3, before we add bells/whistles to sync path, i want
> to know what is expected of sync behavior today.
> And that should be documented in Documentation/scheduler/
> 
> Be it,
> - current way of hint only and scheduler can still choose an idle core/idle cpu etc.
> - Should it be enforcing it to waker cpu if waker cpu has only one task.
> - Whatever the policy maybe.
> 
> Current api usage is tricky to use and effect is visible in real life workloads.
> The case I mentioned in v3 of networking code using sync api leads to strange
> results due to sync mechanism.
> - It depends whether waker/wakee are running on same node.
> - Result of wake_wide.
> In other end, user sees inconsistent latency/throughput.
> 
> We can keep on adding minor changes to sync api path,
> but one benchmark will benefit and one will suffer.
> Having the behavior documented is a good start.
> 
> Peter, Ingo, Vincent, Mel, Prateek,
> What do you guys think?

Currently it is very arbitrary and WF_SYNC may, or may not, indicate a
true voluntary blocking behavior. For example, anon_pipe_read() uses a
wake_up_interruptible_sync_poll() to wake up writers once reader has
drained the pipe but if you think about it, why would the reader block
soon after just having the data it needed?

Here are the results on my Zen4 system from running perf bench
sched messaging (threads + pipes) at varying worker counts with
all wake_up_interruptible_sync_poll converted to
wake_up_interruptible_poll:

Test:                   tip                     no_sync
 1-groups:         3.79 (0.00 pct)         3.35 (11.60 pct)
 2-groups:         3.85 (0.00 pct)         3.41 (11.42 pct)
 4-groups:         4.02 (0.00 pct)         3.32 (17.41 pct)
 8-groups:         4.33 (0.00 pct)         4.38 (-1.15 pct)
16-groups:         6.09 (0.00 pct)         6.12 (-0.49 pct)
---

So seems like WF_SYNC hint on this machine with perf bench sched
messaging (thread + pipes) pattern is actually holding it back.
Lemme check processes ...

Test:                   tip                     no_sync
 1-groups:         3.48 (0.00 pct)         3.08 (11.49 pct)
 2-groups:         3.80 (0.00 pct)         3.07 (19.21 pct)
 4-groups:         3.91 (0.00 pct)         3.09 (20.97 pct)
 8-groups:         4.13 (0.00 pct)         4.10 (0.72 pct)
16-groups:         5.81 (0.00 pct)         5.74 (1.20 pct)

Similar stuff. At some point it was pretty bad for Zen3 but
situation might have changed since ¯\_(ツ)_/¯ I'll let you
know once I have a machine.

But ... If I have true 1:1 waiting on pipe as in the case of
"perf bench sched pipe -l 1000000" I go from ~2.5usecs/op on
average to ~4.2usecs/op which is close to a 50% increase in the
benchmark time so that WF_SYNC hint can also help if all we have
is looping over a read waiting for one page worth of write.

The way I look at WF_SYNC nowadays is that it indicates a local LLC
wakeup is beneficial. wake_wide() doesn't even consider WF_SYNc and
simply uses wake-wakee flips and then only at want_affine() do we
actually check the sync hint.

Most benefit come from wake_affine_idle() for the 1 task case where
target is set to current and select_idle_sibling() uses that as the
target from there on.

It could purely be a coincidence that it benefits at all - most of
these microbenchmark we have always hit the same two syscall (mostly
read() and write() in turns) and a most of benefit for those comes
from kernel instructions being primed in cache.

I remember a while back, removing the effect of WF_SYNC on the
networking side hampered a lot of performance - especially for
localhost communications. This one specifically
https://lore.kernel.org/lkml/20220711224704.1672831-1-libo.chen@oracle.com/

Let me see if things have miraculously changed there too but for
TCP sockets in real world, with blocking for ACKs, I think
WF_SYNC still makes sense there but I feel most of the benefits
from sync are a second order effect.

-- 
Thanks and Regards,
Prateek

Re: [PATCH v4] sched/fair: Preserve wake-affine CPU for non-SMT reciprocal sync wakeups

[Shrikanth Hegde]

Hi Prateek.

On 8/4/26 2:12 PM, K Prateek Nayak wrote:
> Hello Shrikanth,
> 
> On 8/4/2026 10:10 AM, Shrikanth Hegde wrote:
>> As I said in v3, before we add bells/whistles to sync path, i want
>> to know what is expected of sync behavior today.
>> And that should be documented in Documentation/scheduler/
>>
>> Be it,
>> - current way of hint only and scheduler can still choose an idle core/idle cpu etc.
>> - Should it be enforcing it to waker cpu if waker cpu has only one task.
>> - Whatever the policy maybe.
>>
>> Current api usage is tricky to use and effect is visible in real life workloads.
>> The case I mentioned in v3 of networking code using sync api leads to strange
>> results due to sync mechanism.
>> - It depends whether waker/wakee are running on same node.
>> - Result of wake_wide.
>> In other end, user sees inconsistent latency/throughput.
>>
>> We can keep on adding minor changes to sync api path,
>> but one benchmark will benefit and one will suffer.
>> Having the behavior documented is a good start.
>>
>> Peter, Ingo, Vincent, Mel, Prateek,
>> What do you guys think?
> 
> Currently it is very arbitrary and WF_SYNC may, or may not, indicate a
> true voluntary blocking behavior. For example, anon_pipe_read() uses a
> wake_up_interruptible_sync_poll() to wake up writers once reader has
> drained the pipe but if you think about it, why would the reader block
> soon after just having the data it needed?

Doesn't "perf bench sched pipe" also use anon_pipe_read/write?

-   27.49%     0.32%  sched-pipe       [kernel.kallsyms]                 [k] ksys_read
    - 27.17% ksys_read
       - 26.57% vfs_read
          - 22.33% anon_pipe_read

> 
> Here are the results on my Zen4 system from running perf bench
> sched messaging (threads + pipes) at varying worker counts with
> all wake_up_interruptible_sync_poll converted to
> wake_up_interruptible_poll:
> 
> Test:                   tip                     no_sync
>   1-groups:         3.79 (0.00 pct)         3.35 (11.60 pct)
>   2-groups:         3.85 (0.00 pct)         3.41 (11.42 pct)
>   4-groups:         4.02 (0.00 pct)         3.32 (17.41 pct)
>   8-groups:         4.33 (0.00 pct)         4.38 (-1.15 pct)
> 16-groups:         6.09 (0.00 pct)         6.12 (-0.49 pct)
> ---
> 
> So seems like WF_SYNC hint on this machine with perf bench sched
> messaging (thread + pipes) pattern is actually holding it back.
> Lemme check processes ...
> 
> Test:                   tip                     no_sync
>   1-groups:         3.48 (0.00 pct)         3.08 (11.49 pct)
>   2-groups:         3.80 (0.00 pct)         3.07 (19.21 pct)
>   4-groups:         3.91 (0.00 pct)         3.09 (20.97 pct)
>   8-groups:         4.13 (0.00 pct)         4.10 (0.72 pct)
> 16-groups:         5.81 (0.00 pct)         5.74 (1.20 pct)
> 
> Similar stuff. At some point it was pretty bad for Zen3 but
> situation might have changed since ¯\_(ツ)_/¯ I'll let you
> know once I have a machine.
> 
> But ... If I have true 1:1 waiting on pipe as in the case of
> "perf bench sched pipe -l 1000000" I go from ~2.5usecs/op on
> average to ~4.2usecs/op which is close to a 50% increase in the
> benchmark time so that WF_SYNC hint can also help if all we have
> is looping over a read waiting for one page worth of write.
> 
> The way I look at WF_SYNC nowadays is that it indicates a local LLC
> wakeup is beneficial. wake_wide() doesn't even consider WF_SYNc and
> simply uses wake-wakee flips and then only at want_affine() do we
> actually check the sync hint.
> 

Yes, it is difficult to say when would sync actually kick in.
Also, if we say hint, onus now falls on scheduler to optimize all
call sites.

Clearly comments around __wake_up_sync_key are outdated.

> Most benefit come from wake_affine_idle() for the 1 task case where
> target is set to current and select_idle_sibling() uses that as the
> target from there on.
> 
> It could purely be a coincidence that it benefits at all - most of
> these microbenchmark we have always hit the same two syscall (mostly
> read() and write() in turns) and a most of benefit for those comes
> from kernel instructions being primed in cache.
> 
> I remember a while back, removing the effect of WF_SYNC on the
> networking side hampered a lot of performance - especially for
> localhost communications. This one specifically
> https://lore.kernel.org/lkml/20220711224704.1672831-1-libo.chen@oracle.com/
> 
> Let me see if things have miraculously changed there too but for
> TCP sockets in real world, with blocking for ACKs, I think
> WF_SYNC still makes sense there but I feel most of the benefits
> from sync are a second order effect.
> 

But today it calls sync even for non-blocking.

Re: [PATCH v4] sched/fair: Preserve wake-affine CPU for non-SMT reciprocal sync wakeups

[K Prateek Nayak]

Hello Shrikanth,

On 8/4/2026 4:10 PM, Shrikanth Hegde wrote:
> Hi Prateek.
> 
> On 8/4/26 2:12 PM, K Prateek Nayak wrote:
>> Hello Shrikanth,
>>
>> On 8/4/2026 10:10 AM, Shrikanth Hegde wrote:
>>> As I said in v3, before we add bells/whistles to sync path, i want
>>> to know what is expected of sync behavior today.
>>> And that should be documented in Documentation/scheduler/
>>>
>>> Be it,
>>> - current way of hint only and scheduler can still choose an idle core/idle cpu etc.
>>> - Should it be enforcing it to waker cpu if waker cpu has only one task.
>>> - Whatever the policy maybe.
>>>
>>> Current api usage is tricky to use and effect is visible in real life workloads.
>>> The case I mentioned in v3 of networking code using sync api leads to strange
>>> results due to sync mechanism.
>>> - It depends whether waker/wakee are running on same node.
>>> - Result of wake_wide.
>>> In other end, user sees inconsistent latency/throughput.
>>>
>>> We can keep on adding minor changes to sync api path,
>>> but one benchmark will benefit and one will suffer.
>>> Having the behavior documented is a good start.
>>>
>>> Peter, Ingo, Vincent, Mel, Prateek,
>>> What do you guys think?
>>
>> Currently it is very arbitrary and WF_SYNC may, or may not, indicate a
>> true voluntary blocking behavior. For example, anon_pipe_read() uses a
>> wake_up_interruptible_sync_poll() to wake up writers once reader has
>> drained the pipe but if you think about it, why would the reader block
>> soon after just having the data it needed?
> 
> Doesn't "perf bench sched pipe" also use anon_pipe_read/write?
> 
> -   27.49%     0.32%  sched-pipe       [kernel.kallsyms]                 [k] ksys_read
>    - 27.17% ksys_read
>       - 26.57% vfs_read
>          - 22.33% anon_pipe_read

Exactly! Highly depends on the workload - if you are using pipe
for a signal, great, but if you are piping gigabytes of data,
and there is a continuous consumption, then co-locating the
readers and writers makes sense. This is probably why wake_wide
doesn't even care about the sync hint and makes a call purely on
waker_flips.

> 
>>
>> Here are the results on my Zen4 system from running perf bench
>> sched messaging (threads + pipes) at varying worker counts with
>> all wake_up_interruptible_sync_poll converted to
>> wake_up_interruptible_poll:
>>
>> Test:                   tip                     no_sync
>>   1-groups:         3.79 (0.00 pct)         3.35 (11.60 pct)
>>   2-groups:         3.85 (0.00 pct)         3.41 (11.42 pct)
>>   4-groups:         4.02 (0.00 pct)         3.32 (17.41 pct)
>>   8-groups:         4.33 (0.00 pct)         4.38 (-1.15 pct)
>> 16-groups:         6.09 (0.00 pct)         6.12 (-0.49 pct)
>> ---
>>
>> So seems like WF_SYNC hint on this machine with perf bench sched
>> messaging (thread + pipes) pattern is actually holding it back.
>> Lemme check processes ...
>>
>> Test:                   tip                     no_sync
>>   1-groups:         3.48 (0.00 pct)         3.08 (11.49 pct)
>>   2-groups:         3.80 (0.00 pct)         3.07 (19.21 pct)
>>   4-groups:         3.91 (0.00 pct)         3.09 (20.97 pct)
>>   8-groups:         4.13 (0.00 pct)         4.10 (0.72 pct)
>> 16-groups:         5.81 (0.00 pct)         5.74 (1.20 pct)
>>
>> Similar stuff. At some point it was pretty bad for Zen3 but
>> situation might have changed since ¯\_(ツ)_/¯ I'll let you
>> know once I have a machine.
>>
>> But ... If I have true 1:1 waiting on pipe as in the case of
>> "perf bench sched pipe -l 1000000" I go from ~2.5usecs/op on
>> average to ~4.2usecs/op which is close to a 50% increase in the
>> benchmark time so that WF_SYNC hint can also help if all we have
>> is looping over a read waiting for one page worth of write.
>>
>> The way I look at WF_SYNC nowadays is that it indicates a local LLC
>> wakeup is beneficial. wake_wide() doesn't even consider WF_SYNc and
>> simply uses wake-wakee flips and then only at want_affine() do we
>> actually check the sync hint.
>>
> 
> Yes, it is difficult to say when would sync actually kick in.
> Also, if we say hint, onus now falls on scheduler to optimize all
> call sites.
> 
> Clearly comments around __wake_up_sync_key are outdated.

At some point (1da177e4c3f4), sync + wakeup on waker's CPU also
inhibited resched_curr() and allowed the waker to naturally yield
the CPU (which is the reason for the UP comment) but today, we do
a resched_curr() unconditionally. Signs of different times.

> 
>> Most benefit come from wake_affine_idle() for the 1 task case where
>> target is set to current and select_idle_sibling() uses that as the
>> target from there on.
>>
>> It could purely be a coincidence that it benefits at all - most of
>> these microbenchmark we have always hit the same two syscall (mostly
>> read() and write() in turns) and a most of benefit for those comes
>> from kernel instructions being primed in cache.
>>
>> I remember a while back, removing the effect of WF_SYNC on the
>> networking side hampered a lot of performance - especially for
>> localhost communications. This one specifically
>> https://lore.kernel.org/lkml/20220711224704.1672831-1-libo.chen@oracle.com/
>>
>> Let me see if things have miraculously changed there too but for
>> TCP sockets in real world, with blocking for ACKs, I think
>> WF_SYNC still makes sense there but I feel most of the benefits
>> from sync are a second order effect.
>>
> 
> But today it calls sync even for non-blocking.

Ack. I think it just got carried over and for most real world
scenarios, it probably didn't made a difference until the processor
topologies diverged and we have many machines with many small LLCs
and many more machines with many large LLCs on the same socket.

-- 
Thanks and Regards,
Prateek

Re: [PATCH v4] sched/fair: Preserve wake-affine CPU for non-SMT reciprocal sync wakeups

[Shubhang]

Hi Shrikanth,

On Tue, 4 Aug 2026, Shrikanth Hegde wrote:

> Hi Shubhang.
>
> Please give time for discussion/reply for the people looking at
> your patches. Even before I could respond to your v3, you have sent v4.
> And your v4 doesn't addresses the concerns raised in v3.
>

Yes, v4 only narrowed the implementation. It moved the check after 
wake_affine(), only preserves the waker CPU when wake_affine() already 
selected it and keeps SMT out of the direct return path.

But it does not fully address your broader policy/documentation 
concern around what WF_SYNC should mean. I will hold off on another 
revision until there is more agreement there.

> On 8/4/26 7:15 AM, Shubhang Kaushik (Ampere) wrote:
>> Handle only this narrow reciprocal case on non-SMT systems. Once the
>> wake-affine path has selected or kept the waker CPU, preserve that target
>> when the waker rq has no other runnable fair task. Return the waker CPU
>> before select_idle_sibling() so the idle CPU search does not move this
>> handoff away from the wake-affine target.
>> 
>
> Why on non-SMT? Why the same problem cannot happen in SMT systems?
>

On SMT, returning the exact waker CPU is not obviously the right policy. 
An idle SMT sibling on the waker's core may be a better target, or the 
existing SIS choice may still be better depending on the workload and 
topology. Returning the waker CPU early would hide that SMT-specific 
choice from SIS. That was also the concern raised earlier by Vineeth, 
and is why I kept SMT out of this patch.

I did try the SIS-based direction Prateek suggested on the non-SMT Altra 
system. It improved the baseline, but it was still slower than preserving 
the wake-affine CPU directly for the non-SMT reciprocal case.

So my intent with v4 was to keep only that narrower non-SMT case here, 
while seperating the SMT, core sibling question to the separate SIS side 
discussion.

>> ---
>> Tested on 80-core non-SMT Ampere Altra, tip:sched/core baseline.
>
> Where is your LLC? Does it has multiple cores and currently
> you end up choosing an idle core?
> Your numbers below pretty much tell the same story.
>

The v4 numbers were collected with SLC-as-L3 disabled. On Ampere Altra
system, sched_verbose reports:

   domain0: MC,  SD_SHARE_LLC
   domain1: PKG, SD_SHARE_LLC

So, from the scheduler's view, there is still a multi-CPU SD_SHARE_LLC
domain in this configuration. In the baseline, wake_affine() can select
the current waker CPU for the sync handoff, but that target still goes
through select_idle_sibling(). SIS can then move the wakee to another
idle CPU in that domain.

>> 
>> perf bench sched pipe -l 1000000, 20 runs:
>
> IIUC, sched pipe doesn't do any work apart from ping-pong.
>

Yes, agreed. sched pipe mostly exposes handoff cost. I am not trying 
to use it to define generic WF_SYNC behavior.

>> default:
>>    3.985 -> 3.187 usec/op mean,   about 20.0% improvement
>>    4.026 -> 3.181 usec/op median, about 21.0% improvement
>> 
>> taskset -c 78,79:
>>    3.851 -> 3.144 usec/op mean,   about 18.4% improvement
>>    3.804 -> 3.140 usec/op median, about 17.4% improvement
>> 
>> taskset -c 79:
>>    3.055 -> 3.113 usec/op mean,   about 1.9% slower
>>    3.045 -> 3.109 usec/op median, about 2.1% slower
>> 
>
> Which means you get the best result when it runs on same CPU.
> The rest of the changes likely enforce that behavior. Then same issue is
> prevalent in SMT world too.
>

Yes, same locality issue can show up on SMT too. The distinction 
is the target. For non-SMT, the local target can be the waker CPU. For 
SMT, the target may need to be the waker's core, an idle SMT sibling, or 
the normal SIS result. That needs the SMT topology to remain visible.

>> Hackbench process/thread pipe cases with 1/2/4/8 groups were within
>> noise, with mean deltas from -1.8% to +3.7% over 10 runs.
>> 
>> Schbench normal mode at 8/40/80/240 workers and schbench pipe mode at
>> 1/2/4/8 workers showed no material regression.
>> 
>> Baseline: tip/sched/core at 5186ef36909c
>
> As I said in v3, before we add bells/whistles to sync path, i want
> to know what is expected of sync behavior today.
> And that should be documented in Documentation/scheduler/
>

I agree. Currently WF_SYNC is a hint, not a strict placement rule.
The fair wakeup path may use WF_SYNC in wake_affine(), but wake_wide(), 
topology, affinity, capacity, idle state and load can still affect where 
the wakee actually runs.

Prateek's data also shows why this is tricky. WF_SYNC can help a strict 
1:1 pipe handoff, but it can hurt other pipe messaging patterns. So I 
can see that documenting the current behavior first is useful, especially 
because the waitqueue comment reads stronger than what the wakeup path 
actually guarantees today.

I can help with Documentation/scheduler/ update describing the current 
WF_SYNC behavior and the wakeup path before sending another version 
of this optimization.

> Be it,
> - current way of hint only and scheduler can still choose an idle core/idle 
> cpu etc.
> - Should it be enforcing it to waker cpu if waker cpu has only one task.
> - Whatever the policy maybe.
>
> Current api usage is tricky to use and effect is visible in real life 
> workloads.
> The case I mentioned in v3 of networking code using sync api leads to strange
> results due to sync mechanism.
> - It depends whether waker/wakee are running on same node.
> - Result of wake_wide.
> In other end, user sees inconsistent latency/throughput.
>
> We can keep on adding minor changes to sync api path,
> but one benchmark will benefit and one will suffer.
> Having the behavior documented is a good start.
>
> Peter, Ingo, Vincent, Mel, Prateek,
> What do you guys think?
>

Please do let me know if you have any other suggestions in the meantime.

Regards,
Shubhang Kaushik