Re: [PATCH v3] sched/fair: Prefer waker CPU for non-SMT reciprocal sync wakeups

[Shrikanth Hegde <sshegde@linux.ibm.com>]

Hello.

After seeing this and vineeth's patch,
https://lore.kernel.org/all/20260801035532.260625-1-vineethr@linux.ibm.com/
I am bit confused on the policy we are trying to do for sync. Find the details
below.

On 7/28/26 5:28 AM, Shubhang Kaushik (Ampere) wrote:
> Pipe-style ping-pong workloads can be dominated by handoff cost. In
> such cases, placing the wakee on an idle CPU can be slower than keeping
> the pair on the same runqueue.
> 
> Use the existing last_wakee and wake_wide() state to identify narrow
> reciprocal WF_SYNC wakeups:
> 
> A wakes B
> B wakes A
> A wakes B
> ...
> 
> When the wake-affine domain allows SD_WAKE_AFFINE, prefer the waker CPU
> for these narrow reciprocal handoffs on non-SMT systems. Do so only when
> the waker CPU has no other runnable fair task and the wakee fits there on
> asymmetric-capacity systems.
> 
> SMT systems, and wakeups that do not match this pattern, continue through
> the existing wake_affine() and select_idle_sibling() path.
> 

I think we need to think this on the policy notion rather than a usecase specific.
These are api's available to other susystems to make specific call based on its
understand of its requirement. i.e
wake_up_interruptible_sync_poll
vs
wake_up, wake_up_interruptible


If we look at __wake_up_sync*, It says,

/**
  * __wake_up_sync_key - wake up threads blocked on a waitqueue.
  * @wq_head: the waitqueue
  * @mode: which threads
  * @key: opaque value to be passed to wakeup targets
  *
  * The sync wakeup differs that the waker knows that it will schedule
  * away soon, so while the target thread will be woken up, it will not
  * be migrated to another CPU - ie. the two threads are 'synchronized'
  * with each other. This can prevent needless bouncing between CPUs.
  *
  * On UP it can prevent extra preemption.
  *
  * If this function wakes up a task, it executes a full memory barrier before
  * accessing the task state.
  */
void __wake_up_sync_key(struct wait_queue_head *wq_head, unsigned int mode,
                         void *key)
{
         if (unlikely(!wq_head))
                 return;

         __wake_up_common_lock(wq_head, mode, 1, WF_SYNC, key);
}

So, with that, we use introduce the notion that, scheduler wakeup will honor
the sync behaviour based on underlying arch/hw, how will callers ever
know. For example, same SMT system can have all its siblings off, and now it
is !smt system.

There is already use/abuse of sync api in Networking staff.
A recent discussion on it,
https://lore.kernel.org/all/amI22o9MwDoGcBMl@linux.ibm.com/
I am assuming there would be more.

So, What should sync wakeup should do vs non-sync wakeup?
- Should it chose waker's CPU if waker is the only one running.
   - Should it be always?
   - Should it be under specific case such !smt, cas specific?
- Should it still chose an idle core first, if not chose waker CPU/Sibling?
- Should it fallback to waker's LLC vs current LLC. and then choose a CPU
   in that LLC or choose a recently used cpu, prev_cpu etc? (Current logic)

I think we should define the policy for it. (if it is not too late for it)
No?