Re: [PATCH v2 0/9] sched/kvm: Semantics-aware vCPU scheduling for oversubscribed KVM

[Richie Buturla <richie@linux.ibm.com>]

On 01/04/2026 10:34, Wanpeng Li wrote:
> Hi Christian,
> On Thu, 26 Mar 2026 at 22:42, Christian Borntraeger
> <borntraeger@linux.ibm.com> wrote:
>> Am 19.12.25 um 04:53 schrieb Wanpeng Li:
>>> From: Wanpeng Li <wanpengli@tencent.com>
>>>
>>> This series addresses long-standing yield_to() inefficiencies in
>>> virtualized environments through two complementary mechanisms: a vCPU
>>> debooster in the scheduler and IPI-aware directed yield in KVM.
>>>
>>> Problem Statement
>>> -----------------
>>>
>>> In overcommitted virtualization scenarios, vCPUs frequently spin on locks
>>> held by other vCPUs that are not currently running. The kernel's
>>> paravirtual spinlock support detects these situations and calls yield_to()
>>> to boost the lock holder, allowing it to run and release the lock.
>>>
>>> However, the current implementation has two critical limitations:
>>>
>>> 1. Scheduler-side limitation:
>>>
>>>      yield_to_task_fair() relies solely on set_next_buddy() to provide
>>>      preference to the target vCPU. This buddy mechanism only offers
>>>      immediate, transient preference. Once the buddy hint expires (typically
>>>      after one scheduling decision), the yielding vCPU may preempt the target
>>>      again, especially in nested cgroup hierarchies where vruntime domains
>>>      differ.
>>>
>>>      This creates a ping-pong effect: the lock holder runs briefly, gets
>>>      preempted before completing critical sections, and the yielding vCPU
>>>      spins again, triggering another futile yield_to() cycle. The overhead
>>>      accumulates rapidly in workloads with high lock contention.
>> Wanpeng,
>>
>> late but not forgotten.
>>
>> So Richie Buturla gave this a try on s390 with some variations but still
>> without cgroup support (next step).
>> The numbers look very promising (diag 9c is our yieldto hypercall). With
>> super high overcommitment the benefit shrinks again, but results are still
>> positive. We are probably running into other limits.
>>
>> 2:1 Overcommit Ratio:
>> diag9c calls:                       225,804,073 →  213,913,266  (-5.3%)
>> Dbench thrpt (per-run mean):        +1.3%
>> Dbench thrpt (per-run median):      +0.8%
>> Dbench thrpt (total across runs):   +1.3%
>> Dbench thrpt (avg/VM):              +1.3%
>>
>> 4:1:
>> diag9c calls:                       833,455,152 →  556,597,627 (-33.2%)
>> Dbench thrpt (per-run mean):        +7.2%
>> Dbench thrpt (per-run median):      +8.5%
>> Dbench thrpt (total across runs):   +7.2%
>> Dbench thrpt (avg/VM):              +7.2%
>>
>>
>> 6:1:
>> diag9c calls:                       967,501,378 →  737,178,419 (-23.8%)
>> Dbench thrpt (per-run mean):        +5.1%
>> Dbench thrpt (per-run median):      +4.8%
>> Dbench thrpt (total across runs):   +5.1%
>> Dbench thrpt (avg/VM):              +5.1%
>>
>>
>>
>> 8:1:
>> diag9c calls:                       872,165,596 → 653,481,530 (-25.1%)
>> Dbench thrpt (per-run mean):        +11.5%
>> Dbench thrpt (per-run median):      +11.4%
>> Dbench thrpt (total across runs):   +11.5%
>> Dbench thrpt (avg/VM):              +11.5%
>>
>> 9:1:
>> diag9c calls:                       809,384,976  → 587,597,163 (-27.4%)
>> Dbench thrpt (per-run mean):        +4.5%
>> Dbench thrpt (per-run median):      +4.0%
>> Dbench thrpt (total across runs):   +4.5%
>> Dbench thrpt (avg/VM):              +4.5%
>>
>>
>> 10:1:
>> diag9c calls:                       711,772,971 → 477,448,374 (-32.9%)
>> Dbench thrpt (per-run mean):        +3.6%
>> Dbench thrpt (per-run median):      +1.6%
>> Dbench thrpt (total across runs):   +3.6%
>> Dbench thrpt (avg/VM):              +3.6%
> Thanks Christian, and thanks to Richie for running this on s390. :)
>
> This is very valuable independent data. A few things stand out to me:
>
> - The consistent reduction in diag9c calls across all overcommit
> ratios (up to -33.2% at 4:1) confirms that the directed yield
> improvements are effective at reducing unnecessary yield-to
> hypercalls, not just on x86 but across architectures.
> - The fact that these results are without cgroup support is actually
> informative: it tells us the core yield improvement carries its weight
> on its own, which helps me scope the next revision more tightly.
> - The diminishing-but-still-positive returns at very high overcommit
> (9:1, 10:1) match what I see on x86 as well — other bottlenecks start
> dominating but the mechanism does not regress.
>
> Btw, which kernel version were these results collected on?
>
> Regards,
> Wanpeng
>
Hi Wanpeng,

I collected these results on a 6.19 kernel - which should also include 
the existing fixes for yielding and forfeiting vruntime on yield that K 
Prateek mentioned.