Re: [PATCH v3 0/2] accel/tcg/tcg-accel-ops-rr: ensure fairness with icount

["Philippe Mathieu-Daudé" <philmd@linaro.org>]

On 29/4/23 11:28, Richard Henderson wrote:
> On 4/27/23 03:09, Jamie Iles wrote:
>> From: Jamie Iles <jiles@qti.qualcomm.com>
>>
>> The round-robin scheduler will iterate over the CPU list with an
>> assigned budget until the next timer expiry and may exit early because
>> of a TB exit.  This is fine under normal operation but with icount
>> enabled and SMP it is possible for a CPU to be starved of run time and
>> the system live-locks.
>>
>> For example, booting a riscv64 platform with '-icount
>> shift=0,align=off,sleep=on -smp 2' we observe a livelock once the kernel
>> has timers enabled and starts performing TLB shootdowns.  In this case
>> we have CPU 0 in M-mode with interrupts disabled sending an IPI to CPU
>> 1.  As we enter the TCG loop, we assign the icount budget to next timer
>> interrupt to CPU 0 and begin executing where the guest is sat in a busy
>> loop exhausting all of the budget before we try to execute CPU 1 which
>> is the target of the IPI but CPU 1 is left with no budget with which to
>> execute and the process repeats.
>>
>> We try here to add some fairness by splitting the budget across all of
>> the CPUs on the thread fairly before entering each one.  The CPU count
>> is cached on CPU list generation ID to avoid iterating the list on each
>> loop iteration.  With this change it is possible to boot an SMP rv64
>> guest with icount enabled and no hangs.
>>
>> New in v3 (address feedback from Richard Henderson):
>>   - Additional patch to use QEMU_LOCK_GUARD with qemu_cpu_list_lock where
>>     appropriate
>>   - Move rr_cpu_count() call to be conditional on icount_enabled()
>>   - Initialize cpu_budget to 0
>>
>> Jamie Iles (2):
>>    cpu: expose qemu_cpu_list_lock for lock-guard use
>>    accel/tcg/tcg-accel-ops-rr: ensure fairness with icount
> 
> It appears as if one of these two patches causes a failure in replay, e.g.
> 
>    https://gitlab.com/rth7680/qemu/-/jobs/4200609234#L4162
> 
> Would you have a look, please?

cpu count only going forward in rr_cpu_count()?