[Qemu-devel] Re: Role of qemu_fair_mutex

[Anthony Liguori <anthony@codemonkey.ws>]

On 01/04/2011 03:39 PM, Marcelo Tosatti wrote:
> On Tue, Jan 04, 2011 at 08:17:26AM -0600, Anthony Liguori wrote:
>    
>> On 01/03/2011 04:01 AM, Avi Kivity wrote:
>>      
>>> On 01/03/2011 11:46 AM, Jan Kiszka wrote:
>>>        
>>>> Hi,
>>>>
>>>> at least in kvm mode, the qemu_fair_mutex seems to have lost its
>>>> function of balancing qemu_global_mutex access between the io-thread and
>>>> vcpus. It's now only taken by the latter, isn't it?
>>>>
>>>> This and the fact that qemu-kvm does not use this kind of lock made me
>>>> wonder what its role is and if it is still relevant in practice. I'd
>>>> like to unify the execution models of qemu-kvm and qemu, and this lock
>>>> is the most obvious difference (there are surely more subtle ones as
>>>> well...).
>>>>
>>>>          
>>> IIRC it was used for tcg, which has a problem that kvm doesn't
>>> have: a tcg vcpu needs to hold qemu_mutex when it runs, which
>>> means there will always be contention on qemu_mutex.  In the
>>> absence of fairness, the tcg thread could dominate qemu_mutex and
>>> starve the iothread.
>>>        
>> No, it's actually the opposite IIRC.
>>
>> TCG relies on the following behavior.   A guest VCPU runs until 1)
>> it encounters a HLT instruction 2) an event occurs that forces the
>> TCG execution to break.
>>
>> (2) really means that the TCG thread receives a signal.  Usually,
>> this is the periodic timer signal.
>>
>> When the TCG thread, it needs to let the IO thread run for at least
>> one iteration.  Coordinating the execution of the IO thread such
>> that it's guaranteed to run at least once and then having it drop
>> the qemu mutex long enough for the TCG thread to acquire it is the
>> purpose of the qemu_fair_mutex.
>>      
> Its the vcpu threads that starve the IO thread.
>    

I'm not sure if this is a difference in semantics or if we're not 
understanding each other.

With TCG, the VCPU thread will dominate the qemu_mutex and cause the IO 
thread to contend heavily on it.

But the IO thread can always force TCG to exit it's loop (and does so 
when leaving select()).  So the TCG thread make keep the IO thread 
hungry, but it never "starves" it.

OTOH, the TCG thread struggles to hand over execution to the IO thread 
while making sure that it gets back the qemu_mutex in a timely fashion.  
That's the tricky part.  Avi's point is that by giving up the lock at 
select time, we prevent starvation but my concern is that because the 
time between select intervals is unbounded (and potentially very, very 
lock), it's effectively starvation.

Regards,

Anthony Liguori