Re: [Qemu-devel] Re: KVM call minutes for Feb 1

[Anthony Liguori <anthony@codemonkey.ws>]

On 02/01/2011 11:34 AM, Jan Kiszka wrote:
> On 2011-02-01 18:20, Anthony Liguori wrote:
>    
>> On 02/01/2011 11:03 AM, Jan Kiszka wrote:
>>      
>>> On 2011-02-01 17:53, Anthony Liguori wrote:
>>>
>>>        
>>>> On 02/01/2011 10:36 AM, Jan Kiszka wrote:
>>>>
>>>>          
>>>>> On 2011-02-01 16:54, Chris Wright wrote:
>>>>>
>>>>>
>>>>>            
>>>>>> KVM upstream merge: status, plans, coordination
>>>>>> - Jan has a git tree, consolidating
>>>>>> - qemu-kvm io threading is still an issue
>>>>>> - Anthony wants to just merge
>>>>>>      - concerns with non-x86 arch and merge
>>>>>>      - concerns with big-bang patch merge and following stability
>>>>>> - post 0.14 conversion to glib mainloop, non-upstreamed qemu-kvm will be
>>>>>>      a problem if it's not there by then
>>>>>> - testing and nuances are still an issue (e.g. stefan berger's mmio read issue)
>>>>>> - qemu-kvm still evolving, needs to get sync'd or it will keep diverging
>>>>>> - 2 implementations of main init, cpu init, Jan has merged them into one
>>>>>>      - qemu-kvm-x86.c file that's only a few hundred lines
>>>>>> - review as one patch to see the fundamental difference
>>>>>>
>>>>>>
>>>>>>              
>>>>> More precisely, my current work flow is to pick some function(s), e.g.
>>>>> kvm_cpu_exec/kvm_run, and start wondering "What needs to be done to
>>>>> upstream so that qemu-kvm could use that implementation?". If they
>>>>> differ, the reasons need to be understood and patched away, either by
>>>>> fixing/enhancing upstream or simplifying qemu-kvm. Once the upstream
>>>>> changes are merged back, a qemu-kvm patch is posted to switch to that
>>>>> version.
>>>>>
>>>>> Any help will be welcome, either via review of my subtle regressions or
>>>>> on resolving concrete differences.
>>>>>
>>>>> E.g. posix-aio-compat.c: Why does qemu-kvm differ here? If it's because
>>>>> of its own iothread code, can we wrap that away or do we need to
>>>>> consolidate the threading code first? Or do we need to fix something in
>>>>> upstream?
>>>>>
>>>>>
>>>>>            
>>>> I bet it's the eventfd thing.  It's arbitrary.  If you've got a small
>>>> diff post your series, I'd be happy to take a look at it and see what I
>>>> can explain.
>>>>
>>>>
>>>>          
>>> Looks like it's around signalfd and its emulation:
>>>
>>>        
>> I really meant the compatfd thing.
>>
>> signalfd can't really be emulated properly so in upstream we switched to
>> a pipe() which Avi didn't like.
>>
>> But with glib, this all goes away anyway so we should just drop the
>> qemu-kvm changes and use the upstream version.  Once we enable I/O
>> thread in qemu.git, we no longer need to use signals for I/O completion
>> which I think everyone would agree is a better solution.
>>      
> Don't understand: If we do not need SIGIO for AIO emulation in threaded
> mode, why wasn't that stubbed out already?

Historically, we used posix-aio which only notifies completion based on 
signals.

However, because of the signal/select race, there's nothing useful that 
can be done in the signal handler.  So we then added signalfd such that 
we could poll the signal safely from the select loop.

However, signalfd cannot be emulated reliably which was the approach we 
had been using since signalfd is only available in newer kernels.  So we 
switched to having the signal handler write to a pipe() which gives us 
an fd based notification mechanism.  While qemu.git made that change, 
qemu-kvm.git carried the signalfd version probably because we just 
didn't argue about it enough back then.

Now, since we haven't used posix-aio in a very long time, there's really 
no reason to go through this signal non-sense in the first place.  We 
can just make the helper threads write to a file descriptor (eventfd or 
pipe).  At one point, that's what we did in the tree.  However, when TCG 
does TB chaining, the only thing that will break a guest out of a tight 
loop is a signal delivery.  In single threaded TCG, if the guest doesn't 
have a periodic timer enabled and issues an I/O operation, the 
signalling is posix-aio-compat would break it out of the TB loop to let 
it handle the completion.  When we got rid of it, we broke these guests 
with the symptom of I/Os not completing until you typed a key in the 
serial console.

However, once we enable the I/O thread for TCG, the I/O thread can issue 
a select() statement while the TCG thread is doing chaining.  As long as 
we send a signal to the TCG thread after select() returns and then wait 
for qemu_mutex to be released, this problem doesn't exist anymore.

So enabling the I/O thread universally means we can drop signaling in 
posix-aio.

Regards,

Anthony Liguori

>   If that helps reducing
> worries about the signalfd emulation (which is likely a non-issue anyway
> as anyone with serious workload should run a kernel with such support).
>
> Jan
>
>