Re: [Qemu-devel] [PATCH] RFC: Add new block driver for the VDI format

[Avi Kivity <avi@redhat.com>]

On 08/03/2009 05:25 AM, Anthony Liguori wrote:
> Avi Kivity wrote:
>> On 07/06/2009 04:37 PM, Anthony Liguori wrote:
>>>
>>> I'd really like to get rid of synchronous IO functions in the block 
>>> layer.  One way to do this is to insist that all new block drivers 
>>> only implement the AIO functions.
>>>
>>> I think we should make this decree but I'd like to know if other 
>>> people think this is unreasonable first.  One potential model of 
>>> block drivers would involve synchronous IO and threads.  I'm not a 
>>> big fan of that model and I don't think it's an easy conversion from 
>>> today's synchronous IO drivers to that model because the locking and 
>>> re-entrance needs careful consideration.
>>>
>>
>> I agree that sync+threads is not easy, but well performing async is 
>> much, much harder.  Consider that qcow2 still has synchronous 
>> operations, and that eliminating the RMW when writing a partial 
>> cluster concurrently (a very common operation with 64K clusters) is 
>> very hard to do ayncly and much easier syncly.
>
> Supporting parallel RMW operations is certainly difficult, but you're 
> confusing parallel RMW ops with asynchronous RMW ops.  You just have 
> to queue requests and handle them in order.  It's only mildly more 
> difficult to deal with asynchronous I/O and it avoids all the 
> nastiness associated with threads and locking.

I'm talking about a guest sequential write emitted as multiple adjacent 
requests in parallel.  Currently we'll write the first request and the 
second request in different locations, then do a rmw to merge the two 
blocks (I think...).

> Fundamentally, threads don't help the RMW problem because you probably 
> would just hold a look for the entire RMW operation so you're 
> effectively queuing any RMW op.

You do get some queuing but layout is improved, and it's not a W-RMW; 
instead it's a W-WW.

Theoretically anything you can do with threads you can do with async 
operations but experience has proven that async is much more difficult.  
Consider the last qcow2 bug.

>
>> Given in addition the large numbers of format drivers, I think we 
>> should prefer sync+threads over trying to convert all format drivers 
>> to full async.
>
> It's just shifting the problem from one place to another.  Instead of 
> figuring out the state machine, you have to figure out how to do the 
> locking.  The danger of the later is that it gives you the illusion 
> that it's an easy problem and is therefore prone to error.

Locking _is_ an easier problem than figuring out the state machine.  I 
can't prove this but there's numerous anecdotal evidence on the subject.

-- 
error compiling committee.c: too many arguments to function