[Qemu-devel] Re: block: format vs. protocol, and how they stack

[Anthony Liguori <anthony@codemonkey.ws>]

On 06/21/2010 09:01 AM, Kevin Wolf wrote:
> Am 21.06.2010 15:37, schrieb Anthony Liguori:
>    
>> On 06/21/2010 08:30 AM, Kevin Wolf wrote:
>>      
>>> Am 21.06.2010 15:09, schrieb Anthony Liguori:
>>>
>>>        
>>>> On 06/21/2010 03:19 AM, Kevin Wolf wrote:
>>>>
>>>>          
>>>>> Am 20.06.2010 12:51, schrieb Avi Kivity:
>>>>>
>>>>>
>>>>>            
>>>>>> On 06/18/2010 03:59 PM, Markus Armbruster wrote:
>>>>>>
>>>>>>
>>>>>>              
>>>>>>> The code is pretty confused about format vs. protocol, and so are we.
>>>>>>> Let's try to figure them out.
>>>>>>>
>>>>>>>     From cruising altitude, all this format, protocol, stacking business
>>>>>>> doesn't matter.  We provide a bunch of arguments, and get an image.
>>>>>>>
>>>>>>> If you look more closely, providing that image involves sub-tasks.  One
>>>>>>> is to haul bits.  Another one is to translate between bits in different
>>>>>>> formats.
>>>>>>>
>>>>>>> Working hypothesis:
>>>>>>>
>>>>>>> * A protocol hauls image bits.  Examples: file, host_device, nbd.
>>>>>>>
>>>>>>> * A format translates image formats.  Examples: raw, qcow2.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>                
>>>>>> Is there a reason to make the distinction?  Is there a reason to expose
>>>>>> the distinction to the user?
>>>>>>
>>>>>>
>>>>>>              
>>>>> There are good reasons to make that distinction internally. There's no
>>>>> need to expose it to the user - the question is if it helps or not.
>>>>>
>>>>>
>>>>>            
>>>> If we drop the distinction, then I think the remaining issue is how to
>>>> expose the stacking to a user.
>>>>
>>>> Right now, we could have a syntax like:
>>>>
>>>> -blockdev format=file,file=image.qcow2,id=base  \
>>>> -blockdev format=qcow2,backing_dev=base,id=blk1
>>>>
>>>> backing_dev is a sucky name, but hopefully the point is clear.  I think
>>>> the following would be a better user syntax:
>>>>
>>>> -blockdev format=qcow2,file=image.qcow2,id=blk1
>>>>
>>>> I think the easiest way to support this is to make qcow2 take a file
>>>> parameter and have it open the file with default options.  For users
>>>> that need anything more sophisticated a user has to use the former syntax.
>>>>
>>>>          
>>> Not only qcow2, but also raw, qcow, vmdk, vdi, bochs, cow, dmg, ...
>>>
>>> In short: Any format needs an underlying protocol. You may not call it
>>> by its name, but that's effectively what you'd implement. And if you
>>> implemented it in each format driver instead of generic code, you'd be
>>> doing a bad implementation.
>>>
>>>        
>> Sure.  I don't think it would be all that difficult to implement in
>> common code.
>>      
> Probably not. I mean, if you ignore blkdebug for a moment, this is
> what's implemented today. But it makes a difference between formats and
> protocols: If you say format=qcow2, you get qcow2 on file. If you say
> format=file, you get just file with no other protocol because file is
> already one (I think what you really should get is an error message, but
> that's another topic...)
>
>    
>>> The more I think about it, the more I believe that the logic of how qemu
>>> handles things is made much clearer if we actually call it by its name
>>> and expose the distinction to the user.
>>>
>>> "If there is no protocol specified, qemu will pick one automatically"
>>> vs. "If you specify an image in raw, qcow2, qcow, vmdk, vdi, bochs, cow,
>>> dmg or blkdebug format and you have no backing_dev specified, qemu will
>>> pick one automatically; it won't do so for images in file, host_device,
>>> host_flopy, host_cdrom, nbd, http or vvfat format." It's an easy choice.
>>>
>>>        
>> Are you basically saying:
>>
>> [1] -blockdev format=qcow2,protocol=file,file=foo.img,id=blk1
>>
>> Because what I was suggesting is that we don't allow protocol=XX here.
>> We would try to guess the protocol from foo.img.  If a user wishes to
>> override this, they should do the full syntax of:
>>
>> [2] -blockdev protocol=host_dev,file=/dev/sda,id=base \
>> -blockdev format=qcow2,backing_dev=base,id=blk1
>>      
> No, what I'm saying is that even in your model
>
>    -blockdev format=qcow2,file=image.qcow2,id=blk1
>
> becomes qcow2 ->  file automatically, whereas
>
>    -blockdev format=vvfat,file=/tmp/dir/,id=blk1
>
> doesn't become vvfat ->  file, but stays just vvfat.
>
> This is the difference between a protocol and a format.

I can appreciate the desire to keep protocols and formats as an internal 
distinction but as a user visible concept, I think your two examples 
highlight why exposing protocols as formats make sense.  A user doesn't 
necessarily care what's happening under the cover.  I think:

-blockdev format=qcow2,file=image.qcow2,id=blk1

and:

-blockdev protocol=vvfat,file=/tmp/dir,id=blk1

Would cause a bit of confusion.  It's not immediately clear why vvfat is 
a protocol and qcow2 isn't.  It's really an implementation detail that 
we implement qcow2 on top of a "protocol" called file.

>   The logic is
> that you need one protocol and if you don't specify it explicitly, qemu
> will guess it.
>
> If you decide to avoid calling them format and protocol, you need to
> talk about "raw, qcow2, qcow2, vmdk, ..." instead of "format" and about
> "file, host_device, ..." instead of "protocol" - because you have
> removed the terms, but not the concepts.
>    

 From the user interface.  For the internal implementation, if you think 
we should distinguish protocols from formats, I think that's a fair 
thing to do.

> I think the user interface is clearer if we call them by their name.
>    

I think the user interface for you is clearer, but for the majority of 
users including most developers without intimate knowledge of the block 
layer, the distinction will only lead to confusion.

>> But we're also suggesting s/protocol/format/ because the distinction
>> doesn't seem to be important unless you try to support syntax [1] and
>> you need a mechanism to specify the leaf format (which you're calling a
>> protocol).
>>      
> Syntax [1] would be a nice abbreviation when you need to specify the
> protocol explicitly, but it's not what decides about the concept.
>    

I don't think specifying the protocol explicitly is an extremely common 
use case but if you disagree, we can certainly consider abbreviated 
syntax.  I agree though that this is not core to the concepts.

Regards,

Anthony Liguori

> Kevin
>