Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

[Anthony Liguori <anthony@codemonkey.ws>]

On 02/27/2011 03:10 AM, Avi Kivity wrote:
> On 02/24/2011 07:58 PM, Anthony Liguori wrote:
>>> If you move the cdrom to a different IDE channel, you have to update 
>>> the stateful non-config file.
>>>
>>> Whereas if you do
>>>
>>>    $ qemu-img create -f cd-tray -b ~/foo.img ~/foo-media-tray.img
>>>    $ qemu -cdrom ~/foo-media-tray.img
>>>
>>> the cd-rom tray state will be tracked in the image file.
>>
>>
>> Yeah, but how do you move it? 
>
> There is no need to move the file at all.  Simply point the new drive 
> at the media tray.

No, I was asking, how do you move the cdrom to a different IDE channel.  
Are you using QMP?  Are you changing the command line arguments?

>
>> If you do a remove/add through QMP, then the config file will reflect 
>> things just fine.
>
> If all access to the state file is through QMP then it becomes more 
> palatable.  A bit on that later.

As I think I've mentioned before, I hadn't really thought about an 
opaque state file but I'm not necessary opposed to it.  I don't see an 
obvious advantage to making it opaque but I agree it should be 
accessible via QMP.

>> If you want to do it outside of QEMU, then you can just ignore the 
>> config file and manage all of the state yourself.  But it's never 
>> going to work as well (it will be racy) and you're pushing a 
>> tremendous amount of knowledge that ultimately belongs in QEMU (what 
>> state needs to persist) to something that isn't QEMU which means it's 
>> probably not going to be done correctly.
>>
>> I know you're a big fan of the omnipotent management tool but my 
>> experience has been that we need to help the management tooling folks 
>> more by expecting less from them.
>
> I thought that's what I'm doing by separating the state out.  It's 
> easy for management to assemble configuration from their database and 
> convert it into a centralized representation (like a qemu command 
> line).  It's a lot harder to disassemble a central state 
> representation and move it back to the database.
>
> Using QMP is better than directly accessing the state file since qemu 
> does the disassembly for you (provided the command references the 
> device using its normal path, not some random key).  The file just 
> becomes a way to survive a crash, and all management needs to know 
> about is to make it available and back it up.  But it means that 
> everything must be done via QMP, including assembly of the machine, 
> otherwise the state file can become stale.
>
> Separating the state out to the device is even easier, since 
> management is already expected to take care of disk images.  All 
> that's needed is to create the media tray image once, then you can 
> forget about it completely.

Except that instead of having one state file, we might have a dozen 
additional "device state" files.

>>>
>>> Again the question is who is the authoritative source of the 
>>> configuration.  Is it the management tool or is it qemu?
>>
>> QEMU.   No question about it.  At any point in time, we are the 
>> authoritative source of what the guest's configuration is.  There's 
>> no doubt about it.  A management tool can try to keep up with us, but 
>> ultimately we are the only ones that know for sure.
>>
>> We have all of this information internally.  Just persisting it is 
>> not a major architectural change.  It's something we should have been 
>> doing (arguably) from the very beginning.
>
> That's a huge divergence from how management tools are written.

This is one of the reasons why management tooling around QEMU needs 
quite a bit of improving.

There is simply no way a management tool can do a good job of being an 
authoritative source of configuration.  The races we're discussion is a 
good example of why.

But beyond those races, QEMU is the only entity that knows with 
certainty what bits of information are important to persist in order to 
preserve a guest across shutdown/restart.  The fact that we've punted 
this problem for so long has only ensured that management tools are 
either intrinsically broken or only support the most minimal subset of 
functionality we actually support.

>   Currently they contain the required guest configuration, a 
> representation of what's the current live configuration, and they 
> issue monitor commands to move the live configuration towards the 
> required configuration (or just generate a qemu command line).  What 
> you're describing is completely different, I'm not even sure what it is.

Management tools shouldn't have to think about how the monitor commands 
they issue impact the invocation options of QEMU.

>
>>
>>> The management tool already has to keep track of (the optional parts 
>>> of) the guest device tree.  It cannot start reading the stateful 
>>> non-config file at random points in time.  So all that is left is 
>>> the guest controlled portions of the device tree, which are pretty 
>>> rare, and random events like live-copy migration.  I think that 
>>> introducing a new authoritative source of information will create a 
>>> lot of problems.
>>
>> QEMU has always been the authoritative source.  Nothing new has been 
>> introduced.  We never persisted the machine's configuration which 
>> meant management tools had to try to aggressively keep up with us 
>> which is intrinsically error prone.  Fixing this will only improve 
>> existing management tools.
>
> If you look at management tools, they believe they are the 
> authoritative source of configuration information (not guest state, 
> which is more or less ignored).

It's because we've given them no other option.

>>>
>>> Right, but we should make it easy, not hard.
>>
>> Yeah, I fail to see how this makes it hard.  We conveniently are 
>> saying, hey, this is all the state that needs to be persisted.  We'll 
>> persist it for you if you want, otherwise, we'll expose it in a 
>> central location.
>
> The state-in-a-file is just a blob.  Don't expect the tool to parse it 
> and reassociate the various bits to its own representation.  Exposing 
> it via QMP commands is a lot better though.

I don't really see this as being a major issue.  There's no such thing 
as a "blob".  If someone wants to manipulate the state, they will.   We 
need to keep compatibility to support migrating from version-to-version.

I agree that we want to provide QMP interfaces to work with the state 
file.  But I don't think we should be hostile to manual manipulation.

Regards,

Anthony Liguori