Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

[Anthony Liguori <anthony@codemonkey.ws>]

On 02/23/2011 11:18 AM, Avi Kivity wrote:
> On 02/23/2011 06:28 PM, Anthony Liguori wrote:
>>
>>>> Well specifically, it has to ask QEMU and QEMU can tell it the 
>>>> current state via a nice structured data format over QMP.  It's a 
>>>> hell of a lot easier than the management tool trying to do this 
>>>> outside of QEMU.
>>>
>>> So, if qemu crashes, the management tool has to start it up to find 
>>> out what the current state is.
>>
>> Depends on how opaque we make the state file.  I've been thinking a 
>> simple ini syntax with a well supported set of keys.  In that case, a 
>> management tool can read it without starting QEMU.
>
> Then the management stack has to worry about yet another way of 
> interacting via qemu.

{ 'StateItem': { 'key': 'str', 'value': 'str' } }
{ 'StateSection': { 'kind': 'str', 'name': 'str', 'items': [ 'StateItem' 
] } }
{ 'StateInfo': { 'sections': [ 'StateSection' ] } }

{ 'query-state', {}, {}, 'StateInfo' }

A management tool never need to worry about anything other than this 
command if it so chooses.  If we have the pre-machine init mode for 
0.16, then this can even be used to inspect state without running a guest.

The fact that the state is visible in the filesystem is an 
implementation detail.

>   I'd like to limit it to the monitor.
>
>>>
>>> Doesn't the stateful non-config file becomes a failure point?  It 
>>> has to be on shared and redundant storage?
>>
>> It depends on what your availability model is and how frequently your 
>> management tool backs up the config.  As of right now, we have a 
>> pretty glaring reliability hole here so adding a stateful 
>> "non-config" can only improve things.
>
> I think the solutions I pointed out close the hole with the existing 
> interfaces.

It doesn't work for eject unless you interpose an acknowledged event.  
Ultimately, this is a simple problem.  If you want reliability, we 
either need symmetric RPCs so that the device model can call (and wait) 
to the management layer to acknowledge a change or QEMU can post an 
event to the management layer, and maintain the state in a reliable fashion.

>>
>>> To me, it seems a lot easier to require management to replay any 
>>> commands that hadn't been acknowledged (due to management failure), 
>>> or to query qemu as to its current state (if it is alive). 
>>
>> You still have the race condition around guest initiated events like 
>> eject.  Unless you have an acknowledged event from a management tool 
>> (which we can't do in QMP today) whereas you don't complete the guest 
>> initiated eject operation until management ack's it, we need to store 
>> that state ourself.
>
> I don't see why.
>
> If management crashes, it queries the eject state when it reconnects 
> to qemu.
> If qemu crashes, the eject state is lost, but that is fine.  My CD-ROM 
> drive tray pulls itself in when the machine is started.

Pick any of a number of possible events that change the machine's 
state.  We can wave our hands at some things saying they don't matter 
and do one off solutions for others, or we can just have a robust way of 
handling this consistently.

>>
>> I don't like the idea of making a management tool such an integral 
>> part of the functional paths. 
>
> I agree that we don't want qemu to wait on the management stack any 
> more than necessary.
>
>> Not having a stateful config file also means that this problem isn't 
>> solved in any form without a really sophisticated management stack.  
>> I'm a big fan of being robust in the face of not-so sophisticated 
>> management tools.
>
> You're introducing the need for additional code in the management 
> layer, the care and feeding for the stateful non-config file.

If a management layer ignores the stateful non-config file, as you like 
to call it, it'll get the same semantics it has today.  I think managing 
a single thing is a whole lot easier than managing an NVRAM file, a 
block migration layering file, and all of the future things we're going 
to add once we decide they are important too.

>>> If qemu crashes, these events are meaningless.  If management 
>>> crashes, it has to query qemu for all state that it wants to keep 
>>> track of via events.
>>
>> Think power failure, not qemu crash.  In the event of a power 
>> failure, any hardware change initiated by the guest ought to be 
>> consistent with when the guest has restarted.  If you eject the CDROM 
>> tray and then lose power, its still ejected after the power comes 
>> back on.
>
> Not on all machines.
>
> Let's list guest state which is independent of power.  That would be 
> wither NVRAM of various types, or physical alterations.  CD-ROM eject 
> is one.  Are there others?

Any indirect qemu state.  Block migration is an example, but other 
examples would be VNC server information (like current password), WCE 
setting (depending on whether we modelled eeprom for the drivers), and 
persisted device settings (lots of devices have eeprom these days).

>>
>>>>
>>>> I think the nature of a posted event management interface is such 
>>>> that we need a stateful config that persists across QEMU invocations.
>>>
>>> I'm not convinced, and I think making qemu manage even more state 
>>> creates more problems.
>>
>> Well this patch series is making qemu management more state.  The 
>> only question is whether we do this as a one-off mechanism or whether 
>> we architect a general mechanism to do it.
>>
>> How much state we store can always be up for discussion but I think 
>> it's undeniable that we need to store more state than we're storing 
>> today (none).
>
> I think my solution (multiplexing block format driver) fits the 
> requirements for live-copy perfectly.  In fact it has a name - it's a 
> RAID-1 driver started in degraded mode.  It could be useful other use 
> cases.

It feels a bit awkward to me to be honest.

Regards,

Anthony Liguori