Re: [Qemu-devel] [PATCH v4] XBZRLE delta for live migration of large memory apps

[Anthony Liguori <anthony@codemonkey.ws>]

On 08/10/2011 11:40 AM, Avi Kivity wrote:
> On 08/10/2011 07:23 PM, Anthony Liguori wrote:
>>> Right now we have capabilties in the form of -help output.
>>>
>>> If -help says
>>>
>>> -no-xzbrle disable xzbrle support
>>>
>>> (or -migration-compression xzbrle=off, or something) that's sufficient
>>> for management tools.
>>
>>
>> This is static, not dynamic. You may attempt to migrate to another
>> host that supports it and then migrate to a second host that doesn't
>> support it after the first migration fails.
>
> This may be acceptable, wait until the entire migration cluster is
> xzbrle capable before enabling it. If not, add a monitor command.

1) xzbrle needs to be disabled by default.  That way management tools 
don't unknowingly enable it by not passing -no-xzbrle.

2) there needs to be a mechanism for the management tool to query 
whether qemu supports xzbrle.

3) a management tool should be able to query the source and destination, 
and then enable xzbrle if both sides support it.

You can argue that (3) could be static.  A command could be added to 
toggle it dynamically through the monitor.

But no matter what, someone has to touch libvirt and any other tool that 
works with QEMU to make this thing work.  But this is a general problem. 
  Any optional change to the migration protocol has exactly the same 
characteristics whether it's XZBRLE, XZBRLE v2 (if there is a v2), 
ASN.1, or any other form of compression that rolls around.

Instead of teaching management tools how to deal with all of these 
things, let's just fix this problem once.  It just takes:

a) A query-migration-caps command that returns a dict with two lists of 
strings.  Something like:

{ 'execute': 'query-migration-caps' }
{ 'return' : { 'capabilities': [ 'xbzrle' ], 'current': [] } }

b) A set-migration-caps command that takes a list of strings.  It simply 
takes the intersection of the capabilities set with the argument and 
sets the current set to the result.  Something like:

{ 'execute': 'set-migration-caps', 'arguments': { 'set': [ 'xbzrle' ] }}
{ 'return' : {} }

c) An internal interface to register a capability and an internal 
interface to check if a capability is currently enabled.  The xzbrle 
code just needs to disable itself if the capability isn't set.

Then we teach libvirt (and other tools) to query the caps list on the 
source, set the destination, query the current set on the destination, 
and then set that set on the source.

As we introduce new things, like the next great compression protocol, or 
ASN.1, we don't need to touch libvirt again.  libvirt can still know 
about the caps and selectively override QEMU if it's so inclined but it 
prevents us from reinventing the same mechanisms over and over again.

>>> We shouldn't block this feature just because some monitor facility is
>>> not yet implemented.
>>
>> We shouldn't make *any* changes to the migration protocol before we
>> have a feature negotiation capability. I only want to do a hard break
>> of the protocol once.
>
> Didn't we agree that management tool mediated feature negotiation (that
> is, outside the migration protocol itself) is acceptable?

Yes.  But that negotiation needs to become part of the "protocol" for 
migration.  In the absence of that negotiation, we need to use the wire 
protocol we use today.  We cannot have ad-hoc feature negotiation for 
every change we make to the wire protocol.

Regards,

Anthony Liguori

>