Re: [PATCH 0/6] [VERY RFC] Migration Stream v2

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 14/04/14 18:49, George Dunlap wrote:
> On Thu, Apr 10, 2014 at 12:21 PM, Andrew Cooper
> <andrew.cooper3@citrix.com> wrote:
>> On 10/04/14 11:42, Ian Campbell wrote:
>>> On Wed, 2014-04-09 at 19:28 +0100, Andrew Cooper wrote:
>>>> Some design decisions have been take very deliberately (e.g. splitting the
>>>> logic for PV and hvm migration) while others have been more along the lines of
>>>> "I think its a sensible thing to do given a lack of any evidence/opinion to
>>>> the contrary".
>>> Is there some indication of which is which?
>> Not really, given the clean rewrite, and also that it is only partially
>> complete.
>>
>>> Should we check in the desigh/spec which was previously posted as part
>>> of this?
>> I knew I forgot something...
>>
>> http://xenbits.xen.org/people/andrewcoop/domain-save-format-E.pdf
>
> What did you imagine might constitute an "Optional" record?
>
> Other than that, everything looks sensible so far -- but having only
> save/restore of one guest type is the easy bit.  It's when you start
> to have to multiplex across {PV, HVM, PVH} x {disk, network, remus}
> that things are going to get more "interesting".
>
>  -George

I did not opt for optional records, nor did I author them into the
spec.  Frankly, I cannot forsee a need for anything other than mandatory
records.  Nothing required for migration can possibly be optional, and
anything else is likely to be toolstack data which necessarily has to be
ahead of the domain in the migration stream so the receiving toolstack
can create a suitable domain for the new xc_domain_restore() to restore
into.

As for multiplexing, I have been considering that given the
implementation of the live part of migration (which now works, as of a
few hours ago).

Domain save and domain restore are two very very different operations. 
Save involves prodding at a live domain and stuffing values into a
stream.  Restore involves taking stuff from a stream and updating paused
state.

A lot of the structure of save is common to all types of domain and
roughly follows:

send some headers
start logdirty
send some preamble state (p2m size, p2m etc)
while consulting dirty bitmap:
   send some memory
pause domain
send remaining memory
send postamble state (vcpus, tsc, qemu blob etc)
send end record

I was considering writing a common save routine with domain/arch hooks. 
This would allow for one canonical implementation of the save code
(including how to do the 'live' bit, or how to apply remus) while
separating the architecture bits.

Restore on the other hand is much more simple, and just involves reading
records from the stream in the order found and applying the required
changes to the specified domain.  Each architecture can have a do {}
while() loop which accepts valid records for the specific type of domain.

I will experiment with the domain/arch hooks when implementing HVM
migration (which should be rather more simple)

~Andrew