Re: [Qemu-devel] [RFC PATCH 00/16] visitor+BER migration format

["Dr. David Alan Gilbert" <dgilbert@redhat.com>]

* Michael S. Tsirkin (mst@redhat.com) wrote:
> On Tue, Mar 25, 2014 at 08:17:11PM +0000, Dr. David Alan Gilbert (git) wrote:

<snip>

> >    5) At the moment you select BER output format by setting an environment
> >       variable ( export QEMUMIGFORMAT=BER ) , I need to put more thought
> >       in to the right way to do this, there are some harder questions like
> >       what happens to devices that are still using pre-vmstate encodings
> >       (that are currently sent as blobs) when they eventually convert over
> >       and thus how to keep compatibility with earlier BER output versions
> >       where they were blobs.
> 
> A simple way is the way we always changed migration
> format - trying this to the machine type.
> Old machine type gets you binary, newer one BER with blobs,
> then we can add more flags gradually.

Right, that should work; it gets a little messy to tie machine type
to formats for individual devices, but no big issue.

> With BER we'll actually be able to do smarter things in the future,
> such as send out multiple formats and have destination select
> the one it understands, but this isn't mandatory as a first step.
> 
> >    6) At the moment for blobs (i.e. pre-vmstate) the whole blob gets loaded
> >       into memory and then processed; that's not necessary and can be done
> >       with a QEMUFile shim.
> >    7) I need to sort out my Error** handling (probably in a lot of places)
> 
> That's mostly broken in migration as it is.
> 
> >    8) The visitors should be able to share much more code.
> > 
> > The not-yet-done:
> >    a) XBZRLE
> >    b) block-migration
> >    c) SPAPR (it's an iterative migration like RAM and block)
> >    d) RDMA (I'm guessing I've broken it, but haven't tried)
> >    e) Floats are currently sent as blobs.
> 
> So at least a,b and c  need to be done.

Yep, agreed; iterative migration stuff has direct byte/word putting all over
so is rather manual, so I just did enough to get it going first.

> Not sure about RDMA - it was never supported and no one
> seems to work on it.
> Maybe, somehow, it can use the old format when enabled?

I think Michael still throws bug fixes in from time to time, however my worry
is actually more about the Visitor conversion than the actual format change
that might break it; anyway needs testing and then we'll see.

> > What I've done:
> >   There are separate output visitors (binary compatible, debug and BER) and
> > they're selected by setting QEMUMIGFORMAT to one of 'debug', 'BER' or just
> > leaving it unset.
> > 
> >   The Input visitor is selected automatically from the first 4 bytes of the
> > file.
> 
> Not really necessary if you use machine type.

It's easy anyway.

> >   In general most types are BER sequences of indefinite length, with some
> > types that I've allocated an Application specific type tag to.  There is a hook
> > to give any VMState it's own type tag (which I've shown using one example
> > for cpu-common).  Integers and strings are standard 'universal' types.
> > 
> >   Objects with .get/.put methods or register_savevm are saved as an 'octet string'.
> > There are a few places where a device registered with .get/.put calls back into
> > vmstate_save_state for part of their state (typically for devices built on PCI)
> > and when they do that, even when in BER mode, those components get stored inside
> > the octet string in the old format.
> > 
> >   I've used the 'asn1c' tool to validate the schema (which is
> > in docs/specs/migration.schema) and also to verify streams that I've produced
> > match the schema.
> > 
> >   I've tested it with virt-test (hacked to have different source/dest qemu's)
> > and tried bin-bin, ber-ber, and pre-visitor-qemu -> this and
> > this -> pre-visitor-qemu just on the standard migration.default.tcp migration
> > test, but I've not tried a large combination of configurations.
> > 
> >   My fix for qemu_peek_buffer that's on the list is needed to stabilise
> > the binary format input visitor.
> > 
> >   I'll keep chipping away at that list, and would expect to pop another
> > version out in a month or so.
> > 
> > Dave
> > 
> > Dr. David Alan Gilbert (16):
> >   Visitor: Add methods for migration format use
> >   QEMUSizedBuffer/QEMUFile
> >   qemu-file: Add set/get tmp_visitor
> >   Header/constant/types fixes for visitors
> >   Visitor: Binary compatible output visitor
> >   Visitor: Debug output visitor
> >   Visitor: Binary compatible input visitor
> >   Visitor: Output path
> >   Visitor: Load path
> >   Visitor: Common types to use visitors
> >   Choose output visitor based on env variable
> >   BER Visitor: Create output visitor
> >   BER Visitor: Create input visitor
> >   Start some BER format docs
> >   ASN.1 schema for new migration format
> >   Wire in BER visitors
> 
> One nifty feature of Stefan's patches was that a
> set of unit tests for parser was included.
> Worth reusing?

Yes, unit tests are one thing I missed off my list but know
need doing.

Dave
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK