Re: [PATCH for-4.6 2/2] docs: Migration feature document

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 25/08/15 09:49, Wei Liu wrote:
> On Mon, Aug 24, 2015 at 06:37:02PM +0100, Andrew Cooper wrote:
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> ---
>>  docs/features/migration.pandoc |   90 ++++++++++++++++++++++++++++++++++++++++
>>  1 file changed, 90 insertions(+)
>>  create mode 100644 docs/features/migration.pandoc
>>
>> diff --git a/docs/features/migration.pandoc b/docs/features/migration.pandoc
>> new file mode 100644
>> index 0000000..e0422f9
>> --- /dev/null
>> +++ b/docs/features/migration.pandoc
>> @@ -0,0 +1,90 @@
>> +% Migration
> % Revision 1
>
>> +
>> +\clearpage
>> +
>> +# Basics
>> +--------------- -------------
>> +        Status: **Supported**
>> +
>> +  Architecture: x86
>> +
>> +     Component: Toolstack
>> +--------------- -------------
>> +
>> +# Overview
>> +
>> +Migration is a mechanism to move a virtual machine while the VM is running.
>> +Live migration moves a running virtual machine between two physical servers,
>> +but the same mechanism can be used for non-live migrate (pause and copy) and
>> +suspend/resume from disk.
>> +
>> +# User details
>> +
>> +No hardware requirements, although hypervisor logdirty support is required for
>> +live migration.
>> +
>> +From the command line, `xl migrate/save/restore` are the top level
>> +interactions.  e.g.
>> +
>> +    xl create my-vm.cfg
>> +    xl migrate my-vm localhost
>> +
>> +or
>> +
>> +    xl create my-vm.cfg
>> +    xl save my-vm /path/to/save/file
>> +    xl restore /path/to/save/file
>> +
>> +Xen 4.6 sees the instruction of Migration v2.  There is no change for people
>> +using `xl`, although the `libxl` API has had an extension.
>> +
>> +# Technical details
>> +
>> +Migration is of formed of several layers.  `libxc` is responsible for the
> Extraneous "of".
>
>> +contents of the VM (ram, vcpus, etc) and the live migration loop, while
>> +`libxl` is responsible for items such as emulator state.
>> +
>> +The format of the migration v2 stream is specified in two documents, and is
>> +architecture neutral.  Compatibility with legacy streams is maintained via the
>> +`convert-legacy-stream` script which transforms a legacy stream into a
>> +migration v2 stream.
>> +
>> +* Documents
>> +    * `docs/specs/libxc-migration-stream.pandoc`
>> +    * `docs/specs/libxl-migration-stream.pandoc`
>> +* `libxc`
>> +    * `tools/libxc/xc_sr_*.[hc]`
>> +* `libxl`
>> +    * `tools/libxl/libxl_stream_{read,write}.c`
>> +* Scripts
>> +    * `tools/python/xen/migration/*.py`
>> +    * `tools/python/scripts/convert-legacy-stream`
>> +    * `tools/python/scripts/verify-stream-v2`
>> +
>> +Users of the `libxl` API have a new parameter `stream_version` in
>> +`domain_restore_params` which is used to distinguish between legacy and v2
>> +migration streams, and hence whether legacy conversion is required.
>> +
>> +# Limitations
>> +
>> +Hypervisor logdirty support is incompatible with hardware passthrough, as
>> +IOMMU faults cannot be used to track writes.
>> +
>> +While not a bug in migration specifically, VMs are very sensitive to changes
>> +in cpuid information, and cpuid levelling support currently has its issues.
>> +Extreme care should be taken when migrating VMs between non-identical CPUs
>> +until the cpuid levelling improvements are complete.
>> +
>> +# Areas for improvement
>> +
>> +* Arm support
>> +* Linear P2M support for x86 PV
>> +* Live looping parameters
>> +
>> +# Known issues
>> +
>> +* x86 HVM guest physmap operations (not reflected in logdirty bitmap)
>> +* x86 HVM with PoD pages (attempts to map cause PoD allocations)
>> +* x86 HVM with nested-virt (no relevant information included in the stream)
>> +* x86 PV ballooning (P2M marked dirty, target frame not marked)
>> +* x86 PV P2M structure changes (not noticed, stale mappings used)
> TBH I think "Areas for improvement" and "Known issues" sections are very
> cryptic to normal users, but I don't have any suggestion to make them
> better.
>
> In any case, any document is better than no documents. I'm all for this
> doc going in as soon as possible.

This document is deliberately designed to contain both user and
technical information, and be a full overview of the feature, so it is
important for the information to be present.  I am open to alternative
suggestions for how to represent it.

They are basically "proposed new features" and "known bugs".  "Areas for
improvement" could easily be small projects for newcomers/gsoc/opw,
while "Known Issues" probably needs a deeper technical knowledge of the
area.

~Andrew