[PATCH v3 for-4.6 0/2] In-tree feature documentation

[PATCH v3 for-4.6 0/2] In-tree feature documentation

[Andrew Cooper]

An issue which Xen has is an uncertain support statement for features.
Given the success seen with docs/misc/xen-command-line.markdown, and in
particular keeping it up to date, introduce a similar system for
features.

Patch 1 introduces a proposed template (and a makefile tweak to include
the new docs/features subdirectory), while patch 2 is a feature document
covering the topic of migration.

v2 Adds %Revision and #History table, following feedback from v1.
v3 Adds #Testing and #Reference sections.

Andrew Cooper (2):
  docs: Template for feature documents
  docs: Migration feature document

 docs/Makefile                  |    2 +-
 docs/features/migration.pandoc |  123 ++++++++++++++++++++++++++++++++++++++++
 docs/features/template.pandoc  |   75 ++++++++++++++++++++++++
 3 files changed, 199 insertions(+), 1 deletion(-)
 create mode 100644 docs/features/migration.pandoc
 create mode 100644 docs/features/template.pandoc

-- 
1.7.10.4

[PATCH 1/2] docs: Template for feature documents

[Andrew Cooper]

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
v3: Added #Testing and #Reference sections
v2: Added %Revision and #History table
---
 docs/Makefile                 |    2 +-
 docs/features/template.pandoc |   75 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 76 insertions(+), 1 deletion(-)
 create mode 100644 docs/features/template.pandoc

diff --git a/docs/Makefile b/docs/Makefile
index d25e6c7..b9da605 100644
--- a/docs/Makefile
+++ b/docs/Makefile
@@ -15,7 +15,7 @@ MARKDOWNSRC-y := $(sort $(shell find misc -name '*.markdown' -print))
 
 TXTSRC-y := $(sort $(shell find misc -name '*.txt' -print))
 
-PANDOCSRC-y := $(sort $(shell find specs -name '*.pandoc' -print))
+PANDOCSRC-y := $(sort $(shell find features/ misc/ specs/ -name '*.pandoc' -print))
 
 # Documentation targets
 DOC_MAN1 := $(patsubst man/%.pod.1,man1/%.1,$(MAN1SRC-y))
diff --git a/docs/features/template.pandoc b/docs/features/template.pandoc
new file mode 100644
index 0000000..7698291
--- /dev/null
+++ b/docs/features/template.pandoc
@@ -0,0 +1,75 @@
+% Template for feature documents
+% Revision $N
+
+\clearpage
+
+This is a suggested template for formatting of a Xen feature document in
+tree.
+
+The purpose of this document is to provide a concrete support statement
+for the feature (indicating its security status), as well as brief user
+and technical documentation.
+
+# Basics
+
+A table with an overview of the support status and applicability.
+
+---------------- ----------------------------------------------------
+         Status: e.g. **Supported**/**Tech Preview**/**Experimental**
+
+Architecture(s): e.g. x86, arm
+
+   Component(s): e.g. Hypervisor, toolstack, guest
+
+       Hardware: _where applicable_
+---------------- ----------------------------------------------------
+
+# Overview
+
+A short description the feature, similar to an abstract for a
+paper/presentation.
+
+# User information
+
+Information for a user attempting to use the feature.  Should include
+how to enable the feature (is it enabled by default? If not, how to turn
+it on?), and how to interact with the feature (typically via `xl`).
+
+# Limitations
+
+Information concerning incompatibilities with other features or hardware
+combinations.
+
+# Technical information
+
+Information for a developer or power user.  Should include where to look
+in-tree for detailed documents and code.
+
+# Testing
+
+Information concerning how to properly test changes affecting this feature.
+
+# Areas for improvement
+
+List of enhancements which could be undertaken, e.g. to improve the
+feature itself, or improve interaction with other features.
+
+# Known issues
+
+List of known issues or bugs.  For tech preview or experimental
+features, this section must contain the list of items needing fixing for
+its status to be upgraded.
+
+# References
+
+Relevant external references for this feature.
+
+# History
+
+A table of changes to the document, in chronological order.
+
+------------------------------------------------------------------------
+Date       Revision Version  Notes
+---------- -------- -------- -------------------------------------------
+YYYY-MM-DD N        Xen X.Y  ...
+---------- -------- -------- -------------------------------------------
-- 
1.7.10.4

Re: [PATCH 1/2] docs: Template for feature documents

[Ian Campbell]

On Tue, 2015-09-15 at 14:54 +0100, Andrew Cooper wrote:
> +# Basics
> +
> +A table with an overview of the support status and applicability.
> +
> +---------------- ----------------------------------------------------
> +         Status: e.g. **Supported**/**Tech Preview**/**Experimental**

My main concern is still the lack of a precise meaning for these terms.
However this is not a problem with this series per-se and we can always
update the template once we know what they mean.

So I've acked and applied both patches to staging and staging-4.6.

[PATCH 2/2] docs: Migration feature document

[Andrew Cooper]

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
v3:
 * Corrections, and include LIBXL_HAVE_*,
 * #Testing and #Reference, following template review
v2:
 * %Revision and #History, following template review
 * Grammar fixes
---
 docs/features/migration.pandoc |  123 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 123 insertions(+)
 create mode 100644 docs/features/migration.pandoc

diff --git a/docs/features/migration.pandoc b/docs/features/migration.pandoc
new file mode 100644
index 0000000..9852a19
--- /dev/null
+++ b/docs/features/migration.pandoc
@@ -0,0 +1,123 @@
+% Migration
+% Revision 1
+
+\clearpage
+
+# Basics
+--------------- -------------
+        Status: **Supported**
+
+  Architecture: x86
+
+     Component: Toolstack
+--------------- -------------
+
+# Overview
+
+Migration is a mechanism to move a virtual machine while the VM is
+running.  Live migration moves a running virtual machine between two
+physical servers, but the same mechanism can be used for non-live
+migration (pause and copy) and suspend/resume from disk.
+
+# User details
+
+No hardware requirements, although hypervisor logdirty support is
+required for live migration.
+
+From the command line, `xl migrate/save/restore` are the top level
+interactions.  e.g.
+
+    xl create my-vm.cfg
+    xl migrate my-vm localhost
+
+or
+
+    xl create my-vm.cfg
+    xl save my-vm /path/to/save/file
+    xl restore /path/to/save/file
+
+Xen 4.6 sees the introduction of Migration v2.  There is no change for
+people using `xl`, although the `libxl` API has had an extension.
+
+# Technical details
+
+Migration is formed of several layers.  `libxc` is responsible for the
+contents of the VM (ram, vcpus, etc) and the live migration loop, while
+`libxl` is responsible for items such as emulator state.
+
+The format of the migration v2 stream is specified in two documents, and
+is architecture neutral.  Compatibility with legacy streams is
+maintained via the `convert-legacy-stream` script which transforms a
+legacy stream into a migration v2 stream.
+
+* Documents
+    * `docs/specs/libxc-migration-stream.pandoc`
+    * `docs/specs/libxl-migration-stream.pandoc`
+* `libxc`
+    * `tools/libxc/xc_sr_*.[hc]`
+* `libxl`
+    * `tools/libxl/libxl_stream_{read,write}.c`
+    * `tools/libxl/libxl_convert_callout.c`
+* Scripts
+    * `tools/python/xen/migration/*.py`
+    * `tools/python/scripts/convert-legacy-stream`
+    * `tools/python/scripts/verify-stream-v2`
+
+## libxl
+
+With migration v2 support, LIBXL\_HAVE\_SRM\_V2 and LIBXL\_HAVE\_SRM\_V1
+are introduced to indicate support.  `domain_restore_params` gains a new
+parameter, `stream_version`, which is used to distinguish between legacy and
+v2 migration streams, and hence whether legacy conversion is required.
+
+# Limitations
+
+Hypervisor logdirty support is incompatible with hardware passthrough,
+as IOMMU faults cannot be used to track writes.
+
+While not a bug in migration specifically, VMs are very sensitive to
+changes in cpuid information, and cpuid levelling support currently has
+its issues.  Extreme care should be taken when migrating VMs between
+non-identical CPUs until the cpuid levelling improvements are complete.
+
+# Testing
+
+Changes in libxc should be tested with every guest type (32bit PV, 64bit
+PV, HVM), while changes in libxl should test HVM guests with both
+qemu-traditional and qemu-upstream.
+
+In general, testing can be done on a single host using `xl
+save/restore` or `xl migrate $VM localhost`.
+
+Any changes to the conversion script should be tested in all upgrade
+scenarios, which will involve starting with VMs from Xen 4.5
+
+# Areas for improvement
+
+* Arm support
+* Linear P2M support for x86 PV
+* Live looping parameters
+
+# Known issues
+
+* x86 HVM guest physmap operations (not reflected in logdirty bitmap)
+* x86 HVM with PoD pages (attempts to map cause PoD allocations)
+* x86 HVM with nested-virt (no relevant information included in the
+  stream)
+* x86 PV ballooning (P2M marked dirty, target frame not marked)
+* x86 PV P2M structure changes (not noticed, stale mappings used)
+
+# References
+
+Xen Developer Summit 2015 Presentation
+[video](https://www.youtube.com/watch?v=RwiDeG21lrc) and
+[slides](http://events.linuxfoundation.org/sites/events/files/slides/migv2.pdf)
+for Migration v2
+
+# History
+
+------------------------------------------------------------------------
+Date       Revision Version  Notes
+---------- -------- -------- -------------------------------------------
+2015-10-24 1        Xen 4.6  Document written
+---------- -------- -------- -------------------------------------------
-- 
1.7.10.4

Re: [PATCH v3 for-4.6 0/2] In-tree feature documentation

[Wei Liu]

I don't have further comments on this series.

It's definitely a good idea to have this in for 4.6.

If there are no other comments I suggest we commit this series as soon
as possible.

Wei.