Re: [Qemu-devel] [PATCH v4 00/47] Postcopy implementation

["Dr. David Alan Gilbert" <dgilbert@redhat.com>]

I've updated our github at:
https://github.com/orbitfp7/qemu/tree/wp3-postcopy

to have this version.

and it corresponds to the tag:
https://github.com/orbitfp7/qemu/releases/tag/wp3-postcopy-v4

Dave

* Dr. David Alan Gilbert (git) (dgilbert@redhat.com) wrote:
> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> 
> Hi,
>   This is the 4th cut of my version of postcopy; it is designed for use with
> the Linux kernel additions just posted by Andrea Arcangeli here:
> 
> http://marc.info/?l=linux-kernel&m=141235633015100&w=2
> 
> (Note: This is a new version compared to my previous postcopy patchset; you'll
> need to update the kernel to the new version.)
> 
> Other than the new kernel ABI (which is only a small change to the userspace side);
> the major changes are;
> 
>   a) Code for host page size != target page size
>   b) Support for migration over fd 
>      From Cristian Klein; this is for libvirt support which Cristian recently
>      posted to the libvirt list.
>   c) It's now build bisectable and builds on 32bit
> 
> Testing wise; I've now done many thousand of postcopy migrations without
> failure (both of idle and busy guests); so it seems pretty solid.
> 
> Must-TODO's:
>   1) A partially repeatable migration_cancel failure
>   2) virt_test's migrate.with_reboot test is failing
>   3) The ACPI fix in 2.1 that allowed migrating RAMBlocks to be larger than
>     the source feels like it needs looking at for postcopy.
>   4) Paolo's comments with respect to the wakeup_request/is_running code
>      in the migration thread
>   5) xbzrle needs disabling once in postcopy
> 
> Later-TODO's:
>   1) Control the rate of background page transfers during postcopy to
>      reduce their impact on the latency of postcopy requests.
>   2) Work with RDMA
>   3) Could destination RP be made blocking (as per discussion with Paolo;
>      I'm still worried that that changes too many assumptions)
> 
> 
> 
> V4:
>   Initial support for host page size != target page size
>     - tested heavily on hps==tps
>     - only partially tested on hps!=tps systems
>     - This involved quite a bit of rework around the discard code
>   Updated to new kernel userfault ABI
>     - It won't work with the previous version
>   Fix mis-optimisation of postcopy request for wrong RAMBlock
>      request for block A offset n
>      un-needed fault for block B/m (already received - no req sent)
>      request for block B/l  - wrongly sent as request for A/l
>   Fix thinko in discard bitmap processing (missed last word of bitmap)
>      Symptom: remap failures near the top of RAM if postcopy started late
>   Fix bug that caused kernel page acknowledgments to be misaligned
>      May have meant the guest was paused for longer than required
>   Fix potential for crashing cleaning up failed RP
>   Fixes in docs (from Yang)
>   Handle migration by fd as sockets if they are sockets
>   Build tested on 32bit
>   Fully build bisectable (x86-64)
> 
> 
> Dave
> 
> Cristian Klein (1):
>   Handle bi-directional communication for fd migration
> 
> Dr. David Alan Gilbert (46):
>   QEMUSizedBuffer based QEMUFile
>   Tests: QEMUSizedBuffer/QEMUBuffer
>   Start documenting how postcopy works.
>   qemu_ram_foreach_block: pass up error value, and down the ramblock
>     name
>   improve DPRINTF macros, add to savevm
>   Add qemu_get_counted_string to read a string prefixed by a count byte
>   Create MigrationIncomingState
>   socket shutdown
>   Provide runtime Target page information
>   Return path: Open a return path on QEMUFile for sockets
>   Return path: socket_writev_buffer: Block even on non-blocking fd's
>   Migration commands
>   Return path: Control commands
>   Return path: Send responses from destination to source
>   Return path: Source handling of return path
>   qemu_loadvm errors and debug
>   ram_debug_dump_bitmap: Dump a migration bitmap as text
>   Rework loadvm path for subloops
>   Add migration-capability boolean for postcopy-ram.
>   Add wrappers and handlers for sending/receiving the postcopy-ram
>     migration messages.
>   QEMU_VM_CMD_PACKAGED: Send a packaged chunk of migration stream
>   migrate_init: Call from savevm
>   Allow savevm handlers to state whether they could go into postcopy
>   postcopy: OS support test
>   migrate_start_postcopy: Command to trigger transition to postcopy
>   MIG_STATE_POSTCOPY_ACTIVE: Add new migration state
>   qemu_savevm_state_complete: Postcopy changes
>   Postcopy page-map-incoming (PMI) structure
>   Postcopy: Maintain sentmap and calculate discard
>   postcopy: Incoming initialisation
>   postcopy: ram_enable_notify to switch on userfault
>   Postcopy: Postcopy startup in migration thread
>   Postcopy: Create a fault handler thread before marking the ram as
>     userfault
>   Page request:  Add MIG_RPCOMM_REQPAGES reverse command
>   Page request: Process incoming page request
>   Page request: Consume pages off the post-copy queue
>   Add assertion to check migration_dirty_pages
>   postcopy_ram.c: place_page and helpers
>   Postcopy: Use helpers to map pages during migration
>   qemu_ram_block_from_host
>   Don't sync dirty bitmaps in postcopy
>   Host page!=target page: Cleanup bitmaps
>   Postcopy; Handle userfault requests
>   Start up a postcopy/listener thread ready for incoming page data
>   postcopy: Wire up loadvm_postcopy_ram_handle_{run,end} commands
>   End of migration for postcopy
> 
>  Makefile.objs                    |    2 +-
>  arch_init.c                      |  739 +++++++++++++++++++++++++--
>  docs/migration.txt               |  189 +++++++
>  exec.c                           |   76 ++-
>  hmp-commands.hx                  |   15 +
>  hmp.c                            |    7 +
>  hmp.h                            |    1 +
>  include/exec/cpu-common.h        |    8 +-
>  include/migration/migration.h    |  130 +++++
>  include/migration/postcopy-ram.h |  106 ++++
>  include/migration/qemu-file.h    |   47 ++
>  include/migration/vmstate.h      |    2 +-
>  include/qemu/sockets.h           |    1 +
>  include/qemu/typedefs.h          |    9 +-
>  include/sysemu/sysemu.h          |   43 +-
>  migration-fd.c                   |   24 +-
>  migration-rdma.c                 |    4 +-
>  migration.c                      |  693 +++++++++++++++++++++++++-
>  postcopy-ram.c                   | 1016 ++++++++++++++++++++++++++++++++++++++
>  qapi-schema.json                 |   14 +-
>  qemu-file.c                      |  598 +++++++++++++++++++++-
>  qmp-commands.hx                  |   19 +
>  savevm.c                         |  881 +++++++++++++++++++++++++++++++--
>  tests/Makefile                   |    2 +-
>  tests/test-vmstate.c             |   74 +--
>  util/qemu-sockets.c              |   28 ++
>  26 files changed, 4550 insertions(+), 178 deletions(-)
>  create mode 100644 include/migration/postcopy-ram.h
>  create mode 100644 postcopy-ram.c
> 
> -- 
> 1.9.3
> 
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK