Re: [Qemu-devel] [PATCH v2 00/43] Postcopy implementation

[zhanghailiang <zhang.zhanghailiang@huawei.com>]

On 2014/8/11 22:29, Dr. David Alan Gilbert (git) wrote:
> From: "Dr. David Alan Gilbert"<dgilbert@redhat.com>
>
> Hi,
>    This is a 2nd cut of my postcopy implementation; it fixes up some
> of the comments from the 1st posting but it also fixes a lot of bugs,
> (other comments from the 1st round will be fixed later).
>
> The commands to start a postcopy migration have changed to:
>
>    migrate_set_capability x-postcopy-ram on
>    migrate -d   tcp:whereever
>    <some time later>
>    migrate_start_postcopy
>
> Starting the destination in paused mode (-s) now works.
>
> This now survives in a test harness with a few hundred migrations without
> problems (both on heavily loaded and idle VMs).
>
> Other changes:
>    Changed source rp handler to coroutine (from fd_set_handler) - cures
>      occasional loss of requests (and hence huge latency spikes)
>    Added 'SHUT' rp command as a handshake from the destination that it's finished
>    Fixed race during migration of guest with very little page modification
>    Shutdown the fault-thread cleanly
>    Update 'received' map in ADVISE state
>    Turned off nagling on rp - we want our requests to get their ASAP
>    Fixed seg when attempting to postcopy to a stream that couldn't get a return path
>    Got rid of the move of QEMUFile into the .h
>    Fixed up '-ve' and title of assert patch
>    Removed a couple of return-path opening optimisations as per Paolo's comments
>
> Note this version needs the QEMUSizedBuffer/QEMUFile patches that I've posted
> separately.
>
> Current TODO:
>     1) It's not bisectable yet
>     2) There are no testsuite additions (although I have a virt-test modification
>        I've been using).
>     3) End-of-migration cleanup is much better than it was but still needs some work.
>     4) Not all the code is there for systems with hostpagesize!=qemupagesize
>     5) xbzrle needs disabling once in postcopy
>     6) RDMA needs some rework
>     7) The latency measurements are now pretty consistent, no very large spikes,
>        but they're a bit higher than expected, I need to look at rate limiting
>        just the background scan.
>     8) Conversion of return-path to a process and blocking fd needs investigation
>        (as per discussion with Paolo)
>     9) Andrea has suggestions on ways to avoid some of the huge-page splitting
>        that occurs during the discard phase after precopy.
>    10) I'd like to format the data on the return path in a more structured way
>        (i.e. maybe using stuff from my BER world).
>    11) The ACPI fix in 2.1 that allowed migrating RAMBlocks to be larger than
>        the source feels like it needs looking at for postcopy.
> Dave
>
> Dr. David Alan Gilbert (43):
>    qemu_ram_foreach_block: pass up error value, and down the ramblock
>      name
>    improve DPRINTF macros, add to savevm
>    Add qemu_get_counted_string to read a string prefixed by a count byte
>    Create MigrationIncomingState
>
>    Return path: socket_writev_buffer: Block even on non-blocking fd's
>    Migration commands
>    Return path: Control commands
>    Return path: Send responses from destination to source
>    Return path: Source handling of return path
>    qemu_loadvm errors and debug
>    ram_debug_dump_bitmap: Dump a migration bitmap as text
>    Rework loadvm path for subloops
>    Add migration-capability boolean for postcopy-ram.
>    Add wrappers and handlers for sending/receiving the postcopy-ram
>      migration messages.
>    QEMU_VM_CMD_PACKAGED: Send a packaged chunk of migration stream
>    migrate_init: Call from savevm
>    Allow savevm handlers to state whether they could go into postcopy
>    postcopy: OS support test
>    migrate_start_postcopy: Command to trigger transition to postcopy
>    MIG_STATE_POSTCOPY_ACTIVE: Add new migration state
>    qemu_savevm_state_complete: Postcopy changes
>    Postcopy: Maintain sentmap during postcopy pre phase
>    Postcopy page-map-incoming (PMI) structure
>    postcopy: Add incoming_init/cleanup functions
>    postcopy: Incoming initialisation
>    postcopy: ram_enable_notify to switch on userfault
>    Postcopy: postcopy_start
>    Postcopy: Rework migration thread for postcopy mode
>    mig fd_connect: open return path
>    Postcopy: Create a fault handler thread before marking the ram as
>      userfault
>    Page request:  Add MIG_RPCOMM_REQPAGES reverse command
>    Page request: Process incoming page request
>    Page request: Consume pages off the post-copy queue
>    Add assertion to check migration_dirty_pages
>    postcopy_ram.c: place_page and helpers
>    Postcopy: Use helpers to map pages during migration
>    qemu_ram_block_from_host
>    Postcopy; Handle userfault requests
>    Start up a postcopy/listener thread ready for incoming page data
>    postcopy: Wire up loadvm_postcopy_ram_handle_{run,end} commands
>    End of migration for postcopy
>    Start documenting how postcopy works.
>
>   Makefile.objs                    |   2 +-
>   arch_init.c                      | 490 +++++++++++++++++++--
>   docs/migration.txt               | 150 +++++++
>   exec.c                           |  66 ++-
>   hmp-commands.hx                  |  15 +
>   hmp.c                            |   7 +
>   hmp.h                            |   1 +
>   include/exec/cpu-common.h        |   8 +-
>   include/migration/migration.h    | 128 ++++++
>   include/migration/postcopy-ram.h |  89 ++++
>   include/migration/qemu-file.h    |   9 +
>   include/migration/vmstate.h      |   2 +-
>   include/qemu/typedefs.h          |   7 +-
>   include/sysemu/sysemu.h          |  41 +-
>   migration-rdma.c                 |   4 +-
>   migration.c                      | 667 ++++++++++++++++++++++++++--
>   postcopy-ram.c                   | 915 +++++++++++++++++++++++++++++++++++++++
>   qapi-schema.json                 |  14 +-
>   qemu-file.c                      | 124 +++++-
>   qmp-commands.hx                  |  19 +
>   savevm.c                         | 854 +++++++++++++++++++++++++++++++++---
>   21 files changed, 3473 insertions(+), 139 deletions(-)
>   create mode 100644 include/migration/postcopy-ram.h
>   create mode 100644 postcopy-ram.c
>

Hi Dave,

I want to test your patches, but i failed to 'git am' them to the new
qemu-2.1 source. I want to know if you has a git-branch which i can
git clone directly? Thanks.

Best regards,
zhanghailiang