From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38578) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aOOgH-000375-2Q for qemu-devel@nongnu.org; Wed, 27 Jan 2016 06:54:38 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aOOgD-0000rT-Od for qemu-devel@nongnu.org; Wed, 27 Jan 2016 06:54:36 -0500 Received: from mx2.parallels.com ([199.115.105.18]:42513) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aOOgD-0000rB-Fd for qemu-devel@nongnu.org; Wed, 27 Jan 2016 06:54:33 -0500 Message-ID: <56A8AFEE.4070201@virtuozzo.com> Date: Wed, 27 Jan 2016 14:54:22 +0300 From: Vladimir Sementsov-Ogievskiy MIME-Version: 1.0 References: <1446747083-18205-1-git-send-email-dgilbert@redhat.com> In-Reply-To: <1446747083-18205-1-git-send-email-dgilbert@redhat.com> Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH v9 00/56] Postcopy implementation List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Dr. David Alan Gilbert (git)" , qemu-devel@nongnu.org Hello, Dr. Gilbert! As I understand this is not a complete realization of post-copy stage of migration, but only realization of ram post-copy.. I need to implement post-copy migration of block-dirty-bitmaps. And it should work with/without ram post-copy. Did you plan a possibility of post-copying something except ram? If yes, can you please help me with an interface? Should I implement separate thread like postcopy_ram_listen_thread and other things, or something may be reused? On 05.11.2015 21:10, Dr. David Alan Gilbert (git) wrote: > From: "Dr. David Alan Gilbert" > > This is the 9th cut of my version of postcopy. > > The userfaultfd linux kernel code is now in the upstream kernel > tree, and so 4.3 can be used without modification. > > This qemu series can be found at: > https://github.com/orbitfp7/qemu.git > on the wp3-postcopy-v9 tag > > Testing status: > * Tested heavily on x86 > * Smoke tested on aarch64 (so it does work on different page sizes) > > This work has been partially funded by the EU Orbit project: > see http://www.orbitproject.eu/about/ > > v9 > Almost all of the changes are changes from review comments, and most are > minor. > > The following are new patches, mostly split out from earlier patches > (The exception being adding userfaultfd.h header back in - which was in v5 > but we took out, but is needed again due to 1842bdfd) > > 04/56 Move page_size_init earlier > 08/56 qemu_ram_block_by_name > 12/56 Factor out host_from_stream_offset call and check > 15/56 Add Linux userfaultfd.h header back > 21/56 migration_is_setup_or_active > 30/56 migration_completion: Take current state > 34/56 Maintain unsentmap > > The previous patches 03,10,13,36/54 went in upstream already. > > Fix for assert using hotplug (Thanks Bharata for spotting that) > Fix for migrate_cancel a second time > Rework for migration_bitmap_rcu after Denis's deadlock fix > Rework ram_load into a separate postcopy loop > The 'sentmap' is now an 'unsentmap' - this saves a complement step at the end > The unsentmap creation is now split into a separate patch > The unsentmap is now stored in the RCU structure (although it can't > really resize during the migrate) > Fix for block migration > still not a suggested combination. > Minor: > trace_savevm_send_open_return_path added > split migration_is_active function into separate patch and made > migration_is_setup_or_active > move file reads into migrate_handle_advise and migrate_handle_packaged > use of MIN in send-packaged-chunk > migration_thread_started -> migration_thread_running > updated qemu_get_buffer_in_place with Juan's version (and size_t'ified it) > split the host_from_stream_offset change into a separate patch > 'ram_load: Factor out host_from_stream_offset call and check' > split out ram_find_block_by_id into a separate patch and now > called qemu_ram_block_by_name > postcopy_discard_send_range etc now take start/length rather than start/end > (also added another trace) > split ram_save_host_page into ram_save_host_page/ram_save_target_page > split host page cleanup function into a core that handles both passes > > Dave > > Dr. David Alan Gilbert (56): > Add postcopy documentation > Provide runtime Target page information > Move configuration section writing > Move page_size_init earlier > qemu_ram_block_from_host > qemu_ram_block_by_name > Rename mis->file to from_src_file > Add qemu_get_buffer_in_place to avoid copies some of the time > Add wrapper for setting blocking status on a QEMUFile > Add QEMU_MADV_NOHUGEPAGE > ram_debug_dump_bitmap: Dump a migration bitmap as text > ram_load: Factor out host_from_stream_offset call and check > migrate_init: Call from savevm > Rename save_live_complete to save_live_complete_precopy > Add Linux userfaultfd.h header > Return path: Open a return path on QEMUFile for sockets > Return path: socket_writev_buffer: Block even on non-blocking fd's > Migration commands > Return path: Control commands > Return path: Send responses from destination to source > migration_is_setup_or_active > Return path: Source handling of return path > Rework loadvm path for subloops > Add migration-capability boolean for postcopy-ram. > Add wrappers and handlers for sending/receiving the postcopy-ram > migration messages. > MIG_CMD_PACKAGED: Send a packaged chunk of migration stream > Modify save_live_pending for postcopy > postcopy: OS support test > migrate_start_postcopy: Command to trigger transition to postcopy > migration_completion: Take current state > MIGRATION_STATUS_POSTCOPY_ACTIVE: Add new migration state > Avoid sending vmdescription during postcopy > Add qemu_savevm_state_complete_postcopy > Postcopy: Maintain unsentmap > Postcopy: Calculate discard > postcopy: Incoming initialisation > postcopy: ram_enable_notify to switch on userfault > Postcopy: Postcopy startup in migration thread > Postcopy: End of iteration > Page request: Add MIG_RP_MSG_REQ_PAGES reverse command > Page request: Process incoming page request > Page request: Consume pages off the post-copy queue > postcopy_ram.c: place_page and helpers > Postcopy: Use helpers to map pages during migration > postcopy: Check order of received target pages > Don't sync dirty bitmaps in postcopy > Don't iterate on precopy-only devices during postcopy > Host page!=target page: Cleanup bitmaps > Round up RAMBlock sizes to host page sizes > Postcopy; Handle userfault requests > Start up a postcopy/listener thread ready for incoming page data > postcopy: Wire up loadvm_postcopy_handle_ commands > Postcopy: Mark nohugepage before discard > End of migration for postcopy > Disable mlock around incoming postcopy > Inhibit ballooning during postcopy > > balloon.c | 11 + > docs/migration.txt | 191 ++++++++ > exec.c | 92 +++- > hmp-commands.hx | 15 + > hmp.c | 7 + > hmp.h | 1 + > hw/ppc/spapr.c | 2 +- > hw/virtio/virtio-balloon.c | 4 +- > include/exec/cpu-common.h | 4 + > include/exec/exec-all.h | 1 - > include/exec/ram_addr.h | 2 - > include/migration/migration.h | 121 ++++- > include/migration/postcopy-ram.h | 99 ++++ > include/migration/qemu-file.h | 10 + > include/migration/vmstate.h | 8 +- > include/qemu-common.h | 1 + > include/qemu/osdep.h | 9 + > include/qemu/typedefs.h | 3 + > include/sysemu/balloon.h | 2 + > include/sysemu/sysemu.h | 46 +- > kvm-all.c | 1 - > linux-headers/linux/userfaultfd.h | 167 +++++++ > migration/Makefile.objs | 2 +- > migration/block.c | 9 +- > migration/migration.c | 723 +++++++++++++++++++++++++-- > migration/postcopy-ram.c | 760 +++++++++++++++++++++++++++++ > migration/qemu-file-unix.c | 111 ++++- > migration/qemu-file.c | 64 +++ > migration/ram.c | 997 +++++++++++++++++++++++++++++++++++--- > migration/savevm.c | 823 +++++++++++++++++++++++++++---- > qapi-schema.json | 18 +- > qmp-commands.hx | 19 + > qtest.c | 1 - > trace-events | 84 +++- > vl.c | 1 + > 35 files changed, 4159 insertions(+), 250 deletions(-) > create mode 100644 include/migration/postcopy-ram.h > create mode 100644 linux-headers/linux/userfaultfd.h > create mode 100644 migration/postcopy-ram.c > -- Best regards, Vladimir * now, @virtuozzo.com instead of @parallels.com. Sorry for this inconvenience.