From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38435) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1V6Gbn-0000tP-LS for qemu-devel@nongnu.org; Mon, 05 Aug 2013 04:57:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1V6Gbf-0004Om-SI for qemu-devel@nongnu.org; Mon, 05 Aug 2013 04:57:43 -0400 Received: from e23smtp06.au.ibm.com ([202.81.31.148]:39892) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1V6Gbb-0004MX-Gk for qemu-devel@nongnu.org; Mon, 05 Aug 2013 04:57:35 -0400 Received: from /spool/local by e23smtp06.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 5 Aug 2013 18:49:16 +1000 Received: from d23relay04.au.ibm.com (d23relay04.au.ibm.com [9.190.234.120]) by d23dlp02.au.ibm.com (Postfix) with ESMTP id 157352BB0051 for ; Mon, 5 Aug 2013 18:57:19 +1000 (EST) Received: from d23av04.au.ibm.com (d23av04.au.ibm.com [9.190.235.139]) by d23relay04.au.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id r758fXWZ50856080 for ; Mon, 5 Aug 2013 18:41:40 +1000 Received: from d23av04.au.ibm.com (loopback [127.0.0.1]) by d23av04.au.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id r758vAm4004078 for ; Mon, 5 Aug 2013 18:57:11 +1000 Message-ID: <51FF68B4.8020201@linux.vnet.ibm.com> Date: Mon, 05 Aug 2013 16:56:20 +0800 From: Lei Li MIME-Version: 1.0 References: <1374783499-2550-1-git-send-email-lilei@linux.vnet.ibm.com> <51F24452.1010605@redhat.com> In-Reply-To: <51F24452.1010605@redhat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH 0/12 RFC v2] Localhost migration List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Paolo Bonzini Cc: aarcange@redhat.com, aliguori@us.ibm.com, quintela@redhat.com, qemu-devel@nongnu.org, lagarcia@br.ibm.com, rcj@linux.vnet.ibm.com On 07/26/2013 05:41 PM, Paolo Bonzini wrote: > Il 25/07/2013 22:18, Lei Li ha scritto: >> Hi, >> >> This patch series tries to add localhost migration support to >> Qemu. >> >> When doing localhost migration, the host memory will balloon >> up during the period, might consume double memories for some time. >> So we want to add a new live migration mechanism localhost >> migration. >> >> Following I copied from last version that Anthony added for the >> benefit of the other reviewers: >> >> The goal here is to allow "live upgrade" of a running QEMU instance. >> The work flow would look like this: >> >> 1) Guests are running QEMU release 1.6.1 >> 2) Admin installs QEMU release 1.6.2 via RPM or deb >> 3) Admin does localhost migration with page flipping to use new version >> of QEMU. >> >> Page flipping is used in order to avoid requiring that there is enough >> free memory to fit an additional copy of the largest guest which is the >> requirement today with localhost migration. >> >> You can also read from the link below: >> http://lists.gnu.org/archive/html/qemu-devel/2013-06/msg02577.html >> >> The plan is: >> >> 1) Add new command to do localhost migration. >> >> The qmp interface introduced like: >> >> { 'command': 'localhost-migrate', 'data': {'uri': 'str'} } >> >> 2) Use different mechanism than current live migration. >> >> The very basic work flow like: >> >> qemu on the source (the source and destination are both on localhost) >> | >> V >> Stop VM >> | >> V >> Create threads >> | >> V >> Page flipping through vmspice >> | >> V >> MADV_DONTNEED the ram pages which are already flipped >> | >> V >> Migration completes >> >> As stopping VM first, we expect/resume the page flipping through vmspice >> is fast enough to meet *live migration (low downtime). >> >> Notes: >> Currently the work flow is not exactly the same as description >> above. For the first step, the work flow we implemented is: >> stop VM and copy ram pages via unix domain socket, MADV_DONTNEED >> ram pages that already copied. After that, will replace to vmsplice >> mechanism instead of copying pages. >> >> Now it's still a draft version, and as it is implemented separately >> to the current migration code for a easy start, the next step will >> be trying to integrate it into the current migration implementation >> closely. To make sure we are on the right direction that should be >> headed, please let me know your suggestions on this. >> >> For the interface, as Anthony has suggested using a flag or a >> capability, which one would you prefer or any ideas? > Using a capability on the source makes sense; I don't think anything > special is needed on the destination. The destination just sees a > special packet telling it that a pipe is available via SCM_RIGHTS; then > it fetches the file descriptor and uses it for the new protocol. > > It looks like this could reuse a lot of the RAM copying hooks that we > introduced for RDMA. You shouldn't need to change anything in savevm.c > or arch_init.c Hi Paolo, Thanks for your suggestions! And sorry for the late reply, I am looking into RDMA implementation and trying to figure out the way to integrate with its approach as you suggested. For the interface on the source, I am OK with using a capability. For the destination, currently we are copying ram pages via unix domain socket as first step, will replace to pipe later. How about add a new URI prefix as Michael R.Hines suggested? > Paolo > >> Your comments are very welcome! >> >> TODO: >> - Integrate to the current implement of migration closely? >> - Introduce a mechanism to exchange a PIPE via SCM_RIGHTS. >> - benchmark/evaluation. >> >> Lei Li (12): >> migration: export MIG_STATE_xxx flags >> savevm: export qemu_save_device_state() >> rename is_active to is_block_active >> arch_init: introduce ram_page_save() >> arch_init: introduce ram_save_local() >> arch_init: add save_local_setup to savevm_ram_handlers >> savevm: introduce qemu_savevm_local() >> savevm: adjust is_ram check in register_savevm_live() >> migration-local: implementation of outgoing part >> migration-local: implementation of incoming part >> migration-local: add option to command line for local incoming >> hmp:add hmp_localhost_migration interface >> >> Makefile.objs | 1 + >> arch_init.c | 110 +++++++++++++++++++ >> block-migration.c | 2 +- >> hmp-commands.hx | 17 ++++ >> hmp.c | 13 +++ >> hmp.h | 1 + >> include/migration/migration.h | 32 +++++++ >> include/sysemu/sysemu.h | 1 + >> migration-local.c | 228 +++++++++++++++++++++++++++++++++++++++++ >> migration-unix.c | 60 ++++++++++++ >> migration.c | 8 -- >> qapi-schema.json | 14 +++ >> qemu-options.hx | 9 ++ >> qmp-commands.hx | 22 +++++ >> savevm.c | 100 +++++++++++++++-- >> vl.c | 14 +++ >> 16 files changed, 613 insertions(+), 19 deletions(-) >> create mode 100644 migration-local.c >> > -- Lei