From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36441) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VCkpv-0002hK-RQ for qemu-devel@nongnu.org; Fri, 23 Aug 2013 02:27:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1VCkpm-0000Rl-TQ for qemu-devel@nongnu.org; Fri, 23 Aug 2013 02:27:07 -0400 Received: from e23smtp08.au.ibm.com ([202.81.31.141]:54083) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VCkpm-0000RL-Aq for qemu-devel@nongnu.org; Fri, 23 Aug 2013 02:26:58 -0400 Received: from /spool/local by e23smtp08.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 23 Aug 2013 16:23:39 +1000 Received: from d23relay04.au.ibm.com (d23relay04.au.ibm.com [9.190.234.120]) by d23dlp01.au.ibm.com (Postfix) with ESMTP id A3F9B2CE804C for ; Fri, 23 Aug 2013 16:26:50 +1000 (EST) Received: from d23av03.au.ibm.com (d23av03.au.ibm.com [9.190.234.97]) by d23relay04.au.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id r7N6Aotm3932570 for ; Fri, 23 Aug 2013 16:10:50 +1000 Received: from d23av03.au.ibm.com (localhost [127.0.0.1]) by d23av03.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id r7N6QnbU016385 for ; Fri, 23 Aug 2013 16:26:50 +1000 Message-ID: <52170060.4050104@linux.vnet.ibm.com> Date: Fri, 23 Aug 2013 14:25:36 +0800 From: Lei Li MIME-Version: 1.0 References: <1377069536-12658-1-git-send-email-lilei@linux.vnet.ibm.com> <1377069536-12658-14-git-send-email-lilei@linux.vnet.ibm.com> <52149B09.705@redhat.com> In-Reply-To: <52149B09.705@redhat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH 13/18] arch_init: adjust ram_save_setup() for migrate_is_localhost List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Paolo Bonzini Cc: aarcange@redhat.com, quintela@redhat.com, qemu-devel@nongnu.org, mrhines@linux.vnet.ibm.com, Anthony Liguori , lagarcia@br.ibm.com, rcj@linux.vnet.ibm.com On 08/21/2013 06:48 PM, Paolo Bonzini wrote: > Il 21/08/2013 09:18, Lei Li ha scritto: >> Send all the ram blocks hooked by save_page, which will copy >> ram page and MADV_DONTNEED the page just copied. > You should implement this entirely in the hook. > > It will be a little less efficient because of the dirty bitmap overhead, > but you should aim at having *zero* changes in arch_init.c and migration.c. Yes, the reason I modify the migration_thread() to have new process that send all the ram pages in adjusted qemu_savevm_state_begin stage and send device states in qemu_savevm_device_state stage for localhost migration is to avoid the bitmap thing, which is a little less efficient just like you mentioned above. The performance assurance is very important to this feature, our goal is 100ms of downtime for a 1TB guest. > > Paolo > >> Signed-off-by: Lei Li >> --- >> arch_init.c | 19 +++++++++++++------ >> 1 files changed, 13 insertions(+), 6 deletions(-) >> >> diff --git a/arch_init.c b/arch_init.c >> index 434a4ca..cbbb4db 100644 >> --- a/arch_init.c >> +++ b/arch_init.c >> @@ -474,7 +474,7 @@ static int ram_save_block(QEMUFile *f, bool last_stage) >> /* In doubt sent page as normal */ >> bytes_sent = -1; >> ret = ram_control_save_page(f, block->offset, >> - offset, TARGET_PAGE_SIZE, &bytes_sent); >> + offset, TARGET_PAGE_SIZE, &bytes_sent); >> >> if (ret != RAM_SAVE_CONTROL_NOT_SUPP) { >> if (ret != RAM_SAVE_CONTROL_DELAYED) { >> @@ -613,11 +613,13 @@ static int ram_save_setup(QEMUFile *f, void *opaque) >> RAMBlock *block; >> int64_t ram_pages = last_ram_offset() >> TARGET_PAGE_BITS; >> >> - migration_bitmap = bitmap_new(ram_pages); >> - bitmap_set(migration_bitmap, 0, ram_pages); >> - migration_dirty_pages = ram_pages; >> - mig_throttle_on = false; >> - dirty_rate_high_cnt = 0; >> + if (!migrate_is_localhost()) { >> + migration_bitmap = bitmap_new(ram_pages); >> + bitmap_set(migration_bitmap, 0, ram_pages); >> + migration_dirty_pages = ram_pages; >> + mig_throttle_on = false; >> + dirty_rate_high_cnt = 0; >> + } >> >> if (migrate_use_xbzrle()) { >> XBZRLE.cache = cache_init(migrate_xbzrle_cache_size() / >> @@ -641,6 +643,11 @@ static int ram_save_setup(QEMUFile *f, void *opaque) >> migration_bitmap_sync(); >> qemu_mutex_unlock_iothread(); >> >> + if (migrate_is_localhost()) { >> + ram_save_blocks(f); >> + return 0; >> + } >> + >> qemu_put_be64(f, ram_bytes_total() | RAM_SAVE_FLAG_MEM_SIZE); >> >> QTAILQ_FOREACH(block, &ram_list.blocks, next) { >> > -- Lei