From mboxrd@z Thu Jan 1 00:00:00 1970 From: Juan Quintela Subject: Re: [PATCH 09/10] Exit loop if we have been there too long Date: Wed, 01 Dec 2010 14:45:28 +0100 Message-ID: References: <9b23b9b4cee242591bdb356c838a9cfb9af033c1.1290552026.git.quintela@redhat.com> <4CF45D67.5010906@codemonkey.ws> <4CF4A478.8080209@redhat.com> <4CF5008F.2090306@codemonkey.ws> <4CF5030B.40703@redhat.com> <4CF50783.90402@codemonkey.ws> <4CF509C1.9@redhat.com> <20101201102034.07eff649.yoshikawa.takuya@oss.ntt.co.jp> <4CF6412D.5030601@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Takuya Yoshikawa , Anthony Liguori , Paolo Bonzini , qemu-devel@nongnu.org, kvm-devel To: Avi Kivity Return-path: Received: from mx1.redhat.com ([209.132.183.28]:64187 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755173Ab0LANq3 (ORCPT ); Wed, 1 Dec 2010 08:46:29 -0500 In-Reply-To: <4CF6412D.5030601@redhat.com> (Avi Kivity's message of "Wed, 01 Dec 2010 14:35:57 +0200") Sender: kvm-owner@vger.kernel.org List-ID: Avi Kivity wrote: > On 12/01/2010 03:52 AM, Juan Quintela wrote: >> > - 512GB guest is really the target? >> >> no, problems exist with smaller amounts of RAM. with 16GB guest it is >> trivial to get 1s stalls, 64GB guest, 3-4s, with more memory, migration >> is flaky to say the less. >> >> > - how much cpu time can we use for these things? >> >> the problem here is that we are forced to walk the bitmap too many >> times, we want to do it less times. > > How much time is spent walking bitmaps? Are you sure this is the problem? see my 10/10 patch, that one makes ram_save_live() to move from 80% of the time in the profile to the second one with ~5% or so. The important bit is: static uint64_t ram_save_remaining(void) { - RAMBlock *block; - uint64_t count = 0; - - QLIST_FOREACH(block, &ram_list.blocks, next) { - ram_addr_t addr; - for (addr = block->offset; addr < block->offset + block->length; - addr += TARGET_PAGE_SIZE) { - if (cpu_physical_memory_get_dirty(addr, MIGRATION_DIRTY_FLAG)) { - count++; - } - } - } - - return count; + return ram_list.dirty_pages; } We dont' need to walk all bitmap to see how much memory is remaining. syncing the bitmap is also expensive, but just now we have other issues that hide it. That is the reason that I didn't started trynig to get a better interface with kvm, 1st I will try to remove the bigger bottlenecks. >> > - how many dirty pages do we have to care? >> >> default values and assuming 1Gigabit ethernet for ourselves ~9.5MB of >> dirty pages to have only 30ms of downtime. > > 1Gb/s * 30ms = 100 MB/s * 30 ms = 3 MB. I will learn to make math with time and bytes at some point O:-) Later, Juan.