From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=43428 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1PNebc-0006TP-KB for qemu-devel@nongnu.org; Tue, 30 Nov 2010 23:47:51 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1PNQjX-00014y-UU for qemu-devel@nongnu.org; Tue, 30 Nov 2010 08:59:05 -0500 Received: from mx1.redhat.com ([209.132.183.28]:7282) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1PNQjX-0008Kj-Km for qemu-devel@nongnu.org; Tue, 30 Nov 2010 08:59:03 -0500 Message-ID: <4CF5030B.40703@redhat.com> Date: Tue, 30 Nov 2010 15:58:35 +0200 From: Avi Kivity MIME-Version: 1.0 References: <9b23b9b4cee242591bdb356c838a9cfb9af033c1.1290552026.git.quintela@redhat.com> <4CF45D67.5010906@codemonkey.ws> <4CF4A478.8080209@redhat.com> <4CF5008F.2090306@codemonkey.ws> In-Reply-To: <4CF5008F.2090306@codemonkey.ws> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [Qemu-devel] Re: [PATCH 09/10] Exit loop if we have been there too long List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Anthony Liguori Cc: Paolo Bonzini , Juan Quintela , qemu-devel@nongnu.org, kvm-devel , Juan Quintela On 11/30/2010 03:47 PM, Anthony Liguori wrote: > On 11/30/2010 01:15 AM, Paolo Bonzini wrote: >> On 11/30/2010 03:11 AM, Anthony Liguori wrote: >>> >>> BufferedFile should hit the qemu_file_rate_limit check when the socket >>> buffer gets filled up. >> >> The problem is that the file rate limit is not hit because work is >> done elsewhere. The rate can limit the bandwidth used and makes QEMU >> aware that socket operations may block (because that's what the >> buffered file freeze/unfreeze logic does); but it cannot be used to >> limit the _time_ spent in the migration code. > > Yes, it can, if you set the rate limit sufficiently low. > > The caveats are 1) the kvm.ko interface for dirty bits doesn't scale > for large memory guests so we spend a lot more CPU time walking it > than we should 2) zero pages cause us to burn a lot more CPU time than > we otherwise would because compressing them is so effective. What's the problem with burning that cpu? per guest page, compressing takes less than sending. Is it just an issue of qemu mutex hold time? > > In the short term, fixing (2) by accounting zero pages as full sized > pages should "fix" the problem. > > In the long term, we need a new dirty bit interface from kvm.ko that > uses a multi-level table. That should dramatically improve scan > performance. Why would a multi-level table help? (or rather, please explain what you mean by a multi-level table). Something we could do is divide memory into more slots, and polling each slot when we start to scan its page range. That reduces the time between sampling a page's dirtiness and sending it off, and reduces the latency incurred by the sampling. There are also non-interface-changing ways to reduce this latency, like O(1) write protection, or using dirty bits instead of write protection when available. > We also need to implement live migration in a separate thread that > doesn't carry qemu_mutex while it runs. IMO that's the biggest hit currently. -- error compiling committee.c: too many arguments to function