From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1M6QmP-0003xw-TR for qemu-devel@nongnu.org; Tue, 19 May 2009 10:58:58 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1M6QmL-0003su-Td for qemu-devel@nongnu.org; Tue, 19 May 2009 10:58:57 -0400 Received: from [199.232.76.173] (port=56871 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1M6QmL-0003sg-Jf for qemu-devel@nongnu.org; Tue, 19 May 2009 10:58:53 -0400 Received: from mx2.redhat.com ([66.187.237.31]:51533) by monty-python.gnu.org with esmtp (Exim 4.60) (envelope-from ) id 1M6QmK-0005l9-Tz for qemu-devel@nongnu.org; Tue, 19 May 2009 10:58:53 -0400 Message-ID: <4A12C942.4090708@redhat.com> Date: Tue, 19 May 2009 17:59:14 +0300 From: Dor Laor MIME-Version: 1.0 Subject: Re: [Qemu-devel] [PATCH] ram_save_live: add a no-progress convergence rule References: <1242731347-1558-1-git-send-email-uril@redhat.com> <4A12AD80.701@codemonkey.ws> <20090519144101.GA16372@shell.devel.redhat.com> In-Reply-To: <20090519144101.GA16372@shell.devel.redhat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Reply-To: dlaor@redhat.com List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Glauber Costa Cc: Uri Lublin , qemu-devel@nongnu.org Glauber Costa wrote: > On Tue, May 19, 2009 at 08:00:48AM -0500, Anthony Liguori wrote: > >> Uri Lublin wrote: >> >>> Currently the live-part (section QEMU_VM_SECTION_PART) of >>> ram_save_live has only one convergence rule, which is >>> when the number of dirty pages is smaller than a threshold. >>> >>> When the guest uses more memory pages than the threshold (e.g. >>> playing a movie, copying files, sending/receiving many packets), >>> it may take a very long time before convergence according to >>> this rule. >>> >>> This patch (re)introduces a no-progress convergence rule, which limit >>> the number of times the migration process is not progressing >>> (and even regressing), with regards to the number of dirty >>> pages. No-progress means that the number of pages that got >>> dirty is larger than the number of pages that got transferred >>> to the destination during the last transfer. >>> This rule applies only after the first round (in which most >>> memory pages are being transferred). >>> >>> Also this patch enlarges the number-dirty-pages threshold (of >>> the first convergence rule) to 50 pages (was 10) >>> >>> Signed-off-by: Uri Lublin >>> >>> >> The right place to do this is in a management tool. An arbitrary >> convergence rule of 50 can do more damage than good. >> >> For some set of users, it's better that live migration fail than it >> cause an arbitrarily long pause in the guest which can result in dropped >> TCP connections, soft lock ups, and other badness. >> >> A management tool can force convergence by issuing a "stop" command in >> the monitor. I suspect a management tool cares more about wall-clock >> time than number of iterations too so a valid metric would be something >> along the lines of if not converged after N seconds, issue stop monitor >> command where N is calculated from available network bandwidth and guest >> memory size. >> > Another possibility is for the management tool to increase the bandwidth for > little periods if it perceives that no progress is being made. > > Anyhow, I completely agree that we should not introduce this in qemu. > > However, maybe we could augment our "info migrate" to provide more info about > the internal state of migration, so the mgmt tool can take a more informed > decision? > > The problem is that if migration is not progressing since the guest is dirtying pages faster than the migration protocol can send, than we just waist time and cpu. The minimum is to notify the monitor interface in order to let mgmt daemon to trap it. We can easily see this issue while running iperf in the guest or any other high load/dirty pages scenario. We can also make it configurable using the monitor migrate command. For example: migrate -d -no_progress -threshold=x tcp:....