From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49904) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Vjovm-0008UK-4w for qemu-devel@nongnu.org; Fri, 22 Nov 2013 06:29:56 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Vjovf-0006hE-CJ for qemu-devel@nongnu.org; Fri, 22 Nov 2013 06:29:50 -0500 Received: from e23smtp03.au.ibm.com ([202.81.31.145]:53309) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Vjove-0006h0-Jc for qemu-devel@nongnu.org; Fri, 22 Nov 2013 06:29:43 -0500 Received: from /spool/local by e23smtp03.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 22 Nov 2013 21:29:38 +1000 Received: from d23relay05.au.ibm.com (d23relay05.au.ibm.com [9.190.235.152]) by d23dlp02.au.ibm.com (Postfix) with ESMTP id A6AFF2BB0052 for ; Fri, 22 Nov 2013 22:29:36 +1100 (EST) Received: from d23av02.au.ibm.com (d23av02.au.ibm.com [9.190.235.138]) by d23relay05.au.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id rAMBBW3w50462888 for ; Fri, 22 Nov 2013 22:11:38 +1100 Received: from d23av02.au.ibm.com (localhost [127.0.0.1]) by d23av02.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id rAMBTT0c001810 for ; Fri, 22 Nov 2013 22:29:30 +1100 Message-ID: <528F4001.3050600@linux.vnet.ibm.com> Date: Fri, 22 Nov 2013 19:29:05 +0800 From: Lei Li MIME-Version: 1.0 References: <1385025100-3191-1-git-send-email-lilei@linux.vnet.ibm.com> <20131121101934.GA9135@redhat.com> In-Reply-To: <20131121101934.GA9135@redhat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH 0/17 v3] Localhost migration with side channel for ram List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Daniel P. Berrange" Cc: Andrea Arcangeli , quintela@redhat.com, qemu-devel@nongnu.org, mrhines@linux.vnet.ibm.com, mdroth@linux.vnet.ibm.com, aliguori@amazon.com, lagarcia@br.ibm.com, pbonzini@redhat.com, rcj@linux.vnet.ibm.com On 11/21/2013 06:19 PM, Daniel P. Berrange wrote: > On Thu, Nov 21, 2013 at 05:11:23PM +0800, Lei Li wrote: >> This patch series tries to introduce a mechanism using side >> channel pipe for RAM via SCM_RIGHTS with unix domain socket >> protocol migration. >> >> This side channel is used for the page flipping by vmsplice, >> which is the internal mechanism for localhost migration that >> we are trying to add to QEMU. The backgroud info and previous >> patch series for reference, >> >> Localhost migration >> http://lists.nongnu.org/archive/html/qemu-devel/2013-08/msg02916.html >> >> migration: Introduce side channel for RAM >> http://lists.gnu.org/archive/html/qemu-devel/2013-09/msg04043.html >> >> I have picked patches from the localhost migration series and rebased >> it on the series of side channel, now it is a complete series that >> passed the basic test. >> >> Please let me know if there is anything needs to be fixed or improved. >> Your suggestions and comments are very welcome, and thanks to Paolo >> for his continued review and useful suggestions. > In discussions about supporting this for libvirt, we were told that > when this localhost migration fails, you cannot re-start the guest > on the original source QEMU. > > If this is true, this implementation is not satisfactory IMHO. One > of the main motivations of this feature is to allow for in-place > live upgrades of QEMU binaries, for people who can't tolerate the > downtime of restarting their guests, and whom don't have a spare > host to migrate them to. > > If people are using this because they can't tolerate any downtime > of the guest, then we need to be able to fully deal with failure to > complete migration by switching back to the original QEMU process, > as we can do with normal non-localhost migration. Hi Daniel, Page flipping is introduced here not primarily for low downtime, but more to avoid requiring that there is enough free memory to fit an additional copy of the largest guest which is the requirement today with current localhost migration as the additional explanation from Anthony in first proposal version [1]. Of course low downtime is also important to the page flipping migration as the use case of it is to allow 'live' upgrade of a running QEMU instance, so we expect page flipping through vmsplice is fast enough to meet it. As an initial implementation of this feature right now, the downtime is not good, but we are working on it as there has been some work on kernel side [2]. During the page flipping migration, ram page of source guest would be flipped to the destination, that's why the source guest can not be resumed. AFAICT, the page flipping migration may fail at the connection stage (including the exchange of pipe fd) and migration register stage (say any blocker like unsupported migration device), but it could be resumed for such situation since the memory has not been flipped to another content. Once the connection is successfully setup, it would proceed the transmission of ram page which hardly fails. And for the failure handling in Libvirt, ZhengSheng has proposed that restarts the old QEMU instead of resume. I know 'hardly' is not an good answer to your concern, but it is the cost of the limited memory IMO. So if downtime is the key to the user, or if it's *zero toleration of the restarting of QEMU, page flipping migration might not be a good choice. From the perspective of management app like Libvirt, as the 'live upgrade' of QEMU will be done through localhost migration, and there are other migration solutions which have lower downtime, like the real live migration and the postcopy migration that Paolo mentioned in the previous version [3]. Why not have more than one choice for it? [1]http://lists.gnu.org/archive/html/qemu-devel/2013-06/msg02577.html [2]http://article.gmane.org/gmane.linux.kernel/1574277 [3]http://lists.gnu.org/archive/html/qemu-devel/2013-10/msg03212.html > Regards, > Daniel -- Lei