From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49002) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UnA3H-0004qK-Nc for qemu-devel@nongnu.org; Thu, 13 Jun 2013 12:07:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UnA3D-0000M2-5s for qemu-devel@nongnu.org; Thu, 13 Jun 2013 12:07:07 -0400 Received: from e32.co.us.ibm.com ([32.97.110.150]:53347) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UnA3C-0008IY-V9 for qemu-devel@nongnu.org; Thu, 13 Jun 2013 12:07:03 -0400 Received: from /spool/local by e32.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 13 Jun 2013 08:51:58 -0600 Received: from d03relay01.boulder.ibm.com (d03relay01.boulder.ibm.com [9.17.195.226]) by d03dlp01.boulder.ibm.com (Postfix) with ESMTP id 1B35B1FF0060 for ; Thu, 13 Jun 2013 08:41:09 -0600 (MDT) Received: from d03av03.boulder.ibm.com (d03av03.boulder.ibm.com [9.17.195.169]) by d03relay01.boulder.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id r5DEk8Mx060416 for ; Thu, 13 Jun 2013 08:46:08 -0600 Received: from d03av03.boulder.ibm.com (loopback [127.0.0.1]) by d03av03.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id r5DEk71f024350 for ; Thu, 13 Jun 2013 08:46:07 -0600 Message-ID: <51B9DB22.7020305@linux.vnet.ibm.com> Date: Thu, 13 Jun 2013 10:45:54 -0400 From: "Michael R. Hines" MIME-Version: 1.0 References: <1370880226-2208-1-git-send-email-mrhines@linux.vnet.ibm.com> <4168C988EBDF2141B4E0B6475B6A73D10CE2AAC1@G6W2488.americas.hpqcorp.net> <51B60ABA.2070401@linux.vnet.ibm.com> <4168C988EBDF2141B4E0B6475B6A73D10CE2BAE7@G6W2488.americas.hpqcorp.net> <51B7B652.3070905@linux.vnet.ibm.com> <51B85EE5.1050702@hp.com> <51B868B3.9090607@linux.vnet.ibm.com> <51B9A614.2050101@hp.com> <51B9BFCA.4050008@linux.vnet.ibm.com> <51B9CE2E.5080504@hp.com> In-Reply-To: <51B9CE2E.5080504@hp.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH v7 00/12] rdma: migration support List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Chegu Vinod Cc: "qemu-devel@nongnu.org" , Paolo Bonzini , "mrhines@us.ibm.com" , Anthony Liguori , Juan Jose Quintela Carreira On 06/13/2013 09:50 AM, Chegu Vinod wrote: > > Attempted to migrate a smaller guest 10Vcpu/64GB (the guest was just > idle) with the pin-all option. > > It took ~20 sec to do the pin of the guest's RAM (this is the time > when the guest is "frozen") and then the actual migration started... > and took about ~26 secs to complete.. i.e. "info migrate" reported > the total migration time as ~26secs. > > From a user point of view the total clock time from when the migration > operation was actually initiated to the time the guest resumed on the > target host it was : ~20 + ~26 = ~46 secs ...hence my question. > (CC'ing qemu-devel, now.) Ah, ok, yes, I see now - that's a bug that I would recommend reporting to the QEMU maintainer, actually: Here is the sequence of events inside of QEMU: 1. issue the migrate command on the QEMU monitor: 2. qmp_migrate() gets called 3. (tcp|rdma|unix|etc)_start_outgoing_migration() gets called <= pinning occurs here 4. start migration_thread() pthread() <= take first timestamp 5. migration complete <= take another timestamp and subtract for total time 6. exit migration_thread() The problem, as you can see is that "take first timestamp" needs to happen earlier in step #2. This is definitely a "nuisance", but not specific to RDMA, and I think a patch should be submitted, probably by one of the maintainers which moves the timestamp up to a higher level. Does that make sense? - Michael