From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33948) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UkpO0-0004oB-DR for qemu-devel@nongnu.org; Fri, 07 Jun 2013 01:38:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UkpNy-000118-15 for qemu-devel@nongnu.org; Fri, 07 Jun 2013 01:38:52 -0400 Received: from e37.co.us.ibm.com ([32.97.110.158]:55938) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UkpNx-0000yc-QV for qemu-devel@nongnu.org; Fri, 07 Jun 2013 01:38:49 -0400 Received: from /spool/local by e37.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 6 Jun 2013 23:38:26 -0600 Received: from d03relay02.boulder.ibm.com (d03relay02.boulder.ibm.com [9.17.195.227]) by d03dlp02.boulder.ibm.com (Postfix) with ESMTP id CFCF03E40030 for ; Thu, 6 Jun 2013 23:38:05 -0600 (MDT) Received: from d03av03.boulder.ibm.com (d03av03.boulder.ibm.com [9.17.195.169]) by d03relay02.boulder.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id r575cWPX103154 for ; Thu, 6 Jun 2013 23:38:32 -0600 Received: from d03av03.boulder.ibm.com (loopback [127.0.0.1]) by d03av03.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id r575cMlT027336 for ; Thu, 6 Jun 2013 23:38:23 -0600 Message-ID: <51B171CD.6060503@linux.vnet.ibm.com> Date: Fri, 07 Jun 2013 01:38:21 -0400 From: "Michael R. Hines" MIME-Version: 1.0 References: <51844811.4030001@hp.com> <518BDADB.1070705@linux.vnet.ibm.com> <518C2135.306@hp.com> <518C26F6.5080009@linux.vnet.ibm.com> <51AAC589.3080302@linux.vnet.ibm.com> <51B1208C.2080406@hp.com> In-Reply-To: <51B1208C.2080406@hp.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH v6 00/11] rdma: migration support List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Chegu Vinod Cc: Karen Noel , Juan Jose Quintela Carreira , qemu-devel qemu-devel , Orit Wasserman , "Michael R. Hines" , Anthony Liguori , Paolo Bonzini OK, this looks excellent. I think we're ready for a PULL request now - I will submit to Juan - I've already got a signed-off from Paolo & Eric. I think we've got sufficient testing now. I'm expecting to get access to a big machine in the next week or two (256G machine) - and I should be able to reproduce your mlock() delay issue at that time. With such a big VM - I think pinning will help you significantly. Stay tuned, - Michael On 06/06/2013 07:51 PM, Chegu Vinod wrote: > >> > Hi Michael, > > Got some limited time on the systems so gave your latest bits a quick > try today (with the default no pinning) and it seems to be better than > before. > > Ran a Java warehouse workload where the guest was 85-90% busy... > > For both cases > (qemu) migrate_set_speed 40G > (qemu) migrate_set_downtime 2 > (qemu) migrate -d x-rdma:: > > ... > > 20VCPU/256G guest > > (qemu) info migrate > capabilities: xbzrle: off x-rdma-pin-all: off > Migration status: completed > total time: 106994 milliseconds > downtime: 3795 milliseconds > transferred ram: 15425453 kbytes > throughput: 20418.27 mbps > remaining ram: 0 kbytes > total ram: 268444224 kbytes > duplicate: 64707112 pages > skipped: 0 pages > normal: 3839625 pages > normal bytes: 15358500 kbytes > > ---- > > 40VCPU/512G guest <- I had more warehouse threads with higher > heap size etc. to make the guest busy...and hence it seems to have > taken a while to converge. > > (qemu) info migrate > capabilities: xbzrle: off x-rdma-pin-all: off > Migration status: completed > total time: 2470056 milliseconds > downtime: 6254 milliseconds > transferred ram: 3230142002 kbytes > throughput: 22118.67 mbps > remaining ram: 0 kbytes > total ram: 536879680 kbytes > duplicate: 127436402 pages > skipped: 0 pages > normal: 807307274 pages > normal bytes: 3229229096 kbytes > > > <..> >