From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:56635) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UTxvA-0005D4-NH for qemu-devel@nongnu.org; Sun, 21 Apr 2013 13:19:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UTxv7-0001eu-W9 for qemu-devel@nongnu.org; Sun, 21 Apr 2013 13:19:24 -0400 Received: from e8.ny.us.ibm.com ([32.97.182.138]:38332) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UTxv7-0001em-Rt for qemu-devel@nongnu.org; Sun, 21 Apr 2013 13:19:21 -0400 Received: from /spool/local by e8.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Sun, 21 Apr 2013 13:19:20 -0400 Received: from d01relay02.pok.ibm.com (d01relay02.pok.ibm.com [9.56.227.234]) by d01dlp02.pok.ibm.com (Postfix) with ESMTP id 542926E803C for ; Sun, 21 Apr 2013 13:19:15 -0400 (EDT) Received: from d01av05.pok.ibm.com (d01av05.pok.ibm.com [9.56.224.195]) by d01relay02.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id r3LHJITE296024 for ; Sun, 21 Apr 2013 13:19:18 -0400 Received: from d01av05.pok.ibm.com (loopback [127.0.0.1]) by d01av05.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id r3LHJHOs025104 for ; Sun, 21 Apr 2013 13:19:17 -0400 Message-ID: <51741F95.20501@linux.vnet.ibm.com> Date: Sun, 21 Apr 2013 13:19:17 -0400 From: "Michael R. Hines" MIME-Version: 1.0 References: <1366240040-10730-1-git-send-email-mrhines@linux.vnet.ibm.com> <1366240040-10730-8-git-send-email-mrhines@linux.vnet.ibm.com> <51706E9C.9@redhat.com> <20130420170240.GA16115@redhat.com> <5173E759.5080906@redhat.com> <20130421141746.GB13512@redhat.com> In-Reply-To: <20130421141746.GB13512@redhat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PULL v4 07/11] rdma: introduce capability for chunk registration List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Michael S. Tsirkin" Cc: aliguori@us.ibm.com, quintela@redhat.com, qemu-devel@nongnu.org, owasserm@redhat.com, abali@us.ibm.com, mrhines@us.ibm.com, gokul@us.ibm.com, Paolo Bonzini On 04/21/2013 10:17 AM, Michael S. Tsirkin wrote: > On Sun, Apr 21, 2013 at 03:19:21PM +0200, Paolo Bonzini wrote: >> Il 20/04/2013 19:02, Michael S. Tsirkin ha scritto: >>>>> I guess the opposite sense could be named 'x-rdma-pin-all'; default >>>>> false means to do chunk registration and release, >>> chunk release only happens after migration is complete unfortunately. >>> This means that eventually all initialized memory is pinned, regardless >>> of the setting (this is fixable but there's no plan to fix this, at this >>> point). So pin-all might be misleading to some. >>> >>> I agree 'chunk' is unnecessarily low level though. >>> The only difference ATM is pinning of uninitialized memory so I think a >>> better name would be 'x-rdma-pin-uninitialized' or some such. >>> >> x-rdma-pin-all is a better choice. x-rdma-pin-uninitialized is also too >> low level. >> >> Since this series is likely to miss 1.5 at this point, we could >> implement the unregistration part of the protocol in the destination. >> This way, any heuristic we add to the source will not break backwards >> compatibility. >> >> Paolo > To test, you'll have to implement it in the source too. > That's probably a good idea anyway, though doing this > efficiently might need more thought, and some of > the tricks I described earlier (pipelining, > registration cache) might be needed. > Though I'm curious what the performance impact would be > even without these tricks. > We already had a agreement to merge with ulimit -l + ibv_reg_mr() + ERROR + abort migration. We (IBM Research) will not commit to implementing this unless someone provides hard data showing it not to adversely effect migration performance. - Michael