From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:42718) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UPxGL-0004BO-Kt for qemu-devel@nongnu.org; Wed, 10 Apr 2013 11:48:46 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UPxGE-00063E-Tv for qemu-devel@nongnu.org; Wed, 10 Apr 2013 11:48:41 -0400 Received: from e7.ny.us.ibm.com ([32.97.182.137]:48871) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UPxGE-000635-LV for qemu-devel@nongnu.org; Wed, 10 Apr 2013 11:48:34 -0400 Received: from /spool/local by e7.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 10 Apr 2013 11:48:33 -0400 Received: from d01relay06.pok.ibm.com (d01relay06.pok.ibm.com [9.56.227.116]) by d01dlp01.pok.ibm.com (Postfix) with ESMTP id 8447338C8065 for ; Wed, 10 Apr 2013 11:48:31 -0400 (EDT) Received: from d01av02.pok.ibm.com (d01av02.pok.ibm.com [9.56.224.216]) by d01relay06.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id r3AFmQXm30146634 for ; Wed, 10 Apr 2013 11:48:26 -0400 Received: from d01av02.pok.ibm.com (loopback [127.0.0.1]) by d01av02.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id r3AFmODK023025 for ; Wed, 10 Apr 2013 12:48:25 -0300 Message-ID: <516589C8.1040606@linux.vnet.ibm.com> Date: Wed, 10 Apr 2013 11:48:24 -0400 From: "Michael R. Hines" MIME-Version: 1.0 References: <20130324155153.GA8597@redhat.com> <515F3160.4020007@linux.vnet.ibm.com> <515F3948.40205@linux.vnet.ibm.com> <515F42EE.8060203@linux.vnet.ibm.com> <20130409190350.GB8212@redhat.com> <5164BFE3.4040204@linux.vnet.ibm.com> <20130410032449.GA12777@redhat.com> <5164EB5F.80403@linux.vnet.ibm.com> <20130410053255.GC12777@redhat.com> In-Reply-To: <20130410053255.GC12777@redhat.com> Content-Type: multipart/alternative; boundary="------------040004040303040409010709" Subject: Re: [Qemu-devel] [PATCHv2] rdma: add a new IB_ACCESS_GIFT flag List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Michael S. Tsirkin" Cc: Roland Dreier , qemu-devel@nongnu.org, "linux-rdma@vger.kernel.org" , Yishai Hadas , LKML , Hal Rosenstock , Jason Gunthorpe , Sean Hefty , Christoph Lameter This is a multi-part message in MIME format. --------------040004040303040409010709 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit There's a very nice, simple client/server RDMA application on the internet you can use to test your patch. http://thegeekinthecorner.wordpress.com/2010/09/28/rdma-read-and-write-with-ib-verbs/ This guy provides the source code which dumps several gigabytes over RDMA to the other side. There's no need to run QEMU to test your patch, assuming you have access to infiniband hardware. - Michael On 04/10/2013 01:32 AM, Michael S. Tsirkin wrote: > On Wed, Apr 10, 2013 at 12:32:31AM -0400, Michael R. Hines wrote: >> On 04/09/2013 11:24 PM, Michael S. Tsirkin wrote: >>> Which mechanism do you refer to? You patches still seem to pin >>> each page in guest memory at some point, which will break all COW. >>> In particular any pagemap tricks to detect duplicates on source >>> that I suggested won't work. >> Sorry, I mispoke. I'm reffering to dynamic server page registration. >> >> Of course it does not eliminate pinning - but it does mitigate the >> foot print of the VM as a feature that was requested. >> >> I have implemented it and documented it. >> >> - Michael > Okay, but GIFT is supposed to be used on send side: it's only allowed > with local/remote read access, and serves to reduce memory usage > on send side. > For example, disable zero page detection and look at memory usage > on send side before and after migration. > Dynamic registration on the receive side is nice but seems > completely unrelated ... > --------------040004040303040409010709 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit

There's a very nice, simple client/server RDMA application on the
internet you can use to test your patch.

http://thegeekinthecorner.wordpress.com/2010/09/28/rdma-read-and-write-with-ib-verbs/

This guy provides the source code which dumps several gigabytes over RDMA
to the other side.

There's no need to run QEMU to test your patch,
assuming you have access to infiniband hardware.

- Michael

On 04/10/2013 01:32 AM, Michael S. Tsirkin wrote:
On Wed, Apr 10, 2013 at 12:32:31AM -0400, Michael R. Hines wrote:

On 04/09/2013 11:24 PM, Michael S. Tsirkin wrote:

Which mechanism do you refer to? You patches still seem to pin
each page in guest memory at some point, which will break all COW.
In particular any pagemap tricks to detect duplicates on source
that I suggested won't work.

Sorry, I mispoke. I'm reffering to dynamic server page registration.

Of course it does not eliminate pinning - but it does mitigate the
foot print of the VM as a feature that was requested.

I have implemented it and documented it.

- Michael

Okay, but GIFT is supposed to be used on send side: it's only allowed
with local/remote read access, and serves to reduce memory usage
on send side.
For example, disable zero page detection and look at memory usage
on send side before and after migration.
Dynamic registration on the receive side is nice but seems
completely unrelated ...

--------------040004040303040409010709--