From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Michael R. Hines" Subject: Re: [PATCHv2] rdma: add a new IB_ACCESS_GIFT flag Date: Fri, 05 Apr 2013 16:51:20 -0400 Message-ID: <515F3948.40205@linux.vnet.ibm.com> References: <20130324155153.GA8597@redhat.com> <515F3160.4020007@linux.vnet.ibm.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: Sender: linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: Roland Dreier Cc: "Michael S. Tsirkin" , Jason Gunthorpe , Sean Hefty , Hal Rosenstock , Yishai Hadas , Christoph Lameter , "linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , LKML , qemu-devel-qX2TKyscuCcdnm+yROfE0A@public.gmane.org List-Id: linux-rdma@vger.kernel.org Sorry, I was wrong. ignore the comments about cgroups. That's still broken. (i.e. trying to register RDMA memory while using a cgroup swap limit cause the process get killed). But the GIFT flag patch works (my understanding is that GIFT flag allows the adapter to transmit stale memory information, it does not have anything to do with cgroups specifically). Am I missing something? I was only testing the GIFT flag patch. Note: I only turned it on - I did not verify the (non) consitency of the memory that was transmitted. - Michael On 04/05/2013 04:43 PM, Roland Dreier wrote: > On Fri, Apr 5, 2013 at 1:17 PM, Michael R. Hines > wrote: >> I also removed the IBV_*_WRITE flags on the sender-side and activated >> cgroups with the "memory.memsw.limit_in_bytes" activated and the migration >> with RDMA also succeeded without any problems (both with *and* without GIFT >> also worked). > Not sure I'm interpreting this correctly. Are you saying that things > worked without actually setting the GIFT flag? In which case why are > we adding this flag? > > - R. > -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1162997Ab3DEUvb (ORCPT ); Fri, 5 Apr 2013 16:51:31 -0400 Received: from e34.co.us.ibm.com ([32.97.110.152]:42164 "EHLO e34.co.us.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1162134Ab3DEUv3 (ORCPT ); Fri, 5 Apr 2013 16:51:29 -0400 Message-ID: <515F3948.40205@linux.vnet.ibm.com> Date: Fri, 05 Apr 2013 16:51:20 -0400 From: "Michael R. Hines" User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130106 Thunderbird/17.0.2 MIME-Version: 1.0 To: Roland Dreier CC: "Michael S. Tsirkin" , Jason Gunthorpe , Sean Hefty , Hal Rosenstock , Yishai Hadas , Christoph Lameter , "linux-rdma@vger.kernel.org" , LKML , qemu-devel@nongnu.org Subject: Re: [PATCHv2] rdma: add a new IB_ACCESS_GIFT flag References: <20130324155153.GA8597@redhat.com> <515F3160.4020007@linux.vnet.ibm.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-TM-AS-MML: No X-Content-Scanned: Fidelis XPS MAILER x-cbid: 13040520-2876-0000-0000-00000728A9D6 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Sorry, I was wrong. ignore the comments about cgroups. That's still broken. (i.e. trying to register RDMA memory while using a cgroup swap limit cause the process get killed). But the GIFT flag patch works (my understanding is that GIFT flag allows the adapter to transmit stale memory information, it does not have anything to do with cgroups specifically). Am I missing something? I was only testing the GIFT flag patch. Note: I only turned it on - I did not verify the (non) consitency of the memory that was transmitted. - Michael On 04/05/2013 04:43 PM, Roland Dreier wrote: > On Fri, Apr 5, 2013 at 1:17 PM, Michael R. Hines > wrote: >> I also removed the IBV_*_WRITE flags on the sender-side and activated >> cgroups with the "memory.memsw.limit_in_bytes" activated and the migration >> with RDMA also succeeded without any problems (both with *and* without GIFT >> also worked). > Not sure I'm interpreting this correctly. Are you saying that things > worked without actually setting the GIFT flag? In which case why are > we adding this flag? > > - R. > From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:52924) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UODbg-0006s3-Pd for qemu-devel@nongnu.org; Fri, 05 Apr 2013 16:51:35 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UODbb-0004T7-TZ for qemu-devel@nongnu.org; Fri, 05 Apr 2013 16:51:32 -0400 Received: from e9.ny.us.ibm.com ([32.97.182.139]:56069) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UODbb-0004Sv-PQ for qemu-devel@nongnu.org; Fri, 05 Apr 2013 16:51:27 -0400 Received: from /spool/local by e9.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 5 Apr 2013 16:51:27 -0400 Received: from d01relay04.pok.ibm.com (d01relay04.pok.ibm.com [9.56.227.236]) by d01dlp01.pok.ibm.com (Postfix) with ESMTP id ED08E38C801A for ; Fri, 5 Apr 2013 16:51:22 -0400 (EDT) Received: from d03av06.boulder.ibm.com (d03av06.boulder.ibm.com [9.17.195.245]) by d01relay04.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id r35KpM49301816 for ; Fri, 5 Apr 2013 16:51:22 -0400 Received: from d03av06.boulder.ibm.com (loopback [127.0.0.1]) by d03av06.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id r35Ks62A015370 for ; Fri, 5 Apr 2013 14:54:07 -0600 Message-ID: <515F3948.40205@linux.vnet.ibm.com> Date: Fri, 05 Apr 2013 16:51:20 -0400 From: "Michael R. Hines" MIME-Version: 1.0 References: <20130324155153.GA8597@redhat.com> <515F3160.4020007@linux.vnet.ibm.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCHv2] rdma: add a new IB_ACCESS_GIFT flag List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Roland Dreier Cc: qemu-devel@nongnu.org, "Michael S. Tsirkin" , "linux-rdma@vger.kernel.org" , Yishai Hadas , LKML , Hal Rosenstock , Jason Gunthorpe , Sean Hefty , Christoph Lameter Sorry, I was wrong. ignore the comments about cgroups. That's still broken. (i.e. trying to register RDMA memory while using a cgroup swap limit cause the process get killed). But the GIFT flag patch works (my understanding is that GIFT flag allows the adapter to transmit stale memory information, it does not have anything to do with cgroups specifically). Am I missing something? I was only testing the GIFT flag patch. Note: I only turned it on - I did not verify the (non) consitency of the memory that was transmitted. - Michael On 04/05/2013 04:43 PM, Roland Dreier wrote: > On Fri, Apr 5, 2013 at 1:17 PM, Michael R. Hines > wrote: >> I also removed the IBV_*_WRITE flags on the sender-side and activated >> cgroups with the "memory.memsw.limit_in_bytes" activated and the migration >> with RDMA also succeeded without any problems (both with *and* without GIFT >> also worked). > Not sure I'm interpreting this correctly. Are you saying that things > worked without actually setting the GIFT flag? In which case why are > we adding this flag? > > - R. >