From mboxrd@z Thu Jan  1 00:00:00 1970
From: Steve Wise <swise@opengridcomputing.com>
Subject: Re: [ofa-general] Re: Demand paging for memory regions
Date: Thu, 14 Feb 2008 10:23:23 -0600
Message-ID: <47B46AFB.9070009@opengridcomputing.com>
References: <adazlu5vlub.fsf@cisco.com>
	<8A71B368A89016469F72CD08050AD334026D5C23@maui.asicdesigners.com>
	<47B45994.7010805@opengridcomputing.com>
	<20080214155333.GA1029@sgi.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Cc: Rik van Riel <riel@redhat.com>, steiner@sgi.com,
	Andrea Arcangeli <andrea@qumranet.com>, a.p.zijlstra@chello.nl,
	izike@qumranet.com, Roland Dreier <rdreier@cisco.com>,
	linux-kernel@vger.kernel.org, avi@qumranet.com,
	kvm-devel@lists.sourceforge.net, linux-mm@kvack.org,
	daniel.blueman@quadrics.com, general@lists.openfabrics.org,
	Andrew Morton <akpm@linux-foundation.org>,
	Christoph Lameter <clameter@sgi.com>
To: Robin Holt <holt@sgi.com>
Return-path: <general-bounces@lists.openfabrics.org>
In-Reply-To: <20080214155333.GA1029@sgi.com>
List-Unsubscribe: <http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general>,
	<mailto:general-request@lists.openfabrics.org?subject=unsubscribe>
List-Archive: <http://lists.openfabrics.org/pipermail/general>
List-Post: <mailto:general@lists.openfabrics.org>
List-Help: <mailto:general-request@lists.openfabrics.org?subject=help>
List-Subscribe: <http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general>,
	<mailto:general-request@lists.openfabrics.org?subject=subscribe>
Sender: general-bounces@lists.openfabrics.org
Errors-To: general-bounces@lists.openfabrics.org
List-Id: kvm.vger.kernel.org

Robin Holt wrote:
> On Thu, Feb 14, 2008 at 09:09:08AM -0600, Steve Wise wrote:
>> Note that for T3, this involves suspending _all_ rdma connections that are 
>> in the same PD as the MR being remapped.  This is because the driver 
>> doesn't know who the application advertised the rkey/stag to.  So without 
> 
> Is there a reason the driver can not track these.
> 

Because advertising of a MR (ie telling the peer about your rkey/stag, 
offset and length) is application-specific and can be done out of band, 
or in band as simple SEND/RECV payload. Either way, the driver has no 
way of tracking this because the protocol used is application-specific.

>> Point being, it will stop probably all connections that an application is 
>> using (assuming the application uses a single PD).
> 
> It seems like the need to not stop all would be a compelling enough reason
> to modify the driver to track which processes have received the rkey/stag.
> 

Yes, _if_ the driver could track this.

And _if_ the rdma API and paradigm was such that the kernel/driver could 
keep track, then remote revokations of MR tags could be supported.

Stevo