From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jerome Glisse Subject: Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma Date: Tue, 29 Jan 2019 19:08:06 -0500 Message-ID: <20190130000805.GS3176@redhat.com> References: <20190129174728.6430-1-jglisse@redhat.com> <20190129174728.6430-4-jglisse@redhat.com> <20190129191120.GE3176@redhat.com> <20190129193250.GK10108@mellanox.com> <20190129195055.GH3176@redhat.com> <20190129202429.GL10108@mellanox.com> <20190129204359.GM3176@redhat.com> <20190129224016.GD4713@mellanox.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Content-Disposition: inline In-Reply-To: <20190129224016.GD4713@mellanox.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: Jason Gunthorpe Cc: Joerg Roedel , "Rafael J . Wysocki" , Greg Kroah-Hartman , Felix Kuehling , "linux-kernel@vger.kernel.org" , "dri-devel@lists.freedesktop.org" , Christoph Hellwig , "linux-mm@kvack.org" , "iommu@lists.linux-foundation.org" , "linux-pci@vger.kernel.org" , Bjorn Helgaas , Robin Murphy , Logan Gunthorpe , Christian Koenig , Marek Szyprowski List-Id: iommu@lists.linux-foundation.org T24gVHVlLCBKYW4gMjksIDIwMTkgYXQgMTE6MDI6MjVQTSArMDAwMCwgSmFzb24gR3VudGhvcnBl IHdyb3RlOgo+IE9uIFR1ZSwgSmFuIDI5LCAyMDE5IGF0IDAzOjQ0OjAwUE0gLTA1MDAsIEplcm9t ZSBHbGlzc2Ugd3JvdGU6Cj4gCj4gPiA+IEJ1dCB0aGlzIEFQSSBkb2Vzbid0IHNlZW0gdG8gb2Zm ZXIgYW55IGNvbnRyb2wgLSBJIHRob3VnaHQgdGhhdAo+ID4gPiBjb250cm9sIHdhcyBhbGwgY29t aW5nIGZyb20gdGhlIG1tL2htbSBub3RpZmllcnMgdHJpZ2dlcmluZyBwMnBfdW5tYXBzPwo+ID4g Cj4gPiBUaGUgY29udHJvbCBpcyB3aXRoaW4gdGhlIGRyaXZlciBpbXBsZW1lbnRhdGlvbiBvZiB0 aG9zZSBjYWxsYmFja3MuIAo+IAo+IFNlZW1zIGxpa2Ugd2hhdCB5b3UgbWVhbiBieSBjb250cm9s IGlzICd0aGUgZXhwb3J0ZXIgZ2V0cyB0byBjaG9vc2UKPiB0aGUgcGh5c2ljYWwgYWRkcmVzcyBh dCB0aGUgaW5zdGFudCBvZiBtYXAnIC0gd2hpY2ggc2VlbXMgcmVhc29uYWJsZQo+IGZvciBHUFUu Cj4gCj4gCj4gPiB3aWxsIG9ubHkgYWxsb3cgcDJwIG1hcCB0byBzdWNjZWVkIGZvciBvYmplY3Rz IHRoYXQgaGF2ZSBiZWVuIHRhZ2dlZCBieSB0aGUKPiA+IHVzZXJzcGFjZSBpbiBzb21lIHdheSBp ZSB0aGUgdXNlcnNwYWNlIGFwcGxpY2F0aW9uIGlzIGluIGNvbnRyb2wgb2Ygd2hhdAo+ID4gY2Fu IGJlIG1hcCB0byBwZWVyIGRldmljZS4KPiAKPiBJIHdvdWxkIGhhdmUgdGhvdWdodCB0aGlzIG1l YW5zIHRoZSBWTUEgZm9yIHRoZSBvYmplY3QgaXMgY3JlYXRlZAo+IHdpdGhvdXQgdGhlIG1hcC91 bm1hcCBvcHM/IE9yIGFyZSBHUFUgb2JqZWN0cyBhbmQgVk1BcyB1bnJlbGF0ZWQ/CgpHUFUgb2Jq ZWN0IGFuZCBWTUEgYXJlIHVucmVsYXRlZCBpbiBhbGwgb3BlbiBzb3VyY2UgR1BVIGRyaXZlciBp IGFtCnNvbWV3aGF0IGZhbWlsaWFyIHdpdGggKEFNRCwgSW50ZWwsIE5WaWRpYSkuIFlvdSBjYW4g Y3JlYXRlIGEgR1BVCm9iamVjdCBhbmQgbmV2ZXIgbWFwIGl0IChhbmQgdGh1cyBuZXZlciBoYXZl IGl0IGFzc29jaWF0ZWQgd2l0aCBhCnZtYSkgYW5kIGluIGZhY3QgdGhpcyBpcyB2ZXJ5IGNvbW1v bi4gRm9yIGdyYXBoaWMgeW91IHVzdWFseSBvbmx5CmhhdmUgaGFuZCBmdWxsIG9mIHRoZSBodW5k cmVkcyBvZiBHUFUgb2JqZWN0IHlvdXIgYXBwbGljYXRpb24gaGF2ZQptYXBwZWQuCgpUaGUgY29u dHJvbCBmb3IgcGVlciB0byBwZWVyIGNhbiBhbHNvIGJlIGEgbXV0YWJsZSBwcm9wZXJ0aWVzIG9m IHRoZQpvYmplY3QgaWUgdXNlcnNwYWNlIGRvIGlvY3RsIG9uIHRoZSBHUFUgZHJpdmVyIHdoaWNo IGNyZWF0ZSBhbiBvYmplY3Q7ClNvbWUgdGltZXMgYWZ0ZXIgdGhlIG9iamVjdCBpcyBjcmVhdGVk IHRoZSB1c2Vyc3BhY2UgZG8gb3RoZXJzIGlvY3RsCnRvIGFsbG93IHRvIGV4cG9ydCB0aGUgb2Jq ZWN0IHRvIGFub3RoZXIgc3BlY2lmaWMgZGV2aWNlIGFnYWluIHRoaXMKcmVzdWx0IGluIGlvY3Rs IHRvIHRoZSBkZXZpY2UgZHJpdmVyLCB0aG9zZSBpb2N0bCBzZXQgZmxhZ3MgYW5kCnVwZGF0ZSBH UFUgb2JqZWN0IGtlcm5lbCBzdHJ1Y3R1cmUgd2l0aCBhbGwgdGhlIGluZm8uCgpJbiB0aGUgbWVh bnRpbWUgeW91IGhhdmUgbm8gY29udHJvbCBvbiB3aGVuIG90aGVyIGRyaXZlciBtaWdodCBjYWxs CnRoZSB2bWEgcDJwIGNhbGwgYmFja3MuIFNvIHlvdSBtdXN0IGhhdmUgcmVnaXN0ZXIgdGhlIHZt YSB3aXRoCnZtX29wZXJhdGlvbnMgdGhhdCBpbmNsdWRlIHRoZSBwMnBfbWFwIGFuZCBwMnBfdW5t YXAuIFRob3NlIGRyaXZlcgpmdW5jdGlvbiB3aWxsIGNoZWNrIHRoZSBvYmplY3Qga2VybmVsIHN0 cnVjdHVyZSBlYWNoIHRpbWUgdGhleSBnZXQKY2FsbCBhbmQgYWN0IGFjY29yZGluZ2x5LgoKCgo+ ID4gRm9yIG1vdmluZyB0aGluZ3MgYXJvdW5kIGFmdGVyIGEgc3VjY2Vzc2Z1bCBwMnBfbWFwIHll cyB0aGUgZXhwb3J0aW5nCj4gPiBkZXZpY2UgaGF2ZSB0byBjYWxsIGZvciBpbnN0YW5jZSB6YXBf dm1hX3B0ZXMoKSBvciBzb21ldGhpbmcKPiA+IHNpbWlsYXIuCj4gCj4gT2theSwgZ3JlYXQsIFJE TUEgbmVlZHMgdGhpcyBmbG93IGZvciBob3RwbHVnIC0gd2UgemFwIHRoZSBWTUEncyB3aGVuCj4g dW5wbHVnZ2luZyB0aGUgUENJIGRldmljZSBhbmQgd2UgY2FuIGRlbGF5IHRoZSBQQ0kgdW5wbHVn IGNvbXBsZXRpb24KPiB1bnRpbCBhbGwgdGhlIHAycF91bm1hcHMgYXJlIGNhbGxlZC4uLgo+IAo+ IEJ1dCBpbiB0aGlzIGNhc2UgYSBmdXR1cmUgcDJwX21hcCB3aWxsIGhhdmUgdG8gZmFpbCBhcyB0 aGUgQkFSIG5vCj4gbG9uZ2VyIGV4aXN0cy4gSG93IHRvIGhhbmRsZSB0aGlzPwoKU28gdGhlIGNv bW1lbnQgYWJvdmUgdGhlIGNhbGxiYWNrIChpIHNob3VsZCB3cml0ZSBtb3JlIHRob3JvdWdoIGd1 aWRlbGluZQphbmQgZG9jdW1lbnRhdGlvbikgc3RhdGUgdGhhdCBleHBvcnQgc2hvdWxkLyhtdXN0 PykgYmUgcHJlZGljdGFibGUgaWUKaWYgYW4gaW1wb3J0ZXIgZGV2aWNlIGNhbGxzIHAycF9tYXAo KSBvbmNlIG9uIGEgdm1hIGFuZCBpdCBkb2VzIHN1Y2NlZWQKdGhlbiBpZiB0aGUgc2FtZSBkZXZp Y2UgY2FsbHMgYWdhaW4gcDJwX21hcCgpIG9uIHRoZSBzYW1lIHZtYSBhbmQgaWYgdGhlCnZtYSBp cyBzdGlsbCB2YWxpZCAoaWUgbm8gdW5tYXAgb3IgZG9lcyBub3QgY29ycmVzcG9uZCB0byBhIGRp ZmZlcmVudApvYmplY3QgLi4uKSB0aGVuIHRoZSBwMnBfbWFwKCkgc2hvdWxkLyhtdXN0Pykgc3Vj Y2VlZC4KClRoZSBpZGVhIGlzIHRoYXQgdGhlIGltcG9ydGVyIHdvdWxkIGRvIGEgZmlyc3QgY2Fs bCB0byBwMnBfbWFwKCkgd2hlbiBpdApzZXR1cCBpdHMgb3duIG9iamVjdCwgcmVwb3J0IGZhaWx1 cmUgdG8gdXNlcnNwYWNlIGlmIHRoYXQgZmFpbHMuIElmIGl0CmRvZXMgc3VjY2VlZCB0aGVuIHdl IHNob3VsZCBuZXZlciBoYXZlIGFuIGlzc3VlIG5leHQgdGltZSB3ZSBjYWxsIHAycF9tYXAoKQoo YWZ0ZXIgbWFwcGluZyBiZWluZyBpbnZhbGlkYXRlZCBieSBtbXUgbm90aWZpZXIgZm9yIGluc3Rh bmNlKS4gU28gaXQgd2lsbApzdWNjZWVkIGp1c3QgbGlrZSB0aGUgZmlyc3QgY2FsbCAoYWdhaW4g YXNzdW1pbmcgdGhlIHZtYSBpcyBzdGlsbCB2YWxpZCkuCgpJZGVhIGlzIHRoYXQgd2UgY2FuIG9u bHkgYXNrIGV4cG9ydGVyIHRvIGJlIHByZWRpY3RhYmxlIGFuZCBzdGlsbCBhbGxvdwp0aGVtIHRv IGZhaWwgaWYgdGhpbmdzIGFyZSByZWFsbHkgZ29pbmcgYmFkLgoKCj4gPiA+IEkgd291bGQgdGhp bmsgdGhhdCB0aGUgaW1wb3J0aW5nIGRyaXZlciBjYW4gYXNzdW1lIHRoZSBCQVIgcGFnZSBpcwo+ ID4gPiBrZXB0IGFsaXZlIHVudGlsIGl0IGNhbGxzIHVubWFwIChwcmVzdW1hYmx5IHRyaWdnZXJl ZCBieSBub3RpZmllcnMpPwo+ID4gPiAKPiA+ID4gaWUgdGhlIGV4cG9ydGluZyBkcml2ZXIgc2Vl cyB0aGUgQkFSIHBhZ2UgYXMgcGlubmVkIHVudGlsIHVubWFwLgo+ID4gCj4gPiBUaGUgaW50ZW50 aW9uIHdpdGggdGhpcyBwYXRjaHNldCBpcyB0aGF0IGl0IGlzIG5vdCBwaW4gaWUgdGhlIGltcG9y dGVyCj4gPiBkZXZpY2UgX211c3RfIGFiaWRlIGJ5IGFsbCBtbXUgbm90aWZpZXIgaW52YWxpZGF0 aW9ucyBhbmQgdGhleSBjYW4KPiA+IGhhcHBlbiBhdCBhbnl0aW1lLiBUaGUgaW1wb3J0aW5nIGRl dmljZSBjYW4gaG93ZXZlciByZS1wMnBfbWFwIHRoZQo+ID4gc2FtZSByYW5nZSBhZnRlciBhbiBp bnZhbGlkYXRpb24uCj4gPgo+ID4gSSB3b3VsZCBsaWtlIHRvIHJlc3RyaWN0IHRoaXMgdG8gaW1w b3J0ZXIgdGhhdCBjYW4gaW52YWxpZGF0ZSBmb3IKPiA+IG5vdyBiZWNhdXNlIGkgYmVsaWV2ZSBh bGwgdGhlIGZpcnN0IGRldmljZSB0byB1c2UgY2FuIHN1cHBvcnQgdGhlCj4gPiBpbnZhbGlkYXRp b24uCj4gCj4gVGhpcyBzZWVtcyByZWFzb25hYmxlIChhbmQgc29ydCBvZiBzYXlzIGltcG9ydGVy cyBub3QgZ2V0dGluZyB0aGlzCj4gZnJvbSBITU0gbmVlZCBjYXJlZnVsIGNoZWNraW5nKSwgd2Fz IHRoaXMgaW4gdGhlIGNvbW1lbnQgYWJvdmUgdGhlCj4gb3BzPwoKSSB0aGluayBpIHB1dCBpdCBp biB0aGUgY29tbWVudCBhYm92ZSB0aGUgb3BzIGJ1dCBpbiBhbnkgY2FzZXMgaSBzaG91bGQKd3Jp dGUgc29tZXRoaW5nIGluIGRvY3VtZW50YXRpb24gd2l0aCBleGFtcGxlIGFuZCB0aG9yb3VnaCBn dWlkZWxpbmUuCk5vdGUgdGhhdCB0aGVyZSB3b24ndCBiZSBhbnkgbW11IG5vdGlmaWVyIHRvIG1t YXAgb2YgYSBkZXZpY2UgZmlsZQp1bmxlc3MgdGhlIGRldmljZSBkcml2ZXIgY2FsbHMgZm9yIGl0 IG9yIHRoZXJlIGlzIGEgc3lzY2FsbCBsaWtlIG11bm1hcApvciBtcmVtYXAgb3IgbXByb3RlY3Qg d2VsbCBhbnkgc3lzY2FsbCB0aGF0IHdvcmsgb24gdm1hLgoKU28gYXNzdW1pbmcgdGhlIGFwcGxp Y2F0aW9uIGlzIG5vIGRvaW5nIHNvbWV0aGluZyBzdHVwaWQsIG5vciB0aGUKZHJpdmVyLiBUaGVu IHRoZSByZXN1bHQgb2YgcDJwX21hcCBjYW4gc3RheSB2YWxpZCB1bnRpbCB0aGUgaW1wb3J0ZXIK aXMgZG9uZSBhbmQgY2FsbCBwMnBfdW5tYXAgb2YgaXRzIG93biBmcmVlIHdpbGwuIFRoaXMgaXMg d2hhdCBpIGRvCmV4cGVjdCBmb3IgdGhpcy4gQnV0IGZvciBHUFUgaSB3b3VsZCBzdGlsbCBsaWtl IHRvIGFsbG93IEdQVSBkcml2ZXIKdG8gZXZpY3QgKGFuZCB0aHVzIGludmFsaWRhdGUgaW1wb3J0 ZXIgbWFwcGluZykgdG8gbWFpbiBtZW1vcnkgb3IKZGVmcmFnbWVudCB0aGVpciBCQVIgYWRkcmVz cyBzcGFjZSBpZiB0aGUgR1BVIGRyaXZlciBmZWVscyBhIHByZXNzaW5nCm5lZWQgdG8gZG8gc28u CgpJZiB3ZSBldmVyIHdhbnQgdG8gc3VwcG9ydCBmdWxsIHBpbiB0aGVuIHdlIG1pZ2h0IGhhdmUg dG8gYWRkIGEKZmxhZyBzbyB0aGF0IEdQVSBkcml2ZXIgY2FuIHJlZnVzZSBhbiBpbXBvcnRlciB0 aGF0IHdhbnRzIHRoaW5ncwpwaW4gZm9yZXZlci4KCkNoZWVycywKSsOpcsO0bWUKX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KZHJpLWRldmVsIG1haWxpbmcg bGlzdApkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlzdHMuZnJlZWRl c2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJpLWRldmVsCg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS,USER_AGENT_MUTT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D64DEC169C4 for ; Wed, 30 Jan 2019 00:08:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 9D1E62082C for ; Wed, 30 Jan 2019 00:08:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727564AbfA3AIM (ORCPT ); Tue, 29 Jan 2019 19:08:12 -0500 Received: from mx1.redhat.com ([209.132.183.28]:51540 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726951AbfA3AIL (ORCPT ); Tue, 29 Jan 2019 19:08:11 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 98862BDD0; Wed, 30 Jan 2019 00:08:10 +0000 (UTC) Received: from redhat.com (ovpn-122-2.rdu2.redhat.com [10.10.122.2]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3216E53; Wed, 30 Jan 2019 00:08:08 +0000 (UTC) Date: Tue, 29 Jan 2019 19:08:06 -0500 From: Jerome Glisse To: Jason Gunthorpe Cc: Logan Gunthorpe , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , Greg Kroah-Hartman , "Rafael J . Wysocki" , Bjorn Helgaas , Christian Koenig , Felix Kuehling , "linux-pci@vger.kernel.org" , "dri-devel@lists.freedesktop.org" , Christoph Hellwig , Marek Szyprowski , Robin Murphy , Joerg Roedel , "iommu@lists.linux-foundation.org" Subject: Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma Message-ID: <20190130000805.GS3176@redhat.com> References: <20190129174728.6430-1-jglisse@redhat.com> <20190129174728.6430-4-jglisse@redhat.com> <20190129191120.GE3176@redhat.com> <20190129193250.GK10108@mellanox.com> <20190129195055.GH3176@redhat.com> <20190129202429.GL10108@mellanox.com> <20190129204359.GM3176@redhat.com> <20190129224016.GD4713@mellanox.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20190129224016.GD4713@mellanox.com> User-Agent: Mutt/1.10.1 (2018-07-13) X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Wed, 30 Jan 2019 00:08:11 +0000 (UTC) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org On Tue, Jan 29, 2019 at 11:02:25PM +0000, Jason Gunthorpe wrote: > On Tue, Jan 29, 2019 at 03:44:00PM -0500, Jerome Glisse wrote: > > > > But this API doesn't seem to offer any control - I thought that > > > control was all coming from the mm/hmm notifiers triggering p2p_unmaps? > > > > The control is within the driver implementation of those callbacks. > > Seems like what you mean by control is 'the exporter gets to choose > the physical address at the instant of map' - which seems reasonable > for GPU. > > > > will only allow p2p map to succeed for objects that have been tagged by the > > userspace in some way ie the userspace application is in control of what > > can be map to peer device. > > I would have thought this means the VMA for the object is created > without the map/unmap ops? Or are GPU objects and VMAs unrelated? GPU object and VMA are unrelated in all open source GPU driver i am somewhat familiar with (AMD, Intel, NVidia). You can create a GPU object and never map it (and thus never have it associated with a vma) and in fact this is very common. For graphic you usualy only have hand full of the hundreds of GPU object your application have mapped. The control for peer to peer can also be a mutable properties of the object ie userspace do ioctl on the GPU driver which create an object; Some times after the object is created the userspace do others ioctl to allow to export the object to another specific device again this result in ioctl to the device driver, those ioctl set flags and update GPU object kernel structure with all the info. In the meantime you have no control on when other driver might call the vma p2p call backs. So you must have register the vma with vm_operations that include the p2p_map and p2p_unmap. Those driver function will check the object kernel structure each time they get call and act accordingly. > > For moving things around after a successful p2p_map yes the exporting > > device have to call for instance zap_vma_ptes() or something > > similar. > > Okay, great, RDMA needs this flow for hotplug - we zap the VMA's when > unplugging the PCI device and we can delay the PCI unplug completion > until all the p2p_unmaps are called... > > But in this case a future p2p_map will have to fail as the BAR no > longer exists. How to handle this? So the comment above the callback (i should write more thorough guideline and documentation) state that export should/(must?) be predictable ie if an importer device calls p2p_map() once on a vma and it does succeed then if the same device calls again p2p_map() on the same vma and if the vma is still valid (ie no unmap or does not correspond to a different object ...) then the p2p_map() should/(must?) succeed. The idea is that the importer would do a first call to p2p_map() when it setup its own object, report failure to userspace if that fails. If it does succeed then we should never have an issue next time we call p2p_map() (after mapping being invalidated by mmu notifier for instance). So it will succeed just like the first call (again assuming the vma is still valid). Idea is that we can only ask exporter to be predictable and still allow them to fail if things are really going bad. > > > I would think that the importing driver can assume the BAR page is > > > kept alive until it calls unmap (presumably triggered by notifiers)? > > > > > > ie the exporting driver sees the BAR page as pinned until unmap. > > > > The intention with this patchset is that it is not pin ie the importer > > device _must_ abide by all mmu notifier invalidations and they can > > happen at anytime. The importing device can however re-p2p_map the > > same range after an invalidation. > > > > I would like to restrict this to importer that can invalidate for > > now because i believe all the first device to use can support the > > invalidation. > > This seems reasonable (and sort of says importers not getting this > from HMM need careful checking), was this in the comment above the > ops? I think i put it in the comment above the ops but in any cases i should write something in documentation with example and thorough guideline. Note that there won't be any mmu notifier to mmap of a device file unless the device driver calls for it or there is a syscall like munmap or mremap or mprotect well any syscall that work on vma. So assuming the application is no doing something stupid, nor the driver. Then the result of p2p_map can stay valid until the importer is done and call p2p_unmap of its own free will. This is what i do expect for this. But for GPU i would still like to allow GPU driver to evict (and thus invalidate importer mapping) to main memory or defragment their BAR address space if the GPU driver feels a pressing need to do so. If we ever want to support full pin then we might have to add a flag so that GPU driver can refuse an importer that wants things pin forever. Cheers, Jérôme