From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jerome Glisse Subject: Re: [RFC PATCH 00/13] SVM (share virtual memory) with HMM in nouveau Date: Tue, 13 Mar 2018 11:56:46 -0400 Message-ID: <20180313155645.GD3828@redhat.com> References: <20180310032141.6096-1-jglisse@redhat.com> <20180312173009.GN8589@phenom.ffwll.local> <20180312175057.GC4214@redhat.com> <39139ff7-76ad-960c-53f6-46b57525b733@nvidia.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Content-Disposition: inline In-Reply-To: <39139ff7-76ad-960c-53f6-46b57525b733-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: nouveau-bounces-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org Sender: "Nouveau" To: John Hubbard Cc: Ralph Campbell , "Bridgman, John" , nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org, Felix Kuehling , dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org, linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org, Evgeny Baskakov , christian.koenig-5C7GfCeVMHo@public.gmane.org List-Id: nouveau.vger.kernel.org T24gTW9uLCBNYXIgMTIsIDIwMTggYXQgMTE6MTQ6NDdQTSAtMDcwMCwgSm9obiBIdWJiYXJkIHdy b3RlOgo+IE9uIDAzLzEyLzIwMTggMTA6NTAgQU0sIEplcm9tZSBHbGlzc2Ugd3JvdGU6CgpbLi4u XQoKPiBZZXMsIG9uIE5WSURJQSBHUFVzLCB0aGUgSG9zdC9GSUZPIHVuaXQgaXMgbGltaXRlZCB0 byA0MC1iaXQgYWRkcmVzc2VzLCBzbwo+IHRoaW5ncyBzdWNoIGFzIHRoZSBmb2xsb3dpbmcgbmVl ZCB0byBiZSBiZWxvdyAoMSA8PCA0MCksIGFuZCBhbHNvIGFjY2Vzc2libGUgCj4gdG8gYm90aCBD UFUgKHVzZXIgc3BhY2UpIGFuZCBHUFUgaGFyZHdhcmUuIAo+ICAgICAtLSBjb21tYW5kIGJ1ZmZl cnMgKENQVSB1c2VyIHNwYWNlIGRyaXZlciBmaWxscyB0aGVtLCBHUFUgY29uc3VtZXMgdGhlbSks IAo+ICAgICAtLSBzZW1hcGhvcmVzIChoZXJlLCBhIEdQVS1jZW50cmljIHRlcm0sIHJhdGhlciB0 aGFuIE9TLXR5cGU6IHRoZXNlIGFyZQo+ICAgICAgICBtZW1vcnkgbG9jYXRpb25zIHRoYXQsIGZv ciBleGFtcGxlLCB0aGUgR1BVIGhhcmR3YXJlIG1pZ2h0IHdyaXRlIHRvLCBpbgo+ICAgICAgICBv cmRlciB0byBpbmRpY2F0ZSB3b3JrIGNvbXBsZXRpb247IHRoZXJlIGFyZSBvdGhlciB1c2VzIGFz IHdlbGwpLCAKPiAgICAgLS0gYSBmZXcgb3RoZXIgdGhpbmdzIG1vc3QgbGlrZWx5ICh0aGlzIGlz IG5vdCBhIGNvbXBsZXRlIGxpc3QpLgo+IAo+IFNvIHdoYXQgSSdkIHRlbnRhdGl2ZWx5IGV4cGVj dCB0aGF0IHRvIHRyYW5zbGF0ZSBpbnRvIGluIHRoZSBkcml2ZXIgc3RhY2sgaXMsIAo+IGFwcHJv eGltYXRlbHk6Cj4gCj4gICAgIC0tIFVzZXIgc3BhY2UgZHJpdmVyIGNvZGUgbW1hcCdzIGFuIGFy ZWEgYmVsb3cgKDEgPDwgNDApLiBJdCdzIGhhcmQgdG8gYXZvaWQgdGhpcywKPiAgICAgICAgZ2l2 ZW4gdGhhdCB1c2VyIHNwYWNlIG5lZWRzIGFjY2VzcyB0byB0aGUgYXJlYSAoZm9yIGZpbGxpbmcg b3V0IGNvbW1hbmQKPiAgICAgICAgYnVmZmVycyBhbmQgbW9uaXRvcmluZyBzZW1hcGhvcmVzLCB0 aGF0IHNvcnQgb2YgdGhpbmcpLiBUaGVuIHN1YmFsbG9jYXRlCj4gICAgICAgIGZyb20gdGhlcmUg dXNpbmcgbW1hcCdzIE1BUF9GSVhFRCBvciAoZnV0dXJlLWlzaCkgTUFQX0ZJWEVEX1NBRkUgZmxh Z3MuCj4gCj4gICAgICAgIC4uLmdsYW5jaW5nIGF0IHRoZSBvdGhlciBmb3JrIG9mIHRoaXMgdGhy ZWFkLCBJIHRoaW5rIHRoYXQgaXMgZXhhY3RseSB3aGF0Cj4gICAgICAgIEZlbGl4IGlzIHNheWlu ZywgdG9vLiBTbyB0aGF0J3MgZ29vZC4KPiAKPiAgICAgLS0gVGhlIHVzZXIgc3BhY2UgcHJvZ3Jh bSBzaXRzIGFib3ZlIHRoZSB1c2VyIHNwYWNlIGRyaXZlciwgYW5kIGFsdGhvdWdoIHRoZQo+ICAg ICAgICBwcm9ncmFtIGNvdWxkLCBpbiB0aGVvcnksIGludGVyZmVyZSB3aXRoIHRoaXMgbW1hcCdk IGFyZWEsIHRoYXQgd291bGQgYmUKPiAgICAgICAgd3JvbmcgaW4gdGhlIHNhbWUgd2F5IHRoYXQg bXVja2luZyBhcm91bmQgd2l0aCBtYWxsb2MnZCBhcmVhcyAob3V0c2lkZSBvZgo+ICAgICAgICBt YWxsb2MoKSBpdHNlbGYpIGlzIHdyb25nLiBTbyBJIGRvbid0IHNlZSBhbnkgcGFydGljdWxhciBu ZWVkIHRvIGRvIG11Y2gKPiAgICAgICAgbW9yZSB0aGFuIHRoZSBhYm92ZS4KCkkgYW0gd29ycmll ZCB0aGF0IHJvZ3VlIHByb2dyYW0gKGkgYW0gbm90IHdvcnJpZWQgYWJvdXQgYnVnZ3kgcHJvZ3Jh bQppZiBwZW9wbGUgc2hvb3QgdGhlbXNlbGYgaW4gdGhlIGZvb3QgdGhleSBzaG91bGQgZmVlbCB0 aGUgcGFpbikgY291bGQKdXNlIHRoYXQgdG8gYWJ1c2UgY2hhbm5lbCB0byBkbyBzb21ldGhpbmcg aGFybWZ1bC4gSSBhbSBub3QgZmFtaWxpYXIKZW5vdWdoIHdpdGggdGhlIGhhcmR3YXJlIHRvIGNv bXBsZXRlbHkgcnVsZSBvdXQgc3VjaCBzY2VuYXJpby4KCkkgZG8gYmVsaWV2ZSBoYXJkd2FyZSB3 aXRoIHVzZXJzcGFjZSBxdWV1ZSBzdXBwb3J0IGhhdmUgdGhlIG5lY2Vzc2FyeQpib3VuZGFyeSB0 byBrZWVwIHRoaW5nIHNlY3VyZSBhcyBpIHdvdWxkIGFzc3VtZSBmb3IgdGhvc2UgdGhlIGhhcmR3 YXJlCmVuZ2luZWVycyBoYWQgdG8gdGFrZSBzZWN1cml0eSBpbnRvIGNvbnNpZGVyYXRpb24uCgpO b3RlIHRoYXQgaW4gbXkgcGF0Y2hzZXQgdGhlIGNvZGUgdGhhdCBtb25pdG9yIHRoZSBzcGVjaWFs IHZtYSBpcyBzbWFsbApzb21ldGhpbmcgbGlrZSAyMGxpbmVzIG9mIGNvZGUgdGhhdCBvbmx5IGdl dCBjYWxsIGlmIHNvbWV0aGluZyBoYXBwZW4KdG8gdGhlIHJlc2VydmVkIGFyZWEuIFNvIGkgYmVs aWV2ZSBpdCBpcyB3b3J0aCBoYXZpbmcgc3VjaCB0aGluZywgY29zdAppcyBsb3cgZm9yIGxpdHRs ZSBleHRyYSBwZWFjZSBvZiBtaW5kIDopCgpDaGVlcnMsCkrDqXLDtG1lCl9fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCk5vdXZlYXUgbWFpbGluZyBsaXN0Ck5v dXZlYXVAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3Jn L21haWxtYW4vbGlzdGluZm8vbm91dmVhdQo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk0-f199.google.com (mail-qk0-f199.google.com [209.85.220.199]) by kanga.kvack.org (Postfix) with ESMTP id 397CC6B002E for ; Tue, 13 Mar 2018 11:56:50 -0400 (EDT) Received: by mail-qk0-f199.google.com with SMTP id d128so49332qkb.6 for ; Tue, 13 Mar 2018 08:56:50 -0700 (PDT) Received: from mx1.redhat.com (mx3-rdu2.redhat.com. [66.187.233.73]) by mx.google.com with ESMTPS id q45si487384qtq.122.2018.03.13.08.56.49 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 13 Mar 2018 08:56:49 -0700 (PDT) Date: Tue, 13 Mar 2018 11:56:46 -0400 From: Jerome Glisse Subject: Re: [RFC PATCH 00/13] SVM (share virtual memory) with HMM in nouveau Message-ID: <20180313155645.GD3828@redhat.com> References: <20180310032141.6096-1-jglisse@redhat.com> <20180312173009.GN8589@phenom.ffwll.local> <20180312175057.GC4214@redhat.com> <39139ff7-76ad-960c-53f6-46b57525b733@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <39139ff7-76ad-960c-53f6-46b57525b733@nvidia.com> Sender: owner-linux-mm@kvack.org List-ID: To: John Hubbard Cc: christian.koenig@amd.com, dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, Evgeny Baskakov , linux-mm@kvack.org, Ralph Campbell , Felix Kuehling , "Bridgman, John" On Mon, Mar 12, 2018 at 11:14:47PM -0700, John Hubbard wrote: > On 03/12/2018 10:50 AM, Jerome Glisse wrote: [...] > Yes, on NVIDIA GPUs, the Host/FIFO unit is limited to 40-bit addresses, so > things such as the following need to be below (1 << 40), and also accessible > to both CPU (user space) and GPU hardware. > -- command buffers (CPU user space driver fills them, GPU consumes them), > -- semaphores (here, a GPU-centric term, rather than OS-type: these are > memory locations that, for example, the GPU hardware might write to, in > order to indicate work completion; there are other uses as well), > -- a few other things most likely (this is not a complete list). > > So what I'd tentatively expect that to translate into in the driver stack is, > approximately: > > -- User space driver code mmap's an area below (1 << 40). It's hard to avoid this, > given that user space needs access to the area (for filling out command > buffers and monitoring semaphores, that sort of thing). Then suballocate > from there using mmap's MAP_FIXED or (future-ish) MAP_FIXED_SAFE flags. > > ...glancing at the other fork of this thread, I think that is exactly what > Felix is saying, too. So that's good. > > -- The user space program sits above the user space driver, and although the > program could, in theory, interfere with this mmap'd area, that would be > wrong in the same way that mucking around with malloc'd areas (outside of > malloc() itself) is wrong. So I don't see any particular need to do much > more than the above. I am worried that rogue program (i am not worried about buggy program if people shoot themself in the foot they should feel the pain) could use that to abuse channel to do something harmful. I am not familiar enough with the hardware to completely rule out such scenario. I do believe hardware with userspace queue support have the necessary boundary to keep thing secure as i would assume for those the hardware engineers had to take security into consideration. Note that in my patchset the code that monitor the special vma is small something like 20lines of code that only get call if something happen to the reserved area. So i believe it is worth having such thing, cost is low for little extra peace of mind :) Cheers, Jerome