From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jerome Glisse Subject: Re: [RFC PATCH 00/13] SVM (share virtual memory) with HMM in nouveau Date: Tue, 13 Mar 2018 10:28:00 -0400 Message-ID: <20180313142759.GB3828@redhat.com> References: <20180310032141.6096-1-jglisse@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Content-Disposition: inline In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: nouveau-bounces-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org Sender: "Nouveau" To: Felix Kuehling Cc: Ralph Campbell , "Bridgman, John" , nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org, dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org, linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org, Evgeny Baskakov , christian.koenig-5C7GfCeVMHo@public.gmane.org List-Id: nouveau.vger.kernel.org T24gTW9uLCBNYXIgMTIsIDIwMTggYXQgMDI6Mjg6NDJQTSAtMDQwMCwgRmVsaXggS3VlaGxpbmcg d3JvdGU6Cj4gT24gMjAxOC0wMy0xMCAxMDowMSBBTSwgQ2hyaXN0aWFuIEvDtm5pZyB3cm90ZToK PiA+PiBUbyBhY2NvbW9kYXRlIHRob3NlIHdlIG5lZWQgdG8KPiA+PiBjcmVhdGUgYSAiaG9sZSIg aW5zaWRlIHRoZSBwcm9jZXNzIGFkZHJlc3Mgc3BhY2UuIFRoaXMgcGF0Y2hzZXQgaGF2ZQo+ID4+ IGEgaGFjayBmb3IgdGhhdCAocGF0Y2ggMTMgSEFDSyBGT1IgSE1NIEFSRUEpLCBpdCByZXNlcnZl cyBhIHJhbmdlIG9mCj4gPj4gZGV2aWNlIGZpbGUgb2Zmc2V0IHNvIHRoYXQgcHJvY2VzcyBjYW4g bW1hcCB0aGlzIHJhbmdlIHdpdGggUFJPVF9OT05FCj4gPj4gdG8gY3JlYXRlIGEgaG9sZSAocHJv Y2VzcyBtdXN0IG1ha2Ugc3VyZSB0aGUgaG9sZSBpcyBiZWxvdyAxIDw8IDQwKS4KPiA+PiBJIGZl ZWwgdW4tZWFzeSBvZiBkb2luZyBpdCB0aGlzIHdheSBidXQgbWF5YmUgaXQgaXMgb2sgd2l0aCBv dGhlcgo+ID4+IGZvbGtzLgo+ID4KPiA+IFdlbGwgd2UgaGF2ZSBlc3NlbnRpYWxseSB0aGUgc2Ft ZSBwcm9ibGVtIHdpdGggcHJlIGdmeDkgQU1EIGhhcmR3YXJlLgo+ID4gRmVsaXggbWlnaHQgaGF2 ZSBzb21lIGFkdmlzZSBob3cgaXQgd2FzIHNvbHZlZCBmb3IgSFNBLiAKPiAKPiBGb3IgcHJlLWdm eDkgaGFyZHdhcmUgd2UgcmVzZXJ2ZSBhZGRyZXNzIHNwYWNlIGluIHVzZXIgbW9kZSB1c2luZyBh IGJpZwo+IG1tYXAgUFJPVF9OT05FIGNhbGwgYXQgYXBwbGljYXRpb24gc3RhcnQuIFRoZW4gd2Ug bWFuYWdlIHRoZSBhZGRyZXNzCj4gc3BhY2UgaW4gdXNlciBtb2RlIGFuZCB1c2UgTUFQX0ZJWEVE IHRvIG1hcCBidWZmZXJzIGF0IHNwZWNpZmljCj4gYWRkcmVzc2VzIHdpdGhpbiB0aGUgcmVzZXJ2 ZWQgcmFuZ2UuCj4gCj4gVGhlIGJpZyBhZGRyZXNzIHNwYWNlIHJlc2VydmF0aW9uIGNhdXNlcyBp c3N1ZXMgZm9yIHNvbWUgZGVidWdnaW5nIHRvb2xzCj4gKGNsYW5nLXNhbml0aXplciB3YXMgbWVu dGlvbmVkIHRvIG1lKSwgc28gd2l0aCBnZng5IHdlJ3JlIGdvaW5nIHRvIGdldAo+IHJpZCBvZiB0 aGlzIGFkZHJlc3Mgc3BhY2UgcmVzZXJ2YXRpb24uCgpXaGF0IGRvIHlvdSBuZWVkIHRob3NlIG1h cHBpbmcgZm9yID8gV2hhdCBraW5kIG9mIG9iamVjdCAocG00IHBhY2tldApjb21tYW5kIGJ1ZmZl ciwgR1BVIHNlbWFwaG9yZSB8IGZlbmNlLCAuLi4pID8gS2VybmVsIHByaXZhdGUgb2JqZWN0ID8K T24gbnYgd2UgbmVlZCBpdCBmb3IgdGhlIG1haW4gY29tbWFuZCBidWZmZXIgcmluZyB3aGljaCB3 ZSBkbyBub3Qgd2FudAp0byBleHBvc2UgdG8gYXBwbGljYXRpb24uCgpUaHVzIGZvciBudiBncHUg d2UgbmVlZCBrZXJuZWwgdG8gbW9uaXRvciB0aGlzIFBST1RfTk9ORSByZWdpb24gdG8gbWFrZQpz dXJlIHRoYXQgaSBuZXZlciBnb3QgdW5tYXBwZWQsIHJlc2l6ZSwgbW92ZSAuLi4gdGhpcyBpcyBt ZSBmZWFyaW5nIGEKcm9ndWUgdXNlcnNwYWNlIHRoYXQgbXVubWFwIGFuZCB0cnkgdG8gYWJ1c2Ug c29tZSBidWcgaW4gU1ZNL0dQVSBkcml2ZXIKdG8gYWJ1c2Ugb2JqZWN0IG1hcCBiZWhpbmQgdGhv c2UgZml4IG1hcHBpbmcuCgpDaGVlcnMsCkrDqXLDtG1lCl9fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fCk5vdXZlYXUgbWFpbGluZyBsaXN0Ck5vdXZlYXVAbGlz dHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4v bGlzdGluZm8vbm91dmVhdQo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk0-f200.google.com (mail-qk0-f200.google.com [209.85.220.200]) by kanga.kvack.org (Postfix) with ESMTP id 153176B026C for ; Tue, 13 Mar 2018 10:28:05 -0400 (EDT) Received: by mail-qk0-f200.google.com with SMTP id 13so10567445qkg.23 for ; Tue, 13 Mar 2018 07:28:05 -0700 (PDT) Received: from mx1.redhat.com (mx3-rdu2.redhat.com. [66.187.233.73]) by mx.google.com with ESMTPS id w23si334584qtj.181.2018.03.13.07.28.03 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 13 Mar 2018 07:28:03 -0700 (PDT) Date: Tue, 13 Mar 2018 10:28:00 -0400 From: Jerome Glisse Subject: Re: [RFC PATCH 00/13] SVM (share virtual memory) with HMM in nouveau Message-ID: <20180313142759.GB3828@redhat.com> References: <20180310032141.6096-1-jglisse@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Sender: owner-linux-mm@kvack.org List-ID: To: Felix Kuehling Cc: christian.koenig@amd.com, dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, Evgeny Baskakov , linux-mm@kvack.org, Ralph Campbell , John Hubbard , "Bridgman, John" On Mon, Mar 12, 2018 at 02:28:42PM -0400, Felix Kuehling wrote: > On 2018-03-10 10:01 AM, Christian Konig wrote: > >> To accomodate those we need to > >> create a "hole" inside the process address space. This patchset have > >> a hack for that (patch 13 HACK FOR HMM AREA), it reserves a range of > >> device file offset so that process can mmap this range with PROT_NONE > >> to create a hole (process must make sure the hole is below 1 << 40). > >> I feel un-easy of doing it this way but maybe it is ok with other > >> folks. > > > > Well we have essentially the same problem with pre gfx9 AMD hardware. > > Felix might have some advise how it was solved for HSA. > > For pre-gfx9 hardware we reserve address space in user mode using a big > mmap PROT_NONE call at application start. Then we manage the address > space in user mode and use MAP_FIXED to map buffers at specific > addresses within the reserved range. > > The big address space reservation causes issues for some debugging tools > (clang-sanitizer was mentioned to me), so with gfx9 we're going to get > rid of this address space reservation. What do you need those mapping for ? What kind of object (pm4 packet command buffer, GPU semaphore | fence, ...) ? Kernel private object ? On nv we need it for the main command buffer ring which we do not want to expose to application. Thus for nv gpu we need kernel to monitor this PROT_NONE region to make sure that i never got unmapped, resize, move ... this is me fearing a rogue userspace that munmap and try to abuse some bug in SVM/GPU driver to abuse object map behind those fix mapping. Cheers, Jerome