From mboxrd@z Thu Jan 1 00:00:00 1970 From: Daniel Vetter Subject: Re: [PATCH v4 1/2] shmem: Support for registration of driver/file owner specific ops Date: Tue, 26 Apr 2016 14:53:41 +0200 Message-ID: <20160426125341.GF8291@phenom.ffwll.local> References: <1459775891-32442-1-git-send-email-chris@chris-wilson.co.uk> <20160424234250.GB6670@node.shutemov.name> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Received: from mail-wm0-x242.google.com (mail-wm0-x242.google.com [IPv6:2a00:1450:400c:c09::242]) by gabe.freedesktop.org (Postfix) with ESMTPS id 0C1AB6E816 for ; Tue, 26 Apr 2016 12:53:46 +0000 (UTC) Received: by mail-wm0-x242.google.com with SMTP id n3so4738790wmn.1 for ; Tue, 26 Apr 2016 05:53:46 -0700 (PDT) Content-Disposition: inline In-Reply-To: <20160424234250.GB6670@node.shutemov.name> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" To: "Kirill A. Shutemov" Cc: intel-gfx@lists.freedesktop.org, Hugh Dickins , Sourab Gupta , linux-mm@kvack.org, Akash Goel , linux-kernel@vger.linux.org List-Id: intel-gfx@lists.freedesktop.org T24gTW9uLCBBcHIgMjUsIDIwMTYgYXQgMDI6NDI6NTBBTSArMDMwMCwgS2lyaWxsIEEuIFNodXRl bW92IHdyb3RlOgo+IE9uIE1vbiwgQXByIDA0LCAyMDE2IGF0IDAyOjE4OjEwUE0gKzAxMDAsIENo cmlzIFdpbHNvbiB3cm90ZToKPiA+IEZyb206IEFrYXNoIEdvZWwgPGFrYXNoLmdvZWxAaW50ZWwu Y29tPgo+ID4gCj4gPiBUaGlzIHByb3ZpZGVzIHN1cHBvcnQgZm9yIHRoZSBkcml2ZXJzIG9yIHNo bWVtIGZpbGUgb3duZXJzIHRvIHJlZ2lzdGVyCj4gPiBhIHNldCBvZiBjYWxsYmFja3MsIHdoaWNo IGNhbiBiZSBpbnZva2VkIGZyb20gdGhlIGFkZHJlc3Mgc3BhY2UKPiA+IG9wZXJhdGlvbnMgbWV0 aG9kcyBpbXBsZW1lbnRlZCBieSBzaG1lbS4gIFRoaXMgYWxsb3cgdGhlIGZpbGUgb3duZXJzIHRv Cj4gPiBob29rIGludG8gdGhlIHNobWVtIGFkZHJlc3Mgc3BhY2Ugb3BlcmF0aW9ucyB0byBkbyBz b21lIGV4dHJhL2N1c3RvbQo+ID4gb3BlcmF0aW9ucyBpbiBhZGRpdGlvbiB0byB0aGUgZGVmYXVs dCBvbmVzLgo+ID4gCj4gPiBUaGUgcHJpdmF0ZV9kYXRhIGZpZWxkIG9mIGFkZHJlc3Nfc3BhY2Ug c3RydWN0IGlzIHVzZWQgdG8gc3RvcmUgdGhlCj4gPiBwb2ludGVyIHRvIGRyaXZlciBzcGVjaWZp YyBvcHMuICBDdXJyZW50bHkgb25seSBvbmUgb3BzIGZpZWxkIGlzIGRlZmluZWQsCj4gPiB3aGlj aCBpcyBtaWdyYXRlcGFnZSwgYnV0IGNhbiBiZSBleHRlbmRlZCBvbiBhbiBhcy1uZWVkZWQgYmFz aXMuCj4gPiAKPiA+IFRoZSBuZWVkIGZvciBkcml2ZXIgc3BlY2lmaWMgb3BlcmF0aW9ucyBhcmlz ZXMgc2luY2Ugc29tZSBvZiB0aGUKPiA+IG9wZXJhdGlvbnMgKGxpa2UgbWlncmF0ZXBhZ2UpIG1h eSBub3QgYmUgaGFuZGxlZCBjb21wbGV0ZWx5IHdpdGhpbiBzaG1lbSwKPiA+IHNvIGFzIHRvIGJl IGVmZmVjdGl2ZSwgYW5kIHdvdWxkIG5lZWQgc29tZSBkcml2ZXIgc3BlY2lmaWMgaGFuZGxpbmcg YWxzby4KPiA+IFNwZWNpZmljYWxseSwgaTkxNS5rbyB3b3VsZCBsaWtlIHRvIHBhcnRpY2lwYXRl IGluIG1pZ3JhdGVwYWdlKCkuCj4gPiBpOTE1LmtvIHVzZXMgc2htZW1mcyB0byBwcm92aWRlIHN3 YXBwYWJsZSBiYWNraW5nIHN0b3JhZ2UgZm9yIGl0cyB1c2VyCj4gPiBvYmplY3RzLCBidXQgd2hl biB0aG9zZSBvYmplY3RzIGFyZSBpbiB1c2UgYnkgdGhlIEdQVSBpdCBtdXN0IHBpbiB0aGUKPiA+ IGVudGlyZSBvYmplY3QgdW50aWwgdGhlIEdQVSBpcyBpZGxlLiAgQXMgYSByZXN1bHQsIGxhcmdl IGNodW5rcyBvZiBtZW1vcnkKPiA+IGNhbiBiZSBhcmJpdHJhcmlseSB3aXRoZHJhd24gZnJvbSBw YWdlIG1pZ3JhdGlvbiwgcmVzdWx0aW5nIGluIHByZW1hdHVyZQo+ID4gb3V0LW9mLW1lbW9yeSBk dWUgdG8gZnJhZ21lbnRhdGlvbi4gIEhvd2V2ZXIsIGlmIGk5MTUua28gY2FuIHJlY2VpdmUgdGhl Cj4gPiBtaWdyYXRlcGFnZSgpIHJlcXVlc3QsIGl0IGNhbiB0aGVuIGZsdXNoIHRoZSBvYmplY3Qg ZnJvbSB0aGUgR1BVLCByZW1vdmUKPiA+IGl0cyBwaW4gYW5kIHRodXMgZW5hYmxlIHRoZSBtaWdy YXRpb24uCj4gPiAKPiA+IFNpbmNlIGdmeCBhbGxvY2F0aW9ucyBhcmUgb25lIG9mIHRoZSBtYWpv ciBjb25zdW1lciBvZiBzeXN0ZW0gbWVtb3J5LCBpdHMKPiA+IGltcGVyYXRpdmUgdG8gaGF2ZSBz dWNoIGEgbWVjaGFuaXNtIHRvIGVmZmVjdGl2ZWx5IGRlYWwgd2l0aAo+ID4gZnJhZ21lbnRhdGlv bi4gIEFuZCB0aGVyZWZvcmUgdGhlIG5lZWQgZm9yIHN1Y2ggYSBwcm92aXNpb24gZm9yIGluaXRp YXRpbmcKPiA+IGRyaXZlciBzcGVjaWZpYyBhY3Rpb25zIGR1cmluZyBhZGRyZXNzIHNwYWNlIG9w ZXJhdGlvbnMuCj4gCj4gSG0uIFNvcnJ5LCBteSBpZ25vcmFuY2UsIGJ1dCBzaG91bGRuJ3QgdGhp cyBraW5kIG9mIGZsdXNoaW5nIGJlIGRvbmUgaW4KPiByZXNwb25zZSB0byBtbXVfbm90aWZpZXIn cyAtPmludmFsaWRhdGVfcGFnZT8KPiAKPiBJJ20gbm90IGF3YXJlIGFib3V0IGhvdyBpOTE1IHdv cmtzIGFuZCB3aGF0J3MgaXRzIGV4cGVjdGF0aW9uIHdydCBzaG1lbS4KPiBEbyB5b3UgaGF2ZSBz b21lIHVzZXJzcGFjZSBWTUEgd2hpY2ggaXMgbWlycm9yZWQgb24gR1BVIHNpZGU/Cj4gSWYgeWVz LCBtaWdyYXRpb24gd291bGQgY2F1c2UgdW5tYXBwaW5nIG9mIHRoZXNlIHBhZ2VzIGFuZCB0cmln Z2VyIHRoZQo+IG1tdV9ub3RpZmllcidzIGhvb2suCgpXZSBkbyB0aGF0IGZvciB1c2VycHRyIHBh Z2VzIChpLmUuIHN0dWZmIHdlIHN0ZWFsIGZyb20gdXNlcnNwYWNlIGFkZHJlc3MKc3BhY2VzKS4g QnV0IHdlIGFsc28gaGF2ZSBuYXRpdmUgZ2Z4IGJ1ZmZlciBvYmplY3RzIGJhc2VkIG9uIHNobWVt IGZpbGVzLAphbmQgdGh1cyBmYXIgd2UgbmVlZCB0byBhbGxvY2F0ZSB0aGVtIGFzICFHRlBfTU9W RUFCTEUuIEFuZCB3ZSBhbGxvY2F0ZSBhCl9sb3RfIG9mIHRob3NlLiBBbmQgdGhvc2UgZmlsZXMg YXJlbid0IG1hcHBlZCBpbnRvIGFueSBjcHUgYWRkcmVzcyBzcGFjZQoob2ZjIHRoZXkncmUgbWFw cGVkIG9uIHRoZSBncHUgc2lkZSwgYnV0IHRoYXQncyBkcml2ZXIgcHJpdmF0ZSksIGZyb20gdGhl CmNvcmUgbW0gdGhleSBhcmUgcHVyZSBwYWdlY2FjaGUuIEFuZCBhZmFpdWkgZm9yIHRoYXQgd2Ug bmVlZCB0byB3aXJlIHVwCnRoZSBtaWdyYXRlcGFnZSBob29rcyB0aHJvdWdoIHNobWVtIHRvIGk5 MTVfZ2VtLmMKLURhbmllbAotLSAKRGFuaWVsIFZldHRlcgpTb2Z0d2FyZSBFbmdpbmVlciwgSW50 ZWwgQ29ycG9yYXRpb24KaHR0cDovL2Jsb2cuZmZ3bGwuY2gKX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX18KSW50ZWwtZ2Z4IG1haWxpbmcgbGlzdApJbnRlbC1n ZnhAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21h aWxtYW4vbGlzdGluZm8vaW50ZWwtZ2Z4Cg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f69.google.com (mail-wm0-f69.google.com [74.125.82.69]) by kanga.kvack.org (Postfix) with ESMTP id 028B66B0005 for ; Tue, 26 Apr 2016 08:53:47 -0400 (EDT) Received: by mail-wm0-f69.google.com with SMTP id w143so11526244wmw.3 for ; Tue, 26 Apr 2016 05:53:46 -0700 (PDT) Received: from mail-wm0-x243.google.com (mail-wm0-x243.google.com. [2a00:1450:400c:c09::243]) by mx.google.com with ESMTPS id o8si3184278wmg.24.2016.04.26.05.53.45 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 26 Apr 2016 05:53:45 -0700 (PDT) Received: by mail-wm0-x243.google.com with SMTP id w143so4173008wmw.3 for ; Tue, 26 Apr 2016 05:53:45 -0700 (PDT) Date: Tue, 26 Apr 2016 14:53:41 +0200 From: Daniel Vetter Subject: Re: [PATCH v4 1/2] shmem: Support for registration of driver/file owner specific ops Message-ID: <20160426125341.GF8291@phenom.ffwll.local> References: <1459775891-32442-1-git-send-email-chris@chris-wilson.co.uk> <20160424234250.GB6670@node.shutemov.name> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160424234250.GB6670@node.shutemov.name> Sender: owner-linux-mm@kvack.org List-ID: To: "Kirill A. Shutemov" Cc: Chris Wilson , intel-gfx@lists.freedesktop.org, Akash Goel , Hugh Dickins , linux-mm@kvack.org, linux-kernel@vger.linux.org, Sourab Gupta On Mon, Apr 25, 2016 at 02:42:50AM +0300, Kirill A. Shutemov wrote: > On Mon, Apr 04, 2016 at 02:18:10PM +0100, Chris Wilson wrote: > > From: Akash Goel > > > > This provides support for the drivers or shmem file owners to register > > a set of callbacks, which can be invoked from the address space > > operations methods implemented by shmem. This allow the file owners to > > hook into the shmem address space operations to do some extra/custom > > operations in addition to the default ones. > > > > The private_data field of address_space struct is used to store the > > pointer to driver specific ops. Currently only one ops field is defined, > > which is migratepage, but can be extended on an as-needed basis. > > > > The need for driver specific operations arises since some of the > > operations (like migratepage) may not be handled completely within shmem, > > so as to be effective, and would need some driver specific handling also. > > Specifically, i915.ko would like to participate in migratepage(). > > i915.ko uses shmemfs to provide swappable backing storage for its user > > objects, but when those objects are in use by the GPU it must pin the > > entire object until the GPU is idle. As a result, large chunks of memory > > can be arbitrarily withdrawn from page migration, resulting in premature > > out-of-memory due to fragmentation. However, if i915.ko can receive the > > migratepage() request, it can then flush the object from the GPU, remove > > its pin and thus enable the migration. > > > > Since gfx allocations are one of the major consumer of system memory, its > > imperative to have such a mechanism to effectively deal with > > fragmentation. And therefore the need for such a provision for initiating > > driver specific actions during address space operations. > > Hm. Sorry, my ignorance, but shouldn't this kind of flushing be done in > response to mmu_notifier's ->invalidate_page? > > I'm not aware about how i915 works and what's its expectation wrt shmem. > Do you have some userspace VMA which is mirrored on GPU side? > If yes, migration would cause unmapping of these pages and trigger the > mmu_notifier's hook. We do that for userptr pages (i.e. stuff we steal from userspace address spaces). But we also have native gfx buffer objects based on shmem files, and thus far we need to allocate them as !GFP_MOVEABLE. And we allocate a _lot_ of those. And those files aren't mapped into any cpu address space (ofc they're mapped on the gpu side, but that's driver private), from the core mm they are pure pagecache. And afaiui for that we need to wire up the migratepage hooks through shmem to i915_gem.c -Daniel -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org