From mboxrd@z Thu Jan 1 00:00:00 1970 From: Joonas Lahtinen Subject: Re: [PATCH v4 2/2] drm/i915: Make GPU pages movable Date: Tue, 18 Oct 2016 15:05:01 +0300 Message-ID: <1476792301.3117.14.camel@linux.intel.com> References: <1459775891-32442-1-git-send-email-chris@chris-wilson.co.uk> <1459775891-32442-2-git-send-email-chris@chris-wilson.co.uk> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by gabe.freedesktop.org (Postfix) with ESMTPS id D3FD46E6D5 for ; Tue, 18 Oct 2016 12:05:08 +0000 (UTC) In-Reply-To: <1459775891-32442-2-git-send-email-chris@chris-wilson.co.uk> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" To: Chris Wilson , intel-gfx@lists.freedesktop.org Cc: linux-mm@kvack.org, Akash Goel , Hugh Dickins , Sourab Gupta List-Id: intel-gfx@lists.freedesktop.org T24gbWEsIDIwMTYtMDQtMDQgYXQgMTQ6MTggKzAxMDAsIENocmlzIFdpbHNvbiB3cm90ZToKPiBG cm9tOiBBa2FzaCBHb2VsIDxha2FzaC5nb2VsQGludGVsLmNvbT4KPiAKPiBPbiBhIGxvbmcgcnVu IG9mIG1vcmUgdGhhbiAyLTMgZGF5cywgcGh5c2ljYWwgbWVtb3J5IHRlbmRzIHRvIGdldAo+IGZy YWdtZW50ZWQgc2V2ZXJlbHksIHdoaWNoIGNvbnNpZGVyYWJseSBzbG93cyBkb3duIHRoZSBzeXN0 ZW0uIEluIHN1Y2ggYQo+IHNjZW5hcmlvLCB0aGUgc2hyaW5rZXIgaXMgYWxzbyB1bmFibGUgdG8g aGVscCBhcyBsYWNrIG9mIG1lbW9yeSBpcyBub3QKPiB0aGUgYWN0dWFsIHByb2JsZW0sIHNpbmNl IGl0IGhhcyBiZWVuIG9ic2VydmVkIHRoYXQgdGhlcmUgYXJlIGVub3VnaCBmcmVlCj4gcGFnZXMg b2YgMCBvcmRlci4gVGhpcyBhbHNvIG1hbmlmZXN0cyBpdHNlbGYgd2hlbiBhbiBpbmRpdml1YWwg em9uZSBpbgo+IHRoZSBtbSBydW5zIG91dCBvZiBwYWdlcyBhbmQgaWYgd2UgY2Fubm90IG1pZ3Jh dGUgcGFnZXMgYmV0d2VlbiB6b25lcywKPiB0aGUga2VybmVsIGhpdHMgYW4gb3V0LW9mLW1lbW9y eSBldmVuIHRob3VnaCB0aGVyZSBhcmUgZnJlZSBwYWdlcyAoYW5kCj4gb2Z0ZW4gYWxsIG9mIHN3 YXApIGF2YWlsYWJsZS4KPiAKPiBUbyBhZGRyZXNzIHRoZSBpc3N1ZSBvZiBleHRlcm5hbCBmcmFn ZW1lbnRhdGlvbiwga2VybmVsIGRvZXMgYSBjb21wYWN0aW9uCj4gKHdoaWNoIGludm9sdmVzIG1p Z3JhdGlvbiBvZiBwYWdlcykgYnV0IGl0J3MgZWZmaWNhY3kgZGVwZW5kcyB1cG9uIGhvdwo+IG1h bnkgcGFnZXMgYXJlIG1hcmtlZCBhcyBNT1ZBQkxFLCBhcyBvbmx5IHRob3NlIHBhZ2VzIGNhbiBi ZSBtaWdyYXRlZC4KPiAKPiBDdXJyZW50bHkgdGhlIGJhY2tpbmcgcGFnZXMgZm9yIEdGWCBidWZm ZXJzIGFyZSBhbGxvY2F0ZWQgZnJvbSBzaG1lbWZzCj4gd2l0aCBHRlBfUkVDTEFJTUFCTEUgZmxh ZywgaW4gdW5pdHMgb2YgNEtCIHBhZ2VzLsKgwqBJbiB0aGUgY2FzZSBvZiBsaW1pdGVkCj4gc3dh cCBzcGFjZSwgaXQgbWF5IG5vdCBiZSBwb3NzaWJsZSBhbHdheXMgdG8gcmVjbGFpbSBvciBzd2Fw LW91dCBwYWdlcyBvZgo+IGFsbCB0aGUgaW5hY3RpdmUgb2JqZWN0cywgdG8gbWFrZSB3YXkgZm9y IGZyZWUgc3BhY2UgYWxsb3dpbmcgZm9ybWF0aW9uCj4gb2YgaGlnaGVyIG9yZGVyIGdyb3VwcyBv ZiBwaHlzaWNhbGx5LWNvbnRpZ3VvdXMgcGFnZXMgb24gY29tcGFjdGlvbi4KPiAKPiBKdXN0IG1h cmtpbmcgdGhlIEdQVSBwYWdlcyBhcyBNT1ZBQkxFIHdpbGwgbm90IHN1ZmZpY2UsIGFzIGk5MTUu a28gaGFzIHRvCj4gcGluIHRoZSBwYWdlcyBpZiB0aGV5IGFyZSBpbiB1c2UgYnkgR1BVLCB3aGlj aCB3aWxsIHByZXZlbnQgdGhlaXIKPiBtaWdyYXRpb24uIFNvIHRoZSBtaWdyYXRlcGFnZSBjYWxs YmFjayBpbiBzaG1lbSBpcyBhbHNvIGhvb2tlZCB1cCB0byBnZXQKPiBhIG5vdGlmaWNhdGlvbiB3 aGVuIGtlcm5lbCBpbml0aWF0ZXMgdGhlIHBhZ2UgbWlncmF0aW9uLiBPbiB0aGUKPiBub3RpZmlj YXRpb24sIGk5MTUua28gYXBwcm9wcmlhdGVseSB1bnBpbiB0aGUgcGFnZXMuwqDCoFdpdGggdGhp cyB3ZSBjYW4KPiBlZmZlY3RpdmVseSBtYXJrIHRoZSBHUFUgcGFnZXMgYXMgTU9WQUJMRSBhbmQg aGVuY2UgbWl0aWdhdGUgdGhlCj4gZnJhZ21lbnRhdGlvbiBwcm9ibGVtLgo+IAo+IHYyOgo+IMKg LSBSZW5hbWUgdGhlIG1pZ3JhdGlvbiByb3V0aW5lIHRvIGdlbV9zaHJpbmtfbWlncmF0ZXBhZ2Us IG1vdmUgaXQgdG8gdGhlCj4gwqDCoMKgc2hyaW5rZXIgZmlsZSwgYW5kIHVzZSB0aGUgZXhpc3Rp bmcgY29uc3RydWN0cyAoQ2hyaXMpCj4gwqAtIFRvIGNsZWFudXAsIGFkZCBhIG5ldyBoZWxwZXIg ZnVuY3Rpb24gdG8gZW5jYXBzdWxhdGUgYWxsIHBhZ2UgbWlncmF0aW9uCj4gwqDCoMKgc2tpcCBj b25kaXRpb25zIChDaHJpcykKPiDCoC0gQWRkIGEgbmV3IGxvY2FsIGhlbHBlciBmdW5jdGlvbiBp biBzaHJpbmtlciBmaWxlLCBmb3IgZHJvcHBpbmcgdGhlCj4gwqDCoMKgYmFja2luZyBwYWdlcywg YW5kIGNhbGwgdGhlIHNhbWUgZnJvbSBnZW1fc2hyaW5rKCkgYWxzbyAoQ2hyaXMpCj4gCj4gdjM6 Cj4gwqAtIEZpeC9pbnZlcnQgdGhlIGNoZWNrIG9uIHRoZSByZXR1cm4gdmFsdWUgb2YgdW5zYWZl X2Ryb3BfcGFnZXMgKENocmlzKQo+IAo+IHY0Ogo+IMKgLSBNaW5vciB0aWR5Cj4gCj4gVGVzdGNh c2U6IGlndC9nZW1fc2hyaW5rCj4gQnVnemlsbGE6IChlLmcuKSBodHRwczovL2J1Z3MuZnJlZWRl c2t0b3Aub3JnL3Nob3dfYnVnLmNnaT9pZD05MDI1NAo+IENjOiBIdWdoIERpY2tpbnMgPGh1Z2hk QGdvb2dsZS5jb20+Cj4gQ2M6IGxpbnV4LW1tQGt2YWNrLm9yZwo+IFNpZ25lZC1vZmYtYnk6IFNv dXJhYiBHdXB0YSA8c291cmFiLmd1cHRhQGludGVsLmNvbT4KPiBTaWduZWQtb2ZmLWJ5OiBBa2Fz aCBHb2VsIDxha2FzaC5nb2VsQGludGVsLmNvbT4KPiBSZXZpZXdlZC1ieTogQ2hyaXMgV2lsc29u IDxjaHJpc0BjaHJpcy13aWxzb24uY28udWs+CgpDb3VsZCB0aGlzIHBhdGNoIGJlIHJlLXNwaW5u ZWQgb24gdG9wIG9mIGN1cnJlbnQgbmlnaHRseT8KCkFmdGVyIHJlbW92aW5nOwoKPiBXQVJOKHBh Z2VfY291bnQobmV3cGFnZSkgIT0gMSwgIlVuZXhwZWN0ZWQgcmVmIGNvdW50IGZvciBuZXdwYWdl XG4iKQoKYW5kCgo+CWlmIChyZXQpCj4JCURSTV9ERUJVR19EUklWRVIoInBhZ2U9JXAgbWlncmF0 aW9uIHJldHVybmVkICVkXG4iLCBwYWdlLCByZXQpOwoKVGhpcyBpczsKClJldmlld2VkLWJ5OiBK b29uYXMgTGFodGluZW4gPGpvb25hcy5sYWh0aW5lbkBsaW51eC5pbnRlbC5jb20+CgpSZWdhcmRz LCBKb29uYXMKLS0gCkpvb25hcyBMYWh0aW5lbgpPcGVuIFNvdXJjZSBUZWNobm9sb2d5IENlbnRl cgpJbnRlbCBDb3Jwb3JhdGlvbgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fXwpJbnRlbC1nZnggbWFpbGluZyBsaXN0CkludGVsLWdmeEBsaXN0cy5mcmVlZGVz a3RvcC5vcmcKaHR0cHM6Ly9saXN0cy5mcmVlZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5mby9p bnRlbC1nZngK From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pa0-f71.google.com (mail-pa0-f71.google.com [209.85.220.71]) by kanga.kvack.org (Postfix) with ESMTP id 7BCF86B0038 for ; Tue, 18 Oct 2016 08:05:09 -0400 (EDT) Received: by mail-pa0-f71.google.com with SMTP id gg9so229936161pac.6 for ; Tue, 18 Oct 2016 05:05:09 -0700 (PDT) Received: from mga09.intel.com (mga09.intel.com. [134.134.136.24]) by mx.google.com with ESMTPS id c5si32061113pgi.131.2016.10.18.05.05.08 for (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 18 Oct 2016 05:05:08 -0700 (PDT) Message-ID: <1476792301.3117.14.camel@linux.intel.com> Subject: Re: [Intel-gfx] [PATCH v4 2/2] drm/i915: Make GPU pages movable From: Joonas Lahtinen Date: Tue, 18 Oct 2016 15:05:01 +0300 In-Reply-To: <1459775891-32442-2-git-send-email-chris@chris-wilson.co.uk> References: <1459775891-32442-1-git-send-email-chris@chris-wilson.co.uk> <1459775891-32442-2-git-send-email-chris@chris-wilson.co.uk> Content-Type: text/plain; charset="UTF-8" Mime-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: owner-linux-mm@kvack.org List-ID: To: Chris Wilson , intel-gfx@lists.freedesktop.org Cc: linux-mm@kvack.org, Akash Goel , Hugh Dickins , Sourab Gupta On ma, 2016-04-04 at 14:18 +0100, Chris Wilson wrote: > From: Akash Goel > > On a long run of more than 2-3 days, physical memory tends to get > fragmented severely, which considerably slows down the system. In such a > scenario, the shrinker is also unable to help as lack of memory is not > the actual problem, since it has been observed that there are enough free > pages of 0 order. This also manifests itself when an indiviual zone in > the mm runs out of pages and if we cannot migrate pages between zones, > the kernel hits an out-of-memory even though there are free pages (and > often all of swap) available. > > To address the issue of external fragementation, kernel does a compaction > (which involves migration of pages) but it's efficacy depends upon how > many pages are marked as MOVABLE, as only those pages can be migrated. > > Currently the backing pages for GFX buffers are allocated from shmemfs > with GFP_RECLAIMABLE flag, in units of 4KB pages.A A In the case of limited > swap space, it may not be possible always to reclaim or swap-out pages of > all the inactive objects, to make way for free space allowing formation > of higher order groups of physically-contiguous pages on compaction. > > Just marking the GPU pages as MOVABLE will not suffice, as i915.ko has to > pin the pages if they are in use by GPU, which will prevent their > migration. So the migratepage callback in shmem is also hooked up to get > a notification when kernel initiates the page migration. On the > notification, i915.ko appropriately unpin the pages.A A With this we can > effectively mark the GPU pages as MOVABLE and hence mitigate the > fragmentation problem. > > v2: > A - Rename the migration routine to gem_shrink_migratepage, move it to the > A A A shrinker file, and use the existing constructs (Chris) > A - To cleanup, add a new helper function to encapsulate all page migration > A A A skip conditions (Chris) > A - Add a new local helper function in shrinker file, for dropping the > A A A backing pages, and call the same from gem_shrink() also (Chris) > > v3: > A - Fix/invert the check on the return value of unsafe_drop_pages (Chris) > > v4: > A - Minor tidy > > Testcase: igt/gem_shrink > Bugzilla: (e.g.) https://bugs.freedesktop.org/show_bug.cgi?id=90254 > Cc: Hugh Dickins > Cc: linux-mm@kvack.org > Signed-off-by: Sourab Gupta > Signed-off-by: Akash Goel > Reviewed-by: Chris Wilson Could this patch be re-spinned on top of current nightly? After removing; > WARN(page_count(newpage) != 1, "Unexpected ref count for newpage\n") and > if (ret) > DRM_DEBUG_DRIVER("page=%p migration returned %d\n", page, ret); This is; Reviewed-by: Joonas Lahtinen Regards, Joonas -- Joonas Lahtinen Open Source Technology Center Intel Corporation -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org