From mboxrd@z Thu Jan 1 00:00:00 1970 From: Daniel Vetter Subject: Re: [PATCH v2 1/3] drm/i915: Enable lockless lookup of request tracking via RCU Date: Wed, 6 Jan 2016 09:06:58 +0100 Message-ID: <20160106080658.GC8076@phenom.ffwll.local> References: <1450869563-23892-1-git-send-email-chris@chris-wilson.co.uk> <1450877756-2902-1-git-send-email-chris@chris-wilson.co.uk> <20160105145951.GN8076@phenom.ffwll.local> <20160105150213.GP6344@twins.programming.kicks-ass.net> <20160105150648.GT6373@twins.programming.kicks-ass.net> <20160105163537.GL32217@linux.vnet.ibm.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Received: from mail-wm0-f54.google.com (mail-wm0-f54.google.com [74.125.82.54]) by gabe.freedesktop.org (Postfix) with ESMTPS id 40FE16E2A0 for ; Wed, 6 Jan 2016 00:07:01 -0800 (PST) Received: by mail-wm0-f54.google.com with SMTP id f206so50651883wmf.0 for ; Wed, 06 Jan 2016 00:07:01 -0800 (PST) Content-Disposition: inline In-Reply-To: <20160105163537.GL32217@linux.vnet.ibm.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" To: "Paul E. McKenney" Cc: Christoph Lameter , Peter Zijlstra , intel-gfx@lists.freedesktop.org, Linux MM , Pekka Enberg , Jens Axboe , Hugh Dickins List-Id: intel-gfx@lists.freedesktop.org T24gVHVlLCBKYW4gMDUsIDIwMTYgYXQgMDg6MzU6MzdBTSAtMDgwMCwgUGF1bCBFLiBNY0tlbm5l eSB3cm90ZToKPiBPbiBUdWUsIEphbiAwNSwgMjAxNiBhdCAwNDowNjo0OFBNICswMTAwLCBQZXRl ciBaaWpsc3RyYSB3cm90ZToKPiA+IE9uIFR1ZSwgSmFuIDA1LCAyMDE2IGF0IDA0OjAyOjEzUE0g KzAxMDAsIFBldGVyIFppamxzdHJhIHdyb3RlOgo+ID4gPiA+IFNob3VsZG4ndCB0aGUgc2xhYiBz dWJzeXN0ZW0gZG8gdGhpcyBmb3IgdXMgaWYgd2UgcmVxdWVzdCBpdCBkZWxheXMgdGhlCj4gPiA+ ID4gYWN0dWFsIGtmcmVlPyBTZWVtcyBsaWtlIGEgY29yZSBidWcgdG8gbWUgLi4uIEFkZGluZyBt b3JlIGZvbGtzLgo+ID4gPiAKPiA+ID4gbm90ZSB0aGF0IHN5bmNfcmN1KCkgY2FuIHRha2UgYSB0 ZXJyaWJseSBsb25nIHRpbWUuLiBidXQgeWVzLCBJIHNlZW0gdG8KPiA+ID4gcmVtZW1iZXIgUGF1 bCB0YWxraW5nIGFib3V0IGFkZGluZyB0aGlzIHRvIHJlY2xhaW0gcGF0aHMgZm9yIGp1c3QgdGhp cwo+ID4gPiByZWFzb24uIE5vdCBzdXJlIHRoYXQgZXZlciBoYXBwZW5lZCB0aG91aGcuCj4gCj4g VGhlcmUgaXMgYW4gUkNVIE9PTSBub3RpZmllciwgYnV0IGl0IGp1c3QgZW5zdXJlcyB0aGF0IGV4 aXN0aW5nIGNhbGxiYWNrcwo+IGdldCBwcm9jZXNzZWQgaW4gYSB0aW1lbHkgZmFzaGlvbi4gIEl0 IGRvZXMgbm90IGJsb2NrLCBhcyB0aGF0IHdvdWxkCj4gcHJldmVudCBvdGhlciBPT00gbm90aWZp ZXJzIGZyb20gZ2V0dGluZyB0aGVpciBtZW1vcnkgZnJlZWQgcXVpY2tseS4KPiAKPiA+IEFsc28s IHlvdSBtaWdodCBiZSB3YW50aW5nIHJjdV9iYXJyaWVyKCkgaW5zdGVhZCwgdGhhdCBub3Qgb25s eSB3YWl0cwo+ID4gZm9yIGEgR1AgdG8gY29tcGxldGUsIGJ1dCBhbHNvIGZvciBhbGwgcGVuZGlu ZyBjYWxsYmFja3MgdG8gYmUKPiA+IHByb2Nlc3NlZC4KPiAKPiBBbmQgaW4gZmFjdCB3aGF0IHRo ZSBSQ1UgT09NIG5vdGlmaWVyIGRvZXMgY2FuIGJlIHRob3VnaHQgb2YgYXMgYW4KPiBhc3luY2hy b25vdXMgb3Blbi1jb2RlZCByY3VfYmFycmllcigpLiAgSWYgeW91IGFyZSBpbnRlcmVzdGVkLCBw bGVhc2UKPiBzZWUgcmN1X3JlZ2lzdGVyX29vbV9ub3RpZmllcigpIGFuZCBmcmllbmRzLgo+IAo+ ID4gV2l0aG91dCB0aGUgbGF0dGVyIHRoZXJlIG1pZ2h0IHN0aWxsIG5vdCBiZSBhbnl0aGluZyB0 byBmcmVlIGFmdGVyIGl0Lgo+IAo+IEFub3RoZXIgYXBwcm9hY2ggaXMgc3luY2hyb25pemVfcmN1 KCkgYWZ0ZXIgc29tZSBsYXJnaXNoIG51bWJlciBvZgo+IHJlcXVlc3RzLiAgVGhlIGFkdmFudGFn ZSBvZiB0aGlzIGFwcHJvYWNoIGlzIHRoYXQgaXQgdGhyb3R0bGVzIHRoZQo+IHByb2R1Y3Rpb24g b2YgY2FsbGJhY2tzIGF0IHRoZSBzb3VyY2UuICBUaGUgY29ycmVzcG9uZGluZyBkaXNhZHZhbnRh Z2UKPiBpcyB0aGF0IGl0IHNsb3dzIHRoaW5ncyB1cC4KPiAKPiBBbm90aGVyIGFwcHJvYWNoIGlz IHRvIHVzZSBjYWxsX3JjdSgpLCBidXQgaWYgdGhlIHByZXZpb3VzIGNhbGxfcmN1KCkKPiBpcyBz dGlsbCBpbiBmbGlnaHQsIGJsb2NrIHdhaXRpbmcgZm9yIGl0LiAgWWV0IGFub3RoZXIgYXBwcm9h Y2ggaXMKPiB0aGUgZ2V0X3N0YXRlX3N5bmNocm9uaXplX3JjdSgpIC8gY29uZF9zeW5jaHJvbml6 ZV9yY3UoKSBwYWlyLiAgVGhlCj4gaWRlYSBpcyB0byBkbyBzb21ldGhpbmcgbGlrZSB0aGlzOgo+ IAo+IAljb25kX3N5bmNocm9uaXplX3JjdShjb29raWUpOwo+IAljb29raWUgPSBnZXRfc3RhdGVf c3luY2hyb25pemVfcmN1KCk7Cj4gCj4gWW91IHdvdWxkIG9mIGNvdXJzZSBkbyBhbiBpbml0aWFs IGdldF9zdGF0ZV9zeW5jaHJvbml6ZV9yY3UoKSB0bwo+IGdldCB0aGluZ3MgZ29pbmcuICBUaGlz IHdvdWxkIG5vdCBibG9jayB1bmxlc3MgdGhlcmUgd2FzIGxlc3MgdGhhbgo+IG9uZSBncmFjZSBw ZXJpb2QncyB3b3J0aCBvZiB0aW1lIGJldHdlZW4gaW52b2NhdGlvbnMuICBCdXQgdGhpcwo+IGFz c3VtZXMgYSBidXN5IHN5c3RlbSwgd2hlcmUgdGhlcmUgaXMgYWxtb3N0IGFsd2F5cyBhIGdyYWNl IHBlcmlvZAo+IGluIGZsaWdodC4gIEJ1dCB5b3UgY2FuIG1ha2UgdGhhdCBoYXBwZW4gYXMgZm9s bG93czoKPiAKPiAJY29uZF9zeW5jaHJvbml6ZV9yY3UoY29va2llKTsKPiAJY29va2llID0gZ2V0 X3N0YXRlX3N5bmNocm9uaXplX3JjdSgpOwo+IAljYWxsX3JjdSgmbXlfcmN1X2hlYWQsIG5vb3Bf ZnVuY3Rpb24pOwo+IAo+IE5vdGUgdGhhdCB5b3UgbmVlZCBhZGRpdGlvbmFsIGNvZGUgdG8gbWFr ZSBzdXJlIHRoYXQgdGhlIG9sZCBjYWxsYmFjawo+IGhhcyBjb21wbGV0ZWQgYmVmb3JlIGRvaW5n IGEgbmV3IG9uZS4gIFNldHRpbmcgYW5kIGNsZWFyaW5nIGEgZmxhZwo+IHdpdGggYXBwcm9wcmlh dGUgbWVtb3J5IG9yZGVyaW5nIGNvbnRyb2wgc3VmZmljZXMgKGUuZywuIHNtcF9sb2FkX2FjcXVp cmUoKQo+IGFuZCBzbXBfc3RvcmVfcmVsZWFzZSgpKS4KClRoaXMgcHJldHR5IG11Y2ggd2VudCBv dmVyIG15IGhlYWQgOy0pIFdoYXQgSSBuYWl2ZWx5IGhvcGVkIGZvciBpcyB0aGF0CmtmcmVlKCkg b24gYW4gcmN1LWZyZWVpbmcgc2xhYiBjb3VsZCBiZSB0b3VnaHQgdG8gbWFnaWNhbGx5IHN0YWxs IGEgYml0CihvciBhdCBsZWFzdCBleHBlZGl0ZSB0aGUgZGVsYXllZCBmcmVlaW5nKSBpZiB3ZSdy ZSBwaWxpbmcgdXAgdG9vIG1hbnkKZnJlZWQgb2JqZWN0cy4gRG9pbmcgdGhhdCBvbmx5IGluIE9P TSBpcyBwcm9iYWJseSB0b28gbGF0ZSBzaW5jZSBPT00KaGFuZGxpbmcgaXMgYSBiaXQgdW5yZWxp YWJsZS91bnByZWRpY3RhYmxlLiBBbmQgSSB0aG91Z2h0IHdlJ3JlIG5vdCB0aGUKZmlyc3Qgb25l cyBydW5uaW5nIGludG8gdGhpcyBwcm9ibGVtLgoKRG8gYWxsIHRoZSBvdGhlciB1c2VycyBvZiBy Y3UtZnJlZWQgc2xhYnMganVzdCBvcGVuLWNvZGUgdGhlaXIgb3duIGN1c3RvbQphcHByb2FjaD8g SWYgdGhhdCdzIHRoZSByZWNvbW1lbmRhdGlvbiB3ZSBjYW4gY2VydGFpbmx5IGZvbGxvdyB0aGF0 LCB0b28uCi1EYW5pZWwKLS0gCkRhbmllbCBWZXR0ZXIKU29mdHdhcmUgRW5naW5lZXIsIEludGVs IENvcnBvcmF0aW9uCmh0dHA6Ly9ibG9nLmZmd2xsLmNoCl9fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fCkludGVsLWdmeCBtYWlsaW5nIGxpc3QKSW50ZWwtZ2Z4 QGxpc3RzLmZyZWVkZXNrdG9wLm9yZwpodHRwOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxt YW4vbGlzdGluZm8vaW50ZWwtZ2Z4Cg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f43.google.com (mail-wm0-f43.google.com [74.125.82.43]) by kanga.kvack.org (Postfix) with ESMTP id AB96A6B0003 for ; Wed, 6 Jan 2016 03:07:01 -0500 (EST) Received: by mail-wm0-f43.google.com with SMTP id f206so63903566wmf.0 for ; Wed, 06 Jan 2016 00:07:01 -0800 (PST) Received: from mail-wm0-x236.google.com (mail-wm0-x236.google.com. [2a00:1450:400c:c09::236]) by mx.google.com with ESMTPS id bk7si106560971wjb.34.2016.01.06.00.07.00 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 06 Jan 2016 00:07:00 -0800 (PST) Received: by mail-wm0-x236.google.com with SMTP id f206so50651859wmf.0 for ; Wed, 06 Jan 2016 00:07:00 -0800 (PST) Date: Wed, 6 Jan 2016 09:06:58 +0100 From: Daniel Vetter Subject: Re: [Intel-gfx] [PATCH v2 1/3] drm/i915: Enable lockless lookup of request tracking via RCU Message-ID: <20160106080658.GC8076@phenom.ffwll.local> References: <1450869563-23892-1-git-send-email-chris@chris-wilson.co.uk> <1450877756-2902-1-git-send-email-chris@chris-wilson.co.uk> <20160105145951.GN8076@phenom.ffwll.local> <20160105150213.GP6344@twins.programming.kicks-ass.net> <20160105150648.GT6373@twins.programming.kicks-ass.net> <20160105163537.GL32217@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160105163537.GL32217@linux.vnet.ibm.com> Sender: owner-linux-mm@kvack.org List-ID: To: "Paul E. McKenney" Cc: Peter Zijlstra , Chris Wilson , intel-gfx@lists.freedesktop.org, Linux MM , Jens Axboe , Christoph Lameter , Hugh Dickins , Pekka Enberg On Tue, Jan 05, 2016 at 08:35:37AM -0800, Paul E. McKenney wrote: > On Tue, Jan 05, 2016 at 04:06:48PM +0100, Peter Zijlstra wrote: > > On Tue, Jan 05, 2016 at 04:02:13PM +0100, Peter Zijlstra wrote: > > > > Shouldn't the slab subsystem do this for us if we request it delays the > > > > actual kfree? Seems like a core bug to me ... Adding more folks. > > > > > > note that sync_rcu() can take a terribly long time.. but yes, I seem to > > > remember Paul talking about adding this to reclaim paths for just this > > > reason. Not sure that ever happened thouhg. > > There is an RCU OOM notifier, but it just ensures that existing callbacks > get processed in a timely fashion. It does not block, as that would > prevent other OOM notifiers from getting their memory freed quickly. > > > Also, you might be wanting rcu_barrier() instead, that not only waits > > for a GP to complete, but also for all pending callbacks to be > > processed. > > And in fact what the RCU OOM notifier does can be thought of as an > asynchronous open-coded rcu_barrier(). If you are interested, please > see rcu_register_oom_notifier() and friends. > > > Without the latter there might still not be anything to free after it. > > Another approach is synchronize_rcu() after some largish number of > requests. The advantage of this approach is that it throttles the > production of callbacks at the source. The corresponding disadvantage > is that it slows things up. > > Another approach is to use call_rcu(), but if the previous call_rcu() > is still in flight, block waiting for it. Yet another approach is > the get_state_synchronize_rcu() / cond_synchronize_rcu() pair. The > idea is to do something like this: > > cond_synchronize_rcu(cookie); > cookie = get_state_synchronize_rcu(); > > You would of course do an initial get_state_synchronize_rcu() to > get things going. This would not block unless there was less than > one grace period's worth of time between invocations. But this > assumes a busy system, where there is almost always a grace period > in flight. But you can make that happen as follows: > > cond_synchronize_rcu(cookie); > cookie = get_state_synchronize_rcu(); > call_rcu(&my_rcu_head, noop_function); > > Note that you need additional code to make sure that the old callback > has completed before doing a new one. Setting and clearing a flag > with appropriate memory ordering control suffices (e.g,. smp_load_acquire() > and smp_store_release()). This pretty much went over my head ;-) What I naively hoped for is that kfree() on an rcu-freeing slab could be tought to magically stall a bit (or at least expedite the delayed freeing) if we're piling up too many freed objects. Doing that only in OOM is probably too late since OOM handling is a bit unreliable/unpredictable. And I thought we're not the first ones running into this problem. Do all the other users of rcu-freed slabs just open-code their own custom approach? If that's the recommendation we can certainly follow that, too. -Daniel -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org