From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jerome Glisse Subject: Re: [Linaro-mm-sig] [PATCH 1/5] dma-buf: add optional invalidate_mappings callback v2 Date: Mon, 26 Mar 2018 11:42:24 -0400 Message-ID: <20180326154224.GA11930@gmail.com> References: <0bd85f69-c64c-70d1-a4a0-10ae0ed8b4e8@gmail.com> <19ed21a5-805d-271f-9120-49e0c00f510f@amd.com> <20180320140810.GU14155@phenom.ffwll.local> <37ba7394-2a5c-a0bc-cc51-c8a0edc2991d@gmail.com> <20180321082839.GA14155@phenom.ffwll.local> <327c4bc1-5813-16e8-62fc-4301b19a1a22@gmail.com> <20180322071804.GH14155@phenom.ffwll.local> <20180326080121.GO14155@phenom.ffwll.local> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Content-Disposition: inline In-Reply-To: <20180326080121.GO14155-dv86pmgwkMBes7Z6vYuT8azUEOm+Xw19@public.gmane.org> List-Id: Discussion list for AMD gfx List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: amd-gfx-bounces-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org Sender: "amd-gfx" To: Daniel Vetter Cc: Daniel Vetter , dri-devel , "moderated list:DMA BUFFER SHARING FRAMEWORK" , amd-gfx list , christian.koenig-5C7GfCeVMHo@public.gmane.org, "open list:DMA BUFFER SHARING FRAMEWORK" T24gTW9uLCBNYXIgMjYsIDIwMTggYXQgMTA6MDE6MjFBTSArMDIwMCwgRGFuaWVsIFZldHRlciB3 cm90ZToKPiBPbiBUaHUsIE1hciAyMiwgMjAxOCBhdCAxMDo1ODo1NUFNICswMTAwLCBDaHJpc3Rp YW4gS8O2bmlnIHdyb3RlOgo+ID4gQW0gMjIuMDMuMjAxOCB1bSAwODoxOCBzY2hyaWViIERhbmll bCBWZXR0ZXI6Cj4gPiA+IE9uIFdlZCwgTWFyIDIxLCAyMDE4IGF0IDEyOjU0OjIwUE0gKzAxMDAs IENocmlzdGlhbiBLw7ZuaWcgd3JvdGU6Cj4gPiA+ID4gQW0gMjEuMDMuMjAxOCB1bSAwOToyOCBz Y2hyaWViIERhbmllbCBWZXR0ZXI6Cj4gPiA+ID4gPiBPbiBUdWUsIE1hciAyMCwgMjAxOCBhdCAw Njo0Nzo1N1BNICswMTAwLCBDaHJpc3RpYW4gS8O2bmlnIHdyb3RlOgo+ID4gPiA+ID4gPiBBbSAy MC4wMy4yMDE4IHVtIDE1OjA4IHNjaHJpZWIgRGFuaWVsIFZldHRlcjoKPiA+ID4gPiA+ID4gPiBb U05JUF0KPiA+ID4gPiA+ID4gPiBGb3IgdGhlIGluLWRyaXZlciByZXNlcnZhdGlvbiBwYXRoIChD UykgaGF2aW5nIGEgc2xvdy1wYXRoIHRoYXQgZ3JhYnMgYQo+ID4gPiA+ID4gPiA+IHRlbXBvcmFy eSByZWZlcmVuY2UsIGRyb3BzIHRoZSB2cmFtIGxvY2sgYW5kIHRoZW4gbG9ja3MgdGhlIHJlc2Vy dmF0aW9uCj4gPiA+ID4gPiA+ID4gbm9ybWFsbHkgKHVzaW5nIHRoZSBhY3F1aXJlIGNvbnRleHQg dXNlZCBhbHJlYWR5IGZvciB0aGUgZW50aXJlIENTKSBpcyBhCj4gPiA+ID4gPiA+ID4gYml0IHRy aWNreSwgYnV0IHRvdGFsbHkgZmVhc2libGUuIFR0bSBkb2Vzbid0IGRvIHRoYXQgdGhvdWdoLgo+ ID4gPiA+ID4gPiBUaGF0IGlzIGV4YWN0bHkgd2hhdCB3ZSBkbyBpbiBhbWRncHUgYXMgd2VsbCwg aXQncyBqdXN0IG5vdCB2ZXJ5IGVmZmljaWVudAo+ID4gPiA+ID4gPiBub3IgcmVsaWFibGUgdG8g cmV0cnkgZ2V0dGluZyB0aGUgcmlnaHQgcGFnZXMgZm9yIGEgc3VibWlzc2lvbiBvdmVyIGFuZCBv dmVyCj4gPiA+ID4gPiA+IGFnYWluLgo+ID4gPiA+ID4gT3V0IG9mIGN1cmlvc2l0eSwgd2hlcmUn cyB0aGF0IGNvZGU/IEkgZGlkIHJlYWQgdGhlIHR0bSBldmljdGlvbiBjb2RlIHdheQo+ID4gPiA+ ID4gYmFjaywgYW5kIHRoYXQgb25lIGRlZmluaXRlbHkgZGlkbid0IGRvIHRoYXQuIFdvdWxkIGJl IGludGVyZXN0aW5nIHRvCj4gPiA+ID4gPiB1cGRhdGUgbXkgdW5kZXJzdGFuZGluZy4KPiA+ID4g PiBUaGF0IGlzIGluIGFtZGdwdV9jcy5jLiBhbWRncHVfY3NfcGFyc2VyX2JvcygpIGRvZXMgYSBo b3JyaWJsZSBkYW5jZSB3aXRoCj4gPiA+ID4gZ3JhYmJpbmcsIHJlbGVhc2luZyBhbmQgcmVncmFi YmluZyBsb2NrcyBpbiBhIGxvb3AuCj4gPiA+ID4gCj4gPiA+ID4gVGhlbiBpbiBhbWRncHVfY3Nf c3VibWl0KCkgd2UgZ3JhYiBhbiBsb2NrIHByZXZlbnRpbmcgcGFnZSB0YWJsZSB1cGRhdGVzIGFu ZAo+ID4gPiA+IGNoZWNrIGlmIGFsbCBwYWdlcyBhcmUgc3RpbGwgdGhlIG9uZSB3ZSB3YW50IHRv IGhhdmU6Cj4gPiA+ID4gPiAgwqDCoMKgwqDCoMKgwqAgYW1kZ3B1X21uX2xvY2socC0+bW4pOwo+ ID4gPiA+ID4gIMKgwqDCoMKgwqDCoMKgIGlmIChwLT5ib19saXN0KSB7Cj4gPiA+ID4gPiAgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGZvciAoaSA9IHAtPmJvX2xpc3QtPmZpcnN0X3Vz ZXJwdHI7Cj4gPiA+ID4gPiAgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oCBpIDwgcC0+Ym9fbGlzdC0+bnVtX2VudHJpZXM7ICsraSkgewo+ID4gPiA+ID4gIMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgc3RydWN0IGFtZGdwdV9ibyAq Ym8gPSBwLT5ib19saXN0LT5hcnJheVtpXS5yb2JqOwo+ID4gPiA+ID4gCj4gPiA+ID4gPiAgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBpZgo+ID4gPiA+ID4g KGFtZGdwdV90dG1fdHRfdXNlcnB0cl9uZWVkc19wYWdlcyhiby0+dGJvLnR0bSkpIHsKPiA+ID4g PiA+ICDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoCBhbWRncHVfbW5fdW5sb2NrKHAtPm1uKTsKPiA+ID4gPiA+ICDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCByZXR1 cm4gLUVSRVNUQVJUU1lTOwo+ID4gPiA+ID4gIMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqAgfQo+ID4gPiA+ID4gIMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoCB9Cj4gPiA+ID4gPiAgwqDCoMKgwqDCoMKgwqAgfQo+ID4gPiA+IElmIGFueXRoaW5nIGNo YW5nZWQgb24gdGhlIHBhZ2UgdGFibGVzIHdlIHJlc3RhcnQgdGhlIHdob2xlIElPQ1RMIHVzaW5n Cj4gPiA+ID4gLUVSRVNUQVJUU1lTIGFuZCB0cnkgYWdhaW4uCj4gPiA+IEknbSBub3QgdGFsa2lu ZyBhYm91dCB1c2VycHRyIGhlcmUsIGJ1dCBnZW5lcmFsIGJvIGV2aWN0aW9uLiBTb3JyeSBmb3Ig dGhlCj4gPiA+IGNvbmZ1c2lvbi4KPiA+ID4gCj4gPiA+IFRoZSByZWFzb24gSSdtIGRyYWdnaW5n IGFsbCB0aGUgZ2VuZXJhbCBibyBtYW5hZ2VtZW50IGludG8gdGhpcwo+ID4gPiBkaXNjdXNzaW9u cyBpcyBiZWNhdXNlIHdlIGRvIHNlZW0gdG8gaGF2ZSBmYWlybHkgZnVuZGFtZW50YWwgZGlmZmVy ZW5jZSBpbgo+ID4gPiBob3cgdGhhdCdzIGRvbmUsIHdpdGggcmVzdWx0aW5nIGNvbnNlcXVlbmNl cyBmb3IgdGhlIGxvY2tpbmcgaGllcmFyY2h5Lgo+ID4gPiAKPiA+ID4gQW5kIGlmIHRoaXMgaW52 YWxpZGF0ZV9tYXBwaW5nIHN0dWZmIHNob3VsZCB3b3JrLCB0b2dldGhlciB3aXRoIHVzZXJwdHIK PiA+ID4gYW5kIGV2ZXJ5dGhpbmcgZWxzZSwgSSB0aGluayB3ZSdyZSByZXF1aXJlZCB0byBhZ3Jl ZSBvbiBob3cgdGhpcyBpcyBhbGwKPiA+ID4gc3VwcG9zZWQgdG8gbmVzdCwgYW5kIGhvdyBleGFj dGx5IHdlIHNob3VsZCBiYWNrIG9mZiBmb3IgdGhlIG90aGVyIHNpZGUKPiA+ID4gdGhhdCBuZWVk cyB0byBicmVhayB0aGUgbG9ja2luZyBjaXJjbGUuCj4gPiA+IAo+ID4gPiBUaGF0IGFzaWRlLCBJ IGRvbid0IGVudGlyZWx5IHVuZGVyc3RhbmQgd2h5IHlvdSBuZWVkIHRvIHJlc3RhcnQgc28gbXVj aC4gSQo+ID4gPiBmaWd1cmVkIHRoYXQgZ2V0X3VzZXJfcGFnZXMgaXMgb3JkZXJlZCBjb3JyZWN0 bHkgYWdhaW5zdCBtbXUKPiA+ID4gaW52YWxpZGF0aW9ucywgYnV0IEkgZ2V0IHRoZSBpbXByZXNz aW9uIHlvdSB0aGluayB0aGF0J3Mgbm90IHRoZSBjYXNlLiBIb3cKPiA+ID4gZG9lcyB0aGF0IGhh cHBlbj8KPiA+IAo+ID4gQ29ycmVjdC4gSSd2ZSBoYWQgdGhlIHNhbWUgYXNzdW1wdGlvbiwgYnV0 IGJvdGggSmVyb21lIGFzIHdlbGwgYXMgb3VyCj4gPiBpbnRlcm5hbCB0ZXN0cyBwcm92ZWQgbWUg d3Jvbmcgb24gdGhhdC4KPiA+IAo+ID4gS2V5IHRha2UgYXdheSBmcm9tIHRoYXQgd2FzIHRoYXQg eW91IGNhbid0IHRha2UgYW55IGxvY2tzIGZyb20gbmVpdGhlciB0aGUKPiA+IE1NVSBub3RpZmll ciBub3IgdGhlIHNocmlua2VyIHlvdSBhbHNvIHRha2Ugd2hpbGUgY2FsbGluZyBrbWFsbG9jIChv cgo+ID4gc2ltcGxlciBzcGVha2luZyBnZXRfdXNlcl9wYWdlcygpKS4KPiA+IAo+ID4gQWRkaXRp b25hbCB0byB0aGF0IGluIHRoZSBNTVUgb3Igc2hyaW5rZXIgY2FsbGJhY2sgYWxsIGRpZmZlcmVu dCBraW5kcyBvZgo+ID4gbG9ja3MgbWlnaHQgYmUgaGVsZCwgc28geW91IGJhc2ljYWxseSBjYW4n dCBhc3N1bWUgdGhhdCB5b3UgZG8gdGhpbmtzIGxpa2UKPiA+IHJlY3Vyc2l2ZSBwYWdlIHRhYmxl IHdhbGtzIG9yIGNhbGwgZG1hX3VubWFwX2FueXRoaW5nLgo+IAo+IFRoYXQgc291bmRzIGxpa2Ug YSBkZXNpZ24gYnVnIGluIG1tdV9ub3RpZmllcnMsIHNpbmNlIGl0IHdvdWxkIHJlbmRlciB0aGVt Cj4gdXNlbGVzcyBmb3IgS1ZNLiBBbmQgdGhleSB3ZXJlIGRldmVsb3BlZCBmb3IgdGhhdCBvcmln aW5hbGx5LiBJIHRoaW5rIEknbGwKPiBjaGF0IHdpdGggSmVyb21lIHRvIHVuZGVyc3RhbmQgdGhp cywgc2luY2UgaXQncyBhbGwgcmF0aGVyIGNvbmZ1c2luZy4KCkRvaW5nIGRtYV91bm1hcCgpIGR1 cmluZyBtbXVfbm90aWZpZXIgY2FsbGJhY2sgc2hvdWxkIGJlIGZpbmUsIGl0IHdhcyBsYXN0CnRp bWUgaSBjaGVjay4gSG93ZXZlciB0aGVyZSBpcyBubyBmb3JtYWwgY29udHJhY3QgdGhhdCBpdCBp cyBvayB0byBkbyBzby4KQnV0IGkgd2FudCB0byByZXZpc2l0IERNQSBBUEkgdG8gc29tZXRoaW5n IHdoZXJlIGRldmljZSBkcml2ZXIgZ2V0IGNvbnRyb2wKcmF0aGVyIHRoYW4gZGVsZWdhdGUuIEhp Z2ggbGV2ZWwgaWRlYXMgYXJlOgogIC0gZGV2aWNlIGRyaXZlciBhbGxvY2F0ZSBETUEgc3BhY2Ug aWUgcmFuZ2Ugb2YgcGh5c2ljYWwgYnVzIGFkZHJlc3MKICAtIGRldmljZSBkcml2ZXIgY29tcGxl dGVseSBtYW5hZ2UgdGhlIHJhbmdlIGluIHdoaWNoZXZlciB3YXlzIGl0IHdhbnRzCiAgICB1bmRl ciBpdHMgb3duIGxvY2tpbmcuCiAgLSBETUEgcHJvdmlkZSBBUEkgdG8gZmx1c2ggdGxiL2NhY2hl IG9mIGEgcmFuZ2Ugb2YgcGh5c2ljYWwgYnVzIGFkZHJlc3MKICAgIHdpdGggdGhlIGV4cGxpY2l0 IGNvbnRyYWN0IHRoYXQgaXQgY2FuIG9ubHkgdGFrZSBzaG9ydCBsaXZlZCBzcGlubG9jawogICAg dG8gZG8gc28gYW5kIG5vdCBhbGxvY2F0ZSBvciBmcmVlIG1lbW9yeSB3aGlsZSBkb2luZyBzbyAo aWUgaXQgc2hvdWxkCiAgICBvbmx5IGJlIGEgYnVuY2ggb2YgcmVnaXN0ZXJzIHdyaXRlcykKCkkg aGF2ZW4ndCBoYWQgdGltZSB0byBwcm90b3R5cGUgdGhpcyBidXQgaSBmbG9hdGVkIHRoZSBpZGVh IGEgd2hpbGUgYmFjawphbmQgaSBhbSBob3BpbmcgdG8gZGlzY3VzcyBpdCBzb21lIG1vcmUgc29v bmVyIHJhdGhlciB0aGFuIGxhdHRlciAoTFNGL01NCmlzIGNvbWluZyBzb29uKS4KCgpTbyBmcm9t IGEgc2FmZXR5IHBvaW50IG9mIHZpZXcgaSB3b3VsZCBhcmd1ZSB0aGF0IHlvdSBtdXN0IGJlIGRv aW5nIGRtYV91bm1hcCgpCmZyb20gaW52YWxpZGF0ZV9yYW5nZV9zdGFydCgpIGNhbGxiYWNrLiBC dXQgaXQgYWxsIGRlcGVuZHMgb24gaG93IGNvbmZpZGVudAp5b3UgYXJlIGluIHlvdXIgaGFyZHdh cmUgbW11IGFuZCB0aGUgSU9NTVUgKElPTU1VIHVzdWFseSBoYXZlIHNtYWxsIGNhY2hlIHNvCnVu bGVzcyB0aGVyZSBpcyBubyBQQ0lFIHRyYWZmaWMgYWxsIHBlbmRpbmcgUENJRSB0cmFuc2FjdGlv biBzaG91bGQgcG9zdApxdWlja2VyIHRoYW4gaXQgdGFrZXMgdGhlIENQVSB0byBmaW5pc2ggeW91 ciBpbnZhbGlkYXRlX3JhbmdlX3N0YXJ0KCkgZnVuY3Rpb24uCgpTbyBmb3Igbm93IGkgd291bGQg c2F5IGl0IGlzIG9rIHRvIG5vdCBkbWFfdW5tYXAoKS4KCgo+ID4gVGhpbmtpbmcgYSBtb21lbnQg YWJvdXQgdGhhdCBpdCBhY3R1YWxseSBzZWVtcyB0byBtYWtlIHBlcmZlY3Qgc2Vuc2UuCj4gPiBT byBpdCBkb2Vzbid0IG1hdHRlciB3aGF0IG9yZGVyIHlvdSBnb3QgYmV0d2VlbiB0aGUgbW1hcF9z ZW0gYW5kIHlvdXIgYnVmZmVyCj4gPiBvciBhbGxvY2F0aW9uIGxvY2ssIGl0IHdpbGwgc2ltcGx5 IGJlIGluY29ycmVjdCB3aXRoIG90aGVyIGxvY2tzIGluIHRoZQo+ID4gc3lzdGVtIGFueXdheS4K PiAKPiBIbSwgZG9lc24ndCBtYWtlIHNlbnNlIHRvIG1lLCBhdCBsZWFzdCBmcm9tIGEgbG9ja2lu ZyAgaW52ZXJzaW9uIHBvdi4gSQo+IHRob3VnaHQgdGhlIG9ubHkgbG9ja3MgdGhhdCBhcmUgZGVm aW5pdGVseSBwYXJ0IG9mIHRoZSBtbXVfbm9maXRpZXIKPiBjYWxsYmFjayBjb250ZXh0cyBhcmUg dGhlIG1tIGxvY2tzLiBXZSdyZSBkZWZpbml0ZWx5IGZpbmUgd2l0aCB0aG9zZSwgYW5kCj4ga21h bGxvYyBkaWRuJ3QgYml0ZSB1cyB5ZXQuIERpcmVjdCByZWNsYWltIGlzIHJlYWxseSBsaW1pdGVk IGluIHdoYXQgaXQKPiBjYW4gYWNjb21wbGlzaCBmb3IgZ29vZCByZWFzb25zLgo+IAo+IFNvIHRo ZSBvbmx5IHRoaW5nIEknbSB3b3JyaWVkIGhlcmUgaXMgdGhhdCB3ZSBoYXZlIGEgcmFjZSBiZXR3 ZWVuIHRoZQo+IGludmFsaWRhdGUgYW5kIHRoZSBndXAgY2FsbHMsIGFuZCBJIHRoaW5rIGZpeGlu ZyB0aGF0IHJhY2Ugd2lsbCBtYWtlIHRoZQo+IGxvY2tpbmcgbW9yZSBmdW4uIEJ1dCBhZ2Fpbiwg aWYgdGhhdCBtZWFucyB5b3UgY2FuJ2QgZG1hX3VubWFwIHN0dWZmIHRoZW4KPiB0aGF0IGZlZWxz IGxpa2UgbW11IG5vdGlmaWVycyBhcmUgYnJva2VuIGJ5IGRlc2lnbi4KPiAtRGFuaWVsCj4gCj4g PiBUaGUgb25seSBzb2x1dGlvbiB0byB0aGF0IHByb2JsZW0gd2UgaGF2ZSBmb3VuZCBpcyB0aGUg ZGFuY2Ugd2UgaGF2ZSBpbgo+ID4gYW1kZ3B1IG5vdzoKPiA+IAo+ID4gMS4gSW5zaWRlIGludmFs aWRhdGVfcmFuZ2Vfc3RhcnQgeW91IGdyYWIgYSByZWN1cnNpdmUgcmVhZCBzaWRlIGxvY2suCj4g PiAyLiBXYWl0IGZvciBhbGwgR1BVIG9wZXJhdGlvbiBvbiB0aGF0IHBhZ2VzIHRvIGZpbmlzaC4K PiA+IDMuIFRoZW4gbWFyayBhbGwgcGFnZXMgYXMgZGlydHkgYW5kIGFjY2Vzc2VkLgo+ID4gNC4g SW5zaWRlIGludmFsaWRhdGVfcmFuZ2VfZW5kIHlvdSByZWxlYXNlIHlvdXIgcmVjdXJzaXZlIHJl YWQgc2lkZSBsb2NrCj4gPiBhZ2Fpbi4KPiA+IAo+ID4gVGhlbiBkdXJpbmcgY29tbWFuZCBzdWJt aXNzaW9uIHlvdSBkbyB0aGUgZm9sbG93aW5nOgo+ID4gCj4gPiAxLiBUYWtlIHRoZSBsb2NrcyBG aWd1cmUgb3V0IGFsbCB0aGUgdXNlcnB0ciBCT3Mgd2hpY2ggbmVlZHMgbmV3IHBhZ2VzLgo+ID4g Mi4gRHJvcCB0aGUgbG9ja3MgYWdhaW4gY2FsbCBnZXRfdXNlcl9wYWdlcygpLgo+ID4gMy4gSW5z dGFsbCB0aGUgbmV3IHBhZ2UgYXJyYXlzIGFuZCByZWl0ZXJhdGUgd2l0aCAjMQo+ID4gNC4gQXMg c29vbiBhcyBldmVyeWJvZHkgaGFzIHBhZ2VzIHVwZGF0ZSB5b3VyIHBhZ2UgdGFibGVzIGFuZCBw cmVwYXJlIGFsbAo+ID4gY29tbWFuZCBzdWJtaXNzaW9uLgo+ID4gNS4gVGFrZSB0aGUgd3JpdGUg c2lkZSBvZiBvdXIgcmVjdXJzaXZlIGxvY2suCj4gPiA2LiBDaGVjayBpZiBhbGwgcGFnZXMgYXJl IHN0aWxsIHZhbGlkLCBpZiBub3QgcmVzdGFydCB0aGUgd2hvbGUgSU9DVEwuCj4gPiA3LiBTdWJt aXQgdGhlIGpvYiB0byB0aGUgaGFyZHdhcmUuCj4gPiA4LiBEcm9wIHRoZSB3cml0ZSBzaWRlIG9m IG91ciByZWN1cnNpdmUgbG9jay4KCkEgc2xpZ2h0bHkgYmV0dGVyIHNvbHV0aW9uIGlzIHVzaW5n IGF0b21pYyBjb3VudGVyOgogIGRyaXZlcl9yYW5nZV9zdGFydCgpIHsKICAgIGF0b21pY19pbmMo Jm15ZGV2LT5ub3RpZmllcl9jb3VudCk7CiAgICBsb2NrKG15ZGV2LT5ib191cHRyKTsgLy8gcHJv dGVjdGluZyBibyBsaXN0L3RyZWUKICAgIGZvcl9lYWNoX3VwdHJfYm8oKSB7CiAgICAgICBpbnZh bGlkYXRlX2JvKGJvLCBzdGFydCwgZW5kKTsKICAgIH0KICAgIHVubG9jayhteWRldi0+Ym9fdXB0 cik7CiAgfQoKICBkcml2ZXJfcmFuZ2VfZW5kKCkgewogICAgYXRvbWljX2luYygmbXlkZXYtPm5v dGlmaWVyX3NlcSk7CiAgICBhdG9taWNfZGVjKCZteWRldi0+bm90aWZpZXJfY291bnQpOwogIH0K CiAgZHJpdmVyX2JvX2d1cF9wYWdlcyhibywgbXlkZXYpIHsKICAgIHVuc2lnbmVkIGN1cnNlcTsK ICAgIGRvIHsKICAgICAgd2FpdF9vbl9tbXVfbm90aWZpZXJfY291bnRfdG9femVybygpOwogICAg ICBjdXJzZXEgPSBhdG9taWNfcmVhZCgmbXlkZXYtPm5vdGlmaWVyX3NlcSk7CiAgICAgIGZvciAo aSA9IDA7IGkgPCBucGFnZXM7IGkrKykgewogICAgICAgIHBhZ2VzW2ldID0gZ3VwKGFkZHIgKyAo aSA8PCBQQUdFX1NISUZUKSk7CiAgICAgIH0KICAgICAgbG9jayhteWRldi0+Ym9fdXB0cik7IC8v IHByb3RlY3RpbmcgYm8gbGlzdC90cmVlCiAgICAgIGlmIChjdXJzZXEgPT0gYXRvbWljX3JlYWQo Jm15ZGV2LT5ub3RpZmllcl9zZXEpICYmCiAgICAgICAgICAhYXRvbWljX3JlYWQobXlkZXYtPm5v dGlmaWVyX2NvdW50KSkgewogICAgICAgICBhZGRfdXB0cl9ibyhibywgcGFnZXMpOwogICAgICAg ICBiby0+c2VxID0gY3Vyc2VxOwogICAgICAgICB1bmxvY2sobXlkZXYtPmJvX3VwdHIpOwogICAg ICAgICByZXR1cm4gMDsKICAgICAgfQogICAgICB1bmxvY2sobXlkZXYtPmJvX3VwdHIpOwogICAg ICBwdXRfYWxsX3BhZ2VzKHBhZ2VzKTsKICAgIH0gd2hpbGUgKDEpOwogIH0KCk1heWJlIHVzZSBL Vk0gYXMgYSByZWFsIGV4YW1wbGUgb2YgaG93IHRvIGRvIHRoaXMgKHNlZSB2aXJ0L2t2bS9rdm1f bWFpbi5jKQoKTm90ZSB0aGF0IGhvd2V2ZXIgbXkgZW5kZ2FtZSBpcyBiaXQgbW9yZSBzaW1wbGVy LiBDb252ZXJ0IGFtZCwgaW50ZWwgdG8gdXNlCkhNTSBDUFUgcGFnZSB0YWJsZSBzbmFwc2hvdCB0 b29sIChhbmQgbm8geW91IGRvIG5vdCBuZWVkIHBhZ2UgZmF1bHQgZm9yIHRoYXQKeW91IGNhbiB1 c2UgdGhhdCBmZWF0dXJlIGluZGVwZW5kZW50bHkpLgoKU28gaXQgd291bGQgYmU6CiAgZHJpdmVy X2JvX2htbV9pbnZhbGlkYXRlKCkgewogICAgbG9jayhteWRldi0+Ym9fdXB0cik7IC8vIHByb3Rl Y3RpbmcgYm8gbGlzdC90cmVlCiAgICBmb3JfZWFjaF91cHRyX2JvKCkgewogICAgICAgaW52YWxp ZGF0ZV9ibyhibywgc3RhcnQsIGVuZCk7CiAgICB9CiAgICB1bmxvY2sobXlkZXYtPmJvX3VwdHIp OwogIH0KCiAgZHJpdmVyX2JvX2d1cF9wYWdlcyhibywgbXlkZXYpIHsKICAgIGRvIHsKICAgICAg aG1tX3ZtYV9mYXVsdChyYW5nZSwgYm8tPnN0YXJ0LCBiby0+ZW5kLCBiby0+cGZucyk7CiAgICAg IGxvY2sobXlkZXYtPmJvX3VwdHIpOwogICAgICBpZiAoaG1tX3ZtYV9yYW5nZV9kb25lKHJhbmdl KSkgewogICAgICAgICBhZGRfdXB0cl9ibyhibywgcGZucyk7CiAgICAgIH0KICAgICAgdW5sb2Nr KG15ZGV2LT5ib191cHRyKTsKICAgIH0gd2hpbGUgKDEpOwogIH0KCkkgd291bGQgbGlrZSB0byBz ZWUgZHJpdmVyIHVzaW5nIHNhbWUgY29kZSwgYXMgaXQgbWVhbnMgb25lIHBsYWNlIHRvIGZpeApp c3N1ZXMuIEkgaGFkIGZvciBhIGxvbmcgdGltZSBvbiBteSBUT0RPIGxpc3QgZG9pbmcgdGhlIGFi b3ZlIGNvbnZlcnNpb24KdG8gYW1kIG9yIHJhZGVvbiBrZXJuZWwgZHJpdmVyLiBJIGFtIHB1c2hp bmcgdXAgbXkgdG9kbyBsaXN0IGhvcGVmdWxseSBpbgpuZXh0IGZldyB3ZWVrcyBpIGNhbiBzZW5k IGFuIHJmYyBzbyBwZW9wbGUgY2FuIGhhdmUgYSByZWFsIHNlbnNlIG9mIGhvdwppdCBjYW4gbG9v ay4KCgpDaGVlcnMsCkrDqXLDtG1lCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fCmFtZC1nZnggbWFpbGluZyBsaXN0CmFtZC1nZnhAbGlzdHMuZnJlZWRlc2t0 b3Aub3JnCmh0dHBzOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vYW1k LWdmeAo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from mail-qt0-f178.google.com ([209.85.216.178]:44747 "EHLO mail-qt0-f178.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751868AbeCZPme (ORCPT ); Mon, 26 Mar 2018 11:42:34 -0400 Received: by mail-qt0-f178.google.com with SMTP id j26so20005369qtl.11 for ; Mon, 26 Mar 2018 08:42:33 -0700 (PDT) Date: Mon, 26 Mar 2018 11:42:24 -0400 From: Jerome Glisse To: Daniel Vetter Cc: christian.koenig@amd.com, Daniel Vetter , amd-gfx list , "moderated list:DMA BUFFER SHARING FRAMEWORK" , dri-devel , "open list:DMA BUFFER SHARING FRAMEWORK" Subject: Re: [Linaro-mm-sig] [PATCH 1/5] dma-buf: add optional invalidate_mappings callback v2 Message-ID: <20180326154224.GA11930@gmail.com> References: <0bd85f69-c64c-70d1-a4a0-10ae0ed8b4e8@gmail.com> <19ed21a5-805d-271f-9120-49e0c00f510f@amd.com> <20180320140810.GU14155@phenom.ffwll.local> <37ba7394-2a5c-a0bc-cc51-c8a0edc2991d@gmail.com> <20180321082839.GA14155@phenom.ffwll.local> <327c4bc1-5813-16e8-62fc-4301b19a1a22@gmail.com> <20180322071804.GH14155@phenom.ffwll.local> <20180326080121.GO14155@phenom.ffwll.local> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20180326080121.GO14155@phenom.ffwll.local> Sender: linux-media-owner@vger.kernel.org List-ID: On Mon, Mar 26, 2018 at 10:01:21AM +0200, Daniel Vetter wrote: > On Thu, Mar 22, 2018 at 10:58:55AM +0100, Christian König wrote: > > Am 22.03.2018 um 08:18 schrieb Daniel Vetter: > > > On Wed, Mar 21, 2018 at 12:54:20PM +0100, Christian König wrote: > > > > Am 21.03.2018 um 09:28 schrieb Daniel Vetter: > > > > > On Tue, Mar 20, 2018 at 06:47:57PM +0100, Christian König wrote: > > > > > > Am 20.03.2018 um 15:08 schrieb Daniel Vetter: > > > > > > > [SNIP] > > > > > > > For the in-driver reservation path (CS) having a slow-path that grabs a > > > > > > > temporary reference, drops the vram lock and then locks the reservation > > > > > > > normally (using the acquire context used already for the entire CS) is a > > > > > > > bit tricky, but totally feasible. Ttm doesn't do that though. > > > > > > That is exactly what we do in amdgpu as well, it's just not very efficient > > > > > > nor reliable to retry getting the right pages for a submission over and over > > > > > > again. > > > > > Out of curiosity, where's that code? I did read the ttm eviction code way > > > > > back, and that one definitely didn't do that. Would be interesting to > > > > > update my understanding. > > > > That is in amdgpu_cs.c. amdgpu_cs_parser_bos() does a horrible dance with > > > > grabbing, releasing and regrabbing locks in a loop. > > > > > > > > Then in amdgpu_cs_submit() we grab an lock preventing page table updates and > > > > check if all pages are still the one we want to have: > > > > >         amdgpu_mn_lock(p->mn); > > > > >         if (p->bo_list) { > > > > >                 for (i = p->bo_list->first_userptr; > > > > >                      i < p->bo_list->num_entries; ++i) { > > > > >                         struct amdgpu_bo *bo = p->bo_list->array[i].robj; > > > > > > > > > >                         if > > > > > (amdgpu_ttm_tt_userptr_needs_pages(bo->tbo.ttm)) { > > > > >                                 amdgpu_mn_unlock(p->mn); > > > > >                                 return -ERESTARTSYS; > > > > >                         } > > > > >                 } > > > > >         } > > > > If anything changed on the page tables we restart the whole IOCTL using > > > > -ERESTARTSYS and try again. > > > I'm not talking about userptr here, but general bo eviction. Sorry for the > > > confusion. > > > > > > The reason I'm dragging all the general bo management into this > > > discussions is because we do seem to have fairly fundamental difference in > > > how that's done, with resulting consequences for the locking hierarchy. > > > > > > And if this invalidate_mapping stuff should work, together with userptr > > > and everything else, I think we're required to agree on how this is all > > > supposed to nest, and how exactly we should back off for the other side > > > that needs to break the locking circle. > > > > > > That aside, I don't entirely understand why you need to restart so much. I > > > figured that get_user_pages is ordered correctly against mmu > > > invalidations, but I get the impression you think that's not the case. How > > > does that happen? > > > > Correct. I've had the same assumption, but both Jerome as well as our > > internal tests proved me wrong on that. > > > > Key take away from that was that you can't take any locks from neither the > > MMU notifier nor the shrinker you also take while calling kmalloc (or > > simpler speaking get_user_pages()). > > > > Additional to that in the MMU or shrinker callback all different kinds of > > locks might be held, so you basically can't assume that you do thinks like > > recursive page table walks or call dma_unmap_anything. > > That sounds like a design bug in mmu_notifiers, since it would render them > useless for KVM. And they were developed for that originally. I think I'll > chat with Jerome to understand this, since it's all rather confusing. Doing dma_unmap() during mmu_notifier callback should be fine, it was last time i check. However there is no formal contract that it is ok to do so. But i want to revisit DMA API to something where device driver get control rather than delegate. High level ideas are: - device driver allocate DMA space ie range of physical bus address - device driver completely manage the range in whichever ways it wants under its own locking. - DMA provide API to flush tlb/cache of a range of physical bus address with the explicit contract that it can only take short lived spinlock to do so and not allocate or free memory while doing so (ie it should only be a bunch of registers writes) I haven't had time to prototype this but i floated the idea a while back and i am hoping to discuss it some more sooner rather than latter (LSF/MM is coming soon). So from a safety point of view i would argue that you must be doing dma_unmap() from invalidate_range_start() callback. But it all depends on how confident you are in your hardware mmu and the IOMMU (IOMMU usualy have small cache so unless there is no PCIE traffic all pending PCIE transaction should post quicker than it takes the CPU to finish your invalidate_range_start() function. So for now i would say it is ok to not dma_unmap(). > > Thinking a moment about that it actually seems to make perfect sense. > > So it doesn't matter what order you got between the mmap_sem and your buffer > > or allocation lock, it will simply be incorrect with other locks in the > > system anyway. > > Hm, doesn't make sense to me, at least from a locking inversion pov. I > thought the only locks that are definitely part of the mmu_nofitier > callback contexts are the mm locks. We're definitely fine with those, and > kmalloc didn't bite us yet. Direct reclaim is really limited in what it > can accomplish for good reasons. > > So the only thing I'm worried here is that we have a race between the > invalidate and the gup calls, and I think fixing that race will make the > locking more fun. But again, if that means you can'd dma_unmap stuff then > that feels like mmu notifiers are broken by design. > -Daniel > > > The only solution to that problem we have found is the dance we have in > > amdgpu now: > > > > 1. Inside invalidate_range_start you grab a recursive read side lock. > > 2. Wait for all GPU operation on that pages to finish. > > 3. Then mark all pages as dirty and accessed. > > 4. Inside invalidate_range_end you release your recursive read side lock > > again. > > > > Then during command submission you do the following: > > > > 1. Take the locks Figure out all the userptr BOs which needs new pages. > > 2. Drop the locks again call get_user_pages(). > > 3. Install the new page arrays and reiterate with #1 > > 4. As soon as everybody has pages update your page tables and prepare all > > command submission. > > 5. Take the write side of our recursive lock. > > 6. Check if all pages are still valid, if not restart the whole IOCTL. > > 7. Submit the job to the hardware. > > 8. Drop the write side of our recursive lock. A slightly better solution is using atomic counter: driver_range_start() { atomic_inc(&mydev->notifier_count); lock(mydev->bo_uptr); // protecting bo list/tree for_each_uptr_bo() { invalidate_bo(bo, start, end); } unlock(mydev->bo_uptr); } driver_range_end() { atomic_inc(&mydev->notifier_seq); atomic_dec(&mydev->notifier_count); } driver_bo_gup_pages(bo, mydev) { unsigned curseq; do { wait_on_mmu_notifier_count_to_zero(); curseq = atomic_read(&mydev->notifier_seq); for (i = 0; i < npages; i++) { pages[i] = gup(addr + (i << PAGE_SHIFT)); } lock(mydev->bo_uptr); // protecting bo list/tree if (curseq == atomic_read(&mydev->notifier_seq) && !atomic_read(mydev->notifier_count)) { add_uptr_bo(bo, pages); bo->seq = curseq; unlock(mydev->bo_uptr); return 0; } unlock(mydev->bo_uptr); put_all_pages(pages); } while (1); } Maybe use KVM as a real example of how to do this (see virt/kvm/kvm_main.c) Note that however my endgame is bit more simpler. Convert amd, intel to use HMM CPU page table snapshot tool (and no you do not need page fault for that you can use that feature independently). So it would be: driver_bo_hmm_invalidate() { lock(mydev->bo_uptr); // protecting bo list/tree for_each_uptr_bo() { invalidate_bo(bo, start, end); } unlock(mydev->bo_uptr); } driver_bo_gup_pages(bo, mydev) { do { hmm_vma_fault(range, bo->start, bo->end, bo->pfns); lock(mydev->bo_uptr); if (hmm_vma_range_done(range)) { add_uptr_bo(bo, pfns); } unlock(mydev->bo_uptr); } while (1); } I would like to see driver using same code, as it means one place to fix issues. I had for a long time on my TODO list doing the above conversion to amd or radeon kernel driver. I am pushing up my todo list hopefully in next few weeks i can send an rfc so people can have a real sense of how it can look. Cheers, Jérôme