From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jacob Pan Subject: Re: [PATCH v2 01/40] iommu: Introduce Shared Virtual Addressing API Date: Fri, 7 Sep 2018 14:25:04 -0700 Message-ID: <20180907142504.5034351e@jacob-builder> References: <20180511190641.23008-1-jean-philippe.brucker@arm.com> <20180511190641.23008-2-jean-philippe.brucker@arm.com> <03d31ba5-1eda-ea86-8c0c-91d14c86fe83@arm.com> <2fd4a0a1-1a35-bf82-df84-b995cce011d9@amd.com> <65e7accd-4446-19f5-c667-c6407e89cfa6@arm.com> <5bbc0332-b94b-75cc-ca42-a9b196811daf@amd.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: <5bbc0332-b94b-75cc-ca42-a9b196811daf-5C7GfCeVMHo@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org To: Christian =?UTF-8?B?S8O2bmln?= Cc: "xieyisheng1-hv44wF8Li93QT0dZR+AlfA@public.gmane.org" , "ilias.apalodimas-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org" , "kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , Will Deacon , Michal Hocko , "okaya-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org" , "linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org" , "ashok.raj-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org" , Jean-Philippe Brucker , "linux-acpi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "rfranz-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org" , "devicetree-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "kevin.tian-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org" , "rgummal-gjFFaj9aHVfQT0dZR+AlfA@public.gmane.org" , "linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org" , "dwmw2-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org" , "liubo95-hv44wF8Li93QT0dZR+AlfA@public.gmane.org" List-Id: linux-acpi@vger.kernel.org T24gRnJpLCA3IFNlcCAyMDE4IDIwOjAyOjU0ICswMjAwCkNocmlzdGlhbiBLw7ZuaWcgPGNocmlz dGlhbi5rb2VuaWdAYW1kLmNvbT4gd3JvdGU6Cgo+IEFtIDA3LjA5LjIwMTggdW0gMTc6NDUgc2No cmllYiBKZWFuLVBoaWxpcHBlIEJydWNrZXI6Cj4gPiBPbiAwNy8wOS8yMDE4IDA5OjU1LCBDaHJp c3RpYW4gS8O2bmlnIHdyb3RlOiAgCj4gPj4gSSB3aWxsIHRha2UgdGhpcyBhcyBhbiBvcHBvcnR1 bml0eSB0byBzdW1tYXJpemUgc29tZSBvZiB0aGUKPiA+PiByZXF1aXJlbWVudHMgd2UgaGF2ZSBm b3IgUEFTSUQgbWFuYWdlbWVudCBmcm9tIHRoZSBhbWRncHUgZHJpdmVyCj4gPj4gcG9pbnQgb2Yg dmlldzogIAo+ID4gVGhhdCdzIGluY3JlZGlibHkgdXNlZnVsLCB0aGFua3MgOikKPiA+ICAKPiA+ PiAxLiBXZSBuZWVkIHRvIGJlIGFibGUgdG8gYWxsb2NhdGUgUEFTSUQgYmV0d2VlbiAxIGFuZCBz b21lCj4gPj4gbWF4aW11bS4gWmVybyBpcyByZXNlcnZlZCBhcyBmYXIgYXMgSSBrbm93LCBidXQg d2UgZG9uJ3QgbmVjZXNzYXJ5Cj4gPj4gbmVlZCBhIG1pbmltdW0uICAKPiAgWy4uLl0gIAo+ID4+ IDIuIFdlIG5lZWQgdG8gYmUgYWJsZSB0byBhbGxvY2F0ZSBQQVNJRHMgd2l0aG91dCBhIHByb2Nl c3MgYWRkcmVzcwo+ID4+IHNwYWNlIGJhY2tpbmcgaXQuIEUuZy4gb3VyIGhhcmR3YXJlIHVzZXMg UEFTSURzIGV2ZW4gd2l0aG91dAo+ID4+IFNoYXJlZCBWaXJ0dWFsIEFkZHJlc3NpbmcgZW5hYmxl ZCB0byBkaXN0aW5jdCBjbGllbnRzIGZyb20gZWFjaAo+ID4+IG90aGVyLiBXb3VsZCBiZSBhIHBp dHkgaWYgd2UgbmVlZCB0byBzdGlsbCBoYXZlIGEgc2VwYXJhdGUgUEFTSUQKPiA+PiBoYW5kbGlu ZyBiZWNhdXNlIHRoZSBzeXN0ZW0gd2lkZSBpcyBvbmx5IGF2YWlsYWJsZSB3aGVuIElPTU1VIGlz Cj4gPj4gdHVybmVkIG9uLiAgCj4gIFsuLi5dICAKPiAKPiBJIGFncmVlIG9uIHRoYXQuCj4gCj4g PiBpb21tdS1zdmEgZXhwZWN0cyBldmVyeXdoZXJlIHRoYXQgdGhlIGRldmljZSBoYXMgYW4gaW9t bXVfZG9tYWluLAo+ID4gaXQncyB0aGUgZmlyc3QgdGhpbmcgd2UgY2hlY2sgb24gZW50cnkuIEJ5 cGFzc2luZyBhbGwgb2YgdGhpcyB3b3VsZAo+ID4gY2FsbCBpZHJfYWxsb2MoKSBkaXJlY3RseSwg YW5kIHdvdWxkbid0IGhhdmUgYW55IGNvZGUgaW4gY29tbW9uCj4gPiB3aXRoIHRoZSBjdXJyZW50 IGlvbW11LXN2YS4gU28gaXQgc2VlbXMgbGlrZSB5b3UgbmVlZCBhIGxheWVyIG9uCj4gPiB0b3Ag b2YgaW9tbXUtc3ZhIGNhbGxpbmcgaWRyX2FsbG9jKCkgd2hlbiBhbiBJT01NVSBpc24ndCBwcmVz ZW50LAo+ID4gYnV0IEkgZG9uJ3QgdGhpbmsgaXQgc2hvdWxkIGJlIGluIGRyaXZlcnMvaW9tbXUv ICAKPiAKPiBJbiB0aGlzIGNhc2UgSSBxdWVzdGlvbiBpZiB0aGUgUEFTSUQgaGFuZGxpbmcgc2hv dWxkIGJlIHVuZGVyIAo+IGRyaXZlcnMvaW9tbXUgYXQgYWxsLgo+IAo+IFNlZSBJIGNhbiBoYXZl IGEgbWl4IG9mIFZNIGNvbnRleHQgd2hpY2ggYXJlIGJvdW5kIHRvIHByb2Nlc3NlcyAoc29tZSAK PiBmZXcpIGFuZCBWTSBjb250ZXh0cyB3aGljaCBhcmUgc3RhbmRhbG9uZSBhbmQgZG9lc24ndCBj YXJlIGZvciBhCj4gcHJvY2VzcyBhZGRyZXNzIHNwYWNlLiBCdXQgZm9yIGVhY2ggVk0gY29udGV4 dCBJIG5lZWQgYSBkaXN0aW5jdAo+IFBBU0lEIGZvciB0aGUgaGFyZHdhcmUgdG8gd29yay4KPiAK PiBJIGNhbiBsaXZlIGlmIHdlIHNheSBpZiBJT01NVSBpcyBjb21wbGV0ZWx5IGRpc2FibGVkIHdl IHVzZSBhIHNpbXBsZQo+IGlkYSB0byBhbGxvY2F0ZSB0aGVtLCBidXQgd2hlbiBJT01NVSBpcyBl bmFibGVkIEkgY2VydGFpbmx5IG5lZWQgYQo+IHdheSB0byByZXNlcnZlIGEgUEFTSUQgd2l0aG91 dCBhbiBhc3NvY2lhdGVkIHByb2Nlc3MuCj4gClZULWQgd291bGQgYWxzbyBoYXZlIHN1Y2ggcmVx dWlyZW1lbnQuIFRoZXJlIGlzIGEgdmlydHVhbCBjb21tYW5kCnJlZ2lzdGVyIGZvciBhbGxvY2F0 ZSBhbmQgZnJlZSBQQVNJRCBmb3IgVk0gdXNlLiBXaGVuIHRoYXQgUEFTSUQKYWxsb2NhdGlvbiBy ZXF1ZXN0IGdldHMgcHJvcGFnYXRlZCB0byB0aGUgaG9zdCBJT01NVSBkcml2ZXIsIHdlIG5lZWQg dG8KYWxsb2NhdGUgUEFTSUQgdy9vIG1tLgoKSWYgdGhlIFBBU0lEIGFsbG9jYXRpb24gaXMgZG9u ZSB2aWEgVkZJTywgY2FuIHdlIGhhdmUgRkQgdG8gdHJhY2sgUEFTSUQKbGlmZSBjeWNsZSBpbnN0 ZWFkIG9mIG1tX2V4aXQoKT8gaS5lLiBhbGwgRkRzIGdldCBjbG9zZWQgYmVmb3JlCm1tX2V4aXQs IEkgYXNzdW1lPwoKPiA+PiAzLiBFdmVuIGFmdGVyIGRlc3RydWN0aW9uIG9mIGEgcHJvY2VzcyBh ZGRyZXNzIHNwYWNlIHdlIG5lZWQgc29tZQo+ID4+IGdyYWNlIHBlcmlvZCBiZWZvcmUgYSBQQVNJ RCBpcyByZXVzZWQgYmVjYXVzZSBpdCBjYW4gYmUgdGhhdCB0aGUKPiA+PiBzcGVjaWZpYyBQQVNJ RCBpcyBzdGlsbCBpbiBzb21lIGhhcmR3YXJlIHF1ZXVlcyBldGMuLi4KPiA+PiAgIMKgwqDCoCDC oMKgwqAgQXQgYmFyZSBtaW5pbXVtIGFsbCBkZXZpY2UgZHJpdmVycyB1c2luZyBwcm9jZXNzIGJp bmRpbmcKPiA+PiBuZWVkIHRvIGV4cGxpY2l0bHkgbm90ZSB0byB0aGUgY29yZSB3aGVuIHRoZXkg YXJlIGRvbmUgd2l0aCBhCj4gPj4gUEFTSUQuICAKPiA+IFJpZ2h0LCBtdWNoIG9mIHRoZSBob3Jy aWJsZW5lc3MgaW4gaW9tbXUtc3ZhIGRlYWxzIHdpdGggdGhpczoKPiA+Cj4gPiBUaGUgcHJvY2Vz cyBkaWVzLCBpb21tdS1zdmEgaXMgbm90aWZpZWQgYW5kIGNhbGxzIHRoZSBtbV9leGl0KCkKPiA+ IGZ1bmN0aW9uIHBhc3NlZCBieSB0aGUgZGV2aWNlIGRyaXZlciB0byBpb21tdV9zdmFfZGV2aWNl X2luaXQoKS4gSW4KPiA+IG1tX2V4aXQoKSB0aGUgZGV2aWNlIGRyaXZlciBuZWVkcyB0byBjbGVh ciBhbnkgcmVmZXJlbmNlIHRvIHRoZQo+ID4gUEFTSUQgaW4gaGFyZHdhcmUgYW5kIGluIGl0cyBv d24gc3RydWN0dXJlcy4gV2hlbiB0aGUgZGV2aWNlIGRyaXZlcgo+ID4gcmV0dXJucyBmcm9tIG1t X2V4aXQoKSwgaXQgZWZmZWN0aXZlbHkgdGVsbHMgdGhlIGNvcmUgdGhhdCBpdCBoYXMKPiA+IGZp bmlzaGVkIHVzaW5nIHRoZSBQQVNJRCwgYW5kIGlvbW11LXN2YSBjYW4gcmV1c2UgdGhlIFBBU0lE IGZvcgo+ID4gYW5vdGhlciBwcm9jZXNzLiBtbV9leGl0KCkgaXMgYWxsb3dlZCB0byBibG9jaywg c28gdGhlIGRldmljZQo+ID4gZHJpdmVyIGhhcyB0aW1lIHRvIGNsZWFuIHVwIGFuZCBmbHVzaCB0 aGUgcXVldWVzLgo+ID4KPiA+IElmIHRoZSBkZXZpY2UgZHJpdmVyIGZpbmlzaGVzIHVzaW5nIHRo ZSBQQVNJRCBiZWZvcmUgdGhlIHByb2Nlc3MKPiA+IGV4aXRzLCBpdCBqdXN0IGNhbGxzIHVuYmlu ZCgpLiAgCj4gCj4gRXhhY3RseSB0aGF0J3Mgd2hhdCBNaWNoYWwgSG9ja28gaXMgcHJvYmFibHkg Z29pbmcgdG8gbm90IGxpa2UgYXQgYWxsLgo+IAo+IENhbiB3ZSBoYXZlIGEgZGlmZmVyZW50IGFw cHJvYWNoIHdoZXJlIGVhY2ggZHJpdmVyIGlzIGluZm9ybWVkIGJ5IHRoZSAKPiBtbV9leGl0KCks IGJ1dCBuZWVkcyB0byBleHBsaWNpdGx5IGNhbGwgdW5iaW5kKCkgYmVmb3JlIGEgUEFTSUQgaXMK PiByZXVzZWQ/Cj4gCj4gRHVyaW5nIHRoYXQgdGVhcmRvd24gdHJhbnNpdGlvbiBpdCB3b3VsZCBi ZSBpZGVhbCBpZiB0aGF0IFBBU0lEIG9ubHkgCj4gcG9pbnRzIHRvIGEgZHVtbXkgcm9vdCBwYWdl IGRpcmVjdG9yeSB3aXRoIG9ubHkgaW52YWxpZCBlbnRyaWVzLgo+IApJIGd1ZXNzIHRoaXMgY2Fu IGJlIHZlbmRvciBzcGVjaWZpYywgSW4gVlQtZCBJIHBsYW4gdG8gbWFyayBQQVNJRAplbnRyeSBu b3QgcHJlc2VudCBhbmQgZGlzYWJsZSBmYXVsdCByZXBvcnRpbmcgd2hpbGUgZHJhaW5pbmcgcmVt YWluaW5nCmFjdGl2aXRpZXMuCgo+ID4gIAo+ID4+IDQuIEl0IHdvdWxkIGJlIG5pY2UgdG8gaGF2 ZSB0byBiZSBhYmxlIHRvIHNldCBhICJ2b2lkICoiIGZvciBlYWNoCj4gPj4gUEFTSUQvZGV2aWNl IGNvbWJpbmF0aW9uIHdoaWxlIGJpbmRpbmcgdG8gYSBwcm9jZXNzIHdoaWNoIHRoZW4gY2FuCj4g Pj4gYmUgcXVlcmllZCBsYXRlciBvbiBiYXNlZCBvbiB0aGUgUEFTSUQuCj4gPj4gICDCoMKgwqAg wqDCoMKgIEUuZy4gd2hlbiB5b3UgaGF2ZSBhIHBlciBQQVNJRC9kZXZpY2Ugc3RydWN0dXJlIGFy b3VuZAo+ID4+IGFueXdheSwganVzdCBhZGQgYW4gZXh0cmEgZmllbGQuICAKPiA+IGlvbW11X3N2 YV9iaW5kX2RldmljZSgpIHRha2VzIGEgImRydmRhdGEiIHBvaW50ZXIgdGhhdCBpcyBzdG9yZWQK PiA+IGludGVybmFsbHkgZm9yIHRoZSBQQVNJRC9kZXZpY2UgY29tYmluYXRpb24gKGlvbW11X2Jv bmQpLiBJdCBpcwo+ID4gcGFzc2VkIHRvIG1tX2V4aXQoKSwgYnV0IEkgaGF2ZW4ndCBhZGRlZCBh bnl0aGluZyBmb3IgdGhlIGRldmljZQo+ID4gZHJpdmVyIHRvIHF1ZXJ5IGl0IGJhY2suICAKPiAK PiBOaWNlISBMb29rcyBsaWtlIGFsbCB3ZSBuZWVkIGFkZGl0aW9uYWxseSBpcyBhIGZ1bmN0aW9u IHRvIHJldHJpZXZlCj4gdGhhdCBiYXNlZCBvbiB0aGUgUEFTSUQuCj4gCj4gPj4gNS4gSXQgd291 bGQgYmUgbmljZSB0byBoYXZlIHRvIGFsbG9jYXRlIG11bHRpcGxlIFBBU0lEcyBmb3IgdGhlCj4g Pj4gc2FtZSBwcm9jZXNzIGFkZHJlc3Mgc3BhY2UuCj4gPj4gICDCoMKgwqAgwqDCoMKgIEUuZy4g c29tZSB0ZWFtcyBhdCBBTUQgd2FudCB0byB1c2UgYSBzZXBhcmF0ZSBHUFUKPiA+PiBhZGRyZXNz IHNwYWNlIGZvciB0aGVpciB1c2Vyc3BhY2UgY2xpZW50IGxpYnJhcnkuIEknbSBzdGlsbCB0cnlp bmcKPiA+PiB0byBhdm9pZCB0aGF0LCBidXQgaXQgaXMgcGVyZmVjdGx5IHBvc3NpYmxlIHRoYXQg d2UgYXJlIGdvaW5nIHRvCj4gPj4gbmVlZCB0aGF0LiAgCj4gPiBUd28gUEFTSURzIHBvaW50aW5n IHRvIHRoZSBzYW1lIHByb2Nlc3MgcGdkPyBBdCBmaXJzdCBnbGFuY2UgaXQKPiA+IHNlZW1zIGZl YXNpYmxlLCBtYXliZSB3aXRoIGEgZmxhZyBwYXNzZWQgdG8gYmluZCgpIGFuZCBhIGZldwo+ID4g Y2hhbmdlcyB0byBpbnRlcm5hbCBzdHJ1Y3R1cmVzLiBJdCB3aWxsIGR1cGxpY2F0ZSBBVEMgaW52 YWxpZGF0aW9uCj4gPiBjb21tYW5kcyBmb3IgZWFjaCBwcm9jZXNzIGFkZHJlc3Mgc3BhY2UgY2hh bmdlIChtdW5tYXAgZXRjKSBzbyB5b3UKPiA+IG1pZ2h0IHRha2UgYSBwZXJmb3JtYW5jZSBoaXQu Cj4gPgo+ID4gSW50ZWwncyBTVk0gY29kZSBoYXMgdGhlIFNWTV9GTEFHX1BSSVZBVEVfUEFTSUQg d2hpY2ggc2VlbXMgc2ltaWxhcgo+ID4gdG8gd2hhdCB5b3UgZGVzY3JpYmUsIGJ1dCBJIGRvbid0 IHBsYW4gdG8gc3VwcG9ydCBpdCBpbiB0aGlzIHNlcmllcwo+ID4gKHRoZSBpb19tbSBtb2RlbCBp cyBhbHJlYWR5IHByZXR0eSBjb21wbGljYXRlZCkuIEkgdGhpbmsgaXQgY2FuIGJlCj4gPiBhZGRl ZCB3aXRob3V0IHRvbyBtdWNoIGVmZm9ydCBpbiBhIGZ1dHVyZSBzZXJpZXMsIHRob3VnaCB3aXRo IGEKPiA+IGRpZmZlcmVudCBmbGFnIG5hbWUgc2luY2Ugd2UnZCBsaWtlIHRvIHVzZSAicHJpdmF0 ZSBQQVNJRCIgZm9yCj4gPiBzb21ldGhpbmcgZWxzZQo+ID4gKGh0dHBzOi8vd3d3LnNwaW5pY3Mu bmV0L2xpc3RzL2RyaS1kZXZlbC9tc2cxNzcwMDcuaHRtbCkuICAKPiAKPiBUbyBiZSBob25lc3Qg SSBob3BlZCB0aGF0IHlvdSB3b3VsZCBzYXk6IE5vIG5ldmVyISBTbyB0aGF0IEkgaGF2ZSBhCj4g Z29vZCBhcmd1bWVudCB0byBwdXNoYmFjayBvbiBzdWNoIHJlcXVpcmVtZW50cyA6KQo+IAo+IEJ1 dCBpZiBpdCdzIGRvYWJsZSBpdCB3b3VsZCBiZSBhdCBsZWFzdCBuaWNlIHRvIGhhdmUgZm9yIGRl YnVnZ2luZy4KPiAKPiBUaGFua3MgYSBsb3QgZm9yIHdvcmtpbmcgb24gdGhhdCwKPiBDaHJpc3Rp YW4uCj4gCj4gPgo+ID4gVGhhbmtzLAo+ID4gSmVhbgo+ID4gIAo+ID4+ICAgwqDCoMKgIMKgwqDC oCBBZGRpdGlvbmFsIHRvIHRoYXQgaXQgaXMgc29tZXRpbWVzIHF1aXRlIHVzZWZ1bCBmb3IKPiA+ PiBkZWJ1Z2dpbmcgdG8gaXNvbGF0ZSB3aGVyZSBleGFjdGx5IGFuIGluY29ycmVjdCBhY2Nlc3Mg KHNlZ2ZhdWx0KQo+ID4+IGlzIGNvbWluZyBmcm9tLgo+ID4+Cj4gPj4gTGV0IG1lIGtub3cgaWYg dGhlcmUgYXJlIHNvbWUgcHJvYmxlbXMgd2l0aCB0aGF0LCBlc3BlY2lhbGx5IEkKPiA+PiB3YW50 IHRvIGtub3cgaWYgdGhlcmUgaXMgcHVzaGJhY2sgb24gIzUgc28gdGhhdCBJIGNhbiBmb3J3YXJk Cj4gPj4gdGhhdCA6KQo+ID4+Cj4gPj4gVGhhbmtzLAo+ID4+IENocmlzdGlhbi4KPiA+PiAgCj4g Pj4+IFRoYW5rcywKPiA+Pj4gSmVhbiAgCj4gCgpbSmFjb2IgUGFuXQpfX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwppb21tdSBtYWlsaW5nIGxpc3QKaW9tbXVA bGlzdHMubGludXgtZm91bmRhdGlvbi5vcmcKaHR0cHM6Ly9saXN0cy5saW51eGZvdW5kYXRpb24u b3JnL21haWxtYW4vbGlzdGluZm8vaW9tbXU= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Return-Path: Date: Fri, 7 Sep 2018 14:25:04 -0700 From: Jacob Pan To: Christian =?UTF-8?B?S8O2bmln?= Subject: Re: [PATCH v2 01/40] iommu: Introduce Shared Virtual Addressing API Message-ID: <20180907142504.5034351e@jacob-builder> In-Reply-To: <5bbc0332-b94b-75cc-ca42-a9b196811daf@amd.com> References: <20180511190641.23008-1-jean-philippe.brucker@arm.com> <20180511190641.23008-2-jean-philippe.brucker@arm.com> <03d31ba5-1eda-ea86-8c0c-91d14c86fe83@arm.com> <2fd4a0a1-1a35-bf82-df84-b995cce011d9@amd.com> <65e7accd-4446-19f5-c667-c6407e89cfa6@arm.com> <5bbc0332-b94b-75cc-ca42-a9b196811daf@amd.com> MIME-Version: 1.0 List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "xieyisheng1@huawei.com" , "ilias.apalodimas@linaro.org" , "kvm@vger.kernel.org" , "linux-pci@vger.kernel.org" , "xuzaibo@huawei.com" , "jonathan.cameron@huawei.com" , Will Deacon , Michal Hocko , "okaya@codeaurora.org" , "linux-mm@kvack.org" , "yi.l.liu@intel.com" , "ashok.raj@intel.com" , Jean-Philippe Brucker , "tn@semihalf.com" , "joro@8bytes.org" , "robdclark@gmail.com" , "bharatku@xilinx.com" , "linux-acpi@vger.kernel.org" , "liudongdong3@huawei.com" , "rfranz@cavium.com" , "devicetree@vger.kernel.org" , "kevin.tian@intel.com" , jacob.jun.pan@linux.intel.com, Auger Eric , "jcrouse@codeaurora.org" , "rgummal@xilinx.com" , "thunder.leizhen@huawei.com" , "linux-arm-kernel@lists.infradead.org" , "shunyong.yang@hxt-semitech.com" , "dwmw2@infradead.org" , "liubo95@huawei.com" , "alex.williamson@redhat.com" , "iommu@lists.linux-foundation.org" , Robin Murphy , "nwatters@codeaurora.org" , "baolu.lu@linux.intel.com" Content-Type: text/plain; charset="utf-8" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+bjorn=helgaas.com@lists.infradead.org List-ID: T24gRnJpLCA3IFNlcCAyMDE4IDIwOjAyOjU0ICswMjAwCkNocmlzdGlhbiBLw7ZuaWcgPGNocmlz dGlhbi5rb2VuaWdAYW1kLmNvbT4gd3JvdGU6Cgo+IEFtIDA3LjA5LjIwMTggdW0gMTc6NDUgc2No cmllYiBKZWFuLVBoaWxpcHBlIEJydWNrZXI6Cj4gPiBPbiAwNy8wOS8yMDE4IDA5OjU1LCBDaHJp c3RpYW4gS8O2bmlnIHdyb3RlOiAgCj4gPj4gSSB3aWxsIHRha2UgdGhpcyBhcyBhbiBvcHBvcnR1 bml0eSB0byBzdW1tYXJpemUgc29tZSBvZiB0aGUKPiA+PiByZXF1aXJlbWVudHMgd2UgaGF2ZSBm b3IgUEFTSUQgbWFuYWdlbWVudCBmcm9tIHRoZSBhbWRncHUgZHJpdmVyCj4gPj4gcG9pbnQgb2Yg dmlldzogIAo+ID4gVGhhdCdzIGluY3JlZGlibHkgdXNlZnVsLCB0aGFua3MgOikKPiA+ICAKPiA+ PiAxLiBXZSBuZWVkIHRvIGJlIGFibGUgdG8gYWxsb2NhdGUgUEFTSUQgYmV0d2VlbiAxIGFuZCBz b21lCj4gPj4gbWF4aW11bS4gWmVybyBpcyByZXNlcnZlZCBhcyBmYXIgYXMgSSBrbm93LCBidXQg d2UgZG9uJ3QgbmVjZXNzYXJ5Cj4gPj4gbmVlZCBhIG1pbmltdW0uICAKPiAgWy4uLl0gIAo+ID4+ IDIuIFdlIG5lZWQgdG8gYmUgYWJsZSB0byBhbGxvY2F0ZSBQQVNJRHMgd2l0aG91dCBhIHByb2Nl c3MgYWRkcmVzcwo+ID4+IHNwYWNlIGJhY2tpbmcgaXQuIEUuZy4gb3VyIGhhcmR3YXJlIHVzZXMg UEFTSURzIGV2ZW4gd2l0aG91dAo+ID4+IFNoYXJlZCBWaXJ0dWFsIEFkZHJlc3NpbmcgZW5hYmxl ZCB0byBkaXN0aW5jdCBjbGllbnRzIGZyb20gZWFjaAo+ID4+IG90aGVyLiBXb3VsZCBiZSBhIHBp dHkgaWYgd2UgbmVlZCB0byBzdGlsbCBoYXZlIGEgc2VwYXJhdGUgUEFTSUQKPiA+PiBoYW5kbGlu ZyBiZWNhdXNlIHRoZSBzeXN0ZW0gd2lkZSBpcyBvbmx5IGF2YWlsYWJsZSB3aGVuIElPTU1VIGlz Cj4gPj4gdHVybmVkIG9uLiAgCj4gIFsuLi5dICAKPiAKPiBJIGFncmVlIG9uIHRoYXQuCj4gCj4g PiBpb21tdS1zdmEgZXhwZWN0cyBldmVyeXdoZXJlIHRoYXQgdGhlIGRldmljZSBoYXMgYW4gaW9t bXVfZG9tYWluLAo+ID4gaXQncyB0aGUgZmlyc3QgdGhpbmcgd2UgY2hlY2sgb24gZW50cnkuIEJ5 cGFzc2luZyBhbGwgb2YgdGhpcyB3b3VsZAo+ID4gY2FsbCBpZHJfYWxsb2MoKSBkaXJlY3RseSwg YW5kIHdvdWxkbid0IGhhdmUgYW55IGNvZGUgaW4gY29tbW9uCj4gPiB3aXRoIHRoZSBjdXJyZW50 IGlvbW11LXN2YS4gU28gaXQgc2VlbXMgbGlrZSB5b3UgbmVlZCBhIGxheWVyIG9uCj4gPiB0b3Ag b2YgaW9tbXUtc3ZhIGNhbGxpbmcgaWRyX2FsbG9jKCkgd2hlbiBhbiBJT01NVSBpc24ndCBwcmVz ZW50LAo+ID4gYnV0IEkgZG9uJ3QgdGhpbmsgaXQgc2hvdWxkIGJlIGluIGRyaXZlcnMvaW9tbXUv ICAKPiAKPiBJbiB0aGlzIGNhc2UgSSBxdWVzdGlvbiBpZiB0aGUgUEFTSUQgaGFuZGxpbmcgc2hv dWxkIGJlIHVuZGVyIAo+IGRyaXZlcnMvaW9tbXUgYXQgYWxsLgo+IAo+IFNlZSBJIGNhbiBoYXZl IGEgbWl4IG9mIFZNIGNvbnRleHQgd2hpY2ggYXJlIGJvdW5kIHRvIHByb2Nlc3NlcyAoc29tZSAK PiBmZXcpIGFuZCBWTSBjb250ZXh0cyB3aGljaCBhcmUgc3RhbmRhbG9uZSBhbmQgZG9lc24ndCBj YXJlIGZvciBhCj4gcHJvY2VzcyBhZGRyZXNzIHNwYWNlLiBCdXQgZm9yIGVhY2ggVk0gY29udGV4 dCBJIG5lZWQgYSBkaXN0aW5jdAo+IFBBU0lEIGZvciB0aGUgaGFyZHdhcmUgdG8gd29yay4KPiAK PiBJIGNhbiBsaXZlIGlmIHdlIHNheSBpZiBJT01NVSBpcyBjb21wbGV0ZWx5IGRpc2FibGVkIHdl IHVzZSBhIHNpbXBsZQo+IGlkYSB0byBhbGxvY2F0ZSB0aGVtLCBidXQgd2hlbiBJT01NVSBpcyBl bmFibGVkIEkgY2VydGFpbmx5IG5lZWQgYQo+IHdheSB0byByZXNlcnZlIGEgUEFTSUQgd2l0aG91 dCBhbiBhc3NvY2lhdGVkIHByb2Nlc3MuCj4gClZULWQgd291bGQgYWxzbyBoYXZlIHN1Y2ggcmVx dWlyZW1lbnQuIFRoZXJlIGlzIGEgdmlydHVhbCBjb21tYW5kCnJlZ2lzdGVyIGZvciBhbGxvY2F0 ZSBhbmQgZnJlZSBQQVNJRCBmb3IgVk0gdXNlLiBXaGVuIHRoYXQgUEFTSUQKYWxsb2NhdGlvbiBy ZXF1ZXN0IGdldHMgcHJvcGFnYXRlZCB0byB0aGUgaG9zdCBJT01NVSBkcml2ZXIsIHdlIG5lZWQg dG8KYWxsb2NhdGUgUEFTSUQgdy9vIG1tLgoKSWYgdGhlIFBBU0lEIGFsbG9jYXRpb24gaXMgZG9u ZSB2aWEgVkZJTywgY2FuIHdlIGhhdmUgRkQgdG8gdHJhY2sgUEFTSUQKbGlmZSBjeWNsZSBpbnN0 ZWFkIG9mIG1tX2V4aXQoKT8gaS5lLiBhbGwgRkRzIGdldCBjbG9zZWQgYmVmb3JlCm1tX2V4aXQs IEkgYXNzdW1lPwoKPiA+PiAzLiBFdmVuIGFmdGVyIGRlc3RydWN0aW9uIG9mIGEgcHJvY2VzcyBh ZGRyZXNzIHNwYWNlIHdlIG5lZWQgc29tZQo+ID4+IGdyYWNlIHBlcmlvZCBiZWZvcmUgYSBQQVNJ RCBpcyByZXVzZWQgYmVjYXVzZSBpdCBjYW4gYmUgdGhhdCB0aGUKPiA+PiBzcGVjaWZpYyBQQVNJ RCBpcyBzdGlsbCBpbiBzb21lIGhhcmR3YXJlIHF1ZXVlcyBldGMuLi4KPiA+PiAgIMKgwqDCoCDC oMKgwqAgQXQgYmFyZSBtaW5pbXVtIGFsbCBkZXZpY2UgZHJpdmVycyB1c2luZyBwcm9jZXNzIGJp bmRpbmcKPiA+PiBuZWVkIHRvIGV4cGxpY2l0bHkgbm90ZSB0byB0aGUgY29yZSB3aGVuIHRoZXkg YXJlIGRvbmUgd2l0aCBhCj4gPj4gUEFTSUQuICAKPiA+IFJpZ2h0LCBtdWNoIG9mIHRoZSBob3Jy aWJsZW5lc3MgaW4gaW9tbXUtc3ZhIGRlYWxzIHdpdGggdGhpczoKPiA+Cj4gPiBUaGUgcHJvY2Vz cyBkaWVzLCBpb21tdS1zdmEgaXMgbm90aWZpZWQgYW5kIGNhbGxzIHRoZSBtbV9leGl0KCkKPiA+ IGZ1bmN0aW9uIHBhc3NlZCBieSB0aGUgZGV2aWNlIGRyaXZlciB0byBpb21tdV9zdmFfZGV2aWNl X2luaXQoKS4gSW4KPiA+IG1tX2V4aXQoKSB0aGUgZGV2aWNlIGRyaXZlciBuZWVkcyB0byBjbGVh ciBhbnkgcmVmZXJlbmNlIHRvIHRoZQo+ID4gUEFTSUQgaW4gaGFyZHdhcmUgYW5kIGluIGl0cyBv d24gc3RydWN0dXJlcy4gV2hlbiB0aGUgZGV2aWNlIGRyaXZlcgo+ID4gcmV0dXJucyBmcm9tIG1t X2V4aXQoKSwgaXQgZWZmZWN0aXZlbHkgdGVsbHMgdGhlIGNvcmUgdGhhdCBpdCBoYXMKPiA+IGZp bmlzaGVkIHVzaW5nIHRoZSBQQVNJRCwgYW5kIGlvbW11LXN2YSBjYW4gcmV1c2UgdGhlIFBBU0lE IGZvcgo+ID4gYW5vdGhlciBwcm9jZXNzLiBtbV9leGl0KCkgaXMgYWxsb3dlZCB0byBibG9jaywg c28gdGhlIGRldmljZQo+ID4gZHJpdmVyIGhhcyB0aW1lIHRvIGNsZWFuIHVwIGFuZCBmbHVzaCB0 aGUgcXVldWVzLgo+ID4KPiA+IElmIHRoZSBkZXZpY2UgZHJpdmVyIGZpbmlzaGVzIHVzaW5nIHRo ZSBQQVNJRCBiZWZvcmUgdGhlIHByb2Nlc3MKPiA+IGV4aXRzLCBpdCBqdXN0IGNhbGxzIHVuYmlu ZCgpLiAgCj4gCj4gRXhhY3RseSB0aGF0J3Mgd2hhdCBNaWNoYWwgSG9ja28gaXMgcHJvYmFibHkg Z29pbmcgdG8gbm90IGxpa2UgYXQgYWxsLgo+IAo+IENhbiB3ZSBoYXZlIGEgZGlmZmVyZW50IGFw cHJvYWNoIHdoZXJlIGVhY2ggZHJpdmVyIGlzIGluZm9ybWVkIGJ5IHRoZSAKPiBtbV9leGl0KCks IGJ1dCBuZWVkcyB0byBleHBsaWNpdGx5IGNhbGwgdW5iaW5kKCkgYmVmb3JlIGEgUEFTSUQgaXMK PiByZXVzZWQ/Cj4gCj4gRHVyaW5nIHRoYXQgdGVhcmRvd24gdHJhbnNpdGlvbiBpdCB3b3VsZCBi ZSBpZGVhbCBpZiB0aGF0IFBBU0lEIG9ubHkgCj4gcG9pbnRzIHRvIGEgZHVtbXkgcm9vdCBwYWdl IGRpcmVjdG9yeSB3aXRoIG9ubHkgaW52YWxpZCBlbnRyaWVzLgo+IApJIGd1ZXNzIHRoaXMgY2Fu IGJlIHZlbmRvciBzcGVjaWZpYywgSW4gVlQtZCBJIHBsYW4gdG8gbWFyayBQQVNJRAplbnRyeSBu b3QgcHJlc2VudCBhbmQgZGlzYWJsZSBmYXVsdCByZXBvcnRpbmcgd2hpbGUgZHJhaW5pbmcgcmVt YWluaW5nCmFjdGl2aXRpZXMuCgo+ID4gIAo+ID4+IDQuIEl0IHdvdWxkIGJlIG5pY2UgdG8gaGF2 ZSB0byBiZSBhYmxlIHRvIHNldCBhICJ2b2lkICoiIGZvciBlYWNoCj4gPj4gUEFTSUQvZGV2aWNl IGNvbWJpbmF0aW9uIHdoaWxlIGJpbmRpbmcgdG8gYSBwcm9jZXNzIHdoaWNoIHRoZW4gY2FuCj4g Pj4gYmUgcXVlcmllZCBsYXRlciBvbiBiYXNlZCBvbiB0aGUgUEFTSUQuCj4gPj4gICDCoMKgwqAg wqDCoMKgIEUuZy4gd2hlbiB5b3UgaGF2ZSBhIHBlciBQQVNJRC9kZXZpY2Ugc3RydWN0dXJlIGFy b3VuZAo+ID4+IGFueXdheSwganVzdCBhZGQgYW4gZXh0cmEgZmllbGQuICAKPiA+IGlvbW11X3N2 YV9iaW5kX2RldmljZSgpIHRha2VzIGEgImRydmRhdGEiIHBvaW50ZXIgdGhhdCBpcyBzdG9yZWQK PiA+IGludGVybmFsbHkgZm9yIHRoZSBQQVNJRC9kZXZpY2UgY29tYmluYXRpb24gKGlvbW11X2Jv bmQpLiBJdCBpcwo+ID4gcGFzc2VkIHRvIG1tX2V4aXQoKSwgYnV0IEkgaGF2ZW4ndCBhZGRlZCBh bnl0aGluZyBmb3IgdGhlIGRldmljZQo+ID4gZHJpdmVyIHRvIHF1ZXJ5IGl0IGJhY2suICAKPiAK PiBOaWNlISBMb29rcyBsaWtlIGFsbCB3ZSBuZWVkIGFkZGl0aW9uYWxseSBpcyBhIGZ1bmN0aW9u IHRvIHJldHJpZXZlCj4gdGhhdCBiYXNlZCBvbiB0aGUgUEFTSUQuCj4gCj4gPj4gNS4gSXQgd291 bGQgYmUgbmljZSB0byBoYXZlIHRvIGFsbG9jYXRlIG11bHRpcGxlIFBBU0lEcyBmb3IgdGhlCj4g Pj4gc2FtZSBwcm9jZXNzIGFkZHJlc3Mgc3BhY2UuCj4gPj4gICDCoMKgwqAgwqDCoMKgIEUuZy4g c29tZSB0ZWFtcyBhdCBBTUQgd2FudCB0byB1c2UgYSBzZXBhcmF0ZSBHUFUKPiA+PiBhZGRyZXNz IHNwYWNlIGZvciB0aGVpciB1c2Vyc3BhY2UgY2xpZW50IGxpYnJhcnkuIEknbSBzdGlsbCB0cnlp bmcKPiA+PiB0byBhdm9pZCB0aGF0LCBidXQgaXQgaXMgcGVyZmVjdGx5IHBvc3NpYmxlIHRoYXQg d2UgYXJlIGdvaW5nIHRvCj4gPj4gbmVlZCB0aGF0LiAgCj4gPiBUd28gUEFTSURzIHBvaW50aW5n IHRvIHRoZSBzYW1lIHByb2Nlc3MgcGdkPyBBdCBmaXJzdCBnbGFuY2UgaXQKPiA+IHNlZW1zIGZl YXNpYmxlLCBtYXliZSB3aXRoIGEgZmxhZyBwYXNzZWQgdG8gYmluZCgpIGFuZCBhIGZldwo+ID4g Y2hhbmdlcyB0byBpbnRlcm5hbCBzdHJ1Y3R1cmVzLiBJdCB3aWxsIGR1cGxpY2F0ZSBBVEMgaW52 YWxpZGF0aW9uCj4gPiBjb21tYW5kcyBmb3IgZWFjaCBwcm9jZXNzIGFkZHJlc3Mgc3BhY2UgY2hh bmdlIChtdW5tYXAgZXRjKSBzbyB5b3UKPiA+IG1pZ2h0IHRha2UgYSBwZXJmb3JtYW5jZSBoaXQu Cj4gPgo+ID4gSW50ZWwncyBTVk0gY29kZSBoYXMgdGhlIFNWTV9GTEFHX1BSSVZBVEVfUEFTSUQg d2hpY2ggc2VlbXMgc2ltaWxhcgo+ID4gdG8gd2hhdCB5b3UgZGVzY3JpYmUsIGJ1dCBJIGRvbid0 IHBsYW4gdG8gc3VwcG9ydCBpdCBpbiB0aGlzIHNlcmllcwo+ID4gKHRoZSBpb19tbSBtb2RlbCBp cyBhbHJlYWR5IHByZXR0eSBjb21wbGljYXRlZCkuIEkgdGhpbmsgaXQgY2FuIGJlCj4gPiBhZGRl ZCB3aXRob3V0IHRvbyBtdWNoIGVmZm9ydCBpbiBhIGZ1dHVyZSBzZXJpZXMsIHRob3VnaCB3aXRo IGEKPiA+IGRpZmZlcmVudCBmbGFnIG5hbWUgc2luY2Ugd2UnZCBsaWtlIHRvIHVzZSAicHJpdmF0 ZSBQQVNJRCIgZm9yCj4gPiBzb21ldGhpbmcgZWxzZQo+ID4gKGh0dHBzOi8vd3d3LnNwaW5pY3Mu bmV0L2xpc3RzL2RyaS1kZXZlbC9tc2cxNzcwMDcuaHRtbCkuICAKPiAKPiBUbyBiZSBob25lc3Qg SSBob3BlZCB0aGF0IHlvdSB3b3VsZCBzYXk6IE5vIG5ldmVyISBTbyB0aGF0IEkgaGF2ZSBhCj4g Z29vZCBhcmd1bWVudCB0byBwdXNoYmFjayBvbiBzdWNoIHJlcXVpcmVtZW50cyA6KQo+IAo+IEJ1 dCBpZiBpdCdzIGRvYWJsZSBpdCB3b3VsZCBiZSBhdCBsZWFzdCBuaWNlIHRvIGhhdmUgZm9yIGRl YnVnZ2luZy4KPiAKPiBUaGFua3MgYSBsb3QgZm9yIHdvcmtpbmcgb24gdGhhdCwKPiBDaHJpc3Rp YW4uCj4gCj4gPgo+ID4gVGhhbmtzLAo+ID4gSmVhbgo+ID4gIAo+ID4+ICAgwqDCoMKgIMKgwqDC oCBBZGRpdGlvbmFsIHRvIHRoYXQgaXQgaXMgc29tZXRpbWVzIHF1aXRlIHVzZWZ1bCBmb3IKPiA+ PiBkZWJ1Z2dpbmcgdG8gaXNvbGF0ZSB3aGVyZSBleGFjdGx5IGFuIGluY29ycmVjdCBhY2Nlc3Mg KHNlZ2ZhdWx0KQo+ID4+IGlzIGNvbWluZyBmcm9tLgo+ID4+Cj4gPj4gTGV0IG1lIGtub3cgaWYg dGhlcmUgYXJlIHNvbWUgcHJvYmxlbXMgd2l0aCB0aGF0LCBlc3BlY2lhbGx5IEkKPiA+PiB3YW50 IHRvIGtub3cgaWYgdGhlcmUgaXMgcHVzaGJhY2sgb24gIzUgc28gdGhhdCBJIGNhbiBmb3J3YXJk Cj4gPj4gdGhhdCA6KQo+ID4+Cj4gPj4gVGhhbmtzLAo+ID4+IENocmlzdGlhbi4KPiA+PiAgCj4g Pj4+IFRoYW5rcywKPiA+Pj4gSmVhbiAgCj4gCgpbSmFjb2IgUGFuXQoKX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGludXgtYXJtLWtlcm5lbCBtYWlsaW5n IGxpc3QKbGludXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRlYWQub3JnCmh0dHA6Ly9saXN0cy5p bmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8vbGludXgtYXJtLWtlcm5lbAo= From mboxrd@z Thu Jan 1 00:00:00 1970 From: jacob.jun.pan@linux.intel.com (Jacob Pan) Date: Fri, 7 Sep 2018 14:25:04 -0700 Subject: [PATCH v2 01/40] iommu: Introduce Shared Virtual Addressing API In-Reply-To: <5bbc0332-b94b-75cc-ca42-a9b196811daf@amd.com> References: <20180511190641.23008-1-jean-philippe.brucker@arm.com> <20180511190641.23008-2-jean-philippe.brucker@arm.com> <03d31ba5-1eda-ea86-8c0c-91d14c86fe83@arm.com> <2fd4a0a1-1a35-bf82-df84-b995cce011d9@amd.com> <65e7accd-4446-19f5-c667-c6407e89cfa6@arm.com> <5bbc0332-b94b-75cc-ca42-a9b196811daf@amd.com> Message-ID: <20180907142504.5034351e@jacob-builder> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org On Fri, 7 Sep 2018 20:02:54 +0200 Christian K?nig wrote: > Am 07.09.2018 um 17:45 schrieb Jean-Philippe Brucker: > > On 07/09/2018 09:55, Christian K?nig wrote: > >> I will take this as an opportunity to summarize some of the > >> requirements we have for PASID management from the amdgpu driver > >> point of view: > > That's incredibly useful, thanks :) > > > >> 1. We need to be able to allocate PASID between 1 and some > >> maximum. Zero is reserved as far as I know, but we don't necessary > >> need a minimum. > [...] > >> 2. We need to be able to allocate PASIDs without a process address > >> space backing it. E.g. our hardware uses PASIDs even without > >> Shared Virtual Addressing enabled to distinct clients from each > >> other. Would be a pity if we need to still have a separate PASID > >> handling because the system wide is only available when IOMMU is > >> turned on. > [...] > > I agree on that. > > > iommu-sva expects everywhere that the device has an iommu_domain, > > it's the first thing we check on entry. Bypassing all of this would > > call idr_alloc() directly, and wouldn't have any code in common > > with the current iommu-sva. So it seems like you need a layer on > > top of iommu-sva calling idr_alloc() when an IOMMU isn't present, > > but I don't think it should be in drivers/iommu/ > > In this case I question if the PASID handling should be under > drivers/iommu at all. > > See I can have a mix of VM context which are bound to processes (some > few) and VM contexts which are standalone and doesn't care for a > process address space. But for each VM context I need a distinct > PASID for the hardware to work. > > I can live if we say if IOMMU is completely disabled we use a simple > ida to allocate them, but when IOMMU is enabled I certainly need a > way to reserve a PASID without an associated process. > VT-d would also have such requirement. There is a virtual command register for allocate and free PASID for VM use. When that PASID allocation request gets propagated to the host IOMMU driver, we need to allocate PASID w/o mm. If the PASID allocation is done via VFIO, can we have FD to track PASID life cycle instead of mm_exit()? i.e. all FDs get closed before mm_exit, I assume? > >> 3. Even after destruction of a process address space we need some > >> grace period before a PASID is reused because it can be that the > >> specific PASID is still in some hardware queues etc... > >> ??? ??? At bare minimum all device drivers using process binding > >> need to explicitly note to the core when they are done with a > >> PASID. > > Right, much of the horribleness in iommu-sva deals with this: > > > > The process dies, iommu-sva is notified and calls the mm_exit() > > function passed by the device driver to iommu_sva_device_init(). In > > mm_exit() the device driver needs to clear any reference to the > > PASID in hardware and in its own structures. When the device driver > > returns from mm_exit(), it effectively tells the core that it has > > finished using the PASID, and iommu-sva can reuse the PASID for > > another process. mm_exit() is allowed to block, so the device > > driver has time to clean up and flush the queues. > > > > If the device driver finishes using the PASID before the process > > exits, it just calls unbind(). > > Exactly that's what Michal Hocko is probably going to not like at all. > > Can we have a different approach where each driver is informed by the > mm_exit(), but needs to explicitly call unbind() before a PASID is > reused? > > During that teardown transition it would be ideal if that PASID only > points to a dummy root page directory with only invalid entries. > I guess this can be vendor specific, In VT-d I plan to mark PASID entry not present and disable fault reporting while draining remaining activities. > > > >> 4. It would be nice to have to be able to set a "void *" for each > >> PASID/device combination while binding to a process which then can > >> be queried later on based on the PASID. > >> ??? ??? E.g. when you have a per PASID/device structure around > >> anyway, just add an extra field. > > iommu_sva_bind_device() takes a "drvdata" pointer that is stored > > internally for the PASID/device combination (iommu_bond). It is > > passed to mm_exit(), but I haven't added anything for the device > > driver to query it back. > > Nice! Looks like all we need additionally is a function to retrieve > that based on the PASID. > > >> 5. It would be nice to have to allocate multiple PASIDs for the > >> same process address space. > >> ??? ??? E.g. some teams at AMD want to use a separate GPU > >> address space for their userspace client library. I'm still trying > >> to avoid that, but it is perfectly possible that we are going to > >> need that. > > Two PASIDs pointing to the same process pgd? At first glance it > > seems feasible, maybe with a flag passed to bind() and a few > > changes to internal structures. It will duplicate ATC invalidation > > commands for each process address space change (munmap etc) so you > > might take a performance hit. > > > > Intel's SVM code has the SVM_FLAG_PRIVATE_PASID which seems similar > > to what you describe, but I don't plan to support it in this series > > (the io_mm model is already pretty complicated). I think it can be > > added without too much effort in a future series, though with a > > different flag name since we'd like to use "private PASID" for > > something else > > (https://www.spinics.net/lists/dri-devel/msg177007.html). > > To be honest I hoped that you would say: No never! So that I have a > good argument to pushback on such requirements :) > > But if it's doable it would be at least nice to have for debugging. > > Thanks a lot for working on that, > Christian. > > > > > Thanks, > > Jean > > > >> ??? ??? Additional to that it is sometimes quite useful for > >> debugging to isolate where exactly an incorrect access (segfault) > >> is coming from. > >> > >> Let me know if there are some problems with that, especially I > >> want to know if there is pushback on #5 so that I can forward > >> that :) > >> > >> Thanks, > >> Christian. > >> > >>> Thanks, > >>> Jean > [Jacob Pan] From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jacob Pan Subject: Re: [PATCH v2 01/40] iommu: Introduce Shared Virtual Addressing API Date: Fri, 7 Sep 2018 14:25:04 -0700 Message-ID: <20180907142504.5034351e@jacob-builder> References: <20180511190641.23008-1-jean-philippe.brucker@arm.com> <20180511190641.23008-2-jean-philippe.brucker@arm.com> <03d31ba5-1eda-ea86-8c0c-91d14c86fe83@arm.com> <2fd4a0a1-1a35-bf82-df84-b995cce011d9@amd.com> <65e7accd-4446-19f5-c667-c6407e89cfa6@arm.com> <5bbc0332-b94b-75cc-ca42-a9b196811daf@amd.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Cc: "xieyisheng1-hv44wF8Li93QT0dZR+AlfA@public.gmane.org" , "ilias.apalodimas-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org" , "kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , Will Deacon , Michal Hocko , "okaya-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org" , "linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org" , "ashok.raj-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org" , Jean-Philippe Brucker , "linux-acpi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "rfranz-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org" , "devicetree-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "kevin.tian-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org" , "rgummal-gjFFaj9aHVfQT0dZR+AlfA@public.gmane.org" , "linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org" , "dwmw2-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org" , "liubo95-hv44wF8Li93QT0dZR+AlfA@public.gmane.org" Return-path: In-Reply-To: <5bbc0332-b94b-75cc-ca42-a9b196811daf-5C7GfCeVMHo@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org List-Id: kvm.vger.kernel.org T24gRnJpLCA3IFNlcCAyMDE4IDIwOjAyOjU0ICswMjAwCkNocmlzdGlhbiBLw7ZuaWcgPGNocmlz dGlhbi5rb2VuaWdAYW1kLmNvbT4gd3JvdGU6Cgo+IEFtIDA3LjA5LjIwMTggdW0gMTc6NDUgc2No cmllYiBKZWFuLVBoaWxpcHBlIEJydWNrZXI6Cj4gPiBPbiAwNy8wOS8yMDE4IDA5OjU1LCBDaHJp c3RpYW4gS8O2bmlnIHdyb3RlOiAgCj4gPj4gSSB3aWxsIHRha2UgdGhpcyBhcyBhbiBvcHBvcnR1 bml0eSB0byBzdW1tYXJpemUgc29tZSBvZiB0aGUKPiA+PiByZXF1aXJlbWVudHMgd2UgaGF2ZSBm b3IgUEFTSUQgbWFuYWdlbWVudCBmcm9tIHRoZSBhbWRncHUgZHJpdmVyCj4gPj4gcG9pbnQgb2Yg dmlldzogIAo+ID4gVGhhdCdzIGluY3JlZGlibHkgdXNlZnVsLCB0aGFua3MgOikKPiA+ICAKPiA+ PiAxLiBXZSBuZWVkIHRvIGJlIGFibGUgdG8gYWxsb2NhdGUgUEFTSUQgYmV0d2VlbiAxIGFuZCBz b21lCj4gPj4gbWF4aW11bS4gWmVybyBpcyByZXNlcnZlZCBhcyBmYXIgYXMgSSBrbm93LCBidXQg d2UgZG9uJ3QgbmVjZXNzYXJ5Cj4gPj4gbmVlZCBhIG1pbmltdW0uICAKPiAgWy4uLl0gIAo+ID4+ IDIuIFdlIG5lZWQgdG8gYmUgYWJsZSB0byBhbGxvY2F0ZSBQQVNJRHMgd2l0aG91dCBhIHByb2Nl c3MgYWRkcmVzcwo+ID4+IHNwYWNlIGJhY2tpbmcgaXQuIEUuZy4gb3VyIGhhcmR3YXJlIHVzZXMg UEFTSURzIGV2ZW4gd2l0aG91dAo+ID4+IFNoYXJlZCBWaXJ0dWFsIEFkZHJlc3NpbmcgZW5hYmxl ZCB0byBkaXN0aW5jdCBjbGllbnRzIGZyb20gZWFjaAo+ID4+IG90aGVyLiBXb3VsZCBiZSBhIHBp dHkgaWYgd2UgbmVlZCB0byBzdGlsbCBoYXZlIGEgc2VwYXJhdGUgUEFTSUQKPiA+PiBoYW5kbGlu ZyBiZWNhdXNlIHRoZSBzeXN0ZW0gd2lkZSBpcyBvbmx5IGF2YWlsYWJsZSB3aGVuIElPTU1VIGlz Cj4gPj4gdHVybmVkIG9uLiAgCj4gIFsuLi5dICAKPiAKPiBJIGFncmVlIG9uIHRoYXQuCj4gCj4g PiBpb21tdS1zdmEgZXhwZWN0cyBldmVyeXdoZXJlIHRoYXQgdGhlIGRldmljZSBoYXMgYW4gaW9t bXVfZG9tYWluLAo+ID4gaXQncyB0aGUgZmlyc3QgdGhpbmcgd2UgY2hlY2sgb24gZW50cnkuIEJ5 cGFzc2luZyBhbGwgb2YgdGhpcyB3b3VsZAo+ID4gY2FsbCBpZHJfYWxsb2MoKSBkaXJlY3RseSwg YW5kIHdvdWxkbid0IGhhdmUgYW55IGNvZGUgaW4gY29tbW9uCj4gPiB3aXRoIHRoZSBjdXJyZW50 IGlvbW11LXN2YS4gU28gaXQgc2VlbXMgbGlrZSB5b3UgbmVlZCBhIGxheWVyIG9uCj4gPiB0b3Ag b2YgaW9tbXUtc3ZhIGNhbGxpbmcgaWRyX2FsbG9jKCkgd2hlbiBhbiBJT01NVSBpc24ndCBwcmVz ZW50LAo+ID4gYnV0IEkgZG9uJ3QgdGhpbmsgaXQgc2hvdWxkIGJlIGluIGRyaXZlcnMvaW9tbXUv ICAKPiAKPiBJbiB0aGlzIGNhc2UgSSBxdWVzdGlvbiBpZiB0aGUgUEFTSUQgaGFuZGxpbmcgc2hv dWxkIGJlIHVuZGVyIAo+IGRyaXZlcnMvaW9tbXUgYXQgYWxsLgo+IAo+IFNlZSBJIGNhbiBoYXZl IGEgbWl4IG9mIFZNIGNvbnRleHQgd2hpY2ggYXJlIGJvdW5kIHRvIHByb2Nlc3NlcyAoc29tZSAK PiBmZXcpIGFuZCBWTSBjb250ZXh0cyB3aGljaCBhcmUgc3RhbmRhbG9uZSBhbmQgZG9lc24ndCBj YXJlIGZvciBhCj4gcHJvY2VzcyBhZGRyZXNzIHNwYWNlLiBCdXQgZm9yIGVhY2ggVk0gY29udGV4 dCBJIG5lZWQgYSBkaXN0aW5jdAo+IFBBU0lEIGZvciB0aGUgaGFyZHdhcmUgdG8gd29yay4KPiAK PiBJIGNhbiBsaXZlIGlmIHdlIHNheSBpZiBJT01NVSBpcyBjb21wbGV0ZWx5IGRpc2FibGVkIHdl IHVzZSBhIHNpbXBsZQo+IGlkYSB0byBhbGxvY2F0ZSB0aGVtLCBidXQgd2hlbiBJT01NVSBpcyBl bmFibGVkIEkgY2VydGFpbmx5IG5lZWQgYQo+IHdheSB0byByZXNlcnZlIGEgUEFTSUQgd2l0aG91 dCBhbiBhc3NvY2lhdGVkIHByb2Nlc3MuCj4gClZULWQgd291bGQgYWxzbyBoYXZlIHN1Y2ggcmVx dWlyZW1lbnQuIFRoZXJlIGlzIGEgdmlydHVhbCBjb21tYW5kCnJlZ2lzdGVyIGZvciBhbGxvY2F0 ZSBhbmQgZnJlZSBQQVNJRCBmb3IgVk0gdXNlLiBXaGVuIHRoYXQgUEFTSUQKYWxsb2NhdGlvbiBy ZXF1ZXN0IGdldHMgcHJvcGFnYXRlZCB0byB0aGUgaG9zdCBJT01NVSBkcml2ZXIsIHdlIG5lZWQg dG8KYWxsb2NhdGUgUEFTSUQgdy9vIG1tLgoKSWYgdGhlIFBBU0lEIGFsbG9jYXRpb24gaXMgZG9u ZSB2aWEgVkZJTywgY2FuIHdlIGhhdmUgRkQgdG8gdHJhY2sgUEFTSUQKbGlmZSBjeWNsZSBpbnN0 ZWFkIG9mIG1tX2V4aXQoKT8gaS5lLiBhbGwgRkRzIGdldCBjbG9zZWQgYmVmb3JlCm1tX2V4aXQs IEkgYXNzdW1lPwoKPiA+PiAzLiBFdmVuIGFmdGVyIGRlc3RydWN0aW9uIG9mIGEgcHJvY2VzcyBh ZGRyZXNzIHNwYWNlIHdlIG5lZWQgc29tZQo+ID4+IGdyYWNlIHBlcmlvZCBiZWZvcmUgYSBQQVNJ RCBpcyByZXVzZWQgYmVjYXVzZSBpdCBjYW4gYmUgdGhhdCB0aGUKPiA+PiBzcGVjaWZpYyBQQVNJ RCBpcyBzdGlsbCBpbiBzb21lIGhhcmR3YXJlIHF1ZXVlcyBldGMuLi4KPiA+PiAgIMKgwqDCoCDC oMKgwqAgQXQgYmFyZSBtaW5pbXVtIGFsbCBkZXZpY2UgZHJpdmVycyB1c2luZyBwcm9jZXNzIGJp bmRpbmcKPiA+PiBuZWVkIHRvIGV4cGxpY2l0bHkgbm90ZSB0byB0aGUgY29yZSB3aGVuIHRoZXkg YXJlIGRvbmUgd2l0aCBhCj4gPj4gUEFTSUQuICAKPiA+IFJpZ2h0LCBtdWNoIG9mIHRoZSBob3Jy aWJsZW5lc3MgaW4gaW9tbXUtc3ZhIGRlYWxzIHdpdGggdGhpczoKPiA+Cj4gPiBUaGUgcHJvY2Vz cyBkaWVzLCBpb21tdS1zdmEgaXMgbm90aWZpZWQgYW5kIGNhbGxzIHRoZSBtbV9leGl0KCkKPiA+ IGZ1bmN0aW9uIHBhc3NlZCBieSB0aGUgZGV2aWNlIGRyaXZlciB0byBpb21tdV9zdmFfZGV2aWNl X2luaXQoKS4gSW4KPiA+IG1tX2V4aXQoKSB0aGUgZGV2aWNlIGRyaXZlciBuZWVkcyB0byBjbGVh ciBhbnkgcmVmZXJlbmNlIHRvIHRoZQo+ID4gUEFTSUQgaW4gaGFyZHdhcmUgYW5kIGluIGl0cyBv d24gc3RydWN0dXJlcy4gV2hlbiB0aGUgZGV2aWNlIGRyaXZlcgo+ID4gcmV0dXJucyBmcm9tIG1t X2V4aXQoKSwgaXQgZWZmZWN0aXZlbHkgdGVsbHMgdGhlIGNvcmUgdGhhdCBpdCBoYXMKPiA+IGZp bmlzaGVkIHVzaW5nIHRoZSBQQVNJRCwgYW5kIGlvbW11LXN2YSBjYW4gcmV1c2UgdGhlIFBBU0lE IGZvcgo+ID4gYW5vdGhlciBwcm9jZXNzLiBtbV9leGl0KCkgaXMgYWxsb3dlZCB0byBibG9jaywg c28gdGhlIGRldmljZQo+ID4gZHJpdmVyIGhhcyB0aW1lIHRvIGNsZWFuIHVwIGFuZCBmbHVzaCB0 aGUgcXVldWVzLgo+ID4KPiA+IElmIHRoZSBkZXZpY2UgZHJpdmVyIGZpbmlzaGVzIHVzaW5nIHRo ZSBQQVNJRCBiZWZvcmUgdGhlIHByb2Nlc3MKPiA+IGV4aXRzLCBpdCBqdXN0IGNhbGxzIHVuYmlu ZCgpLiAgCj4gCj4gRXhhY3RseSB0aGF0J3Mgd2hhdCBNaWNoYWwgSG9ja28gaXMgcHJvYmFibHkg Z29pbmcgdG8gbm90IGxpa2UgYXQgYWxsLgo+IAo+IENhbiB3ZSBoYXZlIGEgZGlmZmVyZW50IGFw cHJvYWNoIHdoZXJlIGVhY2ggZHJpdmVyIGlzIGluZm9ybWVkIGJ5IHRoZSAKPiBtbV9leGl0KCks IGJ1dCBuZWVkcyB0byBleHBsaWNpdGx5IGNhbGwgdW5iaW5kKCkgYmVmb3JlIGEgUEFTSUQgaXMK PiByZXVzZWQ/Cj4gCj4gRHVyaW5nIHRoYXQgdGVhcmRvd24gdHJhbnNpdGlvbiBpdCB3b3VsZCBi ZSBpZGVhbCBpZiB0aGF0IFBBU0lEIG9ubHkgCj4gcG9pbnRzIHRvIGEgZHVtbXkgcm9vdCBwYWdl IGRpcmVjdG9yeSB3aXRoIG9ubHkgaW52YWxpZCBlbnRyaWVzLgo+IApJIGd1ZXNzIHRoaXMgY2Fu IGJlIHZlbmRvciBzcGVjaWZpYywgSW4gVlQtZCBJIHBsYW4gdG8gbWFyayBQQVNJRAplbnRyeSBu b3QgcHJlc2VudCBhbmQgZGlzYWJsZSBmYXVsdCByZXBvcnRpbmcgd2hpbGUgZHJhaW5pbmcgcmVt YWluaW5nCmFjdGl2aXRpZXMuCgo+ID4gIAo+ID4+IDQuIEl0IHdvdWxkIGJlIG5pY2UgdG8gaGF2 ZSB0byBiZSBhYmxlIHRvIHNldCBhICJ2b2lkICoiIGZvciBlYWNoCj4gPj4gUEFTSUQvZGV2aWNl IGNvbWJpbmF0aW9uIHdoaWxlIGJpbmRpbmcgdG8gYSBwcm9jZXNzIHdoaWNoIHRoZW4gY2FuCj4g Pj4gYmUgcXVlcmllZCBsYXRlciBvbiBiYXNlZCBvbiB0aGUgUEFTSUQuCj4gPj4gICDCoMKgwqAg wqDCoMKgIEUuZy4gd2hlbiB5b3UgaGF2ZSBhIHBlciBQQVNJRC9kZXZpY2Ugc3RydWN0dXJlIGFy b3VuZAo+ID4+IGFueXdheSwganVzdCBhZGQgYW4gZXh0cmEgZmllbGQuICAKPiA+IGlvbW11X3N2 YV9iaW5kX2RldmljZSgpIHRha2VzIGEgImRydmRhdGEiIHBvaW50ZXIgdGhhdCBpcyBzdG9yZWQK PiA+IGludGVybmFsbHkgZm9yIHRoZSBQQVNJRC9kZXZpY2UgY29tYmluYXRpb24gKGlvbW11X2Jv bmQpLiBJdCBpcwo+ID4gcGFzc2VkIHRvIG1tX2V4aXQoKSwgYnV0IEkgaGF2ZW4ndCBhZGRlZCBh bnl0aGluZyBmb3IgdGhlIGRldmljZQo+ID4gZHJpdmVyIHRvIHF1ZXJ5IGl0IGJhY2suICAKPiAK PiBOaWNlISBMb29rcyBsaWtlIGFsbCB3ZSBuZWVkIGFkZGl0aW9uYWxseSBpcyBhIGZ1bmN0aW9u IHRvIHJldHJpZXZlCj4gdGhhdCBiYXNlZCBvbiB0aGUgUEFTSUQuCj4gCj4gPj4gNS4gSXQgd291 bGQgYmUgbmljZSB0byBoYXZlIHRvIGFsbG9jYXRlIG11bHRpcGxlIFBBU0lEcyBmb3IgdGhlCj4g Pj4gc2FtZSBwcm9jZXNzIGFkZHJlc3Mgc3BhY2UuCj4gPj4gICDCoMKgwqAgwqDCoMKgIEUuZy4g c29tZSB0ZWFtcyBhdCBBTUQgd2FudCB0byB1c2UgYSBzZXBhcmF0ZSBHUFUKPiA+PiBhZGRyZXNz IHNwYWNlIGZvciB0aGVpciB1c2Vyc3BhY2UgY2xpZW50IGxpYnJhcnkuIEknbSBzdGlsbCB0cnlp bmcKPiA+PiB0byBhdm9pZCB0aGF0LCBidXQgaXQgaXMgcGVyZmVjdGx5IHBvc3NpYmxlIHRoYXQg d2UgYXJlIGdvaW5nIHRvCj4gPj4gbmVlZCB0aGF0LiAgCj4gPiBUd28gUEFTSURzIHBvaW50aW5n IHRvIHRoZSBzYW1lIHByb2Nlc3MgcGdkPyBBdCBmaXJzdCBnbGFuY2UgaXQKPiA+IHNlZW1zIGZl YXNpYmxlLCBtYXliZSB3aXRoIGEgZmxhZyBwYXNzZWQgdG8gYmluZCgpIGFuZCBhIGZldwo+ID4g Y2hhbmdlcyB0byBpbnRlcm5hbCBzdHJ1Y3R1cmVzLiBJdCB3aWxsIGR1cGxpY2F0ZSBBVEMgaW52 YWxpZGF0aW9uCj4gPiBjb21tYW5kcyBmb3IgZWFjaCBwcm9jZXNzIGFkZHJlc3Mgc3BhY2UgY2hh bmdlIChtdW5tYXAgZXRjKSBzbyB5b3UKPiA+IG1pZ2h0IHRha2UgYSBwZXJmb3JtYW5jZSBoaXQu Cj4gPgo+ID4gSW50ZWwncyBTVk0gY29kZSBoYXMgdGhlIFNWTV9GTEFHX1BSSVZBVEVfUEFTSUQg d2hpY2ggc2VlbXMgc2ltaWxhcgo+ID4gdG8gd2hhdCB5b3UgZGVzY3JpYmUsIGJ1dCBJIGRvbid0 IHBsYW4gdG8gc3VwcG9ydCBpdCBpbiB0aGlzIHNlcmllcwo+ID4gKHRoZSBpb19tbSBtb2RlbCBp cyBhbHJlYWR5IHByZXR0eSBjb21wbGljYXRlZCkuIEkgdGhpbmsgaXQgY2FuIGJlCj4gPiBhZGRl ZCB3aXRob3V0IHRvbyBtdWNoIGVmZm9ydCBpbiBhIGZ1dHVyZSBzZXJpZXMsIHRob3VnaCB3aXRo IGEKPiA+IGRpZmZlcmVudCBmbGFnIG5hbWUgc2luY2Ugd2UnZCBsaWtlIHRvIHVzZSAicHJpdmF0 ZSBQQVNJRCIgZm9yCj4gPiBzb21ldGhpbmcgZWxzZQo+ID4gKGh0dHBzOi8vd3d3LnNwaW5pY3Mu bmV0L2xpc3RzL2RyaS1kZXZlbC9tc2cxNzcwMDcuaHRtbCkuICAKPiAKPiBUbyBiZSBob25lc3Qg SSBob3BlZCB0aGF0IHlvdSB3b3VsZCBzYXk6IE5vIG5ldmVyISBTbyB0aGF0IEkgaGF2ZSBhCj4g Z29vZCBhcmd1bWVudCB0byBwdXNoYmFjayBvbiBzdWNoIHJlcXVpcmVtZW50cyA6KQo+IAo+IEJ1 dCBpZiBpdCdzIGRvYWJsZSBpdCB3b3VsZCBiZSBhdCBsZWFzdCBuaWNlIHRvIGhhdmUgZm9yIGRl YnVnZ2luZy4KPiAKPiBUaGFua3MgYSBsb3QgZm9yIHdvcmtpbmcgb24gdGhhdCwKPiBDaHJpc3Rp YW4uCj4gCj4gPgo+ID4gVGhhbmtzLAo+ID4gSmVhbgo+ID4gIAo+ID4+ICAgwqDCoMKgIMKgwqDC oCBBZGRpdGlvbmFsIHRvIHRoYXQgaXQgaXMgc29tZXRpbWVzIHF1aXRlIHVzZWZ1bCBmb3IKPiA+ PiBkZWJ1Z2dpbmcgdG8gaXNvbGF0ZSB3aGVyZSBleGFjdGx5IGFuIGluY29ycmVjdCBhY2Nlc3Mg KHNlZ2ZhdWx0KQo+ID4+IGlzIGNvbWluZyBmcm9tLgo+ID4+Cj4gPj4gTGV0IG1lIGtub3cgaWYg dGhlcmUgYXJlIHNvbWUgcHJvYmxlbXMgd2l0aCB0aGF0LCBlc3BlY2lhbGx5IEkKPiA+PiB3YW50 IHRvIGtub3cgaWYgdGhlcmUgaXMgcHVzaGJhY2sgb24gIzUgc28gdGhhdCBJIGNhbiBmb3J3YXJk Cj4gPj4gdGhhdCA6KQo+ID4+Cj4gPj4gVGhhbmtzLAo+ID4+IENocmlzdGlhbi4KPiA+PiAgCj4g Pj4+IFRoYW5rcywKPiA+Pj4gSmVhbiAgCj4gCgpbSmFjb2IgUGFuXQpfX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwppb21tdSBtYWlsaW5nIGxpc3QKaW9tbXVA bGlzdHMubGludXgtZm91bmRhdGlvbi5vcmcKaHR0cHM6Ly9saXN0cy5saW51eGZvdW5kYXRpb24u b3JnL21haWxtYW4vbGlzdGluZm8vaW9tbXU= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pg1-f197.google.com (mail-pg1-f197.google.com [209.85.215.197]) by kanga.kvack.org (Postfix) with ESMTP id 538D78E0001 for ; Fri, 7 Sep 2018 17:23:22 -0400 (EDT) Received: by mail-pg1-f197.google.com with SMTP id 186-v6so7676224pgc.12 for ; Fri, 07 Sep 2018 14:23:22 -0700 (PDT) Received: from mga12.intel.com (mga12.intel.com. [192.55.52.136]) by mx.google.com with ESMTPS id h4-v6si9413384pgc.429.2018.09.07.14.23.20 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 07 Sep 2018 14:23:20 -0700 (PDT) Date: Fri, 7 Sep 2018 14:25:04 -0700 From: Jacob Pan Subject: Re: [PATCH v2 01/40] iommu: Introduce Shared Virtual Addressing API Message-ID: <20180907142504.5034351e@jacob-builder> In-Reply-To: <5bbc0332-b94b-75cc-ca42-a9b196811daf@amd.com> References: <20180511190641.23008-1-jean-philippe.brucker@arm.com> <20180511190641.23008-2-jean-philippe.brucker@arm.com> <03d31ba5-1eda-ea86-8c0c-91d14c86fe83@arm.com> <2fd4a0a1-1a35-bf82-df84-b995cce011d9@amd.com> <65e7accd-4446-19f5-c667-c6407e89cfa6@arm.com> <5bbc0332-b94b-75cc-ca42-a9b196811daf@amd.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Sender: owner-linux-mm@kvack.org List-ID: To: Christian =?UTF-8?B?S8O2bmln?= Cc: Jean-Philippe Brucker , Auger Eric , "linux-arm-kernel@lists.infradead.org" , "linux-pci@vger.kernel.org" , "linux-acpi@vger.kernel.org" , "devicetree@vger.kernel.org" , "iommu@lists.linux-foundation.org" , "kvm@vger.kernel.org" , "linux-mm@kvack.org" , "xieyisheng1@huawei.com" , "liubo95@huawei.com" , "xuzaibo@huawei.com" , "thunder.leizhen@huawei.com" , Will Deacon , "okaya@codeaurora.org" , "yi.l.liu@intel.com" , "ashok.raj@intel.com" , "tn@semihalf.com" , "joro@8bytes.org" , "bharatku@xilinx.com" , "liudongdong3@huawei.com" , "rfranz@cavium.com" , "kevin.tian@intel.com" , "jcrouse@codeaurora.org" , "rgummal@xilinx.com" , "jonathan.cameron@huawei.com" , "shunyong.yang@hxt-semitech.com" , Robin Murphy , "ilias.apalodimas@linaro.org" , "alex.williamson@redhat.com" , "robdclark@gmail.com" , "dwmw2@infradead.org" , "nwatters@codeaurora.org" , "baolu.lu@linux.intel.com" , Michal Hocko , jacob.jun.pan@linux.intel.com On Fri, 7 Sep 2018 20:02:54 +0200 Christian K=C3=B6nig wrote: > Am 07.09.2018 um 17:45 schrieb Jean-Philippe Brucker: > > On 07/09/2018 09:55, Christian K=C3=B6nig wrote: =20 > >> I will take this as an opportunity to summarize some of the > >> requirements we have for PASID management from the amdgpu driver > >> point of view: =20 > > That's incredibly useful, thanks :) > > =20 > >> 1. We need to be able to allocate PASID between 1 and some > >> maximum. Zero is reserved as far as I know, but we don't necessary > >> need a minimum. =20 > [...] =20 > >> 2. We need to be able to allocate PASIDs without a process address > >> space backing it. E.g. our hardware uses PASIDs even without > >> Shared Virtual Addressing enabled to distinct clients from each > >> other. Would be a pity if we need to still have a separate PASID > >> handling because the system wide is only available when IOMMU is > >> turned on. =20 > [...] =20 >=20 > I agree on that. >=20 > > iommu-sva expects everywhere that the device has an iommu_domain, > > it's the first thing we check on entry. Bypassing all of this would > > call idr_alloc() directly, and wouldn't have any code in common > > with the current iommu-sva. So it seems like you need a layer on > > top of iommu-sva calling idr_alloc() when an IOMMU isn't present, > > but I don't think it should be in drivers/iommu/ =20 >=20 > In this case I question if the PASID handling should be under=20 > drivers/iommu at all. >=20 > See I can have a mix of VM context which are bound to processes (some=20 > few) and VM contexts which are standalone and doesn't care for a > process address space. But for each VM context I need a distinct > PASID for the hardware to work. >=20 > I can live if we say if IOMMU is completely disabled we use a simple > ida to allocate them, but when IOMMU is enabled I certainly need a > way to reserve a PASID without an associated process. >=20 VT-d would also have such requirement. There is a virtual command register for allocate and free PASID for VM use. When that PASID allocation request gets propagated to the host IOMMU driver, we need to allocate PASID w/o mm. If the PASID allocation is done via VFIO, can we have FD to track PASID life cycle instead of mm_exit()? i.e. all FDs get closed before mm_exit, I assume? > >> 3. Even after destruction of a process address space we need some > >> grace period before a PASID is reused because it can be that the > >> specific PASID is still in some hardware queues etc... > >> =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 At bare minimum all device dri= vers using process binding > >> need to explicitly note to the core when they are done with a > >> PASID. =20 > > Right, much of the horribleness in iommu-sva deals with this: > > > > The process dies, iommu-sva is notified and calls the mm_exit() > > function passed by the device driver to iommu_sva_device_init(). In > > mm_exit() the device driver needs to clear any reference to the > > PASID in hardware and in its own structures. When the device driver > > returns from mm_exit(), it effectively tells the core that it has > > finished using the PASID, and iommu-sva can reuse the PASID for > > another process. mm_exit() is allowed to block, so the device > > driver has time to clean up and flush the queues. > > > > If the device driver finishes using the PASID before the process > > exits, it just calls unbind(). =20 >=20 > Exactly that's what Michal Hocko is probably going to not like at all. >=20 > Can we have a different approach where each driver is informed by the=20 > mm_exit(), but needs to explicitly call unbind() before a PASID is > reused? >=20 > During that teardown transition it would be ideal if that PASID only=20 > points to a dummy root page directory with only invalid entries. >=20 I guess this can be vendor specific, In VT-d I plan to mark PASID entry not present and disable fault reporting while draining remaining activities. > > =20 > >> 4. It would be nice to have to be able to set a "void *" for each > >> PASID/device combination while binding to a process which then can > >> be queried later on based on the PASID. > >> =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 E.g. when you have a per PASID= /device structure around > >> anyway, just add an extra field. =20 > > iommu_sva_bind_device() takes a "drvdata" pointer that is stored > > internally for the PASID/device combination (iommu_bond). It is > > passed to mm_exit(), but I haven't added anything for the device > > driver to query it back. =20 >=20 > Nice! Looks like all we need additionally is a function to retrieve > that based on the PASID. >=20 > >> 5. It would be nice to have to allocate multiple PASIDs for the > >> same process address space. > >> =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 E.g. some teams at AMD want to= use a separate GPU > >> address space for their userspace client library. I'm still trying > >> to avoid that, but it is perfectly possible that we are going to > >> need that. =20 > > Two PASIDs pointing to the same process pgd? At first glance it > > seems feasible, maybe with a flag passed to bind() and a few > > changes to internal structures. It will duplicate ATC invalidation > > commands for each process address space change (munmap etc) so you > > might take a performance hit. > > > > Intel's SVM code has the SVM_FLAG_PRIVATE_PASID which seems similar > > to what you describe, but I don't plan to support it in this series > > (the io_mm model is already pretty complicated). I think it can be > > added without too much effort in a future series, though with a > > different flag name since we'd like to use "private PASID" for > > something else > > (https://www.spinics.net/lists/dri-devel/msg177007.html). =20 >=20 > To be honest I hoped that you would say: No never! So that I have a > good argument to pushback on such requirements :) >=20 > But if it's doable it would be at least nice to have for debugging. >=20 > Thanks a lot for working on that, > Christian. >=20 > > > > Thanks, > > Jean > > =20 > >> =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 Additional to that it is somet= imes quite useful for > >> debugging to isolate where exactly an incorrect access (segfault) > >> is coming from. > >> > >> Let me know if there are some problems with that, especially I > >> want to know if there is pushback on #5 so that I can forward > >> that :) > >> > >> Thanks, > >> Christian. > >> =20 > >>> Thanks, > >>> Jean =20 >=20 [Jacob Pan]