From mboxrd@z Thu Jan 1 00:00:00 1970 From: Kenneth Lee Subject: Re: [RFCv2 PATCH 0/7] A General Accelerator Framework, WarpDrive Date: Fri, 7 Sep 2018 12:01:38 +0800 Message-ID: <20180907040138.GI230707@Turing-Arch-b> References: <20180903005204.26041-1-nek.in.cn@gmail.com> <20180904150019.GA4024@redhat.com> <20180904101509.62314b67@t450s.home> <20180906094532.GG230707@Turing-Arch-b> <20180906133133.GA3830@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Cc: Kenneth Lee , Herbert Xu , kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Jonathan Corbet , Greg Kroah-Hartman , linux-doc-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Sanjay Kumar , Hao Fang , iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linuxarm-hv44wF8Li93QT0dZR+AlfA@public.gmane.org, Alex Williamson , Thomas Gleixner , linux-crypto-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Philippe Ombredanne , "David S . Miller" , linux-accelerators-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org To: Jerome Glisse Return-path: Content-Disposition: inline In-Reply-To: <20180906133133.GA3830-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org List-Id: linux-crypto.vger.kernel.org T24gVGh1LCBTZXAgMDYsIDIwMTggYXQgMDk6MzE6MzNBTSAtMDQwMCwgSmVyb21lIEdsaXNzZSB3 cm90ZToKPiBEYXRlOiBUaHUsIDYgU2VwIDIwMTggMDk6MzE6MzMgLTA0MDAKPiBGcm9tOiBKZXJv bWUgR2xpc3NlIDxqZ2xpc3NlQHJlZGhhdC5jb20+Cj4gVG86IEtlbm5ldGggTGVlIDxsaWd1b3po dUBoaXNpbGljb24uY29tPgo+IENDOiBBbGV4IFdpbGxpYW1zb24gPGFsZXgud2lsbGlhbXNvbkBy ZWRoYXQuY29tPiwgS2VubmV0aCBMZWUKPiAgPG5lay5pbi5jbkBnbWFpbC5jb20+LCBKb25hdGhh biBDb3JiZXQgPGNvcmJldEBsd24ubmV0PiwgSGVyYmVydCBYdQo+ICA8aGVyYmVydEBnb25kb3Iu YXBhbmEub3JnLmF1PiwgIkRhdmlkIFMgLiBNaWxsZXIiIDxkYXZlbUBkYXZlbWxvZnQubmV0PiwK PiAgSm9lcmcgUm9lZGVsIDxqb3JvQDhieXRlcy5vcmc+LCBIYW8gRmFuZyA8ZmFuZ2hhbzExQGh1 YXdlaS5jb20+LCBaaG91IFdhbmcKPiAgPHdhbmd6aG91MUBoaXNpbGljb24uY29tPiwgWmFpYm8g WHUgPHh1emFpYm9AaHVhd2VpLmNvbT4sIFBoaWxpcHBlCj4gIE9tYnJlZGFubmUgPHBvbWJyZWRh bm5lQG5leGIuY29tPiwgR3JlZyBLcm9haC1IYXJ0bWFuCj4gIDxncmVna2hAbGludXhmb3VuZGF0 aW9uLm9yZz4sIFRob21hcyBHbGVpeG5lciA8dGdseEBsaW51dHJvbml4LmRlPiwKPiAgbGludXgt ZG9jQHZnZXIua2VybmVsLm9yZywgbGludXgta2VybmVsQHZnZXIua2VybmVsLm9yZywKPiAgbGlu dXgtY3J5cHRvQHZnZXIua2VybmVsLm9yZywgaW9tbXVAbGlzdHMubGludXgtZm91bmRhdGlvbi5v cmcsCj4gIGt2bUB2Z2VyLmtlcm5lbC5vcmcsIGxpbnV4LWFjY2VsZXJhdG9yc0BsaXN0cy5vemxh YnMub3JnLCBMdSBCYW9sdQo+ICA8YmFvbHUubHVAbGludXguaW50ZWwuY29tPiwgU2FuamF5IEt1 bWFyIDxzYW5qYXkuay5rdW1hckBpbnRlbC5jb20+LAo+ICBsaW51eGFybUBodWF3ZWkuY29tCj4g U3ViamVjdDogUmU6IFtSRkN2MiBQQVRDSCAwLzddIEEgR2VuZXJhbCBBY2NlbGVyYXRvciBGcmFt ZXdvcmssIFdhcnBEcml2ZQo+IFVzZXItQWdlbnQ6IE11dHQvMS4xMC4wICgyMDE4LTA1LTE3KQo+ IE1lc3NhZ2UtSUQ6IDwyMDE4MDkwNjEzMzEzMy5HQTM4MzBAcmVkaGF0LmNvbT4KPiAKPiBPbiBU aHUsIFNlcCAwNiwgMjAxOCBhdCAwNTo0NTozMlBNICswODAwLCBLZW5uZXRoIExlZSB3cm90ZToK PiA+IE9uIFR1ZSwgU2VwIDA0LCAyMDE4IGF0IDEwOjE1OjA5QU0gLTA2MDAsIEFsZXggV2lsbGlh bXNvbiB3cm90ZToKPiA+ID4gRGF0ZTogVHVlLCA0IFNlcCAyMDE4IDEwOjE1OjA5IC0wNjAwCj4g PiA+IEZyb206IEFsZXggV2lsbGlhbXNvbiA8YWxleC53aWxsaWFtc29uQHJlZGhhdC5jb20+Cj4g PiA+IFRvOiBKZXJvbWUgR2xpc3NlIDxqZ2xpc3NlQHJlZGhhdC5jb20+Cj4gPiA+IENDOiBLZW5u ZXRoIExlZSA8bmVrLmluLmNuQGdtYWlsLmNvbT4sIEpvbmF0aGFuIENvcmJldCA8Y29yYmV0QGx3 bi5uZXQ+LAo+ID4gPiAgSGVyYmVydCBYdSA8aGVyYmVydEBnb25kb3IuYXBhbmEub3JnLmF1Piwg IkRhdmlkIFMgLiBNaWxsZXIiCj4gPiA+ICA8ZGF2ZW1AZGF2ZW1sb2Z0Lm5ldD4sIEpvZXJnIFJv ZWRlbCA8am9yb0A4Ynl0ZXMub3JnPiwgS2VubmV0aCBMZWUKPiA+ID4gIDxsaWd1b3podUBoaXNp bGljb24uY29tPiwgSGFvIEZhbmcgPGZhbmdoYW8xMUBodWF3ZWkuY29tPiwgWmhvdSBXYW5nCj4g PiA+ICA8d2FuZ3pob3UxQGhpc2lsaWNvbi5jb20+LCBaYWlibyBYdSA8eHV6YWlib0BodWF3ZWku Y29tPiwgUGhpbGlwcGUKPiA+ID4gIE9tYnJlZGFubmUgPHBvbWJyZWRhbm5lQG5leGIuY29tPiwg R3JlZyBLcm9haC1IYXJ0bWFuCj4gPiA+ICA8Z3JlZ2toQGxpbnV4Zm91bmRhdGlvbi5vcmc+LCBU aG9tYXMgR2xlaXhuZXIgPHRnbHhAbGludXRyb25peC5kZT4sCj4gPiA+ICBsaW51eC1kb2NAdmdl ci5rZXJuZWwub3JnLCBsaW51eC1rZXJuZWxAdmdlci5rZXJuZWwub3JnLAo+ID4gPiAgbGludXgt Y3J5cHRvQHZnZXIua2VybmVsLm9yZywgaW9tbXVAbGlzdHMubGludXgtZm91bmRhdGlvbi5vcmcs Cj4gPiA+ICBrdm1Admdlci5rZXJuZWwub3JnLCBsaW51eC1hY2NlbGVyYXRvcnNAbGlzdHMub3ps YWJzLm9yZywgTHUgQmFvbHUKPiA+ID4gIDxiYW9sdS5sdUBsaW51eC5pbnRlbC5jb20+LCBTYW5q YXkgS3VtYXIgPHNhbmpheS5rLmt1bWFyQGludGVsLmNvbT4sCj4gPiA+ICBsaW51eGFybUBodWF3 ZWkuY29tCj4gPiA+IFN1YmplY3Q6IFJlOiBbUkZDdjIgUEFUQ0ggMC83XSBBIEdlbmVyYWwgQWNj ZWxlcmF0b3IgRnJhbWV3b3JrLCBXYXJwRHJpdmUKPiA+ID4gTWVzc2FnZS1JRDogPDIwMTgwOTA0 MTAxNTA5LjYyMzE0YjY3QHQ0NTBzLmhvbWU+Cj4gPiA+IAo+ID4gPiBPbiBUdWUsIDQgU2VwIDIw MTggMTE6MDA6MTkgLTA0MDAKPiA+ID4gSmVyb21lIEdsaXNzZSA8amdsaXNzZUByZWRoYXQuY29t PiB3cm90ZToKPiA+ID4gCj4gPiA+ID4gT24gTW9uLCBTZXAgMDMsIDIwMTggYXQgMDg6NTE6NTdB TSArMDgwMCwgS2VubmV0aCBMZWUgd3JvdGU6Cj4gPiA+ID4gPiBGcm9tOiBLZW5uZXRoIExlZSA8 bGlndW96aHVAaGlzaWxpY29uLmNvbT4KPiA+ID4gPiA+IAo+ID4gPiA+ID4gV2FycERyaXZlIGlz IGFuIGFjY2VsZXJhdG9yIGZyYW1ld29yayB0byBleHBvc2UgdGhlIGhhcmR3YXJlIGNhcGFiaWxp dGllcwo+ID4gPiA+ID4gZGlyZWN0bHkgdG8gdGhlIHVzZXIgc3BhY2UuIEl0IG1ha2VzIHVzZSBv ZiB0aGUgZXhpc3QgdmZpbyBhbmQgdmZpby1tZGV2Cj4gPiA+ID4gPiBmYWNpbGl0aWVzLiBTbyB0 aGUgdXNlciBhcHBsaWNhdGlvbiBjYW4gc2VuZCByZXF1ZXN0IGFuZCBETUEgdG8gdGhlCj4gPiA+ ID4gPiBoYXJkd2FyZSB3aXRob3V0IGludGVyYWN0aW9uIHdpdGggdGhlIGtlcm5lbC4gVGhpcyBy ZW1vdmVzIHRoZSBsYXRlbmN5Cj4gPiA+ID4gPiBvZiBzeXNjYWxsLgo+ID4gPiA+ID4gCj4gPiA+ ID4gPiBXYXJwRHJpdmUgaXMgdGhlIG5hbWUgZm9yIHRoZSB3aG9sZSBmcmFtZXdvcmsuIFRoZSBj b21wb25lbnQgaW4ga2VybmVsCj4gPiA+ID4gPiBpcyBjYWxsZWQgU0RNREVWLCBTaGFyZSBEb21h aW4gTWVkaWF0ZWQgRGV2aWNlLiBEcml2ZXIgZHJpdmVyIGV4cG9zZXMgaXRzCj4gPiA+ID4gPiBo YXJkd2FyZSByZXNvdXJjZSBieSByZWdpc3RlcmluZyB0byBTRE1ERVYgYXMgYSBWRklPLU1kZXYu IFNvIHRoZSB1c2VyCj4gPiA+ID4gPiBsaWJyYXJ5IG9mIFdhcnBEcml2ZSBjYW4gYWNjZXNzIGl0 IHZpYSBWRklPIGludGVyZmFjZS4KPiA+ID4gPiA+IAo+ID4gPiA+ID4gVGhlIHBhdGNoc2V0IGNv bnRhaW5zIGRvY3VtZW50IGZvciB0aGUgZGV0YWlsLiBQbGVhc2UgcmVmZXIgdG8gaXQgZm9yIG1v cmUKPiA+ID4gPiA+IGluZm9ybWF0aW9uLgo+ID4gPiA+ID4gCj4gPiA+ID4gPiBUaGlzIHBhdGNo c2V0IGlzIGludGVuZGVkIHRvIGJlIHVzZWQgd2l0aCBKZWFuIFBoaWxpcHBlIEJydWNrZXIncyBT VkEKPiA+ID4gPiA+IHBhdGNoIFsxXSwgd2hpY2ggZW5hYmxlcyBub3Qgb25seSBJTyBzaWRlIHBh Z2UgZmF1bHQsIGJ1dCBhbHNvIFBBU0lECj4gPiA+ID4gPiBzdXBwb3J0IHRvIElPTU1VIGFuZCBW RklPLgo+ID4gPiA+ID4gCj4gPiA+ID4gPiBXaXRoIHRoZXNlIGZlYXR1cmVzLCBXYXJwRHJpdmUg Y2FuIHN1cHBvcnQgbm9uLXBpbm5lZCBtZW1vcnkgYW5kCj4gPiA+ID4gPiBtdWx0aS1wcm9jZXNz IGluIHRoZSBzYW1lIGFjY2VsZXJhdG9yIGRldmljZS4gIFdlIHRlc3RlZCBpdCBpbiBvdXIgU29D Cj4gPiA+ID4gPiBpbnRlZ3JhdGVkIEFjY2VsZXJhdG9yIChib2FyZCBJRDogRDA2LCBDaGlwIElE OiBISVAwOCkuIEEgcmVmZXJlbmNlIHdvcmsKPiA+ID4gPiA+IHRyZWUgY2FuIGJlIGZvdW5kIGhl cmU6IFsyXS4KPiA+ID4gPiA+IAo+ID4gPiA+ID4gQnV0IGl0IGlzIG5vdCBtYW5kYXRvcnkuIFRo aXMgcGF0Y2hzZXQgaXMgdGVzdGVkIGluIHRoZSBsYXRlc3QgbWFpbmxpbmUKPiA+ID4gPiA+IGtl cm5lbCB3aXRob3V0IHRoZSBTVkEgcGF0Y2hlcy4gIFNvIGl0IHN1cHBvcnRzIG9ubHkgb25lIHBy b2Nlc3MgZm9yIGVhY2gKPiA+ID4gPiA+IGFjY2VsZXJhdG9yLgo+ID4gPiA+ID4gCj4gPiA+ID4g PiBXZSBoYXZlIG5vdGljZWQgdGhlIElPTU1VIGF3YXJlIG1kZXYgUkZDIGFubm91bmNlZCByZWNl bnRseSBbM10uCj4gPiA+ID4gPiAKPiA+ID4gPiA+IFRoZSBJT01NVSBhd2FyZSBtZGV2IGhhcyBz aW1pbGFyIGlkZWEgYnV0IGRpZmZlcmVudCBpbnRlbnRpb24gY29tcGFyaW5nIHRvCj4gPiA+ID4g PiBXYXJwRHJpdmUuIEl0IGludGVuZHMgdG8gZGVkaWNhdGUgcGFydCBvZiB0aGUgaGFyZHdhcmUg cmVzb3VyY2UgdG8gYSBWTS4KPiA+ID4gPiA+IEFuZCB0aGUgZGVzaWduIGlzIHN1cHBvc2VkIHRv IGJlIHVzZWQgd2l0aCBTY2FsYWJsZSBJL08gVmlydHVhbGl6YXRpb24uCj4gPiA+ID4gPiBXaGls ZSBzZG1kZXYgaXMgaW50ZW5kZWQgdG8gc2hhcmUgdGhlIGhhcmR3YXJlIHJlc291cmNlIHdpdGgg YSBiaWcgYW1vdW50Cj4gPiA+ID4gPiBvZiBwcm9jZXNzZXMuICBJdCBqdXN0IHJlcXVpcmVzIHRo ZSBoYXJkd2FyZSBzdXBwb3J0aW5nIGFkZHJlc3MKPiA+ID4gPiA+IHRyYW5zbGF0aW9uIHBlciBw cm9jZXNzIChQQ0lFJ3MgUEFTSUQgb3IgQVJNIFNNTVUncyBzdWJzdHJlYW0gSUQpLgo+ID4gPiA+ ID4gCj4gPiA+ID4gPiBCdXQgd2UgZG9uJ3Qgc2VlIHNlcmlvdXMgY29uZmxpY3Rpb24gb24gYm90 aCBkZXNpZ24uIFdlIGJlbGlldmUgdGhleSBjYW4gYmUKPiA+ID4gPiA+IG5vcm1hbGl6ZWQgYXMg b25lLgo+ID4gPiA+ID4gICAKPiA+ID4gPiAKPiA+ID4gPiBTbyBvbmNlIGFnYWluIGkgZG8gbm90 IHVuZGVyc3RhbmQgd2h5IHlvdSBhcmUgdHJ5aW5nIHRvIGRvIHRoaW5ncwo+ID4gPiA+IHRoaXMg d2F5LiBLZXJuZWwgYWxyZWFkeSBoYXZlIHRvbnMgb2YgZXhhbXBsZSBvZiBldmVyeXRoaW5nIHlv dQo+ID4gPiA+IHdhbnQgdG8gZG8gd2l0aG91dCBhIG5ldyBmcmFtZXdvcmsuIE1vcmVvdmVyIGkg YmVsaWV2ZSB5b3UgYXJlCj4gPiA+ID4gY29uZnVzZSBieSBWRklPLiBUbyBtZSBWRklPIGlzIGZv ciBWTSBub3QgdG8gY3JlYXRlIGdlbmVyYWwgZGV2aWNlCj4gPiA+ID4gZHJpdmVyIGZyYW1lIHdv cmsuCj4gPiA+IAo+ID4gPiBWRklPIGlzIGEgdXNlcnNwYWNlIGRyaXZlciBmcmFtZXdvcmssIHRo ZSBWTSB1c2UgY2FzZSBqdXN0IGhhcHBlbnMgdG8KPiA+ID4gYmUgYSByYXRoZXIgcHJvbGlmaWMg b25lLiAgVkZJTyB3YXMgbmV2ZXIgaW50ZW5kZWQgdG8gYmUgc29sZWx5IGEgVk0KPiA+ID4gZGV2 aWNlIGludGVyZmFjZSBhbmQgaGFzIHNldmVyYWwgb3RoZXIgdXNlcnNwYWNlIHVzZXJzLCBub3Rh Ymx5IERQREsKPiA+ID4gYW5kIFNQREssIGFuIE5WTWUgYmFja2VuZCBpbiBRRU1VLCBhIHVzZXJz cGFjZSBOVk1lIGRyaXZlciwgYSBydWJ5Cj4gPiA+IHdyYXBwZXIsIGFuZCBwZXJoYXBzIG90aGVy cyB0aGF0IEknbSBub3QgYXdhcmUgb2YuICBXaGV0aGVyIHZmaW8gaXMKPiA+ID4gYXBwcm9wcmlh dGUgaW50ZXJmYWNlIGhlcmUgbWlnaHQgY2VydGFpbmx5IHN0aWxsIGJlIGEgZGViYXRhYmxlIHRv cGljLAo+ID4gPiBidXQgSSB3b3VsZCBzdHJvbmdseSBkaXNhZ3JlZSB3aXRoIHlvdXIgbGFzdCBz ZW50ZW5jZSBhYm92ZS4gIFRoYW5rcywKPiA+ID4gCj4gPiA+IEFsZXgKPiA+ID4gCj4gPiAKPiA+ IFllcywgdGhhdCBpcyBhbHNvIG15IHN0YW5kcG9pbnQgaGVyZS4KPiA+IAo+ID4gPiA+IFNvIGhl cmUgaXMgeW91ciB1c2UgY2FzZSBhcyBpIHVuZGVyc3RhbmQgaXQuIFlvdSBoYXZlIGEgZGV2aWNl Cj4gPiA+ID4gd2l0aCBhIGxpbWl0ZWQgbnVtYmVyIG9mIGNvbW1hbmQgcXVldWVzIChjYW4gYmUg anVzdCBvbmUpIGFuZCBpbgo+ID4gPiA+IHNvbWUgY2FzZSBpdCBjYW4gc3VwcG9ydCBTVkEvU1ZN ICh3aGVuIGhhcmR3YXJlIHN1cHBvcnQgaXQgYW5kIGl0Cj4gPiA+ID4gaXMgbm90IGRpc2FibGVk KS4gRmluYWwgcmVxdWlyZW1lbnQgaXMgYmVpbmcgYWJsZSB0byBzY2hlZHVsZSBjbWRzCj4gPiA+ ID4gZnJvbSB1c2Vyc3BhY2Ugd2l0aG91dCBpb2N0bC4gQWxsIG9mIHRoaXMgZXhpc3RzIGFscmVh ZHkgZXhpc3RzCj4gPiA+ID4gdXBzdHJlYW0gaW4gZmV3IGRldmljZSBkcml2ZXJzLgo+ID4gPiA+ IAo+ID4gPiA+IAo+ID4gPiA+IFNvIGhlcmUgaXMgaG93IGV2ZXJ5IGJvZHkgZWxzZSBpcyBkb2lu ZyBpdC4gUGxlYXNlIGV4cGxhaW4gd2h5Cj4gPiA+ID4gdGhpcyBkb2VzIG5vdCB3b3JrLgo+ID4g PiA+IAo+ID4gPiA+IDEgVXNlcnNwYWNlIG9wZW4gZGV2aWNlIGZpbGUgZHJpdmVyLiBLZXJuZWwg ZGV2aWNlIGRyaXZlciBjcmVhdGUKPiA+ID4gPiAgIGEgY29udGV4dCBhbmQgYXNzb2NpYXRlIGl0 IHdpdGggb24gb3Blbi4gVGhpcyBjb250ZXh0IGNhbiBiZQo+ID4gPiA+ICAgdW5pcSB0byB0aGUg cHJvY2VzcyBhbmQgY2FuIGJpbmQgaGFyZHdhcmUgcmVzb3VyY2VzIChsaWtlIGEKPiA+ID4gPiAg IGNvbW1hbmQgcXVldWUpIHRvIHRoZSBwcm9jZXNzLgo+ID4gPiA+IDIgVXNlcnNwYWNlIGJpbmQv YWNxdWlyZSBhIGNvbW1hbmRzIHF1ZXVlIGFuZCBpbml0aWFsaXplIGl0IHdpdGgKPiA+ID4gPiAg IGFuIGlvY3RsIG9uIHRoZSBkZXZpY2UgZmlsZS4gVGhyb3VnaCB0aGF0IGlvY3RsIHVzZXJzcGFj ZSBjYW4KPiA+ID4gPiAgIGJlIGluZm9ybSB3ZXRoZXIgZWl0aGVyIFNWQS9TVk0gd29ya3MgZm9y IHRoZSBkZXZpY2UuIElmIFNWQS8KPiA+ID4gPiAgIFNWTSB3b3JrcyB0aGVuIGtlcm5lbCBkZXZp Y2UgZHJpdmVyIGJpbmQgdGhlIHByb2Nlc3MgdG8gdGhlCj4gPiA+ID4gICBkZXZpY2UgYXMgcGFy dCBvZiB0aGlzIGlvY3RsLgo+ID4gPiA+IDMgSWYgU1ZNL1NWQSBkb2VzIG5vdCB3b3JrIHVzZXJz cGFjZSBkbyBhbiBpb2N0bCB0byBjcmVhdGUgZG1hCj4gPiA+ID4gICBidWZmZXIgb3Igc29tZXRo aW5nIHRoYXQgZG9lcyBleGFjdGx5IHRoZSBzYW1lIHRoaW5nLgo+ID4gPiA+IDQgVXNlcnNwYWNl IG1tYXAgdGhlIGNvbW1hbmQgcXVldWUgKG1tYXAgb2YgdGhlIGRldmljZSBmaWxlIGJ5Cj4gPiA+ ID4gICB1c2luZyBpbmZvcm1hdGlvbnMgZ2F0aGVyIGF0IHN0ZXAgMikKPiA+ID4gPiA1IFVzZXJz cGFjZSBjYW4gd3JpdGUgY29tbWFuZHMgaW50byB0aGUgcXVldWUgaXQgbWFwcGVkCj4gPiA+ID4g NiBXaGVuIHVzZXJzcGFjZSBjbG9zZSB0aGUgZGV2aWNlIGZpbGUgYWxsIHJlc291cmNlcyBhcmUg cmVsZWFzZQo+ID4gPiA+ICAganVzdCBsaWtlIGFueSBleGlzdGluZyBkZXZpY2UgZHJpdmVycy4K PiA+IAo+ID4gSGksIEplcm9tZSwKPiA+IAo+ID4gSnVzdCBvbmUgdGhpbmcsIGFzIEkgc2FpZCBp biB0aGUgY292ZXIgbGV0dGVyLCBkbWEtYnVmIHJlcXVpcmVzIHRoZSBhcHBsaWNhdGlvbgo+ID4g dG8gdXNlIG1lbW9yeSBjcmVhdGVkIGJ5IHRoZSBkcml2ZXIgZm9yIERNQS4gSSBkaWQgdHJ5IHRo ZSBkbWEtYnVmIHdheSBpbgo+ID4gV3JhcERyaXZlIChyZWZlciB0byBbNF0gaW4gdGhlIGNvdmVy IGxldHRlciksIGl0IGlzIGEgZ29vZCBiYWNrdXAgZm9yIE5PSU9NTVUKPiA+IG1vZGUgb3Igd2Ug Y2Fubm90IHNvbHZlIHRoZSBwcm9ibGVtIGluIFZGSU8uCj4gPiAKPiA+IEJ1dCwgaW4gbWFueSBv ZiBteSBhcHBsaWNhdGlvbiBzY2VuYXJpbywgdGhlIGFwcGxpY2F0aW9uIGFscmVhZHkgaGFzIHNv bWUgbWVtb3J5Cj4gPiBpbiBoYW5kLCBtYXliZSBhbGxvY2F0ZWQgYnkgdGhlIGZyYW1ld29yayBv ciBsaWJyYXJpZXMuIEFueXdheSwgdGhleSBkb24ndCBnZXQKPiA+IG1lbW9yeSBmcm9tIG15IGxp YnJhcnksIGFuZCB0aGV5IHBhc3MgdGhlIHBvaXRlciBmb3IgZGF0YSBvcGVyYXRpb24uIEFuZCB0 aGV5Cj4gPiBtYXkgYWxzbyBoYXZlIHBvaW50ZXIgaW4gdGhlIGJ1ZmZlci4gVGhvc2UgcG9pbnRl ciBtYXkgYmUgdXNlZCBieSB0aGUKPiA+IGFjY2VsZXJhdG9yLiBTbyBJIG5lZWQgaGFyZHdhcmUg ZnVsbHkgc2hhcmUgdGhlIGFkZHJlc3Mgc3BhY2Ugd2l0aCB0aGUKPiA+IGFwcGxpY2F0aW9uLiBU aGF0IGlzIHdoYXQgZG1hYnVmIGNhbm5vdCBkby4KPiAKPiBkbWFidWYgY2FuIGRvIHRoYXQgLi4u IGl0IGlzIGNhbGwgdXB0ciB5b3UgY2FuIGxvb2sgYXQgaTkxNSBmb3IKPiBpbnN0YW5jZS4gU3Rp bGwgdGhpcyBkb2VzIG5vdCBhbnN3ZXIgbXkgcXVlc3Rpb24gYWJvdmUsIHdoeSBkbwo+IHlvdSBu ZWVkIHRvIGJlIGluIFZGSU8gdG8gZG8gYW55IG9mIHRoZSBhYm92ZSB0aGluZyA/IEtlcm5lbCBo YXMKPiB0b25zIG9mIGV4YW1wbGVzIHRoYXQgZG9lcyBhbGwgb2YgdGhlIGFib3ZlIGFuZCBhcmUg bm90IGluIFZGSU8KPiAoaW5jbHVkaW5nIHVzaW5uZyBleGlzdGluZyB1c2VyIHBvaW50ZXIgd2l0 aCBkZXZpY2UpLgo+IAo+IENoZWVycywKPiBKw6lyw7RtZQoKSSB0b29rIGEgbG9vayBhdCBpOTE1 X2dlbV9leGVjYnVmZmVyX2lvY3RsKCkuIEl0IHNlZW1zIGl0ICJjb3B5X2Zyb21fdXNlciIgdGhl CnVzZXIgbWVtb3J5IHRvIHRoZSBrZXJuZWwuIFRoYXQgaXMgbm90IHdoYXQgd2UgbmVlZC4gV2hh dCB3ZSB0cnkgdG8gZ2V0IGlzOiB0aGUKdXNlciBhcHBsaWNhdGlvbiBkbyBzb21ldGhpbmcgb24g aXRzIGRhdGEsIGFuZCBwdXNoIGl0IGF3YXkgdG8gdGhlIGFjY2VsZXJhdG9yLAphbmQgc2F5czog IkknbSB0aWVkLCBpdCBpcyB5b3VyIHR1cm4gdG8gZG8gdGhlIGpvYi4uLiIuIFRoZW4gdGhlIGFj Y2VsZXJhdG9yIGhhcwp0aGUgbWVtb3J5LCByZWZlcnJpbmcgYW55IHBvcnRpb24gb2YgaXQgd2l0 aCB0aGUgc2FtZSBWQXMgb2YgdGhlIGFwcGxpY2F0aW9uLApldmVuIHRoZSBWQXMgYXJlIHN0b3Jl ZCBpbnNpZGUgdGhlIG1lbW9yeSBpdHNlbGYuCgpBbmQgSSBkb24ndCB1bmRlcnN0YW5kIHdoeSBJ IHNob3VsZCBhdm9pZCB0byB1c2UgVkZJTz8gQXMgQWxleCBzYWlkLCBWRklPIGlzIHRoZQp1c2Vy IGRyaXZlciBmcmFtZXdvcmsuIEFuZCBJIG5lZWQgZXhhY3RseSBhIHVzZXIgZHJpdmVyIGludGVy ZmFjZS4gV2h5IHNob3VsZCBJCmludmVudCBhbm90aGVyIHdoZWVsPyBJdCBoYXMgbW9zdCBvZiBz dHVmZiBJIG5lZWQ6CgoxLiBDb25uZWN0aW5nIG11bHRpcGxlIGRldmljZXMgdG8gdGhlIHNhbWUg YXBwbGljYXRpb24gc3BhY2UKMi4gUGlubmluZyBhbmQgRE1BIGZyb20gdGhlIGFwcGxpY2F0aW9u IHNwYWNlIHRvIHRoZSB3aG9sZSBzZXQgb2YgZGV2aWNlCjMuIE1hbmFnaW5nIGhhcmR3YXJlIHJl c291cmNlIGJ5IGRldmljZQoKV2UganVzdCBuZWVkIHRoZSBsYXN0IHN0ZXA6IG1ha2Ugc3VyZSBt dWx0aXBsZSBhcHBsaWNhdGlvbnMgYW5kIHRoZSBrZXJuZWwgY2FuCnNoYXJlIHRoZSBzYW1lIElP TU1VLiBUaGVuIHdoeSBzaG91bGRuJ3Qgd2UgdXNlIFZGSU8/CgpBbmQgcGVyc29uYWxseSwgSSBi ZWxpZXZlIHRoZSBtYXR1cml0eSBhbmQgY29ycmVjdG5lc3Mgb2YgYSBmcmFtZXdvcmsgYXJlIGRy aXZlbgpieSBhcHBsaWNhdGlvbnMuIE5vdyB0aGUgcHJvYmxlbSBpbiBhY2NlbGVyYXRvciB3b3Js ZCBpcyB0aGF0IHdlIGRvbid0IGhhdmUgYQpkaXJlY3Rpb24uIElmIHdlIGJlbGlldmUgdGhlIHJl cXVpcmVtZW50IGlzIHJpZ2h0LCB0aGUgbWV0aG9kIGl0c2VsZiBpcyBub3QgYQpiaWcgcHJvYmxl bSBpbiB0aGUgZW5kLiBXZSBqdXN0IG5lZWQgdG8gbGV0IHBlb3BsZSBoYXZlIGEgdW5pZnkgcGxh dGZvcm0gdG8Kc2hhcmUgdGhlaXIgd29yayB0b2dldGhlci4KCkNoZWVycwotLSAKCQkJLUtlbm5l dGgoSGlzaWxpY29uKQoKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0K5pys6YKu5Lu25Y+K5YW26ZmE 5Lu25ZCr5pyJ5Y2O5Li65YWs5Y+455qE5L+d5a+G5L+h5oGv77yM5LuF6ZmQ5LqO5Y+R6YCB57uZ 5LiK6Z2i5Zyw5Z2A5Lit5YiX5Ye655qE5Liq5Lq65oiW576k57uE44CC56aBCuatouS7u+S9leWF tuS7luS6uuS7peS7u+S9leW9ouW8j+S9v+eUqO+8iOWMheaLrOS9huS4jemZkOS6juWFqOmDqOaI lumDqOWIhuWcsOazhOmcsuOAgeWkjeWItuOAgeaIluaVo+WPke+8ieacrOmCruS7tuS4rQrnmoTk v6Hmga/jgILlpoLmnpzmgqjplJnmlLbkuobmnKzpgq7ku7bvvIzor7fmgqjnq4vljbPnlLXor53m iJbpgq7ku7bpgJrnn6Xlj5Hku7bkurrlubbliKDpmaTmnKzpgq7ku7bvvIEKVGhpcyBlLW1haWwg YW5kIGl0cyBhdHRhY2htZW50cyBjb250YWluIGNvbmZpZGVudGlhbCBpbmZvcm1hdGlvbiBmcm9t IEhVQVdFSSwKd2hpY2ggaXMgaW50ZW5kZWQgb25seSBmb3IgdGhlIHBlcnNvbiBvciBlbnRpdHkg d2hvc2UgYWRkcmVzcyBpcyBsaXN0ZWQgYWJvdmUuCkFueSB1c2Ugb2YgdGhlIAppbmZvcm1hdGlv biBjb250YWluZWQgaGVyZWluIGluIGFueSB3YXkgKGluY2x1ZGluZywgYnV0IG5vdCBsaW1pdGVk IHRvLCB0b3RhbCBvcgpwYXJ0aWFsIGRpc2Nsb3N1cmUsIHJlcHJvZHVjdGlvbiwgb3IgZGlzc2Vt aW5hdGlvbikgYnkgcGVyc29ucyBvdGhlciB0aGFuIHRoZQppbnRlbmRlZCAKcmVjaXBpZW50KHMp IGlzIHByb2hpYml0ZWQuIElmIHlvdSByZWNlaXZlIHRoaXMgZS1tYWlsIGluIGVycm9yLCBwbGVh c2Ugbm90aWZ5CnRoZSBzZW5kZXIgYnkgcGhvbmUgb3IgZW1haWwgaW1tZWRpYXRlbHkgYW5kIGRl bGV0ZSBpdCEKCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f CmlvbW11IG1haWxpbmcgbGlzdAppb21tdUBsaXN0cy5saW51eC1mb3VuZGF0aW9uLm9yZwpodHRw czovL2xpc3RzLmxpbnV4Zm91bmRhdGlvbi5vcmcvbWFpbG1hbi9saXN0aW5mby9pb21tdQ== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on archive.lwn.net X-Spam-Level: X-Spam-Status: No, score=-5.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable autolearn_force=no version=3.4.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by archive.lwn.net (Postfix) with ESMTP id AF03C7D57F for ; Fri, 7 Sep 2018 04:03:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726129AbeIGImd (ORCPT ); Fri, 7 Sep 2018 04:42:33 -0400 Received: from szxga06-in.huawei.com ([45.249.212.32]:48656 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725931AbeIGImc (ORCPT ); Fri, 7 Sep 2018 04:42:32 -0400 Received: from DGGEMS403-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id 03710FFF01337; Fri, 7 Sep 2018 12:03:31 +0800 (CST) Received: from localhost (10.67.212.75) by DGGEMS403-HUB.china.huawei.com (10.3.19.203) with Microsoft SMTP Server (TLS) id 14.3.399.0; Fri, 7 Sep 2018 12:03:27 +0800 Date: Fri, 7 Sep 2018 12:01:38 +0800 From: Kenneth Lee To: Jerome Glisse CC: Alex Williamson , Kenneth Lee , Jonathan Corbet , Herbert Xu , "David S . Miller" , Joerg Roedel , Hao Fang , Zhou Wang , Zaibo Xu , Philippe Ombredanne , Greg Kroah-Hartman , Thomas Gleixner , , , , , , , Lu Baolu , Sanjay Kumar , Subject: Re: [RFCv2 PATCH 0/7] A General Accelerator Framework, WarpDrive Message-ID: <20180907040138.GI230707@Turing-Arch-b> References: <20180903005204.26041-1-nek.in.cn@gmail.com> <20180904150019.GA4024@redhat.com> <20180904101509.62314b67@t450s.home> <20180906094532.GG230707@Turing-Arch-b> <20180906133133.GA3830@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20180906133133.GA3830@redhat.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-Originating-IP: [10.67.212.75] X-CFilter-Loop: Reflected Sender: linux-doc-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-doc@vger.kernel.org On Thu, Sep 06, 2018 at 09:31:33AM -0400, Jerome Glisse wrote: > Date: Thu, 6 Sep 2018 09:31:33 -0400 > From: Jerome Glisse > To: Kenneth Lee > CC: Alex Williamson , Kenneth Lee > , Jonathan Corbet , Herbert Xu > , "David S . Miller" , > Joerg Roedel , Hao Fang , Zhou Wang > , Zaibo Xu , Philippe > Ombredanne , Greg Kroah-Hartman > , Thomas Gleixner , > linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, > linux-crypto@vger.kernel.org, iommu@lists.linux-foundation.org, > kvm@vger.kernel.org, linux-accelerators@lists.ozlabs.org, Lu Baolu > , Sanjay Kumar , > linuxarm@huawei.com > Subject: Re: [RFCv2 PATCH 0/7] A General Accelerator Framework, WarpDrive > User-Agent: Mutt/1.10.0 (2018-05-17) > Message-ID: <20180906133133.GA3830@redhat.com> > > On Thu, Sep 06, 2018 at 05:45:32PM +0800, Kenneth Lee wrote: > > On Tue, Sep 04, 2018 at 10:15:09AM -0600, Alex Williamson wrote: > > > Date: Tue, 4 Sep 2018 10:15:09 -0600 > > > From: Alex Williamson > > > To: Jerome Glisse > > > CC: Kenneth Lee , Jonathan Corbet , > > > Herbert Xu , "David S . Miller" > > > , Joerg Roedel , Kenneth Lee > > > , Hao Fang , Zhou Wang > > > , Zaibo Xu , Philippe > > > Ombredanne , Greg Kroah-Hartman > > > , Thomas Gleixner , > > > linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, > > > linux-crypto@vger.kernel.org, iommu@lists.linux-foundation.org, > > > kvm@vger.kernel.org, linux-accelerators@lists.ozlabs.org, Lu Baolu > > > , Sanjay Kumar , > > > linuxarm@huawei.com > > > Subject: Re: [RFCv2 PATCH 0/7] A General Accelerator Framework, WarpDrive > > > Message-ID: <20180904101509.62314b67@t450s.home> > > > > > > On Tue, 4 Sep 2018 11:00:19 -0400 > > > Jerome Glisse wrote: > > > > > > > On Mon, Sep 03, 2018 at 08:51:57AM +0800, Kenneth Lee wrote: > > > > > From: Kenneth Lee > > > > > > > > > > WarpDrive is an accelerator framework to expose the hardware capabilities > > > > > directly to the user space. It makes use of the exist vfio and vfio-mdev > > > > > facilities. So the user application can send request and DMA to the > > > > > hardware without interaction with the kernel. This removes the latency > > > > > of syscall. > > > > > > > > > > WarpDrive is the name for the whole framework. The component in kernel > > > > > is called SDMDEV, Share Domain Mediated Device. Driver driver exposes its > > > > > hardware resource by registering to SDMDEV as a VFIO-Mdev. So the user > > > > > library of WarpDrive can access it via VFIO interface. > > > > > > > > > > The patchset contains document for the detail. Please refer to it for more > > > > > information. > > > > > > > > > > This patchset is intended to be used with Jean Philippe Brucker's SVA > > > > > patch [1], which enables not only IO side page fault, but also PASID > > > > > support to IOMMU and VFIO. > > > > > > > > > > With these features, WarpDrive can support non-pinned memory and > > > > > multi-process in the same accelerator device. We tested it in our SoC > > > > > integrated Accelerator (board ID: D06, Chip ID: HIP08). A reference work > > > > > tree can be found here: [2]. > > > > > > > > > > But it is not mandatory. This patchset is tested in the latest mainline > > > > > kernel without the SVA patches. So it supports only one process for each > > > > > accelerator. > > > > > > > > > > We have noticed the IOMMU aware mdev RFC announced recently [3]. > > > > > > > > > > The IOMMU aware mdev has similar idea but different intention comparing to > > > > > WarpDrive. It intends to dedicate part of the hardware resource to a VM. > > > > > And the design is supposed to be used with Scalable I/O Virtualization. > > > > > While sdmdev is intended to share the hardware resource with a big amount > > > > > of processes. It just requires the hardware supporting address > > > > > translation per process (PCIE's PASID or ARM SMMU's substream ID). > > > > > > > > > > But we don't see serious confliction on both design. We believe they can be > > > > > normalized as one. > > > > > > > > > > > > > So once again i do not understand why you are trying to do things > > > > this way. Kernel already have tons of example of everything you > > > > want to do without a new framework. Moreover i believe you are > > > > confuse by VFIO. To me VFIO is for VM not to create general device > > > > driver frame work. > > > > > > VFIO is a userspace driver framework, the VM use case just happens to > > > be a rather prolific one. VFIO was never intended to be solely a VM > > > device interface and has several other userspace users, notably DPDK > > > and SPDK, an NVMe backend in QEMU, a userspace NVMe driver, a ruby > > > wrapper, and perhaps others that I'm not aware of. Whether vfio is > > > appropriate interface here might certainly still be a debatable topic, > > > but I would strongly disagree with your last sentence above. Thanks, > > > > > > Alex > > > > > > > Yes, that is also my standpoint here. > > > > > > So here is your use case as i understand it. You have a device > > > > with a limited number of command queues (can be just one) and in > > > > some case it can support SVA/SVM (when hardware support it and it > > > > is not disabled). Final requirement is being able to schedule cmds > > > > from userspace without ioctl. All of this exists already exists > > > > upstream in few device drivers. > > > > > > > > > > > > So here is how every body else is doing it. Please explain why > > > > this does not work. > > > > > > > > 1 Userspace open device file driver. Kernel device driver create > > > > a context and associate it with on open. This context can be > > > > uniq to the process and can bind hardware resources (like a > > > > command queue) to the process. > > > > 2 Userspace bind/acquire a commands queue and initialize it with > > > > an ioctl on the device file. Through that ioctl userspace can > > > > be inform wether either SVA/SVM works for the device. If SVA/ > > > > SVM works then kernel device driver bind the process to the > > > > device as part of this ioctl. > > > > 3 If SVM/SVA does not work userspace do an ioctl to create dma > > > > buffer or something that does exactly the same thing. > > > > 4 Userspace mmap the command queue (mmap of the device file by > > > > using informations gather at step 2) > > > > 5 Userspace can write commands into the queue it mapped > > > > 6 When userspace close the device file all resources are release > > > > just like any existing device drivers. > > > > Hi, Jerome, > > > > Just one thing, as I said in the cover letter, dma-buf requires the application > > to use memory created by the driver for DMA. I did try the dma-buf way in > > WrapDrive (refer to [4] in the cover letter), it is a good backup for NOIOMMU > > mode or we cannot solve the problem in VFIO. > > > > But, in many of my application scenario, the application already has some memory > > in hand, maybe allocated by the framework or libraries. Anyway, they don't get > > memory from my library, and they pass the poiter for data operation. And they > > may also have pointer in the buffer. Those pointer may be used by the > > accelerator. So I need hardware fully share the address space with the > > application. That is what dmabuf cannot do. > > dmabuf can do that ... it is call uptr you can look at i915 for > instance. Still this does not answer my question above, why do > you need to be in VFIO to do any of the above thing ? Kernel has > tons of examples that does all of the above and are not in VFIO > (including usinng existing user pointer with device). > > Cheers, > Jérôme I took a look at i915_gem_execbuffer_ioctl(). It seems it "copy_from_user" the user memory to the kernel. That is not what we need. What we try to get is: the user application do something on its data, and push it away to the accelerator, and says: "I'm tied, it is your turn to do the job...". Then the accelerator has the memory, referring any portion of it with the same VAs of the application, even the VAs are stored inside the memory itself. And I don't understand why I should avoid to use VFIO? As Alex said, VFIO is the user driver framework. And I need exactly a user driver interface. Why should I invent another wheel? It has most of stuff I need: 1. Connecting multiple devices to the same application space 2. Pinning and DMA from the application space to the whole set of device 3. Managing hardware resource by device We just need the last step: make sure multiple applications and the kernel can share the same IOMMU. Then why shouldn't we use VFIO? And personally, I believe the maturity and correctness of a framework are driven by applications. Now the problem in accelerator world is that we don't have a direction. If we believe the requirement is right, the method itself is not a big problem in the end. We just need to let people have a unify platform to share their work together. Cheers -- -Kenneth(Hisilicon) ================================================================================ 本邮件及其附件含有华为公司的保密信息，仅限于发送给上面地址中列出的个人或群组。禁 止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、或散发）本邮件中 的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本邮件！ This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!