From mboxrd@z Thu Jan 1 00:00:00 1970 From: Oded Gabbay Subject: Re: [PATCH v2 00/25] AMDKFD kernel driver Date: Thu, 24 Jul 2014 21:57:16 +0300 Message-ID: <53D1570C.5000704@amd.com> References: <53CD0961.4070505@amd.com> <53CD17FD.3000908@vodafone.de> <20140721152511.GW15237@phenom.ffwll.local> <20140721155851.GB4519@gmail.com> <20140721170546.GB15237@phenom.ffwll.local> <20140723135931.79541a86@jbarnes-desktop> <53D030C5.3030207@amd.com> <20140724154443.GA2951@gmail.com> <20140724184739.GA6177@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2lp0210.outbound.protection.outlook.com [207.46.163.210]) by gabe.freedesktop.org (Postfix) with ESMTP id 330126E739 for ; Thu, 24 Jul 2014 11:57:31 -0700 (PDT) In-Reply-To: <20140724184739.GA6177@gmail.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: Jerome Glisse , Alex Deucher Cc: "Lewycky, Andrew" , "linux-kernel@vger.kernel.org" , "dri-devel@lists.freedesktop.org" List-Id: dri-devel@lists.freedesktop.org T24gMjQvMDcvMTQgMjE6NDcsIEplcm9tZSBHbGlzc2Ugd3JvdGU6Cj4gT24gVGh1LCBKdWwgMjQs IDIwMTQgYXQgMDE6MzU6NTNQTSAtMDQwMCwgQWxleCBEZXVjaGVyIHdyb3RlOgo+PiBPbiBUaHUs IEp1bCAyNCwgMjAxNCBhdCAxMTo0NCBBTSwgSmVyb21lIEdsaXNzZSA8ai5nbGlzc2VAZ21haWwu Y29tPiB3cm90ZToKPj4+IE9uIFRodSwgSnVsIDI0LCAyMDE0IGF0IDAxOjAxOjQxQU0gKzAzMDAs IE9kZWQgR2FiYmF5IHdyb3RlOgo+Pj4+IE9uIDI0LzA3LzE0IDAwOjQ2LCBCcmlkZ21hbiwgSm9o biB3cm90ZToKPj4+Pj4KPj4+Pj4+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tIEZyb206IGRy aS1kZXZlbAo+Pj4+Pj4gW21haWx0bzpkcmktZGV2ZWwtYm91bmNlc0BsaXN0cy5mcmVlZGVza3Rv cC5vcmddIE9uIEJlaGFsZiBPZiBKZXNzZQo+Pj4+Pj4gQmFybmVzIFNlbnQ6IFdlZG5lc2RheSwg SnVseSAyMywgMjAxNCA1OjAwIFBNIFRvOgo+Pj4+Pj4gZHJpLWRldmVsQGxpc3RzLmZyZWVkZXNr dG9wLm9yZyBTdWJqZWN0OiBSZTogW1BBVENIIHYyIDAwLzI1XQo+Pj4+Pj4gQU1ES0ZEIGtlcm5l bCBkcml2ZXIKPj4+Pj4+Cj4+Pj4+PiBPbiBNb24sIDIxIEp1bCAyMDE0IDE5OjA1OjQ2ICswMjAw IGRhbmllbCBhdCBmZndsbC5jaCAoRGFuaWVsCj4+Pj4+PiBWZXR0ZXIpIHdyb3RlOgo+Pj4+Pj4K Pj4+Pj4+PiBPbiBNb24sIEp1bCAyMSwgMjAxNCBhdCAxMTo1ODo1MkFNIC0wNDAwLCBKZXJvbWUg R2xpc3NlIHdyb3RlOgo+Pj4+Pj4+PiBPbiBNb24sIEp1bCAyMSwgMjAxNCBhdCAwNToyNToxMVBN ICswMjAwLCBEYW5pZWwgVmV0dGVyIHdyb3RlOgo+Pj4+Pj4+Pj4gT24gTW9uLCBKdWwgMjEsIDIw MTQgYXQgMDM6Mzk6MDlQTSArMDIwMCwgQ2hyaXN0aWFuIEs/bmlnCj4+Pj4+Pj4+PiB3cm90ZToK Pj4+Pj4+Pj4+PiBBbSAyMS4wNy4yMDE0IDE0OjM2LCBzY2hyaWViIE9kZWQgR2FiYmF5Ogo+Pj4+ Pj4+Pj4+PiBPbiAyMC8wNy8xNCAyMDo0NiwgSmVyb21lIEdsaXNzZSB3cm90ZToKPj4+Pj4+Cj4+ Pj4+PiBbc25pcCEhXQo+Pj4+PiBNeSBCbGFja0JlcnJ5IHRodW1iIHRoYW5rcyB5b3UgOykKPj4+ Pj4+Cj4+Pj4+Pj4+Pj4KPj4+Pj4+Pj4+PiBUaGUgbWFpbiBxdWVzdGlvbnMgaGVyZSBhcmUgaWYg aXQncyBhdm9pZCBhYmxlIHRvIHBpbiBkb3duCj4+Pj4+Pj4+Pj4gdGhlIG1lbW9yeSBhbmQgaWYg dGhlIG1lbW9yeSBpcyBwaW5uZWQgZG93biBhdCBkcml2ZXIgbG9hZCwKPj4+Pj4+Pj4+PiBieSBy ZXF1ZXN0IGZyb20gdXNlcnNwYWNlIG9yIGJ5IGFueXRoaW5nIGVsc2UuCj4+Pj4+Pj4+Pj4KPj4+ Pj4+Pj4+PiBBcyBmYXIgYXMgSSBjYW4gc2VlIG9ubHkgdGhlICJtcWQgcGVyIHVzZXJzcGFjZSBx dWV1ZSIKPj4+Pj4+Pj4+PiBtaWdodCBiZSBhIGJpdCBxdWVzdGlvbmFibGUsIGV2ZXJ5dGhpbmcg ZWxzZSBzb3VuZHMKPj4+Pj4+Pj4+PiByZWFzb25hYmxlLgo+Pj4+Pj4+Pj4KPj4+Pj4+Pj4+IEFz aWRlLCBpOTE1IHBlcnNwZWN0aXZlIGFnYWluIChpLmUuIGhvdyB3ZSBzb2x2ZWQgdGhpcyk6Cj4+ Pj4+Pj4+PiBXaGVuIHNjaGVkdWxpbmcgYXdheSBmcm9tIGNvbnRleHRzIHdlIHVucGluIHRoZW0g YW5kIHB1dCB0aGVtCj4+Pj4+Pj4+PiBpbnRvIHRoZSBscnUuIEFuZCBpbiB0aGUgc2hyaW5rZXIg d2UgaGF2ZSBhIGxhc3QtZGl0Y2gKPj4+Pj4+Pj4+IGNhbGxiYWNrIHRvIHN3aXRjaCB0byBhIGRl ZmF1bHQgY29udGV4dCAoc2luY2UgeW91IGNhbid0IGV2ZXIKPj4+Pj4+Pj4+IGhhdmUgbm8gY29u dGV4dCBvbmNlIHlvdSd2ZSBzdGFydGVkKSB3aGljaCBtZWFucyB3ZSBjYW4gZXZpY3QKPj4+Pj4+ Pj4+IGFueSBjb250ZXh0IG9iamVjdCBpZiBpdCdzCj4+Pj4+PiBnZXR0aW5nIGluIHRoZSB3YXku Cj4+Pj4+Pj4+Cj4+Pj4+Pj4+IFNvIEludGVsIGhhcmR3YXJlIHJlcG9ydCB0aHJvdWdoIHNvbWUg aW50ZXJydXB0IG9yIHNvbWUgY2hhbm5lbAo+Pj4+Pj4+PiB3aGVuIGl0IGlzIG5vdCB1c2luZyBh IGNvbnRleHQgPyBpZSBrZXJuZWwgc2lkZSBnZXQKPj4+Pj4+Pj4gbm90aWZpY2F0aW9uIHdoZW4g c29tZSB1c2VyIGNvbnRleHQgaXMgZG9uZSBleGVjdXRpbmcgPwo+Pj4+Pj4+Cj4+Pj4+Pj4gWWVz LCBhcyBsb25nIGFzIHdlIGRvIHRoZSBzY2hlZHVsaW5nIHdpdGggdGhlIGNwdSB3ZSBnZXQKPj4+ Pj4+PiBpbnRlcnJ1cHRzIGZvciBjb250ZXh0IHN3aXRjaGVzLiBUaGUgbWVjaGFuaWMgaXMgYWxy ZWFkeQo+Pj4+Pj4+IHB1Ymxpc2hlZCBpbiB0aGUgZXhlY2xpc3QgcGF0Y2hlcyBjdXJyZW50bHkg ZmxvYXRpbmcgYXJvdW5kLiBXZQo+Pj4+Pj4+IGdldCBhIHNwZWNpYWwgY29udGV4dCBzd2l0Y2gg aW50ZXJydXB0Lgo+Pj4+Pj4+Cj4+Pj4+Pj4gQnV0IHdlIGhhdmUgdGhpcyB1bnBpbiBsb2dpYyBh bHJlYWR5IG9uIHRoZSBjdXJyZW50IGNvZGUgd2hlcmUKPj4+Pj4+PiB3ZSBzd2l0Y2ggY29udGV4 dHMgdGhyb3VnaCBpbi1saW5lIGNzIGNvbW1hbmRzIGZyb20gdGhlIGtlcm5lbC4KPj4+Pj4+PiBU aGVyZSB3ZSBvYnZpb3VzbHkgdXNlIHRoZSBub3JtYWwgYmF0Y2ggY29tcGxldGlvbiBldmVudHMu Cj4+Pj4+Pgo+Pj4+Pj4gWWVhaCBhbmQgd2UgY2FuIGNvbnRpbnVlIHRoYXQgZ29pbmcgZm9yd2Fy ZC4gIEFuZCBvZiBjb3Vyc2UgaWYgeW91cgo+Pj4+Pj4gaHcgY2FuIGRvIHBhZ2UgZmF1bHRpbmcs IHlvdSBkb24ndCBuZWVkIHRvIHBpbiB0aGUgbm9ybWFsIGRhdGEKPj4+Pj4+IGJ1ZmZlcnMuCj4+ Pj4+Pgo+Pj4+Pj4gVXN1YWxseSB0aGVyZSBhcmUgc29tZSBzcGVjaWFsIGJ1ZmZlcnMgdGhhdCBu ZWVkIHRvIGJlIHBpbm5lZCBmb3IKPj4+Pj4+IGxvbmdlciBwZXJpb2RzIHRob3VnaCwgYW55dGlt ZSB0aGUgY29udGV4dCBjb3VsZCBiZSBhY3RpdmUuICBTb3VuZHMKPj4+Pj4+IGxpa2UgaW4gdGhp cyBjYXNlIHRoZSB1c2VybGFuZCBxdWV1ZXMsIHdoaWNoIG1ha2VzIHNvbWUgc2Vuc2UuICBCdXQK Pj4+Pj4+IG1heWJlIGZvciBzbWFsbGVyIHN5c3RlbXMgdGhlIHNpemUgbGltaXQgY291bGQgYmUg Y2xhbXBlZCB0bwo+Pj4+Pj4gc29tZXRoaW5nIHNtYWxsZXIgdGhhbiAxMjhNLiAgT3IgdGllIGl0 IGludG8gdGhlIHJsaW1pdCBzb21laG93LAo+Pj4+Pj4ganVzdCBsaWtlIHdlIGRvIGZvciBtbG9j aygpIHN0dWZmLgo+Pj4+Pj4KPj4+Pj4gWWVhaCwgZXZlbiB0aGUgcXVldWVzIGFyZSBpbiBwYWdl YWJsZSBtZW1vcnksIGl0J3MganVzdCBhIH4yNTYgYnl0ZQo+Pj4+PiBzdHJ1Y3R1cmUgcGVyIHF1 ZXVlICh0aGUgTWVtb3J5IFF1ZXVlIERlc2NyaXB0b3IpIHRoYXQgZGVzY3JpYmVzIHRoZQo+Pj4+ PiBxdWV1ZSB0byBoYXJkd2FyZSwgcGx1cyBhIGNvdXBsZSBvZiBwYWdlcyBmb3IgZWFjaCBwcm9j ZXNzIHVzaW5nIEhTQQo+Pj4+PiB0byBob2xkIHRoaW5ncyBsaWtlIGRvb3JiZWxscy4gQ3VycmVu dCB0aGlua2luZyBpcyB0byBsaW1pdCAjCj4+Pj4+IHByb2Nlc3NlcyB1c2luZyBIU0EgdG8gfjI1 NiBhbmQgI3F1ZXVlcyBwZXIgcHJvY2VzcyB0byB+MTAyNCBieQo+Pj4+PiBkZWZhdWx0IGluIHRo ZSBpbml0aWFsIGNvZGUsIGFsdGhvdWdoIG15IGd1ZXNzIGlzIHRoYXQgd2UgY291bGQgdGFrZQo+ Pj4+PiB0aGUgI3F1ZXVlcyBwZXIgcHJvY2VzcyBkZWZhdWx0IGxpbWl0IGV2ZW4gbG93ZXIuCj4+ Pj4+Cj4+Pj4KPj4+PiBTbyBteSBtaXN0YWtlLiBzdHJ1Y3QgY2lrX21xZCBpcyBhY3R1YWxseSA2 MDQgYnl0ZXMsIGFuZCBpdCBpcyBhbGxvY2F0ZWQKPj4+PiBvbiAyNTYgYm91bmRhcnkuCj4+Pj4g SSBoYWQgaW4gbWluZCB0byByZXNlcnZlIDY0TUIgb2YgZ2FydCBieSBkZWZhdWx0LCB3aGljaCB0 cmFuc2xhdGVzIHRvCj4+Pj4gNTEyIHF1ZXVlcyBwZXIgcHJvY2Vzcywgd2l0aCAxMjggcHJvY2Vz c2VzLiBBZGQgMiBrZXJuZWwgbW9kdWxlCj4+Pj4gcGFyYW1ldGVycywgIyBvZiBtYXgtcXVldWVz LXBlci1wcm9jZXNzIGFuZCAjIG9mIG1heC1wcm9jZXNzZXMgKGRlZmF1bHQKPj4+PiBpcywgYXMg SSBzYWlkLCA1MTIgYW5kIDEyOCkgZm9yIGJldHRlciBjb250cm9sIG9mIHN5c3RlbSBhZG1pbi4K Pj4+Pgo+Pj4KPj4+IFNvIGFzIGkgc2FpZCBzb21ld2hlcmUgZWxzZSBpbiB0aGlzIHRocmVhZCwg dGhpcyBzaG91bGQgbm90IGJlIHJlc2VydmVkCj4+PiBidXQgdXNlIGEgc3BlY2lhbCBhbGxvY2F0 aW9uLiBBbnkgSFNBIEdQVSB1c2UgdmlydHVhbCBhZGRyZXNzIHNwYWNlIGZvcgo+Pj4gdXNlcnNw YWNlIHNvIG9ubHkgaXNzdWUgaXMgZm9yIGtlcm5lbCBzaWRlIEdUVC4KPj4+Cj4+PiBXaGF0IGkg d291bGQgbGlrZSBpcyBzZWVpbmcgcmFkZW9uIHBpbm5lZCBHVFQgYWxsb2NhdGlvbiBhdCBib3R0 b20gb2YKPj4+IEdUVCBzcGFjZSAoaWUgYWxsIHJpbmcgYnVmZmVyIGFuZCB0aGUgaWIgcG9vbCBi dWZmZXIpLiBUaGVuIGhhdmUgYW4KPj4+IGFsbG9jYXRvciB0aGF0IGFsbG9jYXRlIG5ldyBxdWV1 ZSBmcm9tIHRvcCBvZiBHVFQgYWRkcmVzcyBzcGFjZSBhbmQKPj4+IGdyb3cgdG8gdGhlIGJvdHRv bS4KPj4+Cj4+PiBJdCBzaG91bGQgbm90IHN0YXRpY2x5IHJlc2VydmVkIDY0TSBvciBhbnl0aGlu Zy4gV2hlbiBkb2luZyBhbGxvY2F0aW9uCj4+PiBpdCBzaG91bGQgbW92ZSBhbnkgdHRtIGJ1ZmZl ciB0aGF0IGFyZSBpbiB0aGUgcmVnaW9uIGl0IHdhbnQgdG8gYWxsb2NhdGUKPj4+IHRvIGEgZGlm ZmVyZW50IGxvY2F0aW9uLgo+Pj4KPj4+Cj4+PiBBcyB0aGlzIG5lZWRzIHNvbWUgd29yaywgaSBh bSBub3QgYWdhaW5zdCByZXNlcnZpbmcgc29tZSBzbWFsbCBhbW91bnQKPj4+IChjb3VwbGUgTUIp IGFzIGEgZmlyc3Qgc3RhZ2UgYnV0IGFueXRoaW5nIG1vcmUgd291bGQgbmVlZCBhIHByb3BlciBz b2x1dGlvbgo+Pj4gbGlrZSB0aGUgb25lIGkganVzdCBkZXNjcmliZWQuCj4+Cj4+IEl0J3Mgc3Rp bGwgYSB0cmFkZSBvZmYuICBFdmVuIGlmIHdlIHJlc2VydmUgYSBjb3VwbGUgb2YgbWVncyBpdCds bCBiZQo+PiB3YXN0ZWQgaWYgd2UgYXJlIG5vdCBydW5uaW5nIEhTQSBhcHBzLiBBbmQgZXZlbiB0 b2RheSBpZiB3ZSBydW4gYQo+PiBjb21wdXRlIGpvYiB1c2luZyB0aGUgY3VycmVudCBpbnRlcmZh Y2VzIHdlIGNvdWxkIGVuZCB1cCBpbiB0aGUgc2FtZQo+PiBjYXNlLiAgU28gd2hpbGUgSSB0aGlu ayBpdCdzIGRlZmluaXRlbHkgYSBnb29kIGdvYWwgdG8gY29tZSB1cCB3aXRoCj4+IHNvbWUgc29s dXRpb24gZm9yIGZyYWdtZW50YXRpb24sIEkgZG9uJ3QgdGhpbmsgaXQgc2hvdWxkIGJlIGEKPj4g c2hvdy1zdG9wcGVyIHJpZ2h0IG5vdy4KPj4KPiAKPiBTZWVtcyBpIGFtIGhhdmluZyBhIGhhcmQg dGltZSB0byBleHByZXNzIG15c2VsZi4gSSBhbSBub3Qgc2F5aW5nIGl0IGlzIGEKPiBzaG93c3Rv cHBlciBpIGFtIHNheWluZyB1bnRpbCBwcm9wZXIgc29sdXRpb24gaXMgaW1wbGVtZW50ZWQgS0ZE IHNob3VsZAo+IGxpbWl0IGl0cyBudW1iZXIgb2YgcXVldWUgdG8gY29uc3VtZSBhdCBtb3N0IGNv dXBsZSBNQiBpZSBub3QgNjRNQiBvciBtb3JlCj4gYnV0IDJNQiwgNE1CIHNvbWV0aGluZyBpbiB0 aGF0IHdhdGVyLgpTbyB3ZSB0aG91Z2h0IGludGVybmFsbHkgYWJvdXQgbGltaXRpbmcgb3Vyc2Vs dmVzIHRocm91Z2ggdHdvIGtlcm5lbAptb2R1bGUgcGFyYW1ldGVycywgIyBvZiBxdWV1ZXMgcGVy IHByb2Nlc3MgYW5kICMgb2YgcHJvY2Vzc2VzLiBEZWZhdWx0CnZhbHVlcyB3aWxsIGJlIDEyOCBx dWV1ZXMgcGVyIHByb2Nlc3MgYW5kIDMyIHByb2Nlc3Nlcy4gbXFkIHRha2VzIDc2OApieXRlcyBh dCBtb3N0LCBzbyB0aGF0IGdpdmVzIHVzIGEgbWF4aW11bSBvZiAzTUIuCgpGb3IgYWJzb2x1dGUg bWF4aW11bSwgSSB0aGluayB1c2luZyBIL1cgbGltaXRzIHdoaWNoIGFyZSAxMDI0IHF1ZXVlcyBw ZXIKcHJvY2VzcyBhbmQgNTEyIHByb2Nlc3Nlcy4gVGhhdCBnaXZlcyB1cyAzODRNQi4KCldvdWxk IHRoYXQgYmUgYWNjZXB0YWJsZSA/Cj4gCj4+IEEgYmV0dGVyIHNvbHV0aW9uIHRvIGRlYWwgd2l0 aCBmcmFnbWVudGF0aW9uIG9mIEdUVCBhbmQgcHJvdmlkZSBhCj4+IGJldHRlciB3YXkgdG8gYWxs b2NhdGUgbGFyZ2VyIGJ1ZmZlcnMgaW4gdnJhbSB3b3VsZCBiZSB0byBicmVhayB1cAo+PiB2cmFt IDwtPiBzeXN0ZW0gcG9vbCB0cmFuc2ZlcnMgaW50byBtdWx0aXBsZSB0cmFuc2ZlcnMgZGVwZW5k aW5nIG9uCj4+IHRoZSBhdmFpbGFibGUgR1RUIHNpemUuICBPciB1c2UgR1BVVk0gZHluYW1pY2Fs bHkgZm9yICB2cmFtIDwtPiBzeXN0ZW0KPj4gdHJhbnNmZXJzLgo+IAo+IElzbid0IHRoZSBVVkQg ZW5naW5lIHN0aWxsIHVzaW5nIHRoZSBtYWluIEdUVCA/IEkgaGF2ZSBub3QgbG9vayBtdWNoIGF0 Cj4gVVZEIGluIGEgd2hpbGUuCj4gCj4gWWVzIHRoZXJlIGlzIHdheSB0byBmaXggYnVmZmVyIG1p Z3JhdGlvbiBidXQgaSB3b3VsZCBhbHNvIGxpa2UgdG8gc2VlCj4gYWRkcmVzcyBzcGFjZSBmcmFn bWVudGF0aW9uIHRvIGEgbWluaW11bSB3aGljaCBpcyB0aGUgbWFpbiByZWFzb24gaQo+IHV0ZXJs eSBoYXRlIGFueSBkZXNpZ24gdGhhdCBmb3JiaWQga2VybmVsIHRvIHRha2Ugb3ZlciBhbmQgZG8g aXRzIHRoaW5nLgo+IAo+IEJ1ZmZlciBwaW5pbmcgc2hvdWxkIHJlYWxseSBiZSBvbmx5IGZvciBm cm9udCBidWZmZXIgYW5kIHRoaW5nIGxpa2UgcmluZwo+IGllIGJ1ZmZlciB0aGF0IGhhdmUgYSBs aWZldGltZSBib3VuZCB0byB0aGUgZHJpdmVyIGxpZmV0aW1lLgo+IAo+IENoZWVycywKPiBKw6ly w7RtZQo+IAo+Pgo+PiBBbGV4CgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fXwpkcmktZGV2ZWwgbWFpbGluZyBsaXN0CmRyaS1kZXZlbEBsaXN0cy5mcmVlZGVz a3RvcC5vcmcKaHR0cDovL2xpc3RzLmZyZWVkZXNrdG9wLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2Ry aS1kZXZlbAo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1759720AbaGXS5c (ORCPT ); Thu, 24 Jul 2014 14:57:32 -0400 Received: from mail-bl2lp0209.outbound.protection.outlook.com ([207.46.163.209]:4586 "EHLO na01-bl2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1758964AbaGXS5b convert rfc822-to-8bit (ORCPT ); Thu, 24 Jul 2014 14:57:31 -0400 X-WSS-ID: 0N98CNM-07-SNQ-02 X-M-MSG: Message-ID: <53D1570C.5000704@amd.com> Date: Thu, 24 Jul 2014 21:57:16 +0300 From: Oded Gabbay Organization: AMD User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.0 MIME-Version: 1.0 To: Jerome Glisse , Alex Deucher CC: "Bridgman, John" , Jesse Barnes , "dri-devel@lists.freedesktop.org" , =?UTF-8?B?Q2hyaXN0aWFuIEvDtm5pZw==?= , "Lewycky, Andrew" , "David Airlie" , "linux-kernel@vger.kernel.org" Subject: Re: [PATCH v2 00/25] AMDKFD kernel driver References: <53CD0961.4070505@amd.com> <53CD17FD.3000908@vodafone.de> <20140721152511.GW15237@phenom.ffwll.local> <20140721155851.GB4519@gmail.com> <20140721170546.GB15237@phenom.ffwll.local> <20140723135931.79541a86@jbarnes-desktop> <53D030C5.3030207@amd.com> <20140724154443.GA2951@gmail.com> <20140724184739.GA6177@gmail.com> In-Reply-To: <20140724184739.GA6177@gmail.com> Content-Type: text/plain; charset="utf-8" X-Originating-IP: [10.224.155.208] Content-Transfer-Encoding: 8BIT X-EOPAttributedMessage: 0 X-Forefront-Antispam-Report: CIP:165.204.84.221;CTRY:US;IPV:NLI;IPV:NLI;EFV:NLI;SFV:NSPM;SFS:(6009001)(428002)(377454003)(189002)(199002)(479174003)(51704005)(24454002)(13464003)(19580395003)(23676002)(74662001)(92566001)(83322001)(79102001)(65816999)(36756003)(76176999)(85852003)(80316001)(87266999)(77982001)(50986999)(74502001)(68736004)(93886003)(19580405001)(50466002)(21056001)(83072002)(92726001)(65956001)(33656002)(46102001)(83506001)(65806001)(80022001)(54356999)(20776003)(85306003)(47776003)(64706001)(81342001)(106466001)(86362001)(102836001)(81542001)(105586002)(107046002)(101416001)(87936001)(31966008)(97736001)(99396002)(76482001)(4396001)(84676001)(64126003)(95666004)(44976005);DIR:OUT;SFP:;SCL:1;SRVR:CO1PR02MB046;H:atltwp01.amd.com;FPR:;MLV:sfv;PTR:InfoDomainNonexistent;MX:1;LANG:en; X-Microsoft-Antispam: BCL:0;PCL:0;RULEID: X-Forefront-PRVS: 028256169F Authentication-Results: spf=none (sender IP is 165.204.84.221) smtp.mailfrom=Oded.Gabbay@amd.com; X-OriginatorOrg: amd4.onmicrosoft.com Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 24/07/14 21:47, Jerome Glisse wrote: > On Thu, Jul 24, 2014 at 01:35:53PM -0400, Alex Deucher wrote: >> On Thu, Jul 24, 2014 at 11:44 AM, Jerome Glisse wrote: >>> On Thu, Jul 24, 2014 at 01:01:41AM +0300, Oded Gabbay wrote: >>>> On 24/07/14 00:46, Bridgman, John wrote: >>>>> >>>>>> -----Original Message----- From: dri-devel >>>>>> [mailto:dri-devel-bounces@lists.freedesktop.org] On Behalf Of Jesse >>>>>> Barnes Sent: Wednesday, July 23, 2014 5:00 PM To: >>>>>> dri-devel@lists.freedesktop.org Subject: Re: [PATCH v2 00/25] >>>>>> AMDKFD kernel driver >>>>>> >>>>>> On Mon, 21 Jul 2014 19:05:46 +0200 daniel at ffwll.ch (Daniel >>>>>> Vetter) wrote: >>>>>> >>>>>>> On Mon, Jul 21, 2014 at 11:58:52AM -0400, Jerome Glisse wrote: >>>>>>>> On Mon, Jul 21, 2014 at 05:25:11PM +0200, Daniel Vetter wrote: >>>>>>>>> On Mon, Jul 21, 2014 at 03:39:09PM +0200, Christian K?nig >>>>>>>>> wrote: >>>>>>>>>> Am 21.07.2014 14:36, schrieb Oded Gabbay: >>>>>>>>>>> On 20/07/14 20:46, Jerome Glisse wrote: >>>>>> >>>>>> [snip!!] >>>>> My BlackBerry thumb thanks you ;) >>>>>> >>>>>>>>>> >>>>>>>>>> The main questions here are if it's avoid able to pin down >>>>>>>>>> the memory and if the memory is pinned down at driver load, >>>>>>>>>> by request from userspace or by anything else. >>>>>>>>>> >>>>>>>>>> As far as I can see only the "mqd per userspace queue" >>>>>>>>>> might be a bit questionable, everything else sounds >>>>>>>>>> reasonable. >>>>>>>>> >>>>>>>>> Aside, i915 perspective again (i.e. how we solved this): >>>>>>>>> When scheduling away from contexts we unpin them and put them >>>>>>>>> into the lru. And in the shrinker we have a last-ditch >>>>>>>>> callback to switch to a default context (since you can't ever >>>>>>>>> have no context once you've started) which means we can evict >>>>>>>>> any context object if it's >>>>>> getting in the way. >>>>>>>> >>>>>>>> So Intel hardware report through some interrupt or some channel >>>>>>>> when it is not using a context ? ie kernel side get >>>>>>>> notification when some user context is done executing ? >>>>>>> >>>>>>> Yes, as long as we do the scheduling with the cpu we get >>>>>>> interrupts for context switches. The mechanic is already >>>>>>> published in the execlist patches currently floating around. We >>>>>>> get a special context switch interrupt. >>>>>>> >>>>>>> But we have this unpin logic already on the current code where >>>>>>> we switch contexts through in-line cs commands from the kernel. >>>>>>> There we obviously use the normal batch completion events. >>>>>> >>>>>> Yeah and we can continue that going forward. And of course if your >>>>>> hw can do page faulting, you don't need to pin the normal data >>>>>> buffers. >>>>>> >>>>>> Usually there are some special buffers that need to be pinned for >>>>>> longer periods though, anytime the context could be active. Sounds >>>>>> like in this case the userland queues, which makes some sense. But >>>>>> maybe for smaller systems the size limit could be clamped to >>>>>> something smaller than 128M. Or tie it into the rlimit somehow, >>>>>> just like we do for mlock() stuff. >>>>>> >>>>> Yeah, even the queues are in pageable memory, it's just a ~256 byte >>>>> structure per queue (the Memory Queue Descriptor) that describes the >>>>> queue to hardware, plus a couple of pages for each process using HSA >>>>> to hold things like doorbells. Current thinking is to limit # >>>>> processes using HSA to ~256 and #queues per process to ~1024 by >>>>> default in the initial code, although my guess is that we could take >>>>> the #queues per process default limit even lower. >>>>> >>>> >>>> So my mistake. struct cik_mqd is actually 604 bytes, and it is allocated >>>> on 256 boundary. >>>> I had in mind to reserve 64MB of gart by default, which translates to >>>> 512 queues per process, with 128 processes. Add 2 kernel module >>>> parameters, # of max-queues-per-process and # of max-processes (default >>>> is, as I said, 512 and 128) for better control of system admin. >>>> >>> >>> So as i said somewhere else in this thread, this should not be reserved >>> but use a special allocation. Any HSA GPU use virtual address space for >>> userspace so only issue is for kernel side GTT. >>> >>> What i would like is seeing radeon pinned GTT allocation at bottom of >>> GTT space (ie all ring buffer and the ib pool buffer). Then have an >>> allocator that allocate new queue from top of GTT address space and >>> grow to the bottom. >>> >>> It should not staticly reserved 64M or anything. When doing allocation >>> it should move any ttm buffer that are in the region it want to allocate >>> to a different location. >>> >>> >>> As this needs some work, i am not against reserving some small amount >>> (couple MB) as a first stage but anything more would need a proper solution >>> like the one i just described. >> >> It's still a trade off. Even if we reserve a couple of megs it'll be >> wasted if we are not running HSA apps. And even today if we run a >> compute job using the current interfaces we could end up in the same >> case. So while I think it's definitely a good goal to come up with >> some solution for fragmentation, I don't think it should be a >> show-stopper right now. >> > > Seems i am having a hard time to express myself. I am not saying it is a > showstopper i am saying until proper solution is implemented KFD should > limit its number of queue to consume at most couple MB ie not 64MB or more > but 2MB, 4MB something in that water. So we thought internally about limiting ourselves through two kernel module parameters, # of queues per process and # of processes. Default values will be 128 queues per process and 32 processes. mqd takes 768 bytes at most, so that gives us a maximum of 3MB. For absolute maximum, I think using H/W limits which are 1024 queues per process and 512 processes. That gives us 384MB. Would that be acceptable ? > >> A better solution to deal with fragmentation of GTT and provide a >> better way to allocate larger buffers in vram would be to break up >> vram <-> system pool transfers into multiple transfers depending on >> the available GTT size. Or use GPUVM dynamically for vram <-> system >> transfers. > > Isn't the UVD engine still using the main GTT ? I have not look much at > UVD in a while. > > Yes there is way to fix buffer migration but i would also like to see > address space fragmentation to a minimum which is the main reason i > uterly hate any design that forbid kernel to take over and do its thing. > > Buffer pining should really be only for front buffer and thing like ring > ie buffer that have a lifetime bound to the driver lifetime. > > Cheers, > Jérôme > >> >> Alex