From mboxrd@z Thu Jan 1 00:00:00 1970 From: Oded Gabbay Subject: Re: [PATCH 1/3] amdkfd: Don't clear *kfd2kgd on kfd_module_init Date: Mon, 22 Dec 2014 12:22:18 +0200 Message-ID: <5497F0DA.6050805@amd.com> References: <1419108374-7020-1-git-send-email-oded.gabbay@amd.com> <1419108374-7020-2-git-send-email-oded.gabbay@amd.com> <5496AEAD.3090003@vodafone.de> <5496B04C.50502@amd.com> <5496BAE0.5090901@vodafone.de> <5496C5EA.7050200@amd.com> <5496CA0F.8000800@amd.com> <5496EDF1.7080106@vodafone.de> <5496EF34.70302@amd.com> <5496F0DD.40903@vodafone.de> <5497C98C.2080208@amd.com> <5497CB91.2080306@amd.com> <5497DD0E.7040400@vodafone.de> <5497E3D7.1060405@amd.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8"; Format="flowed" Content-Transfer-Encoding: base64 Return-path: Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1on0146.outbound.protection.outlook.com [157.56.110.146]) by gabe.freedesktop.org (Postfix) with ESMTP id ACD20897DC for ; Mon, 22 Dec 2014 02:22:41 -0800 (PST) In-Reply-To: <5497E3D7.1060405@amd.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: =?UTF-8?B?Q2hyaXN0aWFuIEvDtm5pZw==?= , Dave Airlie Cc: "Deucher, Alexander" , "Elifaz, Dana" , LKML , dri-devel List-Id: dri-devel@lists.freedesktop.org CgpPbiAxMi8yMi8yMDE0IDExOjI2IEFNLCBPZGVkIEdhYmJheSB3cm90ZToKPgo+Cj4gT24gMTIv MjIvMjAxNCAxMDo1NyBBTSwgQ2hyaXN0aWFuIEvDtm5pZyB3cm90ZToKPj4gQW0gMjIuMTIuMjAx NCB1bSAwODo0MyBzY2hyaWViIE9kZWQgR2FiYmF5Ogo+Pj4KPj4+Cj4+PiBPbiAxMi8yMi8yMDE0 IDA5OjQwIEFNLCBEYXZlIEFpcmxpZSB3cm90ZToKPj4+Pj4+Pj4+IFRoZXJlIHNob3VsZCBiZSwg YnV0IHdoZW4gdGhlIG1vZHVsZXMgYXJlIGNvbXBpbGVkIGluLCB0aGV5IGFyZSBsb2FkZWQKPj4+ Pj4+Pj4+IGJhc2VkIG9uCj4+Pj4+Pj4+PiBsaW5rIG9yZGVyIG9ubHksIGlmIHRoZXkgYXJlIGlu IHRoZSBzYW1lIGdyb3VwLCBhbmQgdGhlIGdyb3VwcyBhcmUKPj4+Pj4+Pj4+IGxvYWRlZCBieSBh Cj4+Pj4+Pj4+PiBwcmUtZGVmaW5lZCBvcmRlci4KPj4+Pj4+Pj4KPj4+Pj4+Pj4gSXMgdGhhdCBy ZWFsbHkgc3RpbGwgdXAgdG8gZGF0ZT8gSSd2ZSBzZWVuIGVmZm9ydCB0byBjaGFuZ2UgdGhhdAo+ Pj4+Pj4+PiBzb21ldGhpbmcgbGlrZQo+Pj4+Pj4+PiAxMCsgeWVhcnMgYWdvIHdoZW4gUnVzdHkg cmV3b3JrZWQgdGhlIG1vZHVsZSBzeXN0ZW0uIEFuZCBpdCBpcyBjb21taW5nCj4+Pj4+Pj4+IHVw IG9uIHRoZQo+Pj4+Pj4+PiBsaXN0cyBhZ2FpbiBmcm9tIHRpbWUgdG8gdGltZS4KPj4+Pj4+Pgo+ Pj4+Pj4+ICBGcm9tIHdoYXQgSSBjYW4gc2VlIGluIHRoZSBNYWtlZmlsZSBydWxlcywgY29kZSBh bmQgZ29vZ2xlLCB5ZXMsIHRoYXQncwo+Pj4+Pj4+IHN0aWxsCj4+Pj4+Pj4gdGhlIHNpdHVhdGlv bi4gSWYgc29tZW9uZSB3aWxsIHByb3ZlIG1lIHdyb25nIEkgd2lsbCBiZSBtb3JlIHRoYW4gaGFw cHkKPj4+Pj4+PiB0bwo+Pj4+Pj4+IGNvcnJlY3QgbXkgY29kZS4KPj4+Pj4+Pj4KPj4+Pj4+Pj4K Pj4+Pj4+Pj4+IEkgZG9uJ3Qgd2FudCB0byBtb3ZlIGlvbW11IGJlZm9yZSBncHUsIHNvIEkgZG9u J3QgaGF2ZSBhIHNvbHV0aW9uIGZvcgo+Pj4+Pj4+Pj4gdGhlCj4+Pj4+Pj4+PiBvcmRlciBiZXR3 ZWVuIGFtZGtmZCBhbmQgYW1kX2lvbW11X3YyLgo+Pj4+Pj4+Pgo+Pj4+Pj4+PiBXaHkgbm90PyBU aGF0J3Mgc3RpbGwgYmV0dGVyIHRoYW4gY3JlYXRpbmcgYSBrZXJuZWwgd29ya3F1ZXVlLAo+Pj4+ Pj4+PiBzY2hlZHVsaW5nIG9uZQo+Pj4+Pj4+PiB3b3JrIGl0ZW0gb24gaXQsIHJlc2NoZWR1bGlu ZyB0aGUgdGFzayB1bnRpbCBldmVyeXRoaW5nIGlzIGNvbXBsZXRlZCBhbmQKPj4+Pj4+Pj4geW91 IGNhbgo+Pj4+Pj4+PiBjb250aW51ZS4KPj4+Pj4+Pgo+Pj4+Pj4+IEJlY2F1c2UgSSBkb24ndCBr bm93IHRoZSBjb25zZXF1ZW5jZXMgb2YgbW92aW5nIGFuIGVudGlyZSBzdWJzeXN0ZW0gaW4KPj4+ Pj4+PiBmcm9udAo+Pj4+Pj4+IG9mIGFub3RoZXIgb25lLiBJbiBhZGRpdGlvbiwgZXZlbiBpZiBl dmVyeW9uZSBhZ3JlZXMsIEknbSBwcmV0dHkgc3VyZQo+Pj4+Pj4+IHRoYXQKPj4+Pj4+PiBMaW51 cyB3b24ndCBiZSBoYXBweSB0byBkbyB0aGF0IGluIC1yYyBzdGFnZXMuIFNvIG1heWJlIHRoaXMg aXMgc29tZXRoaW5nCj4+Pj4+Pj4gdG8KPj4+Pj4+PiBjb25zaWRlciBmb3IgMy4yMCBtZXJnZSB3 aW5kb3csIGJ1dCBJIHdvdWxkIHN0aWxsIGxpa2UgdG8gcHJvdmlkZSBhCj4+Pj4+Pj4gc29sdXRp b24KPj4+Pj4+PiBmb3IgMy4xOS4KPj4+Pj4+Cj4+Pj4+Pgo+Pj4+Pj4gWWVhaCwgdHJ1ZSBpbmRl ZWQuIEhvdyBhYm91dCBkZXBlbmRpbmcgb24gZXZlcnl0aGluZyBiZWluZyBjb21waWxlZCBhcwo+ Pj4+Pj4gbW9kdWxlCj4+Pj4+PiBmb3IgMy4xOSB0aGVuPyBTdGlsbCBiZXR0ZXIgdGhhbiBoYXZp bmcgc3VjaCBhIGhhY2sgaW4gdGhlIGRyaXZlciBmb3IgYXMgYQo+Pj4+Pj4gdGVtcG9yYXJ5IHdv cmthcm91bmQgZm9yIG9uZSByZWxlYXNlLgo+Pj4+Pj4KPj4+Pj4gSSB0aG91Z2h0IGFib3V0IGl0 LCBidXQgYmVjYXVzZSB0aGlzIHByb2JsZW0gd2FzIG9yaWdpbmFsbHkgcmVwb3J0ZWQgYnkgYQo+ Pj4+PiB1c2VyIHRoYXQgdG9sZCB1cyBoZSBjb3VsZG4ndCB1c2UgbW9kdWxlcyBiZWNhdXNlIG9m IGhpcyBzZXR1cCwgSSBkZWNpZGVkCj4+Pj4+IG5vdCB0by4KPj4+Pj4gSSBhc3N1bWUgdGhlcmUg YXJlIG90aGVyIHVzZXJzIG91dCB0aGVyZSB3aG8gbmVlZHMgdGhpcyBvcHRpb24gKGNvbXBpbGVk Cj4+Pj4+IGV2ZXJ5dGhpbmcgaW4gdGhlIGtlcm5lbCAtIGVtYmVkZGVkID8pLCBzbyBJIGRvbid0 IHdhbnQgdG8gbWFrZSB0aGVpciBsaWZlCj4+Pj4+IGhhcmRlci4KPj4+Pj4KPj4+Pj4gSW4gYWRk aXRpb24sIHNheWluZyBpdCBpcyBhIHdvcmthcm91bmQgZm9yIG9uZSByZWxlYXNlIGlzIHRydWUg aW4gY2FzZQo+Pj4+PiBtb3ZpbmcgaW9tbXUgc3Vic3lzdGVtIGluIGZyb250IG9mIGdwdSBzdWJz eXN0ZW0gaXMgYWNjZXB0YWJsZSBhbmQgZG9lc24ndAo+Pj4+PiBjYXVzZSBvdGhlciBwcm9ibGVt cywgdW5rbm93biBhdCB0aGlzIHBvaW50Lgo+Pj4+Pgo+Pj4+PiBCb3R0b20gbGluZSwgbXkgcGVy c29uYWwgcHJlZmVyZW5jZSBpcyB0byBoZWxwIHRoZSB1c2VycyBfbm93XyBhbmQgaWYgYQo+Pj4+ PiBiZXR0ZXIgZml4IGlzIGZvdW5kIGluIHRoZSBmdXR1cmUsIGNoYW5nZSB0aGUgY29kZSBhY2Nv cmRpbmdseS4KPj4+Pgo+Pj4+IE15IGd1ZXNzIGlzIG1vdmluZyB0aGUgaW9tbXUgc3Vic3lzdGVt IGluIGZyb250IG9mIHRoZSBHUFUgd291bGQgYmUgcmF0aW9uYWwuCj4+Pj4KPj4+PiBJdCBkb2Vz IHNlZW0gbGlrZSBpdCB3b3VsZCBnZW5lcmFsbHkgaGF2ZSBhIGRlcGVuZCBpbiB0aGF0IG9yZGVy Lgo+Pj4+Cj4+Pj4gRGF2ZS4KPj4+Pgo+Pj4gRGF2ZSwKPj4+IEkgYWdyZWUsIGJ1dCBkb24ndCB5 b3UgdGhpbmsgaXQgaXMgdG9vIHJpc2t5IGZvciAtcmMgc3RhZ2VzID8KPj4+IElmIG5vdCwgSSBj YW4gdHJ5IGl0IGFuZCBpZiBpdCB3b3JrcyBvbiBLViwgSSBjYW4gc3VibWl0IGEgcGF0Y2guCj4+ PiBCdXQgaWYgeW91IGRvIHRoaW5rIGl0IGlzIHJpc2t5LCB3aGF0IGRvIHlvdSByZWNvbW1lbmQg Zm9yIDMuMTkgPyBEbyB0aGUgZml4IEkKPj4+IHN1Z2dlc3RlZCBvciBkaXNhYmxlIGJ1aWxkLWlu IGNvbXBpbGF0aW9uIG9wdGlvbiA/Cj4+Cj4+IEkgd291bGQgc2F5IGNyZWF0ZSB0aGUgcGF0Y2gg b2YgY2hhbmdpbmcgdGhlIG9yZGVyIChzaG91bGQgYmUgdHJpdmlhbCksIGRlc2NyaWJlCj4+IGlu IGRldGFpbCBpbiB0aGUgY29tbWl0IG1lc3NhZ2Ugd2hhdCB0aGlzIGlzIHN1cHBvc2VkIHRvIGZp eCBhbmQgd2h5IHN1Y2ggYW4KPj4gc2V2ZXJlIGNoYW5nZSB3YXMgZG9uZSBpbiAtcmMxIGFuZCBz dWJtaXQgaXQgdXBzdHJlYW0uCj4+Cj4+IFdlIGNhbiBzdGlsbCByZXZlcnQgaXQgaW4gLXJjMiBp ZiBpdCBicmVha3MgYW55dGhpbmcuCj4+Cj4+IENocmlzdGlhbi4KPj4KPj4+Cj4+PiAgICAgT2Rl ZAo+Pgo+Cj4gT0ssIEknbGwgdHJ5IGl0IG9uIG15IG1hY2hpbmUgYW5kIGlmIGl0IHdvcmtzLCBJ IHdpbGwgc2VuZCB0aGUgcGF0Y2ggdG8gdGhlIGxpc3QuCj4KPiAgICAgIE9kZWQKPiBfX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwo+IGRyaS1kZXZlbCBtYWls aW5nIGxpc3QKPiBkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCj4gaHR0cDovL2xpc3Rz LmZyZWVkZXNrdG9wLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2RyaS1kZXZlbAoKU28gSSBjaGVja2Vk IGl0IGFuZCBhbGwgbXkgSFNBIHRlc3RzIGFyZSBwYXNzaW5nIG9uIEtWIG1hY2hpbmUuCkkgd2ls bCBzZW5kIHRoZSBwYXRjaGVzIHRvZGF5LiBQbGVhc2UgZGlzY2FyZCB0aGUgY3VycmVudCBwYXRj aC1zZXQuCgoJT2RlZApfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fXwpkcmktZGV2ZWwgbWFpbGluZyBsaXN0CmRyaS1kZXZlbEBsaXN0cy5mcmVlZGVza3RvcC5v cmcKaHR0cDovL2xpc3RzLmZyZWVkZXNrdG9wLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2RyaS1kZXZl bAo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754261AbaLVKWm (ORCPT ); Mon, 22 Dec 2014 05:22:42 -0500 Received: from mail-bl2on0115.outbound.protection.outlook.com ([65.55.169.115]:54816 "EHLO na01-bl2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1753977AbaLVKWl convert rfc822-to-8bit (ORCPT ); Mon, 22 Dec 2014 05:22:41 -0500 X-WSS-ID: 0NGZBHO-07-6MJ-02 X-M-MSG: Message-ID: <5497F0DA.6050805@amd.com> Date: Mon, 22 Dec 2014 12:22:18 +0200 From: Oded Gabbay Organization: AMD User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: =?UTF-8?B?Q2hyaXN0aWFuIEvDtm5pZw==?= , "Dave Airlie" CC: "Deucher, Alexander" , "Elifaz, Dana" , LKML , dri-devel Subject: Re: [PATCH 1/3] amdkfd: Don't clear *kfd2kgd on kfd_module_init References: <1419108374-7020-1-git-send-email-oded.gabbay@amd.com> <1419108374-7020-2-git-send-email-oded.gabbay@amd.com> <5496AEAD.3090003@vodafone.de> <5496B04C.50502@amd.com> <5496BAE0.5090901@vodafone.de> <5496C5EA.7050200@amd.com> <5496CA0F.8000800@amd.com> <5496EDF1.7080106@vodafone.de> <5496EF34.70302@amd.com> <5496F0DD.40903@vodafone.de> <5497C98C.2080208@amd.com> <5497CB91.2080306@amd.com> <5497DD0E.7040400@vodafone.de> <5497E3D7.1060405@amd.com> In-Reply-To: <5497E3D7.1060405@amd.com> Content-Type: text/plain; charset="utf-8"; format=flowed X-Originating-IP: [10.20.0.84] Content-Transfer-Encoding: 8BIT X-EOPAttributedMessage: 0 Authentication-Results: spf=none (sender IP is 165.204.84.221) smtp.mailfrom=Oded.Gabbay@amd.com; X-Forefront-Antispam-Report: CIP:165.204.84.221;CTRY:US;IPV:NLI;EFV:NLI;SFV:NSPM;SFS:(10019020)(6009001)(428002)(479174004)(199003)(24454002)(51704005)(377454003)(189002)(36756003)(97736003)(47776003)(20776003)(93886004)(21056001)(68736005)(65806001)(15975445007)(65956001)(107046002)(59896002)(4396001)(31966008)(23676002)(62966003)(1720100001)(77156002)(50466002)(64126003)(64706001)(46102003)(2950100001)(92566001)(77096005)(86362001)(87936001)(101416001)(106466001)(87266999)(54356999)(105586002)(76176999)(84676001)(83506001)(120916001)(33656002)(19580395003)(99396003)(19580405001)(80316001)(50986999);DIR:OUT;SFP:1102;SCL:1;SRVR:CO1PR02MB208;H:atltwp01.amd.com;FPR:;SPF:None;MLV:sfv;PTR:InfoDomainNonexistent;A:1;MX:1;LANG:en; X-Microsoft-Antispam: UriScan:; X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:;SRVR:CO1PR02MB208; X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004);SRVR:CO1PR02MB208; X-Forefront-PRVS: 0433DB2766 X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:;SRVR:CO1PR02MB208; X-OriginatorOrg: amd4.onmicrosoft.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Dec 2014 10:22:37.3341 (UTC) X-MS-Exchange-CrossTenant-Id: fde4dada-be84-483f-92cc-e026cbee8e96 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=fde4dada-be84-483f-92cc-e026cbee8e96;Ip=[165.204.84.221] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR02MB208 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 12/22/2014 11:26 AM, Oded Gabbay wrote: > > > On 12/22/2014 10:57 AM, Christian König wrote: >> Am 22.12.2014 um 08:43 schrieb Oded Gabbay: >>> >>> >>> On 12/22/2014 09:40 AM, Dave Airlie wrote: >>>>>>>>> There should be, but when the modules are compiled in, they are loaded >>>>>>>>> based on >>>>>>>>> link order only, if they are in the same group, and the groups are >>>>>>>>> loaded by a >>>>>>>>> pre-defined order. >>>>>>>> >>>>>>>> Is that really still up to date? I've seen effort to change that >>>>>>>> something like >>>>>>>> 10+ years ago when Rusty reworked the module system. And it is comming >>>>>>>> up on the >>>>>>>> lists again from time to time. >>>>>>> >>>>>>> From what I can see in the Makefile rules, code and google, yes, that's >>>>>>> still >>>>>>> the situation. If someone will prove me wrong I will be more than happy >>>>>>> to >>>>>>> correct my code. >>>>>>>> >>>>>>>> >>>>>>>>> I don't want to move iommu before gpu, so I don't have a solution for >>>>>>>>> the >>>>>>>>> order between amdkfd and amd_iommu_v2. >>>>>>>> >>>>>>>> Why not? That's still better than creating a kernel workqueue, >>>>>>>> scheduling one >>>>>>>> work item on it, rescheduling the task until everything is completed and >>>>>>>> you can >>>>>>>> continue. >>>>>>> >>>>>>> Because I don't know the consequences of moving an entire subsystem in >>>>>>> front >>>>>>> of another one. In addition, even if everyone agrees, I'm pretty sure >>>>>>> that >>>>>>> Linus won't be happy to do that in -rc stages. So maybe this is something >>>>>>> to >>>>>>> consider for 3.20 merge window, but I would still like to provide a >>>>>>> solution >>>>>>> for 3.19. >>>>>> >>>>>> >>>>>> Yeah, true indeed. How about depending on everything being compiled as >>>>>> module >>>>>> for 3.19 then? Still better than having such a hack in the driver for as a >>>>>> temporary workaround for one release. >>>>>> >>>>> I thought about it, but because this problem was originally reported by a >>>>> user that told us he couldn't use modules because of his setup, I decided >>>>> not to. >>>>> I assume there are other users out there who needs this option (compiled >>>>> everything in the kernel - embedded ?), so I don't want to make their life >>>>> harder. >>>>> >>>>> In addition, saying it is a workaround for one release is true in case >>>>> moving iommu subsystem in front of gpu subsystem is acceptable and doesn't >>>>> cause other problems, unknown at this point. >>>>> >>>>> Bottom line, my personal preference is to help the users _now_ and if a >>>>> better fix is found in the future, change the code accordingly. >>>> >>>> My guess is moving the iommu subsystem in front of the GPU would be rational. >>>> >>>> It does seem like it would generally have a depend in that order. >>>> >>>> Dave. >>>> >>> Dave, >>> I agree, but don't you think it is too risky for -rc stages ? >>> If not, I can try it and if it works on KV, I can submit a patch. >>> But if you do think it is risky, what do you recommend for 3.19 ? Do the fix I >>> suggested or disable build-in compilation option ? >> >> I would say create the patch of changing the order (should be trivial), describe >> in detail in the commit message what this is supposed to fix and why such an >> severe change was done in -rc1 and submit it upstream. >> >> We can still revert it in -rc2 if it breaks anything. >> >> Christian. >> >>> >>> Oded >> > > OK, I'll try it on my machine and if it works, I will send the patch to the list. > > Oded > _______________________________________________ > dri-devel mailing list > dri-devel@lists.freedesktop.org > http://lists.freedesktop.org/mailman/listinfo/dri-devel So I checked it and all my HSA tests are passing on KV machine. I will send the patches today. Please discard the current patch-set. Oded