From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A20E5CDB474 for ; Fri, 20 Oct 2023 08:47:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:References :In-Reply-To:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=T6+HID7ptz6Lb1nJJaVMdlTVjNtdvZrvV+d1nAlbJoM=; b=oqUhy3NCNoyaBQ oPi4EcBQaHE7a87gLED4hoyOur4T0LukPATsCVMM27hF5YtkzzKMlWLfms0wNIn0MvxeQPIE4a/GW IHU9+KdS0vIibF9AziTu78ZSXfOPA81hKsjn/erht/XJZJUhLwN3dolr1R2ySI6p8S7w3cVuNUN1s SWnrM2IVMyQkjZCNGmeO+/tnvLzf3ZFiHZ7cRHDVir3QCejL06XnKX50mQm/PB65JJ0Fb1Sl6lX4y f5PM2QMCuWjnGYIgBXd2FwlLoUmnvbGiv8SWMl5oZ3+bOB9s3UvnIs7Z0Wx6oUejt1B4GoIgK26Km X6EDwiBhieU9VJBnsFsQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qtlAX-001amT-39; Fri, 20 Oct 2023 08:47:45 +0000 Received: from sin.source.kernel.org ([145.40.73.55]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qtlAT-001alV-36 for linux-riscv@lists.infradead.org; Fri, 20 Oct 2023 08:47:43 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 3E38ECE35C5; Fri, 20 Oct 2023 08:47:37 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id F023CC433C8; Fri, 20 Oct 2023 08:47:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1697791656; bh=yvS57ci7fpW8c627QySzOjgk0/AmuicEXD3GQA5Hz+U=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=u5USE2+U4vh8XD5mIYisLUnT1GNJZJp49PPKHE18d3VDUEQ0SkrSGdbA8H7UVvmrn yrHRSOqdWYUb2fGuE0GjmXM9sUs1GVwHfFwrjZqv53cwcQ0fNqiRATQ9eX8i0ItkCM km8CVxnx/ItN4OqVASHYsu5bp8UadjTNEpNPznuAE7zH0xpLoiXK7EEtmEWwocdscQ Okmk9ofjp4lGSUG3KolizoY/WI9fPQ7BWcZl+CawkxAmnHkrh89gu7iNlQ8nva136U 7i1vATxQ6ttSgT6DKm1m1kVpTkWblbWz8YfqwQE2lUWOimOBz7cwtWLDYa1BUHLZ3h 7Py33kx/e/dMw== From: =?utf-8?B?QmrDtnJuIFTDtnBlbA==?= To: Anup Patel Subject: Re: [PATCH v10 00/15] Linux RISC-V AIA Support In-Reply-To: References: <20231003044403.1974628-1-apatel@ventanamicro.com> <87o7gu7mo9.fsf@all.your.base.are.belong.to.us> Date: Fri, 20 Oct 2023 10:47:33 +0200 Message-ID: <87h6mlbryy.fsf@all.your.base.are.belong.to.us> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231020_014742_340970_5863C4E2 X-CRM114-Status: GOOD ( 31.81 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: devicetree@vger.kernel.org, Conor Dooley , Saravana Kannan , Marc Zyngier , Anup Patel , Atish Patra , linux-kernel@vger.kernel.org, Rob Herring , Palmer Dabbelt , Krzysztof Kozlowski , Paul Walmsley , Thomas Gleixner , Frank Rowand , linux-riscv@lists.infradead.org, Andrew Jones Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org VGhhbmtzIGZvciB0aGUgcXVpY2sgcmVwbHkhCgpBbnVwIFBhdGVsIDxhcGF0ZWxAdmVudGFuYW1p Y3JvLmNvbT4gd3JpdGVzOgoKPiBPbiBUaHUsIE9jdCAxOSwgMjAyMyBhdCA3OjEz4oCvUE0gQmrD tnJuIFTDtnBlbCA8Ympvcm5Aa2VybmVsLm9yZz4gd3JvdGU6Cj4+Cj4+IEhpIEFudXAsCj4+Cj4+ IEFudXAgUGF0ZWwgPGFwYXRlbEB2ZW50YW5hbWljcm8uY29tPiB3cml0ZXM6Cj4+Cj4+ID4gVGhl IFJJU0MtViBBSUEgc3BlY2lmaWNhdGlvbiBpcyByYXRpZmllZCBhcy1wZXIgdGhlIFJJU0MtViBp bnRlcm5hdGlvbmFsCj4+ID4gcHJvY2Vzcy4gVGhlIGxhdGVzdCByYXRpZmllZCBBSUEgc3BlY2lm Y2F0aW9uIGNhbiBiZSBmb3VuZCBhdDoKPj4gPiBodHRwczovL2dpdGh1Yi5jb20vcmlzY3Yvcmlz Y3YtYWlhL3JlbGVhc2VzL2Rvd25sb2FkLzEuMC9yaXNjdi1pbnRlcnJ1cHRzLTEuMC5wZGYKPj4g Pgo+PiA+IEF0IGEgaGlnaC1sZXZlbCwgdGhlIEFJQSBzcGVjaWZpY2F0aW9uIGFkZHMgdGhyZWUg dGhpbmdzOgo+PiA+IDEpIEFJQSBDU1JzCj4+ID4gICAgLSBJbXByb3ZlZCBsb2NhbCBpbnRlcnJ1 cHQgc3VwcG9ydAo+PiA+IDIpIEluY29taW5nIE1lc3NhZ2UgU2lnbmFsZWQgSW50ZXJydXB0IENv bnRyb2xsZXIgKElNU0lDKQo+PiA+ICAgIC0gUGVyLUhBUlQgTVNJIGNvbnRyb2xsZXIKPj4gPiAg ICAtIFN1cHBvcnQgTVNJIHZpcnR1YWxpemF0aW9uCj4+ID4gICAgLSBTdXBwb3J0IElQSSBhbG9u ZyB3aXRoIHZpcnR1YWxpemF0aW9uCj4+ID4gMykgQWR2YW5jZWQgUGxhdGZvcm0tTGV2ZWwgSW50 ZXJydXB0IENvbnRyb2xsZXIgKEFQTElDKQo+PiA+ICAgIC0gV2lyZWQgaW50ZXJydXB0IGNvbnRy b2xsZXIKPj4gPiAgICAtIEluIE1TSS1tb2RlLCBjb252ZXJ0cyB3aXJlZCBpbnRlcnJ1cHQgaW50 byBNU0lzIChpLmUuIE1TSSBnZW5lcmF0b3IpCj4+ID4gICAgLSBJbiBEaXJlY3QtbW9kZSwgaW5q ZWN0cyBleHRlcm5hbCBpbnRlcnJ1cHRzIGRpcmVjdGx5IGludG8gSEFSVHMKPj4KPj4gVGhhbmtz IGZvciB3b3JraW5nIG9uIHRoZSBBSUEgc3VwcG9ydCEgSSBoYWQgYSBsb29rIGF0IHRoZSBzZXJp ZXMsIGFuZAo+PiBoYXZlIHNvbWUgY29uY2VybnMgYWJvdXQgaW50ZXJydXB0IElEIGFic3RyYWN0 aW9uLgo+Pgo+PiBBIGJpdCBvZiBiYWNrZ3JvdW5kLCBmb3IgcmVhZGVycyBub3QgZmFtaWxpYXIg d2l0aCB0aGUgQUlBIGRldGFpbHMuCj4+Cj4+IElNU0lDIGFsbG93cyBmb3IgMjA0NyB1bmlxdWUg TVNJICgibXNpLWlycSIpIHNvdXJjZXMgcGVyIGhhcnQsIGFuZAo+PiBlYWNoIE1TSSBpcyBkZWRp Y2F0ZWQgdG8gYSBjZXJ0YWluIGhhcnQuIFRoZSBzZXJpZXMgdGFrZXMgdGhlIGFwcHJvYWNoCj4+ IHRvIHNheSB0aGF0IHRoZXJlIGFyZSwgZS5nLiwgMjA0NyBpbnRlcnJ1cHRzICgibG54LWlycSIp IGdsb2JhbGx5Lgo+PiBFYWNoIGxueC1pcnEgY29uc2lzdHMgb2YgI2hhcnRzICogbXNpLWlycSAt LSBhIHNsaWNlIC0tIGFuZCBpbiB0aGUKPj4gc2xpY2Ugb25seSAqb25lKiBtc2ktaXJxIGlzIGFj dXRhbGx5IHVzZWQuCj4+Cj4+IFRoaXMgc2NoZW1lIG1ha2VzIGFmZmluaXR5IGNoYW5nZXMgbW9y ZSByb2J1c3QsIGJlY2F1c2UgdGhlIGludGVycnVwdAo+PiBzb3VyY2VzIG9uICJvdGhlciIgaGFy dHMgYXJlIHByZS1hbGxvY2F0ZWQuIE9uIHRoZSBvdGhlciBoYW5kIGl0Cj4+IHJlcXVpcmVzIHRv IHByb3BhZ2F0ZSBpcnEgbWFza2luZyB0byBvdGhlciBoYXJ0cyB2aWEgSVBJcyAodGhpcyBpcwo+ PiBtb3N0bHkgZG9uZSB1cCBzZXR1cC90ZWFyIGRvd24pLiBJdCdzIGFsc28gd2FzdGVmdWwsIGJl Y2F1c2UgbXNpLWlycXMKPj4gYXJlIGhvZ2dlZCwgYW5kIGNhbm5vdCBiZSB1c2VkLgo+Pgo+PiBD b250ZW1wb3Jhcnkgc3RvcmFnZS9uZXR3b3JraW5nIGRyaXZlcnMgdXN1YWxseSB1c2VzIHF1ZXVl cyBwZXIgY29yZQo+PiAob3IgYSBzdWItc2V0IG9mIGNvcmVzKS4gVGhlIGN1cnJlbnQgc2NoZW1l IHdhc3RlcyBhIGxvdCBvZiBtc2ktaXJxcy4KPj4gSWYgd2UgaW5zdGVhZCB1c2VkIGEgc2NoZW1l IHdoZXJlICJtc2ktaXJxID09IGxueC1pcnEiLCBpbnN0ZWFkIG9mCj4+ICJsbnEtaXJxID0ge2hh cnQgMDttc2ktaXJxIHggLCAuLi4gaGFydCBOO21zaS1pcnEgeH0iLCB0aGVyZSB3b3VsZCBiZQo+ PiBhIGxvdCBNU0lzIGZvciBvdGhlciB1c2Vycy4gMS0xIHZzIDEtTi4gRS5nLiwgaWYgYSBzdG9y YWdlIGRldmljZQo+PiB3b3VsZCBsaWtlIHRvIHVzZSA1IHF1ZXVlcyAoNSBjb3Jlcykgb24gYSAx MjggY29yZSBzeXN0ZW0sIHRoZSBjdXJyZW50Cj4+IHNjaGVtZSB3b3VsZCBjb25zdW1lIDUgKiAx MjggTVNJcywgaW5zdGVhZCBvZiBqdXN0IDUuCj4+Cj4+IE9uIHRoZSBwbHVzIHNpZGU6Cj4+ICog Q2hhbmdpbmcgaW50ZXJydXB0cyBhZmZpbml0eSB3aWxsIG5ldmVyIGZhaWwsIGJlY2F1c2UgdGhl IGludGVycnVwdHMKPj4gICBvbiBlYWNoIGhhcnQgaXMgcHJlLWFsbG9jYXRlZC4KPj4KPj4gT24g dGhlIG5lZ2F0aXZlIHNpZGU6Cj4+ICogV2FzdGVmdWwgaW50ZXJydXB0IHVzYWdlLCBhbmQgYSBz eXN0ZW0gY2FuIHBvdGllbnRpYWxseSAicnVuIG91dCIgb2YKPj4gICBpbnRlcnJ1cHRzLiBFc3Bl Y2lhbGx5IGZvciBtYW55IGNvcmUgc3lzdGVtcy4KPj4gKiBJbnRlcnJ1cHQgbWFza2luZyBuZWVk IHRvIHByb2FnYXRlIHRvIGhhcnRzIHZpYSBJUElzICh0aGVyZSdzIG5vCj4+ICAgYnJvYWRjYXN0 IGNzciBpbiBJTVNJQyksIGFuZCBhIG1vcmUgY29tcGxleCBsb2NraW5nIHNjaGVtZSBJTVNJQwo+ Pgo+PiBTdW1tYXJ5Ogo+PiBUaGUgY3VycmVudCBzZXJpZXMgY2FwcyB0aGUgbnVtYmVyIG9mIGds b2JhbCBpbnRlcnJ1cHRzIHRvIG1heGltdW0KPj4gMjA0NyBNU0lzIGZvciBhbGwgY29yZXMgKHdo b2xlIHN5c3RlbSkuIEEgYmV0dGVyIHNjaGVtZSwgSU1PLCB3b3VsZCBiZQo+PiB0byBleHBvc2Ug MjA0NyAqICNoYXJ0cyB1bmlxdWUgTVNJcy4KPj4KPj4gSSB0aGluayB0aGlzIGNvdWxkIHNpbXBs aWZ5L3JlbW92ZSg/KSB0aGUgbG9ja2luZyBhcyB3ZWxsLgo+Cj4gRXhwb3NpbmcgMjA0NyAqICNo YXJ0cyB1bmlxdWUgTVNJcyBoYXMgbXVsdGlwbGUgaXNzdWVzOgo+IDEpIFRoZSBpcnFfc2V0X2Fm ZmluaXR5KCkgZG9lcyBub3Qgd29yayBmb3IgTVNJcyBiZWNhdXNlIGVhY2gKPiAgICAgIElSUSBp cyBub3QgdGllZCB0byBhIHBhcnRpY3VsYXIgSEFSVC4gVGhpcyBtZWFucyB3ZSBjYW4ndAo+ICAg ICAgYmFsYW5jZSB0aGUgSVJRIHByb2Nlc3NpbmcgbG9hZCBhbW9uZyBIQVJUcy4KClllcywgeW91 IGNhbiBiYWxhbmNlLiBJbiB5b3VyIGNvZGUsIGVhY2ggKmFjdGl2ZSogTVNJIGlzIHN0aWxsCmJv dW5kL2FjdGl2ZSB0byBhIHNwZWNpZmljIGhhcmQgdG9nZXRoZXIgd2l0aCB0aGUgYWZmaW5pdHkg bWFzay4gSW4gYW4KMS0xIG1vZGVsIHlvdSB3b3VsZCBzdGlsbCBuZWVkIHRvIHRyYWNrIHRoZSBh ZmZpbml0eSBtYXNrLCBidXQgdGhlCmlycV9zZXRfYWZmaW5pdHkoKSB3b3VsZCBiZSBkaWZmZXJl bnQuIEl0IHdvdWxkIHRyeSB0byBhbGxvY2F0ZSBhIG5ldwpNU0kgZnJvbSB0aGUgdGFyZ2V0IENQ VSwgYW5kIHRoZW4gc3dpdGNoIHRvIGhhdmluZyB0aGF0IE1TSSBhY3RpdmUuCgpUaGF0J3Mgd2hh dCB4ODYgZG9lcyBBRkFJVSwgd2hpY2ggaXMgYWxzbyBjb25zdHJhaW5lZCBieSB0aGUgIyBvZgph dmFpbGFibGUgTVNJcy4KClRoZSBkb3duc2lkZSwgYXMgSSBwb2ludGVkIG91dCwgaXMgdGhhdCB0 aGUgc2V0IGFmZmluaXR5IGFjdGlvbiBjYW4KZmFpbCBmb3IgYSBjZXJ0YWluIHRhcmdldCBDUFUu Cgo+IDIpIEFsbCB3aXJlZCBJUlFzIGZvciBBUExJQyBNU0ktbW9kZSB3aWxsIGFsc28gdGFyZ2V0 IGEKPiAgICAgZml4ZWQgSEFSVCBoZW5jZSBpcnFfc2V0X2FmZmluaXR5KCkgd29uJ3Qgd29yayBm b3Igd2lyZWQKPiAgICAgSVJRcyBhcyB3ZWxsLgoKSSdtIG5vdCBmb2xsb3dpbmcgaGVyZS4gV2h5 IHdvdWxkIEFQTElDIHB1dCBhIGNvbnN0cmFpbnQgaGVyZT8gSSBoYWQgYQpsb29rIGF0IHRoZSBz cGVjcywgYW5kIEkgZGlkbid0IHNlZSBhbnl0aGluZyBzdXBwb3J0aW5nIHRoZSBjdXJyZW50CnNj aGVtZSBleHBsaWNpdGx5LgoKPiAzKSBDb250ZW1wb3Jhcnkgc3RvcmFnZS9uZXR3b3JraW5nIGRy aXZlcnMgd2hpY2ggdXNlIHBlci1jb3JlCj4gICAgICBxdWV1ZXMgdXNlIGlycV9zZXRfYWZmaW5p dHkoKSBvbiBxdWV1ZSBJUlFzIHRvIGJhbGFuY2UKPiAgICAgIGFjcm9zcyBjb3JlcyBidXQgdGhp cyB3aWxsIGZhaWwuCgpPciB2aWEgdGhlIHRoZSBtYW5hZ2VkIGludGVycnVwdHMuIEJ1dCB0aGlz IGlzIGEgbm9uLWlzc3VlLCBhcyBwb2ludGVkCm91dCBpbiBteSByZXBseSB0byAxLgoKPiA0KSBI QVJUIGhvdHBsdWcgYnJlYWtzIGJlY2F1c2Uga2VybmVsIGlycS1zdWJzeXN0ZW0gY2FuJ3QKPiAg ICAgbWlncmF0ZSB0aGUgSVJRcyAoYm90aCBNU0lzIGFuZCBXaXJlZCkgdGFyZ2V0aW5nIEhBUlQg WAo+ICAgICB0byBhbm90aGVyIEhBUlQgWSB3aGVuIHRoZSBIQVJUIFggZ29lcyBkb3duLgoKWWVz LCB3ZSBtaWdodCBlbmQgdXAgaW4gc2NlbmFyaW9zIHdoZXJlIHdlIGNhbid0IG1vdmUgdG8gYSBj ZXJ0YWluCnRhcmdldCBjcHUsIGJ1dCBJIHdvdWxkbid0IGV4cGVjdCB0aGF0IHRvIGJlIGEgY29t bW9uIHNjZW5hcmlvLgoKPiBUaGUgaWRlYSBvZiB0cmVhdGluZyBwZXItSEFSVCBNU0lzIGFzIHNl cGFyYXRlIElSUXMgaGFzCj4gYmVlbiBkaXNjdXNzZWQgaW4gdGhlIHBhc3QuCgpBaGEhIEkgdHJp ZWQgdG8gbG9vayBmb3IgaXQgaW4gbG9yZSwgYnV0IGRpZG4ndCBmaW5kIGFueS4gQ291bGQgeW91 CnBvaW50IG1lIHRvIHRob3NlIGRpc2N1c3Npb25zPwoKPiBBbHNvLCB0aGUgY3VycmVudCBhcHBy b2FjaCBpcyB2ZXJ5IHNpbWlsYXIgdG8gdGhlIEFSTSBHSUN2MyBkcml2ZXIKPiB3aGVyZSBJVFMg TFBJcyBhY3Jvc3MgQ1BVcyBhcmUgdHJlYXRlZCBhcyBzaW5nbGUgSVJRLgoKSSdtIG5vdCBmYW1p bGlhciB3aXRoIHRoZSBHSUMuIElzIHRoZSBHSUN2MyBkZXNpZ24gc2ltaWxhciB0byBJTVNJQz8g SQpoYWQgdGhlIGltcHJlc3Npb24gdGhhdCB0aGUgR0lDIGhhZCBhIG1vcmUgYWR2YW5jZWQgaW50 ZXJydXB0IHJvdXRpbmcKbWVjaGFuaXNtLCB0aGFuIHdoYXQgSU1TSUMgZXhwb3Nlcy4gSSB0aGlu ayB4ODYgQVBJQyB0YWtlcyB0aGUgMS0xCmFwcHJvYWNoICh0aGUgZm9sa3Mgb24gdGhlIFRvOiBs aXN0IGRlZmluaXRlbHkga25vd3MhIDstKSkuCgpNeSBjb25jZXJuIGlzIGludGVycnVwdHMgYmVj b21lIGEgc2NhcmNlIHJlc291cmNlIHdpdGggdGhpcwppbXBsZW1lbnRhdGlvbiwgYnV0IG1heWJl IG15IHZpZXcgaXMgaW5jb3JyZWN0LiBJJ3ZlIHNlZW4gYmFyZS1tZXRhbAp4ODYgc3lzdGVtcyAo bm8gVk1zKSB3aXRoIH4yMDAgY29yZXMsIGFuZCB+MjAwMCBpbnRlcnJ1cHRzLCBidXQgbWF5YmUK dGhhdCBpcyBjb25zaWRlcmVkICJhIGxvdCBvZiBpbnRlcnJ1cHRzIi4KCkFzIGxvbmcgYXMgd2Ug ZG9uJ3QgZ2V0IGludG8gc2NlbmFyaW9zIHdoZXJlIHdlJ3JlIHJ1bm5pbmcgb3V0IG9mCmludGVy cnVwdHMsIGR1ZSB0byB0aGUgc29mdHdhcmUgZGVzaWduLgoKCkJqw7ZybgoKX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGludXgtcmlzY3YgbWFpbGluZyBs aXN0CmxpbnV4LXJpc2N2QGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVh ZC5vcmcvbWFpbG1hbi9saXN0aW5mby9saW51eC1yaXNjdgo= From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E5C0210A03 for ; Fri, 20 Oct 2023 08:47:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="u5USE2+U" Received: by smtp.kernel.org (Postfix) with ESMTPSA id F023CC433C8; Fri, 20 Oct 2023 08:47:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1697791656; bh=yvS57ci7fpW8c627QySzOjgk0/AmuicEXD3GQA5Hz+U=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=u5USE2+U4vh8XD5mIYisLUnT1GNJZJp49PPKHE18d3VDUEQ0SkrSGdbA8H7UVvmrn yrHRSOqdWYUb2fGuE0GjmXM9sUs1GVwHfFwrjZqv53cwcQ0fNqiRATQ9eX8i0ItkCM km8CVxnx/ItN4OqVASHYsu5bp8UadjTNEpNPznuAE7zH0xpLoiXK7EEtmEWwocdscQ Okmk9ofjp4lGSUG3KolizoY/WI9fPQ7BWcZl+CawkxAmnHkrh89gu7iNlQ8nva136U 7i1vATxQ6ttSgT6DKm1m1kVpTkWblbWz8YfqwQE2lUWOimOBz7cwtWLDYa1BUHLZ3h 7Py33kx/e/dMw== From: =?utf-8?B?QmrDtnJuIFTDtnBlbA==?= To: Anup Patel Cc: Palmer Dabbelt , Paul Walmsley , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Frank Rowand , Conor Dooley , Atish Patra , Andrew Jones , Sunil V L , Saravana Kannan , Anup Patel , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org Subject: Re: [PATCH v10 00/15] Linux RISC-V AIA Support In-Reply-To: References: <20231003044403.1974628-1-apatel@ventanamicro.com> <87o7gu7mo9.fsf@all.your.base.are.belong.to.us> Date: Fri, 20 Oct 2023 10:47:33 +0200 Message-ID: <87h6mlbryy.fsf@all.your.base.are.belong.to.us> Precedence: bulk X-Mailing-List: devicetree@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Thanks for the quick reply! Anup Patel writes: > On Thu, Oct 19, 2023 at 7:13=E2=80=AFPM Bj=C3=B6rn T=C3=B6pel wrote: >> >> Hi Anup, >> >> Anup Patel writes: >> >> > The RISC-V AIA specification is ratified as-per the RISC-V internation= al >> > process. The latest ratified AIA specifcation can be found at: >> > https://github.com/riscv/riscv-aia/releases/download/1.0/riscv-interru= pts-1.0.pdf >> > >> > At a high-level, the AIA specification adds three things: >> > 1) AIA CSRs >> > - Improved local interrupt support >> > 2) Incoming Message Signaled Interrupt Controller (IMSIC) >> > - Per-HART MSI controller >> > - Support MSI virtualization >> > - Support IPI along with virtualization >> > 3) Advanced Platform-Level Interrupt Controller (APLIC) >> > - Wired interrupt controller >> > - In MSI-mode, converts wired interrupt into MSIs (i.e. MSI generat= or) >> > - In Direct-mode, injects external interrupts directly into HARTs >> >> Thanks for working on the AIA support! I had a look at the series, and >> have some concerns about interrupt ID abstraction. >> >> A bit of background, for readers not familiar with the AIA details. >> >> IMSIC allows for 2047 unique MSI ("msi-irq") sources per hart, and >> each MSI is dedicated to a certain hart. The series takes the approach >> to say that there are, e.g., 2047 interrupts ("lnx-irq") globally. >> Each lnx-irq consists of #harts * msi-irq -- a slice -- and in the >> slice only *one* msi-irq is acutally used. >> >> This scheme makes affinity changes more robust, because the interrupt >> sources on "other" harts are pre-allocated. On the other hand it >> requires to propagate irq masking to other harts via IPIs (this is >> mostly done up setup/tear down). It's also wasteful, because msi-irqs >> are hogged, and cannot be used. >> >> Contemporary storage/networking drivers usually uses queues per core >> (or a sub-set of cores). The current scheme wastes a lot of msi-irqs. >> If we instead used a scheme where "msi-irq =3D=3D lnx-irq", instead of >> "lnq-irq =3D {hart 0;msi-irq x , ... hart N;msi-irq x}", there would be >> a lot MSIs for other users. 1-1 vs 1-N. E.g., if a storage device >> would like to use 5 queues (5 cores) on a 128 core system, the current >> scheme would consume 5 * 128 MSIs, instead of just 5. >> >> On the plus side: >> * Changing interrupts affinity will never fail, because the interrupts >> on each hart is pre-allocated. >> >> On the negative side: >> * Wasteful interrupt usage, and a system can potientially "run out" of >> interrupts. Especially for many core systems. >> * Interrupt masking need to proagate to harts via IPIs (there's no >> broadcast csr in IMSIC), and a more complex locking scheme IMSIC >> >> Summary: >> The current series caps the number of global interrupts to maximum >> 2047 MSIs for all cores (whole system). A better scheme, IMO, would be >> to expose 2047 * #harts unique MSIs. >> >> I think this could simplify/remove(?) the locking as well. > > Exposing 2047 * #harts unique MSIs has multiple issues: > 1) The irq_set_affinity() does not work for MSIs because each > IRQ is not tied to a particular HART. This means we can't > balance the IRQ processing load among HARTs. Yes, you can balance. In your code, each *active* MSI is still bound/active to a specific hard together with the affinity mask. In an 1-1 model you would still need to track the affinity mask, but the irq_set_affinity() would be different. It would try to allocate a new MSI from the target CPU, and then switch to having that MSI active. That's what x86 does AFAIU, which is also constrained by the # of available MSIs. The downside, as I pointed out, is that the set affinity action can fail for a certain target CPU. > 2) All wired IRQs for APLIC MSI-mode will also target a > fixed HART hence irq_set_affinity() won't work for wired > IRQs as well. I'm not following here. Why would APLIC put a constraint here? I had a look at the specs, and I didn't see anything supporting the current scheme explicitly. > 3) Contemporary storage/networking drivers which use per-core > queues use irq_set_affinity() on queue IRQs to balance > across cores but this will fail. Or via the the managed interrupts. But this is a non-issue, as pointed out in my reply to 1. > 4) HART hotplug breaks because kernel irq-subsystem can't > migrate the IRQs (both MSIs and Wired) targeting HART X > to another HART Y when the HART X goes down. Yes, we might end up in scenarios where we can't move to a certain target cpu, but I wouldn't expect that to be a common scenario. > The idea of treating per-HART MSIs as separate IRQs has > been discussed in the past. Aha! I tried to look for it in lore, but didn't find any. Could you point me to those discussions? > Also, the current approach is very similar to the ARM GICv3 driver > where ITS LPIs across CPUs are treated as single IRQ. I'm not familiar with the GIC. Is the GICv3 design similar to IMSIC? I had the impression that the GIC had a more advanced interrupt routing mechanism, than what IMSIC exposes. I think x86 APIC takes the 1-1 approach (the folks on the To: list definitely knows! ;-)). My concern is interrupts become a scarce resource with this implementation, but maybe my view is incorrect. I've seen bare-metal x86 systems (no VMs) with ~200 cores, and ~2000 interrupts, but maybe that is considered "a lot of interrupts". As long as we don't get into scenarios where we're running out of interrupts, due to the software design. Bj=C3=B6rn