From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CEACCE7716B for ; Wed, 4 Dec 2024 14:14:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:References :In-Reply-To:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=9o/iOcCVsq6WDRcuiRwJ92TkpiPmob5Dgd3FIC6pvwo=; b=E5ZsyMxnjSQ7Fv XfF84F83Jrd4DcvygPHVsjSPSJiLMNA/Ktp8rJhc8mWWelVRbe5zKv0+XkIL+mqdZ+w+aLivONKAh JJ6TylIqU/2UdlU9xFUdN++295rvVJJ31Yy9LXe7cujDiycpm/IZGnjSqVKSKtoZlB29pbDnKT89N 54YlK97h58U1yswqgWcgEx+8pQryiSpiPyiOMgh3RCV53edrzkVm5LnKmjhTGGI3egH6VHUALeG8w WaJxDiEuNPjEKLJB1JSBbQ/1xa9xkK2BMUJGdmrZ8h1vpldjHC7ZnjTDkGAlRVoRr56cyaCYPn621 5D+ioZAbiK8D5SJCjEZg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tIq8f-0000000CnE7-1wxw; Wed, 04 Dec 2024 14:14:01 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tIp4M-0000000CdXu-2ESW; Wed, 04 Dec 2024 13:05:30 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:Content-Type :MIME-Version:Message-ID:Date:References:In-Reply-To:Subject:Cc:To:From: Sender:Reply-To:Content-ID:Content-Description; bh=ROH6zlh02KHc87r45BDTv/cN5afbSIuh9toQ2ctg4yk=; b=lufgGgKCwWsjX/1baicZDVMhDV ZfN/kdGAuI1+JH/hUGSmeAzvvg/Vg/hDEAwBQOyv/jQK1wYym7Nz0TXWtA3TT5cdTgadbuQr4RaxG SatZKKyrhxl+lt3lhl6oY8wHgC4hZFE44bTrN3juZGz/7GPQ744jYukuibGcAe8mLbRRK9tdrAA4L oCtuSSGFMqdOryRWVgPYZ9A2nzDjvuFlm4ixaZVXYKbC8nNY+/NMHFdmU1n1bE2paoNpnP1l8/ysH CeJuaBkh7ic5MCxqBVyYzNnxTiDODnBIex+CcoPY3MXXMeDZ3oONOAmVhEkaUkoGMmt22YgqKWlHN ScFEzePg==; Received: from galois.linutronix.de ([193.142.43.55]) by desiato.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tIp4J-00000002baM-2Jpv; Wed, 04 Dec 2024 13:05:29 +0000 From: Thomas Gleixner DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1733317524; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ROH6zlh02KHc87r45BDTv/cN5afbSIuh9toQ2ctg4yk=; b=wL73UjeXWyYQQmB1Pp1remQXdjAmCrtcWFIChE7E9HHlqL9M/l/akN/1PUyWWcio+M4hsd d7PhJzhCnsWkx4s2pdhDctBT82t3yAu7JL5tjJcYh5qKSbFl1hdbxAKNn5Bt1Rnq8D3xyh 1W12Torjr4e84kQ605+X6R3Xvphy304E3U+iWFWCCSOPeoenFGFisFQOMcWHZkqvkIm/Yo rqrivG3Lb9g3nGu76ZInJIUVeq0VCHCnHokMrMp17Gv781GoLj6bgF39qb7BhDi4tsMRg6 qBrRpQNn0HXqJMPWSPvD3rrF6D76de00gbF5IouDkUgEw4GC/F5dlO0bLlErmA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1733317524; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ROH6zlh02KHc87r45BDTv/cN5afbSIuh9toQ2ctg4yk=; b=qPufNg0aO0AVR52BVTeAWUHTNUKlqiQiXfPzDa82iuoktna7Zd4GaEhPSAChG0cycCtYIt oiuHXrVUC/U2oQBQ== To: Anup Patel Cc: Andrew Jones , iommu@lists.linux.dev, kvm-riscv@lists.infradead.org, kvm@vger.kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, tjeznach@rivosinc.com, zong.li@sifive.com, joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, atishp@atishpatra.org, alex.williamson@redhat.com, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu Subject: Re: [RFC PATCH 01/15] irqchip/riscv-imsic: Use hierarchy to reach irq_set_affinity In-Reply-To: References: <20241114161845.502027-17-ajones@ventanamicro.com> <20241114161845.502027-18-ajones@ventanamicro.com> <87mshcub2u.ffs@tglx> <874j3ktrjv.ffs@tglx> <87ser4s796.ffs@tglx> Date: Wed, 04 Dec 2024 14:05:23 +0100 Message-ID: <87mshbsing.ffs@tglx> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241204_130527_862699_7059D205 X-CRM114-Status: GOOD ( 47.32 ) X-BeenThere: kvm-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "kvm-riscv" Errors-To: kvm-riscv-bounces+kvm-riscv=archiver.kernel.org@lists.infradead.org T24gV2VkLCBEZWMgMDQgMjAyNCBhdCAwOToxMywgQW51cCBQYXRlbCB3cm90ZToKPiBPbiBXZWQs IERlYyA0LCAyMDI0IGF0IDQ6MjnigK9BTSBUaG9tYXMgR2xlaXhuZXIgPHRnbHhAbGludXRyb25p eC5kZT4gd3JvdGU6Cj4+IEFzIEkgd2FzIGxvb2tpbmcgYXQgc29tZXRoaW5nIGVsc2UgTVNJIHJl bGF0ZWQgSSBoYWQgYSBsb29rIGF0Cj4+IGltc2ljX2lycV9zZXRfYWZmaW5pdHkoKSBhZ2Fpbi4K Pj4KPj4gSXQncyBhY3R1YWxseSByZXF1aXJlZCB0byBoYXZlIHRoZSBtZXNzYWdlIHdyaXRlIGlu IHRoYXQgZnVuY3Rpb24gYW5kCj4+IG5vdCBhZnRlcndhcmRzIGFzIHlvdSBpbnZva2UgaW1zaWNf dmVjdG9yX21vdmUoKSBmcm9tIHRoYXQgZnVuY3Rpb24uCj4+Cj4+IFRoYXQncyBvYnZpb3VzbHkg bm90IHRydWUgZm9yIHRoZSByZW1hcCBjYXNlIGFzIHRoYXQgd2lsbCBub3QgY2hhbmdlIHRoZQo+ PiBtZXNzYWdlIGFkZHJlc3MvZGF0YSBwYWlyIGJlY2F1c2UgdGhlIHJlbWFwIHRhYmxlIGVudHJ5 IGlzIGltbXV0YWJsZSAtCj4+IGF0IGxlYXN0IEkgYXNzdW1lIHNvIGZvciBteSBtZW50YWwgc2Fu aXR5IHNha2UgOikKPj4KPj4gQnV0IHRoYXQgYnJpbmdzIG1lIHRvIGEgcmVsYXRlZCBxdWVzdGlv bi4gSG93IGlzIHRoaXMgc3VwcG9zZWQgdG8gd29yawo+PiB3aXRoIG5vbi1hdG9taWMgbWVzc2Fn ZSB1cGRhdGVzPyBQQ0kvTVNJIGRvZXMgbm90IG5lY2Vzc2FyaWx5IHByb3ZpZGUKPj4gbWFza2lu ZywgYW5kIHRoZSB3cml0ZSBvZiB0aGUgYWRkcmVzcy9kYXRhIHBhaXIgaXMgZG9uZSBpbiBiaXRz IGFuZAo+PiBwaWVjZXMuIFNvIHlvdSBjYW4gZW5kIHVwIHdpdGggYW4gaW50ZXJtZWRpYXRlIHN0 YXRlIHNlZW4gYnkgdGhlIGRldmljZQo+PiB3aGljaCBlbmRzIHVwIHNvbWV3aGVyZSBpbiBpbnRl cnJ1cHQgbmlydmFuYSBzcGFjZS4KPj4KPj4gU2VlIHRoZSBkYW5jZSBpbiBtc2lfc2V0X2FmZmlu aXR5KCkgYW5kIGNvbW1pdCA2ZjFhNDg5MWE1OTIKPj4gKCJ4ODYvYXBpYy9tc2k6IFBsdWcgbm9u LW1hc2thYmxlIE1TSSBhZmZpbml0eSByYWNlIikgZm9yIGZ1cnRoZXIKPj4gZXhwbGFuYXRpb24u Cj4+Cj4+IFRoZSB3YXkgaG93IHRoZSBJTVNJQyBkcml2ZXIgd29ya3Mgc2VlbXMgdG8gYmUgcHJl dHR5IG11Y2ggdGhlIHNhbWUgYXMKPj4gdGhlIHg4NiBBUElDIG1lc3M6Cj4+Cj4+ICAgICAgICAg QGFkZHJlc3MgaXMgdGhlIHBoeXNpY2FsIGFkZHJlc3Mgb2YgdGhlIHBlciBDUFUgTVNJIHRhcmdl dAo+PiAgICAgICAgIGFkZHJlc3MgYW5kIEBkYXRhIGlzIHRoZSB2ZWN0b3IgSUQgb24gdGhhdCBD UFUuCj4+Cj4+IFNvIHRoZSBub24tYXRvbWljIHVwZGF0ZSBpbiBjYXNlIG9mIG5vbi1tYXNrYWJs ZSBNU0kgc3VmZmVycyBmcm9tIHRoZQo+PiBzYW1lIHByb2JsZW0uIEl0IHdvcmtzIG1vc3Qgb2Yg dGhlIHRpbWUsIGJ1dCBpZiBpdCBkb2Vzbid0IHlvdSBtaWdodAo+PiBzdGFyZSBhdCB0aGUgb2Nj YXNpb25hbGx5IGxvc3QgaW50ZXJydXB0IGFuZCB0aGUgc3RhbGUgZGV2aWNlIGluCj4+IGRpc2Jl bGllZiBmb3IgcXVpdGUgYSB3aGlsZSA6KQo+Cj4gWWVzLCB3ZSBoYXZlIHRoZSBzYW1lIGNoYWxs ZW5nZXMgYXMgeDg2IEFQSUMgd2hlbiBjaGFuZ2luZwo+IE1TSSBhZmZpbml0eS4KPj4KPj4gSSBt aWdodCBiZSBtaXNzaW5nIHNvbWV0aGluZyB3aGljaCBtYWdpY2FsbHkgcHJldmVudCB0aGF0IHRo b3VnaCA6KQo+Pgo+IFlvdXIgdW5kZXJzdGFuZGluZyBpcyBjb3JyZWN0LiBJbiBmYWN0LCB0aGUg SU1TSUMgbXNpX3NldF9hZmZpbml0eSgpCj4gaGFuZGxpbmcgaXMgaW5zcGlyZWQgZnJvbSB4ODYg QVBJQyBhcHByb2FjaCBkdWUgdG8gc2ltaWxhcml0eSBpbgo+IHRoZSBvdmVyYWxsIE1TSSBjb250 cm9sbGVyLgo+Cj4gVGhlIGhpZ2gtbGV2ZWwgaWRlYSBvZiBpbXNpY19pcnFfc2V0X2FmZmluaXR5 KCkgaXMgYXMgZm9sbG93czoKPgo+IDEpIEFsbG9jYXRlIG5ld192ZWN0b3IgKG5ldyBDUFUgSU1T SUMgYWRkcmVzcyArIG5ldyBJRCBvbiB0aGF0IENQVSkKPgo+IDIpIFVwZGF0ZSB0aGUgTVNJIGFk ZHJlc3MgYW5kIGRhdGEgcHJvZ3JhbW1lZCBpbiB0aGUgZGV2aWNlCj4gYmFzZWQgb24gbmV3X3Zl Y3RvciAoc2VlIGltc2ljX21zaV91cGRhdGVfbXNnKCkpCj4KPiAzKSBBdCB0aGlzIHBvaW50IHRo ZSBkZXZpY2UgcG9pbnRzIHRvIHRoZSBuZXdfdmVjdG9yIGJ1dCBvbGRfdmVjdG9yCj4gKG9sZCBD UFUgSU1TSUMgYWRkcmVzcyArIG9sZCBJRCBvbiB0aGF0IENQVSkgaXMgc3RpbGwgZW5hYmxlZCBh bmQKPiB3ZSBtaWdodCBoYXZlIHJlY2VpdmVkIE1TSSBvbiBvbGRfdmVjdG9yIHdoaWxlIHdlIHdl cmUgYnVzeQo+IHNldHRpbmcgdXAgYSBuZXdfdmVjdG9yIGZvciB0aGUgZGV2aWNlLiBUbyBhZGRy ZXNzIHRoaXMsIHdlIGNhbGwKPiBpbXNpY192ZWN0b3JfbW92ZSgpLgo+Cj4gNCkgVGhlIGltc2lj X3ZlY3Rvcl9tb3ZlKCkgbWFya3MgdGhlIG9sZF92ZWN0b3IgYXMgYmVpbmcKPiBtb3ZlZCBhbmQg c2NoZWR1bGVzIGEgbGF6eSB0aW1lciBvbiB0aGUgb2xkIENQVS4KPgo+IDUpIFRoZSBsYXp5IHRp bWVyIGV4cGlyZXMgb24gdGhlIG9sZCBDUFUgYW5kIHJlc3VsdHMgaW4KPiBfX2ltc2ljX2xvY2Fs X3N5bmMoKSBiZWluZyBjYWxsZWQgb24gdGhlIG9sZCBDUFUuCj4KPiA2KSBJZiB0aGVyZSB3YXMg YSBwZW5kaW5nIE1TSSBvbiB0aGUgb2xkIHZlY3RvciB0aGVuIHRoZQo+IF9faW1zaWNfbG9jYWxf c3luYygpIGZ1bmN0aW9uIGluamVjdHMgYW4gTVNJIHRvIHRoZQo+IG5ld192ZWN0b3IgdXNpbmcg YW4gTU1JTyB3cml0ZS4KPgo+IEl0IGlzIHZlcnkgdW5saWtlbHkgdGhhdCBhbiBNU0kgZnJvbSBk ZXZpY2Ugd2lsbCBiZSBkcm9wcGVkCj4gKHVubGVzcyBJIGFtIG1pc3Npbmcgc29tZXRoaW5nKSBi dXQgdGhlIHVuc29sdmVkIGlzc3VlCj4gaXMgdGhhdCBoYW5kbGluZyBvZiBpbi1mbGlnaHQgTVNJ IHJlY2VpdmVkIG9uIHRoZSBvbGRfdmVjdG9yCj4gZHVyaW5nIHRoZSBNU0kgcmUtcHJvZ3JhbW1p bmcgaXMgZGVsYXllZCB3aGljaCBtYXkgaGF2ZQo+IHNpZGUgZWZmZWN0cyBvbiB0aGUgZGV2aWNl IGRyaXZlciBzaWRlLgoKSW50ZXJydXB0IGRlbGl2ZXJ5IGNhbiBiZSBkZWxheWVkIGZvciB0b25z IG9mIG90aGVyIHJlYXNvbnMuCgpCdXQgeWVzLCB5b3UgYXJlIG1pc3Npbmcgc29tZXRoaW5nIHdo aWNoIGlzIHdvcnNlIHRoYW4gYSBqaWZmaWUgZGVsYXk6CgpDUFUgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgRGV2aWNlCiAgbXNpX3VwZGF0ZV9tc2coKQogICAgIGNvbXBvc2Uo KQogICAgIHdyaXRlKCkKICAgICAgIHdyaXRlKG1zZy0+YWRkcmVzc19sbyk7IFsxXQogICAgICAg d3JpdGUobXNnLT5hZGRyZXNzX2hpKTsgWzJdCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICBSYWlzZXMgaW50ZXJydXB0IFszXQogICAgICAgd3JpdGUobXNnLT5kYXRhKTsg ICAgICAgWzRdCgpbMl0gY2FuIGJlIGlnbm9yZWQgYXMgaXQgc2hvdWxkIG5vdCBjaGFuZ2UgKG90 aGVyd2lzZSAzMmJpdCBvbmx5IGRldmljZXMKd291bGQgbm90IHdvcmspLgoKTGV0cyBhc3N1bWUg dGhhdCB0aGUgb3JpZ2luYWwgbWVzc2FnZSB3YXM6CgogICAgIGFkZHJfbG8gPSAweDEwMDAsIGRh dGEgPSAweDEwICAgIChDUFUxLCB2ZWN0b3IgMHgxMCkKICAgICAgIApUaGUgbmV3IG1lc3NhZ2Ug aXMKCiAgICAgYWRkcl9sbyA9IDB4MjAwMCwgZGF0YSA9IDB4MjAgICAgKENQVTIsIHZlY3RvciAw eDIwKQoKQWZ0ZXIgWzJdIHRoZSBkZXZpY2Ugc2VlczoKCiAgICAgYWRkcl9sbyA9IDB4MjAwMCwg ZGF0YSA9IDB4MTAgICAgKENQVTIsIHZlY3RvciAweDEwKQoKVGhlIGludGVycnVwdCByYWlzZWQg aW4gWzNdIHdpbGwgZW5kIHVwIG9uIENQVTIgYXQgdmVjdG9yIDB4MTAgd2hpY2gKbWlnaHQgYmUg bm90IGluIHVzZSBvciB1c2VkIGJ5IHNvbWUgb3RoZXIgZGV2aWNlLiBJbiBhbnkgY2FzZSB0aGUK aW50ZXJydXB0IGlzIG5vdCByZWFjaGluZyB0aGUgcmVhbCBkZXZpY2UgaGFuZGxlciBhbmQgeW91 IGNhbid0IHNlZSB0aGF0CmludGVycnVwdCBhcyBwZW5kaW5nIGluIENQVTEuCgpUaGF0J3Mgd2h5 IHg4NiBpbiBjYXNlIG9mIG5vbi1yZW1hcHBlZCBpbnRlcnJ1cHRzIGhhcyB0aGlzIGZvciB0aGUg dHdvCnNpdHVhdGlvbnM6CgogIDEpIG9sZC0+ZGF0YSA9PSBuZXctPmRhdGEKCiAgICAgd3JpdGVf bXNnKG5ldyk7IAoKICAgICBUaGUgbmV4dCBpbnRlcnJ1cHQgZWl0aGVyIGFycml2ZXMgb24gdGhl IG9sZCBDUFUgb3Igb24gdGhlIG5ldyBDUFUKICAgICBkZXBlbmRpbmcgb24gdGltaW5nLiBUaGVy ZSBpcyBubyBpbnRlcm1lZGlhdGUgc3RhdGUgYmVjYXVzZSB0aGUKICAgICB2ZWN0b3IgKGRhdGEp IGlzIHRoZSBzYW1lIG9uIGJvdGggQ1BVcy4KCiAgMikgb2xkLT5kYXRhICE9IG5ldy0+ZGF0YQoK ICAgICB0bXBfbXNnLmFkZHIgPSBvbGRfbXNnLmFkZHIKICAgICB0bXBfbXNnLmRhdGEgPSBuZXdf bXNnLmRhdGEKCiAgICAgd3JpdGVfbXNnKHRtcF9tc2cpOwoKICAgICBTbyBhZnRlciB0aGF0IHdy aXRlIHRoZSBkZXZpY2UgbWlnaHQgcmFpc2UgdGhlIGludGVycnVwdCBvbiBDUFUxCiAgICAgYW5k IHZlY3RvciAweDIwLgoKICAgICBUaGUgbmV4dCBzdGVwIGlzCgogICAgIHdyaXRlX21zZyhuZXcp OyAKCiAgICAgd2hpY2ggY2hhbmdlcyB0aGUgZGVzdGluYXRpb24gQ1BVIHRvIENQVTIuCgogICAg IFNvIGRlcGVuZGluZyB3aGF0IHRoZSBkZXZpY2UgaGFzIG9ic2VydmVkIHRoZSBpbnRlcnJ1cHQg bWlnaHQgZW5kCiAgICAgdXAgb24KCiAgICAgQ1BVMSB2ZWN0b3IgMHgxMCAgIChvbGQpCiAgICAg Q1BVMSB2ZWN0b3IgMHgyMCAgICh0bXApCiAgICAgQ1BVMiB2ZWN0b3IgMHgyMCAgIChuZXcpCgog ICAgIENQVTEgdmVjdG9yIDB4MjAgKHRtcCkgaXMgdGhlbiBjaGVja2VkIGluIHRoZSBwZW5kaW5n IHJlZ2lzdGVyIGFuZAogICAgIHRoZSBpbnRlcnJ1cHQgaXMgcmV0cmlnZ2VyZWQgaWYgcGVuZGlu Zy4KClRoYXQgcmVxdWlyZXMgdG8gbW92ZSB0aGUgaW50ZXJydXB0IGZyb20gYWN0dWFsIGludGVy cnVwdCBjb250ZXh0IG9uIHRoZQpvbGQgdGFyZ2V0IENQVS4gSXQgYWxsb3dzIHRvIGV2YWx1YXRl IHRoZSBvbGQgdGFyZ2V0IENQVXMgcGVuZGluZwpyZWdpc3RlciB3aXRoIGxvY2FsIGludGVycnVw dHMgZGlzYWJsZWQsIHdoaWNoIG9idmlvdXNseSBkb2VzIG5vdCB3b3JrCnJlbW90ZS4KCkkgZG9u J3Qgc2VlIGEgd2F5IGhvdyB0aGF0IGNhbiB3b3JrIHJlbW90ZSB3aXRoIHRoZSBJTVNJQyBlaXRo ZXIgZXZlbiBpZgp5b3UgY2FuIGVhc2lseSBhY2Nlc3MgdGhlIHBlbmRpbmcgc3RhdGUgb2YgdGhl IHJlbW90ZSBDUFU6CgpDUFUwICAgICAgICAgICAgICAgICAgICAgICAgICAgIENQVTEgICAgICAg ICAgICAgICAgICAgRGV2aWNlCnNldF9hZmZpbml0eSgpCiAgd3JpdGVfbXNnKHRtcCkKICAgIHdy aXRlKGFkZHIpOyAvLyBDUFUxCiAgICB3cml0ZShkYXRhKTsgLy8gdmVjdG9yIDB4MjAKICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICByYWlzZSBJ UlEKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBoYW5kbGUgdmVjdG9yIDB4MjAKICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAoc3B1cmlvdXMgb3Igb3RoZXIgZGV2aWNlKQoK ICAgIGNoZWNrX3BlbmRpbmcoQ1BVMSwgMHgyMCkgPT0gZmFsc2UgLT4gSW50ZXJydXB0IGlzIGxv c3QKClJlbWFwcGVkIGludGVycnVwdHMgZG8gbm90IGhhdmUgdGhhdCBpc3N1ZSBiZWNhdXNlIHRo ZSB0YWJsZSB1cGRhdGUgaXMKYXRvbWljIHZzLiBhIGNvbmN1cnJlbnRseSByYWlzZWQgaW50ZXJy dXB0LCBzbyB0aGF0IGl0IGVpdGhlciBlbmRzIHVwIG9uCnRoZSBvbGQgb3Igb24gdGhlIG5ldyBk ZXN0aW5hdGlvbi4gVGhlIGRldmljZSBtZXNzYWdlIGRvZXMgbm90IGNoYW5nZSBhcwppdCBhbHdh eXMgdGFyZ2V0cyB0aGUgdGFibGUgZW50cnksIHdoaWNoIGlzIGltbXV0YWJsZSBhZnRlciBzZXR1 cC4KClRoYXQncyB3aHkgeDg2IGhhcyB0aGlzIHNwZWNpYWwgbXNpIGFmZmluaXR5IHNldHRlciBm b3IgdGhlIG5vbiByZW1hcHBlZApjYXNlLiBGb3IgdGhlIHJlbWFwcGVkIGNhc2UgaXQncyBqdXN0 IHVzaW5nIHRoZSBoaWVyYXJjaHkgZGVmYXVsdCB3aGljaAplbmRzIHVwIGF0IHRoZSByZW1hcCBk b21haW4uCgpTbyB0aGUgaGllcmFyY2hpZXMgbG9vayBsaWtlIHRoaXM6CgogICAxKSBub24tcmVt YXAKCiAgICAgIFBDSS9NU0kgZGV2aWNlIGRvbWFpbgogICAgICAgICBpcnFfY2hpcC5pcnFfc2V0 X2FmZmluaXR5ID0gbXNpX3NldF9hZmZpbml0eTsKICAgICAgVkVDVE9SIGRvbWFpbgoKICAgMikg cmVtYXAKCiAgICAgIFBDSS9NU0kgZGV2aWNlIGRvbWFpbgogICAgICAgICBpcnFfY2hpcC5pcnFf c2V0X2FmZmluaXR5ID0gbXNpX2RvbWFpbl9zZXRfYWZmaW5pdHk7CiAgICAgIFJFTUFQIGRvbWFp bgogICAgICAgICBpcnFfY2hpcC5pcnFfc2V0X2FmZmluaXR5ID0gcmVtYXBfc2V0X2FmZmluaXR5 OwogICAgICBWRUNUT1IgZG9tYWluCgpUaGUgc3BlY2lhbCBjYXNlIG9mIG1zaV9zZXRfYWZmaW5p dHkoKSBpcyBub3QgaW52b2x2ZWQgaW4gdGhlIHJlbWFwIGNhc2UKYW5kIHNvbGVseSB1dGlsaXpl ZCBmb3IgdGhlIG5vbi1yZW1hcCBob3Jyb3JzLiBUaGUgcmVtYXAgY2FzZSBkb2VzIG5vdApyZXF1 aXJlIHRoZSBpbnRlcnJ1cHQgdG8gYmUgbW92ZWQgaW4gaW50ZXJydXB0IGNvbnRleHQgZWl0aGVy IGJlY2F1c2UKdGhlcmUgaXMgbm8gaW50ZXJtZWRpYXRlIHN0ZXAgYW5kIHBlbmRpbmcgcmVnaXN0 ZXIgY2hlY2sgb24gdGhlIG9sZCBDUFUKaW52b2x2ZWQuCgpUaGUgdmVjdG9yIGNsZWFudXAgb2Yg dGhlIG9sZCBDUFUgYWx3YXlzIGhhcHBlbnMgd2hlbiB0aGUgaW50ZXJydXB0CmFycml2ZXMgb24g dGhlIG5ldyB0YXJnZXQgZm9yIHRoZSBmaXJzdCB0aW1lIGluZGVwZW5kZW50IG9mIHJlbWFwcGlu Zwptb2RlLiBUaGF0J3MgcmVxdWlyZWQgZm9yIGJvdGggY2FzZXMgdG8gdGFrZSBjYXJlIG9mIHRo ZSBjYXNlIHdoZXJlIHRoZQppbnRlcnJ1cHQgaXMgcmFpc2VkIG9uIHRoZSBvbGQgdmVjdG9yIChj cHUxLCAweDEwKSBiZWZvcmUgdGhlIHdyaXRlCmhhcHBlbnMgYW5kIGlzIHBlbmRpbmcgaW4gQ1BV MS4gU28gYWZ0ZXIgZGVzYzo6bG9jayBpcyByZWxlYXNlZCBhbmQKaW50ZXJydXB0cyBhcmUgcmVl bmFibGVkIHRoZSBpbnRlcnJ1cHQgaXMgaGFuZGxlZCBvbiBDUFUxLiBUaGUgbmV4dCBvbmUKaXMg Z3VhcmFudGVlZCB0byBhcnJpdmUgb24gQ1BVMiBhbmQgdGhhdCB0cmlnZ2VycyB0aGUgY2xlYW4g dXAgb2YgdGhlCkNQVTEgdmVjdG9yLCB3aGljaCByZWxlYXNlcyBpdCBmb3IgcmV1c2UgaW4gdGhl IG1hdHJpeCBhbGxvY2F0b3IuCgpJIHRoaW5rIHlvdSBzaG91bGQgbW9kZWwgaXQgaW4gYSBzaW1p bGFyIHdheSBpbnN0ZWFkIG9mIHRyeWluZyB0bwphcnRpZmljaWFsbHkgcmV1c2UgaW1zaWNfaXJx X3NldF9hZmZpbml0eSgpIGZvciB0aGUgcmVtYXAgY2FzZSwKZXNwZWNpYWxseSB3aGVuIHlvdSBk ZWNpZGUgdG8gY2xvc2UgdGhlIG5vbi1tYXNrYWJsZSBNU0kgaG9sZSBkZXNjcmliZWQKYWJvdmUs IHdoaWNoIEkgcmVjb21tZW5kIHRvIGRvIDopCgpZb3Ugc3RpbGwgY2FuIHV0aWxpemUgbXNpLWxp YiBhbmQganVzdCBoYXZlIHlvdXIgcHJpdmF0ZSBpbXBsZW1lbnRhdGlvbgpvZiBpbml0X2Rldl9t c2lfaW5mbygpIGFzIGEgd3JhcHBlciBhcm91bmQgdGhlIGxpYnJhcnkgc2ltaWxhciB0bwptYmlf aW5pdF9kZXZfbXNpX2luZm8oKSBhbmQgaXRzX2luaXRfZGV2X21zaV9pbmZvKCkuIFRoYXQgd3Jh cHBlciB3b3VsZApqdXN0IGhhbmRsZSB0aGUgbm9uLXJlbWFwIGNhc2UgdG8gc2V0IGluZm86OmNo aXA6aXJxX3NldF9hZmZpbml0eSB0byB0aGUKbWFnaWMgbm9uLXJlbWFwIGZ1bmN0aW9uLgoKSG9w ZSB0aGF0IGhlbHBzLgoKPiBJIGJlbGlldmUgaW4gdGhlIGZ1dHVyZSBSSVNDLVYgQUlBIHYyLjAg KHdoZW5ldmVyIHRoYXQKPiBoYXBwZW5zKSB3aWxsIGFkZHJlc3MgdGhlIGdhcHMgaW4gQUlBIHYx LjAgKGxpa2UgdGhpcyBvbmUpLgoKSWYgdGhhdCBpcyBzdHJpY3RseSB0cmFuc2xhdGlvbiB0YWJs ZSBiYXNlZCwgeWVzLgoKVGhhbmtzLAoKICAgICAgICB0Z2x4CgotLSAKa3ZtLXJpc2N2IG1haWxp bmcgbGlzdAprdm0tcmlzY3ZAbGlzdHMuaW5mcmFkZWFkLm9yZwpodHRwOi8vbGlzdHMuaW5mcmFk ZWFkLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2t2bS1yaXNjdgo= From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EB623746E; Wed, 4 Dec 2024 13:05:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733317533; cv=none; b=CybCbXFp43ptfG/5Zsb3ulyqlRd9zxIb3y/javqtB5u9VwsMDCWE0bHUBMADRBq9eUZ3SNjwdO+NBIO1BktYZ9YPYFwN+EmG1VXIiFMpnyuPSKRPJk64u2HMaCcKVpU34NJZf2IPFLlUJPsGDNDWE5c0jpLWN3Yvxp/Idons5UQ= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733317533; c=relaxed/simple; bh=XVtkq5iJ4Mj7hRv7CqYFdnVlFpaRegC0V8RTwadfHQk=; h=From:To:Cc:Subject:In-Reply-To:References:Date:Message-ID: MIME-Version:Content-Type; b=JX81zvCkiLjglNgOOAL3myYxjYxbujl6bgVlU6ykQU4b4Rlw6J+aSXV54MCb8c0+ru18wWqKuVmXyHP23H3IulaeOBKDDI5P9pRm+2B6jraxANbAJGF6CvxewrTpIpf1ty2+7JC+gLVl/dVVuULkuFCMDz8NMqCdspzlD6PxFng= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=wL73UjeX; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=qPufNg0a; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="wL73UjeX"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="qPufNg0a" From: Thomas Gleixner DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1733317524; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ROH6zlh02KHc87r45BDTv/cN5afbSIuh9toQ2ctg4yk=; b=wL73UjeXWyYQQmB1Pp1remQXdjAmCrtcWFIChE7E9HHlqL9M/l/akN/1PUyWWcio+M4hsd d7PhJzhCnsWkx4s2pdhDctBT82t3yAu7JL5tjJcYh5qKSbFl1hdbxAKNn5Bt1Rnq8D3xyh 1W12Torjr4e84kQ605+X6R3Xvphy304E3U+iWFWCCSOPeoenFGFisFQOMcWHZkqvkIm/Yo rqrivG3Lb9g3nGu76ZInJIUVeq0VCHCnHokMrMp17Gv781GoLj6bgF39qb7BhDi4tsMRg6 qBrRpQNn0HXqJMPWSPvD3rrF6D76de00gbF5IouDkUgEw4GC/F5dlO0bLlErmA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1733317524; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ROH6zlh02KHc87r45BDTv/cN5afbSIuh9toQ2ctg4yk=; b=qPufNg0aO0AVR52BVTeAWUHTNUKlqiQiXfPzDa82iuoktna7Zd4GaEhPSAChG0cycCtYIt oiuHXrVUC/U2oQBQ== To: Anup Patel Cc: Andrew Jones , iommu@lists.linux.dev, kvm-riscv@lists.infradead.org, kvm@vger.kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, tjeznach@rivosinc.com, zong.li@sifive.com, joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, atishp@atishpatra.org, alex.williamson@redhat.com, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu Subject: Re: [RFC PATCH 01/15] irqchip/riscv-imsic: Use hierarchy to reach irq_set_affinity In-Reply-To: References: <20241114161845.502027-17-ajones@ventanamicro.com> <20241114161845.502027-18-ajones@ventanamicro.com> <87mshcub2u.ffs@tglx> <874j3ktrjv.ffs@tglx> <87ser4s796.ffs@tglx> Date: Wed, 04 Dec 2024 14:05:23 +0100 Message-ID: <87mshbsing.ffs@tglx> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On Wed, Dec 04 2024 at 09:13, Anup Patel wrote: > On Wed, Dec 4, 2024 at 4:29=E2=80=AFAM Thomas Gleixner wrote: >> As I was looking at something else MSI related I had a look at >> imsic_irq_set_affinity() again. >> >> It's actually required to have the message write in that function and >> not afterwards as you invoke imsic_vector_move() from that function. >> >> That's obviously not true for the remap case as that will not change the >> message address/data pair because the remap table entry is immutable - >> at least I assume so for my mental sanity sake :) >> >> But that brings me to a related question. How is this supposed to work >> with non-atomic message updates? PCI/MSI does not necessarily provide >> masking, and the write of the address/data pair is done in bits and >> pieces. So you can end up with an intermediate state seen by the device >> which ends up somewhere in interrupt nirvana space. >> >> See the dance in msi_set_affinity() and commit 6f1a4891a592 >> ("x86/apic/msi: Plug non-maskable MSI affinity race") for further >> explanation. >> >> The way how the IMSIC driver works seems to be pretty much the same as >> the x86 APIC mess: >> >> @address is the physical address of the per CPU MSI target >> address and @data is the vector ID on that CPU. >> >> So the non-atomic update in case of non-maskable MSI suffers from the >> same problem. It works most of the time, but if it doesn't you might >> stare at the occasionally lost interrupt and the stale device in >> disbelief for quite a while :) > > Yes, we have the same challenges as x86 APIC when changing > MSI affinity. >> >> I might be missing something which magically prevent that though :) >> > Your understanding is correct. In fact, the IMSIC msi_set_affinity() > handling is inspired from x86 APIC approach due to similarity in > the overall MSI controller. > > The high-level idea of imsic_irq_set_affinity() is as follows: > > 1) Allocate new_vector (new CPU IMSIC address + new ID on that CPU) > > 2) Update the MSI address and data programmed in the device > based on new_vector (see imsic_msi_update_msg()) > > 3) At this point the device points to the new_vector but old_vector > (old CPU IMSIC address + old ID on that CPU) is still enabled and > we might have received MSI on old_vector while we were busy > setting up a new_vector for the device. To address this, we call > imsic_vector_move(). > > 4) The imsic_vector_move() marks the old_vector as being > moved and schedules a lazy timer on the old CPU. > > 5) The lazy timer expires on the old CPU and results in > __imsic_local_sync() being called on the old CPU. > > 6) If there was a pending MSI on the old vector then the > __imsic_local_sync() function injects an MSI to the > new_vector using an MMIO write. > > It is very unlikely that an MSI from device will be dropped > (unless I am missing something) but the unsolved issue > is that handling of in-flight MSI received on the old_vector > during the MSI re-programming is delayed which may have > side effects on the device driver side. Interrupt delivery can be delayed for tons of other reasons. But yes, you are missing something which is worse than a jiffie delay: CPU Device msi_update_msg() compose() write() write(msg->address_lo); [1] write(msg->address_hi); [2] Raises interrupt [3] write(msg->data); [4] [2] can be ignored as it should not change (otherwise 32bit only devices would not work). Lets assume that the original message was: addr_lo =3D 0x1000, data =3D 0x10 (CPU1, vector 0x10) =20=20=20=20=20=20=20 The new message is addr_lo =3D 0x2000, data =3D 0x20 (CPU2, vector 0x20) After [2] the device sees: addr_lo =3D 0x2000, data =3D 0x10 (CPU2, vector 0x10) The interrupt raised in [3] will end up on CPU2 at vector 0x10 which might be not in use or used by some other device. In any case the interrupt is not reaching the real device handler and you can't see that interrupt as pending in CPU1. That's why x86 in case of non-remapped interrupts has this for the two situations: 1) old->data =3D=3D new->data write_msg(new);=20 The next interrupt either arrives on the old CPU or on the new CPU depending on timing. There is no intermediate state because the vector (data) is the same on both CPUs. 2) old->data !=3D new->data tmp_msg.addr =3D old_msg.addr tmp_msg.data =3D new_msg.data write_msg(tmp_msg); So after that write the device might raise the interrupt on CPU1 and vector 0x20. The next step is write_msg(new);=20 which changes the destination CPU to CPU2. So depending what the device has observed the interrupt might end up on CPU1 vector 0x10 (old) CPU1 vector 0x20 (tmp) CPU2 vector 0x20 (new) CPU1 vector 0x20 (tmp) is then checked in the pending register and the interrupt is retriggered if pending. That requires to move the interrupt from actual interrupt context on the old target CPU. It allows to evaluate the old target CPUs pending register with local interrupts disabled, which obviously does not work remote. I don't see a way how that can work remote with the IMSIC either even if you can easily access the pending state of the remote CPU: CPU0 CPU1 Device set_affinity() write_msg(tmp) write(addr); // CPU1 write(data); // vector 0x20 raise IRQ handle vector 0x20 (spurious or other device) check_pending(CPU1, 0x20) =3D=3D false -> Interrupt is lost Remapped interrupts do not have that issue because the table update is atomic vs. a concurrently raised interrupt, so that it either ends up on the old or on the new destination. The device message does not change as it always targets the table entry, which is immutable after setup. That's why x86 has this special msi affinity setter for the non remapped case. For the remapped case it's just using the hierarchy default which ends up at the remap domain. So the hierarchies look like this: 1) non-remap PCI/MSI device domain irq_chip.irq_set_affinity =3D msi_set_affinity; VECTOR domain 2) remap PCI/MSI device domain irq_chip.irq_set_affinity =3D msi_domain_set_affinity; REMAP domain irq_chip.irq_set_affinity =3D remap_set_affinity; VECTOR domain The special case of msi_set_affinity() is not involved in the remap case and solely utilized for the non-remap horrors. The remap case does not require the interrupt to be moved in interrupt context either because there is no intermediate step and pending register check on the old CPU involved. The vector cleanup of the old CPU always happens when the interrupt arrives on the new target for the first time independent of remapping mode. That's required for both cases to take care of the case where the interrupt is raised on the old vector (cpu1, 0x10) before the write happens and is pending in CPU1. So after desc::lock is released and interrupts are reenabled the interrupt is handled on CPU1. The next one is guaranteed to arrive on CPU2 and that triggers the clean up of the CPU1 vector, which releases it for reuse in the matrix allocator. I think you should model it in a similar way instead of trying to artificially reuse imsic_irq_set_affinity() for the remap case, especially when you decide to close the non-maskable MSI hole described above, which I recommend to do :) You still can utilize msi-lib and just have your private implementation of init_dev_msi_info() as a wrapper around the library similar to mbi_init_dev_msi_info() and its_init_dev_msi_info(). That wrapper would just handle the non-remap case to set info::chip:irq_set_affinity to the magic non-remap function. Hope that helps. > I believe in the future RISC-V AIA v2.0 (whenever that > happens) will address the gaps in AIA v1.0 (like this one). If that is strictly translation table based, yes. Thanks, tglx From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2C6BAE7716E for ; Wed, 4 Dec 2024 14:14:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:References :In-Reply-To:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=uU3rAX6I7iRe6My+ofAQsaA9bmOJfvDXcXT/0K9hwI4=; b=QoyfS/UUDV8vQ7 UZymIM4Pue/MeE7Elv/LnP4rHjhHuaiLkExSBIrxL+1/pjLR2OZxhB1D1pbYKoo1Xwj2sNnOaJc+W XDgAfaebJHzAsczA8zVEERTDEHEqdRJFtnZoxukn+5eqDJ0L8IoRg0eZHltTORxrUKEdifOGyZPsa KZ8IM+8eroShtHwouJrC0CbYqelv/p71HifsH+ThPPVJ5fNDNJ8Ulo9pOfxkfK66i37YV/RkNSE0/ 6okw0Wq1WvCknK3spBlAYKSYOYZ4fk2L/4OCq+MLZqtMAhnpdgEfLNsZQOjEXorSpVmPHoOldhy75 yY0zhBGP/Z2mCq7bXRsA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tIq8f-0000000CnEI-3Goh; Wed, 04 Dec 2024 14:14:01 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tIp4M-0000000CdXu-2ESW; Wed, 04 Dec 2024 13:05:30 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:Content-Type :MIME-Version:Message-ID:Date:References:In-Reply-To:Subject:Cc:To:From: Sender:Reply-To:Content-ID:Content-Description; bh=ROH6zlh02KHc87r45BDTv/cN5afbSIuh9toQ2ctg4yk=; b=lufgGgKCwWsjX/1baicZDVMhDV ZfN/kdGAuI1+JH/hUGSmeAzvvg/Vg/hDEAwBQOyv/jQK1wYym7Nz0TXWtA3TT5cdTgadbuQr4RaxG SatZKKyrhxl+lt3lhl6oY8wHgC4hZFE44bTrN3juZGz/7GPQ744jYukuibGcAe8mLbRRK9tdrAA4L oCtuSSGFMqdOryRWVgPYZ9A2nzDjvuFlm4ixaZVXYKbC8nNY+/NMHFdmU1n1bE2paoNpnP1l8/ysH CeJuaBkh7ic5MCxqBVyYzNnxTiDODnBIex+CcoPY3MXXMeDZ3oONOAmVhEkaUkoGMmt22YgqKWlHN ScFEzePg==; Received: from galois.linutronix.de ([193.142.43.55]) by desiato.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tIp4J-00000002baM-2Jpv; Wed, 04 Dec 2024 13:05:29 +0000 From: Thomas Gleixner DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1733317524; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ROH6zlh02KHc87r45BDTv/cN5afbSIuh9toQ2ctg4yk=; b=wL73UjeXWyYQQmB1Pp1remQXdjAmCrtcWFIChE7E9HHlqL9M/l/akN/1PUyWWcio+M4hsd d7PhJzhCnsWkx4s2pdhDctBT82t3yAu7JL5tjJcYh5qKSbFl1hdbxAKNn5Bt1Rnq8D3xyh 1W12Torjr4e84kQ605+X6R3Xvphy304E3U+iWFWCCSOPeoenFGFisFQOMcWHZkqvkIm/Yo rqrivG3Lb9g3nGu76ZInJIUVeq0VCHCnHokMrMp17Gv781GoLj6bgF39qb7BhDi4tsMRg6 qBrRpQNn0HXqJMPWSPvD3rrF6D76de00gbF5IouDkUgEw4GC/F5dlO0bLlErmA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1733317524; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ROH6zlh02KHc87r45BDTv/cN5afbSIuh9toQ2ctg4yk=; b=qPufNg0aO0AVR52BVTeAWUHTNUKlqiQiXfPzDa82iuoktna7Zd4GaEhPSAChG0cycCtYIt oiuHXrVUC/U2oQBQ== To: Anup Patel Cc: Andrew Jones , iommu@lists.linux.dev, kvm-riscv@lists.infradead.org, kvm@vger.kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, tjeznach@rivosinc.com, zong.li@sifive.com, joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, atishp@atishpatra.org, alex.williamson@redhat.com, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu Subject: Re: [RFC PATCH 01/15] irqchip/riscv-imsic: Use hierarchy to reach irq_set_affinity In-Reply-To: References: <20241114161845.502027-17-ajones@ventanamicro.com> <20241114161845.502027-18-ajones@ventanamicro.com> <87mshcub2u.ffs@tglx> <874j3ktrjv.ffs@tglx> <87ser4s796.ffs@tglx> Date: Wed, 04 Dec 2024 14:05:23 +0100 Message-ID: <87mshbsing.ffs@tglx> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241204_130527_862699_7059D205 X-CRM114-Status: GOOD ( 47.32 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org T24gV2VkLCBEZWMgMDQgMjAyNCBhdCAwOToxMywgQW51cCBQYXRlbCB3cm90ZToKPiBPbiBXZWQs IERlYyA0LCAyMDI0IGF0IDQ6MjnigK9BTSBUaG9tYXMgR2xlaXhuZXIgPHRnbHhAbGludXRyb25p eC5kZT4gd3JvdGU6Cj4+IEFzIEkgd2FzIGxvb2tpbmcgYXQgc29tZXRoaW5nIGVsc2UgTVNJIHJl bGF0ZWQgSSBoYWQgYSBsb29rIGF0Cj4+IGltc2ljX2lycV9zZXRfYWZmaW5pdHkoKSBhZ2Fpbi4K Pj4KPj4gSXQncyBhY3R1YWxseSByZXF1aXJlZCB0byBoYXZlIHRoZSBtZXNzYWdlIHdyaXRlIGlu IHRoYXQgZnVuY3Rpb24gYW5kCj4+IG5vdCBhZnRlcndhcmRzIGFzIHlvdSBpbnZva2UgaW1zaWNf dmVjdG9yX21vdmUoKSBmcm9tIHRoYXQgZnVuY3Rpb24uCj4+Cj4+IFRoYXQncyBvYnZpb3VzbHkg bm90IHRydWUgZm9yIHRoZSByZW1hcCBjYXNlIGFzIHRoYXQgd2lsbCBub3QgY2hhbmdlIHRoZQo+ PiBtZXNzYWdlIGFkZHJlc3MvZGF0YSBwYWlyIGJlY2F1c2UgdGhlIHJlbWFwIHRhYmxlIGVudHJ5 IGlzIGltbXV0YWJsZSAtCj4+IGF0IGxlYXN0IEkgYXNzdW1lIHNvIGZvciBteSBtZW50YWwgc2Fu aXR5IHNha2UgOikKPj4KPj4gQnV0IHRoYXQgYnJpbmdzIG1lIHRvIGEgcmVsYXRlZCBxdWVzdGlv bi4gSG93IGlzIHRoaXMgc3VwcG9zZWQgdG8gd29yawo+PiB3aXRoIG5vbi1hdG9taWMgbWVzc2Fn ZSB1cGRhdGVzPyBQQ0kvTVNJIGRvZXMgbm90IG5lY2Vzc2FyaWx5IHByb3ZpZGUKPj4gbWFza2lu ZywgYW5kIHRoZSB3cml0ZSBvZiB0aGUgYWRkcmVzcy9kYXRhIHBhaXIgaXMgZG9uZSBpbiBiaXRz IGFuZAo+PiBwaWVjZXMuIFNvIHlvdSBjYW4gZW5kIHVwIHdpdGggYW4gaW50ZXJtZWRpYXRlIHN0 YXRlIHNlZW4gYnkgdGhlIGRldmljZQo+PiB3aGljaCBlbmRzIHVwIHNvbWV3aGVyZSBpbiBpbnRl cnJ1cHQgbmlydmFuYSBzcGFjZS4KPj4KPj4gU2VlIHRoZSBkYW5jZSBpbiBtc2lfc2V0X2FmZmlu aXR5KCkgYW5kIGNvbW1pdCA2ZjFhNDg5MWE1OTIKPj4gKCJ4ODYvYXBpYy9tc2k6IFBsdWcgbm9u LW1hc2thYmxlIE1TSSBhZmZpbml0eSByYWNlIikgZm9yIGZ1cnRoZXIKPj4gZXhwbGFuYXRpb24u Cj4+Cj4+IFRoZSB3YXkgaG93IHRoZSBJTVNJQyBkcml2ZXIgd29ya3Mgc2VlbXMgdG8gYmUgcHJl dHR5IG11Y2ggdGhlIHNhbWUgYXMKPj4gdGhlIHg4NiBBUElDIG1lc3M6Cj4+Cj4+ICAgICAgICAg QGFkZHJlc3MgaXMgdGhlIHBoeXNpY2FsIGFkZHJlc3Mgb2YgdGhlIHBlciBDUFUgTVNJIHRhcmdl dAo+PiAgICAgICAgIGFkZHJlc3MgYW5kIEBkYXRhIGlzIHRoZSB2ZWN0b3IgSUQgb24gdGhhdCBD UFUuCj4+Cj4+IFNvIHRoZSBub24tYXRvbWljIHVwZGF0ZSBpbiBjYXNlIG9mIG5vbi1tYXNrYWJs ZSBNU0kgc3VmZmVycyBmcm9tIHRoZQo+PiBzYW1lIHByb2JsZW0uIEl0IHdvcmtzIG1vc3Qgb2Yg dGhlIHRpbWUsIGJ1dCBpZiBpdCBkb2Vzbid0IHlvdSBtaWdodAo+PiBzdGFyZSBhdCB0aGUgb2Nj YXNpb25hbGx5IGxvc3QgaW50ZXJydXB0IGFuZCB0aGUgc3RhbGUgZGV2aWNlIGluCj4+IGRpc2Jl bGllZiBmb3IgcXVpdGUgYSB3aGlsZSA6KQo+Cj4gWWVzLCB3ZSBoYXZlIHRoZSBzYW1lIGNoYWxs ZW5nZXMgYXMgeDg2IEFQSUMgd2hlbiBjaGFuZ2luZwo+IE1TSSBhZmZpbml0eS4KPj4KPj4gSSBt aWdodCBiZSBtaXNzaW5nIHNvbWV0aGluZyB3aGljaCBtYWdpY2FsbHkgcHJldmVudCB0aGF0IHRo b3VnaCA6KQo+Pgo+IFlvdXIgdW5kZXJzdGFuZGluZyBpcyBjb3JyZWN0LiBJbiBmYWN0LCB0aGUg SU1TSUMgbXNpX3NldF9hZmZpbml0eSgpCj4gaGFuZGxpbmcgaXMgaW5zcGlyZWQgZnJvbSB4ODYg QVBJQyBhcHByb2FjaCBkdWUgdG8gc2ltaWxhcml0eSBpbgo+IHRoZSBvdmVyYWxsIE1TSSBjb250 cm9sbGVyLgo+Cj4gVGhlIGhpZ2gtbGV2ZWwgaWRlYSBvZiBpbXNpY19pcnFfc2V0X2FmZmluaXR5 KCkgaXMgYXMgZm9sbG93czoKPgo+IDEpIEFsbG9jYXRlIG5ld192ZWN0b3IgKG5ldyBDUFUgSU1T SUMgYWRkcmVzcyArIG5ldyBJRCBvbiB0aGF0IENQVSkKPgo+IDIpIFVwZGF0ZSB0aGUgTVNJIGFk ZHJlc3MgYW5kIGRhdGEgcHJvZ3JhbW1lZCBpbiB0aGUgZGV2aWNlCj4gYmFzZWQgb24gbmV3X3Zl Y3RvciAoc2VlIGltc2ljX21zaV91cGRhdGVfbXNnKCkpCj4KPiAzKSBBdCB0aGlzIHBvaW50IHRo ZSBkZXZpY2UgcG9pbnRzIHRvIHRoZSBuZXdfdmVjdG9yIGJ1dCBvbGRfdmVjdG9yCj4gKG9sZCBD UFUgSU1TSUMgYWRkcmVzcyArIG9sZCBJRCBvbiB0aGF0IENQVSkgaXMgc3RpbGwgZW5hYmxlZCBh bmQKPiB3ZSBtaWdodCBoYXZlIHJlY2VpdmVkIE1TSSBvbiBvbGRfdmVjdG9yIHdoaWxlIHdlIHdl cmUgYnVzeQo+IHNldHRpbmcgdXAgYSBuZXdfdmVjdG9yIGZvciB0aGUgZGV2aWNlLiBUbyBhZGRy ZXNzIHRoaXMsIHdlIGNhbGwKPiBpbXNpY192ZWN0b3JfbW92ZSgpLgo+Cj4gNCkgVGhlIGltc2lj X3ZlY3Rvcl9tb3ZlKCkgbWFya3MgdGhlIG9sZF92ZWN0b3IgYXMgYmVpbmcKPiBtb3ZlZCBhbmQg c2NoZWR1bGVzIGEgbGF6eSB0aW1lciBvbiB0aGUgb2xkIENQVS4KPgo+IDUpIFRoZSBsYXp5IHRp bWVyIGV4cGlyZXMgb24gdGhlIG9sZCBDUFUgYW5kIHJlc3VsdHMgaW4KPiBfX2ltc2ljX2xvY2Fs X3N5bmMoKSBiZWluZyBjYWxsZWQgb24gdGhlIG9sZCBDUFUuCj4KPiA2KSBJZiB0aGVyZSB3YXMg YSBwZW5kaW5nIE1TSSBvbiB0aGUgb2xkIHZlY3RvciB0aGVuIHRoZQo+IF9faW1zaWNfbG9jYWxf c3luYygpIGZ1bmN0aW9uIGluamVjdHMgYW4gTVNJIHRvIHRoZQo+IG5ld192ZWN0b3IgdXNpbmcg YW4gTU1JTyB3cml0ZS4KPgo+IEl0IGlzIHZlcnkgdW5saWtlbHkgdGhhdCBhbiBNU0kgZnJvbSBk ZXZpY2Ugd2lsbCBiZSBkcm9wcGVkCj4gKHVubGVzcyBJIGFtIG1pc3Npbmcgc29tZXRoaW5nKSBi dXQgdGhlIHVuc29sdmVkIGlzc3VlCj4gaXMgdGhhdCBoYW5kbGluZyBvZiBpbi1mbGlnaHQgTVNJ IHJlY2VpdmVkIG9uIHRoZSBvbGRfdmVjdG9yCj4gZHVyaW5nIHRoZSBNU0kgcmUtcHJvZ3JhbW1p bmcgaXMgZGVsYXllZCB3aGljaCBtYXkgaGF2ZQo+IHNpZGUgZWZmZWN0cyBvbiB0aGUgZGV2aWNl IGRyaXZlciBzaWRlLgoKSW50ZXJydXB0IGRlbGl2ZXJ5IGNhbiBiZSBkZWxheWVkIGZvciB0b25z IG9mIG90aGVyIHJlYXNvbnMuCgpCdXQgeWVzLCB5b3UgYXJlIG1pc3Npbmcgc29tZXRoaW5nIHdo aWNoIGlzIHdvcnNlIHRoYW4gYSBqaWZmaWUgZGVsYXk6CgpDUFUgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgRGV2aWNlCiAgbXNpX3VwZGF0ZV9tc2coKQogICAgIGNvbXBvc2Uo KQogICAgIHdyaXRlKCkKICAgICAgIHdyaXRlKG1zZy0+YWRkcmVzc19sbyk7IFsxXQogICAgICAg d3JpdGUobXNnLT5hZGRyZXNzX2hpKTsgWzJdCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICBSYWlzZXMgaW50ZXJydXB0IFszXQogICAgICAgd3JpdGUobXNnLT5kYXRhKTsg ICAgICAgWzRdCgpbMl0gY2FuIGJlIGlnbm9yZWQgYXMgaXQgc2hvdWxkIG5vdCBjaGFuZ2UgKG90 aGVyd2lzZSAzMmJpdCBvbmx5IGRldmljZXMKd291bGQgbm90IHdvcmspLgoKTGV0cyBhc3N1bWUg dGhhdCB0aGUgb3JpZ2luYWwgbWVzc2FnZSB3YXM6CgogICAgIGFkZHJfbG8gPSAweDEwMDAsIGRh dGEgPSAweDEwICAgIChDUFUxLCB2ZWN0b3IgMHgxMCkKICAgICAgIApUaGUgbmV3IG1lc3NhZ2Ug aXMKCiAgICAgYWRkcl9sbyA9IDB4MjAwMCwgZGF0YSA9IDB4MjAgICAgKENQVTIsIHZlY3RvciAw eDIwKQoKQWZ0ZXIgWzJdIHRoZSBkZXZpY2Ugc2VlczoKCiAgICAgYWRkcl9sbyA9IDB4MjAwMCwg ZGF0YSA9IDB4MTAgICAgKENQVTIsIHZlY3RvciAweDEwKQoKVGhlIGludGVycnVwdCByYWlzZWQg aW4gWzNdIHdpbGwgZW5kIHVwIG9uIENQVTIgYXQgdmVjdG9yIDB4MTAgd2hpY2gKbWlnaHQgYmUg bm90IGluIHVzZSBvciB1c2VkIGJ5IHNvbWUgb3RoZXIgZGV2aWNlLiBJbiBhbnkgY2FzZSB0aGUK aW50ZXJydXB0IGlzIG5vdCByZWFjaGluZyB0aGUgcmVhbCBkZXZpY2UgaGFuZGxlciBhbmQgeW91 IGNhbid0IHNlZSB0aGF0CmludGVycnVwdCBhcyBwZW5kaW5nIGluIENQVTEuCgpUaGF0J3Mgd2h5 IHg4NiBpbiBjYXNlIG9mIG5vbi1yZW1hcHBlZCBpbnRlcnJ1cHRzIGhhcyB0aGlzIGZvciB0aGUg dHdvCnNpdHVhdGlvbnM6CgogIDEpIG9sZC0+ZGF0YSA9PSBuZXctPmRhdGEKCiAgICAgd3JpdGVf bXNnKG5ldyk7IAoKICAgICBUaGUgbmV4dCBpbnRlcnJ1cHQgZWl0aGVyIGFycml2ZXMgb24gdGhl IG9sZCBDUFUgb3Igb24gdGhlIG5ldyBDUFUKICAgICBkZXBlbmRpbmcgb24gdGltaW5nLiBUaGVy ZSBpcyBubyBpbnRlcm1lZGlhdGUgc3RhdGUgYmVjYXVzZSB0aGUKICAgICB2ZWN0b3IgKGRhdGEp IGlzIHRoZSBzYW1lIG9uIGJvdGggQ1BVcy4KCiAgMikgb2xkLT5kYXRhICE9IG5ldy0+ZGF0YQoK ICAgICB0bXBfbXNnLmFkZHIgPSBvbGRfbXNnLmFkZHIKICAgICB0bXBfbXNnLmRhdGEgPSBuZXdf bXNnLmRhdGEKCiAgICAgd3JpdGVfbXNnKHRtcF9tc2cpOwoKICAgICBTbyBhZnRlciB0aGF0IHdy aXRlIHRoZSBkZXZpY2UgbWlnaHQgcmFpc2UgdGhlIGludGVycnVwdCBvbiBDUFUxCiAgICAgYW5k IHZlY3RvciAweDIwLgoKICAgICBUaGUgbmV4dCBzdGVwIGlzCgogICAgIHdyaXRlX21zZyhuZXcp OyAKCiAgICAgd2hpY2ggY2hhbmdlcyB0aGUgZGVzdGluYXRpb24gQ1BVIHRvIENQVTIuCgogICAg IFNvIGRlcGVuZGluZyB3aGF0IHRoZSBkZXZpY2UgaGFzIG9ic2VydmVkIHRoZSBpbnRlcnJ1cHQg bWlnaHQgZW5kCiAgICAgdXAgb24KCiAgICAgQ1BVMSB2ZWN0b3IgMHgxMCAgIChvbGQpCiAgICAg Q1BVMSB2ZWN0b3IgMHgyMCAgICh0bXApCiAgICAgQ1BVMiB2ZWN0b3IgMHgyMCAgIChuZXcpCgog ICAgIENQVTEgdmVjdG9yIDB4MjAgKHRtcCkgaXMgdGhlbiBjaGVja2VkIGluIHRoZSBwZW5kaW5n IHJlZ2lzdGVyIGFuZAogICAgIHRoZSBpbnRlcnJ1cHQgaXMgcmV0cmlnZ2VyZWQgaWYgcGVuZGlu Zy4KClRoYXQgcmVxdWlyZXMgdG8gbW92ZSB0aGUgaW50ZXJydXB0IGZyb20gYWN0dWFsIGludGVy cnVwdCBjb250ZXh0IG9uIHRoZQpvbGQgdGFyZ2V0IENQVS4gSXQgYWxsb3dzIHRvIGV2YWx1YXRl IHRoZSBvbGQgdGFyZ2V0IENQVXMgcGVuZGluZwpyZWdpc3RlciB3aXRoIGxvY2FsIGludGVycnVw dHMgZGlzYWJsZWQsIHdoaWNoIG9idmlvdXNseSBkb2VzIG5vdCB3b3JrCnJlbW90ZS4KCkkgZG9u J3Qgc2VlIGEgd2F5IGhvdyB0aGF0IGNhbiB3b3JrIHJlbW90ZSB3aXRoIHRoZSBJTVNJQyBlaXRo ZXIgZXZlbiBpZgp5b3UgY2FuIGVhc2lseSBhY2Nlc3MgdGhlIHBlbmRpbmcgc3RhdGUgb2YgdGhl IHJlbW90ZSBDUFU6CgpDUFUwICAgICAgICAgICAgICAgICAgICAgICAgICAgIENQVTEgICAgICAg ICAgICAgICAgICAgRGV2aWNlCnNldF9hZmZpbml0eSgpCiAgd3JpdGVfbXNnKHRtcCkKICAgIHdy aXRlKGFkZHIpOyAvLyBDUFUxCiAgICB3cml0ZShkYXRhKTsgLy8gdmVjdG9yIDB4MjAKICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICByYWlzZSBJ UlEKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBoYW5kbGUgdmVjdG9yIDB4MjAKICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAoc3B1cmlvdXMgb3Igb3RoZXIgZGV2aWNlKQoK ICAgIGNoZWNrX3BlbmRpbmcoQ1BVMSwgMHgyMCkgPT0gZmFsc2UgLT4gSW50ZXJydXB0IGlzIGxv c3QKClJlbWFwcGVkIGludGVycnVwdHMgZG8gbm90IGhhdmUgdGhhdCBpc3N1ZSBiZWNhdXNlIHRo ZSB0YWJsZSB1cGRhdGUgaXMKYXRvbWljIHZzLiBhIGNvbmN1cnJlbnRseSByYWlzZWQgaW50ZXJy dXB0LCBzbyB0aGF0IGl0IGVpdGhlciBlbmRzIHVwIG9uCnRoZSBvbGQgb3Igb24gdGhlIG5ldyBk ZXN0aW5hdGlvbi4gVGhlIGRldmljZSBtZXNzYWdlIGRvZXMgbm90IGNoYW5nZSBhcwppdCBhbHdh eXMgdGFyZ2V0cyB0aGUgdGFibGUgZW50cnksIHdoaWNoIGlzIGltbXV0YWJsZSBhZnRlciBzZXR1 cC4KClRoYXQncyB3aHkgeDg2IGhhcyB0aGlzIHNwZWNpYWwgbXNpIGFmZmluaXR5IHNldHRlciBm b3IgdGhlIG5vbiByZW1hcHBlZApjYXNlLiBGb3IgdGhlIHJlbWFwcGVkIGNhc2UgaXQncyBqdXN0 IHVzaW5nIHRoZSBoaWVyYXJjaHkgZGVmYXVsdCB3aGljaAplbmRzIHVwIGF0IHRoZSByZW1hcCBk b21haW4uCgpTbyB0aGUgaGllcmFyY2hpZXMgbG9vayBsaWtlIHRoaXM6CgogICAxKSBub24tcmVt YXAKCiAgICAgIFBDSS9NU0kgZGV2aWNlIGRvbWFpbgogICAgICAgICBpcnFfY2hpcC5pcnFfc2V0 X2FmZmluaXR5ID0gbXNpX3NldF9hZmZpbml0eTsKICAgICAgVkVDVE9SIGRvbWFpbgoKICAgMikg cmVtYXAKCiAgICAgIFBDSS9NU0kgZGV2aWNlIGRvbWFpbgogICAgICAgICBpcnFfY2hpcC5pcnFf c2V0X2FmZmluaXR5ID0gbXNpX2RvbWFpbl9zZXRfYWZmaW5pdHk7CiAgICAgIFJFTUFQIGRvbWFp bgogICAgICAgICBpcnFfY2hpcC5pcnFfc2V0X2FmZmluaXR5ID0gcmVtYXBfc2V0X2FmZmluaXR5 OwogICAgICBWRUNUT1IgZG9tYWluCgpUaGUgc3BlY2lhbCBjYXNlIG9mIG1zaV9zZXRfYWZmaW5p dHkoKSBpcyBub3QgaW52b2x2ZWQgaW4gdGhlIHJlbWFwIGNhc2UKYW5kIHNvbGVseSB1dGlsaXpl ZCBmb3IgdGhlIG5vbi1yZW1hcCBob3Jyb3JzLiBUaGUgcmVtYXAgY2FzZSBkb2VzIG5vdApyZXF1 aXJlIHRoZSBpbnRlcnJ1cHQgdG8gYmUgbW92ZWQgaW4gaW50ZXJydXB0IGNvbnRleHQgZWl0aGVy IGJlY2F1c2UKdGhlcmUgaXMgbm8gaW50ZXJtZWRpYXRlIHN0ZXAgYW5kIHBlbmRpbmcgcmVnaXN0 ZXIgY2hlY2sgb24gdGhlIG9sZCBDUFUKaW52b2x2ZWQuCgpUaGUgdmVjdG9yIGNsZWFudXAgb2Yg dGhlIG9sZCBDUFUgYWx3YXlzIGhhcHBlbnMgd2hlbiB0aGUgaW50ZXJydXB0CmFycml2ZXMgb24g dGhlIG5ldyB0YXJnZXQgZm9yIHRoZSBmaXJzdCB0aW1lIGluZGVwZW5kZW50IG9mIHJlbWFwcGlu Zwptb2RlLiBUaGF0J3MgcmVxdWlyZWQgZm9yIGJvdGggY2FzZXMgdG8gdGFrZSBjYXJlIG9mIHRo ZSBjYXNlIHdoZXJlIHRoZQppbnRlcnJ1cHQgaXMgcmFpc2VkIG9uIHRoZSBvbGQgdmVjdG9yIChj cHUxLCAweDEwKSBiZWZvcmUgdGhlIHdyaXRlCmhhcHBlbnMgYW5kIGlzIHBlbmRpbmcgaW4gQ1BV MS4gU28gYWZ0ZXIgZGVzYzo6bG9jayBpcyByZWxlYXNlZCBhbmQKaW50ZXJydXB0cyBhcmUgcmVl bmFibGVkIHRoZSBpbnRlcnJ1cHQgaXMgaGFuZGxlZCBvbiBDUFUxLiBUaGUgbmV4dCBvbmUKaXMg Z3VhcmFudGVlZCB0byBhcnJpdmUgb24gQ1BVMiBhbmQgdGhhdCB0cmlnZ2VycyB0aGUgY2xlYW4g dXAgb2YgdGhlCkNQVTEgdmVjdG9yLCB3aGljaCByZWxlYXNlcyBpdCBmb3IgcmV1c2UgaW4gdGhl IG1hdHJpeCBhbGxvY2F0b3IuCgpJIHRoaW5rIHlvdSBzaG91bGQgbW9kZWwgaXQgaW4gYSBzaW1p bGFyIHdheSBpbnN0ZWFkIG9mIHRyeWluZyB0bwphcnRpZmljaWFsbHkgcmV1c2UgaW1zaWNfaXJx X3NldF9hZmZpbml0eSgpIGZvciB0aGUgcmVtYXAgY2FzZSwKZXNwZWNpYWxseSB3aGVuIHlvdSBk ZWNpZGUgdG8gY2xvc2UgdGhlIG5vbi1tYXNrYWJsZSBNU0kgaG9sZSBkZXNjcmliZWQKYWJvdmUs IHdoaWNoIEkgcmVjb21tZW5kIHRvIGRvIDopCgpZb3Ugc3RpbGwgY2FuIHV0aWxpemUgbXNpLWxp YiBhbmQganVzdCBoYXZlIHlvdXIgcHJpdmF0ZSBpbXBsZW1lbnRhdGlvbgpvZiBpbml0X2Rldl9t c2lfaW5mbygpIGFzIGEgd3JhcHBlciBhcm91bmQgdGhlIGxpYnJhcnkgc2ltaWxhciB0bwptYmlf aW5pdF9kZXZfbXNpX2luZm8oKSBhbmQgaXRzX2luaXRfZGV2X21zaV9pbmZvKCkuIFRoYXQgd3Jh cHBlciB3b3VsZApqdXN0IGhhbmRsZSB0aGUgbm9uLXJlbWFwIGNhc2UgdG8gc2V0IGluZm86OmNo aXA6aXJxX3NldF9hZmZpbml0eSB0byB0aGUKbWFnaWMgbm9uLXJlbWFwIGZ1bmN0aW9uLgoKSG9w ZSB0aGF0IGhlbHBzLgoKPiBJIGJlbGlldmUgaW4gdGhlIGZ1dHVyZSBSSVNDLVYgQUlBIHYyLjAg KHdoZW5ldmVyIHRoYXQKPiBoYXBwZW5zKSB3aWxsIGFkZHJlc3MgdGhlIGdhcHMgaW4gQUlBIHYx LjAgKGxpa2UgdGhpcyBvbmUpLgoKSWYgdGhhdCBpcyBzdHJpY3RseSB0cmFuc2xhdGlvbiB0YWJs ZSBiYXNlZCwgeWVzLgoKVGhhbmtzLAoKICAgICAgICB0Z2x4CgpfX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fXwpsaW51eC1yaXNjdiBtYWlsaW5nIGxpc3QKbGlu dXgtcmlzY3ZAbGlzdHMuaW5mcmFkZWFkLm9yZwpodHRwOi8vbGlzdHMuaW5mcmFkZWFkLm9yZy9t YWlsbWFuL2xpc3RpbmZvL2xpbnV4LXJpc2N2Cg==