From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrew Jones Date: Thu, 19 Jan 2023 09:29:03 +0100 Subject: [PATCH v3 05/13] riscv: cpufeature: extend riscv_cpufeature_patch_func to all ISA extensions In-Reply-To: References: <20230111171027.2392-1-jszhang@kernel.org> <20230111171027.2392-6-jszhang@kernel.org> <2398293.3Lj2Plt8kZ@diego> <20230112092136.f2g43hrhmrqouy4y@orel> Message-ID: <20230119082903.yk3uslfrjtxzassi@orel> List-Id: To: kvm-riscv@lists.infradead.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit On Wed, Jan 18, 2023 at 09:54:46PM +0000, Conor Dooley wrote: > Hey! > > I guess here is the right place to follow up on all of this stuff... > > On Sat, Jan 14, 2023 at 08:32:37PM +0000, Conor Dooley wrote: > > On Fri, Jan 13, 2023 at 03:18:59PM +0000, Conor Dooley wrote: > > > On Thu, Jan 12, 2023 at 10:21:36AM +0100, Andrew Jones wrote: > > > > On Thu, Jan 12, 2023 at 12:29:57AM +0100, Heiko St?bner wrote: > > > > > Am Mittwoch, 11. Januar 2023, 18:10:19 CET schrieb Jisheng Zhang: > > > > > > riscv_cpufeature_patch_func() currently only scans a limited set of > > > > > > cpufeatures, explicitly defined with macros. Extend it to probe for all > > > > > > ISA extensions. > > > > > > > > > > > > Signed-off-by: Jisheng Zhang > > > > > > Reviewed-by: Andrew Jones > > > > > > Reviewed-by: Heiko Stuebner > > > > > > --- > > > > > > arch/riscv/include/asm/errata_list.h | 9 ++-- > > > > > > arch/riscv/kernel/cpufeature.c | 63 ++++------------------------ > > > > > > 2 files changed, 11 insertions(+), 61 deletions(-) > > > > > > > > > > hmmm ... I do see a somewhat big caveat for this. > > > > > and would like to take back my Reviewed-by for now > > > > > > > > > > > > > > > With this change we would limit the patchable cpufeatures to actual > > > > > riscv extensions. But cpufeatures can also be soft features like > > > > > how performant the core handles unaligned accesses. > > > > > > > > I agree that this needs to be addressed and Jisheng also raised this > > > > yesterday here [*]. It seems we need the concept of cpufeatures, which > > > > may be extensions or non-extensions. > > > > > > > > [*] https://lore.kernel.org/all/Y77xyNPNqnFQUqAx at xhacker/ > > > > > > > > > See Palmer's series [0]. > > > > > > > > > > > > > > > Also this essentially codifies that each ALTERNATIVE can only ever > > > > > be attached to exactly one extension. > > > > > > > > > > But contrary to vendor-errata, it is very likely that we will need > > > > > combinations of different extensions for some alternatives in the future. > > > > > > > > One possible approach may be to combine extensions/non-extensions at boot > > > > time into pseudo-cpufeatures. Then, alternatives can continue attaching to > > > > a single "feature". (I'm not saying that's a better approach than the > > > > bitmap, I'm just suggesting it as something else to consider.) > > > > > > > > > > > ALTERNATIVE_2("nop", > > > > > "j strcmp_zbb_unaligned", 0, CPUFEATURE_ZBB | CPUFEATURE_FAST_UNALIGNED, 0, CONFIG_RISCV_ISA_ZBB, > > > > > "j variant_zbb", 0, CPUFEATURE_ZBB, CPUFEATURE_FAST_UNALIGNED, CONFIG_RISCV_ISA_ZBB) > > > > > > > > > > [the additional field there models a "not" component] > > > > > > Since we're discussing theoretical implementations, and it's a little hard > > > to picture all that they entail in my head, I might be making a fool of > > > myself here with assumptions... > > > > > > Heiko's suggestion sounded along the lines of: keep probing individual > > > "features" as we are now. Features in this case being the presence of > > > the extension or non-extension capability. And then in the alternative, > > > make all of the decisions about which to apply. > > > > > > Drew's suggestion would have significantly more defined CPUFEATURE_FOOs, > > > but would offload the decision making about which extensions or non- > > > extension capabilities constitute a feature to regular old cpufeature > > > code. However, the order of precedence would remain in the alt macro, as > > > it does now. > > > > > > I think I am just a wee bit biased, but adding the complexity somewhere > > > other than alternative macros seems a wise choice, especially as we are > > > likely to find that complexity increases over time? > > > > > > The other thing that came to mind, and maybe this is just looking for > > > holes where they don't exist (or are not worth addressing), is that > > > order of precedence. > > > I can imagine that, in some cases, the order of precedence is not a > > > constant per psuedo-cpufeature, but determined by implementation of > > > the capabilities that comprise it? > > > > Having spent longer than I maybe should've looking at your patches > > Heiko, given it's a Saturday evening, the precedence stuff is still > > sticking out to me.. > > > > For Zbb & fast unaligned, the order may be non-controversial, but in > > the general case I don't see how it can be true that the order of > > precedence for variants is a constant. > > > > Creating pseudo cpufeatures as Drew suggested does seem like a way to > > extract complexity from the alternatives themselves (which I think is a > > good thing) but at the expense of eating up cpu_req_feature bits... > > By itself, it doesn't help with precedence, but it may better allow us > > to deal with some of the precedence in cpufeature.c, since the > > alternative would operate based on the pseudo cpufeature rather than on > > the individual capabilities that the pseudo cpufeature depends on. > > > > I tried to come up with a suggestion for what to do about precedence, > > but everything I thought up felt a bit horrific tbh. > > The thing that fits the current model best is just allowing cpu vendors > > to add, yet more, "errata" that pick the variant that works best for > > their implementation... Although I'd be worried about ballooning some of > > these ALT_FOO macros out to a massive degree with that sort of approach. > > > > > If my assumption/understanding holds, moving decision making out of the > > > alternative seems like it would better provision for scenarios like > > > that? I dunno, maybe that is whatever the corollary of "premature > > > optimisation" is for this discussion. > > > > > > That's my unsolicited ? 0.02, hopefully I wasn't off-base with the > > > assumptions I made. > > > > The order in which an alternative is added to the macro does matter, > > right? At least, that's how I thought it worked and hope I've not had > > an incorrect interpretation there all along... I wasn't until I started > > reading your patch and couldn't understand why you had a construct that > > looked like > > > > if (zbb && !fast_unaligned) > > ... > > else if (zbb && fast_unaligned) > > ... > > > > rather than just inverting the order and dropping the !fast_unaligned > > that I realised I might have a gap in my understanding after all.. > > > > > Heiko, I figure you've got some sort of WIP stuff for this anyway since > > > you're interested in the fast unaligned? How close are you to posting any > > > of that? > > > > > > While I think of it, w.r.t. extension versus (pseudo)cpufeature etc > > > naming, it may make sense to call the functions that this series adds > > > in patch 6 has_cpufeature_{un,}likely(), no matter what decision gets > > > made here? > > > IMO using cpufeature seems to make more sense for a general use API that > > > may be used later on for the likes of unaligned access, even if > > > initially it is not used for anything other than extensions. > > Today at [1] we talked a bit about the various bits going on here. > I'll attempt to summarise what I remember, but I meant to do this > several hours ago and am likely to make a hames of it. > > For Zbb/unaligned stuff, the sentiment was along the lines of there > needing to be a performance benefit to justify the inclusion. > Some of us have HW that is (allegedly) capable of Zbb, and, if that's the > case, will give it a go. > I think it was similar for unaligned, since there is nothing yet that > supports this behaviour, we should wait until a benefit is demonstrable. > > On the subject of grouping extension/non-extension capabilities into a > single cpufeature, Palmer mentioned that GCC does something similar, > for the likes of the Ventana vendor extensions, that are unlikely to be > present in isolation. > Those are (or were?) probed as a group of extensions rather than > individually. > I think it was said it'd make sense for us to unify extensions that will > only ever appear together single psuedo cpufeature too. > > For the bitfield approach versus creating pseudo cpufeatures discussion > & how to deal with that in alternatives etc, I'm a bit less sure what the > outcome was. > IIRC, nothing concrete was said about either approach, but maybe it was > implied that we should do as GCC does, only grouping things that won't > ever been seen apart. > Figuring that out seems to have been punted down the road, as the > inclusion of our only current example of this (Zbb + unaligned) is > dependant on hardware showing up that actually benefits from it. > > The plan then seemed to be press ahead with this series & test the > benefits of the Zbb str* functions in Zbb capable hardware before making > a decision there. > > Hopefully I wasn't too far off with that summary... This matches my recollection. Thanks for the summary, Conor. drew > > Thanks, > Conor. > > 1 - https://lore.kernel.org/linux-riscv/mhng-775d4068-6c1e-48a4-a1dc-b4a76ff26bb3 at palmer-ri-x1c9a/ From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BBA0CC004D4 for ; Thu, 19 Jan 2023 08:29:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=29GxmVvTqPXpJkbTxDSgPl/G+xw9MVrlCUZtXFWlNJs=; b=g7jIqvEEVyITfV M9IHUntcsoSay2U/PBM2PSc+H66y5cGBOA/hrAQeGXbf4j0gpZVudXNPcrMu5m1XJbSFJEmpTgIuK f3KtY7+S0DK7AklVeE/of+WecwL9+fDRcsRz4jUnKVnnCpOYQ8DCptuDcl66dUQAF6Ikv7ET0vffu DIf3RqMByQQhmDXkZvLG0+1rwpjcRKxicquqiecgW9NZG+whfZF6D4zNu2DaHQ2tXuGH601gouEfU TBWUC4j2tpg0qZrNUv8OrQo3jLRP4gnfAwAS7iX5U2d/XqHrh/tIjdPHpG0uLmI1yGPQJkEopkXbq mTEUEkwntgru5KhrKPmA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pIQIK-0046Lc-Pi; Thu, 19 Jan 2023 08:29:12 +0000 Received: from mail-ed1-x52c.google.com ([2a00:1450:4864:20::52c]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pIQIH-0046KL-3n for linux-riscv@lists.infradead.org; Thu, 19 Jan 2023 08:29:11 +0000 Received: by mail-ed1-x52c.google.com with SMTP id y19so1942922edc.2 for ; Thu, 19 Jan 2023 00:29:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ventanamicro.com; s=google; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=x5ck+uBW+we5mcml99JIlI5hbqOexLvffduO15vMLCQ=; b=JtUEKYA4ngIiCcO38C3pG5dWS8AmHuQYxbH7z4bS0gjTrWl2JjNzmgu+bHT1rhVvCF 1AyQtvmN+FtmrNaWPPlJxZ1bL2eW7Oditr9V7+Te0BCwylH/bu1UdZdtD74u9sKME4pq 4gLh+tfjkooAP7I3TeDdqGgUeUnakDAAJVoJn1lZJ0nt6ynYWYQ2hSzGBEW18rNJpTE7 X/bTKUyOfdx/C86S3LXuUQzM7SIH2k3OgDl+54FiyHI4lRaVEg82PMf5IzorscxABa4G 3rV6OMUb6N6Qq3ZkX/pp6a00uXYu/B094V6gztspTEq96rPZswMmsEwrXoIDCMXflQrs Yk6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=x5ck+uBW+we5mcml99JIlI5hbqOexLvffduO15vMLCQ=; b=tQWBEli+ElszXxK6c+VZfOgZme3MxHLYsvb26MiMfRqqBb236NHY2W/zam3V2P2hcy 69byGelr4fzCi/EMdoP9EAoX4z27eJHeXZi4t9Wff5NfmyHPN/QiOUvrAsTMXlqQFaLF 115QsRy6mdE3GF/NF70VIfcJ/Nacb1zZzPqkiFeXn/CVM7IxftrpPnAPOFvy5vqlZfcd 5y+iJOtYgoqhiK5G+fbVJVM5gxbJjfLFhIlYb607H+LPDYY3/+Uk2SZDtl9f0q9I+FpL tsyNKgAC5S5it+bQVO8+hu2kEUX79/DJhFDzNAlJEDLZKk4B1YpObjSSzNxPMa8R8gkA sreQ== X-Gm-Message-State: AFqh2kpZImbr8IY9XhOoQJXtp+iD0UtwJLE5mkPDEBmWc0m+0aWnm1qk mx8kmucqPbAkNlkVk+aw4g/H+g== X-Google-Smtp-Source: AMrXdXt37wqPfFZsW/BhY2ZBx33uOivJd/nYfcmjoJqMYW92j1bl/42mtZmEp65y22+4CKilHwwZSg== X-Received: by 2002:a05:6402:b:b0:496:f517:d30d with SMTP id d11-20020a056402000b00b00496f517d30dmr10408911edu.38.1674116945288; Thu, 19 Jan 2023 00:29:05 -0800 (PST) Received: from localhost (cst2-173-16.cust.vodafone.cz. [31.30.173.16]) by smtp.gmail.com with ESMTPSA id a3-20020aa7cf03000000b0049019b48373sm15294210edy.85.2023.01.19.00.29.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Jan 2023 00:29:04 -0800 (PST) Date: Thu, 19 Jan 2023 09:29:03 +0100 From: Andrew Jones To: Conor Dooley Cc: Heiko =?utf-8?Q?St=C3=BCbner?= , Palmer Dabbelt , Paul Walmsley , Albert Ou , Anup Patel , Atish Patra , Jisheng Zhang , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvm-riscv@lists.infradead.org Subject: Re: [PATCH v3 05/13] riscv: cpufeature: extend riscv_cpufeature_patch_func to all ISA extensions Message-ID: <20230119082903.yk3uslfrjtxzassi@orel> References: <20230111171027.2392-1-jszhang@kernel.org> <20230111171027.2392-6-jszhang@kernel.org> <2398293.3Lj2Plt8kZ@diego> <20230112092136.f2g43hrhmrqouy4y@orel> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230119_002909_184901_63D6C83A X-CRM114-Status: GOOD ( 74.16 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org T24gV2VkLCBKYW4gMTgsIDIwMjMgYXQgMDk6NTQ6NDZQTSArMDAwMCwgQ29ub3IgRG9vbGV5IHdy b3RlOgo+IEhleSEKPiAKPiBJIGd1ZXNzIGhlcmUgaXMgdGhlIHJpZ2h0IHBsYWNlIHRvIGZvbGxv dyB1cCBvbiBhbGwgb2YgdGhpcyBzdHVmZi4uLgo+IAo+IE9uIFNhdCwgSmFuIDE0LCAyMDIzIGF0 IDA4OjMyOjM3UE0gKzAwMDAsIENvbm9yIERvb2xleSB3cm90ZToKPiA+IE9uIEZyaSwgSmFuIDEz LCAyMDIzIGF0IDAzOjE4OjU5UE0gKzAwMDAsIENvbm9yIERvb2xleSB3cm90ZToKPiA+ID4gT24g VGh1LCBKYW4gMTIsIDIwMjMgYXQgMTA6MjE6MzZBTSArMDEwMCwgQW5kcmV3IEpvbmVzIHdyb3Rl Ogo+ID4gPiA+IE9uIFRodSwgSmFuIDEyLCAyMDIzIGF0IDEyOjI5OjU3QU0gKzAxMDAsIEhlaWtv IFN0w7xibmVyIHdyb3RlOgo+ID4gPiA+ID4gQW0gTWl0dHdvY2gsIDExLiBKYW51YXIgMjAyMywg MTg6MTA6MTkgQ0VUIHNjaHJpZWIgSmlzaGVuZyBaaGFuZzoKPiA+ID4gPiA+ID4gcmlzY3ZfY3B1 ZmVhdHVyZV9wYXRjaF9mdW5jKCkgY3VycmVudGx5IG9ubHkgc2NhbnMgYSBsaW1pdGVkIHNldCBv Zgo+ID4gPiA+ID4gPiBjcHVmZWF0dXJlcywgZXhwbGljaXRseSBkZWZpbmVkIHdpdGggbWFjcm9z LiBFeHRlbmQgaXQgdG8gcHJvYmUgZm9yIGFsbAo+ID4gPiA+ID4gPiBJU0EgZXh0ZW5zaW9ucy4K PiA+ID4gPiA+ID4gCj4gPiA+ID4gPiA+IFNpZ25lZC1vZmYtYnk6IEppc2hlbmcgWmhhbmcgPGpz emhhbmdAa2VybmVsLm9yZz4KPiA+ID4gPiA+ID4gUmV2aWV3ZWQtYnk6IEFuZHJldyBKb25lcyA8 YWpvbmVzQHZlbnRhbmFtaWNyby5jb20+Cj4gPiA+ID4gPiA+IFJldmlld2VkLWJ5OiBIZWlrbyBT dHVlYm5lciA8aGVpa29Ac250ZWNoLmRlPgo+ID4gPiA+ID4gPiAtLS0KPiA+ID4gPiA+ID4gIGFy Y2gvcmlzY3YvaW5jbHVkZS9hc20vZXJyYXRhX2xpc3QuaCB8ICA5ICsrLS0KPiA+ID4gPiA+ID4g IGFyY2gvcmlzY3Yva2VybmVsL2NwdWZlYXR1cmUuYyAgICAgICB8IDYzICsrKystLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0KPiA+ID4gPiA+ID4gIDIgZmlsZXMgY2hhbmdlZCwgMTEgaW5zZXJ0aW9u cygrKSwgNjEgZGVsZXRpb25zKC0pCj4gPiA+ID4gPiAKPiA+ID4gPiA+IGhtbW0gLi4uIEkgZG8g c2VlIGEgc29tZXdoYXQgYmlnIGNhdmVhdCBmb3IgdGhpcy4KPiA+ID4gPiA+IGFuZCB3b3VsZCBs aWtlIHRvIHRha2UgYmFjayBteSBSZXZpZXdlZC1ieSBmb3Igbm93Cj4gPiA+ID4gPiAKPiA+ID4g PiA+IAo+ID4gPiA+ID4gV2l0aCB0aGlzIGNoYW5nZSB3ZSB3b3VsZCBsaW1pdCB0aGUgcGF0Y2hh YmxlIGNwdWZlYXR1cmVzIHRvIGFjdHVhbAo+ID4gPiA+ID4gcmlzY3YgZXh0ZW5zaW9ucy4gQnV0 IGNwdWZlYXR1cmVzIGNhbiBhbHNvIGJlIHNvZnQgZmVhdHVyZXMgbGlrZQo+ID4gPiA+ID4gaG93 IHBlcmZvcm1hbnQgdGhlIGNvcmUgaGFuZGxlcyB1bmFsaWduZWQgYWNjZXNzZXMuCj4gPiA+ID4g Cj4gPiA+ID4gSSBhZ3JlZSB0aGF0IHRoaXMgbmVlZHMgdG8gYmUgYWRkcmVzc2VkIGFuZCBKaXNo ZW5nIGFsc28gcmFpc2VkIHRoaXMKPiA+ID4gPiB5ZXN0ZXJkYXkgaGVyZSBbKl0uIEl0IHNlZW1z IHdlIG5lZWQgdGhlIGNvbmNlcHQgb2YgY3B1ZmVhdHVyZXMsIHdoaWNoCj4gPiA+ID4gbWF5IGJl IGV4dGVuc2lvbnMgb3Igbm9uLWV4dGVuc2lvbnMuCj4gPiA+ID4gCj4gPiA+ID4gWypdIGh0dHBz Oi8vbG9yZS5rZXJuZWwub3JnL2FsbC9ZNzd4eU5QTnFuRlFVcUF4QHhoYWNrZXIvCj4gPiA+ID4g Cj4gPiA+ID4gPiBTZWUgUGFsbWVyJ3Mgc2VyaWVzIFswXS4KPiA+ID4gPiA+IAo+ID4gPiA+ID4g Cj4gPiA+ID4gPiBBbHNvIHRoaXMgZXNzZW50aWFsbHkgY29kaWZpZXMgdGhhdCBlYWNoIEFMVEVS TkFUSVZFIGNhbiBvbmx5IGV2ZXIKPiA+ID4gPiA+IGJlIGF0dGFjaGVkIHRvIGV4YWN0bHkgb25l IGV4dGVuc2lvbi4KPiA+ID4gPiA+IAo+ID4gPiA+ID4gQnV0IGNvbnRyYXJ5IHRvIHZlbmRvci1l cnJhdGEsIGl0IGlzIHZlcnkgbGlrZWx5IHRoYXQgd2Ugd2lsbCBuZWVkCj4gPiA+ID4gPiBjb21i aW5hdGlvbnMgb2YgZGlmZmVyZW50IGV4dGVuc2lvbnMgZm9yIHNvbWUgYWx0ZXJuYXRpdmVzIGlu IHRoZSBmdXR1cmUuCj4gPiA+ID4gCj4gPiA+ID4gT25lIHBvc3NpYmxlIGFwcHJvYWNoIG1heSBi ZSB0byBjb21iaW5lIGV4dGVuc2lvbnMvbm9uLWV4dGVuc2lvbnMgYXQgYm9vdAo+ID4gPiA+IHRp bWUgaW50byBwc2V1ZG8tY3B1ZmVhdHVyZXMuIFRoZW4sIGFsdGVybmF0aXZlcyBjYW4gY29udGlu dWUgYXR0YWNoaW5nIHRvCj4gPiA+ID4gYSBzaW5nbGUgImZlYXR1cmUiLiAoSSdtIG5vdCBzYXlp bmcgdGhhdCdzIGEgYmV0dGVyIGFwcHJvYWNoIHRoYW4gdGhlCj4gPiA+ID4gYml0bWFwLCBJJ20g anVzdCBzdWdnZXN0aW5nIGl0IGFzIHNvbWV0aGluZyBlbHNlIHRvIGNvbnNpZGVyLikKPiA+ID4g Cj4gPiA+IAo+ID4gPiA+ID4gICAgICAgICBBTFRFUk5BVElWRV8yKCJub3AiLAo+ID4gPiA+ID4g ICAgICAgICAgICAgICAgICAgICAgICJqIHN0cmNtcF96YmJfdW5hbGlnbmVkIiwgMCwgQ1BVRkVB VFVSRV9aQkIgfCBDUFVGRUFUVVJFX0ZBU1RfVU5BTElHTkVELCAwLCBDT05GSUdfUklTQ1ZfSVNB X1pCQiwKPiA+ID4gPiA+ICAgICAgICAgICAgICAgICAgICAgICAiaiB2YXJpYW50X3piYiIsIDAs IENQVUZFQVRVUkVfWkJCLCBDUFVGRUFUVVJFX0ZBU1RfVU5BTElHTkVELCBDT05GSUdfUklTQ1Zf SVNBX1pCQikKPiA+ID4gPiA+IAo+ID4gPiA+ID4gW3RoZSBhZGRpdGlvbmFsIGZpZWxkIHRoZXJl IG1vZGVscyBhICJub3QiIGNvbXBvbmVudF0KPiA+ID4gCj4gPiA+IFNpbmNlIHdlJ3JlIGRpc2N1 c3NpbmcgdGhlb3JldGljYWwgaW1wbGVtZW50YXRpb25zLCBhbmQgaXQncyBhIGxpdHRsZSBoYXJk Cj4gPiA+IHRvIHBpY3R1cmUgYWxsIHRoYXQgdGhleSBlbnRhaWwgaW4gbXkgaGVhZCwgSSBtaWdo dCBiZSBtYWtpbmcgYSBmb29sIG9mCj4gPiA+IG15c2VsZiBoZXJlIHdpdGggYXNzdW1wdGlvbnMu Li4KPiA+ID4gCj4gPiA+IEhlaWtvJ3Mgc3VnZ2VzdGlvbiBzb3VuZGVkIGFsb25nIHRoZSBsaW5l cyBvZjoga2VlcCBwcm9iaW5nIGluZGl2aWR1YWwKPiA+ID4gImZlYXR1cmVzIiBhcyB3ZSBhcmUg bm93LiBGZWF0dXJlcyBpbiB0aGlzIGNhc2UgYmVpbmcgdGhlIHByZXNlbmNlIG9mCj4gPiA+IHRo ZSBleHRlbnNpb24gb3Igbm9uLWV4dGVuc2lvbiBjYXBhYmlsaXR5LiBBbmQgdGhlbiBpbiB0aGUg YWx0ZXJuYXRpdmUsCj4gPiA+IG1ha2UgYWxsIG9mIHRoZSBkZWNpc2lvbnMgYWJvdXQgd2hpY2gg dG8gYXBwbHkuCj4gPiA+IAo+ID4gPiBEcmV3J3Mgc3VnZ2VzdGlvbiB3b3VsZCBoYXZlIHNpZ25p ZmljYW50bHkgbW9yZSBkZWZpbmVkIENQVUZFQVRVUkVfRk9PcywKPiA+ID4gYnV0IHdvdWxkIG9m ZmxvYWQgdGhlIGRlY2lzaW9uIG1ha2luZyBhYm91dCB3aGljaCBleHRlbnNpb25zIG9yIG5vbi0K PiA+ID4gZXh0ZW5zaW9uIGNhcGFiaWxpdGllcyBjb25zdGl0dXRlIGEgZmVhdHVyZSB0byByZWd1 bGFyIG9sZCBjcHVmZWF0dXJlCj4gPiA+IGNvZGUuIEhvd2V2ZXIsIHRoZSBvcmRlciBvZiBwcmVj ZWRlbmNlIHdvdWxkIHJlbWFpbiBpbiB0aGUgYWx0IG1hY3JvLCBhcwo+ID4gPiBpdCBkb2VzIG5v dy4KPiA+ID4gCj4gPiA+IEkgdGhpbmsgSSBhbSBqdXN0IGEgd2VlIGJpdCBiaWFzZWQsIGJ1dCBh ZGRpbmcgdGhlIGNvbXBsZXhpdHkgc29tZXdoZXJlCj4gPiA+IG90aGVyIHRoYW4gYWx0ZXJuYXRp dmUgbWFjcm9zIHNlZW1zIGEgd2lzZSBjaG9pY2UsIGVzcGVjaWFsbHkgYXMgd2UgYXJlCj4gPiA+ IGxpa2VseSB0byBmaW5kIHRoYXQgY29tcGxleGl0eSBpbmNyZWFzZXMgb3ZlciB0aW1lPwo+ID4g PiAKPiA+ID4gVGhlIG90aGVyIHRoaW5nIHRoYXQgY2FtZSB0byBtaW5kLCBhbmQgbWF5YmUgdGhp cyBpcyBqdXN0IGxvb2tpbmcgZm9yCj4gPiA+IGhvbGVzIHdoZXJlIHRoZXkgZG9uJ3QgZXhpc3Qg KG9yIGFyZSBub3Qgd29ydGggYWRkcmVzc2luZyksIGlzIHRoYXQKPiA+ID4gb3JkZXIgb2YgcHJl Y2VkZW5jZS4KPiA+ID4gSSBjYW4gaW1hZ2luZSB0aGF0LCBpbiBzb21lIGNhc2VzLCB0aGUgb3Jk ZXIgb2YgcHJlY2VkZW5jZSBpcyBub3QgYQo+ID4gPiBjb25zdGFudCBwZXIgcHN1ZWRvLWNwdWZl YXR1cmUsIGJ1dCBkZXRlcm1pbmVkIGJ5IGltcGxlbWVudGF0aW9uIG9mCj4gPiA+IHRoZSBjYXBh YmlsaXRpZXMgdGhhdCBjb21wcmlzZSBpdD8KPiA+IAo+ID4gSGF2aW5nIHNwZW50IGxvbmdlciB0 aGFuIEkgbWF5YmUgc2hvdWxkJ3ZlIGxvb2tpbmcgYXQgeW91ciBwYXRjaGVzCj4gPiBIZWlrbywg Z2l2ZW4gaXQncyBhIFNhdHVyZGF5IGV2ZW5pbmcsIHRoZSBwcmVjZWRlbmNlIHN0dWZmIGlzIHN0 aWxsCj4gPiBzdGlja2luZyBvdXQgdG8gbWUuLgo+ID4gCj4gPiBGb3IgWmJiICYgZmFzdCB1bmFs aWduZWQsIHRoZSBvcmRlciBtYXkgYmUgbm9uLWNvbnRyb3ZlcnNpYWwsIGJ1dCBpbgo+ID4gdGhl IGdlbmVyYWwgY2FzZSBJIGRvbid0IHNlZSBob3cgaXQgY2FuIGJlIHRydWUgdGhhdCB0aGUgb3Jk ZXIgb2YKPiA+IHByZWNlZGVuY2UgZm9yIHZhcmlhbnRzIGlzIGEgY29uc3RhbnQuCj4gPiAKPiA+ IENyZWF0aW5nIHBzZXVkbyBjcHVmZWF0dXJlcyBhcyBEcmV3IHN1Z2dlc3RlZCBkb2VzIHNlZW0g bGlrZSBhIHdheSB0bwo+ID4gZXh0cmFjdCBjb21wbGV4aXR5IGZyb20gdGhlIGFsdGVybmF0aXZl cyB0aGVtc2VsdmVzICh3aGljaCBJIHRoaW5rIGlzIGEKPiA+IGdvb2QgdGhpbmcpIGJ1dCBhdCB0 aGUgZXhwZW5zZSBvZiBlYXRpbmcgdXAgY3B1X3JlcV9mZWF0dXJlIGJpdHMuLi4KPiA+IEJ5IGl0 c2VsZiwgaXQgZG9lc24ndCBoZWxwIHdpdGggcHJlY2VkZW5jZSwgYnV0IGl0IG1heSBiZXR0ZXIg YWxsb3cgdXMKPiA+IHRvIGRlYWwgd2l0aCBzb21lIG9mIHRoZSBwcmVjZWRlbmNlIGluIGNwdWZl YXR1cmUuYywgc2luY2UgdGhlCj4gPiBhbHRlcm5hdGl2ZSB3b3VsZCBvcGVyYXRlIGJhc2VkIG9u IHRoZSBwc2V1ZG8gY3B1ZmVhdHVyZSByYXRoZXIgdGhhbiBvbgo+ID4gdGhlIGluZGl2aWR1YWwg Y2FwYWJpbGl0aWVzIHRoYXQgdGhlIHBzZXVkbyBjcHVmZWF0dXJlIGRlcGVuZHMgb24uCj4gPiAK PiA+IEkgdHJpZWQgdG8gY29tZSB1cCB3aXRoIGEgc3VnZ2VzdGlvbiBmb3Igd2hhdCB0byBkbyBh Ym91dCBwcmVjZWRlbmNlLAo+ID4gYnV0IGV2ZXJ5dGhpbmcgSSB0aG91Z2h0IHVwIGZlbHQgYSBi aXQgaG9ycmlmaWMgdGJoLgo+ID4gVGhlIHRoaW5nIHRoYXQgZml0cyB0aGUgY3VycmVudCBtb2Rl bCBiZXN0IGlzIGp1c3QgYWxsb3dpbmcgY3B1IHZlbmRvcnMKPiA+IHRvIGFkZCwgeWV0IG1vcmUs ICJlcnJhdGEiIHRoYXQgcGljayB0aGUgdmFyaWFudCB0aGF0IHdvcmtzIGJlc3QgZm9yCj4gPiB0 aGVpciBpbXBsZW1lbnRhdGlvbi4uLiBBbHRob3VnaCBJJ2QgYmUgd29ycmllZCBhYm91dCBiYWxs b29uaW5nIHNvbWUgb2YKPiA+IHRoZXNlIEFMVF9GT08gbWFjcm9zIG91dCB0byBhIG1hc3NpdmUg ZGVncmVlIHdpdGggdGhhdCBzb3J0IG9mIGFwcHJvYWNoLgo+ID4gCj4gPiA+IElmIG15IGFzc3Vt cHRpb24vdW5kZXJzdGFuZGluZyBob2xkcywgbW92aW5nIGRlY2lzaW9uIG1ha2luZyBvdXQgb2Yg dGhlCj4gPiA+IGFsdGVybmF0aXZlIHNlZW1zIGxpa2UgaXQgd291bGQgYmV0dGVyIHByb3Zpc2lv biBmb3Igc2NlbmFyaW9zIGxpa2UKPiA+ID4gdGhhdD8gSSBkdW5ubywgbWF5YmUgdGhhdCBpcyB3 aGF0ZXZlciB0aGUgY29yb2xsYXJ5IG9mICJwcmVtYXR1cmUKPiA+ID4gb3B0aW1pc2F0aW9uIiBp cyBmb3IgdGhpcyBkaXNjdXNzaW9uLgo+ID4gPiAKPiA+ID4gVGhhdCdzIG15IHVuc29saWNpdGVk IOKCrCAwLjAyLCBob3BlZnVsbHkgSSB3YXNuJ3Qgb2ZmLWJhc2Ugd2l0aCB0aGUKPiA+ID4gYXNz dW1wdGlvbnMgSSBtYWRlLgo+ID4gCj4gPiBUaGUgb3JkZXIgaW4gd2hpY2ggYW4gYWx0ZXJuYXRp dmUgaXMgYWRkZWQgdG8gdGhlIG1hY3JvIGRvZXMgbWF0dGVyLAo+ID4gcmlnaHQ/IEF0IGxlYXN0 LCB0aGF0J3MgaG93IEkgdGhvdWdodCBpdCB3b3JrZWQgYW5kIGhvcGUgSSd2ZSBub3QgaGFkCj4g PiBhbiBpbmNvcnJlY3QgaW50ZXJwcmV0YXRpb24gdGhlcmUgYWxsIGFsb25nLi4uIEkgd2Fzbid0 IHVudGlsIEkgc3RhcnRlZAo+ID4gcmVhZGluZyB5b3VyIHBhdGNoIGFuZCBjb3VsZG4ndCB1bmRl cnN0YW5kIHdoeSB5b3UgaGFkIGEgY29uc3RydWN0IHRoYXQKPiA+IGxvb2tlZCBsaWtlCj4gPiAK PiA+IGlmICh6YmIgJiYgIWZhc3RfdW5hbGlnbmVkKQo+ID4gLi4uCj4gPiBlbHNlIGlmICh6YmIg JiYgZmFzdF91bmFsaWduZWQpCj4gPiAuLi4KPiA+IAo+ID4gcmF0aGVyIHRoYW4ganVzdCBpbnZl cnRpbmcgdGhlIG9yZGVyIGFuZCBkcm9wcGluZyB0aGUgIWZhc3RfdW5hbGlnbmVkCj4gPiB0aGF0 IEkgcmVhbGlzZWQgSSBtaWdodCBoYXZlIGEgZ2FwIGluIG15IHVuZGVyc3RhbmRpbmcgYWZ0ZXIg YWxsLi4KPiA+IAo+ID4gPiBIZWlrbywgSSBmaWd1cmUgeW91J3ZlIGdvdCBzb21lIHNvcnQgb2Yg V0lQIHN0dWZmIGZvciB0aGlzIGFueXdheSBzaW5jZQo+ID4gPiB5b3UncmUgaW50ZXJlc3RlZCBp biB0aGUgZmFzdCB1bmFsaWduZWQ/IEhvdyBjbG9zZSBhcmUgeW91IHRvIHBvc3RpbmcgYW55Cj4g PiA+IG9mIHRoYXQ/Cj4gPiA+IAo+ID4gPiBXaGlsZSBJIHRoaW5rIG9mIGl0LCB3LnIudC4gZXh0 ZW5zaW9uIHZlcnN1cyAocHNldWRvKWNwdWZlYXR1cmUgZXRjCj4gPiA+IG5hbWluZywgaXQgbWF5 IG1ha2Ugc2Vuc2UgdG8gY2FsbCB0aGUgZnVuY3Rpb25zIHRoYXQgdGhpcyBzZXJpZXMgYWRkcwo+ ID4gPiBpbiBwYXRjaCA2IGhhc19jcHVmZWF0dXJlX3t1bix9bGlrZWx5KCksIG5vIG1hdHRlciB3 aGF0IGRlY2lzaW9uIGdldHMKPiA+ID4gbWFkZSBoZXJlPwo+ID4gPiBJTU8gdXNpbmcgY3B1ZmVh dHVyZSBzZWVtcyB0byBtYWtlIG1vcmUgc2Vuc2UgZm9yIGEgZ2VuZXJhbCB1c2UgQVBJIHRoYXQK PiA+ID4gbWF5IGJlIHVzZWQgbGF0ZXIgb24gZm9yIHRoZSBsaWtlcyBvZiB1bmFsaWduZWQgYWNj ZXNzLCBldmVuIGlmCj4gPiA+IGluaXRpYWxseSBpdCBpcyBub3QgdXNlZCBmb3IgYW55dGhpbmcg b3RoZXIgdGhhbiBleHRlbnNpb25zLiAKPiAKPiBUb2RheSBhdCBbMV0gd2UgdGFsa2VkIGEgYml0 IGFib3V0IHRoZSB2YXJpb3VzIGJpdHMgZ29pbmcgb24gaGVyZS4KPiBJJ2xsIGF0dGVtcHQgdG8g c3VtbWFyaXNlIHdoYXQgSSByZW1lbWJlciwgYnV0IEkgbWVhbnQgdG8gZG8gdGhpcwo+IHNldmVy YWwgaG91cnMgYWdvIGFuZCBhbSBsaWtlbHkgdG8gbWFrZSBhIGhhbWVzIG9mIGl0Lgo+IAo+IEZv ciBaYmIvdW5hbGlnbmVkIHN0dWZmLCB0aGUgc2VudGltZW50IHdhcyBhbG9uZyB0aGUgbGluZXMg b2YgdGhlcmUKPiBuZWVkaW5nIHRvIGJlIGEgcGVyZm9ybWFuY2UgYmVuZWZpdCB0byBqdXN0aWZ5 IHRoZSBpbmNsdXNpb24uCj4gU29tZSBvZiB1cyBoYXZlIEhXIHRoYXQgaXMgKGFsbGVnZWRseSkg Y2FwYWJsZSBvZiBaYmIsIGFuZCwgaWYgdGhhdCdzIHRoZQo+IGNhc2UsIHdpbGwgZ2l2ZSBpdCBh IGdvLgo+IEkgdGhpbmsgaXQgd2FzIHNpbWlsYXIgZm9yIHVuYWxpZ25lZCwgc2luY2UgdGhlcmUg aXMgbm90aGluZyB5ZXQgdGhhdAo+IHN1cHBvcnRzIHRoaXMgYmVoYXZpb3VyLCB3ZSBzaG91bGQg d2FpdCB1bnRpbCBhIGJlbmVmaXQgaXMgZGVtb25zdHJhYmxlLgo+IAo+IE9uIHRoZSBzdWJqZWN0 IG9mIGdyb3VwaW5nIGV4dGVuc2lvbi9ub24tZXh0ZW5zaW9uIGNhcGFiaWxpdGllcyBpbnRvIGEK PiBzaW5nbGUgY3B1ZmVhdHVyZSwgUGFsbWVyIG1lbnRpb25lZCB0aGF0IEdDQyBkb2VzIHNvbWV0 aGluZyBzaW1pbGFyLAo+IGZvciB0aGUgbGlrZXMgb2YgdGhlIFZlbnRhbmEgdmVuZG9yIGV4dGVu c2lvbnMsIHRoYXQgYXJlIHVubGlrZWx5IHRvIGJlCj4gcHJlc2VudCBpbiBpc29sYXRpb24uCj4g VGhvc2UgYXJlIChvciB3ZXJlPykgcHJvYmVkIGFzIGEgZ3JvdXAgb2YgZXh0ZW5zaW9ucyByYXRo ZXIgdGhhbgo+IGluZGl2aWR1YWxseS4KPiBJIHRoaW5rIGl0IHdhcyBzYWlkIGl0J2QgbWFrZSBz ZW5zZSBmb3IgdXMgdG8gdW5pZnkgZXh0ZW5zaW9ucyB0aGF0IHdpbGwKPiBvbmx5IGV2ZXIgYXBw ZWFyIHRvZ2V0aGVyIHNpbmdsZSBwc3VlZG8gY3B1ZmVhdHVyZSB0b28uCj4gCj4gRm9yIHRoZSBi aXRmaWVsZCBhcHByb2FjaCB2ZXJzdXMgY3JlYXRpbmcgcHNldWRvIGNwdWZlYXR1cmVzIGRpc2N1 c3Npb24KPiAmIGhvdyB0byBkZWFsIHdpdGggdGhhdCBpbiBhbHRlcm5hdGl2ZXMgZXRjLCBJJ20g YSBiaXQgbGVzcyBzdXJlIHdoYXQgdGhlCj4gb3V0Y29tZSB3YXMuCj4gSUlSQywgbm90aGluZyBj b25jcmV0ZSB3YXMgc2FpZCBhYm91dCBlaXRoZXIgYXBwcm9hY2gsIGJ1dCBtYXliZSBpdCB3YXMK PiBpbXBsaWVkIHRoYXQgd2Ugc2hvdWxkIGRvIGFzIEdDQyBkb2VzLCBvbmx5IGdyb3VwaW5nIHRo aW5ncyB0aGF0IHdvbid0Cj4gZXZlciBiZWVuIHNlZW4gYXBhcnQuCj4gRmlndXJpbmcgdGhhdCBv dXQgc2VlbXMgdG8gaGF2ZSBiZWVuIHB1bnRlZCBkb3duIHRoZSByb2FkLCBhcyB0aGUKPiBpbmNs dXNpb24gb2Ygb3VyIG9ubHkgY3VycmVudCBleGFtcGxlIG9mIHRoaXMgKFpiYiArIHVuYWxpZ25l ZCkgaXMKPiBkZXBlbmRhbnQgb24gaGFyZHdhcmUgc2hvd2luZyB1cCB0aGF0IGFjdHVhbGx5IGJl bmVmaXRzIGZyb20gaXQuCj4gCj4gVGhlIHBsYW4gdGhlbiBzZWVtZWQgdG8gYmUgcHJlc3MgYWhl YWQgd2l0aCB0aGlzIHNlcmllcyAmIHRlc3QgdGhlCj4gYmVuZWZpdHMgb2YgdGhlIFpiYiBzdHIq IGZ1bmN0aW9ucyBpbiBaYmIgY2FwYWJsZSBoYXJkd2FyZSBiZWZvcmUgbWFraW5nCj4gYSBkZWNp c2lvbiB0aGVyZS4KPiAKPiBIb3BlZnVsbHkgSSB3YXNuJ3QgdG9vIGZhciBvZmYgd2l0aCB0aGF0 IHN1bW1hcnkuLi4KClRoaXMgbWF0Y2hlcyBteSByZWNvbGxlY3Rpb24uIFRoYW5rcyBmb3IgdGhl IHN1bW1hcnksIENvbm9yLgoKZHJldwoKPiAKPiBUaGFua3MsCj4gQ29ub3IuCj4gCj4gMSAtIGh0 dHBzOi8vbG9yZS5rZXJuZWwub3JnL2xpbnV4LXJpc2N2L21obmctNzc1ZDQwNjgtNmMxZS00OGE0 LWExZGMtYjRhNzZmZjI2YmIzQHBhbG1lci1yaS14MWM5YS8KCgoKX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGludXgtcmlzY3YgbWFpbGluZyBsaXN0Cmxp bnV4LXJpc2N2QGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVhZC5vcmcv bWFpbG1hbi9saXN0aW5mby9saW51eC1yaXNjdgo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71AF8C004D4 for ; Thu, 19 Jan 2023 08:29:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229696AbjASI3L (ORCPT ); Thu, 19 Jan 2023 03:29:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46776 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229501AbjASI3I (ORCPT ); Thu, 19 Jan 2023 03:29:08 -0500 Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com [IPv6:2a00:1450:4864:20::534]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CCC75AC for ; Thu, 19 Jan 2023 00:29:06 -0800 (PST) Received: by mail-ed1-x534.google.com with SMTP id v13so1881610eda.11 for ; Thu, 19 Jan 2023 00:29:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ventanamicro.com; s=google; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=x5ck+uBW+we5mcml99JIlI5hbqOexLvffduO15vMLCQ=; b=JtUEKYA4ngIiCcO38C3pG5dWS8AmHuQYxbH7z4bS0gjTrWl2JjNzmgu+bHT1rhVvCF 1AyQtvmN+FtmrNaWPPlJxZ1bL2eW7Oditr9V7+Te0BCwylH/bu1UdZdtD74u9sKME4pq 4gLh+tfjkooAP7I3TeDdqGgUeUnakDAAJVoJn1lZJ0nt6ynYWYQ2hSzGBEW18rNJpTE7 X/bTKUyOfdx/C86S3LXuUQzM7SIH2k3OgDl+54FiyHI4lRaVEg82PMf5IzorscxABa4G 3rV6OMUb6N6Qq3ZkX/pp6a00uXYu/B094V6gztspTEq96rPZswMmsEwrXoIDCMXflQrs Yk6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=x5ck+uBW+we5mcml99JIlI5hbqOexLvffduO15vMLCQ=; b=eTFMUwFaZGSlZ7Xi/ERUZ4l2ZUcg/p9r6QBtpeUdFL4NAL+UOO6kVv6Y4GRMongBCz dD6YiBRD2OGd+D2Ho9JvFIQywB78kAVA6H7qyT86K3Hhn6OHFgBsvD7KAps9OcIbVh4b i3WvCc9t9X55KnVdcnCTjwmc3XpChBsDY6pZQlYQR8qCkWYrVJ8DwtlRDvR98DxITahP qTJzcPTNVdnHDSfkDXDzvEGEwa8wSA2339ZZjROZs4AQhak+0haUx/kVJTRrJLMgmY5O ft4NxfhVt5t9sblew6kOHavhiEKJEaNHHwitRt6xuE++bTZ3pKA/TpbpF1vKUzTxFeCn tpew== X-Gm-Message-State: AFqh2kodh4tqMVsCFKNQXMOgCT9JL3ZPazr4T90dcLczB8KjU9EkO7vf A2J2ZGAPWr6Wo/mgCwUbl4LD8wK6V7dy41KZ X-Google-Smtp-Source: AMrXdXt37wqPfFZsW/BhY2ZBx33uOivJd/nYfcmjoJqMYW92j1bl/42mtZmEp65y22+4CKilHwwZSg== X-Received: by 2002:a05:6402:b:b0:496:f517:d30d with SMTP id d11-20020a056402000b00b00496f517d30dmr10408911edu.38.1674116945288; Thu, 19 Jan 2023 00:29:05 -0800 (PST) Received: from localhost (cst2-173-16.cust.vodafone.cz. [31.30.173.16]) by smtp.gmail.com with ESMTPSA id a3-20020aa7cf03000000b0049019b48373sm15294210edy.85.2023.01.19.00.29.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Jan 2023 00:29:04 -0800 (PST) Date: Thu, 19 Jan 2023 09:29:03 +0100 From: Andrew Jones To: Conor Dooley Cc: Heiko =?utf-8?Q?St=C3=BCbner?= , Palmer Dabbelt , Paul Walmsley , Albert Ou , Anup Patel , Atish Patra , Jisheng Zhang , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvm-riscv@lists.infradead.org Subject: Re: [PATCH v3 05/13] riscv: cpufeature: extend riscv_cpufeature_patch_func to all ISA extensions Message-ID: <20230119082903.yk3uslfrjtxzassi@orel> References: <20230111171027.2392-1-jszhang@kernel.org> <20230111171027.2392-6-jszhang@kernel.org> <2398293.3Lj2Plt8kZ@diego> <20230112092136.f2g43hrhmrqouy4y@orel> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org On Wed, Jan 18, 2023 at 09:54:46PM +0000, Conor Dooley wrote: > Hey! > > I guess here is the right place to follow up on all of this stuff... > > On Sat, Jan 14, 2023 at 08:32:37PM +0000, Conor Dooley wrote: > > On Fri, Jan 13, 2023 at 03:18:59PM +0000, Conor Dooley wrote: > > > On Thu, Jan 12, 2023 at 10:21:36AM +0100, Andrew Jones wrote: > > > > On Thu, Jan 12, 2023 at 12:29:57AM +0100, Heiko Stübner wrote: > > > > > Am Mittwoch, 11. Januar 2023, 18:10:19 CET schrieb Jisheng Zhang: > > > > > > riscv_cpufeature_patch_func() currently only scans a limited set of > > > > > > cpufeatures, explicitly defined with macros. Extend it to probe for all > > > > > > ISA extensions. > > > > > > > > > > > > Signed-off-by: Jisheng Zhang > > > > > > Reviewed-by: Andrew Jones > > > > > > Reviewed-by: Heiko Stuebner > > > > > > --- > > > > > > arch/riscv/include/asm/errata_list.h | 9 ++-- > > > > > > arch/riscv/kernel/cpufeature.c | 63 ++++------------------------ > > > > > > 2 files changed, 11 insertions(+), 61 deletions(-) > > > > > > > > > > hmmm ... I do see a somewhat big caveat for this. > > > > > and would like to take back my Reviewed-by for now > > > > > > > > > > > > > > > With this change we would limit the patchable cpufeatures to actual > > > > > riscv extensions. But cpufeatures can also be soft features like > > > > > how performant the core handles unaligned accesses. > > > > > > > > I agree that this needs to be addressed and Jisheng also raised this > > > > yesterday here [*]. It seems we need the concept of cpufeatures, which > > > > may be extensions or non-extensions. > > > > > > > > [*] https://lore.kernel.org/all/Y77xyNPNqnFQUqAx@xhacker/ > > > > > > > > > See Palmer's series [0]. > > > > > > > > > > > > > > > Also this essentially codifies that each ALTERNATIVE can only ever > > > > > be attached to exactly one extension. > > > > > > > > > > But contrary to vendor-errata, it is very likely that we will need > > > > > combinations of different extensions for some alternatives in the future. > > > > > > > > One possible approach may be to combine extensions/non-extensions at boot > > > > time into pseudo-cpufeatures. Then, alternatives can continue attaching to > > > > a single "feature". (I'm not saying that's a better approach than the > > > > bitmap, I'm just suggesting it as something else to consider.) > > > > > > > > > > > ALTERNATIVE_2("nop", > > > > > "j strcmp_zbb_unaligned", 0, CPUFEATURE_ZBB | CPUFEATURE_FAST_UNALIGNED, 0, CONFIG_RISCV_ISA_ZBB, > > > > > "j variant_zbb", 0, CPUFEATURE_ZBB, CPUFEATURE_FAST_UNALIGNED, CONFIG_RISCV_ISA_ZBB) > > > > > > > > > > [the additional field there models a "not" component] > > > > > > Since we're discussing theoretical implementations, and it's a little hard > > > to picture all that they entail in my head, I might be making a fool of > > > myself here with assumptions... > > > > > > Heiko's suggestion sounded along the lines of: keep probing individual > > > "features" as we are now. Features in this case being the presence of > > > the extension or non-extension capability. And then in the alternative, > > > make all of the decisions about which to apply. > > > > > > Drew's suggestion would have significantly more defined CPUFEATURE_FOOs, > > > but would offload the decision making about which extensions or non- > > > extension capabilities constitute a feature to regular old cpufeature > > > code. However, the order of precedence would remain in the alt macro, as > > > it does now. > > > > > > I think I am just a wee bit biased, but adding the complexity somewhere > > > other than alternative macros seems a wise choice, especially as we are > > > likely to find that complexity increases over time? > > > > > > The other thing that came to mind, and maybe this is just looking for > > > holes where they don't exist (or are not worth addressing), is that > > > order of precedence. > > > I can imagine that, in some cases, the order of precedence is not a > > > constant per psuedo-cpufeature, but determined by implementation of > > > the capabilities that comprise it? > > > > Having spent longer than I maybe should've looking at your patches > > Heiko, given it's a Saturday evening, the precedence stuff is still > > sticking out to me.. > > > > For Zbb & fast unaligned, the order may be non-controversial, but in > > the general case I don't see how it can be true that the order of > > precedence for variants is a constant. > > > > Creating pseudo cpufeatures as Drew suggested does seem like a way to > > extract complexity from the alternatives themselves (which I think is a > > good thing) but at the expense of eating up cpu_req_feature bits... > > By itself, it doesn't help with precedence, but it may better allow us > > to deal with some of the precedence in cpufeature.c, since the > > alternative would operate based on the pseudo cpufeature rather than on > > the individual capabilities that the pseudo cpufeature depends on. > > > > I tried to come up with a suggestion for what to do about precedence, > > but everything I thought up felt a bit horrific tbh. > > The thing that fits the current model best is just allowing cpu vendors > > to add, yet more, "errata" that pick the variant that works best for > > their implementation... Although I'd be worried about ballooning some of > > these ALT_FOO macros out to a massive degree with that sort of approach. > > > > > If my assumption/understanding holds, moving decision making out of the > > > alternative seems like it would better provision for scenarios like > > > that? I dunno, maybe that is whatever the corollary of "premature > > > optimisation" is for this discussion. > > > > > > That's my unsolicited € 0.02, hopefully I wasn't off-base with the > > > assumptions I made. > > > > The order in which an alternative is added to the macro does matter, > > right? At least, that's how I thought it worked and hope I've not had > > an incorrect interpretation there all along... I wasn't until I started > > reading your patch and couldn't understand why you had a construct that > > looked like > > > > if (zbb && !fast_unaligned) > > ... > > else if (zbb && fast_unaligned) > > ... > > > > rather than just inverting the order and dropping the !fast_unaligned > > that I realised I might have a gap in my understanding after all.. > > > > > Heiko, I figure you've got some sort of WIP stuff for this anyway since > > > you're interested in the fast unaligned? How close are you to posting any > > > of that? > > > > > > While I think of it, w.r.t. extension versus (pseudo)cpufeature etc > > > naming, it may make sense to call the functions that this series adds > > > in patch 6 has_cpufeature_{un,}likely(), no matter what decision gets > > > made here? > > > IMO using cpufeature seems to make more sense for a general use API that > > > may be used later on for the likes of unaligned access, even if > > > initially it is not used for anything other than extensions. > > Today at [1] we talked a bit about the various bits going on here. > I'll attempt to summarise what I remember, but I meant to do this > several hours ago and am likely to make a hames of it. > > For Zbb/unaligned stuff, the sentiment was along the lines of there > needing to be a performance benefit to justify the inclusion. > Some of us have HW that is (allegedly) capable of Zbb, and, if that's the > case, will give it a go. > I think it was similar for unaligned, since there is nothing yet that > supports this behaviour, we should wait until a benefit is demonstrable. > > On the subject of grouping extension/non-extension capabilities into a > single cpufeature, Palmer mentioned that GCC does something similar, > for the likes of the Ventana vendor extensions, that are unlikely to be > present in isolation. > Those are (or were?) probed as a group of extensions rather than > individually. > I think it was said it'd make sense for us to unify extensions that will > only ever appear together single psuedo cpufeature too. > > For the bitfield approach versus creating pseudo cpufeatures discussion > & how to deal with that in alternatives etc, I'm a bit less sure what the > outcome was. > IIRC, nothing concrete was said about either approach, but maybe it was > implied that we should do as GCC does, only grouping things that won't > ever been seen apart. > Figuring that out seems to have been punted down the road, as the > inclusion of our only current example of this (Zbb + unaligned) is > dependant on hardware showing up that actually benefits from it. > > The plan then seemed to be press ahead with this series & test the > benefits of the Zbb str* functions in Zbb capable hardware before making > a decision there. > > Hopefully I wasn't too far off with that summary... This matches my recollection. Thanks for the summary, Conor. drew > > Thanks, > Conor. > > 1 - https://lore.kernel.org/linux-riscv/mhng-775d4068-6c1e-48a4-a1dc-b4a76ff26bb3@palmer-ri-x1c9a/