From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrew Jones Date: Wed, 24 Apr 2024 12:32:04 +0200 Subject: [PATCH v2 2/6] dt-bindings: riscv: Add Zawrs ISA extension description In-Reply-To: References: <20240419-8c6af6a169a7aa0b9aa5cac5@orel> <20240419-disdain-litmus-82874cc4872e@spud> <20240421-f63f90c2146deb0695b84441@orel> <20240423-ed9ddb701be1df4a25e29f31@orel> <20240424-0e571b29835610c24409762b@orel> Message-ID: <20240424-c24522c332fcaced37339045@orel> List-Id: To: kvm-riscv@lists.infradead.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit On Wed, Apr 24, 2024 at 11:23:00AM +0200, Christoph M?llner wrote: > On Wed, Apr 24, 2024 at 9:34?AM Andrew Jones wrote: > > > > On Tue, Apr 23, 2024 at 03:42:47PM -0400, Charlie Jenkins wrote: > > > On Tue, Apr 23, 2024 at 02:00:53PM -0400, Charlie Jenkins wrote: > > > > On Tue, Apr 23, 2024 at 10:46:01AM +0200, Andrew Jones wrote: > > > > > On Mon, Apr 22, 2024 at 06:36:45PM -0400, Charlie Jenkins wrote: > > > > > > On Sun, Apr 21, 2024 at 12:20:03PM +0200, Andrew Jones wrote: > > > > > > > On Fri, Apr 19, 2024 at 12:40:01PM -0400, Charlie Jenkins wrote: > > > > > ... > > > > > > > > What would be the purpose of a vendor implementing WRS.NTO (and putting > > > > > > > > it in the DT) that never terminates? The spec says "Then a subsequent > > > > > > > > WRS.NTO instruction would cause the hart to temporarily stall execution > > > > > > > > in a low- power state until a store occurs to the reservation set or an > > > > > > > > interrupt is observed." Why is this wording for WRS.NTO not sufficient > > > > > > > > to assume that an implementation of this instruction would eventually > > > > > > > > terminate? > > > > > > > > > > > > > > > > > > > > > > We can invoke smp_cond_load_relaxed(addr, VAL || anything_we_want()). This > > > > > > > means we may not expect VAL ever to be written, which rules out "until a > > > > > > > store occurs". As for "an interrupt is observed", we don't know which one > > > > > > > to expect to arrive within a "reasonable" amount of time. We need to know > > > > > > > which one(s), since, while wrs.nto will terminate even when interrupts are > > > > > > > globally disabled, we still need to have the interrupt(s) we expect to be > > > > > > > locally enabled. And, the interrupts should arrive in a "reasonable" > > > > > > > amount of time since we want to poll anything_we_want() at a "reasonable" > > > > > > > frequency. > > > > > > > > > > > > > > So, we need firmware to promise to enable exceptions if there aren't any > > > > > > > such interrupts. Or, we could require hardware descriptions to identify > > > > > > > which interrupt(s) would be good to have enabled before calling wrs.nto. > > > > > > > Maybe there's already some way to describe something like that? > > > > > > > > > > > > > > Thanks, > > > > > > > drew > > > > > > > > > > > > Ahh okay I am caught up now. So the wording we are looking at in the > > > > > > spec is: > > > > > > > > > > > > "When executing in VS or VU mode, if the VTW bit is set in hstatus, the > > > > > > TW bit in mstatus is clear, and the WRS.NTO does not complete within an > > > > > > implementation-specific bounded time limit, the WRS.NTO instruction will > > > > > > cause a virtual instruction exception." > > > > > > > > > > That's what the hypervisor should promise to do when there's no other > > > > > guarantee of wrs.nto terminating (but the hypervisor likely wants to > > > > > anyway since it wants guests to trap on wrs.nto in order to potentially > > > > > schedule the lock holding VCPU). The firmware of the host should likewise > > > > > promise to set mstatus.TW when there's no guarantee of wrs.nto > > > > > terminating, but that's likely _not_ something it normally would want to > > > > > do, so hopefully there will always be implementation-specific "other > > > > > reasons" which guarantee termination. > > > > > > > > > > > > > > > > > With the concern being that it is possible for "implementation-specific > > > > > > bounded time limit" to be infinite/never times out, > > > > > > > > > > The implementation-defined short timeout cannot be infinite, but it only > > > > > applies to wrs.sto. While using wrs.sto would relieve the concern, it > > > > > cannot be configured to raise exceptions, which means it's not useful in > > > > > guests. If we want to use wrs.sto in hosts and wrs.nto in guests then we > > > > > need a paravirt channel which allows an "enlightened" guest to determine > > > > > that it is a guest and that the hypervisor has configured wrs.nto to > > > > > trap, which then indicates it's a good idea to patch wrs.sto to wrs.nto. > > > > > But, adding paravirt stuff should be avoided whenever possible since it > > > > > adds complexity we'd rather not maintain. > > > > > > > > > > > > > That still wouldn't solve this issue, because the wrs.nto guest may still > > > > never wakeup in the implementation-specific way? > > > > The paravirt approach does solve it, because wrs.nto is specified to raise > > exceptions after an implementation-specific bounded time limit when the > > hypervisor sets hstatus.VTW. > > > > > > > > Thinking about this a bit more, this is a performance penalty and not a > > > correctness issue. > > > > It's incorrect to have a design that is likely to result in bad > > performance. > > > > > This line is an optimization that allows the lock > > > holder to jump the queue if the holder directly in front is a preempted > > > vcpu. Eventually the vcpu will be scheduled again and give up the lock. > > > So an implementation of WRS.NTO that does not have the > > > "implementation-specific bounded time limit" that the spec calls out for > > > WRS.NTO to raise a virtual instruction exception, would still function, > > > just slower. > > > > The problem isn't specific to virtualization. The problem is using wrs.nto > > when it has not been configured to raise exceptions and there are not any > > other guaranteed termination events (other than the reservation set > > becoming invalid). While the paravirt solution is virtualization specific, > > it works, because we would then only use wrs.nto when we know we can, and > > otherwise use wrs.sto. But, as I said, I'd rather not have a paravirt > > solution. > > Andrew, it would be great if you could summarize this finding to the > spec authors. > Maybe a non-normative text added to the spec (that raises awareness > for the issue Sure, I'll write something up pointing out the concern with wrs.nto and post it to a few RVI mailing lists. > and provides a guideline to avoid it) could reduce the risk of triggering > the issue on real HW. This might be enough justification to use WRS.NTO. > > If WRS.NTO is considered as not reliable enough to wake up and therefore causing > performance issues or CPU stalls if used for the spin lock optimization, > it might be also reasonable to use WRS.STO instead. > The cost of having too many wakeups seems much more acceptable than > a stalled CPU. wrs.sto is reasonable to use in all cases, since too many wakeups isn't a concern. But, we can do better in the virtualization case with wrs.nto, where the hypervisor can get involved, so, to avoid paravirt stuff, we'd like to be able to always use wrs.nto. Thanks, drew From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7CDE8C4345F for ; Wed, 24 Apr 2024 10:32:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=4ZsT+u5qNSL+vYZnJsRmwIpSTfk2bl9G/I2t9LYHxmo=; b=Sn+TNRlndp+PFd MWTJebFW0/hnv+NuwY2KvCFtXG5kyaYn0mfhtDhjzYqeybAwV61HxoOVZ5gR+phe66+Yq2mIZ8fQm izW66R0vZGFXq/w/ydk72JX+Wv8rSqGYLXz/9tZY7K8zi7uT1rnlMlVGpnxClVuYt01foDFNSF+B8 RBRLNLYvhZJ0LC3wgHKyxyOILyleb7K5g5JpmSRZT9sP5tDh2CJ/VRGvl+u3M9+wNnI/0LtNIAxK1 dFjvsAkJ62+ZSKOvddXqY4ILOQqHiHgQyJueJXfOafu/Rw1P/IwMB/UyzbHZQfp8UwKC9j5VjmwxD opGvDxjUHAxsdVQAOc/w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rzZvA-00000003hwW-1PG2; Wed, 24 Apr 2024 10:32:12 +0000 Received: from mail-lf1-x12a.google.com ([2a00:1450:4864:20::12a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rzZv7-00000003hvT-29O7 for linux-riscv@lists.infradead.org; Wed, 24 Apr 2024 10:32:11 +0000 Received: by mail-lf1-x12a.google.com with SMTP id 2adb3069b0e04-51abd9fcbf6so9566462e87.1 for ; Wed, 24 Apr 2024 03:32:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ventanamicro.com; s=google; t=1713954726; x=1714559526; darn=lists.infradead.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=69da6UXz2pFiDL31edeqzPN31G6ffxNroQ0Z2N+KdsQ=; b=Nrv766vJ8e6wWAHyMsR30WBHYRseUBniV0fFZgpnZClPPkCxTA29gKlR+mYMlKeLnq wErezrSyifuXw/giY+sDIHrKV/+mdr7ZIJFsG0WT1sJh9oIWVv3Fd81RLKNiGYOwF57b Cg+G8Yc5ITM4bIaGQhg09LCuyt5jRWLByacty71vmRrlrdQQBX4zakel5hlv4/hCMpu7 45HIpiXs32vxmYPi7PzUadd9Q44EwJvJ/DzhSI2AYjjFmxGbjOaW8JUbnXolxfsE1MZB +UugJwUf9QmrY8xeW33xMMU94oNXwAaC2wyeMCe+VwRN9Z8XvKbi6NKNF0F4JHo7d1zs ElWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713954726; x=1714559526; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=69da6UXz2pFiDL31edeqzPN31G6ffxNroQ0Z2N+KdsQ=; b=J1eJaqq2s+BTZyWXnDP53qBjfztRc/N0QZ72W8f63iuTxnQr/DcA8NsqxeFPQkdjhT hvvRe3pQ2SCh+HxyQDHYeitTayfR7rwtdaqbNy0Nz6b+X9l/y2ACeAVDWvJUAAWa/Z9I RlCR2K17FYSdlveA1KQqG3wR+2eZdXRY4PlKnAIkeOAMCgvG2B1ZKraGUJuVH/Z+knFo KRTkurtKlDGxyz3RARVl4f2Q8v3scMJJ3A3P8J31Z9gDpGyvMYYj9pyAxYZ/xWwjVubJ 51iaqZMgwh3HI7zphhK1zxKVu1NN1scilzJ1vBSBBx4bmj6zJAWI+HMhpZh6ugXZ7KPe h8tg== X-Forwarded-Encrypted: i=1; AJvYcCXSKcRjPDQu35wWqC+hRIDD27pyBYfCvX2wgBjUAAF023JXD9wOcTWSbS7WYHqc2HU3JBw7YYtLD9yOUyEL5kC1cb3Tmdbq+6qbTODUoPiI X-Gm-Message-State: AOJu0Yy1nkT3SBJBPfCLfh+jBK56CgcU/o9WGH9vyHAqgOWiSj8TVJV8 0+EOqReD3nhUvaHPA4WSzjFyI3CVi3yWyRcYoSdAH/a6uWp0HwFvV/apgcVyLNQRQytGZFji2f7 UcIM= X-Google-Smtp-Source: AGHT+IEcj0tdrrmR8cPhcTUlqc6Z+F7kWLT0j9ptexs5NM6zhH3y23MG/ibcIMnGQ8n53KYceeEqdw== X-Received: by 2002:a05:6512:793:b0:51a:affe:252e with SMTP id x19-20020a056512079300b0051aaffe252emr1747408lfr.37.1713954726126; Wed, 24 Apr 2024 03:32:06 -0700 (PDT) Received: from localhost (2001-1ae9-1c2-4c00-20f-c6b4-1e57-7965.ip6.tmcz.cz. [2001:1ae9:1c2:4c00:20f:c6b4:1e57:7965]) by smtp.gmail.com with ESMTPSA id z2-20020a17090665c200b00a55bbc4fd16sm3274471ejn.48.2024.04.24.03.32.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Apr 2024 03:32:05 -0700 (PDT) Date: Wed, 24 Apr 2024 12:32:04 +0200 From: Andrew Jones To: Christoph =?utf-8?Q?M=C3=BCllner?= Cc: Charlie Jenkins , Conor Dooley , linux-riscv@lists.infradead.org, kvm-riscv@lists.infradead.org, devicetree@vger.kernel.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, conor.dooley@microchip.com, anup@brainfault.org, atishp@atishpatra.org, robh@kernel.org, krzysztof.kozlowski+dt@linaro.org, conor+dt@kernel.org, heiko@sntech.de, David.Laight@aculab.com, parri.andrea@gmail.com, luxu.kernel@bytedance.com Subject: Re: [PATCH v2 2/6] dt-bindings: riscv: Add Zawrs ISA extension description Message-ID: <20240424-c24522c332fcaced37339045@orel> References: <20240419-8c6af6a169a7aa0b9aa5cac5@orel> <20240419-disdain-litmus-82874cc4872e@spud> <20240421-f63f90c2146deb0695b84441@orel> <20240423-ed9ddb701be1df4a25e29f31@orel> <20240424-0e571b29835610c24409762b@orel> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240424_033209_610507_2C5AD417 X-CRM114-Status: GOOD ( 52.20 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org T24gV2VkLCBBcHIgMjQsIDIwMjQgYXQgMTE6MjM6MDBBTSArMDIwMCwgQ2hyaXN0b3BoIE3DvGxs bmVyIHdyb3RlOgo+IE9uIFdlZCwgQXByIDI0LCAyMDI0IGF0IDk6MzTigK9BTSBBbmRyZXcgSm9u ZXMgPGFqb25lc0B2ZW50YW5hbWljcm8uY29tPiB3cm90ZToKPiA+Cj4gPiBPbiBUdWUsIEFwciAy MywgMjAyNCBhdCAwMzo0Mjo0N1BNIC0wNDAwLCBDaGFybGllIEplbmtpbnMgd3JvdGU6Cj4gPiA+ IE9uIFR1ZSwgQXByIDIzLCAyMDI0IGF0IDAyOjAwOjUzUE0gLTA0MDAsIENoYXJsaWUgSmVua2lu cyB3cm90ZToKPiA+ID4gPiBPbiBUdWUsIEFwciAyMywgMjAyNCBhdCAxMDo0NjowMUFNICswMjAw LCBBbmRyZXcgSm9uZXMgd3JvdGU6Cj4gPiA+ID4gPiBPbiBNb24sIEFwciAyMiwgMjAyNCBhdCAw NjozNjo0NVBNIC0wNDAwLCBDaGFybGllIEplbmtpbnMgd3JvdGU6Cj4gPiA+ID4gPiA+IE9uIFN1 biwgQXByIDIxLCAyMDI0IGF0IDEyOjIwOjAzUE0gKzAyMDAsIEFuZHJldyBKb25lcyB3cm90ZToK PiA+ID4gPiA+ID4gPiBPbiBGcmksIEFwciAxOSwgMjAyNCBhdCAxMjo0MDowMVBNIC0wNDAwLCBD aGFybGllIEplbmtpbnMgd3JvdGU6Cj4gPiA+ID4gPiAuLi4KPiA+ID4gPiA+ID4gPiA+IFdoYXQg d291bGQgYmUgdGhlIHB1cnBvc2Ugb2YgYSB2ZW5kb3IgaW1wbGVtZW50aW5nIFdSUy5OVE8gKGFu ZCBwdXR0aW5nCj4gPiA+ID4gPiA+ID4gPiBpdCBpbiB0aGUgRFQpIHRoYXQgbmV2ZXIgdGVybWlu YXRlcz8gVGhlIHNwZWMgc2F5cyAiVGhlbiBhIHN1YnNlcXVlbnQKPiA+ID4gPiA+ID4gPiA+IFdS Uy5OVE8gaW5zdHJ1Y3Rpb24gd291bGQgY2F1c2UgdGhlIGhhcnQgdG8gdGVtcG9yYXJpbHkgc3Rh bGwgZXhlY3V0aW9uCj4gPiA+ID4gPiA+ID4gPiBpbiBhIGxvdy0gcG93ZXIgc3RhdGUgdW50aWwg YSBzdG9yZSBvY2N1cnMgdG8gdGhlIHJlc2VydmF0aW9uIHNldCBvciBhbgo+ID4gPiA+ID4gPiA+ ID4gaW50ZXJydXB0IGlzIG9ic2VydmVkLiIgV2h5IGlzIHRoaXMgd29yZGluZyBmb3IgV1JTLk5U TyBub3Qgc3VmZmljaWVudAo+ID4gPiA+ID4gPiA+ID4gdG8gYXNzdW1lIHRoYXQgYW4gaW1wbGVt ZW50YXRpb24gb2YgdGhpcyBpbnN0cnVjdGlvbiB3b3VsZCBldmVudHVhbGx5Cj4gPiA+ID4gPiA+ ID4gPiB0ZXJtaW5hdGU/Cj4gPiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+ ID4gV2UgY2FuIGludm9rZSBzbXBfY29uZF9sb2FkX3JlbGF4ZWQoYWRkciwgVkFMIHx8IGFueXRo aW5nX3dlX3dhbnQoKSkuIFRoaXMKPiA+ID4gPiA+ID4gPiBtZWFucyB3ZSBtYXkgbm90IGV4cGVj dCBWQUwgZXZlciB0byBiZSB3cml0dGVuLCB3aGljaCBydWxlcyBvdXQgInVudGlsIGEKPiA+ID4g PiA+ID4gPiBzdG9yZSBvY2N1cnMiLiBBcyBmb3IgImFuIGludGVycnVwdCBpcyBvYnNlcnZlZCIs IHdlIGRvbid0IGtub3cgd2hpY2ggb25lCj4gPiA+ID4gPiA+ID4gdG8gZXhwZWN0IHRvIGFycml2 ZSB3aXRoaW4gYSAicmVhc29uYWJsZSIgYW1vdW50IG9mIHRpbWUuIFdlIG5lZWQgdG8ga25vdwo+ ID4gPiA+ID4gPiA+IHdoaWNoIG9uZShzKSwgc2luY2UsIHdoaWxlIHdycy5udG8gd2lsbCB0ZXJt aW5hdGUgZXZlbiB3aGVuIGludGVycnVwdHMgYXJlCj4gPiA+ID4gPiA+ID4gZ2xvYmFsbHkgZGlz YWJsZWQsIHdlIHN0aWxsIG5lZWQgdG8gaGF2ZSB0aGUgaW50ZXJydXB0KHMpIHdlIGV4cGVjdCB0 byBiZQo+ID4gPiA+ID4gPiA+IGxvY2FsbHkgZW5hYmxlZC4gQW5kLCB0aGUgaW50ZXJydXB0cyBz aG91bGQgYXJyaXZlIGluIGEgInJlYXNvbmFibGUiCj4gPiA+ID4gPiA+ID4gYW1vdW50IG9mIHRp bWUgc2luY2Ugd2Ugd2FudCB0byBwb2xsIGFueXRoaW5nX3dlX3dhbnQoKSBhdCBhICJyZWFzb25h YmxlIgo+ID4gPiA+ID4gPiA+IGZyZXF1ZW5jeS4KPiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+ IFNvLCB3ZSBuZWVkIGZpcm13YXJlIHRvIHByb21pc2UgdG8gZW5hYmxlIGV4Y2VwdGlvbnMgaWYg dGhlcmUgYXJlbid0IGFueQo+ID4gPiA+ID4gPiA+IHN1Y2ggaW50ZXJydXB0cy4gT3IsIHdlIGNv dWxkIHJlcXVpcmUgaGFyZHdhcmUgZGVzY3JpcHRpb25zIHRvIGlkZW50aWZ5Cj4gPiA+ID4gPiA+ ID4gd2hpY2ggaW50ZXJydXB0KHMpIHdvdWxkIGJlIGdvb2QgdG8gaGF2ZSBlbmFibGVkIGJlZm9y ZSBjYWxsaW5nIHdycy5udG8uCj4gPiA+ID4gPiA+ID4gTWF5YmUgdGhlcmUncyBhbHJlYWR5IHNv bWUgd2F5IHRvIGRlc2NyaWJlIHNvbWV0aGluZyBsaWtlIHRoYXQ/Cj4gPiA+ID4gPiA+ID4KPiA+ ID4gPiA+ID4gPiBUaGFua3MsCj4gPiA+ID4gPiA+ID4gZHJldwo+ID4gPiA+ID4gPgo+ID4gPiA+ ID4gPiBBaGggb2theSBJIGFtIGNhdWdodCB1cCBub3cuIFNvIHRoZSB3b3JkaW5nIHdlIGFyZSBs b29raW5nIGF0IGluIHRoZQo+ID4gPiA+ID4gPiBzcGVjIGlzOgo+ID4gPiA+ID4gPgo+ID4gPiA+ ID4gPiAiV2hlbiBleGVjdXRpbmcgaW4gVlMgb3IgVlUgbW9kZSwgaWYgdGhlIFZUVyBiaXQgaXMg c2V0IGluIGhzdGF0dXMsIHRoZQo+ID4gPiA+ID4gPiBUVyBiaXQgaW4gbXN0YXR1cyBpcyBjbGVh ciwgYW5kIHRoZSBXUlMuTlRPIGRvZXMgbm90IGNvbXBsZXRlIHdpdGhpbiBhbgo+ID4gPiA+ID4g PiBpbXBsZW1lbnRhdGlvbi1zcGVjaWZpYyBib3VuZGVkIHRpbWUgbGltaXQsIHRoZSBXUlMuTlRP IGluc3RydWN0aW9uIHdpbGwKPiA+ID4gPiA+ID4gY2F1c2UgYSB2aXJ0dWFsIGluc3RydWN0aW9u IGV4Y2VwdGlvbi4iCj4gPiA+ID4gPgo+ID4gPiA+ID4gVGhhdCdzIHdoYXQgdGhlIGh5cGVydmlz b3Igc2hvdWxkIHByb21pc2UgdG8gZG8gd2hlbiB0aGVyZSdzIG5vIG90aGVyCj4gPiA+ID4gPiBn dWFyYW50ZWUgb2Ygd3JzLm50byB0ZXJtaW5hdGluZyAoYnV0IHRoZSBoeXBlcnZpc29yIGxpa2Vs eSB3YW50cyB0bwo+ID4gPiA+ID4gYW55d2F5IHNpbmNlIGl0IHdhbnRzIGd1ZXN0cyB0byB0cmFw IG9uIHdycy5udG8gaW4gb3JkZXIgdG8gcG90ZW50aWFsbHkKPiA+ID4gPiA+IHNjaGVkdWxlIHRo ZSBsb2NrIGhvbGRpbmcgVkNQVSkuIFRoZSBmaXJtd2FyZSBvZiB0aGUgaG9zdCBzaG91bGQgbGlr ZXdpc2UKPiA+ID4gPiA+IHByb21pc2UgdG8gc2V0IG1zdGF0dXMuVFcgd2hlbiB0aGVyZSdzIG5v IGd1YXJhbnRlZSBvZiB3cnMubnRvCj4gPiA+ID4gPiB0ZXJtaW5hdGluZywgYnV0IHRoYXQncyBs aWtlbHkgX25vdF8gc29tZXRoaW5nIGl0IG5vcm1hbGx5IHdvdWxkIHdhbnQgdG8KPiA+ID4gPiA+ IGRvLCBzbyBob3BlZnVsbHkgdGhlcmUgd2lsbCBhbHdheXMgYmUgaW1wbGVtZW50YXRpb24tc3Bl Y2lmaWMgIm90aGVyCj4gPiA+ID4gPiByZWFzb25zIiB3aGljaCBndWFyYW50ZWUgdGVybWluYXRp b24uCj4gPiA+ID4gPgo+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiBXaXRoIHRoZSBjb25jZXJuIGJl aW5nIHRoYXQgaXQgaXMgcG9zc2libGUgZm9yICJpbXBsZW1lbnRhdGlvbi1zcGVjaWZpYwo+ID4g PiA+ID4gPiBib3VuZGVkIHRpbWUgbGltaXQiIHRvIGJlIGluZmluaXRlL25ldmVyIHRpbWVzIG91 dCwKPiA+ID4gPiA+Cj4gPiA+ID4gPiBUaGUgaW1wbGVtZW50YXRpb24tZGVmaW5lZCBzaG9ydCB0 aW1lb3V0IGNhbm5vdCBiZSBpbmZpbml0ZSwgYnV0IGl0IG9ubHkKPiA+ID4gPiA+IGFwcGxpZXMg dG8gd3JzLnN0by4gV2hpbGUgdXNpbmcgd3JzLnN0byB3b3VsZCByZWxpZXZlIHRoZSBjb25jZXJu LCBpdAo+ID4gPiA+ID4gY2Fubm90IGJlIGNvbmZpZ3VyZWQgdG8gcmFpc2UgZXhjZXB0aW9ucywg d2hpY2ggbWVhbnMgaXQncyBub3QgdXNlZnVsIGluCj4gPiA+ID4gPiBndWVzdHMuIElmIHdlIHdh bnQgdG8gdXNlIHdycy5zdG8gaW4gaG9zdHMgYW5kIHdycy5udG8gaW4gZ3Vlc3RzIHRoZW4gd2UK PiA+ID4gPiA+IG5lZWQgYSBwYXJhdmlydCBjaGFubmVsIHdoaWNoIGFsbG93cyBhbiAiZW5saWdo dGVuZWQiIGd1ZXN0IHRvIGRldGVybWluZQo+ID4gPiA+ID4gdGhhdCBpdCBpcyBhIGd1ZXN0IGFu ZCB0aGF0IHRoZSBoeXBlcnZpc29yIGhhcyBjb25maWd1cmVkIHdycy5udG8gdG8KPiA+ID4gPiA+ IHRyYXAsIHdoaWNoIHRoZW4gaW5kaWNhdGVzIGl0J3MgYSBnb29kIGlkZWEgdG8gcGF0Y2ggd3Jz LnN0byB0byB3cnMubnRvLgo+ID4gPiA+ID4gQnV0LCBhZGRpbmcgcGFyYXZpcnQgc3R1ZmYgc2hv dWxkIGJlIGF2b2lkZWQgd2hlbmV2ZXIgcG9zc2libGUgc2luY2UgaXQKPiA+ID4gPiA+IGFkZHMg Y29tcGxleGl0eSB3ZSdkIHJhdGhlciBub3QgbWFpbnRhaW4uCj4gPiA+ID4gPgo+ID4gPiA+Cj4g PiA+ID4gVGhhdCBzdGlsbCB3b3VsZG4ndCBzb2x2ZSB0aGlzIGlzc3VlLCBiZWNhdXNlIHRoZSB3 cnMubnRvIGd1ZXN0IG1heSBzdGlsbAo+ID4gPiA+IG5ldmVyIHdha2V1cCBpbiB0aGUgaW1wbGVt ZW50YXRpb24tc3BlY2lmaWMgd2F5Pwo+ID4KPiA+IFRoZSBwYXJhdmlydCBhcHByb2FjaCBkb2Vz IHNvbHZlIGl0LCBiZWNhdXNlIHdycy5udG8gaXMgc3BlY2lmaWVkIHRvIHJhaXNlCj4gPiBleGNl cHRpb25zIGFmdGVyIGFuIGltcGxlbWVudGF0aW9uLXNwZWNpZmljIGJvdW5kZWQgdGltZSBsaW1p dCB3aGVuIHRoZQo+ID4gaHlwZXJ2aXNvciBzZXRzIGhzdGF0dXMuVlRXLgo+ID4KPiA+ID4KPiA+ ID4gVGhpbmtpbmcgYWJvdXQgdGhpcyBhIGJpdCBtb3JlLCB0aGlzIGlzIGEgcGVyZm9ybWFuY2Ug cGVuYWx0eSBhbmQgbm90IGEKPiA+ID4gY29ycmVjdG5lc3MgaXNzdWUuCj4gPgo+ID4gSXQncyBp bmNvcnJlY3QgdG8gaGF2ZSBhIGRlc2lnbiB0aGF0IGlzIGxpa2VseSB0byByZXN1bHQgaW4gYmFk Cj4gPiBwZXJmb3JtYW5jZS4KPiA+Cj4gPiA+IFRoaXMgbGluZSBpcyBhbiBvcHRpbWl6YXRpb24g dGhhdCBhbGxvd3MgdGhlIGxvY2sKPiA+ID4gaG9sZGVyIHRvIGp1bXAgdGhlIHF1ZXVlIGlmIHRo ZSBob2xkZXIgZGlyZWN0bHkgaW4gZnJvbnQgaXMgYSBwcmVlbXB0ZWQKPiA+ID4gdmNwdS4gRXZl bnR1YWxseSB0aGUgdmNwdSB3aWxsIGJlIHNjaGVkdWxlZCBhZ2FpbiBhbmQgZ2l2ZSB1cCB0aGUg bG9jay4KPiA+ID4gU28gYW4gaW1wbGVtZW50YXRpb24gb2YgV1JTLk5UTyB0aGF0IGRvZXMgbm90 IGhhdmUgdGhlCj4gPiA+ICJpbXBsZW1lbnRhdGlvbi1zcGVjaWZpYyBib3VuZGVkIHRpbWUgbGlt aXQiIHRoYXQgdGhlIHNwZWMgY2FsbHMgb3V0IGZvcgo+ID4gPiBXUlMuTlRPIHRvIHJhaXNlIGEg dmlydHVhbCBpbnN0cnVjdGlvbiBleGNlcHRpb24sIHdvdWxkIHN0aWxsIGZ1bmN0aW9uLAo+ID4g PiBqdXN0IHNsb3dlci4KPiA+Cj4gPiBUaGUgcHJvYmxlbSBpc24ndCBzcGVjaWZpYyB0byB2aXJ0 dWFsaXphdGlvbi4gVGhlIHByb2JsZW0gaXMgdXNpbmcgd3JzLm50bwo+ID4gd2hlbiBpdCBoYXMg bm90IGJlZW4gY29uZmlndXJlZCB0byByYWlzZSBleGNlcHRpb25zIGFuZCB0aGVyZSBhcmUgbm90 IGFueQo+ID4gb3RoZXIgZ3VhcmFudGVlZCB0ZXJtaW5hdGlvbiBldmVudHMgKG90aGVyIHRoYW4g dGhlIHJlc2VydmF0aW9uIHNldAo+ID4gYmVjb21pbmcgaW52YWxpZCkuIFdoaWxlIHRoZSBwYXJh dmlydCBzb2x1dGlvbiBpcyB2aXJ0dWFsaXphdGlvbiBzcGVjaWZpYywKPiA+IGl0IHdvcmtzLCBi ZWNhdXNlIHdlIHdvdWxkIHRoZW4gb25seSB1c2Ugd3JzLm50byB3aGVuIHdlIGtub3cgd2UgY2Fu LCBhbmQKPiA+IG90aGVyd2lzZSB1c2Ugd3JzLnN0by4gQnV0LCBhcyBJIHNhaWQsIEknZCByYXRo ZXIgbm90IGhhdmUgYSBwYXJhdmlydAo+ID4gc29sdXRpb24uCj4gCj4gQW5kcmV3LCBpdCB3b3Vs ZCBiZSBncmVhdCBpZiB5b3UgY291bGQgc3VtbWFyaXplIHRoaXMgZmluZGluZyB0byB0aGUKPiBz cGVjIGF1dGhvcnMuCj4gTWF5YmUgYSBub24tbm9ybWF0aXZlIHRleHQgYWRkZWQgdG8gdGhlIHNw ZWMgKHRoYXQgcmFpc2VzIGF3YXJlbmVzcwo+IGZvciB0aGUgaXNzdWUKClN1cmUsIEknbGwgd3Jp dGUgc29tZXRoaW5nIHVwIHBvaW50aW5nIG91dCB0aGUgY29uY2VybiB3aXRoIHdycy5udG8gYW5k CnBvc3QgaXQgdG8gYSBmZXcgUlZJIG1haWxpbmcgbGlzdHMuCgo+IGFuZCBwcm92aWRlcyBhIGd1 aWRlbGluZSB0byBhdm9pZCBpdCkgY291bGQgcmVkdWNlIHRoZSByaXNrIG9mIHRyaWdnZXJpbmcK PiB0aGUgaXNzdWUgb24gcmVhbCBIVy4gVGhpcyBtaWdodCBiZSBlbm91Z2gganVzdGlmaWNhdGlv biB0byB1c2UgV1JTLk5UTy4KPiAKPiBJZiBXUlMuTlRPIGlzIGNvbnNpZGVyZWQgYXMgbm90IHJl bGlhYmxlIGVub3VnaCB0byB3YWtlIHVwIGFuZCB0aGVyZWZvcmUgY2F1c2luZwo+IHBlcmZvcm1h bmNlIGlzc3VlcyBvciBDUFUgc3RhbGxzIGlmIHVzZWQgZm9yIHRoZSBzcGluIGxvY2sgb3B0aW1p emF0aW9uLAo+IGl0IG1pZ2h0IGJlIGFsc28gcmVhc29uYWJsZSB0byB1c2UgV1JTLlNUTyBpbnN0 ZWFkLgo+IFRoZSBjb3N0IG9mIGhhdmluZyB0b28gbWFueSB3YWtldXBzIHNlZW1zIG11Y2ggbW9y ZSBhY2NlcHRhYmxlIHRoYW4KPiBhIHN0YWxsZWQgQ1BVLgoKd3JzLnN0byBpcyByZWFzb25hYmxl IHRvIHVzZSBpbiBhbGwgY2FzZXMsIHNpbmNlIHRvbyBtYW55IHdha2V1cHMgaXNuJ3QKYSBjb25j ZXJuLiBCdXQsIHdlIGNhbiBkbyBiZXR0ZXIgaW4gdGhlIHZpcnR1YWxpemF0aW9uIGNhc2Ugd2l0 aCB3cnMubnRvLAp3aGVyZSB0aGUgaHlwZXJ2aXNvciBjYW4gZ2V0IGludm9sdmVkLCBzbywgdG8g YXZvaWQgcGFyYXZpcnQgc3R1ZmYsIHdlJ2QKbGlrZSB0byBiZSBhYmxlIHRvIGFsd2F5cyB1c2Ug d3JzLm50by4KClRoYW5rcywKZHJldwoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX18KbGludXgtcmlzY3YgbWFpbGluZyBsaXN0CmxpbnV4LXJpc2N2QGxpc3Rz LmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVhZC5vcmcvbWFpbG1hbi9saXN0aW5m by9saW51eC1yaXNjdgo= From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-lf1-f42.google.com (mail-lf1-f42.google.com [209.85.167.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4851F41C63 for ; Wed, 24 Apr 2024 10:32:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.42 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713954730; cv=none; b=goBM9muCnRT9MXeg7s0J0HiBNDY04erSbSudETMyHlhisxsRJ3wQPzQmBuDwiDg+ZUuP4LyPs2MSdkno13FQLPO1/eNtXGoVKZfjOxF7uqayE8/rVWz4pkXTbtJLe082W9m2797vzrp9c+9Lt4i8G1NcCmxFnOcberqd0C+2+20= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713954730; c=relaxed/simple; bh=o1xKJQ2Gp2Bmhyjj073820hhH3G67LFsd+0t8zBegAo=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=fVAXRdC1SOOXP20ofherO/FXhq3EkzSpoT3Z4FtT1zjpBRWq+5Oh+xPOMNAkw28rgIHNjRAMFxwZh/J1bzwzh0lwAZx/RvEn8BqMJ+SSWAu56i1w+pzfBcx8tC3mflB0IGC2IasJlPIn9Mar2PtGEWkLRjdyuNm1Snplh8uSw7c= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ventanamicro.com; spf=pass smtp.mailfrom=ventanamicro.com; dkim=pass (2048-bit key) header.d=ventanamicro.com header.i=@ventanamicro.com header.b=B4DUV3o4; arc=none smtp.client-ip=209.85.167.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ventanamicro.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ventanamicro.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ventanamicro.com header.i=@ventanamicro.com header.b="B4DUV3o4" Received: by mail-lf1-f42.google.com with SMTP id 2adb3069b0e04-5176f217b7bso11297655e87.0 for ; Wed, 24 Apr 2024 03:32:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ventanamicro.com; s=google; t=1713954726; x=1714559526; darn=vger.kernel.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=69da6UXz2pFiDL31edeqzPN31G6ffxNroQ0Z2N+KdsQ=; b=B4DUV3o4jMBGDBlFUWjJAUBwPtLKItec1XX1RH1aUmw9nqpbxHU9gvPLYDIE32LJ0F r9TLk++neaU4MItLQMqMHja8SLVa15kzCTVharAf3M0oPsZyA0hduPzDXJR55K5d+EYW wvgRSiQbWwUTVXdOHFYFbLi9CVAli0TzRP6ggsm52UgX+AB/xZq0kDXrmo+DYHU2sW8P zuy9fXFbfOlWwZpuzD4ueJxvM5m6QkjWclcMuaEfEoruuTaGbKIfdd4BWipx+f5a7y61 +EqMa6ezJjdjOqxCDDTlkbiWFPQTAlh9TRgAqInJVLm+8A47n1EM8BaIS6JGVYoyrYdW huTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713954726; x=1714559526; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=69da6UXz2pFiDL31edeqzPN31G6ffxNroQ0Z2N+KdsQ=; b=ihI6t7y7LzCQjF88sS3uBt8znAOq95RJgo+gQ26zJzpEk9fAL1mjbaAroNdTIp2jpf mwPJHxFS2/cqSnl2jSXtAe8GlM7LTZlAckzMtgIi8j/Fc3gNVaUOgLGff6LwQE5FJO/8 Xv286K7FGqHTqPSzt92VgFnGI9qOi7hoEdOpdoC7z4TGmoZv5Tg/3wdcBRqUpF3sixDg SmP/BULM3hkENM1a9hhtgxmBcW9JFXhWebU1J+Gi+zJQrdNg1PUqMLMidYoPdDJs2eCB e6VV1sF0tYiZxTSYK1Q5SdSULqM41udxzRVhRTfUu383GRTELdOz2AQNeByK1vVkr469 Ng+w== X-Forwarded-Encrypted: i=1; AJvYcCX7+vSzxc/ihwuzdQ3Kq9WwzLGqMYkPLDtNCLbbyqUBiKl+mb7vC+wgDi3EqNI+OdtI34Dpl0sTND5B4I9KEqHiNYKURHpKUXp0Wg== X-Gm-Message-State: AOJu0YztNAwuJOyOuhN0R4h2RZr/j4l6mw7dsNtNz+h+h26QohMZ//sO 40OG0LEspa069RmRN4cG2Bcx6MdBiZzUm1wepLRDVE7aIKvkEQkdF6nToVIdX0Y= X-Google-Smtp-Source: AGHT+IEcj0tdrrmR8cPhcTUlqc6Z+F7kWLT0j9ptexs5NM6zhH3y23MG/ibcIMnGQ8n53KYceeEqdw== X-Received: by 2002:a05:6512:793:b0:51a:affe:252e with SMTP id x19-20020a056512079300b0051aaffe252emr1747408lfr.37.1713954726126; Wed, 24 Apr 2024 03:32:06 -0700 (PDT) Received: from localhost (2001-1ae9-1c2-4c00-20f-c6b4-1e57-7965.ip6.tmcz.cz. [2001:1ae9:1c2:4c00:20f:c6b4:1e57:7965]) by smtp.gmail.com with ESMTPSA id z2-20020a17090665c200b00a55bbc4fd16sm3274471ejn.48.2024.04.24.03.32.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Apr 2024 03:32:05 -0700 (PDT) Date: Wed, 24 Apr 2024 12:32:04 +0200 From: Andrew Jones To: Christoph =?utf-8?Q?M=C3=BCllner?= Cc: Charlie Jenkins , Conor Dooley , linux-riscv@lists.infradead.org, kvm-riscv@lists.infradead.org, devicetree@vger.kernel.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, conor.dooley@microchip.com, anup@brainfault.org, atishp@atishpatra.org, robh@kernel.org, krzysztof.kozlowski+dt@linaro.org, conor+dt@kernel.org, heiko@sntech.de, David.Laight@aculab.com, parri.andrea@gmail.com, luxu.kernel@bytedance.com Subject: Re: [PATCH v2 2/6] dt-bindings: riscv: Add Zawrs ISA extension description Message-ID: <20240424-c24522c332fcaced37339045@orel> References: <20240419-8c6af6a169a7aa0b9aa5cac5@orel> <20240419-disdain-litmus-82874cc4872e@spud> <20240421-f63f90c2146deb0695b84441@orel> <20240423-ed9ddb701be1df4a25e29f31@orel> <20240424-0e571b29835610c24409762b@orel> Precedence: bulk X-Mailing-List: devicetree@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: On Wed, Apr 24, 2024 at 11:23:00AM +0200, Christoph Müllner wrote: > On Wed, Apr 24, 2024 at 9:34 AM Andrew Jones wrote: > > > > On Tue, Apr 23, 2024 at 03:42:47PM -0400, Charlie Jenkins wrote: > > > On Tue, Apr 23, 2024 at 02:00:53PM -0400, Charlie Jenkins wrote: > > > > On Tue, Apr 23, 2024 at 10:46:01AM +0200, Andrew Jones wrote: > > > > > On Mon, Apr 22, 2024 at 06:36:45PM -0400, Charlie Jenkins wrote: > > > > > > On Sun, Apr 21, 2024 at 12:20:03PM +0200, Andrew Jones wrote: > > > > > > > On Fri, Apr 19, 2024 at 12:40:01PM -0400, Charlie Jenkins wrote: > > > > > ... > > > > > > > > What would be the purpose of a vendor implementing WRS.NTO (and putting > > > > > > > > it in the DT) that never terminates? The spec says "Then a subsequent > > > > > > > > WRS.NTO instruction would cause the hart to temporarily stall execution > > > > > > > > in a low- power state until a store occurs to the reservation set or an > > > > > > > > interrupt is observed." Why is this wording for WRS.NTO not sufficient > > > > > > > > to assume that an implementation of this instruction would eventually > > > > > > > > terminate? > > > > > > > > > > > > > > > > > > > > > > We can invoke smp_cond_load_relaxed(addr, VAL || anything_we_want()). This > > > > > > > means we may not expect VAL ever to be written, which rules out "until a > > > > > > > store occurs". As for "an interrupt is observed", we don't know which one > > > > > > > to expect to arrive within a "reasonable" amount of time. We need to know > > > > > > > which one(s), since, while wrs.nto will terminate even when interrupts are > > > > > > > globally disabled, we still need to have the interrupt(s) we expect to be > > > > > > > locally enabled. And, the interrupts should arrive in a "reasonable" > > > > > > > amount of time since we want to poll anything_we_want() at a "reasonable" > > > > > > > frequency. > > > > > > > > > > > > > > So, we need firmware to promise to enable exceptions if there aren't any > > > > > > > such interrupts. Or, we could require hardware descriptions to identify > > > > > > > which interrupt(s) would be good to have enabled before calling wrs.nto. > > > > > > > Maybe there's already some way to describe something like that? > > > > > > > > > > > > > > Thanks, > > > > > > > drew > > > > > > > > > > > > Ahh okay I am caught up now. So the wording we are looking at in the > > > > > > spec is: > > > > > > > > > > > > "When executing in VS or VU mode, if the VTW bit is set in hstatus, the > > > > > > TW bit in mstatus is clear, and the WRS.NTO does not complete within an > > > > > > implementation-specific bounded time limit, the WRS.NTO instruction will > > > > > > cause a virtual instruction exception." > > > > > > > > > > That's what the hypervisor should promise to do when there's no other > > > > > guarantee of wrs.nto terminating (but the hypervisor likely wants to > > > > > anyway since it wants guests to trap on wrs.nto in order to potentially > > > > > schedule the lock holding VCPU). The firmware of the host should likewise > > > > > promise to set mstatus.TW when there's no guarantee of wrs.nto > > > > > terminating, but that's likely _not_ something it normally would want to > > > > > do, so hopefully there will always be implementation-specific "other > > > > > reasons" which guarantee termination. > > > > > > > > > > > > > > > > > With the concern being that it is possible for "implementation-specific > > > > > > bounded time limit" to be infinite/never times out, > > > > > > > > > > The implementation-defined short timeout cannot be infinite, but it only > > > > > applies to wrs.sto. While using wrs.sto would relieve the concern, it > > > > > cannot be configured to raise exceptions, which means it's not useful in > > > > > guests. If we want to use wrs.sto in hosts and wrs.nto in guests then we > > > > > need a paravirt channel which allows an "enlightened" guest to determine > > > > > that it is a guest and that the hypervisor has configured wrs.nto to > > > > > trap, which then indicates it's a good idea to patch wrs.sto to wrs.nto. > > > > > But, adding paravirt stuff should be avoided whenever possible since it > > > > > adds complexity we'd rather not maintain. > > > > > > > > > > > > > That still wouldn't solve this issue, because the wrs.nto guest may still > > > > never wakeup in the implementation-specific way? > > > > The paravirt approach does solve it, because wrs.nto is specified to raise > > exceptions after an implementation-specific bounded time limit when the > > hypervisor sets hstatus.VTW. > > > > > > > > Thinking about this a bit more, this is a performance penalty and not a > > > correctness issue. > > > > It's incorrect to have a design that is likely to result in bad > > performance. > > > > > This line is an optimization that allows the lock > > > holder to jump the queue if the holder directly in front is a preempted > > > vcpu. Eventually the vcpu will be scheduled again and give up the lock. > > > So an implementation of WRS.NTO that does not have the > > > "implementation-specific bounded time limit" that the spec calls out for > > > WRS.NTO to raise a virtual instruction exception, would still function, > > > just slower. > > > > The problem isn't specific to virtualization. The problem is using wrs.nto > > when it has not been configured to raise exceptions and there are not any > > other guaranteed termination events (other than the reservation set > > becoming invalid). While the paravirt solution is virtualization specific, > > it works, because we would then only use wrs.nto when we know we can, and > > otherwise use wrs.sto. But, as I said, I'd rather not have a paravirt > > solution. > > Andrew, it would be great if you could summarize this finding to the > spec authors. > Maybe a non-normative text added to the spec (that raises awareness > for the issue Sure, I'll write something up pointing out the concern with wrs.nto and post it to a few RVI mailing lists. > and provides a guideline to avoid it) could reduce the risk of triggering > the issue on real HW. This might be enough justification to use WRS.NTO. > > If WRS.NTO is considered as not reliable enough to wake up and therefore causing > performance issues or CPU stalls if used for the spin lock optimization, > it might be also reasonable to use WRS.STO instead. > The cost of having too many wakeups seems much more acceptable than > a stalled CPU. wrs.sto is reasonable to use in all cases, since too many wakeups isn't a concern. But, we can do better in the virtualization case with wrs.nto, where the hypervisor can get involved, so, to avoid paravirt stuff, we'd like to be able to always use wrs.nto. Thanks, drew