From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B461CC02181 for ; Wed, 22 Jan 2025 11:49:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:MIME-Version: References:In-Reply-To:Subject:Cc:To:From:Message-ID:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=xXO+eeJwoYDq0+ajeKon9Edlp30dkoKqCF75LDwgv34=; b=QtAl7FH4ZLbj0BlJQ5hEkG8wpE m0A29SntWCbsJrlODCPLb1lWqo4W+5oOp1C80aBxa1DHqD6tRk3dYfBMzIfzivQ8vMkh3HHgYnJjf o1GTHBa/01obVq4Q7Sk/QHxNYDpYWb+xzhDqSPq5IIEAJwi44cX/BNTXlFoWZgGWJVll3sQpOyAEp npRNu2ydWshsjN3DbyKbpsBLOaQz0ujqT1Ye2d2sahZd+6AM2nPA68auDPEp8rJGpGqKR+fSyK32r cg7uwnfp5RCMTYjAmPQRUAbBu2JwsT/Qsh8k0aCOOC3/zZOoXeMTeEJ+eqKN8aUnE1hZqw2qQUYwV kKy0NWgA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1taZEF-0000000A5bm-44Zz; Wed, 22 Jan 2025 11:49:03 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1taZBr-0000000A5E6-1lRg for linux-arm-kernel@lists.infradead.org; Wed, 22 Jan 2025 11:46:36 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 283835C581B; Wed, 22 Jan 2025 11:45:54 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3C966C4CEE3; Wed, 22 Jan 2025 11:46:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1737546394; bh=J/3Uv76izXM3DkFhHTfwqy9MkTyaRA0S84/pyF38QyA=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=S8p9UURgZao9lh+1de2TLrojnarOYcZZl77vvfSAh0m7M/vUtsPqLrKrMetuZFpkr qItQoglvTuDwQfHpB3FRZqI+b05vv2uKE1KyWH4AzZwFDciv6bO3MDG48Tp8Hr4q1e SVPUUX0g1/HrHCcPIVxB+nqbHWMOWinv0C5+08ABgHeebXz4qKg2O2OWrBbVpQInUZ sYGY/47MMVqT3AV6elgSgSy0tQ/gnRJlbZmHotRHsT8mN4PbJzL65Jm4taOOrmlLab Kz0Jw3aBn0qjZN30BcFLdNEhiHhWHf30ZKW9J2uZ7TBHibgWkkvMCtWZ/XgOMdfb69 K3gChKxlEK74A== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1taZBn-00ENrU-Gu; Wed, 22 Jan 2025 11:46:32 +0000 Date: Wed, 22 Jan 2025 11:46:31 +0000 Message-ID: <86plkful48.wl-maz@kernel.org> From: Marc Zyngier To: Mark Rutland Cc: linux-arm-kernel@lists.infradead.org, broonie@kernel.org, catalin.marinas@arm.com, eauger@redhat.com, fweimer@redhat.com, jeremy.linton@arm.com, oliver.upton@linux.dev, pbonzini@redhat.com, stable@vger.kernel.org, wilco.dijkstra@arm.com, will@kernel.org Subject: Re: [PATCH] KVM: arm64/sve: Ensure SVE is trapped after guest exit In-Reply-To: References: <20250121100026.3974971-1-mark.rutland@arm.com> <86r04wv2fv.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.4 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: mark.rutland@arm.com, linux-arm-kernel@lists.infradead.org, broonie@kernel.org, catalin.marinas@arm.com, eauger@redhat.com, fweimer@redhat.com, jeremy.linton@arm.com, oliver.upton@linux.dev, pbonzini@redhat.com, stable@vger.kernel.org, wilco.dijkstra@arm.com, will@kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250122_034635_548287_BB98C64D X-CRM114-Status: GOOD ( 51.80 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, 21 Jan 2025 15:37:13 +0000, Mark Rutland wrote: > > On Tue, Jan 21, 2025 at 11:20:04AM +0000, Marc Zyngier wrote: > > Hi Mark, > > [...] > > > diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c > > > index 8c4c1a2186cc5..e4053a90ed240 100644 > > > --- a/arch/arm64/kernel/fpsimd.c > > > +++ b/arch/arm64/kernel/fpsimd.c > > > @@ -1711,8 +1711,24 @@ void fpsimd_kvm_prepare(void) > > > */ > > > get_cpu_fpsimd_context(); > > > > > > - if (test_and_clear_thread_flag(TIF_SVE)) { > > > - sve_to_fpsimd(current); > > > + if (!test_thread_flag(TIF_FOREIGN_FPSTATE) && > > > + test_and_clear_thread_flag(TIF_SVE)) { > > > + sve_user_disable(); > > > > I'm pretty happy with this fix. However... > > > > > + > > > + /* > > > + * The KVM hyp code doesn't set fp_type when saving the host's > > > + * FPSIMD state. Set fp_type here in case the hyp code saves > > > + * the host state. > > > > Should KVM do that? The comment seems to indicate that this is > > papering over yet another bug... > > Yes; really this should be done at hyp (and at that point, hyp could > actually save the entire host SVE state), but that's a larger change and > more painful for backporting, which is why I didn't go that route. I'm > happy to go try to fix hyp to do that, or I can make the comment more > explicit that this is a bodge, if that's all you're after? > > Alternatively, we could take the large hammer approach and always save > and unbind the host state prior to entering the guest, so that hyp > doesn't need to save anything. An unconditional call to > fpsimd_save_and_flush_cpu_state() would suffice, and that'd also > implicitly fix the SME issue below. I think I'd rather see that. Even if that costs us a few hundred cycles on vcpu_load(), I would take that any time over the current fragile/broken behaviour. > > > > + * > > > + * If hyp code does not save the host state, then the host > > > + * state remains live on the CPU and saved fp_type is > > > + * irrelevant until it is overwritten by a later call to > > > + * fpsimd_save_user_state(). > > > > I'm not sure I understand this. If fp_type is irrelevant, surely it is > > *forever* irrelevant, not until something else happens. Or am I > > missing something? > > Sorry, this was not very clear. > > What this is trying to say is that *while the state is live on a CPU* > fp_type is irrelevant, and it's only meaningful when saving/restoring > state. As above, the only reason to set it here is so that *if* hyp > saves and unbinds the state, fp_type will accurately describe what the > hyp code saved. > > The key thing is that there are two possibilities: > > (1) The guest doesn't use FPSIMD/SVE, and no trap is taken to save the > host state. In this case, fp_type is not consumed before the next > time state has to be written back to memory (the act of which will > set fp_type). > > So in this case, setting fp_type is redundant but benign. > > (2) The guest *does* use FPSIMD/SVE, and a trap is taken to hyp to save > the host state. In this case the hyp code will save the task's > FPSIMD state to task->thread.uw.fpsimd_state, but will not update > task->thread.fp_type accordingly, and: > > * If fp_type happened to be FP_STATE_FPSIMD, all is good and a later > restore will load the state saved by the hyp code. > > * If fp_type happened to be FP_STATE_SVE, a later restore will load > stale state from task->thread.sve_state. > > ... does that make sense? It does now, thanks. But with your above alternative suggestion, this becomes completely moot, right? > > > > + * > > > + * This is *NOT* sufficient when CONFIG_ARM64_SME=y, where > > > + * fp_type can be FP_STATE_SVE regardless of TIF_SVE. > > > + */ > > > + BUILD_BUG_ON(IS_ENABLED(CONFIG_ARM64_SME)); > > > > I'd rather not have this build-time failure, as this is very likely to > > annoy a lot of people. Instead, just make SME unselectable with KVM: > > I'm happy to change this, but FWIW I'd used BUILD_BUG() here because it > kept that associated with the comment and logic, and because we disabled > SME in commit: > > 81235ae0c846e1fb4 ("arm64: Kconfig: Make SME depend on BROKEN for now)" > > ... which was CC'd stable, and so this *shouldn't* blow up on anything > with that commit. My experience is that people do set CONFIG_BROKEN, and don't expect the kernel to fail to compile -- they "only" expect it to misbehave. > > So I can: > > (a) Add the dependency, as you suggest. > > (b) Leave that as-is. > > (c) Solve this in a different way so that we don't need a BUILD_BUG() or > dependency. e.g. fix the SME case at the same time, at the cost of > possibly needing to do a bit more work when backporting. > > ... any preference? My preference would be on (c), if at all possible. My understanding is now that the fpsimd_save_and_flush_cpu_state() approach solves all of these problems, at the expense of a bit of overhead. Did I get that correctly? Thanks, M. -- Without deviation from the norm, progress is not possible.