From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8D09AEC5 for ; Wed, 5 Apr 2023 11:59:23 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1B066C433EF; Wed, 5 Apr 2023 11:59:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1680695963; bh=GZfo/elhf7vPGUg+V1tZjMmVivCRFDcBOJg/TBwpr8Y=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=MHf8TjstrquWhrQgffZQpd0nNoqXPA6aeWQbq8PGD58GyFJa8myko+hwtaQMj5NiH 9p/BxPDNT9Hs9jqMZRENhqkjdEyJrNUT2Tive8EJ0EpPzvKxowyypKGgBrdeeQatqH 7+5m+vaIwaJNAwcBdfFkZcmjZ9bPNKXD+hexGH8YnvbywQqMibrcKApWjvPa275Zm9 L4MxpiQ0kc/yYjk+HWgiYprmONPmF8rW6H/d0+DdMzgFHTZcswOf78WCaNyaJer61q RLL9dVMbL1XDiQQTQfY2FV4UzurlTd4l1Sg8iirhm8TxuLCvoy8cCoQA5kyaoLZeEC OyynCzrvibCRw== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1pk1nM-005y7n-Q4; Wed, 05 Apr 2023 12:59:20 +0100 Date: Wed, 05 Apr 2023 12:59:20 +0100 Message-ID: <86pm8iv8tj.wl-maz@kernel.org> From: Marc Zyngier To: Oliver Upton Cc: kvmarm@lists.linux.dev, kvm@vger.kernel.org, Paolo Bonzini , James Morse , Suzuki K Poulose , Zenghui Yu , Sean Christopherson , Salil Mehta Subject: Re: [PATCH v3 08/13] KVM: arm64: Add support for KVM_EXIT_HYPERCALL In-Reply-To: <87o7o26aty.wl-maz@kernel.org> References: <20230404154050.2270077-1-oliver.upton@linux.dev> <20230404154050.2270077-9-oliver.upton@linux.dev> <87o7o26aty.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/28.2 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: oliver.upton@linux.dev, kvmarm@lists.linux.dev, kvm@vger.kernel.org, pbonzini@redhat.com, james.morse@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, seanjc@google.com, salil.mehta@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false On Wed, 05 Apr 2023 08:35:05 +0100, Marc Zyngier wrote: >=20 > On Tue, 04 Apr 2023 16:40:45 +0100, > Oliver Upton wrote: > >=20 > > In anticipation of user hypercall filters, add the necessary plumbing to > > get SMCCC calls out to userspace. Even though the exit structure has > > space for KVM to pass register arguments, let's just avoid it altogether > > and let userspace poke at the registers via KVM_GET_ONE_REG. > >=20 > > This deliberately stretches the definition of a 'hypercall' to cover > > SMCs from EL1 in addition to the HVCs we know and love. KVM doesn't > > support EL1 calls into secure services, but now we can paint that as a > > userspace problem and be done with it. > >=20 > > Finally, we need a flag to let userspace know what conduit instruction > > was used (i.e. SMC vs. HVC). > >=20 > > Signed-off-by: Oliver Upton > > --- > > Documentation/virt/kvm/api.rst | 18 ++++++++++++++++-- > > arch/arm64/include/uapi/asm/kvm.h | 4 ++++ > > arch/arm64/kvm/handle_exit.c | 4 +++- > > arch/arm64/kvm/hypercalls.c | 16 ++++++++++++++++ > > 4 files changed, 39 insertions(+), 3 deletions(-) > >=20 > > diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/ap= i.rst > > index 9b01e3d0e757..9497792c4ee5 100644 > > --- a/Documentation/virt/kvm/api.rst > > +++ b/Documentation/virt/kvm/api.rst > > @@ -6221,11 +6221,25 @@ to the byte array. > > __u64 flags; > > } hypercall; > > =20 > > -Unused. This was once used for 'hypercall to userspace'. To implement > > -such functionality, use KVM_EXIT_IO (x86) or KVM_EXIT_MMIO (all except= s390). > > + > > +It is strongly recommended that userspace use ``KVM_EXIT_IO`` (x86) or > > +``KVM_EXIT_MMIO`` (all except s390) to implement functionality that > > +requires a guest to interact with host userpace. > > =20 > > .. note:: KVM_EXIT_IO is significantly faster than KVM_EXIT_MMIO. > > =20 > > +For arm64: > > +---------- > > + > > +``nr`` contains the function ID of the guest's SMCCC call. Userspace is > > +expected to use the ``KVM_GET_ONE_REG`` ioctl to retrieve the call > > +parameters from the vCPU's GPRs. > > + > > +Definition of ``flags``: > > + - ``KVM_HYPERCALL_EXIT_SMC``: Indicates that the guest used the SMC > > + conduit to initiate the SMCCC call. If this bit is 0 then the guest > > + used the HVC conduit for the SMCCC call. > > + > > :: > > =20 > > /* KVM_EXIT_TPR_ACCESS */ > > diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uap= i/asm/kvm.h > > index f9672ef1159a..f86446c5a7e3 100644 > > --- a/arch/arm64/include/uapi/asm/kvm.h > > +++ b/arch/arm64/include/uapi/asm/kvm.h > > @@ -472,12 +472,16 @@ enum { > > enum kvm_smccc_filter_action { > > KVM_SMCCC_FILTER_HANDLE =3D 0, > > KVM_SMCCC_FILTER_DENY, > > + KVM_SMCCC_FILTER_FWD_TO_USER, > > =20 > > #ifdef __KERNEL__ > > NR_SMCCC_FILTER_ACTIONS > > #endif > > }; > > =20 > > +/* arm64-specific KVM_EXIT_HYPERCALL flags */ > > +#define KVM_HYPERCALL_EXIT_SMC (1U << 0) > > + > > #endif > > =20 > > #endif /* __ARM_KVM_H__ */ > > diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c > > index 68f95dcd41a1..3f43e20c48b6 100644 > > --- a/arch/arm64/kvm/handle_exit.c > > +++ b/arch/arm64/kvm/handle_exit.c > > @@ -71,7 +71,9 @@ static int handle_smc(struct kvm_vcpu *vcpu) > > * Trap exception, not a Secure Monitor Call exception [...]" > > * > > * We need to advance the PC after the trap, as it would > > - * otherwise return to the same address... > > + * otherwise return to the same address. Furthermore, pre-incrementing > > + * the PC before potentially exiting to userspace maintains the same > > + * abstraction for both SMCs and HVCs. >=20 > nit: this comment really needs to find its way in the documentation so > that a VMM author can determine the PC of the SMC/HVC. This is > specially important for 32bit, which has a 16bit encodings for > SMC/HVC. >=20 > And thinking of it, this outlines a small flaw in this API. If > luserspace needs to find out about the address of the HVC/SMC, it > needs to know the *size* of the instruction. But we don't propagate > the ESR value. I think this still works by construction (userspace can > check PSTATE and work out whether we're in ARM or Thumb mode), but > this feels fragile. >=20 > Should we expose the ESR, or at least ESR_EL2.IL as an additional > flag? Just to make this a quicker round trip, I hacked the following together. If you agree with it, I'll stick it on top and get the ball rolling. Thanks, M. =46rom 9b830e7a3819c2771074bebe66c1d5f20394e3cc Mon Sep 17 00:00:00 2001 From: Marc Zyngier Date: Wed, 5 Apr 2023 12:48:58 +0100 Subject: [PATCH] KVM: arm64: Expose SMC/HVC width to userspace When returning to userspace to handle a SMCCC call, we consistently set PC to point to the instruction immediately after the HVC/SMC. However, should userspace need to know the exact address of the trapping instruction, it needs to know about the *size* of that instruction. For AArch64, this is pretty easy. For AArch32, this is a bit more funky, as Thumb has 16bit encodings for both HVC and SMC. Expose this to userspace with a new flag that directly derives from ESR_EL2.IL. Also update the documentation to reflect the PC state at the point of exit. Finally, this fixes a small buglet where the hypercall.{args,ret} fields would not be cleared on exit, and could contain some random junk. Signed-off-by: Marc Zyngier --- Documentation/virt/kvm/api.rst | 8 ++++++++ arch/arm64/include/uapi/asm/kvm.h | 3 ++- arch/arm64/kvm/hypercalls.c | 16 +++++++++++----- 3 files changed, 21 insertions(+), 6 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index c8ab2f730945..103f945959ed 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -6244,6 +6244,14 @@ Definition of ``flags``: conduit to initiate the SMCCC call. If this bit is 0 then the guest used the HVC conduit for the SMCCC call. =20 + - ``KVM_HYPERCALL_EXIT_16BIT``: Indicates that the guest used a 16bit + instruction to initiate the SMCCC call. If this bit is 0 then the + guest used a 32bit instruction. An AArch64 guest always has this + bit set to 0. + +At the point of exit, PC points to the instruction immediately following +the trapping instruction. + :: =20 /* KVM_EXIT_TPR_ACCESS */ diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/as= m/kvm.h index 3dcfa4bfdf83..b1c1edf85480 100644 --- a/arch/arm64/include/uapi/asm/kvm.h +++ b/arch/arm64/include/uapi/asm/kvm.h @@ -491,7 +491,8 @@ struct kvm_smccc_filter { }; =20 /* arm64-specific KVM_EXIT_HYPERCALL flags */ -#define KVM_HYPERCALL_EXIT_SMC (1U << 0) +#define KVM_HYPERCALL_EXIT_SMC (1U << 0) +#define KVM_HYPERCALL_EXIT_16BIT (1U << 1) =20 #endif =20 diff --git a/arch/arm64/kvm/hypercalls.c b/arch/arm64/kvm/hypercalls.c index 9a35d6d18193..3b6523f25afc 100644 --- a/arch/arm64/kvm/hypercalls.c +++ b/arch/arm64/kvm/hypercalls.c @@ -222,13 +222,19 @@ static void kvm_prepare_hypercall_exit(struct kvm_vcp= u *vcpu, u32 func_id) { u8 ec =3D ESR_ELx_EC(kvm_vcpu_get_esr(vcpu)); struct kvm_run *run =3D vcpu->run; - - run->exit_reason =3D KVM_EXIT_HYPERCALL; - run->hypercall.nr =3D func_id; - run->hypercall.flags =3D 0; + u64 flags =3D 0; =20 if (ec =3D=3D ESR_ELx_EC_SMC32 || ec =3D=3D ESR_ELx_EC_SMC64) - run->hypercall.flags |=3D KVM_HYPERCALL_EXIT_SMC; + flags |=3D KVM_HYPERCALL_EXIT_SMC; + + if (!kvm_vcpu_trap_il_is32bit(vcpu)) + flags |=3D KVM_HYPERCALL_EXIT_16BIT; + + run->exit_reason =3D KVM_EXIT_HYPERCALL; + run->hypercall =3D (typeof(run->hypercall)) { + .nr =3D func_id, + .flags =3D flags, + }; } =20 int kvm_smccc_call_handler(struct kvm_vcpu *vcpu) --=20 2.34.1 --=20 Without deviation from the norm, progress is not possible.