From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 32C3A3BBF2; Fri, 25 Jul 2025 16:37:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753461437; cv=none; b=eUVoR2d6VvLnjuCvzmTuixhOPw65IqPiFm1SISKFud5dHybilMchOd6WLm8YWoYVknW65/XZFAueDQ1WcllK2BDdJpyxj7pypktZj63DMNCtTli12j/D8LnkkFWFpz86mL1klju7Zu+qgELMnJ7QSDuoD9rj2x1S3H4JeBHanqQ= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753461437; c=relaxed/simple; bh=/oqgJIktBRx2Oetz49goPLT+JAoQFBLCRGikSQW8otY=; h=Date:Message-ID:From:To:Cc:Subject:In-Reply-To:References: MIME-Version:Content-Type; b=mC9YlJqQG8dcw9i8eBUYkIJvE+e7DGBXAXfIdzMsP6pxbxKr7QC93sa/xLIaf32F/iG9eLmMAbpHMS7w1rozvWnR5Gv4neqsiD2wmecBY0kWgH1JukWk8EkWefmQV+q/m8AxJhOP4tquB4XWPhqi7+IsB9nmkp6ClbSpuoj1qAg= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=rfCzr7Z4; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="rfCzr7Z4" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B6F7EC4CEE7; Fri, 25 Jul 2025 16:37:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1753461434; bh=/oqgJIktBRx2Oetz49goPLT+JAoQFBLCRGikSQW8otY=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=rfCzr7Z4iV0PdLQHU/zwAIaTWQnwdpZD9/MTiLT1U4pF96dV4Qznl1FlxOU1YUONG wMKgjmPP9D7Bv3qewXR+KCO0hlZF8AnG4CLl8ufSVF69CkPO5Lb5GNx1OuKLWuUH1e gUNLRC516FlK4pZ0P0vPU5yjuAe6m2CusWoY3DganhyaBDqA3vRnszEidwJOcFMEuZ uCpjU+i4a29fpRATUoyNHP9bDPYkZzM9QYV1cdi8PgQugVIKsMq7pVKAyCklSac/Xc SqS7X4EWJs8L9iCdc7SkNvUHeeBvOVbtW7n1neBeN6n30dqiXEVOdtlh4Ko2f5psS/ txDcdbNyqIn+g== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1ufLQ0-001OXY-Bd; Fri, 25 Jul 2025 17:37:12 +0100 Date: Fri, 25 Jul 2025 17:37:12 +0100 Message-ID: <86cy9o8bwn.wl-maz@kernel.org> From: Marc Zyngier To: Andre Przywara Cc: Will Deacon , Julien Thierry , kvm@vger.kernel.org, kvmarm@lists.linux.dev, Alexandru Elisei Subject: Re: [PATCH kvmtool v2 5/6] arm64: add FEAT_E2H0 support (TBC) In-Reply-To: <20250725144100.2944226-6-andre.przywara@arm.com> References: <20250725144100.2944226-1-andre.przywara@arm.com> <20250725144100.2944226-6-andre.przywara@arm.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/30.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: andre.przywara@arm.com, will@kernel.org, julien.thierry.kdev@gmail.com, kvm@vger.kernel.org, kvmarm@lists.linux.dev, alexandru.elisei@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Hi Andre, Thanks for picking this. A few nits below. On Fri, 25 Jul 2025 15:40:59 +0100, Andre Przywara wrote: > > From: Marc Zyngier > > To reduce code complexity, KVM only supports nested virtualisation in > VHE mode. So to allow recursive nested virtualisation, and be able to > expose FEAT_NV2 to a guest, we must prevent a guest from turning off > HCR_EL2.E2H, which is covered by not advertising the FEAT_E2H0 architecture > feature. > > To allow people to run a guest in non-VHE mode, KVM introduced the > KVM_ARM_VCPU_HAS_EL2_E2H0 feature flag, which will allow control over > HCR_EL2.E2H, but at the cost of turning off FEAT_NV2. All of that has been captured at length in the kernel code, and I think this is "too much information" for userspace. I'd rather we stick to a pure description of what the various options mean to the user. > Add a kvmtool command line option "--e2h0" to set that feature bit when > creating a guest, to gain non-VHE, but lose recursive nested virt. How about: "The --nested option allows a guest to boot at EL2 without FEAT_E2H0 (i.e. mandating VHE support). While this is great for "modern" operating systems and hypervisors, a few legacy guests are stuck in a distant past. To support those, the --e2h0 option exposes FEAT_E2H0 to the guest, at the expense of a number of other features, such as FEAT_NV2. This is conditioned on the host itself supporting FEAT_E2H0." > > Signed-off-by: Marc Zyngier > Signed-off-by: Andre Przywara > --- > arm64/include/kvm/kvm-config-arch.h | 5 ++++- > arm64/kvm-cpu.c | 2 ++ > 2 files changed, 6 insertions(+), 1 deletion(-) > > diff --git a/arm64/include/kvm/kvm-config-arch.h b/arm64/include/kvm/kvm-config-arch.h > index 44c43367b..73bf4211a 100644 > --- a/arm64/include/kvm/kvm-config-arch.h > +++ b/arm64/include/kvm/kvm-config-arch.h > @@ -11,6 +11,7 @@ struct kvm_config_arch { > bool has_pmuv3; > bool mte_disabled; > bool nested_virt; > + bool e2h0; > u64 kaslr_seed; > enum irqchip_type irqchip; > u64 fw_addr; > @@ -63,6 +64,8 @@ int sve_vl_parser(const struct option *opt, const char *arg, int unset); > OPT_U64('\0', "counter-offset", &(cfg)->counter_offset, \ > "Specify the counter offset, defaulting to 0"), \ > OPT_BOOLEAN('\0', "nested", &(cfg)->nested_virt, \ > - "Start VCPUs in EL2 (for nested virt)"), > + "Start VCPUs in EL2 (for nested virt)"), \ > + OPT_BOOLEAN('\0', "e2h0", &(cfg)->e2h0, \ > + "Create guest without VHE support"), > > #endif /* ARM_COMMON__KVM_CONFIG_ARCH_H */ > diff --git a/arm64/kvm-cpu.c b/arm64/kvm-cpu.c > index 42dc11dad..6eb76dff4 100644 > --- a/arm64/kvm-cpu.c > +++ b/arm64/kvm-cpu.c > @@ -76,6 +76,8 @@ static void kvm_cpu__select_features(struct kvm *kvm, struct kvm_vcpu_init *init > if (!kvm__supports_extension(kvm, KVM_CAP_ARM_EL2)) > die("EL2 (nested virt) is not supported"); > init->features[0] |= 1UL << KVM_ARM_VCPU_HAS_EL2; > + if (kvm->cfg.arch.e2h0) > + init->features[0] |= 1UL << KVM_ARM_VCPU_HAS_EL2_E2H0; This really should also check the capability in order to fail gracefully on system that have no E2H0 support at all (or have it so buggy that it is permanently disabled by the kernel): + if (kvm->cfg.arch.e2h0) { + if (!kvm__supports_extension(kvm, KVM_CAP_ARM_EL2_E2H0)) + die("FEAT_E2H0 is not supported"); + init->features[0] |= 1UL << KVM_ARM_VCPU_HAS_EL2_E2H0; + } Thanks, M. -- Without deviation from the norm, progress is not possible.