Re: [PATCH kvmtool v2 5/6] arm64: add FEAT_E2H0 support (TBC)

[Marc Zyngier <maz@kernel.org>]

Hi Andre,

Thanks for picking this. A few nits below.

On Fri, 25 Jul 2025 15:40:59 +0100,
Andre Przywara <andre.przywara@arm.com> wrote:
> 
> From: Marc Zyngier <maz@kernel.org>
> 
> To reduce code complexity, KVM only supports nested virtualisation in
> VHE mode. So to allow recursive nested virtualisation, and be able to
> expose FEAT_NV2 to a guest, we must prevent a guest from turning off
> HCR_EL2.E2H, which is covered by not advertising the FEAT_E2H0 architecture
> feature.
> 
> To allow people to run a guest in non-VHE mode, KVM introduced the
> KVM_ARM_VCPU_HAS_EL2_E2H0 feature flag, which will allow control over
> HCR_EL2.E2H, but at the cost of turning off FEAT_NV2.

All of that has been captured at length in the kernel code, and I
think this is "too much information" for userspace. I'd rather we
stick to a pure description of what the various options mean to the
user.

> Add a kvmtool command line option "--e2h0" to set that feature bit when
> creating a guest, to gain non-VHE, but lose recursive nested virt.

How about:

"The --nested option allows a guest to boot at EL2 without FEAT_E2H0
 (i.e. mandating VHE support). While this is great for "modern"
 operating systems and hypervisors, a few legacy guests are stuck in a
 distant past.

 To support those, the --e2h0 option exposes FEAT_E2H0 to the guest,
 at the expense of a number of other features, such as FEAT_NV2. This
 is conditioned on the host itself supporting FEAT_E2H0."

> 
> Signed-off-by: Marc Zyngier <maz@kernel.org>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  arm64/include/kvm/kvm-config-arch.h | 5 ++++-
>  arm64/kvm-cpu.c                     | 2 ++
>  2 files changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/arm64/include/kvm/kvm-config-arch.h b/arm64/include/kvm/kvm-config-arch.h
> index 44c43367b..73bf4211a 100644
> --- a/arm64/include/kvm/kvm-config-arch.h
> +++ b/arm64/include/kvm/kvm-config-arch.h
> @@ -11,6 +11,7 @@ struct kvm_config_arch {
>  	bool		has_pmuv3;
>  	bool		mte_disabled;
>  	bool		nested_virt;
> +	bool		e2h0;
>  	u64		kaslr_seed;
>  	enum irqchip_type irqchip;
>  	u64		fw_addr;
> @@ -63,6 +64,8 @@ int sve_vl_parser(const struct option *opt, const char *arg, int unset);
>  	OPT_U64('\0', "counter-offset", &(cfg)->counter_offset,			\
>  		"Specify the counter offset, defaulting to 0"),			\
>  	OPT_BOOLEAN('\0', "nested", &(cfg)->nested_virt,			\
> -		    "Start VCPUs in EL2 (for nested virt)"),
> +		    "Start VCPUs in EL2 (for nested virt)"),			\
> +	OPT_BOOLEAN('\0', "e2h0", &(cfg)->e2h0,					\
> +		    "Create guest without VHE support"),
>  
>  #endif /* ARM_COMMON__KVM_CONFIG_ARCH_H */
> diff --git a/arm64/kvm-cpu.c b/arm64/kvm-cpu.c
> index 42dc11dad..6eb76dff4 100644
> --- a/arm64/kvm-cpu.c
> +++ b/arm64/kvm-cpu.c
> @@ -76,6 +76,8 @@ static void kvm_cpu__select_features(struct kvm *kvm, struct kvm_vcpu_init *init
>  		if (!kvm__supports_extension(kvm, KVM_CAP_ARM_EL2))
>  			die("EL2 (nested virt) is not supported");
>  		init->features[0] |= 1UL << KVM_ARM_VCPU_HAS_EL2;
> +		if (kvm->cfg.arch.e2h0)
> +			init->features[0] |= 1UL << KVM_ARM_VCPU_HAS_EL2_E2H0;

This really should also check the capability in order to fail
gracefully on system that have no E2H0 support at all (or have it so
buggy that it is permanently disabled by the kernel):

+		if (kvm->cfg.arch.e2h0) {
+	  		if (!kvm__supports_extension(kvm, KVM_CAP_ARM_EL2_E2H0))
+				die("FEAT_E2H0 is not supported");
+			init->features[0] |= 1UL << KVM_ARM_VCPU_HAS_EL2_E2H0;
+		}

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.