Re: [PATCH kvmtool v3 5/6] arm64: add FEAT_E2H0 support

[Marc Zyngier <maz@kernel.org>]

On Mon, 04 Aug 2025 15:45:52 +0100,
Alexandru Elisei <alexandru.elisei@arm.com> wrote:
> 
> Hi,
> 
> According to the Arm ARM, FEAT_E2H0 can co-exist with FEAT_VHE; KVM implements
> it differently and disables FEAT_VHE when KVM_ARM_VCPU_HAS_EL2_E2H0. Maybe the
> subject should be "arm64: Add KVM_ARM_VCPU_HAS_EL2_E2H0 support"?

KVM implements something that is strictly compliant with the
architecture. Not "differently", whatever you mean by that. And
FEAT_E2H0 is what you get with this option. The fact that you don't
get FEAT_VHE in this mode is a behaviour that is also compliant with
the architecture.

Whatever flag is used at as the backend is not really relevant, and
describing things in terms of architectural features has the merit of
being clearly documented.

> Also, 'add' should be capitalized.

I think we all got that particular message...

> 
> On Tue, Jul 29, 2025 at 10:57:44AM +0100, Andre Przywara wrote:
> > From: Marc Zyngier <maz@kernel.org>
> > 
> > The --nested option allows a guest to boot at EL2 without FEAT_E2H0
> > (i.e. mandating VHE support). While this is great for "modern" operating
> > systems and hypervisors, a few legacy guests are stuck in a distant past.
> > 
> > To support those, add the --e2h0 command line option, that exposes
> > FEAT_E2H0 to the guest, at the expense of a number of other features, such
> > as FEAT_NV2. This is conditioned on the host itself supporting FEAT_E2H0.
> > 
> > Signed-off-by: Marc Zyngier <maz@kernel.org>
> > Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> > ---
> >  arm64/include/kvm/kvm-config-arch.h | 5 ++++-
> >  arm64/kvm-cpu.c                     | 5 +++++
> >  2 files changed, 9 insertions(+), 1 deletion(-)
> > 
> > diff --git a/arm64/include/kvm/kvm-config-arch.h b/arm64/include/kvm/kvm-config-arch.h
> > index 44c43367b..73bf4211a 100644
> > --- a/arm64/include/kvm/kvm-config-arch.h
> > +++ b/arm64/include/kvm/kvm-config-arch.h
> > @@ -11,6 +11,7 @@ struct kvm_config_arch {
> >  	bool		has_pmuv3;
> >  	bool		mte_disabled;
> >  	bool		nested_virt;
> > +	bool		e2h0;
> >  	u64		kaslr_seed;
> >  	enum irqchip_type irqchip;
> >  	u64		fw_addr;
> > @@ -63,6 +64,8 @@ int sve_vl_parser(const struct option *opt, const char *arg, int unset);
> >  	OPT_U64('\0', "counter-offset", &(cfg)->counter_offset,			\
> >  		"Specify the counter offset, defaulting to 0"),			\
> >  	OPT_BOOLEAN('\0', "nested", &(cfg)->nested_virt,			\
> > -		    "Start VCPUs in EL2 (for nested virt)"),
> > +		    "Start VCPUs in EL2 (for nested virt)"),			\
> > +	OPT_BOOLEAN('\0', "e2h0", &(cfg)->e2h0,					\
> > +		    "Create guest without VHE support"),
> >  
> >  #endif /* ARM_COMMON__KVM_CONFIG_ARCH_H */
> > diff --git a/arm64/kvm-cpu.c b/arm64/kvm-cpu.c
> > index 42dc11dad..5e4f3a7dd 100644
> > --- a/arm64/kvm-cpu.c
> > +++ b/arm64/kvm-cpu.c
> > @@ -76,6 +76,11 @@ static void kvm_cpu__select_features(struct kvm *kvm, struct kvm_vcpu_init *init
> >  		if (!kvm__supports_extension(kvm, KVM_CAP_ARM_EL2))
> >  			die("EL2 (nested virt) is not supported");
> >  		init->features[0] |= 1UL << KVM_ARM_VCPU_HAS_EL2;
> > +		if (kvm->cfg.arch.e2h0) {
> > +			if (!kvm__supports_extension(kvm, KVM_CAP_ARM_EL2_E2H0))
> > +				die("FEAT_E2H0 is not supported");
> > +			init->features[0] |= 1UL << KVM_ARM_VCPU_HAS_EL2_E2H0;
> > +		}
> 
> From the v6.16 documentation (emphasis added by me):
> 
> 	- KVM_ARM_VCPU_HAS_EL2_E2H0: Restrict Nested Virtualisation
> 	  support to HCR_EL2.E2H being RES0 (non-VHE).
> 	  Depends on KVM_CAP_ARM_EL2_E2H0.
> 	  **KVM_ARM_VCPU_HAS_EL2 must also be set**.
> 
> But I am able to run a VM with E2H0 set and EL2 unset:

No. You run a guest with *none* of these flags set, as looking at the
code will tell you.

> 
> # ./lkvm-static run -c2 -m1024 -k Image-v6.16-rc4-upstream --nodefaults -p "earlycon root=/dev/vda" --e2h0
>   Info: # lkvm run -k Image-v6.16-rc4-upstream -m 1024 -c 2 --name guest-165
> [    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd0f0]
> ..
> [    0.390199] kvm [1]: HYP mode not available
> 
> If the documentation is correct, I would suggest that you also add a check for
> nested virtualization being enabled in kvm__arch_validate_cfg().

I think the current behaviour is acceptable. You ask for E2H0 without
--nested, you get something that the guest cannot observe, since it
boots at EL1. Why should that be a problem?

	M.

-- 
Jazz isn't dead. It just smells funny.