Re: [PATCH kvmtool v2 5/6] arm64: add FEAT_E2H0 support (TBC)

[Marc Zyngier <maz@kernel.org>]

On Sat, 26 Jul 2025 11:11:43 +0100,
Wei-Lin Chang <r09922117@csie.ntu.edu.tw> wrote:
> 
> On Sat, Jul 26, 2025 at 10:19:11AM +0100, Marc Zyngier wrote:
> > On Sat, 26 Jul 2025 10:01:25 +0100,
> > Wei-Lin Chang <r09922117@csie.ntu.edu.tw> wrote:
> > > 
> > > Hi all,
> > > 
> > > On Fri, Jul 25, 2025 at 05:37:12PM +0100, Marc Zyngier wrote:
> > > > Hi Andre,
> > > > 
> > > > Thanks for picking this. A few nits below.
> > > > 
> > > > On Fri, 25 Jul 2025 15:40:59 +0100,
> > > > Andre Przywara <andre.przywara@arm.com> wrote:
> > > > > 
> > > > > From: Marc Zyngier <maz@kernel.org>
> > > > > 
> > > > > To reduce code complexity, KVM only supports nested virtualisation in
> > > > > VHE mode. So to allow recursive nested virtualisation, and be able to
> > > > > expose FEAT_NV2 to a guest, we must prevent a guest from turning off
> > > > > HCR_EL2.E2H, which is covered by not advertising the FEAT_E2H0 architecture
> > > > > feature.
> > > > > 
> > > > > To allow people to run a guest in non-VHE mode, KVM introduced the
> > > > > KVM_ARM_VCPU_HAS_EL2_E2H0 feature flag, which will allow control over
> > > > > HCR_EL2.E2H, but at the cost of turning off FEAT_NV2.
> > > > 
> > > > All of that has been captured at length in the kernel code, and I
> > > > think this is "too much information" for userspace. I'd rather we
> > > > stick to a pure description of what the various options mean to the
> > > > user.
> > > > 
> > > > > Add a kvmtool command line option "--e2h0" to set that feature bit when
> > > > > creating a guest, to gain non-VHE, but lose recursive nested virt.
> > > > 
> > > > How about:
> > > > 
> > > > "The --nested option allows a guest to boot at EL2 without FEAT_E2H0
> > > >  (i.e. mandating VHE support). While this is great for "modern"
> > > >  operating systems and hypervisors, a few legacy guests are stuck in a
> > > >  distant past.
> > > > 
> > > >  To support those, the --e2h0 option exposes FEAT_E2H0 to the guest,
> > > >  at the expense of a number of other features, such as FEAT_NV2. This
> > > 
> > > Just a very small thing:
> > > 
> > > Will only mentioning FEAT_NV2 here lead people to think that FEAT_NV is
> > > still available with --e2h0?
> > > Maybe s/FEAT_NV2/FEAT_NV/ makes it clearer?
> > 
> > Maybe. On the other hand, we never advertise the old FEAT_NV as such,
> > irrespective of the state of E2H. This is indicated by
> > ID_AA64MMFR4_EL1.NV_frac==0b0001 when NV is advertised. So I'm not
> > sure this changes anything, really.
> 
> Right, thanks for the input. Even if the user doesn't know what's up the L1
> hypervisor will when it checks ID_AA64MMFR4_EL1 :)

Exactly.

The idea behind this relaxation to the architecture was that SW that
is relying on FEAT_NV being advertised through ID_AA64MMFR2_EL1 would
find that the "old style NV" isn't supported, while SW that has
followed the architecture would also look at NV_frac, and find that
the support that matters actually exists.

It means that we potentially leave behind some hypervisors that
haven't caught up with NV_frac yet, but since we don't know of any
other NV-capable hypervisor that could run under KVM, that's not
really a problem! ;-)

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.