Re: [PATCH v16 44/45] KVM: arm64: CCA: Require ICH_HCR_EL2.TDIR for realms

[Kohei Enju <enju.kohei@fujitsu.com>]

On 08/10 10:41, Marc Zyngier wrote:
> On Mon, 10 Aug 2026 05:58:10 +0100,
> Kohei Enju <enju.kohei@fujitsu.com> wrote:
> > 
> > On 08/03 14:44, Steven Price wrote:
> > > KVM advertises realm support when the RMM is available, and allows
> > > userspace to create a VM with KVM_VM_TYPE_ARM_REALM on that basis.
> > > 
> > > On CPUs that lack ICH_HCR_EL2.TDIR, KVM uses ICH_HCR_EL2.TC for
> > > normal guests so that ICC_DIR_EL1 is still trapped via the common GICv3
> > > CPU interface trap. Realms cannot rely on the normal hyp-side trap
> > > handling for that fallback, so advertising RMI support on such systems
> > > lets userspace create a realm that cannot safely run.
> > > 
> > > Require the finalized ARM64_HAS_ICH_HCR_EL2_TDIR capability when
> > > reporting KVM_CAP_ARM_RMI and when accepting KVM_VM_TYPE_ARM_REALM.
> > > This leaves normal VM creation unchanged on systems that need the TC
> > > workaround.
> > 
> > Hi Steven,
> > 
> > Thanks for your work on upstreaming CCA.
> > 
> > In the v15 discussion [0], you asked whether the system I was testing was a
> > "hacked up test system" or closer to "production hardware", and I said I would
> > share more when the time came. I can now say that this is not a hacked-up test
> > system. At Fujitsu, we have real hardware (FUJITSU-MONAKA) which implements CCA
> > (FEAT_RME) but does not implement FEAT_GICv3_TDIR. The hardware details are as
> > follows:
> > 
> >   - GICv4.2 compliant implementation
> >   - Supports FEAT_GICv3, FEAT_GICv3p1, FEAT_GICv4, FEAT_GICv4p1, and FEAT_GICv3_NMI
> >   - Does not support FEAT_GICv3_LEGACY (deprecated)
> >   - Does not support FEAT_GICv3_TDIR (ICH_VTR_EL2.TDS == 0)
> > 
> > For reference, compared with Arm Neoverse V3, the virtual GIC configuration is
> > largely equivalent. The only missing non-deprecated architectural feature is
> > FEAT_GICv3_TDIR.
> 
> A *very* significant difference. Given the cost of trapping between
> R-EL1 and NS-EL2, something as simple as accesses to ICV_PMR_EL1
> result in an extremely expensive trap.
> 
> > 
> > The issue I see is that the CCA KVM code currently does not support a
> > configuration (non-TDIR/common-trap) that normal KVM already supports. For
> > normal guests, KVM handles systems without TDIR by using ICH_HCR_EL2.TC and the
> > existing GICv3 CPU interface emulation path. However, Realm guests currently
> > fail because the CCA path bypasses that existing emulation path, as Marc also
> > pointed out in [1].
> 
> Plugging CCA in the emulation code will solve the *functional* aspect.
> The performance aspect is still there, unfortunately, and there isn't
> much KVM can do about that.
> 
> > 
> > Also, this is not limited to systems that actually lack TDIR. The same failure
> > can be reproduced on a TDIR-capable system by booting with:
> >   kvm-arm.vgic_v3_common_trap=1
> 
> This is a *debug* option for broken hardware. ThunderX, for
> example. You really are in good company when this bit is set.
> 
> > 
> > So it seems that the current CCA KVM implementation does not yet cover a
> > configuration that normal KVM already supports today, rather than this being a
> > limitation of the RMM specification or the underlying hardware.
> > 
> > I've included a patch below which reuses the existing GICv3 early emulation
> > path for Realm sysreg exits. This patch does not add any new vGIC emulation
> > code, and leaves the existing vGIC emulation code unchanged. So I believe this
> > is in line with Marc's request in [1]. With this patch, Realm guests can run
> > when the common CPU interface trap path is enabled.
> > 
> > I tested the exact patch both on our real silicon and on QEMU, and
> > confirmed that all Realm-related tests in kvm-unit-tests-cca passed.
> 
> KUTs are unfortunately not something that people run in production. I
> wonder why...
> 
> Please run a Linux guest compiled with CONFIG_ARM64_PSEUDO_NMI=y and
> irqchip.gicv3_pseudo_nmi=1 on the command line. Run any significant
> workload (hackbench, for example), and report the overhead. This will
> give you the expected impact introduced by the lack of TDIR.

Sorry for the delayed response.

Understood. I will run a Linux guest with CONFIG_ARM64_PSEUDO_NMI=y and
irqchip.gicv3_pseudo_nmi=1, measure a representative workload such as
hackbench, and report the overhead.

Thanks,
Kohei

> 
> 	M.
> 
> -- 
> Without deviation from the norm, progress is not possible.