Re: [PATCH 03/43] KVM: arm64: gic-v5: Add resident/non-resident hyp calls

[Sascha Bischoff <Sascha.Bischoff@arm.com>]

On Tue, 2026-04-28 at 15:28 +0100, Marc Zyngier wrote:
> On Mon, 27 Apr 2026 17:07:03 +0100,
> Sascha Bischoff <Sascha.Bischoff@arm.com> wrote:
> > 
> > So far the KVM GICv5 support has been limited to PPIs. These only
> > go
> > as far out as the CPU interface, and have no interaction with the
> > host's IRS. Therefore, PPIs can be directly used for guests without
> > host IRS involvement. However, in order to support both SPIs and
> > LPIs
> > IRS involvement is required.
> 
> Please keep this in the cover letter. It isn't required to understand
> this patch.

Done.

> 
> > 
> > GICv5 introduces the concept of VPE residency - a VPE can be either
> > resident or non-resident. When the VPE is resident, the IRS is
> > allowed
> > to select interrupts that target that VPE (or the VM) as the HPPI
> > (Highest Priority Pending Interrupt). As the IRS handles both SPIs
> > and
> > LPIs, these will only be picked as the IRS's HPPI when a VPE is
> > resident.
> > 
> > A GICv5 VPE is made resident by writing to the ICH_CONTEXTR_EL2
> > with a
> > valid VM and VPE ID, and marking it valid in the process. This
> > informs
> > the IRS that a specific VPE is running, and that it can begin HPPI
> > selection for that VPE. Making a VPE non-resident (by making the
> > ICH_CONTEXTR_EL2 invalid) informs the IRS that the VPE is no longer
> > running, and it stops HPPI selection for it.
> > 
> > This change introduces two new hyp calls - one to make a VPE
> > resident
> > and its counterpart to make a VPE non-resident. As part of making a
> > VPE resident, the ICH_CONTEXTR_EL2.F bit is checked in order to
> > catch
> > faults, at which point the kernel will WARN. If everything is
> > configured correctly, this should not happen.
> > 
> > Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
> > ---
> >  arch/arm64/include/asm/kvm_asm.h   |  2 ++
> >  arch/arm64/include/asm/kvm_hyp.h   |  2 ++
> >  arch/arm64/kvm/hyp/nvhe/hyp-main.c | 15 +++++++++++++++
> >  arch/arm64/kvm/hyp/vgic-v5-sr.c    | 25 +++++++++++++++++++++++++
> >  include/kvm/arm_vgic.h             |  3 +++
> >  5 files changed, 47 insertions(+)
> > 
> > diff --git a/arch/arm64/include/asm/kvm_asm.h
> > b/arch/arm64/include/asm/kvm_asm.h
> > index fa033be6141ad..8c69f1f4de534 100644
> > --- a/arch/arm64/include/asm/kvm_asm.h
> > +++ b/arch/arm64/include/asm/kvm_asm.h
> > @@ -79,6 +79,8 @@ enum __kvm_host_smccc_func {
> >  	__KVM_HOST_SMCCC_FUNC___kvm_timer_set_cntvoff,
> >  	__KVM_HOST_SMCCC_FUNC___vgic_v3_save_aprs,
> >  	__KVM_HOST_SMCCC_FUNC___vgic_v3_restore_vmcr_aprs,
> > +	__KVM_HOST_SMCCC_FUNC___vgic_v5_make_resident,
> > +	__KVM_HOST_SMCCC_FUNC___vgic_v5_make_non_resident,
> >  	__KVM_HOST_SMCCC_FUNC___vgic_v5_save_apr,
> >  	__KVM_HOST_SMCCC_FUNC___vgic_v5_restore_vmcr_apr,
> >  
> > diff --git a/arch/arm64/include/asm/kvm_hyp.h
> > b/arch/arm64/include/asm/kvm_hyp.h
> > index 8d06b62e7188c..5f9184276b04e 100644
> > --- a/arch/arm64/include/asm/kvm_hyp.h
> > +++ b/arch/arm64/include/asm/kvm_hyp.h
> > @@ -88,6 +88,8 @@ void __vgic_v3_restore_vmcr_aprs(struct
> > vgic_v3_cpu_if *cpu_if);
> >  int __vgic_v3_perform_cpuif_access(struct kvm_vcpu *vcpu);
> >  
> >  /* GICv5 */
> > +void __vgic_v5_make_resident(struct vgic_v5_cpu_if *cpu_if);
> > +void __vgic_v5_make_non_resident(struct vgic_v5_cpu_if *cpu_if);
> >  void __vgic_v5_save_apr(struct vgic_v5_cpu_if *cpu_if);
> >  void __vgic_v5_restore_vmcr_apr(struct vgic_v5_cpu_if *cpu_if);
> >  /* No hypercalls for the following */
> > diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c
> > b/arch/arm64/kvm/hyp/nvhe/hyp-main.c
> > index 9e44c05cf780e..804a9ffdc8594 100644
> > --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c
> > +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c
> > @@ -672,6 +672,19 @@ static void
> > handle___tracing_write_event(struct kvm_cpu_context *host_ctxt)
> >  	trace_selftest(id);
> >  }
> >  
> > +static void handle___vgic_v5_make_resident(struct kvm_cpu_context
> > *host_ctxt)
> > +{
> > +	DECLARE_REG(struct vgic_v5_cpu_if *, cpu_if, host_ctxt,
> > 1);
> > +
> > +	__vgic_v5_make_resident(kern_hyp_va(cpu_if));
> > +}
> > +static void handle___vgic_v5_make_non_resident(struct
> > kvm_cpu_context *host_ctxt)
> > +{
> > +	DECLARE_REG(struct vgic_v5_cpu_if *, cpu_if, host_ctxt,
> > 1);
> > +
> > +	__vgic_v5_make_non_resident(kern_hyp_va(cpu_if));
> > +}
> > +
> >  static void handle___vgic_v5_save_apr(struct kvm_cpu_context
> > *host_ctxt)
> >  {
> >  	DECLARE_REG(struct vgic_v5_cpu_if *, cpu_if, host_ctxt,
> > 1);
> > @@ -711,6 +724,8 @@ static const hcall_t host_hcall[] = {
> >  	HANDLE_FUNC(__kvm_timer_set_cntvoff),
> >  	HANDLE_FUNC(__vgic_v3_save_aprs),
> >  	HANDLE_FUNC(__vgic_v3_restore_vmcr_aprs),
> > +	HANDLE_FUNC(__vgic_v5_make_resident),
> > +	HANDLE_FUNC(__vgic_v5_make_non_resident),
> >  	HANDLE_FUNC(__vgic_v5_save_apr),
> >  	HANDLE_FUNC(__vgic_v5_restore_vmcr_apr),
> >  
> > diff --git a/arch/arm64/kvm/hyp/vgic-v5-sr.c
> > b/arch/arm64/kvm/hyp/vgic-v5-sr.c
> > index 6d69dfe89a96c..04c5846b9abac 100644
> > --- a/arch/arm64/kvm/hyp/vgic-v5-sr.c
> > +++ b/arch/arm64/kvm/hyp/vgic-v5-sr.c
> > @@ -7,6 +7,31 @@
> >  
> >  #include <asm/kvm_hyp.h>
> >  
> > +void __vgic_v5_make_resident(struct vgic_v5_cpu_if *cpu_if)
> > +{
> > +	write_sysreg_s(cpu_if->vgic_contextr,
> > SYS_ICH_CONTEXTR_EL2);
> > +	isb();
> > +
> > +	/* Catch any faults */
> > +	cpu_if->vgic_contextr =
> > read_sysreg_s(SYS_ICH_CONTEXTR_EL2);
> > +	if (WARN_ON(FIELD_GET(ICH_CONTEXTR_EL2_F, cpu_if-
> > >vgic_contextr)))
> > +		return;
> 
> I don't think this is particularly useful here. With non-VHE, this
> results in a panic. If there is a problem, you need to fail the vcpu
> entry early, and let the caller find out.

OK, noted. I've dropped the WARN_ON() here, have reworked this to catch
the error in the calling function (vgic_v5_load()). Given that we
really are quite dead if we can't make the VPE resident, I catch the
error and mark the VM as dead.
> 
> > +
> > +	cpu_if->gicv5_vpe.resident = true;
> > +}
> > +
> > +void __vgic_v5_make_non_resident(struct vgic_v5_cpu_if *cpu_if)
> > +{
> > +	/*
> > +	 * Make as non-resident before actually making non-
> > resident. Avoids race
> > +	 * with doorbell arriving.
> > +	 */
> > +	cpu_if->gicv5_vpe.resident = false;
> 
> Stores and sysreg accesses are not ordered without a DSB ST in
> between.

Argh, yes. Have added that in.

> 
> > +
> > +	write_sysreg_s(cpu_if->vgic_contextr,
> > SYS_ICH_CONTEXTR_EL2);
> 
> Is this really making anything non-resident? Who clears the V bit?

A vcpu is made resident as part of vgic_v5_load() and non-resident as
part of vgic_v5_put(). In put, the new value of the contextr is
computed as it isn't just a case of zeroing the valid bit. We need to
triage if we need to request a doorbell or not, and what the priority
mask for that is. Therefore, as part of putting (and hence making the
vCPU non-resident) we recalculate a new contextr with V=0, and DB/DBPM
set appropriately.

The commit that plumbs these into load/put this is "KVM: arm64: gic-v5:
Make VPEs (non-)resident in vgic_load/put".

Thanks,
Sascha

> 
> > +	isb();
> > +}
> > +
> >  void __vgic_v5_save_apr(struct vgic_v5_cpu_if *cpu_if)
> >  {
> >  	cpu_if->vgic_apr = read_sysreg_s(SYS_ICH_APR_EL2);
> > diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> > index fe49fb56dc3c9..d14cf4771d606 100644
> > --- a/include/kvm/arm_vgic.h
> > +++ b/include/kvm/arm_vgic.h
> > @@ -495,6 +495,9 @@ struct vgic_v5_cpu_if {
> >  	 */
> >  	u64	vgic_icsr;
> >  
> > +	/* The contextr used to make VPEs resident and non-
> > resident */
> > +	u64	vgic_contextr;
> > +
> >  	struct gicv5_vpe gicv5_vpe;
> >  };
> >  
> 
> Thanks,
> 
> 	M.
>