Re: [PATCH v6 09/49] KVM: arm64: gic-v5: Create and manage VM and VPE tables

[Marc Zyngier <maz@kernel.org>]

On Fri, 04 Sep 2026 12:38:51 +0100,
Sascha Bischoff <Sascha.Bischoff@arm.com> wrote:
> 
> GICv5 uses a set of in-memory tables to track and manage VM state.
> These must be allocated by the hypervisor and provided to the IRS.
> 
> The VMT (Virtual Machine Table) is a linear or two-level table
> comprising VMT Entries (VMTEs). Each VMTE describes the state for a
> single VM. This state includes things such as the SPI and LPI IST
> configuration (coming in a future commit), an implementation-defined
> VM Descriptor, and a VPE Table (VPET).
> 
> The VPET contains one entry per possible VPE ID belonging to a VM. It
> is used to mark a VPE as valid and provide the address of an
> implementation-defined VPE Descriptor (VPED), which the hardware uses
> to track and manage VPE state.
> 
> Allocate each VM's VPEDs as a single dense array indexed by vcpu_idx,
> while the VPET remains indexed by the userspace-provided vcpu_id. This
> keeps VPED storage proportional to the number of vCPUs even when their
> IDs are sparse.
> 
> The VMT and VPET are shared with the IRS. On systems with a
> non-coherent IRS, cache maintenance operates at cache-line
> granularity, while multiple entries can occupy the same cache line.
> Use a common lock for CPU accesses to these tables and IRS command
> processing so that writing back one entry cannot overwrite an IRS
> update to a neighbouring entry.
> 
> The implementation-defined VMD and VPED storage is also visible to the
> IRS. Round these allocations up to whole cache lines to prevent cache
> maintenance from corrupting unrelated slab objects. Initialise the
> storage before publishing its addresses to the IRS.
> 
> This commit adds support for allocating the VMT and its descriptor
> backing state, and for managing VMTEs. The VMTEs can be initialised or
> released for reuse. VM IDs are allocated with an IDA, while an XArray
> tracks the host-side allocations associated with populated VMTEs.
> 
> Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
> ---
>  arch/arm64/kvm/Makefile              |   2 +-
>  arch/arm64/kvm/vgic/vgic-init.c      |   2 +
>  arch/arm64/kvm/vgic/vgic-v5-tables.c | 681 +++++++++++++++++++++++++++
>  arch/arm64/kvm/vgic/vgic-v5-tables.h | 100 ++++
>  arch/arm64/kvm/vgic/vgic-v5.c        |  19 +
>  drivers/irqchip/irq-gic-v5-irs.c     |  12 +-
>  include/kvm/arm_vgic.h               |   4 +
>  include/linux/irqchip/arm-gic-v5.h   |  14 +-
>  8 files changed, 824 insertions(+), 10 deletions(-)
>  create mode 100644 arch/arm64/kvm/vgic/vgic-v5-tables.c
>  create mode 100644 arch/arm64/kvm/vgic/vgic-v5-tables.h
> 

[...]

The lkp bot is reporting some interesting issues in
202609190650.liMAzwjF-lkp@intel.com:

> +	/* Allocate and assign the VM Descriptor, if required. */
> +	if (vmt_info->vmd_size != 0) {
> +		vmd_alloc_size = round_up(vmt_info->vmd_size,
> +					  dma_get_cache_alignment());
> +		vmd = kzalloc(vmd_alloc_size, GFP_KERNEL);
> +		if (!vmd) {
> +			ret = -ENOMEM;
> +			goto out_fail;
> +		}
> +
> +		/* Stash the VA so we can free it later */
> +		vmi->vmd_base = vmd;

The VMD is dynamically allocated.

> +
> +		tmp = FIELD_PREP(GICV5_VMTEL2E_VMD_ADDR,
> +				 virt_to_phys(vmd) >> GICV5_VMTEL2E_VMD_ADDR_SHIFT);
> +		vmte_val0 = tmp;
> +	}
> +
> +	/*
> +	 * Allocate and assign the VPE Table.
> +	 *
> +	 * First of all, iterate over all vcpus to find the highest VPE ID we
> +	 * require - we need to ensure that we have enough storage for all
> +	 * vcpu_id values that userspace has picked and not just the total
> +	 * number of vcpus. This gives us the number of VPEs required for the
> +	 * VM.
> +	 *
> +	 * Round up the number of VPEs to a whole power of two as we cannot
> +	 * describe non-powers-of-two in the VMTE field as it conveys the number
> +	 * of ID bits used and not the number of vPEs. IRS_IDR1.IAFFID_BITS is
> +	 * encoded as N - 1, so expose at least one VPE ID bit even for a
> +	 * single-vCPU VM to keep the views consistent.
> +	 */
> +	kvm_for_each_vcpu(i, vcpu, kvm) {
> +		u16 vpe_id = vgic_v5_vpe_id(vcpu);
> +
> +		if (vpe_id > max_vpe_id)
> +			max_vpe_id = vpe_id;
> +	}
> +
> +	nr_cpus = max(2UL, roundup_pow_of_two(max_vpe_id + 1));
> +	vmi->vpe_id_bits = fls(nr_cpus) - 1;
> +
> +	vpet_alloc_size = round_up((size_t)nr_cpus * sizeof(*vpet),
> +				   dma_get_cache_alignment());
> +	vpet = kzalloc(vpet_alloc_size, GFP_KERNEL);
> +	if (!vpet) {
> +		ret = -ENOMEM;
> +		goto out_fail;
> +	}
> +
> +	/* Stash the VA so we can free it later */
> +	vmi->vpet_base = vpet;

and so is the VPET.

[...]

> diff --git a/arch/arm64/kvm/vgic/vgic-v5-tables.h b/arch/arm64/kvm/vgic/vgic-v5-tables.h
> new file mode 100644
> index 0000000000000..962be0c7cd3f6
> --- /dev/null
> +++ b/arch/arm64/kvm/vgic/vgic-v5-tables.h
> @@ -0,0 +1,100 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2025, 2026 Arm Ltd.
> + */
> +
> +#ifndef __KVM_ARM_VGICV5_TABLES_H__
> +#define __KVM_ARM_VGICV5_TABLES_H__
> +
> +#include <linux/spinlock.h>
> +#include <linux/idr.h>
> +#include <linux/irqchip/arm-gic-v5.h>
> +
> +/* Level 1 Virtual Machine Table Entry */
> +typedef __le64 vmtl1_entry;
> +
> +/* Level 2 Virtual Machine Table Entry */
> +struct vmtl2_entry {
> +	__le64 val[4];
> +};
> +
> +/* Virtual PE Table Entry */
> +typedef __le64 vpe_entry;
> +
> +struct vgic_v5_vm_info {
> +	void __iomem		*vmd_base;
> +	vpe_entry __iomem	*vpet_base;

But pointers to allocated memory cannot be __iomem, as this qualifies
MMIO registers.

I'll queue the change below on top.

	M.

diff --git a/arch/arm64/kvm/vgic/vgic-v5-tables.h b/arch/arm64/kvm/vgic/vgic-v5-tables.h
index b2ffbe68c05b4..89096926563f7 100644
--- a/arch/arm64/kvm/vgic/vgic-v5-tables.h
+++ b/arch/arm64/kvm/vgic/vgic-v5-tables.h
@@ -23,8 +23,8 @@ struct vmtl2_entry {
 typedef __le64 vpe_entry;
 
 struct vgic_v5_vm_info {
-	void __iomem		*vmd_base;
-	vpe_entry __iomem	*vpet_base;
+	void			*vmd_base;
+	vpe_entry		*vpet_base;
 	void			*vped_base;
 	u8			vpe_id_bits;
 


-- 
Jazz isn't dead. It just smells funny.