[PATCH v2 00/16] KVM: arm64: Add GICv5 ITS support

[PATCH v2 00/16] KVM: arm64: Add GICv5 ITS support

[Sascha Bischoff]

Hi all,

This is the second posting of my GICv5 KVM ITS series.

This series builds on the GICv5 IRS support now in 7.4-rc1 and adds
support for the GICv5 ITS, as described by the GICv5 specification
[1]. It allows a GICv5 guest to receive LPIs generated by MSIs,
including those from physical PCI devices mapped into the guest via
VFIO.

The ITS is exposed as a separate KVM device, alongside the VGICv5
device. The guest owns the device table and interrupt translation
tables. KVM walks these tables to resolve a DeviceID and EventID to an
LPI, and caches the result until the guest invalidates the relevant
translation or changes the table configuration.

The main pieces are:

* An emulated GICv5 ITS CONFIG_FRAME, including table configuration,
  event generation, and invalidation registers.

* Guest table walking for linear and two-level device and interrupt
  translation tables, together with a disposable XArray-based
  translation cache.

* MSI injection through KVM_SIGNAL_MSI and irqfd. GICv5 requires a
  DeviceID for each MSI, and the virtual ITS supports 16-bit EventIDs.

* Direct injection through the host ITS when an interrupt is forwarded
  to a vCPU. Forwarding teardown restores the host translation.

* Userspace attributes to configure the ITS address, access its
  registers, and reset the emulated state, together with ABI
  documentation.

* Selftests for the ITS attributes, guest table translation, MSI
  delivery, and guest-driven cache invalidation.

The guest tables remain the authoritative translation state. KVM's
cache can be discarded and rebuilt from guest memory; there is no
separate ITS table data for the VMM to save/restore.

Lorenzo's GICv5 Linux software enablement guide [2] describes how to
set up the GICv5 software stack. These changes have been tested on an
Arm FVP 11.32 release [3].

These changes are based on v7.4-rc1. The corresponding kvmtool patches
have been posted separately [4]. I plan to post an updated kvmtool
series shortly.

These changes are also available as a branch at [5]. For reference, v1
of this series can be found at [6].

This series applies independently of the host GICv5 fixes [7], but
concurrent use may encounter the host driver issues described there. A
branch with both series is available at [8]. Both series need to be
merged for the ITS support to be complete.

The main changes since v1 are:

* Rebased onto v7.4-rc1.

* Reused upstream ITS model tracking, removing the separate has_its
  state.

* Sashiko flagged some races for GICv3 LPIs, which were caused by me
  making the allocation common to both vITSes. Group 1 is set before
  adding the LPI to the tracking XArray. The GICv3 ITS path performs
  one initial lookup before allocating a new LPI.

* Replaced the translation cache hash table with an XArray. This
  avoids the locking/lockup issues raised by Sashiko. Furthermore, it
  is no longer possible for a malicious guest to force all LPI
  mappings into one hashtable bucket.

* Tightened translation and MSI handling: enforce advertised DeviceID
  and EventID limits, report invalid guest translations as blocked
  MSIs, and allow software injection to proceed if caching fails.

* Expanded ITS register handling and the userspace ABI: serialise
  register access, support 32-bit accesses to 64-bit registers, and
  validate restoration of IDR0-IDR2 and READ_EVENT_DATAR while keeping
  them read-only to the guest.

* Hardened direct injection teardown: track which translation owns
  forwarding, synchronise host translation restoration, and retain the
  IRQ association when a timeout requires a later retry.

* Extended selftests for register access widths, read-only behaviour,
  ID limits, malformed MSIs, and cache invalidation.

Thanks for taking the time to look at these changes!

Thanks,
Sascha

[1] https://support.arm.com/documentation/111701/latest
[2] https://linaro.atlassian.net/wiki/x/CQAF-wY
[3] https://support.arm.com/tools-and-software/fixed-virtual-platforms/arm-architecture-fvps
[4] https://lore.kernel.org/all/20261016153939.913764-1-sascha.bischoff@arm.com
[5] https://gitlab.arm.com/linux-arm/linux-sb/-/tree/gicv5_kvm_its_support_v2
[6] https://lore.kernel.org/all/20261016155741.979985-1-sascha.bischoff@arm.com/
[7] https://lore.kernel.org/all/20261102-host_gicv5_fixes-v1-0-743b61626fc5@arm.com/
[8] https://gitlab.arm.com/linux-arm/linux-sb/-/tree/gicv5_kvm_its_support_v2_with_host_fixes

Sascha Bischoff (16):
  KVM: arm64: vgic: Split out common LPI allocation
  KVM: arm64: vgic: Pass opaque device pointer to ITS MMIO callbacks
  KVM: arm64: gic-v5: Share CoreSight ID register emulation
  irqchip/gic-v5: Add ITS MMIO register definitions
  KVM: arm64: gic-v5: Add ITS device skeleton
  KVM: arm64: gic-v5: Add ITS MMIO state emulation
  KVM: arm64: gic-v5: Add ITS guest table translation emulation
  KVM: arm64: gic-v5: Add ITS translation cache maintenance
  KVM: arm64: gic-v5: Add ITS MSI injection
  KVM: arm64: gic-v5: Add ITS userspace register access
  irqchip/gic-v5: Enable direct injection in ITS
  KVM: arm64: vgic: Move direct MSI support helper to common MMIO code
  KVM: arm64: gic-v5: Implement irq forwarding
  KVM: selftests: Add VGICv5 ITS attribute tests
  KVM: selftests: Add VGICv5 ITS translation cache tests
  KVM: arm64: gic-v5: Document ITS userspace ABI

 .../virt/kvm/devices/arm-vgic-v5-its.rst      |  111 ++
 Documentation/virt/kvm/devices/index.rst      |    1 +
 arch/arm64/include/uapi/asm/kvm.h             |    2 +
 arch/arm64/kvm/Makefile                       |    1 +
 arch/arm64/kvm/arm.c                          |    6 +-
 arch/arm64/kvm/vgic/vgic-irqfd.c              |    8 +
 arch/arm64/kvm/vgic/vgic-irs-v5.c             |   75 +-
 arch/arm64/kvm/vgic/vgic-its-v5.c             | 1562 +++++++++++++++++
 arch/arm64/kvm/vgic/vgic-its.c                |  130 +-
 arch/arm64/kvm/vgic/vgic-kvm-device.c         |    4 +
 arch/arm64/kvm/vgic/vgic-mmio-v3.c            |   20 -
 arch/arm64/kvm/vgic/vgic-mmio.c               |   34 +
 arch/arm64/kvm/vgic/vgic-mmio.h               |    6 +-
 arch/arm64/kvm/vgic/vgic-v5.c                 |  182 ++
 arch/arm64/kvm/vgic/vgic.c                    |   99 ++
 arch/arm64/kvm/vgic/vgic.h                    |   20 +
 drivers/irqchip/irq-gic-v5-its.c              |  141 ++
 include/kvm/arm_vgic.h                        |  107 +-
 include/linux/irqchip/arm-gic-v5.h            |  136 +-
 include/uapi/linux/kvm.h                      |    2 +
 tools/include/uapi/linux/kvm.h                |    2 +
 tools/testing/selftests/kvm/arm64/vgic_v5.c   |  882 +++++++++-
 .../selftests/kvm/include/arm64/gic_v5.h      |   88 +
 23 files changed, 3384 insertions(+), 235 deletions(-)
 create mode 100644 Documentation/virt/kvm/devices/arm-vgic-v5-its.rst
 create mode 100644 arch/arm64/kvm/vgic/vgic-its-v5.c


base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
-- 
2.34.1

[PATCH v2 01/16] KVM: arm64: vgic: Split out common LPI allocation

[Sascha Bischoff]

The GICv3 ITS code currently owns vgic_add_lpi(), even though the
first half of the helper is not ITS specific. It allocates or finds
the VM-wide vgic_irq object in lpi_xa and initialises the common LPI
state before the GICv3 ITS code reads the property table and pending
state.

GICv5 needs to reuse the same LPI object lifetime and refcounting, but
without reading the GICv3 property table. Move the allocation and
xarray insertion path into vgic_alloc_lpi() in vgic.c. Both
vgic_add_lpi() and the GICv3 ITS wrapper perform an initial lookup,
then call this helper on a miss. The ITS wrapper continues to load the
property and pending state, while GICv5 can use the common allocator
without GICv3 setup.

Keeping the lookup outside vgic_alloc_lpi() preserves GICv3 behaviour
when another caller inserts the LPI during allocation: the ITS wrapper
still performs its setup on the returned LPI. A second lookup inside
the allocator could instead treat it as already initialised and skip
that setup.

Initialise LPIs as Group 1 before publishing them in lpi_xa. GICv3
LPIs are Group 1, and another vCPU can find a new LPI as soon as it is
published. GICv5 does not use the group field.

This preserves the intended GICv3 behaviour while making the common
LPI storage usable by other VGIC users.

Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
---
 arch/arm64/kvm/vgic/vgic-its.c | 75 ++++---------------------------
 arch/arm64/kvm/vgic/vgic.c     | 82 ++++++++++++++++++++++++++++++++++
 arch/arm64/kvm/vgic/vgic.h     |  4 ++
 3 files changed, 95 insertions(+), 66 deletions(-)

diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c
index 7e9b265b1f521..34c1c89bfbfe1 100644
--- a/arch/arm64/kvm/vgic/vgic-its.c
+++ b/arch/arm64/kvm/vgic/vgic-its.c
@@ -66,76 +66,19 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
 		__ret;							\
 	})
 
-/*
- * Creates a new (reference to a) struct vgic_irq for a given LPI.
- * If this LPI is already mapped on another ITS, we increase its refcount
- * and return a pointer to the existing structure.
- * If this is a "new" LPI, we allocate and initialize a new struct vgic_irq.
- * This function returns a pointer to the _unlocked_ structure.
- */
-static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid,
-				     struct kvm_vcpu *vcpu)
+static struct vgic_irq *vgic_its_add_lpi(struct kvm *kvm, u32 intid,
+					 struct kvm_vcpu *vcpu)
 {
-	struct vgic_dist *dist = &kvm->arch.vgic;
-	struct vgic_irq *irq = vgic_get_irq(kvm, intid), *oldirq;
-	unsigned long flags;
+	struct vgic_irq *irq;
 	int ret;
 
-	/* In this case there is no put, since we keep the reference. */
+	irq = vgic_get_irq(kvm, intid);
 	if (irq)
 		return irq;
 
-	irq = kzalloc_obj(struct vgic_irq, GFP_KERNEL_ACCOUNT);
-	if (!irq)
-		return ERR_PTR(-ENOMEM);
-
-	ret = xa_reserve_irq(&dist->lpi_xa, intid, GFP_KERNEL_ACCOUNT);
-	if (ret) {
-		kfree(irq);
-		return ERR_PTR(ret);
-	}
-
-	INIT_LIST_HEAD(&irq->ap_list);
-	raw_spin_lock_init(&irq->irq_lock);
-
-	irq->config = VGIC_CONFIG_EDGE;
-	refcount_set(&irq->refcount, 1);
-	irq->intid = intid;
-	irq->target_vcpu = vcpu;
-	irq->group = 1;
-
-	xa_lock_irqsave(&dist->lpi_xa, flags);
-
-	/*
-	 * There could be a race with another vgic_add_lpi(), so we need to
-	 * check that we don't add a second list entry with the same LPI.
-	 */
-	oldirq = xa_load(&dist->lpi_xa, intid);
-	if (vgic_try_get_irq_ref(oldirq)) {
-		/* Someone was faster with adding this LPI, lets use that. */
-		kfree(irq);
-		irq = oldirq;
-	} else {
-		/*
-		 * The entry is either empty or contains a dead LPI (refcount=0)
-		 * from the deferred release path, pending cleanup by
-		 * vgic_release_deleted_lpis(). Evict and free it if present.
-		 */
-		oldirq = __xa_store(&dist->lpi_xa, intid, irq,
-				    GFP_NOWAIT | __GFP_ACCOUNT);
-		ret = xa_err(oldirq);
-		if (ret) {
-			xa_unlock_irqrestore(&dist->lpi_xa, flags);
-			kfree(irq);
-
-			return ERR_PTR(ret);
-		}
-
-		if (oldirq && !WARN_ON_ONCE(refcount_read(&oldirq->refcount)))
-			kfree_rcu(oldirq, rcu);
-	}
-
-	xa_unlock_irqrestore(&dist->lpi_xa, flags);
+	irq = vgic_alloc_lpi(kvm, intid, vcpu);
+	if (IS_ERR(irq))
+		return irq;
 
 	/*
 	 * We "cache" the configuration table entries in our struct vgic_irq's.
@@ -1133,7 +1076,7 @@ static int vgic_its_cmd_handle_mapi(struct kvm *kvm, struct vgic_its *its,
 	if (its_is_collection_mapped(collection))
 		vcpu = collection_to_vcpu(kvm, collection);
 
-	irq = vgic_add_lpi(kvm, lpi_nr, vcpu);
+	irq = vgic_its_add_lpi(kvm, lpi_nr, vcpu);
 	if (!IS_ERR(irq) && !vgic_its_map_lpi(kvm, ite, irq, vcpu))
 		irq = ERR_PTR(-EEXIST);
 	if (IS_ERR(irq)) {
@@ -2269,7 +2212,7 @@ static int vgic_its_restore_ite(struct vgic_its *its, u32 event_id,
 	if (its_is_collection_mapped(collection))
 		vcpu = kvm_get_vcpu_by_id(kvm, collection->target_addr);
 
-	irq = vgic_add_lpi(kvm, lpi_id, vcpu);
+	irq = vgic_its_add_lpi(kvm, lpi_id, vcpu);
 	if (IS_ERR(irq)) {
 		its_free_ite(kvm, ite);
 		return PTR_ERR(irq);
diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c
index bea848c93a990..d1a1a2cea386e 100644
--- a/arch/arm64/kvm/vgic/vgic.c
+++ b/arch/arm64/kvm/vgic/vgic.c
@@ -79,6 +79,88 @@ static struct vgic_irq *vgic_get_lpi(struct kvm *kvm, u32 intid)
 	return irq;
 }
 
+/* Allocate an LPI or acquire one inserted by a concurrent caller. */
+struct vgic_irq *vgic_alloc_lpi(struct kvm *kvm, u32 intid,
+				struct kvm_vcpu *vcpu)
+{
+	struct vgic_dist *dist = &kvm->arch.vgic;
+	struct vgic_irq *irq, *oldirq;
+	unsigned long flags;
+	int ret;
+
+	irq = kzalloc_obj(struct vgic_irq, GFP_KERNEL_ACCOUNT);
+	if (!irq)
+		return ERR_PTR(-ENOMEM);
+
+	ret = xa_reserve_irq(&dist->lpi_xa, intid, GFP_KERNEL_ACCOUNT);
+	if (ret) {
+		kfree(irq);
+		return ERR_PTR(ret);
+	}
+
+	INIT_LIST_HEAD(&irq->ap_list);
+	raw_spin_lock_init(&irq->irq_lock);
+
+	irq->config = VGIC_CONFIG_EDGE;
+	refcount_set(&irq->refcount, 1);
+	irq->intid = intid;
+	irq->target_vcpu = vcpu;
+	irq->group = 1;
+
+	xa_lock_irqsave(&dist->lpi_xa, flags);
+
+	/*
+	 * There could be a race with another vgic_add_lpi(), so we need to
+	 * check that we don't add a second list entry with the same LPI.
+	 */
+	oldirq = xa_load(&dist->lpi_xa, intid);
+	if (vgic_try_get_irq_ref(oldirq)) {
+		/* Someone was faster with adding this LPI, lets use that. */
+		kfree(irq);
+		irq = oldirq;
+	} else {
+		/*
+		 * The entry is either empty or contains a dead LPI (refcount=0)
+		 * from the deferred release path, pending cleanup by
+		 * vgic_release_deleted_lpis(). Evict and free it if present.
+		 */
+		oldirq = __xa_store(&dist->lpi_xa, intid, irq,
+				    GFP_NOWAIT | __GFP_ACCOUNT);
+		ret = xa_err(oldirq);
+		if (ret) {
+			xa_unlock_irqrestore(&dist->lpi_xa, flags);
+			kfree(irq);
+
+			return ERR_PTR(ret);
+		}
+
+		if (oldirq && !WARN_ON_ONCE(refcount_read(&oldirq->refcount)))
+			kfree_rcu(oldirq, rcu);
+	}
+
+	xa_unlock_irqrestore(&dist->lpi_xa, flags);
+
+	return irq;
+}
+
+/*
+ * Creates a new (reference to a) struct vgic_irq for a given LPI.
+ * If this LPI already exists, we increase its refcount and return it.
+ * If this is a "new" LPI, we allocate and initialize a new struct vgic_irq.
+ * This function returns a pointer to the _unlocked_ structure.
+ */
+struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid,
+			      struct kvm_vcpu *vcpu)
+{
+	struct vgic_irq *irq = vgic_get_irq(kvm, intid);
+
+	/* In this case there is no put, since we keep the reference. */
+	if (irq)
+		return irq;
+
+	return vgic_alloc_lpi(kvm, intid, vcpu);
+}
+
 /*
  * This looks up the virtual interrupt ID to get the corresponding
  * struct vgic_irq. It also increases the refcount, so any caller is expected
diff --git a/arch/arm64/kvm/vgic/vgic.h b/arch/arm64/kvm/vgic/vgic.h
index 97fc3bdf520e5..8792953057bb7 100644
--- a/arch/arm64/kvm/vgic/vgic.h
+++ b/arch/arm64/kvm/vgic/vgic.h
@@ -280,6 +280,10 @@ vgic_get_mmio_region(struct kvm_vcpu *vcpu, struct vgic_io_device *iodev,
 		     gpa_t addr, int len);
 struct vgic_irq *vgic_get_irq(struct kvm *kvm, u32 intid);
 struct vgic_irq *vgic_get_vcpu_irq(struct kvm_vcpu *vcpu, u32 intid);
+struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid,
+			      struct kvm_vcpu *vcpu);
+struct vgic_irq *vgic_alloc_lpi(struct kvm *kvm, u32 intid,
+				struct kvm_vcpu *vcpu);
 void vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq);
 struct kvm_vcpu *vgic_target_oracle(struct vgic_irq *irq);
 bool vgic_get_phys_line_level(struct vgic_irq *irq);
-- 
2.34.1

[PATCH v2 02/16] KVM: arm64: vgic: Pass opaque device pointer to ITS MMIO callbacks

[Sascha Bischoff]

The ITS MMIO callbacks currently take a struct vgic_its pointer,
which ties them to the GICv3 ITS. GICv5 needs to use the same MMIO
dispatch path with a different device structure.

Pass an opaque device pointer to the callbacks and cast it in the
GICv3 handlers. This does not change GICv3 behaviour.

Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
---
 arch/arm64/kvm/vgic/vgic-its.c  | 55 +++++++++++++++++----------------
 arch/arm64/kvm/vgic/vgic-mmio.h |  6 ++--
 2 files changed, 32 insertions(+), 29 deletions(-)

diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c
index 34c1c89bfbfe1..c6e069af7c614 100644
--- a/arch/arm64/kvm/vgic/vgic-its.c
+++ b/arch/arm64/kvm/vgic/vgic-its.c
@@ -386,10 +386,10 @@ static int its_sync_lpi_pending_table(struct kvm_vcpu *vcpu)
 	return ret;
 }
 
-static unsigned long vgic_mmio_read_its_typer(struct kvm *kvm,
-					      struct vgic_its *its,
+static unsigned long vgic_mmio_read_its_typer(struct kvm *kvm, void *dev,
 					      gpa_t addr, unsigned int len)
 {
+	struct vgic_its *its = dev;
 	const struct vgic_its_abi *abi = vgic_its_get_abi(its);
 	u64 reg = GITS_TYPER_PLPIS;
 
@@ -408,10 +408,10 @@ static unsigned long vgic_mmio_read_its_typer(struct kvm *kvm,
 	return extract_bytes(reg, addr & 7, len);
 }
 
-static unsigned long vgic_mmio_read_its_iidr(struct kvm *kvm,
-					     struct vgic_its *its,
+static unsigned long vgic_mmio_read_its_iidr(struct kvm *kvm, void *dev,
 					     gpa_t addr, unsigned int len)
 {
+	struct vgic_its *its = dev;
 	u32 val;
 
 	val = (its->abi_rev << GITS_IIDR_REV_SHIFT) & GITS_IIDR_REV_MASK;
@@ -419,11 +419,11 @@ static unsigned long vgic_mmio_read_its_iidr(struct kvm *kvm,
 	return val;
 }
 
-static int vgic_mmio_uaccess_write_its_iidr(struct kvm *kvm,
-					    struct vgic_its *its,
+static int vgic_mmio_uaccess_write_its_iidr(struct kvm *kvm, void *dev,
 					    gpa_t addr, unsigned int len,
 					    unsigned long val)
 {
+	struct vgic_its *its = dev;
 	u32 rev = GITS_IIDR_REV(val);
 
 	if (rev >= NR_ITS_ABIS)
@@ -433,7 +433,7 @@ static int vgic_mmio_uaccess_write_its_iidr(struct kvm *kvm,
 }
 
 static unsigned long vgic_mmio_read_its_idregs(struct kvm *kvm,
-					       struct vgic_its *its,
+					       void *dev,
 					       gpa_t addr, unsigned int len)
 {
 	switch (addr & 0xffff) {
@@ -1496,17 +1496,18 @@ static u64 vgic_sanitise_its_cbaser(u64 reg)
 	return reg;
 }
 
-static unsigned long vgic_mmio_read_its_cbaser(struct kvm *kvm,
-					       struct vgic_its *its,
+static unsigned long vgic_mmio_read_its_cbaser(struct kvm *kvm, void *dev,
 					       gpa_t addr, unsigned int len)
 {
+	struct vgic_its *its = dev;
 	return extract_bytes(its->cbaser, addr & 7, len);
 }
 
-static void vgic_mmio_write_its_cbaser(struct kvm *kvm, struct vgic_its *its,
+static void vgic_mmio_write_its_cbaser(struct kvm *kvm, void *dev,
 				       gpa_t addr, unsigned int len,
 				       unsigned long val)
 {
+	struct vgic_its *its = dev;
 	/* When GITS_CTLR.Enable is 1, this register is RO. */
 	if (its->enabled)
 		return;
@@ -1564,10 +1565,11 @@ static void vgic_its_process_commands(struct kvm *kvm, struct vgic_its *its)
  * protects our ring buffer variables, so that there is only one user
  * per ITS handling commands at a given time.
  */
-static void vgic_mmio_write_its_cwriter(struct kvm *kvm, struct vgic_its *its,
+static void vgic_mmio_write_its_cwriter(struct kvm *kvm, void *dev,
 					gpa_t addr, unsigned int len,
 					unsigned long val)
 {
+	struct vgic_its *its = dev;
 	u64 reg;
 
 	if (!its)
@@ -1588,25 +1590,25 @@ static void vgic_mmio_write_its_cwriter(struct kvm *kvm, struct vgic_its *its,
 	mutex_unlock(&its->cmd_lock);
 }
 
-static unsigned long vgic_mmio_read_its_cwriter(struct kvm *kvm,
-						struct vgic_its *its,
+static unsigned long vgic_mmio_read_its_cwriter(struct kvm *kvm, void *dev,
 						gpa_t addr, unsigned int len)
 {
+	struct vgic_its *its = dev;
 	return extract_bytes(its->cwriter, addr & 0x7, len);
 }
 
-static unsigned long vgic_mmio_read_its_creadr(struct kvm *kvm,
-					       struct vgic_its *its,
+static unsigned long vgic_mmio_read_its_creadr(struct kvm *kvm, void *dev,
 					       gpa_t addr, unsigned int len)
 {
+	struct vgic_its *its = dev;
 	return extract_bytes(its->creadr, addr & 0x7, len);
 }
 
-static int vgic_mmio_uaccess_write_its_creadr(struct kvm *kvm,
-					      struct vgic_its *its,
+static int vgic_mmio_uaccess_write_its_creadr(struct kvm *kvm, void *dev,
 					      gpa_t addr, unsigned int len,
 					      unsigned long val)
 {
+	struct vgic_its *its = dev;
 	u32 cmd_offset;
 	int ret = 0;
 
@@ -1630,10 +1632,10 @@ static int vgic_mmio_uaccess_write_its_creadr(struct kvm *kvm,
 }
 
 #define BASER_INDEX(addr) (((addr) / sizeof(u64)) & 0x7)
-static unsigned long vgic_mmio_read_its_baser(struct kvm *kvm,
-					      struct vgic_its *its,
+static unsigned long vgic_mmio_read_its_baser(struct kvm *kvm, void *dev,
 					      gpa_t addr, unsigned int len)
 {
+	struct vgic_its *its = dev;
 	u64 reg;
 
 	switch (BASER_INDEX(addr)) {
@@ -1652,11 +1654,11 @@ static unsigned long vgic_mmio_read_its_baser(struct kvm *kvm,
 }
 
 #define GITS_BASER_RO_MASK	(GENMASK_ULL(52, 48) | GENMASK_ULL(58, 56))
-static void vgic_mmio_write_its_baser(struct kvm *kvm,
-				      struct vgic_its *its,
+static void vgic_mmio_write_its_baser(struct kvm *kvm, void *dev,
 				      gpa_t addr, unsigned int len,
 				      unsigned long val)
 {
+	struct vgic_its *its = dev;
 	const struct vgic_its_abi *abi = vgic_its_get_abi(its);
 	u64 old, entry_size, table_type;
 	u64 reg, *regptr, clearbits = 0;
@@ -1711,10 +1713,10 @@ static void vgic_mmio_write_its_baser(struct kvm *kvm,
 	}
 }
 
-static unsigned long vgic_mmio_read_its_ctlr(struct kvm *vcpu,
-					     struct vgic_its *its,
+static unsigned long vgic_mmio_read_its_ctlr(struct kvm *vcpu, void *dev,
 					     gpa_t addr, unsigned int len)
 {
+	struct vgic_its *its = dev;
 	u32 reg = 0;
 
 	mutex_lock(&its->cmd_lock);
@@ -1727,10 +1729,11 @@ static unsigned long vgic_mmio_read_its_ctlr(struct kvm *vcpu,
 	return reg;
 }
 
-static void vgic_mmio_write_its_ctlr(struct kvm *kvm, struct vgic_its *its,
+static void vgic_mmio_write_its_ctlr(struct kvm *kvm, void *dev,
 				     gpa_t addr, unsigned int len,
 				     unsigned long val)
 {
+	struct vgic_its *its = dev;
 	mutex_lock(&its->cmd_lock);
 
 	/*
@@ -1776,8 +1779,8 @@ static void vgic_mmio_write_its_ctlr(struct kvm *kvm, struct vgic_its *its,
 	.uaccess_its_write = uwr,				\
 }
 
-static void its_mmio_write_wi(struct kvm *kvm, struct vgic_its *its,
-			      gpa_t addr, unsigned int len, unsigned long val)
+static void its_mmio_write_wi(struct kvm *kvm, void *dev, gpa_t addr,
+			      unsigned int len, unsigned long val)
 {
 	/* Ignore */
 }
diff --git a/arch/arm64/kvm/vgic/vgic-mmio.h b/arch/arm64/kvm/vgic/vgic-mmio.h
index 38ed730d68ac3..c3b6d3e752d8d 100644
--- a/arch/arm64/kvm/vgic/vgic-mmio.h
+++ b/arch/arm64/kvm/vgic/vgic-mmio.h
@@ -13,13 +13,13 @@ struct vgic_register_region {
 	union {
 		unsigned long (*read)(struct kvm_vcpu *vcpu, gpa_t addr,
 				      unsigned int len);
-		unsigned long (*its_read)(struct kvm *kvm, struct vgic_its *its,
+		unsigned long (*its_read)(struct kvm *kvm, void *dev,
 					  gpa_t addr, unsigned int len);
 	};
 	union {
 		void (*write)(struct kvm_vcpu *vcpu, gpa_t addr,
 			      unsigned int len, unsigned long val);
-		void (*its_write)(struct kvm *kvm, struct vgic_its *its,
+		void (*its_write)(struct kvm *kvm, void *dev,
 				  gpa_t addr, unsigned int len,
 				  unsigned long val);
 	};
@@ -28,7 +28,7 @@ struct vgic_register_region {
 	union {
 		int (*uaccess_write)(struct kvm_vcpu *vcpu, gpa_t addr,
 				     unsigned int len, unsigned long val);
-		int (*uaccess_its_write)(struct kvm *kvm, struct vgic_its *its,
+		int (*uaccess_its_write)(struct kvm *kvm, void *dev,
 					 gpa_t addr, unsigned int len,
 					 unsigned long val);
 	};
-- 
2.34.1

[PATCH v2 03/16] KVM: arm64: gic-v5: Share CoreSight ID register emulation

[Sascha Bischoff]

The GICv5 IRS and ITS expose the same CoreSight identification register
block. Move the register-value switch into a shared helper and describe
the contiguous ID register range with one register-region descriptor.

This keeps the IRS behaviour unchanged while giving the ITS MMIO
emulation a helper it can reuse when it is added in the following commit.

Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
---
 arch/arm64/kvm/vgic/vgic-irs-v5.c  | 75 +++---------------------------
 arch/arm64/kvm/vgic/vgic-v5.c      | 34 ++++++++++++++
 arch/arm64/kvm/vgic/vgic.h         |  1 +
 include/linux/irqchip/arm-gic-v5.h |  2 +
 4 files changed, 44 insertions(+), 68 deletions(-)

diff --git a/arch/arm64/kvm/vgic/vgic-irs-v5.c b/arch/arm64/kvm/vgic/vgic-irs-v5.c
index 8d0ad74ebc64a..d0ebed2e1663b 100644
--- a/arch/arm64/kvm/vgic/vgic-irs-v5.c
+++ b/arch/arm64/kvm/vgic/vgic-irs-v5.c
@@ -520,38 +520,9 @@ static void vgic_v5_mmio_write_irs_ist(struct kvm_vcpu *vcpu, gpa_t addr,
 static unsigned long vgic_v5_coresight_read(struct kvm_vcpu *vcpu,
 					    gpa_t addr, unsigned int len)
 {
-	const size_t offset = addr & (SZ_64K - 1);
+	u32 offset = addr & (SZ_64K - 1);
 
-	switch (offset) {
-	case GICV5_CORESIGHT_DEVARCH:
-		return GICV5_CORESIGHT_DEVARCH_VAL;
-	case GICV5_CORESIGHT_PIDR4:
-		return GICV5_CORESIGHT_PIDR4_JEP106_CONT;
-	case GICV5_CORESIGHT_PIDR5:
-		return GICV5_CORESIGHT_PIDR5_RES0;
-	case GICV5_CORESIGHT_PIDR6:
-		return GICV5_CORESIGHT_PIDR6_RES0;
-	case GICV5_CORESIGHT_PIDR7:
-		return GICV5_CORESIGHT_PIDR7_RES0;
-	case GICV5_CORESIGHT_PIDR0:
-		return GICV5_CORESIGHT_PIDR0_PART_0;
-	case GICV5_CORESIGHT_PIDR1:
-		return GICV5_CORESIGHT_PIDR1_DES_0_PART_1;
-	case GICV5_CORESIGHT_PIDR2:
-		return GICV5_CORESIGHT_PIDR2_DES_1;
-	case GICV5_CORESIGHT_PIDR3:
-		return GICV5_CORESIGHT_PIDR3_REVAND_CMOD;
-	case GICV5_CORESIGHT_CIDR0:
-		return GICV5_CORESIGHT_CIDR0_VAL;
-	case GICV5_CORESIGHT_CIDR1:
-		return GICV5_CORESIGHT_CIDR1_VAL;
-	case GICV5_CORESIGHT_CIDR2:
-		return GICV5_CORESIGHT_CIDR2_VAL;
-	case GICV5_CORESIGHT_CIDR3:
-		return GICV5_CORESIGHT_CIDR3_VAL;
-	default:
-		return 0;
-	}
+	return vgic_v5_handle_guest_coresight_idreg_read(offset);
 }
 
 static unsigned long vgic_v5_mmio_uaccess_read_irs_status(struct kvm_vcpu *vcpu,
@@ -903,45 +874,13 @@ static const struct vgic_register_region vgic_v5_irs_registers[] = {
 				  vgic_mmio_read_raz, vgic_mmio_write_wi, 8,
 				  VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
 
-	/* CoreSight identification registers */
+	/* Split the ranges so KVM_HAS_DEVICE_ATTR only reports actual registers. */
 	REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_DEVARCH,
-				  vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
-				  VGIC_ACCESS_32bit),
+				  vgic_v5_coresight_read, vgic_mmio_write_wi,
+				  4, VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR4,
-				  vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
-				  VGIC_ACCESS_32bit),
-	REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR5,
-				  vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
-				  VGIC_ACCESS_32bit),
-	REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR6,
-				  vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
-				  VGIC_ACCESS_32bit),
-	REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR7,
-				  vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
-				  VGIC_ACCESS_32bit),
-	REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR0,
-				  vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
-				  VGIC_ACCESS_32bit),
-	REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR1,
-				  vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
-				  VGIC_ACCESS_32bit),
-	REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR2,
-				  vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
-				  VGIC_ACCESS_32bit),
-	REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR3,
-				  vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
-				  VGIC_ACCESS_32bit),
-	REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_CIDR0,
-				  vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
-				  VGIC_ACCESS_32bit),
-	REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_CIDR1,
-				  vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
-				  VGIC_ACCESS_32bit),
-	REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_CIDR2,
-				  vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
-				  VGIC_ACCESS_32bit),
-	REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_CIDR3,
-				  vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
+				  vgic_v5_coresight_read, vgic_mmio_write_wi,
+				  GICV5_CORESIGHT_PIDR_CIDR_SIZE,
 				  VGIC_ACCESS_32bit),
 };
 
diff --git a/arch/arm64/kvm/vgic/vgic-v5.c b/arch/arm64/kvm/vgic/vgic-v5.c
index 91810fbd99afa..18184020e188b 100644
--- a/arch/arm64/kvm/vgic/vgic-v5.c
+++ b/arch/arm64/kvm/vgic/vgic-v5.c
@@ -1601,3 +1601,37 @@ int vgic_v5_irs_restore_ists(struct kvm *kvm, struct kvm_device_attr *attr)
 
 	return ret;
 }
+
+unsigned long vgic_v5_handle_guest_coresight_idreg_read(u32 offset)
+{
+	switch (offset) {
+	case GICV5_CORESIGHT_DEVARCH:
+		return GICV5_CORESIGHT_DEVARCH_VAL;
+	case GICV5_CORESIGHT_PIDR4:
+		return GICV5_CORESIGHT_PIDR4_JEP106_CONT;
+	case GICV5_CORESIGHT_PIDR5:
+		return GICV5_CORESIGHT_PIDR5_RES0;
+	case GICV5_CORESIGHT_PIDR6:
+		return GICV5_CORESIGHT_PIDR6_RES0;
+	case GICV5_CORESIGHT_PIDR7:
+		return GICV5_CORESIGHT_PIDR7_RES0;
+	case GICV5_CORESIGHT_PIDR0:
+		return GICV5_CORESIGHT_PIDR0_PART_0;
+	case GICV5_CORESIGHT_PIDR1:
+		return GICV5_CORESIGHT_PIDR1_DES_0_PART_1;
+	case GICV5_CORESIGHT_PIDR2:
+		return GICV5_CORESIGHT_PIDR2_DES_1;
+	case GICV5_CORESIGHT_PIDR3:
+		return GICV5_CORESIGHT_PIDR3_REVAND_CMOD;
+	case GICV5_CORESIGHT_CIDR0:
+		return GICV5_CORESIGHT_CIDR0_VAL;
+	case GICV5_CORESIGHT_CIDR1:
+		return GICV5_CORESIGHT_CIDR1_VAL;
+	case GICV5_CORESIGHT_CIDR2:
+		return GICV5_CORESIGHT_CIDR2_VAL;
+	case GICV5_CORESIGHT_CIDR3:
+		return GICV5_CORESIGHT_CIDR3_VAL;
+	default:
+		return 0;
+	}
+}
diff --git a/arch/arm64/kvm/vgic/vgic.h b/arch/arm64/kvm/vgic/vgic.h
index 8792953057bb7..f95eec6a01f63 100644
--- a/arch/arm64/kvm/vgic/vgic.h
+++ b/arch/arm64/kvm/vgic/vgic.h
@@ -390,6 +390,7 @@ int vgic_v5_probe(const struct gic_kvm_info *info);
 void vgic_v5_reset(struct kvm_vcpu *vcpu);
 int vgic_v5_init(struct kvm *kvm);
 int kvm_vgic_v5_irs_init(struct kvm *kvm, unsigned int nr_spis);
+unsigned long vgic_v5_handle_guest_coresight_idreg_read(u32 offset);
 void vgic_v5_teardown(struct kvm *kvm);
 int vgic_v5_map_resources(struct kvm *kvm);
 void vgic_v5_set_ppi_ops(struct kvm_vcpu *vcpu, u32 vintid);
diff --git a/include/linux/irqchip/arm-gic-v5.h b/include/linux/irqchip/arm-gic-v5.h
index 299e75bc95255..ee3677413560f 100644
--- a/include/linux/irqchip/arm-gic-v5.h
+++ b/include/linux/irqchip/arm-gic-v5.h
@@ -498,6 +498,8 @@
 #define GICV5_CORESIGHT_CIDR1		0xfff4
 #define GICV5_CORESIGHT_CIDR2		0xfff8
 #define GICV5_CORESIGHT_CIDR3		0xfffc
+#define GICV5_CORESIGHT_PIDR_CIDR_SIZE	\
+	(GICV5_CORESIGHT_CIDR3 + 4 - GICV5_CORESIGHT_PIDR4)
 
 #define GICV5_CORESIGHT_DEVARCH_VAL			\
 	(FIELD_PREP(GENMASK(31, 21), 0x23b) |		\
-- 
2.34.1

[PATCH v2 04/16] irqchip/gic-v5: Add ITS MMIO register definitions

[Sascha Bischoff]

Introduce the GICv5 ITS MMIO register offsets and bit definitions in
one place before the virtual ITS starts consuming them.

Keeping the ITS register frame definitions together avoids spreading
register-definition churn across the subsequent KVM ITS emulation
patches.

Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
---
 include/linux/irqchip/arm-gic-v5.h | 126 +++++++++++++++++++++--------
 1 file changed, 92 insertions(+), 34 deletions(-)

diff --git a/include/linux/irqchip/arm-gic-v5.h b/include/linux/irqchip/arm-gic-v5.h
index ee3677413560f..6e8484fef9faf 100644
--- a/include/linux/irqchip/arm-gic-v5.h
+++ b/include/linux/irqchip/arm-gic-v5.h
@@ -364,50 +364,88 @@
 /*
  * ITS registers and tables structures
  */
-#define GICV5_ITS_IDR1		0x0004
-#define GICV5_ITS_IDR2		0x0008
-#define GICV5_ITS_CR0		0x0080
-#define GICV5_ITS_CR1		0x0084
-#define GICV5_ITS_DT_BASER	0x00c0
-#define GICV5_ITS_DT_CFGR	0x00d0
-#define GICV5_ITS_DIDR		0x0100
-#define GICV5_ITS_EIDR		0x0108
-#define GICV5_ITS_INV_EVENTR	0x010c
-#define GICV5_ITS_INV_DEVICER	0x0110
-#define GICV5_ITS_STATUSR	0x0120
-#define GICV5_ITS_SYNCR		0x0140
-#define GICV5_ITS_SYNC_STATUSR	0x0148
-
-#define GICV5_ITS_IDR1_L2SZ			GENMASK(10, 8)
-#define GICV5_ITS_IDR1_ITT_LEVELS		BIT(7)
-#define GICV5_ITS_IDR1_DT_LEVELS		BIT(6)
-#define GICV5_ITS_IDR1_DEVICEID_BITS		GENMASK(5, 0)
+#define GICV5_ITS_IDR0			0x0000
+#define GICV5_ITS_IDR1			0x0004
+#define GICV5_ITS_IDR2			0x0008
+#define GICV5_ITS_IIDR			0x0040
+#define GICV5_ITS_AIDR			0x0044
+#define GICV5_ITS_CR0			0x0080
+#define GICV5_ITS_CR1			0x0084
+#define GICV5_ITS_DT_BASER		0x00c0
+#define GICV5_ITS_DT_CFGR		0x00d0
+#define GICV5_ITS_DIDR			0x0100
+#define GICV5_ITS_EIDR			0x0108
+#define GICV5_ITS_INV_EVENTR		0x010c
+#define GICV5_ITS_INV_DEVICER		0x0110
+#define GICV5_ITS_READ_EVENTR		0x0114
+#define GICV5_ITS_READ_EVENT_DATAR	0x0118
+#define GICV5_ITS_STATUSR		0x0120
+#define GICV5_ITS_SYNCR			0x0140
+#define GICV5_ITS_SYNC_STATUSR		0x0148
+#define GICV5_ITS_GEN_EVENT_DIDR	0x0180
+#define GICV5_ITS_GEN_EVENT_EIDR	0x0188
+#define GICV5_ITS_GEN_EVENTR		0x018c
+#define GICV5_ITS_GEN_EVENT_STATUSR	0x0190
+#define GICV5_ITS_MEC_IDR		0x01c0
+#define GICV5_ITS_MEC_MECID_R		0x01c4
+#define GICV5_ITS_MPAM_IDR		0x0200
+#define GICV5_ITS_MPAM_PARTID_R		0x0204
+#define GICV5_ITS_SWERR_STATUSR		0x0240
+#define GICV5_ITS_SWERR_SYNDROMER0	0x0248
+#define GICV5_ITS_SWERR_SYNDROMER1	0x0250
+#define GICV5_ITS_MPAM_PMGR		0x0258
+
+#define GICV5_ITS_TRANSLATER		0x0000
+#define GICV5_ITS_RL_TRANSLATER		0x0008
+
+#define GICV5_ITS_IDR0_ITSID		GENMASK(31, 16)
+#define GICV5_ITS_IDR0_SWE		BIT(8)
+#define GICV5_ITS_IDR0_MPAM		BIT(7)
+#define GICV5_ITS_IDR0_MEC		BIT(6)
+#define GICV5_ITS_IDR0_PA_RANGE		GENMASK(5, 2)
+#define GICV5_ITS_IDR0_DOM		GENMASK(1, 0)
+
+#define GICV5_ITS_IDR0_DOMAIN_SECURE		0b00
+#define GICV5_ITS_IDR0_DOMAIN_NON_SECURE	0b01
+#define GICV5_ITS_IDR0_DOMAIN_EL3		0b10
+#define GICV5_ITS_IDR0_DOMAIN_REALM		0b11
+
+#define GICV5_ITS_IDR1_L2SZ		GENMASK(10, 8)
+#define GICV5_ITS_IDR1_ITT_LEVELS	BIT(7)
+#define GICV5_ITS_IDR1_DT_LEVELS	BIT(6)
+#define GICV5_ITS_IDR1_DEVICEID_BITS	GENMASK(5, 0)
 
 #define GICV5_ITS_IDR1_L2SZ_SUPPORT_4KB(r)	FIELD_GET(BIT(8), (r))
 #define GICV5_ITS_IDR1_L2SZ_SUPPORT_16KB(r)	FIELD_GET(BIT(9), (r))
 #define GICV5_ITS_IDR1_L2SZ_SUPPORT_64KB(r)	FIELD_GET(BIT(10), (r))
 
-#define GICV5_ITS_IDR2_XDMN_EVENTs		GENMASK(6, 5)
-#define GICV5_ITS_IDR2_EVENTID_BITS		GENMASK(4, 0)
+#define GICV5_ITS_IDR2_XDMN_EVENTS	GENMASK(6, 5)
+#define GICV5_ITS_IDR2_EVENTID_BITS	GENMASK(4, 0)
 
-#define GICV5_ITS_CR0_IDLE			BIT(1)
-#define GICV5_ITS_CR0_ITSEN			BIT(0)
+#define GICV5_ITS_IIDR_PRODUCT_ID	GENMASK(31, 20)
+#define GICV5_ITS_IIDR_VARIANT		GENMASK(19, 16)
+#define GICV5_ITS_IIDR_REVISION		GENMASK(15, 12)
+#define GICV5_ITS_IIDR_IMPLEMENTER	GENMASK(11, 0)
 
-#define GICV5_ITS_CR1_ITT_RA			BIT(7)
-#define GICV5_ITS_CR1_DT_RA			BIT(6)
-#define GICV5_ITS_CR1_IC			GENMASK(5, 4)
-#define GICV5_ITS_CR1_OC			GENMASK(3, 2)
-#define GICV5_ITS_CR1_SH			GENMASK(1, 0)
+#define GICV5_ITS_AIDR_COMPONENT	GENMASK(11, 8)
+#define GICV5_ITS_AIDR_ARCHMAJORREV	GENMASK(7, 4)
+#define GICV5_ITS_AIDR_ARCHMINORREV	GENMASK(3, 0)
 
-#define GICV5_ITS_DT_CFGR_STRUCTURE		BIT(16)
-#define GICV5_ITS_DT_CFGR_L2SZ			GENMASK(7, 6)
-#define GICV5_ITS_DT_CFGR_DEVICEID_BITS		GENMASK(5, 0)
+#define GICV5_ITS_CR0_IDLE		BIT(1)
+#define GICV5_ITS_CR0_ITSEN		BIT(0)
 
-#define GICV5_ITS_DT_BASER_ADDR_MASK		GENMASK_ULL(55, 3)
+#define GICV5_ITS_CR1_ITT_RA		BIT(7)
+#define GICV5_ITS_CR1_DT_RA		BIT(6)
+#define GICV5_ITS_CR1_IC		GENMASK(5, 4)
+#define GICV5_ITS_CR1_OC		GENMASK(3, 2)
+#define GICV5_ITS_CR1_SH		GENMASK(1, 0)
 
-#define GICV5_ITS_INV_DEVICER_I			BIT(31)
-#define GICV5_ITS_INV_DEVICER_EVENTID_BITS	GENMASK(5, 1)
-#define GICV5_ITS_INV_DEVICER_L1		BIT(0)
+#define GICV5_ITS_DT_BASER_ADDR_MASK	GENMASK_ULL(55, 3)
+#define GICV5_ITS_DT_BASER_ADDR_SHIFT	3ULL
+
+#define GICV5_ITS_DT_CFGR_STRUCTURE	BIT(16)
+#define GICV5_ITS_DT_CFGR_L2SZ		GENMASK(7, 6)
+#define GICV5_ITS_DT_CFGR_DEVICEID_BITS	GENMASK(5, 0)
 
 #define GICV5_ITS_DIDR_DEVICEID			GENMASK_ULL(31, 0)
 
@@ -417,6 +455,17 @@
 #define GICV5_ITS_INV_EVENTR_ITT_L2SZ		GENMASK(2, 1)
 #define GICV5_ITS_INV_EVENTR_L1			BIT(0)
 
+#define GICV5_ITS_INV_DEVICER_I			BIT(31)
+#define GICV5_ITS_INV_DEVICER_EVENTID_BITS	GENMASK(5, 1)
+#define GICV5_ITS_INV_DEVICER_L1		BIT(0)
+
+#define GICV5_ITS_READ_EVENTR_R			BIT(31)
+
+#define GICV5_ITS_READ_EVENT_DATAR_VIRT		BIT_ULL(63)
+#define GICV5_ITS_READ_EVENT_DATAR_VM_ID	GENMASK_ULL(47, 32)
+#define GICV5_ITS_READ_EVENT_DATAR_VALID	BIT_ULL(31)
+#define GICV5_ITS_READ_EVENT_DATAR_LPI_ID	GENMASK_ULL(23, 0)
+
 #define GICV5_ITS_STATUSR_IDLE			BIT(0)
 
 #define GICV5_ITS_SYNCR_SYNC			BIT_ULL(63)
@@ -425,6 +474,15 @@
 
 #define GICV5_ITS_SYNC_STATUSR_IDLE		BIT(0)
 
+#define GICV5_ITS_GEN_EVENT_DIDR_DEVICEID	GENMASK_ULL(31, 0)
+
+#define GICV5_ITS_GEN_EVENT_EIDR_EVENTID	GENMASK(15, 0)
+
+#define GICV5_ITS_GEN_EVENTR_R			BIT(31)
+#define GICV5_ITS_GEN_EVENTR_TARGET_DOMAIN	GENMASK(1, 0)
+
+#define GICV5_ITS_GEN_EVENT_STATUSR_IDLE	BIT(0)
+
 #define GICV5_DTL1E_VALID			BIT_ULL(0)
 /* Note that there is no shift for the address by design */
 #define GICV5_DTL1E_L2_ADDR_MASK		GENMASK_ULL(55, 3)
-- 
2.34.1

[PATCH v2 05/16] KVM: arm64: gic-v5: Add ITS device skeleton

[Sascha Bischoff]

Introduce the basic KVM device plumbing for a GICv5 ITS. This adds the
GICv5 ITS device type and address attribute, registers the device ops,
and wires a GICv5 ITS IODEV into the generic VGIC MMIO dispatch path.

Add struct vgic_v5_its to hold the ITS base address, the IODEV plumbing,
the backing KVM device pointer, and the general ITS mutex. Allocate this
state from the KVM device create path, initialise the mutex and base
address, and mark the VM as having an ITS. GICv5 MSIs require a DeviceID,
so set msis_require_devid when the ITS is created.

Record the VGIC model required by an ITS and reject mismatched device
creation in either order. Check GICv3 ITS creation as well, so the two
ITS models cannot be combined in one VM.

The MMIO region is registered once userspace provides the ITS base
address through KVM_VGIC_V5_ADDR_TYPE_ITS. For now the IODEV only
provides a RAZ/WI placeholder region; the actual ITS register emulation
is added separately.

Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
---
 arch/arm64/include/uapi/asm/kvm.h     |   2 +
 arch/arm64/kvm/Makefile               |   1 +
 arch/arm64/kvm/vgic/vgic-its-v5.c     | 186 ++++++++++++++++++++++++++
 arch/arm64/kvm/vgic/vgic-kvm-device.c |   4 +
 arch/arm64/kvm/vgic/vgic-mmio-v3.c    |   7 -
 arch/arm64/kvm/vgic/vgic-mmio.c       |  13 ++
 arch/arm64/kvm/vgic/vgic.h            |   1 +
 include/kvm/arm_vgic.h                |  16 ++-
 include/uapi/linux/kvm.h              |   2 +
 tools/include/uapi/linux/kvm.h        |   2 +
 10 files changed, 226 insertions(+), 8 deletions(-)
 create mode 100644 arch/arm64/kvm/vgic/vgic-its-v5.c

diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 65ee09bd14636..8c701173a2d88 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -99,8 +99,10 @@ struct kvm_regs {
 
 /* Supported VGICv5 address types  */
 #define KVM_VGIC_V5_ADDR_TYPE_IRS	6
+#define KVM_VGIC_V5_ADDR_TYPE_ITS	7
 
 #define KVM_VGIC_V5_IRS_SIZE		(2 * SZ_64K)
+#define KVM_VGIC_V5_ITS_SIZE		(2 * SZ_64K)
 
 #define KVM_ARM_VCPU_POWER_OFF		0 /* CPU is started in OFF state */
 #define KVM_ARM_VCPU_EL1_32BIT		1 /* CPU running a 32bit VM */
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index 7aaeeb84e788e..47923e1047f16 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -25,6 +25,7 @@ kvm-y += arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \
 	 vgic/vgic-mmio-v3.o vgic/vgic-kvm-device.o \
 	 vgic/vgic-its.o vgic/vgic-debug.o vgic/vgic-v3-nested.o \
 	 vgic/vgic-v5.o vgic/vgic-v5-tables.o vgic/vgic-irs-v5.o \
+	 vgic/vgic-its-v5.o \
 	 vgic-sys-reg-v5.o
 
 kvm-$(CONFIG_HW_PERF_EVENTS)  += pmu-emul.o pmu.o
diff --git a/arch/arm64/kvm/vgic/vgic-its-v5.c b/arch/arm64/kvm/vgic/vgic-its-v5.c
new file mode 100644
index 0000000000000..5790b029f09b6
--- /dev/null
+++ b/arch/arm64/kvm/vgic/vgic-its-v5.c
@@ -0,0 +1,186 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2025 ARM Limited, All Rights Reserved.
+ */
+
+#include <linux/kvm_host.h>
+#include <linux/slab.h>
+
+#include <asm/kvm_mmu.h>
+
+#include <kvm/arm_vgic.h>
+
+#include "vgic.h"
+#include "vgic-mmio.h"
+
+static unsigned long vgic_v5_mmio_read_its_raz(struct kvm *kvm, void *dev,
+					       gpa_t addr, unsigned int len)
+{
+	return 0;
+}
+
+static void vgic_v5_mmio_write_its_wi(struct kvm *kvm, void *dev, gpa_t addr,
+				      unsigned int len, unsigned long val)
+{
+}
+
+#define REGISTER_ITS_V5_DESC(off, rd, wr, length, acc)                 \
+	{                                                              \
+		.reg_offset = off, .len = length, .access_flags = acc, \
+		.its_read = rd, .its_write = wr,                       \
+	}
+
+static const struct vgic_register_region vgic_v5_its_registers[] = {
+	REGISTER_ITS_V5_DESC(0, vgic_v5_mmio_read_its_raz,
+			     vgic_v5_mmio_write_its_wi, KVM_VGIC_V5_ITS_SIZE,
+			     VGIC_ACCESS_32bit),
+};
+
+static int vgic_v5_its_create(struct kvm_device *dev, u32 type)
+{
+	struct vgic_v5_its *its;
+
+	if (type != KVM_DEV_TYPE_ARM_VGIC_V5_ITS)
+		return -ENODEV;
+
+	its = kzalloc_obj(*its, GFP_KERNEL_ACCOUNT);
+	if (!its)
+		return -ENOMEM;
+
+	mutex_init(&its->its_lock);
+
+	guard(mutex)(&dev->kvm->arch.config_lock);
+	if ((irqchip_in_kernel(dev->kvm) && !vgic_is_v5(dev->kvm)) ||
+	    (dev->kvm->arch.vgic.its_model &&
+	     dev->kvm->arch.vgic.its_model != KVM_DEV_TYPE_ARM_VGIC_V5)) {
+		kfree(its);
+		return -ENODEV;
+	}
+
+	its->vgic_v5_its_base = VGIC_ADDR_UNDEF;
+	its->dev = dev;
+
+	dev->kvm->arch.vgic.msis_require_devid = true;
+	dev->kvm->arch.vgic.its_model = KVM_DEV_TYPE_ARM_VGIC_V5;
+	dev->private = its;
+
+	return 0;
+}
+
+static void vgic_v5_its_destroy(struct kvm_device *kvm_dev)
+{
+	struct vgic_v5_its *its = kvm_dev->private;
+
+	kfree(its);
+	kfree(kvm_dev);
+}
+
+static int vgic_v5_register_its_iodev(struct kvm *kvm, struct vgic_v5_its *its,
+				      u64 addr)
+{
+	struct vgic_io_device *iodev = &its->iodev;
+	int ret;
+
+	mutex_lock(&kvm->slots_lock);
+	if (!IS_VGIC_ADDR_UNDEF(its->vgic_v5_its_base)) {
+		ret = -EBUSY;
+		goto out;
+	}
+
+	if (!IS_ALIGNED(addr, SZ_64K)) {
+		kvm_err("ITS Base address is not aligned to 64k\n");
+		ret = -EINVAL;
+		goto out;
+	}
+
+	its->vgic_v5_its_base = addr;
+	iodev->base_addr = addr;
+	iodev->regions = vgic_v5_its_registers;
+	iodev->nr_regions = ARRAY_SIZE(vgic_v5_its_registers);
+	iodev->iodev_type = IODEV_GICV5_ITS;
+	iodev->redist_vcpu = NULL;
+	iodev->v5its = its;
+	kvm_iodevice_init(&iodev->dev, &kvm_io_gic_ops);
+
+	ret = kvm_io_bus_register_dev(kvm, KVM_MMIO_BUS, iodev->base_addr,
+				      KVM_VGIC_V5_ITS_SIZE, &iodev->dev);
+out:
+	mutex_unlock(&kvm->slots_lock);
+
+	return ret;
+}
+
+static int vgic_v5_its_set_attr(struct kvm_device *dev,
+				struct kvm_device_attr *attr)
+{
+	struct vgic_v5_its *its = dev->private;
+	u64 __user *uaddr = (u64 __user *)(long)attr->addr;
+	unsigned long type = (unsigned long)attr->attr;
+	u64 addr;
+	int ret;
+
+	if (attr->group != KVM_DEV_ARM_VGIC_GRP_ADDR)
+		return -ENXIO;
+
+	if (type != KVM_VGIC_V5_ADDR_TYPE_ITS)
+		return -ENODEV;
+
+	if (copy_from_user(&addr, uaddr, sizeof(addr)))
+		return -EFAULT;
+
+	ret = vgic_check_iorange(dev->kvm, its->vgic_v5_its_base, addr,
+				 SZ_64K, KVM_VGIC_V5_ITS_SIZE);
+	if (ret)
+		return ret;
+
+	return vgic_v5_register_its_iodev(dev->kvm, its, addr);
+}
+
+static int vgic_v5_its_get_attr(struct kvm_device *dev,
+				struct kvm_device_attr *attr)
+{
+	struct vgic_v5_its *its = dev->private;
+	u64 __user *uaddr = (u64 __user *)(long)attr->addr;
+	unsigned long type = (unsigned long)attr->attr;
+	u64 addr = its->vgic_v5_its_base;
+
+	if (attr->group != KVM_DEV_ARM_VGIC_GRP_ADDR)
+		return -ENXIO;
+
+	if (type != KVM_VGIC_V5_ADDR_TYPE_ITS)
+		return -ENODEV;
+
+	if (copy_to_user(uaddr, &addr, sizeof(addr)))
+		return -EFAULT;
+
+	return 0;
+}
+
+static int vgic_v5_its_has_attr(struct kvm_device *dev,
+				struct kvm_device_attr *attr)
+{
+	if (attr->group != KVM_DEV_ARM_VGIC_GRP_ADDR)
+		return -ENXIO;
+
+	switch (attr->attr) {
+	case KVM_VGIC_V5_ADDR_TYPE_ITS:
+		return 0;
+	default:
+		return -ENXIO;
+	}
+}
+
+static struct kvm_device_ops kvm_arm_vgic_v5_its_ops = {
+	.name = "kvm-arm-vgic-v5-its",
+	.create = vgic_v5_its_create,
+	.destroy = vgic_v5_its_destroy,
+	.set_attr = vgic_v5_its_set_attr,
+	.get_attr = vgic_v5_its_get_attr,
+	.has_attr = vgic_v5_its_has_attr,
+};
+
+int kvm_vgic_v5_register_its_device(void)
+{
+	return kvm_register_device_ops(&kvm_arm_vgic_v5_its_ops,
+				       KVM_DEV_TYPE_ARM_VGIC_V5_ITS);
+}
diff --git a/arch/arm64/kvm/vgic/vgic-kvm-device.c b/arch/arm64/kvm/vgic/vgic-kvm-device.c
index 1c205cb1361fd..e7630f68e2a74 100644
--- a/arch/arm64/kvm/vgic/vgic-kvm-device.c
+++ b/arch/arm64/kvm/vgic/vgic-kvm-device.c
@@ -369,6 +369,10 @@ int kvm_register_vgic_device(unsigned long type)
 	case KVM_DEV_TYPE_ARM_VGIC_V5:
 		ret = kvm_register_device_ops(&kvm_arm_vgic_v5_ops,
 					      KVM_DEV_TYPE_ARM_VGIC_V5);
+		if (ret)
+			break;
+
+		ret = kvm_vgic_v5_register_its_device();
 		break;
 	}
 
diff --git a/arch/arm64/kvm/vgic/vgic-mmio-v3.c b/arch/arm64/kvm/vgic/vgic-mmio-v3.c
index f9ff553bbbf16..5c7be55ba95d6 100644
--- a/arch/arm64/kvm/vgic/vgic-mmio-v3.c
+++ b/arch/arm64/kvm/vgic/vgic-mmio-v3.c
@@ -38,13 +38,6 @@ u64 update_64bit_reg(u64 reg, unsigned int offset, unsigned int len,
 	return reg | ((u64)val << lower);
 }
 
-bool vgic_has_its(struct kvm *kvm)
-{
-	struct vgic_dist *dist = &kvm->arch.vgic;
-
-	return dist->its_model && dist->its_model == dist->vgic_model;
-}
-
 bool vgic_supports_direct_msis(struct kvm *kvm)
 {
 	/*
diff --git a/arch/arm64/kvm/vgic/vgic-mmio.c b/arch/arm64/kvm/vgic/vgic-mmio.c
index fddb9da0403d5..5371d0b8bd0c6 100644
--- a/arch/arm64/kvm/vgic/vgic-mmio.c
+++ b/arch/arm64/kvm/vgic/vgic-mmio.c
@@ -1068,6 +1068,9 @@ static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
 	case IODEV_GICV5_IRS:
 		data = region->read(vcpu, addr, len);
 		break;
+	case IODEV_GICV5_ITS:
+		data = region->its_read(vcpu->kvm, iodev->v5its, addr, len);
+		break;
 	}
 
 	vgic_data_host_to_mmio_bus(val, len, data);
@@ -1101,6 +1104,9 @@ static int dispatch_mmio_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
 	case IODEV_GICV5_IRS:
 		region->write(vcpu, addr, len, data);
 		break;
+	case IODEV_GICV5_ITS:
+		region->its_write(vcpu->kvm, iodev->v5its, addr, len, data);
+		break;
 	}
 
 	return 0;
@@ -1135,3 +1141,10 @@ int vgic_register_dist_iodev(struct kvm *kvm, gpa_t dist_base_address,
 	return kvm_io_bus_register_dev(kvm, KVM_MMIO_BUS, dist_base_address,
 				       len, &io_device->dev);
 }
+
+bool vgic_has_its(struct kvm *kvm)
+{
+	struct vgic_dist *dist = &kvm->arch.vgic;
+
+	return dist->its_model && dist->its_model == dist->vgic_model;
+}
diff --git a/arch/arm64/kvm/vgic/vgic.h b/arch/arm64/kvm/vgic/vgic.h
index f95eec6a01f63..9e1cbcdedf3f5 100644
--- a/arch/arm64/kvm/vgic/vgic.h
+++ b/arch/arm64/kvm/vgic/vgic.h
@@ -363,6 +363,7 @@ void vgic_v3_put(struct kvm_vcpu *vcpu);
 
 bool vgic_has_its(struct kvm *kvm);
 int kvm_vgic_register_its_device(void);
+int kvm_vgic_v5_register_its_device(void);
 void vgic_enable_lpis(struct kvm_vcpu *vcpu);
 void vgic_flush_pending_lpis(struct kvm_vcpu *vcpu);
 int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi);
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 4b2c41917b77c..4f9909d6c5b11 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -325,7 +325,8 @@ enum iodev_type {
 	IODEV_DIST,
 	IODEV_REDIST,
 	IODEV_ITS,
-	IODEV_GICV5_IRS
+	IODEV_GICV5_IRS,
+	IODEV_GICV5_ITS
 };
 
 struct vgic_io_device {
@@ -333,6 +334,7 @@ struct vgic_io_device {
 	union {
 		struct kvm_vcpu *redist_vcpu;
 		struct vgic_its *its;
+		struct vgic_v5_its *v5its;
 	};
 	const struct vgic_register_region *regions;
 	enum iodev_type iodev_type;
@@ -497,6 +499,18 @@ struct vgic_v5_irs {
 	bool lpi_ist_restore_pending;
 };
 
+/*** GICv5 ***/
+struct vgic_v5_its {
+	/* base addresses in guest physical address space: */
+	gpa_t vgic_v5_its_base;
+
+	struct vgic_io_device iodev;
+	struct kvm_device *dev;
+
+	/* Protects the device in general */
+	struct mutex		its_lock;
+};
+
 struct vgic_dist {
 	bool			in_kernel;
 	bool			ready;
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 8dff2fc1972e9..c0e11993c7c49 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -1239,6 +1239,8 @@ enum kvm_device_type {
 #define KVM_DEV_TYPE_LOONGARCH_DMSINTC	KVM_DEV_TYPE_LOONGARCH_DMSINTC
 	KVM_DEV_TYPE_ARM_VGIC_V5,
 #define KVM_DEV_TYPE_ARM_VGIC_V5	KVM_DEV_TYPE_ARM_VGIC_V5
+	KVM_DEV_TYPE_ARM_VGIC_V5_ITS,
+#define KVM_DEV_TYPE_ARM_VGIC_V5_ITS	KVM_DEV_TYPE_ARM_VGIC_V5_ITS
 
 	KVM_DEV_TYPE_MAX,
 
diff --git a/tools/include/uapi/linux/kvm.h b/tools/include/uapi/linux/kvm.h
index ac2d77d149635..4e16c55902f5e 100644
--- a/tools/include/uapi/linux/kvm.h
+++ b/tools/include/uapi/linux/kvm.h
@@ -1238,6 +1238,8 @@ enum kvm_device_type {
 #define KVM_DEV_TYPE_LOONGARCH_DMSINTC	KVM_DEV_TYPE_LOONGARCH_DMSINTC
 	KVM_DEV_TYPE_ARM_VGIC_V5,
 #define KVM_DEV_TYPE_ARM_VGIC_V5	KVM_DEV_TYPE_ARM_VGIC_V5
+	KVM_DEV_TYPE_ARM_VGIC_V5_ITS,
+#define KVM_DEV_TYPE_ARM_VGIC_V5_ITS	KVM_DEV_TYPE_ARM_VGIC_V5_ITS
 
 	KVM_DEV_TYPE_MAX,
 
-- 
2.34.1

[PATCH v2 06/16] KVM: arm64: gic-v5: Add ITS MMIO state emulation

[Sascha Bischoff]

In order to expose a virtual GICv5 ITS to the guest, KVM needs to
emulate the ITS CONFIG_FRAME and its translation-control registers.
Replace the placeholder ITS IODEV region with the GICv5 ITS register
layout and handlers for the architected MMIO state.

Provide read/write emulation for the ITS ID registers, IIDR, AIDR,
CR0, CR1, DT_BASER, DT_CFGR, DIDR, EIDR, invalidation registers,
READ_EVENT, GEN_EVENT, status registers, and the RAZ/WI MMIO
registers that KVM does not implement. Reuse the shared CoreSight
identification-register helper so the ITS reports the same component
identification values as the IRS.

At this stage the translation-triggering registers only record state.
Walking the guest's DT/ITT, generating READ_EVENT results, and
injecting GEN_EVENT LPIs are added by later commits.

Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
---
 arch/arm64/kvm/vgic/vgic-its-v5.c | 415 +++++++++++++++++++++++++++++-
 include/kvm/arm_vgic.h            |  77 ++++++
 2 files changed, 490 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kvm/vgic/vgic-its-v5.c b/arch/arm64/kvm/vgic/vgic-its-v5.c
index 5790b029f09b6..f04161c315b78 100644
--- a/arch/arm64/kvm/vgic/vgic-its-v5.c
+++ b/arch/arm64/kvm/vgic/vgic-its-v5.c
@@ -6,19 +6,286 @@
 #include <linux/kvm_host.h>
 #include <linux/slab.h>
 
+#include <asm/kvm_arm.h>
 #include <asm/kvm_mmu.h>
 
 #include <kvm/arm_vgic.h>
 
+#include <linux/irqchip/arm-gic-v5.h>
+
 #include "vgic.h"
 #include "vgic-mmio.h"
 
+#define VITS_V5_DEVICE_ID_BITS		16
+#define VITS_V5_EVENT_ID_BITS		16
+
+static unsigned long vgic_v5_mmio_read_its_misc(struct kvm *kvm, void *dev,
+						gpa_t addr, unsigned int len)
+{
+	struct vgic_v5_its *its = dev;
+	u64 value = 0;
+	size_t offset = addr & (SZ_64K - 1);
+
+	guard(mutex)(&its->its_lock);
+
+	switch (offset) {
+	case GICV5_ITS_IDR0:
+		value = FIELD_PREP(GICV5_ITS_IDR0_ITSID, its->idr0.its_id);
+		value |= FIELD_PREP(GICV5_ITS_IDR0_SWE, its->idr0.swe);
+		value |= FIELD_PREP(GICV5_ITS_IDR0_MPAM, its->idr0.mpam);
+		value |= FIELD_PREP(GICV5_ITS_IDR0_MEC, its->idr0.mec);
+		value |= FIELD_PREP(GICV5_ITS_IDR0_PA_RANGE,
+				    its->idr0.pa_range);
+		value |= FIELD_PREP(GICV5_ITS_IDR0_DOM, its->idr0.domain);
+		break;
+	case GICV5_ITS_IDR1:
+		value = FIELD_PREP(GICV5_ITS_IDR1_L2SZ, its->idr1.l2sz);
+		value |= FIELD_PREP(GICV5_ITS_IDR1_ITT_LEVELS,
+				    its->idr1.itt_levels);
+		value |= FIELD_PREP(GICV5_ITS_IDR1_DT_LEVELS,
+				    its->idr1.dt_levels);
+		value |= FIELD_PREP(GICV5_ITS_IDR1_DEVICEID_BITS,
+				    its->idr1.device_id_bits);
+		break;
+	case GICV5_ITS_IDR2:
+		value = FIELD_PREP(GICV5_ITS_IDR2_XDMN_EVENTS,
+				   its->idr2.xdmn_events);
+		value |= FIELD_PREP(GICV5_ITS_IDR2_EVENTID_BITS,
+				    its->idr2.event_id_bits);
+		break;
+	case GICV5_ITS_IIDR:
+		value = FIELD_PREP(GICV5_ITS_IIDR_PRODUCT_ID, PRODUCT_ID_KVM);
+		value |= FIELD_PREP(GICV5_ITS_IIDR_VARIANT, 0);
+		value |= FIELD_PREP(GICV5_ITS_IIDR_REVISION, 0);
+		value |= FIELD_PREP(GICV5_ITS_IIDR_IMPLEMENTER,
+				    IMPLEMENTER_ARM);
+		break;
+	case GICV5_ITS_AIDR:
+		value = FIELD_PREP(GICV5_ITS_AIDR_COMPONENT,
+				   GICV5_AIDR_COMPONENT_ITS);
+		value |= FIELD_PREP(GICV5_ITS_AIDR_ARCHMAJORREV,
+				    GICV5_AIDR_ARCH_MAJ_REV_V5);
+		value |= FIELD_PREP(GICV5_ITS_AIDR_ARCHMINORREV,
+				    GICV5_AIDR_ARCH_MIN_REV_V0);
+		break;
+	case GICV5_ITS_CR0:
+		value = GICV5_ITS_CR0_IDLE;
+		value |= FIELD_PREP(GICV5_ITS_CR0_ITSEN, its->enabled);
+		break;
+	case GICV5_ITS_CR1:
+		value = FIELD_PREP(GICV5_ITS_CR1_ITT_RA, its->cr1.itt_ra);
+		value |= FIELD_PREP(GICV5_ITS_CR1_DT_RA, its->cr1.dt_ra);
+		value |= FIELD_PREP(GICV5_ITS_CR1_IC, its->cr1.ic);
+		value |= FIELD_PREP(GICV5_ITS_CR1_OC, its->cr1.oc);
+		value |= FIELD_PREP(GICV5_ITS_CR1_SH, its->cr1.sh);
+		break;
+	default:
+		return 0;
+	}
+
+	return value;
+}
+
+static void vgic_v5_mmio_write_its_misc(struct kvm *kvm, void *dev, gpa_t addr,
+					unsigned int len, unsigned long val)
+{
+	struct vgic_v5_its *its = dev;
+	size_t offset = addr & (SZ_64K - 1);
+
+	guard(mutex)(&its->its_lock);
+
+	switch (offset) {
+	case GICV5_ITS_CR0:
+		its->enabled = FIELD_GET(GICV5_ITS_CR0_ITSEN, val);
+		return;
+	case GICV5_ITS_CR1:
+		its->cr1.sh = FIELD_GET(GICV5_ITS_CR1_SH, val);
+		its->cr1.oc = FIELD_GET(GICV5_ITS_CR1_OC, val);
+		its->cr1.ic = FIELD_GET(GICV5_ITS_CR1_IC, val);
+		its->cr1.dt_ra = FIELD_GET(GICV5_ITS_CR1_DT_RA, val);
+		its->cr1.itt_ra = FIELD_GET(GICV5_ITS_CR1_ITT_RA, val);
+		return;
+	default:
+		return;
+	}
+}
+
+static void vgic_v5_write_its_dt_reg(struct vgic_v5_its *its, size_t offset,
+				     unsigned long val)
+{
+	switch (offset) {
+	case GICV5_ITS_DT_CFGR:
+		its->dt_cfgr.device_id_bits =
+			FIELD_GET(GICV5_ITS_DT_CFGR_DEVICEID_BITS, val);
+		its->dt_cfgr.l2sz =
+			FIELD_GET(GICV5_ITS_DT_CFGR_L2SZ, val);
+		its->dt_cfgr.structure =
+			FIELD_GET(GICV5_ITS_DT_CFGR_STRUCTURE, val);
+		return;
+	case GICV5_ITS_DT_BASER:
+		its->dt_baser.addr =
+			FIELD_GET(GICV5_ITS_DT_BASER_ADDR_MASK, val)
+			<< GICV5_ITS_DT_BASER_ADDR_SHIFT;
+		return;
+	}
+}
+
+static u64 vgic_v5_read_its_dt_baser(struct vgic_v5_its *its)
+{
+	return FIELD_PREP(GICV5_ITS_DT_BASER_ADDR_MASK,
+			  its->dt_baser.addr >> GICV5_ITS_DT_BASER_ADDR_SHIFT);
+}
+
+static unsigned long vgic_v5_mmio_read_its_dt_itt(struct kvm *kvm, void *dev,
+						  gpa_t addr, unsigned int len)
+{
+	struct vgic_v5_its *its = dev;
+	u64 value = 0;
+	size_t offset = addr & (SZ_64K - 1);
+
+	guard(mutex)(&its->its_lock);
+
+	switch (offset) {
+	case GICV5_ITS_DT_CFGR:
+		value = FIELD_PREP(GICV5_ITS_DT_CFGR_STRUCTURE,
+				   its->dt_cfgr.structure);
+		value |= FIELD_PREP(GICV5_ITS_DT_CFGR_L2SZ,
+				    its->dt_cfgr.l2sz);
+		value |= FIELD_PREP(GICV5_ITS_DT_CFGR_DEVICEID_BITS,
+				    its->dt_cfgr.device_id_bits);
+		break;
+	case GICV5_ITS_DT_BASER:
+	case GICV5_ITS_DT_BASER + 4:
+		value = vgic_v5_read_its_dt_baser(its);
+		return extract_bytes(value, offset & 4, len);
+	case GICV5_ITS_DIDR:
+	case GICV5_ITS_DIDR + 4:
+		value = FIELD_PREP(GICV5_ITS_DIDR_DEVICEID,
+				   its->didr.device_id);
+		return extract_bytes(value, offset & 4, len);
+	case GICV5_ITS_EIDR:
+		value = FIELD_PREP(GICV5_ITS_EIDR_EVENTID,
+				   its->eidr.event_id);
+		break;
+	case GICV5_ITS_STATUSR:
+		value = GICV5_ITS_STATUSR_IDLE;
+		break;
+	case GICV5_ITS_SYNC_STATUSR:
+		value = GICV5_ITS_SYNC_STATUSR_IDLE;
+		break;
+	case GICV5_ITS_READ_EVENT_DATAR:
+	case GICV5_ITS_READ_EVENT_DATAR + 4:
+		value = FIELD_PREP(GICV5_ITS_READ_EVENT_DATAR_VIRT,
+				   its->read_event_datar.virt);
+		value |= FIELD_PREP(GICV5_ITS_READ_EVENT_DATAR_VM_ID,
+				    its->read_event_datar.vm_id);
+		value |= FIELD_PREP(GICV5_ITS_READ_EVENT_DATAR_VALID,
+				    its->read_event_datar.valid);
+		value |= FIELD_PREP(GICV5_ITS_READ_EVENT_DATAR_LPI_ID,
+				    its->read_event_datar.lpi_id);
+		return extract_bytes(value, offset & 4, len);
+	case GICV5_ITS_GEN_EVENT_EIDR:
+		value = FIELD_PREP(GICV5_ITS_GEN_EVENT_EIDR_EVENTID,
+				   its->gen_event_eidr.event_id);
+		break;
+	case GICV5_ITS_GEN_EVENT_DIDR:
+	case GICV5_ITS_GEN_EVENT_DIDR + 4:
+		value = FIELD_PREP(GICV5_ITS_GEN_EVENT_DIDR_DEVICEID,
+				   its->gen_event_didr.device_id);
+		return extract_bytes(value, offset & 4, len);
+	case GICV5_ITS_GEN_EVENT_STATUSR:
+		value = GICV5_ITS_GEN_EVENT_STATUSR_IDLE;
+		break;
+	default:
+		return 0;
+	}
+
+	return value;
+}
+
+static void vgic_v5_mmio_write_its_dt_itt(struct kvm *kvm, void *dev,
+					  gpa_t addr, unsigned int len,
+					  unsigned long val)
+{
+	struct vgic_v5_its *its = dev;
+	size_t offset = addr & (SZ_64K - 1);
+
+	guard(mutex)(&its->its_lock);
+
+	switch (offset) {
+	case GICV5_ITS_DT_CFGR:
+		if (!its->enabled)
+			vgic_v5_write_its_dt_reg(its, offset, val);
+		return;
+	case GICV5_ITS_DT_BASER:
+	case GICV5_ITS_DT_BASER + 4:
+		if (!its->enabled) {
+			u64 baser = vgic_v5_read_its_dt_baser(its);
+
+			baser = update_64bit_reg(baser, offset & 4, len, val);
+			vgic_v5_write_its_dt_reg(its, GICV5_ITS_DT_BASER,
+						 baser);
+		}
+		return;
+	case GICV5_ITS_INV_DEVICER:
+		its->inv_devicer.l1 =
+			FIELD_GET(GICV5_ITS_INV_DEVICER_L1, val);
+		its->inv_devicer.event_id_bits =
+			FIELD_GET(GICV5_ITS_INV_DEVICER_EVENTID_BITS, val);
+		its->inv_devicer.i =
+			FIELD_GET(GICV5_ITS_INV_DEVICER_I, val);
+		return;
+	case GICV5_ITS_DIDR:
+	case GICV5_ITS_DIDR + 4:
+		if (offset & 4)
+			return;
+		its->didr.device_id = FIELD_GET(GICV5_ITS_DIDR_DEVICEID, val);
+		return;
+	case GICV5_ITS_EIDR:
+		its->eidr.event_id = FIELD_GET(GICV5_ITS_EIDR_EVENTID, val);
+		return;
+	case GICV5_ITS_INV_EVENTR:
+		its->inv_eventr.l1 =
+			FIELD_GET(GICV5_ITS_INV_EVENTR_L1, val);
+		its->inv_eventr.itt_l2sz =
+			FIELD_GET(GICV5_ITS_INV_EVENTR_ITT_L2SZ, val);
+		its->inv_eventr.i = FIELD_GET(GICV5_ITS_INV_EVENTR_I, val);
+		return;
+	case GICV5_ITS_SYNCR:
+	case GICV5_ITS_READ_EVENTR:
+		return;
+	case GICV5_ITS_GEN_EVENT_EIDR:
+		its->gen_event_eidr.event_id =
+			FIELD_GET(GICV5_ITS_GEN_EVENT_EIDR_EVENTID, val);
+		return;
+	case GICV5_ITS_GEN_EVENT_DIDR:
+	case GICV5_ITS_GEN_EVENT_DIDR + 4:
+		if (offset & 4)
+			return;
+		its->gen_event_didr.device_id =
+			FIELD_GET(GICV5_ITS_GEN_EVENT_DIDR_DEVICEID, val);
+		return;
+	case GICV5_ITS_GEN_EVENTR:
+		return;
+	default:
+		return;
+	}
+}
+
 static unsigned long vgic_v5_mmio_read_its_raz(struct kvm *kvm, void *dev,
 					       gpa_t addr, unsigned int len)
 {
 	return 0;
 }
 
+static unsigned long vgic_v5_its_coresight_read(struct kvm *kvm, void *dev,
+						gpa_t addr, unsigned int len)
+{
+	u32 offset = addr & (SZ_64K - 1);
+
+	return vgic_v5_handle_guest_coresight_idreg_read(offset);
+}
+
 static void vgic_v5_mmio_write_its_wi(struct kvm *kvm, void *dev, gpa_t addr,
 				      unsigned int len, unsigned long val)
 {
@@ -31,11 +298,153 @@ static void vgic_v5_mmio_write_its_wi(struct kvm *kvm, void *dev, gpa_t addr,
 	}
 
 static const struct vgic_register_region vgic_v5_its_registers[] = {
-	REGISTER_ITS_V5_DESC(0, vgic_v5_mmio_read_its_raz,
-			     vgic_v5_mmio_write_its_wi, KVM_VGIC_V5_ITS_SIZE,
+	REGISTER_ITS_V5_DESC(GICV5_ITS_IDR0, vgic_v5_mmio_read_its_misc,
+			     vgic_v5_mmio_write_its_wi, 4, VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_IDR1, vgic_v5_mmio_read_its_misc,
+			     vgic_v5_mmio_write_its_wi, 4, VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_IDR2, vgic_v5_mmio_read_its_misc,
+			     vgic_v5_mmio_write_its_wi, 4, VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_IIDR, vgic_v5_mmio_read_its_misc,
+			     vgic_v5_mmio_write_its_wi, 4, VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_AIDR, vgic_v5_mmio_read_its_misc,
+			     vgic_v5_mmio_write_its_wi, 4, VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_CR0, vgic_v5_mmio_read_its_misc,
+			     vgic_v5_mmio_write_its_misc, 4, VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_CR1, vgic_v5_mmio_read_its_misc,
+			     vgic_v5_mmio_write_its_misc, 4, VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_DT_BASER,
+			     vgic_v5_mmio_read_its_dt_itt,
+			     vgic_v5_mmio_write_its_dt_itt, 8,
+			     VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_DT_CFGR,
+			     vgic_v5_mmio_read_its_dt_itt,
+			     vgic_v5_mmio_write_its_dt_itt, 4,
+			     VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_DIDR, vgic_v5_mmio_read_its_dt_itt,
+			     vgic_v5_mmio_write_its_dt_itt, 8,
+			     VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_EIDR, vgic_v5_mmio_read_its_dt_itt,
+			     vgic_v5_mmio_write_its_dt_itt, 4,
+			     VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_INV_EVENTR, vgic_v5_mmio_read_its_raz,
+			     vgic_v5_mmio_write_its_dt_itt, 4,
+			     VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_INV_DEVICER, vgic_v5_mmio_read_its_raz,
+			     vgic_v5_mmio_write_its_dt_itt, 4,
+			     VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_READ_EVENTR, vgic_v5_mmio_read_its_raz,
+			     vgic_v5_mmio_write_its_dt_itt, 4,
+			     VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_READ_EVENT_DATAR,
+			     vgic_v5_mmio_read_its_dt_itt,
+			     vgic_v5_mmio_write_its_wi, 8,
+			     VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_STATUSR, vgic_v5_mmio_read_its_dt_itt,
+			     vgic_v5_mmio_write_its_wi, 4, VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_SYNCR, vgic_v5_mmio_read_its_raz,
+			     vgic_v5_mmio_write_its_dt_itt, 4,
+			     VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_SYNC_STATUSR,
+			     vgic_v5_mmio_read_its_dt_itt,
+			     vgic_v5_mmio_write_its_wi, 4, VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_GEN_EVENT_DIDR,
+			     vgic_v5_mmio_read_its_dt_itt,
+			     vgic_v5_mmio_write_its_dt_itt, 8,
+			     VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_GEN_EVENT_EIDR,
+			     vgic_v5_mmio_read_its_dt_itt,
+			     vgic_v5_mmio_write_its_dt_itt, 4,
+			     VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_GEN_EVENTR, vgic_v5_mmio_read_its_raz,
+			     vgic_v5_mmio_write_its_dt_itt, 4,
+			     VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_GEN_EVENT_STATUSR,
+			     vgic_v5_mmio_read_its_dt_itt,
+			     vgic_v5_mmio_write_its_wi, 4, VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_MEC_IDR, vgic_v5_mmio_read_its_raz,
+			     vgic_v5_mmio_write_its_wi, 4, VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_MEC_MECID_R, vgic_v5_mmio_read_its_raz,
+			     vgic_v5_mmio_write_its_wi, 4, VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_MPAM_IDR, vgic_v5_mmio_read_its_raz,
+			     vgic_v5_mmio_write_its_wi, 4, VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_MPAM_PARTID_R,
+			     vgic_v5_mmio_read_its_raz,
+			     vgic_v5_mmio_write_its_wi, 4, VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_SWERR_STATUSR, vgic_v5_mmio_read_its_raz,
+			     vgic_v5_mmio_write_its_wi, 8, VGIC_ACCESS_64bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_SWERR_SYNDROMER0,
+			     vgic_v5_mmio_read_its_raz,
+			     vgic_v5_mmio_write_its_wi, 8, VGIC_ACCESS_64bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_SWERR_SYNDROMER1,
+			     vgic_v5_mmio_read_its_raz,
+			     vgic_v5_mmio_write_its_wi, 8,
+			     VGIC_ACCESS_64bit),
+	REGISTER_ITS_V5_DESC(GICV5_ITS_MPAM_PMGR, vgic_v5_mmio_read_its_raz,
+			     vgic_v5_mmio_write_its_wi, 8, VGIC_ACCESS_64bit),
+	/* Split the ranges so KVM_HAS_DEVICE_ATTR only reports actual registers. */
+	REGISTER_ITS_V5_DESC(GICV5_CORESIGHT_DEVARCH,
+			     vgic_v5_its_coresight_read,
+			     vgic_v5_mmio_write_its_wi, 4,
+			     VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC(GICV5_CORESIGHT_PIDR4,
+			     vgic_v5_its_coresight_read,
+			     vgic_v5_mmio_write_its_wi,
+			     GICV5_CORESIGHT_PIDR_CIDR_SIZE,
 			     VGIC_ACCESS_32bit),
 };
 
+static void vgic_v5_its_reset(struct vgic_v5_its *its)
+{
+	u64 mmfr0;
+
+	its->enabled = false;
+
+	mmfr0 = read_sanitised_ftr_reg(SYS_ID_AA64MMFR0_EL1);
+	its->idr0.pa_range = cpuid_feature_extract_unsigned_field(
+		mmfr0, ID_AA64MMFR0_EL1_PARANGE_SHIFT);
+	its->idr0.domain = GICV5_ITS_IDR0_DOMAIN_NON_SECURE;
+	its->idr0.mec = 0;
+	its->idr0.mpam = 0;
+	its->idr0.swe = 0;
+	its->idr0.its_id = 0;
+
+	its->idr1.dt_levels = 1;
+	its->idr1.itt_levels = 1;
+	its->idr1.l2sz = 0;
+	its->idr2.xdmn_events = 0;
+
+	its->cr1.sh = 0;
+	its->cr1.oc = 0;
+	its->cr1.ic = 0;
+	its->cr1.dt_ra = false;
+	its->cr1.itt_ra = false;
+
+	its->idr1.device_id_bits = VITS_V5_DEVICE_ID_BITS;
+	its->idr2.event_id_bits = VITS_V5_EVENT_ID_BITS;
+
+	its->dt_cfgr.device_id_bits = 0;
+	its->dt_cfgr.l2sz = 0;
+	its->dt_cfgr.structure = false;
+	its->dt_baser.addr = 0;
+
+	its->inv_devicer.l1 = false;
+	its->inv_devicer.event_id_bits = 0;
+	its->inv_devicer.i = false;
+	its->didr.device_id = 0;
+	its->eidr.event_id = 0;
+	its->inv_eventr.l1 = false;
+	its->inv_eventr.itt_l2sz = 0;
+	its->inv_eventr.i = false;
+
+	its->read_event_datar.lpi_id = 0;
+	its->read_event_datar.valid = false;
+	its->read_event_datar.vm_id = 0;
+	its->read_event_datar.virt = false;
+	its->gen_event_eidr.event_id = 0;
+	its->gen_event_didr.device_id = 0;
+
+}
+
 static int vgic_v5_its_create(struct kvm_device *dev, u32 type)
 {
 	struct vgic_v5_its *its;
@@ -60,6 +469,8 @@ static int vgic_v5_its_create(struct kvm_device *dev, u32 type)
 	its->vgic_v5_its_base = VGIC_ADDR_UNDEF;
 	its->dev = dev;
 
+	vgic_v5_its_reset(its);
+
 	dev->kvm->arch.vgic.msis_require_devid = true;
 	dev->kvm->arch.vgic.its_model = KVM_DEV_TYPE_ARM_VGIC_V5;
 	dev->private = its;
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 4f9909d6c5b11..53aa5c857e295 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -507,6 +507,83 @@ struct vgic_v5_its {
 	struct vgic_io_device iodev;
 	struct kvm_device *dev;
 
+	bool enabled;
+
+	/* ITS state - used for registers etc */
+	struct its_idr0 {
+		u8 domain;
+		u8 pa_range;
+		bool mec;
+		bool mpam;
+		bool swe;
+		u16 its_id;
+	} idr0;
+
+	struct its_idr1 {
+		u8 device_id_bits;
+		u8 dt_levels;
+		u8 itt_levels;
+		u8 l2sz;
+	} idr1;
+
+	struct its_idr2 {
+		u8 event_id_bits;
+		u8 xdmn_events;
+	} idr2;
+
+	struct its_cr1 {
+		u8 sh;
+		u8 oc;
+		u8 ic;
+		bool dt_ra;
+		bool itt_ra;
+	} cr1;
+
+	struct its_dt_cfgr {
+		u8 device_id_bits;
+		u8 l2sz;
+		bool structure;
+	} dt_cfgr;
+
+	struct its_dt_baser {
+		u64 addr;
+	} dt_baser;
+
+	struct its_inv_devicer {
+		bool l1;
+		u8 event_id_bits;
+		bool i;
+	} inv_devicer;
+
+	struct its_didr {
+		u32 device_id;
+	} didr;
+
+	struct its_eidr {
+		u16 event_id;
+	} eidr;
+
+	struct its_inv_eventr {
+		bool l1;
+		u8 itt_l2sz;
+		bool i;
+	} inv_eventr;
+
+	struct its_read_event_datar {
+		u32 lpi_id;
+		bool valid;
+		u16 vm_id;
+		bool virt;
+	} read_event_datar;
+
+	struct its_gen_event_eidr {
+		u16 event_id;
+	} gen_event_eidr;
+
+	struct its_gen_event_didr {
+		u32 device_id;
+	} gen_event_didr;
+
 	/* Protects the device in general */
 	struct mutex		its_lock;
 };
-- 
2.34.1

[PATCH v2 07/16] KVM: arm64: gic-v5: Add ITS guest table translation emulation

[Sascha Bischoff]

The GICv5 ITS consumes guest-owned translation tables. Unlike the
GICv3 ITS, KVM does not maintain a separate software device/table
model for GICv5 translations; the guest's DT and ITTs are the
authoritative source of DeviceID/EventID to LPI mappings.

Add helpers to walk both linear and two-level guest Device Tables and
Interrupt Translation Tables. The walker reads the guest table entries
under the ITS lock, validates the relevant valid bits and range fields,
and resolves a DeviceID/EventID pair to the LPI encoded in the guest's
ITTE.

Use the table walker to implement the translation side effects of the
MMIO interface. READ_EVENTR now resolves DIDR/EIDR and reports the LPI
through READ_EVENT_DATAR, while GEN_EVENTR resolves GEN_EVENT_DIDR and
GEN_EVENT_EIDR and injects the resulting LPI into the VM.

Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
---
 arch/arm64/kvm/vgic/vgic-its-v5.c | 300 ++++++++++++++++++++++++++++++
 1 file changed, 300 insertions(+)

diff --git a/arch/arm64/kvm/vgic/vgic-its-v5.c b/arch/arm64/kvm/vgic/vgic-its-v5.c
index f04161c315b78..caa68c4680d02 100644
--- a/arch/arm64/kvm/vgic/vgic-its-v5.c
+++ b/arch/arm64/kvm/vgic/vgic-its-v5.c
@@ -19,6 +19,302 @@
 #define VITS_V5_DEVICE_ID_BITS		16
 #define VITS_V5_EVENT_ID_BITS		16
 
+static int parse_linear_dt(struct kvm *kvm, gpa_t dt_base, u32 device_id,
+			   gpa_t *itt_base, bool *two_level_itt,
+			   unsigned int *event_id_bits, unsigned int *itt_l2sz)
+{
+	__le64 dt_l2_entry;
+	gpa_t entry_addr;
+	u64 val;
+	int ret;
+
+	entry_addr = dt_base + device_id * sizeof(__le64);
+	ret = kvm_read_guest_lock(kvm, entry_addr, &dt_l2_entry,
+				  sizeof(dt_l2_entry));
+	if (ret) {
+		kvm_debug("Failed to read guest memory for L2 DT\n");
+		return ret;
+	}
+
+	val = le64_to_cpu(dt_l2_entry);
+	if (!FIELD_GET(GICV5_DTL2E_VALID, val)) {
+		kvm_debug("Guest DT entry is invalid\n");
+		return -EINVAL;
+	}
+
+	*itt_base = FIELD_GET(GICV5_DTL2E_ITT_ADDR_MASK, val) << 3;
+	*two_level_itt = FIELD_GET(GICV5_DTL2E_ITT_STRUCTURE, val);
+	*event_id_bits = FIELD_GET(GICV5_DTL2E_EVENT_ID_BITS, val);
+	*itt_l2sz = FIELD_GET(GICV5_DTL2E_ITT_L2SZ, val);
+
+	return 0;
+}
+
+static int parse_two_level_dt(struct kvm *kvm, gpa_t dt_base,
+			      unsigned int dt_l2sz, u32 device_id,
+			      gpa_t *itt_base, bool *two_level_itt,
+			      unsigned int *event_id_bits,
+			      unsigned int *itt_l2sz)
+{
+	unsigned int device_id_l1_shift, l2_device_id;
+	__le64 dt_l1_entry;
+	gpa_t dt_l2_base;
+	gpa_t entry_addr;
+	u64 val;
+	int ret;
+
+	switch (dt_l2sz) {
+	case 0:
+		device_id_l1_shift = 9;
+		break;
+	case 1:
+		device_id_l1_shift = 11;
+		break;
+	case 2:
+		device_id_l1_shift = 13;
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	entry_addr = dt_base +
+		     (device_id >> device_id_l1_shift) * sizeof(__le64);
+
+	ret = kvm_read_guest_lock(kvm, entry_addr, &dt_l1_entry,
+				  sizeof(dt_l1_entry));
+	if (ret) {
+		kvm_debug("Failed to read guest memory for L1 DT\n");
+		return ret;
+	}
+
+	val = le64_to_cpu(dt_l1_entry);
+	if (!FIELD_GET(GICV5_DTL1E_VALID, val)) {
+		kvm_debug("Guest DT L1 entry is invalid\n");
+		return -EINVAL;
+	}
+
+	dt_l2_base = FIELD_GET(GICV5_DTL1E_L2_ADDR_MASK, val) << 3;
+	l2_device_id = device_id % (1UL << device_id_l1_shift);
+	if (l2_device_id >= (1UL << FIELD_GET(GICV5_DTL1E_SPAN, val))) {
+		kvm_debug("DeviceID 0x%x out of range of guest DT\n",
+			  device_id);
+		return -ENXIO;
+	}
+
+	return parse_linear_dt(kvm, dt_l2_base, l2_device_id, itt_base,
+			       two_level_itt, event_id_bits, itt_l2sz);
+}
+
+static int get_itt(struct kvm *kvm, struct vgic_v5_its *its, u32 device_id,
+		   gpa_t *itt_base, bool *two_level_itt,
+		   unsigned int *event_id_bits, unsigned int *itt_l2sz)
+{
+	unsigned int dt_l2sz, device_id_bits;
+	bool two_level_dt;
+	gpa_t dt_base;
+
+	dt_base = its->dt_baser.addr;
+	two_level_dt = its->dt_cfgr.structure;
+	dt_l2sz = its->dt_cfgr.l2sz;
+	/* An oversized DT_CFGR value reads back as written but acts as IDR1's max. */
+	device_id_bits = min(its->dt_cfgr.device_id_bits,
+			     its->idr1.device_id_bits);
+
+	if (device_id >= 1UL << device_id_bits) {
+		kvm_debug("Device ID out of range\n");
+		return -E2BIG;
+	}
+
+	if (!two_level_dt)
+		return parse_linear_dt(kvm, dt_base, device_id, itt_base,
+				       two_level_itt, event_id_bits, itt_l2sz);
+
+	return parse_two_level_dt(kvm, dt_base, dt_l2sz, device_id, itt_base,
+				  two_level_itt, event_id_bits, itt_l2sz);
+}
+
+static int parse_linear_itt(struct kvm *kvm, u32 event_id, gpa_t itt_base,
+			    __le64 *itt_l2_entry)
+{
+	gpa_t entry_addr = itt_base + event_id * sizeof(*itt_l2_entry);
+	int ret;
+
+	ret = kvm_read_guest_lock(kvm, entry_addr, itt_l2_entry,
+				  sizeof(*itt_l2_entry));
+	if (ret)
+		kvm_debug("Failed to read guest memory for L2 ITT\n");
+
+	return ret;
+}
+
+static int parse_two_level_itt(struct kvm *kvm, u32 event_id, gpa_t itt_base,
+			       unsigned int itt_l2sz, __le64 *itt_l2_entry)
+{
+	unsigned int itt_split_bits, l2_event_id_bits;
+	gpa_t itt_l2_base, entry_addr;
+	u32 l1_event_id, l2_event_id;
+	__le64 itt_l1_entry;
+	u64 val;
+	int ret;
+
+	switch (itt_l2sz) {
+	case 0:
+		itt_split_bits = 9;
+		break;
+	case 1:
+		itt_split_bits = 11;
+		break;
+	case 2:
+		itt_split_bits = 13;
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	l1_event_id = event_id >> itt_split_bits;
+	entry_addr = itt_base + l1_event_id * sizeof(itt_l1_entry);
+	ret = kvm_read_guest_lock(kvm, entry_addr, &itt_l1_entry,
+				  sizeof(itt_l1_entry));
+	if (ret) {
+		kvm_debug("Failed to read guest memory for L1 ITT\n");
+		return ret;
+	}
+
+	val = le64_to_cpu(itt_l1_entry);
+	if (!FIELD_GET(GICV5_ITTL1E_VALID, val)) {
+		kvm_debug("Guest ITT L1 entry is invalid\n");
+		return -EINVAL;
+	}
+
+	itt_l2_base = FIELD_GET(GICV5_ITTL1E_L2_ADDR_MASK, val) << 3;
+	l2_event_id_bits =
+		min(FIELD_GET(GICV5_ITTL1E_SPAN, val), itt_split_bits);
+	l2_event_id = event_id % (1UL << l2_event_id_bits);
+
+	if ((itt_split_bits != l2_event_id_bits) &&
+	    (event_id & ((1UL << itt_split_bits) - 1)) != l2_event_id) {
+		kvm_debug("EventID is out of range of L2 ITT\n");
+		return -ENXIO;
+	}
+
+	return parse_linear_itt(kvm, l2_event_id, itt_l2_base, itt_l2_entry);
+}
+
+static int get_itte(struct kvm *kvm, u32 event_id, gpa_t itt_base,
+		    bool two_level_itt, unsigned int itt_l2sz,
+		    __le64 *itt_l2_entry)
+{
+	if (!two_level_itt)
+		return parse_linear_itt(kvm, event_id, itt_base, itt_l2_entry);
+
+	return parse_two_level_itt(kvm, event_id, itt_base, itt_l2sz,
+				   itt_l2_entry);
+}
+
+static int __vgic_v5_its_resolve_lpi(struct kvm *kvm, struct vgic_v5_its *its,
+				     u32 device_id, u16 event_id,
+				     struct vgic_irq **irq)
+{
+	unsigned int itt_l2sz, event_id_bits;
+	__le64 itt_l2_entry;
+	bool two_level_itt;
+	gpa_t itt_base;
+	u32 lpi;
+	u64 val;
+	int ret;
+
+	lockdep_assert_held(&its->its_lock);
+
+	if (!its->enabled) {
+		kvm_debug("Cannot look up LPI; vITS is disabled\n");
+		return -ENXIO;
+	}
+
+	if (WARN_ON_ONCE(kvm->mm != get_current()->mm))
+		return -EWOULDBLOCK;
+
+	ret = get_itt(kvm, its, device_id, &itt_base, &two_level_itt,
+		      &event_id_bits, &itt_l2sz);
+	if (ret) {
+		kvm_debug("Error parsing guest's DT\n");
+		return ret;
+	}
+
+	if (event_id >= 1UL << event_id_bits) {
+		kvm_debug("Event ID out of range\n");
+		return -E2BIG;
+	}
+
+	ret = get_itte(kvm, event_id, itt_base, two_level_itt, itt_l2sz,
+		       &itt_l2_entry);
+	if (ret)
+		return ret;
+
+	val = le64_to_cpu(itt_l2_entry);
+	if (!FIELD_GET(GICV5_ITTL2E_VALID, val)) {
+		kvm_debug("ITTE is not valid\n");
+		return -EINVAL;
+	}
+
+	lpi = FIELD_GET(GICV5_ITTL2E_LPI_ID, val);
+
+	*irq = vgic_add_lpi(kvm, vgic_v5_make_lpi(lpi), NULL);
+	if (IS_ERR(*irq))
+		return PTR_ERR(*irq);
+
+	return 0;
+}
+
+static void vgic_v5_its_inject_lpi(struct kvm *kvm, struct vgic_irq *irq)
+{
+	kvm_call_hyp(__vgic_v5_vdpend, irq->intid, true,
+		     kvm->arch.vgic.gicv5_vm.vm_id);
+}
+
+static void vgic_v5_its_handle_read_event(struct kvm *kvm,
+					  struct vgic_v5_its *its)
+{
+	struct vgic_irq *irq;
+	int ret;
+
+	its->read_event_datar.valid = false;
+	its->read_event_datar.lpi_id = 0;
+	its->read_event_datar.vm_id = 0;
+	its->read_event_datar.virt = false;
+
+	ret = __vgic_v5_its_resolve_lpi(kvm, its, its->didr.device_id,
+					its->eidr.event_id, &irq);
+	if (ret)
+		return;
+
+	its->read_event_datar.lpi_id = vgic_v5_get_hwirq_id(irq->intid);
+	its->read_event_datar.valid = true;
+	vgic_put_irq(kvm, irq);
+}
+
+static void vgic_v5_its_handle_gen_event(struct kvm *kvm,
+					 struct vgic_v5_its *its,
+					 unsigned long val)
+{
+	struct vgic_irq *irq;
+	int ret;
+
+	if (!FIELD_GET(GICV5_ITS_GEN_EVENTR_R, val))
+		return;
+
+	if (FIELD_GET(GICV5_ITS_GEN_EVENTR_TARGET_DOMAIN, val))
+		return;
+
+	ret = __vgic_v5_its_resolve_lpi(kvm, its,
+					its->gen_event_didr.device_id,
+					its->gen_event_eidr.event_id, &irq);
+	if (ret)
+		return;
+
+	vgic_v5_its_inject_lpi(kvm, irq);
+	vgic_put_irq(kvm, irq);
+}
+
 static unsigned long vgic_v5_mmio_read_its_misc(struct kvm *kvm, void *dev,
 						gpa_t addr, unsigned int len)
 {
@@ -252,7 +548,10 @@ static void vgic_v5_mmio_write_its_dt_itt(struct kvm *kvm, void *dev,
 		its->inv_eventr.i = FIELD_GET(GICV5_ITS_INV_EVENTR_I, val);
 		return;
 	case GICV5_ITS_SYNCR:
+		return;
 	case GICV5_ITS_READ_EVENTR:
+		if (FIELD_GET(GICV5_ITS_READ_EVENTR_R, val))
+			vgic_v5_its_handle_read_event(kvm, its);
 		return;
 	case GICV5_ITS_GEN_EVENT_EIDR:
 		its->gen_event_eidr.event_id =
@@ -266,6 +565,7 @@ static void vgic_v5_mmio_write_its_dt_itt(struct kvm *kvm, void *dev,
 			FIELD_GET(GICV5_ITS_GEN_EVENT_DIDR_DEVICEID, val);
 		return;
 	case GICV5_ITS_GEN_EVENTR:
+		vgic_v5_its_handle_gen_event(kvm, its, val);
 		return;
 	default:
 		return;
-- 
2.34.1

[PATCH v2 08/16] KVM: arm64: gic-v5: Add ITS translation cache maintenance

[Sascha Bischoff]

Walking the guest DT and ITTs on every MSI or ITS event is unnecessary
once a DeviceID/EventID pair has been resolved. Add an XArray indexed
by the DeviceID/EventID pair to cache resolved translations as a
disposable acceleration structure, avoiding repeated walks of guest
memory.

Cache entries hold references to the resolved struct vgic_irq objects
stored in the VM-wide LPI xarray. Lookups use RCU to access entries in
interrupt context without taking a raw spinlock, and take an IRQ
reference before returning an entry to the caller. Cache eviction drops
the reference held by the cache and defers freeing the entry until RCU
readers have finished. This keeps the cache tied to the normal LPI
object lifetime instead of storing raw LPI numbers.

Invalidate cached translations when the table configuration or guest
invalidation commands make them stale. Changes to DT_BASER or DT_CFGR
drop the entire cache, while writes of unchanged values leave it intact.
INV_EVENTR removes either one event or the events covered by an L1 ITT
entry, and INV_DEVICER removes the translations covered by either an L2
DT entry or an L1 DT entry.

The cache is not part of any userspace ABI and is never authoritative:
entries can always be rebuilt from guest memory on the next lookup.

The vgic_irq objects for LPIs track valid translations rather than
mirror interrupt state. Mirroring that state would require querying
the GIC on every guest exit. Once direct injection is added, the
objects also record whether an LPI is mapped into the host ITT.

Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
---
 arch/arm64/kvm/vgic/vgic-its-v5.c | 219 ++++++++++++++++++++++++++++--
 include/kvm/arm_vgic.h            |   3 +
 2 files changed, 208 insertions(+), 14 deletions(-)

diff --git a/arch/arm64/kvm/vgic/vgic-its-v5.c b/arch/arm64/kvm/vgic/vgic-its-v5.c
index caa68c4680d02..9f4783b243221 100644
--- a/arch/arm64/kvm/vgic/vgic-its-v5.c
+++ b/arch/arm64/kvm/vgic/vgic-its-v5.c
@@ -4,6 +4,7 @@
  */
 
 #include <linux/kvm_host.h>
+#include <linux/rcupdate.h>
 #include <linux/slab.h>
 
 #include <asm/kvm_arm.h>
@@ -19,6 +20,153 @@
 #define VITS_V5_DEVICE_ID_BITS		16
 #define VITS_V5_EVENT_ID_BITS		16
 
+struct its_cache_translation_entry {
+	struct rcu_head rcu;
+	struct vgic_irq *irq;
+};
+
+/* Put EventID in the low bits so each device's events form one range. */
+static unsigned long its_cache_key(u32 device_id, u16 event_id)
+{
+	return ((unsigned long)device_id << VITS_V5_EVENT_ID_BITS) | event_id;
+}
+
+static void its_cache_drop_translation(struct kvm *kvm,
+				       struct its_cache_translation_entry *entry)
+{
+	struct vgic_irq *irq = entry->irq;
+
+	vgic_put_irq(kvm, irq);
+	/* An interrupt-context lookup may still be using the erased entry. */
+	kfree_rcu(entry, rcu);
+}
+
+static void its_cache_erase_translation(struct vgic_v5_its *its,
+					unsigned long key)
+{
+	struct its_cache_translation_entry *entry;
+
+	entry = xa_erase(&its->translation_cache, key);
+	if (entry)
+		its_cache_drop_translation(its->dev->kvm, entry);
+}
+
+static void its_cache_erase_range(struct vgic_v5_its *its,
+				  unsigned long first, unsigned long last)
+{
+	struct its_cache_translation_entry *entry;
+	unsigned long key;
+
+	xa_for_each_range(&its->translation_cache, key, entry, first, last)
+		its_cache_erase_translation(its, key);
+}
+
+static void its_cache_erase_span(struct vgic_v5_its *its,
+				unsigned long key, unsigned int bits)
+{
+	unsigned long mask = (1UL << bits) - 1;
+	unsigned long first = key & ~mask;
+
+	its_cache_erase_range(its, first, first | mask);
+}
+
+static int its_cache_add_translation(struct kvm *kvm, struct vgic_v5_its *its,
+				     u32 device_id, u16 event_id,
+				     struct vgic_irq *irq)
+{
+	unsigned long key = its_cache_key(device_id, event_id);
+	struct its_cache_translation_entry *entry;
+	struct its_cache_translation_entry *old;
+
+	entry = kzalloc_obj(*entry, GFP_KERNEL_ACCOUNT);
+	if (!entry)
+		return -ENOMEM;
+
+	entry->irq = irq;
+
+	old = xa_store(&its->translation_cache, key, entry,
+		       GFP_KERNEL_ACCOUNT);
+	if (xa_is_err(old)) {
+		kfree(entry);
+		return xa_err(old);
+	}
+
+	if (old)
+		its_cache_drop_translation(kvm, old);
+
+	return 0;
+}
+
+static void its_cache_clear_translations(struct vgic_v5_its *its)
+{
+	struct its_cache_translation_entry *entry;
+	unsigned long key;
+
+	xa_for_each(&its->translation_cache, key, entry)
+		its_cache_erase_translation(its, key);
+}
+
+static int its_cache_look_up_translation(struct vgic_v5_its *its,
+					 u32 device_id, u16 event_id,
+					 struct vgic_irq **irq)
+{
+	unsigned long key = its_cache_key(device_id, event_id);
+	struct its_cache_translation_entry *entry;
+	int ret = -ENXIO;
+
+	rcu_read_lock();
+	entry = xa_load(&its->translation_cache, key);
+	if (entry && vgic_try_get_irq_ref(entry->irq)) {
+		*irq = entry->irq;
+		ret = 0;
+	}
+	rcu_read_unlock();
+
+	return ret;
+}
+
+static int its_cache_l2sz_bits(u8 l2sz)
+{
+	if (l2sz > GICV5_ITS_DT_ITT_CFGR_L2SZ_64k)
+		return -EINVAL;
+
+	/* L2SZ 0, 1, and 2 encode 4K, 16K, and 64K tables of 8-byte entries. */
+	return 9 + 2 * l2sz;
+}
+
+static void its_cache_handle_inv_devicer(struct vgic_v5_its *its)
+{
+	unsigned int bits = its->inv_devicer.event_id_bits;
+	int l2_bits;
+
+	if (its->inv_devicer.l1 && its->dt_cfgr.structure) {
+		l2_bits = its_cache_l2sz_bits(its->dt_cfgr.l2sz);
+		if (l2_bits < 0)
+			return;
+		/* An L1 DTE covers the L2 DT's devices and all their EventIDs. */
+		bits = l2_bits + VITS_V5_EVENT_ID_BITS;
+	} else if (bits > VITS_V5_EVENT_ID_BITS) {
+		bits = VITS_V5_EVENT_ID_BITS;
+	}
+
+	its_cache_erase_span(its, its_cache_key(its->didr.device_id, 0), bits);
+}
+
+static void its_cache_handle_inv_eventr(struct vgic_v5_its *its)
+{
+	int bits = 0;
+
+	if (its->inv_eventr.l1) {
+		/* An L1 ITTE covers the EventIDs in its L2 ITT. */
+		bits = its_cache_l2sz_bits(its->inv_eventr.itt_l2sz);
+		if (bits < 0)
+			return;
+	}
+
+	its_cache_erase_span(its, its_cache_key(its->didr.device_id,
+						  its->eidr.event_id), bits);
+}
+
 static int parse_linear_dt(struct kvm *kvm, gpa_t dt_base, u32 device_id,
 			   gpa_t *itt_base, bool *two_level_itt,
 			   unsigned int *event_id_bits, unsigned int *itt_l2sz)
@@ -230,6 +378,10 @@ static int __vgic_v5_its_resolve_lpi(struct kvm *kvm, struct vgic_v5_its *its,
 		return -ENXIO;
 	}
 
+	ret = its_cache_look_up_translation(its, device_id, event_id, irq);
+	if (!ret)
+		return 0;
+
 	if (WARN_ON_ONCE(kvm->mm != get_current()->mm))
 		return -EWOULDBLOCK;
 
@@ -262,7 +414,14 @@ static int __vgic_v5_its_resolve_lpi(struct kvm *kvm, struct vgic_v5_its *its,
 	if (IS_ERR(*irq))
 		return PTR_ERR(*irq);
 
-	return 0;
+	vgic_get_irq_ref(*irq);
+	ret = its_cache_add_translation(kvm, its, device_id, event_id, *irq);
+	if (ret) {
+		vgic_put_irq(kvm, *irq);
+		return 0;
+	}
+
+	return ret;
 }
 
 static void vgic_v5_its_inject_lpi(struct kvm *kvm, struct vgic_irq *irq)
@@ -392,7 +551,12 @@ static void vgic_v5_mmio_write_its_misc(struct kvm *kvm, void *dev, gpa_t addr,
 
 	switch (offset) {
 	case GICV5_ITS_CR0:
-		its->enabled = FIELD_GET(GICV5_ITS_CR0_ITSEN, val);
+		if (FIELD_GET(GICV5_ITS_CR0_ITSEN, val)) {
+			its->enabled = true;
+		} else {
+			its->enabled = false;
+			its_cache_clear_translations(its);
+		}
 		return;
 	case GICV5_ITS_CR1:
 		its->cr1.sh = FIELD_GET(GICV5_ITS_CR1_SH, val);
@@ -409,21 +573,39 @@ static void vgic_v5_mmio_write_its_misc(struct kvm *kvm, void *dev, gpa_t addr,
 static void vgic_v5_write_its_dt_reg(struct vgic_v5_its *its, size_t offset,
 				     unsigned long val)
 {
+	lockdep_assert_held(&its->its_lock);
+
 	switch (offset) {
-	case GICV5_ITS_DT_CFGR:
-		its->dt_cfgr.device_id_bits =
-			FIELD_GET(GICV5_ITS_DT_CFGR_DEVICEID_BITS, val);
-		its->dt_cfgr.l2sz =
-			FIELD_GET(GICV5_ITS_DT_CFGR_L2SZ, val);
-		its->dt_cfgr.structure =
-			FIELD_GET(GICV5_ITS_DT_CFGR_STRUCTURE, val);
-		return;
-	case GICV5_ITS_DT_BASER:
-		its->dt_baser.addr =
-			FIELD_GET(GICV5_ITS_DT_BASER_ADDR_MASK, val)
-			<< GICV5_ITS_DT_BASER_ADDR_SHIFT;
+	case GICV5_ITS_DT_CFGR: {
+		u8 device_id_bits = FIELD_GET(GICV5_ITS_DT_CFGR_DEVICEID_BITS, val);
+		u8 l2sz = FIELD_GET(GICV5_ITS_DT_CFGR_L2SZ, val);
+		bool structure = FIELD_GET(GICV5_ITS_DT_CFGR_STRUCTURE, val);
+
+		if (its->dt_cfgr.device_id_bits == device_id_bits &&
+		    its->dt_cfgr.l2sz == l2sz &&
+		    its->dt_cfgr.structure == structure)
+			return;
+
+		its->dt_cfgr.device_id_bits = device_id_bits;
+		its->dt_cfgr.l2sz = l2sz;
+		its->dt_cfgr.structure = structure;
+		break;
+	}
+	case GICV5_ITS_DT_BASER: {
+		u64 base = FIELD_GET(GICV5_ITS_DT_BASER_ADDR_MASK, val)
+			   << GICV5_ITS_DT_BASER_ADDR_SHIFT;
+
+		if (its->dt_baser.addr == base)
+			return;
+
+		its->dt_baser.addr = base;
+		break;
+	}
+	default:
 		return;
 	}
+
+	its_cache_clear_translations(its);
 }
 
 static u64 vgic_v5_read_its_dt_baser(struct vgic_v5_its *its)
@@ -530,6 +712,8 @@ static void vgic_v5_mmio_write_its_dt_itt(struct kvm *kvm, void *dev,
 			FIELD_GET(GICV5_ITS_INV_DEVICER_EVENTID_BITS, val);
 		its->inv_devicer.i =
 			FIELD_GET(GICV5_ITS_INV_DEVICER_I, val);
+		if (its->inv_devicer.i)
+			its_cache_handle_inv_devicer(its);
 		return;
 	case GICV5_ITS_DIDR:
 	case GICV5_ITS_DIDR + 4:
@@ -546,6 +730,8 @@ static void vgic_v5_mmio_write_its_dt_itt(struct kvm *kvm, void *dev,
 		its->inv_eventr.itt_l2sz =
 			FIELD_GET(GICV5_ITS_INV_EVENTR_ITT_L2SZ, val);
 		its->inv_eventr.i = FIELD_GET(GICV5_ITS_INV_EVENTR_I, val);
+		if (its->inv_eventr.i)
+			its_cache_handle_inv_eventr(its);
 		return;
 	case GICV5_ITS_SYNCR:
 		return;
@@ -743,6 +929,7 @@ static void vgic_v5_its_reset(struct vgic_v5_its *its)
 	its->gen_event_eidr.event_id = 0;
 	its->gen_event_didr.device_id = 0;
 
+	its_cache_clear_translations(its);
 }
 
 static int vgic_v5_its_create(struct kvm_device *dev, u32 type)
@@ -766,6 +953,8 @@ static int vgic_v5_its_create(struct kvm_device *dev, u32 type)
 		return -ENODEV;
 	}
 
+	xa_init(&its->translation_cache);
+
 	its->vgic_v5_its_base = VGIC_ADDR_UNDEF;
 	its->dev = dev;
 
@@ -782,6 +971,8 @@ static void vgic_v5_its_destroy(struct kvm_device *kvm_dev)
 {
 	struct vgic_v5_its *its = kvm_dev->private;
 
+	its_cache_clear_translations(its);
+	xa_destroy(&its->translation_cache);
 	kfree(its);
 	kfree(kvm_dev);
 }
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 53aa5c857e295..376c6efa9779e 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -584,6 +584,9 @@ struct vgic_v5_its {
 		u32 device_id;
 	} gen_event_didr;
 
+	/* Caches DeviceID/EventID translations. */
+	struct xarray translation_cache;
+
 	/* Protects the device in general */
 	struct mutex		its_lock;
 };
-- 
2.34.1

[PATCH v2 09/16] KVM: arm64: gic-v5: Add ITS MSI injection

[Sascha Bischoff]

Now that KVM can resolve GICv5 ITS translations from guest memory,
allow userspace and irqfd MSI injection to target a virtual GICv5 ITS.

Add helpers to validate that an MSI address resolves to a GICv5 ITS
IODEV, find the ITS for an MSI, resolve the DeviceID/EventID pair to a
vgic_irq, and inject the resulting LPI with the GICv5 VDPEND path.
GICv5 MSIs require KVM_MSI_VALID_DEVID, as the DeviceID is part of the
ITS translation key.

Although MSI data is 32 bits, GICv5 supports only 16-bit EventIDs.
Reject a nonzero upper half before narrowing the EventID for cache
lookups, so malformed MSI data cannot alias a valid translation.

Provide both the slow path, which resolves the translation from guest
memory and populates the ITS cache, and the inatomic cached path used
by irqfd. If the cached path misses, irqfd retries in a work item. The
worker borrows the VM's mm while walking the guest tables, then drops
it before injecting the LPI.

Wire kvm_set_msi() and kvm_arch_set_irq_inatomic() to use the GICv5
ITS injection helpers for GICv5 VMs, while keeping the existing GICv3
ITS path unchanged for GICv3 VMs.

Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
---
 arch/arm64/kvm/vgic/vgic-irqfd.c  |   8 ++
 arch/arm64/kvm/vgic/vgic-its-v5.c | 132 ++++++++++++++++++++++++++++--
 arch/arm64/kvm/vgic/vgic.h        |   6 ++
 3 files changed, 139 insertions(+), 7 deletions(-)

diff --git a/arch/arm64/kvm/vgic/vgic-irqfd.c b/arch/arm64/kvm/vgic/vgic-irqfd.c
index ee1a1b214f03c..c5f5ce8b80c1d 100644
--- a/arch/arm64/kvm/vgic/vgic-irqfd.c
+++ b/arch/arm64/kvm/vgic/vgic-irqfd.c
@@ -114,6 +114,10 @@ int kvm_set_msi(struct kvm_kernel_irq_routing_entry *e,
 		return -1;
 
 	kvm_populate_msi(e, &msi);
+
+	if (vgic_is_v5(kvm))
+		return vgic_v5_its_inject_msi(kvm, &msi);
+
 	return vgic_its_inject_msi(kvm, &msi);
 }
 
@@ -135,6 +139,10 @@ int kvm_arch_set_irq_inatomic(struct kvm_kernel_irq_routing_entry *e,
 			break;
 
 		kvm_populate_msi(e, &msi);
+
+		if (vgic_is_v5(kvm))
+			return vgic_v5_its_inject_cached_translation(kvm, &msi);
+
 		return vgic_its_inject_cached_translation(kvm, &msi);
 	}
 
diff --git a/arch/arm64/kvm/vgic/vgic-its-v5.c b/arch/arm64/kvm/vgic/vgic-its-v5.c
index 9f4783b243221..b69a1d107c0ae 100644
--- a/arch/arm64/kvm/vgic/vgic-its-v5.c
+++ b/arch/arm64/kvm/vgic/vgic-its-v5.c
@@ -3,8 +3,10 @@
  * Copyright (C) 2025 ARM Limited, All Rights Reserved.
  */
 
+#include <linux/kthread.h>
 #include <linux/kvm_host.h>
 #include <linux/rcupdate.h>
+#include <linux/sched/mm.h>
 #include <linux/slab.h>
 
 #include <asm/kvm_arm.h>
@@ -19,6 +21,7 @@
 
 #define VITS_V5_DEVICE_ID_BITS		16
 #define VITS_V5_EVENT_ID_BITS		16
+#define VITS_V5_UNMAPPED		1
 
 struct its_cache_translation_entry {
 	struct rcu_head rcu;
@@ -359,8 +362,46 @@ static int get_itte(struct kvm *kvm, u32 event_id, gpa_t itt_base,
 				   itt_l2_entry);
 }
 
+struct vgic_v5_its *vgic_v5_msi_to_its(struct kvm *kvm, struct kvm_msi *msi)
+{
+	struct kvm_io_device *kvm_io_dev;
+	struct vgic_io_device *iodev;
+	u64 address;
+
+	if (!vgic_has_its(kvm))
+		return ERR_PTR(-ENODEV);
+
+	if (!(msi->flags & KVM_MSI_VALID_DEVID))
+		return ERR_PTR(-EINVAL);
+
+	/*
+	 * MSI data is 32 bits, but GICv5 limits EventIDs to 16 bits.
+	 * Match that limit in the vITS.
+	 */
+	if (msi->data >> VITS_V5_EVENT_ID_BITS)
+		return ERR_PTR(-EINVAL);
+
+	address = (u64)msi->address_hi << 32 | msi->address_lo;
+
+	guard(srcu)(&kvm->srcu);
+
+	kvm_io_dev = kvm_io_bus_get_dev(kvm, KVM_MMIO_BUS, address);
+	if (!kvm_io_dev)
+		return ERR_PTR(-EINVAL);
+
+	if (kvm_io_dev->ops != &kvm_io_gic_ops)
+		return ERR_PTR(-EINVAL);
+
+	iodev = container_of(kvm_io_dev, struct vgic_io_device, dev);
+	if (iodev->iodev_type != IODEV_GICV5_ITS)
+		return ERR_PTR(-EINVAL);
+
+	return iodev->v5its;
+}
+
+/* A positive return means the guest has no usable translation. */
 static int __vgic_v5_its_resolve_lpi(struct kvm *kvm, struct vgic_v5_its *its,
-				     u32 device_id, u16 event_id,
+				     u32 device_id, u32 event_id,
 				     struct vgic_irq **irq)
 {
 	unsigned int itt_l2sz, event_id_bits;
@@ -372,13 +413,15 @@ static int __vgic_v5_its_resolve_lpi(struct kvm *kvm, struct vgic_v5_its *its,
 	int ret;
 
 	lockdep_assert_held(&its->its_lock);
+	if (event_id >> VITS_V5_EVENT_ID_BITS)
+		return -EINVAL;
 
 	if (!its->enabled) {
 		kvm_debug("Cannot look up LPI; vITS is disabled\n");
 		return -ENXIO;
 	}
 
-	ret = its_cache_look_up_translation(its, device_id, event_id, irq);
+	ret = its_cache_look_up_translation(its, device_id, (u16)event_id, irq);
 	if (!ret)
 		return 0;
 
@@ -389,23 +432,23 @@ static int __vgic_v5_its_resolve_lpi(struct kvm *kvm, struct vgic_v5_its *its,
 		      &event_id_bits, &itt_l2sz);
 	if (ret) {
 		kvm_debug("Error parsing guest's DT\n");
-		return ret;
+		return VITS_V5_UNMAPPED;
 	}
 
 	if (event_id >= 1UL << event_id_bits) {
 		kvm_debug("Event ID out of range\n");
-		return -E2BIG;
+		return VITS_V5_UNMAPPED;
 	}
 
 	ret = get_itte(kvm, event_id, itt_base, two_level_itt, itt_l2sz,
 		       &itt_l2_entry);
 	if (ret)
-		return ret;
+		return VITS_V5_UNMAPPED;
 
 	val = le64_to_cpu(itt_l2_entry);
 	if (!FIELD_GET(GICV5_ITTL2E_VALID, val)) {
 		kvm_debug("ITTE is not valid\n");
-		return -EINVAL;
+		return VITS_V5_UNMAPPED;
 	}
 
 	lpi = FIELD_GET(GICV5_ITTL2E_LPI_ID, val);
@@ -415,7 +458,8 @@ static int __vgic_v5_its_resolve_lpi(struct kvm *kvm, struct vgic_v5_its *its,
 		return PTR_ERR(*irq);
 
 	vgic_get_irq_ref(*irq);
-	ret = its_cache_add_translation(kvm, its, device_id, event_id, *irq);
+	ret = its_cache_add_translation(kvm, its, device_id, (u16)event_id,
+					 *irq);
 	if (ret) {
 		vgic_put_irq(kvm, *irq);
 		return 0;
@@ -424,12 +468,86 @@ static int __vgic_v5_its_resolve_lpi(struct kvm *kvm, struct vgic_v5_its *its,
 	return ret;
 }
 
+int vgic_v5_its_resolve_lpi(struct kvm *kvm, struct vgic_v5_its *its,
+			    u32 device_id, u32 event_id,
+			    struct vgic_irq **irq)
+{
+	guard(mutex)(&its->its_lock);
+
+	return __vgic_v5_its_resolve_lpi(kvm, its, device_id, event_id, irq);
+}
+
 static void vgic_v5_its_inject_lpi(struct kvm *kvm, struct vgic_irq *irq)
 {
 	kvm_call_hyp(__vgic_v5_vdpend, irq->intid, true,
 		     kvm->arch.vgic.gicv5_vm.vm_id);
 }
 
+int vgic_v5_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi)
+{
+	struct vgic_v5_its *its;
+	struct vgic_irq *irq;
+	bool borrowed_mm = false;
+	int ret;
+
+	its = vgic_v5_msi_to_its(kvm, msi);
+	if (IS_ERR(its))
+		return PTR_ERR(its);
+
+	/* An irqfd cache miss is resolved by a worker outside the VM's mm. */
+	if (current->mm != kvm->mm) {
+		if (WARN_ON_ONCE(current->mm || !(current->flags & PF_KTHREAD)))
+			return -EWOULDBLOCK;
+
+		if (!mmget_not_zero(kvm->mm))
+			return -EFAULT;
+
+		kthread_use_mm(kvm->mm);
+		borrowed_mm = true;
+	}
+
+	ret = vgic_v5_its_resolve_lpi(kvm, its, msi->devid, msi->data, &irq);
+	if (borrowed_mm) {
+		kthread_unuse_mm(kvm->mm);
+		mmput(kvm->mm);
+	}
+
+	if (ret < 0)
+		return ret;
+	/* KVM_SIGNAL_MSI reports guest translation failures as blocked MSIs. */
+	if (ret)
+		return 0;
+
+	vgic_v5_its_inject_lpi(kvm, irq);
+	vgic_put_irq(kvm, irq);
+
+	return 1;
+}
+
+int vgic_v5_its_inject_cached_translation(struct kvm *kvm, struct kvm_msi *msi)
+{
+	struct vgic_v5_its *its;
+	struct vgic_irq *irq;
+	int ret;
+
+	its = vgic_v5_msi_to_its(kvm, msi);
+	if (IS_ERR(its))
+		return PTR_ERR(its);
+
+	if (!READ_ONCE(its->enabled))
+		return -EWOULDBLOCK;
+
+	ret = its_cache_look_up_translation(its, msi->devid, (u16)msi->data,
+					    &irq);
+	if (ret)
+		return -EWOULDBLOCK;
+
+	vgic_v5_its_inject_lpi(kvm, irq);
+	vgic_put_irq(kvm, irq);
+
+	return 0;
+}
+
 static void vgic_v5_its_handle_read_event(struct kvm *kvm,
 					  struct vgic_v5_its *its)
 {
diff --git a/arch/arm64/kvm/vgic/vgic.h b/arch/arm64/kvm/vgic/vgic.h
index 9e1cbcdedf3f5..64f6229fe555e 100644
--- a/arch/arm64/kvm/vgic/vgic.h
+++ b/arch/arm64/kvm/vgic/vgic.h
@@ -364,6 +364,12 @@ void vgic_v3_put(struct kvm_vcpu *vcpu);
 bool vgic_has_its(struct kvm *kvm);
 int kvm_vgic_register_its_device(void);
 int kvm_vgic_v5_register_its_device(void);
+struct vgic_v5_its *vgic_v5_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
+int vgic_v5_its_resolve_lpi(struct kvm *kvm, struct vgic_v5_its *its,
+			    u32 device_id, u32 event_id,
+			    struct vgic_irq **irq);
+int vgic_v5_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi);
+int vgic_v5_its_inject_cached_translation(struct kvm *kvm, struct kvm_msi *msi);
 void vgic_enable_lpis(struct kvm_vcpu *vcpu);
 void vgic_flush_pending_lpis(struct kvm_vcpu *vcpu);
 int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi);
-- 
2.34.1

[PATCH v2 10/16] KVM: arm64: gic-v5: Add ITS userspace register access

[Sascha Bischoff]

Userspace needs access to the emulated GICv5 ITS MMIO state in order
to inspect and restore the device state around VM reset and migration.
Expose the GICv5 ITS register frame through KVM_DEV_ARM_VGIC_GRP_ITS_REGS.

Reuse the same MMIO register descriptors used by the guest-facing
IODEV, but apply the usual userspace access constraints: the ITS base
address must have been configured, the register offset must map to an
emulated ITS register, and userspace accesses must use the natural
32-bit or 64-bit register alignment.

Add control handling for KVM_DEV_ARM_ITS_CTRL_RESET. The reset is
performed while all vCPUs are stopped and while holding the ITS lock,
so it cannot race with guest MMIO updates or translation lookups. The
reset clears guest-programmed state and cached translations while
preserving the advertised ITS capability defaults.

Do not expose GICv5 ITS SAVE_TABLES or RESTORE_TABLES controls. For
GICv5, the guest-owned DT and ITTs are the authoritative translation
state, and the KVM translation cache is disposable state that can be
rebuilt from guest memory.

Restore guest read-only IDR0-IDR2 through dedicated userspace writes,
validating the advertised capabilities before changing them. Also
allow userspace to restore READ_EVENT_DATAR while keeping guest writes
to that result register ignored.

Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
---
 arch/arm64/kvm/vgic/vgic-its-v5.c | 371 ++++++++++++++++++++++++++----
 1 file changed, 322 insertions(+), 49 deletions(-)

diff --git a/arch/arm64/kvm/vgic/vgic-its-v5.c b/arch/arm64/kvm/vgic/vgic-its-v5.c
index b69a1d107c0ae..638ee47f6e27d 100644
--- a/arch/arm64/kvm/vgic/vgic-its-v5.c
+++ b/arch/arm64/kvm/vgic/vgic-its-v5.c
@@ -688,6 +688,70 @@ static void vgic_v5_mmio_write_its_misc(struct kvm *kvm, void *dev, gpa_t addr,
 	}
 }
 
+static int vgic_v5_uaccess_write_its_idr(struct kvm *kvm, void *dev,
+					gpa_t addr, unsigned int len,
+					unsigned long val)
+{
+	struct vgic_v5_its *its = dev;
+	size_t offset = addr & (SZ_64K - 1);
+	u64 mmfr0;
+
+	guard(mutex)(&its->its_lock);
+
+	switch (offset) {
+	case GICV5_ITS_IDR0:
+		if (FIELD_GET(GICV5_ITS_IDR0_DOM, val) !=
+		    GICV5_ITS_IDR0_DOMAIN_NON_SECURE ||
+		    (val & ~(GICV5_ITS_IDR0_ITSID |
+			     GICV5_ITS_IDR0_PA_RANGE | GICV5_ITS_IDR0_DOM)))
+			return -EINVAL;
+
+		mmfr0 = read_sanitised_ftr_reg(SYS_ID_AA64MMFR0_EL1);
+		if (FIELD_GET(GICV5_ITS_IDR0_PA_RANGE, val) >
+		    cpuid_feature_extract_unsigned_field(mmfr0,
+					ID_AA64MMFR0_EL1_PARANGE_SHIFT))
+			return -EINVAL;
+
+		its->idr0.pa_range = FIELD_GET(GICV5_ITS_IDR0_PA_RANGE, val);
+		its->idr0.its_id = FIELD_GET(GICV5_ITS_IDR0_ITSID, val);
+		return 0;
+	case GICV5_ITS_IDR1:
+		if ((val & ~(GICV5_ITS_IDR1_L2SZ |
+			     GICV5_ITS_IDR1_ITT_LEVELS |
+			     GICV5_ITS_IDR1_DT_LEVELS |
+			     GICV5_ITS_IDR1_DEVICEID_BITS)) ||
+		    FIELD_GET(GICV5_ITS_IDR1_DEVICEID_BITS, val) >
+		    VITS_V5_DEVICE_ID_BITS)
+			return -EINVAL;
+
+		/*
+		 * Userspace can change the DeviceID limit after translations have
+		 * been cached. Drop them so the old limit cannot remain in effect.
+		 */
+		if (its->idr1.device_id_bits !=
+		    FIELD_GET(GICV5_ITS_IDR1_DEVICEID_BITS, val))
+			its_cache_clear_translations(its);
+
+		its->idr1.device_id_bits = FIELD_GET(GICV5_ITS_IDR1_DEVICEID_BITS,
+							 val);
+		its->idr1.dt_levels = !!(val & GICV5_ITS_IDR1_DT_LEVELS);
+		its->idr1.itt_levels = !!(val & GICV5_ITS_IDR1_ITT_LEVELS);
+		its->idr1.l2sz = FIELD_GET(GICV5_ITS_IDR1_L2SZ, val);
+		return 0;
+	case GICV5_ITS_IDR2:
+		if ((val & ~GICV5_ITS_IDR2_EVENTID_BITS) ||
+		    FIELD_GET(GICV5_ITS_IDR2_EVENTID_BITS, val) >
+		    VITS_V5_EVENT_ID_BITS)
+			return -EINVAL;
+
+		its->idr2.event_id_bits = FIELD_GET(GICV5_ITS_IDR2_EVENTID_BITS,
+							 val);
+		return 0;
+	default:
+		return -EINVAL;
+	}
+}
+
 static void vgic_v5_write_its_dt_reg(struct vgic_v5_its *its, size_t offset,
 				     unsigned long val)
 {
@@ -732,6 +796,37 @@ static u64 vgic_v5_read_its_dt_baser(struct vgic_v5_its *its)
 			  its->dt_baser.addr >> GICV5_ITS_DT_BASER_ADDR_SHIFT);
 }
 
+static int vgic_v5_uaccess_write_its_dt_itt(struct kvm *kvm, void *dev,
+					    gpa_t addr, unsigned int len,
+					    unsigned long val)
+{
+	struct vgic_v5_its *its = dev;
+	size_t offset = addr & (SZ_64K - 1);
+
+	guard(mutex)(&its->its_lock);
+
+	vgic_v5_write_its_dt_reg(its, offset, val);
+	return 0;
+}
+
+static int vgic_v5_uaccess_write_its_event_datar(struct kvm *kvm, void *dev,
+						gpa_t addr, unsigned int len,
+						unsigned long val)
+{
+	struct vgic_v5_its *its = dev;
+
+	guard(mutex)(&its->its_lock);
+
+	its->read_event_datar.valid = !!(val & GICV5_ITS_READ_EVENT_DATAR_VALID);
+	its->read_event_datar.lpi_id = FIELD_GET(GICV5_ITS_READ_EVENT_DATAR_LPI_ID,
+							 val);
+	its->read_event_datar.virt = !!(val & GICV5_ITS_READ_EVENT_DATAR_VIRT);
+	its->read_event_datar.vm_id = FIELD_GET(GICV5_ITS_READ_EVENT_DATAR_VM_ID,
+							 val);
+
+	return 0;
+}
+
 static unsigned long vgic_v5_mmio_read_its_dt_itt(struct kvm *kvm, void *dev,
 						  gpa_t addr, unsigned int len)
 {
@@ -901,13 +996,26 @@ static void vgic_v5_mmio_write_its_wi(struct kvm *kvm, void *dev, gpa_t addr,
 		.its_read = rd, .its_write = wr,                       \
 	}
 
+#define REGISTER_ITS_V5_DESC_UACCESS(off, rd, wr, uwr, length, acc)    \
+	{                                                              \
+		.reg_offset = off, .len = length, .access_flags = acc, \
+		.its_read = rd, .its_write = wr,                       \
+		.uaccess_its_write = uwr,                              \
+	}
+
 static const struct vgic_register_region vgic_v5_its_registers[] = {
-	REGISTER_ITS_V5_DESC(GICV5_ITS_IDR0, vgic_v5_mmio_read_its_misc,
-			     vgic_v5_mmio_write_its_wi, 4, VGIC_ACCESS_32bit),
-	REGISTER_ITS_V5_DESC(GICV5_ITS_IDR1, vgic_v5_mmio_read_its_misc,
-			     vgic_v5_mmio_write_its_wi, 4, VGIC_ACCESS_32bit),
-	REGISTER_ITS_V5_DESC(GICV5_ITS_IDR2, vgic_v5_mmio_read_its_misc,
-			     vgic_v5_mmio_write_its_wi, 4, VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC_UACCESS(GICV5_ITS_IDR0, vgic_v5_mmio_read_its_misc,
+				     vgic_v5_mmio_write_its_wi,
+				     vgic_v5_uaccess_write_its_idr, 4,
+				     VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC_UACCESS(GICV5_ITS_IDR1, vgic_v5_mmio_read_its_misc,
+				     vgic_v5_mmio_write_its_wi,
+				     vgic_v5_uaccess_write_its_idr, 4,
+				     VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC_UACCESS(GICV5_ITS_IDR2, vgic_v5_mmio_read_its_misc,
+				     vgic_v5_mmio_write_its_wi,
+				     vgic_v5_uaccess_write_its_idr, 4,
+				     VGIC_ACCESS_32bit),
 	REGISTER_ITS_V5_DESC(GICV5_ITS_IIDR, vgic_v5_mmio_read_its_misc,
 			     vgic_v5_mmio_write_its_wi, 4, VGIC_ACCESS_32bit),
 	REGISTER_ITS_V5_DESC(GICV5_ITS_AIDR, vgic_v5_mmio_read_its_misc,
@@ -916,14 +1024,16 @@ static const struct vgic_register_region vgic_v5_its_registers[] = {
 			     vgic_v5_mmio_write_its_misc, 4, VGIC_ACCESS_32bit),
 	REGISTER_ITS_V5_DESC(GICV5_ITS_CR1, vgic_v5_mmio_read_its_misc,
 			     vgic_v5_mmio_write_its_misc, 4, VGIC_ACCESS_32bit),
-	REGISTER_ITS_V5_DESC(GICV5_ITS_DT_BASER,
-			     vgic_v5_mmio_read_its_dt_itt,
-			     vgic_v5_mmio_write_its_dt_itt, 8,
-			     VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
-	REGISTER_ITS_V5_DESC(GICV5_ITS_DT_CFGR,
-			     vgic_v5_mmio_read_its_dt_itt,
-			     vgic_v5_mmio_write_its_dt_itt, 4,
-			     VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC_UACCESS(GICV5_ITS_DT_BASER,
+				     vgic_v5_mmio_read_its_dt_itt,
+				     vgic_v5_mmio_write_its_dt_itt,
+				     vgic_v5_uaccess_write_its_dt_itt, 8,
+				     VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
+	REGISTER_ITS_V5_DESC_UACCESS(GICV5_ITS_DT_CFGR,
+				     vgic_v5_mmio_read_its_dt_itt,
+				     vgic_v5_mmio_write_its_dt_itt,
+				     vgic_v5_uaccess_write_its_dt_itt, 4,
+				     VGIC_ACCESS_32bit),
 	REGISTER_ITS_V5_DESC(GICV5_ITS_DIDR, vgic_v5_mmio_read_its_dt_itt,
 			     vgic_v5_mmio_write_its_dt_itt, 8,
 			     VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
@@ -939,9 +1049,10 @@ static const struct vgic_register_region vgic_v5_its_registers[] = {
 	REGISTER_ITS_V5_DESC(GICV5_ITS_READ_EVENTR, vgic_v5_mmio_read_its_raz,
 			     vgic_v5_mmio_write_its_dt_itt, 4,
 			     VGIC_ACCESS_32bit),
-	REGISTER_ITS_V5_DESC(GICV5_ITS_READ_EVENT_DATAR,
+	REGISTER_ITS_V5_DESC_UACCESS(GICV5_ITS_READ_EVENT_DATAR,
 			     vgic_v5_mmio_read_its_dt_itt,
-			     vgic_v5_mmio_write_its_wi, 8,
+			     vgic_v5_mmio_write_its_wi,
+			     vgic_v5_uaccess_write_its_event_datar, 8,
 			     VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
 	REGISTER_ITS_V5_DESC(GICV5_ITS_STATUSR, vgic_v5_mmio_read_its_dt_itt,
 			     vgic_v5_mmio_write_its_wi, 4, VGIC_ACCESS_32bit),
@@ -1130,64 +1241,226 @@ static int vgic_v5_register_its_iodev(struct kvm *kvm, struct vgic_v5_its *its,
 	return ret;
 }
 
+static int vgic_v5_its_has_attr_regs(struct kvm_device *dev,
+				     struct kvm_device_attr *attr)
+{
+	const struct vgic_register_region *region;
+	gpa_t offset = attr->attr;
+	int align;
+
+	region = vgic_find_mmio_region(vgic_v5_its_registers,
+				       ARRAY_SIZE(vgic_v5_its_registers),
+				       offset);
+	if (!region)
+		return -ENXIO;
+
+	align = region->access_flags & VGIC_ACCESS_64bit ? 0x7 : 0x3;
+	if (offset & align)
+		return -EINVAL;
+
+	return 0;
+}
+
+static int vgic_v5_its_attr_regs_access(struct kvm_device *dev,
+					struct kvm_device_attr *attr,
+					u64 *reg, bool is_write)
+{
+	const struct vgic_register_region *region;
+	struct vgic_v5_its *its = dev->private;
+	gpa_t offset = attr->attr;
+	unsigned int len;
+	gpa_t addr;
+	int align;
+	int ret = 0;
+
+	mutex_lock(&dev->kvm->lock);
+
+	if (kvm_trylock_all_vcpus(dev->kvm)) {
+		mutex_unlock(&dev->kvm->lock);
+		return -EBUSY;
+	}
+
+	mutex_lock(&dev->kvm->arch.config_lock);
+
+	if (IS_VGIC_ADDR_UNDEF(its->vgic_v5_its_base)) {
+		ret = -ENXIO;
+		goto out;
+	}
+
+	region = vgic_find_mmio_region(vgic_v5_its_registers,
+				       ARRAY_SIZE(vgic_v5_its_registers),
+				       offset);
+	if (!region) {
+		ret = -ENXIO;
+		goto out;
+	}
+
+	/*
+	 * Although the spec supports upper/lower 32-bit accesses to
+	 * 64-bit ITS registers, the userspace ABI requires 64-bit
+	 * accesses to all 64-bit wide registers.
+	 */
+	align = region->access_flags & VGIC_ACCESS_64bit ? 0x7 : 0x3;
+	len = region->access_flags & VGIC_ACCESS_64bit ? 8 : 4;
+
+	if (offset & align) {
+		ret = -EINVAL;
+		goto out;
+	}
+
+	addr = its->vgic_v5_its_base + offset;
+
+	if (is_write) {
+		if (region->uaccess_its_write)
+			ret = region->uaccess_its_write(dev->kvm, its, addr,
+							len, *reg);
+		else
+			region->its_write(dev->kvm, its, addr, len, *reg);
+	} else {
+		*reg = region->its_read(dev->kvm, its, addr, len);
+	}
+out:
+	mutex_unlock(&dev->kvm->arch.config_lock);
+	kvm_unlock_all_vcpus(dev->kvm);
+	mutex_unlock(&dev->kvm->lock);
+
+	return ret;
+}
+
+static int vgic_v5_its_ctrl(struct kvm *kvm, struct vgic_v5_its *its, u64 attr)
+{
+	int ret = 0;
+
+	if (attr == KVM_DEV_ARM_VGIC_CTRL_INIT)
+		return 0;
+
+	mutex_lock(&kvm->lock);
+
+	if (kvm_trylock_all_vcpus(kvm)) {
+		mutex_unlock(&kvm->lock);
+		return -EBUSY;
+	}
+
+	mutex_lock(&kvm->arch.config_lock);
+	mutex_lock(&its->its_lock);
+
+	switch (attr) {
+	case KVM_DEV_ARM_ITS_CTRL_RESET:
+		vgic_v5_its_reset(its);
+		break;
+	default:
+		ret = -ENXIO;
+		break;
+	}
+
+	mutex_unlock(&its->its_lock);
+	mutex_unlock(&kvm->arch.config_lock);
+	kvm_unlock_all_vcpus(kvm);
+	mutex_unlock(&kvm->lock);
+
+	return ret;
+}
+
 static int vgic_v5_its_set_attr(struct kvm_device *dev,
 				struct kvm_device_attr *attr)
 {
 	struct vgic_v5_its *its = dev->private;
-	u64 __user *uaddr = (u64 __user *)(long)attr->addr;
-	unsigned long type = (unsigned long)attr->attr;
-	u64 addr;
 	int ret;
 
-	if (attr->group != KVM_DEV_ARM_VGIC_GRP_ADDR)
-		return -ENXIO;
+	switch (attr->group) {
+	case KVM_DEV_ARM_VGIC_GRP_ADDR: {
+		u64 __user *uaddr = (u64 __user *)(long)attr->addr;
+		unsigned long type = (unsigned long)attr->attr;
+		u64 addr;
 
-	if (type != KVM_VGIC_V5_ADDR_TYPE_ITS)
-		return -ENODEV;
+		if (type != KVM_VGIC_V5_ADDR_TYPE_ITS)
+			return -ENODEV;
 
-	if (copy_from_user(&addr, uaddr, sizeof(addr)))
-		return -EFAULT;
+		if (copy_from_user(&addr, uaddr, sizeof(addr)))
+			return -EFAULT;
 
-	ret = vgic_check_iorange(dev->kvm, its->vgic_v5_its_base, addr,
-				 SZ_64K, KVM_VGIC_V5_ITS_SIZE);
-	if (ret)
-		return ret;
+		ret = vgic_check_iorange(dev->kvm, its->vgic_v5_its_base,
+					 addr, SZ_64K, KVM_VGIC_V5_ITS_SIZE);
+		if (ret)
+			return ret;
+
+		return vgic_v5_register_its_iodev(dev->kvm, its, addr);
+	}
+	case KVM_DEV_ARM_VGIC_GRP_CTRL:
+		return vgic_v5_its_ctrl(dev->kvm, its, attr->attr);
+	case KVM_DEV_ARM_VGIC_GRP_ITS_REGS: {
+		u64 __user *uaddr = (u64 __user *)(long)attr->addr;
+		u64 reg;
+
+		if (get_user(reg, uaddr))
+			return -EFAULT;
 
-	return vgic_v5_register_its_iodev(dev->kvm, its, addr);
+		return vgic_v5_its_attr_regs_access(dev, attr, &reg, true);
+	}
+	}
+
+	return -ENXIO;
 }
 
 static int vgic_v5_its_get_attr(struct kvm_device *dev,
 				struct kvm_device_attr *attr)
 {
-	struct vgic_v5_its *its = dev->private;
-	u64 __user *uaddr = (u64 __user *)(long)attr->addr;
-	unsigned long type = (unsigned long)attr->attr;
-	u64 addr = its->vgic_v5_its_base;
+	switch (attr->group) {
+	case KVM_DEV_ARM_VGIC_GRP_ADDR: {
+		struct vgic_v5_its *its = dev->private;
+		u64 __user *uaddr = (u64 __user *)(long)attr->addr;
+		unsigned long type = (unsigned long)attr->attr;
+		u64 addr = its->vgic_v5_its_base;
 
-	if (attr->group != KVM_DEV_ARM_VGIC_GRP_ADDR)
-		return -ENXIO;
+		if (type != KVM_VGIC_V5_ADDR_TYPE_ITS)
+			return -ENODEV;
 
-	if (type != KVM_VGIC_V5_ADDR_TYPE_ITS)
-		return -ENODEV;
+		if (copy_to_user(uaddr, &addr, sizeof(addr)))
+			return -EFAULT;
+
+		return 0;
+	}
+	case KVM_DEV_ARM_VGIC_GRP_ITS_REGS: {
+		u64 __user *uaddr = (u64 __user *)(long)attr->addr;
+		u64 reg;
+		int ret;
 
-	if (copy_to_user(uaddr, &addr, sizeof(addr)))
-		return -EFAULT;
+		ret = vgic_v5_its_attr_regs_access(dev, attr, &reg, false);
+		if (ret)
+			return ret;
 
-	return 0;
+		if (put_user(reg, uaddr))
+			return -EFAULT;
+
+		return 0;
+	}
+	}
+
+	return -ENXIO;
 }
 
 static int vgic_v5_its_has_attr(struct kvm_device *dev,
 				struct kvm_device_attr *attr)
 {
-	if (attr->group != KVM_DEV_ARM_VGIC_GRP_ADDR)
-		return -ENXIO;
-
-	switch (attr->attr) {
-	case KVM_VGIC_V5_ADDR_TYPE_ITS:
-		return 0;
-	default:
-		return -ENXIO;
+	switch (attr->group) {
+	case KVM_DEV_ARM_VGIC_GRP_ADDR:
+		switch (attr->attr) {
+		case KVM_VGIC_V5_ADDR_TYPE_ITS:
+			return 0;
+		}
+		break;
+	case KVM_DEV_ARM_VGIC_GRP_CTRL:
+		switch (attr->attr) {
+		case KVM_DEV_ARM_VGIC_CTRL_INIT:
+		case KVM_DEV_ARM_ITS_CTRL_RESET:
+			return 0;
+		}
+		break;
+	case KVM_DEV_ARM_VGIC_GRP_ITS_REGS:
+		return vgic_v5_its_has_attr_regs(dev, attr);
 	}
+
+	return -ENXIO;
 }
 
 static struct kvm_device_ops kvm_arm_vgic_v5_its_ops = {
-- 
2.34.1

[PATCH v2 11/16] irqchip/gic-v5: Enable direct injection in ITS

[Sascha Bischoff]

Allow KVM to replace a host ITTE with a virtual translation that
directly injects an interrupt into a guest. Add an irq_chip callback to
install and remove the virtual translation, and disable lazy masking
while the interrupt is forwarded. The KVM caller is added later.

The ITS writes the ITTE before invalidating its cache. If invalidation
times out, hardware may still use the virtual translation even though
setup failed. Allow disable to restore the host ITTE when the forwarded
flag is clear, and retain retryable state if restoration also times
out. Keep lazy masking disabled until the host mapping is restored.

Co-authored-by: Timothy Hayes <timothy.hayes@arm.com>
Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
---
 drivers/irqchip/irq-gic-v5-its.c   | 141 +++++++++++++++++++++++++++++
 include/linux/irqchip/arm-gic-v5.h |   8 ++
 2 files changed, 149 insertions(+)

diff --git a/drivers/irqchip/irq-gic-v5-its.c b/drivers/irqchip/irq-gic-v5-its.c
index 28e39b065de0e..d91f4ea1ef440 100644
--- a/drivers/irqchip/irq-gic-v5-its.c
+++ b/drivers/irqchip/irq-gic-v5-its.c
@@ -61,6 +61,48 @@ static void its_writeq_relaxed(struct gicv5_its_chip_data *its_node, const u64 v
 	writeq_relaxed(val, its_node->its_base + reg_offset);
 }
 
+#ifdef CONFIG_KVM
+int gicv5_its_enable_direct_injection(int virq, u16 vm_id, u32 guest_lpi)
+{
+	struct virt_intr_data virt_intr = {
+		.vm_id		= vm_id,
+		.guest_lpi	= guest_lpi,
+	};
+	int ret;
+
+	/*
+	 * The host will never see that interrupt firing again, so it
+	 * is vital that we don't do any lazy masking.
+	 */
+	irq_set_status_flags(virq, IRQ_DISABLE_UNLAZY);
+
+	ret = irq_set_vcpu_affinity(virq, &virt_intr);
+	if (ret && ret != -ETIMEDOUT && ret != -EBUSY)
+		irq_clear_status_flags(virq, IRQ_DISABLE_UNLAZY);
+
+	return ret;
+}
+
+int gicv5_its_disable_direct_injection(int virq)
+{
+	int ret;
+
+	ret = irq_set_vcpu_affinity(virq, NULL);
+	if (ret) {
+		pr_err("Failed to restore host interrupt\n");
+		return ret;
+	}
+
+	/*
+	 * In theory the host could see the interrupt again now. Undo this
+	 * again.
+	 */
+	irq_clear_status_flags(virq, IRQ_DISABLE_UNLAZY);
+
+	return 0;
+}
+#endif /* CONFIG_KVM */
+
 static void gicv5_its_dcache_clean(struct gicv5_its_chip_data *its, void *start,
 				   size_t sz)
 {
@@ -723,6 +765,102 @@ static void gicv5_its_compose_msi_msg(struct irq_data *d, struct msi_msg *msg)
 	msi_msg_set_addr(irq_data_get_msi_desc(d), msg, addr);
 }
 
+#ifdef CONFIG_KVM
+static int gicv5_its_set_vcpu_affinity(struct irq_data *d, void *int_data)
+{
+	struct gicv5_its_dev *its_dev = irq_data_get_irq_chip_data(d);
+	u16 event_id = FIELD_GET(GICV5_ITS_HWIRQ_EVENT_ID, d->hwirq);
+	u32 dev_id = FIELD_GET(GICV5_ITS_HWIRQ_DEVICE_ID, d->hwirq);
+	struct virt_intr_data *data = int_data;
+	struct gicv5_its_chip_data *its;
+	__le64 *itte;
+	u64 itte_val, tmp;
+	int ret = 0;
+
+	if (IS_ERR(its_dev)) {
+		pr_err("Failed to find ITS entry for DeviceID 0x%x", dev_id);
+		return PTR_ERR(its_dev);
+	}
+
+	its = its_dev->its_node;
+
+	itte = gicv5_its_device_get_itte_ref(its_dev, event_id);
+	itte_val = le64_to_cpu(*itte);
+
+	/*
+	 * The host entry should be valid, else something strange is going on!
+	 */
+	if (!(itte_val & GICV5_ITTL2E_VALID)) {
+		if (data)
+			return -EINVAL;
+		if (!irqd_is_forwarded_to_vcpu(d))
+			return 0;
+		goto invalidate;
+	}
+
+	if (data) {
+		if (irqd_is_forwarded_to_vcpu(d)) {
+			if ((itte_val & GICV5_ITTL2E_VIRTUAL) &&
+			    FIELD_GET(GICV5_ITTL2E_LPI_ID, itte_val) == data->guest_lpi &&
+			    FIELD_GET(GICV5_ITTL2E_VM_ID, itte_val) == data->vm_id)
+				return 0;
+			return -EBUSY;
+		}
+
+		/* Stale virtual translation without forwarded state. */
+		if (itte_val & GICV5_ITTL2E_VIRTUAL)
+			return -EBUSY;
+
+		tmp = FIELD_PREP(GICV5_ITTL2E_LPI_ID, data->guest_lpi);
+		tmp |= FIELD_PREP(GICV5_ITTL2E_VM_ID, data->vm_id);
+		tmp |= GICV5_ITTL2E_VIRTUAL;
+		tmp |= GICV5_ITTL2E_VALID;
+
+		its_write_table_entry(its, itte, tmp);
+	} else {
+		/*
+		 * A failed sync still needs retrying, even if the ITTE is
+		 * host-owned.
+		 */
+		if (!(itte_val & GICV5_ITTL2E_VIRTUAL) &&
+		    !irqd_is_forwarded_to_vcpu(d))
+			return 0;
+
+		if (itte_val & GICV5_ITTL2E_VIRTUAL) {
+			tmp = FIELD_PREP(GICV5_ITTL2E_LPI_ID,
+					 d->parent_data->hwirq);
+			tmp |= GICV5_ITTL2E_VALID;
+			its_write_table_entry(its, itte, tmp);
+		}
+	}
+
+invalidate:
+	/* Make the change ITS-visible */
+	dsb(st);
+	ret = gicv5_its_itt_cache_inv(its, dev_id, event_id);
+	if (!ret && !data) {
+		/*
+		 * Invalidation sends events using the old translation to the
+		 * IRS, but does not wait for them to be processed.
+		 */
+		gicv5_irs_syncr();
+	}
+	if (ret) {
+		/* Keep a failed restoration distinguishable from a clean host ITTE. */
+		if (!data)
+			irqd_set_forwarded_to_vcpu(d);
+		return ret;
+	}
+
+	if (data)
+		irqd_set_forwarded_to_vcpu(d);
+	else
+		irqd_clr_forwarded_to_vcpu(d);
+
+	return 0;
+}
+#endif /* CONFIG_KVM */
+
 static const struct irq_chip gicv5_its_irq_chip = {
 	.name			= "GICv5-ITS-MSI",
 	.irq_mask		= irq_chip_mask_parent,
@@ -732,6 +870,9 @@ static const struct irq_chip gicv5_its_irq_chip = {
 	.irq_get_irqchip_state	= irq_chip_get_parent_state,
 	.irq_set_irqchip_state	= irq_chip_set_parent_state,
 	.irq_compose_msi_msg	= gicv5_its_compose_msi_msg,
+#ifdef CONFIG_KVM
+	.irq_set_vcpu_affinity	= gicv5_its_set_vcpu_affinity,
+#endif /* CONFIG_KVM */
 };
 
 static struct gicv5_its_dev *gicv5_its_find_device(struct gicv5_its_chip_data *its,
diff --git a/include/linux/irqchip/arm-gic-v5.h b/include/linux/irqchip/arm-gic-v5.h
index 6e8484fef9faf..26ca29794a960 100644
--- a/include/linux/irqchip/arm-gic-v5.h
+++ b/include/linux/irqchip/arm-gic-v5.h
@@ -727,4 +727,12 @@ enum gicv5_vcpu_cmd {
 	LPI_VIST_MAKE_INVALID,	/* Triggered by a guest */
 };
 
+struct virt_intr_data {
+	u16 vm_id;
+	u32 guest_lpi;
+};
+
+int gicv5_its_enable_direct_injection(int virq, u16 vm_id, u32 guest_lpi);
+int gicv5_its_disable_direct_injection(int virq);
+
 #endif
-- 
2.34.1

[PATCH v2 12/16] KVM: arm64: vgic: Move direct MSI support helper to common MMIO code

[Sascha Bischoff]

Move vgic_supports_direct_msis() from the GICv3 MMIO code to the
common VGIC MMIO file without changing its behavior. GICv5 will extend
this helper when irq forwarding support is added.

Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
---
 arch/arm64/kvm/vgic/vgic-mmio-v3.c | 13 -------------
 arch/arm64/kvm/vgic/vgic-mmio.c    | 13 +++++++++++++
 2 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/arch/arm64/kvm/vgic/vgic-mmio-v3.c b/arch/arm64/kvm/vgic/vgic-mmio-v3.c
index 5c7be55ba95d6..b9b0531954f52 100644
--- a/arch/arm64/kvm/vgic/vgic-mmio-v3.c
+++ b/arch/arm64/kvm/vgic/vgic-mmio-v3.c
@@ -38,19 +38,6 @@ u64 update_64bit_reg(u64 reg, unsigned int offset, unsigned int len,
 	return reg | ((u64)val << lower);
 }
 
-bool vgic_supports_direct_msis(struct kvm *kvm)
-{
-	/*
-	 * Deliberately conflate vLPI and vSGI support on GICv4.1 hardware,
-	 * indirectly allowing userspace to control whether or not vPEs are
-	 * allocated for the VM.
-	 */
-	if (system_supports_direct_sgis() && !vgic_supports_direct_sgis(kvm))
-		return false;
-
-	return kvm_vgic_global_state.has_gicv4 && vgic_has_its(kvm);
-}
-
 bool system_supports_direct_sgis(void)
 {
 	return kvm_vgic_global_state.has_gicv4_1 && gic_cpuif_has_vsgi();
diff --git a/arch/arm64/kvm/vgic/vgic-mmio.c b/arch/arm64/kvm/vgic/vgic-mmio.c
index 5371d0b8bd0c6..3c0c858608752 100644
--- a/arch/arm64/kvm/vgic/vgic-mmio.c
+++ b/arch/arm64/kvm/vgic/vgic-mmio.c
@@ -1142,6 +1142,19 @@ int vgic_register_dist_iodev(struct kvm *kvm, gpa_t dist_base_address,
 				       len, &io_device->dev);
 }
 
+bool vgic_supports_direct_msis(struct kvm *kvm)
+{
+	/*
+	 * Deliberately conflate vLPI and vSGI support on GICv4.1 hardware,
+	 * indirectly allowing userspace to control whether or not vPEs are
+	 * allocated for the VM.
+	 */
+	if (system_supports_direct_sgis() && !vgic_supports_direct_sgis(kvm))
+		return false;
+
+	return kvm_vgic_global_state.has_gicv4 && vgic_has_its(kvm);
+}
+
 bool vgic_has_its(struct kvm *kvm)
 {
 	struct vgic_dist *dist = &kvm->arch.vgic;
-- 
2.34.1

[PATCH v2 13/16] KVM: arm64: gic-v5: Implement irq forwarding

[Sascha Bischoff]

GICv5, like GICv4, supports direct injection of interrupts via the
ITS. Move the existing GICv4 interface behind a common shim that selects
the appropriate implementation. For GICv5 forwarding, the ITS driver
replaces a host ITTE with one targeting a guest. The host no longer
receives that interrupt until forwarding is removed and the host ITTE is
restored.

A guest can make multiple DeviceID/EventID pairs resolve to the same
LPI, even though it should not. Track the cache entry responsible for
direct injection, and let only that entry tear it down during guest
invalidation. Explicitly unforwarding an IRQ clears the owner without
removing the cached translation.

The ITS driver writes the virtual ITTE before invalidating its cache.
If invalidation times out, attempt to restore the host ITTE before
falling back to software injection. If the restoration also times out,
keep the host IRQ association and a separate LPI reference so
unforwarding can be retried even after the guest invalidates the
translation. Clear KVM's forwarding state only after the host ITTE has
been restored.

Forwarding can be set up from another process's address space through
IRQ bypass. A cache miss in this path must still walk the guest's DT and
ITT. Read the entries from the VM's mm using access_remote_vm() when
the caller does not have that mm. Hold an mm reference and protect the
memslot lookup with SRCU, and fail the lookup on a short read.

Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
---
 arch/arm64/kvm/arm.c              |   6 +-
 arch/arm64/kvm/vgic/vgic-its-v5.c | 129 +++++++++++++++++++++-----
 arch/arm64/kvm/vgic/vgic-mmio.c   |  10 +-
 arch/arm64/kvm/vgic/vgic-v5.c     | 148 ++++++++++++++++++++++++++++++
 arch/arm64/kvm/vgic/vgic.c        |  17 ++++
 arch/arm64/kvm/vgic/vgic.h        |   8 ++
 include/kvm/arm_vgic.h            |  11 +++
 7 files changed, 302 insertions(+), 27 deletions(-)

diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 157bf4b812020..5c7cbf2afaa1f 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -3193,7 +3193,7 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
 	if (irq_entry->type != KVM_IRQ_ROUTING_MSI)
 		return 0;
 
-	return kvm_vgic_v4_set_forwarding(irqfd->kvm, prod->irq,
+	return kvm_vgic_set_forwarding(irqfd->kvm, prod->irq,
 					  &irqfd->irq_entry);
 }
 
@@ -3207,7 +3207,7 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
 	if (irq_entry->type != KVM_IRQ_ROUTING_MSI)
 		return;
 
-	kvm_vgic_v4_unset_forwarding(irqfd->kvm, prod->irq);
+	kvm_vgic_unset_forwarding(irqfd->kvm, prod->irq);
 }
 
 void kvm_arch_update_irqfd_routing(struct kvm_kernel_irqfd *irqfd,
@@ -3226,7 +3226,7 @@ void kvm_arch_update_irqfd_routing(struct kvm_kernel_irqfd *irqfd,
 	 *
 	 * Unmap the vLPI and fall back to software LPI injection.
 	 */
-	return kvm_vgic_v4_unset_forwarding(irqfd->kvm, irqfd->producer->irq);
+	kvm_vgic_unset_forwarding(irqfd->kvm, irqfd->producer->irq);
 }
 
 void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
diff --git a/arch/arm64/kvm/vgic/vgic-its-v5.c b/arch/arm64/kvm/vgic/vgic-its-v5.c
index 638ee47f6e27d..3648d3f7c1b29 100644
--- a/arch/arm64/kvm/vgic/vgic-its-v5.c
+++ b/arch/arm64/kvm/vgic/vgic-its-v5.c
@@ -5,6 +5,7 @@
 
 #include <linux/kthread.h>
 #include <linux/kvm_host.h>
+#include <linux/mm.h>
 #include <linux/rcupdate.h>
 #include <linux/sched/mm.h>
 #include <linux/slab.h>
@@ -38,7 +39,35 @@ static void its_cache_drop_translation(struct kvm *kvm,
 				       struct its_cache_translation_entry *entry)
 {
 	struct vgic_irq *irq = entry->irq;
+	unsigned long flags;
+	bool drop_forwarding_ref = false;
 
+	/*
+	 * Multiple translations can share an LPI as we have no way of stopping
+	 * the guest from doing so. Only the entry associated with direct injection
+	 * may tear down the IRQ's forwarding state, as otherwise we can end up
+	 * accidentally tearing something down. Realistically, this only would
+	 * result in a performance degradation, but we might as well avoid that
+	 * in the first place if we can.
+	 *
+	 * Keep the lock until the host IRQ is restored, so the producer cannot
+	 * free it while we are using its IRQ number.
+	 */
+	raw_spin_lock_irqsave(&irq->irq_lock, flags);
+	if (irq->forwarding_owner == entry) {
+		/* The entry is about to be freed, even if unforwarding fails. */
+		irq->forwarding_owner = NULL;
+		if (irq->hw &&
+		    !WARN_ON_ONCE(gicv5_its_disable_direct_injection(irq->host_irq))) {
+			irq->hw = false;
+			irq->host_irq = 0;
+			drop_forwarding_ref = true;
+		}
+	}
+	raw_spin_unlock_irqrestore(&irq->irq_lock, flags);
+
+	if (drop_forwarding_ref)
+		vgic_put_irq(kvm, irq);
 	vgic_put_irq(kvm, irq);
 	/* An interrupt-context lookup may still be using the erased entry. */
 	kfree_rcu(entry, rcu);
@@ -75,7 +104,8 @@ static void its_cache_erase_span(struct vgic_v5_its *its,
 
 static int its_cache_add_translation(struct kvm *kvm, struct vgic_v5_its *its,
 				     u32 device_id, u16 event_id,
-				     struct vgic_irq *irq)
+				     struct vgic_irq *irq,
+				     void **owner)
 {
 	unsigned long key = its_cache_key(device_id, event_id);
 	struct its_cache_translation_entry *entry;
@@ -96,6 +126,8 @@ static int its_cache_add_translation(struct kvm *kvm, struct vgic_v5_its *its,
 
 	if (old)
 		its_cache_drop_translation(kvm, old);
+	if (owner)
+		*owner = entry;
 
 	return 0;
 }
@@ -111,16 +143,23 @@ static void its_cache_clear_translations(struct vgic_v5_its *its)
 
 static int its_cache_look_up_translation(struct vgic_v5_its *its,
 					 u32 device_id, u16 event_id,
-					 struct vgic_irq **irq)
+					 struct vgic_irq **irq,
+					 void **owner)
 {
 	unsigned long key = its_cache_key(device_id, event_id);
 	struct its_cache_translation_entry *entry;
 	int ret = -ENXIO;
 
+	/* The owner can escape the RCU read section only under its_lock. */
+	if (owner)
+		lockdep_assert_held(&its->its_lock);
+
 	rcu_read_lock();
 	entry = xa_load(&its->translation_cache, key);
 	if (entry && vgic_try_get_irq_ref(entry->irq)) {
 		*irq = entry->irq;
+		if (owner)
+			*owner = entry;
 		ret = 0;
 	}
 	rcu_read_unlock();
@@ -170,6 +209,37 @@ static void its_cache_handle_inv_eventr(struct vgic_v5_its *its)
 						  its->eidr.event_id), bits);
 }
 
+static int vgic_v5_its_read_entry(struct kvm *kvm, gpa_t gpa, __le64 *entry)
+{
+	struct mm_struct *mm = kvm->mm;
+	unsigned long hva;
+	int srcu_idx, ret;
+
+	if (current->mm == mm)
+		return kvm_read_guest_lock(kvm, gpa, entry, sizeof(*entry));
+
+	/* Forwarding setup can run in the IRQ producer's address space. */
+	if (!mmget_not_zero(mm))
+		return -EFAULT;
+
+	srcu_idx = srcu_read_lock(&kvm->srcu);
+	hva = gfn_to_hva_prot(kvm, gpa >> PAGE_SHIFT, NULL);
+	if (kvm_is_error_hva(hva)) {
+		ret = -EFAULT;
+		goto out;
+	}
+
+	/* Table entries are 8-byte aligned, so each read fits in one page. */
+	ret = access_remote_vm(mm, hva + offset_in_page(gpa), entry,
+			       sizeof(*entry), 0);
+
+out:
+	srcu_read_unlock(&kvm->srcu, srcu_idx);
+	mmput(mm);
+
+	return ret == sizeof(*entry) ? 0 : -EFAULT;
+}
+
 static int parse_linear_dt(struct kvm *kvm, gpa_t dt_base, u32 device_id,
 			   gpa_t *itt_base, bool *two_level_itt,
 			   unsigned int *event_id_bits, unsigned int *itt_l2sz)
@@ -180,8 +250,7 @@ static int parse_linear_dt(struct kvm *kvm, gpa_t dt_base, u32 device_id,
 	int ret;
 
 	entry_addr = dt_base + device_id * sizeof(__le64);
-	ret = kvm_read_guest_lock(kvm, entry_addr, &dt_l2_entry,
-				  sizeof(dt_l2_entry));
+	ret = vgic_v5_its_read_entry(kvm, entry_addr, &dt_l2_entry);
 	if (ret) {
 		kvm_debug("Failed to read guest memory for L2 DT\n");
 		return ret;
@@ -231,8 +300,7 @@ static int parse_two_level_dt(struct kvm *kvm, gpa_t dt_base,
 	entry_addr = dt_base +
 		     (device_id >> device_id_l1_shift) * sizeof(__le64);
 
-	ret = kvm_read_guest_lock(kvm, entry_addr, &dt_l1_entry,
-				  sizeof(dt_l1_entry));
+	ret = vgic_v5_its_read_entry(kvm, entry_addr, &dt_l1_entry);
 	if (ret) {
 		kvm_debug("Failed to read guest memory for L1 DT\n");
 		return ret;
@@ -290,8 +358,7 @@ static int parse_linear_itt(struct kvm *kvm, u32 event_id, gpa_t itt_base,
 	gpa_t entry_addr = itt_base + event_id * sizeof(*itt_l2_entry);
 	int ret;
 
-	ret = kvm_read_guest_lock(kvm, entry_addr, itt_l2_entry,
-				  sizeof(*itt_l2_entry));
+	ret = vgic_v5_its_read_entry(kvm, entry_addr, itt_l2_entry);
 	if (ret)
 		kvm_debug("Failed to read guest memory for L2 ITT\n");
 
@@ -324,8 +391,7 @@ static int parse_two_level_itt(struct kvm *kvm, u32 event_id, gpa_t itt_base,
 
 	l1_event_id = event_id >> itt_split_bits;
 	entry_addr = itt_base + l1_event_id * sizeof(itt_l1_entry);
-	ret = kvm_read_guest_lock(kvm, entry_addr, &itt_l1_entry,
-				  sizeof(itt_l1_entry));
+	ret = vgic_v5_its_read_entry(kvm, entry_addr, &itt_l1_entry);
 	if (ret) {
 		kvm_debug("Failed to read guest memory for L1 ITT\n");
 		return ret;
@@ -402,7 +468,8 @@ struct vgic_v5_its *vgic_v5_msi_to_its(struct kvm *kvm, struct kvm_msi *msi)
 /* A positive return means the guest has no usable translation. */
 static int __vgic_v5_its_resolve_lpi(struct kvm *kvm, struct vgic_v5_its *its,
 				     u32 device_id, u32 event_id,
-				     struct vgic_irq **irq)
+				     struct vgic_irq **irq,
+				     void **owner)
 {
 	unsigned int itt_l2sz, event_id_bits;
 	__le64 itt_l2_entry;
@@ -421,13 +488,11 @@ static int __vgic_v5_its_resolve_lpi(struct kvm *kvm, struct vgic_v5_its *its,
 		return -ENXIO;
 	}
 
-	ret = its_cache_look_up_translation(its, device_id, (u16)event_id, irq);
+	ret = its_cache_look_up_translation(its, device_id, (u16)event_id, irq,
+					    owner);
 	if (!ret)
 		return 0;
 
-	if (WARN_ON_ONCE(kvm->mm != get_current()->mm))
-		return -EWOULDBLOCK;
-
 	ret = get_itt(kvm, its, device_id, &itt_base, &two_level_itt,
 		      &event_id_bits, &itt_l2sz);
 	if (ret) {
@@ -458,23 +523,41 @@ static int __vgic_v5_its_resolve_lpi(struct kvm *kvm, struct vgic_v5_its *its,
 		return PTR_ERR(*irq);
 
 	vgic_get_irq_ref(*irq);
-	ret = its_cache_add_translation(kvm, its, device_id, (u16)event_id,
-					 *irq);
+	ret = its_cache_add_translation(kvm, its, device_id, (u16)event_id, *irq,
+					owner);
 	if (ret) {
+		/*
+		 * Drop the reference intended for the cache. Software callers
+		 * retain theirs, but direct forwarding needs an entry for ownership.
+		 */
 		vgic_put_irq(kvm, *irq);
-		return 0;
+		if (!owner)
+			return 0;
+		vgic_put_irq(kvm, *irq);
+		*irq = NULL;
 	}
 
 	return ret;
 }
 
+int vgic_v5_its_resolve_lpi_locked(struct kvm *kvm,
+				   struct vgic_v5_its *its,
+				   u32 device_id, u32 event_id,
+				   struct vgic_irq **irq,
+				   void **owner)
+{
+	return __vgic_v5_its_resolve_lpi(kvm, its, device_id, event_id, irq,
+					 owner);
+}
+
 int vgic_v5_its_resolve_lpi(struct kvm *kvm, struct vgic_v5_its *its,
 			    u32 device_id, u32 event_id,
 			    struct vgic_irq **irq)
 {
 	guard(mutex)(&its->its_lock);
 
-	return __vgic_v5_its_resolve_lpi(kvm, its, device_id, event_id, irq);
+	return __vgic_v5_its_resolve_lpi(kvm, its, device_id, event_id, irq,
+					 NULL);
 }
 
 static void vgic_v5_its_inject_lpi(struct kvm *kvm, struct vgic_irq *irq)
@@ -537,8 +620,8 @@ int vgic_v5_its_inject_cached_translation(struct kvm *kvm, struct kvm_msi *msi)
 	if (!READ_ONCE(its->enabled))
 		return -EWOULDBLOCK;
 
-	ret = its_cache_look_up_translation(its, msi->devid, (u16)msi->data,
-					    &irq);
+	ret = its_cache_look_up_translation(its, msi->devid, (u16)msi->data, &irq,
+					    NULL);
 	if (ret)
 		return -EWOULDBLOCK;
 
@@ -560,7 +643,7 @@ static void vgic_v5_its_handle_read_event(struct kvm *kvm,
 	its->read_event_datar.virt = false;
 
 	ret = __vgic_v5_its_resolve_lpi(kvm, its, its->didr.device_id,
-					its->eidr.event_id, &irq);
+					its->eidr.event_id, &irq, NULL);
 	if (ret)
 		return;
 
@@ -584,7 +667,7 @@ static void vgic_v5_its_handle_gen_event(struct kvm *kvm,
 
 	ret = __vgic_v5_its_resolve_lpi(kvm, its,
 					its->gen_event_didr.device_id,
-					its->gen_event_eidr.event_id, &irq);
+					its->gen_event_eidr.event_id, &irq, NULL);
 	if (ret)
 		return;
 
diff --git a/arch/arm64/kvm/vgic/vgic-mmio.c b/arch/arm64/kvm/vgic/vgic-mmio.c
index 3c0c858608752..3268e3d3d5170 100644
--- a/arch/arm64/kvm/vgic/vgic-mmio.c
+++ b/arch/arm64/kvm/vgic/vgic-mmio.c
@@ -1144,6 +1144,8 @@ int vgic_register_dist_iodev(struct kvm *kvm, gpa_t dist_base_address,
 
 bool vgic_supports_direct_msis(struct kvm *kvm)
 {
+	const struct vgic_dist *dist = &kvm->arch.vgic;
+
 	/*
 	 * Deliberately conflate vLPI and vSGI support on GICv4.1 hardware,
 	 * indirectly allowing userspace to control whether or not vPEs are
@@ -1152,7 +1154,13 @@ bool vgic_supports_direct_msis(struct kvm *kvm)
 	if (system_supports_direct_sgis() && !vgic_supports_direct_sgis(kvm))
 		return false;
 
-	return kvm_vgic_global_state.has_gicv4 && vgic_has_its(kvm);
+	if (kvm_vgic_global_state.has_gicv4 && vgic_has_its(kvm))
+		return true;
+	if (kvm_vgic_global_state.type == VGIC_V5 && vgic_has_its(kvm) &&
+	    dist->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V5)
+		return true;
+
+	return false;
 }
 
 bool vgic_has_its(struct kvm *kvm)
diff --git a/arch/arm64/kvm/vgic/vgic-v5.c b/arch/arm64/kvm/vgic/vgic-v5.c
index 18184020e188b..c2ae7b0fb3a7f 100644
--- a/arch/arm64/kvm/vgic/vgic-v5.c
+++ b/arch/arm64/kvm/vgic/vgic-v5.c
@@ -1635,3 +1635,151 @@ unsigned long vgic_v5_handle_guest_coresight_idreg_read(u32 offset)
 		return 0;
 	}
 }
+
+static struct vgic_v5_its *
+vgic_v5_get_its(struct kvm *kvm, struct kvm_kernel_irq_routing_entry *irq_entry)
+{
+	struct kvm_msi msi = (struct kvm_msi){
+		.address_lo = irq_entry->msi.address_lo,
+		.address_hi = irq_entry->msi.address_hi,
+		.data = irq_entry->msi.data,
+		.flags = irq_entry->msi.flags,
+		.devid = irq_entry->msi.devid,
+	};
+
+	return vgic_v5_msi_to_its(kvm, &msi);
+}
+
+int kvm_vgic_v5_set_forwarding(struct kvm *kvm, int virq,
+			       struct kvm_kernel_irq_routing_entry *irq_entry)
+{
+	struct vgic_v5_its *its_data;
+	struct vgic_irq *irq;
+	void *owner;
+	bool hw;
+	int ret;
+
+	if (!vgic_has_its(kvm))
+		return 0;
+
+	/* Look up the ITS; bail out if we can't look it up */
+	its_data = vgic_v5_get_its(kvm, irq_entry);
+	if (IS_ERR(its_data))
+		return 0;
+
+	/*
+	 * Look up and cache the translation. Bail out if we failed to look up
+	 * the translation at this stage. If the translation becomes valid at a
+	 * later stage, we should fall back to software-based injection.
+	 */
+	guard(mutex)(&its_data->its_lock);
+
+	ret = vgic_v5_its_resolve_lpi_locked(kvm, its_data,
+					     irq_entry->msi.devid,
+					     irq_entry->msi.data, &irq, &owner);
+	if (ret)
+		return 0;
+
+	scoped_guard(raw_spinlock_irqsave, &irq->irq_lock) {
+		hw = irq->hw;
+	}
+
+	if (hw) {
+		vgic_put_irq(kvm, irq);
+		return 0;
+	}
+
+	ret = gicv5_its_enable_direct_injection(virq, vgic_v5_vm_id(kvm),
+						vgic_v5_get_hwirq_id(irq->intid));
+	if (ret) {
+		if (ret != -ETIMEDOUT) {
+			vgic_put_irq(kvm, irq);
+			return 0;
+		}
+
+		/* The ITTE was written; confirm restoration before falling back. */
+		if (!gicv5_its_disable_direct_injection(virq)) {
+			vgic_put_irq(kvm, irq);
+			return 0;
+		}
+
+		/* Keep the host IRQ association so a later unforward can retry. */
+		ret = 0;
+	}
+
+	scoped_guard(raw_spinlock_irqsave, &irq->irq_lock) {
+		if (!irq->hw) {
+			/* Keep the LPI available if invalidation must retry unforwarding. */
+			vgic_get_irq_ref(irq);
+			irq->hw = true;
+			irq->host_irq = virq;
+			/*
+			 * Record the entry responsible for this host IRQ, even if
+			 * cleanup after a failed sync is still pending. Only this
+			 * entry may unforward on invalidation; another translation
+			 * could map to the same LPI.
+			 */
+			irq->forwarding_owner = owner;
+		} else {
+			ret = -EBUSY;
+		}
+	}
+
+	if (ret)
+		WARN_ON_ONCE(gicv5_its_disable_direct_injection(virq));
+
+	vgic_put_irq(kvm, irq);
+
+	return 0;
+}
+
+static struct vgic_irq *vgic_v5_host_irq_get_lpi(struct kvm *kvm, int host_irq)
+{
+	struct vgic_irq *irq;
+	unsigned long idx;
+
+	guard(rcu)();
+	xa_for_each(&kvm->arch.vgic.lpi_xa, idx, irq) {
+		scoped_guard(raw_spinlock_irqsave, &irq->irq_lock) {
+			if (irq->hw && irq->host_irq == host_irq &&
+			    vgic_try_get_irq_ref(irq))
+				return irq;
+		}
+	}
+
+	return NULL;
+}
+
+void kvm_vgic_v5_unset_forwarding(struct kvm *kvm, int virq)
+{
+	struct vgic_irq *irq;
+	bool drop_forwarding_ref = false;
+
+	if (!vgic_has_its(kvm))
+		return;
+
+	/*
+	 * Note: We don't remove the entry from our translation cache here. It
+	 * is up to the guest to invalidate the old translations, and hence the
+	 * removal path is tied into ITS_INV_DEVICER and ITS_INV_EVENTR, rather
+	 * than being explicitly called here.
+	 */
+	irq = vgic_v5_host_irq_get_lpi(kvm, virq);
+	if (!irq)
+		return;
+
+	scoped_guard(raw_spinlock_irqsave, &irq->irq_lock) {
+		if (irq->hw && irq->host_irq == virq &&
+		    !WARN_ON_ONCE(gicv5_its_disable_direct_injection(virq))) {
+			irq->hw = false;
+			irq->host_irq = 0;
+			irq->forwarding_owner = NULL;
+			drop_forwarding_ref = true;
+		}
+	}
+
+	if (drop_forwarding_ref)
+		vgic_put_irq(kvm, irq);
+
+	vgic_put_irq(kvm, irq);
+}
diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c
index d1a1a2cea386e..32cde9316df1b 100644
--- a/arch/arm64/kvm/vgic/vgic.c
+++ b/arch/arm64/kvm/vgic/vgic.c
@@ -1442,3 +1442,20 @@ void vgic_irq_handle_resampling(struct vgic_irq *irq,
 			vgic_irq_set_phys_active(irq, false);
 	}
 }
+
+int kvm_vgic_set_forwarding(struct kvm *kvm, int irq,
+			    struct kvm_kernel_irq_routing_entry *irq_entry)
+{
+	if (kvm->arch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V5)
+		return kvm_vgic_v5_set_forwarding(kvm, irq, irq_entry);
+	else
+		return kvm_vgic_v4_set_forwarding(kvm, irq, irq_entry);
+}
+
+void kvm_vgic_unset_forwarding(struct kvm *kvm, int irq)
+{
+	if (kvm->arch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V5)
+		kvm_vgic_v5_unset_forwarding(kvm, irq);
+	else
+		kvm_vgic_v4_unset_forwarding(kvm, irq);
+}
diff --git a/arch/arm64/kvm/vgic/vgic.h b/arch/arm64/kvm/vgic/vgic.h
index 64f6229fe555e..deb2331a0d4e1 100644
--- a/arch/arm64/kvm/vgic/vgic.h
+++ b/arch/arm64/kvm/vgic/vgic.h
@@ -368,8 +368,16 @@ struct vgic_v5_its *vgic_v5_msi_to_its(struct kvm *kvm, struct kvm_msi *msi);
 int vgic_v5_its_resolve_lpi(struct kvm *kvm, struct vgic_v5_its *its,
 			    u32 device_id, u32 event_id,
 			    struct vgic_irq **irq);
+int vgic_v5_its_resolve_lpi_locked(struct kvm *kvm,
+				   struct vgic_v5_its *its,
+				   u32 device_id, u32 event_id,
+				   struct vgic_irq **irq,
+				   void **owner);
 int vgic_v5_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi);
 int vgic_v5_its_inject_cached_translation(struct kvm *kvm, struct kvm_msi *msi);
+int kvm_vgic_v5_set_forwarding(struct kvm *kvm, int irq,
+			       struct kvm_kernel_irq_routing_entry *irq_entry);
+void kvm_vgic_v5_unset_forwarding(struct kvm *kvm, int irq);
 void vgic_enable_lpis(struct kvm_vcpu *vcpu);
 void vgic_flush_pending_lpis(struct kvm_vcpu *vcpu);
 int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi);
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 376c6efa9779e..655a42bd5fcee 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -296,6 +296,13 @@ struct vgic_irq {
 	refcount_t refcount;		/* Used for LPIs */
 	u32 hwintid;			/* HW INTID number */
 	unsigned int host_irq;		/* linux irq corresponding to hwintid */
+	/*
+	 * Opaque identity of the mapping responsible for direct injection. On
+	 * invalidation, only the matching mapping may disable forwarding.
+	 * Protected by irq_lock. It can be NULL while an unforward retry is
+	 * pending after the mapping has been invalidated.
+	 */
+	void *forwarding_owner;
 	union {
 		u8 targets;			/* GICv2 target VCPUs mask */
 		u32 mpidr;			/* GICv3 target VCPU */
@@ -927,6 +934,10 @@ bool vgic_v5_ppi_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq,
 				  unsigned long flags);
 void vgic_v5_set_ppi_dvi(struct kvm_vcpu *vcpu, struct vgic_irq *irq, bool dvi);
 
+int kvm_vgic_set_forwarding(struct kvm *kvm, int irq,
+			    struct kvm_kernel_irq_routing_entry *irq_entry);
+void kvm_vgic_unset_forwarding(struct kvm *kvm, int irq);
+
 bool vgic_state_is_nested(struct kvm_vcpu *vcpu);
 
 /* CPU HP callbacks */
-- 
2.34.1

[PATCH v2 14/16] KVM: selftests: Add VGICv5 ITS attribute tests

[Sascha Bischoff]

Add coverage for the VGICv5 ITS device address and ITS_REGS attribute
groups. Userspace needs to be able to discover and save the virtual
ITS register state, while KVM must reject invalid address placement
and malformed register accesses.

Verify that the ITS device exposes only the ITS address type, reports
VGIC_ADDR_UNDEF before configuration, rejects misaligned and
out-of-IPA bases, accepts a valid 64K-aligned base, reports it back,
and rejects a second base assignment.

Exercise ITS_REGS after the ITS base is configured by checking all
exposed offsets, read/write-back access, bad offsets, bad alignment,
and bad userspace pointers. Also initialise the shared VM wrapper so
non-ITS tests safely ignore the optional ITS fd during cleanup.

Verify that nonzero IDR0-IDR2, READ_EVENT_DATAR and DT_BASER values
round-trip through ITS_REGS, and reject unsupported ID register
values.

Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
---
 tools/testing/selftests/kvm/arm64/vgic_v5.c   | 364 +++++++++++++++++-
 .../selftests/kvm/include/arm64/gic_v5.h      |  71 ++++
 2 files changed, 424 insertions(+), 11 deletions(-)

diff --git a/tools/testing/selftests/kvm/arm64/vgic_v5.c b/tools/testing/selftests/kvm/arm64/vgic_v5.c
index 5bf7d99d29147..92da9753ce8e6 100644
--- a/tools/testing/selftests/kvm/arm64/vgic_v5.c
+++ b/tools/testing/selftests/kvm/arm64/vgic_v5.c
@@ -43,6 +43,7 @@ static u64 max_phys_size;
 struct vm_gic {
 	struct kvm_vm *vm;
 	int gic_fd;
+	int its_fd;
 	u32 gic_dev_type;
 };
 
@@ -581,9 +582,20 @@ static int run_vcpu(struct kvm_vcpu *vcpu)
 	return __vcpu_run(vcpu) ? -errno : 0;
 }
 
+static void vm_gic_init(struct vm_gic *v)
+{
+	v->vm = NULL;
+	v->gic_fd = -1;
+	v->its_fd = -1;
+	v->gic_dev_type = KVM_DEV_TYPE_ARM_VGIC_V5;
+}
+
 static void vm_gic_destroy(struct vm_gic *v)
 {
-	close(v->gic_fd);
+	if (v->its_fd >= 0)
+		close(v->its_fd);
+	if (v->gic_fd >= 0)
+		close(v->gic_fd);
 	kvm_vm_free(v->vm);
 }
 
@@ -623,7 +635,7 @@ static void vgic_v5_spi_line_vm_create(struct vm_gic *v,
 {
 	u64 attr;
 
-	v->gic_dev_type = KVM_DEV_TYPE_ARM_VGIC_V5;
+	vm_gic_init(v);
 	v->vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);
 	v->gic_fd = kvm_create_device(v->vm, v->gic_dev_type);
 	*vcpu = vm_vcpu_add(v->vm, 0, guest_spi_line_code);
@@ -654,7 +666,7 @@ static void vgic_v5_lpi_ist_vm_create(struct vm_gic *v,
 	u64 attr;
 	int i;
 
-	v->gic_dev_type = KVM_DEV_TYPE_ARM_VGIC_V5;
+	vm_gic_init(v);
 	v->vm = __vm_create(VM_SHAPE_DEFAULT, VGIC_V5_LPI_NR_VCPUS, 0);
 	v->gic_fd = kvm_create_device(v->vm, v->gic_dev_type);
 
@@ -779,6 +791,12 @@ static const struct vgic_region_attr gic_v5_irs_region = {
 	.alignment = GICV5_IRS_ALIGN,
 };
 
+static const struct vgic_region_attr gic_v5_its_region = {
+	.attr = KVM_VGIC_V5_ADDR_TYPE_ITS,
+	.size = GICV5_ITS_SIZE,
+	.alignment = GICV5_ITS_ALIGN,
+};
+
 struct vgic_sysreg_attr {
 	const char	*name;
 	u32		encoding;
@@ -863,6 +881,39 @@ static const struct vgic_irs_reg_attr gic_v5_irs_regs[] = {
 	IRS_REG(GICV5_IRS_SWERR_SYNDROMER1),
 };
 
+static const struct vgic_irs_reg_attr gic_v5_its_regs[] = {
+	IRS_REG(GICV5_ITS_IDR0),
+	IRS_REG(GICV5_ITS_IDR1),
+	IRS_REG(GICV5_ITS_IDR2),
+	IRS_REG(GICV5_ITS_IIDR),
+	IRS_REG(GICV5_ITS_AIDR),
+	IRS_REG(GICV5_ITS_CR0),
+	IRS_REG(GICV5_ITS_CR1),
+	IRS_REG(GICV5_ITS_DT_BASER),
+	IRS_REG(GICV5_ITS_DT_CFGR),
+	IRS_REG(GICV5_ITS_DIDR),
+	IRS_REG(GICV5_ITS_EIDR),
+	IRS_REG(GICV5_ITS_INV_EVENTR),
+	IRS_REG(GICV5_ITS_INV_DEVICER),
+	IRS_REG(GICV5_ITS_READ_EVENTR),
+	IRS_REG(GICV5_ITS_READ_EVENT_DATAR),
+	IRS_REG(GICV5_ITS_STATUSR),
+	IRS_REG(GICV5_ITS_SYNCR),
+	IRS_REG(GICV5_ITS_SYNC_STATUSR),
+	IRS_REG(GICV5_ITS_GEN_EVENT_DIDR),
+	IRS_REG(GICV5_ITS_GEN_EVENT_EIDR),
+	IRS_REG(GICV5_ITS_GEN_EVENTR),
+	IRS_REG(GICV5_ITS_GEN_EVENT_STATUSR),
+	IRS_REG(GICV5_ITS_MEC_IDR),
+	IRS_REG(GICV5_ITS_MEC_MECID_R),
+	IRS_REG(GICV5_ITS_MPAM_IDR),
+	IRS_REG(GICV5_ITS_MPAM_PARTID_R),
+	IRS_REG(GICV5_ITS_SWERR_STATUSR),
+	IRS_REG(GICV5_ITS_SWERR_SYNDROMER0),
+	IRS_REG(GICV5_ITS_SWERR_SYNDROMER1),
+	IRS_REG(GICV5_ITS_MPAM_PMGR),
+};
+
 static const struct vgic_sysreg_attr gic_v5_cpu_sysregs[] = {
 	SR(SYS_ICC_ICSR_EL1),
 	SR(SYS_ICC_PPI_ENABLER0_EL1),
@@ -945,7 +996,7 @@ static void test_vgic_v5_addr_attrs(void)
 	u64 addr;
 	int ret;
 
-	v.gic_dev_type = KVM_DEV_TYPE_ARM_VGIC_V5;
+	vm_gic_init(&v);
 	v.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);
 	v.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);
 
@@ -1001,6 +1052,7 @@ static void test_vgic_v5_addr_attrs(void)
 	vm_gic_destroy(&v);
 
 	/* Try running a VM without ever setting the IRS base addr */
+	vm_gic_init(&v);
 	v.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);
 	v.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);
 	vcpu = vm_vcpu_add(v.vm, 0, NULL);
@@ -1019,7 +1071,7 @@ static void test_vgic_v5_nr_irqs_attrs(void)
 	u32 nr_irqs;
 	int ret;
 
-	v.gic_dev_type = KVM_DEV_TYPE_ARM_VGIC_V5;
+	vm_gic_init(&v);
 	v.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);
 	v.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);
 
@@ -1077,6 +1129,7 @@ static void test_vgic_v5_nr_irqs_attrs(void)
 	vm_gic_destroy(&v);
 
 	/* If userspace does not set NR_IRQS, init selects the default. */
+	vm_gic_init(&v);
 	v.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);
 	v.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);
 	vcpu = vm_vcpu_add(v.vm, 0, NULL);
@@ -1096,7 +1149,6 @@ static void test_vgic_v5_nr_irqs_attrs(void)
 	TEST_ASSERT(ret && errno == EBUSY, "GICv5 NR_IRQS set after init");
 
 	vm_gic_destroy(&v);
-
 }
 
 static void test_vgic_v5_irs_regs_attrs(void)
@@ -1106,7 +1158,7 @@ static void test_vgic_v5_irs_regs_attrs(void)
 	u64 attr, val;
 	int ret, i;
 
-	v.gic_dev_type = KVM_DEV_TYPE_ARM_VGIC_V5;
+	vm_gic_init(&v);
 	v.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);
 	v.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);
 	vcpu = vm_vcpu_add(v.vm, 0, NULL);
@@ -1287,6 +1339,290 @@ static void test_vgic_v5_irs_regs_attrs(void)
 	vm_gic_destroy(&v);
 }
 
+static void test_vgic_v5_its_attrs(void)
+{
+	struct vm_gic v;
+	u64 attr, val;
+	int ret, i;
+
+	vm_gic_init(&v);
+	v.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);
+	v.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);
+	v.its_fd = kvm_create_device(v.vm, KVM_DEV_TYPE_ARM_VGIC_V5_ITS);
+
+	kvm_has_device_attr(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
+			    gic_v5_its_region.attr);
+
+	/* We should have the ITS Address attr */
+	ret = __kvm_has_device_attr(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
+				    KVM_VGIC_V5_ADDR_TYPE_ITS);
+	TEST_ASSERT(!ret, "GICv5 ITS address attr missing");
+
+	/* But not the IRS one */
+	ret = __kvm_has_device_attr(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
+				    KVM_VGIC_V5_ADDR_TYPE_IRS);
+	TEST_ASSERT(ret && errno == ENXIO, "GICv5 ITS accepted IRS address attr");
+
+	/* Default address is undefined */
+	ret = __kvm_device_attr_get(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
+				    KVM_VGIC_V5_ADDR_TYPE_ITS, &attr);
+	TEST_ASSERT(!ret && attr == (-1ULL),
+		    "GICv5 ITS returns VGIC_ADDR_UNDEF");
+
+	/* Misaligned address */
+	attr = gic_v5_its_region.alignment / 0x10;
+	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
+				    KVM_VGIC_V5_ADDR_TYPE_ITS, &attr);
+	TEST_ASSERT(ret && errno == EINVAL, "GICv5 ITS base not aligned");
+
+	/* Address out of PA range */
+	attr = max_phys_size;
+	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
+				    KVM_VGIC_V5_ADDR_TYPE_ITS, &attr);
+	TEST_ASSERT(ret && errno == E2BIG, "GICv5 ITS address beyond IPA limit");
+
+	/* Address + size too big to fit in PA range */
+	attr = max_phys_size - gic_v5_its_region.alignment;
+	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
+				    KVM_VGIC_V5_ADDR_TYPE_ITS, &attr);
+	TEST_ASSERT(ret && errno == E2BIG,
+		    "half of the ITS is beyond IPA limit");
+
+	/* Set a valid address */
+	attr = GICV5_ITS_BASE_GPA;
+	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
+				    KVM_VGIC_V5_ADDR_TYPE_ITS, &attr);
+	TEST_ASSERT(!ret, "GICv5 ITS base correctly set");
+
+	/* Read back the set address */
+	attr = 0xbad;
+	ret = __kvm_device_attr_get(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
+				    KVM_VGIC_V5_ADDR_TYPE_ITS, &attr);
+	TEST_ASSERT(!ret && attr == GICV5_ITS_BASE_GPA,
+		    "GICv5 ITS base readback mismatch");
+
+	/* Attempt to set the address again */
+	attr = GICV5_ITS_BASE_GPA + gic_v5_its_region.alignment;
+	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
+				    KVM_VGIC_V5_ADDR_TYPE_ITS, &attr);
+	TEST_ASSERT(ret && errno == EEXIST, "GICv5 ITS base set again");
+
+	/* Check existing group/attribute */
+	kvm_has_device_attr(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+			    GICV5_ITS_IDR0);
+
+	/* Check MMIO by checking if present, reading, then writing back */
+	for (i = 0; i < ARRAY_SIZE(gic_v5_its_regs); i++) {
+		attr = gic_v5_its_regs[i].attr;
+		ret = __kvm_has_device_attr(v.its_fd,
+					    KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+					    attr);
+		TEST_ASSERT(!ret, "GICv5 ITS_REGS missing %s",
+			    gic_v5_its_regs[i].name);
+
+		val = 0xbad;
+		ret = __kvm_device_attr_get(v.its_fd,
+					    KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+					    attr, &val);
+		TEST_ASSERT(!ret, "GICv5 ITS_REGS get failed for %s",
+			    gic_v5_its_regs[i].name);
+
+		ret = __kvm_device_attr_set(v.its_fd,
+					    KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+					    attr, &val);
+		TEST_ASSERT(!ret, "GICv5 ITS_REGS set failed for %s",
+			    gic_v5_its_regs[i].name);
+	}
+
+	/* ID registers are guest read-only, but userspace can restore sane values. */
+	ret = __kvm_device_attr_get(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_IDR0, &val);
+	TEST_ASSERT(!ret, "GICv5 ITS_REGS get IDR0 failed");
+	val |= FIELD_PREP(GICV5_ITS_IDR0_ITSID, 1);
+	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_IDR0, &val);
+	TEST_ASSERT(!ret, "GICv5 ITS_REGS rejected IDR0 restore");
+	ret = __kvm_device_attr_get(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_IDR0, &attr);
+	TEST_ASSERT(!ret && attr == val, "GICv5 ITS_REGS IDR0 restore mismatch");
+	attr = (val & ~GICV5_ITS_IDR0_DOM) |
+		FIELD_PREP(GICV5_ITS_IDR0_DOM, GICV5_ITS_IDR0_DOMAIN_SECURE);
+	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_IDR0, &attr);
+	TEST_ASSERT(ret && errno == EINVAL, "GICv5 ITS_REGS accepted secure IDR0");
+	attr = val | GICV5_ITS_IDR0_SWE;
+	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_IDR0, &attr);
+	TEST_ASSERT(ret && errno == EINVAL, "GICv5 ITS_REGS accepted SWE");
+	attr = (val & ~GICV5_ITS_IDR0_PA_RANGE) |
+		FIELD_PREP(GICV5_ITS_IDR0_PA_RANGE, 0xf);
+	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_IDR0, &attr);
+	TEST_ASSERT(ret && errno == EINVAL, "GICv5 ITS_REGS accepted PA range");
+	attr = val | BIT(9);
+	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_IDR0, &attr);
+	TEST_ASSERT(ret && errno == EINVAL, "GICv5 ITS_REGS accepted IDR0 reserved bit");
+	attr = val | BIT_ULL(32);
+	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_IDR0, &attr);
+	TEST_ASSERT(ret && errno == EINVAL, "GICv5 ITS_REGS accepted IDR0 upper bit");
+
+	ret = __kvm_device_attr_get(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_IDR1, &val);
+	TEST_ASSERT(!ret, "GICv5 ITS_REGS get IDR1 failed");
+	val = (val & ~GICV5_ITS_IDR1_DEVICEID_BITS) |
+	      FIELD_PREP(GICV5_ITS_IDR1_DEVICEID_BITS, 8);
+	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_IDR1, &val);
+	TEST_ASSERT(!ret, "GICv5 ITS_REGS rejected IDR1 restore");
+	ret = __kvm_device_attr_get(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_IDR1, &attr);
+	TEST_ASSERT(!ret && attr == val, "GICv5 ITS_REGS IDR1 restore mismatch");
+	attr = (val & ~GICV5_ITS_IDR1_DEVICEID_BITS) |
+		FIELD_PREP(GICV5_ITS_IDR1_DEVICEID_BITS, 17);
+	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_IDR1, &attr);
+	TEST_ASSERT(ret && errno == EINVAL, "GICv5 ITS_REGS accepted bad IDR1");
+	attr = val | BIT(11);
+	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_IDR1, &attr);
+	TEST_ASSERT(ret && errno == EINVAL, "GICv5 ITS_REGS accepted IDR1 reserved bit");
+
+	ret = __kvm_device_attr_get(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_IDR2, &val);
+	TEST_ASSERT(!ret, "GICv5 ITS_REGS get IDR2 failed");
+	val = (val & ~GICV5_ITS_IDR2_EVENTID_BITS) |
+	      FIELD_PREP(GICV5_ITS_IDR2_EVENTID_BITS, 8);
+	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_IDR2, &val);
+	TEST_ASSERT(!ret, "GICv5 ITS_REGS rejected IDR2 restore");
+	ret = __kvm_device_attr_get(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_IDR2, &attr);
+	TEST_ASSERT(!ret && attr == val, "GICv5 ITS_REGS IDR2 restore mismatch");
+	attr = val | GICV5_ITS_IDR2_XDMN_EVENTS;
+	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_IDR2, &attr);
+	TEST_ASSERT(ret && errno == EINVAL, "GICv5 ITS_REGS accepted XDMN events");
+	attr = (val & ~GICV5_ITS_IDR2_EVENTID_BITS) |
+		FIELD_PREP(GICV5_ITS_IDR2_EVENTID_BITS, 17);
+	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_IDR2, &attr);
+	TEST_ASSERT(ret && errno == EINVAL, "GICv5 ITS_REGS accepted bad IDR2");
+	attr = val | BIT(7);
+	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_IDR2, &attr);
+	TEST_ASSERT(ret && errno == EINVAL, "GICv5 ITS_REGS accepted IDR2 reserved bit");
+
+	/* Userspace can restore the complete read-event result. */
+	val = GICV5_ITS_READ_EVENT_DATAR_VIRT |
+	      GICV5_ITS_READ_EVENT_DATAR_VALID |
+	      FIELD_PREP(GICV5_ITS_READ_EVENT_DATAR_VM_ID, 0x1234) |
+	      FIELD_PREP(GICV5_ITS_READ_EVENT_DATAR_LPI_ID, 0x12345);
+	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_READ_EVENT_DATAR, &val);
+	TEST_ASSERT(!ret, "GICv5 ITS_REGS rejected READ_EVENT_DATAR restore");
+	ret = __kvm_device_attr_get(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_READ_EVENT_DATAR, &attr);
+	TEST_ASSERT(!ret && attr == val,
+		    "GICv5 ITS_REGS READ_EVENT_DATAR restore mismatch");
+
+	/* DT_BASER uses a full-width userspace access. */
+	val = BIT_ULL(40) | BIT_ULL(16);
+	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_DT_BASER, &val);
+	TEST_ASSERT(!ret, "GICv5 ITS_REGS rejected DT_BASER update");
+	ret = __kvm_device_attr_get(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_DT_BASER, &attr);
+	TEST_ASSERT(!ret && attr == val,
+		    "GICv5 ITS_REGS DT_BASER update mismatch");
+
+	/* Check setting and getting an out-of-range offset */
+	attr = GICV5_ITS_SIZE;
+	ret = __kvm_has_device_attr(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    attr);
+	TEST_ASSERT(ret && errno == ENXIO, "GICv5 ITS_REGS accepted bad offset");
+
+	ret = __kvm_device_attr_get(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    attr, &val);
+	TEST_ASSERT(ret && errno == ENXIO, "GICv5 ITS_REGS get bad offset");
+
+	/* Check (mis)alignment */
+	attr = GICV5_ITS_IDR0 + 2;
+	ret = __kvm_has_device_attr(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    attr);
+	TEST_ASSERT(ret && errno == EINVAL,
+		    "GICv5 ITS_REGS accepted unaligned 32-bit attr");
+
+	attr = GICV5_ITS_DT_BASER + 4;
+	ret = __kvm_has_device_attr(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    attr);
+	TEST_ASSERT(ret && errno == EINVAL,
+		    "GICv5 ITS_REGS accepted unaligned 64-bit attr");
+
+	/* Fail to provide attr pointer */
+	ret = __kvm_device_attr_get(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_IDR0, NULL);
+	TEST_ASSERT(ret && errno == EFAULT, "GICv5 ITS_REGS get with bad pointer");
+
+	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_IDR0, NULL);
+	TEST_ASSERT(ret && errno == EFAULT, "GICv5 ITS_REGS set with bad pointer");
+
+	/* Make sure we can restore the vITS's enable state */
+	val = GICV5_ITS_CR0_ITSEN;
+	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_CR0, &val);
+	TEST_ASSERT(!ret, "GICv5 ITS_REGS failed to enable ITS");
+	ret = __kvm_device_attr_get(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_CR0, &attr);
+	TEST_ASSERT(!ret && (attr & GICV5_ITS_CR0_ITSEN),
+		    "GICv5 ITS_REGS did not enable ITS");
+
+	/* And DT CFGR/BASER */
+	val = FIELD_PREP(GICV5_ITS_DT_CFGR_DEVICEID_BITS, 8);
+	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_DT_CFGR, &val);
+	TEST_ASSERT(!ret, "GICv5 ITS_REGS rejected enabled DT_CFGR update");
+	ret = __kvm_device_attr_get(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_DT_CFGR, &attr);
+	TEST_ASSERT(!ret && attr == val,
+		    "GICv5 ITS_REGS DT_CFGR restore mismatch");
+
+	val = 0;
+	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_DT_BASER, &val);
+	TEST_ASSERT(!ret, "GICv5 ITS_REGS rejected enabled DT_BASER update");
+	ret = __kvm_device_attr_get(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_DT_BASER, &val);
+	TEST_ASSERT(!ret && !val, "GICv5 ITS_REGS DT_BASER restore mismatch");
+
+	vm_gic_destroy(&v);
+
+	vm_gic_init(&v);
+	v.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);
+	v.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);
+	v.its_fd = kvm_create_device(v.vm, KVM_DEV_TYPE_ARM_VGIC_V5_ITS);
+
+	/* The registers exist before setting the ITS address. */
+	ret = __kvm_has_device_attr(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_IDR0);
+	TEST_ASSERT(!ret, "GICv5 ITS_REGS missing without ITS base");
+
+	/* Accessing them still requires the ITS address. */
+	val = 0;
+	ret = __kvm_device_attr_get(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_IDR0, &val);
+	TEST_ASSERT(ret && errno == ENXIO,
+		    "GICv5 ITS_REGS get accepted without ITS base");
+	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_IDR0, &val);
+	TEST_ASSERT(ret && errno == ENXIO,
+		    "GICv5 ITS_REGS set accepted without ITS base");
+
+	vm_gic_destroy(&v);
+}
+
 static void test_vgic_v5_ist_attrs(void)
 {
 	struct kvm_vgic_v5_ist ist_attr;
@@ -1296,7 +1632,7 @@ static void test_vgic_v5_ist_attrs(void)
 	u64 attr;
 	int ret;
 
-	v.gic_dev_type = KVM_DEV_TYPE_ARM_VGIC_V5;
+	vm_gic_init(&v);
 	v.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);
 	v.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);
 	vcpu = vm_vcpu_add(v.vm, 0, NULL);
@@ -1383,6 +1719,7 @@ static void test_vgic_v5_ist_attrs(void)
 	vm_gic_destroy(&v);
 
 	/* IST restore is rejected after the VM has run. */
+	vm_gic_init(&v);
 	v.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);
 	v.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);
 	vcpu = vm_vcpu_add(v.vm, 0, guest_code);
@@ -1411,7 +1748,7 @@ static void test_vgic_v5_userspace_ppis_attrs(void)
 	u64 user_ppis[2];
 	int ret;
 
-	v.gic_dev_type = KVM_DEV_TYPE_ARM_VGIC_V5;
+	vm_gic_init(&v);
 	v.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);
 	v.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);
 	vcpu = vm_vcpu_add(v.vm, 0, NULL);
@@ -1452,7 +1789,7 @@ static void test_vgic_v5_cpu_sysreg_attrs(void)
 	u64 attr, val;
 	int ret, i;
 
-	v.gic_dev_type = KVM_DEV_TYPE_ARM_VGIC_V5;
+	vm_gic_init(&v);
 	v.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);
 	v.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);
 	vcpu = vm_vcpu_add(v.vm, 0, NULL);
@@ -1544,7 +1881,7 @@ static void test_vgic_v5_sparse_vcpu_ids(void)
 	struct vm_gic v;
 	u64 attr;
 
-	v.gic_dev_type = KVM_DEV_TYPE_ARM_VGIC_V5;
+	vm_gic_init(&v);
 	v.vm = __vm_create(VM_SHAPE_DEFAULT, VGIC_V5_SPARSE_NR_VCPUS, 0);
 	v.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);
 
@@ -1588,6 +1925,7 @@ static void test_vgic_v5_ppis(u32 gic_dev_type)
 	uint64_t attr;
 	int ret, i;
 
+	vm_gic_init(&v);
 	v.gic_dev_type = gic_dev_type;
 	v.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);
 
@@ -1983,6 +2321,7 @@ int test_kvm_device(u32 gic_dev_type)
 	struct vm_gic v;
 	int ret;
 
+	vm_gic_init(&v);
 	v.vm = vm_create_with_vcpus(NR_VCPUS, guest_code, vcpus);
 
 	/* try to create a non existing KVM device */
@@ -2046,6 +2385,9 @@ void run_tests(u32 gic_dev_type)
 	pr_info("Test VGICv5 IRS_REGS attrs\n");
 	test_vgic_v5_irs_regs_attrs();
 
+	pr_info("Test VGICv5 ITS attrs\n");
+	test_vgic_v5_its_attrs();
+
 	pr_info("Test VGICv5 IST attrs\n");
 	test_vgic_v5_ist_attrs();
 
diff --git a/tools/testing/selftests/kvm/include/arm64/gic_v5.h b/tools/testing/selftests/kvm/include/arm64/gic_v5.h
index 2d455515563e0..2e4472489601c 100644
--- a/tools/testing/selftests/kvm/include/arm64/gic_v5.h
+++ b/tools/testing/selftests/kvm/include/arm64/gic_v5.h
@@ -13,6 +13,8 @@
 /* GIC component base address is guest PA space */
 #define GICV5_IRS_CONFIG_BASE_GPA	0x8000000ULL
 #define GICV5_IRS_CONFIG_BASE_GVA	((volatile void *)GICV5_IRS_CONFIG_BASE_GPA)
+#define GICV5_ITS_BASE_GPA		0x8030000ULL
+#define GICV5_ITS_BASE_GVA		((volatile void *)GICV5_ITS_BASE_GPA)
 
 #define GICV5_IRS_SIZE			0x20000
 #define GICV5_IRS_ALIGN			0x10000
@@ -74,6 +76,75 @@
 #define GICV5_IRS_SWERR_SYNDROMER0	0x03c8
 #define GICV5_IRS_SWERR_SYNDROMER1	0x03d0
 
+/*
+ * ITS register offsets and fields used by the GICv5 ITS selftests.
+ */
+#define GICV5_ITS_IDR0			0x0000
+#define GICV5_ITS_IDR1			0x0004
+#define GICV5_ITS_IDR2			0x0008
+#define GICV5_ITS_IIDR			0x0040
+#define GICV5_ITS_AIDR			0x0044
+#define GICV5_ITS_CR0			0x0080
+#define GICV5_ITS_CR1			0x0084
+#define GICV5_ITS_DT_BASER		0x00c0
+#define GICV5_ITS_DT_CFGR		0x00d0
+#define GICV5_ITS_DIDR			0x0100
+#define GICV5_ITS_EIDR			0x0108
+#define GICV5_ITS_INV_EVENTR		0x010c
+#define GICV5_ITS_INV_DEVICER		0x0110
+#define GICV5_ITS_READ_EVENTR		0x0114
+#define GICV5_ITS_READ_EVENT_DATAR	0x0118
+#define GICV5_ITS_STATUSR		0x0120
+#define GICV5_ITS_SYNCR			0x0140
+#define GICV5_ITS_SYNC_STATUSR		0x0148
+#define GICV5_ITS_GEN_EVENT_DIDR	0x0180
+#define GICV5_ITS_GEN_EVENT_EIDR	0x0188
+#define GICV5_ITS_GEN_EVENTR		0x018c
+#define GICV5_ITS_GEN_EVENT_STATUSR	0x0190
+#define GICV5_ITS_MEC_IDR		0x01c0
+#define GICV5_ITS_MEC_MECID_R		0x01c4
+#define GICV5_ITS_MPAM_IDR		0x0200
+#define GICV5_ITS_MPAM_PARTID_R		0x0204
+#define GICV5_ITS_SWERR_STATUSR		0x0240
+#define GICV5_ITS_SWERR_SYNDROMER0	0x0248
+#define GICV5_ITS_SWERR_SYNDROMER1	0x0250
+#define GICV5_ITS_MPAM_PMGR		0x0258
+#define GICV5_ITS_TRANSLATER		0x10000
+#define GICV5_ITS_RL_TRANSLATER		0x10008
+
+#define GICV5_ITS_SIZE			0x20000
+#define GICV5_ITS_ALIGN			0x10000
+#define GICV5_ITS_IDR0_ITSID		GENMASK(31, 16)
+#define GICV5_ITS_IDR0_SWE		BIT(8)
+#define GICV5_ITS_IDR0_PA_RANGE		GENMASK(5, 2)
+#define GICV5_ITS_IDR0_DOM		GENMASK(1, 0)
+#define GICV5_ITS_IDR0_DOMAIN_SECURE	0b00
+#define GICV5_ITS_IDR1_DEVICEID_BITS	GENMASK(5, 0)
+#define GICV5_ITS_IDR2_XDMN_EVENTS	GENMASK(6, 5)
+#define GICV5_ITS_IDR2_EVENTID_BITS	GENMASK(4, 0)
+
+#define GICV5_ITS_CR0_ITSEN		BIT(0)
+#define GICV5_ITS_DT_BASER_ADDR_MASK	GENMASK_ULL(55, 3)
+#define GICV5_ITS_DT_BASER_ADDR_SHIFT	3ULL
+#define GICV5_ITS_DT_CFGR_STRUCTURE	BIT(16)
+#define GICV5_ITS_DT_CFGR_L2SZ		GENMASK(7, 6)
+#define GICV5_ITS_DT_CFGR_DEVICEID_BITS	GENMASK(5, 0)
+#define GICV5_ITS_DIDR_DEVICEID		GENMASK_ULL(31, 0)
+#define GICV5_ITS_EIDR_EVENTID		GENMASK(15, 0)
+#define GICV5_ITS_INV_EVENTR_I		BIT(31)
+#define GICV5_ITS_INV_EVENTR_ITT_L2SZ	GENMASK(2, 1)
+#define GICV5_ITS_INV_EVENTR_L1		BIT(0)
+#define GICV5_ITS_INV_DEVICER_I		BIT(31)
+#define GICV5_ITS_INV_DEVICER_EVENTID_BITS GENMASK(5, 1)
+#define GICV5_ITS_INV_DEVICER_L1	BIT(0)
+#define GICV5_ITS_READ_EVENT_DATAR_VIRT	BIT_ULL(63)
+#define GICV5_ITS_READ_EVENT_DATAR_VM_ID GENMASK_ULL(47, 32)
+#define GICV5_ITS_READ_EVENT_DATAR_VALID BIT_ULL(31)
+#define GICV5_ITS_READ_EVENT_DATAR_LPI_ID GENMASK_ULL(23, 0)
+#define GICV5_ITS_STATUSR_IDLE		BIT(0)
+#define GICV5_ITS_SYNC_STATUSR_IDLE	BIT(0)
+#define GICV5_ITS_GEN_EVENT_STATUSR_IDLE BIT(0)
+
 #define GICV5_IRS_IDR0_SETLPI		BIT(9)
 #define GICV5_IRS_IDR0_INT_DOM		GENMASK(1, 0)
 #define GICV5_IRS_IDR0_INT_DOM_SECURE		0b00
-- 
2.34.1

[PATCH v2 15/16] KVM: selftests: Add VGICv5 ITS translation cache tests

[Sascha Bischoff]

Add end-to-end coverage for VGICv5 ITS translation and cache
invalidation. The ITS attribute test only checks the userspace API
surface; this test verifies that KVM can parse guest-provided linear
DT and ITT state and use it to deliver LPIs through guest GEN_EVENTR
and KVM_SIGNAL_MSI.

Build a two-VPE VM with IRS, ITS and an LPI IST. VPE0 programs a
linear device table and interrupt translation table, enables the
virtual ITS, enables two LPIs targeting VPE1, and sends GEN_EVENTR to
populate the translation cache with EventID to LPI A. VPE1 reports the
LPI identity it consumed.

VPE0 then rewrites the ITT to point at LPI B without invalidation, and
a host KVM_SIGNAL_MSI injection verifies that the stale cached LPI A
translation is reused. VPE0 invalidates the event with ITS_INV_EVENTR
and the host verifies that LPI B is delivered.

VPE0 then rewrites the ITT back to LPI A, invalidates the device with
ITS_INV_DEVICER, and the host verifies that LPI A is delivered again.
This exercises the guest-side event and device invalidation paths
rather than invalidating translations directly from the test host.

Also cover negative MSI paths by checking that KVM_SIGNAL_MSI rejects a
message without KVM_MSI_VALID_DEVID and rejects a message whose address
does not target the GICv5 ITS. Finally, inject once before guest-side
ITS disable to prove delivery still works, then have VPE0 disable the
ITS and verify that a cached KVM_SIGNAL_MSI delivery is rejected with
ENXIO.

Check 32-bit accesses to both halves of DT_BASER and preservation of
the untouched half. Verify that guest writes to the ITS ID registers
and READ_EVENT_DATAR are ignored, and that READ_EVENT_DATAR can be read
at both 32-bit and 64-bit widths.

Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
---
 tools/testing/selftests/kvm/arm64/vgic_v5.c   | 518 +++++++++++++++++-
 .../selftests/kvm/include/arm64/gic_v5.h      |  17 +
 2 files changed, 532 insertions(+), 3 deletions(-)

diff --git a/tools/testing/selftests/kvm/arm64/vgic_v5.c b/tools/testing/selftests/kvm/arm64/vgic_v5.c
index 92da9753ce8e6..a9b0bca24ec08 100644
--- a/tools/testing/selftests/kvm/arm64/vgic_v5.c
+++ b/tools/testing/selftests/kvm/arm64/vgic_v5.c
@@ -18,6 +18,7 @@
 #define VGIC_V5_SPARSE_VCPU0_ID	17
 #define VGIC_V5_SPARSE_VCPU1_ID	3
 #define VGIC_V5_LPI_MEMSLOT	1
+#define VGIC_V5_ITS_TABLE_MEMSLOT	2
 #define VGIC_V5_NR_PRIVATE_IRQS	64
 #define VGIC_V5_DEFAULT_NR_SPIS	32
 #define VGIC_V5_MAX_NR_SPIS	BIT(10)
@@ -25,8 +26,19 @@
 #define VGIC_V5_IRS_WAIT_RETRIES	1000000
 #define VGIC_V5_LPI_IST_BASE_GPA	(GICV5_IRS_CONFIG_BASE_GPA + VGIC_V5_IRS_SIZE)
 #define VGIC_V5_LPI_IST_SIZE		0x10000
+/* A linear DT with 17 DeviceID bits needs 1 MiB of aligned storage. */
+#define VGIC_V5_ITS_DT_BASE_GPA		0x8100000ULL
+#define VGIC_V5_ITS_ITT_BASE_GPA	(VGIC_V5_ITS_DT_BASE_GPA + 0x100000)
+#define VGIC_V5_ITS_TABLE_SIZE		0x110000
 #define LPI_TEST_TO_VPE1		0
 #define LPI_TEST_TO_VPE0		1
+#define ITS_TEST_DEVICE_ID		5
+#define ITS_TEST_OVERWIDE_DEVICE_ID	BIT(16)
+#define ITS_TEST_EVENT_ID		3
+#define ITS_TEST_LPI_A			2
+#define ITS_TEST_LPI_B			3
+#define ITS_TEST_DEVICE_ID_BITS		17
+#define ITS_TEST_EVENT_ID_BITS		4
 #define RESTORE_TEST_LPI_PENDING		0
 #define RESTORE_TEST_LPI_ENABLED		1
 #define RESTORE_TEST_SPI_PRIO_HIGH		0
@@ -56,6 +68,13 @@ struct vm_gic {
 #define GUEST_CMD_IST_SOURCE_READY	16
 #define GUEST_CMD_IST_SPI_INJECT_READY	17
 #define GUEST_CMD_IST_LPI_INJECT_READY	18
+#define GUEST_CMD_ITS_READY		19
+#define GUEST_CMD_ITS_LPI_DELIVERED	20
+#define GUEST_CMD_ITS_GEN_EVENT_SENT	21
+#define GUEST_CMD_ITS_ITT_REWRITTEN	22
+#define GUEST_CMD_ITS_EVENT_INVALIDATED	23
+#define GUEST_CMD_ITS_DEVICE_INVALIDATED	24
+#define GUEST_CMD_ITS_DISABLED		25
 
 static struct kvm_vgic_v5_ist vgic_v5_ist_attr(void *spi_ist, size_t spi_size,
 					       void *lpi_ist, size_t lpi_size)
@@ -71,6 +90,7 @@ static struct kvm_vgic_v5_ist vgic_v5_ist_attr(void *spi_ist, size_t spi_size,
 static u32 spi_line_expected;
 static bool spi_line_level_sensitive;
 static bool lpi_ist_ready;
+static u32 its_expected_lpi;
 static u32 ist_restore_irq_count[VGIC_V5_LPI_NR_VCPUS];
 
 static u64 gicv5_hwirq(u32 type, u32 intid)
@@ -222,6 +242,199 @@ static void gicv5_configure_lpi_ist(void)
 			      GICV5_IRS_IST_BASER), val);
 }
 
+/* We only use a single DeviceID and hence only have a single ITT */
+static void gicv5_write_its_itte(u32 event_id, u32 lpi)
+{
+	u64 val;
+
+	val = FIELD_PREP(GICV5_ITTL2E_LPI_ID, lpi) | GICV5_ITTL2E_VALID;
+	WRITE_ONCE(*(u64 *)(VGIC_V5_ITS_ITT_BASE_GPA +
+			    event_id * sizeof(u64)), val);
+}
+
+/*
+ * Set up the ITS "tables". Point the DTE to the ITT and make valid. Then set up
+ * the translation in the ITT. We are cheating a bit here, and have the most
+ * simplistic table setup possible. A linear DT with two DeviceIDs pointing
+ * to the same linear ITT; the second ID exceeds the advertised width.
+ */
+static void gicv5_configure_its_tables(u32 lpi)
+{
+	u64 val;
+
+	val = FIELD_PREP(GICV5_DTL2E_ITT_ADDR_MASK,
+			 VGIC_V5_ITS_ITT_BASE_GPA >> GICV5_ITS_DT_BASER_ADDR_SHIFT) |
+	      FIELD_PREP(GICV5_DTL2E_EVENT_ID_BITS, ITS_TEST_EVENT_ID_BITS) |
+	      GICV5_DTL2E_VALID;
+	WRITE_ONCE(*(u64 *)(VGIC_V5_ITS_DT_BASE_GPA +
+			    ITS_TEST_DEVICE_ID * sizeof(u64)), val);
+	WRITE_ONCE(*(u64 *)(VGIC_V5_ITS_DT_BASE_GPA +
+			    ITS_TEST_OVERWIDE_DEVICE_ID * sizeof(u64)), val);
+
+	gicv5_write_its_itte(ITS_TEST_EVENT_ID, lpi);
+}
+
+/* Set ITS_DT_CFGR, ITS_DT_BASER, and enable the ITS */
+static void gicv5_configure_its(void)
+{
+	u64 val;
+	u64 test_baser;
+	u32 idr0, lower, upper;
+
+	idr0 = readl(GICV5_ITS_BASE_GVA + GICV5_ITS_IDR0);
+	writel(idr0 ^ GICV5_ITS_IDR0_SWE, GICV5_ITS_BASE_GVA + GICV5_ITS_IDR0);
+	GUEST_ASSERT_EQ(readl(GICV5_ITS_BASE_GVA + GICV5_ITS_IDR0), idr0);
+
+	/* A 32-bit write must leave the other half of DT_BASER intact. */
+	test_baser = BIT_ULL(40) | BIT_ULL(16);
+	writeq(test_baser, GICV5_ITS_BASE_GVA + GICV5_ITS_DT_BASER);
+	lower = BIT(24);
+	writel(lower, GICV5_ITS_BASE_GVA + GICV5_ITS_DT_BASER);
+	GUEST_ASSERT_EQ(readq(GICV5_ITS_BASE_GVA + GICV5_ITS_DT_BASER),
+			(BIT_ULL(40) | lower));
+	upper = BIT(20);
+	writel(upper, GICV5_ITS_BASE_GVA + GICV5_ITS_DT_BASER + 4);
+	GUEST_ASSERT_EQ(readq(GICV5_ITS_BASE_GVA + GICV5_ITS_DT_BASER),
+			(((u64)upper << 32) | lower));
+	GUEST_ASSERT_EQ(readl(GICV5_ITS_BASE_GVA + GICV5_ITS_DT_BASER), lower);
+	GUEST_ASSERT_EQ(readl(GICV5_ITS_BASE_GVA + GICV5_ITS_DT_BASER + 4),
+			upper);
+
+	val = FIELD_PREP(GICV5_ITS_DT_BASER_ADDR_MASK,
+			 VGIC_V5_ITS_DT_BASE_GPA >> GICV5_ITS_DT_BASER_ADDR_SHIFT);
+	writeq(val, GICV5_ITS_BASE_GVA + GICV5_ITS_DT_BASER);
+
+	val = FIELD_PREP(GICV5_ITS_DT_CFGR_DEVICEID_BITS,
+			 ITS_TEST_DEVICE_ID_BITS);
+	writel(val, GICV5_ITS_BASE_GVA + GICV5_ITS_DT_CFGR);
+
+	writel(GICV5_ITS_CR0_ITSEN, GICV5_ITS_BASE_GVA + GICV5_ITS_CR0);
+	GUEST_ASSERT(readl(GICV5_ITS_BASE_GVA + GICV5_ITS_CR0) & GICV5_ITS_CR0_ITSEN);
+}
+
+/*
+ * The DT registers are read-only to the guest while the ITS is enabled. Write
+ * modified versions of the current values and check that the writes are
+ * ignored. Avoid literal zero here: the arm64 selftest MMIO helpers allow a
+ * zero-immediate operand, but a store instruction needs a register operand.
+ */
+static void gicv5_check_enabled_its_dt_regs_ro(void)
+{
+	const u64 dt_baser = readq(GICV5_ITS_BASE_GVA + GICV5_ITS_DT_BASER);
+	const u32 dt_cfgr = readl(GICV5_ITS_BASE_GVA + GICV5_ITS_DT_CFGR);
+	const u64 new_dt_baser = dt_baser ^ BIT_ULL(16);
+	const u32 new_dt_cfgr = dt_cfgr ^ BIT(0);
+
+	writeq(new_dt_baser, GICV5_ITS_BASE_GVA + GICV5_ITS_DT_BASER);
+	writel((u32)(dt_baser >> 32) ^ BIT(0),
+	       GICV5_ITS_BASE_GVA + GICV5_ITS_DT_BASER + 4);
+	writel(new_dt_cfgr, GICV5_ITS_BASE_GVA + GICV5_ITS_DT_CFGR);
+
+	GUEST_ASSERT_EQ(readq(GICV5_ITS_BASE_GVA + GICV5_ITS_DT_BASER),
+			dt_baser);
+	GUEST_ASSERT_EQ(readl(GICV5_ITS_BASE_GVA + GICV5_ITS_DT_CFGR),
+			dt_cfgr);
+}
+
+/*
+ * Read out the Event Data from ITS_READ_EVENT_DATAR and check it to ensure the
+ * expected translation is returned.
+ */
+static void gicv5_check_its_read_event(u32 device_id, u32 event_id, u32 lpi)
+{
+	u64 val;
+	u32 new_datar;
+
+	val = FIELD_PREP(GICV5_ITS_DIDR_DEVICEID, device_id);
+	writeq(val, GICV5_ITS_BASE_GVA + GICV5_ITS_DIDR);
+
+	val = FIELD_PREP(GICV5_ITS_EIDR_EVENTID, event_id);
+	writel(val, GICV5_ITS_BASE_GVA + GICV5_ITS_EIDR);
+
+	writel(GICV5_ITS_READ_EVENTR_R,
+	       GICV5_ITS_BASE_GVA + GICV5_ITS_READ_EVENTR);
+
+	val = readq(GICV5_ITS_BASE_GVA + GICV5_ITS_READ_EVENT_DATAR);
+	GUEST_ASSERT(val & GICV5_ITS_READ_EVENT_DATAR_VALID);
+	GUEST_ASSERT_EQ(FIELD_GET(GICV5_ITS_READ_EVENT_DATAR_LPI_ID, val),
+			lpi);
+	/* The result register remains read-only to the guest at either width. */
+	new_datar = (u32)val ^ BIT(0);
+	writel(new_datar, GICV5_ITS_BASE_GVA + GICV5_ITS_READ_EVENT_DATAR);
+	writel((u32)(val >> 32) ^ BIT(0),
+	       GICV5_ITS_BASE_GVA + GICV5_ITS_READ_EVENT_DATAR + 4);
+	writeq(val ^ BIT_ULL(0), GICV5_ITS_BASE_GVA + GICV5_ITS_READ_EVENT_DATAR);
+	GUEST_ASSERT_EQ(readq(GICV5_ITS_BASE_GVA + GICV5_ITS_READ_EVENT_DATAR),
+			val);
+	GUEST_ASSERT_EQ(readl(GICV5_ITS_BASE_GVA + GICV5_ITS_READ_EVENT_DATAR),
+			(u32)val);
+	GUEST_ASSERT_EQ(readl(GICV5_ITS_BASE_GVA + GICV5_ITS_READ_EVENT_DATAR + 4),
+			(u32)(val >> 32));
+}
+
+static void gicv5_check_its_deviceid_cap(void)
+{
+	u32 idr1 = readl(GICV5_ITS_BASE_GVA + GICV5_ITS_IDR1);
+	u32 dt_cfgr = readl(GICV5_ITS_BASE_GVA + GICV5_ITS_DT_CFGR);
+	u64 val;
+
+	GUEST_ASSERT_EQ(FIELD_GET(GICV5_ITS_IDR1_DEVICEID_BITS, idr1),
+			ITS_TEST_DEVICE_ID_BITS - 1);
+	GUEST_ASSERT_EQ(FIELD_GET(GICV5_ITS_DT_CFGR_DEVICEID_BITS, dt_cfgr),
+			ITS_TEST_DEVICE_ID_BITS);
+
+	writeq(ITS_TEST_OVERWIDE_DEVICE_ID,
+	       GICV5_ITS_BASE_GVA + GICV5_ITS_DIDR);
+	writel(ITS_TEST_EVENT_ID, GICV5_ITS_BASE_GVA + GICV5_ITS_EIDR);
+	writel(GICV5_ITS_READ_EVENTR_R,
+	       GICV5_ITS_BASE_GVA + GICV5_ITS_READ_EVENTR);
+
+	val = readq(GICV5_ITS_BASE_GVA + GICV5_ITS_READ_EVENT_DATAR);
+	GUEST_ASSERT(!(val & GICV5_ITS_READ_EVENT_DATAR_VALID));
+}
+
+/* Inject a manually triggered translation via the ITS_GEN_EVENTR */
+static void gicv5_send_its_gen_event(u32 device_id, u32 event_id)
+{
+	u64 val;
+
+	val = FIELD_PREP(GICV5_ITS_GEN_EVENT_DIDR_DEVICEID, device_id);
+	writeq(val, GICV5_ITS_BASE_GVA + GICV5_ITS_GEN_EVENT_DIDR);
+
+	val = FIELD_PREP(GICV5_ITS_GEN_EVENT_EIDR_EVENTID, event_id);
+	writel(val, GICV5_ITS_BASE_GVA + GICV5_ITS_GEN_EVENT_EIDR);
+
+	writel(GICV5_ITS_GEN_EVENTR_R |
+	       FIELD_PREP(GICV5_ITS_GEN_EVENTR_TARGET_DOMAIN, 0),
+	       GICV5_ITS_BASE_GVA + GICV5_ITS_GEN_EVENTR);
+}
+
+static void gicv5_invalidate_its_event(u32 device_id, u32 event_id)
+{
+	u64 val;
+
+	val = FIELD_PREP(GICV5_ITS_DIDR_DEVICEID, device_id);
+	writeq(val, GICV5_ITS_BASE_GVA + GICV5_ITS_DIDR);
+
+	val = FIELD_PREP(GICV5_ITS_EIDR_EVENTID, event_id);
+	writel(val, GICV5_ITS_BASE_GVA + GICV5_ITS_EIDR);
+
+	writel(GICV5_ITS_INV_EVENTR_I,
+	       GICV5_ITS_BASE_GVA + GICV5_ITS_INV_EVENTR);
+}
+
+static void gicv5_invalidate_its_device(u32 device_id, u32 event_id_bits)
+{
+	u64 val;
+
+	val = FIELD_PREP(GICV5_ITS_DIDR_DEVICEID, device_id);
+	writeq(val, GICV5_ITS_BASE_GVA + GICV5_ITS_DIDR);
+
+	val = GICV5_ITS_INV_DEVICER_I |
+	      FIELD_PREP(GICV5_ITS_INV_DEVICER_EVENTID_BITS, event_id_bits);
+	writel(val, GICV5_ITS_BASE_GVA + GICV5_ITS_INV_DEVICER);
+}
+
 static void gicv5_enable_irs(void)
 {
 	writel(GICV5_IRS_CR0_IRSEN,
@@ -417,6 +630,81 @@ static void guest_lpi_code(void)
 		wfi();
 }
 
+static void guest_its_irq_handler(struct ex_regs *regs)
+{
+	u64 expected_hwirq = gicv5_lpi_hwirq(READ_ONCE(its_expected_lpi));
+	u32 hwirq;
+	u64 ia;
+
+	ia = gicr_insn(CDIA);
+	if (!GICV5_GICR_CDIA_VALID(ia))
+		return;
+
+	gsb_ack();
+	isb();
+
+	hwirq = FIELD_GET(GICV5_GICR_CDIA_INTID, ia);
+	GUEST_ASSERT_EQ(hwirq, expected_hwirq);
+
+	gic_insn(hwirq, CDDI);
+	gic_insn(0, CDEOI);
+
+	GUEST_SYNC(GUEST_CMD_ITS_LPI_DELIVERED);
+}
+
+static void guest_its_code(void)
+{
+	local_irq_disable();
+
+	if (!guest_get_vcpuid()) { /* VCPU0 */
+		gicv5_enable_irs();
+		gicv5_configure_lpi_ist();
+		gicv5_enable_lpi(ITS_TEST_LPI_A, 1);
+		gicv5_enable_lpi(ITS_TEST_LPI_B, 1);
+		gicv5_configure_its_tables(ITS_TEST_LPI_A);
+		gicv5_configure_its();
+		gicv5_check_enabled_its_dt_regs_ro();
+		gicv5_check_its_read_event(ITS_TEST_DEVICE_ID,
+					   ITS_TEST_EVENT_ID,
+					   ITS_TEST_LPI_A);
+		gicv5_check_its_deviceid_cap();
+		WRITE_ONCE(lpi_ist_ready, true);
+
+		GUEST_SYNC(GUEST_CMD_ITS_READY);
+
+		gicv5_send_its_gen_event(ITS_TEST_DEVICE_ID,
+					 ITS_TEST_EVENT_ID);
+
+		GUEST_SYNC(GUEST_CMD_ITS_GEN_EVENT_SENT);
+
+		gicv5_write_its_itte(ITS_TEST_EVENT_ID, ITS_TEST_LPI_B);
+		GUEST_SYNC(GUEST_CMD_ITS_ITT_REWRITTEN);
+
+		gicv5_invalidate_its_event(ITS_TEST_DEVICE_ID,
+					   ITS_TEST_EVENT_ID);
+		GUEST_SYNC(GUEST_CMD_ITS_EVENT_INVALIDATED);
+
+		gicv5_write_its_itte(ITS_TEST_EVENT_ID, ITS_TEST_LPI_A);
+		gicv5_invalidate_its_device(ITS_TEST_DEVICE_ID,
+					    ITS_TEST_EVENT_ID_BITS);
+		GUEST_SYNC(GUEST_CMD_ITS_DEVICE_INVALIDATED);
+
+		writel(0, GICV5_ITS_BASE_GVA + GICV5_ITS_CR0);
+		GUEST_ASSERT(!(readl(GICV5_ITS_BASE_GVA + GICV5_ITS_CR0) &
+			       GICV5_ITS_CR0_ITSEN));
+		GUEST_SYNC(GUEST_CMD_ITS_DISABLED);
+	} else { /* VCPU1 */
+		GUEST_ASSERT(READ_ONCE(lpi_ist_ready));
+		gicv5_cpu_enable_interrupts();
+		local_irq_enable();
+
+		GUEST_SYNC(GUEST_CMD_ITS_READY);
+	}
+
+	while (1)
+		wfi();
+}
+
 static void restore_test_setup_hwirq(u64 hwirq, u32 target_vpe,
 				     u32 priority, bool enable)
 {
@@ -610,6 +898,29 @@ static void vgic_v5_map_irs(struct kvm_vm *vm)
 		 GICV5_IRS_CONFIG_BASE_GPA, nr_irs_pages);
 }
 
+static void vgic_v5_map_its(struct kvm_vm *vm)
+{
+	unsigned int nr_its_pages;
+
+	nr_its_pages = vm_calc_num_guest_pages(vm->mode, GICV5_ITS_SIZE);
+
+	/* Map the ITS at VA == IPA so guest MMIO writes hit the ITS IODEV. */
+	virt_map(vm, GICV5_ITS_BASE_GPA, GICV5_ITS_BASE_GPA, nr_its_pages);
+}
+
+static void vgic_v5_add_its_table_mem(struct kvm_vm *vm)
+{
+	unsigned int nr_pages;
+
+	nr_pages = vm_calc_num_guest_pages(vm->mode, VGIC_V5_ITS_TABLE_SIZE);
+	vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS,
+				    VGIC_V5_ITS_DT_BASE_GPA,
+				    VGIC_V5_ITS_TABLE_MEMSLOT,
+				    nr_pages, 0);
+	virt_map(vm, VGIC_V5_ITS_DT_BASE_GPA, VGIC_V5_ITS_DT_BASE_GPA,
+		 nr_pages);
+}
+
 static u32 vgic_v5_irq_line_payload(u32 type, u32 num)
 {
 	return (type << KVM_ARM_IRQ_TYPE_SHIFT) |
@@ -660,7 +971,8 @@ static void vgic_v5_spi_line_vm_create(struct vm_gic *v,
 static void vgic_v5_lpi_ist_vm_create(struct vm_gic *v,
 				      struct kvm_vcpu *vcpus[VGIC_V5_LPI_NR_VCPUS],
 				      void (*guest_code)(void),
-				      void (*irq_handler)(struct ex_regs *))
+				      void (*irq_handler)(struct ex_regs *),
+				      bool with_its)
 {
 	unsigned int nr_lpi_ist_pages;
 	u64 attr;
@@ -669,6 +981,9 @@ static void vgic_v5_lpi_ist_vm_create(struct vm_gic *v,
 	vm_gic_init(v);
 	v->vm = __vm_create(VM_SHAPE_DEFAULT, VGIC_V5_LPI_NR_VCPUS, 0);
 	v->gic_fd = kvm_create_device(v->vm, v->gic_dev_type);
+	if (with_its)
+		v->its_fd = kvm_create_device(v->vm,
+					      KVM_DEV_TYPE_ARM_VGIC_V5_ITS);
 
 	for (i = 0; i < VGIC_V5_LPI_NR_VCPUS; i++) {
 		vcpus[i] = vm_vcpu_add(v->vm, i, guest_code);
@@ -687,6 +1002,16 @@ static void vgic_v5_lpi_ist_vm_create(struct vm_gic *v,
 			    KVM_VGIC_V5_ADDR_TYPE_IRS, &attr);
 	vgic_v5_map_irs(v->vm);
 
+	if (with_its) {
+		attr = GICV5_ITS_BASE_GPA;
+		kvm_device_attr_set(v->its_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
+				    KVM_VGIC_V5_ADDR_TYPE_ITS, &attr);
+		kvm_device_attr_set(v->its_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
+				    KVM_DEV_ARM_VGIC_CTRL_INIT, NULL);
+		vgic_v5_map_its(v->vm);
+		vgic_v5_add_its_table_mem(v->vm);
+	}
+
 	nr_lpi_ist_pages = vm_calc_num_guest_pages(v->vm->mode,
 						   VGIC_V5_LPI_IST_SIZE);
 	vm_userspace_mem_region_add(v->vm, VM_MEM_SRC_ANONYMOUS,
@@ -708,7 +1033,14 @@ static void vgic_v5_lpi_vm_create(struct vm_gic *v,
 				  struct kvm_vcpu *vcpus[VGIC_V5_LPI_NR_VCPUS])
 {
 	vgic_v5_lpi_ist_vm_create(v, vcpus, guest_lpi_code,
-				  guest_lpi_irq_handler);
+				  guest_lpi_irq_handler, false);
+}
+
+static void vgic_v5_its_vm_create(struct vm_gic *v,
+				  struct kvm_vcpu *vcpus[VGIC_V5_LPI_NR_VCPUS])
+{
+	vgic_v5_lpi_ist_vm_create(v, vcpus, guest_its_code,
+				  guest_its_irq_handler, true);
 }
 
 static void vgic_v5_ist_restore_vm_create(struct vm_gic *v,
@@ -716,7 +1048,7 @@ static void vgic_v5_ist_restore_vm_create(struct vm_gic *v,
 					  void (*guest_code)(void),
 					  void (*irq_handler)(struct ex_regs *))
 {
-	vgic_v5_lpi_ist_vm_create(v, vcpus, guest_code, irq_handler);
+	vgic_v5_lpi_ist_vm_create(v, vcpus, guest_code, irq_handler, false);
 }
 
 static void vgic_v5_run_spi_line_test(u32 nr_spis, u32 expected_spi,
@@ -2312,6 +2644,183 @@ static void test_vgic_v5_ist_save_restore(void)
 	vm_gic_destroy(&dst);
 }
 
+static int __vgic_v5_signal_msi(struct kvm_vm *vm, u64 address,
+				u32 device_id, u32 event_id, u32 flags)
+{
+	struct kvm_msi msi = {
+		.address_lo	= address,
+		.address_hi	= address >> 32,
+		.data		= event_id,
+		.devid		= device_id,
+		.flags		= flags,
+	};
+
+	return __vm_ioctl(vm, KVM_SIGNAL_MSI, &msi);
+}
+
+static void vgic_v5_signal_msi(struct kvm_vm *vm, u32 device_id,
+			       u32 event_id)
+{
+	u64 addr = GICV5_ITS_BASE_GPA + GICV5_ITS_TRANSLATER;
+	int ret;
+
+	ret = __vgic_v5_signal_msi(vm, addr, device_id, event_id,
+				   KVM_MSI_VALID_DEVID);
+	TEST_ASSERT(ret == 1, "KVM_SIGNAL_MSI failed for GICv5 ITS");
+}
+
+static void vgic_v5_expect_its_lpi(struct vm_gic *v, struct kvm_vcpu *vcpu,
+				   u32 lpi)
+{
+	int ret;
+
+	its_expected_lpi = lpi;
+	sync_global_to_guest(v->vm, its_expected_lpi);
+
+	vgic_v5_signal_msi(v->vm, ITS_TEST_DEVICE_ID, ITS_TEST_EVENT_ID);
+
+	ret = run_vcpu(vcpu);
+	TEST_ASSERT(!ret, "Failed to run GICv5 ITS receiver");
+	vgic_v5_expect_sync(vcpu, GUEST_CMD_ITS_LPI_DELIVERED,
+			    "GICv5 ITS guest did not consume expected LPI");
+}
+
+static void test_vgic_v5_its_translation_cache(void)
+{
+	struct kvm_vcpu *vcpus[VGIC_V5_LPI_NR_VCPUS];
+	struct vm_gic v;
+	u64 wrong_addr, idr1;
+	int ret;
+
+	lpi_ist_ready = false;
+	its_expected_lpi = ITS_TEST_LPI_A;
+	vgic_v5_its_vm_create(&v, vcpus);
+	sync_global_to_guest(v.vm, lpi_ist_ready);
+	sync_global_to_guest(v.vm, its_expected_lpi);
+
+	ret = run_vcpu(vcpus[0]);
+	TEST_ASSERT(!ret, "Failed to run GICv5 ITS setup vCPU");
+	vgic_v5_expect_sync(vcpus[0], GUEST_CMD_ITS_READY,
+			    "GICv5 ITS setup guest did not become ready");
+
+	ret = run_vcpu(vcpus[1]);
+	TEST_ASSERT(!ret, "Failed to run GICv5 ITS receiver vCPU");
+	vgic_v5_expect_sync(vcpus[1], GUEST_CMD_ITS_READY,
+			    "GICv5 ITS receiver guest did not become ready");
+
+	ret = run_vcpu(vcpus[0]);
+	TEST_ASSERT(!ret, "Failed to run GICv5 ITS GEN_EVENTR sender");
+	vgic_v5_expect_sync(vcpus[0], GUEST_CMD_ITS_GEN_EVENT_SENT,
+			    "GICv5 ITS setup guest did not send GEN_EVENTR");
+
+	ret = run_vcpu(vcpus[1]);
+	TEST_ASSERT(!ret, "Failed to run GICv5 ITS GEN_EVENTR receiver");
+	vgic_v5_expect_sync(vcpus[1], GUEST_CMD_ITS_LPI_DELIVERED,
+			    "GICv5 ITS GEN_EVENTR did not deliver expected LPI");
+
+	ret = __vgic_v5_signal_msi(v.vm, GICV5_ITS_BASE_GPA +
+				   GICV5_ITS_TRANSLATER,
+				   ITS_TEST_DEVICE_ID, ITS_TEST_EVENT_ID, 0);
+	TEST_ASSERT(ret < 0 && errno == EINVAL,
+		    "GICv5 ITS accepted MSI without device ID flag");
+
+	wrong_addr = GICV5_ITS_BASE_GPA + GICV5_ITS_SIZE;
+	ret = __vgic_v5_signal_msi(v.vm, wrong_addr, ITS_TEST_DEVICE_ID,
+				   ITS_TEST_EVENT_ID, KVM_MSI_VALID_DEVID);
+	TEST_ASSERT(ret < 0 && errno == EINVAL,
+		    "GICv5 ITS accepted MSI for wrong address");
+
+	ret = __vgic_v5_signal_msi(v.vm, GICV5_ITS_BASE_GPA +
+				   GICV5_ITS_TRANSLATER,
+				   ITS_TEST_DEVICE_ID, BIT(16),
+				   KVM_MSI_VALID_DEVID);
+	TEST_ASSERT(ret < 0 && errno == EINVAL,
+		    "GICv5 ITS accepted MSI data wider than EventID");
+
+	ret = __vgic_v5_signal_msi(v.vm, GICV5_ITS_BASE_GPA +
+				   GICV5_ITS_TRANSLATER,
+				   ITS_TEST_DEVICE_ID, BIT(ITS_TEST_EVENT_ID_BITS),
+				   KVM_MSI_VALID_DEVID);
+	TEST_ASSERT(ret == 0,
+		    "GICv5 ITS did not block MSI with out-of-range EventID");
+
+	ret = __vgic_v5_signal_msi(v.vm, GICV5_ITS_BASE_GPA +
+				   GICV5_ITS_TRANSLATER,
+				   ITS_TEST_DEVICE_ID, ITS_TEST_EVENT_ID + 1,
+				   KVM_MSI_VALID_DEVID);
+	TEST_ASSERT(ret == 0,
+		    "GICv5 ITS did not block MSI with invalid ITTE");
+
+	/*
+	 * The earlier GEN_EVENTR has cached EventID -> LPI_A. At this stage,
+	 * VCPU0 rewrites the event, but doesn't invalidate it. The result is
+	 * that the MSI injection reuses the (stale) cache translation. This is
+	 * broken guest behaviour, as it should always issue an invalidation
+	 * when a translation changes.
+	 */
+	ret = run_vcpu(vcpus[0]);
+	TEST_ASSERT(!ret, "Failed to run GICv5 ITS ITT rewrite");
+	vgic_v5_expect_sync(vcpus[0], GUEST_CMD_ITS_ITT_REWRITTEN,
+			    "GICv5 ITS guest did not rewrite ITT");
+	vgic_v5_expect_its_lpi(&v, vcpus[1], ITS_TEST_LPI_A);
+
+	/*
+	 * Event invalidation must drop the stale EventID -> LPI_A entry. We now
+	 * expect LPI B.
+	 */
+	ret = run_vcpu(vcpus[0]);
+	TEST_ASSERT(!ret, "Failed to run GICv5 ITS event invalidation");
+	vgic_v5_expect_sync(vcpus[0], GUEST_CMD_ITS_EVENT_INVALIDATED,
+			    "GICv5 ITS guest did not invalidate event");
+	vgic_v5_expect_its_lpi(&v, vcpus[1], ITS_TEST_LPI_B);
+
+	/*
+	 * Device invalidation must cover all cached EventIDs for the
+	 * DeviceID. The guest has again re-written the event to translate to
+	 * LPI A.
+	 */
+	ret = run_vcpu(vcpus[0]);
+	TEST_ASSERT(!ret, "Failed to run GICv5 ITS device invalidation");
+	vgic_v5_expect_sync(vcpus[0], GUEST_CMD_ITS_DEVICE_INVALIDATED,
+			    "GICv5 ITS guest did not invalidate device");
+	vgic_v5_expect_its_lpi(&v, vcpus[1], ITS_TEST_LPI_A);
+
+	/* A narrower userspace IDR1 must invalidate the cached DeviceID. */
+	ret = __kvm_device_attr_get(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_IDR1, &idr1);
+	TEST_ASSERT(!ret, "GICv5 ITS_REGS failed to read IDR1");
+	idr1 = (idr1 & ~GICV5_ITS_IDR1_DEVICEID_BITS) |
+		FIELD_PREP(GICV5_ITS_IDR1_DEVICEID_BITS, 2);
+	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
+				    GICV5_ITS_IDR1, &idr1);
+	TEST_ASSERT(!ret, "GICv5 ITS_REGS failed to narrow DeviceID width");
+	ret = __vgic_v5_signal_msi(v.vm, GICV5_ITS_BASE_GPA +
+				   GICV5_ITS_TRANSLATER,
+				   ITS_TEST_DEVICE_ID, ITS_TEST_EVENT_ID,
+				   KVM_MSI_VALID_DEVID);
+	TEST_ASSERT(ret == 0,
+		    "GICv5 ITS did not block MSI after DeviceID width changed");
+
+	/* Disabling the ITS must also prevent delivery from cached entries. */
+	ret = run_vcpu(vcpus[0]);
+	TEST_ASSERT(!ret, "Failed to run GICv5 ITS disable");
+	vgic_v5_expect_sync(vcpus[0], GUEST_CMD_ITS_DISABLED,
+			    "GICv5 ITS guest did not disable ITS");
+
+	/*
+	 * We should fail to inject this MSI. Use the raw call to ensure we
+	 * catch the return value.
+	 */
+	ret = __vgic_v5_signal_msi(v.vm, GICV5_ITS_BASE_GPA +
+				   GICV5_ITS_TRANSLATER,
+				   ITS_TEST_DEVICE_ID, ITS_TEST_EVENT_ID,
+				   KVM_MSI_VALID_DEVID);
+	TEST_ASSERT(ret < 0 && errno == ENXIO,
+		    "GICv5 ITS accepted cached MSI while disabled");
+
+	vm_gic_destroy(&v);
+}
+
 /*
  * Returns 0 if it's possible to create GIC device of a given type (V5).
  */
@@ -2411,6 +2920,9 @@ void run_tests(u32 gic_dev_type)
 
 	pr_info("Test VGICv5 IST save/restore\n");
 	test_vgic_v5_ist_save_restore();
+
+	pr_info("Test VGICv5 ITS translation cache\n");
+	test_vgic_v5_its_translation_cache();
 }
 
 int main(int ac, char **av)
diff --git a/tools/testing/selftests/kvm/include/arm64/gic_v5.h b/tools/testing/selftests/kvm/include/arm64/gic_v5.h
index 2e4472489601c..895215a813b8d 100644
--- a/tools/testing/selftests/kvm/include/arm64/gic_v5.h
+++ b/tools/testing/selftests/kvm/include/arm64/gic_v5.h
@@ -137,14 +137,31 @@
 #define GICV5_ITS_INV_DEVICER_I		BIT(31)
 #define GICV5_ITS_INV_DEVICER_EVENTID_BITS GENMASK(5, 1)
 #define GICV5_ITS_INV_DEVICER_L1	BIT(0)
+#define GICV5_ITS_READ_EVENTR_R		BIT(31)
 #define GICV5_ITS_READ_EVENT_DATAR_VIRT	BIT_ULL(63)
 #define GICV5_ITS_READ_EVENT_DATAR_VM_ID GENMASK_ULL(47, 32)
 #define GICV5_ITS_READ_EVENT_DATAR_VALID BIT_ULL(31)
 #define GICV5_ITS_READ_EVENT_DATAR_LPI_ID GENMASK_ULL(23, 0)
+#define GICV5_ITS_GEN_EVENT_DIDR_DEVICEID GENMASK_ULL(31, 0)
+#define GICV5_ITS_GEN_EVENT_EIDR_EVENTID GENMASK(15, 0)
+#define GICV5_ITS_GEN_EVENTR_R		BIT(31)
+#define GICV5_ITS_GEN_EVENTR_TARGET_DOMAIN GENMASK(1, 0)
 #define GICV5_ITS_STATUSR_IDLE		BIT(0)
 #define GICV5_ITS_SYNC_STATUSR_IDLE	BIT(0)
 #define GICV5_ITS_GEN_EVENT_STATUSR_IDLE BIT(0)
 
+#define GICV5_DTL2E_VALID		BIT_ULL(0)
+#define GICV5_DTL2E_ITT_L2SZ		GENMASK_ULL(2, 1)
+#define GICV5_DTL2E_ITT_ADDR_MASK	GENMASK_ULL(55, 3)
+#define GICV5_DTL2E_ITT_DSWE		BIT_ULL(57)
+#define GICV5_DTL2E_ITT_STRUCTURE	BIT_ULL(58)
+#define GICV5_DTL2E_EVENT_ID_BITS	GENMASK_ULL(63, 59)
+
+#define GICV5_ITTL2E_LPI_ID		GENMASK_ULL(23, 0)
+#define GICV5_ITTL2E_DAC		GENMASK_ULL(29, 28)
+#define GICV5_ITTL2E_VIRTUAL		BIT_ULL(30)
+#define GICV5_ITTL2E_VALID		BIT_ULL(31)
+
 #define GICV5_IRS_IDR0_SETLPI		BIT(9)
 #define GICV5_IRS_IDR0_INT_DOM		GENMASK(1, 0)
 #define GICV5_IRS_IDR0_INT_DOM_SECURE		0b00
-- 
2.34.1

[PATCH v2 16/16] KVM: arm64: gic-v5: Document ITS userspace ABI

[Sascha Bischoff]

Document the GICv5 ITS device and its userspace ABI, including the
MMIO address, reset control, register attributes, and MSI requirements.
State that software should target ITS_TRANSLATER, while KVM uses the
MSI address to identify the virtual ITS.

Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
---
 .../virt/kvm/devices/arm-vgic-v5-its.rst      | 111 ++++++++++++++++++
 Documentation/virt/kvm/devices/index.rst      |   1 +
 2 files changed, 112 insertions(+)
 create mode 100644 Documentation/virt/kvm/devices/arm-vgic-v5-its.rst

diff --git a/Documentation/virt/kvm/devices/arm-vgic-v5-its.rst b/Documentation/virt/kvm/devices/arm-vgic-v5-its.rst
new file mode 100644
index 0000000000000..ae1a84deef98b
--- /dev/null
+++ b/Documentation/virt/kvm/devices/arm-vgic-v5-its.rst
@@ -0,0 +1,111 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+============================================================
+ARM Virtual Generic Interrupt Controller v5 ITS (VGICv5 ITS)
+============================================================
+
+Device types supported:
+  KVM_DEV_TYPE_ARM_VGIC_V5_ITS    ARM GICv5 Interrupt Translation Service
+
+The GICv5 ITS translates MSI(-X) interrupts into LPIs. It is optional and
+requires a GICv5 host and a KVM_DEV_TYPE_ARM_VGIC_V5 device for the VM. The
+ITS and VGIC devices may be created in either order, but their models must
+match. Userspace should assign each ITS a separate, non-overlapping MMIO
+region.
+
+MSIs injected through KVM_SIGNAL_MSI must provide a DeviceID using
+KVM_MSI_VALID_DEVID. The MSI address should target ITS_TRANSLATER in the
+ITS_TRANSLATE_FRAME. KVM uses the address to identify the virtual ITS, and
+the MSI data supplies the EventID. The virtual ITS supports 16-bit EventIDs;
+the upper 16 bits of the MSI data must be zero.
+
+
+Groups
+======
+
+KVM_DEV_ARM_VGIC_GRP_ADDR
+-------------------------
+
+  Attributes:
+    KVM_VGIC_V5_ADDR_TYPE_ITS (rw, 64-bit)
+      Base address of the ITS in the guest physical address space. The address
+      must be 64K aligned. The region covers 128K: a 64K control frame and a
+      64K translation frame. The address must be set before accessing ITS
+      registers through KVM_DEV_ARM_VGIC_GRP_ITS_REGS. It can only be set once.
+
+  Errors:
+
+    =======  =========================================================
+    -E2BIG   Address outside the guest's addressable IPA range
+    -EINVAL  Address is not 64K aligned, or the 128K range wraps
+    -EEXIST  Address has already been configured
+    -EFAULT  Invalid user pointer for kvm_device_attr.addr
+    -ENODEV  Unsupported address attribute for GET or SET
+    -ENXIO   Unsupported address attribute for HAS
+    -EBUSY   Concurrent address configuration has already set the base
+    -ENOMEM  Unable to allocate space for the MMIO bus entry
+    -ENOSPC  MMIO bus device limit reached
+    =======  =========================================================
+
+
+KVM_DEV_ARM_VGIC_GRP_CTRL
+-------------------------
+
+  Attributes:
+    KVM_DEV_ARM_VGIC_CTRL_INIT
+      Accepted for consistency with other VGIC devices; no additional ITS
+      initialization is required.
+
+    KVM_DEV_ARM_ITS_CTRL_RESET
+      Reset the emulated ITS registers and discard cached translations. The
+      configured MMIO base address is retained. Guest memory containing the
+      device and interrupt translation tables is not modified.
+
+  Errors:
+
+    =======  =========================================================
+    -EBUSY   A vCPU could not be locked for reset
+    -ENXIO   Unsupported control attribute for SET, or any GET
+    =======  =========================================================
+
+
+KVM_DEV_ARM_VGIC_GRP_ITS_REGS
+-----------------------------
+
+  Attributes:
+    The attr field of kvm_device_attr contains the ITS register offset from
+    the control frame base address. kvm_device_attr.addr points to a __u64,
+    regardless of whether the register is 32 or 64 bits wide. A 64-bit
+    register must be addressed at its base offset; userspace cannot access
+    its upper or lower half separately. Guest MMIO may use aligned 32-bit
+    accesses to DT_BASER, DIDR, READ_EVENT_DATAR, and GEN_EVENT_DIDR.
+
+    KVM_HAS_DEVICE_ATTR can probe supported offsets before the ITS base
+    address is configured. KVM_GET_DEVICE_ATTR and KVM_SET_DEVICE_ATTR require
+    the base address to be configured first.
+    Writes to read-only registers are ignored, except for IDR0-IDR2 and
+    READ_EVENT_DATAR, which can be restored through this interface. IDR0
+    requires the non-secure domain, a PA range supported by the host, and no
+    unsupported features. IDR1 cannot advertise more than 16 DeviceID bits.
+    IDR2 cannot advertise more than 16 EventID bits or cross-domain events.
+    Guest MMIO writes to these registers are ignored.
+
+    Writes to DT_CFGR and DT_BASER through this interface update their values
+    even when the ITS is enabled; guest MMIO writes to those registers are
+    ignored while it is enabled.
+
+    The registers with readable, writable state are CR0, CR1, DT_CFGR,
+    DT_BASER, DIDR, EIDR, GEN_EVENT_DIDR, and GEN_EVENT_EIDR.
+
+    Some registers initiate guest-visible operations when written, including
+    INV_EVENTR, INV_DEVICER, READ_EVENTR, and GEN_EVENTR.
+
+  Errors:
+
+    =======  =========================================================
+    -ENXIO   ITS base address is unset for GET or SET, or offset is unsupported
+    -EINVAL  Offset is not aligned to the register width, or an ID register
+             contains an unsupported value
+    -EFAULT  Invalid user pointer for kvm_device_attr.addr
+    -EBUSY   A vCPU could not be locked
+    =======  =========================================================
diff --git a/Documentation/virt/kvm/devices/index.rst b/Documentation/virt/kvm/devices/index.rst
index 70845aba38f45..d6d64b7266e83 100644
--- a/Documentation/virt/kvm/devices/index.rst
+++ b/Documentation/virt/kvm/devices/index.rst
@@ -11,6 +11,7 @@ Devices
    arm-vgic
    arm-vgic-v3
    arm-vgic-v5
+   arm-vgic-v5-its
    mpic
    s390_flic
    vcpu
-- 
2.34.1

Re: [PATCH v2 06/16] KVM: arm64: gic-v5: Add ITS MMIO state emulation

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] KVM: arm64: gic-v5: GICV5_ITS_SYNCR registered as 32-bit instead of 64-bit
- [Medium] KVM: arm64: gic-v5: vgic_v5_its_reset() sets idr1.l2sz=0 for 2-level tables
--

commit 26a03544ad8ae134679e971108dc975221b2093a
Author: Sascha Bischoff <sascha.bischoff@arm.com>

KVM: arm64: gic-v5: Add ITS MMIO state emulation

This commit replaces the placeholder GICv5 ITS IODEV region with the
architected register layout and adds MMIO read/write handlers and reset
state for the ITS configuration and translation-control registers.

> diff --git a/arch/arm64/kvm/vgic/vgic-its-v5.c b/arch/arm64/kvm/vgic/vgic-its-v5.c
> index 5790b029f09b6..f04161c315b78 100644
> --- a/arch/arm64/kvm/vgic/vgic-its-v5.c
> +++ b/arch/arm64/kvm/vgic/vgic-its-v5.c

[ ... ]

> +static void vgic_v5_mmio_write_its_dt_itt(struct kvm *kvm, void *dev,
> +					  gpa_t addr, unsigned int len,
> +					  unsigned long val)
> +{
> +	struct vgic_v5_its *its = dev;
> +	size_t offset = addr & (SZ_64K - 1);
> +
> +	guard(mutex)(&its->its_lock);
> +
> +	switch (offset) {

[ ... ]

> +	case GICV5_ITS_INV_EVENTR:
> +		its->inv_eventr.l1 =
> +			FIELD_GET(GICV5_ITS_INV_EVENTR_L1, val);
> +		its->inv_eventr.itt_l2sz =
> +			FIELD_GET(GICV5_ITS_INV_EVENTR_ITT_L2SZ, val);
> +		its->inv_eventr.i = FIELD_GET(GICV5_ITS_INV_EVENTR_I, val);
> +		return;
> +	case GICV5_ITS_SYNCR:
> +	case GICV5_ITS_READ_EVENTR:
> +		return;

[ ... ]

> @@ -31,11 +298,153 @@ static void vgic_v5_mmio_write_its_wi(struct kvm *kvm, void *dev, gpa_t addr,
>  	}
>  
>  static const struct vgic_register_region vgic_v5_its_registers[] = {

[ ... ]

> +	REGISTER_ITS_V5_DESC(GICV5_ITS_STATUSR, vgic_v5_mmio_read_its_dt_itt,
> +			     vgic_v5_mmio_write_its_wi, 4, VGIC_ACCESS_32bit),
> +	REGISTER_ITS_V5_DESC(GICV5_ITS_SYNCR, vgic_v5_mmio_read_its_raz,
> +			     vgic_v5_mmio_write_its_dt_itt, 4,
> +			     VGIC_ACCESS_32bit),

[Severity: Medium]
Should GICV5_ITS_SYNCR be registered as an 8-byte register with
VGIC_ACCESS_64bit | VGIC_ACCESS_32bit (and handle GICV5_ITS_SYNCR + 4 in
vgic_v5_mmio_write_its_dt_itt())?

In include/linux/irqchip/arm-gic-v5.h, GICV5_ITS_SYNCR is a 64-bit register
with GICV5_ITS_SYNCR_SYNC at BIT_ULL(63) and GICV5_ITS_SYNCR_SYNCALL at
BIT_ULL(32):

#define GICV5_ITS_SYNCR_SYNC			BIT_ULL(63)
#define GICV5_ITS_SYNCR_SYNCALL			BIT_ULL(32)
#define GICV5_ITS_SYNCR_DEVICEID		GENMASK_ULL(31, 0)

In drivers/irqchip/irq-gic-v5-its.c, gicv5_its_syncr() performs a 64-bit MMIO
write to this register:

	syncr = FIELD_PREP(GICV5_ITS_SYNCR_SYNC, 1) |
		FIELD_PREP(GICV5_ITS_SYNCR_DEVICEID, its_dev->device_id);

	its_writeq_relaxed(its, syncr, GICV5_ITS_SYNCR);

With length 4 and VGIC_ACCESS_32bit in vgic_v5_its_registers[],
vgic_get_mmio_region() -> check_region() rejects 64-bit guest accesses to
GICV5_ITS_SYNCR, and vgic_find_mmio_region() fails to match 32-bit accesses to
the upper word at GICV5_ITS_SYNCR + 4. This also causes the
KVM_DEV_ARM_VGIC_GRP_ITS_REGS userspace attribute access added later in the
series to treat GICV5_ITS_SYNCR as a 32-bit register rather than a 64-bit
register.

[ ... ]

> +static void vgic_v5_its_reset(struct vgic_v5_its *its)
> +{
> +	u64 mmfr0;
> +
> +	its->enabled = false;
> +
> +	mmfr0 = read_sanitised_ftr_reg(SYS_ID_AA64MMFR0_EL1);
> +	its->idr0.pa_range = cpuid_feature_extract_unsigned_field(
> +		mmfr0, ID_AA64MMFR0_EL1_PARANGE_SHIFT);
> +	its->idr0.domain = GICV5_ITS_IDR0_DOMAIN_NON_SECURE;
> +	its->idr0.mec = 0;
> +	its->idr0.mpam = 0;
> +	its->idr0.swe = 0;
> +	its->idr0.its_id = 0;
> +
> +	its->idr1.dt_levels = 1;
> +	its->idr1.itt_levels = 1;
> +	its->idr1.l2sz = 0;

[Severity: Medium]
In vgic_v5_its_reset(), dt_levels and itt_levels are set to 1 to advertise
2-level Device Tables and Interrupt Translation Tables, while its->idr1.l2sz
is initialized to 0.

In include/linux/irqchip/arm-gic-v5.h, GICV5_ITS_IDR1_L2SZ (bits [10:8]) is a
3-bit support bitmap rather than the 2-bit size encoding used in
GICV5_ITS_DT_CFGR_L2SZ:

#define GICV5_ITS_IDR1_L2SZ		GENMASK(10, 8)
#define GICV5_ITS_IDR1_ITT_LEVELS	BIT(7)
#define GICV5_ITS_IDR1_DT_LEVELS	BIT(6)
#define GICV5_ITS_IDR1_DEVICEID_BITS	GENMASK(5, 0)

#define GICV5_ITS_IDR1_L2SZ_SUPPORT_4KB(r)	FIELD_GET(BIT(8), (r))
#define GICV5_ITS_IDR1_L2SZ_SUPPORT_16KB(r)	FIELD_GET(BIT(9), (r))
#define GICV5_ITS_IDR1_L2SZ_SUPPORT_64KB(r)	FIELD_GET(BIT(10), (r))

When a guest reads GICV5_ITS_IDR1 through vgic_v5_mmio_read_its_misc(),
FIELD_PREP(GICV5_ITS_IDR1_L2SZ, its->idr1.l2sz) reports 0b000, indicating that
none of the 4KB, 16KB, or 64KB L2 granule sizes are supported. In
drivers/irqchip/irq-gic-v5-its.c, gicv5_its_l2sz_two_level() checks those
support bits and fails all of them (falling back to 4KB even on 16KB or 64KB
page size guests, despite parse_two_level_dt() and parse_two_level_itt()
supporting all three sizes).

Should its->idr1.l2sz be initialized with the supported L2 table size bits set
instead of 0?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261102194454.4086911-1-sascha.bischoff@arm.com?part=6

Re: [PATCH v2 05/16] KVM: arm64: gic-v5: Add ITS device skeleton

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [Medium] KVM: arm64: gic-v5: unrestored base address in vgic_v5_register_its_iodev()
- [Low] KVM: arm64: gic-v5: missing tools UAPI header sync for GICv5 ITS definitions
- [Low] KVM: arm64: gic-v5: dead code in vgic_v5_register_its_iodev()
- [Low] KVM: arm64: gic-v5: stale GICv3 ITS description in commit message
--

commit bdded985e667a6d1396b4ac9faceeadebc56557f
Author: Sascha Bischoff <sascha.bischoff@arm.com>

KVM: arm64: gic-v5: Add ITS device skeleton

This commit introduces the initial KVM device plumbing for a GICv5 ITS,
including the UAPI device type and address attribute, struct vgic_v5_its
state, and a RAZ/WI MMIO region placeholder registered when userspace sets
the ITS base address.

>     KVM: arm64: gic-v5: Add ITS device skeleton
>     
>     Introduce the basic KVM device plumbing for a GICv5 ITS. This adds the
>     GICv5 ITS device type and address attribute, registers the device ops,
>     and wires a GICv5 ITS IODEV into the generic VGIC MMIO dispatch path.
>     
>     Add struct vgic_v5_its to hold the ITS base address, the IODEV plumbing,
>     the backing KVM device pointer, and the general ITS mutex. Allocate this
>     state from the KVM device create path, initialise the mutex and base
>     address, and mark the VM as having an ITS. GICv5 MSIs require a DeviceID,
>     so set msis_require_devid when the ITS is created.
>     
>     Record the VGIC model required by an ITS and reject mismatched device
>     creation in either order. Check GICv3 ITS creation as well, so the two
>     ITS models cannot be combined in one VM.
>     
>     The MMIO region is registered once userspace provides the ITS base
>     address through KVM_VGIC_V5_ADDR_TYPE_ITS. For now the IODEV only
>     provides a RAZ/WI placeholder region; the actual ITS register emulation
>     is added separately.
>     
>     Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>

[Severity: Low]
This isn't a bug, but is the third paragraph of the commit message stale?
struct vgic_dist::its_model, the check in kvm_vgic_create(), and the GICv3
ITS creation check in vgic_its_create() were already introduced in commit
05620a314219 ("KVM: arm64: vgic: Require matching VGIC and ITS models"),
whereas this commit only checks and sets its_model in vgic_v5_its_create().

> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
> index 65ee09bd14636..8c701173a2d88 100644
> --- a/arch/arm64/include/uapi/asm/kvm.h
> +++ b/arch/arm64/include/uapi/asm/kvm.h
> @@ -99,8 +99,10 @@ struct kvm_regs {
>  
>  /* Supported VGICv5 address types  */
>  #define KVM_VGIC_V5_ADDR_TYPE_IRS	6
> +#define KVM_VGIC_V5_ADDR_TYPE_ITS	7
>  
>  #define KVM_VGIC_V5_IRS_SIZE		(2 * SZ_64K)
> +#define KVM_VGIC_V5_ITS_SIZE		(2 * SZ_64K)
>  
>  #define KVM_ARM_VCPU_POWER_OFF		0 /* CPU is started in OFF state */
>  #define KVM_ARM_VCPU_EL1_32BIT		1 /* CPU running a 32bit VM */

[Severity: Low]
Should tools/arch/arm64/include/uapi/asm/kvm.h also be updated with
KVM_VGIC_V5_ADDR_TYPE_ITS and KVM_VGIC_V5_ITS_SIZE so that
tools/perf/check-headers.sh does not report an out-of-sync header warning
during tools builds?

[ ... ]

> diff --git a/arch/arm64/kvm/vgic/vgic-its-v5.c b/arch/arm64/kvm/vgic/vgic-its-v5.c
> new file mode 100644
> index 0000000000000..5790b029f09b6
> --- /dev/null
> +++ b/arch/arm64/kvm/vgic/vgic-its-v5.c
> @@ -0,0 +1,186 @@

[ ... ]

> +static int vgic_v5_register_its_iodev(struct kvm *kvm, struct vgic_v5_its *its,
> +				      u64 addr)
> +{
> +	struct vgic_io_device *iodev = &its->iodev;
> +	int ret;
> +
> +	mutex_lock(&kvm->slots_lock);
> +	if (!IS_VGIC_ADDR_UNDEF(its->vgic_v5_its_base)) {
> +		ret = -EBUSY;
> +		goto out;
> +	}
> +
> +	if (!IS_ALIGNED(addr, SZ_64K)) {
> +		kvm_err("ITS Base address is not aligned to 64k\n");
> +		ret = -EINVAL;
> +		goto out;
> +	}
> +
> +	its->vgic_v5_its_base = addr;
> +	iodev->base_addr = addr;
> +	iodev->regions = vgic_v5_its_registers;
> +	iodev->nr_regions = ARRAY_SIZE(vgic_v5_its_registers);
> +	iodev->iodev_type = IODEV_GICV5_ITS;
> +	iodev->redist_vcpu = NULL;
> +	iodev->v5its = its;

[Severity: Low]
This isn't a bug, but since vgic_v5_its_set_attr() is the only caller of
vgic_v5_register_its_iodev() and already checks SZ_64K alignment in
vgic_check_iorange(), is the !IS_ALIGNED(addr, SZ_64K) check here needed?

Also, redist_vcpu and v5its are members of the same anonymous union in
struct vgic_io_device, so setting iodev->redist_vcpu = NULL is immediately
overwritten by iodev->v5its = its.

> +	kvm_iodevice_init(&iodev->dev, &kvm_io_gic_ops);
> +
> +	ret = kvm_io_bus_register_dev(kvm, KVM_MMIO_BUS, iodev->base_addr,
> +				      KVM_VGIC_V5_ITS_SIZE, &iodev->dev);
> +out:
> +	mutex_unlock(&kvm->slots_lock);
> +
> +	return ret;
> +}

[Severity: Medium]
If kvm_io_bus_register_dev() fails in vgic_v5_register_its_iodev() (for
example with -ENOMEM or -ENOSPC), should its->vgic_v5_its_base be reset to
VGIC_ADDR_UNDEF before returning?

Leaving its->vgic_v5_its_base set to addr on failure causes
vgic_v5_its_get_attr() to report the address as configured even though the
MMIO device was never registered on KVM_MMIO_BUS, and causes any subsequent
retry of KVM_SET_DEVICE_ATTR via vgic_v5_its_set_attr() to fail with
-EEXIST in vgic_check_iorange().

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261102194454.4086911-1-sascha.bischoff@arm.com?part=5

Re: [PATCH v2 07/16] KVM: arm64: gic-v5: Add ITS guest table translation emulation

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Critical] KVM: arm64: gic-v5: missing vgic init check in __vgic_v5_its_resolve_lpi()
- [Low] KVM: arm64: gic-v5: unclamped event_id_bits in __vgic_v5_its_resolve_lpi()
- [Low] KVM: arm64: gic-v5: duplicated L2SZ decoding in parse_two_level_itt()
--

commit 48d54735882d147618fe3ac1bc65671388c7716f
Author: Sascha Bischoff <sascha.bischoff@arm.com>

KVM: arm64: gic-v5: Add ITS guest table translation emulation

This commit adds helpers to walk linear and two-level guest Device Tables
and Interrupt Translation Tables for GICv5 ITS emulation, resolving
DeviceID/EventID pairs to LPIs and wiring up the READ_EVENTR and
GEN_EVENTR MMIO handlers.

> diff --git a/arch/arm64/kvm/vgic/vgic-its-v5.c b/arch/arm64/kvm/vgic/vgic-its-v5.c
> index f04161c315b78..caa68c4680d02 100644
> --- a/arch/arm64/kvm/vgic/vgic-its-v5.c
> +++ b/arch/arm64/kvm/vgic/vgic-its-v5.c
> @@ -19,6 +19,302 @@
>  #define VITS_V5_DEVICE_ID_BITS		16
>  #define VITS_V5_EVENT_ID_BITS		16
>  
> +static int parse_linear_dt(struct kvm *kvm, gpa_t dt_base, u32 device_id,
> +			   gpa_t *itt_base, bool *two_level_itt,
> +			   unsigned int *event_id_bits, unsigned int *itt_l2sz)
> +{
> +	__le64 dt_l2_entry;
> +	gpa_t entry_addr;
> +	u64 val;
> +	int ret;
> +
> +	entry_addr = dt_base + device_id * sizeof(__le64);
> +	ret = kvm_read_guest_lock(kvm, entry_addr, &dt_l2_entry,
> +				  sizeof(dt_l2_entry));
> +	if (ret) {
> +		kvm_debug("Failed to read guest memory for L2 DT\n");
> +		return ret;
> +	}
> +
> +	val = le64_to_cpu(dt_l2_entry);
> +	if (!FIELD_GET(GICV5_DTL2E_VALID, val)) {
> +		kvm_debug("Guest DT entry is invalid\n");
> +		return -EINVAL;
> +	}
> +
> +	*itt_base = FIELD_GET(GICV5_DTL2E_ITT_ADDR_MASK, val) << 3;
> +	*two_level_itt = FIELD_GET(GICV5_DTL2E_ITT_STRUCTURE, val);
> +	*event_id_bits = FIELD_GET(GICV5_DTL2E_EVENT_ID_BITS, val);
> +	*itt_l2sz = FIELD_GET(GICV5_DTL2E_ITT_L2SZ, val);
> +
> +	return 0;
> +}
> +
> +static int parse_two_level_dt(struct kvm *kvm, gpa_t dt_base,
> +			      unsigned int dt_l2sz, u32 device_id,
> +			      gpa_t *itt_base, bool *two_level_itt,
> +			      unsigned int *event_id_bits,
> +			      unsigned int *itt_l2sz)
> +{
> +	unsigned int device_id_l1_shift, l2_device_id;
> +	__le64 dt_l1_entry;
> +	gpa_t dt_l2_base;
> +	gpa_t entry_addr;
> +	u64 val;
> +	int ret;
> +
> +	switch (dt_l2sz) {
> +	case 0:
> +		device_id_l1_shift = 9;
> +		break;
> +	case 1:
> +		device_id_l1_shift = 11;
> +		break;
> +	case 2:
> +		device_id_l1_shift = 13;
> +		break;
> +	default:
> +		return -EINVAL;
> +	}
> +
> +	entry_addr = dt_base +
> +		     (device_id >> device_id_l1_shift) * sizeof(__le64);
> +
> +	ret = kvm_read_guest_lock(kvm, entry_addr, &dt_l1_entry,
> +				  sizeof(dt_l1_entry));
> +	if (ret) {
> +		kvm_debug("Failed to read guest memory for L1 DT\n");
> +		return ret;
> +	}
> +
> +	val = le64_to_cpu(dt_l1_entry);
> +	if (!FIELD_GET(GICV5_DTL1E_VALID, val)) {
> +		kvm_debug("Guest DT L1 entry is invalid\n");
> +		return -EINVAL;
> +	}
> +
> +	dt_l2_base = FIELD_GET(GICV5_DTL1E_L2_ADDR_MASK, val) << 3;
> +	l2_device_id = device_id % (1UL << device_id_l1_shift);
> +	if (l2_device_id >= (1UL << FIELD_GET(GICV5_DTL1E_SPAN, val))) {
> +		kvm_debug("DeviceID 0x%x out of range of guest DT\n",
> +			  device_id);
> +		return -ENXIO;
> +	}
> +
> +	return parse_linear_dt(kvm, dt_l2_base, l2_device_id, itt_base,
> +			       two_level_itt, event_id_bits, itt_l2sz);
> +}

[ ... ]

> +static int parse_two_level_itt(struct kvm *kvm, u32 event_id, gpa_t itt_base,
> +			       unsigned int itt_l2sz, __le64 *itt_l2_entry)
> +{
> +	unsigned int itt_split_bits, l2_event_id_bits;
> +	gpa_t itt_l2_base, entry_addr;
> +	u32 l1_event_id, l2_event_id;
> +	__le64 itt_l1_entry;
> +	u64 val;
> +	int ret;
> +
> +	switch (itt_l2sz) {
> +	case 0:
> +		itt_split_bits = 9;
> +		break;
> +	case 1:
> +		itt_split_bits = 11;
> +		break;
> +	case 2:
> +		itt_split_bits = 13;
> +		break;
> +	default:
> +		return -EINVAL;
> +	}
> +
> +	l1_event_id = event_id >> itt_split_bits;
> +	entry_addr = itt_base + l1_event_id * sizeof(itt_l1_entry);
> +	ret = kvm_read_guest_lock(kvm, entry_addr, &itt_l1_entry,
> +				  sizeof(itt_l1_entry));
> +	if (ret) {
> +		kvm_debug("Failed to read guest memory for L1 ITT\n");
> +		return ret;
> +	}
> +
> +	val = le64_to_cpu(itt_l1_entry);
> +	if (!FIELD_GET(GICV5_ITTL1E_VALID, val)) {
> +		kvm_debug("Guest ITT L1 entry is invalid\n");
> +		return -EINVAL;
> +	}
> +
> +	itt_l2_base = FIELD_GET(GICV5_ITTL1E_L2_ADDR_MASK, val) << 3;
> +	l2_event_id_bits =
> +		min(FIELD_GET(GICV5_ITTL1E_SPAN, val), itt_split_bits);
> +	l2_event_id = event_id % (1UL << l2_event_id_bits);
> +
> +	if ((itt_split_bits != l2_event_id_bits) &&
> +	    (event_id & ((1UL << itt_split_bits) - 1)) != l2_event_id) {
> +		kvm_debug("EventID is out of range of L2 ITT\n");
> +		return -ENXIO;
> +	}
> +
> +	return parse_linear_itt(kvm, l2_event_id, itt_l2_base, itt_l2_entry);
> +}

[Severity: Low]
This isn't a bug, but both parse_two_level_dt() and parse_two_level_itt()
duplicate the same switch statement mapping L2SZ encodings (0, 1, 2) to bit
shifts (9, 11, 13).

For validating the L2 index against the L1 entry's SPAN field,
parse_two_level_dt() computes l2_device_id = device_id % (1UL <<
device_id_l1_shift) and directly checks l2_device_id >= (1UL <<
FIELD_GET(GICV5_DTL1E_SPAN, val)), whereas parse_two_level_itt() clamps
l2_event_id_bits with min() and compares (event_id & ((1UL << itt_split_bits)
- 1)) against l2_event_id.

Could the L2SZ-to-shift conversion be factored into a small helper, and could
parse_two_level_itt() use the same direct SPAN bounds check as
parse_two_level_dt()?

[ ... ]

> +static int __vgic_v5_its_resolve_lpi(struct kvm *kvm, struct vgic_v5_its *its,
> +				     u32 device_id, u16 event_id,
> +				     struct vgic_irq **irq)
> +{
> +	unsigned int itt_l2sz, event_id_bits;
> +	__le64 itt_l2_entry;
> +	bool two_level_itt;
> +	gpa_t itt_base;
> +	u32 lpi;
> +	u64 val;
> +	int ret;
> +
> +	lockdep_assert_held(&its->its_lock);
> +
> +	if (!its->enabled) {
> +		kvm_debug("Cannot look up LPI; vITS is disabled\n");
> +		return -ENXIO;
> +	}
> +
> +	if (WARN_ON_ONCE(kvm->mm != get_current()->mm))
> +		return -EWOULDBLOCK;
> +
> +	ret = get_itt(kvm, its, device_id, &itt_base, &two_level_itt,
> +		      &event_id_bits, &itt_l2sz);
> +	if (ret) {
> +		kvm_debug("Error parsing guest's DT\n");
> +		return ret;
> +	}
> +
> +	if (event_id >= 1UL << event_id_bits) {
> +		kvm_debug("Event ID out of range\n");
> +		return -E2BIG;
> +	}

[Severity: Low]
In get_itt(), the guest-configured DeviceID width is clamped to the maximum
supported by the ITS in IDR1 via min(its->dt_cfgr.device_id_bits,
its->idr1.device_id_bits).

In parse_linear_dt(), *event_id_bits = FIELD_GET(GICV5_DTL2E_EVENT_ID_BITS,
val) extracts the 5-bit EVENT_ID_BITS field (0..31) from the guest DTL2E, and
__vgic_v5_its_resolve_lpi() checks event_id >= 1UL << event_id_bits without
clamping event_id_bits to or validating it against its->idr2.event_id_bits.

This rests on a GICv5 architectural detail that could not be confirmed from
the kernel source: when GICV5_DTL2E_EVENT_ID_BITS exceeds
GICV5_ITS_IDR2_EVENTID_BITS, does the architecture require clamping the
effective EventID width to ITS_IDR2.EVENTID_BITS or treating the DTL2E as
invalid? If yes, parse_linear_dt() and __vgic_v5_its_resolve_lpi() will accept
an oversized DTL2E.EVENT_ID_BITS without capping or rejecting it when
its->idr2.event_id_bits is smaller than DTL2E.EVENT_ID_BITS.

> +
> +	ret = get_itte(kvm, event_id, itt_base, two_level_itt, itt_l2sz,
> +		       &itt_l2_entry);
> +	if (ret)
> +		return ret;
> +
> +	val = le64_to_cpu(itt_l2_entry);
> +	if (!FIELD_GET(GICV5_ITTL2E_VALID, val)) {
> +		kvm_debug("ITTE is not valid\n");
> +		return -EINVAL;
> +	}
> +
> +	lpi = FIELD_GET(GICV5_ITTL2E_LPI_ID, val);
> +
> +	*irq = vgic_add_lpi(kvm, vgic_v5_make_lpi(lpi), NULL);
> +	if (IS_ERR(*irq))
> +		return PTR_ERR(*irq);
> +
> +	return 0;
> +}
> +
> +static void vgic_v5_its_inject_lpi(struct kvm *kvm, struct vgic_irq *irq)
> +{
> +	kvm_call_hyp(__vgic_v5_vdpend, irq->intid, true,
> +		     kvm->arch.vgic.gicv5_vm.vm_id);
> +}

[Severity: Critical]
What happens if userspace creates a KVM_DEV_TYPE_ARM_VGIC_V5_ITS device, sets
KVM_VGIC_V5_ADDR_TYPE_ITS, and runs a vCPU without ever creating a
KVM_DEV_TYPE_ARM_VGIC_V5 irqchip device?

In vgic_v5_its_create(), creating the ITS device is permitted when
irqchip_in_kernel(dev->kvm) is false:

arch/arm64/kvm/vgic/vgic-its-v5.c:vgic_v5_its_create() {
	...
	if ((irqchip_in_kernel(dev->kvm) && !vgic_is_v5(dev->kvm)) ||
	    (dev->kvm->arch.vgic.its_model &&
	     dev->kvm->arch.vgic.its_model != KVM_DEV_TYPE_ARM_VGIC_V5)) {
		kfree(its);
		return -ENODEV;
	}
	...
}

Setting KVM_VGIC_V5_ADDR_TYPE_ITS via vgic_v5_its_set_attr() immediately
registers the ITS MMIO device on KVM_MMIO_BUS via
vgic_v5_register_its_iodev(). When userspace runs a vCPU via KVM_RUN,
kvm_arch_vcpu_run_pid_change() only calls kvm_vgic_map_resources(kvm) when
irqchip_in_kernel(kvm) is true:

arch/arm64/kvm/arm.c:kvm_arch_vcpu_run_pid_change() {
	...
	if (likely(irqchip_in_kernel(kvm))) {
		/*
		 * Map the VGIC hardware resources before running a vcpu the
		 * first time on this VM.
		 */
		ret = kvm_vgic_map_resources(kvm);
		if (ret)
			return ret;
	}
	...
}

Because kvm_vgic_create() was never called to initialize
kvm->arch.vgic.gicv5_vm.vm_id to VGIC_V5_VM_ID_INVAL (and
vgic_v5_allocate_vm_id() was never called), kvm->arch.vgic.gicv5_vm.vm_id
remains 0 from the zero-initialized struct kvm allocation.

When the guest enables the vITS via GICV5_ITS_CR0 and writes
GICV5_ITS_GEN_EVENTR_R to GICV5_ITS_GEN_EVENTR,
vgic_v5_its_handle_gen_event() calls __vgic_v5_its_resolve_lpi() and
vgic_v5_its_inject_lpi(). Neither function checks vgic_initialized(kvm) or
vgic_has_its(kvm), so __vgic_v5_its_resolve_lpi() allocates LPIs in lpi_xa and
vgic_v5_its_inject_lpi() executes kvm_call_hyp(__vgic_v5_vdpend, irq->intid,
true, kvm->arch.vgic.gicv5_vm.vm_id) with vm_id == 0.

Since vgic_v5_allocate_vm_id() allocates valid hardware VM IDs from
vmt_info->vm_id_ida starting at 0, won't the EL2 VDPEND instruction inject
guest-controlled LPIs directly into whichever active GICv5 VM on the host owns
hardware VM ID 0? Should __vgic_v5_its_resolve_lpi() (or vgic_v5_its_create())
verify that the in-kernel GICv5 irqchip is initialized before resolving and
injecting LPIs?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261102194454.4086911-1-sascha.bischoff@arm.com?part=7

Re: [PATCH v2 09/16] KVM: arm64: gic-v5: Add ITS MSI injection

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] KVM: arm64: gic-v5: missing vgic_initialized() check in MSI injection
--

commit b2f7c0263871635da8ddb0614e8b96f1cb1f641c
Author: Sascha Bischoff <sascha.bischoff@arm.com>

KVM: arm64: gic-v5: Add ITS MSI injection

This commit adds MSI injection support for the virtual GICv5 ITS via both
the slow guest-memory translation path and the inatomic cached translation
path. It wires kvm_set_msi() and kvm_arch_set_irq_inatomic() to invoke the
new GICv5 ITS injection helpers for GICv5 VMs.

> diff --git a/arch/arm64/kvm/vgic/vgic-its-v5.c b/arch/arm64/kvm/vgic/vgic-its-v5.c
> index 9f4783b243221..b69a1d107c0ae 100644
> --- a/arch/arm64/kvm/vgic/vgic-its-v5.c
> +++ b/arch/arm64/kvm/vgic/vgic-its-v5.c

[ ... ]

> @@ -424,12 +468,86 @@ static int __vgic_v5_its_resolve_lpi(struct kvm *kvm, struct vgic_v5_its *its,
>  	return ret;
>  }
>  
> +int vgic_v5_its_resolve_lpi(struct kvm *kvm, struct vgic_v5_its *its,
> +			    u32 device_id, u32 event_id,
> +			    struct vgic_irq **irq)
> +{
> +	guard(mutex)(&its->its_lock);
> +
> +	return __vgic_v5_its_resolve_lpi(kvm, its, device_id, event_id, irq);
> +}
> +
>  static void vgic_v5_its_inject_lpi(struct kvm *kvm, struct vgic_irq *irq)
>  {
>  	kvm_call_hyp(__vgic_v5_vdpend, irq->intid, true,
>  		     kvm->arch.vgic.gicv5_vm.vm_id);
>  }
>  
> +int vgic_v5_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi)
> +{
> +	struct vgic_v5_its *its;
> +	struct vgic_irq *irq;
> +	bool borrowed_mm = false;
> +	int ret;
> +
> +	its = vgic_v5_msi_to_its(kvm, msi);
> +	if (IS_ERR(its))
> +		return PTR_ERR(its);
> +
> +	/* An irqfd cache miss is resolved by a worker outside the VM's mm. */
> +	if (current->mm != kvm->mm) {
> +		if (WARN_ON_ONCE(current->mm || !(current->flags & PF_KTHREAD)))
> +			return -EWOULDBLOCK;
> +
> +		if (!mmget_not_zero(kvm->mm))
> +			return -EFAULT;
> +
> +		kthread_use_mm(kvm->mm);
> +		borrowed_mm = true;
> +	}
> +
> +	ret = vgic_v5_its_resolve_lpi(kvm, its, msi->devid, msi->data, &irq);
> +	if (borrowed_mm) {
> +		kthread_unuse_mm(kvm->mm);
> +		mmput(kvm->mm);
> +	}
> +
> +	if (ret < 0)
> +		return ret;
> +	/* KVM_SIGNAL_MSI reports guest translation failures as blocked MSIs. */
> +	if (ret)
> +		return 0;
> +
> +	vgic_v5_its_inject_lpi(kvm, irq);
> +	vgic_put_irq(kvm, irq);
> +
> +	return 1;
> +}
> +
> +int vgic_v5_its_inject_cached_translation(struct kvm *kvm, struct kvm_msi *msi)
> +{
> +	struct vgic_v5_its *its;
> +	struct vgic_irq *irq;
> +	int ret;
> +
> +	its = vgic_v5_msi_to_its(kvm, msi);
> +	if (IS_ERR(its))
> +		return PTR_ERR(its);
> +
> +	if (!READ_ONCE(its->enabled))
> +		return -EWOULDBLOCK;
> +
> +	ret = its_cache_look_up_translation(its, msi->devid, (u16)msi->data,
> +					    &irq);
> +	if (ret)
> +		return -EWOULDBLOCK;
> +
> +	vgic_v5_its_inject_lpi(kvm, irq);
> +	vgic_put_irq(kvm, irq);
> +
> +	return 0;
> +}

[Severity: High]
Can userspace trigger vgic_v5_its_inject_lpi() before the GICv5 VM has been
initialized?

When a VM creates KVM_DEV_TYPE_ARM_VGIC_V5 and KVM_DEV_TYPE_ARM_VGIC_V5_ITS,
kvm_vgic_create() initializes kvm->arch.vgic.gicv5_vm.vm_id to
VGIC_V5_VM_ID_INVAL (-1), and a valid VM ID is only allocated later during
KVM_DEV_ARM_VGIC_CTRL_INIT via:

  vgic_init()
    vgic_v5_init()
      vgic_v5_allocate_vm_id()

However, vgic_v5_register_its_iodev() registers the ITS on KVM_MMIO_BUS as
soon as KVM_VGIC_V5_ADDR_TYPE_ITS is set, and userspace register access
allows writing GICV5_ITS_CR0 (setting its->enabled = true),
GICV5_ITS_DT_CFGR, and GICV5_ITS_DT_BASER before vgic_init() is called.

If userspace invokes KVM_SIGNAL_MSI or triggers an irqfd before
KVM_DEV_ARM_VGIC_CTRL_INIT, neither __vgic_v5_its_resolve_lpi(),
vgic_v5_its_inject_msi(), nor vgic_v5_its_inject_cached_translation() checks
vgic_initialized(kvm). As a result, vgic_v5_its_inject_lpi() will pass
VGIC_V5_VM_ID_INVAL truncated to u16 (0xffff) to __vgic_v5_vdpend(), which
executes the hardware VDPEND instruction with an unallocated VM ID (or
targets another active VM using VM ID 0xffff).

Should the MSI resolution and injection paths check vgic_initialized(kvm)
before resolving translations or calling vgic_v5_its_inject_lpi()?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261102194454.4086911-1-sascha.bischoff@arm.com?part=9

Re: [PATCH v2 10/16] KVM: arm64: gic-v5: Add ITS userspace register access

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [High] KVM: arm64: gic-v5: missing vgic_initialized() check before LPI injection
- [Medium] KVM: arm64: gic-v5: unenforced IDR1/IDR2 limits and stale cache on write
- [Medium] KVM: arm64: gic-v5: GICV5_ITS_SYNCR registered as 32-bit instead of 64-bit
- [Low] KVM: arm64: gic-v5: 64-bit attr->attr offset truncated to 32-bit in ITS regs
--

commit 3a58686064e4b9037bb8becf8b235cece1c6b8e0
Author: Sascha Bischoff <sascha.bischoff@arm.com>

KVM: arm64: gic-v5: Add ITS userspace register access

This commit exposes the emulated GICv5 ITS MMIO register frame to userspace
via KVM_DEV_ARM_VGIC_GRP_ITS_REGS and adds support for resetting the ITS via
KVM_DEV_ARM_ITS_CTRL_RESET. It also adds dedicated userspace write handlers
for restoring IDR0-IDR2, DT_BASER, DT_CFGR, and READ_EVENT_DATAR.

> diff --git a/arch/arm64/kvm/vgic/vgic-its-v5.c b/arch/arm64/kvm/vgic/vgic-its-v5.c
> index b69a1d107c0ae..638ee47f6e27d 100644
> --- a/arch/arm64/kvm/vgic/vgic-its-v5.c
> +++ b/arch/arm64/kvm/vgic/vgic-its-v5.c
> @@ -688,6 +688,70 @@ static void vgic_v5_mmio_write_its_misc(struct kvm *kvm, void *dev, gpa_t addr,
>  	}
>  }
>  
> +static int vgic_v5_uaccess_write_its_idr(struct kvm *kvm, void *dev,
> +					gpa_t addr, unsigned int len,
> +					unsigned long val)
> +{
> +	struct vgic_v5_its *its = dev;
> +	size_t offset = addr & (SZ_64K - 1);
> +	u64 mmfr0;
> +
> +	guard(mutex)(&its->its_lock);
> +
> +	switch (offset) {
[ ... ]
> +	case GICV5_ITS_IDR1:
> +		if ((val & ~(GICV5_ITS_IDR1_L2SZ |
> +			     GICV5_ITS_IDR1_ITT_LEVELS |
> +			     GICV5_ITS_IDR1_DT_LEVELS |
> +			     GICV5_ITS_IDR1_DEVICEID_BITS)) ||
> +		    FIELD_GET(GICV5_ITS_IDR1_DEVICEID_BITS, val) >
> +		    VITS_V5_DEVICE_ID_BITS)
> +			return -EINVAL;
> +
> +		/*
> +		 * Userspace can change the DeviceID limit after translations have
> +		 * been cached. Drop them so the old limit cannot remain in effect.
> +		 */
> +		if (its->idr1.device_id_bits !=
> +		    FIELD_GET(GICV5_ITS_IDR1_DEVICEID_BITS, val))
> +			its_cache_clear_translations(its);
> +
> +		its->idr1.device_id_bits = FIELD_GET(GICV5_ITS_IDR1_DEVICEID_BITS,
> +							 val);
> +		its->idr1.dt_levels = !!(val & GICV5_ITS_IDR1_DT_LEVELS);
> +		its->idr1.itt_levels = !!(val & GICV5_ITS_IDR1_ITT_LEVELS);
> +		its->idr1.l2sz = FIELD_GET(GICV5_ITS_IDR1_L2SZ, val);
> +		return 0;
> +	case GICV5_ITS_IDR2:
> +		if ((val & ~GICV5_ITS_IDR2_EVENTID_BITS) ||
> +		    FIELD_GET(GICV5_ITS_IDR2_EVENTID_BITS, val) >
> +		    VITS_V5_EVENT_ID_BITS)
> +			return -EINVAL;
> +
> +		its->idr2.event_id_bits = FIELD_GET(GICV5_ITS_IDR2_EVENTID_BITS,
> +							 val);
> +		return 0;

[Severity: Medium]
When userspace writes GICV5_ITS_IDR1 or GICV5_ITS_IDR2 in
vgic_v5_uaccess_write_its_idr(), changes to its->idr1.device_id_bits clear
the translation cache and are enforced in get_itt() via:

	device_id_bits = min(its->dt_cfgr.device_id_bits,
			     its->idr1.device_id_bits);

However, for GICV5_ITS_IDR2, updating its->idr2.event_id_bits does not call
its_cache_clear_translations(its), and __vgic_v5_its_resolve_lpi() checks
event_id against the unclamped event_id_bits read from the guest DTE without
clamping to its->idr2.event_id_bits:

	ret = get_itt(kvm, its, device_id, &itt_base, &two_level_itt,
		      &event_id_bits, &itt_l2sz);
	...
	if (event_id >= 1UL << event_id_bits) {
		kvm_debug("Event ID out of range\n");
		return VITS_V5_UNMAPPED;
	}

Similarly, in GICV5_ITS_IDR1, updates to its->idr1.dt_levels,
its->idr1.itt_levels, and its->idr1.l2sz do not invalidate cached
translations, and get_itt(), parse_two_level_dt(), get_itte(), and
parse_two_level_itt() do not check whether two-level tables or the selected
L2 sizes are enabled in its->idr1 (and vgic_v5_its_reset() initializes
its->idr1.l2sz to 0 while advertising dt_levels = 1 and itt_levels = 1).

Should changes to these IDR1 and IDR2 fields also flush the translation cache
and be enforced during DT/ITT translation walks?

[ ... ]
> @@ -1130,64 +1241,226 @@ static int vgic_v5_register_its_iodev(struct kvm *kvm, struct vgic_v5_its *its,
>  	return ret;
>  }
>  
> +static int vgic_v5_its_has_attr_regs(struct kvm_device *dev,
> +				     struct kvm_device_attr *attr)
> +{
> +	const struct vgic_register_region *region;
> +	gpa_t offset = attr->attr;
> +	int align;
> +
> +	region = vgic_find_mmio_region(vgic_v5_its_registers,
> +				       ARRAY_SIZE(vgic_v5_its_registers),
> +				       offset);

[Severity: Low]
In both vgic_v5_its_has_attr_regs() and vgic_v5_its_attr_regs_access(),
offset is a 64-bit gpa_t taken from attr->attr, but vgic_find_mmio_region()
takes a 32-bit unsigned int offset parameter:

const struct vgic_register_region *
vgic_find_mmio_region(const struct vgic_register_region *regions,
		      int nr_regions, unsigned int offset)

If userspace passes an attr->attr value with non-zero upper 32 bits and a
valid lower 32-bit register offset, vgic_find_mmio_region() truncates offset
to 32 bits, the alignment check only tests the low bits, and the register
callbacks mask addr with (SZ_64K - 1).

Should vgic_v5_its_has_attr_regs() and vgic_v5_its_attr_regs_access() reject
offsets outside the ITS register frame before calling vgic_find_mmio_region()?

> +	if (!region)
> +		return -ENXIO;
> +
> +	align = region->access_flags & VGIC_ACCESS_64bit ? 0x7 : 0x3;
> +	if (offset & align)
> +		return -EINVAL;
> +
> +	return 0;
> +}
> +
> +static int vgic_v5_its_attr_regs_access(struct kvm_device *dev,
> +					struct kvm_device_attr *attr,
> +					u64 *reg, bool is_write)
> +{
> +	const struct vgic_register_region *region;
> +	struct vgic_v5_its *its = dev->private;
> +	gpa_t offset = attr->attr;
> +	unsigned int len;
> +	gpa_t addr;
> +	int align;
> +	int ret = 0;
> +
> +	mutex_lock(&dev->kvm->lock);
> +
> +	if (kvm_trylock_all_vcpus(dev->kvm)) {
> +		mutex_unlock(&dev->kvm->lock);
> +		return -EBUSY;
> +	}
> +
> +	mutex_lock(&dev->kvm->arch.config_lock);
> +
> +	if (IS_VGIC_ADDR_UNDEF(its->vgic_v5_its_base)) {
> +		ret = -ENXIO;
> +		goto out;
> +	}
> +
> +	region = vgic_find_mmio_region(vgic_v5_its_registers,
> +				       ARRAY_SIZE(vgic_v5_its_registers),
> +				       offset);
> +	if (!region) {
> +		ret = -ENXIO;
> +		goto out;
> +	}
> +
> +	/*
> +	 * Although the spec supports upper/lower 32-bit accesses to
> +	 * 64-bit ITS registers, the userspace ABI requires 64-bit
> +	 * accesses to all 64-bit wide registers.
> +	 */
> +	align = region->access_flags & VGIC_ACCESS_64bit ? 0x7 : 0x3;
> +	len = region->access_flags & VGIC_ACCESS_64bit ? 8 : 4;

[Severity: Medium]
vgic_v5_its_has_attr_regs() and vgic_v5_its_attr_regs_access() determine the
userspace ABI alignment and access width from region->access_flags &
VGIC_ACCESS_64bit.

In vgic_v5_its_registers[], GICV5_ITS_SYNCR is declared with length 4 and
VGIC_ACCESS_32bit:

	REGISTER_ITS_V5_DESC(GICV5_ITS_SYNCR, vgic_v5_mmio_read_its_raz,
			     vgic_v5_mmio_write_its_dt_itt, 4,
			     VGIC_ACCESS_32bit),

However, in include/linux/irqchip/arm-gic-v5.h, GICV5_ITS_SYNCR is a 64-bit
register at offset 0x0140 with GICV5_ITS_SYNCR_SYNC at BIT_ULL(63) and
GICV5_ITS_SYNCR_SYNCALL at BIT_ULL(32), and the host driver in
drivers/irqchip/irq-gic-v5-its.c writes it with a 64-bit access.

Because its descriptor uses length 4 and VGIC_ACCESS_32bit, userspace
accesses treat GICV5_ITS_SYNCR as a 32-bit register (align 0x3, len 4), and
guest 64-bit MMIO writes or 32-bit writes to GICV5_ITS_SYNCR + 4 fail lookup
in vgic_get_mmio_region(). (Conversely, GICV5_ITS_SWERR_STATUSR,
GICV5_ITS_SWERR_SYNDROMER0, GICV5_ITS_SWERR_SYNDROMER1, and
GICV5_ITS_MPAM_PMGR are declared with VGIC_ACCESS_64bit without
VGIC_ACCESS_32bit.)

Should GICV5_ITS_SYNCR be registered with length 8 and
VGIC_ACCESS_64bit | VGIC_ACCESS_32bit?

> +
> +	if (offset & align) {
> +		ret = -EINVAL;
> +		goto out;
> +	}
> +
> +	addr = its->vgic_v5_its_base + offset;
> +
> +	if (is_write) {
> +		if (region->uaccess_its_write)
> +			ret = region->uaccess_its_write(dev->kvm, its, addr,
> +							len, *reg);
> +		else
> +			region->its_write(dev->kvm, its, addr, len, *reg);

[Severity: High]
Comparing all 32 register descriptors (covering 43 registers, including the
12 CoreSight PIDR/CIDR registers in the 48-byte GICV5_CORESIGHT_PIDR4 region)
in vgic_v5_its_registers[] between guest MMIO and userspace access in
vgic_v5_its_attr_regs_access():

- 6 descriptors (6 registers: GICV5_ITS_IDR0, GICV5_ITS_IDR1, GICV5_ITS_IDR2,
  GICV5_ITS_DT_BASER, GICV5_ITS_DT_CFGR, GICV5_ITS_READ_EVENT_DATAR) use a
  dedicated uaccess_its_write callback.
- 6 descriptors (6 registers: GICV5_ITS_CR0, GICV5_ITS_CR1, GICV5_ITS_DIDR,
  GICV5_ITS_EIDR, GICV5_ITS_GEN_EVENT_DIDR, GICV5_ITS_GEN_EVENT_EIDR) are RW
  state registers whose userspace writes fall back to region->its_write.
- 15 descriptors (26 registers: GICV5_ITS_IIDR, GICV5_ITS_AIDR,
  GICV5_ITS_STATUSR, GICV5_ITS_SYNC_STATUSR, GICV5_ITS_GEN_EVENT_STATUSR,
  GICV5_ITS_MEC_IDR, GICV5_ITS_MEC_MECID_R, GICV5_ITS_MPAM_IDR,
  GICV5_ITS_MPAM_PARTID_R, GICV5_ITS_SWERR_STATUSR,
  GICV5_ITS_SWERR_SYNDROMER0, GICV5_ITS_SWERR_SYNDROMER1,
  GICV5_ITS_MPAM_PMGR, GICV5_CORESIGHT_DEVARCH, and the 12 registers in
  GICV5_CORESIGHT_PIDR4) use vgic_v5_mmio_write_its_wi().
- 5 descriptors (5 registers: GICV5_ITS_INV_EVENTR, GICV5_ITS_INV_DEVICER,
  GICV5_ITS_READ_EVENTR, GICV5_ITS_SYNCR, GICV5_ITS_GEN_EVENTR) are action
  registers that read as zero via vgic_v5_mmio_read_its_raz() and have no
  uaccess_its_write callback, so userspace writes execute
  vgic_v5_mmio_write_its_dt_itt().

In particular, vgic_v5_its_attr_regs_access() only checks that
its->vgic_v5_its_base is defined; neither it nor the LPI translation and
injection paths (vgic_v5_its_handle_gen_event(), __vgic_v5_its_resolve_lpi(),
vgic_v5_its_inject_msi(), and vgic_v5_its_inject_lpi()) check
vgic_initialized(kvm) or irqchip_in_kernel(kvm):

static void vgic_v5_its_inject_lpi(struct kvm *kvm, struct vgic_irq *irq)
{
	kvm_call_hyp(__vgic_v5_vdpend, irq->intid, true,
		     kvm->arch.vgic.gicv5_vm.vm_id);
}

Because vgic_v5_its_create() allows creating the ITS device before
KVM_DEV_TYPE_ARM_VGIC_V5 is created, kvm->arch.vgic.gicv5_vm.vm_id is 0 from
the zero-allocation of struct kvm (since VGIC_V5_VM_ID_INVAL is only assigned
in kvm_vgic_create(), and vgic_v5_allocate_vm_id() allocates hardware VM IDs
starting from 0). If KVM_DEV_TYPE_ARM_VGIC_V5 has been created but not yet
initialized via KVM_DEV_ARM_VGIC_CTRL_INIT, vm_id is VGIC_V5_VM_ID_INVAL (-1,
truncated to 0xffff).

If userspace enables GICV5_ITS_CR0 via KVM_DEV_ARM_VGIC_GRP_ITS_REGS and
triggers LPI injection via GICV5_ITS_GEN_EVENTR (with GICV5_ITS_GEN_EVENTR_R
set) or KVM_SIGNAL_MSI before the VGICv5 is created or initialized, can this
execute __vgic_v5_vdpend at EL2 with hardware vm_id 0 (injecting LPIs into
another guest VM owning VM ID 0) or with an uninitialized vm_id of 0xffff?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261102194454.4086911-1-sascha.bischoff@arm.com?part=10

Re: [PATCH v2 11/16] irqchip/gic-v5: Enable direct injection in ITS

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [High] irqchip/gic-v5: sleep in atomic context via gicv5_irs_syncr() under desc->lock
- [High] irqchip/gic-v5: unlocked concurrent MMIO access in gicv5_its_itt_cache_inv()
- [Medium] irqchip/gic-v5: ignored timeout in gicv5_irs_syncr() during ITTE restoration
--

commit 0ced984dd99b707bdb6169e0051131c256a7148e
Author: Sascha Bischoff <sascha.bischoff@arm.com>

irqchip/gic-v5: Enable direct injection in ITS

This commit adds the irq_set_vcpu_affinity callback and helper functions to
install and remove virtual ITTE translations in the GICv5 ITS for KVM direct
interrupt injection, while disabling lazy masking for forwarded interrupts.

> diff --git a/drivers/irqchip/irq-gic-v5-its.c b/drivers/irqchip/irq-gic-v5-its.c
> index 28e39b065de0e..d91f4ea1ef440 100644
> --- a/drivers/irqchip/irq-gic-v5-its.c
> +++ b/drivers/irqchip/irq-gic-v5-its.c

[ ... ]

> @@ -723,6 +765,102 @@ static void gicv5_its_compose_msi_msg(struct irq_data *d, struct msi_msg *msg)
>  	msi_msg_set_addr(irq_data_get_msi_desc(d), msg, addr);
>  }
>  
> +#ifdef CONFIG_KVM
> +static int gicv5_its_set_vcpu_affinity(struct irq_data *d, void *int_data)
> +{
> +	struct gicv5_its_dev *its_dev = irq_data_get_irq_chip_data(d);
> +	u16 event_id = FIELD_GET(GICV5_ITS_HWIRQ_EVENT_ID, d->hwirq);
> +	u32 dev_id = FIELD_GET(GICV5_ITS_HWIRQ_DEVICE_ID, d->hwirq);
> +	struct virt_intr_data *data = int_data;
> +	struct gicv5_its_chip_data *its;
> +	__le64 *itte;
> +	u64 itte_val, tmp;
> +	int ret = 0;

[ ... ]

> +invalidate:
> +	/* Make the change ITS-visible */
> +	dsb(st);
> +	ret = gicv5_its_itt_cache_inv(its, dev_id, event_id);

[Severity: High]
As reported by Sashiko on v1 [1], can concurrent calls to
gicv5_its_itt_cache_inv() on the same physical ITS race with each other?

Both gicv5_its_enable_direct_injection() and
gicv5_its_disable_direct_injection() invoke irq_set_vcpu_affinity(), which
only holds the per-IRQ desc->lock before calling
gicv5_its_set_vcpu_affinity().

Inside gicv5_its_itt_cache_inv():

drivers/irqchip/irq-gic-v5-its.c:gicv5_its_itt_cache_inv() {
	...
	its_writeq_relaxed(its, didr, GICV5_ITS_DIDR);
	its_writel_relaxed(its, eidr, GICV5_ITS_EIDR);
	its_writel_relaxed(its, eventr, GICV5_ITS_INV_EVENTR);

	return gicv5_its_cache_sync(its);
}

Because struct gicv5_its_chip_data has no spinlock serializing access to the
shared ITS MMIO registers, two CPUs concurrently configuring different IRQs
on the same ITS (or racing with gicv5_its_irq_domain_activate(),
gicv5_its_irq_domain_deactivate(), or gicv5_its_device_cache_inv()) can
overwrite each other's GICV5_ITS_DIDR and GICV5_ITS_EIDR values before
GICV5_ITS_INV_EVENTR is written, invalidating the wrong (DeviceID, EventID)
entry and leaving stale translations cached in the ITS.

> +	if (!ret && !data) {
> +		/*
> +		 * Invalidation sends events using the old translation to the
> +		 * IRS, but does not wait for them to be processed.
> +		 */
> +		gicv5_irs_syncr();
> +	}

[Severity: High]
Will calling gicv5_irs_syncr() here sleep in atomic context?

gicv5_its_disable_direct_injection() calls irq_set_vcpu_affinity(virq, NULL),
which acquires raw_spin_lock_irqsave(&desc->lock, flags) via
scoped_irqdesc_get_and_lock() before invoking gicv5_its_set_vcpu_affinity().

When gicv5_its_set_vcpu_affinity() calls gicv5_irs_syncr():

drivers/irqchip/irq-gic-v5-irs.c:gicv5_irs_syncr() {
	...
	syncr = FIELD_PREP(GICV5_IRS_SYNCR_SYNC, 1);
	irs_writel_relaxed(irs_data, syncr, GICV5_IRS_SYNCR);

	gicv5_wait_for_op(irs_data->irs_base, GICV5_IRS_SYNC_STATUSR,
			  GICV5_IRS_SYNC_STATUSR_IDLE);
}

Unlike gicv5_wait_for_op_atomic(), gicv5_wait_for_op() expands to
gicv5_wait_for_op_s(), which calls readl_poll_timeout() with a sleep_us of 1,
executing might_sleep_if(1) and usleep_range() while holding desc->lock with
interrupts disabled.

> +	if (ret) {
> +		/* Keep a failed restoration distinguishable from a clean host ITTE. */
> +		if (!data)
> +			irqd_set_forwarded_to_vcpu(d);
> +		return ret;
> +	}

[Severity: Medium]
What happens if polling GICV5_IRS_SYNC_STATUSR times out inside
gicv5_irs_syncr() while restoring a host ITTE?

gicv5_irs_syncr() returns void and discards the -ETIMEDOUT return value from
gicv5_wait_for_op(). If the IRS synchronization times out after
gicv5_its_itt_cache_inv() succeeds, ret remains 0.

As a result, gicv5_its_set_vcpu_affinity() will proceed to clear
IRQD_FORWARDED_TO_VCPU instead of keeping the retryable forwarded state, and
gicv5_its_disable_direct_injection() will clear IRQ_DISABLE_UNLAZY and return
0 to the caller while events using the old virtual translation may still be
in flight at the IRS. Should gicv5_irs_syncr() propagate its error so ret can
be checked here?

[1] https://lore.kernel.org/r/sashiko-outbox-174629@kernel.org

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261102194454.4086911-1-sascha.bischoff@arm.com?part=11

Re: [PATCH v2 15/16] KVM: selftests: Add VGICv5 ITS translation cache tests

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] KVM: selftests: IDR1 narrowing clears cache before ITS disable in vgic_v5
--

commit a7e0ed97dc0fc966b7b8a5fa6982c6d6e3d8240f
Author: Sascha Bischoff <sascha.bischoff@arm.com>

KVM: selftests: Add VGICv5 ITS translation cache tests

This commit adds a KVM selftest for VGICv5 ITS translation caching, guest
invalidation commands, register width accesses, and negative KVM_SIGNAL_MSI
paths.

>     KVM: selftests: Add VGICv5 ITS translation cache tests
>     
>     Add end-to-end coverage for VGICv5 ITS translation and cache
>     invalidation. The ITS attribute test only checks the userspace API
>     surface; this test verifies that KVM can parse guest-provided linear
>     DT and ITT state and use it to deliver LPIs through guest GEN_EVENTR
>     and KVM_SIGNAL_MSI.
>     
>     Build a two-VPE VM with IRS, ITS and an LPI IST. VPE0 programs a
>     linear device table and interrupt translation table, enables the
>     virtual ITS, enables two LPIs targeting VPE1, and sends GEN_EVENTR to
>     populate the translation cache with EventID to LPI A. VPE1 reports the
>     LPI identity it consumed.
>     
>     VPE0 then rewrites the ITT to point at LPI B without invalidation, and
>     a host KVM_SIGNAL_MSI injection verifies that the stale cached LPI A
>     translation is reused. VPE0 invalidates the event with ITS_INV_EVENTR
>     and the host verifies that LPI B is delivered.
>     
>     VPE0 then rewrites the ITT back to LPI A, invalidates the device with
>     ITS_INV_DEVICER, and the host verifies that LPI A is delivered again.
>     This exercises the guest-side event and device invalidation paths
>     rather than invalidating translations directly from the test host.
>     
>     Also cover negative MSI paths by checking that KVM_SIGNAL_MSI rejects a
>     message without KVM_MSI_VALID_DEVID and rejects a message whose address
>     does not target the GICv5 ITS. Finally, inject once before guest-side
>     ITS disable to prove delivery still works, then have VPE0 disable the
>     ITS and verify that a cached KVM_SIGNAL_MSI delivery is rejected with
>     ENXIO.
>     
>     Check 32-bit accesses to both halves of DT_BASER and preservation of
>     the untouched half. Verify that guest writes to the ITS ID registers
>     and READ_EVENT_DATAR are ignored, and that READ_EVENT_DATAR can be read
>     at both 32-bit and 64-bit widths.
>     
>     Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
> 
> diff --git a/tools/testing/selftests/kvm/arm64/vgic_v5.c b/tools/testing/selftests/kvm/arm64/vgic_v5.c
> index 92da9753ce8e6..a9b0bca24ec08 100644
> --- a/tools/testing/selftests/kvm/arm64/vgic_v5.c
> +++ b/tools/testing/selftests/kvm/arm64/vgic_v5.c

[ ... ]

> @@ -2312,6 +2644,183 @@ static void test_vgic_v5_ist_save_restore(void)

[ ... ]

> +static void test_vgic_v5_its_translation_cache(void)
> +{

[ ... ]

> +	/*
> +	 * Device invalidation must cover all cached EventIDs for the
> +	 * DeviceID. The guest has again re-written the event to translate to
> +	 * LPI A.
> +	 */
> +	ret = run_vcpu(vcpus[0]);
> +	TEST_ASSERT(!ret, "Failed to run GICv5 ITS device invalidation");
> +	vgic_v5_expect_sync(vcpus[0], GUEST_CMD_ITS_DEVICE_INVALIDATED,
> +			    "GICv5 ITS guest did not invalidate device");
> +	vgic_v5_expect_its_lpi(&v, vcpus[1], ITS_TEST_LPI_A);
> +
> +	/* A narrower userspace IDR1 must invalidate the cached DeviceID. */
> +	ret = __kvm_device_attr_get(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
> +				    GICV5_ITS_IDR1, &idr1);
> +	TEST_ASSERT(!ret, "GICv5 ITS_REGS failed to read IDR1");
> +	idr1 = (idr1 & ~GICV5_ITS_IDR1_DEVICEID_BITS) |
> +		FIELD_PREP(GICV5_ITS_IDR1_DEVICEID_BITS, 2);
> +	ret = __kvm_device_attr_set(v.its_fd, KVM_DEV_ARM_VGIC_GRP_ITS_REGS,
> +				    GICV5_ITS_IDR1, &idr1);
> +	TEST_ASSERT(!ret, "GICv5 ITS_REGS failed to narrow DeviceID width");
> +	ret = __vgic_v5_signal_msi(v.vm, GICV5_ITS_BASE_GPA +
> +				   GICV5_ITS_TRANSLATER,
> +				   ITS_TEST_DEVICE_ID, ITS_TEST_EVENT_ID,
> +				   KVM_MSI_VALID_DEVID);
> +	TEST_ASSERT(ret == 0,
> +		    "GICv5 ITS did not block MSI after DeviceID width changed");
> +
> +	/* Disabling the ITS must also prevent delivery from cached entries. */
> +	ret = run_vcpu(vcpus[0]);
> +	TEST_ASSERT(!ret, "Failed to run GICv5 ITS disable");
> +	vgic_v5_expect_sync(vcpus[0], GUEST_CMD_ITS_DISABLED,
> +			    "GICv5 ITS guest did not disable ITS");
> +
> +	/*
> +	 * We should fail to inject this MSI. Use the raw call to ensure we
> +	 * catch the return value.
> +	 */
> +	ret = __vgic_v5_signal_msi(v.vm, GICV5_ITS_BASE_GPA +
> +				   GICV5_ITS_TRANSLATER,
> +				   ITS_TEST_DEVICE_ID, ITS_TEST_EVENT_ID,
> +				   KVM_MSI_VALID_DEVID);
> +	TEST_ASSERT(ret < 0 && errno == ENXIO,
> +		    "GICv5 ITS accepted cached MSI while disabled");

[Severity: Medium]
Does this final check in test_vgic_v5_its_translation_cache() actually test
disabling the ITS while a valid entry is present in the translation cache?

In the preceding block, writing GICV5_ITS_IDR1 via
KVM_DEV_ARM_VGIC_GRP_ITS_REGS to narrow GICV5_ITS_IDR1_DEVICEID_BITS to 2
causes vgic_v5_uaccess_write_its_idr() -> its_cache_clear_translations() to
flush the translation cache, and leaves ITS_TEST_DEVICE_ID (5) out of range
so the subsequent __vgic_v5_signal_msi() call fails in get_itt() without
repopulating the cache.

Because GICV5_ITS_IDR1 is not restored and vgic_v5_expect_its_lpi() is not
called again before VCPU0 runs to GUEST_CMD_ITS_DISABLED, the cache is
already empty when the ITS is disabled, which also differs from the sequence
described in the commit message ("Finally, inject once before guest-side ITS
disable to prove delivery still works, then have VPE0 disable the ITS and
verify that a cached KVM_SIGNAL_MSI delivery is rejected with ENXIO").

Should GICV5_ITS_IDR1 be restored and the translation cache repopulated via
vgic_v5_expect_its_lpi() before disabling the ITS (and should the commit
message also mention the IDR1 narrowing, read-only DT register, and DeviceID
cap checks)?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261102194454.4086911-1-sascha.bischoff@arm.com?part=15

Re: [PATCH v2 13/16] KVM: arm64: gic-v5: Implement irq forwarding

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [High] KVM: arm64: gic-v5: sleep in atomic context when disabling direct injection
- [High] KVM: arm64: gic-v5: stale host_irq and leaked LPI on unforward failure
- [High] KVM: arm64: gic-v5: unserialized physical ITS MMIO in gicv5_its_itt_cache_inv()
- [Low] KVM: arm64: gic-v5: unused VGIC_V5 check in vgic_supports_direct_msis()
--

commit deaf3d7036edf849d625db99977af89a6c3e3411
Author: Sascha Bischoff <sascha.bischoff@arm.com>

KVM: arm64: gic-v5: Implement irq forwarding

This commit implements direct interrupt injection (IRQ forwarding) for GICv5
via the ITS, placing the GICv4 and GICv5 forwarding hooks behind common
kvm_vgic_set_forwarding() and kvm_vgic_unset_forwarding() shims. It also
tracks the translation cache entry that owns forwarding on each LPI and uses
access_remote_vm() when resolving guest DT/ITT entries from another
process's address space.

> diff --git a/arch/arm64/kvm/vgic/vgic-its-v5.c b/arch/arm64/kvm/vgic/vgic-its-v5.c
> index 638ee47f6e27d..3648d3f7c1b29 100644
> --- a/arch/arm64/kvm/vgic/vgic-its-v5.c
> +++ b/arch/arm64/kvm/vgic/vgic-its-v5.c
> @@ -38,7 +39,35 @@ static void its_cache_drop_translation(struct kvm *kvm,
>  				       struct its_cache_translation_entry *entry)
>  {
>  	struct vgic_irq *irq = entry->irq;
> +	unsigned long flags;
> +	bool drop_forwarding_ref = false;
>  
> +	/*
> +	 * Multiple translations can share an LPI as we have no way of stopping
> +	 * the guest from doing so. Only the entry associated with direct injection
> +	 * may tear down the IRQ's forwarding state, as otherwise we can end up
> +	 * accidentally tearing something down. Realistically, this only would
> +	 * result in a performance degradation, but we might as well avoid that
> +	 * in the first place if we can.
> +	 *
> +	 * Keep the lock until the host IRQ is restored, so the producer cannot
> +	 * free it while we are using its IRQ number.
> +	 */
> +	raw_spin_lock_irqsave(&irq->irq_lock, flags);
> +	if (irq->forwarding_owner == entry) {
> +		/* The entry is about to be freed, even if unforwarding fails. */
> +		irq->forwarding_owner = NULL;
> +		if (irq->hw &&
> +		    !WARN_ON_ONCE(gicv5_its_disable_direct_injection(irq->host_irq))) {
> +			irq->hw = false;
> +			irq->host_irq = 0;
> +			drop_forwarding_ref = true;
> +		}
> +	}
> +	raw_spin_unlock_irqrestore(&irq->irq_lock, flags);

[Severity: High]
Can calling gicv5_its_disable_direct_injection() here and in
kvm_vgic_v5_unset_forwarding() sleep in atomic context?

Both its_cache_drop_translation() and kvm_vgic_v5_unset_forwarding() hold
raw_spin_lock_irqsave(&irq->irq_lock, flags) (and
kvm_arch_update_irqfd_routing() -> kvm_vgic_unset_forwarding() is also
invoked under spin_lock_irq(&kvm->irqfds.lock)). The call chain is:

  gicv5_its_disable_direct_injection()
    -> irq_set_vcpu_affinity(virq, NULL) [acquires desc->lock raw_spinlock]
      -> gicv5_its_set_vcpu_affinity(d, NULL)
        -> gicv5_irs_syncr()
          -> gicv5_wait_for_op(..., GICV5_IRS_SYNC_STATUSR,
                               GICV5_IRS_SYNC_STATUSR_IDLE)

In drivers/irqchip/irq-gic-v5-irs.c, gicv5_irs_syncr() uses the sleeping
gicv5_wait_for_op() macro (which expands to readl_poll_timeout() with a 1 us
sleep and calls might_sleep() and usleep_range()) rather than
gicv5_wait_for_op_atomic().

Should gicv5_irs_syncr() use gicv5_wait_for_op_atomic() since it is called
under desc->lock and irq->irq_lock?

[ ... ]

> diff --git a/arch/arm64/kvm/vgic/vgic-mmio.c b/arch/arm64/kvm/vgic/vgic-mmio.c
> index 3c0c858608752..3268e3d3d5170 100644
> --- a/arch/arm64/kvm/vgic/vgic-mmio.c
> +++ b/arch/arm64/kvm/vgic/vgic-mmio.c
> @@ -1144,6 +1144,8 @@ int vgic_register_dist_iodev(struct kvm *kvm, gpa_t dist_base_address,
>  
>  bool vgic_supports_direct_msis(struct kvm *kvm)
>  {
> +	const struct vgic_dist *dist = &kvm->arch.vgic;
> +
>  	/*
>  	 * Deliberately conflate vLPI and vSGI support on GICv4.1 hardware,
>  	 * indirectly allowing userspace to control whether or not vPEs are
> @@ -1152,7 +1154,13 @@ bool vgic_supports_direct_msis(struct kvm *kvm)
>  	if (system_supports_direct_sgis() && !vgic_supports_direct_sgis(kvm))
>  		return false;
>  
> -	return kvm_vgic_global_state.has_gicv4 && vgic_has_its(kvm);
> +	if (kvm_vgic_global_state.has_gicv4 && vgic_has_its(kvm))
> +		return true;
> +	if (kvm_vgic_global_state.type == VGIC_V5 && vgic_has_its(kvm) &&
> +	    dist->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V5)
> +		return true;
> +
> +	return false;
>  }

[Severity: Low]
This isn't a bug, but is the new VGIC_V5 check in vgic_supports_direct_msis()
ever reached?

In arch/arm64/kvm/vgic/vgic.h, vgic_supports_direct_irqs() already returns
true early when vgic_is_v5(kvm) is true before calling
vgic_supports_direct_msis(kvm). The only other callers of
vgic_supports_direct_msis() are kvm_vgic_v4_set_forwarding() and
kvm_vgic_v4_unset_forwarding(), whereas kvm_vgic_v5_set_forwarding() and
kvm_vgic_v5_unset_forwarding() check !vgic_has_its(kvm) directly.

Should kvm_vgic_v5_set_forwarding() and kvm_vgic_v5_unset_forwarding() (or
the common kvm_vgic_set_forwarding() and kvm_vgic_unset_forwarding() shims)
call vgic_supports_direct_msis(kvm) instead of vgic_has_its(kvm)?

[ ... ]

> diff --git a/arch/arm64/kvm/vgic/vgic-v5.c b/arch/arm64/kvm/vgic/vgic-v5.c
> index 18184020e188b..c2ae7b0fb3a7f 100644
> --- a/arch/arm64/kvm/vgic/vgic-v5.c
> +++ b/arch/arm64/kvm/vgic/vgic-v5.c
> @@ -1635,3 +1635,151 @@ unsigned long vgic_v5_handle_guest_coresight_idreg_read(u32 offset)
> +int kvm_vgic_v5_set_forwarding(struct kvm *kvm, int virq,
> +			       struct kvm_kernel_irq_routing_entry *irq_entry)
> +{

[ ... ]

> +	ret = gicv5_its_enable_direct_injection(virq, vgic_v5_vm_id(kvm),
> +						vgic_v5_get_hwirq_id(irq->intid));

[Severity: High]
What serializes concurrent physical ITS MMIO register accesses in
gicv5_its_itt_cache_inv() when forwarding or unforwarding interrupts?

Enabling and disabling direct injection calls:

  gicv5_its_enable_direct_injection() / gicv5_its_disable_direct_injection()
    -> irq_set_vcpu_affinity() [locks per-IRQ desc->lock]
      -> gicv5_its_set_vcpu_affinity()
        -> gicv5_its_itt_cache_inv()

In drivers/irqchip/irq-gic-v5-its.c, gicv5_its_itt_cache_inv() writes the
shared physical ITS registers and waits for completion without a per-ITS lock
in struct gicv5_its_chip_data:

	its_writeq_relaxed(its, didr, GICV5_ITS_DIDR);
	its_writel_relaxed(its, eidr, GICV5_ITS_EIDR);
	its_writel_relaxed(its, eventr, GICV5_ITS_INV_EVENTR);

	return gicv5_its_cache_sync(its);

Because irq_set_vcpu_affinity() only holds the per-IRQ desc->lock and
kvm_vgic_v5_unset_forwarding() only holds the per-LPI irq->irq_lock (without
its->its_lock), two CPUs forwarding, unforwarding, or invalidating different
IRQs on the same physical ITS (within the same VM, across VMs, or racing with
host gicv5_its_map_event() / gicv5_its_unmap_event()) can interleave writes
to GICV5_ITS_DIDR and GICV5_ITS_EIDR before GICV5_ITS_INV_EVENTR is written.

Could one CPU overwrite GICV5_ITS_DIDR or GICV5_ITS_EIDR so that the physical
ITS invalidates the wrong DeviceID/EventID entry while gicv5_its_cache_sync()
still sees GICV5_ITS_STATUSR_IDLE and returns success, leaving a stale
virtual or host ITTE cached in the physical ITS?

> +	if (ret) {
> +		if (ret != -ETIMEDOUT) {
> +			vgic_put_irq(kvm, irq);
> +			return 0;
> +		}

[ ... ]

> +void kvm_vgic_v5_unset_forwarding(struct kvm *kvm, int virq)
> +{
> +	struct vgic_irq *irq;
> +	bool drop_forwarding_ref = false;
> +
> +	if (!vgic_has_its(kvm))
> +		return;
> +
> +	/*
> +	 * Note: We don't remove the entry from our translation cache here. It
> +	 * is up to the guest to invalidate the old translations, and hence the
> +	 * removal path is tied into ITS_INV_DEVICER and ITS_INV_EVENTR, rather
> +	 * than being explicitly called here.
> +	 */
> +	irq = vgic_v5_host_irq_get_lpi(kvm, virq);
> +	if (!irq)
> +		return;
> +
> +	scoped_guard(raw_spinlock_irqsave, &irq->irq_lock) {
> +		if (irq->hw && irq->host_irq == virq &&
> +		    !WARN_ON_ONCE(gicv5_its_disable_direct_injection(virq))) {
> +			irq->hw = false;
> +			irq->host_irq = 0;
> +			irq->forwarding_owner = NULL;
> +			drop_forwarding_ref = true;
> +		}
> +	}
> +
> +	if (drop_forwarding_ref)
> +		vgic_put_irq(kvm, irq);
> +
> +	vgic_put_irq(kvm, irq);
> +}

[Severity: High]
If gicv5_its_disable_direct_injection(virq) fails in
kvm_vgic_v5_unset_forwarding(), can a subsequent cache eviction operate on a
freed or reallocated host Linux IRQ number, or leak the LPI and reuse the VM
ID at teardown?

1. kvm_arch_irq_bypass_del_producer() returns void to __disconnect() in
   virt/lib/irqbypass.c, so producer unregistration cannot be aborted when
   gicv5_its_disable_direct_injection(virq) fails. Once
   irq_bypass_unregister_producer() returns, the producer (e.g. VFIO) tears
   down and frees virq, which can then be reallocated to another device.
   Because irq->forwarding_owner, irq->hw, and irq->host_irq remain set here,
   a later guest invalidation or vgic_v5_its_destroy() ->
   its_cache_clear_translations() -> its_cache_drop_translation() will call
   gicv5_its_disable_direct_injection(irq->host_irq) on that freed or
   reallocated virq.

2. Similarly, if kvm_arch_update_irqfd_routing() calls
   kvm_vgic_unset_forwarding() when an irqfd routing entry changes from
   KVM_IRQ_ROUTING_MSI to a non-MSI type and
   gicv5_its_disable_direct_injection() fails, a subsequent
   kvm_arch_irq_bypass_del_producer() sees irq_entry->type !=
   KVM_IRQ_ROUTING_MSI and returns early without retrying
   kvm_vgic_unset_forwarding().

3. If gicv5_its_disable_direct_injection() fails in both
   its_cache_drop_translation() and kvm_vgic_v5_unset_forwarding(),
   drop_forwarding_ref is never set. When the VM is destroyed,
   kvm_vgic_dist_destroy() runs xa_destroy(&dist->lpi_xa) and leaks the
   struct vgic_irq, while vgic_v5_teardown() unconditionally calls
   vgic_v5_release_vm_id(kvm) even though a physical ITS virtual ITTE may
   still reference that VM ID.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261102194454.4086911-1-sascha.bischoff@arm.com?part=13

Re: [PATCH v2 01/16] KVM: arm64: vgic: Split out common LPI allocation

[Fuad Tabba]

Hi Sascha,

On Mon, 02 Nov 2026 19:44:39 +0000, Sascha Bischoff
<sascha.bischoff@arm.com> wrote:
[...]
> Initialise LPIs as Group 1 before publishing them in lpi_xa. GICv3
> LPIs are Group 1, and another vCPU can find a new LPI as soon as it is
> published. GICv5 does not use the group field.

Two nits: the old vgic_add_lpi() already set the group before
publishing. The comments in its_free_ite() and vgic_alloc_lpi() still
name vgic_add_lpi() for the GICv3 path.

Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>

Thanks!
/fuad

Re: [PATCH v2 02/16] KVM: arm64: vgic: Pass opaque device pointer to ITS MMIO callbacks

[Fuad Tabba]

On Mon, 2 Nov 2026 at 19:45, Sascha Bischoff <sascha.bischoff@arm.com> wrote:
>
> The ITS MMIO callbacks currently take a struct vgic_its pointer,
> which ties them to the GICv3 ITS. GICv5 needs to use the same MMIO
> dispatch path with a different device structure.
>
> Pass an opaque device pointer to the callbacks and cast it in the
> GICv3 handlers. This does not change GICv3 behaviour.
>
> Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>

Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>

Cheers,
/fuad

Re: [PATCH v2 03/16] KVM: arm64: gic-v5: Share CoreSight ID register emulation

[Fuad Tabba]

On Mon, 02 Nov 2026 19:44:41 +0000, Sascha Bischoff
<sascha.bischoff@arm.com> wrote:
[...]
> This keeps the IRS behaviour unchanged while giving the ITS MMIO
> emulation a helper it can reuse when it is added in the following commit.

nit: the ITS emulation that uses it arrives in patch 6.

Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>

Thanks!
/fuad

Re: [PATCH v2 04/16] irqchip/gic-v5: Add ITS MMIO register definitions

[Fuad Tabba]

On Mon, 02 Nov 2026 19:44:42 +0000, Sascha Bischoff
<sascha.bischoff@arm.com> wrote:
[...]
> Keeping the ITS register frame definitions together avoids spreading
> register-definition churn across the subsequent KVM ITS emulation
> patches.

nit: the re-tab and the XDMN_EVENTs rename worth a mention here?

Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>

Thanks!
/fuad

Re: [PATCH v2 06/16] KVM: arm64: gic-v5: Add ITS MMIO state emulation

[Fuad Tabba]

On Mon, 02 Nov 2026 19:44:44 +0000, Sascha Bischoff
<sascha.bischoff@arm.com> wrote:
[...]
> diff --git a/arch/arm64/kvm/vgic/vgic-its-v5.c b/arch/arm64/kvm/vgic/vgic-its-v5.c
[...]
> +static void vgic_v5_its_reset(struct vgic_v5_its *its)
> +{
[...]
> +        its->idr1.dt_levels = 1;
> +        its->idr1.itt_levels = 1;
> +        its->idr1.l2sz = 0;

L2SZ 0 is reserved in the spec when either LEVELS bit is set. Could
this advertise 0b111, as the spec recommends? (Sashiko [1])

Thanks!
/fuad

[1] https://lore.kernel.org/all/sashiko-outbox-194056@kernel.org/

Re: [PATCH v2 08/16] KVM: arm64: gic-v5: Add ITS translation cache maintenance

[Fuad Tabba]

On Mon, 02 Nov 2026 19:44:46 +0000, Sascha Bischoff
<sascha.bischoff@arm.com> wrote:
[...]
> diff --git a/arch/arm64/kvm/vgic/vgic-its-v5.c b/arch/arm64/kvm/vgic/vgic-its-v5.c
[...]
> +static int its_cache_add_translation(struct kvm *kvm, struct vgic_v5_its *its,
[...]
> +        old = xa_store(&its->translation_cache, key, entry,
> +                       GFP_KERNEL_ACCOUNT);

nit: fits on one line.

Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>

Thanks!
/fuad

Re: [PATCH v2 09/16] KVM: arm64: gic-v5: Add ITS MSI injection

[Fuad Tabba]

On Mon, 02 Nov 2026 19:44:47 +0000, Sascha Bischoff
<sascha.bischoff@arm.com> wrote:
[...]
> diff --git a/arch/arm64/kvm/vgic/vgic-its-v5.c b/arch/arm64/kvm/vgic/vgic-its-v5.c
[...]
> +int vgic_v5_its_inject_cached_translation(struct kvm *kvm, struct kvm_msi *msi)
> +{
[...]
> +        if (!READ_ONCE(its->enabled))
> +                return -EWOULDBLOCK;

Should the plain stores to its->enabled should be WRITE_ONCE() to pair
with this?

Thanks!
/fuad

Re: [PATCH v2 10/16] KVM: arm64: gic-v5: Add ITS userspace register access

[Fuad Tabba]

On Mon, 02 Nov 2026 19:44:48 +0000, Sascha Bischoff
<sascha.bischoff@arm.com> wrote:
[...]
> diff --git a/arch/arm64/kvm/vgic/vgic-its-v5.c b/arch/arm64/kvm/vgic/vgic-its-v5.c
[...]
> +static int vgic_v5_its_attr_regs_access(struct kvm_device *dev,
> +                                        struct kvm_device_attr *attr,
> +                                        u64 *reg, bool is_write)
> +{
[...]
> +        if (IS_VGIC_ADDR_UNDEF(its->vgic_v5_its_base)) {
> +                ret = -ENXIO;
> +                goto out;
> +        }

This lets userspace set CR0.ITSEN before the vgic is initialised,
while the VM ID is still VGIC_V5_VM_ID_INVAL. An MSI (KVM_SIGNAL_MSI
or an irqfd) then reaches EL2 with VM ID 0xffff, and forwarding writes
the same into the ITTE. With 16 VM ID bits and every lower ID in use,
that lands in another VM. Could enabling the ITS be rejected until the
vgic is initialised? The existing its->enabled checks then cover
injection and forwarding. (Sashiko raised this on patch 7 [1])

Thanks!
/fuad

[1] https://lore.kernel.org/all/sashiko-outbox-194102@kernel.org/

Re: [PATCH v2 11/16] irqchip/gic-v5: Enable direct injection in ITS

[Fuad Tabba]

On Mon, 02 Nov 2026 19:44:49 +0000, Sascha Bischoff
<sascha.bischoff@arm.com> wrote:
[...]
> Co-authored-by: Timothy Hayes <timothy.hayes@arm.com>
> Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>

This should be Co-developed-by:, followed by Timothy's Signed-off-by:.

[...]
> diff --git a/drivers/irqchip/irq-gic-v5-its.c b/drivers/irqchip/irq-gic-v5-its.c
[...]
> +static int gicv5_its_set_vcpu_affinity(struct irq_data *d, void *int_data)
[...]
> +invalidate:
> +        /* Make the change ITS-visible */
> +        dsb(st);
> +        ret = gicv5_its_itt_cache_inv(its, dev_id, event_id);
> +        if (!ret && !data) {
> +                /*
> +                 * Invalidation sends events using the old translation to the
> +                 * IRS, but does not wait for them to be processed.
> +                 */
> +                gicv5_irs_syncr();
> +        }

gicv5_irs_syncr() returns void, so a timed-out sync looks like success
here. Could it return the poll's result into ret, so the if (ret) path
keeps the flag for a retry? (Sashiko [1])

This relies on your host series' patches 3 and 4. Without them the
sync sleeps under desc->lock, and this invalidation races the host's.
Should the cover mention the dependency?

Thanks!
/fuad

[1] https://lore.kernel.org/all/sashiko-outbox-194164@kernel.org/

Re: [PATCH v2 13/16] KVM: arm64: gic-v5: Implement irq forwarding

[Fuad Tabba]

On Mon, 02 Nov 2026 19:44:51 +0000, Sascha Bischoff
<sascha.bischoff@arm.com> wrote:
[...]
> diff --git a/arch/arm64/kvm/vgic/vgic-mmio.c b/arch/arm64/kvm/vgic/vgic-mmio.c
[...]
> @@ -1152,7 +1154,13 @@ bool vgic_supports_direct_msis(struct kvm *kvm)
>          if (system_supports_direct_sgis() && !vgic_supports_direct_sgis(kvm))
>                  return false;
>
> -        return kvm_vgic_global_state.has_gicv4 && vgic_has_its(kvm);
> +        if (kvm_vgic_global_state.has_gicv4 && vgic_has_its(kvm))
> +                return true;
> +        if (kvm_vgic_global_state.type == VGIC_V5 && vgic_has_its(kvm) &&
> +            dist->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V5)
> +                return true;

No GICv5 VM gets here, since every caller diverts v5 first. Could this
hunk, and patch 12, go? (Sashiko [1])

[...]
> diff --git a/arch/arm64/kvm/vgic/vgic-v5.c b/arch/arm64/kvm/vgic/vgic-v5.c
[...]
> +int kvm_vgic_v5_set_forwarding(struct kvm *kvm, int virq,
> +                               struct kvm_kernel_irq_routing_entry *irq_entry)
[...]
> +        ret = gicv5_its_enable_direct_injection(virq, vgic_v5_vm_id(kvm),
> +                                                vgic_v5_get_hwirq_id(irq->intid));

Before init this writes VM ID 0xffff into the ITTE. See my reply to patch 10.

[...]
> +                /* Keep the host IRQ association so a later unforward can retry. */
> +                ret = 0;

If the retry never succeeds, teardown still frees the VM ID while the
ITS may hold the cached ITTE. Could it keep the ID, as it does when
its own teardown fails? Only misbehaving hardware gets here (also
Sashiko).

[...]
> +                } else {
> +                        ret = -EBUSY;
> +                }

This is unreachable, as is the WARN_ON_ONCE() below (the only caller
holds irqbypass's mutex, and irq->hw was checked above). Could both
go?

Thanks!
/fuad

[1] https://lore.kernel.org/all/sashiko-outbox-194187@kernel.org/

Re: [PATCH v2 15/16] KVM: selftests: Add VGICv5 ITS translation cache tests

[Fuad Tabba]

On Mon, 02 Nov 2026 19:44:53 +0000, Sascha Bischoff
<sascha.bischoff@arm.com> wrote:
[...]
> Also cover negative MSI paths by checking that KVM_SIGNAL_MSI rejects a
> message without KVM_MSI_VALID_DEVID and rejects a message whose address
> does not target the GICv5 ITS. Finally, inject once before guest-side
> ITS disable to prove delivery still works, then have VPE0 disable the
> ITS and verify that a cached KVM_SIGNAL_MSI delivery is rejected with
> ENXIO.

The MSI just before the disable is expected to be blocked (IDR1 is
narrowed), and the disabled check runs before any cache lookup, so the
final ENXIO doesn't test the cache. Could the test restore IDR1 and
deliver an MSI first, and the message describe what the last step
checks? (Sashiko [1])

Thanks!
/fuad

[1] https://lore.kernel.org/all/sashiko-outbox-194181@kernel.org/

Re: [PATCH v2 03/16] KVM: arm64: gic-v5: Share CoreSight ID register emulation

[Wei-Lin Chang]

On Mon, Nov 02, 2026 at 07:44:41PM +0000, Sascha Bischoff wrote:
> The GICv5 IRS and ITS expose the same CoreSight identification register
> block. Move the register-value switch into a shared helper and describe
> the contiguous ID register range with one register-region descriptor.
> 
> This keeps the IRS behaviour unchanged while giving the ITS MMIO
> emulation a helper it can reuse when it is added in the following commit.
> 
> Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
> ---
>  arch/arm64/kvm/vgic/vgic-irs-v5.c  | 75 +++---------------------------
>  arch/arm64/kvm/vgic/vgic-v5.c      | 34 ++++++++++++++
>  arch/arm64/kvm/vgic/vgic.h         |  1 +
>  include/linux/irqchip/arm-gic-v5.h |  2 +
>  4 files changed, 44 insertions(+), 68 deletions(-)
> 
> diff --git a/arch/arm64/kvm/vgic/vgic-irs-v5.c b/arch/arm64/kvm/vgic/vgic-irs-v5.c
> index 8d0ad74ebc64a..d0ebed2e1663b 100644
> --- a/arch/arm64/kvm/vgic/vgic-irs-v5.c
> +++ b/arch/arm64/kvm/vgic/vgic-irs-v5.c
> @@ -520,38 +520,9 @@ static void vgic_v5_mmio_write_irs_ist(struct kvm_vcpu *vcpu, gpa_t addr,
>  static unsigned long vgic_v5_coresight_read(struct kvm_vcpu *vcpu,
>  					    gpa_t addr, unsigned int len)
>  {
> -	const size_t offset = addr & (SZ_64K - 1);
> +	u32 offset = addr & (SZ_64K - 1);

nit: I think we can keep the const here?

>  
> -	switch (offset) {
> -	case GICV5_CORESIGHT_DEVARCH:
> -		return GICV5_CORESIGHT_DEVARCH_VAL;
> -	case GICV5_CORESIGHT_PIDR4:
> -		return GICV5_CORESIGHT_PIDR4_JEP106_CONT;
> -	case GICV5_CORESIGHT_PIDR5:
> -		return GICV5_CORESIGHT_PIDR5_RES0;
> -	case GICV5_CORESIGHT_PIDR6:
> -		return GICV5_CORESIGHT_PIDR6_RES0;
> -	case GICV5_CORESIGHT_PIDR7:
> -		return GICV5_CORESIGHT_PIDR7_RES0;
> -	case GICV5_CORESIGHT_PIDR0:
> -		return GICV5_CORESIGHT_PIDR0_PART_0;
> -	case GICV5_CORESIGHT_PIDR1:
> -		return GICV5_CORESIGHT_PIDR1_DES_0_PART_1;
> -	case GICV5_CORESIGHT_PIDR2:
> -		return GICV5_CORESIGHT_PIDR2_DES_1;
> -	case GICV5_CORESIGHT_PIDR3:
> -		return GICV5_CORESIGHT_PIDR3_REVAND_CMOD;
> -	case GICV5_CORESIGHT_CIDR0:
> -		return GICV5_CORESIGHT_CIDR0_VAL;
> -	case GICV5_CORESIGHT_CIDR1:
> -		return GICV5_CORESIGHT_CIDR1_VAL;
> -	case GICV5_CORESIGHT_CIDR2:
> -		return GICV5_CORESIGHT_CIDR2_VAL;
> -	case GICV5_CORESIGHT_CIDR3:
> -		return GICV5_CORESIGHT_CIDR3_VAL;
> -	default:
> -		return 0;
> -	}
> +	return vgic_v5_handle_guest_coresight_idreg_read(offset);
>  }
>  
>  static unsigned long vgic_v5_mmio_uaccess_read_irs_status(struct kvm_vcpu *vcpu,
> @@ -903,45 +874,13 @@ static const struct vgic_register_region vgic_v5_irs_registers[] = {
>  				  vgic_mmio_read_raz, vgic_mmio_write_wi, 8,
>  				  VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
>  
> -	/* CoreSight identification registers */
> +	/* Split the ranges so KVM_HAS_DEVICE_ATTR only reports actual registers. */

Would "Split the CoreSight range into two to account for the gap
between DEVARCH and PIDR/CIDR registers." be clearer?

Thanks,
Wei-Lin Chang

>  	REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_DEVARCH,
> -				  vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
> -				  VGIC_ACCESS_32bit),
> +				  vgic_v5_coresight_read, vgic_mmio_write_wi,
> +				  4, VGIC_ACCESS_32bit),
>  	REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR4,
> -				  vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
> -				  VGIC_ACCESS_32bit),
> -	REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR5,
> -				  vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
> -				  VGIC_ACCESS_32bit),
> -	REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR6,
> -				  vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
> -				  VGIC_ACCESS_32bit),
> -	REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR7,
> -				  vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
> -				  VGIC_ACCESS_32bit),
> -	REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR0,
> -				  vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
> -				  VGIC_ACCESS_32bit),
> -	REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR1,
> -				  vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
> -				  VGIC_ACCESS_32bit),
> -	REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR2,
> -				  vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
> -				  VGIC_ACCESS_32bit),
> -	REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR3,
> -				  vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
> -				  VGIC_ACCESS_32bit),
> -	REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_CIDR0,
> -				  vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
> -				  VGIC_ACCESS_32bit),
> -	REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_CIDR1,
> -				  vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
> -				  VGIC_ACCESS_32bit),
> -	REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_CIDR2,
> -				  vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
> -				  VGIC_ACCESS_32bit),
> -	REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_CIDR3,
> -				  vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
> +				  vgic_v5_coresight_read, vgic_mmio_write_wi,
> +				  GICV5_CORESIGHT_PIDR_CIDR_SIZE,
>  				  VGIC_ACCESS_32bit),
>  };
>  

[...]