[PATCH v5 0/2] RISC-V: KVM: Convert HGEI management to fully per-HART

[PATCH v5 0/2] RISC-V: KVM: Convert HGEI management to fully per-HART

[Anup Patel]

From: "Guo Ren (Alibaba DAMO Academy)" <guoren@kernel.org>

This short series converts RISC-V KVM AIA's Hypervisor Guest External
Interrupt (HGEI) line management from a global "one-size-fits-all"
model to a fully per-HART (per-CPU) model. It also performs the
corresponding changes in the IMSIC irqchip driver.

The motivation is to properly support heterogeneous RISC-V SoCs
(big.LITTLE, multi-vendor core mixes) where different HARTs may
expose different numbers of guest interrupt files / HGEIE bits.

After this series, HGEI allocation, freeing, and interrupt delivery
are completely per-HART and there are no remaining global assumptions
about number of guest interrupt files.

Changelog:
V5:
 - Avoid re-initializing hgctrl->lock in kvm_riscv_aia_enable()
 - Setup free_bitmap only once in kvm_riscv_aia_enable()
 - Make kvm_riscv_aia_nr_hgei so that it can be concurrently
   updated from kvm_riscv_aia_enable()

V4:
 - Don't modify kvm_riscv_aia_nr_hgei in kvm_riscv_aia_enable()
   since we use kvm_riscv_aia_nr_hgei mostly for HGEI enable/disable
   checks.
 - Use GENMASK_ULL() instead of `BIT(hgctrl->nr_hgei + 1) - 1`
 - Always setup SGEI interrupt irrespective of kvm_riscv_aia_nr_hgei

V3:
 - Move PATCH3 as first patch and kept the nr_guest_files in
   global config as the number of guest files across all CPUs
 - Droped original PATCH2 and squashed original PATCH4 into
   original PATCH1 which is now PATCH2.
 - Addressed comments in original PATCH1.

V2:
 - Add per-HART IMSIC guest files to compute final HGEI count.
 - Add min(local->nr_guest_files, nr_guest_files);

V1:
 - https://lore.kernel.org/kvm-riscv/20260421145451.1597930-1-guoren@kernel.org/

Guo Ren (Alibaba DAMO Academy) (2):
  irqchip/riscv-imsic: Add nr_guest_files in per-HART local config
  RISC-V: KVM: AIA: Make HGEI number management fully per-CPU

 arch/riscv/include/asm/kvm_aia.h        |  2 +-
 arch/riscv/kvm/aia.c                    | 88 +++++++++++++++----------
 arch/riscv/kvm/aia_device.c             |  4 +-
 arch/riscv/kvm/main.c                   |  8 +--
 drivers/irqchip/irq-riscv-imsic-state.c |  9 ++-
 include/linux/irqchip/riscv-imsic.h     |  5 +-
 6 files changed, 68 insertions(+), 48 deletions(-)

-- 
2.43.0


-- 
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

[PATCH v5 0/2] RISC-V: KVM: Convert HGEI management to fully per-HART

[Anup Patel]

From: "Guo Ren (Alibaba DAMO Academy)" <guoren@kernel.org>

This short series converts RISC-V KVM AIA's Hypervisor Guest External
Interrupt (HGEI) line management from a global "one-size-fits-all"
model to a fully per-HART (per-CPU) model. It also performs the
corresponding changes in the IMSIC irqchip driver.

The motivation is to properly support heterogeneous RISC-V SoCs
(big.LITTLE, multi-vendor core mixes) where different HARTs may
expose different numbers of guest interrupt files / HGEIE bits.

After this series, HGEI allocation, freeing, and interrupt delivery
are completely per-HART and there are no remaining global assumptions
about number of guest interrupt files.

Changelog:
V5:
 - Avoid re-initializing hgctrl->lock in kvm_riscv_aia_enable()
 - Setup free_bitmap only once in kvm_riscv_aia_enable()
 - Make kvm_riscv_aia_nr_hgei so that it can be concurrently
   updated from kvm_riscv_aia_enable()

V4:
 - Don't modify kvm_riscv_aia_nr_hgei in kvm_riscv_aia_enable()
   since we use kvm_riscv_aia_nr_hgei mostly for HGEI enable/disable
   checks.
 - Use GENMASK_ULL() instead of `BIT(hgctrl->nr_hgei + 1) - 1`
 - Always setup SGEI interrupt irrespective of kvm_riscv_aia_nr_hgei

V3:
 - Move PATCH3 as first patch and kept the nr_guest_files in
   global config as the number of guest files across all CPUs
 - Droped original PATCH2 and squashed original PATCH4 into
   original PATCH1 which is now PATCH2.
 - Addressed comments in original PATCH1.

V2:
 - Add per-HART IMSIC guest files to compute final HGEI count.
 - Add min(local->nr_guest_files, nr_guest_files);

V1:
 - https://lore.kernel.org/kvm-riscv/20260421145451.1597930-1-guoren@kernel.org/

Guo Ren (Alibaba DAMO Academy) (2):
  irqchip/riscv-imsic: Add nr_guest_files in per-HART local config
  RISC-V: KVM: AIA: Make HGEI number management fully per-CPU

 arch/riscv/include/asm/kvm_aia.h        |  2 +-
 arch/riscv/kvm/aia.c                    | 88 +++++++++++++++----------
 arch/riscv/kvm/aia_device.c             |  4 +-
 arch/riscv/kvm/main.c                   |  8 +--
 drivers/irqchip/irq-riscv-imsic-state.c |  9 ++-
 include/linux/irqchip/riscv-imsic.h     |  5 +-
 6 files changed, 68 insertions(+), 48 deletions(-)

-- 
2.43.0

[PATCH v5 0/2] RISC-V: KVM: Convert HGEI management to fully per-HART

[Anup Patel]

From: "Guo Ren (Alibaba DAMO Academy)" <guoren@kernel.org>

This short series converts RISC-V KVM AIA's Hypervisor Guest External
Interrupt (HGEI) line management from a global "one-size-fits-all"
model to a fully per-HART (per-CPU) model. It also performs the
corresponding changes in the IMSIC irqchip driver.

The motivation is to properly support heterogeneous RISC-V SoCs
(big.LITTLE, multi-vendor core mixes) where different HARTs may
expose different numbers of guest interrupt files / HGEIE bits.

After this series, HGEI allocation, freeing, and interrupt delivery
are completely per-HART and there are no remaining global assumptions
about number of guest interrupt files.

Changelog:
V5:
 - Avoid re-initializing hgctrl->lock in kvm_riscv_aia_enable()
 - Setup free_bitmap only once in kvm_riscv_aia_enable()
 - Make kvm_riscv_aia_nr_hgei so that it can be concurrently
   updated from kvm_riscv_aia_enable()

V4:
 - Don't modify kvm_riscv_aia_nr_hgei in kvm_riscv_aia_enable()
   since we use kvm_riscv_aia_nr_hgei mostly for HGEI enable/disable
   checks.
 - Use GENMASK_ULL() instead of `BIT(hgctrl->nr_hgei + 1) - 1`
 - Always setup SGEI interrupt irrespective of kvm_riscv_aia_nr_hgei

V3:
 - Move PATCH3 as first patch and kept the nr_guest_files in
   global config as the number of guest files across all CPUs
 - Droped original PATCH2 and squashed original PATCH4 into
   original PATCH1 which is now PATCH2.
 - Addressed comments in original PATCH1.

V2:
 - Add per-HART IMSIC guest files to compute final HGEI count.
 - Add min(local->nr_guest_files, nr_guest_files);

V1:
 - https://lore.kernel.org/kvm-riscv/20260421145451.1597930-1-guoren@kernel.org/

Guo Ren (Alibaba DAMO Academy) (2):
  irqchip/riscv-imsic: Add nr_guest_files in per-HART local config
  RISC-V: KVM: AIA: Make HGEI number management fully per-CPU

 arch/riscv/include/asm/kvm_aia.h        |  2 +-
 arch/riscv/kvm/aia.c                    | 88 +++++++++++++++----------
 arch/riscv/kvm/aia_device.c             |  4 +-
 arch/riscv/kvm/main.c                   |  8 +--
 drivers/irqchip/irq-riscv-imsic-state.c |  9 ++-
 include/linux/irqchip/riscv-imsic.h     |  5 +-
 6 files changed, 68 insertions(+), 48 deletions(-)

-- 
2.43.0


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

[PATCH v5 1/2] irqchip/riscv-imsic: Add nr_guest_files in per-HART local config

[Anup Patel]

From: "Guo Ren (Alibaba DAMO Academy)" <guoren@kernel.org>

Add nr_guest_files in per-HART local config to represent the number of
guest files available on a particular HART whereas the nr_guest_files
in the global config represents the number of guest files available
across all HARTs.

This allows KVM RISC-V to use nr_guest_files from per-HART local
config for asymmetric big.Little systems.

Signed-off-by: Guo Ren (Alibaba DAMO Academy) <guoren@kernel.org>
Acked-by: Thomas Gleixner <tglx@kernel.org>
Signed-off-by: Anup Patel <anup.patel@oss.qualcomm.com>
---
 drivers/irqchip/irq-riscv-imsic-state.c | 9 ++++-----
 include/linux/irqchip/riscv-imsic.h     | 5 ++++-
 2 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/drivers/irqchip/irq-riscv-imsic-state.c b/drivers/irqchip/irq-riscv-imsic-state.c
index e3ed874d89e7..b8d1bbbf42f7 100644
--- a/drivers/irqchip/irq-riscv-imsic-state.c
+++ b/drivers/irqchip/irq-riscv-imsic-state.c
@@ -920,13 +920,12 @@ int __init imsic_setup_state(struct fwnode_handle *fwnode, void *opaque)
 		local->msi_va = mmios_va[index] + reloff;
 
 		/*
-		 * KVM uses global->nr_guest_files to determine the available guest
-		 * interrupt files on each CPU. Take the minimum number of guest
-		 * interrupt files across all CPUs to avoid KVM incorrectly allocating
-		 * an unexisted or unmapped guest interrupt file on some CPUs.
+		 * KVM uses both local->nr_guest_files and global->nr_guest_files
+		 * to determine the available guest interrupt files on each CPU.
 		 */
 		nr_guest_files = (resource_size(&mmios[index]) - reloff) / IMSIC_MMIO_PAGE_SZ - 1;
-		global->nr_guest_files = min(global->nr_guest_files, nr_guest_files);
+		local->nr_guest_files = min((BIT(global->guest_index_bits) - 1), nr_guest_files);
+		global->nr_guest_files = min(global->nr_guest_files, local->nr_guest_files);
 
 		nr_handlers++;
 	}
diff --git a/include/linux/irqchip/riscv-imsic.h b/include/linux/irqchip/riscv-imsic.h
index 4b348836de7a..61af3a5bea09 100644
--- a/include/linux/irqchip/riscv-imsic.h
+++ b/include/linux/irqchip/riscv-imsic.h
@@ -40,6 +40,9 @@
 struct imsic_local_config {
 	phys_addr_t				msi_pa;
 	void __iomem				*msi_va;
+
+	/* Number of guest interrupt files per-HART */
+	u32					nr_guest_files;
 };
 
 struct imsic_global_config {
@@ -68,7 +71,7 @@ struct imsic_global_config {
 	/* Number of guest interrupt identities */
 	u32					nr_guest_ids;
 
-	/* Number of guest interrupt files per core */
+	/* Number of guest interrupt files across all HARTs */
 	u32					nr_guest_files;
 
 	/* Per-CPU IMSIC addresses */
-- 
2.43.0


-- 
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

[PATCH v5 1/2] irqchip/riscv-imsic: Add nr_guest_files in per-HART local config

[Anup Patel]

From: "Guo Ren (Alibaba DAMO Academy)" <guoren@kernel.org>

Add nr_guest_files in per-HART local config to represent the number of
guest files available on a particular HART whereas the nr_guest_files
in the global config represents the number of guest files available
across all HARTs.

This allows KVM RISC-V to use nr_guest_files from per-HART local
config for asymmetric big.Little systems.

Signed-off-by: Guo Ren (Alibaba DAMO Academy) <guoren@kernel.org>
Acked-by: Thomas Gleixner <tglx@kernel.org>
Signed-off-by: Anup Patel <anup.patel@oss.qualcomm.com>
---
 drivers/irqchip/irq-riscv-imsic-state.c | 9 ++++-----
 include/linux/irqchip/riscv-imsic.h     | 5 ++++-
 2 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/drivers/irqchip/irq-riscv-imsic-state.c b/drivers/irqchip/irq-riscv-imsic-state.c
index e3ed874d89e7..b8d1bbbf42f7 100644
--- a/drivers/irqchip/irq-riscv-imsic-state.c
+++ b/drivers/irqchip/irq-riscv-imsic-state.c
@@ -920,13 +920,12 @@ int __init imsic_setup_state(struct fwnode_handle *fwnode, void *opaque)
 		local->msi_va = mmios_va[index] + reloff;
 
 		/*
-		 * KVM uses global->nr_guest_files to determine the available guest
-		 * interrupt files on each CPU. Take the minimum number of guest
-		 * interrupt files across all CPUs to avoid KVM incorrectly allocating
-		 * an unexisted or unmapped guest interrupt file on some CPUs.
+		 * KVM uses both local->nr_guest_files and global->nr_guest_files
+		 * to determine the available guest interrupt files on each CPU.
 		 */
 		nr_guest_files = (resource_size(&mmios[index]) - reloff) / IMSIC_MMIO_PAGE_SZ - 1;
-		global->nr_guest_files = min(global->nr_guest_files, nr_guest_files);
+		local->nr_guest_files = min((BIT(global->guest_index_bits) - 1), nr_guest_files);
+		global->nr_guest_files = min(global->nr_guest_files, local->nr_guest_files);
 
 		nr_handlers++;
 	}
diff --git a/include/linux/irqchip/riscv-imsic.h b/include/linux/irqchip/riscv-imsic.h
index 4b348836de7a..61af3a5bea09 100644
--- a/include/linux/irqchip/riscv-imsic.h
+++ b/include/linux/irqchip/riscv-imsic.h
@@ -40,6 +40,9 @@
 struct imsic_local_config {
 	phys_addr_t				msi_pa;
 	void __iomem				*msi_va;
+
+	/* Number of guest interrupt files per-HART */
+	u32					nr_guest_files;
 };
 
 struct imsic_global_config {
@@ -68,7 +71,7 @@ struct imsic_global_config {
 	/* Number of guest interrupt identities */
 	u32					nr_guest_ids;
 
-	/* Number of guest interrupt files per core */
+	/* Number of guest interrupt files across all HARTs */
 	u32					nr_guest_files;
 
 	/* Per-CPU IMSIC addresses */
-- 
2.43.0

[PATCH v5 1/2] irqchip/riscv-imsic: Add nr_guest_files in per-HART local config

[Anup Patel]

From: "Guo Ren (Alibaba DAMO Academy)" <guoren@kernel.org>

Add nr_guest_files in per-HART local config to represent the number of
guest files available on a particular HART whereas the nr_guest_files
in the global config represents the number of guest files available
across all HARTs.

This allows KVM RISC-V to use nr_guest_files from per-HART local
config for asymmetric big.Little systems.

Signed-off-by: Guo Ren (Alibaba DAMO Academy) <guoren@kernel.org>
Acked-by: Thomas Gleixner <tglx@kernel.org>
Signed-off-by: Anup Patel <anup.patel@oss.qualcomm.com>
---
 drivers/irqchip/irq-riscv-imsic-state.c | 9 ++++-----
 include/linux/irqchip/riscv-imsic.h     | 5 ++++-
 2 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/drivers/irqchip/irq-riscv-imsic-state.c b/drivers/irqchip/irq-riscv-imsic-state.c
index e3ed874d89e7..b8d1bbbf42f7 100644
--- a/drivers/irqchip/irq-riscv-imsic-state.c
+++ b/drivers/irqchip/irq-riscv-imsic-state.c
@@ -920,13 +920,12 @@ int __init imsic_setup_state(struct fwnode_handle *fwnode, void *opaque)
 		local->msi_va = mmios_va[index] + reloff;
 
 		/*
-		 * KVM uses global->nr_guest_files to determine the available guest
-		 * interrupt files on each CPU. Take the minimum number of guest
-		 * interrupt files across all CPUs to avoid KVM incorrectly allocating
-		 * an unexisted or unmapped guest interrupt file on some CPUs.
+		 * KVM uses both local->nr_guest_files and global->nr_guest_files
+		 * to determine the available guest interrupt files on each CPU.
 		 */
 		nr_guest_files = (resource_size(&mmios[index]) - reloff) / IMSIC_MMIO_PAGE_SZ - 1;
-		global->nr_guest_files = min(global->nr_guest_files, nr_guest_files);
+		local->nr_guest_files = min((BIT(global->guest_index_bits) - 1), nr_guest_files);
+		global->nr_guest_files = min(global->nr_guest_files, local->nr_guest_files);
 
 		nr_handlers++;
 	}
diff --git a/include/linux/irqchip/riscv-imsic.h b/include/linux/irqchip/riscv-imsic.h
index 4b348836de7a..61af3a5bea09 100644
--- a/include/linux/irqchip/riscv-imsic.h
+++ b/include/linux/irqchip/riscv-imsic.h
@@ -40,6 +40,9 @@
 struct imsic_local_config {
 	phys_addr_t				msi_pa;
 	void __iomem				*msi_va;
+
+	/* Number of guest interrupt files per-HART */
+	u32					nr_guest_files;
 };
 
 struct imsic_global_config {
@@ -68,7 +71,7 @@ struct imsic_global_config {
 	/* Number of guest interrupt identities */
 	u32					nr_guest_ids;
 
-	/* Number of guest interrupt files per core */
+	/* Number of guest interrupt files across all HARTs */
 	u32					nr_guest_files;
 
 	/* Per-CPU IMSIC addresses */
-- 
2.43.0


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

[PATCH v5 2/2] RISC-V: KVM: AIA: Make HGEI number management fully per-CPU

[Anup Patel]

From: "Guo Ren (Alibaba DAMO Academy)" <guoren@kernel.org>

Previously, the number of Hypervisor Guest External Interrupt (HGEI)
lines was stored in a single global variable `kvm_riscv_aia_nr_hgei`
and assumed to be the same for all HARTs. This assumption does not
hold on heterogeneous RISC-V SoCs where different cores may expose
different HGEIE CSR widths.

Introduce `nr_hgei` field into the per-CPU `struct aia_hgei_control`
and probe the actual supported HGEI count for the current HART in
`kvm_riscv_aia_enable()` using the standard RISC-V CSR probe technique:

    csr_write(CSR_HGEIE, -1UL);
    nr = fls_long(csr_read(CSR_HGEIE));
    if (nr)
        nr--;

All HGEI allocation, free and disable paths (`kvm_riscv_aia_free_hgei()`,
`kvm_riscv_aia_disable()`, etc.) now use the per-CPU value instead of
the global one.

The global `kvm_riscv_aia_nr_hgei` now represents the minimum number
of HGEI lines across HARTs and can be used to check whether HGEI
support is available or not.

This makes KVM AIA robust on big.LITTLE-style asymmetric platforms.

Signed-off-by: Guo Ren (Alibaba DAMO Academy) <guoren@kernel.org>
Signed-off-by: Anup Patel <anup.patel@oss.qualcomm.com>
---
 arch/riscv/include/asm/kvm_aia.h |  2 +-
 arch/riscv/kvm/aia.c             | 88 +++++++++++++++++++-------------
 arch/riscv/kvm/aia_device.c      |  4 +-
 arch/riscv/kvm/main.c            |  8 +--
 4 files changed, 60 insertions(+), 42 deletions(-)

diff --git a/arch/riscv/include/asm/kvm_aia.h b/arch/riscv/include/asm/kvm_aia.h
index b04ecdd1a860..c67ec5ac0a14 100644
--- a/arch/riscv/include/asm/kvm_aia.h
+++ b/arch/riscv/include/asm/kvm_aia.h
@@ -79,7 +79,7 @@ struct kvm_vcpu_aia {
 
 #define irqchip_in_kernel(k)		((k)->arch.aia.in_kernel)
 
-extern unsigned int kvm_riscv_aia_nr_hgei;
+extern atomic_t kvm_riscv_aia_nr_hgei;
 extern unsigned int kvm_riscv_aia_max_ids;
 DECLARE_STATIC_KEY_FALSE(kvm_riscv_aia_available);
 #define kvm_riscv_aia_available() \
diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
index 5ec503288555..bafb009c5ce5 100644
--- a/arch/riscv/kvm/aia.c
+++ b/arch/riscv/kvm/aia.c
@@ -21,13 +21,15 @@
 
 struct aia_hgei_control {
 	raw_spinlock_t lock;
+	bool free_bitmap_initialized;
 	unsigned long free_bitmap;
 	struct kvm_vcpu *owners[BITS_PER_LONG];
+	unsigned int nr_hgei;
 };
 static DEFINE_PER_CPU(struct aia_hgei_control, aia_hgei);
 static int hgei_parent_irq;
 
-unsigned int kvm_riscv_aia_nr_hgei;
+atomic_t kvm_riscv_aia_nr_hgei;
 unsigned int kvm_riscv_aia_max_ids;
 DEFINE_STATIC_KEY_FALSE(kvm_riscv_aia_available);
 
@@ -452,7 +454,7 @@ void kvm_riscv_aia_free_hgei(int cpu, int hgei)
 
 	raw_spin_lock_irqsave(&hgctrl->lock, flags);
 
-	if (hgei > 0 && hgei <= kvm_riscv_aia_nr_hgei) {
+	if (hgei > 0 && hgei <= hgctrl->nr_hgei) {
 		if (!(hgctrl->free_bitmap & BIT(hgei))) {
 			hgctrl->free_bitmap |= BIT(hgei);
 			hgctrl->owners[hgei] = NULL;
@@ -486,26 +488,18 @@ static irqreturn_t hgei_interrupt(int irq, void *dev_id)
 
 static int aia_hgei_init(void)
 {
-	int cpu, rc;
-	struct irq_domain *domain;
 	struct aia_hgei_control *hgctrl;
+	struct irq_domain *domain;
+	int cpu, rc;
 
 	/* Initialize per-CPU guest external interrupt line management */
 	for_each_possible_cpu(cpu) {
 		hgctrl = per_cpu_ptr(&aia_hgei, cpu);
 		raw_spin_lock_init(&hgctrl->lock);
-		if (kvm_riscv_aia_nr_hgei) {
-			hgctrl->free_bitmap =
-				BIT(kvm_riscv_aia_nr_hgei + 1) - 1;
-			hgctrl->free_bitmap &= ~BIT(0);
-		} else
-			hgctrl->free_bitmap = 0;
+		hgctrl->free_bitmap_initialized = false;
+		hgctrl->free_bitmap = 0;
 	}
 
-	/* Skip SGEI interrupt setup for zero guest external interrupts */
-	if (!kvm_riscv_aia_nr_hgei)
-		goto skip_sgei_interrupt;
-
 	/* Find INTC irq domain */
 	domain = irq_find_matching_fwnode(riscv_get_intc_hwnode(),
 					  DOMAIN_BUS_ANY);
@@ -529,25 +523,60 @@ static int aia_hgei_init(void)
 		return rc;
 	}
 
-skip_sgei_interrupt:
 	return 0;
 }
 
 static void aia_hgei_exit(void)
 {
-	/* Do nothing for zero guest external interrupts */
-	if (!kvm_riscv_aia_nr_hgei)
-		return;
-
 	/* Free per-CPU SGEI interrupt */
 	free_percpu_irq(hgei_parent_irq, &aia_hgei);
 }
 
 void kvm_riscv_aia_enable(void)
 {
+	const struct imsic_global_config *gc;
+	const struct imsic_local_config *lc;
+	struct aia_hgei_control *hgctrl;
+	unsigned long flags;
+	int aia_nr_hgei;
+
 	if (!kvm_riscv_aia_available())
 		return;
 
+	gc = imsic_get_global_config();
+	lc = (gc) ? this_cpu_ptr(gc->local) : NULL;
+	hgctrl = this_cpu_ptr(&aia_hgei);
+
+	/* Figure-out number of bits in HGEIE */
+	csr_write(CSR_HGEIE, -1UL);
+	hgctrl->nr_hgei = fls_long(csr_read(CSR_HGEIE));
+	csr_write(CSR_HGEIE, 0);
+	if (hgctrl->nr_hgei)
+		hgctrl->nr_hgei--;
+
+	/*
+	 * Number of usable per-HART HGEI lines should be minimum of
+	 * per-HART IMSIC guest files and number of bits in HGEIE.
+	 */
+	if (lc)
+		hgctrl->nr_hgei = min((ulong)hgctrl->nr_hgei, lc->nr_guest_files);
+	else
+		hgctrl->nr_hgei = 0;
+
+	/* Update the number of IMSIC guest files across all HARTs */
+	aia_nr_hgei = atomic_read(&kvm_riscv_aia_nr_hgei);
+	do {
+		if (aia_nr_hgei <= hgctrl->nr_hgei)
+			break;
+	} while (!atomic_try_cmpxchg(&kvm_riscv_aia_nr_hgei, &aia_nr_hgei, hgctrl->nr_hgei));
+
+	raw_spin_lock_irqsave(&hgctrl->lock, flags);
+	if (!hgctrl->free_bitmap_initialized) {
+		hgctrl->free_bitmap = (hgctrl->nr_hgei) ? GENMASK_ULL(hgctrl->nr_hgei, 1) : 0;
+		hgctrl->free_bitmap_initialized = true;
+	}
+	raw_spin_unlock_irqrestore(&hgctrl->lock, flags);
+
 	csr_write(CSR_HVICTL, aia_hvictl_value(false));
 	csr_write(CSR_HVIPRIO1, 0x0);
 	csr_write(CSR_HVIPRIO2, 0x0);
@@ -588,7 +617,7 @@ void kvm_riscv_aia_disable(void)
 
 	raw_spin_lock_irqsave(&hgctrl->lock, flags);
 
-	for (i = 0; i <= kvm_riscv_aia_nr_hgei; i++) {
+	for (i = 0; i <= hgctrl->nr_hgei; i++) {
 		vcpu = hgctrl->owners[i];
 		if (!vcpu)
 			continue;
@@ -628,26 +657,15 @@ int kvm_riscv_aia_init(void)
 		return -ENODEV;
 	gc = imsic_get_global_config();
 
-	/* Figure-out number of bits in HGEIE */
-	csr_write(CSR_HGEIE, -1UL);
-	kvm_riscv_aia_nr_hgei = fls_long(csr_read(CSR_HGEIE));
-	csr_write(CSR_HGEIE, 0);
-	if (kvm_riscv_aia_nr_hgei)
-		kvm_riscv_aia_nr_hgei--;
-
-	/*
-	 * Number of usable HGEI lines should be minimum of per-HART
-	 * IMSIC guest files and number of bits in HGEIE
-	 */
+	/* Set initial value of IMSIC guest files across all HARTs */
 	if (gc)
-		kvm_riscv_aia_nr_hgei = min((ulong)kvm_riscv_aia_nr_hgei,
-					    gc->nr_guest_files);
+		atomic_set(&kvm_riscv_aia_nr_hgei, gc->nr_guest_files);
 	else
-		kvm_riscv_aia_nr_hgei = 0;
+		atomic_set(&kvm_riscv_aia_nr_hgei, 0);
 
 	/* Find number of guest MSI IDs */
 	kvm_riscv_aia_max_ids = IMSIC_MAX_ID;
-	if (gc && kvm_riscv_aia_nr_hgei)
+	if (gc)
 		kvm_riscv_aia_max_ids = gc->nr_guest_ids + 1;
 
 	/* Initialize guest external interrupt line management */
diff --git a/arch/riscv/kvm/aia_device.c b/arch/riscv/kvm/aia_device.c
index 3d1e81e2a36b..be83c2d5fc30 100644
--- a/arch/riscv/kvm/aia_device.c
+++ b/arch/riscv/kvm/aia_device.c
@@ -71,7 +71,7 @@ static int aia_config(struct kvm *kvm, unsigned long type,
 				 * external interrupts (i.e. non-zero
 				 * VS-level IMSIC pages).
 				 */
-				if (!kvm_riscv_aia_nr_hgei)
+				if (!atomic_read(&kvm_riscv_aia_nr_hgei))
 					return -EINVAL;
 				break;
 			default:
@@ -628,7 +628,7 @@ void kvm_riscv_aia_init_vm(struct kvm *kvm)
 	 */
 
 	/* Initialize default values in AIA global context */
-	aia->mode = (kvm_riscv_aia_nr_hgei) ?
+	aia->mode = (atomic_read(&kvm_riscv_aia_nr_hgei)) ?
 		KVM_DEV_RISCV_AIA_MODE_AUTO : KVM_DEV_RISCV_AIA_MODE_EMUL;
 	aia->nr_ids = kvm_riscv_aia_max_ids - 1;
 	aia->nr_sources = 0;
diff --git a/arch/riscv/kvm/main.c b/arch/riscv/kvm/main.c
index cb8a65273c1f..0924c75100a2 100644
--- a/arch/riscv/kvm/main.c
+++ b/arch/riscv/kvm/main.c
@@ -168,10 +168,6 @@ static int __init riscv_kvm_init(void)
 
 	kvm_info("VMID %ld bits available\n", kvm_riscv_gstage_vmid_bits());
 
-	if (kvm_riscv_aia_available())
-		kvm_info("AIA available with %d guest external interrupts\n",
-			 kvm_riscv_aia_nr_hgei);
-
 	kvm_riscv_setup_vendor_features();
 
 	kvm_register_perf_callbacks();
@@ -182,6 +178,10 @@ static int __init riscv_kvm_init(void)
 		return rc;
 	}
 
+	if (kvm_riscv_aia_available())
+		kvm_info("AIA available with %d guest external interrupts\n",
+			 atomic_read(&kvm_riscv_aia_nr_hgei));
+
 	return 0;
 }
 module_init(riscv_kvm_init);
-- 
2.43.0


-- 
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

[PATCH v5 2/2] RISC-V: KVM: AIA: Make HGEI number management fully per-CPU

[Anup Patel]

From: "Guo Ren (Alibaba DAMO Academy)" <guoren@kernel.org>

Previously, the number of Hypervisor Guest External Interrupt (HGEI)
lines was stored in a single global variable `kvm_riscv_aia_nr_hgei`
and assumed to be the same for all HARTs. This assumption does not
hold on heterogeneous RISC-V SoCs where different cores may expose
different HGEIE CSR widths.

Introduce `nr_hgei` field into the per-CPU `struct aia_hgei_control`
and probe the actual supported HGEI count for the current HART in
`kvm_riscv_aia_enable()` using the standard RISC-V CSR probe technique:

    csr_write(CSR_HGEIE, -1UL);
    nr = fls_long(csr_read(CSR_HGEIE));
    if (nr)
        nr--;

All HGEI allocation, free and disable paths (`kvm_riscv_aia_free_hgei()`,
`kvm_riscv_aia_disable()`, etc.) now use the per-CPU value instead of
the global one.

The global `kvm_riscv_aia_nr_hgei` now represents the minimum number
of HGEI lines across HARTs and can be used to check whether HGEI
support is available or not.

This makes KVM AIA robust on big.LITTLE-style asymmetric platforms.

Signed-off-by: Guo Ren (Alibaba DAMO Academy) <guoren@kernel.org>
Signed-off-by: Anup Patel <anup.patel@oss.qualcomm.com>
---
 arch/riscv/include/asm/kvm_aia.h |  2 +-
 arch/riscv/kvm/aia.c             | 88 +++++++++++++++++++-------------
 arch/riscv/kvm/aia_device.c      |  4 +-
 arch/riscv/kvm/main.c            |  8 +--
 4 files changed, 60 insertions(+), 42 deletions(-)

diff --git a/arch/riscv/include/asm/kvm_aia.h b/arch/riscv/include/asm/kvm_aia.h
index b04ecdd1a860..c67ec5ac0a14 100644
--- a/arch/riscv/include/asm/kvm_aia.h
+++ b/arch/riscv/include/asm/kvm_aia.h
@@ -79,7 +79,7 @@ struct kvm_vcpu_aia {
 
 #define irqchip_in_kernel(k)		((k)->arch.aia.in_kernel)
 
-extern unsigned int kvm_riscv_aia_nr_hgei;
+extern atomic_t kvm_riscv_aia_nr_hgei;
 extern unsigned int kvm_riscv_aia_max_ids;
 DECLARE_STATIC_KEY_FALSE(kvm_riscv_aia_available);
 #define kvm_riscv_aia_available() \
diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
index 5ec503288555..bafb009c5ce5 100644
--- a/arch/riscv/kvm/aia.c
+++ b/arch/riscv/kvm/aia.c
@@ -21,13 +21,15 @@
 
 struct aia_hgei_control {
 	raw_spinlock_t lock;
+	bool free_bitmap_initialized;
 	unsigned long free_bitmap;
 	struct kvm_vcpu *owners[BITS_PER_LONG];
+	unsigned int nr_hgei;
 };
 static DEFINE_PER_CPU(struct aia_hgei_control, aia_hgei);
 static int hgei_parent_irq;
 
-unsigned int kvm_riscv_aia_nr_hgei;
+atomic_t kvm_riscv_aia_nr_hgei;
 unsigned int kvm_riscv_aia_max_ids;
 DEFINE_STATIC_KEY_FALSE(kvm_riscv_aia_available);
 
@@ -452,7 +454,7 @@ void kvm_riscv_aia_free_hgei(int cpu, int hgei)
 
 	raw_spin_lock_irqsave(&hgctrl->lock, flags);
 
-	if (hgei > 0 && hgei <= kvm_riscv_aia_nr_hgei) {
+	if (hgei > 0 && hgei <= hgctrl->nr_hgei) {
 		if (!(hgctrl->free_bitmap & BIT(hgei))) {
 			hgctrl->free_bitmap |= BIT(hgei);
 			hgctrl->owners[hgei] = NULL;
@@ -486,26 +488,18 @@ static irqreturn_t hgei_interrupt(int irq, void *dev_id)
 
 static int aia_hgei_init(void)
 {
-	int cpu, rc;
-	struct irq_domain *domain;
 	struct aia_hgei_control *hgctrl;
+	struct irq_domain *domain;
+	int cpu, rc;
 
 	/* Initialize per-CPU guest external interrupt line management */
 	for_each_possible_cpu(cpu) {
 		hgctrl = per_cpu_ptr(&aia_hgei, cpu);
 		raw_spin_lock_init(&hgctrl->lock);
-		if (kvm_riscv_aia_nr_hgei) {
-			hgctrl->free_bitmap =
-				BIT(kvm_riscv_aia_nr_hgei + 1) - 1;
-			hgctrl->free_bitmap &= ~BIT(0);
-		} else
-			hgctrl->free_bitmap = 0;
+		hgctrl->free_bitmap_initialized = false;
+		hgctrl->free_bitmap = 0;
 	}
 
-	/* Skip SGEI interrupt setup for zero guest external interrupts */
-	if (!kvm_riscv_aia_nr_hgei)
-		goto skip_sgei_interrupt;
-
 	/* Find INTC irq domain */
 	domain = irq_find_matching_fwnode(riscv_get_intc_hwnode(),
 					  DOMAIN_BUS_ANY);
@@ -529,25 +523,60 @@ static int aia_hgei_init(void)
 		return rc;
 	}
 
-skip_sgei_interrupt:
 	return 0;
 }
 
 static void aia_hgei_exit(void)
 {
-	/* Do nothing for zero guest external interrupts */
-	if (!kvm_riscv_aia_nr_hgei)
-		return;
-
 	/* Free per-CPU SGEI interrupt */
 	free_percpu_irq(hgei_parent_irq, &aia_hgei);
 }
 
 void kvm_riscv_aia_enable(void)
 {
+	const struct imsic_global_config *gc;
+	const struct imsic_local_config *lc;
+	struct aia_hgei_control *hgctrl;
+	unsigned long flags;
+	int aia_nr_hgei;
+
 	if (!kvm_riscv_aia_available())
 		return;
 
+	gc = imsic_get_global_config();
+	lc = (gc) ? this_cpu_ptr(gc->local) : NULL;
+	hgctrl = this_cpu_ptr(&aia_hgei);
+
+	/* Figure-out number of bits in HGEIE */
+	csr_write(CSR_HGEIE, -1UL);
+	hgctrl->nr_hgei = fls_long(csr_read(CSR_HGEIE));
+	csr_write(CSR_HGEIE, 0);
+	if (hgctrl->nr_hgei)
+		hgctrl->nr_hgei--;
+
+	/*
+	 * Number of usable per-HART HGEI lines should be minimum of
+	 * per-HART IMSIC guest files and number of bits in HGEIE.
+	 */
+	if (lc)
+		hgctrl->nr_hgei = min((ulong)hgctrl->nr_hgei, lc->nr_guest_files);
+	else
+		hgctrl->nr_hgei = 0;
+
+	/* Update the number of IMSIC guest files across all HARTs */
+	aia_nr_hgei = atomic_read(&kvm_riscv_aia_nr_hgei);
+	do {
+		if (aia_nr_hgei <= hgctrl->nr_hgei)
+			break;
+	} while (!atomic_try_cmpxchg(&kvm_riscv_aia_nr_hgei, &aia_nr_hgei, hgctrl->nr_hgei));
+
+	raw_spin_lock_irqsave(&hgctrl->lock, flags);
+	if (!hgctrl->free_bitmap_initialized) {
+		hgctrl->free_bitmap = (hgctrl->nr_hgei) ? GENMASK_ULL(hgctrl->nr_hgei, 1) : 0;
+		hgctrl->free_bitmap_initialized = true;
+	}
+	raw_spin_unlock_irqrestore(&hgctrl->lock, flags);
+
 	csr_write(CSR_HVICTL, aia_hvictl_value(false));
 	csr_write(CSR_HVIPRIO1, 0x0);
 	csr_write(CSR_HVIPRIO2, 0x0);
@@ -588,7 +617,7 @@ void kvm_riscv_aia_disable(void)
 
 	raw_spin_lock_irqsave(&hgctrl->lock, flags);
 
-	for (i = 0; i <= kvm_riscv_aia_nr_hgei; i++) {
+	for (i = 0; i <= hgctrl->nr_hgei; i++) {
 		vcpu = hgctrl->owners[i];
 		if (!vcpu)
 			continue;
@@ -628,26 +657,15 @@ int kvm_riscv_aia_init(void)
 		return -ENODEV;
 	gc = imsic_get_global_config();
 
-	/* Figure-out number of bits in HGEIE */
-	csr_write(CSR_HGEIE, -1UL);
-	kvm_riscv_aia_nr_hgei = fls_long(csr_read(CSR_HGEIE));
-	csr_write(CSR_HGEIE, 0);
-	if (kvm_riscv_aia_nr_hgei)
-		kvm_riscv_aia_nr_hgei--;
-
-	/*
-	 * Number of usable HGEI lines should be minimum of per-HART
-	 * IMSIC guest files and number of bits in HGEIE
-	 */
+	/* Set initial value of IMSIC guest files across all HARTs */
 	if (gc)
-		kvm_riscv_aia_nr_hgei = min((ulong)kvm_riscv_aia_nr_hgei,
-					    gc->nr_guest_files);
+		atomic_set(&kvm_riscv_aia_nr_hgei, gc->nr_guest_files);
 	else
-		kvm_riscv_aia_nr_hgei = 0;
+		atomic_set(&kvm_riscv_aia_nr_hgei, 0);
 
 	/* Find number of guest MSI IDs */
 	kvm_riscv_aia_max_ids = IMSIC_MAX_ID;
-	if (gc && kvm_riscv_aia_nr_hgei)
+	if (gc)
 		kvm_riscv_aia_max_ids = gc->nr_guest_ids + 1;
 
 	/* Initialize guest external interrupt line management */
diff --git a/arch/riscv/kvm/aia_device.c b/arch/riscv/kvm/aia_device.c
index 3d1e81e2a36b..be83c2d5fc30 100644
--- a/arch/riscv/kvm/aia_device.c
+++ b/arch/riscv/kvm/aia_device.c
@@ -71,7 +71,7 @@ static int aia_config(struct kvm *kvm, unsigned long type,
 				 * external interrupts (i.e. non-zero
 				 * VS-level IMSIC pages).
 				 */
-				if (!kvm_riscv_aia_nr_hgei)
+				if (!atomic_read(&kvm_riscv_aia_nr_hgei))
 					return -EINVAL;
 				break;
 			default:
@@ -628,7 +628,7 @@ void kvm_riscv_aia_init_vm(struct kvm *kvm)
 	 */
 
 	/* Initialize default values in AIA global context */
-	aia->mode = (kvm_riscv_aia_nr_hgei) ?
+	aia->mode = (atomic_read(&kvm_riscv_aia_nr_hgei)) ?
 		KVM_DEV_RISCV_AIA_MODE_AUTO : KVM_DEV_RISCV_AIA_MODE_EMUL;
 	aia->nr_ids = kvm_riscv_aia_max_ids - 1;
 	aia->nr_sources = 0;
diff --git a/arch/riscv/kvm/main.c b/arch/riscv/kvm/main.c
index cb8a65273c1f..0924c75100a2 100644
--- a/arch/riscv/kvm/main.c
+++ b/arch/riscv/kvm/main.c
@@ -168,10 +168,6 @@ static int __init riscv_kvm_init(void)
 
 	kvm_info("VMID %ld bits available\n", kvm_riscv_gstage_vmid_bits());
 
-	if (kvm_riscv_aia_available())
-		kvm_info("AIA available with %d guest external interrupts\n",
-			 kvm_riscv_aia_nr_hgei);
-
 	kvm_riscv_setup_vendor_features();
 
 	kvm_register_perf_callbacks();
@@ -182,6 +178,10 @@ static int __init riscv_kvm_init(void)
 		return rc;
 	}
 
+	if (kvm_riscv_aia_available())
+		kvm_info("AIA available with %d guest external interrupts\n",
+			 atomic_read(&kvm_riscv_aia_nr_hgei));
+
 	return 0;
 }
 module_init(riscv_kvm_init);
-- 
2.43.0

[PATCH v5 2/2] RISC-V: KVM: AIA: Make HGEI number management fully per-CPU

[Anup Patel]

From: "Guo Ren (Alibaba DAMO Academy)" <guoren@kernel.org>

Previously, the number of Hypervisor Guest External Interrupt (HGEI)
lines was stored in a single global variable `kvm_riscv_aia_nr_hgei`
and assumed to be the same for all HARTs. This assumption does not
hold on heterogeneous RISC-V SoCs where different cores may expose
different HGEIE CSR widths.

Introduce `nr_hgei` field into the per-CPU `struct aia_hgei_control`
and probe the actual supported HGEI count for the current HART in
`kvm_riscv_aia_enable()` using the standard RISC-V CSR probe technique:

    csr_write(CSR_HGEIE, -1UL);
    nr = fls_long(csr_read(CSR_HGEIE));
    if (nr)
        nr--;

All HGEI allocation, free and disable paths (`kvm_riscv_aia_free_hgei()`,
`kvm_riscv_aia_disable()`, etc.) now use the per-CPU value instead of
the global one.

The global `kvm_riscv_aia_nr_hgei` now represents the minimum number
of HGEI lines across HARTs and can be used to check whether HGEI
support is available or not.

This makes KVM AIA robust on big.LITTLE-style asymmetric platforms.

Signed-off-by: Guo Ren (Alibaba DAMO Academy) <guoren@kernel.org>
Signed-off-by: Anup Patel <anup.patel@oss.qualcomm.com>
---
 arch/riscv/include/asm/kvm_aia.h |  2 +-
 arch/riscv/kvm/aia.c             | 88 +++++++++++++++++++-------------
 arch/riscv/kvm/aia_device.c      |  4 +-
 arch/riscv/kvm/main.c            |  8 +--
 4 files changed, 60 insertions(+), 42 deletions(-)

diff --git a/arch/riscv/include/asm/kvm_aia.h b/arch/riscv/include/asm/kvm_aia.h
index b04ecdd1a860..c67ec5ac0a14 100644
--- a/arch/riscv/include/asm/kvm_aia.h
+++ b/arch/riscv/include/asm/kvm_aia.h
@@ -79,7 +79,7 @@ struct kvm_vcpu_aia {
 
 #define irqchip_in_kernel(k)		((k)->arch.aia.in_kernel)
 
-extern unsigned int kvm_riscv_aia_nr_hgei;
+extern atomic_t kvm_riscv_aia_nr_hgei;
 extern unsigned int kvm_riscv_aia_max_ids;
 DECLARE_STATIC_KEY_FALSE(kvm_riscv_aia_available);
 #define kvm_riscv_aia_available() \
diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
index 5ec503288555..bafb009c5ce5 100644
--- a/arch/riscv/kvm/aia.c
+++ b/arch/riscv/kvm/aia.c
@@ -21,13 +21,15 @@
 
 struct aia_hgei_control {
 	raw_spinlock_t lock;
+	bool free_bitmap_initialized;
 	unsigned long free_bitmap;
 	struct kvm_vcpu *owners[BITS_PER_LONG];
+	unsigned int nr_hgei;
 };
 static DEFINE_PER_CPU(struct aia_hgei_control, aia_hgei);
 static int hgei_parent_irq;
 
-unsigned int kvm_riscv_aia_nr_hgei;
+atomic_t kvm_riscv_aia_nr_hgei;
 unsigned int kvm_riscv_aia_max_ids;
 DEFINE_STATIC_KEY_FALSE(kvm_riscv_aia_available);
 
@@ -452,7 +454,7 @@ void kvm_riscv_aia_free_hgei(int cpu, int hgei)
 
 	raw_spin_lock_irqsave(&hgctrl->lock, flags);
 
-	if (hgei > 0 && hgei <= kvm_riscv_aia_nr_hgei) {
+	if (hgei > 0 && hgei <= hgctrl->nr_hgei) {
 		if (!(hgctrl->free_bitmap & BIT(hgei))) {
 			hgctrl->free_bitmap |= BIT(hgei);
 			hgctrl->owners[hgei] = NULL;
@@ -486,26 +488,18 @@ static irqreturn_t hgei_interrupt(int irq, void *dev_id)
 
 static int aia_hgei_init(void)
 {
-	int cpu, rc;
-	struct irq_domain *domain;
 	struct aia_hgei_control *hgctrl;
+	struct irq_domain *domain;
+	int cpu, rc;
 
 	/* Initialize per-CPU guest external interrupt line management */
 	for_each_possible_cpu(cpu) {
 		hgctrl = per_cpu_ptr(&aia_hgei, cpu);
 		raw_spin_lock_init(&hgctrl->lock);
-		if (kvm_riscv_aia_nr_hgei) {
-			hgctrl->free_bitmap =
-				BIT(kvm_riscv_aia_nr_hgei + 1) - 1;
-			hgctrl->free_bitmap &= ~BIT(0);
-		} else
-			hgctrl->free_bitmap = 0;
+		hgctrl->free_bitmap_initialized = false;
+		hgctrl->free_bitmap = 0;
 	}
 
-	/* Skip SGEI interrupt setup for zero guest external interrupts */
-	if (!kvm_riscv_aia_nr_hgei)
-		goto skip_sgei_interrupt;
-
 	/* Find INTC irq domain */
 	domain = irq_find_matching_fwnode(riscv_get_intc_hwnode(),
 					  DOMAIN_BUS_ANY);
@@ -529,25 +523,60 @@ static int aia_hgei_init(void)
 		return rc;
 	}
 
-skip_sgei_interrupt:
 	return 0;
 }
 
 static void aia_hgei_exit(void)
 {
-	/* Do nothing for zero guest external interrupts */
-	if (!kvm_riscv_aia_nr_hgei)
-		return;
-
 	/* Free per-CPU SGEI interrupt */
 	free_percpu_irq(hgei_parent_irq, &aia_hgei);
 }
 
 void kvm_riscv_aia_enable(void)
 {
+	const struct imsic_global_config *gc;
+	const struct imsic_local_config *lc;
+	struct aia_hgei_control *hgctrl;
+	unsigned long flags;
+	int aia_nr_hgei;
+
 	if (!kvm_riscv_aia_available())
 		return;
 
+	gc = imsic_get_global_config();
+	lc = (gc) ? this_cpu_ptr(gc->local) : NULL;
+	hgctrl = this_cpu_ptr(&aia_hgei);
+
+	/* Figure-out number of bits in HGEIE */
+	csr_write(CSR_HGEIE, -1UL);
+	hgctrl->nr_hgei = fls_long(csr_read(CSR_HGEIE));
+	csr_write(CSR_HGEIE, 0);
+	if (hgctrl->nr_hgei)
+		hgctrl->nr_hgei--;
+
+	/*
+	 * Number of usable per-HART HGEI lines should be minimum of
+	 * per-HART IMSIC guest files and number of bits in HGEIE.
+	 */
+	if (lc)
+		hgctrl->nr_hgei = min((ulong)hgctrl->nr_hgei, lc->nr_guest_files);
+	else
+		hgctrl->nr_hgei = 0;
+
+	/* Update the number of IMSIC guest files across all HARTs */
+	aia_nr_hgei = atomic_read(&kvm_riscv_aia_nr_hgei);
+	do {
+		if (aia_nr_hgei <= hgctrl->nr_hgei)
+			break;
+	} while (!atomic_try_cmpxchg(&kvm_riscv_aia_nr_hgei, &aia_nr_hgei, hgctrl->nr_hgei));
+
+	raw_spin_lock_irqsave(&hgctrl->lock, flags);
+	if (!hgctrl->free_bitmap_initialized) {
+		hgctrl->free_bitmap = (hgctrl->nr_hgei) ? GENMASK_ULL(hgctrl->nr_hgei, 1) : 0;
+		hgctrl->free_bitmap_initialized = true;
+	}
+	raw_spin_unlock_irqrestore(&hgctrl->lock, flags);
+
 	csr_write(CSR_HVICTL, aia_hvictl_value(false));
 	csr_write(CSR_HVIPRIO1, 0x0);
 	csr_write(CSR_HVIPRIO2, 0x0);
@@ -588,7 +617,7 @@ void kvm_riscv_aia_disable(void)
 
 	raw_spin_lock_irqsave(&hgctrl->lock, flags);
 
-	for (i = 0; i <= kvm_riscv_aia_nr_hgei; i++) {
+	for (i = 0; i <= hgctrl->nr_hgei; i++) {
 		vcpu = hgctrl->owners[i];
 		if (!vcpu)
 			continue;
@@ -628,26 +657,15 @@ int kvm_riscv_aia_init(void)
 		return -ENODEV;
 	gc = imsic_get_global_config();
 
-	/* Figure-out number of bits in HGEIE */
-	csr_write(CSR_HGEIE, -1UL);
-	kvm_riscv_aia_nr_hgei = fls_long(csr_read(CSR_HGEIE));
-	csr_write(CSR_HGEIE, 0);
-	if (kvm_riscv_aia_nr_hgei)
-		kvm_riscv_aia_nr_hgei--;
-
-	/*
-	 * Number of usable HGEI lines should be minimum of per-HART
-	 * IMSIC guest files and number of bits in HGEIE
-	 */
+	/* Set initial value of IMSIC guest files across all HARTs */
 	if (gc)
-		kvm_riscv_aia_nr_hgei = min((ulong)kvm_riscv_aia_nr_hgei,
-					    gc->nr_guest_files);
+		atomic_set(&kvm_riscv_aia_nr_hgei, gc->nr_guest_files);
 	else
-		kvm_riscv_aia_nr_hgei = 0;
+		atomic_set(&kvm_riscv_aia_nr_hgei, 0);
 
 	/* Find number of guest MSI IDs */
 	kvm_riscv_aia_max_ids = IMSIC_MAX_ID;
-	if (gc && kvm_riscv_aia_nr_hgei)
+	if (gc)
 		kvm_riscv_aia_max_ids = gc->nr_guest_ids + 1;
 
 	/* Initialize guest external interrupt line management */
diff --git a/arch/riscv/kvm/aia_device.c b/arch/riscv/kvm/aia_device.c
index 3d1e81e2a36b..be83c2d5fc30 100644
--- a/arch/riscv/kvm/aia_device.c
+++ b/arch/riscv/kvm/aia_device.c
@@ -71,7 +71,7 @@ static int aia_config(struct kvm *kvm, unsigned long type,
 				 * external interrupts (i.e. non-zero
 				 * VS-level IMSIC pages).
 				 */
-				if (!kvm_riscv_aia_nr_hgei)
+				if (!atomic_read(&kvm_riscv_aia_nr_hgei))
 					return -EINVAL;
 				break;
 			default:
@@ -628,7 +628,7 @@ void kvm_riscv_aia_init_vm(struct kvm *kvm)
 	 */
 
 	/* Initialize default values in AIA global context */
-	aia->mode = (kvm_riscv_aia_nr_hgei) ?
+	aia->mode = (atomic_read(&kvm_riscv_aia_nr_hgei)) ?
 		KVM_DEV_RISCV_AIA_MODE_AUTO : KVM_DEV_RISCV_AIA_MODE_EMUL;
 	aia->nr_ids = kvm_riscv_aia_max_ids - 1;
 	aia->nr_sources = 0;
diff --git a/arch/riscv/kvm/main.c b/arch/riscv/kvm/main.c
index cb8a65273c1f..0924c75100a2 100644
--- a/arch/riscv/kvm/main.c
+++ b/arch/riscv/kvm/main.c
@@ -168,10 +168,6 @@ static int __init riscv_kvm_init(void)
 
 	kvm_info("VMID %ld bits available\n", kvm_riscv_gstage_vmid_bits());
 
-	if (kvm_riscv_aia_available())
-		kvm_info("AIA available with %d guest external interrupts\n",
-			 kvm_riscv_aia_nr_hgei);
-
 	kvm_riscv_setup_vendor_features();
 
 	kvm_register_perf_callbacks();
@@ -182,6 +178,10 @@ static int __init riscv_kvm_init(void)
 		return rc;
 	}
 
+	if (kvm_riscv_aia_available())
+		kvm_info("AIA available with %d guest external interrupts\n",
+			 atomic_read(&kvm_riscv_aia_nr_hgei));
+
 	return 0;
 }
 module_init(riscv_kvm_init);
-- 
2.43.0


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v5 1/2] irqchip/riscv-imsic: Add nr_guest_files in per-HART local config

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Pre-existing issues:
- [High] The loop searching for the correct MMIO location uses `nr_mmios` as its continuation condition instead of `j < nr_mmios`. This acts as an infinite loop if `nr_mmios > 0` and can cause an out-of-bounds read on the `mmios` array.
- [High] The calculation of `nr_guest_files` can silently underflow to `0xFFFFFFFF` if the remaining MMIO resource size is smaller than a single page, leading to an incorrectly maximized guest file count.
--

commit 15182effc09715c1274ad73e7c779ccf2a70e30d
Author: Guo Ren (Alibaba DAMO Academy) <guoren@kernel.org>

irqchip/riscv-imsic: Add nr_guest_files in per-HART local config

This commit adds nr_guest_files to the per-HART local config to represent
the number of guest files available on a particular HART. This allows KVM
to use per-HART configurations for asymmetric big.Little systems instead
of relying on a global minimum.

> diff --git a/drivers/irqchip/irq-riscv-imsic-state.c b/drivers/irqchip/irq-riscv-imsic-state.c
> index e3ed874d89e73..b8d1bbbf42f73 100644
> --- a/drivers/irqchip/irq-riscv-imsic-state.c
> +++ b/drivers/irqchip/irq-riscv-imsic-state.c
> @@ -920,13 +920,12 @@ int __init imsic_setup_state(struct fwnode_handle *fwnode, void *opaque)
>  		local->msi_va = mmios_va[index] + reloff;

[Severity: High]
This is a pre-existing issue, but looking at the code just above this hunk
in imsic_setup_state(), the loop searching for the correct MMIO location
uses nr_mmios as its continuation condition:

	for (j = 0; nr_mmios; j++) {
		if (reloff < resource_size(&mmios[j])) {
			...

Since nr_mmios is a constant integer greater than 0, this acts as an
unconditional loop. If the condition is never satisfied, could j increment
past nr_mmios and cause an out-of-bounds read on the dynamically allocated
mmios array?

>  
>  		/*
> -		 * KVM uses global->nr_guest_files to determine the available guest
> -		 * interrupt files on each CPU. Take the minimum number of guest
> -		 * interrupt files across all CPUs to avoid KVM incorrectly allocating
> -		 * an unexisted or unmapped guest interrupt file on some CPUs.
> +		 * KVM uses both local->nr_guest_files and global->nr_guest_files
> +		 * to determine the available guest interrupt files on each CPU.
>  		 */
>  		nr_guest_files = (resource_size(&mmios[index]) - reloff) / IMSIC_MMIO_PAGE_SZ - 1;

[Severity: High]
This is a pre-existing issue, but can this calculation silently underflow to
0xFFFFFFFF if the remaining MMIO resource size is smaller than a single
page?

If the remaining resource size is strictly greater than 0 but less than
IMSIC_MMIO_PAGE_SZ, the division yields 0. Subtracting 1 from 0 causes an
unsigned underflow.

Because min() is used in the following lines, it compares both as unsigned
and selects the maximum allowed guest files instead of 0. Could this
artificially inflate the guest file count and lead KVM to make out-of-bounds
MMIO accesses?

> -		global->nr_guest_files = min(global->nr_guest_files, nr_guest_files);
> +		local->nr_guest_files = min((BIT(global->guest_index_bits) - 1), nr_guest_files);
> +		global->nr_guest_files = min(global->nr_guest_files, local->nr_guest_files);
>  
>  		nr_handlers++;
>  	}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260525094945.3721783-1-anup.patel@oss.qualcomm.com?part=1

Re: [PATCH v5 2/2] RISC-V: KVM: AIA: Make HGEI number management fully per-CPU

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [Critical] This problem wasn't introduced by this patch, but a Use-After-Free vulnerability exists in `kvm_riscv_aia_disable()` during concurrent VM destruction and CPU offline.
--

commit b20fb1dd06460a83c6a2cbe4fe6445306d7467e2
Author: Guo Ren (Alibaba DAMO Academy) <guoren@kernel.org>

RISC-V: KVM: AIA: Make HGEI number management fully per-CPU

This commit introduces a per-CPU field for the number of Hypervisor Guest
External Interrupt (HGEI) lines, replacing a single global variable. It
updates allocation, freeing, and disabling paths to use the probed
per-CPU values. The global value is retained to represent the minimum
number of HGEI lines across all HARTs.

> diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
> index 5ec503288555d..bafb009c5ce55 100644
> --- a/arch/riscv/kvm/aia.c
> +++ b/arch/riscv/kvm/aia.c
[ ... ]
> @@ -588,7 +617,7 @@ void kvm_riscv_aia_disable(void)
>  
>  	raw_spin_lock_irqsave(&hgctrl->lock, flags);
>  
> -	for (i = 0; i <= kvm_riscv_aia_nr_hgei; i++) {
> +	for (i = 0; i <= hgctrl->nr_hgei; i++) {
>  		vcpu = hgctrl->owners[i];
>  		if (!vcpu)
>  			continue;

[Severity: Critical]
This is a pre-existing issue, but could this lead to a use-after-free
during concurrent VM destruction and CPU offline?

Looking at kvm_riscv_aia_disable(), the code reads the vcpu pointer and
temporarily drops the lock before using it:

    vcpu = hgctrl->owners[i];
    if (!vcpu)
        continue;

    raw_spin_unlock_irqrestore(&hgctrl->lock, flags);
    kvm_riscv_vcpu_aia_imsic_release(vcpu);

If CPU hotplug runs concurrently with VM teardown:

CPU1 (CPU offline)
kvm_riscv_aia_disable()
    vcpu = hgctrl->owners[i];
    raw_spin_unlock_irqrestore(&hgctrl->lock, flags);

CPU2 (VM teardown)
kvm_destroy_vm()
    kvm_vcpu_destroy()
        kvm_riscv_vcpu_aia_deinit()
            imsic_vsfile_cleanup()
                kvm_riscv_aia_free_hgei()
                    // acquires hgctrl->lock, clears the owners slot
        kmem_cache_free(vcpu)

CPU1 (CPU offline resumes)
kvm_riscv_aia_disable()
    kvm_riscv_vcpu_aia_imsic_release(vcpu); // dereferences freed vcpu

Since kvm_vcpu_destroy() does not use RCU for reclamation and no reference
counts are held for the vcpu while the lock is dropped, the memory appears
to be freed immediately.

Is there a mechanism preventing this race?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260525094945.3721783-1-anup.patel@oss.qualcomm.com?part=2

Re: [PATCH v5 2/2] RISC-V: KVM: AIA: Make HGEI number management fully per-CPU

[Guo Ren]

On Mon, May 25, 2026 at 5:49 PM Anup Patel <anup.patel@oss.qualcomm.com> wrote:
>
> From: "Guo Ren (Alibaba DAMO Academy)" <guoren@kernel.org>
>
> Previously, the number of Hypervisor Guest External Interrupt (HGEI)
> lines was stored in a single global variable `kvm_riscv_aia_nr_hgei`
> and assumed to be the same for all HARTs. This assumption does not
> hold on heterogeneous RISC-V SoCs where different cores may expose
> different HGEIE CSR widths.
>
> Introduce `nr_hgei` field into the per-CPU `struct aia_hgei_control`
> and probe the actual supported HGEI count for the current HART in
> `kvm_riscv_aia_enable()` using the standard RISC-V CSR probe technique:
>
>     csr_write(CSR_HGEIE, -1UL);
>     nr = fls_long(csr_read(CSR_HGEIE));
>     if (nr)
>         nr--;
>
> All HGEI allocation, free and disable paths (`kvm_riscv_aia_free_hgei()`,
> `kvm_riscv_aia_disable()`, etc.) now use the per-CPU value instead of
> the global one.
>
> The global `kvm_riscv_aia_nr_hgei` now represents the minimum number
> of HGEI lines across HARTs and can be used to check whether HGEI
> support is available or not.
>
> This makes KVM AIA robust on big.LITTLE-style asymmetric platforms.
>
> Signed-off-by: Guo Ren (Alibaba DAMO Academy) <guoren@kernel.org>
> Signed-off-by: Anup Patel <anup.patel@oss.qualcomm.com>
> ---
>  arch/riscv/include/asm/kvm_aia.h |  2 +-
>  arch/riscv/kvm/aia.c             | 88 +++++++++++++++++++-------------
>  arch/riscv/kvm/aia_device.c      |  4 +-
>  arch/riscv/kvm/main.c            |  8 +--
>  4 files changed, 60 insertions(+), 42 deletions(-)
>
> diff --git a/arch/riscv/include/asm/kvm_aia.h b/arch/riscv/include/asm/kvm_aia.h
> index b04ecdd1a860..c67ec5ac0a14 100644
> --- a/arch/riscv/include/asm/kvm_aia.h
> +++ b/arch/riscv/include/asm/kvm_aia.h
> @@ -79,7 +79,7 @@ struct kvm_vcpu_aia {
>
>  #define irqchip_in_kernel(k)           ((k)->arch.aia.in_kernel)
>
> -extern unsigned int kvm_riscv_aia_nr_hgei;
> +extern atomic_t kvm_riscv_aia_nr_hgei;
>  extern unsigned int kvm_riscv_aia_max_ids;
>  DECLARE_STATIC_KEY_FALSE(kvm_riscv_aia_available);
>  #define kvm_riscv_aia_available() \
> diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
> index 5ec503288555..bafb009c5ce5 100644
> --- a/arch/riscv/kvm/aia.c
> +++ b/arch/riscv/kvm/aia.c
> @@ -21,13 +21,15 @@
>
>  struct aia_hgei_control {
>         raw_spinlock_t lock;
> +       bool free_bitmap_initialized;
>         unsigned long free_bitmap;
>         struct kvm_vcpu *owners[BITS_PER_LONG];
> +       unsigned int nr_hgei;
>  };
>  static DEFINE_PER_CPU(struct aia_hgei_control, aia_hgei);
>  static int hgei_parent_irq;
>
> -unsigned int kvm_riscv_aia_nr_hgei;
> +atomic_t kvm_riscv_aia_nr_hgei;
>  unsigned int kvm_riscv_aia_max_ids;
>  DEFINE_STATIC_KEY_FALSE(kvm_riscv_aia_available);
>
> @@ -452,7 +454,7 @@ void kvm_riscv_aia_free_hgei(int cpu, int hgei)
>
>         raw_spin_lock_irqsave(&hgctrl->lock, flags);
>
> -       if (hgei > 0 && hgei <= kvm_riscv_aia_nr_hgei) {
> +       if (hgei > 0 && hgei <= hgctrl->nr_hgei) {
>                 if (!(hgctrl->free_bitmap & BIT(hgei))) {
>                         hgctrl->free_bitmap |= BIT(hgei);
>                         hgctrl->owners[hgei] = NULL;
> @@ -486,26 +488,18 @@ static irqreturn_t hgei_interrupt(int irq, void *dev_id)
>
>  static int aia_hgei_init(void)
>  {
> -       int cpu, rc;
> -       struct irq_domain *domain;
>         struct aia_hgei_control *hgctrl;
> +       struct irq_domain *domain;
> +       int cpu, rc;
>
>         /* Initialize per-CPU guest external interrupt line management */
>         for_each_possible_cpu(cpu) {
>                 hgctrl = per_cpu_ptr(&aia_hgei, cpu);
>                 raw_spin_lock_init(&hgctrl->lock);
> -               if (kvm_riscv_aia_nr_hgei) {
> -                       hgctrl->free_bitmap =
> -                               BIT(kvm_riscv_aia_nr_hgei + 1) - 1;
> -                       hgctrl->free_bitmap &= ~BIT(0);
> -               } else
> -                       hgctrl->free_bitmap = 0;
> +               hgctrl->free_bitmap_initialized = false;
> +               hgctrl->free_bitmap = 0;
>         }
>
> -       /* Skip SGEI interrupt setup for zero guest external interrupts */
> -       if (!kvm_riscv_aia_nr_hgei)
> -               goto skip_sgei_interrupt;
> -
>         /* Find INTC irq domain */
>         domain = irq_find_matching_fwnode(riscv_get_intc_hwnode(),
>                                           DOMAIN_BUS_ANY);
> @@ -529,25 +523,60 @@ static int aia_hgei_init(void)
>                 return rc;
>         }
>
> -skip_sgei_interrupt:
>         return 0;
>  }
>
>  static void aia_hgei_exit(void)
>  {
> -       /* Do nothing for zero guest external interrupts */
> -       if (!kvm_riscv_aia_nr_hgei)
> -               return;
> -
>         /* Free per-CPU SGEI interrupt */
>         free_percpu_irq(hgei_parent_irq, &aia_hgei);
>  }
>
>  void kvm_riscv_aia_enable(void)
>  {
> +       const struct imsic_global_config *gc;
> +       const struct imsic_local_config *lc;
> +       struct aia_hgei_control *hgctrl;
> +       unsigned long flags;
> +       int aia_nr_hgei;
> +
>         if (!kvm_riscv_aia_available())
>                 return;
>
> +       gc = imsic_get_global_config();
> +       lc = (gc) ? this_cpu_ptr(gc->local) : NULL;
> +       hgctrl = this_cpu_ptr(&aia_hgei);
> +
> +       /* Figure-out number of bits in HGEIE */
> +       csr_write(CSR_HGEIE, -1UL);
> +       hgctrl->nr_hgei = fls_long(csr_read(CSR_HGEIE));
> +       csr_write(CSR_HGEIE, 0);
> +       if (hgctrl->nr_hgei)
> +               hgctrl->nr_hgei--;
> +
> +       /*
> +        * Number of usable per-HART HGEI lines should be minimum of
> +        * per-HART IMSIC guest files and number of bits in HGEIE.
> +        */
> +       if (lc)
> +               hgctrl->nr_hgei = min((ulong)hgctrl->nr_hgei, lc->nr_guest_files);
Great! You correct that.

> +       else
> +               hgctrl->nr_hgei = 0;
> +
> +       /* Update the number of IMSIC guest files across all HARTs */
> +       aia_nr_hgei = atomic_read(&kvm_riscv_aia_nr_hgei);
> +       do {
> +               if (aia_nr_hgei <= hgctrl->nr_hgei)
> +                       break;
> +       } while (!atomic_try_cmpxchg(&kvm_riscv_aia_nr_hgei, &aia_nr_hgei, hgctrl->nr_hgei));
> +
> +       raw_spin_lock_irqsave(&hgctrl->lock, flags);
> +       if (!hgctrl->free_bitmap_initialized) {
> +               hgctrl->free_bitmap = (hgctrl->nr_hgei) ? GENMASK_ULL(hgctrl->nr_hgei, 1) : 0;
> +               hgctrl->free_bitmap_initialized = true;
> +       }
> +       raw_spin_unlock_irqrestore(&hgctrl->lock, flags);
> +
>         csr_write(CSR_HVICTL, aia_hvictl_value(false));
>         csr_write(CSR_HVIPRIO1, 0x0);
>         csr_write(CSR_HVIPRIO2, 0x0);
> @@ -588,7 +617,7 @@ void kvm_riscv_aia_disable(void)
>
>         raw_spin_lock_irqsave(&hgctrl->lock, flags);
>
> -       for (i = 0; i <= kvm_riscv_aia_nr_hgei; i++) {
> +       for (i = 0; i <= hgctrl->nr_hgei; i++) {
>                 vcpu = hgctrl->owners[i];
>                 if (!vcpu)
>                         continue;
> @@ -628,26 +657,15 @@ int kvm_riscv_aia_init(void)
>                 return -ENODEV;
>         gc = imsic_get_global_config();
>
> -       /* Figure-out number of bits in HGEIE */
> -       csr_write(CSR_HGEIE, -1UL);
> -       kvm_riscv_aia_nr_hgei = fls_long(csr_read(CSR_HGEIE));
> -       csr_write(CSR_HGEIE, 0);
> -       if (kvm_riscv_aia_nr_hgei)
> -               kvm_riscv_aia_nr_hgei--;
> -
> -       /*
> -        * Number of usable HGEI lines should be minimum of per-HART
> -        * IMSIC guest files and number of bits in HGEIE
> -        */
> +       /* Set initial value of IMSIC guest files across all HARTs */
>         if (gc)
> -               kvm_riscv_aia_nr_hgei = min((ulong)kvm_riscv_aia_nr_hgei,
> -                                           gc->nr_guest_files);
> +               atomic_set(&kvm_riscv_aia_nr_hgei, gc->nr_guest_files);
>         else
> -               kvm_riscv_aia_nr_hgei = 0;
> +               atomic_set(&kvm_riscv_aia_nr_hgei, 0);
>
>         /* Find number of guest MSI IDs */
>         kvm_riscv_aia_max_ids = IMSIC_MAX_ID;
> -       if (gc && kvm_riscv_aia_nr_hgei)
> +       if (gc)
>                 kvm_riscv_aia_max_ids = gc->nr_guest_ids + 1;
>
>         /* Initialize guest external interrupt line management */
> diff --git a/arch/riscv/kvm/aia_device.c b/arch/riscv/kvm/aia_device.c
> index 3d1e81e2a36b..be83c2d5fc30 100644
> --- a/arch/riscv/kvm/aia_device.c
> +++ b/arch/riscv/kvm/aia_device.c
> @@ -71,7 +71,7 @@ static int aia_config(struct kvm *kvm, unsigned long type,
>                                  * external interrupts (i.e. non-zero
>                                  * VS-level IMSIC pages).
>                                  */
> -                               if (!kvm_riscv_aia_nr_hgei)
> +                               if (!atomic_read(&kvm_riscv_aia_nr_hgei))
>                                         return -EINVAL;
>                                 break;
>                         default:
> @@ -628,7 +628,7 @@ void kvm_riscv_aia_init_vm(struct kvm *kvm)
>          */
>
>         /* Initialize default values in AIA global context */
> -       aia->mode = (kvm_riscv_aia_nr_hgei) ?
> +       aia->mode = (atomic_read(&kvm_riscv_aia_nr_hgei)) ?
>                 KVM_DEV_RISCV_AIA_MODE_AUTO : KVM_DEV_RISCV_AIA_MODE_EMUL;
>         aia->nr_ids = kvm_riscv_aia_max_ids - 1;
>         aia->nr_sources = 0;
> diff --git a/arch/riscv/kvm/main.c b/arch/riscv/kvm/main.c
> index cb8a65273c1f..0924c75100a2 100644
> --- a/arch/riscv/kvm/main.c
> +++ b/arch/riscv/kvm/main.c
> @@ -168,10 +168,6 @@ static int __init riscv_kvm_init(void)
>
>         kvm_info("VMID %ld bits available\n", kvm_riscv_gstage_vmid_bits());
>
> -       if (kvm_riscv_aia_available())
> -               kvm_info("AIA available with %d guest external interrupts\n",
> -                        kvm_riscv_aia_nr_hgei);
> -
>         kvm_riscv_setup_vendor_features();
>
>         kvm_register_perf_callbacks();
> @@ -182,6 +178,10 @@ static int __init riscv_kvm_init(void)
>                 return rc;
>         }
>
> +       if (kvm_riscv_aia_available())
> +               kvm_info("AIA available with %d guest external interrupts\n",
> +                        atomic_read(&kvm_riscv_aia_nr_hgei));
> +
>         return 0;
>  }
>  module_init(riscv_kvm_init);
> --
> 2.43.0
>


-- 
Best Regards
 Guo Ren

-- 
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

Re: [PATCH v5 2/2] RISC-V: KVM: AIA: Make HGEI number management fully per-CPU

[Guo Ren]

On Mon, May 25, 2026 at 5:49 PM Anup Patel <anup.patel@oss.qualcomm.com> wrote:
>
> From: "Guo Ren (Alibaba DAMO Academy)" <guoren@kernel.org>
>
> Previously, the number of Hypervisor Guest External Interrupt (HGEI)
> lines was stored in a single global variable `kvm_riscv_aia_nr_hgei`
> and assumed to be the same for all HARTs. This assumption does not
> hold on heterogeneous RISC-V SoCs where different cores may expose
> different HGEIE CSR widths.
>
> Introduce `nr_hgei` field into the per-CPU `struct aia_hgei_control`
> and probe the actual supported HGEI count for the current HART in
> `kvm_riscv_aia_enable()` using the standard RISC-V CSR probe technique:
>
>     csr_write(CSR_HGEIE, -1UL);
>     nr = fls_long(csr_read(CSR_HGEIE));
>     if (nr)
>         nr--;
>
> All HGEI allocation, free and disable paths (`kvm_riscv_aia_free_hgei()`,
> `kvm_riscv_aia_disable()`, etc.) now use the per-CPU value instead of
> the global one.
>
> The global `kvm_riscv_aia_nr_hgei` now represents the minimum number
> of HGEI lines across HARTs and can be used to check whether HGEI
> support is available or not.
>
> This makes KVM AIA robust on big.LITTLE-style asymmetric platforms.
>
> Signed-off-by: Guo Ren (Alibaba DAMO Academy) <guoren@kernel.org>
> Signed-off-by: Anup Patel <anup.patel@oss.qualcomm.com>
> ---
>  arch/riscv/include/asm/kvm_aia.h |  2 +-
>  arch/riscv/kvm/aia.c             | 88 +++++++++++++++++++-------------
>  arch/riscv/kvm/aia_device.c      |  4 +-
>  arch/riscv/kvm/main.c            |  8 +--
>  4 files changed, 60 insertions(+), 42 deletions(-)
>
> diff --git a/arch/riscv/include/asm/kvm_aia.h b/arch/riscv/include/asm/kvm_aia.h
> index b04ecdd1a860..c67ec5ac0a14 100644
> --- a/arch/riscv/include/asm/kvm_aia.h
> +++ b/arch/riscv/include/asm/kvm_aia.h
> @@ -79,7 +79,7 @@ struct kvm_vcpu_aia {
>
>  #define irqchip_in_kernel(k)           ((k)->arch.aia.in_kernel)
>
> -extern unsigned int kvm_riscv_aia_nr_hgei;
> +extern atomic_t kvm_riscv_aia_nr_hgei;
>  extern unsigned int kvm_riscv_aia_max_ids;
>  DECLARE_STATIC_KEY_FALSE(kvm_riscv_aia_available);
>  #define kvm_riscv_aia_available() \
> diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
> index 5ec503288555..bafb009c5ce5 100644
> --- a/arch/riscv/kvm/aia.c
> +++ b/arch/riscv/kvm/aia.c
> @@ -21,13 +21,15 @@
>
>  struct aia_hgei_control {
>         raw_spinlock_t lock;
> +       bool free_bitmap_initialized;
>         unsigned long free_bitmap;
>         struct kvm_vcpu *owners[BITS_PER_LONG];
> +       unsigned int nr_hgei;
>  };
>  static DEFINE_PER_CPU(struct aia_hgei_control, aia_hgei);
>  static int hgei_parent_irq;
>
> -unsigned int kvm_riscv_aia_nr_hgei;
> +atomic_t kvm_riscv_aia_nr_hgei;
>  unsigned int kvm_riscv_aia_max_ids;
>  DEFINE_STATIC_KEY_FALSE(kvm_riscv_aia_available);
>
> @@ -452,7 +454,7 @@ void kvm_riscv_aia_free_hgei(int cpu, int hgei)
>
>         raw_spin_lock_irqsave(&hgctrl->lock, flags);
>
> -       if (hgei > 0 && hgei <= kvm_riscv_aia_nr_hgei) {
> +       if (hgei > 0 && hgei <= hgctrl->nr_hgei) {
>                 if (!(hgctrl->free_bitmap & BIT(hgei))) {
>                         hgctrl->free_bitmap |= BIT(hgei);
>                         hgctrl->owners[hgei] = NULL;
> @@ -486,26 +488,18 @@ static irqreturn_t hgei_interrupt(int irq, void *dev_id)
>
>  static int aia_hgei_init(void)
>  {
> -       int cpu, rc;
> -       struct irq_domain *domain;
>         struct aia_hgei_control *hgctrl;
> +       struct irq_domain *domain;
> +       int cpu, rc;
>
>         /* Initialize per-CPU guest external interrupt line management */
>         for_each_possible_cpu(cpu) {
>                 hgctrl = per_cpu_ptr(&aia_hgei, cpu);
>                 raw_spin_lock_init(&hgctrl->lock);
> -               if (kvm_riscv_aia_nr_hgei) {
> -                       hgctrl->free_bitmap =
> -                               BIT(kvm_riscv_aia_nr_hgei + 1) - 1;
> -                       hgctrl->free_bitmap &= ~BIT(0);
> -               } else
> -                       hgctrl->free_bitmap = 0;
> +               hgctrl->free_bitmap_initialized = false;
> +               hgctrl->free_bitmap = 0;
>         }
>
> -       /* Skip SGEI interrupt setup for zero guest external interrupts */
> -       if (!kvm_riscv_aia_nr_hgei)
> -               goto skip_sgei_interrupt;
> -
>         /* Find INTC irq domain */
>         domain = irq_find_matching_fwnode(riscv_get_intc_hwnode(),
>                                           DOMAIN_BUS_ANY);
> @@ -529,25 +523,60 @@ static int aia_hgei_init(void)
>                 return rc;
>         }
>
> -skip_sgei_interrupt:
>         return 0;
>  }
>
>  static void aia_hgei_exit(void)
>  {
> -       /* Do nothing for zero guest external interrupts */
> -       if (!kvm_riscv_aia_nr_hgei)
> -               return;
> -
>         /* Free per-CPU SGEI interrupt */
>         free_percpu_irq(hgei_parent_irq, &aia_hgei);
>  }
>
>  void kvm_riscv_aia_enable(void)
>  {
> +       const struct imsic_global_config *gc;
> +       const struct imsic_local_config *lc;
> +       struct aia_hgei_control *hgctrl;
> +       unsigned long flags;
> +       int aia_nr_hgei;
> +
>         if (!kvm_riscv_aia_available())
>                 return;
>
> +       gc = imsic_get_global_config();
> +       lc = (gc) ? this_cpu_ptr(gc->local) : NULL;
> +       hgctrl = this_cpu_ptr(&aia_hgei);
> +
> +       /* Figure-out number of bits in HGEIE */
> +       csr_write(CSR_HGEIE, -1UL);
> +       hgctrl->nr_hgei = fls_long(csr_read(CSR_HGEIE));
> +       csr_write(CSR_HGEIE, 0);
> +       if (hgctrl->nr_hgei)
> +               hgctrl->nr_hgei--;
> +
> +       /*
> +        * Number of usable per-HART HGEI lines should be minimum of
> +        * per-HART IMSIC guest files and number of bits in HGEIE.
> +        */
> +       if (lc)
> +               hgctrl->nr_hgei = min((ulong)hgctrl->nr_hgei, lc->nr_guest_files);
Great! You correct that.

> +       else
> +               hgctrl->nr_hgei = 0;
> +
> +       /* Update the number of IMSIC guest files across all HARTs */
> +       aia_nr_hgei = atomic_read(&kvm_riscv_aia_nr_hgei);
> +       do {
> +               if (aia_nr_hgei <= hgctrl->nr_hgei)
> +                       break;
> +       } while (!atomic_try_cmpxchg(&kvm_riscv_aia_nr_hgei, &aia_nr_hgei, hgctrl->nr_hgei));
> +
> +       raw_spin_lock_irqsave(&hgctrl->lock, flags);
> +       if (!hgctrl->free_bitmap_initialized) {
> +               hgctrl->free_bitmap = (hgctrl->nr_hgei) ? GENMASK_ULL(hgctrl->nr_hgei, 1) : 0;
> +               hgctrl->free_bitmap_initialized = true;
> +       }
> +       raw_spin_unlock_irqrestore(&hgctrl->lock, flags);
> +
>         csr_write(CSR_HVICTL, aia_hvictl_value(false));
>         csr_write(CSR_HVIPRIO1, 0x0);
>         csr_write(CSR_HVIPRIO2, 0x0);
> @@ -588,7 +617,7 @@ void kvm_riscv_aia_disable(void)
>
>         raw_spin_lock_irqsave(&hgctrl->lock, flags);
>
> -       for (i = 0; i <= kvm_riscv_aia_nr_hgei; i++) {
> +       for (i = 0; i <= hgctrl->nr_hgei; i++) {
>                 vcpu = hgctrl->owners[i];
>                 if (!vcpu)
>                         continue;
> @@ -628,26 +657,15 @@ int kvm_riscv_aia_init(void)
>                 return -ENODEV;
>         gc = imsic_get_global_config();
>
> -       /* Figure-out number of bits in HGEIE */
> -       csr_write(CSR_HGEIE, -1UL);
> -       kvm_riscv_aia_nr_hgei = fls_long(csr_read(CSR_HGEIE));
> -       csr_write(CSR_HGEIE, 0);
> -       if (kvm_riscv_aia_nr_hgei)
> -               kvm_riscv_aia_nr_hgei--;
> -
> -       /*
> -        * Number of usable HGEI lines should be minimum of per-HART
> -        * IMSIC guest files and number of bits in HGEIE
> -        */
> +       /* Set initial value of IMSIC guest files across all HARTs */
>         if (gc)
> -               kvm_riscv_aia_nr_hgei = min((ulong)kvm_riscv_aia_nr_hgei,
> -                                           gc->nr_guest_files);
> +               atomic_set(&kvm_riscv_aia_nr_hgei, gc->nr_guest_files);
>         else
> -               kvm_riscv_aia_nr_hgei = 0;
> +               atomic_set(&kvm_riscv_aia_nr_hgei, 0);
>
>         /* Find number of guest MSI IDs */
>         kvm_riscv_aia_max_ids = IMSIC_MAX_ID;
> -       if (gc && kvm_riscv_aia_nr_hgei)
> +       if (gc)
>                 kvm_riscv_aia_max_ids = gc->nr_guest_ids + 1;
>
>         /* Initialize guest external interrupt line management */
> diff --git a/arch/riscv/kvm/aia_device.c b/arch/riscv/kvm/aia_device.c
> index 3d1e81e2a36b..be83c2d5fc30 100644
> --- a/arch/riscv/kvm/aia_device.c
> +++ b/arch/riscv/kvm/aia_device.c
> @@ -71,7 +71,7 @@ static int aia_config(struct kvm *kvm, unsigned long type,
>                                  * external interrupts (i.e. non-zero
>                                  * VS-level IMSIC pages).
>                                  */
> -                               if (!kvm_riscv_aia_nr_hgei)
> +                               if (!atomic_read(&kvm_riscv_aia_nr_hgei))
>                                         return -EINVAL;
>                                 break;
>                         default:
> @@ -628,7 +628,7 @@ void kvm_riscv_aia_init_vm(struct kvm *kvm)
>          */
>
>         /* Initialize default values in AIA global context */
> -       aia->mode = (kvm_riscv_aia_nr_hgei) ?
> +       aia->mode = (atomic_read(&kvm_riscv_aia_nr_hgei)) ?
>                 KVM_DEV_RISCV_AIA_MODE_AUTO : KVM_DEV_RISCV_AIA_MODE_EMUL;
>         aia->nr_ids = kvm_riscv_aia_max_ids - 1;
>         aia->nr_sources = 0;
> diff --git a/arch/riscv/kvm/main.c b/arch/riscv/kvm/main.c
> index cb8a65273c1f..0924c75100a2 100644
> --- a/arch/riscv/kvm/main.c
> +++ b/arch/riscv/kvm/main.c
> @@ -168,10 +168,6 @@ static int __init riscv_kvm_init(void)
>
>         kvm_info("VMID %ld bits available\n", kvm_riscv_gstage_vmid_bits());
>
> -       if (kvm_riscv_aia_available())
> -               kvm_info("AIA available with %d guest external interrupts\n",
> -                        kvm_riscv_aia_nr_hgei);
> -
>         kvm_riscv_setup_vendor_features();
>
>         kvm_register_perf_callbacks();
> @@ -182,6 +178,10 @@ static int __init riscv_kvm_init(void)
>                 return rc;
>         }
>
> +       if (kvm_riscv_aia_available())
> +               kvm_info("AIA available with %d guest external interrupts\n",
> +                        atomic_read(&kvm_riscv_aia_nr_hgei));
> +
>         return 0;
>  }
>  module_init(riscv_kvm_init);
> --
> 2.43.0
>


-- 
Best Regards
 Guo Ren

Re: [PATCH v5 2/2] RISC-V: KVM: AIA: Make HGEI number management fully per-CPU

[Guo Ren]

On Mon, May 25, 2026 at 5:49 PM Anup Patel <anup.patel@oss.qualcomm.com> wrote:
>
> From: "Guo Ren (Alibaba DAMO Academy)" <guoren@kernel.org>
>
> Previously, the number of Hypervisor Guest External Interrupt (HGEI)
> lines was stored in a single global variable `kvm_riscv_aia_nr_hgei`
> and assumed to be the same for all HARTs. This assumption does not
> hold on heterogeneous RISC-V SoCs where different cores may expose
> different HGEIE CSR widths.
>
> Introduce `nr_hgei` field into the per-CPU `struct aia_hgei_control`
> and probe the actual supported HGEI count for the current HART in
> `kvm_riscv_aia_enable()` using the standard RISC-V CSR probe technique:
>
>     csr_write(CSR_HGEIE, -1UL);
>     nr = fls_long(csr_read(CSR_HGEIE));
>     if (nr)
>         nr--;
>
> All HGEI allocation, free and disable paths (`kvm_riscv_aia_free_hgei()`,
> `kvm_riscv_aia_disable()`, etc.) now use the per-CPU value instead of
> the global one.
>
> The global `kvm_riscv_aia_nr_hgei` now represents the minimum number
> of HGEI lines across HARTs and can be used to check whether HGEI
> support is available or not.
>
> This makes KVM AIA robust on big.LITTLE-style asymmetric platforms.
>
> Signed-off-by: Guo Ren (Alibaba DAMO Academy) <guoren@kernel.org>
> Signed-off-by: Anup Patel <anup.patel@oss.qualcomm.com>
> ---
>  arch/riscv/include/asm/kvm_aia.h |  2 +-
>  arch/riscv/kvm/aia.c             | 88 +++++++++++++++++++-------------
>  arch/riscv/kvm/aia_device.c      |  4 +-
>  arch/riscv/kvm/main.c            |  8 +--
>  4 files changed, 60 insertions(+), 42 deletions(-)
>
> diff --git a/arch/riscv/include/asm/kvm_aia.h b/arch/riscv/include/asm/kvm_aia.h
> index b04ecdd1a860..c67ec5ac0a14 100644
> --- a/arch/riscv/include/asm/kvm_aia.h
> +++ b/arch/riscv/include/asm/kvm_aia.h
> @@ -79,7 +79,7 @@ struct kvm_vcpu_aia {
>
>  #define irqchip_in_kernel(k)           ((k)->arch.aia.in_kernel)
>
> -extern unsigned int kvm_riscv_aia_nr_hgei;
> +extern atomic_t kvm_riscv_aia_nr_hgei;
>  extern unsigned int kvm_riscv_aia_max_ids;
>  DECLARE_STATIC_KEY_FALSE(kvm_riscv_aia_available);
>  #define kvm_riscv_aia_available() \
> diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
> index 5ec503288555..bafb009c5ce5 100644
> --- a/arch/riscv/kvm/aia.c
> +++ b/arch/riscv/kvm/aia.c
> @@ -21,13 +21,15 @@
>
>  struct aia_hgei_control {
>         raw_spinlock_t lock;
> +       bool free_bitmap_initialized;
>         unsigned long free_bitmap;
>         struct kvm_vcpu *owners[BITS_PER_LONG];
> +       unsigned int nr_hgei;
>  };
>  static DEFINE_PER_CPU(struct aia_hgei_control, aia_hgei);
>  static int hgei_parent_irq;
>
> -unsigned int kvm_riscv_aia_nr_hgei;
> +atomic_t kvm_riscv_aia_nr_hgei;
>  unsigned int kvm_riscv_aia_max_ids;
>  DEFINE_STATIC_KEY_FALSE(kvm_riscv_aia_available);
>
> @@ -452,7 +454,7 @@ void kvm_riscv_aia_free_hgei(int cpu, int hgei)
>
>         raw_spin_lock_irqsave(&hgctrl->lock, flags);
>
> -       if (hgei > 0 && hgei <= kvm_riscv_aia_nr_hgei) {
> +       if (hgei > 0 && hgei <= hgctrl->nr_hgei) {
>                 if (!(hgctrl->free_bitmap & BIT(hgei))) {
>                         hgctrl->free_bitmap |= BIT(hgei);
>                         hgctrl->owners[hgei] = NULL;
> @@ -486,26 +488,18 @@ static irqreturn_t hgei_interrupt(int irq, void *dev_id)
>
>  static int aia_hgei_init(void)
>  {
> -       int cpu, rc;
> -       struct irq_domain *domain;
>         struct aia_hgei_control *hgctrl;
> +       struct irq_domain *domain;
> +       int cpu, rc;
>
>         /* Initialize per-CPU guest external interrupt line management */
>         for_each_possible_cpu(cpu) {
>                 hgctrl = per_cpu_ptr(&aia_hgei, cpu);
>                 raw_spin_lock_init(&hgctrl->lock);
> -               if (kvm_riscv_aia_nr_hgei) {
> -                       hgctrl->free_bitmap =
> -                               BIT(kvm_riscv_aia_nr_hgei + 1) - 1;
> -                       hgctrl->free_bitmap &= ~BIT(0);
> -               } else
> -                       hgctrl->free_bitmap = 0;
> +               hgctrl->free_bitmap_initialized = false;
> +               hgctrl->free_bitmap = 0;
>         }
>
> -       /* Skip SGEI interrupt setup for zero guest external interrupts */
> -       if (!kvm_riscv_aia_nr_hgei)
> -               goto skip_sgei_interrupt;
> -
>         /* Find INTC irq domain */
>         domain = irq_find_matching_fwnode(riscv_get_intc_hwnode(),
>                                           DOMAIN_BUS_ANY);
> @@ -529,25 +523,60 @@ static int aia_hgei_init(void)
>                 return rc;
>         }
>
> -skip_sgei_interrupt:
>         return 0;
>  }
>
>  static void aia_hgei_exit(void)
>  {
> -       /* Do nothing for zero guest external interrupts */
> -       if (!kvm_riscv_aia_nr_hgei)
> -               return;
> -
>         /* Free per-CPU SGEI interrupt */
>         free_percpu_irq(hgei_parent_irq, &aia_hgei);
>  }
>
>  void kvm_riscv_aia_enable(void)
>  {
> +       const struct imsic_global_config *gc;
> +       const struct imsic_local_config *lc;
> +       struct aia_hgei_control *hgctrl;
> +       unsigned long flags;
> +       int aia_nr_hgei;
> +
>         if (!kvm_riscv_aia_available())
>                 return;
>
> +       gc = imsic_get_global_config();
> +       lc = (gc) ? this_cpu_ptr(gc->local) : NULL;
> +       hgctrl = this_cpu_ptr(&aia_hgei);
> +
> +       /* Figure-out number of bits in HGEIE */
> +       csr_write(CSR_HGEIE, -1UL);
> +       hgctrl->nr_hgei = fls_long(csr_read(CSR_HGEIE));
> +       csr_write(CSR_HGEIE, 0);
> +       if (hgctrl->nr_hgei)
> +               hgctrl->nr_hgei--;
> +
> +       /*
> +        * Number of usable per-HART HGEI lines should be minimum of
> +        * per-HART IMSIC guest files and number of bits in HGEIE.
> +        */
> +       if (lc)
> +               hgctrl->nr_hgei = min((ulong)hgctrl->nr_hgei, lc->nr_guest_files);
Great! You correct that.

> +       else
> +               hgctrl->nr_hgei = 0;
> +
> +       /* Update the number of IMSIC guest files across all HARTs */
> +       aia_nr_hgei = atomic_read(&kvm_riscv_aia_nr_hgei);
> +       do {
> +               if (aia_nr_hgei <= hgctrl->nr_hgei)
> +                       break;
> +       } while (!atomic_try_cmpxchg(&kvm_riscv_aia_nr_hgei, &aia_nr_hgei, hgctrl->nr_hgei));
> +
> +       raw_spin_lock_irqsave(&hgctrl->lock, flags);
> +       if (!hgctrl->free_bitmap_initialized) {
> +               hgctrl->free_bitmap = (hgctrl->nr_hgei) ? GENMASK_ULL(hgctrl->nr_hgei, 1) : 0;
> +               hgctrl->free_bitmap_initialized = true;
> +       }
> +       raw_spin_unlock_irqrestore(&hgctrl->lock, flags);
> +
>         csr_write(CSR_HVICTL, aia_hvictl_value(false));
>         csr_write(CSR_HVIPRIO1, 0x0);
>         csr_write(CSR_HVIPRIO2, 0x0);
> @@ -588,7 +617,7 @@ void kvm_riscv_aia_disable(void)
>
>         raw_spin_lock_irqsave(&hgctrl->lock, flags);
>
> -       for (i = 0; i <= kvm_riscv_aia_nr_hgei; i++) {
> +       for (i = 0; i <= hgctrl->nr_hgei; i++) {
>                 vcpu = hgctrl->owners[i];
>                 if (!vcpu)
>                         continue;
> @@ -628,26 +657,15 @@ int kvm_riscv_aia_init(void)
>                 return -ENODEV;
>         gc = imsic_get_global_config();
>
> -       /* Figure-out number of bits in HGEIE */
> -       csr_write(CSR_HGEIE, -1UL);
> -       kvm_riscv_aia_nr_hgei = fls_long(csr_read(CSR_HGEIE));
> -       csr_write(CSR_HGEIE, 0);
> -       if (kvm_riscv_aia_nr_hgei)
> -               kvm_riscv_aia_nr_hgei--;
> -
> -       /*
> -        * Number of usable HGEI lines should be minimum of per-HART
> -        * IMSIC guest files and number of bits in HGEIE
> -        */
> +       /* Set initial value of IMSIC guest files across all HARTs */
>         if (gc)
> -               kvm_riscv_aia_nr_hgei = min((ulong)kvm_riscv_aia_nr_hgei,
> -                                           gc->nr_guest_files);
> +               atomic_set(&kvm_riscv_aia_nr_hgei, gc->nr_guest_files);
>         else
> -               kvm_riscv_aia_nr_hgei = 0;
> +               atomic_set(&kvm_riscv_aia_nr_hgei, 0);
>
>         /* Find number of guest MSI IDs */
>         kvm_riscv_aia_max_ids = IMSIC_MAX_ID;
> -       if (gc && kvm_riscv_aia_nr_hgei)
> +       if (gc)
>                 kvm_riscv_aia_max_ids = gc->nr_guest_ids + 1;
>
>         /* Initialize guest external interrupt line management */
> diff --git a/arch/riscv/kvm/aia_device.c b/arch/riscv/kvm/aia_device.c
> index 3d1e81e2a36b..be83c2d5fc30 100644
> --- a/arch/riscv/kvm/aia_device.c
> +++ b/arch/riscv/kvm/aia_device.c
> @@ -71,7 +71,7 @@ static int aia_config(struct kvm *kvm, unsigned long type,
>                                  * external interrupts (i.e. non-zero
>                                  * VS-level IMSIC pages).
>                                  */
> -                               if (!kvm_riscv_aia_nr_hgei)
> +                               if (!atomic_read(&kvm_riscv_aia_nr_hgei))
>                                         return -EINVAL;
>                                 break;
>                         default:
> @@ -628,7 +628,7 @@ void kvm_riscv_aia_init_vm(struct kvm *kvm)
>          */
>
>         /* Initialize default values in AIA global context */
> -       aia->mode = (kvm_riscv_aia_nr_hgei) ?
> +       aia->mode = (atomic_read(&kvm_riscv_aia_nr_hgei)) ?
>                 KVM_DEV_RISCV_AIA_MODE_AUTO : KVM_DEV_RISCV_AIA_MODE_EMUL;
>         aia->nr_ids = kvm_riscv_aia_max_ids - 1;
>         aia->nr_sources = 0;
> diff --git a/arch/riscv/kvm/main.c b/arch/riscv/kvm/main.c
> index cb8a65273c1f..0924c75100a2 100644
> --- a/arch/riscv/kvm/main.c
> +++ b/arch/riscv/kvm/main.c
> @@ -168,10 +168,6 @@ static int __init riscv_kvm_init(void)
>
>         kvm_info("VMID %ld bits available\n", kvm_riscv_gstage_vmid_bits());
>
> -       if (kvm_riscv_aia_available())
> -               kvm_info("AIA available with %d guest external interrupts\n",
> -                        kvm_riscv_aia_nr_hgei);
> -
>         kvm_riscv_setup_vendor_features();
>
>         kvm_register_perf_callbacks();
> @@ -182,6 +178,10 @@ static int __init riscv_kvm_init(void)
>                 return rc;
>         }
>
> +       if (kvm_riscv_aia_available())
> +               kvm_info("AIA available with %d guest external interrupts\n",
> +                        atomic_read(&kvm_riscv_aia_nr_hgei));
> +
>         return 0;
>  }
>  module_init(riscv_kvm_init);
> --
> 2.43.0
>


-- 
Best Regards
 Guo Ren

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v5 0/2] RISC-V: KVM: Convert HGEI management to fully per-HART

[Anup Patel]

On Mon, May 25, 2026 at 3:19 PM Anup Patel <anup.patel@oss.qualcomm.com> wrote:
>
> From: "Guo Ren (Alibaba DAMO Academy)" <guoren@kernel.org>
>
> This short series converts RISC-V KVM AIA's Hypervisor Guest External
> Interrupt (HGEI) line management from a global "one-size-fits-all"
> model to a fully per-HART (per-CPU) model. It also performs the
> corresponding changes in the IMSIC irqchip driver.
>
> The motivation is to properly support heterogeneous RISC-V SoCs
> (big.LITTLE, multi-vendor core mixes) where different HARTs may
> expose different numbers of guest interrupt files / HGEIE bits.
>
> After this series, HGEI allocation, freeing, and interrupt delivery
> are completely per-HART and there are no remaining global assumptions
> about number of guest interrupt files.
>
> Changelog:
> V5:
>  - Avoid re-initializing hgctrl->lock in kvm_riscv_aia_enable()
>  - Setup free_bitmap only once in kvm_riscv_aia_enable()
>  - Make kvm_riscv_aia_nr_hgei so that it can be concurrently
>    updated from kvm_riscv_aia_enable()
>
> V4:
>  - Don't modify kvm_riscv_aia_nr_hgei in kvm_riscv_aia_enable()
>    since we use kvm_riscv_aia_nr_hgei mostly for HGEI enable/disable
>    checks.
>  - Use GENMASK_ULL() instead of `BIT(hgctrl->nr_hgei + 1) - 1`
>  - Always setup SGEI interrupt irrespective of kvm_riscv_aia_nr_hgei
>
> V3:
>  - Move PATCH3 as first patch and kept the nr_guest_files in
>    global config as the number of guest files across all CPUs
>  - Droped original PATCH2 and squashed original PATCH4 into
>    original PATCH1 which is now PATCH2.
>  - Addressed comments in original PATCH1.
>
> V2:
>  - Add per-HART IMSIC guest files to compute final HGEI count.
>  - Add min(local->nr_guest_files, nr_guest_files);
>
> V1:
>  - https://lore.kernel.org/kvm-riscv/20260421145451.1597930-1-guoren@kernel.org/
>
> Guo Ren (Alibaba DAMO Academy) (2):
>   irqchip/riscv-imsic: Add nr_guest_files in per-HART local config
>   RISC-V: KVM: AIA: Make HGEI number management fully per-CPU
>
>  arch/riscv/include/asm/kvm_aia.h        |  2 +-
>  arch/riscv/kvm/aia.c                    | 88 +++++++++++++++----------
>  arch/riscv/kvm/aia_device.c             |  4 +-
>  arch/riscv/kvm/main.c                   |  8 +--
>  drivers/irqchip/irq-riscv-imsic-state.c |  9 ++-
>  include/linux/irqchip/riscv-imsic.h     |  5 +-
>  6 files changed, 68 insertions(+), 48 deletions(-)
>
> --
> 2.43.0
>

Queued this series for Linux-7.2

Thanks,
Anup

-- 
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

Re: [PATCH v5 0/2] RISC-V: KVM: Convert HGEI management to fully per-HART

[Anup Patel]

On Mon, May 25, 2026 at 3:19 PM Anup Patel <anup.patel@oss.qualcomm.com> wrote:
>
> From: "Guo Ren (Alibaba DAMO Academy)" <guoren@kernel.org>
>
> This short series converts RISC-V KVM AIA's Hypervisor Guest External
> Interrupt (HGEI) line management from a global "one-size-fits-all"
> model to a fully per-HART (per-CPU) model. It also performs the
> corresponding changes in the IMSIC irqchip driver.
>
> The motivation is to properly support heterogeneous RISC-V SoCs
> (big.LITTLE, multi-vendor core mixes) where different HARTs may
> expose different numbers of guest interrupt files / HGEIE bits.
>
> After this series, HGEI allocation, freeing, and interrupt delivery
> are completely per-HART and there are no remaining global assumptions
> about number of guest interrupt files.
>
> Changelog:
> V5:
>  - Avoid re-initializing hgctrl->lock in kvm_riscv_aia_enable()
>  - Setup free_bitmap only once in kvm_riscv_aia_enable()
>  - Make kvm_riscv_aia_nr_hgei so that it can be concurrently
>    updated from kvm_riscv_aia_enable()
>
> V4:
>  - Don't modify kvm_riscv_aia_nr_hgei in kvm_riscv_aia_enable()
>    since we use kvm_riscv_aia_nr_hgei mostly for HGEI enable/disable
>    checks.
>  - Use GENMASK_ULL() instead of `BIT(hgctrl->nr_hgei + 1) - 1`
>  - Always setup SGEI interrupt irrespective of kvm_riscv_aia_nr_hgei
>
> V3:
>  - Move PATCH3 as first patch and kept the nr_guest_files in
>    global config as the number of guest files across all CPUs
>  - Droped original PATCH2 and squashed original PATCH4 into
>    original PATCH1 which is now PATCH2.
>  - Addressed comments in original PATCH1.
>
> V2:
>  - Add per-HART IMSIC guest files to compute final HGEI count.
>  - Add min(local->nr_guest_files, nr_guest_files);
>
> V1:
>  - https://lore.kernel.org/kvm-riscv/20260421145451.1597930-1-guoren@kernel.org/
>
> Guo Ren (Alibaba DAMO Academy) (2):
>   irqchip/riscv-imsic: Add nr_guest_files in per-HART local config
>   RISC-V: KVM: AIA: Make HGEI number management fully per-CPU
>
>  arch/riscv/include/asm/kvm_aia.h        |  2 +-
>  arch/riscv/kvm/aia.c                    | 88 +++++++++++++++----------
>  arch/riscv/kvm/aia_device.c             |  4 +-
>  arch/riscv/kvm/main.c                   |  8 +--
>  drivers/irqchip/irq-riscv-imsic-state.c |  9 ++-
>  include/linux/irqchip/riscv-imsic.h     |  5 +-
>  6 files changed, 68 insertions(+), 48 deletions(-)
>
> --
> 2.43.0
>

Queued this series for Linux-7.2

Thanks,
Anup

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v5 0/2] RISC-V: KVM: Convert HGEI management to fully per-HART

[Anup Patel]

On Mon, May 25, 2026 at 3:19 PM Anup Patel <anup.patel@oss.qualcomm.com> wrote:
>
> From: "Guo Ren (Alibaba DAMO Academy)" <guoren@kernel.org>
>
> This short series converts RISC-V KVM AIA's Hypervisor Guest External
> Interrupt (HGEI) line management from a global "one-size-fits-all"
> model to a fully per-HART (per-CPU) model. It also performs the
> corresponding changes in the IMSIC irqchip driver.
>
> The motivation is to properly support heterogeneous RISC-V SoCs
> (big.LITTLE, multi-vendor core mixes) where different HARTs may
> expose different numbers of guest interrupt files / HGEIE bits.
>
> After this series, HGEI allocation, freeing, and interrupt delivery
> are completely per-HART and there are no remaining global assumptions
> about number of guest interrupt files.
>
> Changelog:
> V5:
>  - Avoid re-initializing hgctrl->lock in kvm_riscv_aia_enable()
>  - Setup free_bitmap only once in kvm_riscv_aia_enable()
>  - Make kvm_riscv_aia_nr_hgei so that it can be concurrently
>    updated from kvm_riscv_aia_enable()
>
> V4:
>  - Don't modify kvm_riscv_aia_nr_hgei in kvm_riscv_aia_enable()
>    since we use kvm_riscv_aia_nr_hgei mostly for HGEI enable/disable
>    checks.
>  - Use GENMASK_ULL() instead of `BIT(hgctrl->nr_hgei + 1) - 1`
>  - Always setup SGEI interrupt irrespective of kvm_riscv_aia_nr_hgei
>
> V3:
>  - Move PATCH3 as first patch and kept the nr_guest_files in
>    global config as the number of guest files across all CPUs
>  - Droped original PATCH2 and squashed original PATCH4 into
>    original PATCH1 which is now PATCH2.
>  - Addressed comments in original PATCH1.
>
> V2:
>  - Add per-HART IMSIC guest files to compute final HGEI count.
>  - Add min(local->nr_guest_files, nr_guest_files);
>
> V1:
>  - https://lore.kernel.org/kvm-riscv/20260421145451.1597930-1-guoren@kernel.org/
>
> Guo Ren (Alibaba DAMO Academy) (2):
>   irqchip/riscv-imsic: Add nr_guest_files in per-HART local config
>   RISC-V: KVM: AIA: Make HGEI number management fully per-CPU
>
>  arch/riscv/include/asm/kvm_aia.h        |  2 +-
>  arch/riscv/kvm/aia.c                    | 88 +++++++++++++++----------
>  arch/riscv/kvm/aia_device.c             |  4 +-
>  arch/riscv/kvm/main.c                   |  8 +--
>  drivers/irqchip/irq-riscv-imsic-state.c |  9 ++-
>  include/linux/irqchip/riscv-imsic.h     |  5 +-
>  6 files changed, 68 insertions(+), 48 deletions(-)
>
> --
> 2.43.0
>

Queued this series for Linux-7.2

Thanks,
Anup