[PATCH 0/3] RISC-V: KVM: AIA: Convert HGEI management to fully per-HART

[PATCH 0/3] RISC-V: KVM: AIA: Convert HGEI management to fully per-HART

[guoren]

From: "Guo Ren (Alibaba DAMO Academy)" <guoren@kernel.org>

This short series converts RISC-V KVM AIA's Hypervisor Guest External
Interrupt (HGEI) line management from a global "one-size-fits-all"
model to a fully per-HART (per-CPU) model. It also performs the
corresponding cleanup in the IMSIC irqchip driver.

The motivation is to properly support heterogeneous RISC-V SoCs
(big.LITTLE, multi-vendor core mixes) where different HARTs may
expose different numbers of guest interrupt files / HGEIE bits.
The old global `kvm_riscv_aia_nr_hgei` and
`imsic_global_config::nr_guest_files` assumptions are no longer
tenable and are removed.

After this series, HGEI allocation, freeing, and interrupt delivery
are completely per-HART, the code is simpler and clearer, and there
are no remaining global assumptions about guest interrupt file
counts.

Guo Ren (Alibaba DAMO Academy) (3):
  RISC-V: KVM: AIA: Make HGEI number and management fully per-CPU
  RISC-V: KVM: AIA: Replace global HGEI count with simple enabled bool
  irqchip/riscv-imsic: Remove global nr_guest_files after KVM AIA
    per-HART conversion

 arch/riscv/include/asm/kvm_aia.h        |  2 +-
 arch/riscv/kvm/aia.c                    | 58 ++++++++++++++-----------
 arch/riscv/kvm/aia_device.c             |  4 +-
 arch/riscv/kvm/main.c                   |  3 +-
 drivers/irqchip/irq-riscv-imsic-state.c | 12 +----
 include/linux/irqchip/riscv-imsic.h     |  3 --
 6 files changed, 37 insertions(+), 45 deletions(-)

-- 
2.40.1


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

[PATCH 1/3] RISC-V: KVM: AIA: Make HGEI number and management fully per-CPU

[guoren]

From: "Guo Ren (Alibaba DAMO Academy)" <guoren@kernel.org>

Previously the number of Hypervisor Guest External Interrupt (HGEI)
lines was stored in a single global variable `kvm_riscv_aia_nr_hgei`
and assumed to be the same for all HARTs. This assumption does not
hold on heterogeneous RISC-V SoCs where different cores may expose
different HGEIE CSR widths.

Introduce `nr_hgei` field into the per-CPU `struct aia_hgei_control`
and probe the actual supported HGEI count for the current HART in
`kvm_riscv_aia_enable()` using the standard RISC-V CSR probe
technique:

    csr_write(CSR_HGEIE, -1UL);
    nr = fls_long(csr_read(CSR_HGEIE));
    if (nr)
        nr--;

All HGEI allocation, free and disable paths (`kvm_riscv_aia_free_hgei()`,
`kvm_riscv_aia_disable()`, etc.) now use the per-CPU value instead of
the global one.

The early global `kvm_riscv_aia_nr_hgei` is kept only for deciding
whether SGEI interrupt registration is needed; the real per-HART
initialization of lock and free_bitmap is moved to enable time.

This makes KVM AIA robust on big.LITTLE-style and multi-vendor
asymmetric platforms.

Signed-off-by: Guo Ren (Alibaba DAMO Academy) <guoren@kernel.org>
---
 arch/riscv/kvm/aia.c | 40 ++++++++++++++++++++++++----------------
 1 file changed, 24 insertions(+), 16 deletions(-)

diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
index 5ec503288555..a23729052cfb 100644
--- a/arch/riscv/kvm/aia.c
+++ b/arch/riscv/kvm/aia.c
@@ -23,6 +23,7 @@ struct aia_hgei_control {
 	raw_spinlock_t lock;
 	unsigned long free_bitmap;
 	struct kvm_vcpu *owners[BITS_PER_LONG];
+	unsigned int nr_hgei;
 };
 static DEFINE_PER_CPU(struct aia_hgei_control, aia_hgei);
 static int hgei_parent_irq;
@@ -452,7 +453,7 @@ void kvm_riscv_aia_free_hgei(int cpu, int hgei)
 
 	raw_spin_lock_irqsave(&hgctrl->lock, flags);
 
-	if (hgei > 0 && hgei <= kvm_riscv_aia_nr_hgei) {
+	if (hgei > 0 && hgei <= hgctrl->nr_hgei) {
 		if (!(hgctrl->free_bitmap & BIT(hgei))) {
 			hgctrl->free_bitmap |= BIT(hgei);
 			hgctrl->owners[hgei] = NULL;
@@ -486,21 +487,8 @@ static irqreturn_t hgei_interrupt(int irq, void *dev_id)
 
 static int aia_hgei_init(void)
 {
-	int cpu, rc;
+	int rc;
 	struct irq_domain *domain;
-	struct aia_hgei_control *hgctrl;
-
-	/* Initialize per-CPU guest external interrupt line management */
-	for_each_possible_cpu(cpu) {
-		hgctrl = per_cpu_ptr(&aia_hgei, cpu);
-		raw_spin_lock_init(&hgctrl->lock);
-		if (kvm_riscv_aia_nr_hgei) {
-			hgctrl->free_bitmap =
-				BIT(kvm_riscv_aia_nr_hgei + 1) - 1;
-			hgctrl->free_bitmap &= ~BIT(0);
-		} else
-			hgctrl->free_bitmap = 0;
-	}
 
 	/* Skip SGEI interrupt setup for zero guest external interrupts */
 	if (!kvm_riscv_aia_nr_hgei)
@@ -545,9 +533,29 @@ static void aia_hgei_exit(void)
 
 void kvm_riscv_aia_enable(void)
 {
+	struct aia_hgei_control *hgctrl;
+
 	if (!kvm_riscv_aia_available())
 		return;
 
+	hgctrl = this_cpu_ptr(&aia_hgei);
+
+	/* Figure-out number of bits in HGEIE */
+	csr_write(CSR_HGEIE, -1UL);
+	hgctrl->nr_hgei = fls_long(csr_read(CSR_HGEIE));
+	csr_write(CSR_HGEIE, 0);
+	if (hgctrl->nr_hgei)
+		hgctrl->nr_hgei--;
+
+	if (hgctrl->nr_hgei) {
+		hgctrl->free_bitmap = BIT(hgctrl->nr_hgei + 1) - 1;
+		hgctrl->free_bitmap &= ~BIT(0);
+	} else {
+		hgctrl->free_bitmap = 0;
+	}
+
+	raw_spin_lock_init(&hgctrl->lock);
+
 	csr_write(CSR_HVICTL, aia_hvictl_value(false));
 	csr_write(CSR_HVIPRIO1, 0x0);
 	csr_write(CSR_HVIPRIO2, 0x0);
@@ -588,7 +596,7 @@ void kvm_riscv_aia_disable(void)
 
 	raw_spin_lock_irqsave(&hgctrl->lock, flags);
 
-	for (i = 0; i <= kvm_riscv_aia_nr_hgei; i++) {
+	for (i = 0; i <= hgctrl->nr_hgei; i++) {
 		vcpu = hgctrl->owners[i];
 		if (!vcpu)
 			continue;
-- 
2.40.1


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

[PATCH 2/3] RISC-V: KVM: AIA: Replace global HGEI count with simple enabled bool

[guoren]

From: "Guo Ren (Alibaba DAMO Academy)" <guoren@kernel.org>

Now that HGEI line management is fully per-CPU (via struct
aia_hgei_control::nr_hgei), the global `kvm_riscv_aia_nr_hgei`
is no longer needed. Replace it with a simple `bool
kvm_riscv_aia_hgei_enabled` that only indicates whether HGEI
support is available at all.

Signed-off-by: Guo Ren (Alibaba DAMO Academy) <guoren@kernel.org>
---
 arch/riscv/include/asm/kvm_aia.h |  2 +-
 arch/riscv/kvm/aia.c             | 18 ++++++++----------
 arch/riscv/kvm/aia_device.c      |  4 ++--
 arch/riscv/kvm/main.c            |  3 +--
 4 files changed, 12 insertions(+), 15 deletions(-)

diff --git a/arch/riscv/include/asm/kvm_aia.h b/arch/riscv/include/asm/kvm_aia.h
index b04ecdd1a860..e8749ac6bed7 100644
--- a/arch/riscv/include/asm/kvm_aia.h
+++ b/arch/riscv/include/asm/kvm_aia.h
@@ -79,7 +79,7 @@ struct kvm_vcpu_aia {
 
 #define irqchip_in_kernel(k)		((k)->arch.aia.in_kernel)
 
-extern unsigned int kvm_riscv_aia_nr_hgei;
+extern bool kvm_riscv_aia_hgei_enabled;
 extern unsigned int kvm_riscv_aia_max_ids;
 DECLARE_STATIC_KEY_FALSE(kvm_riscv_aia_available);
 #define kvm_riscv_aia_available() \
diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
index a23729052cfb..70ff1d25dd99 100644
--- a/arch/riscv/kvm/aia.c
+++ b/arch/riscv/kvm/aia.c
@@ -28,7 +28,7 @@ struct aia_hgei_control {
 static DEFINE_PER_CPU(struct aia_hgei_control, aia_hgei);
 static int hgei_parent_irq;
 
-unsigned int kvm_riscv_aia_nr_hgei;
+bool kvm_riscv_aia_hgei_enabled;
 unsigned int kvm_riscv_aia_max_ids;
 DEFINE_STATIC_KEY_FALSE(kvm_riscv_aia_available);
 
@@ -491,7 +491,7 @@ static int aia_hgei_init(void)
 	struct irq_domain *domain;
 
 	/* Skip SGEI interrupt setup for zero guest external interrupts */
-	if (!kvm_riscv_aia_nr_hgei)
+	if (!kvm_riscv_aia_hgei_enabled)
 		goto skip_sgei_interrupt;
 
 	/* Find INTC irq domain */
@@ -524,7 +524,7 @@ static int aia_hgei_init(void)
 static void aia_hgei_exit(void)
 {
 	/* Do nothing for zero guest external interrupts */
-	if (!kvm_riscv_aia_nr_hgei)
+	if (!kvm_riscv_aia_hgei_enabled)
 		return;
 
 	/* Free per-CPU SGEI interrupt */
@@ -631,6 +631,7 @@ int kvm_riscv_aia_init(void)
 {
 	int rc;
 	const struct imsic_global_config *gc;
+	unsigned int kvm_riscv_aia_nr_hgei;
 
 	if (!riscv_isa_extension_available(NULL, SxAIA))
 		return -ENODEV;
@@ -641,21 +642,18 @@ int kvm_riscv_aia_init(void)
 	kvm_riscv_aia_nr_hgei = fls_long(csr_read(CSR_HGEIE));
 	csr_write(CSR_HGEIE, 0);
 	if (kvm_riscv_aia_nr_hgei)
-		kvm_riscv_aia_nr_hgei--;
+		kvm_riscv_aia_hgei_enabled = true;
 
 	/*
 	 * Number of usable HGEI lines should be minimum of per-HART
 	 * IMSIC guest files and number of bits in HGEIE
 	 */
-	if (gc)
-		kvm_riscv_aia_nr_hgei = min((ulong)kvm_riscv_aia_nr_hgei,
-					    gc->nr_guest_files);
-	else
-		kvm_riscv_aia_nr_hgei = 0;
+	if (!gc)
+		kvm_riscv_aia_hgei_enabled = 0;
 
 	/* Find number of guest MSI IDs */
 	kvm_riscv_aia_max_ids = IMSIC_MAX_ID;
-	if (gc && kvm_riscv_aia_nr_hgei)
+	if (gc && kvm_riscv_aia_hgei_enabled)
 		kvm_riscv_aia_max_ids = gc->nr_guest_ids + 1;
 
 	/* Initialize guest external interrupt line management */
diff --git a/arch/riscv/kvm/aia_device.c b/arch/riscv/kvm/aia_device.c
index 3d1e81e2a36b..5b45b1114425 100644
--- a/arch/riscv/kvm/aia_device.c
+++ b/arch/riscv/kvm/aia_device.c
@@ -71,7 +71,7 @@ static int aia_config(struct kvm *kvm, unsigned long type,
 				 * external interrupts (i.e. non-zero
 				 * VS-level IMSIC pages).
 				 */
-				if (!kvm_riscv_aia_nr_hgei)
+				if (!kvm_riscv_aia_hgei_enabled)
 					return -EINVAL;
 				break;
 			default:
@@ -628,7 +628,7 @@ void kvm_riscv_aia_init_vm(struct kvm *kvm)
 	 */
 
 	/* Initialize default values in AIA global context */
-	aia->mode = (kvm_riscv_aia_nr_hgei) ?
+	aia->mode = (kvm_riscv_aia_hgei_enabled) ?
 		KVM_DEV_RISCV_AIA_MODE_AUTO : KVM_DEV_RISCV_AIA_MODE_EMUL;
 	aia->nr_ids = kvm_riscv_aia_max_ids - 1;
 	aia->nr_sources = 0;
diff --git a/arch/riscv/kvm/main.c b/arch/riscv/kvm/main.c
index cb8a65273c1f..bcfa139f4871 100644
--- a/arch/riscv/kvm/main.c
+++ b/arch/riscv/kvm/main.c
@@ -169,8 +169,7 @@ static int __init riscv_kvm_init(void)
 	kvm_info("VMID %ld bits available\n", kvm_riscv_gstage_vmid_bits());
 
 	if (kvm_riscv_aia_available())
-		kvm_info("AIA available with %d guest external interrupts\n",
-			 kvm_riscv_aia_nr_hgei);
+		kvm_info("AIA available with guest external interrupts\n");
 
 	kvm_riscv_setup_vendor_features();
 
-- 
2.40.1


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

[PATCH 3/3] irqchip/riscv-imsic: Remove global nr_guest_files after KVM AIA per-HART conversion

[guoren]

From: "Guo Ren (Alibaba DAMO Academy)" <guoren@kernel.org>

With the recent KVM AIA changes, HGEI line management is now fully
per-CPU (via struct aia_hgei_control::nr_hgei) and the global
kvm_riscv_aia_nr_hgei has been replaced by a simple enabled flag.
The IMSIC driver no longer needs to compute and enforce a global
minimum number of guest interrupt files across all HARTs.

Remove:
 - `u32 nr_guest_files` from `struct imsic_global_config`
 - the initial `BIT(global->guest_index_bits) - 1` assignment
 - the entire per-CPU MMIO-based min() calculation and its comment
   (which was specifically there to protect old KVM)

The per-HART guest file count is already handled locally in
`imsic_local_config` during the parent IRQ loop, so this global
field was redundant.

This completes the cleanup series, eliminates the last global
assumption about guest files, and improves support for heterogeneous
(big.LITTLE / multi-vendor) RISC-V platforms.

Signed-off-by: Guo Ren (Alibaba DAMO Academy) <guoren@kernel.org>
---
 drivers/irqchip/irq-riscv-imsic-state.c | 12 +-----------
 include/linux/irqchip/riscv-imsic.h     |  3 ---
 2 files changed, 1 insertion(+), 14 deletions(-)

diff --git a/drivers/irqchip/irq-riscv-imsic-state.c b/drivers/irqchip/irq-riscv-imsic-state.c
index e3ed874d89e7..fef27247a34f 100644
--- a/drivers/irqchip/irq-riscv-imsic-state.c
+++ b/drivers/irqchip/irq-riscv-imsic-state.c
@@ -784,7 +784,7 @@ static int __init imsic_parse_fwnode(struct fwnode_handle *fwnode,
 
 int __init imsic_setup_state(struct fwnode_handle *fwnode, void *opaque)
 {
-	u32 i, j, index, nr_parent_irqs, nr_mmios, nr_guest_files, nr_handlers = 0;
+	u32 i, j, index, nr_parent_irqs, nr_mmios, nr_handlers = 0;
 	struct imsic_global_config *global;
 	struct imsic_local_config *local;
 	void __iomem **mmios_va = NULL;
@@ -878,7 +878,6 @@ int __init imsic_setup_state(struct fwnode_handle *fwnode, void *opaque)
 	}
 
 	/* Configure handlers for target CPUs */
-	global->nr_guest_files = BIT(global->guest_index_bits) - 1;
 	for (i = 0; i < nr_parent_irqs; i++) {
 		rc = imsic_get_parent_hartid(fwnode, i, &hartid);
 		if (rc) {
@@ -919,15 +918,6 @@ int __init imsic_setup_state(struct fwnode_handle *fwnode, void *opaque)
 		local->msi_pa = mmios[index].start + reloff;
 		local->msi_va = mmios_va[index] + reloff;
 
-		/*
-		 * KVM uses global->nr_guest_files to determine the available guest
-		 * interrupt files on each CPU. Take the minimum number of guest
-		 * interrupt files across all CPUs to avoid KVM incorrectly allocating
-		 * an unexisted or unmapped guest interrupt file on some CPUs.
-		 */
-		nr_guest_files = (resource_size(&mmios[index]) - reloff) / IMSIC_MMIO_PAGE_SZ - 1;
-		global->nr_guest_files = min(global->nr_guest_files, nr_guest_files);
-
 		nr_handlers++;
 	}
 
diff --git a/include/linux/irqchip/riscv-imsic.h b/include/linux/irqchip/riscv-imsic.h
index 4b348836de7a..7f3ff5c5ea53 100644
--- a/include/linux/irqchip/riscv-imsic.h
+++ b/include/linux/irqchip/riscv-imsic.h
@@ -68,9 +68,6 @@ struct imsic_global_config {
 	/* Number of guest interrupt identities */
 	u32					nr_guest_ids;
 
-	/* Number of guest interrupt files per core */
-	u32					nr_guest_files;
-
 	/* Per-CPU IMSIC addresses */
 	struct imsic_local_config __percpu	*local;
 };
-- 
2.40.1


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH 3/3] irqchip/riscv-imsic: Remove global nr_guest_files after KVM AIA per-HART conversion

[Guo Ren]

Hi all,

This patch simply removed `nr_guest_files`, which is suboptimal. For
v2, I will keep `nr_guest_files` inside `struct imsic_local_config`
(per-HART) and compute the effective HGEI count in
`kvm_riscv_aia_enable()` as:

min(hgctrl->nr_hgei, lc->nr_guest_files)

This provides a more accurate and robust per-HART configuration.

Here are the updated two patches for your review:

    irqchip/riscv-imsic: Move nr_guest_files to per-HART local config

    With the recent KVM AIA per-HART HGEI conversion, the global
    nr_guest_files is no longer appropriate. Different HARTs in
    heterogeneous SoCs may have different numbers of guest interrupt
    files.

    Move `nr_guest_files` from `struct imsic_global_config` to
    `struct imsic_local_config`, and compute it per-CPU in
    imsic_setup_state() based on the actual MMIO guest file region size.

    Update the related comment to reflect that KVM now uses the
    per-HART value.

    This eliminates the last global assumption about guest files and
    completes the per-HART conversion series for RISC-V AIA/IMSIC.

    Signed-off-by: Guo Ren (Alibaba DAMO Academy) <guoren@kernel.org>

diff --git a/drivers/irqchip/irq-riscv-imsic-state.c
b/drivers/irqchip/irq-riscv-imsic-state.c
index e3ed874d89e7..3d455ef43298 100644
--- a/drivers/irqchip/irq-riscv-imsic-state.c
+++ b/drivers/irqchip/irq-riscv-imsic-state.c
@@ -784,7 +784,7 @@ static int __init imsic_parse_fwnode(struct
fwnode_handle *fwnode,

 int __init imsic_setup_state(struct fwnode_handle *fwnode, void *opaque)
 {
-       u32 i, j, index, nr_parent_irqs, nr_mmios, nr_guest_files,
nr_handlers = 0;
+       u32 i, j, index, nr_parent_irqs, nr_mmios, nr_handlers = 0;
        struct imsic_global_config *global;
        struct imsic_local_config *local;
        void __iomem **mmios_va = NULL;
@@ -878,7 +878,6 @@ int __init imsic_setup_state(struct fwnode_handle
*fwnode, void *opaque)
        }

        /* Configure handlers for target CPUs */
-       global->nr_guest_files = BIT(global->guest_index_bits) - 1;
        for (i = 0; i < nr_parent_irqs; i++) {
                rc = imsic_get_parent_hartid(fwnode, i, &hartid);
                if (rc) {
@@ -920,13 +919,13 @@ int __init imsic_setup_state(struct
fwnode_handle *fwnode, void *opaque)
                local->msi_va = mmios_va[index] + reloff;

                /*
-                * KVM uses global->nr_guest_files to determine the
available guest
+                * KVM uses local->nr_guest_files to determine the
available guest
                 * interrupt files on each CPU. Take the minimum number of guest
                 * interrupt files across all CPUs to avoid KVM
incorrectly allocating
                 * an unexisted or unmapped guest interrupt file on some CPUs.
                 */
-               nr_guest_files = (resource_size(&mmios[index]) -
reloff) / IMSIC_MMIO_PAGE_SZ - 1;
-               global->nr_guest_files = min(global->nr_guest_files,
nr_guest_files);
+               local->nr_guest_files =
+                       (resource_size(&mmios[index]) - reloff) /
IMSIC_MMIO_PAGE_SZ - 1;

                nr_handlers++;
        }
diff --git a/include/linux/irqchip/riscv-imsic.h
b/include/linux/irqchip/riscv-imsic.h
index 4b348836de7a..13e8bd7ff4b4 100644
--- a/include/linux/irqchip/riscv-imsic.h
+++ b/include/linux/irqchip/riscv-imsic.h
@@ -40,6 +40,9 @@
 struct imsic_local_config {
        phys_addr_t                             msi_pa;
        void __iomem                            *msi_va;
+
+       /* Number of guest interrupt files per core */
+       u32                                     nr_guest_files;
 };

 struct imsic_global_config {
@@ -68,9 +71,6 @@ struct imsic_global_config {
        /* Number of guest interrupt identities */
        u32                                     nr_guest_ids;

-       /* Number of guest interrupt files per core */
-       u32                                     nr_guest_files;
-
        /* Per-CPU IMSIC addresses */
        struct imsic_local_config __percpu      *local;
 };

------------------------
The other one:

    RISC-V: KVM: AIA: Use per-HART IMSIC guest files to compute final HGEI count

    Now that `nr_guest_files` has been moved to `struct imsic_local_config`
    and is computed per-HART, KVM must respect the actual number of guest
    interrupt files available on each HART when setting up HGEI.

    In `kvm_riscv_aia_enable()`:

     - Retrieve the per-CPU IMSIC local config
     - Take `min(hgctrl->nr_hgei, lc->nr_guest_files)` as the final usable
       HGEI count for this HART
     - Use the result to initialize `free_bitmap`

    This ensures correct HGEI allocation on heterogeneous RISC-V SoCs
    where different cores may have different IMSIC guest file counts, and
    completes the per-HART conversion series.

    Signed-off-by: Guo Ren (Alibaba DAMO Academy) <guoren@kernel.org>

diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
index 70ff1d25dd99..a5f4b7fe1dce 100644
--- a/arch/riscv/kvm/aia.c
+++ b/arch/riscv/kvm/aia.c
@@ -534,6 +534,9 @@ static void aia_hgei_exit(void)
 void kvm_riscv_aia_enable(void)
 {
        struct aia_hgei_control *hgctrl;
+       const struct imsic_global_config *gc;
+       const struct imsic_local_config *lc;
+       unsigned int nr_hgei;

        if (!kvm_riscv_aia_available())
                return;
@@ -547,8 +550,13 @@ void kvm_riscv_aia_enable(void)
        if (hgctrl->nr_hgei)
                hgctrl->nr_hgei--;

-       if (hgctrl->nr_hgei) {
-               hgctrl->free_bitmap = BIT(hgctrl->nr_hgei + 1) - 1;
+       gc = imsic_get_global_config();
+       lc = (gc) ? this_cpu_ptr(gc->local) : NULL;
+       if (lc)
+               nr_hgei = min(hgctrl->nr_hgei, lc->nr_guest_files);
+
+       if (nr_hgei) {
+               hgctrl->free_bitmap = BIT(nr_hgei + 1) - 1;
                hgctrl->free_bitmap &= ~BIT(0);
        } else {
                hgctrl->free_bitmap = 0;


On Tue, Apr 21, 2026 at 10:55 PM <guoren@kernel.org> wrote:
>
> From: "Guo Ren (Alibaba DAMO Academy)" <guoren@kernel.org>
>
> With the recent KVM AIA changes, HGEI line management is now fully
> per-CPU (via struct aia_hgei_control::nr_hgei) and the global
> kvm_riscv_aia_nr_hgei has been replaced by a simple enabled flag.
> The IMSIC driver no longer needs to compute and enforce a global
> minimum number of guest interrupt files across all HARTs.
>
> Remove:
>  - `u32 nr_guest_files` from `struct imsic_global_config`
>  - the initial `BIT(global->guest_index_bits) - 1` assignment
>  - the entire per-CPU MMIO-based min() calculation and its comment
>    (which was specifically there to protect old KVM)
>
> The per-HART guest file count is already handled locally in
> `imsic_local_config` during the parent IRQ loop, so this global
> field was redundant.
>
> This completes the cleanup series, eliminates the last global
> assumption about guest files, and improves support for heterogeneous
> (big.LITTLE / multi-vendor) RISC-V platforms.
>
> Signed-off-by: Guo Ren (Alibaba DAMO Academy) <guoren@kernel.org>
> ---
>  drivers/irqchip/irq-riscv-imsic-state.c | 12 +-----------
>  include/linux/irqchip/riscv-imsic.h     |  3 ---
>  2 files changed, 1 insertion(+), 14 deletions(-)
>
> diff --git a/drivers/irqchip/irq-riscv-imsic-state.c b/drivers/irqchip/irq-riscv-imsic-state.c
> index e3ed874d89e7..fef27247a34f 100644
> --- a/drivers/irqchip/irq-riscv-imsic-state.c
> +++ b/drivers/irqchip/irq-riscv-imsic-state.c
> @@ -784,7 +784,7 @@ static int __init imsic_parse_fwnode(struct fwnode_handle *fwnode,
>
>  int __init imsic_setup_state(struct fwnode_handle *fwnode, void *opaque)
>  {
> -       u32 i, j, index, nr_parent_irqs, nr_mmios, nr_guest_files, nr_handlers = 0;
> +       u32 i, j, index, nr_parent_irqs, nr_mmios, nr_handlers = 0;
>         struct imsic_global_config *global;
>         struct imsic_local_config *local;
>         void __iomem **mmios_va = NULL;
> @@ -878,7 +878,6 @@ int __init imsic_setup_state(struct fwnode_handle *fwnode, void *opaque)
>         }
>
>         /* Configure handlers for target CPUs */
> -       global->nr_guest_files = BIT(global->guest_index_bits) - 1;
>         for (i = 0; i < nr_parent_irqs; i++) {
>                 rc = imsic_get_parent_hartid(fwnode, i, &hartid);
>                 if (rc) {
> @@ -919,15 +918,6 @@ int __init imsic_setup_state(struct fwnode_handle *fwnode, void *opaque)
>                 local->msi_pa = mmios[index].start + reloff;
>                 local->msi_va = mmios_va[index] + reloff;
>
> -               /*
> -                * KVM uses global->nr_guest_files to determine the available guest
> -                * interrupt files on each CPU. Take the minimum number of guest
> -                * interrupt files across all CPUs to avoid KVM incorrectly allocating
> -                * an unexisted or unmapped guest interrupt file on some CPUs.
> -                */
> -               nr_guest_files = (resource_size(&mmios[index]) - reloff) / IMSIC_MMIO_PAGE_SZ - 1;
> -               global->nr_guest_files = min(global->nr_guest_files, nr_guest_files);
> -
>                 nr_handlers++;
>         }
>
> diff --git a/include/linux/irqchip/riscv-imsic.h b/include/linux/irqchip/riscv-imsic.h
> index 4b348836de7a..7f3ff5c5ea53 100644
> --- a/include/linux/irqchip/riscv-imsic.h
> +++ b/include/linux/irqchip/riscv-imsic.h
> @@ -68,9 +68,6 @@ struct imsic_global_config {
>         /* Number of guest interrupt identities */
>         u32                                     nr_guest_ids;
>
> -       /* Number of guest interrupt files per core */
> -       u32                                     nr_guest_files;
> -
>         /* Per-CPU IMSIC addresses */
>         struct imsic_local_config __percpu      *local;
>  };
> --
> 2.40.1
>


--
Best Regards
 Guo Ren

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH 3/3] irqchip/riscv-imsic: Remove global nr_guest_files after KVM AIA per-HART conversion

[Anup Patel]

On Wed, Apr 22, 2026 at 8:39 AM Guo Ren <guoren@kernel.org> wrote:
>
> Hi all,
>
> This patch simply removed `nr_guest_files`, which is suboptimal. For
> v2, I will keep `nr_guest_files` inside `struct imsic_local_config`
> (per-HART) and compute the effective HGEI count in
> `kvm_riscv_aia_enable()` as:
>
> min(hgctrl->nr_hgei, lc->nr_guest_files)

Yes, I was going to comment the exact same thing. Good that you
realized it early.

Regards,
Anup

>
> This provides a more accurate and robust per-HART configuration.
>
> Here are the updated two patches for your review:
>
>     irqchip/riscv-imsic: Move nr_guest_files to per-HART local config
>
>     With the recent KVM AIA per-HART HGEI conversion, the global
>     nr_guest_files is no longer appropriate. Different HARTs in
>     heterogeneous SoCs may have different numbers of guest interrupt
>     files.
>
>     Move `nr_guest_files` from `struct imsic_global_config` to
>     `struct imsic_local_config`, and compute it per-CPU in
>     imsic_setup_state() based on the actual MMIO guest file region size.
>
>     Update the related comment to reflect that KVM now uses the
>     per-HART value.
>
>     This eliminates the last global assumption about guest files and
>     completes the per-HART conversion series for RISC-V AIA/IMSIC.
>
>     Signed-off-by: Guo Ren (Alibaba DAMO Academy) <guoren@kernel.org>
>
> diff --git a/drivers/irqchip/irq-riscv-imsic-state.c
> b/drivers/irqchip/irq-riscv-imsic-state.c
> index e3ed874d89e7..3d455ef43298 100644
> --- a/drivers/irqchip/irq-riscv-imsic-state.c
> +++ b/drivers/irqchip/irq-riscv-imsic-state.c
> @@ -784,7 +784,7 @@ static int __init imsic_parse_fwnode(struct
> fwnode_handle *fwnode,
>
>  int __init imsic_setup_state(struct fwnode_handle *fwnode, void *opaque)
>  {
> -       u32 i, j, index, nr_parent_irqs, nr_mmios, nr_guest_files,
> nr_handlers = 0;
> +       u32 i, j, index, nr_parent_irqs, nr_mmios, nr_handlers = 0;
>         struct imsic_global_config *global;
>         struct imsic_local_config *local;
>         void __iomem **mmios_va = NULL;
> @@ -878,7 +878,6 @@ int __init imsic_setup_state(struct fwnode_handle
> *fwnode, void *opaque)
>         }
>
>         /* Configure handlers for target CPUs */
> -       global->nr_guest_files = BIT(global->guest_index_bits) - 1;
>         for (i = 0; i < nr_parent_irqs; i++) {
>                 rc = imsic_get_parent_hartid(fwnode, i, &hartid);
>                 if (rc) {
> @@ -920,13 +919,13 @@ int __init imsic_setup_state(struct
> fwnode_handle *fwnode, void *opaque)
>                 local->msi_va = mmios_va[index] + reloff;
>
>                 /*
> -                * KVM uses global->nr_guest_files to determine the
> available guest
> +                * KVM uses local->nr_guest_files to determine the
> available guest
>                  * interrupt files on each CPU. Take the minimum number of guest
>                  * interrupt files across all CPUs to avoid KVM
> incorrectly allocating
>                  * an unexisted or unmapped guest interrupt file on some CPUs.
>                  */
> -               nr_guest_files = (resource_size(&mmios[index]) -
> reloff) / IMSIC_MMIO_PAGE_SZ - 1;
> -               global->nr_guest_files = min(global->nr_guest_files,
> nr_guest_files);
> +               local->nr_guest_files =
> +                       (resource_size(&mmios[index]) - reloff) /
> IMSIC_MMIO_PAGE_SZ - 1;
>
>                 nr_handlers++;
>         }
> diff --git a/include/linux/irqchip/riscv-imsic.h
> b/include/linux/irqchip/riscv-imsic.h
> index 4b348836de7a..13e8bd7ff4b4 100644
> --- a/include/linux/irqchip/riscv-imsic.h
> +++ b/include/linux/irqchip/riscv-imsic.h
> @@ -40,6 +40,9 @@
>  struct imsic_local_config {
>         phys_addr_t                             msi_pa;
>         void __iomem                            *msi_va;
> +
> +       /* Number of guest interrupt files per core */
> +       u32                                     nr_guest_files;
>  };
>
>  struct imsic_global_config {
> @@ -68,9 +71,6 @@ struct imsic_global_config {
>         /* Number of guest interrupt identities */
>         u32                                     nr_guest_ids;
>
> -       /* Number of guest interrupt files per core */
> -       u32                                     nr_guest_files;
> -
>         /* Per-CPU IMSIC addresses */
>         struct imsic_local_config __percpu      *local;
>  };
>
> ------------------------
> The other one:
>
>     RISC-V: KVM: AIA: Use per-HART IMSIC guest files to compute final HGEI count
>
>     Now that `nr_guest_files` has been moved to `struct imsic_local_config`
>     and is computed per-HART, KVM must respect the actual number of guest
>     interrupt files available on each HART when setting up HGEI.
>
>     In `kvm_riscv_aia_enable()`:
>
>      - Retrieve the per-CPU IMSIC local config
>      - Take `min(hgctrl->nr_hgei, lc->nr_guest_files)` as the final usable
>        HGEI count for this HART
>      - Use the result to initialize `free_bitmap`
>
>     This ensures correct HGEI allocation on heterogeneous RISC-V SoCs
>     where different cores may have different IMSIC guest file counts, and
>     completes the per-HART conversion series.
>
>     Signed-off-by: Guo Ren (Alibaba DAMO Academy) <guoren@kernel.org>
>
> diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
> index 70ff1d25dd99..a5f4b7fe1dce 100644
> --- a/arch/riscv/kvm/aia.c
> +++ b/arch/riscv/kvm/aia.c
> @@ -534,6 +534,9 @@ static void aia_hgei_exit(void)
>  void kvm_riscv_aia_enable(void)
>  {
>         struct aia_hgei_control *hgctrl;
> +       const struct imsic_global_config *gc;
> +       const struct imsic_local_config *lc;
> +       unsigned int nr_hgei;
>
>         if (!kvm_riscv_aia_available())
>                 return;
> @@ -547,8 +550,13 @@ void kvm_riscv_aia_enable(void)
>         if (hgctrl->nr_hgei)
>                 hgctrl->nr_hgei--;
>
> -       if (hgctrl->nr_hgei) {
> -               hgctrl->free_bitmap = BIT(hgctrl->nr_hgei + 1) - 1;
> +       gc = imsic_get_global_config();
> +       lc = (gc) ? this_cpu_ptr(gc->local) : NULL;
> +       if (lc)
> +               nr_hgei = min(hgctrl->nr_hgei, lc->nr_guest_files);
> +
> +       if (nr_hgei) {
> +               hgctrl->free_bitmap = BIT(nr_hgei + 1) - 1;
>                 hgctrl->free_bitmap &= ~BIT(0);
>         } else {
>                 hgctrl->free_bitmap = 0;
>
>
> On Tue, Apr 21, 2026 at 10:55 PM <guoren@kernel.org> wrote:
> >
> > From: "Guo Ren (Alibaba DAMO Academy)" <guoren@kernel.org>
> >
> > With the recent KVM AIA changes, HGEI line management is now fully
> > per-CPU (via struct aia_hgei_control::nr_hgei) and the global
> > kvm_riscv_aia_nr_hgei has been replaced by a simple enabled flag.
> > The IMSIC driver no longer needs to compute and enforce a global
> > minimum number of guest interrupt files across all HARTs.
> >
> > Remove:
> >  - `u32 nr_guest_files` from `struct imsic_global_config`
> >  - the initial `BIT(global->guest_index_bits) - 1` assignment
> >  - the entire per-CPU MMIO-based min() calculation and its comment
> >    (which was specifically there to protect old KVM)
> >
> > The per-HART guest file count is already handled locally in
> > `imsic_local_config` during the parent IRQ loop, so this global
> > field was redundant.
> >
> > This completes the cleanup series, eliminates the last global
> > assumption about guest files, and improves support for heterogeneous
> > (big.LITTLE / multi-vendor) RISC-V platforms.
> >
> > Signed-off-by: Guo Ren (Alibaba DAMO Academy) <guoren@kernel.org>
> > ---
> >  drivers/irqchip/irq-riscv-imsic-state.c | 12 +-----------
> >  include/linux/irqchip/riscv-imsic.h     |  3 ---
> >  2 files changed, 1 insertion(+), 14 deletions(-)
> >
> > diff --git a/drivers/irqchip/irq-riscv-imsic-state.c b/drivers/irqchip/irq-riscv-imsic-state.c
> > index e3ed874d89e7..fef27247a34f 100644
> > --- a/drivers/irqchip/irq-riscv-imsic-state.c
> > +++ b/drivers/irqchip/irq-riscv-imsic-state.c
> > @@ -784,7 +784,7 @@ static int __init imsic_parse_fwnode(struct fwnode_handle *fwnode,
> >
> >  int __init imsic_setup_state(struct fwnode_handle *fwnode, void *opaque)
> >  {
> > -       u32 i, j, index, nr_parent_irqs, nr_mmios, nr_guest_files, nr_handlers = 0;
> > +       u32 i, j, index, nr_parent_irqs, nr_mmios, nr_handlers = 0;
> >         struct imsic_global_config *global;
> >         struct imsic_local_config *local;
> >         void __iomem **mmios_va = NULL;
> > @@ -878,7 +878,6 @@ int __init imsic_setup_state(struct fwnode_handle *fwnode, void *opaque)
> >         }
> >
> >         /* Configure handlers for target CPUs */
> > -       global->nr_guest_files = BIT(global->guest_index_bits) - 1;
> >         for (i = 0; i < nr_parent_irqs; i++) {
> >                 rc = imsic_get_parent_hartid(fwnode, i, &hartid);
> >                 if (rc) {
> > @@ -919,15 +918,6 @@ int __init imsic_setup_state(struct fwnode_handle *fwnode, void *opaque)
> >                 local->msi_pa = mmios[index].start + reloff;
> >                 local->msi_va = mmios_va[index] + reloff;
> >
> > -               /*
> > -                * KVM uses global->nr_guest_files to determine the available guest
> > -                * interrupt files on each CPU. Take the minimum number of guest
> > -                * interrupt files across all CPUs to avoid KVM incorrectly allocating
> > -                * an unexisted or unmapped guest interrupt file on some CPUs.
> > -                */
> > -               nr_guest_files = (resource_size(&mmios[index]) - reloff) / IMSIC_MMIO_PAGE_SZ - 1;
> > -               global->nr_guest_files = min(global->nr_guest_files, nr_guest_files);
> > -
> >                 nr_handlers++;
> >         }
> >
> > diff --git a/include/linux/irqchip/riscv-imsic.h b/include/linux/irqchip/riscv-imsic.h
> > index 4b348836de7a..7f3ff5c5ea53 100644
> > --- a/include/linux/irqchip/riscv-imsic.h
> > +++ b/include/linux/irqchip/riscv-imsic.h
> > @@ -68,9 +68,6 @@ struct imsic_global_config {
> >         /* Number of guest interrupt identities */
> >         u32                                     nr_guest_ids;
> >
> > -       /* Number of guest interrupt files per core */
> > -       u32                                     nr_guest_files;
> > -
> >         /* Per-CPU IMSIC addresses */
> >         struct imsic_local_config __percpu      *local;
> >  };
> > --
> > 2.40.1
> >
>
>
> --
> Best Regards
>  Guo Ren

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv