[PATCH RFC v2 0/2] irqchip/ls-scfg-msi: add multi-MSI support

[PATCH RFC v2 0/2] irqchip/ls-scfg-msi: add multi-MSI support

[Alexander Wilhelm]

This series enables multi-MSI (nr_irqs > 1) allocation on the Layerscape
SCFG MSI controller so PCI endpoints that request contiguous MSI vector
blocks (for example ath12k, which asks for 16 vectors: 3 MHI + 5 CE + 8 DP)
no longer fall back to single-MSI operation with all interrupts multiplexed
onto one CPU.

The first patch is a preparation refactor: switch the hwirq bookkeeping
from find_first_zero_bit() / __set_bit() to bitmap_find_free_region() /
bitmap_release_region(), release the region on iommu_dma_prepare_msi()
error, and loop irq_domain_set_info() over nr_irqs. For the current
single-MSI case (order 0) this is functionally equivalent; the only
externally visible change is -ENOMEM instead of -ENOSPC on exhaustion.

The second patch enables MSI_FLAG_MULTI_PCI_MSI on the parent domain, drops
the WARN_ON(nr_irqs != 1) guard, and statically pins each MSIR's chained
parent IRQ to its matching CPU in no-affinity mode. That pinning is
required because affinity mode only releases every (1 << ibs_shift)-th
hwirq and cannot satisfy aligned power-of-two allocations of size > 1;
users of multi-MSI must therefore boot with lsmsi=no-affinity.

Signed-off-by: Alexander Wilhelm <alexander.wilhelm@westermo.com>
---
Changes in v2:
- Rebase on latest torvalds/master
- Adapt code to match kernel style guide
- Improve commit descriptions
- Use round-robin for MSIR distribution instead of ignoring the surplus
- Link to v1: https://lore.kernel.org/r/20260716-irqchip-ls-scfg-msi-add-multi-msi-support-v1-0-9795356d0ebc@westermo.com

---
Alexander Wilhelm (2):
      irqchip/ls-scfg-msi: refactor allocation to bitmap_find_free_region()
      irqchip/ls-scfg-msi: enable multi-MSI allocation

 drivers/irqchip/irq-ls-scfg-msi.c | 55 +++++++++++++++++++++++----------------
 1 file changed, 32 insertions(+), 23 deletions(-)
---
base-commit: 248951ddc14de84de3910f9b13f51491a8cd91df
change-id: 20260716-irqchip-ls-scfg-msi-add-multi-msi-support-5e7538460704

Best regards,
-- 
Alexander Wilhelm <alexander.wilhelm@westermo.com>

[PATCH RFC v2 1/2] irqchip/ls-scfg-msi: refactor allocation to bitmap_find_free_region()

[Alexander Wilhelm]

MSI hwirqs are allocated from a per-instance bitmap. The current
bookkeeping picks and marks free bits by hand, treats every allocation as a
single vector, and on iommu prepare failure returns without releasing the
bit that was just reserved.

Enabling multi-MSI requires atomically reserving an aligned power-of-two
region under the lock, which the split find/mark sequence cannot express,
and every callsite around it must handle a range of vectors instead of one.
The current error path also leaks a hwirq on every iommu prepare failure.

Switch to the bitmap_find_free_region() family, which reserves
order-aligned regions atomically, release on the iommu error path, and
range-check and iterate over the requested vector count. The single-vector
case stays functionally equivalent; the only externally visible change is
-ENOMEM instead of -ENOSPC on exhaustion.

Assisted-by: Copilot:claude-opus-4.7
Signed-off-by: Alexander Wilhelm <alexander.wilhelm@westermo.com>
---
 drivers/irqchip/irq-ls-scfg-msi.c | 42 ++++++++++++++++++++-------------------
 1 file changed, 22 insertions(+), 20 deletions(-)

diff --git a/drivers/irqchip/irq-ls-scfg-msi.c b/drivers/irqchip/irq-ls-scfg-msi.c
index 4910f364e568..50bd84644769 100644
--- a/drivers/irqchip/irq-ls-scfg-msi.c
+++ b/drivers/irqchip/irq-ls-scfg-msi.c
@@ -139,30 +139,32 @@ static int ls_scfg_msi_domain_irq_alloc(struct irq_domain *domain,
 					unsigned int nr_irqs,
 					void *args)
 {
-	msi_alloc_info_t *info = args;
 	struct ls_scfg_msi *msi_data = domain->host_data;
-	int pos, err = 0;
+	int order = get_count_order(nr_irqs);
+	msi_alloc_info_t *info = args;
+	unsigned int i;
+	int pos, err;
 
 	WARN_ON(nr_irqs != 1);
 
-	spin_lock(&msi_data->lock);
-	pos = find_first_zero_bit(msi_data->used, msi_data->irqs_num);
-	if (pos < msi_data->irqs_num)
-		__set_bit(pos, msi_data->used);
-	else
-		err = -ENOSPC;
-	spin_unlock(&msi_data->lock);
+	scoped_guard(spinlock, &msi_data->lock)
+		pos = bitmap_find_free_region(msi_data->used, msi_data->irqs_num, order);
 
-	if (err)
-		return err;
+	if (pos < 0)
+		return pos;
 
 	err = iommu_dma_prepare_msi(info->desc, msi_data->msiir_addr);
-	if (err)
+	if (err) {
+		scoped_guard(spinlock, &msi_data->lock)
+			bitmap_release_region(msi_data->used, pos, order);
 		return err;
+	}
 
-	irq_domain_set_info(domain, virq, pos,
-			    &ls_scfg_msi_parent_chip, msi_data,
-			    handle_simple_irq, NULL, NULL);
+	for (i = 0; i < nr_irqs; i++) {
+		irq_domain_set_info(domain, virq + i, pos + i,
+				    &ls_scfg_msi_parent_chip, msi_data,
+				    handle_simple_irq, NULL, NULL);
+	}
 
 	return 0;
 }
@@ -172,17 +174,17 @@ static void ls_scfg_msi_domain_irq_free(struct irq_domain *domain,
 {
 	struct irq_data *d = irq_domain_get_irq_data(domain, virq);
 	struct ls_scfg_msi *msi_data = irq_data_get_irq_chip_data(d);
+	int order = get_count_order(nr_irqs);
 	int pos;
 
 	pos = d->hwirq;
-	if (pos < 0 || pos >= msi_data->irqs_num) {
-		pr_err("failed to teardown msi. Invalid hwirq %d\n", pos);
+	if (pos < 0 || pos + nr_irqs > msi_data->irqs_num) {
+		pr_err("failed to teardown msi. Invalid hwirq %d nr %u\n", pos, nr_irqs);
 		return;
 	}
 
-	spin_lock(&msi_data->lock);
-	__clear_bit(pos, msi_data->used);
-	spin_unlock(&msi_data->lock);
+	scoped_guard(spinlock, &msi_data->lock)
+		bitmap_release_region(msi_data->used, pos, order);
 }
 
 static const struct irq_domain_ops ls_scfg_msi_domain_ops = {

-- 
2.43.0

[PATCH RFC v2 2/2] irqchip/ls-scfg-msi: enable multi-MSI allocation

[Alexander Wilhelm]

PCI endpoints that ask for a contiguous multi-vector MSI block currently
fall back to single-MSI on Layerscape SCFG and multiplex every device
interrupt onto one CPU. The hwirq allocator already reserves
order-aligned power-of-two regions, but the parent domain does not
advertise multi-MSI support and a leftover single-vector guard rejects
any request for more than one vector.

Simply lifting those two restrictions is not enough. A multi-vector
block spans several MSIRs by construction of the hwirq layout, and in
no-affinity routing mode every MSIR's chained parent IRQ defaults to
CPU0 with no way to rebalance individual MSIs at runtime. Allowing
nr_irqs > 1 without a distribution hint would therefore pile all
vectors of a single device onto the boot CPU and defeat the per-CPU
scaling that multi-MSI is supposed to buy.

Advertise multi-MSI support on the parent domain and let the allocator
serve requests larger than one vector. Distribute the MSIR chained
parent IRQs across the online CPUs via a modular round-robin at setup,
so that multi-MSI-capable devices see genuine per-CPU parallelism
instead of piling every interrupt onto CPU0.

Assisted-by: Copilot:claude-opus-4.7
Signed-off-by: Alexander Wilhelm <alexander.wilhelm@westermo.com>
---
 drivers/irqchip/irq-ls-scfg-msi.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/drivers/irqchip/irq-ls-scfg-msi.c b/drivers/irqchip/irq-ls-scfg-msi.c
index 50bd84644769..6c05af6b3fa2 100644
--- a/drivers/irqchip/irq-ls-scfg-msi.c
+++ b/drivers/irqchip/irq-ls-scfg-msi.c
@@ -60,6 +60,7 @@ struct ls_scfg_msi {
 #define MPIC_MSI_FLAGS_REQUIRED (MSI_FLAG_USE_DEF_DOM_OPS | \
 				 MSI_FLAG_USE_DEF_CHIP_OPS)
 #define MPIC_MSI_FLAGS_SUPPORTED (MSI_FLAG_PCI_MSIX       | \
+				  MSI_FLAG_MULTI_PCI_MSI  | \
 				  MSI_GENERIC_FLAGS_MASK)
 
 static const struct msi_parent_ops ls_scfg_msi_parent_ops = {
@@ -145,8 +146,6 @@ static int ls_scfg_msi_domain_irq_alloc(struct irq_domain *domain,
 	unsigned int i;
 	int pos, err;
 
-	WARN_ON(nr_irqs != 1);
-
 	scoped_guard(spinlock, &msi_data->lock)
 		pos = bitmap_find_free_region(msi_data->used, msi_data->irqs_num, order);
 
@@ -267,8 +266,16 @@ static int ls_scfg_msi_setup_hwirq(struct ls_scfg_msi *msi_data, int index)
 		/* Associate MSIR interrupt to the cpu */
 		irq_set_affinity(msir->gic_irq, get_cpu_mask(index));
 		msir->srs = 0; /* This value is determined by the CPU */
-	} else
+	} else {
 		msir->srs = index;
+		/*
+		 * Distribute MSI processing across all CPUs so heavy traffic is
+		 * not throttled by a single core saturating on interrupts.
+		 * No-affinity mode disables per-IRQ rebalancing and without a
+		 * hint here every MSIR's chained handler would default to CPU0.
+		 */
+		irq_set_affinity(msir->gic_irq, get_cpu_mask(index % num_possible_cpus()));
+	}
 
 	/* Release the hwirqs corresponding to this MSIR */
 	if (!msi_affinity_flag || msir->index == 0) {

-- 
2.43.0

Re: [PATCH RFC v2 0/2] irqchip/ls-scfg-msi: add multi-MSI support

[Thomas Gleixner]

On Wed, Jul 22 2026 at 10:26, Alexander Wilhelm wrote:
> This series enables multi-MSI (nr_irqs > 1) allocation on the Layerscape
> SCFG MSI controller so PCI endpoints that request contiguous MSI vector
> blocks (for example ath12k, which asks for 16 vectors: 3 MHI + 5 CE + 8 DP)
> no longer fall back to single-MSI operation with all interrupts multiplexed
> onto one CPU.
>
> The first patch is a preparation refactor: switch the hwirq bookkeeping
> from find_first_zero_bit() / __set_bit() to bitmap_find_free_region() /
> bitmap_release_region(), release the region on iommu_dma_prepare_msi()
> error, and loop irq_domain_set_info() over nr_irqs. For the current
> single-MSI case (order 0) this is functionally equivalent; the only
> externally visible change is -ENOMEM instead of -ENOSPC on exhaustion.
>
> The second patch enables MSI_FLAG_MULTI_PCI_MSI on the parent domain, drops
> the WARN_ON(nr_irqs != 1) guard, and statically pins each MSIR's chained
> parent IRQ to its matching CPU in no-affinity mode. That pinning is
> required because affinity mode only releases every (1 << ibs_shift)-th
> hwirq and cannot satisfy aligned power-of-two allocations of size > 1;
> users of multi-MSI must therefore boot with lsmsi=no-affinity.

That needs eyeballs from the NXP people.