Re: [PATCH] PCI: brcmstb: Reserve only the MSI vectors that are handed out

[Bjorn Helgaas <helgaas@kernel.org>]

On Thu, Oct 01, 2026 at 09:29:27PM +0200, Thomas Gleixner wrote:
> On Mon, Sep 21 2026 at 18:05, Han / 한상우Sangwoo wrote:
> > I tested Thomas's patch on the same hardware setup with the 5-vector
> > MSI endpoint. The patch was applied as posted on top of 6.12.93
> > (rpi-6.12.y).
> >
> > With the patch applied:
> >
> > - The MSI base hwirq remained at 0x8 across 200 driver reload cycles.
> > - The driver got all 5 requested vectors on every cycle, with no
> > single-MSI fallback. Multiple Message Enable remained at 8.
> > - The brcmstb inner-domain mapping returned to baseline after each
> > unload/reload, with 12 mapped while the driver was loaded and 4 after
> > unload.
> > - A kprobe showed one allocation of 8 vectors followed by eight
> > single-vector frees, with nothing left over.
> 
> 8 single vector frees?
> 
> Seems I got something wrong there verus the bulk free. Updated patch
> below.
> 
> > The MSI vector exhaustion issue I originally observed no longer
> > reproduces with the patch.
> 
> Good. Can you please retest with the updated patch?

Sangwoo, any update on this?

I don't see this patch in linux-next (next-20261012), so I don't think
this will be fixed for v7.4 unless something moves it ahead.

> > I also observed a KASAN report with managed affinity. Using a small
> > out-of-tree test module bound to the same endpoint and requesting 1..3
> > vectors with PCI_IRQ_MSI | PCI_IRQ_AFFINITY, a request for 3 vectors
> > resulted in:
> >
> > BUG: KASAN: slab-out-of-bounds in __irq_alloc_descs+0x158/0x460
> >
> > Requests for 5 and 7 vectors were capped to 4 on this 4-CPU system and
> > did not trigger the report. I have not checked this against the
> > unpatched kernel yet, so I cannot tell whether it is related to the
> > patch.
> 
> Any updates on that? Also please provide the source for that test.
> 
> Thanks,
> 
>         tglx
> ---
> --- a/kernel/irq/irqdomain.c
> +++ b/kernel/irq/irqdomain.c
> @@ -1610,6 +1610,17 @@ static void irq_domain_free_irqs_hierarc
>  	if (!domain->ops->free)
>  		return;
>  
> +	/*
> +	 * MSI device domains are capable of bulk free.
> +	 *
> +	 * CHECKME: Are all MSI parent domains capable?
> +	 */
> +	if (domain->flags & (IRQ_DOMAIN_FLAG_MSI_DEVICE | IRQ_DOMAIN_FLAG_MSI_PARENT)) {
> +		if (irq_domain_get_irq_data(domain, irq_base))
> +			domain->ops->free(domain, irq_base, nr_irqs);
> +		return;
> +	}
> +
>  	for (i = 0; i < nr_irqs; i++) {
>  		if (irq_domain_get_irq_data(domain, irq_base + i))
>  			domain->ops->free(domain, irq_base + i, 1);
> --- a/kernel/irq/msi.c
> +++ b/kernel/irq/msi.c
> @@ -1333,20 +1333,28 @@ static int __msi_domain_alloc_irqs(struc
>  
>  		ops->set_desc(&arg, desc);
>  
> -		virq = __irq_domain_alloc_irqs(domain, -1, desc->nvec_used,
> +		/* Make sure a MULTI-MSI allocation is power of two */
> +		unsigned int nvec_aligned = roundup_pow_of_two(desc->nvec_used);
> +
> +		virq = __irq_domain_alloc_irqs(domain, -1, nvec_aligned,
>  					       dev_to_node(dev), &arg, false,
>  					       desc->affinity);
>  		if (virq < 0)
>  			return msi_handle_pci_fail(domain, desc, allocated);
>  
> -		for (i = 0; i < desc->nvec_used; i++) {
> +		for (i = 0; i < nvec_aligned; i++) {
>  			irq_set_msi_desc_off(virq, i, desc);
>  			irq_debugfs_copy_devname(virq + i, dev);
>  			ret = msi_init_virq(domain, virq + i, vflags);
>  			if (ret)
>  				return ret;
>  		}
> +
>  		if (info->flags & MSI_FLAG_DEV_SYSFS) {
> +			/*
> +			 * This only exposes desc->nvec_used and ignores the
> +			 * overallocated MULTI-MSI ones.
> +			 */
>  			ret = msi_sysfs_populate_desc(dev, desc);
>  			if (ret)
>  				return ret;
> @@ -1610,13 +1618,15 @@ static void __msi_domain_free_irqs(struc
>  			continue;
>  
>  		/* Make sure all interrupts are deactivated */
> -		for (i = 0; i < desc->nvec_used; i++) {
> +		unsigned int nvec_aligned = roundup_pow_of_two(desc->nvec_used);
> +
> +		for (i = 0; i < nvec_aligned; i++) {
>  			irqd = irq_domain_get_irq_data(domain, desc->irq + i);
>  			if (irqd && irqd_is_activated(irqd))
>  				irq_domain_deactivate_irq(irqd);
>  		}
>  
> -		irq_domain_free_irqs(desc->irq, desc->nvec_used);
> +		irq_domain_free_irqs(desc->irq, nvec_aligned);
>  		if (info->flags & MSI_FLAG_DEV_SYSFS)
>  			msi_sysfs_remove_desc(dev, desc);
>  		desc->irq = 0;