[PATCH 0/2] Update MSI-X irq domain hwsize

[PATCH 0/2] Update MSI-X irq domain hwsize

[Guixin Liu]

Hi,
  These two patches provide the ability to update hwsize during re-probe.
All comments are welcome.

Guixin Liu (2):
  genirq/msi: Introduce update hwsize helper
  PCI/MSI: Update MSI-X irq domain hwsize

 drivers/pci/msi/irqdomain.c |  5 ++++-
 include/linux/msi.h         |  3 +++
 kernel/irq/msi.c            | 22 ++++++++++++++++++++++
 3 files changed, 29 insertions(+), 1 deletion(-)

-- 
2.32.0.3.g01195cf9f

[PATCH 1/2] genirq/msi: Introduce update hwsize helper

[Guixin Liu]

Introduce msi_domain_update_hwsize() helper for updating hwsize.

Signed-off-by: Guixin Liu <kanie@linux.alibaba.com>
Reviewed-by: Guanghui Feng <guanghuifeng@linux.alibaba.com>
---
 include/linux/msi.h |  3 +++
 kernel/irq/msi.c    | 22 ++++++++++++++++++++++
 2 files changed, 25 insertions(+)

diff --git a/include/linux/msi.h b/include/linux/msi.h
index fa41eed62868..6de938ba2245 100644
--- a/include/linux/msi.h
+++ b/include/linux/msi.h
@@ -647,6 +647,9 @@ bool msi_create_device_irq_domain(struct device *dev, unsigned int domid,
 				  void *chip_data);
 void msi_remove_device_irq_domain(struct device *dev, unsigned int domid);
 
+int msi_domain_update_hwsize(struct device *dev, unsigned int domid,
+			     unsigned int hwsize_new);
+
 bool msi_match_device_irq_domain(struct device *dev, unsigned int domid,
 				 enum irq_domain_bus_token bus_token);
 
diff --git a/kernel/irq/msi.c b/kernel/irq/msi.c
index 3cafa40e6ce3..162e7263ef7f 100644
--- a/kernel/irq/msi.c
+++ b/kernel/irq/msi.c
@@ -1133,6 +1133,28 @@ void msi_remove_device_irq_domain(struct device *dev, unsigned int domid)
 	kfree(container_of(info, struct msi_domain_template, info));
 }
 
+int msi_domain_update_hwsize(struct device *dev, unsigned int domid,
+			     unsigned int hwsize_new)
+{
+	struct msi_domain_info *info;
+	struct irq_domain *domain;
+
+	if (hwsize_new > MSI_XA_DOMAIN_SIZE)
+		return -EINVAL;
+	if (!hwsize_new)
+		hwsize_new = MSI_XA_DOMAIN_SIZE;
+
+	domain = msi_get_device_domain(dev, domid);
+	if (!domain)
+		return -ENODEV;
+
+	info = domain->host_data;
+	if (hwsize_new != info->hwsize)
+		info->hwsize = hwsize_new;
+
+	return 0;
+}
+
 /**
  * msi_match_device_irq_domain - Match a device irq domain against a bus token
  * @dev:	Pointer to the device
-- 
2.32.0.3.g01195cf9f

[PATCH 2/2] PCI/MSI: Update MSI-X irq domain hwsize

[Guixin Liu]

After the upper-layer driver removes the device and before the next
probe, events such as firmware updates may increase the number of
interrupts supported by the device. However, the irq_domain still
retains the old hwsize, which causes subsequent interrupt allocation
failures. Update hwsize during MSI-X device domain setup to fix this
issue.

Signed-off-by: Guixin Liu <kanie@linux.alibaba.com>
Reviewed-by: Guanghui Feng <guanghuifeng@linux.alibaba.com>
---
 drivers/pci/msi/irqdomain.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/msi/irqdomain.c b/drivers/pci/msi/irqdomain.c
index 6e65f0f44112..485bfba059cc 100644
--- a/drivers/pci/msi/irqdomain.c
+++ b/drivers/pci/msi/irqdomain.c
@@ -274,8 +274,11 @@ bool pci_setup_msix_device_domain(struct pci_dev *pdev, unsigned int hwsize)
 	if (WARN_ON_ONCE(pdev->msi_enabled))
 		return false;
 
-	if (pci_match_device_domain(pdev, DOMAIN_BUS_PCI_DEVICE_MSIX))
+	if (pci_match_device_domain(pdev, DOMAIN_BUS_PCI_DEVICE_MSIX)) {
+		if (msi_domain_update_hwsize(&pdev->dev, MSI_DEFAULT_DOMAIN, hwsize))
+			pr_warn("too big MSI-X hwsize:%u\n", hwsize);
 		return true;
+	}
 	if (pci_match_device_domain(pdev, DOMAIN_BUS_PCI_DEVICE_MSI))
 		msi_remove_device_irq_domain(&pdev->dev, MSI_DEFAULT_DOMAIN);
 
-- 
2.32.0.3.g01195cf9f

Re: [PATCH 2/2] PCI/MSI: Update MSI-X irq domain hwsize

[Thomas Gleixner]

On Tue, Mar 24 2026 at 09:47, Guixin Liu wrote:
> After the upper-layer driver removes the device and before the next
> probe, events such as firmware updates may increase the number of
> interrupts supported by the device. However, the irq_domain still
> retains the old hwsize, which causes subsequent interrupt allocation
> failures. Update hwsize during MSI-X device domain setup to fix this
> issue.

When a device is removed then the corresponding struct device is torn
down, which implies that the device domain is freed as well. So how can
this end up with the old state on the next probe?

Thanks,

        tglx

Re: [PATCH 2/2] PCI/MSI: Update MSI-X irq domain hwsize

[Guixin Liu]

在 2026/3/24 21:59, Thomas Gleixner 写道:
> On Tue, Mar 24 2026 at 09:47, Guixin Liu wrote:
>> After the upper-layer driver removes the device and before the next
>> probe, events such as firmware updates may increase the number of
>> interrupts supported by the device. However, the irq_domain still
>> retains the old hwsize, which causes subsequent interrupt allocation
>> failures. Update hwsize during MSI-X device domain setup to fix this
>> issue.
> When a device is removed then the corresponding struct device is torn
> down, which implies that the device domain is freed as well. So how can
> this end up with the old state on the next probe?
>
> Thanks,
>
>          tglx
Hi, My description was a bit ambiguous. The msi_device_data_release()
path to remove the irq_domain is only triggered when the device is
removed at the PCI layer. If only the upper-layer driver unbinds,
this path will not be reached.

Best Regards,
Guixin Liu

Re: [PATCH 2/2] PCI/MSI: Update MSI-X irq domain hwsize

[Thomas Gleixner]

On Wed, Mar 25 2026 at 09:34, Guixin Liu wrote:
> 在 2026/3/24 21:59, Thomas Gleixner 写道:
>> On Tue, Mar 24 2026 at 09:47, Guixin Liu wrote:
>>> After the upper-layer driver removes the device and before the next
>>> probe, events such as firmware updates may increase the number of
>>> interrupts supported by the device. However, the irq_domain still
>>> retains the old hwsize, which causes subsequent interrupt allocation
>>> failures. Update hwsize during MSI-X device domain setup to fix this
>>> issue.
>> When a device is removed then the corresponding struct device is torn
>> down, which implies that the device domain is freed as well. So how can
>> this end up with the old state on the next probe?
>
> Hi, My description was a bit ambiguous. The msi_device_data_release()
> path to remove the irq_domain is only triggered when the device is
> removed at the PCI layer. If only the upper-layer driver unbinds,
> this path will not be reached.

What's an upper-layer driver? Please be precise.

I assume you are talking about the device driver itself. Right, the
unbind of the driver won't remove the domain. But there is no real good
reason for keeping it around at that point.

So the straight forward solution is to free the MSI domain when the
driver shuts down and tears the MSI interrupts down.

Thanks,

        tglx

Re: [PATCH 1/2] genirq/msi: Introduce update hwsize helper

[Thomas Gleixner]

On Tue, Mar 24 2026 at 09:47, Guixin Liu wrote:
> +int msi_domain_update_hwsize(struct device *dev, unsigned int domid,
> +			     unsigned int hwsize_new)
> +{
> +	struct msi_domain_info *info;
> +	struct irq_domain *domain;
> +
> +	if (hwsize_new > MSI_XA_DOMAIN_SIZE)
> +		return -EINVAL;
> +	if (!hwsize_new)
> +		hwsize_new = MSI_XA_DOMAIN_SIZE;
> +
> +	domain = msi_get_device_domain(dev, domid);
> +	if (!domain)
> +		return -ENODEV;
> +
> +	info = domain->host_data;
> +	if (hwsize_new != info->hwsize)
> +		info->hwsize = hwsize_new;

This is incorrect because the underlying irqdomain has been created with
the original hardware size. So just fiddling with info->hwsize results
in inconsistent state.

Re: [PATCH 2/2] PCI/MSI: Update MSI-X irq domain hwsize

[Guixin Liu]

在 2026/3/25 15:42, Thomas Gleixner 写道:
> On Wed, Mar 25 2026 at 09:34, Guixin Liu wrote:
>> 在 2026/3/24 21:59, Thomas Gleixner 写道:
>>> On Tue, Mar 24 2026 at 09:47, Guixin Liu wrote:
>>>> After the upper-layer driver removes the device and before the next
>>>> probe, events such as firmware updates may increase the number of
>>>> interrupts supported by the device. However, the irq_domain still
>>>> retains the old hwsize, which causes subsequent interrupt allocation
>>>> failures. Update hwsize during MSI-X device domain setup to fix this
>>>> issue.
>>> When a device is removed then the corresponding struct device is torn
>>> down, which implies that the device domain is freed as well. So how can
>>> this end up with the old state on the next probe?
>> Hi, My description was a bit ambiguous. The msi_device_data_release()
>> path to remove the irq_domain is only triggered when the device is
>> removed at the PCI layer. If only the upper-layer driver unbinds,
>> this path will not be reached.
> What's an upper-layer driver? Please be precise.
>
> I assume you are talking about the device driver itself.
Sorry, My description is not very accurate, yes, it's device driver.
> Right, the
> unbind of the driver won't remove the domain. But there is no real good
> reason for keeping it around at that point.
>
> So the straight forward solution is to free the MSI domain when the
> driver shuts down and tears the MSI interrupts down.
Yes, I had also considered this aspect before, I will change the scheme 
to this,
and send another patch, thanks.

Best Regards,
Guixin Liu
>
> Thanks,
>
>          tglx
>

Re: [PATCH 1/2] genirq/msi: Introduce update hwsize helper

[Guixin Liu]

在 2026/3/25 16:15, Thomas Gleixner 写道:
> On Tue, Mar 24 2026 at 09:47, Guixin Liu wrote:
>> +int msi_domain_update_hwsize(struct device *dev, unsigned int domid,
>> +			     unsigned int hwsize_new)
>> +{
>> +	struct msi_domain_info *info;
>> +	struct irq_domain *domain;
>> +
>> +	if (hwsize_new > MSI_XA_DOMAIN_SIZE)
>> +		return -EINVAL;
>> +	if (!hwsize_new)
>> +		hwsize_new = MSI_XA_DOMAIN_SIZE;
>> +
>> +	domain = msi_get_device_domain(dev, domid);
>> +	if (!domain)
>> +		return -ENODEV;
>> +
>> +	info = domain->host_data;
>> +	if (hwsize_new != info->hwsize)
>> +		info->hwsize = hwsize_new;
> This is incorrect because the underlying irqdomain has been created with
> the original hardware size. So just fiddling with info->hwsize results
> in inconsistent state.
I found that hwsize only affects the range checking when allocating and
releasing intterrupts after the irq_domain is established,
and the kernel does not allocate related resources based on it.
Cloud you plz tell me specifically where the inconsistency is? thanks.

Best Regards,
Guixin Liu

Re: [PATCH 2/2] PCI/MSI: Update MSI-X irq domain hwsize

[Thomas Gleixner]

On Wed, Mar 25 2026 at 16:40, Guixin Liu wrote:
> 在 2026/3/25 15:42, Thomas Gleixner 写道:
>> So the straight forward solution is to free the MSI domain when the
>> driver shuts down and tears the MSI interrupts down.
> Yes, I had also considered this aspect before, I will change the scheme 
> to this, and send another patch, thanks.

Actually none of that is required. When you update the firmware then
just let the PCI core rescan the device. That makes way more sense as
the new firmware might change other config entries as well not only the
MSI ones.

Thanks,

        tglx

Re: [PATCH 2/2] PCI/MSI: Update MSI-X irq domain hwsize

[Guixin Liu]

在 2026/3/25 21:58, Thomas Gleixner 写道:
> On Wed, Mar 25 2026 at 16:40, Guixin Liu wrote:
>> 在 2026/3/25 15:42, Thomas Gleixner 写道:
>>> So the straight forward solution is to free the MSI domain when the
>>> driver shuts down and tears the MSI interrupts down.
>> Yes, I had also considered this aspect before, I will change the scheme
>> to this, and send another patch, thanks.
> Actually none of that is required. When you update the firmware then
> just let the PCI core rescan the device. That makes way more sense as
> the new firmware might change other config entries as well not only the
> MSI ones.
>
> Thanks,
>
>          tglx
You are right, besides MSI, other attributes may also change.
Removing and then recanning the PCI device is the best approach.

Best Regards,
Guixin Liu