Re: [PATCH V1] PCI/MSI: Don't touch the MSI-X table while the Link is contained

[Vidya Sagar <vidyas@nvidia.com>]

On 18-08-2026 01:42, sashiko-bot@kernel.org wrote:
> External email: Use caution opening links or attachments
> 
> 
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> 
> Pre-existing issues:
> - [High] Unprotected `readl()` operations in `pci_msix_write_tph_tag()`, `__pci_read_msi_msg()`, and `msix_prepare_msi_desc()` circumvent the MSI-X MMIO containment fix.
Thanks for looking at this. The three sites do exist as described, but
they are not equivalent, and only one of them needs a change. Taking
them in turn.
> --
> 
> --- Patch [1]: [PATCH V1] PCI/MSI: Don't touch the MSI-X table while the Link is contained ---
> commit aba17efd26855fb96d48aa82a9f367120d469a3f
> Author: Vidya Sagar <vidyas@nvidia.com>
> 
> PCI/MSI: Don't touch the MSI-X table while the Link is contained
> 
> The MSI-X table lives in device MMIO space behind a BAR, so it is only
> reachable while the Link is up. While a Downstream Port has the Link
> contained by DPC it completes accesses to the table with Unsupported
> Request, and the read returns all ones.
> 
>> diff --git a/drivers/pci/msi/msi.c b/drivers/pci/msi/msi.c
>> index 209373c92e9e1..5612cf2ff2800 100644
>> --- a/drivers/pci/msi/msi.c
>> +++ b/drivers/pci/msi/msi.c
>> @@ -249,7 +249,7 @@ void __pci_write_msi_msg(struct msi_desc *entry, struct msi_msg *msg)
>>  {
>>       struct pci_dev *dev = msi_desc_to_pci_dev(entry);
>>
>> -     if (dev->current_state != PCI_D0 || pci_dev_is_disconnected(dev)) {
>> +     if (dev->current_state != PCI_D0 || pci_msix_mmio_unsafe(dev)) {
>>               /* Don't touch the hardware now */
>>       } else if (entry->pci.msi_attrib.is_msix) {
>>               pci_write_msg_msix(entry, msg);
>> [ ... ]
>> diff --git a/drivers/pci/msi/msi.h b/drivers/pci/msi/msi.h
>> index 0b420b319f50f..e372e9c875087 100644
>> --- a/drivers/pci/msi/msi.h
>> +++ b/drivers/pci/msi/msi.h
>> [ ... ]
>> +/*
>> + * The MSI-X Table lives in device MMIO space, so it is only reachable while
>> + * the Link is usable. While a Downstream Port has the Link contained by DPC
>> + * it completes these accesses with Unsupported Request, which the Root Port
>> + * in turn reports as an RP PIO error and answers with a DPC of its own,
>> + * taking down every other device below it.
>> + */
>> +static inline bool pci_msix_mmio_unsafe(struct pci_dev *pdev)
>> +{
>> +     return pdev->error_state != pci_channel_io_normal ||
>> +            pci_dev_is_disconnected(pdev);
>> +}
> 
> [Severity: High]
> This is a pre-existing issue, but could this regression still occur through
> other unprotected MSI-X MMIO accesses?
> 
> While this patch successfully shields the vector control write inside
> pci_msix_write_vector_ctrl() using this new check, there appear to be
> unprotected readl() operations that still bypass it.
> 
> For example, in drivers/pci/msi/msi.c:pci_msix_write_tph_tag():
> 
>     FIELD_MODIFY(PCI_MSIX_ENTRY_CTRL_ST, &msi_desc->pci.msix_ctrl, tag);
>     pci_msix_write_vector_ctrl(msi_desc, msi_desc->pci.msix_ctrl);
>     /* Flush the write */
>     readl(pci_msix_desc_addr(msi_desc));
> 
> The flush readl() is unconditional. It bypasses the safety check and
> would complete with an Unsupported Request if the link is contained.
Agreed, and this one is worth fixing. It is also worse than you state:
it is not purely pre-existing, because this patch is what makes the
write conditional while leaving its paired flush read unconditional. So
after this patch the function can issue a non-posted read to flush a
write that never happened.

pcie_tph_set_st_entry() already propagates an error by disabling TPH, so
the caller can cope. I will bail out early in v2:

	if (!pdev->msix_enabled)
		return -ENXIO;

+	/*
+	 * The tag update below is a write to the MSI-X Table followed by a
+	 * flush read, neither of which can be completed while the Link is
+	 * contained. Let the caller disable TPH instead.
+	 */
+	if (pci_msi_dev_inaccessible(pdev))
+		return -EIO;
+
	virq = msi_get_virq(&pdev->dev, index);

> 
> Similarly, in drivers/pci/msi/msi.c:__pci_read_msi_msg():
> 
>     msg->address_lo = readl(base + PCI_MSIX_ENTRY_LOWER_ADDR);
>     msg->address_hi = readl(base + PCI_MSIX_ENTRY_UPPER_ADDR);
>     msg->data = readl(base + PCI_MSIX_ENTRY_DATA);
The only caller of __pci_read_msi_msg() in the tree is
pseries_msi_compose_msg() on powerpc/pseries, which already selects
between reading the device and using the cached message based on device
state. pseries uses firmware-mediated EEH rather than native DPC, so the
escalation this patch is about does not arise there.

More importantly, this is a read whose value the caller consumes. The
write sites are safe to skip only because msi_desc::msix_ctrl caches the
intent and __pci_restore_msix_state() replays it once the Link is back
up. A read has no such fallback, so "guarding" it would mean returning
uninitialised or all-ones data, which is worse than the UR. Making it
safe would require an error return and propagation through
irq_compose_msi_msg(), which is a different and much larger change.

> 
> And in drivers/pci/msi/msi.c:msix_prepare_msi_desc():
> 
>     if (dev->dev_flags & PCI_DEV_FLAGS_MSIX_TOUCH_ENTRY_DATA_FIRST)
>         writel(0, addr + PCI_MSIX_ENTRY_DATA);
>     desc->pci.msix_ctrl = readl(addr + PCI_MSIX_ENTRY_VECTOR_CTRL);
> 
> If an IRQ affinity rebalancing event triggers a TPH tag update while
> the link is contained, or if the link becomes contained concurrently
> during MSI-X initialization, won't these unprotected accesses still
> escalate to a Root Port PIO error and trigger a DPC?
Guarding this one would introduce a bug. That readl() seeds the
msix_ctrl cache from hardware. If it were skipped the cache would read
back as zero, i.e. unmasked, while the hardware vector is masked, which
is the reset default. Every later mask/unmask decision would then work
off an inverted cache, and __pci_restore_msix_state() would replay the
wrong state.

The specific sequence you describe also has no deterministic path.
report_slot_reset() moves the device back to pci_channel_io_normal
before it invokes the driver callback:

	if (!pci_dev_set_io_state(dev, pci_channel_io_normal) ||
	    !pdrv || !pdrv->err_handler || !pdrv->err_handler->slot_reset)
		goto out;

	vote = err_handler->slot_reset(dev);

So a driver re-enabling MSI-X from .slot_reset() runs with error_state
already normal and nothing is skipped. That same ordering is why this
patch does not break pci_restore_state() ->
__pci_restore_msix_state() during recovery.

In principle yes, and that is inherent rather than something this patch
regresses. error_state is a notification set by the DPC/AER handler
after containment has already happened; it is not a lock. Containment is
asynchronous, so any MMIO to any device can race with it, and no
placement of these checks changes that. Closing the race properly would
require serialising every MSI-X mask against containment, i.e. a lock in
the interrupt masking path.

What the patch does close is the deterministic window, which is the
interval between report_frozen_detected() and report_slot_reset(). In
that window the kernel already knows the Link is contained, and driver
.error_detected() and prepare-for-reset callbacks nonetheless call
pci_free_irq_vectors(), which masks every descriptor and flushes each
mask with a non-posted read. That is reproducible on every contained
device that tears down its interrupts before the reset, and it is what I
observed escalating a Downstream Port containment event into one at the
Root Port.

For v2 I will also rename the helper to pci_msi_dev_inaccessible(),
since in __pci_write_msi_msg() it gates the Configuration Space MSI path
as well, and the previous name implied MSI-X MMIO only.

Thanks,
Vidya Sagar
> 
> --
> Sashiko AI review · https://sashiko.dev/#/patchset/20260817195626.3091331-1-vidyas@nvidia.com?part=1