Re: [PATCH] pciehp: sync interrupts for bus resets

[Keith Busch <kbusch@kernel.org>]

On Wed, Sep 03, 2025 at 10:21:34AM +0200, Lukas Wunner wrote:
> On Tue, Sep 02, 2025 at 11:59:11AM -0600, Keith Busch wrote:
> > Hm, I think you're right. We are definitely seeing pciehp requeue itself
> > with the link/presence events that we want to be ignored, so we're
> > getting re-enumeration when we didn't expect it. I thought the
> > back-to-back resets that we're causing vfio to initiate was the problem,
> > but maybe not. I think the switch and/or end device we're using have
> > some unusual link timings that defeats the pciehp ignore logic.
> 
> pci_bridge_secondary_bus_reset() calls pci_bridge_wait_for_secondary_bus()
> to await Link Up.  So unless the link flaps afterwards, this should be
> fine.
> 
> Another possibility is that the pciehp_device_replaced() check triggers,
> e.g. because the Endpoint's Device Serial Number or other data in Config
> Space changed after the second reset.

That can happen because we're using switches that insert a fake
"placeholder" device when a link is down.
 
> Maybe you can instrument the code with a few printk()'s to see what's
> going on.

But it looks like we're more frequently seeing the link not active.
Here's the existing messages printed:


[ 7904.749658] vfio-pci 0000:05:00.0: disabling bus mastering
[ 7904.756595] vfio-pci 0000:05:00.0: reset via bus
[ 7904.759975] pcieport 0000:02:02.0: waiting 100 ms for downstream link, after activation
[ 7905.908987] vfio-pci 0000:05:00.0: ready 0ms after bus reset
[ 7905.909003] pcieport 0000:02:02.0: pciehp: Slot(314): Link Down/Up ignored
[ 7906.847973] vfio-pci 0000:05:00.0: resetting
[ 7906.856312] vfio-pci 0000:05:00.0: reset via bus
[ 7906.862967] pcieport 0000:02:02.0: waiting 100 ms for downstream link, after activation
[ 7909.915925] pcieport 0000:02:02.0: Data Link Layer Link Active not set in 100 msec
[ 7909.915953] pcieport 0000:02:02.0: pciehp: Slot(314): Link Down/Up ignored
[ 7909.915977] pcieport 0000:02:02.0: pciehp: Slot(314): Link Down
[ 7909.915978] pcieport 0000:02:02.0: pciehp: Slot(314): Card not present
[ 7909.918934] pcieport 0000:02:02.0: waiting 100 ms for downstream link, after activation
[ 7911.923899] vfio-pci 0000:05:00.0: disconnected; not waiting
[ 7911.923905] vfio-pci 0000:05:00.0: bus failed with -25