Re: [PATCH RFC v4 0/5] PCI: pciehp: Report surprise removal during safe removal

["Michael S. Tsirkin" <mst@redhat.com>]

On Mon, Oct 12, 2026 at 08:59:33PM +0200, Abhin Parekadan Jose wrote:
> On Mon, Oct 12, 2026 at 02:32:02AM -0400, Michael S. Tsirkin wrote:
> > On Mon, Oct 12, 2026 at 07:39:34AM +0200, Abhin Parekadan Jose wrote:
> > > Gentle ping on this RFC.
> > >
> > > Lukas, Bjorn, could you let me know whether the general direction is
> > > acceptable before I work on a non-RFC version? Any comments on the
> > > approach would help.
> > >
> > > Thanks,
> > > Abhin
> >
> > I'd say, complete the picture. acpi,shpc,polling...
> 
> Ok.
> 
> >
> > Maybe think about how this interacts with pci_ignore_hotplug
> > (seems to weirdly check presence then ignore)?
> 
> Yeah, the order is wrong in v4. The presence read should come after
> the `ignore_hotplug` check. Will fix.
> (Need to still find out why this was introduced. Looks like only the GPU drivers
> use it.)
> 
> >
> > And test with virtio not with edu.
> 
> Bjorn had asked me to keep this puerly PCI[1], So I had decoupled it from the
> virtio changes. I could post another series with those changes with a dependecy
> to this series.
> 
> [1] https://lore.kernel.org/all/20260826194815.GA1552818@bhelgaas/

yea sure.

> >
> > > On Sun, Sep 27, 2026 at 06:20:11PM +0000, Abhin Parekadan Jose wrote:
> > > > If a safe removal is already in progress when the device is surprise
> > > > removed, pciehp cannot report the disconnect [1]. The removal blocks
> > > > waiting on a device interrupt or status read, and pciehp's IRQ thread is
> > > > single-threaded and is itself executing that removal, so it never runs
> > > > again to report the device gone. The removal hangs indefinitely.
> > > >
> > > > pciehp_isr() does run while the IRQ thread is blocked, but pciehp_ist()
> > > > must ignore link and presence changes caused by Secondary Bus Reset or
> > > > DPC, and telling those apart takes seconds which cannot be spent in
> > > > hardirq.
> > > >
> > > > Patch 4 does not do that work in hardirq. pciehp_isr() only checks
> > > > whether the card is present or the link is active, and defers the rest
> > > > to a work item in process context. The work item does not need to know
> > > > why the link changed, only whether the card is gone: it waits for DPC
> > > > recovery or a Secondary Bus Reset in progress to complete, then checks
> > > > again under reset_lock, and if the card is absent and the link is down
> > > > marks the devices below disconnected, which schedules the driver's
> > > > disconnect work from patch 1. pciehp_ist() is unchanged and
> > > > remains the only consumer of the one-shot flags PCI_DPC_RECOVERED and
> > > > PCI_LINK_CHANGED, which tell it whether a link change can be ignored.
> > > >
> > > > Patches:
> > > >
> > > >   1/5 Michael's "PCI: Report surprise removal event" from his RFC v5,
> > > >       which adds disconnect_work_enable and pdev->disconnect_work.
> > > >       Changed to serialize disconnect_work_enable with a per-device
> > > >       spinlock.
> > > >
> > > >   2/5 Add pci_hp_wait_link_change(), which awaits a Secondary Bus Reset
> > > >       in progress without consuming PCI_LINK_CHANGED.
> > > >
> > > >   3/5 Add pci_dpc_wait_recovery(), which awaits DPC recovery without
> > > >       consuming PCI_DPC_RECOVERED.
> > > >
> > > >   4/5 The pciehp change. Adds disconnect_work to struct controller,
> > > >       scheduled from pciehp_isr() on PDC or DLLSC when neither Presence
> > > >       Detect State nor Data Link Layer Link Active indicates a card.
> >
> >
> > how about including the virtio patch? seems more relevant than
> > fooling around with edu.
> >
> > > >   5/5 A POC driver for the QEMU edu device that blocks in remove()
> > > >       waiting for an interrupt, standing in for del_gendisk() stuck in
> > > >       blk_mq_freeze_queue_wait(). Not for merge -- included so the
> > > >       hang can be reproduced.
> > > >
> > > > Changes since RFC v3 [5]:
> > > >
> > > >   - 5/5: Reinitialize the completion in remove() before requesting the
> > > >     delayed interrupt, so that an earlier completion cannot let remove()
> > > >     return while the interrupt is still pending. (Sashiko)
> > > >   - 1/5 to 4/5: No changes.
> > > >
> > > > Changes since RFC v2 [4]:
> > > >
> > > >   - 1/5: Protect disconnect_work_enable with a per-device spinlock, held
> > > >     while testing it and scheduling the work, instead of lockless
> > > >     accesses and barriers.  pci_dev_set_disconnected() could otherwise
> > > >     test the flag, get preempted, and queue the work while a newly bound
> > > >     driver re-initializes it.  With the lock, cancel_work_sync() is
> > > >     sufficient again, so go back to it from disable_work_sync().
> > > >     (Sashiko)
> > > >   - 2/5: Test PCI_LINK_CHANGING with test_bit_acquire(), so that the
> > > >     caller's subsequent accesses are ordered after the end of the code
> > > >     section even if wait_event() returns without sleeping. (Sashiko)
> > > >   - 3/5, 4/5, 5/5: No changes.
> > > >
> > > > Changes since RFC v1 [2]:
> > > >
> > > >   - 1/5: Use disable_work_sync() instead of cancel_work_sync() in
> > > >     pci_clear_disconnect_work(), so that a racing schedule_work() cannot
> > > >     queue the work after remove() has returned. (Sashiko)
> > > >   - 2/5, 3/5: New.
> > > >   - 4/5: Drop schedule_notification_work(); pci_dev_set_disconnected()
> > > >     schedules the driver's disconnect work for all callers again.
> > > >     (Michael)
> > > >   - 4/5: Don't consume PCI_DPC_RECOVERED and PCI_LINK_CHANGED in the
> > > >     work item.  In v1 it ran the same spurious link change test as
> > > >     pciehp_ist(), and whichever ran first took the flags, so pciehp_ist()
> > > >     could tear down a device that was only reset. (Sashiko)
> > > >   - 4/5: Check presence with pciehp_card_present_or_link_active() in
> > > >     both pciehp_isr() and the work item, as pciehp_ist() does, so that a
> > > >     port with Presence Detect State hardwired to zero is not mistaken
> > > >     for an empty slot.
> > > >   - 4/5: Return early from the work item if pciehp_ist() has already
> > > >     taken the pending events, and treat a read error of the presence
> > > >     check as "not present". (Sashiko)
> > > >   - 4/5: In pciehp_isr(), check presence before dropping the runtime PM
> > > >     reference on the port's parent.  In the work item, take a runtime PM
> > > >     reference and check presence under reset_lock, since a slot reset
> > > >     may make Presence Detect State and Link Active flap.
> > > >   - 5/5: Build only with CONFIG_EDU_SRPOC, which depends on PCI; don't
> > > >     claim the interrupt when the device reads all ones; clear bus
> > > >     mastering on teardown. (Sashiko)
> > > >
> > > > Testing
> > > >
> > > > Reproducing this needs QEMU changes, since neither device_del nor the
> > > > attention button produces a true surprise removal.  A branch with both
> > > > is here [3]:
> > > >
> > > >   - a delayed-IRQ register on the edu device (BAR0 0x30, write N ms)
> > > >   - a pcie_surprise_del monitor command that drops the device and
> > > >     generates PDC=1, DLLSC=1, PDS=0
> > > >
> > > > Both tests use:
> > > >
> > > >   ./qemu-system-aarch64 -machine virt,gic-version=3 -cpu cortex-a57 \
> > > >     -m 512 -smp 2 -kernel Image -initrd initramfs.cpio.gz \
> > > >     -device pcie-root-port,id=rp1,chassis=1,slot=1 \
> > > >     -device edu,bus=rp1,id=edu0 -append "console=ttyAMA0 rdinit=/init" \
> > > >     -nographic -monitor unix:/tmp/qemu-mon.sock,server,nowait
> > > >
> > > >   and need a guest kernel with CONFIG_EDU_SRPOC=y.
> > > >
> > > > Test 1 (Hang in remove() on a user thread, then surprise removal):
> > > >
> > > >   guest# echo 1 > /sys/bus/pci/devices/0000:01:00.0/remove &
> > > >   host$  echo "pcie_surprise_del edu0" | socat - unix-connect:/tmp/qemu-mon.sock
> > > >
> > > >   This is the case patch 1 solves on its own: pciehp's IRQ thread is
> > > >   free and handles the removal.
> > > >
> > > > Test 2 (Hang in remove() on pciehp's IRQ thread, then surprise removal):
> > > >
> > > >   guest# echo 0 > /sys/bus/pci/slots/1/power &
> > > >   host$  echo "pcie_surprise_del edu0" | socat - unix-connect:/tmp/qemu-mon.sock
> > > >
> > > >   Without patch 4 the safe removal does not return until the delayed
> > > >   interrupt fires 600 s later.  With it,
> > > >   pciehp_isr() schedules ctrl->disconnect_work, which marks the device
> > > >   disconnected; the POC driver's disconnect work completes the wait and
> > > >   remove() proceeds.
> > > >
> > > > Open questions
> > > >
> > > >   - Is this a viable approach?
> > > >
> > > > [1] https://lore.kernel.org/all/aHlZE18kPuHuDtTT@wunner.de/
> > > > [2] https://lore.kernel.org/all/20260905183905.997833-1-abhinjoses@gmail.com/
> > > > [3] https://gitlab.com/abhinkop/qemu/-/commits/suprise-removal
> > > > [4] https://lore.kernel.org/all/20260927165459.829900-1-abhinjoses@gmail.com/
> > > > [5] https://lore.kernel.org/all/20260927175203.928270-1-abhinjoses@gmail.com/
> > > >
> > > > Assisted-by: LLM
> > > >
> > > > Abhin Parekadan Jose (4):
> > > >   PCI: pciehp: Add pci_hp_wait_link_change()
> > > >   PCI/DPC: Add pci_dpc_wait_recovery()
> > > >   PCI: pciehp: Report surprise removal from pciehp_isr()
> > > >   misc: Add edu_srpoc surprise removal POC driver
> > > >
> > > > Michael S. Tsirkin (1):
> > > >   PCI: Report surprise removal event
> > > >
> > > >  drivers/misc/Kconfig                   |  11 ++
> > > >  drivers/misc/Makefile                  |   1 +
> > > >  drivers/misc/edu_srpoc.c               | 172 +++++++++++++++++++++++++
> > > >  drivers/pci/hotplug/pci_hotplug_core.c |  21 ++-
> > > >  drivers/pci/hotplug/pciehp.h           |   1 +
> > > >  drivers/pci/hotplug/pciehp_hpc.c       |  67 ++++++++++
> > > >  drivers/pci/pci.h                      |  10 ++
> > > >  drivers/pci/pcie/dpc.c                 |  54 ++++++--
> > > >  drivers/pci/probe.c                    |   1 +
> > > >  include/linux/pci.h                    |  57 ++++++++
> > > >  10 files changed, 381 insertions(+), 14 deletions(-)
> > > >  create mode 100644 drivers/misc/edu_srpoc.c
> > > >
> > > >
> > > > base-commit: fd179f8a05be3ccae366b9b96e176b51fbe54aab
> > > > --
> > > > 2.51.1
> > > >
> >