Re: [PATCH v6] PCI/PM: Skip resuming to D0 if disconnected

["Mario Limonciello (kernel.org)" <superm1@kernel.org>]

On 9/8/2025 4:48 PM, Bjorn Helgaas wrote:
> On Mon, Sep 08, 2025 at 04:29:32PM -0500, Mario Limonciello (kernel.org) wrote:
>> On 9/8/2025 4:03 PM, Bjorn Helgaas wrote:
>>> On Mon, Aug 11, 2025 at 11:35:10AM -0500, Mario Limonciello (AMD) wrote:
>>>> When a USB4 dock is unplugged the PCIe bridge it's connected to will
>>>> issue a "Link Down" and "Card not detected event". The PCI core will
>>>> treat this as a surprise hotplug event and unconfigure all downstream
>>>> devices. This involves setting the device error state to
>>>> `pci_channel_io_perm_failure` which pci_dev_is_disconnected() will check.
>>>
>>> There's nothing special about USB4 here, right?  I guess the same
>>> happens with any surprise hotplug remove?
>>
>> Correct.
>>
>>> pciehp_unconfigure_device() does the pci_dev_set_io_state(dev,
>>> pci_channel_io_perm_failure) part on everything that got removed, so
>>> that part is pretty straightforward.
>>>
>>>> It doesn't make sense to runtime resume disconnected devices to D0 and
>>>> report the (expected) error, so bail early.
>>>
>>> Can you include a hint about where the runtime resume happens?  It
>>> seems unintuitive to power up removed devices.
>>
>> Here's the whole trace.  Basically as part of the base driver release the
>> device is runtime resumed.
>>
>> Do you want me to respin and try to incorporate a sentence or two into the
>> commit text?
> 
> Yes, please.  

OK, will do.
> I guess maybe we are basically here inside the
> pm_runtime_get_sync() here?
> 
>    pciehp_unconfigure_device
>      pci_stop_and_remove_bus_device
>        device_release_driver
>          device_remove
>            pci_device_remove
>              pm_runtime_get_sync    <--
>                drv->remove
> 
> I guess it's the pm_runtime_get_sync() that eventually lands us in
> pci_power_up()?
> 
> And I guess we can't really check whether the device is already gone
> in pci_device_remove() because we still want to run the driver
> .remove() method and other things to clean up.
> 
> I suppose everything we call just needs to be aware that its device
> may be already gone?

Yes, exactly.

> 
>> [   37.237841] dump_stack_lvl (lib/dump_stack.c:94 lib/dump_stack.c:120)
>> [   37.237858] pci_power_up.cold+0x86/0xc1
>> [   37.237867] ? __pfx_pci_power_up (drivers/pci/pci.c:1360)
>> [   37.237876] ? prb_commit (kernel/printk/printk_ringbuffer.c:1748)
>> [   37.237883] ? __pfx_prb_read_valid
>> (kernel/printk/printk_ringbuffer.c:2184)
>> [   37.237889] pci_pm_power_up_and_verify_state (drivers/pci/pci.c:1146
>> drivers/pci/pci.c:1201 drivers/pci/pci.c:3206)
>> [   37.237897] ? __pfx_pci_pm_power_up_and_verify_state
>> (drivers/pci/pci.c:3204)
>> [   37.237903] ? __pfx_rpm_resume (drivers/base/power/runtime.c:785)
>> [   37.237911] pci_pm_runtime_resume (drivers/pci/pci-driver.c:561
>> drivers/pci/pci-driver.c:1349)
>> [   37.237918] __rpm_callback (drivers/base/power/runtime.c:406)
>> [   37.237923] ? __pfx__raw_spin_lock (kernel/locking/spinlock.c:153)
>> [   37.237932] rpm_callback (drivers/base/power/runtime.c:444)
>> [   37.237936] ? __pfx_pci_pm_runtime_resume (drivers/pci/pci-driver.c:1338)
>> [   37.237941] rpm_resume (drivers/base/power/runtime.c:943)
>> [   37.237946] ? __pfx_rpm_resume (drivers/base/power/runtime.c:785)
>> [   37.237951] ? _raw_spin_lock_irqsave (./include/linux/instrumented.h:96
>> ./include/linux/atomic/atomic-instrumented.h:1301
>> ./include/asm-generic/qspinlock.h:111 ./include/linux/spinlock.h:187
>> ./include/linux/spinlock_api_smp.h:111 kernel/locking/spinlock.c:162)
>> [   37.237954] ? __pfx__raw_spin_lock_irqsave
>> (kernel/locking/spinlock.c:161)
>> [   37.237958] ? mutex_lock (./include/linux/instrumented.h:96
>> ./include/linux/atomic/atomic-instrumented.h:4457 kernel/locking/mutex.c:157
>> kernel/locking/mutex.c:273)
>> [   37.237965] __pm_runtime_resume (drivers/base/power/runtime.c:1192)
>> [   37.237971] device_release_driver_internal (drivers/base/dd.c:1111
>> (discriminator 1) drivers/base/dd.c:1252 (discriminator 1)
>> drivers/base/dd.c:1297 (discriminator 1))
>> [   37.237978] ? pci_pme_active (drivers/pci/pci.c:2521)
>> [   37.237984] pci_stop_bus_device (drivers/pci/remove.c:44
>> drivers/pci/remove.c:107)
>> [   37.237992] pci_stop_bus_device (drivers/pci/remove.c:102 (discriminator
>> 1))
>> [   37.237997] pci_stop_and_remove_bus_device (drivers/pci/remove.c:142)
>> [   37.238003] pciehp_unconfigure_device
>> (drivers/pci/hotplug/pciehp_pci.c:124)
>> [   37.238011] ? __pfx_pciehp_unconfigure_device
>> (drivers/pci/hotplug/pciehp_pci.c:96)
>> [   37.238017] ? _dev_info (drivers/base/core.c:4983)
>> [   37.238025] pciehp_disable_slot (drivers/pci/hotplug/pciehp_ctrl.c:115
>> drivers/pci/hotplug/pciehp_ctrl.c:355 drivers/pci/hotplug/pciehp_ctrl.c:364)
>> [   37.238030] ? __pfx_pciehp_disable_slot
>> (drivers/pci/hotplug/pciehp_ctrl.c:360)
>> [   37.238035] ? __pfx_mutex_unlock (kernel/locking/mutex.c:531)
>> [   37.238039] ? mutex_lock_interruptible (kernel/locking/mutex.c:992)
>> [   37.238047] pciehp_handle_presence_or_link_change
>> (drivers/pci/hotplug/pciehp_ctrl.c:253)
>> [   37.238054] ? down_read (kernel/locking/rwsem.c:174
>> kernel/locking/rwsem.c:182 kernel/locking/rwsem.c:257
>> kernel/locking/rwsem.c:249 kernel/locking/rwsem.c:1260
>> kernel/locking/rwsem.c:1274 kernel/locking/rwsem.c:1539)
>> [   37.238061] ? __pfx_pciehp_handle_presence_or_link_change
>> (drivers/pci/hotplug/pciehp_ctrl.c:232)
>> [   37.238068] ? __pfx_down_read (kernel/locking/rwsem.c:1535)
>> [   37.238074] ? __pfx_pciehp_ist (drivers/pci/hotplug/pciehp_hpc.c:728)
>> [   37.238079] pciehp_ist (drivers/pci/hotplug/pciehp_hpc.c:788)
>> [   37.238085] ? __pfx_pciehp_ist (drivers/pci/hotplug/pciehp_hpc.c:728)
>> [   37.238091] irq_thread_fn (kernel/irq/manage.c:1131)
>> [   37.238098] irq_thread (./arch/x86/include/asm/bitops.h:206
>> ./arch/x86/include/asm/bitops.h:238
>> ./include/asm-generic/bitops/instrumented-non-atomic.h:142
>> kernel/irq/manage.c:1244)
>> [   37.238104] ? __pfx_irq_thread_fn (kernel/irq/manage.c:1130)
>> [   37.238110] ? __pfx_irq_thread (kernel/irq/manage.c:1233)
>> [   37.238115] ? __pfx_irq_thread_dtor (kernel/irq/manage.c:1167)
>> [   37.238123] ? __kthread_parkme (./arch/x86/include/asm/bitops.h:206
>> ./arch/x86/include/asm/bitops.h:238
>> ./include/asm-generic/bitops/instrumented-non-atomic.h:142
>> kernel/kthread.c:290)
>> [   37.238130] ? __pfx_irq_thread (kernel/irq/manage.c:1233)
>> [   37.238136] ? __pfx_irq_thread (kernel/irq/manage.c:1233)
>> [   37.238141] kthread (kernel/kthread.c:463)
>> [   37.238147] ? __pfx_kthread (kernel/kthread.c:412)
>> [   37.238153] ? finish_task_switch.isra.0
>> (./arch/x86/include/asm/irqflags.h:42 ./arch/x86/include/asm/irqflags.h:119
>> kernel/sched/sched.h:1531 kernel/sched/core.c:5105 kernel/sched/core.c:5223)
>> [   37.238161] ? __pfx_kthread (kernel/kthread.c:412)
>> [   37.238166] ret_from_fork (arch/x86/kernel/process.c:154)
>> [   37.238175] ? __pfx_kthread (kernel/kthread.c:412)
>> [   37.238180] ? __pfx_kthread (kernel/kthread.c:412)
>> [   37.238184] ret_from_fork_asm (arch/x86/entry/entry_64.S:258)
>> [   37.238192]  </TASK>
>>
>>>
>>>> Suggested-by: Lukas Wunner <lukas@wunner.de>
>>>> Reviewed-by: Lukas Wunner <lukas@wunner.de>
>>>> Reviewed-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
>>>> Acked-by: Rafael J. Wysocki <rafael@kernel.org>
>>>> Signed-off-by: Mario Limonciello (AMD) <superm1@kernel.org>
>>>> ---
>>>> v6:
>>>>    * rebase on v6.17-rc1
>>>> v5:
>>>>    * Pick up tags, rebase on linux-next
>>>>    * https://lore.kernel.org/linux-pci/20250709205948.3888045-1-superm1@kernel.org/T/#mbd784f786c50a3d1b5ab1833520995c01eae2fd2
>>>> ---
>>>>    drivers/pci/pci.c | 5 +++++
>>>>    1 file changed, 5 insertions(+)
>>>>
>>>> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
>>>> index b0f4d98036cdd..036511f5b2625 100644
>>>> --- a/drivers/pci/pci.c
>>>> +++ b/drivers/pci/pci.c
>>>> @@ -1374,6 +1374,11 @@ int pci_power_up(struct pci_dev *dev)
>>>>    		return -EIO;
>>>>    	}
>>>> +	if (pci_dev_is_disconnected(dev)) {
>>>> +		dev->current_state = PCI_D3cold;
>>>> +		return -EIO;
>>>> +	}
>>>> +
>>>>    	pci_read_config_word(dev, dev->pm_cap + PCI_PM_CTRL, &pmcsr);
>>>>    	if (PCI_POSSIBLE_ERROR(pmcsr)) {
>>>>    		pci_err(dev, "Unable to change power state from %s to D0, device inaccessible\n",
>>>>
>>>> base-commit: 8f5ae30d69d7543eee0d70083daf4de8fe15d585
>>>> -- 
>>>> 2.43.0
>>>>
>>