* [PATCH v2] pci: quirks: Disable native PCIe hotplug on MSI Claw A8 root bridge
@ 2026-08-06 21:48 Derek J. Clark
2026-08-07 8:53 ` Lukas Wunner
0 siblings, 1 reply; 10+ messages in thread
From: Derek J. Clark @ 2026-08-06 21:48 UTC (permalink / raw)
To: Bjorn Helgaas
Cc: Pierre-Loup A . Griffais, Derek J . Clark, linux-pci,
linux-kernel
The MSI Claw A8 (MS-1T8K) hard-locks on resume from s2idle whenever an
SD/MMC card is present in the onboard Realtek RTS525A card reader
(10ec:525a), which sits behind root port 0000:00:02.2. The reader's
PCIe endpoint is fixed/soldered and not user-hotpluggable.
Firmware advertises this root port as PCIe hotplug-capable, and _OSC
grants the OS native hotplug control. When native hotplug is OS-owned,
resuming from s2idle races the port's hotplug/PME resume handling
against the RTS525A's own resume sequence when a card is present,
hard-locking the system.
Two kernel command line parameters were confirmed to prevent the hang:
pcie_aspm=off and pcie_ports=compat. Both stop the OS from running
its own hotplug/PME resume path against this port. The former does
so by dropping _OSC negotiation during boot, skipping the path that
enables PCIeHotPlug. The latter prevents the PME service resume
handling by aborting registration of the ports native services. Testing
different kernel boot commands, including pciehp=off and pcie_pme=off
did not resolve the issue. AER, DPC, SHPC, and LTR flags were not tested
because the hardware reports that the platform doesn't support those
features.
This quirk implements as narrow a fix as possible, clearing
is_hotplug_bridge and is_pciehp on the root bridge, scoped by DMI board
name and this root port's bus/device/function, so the port driver never
registers a native hotplug service against it.
SD card insertion is handled entirely by rtsx_pci's own card-detect logic,
so is unaffected by this quirk.
Signed-off-by: Derek J. Clark <derekjohn.clark@gmail.com>
---
v2:
- Switch to pci_info vice dev_info.
- use is_hotplug_bridge and is_pciehp on the root port instead of
native_pcie_hotplug on the root bridge.
v1: https://lore.kernel.org/linux-pci/20260806190439.12022-1-derekjohn.clark@gmail.com/
---
drivers/pci/quirks.c | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 6501c949c5b7..62480fe10bcd 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -6420,3 +6420,26 @@ static void pci_mask_replay_timer_timeout(struct pci_dev *pdev)
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_GLI, 0x9750, pci_mask_replay_timer_timeout);
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_GLI, 0x9755, pci_mask_replay_timer_timeout);
#endif
+
+/*
+ * The MSI Claw A8 firmware advertises native PCIe hotplug support for
+ * this root port, but native hotplug handling causes resume failures.
+ * Prevent the PCIe port driver from claiming native hotplug ownership
+ * of this port.
+ */
+static void quirk_claw_a8_no_native_hotplug(struct pci_dev *pdev)
+{
+ if (!dmi_match(DMI_BOARD_NAME, "MS-1T8K"))
+ return;
+
+ if (pdev->bus->number != 0 ||
+ PCI_SLOT(pdev->devfn) != 2 ||
+ PCI_FUNC(pdev->devfn) != 2)
+ return;
+
+ pci_info(pdev, "disabling native PCIe hotplug\n");
+ pdev->is_hotplug_bridge = 0;
+ pdev->is_pciehp = 0;
+}
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_AMD, 0x150b,
+ quirk_claw_a8_no_native_hotplug);
--
2.55.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* Re: [PATCH v2] pci: quirks: Disable native PCIe hotplug on MSI Claw A8 root bridge 2026-08-06 21:48 [PATCH v2] pci: quirks: Disable native PCIe hotplug on MSI Claw A8 root bridge Derek J. Clark @ 2026-08-07 8:53 ` Lukas Wunner 2026-08-08 0:31 ` Derek John Clark 0 siblings, 1 reply; 10+ messages in thread From: Lukas Wunner @ 2026-08-07 8:53 UTC (permalink / raw) To: Derek J. Clark Cc: Bjorn Helgaas, Pierre-Loup A . Griffais, linux-pci, linux-kernel On Thu, Aug 06, 2026 at 09:48:08PM +0000, Derek J. Clark wrote: > The MSI Claw A8 (MS-1T8K) hard-locks on resume from s2idle whenever an > SD/MMC card is present in the onboard Realtek RTS525A card reader > (10ec:525a), which sits behind root port 0000:00:02.2. The RTS525A is known to falsely claim that it supports ASPM L0s, even though it only supports L1. There is a patch pending to work around this hardware erratum, I'm wondering whether it makes the issue go away on the MSI Claw A8? https://lore.kernel.org/r/20260707021527.639611-1-max.lee@canonical.com/ > The reader's PCIe endpoint is fixed/soldered and not user-hotpluggable. > Firmware advertises this root port as PCIe hotplug-capable That looks like an oversight on the part of MSI. Is there a BIOS update available which clears the Hot-Plug Capable bit in the Slot Capabilities Register? (I'm *assuming* that bit is set -- I haven't seen any dmesg or lspci -vvv output in your e-mails, perhaps you could open a bug at bugzilla.kernel.org and upload it there.) > When native hotplug is OS-owned, > resuming from s2idle races the port's hotplug/PME resume handling > against the RTS525A's own resume sequence when a card is present, > hard-locking the system. This sounds like a driver bug in rtsx_pci which needs to be root-caused and fixed, rather than worked around. pciehp (the OS-native PCIe hotplug driver) normally doesn't de-enumerate and re-enumerate devices on resume from system sleep. It uses a heuristic to determine whether the device in the slot was replaced during system sleep and synthesizes a hotplug event if it believes there's now a different device in the slot. Otherwise it just leaves the device in the slot alone. The replacement detection is in pciehp_device_replaced(), drivers/pci/hotplug/pciehp_hpc.c line 567. You could try hacking that function to return false and see if it helps. The heuristic looks at the Vendor ID and Device ID in config space. I do recall that a different type of card readers, RTS5261, trigger a hotplug event when a regular UHS card is replaced with an SD Express card. The card reader is hot-unplugged and the SD Express card appears as an NVMe drive with a different Device ID in config space: https://lore.kernel.org/r/20231016040132.23824-1-kai.heng.feng@canonical.com/ That said, I'm not even sure RTS525A supports SD Express. What kind of card do you have inserted in the card reader, is it UHS or SD Express? Does the issue go away if you eject the card before going to system sleep? > +++ b/drivers/pci/quirks.c [...] > +/* > + * The MSI Claw A8 firmware advertises native PCIe hotplug support for > + * this root port, but native hotplug handling causes resume failures. > + * Prevent the PCIe port driver from claiming native hotplug ownership > + * of this port. > + */ > +static void quirk_claw_a8_no_native_hotplug(struct pci_dev *pdev) > +{ > + if (!dmi_match(DMI_BOARD_NAME, "MS-1T8K")) > + return; > + > + if (pdev->bus->number != 0 || > + PCI_SLOT(pdev->devfn) != 2 || > + PCI_FUNC(pdev->devfn) != 2) > + return; > + > + pci_info(pdev, "disabling native PCIe hotplug\n"); > + pdev->is_hotplug_bridge = 0; > + pdev->is_pciehp = 0; > +} > +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_AMD, 0x150b, > + quirk_claw_a8_no_native_hotplug); Unfortunately this is just a workaround that happens to make the issue disappear, but we'd really want to identify the root cause first. If you boot with no_console_suspend, are you able to see any messages on suspend or resume that could help us understand what is going on? Thanks, Lukas ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] pci: quirks: Disable native PCIe hotplug on MSI Claw A8 root bridge 2026-08-07 8:53 ` Lukas Wunner @ 2026-08-08 0:31 ` Derek John Clark 2026-08-08 8:16 ` Lukas Wunner 0 siblings, 1 reply; 10+ messages in thread From: Derek John Clark @ 2026-08-08 0:31 UTC (permalink / raw) To: lukas; +Cc: Bjorn Helgaas, Pierre-Loup A . Griffais, linux-pci, linux-kernel On Fri, Aug 7, 2026 at 1:53 AM Lukas Wunner <lukas@wunner.de> wrote: > > On Thu, Aug 06, 2026 at 09:48:08PM +0000, Derek J. Clark wrote: > > The MSI Claw A8 (MS-1T8K) hard-locks on resume from s2idle whenever an > > SD/MMC card is present in the onboard Realtek RTS525A card reader > > (10ec:525a), which sits behind root port 0000:00:02.2. > > The RTS525A is known to falsely claim that it supports ASPM L0s, > even though it only supports L1. There is a patch pending to > work around this hardware erratum, I'm wondering whether it > makes the issue go away on the MSI Claw A8? > > https://lore.kernel.org/r/20260707021527.639611-1-max.lee@canonical.com/ > Hi Lukas, Sending again as i inadvertently hit reply instead of reply all. The freeze still occurs with that patch applied, yes. > > The reader's PCIe endpoint is fixed/soldered and not user-hotpluggable. > > Firmware advertises this root port as PCIe hotplug-capable > > That looks like an oversight on the part of MSI. Is there a BIOS update > available which clears the Hot-Plug Capable bit in the Slot Capabilities > Register? (I'm *assuming* that bit is set -- I haven't seen any dmesg > or lspci -vvv output in your e-mails, perhaps you could open a bug > at bugzilla.kernel.org and upload it there.) > I'm on the latest firmware they have as of abuut a month ago. They don't participate in LVFS AFAIK but I made sure to update before switching to Linux. I posted them here for the entire device tree for the SD card reader: https://github.com/ValveSoftware/SteamOS/issues/2473#issuecomment-5222972815 > > When native hotplug is OS-owned, > > resuming from s2idle races the port's hotplug/PME resume handling > > against the RTS525A's own resume sequence when a card is present, > > hard-locking the system. > > This sounds like a driver bug in rtsx_pci which needs to be root-caused > and fixed, rather than worked around. > > pciehp (the OS-native PCIe hotplug driver) normally doesn't de-enumerate > and re-enumerate devices on resume from system sleep. It uses a heuristic > to determine whether the device in the slot was replaced during system > sleep and synthesizes a hotplug event if it believes there's now a different > device in the slot. Otherwise it just leaves the device in the slot alone. > The replacement detection is in pciehp_device_replaced(), > drivers/pci/hotplug/pciehp_hpc.c line 567. You could try hacking > that function to return false and see if it helps. > That worked! That strikes me as odd since the card is never replaced. > The heuristic looks at the Vendor ID and Device ID in config space. > I do recall that a different type of card readers, RTS5261, trigger > a hotplug event when a regular UHS card is replaced with an SD Express > card. The card reader is hot-unplugged and the SD Express card appears > as an NVMe drive with a different Device ID in config space: > > https://lore.kernel.org/r/20231016040132.23824-1-kai.heng.feng@canonical.com/ > > That said, I'm not even sure RTS525A supports SD Express. > > What kind of card do you have inserted in the card reader, is it UHS > or SD Express? Does the issue go away if you eject the card before > going to system sleep? > I've been using a SanDisk Ultra 32GB microSDHC UHS-I Card. And yes, it only happens when the card is present. > > +++ b/drivers/pci/quirks.c > [...] > > +/* > > + * The MSI Claw A8 firmware advertises native PCIe hotplug support for > > + * this root port, but native hotplug handling causes resume failures. > > + * Prevent the PCIe port driver from claiming native hotplug ownership > > + * of this port. > > + */ > > +static void quirk_claw_a8_no_native_hotplug(struct pci_dev *pdev) > > +{ > > + if (!dmi_match(DMI_BOARD_NAME, "MS-1T8K")) > > + return; > > + > > + if (pdev->bus->number != 0 || > > + PCI_SLOT(pdev->devfn) != 2 || > > + PCI_FUNC(pdev->devfn) != 2) > > + return; > > + > > + pci_info(pdev, "disabling native PCIe hotplug\n"); > > + pdev->is_hotplug_bridge = 0; > > + pdev->is_pciehp = 0; > > +} > > +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_AMD, 0x150b, > > + quirk_claw_a8_no_native_hotplug); > > Unfortunately this is just a workaround that happens to make the > issue disappear, but we'd really want to identify the root cause > first. > > If you boot with no_console_suspend, are you able to see any messages > on suspend or resume that could help us understand what is going on? > Nothing of note. After amdgpu finishes its smu resume sequence the console _ flashes and doesn't progress. I did try adding some debug printing to pciehp_device_replaced() but saw no output for that either. Since that didn't work I tried replacing each true result in the function one at a time and found the offending case if (pdev->hdr_type == PCI_HEADER_TYPE_NORMAL && (pci_read_config_dword(pdev, PCI_SUBSYSTEM_VENDOR_ID, ®) || - reg != (pdev->subsystem_vendor | (pdev->subsystem_device << 16)))) - return true; + reg != (pdev->subsystem_vendor | (pdev->subsystem_device << 16)))) { + pci_info(pdev, "pdev subsystem vendor ID mismatch"); + return false; + + } [ 25.686128] rtsx_pci 0000:c2:00.0: pciehp: pdev subsystem vendor ID mismatch Any insight on why that might be occurring and what my next troubleshooting steps should be are appreciated. ** Update: after my initial email & before I sent it back to the entire ML I did another test and printed the reg vs expected values. It seems the SD card reader ID is reflected when it is enumerated during resume. reg=0x525a10ec expected=0x14af1462 Thanks, Derek > Thanks, > > Lukas ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] pci: quirks: Disable native PCIe hotplug on MSI Claw A8 root bridge 2026-08-08 0:31 ` Derek John Clark @ 2026-08-08 8:16 ` Lukas Wunner 2026-08-09 3:34 ` Derek John Clark 0 siblings, 1 reply; 10+ messages in thread From: Lukas Wunner @ 2026-08-08 8:16 UTC (permalink / raw) To: Derek John Clark Cc: Bjorn Helgaas, Pierre-Loup A . Griffais, linux-pci, linux-kernel On Fri, Aug 07, 2026 at 05:31:04PM -0700, Derek John Clark wrote: > [ 25.686128] rtsx_pci 0000:c2:00.0: pciehp: pdev subsystem vendor ID mismatch [...] > reg=0x525a10ec expected=0x14af1462 The subsystem vendor ID was 0x1462 (MSI) when the card reader was enumerated on boot and it is 0x10ec (Realtek) on resume. Likewise the subystem device ID was 0x14af on boot, 0x525a on resume. It's another bug in the MSI BIOS (in addition to marking the Root Port hotplug-capable): The BIOS neglected to re-initialize the subsystem vendor/device ID registers in config space on resume to the same values that it set on boot. The card reader was likely reset while going through the system sleep transition and its config space thus needs to be re-initialized. What happens is that pciehp marks the card reader disconnected because it assumes that it was replaced during system sleep. It then synthesizes a Presence Detect Changed event: https://elixir.bootlin.com/linux/v7.2-rc6/source/drivers/pci/hotplug/pciehp_core.c#L300 So the pciehp interrupt thread pciehp_ist() will remove the rtsx_pci driver and de-enumerate the device. Replacing or removing PCIe devices during system sleep happens all the time (e.g. with Thunderbolt) and we haven't seen issues with regards to that. The likelihood is higher that the lockup is caused by rtsx_pci. Maybe rtsx_pci_remove() blocks somewhere and prevents forward progress of the resume transition. In particular, it calls pm_runtime_get_sync() and cancel_delayed_work_sync(), which are blocking calls, i.e. they wait for something else to happen. You could try commenting those calls out and see if it avoids the lockup: https://elixir.bootlin.com/linux/v7.2-rc6/source/drivers/misc/cardreader/rtsx_pcr.c#L1608 Another possibility is a crash due to a NULL pointer deref or GPF somewhere in that driver. This is pretty difficult to debug if there's no possibility to see any dmesg output. Even if we manage to find the root cause of the lockup, as long as the BIOS isn't fixed, the card reader will always be de-enumerated and re-enumerated on resume and consequently the card will briefly disappear. This will make it impossible to use the card as root filesystem. > I've been using a SanDisk Ultra 32GB microSDHC UHS-I Card. And yes, it > only happens when the card is present. If you go to sleep and resume without SD card, do you then see in dmesg that the card reader was de-enumerated and re-enumerated? Normally there should be at least a "Card not present" / "Card present" message from pciehp, if it sensed a replaced device. ("Card" refers to PCIe card, not SD card.) Also, what's the subsystem vendor/device ID as seen with lspci after a successful system sleep transition (successful = without SD card)? Thanks, Lukas ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] pci: quirks: Disable native PCIe hotplug on MSI Claw A8 root bridge 2026-08-08 8:16 ` Lukas Wunner @ 2026-08-09 3:34 ` Derek John Clark 2026-08-09 6:22 ` Lukas Wunner 0 siblings, 1 reply; 10+ messages in thread From: Derek John Clark @ 2026-08-09 3:34 UTC (permalink / raw) To: Lukas Wunner Cc: Bjorn Helgaas, Pierre-Loup A . Griffais, linux-pci, linux-kernel On Sat, Aug 8, 2026 at 1:16 AM Lukas Wunner <lukas@wunner.de> wrote: > > On Fri, Aug 07, 2026 at 05:31:04PM -0700, Derek John Clark wrote: > > [ 25.686128] rtsx_pci 0000:c2:00.0: pciehp: pdev subsystem vendor ID mismatch > [...] > > reg=0x525a10ec expected=0x14af1462 > > The subsystem vendor ID was 0x1462 (MSI) when the card reader was > enumerated on boot and it is 0x10ec (Realtek) on resume. > > Likewise the subystem device ID was 0x14af on boot, 0x525a on resume. > > It's another bug in the MSI BIOS (in addition to marking the Root Port > hotplug-capable): The BIOS neglected to re-initialize the subsystem > vendor/device ID registers in config space on resume to the same > values that it set on boot. The card reader was likely reset while > going through the system sleep transition and its config space > thus needs to be re-initialized. > > What happens is that pciehp marks the card reader disconnected > because it assumes that it was replaced during system sleep. > It then synthesizes a Presence Detect Changed event: > > https://elixir.bootlin.com/linux/v7.2-rc6/source/drivers/pci/hotplug/pciehp_core.c#L300 > > So the pciehp interrupt thread pciehp_ist() will remove the > rtsx_pci driver and de-enumerate the device. > > Replacing or removing PCIe devices during system sleep happens > all the time (e.g. with Thunderbolt) and we haven't seen issues > with regards to that. The likelihood is higher that the lockup > is caused by rtsx_pci. > > Maybe rtsx_pci_remove() blocks somewhere and prevents forward > progress of the resume transition. In particular, it calls > pm_runtime_get_sync() and cancel_delayed_work_sync(), which are > blocking calls, i.e. they wait for something else to happen. > You could try commenting those calls out and see if it avoids > the lockup: > > https://elixir.bootlin.com/linux/v7.2-rc6/source/drivers/misc/cardreader/rtsx_pcr.c#L1608 This was good instinct. I was able to drill down and find approximately where it is hanging. rtsx_pci_remove() ->mfd_remove_devices() -> mfd_remove_devices_fn() -> platform_device_unregister() -> platform_device_del() -> device_del() -> bus_remove_device() -> device_release_driver() -> device_release_driver_internal() -> __device_release_driver() -> device_remove() In device_remove() (drivers/base/dd.c) it gets past device_remove_groups() and hangs before completing this if block: if (dev->bus && dev->bus->remove) dev->bus->remove(dev); else if (dev->driver->remove) dev->driver->remove(dev); After this logging stops. - Derek > Another possibility is a crash due to a NULL pointer deref or > GPF somewhere in that driver. This is pretty difficult to debug > if there's no possibility to see any dmesg output. > Even if we manage to find the root cause of the lockup, as long as > the BIOS isn't fixed, the card reader will always be de-enumerated > and re-enumerated on resume and consequently the card will briefly > disappear. This will make it impossible to use the card as root > filesystem. I agree. Even if we were to convince them to fix it, there are plenty of devices in the wild that will likely never get the BIOS update since MSI isn't using LFVS for this device. (I'm not sure if they do that for any device) > > I've been using a SanDisk Ultra 32GB microSDHC UHS-I Card. And yes, it > > only happens when the card is present. > > If you go to sleep and resume without SD card, do you then see in dmesg > that the card reader was de-enumerated and re-enumerated? Normally > there should be at least a "Card not present" / "Card present" message > from pciehp, if it sensed a replaced device. ("Card" refers to PCIe card, > not SD card.) i get card not present, then card present, then link up. > > Also, what's the subsystem vendor/device ID as seen with lspci after > a successful system sleep transition (successful = without SD card)? it reflects the Realtek device VID/PID. I also tested resume after the 1 cycle with it re-inserted and it works fine after the first cycle, presumably because the device id matches on the second run and beyond Thanks, Derek. > Thanks, > > Lukas ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] pci: quirks: Disable native PCIe hotplug on MSI Claw A8 root bridge 2026-08-09 3:34 ` Derek John Clark @ 2026-08-09 6:22 ` Lukas Wunner 2026-08-10 20:57 ` Derek John Clark 0 siblings, 1 reply; 10+ messages in thread From: Lukas Wunner @ 2026-08-09 6:22 UTC (permalink / raw) To: Derek John Clark Cc: Bjorn Helgaas, Pierre-Loup A . Griffais, linux-pci, linux-kernel On Sat, Aug 08, 2026 at 08:34:38PM -0700, Derek John Clark wrote: > This was good instinct. I was able to drill down and find > approximately where it is hanging. > > rtsx_pci_remove() ->mfd_remove_devices() -> mfd_remove_devices_fn() -> > platform_device_unregister() -> platform_device_del() -> device_del() > -> bus_remove_device() -> device_release_driver() -> > device_release_driver_internal() -> __device_release_driver() -> > device_remove() > > In device_remove() (drivers/base/dd.c) it gets past > device_remove_groups() and hangs before completing this if block: > > if (dev->bus && dev->bus->remove) > dev->bus->remove(dev); > else if (dev->driver->remove) > dev->driver->remove(dev); Okay, that ->remove callback should be rtsx_pci_sdmmc_drv_remove() in drivers/mmc/host/rtsx_pci_sdmmc.c. Can you maybe identify where execution stops in that function? Basically the rtsx_pci driver creates a child device of the PCI device and the mmc host driver binds to that child device. Removing the mmc host driver fails here for some reason. > > If you go to sleep and resume without SD card, do you then see in dmesg > > that the card reader was de-enumerated and re-enumerated? Normally > > there should be at least a "Card not present" / "Card present" message > > from pciehp, if it sensed a replaced device. ("Card" refers to PCIe card, > > not SD card.) > > i get card not present, then card present, then link up. > > > Also, what's the subsystem vendor/device ID as seen with lspci after > > a successful system sleep transition (successful = without SD card)? > > it reflects the Realtek device VID/PID. I also tested resume after the > 1 cycle with it re-inserted and it works fine after the first cycle, > presumably because the device id matches on the second run and beyond Right, when the device is re-enumerated after the first system sleep transition, the Realtek subdevice vendor/device ID is cached and that matches on all subsequent system sleep transitions. So if we manage to find and eliminate the cause of the hang, there'll only be the inconvenience of the removal/re-enumeration on first sleep. Thanks, Lukas ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] pci: quirks: Disable native PCIe hotplug on MSI Claw A8 root bridge 2026-08-09 6:22 ` Lukas Wunner @ 2026-08-10 20:57 ` Derek John Clark 2026-08-11 5:19 ` Lukas Wunner 0 siblings, 1 reply; 10+ messages in thread From: Derek John Clark @ 2026-08-10 20:57 UTC (permalink / raw) To: Lukas Wunner Cc: Bjorn Helgaas, Pierre-Loup A . Griffais, linux-pci, linux-kernel On Sat, Aug 8, 2026 at 11:22 PM Lukas Wunner <lukas@wunner.de> wrote: > > On Sat, Aug 08, 2026 at 08:34:38PM -0700, Derek John Clark wrote: > > This was good instinct. I was able to drill down and find > > approximately where it is hanging. > > > > rtsx_pci_remove() ->mfd_remove_devices() -> mfd_remove_devices_fn() -> > > platform_device_unregister() -> platform_device_del() -> device_del() > > -> bus_remove_device() -> device_release_driver() -> > > device_release_driver_internal() -> __device_release_driver() -> > > device_remove() > > > > In device_remove() (drivers/base/dd.c) it gets past > > device_remove_groups() and hangs before completing this if block: > > > > if (dev->bus && dev->bus->remove) > > dev->bus->remove(dev); > > else if (dev->driver->remove) > > dev->driver->remove(dev); > > Okay, that ->remove callback should be rtsx_pci_sdmmc_drv_remove() > in drivers/mmc/host/rtsx_pci_sdmmc.c. Can you maybe identify where > execution stops in that function? > > Basically the rtsx_pci driver creates a child device of the PCI device > and the mmc host driver binds to that child device. Removing the > mmc host driver fails here for some reason. Hi Lukas, I was able to drill down further. When the mmc device gets to blk_report_disk_dead() in block/genhd.c there is an xa_for_each loop. On the second loop of that it seems to hang in bdev_mark_dead(). That sets a callback that runs fs_bdev_mark_dead() which then runs sync_filesystem(). This is all hit because the "surprise" bool is set to false unconditionally in __del_gendisk(). Commenting out this from __del_gendisk(): if (!test_bit(GD_DEAD, &disk->state)) blk_report_disk_dead(disk, false); Avoids the hang. I'm not sure how much further we want to go down into this as we've moved quite far out of the driver space and are now well into the block subsystem. I'm working on a proof of concept that would allow the driver to inform the mmc core that this was a surprise, but I'm not sure how that would land. From what I've read the mmc core was intentionally detached from higher level drivers, though a few notification methods do exist. If I get that fully working I'll post it here but it will probably be a few days as I've run into some other obligations this week. > > > If you go to sleep and resume without SD card, do you then see in dmesg > > > that the card reader was de-enumerated and re-enumerated? Normally > > > there should be at least a "Card not present" / "Card present" message > > > from pciehp, if it sensed a replaced device. ("Card" refers to PCIe card, > > > not SD card.) > > > > i get card not present, then card present, then link up. > > > > > Also, what's the subsystem vendor/device ID as seen with lspci after > > > a successful system sleep transition (successful = without SD card)? > > > > it reflects the Realtek device VID/PID. I also tested resume after the > > 1 cycle with it re-inserted and it works fine after the first cycle, > > presumably because the device id matches on the second run and beyond > > Right, when the device is re-enumerated after the first system sleep > transition, the Realtek subdevice vendor/device ID is cached and that > matches on all subsequent system sleep transitions. So if we manage > to find and eliminate the cause of the hang, there'll only be the > inconvenience of the removal/re-enumeration on first sleep. I suppose that is true, but in that case we still wouldn't be able to use it as the root fs since it gets removed/re-added during a suspend. Would the quirk I submitted not have this compromise? > Thanks, > > Lukas ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] pci: quirks: Disable native PCIe hotplug on MSI Claw A8 root bridge 2026-08-10 20:57 ` Derek John Clark @ 2026-08-11 5:19 ` Lukas Wunner 2026-08-11 8:59 ` Lukas Wunner 0 siblings, 1 reply; 10+ messages in thread From: Lukas Wunner @ 2026-08-11 5:19 UTC (permalink / raw) To: Derek John Clark Cc: Bjorn Helgaas, Pierre-Loup A . Griffais, linux-pci, linux-kernel On Mon, Aug 10, 2026 at 01:57:58PM -0700, Derek John Clark wrote: > I was able to drill down further. When the mmc device gets to > blk_report_disk_dead() in block/genhd.c there is an xa_for_each loop. > On the second loop of that it seems to hang in bdev_mark_dead(). That > sets a callback that runs fs_bdev_mark_dead() which then runs > sync_filesystem(). This is all hit because the "surprise" bool is set > to false unconditionally in __del_gendisk(). > > Commenting out this from __del_gendisk(): > if (!test_bit(GD_DEAD, &disk->state)) > blk_report_disk_dead(disk, false); > > Avoids the hang. Thank you so much, you've root-caused the issue: We're missing a call to blk_mark_disk_dead() at the top of rtsx_pci_sdmmc_drv_remove() if the underlying pci_dev is marked disconnected. Let me get back to you with a fix in a bit. > I'm not sure how much further we want to go down into this as we've > moved quite far out of the driver space and are now well into the > block subsystem. I'm working on a proof of concept that would allow > the driver to inform the mmc core that this was a surprise, but I'm > not sure how that would land. From what I've read the mmc core was > intentionally detached from higher level drivers, though a few > notification methods do exist. If I get that fully working I'll post > it here but it will probably be a few days as I've run into some other > obligations this week. It's fine, you don't need to invest any more time into that and I understand your frustration about how much effort is necessary to fix this. The issue you've found affects anyone removing e.g. a Thunderbolt- attached dock with an MMC card reader during system sleep. So it's useful and important to have root-caused and fix that. Unfortunately kernel development often means getting sidetracked like this. > I suppose that is true, but in that case we still wouldn't be able to > use it as the root fs since it gets removed/re-added during a suspend. > Would the quirk I submitted not have this compromise? Okay I wasn't sure that using the MMC card as root filesystem is even possible on this product. If that's a potential use case, it needs a separate fix. I'll come up with a proposal for that one as well. Thanks, Lukas ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] pci: quirks: Disable native PCIe hotplug on MSI Claw A8 root bridge 2026-08-11 5:19 ` Lukas Wunner @ 2026-08-11 8:59 ` Lukas Wunner 2026-08-11 9:19 ` Derek J. Clark 0 siblings, 1 reply; 10+ messages in thread From: Lukas Wunner @ 2026-08-11 8:59 UTC (permalink / raw) To: Derek John Clark Cc: Bjorn Helgaas, Pierre-Loup A . Griffais, linux-pci, linux-kernel, Ulf Hansson [cc += Ulf, start of thread is here: https://lore.kernel.org/all/20260806214808.1202819-1-derekjohn.clark@gmail.com/ ] On Tue, Aug 11, 2026 at 07:19:12AM +0200, Lukas Wunner wrote: > On Mon, Aug 10, 2026 at 01:57:58PM -0700, Derek John Clark wrote: > > I was able to drill down further. When the mmc device gets to > > blk_report_disk_dead() in block/genhd.c there is an xa_for_each loop. > > On the second loop of that it seems to hang in bdev_mark_dead(). That > > sets a callback that runs fs_bdev_mark_dead() which then runs > > sync_filesystem(). This is all hit because the "surprise" bool is set > > to false unconditionally in __del_gendisk(). > > > > Commenting out this from __del_gendisk(): > > if (!test_bit(GD_DEAD, &disk->state)) > > blk_report_disk_dead(disk, false); > > > > Avoids the hang. > > Thank you so much, you've root-caused the issue: We're missing a call > to blk_mark_disk_dead() at the top of rtsx_pci_sdmmc_drv_remove() > if the underlying pci_dev is marked disconnected. Let me get back > to you with a fix in a bit. So the completely untested patch below might be an upstreamable approach. I'm adding MMC maintainer Ulf to cc in case he has early feedback. If this works, feel free to submit a proper patch and claim authorship if you want. I'll gladly let you have that given the amount of time you've already sunk into it. If you'd rather have me submit a patch (and deal with any regressions caused by it), I'll be happy to do that as well. Thanks! -- >8 -- diff --git a/drivers/mmc/core/block.c b/drivers/mmc/core/block.c index 0274e8d..1a82233 100644 --- a/drivers/mmc/core/block.c +++ b/drivers/mmc/core/block.c @@ -2978,8 +2978,11 @@ static int mmc_blk_alloc_parts(struct mmc_card *card, struct mmc_blk_data *md) return 0; } -static void mmc_blk_remove_req(struct mmc_blk_data *md) +static void mmc_blk_remove_req(struct mmc_card *card, struct mmc_blk_data *md) { + if (mmc_card_removed(card)) + blk_mark_disk_dead(md->disk); + /* * Flush remaining requests and free queues. It is freeing the queue * that stops new requests from being accepted. @@ -3006,7 +3009,7 @@ static void mmc_blk_remove_parts(struct mmc_card *card, list_for_each_safe(pos, q, &md->part) { part_md = list_entry(pos, struct mmc_blk_data, part); list_del(pos); - mmc_blk_remove_req(part_md); + mmc_blk_remove_req(card, part_md); } } @@ -3252,7 +3255,7 @@ static int mmc_blk_probe(struct mmc_card *card) out: mmc_blk_remove_parts(card, md); - mmc_blk_remove_req(md); + mmc_blk_remove_req(card, md); out_free: destroy_workqueue(card->complete_wq); return ret; @@ -3273,7 +3276,7 @@ static void mmc_blk_remove(struct mmc_card *card) if (!mmc_card_sd_combo(card)) pm_runtime_disable(&card->dev); pm_runtime_put_noidle(&card->dev); - mmc_blk_remove_req(md); + mmc_blk_remove_req(card, md); destroy_workqueue(card->complete_wq); } diff --git a/drivers/mmc/core/host.c b/drivers/mmc/core/host.c index b7ce313..658b241 100644 --- a/drivers/mmc/core/host.c +++ b/drivers/mmc/core/host.c @@ -703,3 +703,17 @@ void mmc_free_host(struct mmc_host *host) } EXPORT_SYMBOL(mmc_free_host); + +/** + * mmc_host_set_removed - declare host removed + * @host: mmc host + * + * Declare the host (and any inserted card) removed and inaccessible. + */ +void mmc_host_set_removed(struct mmc_host *host) +{ + if (host->card) + mmc_card_set_removed(host->card); +} + +EXPORT_SYMBOL(mmc_host_set_removed); diff --git a/drivers/mmc/host/rtsx_pci_sdmmc.c b/drivers/mmc/host/rtsx_pci_sdmmc.c index 8dfbc62..3f97659f 100644 --- a/drivers/mmc/host/rtsx_pci_sdmmc.c +++ b/drivers/mmc/host/rtsx_pci_sdmmc.c @@ -1511,6 +1511,9 @@ static void rtsx_pci_sdmmc_drv_remove(struct platform_device *pdev) pcr->slots[RTSX_SD_CARD].card_event = NULL; mmc = host->mmc; + if (pci_dev_is_disconnected(pcr->pci)) + mmc_host_set_removed(mmc); + cancel_work_sync(&host->work); mutex_lock(&host->host_mutex); diff --git a/include/linux/mmc/host.h b/include/linux/mmc/host.h index ba84f02..a240306 100644 --- a/include/linux/mmc/host.h +++ b/include/linux/mmc/host.h @@ -588,6 +588,7 @@ struct mmc_host { int mmc_add_host(struct mmc_host *); void mmc_remove_host(struct mmc_host *); void mmc_free_host(struct mmc_host *); +void mmc_host_set_removed(struct mmc_host *host); void mmc_of_parse_clk_phase(struct device *dev, struct mmc_clk_phase_map *map); int mmc_of_parse(struct mmc_host *host); ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH v2] pci: quirks: Disable native PCIe hotplug on MSI Claw A8 root bridge 2026-08-11 8:59 ` Lukas Wunner @ 2026-08-11 9:19 ` Derek J. Clark 0 siblings, 0 replies; 10+ messages in thread From: Derek J. Clark @ 2026-08-11 9:19 UTC (permalink / raw) To: Lukas Wunner Cc: Bjorn Helgaas, Pierre-Loup A . Griffais, linux-pci, linux-kernel, Ulf Hansson On August 11, 2026 5:59:09 PM GMT+09:00, Lukas Wunner <lukas@wunner.de> wrote: >[cc += Ulf, start of thread is here: >https://lore.kernel.org/all/20260806214808.1202819-1-derekjohn.clark@gmail.com/ >] > >On Tue, Aug 11, 2026 at 07:19:12AM +0200, Lukas Wunner wrote: >> On Mon, Aug 10, 2026 at 01:57:58PM -0700, Derek John Clark wrote: >> > I was able to drill down further. When the mmc device gets to >> > blk_report_disk_dead() in block/genhd.c there is an xa_for_each loop. >> > On the second loop of that it seems to hang in bdev_mark_dead(). That >> > sets a callback that runs fs_bdev_mark_dead() which then runs >> > sync_filesystem(). This is all hit because the "surprise" bool is set >> > to false unconditionally in __del_gendisk(). >> > >> > Commenting out this from __del_gendisk(): >> > if (!test_bit(GD_DEAD, &disk->state)) >> > blk_report_disk_dead(disk, false); >> > >> > Avoids the hang. >> >> Thank you so much, you've root-caused the issue: We're missing a call >> to blk_mark_disk_dead() at the top of rtsx_pci_sdmmc_drv_remove() >> if the underlying pci_dev is marked disconnected. Let me get back >> to you with a fix in a bit. > >So the completely untested patch below might be an upstreamable approach. >I'm adding MMC maintainer Ulf to cc in case he has early feedback. > Great, I'll give it a test soon and follow up. >If this works, feel free to submit a proper patch and claim authorship >if you want. I'll gladly let you have that given the amount of time >you've already sunk into it. If you'd rather have me submit a patch >(and deal with any regressions caused by it), I'll be happy to do >that as well. Thanks! > I think tested by tags will be sufficient once I've completed that. The shape of your patch is a bit different than what I was working on and is a better approach. I appreciate your help on this so far. Thanks, Derek >-- >8 -- > >diff --git a/drivers/mmc/core/block.c b/drivers/mmc/core/block.c >index 0274e8d..1a82233 100644 >--- a/drivers/mmc/core/block.c >+++ b/drivers/mmc/core/block.c >@@ -2978,8 +2978,11 @@ static int mmc_blk_alloc_parts(struct mmc_card *card, struct mmc_blk_data *md) > return 0; > } > >-static void mmc_blk_remove_req(struct mmc_blk_data *md) >+static void mmc_blk_remove_req(struct mmc_card *card, struct mmc_blk_data *md) > { >+ if (mmc_card_removed(card)) >+ blk_mark_disk_dead(md->disk); >+ > /* > * Flush remaining requests and free queues. It is freeing the queue > * that stops new requests from being accepted. >@@ -3006,7 +3009,7 @@ static void mmc_blk_remove_parts(struct mmc_card *card, > list_for_each_safe(pos, q, &md->part) { > part_md = list_entry(pos, struct mmc_blk_data, part); > list_del(pos); >- mmc_blk_remove_req(part_md); >+ mmc_blk_remove_req(card, part_md); > } > } > >@@ -3252,7 +3255,7 @@ static int mmc_blk_probe(struct mmc_card *card) > > out: > mmc_blk_remove_parts(card, md); >- mmc_blk_remove_req(md); >+ mmc_blk_remove_req(card, md); > out_free: > destroy_workqueue(card->complete_wq); > return ret; >@@ -3273,7 +3276,7 @@ static void mmc_blk_remove(struct mmc_card *card) > if (!mmc_card_sd_combo(card)) > pm_runtime_disable(&card->dev); > pm_runtime_put_noidle(&card->dev); >- mmc_blk_remove_req(md); >+ mmc_blk_remove_req(card, md); > destroy_workqueue(card->complete_wq); > } > >diff --git a/drivers/mmc/core/host.c b/drivers/mmc/core/host.c >index b7ce313..658b241 100644 >--- a/drivers/mmc/core/host.c >+++ b/drivers/mmc/core/host.c >@@ -703,3 +703,17 @@ void mmc_free_host(struct mmc_host *host) > } > > EXPORT_SYMBOL(mmc_free_host); >+ >+/** >+ * mmc_host_set_removed - declare host removed >+ * @host: mmc host >+ * >+ * Declare the host (and any inserted card) removed and inaccessible. >+ */ >+void mmc_host_set_removed(struct mmc_host *host) >+{ >+ if (host->card) >+ mmc_card_set_removed(host->card); >+} >+ >+EXPORT_SYMBOL(mmc_host_set_removed); >diff --git a/drivers/mmc/host/rtsx_pci_sdmmc.c b/drivers/mmc/host/rtsx_pci_sdmmc.c >index 8dfbc62..3f97659f 100644 >--- a/drivers/mmc/host/rtsx_pci_sdmmc.c >+++ b/drivers/mmc/host/rtsx_pci_sdmmc.c >@@ -1511,6 +1511,9 @@ static void rtsx_pci_sdmmc_drv_remove(struct platform_device *pdev) > pcr->slots[RTSX_SD_CARD].card_event = NULL; > mmc = host->mmc; > >+ if (pci_dev_is_disconnected(pcr->pci)) >+ mmc_host_set_removed(mmc); >+ > cancel_work_sync(&host->work); > > mutex_lock(&host->host_mutex); >diff --git a/include/linux/mmc/host.h b/include/linux/mmc/host.h >index ba84f02..a240306 100644 >--- a/include/linux/mmc/host.h >+++ b/include/linux/mmc/host.h >@@ -588,6 +588,7 @@ struct mmc_host { > int mmc_add_host(struct mmc_host *); > void mmc_remove_host(struct mmc_host *); > void mmc_free_host(struct mmc_host *); >+void mmc_host_set_removed(struct mmc_host *host); > void mmc_of_parse_clk_phase(struct device *dev, > struct mmc_clk_phase_map *map); > int mmc_of_parse(struct mmc_host *host); ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2026-08-11 9:19 UTC | newest] Thread overview: 10+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-08-06 21:48 [PATCH v2] pci: quirks: Disable native PCIe hotplug on MSI Claw A8 root bridge Derek J. Clark 2026-08-07 8:53 ` Lukas Wunner 2026-08-08 0:31 ` Derek John Clark 2026-08-08 8:16 ` Lukas Wunner 2026-08-09 3:34 ` Derek John Clark 2026-08-09 6:22 ` Lukas Wunner 2026-08-10 20:57 ` Derek John Clark 2026-08-11 5:19 ` Lukas Wunner 2026-08-11 8:59 ` Lukas Wunner 2026-08-11 9:19 ` Derek J. Clark
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox