Re: [REGRESSION][BISECTED] usb: hub: total system freeze after running adb

Re: [REGRESSION][BISECTED] usb: hub: total system freeze after running adb

[Slavik Dev]

Hi,

I'm hitting what appears to be the same regression, on the same xHCI
hardware as the original report. I spent months chasing this as a
hardware problem (RAM, PSU, BIOS) before the adb correlation
eventually led me to this thread. Earlier today I applied the
workaround discussed here (disabling USB runtime PM), and so far it
is holding up. Sharing my data in case it helps revive the
investigation -- until the workaround, this reproduced reliably for
me, up to several freezes per evening.

Hardware:
- AMD Ryzen 7 7700 (Raphael, display on iGPU), ASUS TUF GAMING B650-E
  WIFI, BIOS 3881 (AGESA 1.3.0.1b)
- The affected controller from the original report is present:
  0b:00.0 USB controller [1022:15b8] (AMD Raphael USB 2.0 xHCI)

Kernels (Linux Mint 22.3 / Ubuntu HWE builds):
- Freezes observed on 6.17.0-14/-19-generic and 6.14.0-37-generic
  (based on 6.14.11 + stable backports, built 2025-11-20).
- First freeze on 2026-03-10, shortly after kernel package updates;
  16 freezes total through 2026-08-03.

Correlation with adb:
- The last 4 freezes occurred 5, 12, 35 and 81 minutes after the adb
  server was started (using the once-per-boot "adb uses wireless
  extensions" kernel warning as the start marker in the journal).
- Earlier freezes also occurred with an Android phone plugged in while
  the adb server was NOT running -- including one freeze 2 seconds
  after unplugging the phone. Consistent with hub runtime
  suspend/resume transitions being the trigger rather than adb itself.

Freeze signature:
- Journal stops mid-second, no oops/panic, no MCE/WHEA, efi-pstore
  empty. nmi_watchdog=1 and hardlockup_panic=1 never fire.
- Notably: the chipset watchdog (sp5100_tco + systemd
  RuntimeWatchdogSec=30) DOES fire and resets the machine ~30 s after
  the hang, so the platform stays alive -- it is the kernel/CPUs that
  stop responding.

Ruled out on this machine:
- RAM: freezes identically with DDR5 at EXPO 6000, EXPO 5600 and
  JEDEC stock 4800.
- Pure load: 60 minutes of stress-ng (16 CPU workers + 16 GB VM
  workers, no adb running) survives without issues.
- Firmware: BIOS update 3223 -> 3881 (AGESA 1.3.0.1b) made no
  difference.

Side observation: in freezing sessions, Bluetooth (MT7922 on the same
bus as most devices) repeatedly logs "hci0: ACL packet for unknown
connection handle" before the freeze.

Workaround status:
- Running with runtime PM disabled for all USB devices
  (power/control=on via udev rule) since 2026-08-03. Reporting while
  it's fresh: the adb server has since been running continuously,
  already well past my longest previously observed time-to-freeze
  (81 min), and so far so good.

Re: [REGRESSION][BISECTED] usb: hub: total system freeze after running adb

[Michal Pecio]

On Mon, 3 Aug 2026 16:54:58 +0300, Slavik Dev wrote:
> Hi,
> 
> I'm hitting what appears to be the same regression, on the same xHCI
> hardware as the original report.

Hi,

I got your email from linux-usb with no working reference to the
original thread, I presume you mean this one:

https://lore.kernel.org/regressions/qc0nhk9c6l0a08bkfeplrm3qjssgrjkvkp@sonic.net/

And it appears that this (later) bugzilla issue was another dupe,
which fact had been overlooked at the time:

https://bugzilla.kernel.org/show_bug.cgi?id=221103

> Earlier today I applied the workaround discussed here (disabling USB
> runtime PM), and so far it is holding up.

Slightly simpler workaround is described in bugzilla comment 16
and it should work for you too if it's the same bug.

In comment 26 there is a reference to another bug affecting AMD
hardware power management, where some success has been achieved.
AFAIK no one tried if those patches would work for the ADB issue.

Somebody has apparently bisected this to the commit below.
Failing the above, does reverting the commit work for you?

commit 8f5b7e2bec1c36578fdaa74a6951833541103e27
2025-06-11 14:24:41 +0300
usb: hub: fix detection of high tier USB3 devices behind suspended hubs

> Hardware:
> - AMD Ryzen 7 7700 (Raphael, display on iGPU), ASUS TUF GAMING B650-E
>   WIFI, BIOS 3881 (AGESA 1.3.0.1b)
> - The affected controller from the original report is present:
>   0b:00.0 USB controller [1022:15b8] (AMD Raphael USB 2.0 xHCI)
> 
> Kernels (Linux Mint 22.3 / Ubuntu HWE builds):
> - Freezes observed on 6.17.0-14/-19-generic and 6.14.0-37-generic
>   (based on 6.14.11 + stable backports, built 2025-11-20).
> - First freeze on 2026-03-10, shortly after kernel package updates;
>   16 freezes total through 2026-08-03.
> 
> Correlation with adb:
> - The last 4 freezes occurred 5, 12, 35 and 81 minutes after the adb
>   server was started (using the once-per-boot "adb uses wireless
>   extensions" kernel warning as the start marker in the journal).
> - Earlier freezes also occurred with an Android phone plugged in while
>   the adb server was NOT running -- including one freeze 2 seconds
>   after unplugging the phone. Consistent with hub runtime
>   suspend/resume transitions being the trigger rather than adb itself.
> 
> Freeze signature:
> - Journal stops mid-second, no oops/panic, no MCE/WHEA, efi-pstore
>   empty. nmi_watchdog=1 and hardlockup_panic=1 never fire.
> - Notably: the chipset watchdog (sp5100_tco + systemd
>   RuntimeWatchdogSec=30) DOES fire and resets the machine ~30 s after
>   the hang, so the platform stays alive -- it is the kernel/CPUs that
>   stop responding.
> 
> Ruled out on this machine:
> - RAM: freezes identically with DDR5 at EXPO 6000, EXPO 5600 and
>   JEDEC stock 4800.
> - Pure load: 60 minutes of stress-ng (16 CPU workers + 16 GB VM
>   workers, no adb running) survives without issues.
> - Firmware: BIOS update 3223 -> 3881 (AGESA 1.3.0.1b) made no
>   difference.
> 
> Side observation: in freezing sessions, Bluetooth (MT7922 on the same
> bus as most devices) repeatedly logs "hci0: ACL packet for unknown
> connection handle" before the freeze.
> 
> Workaround status:
> - Running with runtime PM disabled for all USB devices
>   (power/control=on via udev rule) since 2026-08-03. Reporting while
>   it's fresh: the adb server has since been running continuously,
>   already well past my longest previously observed time-to-freeze
>   (81 min), and so far so good.

[REGRESSION][BISECTED] usb: hub: total system freeze after running adb

[Forest]

#regzbot introduced 8f5b7e2bec1c

Running adb (android debug bridge) commands causes the entire system to
freeze some time later.

The freeze usually occurs within two hours of running adb, but not always.
Stopping adb's persistent background process with `adb kill-server` seems to
avoid the freeze. (However, this is not always possible, since the system
sometimes freezes just a few seconds after adb is launched.)

It occurs even when no android devices are connected to the system.

I am unsure whether user activity affects how soon the freeze happens.

Observations when it happens:

- No messages are written to dmesg or syslog.
- Virtual console switching no longer works.
- SysRq key commands no longer work, as far as I can tell.
- Whatever was on the display remains intact.
- Any already-playing audio enters a short loop, as though the motherboard's
  sound device is playing from a ring buffer that no longer gets updated.

I have reproduced it on mainline (6.18.0-rc5) and various older kernel
versions as early as 6.12.48, built with Debian config files and
`make olddefconfig`. Tested using Debian Stable 13.2 (trixie), and its
adb 1:34.0.5-12 package.

Git bisect identifies the culprit:
commit 8f5b7e2bec1c36578fdaa74a6951833541103e27
2025-06-11 14:24:41 +0300
usb: hub: fix detection of high tier USB3 devices behind suspended hubs

Reverting that commit (along with two later commits that build upon it)
seems to fix the problem in the mainline and longterm kernels that I tried.

Re: [REGRESSION][BISECTED] usb: hub: total system freeze after running adb

[Mathias Nyman]

Hi

On 11/17/25 22:18, Forest wrote:
> #regzbot introduced 8f5b7e2bec1c
> 
> Running adb (android debug bridge) commands causes the entire system to
> freeze some time later.
> 
> The freeze usually occurs within two hours of running adb, but not always.
> Stopping adb's persistent background process with `adb kill-server` seems to
> avoid the freeze. (However, this is not always possible, since the system
> sometimes freezes just a few seconds after adb is launched.)
> 
> It occurs even when no android devices are connected to the system.
> 
> I am unsure whether user activity affects how soon the freeze happens.
> 
> Observations when it happens:
> 
> - No messages are written to dmesg or syslog.
> - Virtual console switching no longer works.
> - SysRq key commands no longer work, as far as I can tell.
> - Whatever was on the display remains intact.
> - Any already-playing audio enters a short loop, as though the motherboard's
>    sound device is playing from a ring buffer that no longer gets updated.
> 
> I have reproduced it on mainline (6.18.0-rc5) and various older kernel
> versions as early as 6.12.48, built with Debian config files and
> `make olddefconfig`. Tested using Debian Stable 13.2 (trixie), and its
> adb 1:34.0.5-12 package.
> 
> Git bisect identifies the culprit:
> commit 8f5b7e2bec1c36578fdaa74a6951833541103e27
> 2025-06-11 14:24:41 +0300
> usb: hub: fix detection of high tier USB3 devices behind suspended hubs
> 
> Reverting that commit (along with two later commits that build upon it)
> seems to fix the problem in the mainline and longterm kernels that I tried.

Could you share some info about this setup:

lspci -nnv -d ::0c03
lsusb -t

Enabling usb core and xhci dynamic debug could share some light on this:

mount -t debugfs none /sys/kernel/debug
echo 'module xhci_hcd =p' >/sys/kernel/debug/dynamic_debug/control
echo 'module usbcore =p' >/sys/kernel/debug/dynamic_debug/control
< Reproduce issue >
Send output of dmesg

If the bisected patch is the issue then disabling runtime suspend could help.
For all usb* roothubs:

echo on > /sys/bus/usb/devices/usb*/power/control

Does it affect anything?

Thanks
Mathias

Re: [REGRESSION][BISECTED] usb: hub: total system freeze after running adb

[Forest]

On Tue, 18 Nov 2025 11:21:18 +0200, Mathias Nyman wrote:

>If the bisected patch is the issue then disabling runtime suspend could help.
>For all usb* roothubs:
>
>echo on > /sys/bus/usb/devices/usb*/power/control
>
>Does it affect anything?

Yes. After setting all of those to 'on', I have been unable to
reproduce the freeze. Tested for nearly 12 hours so far, so I think
it does avoid the problem. (I'll do some more testing and report
back if that changes.)

>Could you share some info about this setup:

$ sudo lspci -nnv -d ::0c03
0c:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset USB 3.2 Controller [1022:43f7] (rev 01) (prog-if 30 [XHCI])
        Subsystem: ASMedia Technology Inc. Device [1b21:1142]
        Flags: bus master, fast devsel, latency 0, IRQ 24, IOMMU group 23
        Memory at f6900000 (64-bit, non-prefetchable) [size=32K]
        Capabilities: [50] MSI: Enable- Count=1/8 Maskable- 64bit+
        Capabilities: [68] MSI-X: Enable+ Count=8 Masked-
        Capabilities: [78] Power Management version 3
        Capabilities: [80] Express Legacy Endpoint, IntMsgNum 0
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [160] Latency Tolerance Reporting
        Kernel driver in use: xhci_hcd
        Kernel modules: xhci_pci

0e:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge USB 3.1 xHCI [1022:15b6] (prog-if 30 [XHCI])
        Subsystem: ASUSTeK Computer Inc. Device [1043:8877]
        Flags: fast devsel, IRQ 86, IOMMU group 28
        Memory at f6500000 (64-bit, non-prefetchable) [size=1M]
        Capabilities: [48] Vendor Specific Information: Len=08 <?>
        Capabilities: [50] Power Management version 3
        Capabilities: [64] Express Endpoint, IntMsgNum 0
        Capabilities: [a0] MSI: Enable- Count=1/8 Maskable- 64bit+
        Capabilities: [c0] MSI-X: Enable+ Count=8 Masked-
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [2a0] Access Control Services
        Kernel driver in use: xhci_hcd
        Kernel modules: xhci_pci

0e:00.4 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge USB 3.1 xHCI [1022:15b7] (prog-if 30 [XHCI])
        Subsystem: ASUSTeK Computer Inc. Device [1043:8877]
        Flags: fast devsel, IRQ 95, IOMMU group 29
        Memory at f6400000 (64-bit, non-prefetchable) [size=1M]
        Capabilities: [48] Vendor Specific Information: Len=08 <?>
        Capabilities: [50] Power Management version 3
        Capabilities: [64] Express Endpoint, IntMsgNum 0
        Capabilities: [a0] MSI: Enable- Count=1/8 Maskable- 64bit+
        Capabilities: [c0] MSI-X: Enable+ Count=8 Masked-
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [2a0] Access Control Services
        Kernel driver in use: xhci_hcd
        Kernel modules: xhci_pci

0f:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge USB 2.0 xHCI [1022:15b8] (prog-if 30 [XHCI])
        Subsystem: ASUSTeK Computer Inc. Device [1043:8877]
        Flags: fast devsel, IRQ 24, IOMMU group 31
        Memory at f6e00000 (64-bit, non-prefetchable) [size=1M]
        Capabilities: [48] Vendor Specific Information: Len=08 <?>
        Capabilities: [50] Power Management version 3
        Capabilities: [64] Express Endpoint, IntMsgNum 0
        Capabilities: [a0] MSI: Enable- Count=1/8 Maskable- 64bit+
        Capabilities: [c0] MSI-X: Enable+ Count=8 Masked-
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [270] Secondary PCI Express
        Capabilities: [2a0] Access Control Services
        Capabilities: [410] Physical Layer 16.0 GT/s <?>
        Capabilities: [450] Lane Margining at the Receiver
        Kernel driver in use: xhci_hcd
        Kernel modules: xhci_pci


$ sudo lsusb -t
/:  Bus 001.Port 001: Dev 001, Class=root_hub, Driver=xhci_hcd/12p, 480M
    |__ Port 005: Dev 002, If 0, Class=Wireless, Driver=btusb, 480M
    |__ Port 005: Dev 002, If 1, Class=Wireless, Driver=btusb, 480M
    |__ Port 005: Dev 002, If 2, Class=Wireless, Driver=[none], 480M
    |__ Port 006: Dev 003, If 0, Class=Human Interface Device, Driver=usbhid, 12M
    |__ Port 006: Dev 003, If 1, Class=Human Interface Device, Driver=usbhid, 12M
    |__ Port 006: Dev 003, If 2, Class=Human Interface Device, Driver=usbhid, 12M
    |__ Port 006: Dev 003, If 3, Class=Human Interface Device, Driver=usbhid, 12M
    |__ Port 007: Dev 004, If 0, Class=Human Interface Device, Driver=usbhid, 12M
    |__ Port 007: Dev 004, If 1, Class=Human Interface Device, Driver=usbhid, 12M
    |__ Port 011: Dev 005, If 0, Class=Vendor Specific Class, Driver=[none], 12M
    |__ Port 011: Dev 005, If 2, Class=Human Interface Device, Driver=usbhid, 12M
/:  Bus 002.Port 001: Dev 001, Class=root_hub, Driver=xhci_hcd/5p, 20000M/x2
/:  Bus 003.Port 001: Dev 001, Class=root_hub, Driver=xhci_hcd/2p, 480M
/:  Bus 004.Port 001: Dev 001, Class=root_hub, Driver=xhci_hcd/2p, 10000M
/:  Bus 005.Port 001: Dev 001, Class=root_hub, Driver=xhci_hcd/2p, 480M
/:  Bus 006.Port 001: Dev 001, Class=root_hub, Driver=xhci_hcd/2p, 10000M
/:  Bus 007.Port 001: Dev 001, Class=root_hub, Driver=xhci_hcd/1p, 480M


>Enabling usb core and xhci dynamic debug could share some light on this:
>
>mount -t debugfs none /sys/kernel/debug
>echo 'module xhci_hcd =p' >/sys/kernel/debug/dynamic_debug/control
>echo 'module usbcore =p' >/sys/kernel/debug/dynamic_debug/control
>< Reproduce issue >
>Send output of dmesg

With these enabled and adb running, dmesg produces a 90-line burst of
output roughly each second. This generated megabytes of output before
the freeze happened. Comparing the last three bursts, I found them to
be identical except for the relative order of a few lines, and the
timestamps.

I was redirecting `dmesg --follow` to a file while watching it in a
terminal, in hopes that any buffered output that wasn't written
to the file when the freeze happened would be visible on-screen.
A photo of the screen showed the same messages as found at the tail
of the file, but with newer timestamps.

Here is the last burst of logged messages:

[ 3516.610664] xhci_hcd 0000:0f:00.0: Setting command ring address to 0xffffe001
[ 3516.611616] xhci_hcd 0000:0f:00.0: xhci_resume: starting usb7 port polling.
[ 3516.611619] xhci_hcd 0000:0f:00.0: xhci_hub_status_data: stopping usb7 port polling
[ 3516.611621] xhci_hcd 0000:0f:00.0: hcd_pci_runtime_resume: 0
[ 3516.611624] usb usb7: usb auto-resume
[ 3516.611628] hub 7-0:1.0: hub_resume
[ 3516.611632] xhci_hcd 0000:0f:00.0: Get port status 7-1 read: 0x2a0, return 0x100
[ 3516.611689] hub 7-0:1.0: state 7 ports 1 chg 0000 evt 0000
[ 3516.611695] hub 7-0:1.0: hub_suspend
[ 3516.611704] usb usb7: bus auto-suspend, wakeup 1
[ 3516.611708] xhci_hcd 0000:0f:00.0: xhci_hub_status_data: stopping usb7 port polling
[ 3516.611712] xhci_hcd 0000:0f:00.0: config port 7-1 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
[ 3516.611714] xhci_hcd 0000:0f:00.0: xhci_suspend: stopping usb7 port polling.
[ 3516.611726] xhci_hcd 0000:0f:00.0: Setting command ring address to 0xffffe001
[ 3516.612681] xhci_hcd 0000:0f:00.0: hcd_pci_runtime_suspend: 0
[ 3516.622567] xhci_hcd 0000:0e:00.4: Setting command ring address to 0xffffe001
[ 3516.623346] xhci_hcd 0000:0e:00.4: xhci_resume: starting usb5 port polling.
[ 3516.623349] xhci_hcd 0000:0e:00.4: xhci_hub_status_data: stopping usb6 port polling
[ 3516.623351] xhci_hcd 0000:0e:00.4: xhci_hub_status_data: stopping usb5 port polling
[ 3516.623352] xhci_hcd 0000:0e:00.4: hcd_pci_runtime_resume: 0
[ 3516.623354] usb usb6: usb auto-resume
[ 3516.623378] hub 6-0:1.0: hub_resume
[ 3516.650531] xhci_hcd 0000:0e:00.4: Get port status 6-1 read: 0x2a0, return 0x2a0
[ 3516.650565] xhci_hcd 0000:0e:00.4: Get port status 6-2 read: 0x2a0, return 0x2a0
[ 3516.650614] usb usb5: usb auto-resume
[ 3516.650632] hub 5-0:1.0: hub_resume
[ 3516.650635] xhci_hcd 0000:0e:00.4: Get port status 5-1 read: 0x2a0, return 0x100
[ 3516.650642] hub 6-0:1.0: state 7 ports 2 chg 0000 evt 0000
[ 3516.650664] xhci_hcd 0000:0e:00.4: Get port status 5-2 read: 0x2a0, return 0x100
[ 3516.650689] hub 5-0:1.0: state 7 ports 2 chg 0000 evt 0000
[ 3516.650697] hub 5-0:1.0: hub_suspend
[ 3516.650715] usb usb5: bus auto-suspend, wakeup 1
[ 3516.650733] xhci_hcd 0000:0e:00.4: xhci_hub_status_data: stopping usb5 port polling
[ 3516.662574] xhci_hcd 0000:0e:00.3: Setting command ring address to 0xffffe001
[ 3516.663352] xhci_hcd 0000:0e:00.3: xhci_resume: starting usb3 port polling.
[ 3516.663355] xhci_hcd 0000:0e:00.3: xhci_hub_status_data: stopping usb4 port polling
[ 3516.663357] xhci_hcd 0000:0e:00.3: xhci_hub_status_data: stopping usb3 port polling
[ 3516.663358] xhci_hcd 0000:0e:00.3: hcd_pci_runtime_resume: 0
[ 3516.663360] usb usb4: usb auto-resume
[ 3516.663383] hub 4-0:1.0: hub_resume
[ 3516.686531] xhci_hcd 0000:0e:00.3: Get port status 4-1 read: 0x2a0, return 0x2a0
[ 3516.686560] xhci_hcd 0000:0e:00.3: Get port status 4-2 read: 0x2a0, return 0x2a0
[ 3516.686588] hub 4-0:1.0: state 7 ports 2 chg 0000 evt 0000
[ 3516.686616] usb usb3: usb auto-resume
[ 3516.686637] hub 3-0:1.0: hub_resume
[ 3516.686640] xhci_hcd 0000:0e:00.3: Get port status 3-1 read: 0x2a0, return 0x100
[ 3516.686655] xhci_hcd 0000:0e:00.3: Get port status 3-2 read: 0x2a0, return 0x100
[ 3516.686676] hub 3-0:1.0: state 7 ports 2 chg 0000 evt 0000
[ 3516.686684] hub 3-0:1.0: hub_suspend
[ 3516.686704] usb usb3: bus auto-suspend, wakeup 1
[ 3516.686722] xhci_hcd 0000:0e:00.3: xhci_hub_status_data: stopping usb3 port polling
[ 3516.686736] usb usb2: usb auto-resume
[ 3516.686744] hub 2-0:1.0: hub_resume
[ 3516.710545] xhci_hcd 0000:0c:00.0: Get port status 2-1 read: 0x2a0, return 0x2a0
[ 3516.710578] xhci_hcd 0000:0c:00.0: Get port status 2-2 read: 0x2a0, return 0x2a0
[ 3516.710599] xhci_hcd 0000:0c:00.0: Get port status 2-3 read: 0x2a0, return 0x2a0
[ 3516.710624] xhci_hcd 0000:0c:00.0: Get port status 2-4 read: 0x2a0, return 0x2a0
[ 3516.710645] xhci_hcd 0000:0c:00.0: Get port status 2-5 read: 0x2a0, return 0x2a0
[ 3516.710670] hub 2-0:1.0: state 7 ports 5 chg 0000 evt 0000
[ 3516.858535] xhci_hcd 0000:0e:00.4: set port remote wake mask, actual port 6-1 status  = 0xe0002a0
[ 3516.858551] xhci_hcd 0000:0e:00.4: set port remote wake mask, actual port 6-2 status  = 0xe0002a0
[ 3516.858559] hub 6-0:1.0: hub_suspend
[ 3516.858567] usb usb6: bus auto-suspend, wakeup 1
[ 3516.858594] xhci_hcd 0000:0e:00.4: xhci_hub_status_data: stopping usb6 port polling
[ 3516.858600] xhci_hcd 0000:0e:00.4: config port 6-1 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
[ 3516.858615] xhci_hcd 0000:0e:00.4: config port 6-2 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
[ 3516.858636] xhci_hcd 0000:0e:00.4: config port 5-1 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
[ 3516.858658] xhci_hcd 0000:0e:00.4: config port 5-2 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
[ 3516.858660] xhci_hcd 0000:0e:00.4: xhci_suspend: stopping usb5 port polling.
[ 3516.858677] xhci_hcd 0000:0e:00.4: Setting command ring address to 0xffffe001
[ 3516.859456] xhci_hcd 0000:0e:00.4: hcd_pci_runtime_suspend: 0
[ 3516.894548] xhci_hcd 0000:0e:00.3: set port remote wake mask, actual port 4-1 status  = 0xe0002a0
[ 3516.894567] xhci_hcd 0000:0e:00.3: set port remote wake mask, actual port 4-2 status  = 0xe0002a0
[ 3516.894575] hub 4-0:1.0: hub_suspend
[ 3516.894582] usb usb4: bus auto-suspend, wakeup 1
[ 3516.894610] xhci_hcd 0000:0e:00.3: xhci_hub_status_data: stopping usb4 port polling
[ 3516.894615] xhci_hcd 0000:0e:00.3: config port 4-1 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
[ 3516.894631] xhci_hcd 0000:0e:00.3: config port 4-2 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
[ 3516.894653] xhci_hcd 0000:0e:00.3: config port 3-1 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
[ 3516.894674] xhci_hcd 0000:0e:00.3: config port 3-2 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
[ 3516.894676] xhci_hcd 0000:0e:00.3: xhci_suspend: stopping usb3 port polling.
[ 3516.894693] xhci_hcd 0000:0e:00.3: Setting command ring address to 0xffffe001
[ 3516.895472] xhci_hcd 0000:0e:00.3: hcd_pci_runtime_suspend: 0
[ 3516.918537] xhci_hcd 0000:0c:00.0: set port remote wake mask, actual port 2-1 status  = 0xe0002a0
[ 3516.918551] xhci_hcd 0000:0c:00.0: set port remote wake mask, actual port 2-2 status  = 0xe0002a0
[ 3516.918561] xhci_hcd 0000:0c:00.0: set port remote wake mask, actual port 2-3 status  = 0xe0002a0
[ 3516.918569] xhci_hcd 0000:0c:00.0: set port remote wake mask, actual port 2-4 status  = 0xe0002a0
[ 3516.918578] xhci_hcd 0000:0c:00.0: set port remote wake mask, actual port 2-5 status  = 0xe0002a0
[ 3516.918584] hub 2-0:1.0: hub_suspend
[ 3516.918591] usb usb2: bus auto-suspend, wakeup 1
[ 3516.918599] xhci_hcd 0000:0c:00.0: xhci_hub_status_data: stopping usb2 port polling

Re: [REGRESSION][BISECTED] usb: hub: total system freeze after running adb

[Forest]

On Thu, 20 Nov 2025 14:04:45 -0800, Forest wrote:

>On Tue, 18 Nov 2025 11:21:18 +0200, Mathias Nyman wrote:
>
>>If the bisected patch is the issue then disabling runtime suspend could help.
>>For all usb* roothubs:
>>
>>echo on > /sys/bus/usb/devices/usb*/power/control
>>
>>Does it affect anything?
>
>Yes. After setting all of those to 'on', I have been unable to
>reproduce the freeze. Tested for nearly 12 hours so far, so I think
>it does avoid the problem. (I'll do some more testing and report
>back if that changes.)

I've now tested this workaround for about 36 hours with no freeze. 
Looks to me like the bisected commit is indeed the source of the bug.

Re: [REGRESSION][BISECTED] usb: hub: total system freeze after running adb

[Mathias Nyman]

Thanks for the info and testing, got hung up on some other tasks for a while

On 11/21/25 00:04, Forest wrote:
> On Tue, 18 Nov 2025 11:21:18 +0200, Mathias Nyman wrote:
> 
>> If the bisected patch is the issue then disabling runtime suspend could help.
>> For all usb* roothubs:
>>
>> echo on > /sys/bus/usb/devices/usb*/power/control
>>
>> Does it affect anything?
> 
> Yes. After setting all of those to 'on', I have been unable to
> reproduce the freeze. Tested for nearly 12 hours so far, so I think
> it does avoid the problem. (I'll do some more testing and report
> back if that changes.)
> 
>> Could you share some info about this setup:
> 
...

> 0f:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Raphael/Granite Ridge USB 2.0 xHCI [1022:15b8] (prog-if 30 [XHCI])
>          Subsystem: ASUSTeK Computer Inc. Device [1043:8877]
>          Flags: fast devsel, IRQ 24, IOMMU group 31
>          Memory at f6e00000 (64-bit, non-prefetchable) [size=1M]
>          Capabilities: [48] Vendor Specific Information: Len=08 <?>
>          Capabilities: [50] Power Management version 3
>          Capabilities: [64] Express Endpoint, IntMsgNum 0
>          Capabilities: [a0] MSI: Enable- Count=1/8 Maskable- 64bit+
>          Capabilities: [c0] MSI-X: Enable+ Count=8 Masked-
>          Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
>          Capabilities: [270] Secondary PCI Express
>          Capabilities: [2a0] Access Control Services
>          Capabilities: [410] Physical Layer 16.0 GT/s <?>
>          Capabilities: [450] Lane Margining at the Receiver
>          Kernel driver in use: xhci_hcd
>          Kernel modules: xhci_pci

This looks like a special xhci that only has a usb2 roothub.
Those did cause issues earlier with hub post resume work, but should be fixed now.

> 
>> Enabling usb core and xhci dynamic debug could share some light on this:
>>
>> mount -t debugfs none /sys/kernel/debug
>> echo 'module xhci_hcd =p' >/sys/kernel/debug/dynamic_debug/control
>> echo 'module usbcore =p' >/sys/kernel/debug/dynamic_debug/control
>> < Reproduce issue >
>> Send output of dmesg
> 
> With these enabled and adb running, dmesg produces a 90-line burst of
> output roughly each second. This generated megabytes of output before
> the freeze happened. Comparing the last three bursts, I found them to
> be identical except for the relative order of a few lines, and the
> timestamps.

Ok, so with this setup we end up running a runtime suspend/resume stress
test that wakes up and suspends the hosts about once a second.

> 
> I was redirecting `dmesg --follow` to a file while watching it in a
> terminal, in hopes that any buffered output that wasn't written
> to the file when the freeze happened would be visible on-screen.
> A photo of the screen showed the same messages as found at the tail
> of the file, but with newer timestamps.
> 
> Here is the last burst of logged messages:
> 
> [ 3516.610664] xhci_hcd 0000:0f:00.0: Setting command ring address to 0xffffe001
> [ 3516.611616] xhci_hcd 0000:0f:00.0: xhci_resume: starting usb7 port polling.
> [ 3516.611619] xhci_hcd 0000:0f:00.0: xhci_hub_status_data: stopping usb7 port polling
> [ 3516.611621] xhci_hcd 0000:0f:00.0: hcd_pci_runtime_resume: 0
> [ 3516.611624] usb usb7: usb auto-resume
> [ 3516.611628] hub 7-0:1.0: hub_resume
> [ 3516.611632] xhci_hcd 0000:0f:00.0: Get port status 7-1 read: 0x2a0, return 0x100
> [ 3516.611689] hub 7-0:1.0: state 7 ports 1 chg 0000 evt 0000
> [ 3516.611695] hub 7-0:1.0: hub_suspend
> [ 3516.611704] usb usb7: bus auto-suspend, wakeup 1
> [ 3516.611708] xhci_hcd 0000:0f:00.0: xhci_hub_status_data: stopping usb7 port polling
> [ 3516.611712] xhci_hcd 0000:0f:00.0: config port 7-1 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
> [ 3516.611714] xhci_hcd 0000:0f:00.0: xhci_suspend: stopping usb7 port polling.
> [ 3516.611726] xhci_hcd 0000:0f:00.0: Setting command ring address to 0xffffe001
> [ 3516.612681] xhci_hcd 0000:0f:00.0: hcd_pci_runtime_suspend: 0
> [ 3516.622567] xhci_hcd 0000:0e:00.4: Setting command ring address to 0xffffe001
> [ 3516.623346] xhci_hcd 0000:0e:00.4: xhci_resume: starting usb5 port polling.
> [ 3516.623349] xhci_hcd 0000:0e:00.4: xhci_hub_status_data: stopping usb6 port polling
> [ 3516.623351] xhci_hcd 0000:0e:00.4: xhci_hub_status_data: stopping usb5 port polling
> [ 3516.623352] xhci_hcd 0000:0e:00.4: hcd_pci_runtime_resume: 0
> [ 3516.623354] usb usb6: usb auto-resume
> [ 3516.623378] hub 6-0:1.0: hub_resume
> [ 3516.650531] xhci_hcd 0000:0e:00.4: Get port status 6-1 read: 0x2a0, return 0x2a0
> [ 3516.650565] xhci_hcd 0000:0e:00.4: Get port status 6-2 read: 0x2a0, return 0x2a0
> [ 3516.650614] usb usb5: usb auto-resume
> [ 3516.650632] hub 5-0:1.0: hub_resume
> [ 3516.650635] xhci_hcd 0000:0e:00.4: Get port status 5-1 read: 0x2a0, return 0x100
> [ 3516.650642] hub 6-0:1.0: state 7 ports 2 chg 0000 evt 0000
> [ 3516.650664] xhci_hcd 0000:0e:00.4: Get port status 5-2 read: 0x2a0, return 0x100
> [ 3516.650689] hub 5-0:1.0: state 7 ports 2 chg 0000 evt 0000
> [ 3516.650697] hub 5-0:1.0: hub_suspend
> [ 3516.650715] usb usb5: bus auto-suspend, wakeup 1
> [ 3516.650733] xhci_hcd 0000:0e:00.4: xhci_hub_status_data: stopping usb5 port polling
> [ 3516.662574] xhci_hcd 0000:0e:00.3: Setting command ring address to 0xffffe001
> [ 3516.663352] xhci_hcd 0000:0e:00.3: xhci_resume: starting usb3 port polling.
> [ 3516.663355] xhci_hcd 0000:0e:00.3: xhci_hub_status_data: stopping usb4 port polling
> [ 3516.663357] xhci_hcd 0000:0e:00.3: xhci_hub_status_data: stopping usb3 port polling
> [ 3516.663358] xhci_hcd 0000:0e:00.3: hcd_pci_runtime_resume: 0
> [ 3516.663360] usb usb4: usb auto-resume
> [ 3516.663383] hub 4-0:1.0: hub_resume
> [ 3516.686531] xhci_hcd 0000:0e:00.3: Get port status 4-1 read: 0x2a0, return 0x2a0
> [ 3516.686560] xhci_hcd 0000:0e:00.3: Get port status 4-2 read: 0x2a0, return 0x2a0
> [ 3516.686588] hub 4-0:1.0: state 7 ports 2 chg 0000 evt 0000
> [ 3516.686616] usb usb3: usb auto-resume
> [ 3516.686637] hub 3-0:1.0: hub_resume
> [ 3516.686640] xhci_hcd 0000:0e:00.3: Get port status 3-1 read: 0x2a0, return 0x100
> [ 3516.686655] xhci_hcd 0000:0e:00.3: Get port status 3-2 read: 0x2a0, return 0x100
> [ 3516.686676] hub 3-0:1.0: state 7 ports 2 chg 0000 evt 0000
> [ 3516.686684] hub 3-0:1.0: hub_suspend
> [ 3516.686704] usb usb3: bus auto-suspend, wakeup 1
> [ 3516.686722] xhci_hcd 0000:0e:00.3: xhci_hub_status_data: stopping usb3 port polling
> [ 3516.686736] usb usb2: usb auto-resume
> [ 3516.686744] hub 2-0:1.0: hub_resume
> [ 3516.710545] xhci_hcd 0000:0c:00.0: Get port status 2-1 read: 0x2a0, return 0x2a0
> [ 3516.710578] xhci_hcd 0000:0c:00.0: Get port status 2-2 read: 0x2a0, return 0x2a0
> [ 3516.710599] xhci_hcd 0000:0c:00.0: Get port status 2-3 read: 0x2a0, return 0x2a0
> [ 3516.710624] xhci_hcd 0000:0c:00.0: Get port status 2-4 read: 0x2a0, return 0x2a0
> [ 3516.710645] xhci_hcd 0000:0c:00.0: Get port status 2-5 read: 0x2a0, return 0x2a0
> [ 3516.710670] hub 2-0:1.0: state 7 ports 5 chg 0000 evt 0000
> [ 3516.858535] xhci_hcd 0000:0e:00.4: set port remote wake mask, actual port 6-1 status  = 0xe0002a0
> [ 3516.858551] xhci_hcd 0000:0e:00.4: set port remote wake mask, actual port 6-2 status  = 0xe0002a0
> [ 3516.858559] hub 6-0:1.0: hub_suspend
> [ 3516.858567] usb usb6: bus auto-suspend, wakeup 1
> [ 3516.858594] xhci_hcd 0000:0e:00.4: xhci_hub_status_data: stopping usb6 port polling
> [ 3516.858600] xhci_hcd 0000:0e:00.4: config port 6-1 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
> [ 3516.858615] xhci_hcd 0000:0e:00.4: config port 6-2 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
> [ 3516.858636] xhci_hcd 0000:0e:00.4: config port 5-1 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
> [ 3516.858658] xhci_hcd 0000:0e:00.4: config port 5-2 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
> [ 3516.858660] xhci_hcd 0000:0e:00.4: xhci_suspend: stopping usb5 port polling.
> [ 3516.858677] xhci_hcd 0000:0e:00.4: Setting command ring address to 0xffffe001
> [ 3516.859456] xhci_hcd 0000:0e:00.4: hcd_pci_runtime_suspend: 0
> [ 3516.894548] xhci_hcd 0000:0e:00.3: set port remote wake mask, actual port 4-1 status  = 0xe0002a0
> [ 3516.894567] xhci_hcd 0000:0e:00.3: set port remote wake mask, actual port 4-2 status  = 0xe0002a0
> [ 3516.894575] hub 4-0:1.0: hub_suspend
> [ 3516.894582] usb usb4: bus auto-suspend, wakeup 1
> [ 3516.894610] xhci_hcd 0000:0e:00.3: xhci_hub_status_data: stopping usb4 port polling
> [ 3516.894615] xhci_hcd 0000:0e:00.3: config port 4-1 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
> [ 3516.894631] xhci_hcd 0000:0e:00.3: config port 4-2 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
> [ 3516.894653] xhci_hcd 0000:0e:00.3: config port 3-1 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
> [ 3516.894674] xhci_hcd 0000:0e:00.3: config port 3-2 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
> [ 3516.894676] xhci_hcd 0000:0e:00.3: xhci_suspend: stopping usb3 port polling.
> [ 3516.894693] xhci_hcd 0000:0e:00.3: Setting command ring address to 0xffffe001
> [ 3516.895472] xhci_hcd 0000:0e:00.3: hcd_pci_runtime_suspend: 0
> [ 3516.918537] xhci_hcd 0000:0c:00.0: set port remote wake mask, actual port 2-1 status  = 0xe0002a0
> [ 3516.918551] xhci_hcd 0000:0c:00.0: set port remote wake mask, actual port 2-2 status  = 0xe0002a0
> [ 3516.918561] xhci_hcd 0000:0c:00.0: set port remote wake mask, actual port 2-3 status  = 0xe0002a0
> [ 3516.918569] xhci_hcd 0000:0c:00.0: set port remote wake mask, actual port 2-4 status  = 0xe0002a0
> [ 3516.918578] xhci_hcd 0000:0c:00.0: set port remote wake mask, actual port 2-5 status  = 0xe0002a0
> [ 3516.918584] hub 2-0:1.0: hub_suspend
> [ 3516.918591] usb usb2: bus auto-suspend, wakeup 1
> [ 3516.918599] xhci_hcd 0000:0c:00.0: xhci_hub_status_data: stopping usb2 port polling
> 

I can't see anything suspicious in here.
xHCI hosts and their roothubs resume, USB2 roothubs suspend back almost immediately, USB 3 hubs after
200ms of post resume delay. host controller suspend right after that.

Best guess is that something is still messed up with flushing the work.

If possible, could you try with the same usb debugging enabled as last time,
and add HUNG_TASK debugging entries in kernel config:

CONFIG_DETECT_HUNG_TASK=y
CONFIG_DEFAULT_HUNG_TASK_TIMEOUT=30
CONFIG_DETECT_HUNG_TASK_BLOCKER=y

And add the following patch to hub driver:

diff --git a/drivers/usb/core/hub.c b/drivers/usb/core/hub.c
index 256fe8c86828..6863a2feb1d5 100644
--- a/drivers/usb/core/hub.c
+++ b/drivers/usb/core/hub.c
@@ -1383,6 +1383,7 @@ static void hub_post_resume(struct work_struct *ws)
  {
  	struct usb_hub *hub = container_of(ws, struct usb_hub, post_resume_work.work);
  
+	dev_err(hub->intfdev, "HUBHANG hub_post_resume()\n");
  	usb_autopm_put_interface_async(to_usb_interface(hub->intfdev));
  	hub_put(hub);
  }
@@ -1395,6 +1396,7 @@ static void hub_quiesce(struct usb_hub *hub, enum hub_quiescing_type type)
  {
  	struct usb_device *hdev = hub->hdev;
  	unsigned long flags;
+	bool flushed;
  	int i;
  
  	/* hub_wq and related activity won't re-trigger */
@@ -1412,7 +1414,11 @@ static void hub_quiesce(struct usb_hub *hub, enum hub_quiescing_type type)
  
  	/* Stop hub_wq and related activity */
  	timer_delete_sync(&hub->irq_urb_retry);
-	flush_delayed_work(&hub->post_resume_work);
+	dev_err(hub->intfdev, "HUBHANG %pS -> hub_quiesce() flush post resume work\n",
+		  __builtin_return_address(0));
+	flushed = flush_delayed_work(&hub->post_resume_work);
+	dev_err(hub->intfdev, "HUBHANG post resume work flush%s\n",
+		  flushed ? "ed" : " not needed");
  	usb_kill_urb(hub->urb);
  	if (hub->has_indicators)
  		cancel_delayed_work_sync(&hub->leds);
@@ -1781,6 +1787,7 @@ static void hub_release(struct kref *kref)
  {
  	struct usb_hub *hub = container_of(kref, struct usb_hub, kref);
  
+	dev_err(hub->intfdev, "hub_release()\n");
  	usb_put_dev(hub->hdev);
  	usb_put_intf(to_usb_interface(hub->intfdev));
  	kfree(hub);
-- 
2.43.0

Thanks
Mathias

Re: [REGRESSION][BISECTED] usb: hub: total system freeze after running adb

[Forest]

On Thu, 4 Dec 2025 01:41:37 +0200, Mathias Nyman wrote:

>If possible, could you try with the same usb debugging enabled as last time,
>and add HUNG_TASK debugging entries in kernel config:
>
>CONFIG_DETECT_HUNG_TASK=y
>CONFIG_DEFAULT_HUNG_TASK_TIMEOUT=30
>CONFIG_DETECT_HUNG_TASK_BLOCKER=y
>
>And add the following patch to hub driver:

That patch failed to apply, due to tab/space differences. I guess Thunderbird
might have taken liberties when you sent it. It was easy enough to make those
edits by hand, though.

The result was again bursts of dmesg output roughly once per second, this time
107 lines each. I compared the last three bursts that made it to my redirected
output file, and they were again the same except for timestamps and a few lines
being reordered.

A screen photo once again shows newer timestamps than the file. I did not
read every line in the photo to compare it with the log file, but the last
dozen lines look the same. I suppose I can send you the photo if you like.

Here is the last dmesg burst that was captured in the file:


[ 2875.178933] xhci_hcd 0000:0f:00.0: Setting command ring address to 0xffffe001
[ 2875.179881] xhci_hcd 0000:0f:00.0: xhci_resume: starting usb7 port polling.
[ 2875.179884] xhci_hcd 0000:0f:00.0: xhci_hub_status_data: stopping usb7 port polling
[ 2875.179886] xhci_hcd 0000:0f:00.0: hcd_pci_runtime_resume: 0
[ 2875.179890] usb usb7: usb auto-resume
[ 2875.179895] hub 7-0:1.0: hub_resume
[ 2875.179899] xhci_hcd 0000:0f:00.0: Get port status 7-1 read: 0x2a0, return 0x100
[ 2875.179974] hub 7-0:1.0: state 7 ports 1 chg 0000 evt 0000
[ 2875.179979] hub 7-0:1.0: hub_suspend
[ 2875.179981] hub 7-0:1.0: HUBHANG hub_suspend+0xf1/0x250 [usbcore] -> hub_quiesce() flush post resume work
[ 2875.179996] hub 7-0:1.0: HUBHANG post resume work flush not needed
[ 2875.180005] usb usb7: bus auto-suspend, wakeup 1
[ 2875.180009] xhci_hcd 0000:0f:00.0: xhci_hub_status_data: stopping usb7 port polling
[ 2875.180013] xhci_hcd 0000:0f:00.0: config port 7-1 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
[ 2875.180015] xhci_hcd 0000:0f:00.0: xhci_suspend: stopping usb7 port polling.
[ 2875.180029] xhci_hcd 0000:0f:00.0: Setting command ring address to 0xffffe001
[ 2875.180984] xhci_hcd 0000:0f:00.0: hcd_pci_runtime_suspend: 0
[ 2875.192035] xhci_hcd 0000:0e:00.4: Setting command ring address to 0xffffe001
[ 2875.192818] xhci_hcd 0000:0e:00.4: xhci_resume: starting usb5 port polling.
[ 2875.192821] xhci_hcd 0000:0e:00.4: xhci_hub_status_data: stopping usb6 port polling
[ 2875.192823] xhci_hcd 0000:0e:00.4: xhci_hub_status_data: stopping usb5 port polling
[ 2875.192824] xhci_hcd 0000:0e:00.4: hcd_pci_runtime_resume: 0
[ 2875.192826] usb usb6: usb auto-resume
[ 2875.192849] hub 6-0:1.0: hub_resume
[ 2875.214526] xhci_hcd 0000:0e:00.4: Get port status 6-1 read: 0x2a0, return 0x2a0
[ 2875.214545] xhci_hcd 0000:0e:00.4: Get port status 6-2 read: 0x2a0, return 0x2a0
[ 2875.214562] hub 6-0:1.0: state 7 ports 2 chg 0000 evt 0000
[ 2875.214584] usb usb5: usb auto-resume
[ 2875.214598] hub 5-0:1.0: hub_resume
[ 2875.214600] xhci_hcd 0000:0e:00.4: Get port status 5-1 read: 0x2a0, return 0x100
[ 2875.214610] xhci_hcd 0000:0e:00.4: Get port status 5-2 read: 0x2a0, return 0x100
[ 2875.214622] hub 5-0:1.0: state 7 ports 2 chg 0000 evt 0000
[ 2875.214629] hub 5-0:1.0: hub_suspend
[ 2875.214630] hub 5-0:1.0: HUBHANG hub_suspend+0xf1/0x250 [usbcore] -> hub_quiesce() flush post resume work
[ 2875.214641] hub 5-0:1.0: HUBHANG post resume work flush not needed
[ 2875.214649] usb usb5: bus auto-suspend, wakeup 1
[ 2875.214673] xhci_hcd 0000:0e:00.4: xhci_hub_status_data: stopping usb5 port polling
[ 2875.226565] xhci_hcd 0000:0e:00.3: Setting command ring address to 0xffffe001
[ 2875.227340] xhci_hcd 0000:0e:00.3: xhci_resume: starting usb3 port polling.
[ 2875.227343] xhci_hcd 0000:0e:00.3: xhci_hub_status_data: stopping usb4 port polling
[ 2875.227346] xhci_hcd 0000:0e:00.3: xhci_hub_status_data: stopping usb3 port polling
[ 2875.227347] xhci_hcd 0000:0e:00.3: hcd_pci_runtime_resume: 0
[ 2875.227350] usb usb4: usb auto-resume
[ 2875.227372] hub 4-0:1.0: hub_resume
[ 2875.250541] xhci_hcd 0000:0e:00.3: Get port status 4-1 read: 0x2a0, return 0x2a0
[ 2875.250574] xhci_hcd 0000:0e:00.3: Get port status 4-2 read: 0x2a0, return 0x2a0
[ 2875.250596] hub 4-0:1.0: state 7 ports 2 chg 0000 evt 0000
[ 2875.250629] usb usb3: usb auto-resume
[ 2875.250646] hub 3-0:1.0: hub_resume
[ 2875.250649] xhci_hcd 0000:0e:00.3: Get port status 3-1 read: 0x2a0, return 0x100
[ 2875.250662] xhci_hcd 0000:0e:00.3: Get port status 3-2 read: 0x2a0, return 0x100
[ 2875.250677] hub 3-0:1.0: state 7 ports 2 chg 0000 evt 0000
[ 2875.250685] hub 3-0:1.0: hub_suspend
[ 2875.250687] hub 3-0:1.0: HUBHANG hub_suspend+0xf1/0x250 [usbcore] -> hub_quiesce() flush post resume work
[ 2875.250701] hub 3-0:1.0: HUBHANG post resume work flush not needed
[ 2875.250712] usb usb3: bus auto-suspend, wakeup 1
[ 2875.250732] xhci_hcd 0000:0e:00.3: xhci_hub_status_data: stopping usb3 port polling
[ 2875.250747] usb usb2: usb auto-resume
[ 2875.250754] hub 2-0:1.0: hub_resume
[ 2875.278530] xhci_hcd 0000:0c:00.0: Get port status 2-1 read: 0x2a0, return 0x2a0
[ 2875.278553] xhci_hcd 0000:0c:00.0: Get port status 2-2 read: 0x2a0, return 0x2a0
[ 2875.278568] xhci_hcd 0000:0c:00.0: Get port status 2-3 read: 0x2a0, return 0x2a0
[ 2875.278581] xhci_hcd 0000:0c:00.0: Get port status 2-4 read: 0x2a0, return 0x2a0
[ 2875.278593] xhci_hcd 0000:0c:00.0: Get port status 2-5 read: 0x2a0, return 0x2a0
[ 2875.278610] hub 2-0:1.0: state 7 ports 5 chg 0000 evt 0000
[ 2875.418535] hub 6-0:1.0: HUBHANG hub_post_resume()
[ 2875.418551] xhci_hcd 0000:0e:00.4: set port remote wake mask, actual port 6-1 status  = 0xe0002a0
[ 2875.418571] xhci_hcd 0000:0e:00.4: set port remote wake mask, actual port 6-2 status  = 0xe0002a0
[ 2875.418578] hub 6-0:1.0: hub_suspend
[ 2875.418580] hub 6-0:1.0: HUBHANG hub_suspend+0xf1/0x250 [usbcore] -> hub_quiesce() flush post resume work
[ 2875.418597] hub 6-0:1.0: HUBHANG post resume work flush not needed
[ 2875.418606] usb usb6: bus auto-suspend, wakeup 1
[ 2875.418630] xhci_hcd 0000:0e:00.4: xhci_hub_status_data: stopping usb6 port polling
[ 2875.418635] xhci_hcd 0000:0e:00.4: config port 6-1 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
[ 2875.418651] xhci_hcd 0000:0e:00.4: config port 6-2 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
[ 2875.418672] xhci_hcd 0000:0e:00.4: config port 5-1 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
[ 2875.418693] xhci_hcd 0000:0e:00.4: config port 5-2 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
[ 2875.418695] xhci_hcd 0000:0e:00.4: xhci_suspend: stopping usb5 port polling.
[ 2875.418713] xhci_hcd 0000:0e:00.4: Setting command ring address to 0xffffe001
[ 2875.419493] xhci_hcd 0000:0e:00.4: hcd_pci_runtime_suspend: 0
[ 2875.458525] hub 4-0:1.0: HUBHANG hub_post_resume()
[ 2875.458537] xhci_hcd 0000:0e:00.3: set port remote wake mask, actual port 4-1 status  = 0xe0002a0
[ 2875.458555] xhci_hcd 0000:0e:00.3: set port remote wake mask, actual port 4-2 status  = 0xe0002a0
[ 2875.458560] hub 4-0:1.0: hub_suspend
[ 2875.458562] hub 4-0:1.0: HUBHANG hub_suspend+0xf1/0x250 [usbcore] -> hub_quiesce() flush post resume work
[ 2875.458575] hub 4-0:1.0: HUBHANG post resume work flush not needed
[ 2875.458583] usb usb4: bus auto-suspend, wakeup 1
[ 2875.458603] xhci_hcd 0000:0e:00.3: xhci_hub_status_data: stopping usb4 port polling
[ 2875.458608] xhci_hcd 0000:0e:00.3: config port 4-1 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
[ 2875.458624] xhci_hcd 0000:0e:00.3: config port 4-2 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
[ 2875.458646] xhci_hcd 0000:0e:00.3: config port 3-1 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
[ 2875.458667] xhci_hcd 0000:0e:00.3: config port 3-2 wake bits, portsc: 0xa0002a0, write: 0xa0202a0
[ 2875.458668] xhci_hcd 0000:0e:00.3: xhci_suspend: stopping usb3 port polling.
[ 2875.458686] xhci_hcd 0000:0e:00.3: Setting command ring address to 0xffffe001
[ 2875.459466] xhci_hcd 0000:0e:00.3: hcd_pci_runtime_suspend: 0
[ 2875.486515] hub 2-0:1.0: HUBHANG hub_post_resume()
[ 2875.486526] xhci_hcd 0000:0c:00.0: set port remote wake mask, actual port 2-1 status  = 0xe0002a0
[ 2875.486539] xhci_hcd 0000:0c:00.0: set port remote wake mask, actual port 2-2 status  = 0xe0002a0
[ 2875.486548] xhci_hcd 0000:0c:00.0: set port remote wake mask, actual port 2-3 status  = 0xe0002a0
[ 2875.486557] xhci_hcd 0000:0c:00.0: set port remote wake mask, actual port 2-4 status  = 0xe0002a0
[ 2875.486566] xhci_hcd 0000:0c:00.0: set port remote wake mask, actual port 2-5 status  = 0xe0002a0
[ 2875.486572] hub 2-0:1.0: hub_suspend
[ 2875.486574] hub 2-0:1.0: HUBHANG hub_suspend+0xf1/0x250 [usbcore] -> hub_quiesce() flush post resume work
[ 2875.486586] hub 2-0:1.0: HUBHANG post resume work flush not needed
[ 2875.486593] usb usb2: bus auto-suspend, wakeup 1
[ 2875.486601] xhci_hcd 0000:0c:00.0: xhci_hub_status_data: stopping usb2 port polling

Re: [REGRESSION][BISECTED] usb: hub: total system freeze after running adb

[Mathias Nyman]

On 12/5/25 03:56, Forest wrote:
> On Thu, 4 Dec 2025 01:41:37 +0200, Mathias Nyman wrote:
> 
>> If possible, could you try with the same usb debugging enabled as last time,
>> and add HUNG_TASK debugging entries in kernel config:
>>
>> CONFIG_DETECT_HUNG_TASK=y
>> CONFIG_DEFAULT_HUNG_TASK_TIMEOUT=30
>> CONFIG_DETECT_HUNG_TASK_BLOCKER=y
>>
>> And add the following patch to hub driver:
> 
> That patch failed to apply, due to tab/space differences. I guess Thunderbird
> might have taken liberties when you sent it. It was easy enough to make those
> edits by hand, though.
> 
> The result was again bursts of dmesg output roughly once per second, this time
> 107 lines each. I compared the last three bursts that made it to my redirected
> output file, and they were again the same except for timestamps and a few lines
> being reordered.
> 
> A screen photo once again shows newer timestamps than the file. I did not
> read every line in the photo to compare it with the log file, but the last
> dozen lines look the same. I suppose I can send you the photo if you like.
> 

Yes please, small differences, ordering, and timing matter in this case

Thanks
Mathias

Re: [REGRESSION][BISECTED] usb: hub: total system freeze after running adb

[Forest]

For the record, I (the reporter) have a new email address.

It was: forestix@gaga.casa
It is now: forestix@kcat.cc


On Fri, 5 Dec 2025 16:57:45 +0200, Mathias Nyman wrote:

>> A screen photo once again shows newer timestamps than the file. I did not
>> read every line in the photo to compare it with the log file, but the last
>> dozen lines look the same. I suppose I can send you the photo if you like.
>> 
>
>Yes please, small differences, ordering, and timing matter in this case

Mathias, I sent the screen shot directly to your intel email address
back in December. (I sent it only to you, since I'm not sure binaries
would be welcome on the list.)

Did you get a chance to look it over?

Forest

Re: [REGRESSION][BISECTED] usb: hub: total system freeze after running adb

[Michal Pecio]

On Mon, 17 Nov 2025 12:18:54 -0800, Forest wrote:
> #regzbot introduced 8f5b7e2bec1c
> 
> Running adb (android debug bridge) commands causes the entire system to
> freeze some time later.
> 
> The freeze usually occurs within two hours of running adb, but not always.
> Stopping adb's persistent background process with `adb kill-server` seems to
> avoid the freeze. (However, this is not always possible, since the system
> sometimes freezes just a few seconds after adb is launched.)
> 
> It occurs even when no android devices are connected to the system.

Hi,

If this is caused by running some demon without much hardware connected
(besides hubs, I guess), I wonder if it might be useful to run strace on
that thing to see what it is even doing to trigger this crash.

> - No messages are written to dmesg or syslog.
> - Virtual console switching no longer works.
> - SysRq key commands no longer work, as far as I can tell.
> - Whatever was on the display remains intact.
> - Any already-playing audio enters a short loop, as though the
>   motherboard's sound device is playing from a ring buffer that
>   no longer gets updated.

How can you know nothing is logged if the machine becomes unresponsive?
For such occasions I keep a PCIe card with a serial port to get logs.
Maybe xHCI debug capability could work too (?), but I've never tried.

Regards,
Michal

Re: [REGRESSION][BISECTED] usb: hub: total system freeze after running adb

[Forest]

On Tue, 18 Nov 2025 10:59:29 +0100, Michal Pecio wrote:

>If this is caused by running some demon without much hardware connected
>(besides hubs, I guess), I wonder if it might be useful to run strace on
>that thing to see what it is even doing to trigger this crash.

# strace -p 2862
strace: Process 2862 attached
gettid()                                = 2862
epoll_wait(6,

>How can you know nothing is logged if the machine becomes unresponsive?

By watching the output of `stdbuf -o0 dmesg --follow` when it happens.

Same results in a GUI terminal, a text console, and output redirected to
a `mount -o sync` non-journaled filesystem. I guess it's possible that
something *tried* to log a message, but if so, it didn't get logged. 

Re: [REGRESSION][BISECTED] usb: hub: total system freeze after running adb

[Michal Pecio]

On Thu, 20 Nov 2025 14:18:00 -0800, Forest wrote:
> On Tue, 18 Nov 2025 10:59:29 +0100, Michal Pecio wrote:
> >How can you know nothing is logged if the machine becomes
> >unresponsive?  
> 
> By watching the output of `stdbuf -o0 dmesg --follow` when it happens.
> 
> Same results in a GUI terminal, a text console, and output redirected
> to a `mount -o sync` non-journaled filesystem. I guess it's possible
> that something *tried* to log a message, but if so, it didn't get
> logged. 

This involves userspace, which stops working in cases like panic or
full lockup of all CPUs. So you may be missing some final messages.

If you have a PS/2 port, connect some keyboard and see if LEDs will
start blinking - this means kernel panic.

More reliable solution for getting logs from a broken system is
serial console. Maybe netconsole would work too, I have never tried.
And it seems it may also be possible to use xHCI debug capability
(but it needs a special cable: A-to-A without Vbus).

You could try CONFIG_DRM_PANIC, but it doesn't work on all GPUs yet
and it will only help if it's a panic.

Regards,
Michal