Re: [PATCH] usb: xhci: bound wait command completion to avoid kdump deadlock

[Michal Pecio <michal.pecio@gmail.com>]

On Wed, 29 Apr 2026 22:48:17 -0300, Desnes Nunes wrote:
> The following deadlock in the usb subsystem can be triggered during kdump:
> 
> systemd-udevd[402]: usb3: Worker [419] processing SEQNUM=2194 is taking a long time
> dracut-initqueue[432]: Timed out while waiting for udev queue to empty.
> systemd-udevd[402]: usb3: Worker [419] processing SEQNUM=2194 killed
> systemd-udevd[402]: usb3: Worker [419] terminated by signal 9 (KILL).
> ...
> kdump[720]: saving vmcore complete
> ...
> systemd-shutdown[1]: Rebooting.
> INFO: task kworker/0:6:76 blocked for more than 122 seconds.

That's suspiciously long indeed.

>       Not tainted 6.12.0-223.2443_2475543665.el10.x86_64 #1

Pretty old kernel, and distribution to boot.
Have you tried 7.x, does the bug still exist?

> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> task:kworker/0:6     state:D stack:0     pid:76    tgid:76    ppid:2      task_flags:0x4208060 flags:0x00004000
> Workqueue: usb_hub_wq hub_event
> Call Trace:
>  <TASK>
>  __schedule+0x2a5/0x630
>  schedule+0x27/0x80
>  schedule_timeout+0xbf/0x100
>  __wait_for_common+0x95/0x1b0
>  ? __pfx_schedule_timeout+0x10/0x10
>  xhci_alloc_dev+0x9e/0x290
>  usb_alloc_dev+0x77/0x3a0
>  hub_port_connect+0x293/0x9a0
>  hub_port_connect_change+0x94/0x260
>  port_event+0x4d1/0x7f0
>  hub_event+0x16f/0x480
>  process_one_work+0x174/0x330
>  worker_thread+0x256/0x3a0
>  ? __pfx_worker_thread+0x10/0x10
>  kthread+0xfa/0x240
>  ? __pfx_kthread+0x10/0x10
>  ret_from_fork+0x31/0x50
>  ? __pfx_kthread+0x10/0x10
>  ret_from_fork_asm+0x1a/0x30
>  </TASK>
> INFO: task systemd-shutdow:1 blocked for more than 122 seconds.
>       Not tainted 6.12.0-223.2443_2475543665.el10.x86_64 #1
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> task:systemd-shutdow state:D stack:0     pid:1     tgid:1     ppid:0      task_flags:0x400100 flags:0x00000002
> Call Trace:
>  <TASK>
>  __schedule+0x2a5/0x630
>  schedule+0x27/0x80
>  schedule_preempt_disabled+0x15/0x30
>  __mutex_lock.constprop.0+0x497/0x860
>  device_shutdown+0xac/0x190
>  kernel_restart+0x3a/0x70
>  __do_sys_reboot+0x146/0x240
>  do_syscall_64+0x7d/0x160
>  ? devkmsg_write.cold+0x24/0x4a
>  ? update_load_avg+0x7f/0x730
>  ? __dequeue_entity+0x3ec/0x4a0
>  ? update_load_avg+0x7f/0x730
>  ? pick_next_task_fair+0x1e6/0x330
>  ? finish_task_switch.isra.0+0x97/0x2a0
>  ? rseq_get_rseq_cs+0x1d/0x220
>  ? rseq_ip_fixup+0x8d/0x1d0
>  ? arch_exit_to_user_mode_prepare.isra.0+0xa5/0xd0
>  ? syscall_exit_to_user_mode+0x32/0x190
>  ? do_syscall_64+0x89/0x160
>  ? handle_mm_fault+0x110/0x370
>  ? do_user_addr_fault+0x606/0x830
>  ? exc_page_fault+0x7f/0x150
>  entry_SYSCALL_64_after_hwframe+0x76/0x7e
> RIP: 0033:0x7f32517d9917
> RSP: 002b:00007ffc018d4fb8 EFLAGS: 00000206 ORIG_RAX: 00000000000000a9
> RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f32517d9917
> RDX: 0000000001234567 RSI: 0000000028121969 RDI: 00000000fee1dead
> RBP: 00007ffc018d5130 R08: 0000000000000069 R09: 00000000ffffffff
> R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000000
> R13: 0000000000000000 R14: 00007ffc018d5258 R15: 0000000000000000
>  </TASK>
> 
> During crashkernel's boot, hub_event() takes usb_lock_device(hdev) of the
> root hub and keeps it for the whole hub processing loop, since it calls
> hub_port_connect() -> usb_alloc_dev() -> xhci_alloc_dev(). If during kdump
> another device (e.g., a mis-initialized dGPU) hogs interrupts or DMAs, the
> TRB_ENABLE_SLOT command will be blocked from completion in time, moving
> the HC to an unstable condition (e.g., HSE in USBSTS).

What specifically have you seen?

If you have actually observed HSE (how?), maybe xhci-hcd could detect
it automatically by the same means and clean up immediately.

> After vmcore gets captured, init calls device_shutdown() trying to
> shut down the hub device, by also trying to take the same lock still
> held by the hub kworker task.
> 
> Avoid the deadlock by adding a 2x timeout for command completion

nit: not a deadlock if X waits for Y and Y is just stuck by itself.

> before calling xhci_hc_died(). This gives enough time before marking
> the host un- stable, dying and calling xhci_cleanup_command_queue();
> which unblocks the hub worker into releasing the lock, allowing
> device_shutdown() to proceed.

Many functions which wait for command completion without timeouts.
Patch this one and you will get stuck in the next.

This shouldn't be happening in the first place. If a command doesn't
complete normally in time then xhci_handle_command_timeout() should
abort it, and if that times out too, then hc_died().

So not sure why this hasn't happened here.
Is it reproducible? Can you try again with debug logs?

echo 'module xhci_hcd +p' >/proc/dynamic_debug/control

Regards,
Michal