Thunderbolt: reboot hangs in nhi_pci_remove() with CalDigit TS5 Plus on AMD Strix Halo

Thunderbolt: reboot hangs in nhi_pci_remove() with CalDigit TS5 Plus on AMD Strix Halo

[brandon.proulx]

[-- Attachment #1.1: Type: text/plain, Size: 1858 bytes --]

Hi,
On a Framework Desktop (AMD Ryzen AI Max 300 / Strix Halo) with a CalDigit
TS5 Plus attached, reboot never completes. After systemd finishes, PID 1
sits in D state in nhi_pci_remove() during device_shutdown() and the
machine never resets. The hardware watchdog does not fire either. Only the
power button recovers it.

Hardware:
  System:  Framework Desktop (AMD Ryzen AI Max 300 Series), BIOS 03.06 (2026-07-13)
  Host routers:
    c5:00.5 AMD Strix Halo USB4 Host Router [1022:158d], subsys [f111:000a]
    c5:00.6 AMD Strix Halo USB4 Host Router [1022:158e], subsys [f111:000a]
  Device:  CalDigit TS5 Plus (thunderbolt 1-2, domain1 on c5:00.6), gen 4, NVM 64.c1

Kernel: 7.2.5 (distribution build 7.2.5-3-omarchy, Arch-based)
thunderbolt.host_reset=Y (default)

Stack of PID 1, sampled every few seconds from reboot until power-off.
Unchanged from 21:02:02 to 21:07:27:

  [<0>] nhi_pci_remove+0x53/0x60 [thunderbolt]
  [<0>] pci_device_shutdown+0x3e/0x80
  [<0>] device_shutdown+0x169/0x1d0
  [<0>] kernel_restart+0x3a/0x70
  [<0>] __do_sys_reboot+0x22b/0x270
  [<0>] do_syscall_64+0xaa/0x690
  [<0>] entry_SYSCALL_64_after_hwframe+0x76/0x7e

The offset lands on the wait_for_completion(&nhi->domain_released) after
tb_domain_remove(), so it looks like the last reference to the domain is
never dropped while the TS5 Plus is connected. Unbinding the NHI via sysfs
(echo 0000:c5:00.6 > /sys/bus/pci/drivers/thunderbolt/unbind) before reboot
blocks in the same place (unbind_store -> nhi_pci_remove).

Reproduction:
  1. Boot with the TS5 Plus connected to a rear USB4 port.
  2. systemctl reboot
  3. Machine hangs after the last systemd message and never resets.

Current workaround: sync, then SysRq-b (emergency_restart), which skips the
device_shutdown() walk.

Thanks,
Brandon Proulx

[-- Attachment #1.2: publickey - brandon.proulx@42fifty.dev - 0x5945A321.asc --]
[-- Type: application/pgp-keys, Size: 856 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 343 bytes --]

Re: Thunderbolt: reboot hangs in nhi_pci_remove() with CalDigit TS5 Plus on AMD Strix Halo

[Mika Westerberg]

Hi,

On Sat, Sep 26, 2026 at 02:06:42AM +0000, brandon.proulx@42fifty.dev wrote:
> Hi,
> On a Framework Desktop (AMD Ryzen AI Max 300 / Strix Halo) with a CalDigit
> TS5 Plus attached, reboot never completes. After systemd finishes, PID 1
> sits in D state in nhi_pci_remove() during device_shutdown() and the
> machine never resets. The hardware watchdog does not fire either. Only the
> power button recovers it.
> 
> Hardware:
>   System:  Framework Desktop (AMD Ryzen AI Max 300 Series), BIOS 03.06 (2026-07-13)
>   Host routers:
>     c5:00.5 AMD Strix Halo USB4 Host Router [1022:158d], subsys [f111:000a]
>     c5:00.6 AMD Strix Halo USB4 Host Router [1022:158e], subsys [f111:000a]
>   Device:  CalDigit TS5 Plus (thunderbolt 1-2, domain1 on c5:00.6), gen 4, NVM 64.c1
> 
> Kernel: 7.2.5 (distribution build 7.2.5-3-omarchy, Arch-based)
> thunderbolt.host_reset=Y (default)
> 
> Stack of PID 1, sampled every few seconds from reboot until power-off.
> Unchanged from 21:02:02 to 21:07:27:
> 
>   [<0>] nhi_pci_remove+0x53/0x60 [thunderbolt]
>   [<0>] pci_device_shutdown+0x3e/0x80
>   [<0>] device_shutdown+0x169/0x1d0
>   [<0>] kernel_restart+0x3a/0x70
>   [<0>] __do_sys_reboot+0x22b/0x270
>   [<0>] do_syscall_64+0xaa/0x690
>   [<0>] entry_SYSCALL_64_after_hwframe+0x76/0x7e
> 
> The offset lands on the wait_for_completion(&nhi->domain_released) after
> tb_domain_remove(), so it looks like the last reference to the domain is
> never dropped while the TS5 Plus is connected. Unbinding the NHI via sysfs
> (echo 0000:c5:00.6 > /sys/bus/pci/drivers/thunderbolt/unbind) before reboot
> blocks in the same place (unbind_store -> nhi_pci_remove).
> 
> Reproduction:
>   1. Boot with the TS5 Plus connected to a rear USB4 port.
>   2. systemctl reboot
>   3. Machine hangs after the last systemd message and never resets.
> 
> Current workaround: sync, then SysRq-b (emergency_restart), which skips the
> device_shutdown() walk.

There are bunch of fixes for this area in my "fixes" branch here:

https://git.kernel.org/pub/scm/linux/kernel/git/westeri/thunderbolt.git/

I wonder if you could try those and see if the problem goes away?

Re: Thunderbolt: reboot hangs in nhi_pci_remove() with CalDigit TS5 Plus on AMD Strix Halo

[brandon.proulx]

[-- Attachment #1.1: Type: text/plain, Size: 3448 bytes --]


Hi Mika,

Yes. Those fixes clear the hang on my system.

I could not boot the fixes branch as-is. It is based on 7.3-rc1, and this system runs 7.2.5-3-omarchy. I applied these four commits onto the 7.2.5 thunderbolt driver and loaded that module:

• thunderbolt: Fix domain reference leak when DPRX read is canceled
• thunderbolt: Don't access a DP tunnel after its DPRX read was canceled
• thunderbolt: Mark discovered tunnels as active
• thunderbolt: Tear down inactive DP tunnels when the domain is stopped

Before that, unbinding 0000:c5:00.6 with the TS5 Plus attached blocked in nhi_pci_remove() at wait_for_completion(&nhi->domain_released), and reboot(2) stuck in the same place until I held the power button. dprx_timeout is 12000, but the wait lasted many minutes.

With the patched module, that unbind returned in about a second and the dock came back on rebind. A normal systemctl reboot then finished: systemd unmounted the filesystems and the machine reset, instead of sitting on the splash.

Thanks!


On Saturday, September 26th, 2026 at 06:13, Mika Westerberg <mika.westerberg@linux.intel.com> wrote:

> Hi,
> 

> On Sat, Sep 26, 2026 at 02:06:42AM +0000, brandon.proulx@42fifty.dev wrote:
> > Hi,
> > On a Framework Desktop (AMD Ryzen AI Max 300 / Strix Halo) with a CalDigit
> > TS5 Plus attached, reboot never completes. After systemd finishes, PID 1
> > sits in D state in nhi_pci_remove() during device_shutdown() and the
> > machine never resets. The hardware watchdog does not fire either. Only the
> > power button recovers it.
> >
> > Hardware:
> >   System:  Framework Desktop (AMD Ryzen AI Max 300 Series), BIOS 03.06 (2026-07-13)
> >   Host routers:
> >     c5:00.5 AMD Strix Halo USB4 Host Router [1022:158d], subsys [f111:000a]
> >     c5:00.6 AMD Strix Halo USB4 Host Router [1022:158e], subsys [f111:000a]
> >   Device:  CalDigit TS5 Plus (thunderbolt 1-2, domain1 on c5:00.6), gen 4, NVM 64.c1
> >
> > Kernel: 7.2.5 (distribution build 7.2.5-3-omarchy, Arch-based)
> > thunderbolt.host_reset=Y (default)
> >
> > Stack of PID 1, sampled every few seconds from reboot until power-off.
> > Unchanged from 21:02:02 to 21:07:27:
> >
> >   [<0>] nhi_pci_remove+0x53/0x60 [thunderbolt]
> >   [<0>] pci_device_shutdown+0x3e/0x80
> >   [<0>] device_shutdown+0x169/0x1d0
> >   [<0>] kernel_restart+0x3a/0x70
> >   [<0>] __do_sys_reboot+0x22b/0x270
> >   [<0>] do_syscall_64+0xaa/0x690
> >   [<0>] entry_SYSCALL_64_after_hwframe+0x76/0x7e
> >
> > The offset lands on the wait_for_completion(&nhi->domain_released) after
> > tb_domain_remove(), so it looks like the last reference to the domain is
> > never dropped while the TS5 Plus is connected. Unbinding the NHI via sysfs
> > (echo 0000:c5:00.6 > /sys/bus/pci/drivers/thunderbolt/unbind) before reboot
> > blocks in the same place (unbind_store -> nhi_pci_remove).
> >
> > Reproduction:
> >   1. Boot with the TS5 Plus connected to a rear USB4 port.
> >   2. systemctl reboot
> >   3. Machine hangs after the last systemd message and never resets.
> >
> > Current workaround: sync, then SysRq-b (emergency_restart), which skips the
> > device_shutdown() walk.
> 

> There are bunch of fixes for this area in my "fixes" branch here:
> 

> https://git.kernel.org/pub/scm/linux/kernel/git/westeri/thunderbolt.git/
> 

> I wonder if you could try those and see if the problem goes away?
> 

[-- Attachment #1.2: publickey - brandon.proulx@42fifty.dev - 0x5945A321.asc --]
[-- Type: application/pgp-keys, Size: 856 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 343 bytes --]

Re: Thunderbolt: reboot hangs in nhi_pci_remove() with CalDigit TS5 Plus on AMD Strix Halo

[Mika Westerberg]

On Mon, Sep 28, 2026 at 08:29:55PM +0000, brandon.proulx@42fifty.dev wrote:
> 
> Hi Mika,
> 
> Yes. Those fixes clear the hang on my system.
> 
> I could not boot the fixes branch as-is. It is based on 7.3-rc1, and this system runs 7.2.5-3-omarchy. I applied these four commits onto the 7.2.5 thunderbolt driver and loaded that module:
> 
> • thunderbolt: Fix domain reference leak when DPRX read is canceled
> • thunderbolt: Don't access a DP tunnel after its DPRX read was canceled
> • thunderbolt: Mark discovered tunnels as active
> • thunderbolt: Tear down inactive DP tunnels when the domain is stopped
> 
> Before that, unbinding 0000:c5:00.6 with the TS5 Plus attached blocked in nhi_pci_remove() at wait_for_completion(&nhi->domain_released), and reboot(2) stuck in the same place until I held the power button. dprx_timeout is 12000, but the wait lasted many minutes.
> 
> With the patched module, that unbind returned in about a second and the dock came back on rebind. A normal systemctl reboot then finished: systemd unmounted the filesystems and the machine reset, instead of sitting on the splash.
> 

Great to hear that it helped :)