All of lore.kernel.org
 help / color / mirror / Atom feed
* thunderbolt: PCIe tunnel creation fails for USB4 eGPU dock on Titan Ridge host
@ 2026-08-06 21:48 David GUENAULT
  2026-08-07  4:37 ` Mika Westerberg
  0 siblings, 1 reply; 7+ messages in thread
From: David GUENAULT @ 2026-08-06 21:48 UTC (permalink / raw)
  To: Andreas Noever, Mika Westerberg, Yehezkel Bernat; +Cc: linux-usb

Hi,

PCIe tunneling never succeeds between an Intel Titan Ridge 2C host router
and a USB4 (Thunderbolt 5) eGPU dock, although the Thunderbolt link itself
comes up fully bonded at 40 Gb/s and both routers enumerate cleanly.
tb_tunnel_pci() fails silently -- the only message is "PCIe tunnel creation
failed", with no path/credit/activation diagnostics even with full dyndbg.

=======================================================================
HARDWARE
=======================================================================
Host    : Intel NUC10i7FNH (Comet Lake, i7-10710U)
          BIOS FNCML357.0067.2026.0429.1614 (04/29/2026)
Host TBT: JHL7540 Titan Ridge 2C 2018
          Root port      00:1c.0  [8086:02bc]
          Bridges        01:00.0, 02:00.0, 02:01.0, 02:02.0  [8086:15e7]
          NHI            03:00.0  [8086:15e8]
          USB ctrl       39:00.0  [8086:15e9]
          Host router config space vendor/device = 0x15e78086
          => tb_switch_is_titan_ridge() == true
          NVM 42.0, generation 3

Device  : Minisforum DEG2 eGPU dock (OCuLink/TBT5, switch set to TBT)
          Thunderbolt router: "Micro Computer (HK) Tech. Ltd. TBGAA"
          vendor=0x41f device=0xd002
          Router config space: USB4 Switch 8087:5786 (Revision 133, TB
Version 64)
          NVM 62.42, generation 4 (USB4)
          GPU inside the dock: NVIDIA GTX 1060 (dock reports GPU
present via LED)

=======================================================================
SOFTWARE
=======================================================================
Ubuntu 24.04.4 LTS

Reproduced identically, byte-for-byte same failure, on BOTH:
  - 7.0.0-28-generic #28~24.04.1-Ubuntu   (distro kernel, based on 7.0.12)
  - 7.1.5-070105-generic                  (vanilla mainline v7.1.5 build from
                                           kernel.ubuntu.com/mainline, no
                                           distro patches)

Cmdline: pcie_ports=native pcie_aspm.policy=performance
         thunderbolt.start_icm=1 thunderbolt.host_reset=0

Connection manager: software CM. The driver enumerates all 23 adapters of
the remote router itself, so ICM is not running. thunderbolt.start_icm=1
does NOT start it (BIOS "Ignore Thunderbolt Option ROM" is unchecked, so
the option ROM is loaded).

=======================================================================
WHAT WORKS
=======================================================================
- ACPI _OSC: OS now controls [PCIeHotplug SHPCHotplug PME
PCIeCapability LTR DPC]
- pciehp armed on the tunnel bridge:
      pcieport 0000:02:01.0: pciehp: Slot #1 ... HotPlug+ Surprise+ LLActRep+
      /sys/bus/pci/slots/1 -> 0000:04:00
- Thunderbolt link fully up and bonded:
      rx/tx 20.0 Gb/s per lane, 2 lanes = 40 Gb/s each direction
      lane adapter error counters: all zero
- Remote router enumerates completely, no errors:
      DROM read, uid 0xf87e040080877eba, all 23 adapters read,
      quirk_usb3_maximum_bandwidth applied, NVM 62.42 read
- Adapters present on both sides:
      host 0-0: port8, port9 = TB_TYPE_PCIE_DOWN (0x100101)
                port5, port6 = DP_HDMI_IN
      dock 0-1: port9  = TB_TYPE_PCIE_UP   (0x100102)
                port10 = TB_TYPE_PCIE_DOWN (to the internal GPU)
                port11/12/19 = DP_HDMI_OUT, port20-23 = USB3
- Host PCIe down adapters are NOT pre-enabled by firmware:
      0-0/port8 cap_id 0x04 (TB_PORT_CAP_ADAP) = 0x2126043a  -> PE (bit31) = 0
      0-0/port9 cap_id 0x04 (TB_PORT_CAP_ADAP) = 0x2126043a  -> PE (bit31) = 0
  so tb_find_pcie_down() should not reject them.
- Expected mapping is satisfied: dock arrives on lane adapter 1,
  tb_phy_port_from_link(1) = 0, Titan Ridge => index 8 => port8 (free
PCIe down).
- BIOS resources reserved behind 02:01.0 are ample for a GTX 1060:
      bus 04-38, I/O 4K, mem 351M, prefetchable 64-bit 544M
      (GTX 1060 needs ~304M: BAR0 16M + BAR1 256M + BAR3 32M)

=======================================================================
WHAT FAILS
=======================================================================
With BIOS Thunderbolt Security Level = "Unique ID" (SL1, sysfs security=user),
boltd authorizes the device, which triggers tb_approve_switch(), and:

    thunderbolt 0-1: new device found, vendor=0x41f device=0xd002
    thunderbolt 0-1: Micro Computer (HK) Tech. Ltd. TBGAA
    thunderbolt 0000:03:00.0: 1: NVM version 62.42
    thunderbolt 0000:03:00.0: PCIe tunnel creation failed

boltctl then reports "authorization error"; /sys/.../0-1/authorized = 0.
02:01.0 stays at PresDet-, no device ever appears on bus 04, no GPU in lspci.

Notably there is NO "PCIe tunnel activation failed, aborting" message, which
suggests the failure happens in tb_tunnel_alloc_pci() rather than in
tb_tunnel_activate().

Full dyndbg was enabled ("module thunderbolt +p", 314 active sites) for all
captures. Nothing is logged between "NVM version 62.42" and the failure.

=======================================================================
ANALYSIS / SUSPICION
=======================================================================
Since the down port is found, free and correctly mapped, the most plausible
remaining failure point is credit allocation in tb_pci_init_credits():

    if (tb_port_use_credit_allocation(port)) {
            credits = sw->max_pcie_credits;
            available = tb_available_credits(port, NULL);
            if (available < credits)
                    return -ENOSPC;      /* silent */
    }

tb_port_use_credit_allocation() is true for USB4 routers, and the dock IS a
USB4 router -- but the path terminates on a Thunderbolt 3 Titan Ridge host.
Adapter credits reported by the dock:
    lane adapters (port1-4): Credits (total/control) 160/2
    PCIe adapters (port9/10): Credits (total/control) 8/0, NFC 0x40800000
    USB3/DP adapters:         Credits (total/control) 8/0

I suspect the USB4 credit-allocation path is applied to a tunnel that
traverses a non-USB4 host router, yielding -ENOSPC. I could not confirm this
without instrumenting the driver; happy to test any patch or add tracing.

=======================================================================
NOTES ON SETTINGS THAT MATTER
=======================================================================
1. With BIOS Security Level = "No Security" (sysfs security=none), the device
   is auto-authorized and tb_approve_switch() is never called, so NO tunnel is
   ever attempted and nothing at all is logged. Switching to SL1 ("Unique ID")
   is what makes the driver actually try. This asymmetry is confusing: with
   security=none plus software CM and no ICM running, nobody creates the
   PCIe tunnel and the device silently looks "authorized" but is unusable.

2. thunderbolt.host_reset=0 measurably improves link behaviour on this TBT3
   host: with the default (host_reset=1) the link never came up at boot and
   required a manual unplug/replug; with host_reset=0 the router is connected
   and bonded at 40 Gb/s directly at boot. The parameter is documented as
   targeting USB4 host routers, yet it clearly affects this Titan Ridge host.

3. Unrelated but worth noting: passing pcie_aspm=off makes
   negotiate_os_control() bail out entirely --
   "_OSC: not requesting OS control; OS requires [ExtendedConfig ASPM
ClockPM MSI]"
   -- which silently drops native PCIe hotplug, so pciehp never binds to the
   Thunderbolt downstream port and no tunnelled device can ever be enumerated.
   pcie_aspm.policy=performance avoids this.

=======================================================================
WORKAROUNDS TRIED WITHOUT SUCCESS
=======================================================================
- thunderbolt.clx=0            (no change; link still dropped at the time)
- thunderbolt.start_icm=1      (ICM does not start; still software CM)
- pci=realloc / assign-busses / nocrs / hpbussize / hpmemsize
  (assign-busses is actively harmful: it shrinks the BIOS reservation
   [bus 04-38] down to [bus 04])
- Cold power cycle (AC unplugged), several cable replug sequences,
  certified TBT3/4 cable, dock's OCuLink/TBT switch verified on TBT.
- Runtime PM: the remote router suspends and then fails to resume
  ("thunderbolt 0-1: runtime resuming timed out"), which prevents tunnel
  creation as well. Worked around with a udev rule setting
  ATTR{power/control}="on" on SUBSYSTEM=="thunderbolt" and on the
  8086:15e7/15e8/15e9 PCI devices.

I can provide full dyndbg captures, debugfs register dumps for any adapter,
and test patches. Thanks!

^ permalink raw reply	[flat|nested] 7+ messages in thread
end of thread, other threads:[~2026-08-07 11:35 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-08-06 21:48 thunderbolt: PCIe tunnel creation fails for USB4 eGPU dock on Titan Ridge host David GUENAULT
2026-08-07  4:37 ` Mika Westerberg
2026-08-07  9:55   ` David GUENAULT
2026-08-07 10:18     ` Mika Westerberg
2026-08-07 11:01       ` David GUENAULT
2026-08-07 11:29         ` Mika Westerberg
2026-08-07 11:35           ` Mika Westerberg

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.