Re: [PATCH] PCI/portdrv: Disable bwctrl service if port is fixed at 2.5 GT/s

[Mika Westerberg <mika.westerberg@linux.intel.com>]

On Fri, Dec 13, 2024 at 09:37:00AM +0100, Lukas Wunner wrote:
> On Thu, Dec 12, 2024 at 09:40:04PM +0100, Niklas Schnelle wrote:
> > On Thu, 2024-12-12 at 13:32 +0100, Lukas Wunner wrote:
> > > pcie_get_supported_speeds() is used to fill in the supported_speeds
> > > field in struct pci_dev.
> > > 
> > > And that field is used in a number of places (exposure of the max link
> > > speed in sysfs, delay handling in pci_bridge_wait_for_secondary_bus(),
> > > link tuning in radeon/amdgpu drivers, etc).
> > 
> > Side question. If this is used in radeon/amdgpu could detecting the
> > thunderbolt port's max link speed as 2.5 GT/s cause issues for external
> > GPUs?
> 
> I don't think so:
> 
> An attached Thunderbolt gadget (e.g. eGPU) is visible to the OS as a
> PCIe switch.  A portion of the Switch Downstream Ports is used to
> attach Endpoints (e.g. GPU) and the remainder is used for tunneling,
> i.e. to extend the hierarchy further if multiple Thunderbolt devices
> are daisy-chained.
> 
> My expectation is that the Max Link Speed is 8 GT/s on those Downstream
> Ports leading to Endpoints and 2.5 GT/s on those Downstream Ports used
> for tunneling (to conform with the USB4/Thunderbolt spec).  In other words,
> the Supported Link Speeds is the same on all of them, but Max Link Speed
> is reduced to 2.5 GT/s on so-called PCIe Adapters (in USB4/Thunderbolt
> terminology).
> 
> The PCIe Adapters encapsulate PCIe TLPs into Thunderbolt packets and
> send them over the Thunderbolt fabric, and similarly decapsulate TLPs
> received from the fabric.
> 
> There are some illustrations available here which explain the distinction
> between the two types of Downstream Ports:
> 
> https://developer.apple.com/library/archive/documentation/HardwareDrivers/Conceptual/ThunderboltDevGuide/Basics/Basics.html
> 
> I'm hoping Mika or Ilpo can verify the above information.  I have
> lspci dumps here of MeteorLake-P and BarlowRidge host controllers,
> but without any attached USB4/Thunderbolt gadgets.

That's right.

The bandwidth in the PCIe tunnel is dynamic but the adapters announce Gen 1
x 4 according to USB4 spec. You can get up to the 90% of the available
TB/USB4 link bandwidth for PCIe depending what is being tunneled. Graphics
drivers should not really look for these "virtual" PCIe links but instead
the native link if they need to.