Re: ReBAR over Thunderbolt

["Ilpo Järvinen" <ilpo.jarvinen@linux.intel.com>]

[-- Attachment #1: Type: text/plain, Size: 3615 bytes --]

On Thu, 26 Mar 2026, Cristian Cocos wrote:

> [This has been previously posted on the linux-hotplug list, though it
> has been suggested that I post it here as well.]
> 
> The short story is that ReBAR does not work in eGPU hotplug(!)
> scenarios: hotplugged Thunderbolt eGPUs are forced onto a 256MB BAR
> regardless of the system's ReBAR capabilities, and this because during
> hotplug the Linux kernel does not consult the ReBAR capability. As
> eGPUs are becoming more and more popular, accommodating eGPU hotplug
> scenarios has become imperative.
> 
> The longer story:
> 
> The current Thunderbolt eGPU *hotplug* sequence of events is this: BAR
> 2's hardware register powers up at 256 MB — the default size programmed
> into the BAR's address decoder by Intel at the factory. The PCIe
> Resizable BAR capability advertises support for up to 16 GB, but it's
> passive — software must explicitly exercise it. When a Thunderbolt eGPU
> is hotplugged at runtime, the kernel's PCI subsystem enumerates the new
> device, reads the BAR at its 256 MB default, sizes the bridge windows
> to match, and assigns addresses — all before any driver loads. The
> ReBAR capability is never consulted(!) during this process.

Hi,

The intel GPU drivers do attempt to perform the resize at probe time, 
though it's often not successful.

You haven't indicated what kernel you run nor provided any logs, but there 
have been some improvements to this in general in the recent kernels. And 
then there's also this patch which relates to thunderbolt cases (which was 
only accepted yesterday):

https://lore.kernel.org/linux-pci/20260219153951.68869-1-ilpo.jarvinen@linux.intel.com

> A workaround is available for cold-plug scenarios only, and is
> achieved by means of the **thunderbolt.host_reset=0** kernel parameter:
> this preserves the BIOS's PCIe tunnel and BAR assignments from POST
> (where the BIOS *does* exercise ReBAR). This delivers the full 16 GB
> BAR but, as just mentioned, only works for cold-plug(!) scenarios — if
> the eGPU is power-cycled at runtime, the new tunnel gets the 256 MB
> default.
> 
> The proper fix would be for the kernel's PCI hotplug resource
> assignment to *first* check for ReBAR capability during enumeration,
> resize the BAR to the largest supported size that fits within available
> bridge headroom, and *then* commit bridge windows and assign addresses.

That's being worked on. But it's not as easy as you depict it as kernel 
needs to be also able to fallback to some intermediate size if the largest 
supported size fails.

Thanks to how size calculations and assignment are done at different 
phases of the algorithm, it's not as easy to fallback to something else as 
you indicate as the secondary failures would still persist from the 
attempt with the largest window. So effectively, the entire sizing has to 
be recalculated.

> This is essentially what the BIOS does during POST. It hasn't been
> implemented yet because eGPU-over-Thunderbolt-with-ReBAR is (was?) a
> niche use case.
> 
> Seeing as eGPU-over-Thunderbolt is poised to become mainstream, this
> can no longer be considered a niche use case.

We don't consider it unimportant usecase. It's just the resource fitting 
and assignment algorithm is very complex and hard to improve, because 
something usually breaks when trying to improve it. Thus, the progress is 
slow and everything has to be done carefully (when things break badly in 
this area, system doesn't even come up so there's not much room for 
error).

-- 
 i.