Re: Make ReBAR Great Again

[Cristian Cocos <cristi@ieee.org>]

Is this the right forum to post this? The hotplug list looked to me to
be the most appropriate venue, though please advise if you guys can
think of a more appropriate recipient.

The short story is that ReBAR does not work in hotplug scenarios:
hotplugged eGPUs are forced onto a 256MB BAR regardless of the system's
ReBAR capabilities, and this because during hotplug the Linux kernel
does not consult the ReBAR capability. It's that simple. Seeing as
eGPUs are becoming more popular, accommodating eGPU hotplug scenarios
becomes stringent.

On Sun, 2026-03-08 at 15:35 -0400, Cristian Cocos wrote:
> The following is a suggestion for the Linux kernel devs.
> 
> Now that Thunderbolt eGPUs have become more common, it might be a
> good
> idea to make the Linux kernel ReBAR-over-Thunderbolt friendly. This,
> I
> contend, can be done by simply mimicking the system behavior at boot
> time.
> 
> The current Thunderbolt eGPU *hotplug* sequence of events is this:
> BAR
> 2's hardware register powers up at 256 MB — the default size
> programmed
> into the BAR's address decoder by Intel at the factory. The PCIe
> Resizable BAR capability advertises support for up to 16 GB, but it's
> passive — software must explicitly exercise it. When a Thunderbolt
> eGPU
> is hotplugged at runtime, the kernel's PCI subsystem enumerates the
> new
> device, reads the BAR at its 256 MB default, sizes the bridge windows
> to match, and assigns addresses — all before any driver loads. The
> ReBAR capability is never consulted(!) during this process.
> 
> A workaround is available for cold-plug scenarios only, and is
> achieved by means of the **thunderbolt.host_reset=0** kernel
> parameter:
> this preserves the BIOS's PCIe tunnel and BAR assignments from POST
> (where the BIOS *does* exercise ReBAR). This delivers the full 16 GB
> BAR but, as just mentioned, only works for cold-plug(!) scenarios —
> if
> the eGPU is power-cycled at runtime, the new tunnel gets the 256 MB
> default.
> 
> The proper fix would be for the kernel's PCI hotplug resource
> assignment to *first* check for ReBAR capability during enumeration,
> resize the BAR to the largest supported size that fits within
> available
> bridge headroom, and *then* commit bridge windows and assign
> addresses.
> This is essentially what the BIOS does during POST. It hasn't been
> implemented yet because eGPU-over-Thunderbolt-with-ReBAR is (was?) a
> niche use case.
> 
> Well, no more niche use case. eGPU-over-Thunderbolt is becoming
> mainstream.
> 
>