Make ReBAR Great Again

Make ReBAR Great Again

[Cristian Cocos]

The following is a suggestion for the Linux kernel devs.

Now that Thunderbolt eGPUs have become more common, it might be a good
idea to make the Linux kernel ReBAR-over-Thunderbolt friendly. This, I
contend, can be done by simply mimicking the system behavior at boot
time.

The current Thunderbolt eGPU *hotplug* sequence of events is this: BAR
2's hardware register powers up at 256 MB — the default size programmed
into the BAR's address decoder by Intel at the factory. The PCIe
Resizable BAR capability advertises support for up to 16 GB, but it's
passive — software must explicitly exercise it. When a Thunderbolt eGPU
is hotplugged at runtime, the kernel's PCI subsystem enumerates the new
device, reads the BAR at its 256 MB default, sizes the bridge windows
to match, and assigns addresses — all before any driver loads. The
ReBAR capability is never consulted(!) during this process.

A workaround is available for cold-plug scenarios only, and is
achieved by means of the **thunderbolt.host_reset=0** kernel parameter:
this preserves the BIOS's PCIe tunnel and BAR assignments from POST
(where the BIOS *does* exercise ReBAR). This delivers the full 16 GB
BAR but, as just mentioned, only works for cold-plug(!) scenarios — if
the eGPU is power-cycled at runtime, the new tunnel gets the 256 MB
default.

The proper fix would be for the kernel's PCI hotplug resource
assignment to *first* check for ReBAR capability during enumeration,
resize the BAR to the largest supported size that fits within available
bridge headroom, and *then* commit bridge windows and assign addresses.
This is essentially what the BIOS does during POST. It hasn't been
implemented yet because eGPU-over-Thunderbolt-with-ReBAR is (was?) a
niche use case.

Well, no more niche use case. eGPU-over-Thunderbolt is becoming
mainstream.

Re: Make ReBAR Great Again

[Cristian Cocos]

Is this the right forum to post this? The hotplug list looked to me to
be the most appropriate venue, though please advise if you guys can
think of a more appropriate recipient.

The short story is that ReBAR does not work in hotplug scenarios:
hotplugged eGPUs are forced onto a 256MB BAR regardless of the system's
ReBAR capabilities, and this because during hotplug the Linux kernel
does not consult the ReBAR capability. It's that simple. Seeing as
eGPUs are becoming more popular, accommodating eGPU hotplug scenarios
becomes stringent.

On Sun, 2026-03-08 at 15:35 -0400, Cristian Cocos wrote:
> The following is a suggestion for the Linux kernel devs.
> 
> Now that Thunderbolt eGPUs have become more common, it might be a
> good
> idea to make the Linux kernel ReBAR-over-Thunderbolt friendly. This,
> I
> contend, can be done by simply mimicking the system behavior at boot
> time.
> 
> The current Thunderbolt eGPU *hotplug* sequence of events is this:
> BAR
> 2's hardware register powers up at 256 MB — the default size
> programmed
> into the BAR's address decoder by Intel at the factory. The PCIe
> Resizable BAR capability advertises support for up to 16 GB, but it's
> passive — software must explicitly exercise it. When a Thunderbolt
> eGPU
> is hotplugged at runtime, the kernel's PCI subsystem enumerates the
> new
> device, reads the BAR at its 256 MB default, sizes the bridge windows
> to match, and assigns addresses — all before any driver loads. The
> ReBAR capability is never consulted(!) during this process.
> 
> A workaround is available for cold-plug scenarios only, and is
> achieved by means of the **thunderbolt.host_reset=0** kernel
> parameter:
> this preserves the BIOS's PCIe tunnel and BAR assignments from POST
> (where the BIOS *does* exercise ReBAR). This delivers the full 16 GB
> BAR but, as just mentioned, only works for cold-plug(!) scenarios —
> if
> the eGPU is power-cycled at runtime, the new tunnel gets the 256 MB
> default.
> 
> The proper fix would be for the kernel's PCI hotplug resource
> assignment to *first* check for ReBAR capability during enumeration,
> resize the BAR to the largest supported size that fits within
> available
> bridge headroom, and *then* commit bridge windows and assign
> addresses.
> This is essentially what the BIOS does during POST. It hasn't been
> implemented yet because eGPU-over-Thunderbolt-with-ReBAR is (was?) a
> niche use case.
> 
> Well, no more niche use case. eGPU-over-Thunderbolt is becoming
> mainstream.
> 
> 

Re: Make ReBAR Great Again

[Greg KH]

On Thu, Mar 26, 2026 at 11:01:02AM -0400, Cristian Cocos wrote:
> Is this the right forum to post this? The hotplug list looked to me to
> be the most appropriate venue, though please advise if you guys can
> think of a more appropriate recipient.

linux-pci?