Make ReBAR Great Again

[Cristian Cocos <cristi@ieee.org>]

The following is a suggestion for the Linux kernel devs.

Now that Thunderbolt eGPUs have become more common, it might be a good
idea to make the Linux kernel ReBAR-over-Thunderbolt friendly. This, I
contend, can be done by simply mimicking the system behavior at boot
time.

The current Thunderbolt eGPU *hotplug* sequence of events is this: BAR
2's hardware register powers up at 256 MB — the default size programmed
into the BAR's address decoder by Intel at the factory. The PCIe
Resizable BAR capability advertises support for up to 16 GB, but it's
passive — software must explicitly exercise it. When a Thunderbolt eGPU
is hotplugged at runtime, the kernel's PCI subsystem enumerates the new
device, reads the BAR at its 256 MB default, sizes the bridge windows
to match, and assigns addresses — all before any driver loads. The
ReBAR capability is never consulted(!) during this process.

A workaround is available for cold-plug scenarios only, and is
achieved by means of the **thunderbolt.host_reset=0** kernel parameter:
this preserves the BIOS's PCIe tunnel and BAR assignments from POST
(where the BIOS *does* exercise ReBAR). This delivers the full 16 GB
BAR but, as just mentioned, only works for cold-plug(!) scenarios — if
the eGPU is power-cycled at runtime, the new tunnel gets the 256 MB
default.

The proper fix would be for the kernel's PCI hotplug resource
assignment to *first* check for ReBAR capability during enumeration,
resize the BAR to the largest supported size that fits within available
bridge headroom, and *then* commit bridge windows and assign addresses.
This is essentially what the BIOS does during POST. It hasn't been
implemented yet because eGPU-over-Thunderbolt-with-ReBAR is (was?) a
niche use case.

Well, no more niche use case. eGPU-over-Thunderbolt is becoming
mainstream.