Linux PCI subsystem development
 help / color / mirror / Atom feed
* PCI: Resizable BAR never considered for Thunderbolt/USB4 hotplugged eGPUs
@ 2026-09-11  8:58 鳥井原茂
  2026-09-11  9:08 ` Mika Westerberg
  0 siblings, 1 reply; 4+ messages in thread
From: 鳥井原茂 @ 2026-09-11  8:58 UTC (permalink / raw)
  To: linux-pci
  Cc: bhelgaas, ilpo.jarvinen, linux-usb, westeri, andreas.noever,
	YehezkelShB

Hi,

I am reporting a case where a Thunderbolt/USB4-attached eGPU is
enumerated and links
correctly but can never be used for compute, because its BAR stays at
the power-on
default of 256MB and the Resizable BAR capability is never exercised.

I am not sure whether this is considered a known limitation or a bug
worth fixing, and
I would like to ask before assuming either. I have the hardware and am
happy to test
patches.

Environment
-----------
  Host:      ThinkPad X1 Carbon Gen 13 (21NS0000JP), Lunar Lake
             BIOS N4BET77W (1.47), 2026-06-30
  Kernel:    7.0.0-31 (Ubuntu 26.04.1)
  Enclosure: AOOSTAR AG03, connected over USB4 (not OCuLink)
  GPU:       Intel Arc B580 (Battlemage, 12GB), driver xe

What happens
------------
The card enumerates, binds to xe, and its edge connector trains at
16GT/s x4, which is
the expected ceiling for a USB4 PCIe tunnel. The link is healthy.

BAR2 stays at 256 MiB. The card advertises Resizable BAR support for
256MB..16GB:

  # cat /sys/bus/pci/devices/0000:24:00.0/resource2_resize
  0000000000007f00

Level Zero (intel-compute-runtime 26.31.39395.13) prints

  WARNING: Resizable BAR not detected for device 0000:24:00.0

and then drops the device from enumeration entirely, so it never
appears to any compute
API. The integrated Arc 140V is enumerated normally.

Why the BAR cannot be grown after the fact
------------------------------------------
Attempting the resize by hand fails for every size down to 512MB:

  # echo 0000:24:00.0 > /sys/bus/pci/drivers/xe/unbind
  # echo 14 > /sys/bus/pci/devices/0000:24:00.0/resource2_resize   #
16GB -> fails
  # echo 9  > /sys/bus/pci/devices/0000:24:00.0/resource2_resize   #
512MB -> fails

BAR2 is placed at 0x2010000000, which is only 256MB-aligned, so a
larger size cannot be
aligned to the bridge base. The prefetchable window of the Thunderbolt root port
(0000:00:07.0) is only a few hundred MB and comes from firmware:

  PCI: Using host bridge windows from ACPI; if necessary, use
"pci=nocrs" and report a bug

I measured 711 MiB and 448 MiB for that window across two boots.

Things that do not help
-----------------------
  pci=realloc                  no effect on already-populated bridges

  pci=hpmmioprefsize=32G       makes it worse: the Thunderbolt
downstream port the GPU
                               sits behind gets no window at all, and
the whole card
                               disappears from the bus:

    pci 0000:22:00.0: BAR 0 [mem 0x00800000-0x00ffffff 64bit pref]: assigned
    pcieport 0000:21:00.0: bridge window [mem 0x00800000-0x00ffffff 64bit pref]:
        can't claim; no compatible bridge window

  pci=nocrs                    the machine does not boot: the NVMe
holding the rootfs
                               fails to get resources and probe returns -ENOMEM
    nvme 0000:04:00.0: probe with driver nvme failed with error -12

  thunderbolt.host_reset=0     no effect here, and I believe the
reason is that the
                               firmware never enumerates the tunnel at
POST. The eGPU
                               first appears 63 seconds after kernel
boot (16:05:11 boot,
                               16:06:14 first sighting of the card),
i.e. purely as an
                               OS-side hotplug event. There is nothing
from POST to
                               preserve.

The BIOS has no relevant setup option: under Config > Thunderbolt 4
there is only
"PCIe Tunneling" (on/off). No Resizable BAR, no Above 4G Decoding, no
BIOS Assist Mode.
I have filed a separate request with the vendor about that.

What I think the underlying issue is
------------------------------------
On hotplug, the PCI core reads the BAR at its power-on size, sizes the
bridge windows to
match and assigns addresses, all before any driver binds -- and the
ReBAR capability is
not consulted at any point in that path. By the time a driver could
ask for a larger BAR,
the surrounding windows are already committed and too small to grow into.

This looks like exactly what "PCI: Allow BAR movement during boot and hotplug"
(Sergei Miroshnichenko, v9, Dec 2020, 26 patches) was written to solve
-- the cover letter
explicitly lists Resizable BARs as a use case. As far as I can tell it
never landed:

  # grep -c -e rescan_prepare -e rescan_done -e bar_fixed -e
movable_bars /proc/kallsyms
  0

Questions
---------
1. Is ReBAR-over-Thunderbolt-hotplug considered a known limitation, or
is this worth
   treating as a bug?
2. Is there any current plan to consult the ReBAR capability during
hotplug resource
   assignment?
3. What is the status of the movable-BARs series? Is it abandoned,
superseded, or waiting
   on something?
4. Is there a kernel parameter or sysfs path I have missed that would
let the BAR grow
   after enumeration?

Thanks, s-tory.

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: PCI: Resizable BAR never considered for Thunderbolt/USB4 hotplugged eGPUs
  2026-09-11  8:58 PCI: Resizable BAR never considered for Thunderbolt/USB4 hotplugged eGPUs 鳥井原茂
@ 2026-09-11  9:08 ` Mika Westerberg
  2026-09-29 12:17   ` Ilpo Järvinen
  0 siblings, 1 reply; 4+ messages in thread
From: Mika Westerberg @ 2026-09-11  9:08 UTC (permalink / raw)
  To: 鳥井原茂
  Cc: linux-pci, bhelgaas, ilpo.jarvinen, linux-usb, westeri,
	andreas.noever, YehezkelShB

Hi,

On Fri, Sep 11, 2026 at 05:58:26PM +0900, 鳥井原茂 wrote:
> Hi,
> 
> I am reporting a case where a Thunderbolt/USB4-attached eGPU is
> enumerated and links
> correctly but can never be used for compute, because its BAR stays at
> the power-on
> default of 256MB and the Resizable BAR capability is never exercised.
> 
> I am not sure whether this is considered a known limitation or a bug
> worth fixing, and
> I would like to ask before assuming either. I have the hardware and am
> happy to test
> patches.
> 
> Environment
> -----------
>   Host:      ThinkPad X1 Carbon Gen 13 (21NS0000JP), Lunar Lake
>              BIOS N4BET77W (1.47), 2026-06-30

I don't know if we support REBAR or how well in PCIe side, Ilpo probably
knows that. Typically there is just certain amount of resources allocated
for each PCIe root port that gets tunneled so the way to do this is to
increase that in the BIOS. Having said that most of the vendors don't
actually allow it to be changed.

>   Kernel:    7.0.0-31 (Ubuntu 26.04.1)
>   Enclosure: AOOSTAR AG03, connected over USB4 (not OCuLink)
>   GPU:       Intel Arc B580 (Battlemage, 12GB), driver xe
> 
> What happens
> ------------
> The card enumerates, binds to xe, and its edge connector trains at
> 16GT/s x4, which is
> the expected ceiling for a USB4 PCIe tunnel. The link is healthy.
> 
> BAR2 stays at 256 MiB. The card advertises Resizable BAR support for
> 256MB..16GB:
> 
>   # cat /sys/bus/pci/devices/0000:24:00.0/resource2_resize
>   0000000000007f00
> 
> Level Zero (intel-compute-runtime 26.31.39395.13) prints
> 
>   WARNING: Resizable BAR not detected for device 0000:24:00.0
> 
> and then drops the device from enumeration entirely, so it never
> appears to any compute
> API. The integrated Arc 140V is enumerated normally.
> 
> Why the BAR cannot be grown after the fact
> ------------------------------------------
> Attempting the resize by hand fails for every size down to 512MB:
> 
>   # echo 0000:24:00.0 > /sys/bus/pci/drivers/xe/unbind
>   # echo 14 > /sys/bus/pci/devices/0000:24:00.0/resource2_resize   #
> 16GB -> fails
>   # echo 9  > /sys/bus/pci/devices/0000:24:00.0/resource2_resize   #
> 512MB -> fails
> 
> BAR2 is placed at 0x2010000000, which is only 256MB-aligned, so a
> larger size cannot be
> aligned to the bridge base. The prefetchable window of the Thunderbolt root port
> (0000:00:07.0) is only a few hundred MB and comes from firmware:
> 
>   PCI: Using host bridge windows from ACPI; if necessary, use
> "pci=nocrs" and report a bug
> 
> I measured 711 MiB and 448 MiB for that window across two boots.
> 
> Things that do not help
> -----------------------
>   pci=realloc                  no effect on already-populated bridges
> 
>   pci=hpmmioprefsize=32G       makes it worse: the Thunderbolt
> downstream port the GPU
>                                sits behind gets no window at all, and
> the whole card
>                                disappears from the bus:
> 
>     pci 0000:22:00.0: BAR 0 [mem 0x00800000-0x00ffffff 64bit pref]: assigned
>     pcieport 0000:21:00.0: bridge window [mem 0x00800000-0x00ffffff 64bit pref]:
>         can't claim; no compatible bridge window
> 
>   pci=nocrs                    the machine does not boot: the NVMe
> holding the rootfs
>                                fails to get resources and probe returns -ENOMEM
>     nvme 0000:04:00.0: probe with driver nvme failed with error -12
> 
>   thunderbolt.host_reset=0     no effect here, and I believe the
> reason is that the
>                                firmware never enumerates the tunnel at
> POST. The eGPU
>                                first appears 63 seconds after kernel
> boot (16:05:11 boot,
>                                16:06:14 first sighting of the card),
> i.e. purely as an
>                                OS-side hotplug event. There is nothing
> from POST to
>                                preserve.
> 
> The BIOS has no relevant setup option: under Config > Thunderbolt 4
> there is only
> "PCIe Tunneling" (on/off). No Resizable BAR, no Above 4G Decoding, no
> BIOS Assist Mode.
> I have filed a separate request with the vendor about that.

BIOS assist mode is a workaround for early Windows systems that did not
support native PCIe hotplug. It should not be used in any modern systems
and I doubt it's not even capable of what you are doing.

> What I think the underlying issue is
> ------------------------------------
> On hotplug, the PCI core reads the BAR at its power-on size, sizes the
> bridge windows to
> match and assigns addresses, all before any driver binds -- and the
> ReBAR capability is
> not consulted at any point in that path. By the time a driver could
> ask for a larger BAR,
> the surrounding windows are already committed and too small to grow into.
> 
> This looks like exactly what "PCI: Allow BAR movement during boot and hotplug"
> (Sergei Miroshnichenko, v9, Dec 2020, 26 patches) was written to solve
> -- the cover letter
> explicitly lists Resizable BARs as a use case. As far as I can tell it
> never landed:
> 
>   # grep -c -e rescan_prepare -e rescan_done -e bar_fixed -e
> movable_bars /proc/kallsyms
>   0
> 
> Questions
> ---------
> 1. Is ReBAR-over-Thunderbolt-hotplug considered a known limitation, or
> is this worth
>    treating as a bug?

I would say here that it's not even related to the underlying transport,
could be something else than TB too, as long as PCIe goes inside.

> 2. Is there any current plan to consult the ReBAR capability during
> hotplug resource
>    assignment?
> 3. What is the status of the movable-BARs series? Is it abandoned,
> superseded, or waiting
>    on something?
> 4. Is there a kernel parameter or sysfs path I have missed that would
> let the BAR grow
>    after enumeration?
> 
> Thanks, s-tory.

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: PCI: Resizable BAR never considered for Thunderbolt/USB4 hotplugged eGPUs
  2026-09-11  9:08 ` Mika Westerberg
@ 2026-09-29 12:17   ` Ilpo Järvinen
  2026-10-01  7:55     ` 鳥井原茂
  0 siblings, 1 reply; 4+ messages in thread
From: Ilpo Järvinen @ 2026-09-29 12:17 UTC (permalink / raw)
  To: Mika Westerberg
  Cc: 鳥井原茂, linux-pci, bhelgaas, linux-usb,
	westeri, andreas.noever, YehezkelShB

[-- Attachment #1: Type: text/plain, Size: 8080 bytes --]

On Fri, 11 Sep 2026, Mika Westerberg wrote:
> On Fri, Sep 11, 2026 at 05:58:26PM +0900, 鳥井原茂 wrote:
> > 
> > I am reporting a case where a Thunderbolt/USB4-attached eGPU is
> > enumerated and links
> > correctly but can never be used for compute, because its BAR stays at
> > the power-on
> > default of 256MB and the Resizable BAR capability is never exercised.
> > 
> > I am not sure whether this is considered a known limitation or a bug
> > worth fixing, and
> > I would like to ask before assuming either. I have the hardware and am
> > happy to test
> > patches.
> > 
> > Environment
> > -----------
> >   Host:      ThinkPad X1 Carbon Gen 13 (21NS0000JP), Lunar Lake
> >              BIOS N4BET77W (1.47), 2026-06-30
> 
> I don't know if we support REBAR or how well in PCIe side, Ilpo probably
> knows that. Typically there is just certain amount of resources allocated
> for each PCIe root port that gets tunneled so the way to do this is to
> increase that in the BIOS. Having said that most of the vendors don't
> actually allow it to be changed.
> 
> >   Kernel:    7.0.0-31 (Ubuntu 26.04.1)
> >   Enclosure: AOOSTAR AG03, connected over USB4 (not OCuLink)
> >   GPU:       Intel Arc B580 (Battlemage, 12GB), driver xe
> > 
> > What happens
> > ------------
> > The card enumerates, binds to xe, and its edge connector trains at
> > 16GT/s x4, which is
> > the expected ceiling for a USB4 PCIe tunnel. The link is healthy.
> > 
> > BAR2 stays at 256 MiB. The card advertises Resizable BAR support for
> > 256MB..16GB:
> > 
> >   # cat /sys/bus/pci/devices/0000:24:00.0/resource2_resize
> >   0000000000007f00
> > 
> > Level Zero (intel-compute-runtime 26.31.39395.13) prints
> > 
> >   WARNING: Resizable BAR not detected for device 0000:24:00.0
> > 
> > and then drops the device from enumeration entirely, so it never
> > appears to any compute
> > API. The integrated Arc 140V is enumerated normally.
> > 
> > Why the BAR cannot be grown after the fact
> > ------------------------------------------
> > Attempting the resize by hand fails for every size down to 512MB:
> > 
> >   # echo 0000:24:00.0 > /sys/bus/pci/drivers/xe/unbind
> >   # echo 14 > /sys/bus/pci/devices/0000:24:00.0/resource2_resize   #
> > 16GB -> fails
> >   # echo 9  > /sys/bus/pci/devices/0000:24:00.0/resource2_resize   #
> > 512MB -> fails
> > 
> > BAR2 is placed at 0x2010000000, which is only 256MB-aligned, so a
> > larger size cannot be
> > aligned to the bridge base. The prefetchable window of the Thunderbolt root port
> > (0000:00:07.0) is only a few hundred MB and comes from firmware:

The usual result.

> >   PCI: Using host bridge windows from ACPI; if necessary, use
> > "pci=nocrs" and report a bug
> > 
> > I measured 711 MiB and 448 MiB for that window across two boots.
> > 
> > Things that do not help
> > -----------------------
> >   pci=realloc                  no effect on already-populated bridges

This was my discovery as well which blocked my Resizable BAR aware 
resource sizing from doing anything. I did some work to solve it but then 
got regressions from the earlier resource work to address so I've not had 
time to look into this for a while.

Solving this is among the last few blockers for the resizable BAR-aware 
resource sizing (sans the yet to be discovered bugs in the resizable 
BAR-aware code itself).

> >   pci=hpmmioprefsize=32G       makes it worse: the Thunderbolt
> > downstream port the GPU
> >                                sits behind gets no window at all, and
> > the whole card
> >                                disappears from the bus:
> > 
> >     pci 0000:22:00.0: BAR 0 [mem 0x00800000-0x00ffffff 64bit pref]: assigned
> >     pcieport 0000:21:00.0: bridge window [mem 0x00800000-0x00ffffff 64bit pref]:
> >         can't claim; no compatible bridge window
> > 
> >   pci=nocrs                    the machine does not boot: the NVMe
> > holding the rootfs
> >                                fails to get resources and probe returns -ENOMEM
> >     nvme 0000:04:00.0: probe with driver nvme failed with error -12
> > 
> >   thunderbolt.host_reset=0     no effect here, and I believe the
> > reason is that the
> >                                firmware never enumerates the tunnel at
> > POST. The eGPU
> >                                first appears 63 seconds after kernel
> > boot (16:05:11 boot,
> >                                16:06:14 first sighting of the card),
> > i.e. purely as an
> >                                OS-side hotplug event. There is nothing
> > from POST to
> >                                preserve.
> > 
> > The BIOS has no relevant setup option: under Config > Thunderbolt 4
> > there is only
> > "PCIe Tunneling" (on/off). No Resizable BAR, no Above 4G Decoding, no
> > BIOS Assist Mode.
> > I have filed a separate request with the vendor about that.
> 
> BIOS assist mode is a workaround for early Windows systems that did not
> support native PCIe hotplug. It should not be used in any modern systems
> and I doubt it's not even capable of what you are doing.
> 
> > What I think the underlying issue is
> > ------------------------------------
> > On hotplug, the PCI core reads the BAR at its power-on size, sizes the
> > bridge windows to
> > match and assigns addresses, all before any driver binds -- and the
> > ReBAR capability is
> > not consulted at any point in that path. By the time a driver could
> > ask for a larger BAR,
> > the surrounding windows are already committed and too small to grow into.
> > 
> > This looks like exactly what "PCI: Allow BAR movement during boot and hotplug"
> > (Sergei Miroshnichenko, v9, Dec 2020, 26 patches) was written to solve
> > -- the cover letter
> > explicitly lists Resizable BARs as a use case. As far as I can tell it
> > never landed:
> > 
> >   # grep -c -e rescan_prepare -e rescan_done -e bar_fixed -e
> > movable_bars /proc/kallsyms
> >   0
> > 
> > Questions
> > ---------
> > 1. Is ReBAR-over-Thunderbolt-hotplug considered a known limitation, or
> > is this worth
> >    treating as a bug?
> 
> I would say here that it's not even related to the underlying transport,
> could be something else than TB too, as long as PCIe goes inside.

It's a known problem, but fixing it without introducing regressions is not 
easy. The solution must be able to fallback gracefully to smaller sizes 
as needed. Without that, a regression that will surely come up soonish 
would just lead reverting the improvment.

> > 2. Is there any current plan to consult the ReBAR capability during
> > hotplug resource
> >    assignment?

Yes, infact patches already exist to consired ReBAR during resource 
fitting, but they just don't run as is when a bridge is already assigned
so more work is still needed once I get the most recent set of regressions 
out of the way first.

> > 3. What is the status of the movable-BARs series? Is it abandoned,
> > superseded, or waiting
> >    on something?

I've no plans on basing anything on that work.

> > 4. Is there a kernel parameter or sysfs path I have missed that would
> > let the BAR grow
> >    after enumeration?

In some cases, removing siblings that pin any of the upstream bridge 
windows in place prior to executing the resize may allow the entire window 
to move elsewhere.

There's even a patch from Geramy Loveless that would do this automatically 
for empty siblings [1] which is definitely worth a try. It does not work 
universally, however, so YMMV (I don't see your /proc/iomem anywhere so I 
cannot tell whether it likely works in your case).

Somebody was against Geramy's patch as it didn't (and kind of doesn't 
even want to) ensure the unused siblings would be guaranteed to be 
allocated at least some space after the resize (any unused hotplug 
reservation is "optional" so it would be a major change to alter that 
differentiation to address that review comment).

But if that patch works for you, sending your Tested-by tag to its thread 
might actually be helpful.

[1] https://lore.kernel.org/linux-pci/20260428225146.3104063-1-gloveless@jqluv.com/

-- 
 i.

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: PCI: Resizable BAR never considered for Thunderbolt/USB4 hotplugged eGPUs
  2026-09-29 12:17   ` Ilpo Järvinen
@ 2026-10-01  7:55     ` 鳥井原茂
  0 siblings, 0 replies; 4+ messages in thread
From: 鳥井原茂 @ 2026-10-01  7:55 UTC (permalink / raw)
  To: Ilpo Järvinen
  Cc: Mika Westerberg, linux-pci, bhelgaas, linux-usb, westeri,
	andreas.noever, YehezkelShB

Hi Ilpo,

Thank you very much for the detailed reply! I'll try the patch when I
get a chance and share the results.

Best regards, s-tory.

2026年9月29日(火) 21:17 Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>:
>
> On Fri, 11 Sep 2026, Mika Westerberg wrote:
> > On Fri, Sep 11, 2026 at 05:58:26PM +0900, 鳥井原茂 wrote:
> > >
> > > I am reporting a case where a Thunderbolt/USB4-attached eGPU is
> > > enumerated and links
> > > correctly but can never be used for compute, because its BAR stays at
> > > the power-on
> > > default of 256MB and the Resizable BAR capability is never exercised.
> > >
> > > I am not sure whether this is considered a known limitation or a bug
> > > worth fixing, and
> > > I would like to ask before assuming either. I have the hardware and am
> > > happy to test
> > > patches.
> > >
> > > Environment
> > > -----------
> > >   Host:      ThinkPad X1 Carbon Gen 13 (21NS0000JP), Lunar Lake
> > >              BIOS N4BET77W (1.47), 2026-06-30
> >
> > I don't know if we support REBAR or how well in PCIe side, Ilpo probably
> > knows that. Typically there is just certain amount of resources allocated
> > for each PCIe root port that gets tunneled so the way to do this is to
> > increase that in the BIOS. Having said that most of the vendors don't
> > actually allow it to be changed.
> >
> > >   Kernel:    7.0.0-31 (Ubuntu 26.04.1)
> > >   Enclosure: AOOSTAR AG03, connected over USB4 (not OCuLink)
> > >   GPU:       Intel Arc B580 (Battlemage, 12GB), driver xe
> > >
> > > What happens
> > > ------------
> > > The card enumerates, binds to xe, and its edge connector trains at
> > > 16GT/s x4, which is
> > > the expected ceiling for a USB4 PCIe tunnel. The link is healthy.
> > >
> > > BAR2 stays at 256 MiB. The card advertises Resizable BAR support for
> > > 256MB..16GB:
> > >
> > >   # cat /sys/bus/pci/devices/0000:24:00.0/resource2_resize
> > >   0000000000007f00
> > >
> > > Level Zero (intel-compute-runtime 26.31.39395.13) prints
> > >
> > >   WARNING: Resizable BAR not detected for device 0000:24:00.0
> > >
> > > and then drops the device from enumeration entirely, so it never
> > > appears to any compute
> > > API. The integrated Arc 140V is enumerated normally.
> > >
> > > Why the BAR cannot be grown after the fact
> > > ------------------------------------------
> > > Attempting the resize by hand fails for every size down to 512MB:
> > >
> > >   # echo 0000:24:00.0 > /sys/bus/pci/drivers/xe/unbind
> > >   # echo 14 > /sys/bus/pci/devices/0000:24:00.0/resource2_resize   #
> > > 16GB -> fails
> > >   # echo 9  > /sys/bus/pci/devices/0000:24:00.0/resource2_resize   #
> > > 512MB -> fails
> > >
> > > BAR2 is placed at 0x2010000000, which is only 256MB-aligned, so a
> > > larger size cannot be
> > > aligned to the bridge base. The prefetchable window of the Thunderbolt root port
> > > (0000:00:07.0) is only a few hundred MB and comes from firmware:
>
> The usual result.
>
> > >   PCI: Using host bridge windows from ACPI; if necessary, use
> > > "pci=nocrs" and report a bug
> > >
> > > I measured 711 MiB and 448 MiB for that window across two boots.
> > >
> > > Things that do not help
> > > -----------------------
> > >   pci=realloc                  no effect on already-populated bridges
>
> This was my discovery as well which blocked my Resizable BAR aware
> resource sizing from doing anything. I did some work to solve it but then
> got regressions from the earlier resource work to address so I've not had
> time to look into this for a while.
>
> Solving this is among the last few blockers for the resizable BAR-aware
> resource sizing (sans the yet to be discovered bugs in the resizable
> BAR-aware code itself).
>
> > >   pci=hpmmioprefsize=32G       makes it worse: the Thunderbolt
> > > downstream port the GPU
> > >                                sits behind gets no window at all, and
> > > the whole card
> > >                                disappears from the bus:
> > >
> > >     pci 0000:22:00.0: BAR 0 [mem 0x00800000-0x00ffffff 64bit pref]: assigned
> > >     pcieport 0000:21:00.0: bridge window [mem 0x00800000-0x00ffffff 64bit pref]:
> > >         can't claim; no compatible bridge window
> > >
> > >   pci=nocrs                    the machine does not boot: the NVMe
> > > holding the rootfs
> > >                                fails to get resources and probe returns -ENOMEM
> > >     nvme 0000:04:00.0: probe with driver nvme failed with error -12
> > >
> > >   thunderbolt.host_reset=0     no effect here, and I believe the
> > > reason is that the
> > >                                firmware never enumerates the tunnel at
> > > POST. The eGPU
> > >                                first appears 63 seconds after kernel
> > > boot (16:05:11 boot,
> > >                                16:06:14 first sighting of the card),
> > > i.e. purely as an
> > >                                OS-side hotplug event. There is nothing
> > > from POST to
> > >                                preserve.
> > >
> > > The BIOS has no relevant setup option: under Config > Thunderbolt 4
> > > there is only
> > > "PCIe Tunneling" (on/off). No Resizable BAR, no Above 4G Decoding, no
> > > BIOS Assist Mode.
> > > I have filed a separate request with the vendor about that.
> >
> > BIOS assist mode is a workaround for early Windows systems that did not
> > support native PCIe hotplug. It should not be used in any modern systems
> > and I doubt it's not even capable of what you are doing.
> >
> > > What I think the underlying issue is
> > > ------------------------------------
> > > On hotplug, the PCI core reads the BAR at its power-on size, sizes the
> > > bridge windows to
> > > match and assigns addresses, all before any driver binds -- and the
> > > ReBAR capability is
> > > not consulted at any point in that path. By the time a driver could
> > > ask for a larger BAR,
> > > the surrounding windows are already committed and too small to grow into.
> > >
> > > This looks like exactly what "PCI: Allow BAR movement during boot and hotplug"
> > > (Sergei Miroshnichenko, v9, Dec 2020, 26 patches) was written to solve
> > > -- the cover letter
> > > explicitly lists Resizable BARs as a use case. As far as I can tell it
> > > never landed:
> > >
> > >   # grep -c -e rescan_prepare -e rescan_done -e bar_fixed -e
> > > movable_bars /proc/kallsyms
> > >   0
> > >
> > > Questions
> > > ---------
> > > 1. Is ReBAR-over-Thunderbolt-hotplug considered a known limitation, or
> > > is this worth
> > >    treating as a bug?
> >
> > I would say here that it's not even related to the underlying transport,
> > could be something else than TB too, as long as PCIe goes inside.
>
> It's a known problem, but fixing it without introducing regressions is not
> easy. The solution must be able to fallback gracefully to smaller sizes
> as needed. Without that, a regression that will surely come up soonish
> would just lead reverting the improvment.
>
> > > 2. Is there any current plan to consult the ReBAR capability during
> > > hotplug resource
> > >    assignment?
>
> Yes, infact patches already exist to consired ReBAR during resource
> fitting, but they just don't run as is when a bridge is already assigned
> so more work is still needed once I get the most recent set of regressions
> out of the way first.
>
> > > 3. What is the status of the movable-BARs series? Is it abandoned,
> > > superseded, or waiting
> > >    on something?
>
> I've no plans on basing anything on that work.
>
> > > 4. Is there a kernel parameter or sysfs path I have missed that would
> > > let the BAR grow
> > >    after enumeration?
>
> In some cases, removing siblings that pin any of the upstream bridge
> windows in place prior to executing the resize may allow the entire window
> to move elsewhere.
>
> There's even a patch from Geramy Loveless that would do this automatically
> for empty siblings [1] which is definitely worth a try. It does not work
> universally, however, so YMMV (I don't see your /proc/iomem anywhere so I
> cannot tell whether it likely works in your case).
>
> Somebody was against Geramy's patch as it didn't (and kind of doesn't
> even want to) ensure the unused siblings would be guaranteed to be
> allocated at least some space after the resize (any unused hotplug
> reservation is "optional" so it would be a major change to alter that
> differentiation to address that review comment).
>
> But if that patch works for you, sending your Tested-by tag to its thread
> might actually be helpful.
>
> [1] https://lore.kernel.org/linux-pci/20260428225146.3104063-1-gloveless@jqluv.com/
>
> --
>  i.

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2026-10-01  7:55 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-09-11  8:58 PCI: Resizable BAR never considered for Thunderbolt/USB4 hotplugged eGPUs 鳥井原茂
2026-09-11  9:08 ` Mika Westerberg
2026-09-29 12:17   ` Ilpo Järvinen
2026-10-01  7:55     ` 鳥井原茂

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox