[PATCH v2] PCI: Fix BAR resize for devices on a root bus

[PATCH v2] PCI: Fix BAR resize for devices on a root bus

[Liz Fong-Jones via B4 Relay]

From: Liz Fong-Jones <lizf@honeycomb.io>

pci_do_resource_release_and_resize() releases the device BARs that
share a bridge window with the BAR being resized, but when the device
sits directly on a root bus (pdev->bus->self == NULL) it then skips
resource assignment entirely and returns success, leaving the BARs it
just released unassigned (IORESOURCE_UNSET).

Skipping pbus_reassign_bridge_resources() is correct in that case --
there is no bridge window to adjust -- but the device BARs still have
to be reassigned. Before the BAR release was consolidated into the PCI
core, this case worked for amdgpu because the driver released the BARs
itself and then called pci_assign_unassigned_bus_resources()
unconditionally after the resize, which assigns unassigned device BARs
also on a root bus. Commit db92e3fef53e ("drm/amdgpu: Remove driver
side BAR release before resize") removed that call, so nothing assigns
the released BARs anymore.

This breaks amdgpu completely on the SolidRun HoneyComb LX2 (NXP
LX2160A, arm64, ACPI), where the GPU endpoint is enumerated directly
on the root bus of its segment (there is no root port device, so
pdev->bus->self is NULL):

  amdgpu 0004:01:00.0: BAR 0 [mem 0xa400000000-0xa40fffffff 64bit pref]: releasing
  amdgpu 0004:01:00.0: BAR 2 [mem 0xa410000000-0xa4101fffff 64bit pref]: releasing
  amdgpu 0004:01:00.0: sw_init of IP block <gmc_v8_0> failed -19
  amdgpu 0004:01:00.0: amdgpu_device_ip_init failed
  amdgpu 0004:01:00.0: Fatal error during GPU init

No error is logged because the resize path reports success; amdgpu
then finds BAR0 IORESOURCE_UNSET and bails out with -ENODEV.

Assign the released BARs directly from the root bus windows when there
is no upstream bridge. On failure, roll back through the existing
restore path exactly as in the bridged case.

The root bus path also had a locking bug that any fix here necessarily
touches: the old "goto out" jumped to up_read(&pci_bus_sem) without a
matching down_read() (as does the "goto restore" taken when
pci_dev_res_add_to_list() fails in the release loop). Take pci_bus_sem
before the BAR release loop so every path through the function holds
it exactly once.

Fixes: 337b1b566db0 ("PCI: Fix restoring BARs on BAR resize rollback path")
Cc: stable@vger.kernel.org
Link: https://bugs.launchpad.net/ubuntu/+source/linux-hwe-7.0/+bug/2159596
Assisted-by: Claude:claude-fable-5 checkpatch
Assisted-by: Claude:claude-sonnet-5
Signed-off-by: Liz Fong-Jones <lizf@honeycomb.io>
---
#regzbot introduced: 337b1b566db0

Observed at runtime on Ubuntu's linux-hwe-7.0 (7.0.0-14, broken) vs
linux-hwe-6.17 (working), but nothing here is distro-specific: Ubuntu
carries this code unmodified, and the affected function is identical
to current mainline. By source inspection the regression window is
v6.18 (old code paths) to v6.19 (consolidation). Workaround for
affected users: amdgpu.rebar=0.
---
Changes in v2:
- Add Assisted-by tags (missing from v1; required per
  Documentation/process/coding-assistants.rst)
- Add Link: to the corresponding Ubuntu bug report
- Drop the "# v6.19+" annotation on Cc: stable; unnecessary noise
  given the Fixes: tag already lets the stable team derive applicable
  versions (per stable-kernel-rules.rst)
- Link to v1: https://patch.msgid.link/20260705-pci-rebar-root-bus-v1-1-55df70cbdd88@honeycomb.io

To: Bjorn Helgaas <bhelgaas@google.com>
Cc: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Cc: linux-pci@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: regressions@lists.linux.dev
Cc: amd-gfx@lists.freedesktop.org
Cc: Jon Nettleton <jon@solid-run.com>
Cc: Jon Nettleton <jon.nettleton@gmail.com>
---
 drivers/pci/setup-bus.c | 25 +++++++++++++++++++------
 1 file changed, 19 insertions(+), 6 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index c0a949f2c995..9db1951f6e5c 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -2397,6 +2397,8 @@ int pci_do_resource_release_and_resize(struct pci_dev *pdev, int resno, int size
 	if (ret)
 		return ret;
 
+	down_read(&pci_bus_sem);
+
 	pci_dev_for_each_resource(pdev, r, i) {
 		if (i >= PCI_BRIDGE_RESOURCES)
 			break;
@@ -2415,13 +2417,24 @@ int pci_do_resource_release_and_resize(struct pci_dev *pdev, int resno, int size
 
 	pci_resize_resource_set_size(pdev, resno, size);
 
-	if (!bus->self)
-		goto out;
+	if (bus->self) {
+		ret = pbus_reassign_bridge_resources(bus, res, &saved);
+		if (ret)
+			goto restore;
+	} else {
+		/*
+		 * A device on a root bus has no bridge windows to adjust.
+		 * Assign the BARs released above directly from the root bus
+		 * windows.
+		 */
+		list_for_each_entry(dev_res, &saved, list) {
+			i = pci_resource_num(pdev, dev_res->res);
 
-	down_read(&pci_bus_sem);
-	ret = pbus_reassign_bridge_resources(bus, res, &saved);
-	if (ret)
-		goto restore;
+			ret = pci_assign_resource(pdev, i);
+			if (ret)
+				goto restore;
+		}
+	}
 
 out:
 	up_read(&pci_bus_sem);

---
base-commit: 7404ce51637231382873d0b55edabc2f3b841a9d
change-id: 20260704-pci-rebar-root-bus-f3123fe10fc7

Best regards,
--  
Liz Fong-Jones <lizf@honeycomb.io>

Re: [PATCH v2] PCI: Fix BAR resize for devices on a root bus

[sashiko-bot]

> From: Liz Fong-Jones <lizf@honeycomb.io>
> 
> pci_do_resource_release_and_resize() releases the device BARs that
> share a bridge window with the BAR being resized, but when the device
> sits directly on a root bus (pdev->bus->self == NULL) it then skips
> resource assignment entirely and returns success, leaving the BARs it
> just released unassigned (IORESOURCE_UNSET).
> 
> Skipping pbus_reassign_bridge_resources() is correct in that case --
> there is no bridge window to adjust -- but the device BARs still have
> to be reassigned. Before the BAR release was consolidated into the PCI
> core, this case worked for amdgpu because the driver released the BARs
> itself and then called pci_assign_unassigned_bus_resources()
> unconditionally after the resize, which assigns unassigned device BARs
> also on a root bus. Commit db92e3fef53e ("drm/amdgpu: Remove driver
> [ ... ]
> Cc: stable@vger.kernel.org
> Link: https://bugs.launchpad.net/ubuntu/+source/linux-hwe-7.0/+bug/2159596
> Assisted-by: Claude:claude-fable-5 checkpatch
> Assisted-by: Claude:claude-sonnet-5
> Signed-off-by: Liz Fong-Jones <lizf@honeycomb.io>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260712-pci-rebar-root-bus-v2-1-a1b9107a82dc@honeycomb.io?part=1

Re: [PATCH v2] PCI: Fix BAR resize for devices on a root bus

[Krzysztof Wilczyński]

Hello,

> Assign the released BARs directly from the root bus windows when there
> is no upstream bridge. On failure, roll back through the existing
> restore path exactly as in the bridged case.
> 
> The root bus path also had a locking bug that any fix here necessarily
> touches: the old "goto out" jumped to up_read(&pci_bus_sem) without a
> matching down_read() (as does the "goto restore" taken when
> pci_dev_res_add_to_list() fails in the release loop). Take pci_bus_sem
> before the BAR release loop so every path through the function holds
> it exactly once.

Thank you for also fixing this pci_bus_sem imbalance, appreciated.

With that,

  Reviewed-by: Krzysztof Wilczyński <kwilczynski@kernel.org>

Thank you!

	Krzysztof

Re: [PATCH v2] PCI: Fix BAR resize for devices on a root bus

[Bjorn Helgaas]

[cc->to: Ilpo, I think we need your review/ack for this]

On Sun, Jul 12, 2026 at 02:49:43PM -0700, Liz Fong-Jones via B4 Relay wrote:
> From: Liz Fong-Jones <lizf@honeycomb.io>
> 
> pci_do_resource_release_and_resize() releases the device BARs that
> share a bridge window with the BAR being resized, but when the device
> sits directly on a root bus (pdev->bus->self == NULL) it then skips
> resource assignment entirely and returns success, leaving the BARs it
> just released unassigned (IORESOURCE_UNSET).
> 
> Skipping pbus_reassign_bridge_resources() is correct in that case --
> there is no bridge window to adjust -- but the device BARs still have
> to be reassigned. Before the BAR release was consolidated into the PCI
> core, this case worked for amdgpu because the driver released the BARs
> itself and then called pci_assign_unassigned_bus_resources()
> unconditionally after the resize, which assigns unassigned device BARs
> also on a root bus. Commit db92e3fef53e ("drm/amdgpu: Remove driver
> side BAR release before resize") removed that call, so nothing assigns
> the released BARs anymore.
> 
> This breaks amdgpu completely on the SolidRun HoneyComb LX2 (NXP
> LX2160A, arm64, ACPI), where the GPU endpoint is enumerated directly
> on the root bus of its segment (there is no root port device, so
> pdev->bus->self is NULL):

Maybe this is a tangent, but I'm curious about this topology.  Is this
running as a virtualized guest?  At the hardware level, a PCIe
endpoint must have a Root Port somewhere, even if it isn't visible to
the OS (as is often the case in virtualized guests).

>   amdgpu 0004:01:00.0: BAR 0 [mem 0xa400000000-0xa40fffffff 64bit pref]: releasing
>   amdgpu 0004:01:00.0: BAR 2 [mem 0xa410000000-0xa4101fffff 64bit pref]: releasing
>   amdgpu 0004:01:00.0: sw_init of IP block <gmc_v8_0> failed -19
>   amdgpu 0004:01:00.0: amdgpu_device_ip_init failed
>   amdgpu 0004:01:00.0: Fatal error during GPU init
> 
> No error is logged because the resize path reports success; amdgpu
> then finds BAR0 IORESOURCE_UNSET and bails out with -ENODEV.
> 
> Assign the released BARs directly from the root bus windows when there
> is no upstream bridge. On failure, roll back through the existing
> restore path exactly as in the bridged case.
> 
> The root bus path also had a locking bug that any fix here necessarily
> touches: the old "goto out" jumped to up_read(&pci_bus_sem) without a
> matching down_read() (as does the "goto restore" taken when
> pci_dev_res_add_to_list() fails in the release loop). Take pci_bus_sem
> before the BAR release loop so every path through the function holds
> it exactly once.
> 
> Fixes: 337b1b566db0 ("PCI: Fix restoring BARs on BAR resize rollback path")
> Cc: stable@vger.kernel.org
> Link: https://bugs.launchpad.net/ubuntu/+source/linux-hwe-7.0/+bug/2159596
> Assisted-by: Claude:claude-fable-5 checkpatch
> Assisted-by: Claude:claude-sonnet-5
> Signed-off-by: Liz Fong-Jones <lizf@honeycomb.io>
> ---
> #regzbot introduced: 337b1b566db0
> 
> Observed at runtime on Ubuntu's linux-hwe-7.0 (7.0.0-14, broken) vs
> linux-hwe-6.17 (working), but nothing here is distro-specific: Ubuntu
> carries this code unmodified, and the affected function is identical
> to current mainline. By source inspection the regression window is
> v6.18 (old code paths) to v6.19 (consolidation). Workaround for
> affected users: amdgpu.rebar=0.
> ---
> Changes in v2:
> - Add Assisted-by tags (missing from v1; required per
>   Documentation/process/coding-assistants.rst)
> - Add Link: to the corresponding Ubuntu bug report
> - Drop the "# v6.19+" annotation on Cc: stable; unnecessary noise
>   given the Fixes: tag already lets the stable team derive applicable
>   versions (per stable-kernel-rules.rst)
> - Link to v1: https://patch.msgid.link/20260705-pci-rebar-root-bus-v1-1-55df70cbdd88@honeycomb.io
> 
> To: Bjorn Helgaas <bhelgaas@google.com>
> Cc: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
> Cc: linux-pci@vger.kernel.org
> Cc: linux-kernel@vger.kernel.org
> Cc: regressions@lists.linux.dev
> Cc: amd-gfx@lists.freedesktop.org
> Cc: Jon Nettleton <jon@solid-run.com>
> Cc: Jon Nettleton <jon.nettleton@gmail.com>
> ---
>  drivers/pci/setup-bus.c | 25 +++++++++++++++++++------
>  1 file changed, 19 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
> index c0a949f2c995..9db1951f6e5c 100644
> --- a/drivers/pci/setup-bus.c
> +++ b/drivers/pci/setup-bus.c
> @@ -2397,6 +2397,8 @@ int pci_do_resource_release_and_resize(struct pci_dev *pdev, int resno, int size
>  	if (ret)
>  		return ret;
>  
> +	down_read(&pci_bus_sem);
> +
>  	pci_dev_for_each_resource(pdev, r, i) {
>  		if (i >= PCI_BRIDGE_RESOURCES)
>  			break;
> @@ -2415,13 +2417,24 @@ int pci_do_resource_release_and_resize(struct pci_dev *pdev, int resno, int size
>  
>  	pci_resize_resource_set_size(pdev, resno, size);
>  
> -	if (!bus->self)
> -		goto out;
> +	if (bus->self) {

I think pci_upstream_bridge() would be better here.  Not that this
device will be a VF (where pci_upstream_bridge() makes a difference),
but just to be a little more generic.

> +		ret = pbus_reassign_bridge_resources(bus, res, &saved);
> +		if (ret)
> +			goto restore;
> +	} else {
> +		/*
> +		 * A device on a root bus has no bridge windows to adjust.
> +		 * Assign the BARs released above directly from the root bus
> +		 * windows.
> +		 */
> +		list_for_each_entry(dev_res, &saved, list) {
> +			i = pci_resource_num(pdev, dev_res->res);
>  
> -	down_read(&pci_bus_sem);
> -	ret = pbus_reassign_bridge_resources(bus, res, &saved);
> -	if (ret)
> -		goto restore;
> +			ret = pci_assign_resource(pdev, i);
> +			if (ret)
> +				goto restore;
> +		}
> +	}
>  
>  out:
>  	up_read(&pci_bus_sem);
> 
> ---
> base-commit: 7404ce51637231382873d0b55edabc2f3b841a9d
> change-id: 20260704-pci-rebar-root-bus-f3123fe10fc7
> 
> Best regards,
> --  
> Liz Fong-Jones <lizf@honeycomb.io>
> 
> 

Re: [PATCH v2] PCI: Fix BAR resize for devices on a root bus

[Bjorn Helgaas]

Liz Fong-Jones <lizf@honeycomb.io> wrote:
>>> This breaks amdgpu completely on the SolidRun HoneyComb LX2 (NXP
>>> LX2160A, arm64, ACPI), where the GPU endpoint is enumerated directly
>>> on the root bus of its segment (there is no root port device, so
>>> pdev->bus->self is NULL):

On Fri, Jul 31, 2026 at 12:15 PM Liz Fong-Jones <lizf@honeycomb.io> wrote:
>> Maybe this is a tangent, but I'm curious about this topology.  Is this
>> running as a virtualized guest?  At the hardware level, a PCIe
>> endpoint must have a Root Port somewhere, even if it isn't visible to
>> the OS (as is often the case in virtualized guests).
>
> Defer to Jon on what the actual hardware configuration is, but this is
> physical hardware, no virtualisation, running as the UEFI rather than U-Boot
> (devicetree) variant. See:
> https://dev.solid-run.com/nxp/lx2160a/com-som/lx2160a-com-hardware-user-manual#block-diagram

I guess this must be the drivers/pci/controller/mobiveil/pcie-layerscape-gen4.c
driver.  The commit log mentioned this:

>>>   amdgpu 0004:01:00.0: BAR 0 [mem 0xa400000000-0xa40fffffff 64bit pref]: releasing

so the GPU is on bus 01, which isn't a nice round bus number like root buses
usually are.  The typical thing would be a root port on bus 00 with a secondary
bus 01, and amdgpu being on that bus 01.  Can you share a dmesg log?  It
should include the lx2160a host bridge details and the PCIe topology below it.

Re: [PATCH v3] PCI: Fix BAR resize for devices on a root bus

[Liz Fong-Jones]

> I guess this must be the drivers/pci/controller/mobiveil/pcie-layerscape-gen4.c
> driver.

Close, but not in this configuration -- that driver binds on the
U-Boot/devicetree variant of this platform. This machine runs the
UEFI/ACPI firmware variant (EDK II), where each PCIe controller is
exposed as a plain PNP0A08 host bridge with standard ECAM (MCFG OEM
"NXP LX2160", 2 entries, one per segment), so it goes through the
generic ACPI ECAM path with no controller-specific driver and no MCFG
quirk.

> so the GPU is on bus 01, which isn't a nice round bus number like
> root buses usually are.  The typical thing would be a root port on
> bus 00 with a secondary bus 01, and amdgpu being on that bus 01.

The firmware genuinely declares the root bus as 01: the host bridge
_CRS bus range is [bus 01-ff], there is no bus 00 in the segment at
all, and no root port device is visible anywhere -- the endpoint is
enumerated directly on the root bus. Both segments follow the same
pattern (NVMe on 0002:01:00.0, GPU on 0004:01:00.0). I'll defer to Jon
(CC'd) on why the firmware hides the root port, but the OS-visible
topology is unambiguous.

> Can you share a dmesg log?

Relevant excerpt below; happy to send the full log off-list or attach
to the Launchpad bug. Note this boot is running with the
amdgpu.rebar=0 workaround on a 7.0.0-28 distro kernel, so amdgpu
initializes successfully (on the native 256MB BAR) -- the host bridge
and topology enumeration is identical on the boots that fail.

  ACPI: MCFG 0x00000000EBE28E98 00004C (v01 NXP    LX2160   000000FF INTL 20151124)
  ...
  ACPI: MCFG table detected, 2 entries
  ...
  ACPI: PCI Root Bridge [PCI0] (domain 0002 [bus 01-ff])
  acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI EDR HPX-Type3]
  acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug SHPCHotplug PME AER PCIeCapability LTR DPC]
  acpi PNP0A08:00: ECAM area [mem 0x9000100000-0x900fffffff] reserved by PNP0C02:00
  acpi PNP0A08:00: ECAM at [mem 0x9000100000-0x900fffffff] for [bus 01-ff]
  PCI host bridge to bus 0002:01
  pci_bus 0002:01: root bus resource [mem 0x9040000000-0x90ffffffff window] (bus address [0x40000000-0xffffffff])
  pci_bus 0002:01: root bus resource [mem 0x9400000000-0x97ffffffff window]
  pci_bus 0002:01: root bus resource [io  0x0000-0xffff window]
  pci_bus 0002:01: root bus resource [bus 01-ff]
  pci 0002:01:00.0: [144d:a808] type 00 class 0x010802 PCIe Endpoint
  pci 0002:01:00.0: BAR 0 [mem 0x9400000000-0x9400003fff 64bit]
  pci 0002:01:00.0: BAR 0 [mem 0x9400000000-0x9400003fff 64bit]: assigned
  ACPI: PCI Root Bridge [PCI1] (domain 0004 [bus 01-ff])
  acpi PNP0A08:01: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI EDR HPX-Type3]
  acpi PNP0A08:01: _OSC: OS now controls [PCIeHotplug SHPCHotplug PME AER PCIeCapability LTR DPC]
  acpi PNP0A08:01: ECAM area [mem 0xa000100000-0xa00fffffff] reserved by PNP0C02:00
  acpi PNP0A08:01: ECAM at [mem 0xa000100000-0xa00fffffff] for [bus 01-ff]
  PCI host bridge to bus 0004:01
  pci_bus 0004:01: root bus resource [mem 0xa040000000-0xa0ffffffff window] (bus address [0x40000000-0xffffffff])
  pci_bus 0004:01: root bus resource [mem 0xa400000000-0xa7ffffffff window]
  pci_bus 0004:01: root bus resource [io  0x10000-0x1ffff window] (bus address [0x0000-0xffff])
  pci_bus 0004:01: root bus resource [bus 01-ff]
  pci 0004:01:00.0: [1002:6981] type 00 class 0x030000 PCIe Legacy Endpoint
  pci 0004:01:00.0: BAR 0 [mem 0xa400000000-0xa40fffffff 64bit pref]
  pci 0004:01:00.0: BAR 2 [mem 0xa410000000-0xa4101fffff 64bit pref]
  pci 0004:01:00.0: BAR 4 [io  0x10000-0x100ff]
  pci 0004:01:00.0: BAR 5 [mem 0xa046000000-0xa04603ffff]
  pci 0004:01:00.0: ROM [mem 0xa0fffe0000-0xa0ffffffff pref]
  pci 0004:01:00.1: [1002:aae0] type 00 class 0x040300 PCIe Legacy Endpoint
  pci 0004:01:00.1: BAR 0 [mem 0xa410200000-0xa410203fff 64bit]
  pci 0004:01:00.0: BAR 0 [mem 0xa400000000-0xa40fffffff 64bit pref]: assigned
  pci 0004:01:00.0: BAR 2 [mem 0xa410000000-0xa4101fffff 64bit pref]: assigned
  pci 0004:01:00.0: BAR 5 [mem 0xa040000000-0xa04003ffff]: assigned
  pci 0004:01:00.0: ROM [mem 0xa040040000-0xa04005ffff pref]: assigned
  pci 0004:01:00.1: BAR 0 [mem 0xa410200000-0xa410203fff 64bit]: assigned
  pci 0004:01:00.0: BAR 4 [io  0x10000-0x100ff]: assigned
  pci_bus 0004:01: resource 4 [mem 0xa040000000-0xa0ffffffff window]
  pci_bus 0004:01: resource 5 [mem 0xa400000000-0xa7ffffffff window]
  pci_bus 0004:01: resource 6 [io  0x10000-0x1ffff window]