[PATCH] PCI: release empty sibling resources during bridge window resize

[PATCH] PCI: release empty sibling resources during bridge window resize

[Geramy Loveless]

When pci_resize_resource() walks up the bridge hierarchy via
pbus_reassign_bridge_resources(), bridge windows with any child
resources are refused release. This prevents BAR resize on devices
behind multi-port PCIe switches (such as Thunderbolt docks) where
empty sibling downstream ports hold small reservations that block the
parent bridge window from being freed and re-sized.

Add pci_bus_release_bridge_resources_safe() which verifies that a
resource subtree contains no active children before releasing it,
and use it in pbus_reassign_bridge_resources() to clear empty sibling
reservations so the bridge window can grow.

Signed-off-by: Geramy Loveless <gloveless@jqluv.com>
---
drivers/pci/setup-bus.c | 30 +++++++++++++++++++++++++++---
1 file changed, 27 insertions(+), 3 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 4cf120ebe..9a3a23819 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -2297,6 +2297,30 @@
EXPORT_SYMBOL_GPL(pci_assign_unassigned_bridge_resources);
* release it when possible. If the bridge window contains assigned
* resources, it cannot be released.
*/
+
+/*
+ * Release child resources from a bridge window so it can be freed and
+ * re-sized, but only if the entire subtree is empty (no active device
+ * resources underneath). Walks the resource tree recursively to handle
+ * arbitrarily deep bridge hierarchies. Returns true if the resources
+ * were released or the window was already empty.
+ */
+static bool pci_bus_release_bridge_resources_safe(struct pci_bus *bus,
+ struct resource *res)
+{
+ struct resource *child;
+
+ for (child = res->child; child; child = child->sibling) {
+ if (child->child &&
+ !pci_bus_release_bridge_resources_safe(bus, child))
+ return false;
+ }
+
+ if (res->child)
+ pci_bus_release_bridge_resources(bus, res, whole_subtree);
+ return true;
+}
+
static int pbus_reassign_bridge_resources(struct pci_bus *bus, struct
resource *res,
struct list_head *saved)
{
@@ -2316,8 +2340,8 @@ static int pbus_reassign_bridge_resources(struct
pci_bus *bus, struct resource *
i = pci_resource_num(bridge, res);
- /* Ignore BARs which are still in use */
- if (!res->child) {
+ if (!res->child ||
+ pci_bus_release_bridge_resources_safe(bridge->subordinate, res)) {
ret = pci_dev_res_add_to_list(saved, bridge, res, 0, 0);
if (ret)
return ret;
@@ -2327,7 +2351,7 @@ static int pbus_reassign_bridge_resources(struct
pci_bus *bus, struct resource *
const char *res_name = pci_resource_name(bridge, i);
pci_warn(bridge,
- "%s %pR: was not released (still contains assigned resources)\n",
+ "%s %pR: not released, active children present\n",
res_name, res);
}
-- 
2.51.0

Re: [PATCH] PCI: release empty sibling resources during bridge window resize

[Ilpo Järvinen]

On Wed, 8 Apr 2026, Geramy Loveless wrote:

Thanks for the patch. The patch need some work but the general direction 
looks acceptable.

FYI, your patch is misformatted, tabs got eaten, probably something in 
tha way you send the email does that.

Please Cc also Cristian Cocos <cristi@ieee.org> on further submissions.

> When pci_resize_resource() walks up the bridge hierarchy via
> pbus_reassign_bridge_resources(), bridge windows with any child
> resources are refused release. This prevents BAR resize on devices
> behind multi-port PCIe switches (such as Thunderbolt docks) where
> empty sibling downstream ports hold small reservations that block the
> parent bridge window from being freed and re-sized.
> 
> Add pci_bus_release_bridge_resources_safe() which verifies that a
> resource subtree contains no active children before releasing it,
> and use it in pbus_reassign_bridge_resources() to clear empty sibling
> reservations so the bridge window can grow.
> 
> Signed-off-by: Geramy Loveless <gloveless@jqluv.com>

You may want to add Suggested-by tag.

> ---
> drivers/pci/setup-bus.c | 30 +++++++++++++++++++++++++++---
> 1 file changed, 27 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
> index 4cf120ebe..9a3a23819 100644
> --- a/drivers/pci/setup-bus.c
> +++ b/drivers/pci/setup-bus.c
> @@ -2297,6 +2297,30 @@
> EXPORT_SYMBOL_GPL(pci_assign_unassigned_bridge_resources);
> * release it when possible. If the bridge window contains assigned
> * resources, it cannot be released.
> */
> +
> +/*
> + * Release child resources from a bridge window so it can be freed and
> + * re-sized, but only if the entire subtree is empty (no active device
> + * resources underneath). Walks the resource tree recursively to handle
> + * arbitrarily deep bridge hierarchies. Returns true if the resources
> + * were released or the window was already empty.
> + */
> +static bool pci_bus_release_bridge_resources_safe(struct pci_bus *bus,
> + struct resource *res)
> +{
> + struct resource *child;
> +
> + for (child = res->child; child; child = child->sibling) {
> + if (child->child &&
> + !pci_bus_release_bridge_resources_safe(bus, child))
> + return false;

This should walk bus devices and device resources using PCI side
iterators (not using the low-level resource list). This does not look 
safe iteration anyway because the code is also removing entries from the 
resource tree (it only works because resource code leaves stale sibling 
pointers which should be corrected one day).

PCI side should be enough to capture all resources, and that way you 
still have access to all PCI side structs.

Perhaps sanity checking ! ->child at the end wouldn't hurt but it the
release should be doable.

You should also pass the saved list and store any released resources so 
you can properly rollback in case of a failure (I think the rollback 
code will already just-work as long as you add the entries there).

> + }
> +
> + if (res->child)
> + pci_bus_release_bridge_resources(bus, res, whole_subtree);

I think this could pass non-bridge window resources which looks wrong but 
you need to rewrite the algorithm in this function anyway.

> + return true;
> +}
> +
> static int pbus_reassign_bridge_resources(struct pci_bus *bus, struct
> resource *res,
> struct list_head *saved)
> {
> @@ -2316,8 +2340,8 @@ static int pbus_reassign_bridge_resources(struct
> pci_bus *bus, struct resource *
> i = pci_resource_num(bridge, res);
> - /* Ignore BARs which are still in use */
> - if (!res->child) {
> + if (!res->child ||
> + pci_bus_release_bridge_resources_safe(bridge->subordinate, res)) {

Just call this release always before checking !res->child.

You need to check bridge->subordinate before calling it.

> ret = pci_dev_res_add_to_list(saved, bridge, res, 0, 0);
> if (ret)
> return ret;
> @@ -2327,7 +2351,7 @@ static int pbus_reassign_bridge_resources(struct
> pci_bus *bus, struct resource *
> const char *res_name = pci_resource_name(bridge, i);
> pci_warn(bridge,
> - "%s %pR: was not released (still contains assigned resources)\n",
> + "%s %pR: not released, active children present\n",
> res_name, res);
> }
> 

-- 
 i.

Re: [PATCH] PCI: release empty sibling resources during bridge window resize

[Ilpo Järvinen]

[-- Attachment #1: Type: text/plain, Size: 5983 bytes --]

On Thu, 9 Apr 2026, Geramy Loveless wrote:

> Perfect I’ll get started on these changes, while I’m here, does the other bug create
> a cascade of problems if fixed? 

Hi,

I'm not sure what "other bug" refers to?

If you meant Cristian problem, he seemed to have empty bridge windows 
which this approach should be able to solve without extra problems 
expected.

If you refer to the resource entries having dangling resource tree 
pointers (sibling, etc.), it's not a big problem when the resource entry 
has been detached from the tree. It has some impact on whether 
resource_assigned() can always be trusted so it would be nice to clear all 
those links on removal (the current remove algorithm shortcuts the 
subtree).

-- 
 i.

> On Thu, Apr 9, 2026 at 1:03 AM Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> wrote:
>       On Wed, 8 Apr 2026, Geramy Loveless wrote:
> 
>       Thanks for the patch. The patch need some work but the general direction
>       looks acceptable.
> 
>       FYI, your patch is misformatted, tabs got eaten, probably something in
>       tha way you send the email does that.
> 
>       Please Cc also Cristian Cocos <cristi@ieee.org> on further submissions.
> 
>       > When pci_resize_resource() walks up the bridge hierarchy via
>       > pbus_reassign_bridge_resources(), bridge windows with any child
>       > resources are refused release. This prevents BAR resize on devices
>       > behind multi-port PCIe switches (such as Thunderbolt docks) where
>       > empty sibling downstream ports hold small reservations that block the
>       > parent bridge window from being freed and re-sized.
>       >
>       > Add pci_bus_release_bridge_resources_safe() which verifies that a
>       > resource subtree contains no active children before releasing it,
>       > and use it in pbus_reassign_bridge_resources() to clear empty sibling
>       > reservations so the bridge window can grow.
>       >
>       > Signed-off-by: Geramy Loveless <gloveless@jqluv.com>
> 
>       You may want to add Suggested-by tag.
> 
>       > ---
>       > drivers/pci/setup-bus.c | 30 +++++++++++++++++++++++++++---
>       > 1 file changed, 27 insertions(+), 3 deletions(-)
>       >
>       > diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
>       > index 4cf120ebe..9a3a23819 100644
>       > --- a/drivers/pci/setup-bus.c
>       > +++ b/drivers/pci/setup-bus.c
>       > @@ -2297,6 +2297,30 @@
>       > EXPORT_SYMBOL_GPL(pci_assign_unassigned_bridge_resources);
>       > * release it when possible. If the bridge window contains assigned
>       > * resources, it cannot be released.
>       > */
>       > +
>       > +/*
>       > + * Release child resources from a bridge window so it can be freed and
>       > + * re-sized, but only if the entire subtree is empty (no active device
>       > + * resources underneath). Walks the resource tree recursively to
>       handle
>       > + * arbitrarily deep bridge hierarchies. Returns true if the resources
>       > + * were released or the window was already empty.
>       > + */
>       > +static bool pci_bus_release_bridge_resources_safe(struct pci_bus *bus,
>       > + struct resource *res)
>       > +{
>       > + struct resource *child;
>       > +
>       > + for (child = res->child; child; child = child->sibling) {
>       > + if (child->child &&
>       > + !pci_bus_release_bridge_resources_safe(bus, child))
>       > + return false;
> 
>       This should walk bus devices and device resources using PCI side
>       iterators (not using the low-level resource list). This does not look
>       safe iteration anyway because the code is also removing entries from the
>       resource tree (it only works because resource code leaves stale sibling
>       pointers which should be corrected one day).
> 
>       PCI side should be enough to capture all resources, and that way you
>       still have access to all PCI side structs.
> 
>       Perhaps sanity checking ! ->child at the end wouldn't hurt but it the
>       release should be doable.
> 
>       You should also pass the saved list and store any released resources so
>       you can properly rollback in case of a failure (I think the rollback
>       code will already just-work as long as you add the entries there).
> 
>       > + }
>       > +
>       > + if (res->child)
>       > + pci_bus_release_bridge_resources(bus, res, whole_subtree);
> 
>       I think this could pass non-bridge window resources which looks wrong but
>       you need to rewrite the algorithm in this function anyway.
> 
>       > + return true;
>       > +}
>       > +
>       > static int pbus_reassign_bridge_resources(struct pci_bus *bus, struct
>       > resource *res,
>       > struct list_head *saved)
>       > {
>       > @@ -2316,8 +2340,8 @@ static int pbus_reassign_bridge_resources(struct
>       > pci_bus *bus, struct resource *
>       > i = pci_resource_num(bridge, res);
>       > - /* Ignore BARs which are still in use */
>       > - if (!res->child) {
>       > + if (!res->child ||
>       > + pci_bus_release_bridge_resources_safe(bridge->subordinate, res)) {
> 
>       Just call this release always before checking !res->child.
> 
>       You need to check bridge->subordinate before calling it.
> 
>       > ret = pci_dev_res_add_to_list(saved, bridge, res, 0, 0);
>       > if (ret)
>       > return ret;
>       > @@ -2327,7 +2351,7 @@ static int pbus_reassign_bridge_resources(struct
>       > pci_bus *bus, struct resource *
>       > const char *res_name = pci_resource_name(bridge, i);
>       > pci_warn(bridge,
>       > - "%s %pR: was not released (still contains assigned resources)\n",
>       > + "%s %pR: not released, active children present\n",
>       > res_name, res);
>       > }
>       >
> 
>       --
>        i.
> 
> 
> 

Re: [PATCH] PCI: release empty sibling resources during bridge window resize

[Cristian Cocos]

This looks very encouraging!

Thanks for keeping me in the loop!

C

On Thu, 2026-04-09 at 16:26 +0300, Ilpo Järvinen wrote:
> On Thu, 9 Apr 2026, Geramy Loveless wrote:
> 
> > Perfect I’ll get started on these changes, while I’m here, does the
> > other bug create
> > a cascade of problems if fixed? 
> 
> Hi,
> 
> I'm not sure what "other bug" refers to?
> 
> If you meant Cristian problem, he seemed to have empty bridge windows
> which this approach should be able to solve without extra problems 
> expected.
> 
> If you refer to the resource entries having dangling resource tree 
> pointers (sibling, etc.), it's not a big problem when the resource
> entry 
> has been detached from the tree. It has some impact on whether 
> resource_assigned() can always be trusted so it would be nice to
> clear all 
> those links on removal (the current remove algorithm shortcuts the 
> subtree).

[PATCH v2] PCI: release empty sibling bridge windows during rebar expansion

[Geramy Loveless]

When pbus_reassign_bridge_resources() walks up the bridge hierarchy
to expand a window (e.g. for resizable BAR), it refuses to release
any bridge window that has children.  This prevents BAR resize on
devices behind multi-port PCIe switches (such as Thunderbolt docks)
where empty sibling downstream ports hold small reservations that
block the parent bridge window from being freed and re-sized.

Add pci_bus_subtree_empty() to check whether a bus subtree contains
any assigned device BARs, and pci_bus_release_empty_bridges() to
release bridge window resources of empty sibling bridges, saving
them to the rollback list so failures can be properly unwound.

In pbus_reassign_bridge_resources(), call pci_bus_release_empty_bridges()
before checking res->child, so empty sibling windows are cleared first
and the parent window can then be released and grown.

Uses PCI bus/device iterators rather than walking the raw resource
tree, which avoids issues with stale sibling pointers after resource
release.

Suggested-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Signed-off-by: Geramy Loveless <gloveless@jqluv.com>
---
 drivers/pci/setup-bus.c | 99 ++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 97 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 4cf120ebe5a..7a182cd7e4d 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -2292,6 +2292,94 @@ void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge)
 }
 EXPORT_SYMBOL_GPL(pci_assign_unassigned_bridge_resources);
 
+/*
+ * pci_bus_subtree_empty - Check whether a bus subtree has any assigned
+ * non-bridge device resources.
+ * @bus: PCI bus to check
+ *
+ * Returns true if no device on @bus or its descendant buses has any
+ * assigned BARs (bridge window resources are not considered).
+ */
+static bool pci_bus_subtree_empty(struct pci_bus *bus)
+{
+	struct pci_dev *dev;
+
+	list_for_each_entry(dev, &bus->devices, bus_list) {
+		struct resource *r;
+		unsigned int i;
+
+		pci_dev_for_each_resource(dev, r, i) {
+			if (i >= PCI_BRIDGE_RESOURCES)
+				break;
+			if (resource_assigned(r))
+				return false;
+		}
+
+		if (dev->subordinate &&
+		    !pci_bus_subtree_empty(dev->subordinate))
+			return false;
+	}
+
+	return true;
+}
+
+/*
+ * pci_bus_release_empty_bridges - Release bridge window resources of
+ * empty sibling bridges so the parent window can be freed and re-sized.
+ * @bus: PCI bus whose child bridges to scan
+ * @b_win: Parent bridge window resource; only children of this window
+ *         are released
+ * @saved: List to save released resources for rollback
+ *
+ * For each PCI-to-PCI bridge on @bus whose subtree is empty (no assigned
+ * device BARs), releases bridge window resources that are children of
+ * @b_win, saving them for rollback via @saved.
+ *
+ * Returns 0 on success, negative errno on failure.
+ */
+static int pci_bus_release_empty_bridges(struct pci_bus *bus,
+					 struct resource *b_win,
+					 struct list_head *saved)
+{
+	struct pci_dev *dev;
+
+	list_for_each_entry(dev, &bus->devices, bus_list) {
+		struct resource *r;
+		unsigned int i;
+
+		if (!dev->subordinate)
+			continue;
+
+		if ((dev->class >> 8) != PCI_CLASS_BRIDGE_PCI)
+			continue;
+
+		if (!pci_bus_subtree_empty(dev->subordinate))
+			continue;
+
+		pci_dev_for_each_resource(dev, r, i) {
+			int ret;
+
+			if (!pci_resource_is_bridge_win(i))
+				continue;
+
+			if (!resource_assigned(r))
+				continue;
+
+			if (r->parent != b_win)
+				continue;
+
+			ret = pci_dev_res_add_to_list(saved, dev, r, 0, 0);
+			if (ret)
+				return ret;
+
+			release_child_resources(r);
+			pci_release_resource(dev, i);
+		}
+	}
+
+	return 0;
+}
+
 /*
  * Walk to the root bus, find the bridge window relevant for @res and
  * release it when possible. If the bridge window contains assigned
@@ -2316,7 +2404,14 @@ static int pbus_reassign_bridge_resources(struct pci_bus *bus, struct resource *
 
 		i = pci_resource_num(bridge, res);
 
-		/* Ignore BARs which are still in use */
+		/* Release empty sibling bridge windows first */
+		if (bridge->subordinate) {
+			ret = pci_bus_release_empty_bridges(
+					bridge->subordinate, res, saved);
+			if (ret)
+				return ret;
+		}
+
 		if (!res->child) {
 			ret = pci_dev_res_add_to_list(saved, bridge, res, 0, 0);
 			if (ret)
@@ -2327,7 +2422,7 @@ static int pbus_reassign_bridge_resources(struct pci_bus *bus, struct resource *
 			const char *res_name = pci_resource_name(bridge, i);
 
 			pci_warn(bridge,
-				 "%s %pR: was not released (still contains assigned resources)\n",
+				 "%s %pR: not released, active children present\n",
 				 res_name, res);
 		}
 
-- 
2.43.0

Re: [PATCH v2] PCI: release empty sibling bridge windows during rebar expansion

[Ilpo Järvinen]

[-- Attachment #1: Type: text/plain, Size: 7314 bytes --]

On Thu, 9 Apr 2026, Geramy Loveless wrote:

> When pbus_reassign_bridge_resources() walks up the bridge hierarchy
> to expand a window (e.g. for resizable BAR), it refuses to release
> any bridge window that has children.  This prevents BAR resize on
> devices behind multi-port PCIe switches (such as Thunderbolt docks)
> where empty sibling downstream ports hold small reservations that
> block the parent bridge window from being freed and re-sized.
> 
> Add pci_bus_subtree_empty() to check whether a bus subtree contains
> any assigned device BARs, and pci_bus_release_empty_bridges() to
> release bridge window resources of empty sibling bridges, saving
> them to the rollback list so failures can be properly unwound.
> 
> In pbus_reassign_bridge_resources(), call pci_bus_release_empty_bridges()
> before checking res->child, so empty sibling windows are cleared first
> and the parent window can then be released and grown.
> 
> Uses PCI bus/device iterators rather than walking the raw resource
> tree, which avoids issues with stale sibling pointers after resource
> release.

This paragraph can be dropped. And it's not exactly correct either as 
the pointers are only stale for resource entries that reside outside of 
the resource tree (after they've been released in a specific way) so if 
you start from a resource tree entry, you should never encounter a stale 
pointer.

> Suggested-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
> Signed-off-by: Geramy Loveless <gloveless@jqluv.com>
> ---
>  drivers/pci/setup-bus.c | 99 ++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 97 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
> index 4cf120ebe5a..7a182cd7e4d 100644
> --- a/drivers/pci/setup-bus.c
> +++ b/drivers/pci/setup-bus.c
> @@ -2292,6 +2292,94 @@ void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge)
>  }
>  EXPORT_SYMBOL_GPL(pci_assign_unassigned_bridge_resources);
>  
> +/*
> + * pci_bus_subtree_empty - Check whether a bus subtree has any assigned
> + * non-bridge device resources.
> + * @bus: PCI bus to check
> + *
> + * Returns true if no device on @bus or its descendant buses has any
> + * assigned BARs (bridge window resources are not considered).
> + */
> +static bool pci_bus_subtree_empty(struct pci_bus *bus)
> +{
> +	struct pci_dev *dev;
> +
> +	list_for_each_entry(dev, &bus->devices, bus_list) {
> +		struct resource *r;
> +		unsigned int i;
> +
> +		pci_dev_for_each_resource(dev, r, i) {
> +			if (i >= PCI_BRIDGE_RESOURCES)
> +				break;
> +			if (resource_assigned(r))
> +				return false;
> +		}
> +
> +		if (dev->subordinate &&
> +		    !pci_bus_subtree_empty(dev->subordinate))
> +			return false;
> +	}
> +
> +	return true;
> +}
> +
> +/*
> + * pci_bus_release_empty_bridges - Release bridge window resources of
> + * empty sibling bridges so the parent window can be freed and re-sized.
> + * @bus: PCI bus whose child bridges to scan
> + * @b_win: Parent bridge window resource; only children of this window
> + *         are released
> + * @saved: List to save released resources for rollback
> + *
> + * For each PCI-to-PCI bridge on @bus whose subtree is empty (no assigned
> + * device BARs), releases bridge window resources that are children of
> + * @b_win, saving them for rollback via @saved.
> + *
> + * Returns 0 on success, negative errno on failure.
> + */
> +static int pci_bus_release_empty_bridges(struct pci_bus *bus,
> +					 struct resource *b_win,
> +					 struct list_head *saved)
> +{
> +	struct pci_dev *dev;
> +
> +	list_for_each_entry(dev, &bus->devices, bus_list) {
> +		struct resource *r;
> +		unsigned int i;
> +
> +		if (!dev->subordinate)
> +			continue;
> +
> +		if ((dev->class >> 8) != PCI_CLASS_BRIDGE_PCI)
> +			continue;

I suppose dev->subordinate check is enough for what we're doing so this 
looks redundant.

> +
> +		if (!pci_bus_subtree_empty(dev->subordinate))
> +			continue;
> +
> +		pci_dev_for_each_resource(dev, r, i) {
> +			int ret;
> +
> +			if (!pci_resource_is_bridge_win(i))
> +				continue;
> +
> +			if (!resource_assigned(r))
> +				continue;
> +
> +			if (r->parent != b_win)
> +				continue;
> +
> +			ret = pci_dev_res_add_to_list(saved, dev, r, 0, 0);
> +			if (ret)
> +				return ret;
> +
> +			release_child_resources(r);

Unfortunately you cannot call this low-level function because it 
recursively frees child resources which means you won't be able to 
rollback them as they were not added to the saved list.

I think the release algorithm should basically do this:

- Recurse to the subordinate buses
- Loop through bridge window resources of this bus
	- Skip resources that are not assigned or are not parented by b_win
	- If the resource still has childs, leave the resource alone
	  (+ log it for easier troubleshooting these cases; any failure
	     will also cascade to upstream so it may be possible to 
	     shortcut something but it will also make the algorithm more
	     complicated)
	- Save and free the resource

It might be better to move some of the code from 
pbus_reassign_bridge_resources() here as there's overlap with the sketched 
algorithm (but I'm not sure until I see the updated version but keep this 
in mind).

Doing pci_bus_subtree_empty() before any removal is fine with me, but I 
see it just an optimization.

> +			pci_release_resource(dev, i);
> +		}
> +	}
> +
> +	return 0;
> +}
> +
>  /*
>   * Walk to the root bus, find the bridge window relevant for @res and
>   * release it when possible. If the bridge window contains assigned
> @@ -2316,7 +2404,14 @@ static int pbus_reassign_bridge_resources(struct pci_bus *bus, struct resource *
>  
>  		i = pci_resource_num(bridge, res);
>  
> -		/* Ignore BARs which are still in use */

I don't know why you removed this comment (I admit though "BARs" could 
have been worded better as it's bridge windows we're dealing here).

> +		/* Release empty sibling bridge windows first */
> +		if (bridge->subordinate) {
> +			ret = pci_bus_release_empty_bridges(
> +					bridge->subordinate, res, saved);

First arg fits to the previous line.

Align the second line to (.

But consider also rearranging code as I mentioned above.

> +			if (ret)
> +				return ret;

Consider proceeding with the resize even if something failed as there are 
cases where the bridge windows are large enough (admittedly, you seem to 
only bail out in case of alloc error).

In to the same vein, there seems to be one existing goto restore (that was 
added by me), which could also probably do continue instead (but changing 
it would be worth another patch).

> +		}
> +
>  		if (!res->child) {
>  			ret = pci_dev_res_add_to_list(saved, bridge, res, 0, 0);
>  			if (ret)
> @@ -2327,7 +2422,7 @@ static int pbus_reassign_bridge_resources(struct pci_bus *bus, struct resource *
>  			const char *res_name = pci_resource_name(bridge, i);
>  
>  			pci_warn(bridge,
> -				 "%s %pR: was not released (still contains assigned resources)\n",
> +				 "%s %pR: not released, active children present\n",
>  				 res_name, res);
>  		}
>  
> 

-- 
 i.

Re: [PATCH v2] PCI: release empty sibling bridge windows during rebar expansion

[Geramy Loveless]

 I'm going to loop in Christian Koenig over at AMD he has been working
with me on resolving or attempting to figure out whats going on with
my gfx1201 connected to a tb5 dock to the host.
I am currently having problems with the GPU basically loosing MMIO and
crashing randomly. This recent patch change I believe helped but its
really hard to say at this point.
Without this patch of course the bar size would be 256MB and cause
huge performance problems or feature loss. I am able to load up AI
models and run workloads at nearly 100% gpu usage, i'm seeing 205W
power draw out of the maximum 300W. But after sustained load I still
get a crash.

Maybe you would have an idea as to what is causing that crash or where
I should be looking to find the cause?
Here are some relevant logs, from what I can tell something is going
on with MMIO, but the config bar as i understand it is still alive.
This let me to believe maybe the router was getting put into suspend
mode which wouldnt make sense for a GPU that is active and busy
because the pcie tunnel would be active.

Any advice or tips would be helpful thank you for the suggestions I
will get started on writing the patch based on those recommendations.

## SMU Firmware Version

```
smu driver if version = 0x0000002e
smu fw if version = 0x00000032
smu fw program = 0
smu fw version = 0x00684b00 (104.75.0)
```

Note: Driver interface version (0x2e / 46) does not match firmware
interface version (0x32 / 50).

## PCI Topology

```
65:00.0 PCI bridge: Intel Barlow Ridge Host 80G (rev 84)
66:00.0 PCI bridge: Intel Barlow Ridge Host 80G (rev 84) → NHI
66:01.0 PCI bridge: Intel Barlow Ridge Host 80G (rev 84) → empty hotplug port
66:02.0 PCI bridge: Intel Barlow Ridge Host 80G (rev 84) → USB
66:03.0 PCI bridge: Intel Barlow Ridge Host 80G (rev 84) → dock
93:00.0 PCI bridge: Intel Barlow Ridge Hub 80G (rev 85) → dock switch
94:00.0 PCI bridge: Intel Barlow Ridge Hub 80G (rev 85) → downstream
95:00.0 PCI bridge: AMD Navi 10 XL Upstream Port (rev 24)
96:00.0 PCI bridge: AMD Navi 10 XL Downstream Port (rev 24)
97:00.0 VGA: AMD [1002:7551] (rev c0) ← GPU
97:00.1 Audio: AMD [1002:ab40]
```

## Workload

GPU compute via llama.cpp (ROCm/HIP backend), running
Qwen3.5-35B-A3B-Q4_K_M.gguf model (20.49 GiB, fully offloaded to
VRAM). Flash attention enabled, 128K context, 32 threads.

## Crash Timeline

All timestamps from `dmesg -T`, kernel boot-relative times in brackets.

### GPU initialization (successful)

```
[603.644s] GPU probe: IP DISCOVERY 0x1002:0x7551
[603.653s] Detected IP block: smu_v14_0_0, gfx_v12_0_0
[603.771s] Detected VRAM RAM=32624M, BAR=32768M, RAM width 256bits GDDR6
[604.014s] SMU driver IF 0x2e, FW IF 0x32, FW version 104.75.0
[604.049s] SMU is initialized successfully!
[604.119s] Runtime PM manually disabled (amdgpu.runpm=0)
[604.119s] Initialized amdgpu 3.64.0 for 0000:97:00.0
```

### SMU stops responding [T+4238s after init, ~70 minutes]

```
[4841.828s] SMU: No response msg_reg: 12 resp_reg: 0
[4841.828s] [smu_v14_0_2_get_power_profile_mode] Failed to get activity monitor!
[4849.393s] SMU: No response msg_reg: 12 resp_reg: 0
[4849.393s] Failed to export SMU metrics table!
```

15 consecutive `SMU: No response` messages logged between [4841s] and
[4948s], approximately every 7-8 seconds. All with `msg_reg: 12
resp_reg: 0`. Failed operations include:
- `smu_v14_0_2_get_power_profile_mode` — Failed to get activity monitor
- `Failed to export SMU metrics table`
- `Failed to get current clock freq`

### Page faults begin [T+4349s after init, ~111s after first SMU failure]

```
[4948.927s] [gfxhub] page fault (src_id:0 ring:40 vmid:9 pasid:108)
Process llama-cli pid 35632
GCVM_L2_PROTECTION_FAULT_STATUS: 0x00941051
Faulty UTCL2 client ID: TCP (0x8)
PERMISSION_FAULTS: 0x5
WALKER_ERROR: 0x0
MAPPING_ERROR: 0x0
RW: 0x1 (write)
```

10 page faults logged at [4948s], all from TCP (Texture Cache Pipe),
all PERMISSION_FAULTS=0x5, WALKER_ERROR=0x0, MAPPING_ERROR=0x0. 7
unique faulting addresses:
- 0x000072ce90828000
- 0x000072ce90a88000
- 0x000072ce90a89000
- 0x000072ce90cde000
- 0x000072ce90ce1000
- 0x000072ce90f51000
- 0x000072ce90f52000

### MES failure and GPU reset [T+4349s]

```
[4952.809s] MES(0) failed to respond to msg=REMOVE_QUEUE
[4952.809s] failed to remove hardware queue from MES, doorbell=0x1806
[4952.809s] MES might be in unrecoverable state, issue a GPU reset
[4952.809s] Failed to evict queue 4
[4952.809s] Failed to evict process queues
[4952.809s] GPU reset begin!. Source: 3
```

### GPU reset fails

```
[4953.121s] Failed to evict queue 4
[4953.121s] Failed to suspend process pid 28552
[4953.121s] remove_all_kfd_queues_mes: Failed to remove queue 3 for dev 62536
```

6 MES(1) REMOVE_QUEUE failures, each timing out after ~2.5 seconds:
```
[4955.720s] MES(1) failed to respond to msg=REMOVE_QUEUE → failed to
unmap legacy queue
[4958.283s] MES(1) failed to respond to msg=REMOVE_QUEUE → failed to
unmap legacy queue
[4960.847s] MES(1) failed to respond to msg=REMOVE_QUEUE → failed to
unmap legacy queue
[4963.411s] MES(1) failed to respond to msg=REMOVE_QUEUE → failed to
unmap legacy queue
[4965.976s] MES(1) failed to respond to msg=REMOVE_QUEUE → failed to
unmap legacy queue
[4968.540s] MES(1) failed to respond to msg=REMOVE_QUEUE → failed to
unmap legacy queue
```

### PSP suspend fails

```
[4971.164s] psp gfx command LOAD_IP_FW(0x6) failed and response status is (0x0)
[4971.164s] Failed to terminate ras ta
[4971.164s] suspend of IP block <psp> failed -22
```

### Suspend unwind fails — SMU not ready

```
[4971.164s] SMU is resuming...
[4971.164s] SMC is not ready
[4971.164s] SMC engine is not correctly up!
[4971.164s] resume of IP block <smu> failed -5
[4971.164s] amdgpu_device_ip_resume_phase2 failed during unwind: -5
[4971.164s] GPU pre asic reset failed with err, -22 for drm dev, 0000:97:00.0
```

### MODE1 reset — SMU still dead

```
[4971.164s] MODE1 reset
[4971.164s] GPU mode1 reset
[4971.164s] GPU smu mode1 reset
[4972.193s] GPU reset succeeded, trying to resume
[4972.193s] VRAM is lost due to GPU reset!
[4972.193s] SMU is resuming...
[4972.193s] SMC is not ready
[4972.193s] SMC engine is not correctly up!
[4972.193s] resume of IP block <smu> failed -5
[4972.193s] GPU reset end with ret = -5
```




On Fri, Apr 10, 2026 at 3:09 AM Ilpo Järvinen
<ilpo.jarvinen@linux.intel.com> wrote:
>
> On Thu, 9 Apr 2026, Geramy Loveless wrote:
>
> > When pbus_reassign_bridge_resources() walks up the bridge hierarchy
> > to expand a window (e.g. for resizable BAR), it refuses to release
> > any bridge window that has children.  This prevents BAR resize on
> > devices behind multi-port PCIe switches (such as Thunderbolt docks)
> > where empty sibling downstream ports hold small reservations that
> > block the parent bridge window from being freed and re-sized.
> >
> > Add pci_bus_subtree_empty() to check whether a bus subtree contains
> > any assigned device BARs, and pci_bus_release_empty_bridges() to
> > release bridge window resources of empty sibling bridges, saving
> > them to the rollback list so failures can be properly unwound.
> >
> > In pbus_reassign_bridge_resources(), call pci_bus_release_empty_bridges()
> > before checking res->child, so empty sibling windows are cleared first
> > and the parent window can then be released and grown.
> >
> > Uses PCI bus/device iterators rather than walking the raw resource
> > tree, which avoids issues with stale sibling pointers after resource
> > release.
>
> This paragraph can be dropped. And it's not exactly correct either as
> the pointers are only stale for resource entries that reside outside of
> the resource tree (after they've been released in a specific way) so if
> you start from a resource tree entry, you should never encounter a stale
> pointer.
>
> > Suggested-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
> > Signed-off-by: Geramy Loveless <gloveless@jqluv.com>
> > ---
> >  drivers/pci/setup-bus.c | 99 ++++++++++++++++++++++++++++++++++++++++-
> >  1 file changed, 97 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
> > index 4cf120ebe5a..7a182cd7e4d 100644
> > --- a/drivers/pci/setup-bus.c
> > +++ b/drivers/pci/setup-bus.c
> > @@ -2292,6 +2292,94 @@ void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge)
> >  }
> >  EXPORT_SYMBOL_GPL(pci_assign_unassigned_bridge_resources);
> >
> > +/*
> > + * pci_bus_subtree_empty - Check whether a bus subtree has any assigned
> > + * non-bridge device resources.
> > + * @bus: PCI bus to check
> > + *
> > + * Returns true if no device on @bus or its descendant buses has any
> > + * assigned BARs (bridge window resources are not considered).
> > + */
> > +static bool pci_bus_subtree_empty(struct pci_bus *bus)
> > +{
> > +     struct pci_dev *dev;
> > +
> > +     list_for_each_entry(dev, &bus->devices, bus_list) {
> > +             struct resource *r;
> > +             unsigned int i;
> > +
> > +             pci_dev_for_each_resource(dev, r, i) {
> > +                     if (i >= PCI_BRIDGE_RESOURCES)
> > +                             break;
> > +                     if (resource_assigned(r))
> > +                             return false;
> > +             }
> > +
> > +             if (dev->subordinate &&
> > +                 !pci_bus_subtree_empty(dev->subordinate))
> > +                     return false;
> > +     }
> > +
> > +     return true;
> > +}
> > +
> > +/*
> > + * pci_bus_release_empty_bridges - Release bridge window resources of
> > + * empty sibling bridges so the parent window can be freed and re-sized.
> > + * @bus: PCI bus whose child bridges to scan
> > + * @b_win: Parent bridge window resource; only children of this window
> > + *         are released
> > + * @saved: List to save released resources for rollback
> > + *
> > + * For each PCI-to-PCI bridge on @bus whose subtree is empty (no assigned
> > + * device BARs), releases bridge window resources that are children of
> > + * @b_win, saving them for rollback via @saved.
> > + *
> > + * Returns 0 on success, negative errno on failure.
> > + */
> > +static int pci_bus_release_empty_bridges(struct pci_bus *bus,
> > +                                      struct resource *b_win,
> > +                                      struct list_head *saved)
> > +{
> > +     struct pci_dev *dev;
> > +
> > +     list_for_each_entry(dev, &bus->devices, bus_list) {
> > +             struct resource *r;
> > +             unsigned int i;
> > +
> > +             if (!dev->subordinate)
> > +                     continue;
> > +
> > +             if ((dev->class >> 8) != PCI_CLASS_BRIDGE_PCI)
> > +                     continue;
>
> I suppose dev->subordinate check is enough for what we're doing so this
> looks redundant.
>
> > +
> > +             if (!pci_bus_subtree_empty(dev->subordinate))
> > +                     continue;
> > +
> > +             pci_dev_for_each_resource(dev, r, i) {
> > +                     int ret;
> > +
> > +                     if (!pci_resource_is_bridge_win(i))
> > +                             continue;
> > +
> > +                     if (!resource_assigned(r))
> > +                             continue;
> > +
> > +                     if (r->parent != b_win)
> > +                             continue;
> > +
> > +                     ret = pci_dev_res_add_to_list(saved, dev, r, 0, 0);
> > +                     if (ret)
> > +                             return ret;
> > +
> > +                     release_child_resources(r);
>
> Unfortunately you cannot call this low-level function because it
> recursively frees child resources which means you won't be able to
> rollback them as they were not added to the saved list.
>
> I think the release algorithm should basically do this:
>
> - Recurse to the subordinate buses
> - Loop through bridge window resources of this bus
>         - Skip resources that are not assigned or are not parented by b_win
>         - If the resource still has childs, leave the resource alone
>           (+ log it for easier troubleshooting these cases; any failure
>              will also cascade to upstream so it may be possible to
>              shortcut something but it will also make the algorithm more
>              complicated)
>         - Save and free the resource
>
> It might be better to move some of the code from
> pbus_reassign_bridge_resources() here as there's overlap with the sketched
> algorithm (but I'm not sure until I see the updated version but keep this
> in mind).
>
> Doing pci_bus_subtree_empty() before any removal is fine with me, but I
> see it just an optimization.
>
> > +                     pci_release_resource(dev, i);
> > +             }
> > +     }
> > +
> > +     return 0;
> > +}
> > +
> >  /*
> >   * Walk to the root bus, find the bridge window relevant for @res and
> >   * release it when possible. If the bridge window contains assigned
> > @@ -2316,7 +2404,14 @@ static int pbus_reassign_bridge_resources(struct pci_bus *bus, struct resource *
> >
> >               i = pci_resource_num(bridge, res);
> >
> > -             /* Ignore BARs which are still in use */
>
> I don't know why you removed this comment (I admit though "BARs" could
> have been worded better as it's bridge windows we're dealing here).
>
> > +             /* Release empty sibling bridge windows first */
> > +             if (bridge->subordinate) {
> > +                     ret = pci_bus_release_empty_bridges(
> > +                                     bridge->subordinate, res, saved);
>
> First arg fits to the previous line.
>
> Align the second line to (.
>
> But consider also rearranging code as I mentioned above.
>
> > +                     if (ret)
> > +                             return ret;
>
> Consider proceeding with the resize even if something failed as there are
> cases where the bridge windows are large enough (admittedly, you seem to
> only bail out in case of alloc error).
>
> In to the same vein, there seems to be one existing goto restore (that was
> added by me), which could also probably do continue instead (but changing
> it would be worth another patch).
>
> > +             }
> > +
> >               if (!res->child) {
> >                       ret = pci_dev_res_add_to_list(saved, bridge, res, 0, 0);
> >                       if (ret)
> > @@ -2327,7 +2422,7 @@ static int pbus_reassign_bridge_resources(struct pci_bus *bus, struct resource *
> >                       const char *res_name = pci_resource_name(bridge, i);
> >
> >                       pci_warn(bridge,
> > -                              "%s %pR: was not released (still contains assigned resources)\n",
> > +                              "%s %pR: not released, active children present\n",
> >                                res_name, res);
> >               }
> >
> >
>
> --
>  i.

Re: [PATCH v2] PCI: release empty sibling bridge windows during rebar expansion

[Cristian Cocos]

My experience with my 9060XT Thunderbolt eGPU is that the current
amdgpu driver is full of bugs, and this *specifically* in a Thunderbolt
eGPU configuration. I have attempted to document some of these bugs
here:
https://pcforum.amd.com/s/question/0D5Pd00001S3Av9KAF/linux-9060xt-egpuoverthunderbolt-bugs-galore

Apologies for posting this here, as most of these may not be relevant
to ReBAR, yet an AMD representative may still benefit from this
multiple bug report.

C

On Fri, 2026-04-10 at 10:53 -0700, Geramy Loveless wrote:
>  I'm going to loop in Christian Koenig over at AMD he has been
> working
> with me on resolving or attempting to figure out whats going on with
> my gfx1201 connected to a tb5 dock to the host.
> I am currently having problems with the GPU basically loosing MMIO
> and
> crashing randomly. This recent patch change I believe helped but its
> really hard to say at this point.
> Without this patch of course the bar size would be 256MB and cause
> huge performance problems or feature loss. I am able to load up AI
> models and run workloads at nearly 100% gpu usage, i'm seeing 205W
> power draw out of the maximum 300W. But after sustained load I still
> get a crash.
> 
> Maybe you would have an idea as to what is causing that crash or
> where
> I should be looking to find the cause?
> Here are some relevant logs, from what I can tell something is going
> on with MMIO, but the config bar as i understand it is still alive.
> This let me to believe maybe the router was getting put into suspend
> mode which wouldnt make sense for a GPU that is active and busy
> because the pcie tunnel would be active.
> 
> Any advice or tips would be helpful thank you for the suggestions I
> will get started on writing the patch based on those recommendations.
> 
> ## SMU Firmware Version
> 
> ```
> smu driver if version = 0x0000002e
> smu fw if version = 0x00000032
> smu fw program = 0
> smu fw version = 0x00684b00 (104.75.0)
> ```
> 
> Note: Driver interface version (0x2e / 46) does not match firmware
> interface version (0x32 / 50).
> 
> ## PCI Topology
> 
> ```
> 65:00.0 PCI bridge: Intel Barlow Ridge Host 80G (rev 84)
> 66:00.0 PCI bridge: Intel Barlow Ridge Host 80G (rev 84) → NHI
> 66:01.0 PCI bridge: Intel Barlow Ridge Host 80G (rev 84) → empty
> hotplug port
> 66:02.0 PCI bridge: Intel Barlow Ridge Host 80G (rev 84) → USB
> 66:03.0 PCI bridge: Intel Barlow Ridge Host 80G (rev 84) → dock
> 93:00.0 PCI bridge: Intel Barlow Ridge Hub 80G (rev 85) → dock switch
> 94:00.0 PCI bridge: Intel Barlow Ridge Hub 80G (rev 85) → downstream
> 95:00.0 PCI bridge: AMD Navi 10 XL Upstream Port (rev 24)
> 96:00.0 PCI bridge: AMD Navi 10 XL Downstream Port (rev 24)
> 97:00.0 VGA: AMD [1002:7551] (rev c0) ← GPU
> 97:00.1 Audio: AMD [1002:ab40]
> ```
> 
> ## Workload
> 
> GPU compute via llama.cpp (ROCm/HIP backend), running
> Qwen3.5-35B-A3B-Q4_K_M.gguf model (20.49 GiB, fully offloaded to
> VRAM). Flash attention enabled, 128K context, 32 threads.
> 
> ## Crash Timeline
> 
> All timestamps from `dmesg -T`, kernel boot-relative times in
> brackets.
> 
> ### GPU initialization (successful)
> 
> ```
> [603.644s] GPU probe: IP DISCOVERY 0x1002:0x7551
> [603.653s] Detected IP block: smu_v14_0_0, gfx_v12_0_0
> [603.771s] Detected VRAM RAM=32624M, BAR=32768M, RAM width 256bits
> GDDR6
> [604.014s] SMU driver IF 0x2e, FW IF 0x32, FW version 104.75.0
> [604.049s] SMU is initialized successfully!
> [604.119s] Runtime PM manually disabled (amdgpu.runpm=0)
> [604.119s] Initialized amdgpu 3.64.0 for 0000:97:00.0
> ```
> 
> ### SMU stops responding [T+4238s after init, ~70 minutes]
> 
> ```
> [4841.828s] SMU: No response msg_reg: 12 resp_reg: 0
> [4841.828s] [smu_v14_0_2_get_power_profile_mode] Failed to get
> activity monitor!
> [4849.393s] SMU: No response msg_reg: 12 resp_reg: 0
> [4849.393s] Failed to export SMU metrics table!
> ```
> 
> 15 consecutive `SMU: No response` messages logged between [4841s] and
> [4948s], approximately every 7-8 seconds. All with `msg_reg: 12
> resp_reg: 0`. Failed operations include:
> - `smu_v14_0_2_get_power_profile_mode` — Failed to get activity
> monitor
> - `Failed to export SMU metrics table`
> - `Failed to get current clock freq`
> 
> ### Page faults begin [T+4349s after init, ~111s after first SMU
> failure]
> 
> ```
> [4948.927s] [gfxhub] page fault (src_id:0 ring:40 vmid:9 pasid:108)
> Process llama-cli pid 35632
> GCVM_L2_PROTECTION_FAULT_STATUS: 0x00941051
> Faulty UTCL2 client ID: TCP (0x8)
> PERMISSION_FAULTS: 0x5
> WALKER_ERROR: 0x0
> MAPPING_ERROR: 0x0
> RW: 0x1 (write)
> ```
> 
> 10 page faults logged at [4948s], all from TCP (Texture Cache Pipe),
> all PERMISSION_FAULTS=0x5, WALKER_ERROR=0x0, MAPPING_ERROR=0x0. 7
> unique faulting addresses:
> - 0x000072ce90828000
> - 0x000072ce90a88000
> - 0x000072ce90a89000
> - 0x000072ce90cde000
> - 0x000072ce90ce1000
> - 0x000072ce90f51000
> - 0x000072ce90f52000
> 
> ### MES failure and GPU reset [T+4349s]
> 
> ```
> [4952.809s] MES(0) failed to respond to msg=REMOVE_QUEUE
> [4952.809s] failed to remove hardware queue from MES, doorbell=0x1806
> [4952.809s] MES might be in unrecoverable state, issue a GPU reset
> [4952.809s] Failed to evict queue 4
> [4952.809s] Failed to evict process queues
> [4952.809s] GPU reset begin!. Source: 3
> ```
> 
> ### GPU reset fails
> 
> ```
> [4953.121s] Failed to evict queue 4
> [4953.121s] Failed to suspend process pid 28552
> [4953.121s] remove_all_kfd_queues_mes: Failed to remove queue 3 for
> dev 62536
> ```
> 
> 6 MES(1) REMOVE_QUEUE failures, each timing out after ~2.5 seconds:
> ```
> [4955.720s] MES(1) failed to respond to msg=REMOVE_QUEUE → failed to
> unmap legacy queue
> [4958.283s] MES(1) failed to respond to msg=REMOVE_QUEUE → failed to
> unmap legacy queue
> [4960.847s] MES(1) failed to respond to msg=REMOVE_QUEUE → failed to
> unmap legacy queue
> [4963.411s] MES(1) failed to respond to msg=REMOVE_QUEUE → failed to
> unmap legacy queue
> [4965.976s] MES(1) failed to respond to msg=REMOVE_QUEUE → failed to
> unmap legacy queue
> [4968.540s] MES(1) failed to respond to msg=REMOVE_QUEUE → failed to
> unmap legacy queue
> ```
> 
> ### PSP suspend fails
> 
> ```
> [4971.164s] psp gfx command LOAD_IP_FW(0x6) failed and response
> status is (0x0)
> [4971.164s] Failed to terminate ras ta
> [4971.164s] suspend of IP block <psp> failed -22
> ```
> 
> ### Suspend unwind fails — SMU not ready
> 
> ```
> [4971.164s] SMU is resuming...
> [4971.164s] SMC is not ready
> [4971.164s] SMC engine is not correctly up!
> [4971.164s] resume of IP block <smu> failed -5
> [4971.164s] amdgpu_device_ip_resume_phase2 failed during unwind: -5
> [4971.164s] GPU pre asic reset failed with err, -22 for drm dev,
> 0000:97:00.0
> ```
> 
> ### MODE1 reset — SMU still dead
> 
> ```
> [4971.164s] MODE1 reset
> [4971.164s] GPU mode1 reset
> [4971.164s] GPU smu mode1 reset
> [4972.193s] GPU reset succeeded, trying to resume
> [4972.193s] VRAM is lost due to GPU reset!
> [4972.193s] SMU is resuming...
> [4972.193s] SMC is not ready
> [4972.193s] SMC engine is not correctly up!
> [4972.193s] resume of IP block <smu> failed -5
> [4972.193s] GPU reset end with ret = -5
> ```
> 
> 
> 
> 
> On Fri, Apr 10, 2026 at 3:09 AM Ilpo Järvinen
> <ilpo.jarvinen@linux.intel.com> wrote:
> > 
> > On Thu, 9 Apr 2026, Geramy Loveless wrote:
> > 
> > > When pbus_reassign_bridge_resources() walks up the bridge
> > > hierarchy
> > > to expand a window (e.g. for resizable BAR), it refuses to
> > > release
> > > any bridge window that has children.  This prevents BAR resize on
> > > devices behind multi-port PCIe switches (such as Thunderbolt
> > > docks)
> > > where empty sibling downstream ports hold small reservations that
> > > block the parent bridge window from being freed and re-sized.
> > > 
> > > Add pci_bus_subtree_empty() to check whether a bus subtree
> > > contains
> > > any assigned device BARs, and pci_bus_release_empty_bridges() to
> > > release bridge window resources of empty sibling bridges, saving
> > > them to the rollback list so failures can be properly unwound.
> > > 
> > > In pbus_reassign_bridge_resources(), call
> > > pci_bus_release_empty_bridges()
> > > before checking res->child, so empty sibling windows are cleared
> > > first
> > > and the parent window can then be released and grown.
> > > 
> > > Uses PCI bus/device iterators rather than walking the raw
> > > resource
> > > tree, which avoids issues with stale sibling pointers after
> > > resource
> > > release.
> > 
> > This paragraph can be dropped. And it's not exactly correct either
> > as
> > the pointers are only stale for resource entries that reside
> > outside of
> > the resource tree (after they've been released in a specific way)
> > so if
> > you start from a resource tree entry, you should never encounter a
> > stale
> > pointer.
> > 
> > > Suggested-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
> > > Signed-off-by: Geramy Loveless <gloveless@jqluv.com>
> > > ---
> > >  drivers/pci/setup-bus.c | 99
> > > ++++++++++++++++++++++++++++++++++++++++-
> > >  1 file changed, 97 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
> > > index 4cf120ebe5a..7a182cd7e4d 100644
> > > --- a/drivers/pci/setup-bus.c
> > > +++ b/drivers/pci/setup-bus.c
> > > @@ -2292,6 +2292,94 @@ void
> > > pci_assign_unassigned_bridge_resources(struct pci_dev *bridge)
> > >  }
> > >  EXPORT_SYMBOL_GPL(pci_assign_unassigned_bridge_resources);
> > > 
> > > +/*
> > > + * pci_bus_subtree_empty - Check whether a bus subtree has any
> > > assigned
> > > + * non-bridge device resources.
> > > + * @bus: PCI bus to check
> > > + *
> > > + * Returns true if no device on @bus or its descendant buses has
> > > any
> > > + * assigned BARs (bridge window resources are not considered).
> > > + */
> > > +static bool pci_bus_subtree_empty(struct pci_bus *bus)
> > > +{
> > > +     struct pci_dev *dev;
> > > +
> > > +     list_for_each_entry(dev, &bus->devices, bus_list) {
> > > +             struct resource *r;
> > > +             unsigned int i;
> > > +
> > > +             pci_dev_for_each_resource(dev, r, i) {
> > > +                     if (i >= PCI_BRIDGE_RESOURCES)
> > > +                             break;
> > > +                     if (resource_assigned(r))
> > > +                             return false;
> > > +             }
> > > +
> > > +             if (dev->subordinate &&
> > > +                 !pci_bus_subtree_empty(dev->subordinate))
> > > +                     return false;
> > > +     }
> > > +
> > > +     return true;
> > > +}
> > > +
> > > +/*
> > > + * pci_bus_release_empty_bridges - Release bridge window
> > > resources of
> > > + * empty sibling bridges so the parent window can be freed and
> > > re-sized.
> > > + * @bus: PCI bus whose child bridges to scan
> > > + * @b_win: Parent bridge window resource; only children of this
> > > window
> > > + *         are released
> > > + * @saved: List to save released resources for rollback
> > > + *
> > > + * For each PCI-to-PCI bridge on @bus whose subtree is empty (no
> > > assigned
> > > + * device BARs), releases bridge window resources that are
> > > children of
> > > + * @b_win, saving them for rollback via @saved.
> > > + *
> > > + * Returns 0 on success, negative errno on failure.
> > > + */
> > > +static int pci_bus_release_empty_bridges(struct pci_bus *bus,
> > > +                                      struct resource *b_win,
> > > +                                      struct list_head *saved)
> > > +{
> > > +     struct pci_dev *dev;
> > > +
> > > +     list_for_each_entry(dev, &bus->devices, bus_list) {
> > > +             struct resource *r;
> > > +             unsigned int i;
> > > +
> > > +             if (!dev->subordinate)
> > > +                     continue;
> > > +
> > > +             if ((dev->class >> 8) != PCI_CLASS_BRIDGE_PCI)
> > > +                     continue;
> > 
> > I suppose dev->subordinate check is enough for what we're doing so
> > this
> > looks redundant.
> > 
> > > +
> > > +             if (!pci_bus_subtree_empty(dev->subordinate))
> > > +                     continue;
> > > +
> > > +             pci_dev_for_each_resource(dev, r, i) {
> > > +                     int ret;
> > > +
> > > +                     if (!pci_resource_is_bridge_win(i))
> > > +                             continue;
> > > +
> > > +                     if (!resource_assigned(r))
> > > +                             continue;
> > > +
> > > +                     if (r->parent != b_win)
> > > +                             continue;
> > > +
> > > +                     ret = pci_dev_res_add_to_list(saved, dev,
> > > r, 0, 0);
> > > +                     if (ret)
> > > +                             return ret;
> > > +
> > > +                     release_child_resources(r);
> > 
> > Unfortunately you cannot call this low-level function because it
> > recursively frees child resources which means you won't be able to
> > rollback them as they were not added to the saved list.
> > 
> > I think the release algorithm should basically do this:
> > 
> > - Recurse to the subordinate buses
> > - Loop through bridge window resources of this bus
> >         - Skip resources that are not assigned or are not parented
> > by b_win
> >         - If the resource still has childs, leave the resource
> > alone
> >           (+ log it for easier troubleshooting these cases; any
> > failure
> >              will also cascade to upstream so it may be possible to
> >              shortcut something but it will also make the algorithm
> > more
> >              complicated)
> >         - Save and free the resource
> > 
> > It might be better to move some of the code from
> > pbus_reassign_bridge_resources() here as there's overlap with the
> > sketched
> > algorithm (but I'm not sure until I see the updated version but
> > keep this
> > in mind).
> > 
> > Doing pci_bus_subtree_empty() before any removal is fine with me,
> > but I
> > see it just an optimization.
> > 
> > > +                     pci_release_resource(dev, i);
> > > +             }
> > > +     }
> > > +
> > > +     return 0;
> > > +}
> > > +
> > >  /*
> > >   * Walk to the root bus, find the bridge window relevant for
> > > @res and
> > >   * release it when possible. If the bridge window contains
> > > assigned
> > > @@ -2316,7 +2404,14 @@ static int
> > > pbus_reassign_bridge_resources(struct pci_bus *bus, struct
> > > resource *
> > > 
> > >               i = pci_resource_num(bridge, res);
> > > 
> > > -             /* Ignore BARs which are still in use */
> > 
> > I don't know why you removed this comment (I admit though "BARs"
> > could
> > have been worded better as it's bridge windows we're dealing here).
> > 
> > > +             /* Release empty sibling bridge windows first */
> > > +             if (bridge->subordinate) {
> > > +                     ret = pci_bus_release_empty_bridges(
> > > +                                     bridge->subordinate, res,
> > > saved);
> > 
> > First arg fits to the previous line.
> > 
> > Align the second line to (.
> > 
> > But consider also rearranging code as I mentioned above.
> > 
> > > +                     if (ret)
> > > +                             return ret;
> > 
> > Consider proceeding with the resize even if something failed as
> > there are
> > cases where the bridge windows are large enough (admittedly, you
> > seem to
> > only bail out in case of alloc error).
> > 
> > In to the same vein, there seems to be one existing goto restore
> > (that was
> > added by me), which could also probably do continue instead (but
> > changing
> > it would be worth another patch).
> > 
> > > +             }
> > > +
> > >               if (!res->child) {
> > >                       ret = pci_dev_res_add_to_list(saved,
> > > bridge, res, 0, 0);
> > >                       if (ret)
> > > @@ -2327,7 +2422,7 @@ static int
> > > pbus_reassign_bridge_resources(struct pci_bus *bus, struct
> > > resource *
> > >                       const char *res_name =
> > > pci_resource_name(bridge, i);
> > > 
> > >                       pci_warn(bridge,
> > > -                              "%s %pR: was not released (still
> > > contains assigned resources)\n",
> > > +                              "%s %pR: not released, active
> > > children present\n",
> > >                                res_name, res);
> > >               }
> > > 
> > > 
> > 
> > --
> >  i.

[PATCH] PCI: release empty sibling bridge resources during window resize

[Geramy Loveless]

pbus_reassign_bridge_resources() refuses to release bridge windows that
have child resources.  On multi-port PCIe switches (e.g. Thunderbolt
docks), empty sibling downstream ports hold small reservations that
prevent the parent window from being freed and re-sized for rebar.

Add a bottom-up recursive release that saves and frees each empty bridge
window individually, so the parent window can then be released and grown.

Suggested-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Signed-off-by: Geramy Loveless <gloveless@jqluv.com>
---
 drivers/pci/setup-bus.c | 63 +++++++++++++++++++++++++++++++++++++++--
 1 file changed, 61 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 4cf120ebe5ad..0c1fb654c9cc 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -2292,6 +2292,60 @@ void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge)
 }
 EXPORT_SYMBOL_GPL(pci_assign_unassigned_bridge_resources);
 
+/*
+ * pci_bus_release_empty_bridge_resources - Bottom-up release of bridge
+ * window resources in empty subtrees.
+ * @bus: PCI bus whose child bridges to process
+ * @b_win: Ancestor bridge window; only children of this resource are released
+ * @saved: List to save released resources for rollback
+ *
+ * Recurses depth-first into subordinate buses, then releases bridge windows
+ * on the way back up.  Each resource is individually saved before release so
+ * the entire operation can be rolled back.
+ */
+static void pci_bus_release_empty_bridge_resources(struct pci_bus *bus,
+						   struct resource *b_win,
+						   struct list_head *saved)
+{
+	struct pci_dev *dev;
+
+	list_for_each_entry(dev, &bus->devices, bus_list) {
+		struct resource *r;
+		unsigned int i;
+
+		if (!dev->subordinate)
+			continue;
+
+		/* Recurse first — release deepest resources before parents */
+		pci_bus_release_empty_bridge_resources(dev->subordinate,
+						      b_win, saved);
+
+		pci_dev_for_each_resource(dev, r, i) {
+			if (!pci_resource_is_bridge_win(i))
+				continue;
+
+			if (!resource_assigned(r))
+				continue;
+
+			if (r->parent != b_win)
+				continue;
+
+			if (r->child) {
+				const char *res_name = pci_resource_name(dev, i);
+
+				pci_info(dev, "%s %pR: not released, children still present\n",
+					 res_name, r);
+				continue;
+			}
+
+			if (pci_dev_res_add_to_list(saved, dev, r, 0, 0))
+				continue;
+
+			pci_release_resource(dev, i);
+		}
+	}
+}
+
 /*
  * Walk to the root bus, find the bridge window relevant for @res and
  * release it when possible. If the bridge window contains assigned
@@ -2316,7 +2370,12 @@ static int pbus_reassign_bridge_resources(struct pci_bus *bus, struct resource *
 
 		i = pci_resource_num(bridge, res);
 
-		/* Ignore BARs which are still in use */
+		/* Release empty sibling bridge windows bottom-up */
+		if (bridge->subordinate)
+			pci_bus_release_empty_bridge_resources(
+				bridge->subordinate, res, saved);
+
+		/* Ignore bridge windows which are still in use */
 		if (!res->child) {
 			ret = pci_dev_res_add_to_list(saved, bridge, res, 0, 0);
 			if (ret)
@@ -2327,7 +2386,7 @@ static int pbus_reassign_bridge_resources(struct pci_bus *bus, struct resource *
 			const char *res_name = pci_resource_name(bridge, i);
 
 			pci_warn(bridge,
-				 "%s %pR: was not released (still contains assigned resources)\n",
+				 "%s %pR: not released, active children present\n",
 				 res_name, res);
 		}
 
-- 
2.51.0

Re: [PATCH v2] PCI: release empty sibling bridge windows during rebar expansion

[Geramy Loveless]

Oh wow, would you mind me adding you to my existing bug ticket with AMD?
I am having the exact same problem, they are working on some fixes for
some of the bugs I ran across. I ended adding in some device active
and alive checks basically to stop the driver from crashing the
kernel. But I could add you in there so we can get both of our logs
which are the same problem I believe in the same "ticket" if you will.
It might also help having two views of the same issue. We have the
same dock.




Geramy L. Loveless
Founder & Chief Innovation Officer

JQluv.net, Inc.
Site: JQluv.com
Mobile: 559.999.1557
Office: 1 (877) 44 JQluv




On Fri, Apr 10, 2026 at 11:58 AM Cristian Cocos <cristi@ieee.org> wrote:
>
> My experience with my 9060XT Thunderbolt eGPU is that the current
> amdgpu driver is full of bugs, and this *specifically* in a Thunderbolt
> eGPU configuration. I have attempted to document some of these bugs
> here:
> https://pcforum.amd.com/s/question/0D5Pd00001S3Av9KAF/linux-9060xt-egpuoverthunderbolt-bugs-galore
>
> Apologies for posting this here, as most of these may not be relevant
> to ReBAR, yet an AMD representative may still benefit from this
> multiple bug report.
>
> C
>
> On Fri, 2026-04-10 at 10:53 -0700, Geramy Loveless wrote:
> >  I'm going to loop in Christian Koenig over at AMD he has been
> > working
> > with me on resolving or attempting to figure out whats going on with
> > my gfx1201 connected to a tb5 dock to the host.
> > I am currently having problems with the GPU basically loosing MMIO
> > and
> > crashing randomly. This recent patch change I believe helped but its
> > really hard to say at this point.
> > Without this patch of course the bar size would be 256MB and cause
> > huge performance problems or feature loss. I am able to load up AI
> > models and run workloads at nearly 100% gpu usage, i'm seeing 205W
> > power draw out of the maximum 300W. But after sustained load I still
> > get a crash.
> >
> > Maybe you would have an idea as to what is causing that crash or
> > where
> > I should be looking to find the cause?
> > Here are some relevant logs, from what I can tell something is going
> > on with MMIO, but the config bar as i understand it is still alive.
> > This let me to believe maybe the router was getting put into suspend
> > mode which wouldnt make sense for a GPU that is active and busy
> > because the pcie tunnel would be active.
> >
> > Any advice or tips would be helpful thank you for the suggestions I
> > will get started on writing the patch based on those recommendations.
> >
> > ## SMU Firmware Version
> >
> > ```
> > smu driver if version = 0x0000002e
> > smu fw if version = 0x00000032
> > smu fw program = 0
> > smu fw version = 0x00684b00 (104.75.0)
> > ```
> >
> > Note: Driver interface version (0x2e / 46) does not match firmware
> > interface version (0x32 / 50).
> >
> > ## PCI Topology
> >
> > ```
> > 65:00.0 PCI bridge: Intel Barlow Ridge Host 80G (rev 84)
> > 66:00.0 PCI bridge: Intel Barlow Ridge Host 80G (rev 84) → NHI
> > 66:01.0 PCI bridge: Intel Barlow Ridge Host 80G (rev 84) → empty
> > hotplug port
> > 66:02.0 PCI bridge: Intel Barlow Ridge Host 80G (rev 84) → USB
> > 66:03.0 PCI bridge: Intel Barlow Ridge Host 80G (rev 84) → dock
> > 93:00.0 PCI bridge: Intel Barlow Ridge Hub 80G (rev 85) → dock switch
> > 94:00.0 PCI bridge: Intel Barlow Ridge Hub 80G (rev 85) → downstream
> > 95:00.0 PCI bridge: AMD Navi 10 XL Upstream Port (rev 24)
> > 96:00.0 PCI bridge: AMD Navi 10 XL Downstream Port (rev 24)
> > 97:00.0 VGA: AMD [1002:7551] (rev c0) ← GPU
> > 97:00.1 Audio: AMD [1002:ab40]
> > ```
> >
> > ## Workload
> >
> > GPU compute via llama.cpp (ROCm/HIP backend), running
> > Qwen3.5-35B-A3B-Q4_K_M.gguf model (20.49 GiB, fully offloaded to
> > VRAM). Flash attention enabled, 128K context, 32 threads.
> >
> > ## Crash Timeline
> >
> > All timestamps from `dmesg -T`, kernel boot-relative times in
> > brackets.
> >
> > ### GPU initialization (successful)
> >
> > ```
> > [603.644s] GPU probe: IP DISCOVERY 0x1002:0x7551
> > [603.653s] Detected IP block: smu_v14_0_0, gfx_v12_0_0
> > [603.771s] Detected VRAM RAM=32624M, BAR=32768M, RAM width 256bits
> > GDDR6
> > [604.014s] SMU driver IF 0x2e, FW IF 0x32, FW version 104.75.0
> > [604.049s] SMU is initialized successfully!
> > [604.119s] Runtime PM manually disabled (amdgpu.runpm=0)
> > [604.119s] Initialized amdgpu 3.64.0 for 0000:97:00.0
> > ```
> >
> > ### SMU stops responding [T+4238s after init, ~70 minutes]
> >
> > ```
> > [4841.828s] SMU: No response msg_reg: 12 resp_reg: 0
> > [4841.828s] [smu_v14_0_2_get_power_profile_mode] Failed to get
> > activity monitor!
> > [4849.393s] SMU: No response msg_reg: 12 resp_reg: 0
> > [4849.393s] Failed to export SMU metrics table!
> > ```
> >
> > 15 consecutive `SMU: No response` messages logged between [4841s] and
> > [4948s], approximately every 7-8 seconds. All with `msg_reg: 12
> > resp_reg: 0`. Failed operations include:
> > - `smu_v14_0_2_get_power_profile_mode` — Failed to get activity
> > monitor
> > - `Failed to export SMU metrics table`
> > - `Failed to get current clock freq`
> >
> > ### Page faults begin [T+4349s after init, ~111s after first SMU
> > failure]
> >
> > ```
> > [4948.927s] [gfxhub] page fault (src_id:0 ring:40 vmid:9 pasid:108)
> > Process llama-cli pid 35632
> > GCVM_L2_PROTECTION_FAULT_STATUS: 0x00941051
> > Faulty UTCL2 client ID: TCP (0x8)
> > PERMISSION_FAULTS: 0x5
> > WALKER_ERROR: 0x0
> > MAPPING_ERROR: 0x0
> > RW: 0x1 (write)
> > ```
> >
> > 10 page faults logged at [4948s], all from TCP (Texture Cache Pipe),
> > all PERMISSION_FAULTS=0x5, WALKER_ERROR=0x0, MAPPING_ERROR=0x0. 7
> > unique faulting addresses:
> > - 0x000072ce90828000
> > - 0x000072ce90a88000
> > - 0x000072ce90a89000
> > - 0x000072ce90cde000
> > - 0x000072ce90ce1000
> > - 0x000072ce90f51000
> > - 0x000072ce90f52000
> >
> > ### MES failure and GPU reset [T+4349s]
> >
> > ```
> > [4952.809s] MES(0) failed to respond to msg=REMOVE_QUEUE
> > [4952.809s] failed to remove hardware queue from MES, doorbell=0x1806
> > [4952.809s] MES might be in unrecoverable state, issue a GPU reset
> > [4952.809s] Failed to evict queue 4
> > [4952.809s] Failed to evict process queues
> > [4952.809s] GPU reset begin!. Source: 3
> > ```
> >
> > ### GPU reset fails
> >
> > ```
> > [4953.121s] Failed to evict queue 4
> > [4953.121s] Failed to suspend process pid 28552
> > [4953.121s] remove_all_kfd_queues_mes: Failed to remove queue 3 for
> > dev 62536
> > ```
> >
> > 6 MES(1) REMOVE_QUEUE failures, each timing out after ~2.5 seconds:
> > ```
> > [4955.720s] MES(1) failed to respond to msg=REMOVE_QUEUE → failed to
> > unmap legacy queue
> > [4958.283s] MES(1) failed to respond to msg=REMOVE_QUEUE → failed to
> > unmap legacy queue
> > [4960.847s] MES(1) failed to respond to msg=REMOVE_QUEUE → failed to
> > unmap legacy queue
> > [4963.411s] MES(1) failed to respond to msg=REMOVE_QUEUE → failed to
> > unmap legacy queue
> > [4965.976s] MES(1) failed to respond to msg=REMOVE_QUEUE → failed to
> > unmap legacy queue
> > [4968.540s] MES(1) failed to respond to msg=REMOVE_QUEUE → failed to
> > unmap legacy queue
> > ```
> >
> > ### PSP suspend fails
> >
> > ```
> > [4971.164s] psp gfx command LOAD_IP_FW(0x6) failed and response
> > status is (0x0)
> > [4971.164s] Failed to terminate ras ta
> > [4971.164s] suspend of IP block <psp> failed -22
> > ```
> >
> > ### Suspend unwind fails — SMU not ready
> >
> > ```
> > [4971.164s] SMU is resuming...
> > [4971.164s] SMC is not ready
> > [4971.164s] SMC engine is not correctly up!
> > [4971.164s] resume of IP block <smu> failed -5
> > [4971.164s] amdgpu_device_ip_resume_phase2 failed during unwind: -5
> > [4971.164s] GPU pre asic reset failed with err, -22 for drm dev,
> > 0000:97:00.0
> > ```
> >
> > ### MODE1 reset — SMU still dead
> >
> > ```
> > [4971.164s] MODE1 reset
> > [4971.164s] GPU mode1 reset
> > [4971.164s] GPU smu mode1 reset
> > [4972.193s] GPU reset succeeded, trying to resume
> > [4972.193s] VRAM is lost due to GPU reset!
> > [4972.193s] SMU is resuming...
> > [4972.193s] SMC is not ready
> > [4972.193s] SMC engine is not correctly up!
> > [4972.193s] resume of IP block <smu> failed -5
> > [4972.193s] GPU reset end with ret = -5
> > ```
> >
> >
> >
> >
> > On Fri, Apr 10, 2026 at 3:09 AM Ilpo Järvinen
> > <ilpo.jarvinen@linux.intel.com> wrote:
> > >
> > > On Thu, 9 Apr 2026, Geramy Loveless wrote:
> > >
> > > > When pbus_reassign_bridge_resources() walks up the bridge
> > > > hierarchy
> > > > to expand a window (e.g. for resizable BAR), it refuses to
> > > > release
> > > > any bridge window that has children.  This prevents BAR resize on
> > > > devices behind multi-port PCIe switches (such as Thunderbolt
> > > > docks)
> > > > where empty sibling downstream ports hold small reservations that
> > > > block the parent bridge window from being freed and re-sized.
> > > >
> > > > Add pci_bus_subtree_empty() to check whether a bus subtree
> > > > contains
> > > > any assigned device BARs, and pci_bus_release_empty_bridges() to
> > > > release bridge window resources of empty sibling bridges, saving
> > > > them to the rollback list so failures can be properly unwound.
> > > >
> > > > In pbus_reassign_bridge_resources(), call
> > > > pci_bus_release_empty_bridges()
> > > > before checking res->child, so empty sibling windows are cleared
> > > > first
> > > > and the parent window can then be released and grown.
> > > >
> > > > Uses PCI bus/device iterators rather than walking the raw
> > > > resource
> > > > tree, which avoids issues with stale sibling pointers after
> > > > resource
> > > > release.
> > >
> > > This paragraph can be dropped. And it's not exactly correct either
> > > as
> > > the pointers are only stale for resource entries that reside
> > > outside of
> > > the resource tree (after they've been released in a specific way)
> > > so if
> > > you start from a resource tree entry, you should never encounter a
> > > stale
> > > pointer.
> > >
> > > > Suggested-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
> > > > Signed-off-by: Geramy Loveless <gloveless@jqluv.com>
> > > > ---
> > > >  drivers/pci/setup-bus.c | 99
> > > > ++++++++++++++++++++++++++++++++++++++++-
> > > >  1 file changed, 97 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
> > > > index 4cf120ebe5a..7a182cd7e4d 100644
> > > > --- a/drivers/pci/setup-bus.c
> > > > +++ b/drivers/pci/setup-bus.c
> > > > @@ -2292,6 +2292,94 @@ void
> > > > pci_assign_unassigned_bridge_resources(struct pci_dev *bridge)
> > > >  }
> > > >  EXPORT_SYMBOL_GPL(pci_assign_unassigned_bridge_resources);
> > > >
> > > > +/*
> > > > + * pci_bus_subtree_empty - Check whether a bus subtree has any
> > > > assigned
> > > > + * non-bridge device resources.
> > > > + * @bus: PCI bus to check
> > > > + *
> > > > + * Returns true if no device on @bus or its descendant buses has
> > > > any
> > > > + * assigned BARs (bridge window resources are not considered).
> > > > + */
> > > > +static bool pci_bus_subtree_empty(struct pci_bus *bus)
> > > > +{
> > > > +     struct pci_dev *dev;
> > > > +
> > > > +     list_for_each_entry(dev, &bus->devices, bus_list) {
> > > > +             struct resource *r;
> > > > +             unsigned int i;
> > > > +
> > > > +             pci_dev_for_each_resource(dev, r, i) {
> > > > +                     if (i >= PCI_BRIDGE_RESOURCES)
> > > > +                             break;
> > > > +                     if (resource_assigned(r))
> > > > +                             return false;
> > > > +             }
> > > > +
> > > > +             if (dev->subordinate &&
> > > > +                 !pci_bus_subtree_empty(dev->subordinate))
> > > > +                     return false;
> > > > +     }
> > > > +
> > > > +     return true;
> > > > +}
> > > > +
> > > > +/*
> > > > + * pci_bus_release_empty_bridges - Release bridge window
> > > > resources of
> > > > + * empty sibling bridges so the parent window can be freed and
> > > > re-sized.
> > > > + * @bus: PCI bus whose child bridges to scan
> > > > + * @b_win: Parent bridge window resource; only children of this
> > > > window
> > > > + *         are released
> > > > + * @saved: List to save released resources for rollback
> > > > + *
> > > > + * For each PCI-to-PCI bridge on @bus whose subtree is empty (no
> > > > assigned
> > > > + * device BARs), releases bridge window resources that are
> > > > children of
> > > > + * @b_win, saving them for rollback via @saved.
> > > > + *
> > > > + * Returns 0 on success, negative errno on failure.
> > > > + */
> > > > +static int pci_bus_release_empty_bridges(struct pci_bus *bus,
> > > > +                                      struct resource *b_win,
> > > > +                                      struct list_head *saved)
> > > > +{
> > > > +     struct pci_dev *dev;
> > > > +
> > > > +     list_for_each_entry(dev, &bus->devices, bus_list) {
> > > > +             struct resource *r;
> > > > +             unsigned int i;
> > > > +
> > > > +             if (!dev->subordinate)
> > > > +                     continue;
> > > > +
> > > > +             if ((dev->class >> 8) != PCI_CLASS_BRIDGE_PCI)
> > > > +                     continue;
> > >
> > > I suppose dev->subordinate check is enough for what we're doing so
> > > this
> > > looks redundant.
> > >
> > > > +
> > > > +             if (!pci_bus_subtree_empty(dev->subordinate))
> > > > +                     continue;
> > > > +
> > > > +             pci_dev_for_each_resource(dev, r, i) {
> > > > +                     int ret;
> > > > +
> > > > +                     if (!pci_resource_is_bridge_win(i))
> > > > +                             continue;
> > > > +
> > > > +                     if (!resource_assigned(r))
> > > > +                             continue;
> > > > +
> > > > +                     if (r->parent != b_win)
> > > > +                             continue;
> > > > +
> > > > +                     ret = pci_dev_res_add_to_list(saved, dev,
> > > > r, 0, 0);
> > > > +                     if (ret)
> > > > +                             return ret;
> > > > +
> > > > +                     release_child_resources(r);
> > >
> > > Unfortunately you cannot call this low-level function because it
> > > recursively frees child resources which means you won't be able to
> > > rollback them as they were not added to the saved list.
> > >
> > > I think the release algorithm should basically do this:
> > >
> > > - Recurse to the subordinate buses
> > > - Loop through bridge window resources of this bus
> > >         - Skip resources that are not assigned or are not parented
> > > by b_win
> > >         - If the resource still has childs, leave the resource
> > > alone
> > >           (+ log it for easier troubleshooting these cases; any
> > > failure
> > >              will also cascade to upstream so it may be possible to
> > >              shortcut something but it will also make the algorithm
> > > more
> > >              complicated)
> > >         - Save and free the resource
> > >
> > > It might be better to move some of the code from
> > > pbus_reassign_bridge_resources() here as there's overlap with the
> > > sketched
> > > algorithm (but I'm not sure until I see the updated version but
> > > keep this
> > > in mind).
> > >
> > > Doing pci_bus_subtree_empty() before any removal is fine with me,
> > > but I
> > > see it just an optimization.
> > >
> > > > +                     pci_release_resource(dev, i);
> > > > +             }
> > > > +     }
> > > > +
> > > > +     return 0;
> > > > +}
> > > > +
> > > >  /*
> > > >   * Walk to the root bus, find the bridge window relevant for
> > > > @res and
> > > >   * release it when possible. If the bridge window contains
> > > > assigned
> > > > @@ -2316,7 +2404,14 @@ static int
> > > > pbus_reassign_bridge_resources(struct pci_bus *bus, struct
> > > > resource *
> > > >
> > > >               i = pci_resource_num(bridge, res);
> > > >
> > > > -             /* Ignore BARs which are still in use */
> > >
> > > I don't know why you removed this comment (I admit though "BARs"
> > > could
> > > have been worded better as it's bridge windows we're dealing here).
> > >
> > > > +             /* Release empty sibling bridge windows first */
> > > > +             if (bridge->subordinate) {
> > > > +                     ret = pci_bus_release_empty_bridges(
> > > > +                                     bridge->subordinate, res,
> > > > saved);
> > >
> > > First arg fits to the previous line.
> > >
> > > Align the second line to (.
> > >
> > > But consider also rearranging code as I mentioned above.
> > >
> > > > +                     if (ret)
> > > > +                             return ret;
> > >
> > > Consider proceeding with the resize even if something failed as
> > > there are
> > > cases where the bridge windows are large enough (admittedly, you
> > > seem to
> > > only bail out in case of alloc error).
> > >
> > > In to the same vein, there seems to be one existing goto restore
> > > (that was
> > > added by me), which could also probably do continue instead (but
> > > changing
> > > it would be worth another patch).
> > >
> > > > +             }
> > > > +
> > > >               if (!res->child) {
> > > >                       ret = pci_dev_res_add_to_list(saved,
> > > > bridge, res, 0, 0);
> > > >                       if (ret)
> > > > @@ -2327,7 +2422,7 @@ static int
> > > > pbus_reassign_bridge_resources(struct pci_bus *bus, struct
> > > > resource *
> > > >                       const char *res_name =
> > > > pci_resource_name(bridge, i);
> > > >
> > > >                       pci_warn(bridge,
> > > > -                              "%s %pR: was not released (still
> > > > contains assigned resources)\n",
> > > > +                              "%s %pR: not released, active
> > > > children present\n",
> > > >                                res_name, res);
> > > >               }
> > > >
> > > >
> > >
> > > --
> > >  i.

Re: [PATCH v2] PCI: release empty sibling bridge windows during rebar expansion

[Geramy Loveless]

So after a while of doing investigations I keep thinking, gpus have
supported hotplug or so i would believe for a while ever since at
least thunderbolt 4 and possibly older styles too of hotplug as well
as dual gpu machines like laptops that have a dedicated and
integrated. I would think all of these problems we are seeing is most
likely due to something very specific to thunderbolt 5. So here is a
theory.

Root port ACPI device: \_SB_.PCI0.GP10 at address 0x00030002 = PCI 00:03.2
Its _DSD (Device-Specific Data) contains only:
"FundamentalDeviceResetTriggeredOnD3ToD0" = 1

No ExternalFacingPort property. The firmware gives it a D3-to-D0 reset
hint but never declares it as external-facing.
For comparison, a properly configured system would have:

Package {"ExternalFacingPort", 1},
Package {"DmaProperty", 1},

The complete proof chain:

DSDT (/tmp/dsdt.dsl:4194-4205): GP10 (00:03.2) _DSD has no ExternalFacingPort
pci_acpi_set_external_facing() (pci-acpi.c:1422):
device_property_read_u8("ExternalFacingPort") fails → external_facing
stays 0
pci_set_removable() (probe.c:1780): parent not removable → falls through
arch_pci_dev_is_removable() (arch/x86/pci/acpi.c:353):
!root->external_facing → returns false
Result: dev->removable never set to DEVICE_REMOVABLE on any device in the chain

Sysfs proof:
  00:03.2: removable=N/A, external_facing=N/A
  65:00.0: removable=N/A
  66:03.0: removable=N/A  (TB bridge with HotPlug+ Surprise+)
  93:00.0: removable=N/A
  97:00.0: removable=N/A  (eGPU — dev_is_removable() returns false)

I'm not sure its correct so much but I have patched up the amdgpu
driver myself to use what its using now and "||
pci_is_thunderbolt_attached"
This might be entirely completely wrong :D but I figured I would see
what happens.

On Fri, Apr 10, 2026 at 12:14 PM Geramy Loveless <gloveless@jqluv.com> wrote:
>
> Oh wow, would you mind me adding you to my existing bug ticket with AMD?
> I am having the exact same problem, they are working on some fixes for
> some of the bugs I ran across. I ended adding in some device active
> and alive checks basically to stop the driver from crashing the
> kernel. But I could add you in there so we can get both of our logs
> which are the same problem I believe in the same "ticket" if you will.
> It might also help having two views of the same issue. We have the
> same dock.
>
>
>
>
>
>
>
>
> On Fri, Apr 10, 2026 at 11:58 AM Cristian Cocos <cristi@ieee.org> wrote:
> >
> > My experience with my 9060XT Thunderbolt eGPU is that the current
> > amdgpu driver is full of bugs, and this *specifically* in a Thunderbolt
> > eGPU configuration. I have attempted to document some of these bugs
> > here:
> > https://pcforum.amd.com/s/question/0D5Pd00001S3Av9KAF/linux-9060xt-egpuoverthunderbolt-bugs-galore
> >
> > Apologies for posting this here, as most of these may not be relevant
> > to ReBAR, yet an AMD representative may still benefit from this
> > multiple bug report.
> >
> > C
> >
> > On Fri, 2026-04-10 at 10:53 -0700, Geramy Loveless wrote:
> > >  I'm going to loop in Christian Koenig over at AMD he has been
> > > working
> > > with me on resolving or attempting to figure out whats going on with
> > > my gfx1201 connected to a tb5 dock to the host.
> > > I am currently having problems with the GPU basically loosing MMIO
> > > and
> > > crashing randomly. This recent patch change I believe helped but its
> > > really hard to say at this point.
> > > Without this patch of course the bar size would be 256MB and cause
> > > huge performance problems or feature loss. I am able to load up AI
> > > models and run workloads at nearly 100% gpu usage, i'm seeing 205W
> > > power draw out of the maximum 300W. But after sustained load I still
> > > get a crash.
> > >
> > > Maybe you would have an idea as to what is causing that crash or
> > > where
> > > I should be looking to find the cause?
> > > Here are some relevant logs, from what I can tell something is going
> > > on with MMIO, but the config bar as i understand it is still alive.
> > > This let me to believe maybe the router was getting put into suspend
> > > mode which wouldnt make sense for a GPU that is active and busy
> > > because the pcie tunnel would be active.
> > >
> > > Any advice or tips would be helpful thank you for the suggestions I
> > > will get started on writing the patch based on those recommendations.
> > >
> > > ## SMU Firmware Version
> > >
> > > ```
> > > smu driver if version = 0x0000002e
> > > smu fw if version = 0x00000032
> > > smu fw program = 0
> > > smu fw version = 0x00684b00 (104.75.0)
> > > ```
> > >
> > > Note: Driver interface version (0x2e / 46) does not match firmware
> > > interface version (0x32 / 50).
> > >
> > > ## PCI Topology
> > >
> > > ```
> > > 65:00.0 PCI bridge: Intel Barlow Ridge Host 80G (rev 84)
> > > 66:00.0 PCI bridge: Intel Barlow Ridge Host 80G (rev 84) → NHI
> > > 66:01.0 PCI bridge: Intel Barlow Ridge Host 80G (rev 84) → empty
> > > hotplug port
> > > 66:02.0 PCI bridge: Intel Barlow Ridge Host 80G (rev 84) → USB
> > > 66:03.0 PCI bridge: Intel Barlow Ridge Host 80G (rev 84) → dock
> > > 93:00.0 PCI bridge: Intel Barlow Ridge Hub 80G (rev 85) → dock switch
> > > 94:00.0 PCI bridge: Intel Barlow Ridge Hub 80G (rev 85) → downstream
> > > 95:00.0 PCI bridge: AMD Navi 10 XL Upstream Port (rev 24)
> > > 96:00.0 PCI bridge: AMD Navi 10 XL Downstream Port (rev 24)
> > > 97:00.0 VGA: AMD [1002:7551] (rev c0) ← GPU
> > > 97:00.1 Audio: AMD [1002:ab40]
> > > ```
> > >
> > > ## Workload
> > >
> > > GPU compute via llama.cpp (ROCm/HIP backend), running
> > > Qwen3.5-35B-A3B-Q4_K_M.gguf model (20.49 GiB, fully offloaded to
> > > VRAM). Flash attention enabled, 128K context, 32 threads.
> > >
> > > ## Crash Timeline
> > >
> > > All timestamps from `dmesg -T`, kernel boot-relative times in
> > > brackets.
> > >
> > > ### GPU initialization (successful)
> > >
> > > ```
> > > [603.644s] GPU probe: IP DISCOVERY 0x1002:0x7551
> > > [603.653s] Detected IP block: smu_v14_0_0, gfx_v12_0_0
> > > [603.771s] Detected VRAM RAM=32624M, BAR=32768M, RAM width 256bits
> > > GDDR6
> > > [604.014s] SMU driver IF 0x2e, FW IF 0x32, FW version 104.75.0
> > > [604.049s] SMU is initialized successfully!
> > > [604.119s] Runtime PM manually disabled (amdgpu.runpm=0)
> > > [604.119s] Initialized amdgpu 3.64.0 for 0000:97:00.0
> > > ```
> > >
> > > ### SMU stops responding [T+4238s after init, ~70 minutes]
> > >
> > > ```
> > > [4841.828s] SMU: No response msg_reg: 12 resp_reg: 0
> > > [4841.828s] [smu_v14_0_2_get_power_profile_mode] Failed to get
> > > activity monitor!
> > > [4849.393s] SMU: No response msg_reg: 12 resp_reg: 0
> > > [4849.393s] Failed to export SMU metrics table!
> > > ```
> > >
> > > 15 consecutive `SMU: No response` messages logged between [4841s] and
> > > [4948s], approximately every 7-8 seconds. All with `msg_reg: 12
> > > resp_reg: 0`. Failed operations include:
> > > - `smu_v14_0_2_get_power_profile_mode` — Failed to get activity
> > > monitor
> > > - `Failed to export SMU metrics table`
> > > - `Failed to get current clock freq`
> > >
> > > ### Page faults begin [T+4349s after init, ~111s after first SMU
> > > failure]
> > >
> > > ```
> > > [4948.927s] [gfxhub] page fault (src_id:0 ring:40 vmid:9 pasid:108)
> > > Process llama-cli pid 35632
> > > GCVM_L2_PROTECTION_FAULT_STATUS: 0x00941051
> > > Faulty UTCL2 client ID: TCP (0x8)
> > > PERMISSION_FAULTS: 0x5
> > > WALKER_ERROR: 0x0
> > > MAPPING_ERROR: 0x0
> > > RW: 0x1 (write)
> > > ```
> > >
> > > 10 page faults logged at [4948s], all from TCP (Texture Cache Pipe),
> > > all PERMISSION_FAULTS=0x5, WALKER_ERROR=0x0, MAPPING_ERROR=0x0. 7
> > > unique faulting addresses:
> > > - 0x000072ce90828000
> > > - 0x000072ce90a88000
> > > - 0x000072ce90a89000
> > > - 0x000072ce90cde000
> > > - 0x000072ce90ce1000
> > > - 0x000072ce90f51000
> > > - 0x000072ce90f52000
> > >
> > > ### MES failure and GPU reset [T+4349s]
> > >
> > > ```
> > > [4952.809s] MES(0) failed to respond to msg=REMOVE_QUEUE
> > > [4952.809s] failed to remove hardware queue from MES, doorbell=0x1806
> > > [4952.809s] MES might be in unrecoverable state, issue a GPU reset
> > > [4952.809s] Failed to evict queue 4
> > > [4952.809s] Failed to evict process queues
> > > [4952.809s] GPU reset begin!. Source: 3
> > > ```
> > >
> > > ### GPU reset fails
> > >
> > > ```
> > > [4953.121s] Failed to evict queue 4
> > > [4953.121s] Failed to suspend process pid 28552
> > > [4953.121s] remove_all_kfd_queues_mes: Failed to remove queue 3 for
> > > dev 62536
> > > ```
> > >
> > > 6 MES(1) REMOVE_QUEUE failures, each timing out after ~2.5 seconds:
> > > ```
> > > [4955.720s] MES(1) failed to respond to msg=REMOVE_QUEUE → failed to
> > > unmap legacy queue
> > > [4958.283s] MES(1) failed to respond to msg=REMOVE_QUEUE → failed to
> > > unmap legacy queue
> > > [4960.847s] MES(1) failed to respond to msg=REMOVE_QUEUE → failed to
> > > unmap legacy queue
> > > [4963.411s] MES(1) failed to respond to msg=REMOVE_QUEUE → failed to
> > > unmap legacy queue
> > > [4965.976s] MES(1) failed to respond to msg=REMOVE_QUEUE → failed to
> > > unmap legacy queue
> > > [4968.540s] MES(1) failed to respond to msg=REMOVE_QUEUE → failed to
> > > unmap legacy queue
> > > ```
> > >
> > > ### PSP suspend fails
> > >
> > > ```
> > > [4971.164s] psp gfx command LOAD_IP_FW(0x6) failed and response
> > > status is (0x0)
> > > [4971.164s] Failed to terminate ras ta
> > > [4971.164s] suspend of IP block <psp> failed -22
> > > ```
> > >
> > > ### Suspend unwind fails — SMU not ready
> > >
> > > ```
> > > [4971.164s] SMU is resuming...
> > > [4971.164s] SMC is not ready
> > > [4971.164s] SMC engine is not correctly up!
> > > [4971.164s] resume of IP block <smu> failed -5
> > > [4971.164s] amdgpu_device_ip_resume_phase2 failed during unwind: -5
> > > [4971.164s] GPU pre asic reset failed with err, -22 for drm dev,
> > > 0000:97:00.0
> > > ```
> > >
> > > ### MODE1 reset — SMU still dead
> > >
> > > ```
> > > [4971.164s] MODE1 reset
> > > [4971.164s] GPU mode1 reset
> > > [4971.164s] GPU smu mode1 reset
> > > [4972.193s] GPU reset succeeded, trying to resume
> > > [4972.193s] VRAM is lost due to GPU reset!
> > > [4972.193s] SMU is resuming...
> > > [4972.193s] SMC is not ready
> > > [4972.193s] SMC engine is not correctly up!
> > > [4972.193s] resume of IP block <smu> failed -5
> > > [4972.193s] GPU reset end with ret = -5
> > > ```
> > >
> > >
> > >
> > >
> > > On Fri, Apr 10, 2026 at 3:09 AM Ilpo Järvinen
> > > <ilpo.jarvinen@linux.intel.com> wrote:
> > > >
> > > > On Thu, 9 Apr 2026, Geramy Loveless wrote:
> > > >
> > > > > When pbus_reassign_bridge_resources() walks up the bridge
> > > > > hierarchy
> > > > > to expand a window (e.g. for resizable BAR), it refuses to
> > > > > release
> > > > > any bridge window that has children.  This prevents BAR resize on
> > > > > devices behind multi-port PCIe switches (such as Thunderbolt
> > > > > docks)
> > > > > where empty sibling downstream ports hold small reservations that
> > > > > block the parent bridge window from being freed and re-sized.
> > > > >
> > > > > Add pci_bus_subtree_empty() to check whether a bus subtree
> > > > > contains
> > > > > any assigned device BARs, and pci_bus_release_empty_bridges() to
> > > > > release bridge window resources of empty sibling bridges, saving
> > > > > them to the rollback list so failures can be properly unwound.
> > > > >
> > > > > In pbus_reassign_bridge_resources(), call
> > > > > pci_bus_release_empty_bridges()
> > > > > before checking res->child, so empty sibling windows are cleared
> > > > > first
> > > > > and the parent window can then be released and grown.
> > > > >
> > > > > Uses PCI bus/device iterators rather than walking the raw
> > > > > resource
> > > > > tree, which avoids issues with stale sibling pointers after
> > > > > resource
> > > > > release.
> > > >
> > > > This paragraph can be dropped. And it's not exactly correct either
> > > > as
> > > > the pointers are only stale for resource entries that reside
> > > > outside of
> > > > the resource tree (after they've been released in a specific way)
> > > > so if
> > > > you start from a resource tree entry, you should never encounter a
> > > > stale
> > > > pointer.
> > > >
> > > > > Suggested-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
> > > > > Signed-off-by: Geramy Loveless <gloveless@jqluv.com>
> > > > > ---
> > > > >  drivers/pci/setup-bus.c | 99
> > > > > ++++++++++++++++++++++++++++++++++++++++-
> > > > >  1 file changed, 97 insertions(+), 2 deletions(-)
> > > > >
> > > > > diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
> > > > > index 4cf120ebe5a..7a182cd7e4d 100644
> > > > > --- a/drivers/pci/setup-bus.c
> > > > > +++ b/drivers/pci/setup-bus.c
> > > > > @@ -2292,6 +2292,94 @@ void
> > > > > pci_assign_unassigned_bridge_resources(struct pci_dev *bridge)
> > > > >  }
> > > > >  EXPORT_SYMBOL_GPL(pci_assign_unassigned_bridge_resources);
> > > > >
> > > > > +/*
> > > > > + * pci_bus_subtree_empty - Check whether a bus subtree has any
> > > > > assigned
> > > > > + * non-bridge device resources.
> > > > > + * @bus: PCI bus to check
> > > > > + *
> > > > > + * Returns true if no device on @bus or its descendant buses has
> > > > > any
> > > > > + * assigned BARs (bridge window resources are not considered).
> > > > > + */
> > > > > +static bool pci_bus_subtree_empty(struct pci_bus *bus)
> > > > > +{
> > > > > +     struct pci_dev *dev;
> > > > > +
> > > > > +     list_for_each_entry(dev, &bus->devices, bus_list) {
> > > > > +             struct resource *r;
> > > > > +             unsigned int i;
> > > > > +
> > > > > +             pci_dev_for_each_resource(dev, r, i) {
> > > > > +                     if (i >= PCI_BRIDGE_RESOURCES)
> > > > > +                             break;
> > > > > +                     if (resource_assigned(r))
> > > > > +                             return false;
> > > > > +             }
> > > > > +
> > > > > +             if (dev->subordinate &&
> > > > > +                 !pci_bus_subtree_empty(dev->subordinate))
> > > > > +                     return false;
> > > > > +     }
> > > > > +
> > > > > +     return true;
> > > > > +}
> > > > > +
> > > > > +/*
> > > > > + * pci_bus_release_empty_bridges - Release bridge window
> > > > > resources of
> > > > > + * empty sibling bridges so the parent window can be freed and
> > > > > re-sized.
> > > > > + * @bus: PCI bus whose child bridges to scan
> > > > > + * @b_win: Parent bridge window resource; only children of this
> > > > > window
> > > > > + *         are released
> > > > > + * @saved: List to save released resources for rollback
> > > > > + *
> > > > > + * For each PCI-to-PCI bridge on @bus whose subtree is empty (no
> > > > > assigned
> > > > > + * device BARs), releases bridge window resources that are
> > > > > children of
> > > > > + * @b_win, saving them for rollback via @saved.
> > > > > + *
> > > > > + * Returns 0 on success, negative errno on failure.
> > > > > + */
> > > > > +static int pci_bus_release_empty_bridges(struct pci_bus *bus,
> > > > > +                                      struct resource *b_win,
> > > > > +                                      struct list_head *saved)
> > > > > +{
> > > > > +     struct pci_dev *dev;
> > > > > +
> > > > > +     list_for_each_entry(dev, &bus->devices, bus_list) {
> > > > > +             struct resource *r;
> > > > > +             unsigned int i;
> > > > > +
> > > > > +             if (!dev->subordinate)
> > > > > +                     continue;
> > > > > +
> > > > > +             if ((dev->class >> 8) != PCI_CLASS_BRIDGE_PCI)
> > > > > +                     continue;
> > > >
> > > > I suppose dev->subordinate check is enough for what we're doing so
> > > > this
> > > > looks redundant.
> > > >
> > > > > +
> > > > > +             if (!pci_bus_subtree_empty(dev->subordinate))
> > > > > +                     continue;
> > > > > +
> > > > > +             pci_dev_for_each_resource(dev, r, i) {
> > > > > +                     int ret;
> > > > > +
> > > > > +                     if (!pci_resource_is_bridge_win(i))
> > > > > +                             continue;
> > > > > +
> > > > > +                     if (!resource_assigned(r))
> > > > > +                             continue;
> > > > > +
> > > > > +                     if (r->parent != b_win)
> > > > > +                             continue;
> > > > > +
> > > > > +                     ret = pci_dev_res_add_to_list(saved, dev,
> > > > > r, 0, 0);
> > > > > +                     if (ret)
> > > > > +                             return ret;
> > > > > +
> > > > > +                     release_child_resources(r);
> > > >
> > > > Unfortunately you cannot call this low-level function because it
> > > > recursively frees child resources which means you won't be able to
> > > > rollback them as they were not added to the saved list.
> > > >
> > > > I think the release algorithm should basically do this:
> > > >
> > > > - Recurse to the subordinate buses
> > > > - Loop through bridge window resources of this bus
> > > >         - Skip resources that are not assigned or are not parented
> > > > by b_win
> > > >         - If the resource still has childs, leave the resource
> > > > alone
> > > >           (+ log it for easier troubleshooting these cases; any
> > > > failure
> > > >              will also cascade to upstream so it may be possible to
> > > >              shortcut something but it will also make the algorithm
> > > > more
> > > >              complicated)
> > > >         - Save and free the resource
> > > >
> > > > It might be better to move some of the code from
> > > > pbus_reassign_bridge_resources() here as there's overlap with the
> > > > sketched
> > > > algorithm (but I'm not sure until I see the updated version but
> > > > keep this
> > > > in mind).
> > > >
> > > > Doing pci_bus_subtree_empty() before any removal is fine with me,
> > > > but I
> > > > see it just an optimization.
> > > >
> > > > > +                     pci_release_resource(dev, i);
> > > > > +             }
> > > > > +     }
> > > > > +
> > > > > +     return 0;
> > > > > +}
> > > > > +
> > > > >  /*
> > > > >   * Walk to the root bus, find the bridge window relevant for
> > > > > @res and
> > > > >   * release it when possible. If the bridge window contains
> > > > > assigned
> > > > > @@ -2316,7 +2404,14 @@ static int
> > > > > pbus_reassign_bridge_resources(struct pci_bus *bus, struct
> > > > > resource *
> > > > >
> > > > >               i = pci_resource_num(bridge, res);
> > > > >
> > > > > -             /* Ignore BARs which are still in use */
> > > >
> > > > I don't know why you removed this comment (I admit though "BARs"
> > > > could
> > > > have been worded better as it's bridge windows we're dealing here).
> > > >
> > > > > +             /* Release empty sibling bridge windows first */
> > > > > +             if (bridge->subordinate) {
> > > > > +                     ret = pci_bus_release_empty_bridges(
> > > > > +                                     bridge->subordinate, res,
> > > > > saved);
> > > >
> > > > First arg fits to the previous line.
> > > >
> > > > Align the second line to (.
> > > >
> > > > But consider also rearranging code as I mentioned above.
> > > >
> > > > > +                     if (ret)
> > > > > +                             return ret;
> > > >
> > > > Consider proceeding with the resize even if something failed as
> > > > there are
> > > > cases where the bridge windows are large enough (admittedly, you
> > > > seem to
> > > > only bail out in case of alloc error).
> > > >
> > > > In to the same vein, there seems to be one existing goto restore
> > > > (that was
> > > > added by me), which could also probably do continue instead (but
> > > > changing
> > > > it would be worth another patch).
> > > >
> > > > > +             }
> > > > > +
> > > > >               if (!res->child) {
> > > > >                       ret = pci_dev_res_add_to_list(saved,
> > > > > bridge, res, 0, 0);
> > > > >                       if (ret)
> > > > > @@ -2327,7 +2422,7 @@ static int
> > > > > pbus_reassign_bridge_resources(struct pci_bus *bus, struct
> > > > > resource *
> > > > >                       const char *res_name =
> > > > > pci_resource_name(bridge, i);
> > > > >
> > > > >                       pci_warn(bridge,
> > > > > -                              "%s %pR: was not released (still
> > > > > contains assigned resources)\n",
> > > > > +                              "%s %pR: not released, active
> > > > > children present\n",
> > > > >                                res_name, res);
> > > > >               }
> > > > >
> > > > >
> > > >
> > > > --
> > > >  i.

Re: [PATCH v2] PCI: release empty sibling bridge windows during rebar expansion

[Cristian Cocos]

In *Windows*, yes, GPUs have indeed been supported in eGPU
configuration, though not so much in Linux(!). The troubles we are
currently experiencing, esp. concerning hot-plugging, have not been
reported simply because eGPUs in Linux has, so far, been a very niche,
limited application. In other words, it's not that these issues have
not been around, it's just that they have not been REPORTED due to a
very limited pool of users running Linux in eGPU configuration. GPU
manufacturers have thus focused on patching up bugs in Windows, while
hot-plugging-Thunderbolt-eGPUs-in-Linux has fallen onto the back burner
due to its small market share. As such, I doubt that TB5 has anything
to do with all this.

On Fri, 2026-04-10 at 16:01 -0700, Geramy Loveless wrote:
> So after a while of doing investigations I keep thinking, gpus have
> supported hotplug or so i would believe for a while ever since at
> least thunderbolt 4 and possibly older styles too of hotplug as well
> as dual gpu machines like laptops that have a dedicated and
> integrated. I would think all of these problems we are seeing is most
> likely due to something very specific to thunderbolt 5. So here is a
> theory.
> 
> Root port ACPI device: \_SB_.PCI0.GP10 at address 0x00030002 = PCI
> 00:03.2
> Its _DSD (Device-Specific Data) contains only:
> "FundamentalDeviceResetTriggeredOnD3ToD0" = 1
> 
> No ExternalFacingPort property. The firmware gives it a D3-to-D0
> reset
> hint but never declares it as external-facing.
> For comparison, a properly configured system would have:
> 
> Package {"ExternalFacingPort", 1},
> Package {"DmaProperty", 1},
> 
> The complete proof chain:
> 
> DSDT (/tmp/dsdt.dsl:4194-4205): GP10 (00:03.2) _DSD has no
> ExternalFacingPort
> pci_acpi_set_external_facing() (pci-acpi.c:1422):
> device_property_read_u8("ExternalFacingPort") fails → external_facing
> stays 0
> pci_set_removable() (probe.c:1780): parent not removable → falls
> through
> arch_pci_dev_is_removable() (arch/x86/pci/acpi.c:353):
> !root->external_facing → returns false
> Result: dev->removable never set to DEVICE_REMOVABLE on any device in
> the chain
> 
> Sysfs proof:
>   00:03.2: removable=N/A, external_facing=N/A
>   65:00.0: removable=N/A
>   66:03.0: removable=N/A  (TB bridge with HotPlug+ Surprise+)
>   93:00.0: removable=N/A
>   97:00.0: removable=N/A  (eGPU — dev_is_removable() returns false)
> 
> I'm not sure its correct so much but I have patched up the amdgpu
> driver myself to use what its using now and "||
> pci_is_thunderbolt_attached"
> This might be entirely completely wrong :D but I figured I would see
> what happens.
> 
> On Fri, Apr 10, 2026 at 12:14 PM Geramy Loveless
> <gloveless@jqluv.com> wrote:
> > 
> > Oh wow, would you mind me adding you to my existing bug ticket with
> > AMD?
> > I am having the exact same problem, they are working on some fixes
> > for
> > some of the bugs I ran across. I ended adding in some device active
> > and alive checks basically to stop the driver from crashing the
> > kernel. But I could add you in there so we can get both of our logs
> > which are the same problem I believe in the same "ticket" if you
> > will.
> > It might also help having two views of the same issue. We have the
> > same dock.
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > On Fri, Apr 10, 2026 at 11:58 AM Cristian Cocos <cristi@ieee.org>
> > wrote:
> > > 
> > > My experience with my 9060XT Thunderbolt eGPU is that the current
> > > amdgpu driver is full of bugs, and this *specifically* in a
> > > Thunderbolt
> > > eGPU configuration. I have attempted to document some of these
> > > bugs
> > > here:
> > > https://pcforum.amd.com/s/question/0D5Pd00001S3Av9KAF/linux-
> > > 9060xt-egpuoverthunderbolt-bugs-galore
> > > 
> > > Apologies for posting this here, as most of these may not be
> > > relevant
> > > to ReBAR, yet an AMD representative may still benefit from this
> > > multiple bug report.
> > > 
> > > C
> > > 
> > > On Fri, 2026-04-10 at 10:53 -0700, Geramy Loveless wrote:
> > > >  I'm going to loop in Christian Koenig over at AMD he has been
> > > > working
> > > > with me on resolving or attempting to figure out whats going on
> > > > with
> > > > my gfx1201 connected to a tb5 dock to the host.
> > > > I am currently having problems with the GPU basically loosing
> > > > MMIO
> > > > and
> > > > crashing randomly. This recent patch change I believe helped
> > > > but its
> > > > really hard to say at this point.
> > > > Without this patch of course the bar size would be 256MB and
> > > > cause
> > > > huge performance problems or feature loss. I am able to load up
> > > > AI
> > > > models and run workloads at nearly 100% gpu usage, i'm seeing
> > > > 205W
> > > > power draw out of the maximum 300W. But after sustained load I
> > > > still
> > > > get a crash.
> > > > 
> > > > Maybe you would have an idea as to what is causing that crash
> > > > or
> > > > where
> > > > I should be looking to find the cause?
> > > > Here are some relevant logs, from what I can tell something is
> > > > going
> > > > on with MMIO, but the config bar as i understand it is still
> > > > alive.
> > > > This let me to believe maybe the router was getting put into
> > > > suspend
> > > > mode which wouldnt make sense for a GPU that is active and busy
> > > > because the pcie tunnel would be active.
> > > > 
> > > > Any advice or tips would be helpful thank you for the
> > > > suggestions I
> > > > will get started on writing the patch based on those
> > > > recommendations.
> > > > 
> > > > ## SMU Firmware Version
> > > > 
> > > > ```
> > > > smu driver if version = 0x0000002e
> > > > smu fw if version = 0x00000032
> > > > smu fw program = 0
> > > > smu fw version = 0x00684b00 (104.75.0)
> > > > ```
> > > > 
> > > > Note: Driver interface version (0x2e / 46) does not match
> > > > firmware
> > > > interface version (0x32 / 50).
> > > > 
> > > > ## PCI Topology
> > > > 
> > > > ```
> > > > 65:00.0 PCI bridge: Intel Barlow Ridge Host 80G (rev 84)
> > > > 66:00.0 PCI bridge: Intel Barlow Ridge Host 80G (rev 84) → NHI
> > > > 66:01.0 PCI bridge: Intel Barlow Ridge Host 80G (rev 84) →
> > > > empty
> > > > hotplug port
> > > > 66:02.0 PCI bridge: Intel Barlow Ridge Host 80G (rev 84) → USB
> > > > 66:03.0 PCI bridge: Intel Barlow Ridge Host 80G (rev 84) → dock
> > > > 93:00.0 PCI bridge: Intel Barlow Ridge Hub 80G (rev 85) → dock
> > > > switch
> > > > 94:00.0 PCI bridge: Intel Barlow Ridge Hub 80G (rev 85) →
> > > > downstream
> > > > 95:00.0 PCI bridge: AMD Navi 10 XL Upstream Port (rev 24)
> > > > 96:00.0 PCI bridge: AMD Navi 10 XL Downstream Port (rev 24)
> > > > 97:00.0 VGA: AMD [1002:7551] (rev c0) ← GPU
> > > > 97:00.1 Audio: AMD [1002:ab40]
> > > > ```
> > > > 
> > > > ## Workload
> > > > 
> > > > GPU compute via llama.cpp (ROCm/HIP backend), running
> > > > Qwen3.5-35B-A3B-Q4_K_M.gguf model (20.49 GiB, fully offloaded
> > > > to
> > > > VRAM). Flash attention enabled, 128K context, 32 threads.
> > > > 
> > > > ## Crash Timeline
> > > > 
> > > > All timestamps from `dmesg -T`, kernel boot-relative times in
> > > > brackets.
> > > > 
> > > > ### GPU initialization (successful)
> > > > 
> > > > ```
> > > > [603.644s] GPU probe: IP DISCOVERY 0x1002:0x7551
> > > > [603.653s] Detected IP block: smu_v14_0_0, gfx_v12_0_0
> > > > [603.771s] Detected VRAM RAM=32624M, BAR=32768M, RAM width
> > > > 256bits
> > > > GDDR6
> > > > [604.014s] SMU driver IF 0x2e, FW IF 0x32, FW version 104.75.0
> > > > [604.049s] SMU is initialized successfully!
> > > > [604.119s] Runtime PM manually disabled (amdgpu.runpm=0)
> > > > [604.119s] Initialized amdgpu 3.64.0 for 0000:97:00.0
> > > > ```
> > > > 
> > > > ### SMU stops responding [T+4238s after init, ~70 minutes]
> > > > 
> > > > ```
> > > > [4841.828s] SMU: No response msg_reg: 12 resp_reg: 0
> > > > [4841.828s] [smu_v14_0_2_get_power_profile_mode] Failed to get
> > > > activity monitor!
> > > > [4849.393s] SMU: No response msg_reg: 12 resp_reg: 0
> > > > [4849.393s] Failed to export SMU metrics table!
> > > > ```
> > > > 
> > > > 15 consecutive `SMU: No response` messages logged between
> > > > [4841s] and
> > > > [4948s], approximately every 7-8 seconds. All with `msg_reg: 12
> > > > resp_reg: 0`. Failed operations include:
> > > > - `smu_v14_0_2_get_power_profile_mode` — Failed to get activity
> > > > monitor
> > > > - `Failed to export SMU metrics table`
> > > > - `Failed to get current clock freq`
> > > > 
> > > > ### Page faults begin [T+4349s after init, ~111s after first
> > > > SMU
> > > > failure]
> > > > 
> > > > ```
> > > > [4948.927s] [gfxhub] page fault (src_id:0 ring:40 vmid:9
> > > > pasid:108)
> > > > Process llama-cli pid 35632
> > > > GCVM_L2_PROTECTION_FAULT_STATUS: 0x00941051
> > > > Faulty UTCL2 client ID: TCP (0x8)
> > > > PERMISSION_FAULTS: 0x5
> > > > WALKER_ERROR: 0x0
> > > > MAPPING_ERROR: 0x0
> > > > RW: 0x1 (write)
> > > > ```
> > > > 
> > > > 10 page faults logged at [4948s], all from TCP (Texture Cache
> > > > Pipe),
> > > > all PERMISSION_FAULTS=0x5, WALKER_ERROR=0x0, MAPPING_ERROR=0x0.
> > > > 7
> > > > unique faulting addresses:
> > > > - 0x000072ce90828000
> > > > - 0x000072ce90a88000
> > > > - 0x000072ce90a89000
> > > > - 0x000072ce90cde000
> > > > - 0x000072ce90ce1000
> > > > - 0x000072ce90f51000
> > > > - 0x000072ce90f52000
> > > > 
> > > > ### MES failure and GPU reset [T+4349s]
> > > > 
> > > > ```
> > > > [4952.809s] MES(0) failed to respond to msg=REMOVE_QUEUE
> > > > [4952.809s] failed to remove hardware queue from MES,
> > > > doorbell=0x1806
> > > > [4952.809s] MES might be in unrecoverable state, issue a GPU
> > > > reset
> > > > [4952.809s] Failed to evict queue 4
> > > > [4952.809s] Failed to evict process queues
> > > > [4952.809s] GPU reset begin!. Source: 3
> > > > ```
> > > > 
> > > > ### GPU reset fails
> > > > 
> > > > ```
> > > > [4953.121s] Failed to evict queue 4
> > > > [4953.121s] Failed to suspend process pid 28552
> > > > [4953.121s] remove_all_kfd_queues_mes: Failed to remove queue 3
> > > > for
> > > > dev 62536
> > > > ```
> > > > 
> > > > 6 MES(1) REMOVE_QUEUE failures, each timing out after ~2.5
> > > > seconds:
> > > > ```
> > > > [4955.720s] MES(1) failed to respond to msg=REMOVE_QUEUE →
> > > > failed to
> > > > unmap legacy queue
> > > > [4958.283s] MES(1) failed to respond to msg=REMOVE_QUEUE →
> > > > failed to
> > > > unmap legacy queue
> > > > [4960.847s] MES(1) failed to respond to msg=REMOVE_QUEUE →
> > > > failed to
> > > > unmap legacy queue
> > > > [4963.411s] MES(1) failed to respond to msg=REMOVE_QUEUE →
> > > > failed to
> > > > unmap legacy queue
> > > > [4965.976s] MES(1) failed to respond to msg=REMOVE_QUEUE →
> > > > failed to
> > > > unmap legacy queue
> > > > [4968.540s] MES(1) failed to respond to msg=REMOVE_QUEUE →
> > > > failed to
> > > > unmap legacy queue
> > > > ```
> > > > 
> > > > ### PSP suspend fails
> > > > 
> > > > ```
> > > > [4971.164s] psp gfx command LOAD_IP_FW(0x6) failed and response
> > > > status is (0x0)
> > > > [4971.164s] Failed to terminate ras ta
> > > > [4971.164s] suspend of IP block <psp> failed -22
> > > > ```
> > > > 
> > > > ### Suspend unwind fails — SMU not ready
> > > > 
> > > > ```
> > > > [4971.164s] SMU is resuming...
> > > > [4971.164s] SMC is not ready
> > > > [4971.164s] SMC engine is not correctly up!
> > > > [4971.164s] resume of IP block <smu> failed -5
> > > > [4971.164s] amdgpu_device_ip_resume_phase2 failed during
> > > > unwind: -5
> > > > [4971.164s] GPU pre asic reset failed with err, -22 for drm
> > > > dev,
> > > > 0000:97:00.0
> > > > ```
> > > > 
> > > > ### MODE1 reset — SMU still dead
> > > > 
> > > > ```
> > > > [4971.164s] MODE1 reset
> > > > [4971.164s] GPU mode1 reset
> > > > [4971.164s] GPU smu mode1 reset
> > > > [4972.193s] GPU reset succeeded, trying to resume
> > > > [4972.193s] VRAM is lost due to GPU reset!
> > > > [4972.193s] SMU is resuming...
> > > > [4972.193s] SMC is not ready
> > > > [4972.193s] SMC engine is not correctly up!
> > > > [4972.193s] resume of IP block <smu> failed -5
> > > > [4972.193s] GPU reset end with ret = -5
> > > > ```
> > > > 
> > > > 
> > > > 
> > > > 
> > > > On Fri, Apr 10, 2026 at 3:09 AM Ilpo Järvinen
> > > > <ilpo.jarvinen@linux.intel.com> wrote:
> > > > > 
> > > > > On Thu, 9 Apr 2026, Geramy Loveless wrote:
> > > > > 
> > > > > > When pbus_reassign_bridge_resources() walks up the bridge
> > > > > > hierarchy
> > > > > > to expand a window (e.g. for resizable BAR), it refuses to
> > > > > > release
> > > > > > any bridge window that has children.  This prevents BAR
> > > > > > resize on
> > > > > > devices behind multi-port PCIe switches (such as
> > > > > > Thunderbolt
> > > > > > docks)
> > > > > > where empty sibling downstream ports hold small
> > > > > > reservations that
> > > > > > block the parent bridge window from being freed and re-
> > > > > > sized.
> > > > > > 
> > > > > > Add pci_bus_subtree_empty() to check whether a bus subtree
> > > > > > contains
> > > > > > any assigned device BARs, and
> > > > > > pci_bus_release_empty_bridges() to
> > > > > > release bridge window resources of empty sibling bridges,
> > > > > > saving
> > > > > > them to the rollback list so failures can be properly
> > > > > > unwound.
> > > > > > 
> > > > > > In pbus_reassign_bridge_resources(), call
> > > > > > pci_bus_release_empty_bridges()
> > > > > > before checking res->child, so empty sibling windows are
> > > > > > cleared
> > > > > > first
> > > > > > and the parent window can then be released and grown.
> > > > > > 
> > > > > > Uses PCI bus/device iterators rather than walking the raw
> > > > > > resource
> > > > > > tree, which avoids issues with stale sibling pointers after
> > > > > > resource
> > > > > > release.
> > > > > 
> > > > > This paragraph can be dropped. And it's not exactly correct
> > > > > either
> > > > > as
> > > > > the pointers are only stale for resource entries that reside
> > > > > outside of
> > > > > the resource tree (after they've been released in a specific
> > > > > way)
> > > > > so if
> > > > > you start from a resource tree entry, you should never
> > > > > encounter a
> > > > > stale
> > > > > pointer.
> > > > > 
> > > > > > Suggested-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
> > > > > > Signed-off-by: Geramy Loveless <gloveless@jqluv.com>
> > > > > > ---
> > > > > >  drivers/pci/setup-bus.c | 99
> > > > > > ++++++++++++++++++++++++++++++++++++++++-
> > > > > >  1 file changed, 97 insertions(+), 2 deletions(-)
> > > > > > 
> > > > > > diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-
> > > > > > bus.c
> > > > > > index 4cf120ebe5a..7a182cd7e4d 100644
> > > > > > --- a/drivers/pci/setup-bus.c
> > > > > > +++ b/drivers/pci/setup-bus.c
> > > > > > @@ -2292,6 +2292,94 @@ void
> > > > > > pci_assign_unassigned_bridge_resources(struct pci_dev
> > > > > > *bridge)
> > > > > >  }
> > > > > >  EXPORT_SYMBOL_GPL(pci_assign_unassigned_bridge_resources);
> > > > > > 
> > > > > > +/*
> > > > > > + * pci_bus_subtree_empty - Check whether a bus subtree has
> > > > > > any
> > > > > > assigned
> > > > > > + * non-bridge device resources.
> > > > > > + * @bus: PCI bus to check
> > > > > > + *
> > > > > > + * Returns true if no device on @bus or its descendant
> > > > > > buses has
> > > > > > any
> > > > > > + * assigned BARs (bridge window resources are not
> > > > > > considered).
> > > > > > + */
> > > > > > +static bool pci_bus_subtree_empty(struct pci_bus *bus)
> > > > > > +{
> > > > > > +     struct pci_dev *dev;
> > > > > > +
> > > > > > +     list_for_each_entry(dev, &bus->devices, bus_list) {
> > > > > > +             struct resource *r;
> > > > > > +             unsigned int i;
> > > > > > +
> > > > > > +             pci_dev_for_each_resource(dev, r, i) {
> > > > > > +                     if (i >= PCI_BRIDGE_RESOURCES)
> > > > > > +                             break;
> > > > > > +                     if (resource_assigned(r))
> > > > > > +                             return false;
> > > > > > +             }
> > > > > > +
> > > > > > +             if (dev->subordinate &&
> > > > > > +                 !pci_bus_subtree_empty(dev->subordinate))
> > > > > > +                     return false;
> > > > > > +     }
> > > > > > +
> > > > > > +     return true;
> > > > > > +}
> > > > > > +
> > > > > > +/*
> > > > > > + * pci_bus_release_empty_bridges - Release bridge window
> > > > > > resources of
> > > > > > + * empty sibling bridges so the parent window can be freed
> > > > > > and
> > > > > > re-sized.
> > > > > > + * @bus: PCI bus whose child bridges to scan
> > > > > > + * @b_win: Parent bridge window resource; only children of
> > > > > > this
> > > > > > window
> > > > > > + *         are released
> > > > > > + * @saved: List to save released resources for rollback
> > > > > > + *
> > > > > > + * For each PCI-to-PCI bridge on @bus whose subtree is
> > > > > > empty (no
> > > > > > assigned
> > > > > > + * device BARs), releases bridge window resources that are
> > > > > > children of
> > > > > > + * @b_win, saving them for rollback via @saved.
> > > > > > + *
> > > > > > + * Returns 0 on success, negative errno on failure.
> > > > > > + */
> > > > > > +static int pci_bus_release_empty_bridges(struct pci_bus
> > > > > > *bus,
> > > > > > +                                      struct resource
> > > > > > *b_win,
> > > > > > +                                      struct list_head
> > > > > > *saved)
> > > > > > +{
> > > > > > +     struct pci_dev *dev;
> > > > > > +
> > > > > > +     list_for_each_entry(dev, &bus->devices, bus_list) {
> > > > > > +             struct resource *r;
> > > > > > +             unsigned int i;
> > > > > > +
> > > > > > +             if (!dev->subordinate)
> > > > > > +                     continue;
> > > > > > +
> > > > > > +             if ((dev->class >> 8) !=
> > > > > > PCI_CLASS_BRIDGE_PCI)
> > > > > > +                     continue;
> > > > > 
> > > > > I suppose dev->subordinate check is enough for what we're
> > > > > doing so
> > > > > this
> > > > > looks redundant.
> > > > > 
> > > > > > +
> > > > > > +             if (!pci_bus_subtree_empty(dev->subordinate))
> > > > > > +                     continue;
> > > > > > +
> > > > > > +             pci_dev_for_each_resource(dev, r, i) {
> > > > > > +                     int ret;
> > > > > > +
> > > > > > +                     if (!pci_resource_is_bridge_win(i))
> > > > > > +                             continue;
> > > > > > +
> > > > > > +                     if (!resource_assigned(r))
> > > > > > +                             continue;
> > > > > > +
> > > > > > +                     if (r->parent != b_win)
> > > > > > +                             continue;
> > > > > > +
> > > > > > +                     ret = pci_dev_res_add_to_list(saved,
> > > > > > dev,
> > > > > > r, 0, 0);
> > > > > > +                     if (ret)
> > > > > > +                             return ret;
> > > > > > +
> > > > > > +                     release_child_resources(r);
> > > > > 
> > > > > Unfortunately you cannot call this low-level function because
> > > > > it
> > > > > recursively frees child resources which means you won't be
> > > > > able to
> > > > > rollback them as they were not added to the saved list.
> > > > > 
> > > > > I think the release algorithm should basically do this:
> > > > > 
> > > > > - Recurse to the subordinate buses
> > > > > - Loop through bridge window resources of this bus
> > > > >         - Skip resources that are not assigned or are not
> > > > > parented
> > > > > by b_win
> > > > >         - If the resource still has childs, leave the
> > > > > resource
> > > > > alone
> > > > >           (+ log it for easier troubleshooting these cases;
> > > > > any
> > > > > failure
> > > > >              will also cascade to upstream so it may be
> > > > > possible to
> > > > >              shortcut something but it will also make the
> > > > > algorithm
> > > > > more
> > > > >              complicated)
> > > > >         - Save and free the resource
> > > > > 
> > > > > It might be better to move some of the code from
> > > > > pbus_reassign_bridge_resources() here as there's overlap with
> > > > > the
> > > > > sketched
> > > > > algorithm (but I'm not sure until I see the updated version
> > > > > but
> > > > > keep this
> > > > > in mind).
> > > > > 
> > > > > Doing pci_bus_subtree_empty() before any removal is fine with
> > > > > me,
> > > > > but I
> > > > > see it just an optimization.
> > > > > 
> > > > > > +                     pci_release_resource(dev, i);
> > > > > > +             }
> > > > > > +     }
> > > > > > +
> > > > > > +     return 0;
> > > > > > +}
> > > > > > +
> > > > > >  /*
> > > > > >   * Walk to the root bus, find the bridge window relevant
> > > > > > for
> > > > > > @res and
> > > > > >   * release it when possible. If the bridge window contains
> > > > > > assigned
> > > > > > @@ -2316,7 +2404,14 @@ static int
> > > > > > pbus_reassign_bridge_resources(struct pci_bus *bus, struct
> > > > > > resource *
> > > > > > 
> > > > > >               i = pci_resource_num(bridge, res);
> > > > > > 
> > > > > > -             /* Ignore BARs which are still in use */
> > > > > 
> > > > > I don't know why you removed this comment (I admit though
> > > > > "BARs"
> > > > > could
> > > > > have been worded better as it's bridge windows we're dealing
> > > > > here).
> > > > > 
> > > > > > +             /* Release empty sibling bridge windows first
> > > > > > */
> > > > > > +             if (bridge->subordinate) {
> > > > > > +                     ret = pci_bus_release_empty_bridges(
> > > > > > +                                     bridge->subordinate,
> > > > > > res,
> > > > > > saved);
> > > > > 
> > > > > First arg fits to the previous line.
> > > > > 
> > > > > Align the second line to (.
> > > > > 
> > > > > But consider also rearranging code as I mentioned above.
> > > > > 
> > > > > > +                     if (ret)
> > > > > > +                             return ret;
> > > > > 
> > > > > Consider proceeding with the resize even if something failed
> > > > > as
> > > > > there are
> > > > > cases where the bridge windows are large enough (admittedly,
> > > > > you
> > > > > seem to
> > > > > only bail out in case of alloc error).
> > > > > 
> > > > > In to the same vein, there seems to be one existing goto
> > > > > restore
> > > > > (that was
> > > > > added by me), which could also probably do continue instead
> > > > > (but
> > > > > changing
> > > > > it would be worth another patch).
> > > > > 
> > > > > > +             }
> > > > > > +
> > > > > >               if (!res->child) {
> > > > > >                       ret = pci_dev_res_add_to_list(saved,
> > > > > > bridge, res, 0, 0);
> > > > > >                       if (ret)
> > > > > > @@ -2327,7 +2422,7 @@ static int
> > > > > > pbus_reassign_bridge_resources(struct pci_bus *bus, struct
> > > > > > resource *
> > > > > >                       const char *res_name =
> > > > > > pci_resource_name(bridge, i);
> > > > > > 
> > > > > >                       pci_warn(bridge,
> > > > > > -                              "%s %pR: was not released
> > > > > > (still
> > > > > > contains assigned resources)\n",
> > > > > > +                              "%s %pR: not released,
> > > > > > active
> > > > > > children present\n",
> > > > > >                                res_name, res);
> > > > > >               }
> > > > > > 
> > > > > > 
> > > > > 
> > > > > --
> > > > >  i.