[BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery

[BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery

[Shawn Jin]

[-- Attachment #1: Type: text/plain, Size: 3938 bytes --]

Hello,

I'm reporting a potential critical bug in the Linux kernel's PCIe resource allocation code that creates invalid bridge window limit addresses during hotplug re-enumeration after Secondary Bus Reset (SBR) recovery.

## AFFECTED KERNEL VERSIONS
- Confirmed: 5.15.0, 6.8.0 (Ubuntu 6.8.0-88-generic, 6.8.0-90-generic)
- Likely affected: All recent kernels including 6.19

## HARDWARE CONFIGURATION
Intel Ice Lake server with PCIe Gen5 switches and endpoints:

Topology 1:
  Root Port 96:01.0 → 98:00.0 → 99:01.0 → 9b:00.0 (NVIDIA L20 GPU)

Kernel parameter: pci=realloc=on

## PROBLEM DESCRIPTION

After performing Secondary Bus Reset on a PCIe switch port and clearing the reset bit, the kernel re-enumerates devices and assigns bridge window resources. However, the assigned memory window limit addresses are INVALID according to PCIe specification.

### Evidence from dmesg (Topology 1):

**Before SBR (correct allocation):**
```
[    6.636493] pci 0000:98:00.0: PCI bridge to [bus 99-9c]
[    6.636539] pci 0000:98:00.0:   bridge window [mem 0xe9600000-0xe96fffff]
[    6.636645] pci 0000:98:00.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]

[    6.644429] pci 0000:99:01.0: PCI bridge to [bus 9b]
[    6.644476] pci 0000:99:01.0:   bridge window [mem 0xe9600000-0xe96fffff]
[    6.644656] pci 0000:99:01.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]

[    6.654203] pci 0000:9b:00.0: [1e3e:0002] type 00 class 0x120000 PCIe Endpoint
[    6.654652] pci 0000:9b:00.0: BAR 0 [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
[    6.654666] pci 0000:9b:00.0: BAR 2 [mem 0xe9600000-0xe963ffff]
```

**After SBR clear (INVALID allocation):**
```
[  656.644184] pci 0000:98:00.0: bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]: assigned
[  656.644186] pci 0000:98:00.0: bridge window [mem 0xe9600000-0xe96fffff]: assigned
[  656.644188] pci 0000:99:01.0: bridge window [mem 0x13b000000000-0x13b7fffffffe 64bit pref]: assigned
[  656.644189] pci 0000:99:01.0: bridge window [mem 0xe9600000-0xe96ffffe]: assigned

[  656.644830] pci 0000:9b:00.0: BAR 0 [mem size 0x800000000 64bit pref]: can't assign; no space
[  656.644831] pci 0000:9b:00.0: BAR 0 [mem size 0x800000000 64bit pref]: failed to assign
// BAR2 can still be assigned because the size is only 256KB, while the min window in the bridge is 1MB
[  656.644832] pci 0000:9b:00.0: BAR 2 [mem 0xe9600000-0xe963ffff]: assigned

```

### Invalid Addresses Created by Kernel:
- `0x13b7ffffffff` (ends in 0xFFFE - **2 bytes short**)
- `0xe96ffffe`  (ends in 0xFFFE - **2 bytes short**)

## IMPACT

1. **Device initialization failure**: Endpoints cannot allocate required BARs
   ```
[  656.644830] pci 0000:9b:00.0: BAR 0 [mem size 0x800000000 64bit pref]: can't assign; no space
[  656.644831] pci 0000:9b:00.0: BAR 0 [mem size 0x800000000 64bit pref]: failed to assign
   ```

2. **Consistent across multiple hierarchies**: Affects different PCIe topologies independently

## REPRODUCTION

The attached script test_rc_sbr.sh.txt issues a SBR to the root port.

## SUSPECTED ROOT CAUSE

The bug appears to be in `drivers/pci/setup-bus.c`, likely in:
- `pci_bus_distribute_available_resources()`
- `adjust_bridge_window()`
- `pci_assign_unassigned_bridge_resources()`

The resource end address calculation appears to perform multiple subtractions:
1. Initial calculation: `res->end = res->start + size - 1` (correct)
2. During redistribution: Another subtraction occurs, creating `res->end = ... - 2`

## WORKAROUND ATTEMPTS

- `pci=realloc=on`: Does NOT fix the issue
- Manual remove/rescan from root: Does NOT fix the issue
- Initial boot allocation: Works correctly (bug only occurs during hotplug re-enumeration)

## REQUEST

I want to track how the bridge windows are allocated. Is there a way to enable additional kernel messages to show the path? Please investigate if this is a real kernel bug.

Thank you,
Shawn

[-- Attachment #2: test_rc_sbr.sh.txt --]
[-- Type: text/plain, Size: 1347 bytes --]

#!/bin/bash

# Function to display usage
usage() {
    echo "Usage: $0 -rp <ROOT_PORT_BDF> -usp <USP_BDF>"
    echo "Example: $0 -rp c6:01.0 -usp c7:00.0"
    exit 1
}

# Initialize variables
ROOT_PORT_BDF=""
USP_BDF=""

# Parse command-line arguments
while [[ $# -gt 0 ]]; do
    case $1 in
        -rp)
            ROOT_PORT_BDF="$2"
            shift 2
            ;;
        -usp)
            USP_BDF="$2"
            shift 2
            ;;
        -h|--help)
            usage
            ;;
        *)
            echo "Unknown option: $1"
            usage
            ;;
    esac
done

# Validate that both arguments are provided
if [ -z "$ROOT_PORT_BDF" ] || [ -z "$USP_BDF" ]; then
    echo "Error: Both -rp and -usp arguments are required"
    usage
fi

echo "Root Port BDF: $ROOT_PORT_BDF"
echo "USP BDF: $USP_BDF"
echo ""

# Remove the USP device
echo 1 | sudo tee /sys/bus/pci/devices/0000:${USP_BDF}/remove

# Trigger SBR via Bridge Control register
BRIDGE_CTL=$(sudo setpci -s ${ROOT_PORT_BDF} 0x3E.w)
BRIDGE_CTL_RESET=$(printf "0x%04x" $((0x$BRIDGE_CTL | 0x0040)))

echo "Asserting Secondary Bus Reset..."
sudo setpci -s ${ROOT_PORT_BDF} 0x3E.w=$BRIDGE_CTL_RESET
sleep 1

echo "De-asserting Secondary Bus Reset..."
sudo setpci -s ${ROOT_PORT_BDF} 0x3E.w=$BRIDGE_CTL
sleep 2

# Rescan PCI bus
echo 1 | sudo tee /sys/bus/pci/rescan

Re: [BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery

[Bjorn Helgaas]

[+cc Ilpo]

On Wed, Mar 11, 2026 at 10:00:39PM +0000, Shawn Jin wrote:
> Hello,
> 
> I'm reporting a potential critical bug in the Linux kernel's PCIe
> resource allocation code that creates invalid bridge window limit
> addresses during hotplug re-enumeration after Secondary Bus Reset
> (SBR) recovery.

Thanks for the report and the repro and debugging information!

> ## AFFECTED KERNEL VERSIONS
> - Confirmed: 5.15.0, 6.8.0 (Ubuntu 6.8.0-88-generic, 6.8.0-90-generic)
> - Likely affected: All recent kernels including 6.19

Do you know of any kernels that are *not* affected?  If you do, we
could bisect.

> ## HARDWARE CONFIGURATION
> Intel Ice Lake server with PCIe Gen5 switches and endpoints:
> 
> Topology 1:
>   Root Port 96:01.0 → 98:00.0 → 99:01.0 → 9b:00.0 (NVIDIA L20 GPU)
> 
> Kernel parameter: pci=realloc=on
> 
> ## PROBLEM DESCRIPTION
> 
> After performing Secondary Bus Reset on a PCIe switch port and
> clearing the reset bit, the kernel re-enumerates devices and assigns
> bridge window resources. However, the assigned memory window limit
> addresses are INVALID according to PCIe specification.
> 
> ### Evidence from dmesg (Topology 1):
> 
> **Before SBR (correct allocation):**
> ```
> [    6.636493] pci 0000:98:00.0: PCI bridge to [bus 99-9c]
> [    6.636539] pci 0000:98:00.0:   bridge window [mem 0xe9600000-0xe96fffff]
> [    6.636645] pci 0000:98:00.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
> 
> [    6.644429] pci 0000:99:01.0: PCI bridge to [bus 9b]
> [    6.644476] pci 0000:99:01.0:   bridge window [mem 0xe9600000-0xe96fffff]
> [    6.644656] pci 0000:99:01.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
> 
> [    6.654203] pci 0000:9b:00.0: [1e3e:0002] type 00 class 0x120000 PCIe Endpoint
> [    6.654652] pci 0000:9b:00.0: BAR 0 [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
> [    6.654666] pci 0000:9b:00.0: BAR 2 [mem 0xe9600000-0xe963ffff]
> ```
> 
> **After SBR clear (INVALID allocation):**
> ```
> [  656.644184] pci 0000:98:00.0: bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]: assigned
> [  656.644186] pci 0000:98:00.0: bridge window [mem 0xe9600000-0xe96fffff]: assigned
> [  656.644188] pci 0000:99:01.0: bridge window [mem 0x13b000000000-0x13b7fffffffe 64bit pref]: assigned
> [  656.644189] pci 0000:99:01.0: bridge window [mem 0xe9600000-0xe96ffffe]: assigned
> 
> [  656.644830] pci 0000:9b:00.0: BAR 0 [mem size 0x800000000 64bit pref]: can't assign; no space
> [  656.644831] pci 0000:9b:00.0: BAR 0 [mem size 0x800000000 64bit pref]: failed to assign
> // BAR2 can still be assigned because the size is only 256KB, while the min window in the bridge is 1MB
> [  656.644832] pci 0000:9b:00.0: BAR 2 [mem 0xe9600000-0xe963ffff]: assigned
> 
> ```
> 
> ### Invalid Addresses Created by Kernel:
> - `0x13b7ffffffff` (ends in 0xFFFE - **2 bytes short**)
> - `0xe96ffffe`  (ends in 0xFFFE - **2 bytes short**)
> 
> ## IMPACT
> 
> 1. **Device initialization failure**: Endpoints cannot allocate required BARs
>    ```
> [  656.644830] pci 0000:9b:00.0: BAR 0 [mem size 0x800000000 64bit pref]: can't assign; no space
> [  656.644831] pci 0000:9b:00.0: BAR 0 [mem size 0x800000000 64bit pref]: failed to assign
>    ```
> 
> 2. **Consistent across multiple hierarchies**: Affects different PCIe topologies independently
> 
> ## REPRODUCTION
> 
> The attached script test_rc_sbr.sh.txt issues a SBR to the root port.
> 
> ## SUSPECTED ROOT CAUSE
> 
> The bug appears to be in `drivers/pci/setup-bus.c`, likely in:
> - `pci_bus_distribute_available_resources()`
> - `adjust_bridge_window()`
> - `pci_assign_unassigned_bridge_resources()`
> 
> The resource end address calculation appears to perform multiple subtractions:
> 1. Initial calculation: `res->end = res->start + size - 1` (correct)
> 2. During redistribution: Another subtraction occurs, creating `res->end = ... - 2`
> 
> ## WORKAROUND ATTEMPTS
> 
> - `pci=realloc=on`: Does NOT fix the issue
> - Manual remove/rescan from root: Does NOT fix the issue
> - Initial boot allocation: Works correctly (bug only occurs during hotplug re-enumeration)
> 
> ## REQUEST
> 
> I want to track how the bridge windows are allocated. Is there a way
> to enable additional kernel messages to show the path? Please
> investigate if this is a real kernel bug.
> 
> Thank you,
> Shawn

> #!/bin/bash
> 
> # Function to display usage
> usage() {
>     echo "Usage: $0 -rp <ROOT_PORT_BDF> -usp <USP_BDF>"
>     echo "Example: $0 -rp c6:01.0 -usp c7:00.0"
>     exit 1
> }
> 
> # Initialize variables
> ROOT_PORT_BDF=""
> USP_BDF=""
> 
> # Parse command-line arguments
> while [[ $# -gt 0 ]]; do
>     case $1 in
>         -rp)
>             ROOT_PORT_BDF="$2"
>             shift 2
>             ;;
>         -usp)
>             USP_BDF="$2"
>             shift 2
>             ;;
>         -h|--help)
>             usage
>             ;;
>         *)
>             echo "Unknown option: $1"
>             usage
>             ;;
>     esac
> done
> 
> # Validate that both arguments are provided
> if [ -z "$ROOT_PORT_BDF" ] || [ -z "$USP_BDF" ]; then
>     echo "Error: Both -rp and -usp arguments are required"
>     usage
> fi
> 
> echo "Root Port BDF: $ROOT_PORT_BDF"
> echo "USP BDF: $USP_BDF"
> echo ""
> 
> # Remove the USP device
> echo 1 | sudo tee /sys/bus/pci/devices/0000:${USP_BDF}/remove
> 
> # Trigger SBR via Bridge Control register
> BRIDGE_CTL=$(sudo setpci -s ${ROOT_PORT_BDF} 0x3E.w)
> BRIDGE_CTL_RESET=$(printf "0x%04x" $((0x$BRIDGE_CTL | 0x0040)))
> 
> echo "Asserting Secondary Bus Reset..."
> sudo setpci -s ${ROOT_PORT_BDF} 0x3E.w=$BRIDGE_CTL_RESET
> sleep 1
> 
> echo "De-asserting Secondary Bus Reset..."
> sudo setpci -s ${ROOT_PORT_BDF} 0x3E.w=$BRIDGE_CTL
> sleep 2
> 
> # Rescan PCI bus
> echo 1 | sudo tee /sys/bus/pci/rescan

Re: [BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery

[Shawn Jin]

Hi Bjorn,

Thanks for the quick response.

[+cc Ilpo]

On Wed, Mar 11, 2026 at 10:00:39PM +0000, Shawn Jin wrote:
> Hello,
>
> I'm reporting a potential critical bug in the Linux kernel's PCIe
> resource allocation code that creates invalid bridge window limit
> addresses during hotplug re-enumeration after Secondary Bus Reset
> (SBR) recovery.

Thanks for the report and the repro and debugging information!

> ## AFFECTED KERNEL VERSIONS
> - Confirmed: 5.15.0, 6.8.0 (Ubuntu 6.8.0-88-generic, 6.8.0-90-generic)
> - Likely affected: All recent kernels including 6.19

Do you know of any kernels that are *not* affected?  If you do, we
could bisect.

<SJ> I haven't tried the latest kernel 6.19 yet. I'm planning to install the openSUSE Tumbleweed with kernel 6.19.

Thanks,
Shawn.

Re: [BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery

[Shawn Jin]

After enabling more debugging messages for PCI subsystem, it shows the pref bridge window was shrunken by 1 at [  796.604883].

Again, this is the kernel 6.8. I'll test 6.19 later using openSUSE Tumbleweed. This time the realloc was on.

[  796.604869] pcieport 0000:96:01.0: distributing available resources
[  796.604872] pci 0000:98:00.0: bridge window [mem 0x00100000-0x001fffff] extended by 0x0000000000100000
[  796.604876] pci 0000:99:00.0: bridge window [??? 0x00000000 flags 0x0] extended by 0x0000000000055555
[  796.604880] pci 0000:99:01.0: bridge window [mem 0x00100000-0x001fffff] shrunken by 0x00000000000aaaac
[  796.604883] pci 0000:99:01.0: bridge window [mem 0x800000000-0xfffffffff 64bit pref] shrunken by 0x0000000000000001
[  796.604886] pci 0000:99:02.0: bridge window [??? 0x00000000 flags 0x0] extended by 0x0000000000055555

[  796.607244] PCI: No. 2 try to assign unassigned res
[  796.607246] pci 0000:99:01.0: resource 15 [mem 0x134000000000-0x1347fffffffe 64bit pref] released
[  796.607247] pci 0000:99:01.0: PCI bridge to [bus 9b]
[  796.607599] pcieport 0000:96:01.0: bridge window [io  0x1000-0x0fff] to [bus 98-9c] add_size 1000
[  796.607600] pcieport 0000:96:01.0: distributing available resources
[  796.607602] pci 0000:99:01.0: bridge window [mem 0x800000000-0xfffffffff 64bit pref] shrunken by 0x0000000000000001
[  796.607603] pci 0000:99:02.0: bridge window [mem 0x00100000-0x001fffff] shrunken by 0x0000000000055554
[  796.607605] pcieport 0000:96:01.0: bridge window [io  size 0x1000]: can't assign; no space
[  796.607606] pcieport 0000:96:01.0: bridge window [io  size 0x1000]: failed to assign
[  796.607607] pcieport 0000:96:01.0: bridge window [io  size 0x1000]: can't assign; no space
[  796.607607] pcieport 0000:96:01.0: bridge window [io  size 0x1000]: failed to assign
[  796.607609] pci 0000:99:01.0: bridge window [mem 0x134000000000-0x1347fffffffe 64bit pref]: assigned
[  796.607610] pci 0000:99:00.0: bridge window [mem 0xe1b00000-0xe1bfffff]: assigned
[  796.607611] pci 0000:99:02.0: bridge window [mem size 0x000aaaac]: can't assign; no space
[  796.607613] pci 0000:99:02.0: bridge window [mem size 0x000aaaac]: failed to assign
[  796.607614] pci 0000:99:00.0: PCI bridge to [bus 9a]
[  796.607832] pci 0000:99:00.0:   bridge window [mem 0xe1b00000-0xe1bfffff]
[  796.608262] pci 0000:9b:00.0: BAR 0 [mem size 0x800000000 64bit pref]: can't assign; no space
[  796.608264] pci 0000:9b:00.0: BAR 0 [mem size 0x800000000 64bit pref]: failed to assign
[  796.608265] pci 0000:99:01.0: PCI bridge to [bus 9b]
[  796.608473] pci 0000:99:01.0:   bridge window [mem 0xe1a00000-0xe1a55553]
[  796.608615] pci 0000:99:01.0:   bridge window [mem 0x134000000000-0x1347fffffffe 64bit pref]

Thanks,
Shawn.
________________________________________
From: Shawn Jin <shawn.jin@asteralabs.com>
Sent: Wednesday, March 11, 2026 5:02 PM
To: Bjorn Helgaas <helgaas@kernel.org>
Cc: Bjorn Helgaas <bhelgaas@google.com>; linux-kernel@vger.kernel.org <linux-kernel@vger.kernel.org>; linux-pci@vger.kernel.org <linux-pci@vger.kernel.org>; Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Subject: Re: [BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery
 
Hi Bjorn,

Thanks for the quick response.

[+cc Ilpo]

On Wed, Mar 11, 2026 at 10:00:39PM +0000, Shawn Jin wrote:
> Hello,
>
> I'm reporting a potential critical bug in the Linux kernel's PCIe
> resource allocation code that creates invalid bridge window limit
> addresses during hotplug re-enumeration after Secondary Bus Reset
> (SBR) recovery.

Thanks for the report and the repro and debugging information!

> ## AFFECTED KERNEL VERSIONS
> - Confirmed: 5.15.0, 6.8.0 (Ubuntu 6.8.0-88-generic, 6.8.0-90-generic)
> - Likely affected: All recent kernels including 6.19

Do you know of any kernels that are *not* affected?  If you do, we
could bisect.

<SJ> I haven't tried the latest kernel 6.19 yet. I'm planning to install the openSUSE Tumbleweed with kernel 6.19.

Thanks,
Shawn.

Re: [BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery

[Ilpo Järvinen]

[-- Attachment #1: Type: text/plain, Size: 4629 bytes --]

On Thu, 12 Mar 2026, Shawn Jin wrote:

> After enabling more debugging messages for PCI subsystem, it shows the 
> pref bridge window was shrunken by 1 at [  796.604883]. 
> 
> Again, this is the kernel 6.8. I'll test 6.19 later using openSUSE 
> Tumbleweed. This time the realloc was on.

Hi,

First of all, many of the changes related to this additional resource 
distribution do not really make sense to me.

There's one fix that tries to rectify to most gross wrongness in the 
adjustment itself because it caused a regression (not yet applied):

https://lore.kernel.org/linux-pci/20260219153951.68869-1-ilpo.jarvinen@linux.intel.com/

It probably doesn't address the bug you see here though it may hide it.

> [  796.604869] pcieport 0000:96:01.0: distributing available resources
> [  796.604872] pci 0000:98:00.0: bridge window [mem 0x00100000-0x001fffff] extended by 0x0000000000100000
> [  796.604876] pci 0000:99:00.0: bridge window [??? 0x00000000 flags 0x0] extended by 0x0000000000055555
> [  796.604880] pci 0000:99:01.0: bridge window [mem 0x00100000-0x001fffff] shrunken by 0x00000000000aaaac
> [  796.604883] pci 0000:99:01.0: bridge window [mem 0x800000000-0xfffffffff 64bit pref] shrunken by 0x0000000000000001
> [  796.604886] pci 0000:99:02.0: bridge window [??? 0x00000000 flags 0x0] extended by 0x0000000000055555

Note how it's not just -1 that seems wrong, the other numbers too seem 
non-round (to 1M) as well.

There seems to be a number of problems with this entire algorithm.

I think the -1 comes from remove_dev_resource() that is called for a 
resource which does have !res->flags (+ res->start=res->end=0 -> 
resource_size(res) == 1).

So could you please try the patch below.

Not that the fix makes everything okay IMO but perhaps it addresses this 
particular case. It's not even clear to me how some of the cases with not 
valid bridge windows should be dealt with while distributing the 
additional memory between the bridges.


The 55555 cases probably come from align == 0 which makes 
ALIGN_DOWN_IF_NONZERO() to produce non-aligning sizes. The fix patch below 
might address it.

I don't (yet) understand how that aaaac came to be, I suppose it somehow 
relates to this doing something unexpected so if you want to check that 
out, it would be helpful:

		align = pci_resource_alignment(bridge, res);
                if (!resource_assigned(res) && align)
                        available[i].start = min(ALIGN(available[i].start, align),
                                                 available[i].end + 1);

Another option is remove_dev_resource() doing something that leads to it.

It might be worth adding debug traps for available[i] getting non-1M 
aligned but it's not very clear to me how and where.


From: =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= <ilpo.jarvinen@linux.intel.com>
Date: Thu, 12 Mar 2026 14:59:43 +0200
Subject: [PATCH 1/1] PCI: Skip not valid bridge windows while distributing
 resources
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

pci_bus_distribute_available_resources() distributes available
resources between downstream bridges, however, it does not take
into account cases where a bridge window is not valid.

Skip touching bridge windows that are not valid.

Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
---
 drivers/pci/setup-bus.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 61f769aaa2f6..4fcb5e0c44b4 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1868,6 +1868,9 @@ static void remove_dev_resource(struct resource *avail, struct pci_dev *dev,
 {
 	resource_size_t size, align, tmp;
 
+	if (!res->flags)
+		return;
+
 	size = resource_size(res);
 	if (!size)
 		return;
@@ -1922,6 +1925,8 @@ static void pci_bus_distribute_available_resources(struct pci_bus *bus,
 
 		available[i] = available_in[i];
 
+		if (!res->flags)
+			continue;
 		/*
 		 * The alignment of this bridge is yet to be considered,
 		 * hence it must be done now before extending its bridge
@@ -1993,6 +1998,8 @@ static void pci_bus_distribute_available_resources(struct pci_bus *bus,
 		for (i = 0; i < PCI_P2P_BRIDGE_RESOURCE_NUM; i++) {
 			res = pci_resource_n(dev, PCI_BRIDGE_RESOURCES + i);
 
+			if (!res->flags)
+				continue;
 			/*
 			 * Make sure the split resource space is properly
 			 * aligned for bridge windows (align it down to

base-commit: 6de23f81a5e08be8fbf5e8d7e9febc72a5b5f27f
-- 
2.39.5

Re: [BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery

[Shawn Jin]

Hi Ilpo,

Thanks for looking into the issue. Just a quick question. So the patch you linked and the patch you attached is for the latest kernel, not 6.8. Right? I'll try to install the latest kernel first and rerun my test and keep you updated.

Thanks,
Shawn.

________________________________________
From: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Sent: Thursday, March 12, 2026 6:24 AM
To: Shawn Jin <shawn.jin@asteralabs.com>
Cc: Bjorn Helgaas <helgaas@kernel.org>; Bjorn Helgaas <bhelgaas@google.com>; linux-kernel@vger.kernel.org <linux-kernel@vger.kernel.org>; linux-pci@vger.kernel.org <linux-pci@vger.kernel.org>
Subject: Re: [BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery

[EXTERNAL EMAIL]

Hi,
First of all, many of the changes related to this additional resource
distribution do not really make sense to me.

There's one fix that tries to rectify to most gross wrongness in the
adjustment itself because it caused a regression (not yet applied):

https://lore.kernel.org/linux-pci/20260219153951.68869-1-ilpo.jarvinen@linux.intel.com/

It probably doesn't address the bug you see here though it may hide it.

> [  796.604869] pcieport 0000:96:01.0: distributing available resources
> [  796.604872] pci 0000:98:00.0: bridge window [mem 0x00100000-0x001fffff] extended by 0x0000000000100000
> [  796.604876] pci 0000:99:00.0: bridge window [??? 0x00000000 flags 0x0] extended by 0x0000000000055555
> [  796.604880] pci 0000:99:01.0: bridge window [mem 0x00100000-0x001fffff] shrunken by 0x00000000000aaaac
> [  796.604883] pci 0000:99:01.0: bridge window [mem 0x800000000-0xfffffffff 64bit pref] shrunken by 0x0000000000000001
> [  796.604886] pci 0000:99:02.0: bridge window [??? 0x00000000 flags 0x0] extended by 0x0000000000055555

Note how it's not just -1 that seems wrong, the other numbers too seem
non-round (to 1M) as well.

There seems to be a number of problems with this entire algorithm.
I think the -1 comes from remove_dev_resource() that is called for a
resource which does have !res->flags (+ res->start=res->end=0 ->
resource_size(res) == 1).

So could you please try the patch below.

Not that the fix makes everything okay IMO but perhaps it addresses this
particular case. It's not even clear to me how some of the cases with not
valid bridge windows should be dealt with while distributing the
additional memory between the bridges.

The 55555 cases probably come from align == 0 which makes
ALIGN_DOWN_IF_NONZERO() to produce non-aligning sizes. The fix patch below
might address it.

I don't (yet) understand how that aaaac came to be, I suppose it somehow
relates to this doing something unexpected so if you want to check that
out, it would be helpful:
                align = pci_resource_alignment(bridge, res);
                if (!resource_assigned(res) && align)
                        available[i].start = min(ALIGN(available[i].start, align),
                                                 available[i].end + 1);

Another option is remove_dev_resource() doing something that leads to it.

It might be worth adding debug traps for available[i] getting non-1M
aligned but it's not very clear to me how and where.

From: =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= <ilpo.jarvinen@linux.intel.com>
Date: Thu, 12 Mar 2026 14:59:43 +0200
Subject: [PATCH 1/1] PCI: Skip not valid bridge windows while distributing
 resources
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

pci_bus_distribute_available_resources() distributes available
resources between downstream bridges, however, it does not take
into account cases where a bridge window is not valid.

Skip touching bridge windows that are not valid.

Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
---
 drivers/pci/setup-bus.c | 7 +++++++
 1 file changed, 7 insertions(+)
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 61f769aaa2f6..4fcb5e0c44b4 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1868,6 +1868,9 @@ static void remove_dev_resource(struct resource *avail, struct pci_dev *dev,
 {
        resource_size_t size, align, tmp;

+       if (!res->flags)
+               return;
+
        size = resource_size(res);
        if (!size)
                return;
@@ -1922,6 +1925,8 @@ static void pci_bus_distribute_available_resources(struct pci_bus *bus,
                available[i] = available_in[i];
+               if (!res->flags)
+                       continue;
                /*
                 * The alignment of this bridge is yet to be considered,
                 * hence it must be done now before extending its bridge
@@ -1993,6 +1998,8 @@ static void pci_bus_distribute_available_resources(struct pci_bus *bus,
                for (i = 0; i < PCI_P2P_BRIDGE_RESOURCE_NUM; i++) {
                        res = pci_resource_n(dev, PCI_BRIDGE_RESOURCES + i);

+                       if (!res->flags)
+                               continue;
                        /*
                         * Make sure the split resource space is properly
                         * aligned for bridge windows (align it down to
base-commit: 6de23f81a5e08be8fbf5e8d7e9febc72a5b5f27f
--
2.39.5

Re: [BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery

[Bjorn Helgaas]

On Thu, Mar 12, 2026 at 12:02:14AM +0000, Shawn Jin wrote:
> On Wed, Mar 11, 2026 at 10:00:39PM +0000, Shawn Jin wrote:
> > I'm reporting a potential critical bug in the Linux kernel's PCIe
> > resource allocation code that creates invalid bridge window limit
> > addresses during hotplug re-enumeration after Secondary Bus Reset
> > (SBR) recovery.
> 
> Thanks for the report and the repro and debugging information!
> 
> > ## AFFECTED KERNEL VERSIONS
> > - Confirmed: 5.15.0, 6.8.0 (Ubuntu 6.8.0-88-generic, 6.8.0-90-generic)
> > - Likely affected: All recent kernels including 6.19
> 
> Do you know of any kernels that are *not* affected?  If you do, we
> could bisect.
> 
> <SJ> I haven't tried the latest kernel 6.19 yet. I'm planning to
> install the openSUSE Tumbleweed with kernel 6.19.

I didn't notice at first, but AFAICS the most recent kernel you've
actually tested is v6.8, and "6.19 likely affected" is just
speculation.

There have been a lot of resource assignment changes since then, so
I doubt it's worth testing/debugging these old kernels until we
actually know the problem happens on v6.19 or v7.0-rc1.

IMO we should start by debugging the current kernel, fix the issue
there, then decide about backporting.

Bjorn

Re: [BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery

[Shawn Jin]

Hi Bjorn,

I agree we should work on the latest kernel instead. I'll try the latest kernel and report if similar issue exists.

Thanks,
Shawn.

________________________________________
From: Bjorn Helgaas <helgaas@kernel.org>
Sent: Thursday, March 12, 2026 10:34 AM
To: Shawn Jin <shawn.jin@asteralabs.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>; linux-kernel@vger.kernel.org <linux-kernel@vger.kernel.org>; linux-pci@vger.kernel.org <linux-pci@vger.kernel.org>; Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Subject: Re: [BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery
 
[EXTERNAL EMAIL]

On Thu, Mar 12, 2026 at 12:02:14AM +0000, Shawn Jin wrote:
> On Wed, Mar 11, 2026 at 10:00:39PM +0000, Shawn Jin wrote:
> > I'm reporting a potential critical bug in the Linux kernel's PCIe
> > resource allocation code that creates invalid bridge window limit
> > addresses during hotplug re-enumeration after Secondary Bus Reset
> > (SBR) recovery.
>
> Thanks for the report and the repro and debugging information!
>
> > ## AFFECTED KERNEL VERSIONS
> > - Confirmed: 5.15.0, 6.8.0 (Ubuntu 6.8.0-88-generic, 6.8.0-90-generic)
> > - Likely affected: All recent kernels including 6.19
>
> Do you know of any kernels that are *not* affected?  If you do, we
> could bisect.
>
> <SJ> I haven't tried the latest kernel 6.19 yet. I'm planning to
> install the openSUSE Tumbleweed with kernel 6.19.

I didn't notice at first, but AFAICS the most recent kernel you've
actually tested is v6.8, and "6.19 likely affected" is just
speculation.

There have been a lot of resource assignment changes since then, so
I doubt it's worth testing/debugging these old kernels until we
actually know the problem happens on v6.19 or v7.0-rc1.

IMO we should start by debugging the current kernel, fix the issue
there, then decide about backporting.

Bjorn

Re: [BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery

[Ilpo Järvinen]

[-- Attachment #1: Type: text/plain, Size: 5725 bytes --]

On Thu, 12 Mar 2026, Shawn Jin wrote:

> Hi Ilpo,
> 
> Thanks for looking into the issue. Just a quick question. So the patch 
> you linked and the patch you attached is for the latest kernel, not 6.8. 
> Right? I'll try to install the latest kernel first and rerun my test and 
> keep you updated.

Yes, they are for the latest trees but I can look into backporting them if 
needed.

--
 i.

> ________________________________________
> From: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
> Sent: Thursday, March 12, 2026 6:24 AM
> To: Shawn Jin <shawn.jin@asteralabs.com>
> Cc: Bjorn Helgaas <helgaas@kernel.org>; Bjorn Helgaas <bhelgaas@google.com>; linux-kernel@vger.kernel.org <linux-kernel@vger.kernel.org>; linux-pci@vger.kernel.org <linux-pci@vger.kernel.org>
> Subject: Re: [BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery
> 
> [EXTERNAL EMAIL]
> 
> Hi,
> First of all, many of the changes related to this additional resource
> distribution do not really make sense to me.
> 
> There's one fix that tries to rectify to most gross wrongness in the
> adjustment itself because it caused a regression (not yet applied):
> 
> https://lore.kernel.org/linux-pci/20260219153951.68869-1-ilpo.jarvinen@linux.intel.com/
> 
> It probably doesn't address the bug you see here though it may hide it.
> 
> > [  796.604869] pcieport 0000:96:01.0: distributing available resources
> > [  796.604872] pci 0000:98:00.0: bridge window [mem 0x00100000-0x001fffff] extended by 0x0000000000100000
> > [  796.604876] pci 0000:99:00.0: bridge window [??? 0x00000000 flags 0x0] extended by 0x0000000000055555
> > [  796.604880] pci 0000:99:01.0: bridge window [mem 0x00100000-0x001fffff] shrunken by 0x00000000000aaaac
> > [  796.604883] pci 0000:99:01.0: bridge window [mem 0x800000000-0xfffffffff 64bit pref] shrunken by 0x0000000000000001
> > [  796.604886] pci 0000:99:02.0: bridge window [??? 0x00000000 flags 0x0] extended by 0x0000000000055555
> 
> Note how it's not just -1 that seems wrong, the other numbers too seem
> non-round (to 1M) as well.
> 
> There seems to be a number of problems with this entire algorithm.
> I think the -1 comes from remove_dev_resource() that is called for a
> resource which does have !res->flags (+ res->start=res->end=0 ->
> resource_size(res) == 1).
> 
> So could you please try the patch below.
> 
> Not that the fix makes everything okay IMO but perhaps it addresses this
> particular case. It's not even clear to me how some of the cases with not
> valid bridge windows should be dealt with while distributing the
> additional memory between the bridges.
> 
> The 55555 cases probably come from align == 0 which makes
> ALIGN_DOWN_IF_NONZERO() to produce non-aligning sizes. The fix patch below
> might address it.
> 
> I don't (yet) understand how that aaaac came to be, I suppose it somehow
> relates to this doing something unexpected so if you want to check that
> out, it would be helpful:
>                 align = pci_resource_alignment(bridge, res);
>                 if (!resource_assigned(res) && align)
>                         available[i].start = min(ALIGN(available[i].start, align),
>                                                  available[i].end + 1);
> 
> Another option is remove_dev_resource() doing something that leads to it.
> 
> It might be worth adding debug traps for available[i] getting non-1M
> aligned but it's not very clear to me how and where.
> 
> From: =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= <ilpo.jarvinen@linux.intel.com>
> Date: Thu, 12 Mar 2026 14:59:43 +0200
> Subject: [PATCH 1/1] PCI: Skip not valid bridge windows while distributing
>  resources
> MIME-Version: 1.0
> Content-Type: text/plain; charset=UTF-8
> Content-Transfer-Encoding: 8bit
> 
> pci_bus_distribute_available_resources() distributes available
> resources between downstream bridges, however, it does not take
> into account cases where a bridge window is not valid.
> 
> Skip touching bridge windows that are not valid.
> 
> Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
> ---
>  drivers/pci/setup-bus.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
> index 61f769aaa2f6..4fcb5e0c44b4 100644
> --- a/drivers/pci/setup-bus.c
> +++ b/drivers/pci/setup-bus.c
> @@ -1868,6 +1868,9 @@ static void remove_dev_resource(struct resource *avail, struct pci_dev *dev,
>  {
>         resource_size_t size, align, tmp;
> 
> +       if (!res->flags)
> +               return;
> +
>         size = resource_size(res);
>         if (!size)
>                 return;
> @@ -1922,6 +1925,8 @@ static void pci_bus_distribute_available_resources(struct pci_bus *bus,
>                 available[i] = available_in[i];
> +               if (!res->flags)
> +                       continue;
>                 /*
>                  * The alignment of this bridge is yet to be considered,
>                  * hence it must be done now before extending its bridge
> @@ -1993,6 +1998,8 @@ static void pci_bus_distribute_available_resources(struct pci_bus *bus,
>                 for (i = 0; i < PCI_P2P_BRIDGE_RESOURCE_NUM; i++) {
>                         res = pci_resource_n(dev, PCI_BRIDGE_RESOURCES + i);
> 
> +                       if (!res->flags)
> +                               continue;
>                         /*
>                          * Make sure the split resource space is properly
>                          * aligned for bridge windows (align it down to
> base-commit: 6de23f81a5e08be8fbf5e8d7e9febc72a5b5f27f
> --
> 2.39.5
> 

Re: [BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery

[Shawn Jin]

Hi Ilpo and Bjorn,

A quick update on the test result with the kernel 6.19.6-1-default in openSUSE Tumbleweed.

Summary:
1. With PCI realloc off, the bridge windows were reassigned as before, the ending addresses look normal, all ending with 0xffff.
2. With PCI realloc on, the non-pref window ending address is still incorrect: 0xe1a55554. But the pref window looks right.
[  482.943430] [   T1435] pci 0000:99:01.0: bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]: assigned
[  482.943431] [   T1435] pci 0000:99:01.0: bridge window [mem 0xe1a00000-0xe1a55554]: assigned

Details:
Focused topology:
-[0000:94]---01.0-[95-9e]----00.0-[96-9e]----01.0-[98-9c]----00.0-[99-9c]--+-00.0-[9a]--
                                                                           +-01.0-[9b]----00.0  Device 1e3e:0002
                                                                           \-02.0-[9c]--
Test 1: realloc OFF
// Cold boot
[    2.132565] [      T1] pci 0000:99:00.0: PCI bridge to [bus 9a]
[    2.133202] [      T1] pci 0000:99:01.0: PCI bridge to [bus 9b]
[    2.133413] [      T1] pci 0000:99:01.0:   bridge window [mem 0xe9600000-0xe96fffff]
[    2.133555] [      T1] pci 0000:99:01.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
[    2.133827] [      T1] pci 0000:99:02.0: PCI bridge to [bus 9c]
[    2.134453] [      T1] pci 0000:98:00.0: PCI bridge to [bus 99-9c]
[    2.134565] [      T1] pci 0000:98:00.0:   bridge window [mem 0xe9600000-0xe96fffff]
[    2.134647] [      T1] pci 0000:98:00.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
[    2.134809] [      T1] pci 0000:96:01.0: PCI bridge to [bus 98-9c]
[    2.134830] [      T1] pci 0000:96:01.0:   bridge window [mem 0xe9600000-0xe96fffff]
[    2.134845] [      T1] pci 0000:96:01.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
// after SBR
[  202.341320] [   T1430] pci 0000:99:01.0: PCI bridge to [bus 9b]
[  202.341539] [   T1430] pci 0000:99:01.0:   bridge window [mem 0xe9600000-0xe96fffff]
[  202.341680] [   T1430] pci 0000:99:01.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
[  202.341950] [   T1430] pci 0000:99:02.0: PCI bridge to [bus 9c]
[  202.342573] [   T1430] pci 0000:98:00.0: PCI bridge to [bus 99-9c]
[  202.342685] [   T1430] pci 0000:98:00.0:   bridge window [mem 0xe9600000-0xe96fffff]
[  202.342766] [   T1430] pci 0000:98:00.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
[  202.342929] [   T1430] pcieport 0000:96:01.0: PCI bridge to [bus 98-9c]
[  202.342951] [   T1430] pcieport 0000:96:01.0:   bridge window [mem 0xe9600000-0xe96fffff]
[  202.342964] [   T1430] pcieport 0000:96:01.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]

// although the bridges don't have IO windows, the dbg message showing the window extension may be a bit worrisome.
[  202.340536] [   T1430] pci 0000:99:00.0: bridge window [io  size 0x0000 disabled] extended by 0x0000000000000555
[  202.340538] [   T1430] pci 0000:99:01.0: bridge window [io  size 0x0000 disabled] extended by 0x0000000000000555
[  202.340538] [   T1430] pci 0000:99:02.0: bridge window [io  size 0x0000 disabled] extended by 0x0000000000000555

Test 2: realloc ON
// code boot
[    2.146003] [      T1] pci_bus 0000:96: resource 1 [mem 0xe1800000-0xe1dfffff]
[    2.146004] [      T1] pci_bus 0000:96: resource 2 [mem 0x134000000000-0x1348004fffff 64bit pref]
[    2.146007] [      T1] pci_bus 0000:98: resource 0 [io  size 0x1000]
[    2.146008] [      T1] pci_bus 0000:98: resource 1 [mem 0xe1a00000-0xe1bfffff]
[    2.146009] [      T1] pci_bus 0000:98: resource 2 [mem 0x134000000000-0x1347ffffffff 64bit pref]
[    2.146010] [      T1] pci_bus 0000:99: resource 1 [mem 0xe1a00000-0xe1afffff]
[    2.146011] [      T1] pci_bus 0000:99: resource 2 [mem 0x134000000000-0x1347ffffffff 64bit pref]
[    2.146012] [      T1] pci_bus 0000:9b: resource 1 [mem 0xe1a00000-0xe1afffff]
[    2.146013] [      T1] pci_bus 0000:9b: resource 2 [mem 0x134000000000-0x1347ffffffff 64bit pref]

// after SBR
// USP 98:00.0's windows still look correct
[  482.943427] [   T1435] pci 0000:98:00.0: bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]: assigned
[  482.943429] [   T1435] pci 0000:98:00.0: bridge window [mem 0xe1a00000-0xe1bfffff]: assigned
// DSP 99:01.0 pref window is good
[  482.943430] [   T1435] pci 0000:99:01.0: bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]: assigned
// DSP 99:01.0 non-pref window ending addr is wrong: 0xe1a55554
[  482.943431] [   T1435] pci 0000:99:01.0: bridge window [mem 0xe1a00000-0xe1a55554]: assigned
[  482.943432] [   T1435] pci 0000:99:00.0: PCI bridge to [bus 9a]
[  482.944063] [   T1435] pci 0000:9b:00.0: BAR 0 [mem 0x134000000000-0x1347ffffffff 64bit pref]: assigned
[  482.944164] [   T1435] pci 0000:9b:00.0: BAR 2 [mem 0xe1a00000-0xe1a3ffff]: assigned
[  482.944202] [   T1435] pci 0000:99:01.0: PCI bridge to [bus 9b]
[  482.944424] [   T1435] pci 0000:99:01.0:   bridge window [mem 0xe1a00000-0xe1a55554]
[  482.944565] [   T1435] pci 0000:99:01.0:   bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]
[  482.944837] [   T1435] pci 0000:99:02.0: PCI bridge to [bus 9c]
[  482.945463] [   T1435] pci 0000:98:00.0: PCI bridge to [bus 99-9c]
[  482.945575] [   T1435] pci 0000:98:00.0:   bridge window [mem 0xe1a00000-0xe1bfffff]
[  482.945657] [   T1435] pci 0000:98:00.0:   bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]
[  482.945820] [   T1435] pcieport 0000:96:01.0: PCI bridge to [bus 98-9c]
[  482.945842] [   T1435] pcieport 0000:96:01.0:   bridge window [mem 0xe1a00000-0xe1bfffff]
[  482.945856] [   T1435] pcieport 0000:96:01.0:   bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]
[  482.945883] [   T1435] PCI: No. 2 try to assign unassigned res
[  482.945885] [   T1435] pci 0000:99:00.0: disabling bridge window [mem size 0x00000000 64bit pref disabled] to [bus 9a] (unused)
// DSP 99:00.0 doesn't have any EP attached. But the mem size 0x55555 looks very suspicious.
[  482.945886] [   T1435] pci 0000:99:00.0: disabling bridge window [mem size 0x00055555 disabled] to [bus 9a] (unused)
[  482.945887] [   T1435] pci 0000:99:02.0: disabling bridge window [mem size 0x00000000 64bit pref disabled] to [bus 9c] (unused)
// Same for DSP 99:02.0
[  482.945888] [   T1435] pci 0000:99:02.0: disabling bridge window [mem size 0x00055555 disabled] to [bus 9c] (unused)

Hope the above info gives you some hints.

Thanks,
Shawn.
________________________________________
From: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Sent: Thursday, March 12, 2026 10:48 AM
To: Shawn Jin <shawn.jin@asteralabs.com>
Cc: Bjorn Helgaas <helgaas@kernel.org>; Bjorn Helgaas <bhelgaas@google.com>; linux-kernel@vger.kernel.org <linux-kernel@vger.kernel.org>; linux-pci@vger.kernel.org <linux-pci@vger.kernel.org>
Subject: Re: [BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery

[EXTERNAL EMAIL]

On Thu, 12 Mar 2026, Shawn Jin wrote:

> Hi Ilpo,
>
> Thanks for looking into the issue. Just a quick question. So the patch
> you linked and the patch you attached is for the latest kernel, not 6.8.
> Right? I'll try to install the latest kernel first and rerun my test and
> keep you updated.

Yes, they are for the latest trees but I can look into backporting them if
needed.

--
 i.

> ________________________________________
> From: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
> Sent: Thursday, March 12, 2026 6:24 AM
> To: Shawn Jin <shawn.jin@asteralabs.com>
> Cc: Bjorn Helgaas <helgaas@kernel.org>; Bjorn Helgaas <bhelgaas@google.com>; linux-kernel@vger.kernel.org <linux-kernel@vger.kernel.org>; linux-pci@vger.kernel.org <linux-pci@vger.kernel.org>
> Subject: Re: [BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery
>
> [EXTERNAL EMAIL]
>
> Hi,
> First of all, many of the changes related to this additional resource
> distribution do not really make sense to me.
>
> There's one fix that tries to rectify to most gross wrongness in the
> adjustment itself because it caused a regression (not yet applied):
>
> https://lore.kernel.org/linux-pci/20260219153951.68869-1-ilpo.jarvinen@linux.intel.com/
>
> It probably doesn't address the bug you see here though it may hide it.
>
> > [  796.604869] pcieport 0000:96:01.0: distributing available resources
> > [  796.604872] pci 0000:98:00.0: bridge window [mem 0x00100000-0x001fffff] extended by 0x0000000000100000
> > [  796.604876] pci 0000:99:00.0: bridge window [??? 0x00000000 flags 0x0] extended by 0x0000000000055555
> > [  796.604880] pci 0000:99:01.0: bridge window [mem 0x00100000-0x001fffff] shrunken by 0x00000000000aaaac
> > [  796.604883] pci 0000:99:01.0: bridge window [mem 0x800000000-0xfffffffff 64bit pref] shrunken by 0x0000000000000001
> > [  796.604886] pci 0000:99:02.0: bridge window [??? 0x00000000 flags 0x0] extended by 0x0000000000055555
>
> Note how it's not just -1 that seems wrong, the other numbers too seem
> non-round (to 1M) as well.
>
> There seems to be a number of problems with this entire algorithm.
> I think the -1 comes from remove_dev_resource() that is called for a
> resource which does have !res->flags (+ res->start=res->end=0 ->
> resource_size(res) == 1).
>
> So could you please try the patch below.
>
> Not that the fix makes everything okay IMO but perhaps it addresses this
> particular case. It's not even clear to me how some of the cases with not
> valid bridge windows should be dealt with while distributing the
> additional memory between the bridges.
>
> The 55555 cases probably come from align == 0 which makes
> ALIGN_DOWN_IF_NONZERO() to produce non-aligning sizes. The fix patch below
> might address it.
>
> I don't (yet) understand how that aaaac came to be, I suppose it somehow
> relates to this doing something unexpected so if you want to check that
> out, it would be helpful:
>                 align = pci_resource_alignment(bridge, res);
>                 if (!resource_assigned(res) && align)
>                         available[i].start = min(ALIGN(available[i].start, align),
>                                                  available[i].end + 1);
>
> Another option is remove_dev_resource() doing something that leads to it.
>
> It might be worth adding debug traps for available[i] getting non-1M
> aligned but it's not very clear to me how and where.
>
> From: =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= <ilpo.jarvinen@linux.intel.com>
> Date: Thu, 12 Mar 2026 14:59:43 +0200
> Subject: [PATCH 1/1] PCI: Skip not valid bridge windows while distributing
>  resources
> MIME-Version: 1.0
> Content-Type: text/plain; charset=UTF-8
> Content-Transfer-Encoding: 8bit
>
> pci_bus_distribute_available_resources() distributes available
> resources between downstream bridges, however, it does not take
> into account cases where a bridge window is not valid.
>
> Skip touching bridge windows that are not valid.
>
> Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
> ---
>  drivers/pci/setup-bus.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
> index 61f769aaa2f6..4fcb5e0c44b4 100644
> --- a/drivers/pci/setup-bus.c
> +++ b/drivers/pci/setup-bus.c
> @@ -1868,6 +1868,9 @@ static void remove_dev_resource(struct resource *avail, struct pci_dev *dev,
>  {
>         resource_size_t size, align, tmp;
>
> +       if (!res->flags)
> +               return;
> +
>         size = resource_size(res);
>         if (!size)
>                 return;
> @@ -1922,6 +1925,8 @@ static void pci_bus_distribute_available_resources(struct pci_bus *bus,
>                 available[i] = available_in[i];
> +               if (!res->flags)
> +                       continue;
>                 /*
>                  * The alignment of this bridge is yet to be considered,
>                  * hence it must be done now before extending its bridge
> @@ -1993,6 +1998,8 @@ static void pci_bus_distribute_available_resources(struct pci_bus *bus,
>                 for (i = 0; i < PCI_P2P_BRIDGE_RESOURCE_NUM; i++) {
>                         res = pci_resource_n(dev, PCI_BRIDGE_RESOURCES + i);
>
> +                       if (!res->flags)
> +                               continue;
>                         /*
>                          * Make sure the split resource space is properly
>                          * aligned for bridge windows (align it down to
> base-commit: 6de23f81a5e08be8fbf5e8d7e9febc72a5b5f27f
> --
> 2.39.5
>

Re: [BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery

[Ilpo Järvinen]

[-- Attachment #1: Type: text/plain, Size: 13843 bytes --]

On Fri, 13 Mar 2026, Shawn Jin wrote:

> Hi Ilpo and Bjorn,
> 
> A quick update on the test result with the kernel 6.19.6-1-default in openSUSE Tumbleweed.
> 
> Summary:
> 1. With PCI realloc off, the bridge windows were reassigned as before, the ending addresses look normal, all ending with 0xffff.
> 2. With PCI realloc on, the non-pref window ending address is still incorrect: 0xe1a55554. But the pref window looks right.
> [  482.943430] [   T1435] pci 0000:99:01.0: bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]: assigned
> [  482.943431] [   T1435] pci 0000:99:01.0: bridge window [mem 0xe1a00000-0xe1a55554]: assigned

Was this window initially zero sized? That would explain why align becomes 
zero.

> Details:
> Focused topology:
> -[0000:94]---01.0-[95-9e]----00.0-[96-9e]----01.0-[98-9c]----00.0-[99-9c]--+-00.0-[9a]--
>                                                                            +-01.0-[9b]----00.0  Device 1e3e:0002
>                                                                            \-02.0-[9c]--
> Test 1: realloc OFF
> // Cold boot
> [    2.132565] [      T1] pci 0000:99:00.0: PCI bridge to [bus 9a]
> [    2.133202] [      T1] pci 0000:99:01.0: PCI bridge to [bus 9b]
> [    2.133413] [      T1] pci 0000:99:01.0:   bridge window [mem 0xe9600000-0xe96fffff]
> [    2.133555] [      T1] pci 0000:99:01.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
> [    2.133827] [      T1] pci 0000:99:02.0: PCI bridge to [bus 9c]
> [    2.134453] [      T1] pci 0000:98:00.0: PCI bridge to [bus 99-9c]
> [    2.134565] [      T1] pci 0000:98:00.0:   bridge window [mem 0xe9600000-0xe96fffff]
> [    2.134647] [      T1] pci 0000:98:00.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
> [    2.134809] [      T1] pci 0000:96:01.0: PCI bridge to [bus 98-9c]
> [    2.134830] [      T1] pci 0000:96:01.0:   bridge window [mem 0xe9600000-0xe96fffff]
> [    2.134845] [      T1] pci 0000:96:01.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
> // after SBR
> [  202.341320] [   T1430] pci 0000:99:01.0: PCI bridge to [bus 9b]
> [  202.341539] [   T1430] pci 0000:99:01.0:   bridge window [mem 0xe9600000-0xe96fffff]
> [  202.341680] [   T1430] pci 0000:99:01.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
> [  202.341950] [   T1430] pci 0000:99:02.0: PCI bridge to [bus 9c]
> [  202.342573] [   T1430] pci 0000:98:00.0: PCI bridge to [bus 99-9c]
> [  202.342685] [   T1430] pci 0000:98:00.0:   bridge window [mem 0xe9600000-0xe96fffff]
> [  202.342766] [   T1430] pci 0000:98:00.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
> [  202.342929] [   T1430] pcieport 0000:96:01.0: PCI bridge to [bus 98-9c]
> [  202.342951] [   T1430] pcieport 0000:96:01.0:   bridge window [mem 0xe9600000-0xe96fffff]
> [  202.342964] [   T1430] pcieport 0000:96:01.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
> 
> // although the bridges don't have IO windows, the dbg message showing the window extension may be a bit worrisome.
> [  202.340536] [   T1430] pci 0000:99:00.0: bridge window [io  size 0x0000 disabled] extended by 0x0000000000000555
> [  202.340538] [   T1430] pci 0000:99:01.0: bridge window [io  size 0x0000 disabled] extended by 0x0000000000000555
> [  202.340538] [   T1430] pci 0000:99:02.0: bridge window [io  size 0x0000 disabled] extended by 0x0000000000000555

This again is thanks to align being 0 because resource_size() == 0 so 
ALIGN_DOWN_IF_NONZERO() doesn't do anything useful.

I wonder if it would make sense to handle the minimum alignment of bridge 
windows properly in pci_resource_alignment(). It currently can return 0 
for bridge windows but we've well-defined alignment lower bounds for those 
which I think could be applied right there at source instead of having 
the callers have to worry about getting the alignment lower bounding 
right.

-- 
 i.

> Test 2: realloc ON
> // code boot
> [    2.146003] [      T1] pci_bus 0000:96: resource 1 [mem 0xe1800000-0xe1dfffff]
> [    2.146004] [      T1] pci_bus 0000:96: resource 2 [mem 0x134000000000-0x1348004fffff 64bit pref]
> [    2.146007] [      T1] pci_bus 0000:98: resource 0 [io  size 0x1000]
> [    2.146008] [      T1] pci_bus 0000:98: resource 1 [mem 0xe1a00000-0xe1bfffff]
> [    2.146009] [      T1] pci_bus 0000:98: resource 2 [mem 0x134000000000-0x1347ffffffff 64bit pref]
> [    2.146010] [      T1] pci_bus 0000:99: resource 1 [mem 0xe1a00000-0xe1afffff]
> [    2.146011] [      T1] pci_bus 0000:99: resource 2 [mem 0x134000000000-0x1347ffffffff 64bit pref]
> [    2.146012] [      T1] pci_bus 0000:9b: resource 1 [mem 0xe1a00000-0xe1afffff]
> [    2.146013] [      T1] pci_bus 0000:9b: resource 2 [mem 0x134000000000-0x1347ffffffff 64bit pref]
> 
> // after SBR
> // USP 98:00.0's windows still look correct
> [  482.943427] [   T1435] pci 0000:98:00.0: bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]: assigned
> [  482.943429] [   T1435] pci 0000:98:00.0: bridge window [mem 0xe1a00000-0xe1bfffff]: assigned
> // DSP 99:01.0 pref window is good
> [  482.943430] [   T1435] pci 0000:99:01.0: bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]: assigned
> // DSP 99:01.0 non-pref window ending addr is wrong: 0xe1a55554
> [  482.943431] [   T1435] pci 0000:99:01.0: bridge window [mem 0xe1a00000-0xe1a55554]: assigned
> [  482.943432] [   T1435] pci 0000:99:00.0: PCI bridge to [bus 9a]
> [  482.944063] [   T1435] pci 0000:9b:00.0: BAR 0 [mem 0x134000000000-0x1347ffffffff 64bit pref]: assigned
> [  482.944164] [   T1435] pci 0000:9b:00.0: BAR 2 [mem 0xe1a00000-0xe1a3ffff]: assigned
> [  482.944202] [   T1435] pci 0000:99:01.0: PCI bridge to [bus 9b]
> [  482.944424] [   T1435] pci 0000:99:01.0:   bridge window [mem 0xe1a00000-0xe1a55554]
> [  482.944565] [   T1435] pci 0000:99:01.0:   bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]
> [  482.944837] [   T1435] pci 0000:99:02.0: PCI bridge to [bus 9c]
> [  482.945463] [   T1435] pci 0000:98:00.0: PCI bridge to [bus 99-9c]
> [  482.945575] [   T1435] pci 0000:98:00.0:   bridge window [mem 0xe1a00000-0xe1bfffff]
> [  482.945657] [   T1435] pci 0000:98:00.0:   bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]
> [  482.945820] [   T1435] pcieport 0000:96:01.0: PCI bridge to [bus 98-9c]
> [  482.945842] [   T1435] pcieport 0000:96:01.0:   bridge window [mem 0xe1a00000-0xe1bfffff]
> [  482.945856] [   T1435] pcieport 0000:96:01.0:   bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]
> [  482.945883] [   T1435] PCI: No. 2 try to assign unassigned res
> [  482.945885] [   T1435] pci 0000:99:00.0: disabling bridge window [mem size 0x00000000 64bit pref disabled] to [bus 9a] (unused)
> // DSP 99:00.0 doesn't have any EP attached. But the mem size 0x55555 looks very suspicious.
> [  482.945886] [   T1435] pci 0000:99:00.0: disabling bridge window [mem size 0x00055555 disabled] to [bus 9a] (unused)
> [  482.945887] [   T1435] pci 0000:99:02.0: disabling bridge window [mem size 0x00000000 64bit pref disabled] to [bus 9c] (unused)
> // Same for DSP 99:02.0
> [  482.945888] [   T1435] pci 0000:99:02.0: disabling bridge window [mem size 0x00055555 disabled] to [bus 9c] (unused)
> 
> Hope the above info gives you some hints.
>
> From: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
> Sent: Thursday, March 12, 2026 10:48 AM
> To: Shawn Jin <shawn.jin@asteralabs.com>
> Cc: Bjorn Helgaas <helgaas@kernel.org>; Bjorn Helgaas <bhelgaas@google.com>; linux-kernel@vger.kernel.org <linux-kernel@vger.kernel.org>; linux-pci@vger.kernel.org <linux-pci@vger.kernel.org>
> Subject: Re: [BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery
> 
> [EXTERNAL EMAIL]
> 
> On Thu, 12 Mar 2026, Shawn Jin wrote:
> 
> > Hi Ilpo,
> >
> > Thanks for looking into the issue. Just a quick question. So the patch
> > you linked and the patch you attached is for the latest kernel, not 6.8.
> > Right? I'll try to install the latest kernel first and rerun my test and
> > keep you updated.
> 
> Yes, they are for the latest trees but I can look into backporting them if
> needed.
> 
> --
>  i.
> 
> > ________________________________________
> > From: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
> > Sent: Thursday, March 12, 2026 6:24 AM
> > To: Shawn Jin <shawn.jin@asteralabs.com>
> > Cc: Bjorn Helgaas <helgaas@kernel.org>; Bjorn Helgaas <bhelgaas@google.com>; linux-kernel@vger.kernel.org <linux-kernel@vger.kernel.org>; linux-pci@vger.kernel.org <linux-pci@vger.kernel.org>
> > Subject: Re: [BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery
> >
> > [EXTERNAL EMAIL]
> >
> > Hi,
> > First of all, many of the changes related to this additional resource
> > distribution do not really make sense to me.
> >
> > There's one fix that tries to rectify to most gross wrongness in the
> > adjustment itself because it caused a regression (not yet applied):
> >
> > https://lore.kernel.org/linux-pci/20260219153951.68869-1-ilpo.jarvinen@linux.intel.com/
> >
> > It probably doesn't address the bug you see here though it may hide it.
> >
> > > [  796.604869] pcieport 0000:96:01.0: distributing available resources
> > > [  796.604872] pci 0000:98:00.0: bridge window [mem 0x00100000-0x001fffff] extended by 0x0000000000100000
> > > [  796.604876] pci 0000:99:00.0: bridge window [??? 0x00000000 flags 0x0] extended by 0x0000000000055555
> > > [  796.604880] pci 0000:99:01.0: bridge window [mem 0x00100000-0x001fffff] shrunken by 0x00000000000aaaac
> > > [  796.604883] pci 0000:99:01.0: bridge window [mem 0x800000000-0xfffffffff 64bit pref] shrunken by 0x0000000000000001
> > > [  796.604886] pci 0000:99:02.0: bridge window [??? 0x00000000 flags 0x0] extended by 0x0000000000055555
> >
> > Note how it's not just -1 that seems wrong, the other numbers too seem
> > non-round (to 1M) as well.
> >
> > There seems to be a number of problems with this entire algorithm.
> > I think the -1 comes from remove_dev_resource() that is called for a
> > resource which does have !res->flags (+ res->start=res->end=0 ->
> > resource_size(res) == 1).
> >
> > So could you please try the patch below.
> >
> > Not that the fix makes everything okay IMO but perhaps it addresses this
> > particular case. It's not even clear to me how some of the cases with not
> > valid bridge windows should be dealt with while distributing the
> > additional memory between the bridges.
> >
> > The 55555 cases probably come from align == 0 which makes
> > ALIGN_DOWN_IF_NONZERO() to produce non-aligning sizes. The fix patch below
> > might address it.
> >
> > I don't (yet) understand how that aaaac came to be, I suppose it somehow
> > relates to this doing something unexpected so if you want to check that
> > out, it would be helpful:
> >                 align = pci_resource_alignment(bridge, res);
> >                 if (!resource_assigned(res) && align)
> >                         available[i].start = min(ALIGN(available[i].start, align),
> >                                                  available[i].end + 1);
> >
> > Another option is remove_dev_resource() doing something that leads to it.
> >
> > It might be worth adding debug traps for available[i] getting non-1M
> > aligned but it's not very clear to me how and where.
> >
> > From: =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= <ilpo.jarvinen@linux.intel.com>
> > Date: Thu, 12 Mar 2026 14:59:43 +0200
> > Subject: [PATCH 1/1] PCI: Skip not valid bridge windows while distributing
> >  resources
> > MIME-Version: 1.0
> > Content-Type: text/plain; charset=UTF-8
> > Content-Transfer-Encoding: 8bit
> >
> > pci_bus_distribute_available_resources() distributes available
> > resources between downstream bridges, however, it does not take
> > into account cases where a bridge window is not valid.
> >
> > Skip touching bridge windows that are not valid.
> >
> > Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
> > ---
> >  drivers/pci/setup-bus.c | 7 +++++++
> >  1 file changed, 7 insertions(+)
> > diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
> > index 61f769aaa2f6..4fcb5e0c44b4 100644
> > --- a/drivers/pci/setup-bus.c
> > +++ b/drivers/pci/setup-bus.c
> > @@ -1868,6 +1868,9 @@ static void remove_dev_resource(struct resource *avail, struct pci_dev *dev,
> >  {
> >         resource_size_t size, align, tmp;
> >
> > +       if (!res->flags)
> > +               return;
> > +
> >         size = resource_size(res);
> >         if (!size)
> >                 return;
> > @@ -1922,6 +1925,8 @@ static void pci_bus_distribute_available_resources(struct pci_bus *bus,
> >                 available[i] = available_in[i];
> > +               if (!res->flags)
> > +                       continue;
> >                 /*
> >                  * The alignment of this bridge is yet to be considered,
> >                  * hence it must be done now before extending its bridge
> > @@ -1993,6 +1998,8 @@ static void pci_bus_distribute_available_resources(struct pci_bus *bus,
> >                 for (i = 0; i < PCI_P2P_BRIDGE_RESOURCE_NUM; i++) {
> >                         res = pci_resource_n(dev, PCI_BRIDGE_RESOURCES + i);
> >
> > +                       if (!res->flags)
> > +                               continue;
> >                         /*
> >                          * Make sure the split resource space is properly
> >                          * aligned for bridge windows (align it down to
> > base-commit: 6de23f81a5e08be8fbf5e8d7e9febc72a5b5f27f
> > --
> > 2.39.5
> >
> 

Re: [BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery

[Shawn Jin]

________________________________________
From: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Sent: Monday, March 16, 2026 3:28 AM
To: Shawn Jin <shawn.jin@asteralabs.com>
Cc: Bjorn Helgaas <helgaas@kernel.org>; Bjorn Helgaas <bhelgaas@google.com>; linux-kernel@vger.kernel.org <linux-kernel@vger.kernel.org>; linux-pci@vger.kernel.org <linux-pci@vger.kernel.org>
Subject: Re: [BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery
 
[EXTERNAL EMAIL]

On Fri, 13 Mar 2026, Shawn Jin wrote:

> Hi Ilpo and Bjorn,
>
> A quick update on the test result with the kernel 6.19.6-1-default in openSUSE Tumbleweed.
>
> Summary:
> 1. With PCI realloc off, the bridge windows were reassigned as before, the ending addresses look normal, all ending with 0xffff.
> 2. With PCI realloc on, the non-pref window ending address is still incorrect: 0xe1a55554. But the pref window looks right.
> [  482.943430] [   T1435] pci 0000:99:01.0: bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]: assigned
> [  482.943431] [   T1435] pci 0000:99:01.0: bridge window [mem 0xe1a00000-0xe1a55554]: assigned

Was this window initially zero sized? That would explain why align becomes
zero.

<SJ> If "initially" means the windows assigned during the cold boot, then the answer is no. But the window resources were freed because of the SBR test it underwent.

The test script is as below.
# Remove the USP device
echo 1 | sudo tee /sys/bus/pci/devices/0000:${USP_BDF}/remove

# Trigger SBR via Bridge Control register
BRIDGE_CTL=$(sudo setpci -s ${ROOT_PORT_BDF} 0x3E.w)
BRIDGE_CTL_RESET=$(printf "0x%04x" $((0x$BRIDGE_CTL | 0x0040)))

echo "Asserting Secondary Bus Reset..."
sudo setpci -s ${ROOT_PORT_BDF} 0x3E.w=$BRIDGE_CTL_RESET
sleep 1

echo "De-asserting Secondary Bus Reset..."
sudo setpci -s ${ROOT_PORT_BDF} 0x3E.w=$BRIDGE_CTL
sleep 2

# Rescan PCI bus
echo 1 | sudo tee /sys/bus/pci/rescan

Additional dmesg showed that the window sizes were reset to 0.
[  482.935950] [   T1435] pci 0000:99:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[  482.936183] [   T1435] pci 0000:99:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[  482.936401] [   T1435] pci 0000:99:02.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[  482.938545] [   T1435] pci 0000:99:00.0: PCI bridge to [bus 9a-9c]
[  482.938611] [   T1435] pci_bus 0000:9a: busn_res: [bus 9a-9c] end is updated to 9a
[  482.939301] [   T1435] pci 0000:9b:00.0: [1e3e:0002] type 00 class 0x120000 PCIe Endpoint
[  482.939735] [   T1435] pci 0000:9b:00.0: BAR 0 [mem 0x00000000-0x7ffffffff 64bit pref]
[  482.939743] [   T1435] pci 0000:9b:00.0: BAR 2 [mem 0x00000000-0x0003ffff]
[  482.939843] [   T1435] pci 0000:9b:00.0: Max Payload Size set to 256 (was 128, max 256)
[  482.940817] [   T1435] pci 0000:9b:00.0: PME# supported from D0 D3hot
[  482.942468] [   T1435] pci 0000:99:01.0: PCI bridge to [bus 9b-9c]
[  482.942535] [   T1435] pci_bus 0000:9b: busn_res: [bus 9b-9c] end is updated to 9b
[  482.943101] [   T1435] pci 0000:99:02.0: PCI bridge to [bus 9c]
[  482.943169] [   T1435] pci_bus 0000:9c: busn_res: [bus 9c] end is updated to 9c
[  482.943295] [   T1435] pci_bus 0000:99: busn_res: [bus 99-9c] end is updated to 9c
[  482.943402] [   T1435] pci 0000:99:00.0: disabling bridge window [io  size 0x0000 disabled] to [bus 9a] (unused)
[  482.943405] [   T1435] pci 0000:99:00.0: disabling bridge window [mem size 0x00000000 64bit pref disabled] to [bus 9a] (unused)
[  482.943406] [   T1435] pci 0000:99:00.0: disabling bridge window [mem size 0x00000000 disabled] to [bus 9a] (unused)
[  482.943407] [   T1435] pci 0000:99:01.0: disabling bridge window [io  size 0x0000 disabled] to [bus 9b] (unused)
[  482.943409] [   T1435] pci 0000:99:02.0: disabling bridge window [io  size 0x0000 disabled] to [bus 9c] (unused)
[  482.943410] [   T1435] pci 0000:99:02.0: disabling bridge window [mem size 0x00000000 64bit pref disabled] to [bus 9c] (unused)
[  482.943411] [   T1435] pci 0000:99:02.0: disabling bridge window [mem size 0x00000000 disabled] to [bus 9c] (unused)
[  482.943412] [   T1435] pci 0000:98:00.0: disabling bridge window [io  size 0x0000 disabled] to [bus 99-9c] (unused)

Thanks,
Shawn

> Details:
> Focused topology:
> -[0000:94]---01.0-[95-9e]----00.0-[96-9e]----01.0-[98-9c]----00.0-[99-9c]--+-00.0-[9a]--
>                                                                            +-01.0-[9b]----00.0  Device 1e3e:0002
>                                                                            \-02.0-[9c]--
> Test 1: realloc OFF
> // Cold boot
> [    2.132565] [      T1] pci 0000:99:00.0: PCI bridge to [bus 9a]
> [    2.133202] [      T1] pci 0000:99:01.0: PCI bridge to [bus 9b]
> [    2.133413] [      T1] pci 0000:99:01.0:   bridge window [mem 0xe9600000-0xe96fffff]
> [    2.133555] [      T1] pci 0000:99:01.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
> [    2.133827] [      T1] pci 0000:99:02.0: PCI bridge to [bus 9c]
> [    2.134453] [      T1] pci 0000:98:00.0: PCI bridge to [bus 99-9c]
> [    2.134565] [      T1] pci 0000:98:00.0:   bridge window [mem 0xe9600000-0xe96fffff]
> [    2.134647] [      T1] pci 0000:98:00.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
> [    2.134809] [      T1] pci 0000:96:01.0: PCI bridge to [bus 98-9c]
> [    2.134830] [      T1] pci 0000:96:01.0:   bridge window [mem 0xe9600000-0xe96fffff]
> [    2.134845] [      T1] pci 0000:96:01.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
> // after SBR
> [  202.341320] [   T1430] pci 0000:99:01.0: PCI bridge to [bus 9b]
> [  202.341539] [   T1430] pci 0000:99:01.0:   bridge window [mem 0xe9600000-0xe96fffff]
> [  202.341680] [   T1430] pci 0000:99:01.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
> [  202.341950] [   T1430] pci 0000:99:02.0: PCI bridge to [bus 9c]
> [  202.342573] [   T1430] pci 0000:98:00.0: PCI bridge to [bus 99-9c]
> [  202.342685] [   T1430] pci 0000:98:00.0:   bridge window [mem 0xe9600000-0xe96fffff]
> [  202.342766] [   T1430] pci 0000:98:00.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
> [  202.342929] [   T1430] pcieport 0000:96:01.0: PCI bridge to [bus 98-9c]
> [  202.342951] [   T1430] pcieport 0000:96:01.0:   bridge window [mem 0xe9600000-0xe96fffff]
> [  202.342964] [   T1430] pcieport 0000:96:01.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
>
> // although the bridges don't have IO windows, the dbg message showing the window extension may be a bit worrisome.
> [  202.340536] [   T1430] pci 0000:99:00.0: bridge window [io  size 0x0000 disabled] extended by 0x0000000000000555
> [  202.340538] [   T1430] pci 0000:99:01.0: bridge window [io  size 0x0000 disabled] extended by 0x0000000000000555
> [  202.340538] [   T1430] pci 0000:99:02.0: bridge window [io  size 0x0000 disabled] extended by 0x0000000000000555

This again is thanks to align being 0 because resource_size() == 0 so
ALIGN_DOWN_IF_NONZERO() doesn't do anything useful.

I wonder if it would make sense to handle the minimum alignment of bridge
windows properly in pci_resource_alignment(). It currently can return 0
for bridge windows but we've well-defined alignment lower bounds for those
which I think could be applied right there at source instead of having
the callers have to worry about getting the alignment lower bounding
right.

--
 i.

> Test 2: realloc ON
> // code boot
> [    2.146003] [      T1] pci_bus 0000:96: resource 1 [mem 0xe1800000-0xe1dfffff]
> [    2.146004] [      T1] pci_bus 0000:96: resource 2 [mem 0x134000000000-0x1348004fffff 64bit pref]
> [    2.146007] [      T1] pci_bus 0000:98: resource 0 [io  size 0x1000]
> [    2.146008] [      T1] pci_bus 0000:98: resource 1 [mem 0xe1a00000-0xe1bfffff]
> [    2.146009] [      T1] pci_bus 0000:98: resource 2 [mem 0x134000000000-0x1347ffffffff 64bit pref]
> [    2.146010] [      T1] pci_bus 0000:99: resource 1 [mem 0xe1a00000-0xe1afffff]
> [    2.146011] [      T1] pci_bus 0000:99: resource 2 [mem 0x134000000000-0x1347ffffffff 64bit pref]
> [    2.146012] [      T1] pci_bus 0000:9b: resource 1 [mem 0xe1a00000-0xe1afffff]
> [    2.146013] [      T1] pci_bus 0000:9b: resource 2 [mem 0x134000000000-0x1347ffffffff 64bit pref]
>
> // after SBR
> // USP 98:00.0's windows still look correct
> [  482.943427] [   T1435] pci 0000:98:00.0: bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]: assigned
> [  482.943429] [   T1435] pci 0000:98:00.0: bridge window [mem 0xe1a00000-0xe1bfffff]: assigned
> // DSP 99:01.0 pref window is good
> [  482.943430] [   T1435] pci 0000:99:01.0: bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]: assigned
> // DSP 99:01.0 non-pref window ending addr is wrong: 0xe1a55554
> [  482.943431] [   T1435] pci 0000:99:01.0: bridge window [mem 0xe1a00000-0xe1a55554]: assigned
> [  482.943432] [   T1435] pci 0000:99:00.0: PCI bridge to [bus 9a]
> [  482.944063] [   T1435] pci 0000:9b:00.0: BAR 0 [mem 0x134000000000-0x1347ffffffff 64bit pref]: assigned
> [  482.944164] [   T1435] pci 0000:9b:00.0: BAR 2 [mem 0xe1a00000-0xe1a3ffff]: assigned
> [  482.944202] [   T1435] pci 0000:99:01.0: PCI bridge to [bus 9b]
> [  482.944424] [   T1435] pci 0000:99:01.0:   bridge window [mem 0xe1a00000-0xe1a55554]
> [  482.944565] [   T1435] pci 0000:99:01.0:   bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]
> [  482.944837] [   T1435] pci 0000:99:02.0: PCI bridge to [bus 9c]
> [  482.945463] [   T1435] pci 0000:98:00.0: PCI bridge to [bus 99-9c]
> [  482.945575] [   T1435] pci 0000:98:00.0:   bridge window [mem 0xe1a00000-0xe1bfffff]
> [  482.945657] [   T1435] pci 0000:98:00.0:   bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]
> [  482.945820] [   T1435] pcieport 0000:96:01.0: PCI bridge to [bus 98-9c]
> [  482.945842] [   T1435] pcieport 0000:96:01.0:   bridge window [mem 0xe1a00000-0xe1bfffff]
> [  482.945856] [   T1435] pcieport 0000:96:01.0:   bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]
> [  482.945883] [   T1435] PCI: No. 2 try to assign unassigned res
> [  482.945885] [   T1435] pci 0000:99:00.0: disabling bridge window [mem size 0x00000000 64bit pref disabled] to [bus 9a] (unused)
> // DSP 99:00.0 doesn't have any EP attached. But the mem size 0x55555 looks very suspicious.
> [  482.945886] [   T1435] pci 0000:99:00.0: disabling bridge window [mem size 0x00055555 disabled] to [bus 9a] (unused)
> [  482.945887] [   T1435] pci 0000:99:02.0: disabling bridge window [mem size 0x00000000 64bit pref disabled] to [bus 9c] (unused)
> // Same for DSP 99:02.0
> [  482.945888] [   T1435] pci 0000:99:02.0: disabling bridge window [mem size 0x00055555 disabled] to [bus 9c] (unused)
>
> Hope the above info gives you some hints.
>