Re: [BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery

Shawn Jin
2026-03-16T17:26:48Z

________________________________________
From: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Sent: Monday, March 16, 2026 3:28 AM
To: Shawn Jin <shawn.jin@asteralabs.com>
Cc: Bjorn Helgaas <helgaas@kernel.org>; Bjorn Helgaas <bhelgaas@google.com>; linux-kernel@vger.kernel.org <linux-kernel@vger.kernel.org>; linux-pci@vger.kernel.org <linux-pci@vger.kernel.org>
Subject: Re: [BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery

[EXTERNAL EMAIL]

On Fri, 13 Mar 2026, Shawn Jin wrote:

> Hi Ilpo and Bjorn,
>
> A quick update on the test result with the kernel 6.19.6-1-default in openSUSE Tumbleweed.
>
> Summary:
> 1. With PCI realloc off, the bridge windows were reassigned as before, the ending addresses look normal, all ending with 0xffff.
> 2. With PCI realloc on, the non-pref window ending address is still incorrect: 0xe1a55554. But the pref window looks right.
> [  482.943430] [   T1435] pci 0000:99:01.0: bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]: assigned
> [  482.943431] [   T1435] pci 0000:99:01.0: bridge window [mem 0xe1a00000-0xe1a55554]: assigned

Was this window initially zero sized? That would explain why align becomes
zero.

<SJ> If "initially" means the windows assigned during the cold boot, then the answer is no. But the window resources were freed because of the SBR test it underwent.

The test script is as below.
# Remove the USP device
echo 1 | sudo tee /sys/bus/pci/devices/0000:${USP_BDF}/remove

# Trigger SBR via Bridge Control register
BRIDGE_CTL=$(sudo setpci -s ${ROOT_PORT_BDF} 0x3E.w)
BRIDGE_CTL_RESET=$(printf "0x%04x" $((0x$BRIDGE_CTL | 0x0040)))

echo "Asserting Secondary Bus Reset..."
sudo setpci -s ${ROOT_PORT_BDF} 0x3E.w=$BRIDGE_CTL_RESET
sleep 1

echo "De-asserting Secondary Bus Reset..."
sudo setpci -s ${ROOT_PORT_BDF} 0x3E.w=$BRIDGE_CTL
sleep 2

# Rescan PCI bus
echo 1 | sudo tee /sys/bus/pci/rescan

Additional dmesg showed that the window sizes were reset to 0.
[ 482.935950] [ T1435] pci 0000:99:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 482.936183] [ T1435] pci 0000:99:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 482.936401] [ T1435] pci 0000:99:02.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 482.938545] [ T1435] pci 0000:99:00.0: PCI bridge to [bus 9a-9c]
[ 482.938611] [ T1435] pci_bus 0000:9a: busn_res: [bus 9a-9c] end is updated to 9a
[ 482.939301] [ T1435] pci 0000:9b:00.0: [1e3e:0002] type 00 class 0x120000 PCIe Endpoint
[ 482.939735] [ T1435] pci 0000:9b:00.0: BAR 0 [mem 0x00000000-0x7ffffffff 64bit pref]
[ 482.939743] [ T1435] pci 0000:9b:00.0: BAR 2 [mem 0x00000000-0x0003ffff]
[ 482.939843] [ T1435] pci 0000:9b:00.0: Max Payload Size set to 256 (was 128, max 256)
[ 482.940817] [ T1435] pci 0000:9b:00.0: PME# supported from D0 D3hot
[ 482.942468] [ T1435] pci 0000:99:01.0: PCI bridge to [bus 9b-9c]
[ 482.942535] [ T1435] pci_bus 0000:9b: busn_res: [bus 9b-9c] end is updated to 9b
[ 482.943101] [ T1435] pci 0000:99:02.0: PCI bridge to [bus 9c]
[ 482.943169] [ T1435] pci_bus 0000:9c: busn_res: [bus 9c] end is updated to 9c
[ 482.943295] [ T1435] pci_bus 0000:99: busn_res: [bus 99-9c] end is updated to 9c
[ 482.943402] [ T1435] pci 0000:99:00.0: disabling bridge window [io size 0x0000 disabled] to [bus 9a] (unused)
[ 482.943405] [ T1435] pci 0000:99:00.0: disabling bridge window [mem size 0x00000000 64bit pref disabled] to [bus 9a] (unused)
[ 482.943406] [ T1435] pci 0000:99:00.0: disabling bridge window [mem size 0x00000000 disabled] to [bus 9a] (unused)
[ 482.943407] [ T1435] pci 0000:99:01.0: disabling bridge window [io size 0x0000 disabled] to [bus 9b] (unused)
[ 482.943409] [ T1435] pci 0000:99:02.0: disabling bridge window [io size 0x0000 disabled] to [bus 9c] (unused)
[ 482.943410] [ T1435] pci 0000:99:02.0: disabling bridge window [mem size 0x00000000 64bit pref disabled] to [bus 9c] (unused)
[ 482.943411] [ T1435] pci 0000:99:02.0: disabling bridge window [mem size 0x00000000 disabled] to [bus 9c] (unused)
[ 482.943412] [ T1435] pci 0000:98:00.0: disabling bridge window [io size 0x0000 disabled] to [bus 99-9c] (unused)

Thanks,
Shawn

> Details:
> Focused topology:
> -[0000:94]---01.0-[95-9e]----00.0-[96-9e]----01.0-[98-9c]----00.0-[99-9c]--+-00.0-[9a]--
>                                                                            +-01.0-[9b]----00.0  Device 1e3e:0002
>                                                                            \-02.0-[9c]--
> Test 1: realloc OFF
> // Cold boot
> [    2.132565] [      T1] pci 0000:99:00.0: PCI bridge to [bus 9a]
> [    2.133202] [      T1] pci 0000:99:01.0: PCI bridge to [bus 9b]
> [    2.133413] [      T1] pci 0000:99:01.0:   bridge window [mem 0xe9600000-0xe96fffff]
> [    2.133555] [      T1] pci 0000:99:01.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
> [    2.133827] [      T1] pci 0000:99:02.0: PCI bridge to [bus 9c]
> [    2.134453] [      T1] pci 0000:98:00.0: PCI bridge to [bus 99-9c]
> [    2.134565] [      T1] pci 0000:98:00.0:   bridge window [mem 0xe9600000-0xe96fffff]
> [    2.134647] [      T1] pci 0000:98:00.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
> [    2.134809] [      T1] pci 0000:96:01.0: PCI bridge to [bus 98-9c]
> [    2.134830] [      T1] pci 0000:96:01.0:   bridge window [mem 0xe9600000-0xe96fffff]
> [    2.134845] [      T1] pci 0000:96:01.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
> // after SBR
> [  202.341320] [   T1430] pci 0000:99:01.0: PCI bridge to [bus 9b]
> [  202.341539] [   T1430] pci 0000:99:01.0:   bridge window [mem 0xe9600000-0xe96fffff]
> [  202.341680] [   T1430] pci 0000:99:01.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
> [  202.341950] [   T1430] pci 0000:99:02.0: PCI bridge to [bus 9c]
> [  202.342573] [   T1430] pci 0000:98:00.0: PCI bridge to [bus 99-9c]
> [  202.342685] [   T1430] pci 0000:98:00.0:   bridge window [mem 0xe9600000-0xe96fffff]
> [  202.342766] [   T1430] pci 0000:98:00.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
> [  202.342929] [   T1430] pcieport 0000:96:01.0: PCI bridge to [bus 98-9c]
> [  202.342951] [   T1430] pcieport 0000:96:01.0:   bridge window [mem 0xe9600000-0xe96fffff]
> [  202.342964] [   T1430] pcieport 0000:96:01.0:   bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
>
> // although the bridges don't have IO windows, the dbg message showing the window extension may be a bit worrisome.
> [  202.340536] [   T1430] pci 0000:99:00.0: bridge window [io  size 0x0000 disabled] extended by 0x0000000000000555
> [  202.340538] [   T1430] pci 0000:99:01.0: bridge window [io  size 0x0000 disabled] extended by 0x0000000000000555
> [  202.340538] [   T1430] pci 0000:99:02.0: bridge window [io  size 0x0000 disabled] extended by 0x0000000000000555

This again is thanks to align being 0 because resource_size() == 0 so
ALIGN_DOWN_IF_NONZERO() doesn't do anything useful.

I wonder if it would make sense to handle the minimum alignment of bridge
windows properly in pci_resource_alignment(). It currently can return 0
for bridge windows but we've well-defined alignment lower bounds for those
which I think could be applied right there at source instead of having
the callers have to worry about getting the alignment lower bounding
right.

--
i.

> Test 2: realloc ON
> // code boot
> [    2.146003] [      T1] pci_bus 0000:96: resource 1 [mem 0xe1800000-0xe1dfffff]
> [    2.146004] [      T1] pci_bus 0000:96: resource 2 [mem 0x134000000000-0x1348004fffff 64bit pref]
> [    2.146007] [      T1] pci_bus 0000:98: resource 0 [io  size 0x1000]
> [    2.146008] [      T1] pci_bus 0000:98: resource 1 [mem 0xe1a00000-0xe1bfffff]
> [    2.146009] [      T1] pci_bus 0000:98: resource 2 [mem 0x134000000000-0x1347ffffffff 64bit pref]
> [    2.146010] [      T1] pci_bus 0000:99: resource 1 [mem 0xe1a00000-0xe1afffff]
> [    2.146011] [      T1] pci_bus 0000:99: resource 2 [mem 0x134000000000-0x1347ffffffff 64bit pref]
> [    2.146012] [      T1] pci_bus 0000:9b: resource 1 [mem 0xe1a00000-0xe1afffff]
> [    2.146013] [      T1] pci_bus 0000:9b: resource 2 [mem 0x134000000000-0x1347ffffffff 64bit pref]
>
> // after SBR
> // USP 98:00.0's windows still look correct
> [  482.943427] [   T1435] pci 0000:98:00.0: bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]: assigned
> [  482.943429] [   T1435] pci 0000:98:00.0: bridge window [mem 0xe1a00000-0xe1bfffff]: assigned
> // DSP 99:01.0 pref window is good
> [  482.943430] [   T1435] pci 0000:99:01.0: bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]: assigned
> // DSP 99:01.0 non-pref window ending addr is wrong: 0xe1a55554
> [  482.943431] [   T1435] pci 0000:99:01.0: bridge window [mem 0xe1a00000-0xe1a55554]: assigned
> [  482.943432] [   T1435] pci 0000:99:00.0: PCI bridge to [bus 9a]
> [  482.944063] [   T1435] pci 0000:9b:00.0: BAR 0 [mem 0x134000000000-0x1347ffffffff 64bit pref]: assigned
> [  482.944164] [   T1435] pci 0000:9b:00.0: BAR 2 [mem 0xe1a00000-0xe1a3ffff]: assigned
> [  482.944202] [   T1435] pci 0000:99:01.0: PCI bridge to [bus 9b]
> [  482.944424] [   T1435] pci 0000:99:01.0:   bridge window [mem 0xe1a00000-0xe1a55554]
> [  482.944565] [   T1435] pci 0000:99:01.0:   bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]
> [  482.944837] [   T1435] pci 0000:99:02.0: PCI bridge to [bus 9c]
> [  482.945463] [   T1435] pci 0000:98:00.0: PCI bridge to [bus 99-9c]
> [  482.945575] [   T1435] pci 0000:98:00.0:   bridge window [mem 0xe1a00000-0xe1bfffff]
> [  482.945657] [   T1435] pci 0000:98:00.0:   bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]
> [  482.945820] [   T1435] pcieport 0000:96:01.0: PCI bridge to [bus 98-9c]
> [  482.945842] [   T1435] pcieport 0000:96:01.0:   bridge window [mem 0xe1a00000-0xe1bfffff]
> [  482.945856] [   T1435] pcieport 0000:96:01.0:   bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]
> [  482.945883] [   T1435] PCI: No. 2 try to assign unassigned res
> [  482.945885] [   T1435] pci 0000:99:00.0: disabling bridge window [mem size 0x00000000 64bit pref disabled] to [bus 9a] (unused)
> // DSP 99:00.0 doesn't have any EP attached. But the mem size 0x55555 looks very suspicious.
> [  482.945886] [   T1435] pci 0000:99:00.0: disabling bridge window [mem size 0x00055555 disabled] to [bus 9a] (unused)
> [  482.945887] [   T1435] pci 0000:99:02.0: disabling bridge window [mem size 0x00000000 64bit pref disabled] to [bus 9c] (unused)
> // Same for DSP 99:02.0
> [  482.945888] [   T1435] pci 0000:99:02.0: disabling bridge window [mem size 0x00055555 disabled] to [bus 9c] (unused)
>
> Hope the above info gives you some hints.
>