Re: [BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery

Bjorn Helgaas
2026-03-11T23:19:44Z

[+cc Ilpo]

On Wed, Mar 11, 2026 at 10:00:39PM +0000, Shawn Jin wrote:
> Hello,
>
> I'm reporting a potential critical bug in the Linux kernel's PCIe
> resource allocation code that creates invalid bridge window limit
> addresses during hotplug re-enumeration after Secondary Bus Reset
> (SBR) recovery.

Thanks for the report and the repro and debugging information!

> ## AFFECTED KERNEL VERSIONS
> - Confirmed: 5.15.0, 6.8.0 (Ubuntu 6.8.0-88-generic, 6.8.0-90-generic)
> - Likely affected: All recent kernels including 6.19

Do you know of any kernels that are *not* affected? If you do, we
could bisect.

> ## HARDWARE CONFIGURATION
> Intel Ice Lake server with PCIe Gen5 switches and endpoints:
>
> Topology 1:
>   Root Port 96:01.0 → 98:00.0 → 99:01.0 → 9b:00.0 (NVIDIA L20 GPU)
>
> Kernel parameter: pci=realloc=on
>
> ## PROBLEM DESCRIPTION
>
> After performing Secondary Bus Reset on a PCIe switch port and
> clearing the reset bit, the kernel re-enumerates devices and assigns
> bridge window resources. However, the assigned memory window limit
> addresses are INVALID according to PCIe specification.
>
> ### Evidence from dmesg (Topology 1):
>
> **Before SBR (correct allocation):**
> ```
> [ 6.636493] pci 0000:98:00.0: PCI bridge to [bus 99-9c]
> [ 6.636539] pci 0000:98:00.0: bridge window [mem 0xe9600000-0xe96fffff]
> [ 6.636645] pci 0000:98:00.0: bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
>
> [ 6.644429] pci 0000:99:01.0: PCI bridge to [bus 9b]
> [ 6.644476] pci 0000:99:01.0: bridge window [mem 0xe9600000-0xe96fffff]
> [ 6.644656] pci 0000:99:01.0: bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
>
> [ 6.654203] pci 0000:9b:00.0: [1e3e:0002] type 00 class 0x120000 PCIe Endpoint
> [ 6.654652] pci 0000:9b:00.0: BAR 0 [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
> [ 6.654666] pci 0000:9b:00.0: BAR 2 [mem 0xe9600000-0xe963ffff]
> ```
>
> **After SBR clear (INVALID allocation):**
> ```
> [ 656.644184] pci 0000:98:00.0: bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]: assigned
> [ 656.644186] pci 0000:98:00.0: bridge window [mem 0xe9600000-0xe96fffff]: assigned
> [ 656.644188] pci 0000:99:01.0: bridge window [mem 0x13b000000000-0x13b7fffffffe 64bit pref]: assigned
> [ 656.644189] pci 0000:99:01.0: bridge window [mem 0xe9600000-0xe96ffffe]: assigned
>
> [ 656.644830] pci 0000:9b:00.0: BAR 0 [mem size 0x800000000 64bit pref]: can't assign; no space
> [ 656.644831] pci 0000:9b:00.0: BAR 0 [mem size 0x800000000 64bit pref]: failed to assign
> // BAR2 can still be assigned because the size is only 256KB, while the min window in the bridge is 1MB
> [ 656.644832] pci 0000:9b:00.0: BAR 2 [mem 0xe9600000-0xe963ffff]: assigned
>
> ```
>
> ### Invalid Addresses Created by Kernel:
> - `0x13b7ffffffff` (ends in 0xFFFE - **2 bytes short**)
> - `0xe96ffffe` (ends in 0xFFFE - **2 bytes short**)
>
> ## IMPACT
>
> 1. **Device initialization failure**: Endpoints cannot allocate required BARs
>    ```
> [ 656.644830] pci 0000:9b:00.0: BAR 0 [mem size 0x800000000 64bit pref]: can't assign; no space
> [ 656.644831] pci 0000:9b:00.0: BAR 0 [mem size 0x800000000 64bit pref]: failed to assign
>    ```
>
> 2. **Consistent across multiple hierarchies**: Affects different PCIe topologies independently
>
> ## REPRODUCTION
>
> The attached script test_rc_sbr.sh.txt issues a SBR to the root port.
>
> ## SUSPECTED ROOT CAUSE
>
> The bug appears to be in `drivers/pci/setup-bus.c`, likely in:
> - `pci_bus_distribute_available_resources()`
> - `adjust_bridge_window()`
> - `pci_assign_unassigned_bridge_resources()`
>
> The resource end address calculation appears to perform multiple subtractions:
> 1. Initial calculation: `res->end = res->start + size - 1` (correct)
> 2. During redistribution: Another subtraction occurs, creating `res->end = ... - 2`
>
> ## WORKAROUND ATTEMPTS
>
> - `pci=realloc=on`: Does NOT fix the issue
> - Manual remove/rescan from root: Does NOT fix the issue
> - Initial boot allocation: Works correctly (bug only occurs during hotplug re-enumeration)
>
> ## REQUEST
>
> I want to track how the bridge windows are allocated. Is there a way
> to enable additional kernel messages to show the path? Please
> investigate if this is a real kernel bug.
>
> Thank you,
> Shawn

> #!/bin/bash
>
> # Function to display usage
> usage() {
> echo "Usage: $0 -rp <ROOT_PORT_BDF> -usp <USP_BDF>"
> echo "Example: $0 -rp c6:01.0 -usp c7:00.0"
> exit 1
> }
>
> # Initialize variables
> ROOT_PORT_BDF=""
> USP_BDF=""
>
> # Parse command-line arguments
> while [[ $# -gt 0 ]]; do
> case $1 in
> -rp)
> ROOT_PORT_BDF="$2"
> shift 2
> ;;
> -usp)
> USP_BDF="$2"
> shift 2
> ;;
> -h|--help)
> usage
> ;;
> *)
> echo "Unknown option: $1"
> usage
> ;;
> esac
> done
>
> # Validate that both arguments are provided
> if [ -z "$ROOT_PORT_BDF" ] || [ -z "$USP_BDF" ]; then
> echo "Error: Both -rp and -usp arguments are required"
> usage
> fi
>
> echo "Root Port BDF: $ROOT_PORT_BDF"
> echo "USP BDF: $USP_BDF"
> echo ""
>
> # Remove the USP device
> echo 1 | sudo tee /sys/bus/pci/devices/0000:${USP_BDF}/remove
>
> # Trigger SBR via Bridge Control register
> BRIDGE_CTL=$(sudo setpci -s ${ROOT_PORT_BDF} 0x3E.w)
> BRIDGE_CTL_RESET=$(printf "0x%04x" $((0x$BRIDGE_CTL | 0x0040)))
>
> echo "Asserting Secondary Bus Reset..."
> sudo setpci -s ${ROOT_PORT_BDF} 0x3E.w=$BRIDGE_CTL_RESET
> sleep 1
>
> echo "De-asserting Secondary Bus Reset..."
> sudo setpci -s ${ROOT_PORT_BDF} 0x3E.w=$BRIDGE_CTL
> sleep 2
>
> # Rescan PCI bus
> echo 1 | sudo tee /sys/bus/pci/rescan