Re: [PATCH 24/25] PCI: Perform reset_resource() and build fail list in sync

["Ilpo Järvinen" <ilpo.jarvinen@linux.intel.com>]

[-- Attachment #1: Type: text/plain, Size: 9223 bytes --]

On Mon, 31 Mar 2025, Guenter Roeck wrote:
> On Mon, Dec 16, 2024 at 07:56:31PM +0200, Ilpo Järvinen wrote:
> > Resetting resource is problematic as it prevent attempting to allocate
> > the resource later, unless something in between restores the resource.
> > Similarly, if fail_head does not contain all resources that were reset,
> > those resource cannot be restored later.
> > 
> > The entire reset/restore cycle adds complexity and leaving resources
> > into reseted state causes issues to other code such as for checks done
> > in pci_enable_resources(). Take a small step towards not resetting
> > resources by delaying reset until the end of resource assignment and
> > build failure list (fail_head) in sync with the reset to avoid leaving
> > behind resources that cannot be restored (for the case where the caller
> > provides fail_head in the first place to allow restore somewhere in the
> > callchain, as is not all callers pass non-NULL fail_head).
> > 
> > The Expansion ROM check is temporarily left in place while building the
> > failure list until the upcoming change which reworks optional resource
> > handling.
> > 
> > Ideally, whole resource reset could be removed but doing that in a big
> > step would make the impact non-tractable due to complexity of all
> > related code.
> > 
> > Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
> 
> With this patch in the mainline kernel, all mips:boston qemu emulations
> fail when running a 64-bit little endian configuration (64r6el_defconfig).
> 
> The problem is that the PCI based IDE/ATA controller is not initialized.
> There are a number of pci error messages.
> 
> pci_bus 0002:01: extended config space not accessible
> pci 0002:01:00.0: [8086:2922] type 00 class 0x010601 conventional PCI endpoint
> pci 0002:01:00.0: BAR 4 [io  0x0000-0x001f]
> pci 0002:01:00.0: BAR 5 [mem 0x00000000-0x00000fff]
> pci 0002:00:00.0: PCI bridge to [bus 01-ff]
> pci_bus 0002:01: busn_res: [bus 01-ff] end is updated to 01
> pci 0002:00:00.0: bridge window [mem 0x16000000-0x160fffff]: assigned
> pci 0002:00:00.0: bridge window [mem size 0x00100000 64bit pref]: can't assign; no space
> pci 0002:00:00.0: bridge window [mem size 0x00100000 64bit pref]: failed to assign
> pci 0002:00:00.0: bridge window [io  size 0x1000]: can't assign; no space
> pci 0002:00:00.0: bridge window [io  size 0x1000]: failed to assign
> pci 0002:00:00.0: bridge window [mem size 0x00100000]: can't assign; bogus alignment
> pci 0002:00:00.0: bridge window [mem 0x16000000-0x160fffff 64bit pref]: assigned
> pci 0002:00:00.0: bridge window [io  size 0x1000]: can't assign; no space
> pci 0002:00:00.0: bridge window [io  size 0x1000]: failed to assign
> pci 0002:01:00.0: BAR 5 [mem size 0x00001000]: can't assign; no space
> pci 0002:01:00.0: BAR 5 [mem size 0x00001000]: failed to assign
> pci 0002:01:00.0: BAR 4 [io  size 0x0020]: can't assign; no space
> pci 0002:01:00.0: BAR 4 [io  size 0x0020]: failed to assign
> pci 0002:01:00.0: BAR 5 [mem size 0x00001000]: can't assign; no space
> pci 0002:01:00.0: BAR 5 [mem size 0x00001000]: failed to assign
> pci 0002:01:00.0: BAR 4 [io  size 0x0020]: can't assign; no space
> pci 0002:01:00.0: BAR 4 [io  size 0x0020]: failed to assign
> pci 0002:00:00.0: PCI bridge to [bus 01]
> pci 0002:00:00.0:   bridge window [mem 0x16000000-0x160fffff 64bit pref]
> pci_bus 0002:00: Some PCI device resources are unassigned, try booting with pci=realloc
> pci_bus 0002:00: resource 4 [mem 0x16000000-0x160fffff]
> pci_bus 0002:01: resource 2 [mem 0x16000000-0x160fffff 64bit pref]
> ...
> pci 0002:00:00.0: enabling device (0000 -> 0002)
> ahci 0002:01:00.0: probe with driver ahci failed with error -12
> 
> Bisect points to this patch. Reverting it together with "PCI: Rework
> optional resource handling" fixes the problem. For comparison, after
> reverting the offending patches, the log messages are as follows.
> 
> pci_bus 0002:00: root bus resource [bus 00-ff]
> pci_bus 0002:00: root bus resource [mem 0x16000000-0x160fffff]
> pci 0002:00:00.0: [10ee:7021] type 01 class 0x060400 PCIe Root Complex Integrated Endpoint
> pci 0002:00:00.0: PCI bridge to [bus 00]
> pci 0002:00:00.0:   bridge window [io  0x0000-0x0fff]
> pci 0002:00:00.0:   bridge window [mem 0x00000000-0x000fffff]
> pci 0002:00:00.0:   bridge window [mem 0x00000000-0x000fffff 64bit pref]
> pci 0002:00:00.0: enabling Extended Tags
> pci 0002:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> pci_bus 0002:01: extended config space not accessible
> pci 0002:01:00.0: [8086:2922] type 00 class 0x010601 conventional PCI endpoint
> pci 0002:01:00.0: BAR 4 [io  0x0000-0x001f]
> pci 0002:01:00.0: BAR 5 [mem 0x00000000-0x00000fff]
> pci 0002:00:00.0: PCI bridge to [bus 01-ff]
> pci_bus 0002:01: busn_res: [bus 01-ff] end is updated to 01
> pci 0002:00:00.0: bridge window [mem 0x16000000-0x160fffff]: assigned
> pci 0002:00:00.0: bridge window [mem size 0x00100000 64bit pref]: can't assign; no space
> pci 0002:00:00.0: bridge window [mem size 0x00100000 64bit pref]: failed to assign
> pci 0002:00:00.0: bridge window [io  size 0x1000]: can't assign; no space
> pci 0002:00:00.0: bridge window [io  size 0x1000]: failed to assign
> pci 0002:01:00.0: BAR 5 [mem 0x16000000-0x16000fff]: assigned
> pci 0002:01:00.0: BAR 4 [io  size 0x0020]: can't assign; no space
> pci 0002:01:00.0: BAR 4 [io  size 0x0020]: failed to assign
> pci 0002:00:00.0: PCI bridge to [bus 01]
> pci 0002:00:00.0:   bridge window [mem 0x16000000-0x160fffff]
> pci_bus 0002:00: Some PCI device resources are unassigned, try booting with pci=realloc
> pci_bus 0002:00: resource 4 [mem 0x16000000-0x160fffff]
> pci_bus 0002:01: resource 1 [mem 0x16000000-0x160fffff]
> ...
> pci 0002:00:00.0: enabling device (0000 -> 0002)
> ahci 0002:01:00.0: enabling device (0000 -> 0002)
> ahci 0002:01:00.0: AHCI vers 0001.0000, 32 command slots, 1.5 Gbps, SATA mode
> ahci 0002:01:00.0: 6/6 ports implemented (port mask 0x3f)
> ahci 0002:01:00.0: flags: 64bit ncq only

Hi,

Thanks for reporting. Please add this to the command line to get the 
resource releasing between the steps to show:

dyndbg="file drivers/pci/setup-bus.c +p"

Also, the log snippet just shows it fails but it is impossible to know 
from it why the resource assigments do not fit so could you please provide 
a complete dmesg logs. Also providing the contents of /proc/iomem from the 
working case would save me quite a bit of decoding the iomem layout from 
the dmesgs.

> Bisect log is attached for reference.
> 
> Guenter
> ---
> # bad: [609706855d90bcab6080ba2cd030b9af322a1f0c] Merge tag 'trace-latency-v6.15-3' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
> # good: [acb4f33713b9f6cadb6143f211714c343465411c] Merge tag 'm68knommu-for-v6.15' of git://git.kernel.org/pub/scm/linux/kernel/git/gerg/m68knommu
> git bisect start 'HEAD' 'acb4f33713b9'
> # good: [cf05922d63e2ae6a9b1b52ff5236a44c3b29f78c] Merge tag 'drm-intel-gt-next-2025-03-12' of https://gitlab.freedesktop.org/drm/i915/kernel into drm-next
> git bisect good cf05922d63e2ae6a9b1b52ff5236a44c3b29f78c
> # bad: [93d52288679e29aaa44a6f12d5a02e8a90e742c5] Merge tag 'backlight-next-6.15' of git://git.kernel.org/pub/scm/linux/kernel/git/lee/backlight
> git bisect bad 93d52288679e29aaa44a6f12d5a02e8a90e742c5
> # bad: [e5e0e6bebef3a21081fd1057c40468d4cff1a60d] Merge tag 'v6.15-p1' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6
> git bisect bad e5e0e6bebef3a21081fd1057c40468d4cff1a60d
> # bad: [dea140198b846f7432d78566b7b0b83979c72c2b] Merge branch 'pci/misc'
> git bisect bad dea140198b846f7432d78566b7b0b83979c72c2b
> # bad: [a113afb84ae63ec4c893bc3204945ef6f3bb89f7] Merge branch 'pci/endpoint'
> git bisect bad a113afb84ae63ec4c893bc3204945ef6f3bb89f7
> # good: [a7a8e7996c1c114b50df5599229b1e7be38be3db] Merge branch 'pci/reset'
> git bisect good a7a8e7996c1c114b50df5599229b1e7be38be3db
> # bad: [7d4bcc0f2631e4ee10b5bcfff24a423d1c3c02a3] PCI: Move resource reassignment func declarations into pci/pci.h
> git bisect bad 7d4bcc0f2631e4ee10b5bcfff24a423d1c3c02a3
> # good: [acba174d2e754346c07578ad2220258706a203e2] PCI: Use while loop and break instead of gotos
> git bisect good acba174d2e754346c07578ad2220258706a203e2
> # good: [8884b5637b794ae541e8d6fb72102b1d8dba2b8d] PCI: Add debug print when releasing resources before retry
> git bisect good 8884b5637b794ae541e8d6fb72102b1d8dba2b8d
> # bad: [5af473941b56189423a7d16c05efabaf77299847] PCI: Increase Resizable BAR support from 512 GB to 128 TB
> git bisect bad 5af473941b56189423a7d16c05efabaf77299847
> # bad: [96336ec702643aec2addb3b1cdb81d687fe362f0] PCI: Perform reset_resource() and build fail list in sync
> git bisect bad 96336ec702643aec2addb3b1cdb81d687fe362f0
> # good: [e89df6d2beae847e931d84a190b192dfac41eba7] PCI: Use res->parent to check if resource is assigned
> git bisect good e89df6d2beae847e931d84a190b192dfac41eba7
> # first bad commit: [96336ec702643aec2addb3b1cdb81d687fe362f0] PCI: Perform reset_resource() and build fail list in sync
> 

-- 
 i.