Re: [PATCH v7 1/9] arm64: dts: lx2160a-rev2: extend 32-bit, and add 64-bit pci regions

[Josua Mayer <josua@solid-run.com>]

Am 24.05.26 um 16:54 schrieb Josua Mayer:
> LX2160 SoC pci-e controller supports 64-bit memory regions up to 16GB,
> 32-bit regions up to 3GB and 16-bit regions up to 64k.
>
> For each pci-e controller:
> - extend the existing 32-bit regions to 3GB size
> - add 64-bit region
> See [1] and [2] for boot messages showing ranges before and after.
>
> On LX2160A Silicon revision 1, the pcie driver fails to program atu for
> ranges larger than 4GB [3]. Therefore changes are limited to revision 2.
>
> Similar memory allocation with similar flags was tested with UEFI and ACPI
> on pcie3 and pcie5, on a variety of nxp vendor fork versions.
>
> This patch was tested on Linux v7.1-rc1 and u-boot, with two pcie cards:
> - pcie5: Radeon Pro WX2100
> - pcie3: ADATA NVME
>
> This fixes allocation of large, and 64-bit BARs as requested by many pci
> cards - especially graphics processors or AI accelerators, e.g.:
>
> [    2.941187] pci 0000:01:00.0: BAR 0: no space for [mem size 0x200000000 64bit pref]
> [    2.948834] pci 0000:01:00.0: BAR 0: failed to assign [mem size 0x200000000 64bit pref]
>
> [1] example of new allocations (pcie5):
> [    1.182745] layerscape-pcie 3800000.pcie: host bridge /soc/pcie@3800000 ranges:
> [    1.182760] layerscape-pcie 3800000.pcie:      MEM 0xa400000000..0xa7ffffffff -> 0xa400000000
> [    1.182771] layerscape-pcie 3800000.pcie:      MEM 0xa040000000..0xa0ffffffff -> 0x0040000000
> [    1.182778] layerscape-pcie 3800000.pcie:       IO 0xa000010000..0xa00001ffff -> 0x0000000000
> [    1.183642] layerscape-pcie 3800000.pcie: iATU: unroll F, 256 ob, 24 ib, align 4K, limit 4G
> [    1.385429] layerscape-pcie 3800000.pcie: PCIe Gen.3 x8 link up
> [    1.385481] layerscape-pcie 3800000.pcie: PCI host bridge to bus 0001:00
> [    1.385484] pci_bus 0001:00: root bus resource [bus 00-ff]
> [    1.385488] pci_bus 0001:00: root bus resource [mem 0xa400000000-0xa7ffffffff pref]
> [    1.385491] pci_bus 0001:00: root bus resource [mem 0xa040000000-0xa0ffffffff] (bus address [0x40000000-0xffffffff])
> [    1.385494] pci_bus 0001:00: root bus resource [io  0x10000-0x1ffff] (bus address [0x0000-0xffff])
> [    1.385516] pci 0001:00:00.0: [1957:8d80] type 01 class 0x060400 PCIe Root Port
> [    1.385538] pci 0001:00:00.0: PCI bridge to [bus 01-ff]
> [    1.385544] pci 0001:00:00.0:   bridge window [io  0x11000-0x11fff]
> [    1.385548] pci 0001:00:00.0:   bridge window [mem 0xa040000000-0xa0502fffff]
> [    1.385605] pci 0001:00:00.0: supports D1 D2
> [    1.385607] pci 0001:00:00.0: PME# supported from D0 D1 D2 D3hot
> [    1.386778] pci 0001:01:00.0: [1002:6995] type 00 class 0x030000 PCIe Legacy Endpoint
> [    1.387336] pci 0001:01:00.0: BAR 0 [mem 0xa040000000-0xa04fffffff 64bit pref]
> [    1.387368] pci 0001:01:00.0: BAR 2 [mem 0xa050000000-0xa0501fffff 64bit pref]
> [    1.387385] pci 0001:01:00.0: BAR 4 [io  0x11000-0x110ff]
> [    1.387402] pci 0001:01:00.0: BAR 5 [mem 0xa050200000-0xa05023ffff]
> [    1.387418] pci 0001:01:00.0: ROM [mem 0xa050240000-0xa05025ffff pref]
> [    1.387493] pci 0001:01:00.0: enabling Extended Tags
> [    1.388960] pci 0001:01:00.0: supports D1 D2
>
> [2] example of previous allocations (pcie5):
> [    1.716744] layerscape-pcie 3800000.pcie: host bridge /soc/pcie@3800000 ranges:
> [    1.724060] layerscape-pcie 3800000.pcie:      MEM 0xa040000000..0xa07fffffff -> 0x0040000000
> [    1.733277] layerscape-pcie 3800000.pcie: iATU: unroll F, 256 ob, 24 ib, align 4K, limit 4G
> [    1.836220] layerscape-pcie 3800000.pcie: PCIe Gen.3 x8 link up
> [    1.842186] layerscape-pcie 3800000.pcie: PCI host bridge to bus 0001:00
> [    1.848883] pci_bus 0001:00: root bus resource [bus 00-ff]
> [    1.854363] pci_bus 0001:00: root bus resource [mem 0xa040000000-0xa07fffffff] (bus address [0x40000000-0x7fffffff])
> [    1.864892] pci 0001:00:00.0: [1957:8d80] type 01 class 0x060400 PCIe Root Port
> [    1.872216] pci 0001:00:00.0: PCI bridge to [bus 01-ff]
> [    1.877438] pci 0001:00:00.0:   bridge window [io  0x1000-0x1fff]
> [    1.883526] pci 0001:00:00.0:   bridge window [mem 0xa040000000-0xa0502fffff]
>
> [3] error programming atu beyond 4GB:
> [    1.716762] layerscape-pcie 3800000.pcie: host bridge /soc/pcie@3800000 ranges:
> [    1.724080] layerscape-pcie 3800000.pcie:      MEM 0xa400000000..0xa7ffffffff -> 0xa400000000
> [    1.732615] layerscape-pcie 3800000.pcie:      MEM 0xa040000000..0xa0ffffffff -> 0x0040000000
> [    1.741142] layerscape-pcie 3800000.pcie:       IO 0xa010000000..0xa01000ffff -> 0x0000000000
> [    1.750379] layerscape-pcie 3800000.pcie: iATU: unroll F, 256 ob, 24 ib, align 4K, limit 4G
> [    1.759089] layerscape-pcie 3800000.pcie: Failed to set MEM range [mem 0xa400000000-0xa7ffffffff flags 0x2200]
> [    1.769089] layerscape-pcie 3800000.pcie: probe with driver layerscape-pcie failed with error -22
>
> [4] pci bootloaderp atching related errors with IORESOURCE_MEM_64 flag:
> [    0.967809] layerscape-pcie 3800000.pcie: host bridge /soc/pcie@3800000 ranges:
> [    0.967830] layerscape-pcie 3800000.pcie:      MEM 0xa400000000..0xa7ffffffff -> 0xa400000000
> [    0.967842] layerscape-pcie 3800000.pcie:      MEM 0xa040000000..0xa0ffffffff -> 0x0040000000
> [    0.967849] layerscape-pcie 3800000.pcie:       IO 0xa000010000..0xa00001ffff -> 0x0000000000
> [    1.169315] pci 0000:01:00.0: [8086:1572] type 00 class 0x020000 PCIe Endpoint
> [    1.169733] pci 0000:01:00.0: BAR 0 [mem 0x00000000-0x00ffffff 64bit pref]
> [    1.169771] pci 0000:01:00.0: BAR 3 [mem 0x00000000-0x00007fff 64bit pref]
> [    1.169796] pci 0000:01:00.0: ROM [mem 0x00000000-0x0007ffff pref]
> [    1.173389] OF: /soc/pcie@3800000: no msi-map translation for id 0x100 on (null)
> [    1.173515] OF: /soc/pcie@3800000: no iommu-map translation for id 0x100 on (null)
I meant to drop this bootloader error log, because after fixing the ranges flags
this is no longer an issue with this v7 patch.