Re: [PATCH v7 1/9] arm64: dts: lx2160a-rev2: extend 32-bit, and add 64-bit pci regions

[Josua Mayer <josua@solid-run.com>]

Am 24.05.26 um 17:13 schrieb sashiko-bot@kernel.org:
> Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
> - [High] Expanding the 32-bit PCI outbound window to 3GB causes it to overlap with the inbound system memory address space, breaking DMA.
> - [High] The introduction of the 64-bit PCI memory range causes existing U-Boot bootloaders to fail device tree patching, breaking MSI and IOMMU.
> --
>
> commit a80e2e3f97808fdad781240469c5e07e019191e9
> Author: Josua Mayer <josua@solid-run.com>
>
> arm64: dts: lx2160a-rev2: extend 32-bit, and add 64-bit pci regions
>
> This commit extends the 32-bit PCIe memory regions to 3GB and adds 64-bit
> PCIe memory regions for LX2160A rev2. This allows allocation of large BARs
> requested by graphics processors or AI accelerators.
>
>> [4] pci bootloaderp atching related errors with IORESOURCE_MEM_64 flag:
>> [    0.967809] layerscape-pcie 3800000.pcie: host bridge /soc/pcie@3800000 ranges:
>> [    0.967830] layerscape-pcie 3800000.pcie:      MEM 0xa400000000..0xa7ffffffff -> 0xa400000000
>> [    0.967842] layerscape-pcie 3800000.pcie:      MEM 0xa040000000..0xa0ffffffff -> 0x0040000000
>> [    0.967849] layerscape-pcie 3800000.pcie:       IO 0xa000010000..0xa00001ffff -> 0x0000000000
>> [    1.169315] pci 0000:01:00.0: [8086:1572] type 00 class 0x020000 PCIe Endpoint
>> [    1.169733] pci 0000:01:00.0: BAR 0 [mem 0x00000000-0x00ffffff 64bit pref]
>> [    1.169771] pci 0000:01:00.0: BAR 3 [mem 0x00000000-0x00007fff 64bit pref]
>> [    1.169796] pci 0000:01:00.0: ROM [mem 0x00000000-0x0007ffff pref]
>> [    1.173389] OF: /soc/pcie@3800000: no msi-map translation for id 0x100 on (null)
>> [    1.173515] OF: /soc/pcie@3800000: no iommu-map translation for id 0x100 on (null)
> [Severity: High]
> Does adding the 64-bit PCI memory range cause existing U-Boot bootloaders to
> fail device tree patching?
>
> When U-Boot encounters the 64-bit range, it appears fdt_fixup_pcie() aborts
> and fails to append the msi-map and iommu-map properties.
>
> Without iommu-map, the kernel will not attach the SMMU to the PCIe devices.
> Without msi-map, the kernel cannot translate PCIe Requester IDs to ITS
> Device IDs, which breaks MSI/MSI-X allocations.
>
> Could this prevent devices like NVMe drives from initializing and break
> compatibility with existing deployed bootloaders?
This was a stray error from previous version and was resolved by passing
correct flags, as present in v7.
>
>> diff --git a/arch/arm64/boot/dts/freescale/fsl-lx2160a-rev2.dtsi b/arch/arm64/boot/dts/freescale/fsl-lx2160a-rev2.dtsi
>> index f54005e37924b..db1ebee53f6f0 100644
>> --- a/arch/arm64/boot/dts/freescale/fsl-lx2160a-rev2.dtsi
>> +++ b/arch/arm64/boot/dts/freescale/fsl-lx2160a-rev2.dtsi
>> @@ -14,8 +14,9 @@ &pcie1 {
>>  	      0x80 0x00000000 0x0 0x00002000>; /* configuration space */
>>  	reg-names = "regs", "config";
>>  
>> -	ranges = <0x81000000 0x0 0x00000000 0x80 0x00010000 0x0 0x00010000
>> -		  0x82000000 0x0 0x40000000 0x80 0x40000000 0x0 0x40000000>;
>> +	ranges = <0x81000000 0x00 0x00000000 0x80 0x00010000 0x00 0x00010000>, /* 16-Bit IO Window */
>> +		 <0x82000000 0x00 0x40000000 0x80 0x40000000 0x00 0xc0000000>, /* 32-Bit - non-prefetchable */
>> +		 <0xC3000000 0x84 0x00000000 0x84 0x00000000 0x04 0x00000000>; /* 64-Bit - prefetchable - 16GB */
> [Severity: High]
> Does expanding the 32-bit PCI outbound window to 3GB (0xc0000000) cause it to
> overlap with inbound system memory?
>
> System memory starts at physical address 0x80000000. Since there isn't a
> dma-ranges property mapping inbound DMA to a different bus address, system
> memory uses a 1:1 mapping, meaning inbound DMA targets 0x80000000 and above.
>
> If a PCIe device requests a large 32-bit BAR, the kernel might allocate it
> within this 0x40000000-0xffffffff range. This causes the Root Port's
> downstream bridge Memory Limit register to be programmed to a value >=
> 0x80000000.
>
> A Root Port won't forward upstream memory transactions if the address falls
> within its programmed downstream window, as it assumes it is peer-to-peer
> traffic.
>
> Could this intercept endpoint DMA targeting system memory in the overlapped
> range and cause system hangs?
This configuration was tested with NVMe and GPUs,
the SMMU should ensure isolation pci bus addresses do not overlap with
physical ram base address.