Re: [PATCH 9/9] [DNI] ARM: multi_v7_defconfig: Enable CONFIG_ARM_LPAE for multi_v7_config

[Alexander Stein <alexander.stein@ew.tq-group.com>]

Hi Arnd,

Am Donnerstag, 19. Januar 2023, 17:07:30 CET schrieb Arnd Bergmann:
> On Thu, Jan 19, 2023, at 16:27, Alexander Stein wrote:
> > Am Donnerstag, 19. Januar 2023, 16:09:05 CET schrieb Arnd Bergmann:
> >> On Thu, Jan 19, 2023, at 15:42, Alexander Stein wrote:
> >> > This is necessary to support PCIe on LS1021A.
> >> > 
> >> > Signed-off-by: Alexander Stein <alexander.stein@ew.tq-group.com>
> >> 
> >> Can you explain why this is actually required? I can see that the
> >> ranges in the PCIe device point to a high address (0x4000000000,
> >> 2^40), but I can't tell if this is hardwired in the SoC or a
> >> setting that is applied by software (either the bootloader or
> >> the PCIe driver).
> > 
> > The RM ([1]) memory map (Table 2-1) says that 'PCI Express 1' is located
> > at
> > '400000_0000', 'PCI Express 2' at '480000_0000', so I assume this is
> > hardcoded in SoC.
> > It also explicitly lists in that table PCIe 1&2 is only accessible with
> > 40-bit addressing.
> > 
> >> If you can reprogram the memory map, I would expect this to fit
> >> easily into the 32-bit address space, with 1GB for DDR3 memory
> >> and 1GB for PCIe BARs.
> > 
> > I'm not sure which part of memory map you can reprogram and where, but I
> > guess this is fixed on this SoC.
> 
> Ok, I see it now. It looks like they fell victim to the
> cursed "Principles of ARM® Memory Maps White Paper"
> document and messed it up even further ;-)
> 
> In particular, it seems that the memory map of the PCI address
> spaces is configurable, but only within that area you listed.
> I see that section "28.4.2 PEX register descriptions" does list
> a 64-bit prefetchable address space in addition to the 32-bit
> non-prefetchable memory space, but the 64-bit space is not
> listed in the DT. It would be a good idea to configure that
> as well in order for devices to work that need a larger BAR,
> such as a GPU, but it wouldn't help with fitting the PCIe
> into non-LPAE 32-bit CPU address space.

I'm not sure if I can follow you here. Do you have some keywords of what's 
missing there?

> In the datasheet I also see that the chip theoretically
> supports 8GB of DDR4, which would definitely put it beyond
> the highmem limit, even with the 4G:4G memory split. Do you
> know if there are ls1021a devices with more than 4GB of
> installed memory?

Where did you find those 8GB? Section 16.2 mentions it supports up to 4 banks/
chip-selects which I would assume is much more. Also the memory map has a DRAM 
region 2 for memory region 2-32GB. But yes this exceeds 32bit addressing.
I'm not aware of ls1021 devices with more than 4GB memory. Our modules only 
support up to 2GB.

Best regards,
Alexander