Re: [PATCH] fsldma: Support 40 bit DMA addresses where capable

["Arnd Bergmann" <arnd@arndb.de>]

On Tue, Apr 22, 2025, at 10:56, Ben Collins wrote:
> On Tue, Apr 22, 2025 at 09:59:42AM -0500, Arnd Bergmann wrote:
>> 
>> Right, but this could just mean that they end up using SWIOTLB
>> to bounce the high DMA pages or use an IOMMU rather than actually
>> translating the physical address to a dma address.
>
> There's a few things going on. The Local Address Window can shift
> anywhere in the 64-bit address space and be as wide as the physical
> address (40-bit on T4240, 36-bit on P4080). I think this is mainly for
> IO to PCIe and RapidIO, though.

There are usually two sets of registers, not sure which one the Local
Address Window refers to:

- Translation of MMIO addresses (PCI BAR and device registers) when
  accessed from CPU and possibly from P2P DMA, these are represented
  by the 'ranges' property in DT.

- Translation of physical memory when accessed from a DMA bus master,
  represented by the 'dma-ranges' property.

The latter is what the dma-mapping API needs. This code has changed
a lot over the years, but in the current version the idea is that
the limit enforced by the driver through dma_set_mask() is independent
of the limit enforced by the platform bus based on the dma-ranges
property. 

The bit that matters in the end is the intersection of both,
so dma_map_single() etc only maps a page that is addressable
by both the device and the bus.

>> > I'll check on this, but I think it's a seperate issue. The main thing is
>> > just to configure the dma hw correctly.
>> 
>> I think it's still important to check this before changing the
>> driver: if the larger mask doesn't actually have any effect now
>> because the DT caps the DMA at 4GB, then it might break later
>> when someone adds the correct dma-ranges properties.
>
> I'm adding dma-ranges to my dt for testing.

Ok. The other thing you can try is to printk() the dev->bus_dma_limit
to see if it even tries to use >32bit addressing.

>> > So a little research shows that these 3 compatible strings in
>> > the fsldma are:
>> >
>> > fsl,elo3-dma:		40-bit
>> > fsl,eloplus-dma:	36-bit
>> > fsl,elo-dma:		32-bit
>> >
>> > I'll rework it so addressing is based on the compatible string.
>> 
>> Sounds good, yes. Just to clarify: where did you find those
>> limits? Are you sure those are not just the maximum addressable
>> amounts of physical RAM on the chips that use the respective
>> controllers?
>
> This is where things might be more interesting. The P4080RM and T4240RM
> is where I got this information. Register "cdar" in the fsldma code. This
> makes up 0x08 and 0x0c registers.

> In the RM 0x08 is the extended address register. On P4080 it says this
> holds the top 4 bits of the 36-bit address, and on T4240 it says the top
> 8 bits of the 40-bit address. So the asynx_tx physical address needs to
> be masked to the 36-bit or 40-bit.

Ok, makes sense.

       Arnd