Re: [PATCH v6 0/6] Improve PCI memory mapping API

[Damien Le Moal <dlemoal@kernel.org>]

On 10/23/24 00:30, Manivannan Sadhasivam wrote:
> On Tue, Oct 22, 2024 at 04:16:24PM +0200, Niklas Cassel wrote:
>> On Tue, Oct 22, 2024 at 07:26:31PM +0530, Manivannan Sadhasivam wrote:
>>> On Tue, Oct 22, 2024 at 10:38:58AM +0200, Niklas Cassel wrote:
>>>> On Tue, Oct 22, 2024 at 10:51:53AM +0900, Damien Le Moal wrote:
>>>>> On 10/22/24 07:19, Bjorn Helgaas wrote:
>>>>>> On Sat, Oct 12, 2024 at 08:32:40PM +0900, Damien Le Moal wrote:
>>>
>>>> However, if you think about a generic DMA controller, e.g. an ARM primecell
>>>> pl330, I don't see how it that DMA controller will be able to perform
>>>> transfers correctly if there is not an iATU mapping for the region that it
>>>> is reading/writing to.
>>>>
>>>
>>> I don't think the generic DMA controller can be used to read/write to remote
>>> memory. It needs to be integrated with the PCIe IP so that it can issue PCIe
>>> transactions.
>>
>> I'm not an expert, so I might of course be misunderstanding how things work.
>>
> 
> Neither am I :) I'm just sharing my understanding based on reading the DWC spec
> and open to get corrected if I'm wrong.
> 
>> When the CPU performs an AXI read/write to a MMIO address within the PCIe
>> controller (specifically the PCIe controller's outbound memory window),
>> the PCIe controller translates that incoming read/write to a read/write on the
>> PCIe bus.
>>
> 
> I don't think the *PCIe controller* translates the read/writes, but the iATU. If
> we use iATU, then the remote address needs to be mapped to the endpoint DDR and
> if CPU performs AXI read/write to that address, then iATU will translate the DDR
> address to remote address and then issue PCIe transactions (together with the
> PCIe controller).
> 
> And if DMA is used, then DMA controller can issue PCIe transactions to the
> remote memory itself (again, together with the PCIe controller). So no mapping
> is required here.

Caveat: DMA here cannot be the case "using the generic memory copy offload DMA
channel". It must be DMA driven by hardware rx/tx DMA channels. For memory copy
offload DMA channel, an EPF must map the address range to "DMA" to (which is
really memcpy_toio/fromio() but the DMA API will hide that...).

Would love to have so EPF wrapper API around all that mess to simplify EPF
drivers. They all end up having to do the exact same things.

> 
> - Mani
> 


-- 
Damien Le Moal
Western Digital Research