Re: [PATCH v6 0/6] Improve PCI memory mapping API

[Niklas Cassel <cassel@kernel.org>]

On Wed, Oct 23, 2024 at 11:51:41AM +0900, Damien Le Moal wrote:

(snip)

> For an endpoint initiated transfer, typically, the PCI address is obtained from
> some "command" received through a BAR or through DMA. The command has the PCI
> addresses to use for transfering data to/from the host (e.g. an nvme rw command
> uses PRPs or SGLs to specify the PCI address segments for the data buffer of a
> command). For this case, the EPF driver calls calls pci_epc_mem_map() for the
> command buffer, does the transfer (memcpy_toio/fromio()) and unmaps with
> pci_epc_mem_unmap(). Note though that here, if an eDMA channel is used for the
> transfer, the DMA engine will do the mapping automatically and the epf does not
> need to call pci_epc_mem_map()/pci_epc_mem_unmap(). There is still an issue in
> this area which is that it is *not* clear if the DMA channel used can actually
> do the mapping automatically or not. E.g. the generic DMA channel (mem copy
> offload engine) will not. So there is still some API improvement needed to
> abstract more HW dependent things here.

FWIW, in my final reply here:
https://lore.kernel.org/lkml/ZiYuIaX7ZV0exKMt@ryzen/

"
I did suggest that DWC-based drivers could set a DMA_SLAVE_SKIP_MEM_MAP flag
or similar when registering the eDMA, which pci-epf-test then could check,
but I got no response if anyone else thought that this was a good idea.
"


For DMA_SLAVE (private tx/rx DMA channels):
For DWC-based controllers, we can definitely set DMA_SLAVE_SKIP_MEM_MAP when
registering the eDMA (e.g. in dw_pcie_edma_detect()).

However, I don't know how the DMA hardware (if any) in:
drivers/pci/controller/cadence/pcie-cadence-ep.c
drivers/pci/controller/pcie-rcar-ep.c
drivers/pci/controller/pcie-rockchip-ep.c
works, so I'm not sure if those drivers can set DMA_SLAVE_SKIP_MEM_MAP
(the safest thing is to not set that flag, until we know how they work).


For DMA_MEMCPY:
We know that we need to perform pci_epc_mem_map() when using
DMA API + "dummy" memcpy dma-channel (DMA_MEMCPY).
I could imagine that some embedded DMA controllers also provide
DMA_MEMCPY capabilities (in addition to DMA_SLAVE).


So I guess the safest thing is to call the flag something like:
PCI_EPC_DMA_SKIP_MEM_MAP
(rather than PCI_EPC_DMA_SLAVE_SKIP_MEM_MAP).
(since the embedded DMA controller might provide both DMA_SLAVE and DMA_MEMCPY).

And let the EPC driver (e.g. dw_pcie_edma_detect()), or possibly the DMA
driver itself to provide/set this flag.


Kind regards,
Niklas