Re: [RFC 0/7] Peer-direct memory

[Jason Gunthorpe <jgunthorpe@obsidianresearch.com>]

On Tue, Feb 16, 2016 at 08:13:58PM -0800, davide rossetti wrote:

> Bottom line is, BAR mappings are not like plain memory.

As I understand it the actual use of this in fact when user space
manages to map BAR memory into it's address space and attempts to do DMA
from it. So, I'm not sure I agree at all with this assement.

ie I gather with NVMe the desire is this could happen through the
filesystem with the right open/mmap flags.

So, saying this has nothing to do with core kernel code, or with mm,
is a really big leap.

> 2) Instead, I see appropriate that two sophisticated devices, like an
> IB NIC and a storage/accelerator device, can freely target each
> other

There is nothing special about IB, and no 'sophistication' of the
DMA'ing device is required.

All other DMA devices should be able to target BAR memory. eg TCP TSO,
or storage-to-storage copies from BAR to SCSI immediately come to
mind.

> for I/O, i.e. exchanging peer-to-peer PCIe transactions. And as long
> as the existing sophisticated initiators are confined to the RDMA
> subsystem, that is where this support belongs to.

I would not object to this stuff living in the PCI subsystem, but
living in rdma and having this narrrow focus that it should only
work with IB is not good.

> On a different note, this reminds me that the current patch set may be
> missing a way to disable the use of platform PCIe atomics when the
> target is the BAR of a peer device.

There is a general open question with all PCI peer to peer
transactions on how to negotiate all the relevant PCI
parameters. Supported vendor extensions and supported standardized
features seems like just one piece of a larger problem. Again well
outside the scope of IB.

Jason

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>