Re: [PATCH v2 10/10] nvmet: Optionally use PCI P2P memory

Logan Gunthorpe
2018-03-01T19:27:03Z

On 01/03/18 11:42 AM, Jason Gunthorpe wrote:
> On Thu, Mar 01, 2018 at 08:35:55PM +0200, Sagi Grimberg wrote:
> This is also why I don't entirely understand why this series has a
> generic allocator for p2p mem, it makes little sense to me.

> Why wouldn't the nmve driver just claim the entire CMB of its local
> device for its own use? Why involve p2p core code in this?

We'd prefer to have a generic way to get p2pmem instead of restricting
ourselves to only using CMBs. We did work in the past where the P2P
memory was part of an IB adapter and not the NVMe card. So this won't
work if it's an NVMe only interface.

As Stephen mentioned, we also use a couple devices that only exposes P2P
memory and this isn't related to NVMe at all. So there's value in having
a generic interface and allocator to enable all devices to provide this
memory.

If there were a hypothetical situation where a driver wants to use some
of the memory for P2P and some of it for other purposes then they'd just
divide it themselves and only pass a subset to
pci_p2pdma_add_resource(). However, as per our changes to the nvme-pci
code, it's really just easier to use the allocator for everything inside
the driver.

Logan