Re: [RFC] libibverbs IB Device Memory support

[Christoph Hellwig <hch-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>]

On Mon, Jun 05, 2017 at 11:44:00AM -0500, Christoph Lameter wrote:
> On Wed, 10 May 2017, ahmad omary wrote:
> 
> > We have considered using mmap(), but As the size of device memory may be limited
> > ,the way to access it from host cpu may differ from vendor to vendor, due to
> > the 4K (page) aligment limitation of mmap() and the need not to directly
> > allow user to access the device memory, there is a need for a wrapper access
> > methods API that allows allocating and managing chunks that are smaller than
> > 4KB and not necessarily aligned to 4KB (page size).
> 
> Why are 4k sized chunks a problem given that there are megabytes of memory
> in these devices? We are using various adapters already with an mmapped
> solution here.
> 
> And I would prefer direct user space access to the memory. Fast access to
> the data stored in the NIC is important and it would be best not to have
> an intermediate layer that requires memcpy.

Agreed.  The current design looks incredibly stupid.  Also makes sure
it works with the cxgb4 version of this feature that has been posted
as part of the NVMe P2P patches.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html