Re: [RFC rdma-core 2/2] verbs: Introduce non-contiguous memory registration

[Jason Gunthorpe <jgg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>]

On Tue, Jan 30, 2018 at 01:35:21PM +0200, Marcel Apfelbaum wrote:
> On 29/01/2018 19:27, Jason Gunthorpe wrote:
> >On Sun, Jan 28, 2018 at 10:37:47PM +0200, Yuval Shaia wrote:
> >
> 
> Hi Jason,
> 
> >>But let's try to take it one step further, what if all my buffers are the
> >>same size, of even better, all are PAGE_SIZE. So in case of "composite"
> >>array of let's say 262144 elements i would have wasteful 262144 * 8 bytes.
> >>
> >>This problem could be solved with a bitmap to a given range where only the
> >>bits that are set composed the MR.
> >
> >You want this for the host on virtualization right?
> 
> Yes. (actually is more about us needing rather that wanting :) )
> 
> >Like we talked
> >about at plumbers?
> > > Is it really necessary to be so optimal? A list of SGLs is not good
> >enough?
> 
> It is not. We think the list would need to be limited to a single page,
> (system calls limitation? maybe we are wrong?)

The new ioctl interface isn't really limited.
This new API(s) will run over ioctl.

> By the way, doing that would only solve half of our problem.

Well, actually, only a 3rd :| The new MR would likely be 0 based, but
the VM guest doesn't know about this. So you'd need an API that can do
arbitrary based to really solve your probably. I guess all HW should
be able to do this so maybe it is OK?

> The other problem is what is happening on post-send. We don't have a
> virtually contiguous range to pass to post-send, and breaking the
> Work Request into several work requests using pages as boundaries
> will become again a problem if we want to send a big chunk (the HW
> has a rather limited max sg elements).  We can solve it by using 0
> based MRs, do you know if the current HW supports it?

I think some does.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html