Re: [RFC rdma-core 2/2] verbs: Introduce non-contiguous memory registration

[Jason Gunthorpe <jgg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>]

On Wed, Jan 31, 2018 at 02:27:01PM +0200, Marcel Apfelbaum wrote:

> It is good to know, but still, passing so much information to kernel
> when we can rather "compress" it, maybe it worth a second thought.

Not sure. Have to see the whole thing..

> > Well, actually, only a 3rd :| The new MR would likely be 0 based, but
> > the VM guest doesn't know about this. So you'd need an API that can do
> > arbitrary based to really solve your probably. I guess all HW should
> > be able to do this so maybe it is OK?
> 
> The way we solve "the other" half is by intercepting the post-send
> requests in hypervisor. At hypervisor level we don't have contiguous virtual
> addresses anymore, but we don't need them for 0 based MRs:
> The guest still register regular MRs, while the hypervisor will
> register a 0 based MR save the guest virtual address of the MR.
> At post-send we simply substract the saved MR base address from the work request
> buffers and we are back to 0 based MR.

That only works for lkeys, the rkey expoeses the base address to the
remote - the HV can't fix it..

> Do you have a model in mind? We would really want to try it out.
> 
> By the way, I tried to search in the kernel for vendors implementing
> it and I saw maybe one vendor... so maybe 0 based MR is a nice idea
> but nothing more.

Try the mlx drivers, I think at least one of them can do it today.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html