Re: [PATCH][RFC][0/4] InfiniBand userspace verbs implementation

Andrew Morton
2005-04-25T22:34:06Z

Timur Tabi <timur.tabi@ammasso.com> wrote:
>
> Andrew Morton wrote:
>
> > The way we expect get_user_pages() to be used is that the kernel will use
> > get_user_pages() once per application I/O request.
> >
> > Are you saying that RDMA clients will semi-permanently own pages which were
> > pinned by get_user_pages()? That those pages will be used for multiple
> > separate I/O operations?
>
> Yes, absolutely!
>
> The memory buffer is allocated by the process (usually just via malloc) and
> registed/pinned by the driver. It then stays pinned for the life of the process (typically).

ug. What stops the memory from leaking if the process exits?

I hope this is a privileged operation?

> > If so, then that's a significant design departure and it would be good to
> > hear why it is necessary.
>
> That's just how RMDA works. Once the memory is pinned, if the app wants to send data to
> another node, it does two things:
>
> 1) Puts the data into its buffer
> 2) Sends a "work request" to the driver with (among other things) the offset and length of
> the data.
>
> This is a time-critical operation. It must occurs as fast as possible, which means the
> memory must have already been pinned.

It would be better to obtain this memory via a mmap() of some special
device node, so we can perform appropriate permission checking and clean
everything up on unclean application exit.