Re: [PATCH][RFC][0/4] InfiniBand userspace verbs implementation

Timur Tabi
2005-04-25T22:22:53Z

Andrew Morton wrote:

> The way we expect get_user_pages() to be used is that the kernel will use
> get_user_pages() once per application I/O request.
>
> Are you saying that RDMA clients will semi-permanently own pages which were
> pinned by get_user_pages()? That those pages will be used for multiple
> separate I/O operations?

Yes, absolutely!

The memory buffer is allocated by the process (usually just via malloc) and
registed/pinned by the driver. It then stays pinned for the life of the process (typically).

> If so, then that's a significant design departure and it would be good to
> hear why it is necessary.

That's just how RMDA works. Once the memory is pinned, if the app wants to send data to
another node, it does two things:

1) Puts the data into its buffer
2) Sends a "work request" to the driver with (among other things) the offset and length of
the data.

This is a time-critical operation. It must occurs as fast as possible, which means the
memory must have already been pinned.

--
Timur Tabi
Staff Software Engineer
timur.tabi@ammasso.com

One thing a Southern boy will never say is,
"I don't think duct tape will fix it."
-- Ed Smylie, NASA engineer for Apollo 13