Re: [openib-general] Re: [PATCH][RFC][0/4] InfiniBand userspace verbs implementation

Timur Tabi
2005-04-21T18:41:06Z

Andy Isaacson wrote:

> If you take the hardline position that "the app is the only thing that
> matters", your code is unlikely to get merged. Linux is a
> general-purpose OS.

The problem is that our driver and library implement an API that we don't fully control.
The API states that the application allocates the memory and tells the library to register
it. The app then goes on its merry way until it's done, at which point it tells the
library to deregister the memory. Neither the app nor the API has any provision for the
app to be notified that the memory is no longer pinned and therefore can't be trusted.
That would be considered a critical failure from the app's perspective, so the kernel
would be doing it a favor by killing the process.

> You might want to consider what happens with your communication system
> in a machine running power-saving modes (in the limit, suspend-to-disk).
> Of course most machines with Infiniband adapters aren't running swsusp,
> but it's not inconceivable that blade servers might sleep to lower power
> and cooling costs.

Any application that registers memory, will in all likelihood be running at 100% CPU
non-stop. The computer is not going to be doing anything else but whatever that app is
trying to do. The application could conceiveable register gigabytes of RAM, and if even a
single page becomes unpinned, the whole thing is worthless. The application cannot do
anything meaningful if it gets a message saying that some of the memory has become
unpinned and should not be used.

So the real question is: how important is it to the kernel developers that Linux support
these kinds of enterprise-class applications?

--
Timur Tabi
Staff Software Engineer
timur.tabi@ammasso.com

One thing a Southern boy will never say is,
"I don't think duct tape will fix it."
-- Ed Smylie, NASA engineer for Apollo 13