Re: [RFC] libibverbs IB device hotplug support

[Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>]

On Tue, Feb 28, 2017 at 04:47:05PM -0500, Doug Ledford wrote:

> > I don't think that we should introduce an asych context into libibverbs.
> 
> Why not?

Generally, I dislike the idea of running threads from libraries,
particularly libraries like ibverbs. So many apps get no benefit from
the thread, but it sits there connected to udev..

But aside from that, threads create races, and in this case if the
hidden thread has to run to get the up to date device list, then the
application has to synchronize with it to get the up to date
information.

This is practically problematic because one of the areas we need to
fix is rdma-cm which is already getting a kind of new-device
notification from kernel messages that are not synchronized with udev
or any other thread.

In this case I think the simple solution is the most correct one - run
readdir on every call to ibv_get_devices and do set-interesection with
the internal list before returning the result. Easy, simple locking,
no races to worry about.

We probably also need to add some retries for opening /dev/uverbsXX
based on sysfs to really solve all the races...

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html