Re: [rdma-core v5 0/8] Broadcom User Space RoCE Driver

[Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>]

On Mon, Apr 24, 2017 at 04:57:23PM -0400, Doug Ledford wrote:

> I'm not aware of anyway, which is always possible, I prefer to see
> mutexes instead of spin locks in user space because they are safer and
> more likely to result in forward progress in the event of
> contention.

It is true that if the callers of verbs adjust their thread priorities
we will certainly completely fail to make forward progess in the worst
case situations, as high priority waiters will no yield the CPU.

For this reason alone it is probably a really bad idea to use
spinlocks at all in library code.

However, AFAIK, default pthread mutexes well known to be much slower
than spinlocks for various cases. They have weird performance aspects
from the fairness policy on unlock we may not really want.

Perahps this is why Sean open-coded locks using atomics in rdmacm and
acm?

However, there are many kinds of mutex locks in glibc, and perhaps
something like PTHREAD_MUTEX_ADAPTIVE_NP gives performance close to a
spinlock?

Someone would have to benchmark different options and see.

Since we are so very performance sensitive, we might do well to
open-code our own locking primitive, similar to
PTHREAD_MUTEX_ADAPTIVE_NP, based on futexes. Inlining the
non-contended case alone would probably be notably helpful.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html