Re: [PATCH] RDMA/ucma: Put a lock around every call to the rdma_cm layer

[Jason Gunthorpe <jgg@mellanox.com>]

On Tue, Feb 18, 2020 at 10:07:01PM -0800, Eric Biggers wrote:
> > these 11 lets include them as  well. I wasn't able to find a way to
> > search for things, this list is from your past email, thanks.
> > 
> 
> Unfortunately I haven't had time to work on syzkaller bugs lately, so I can't
> provide an updated list until I go through the long backlog of bugs.

Ok

> A comment on the patch below:
> 
> > @@ -1112,13 +1134,17 @@ static ssize_t ucma_accept(struct ucma_file *file, const char __user *inbuf,
> >  	if (cmd.conn_param.valid) {
> >  		ucma_copy_conn_param(ctx->cm_id, &conn_param, &cmd.conn_param);
> >  		mutex_lock(&file->mut);
> > +		mutex_lock(&ctx->mutex);
> >  		ret = __rdma_accept(ctx->cm_id, &conn_param, NULL);
> > +		mutex_unlock(&ctx->mutex);
> >  		if (!ret)
> >  			ctx->uid = cmd.uid;
> >  		mutex_unlock(&file->mut);
> 
> This is nesting the new ucma_context::mutex inside the existing ucma_file::mut.

Ah, indeed
 
> > @@ -1403,6 +1443,7 @@ static ssize_t ucma_process_join(struct ucma_file *file,
> >  	if (IS_ERR(ctx))
> >  		return PTR_ERR(ctx);
> >  
> > +	mutex_lock(&ctx->mutex);
> >  	mutex_lock(&file->mut);
> >  	mc = ucma_alloc_multicast(ctx);
> >  	if (!mc) {
> 
> ... but this is doing the opposite.  So it can deadlock.

Lets narrow this one to just rdma_join_multicast(), looks safe

> What's the intended order?

The code works better if mut is the exterior lock, it seems
 
> Also, are these two separate mutexes actually needed?  I.e., did you consider
> using the existing ucma_file::mut, but it didn't work or it wasn't fine-grained
> enough?  (It looks like one ucma_file can have multiple ucma_contexts.)

It would probably work, but it is not as fine grained as a per-ctx
lock, and some people do care about performance with this stuff.

The file->mut is protecting some global per-fd lists it seems.

Thanks,
Jason