Re: [PATCH rdma-next] RDMA/mlx5: Add a umr recovery flow

[Leon Romanovsky <leon@kernel.org>]

On Thu, May 26, 2022 at 11:32:12AM -0300, Jason Gunthorpe wrote:
> On Sun, May 15, 2022 at 07:19:53AM +0300, Leon Romanovsky wrote:
> > @@ -270,17 +296,49 @@ static int mlx5r_umr_post_send_wait(struct mlx5_ib_dev *dev, u32 mkey,
> >  	mlx5r_umr_init_context(&umr_context);
> >  
> >  	down(&umrc->sem);
> > +	while (true) {
> > +		mutex_lock(&umrc->lock);
> 
> You need to test this with lockdep, nesing a mutex under a semaphor is
> not allowed, AFAIK.

We are running with lockdep all our tests.

> 
> > +		err = mlx5r_umr_post_send(umrc->qp, mkey, &umr_context.cqe, wqe,
> > +					  with_data);
> > +		mutex_unlock(&umrc->lock);
> > +		if (err) {
> > +			mlx5_ib_warn(dev, "UMR post send failed, err %d\n",
> > +				     err);
> > +			break;
> >  		}
> > +
> > +		wait_for_completion(&umr_context.done);
> 
> Nor is sleeping under a semaphore.

Not according to the kernel/locking/semaphore.c. Semaphores can sleep
and the code protected by semaphores can sleep too.

   53 void down(struct semaphore *sem)
   54 {
   55         unsigned long flags;
   56
   57         might_sleep();
   ....
   64 }
   65 EXPORT_SYMBOL(down);

> 
> And, I'm pretty sure, this entire function is called under a spinlock
> in some cases.

Can you point to such flow?

Thanks

> 
> Jason