Re: [PATCH v2 00/14] Introduce AMD Pensando RDMA driver

[Jason Gunthorpe <jgg@nvidia.com>]

On Thu, May 22, 2025 at 04:58:19PM +0530, Abhijit Gangurde wrote:
> > @@ -1231,6 +1257,7 @@ void ionic_destroy_rdma_admin(struct ionic_ibdev *dev)
> >   	}
> >   	if (dev->eq_vec) {
> > +		// Locking? Add a lockdep assertion if caller is holding the lock
> >   		while (dev->lif_cfg.eq_count > 0) {
> >   			eq = dev->eq_vec[--dev->lif_cfg.eq_count];
> >   			ionic_destroy_eq(eq);
> I don't think there is a need for the lock here because the device is
> unregistered and the queues are all stopped.

Add a comment then

> > @@ -887,6 +893,8 @@ static struct ib_mr *ionic_get_dma_mr(struct ib_pd *ibpd, int access)
> >   	if (!mr)
> >   		return ERR_PTR(-ENOMEM);
> > +	// This seems strange, shouldn't this do something? If you don't support an all address MR then don't define this op.
> > +
> >   	return &mr->ibmr;
> >   }
> From hardware lkey zero is reserved as a local dma lkey for all address MR. 
> I would make it more explicit as mr.ibmr.lkey = IONIC_DMA_LKEY (same for
> RKEY) with that defined to be zero.

Yeah, that's a lot clearer

> > @@ -1454,11 +1466,15 @@ static int ionic_destroy_cq(struct ib_cq *ibcq, struct ib_udata *udata)
> >   static bool pd_local_privileged(struct ib_pd *pd)
> >   {
> > +	/* That isn't how it works, only the lkey get_dma_mr() returns is
> > +	special and must be used on any WRs that require it. WRs refering to any
> > +	other lkeys must behave normally. */
> >   	return !pd->uobject;
> >   }
> >   static bool pd_remote_privileged(struct ib_pd *pd)
> >   {
> > +	/* Same comment, except about rkeys now. */
> >   	return pd->flags & IB_PD_UNSAFE_GLOBAL_RKEY;
> >   }
> This is how we allow the qp to use the dma lkey.  If the qp is a kernel
> space qp (its pd has no uobject) then we allow use of the dma lkey by that
> qp.  We do not allow use of dma lkey by user qps.  If the pd flags has the
> unsafe rkey flag, then we also allow the qp use it for remote access.

OK, so this is just about permissions because you don't attach the 0
l/rkey to a PD? Add a comment in the ionic_get_dma_mr() that the QP
needs special flags to be allowed to use the 0 l/rkey.

Jason