Re: [PATCH for-next 2/2] RDMA/efa: Add CQ with external memory support

[Jason Gunthorpe <jgg@nvidia.com>]

On Mon, Jul 07, 2025 at 08:07:43PM +0300, Margolin, Michael wrote:
> 
> On 7/4/2025 9:28 PM, Jason Gunthorpe wrote:
> > CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
> > 
> > 
> > 
> > On Tue, Jul 01, 2025 at 11:15:45PM +0000, Michael Margolin wrote:
> > > +     if (umem) {
> > > +             umem_sgl = ibcq->umem->sgt_append.sgt.sgl;
> > > +             if (sg_dma_len(umem_sgl) < ib_umem_offset(umem) + cq->size) {
> > > +                     ibdev_dbg(&dev->ibdev, "Non contiguous CQ unsupported\n");
> > > +                     err = -EINVAL;
> > > +                     goto err_free_mem;
> > > +             }
> > I'd rather see this call ib_umem_find_best_pgsz()
> > 
> > Maybe like:
> >    len = ib_umem_offset(umem) + cq->size
> >    pgsz = roundup_pow_of_two(len);
> >    ib_umem_find_best_pgoff(umem, pgsz, U64_MAX);
> >    rdma_umem_for_each_dma_block(umem, biter, pgsz) {
> >         dma = rdma_block_iter_dma_address(&biter) +  ib_umem_dma_offset(umem, pgsz);
> >         break;
> >    }
> > 
> > It turns out the scatterlists can be irregular in some cases so this
> > will handle that properly while the little test above cannot.
> > 
> > And maybe the above thing could be a little helper:
> > 
> > bool ib_umem_get_contiguous_dma(umem, &dma_addr)
> > 
> > Also I am trying hard to keep scatterlist APIS out of the drivers.
> > 
> > Jason
> 
> Some disadvantage of this approach is that if the provided memory
> is contiguous in its first "roundup_pow_of_two(len)" bytes but consists of
> additional smaller or unaligned sgl entries, we will refuse to use
> it.

I intended "len" to be large enough that there was no additional
portion that is not covered.

Jason