Re: [PATCH for-next 2/2] RDMA/efa: Add CQ with external memory support

["Margolin, Michael" <mrgolin@amazon.com>]

On 7/4/2025 9:28 PM, Jason Gunthorpe wrote:
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
> On Tue, Jul 01, 2025 at 11:15:45PM +0000, Michael Margolin wrote:
>> +     if (umem) {
>> +             umem_sgl = ibcq->umem->sgt_append.sgt.sgl;
>> +             if (sg_dma_len(umem_sgl) < ib_umem_offset(umem) + cq->size) {
>> +                     ibdev_dbg(&dev->ibdev, "Non contiguous CQ unsupported\n");
>> +                     err = -EINVAL;
>> +                     goto err_free_mem;
>> +             }
> I'd rather see this call ib_umem_find_best_pgsz()
>
> Maybe like:
>    len = ib_umem_offset(umem) + cq->size
>    pgsz = roundup_pow_of_two(len);
>    ib_umem_find_best_pgoff(umem, pgsz, U64_MAX);
>    rdma_umem_for_each_dma_block(umem, biter, pgsz) {
>         dma = rdma_block_iter_dma_address(&biter) +  ib_umem_dma_offset(umem, pgsz);
>         break;
>    }
>
> It turns out the scatterlists can be irregular in some cases so this
> will handle that properly while the little test above cannot.
>
> And maybe the above thing could be a little helper:
>
> bool ib_umem_get_contiguous_dma(umem, &dma_addr)
>
> Also I am trying hard to keep scatterlist APIS out of the drivers.
>
> Jason

Some disadvantage of this approach is that if the provided memory 
is contiguous in its first "roundup_pow_of_two(len)" bytes but consists 
of additional smaller or unaligned sgl entries, we will refuse to use it.

I will try to compile some combination of the two approaches into a 
helper function.

Michael