Re: [PATCH for-next v2 1/4] RDMA/erdma: Support non-contiguous kernel QP buffers

[Leon Romanovsky <leon@kernel.org>]

On Wed, Sep 09, 2026 at 02:21:37PM +0800, Cheng Xu wrote:
> 
> 
> On 9/3/26 8:38 PM, Cheng Xu wrote:
> > 
> > 
> > On 9/3/26 5:10 PM, Leon Romanovsky wrote:
> >> On Thu, Aug 27, 2026 at 04:25:20PM +0800, Cheng Xu wrote:
> >>> A single coherent allocation for kernel QP queues can fail for large
> >>> queues when memory is fragmented.
> >>>
> >>> Allocate page-sized coherent buffers and describe them with the existing
> >>> MTT. Keep the userspace QP path unchanged.
> >>>
> >>> Signed-off-by: Cheng Xu <chengyou@linux.alibaba.com>
> >>> ---
> >>>  drivers/infiniband/hw/erdma/erdma_cq.c    |   4 +-
> >>>  drivers/infiniband/hw/erdma/erdma_qp.c    |  38 +++--
> >>>  drivers/infiniband/hw/erdma/erdma_verbs.c | 195 +++++++++++++---------
> >>>  drivers/infiniband/hw/erdma/erdma_verbs.h |  44 ++++-
> >>>  4 files changed, 179 insertions(+), 102 deletions(-)
> >>
> >> <...>
> >>
> 
> <...>
> 
> >>
> >>> +struct erdma_buf_list {
> >>> +	void *buf;
> >>> +	dma_addr_t dma_addr;
> >>> +};
> >>
> >> This struct is very similar to scatter-gather list, why don't you use it
> >> directly?
> > 
> > Good idea. I will use struct scatterlist in the next revision.
> 
> Hi Leon,
> 
> I switched to scatterlist in v3, but Sashiko pointed out an issue with
> using sg_set_buf() and sg_virt() on dma_alloc_coherent() memory [1].
> 
> To handle this correctly, the driver would still need to retain the
> original CPU addresses returned by dma_alloc_coherent(). Using scatterlist
> does not simplify this implementation: we still need separate storage for
> the CPU addresses, and the only scatterlist field we actually need is the
> DMA address. A small structure holding both addresses would therefore be
> simpler.

Sashiko thinks that you are creating SG list to feed it to dma_map_sg()
later which is not. You are using SG as simple database and you will get
iterators for free.

cpu_address = sg_page()
dma_address = sg_dma_address()

Thanks

> 
> I would like to switch back to a small structure holding the CPU and DMA
> addresses in v4. This would keep the implementation simpler while
> preserving the original addresses returned by dma_alloc_coherent().
> 
> [1] https://sashiko.dev/#/patchset/20260908033709.89898-1-chengyou%40linux.alibaba.com
> 
> Thanks,
> Cheng Xu
> 
> 
> > Thanks,
> > Cheng Xu
> > 
> >>
> >> Thanks