Re: [PATCH for-next 1/3] RDMA/erdma: Use dma_map_page to map scatter MTT buffer

[Leon Romanovsky <leon@kernel.org>]

On Mon, Jul 28, 2025 at 11:08:46AM +0800, Boshi Yu wrote:
> 
> 
> On 2025/7/27 19:27, Leon Romanovsky wrote:
> > On Fri, Jul 25, 2025 at 01:53:54PM +0800, Boshi Yu wrote:
> > > Each high-level indirect MTT entry is assumed to point to exactly one page
> > > of the low-level MTT buffer, but dma_map_sg may merge contiguous physical
> > > pages when mapping. To avoid extra overhead from splitting merged regions,
> > > use dma_map_page to map the scatter MTT buffer page by page.
> > > 
> > > Reviewed-by: Cheng Xu <chengyou@linux.alibaba.com>
> > > Signed-off-by: Boshi Yu <boshiyu@linux.alibaba.com>
> > > ---
> > >   drivers/infiniband/hw/erdma/erdma_verbs.c | 110 ++++++++++++++--------
> > >   drivers/infiniband/hw/erdma/erdma_verbs.h |   4 +-
> > >   2 files changed, 71 insertions(+), 43 deletions(-)
> > 
> > <...>
> > 
> > > +	pg_dma = vzalloc(npages * sizeof(dma_addr_t));
> > > +	if (!pg_dma)
> > > +		return 0;
> > > -	sg_init_table(sglist, npages);
> > > +	addr = buf;
> > >   	for (i = 0; i < npages; i++) {
> > > -		pg = vmalloc_to_page(buf);
> > > +		pg = vmalloc_to_page(addr);
> > 
> > <...>
> > > +
> > > +		pg_dma[i] = dma_map_page(&dev->pdev->dev, pg, 0, PAGE_SIZE,
> > > +					 DMA_TO_DEVICE);
> > 
> > Does it work?
> 
> Hi Leon,
> 
> I would like to confirm which part you think is not working properly. I
> guess that you might be concerned that if the buffer is not page-aligned, it
> could cause problems with dma_map_page.
> 
> In fact, when allocating the MTT buffer, we ensure that it is always
> page-aligned and that its length is a multiple of PAGE_SIZE. We have also
> tested the new code in our production environment, and it works well.
> 
> Look forward to your further reply if I have misunderstood your concerns.

DMA API expects Kmalloc addresses and not Vmalloc ones.

Thanks