Re: [RFC] RDMA with Continuous Memory Allocator

[Leon Romanovsky <leon@kernel.org>]

On Fri, Jul 09, 2021 at 07:24:48PM +0900, Shunsuke Mie wrote:
> Hi all,
> 
> I tried to use Continuous Memory Allocator (CMA) allocated memory for
> RDMA transfer buffer in userspace, but it failed.

Sorry for my question, but why do you need it?
From that I remember, CMA memory is used for the devices that doesn't
support scatter-gather, while RDMA devices (umem) need SG.

Thanks

> 
> For more details, an ibv_reg_mr() API fails when I pass an mmaped
> memory region of allocated memory by the CMA. The reason why, a CMA
> mmap function, __dma_mmap_from_coherent(), sets  VM_IO and VM_PFNMAP
> to vma->vm_flags, and an ib_uverbs_reg_mr(), kernel function
> corresponding to the ibv_reg_mr(), tries to pin the memory region but,
> it becomes to fail. because VM_IO or VM_PFNMAP regions cannot be
> pined. As a result, the ibv_reg_mr() returns an error.
> 
> I think the ib_umem_get() that is called in the ib_uverbs_reg_mr() and
> pins memories needs some modifications to support RDMA transfer
> to/from the CMA memory.
> 
> I’d like to know your comments, ideas, and other solution.
> 
> Thanks a lot,
> Shunsuke.