Re: [PATCH rdma-next v4 11/16] RDMA/mlx4: Use ib_umem_get_cq_buf() for user CQ buffer

[Jiri Pirko <jiri@resnulli.us>]

Tue, May 12, 2026 at 08:29:27PM CEST, jgg@ziepe.ca wrote:
>On Thu, May 07, 2026 at 02:52:26PM +0200, Jiri Pirko wrote:
>> +	cq->umem = ib_umem_get_cq_buf(&dev->ib_dev, udata, entries * cqe_size,
>> +				      IB_ACCESS_LOCAL_WRITE);
>> +	if (IS_ERR(cq->umem)) {
>> +		err = PTR_ERR(cq->umem);
>>  		goto err_cq;
>>  	}
>> +	if (cq->umem) {
>> +		if (dev->dev->caps.flags2 & MLX4_DEV_CAP_FLAG2_SW_CQ_INIT) {
>> +			err = -EOPNOTSUPP;
>> +			goto err_umem;
>
>Huh. this is getting pretty hacky.. The driver wants to memset the
>user buf to 0xcc for some reason, and it already has a nice flow that
>if that fails it tells the FW it fails and presumably is Ok.
>
>The issue is it passes buf_addr around insead of having made an
>ib_umem_memset() (which can reject dmabuf).
>
>Looks easy enough, change sg_zero_buffer() to sg_fill_buffer() to
>accept the 0xcc, ib_umem_memset() trivially calls it, remove the
>buf_addr from the call chain, directly use the umem in the
>mlx4_init_user_cqes(), remove the if above, use the
>ib_umem_get_cq_buf_or_va() in the driver..
>
>Leaving it like this just means the driver won't work with the new
>uAPI with normal VA which is not desirable..

Agreed. I would like to fix this in a follow-up patchset which would
look more or less like this (Claude generated):

 1) lib/scatterlist: add sg_fill_buffer()
    Generalize sg_zero_buffer() to take a fill byte. Keep
    sg_zero_buffer() as a thin static inline wrapper around
    sg_fill_buffer(..., 0) so existing callers (nvmet, scsi_debug,
    ccree, jh7110-aes, krb5) don't have to change.
 2) RDMA/umem: add ib_umem_memset()
    Walks the umem's sg list via sg_fill_buffer(). Rejects dmabuf and
    ODP umems with -EOPNOTSUPP. Honors umem offset/length bounds.
 3) net/mlx4: drop buf_addr/user_cq from mlx4_cq_alloc()
    Replace "void *buf_addr, bool user_cq" with
    "struct mlx4_buf *kbuf, bool sw_cq_init". The function only owns
    the kernel-side init via mlx4_init_kernel_cqes(); user-side init
    becomes the caller's responsibility. mlx4_init_user_cqes() goes
    away. mlx4_en and the kernel mlx4_ib path are updated to the new
    signature.
 4) RDMA/mlx4: switch to ib_umem_get_cq_buf_or_va() and ib_umem_memset()
    The user-CQ create path collapses to a single
    ib_umem_get_cq_buf_or_va() followed by an
    ib_umem_memset(cq->umem, 0xcc, ...) before mlx4_cq_alloc(). If the
    memset succeeds, tell FW sw_cq_init=true; otherwise fall back to
    FW-side init. dmabuf / ODP umems fall back naturally via
    ib_umem_memset() returning -EOPNOTSUPP, and the explicit
    MLX4_DEV_CAP_FLAG2_SW_CQ_INIT -EOPNOTSUPP branch goes away.

Makes sense?