Re: [PATCH rdma-next v9 4/5] RDMA/bnxt_re: Direct Verbs: Support DBR verbs

[Jason Gunthorpe <jgg@ziepe.ca>]

On Tue, Jan 27, 2026 at 04:07:41PM +0100, Jiri Pirko wrote:
> Tue, Jan 27, 2026 at 03:15:45PM +0100, jgg@ziepe.ca wrote:
> >On Tue, Jan 27, 2026 at 01:30:03PM +0100, Jiri Pirko wrote:
> >> Tue, Jan 27, 2026 at 11:31:08AM +0100, sriharsha.basavapatna@broadcom.com wrote:
> >> >From: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
> >> >
> >> >The following Direct Verbs (DV) methods have been implemented in
> >> >this patch.
> >> >
> >> >Doorbell Region Direct Verbs:
> >> >-----------------------------
> >> >- BNXT_RE_METHOD_DBR_ALLOC:
> >> >  This will allow the appliation to create extra doorbell regions
> >> >  and use the associated doorbell page index in DV_CREATE_QP and
> >> >  use the associated DB address while ringing the doorbell.
> >> >
> >> >- BNXT_RE_METHOD_DBR_FREE:
> >> >  Free the allocated doorbell region.
> >> >
> >> >- BNXT_RE_METHOD_GET_DEFAULT_DBR:
> >> >  Return the default doorbell page index and doorbell page address
> >> >  associated with the ucontext.
> >> >
> >> 
> >> Similar to CQ/QP, why this is bnxt specific? I know a little about rdma,
> >> but I believe we use it in mlx5 too, no?
> >
> >mlx5 has a specific thing too, the doorbell has enough fairly hw
> >specific properties and never leaks outside the userspace provider.
> >
> >We consolidated the internal code to manage the mmaps, beyond that
> >there hasn't been a big push to consolidate more.
> 
> I'm a bit lost about what this patchset tries to do. The cover letter
> does not mention dmabuf at all. Not sure why. I understand that create
> cq/qp is enabled to work with user-passed dma-buf info. So that makes me
> assume the same for DBR. I guess I'm wrong.

This series doesn't really clearly explain what it is actually for,
but it is almost certianly about supporting what NCCL calls "GPU
Initiated Networking (GIN)".

To do this you need a couple of components:
 1) "DV" verbs to allow direct access to the underlying HW queues
    under a QP/CQ. This is because you will write a "RDMA provider"
    that runs on the GPU
 2) DMABUF support for QP/CQ because you will use DMABUF to place
    the QP/CQ inside GPU VRAM so that the "RDMA provider" running in
    the GPU can access the rings at full performance
 3) A doorebell ring that is compatible with the GPU, usually meaning
    dedicated doorbell registers because the GPU can't do locking
    coordinated with the CPU.

Of course there are other ways to use these APIs and DV was first
invented for DPDK not NCCL..

Jason