Re: [RFC PATCH 1/5] IB/core: Add Core Capability flags to ib_device

Dave Goodell (dgoodell)
2015-05-12T17:41:14Z

On May 11, 2015, at 3:39 PM, Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org> wrote:

> So, I look at this:
>
> int usnic_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
> struct ib_send_wr **bad_wr)
> {
> usnic_dbg("\n");
> return -EINVAL;
> }
>
> And conclude that usnic doesn't actually provide any RDMA services to
> in-kernel users, correct?

Correct.

> I'd then guess the entire point of this is to hack some non-RDMA
> device to export it's interface via uverbs?

If by "non-RDMA", you mean "does not currently support hardware offload of direct data placement", then yes. There are a couple of things to keep in mind:

1) usNIC is UD-only at this time. Are there any kernel clients that make use of UD without also requiring RC? (this is a genuine question)

2) When using UDP, usNIC deposits 42-byte UDP/IP/Ethernet headers into the start of the recv buffer, not a 40-byte GRH. Unfortunately, the current verbs stack has no way to indicate the device's/port's/QP's header size, let alone format, to any client. So clients must be aware they are using a usNIC device under the hood.

> Presumably because someone
> nack'd using your own char device or whatever?

I wasn't participating on the kernel side of the usNIC project at the time that some of the early decisions were made, but to my knowledge the uverbs stack was selected as the most appropriate way to expose this technology to users after evaluation of several approaches. I don't think that any other approach was previously submitted and NACK-ed.

-Dave