[PATCH 17/17] nvme-rdma: Add T10-PI support

[sagi@grimberg.me (Sagi Grimberg)]

>> Hmm, in my mind, overloading the interface to be generic was counter
>> productive specifically for PI. I know that there are bunch of
>> operations to do on an MR, but I think that each of them is better off
>> with its own clean and simple API that gives the user the minimum stuff
>> it needs to understand or perform in order to get the task done.
> 
> You can say the opposite too... When you see different use cases converge
> into one elegant API then you know you've done something right.
> I think this is one of those cases. DIF and other signatures, simple layouts,
> indirections, complex n-dim matrix layouts are all attributes assigned to
> memory, and apply when this memory is accessed.

I'm not opposed to have a common API, but I would also think that we
should layer feature specific API on top of it such that the actual
consumer will only need to care about what it needs to.

>> Note that my comments are mostly directed to the MR hierarchy that PI
>> is presenting to the consumer. I think we are much better off with
>> hiding it down in the providers such that the ULP will see yet another
>> type of a memory region that can do PI.
> 
> We'll look into that. Basically I don't mind the API of
> set_mr_layout_signature()
> to take (data_sgl, dif_sgl) instead of (data_sge, dif_sge). Just need to make
> sure we can hide in provider the pre-allocation of all the "hidden" mkeys for
> that case. We don't want ulp to experience hickups due to internal key
> allocations and management...

100% agree. All allocations must not be made in the data path.