Re: [PATCH rdma-core 0/7] Add mlx5 direct verbs

[Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>]

On Mon, Feb 06, 2017 at 01:46:09PM +0200, Leon Romanovsky wrote:
> On Mon, Feb 06, 2017 at 10:56:56AM +0000, Amrani, Ram wrote:
> > > This patchset from Leon adds direct access to mlx5 devices.
> > >
> > > The libibverbs API is an abstract API. It is agnostic to any underlying
> > > provider specific implementation. While this abstraction has the advantage of
> > > user applications portability it has a performance penalty. For some
> > > applications optimizing performance is more important than portability.
> > >
> > > The mlx5 direct verbs API introduced in this patchset is intended for such
> > > applications. It exposes mlx5 specific low level data path
> > > (send/receive/completion) operations, allowing the application to bypass the
> > > libibverbs data path API.
> > >
> > > The proposed interface consists from small number of hardware specific headers
> > > with relevant inline functions and conversion logic from ibverbs structures to
> > > mlx5 related structures.
> > >

> > While I'm always for better performance, I'm not sure driver specific APIs is the way.
> > This will hurt portability for users.
> 
> As it was expressed in cover letter, this feature is intended for
> users who want performance and understand the disadvantages of such
> direct approach.

To expound on that - the trade off we discussed in-person was we
either do something like this or continually take endless
vendor-specific patches to add 'common' core code for unstandardized
chip-specific features.

This seemed like the least bad trade off, if another vendor wants the
same things then they could work together and make an actual
multi-chip common API. I was expecting this interface to be used in
only a few places, like the next level of middleware libraries
(libfabric, mxm, etc) not for many end-users.

I'm still not completely sure what this API is intended to do, if it is
really just performance then I am less supportive and would rather see
an approach like libfabric where a user can build a single provider verbs
with the same source API, but optimized....

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html