rdma-core out-of-tree user library

rdma-core out-of-tree user library

[Amrani, Ram]

Hi Jason, Leon, Doug, All,
A provider's out-of-tree user library cannot be built against rdma-core, because the
API is hidden to out-of-tree libraries. This was due to a decision taken months back.
The distros have adopted this approach as-is and it is going to be part of the next
releases. While this approach ensures all of the components are in-sync, it has its
disadvantages.

There is typically a gap of several months between two inbox releases.
New features and fixes are delivered to the next inbox releases, but we would like to
have the ability to supply an outbox user library to customers in between. It may
also happen that customers can only adopt the new inbox version with a delay, or even
worse, if they cannot adopt it at all due to various reasons.

I guess this is something you have thought of. If so, what were/are your thoughts about
creating intermediate out-of-box versions?

How are other providers handling this?

Thanks,
Ram

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: rdma-core out-of-tree user library

[Jason Gunthorpe]

On Tue, Jun 06, 2017 at 12:49:21PM +0000, Amrani, Ram wrote:
> Hi Jason, Leon, Doug, All,

> A provider's out-of-tree user library cannot be built against
> rdma-core, because the API is hidden to out-of-tree libraries. This
> was due to a decision taken months back.  The distros have adopted
> this approach as-is and it is going to be part of the next
> releases. While this approach ensures all of the components are
> in-sync, it has its disadvantages.
> 
> There is typically a gap of several months between two inbox
> releases.  New features and fixes are delivered to the next inbox
> releases, but we would like to have the ability to supply an outbox
> user library to customers in between. It may also happen that
> customers can only adopt the new inbox version with a delay, or even
> worse, if they cannot adopt it at all due to various reasons.

Two options,

1) The 'ofed' approach: just use the latest rdma-core. We want
   rdma-core to run on any kernel version and any distro release, so
   this should not be a major problem
2) For each targetted distro release back port the changes you want
   into the exact distro rdma-core source and distribute your driver
   binary from that build. This keeps everything very much the same
   but upgrades that driver.

> How are other providers handling this?

Most things I've seen have followed an OFED like approach because they
also want to ship an updated kernel component. Historically we have
not seen that much churn in the user space provider that is not linked
to kernel changes.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: rdma-core out-of-tree user library

[Amrani, Ram]

> Two options,
> 
> 1) The 'ofed' approach: just use the latest rdma-core. We want
>    rdma-core to run on any kernel version and any distro release, so
>    this should not be a major problem

My major concern was that when it comes to distros we don't have their
exact code base and/or can't release it. They take rdma-core X and
apply patches on it so it becomes rdma-core X.Y.
Hence if I take rdma-core X and apply my changes I will be actually
missing several patches.
Assuming backward-and-forward support, I might as well take rdma-core X+1
and base upon it my newer patches.

> 2) For each targetted distro release back port the changes you want
>    into the exact distro rdma-core source and distribute your driver
>    binary from that build. This keeps everything very much the same
>    but upgrades that driver.
> > How are other providers handling this?

As I've indicated above we don't have their exact code base and/or can't release it.

> Most things I've seen have followed an OFED like approach because they
> also want to ship an updated kernel component. Historically we have
> not seen that much churn in the user space provider that is not linked
> to kernel changes.
> 
> Jason

Thanks,
Ram

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: rdma-core out-of-tree user library

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 1717 bytes --]

On Wed, Jun 07, 2017 at 07:30:43AM +0000, Amrani, Ram wrote:
> > Two options,
> >
> > 1) The 'ofed' approach: just use the latest rdma-core. We want
> >    rdma-core to run on any kernel version and any distro release, so
> >    this should not be a major problem
>
> My major concern was that when it comes to distros we don't have their
> exact code base and/or can't release it. They take rdma-core X and
> apply patches on it so it becomes rdma-core X.Y.
> Hence if I take rdma-core X and apply my changes I will be actually
> missing several patches.
> Assuming backward-and-forward support, I might as well take rdma-core X+1
> and base upon it my newer patches.

You should leverage your partnership program with various distributions.
It will give you the actual code base, on which they are basing their
packages.

>
> > 2) For each targetted distro release back port the changes you want
> >    into the exact distro rdma-core source and distribute your driver
> >    binary from that build. This keeps everything very much the same
> >    but upgrades that driver.
> > > How are other providers handling this?
>
> As I've indicated above we don't have their exact code base and/or can't release it.
>
> > Most things I've seen have followed an OFED like approach because they
> > also want to ship an updated kernel component. Historically we have
> > not seen that much churn in the user space provider that is not linked
> > to kernel changes.
> >
> > Jason
>
> Thanks,
> Ram
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: rdma-core out-of-tree user library

[Jason Gunthorpe]

On Wed, Jun 07, 2017 at 07:30:43AM +0000, Amrani, Ram wrote:
> > Two options,
> > 
> > 1) The 'ofed' approach: just use the latest rdma-core. We want
> >    rdma-core to run on any kernel version and any distro release, so
> >    this should not be a major problem
> 
> My major concern was that when it comes to distros we don't have their
> exact code base and/or can't release it. They take rdma-core X and
> apply patches on it so it becomes rdma-core X.Y.

All distros I know of provide full source - GPL requires it for most
packages, and you would have a right to redistibute it, of
course.. This is no different than the kernel.

> Assuming backward-and-forward support, I might as well take rdma-core X+1
> and base upon it my newer patches.

Right, that is what I ment when I said 'latest rdma-core'

The very best thing you can do for your cusomters is to provide your
changes in a way that the distro itself would be willing to backport
and release as a minor update to their users. eg provide clear commit
messages, a fixes tag, content that is appropriate for stable
backporting and inform the distro via your corporate relationship to
take the patch. This is what Leon is talking about.

This is no different than the process for the kernel.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: rdma-core out-of-tree user library

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 456 bytes --]

On Tue, Jun 06, 2017 at 12:49:21PM +0000, Amrani, Ram wrote:
> How are other providers handling this?

We are opening bugzilla tickets in RH system (other distro similar) to
update relevant libraries.

Thanks

>
> Thanks,
> Ram
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]