[RFC] Move InfiniBand .h files

[RFC] Move InfiniBand .h files

[Roland Dreier]

I would like to get people's reactions to moving the InfiniBand .h
files from their current location in drivers/infiniband/include/ to
include/linux/rdma/.  If we agree that this is a good idea then I'll
push this change as soon as 2.6.14 starts.

The advantages of doing this are:

  - The headers become more easily accessible to other parts of the
    tree that might want to use IB support.  For example, an NFS/RDMA
    client probably wants to live under fs/
  - It makes it easier to build IB modules outside the tree, since
    include/linux gets put in /lib/modules/<ver>/build.  I realize
    that we don't really care about out-of-tree modules, but it is
    convenient to be able to develop and distribute new drivers that
    build against someone's existing kernels.
  - We can kill off the ugly

        EXTRA_CFLAGS += -Idrivers/infiniband/include

    lines in our Makefiles.

The disadvantages are:

  - It's churn with little technical merit.
  - It makes it a little harder to pull the OpenIB svn tree into a
    kernel tree, since one would have to link both drivers/infiniband
    and include/linux/rdma instead of just drivers/infiniband.  This
    problem goes away if/when OpenIB shifts over to a new source code
    control system.

Thanks,
  Roland

Re: [openib-general] [RFC] Move InfiniBand .h files

[Tom Duffy]

[-- Attachment #1: Type: text/plain, Size: 1136 bytes --]

On Thu, 2005-08-04 at 10:32 -0700, Roland Dreier wrote:
> I would like to get people's reactions to moving the InfiniBand .h
> files from their current location in drivers/infiniband/include/ to
> include/linux/rdma/.  If we agree that this is a good idea then I'll
> push this change as soon as 2.6.14 starts.

I think it is a great idea.

> The advantages of doing this are:
> 
>   - The headers become more easily accessible to other parts of the
>     tree that might want to use IB support.  For example, an NFS/RDMA
>     client probably wants to live under fs/
>   - It makes it easier to build IB modules outside the tree, since
>     include/linux gets put in /lib/modules/<ver>/build.  I realize
>     that we don't really care about out-of-tree modules, but it is
>     convenient to be able to develop and distribute new drivers that
>     build against someone's existing kernels.
>   - We can kill off the ugly
> 
>         EXTRA_CFLAGS += -Idrivers/infiniband/include
> 
>     lines in our Makefiles.

One more advantage:

   - It shows our willingness to push past just infiniband.

-tduffy

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFC] Move InfiniBand .h files

[Arjan van de Ven]

On Thu, 2005-08-04 at 10:32 -0700, Roland Dreier wrote:
> I would like to get people's reactions to moving the InfiniBand .h
> files from their current location in drivers/infiniband/include/ to
> include/linux/rdma/.  If we agree that this is a good idea then I'll
> push this change as soon as 2.6.14 starts.

please only put userspace clean headers here; the rest is more or less
private headers for your subsystem. 

At minimum the headers should be split in separate files for
shared-userspace and kernel (eg no overlap at all), but I'd vote for
keeping the headers in your own dir.

Re: [openib-general] Re: [RFC] Move InfiniBand .h files

[Grant Grundler]

On Thu, Aug 04, 2005 at 07:53:58PM +0200, Arjan van de Ven wrote:
> On Thu, 2005-08-04 at 10:32 -0700, Roland Dreier wrote:
> > I would like to get people's reactions to moving the InfiniBand .h
> > files from their current location in drivers/infiniband/include/ to
> > include/linux/rdma/.  If we agree that this is a good idea then I'll
> > push this change as soon as 2.6.14 starts.
> 
> please only put userspace clean headers here; the rest is more or less
> private headers for your subsystem. 

Sorry...this smells like a rathole...but does this mean
linus agrees the kernel subsystems should export headers suitable for
both user space and kernel driver modules?

Historical, I thought glibc and other user space libs were expected to
maintain their own set of header files. Maybe I'm just confused...

thanks,
grant

Re: [openib-general] Re: [RFC] Move InfiniBand .h files

[Arjan van de Ven]

On Thu, 2005-08-04 at 11:26 -0700, Grant Grundler wrote:
> On Thu, Aug 04, 2005 at 07:53:58PM +0200, Arjan van de Ven wrote:
> > On Thu, 2005-08-04 at 10:32 -0700, Roland Dreier wrote:
> > > I would like to get people's reactions to moving the InfiniBand .h
> > > files from their current location in drivers/infiniband/include/ to
> > > include/linux/rdma/.  If we agree that this is a good idea then I'll
> > > push this change as soon as 2.6.14 starts.
> > 
> > please only put userspace clean headers here; the rest is more or less
> > private headers for your subsystem. 
> 
> Sorry...this smells like a rathole...but does this mean
> linus agrees the kernel subsystems should export headers suitable for
> both user space and kernel driver modules?
> 
> Historical, I thought glibc and other user space libs were expected to
> maintain their own set of header files. Maybe I'm just confused...

there is a definite requirement for the kernel to expose SOME things to
userspace. Well for SOMETHING to expose them. Right now most distros
ship a hacked up version of the kernel headers (eg removed of all the
kernel specific stuff and all the gpl inline code etc). A good part of
making such an external project possible is to make a clean separation
between userspace shared stuff and pure kernel internals.

Re: [RFC] Move InfiniBand .h files

[Roland Dreier]

    Arjan> At minimum the headers should be split in separate files
    Arjan> for shared-userspace and kernel (eg no overlap at all), but
    Arjan> I'd vote for keeping the headers in your own dir.

This is already done -- the userspace ABI is defined in ib_user_mad.h,
ib_user_verbs.h, etc.

The problem with keeping subsystem headers under drivers/infiniband is
that it's ugly for, say, fs/nfs/Makefile to have to add
-Idrivers/infiniband/include to its CFLAGS just because it's
implementing NFS/RDMA.

Also, drivers/infiniband/include doesn't get put into the
/lib/modules/<ver>/build directory, so it's a pain for people
developing new drivers (this is a real complaint that came to me from
a vendor developing a driver for a new piece of IB hardware).

Thanks,
  Roland

Re: [RFC] Move InfiniBand .h files

[Arjan van de Ven]

> Also, drivers/infiniband/include doesn't get put into the
> /lib/modules/<ver>/build directory,

that is a symlink not a directory, and a symlink to the full source...

Re: [RFC] Move InfiniBand .h files

[Roland Dreier]

    Roland> Also, drivers/infiniband/include doesn't get put into the
    Roland> /lib/modules/<ver>/build directory,

    Arjan> that is a symlink not a directory, and a symlink to the
    Arjan> full source...

Sorry, I was too terse about the problem.  You're right, but typical
distros don't ship full kernel source in their "support kernel builds"
package.  And if I use an external build directory (ie "O=") then
the symlink just points to my external build directory, which doesn't
include the source to drivers/, just links to include/

 - R.

Re: [RFC] Move InfiniBand .h files

[Sam Ravnborg]

On Thu, Aug 04, 2005 at 11:57:55AM -0700, Roland Dreier wrote:
>     Roland> Also, drivers/infiniband/include doesn't get put into the
>     Roland> /lib/modules/<ver>/build directory,
> 
>     Arjan> that is a symlink not a directory, and a symlink to the
>     Arjan> full source...
> 
> Sorry, I was too terse about the problem.  You're right, but typical
> distros don't ship full kernel source in their "support kernel builds"
> package.  And if I use an external build directory (ie "O=") then
> the symlink just points to my external build directory, which doesn't
> include the source to drivers/, just links to include/

If the external module uses a Kbuild file as explained in
Documentation/kbuild/makefiles.txt and then uses both O= and M=
when compiling the module there is no issue.

With respect to moving the .h files - please do so.
drivers/infiniband should only include header used in that same
directory. Not header files potentially uased by fs/.

	Sam

Re: [RFC] Move InfiniBand .h files

[Christopher Friesen]

Sam Ravnborg wrote:
> On Thu, Aug 04, 2005 at 11:57:55AM -0700, Roland Dreier wrote:

>>Sorry, I was too terse about the problem.  You're right, but typical
>>distros don't ship full kernel source in their "support kernel builds"
>>package.  And if I use an external build directory (ie "O=") then
>>the symlink just points to my external build directory, which doesn't
>>include the source to drivers/, just links to include/
> 
> 
> If the external module uses a Kbuild file as explained in
> Documentation/kbuild/makefiles.txt and then uses both O= and M=
> when compiling the module there is no issue.
> 
> With respect to moving the .h files - please do so.
> drivers/infiniband should only include header used in that same
> directory. Not header files potentially uased by fs/.

I think Roland was talking about the case where the running kernel was 
built with "O=", in which case the /lib/modules.../build symlink points 
to the build directory rather than the original source tree.

Does Kbuild handle this case properly?

Chris

Re: [RFC] Move InfiniBand .h files

[Sam Ravnborg]

On Thu, Aug 04, 2005 at 01:54:56PM -0600, Christopher Friesen wrote:
> Sam Ravnborg wrote:
> >On Thu, Aug 04, 2005 at 11:57:55AM -0700, Roland Dreier wrote:
> 
> >>Sorry, I was too terse about the problem.  You're right, but typical
> >>distros don't ship full kernel source in their "support kernel builds"
> >>package.  And if I use an external build directory (ie "O=") then
> >>the symlink just points to my external build directory, which doesn't
> >>include the source to drivers/, just links to include/
> >
> >
> >If the external module uses a Kbuild file as explained in
> >Documentation/kbuild/makefiles.txt and then uses both O= and M=
> >when compiling the module there is no issue.
> >
> >With respect to moving the .h files - please do so.
> >drivers/infiniband should only include header used in that same
> >directory. Not header files potentially uased by fs/.
> 
> I think Roland was talking about the case where the running kernel was 
> built with "O=", in which case the /lib/modules.../build symlink points 
> to the build directory rather than the original source tree.
> 
> Does Kbuild handle this case properly?

Yes it does.
/lib/modules/.../ contains two symlinks these days:

build -> always point to the directory containing the output of the build
source -> always point to the kernel source

In the 'make' case where the kernel is built without using O= they point
to the same directory.
In the 'make O=' case they point to different directories.

SUSE does ship with a make O= build kernel these days.
Fedora IIRC has done an ugly hack and just copied over a number of files
so a compile works in most cases - but then also use both symlink.

It has never been easier to build a module if the target is only the
running kernel. Only when you adds backwards compatibility it gets messy :-(

	Sam

Re: [RFC] Move InfiniBand .h files

[Arjan van de Ven]

On Thu, 2005-08-04 at 11:57 -0700, Roland Dreier wrote:
>     Roland> Also, drivers/infiniband/include doesn't get put into the
>     Roland> /lib/modules/<ver>/build directory,
> 
>     Arjan> that is a symlink not a directory, and a symlink to the
>     Arjan> full source...
> 
> Sorry, I was too terse about the problem.  You're right, but typical
> distros don't ship full kernel source in their "support kernel builds"
> package.

so what makes you think they will ship include/infiniband ?

RE: [openib-general] [RFC] Move InfiniBand .h files

[Steve Wise]

Seems reasonable to me...


Steve.

 

> -----Original Message-----
> From: openib-general-bounces@openib.org 
> [mailto:openib-general-bounces@openib.org] On Behalf Of Roland Dreier
> Sent: Thursday, August 04, 2005 12:32 PM
> To: openib-general@openib.org; linux-kernel@vger.kernel.org
> Subject: [openib-general] [RFC] Move InfiniBand .h files
> 
> I would like to get people's reactions to moving the InfiniBand .h
> files from their current location in drivers/infiniband/include/ to
> include/linux/rdma/.  If we agree that this is a good idea then I'll
> push this change as soon as 2.6.14 starts.
> 
> The advantages of doing this are:
> 
>   - The headers become more easily accessible to other parts of the
>     tree that might want to use IB support.  For example, an NFS/RDMA
>     client probably wants to live under fs/
>   - It makes it easier to build IB modules outside the tree, since
>     include/linux gets put in /lib/modules/<ver>/build.  I realize
>     that we don't really care about out-of-tree modules, but it is
>     convenient to be able to develop and distribute new drivers that
>     build against someone's existing kernels.
>   - We can kill off the ugly
> 
>         EXTRA_CFLAGS += -Idrivers/infiniband/include
> 
>     lines in our Makefiles.
> 
> The disadvantages are:
> 
>   - It's churn with little technical merit.
>   - It makes it a little harder to pull the OpenIB svn tree into a
>     kernel tree, since one would have to link both drivers/infiniband
>     and include/linux/rdma instead of just drivers/infiniband.  This
>     problem goes away if/when OpenIB shifts over to a new source code
>     control system.
> 
> Thanks,
>   Roland
> _______________________________________________
> openib-general mailing list
> openib-general@openib.org
> http://openib.org/mailman/listinfo/openib-general
> 
> To unsubscribe, please visit 
> http://openib.org/mailman/listinfo/openib-general
> 

Re: [openib-general] [RFC] Move InfiniBand .h files

[Grant Grundler]

On Thu, Aug 04, 2005 at 10:32:14AM -0700, Roland Dreier wrote:
...
I agree with the rename/relocation of the header files for
the reasons you mentioned.

>   - It makes it a little harder to pull the OpenIB svn tree into a
>     kernel tree, since one would have to link both drivers/infiniband
>     and include/linux/rdma instead of just drivers/infiniband.  This
>     problem goes away if/when OpenIB shifts over to a new source code
>     control system.

Any thoughts on renaming drivers/infiniband to drivers/rdma
at the same time?

If you are going to churn...don't be shy about it :^)

grant

Re: [openib-general] [RFC] Move InfiniBand .h files

[Roland Dreier]

    Grant> Any thoughts on renaming drivers/infiniband to drivers/rdma
    Grant> at the same time?

    Grant> If you are going to churn...don't be shy about it :^)

Well, I'd rather avoid churn for purely political reasons.  The main
point of my proposal is to move the includes from drivers/ to
include/, but while we're at it me might as well pick a more neutral
directory name.

Moving drivers/infiniband to drivers/rdma has no technical merit right
now, so I'd rather wait and see how it makes sense to organize the
code we end up with.

 - R.

Re: [openib-general] [RFC] Move InfiniBand .h files

[James Lentini]

On Thu, 4 Aug 2005, Roland Dreier wrote:

> I would like to get people's reactions to moving the InfiniBand .h
> files from their current location in drivers/infiniband/include/ to
> include/linux/rdma/.  If we agree that this is a good idea then I'll
> push this change as soon as 2.6.14 starts.

I think it is a good idea.

> The advantages of doing this are:
>
>  - The headers become more easily accessible to other parts of the
>    tree that might want to use IB support.  For example, an NFS/RDMA
>    client probably wants to live under fs/

net/sunrpc/ has also been proposed.

Re: [RFC] Move InfiniBand .h files

[Christoph Hellwig]

On Thu, Aug 04, 2005 at 10:32:14AM -0700, Roland Dreier wrote:
> I would like to get people's reactions to moving the InfiniBand .h
> files from their current location in drivers/infiniband/include/ to
> include/linux/rdma/.  If we agree that this is a good idea then I'll
> push this change as soon as 2.6.14 starts.

include/rmda, please.  not need for the linux/ component.

> 
> The advantages of doing this are:
> 

Re: [RFC] Move InfiniBand .h files

[Roland Dreier]

    Christoph> include/rmda, please.  not need for the linux/ component.

OK, fair enough.  Any objection to this?

 - R.

Re: [RFC] Move InfiniBand .h files

[Sam Ravnborg]

On Fri, Aug 05, 2005 at 09:08:41AM -0700, Roland Dreier wrote:
>     Christoph> include/rmda, please.  not need for the linux/ component.
> 
> OK, fair enough.  Any objection to this?
No, makes sense.

	Sam