Using kernel headers that are not for the running kernel

Using kernel headers that are not for the running kernel

[Nick Bartos]

I have a little distro that I am trying to upgrade to 2.6.x.

The problem is that when I use the headers for 2.6.x, glibc 2.2.5 won't
compile.  Eventually I want to upgrade glibc/gcc, but not at the moment. 
If I use the headers from 2.4.26 for the system, but just compile the
2.6.7 kernel, things do compile fine for everything.

This distro is small, and I can rebuild the entire thing in about 90 mins,
so if I change the kernel (or really anything that has other deps), I just
rebuild the entire thing to make sure everything is in sync.

I see that a lot of distros use a separate package for the kernel headers,
which do not necessarily coincide with the running kernel.

I am wondering what (if any) are the side effects of doing this are,
especially when the kernel versions are so different.  I was thinking that
there may be issues with some progs if the prototypes for certain kernel
functions weren't the same.  However people are doing it and it does seem
to work, but I am wondering how it fends for stability.

Comments?

RE: Using kernel headers that are not for the running kernel

[David Schwartz]

> I have a little distro that I am trying to upgrade to 2.6.x.

	Okay.

> The problem is that when I use the headers for 2.6.x, glibc 2.2.5 won't
> compile.  Eventually I want to upgrade glibc/gcc, but not at the moment.
> If I use the headers from 2.4.26 for the system, but just compile the
> 2.6.7 kernel, things do compile fine for everything.

	You should not be using the kernel headers to compile glibc.

> This distro is small, and I can rebuild the entire thing in about 90 mins,
> so if I change the kernel (or really anything that has other deps), I just
> rebuild the entire thing to make sure everything is in sync.

	The running kernel should not be in sync with glibc.

> I see that a lot of distros use a separate package for the kernel headers,
> which do not necessarily coincide with the running kernel.

	Right, there is no reason they should.

> I am wondering what (if any) are the side effects of doing this are,
> especially when the kernel versions are so different.  I was thinking that
> there may be issues with some progs if the prototypes for certain kernel
> functions weren't the same.  However people are doing it and it does seem
> to work, but I am wondering how it fends for stability.

	It creates a stable system. Things become much less stable if you mess
around with all of userspace just because the kernel changes. There is no
reason user space should be in sync with the running kernel. User space
should be stable.

	The later the kernel headers you use in user space, the less compatability
you will have with earlier kernels. A program compiled with 2.4.26 header
files should work with 2.6.x, but not vice versa. When new kernel interfaces
are added, the old ones are retained to keep compatability with user space,
but programs compiled against the newer headers may fail to work with older
kernels that lack the newer interfaces. Of course, user space programs
compiled against the older kernel header files can't get the advantages of
the newer interfaces.

	The kernel-user interface is supposed to stay stable, so you shouldn't need
to make significant user space changes when you upgrade the kernel. Only
specific applications that need to get specific new features that require
changes to the kernel-user interface need to change. It has been a very long
time since compiling user space programs against the header files for the
kernel you happened to be running was considered good practice.

	DS

Re: Using kernel headers that are not for the running kernel

[Krzysztof Halasa]

"David Schwartz" <davids@webmaster.com> writes:

> 	It creates a stable system. Things become much less stable if you mess
> around with all of userspace just because the kernel changes. There is no
> reason user space should be in sync with the running kernel. User space
> should be stable.

Kernel headers should and in fact are stable WRT common userspace
interface. If they aren't, you're forced to recompile the userspace
anyway.

BTW: it's ok to compile things like glibc against new kernel headers
(say, 2.6.x) and use the resulting library with older kernels (as old
as 2.2 I think). In fact it's the preferred way to compile glibc.
You can disable support for older kernels with glibc/configure
--enable-kernel=VERSION --enable-oldest-abi=ABI.

> The kernel-user interface is supposed to stay stable, so you shouldn't need
> to make significant user space changes when you upgrade the kernel. Only
> specific applications that need to get specific new features that require
> changes to the kernel-user interface need to change.

Sure. Examples: ioctls for configuring the system.

> It has been a very long
> time since compiling user space programs against the header files for the
> kernel you happened to be running was considered good practice.

I think at this point we have to create include/abi (or api) as a part
of the kernel. The mess with distributions-provided "glibc kernel
headers" must at last be cleaned.

Should no one have time for doing that I'm going to start.
-- 
Krzysztof Halasa, B*FH

Re: Using kernel headers that are not for the running kernel

[Jesper Juhl]

On Thu, 17 Jun 2004, Nick Bartos wrote:

> I have a little distro that I am trying to upgrade to 2.6.x.
>
> The problem is that when I use the headers for 2.6.x, glibc 2.2.5 won't
> compile.  Eventually I want to upgrade glibc/gcc, but not at the moment.
> If I use the headers from 2.4.26 for the system, but just compile the
> 2.6.7 kernel, things do compile fine for everything.
>
[snip]
>
> Comments?
> -

I think this covers the issues you are having pretty well :
http://www.linuxmafia.com/faq/Kernel/usr-src-linux-symlink.html


--
Jesper Juhl <juhl-lkml@dif.dk>

Re: Using kernel headers that are not for the running kernel

[Rob Landley]

On Thursday 17 June 2004 18:28, Nick Bartos wrote:
> I have a little distro that I am trying to upgrade to 2.6.x.
>
> The problem is that when I use the headers for 2.6.x, glibc 2.2.5 won't
> compile.  Eventually I want to upgrade glibc/gcc, but not at the moment.
> If I use the headers from 2.4.26 for the system, but just compile the
> 2.6.7 kernel, things do compile fine for everything.

The linux-kernel maintainers apparently decided that C libraries using kernel 
headers to actually interface with the kernel was a bad idea.  Apparently, 
interfacing with the kernel from a C library is not a proper use for kernel 
headers, or something.  (I tried to follow the logic in this discussion, but 
never actually found any, despite repeated attempts.  It always seemed to 
boil down to "can't be bothered", "userspace shouldn't use kernel headers and 
this includes the C library", etc...)

On a pragmatic level, you want the cleaned up 2.6 kernel headers maintained by 
Mariusz Mazur <mmazur@kernel.pl> which you can download from:

http://ep09.pld-linux.org/~mmazur/linux-libc-headers/

> I see that a lot of distros use a separate package for the kernel headers,
> which do not necessarily coincide with the running kernel.

Yup.  It sucks, but there's no alternative for 2.6.

Mazur does a good job, though.

> I am wondering what (if any) are the side effects of doing this are,
> especially when the kernel versions are so different.

You're generally safe using older header versions.  (Heck, you can use 2.4 
headers to build your C library if you like.  The resulting C library just 
won't be able to use various new features like more than 256 minor numbers 
for devices, or possibly the ability to burn with ATA CD-ROM drives without 
SCSI emulation...)

> I was thinking that
> there may be issues with some progs if the prototypes for certain kernel
> functions weren't the same.  However people are doing it and it does seem
> to work, but I am wondering how it fends for stability.

If using old headers didn't work on new kernels then using old binaries 
wouldn't work on new kernels.  And that would be noticed big time.  (We can 
still run binaries that ran on Linux 0.01.  A couple minor things have been 
ripped out over the years, but not much.  The big backwards compatability 
issue is generally making sure you have the relevant old versions of the C 
library installed if you really want to run WordPerfect 6 or Netscape or 
such....)

Rob
-- 
www.linucon.org: Linux Expo and Science Fiction Convention
October 8-10, 2004 in Austin Texas.  (I'm the con chair.)

Re: Using kernel headers that are not for the running kernel

[Jeff Garzik]

On Sat, Jun 19, 2004 at 05:46:50AM -0500, Rob Landley wrote:
> The linux-kernel maintainers apparently decided that C libraries using kernel 
> headers to actually interface with the kernel was a bad idea.  Apparently, 
> interfacing with the kernel from a C library is not a proper use for kernel 
> headers, or something.  (I tried to follow the logic in this discussion, but 
> never actually found any, despite repeated attempts.  It always seemed to 
> boil down to "can't be bothered", "userspace shouldn't use kernel headers and 
> this includes the C library", etc...)

No, the problem is that the only thing that needs to be shared are the
_ABI_ headers, which are unfortunately mixed in with kernel-internal
headers and definitions.  This leads to use of kernel-internal
definitions in userspace, which leads to breakage.  This also leads to
restrictions on changing -kernel-internal- headers, because some
userspace wanker is complaining.

Kernel-internal headers and definitions should absolutely never be used
in userspace.

H. Peter Anvin has suggested an include/abi which could be shared, and
this seem quite reasonable to me.  However, the monumental task of
separating kernel-internal definitions from ABI definitions still
remains.

	Jeff, really glad the linux-libc-headers guys started his effort

Re: Using kernel headers that are not for the running kernel

[Rob Landley]

On Sunday 20 June 2004 11:24, Jeff Garzik wrote:

> Kernel-internal headers and definitions should absolutely never be used
> in userspace.

Hence the old #ifdef KERNEL stuff, or whatever the guard was...

My only confusion was that when the #ifdefs stopped being maintained (written 
off as inherently unworkable because people just #defined KERNEL when they 
shouldn't), no actual replacement was pursued.  Instead the attitude seemed 
to be "this is glibc's problem", we're too busy trying to get 2.6 out to 
actually worry about anybody using it.  And calling it glibc's problem 
doesn't work for me, because want to use uclibc...

> H. Peter Anvin has suggested an include/abi which could be shared, and
> this seem quite reasonable to me.  However, the monumental task of
> separating kernel-internal definitions from ABI definitions still
> remains.
>
> 	Jeff, really glad the linux-libc-headers guys started his effort

Mazur seems to be doing a really nice job of it so far.  I'm building a small 
distro based on it and sending him bug reports when I can't get something to 
compile.  I'm happy to use his work, but I'd rather it got integrated into 
the kernel.

Now that it's mostly stabilized, it seems that the remaining work is mostly 
auditing, integrating it in under the include/abi directory, and cleaning up 
the normal kernel headers to include the abi stuff rather than defining their 
own copies in the kernel internal headers.

If an abi directory was created, I'd be happy to submit a file or two at a 
time into it (with the corresponding patches to remove the definitions from 
the main include directory and #include abi/whatever instead...)

(Is there some effort _other_ than Mazur's work I should know about?  Or 
something wrong with Mazur's cleanups?  Or somebody already doing this...?)

Rob
-- 
www.linucon.org: Linux Expo and Science Fiction Convention
October 8-10, 2004 in Austin Texas.  (I'm the con chair.)

Re: Using kernel headers that are not for the running kernel

[Randy.Dunlap]

On Sun, 20 Jun 2004 19:37:20 -0500 Rob Landley wrote:

| On Sunday 20 June 2004 11:24, Jeff Garzik wrote:
| 
| > Kernel-internal headers and definitions should absolutely never be used
| > in userspace.
| 
| Hence the old #ifdef KERNEL stuff, or whatever the guard was...
| 
| My only confusion was that when the #ifdefs stopped being maintained (written 
| off as inherently unworkable because people just #defined KERNEL when they 
| shouldn't), no actual replacement was pursued.  Instead the attitude seemed 
| to be "this is glibc's problem", we're too busy trying to get 2.6 out to 
| actually worry about anybody using it.  And calling it glibc's problem 
| doesn't work for me, because want to use uclibc...
| 
| > H. Peter Anvin has suggested an include/abi which could be shared, and
| > this seem quite reasonable to me.  However, the monumental task of
| > separating kernel-internal definitions from ABI definitions still
| > remains.
| >
| > 	Jeff, really glad the linux-libc-headers guys started his effort
| 
| Mazur seems to be doing a really nice job of it so far.  I'm building a small 
| distro based on it and sending him bug reports when I can't get something to 
| compile.  I'm happy to use his work, but I'd rather it got integrated into 
| the kernel.
| 
| Now that it's mostly stabilized, it seems that the remaining work is mostly 
| auditing, integrating it in under the include/abi directory, and cleaning up 
| the normal kernel headers to include the abi stuff rather than defining their 
| own copies in the kernel internal headers.
| 
| If an abi directory was created, I'd be happy to submit a file or two at a 
| time into it (with the corresponding patches to remove the definitions from 
| the main include directory and #include abi/whatever instead...)
| 
| (Is there some effort _other_ than Mazur's work I should know about?  Or 
| something wrong with Mazur's cleanups?  Or somebody already doing this...?)

Yes, or sort of.  There's a linuxabi mailing list:

http://zytor.com/mailman/listinfo/linuxabi

and talk about doing a big push on this in early 2.7.
There are several interested parties, but I don't know how interested
the top-level maintainer is in accepting such patches.


--
~Randy