[ANNOUNCE] linux-libc-headers 2.6.3.0

[ANNOUNCE] linux-libc-headers 2.6.3.0

[Mariusz Mazur]

Available at http://ep09.pld-linux.org/~mmazur/linux-libc-headers/
Changes:
- updated to 2.6.3
- linux/hdreg.h - fixed types in defs exposed to userspace
- using sys/resource.h instead of linux/resource.h
- couple of minor fixes

Enjoy.

-- 
In the year eighty five ten
God is gonna shake his mighty head
He'll either say,
"I'm pleased where man has been"
Or tear it down, and start again

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[H. Peter Anvin]

Followup to:  <200402291942.45392.mmazur@kernel.pl>
By author:    Mariusz Mazur <mmazur@kernel.pl>
In newsgroup: linux.dev.kernel
>
> Available at http://ep09.pld-linux.org/~mmazur/linux-libc-headers/
> Changes:
> - updated to 2.6.3
> - linux/hdreg.h - fixed types in defs exposed to userspace
> - using sys/resource.h instead of linux/resource.h
> - couple of minor fixes
> 

Dumb but important question: what is the copyright on these headers?

	-hpa

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[Mariusz Mazur]

On Sunday 29 of February 2004 21:19, H. Peter Anvin wrote:
> Dumb but important question: what is the copyright on these headers?

They are based on original linux headers, so gpl of course.


-- 
In the year eighty five ten
God is gonna shake his mighty head
He'll either say,
"I'm pleased where man has been"
Or tear it down, and start again

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[H. Peter Anvin]

Followup to:  <200402292130.55743.mmazur@kernel.pl>
By author:    Mariusz Mazur <mmazur@kernel.pl>
In newsgroup: linux.dev.kernel
>
> On Sunday 29 of February 2004 21:19, H. Peter Anvin wrote:
> > Dumb but important question: what is the copyright on these headers?
> 
> They are based on original linux headers, so gpl of course.
> 

If so, we actually have a bit of an issue w.r.t. the potential
legality of these headers.  Technically they're incompatible with LGPL
and BSD-licensed libraries; I think we need some kind of official
declaration that compiling against them is permitted.

	-hpa

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[Mariusz Mazur]

On Sunday 29 of February 2004 22:03, H. Peter Anvin wrote:
> If so, we actually have a bit of an issue w.r.t. the potential
> legality of these headers.  Technically they're incompatible with LGPL
> and BSD-licensed libraries; I think we need some kind of official
> declaration that compiling against them is permitted.

/me knows nothing about legalities. So if you could explain exactly why, what 
to do about it and how can that help... (and why there wasn't any problem 
earlier, when people used headers from linux tarballs?)


-- 
In the year eighty five ten
God is gonna shake his mighty head
He'll either say,
"I'm pleased where man has been"
Or tear it down, and start again

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[Måns Rullgård]

Mariusz Mazur <mmazur@kernel.pl> writes:

> On Sunday 29 of February 2004 22:03, H. Peter Anvin wrote:
>> If so, we actually have a bit of an issue w.r.t. the potential
>> legality of these headers.  Technically they're incompatible with LGPL
>> and BSD-licensed libraries; I think we need some kind of official
>> declaration that compiling against them is permitted.
>
> /me knows nothing about legalities. So if you could explain exactly why, what 
> to do about it and how can that help... (and why there wasn't any problem 
> earlier, when people used headers from linux tarballs?)

Excuse my ignorance, but why can't the headers from the kernel still
be used.  They seem to be working fine here.

-- 
Måns Rullgård
mru@kth.se

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[Chris Friesen]

Måns Rullgård wrote:

> Excuse my ignorance, but why can't the headers from the kernel still
> be used.  They seem to be working fine here.

For current kernels, the "official" method is to have cleaned up copies 
of the kernel headers shipped with glibc and placed in 
/usr/include/linux and /usr/include/asm.  The "real" headers will often 
work, but not always,

To complicate things, if you add new stuff to the kernel (new ioctl 
commands, etc.) then your app needs to either link against the "real" 
headers, or else duplicate the definitions.

Its kind of a mess.

In an ideal world there would be clean "userspace" headers shipped with 
the kernel, and the kernel would then use those headers plus the 
kernel-only stuff.

Chris



-- 
Chris Friesen                    | MailStop: 043/33/F10
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[Krzysztof Halasa]

Chris Friesen <cfriesen@nortelnetworks.com> writes:

> For current kernels, the "official" method is to have cleaned up
> copies of the kernel headers shipped with glibc and placed in
> /usr/include/linux and /usr/include/asm.

Not sure about it being "official".
It may make sense if it's a distribution - many users don't install
kernel sources. Still, from a technical point of view, it should be
a straight copy of kernel includes - we don't want to maintain two
separate sets, do we?

>  The "real" headers will
> often work, but not always,

Then they should be fixed.
I.e. parts for internal kernel use should be wrapped with #ifdef __KERNEL__.
Personally I consider every kernel header which prevents successful
user space compilation buggy.

> To complicate things, if you add new stuff to the kernel (new ioctl
> commands, etc.) then your app needs to either link against the "real"
> headers, or else duplicate the definitions.
>
> Its kind of a mess.

Precisely. This is why we need just one header set.

> In an ideal world there would be clean "userspace" headers shipped
> with the kernel, and the kernel would then use those headers plus the
> kernel-only stuff.

Not sure about it. How is it different from clean "kernel" headers
shipped (obviously) with the kernel?


The "non-problem" here is, IMHO, that the cleaning of kernel headers
is quite trivial, and thus nobody is interested :-)
-- 
Krzysztof Halasa, B*FH

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[Måns Rullgård]

Krzysztof Halasa <khc@pm.waw.pl> writes:

> The "non-problem" here is, IMHO, that the cleaning of kernel headers
> is quite trivial, and thus nobody is interested :-)

It usually is.  I guess I'll start sending patches whenever I need to
change one to get something compiled.  Usually it's so trivial that I
think I must have made a mistake somewhere else.

-- 
Måns Rullgård
mru@kth.se

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[Sam Ravnborg]

[Also sent to linuxabi@zytor.com]

On Mon, Mar 01, 2004 at 07:10:31PM +0100, Krzysztof Halasa wrote:
> 
> The "non-problem" here is, IMHO, that the cleaning of kernel headers
> is quite trivial, and thus nobody is interested :-)

The problem is that there is no infrastructure for abi only .h files (mainly).
Matthew Wilcox + others IIRC has already posted a few patches, but I do not see
this happening until 2.7 opens up.

When the proper infrastructure is agreed upon on lot's of people will put some effort
in this janitorial type of work. But imagine all the small mistakes, something we do not
want in 2.6.

IIRC the current agreed scheme is something along the lines of this:

abi/abi-linux/* Userspace relevant parts of include/linux
abi/abi-asm/ symlink to abi/abi-$(ARCH)
abi/abi-i386 i386 specific userland abi
abi/abi-ppc  ppc ....


So a header file in include/linux with a counterpart in abi could look like this:

include/linux/wait.h:
#include <abi-linux/wait.h>

#include <linux/config.h>
typedef struct __wait_queue wait_queue_t;
...


abi/abi-linux/wait.h:
#define WNOHANG         0x00000001
#define WUNTRACED       0x00000002



This proposal meets some resistence related to internal issues such as
renaming of internal types etc.
But in the end the gain from a scheme like this outweights the drawbacks - IMHO. 

And the backward compatible stuff can be located in abi where it may belong -
if really needed.

	Sam

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[Krzysztof Halasa]

Sam Ravnborg <sam@ravnborg.org> writes:

> IIRC the current agreed scheme is something along the lines of this:
>
> abi/abi-linux/* Userspace relevant parts of include/linux
> abi/abi-asm/ symlink to abi/abi-$(ARCH)
> abi/abi-i386 i386 specific userland abi
> abi/abi-ppc  ppc ....

More efforts, no real effects.
I don't think we need such an infrastructure.
The normal headers should just be usable for user-space inclusion.

However I realize that the difference isn't that important, as long
as we don't duplicate the definitions etc.

> So a header file in include/linux with a counterpart in abi could look
> like this:
>
> include/linux/wait.h:
> #include <abi-linux/wait.h>
>
> #include <linux/config.h>
> typedef struct __wait_queue wait_queue_t;
> ...
>
>
> abi/abi-linux/wait.h:
> #define WNOHANG         0x00000001
> #define WUNTRACED       0x00000002

why not:

include/linux/wait.h:

#define WNOHANG         0x00000001
#define WUNTRACED       0x00000002

#ifdef __KERNEL__

#include <linux/config.h>
...

#endif /* __KERNEL__ */

> But in the end the gain from a scheme like this outweights the drawbacks
> - IMHO.

Such as? In comparison to a (fixed) present situation?
-- 
Krzysztof Halasa, B*FH

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[Jeff Garzik]

Krzysztof Halasa wrote:
> Sam Ravnborg <sam@ravnborg.org> writes:
> 
> 
>>IIRC the current agreed scheme is something along the lines of this:
>>
>>abi/abi-linux/* Userspace relevant parts of include/linux
>>abi/abi-asm/ symlink to abi/abi-$(ARCH)
>>abi/abi-i386 i386 specific userland abi
>>abi/abi-ppc  ppc ....
> 
> 
> More efforts, no real effects.
> I don't think we need such an infrastructure.
> The normal headers should just be usable for user-space inclusion.

No, this is a big pain :)

The ABI headers shared with userspace need to be split from definitions 
that are only used in kernel space.  #ifdef __KERNEL__ is a source of 
frustration :)

	Jeff

Re: [Linuxabi] Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[H. Peter Anvin]

Jeff Garzik wrote:
> Krzysztof Halasa wrote:
> 
>> Sam Ravnborg <sam@ravnborg.org> writes:
>>
>>
>>> IIRC the current agreed scheme is something along the lines of this:
>>>
>>> abi/abi-linux/* Userspace relevant parts of include/linux
>>> abi/abi-asm/ symlink to abi/abi-$(ARCH)
>>> abi/abi-i386 i386 specific userland abi
>>> abi/abi-ppc  ppc ....
>>
>>
>>
>> More efforts, no real effects.
>> I don't think we need such an infrastructure.
>> The normal headers should just be usable for user-space inclusion.
> 
> 
> No, this is a big pain :)
> 
> The ABI headers shared with userspace need to be split from definitions 
> that are only used in kernel space.  #ifdef __KERNEL__ is a source of 
> frustration :)
> 

The biggest problem is that what is kernel ABI hasn't been very well 
thought about.  The result is that the split between the what the kernel 
exports and what libc is supposed to provide itself is very haphazard.

And yes, #ifdef __KERNEL__ is a reflection of that.

	-hpa

Re: [Linuxabi] Re: linux-libc-headers 2.6.3.0

[H. Peter Anvin]

Sam Ravnborg wrote:
> 
> IIRC the current agreed scheme is something along the lines of this:
> 
> abi/abi-linux/* Userspace relevant parts of include/linux
> abi/abi-asm/ symlink to abi/abi-$(ARCH)
> abi/abi-i386 i386 specific userland abi
> abi/abi-ppc  ppc ....
> 
> So a header file in include/linux with a counterpart in abi could look like this:
> 
> include/linux/wait.h:
> #include <abi-linux/wait.h>
> 
> #include <linux/config.h>
> typedef struct __wait_queue wait_queue_t;
> ...
> 
> 
> abi/abi-linux/wait.h:
> #define WNOHANG         0x00000001
> #define WUNTRACED       0x00000002
> 
> 
> This proposal meets some resistence related to internal issues such as
> renaming of internal types etc.
> But in the end the gain from a scheme like this outweights the drawbacks - IMHO. 
> 
> And the backward compatible stuff can be located in abi where it may belong -
> if really needed.
> 

I think the main issue is that it's going to take a fair amount of work, 
and hence needs to wait until 2.7.

The other thing to consider if whether or not there should be some way 
to export things that aren't easily expressible as #defines...

	-hpa

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[Mariusz Mazur]

On Monday 01 of March 2004 19:10, Krzysztof Halasa wrote:
> Not sure about it being "official".
> It may make sense if it's a distribution - many users don't install
> kernel sources. Still, from a technical point of view, it should be
> a straight copy of kernel includes - we don't want to maintain two
> separate sets, do we?

We do. Abi changes are rather rare so keeping them as a separate tree wouldn't 
add too much work for subsystem maintainers, and it would be a Good Thing 
(tm) to have one place where the whole current abi can be seen. One thing to 
note is that linux headers duplicate many structures and definitions that 
should be (and are) provided by glibc. This causes collisions. My current 
practice is to clear offending linux headers of their content, and simply 
include appropriate libc headers (with a nice warning) from them.
I can say that about 60-80% of current linux headers do not have proper 
separation of kernel only code (counting in headers that are kernel only, but 
have no visible signs of that fact) and adding that separation would take a 
while. And even if successfull, it would add significant maintainer burden to 
keep the whole thing working (it would probably look like crap too). 
And we have to remember 2.4 compatibilities (which linux-libc-headers have) - 
is 2.6 kernel a place for them?
(keep in mind I know squat about kernel development, so don't rely on my 
opinions too much)

My current objective is to remove *all* remaining kernel code from 
linux-libc-headers and add as much 2.4 compatibilities as possible. And this 
will take a loooong time since the majority of headers are kernel only, but 
are not marked as such, or are marked as such on one arch only (ppc guys 
rock :), and assumption that removal of those headers from other archs is 
safe is... well... just an assumption.

And as to leaving this as a separate project, or trying to push it into the 
kernel - well, I see pros and cons of both options (and it's not my 
decision :).


-- 
In the year eighty five ten
God is gonna shake his mighty head
He'll either say,
"I'm pleased where man has been"
Or tear it down, and start again

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[Grzegorz Kulewski]

On Wed, 3 Mar 2004, Mariusz Mazur wrote:

> On Monday 01 of March 2004 19:10, Krzysztof Halasa wrote:
> > Not sure about it being "official".
> > It may make sense if it's a distribution - many users don't install
> > kernel sources. Still, from a technical point of view, it should be
> > a straight copy of kernel includes - we don't want to maintain two
> > separate sets, do we?
> 
> We do. Abi changes are rather rare so keeping them as a separate tree wouldn't 
> add too much work for subsystem maintainers, and it would be a Good Thing 
> (tm) to have one place where the whole current abi can be seen. One thing to 
> note is that linux headers duplicate many structures and definitions that 
> should be (and are) provided by glibc. This causes collisions. My current 
> practice is to clear offending linux headers of their content, and simply 
> include appropriate libc headers (with a nice warning) from them.
> I can say that about 60-80% of current linux headers do not have proper 
> separation of kernel only code (counting in headers that are kernel only, but 
> have no visible signs of that fact) and adding that separation would take a 
> while. And even if successfull, it would add significant maintainer burden to 
> keep the whole thing working (it would probably look like crap too). 
> And we have to remember 2.4 compatibilities (which linux-libc-headers have) - 
> is 2.6 kernel a place for them?
> (keep in mind I know squat about kernel development, so don't rely on my 
> opinions too much)
> 
> My current objective is to remove *all* remaining kernel code from 
> linux-libc-headers and add as much 2.4 compatibilities as possible. And this 
> will take a loooong time since the majority of headers are kernel only, but 
> are not marked as such, or are marked as such on one arch only (ppc guys 
> rock :), and assumption that removal of those headers from other archs is 
> safe is... well... just an assumption.
> 
> And as to leaving this as a separate project, or trying to push it into the 
> kernel - well, I see pros and cons of both options (and it's not my 
> decision :).

Then maybe we need 3 types of includes (each in separate directory I 
think):
1. kernel only - cannot be seen from user space, for current kernel, no 
comatibility
2. user only - cannot be seen from kernel (mainly for compatibility), will 
not change very often
3. shared - really portions of includes with definitions that are shared 
between current kernel headers and user headers, are included by both 1. 
and 2. internally (= both kernel and user space does not include them 
directly), extra care should be taken when changing it

This way there are no _KERNEL_ ifdefs needed any more and there is no code 
or data duplications (= no inconsistencies between 1. and 2.).

And if some definitions in kernel 6.6.6 will be obsolete but should be 
kept for compatibility with user space then it will be moved from 3. to 2. 
and apropriate #include commands will be removed from 1. (and from 2. 
too).

What do You think about this? (I know I am completly wrong :))


Grzegorz Kulewski

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[Krzysztof Halasa]

Mariusz Mazur <mmazur@kernel.pl> writes:

> We do. Abi changes are rather rare so keeping them as a separate tree
> wouldn't 
> add too much work for subsystem maintainers, and it would be a Good Thing 
> (tm) to have one place where the whole current abi can be seen.

Sure. One place, yes. But separate header sets are IMHO too much crazy.

> One thing to 
> note is that linux headers duplicate many structures and definitions that 
> should be (and are) provided by glibc. This causes collisions. My current 
> practice is to clear offending linux headers of their content, and simply 
> include appropriate libc headers (with a nice warning) from them.

Is it the kernel which is based on glibc, or is it glibc (and the rest of
the userland as glibc isn't that special) using the kernel interface?

The kernel doesn't need glibc at all, I don't know why do you want it
to require some external headers to compile.
Should the kernel behave differently when compiled with different glibc
header sets? :-)

IMHO all the defines should be in the kernel tree. Glibc can and should
use them, as it uses the ABI.

> I can say that about 60-80% of current linux headers do not have proper 
> separation of kernel only code (counting in headers that are kernel only,
> but 
> have no visible signs of that fact)

No doubt.

> and adding that separation would take a 
> while. And even if successfull, it would add significant maintainer burden
> to 
> keep the whole thing working (it would probably look like crap too).

Don't think so. Anyway, there is no other acceptable option. The kernel
must compile without glibc headers (without any external headers, to
be precise) - as it doesn't need any external software to work.

The open question (of much less importance) is if we want to keep
the existing include/ layout or to move public parts to include/linux-abi
etc. It still has to reside in the kernel tree, though. I'd go with the
former for now as it requires less work. OTOH the latter might be
cleaner.

> And we have to remember 2.4 compatibilities (which linux-libc-headers
> have) - 
> is 2.6 kernel a place for them?

Examples?
If they are part of kernel API/ABI, then of course they are still used
by 2.6 kernel and they need to be there. If they aren't used by the
kernel (old #define names for instance) they should go to glibc headers
(#ifndef xxx #define xxx etc.).
-- 
Krzysztof Halasa, B*FH

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[Mariusz Mazur]

On Thursday 04 of March 2004 15:13, Krzysztof Halasa wrote:
> Is it the kernel which is based on glibc, or is it glibc (and the rest of
> the userland as glibc isn't that special) using the kernel interface?
>
> The kernel doesn't need glibc at all, I don't know why do you want it
> to require some external headers to compile.
> Should the kernel behave differently when compiled with different glibc
> header sets? :-)

I never said kernel should require glibc - it shouldn't (mind you I don't do 
kernelland headers). But kernel does duplicate each and every structure 
provided by glibc. It has to. The Bad Thing (tm) is that all (well... allmost 
all - lots of linux headers don't parse correctly in userspace) of those 
structures get exported to userland. And programmers use them. They don't 
include <sys/resource.h>, but <linux/resource.h>. And that causes conflicts 
(and is bad practice).

> IMHO all the defines should be in the kernel tree. Glibc can and should
> use them, as it uses the ABI.

Parts of abi that are standardized 
(http://www.opengroup.org/onlinepubs/007904975/ - this thing; check the 
headers section), should be imho provided by C libs. These things do not 
change (they can't or everything would blow up) and I see no reason why glibc 
should rely on having additional headers available, just to do what it's 
supposed to.

> The open question (of much less importance) is if we want to keep
> the existing include/ layout or to move public parts to include/linux-abi
> etc. It still has to reside in the kernel tree, though. I'd go with the
> former for now as it requires less work. OTOH the latter might be
> cleaner.

Userland headers should be kept in /usr/include/{asm,linux}. I see no reason 
to change that (kernel headers have no business being in /usr/include btw). 
As to linux-common linux-kernelonly and linux-userland headers (linux-common 
used by both) - I just find it weird for userland to require kernel sources. 
Linux is supposed to have stable abi.

> > And we have to remember 2.4 compatibilities (which linux-libc-headers
> > have) -
> > is 2.6 kernel a place for them?
>
> Examples?
> If they are part of kernel API/ABI, then of course they are still used
> by 2.6 kernel and they need to be there. If they aren't used by the
> kernel (old #define names for instance) they should go to glibc headers
> (#ifndef xxx #define xxx etc.).

Additionall defines mostly. Probably some extra structures.


-- 
In the year eighty five ten
God is gonna shake his mighty head
He'll either say,
"I'm pleased where man has been"
Or tear it down, and start again

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[Chris Friesen]

Mariusz Mazur wrote:
> Parts of abi that are standardized 
> (http://www.opengroup.org/onlinepubs/007904975/ - this thing; check the 
> headers section), should be imho provided by C libs. These things do not 
> change (they can't or everything would blow up) and I see no reason why glibc 
> should rely on having additional headers available, just to do what it's 
> supposed to.

And how do you propose to handle a kernel developer that wants to add a 
new feature (a scheduling class, for instance) and make it available to 
userspace apps?

For the sched class example, the standard says, "Four scheduling 
policies are defined; others may be defined by the implementation."  I 
would like to make it easy for kernel developers to extend the 
implementation and expose that to userspace.

It would be nice to have apps be able to include <linux/sched.h> and get 
the *real* kernel sched.h userspace export.  Then when I add 
SCHED_NEW_CLASS to the kernel, userspace can pick up the changes without 
me having to modify glibc headers as well.

> As to linux-common linux-kernelonly and linux-userland headers (linux-common 
> used by both) - I just find it weird for userland to require kernel sources. 
> Linux is supposed to have stable abi.

I would prefer to have the standard userspace kernel headers be owned by 
the kernel.  If we do it right, we could go back to the days of the 
symlink to the headers of the real running kernel.  As far as I can 
tell, that's the only way to avoid having to make changes in multiple 
places.

Chris






-- 
Chris Friesen                    | MailStop: 043/33/F10
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[Mariusz Mazur]

On Thursday 04 of March 2004 22:27, Chris Friesen wrote:
> And how do you propose to handle a kernel developer that wants to add a
> new feature (a scheduling class, for instance) and make it available to
> userspace apps?

Dunno. That's something one should ask the glibc people.


-- 
In the year eighty five ten
God is gonna shake his mighty head
He'll either say,
"I'm pleased where man has been"
Or tear it down, and start again

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[Chris Friesen]

Mariusz Mazur wrote:
> On Thursday 04 of March 2004 22:27, Chris Friesen wrote:
> 
>>And how do you propose to handle a kernel developer that wants to add a
>>new feature (a scheduling class, for instance) and make it available to
>>userspace apps?
>>
> 
> Dunno. That's something one should ask the glibc people.

That's exactly my point.  I don't *want* to have to ask the glibc people.

I want to have the kernel ABI headers ship with the kernel, and only 
with the kernel.

I want to be able to change one header file, recompile my kernel, 
recompile my app, reboot, and pick up the change.  glibc should not even 
be a factor.

Chris




-- 
Chris Friesen                    | MailStop: 043/33/F10
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[Krzysztof Halasa]

Mariusz Mazur <mmazur@kernel.pl> writes:

> I never said kernel should require glibc - it shouldn't (mind you I don't do 
> kernelland headers). But kernel does duplicate each and every structure 
> provided by glibc. It has to.

No, there is no such requirement. Glibc doesn't need to duplicate kernel
defs either.

Do you really want duplicated code (definitions etc)?

> The Bad Thing (tm) is that all (well... allmost 
> all - lots of linux headers don't parse correctly in userspace)

That's bad, for sure.

> of those 
> structures get exported to userland. And programmers use them. They don't 
> include <sys/resource.h>, but <linux/resource.h>. And that causes conflicts 
> (and is bad practice).

So the kernel headers are buggy (and should be fixed) and the glibc
headers contain unneeded code (and they should be fixed as well).

>> IMHO all the defines should be in the kernel tree. Glibc can and should
>> use them, as it uses the ABI.
>
> Parts of abi that are standardized 
> (http://www.opengroup.org/onlinepubs/007904975/ - this thing; check the 
> headers section), should be imho provided by C libs.

The C library alone obviously provide some functionality such as string.h
functions and such API should be defined in glibc headers.
However, a part of the glibc (or any libc) functionality is provided by
the kernel itself. It is still correct to say that those are provided
by libc, and that this is libc providing the headers. It doesn't mean
the syscalls can't go straight to the kernel and that the libc
headers can't just include/be the kernel ones.

> These things do not 
> change (they can't or everything would blow up) and I see no reason why glibc 
> should rely on having additional headers available, just to do what it's 
> supposed to.

Because glibc has to rely on underlying kernel services while the user
space programs are executed.

You know, Linux can be run on different architectures. Do you want to
keep different sets of header files for each arch? Yes, the APIs differs.

BTW: the user space programs don't necessarily need glibc. They may need
the kernel API headers, of course.
You may even want to use another C library. Would you copy the headers
in such case again?

> Userland headers should be kept in /usr/include/{asm,linux}.

Because?

> As to linux-common linux-kernelonly and linux-userland headers (linux-common 
> used by both) - I just find it weird for userland to require kernel sources. 
> Linux is supposed to have stable abi.

No, it isn't. It's supposed to have a backward-compatible ABI.
You don't want to block progress, do you?

However, this isn't only about progress. A functionality can be removed,
too.

>> Examples?
>> If they are part of kernel API/ABI, then of course they are still used
>> by 2.6 kernel and they need to be there. If they aren't used by the
>> kernel (old #define names for instance) they should go to glibc headers
>> (#ifndef xxx #define xxx etc.).
>
> Additionall defines mostly. Probably some extra structures.

I'd go with the above, then.
-- 
Krzysztof Halasa, B*FH

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[Grzegorz Kulewski]

On Fri, 5 Mar 2004, Krzysztof Halasa wrote:
> BTW: the user space programs don't necessarily need glibc. They may need
> the kernel API headers, of course.
> You may even want to use another C library. Would you copy the headers
> in such case again?

I agree.


> > As to linux-common linux-kernelonly and linux-userland headers (linux-common 
> > used by both) - I just find it weird for userland to require kernel sources. 

But some programs (hdparm?) and/or libs must (for example most C libs like 
glibc)...


> >> If they are part of kernel API/ABI, then of course they are still used
> >> by 2.6 kernel and they need to be there. If they aren't used by the
> >> kernel (old #define names for instance) they should go to glibc headers
> >> (#ifndef xxx #define xxx etc.).
> >
> > Additionall defines mostly. Probably some extra structures.
> 
> I'd go with the above, then.

But how synchronize kernel development with glibcs (+ all other C libs + 
all other programs that must interface directly with the kernel)?

When kernel developers want to stop using something, how they should tell 
that to glibc developers and others. And how these non-kernel developers 
can change their projects in very short time to start using new kernel 
headers (= define things removed from the kernel in their own headers)? 
Most admins change kernels more often than all other programs and libs 
and these programs can potenitially not be updated yet.

My proposal is to move these things from linux-common to linux-userland 
instead of removng them from the kernel immendiatelly.
So no compatibility with user program and libs will be broken (and after a 
few releases of kernel, when all programs and libs will be updated, these 
things can be removed completly from kernel headers of all three types).


Grzegorz Kulewski
ex inx user :)

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[Krzysztof Halasa]

Grzegorz Kulewski <kangur@polcom.net> writes:

> But how synchronize kernel development with glibcs (+ all other C libs + 
> all other programs that must interface directly with the kernel)?

The kernel API should be stable enough, at least within stable trees.
Outside stable trees we should still maintain backward compatibility,
except when it comes to things which wouldn't work anyway (non-libc API
such as ipfw/ipchains/iptables etc).

> When kernel developers want to stop using something, how they should tell 
> that to glibc developers and others.

They should not stop :-)
Adding things is another story.

> Most admins change kernels more often than all other programs and libs 
> and these programs can potenitially not be updated yet.

I don't think so.
Applying a kernel patch usually doesn't change the API. Changing kernel
config should never change the API.

> My proposal is to move these things from linux-common to linux-userland 
> instead of removng them from the kernel immendiatelly.
> So no compatibility with user program and libs will be broken (and after a 
> few releases of kernel, when all programs and libs will be updated, these 
> things can be removed completly from kernel headers of all three types).

Things which are IMHO not options:
- removing kernel headers (i.e. headers which are required by kernel
  files to compile) from the kernel tree,
- duplicating kernel header definitions in external packages (a straight
  copy for distro users is ok, things which need maintenance aren't).
  Existing duplicates should be dealt with.
-- 
Krzysztof Halasa, B*FH

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[Paul Jackson]

> Changing kernel config should never change the API.

Linux would be in pretty sad shape if that held.

Incompatible API changes should be rare, but they are an essential part
of our continuing healthy evolution.  In particular, there's a _long_
list of hardware devices that have at some time or other worked on
Linux, but don't anymore - usually because no one is still maintaining
the driver needed.  But sometimes other API's have to change or
disappear as well.

Would you say that a city or county should _never_ raze a building or
remove a road as part of development?

Change happens.  Deal with it.

-- 
                          I won't rest till it's the best ...
                          Programmer, Linux Scalability
                          Paul Jackson <pj@sgi.com> 1.650.933.1373

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[Krzysztof Halasa]

Paul Jackson <pj@sgi.com> writes:

>> Changing kernel config should never change the API.
>
> Linux would be in pretty sad shape if that held.
>
> Incompatible API changes should be rare, but they are an essential part
> of our continuing healthy evolution.  In particular, there's a _long_
> list of hardware devices that have at some time or other worked on
> Linux, but don't anymore - usually because no one is still maintaining
> the driver needed.  But sometimes other API's have to change or
> disappear as well.

You're talking about the kernel development while I meant changing
kernel configuration. They're very distant things.

And note that it's all about C API and not about some API between
different parts of the kernel.
-- 
Krzysztof Halasa, B*FH

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[Paul Jackson]

> You're talking about the kernel development ...

No.  I mean that even the C API that the kernel presents to
user code will sometimes change or have parts disappear.

To be avoided, yes.  Worth going to substantial lengths
to avoid, yes.  But not absolutely prohibited.

-- 
                          I won't rest till it's the best ...
                          Programmer, Linux Scalability
                          Paul Jackson <pj@sgi.com> 1.650.933.1373

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[Krzysztof Halasa]

Paul Jackson <pj@sgi.com> writes:

>> You're talking about the kernel development ...
>
> No.  I mean that even the C API that the kernel presents to
> user code will sometimes change or have parts disappear.

And it can be caused by changing kernel config?

Care to show an example?
-- 
Krzysztof Halasa, B*FH

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[Chris Friesen]

Krzysztof Halasa wrote:
> Paul Jackson <pj@sgi.com> writes:
> 
> 
>>>You're talking about the kernel development ...
>>>
>>No.  I mean that even the C API that the kernel presents to
>>user code will sometimes change or have parts disappear.
>>
> 
> And it can be caused by changing kernel config?
> 
> Care to show an example?

Generally it cannot be caused by changing kernel config (although kernel 
config can cause syscalls to be valid or not).

However, new kernel versions can add/remove stuff.  Usually stuff isn't 
removed for a while after its deprecated, but there have been instances 
where stuff has been added/changed/removed during unstable kernel 
development.

Chris


-- 
Chris Friesen                    | MailStop: 043/33/F10
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[Krzysztof Halasa]

Chris Friesen <cfriesen@nortelnetworks.com> writes:

> Generally it cannot be caused by changing kernel config (although
> kernel config can cause syscalls to be valid or not).
>
> However, new kernel versions can add/remove stuff.  Usually stuff
> isn't removed for a while after its deprecated, but there have been
> instances where stuff has been added/changed/removed during unstable
> kernel development.

Well, _this_ fits nicely my own idea of the situation.
-- 
Krzysztof Halasa, B*FH

Re: [ANNOUNCE] linux-libc-headers 2.6.3.0

[Benjamin Herrenschmidt]

On Mon, 2004-03-01 at 07:30, Mariusz Mazur wrote:
> On Sunday 29 of February 2004 21:19, H. Peter Anvin wrote:
> > Dumb but important question: what is the copyright on these headers?
> 
> They are based on original linux headers, so gpl of course.

GPL isn't a copyright ... :)

Ben.