[ANNOUNCE] linux-libc-headers dead

[ANNOUNCE] linux-libc-headers dead

[Mariusz Mazur]

LLH hasn't seen a new release for a lot more than six months now and up until 
today I hoped to get back on track with new releases. But I've just spent 
some time doing a 2.6.14 update, and it came back to me, that I'd have to 
spend up to 10 hours just to get a basic 2.6.14.0 ready. And there'd still be 
2.6.15 waiting, 2.6.16 just around the corner plus sorting through all the 
bug reports that came in during those months and all the internal rearranging 
I either had planned or that's being forced by new kernel releases (eg. 
addition of asm-powerpc).

I stopped having both the time and the will for such commitments a couple of 
months ago.

Should anyone want to take over, I'd be happy to give hints, pointers, and 
whatnot. Just don't get overexcited -- diffs between new kernel versions get 
bigger, not smaller, and after a couple of years there's still no long term 
solution in sight.

Oh, and women don't fall for the "I hack kernel stuff" line. I was lied to.


-- 
In the year eighty five ten
God is gonna shake his mighty head
He'll either say,
"I'm pleased where man has been"
Or tear it down, and start again

Re: [ANNOUNCE] linux-libc-headers dead

[Ismail Donmez]

[-- Attachment #1: Type: text/plain, Size: 1051 bytes --]

Hi,
Salı 14 Mart 2006 17:19 tarihinde şunları yazmıştınız:
> LLH hasn't seen a new release for a lot more than six months now and up
> until today I hoped to get back on track with new releases. But I've just
> spent some time doing a 2.6.14 update, and it came back to me, that I'd
> have to spend up to 10 hours just to get a basic 2.6.14.0 ready. And
> there'd still be 2.6.15 waiting, 2.6.16 just around the corner plus sorting
> through all the bug reports that came in during those months and all the
> internal rearranging I either had planned or that's being forced by new
> kernel releases (eg. addition of asm-powerpc).

Thank you for all your work, its greatly appreciated. I think its time to fix 
public kernel headers so they can be included in userspace oh well I 
hear "Don't include kernel headers in userspace" flames already but some 
things are impossible without including kernel headers ( ide/v4l etc comes to 
mind ).

Regards,
ismail


-- 
An eye for eye will make the whole world blind -- Gandhi

[-- Attachment #2: Type: application/pgp-signature, Size: 191 bytes --]

Re: [llh-announce] [ANNOUNCE] linux-libc-headers dead

[Nigel Kukard]

> LLH hasn't seen a new release for a lot more than six months now and up until
> today I hoped to get back on track with new releases. But I've just spent
> some time doing a 2.6.14 update, and it came back to me, that I'd have to
> spend up to 10 hours just to get a basic 2.6.14.0 ready. And there'd still be
> 2.6.15 waiting, 2.6.16 just around the corner plus sorting through all the
> bug reports that came in during those months and all the internal rearranging
> I either had planned or that's being forced by new kernel releases (eg.
> addition of asm-powerpc).
>
> I stopped having both the time and the will for such commitments a couple of
> months ago.
>
> Should anyone want to take over, I'd be happy to give hints, pointers, and
> whatnot. Just don't get overexcited -- diffs between new kernel versions get
> bigger, not smaller, and after a couple of years there's still no long term
> solution in sight.

I'm interested, more so to be part of a team which as a group is 
responsible for linux-libc-headers. This way if one or two persons 
don't have much time available, the others can share the load.

If there is no-one else willing to maintain linux-libc-headers, I'll be 
more than happy to take it over and try get more developers interested.


Kind Regards

-- 
Nigel Kukard, PhD CompSc

Re: [llh-announce] [ANNOUNCE] linux-libc-headers dead

[Jan Engelhardt]

>
> If there is no-one else willing to maintain linux-libc-headers, I'll be more
> than happy to take it over and try get more developers interested.
>

Some linux distributions (I know of Novell who do it for SUSE Linux)
seem to roll their own thing AFAICS. The glibc.src.rpm from them contains
a userspacified copy of the kernel headers.
Well, it's probably not bleeeeding edge uptodate, but provides a starting 
point.


Jan Engelhardt
-- 

Re: [llh-announce] [ANNOUNCE] linux-libc-headers dead

[Dan Kegel]

On 3/16/06, Jan Engelhardt <jengelh@linux01.gwdg.de> wrote:
> Some linux distributions (I know of Novell who do it for SUSE Linux)
> seem to roll their own thing AFAICS. The glibc.src.rpm from them contains
> a userspacified copy of the kernel headers.

Yep.  Red Hat does, too.  They had to, since there wasn't
a linux-libc-headers project when they started.

Maybe the way to move forward is to see if we can
get the linux-libc-headers accepted as is into
the kernel.org tree in a userspace-include directory,
then maintain it there.
- Dan

--
Wine for Windows ISVs: http://kegel.com/wine/isv

Re: [ANNOUNCE] linux-libc-headers dead

[DervishD]

    Hi Mariusz :)

 * Mariusz Mazur <mmazur@kernel.pl> dixit:
> I stopped having both the time and the will for such commitments a
> couple of months ago.

    I understand. And it's a pity, because LLH is very useful, and
you did a GREAT job with it.
 
> Oh, and women don't fall for the "I hack kernel stuff" line. I was lied to.

    Yes... XDDD

    Well, I can volunteer to test LLH and contribute a bit, fixing
things I can fix, but IMHO this kind of work should be done by the
kernel itself: it should provide both user-space and kernel-space
headers (yes, I know, "don't-include-kernel-headers-in-userspace".
Tell that to iproute2, iptables, etc...).

    Thanks a lot for your work, your effort and your interest in this
problem. You have done a lot for the linux community (at least,
you've done a lot for me).

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736 | http://www.dervishd.net
http://www.pleyades.net & http://www.gotesdelluna.net
It's my PC and I'll cry if I want to... RAmen!

State of userland headers

[Mariusz Mazur]

(I'm CCing everyone who mailed me during the past week and a half)

When I first created llh, it was a couple of weeks before 2.6.0 came out. I
had a need for those headers, since a new version of PLD distro was supposed
to be based on 2.6 kernels and one couldn't even build glibc with the
internal headers of the last 2.5 kernels (I assume the breakage occurred a
lot earlier, though).

And just because I quit, doesn't make the problem go away for my distro.
Worse, currently also a couple of other projects depend on llh (LFS and
uclibc are the first to come to my mind), so one way or the other, someone's
eventually got to do something about it. And I did get a couple of offers to
take over.

So here's how it goes.

LLH as it is is not and never will be mergable into mainline kernel. For a
simple reason -- it's a piece of total crap (poetically called a hack). There
are some nicely polished parts, mostly the core 'infrastructure' (endianness
support, basic data types, etc.), but the rest are mostly my best guesses at
'whether this looks like a userland header', and I can't even call them
educated guesses, since I simply have no clue about kernel internals. Plus
count in the various bad calls I've made during those two years.

So on one side we've got a barely working hack, that is not mergable, a royal
pita to maintain and gets harder to update with each kernel version. If
anyone wants to take a stab at it, then be my guest, I can tell you how I've
done it, but since we've got at least a couple of people directly interested
in this thing, I do believe it'd be a good idea to try to do it The Right
Way.

There was a thread on lkml on this topic about a year ago. IIRC I've
suggested, that the best option would be to get a new set of dirs somewhere
inside the kernel, and gradually export the userland usable stuff from the
kernel headers, so to (a) achieve full separation and (b) avoid duplication
of definitions (meaning that kernel headers would simply include the userland
ones where required). Linus said, that it would break stuff and so is
unacceptable.

Unfortunately I must agree with him -- I don't think it is possible to
completely avoid duplication of definitions and all tries would lead to
breakage of some obscure configurations -- kernel headers sometimes
require various magic that should be avoided inside the userland headers
at all cost. This means that initially the llh-ng project would need to
start as a completely separate entity that would not require the
original kernel headers for anything, and only later, after achieving
some level of maturity and getting merged into the kernel, would come
the time for removing some duplication. Of course that means that kernel
hackers would (a) need to keep in mind that they have two places to
update, (b) once llh-ng got merged, figure out where duplication can be
avoided and do something about it and (c) even when that'd get done,
still remember about double updates for places where duplication
couldn't have been avoided.

I won't be loosing any sleep about it, though. Those mean kernel hackers
never cared about us poor userland folks, so there's no reason we should
feel sorry for them. Payback time.

And now for some technical details on how I see it. I'd appreciate any input.

First thing, is that we'd probably need to use a distributed rcs, since it's
more flexible, and, well, distributed. Never used any, so git sounds as good
an option, as anything else.

Now, what I propose we do initially, is get a bunch of headers from current
llh for the most popular archs and figure out how to build all the various
*libcs against them. And of course update them to the newest kernel by the
way.

It'd probably be best if I did it. PLD has support for x86, x86_64, sparc, ppc
and alpha, so that's what I'd take care of first. It might sound like a good
idea to start with all available archs, but in reality we wouldn't even be
able to test them most of the times, so it's best to stick with the
archs we're directly interested in. They should be enough to get the
'design' of the thing right. And should anyone come by with a new arch
he's interested in, he'd have a quick time adding it, since the
foundations would already be there.

And here's where the first problem arises -- llh were so great
initially, because they removed a lot of conflicts with glibc, by simply
removing 'offending' linux headers and including the glibc counterparts
(eg. linux/socket.h would do nothing else, than include sys/socket.h).
Glibc's known for having lots of stuff simply 'hardcoded' into it's own
set of headers, and more often than not, people do need to include
headers from both places. I don't know about uclibc, but klibc afaik
expects a lot more of the linux headers to be present, than glibc does.
Hell, if llh-ng is supposed to be a full set of apis, we can't expect any of
the headers to come from other places, so if any other app (libcs included)
has duplicates, that's too bad for the app, since it's in need of some
patching. Unfortunately the glibc's build system is one hell of a scary
place to lurk around.

Ok, assuming we've got the libcs covered for the most popular archs, we're in
for supporting the rest of the apps. I can just throw whatever headers
we come up with at my distro and wait for the bug reports (along with
colorful descriptions of how slow and painful my death is going to be),
and I believe LFS guys can more or less do the same.  So after some
time, we'd hopefully have a bunch of mostly working headers, that are
clean and rather easy to maintain (although probably require a lot of
patches for various apps; but those would eventually get merged
upstream; at least most of the times, I hope; keep in mind my comment
about glibc being a scary and evil place).  *But* in the meantime we'd
be stuck with the old llh 2.6.13, so, like I've said before, if
anybody's still interested in doing an update with the old llh, mail me
(again) and I'll give you the details on how I've done it in the past.

And one other thing -- the assumption is that kernel<->userland apis/abis
(mostly the latter) shouldn't change in an incompatible way and if they do,
that's considered a bug. But until llh-ng got merged into the kernel,
we'd have to figure out a way to cope with new kernel releases. Eg. what
about netfilter headers, are they backwards compatible (yeah, I know
iptables have their own set of netfilter headers, but apps not needing
such hacks is kind of the point of the whole exercise)? And what about
different paces of updates for various archs. Eg. most people are happy
with the most popular archs, so it'd be a bad idea not to do a release
just because we're waiting for one of the more exotic archs. How does
one version such a project? Not like the old llh (x.y.z.t, where x.y.z
corresponds to the kernel version), since not all releases would be
always fully up to date with a given kernel. So maybe just automatic
snapshots every day (assuming a change has been made). But if so, who's
tree gets to be treated as the main one? (Distributed rcs, remember).
And should we choose such a person, what is the chance he'll end up just
like I did (a lazy schmuck that is)?

Ok, I'm getting too detailed here. One thing at a time.

So, waiting for any insights, counterproposals and/or declarations who's
willing to work on old llh, llh-ng or both. Shoot away.

I'm CCing this to llh-discuss@lists.pld-linux.org
(http://lists.pld-linux.org and click away; mailman over there, so you
know the drill). I've created that list about a month ago, when I
figured that when people mail me, they never get to see each other's
posts. I thought of deleting it a week ago, but, well, looks like it
might come in handy after all.


-- 
Judge others by their intentions and yourself by your results.
                                                                 Guy Kawasaki
Education is an admirable thing, but it is well to remember from
time to time that nothing that is worth knowing can be taught.
                                                                  Oscar Wilde

Re: State of userland headers

[Rob Landley]

On Thursday 23 March 2006 12:11 pm, Mariusz Mazur wrote:

> This means that initially the llh-ng project would need to 
> start as a completely separate entity that would not require the
> original kernel headers for anything, and only later, after achieving
> some level of maturity and getting merged into the kernel, would come
> the time for removing some duplication. Of course that means that kernel
> hackers would (a) need to keep in mind that they have two places to
> update, (b) once llh-ng got merged, figure out where duplication can be
> avoided and do something about it and (c) even when that'd get done,
> still remember about double updates for places where duplication
> couldn't have been avoided.

I mentioned earlier that klibc seems like a good test case.  Make userspace 
headers that klibc can build against, and work up from there...

> I won't be loosing any sleep about it, though. Those mean kernel hackers
> never cared about us poor userland folks, so there's no reason we should
> feel sorry for them. Payback time.
>
> And now for some technical details on how I see it. I'd appreciate any
> input.
>
> First thing, is that we'd probably need to use a distributed rcs, since
> it's more flexible, and, well, distributed. Never used any, so git sounds
> as good an option, as anything else.
>
> Now, what I propose we do initially, is get a bunch of headers from current
> llh for the most popular archs and figure out how to build all the various
> *libcs against them. And of course update them to the newest kernel by the
> way.
>
> It'd probably be best if I did it. PLD has support for x86, x86_64, sparc,
> ppc and alpha, so that's what I'd take care of first. It might sound like a
> good idea to start with all available archs, but in reality we wouldn't
> even be able to test them most of the times, so it's best to stick with the 
> archs we're directly interested in.

Here at timesys I've got access to arm, mips, and ppc.  (Can't say I really 
know what I'm doing on any of them, but I'm learning.)

> just because we're waiting for one of the more exotic archs. How does
> one version such a project? Not like the old llh (x.y.z.t, where x.y.z
> corresponds to the kernel version), since not all releases would be
> always fully up to date with a given kernel. So maybe just automatic
> snapshots every day (assuming a change has been made). But if so, who's
> tree gets to be treated as the main one? (Distributed rcs, remember).
> And should we choose such a person, what is the chance he'll end up just
> like I did (a lazy schmuck that is)?
>
> Ok, I'm getting too detailed here. One thing at a time.
>
> So, waiting for any insights, counterproposals and/or declarations who's
> willing to work on old llh, llh-ng or both. Shoot away.

I'm a little confused about the goals of the project.

In order to take advantage of new kernel features, like major/minor numbers 
greater than 8 bits or DMA for cd burning, we need new kernel headers.  That 
sort of thing is why we didn't just build against 2.4 kernel headers, which 
for some reason seemed to be usable from userspace just fine.

You also don't want to run a libc built with newer headers than the kernel 
you're running on, or it'll try to use stuff that isn't there.

You're saying that the new kernel headers wouldn't be versioned using the 
kernel's release numbers.  How do we know what kernel version their feature 
set matches then?  (I'm confused.  This happens easily...)

Rob
-- 
Never bet against the cheap plastic solution.

Re: State of userland headers

[Mariusz Mazur]

On Friday 24 March 2006 00:04, Rob Landley wrote:
> You also don't want to run a libc built with newer headers than the kernel
> you're running on, or it'll try to use stuff that isn't there.
>
> You're saying that the new kernel headers wouldn't be versioned using the
> kernel's release numbers.  How do we know what kernel version their feature
> set matches then?  (I'm confused.  This happens easily...)

That's a tradeoff. You either version the headers just like I did, meaning 
that a given version corresponds to a given kernel, but that means you can't 
release before all of the archs are fully updated (and not relying on a 
single person to do all of the updates is one of the points of the exercise; 
and with more people, one can have delays) or you're forced to figure out 
some other way to version the headers.

-- 
Judge others by their intentions and yourself by your results.
                                                                 Guy Kawasaki
Education is an admirable thing, but it is well to remember from
time to time that nothing that is worth knowing can be taught.
                                                                  Oscar Wilde

Re: State of userland headers

[Rob Landley]

On Sunday 26 March 2006 8:12 am, Mariusz Mazur wrote:
> On Friday 24 March 2006 00:04, Rob Landley wrote:
> > You also don't want to run a libc built with newer headers than the
> > kernel you're running on, or it'll try to use stuff that isn't there.
> >
> > You're saying that the new kernel headers wouldn't be versioned using the
> > kernel's release numbers.  How do we know what kernel version their
> > feature set matches then?  (I'm confused.  This happens easily...)
>
> That's a tradeoff. You either version the headers just like I did, meaning
> that a given version corresponds to a given kernel, but that means you
> can't release before all of the archs are fully updated (and not relying on
> a single person to do all of the updates is one of the points of the
> exercise; and with more people, one can have delays) or you're forced to
> figure out some other way to version the headers.

If somebody #includes <linux/version.h> from the new headers, and can 
determine what kernel version it goes with for their platform from that (even 
if the different platforms are out of sync), and said information also goes 
in a README or release notes or some such, I'll live.

Rob
-- 
Never bet against the cheap plastic solution.

Re: State of userland headers

[Kyle Moffett]

On Mar 23, 2006, at 12:11:26, Mariusz Mazur wrote:
> There was a thread on lkml on this topic about a year ago. IIRC  
> I've suggested, that the best option would be to get a new set of  
> dirs somewhere inside the kernel, and gradually export the userland  
> usable stuff from the kernel headers, so to (a) achieve full  
> separation and (b) avoid duplication of definitions (meaning that  
> kernel headers would simply include the userland ones where  
> required). Linus said, that it would break stuff and so is  
> unacceptable.

I seem to remember Linus saying that "breaking things is  
unacceptable", not that the project was guaranteed to break things  
(we would just need to be much more careful about it than most kernel  
patches).  What that seems to indicate to me is that an in-kernel  
version would need to do the following for userspace-accessible  
header files for a large number of kernel releases:

#ifndef _LINUX_HEADER_H
#define _LINUX_HEADER_H
#include <kabi/header.h>
   /* Define or typedef a bunch of __kabi_ prefixes to the old  
prefixes they used to have in the kernel header */
#ifndef __KERNEL__
# warning "The header file <linux/header.h> is deprecated for"
# warning "userspace, please use <kabi/header.h> instead."
#else
   /* Kernel-only declarations/definitions */
#endif

If this were done carefully, all programs that compile against kernel  
headers could be _fixed_ in the short term (they'd go from throwing  
errors to giving a couple deprecation warnings).  In the long term,  
the extra ifdeffery could be removed and the <linux/*.h> headers for  
which a <kabi/*.h> replacement had existed for a couple versions  
could be removed.  New ABIs (including IOCTLs, new syscalls, etc)  
could be required to use <kabi/*.h> in the first place.

> Unfortunately I must agree with him -- I don't think it is possible  
> to completely avoid duplication of definitions and all tries would  
> lead to breakage of some obscure configurations -- kernel headers  
> sometimes require various magic that should be avoided inside the  
> userland headers at all cost. This means that initially the llh-ng  
> project would need to start as a completely separate entity that  
> would not require the original kernel headers for anything, and  
> only later, after achieving some level of maturity and getting  
> merged into the kernel, would come the time for removing some  
> duplication.

I think that requiring any kind of duplication of effort on that  
large a scale is virtually guaranteed not to work.  It will break  
down and be really painful for a long time.

> And here's where the first problem arises -- llh were so great  
> initially, because they removed a lot of conflicts with glibc, by  
> simply removing 'offending' linux headers and including the glibc  
> counterparts (eg. linux/socket.h would do nothing else, than  
> include sys/socket.h).  Glibc's known for having lots of stuff  
> simply 'hardcoded' into it's own set of headers, and more often  
> than not, people do need to include headers from both places.

1: Ewww, bad glibc!
2: The symbols in kabi/*.h should probably all start with __kabi_

The kernel _internally_ would "#include <kabi/foo.h>" and then  
promptly redefine or typedef all of those objects.  If the kernel or  
GLIBC want a struct renamed, they should #define  
__kabi_name_of_struct name_of_struct just before including the header  
file. We should also kind of frown upon non-libc userspace including  
many <kabi/*.h> files, since if they get included before libc can  
redefine the names to what it wants.

> I don't know about uclibc, but klibc afaik expects a lot more of  
> the linux headers to be present, than glibc does.

Perhaps we should use klibc as our test-case for functional headers,  
then, instead of glibc.

> Hell, if llh-ng is supposed to be a full set of apis, we can't  
> expect any of the headers to come from other places, so if any  
> other app (libcs included) has duplicates, that's too bad for the  
> app, since it's in need of some patching.

The solution to that is to make sure that the new exported kernel  
ABIs always have the __kabi_ prefix, so that we don't smash the  
namespace of various programs.

Cheers,
Kyle Moffett

Re: State of userland headers

[Rob Landley]

On Friday 24 March 2006 1:51 pm, Kyle Moffett wrote:
> On Mar 23, 2006, at 12:11:26, Mariusz Mazur wrote:
> > There was a thread on lkml on this topic about a year ago. IIRC
> > I've suggested, that the best option would be to get a new set of
> > dirs somewhere inside the kernel, and gradually export the userland
> > usable stuff from the kernel headers, so to (a) achieve full
> > separation and (b) avoid duplication of definitions (meaning that
> > kernel headers would simply include the userland ones where
> > required). Linus said, that it would break stuff and so is
> > unacceptable.
>
> I seem to remember Linus saying that "breaking things is
> unacceptable", not that the project was guaranteed to break things
> (we would just need to be much more careful about it than most kernel
> patches).

The gentoo guys clean up their own headers, apparently.  I'm told they do so 
by moving around the #ifdef __KERNEL__ stuff to be in the correct places, and 
that they're currently working on a 2.6.14 or 2.6.15 version of the headers:

http://www.gentoo.org/cgi-bin/viewcvs.cgi/src/patchsets/gentoo-headers/?root=gentoo

> What that seems to indicate to me is that an in-kernel 
> version would need to do the following for userspace-accessible
> header files for a large number of kernel releases:
>
> #ifndef _LINUX_HEADER_H
> #define _LINUX_HEADER_H
> #include <kabi/header.h>
>    /* Define or typedef a bunch of __kabi_ prefixes to the old
> prefixes they used to have in the kernel header */
> #ifndef __KERNEL__
> # warning "The header file <linux/header.h> is deprecated for"
> # warning "userspace, please use <kabi/header.h> instead."
> #else
>    /* Kernel-only declarations/definitions */
> #endif

Changing the #include paths in all deployed software will basically never 
happen.  If this header package requires that, I'm not interested in it 
because I can't build existing software against it, and I don't expect anyone 
else to be.

I was thinking of possibly a parallel header set under linux-2.6.x/usr/include 
which the linux-2.6.x/include/*.h could #include to clean out their #ifndef 
__KERNEL__ stuff, and that eventually the usr/include stuff would contain 
approximately what Mazur's headers had contained.  Unfortunately, I'm under 
the impression that's not a realistic approach.

> If this were done carefully, all programs that compile against kernel
> headers could be _fixed_ in the short term (they'd go from throwing
> errors to giving a couple deprecation warnings).  In the long term,
> the extra ifdeffery could be removed and the <linux/*.h> headers for
> which a <kabi/*.h> replacement had existed for a couple versions
> could be removed.  New ABIs (including IOCTLs, new syscalls, etc)
> could be required to use <kabi/*.h> in the first place.

A program that includes kabi/* instead of linux/* won't build against older C 
libraries with older headers from older kernel versions (or older project's 
like Mazur's headers).

> 1: Ewww, bad glibc!
> 2: The symbols in kabi/*.h should probably all start with __kabi_

Any grand new incompatible thing is something I will happily ignore for as 
long as I am able to, and I'm not alone here.  Your uptake will be zero.

Rob
-- 
Never bet against the cheap plastic solution.

Re: State of userland headers

[Kyle Moffett]

On Mar 24, 2006, at 16:48:47, Nix wrote:
> On 24 Mar 2006, Rob Landley suggested tentatively:
>> On Friday 24 March 2006 1:51 pm, Kyle Moffett wrote:
>>> 1: Ewww, bad glibc!
>>> 2: The symbols in kabi/*.h should probably all start with __kabi_
>>
>> Any grand new incompatible thing is something I will happily  
>> ignore for as long as I am able to, and I'm not alone here.  Your  
>> uptake will be zero.
>
> I concur. The purpose of this thing is by definition to provide  
> libcs with the kernel/user interface stuff they need in order for  
> userspace programs to be compiled. There's no point defining a new  
> interface because there is a massive quantity of *existing* code  
> out there that we must work with. (Plus, it can be, uh, difficult  
> to get changes of this nature into glibc in particular, and glibc  
> is the 300-pound gorilla in this particular room. If the headers  
> don't have working with it as a goal, they are pointless.)

Hmm, I didn't really explain my idea very well.  Let me start with a  
list of a facts.  If anybody disagrees with any part of this, please  
let me know.

1)  The <linux/*.h> headers include a lot of information essential to  
compiling userspace applications and libraries (libcs in  
particular).  That same information is also required while building  
the kernel (IE: The ABI).
2)  Those headers have a lot of declarations and definitions which  
must *not* be present while compiling userspace applications, and is  
basically kernel-only stuff.
3)  Glibc is extremely large and complex 500-pound gorilla and  
contains an ugly build process and a lot of static definitions in its  
own header files that conflict with the definitions in the kernel  
headers.
4)  UML runs into a lot of problems when glibc's headers and the  
native kernel headers headers conflict.

Here's some of my opinions about this:

1)  Trying to create and maintain 2 separate versions of an ABI as  
large and complex as the kernel<=>userspace ABI across new versions  
and features would be extremely difficult and result in subtle bugs  
and missing features, even over a short period of time.
2)  Ideally there should be three distinct pieces, the kernel, the  
ABI, and userspace.  Compiling either the kernel or userspace  
requires the ABI, but the ABI depends only on the compiler.
3)  Breaking any compatibility is bad
4)  Trying to continue to maintain the glibc custom-header-file  
status-quo as more APIs and architectures get added to the kernel is  
going to become an increasingly difficult and tedious task.

My proposal (which I'm working on sample patches for) would be to  
divide up the kernel headers into 2 parts.  The first part would be  
<kabi/*.h>, and the second would be all the traditional kernel-only  
headers.  The kabi headers would *only* define things that begin with  
the prefix __kabi_.  This would mean that the kabi headers have no  
risk of namespace contamination with anything else existing in the  
kernel or userspace, and since they would depend only on the  
compiler, they would be useable anywhere.

The second step would be to convert the traditional linux header to  
include the corresponding kabi header, then redefine its own  
structures and defines in terms of those in the kabi header.  This  
would provide complete backwards compatibility to all kernel code, as  
well as to anything that currently compiles using the existing kernel  
headers.  The entire rest of the <linux/*.h> header file would be  
wrapped in #ifdef __KERNEL__, as it should not be needed by anything  
in userspace.

In the process of those two steps, we would relocate many of the  
misplaced "#ifdef __KERNEL__" and "#endif /* __KERNEL__ */".  The  
kabi headers should not mention __KERNEL__ at all, and the linux/*  
headers should be almost completely wrapped in __KERNEL__ ifdefs.   
That should be enough to make klibc build correctly, although from  
the description glibc needs significantly more work.

Once a significant portion of the kernel headers have been split that  
way (preserving complete backwards compatibility), external projects  
_may_ be converted to #include <kabi/*.h> instead of #include <linux/ 
*.h>, although this would require other changes to the source to  
handle the __kabi_ prefix.  Most of those should be straightforward,  
however.  Since the kabi/*.h headers would not be kernel-version- 
specific, they could be copied to a system running an older kernel  
and reused there without problems.  Even though some of the syscalls  
and ioctls referenced in the kabi headers might not be present on the  
running kernel, portable programs are expected to be able to sanely  
handle older kernels.

Once the kabi headers are available, it would be possible to begin  
cleaning up many of the glibc headers without worrying about  
differences between architectures.  If all critical constants and  
datatypes are already defined in <kabi/*.h> with __kabi_ or __KABI_  
prefixes, it should be possible to import those definitions into  
klibc and glibc without much effort.

UML has other issues with conflicts between the native kernel headers  
and the GLIBC-provided stubs.  It's been mentioned on the prior  
threads about this topic that this sort of system would ease most of  
the issues that UML runs into.

I'm working on some sample patches now which I'll try to post in a  
few days if I get the time.

Cheers,
Kyle Moffett

Re: State of userland headers

[Randy.Dunlap]

On Fri, 24 Mar 2006 17:46:27 -0500 Kyle Moffett wrote:

> On Mar 24, 2006, at 16:48:47, Nix wrote:
> > On 24 Mar 2006, Rob Landley suggested tentatively:
> >> On Friday 24 March 2006 1:51 pm, Kyle Moffett wrote:
> >>> 1: Ewww, bad glibc!
> >>> 2: The symbols in kabi/*.h should probably all start with __kabi_
> >>
> >> Any grand new incompatible thing is something I will happily  
> >> ignore for as long as I am able to, and I'm not alone here.  Your  
> >> uptake will be zero.
> >
> > I concur. The purpose of this thing is by definition to provide  
> > libcs with the kernel/user interface stuff they need in order for  
> > userspace programs to be compiled. There's no point defining a new  
> > interface because there is a massive quantity of *existing* code  
> > out there that we must work with. (Plus, it can be, uh, difficult  
> > to get changes of this nature into glibc in particular, and glibc  
> > is the 300-pound gorilla in this particular room. If the headers  
> > don't have working with it as a goal, they are pointless.)
> 
> Hmm, I didn't really explain my idea very well.  Let me start with a  
> list of a facts.  If anybody disagrees with any part of this, please  
> let me know.
> 
> 1)  The <linux/*.h> headers include a lot of information essential to  
> compiling userspace applications and libraries (libcs in  
> particular).  That same information is also required while building  
> the kernel (IE: The ABI).
> 2)  Those headers have a lot of declarations and definitions which  
> must *not* be present while compiling userspace applications, and is  
> basically kernel-only stuff.
> 3)  Glibc is extremely large and complex 500-pound gorilla and  
> contains an ugly build process and a lot of static definitions in its  
> own header files that conflict with the definitions in the kernel  
> headers.
> 4)  UML runs into a lot of problems when glibc's headers and the  
> native kernel headers headers conflict.
> 
> Here's some of my opinions about this:
> 
> 1)  Trying to create and maintain 2 separate versions of an ABI as  
> large and complex as the kernel<=>userspace ABI across new versions  
> and features would be extremely difficult and result in subtle bugs  
> and missing features, even over a short period of time.
> 2)  Ideally there should be three distinct pieces, the kernel, the  
> ABI, and userspace.  Compiling either the kernel or userspace  
> requires the ABI, but the ABI depends only on the compiler.
> 3)  Breaking any compatibility is bad
> 4)  Trying to continue to maintain the glibc custom-header-file  
> status-quo as more APIs and architectures get added to the kernel is  
> going to become an increasingly difficult and tedious task.
> 
> My proposal (which I'm working on sample patches for) would be to  
> divide up the kernel headers into 2 parts.  The first part would be  
> <kabi/*.h>, and the second would be all the traditional kernel-only  
> headers.  The kabi headers would *only* define things that begin with  
> the prefix __kabi_.  This would mean that the kabi headers have no  
> risk of namespace contamination with anything else existing in the  
> kernel or userspace, and since they would depend only on the  
> compiler, they would be useable anywhere.
> 
> The second step would be to convert the traditional linux header to  
> include the corresponding kabi header, then redefine its own  
> structures and defines in terms of those in the kabi header.  This  
> would provide complete backwards compatibility to all kernel code, as  
> well as to anything that currently compiles using the existing kernel  
> headers.  The entire rest of the <linux/*.h> header file would be  
> wrapped in #ifdef __KERNEL__, as it should not be needed by anything  
> in userspace.
> 
> In the process of those two steps, we would relocate many of the  
> misplaced "#ifdef __KERNEL__" and "#endif /* __KERNEL__ */".  The  
> kabi headers should not mention __KERNEL__ at all, and the linux/*  
> headers should be almost completely wrapped in __KERNEL__ ifdefs.   
> That should be enough to make klibc build correctly, although from  
> the description glibc needs significantly more work.
> 
> Once a significant portion of the kernel headers have been split that  
> way (preserving complete backwards compatibility), external projects  
> _may_ be converted to #include <kabi/*.h> instead of #include <linux/ 
> *.h>, although this would require other changes to the source to  
> handle the __kabi_ prefix.  Most of those should be straightforward,  
> however.  Since the kabi/*.h headers would not be kernel-version- 
> specific, they could be copied to a system running an older kernel  
> and reused there without problems.  Even though some of the syscalls  
> and ioctls referenced in the kabi headers might not be present on the  
> running kernel, portable programs are expected to be able to sanely  
> handle older kernels.
> 
> Once the kabi headers are available, it would be possible to begin  
> cleaning up many of the glibc headers without worrying about  
> differences between architectures.  If all critical constants and  
> datatypes are already defined in <kabi/*.h> with __kabi_ or __KABI_  
> prefixes, it should be possible to import those definitions into  
> klibc and glibc without much effort.
> 
> UML has other issues with conflicts between the native kernel headers  
> and the GLIBC-provided stubs.  It's been mentioned on the prior  
> threads about this topic that this sort of system would ease most of  
> the issues that UML runs into.
> 
> I'm working on some sample patches now which I'll try to post in a  
> few days if I get the time.

Kyle,
Do you have (recorded) or recall any constraints or requirements
on this $subject from Linus or Andrew or others?
I mean just basic big items, like "thou shalt not mix foo and bar".

I'm just looking for the basic parameters of this task.

Thanks,
---
~Randy

Re: State of userland headers

[Kyle Moffett]

On Mar 24, 2006, at 18:01:00, Randy.Dunlap wrote:
> Kyle,
> Do you have (recorded) or recall any constraints or requirements on  
> this $subject from Linus or Andrew or others?  I mean just basic  
> big items, like "thou shalt not mix foo and bar".
>
> I'm just looking for the basic parameters of this task.

Well, to a large extent he's actually wisely kind of stayed out of  
the flamewars :-D  I'm guessing he's hoping that we'll figure  
something acceptable out on our own and send him patches without him  
having to think about it much.  He has said this, though:
> On the bigger question of what to do with kernel headers in  
> general, let's just make one thing clear: the kernel headers are  
> for the kernel, and big and painful re-organizations that don't  
> help _existing_ user space are not going to happen.

He's also said this:
> On Tue, 30 Nov 2004, Alexandre Oliva wrote:
>> Then maybe this is the fundamental problem. As long as the kernel  
>> doesn't recognize that an ABI is a contract, rather than an  
>> imposition, kernel developers won't care.
>
> That's a silly analogy. Worse, it's a very flawed analogy.
>
> If you want to use a legal analogy, the ABI is not a contract, it's  
> a public _license_.
>
> Why? A contract is something that you cannot change unilaterally.  
> Clearly the kernel _can_ and will change the ABI without asking  
> every existing program for permission.
>
> In a license, you can always just _expand_ the license eventually.  
> Maybe you originally licensed for "fly-fishing for trout", and  
> later you can expand that license to say "you can also catch  
> crawfish" without impacting your existing licensees.
>
> And exactly as with an ABI, the only thing you can't do is _remove_  
> rights without getting into trouble.
>
> So get your analogies straight. The kernel ABI is _not_ a contract.

My hope for this is that we can start doing _little_ and clearly  
obvious changes that clean up header files.  Basically a lot of the  
__KERNEL__ definitions need janitorial work.  Either they need to be  
removed or they need to be put in the right places.  Also it seems  
like there's a lot of duplication between architectures; for one  
example look at all the varieties of FD_SET code in the different  
linux/include/asm-*/posix_types.h files.  That's one of the areas I'm  
trying to clean up into a single linux/fdset.h included from the  
pertinent files.  Notice how the file forcibly overrides GLIBC; I  
think if we can define it as __KABI_FD* and only define the non- 
prefixed version in __KERNEL__, then we can provide something that  
GLIBC and klibc can get for free, without having to figure out their  
own bitmaps.  Likewise some of the user-accessible types are uniform  
across architectures, and others are haphazardly arrayed, some to be  
compatible with other OSes on that architecture, others just because  
that's what the arch they copied from used.  I'm hoping if I can get  
enough small patches flowing, others will join in too and the process  
will go easier.

Cheers,
Kyle Moffett

Re: State of userland headers

[Jim Gifford]

Just to through this out, I'm working on project right now where we want 
to use the latest Kernel Headers, but since LLH is pretty much dead we 
came up with a interim solution until something else gets worked out. 
The script lists below only sanitizes the headers that are needed by 
userspace, there may be some missing, but this has worked on the systems 
I've built up to this point.

http://ftp.jg555.com/headers/headers2

-- 
----
Jim Gifford
maillist@jg555.com

Re: State of userland headers

[Jeff Dike]

On Fri, Mar 24, 2006 at 05:46:27PM -0500, Kyle Moffett wrote:
> 4)  UML runs into a lot of problems when glibc's headers and the  
> native kernel headers headers conflict.

> UML has other issues with conflicts between the native kernel headers  
> and the GLIBC-provided stubs.  It's been mentioned on the prior  
> threads about this topic that this sort of system would ease most of  
> the issues that UML runs into.

Actually, this isn't quite the same as what UML hits.  I have an
amicable solution (with some warts) to the glibc/kernel header
conflicts - files build against either glibc headers or kernel
headers, but never both.  

The warts are where I pass information between those two sets of files
that could be interpretted differently on either side, but aren't
because both sides are Linux.  For example, I freely pass errno values
across that interface in the hope that the glibc headers and the
kernel headers agree on what they mean. 

My problem with the kernel headers is that they are a mixture of
things that are usable in userspace and things that aren't.  This is
closely related, but not identical to, things which are part of the
ABI and things which aren't.

For example, the kernel locks are quite usable in userspace, but you
would never make them part of the ABI.

So, a set of KABI headers would likely make UML's headers cleaner, by
avoiding copying arch headers and using various nasty tricks to
disable objectionable pieces of headers which I steal from the arch.

So what I really want is a superset of the KABI headers, but the KABI
will give me most of what I want.

				Jeff

Re: State of userland headers

[Kyle Moffett]

On Mar 24, 2006, at 20:36:15, Jeff Dike wrote:
> On Fri, Mar 24, 2006 at 05:46:27PM -0500, Kyle Moffett wrote:
>> 4)  UML runs into a lot of problems when glibc's headers and the
>> native kernel headers headers conflict.
>>
>> UML has other issues with conflicts between the native kernel  
>> headers and the GLIBC-provided stubs.  It's been mentioned on the  
>> prior threads about this topic that this sort of system would ease  
>> most of the issues that UML runs into.
>
> Actually, this isn't quite the same as what UML hits.  My problem  
> with the kernel headers is that they are a mixture of things that  
> are usable in userspace and things that aren't.  This is closely  
> related, but not identical to, things which are part of the ABI and  
> things which aren't.
>
> For example, the kernel locks are quite usable in userspace, but  
> you would never make them part of the ABI.
>
> So, a set of KABI headers would likely make UML's headers cleaner,  
> by avoiding copying arch headers and using various nasty tricks to  
> disable objectionable pieces of headers which I steal from the arch.
>
> So what I really want is a superset of the KABI headers, but the  
> KABI will give me most of what I want.

So perhaps could we define an informal subset of the kernel code that  
works in both userspace and kernel-space and put it in include/libk?   
Stuff like linked lists, spinlocks (depends on arch, may not be  
supported), etc could be in linux/libk and linux/include/libk or  
similar, and then from there included into linux/include/linux/ 
list.h, etc, as well as into the UML files that need it.  Since the  
provider and user would both be the Linux kernel, I see no issues  
with trying to provide a stable interface of any kind, especially if  
we document it as "PRIVATE - FOR KERNEL USE ONLY!!!" with big warning  
signs. As a nice bonus, this would make it possible to implement some  
user-space unit tests of various pieces.

Cheers,
Kyle Moffett

Re: State of userland headers

[Jeff Dike]

On Sat, Mar 25, 2006 at 01:33:55AM -0500, Kyle Moffett wrote:
> So perhaps could we define an informal subset of the kernel code that  
> works in both userspace and kernel-space and put it in include/libk?   
> Stuff like linked lists, spinlocks (depends on arch, may not be  
> supported), etc could be in linux/libk and linux/include/libk or  
> similar, and then from there included into linux/include/linux/ 
> list.h, etc, as well as into the UML files that need it.  Since the  
> provider and user would both be the Linux kernel, I see no issues  
> with trying to provide a stable interface of any kind, especially if  
> we document it as "PRIVATE - FOR KERNEL USE ONLY!!!" with big warning  
> signs. As a nice bonus, this would make it possible to implement some  
> user-space unit tests of various pieces.

This would be perfect.

			Jeff

Re: State of userland headers

[Rob Landley]

On Friday 24 March 2006 5:46 pm, Kyle Moffett wrote:

> however.  Since the kabi/*.h headers would not be kernel-version-
> specific, they could be copied to a system running an older kernel
> and reused there without problems.

Since when is the kernel ABI not kernel version specific?  You can use an 
older ABI on a newer kernel, but you can't use a newer ABI on an older 
kernel.

> Even though some of the syscalls 
> and ioctls referenced in the kabi headers might not be present on the
> running kernel, portable programs are expected to be able to sanely
> handle older kernels.

At the source level, maybe.  At the binary level?  Not really.

Rob
-- 
Never bet against the cheap plastic solution.

Re: State of userland headers

[Kyle Moffett]

On Mar 24, 2006, at 22:19:54, Rob Landley wrote:
> On Friday 24 March 2006 5:46 pm, Kyle Moffett wrote:
>> however.  Since the kabi/*.h headers would not be kernel-version- 
>> specific, they could be copied to a system running an older kernel  
>> and reused there without problems.
>
> Since when is the kernel ABI not kernel version specific?  You can  
> use an older ABI on a newer kernel, but you can't use a newer ABI  
> on an older kernel.

By the very definition of "ABI", it _must_ not be kernel version  
specific.  A program written for a newer ABI and compiled against it  
_must_ be able to gracefully handle syscalls not existing when run on  
an older kernel.  The program may fail to work entirely because the  
syscalls it needs are not present, but that's no different than a  
"Kernel does not have feature CONFIG_FOO enabled" issue.

>> Even though some of the syscalls and ioctls referenced in the kabi  
>> headers might not be present on the running kernel, portable  
>> programs are expected to be able to sanely handle older kernels.
>
> At the source level, maybe.  At the binary level?  Not really.

The ABI must be compatible at both source and binary interfaces,  
"Application _Binary_ Interface" is the name, after all.  It's OK to  
introduce new features so long as you can gracefully handle them not  
being present.  Look for example at what happens if I take V4L apps  
and run them on a 2.4.0 kernel without V4L.  They report that the  
kernel is too old or missing features, but aside from V4L not being  
present, everything just works.

As a result, I think that if we can clean up the headers in kernel,  
we can reuse those same headers for compiling userspace everywhere,  
even if the userspace is going to be run on an older kernel.

Cheers,
Kyle Moffett

[RFC][PATCH 0/2] KABI example conversion and cleanup

[Kyle Moffett]

On Fri, 24 Mar 2006 17:46:27 -0500 Kyle Moffett <mrmacman_g4@mac.com> wrote:
> I'm working on some sample patches now which I'll try to post in a
> few days if I get the time.

Ok, here's a sample of the KABI conversion and cleanup patches that I'm
proposing.  I have a few fundamental goals for these patches:
1)  The Linux kernel compiles and works at every step along the way
2)  Since most of the headers are currently quite broken with respect to
    GLIBC and userspace, I won't spend much extra time preserving
    compatibility with GLIBC, userspace, or non-GCC compilers.
3)  Everything in include/kabi will have a __kabi_ or __KABI_ prefix.
4)  Headers in include/linux that need the KABI interfaces will include
    the corresponding <kabi/*.h> header and define or typedef the
    necessary KABI definitions to the names the kernel wants.
5)  The stuff in include/kabi/*.h should always be completely independent
    of userspace/kernelspace and not require any includes outside of
    <kabi/*>.  This means that the only preprocessor symbols that we can
    assume are present are those provided by the compiler itself.

Cheers,
Kyle Moffett

[RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

[Kyle Moffett]

Create initial kernel ABI header infrastructure

The model proposed for the new kernel ABI headers is that the headers
pertinent to both kernelspace and userspace would be reorganized and
cleaned out into include/kabi.  The new headers should stay in a restricted
namespace such that they are useable anywhere.  Just as include/asm is
symlinked to include/asm-$ARCH, include/kabi/arch will be symlinked to
include/kabi/arch-$ARCH to provide a straightforward way of selecting the
correct architecture ABI.

This creates the first architecture-specific header file, "stddef.h", which
implements a basic BITS_PER_LONG definition, and adds the corresponding
Makefile bits to create the include/kabi/arch symlink.

There's probably something else that needs to be done about getting the
correct include/kabi/* files in the tarball but I'm not aware of it at this
time, so I'll just pretend it doesn't exist :-D.

This appears to compile correctly with allmodconfig on powerpc (32-bit)

Signed-off-by: Kyle Moffett <mrmacman_g4@mac.com>

---
commit affadc51c2b4c96785809a352973a9f6a51a4c37
tree e0979fa864eab8902ccae459060b18f077ad1d4e
parent 36ddf5bbdea7ba4582abc62f106f0f0e9f0b6b91
author Kyle Moffett <mrmacman_g4@mac.com> Sun, 26 Mar 2006 06:18:06 -0500
committer Kyle Moffett <mrmacman_g4@mac.com> Sun, 26 Mar 2006 06:18:06 -0500

 Makefile                             |   22 +++++++++++++++++++---
 include/kabi/arch-alpha/stddef.h     |    7 +++++++
 include/kabi/arch-arm/stddef.h       |    7 +++++++
 include/kabi/arch-arm26/stddef.h     |    7 +++++++
 include/kabi/arch-cris/stddef.h      |    7 +++++++
 include/kabi/arch-frv/stddef.h       |    7 +++++++
 include/kabi/arch-h8300/stddef.h     |    7 +++++++
 include/kabi/arch-i386/stddef.h      |    7 +++++++
 include/kabi/arch-ia64/stddef.h      |    7 +++++++
 include/kabi/arch-m32r/stddef.h      |    7 +++++++
 include/kabi/arch-m68k/stddef.h      |    7 +++++++
 include/kabi/arch-m68knommu/stddef.h |    1 +
 include/kabi/arch-mips/stddef.h      |   12 ++++++++++++
 include/kabi/arch-parisc/stddef.h    |   12 ++++++++++++
 include/kabi/arch-powerpc/stddef.h   |   12 ++++++++++++
 include/kabi/arch-ppc/stddef.h       |    1 +
 include/kabi/arch-s390/stddef.h      |   12 ++++++++++++
 include/kabi/arch-sh/stddef.h        |    7 +++++++
 include/kabi/arch-sh64/stddef.h      |    7 +++++++
 include/kabi/arch-sparc/stddef.h     |    7 +++++++
 include/kabi/arch-sparc64/stddef.h   |    7 +++++++
 include/kabi/arch-v850/stddef.h      |    7 +++++++
 include/kabi/arch-x86_64/stddef.h    |    7 +++++++
 include/kabi/arch-xtensa/stddef.h    |    7 +++++++
 24 files changed, 188 insertions(+), 3 deletions(-)

diff --git a/Makefile b/Makefile
index af6210d..8e9045a 100644
--- a/Makefile
+++ b/Makefile
@@ -787,13 +787,15 @@ ifneq ($(KBUILD_SRC),)
 		/bin/false; \
 	fi;
 	$(Q)if [ ! -d include2 ]; then mkdir -p include2; fi;
+	$(Q)if [ ! -d include2/kabi ]; then mkdir -p include2/kabi; fi;
 	$(Q)ln -fsn $(srctree)/include/asm-$(ARCH) include2/asm
+	$(Q)ln -fsn $(srctree)/include/kabi/arch-$(ARCH) include2/kabi/arch
 endif
 
 # prepare2 creates a makefile if using a separate output directory
 prepare2: prepare3 outputmakefile
 
-prepare1: prepare2 include/linux/version.h include/asm \
+prepare1: prepare2 include/linux/version.h include/asm include/kabi/arch \
                    include/config/MARKER
 ifneq ($(KBUILD_MODULES),)
 	$(Q)rm -rf $(MODVERDIR)
@@ -822,6 +824,12 @@ include/asm:
 	$(Q)if [ ! -d include ]; then mkdir -p include; fi;
 	@ln -fsn asm-$(ARCH) $@
 
+include/kabi/arch:
+	@echo '  SYMLINK $@ -> include/kabi/arch-$(ARCH)'
+	$(Q)if [ ! -d include ]; then mkdir -p include; fi;
+	$(Q)if [ ! -d include/kabi ]; then mkdir -p include/kabi; fi;
+	@ln -fsn arch-$(ARCH) $@
+
 # 	Split autoconf.h into include/linux/config/*
 
 include/config/MARKER: scripts/basic/split-include include/linux/autoconf.h
@@ -944,7 +952,8 @@ CLEAN_FILES +=	vmlinux System.map \
 MRPROPER_DIRS  += include/config include2
 MRPROPER_FILES += .config .config.old include/asm .version .old_version \
                   include/linux/autoconf.h include/linux/version.h \
-		  .kernelrelease Module.symvers tags TAGS cscope*
+		  .kernelrelease Module.symvers tags TAGS cscope* \
+		  include/kabi/arch
 
 # clean - Delete most, but leave enough to build external modules
 #
@@ -1195,12 +1204,19 @@ define all-sources
 	  find $(__srctree)security/selinux/include $(RCS_FIND_IGNORE) \
 	       -name '*.[chS]' -print; \
 	  find $(__srctree)include $(RCS_FIND_IGNORE) \
-	       \( -name config -o -name 'asm-*' \) -prune \
+	       \( -name config -o -name 'asm-*' -o -name 'kabi' \) -prune \
+	       -o -name '*.[chS]' -print; \
+	  find $(__srctree)include/kabi $(RCS_FIND_IGNORE) \
+	       \( -name 'arch-*' \) -prune \
 	       -o -name '*.[chS]' -print; \
 	  for ARCH in $(ALLINCLUDE_ARCHS) ; do \
 	       find $(__srctree)include/asm-$${ARCH} $(RCS_FIND_IGNORE) \
 	            -name '*.[chS]' -print; \
 	  done ; \
+	  for ARCH in $(ALLINCLUDE_ARCHS) ; do \
+	       find $(__srctree)include/kabi/arch-$${ARCH} $(RCS_FIND_IGNORE) \
+	            -name '*.[chS]' -print; \
+	  done ; \
 	  find $(__srctree)include/asm-generic $(RCS_FIND_IGNORE) \
 	       -name '*.[chS]' -print )
 endef
diff --git a/include/kabi/arch-alpha/stddef.h b/include/kabi/arch-alpha/stddef.h
new file mode 100644
index 0000000..dd0ea38
--- /dev/null
+++ b/include/kabi/arch-alpha/stddef.h
@@ -0,0 +1,7 @@
+#ifndef  __KABI_ARCH_ALPHA_STDDEF_H
+# define __KABI_ARCH_ALPHA_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 64
+# define __KABI_BITS_PER_LONG_SHIFT 6
+
+#endif /* not __KABI_ARCH_ALPHA_STDDEF_H */
diff --git a/include/kabi/arch-arm/stddef.h b/include/kabi/arch-arm/stddef.h
new file mode 100644
index 0000000..556b13a
--- /dev/null
+++ b/include/kabi/arch-arm/stddef.h
@@ -0,0 +1,7 @@
+#ifndef  __KABI_ARCH_ARM_STDDEF_H
+# define __KABI_ARCH_ARM_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+
+#endif /* not __KABI_ARCH_ARM_STDDEF_H */
diff --git a/include/kabi/arch-arm26/stddef.h b/include/kabi/arch-arm26/stddef.h
new file mode 100644
index 0000000..74a6959
--- /dev/null
+++ b/include/kabi/arch-arm26/stddef.h
@@ -0,0 +1,7 @@
+#ifndef  __KABI_ARCH_ARM26_STDDEF_H
+# define __KABI_ARCH_ARM26_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+
+#endif /* not __KABI_ARCH_ARM26_STDDEF_H */
diff --git a/include/kabi/arch-cris/stddef.h b/include/kabi/arch-cris/stddef.h
new file mode 100644
index 0000000..ca3a965
--- /dev/null
+++ b/include/kabi/arch-cris/stddef.h
@@ -0,0 +1,7 @@
+#ifndef  __KABI_ARCH_CRIS_STDDEF_H
+# define __KABI_ARCH_CRIS_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+
+#endif /* not __KABI_ARCH_CRIS_STDDEF_H */
diff --git a/include/kabi/arch-frv/stddef.h b/include/kabi/arch-frv/stddef.h
new file mode 100644
index 0000000..6d47d06
--- /dev/null
+++ b/include/kabi/arch-frv/stddef.h
@@ -0,0 +1,7 @@
+#ifndef  __KABI_ARCH_FRV_STDDEF_H
+# define __KABI_ARCH_FRV_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+
+#endif /* not __KABI_ARCH_FRV_STDDEF_H */
diff --git a/include/kabi/arch-h8300/stddef.h b/include/kabi/arch-h8300/stddef.h
new file mode 100644
index 0000000..fc05a8e
--- /dev/null
+++ b/include/kabi/arch-h8300/stddef.h
@@ -0,0 +1,7 @@
+#ifndef  __KABI_ARCH_H8300_STDDEF_H
+# define __KABI_ARCH_H8300_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+
+#endif /* not __KABI_ARCH_H8300_STDDEF_H */
diff --git a/include/kabi/arch-i386/stddef.h b/include/kabi/arch-i386/stddef.h
new file mode 100644
index 0000000..a2006b9
--- /dev/null
+++ b/include/kabi/arch-i386/stddef.h
@@ -0,0 +1,7 @@
+#ifndef  __KABI_ARCH_I386_STDDEF_H
+# define __KABI_ARCH_I386_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+
+#endif /* not __KABI_ARCH_I386_STDDEF_H */
diff --git a/include/kabi/arch-ia64/stddef.h b/include/kabi/arch-ia64/stddef.h
new file mode 100644
index 0000000..6847a92
--- /dev/null
+++ b/include/kabi/arch-ia64/stddef.h
@@ -0,0 +1,7 @@
+#ifndef  __KABI_ARCH_IA64_STDDEF_H
+# define __KABI_ARCH_IA64_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 64
+# define __KABI_BITS_PER_LONG_SHIFT 6
+
+#endif /* not __KABI_ARCH_IA64_STDDEF_H */
diff --git a/include/kabi/arch-m32r/stddef.h b/include/kabi/arch-m32r/stddef.h
new file mode 100644
index 0000000..fdb97ad
--- /dev/null
+++ b/include/kabi/arch-m32r/stddef.h
@@ -0,0 +1,7 @@
+#ifndef  __KABI_ARCH_M32R_STDDEF_H
+# define __KABI_ARCH_M32R_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+
+#endif /* not __KABI_ARCH_M32R_STDDEF_H */
diff --git a/include/kabi/arch-m68k/stddef.h b/include/kabi/arch-m68k/stddef.h
new file mode 100644
index 0000000..b2c1490
--- /dev/null
+++ b/include/kabi/arch-m68k/stddef.h
@@ -0,0 +1,7 @@
+#ifndef  __KABI_ARCH_M68K_STDDEF_H
+# define __KABI_ARCH_M68K_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+
+#endif /* not __KABI_ARCH_M68K_STDDEF_H */
diff --git a/include/kabi/arch-m68knommu/stddef.h b/include/kabi/arch-m68knommu/stddef.h
new file mode 100644
index 0000000..f01d529
--- /dev/null
+++ b/include/kabi/arch-m68knommu/stddef.h
@@ -0,0 +1 @@
+#include <kabi/arch-m68k/stddef.h>
diff --git a/include/kabi/arch-mips/stddef.h b/include/kabi/arch-mips/stddef.h
new file mode 100644
index 0000000..318a2d1
--- /dev/null
+++ b/include/kabi/arch-mips/stddef.h
@@ -0,0 +1,12 @@
+#ifndef  __KABI_ARCH_MIPS_STDDEF_H
+# define __KABI_ARCH_MIPS_STDDEF_H 1
+
+# if _MIPS_SZLONG == 64
+#  define __KABI_BITS_PER_LONG 64
+#  define __KABI_BITS_PER_LONG_SHIFT 6
+# else
+#  define __KABI_BITS_PER_LONG 32
+#  define __KABI_BITS_PER_LONG_SHIFT 5
+# endif
+
+#endif /* not __KABI_ARCH_MIPS_STDDEF_H */
diff --git a/include/kabi/arch-parisc/stddef.h b/include/kabi/arch-parisc/stddef.h
new file mode 100644
index 0000000..ae24ae9
--- /dev/null
+++ b/include/kabi/arch-parisc/stddef.h
@@ -0,0 +1,12 @@
+#ifndef  __KABI_ARCH_PARISC_STDDEF_H
+# define __KABI_ARCH_PARISC_STDDEF_H 1
+
+# ifdef __LP64__
+#  define __KABI_BITS_PER_LONG 64
+#  define __KABI_BITS_PER_LONG_SHIFT 6
+# else
+#  define __KABI_BITS_PER_LONG 32
+#  define __KABI_BITS_PER_LONG_SHIFT 5
+# endif
+
+#endif /* not __KABI_ARCH_PARISC_STDDEF_H */
diff --git a/include/kabi/arch-powerpc/stddef.h b/include/kabi/arch-powerpc/stddef.h
new file mode 100644
index 0000000..f9acee1
--- /dev/null
+++ b/include/kabi/arch-powerpc/stddef.h
@@ -0,0 +1,12 @@
+#ifndef  __KABI_ARCH_POWERPC_STDDEF_H
+# define __KABI_ARCH_POWERPC_STDDEF_H 1
+
+# ifdef __powerpc64__
+#  define __KABI_BITS_PER_LONG 64
+#  define __KABI_BITS_PER_LONG_SHIFT 6
+# else
+#  define __KABI_BITS_PER_LONG 32
+#  define __KABI_BITS_PER_LONG_SHIFT 5
+# endif
+
+#endif /* not __KABI_ARCH_POWERPC_STDDEF_H */
diff --git a/include/kabi/arch-ppc/stddef.h b/include/kabi/arch-ppc/stddef.h
new file mode 100644
index 0000000..a244f18
--- /dev/null
+++ b/include/kabi/arch-ppc/stddef.h
@@ -0,0 +1 @@
+#include <kabi/arch-powerpc/stddef.h>
diff --git a/include/kabi/arch-s390/stddef.h b/include/kabi/arch-s390/stddef.h
new file mode 100644
index 0000000..d1f26b0
--- /dev/null
+++ b/include/kabi/arch-s390/stddef.h
@@ -0,0 +1,12 @@
+#ifndef  __KABI_ARCH_S390_STDDEF_H
+# define __KABI_ARCH_S390_STDDEF_H 1
+
+# ifdef __s390x__
+#  define __KABI_BITS_PER_LONG 64
+#  define __KABI_BITS_PER_LONG_SHIFT 6
+# else
+#  define __KABI_BITS_PER_LONG 32
+#  define __KABI_BITS_PER_LONG_SHIFT 5
+# endif
+
+#endif /* not __KABI_ARCH_S390_STDDEF_H */
diff --git a/include/kabi/arch-sh/stddef.h b/include/kabi/arch-sh/stddef.h
new file mode 100644
index 0000000..d0f81dd
--- /dev/null
+++ b/include/kabi/arch-sh/stddef.h
@@ -0,0 +1,7 @@
+#ifndef  __KABI_ARCH_SH_STDDEF_H
+# define __KABI_ARCH_SH_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+
+#endif /* not __KABI_ARCH_SH_STDDEF_H */
diff --git a/include/kabi/arch-sh64/stddef.h b/include/kabi/arch-sh64/stddef.h
new file mode 100644
index 0000000..5d00497
--- /dev/null
+++ b/include/kabi/arch-sh64/stddef.h
@@ -0,0 +1,7 @@
+#ifndef  __KABI_ARCH_SH64_STDDEF_H
+# define __KABI_ARCH_SH64_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+
+#endif /* not __KABI_ARCH_SH64_STDDEF_H */
diff --git a/include/kabi/arch-sparc/stddef.h b/include/kabi/arch-sparc/stddef.h
new file mode 100644
index 0000000..e00c10c
--- /dev/null
+++ b/include/kabi/arch-sparc/stddef.h
@@ -0,0 +1,7 @@
+#ifndef  __KABI_ARCH_SPARC_STDDEF_H
+# define __KABI_ARCH_SPARC_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+
+#endif /* not __KABI_ARCH_SPARC_STDDEF_H */
diff --git a/include/kabi/arch-sparc64/stddef.h b/include/kabi/arch-sparc64/stddef.h
new file mode 100644
index 0000000..5125f4b
--- /dev/null
+++ b/include/kabi/arch-sparc64/stddef.h
@@ -0,0 +1,7 @@
+#ifndef  __KABI_ARCH_SPARC64_STDDEF_H
+# define __KABI_ARCH_SPARC64_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 64
+# define __KABI_BITS_PER_LONG_SHIFT 6
+
+#endif /* not __KABI_ARCH_SPARC64_STDDEF_H */
diff --git a/include/kabi/arch-v850/stddef.h b/include/kabi/arch-v850/stddef.h
new file mode 100644
index 0000000..84328e5
--- /dev/null
+++ b/include/kabi/arch-v850/stddef.h
@@ -0,0 +1,7 @@
+#ifndef  __KABI_ARCH_V850_STDDEF_H
+# define __KABI_ARCH_V850_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+
+#endif /* not __KABI_ARCH_V850_STDDEF_H */
diff --git a/include/kabi/arch-x86_64/stddef.h b/include/kabi/arch-x86_64/stddef.h
new file mode 100644
index 0000000..e20e620
--- /dev/null
+++ b/include/kabi/arch-x86_64/stddef.h
@@ -0,0 +1,7 @@
+#ifndef  __KABI_ARCH_X86_64_STDDEF_H
+# define __KABI_ARCH_X86_64_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 64
+# define __KABI_BITS_PER_LONG_SHIFT 6
+
+#endif /* not __KABI_ARCH_X86_64_STDDEF_H */
diff --git a/include/kabi/arch-xtensa/stddef.h b/include/kabi/arch-xtensa/stddef.h
new file mode 100644
index 0000000..672c19f
--- /dev/null
+++ b/include/kabi/arch-xtensa/stddef.h
@@ -0,0 +1,7 @@
+#ifndef  __KABI_ARCH_XTENSA_STDDEF_H
+# define __KABI_ARCH_XTENSA_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+
+#endif /* not __KABI_ARCH_XTENSA_STDDEF_H */

Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

[Arjan van de Ven]

On Sun, 2006-03-26 at 06:54 -0500, Kyle Moffett wrote:
> Create initial kernel ABI header infrastructure


it's nice that you picked this one;
for this you want an arch-generic/stddef32.h and stddef64.h

and have arch-foo just only include the proper generic one..

(and... why do you prefix these with _KABI? that's a mistake imo. Don't
both with that. Really. Either these need exporting to userspace, but
then either use __ as prefix or don't use a prefix. But KABI.. No.)

Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

[Kyle Moffett]

On Mar 26, 2006, at 07:32:31, Arjan van de Ven wrote:
> On Sun, 2006-03-26 at 06:54 -0500, Kyle Moffett wrote:
>> Create initial kernel ABI header infrastructure
>
> it's nice that you picked this one;
> for this you want an arch-generic/stddef32.h and stddef64.h
>
> and have arch-foo just only include the proper generic one..

I plan to add a lot of other definitions to this file later on.  For  
example different architectures have different notions of what a  
__kernel_ino_t is (unsigned int versus unsigned long).  I may rename  
this file as types.h, but from looking through the code I figure I'll  
have enough general purpose declarations about "This architecture has  
blah" that a separate stddef.h file will be worth it.

> (and... why do you prefix these with _KABI? that's a mistake imo.  
> Don't bother with that. Really. Either these need exporting to  
> userspace, but then either use __ as prefix or don't use a prefix.  
> But KABI.. No.)

According to the various standards all symbols beginning with __ are  
reserved for "The Implementation", including the compiler, the  
standard library, the kernel, etc.  In order to avoid clashing with  
any/all of those, I picked the __KABI_ and __kabi_ prefixes for  
uniqueness.  In theory I could just use __, but there are problems  
with that too.  For example, note how the current compiler.h files  
redefine __always_inline to mean something kinda different.  The GCC  
manual says we should be able to write this:

inline __attribute__((__always_inline)) int increment(int x)
{
	return x+1;
}

Except when compiling the kernel headers turn that into this (which  
obviously doesn't compile):
inline __attribute__((__attribute__((always_inline)))) int increment 
(int x)
{
	return x+1;
}

As a result, I kinda want to stay away from anything that remotely  
looks like a conflicting namespace.  Using such a unique namespace  
means we can also safely do this if necessary (Since you can't  
"typedef struct foo struct bar"):

kabi/foo.h:
   struct __kabi_foo {
   	int x;
   	int y;
   };

linux/foo.h:
   #define __kabi_foo foo
   #include <kabi/foo.h>

drivers/foo/foo.h:
   #include <linux/foo.h>
   void func()
   {
   	struct foo = { .x = 1, .y = 2 };
   }

Cheers,
Kyle Moffett

Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

[Martin Mares]

Hello!

> As a result, I kinda want to stay away from anything that remotely  
> looks like a conflicting namespace.  Using such a unique namespace  
> means we can also safely do this if necessary (Since you can't  
> "typedef struct foo struct bar"):
> 
> kabi/foo.h:
>   struct __kabi_foo {
>   	int x;
>   	int y;
>   };
> 
> linux/foo.h:
>   #define __kabi_foo foo
>   #include <kabi/foo.h>

This looks very fragile -- <kabi/foo.h> can be included earlier by another
header.

				Have a nice fortnight
-- 
Martin `MJ' Mares   <mj@ucw.cz>   http://atrey.karlin.mff.cuni.cz/~mj/
Faculty of Math and Physics, Charles University, Prague, Czech Rep., Earth
And God said: E = 1/2mv^2 - Ze^2/r ...and there *WAS* light!

Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

[Kyle Moffett]

On Mar 26, 2006, at 07:59:26, Martin Mares wrote:
> Hello!
>
>> As a result, I kinda want to stay away from anything that remotely  
>> looks like a conflicting namespace.  Using such a unique namespace  
>> means we can also safely do this if necessary (Since you can't  
>> "typedef struct foo struct bar"):
>>
>> kabi/foo.h:
>>   struct __kabi_foo {
>>   	int x;
>>   	int y;
>>   };
>>
>> linux/foo.h:
>>   #define __kabi_foo foo
>>   #include <kabi/foo.h>
>
> This looks very fragile -- <kabi/foo.h> can be included earlier by  
> another header.

It _is_ fragile, but for a number of POSIX-defined structs that's  
actually the only way to do it without duplicating the data structure  
in entirety, unless the GCC people can implement a "typedef struct  
foo struct bar;"  Hopefully it would be a relatively rare occurrence  
and carefully thought out in any case.  We may end up with some  
duplication regardless :-\.

Cheers,
Kyle Moffett

Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

[Martin Mares]

Hello!

> It _is_ fragile, but for a number of POSIX-defined structs that's  
> actually the only way to do it without duplicating the data structure  
> in entirety, unless the GCC people can implement a "typedef struct  
> foo struct bar;"

Actually, something like that can be achieved using anonymous structure
members:

struct xxx {
	struct yyy;
};

However, I'm not sure how old versions of GCC support this feature.

				Have a nice fortnight
-- 
Martin `MJ' Mares   <mj@ucw.cz>   http://atrey.karlin.mff.cuni.cz/~mj/
Faculty of Math and Physics, Charles University, Prague, Czech Rep., Earth
And God said: E = 1/2mv^2 - Ze^2/r ...and there *WAS* light!

Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

[Kyle Moffett]

On Mar 26, 2006, at 10:38:59, Martin Mares wrote:
>> It _is_ fragile, but for a number of POSIX-defined structs that's  
>> actually the only way to do it without duplicating the data  
>> structure in entirety, unless the GCC people can implement a  
>> "typedef struct foo struct bar;"
>
> Actually, something like that can be achieved using anonymous  
> structure members:
>
> struct xxx {
> 	struct yyy;
> };

Oh, if only that worked.  Actually, what happens is the "struct yyy;"  
declaration inside of struct xxx looks just like "struct yyy;" out in  
the middle of some random header file.  It predeclares the existence  
of a struct yyy and does nothing else.

For instance, the following sample program:
   struct foo {
   	int a;
   	int b;
   };

   struct bar {
   	struct foo;
   };

   int main()
   {
   	struct foo myfoo = { .a = 1, .b = 2 };
   	struct bar mybar = { .a = 1, .b = 2 };
   	return 0;
   }

Compiled like this:
   gcc mytest.c -o mytest

Generates these errors:
   mytest.c:7: warning: declaration does not declare anything
   mytest.c: In function `main':
   mytest.c:12: error: unknown field `a' specified in initializer
   mytest.c:12: warning: excess elements in struct initializer
   mytest.c:12: warning: (near initialization for `mybar')
   mytest.c:12: error: unknown field `b' specified in initializer
   mytest.c:12: warning: excess elements in struct initializer
   mytest.c:12: warning: (near initialization for `mybar')

Cheers,
Kyle Moffett

Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

[Arjan van de Ven]

> 
> According to the various standards all symbols beginning with __ are  
> reserved for "The Implementation", including the compiler, the  
> standard library, the kernel, etc.  In order to avoid clashing with  
> any/all of those, I picked the __KABI_ and __kabi_ prefixes for  
> uniqueness.  In theory I could just use __, but there are problems  
> with that too.  For example, note how the current compiler.h files  
> redefine __always_inline to mean something kinda different.

well... the "problem" is there today, and... it's not much of a problem
if at all; there's just a few simple rules to keep it that way which
seem to wkr.

And your __alway_inline example.. that's something that really is kernel
internal and shouldn't be exposed to userland. 


I think the "problem" really is not there if
1) we only use __ symbols like we do today for non-structs
2) avoid including kernel headers in kernel headers as far as possible.
   This means, that if an application wants to use MTD struct 
   "struct mtd_foo" it will have to include the MTD header, but that
   he otherwise never gets it. Eg all such symbols are in a "Yes I
   really want it" header.
3) keep the userspace exposed stuff as small as reasonable. Your
   __always_inline example doesn't make that cut. A struct used as
   ioctl of a subsystem/driver in the header specially for that 
   subsystem/driver does.

Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

[Kyle Moffett]

On Mar 26, 2006, at 09:39:28, Arjan van de Ven wrote:
>> According to the various standards all symbols beginning with __  
>> are reserved for "The Implementation", including the compiler, the  
>> standard library, the kernel, etc.  In order to avoid clashing  
>> with any/all of those, I picked the __KABI_ and __kabi_ prefixes  
>> for uniqueness.  In theory I could just use __, but there are  
>> problems with that too.  For example, note how the current  
>> compiler.h files redefine __always_inline to mean something kinda  
>> different.
>
> well... the "problem" is there today, and... it's not much of a  
> problem if at all; there's just a few simple rules to keep it that  
> way which seem to work.

The current rules work for things kept completely private to the  
kernel.  If we start exporting things into the guts of libc, we can  
expect that those guys have _also_ been using all sorts of double- 
underscore symbols in conflicting ways.

> And your __alway_inline example.. that's something that really is  
> kernel internal and shouldn't be exposed to userland.

Yes, but it illustrates how a supposedly-innocuous double-underscore  
symbol actually conflicts with a GCC builtin, and that's just GCC!   
That's not including the massive "The Implementation" codebase that  
is GLIBC.

> 2) avoid including kernel headers in kernel headers as far as  
> possible.  This means, that if an application wants to use MTD  
> struct "struct mtd_foo" it will have to include the MTD header, but  
> that he otherwise never gets it. Eg all such symbols are in a "Yes  
> I really want it" header.

This is Hard(TM).  Take a look at the interdependent mess which is  
kernel.h, or sched.h.  Feel free to submit patches :-D  That's  
actually part of the reason why I'm trying this kabi/*.h bit.  I  
think that if I can pull out userspace-required struct definitions  
from the rest of the code, we'll notice that some of the header files  
don't need as many dependencies anymore and can be cleaned out a bit.

I've also been looking at GCC's "precompiled header" stuff.  I think  
that a single kabi.h file that includes all other kabi headers,  
precompiled to "kabi.h.gch", could potentially speed up a portion of  
the linux kernel build by removing a lot of text parsing.  I can't  
tell for sure till I've cleaned up more crap and done benchmarks, but  
I'm willing to give it a shot. :-D

Cheers,
Kyle Moffett

Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

[Pavel Machek]

Hi!

> I plan to add a lot of other definitions to this file 
> later on.  For  example different architectures have 
> different notions of what a  __kernel_ino_t is (unsigned 
> int versus unsigned long).  I may rename  this file as 
> types.h, but from looking through the code I figure I'll  
> have enough general purpose declarations about "This 
> architecture has  blah" that a separate stddef.h file 
> will be worth it.
> 
> >(and... why do you prefix these with _KABI? that's a 
> >mistake imo.  Don't bother with that. Really. Either 
> >these need exporting to  userspace, but then either use 
> >__ as prefix or don't use a prefix.  But KABI.. No.)
> 
> According to the various standards all symbols beginning 
> with __ are  reserved for "The Implementation", including 
> the compiler, the  standard library, the kernel, etc.  In 
> order to avoid clashing with  any/all of those, I picked 
> the __KABI_ and __kabi_ prefixes for  uniqueness.  In 
> theory I could just use __, but there are problems  with 
> that too.  For example, note how the current compiler.h 
> files  redefine __always_inline to mean something kinda 
> different.  The GCC  manual says we should be able to 
> write this:

__KABI_ everywhere will just make your headers totally unreadable.
Please don't do that.

-- 
Thanks, Sharp!

Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

[Randy.Dunlap]

On Wed, 29 Mar 2006 22:26:41 +0000 Pavel Machek wrote:

> Hi!
> 
> > I plan to add a lot of other definitions to this file 
> > later on.  For  example different architectures have 
> > different notions of what a  __kernel_ino_t is (unsigned 
> > int versus unsigned long).  I may rename  this file as 
> > types.h, but from looking through the code I figure I'll  
> > have enough general purpose declarations about "This 
> > architecture has  blah" that a separate stddef.h file 
> > will be worth it.
> > 
> > >(and... why do you prefix these with _KABI? that's a 
> > >mistake imo.  Don't bother with that. Really. Either 
> > >these need exporting to  userspace, but then either use 
> > >__ as prefix or don't use a prefix.  But KABI.. No.)
> > 
> > According to the various standards all symbols beginning 
> > with __ are  reserved for "The Implementation", including 
> > the compiler, the  standard library, the kernel, etc.  In 
> > order to avoid clashing with  any/all of those, I picked 
> > the __KABI_ and __kabi_ prefixes for  uniqueness.  In 
> > theory I could just use __, but there are problems  with 
> > that too.  For example, note how the current compiler.h 
> > files  redefine __always_inline to mean something kinda 
> > different.  The GCC  manual says we should be able to 
> > write this:
> 
> __KABI_ everywhere will just make your headers totally unreadable.
> Please don't do that.

Ack, I agree.

---
~Randy

Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

[Kyle Moffett]

On Apr 1, 2006, at 19:22:13, Randy.Dunlap wrote:
> On Wed, 29 Mar 2006 22:26:41 +0000 Pavel Machek wrote:
>>> I plan to add a lot of other definitions to this file later on.   
>>> For  example different architectures have
>>> different notions of what a  __kernel_ino_t is (unsigned int  
>>> versus unsigned long).  I may rename  this file as types.h, but  
>>> from looking through the code I figure I'll have enough general  
>>> purpose declarations about "This architecture has  blah" that a  
>>> separate stddef.h file will be worth it.
>>>
>>>> (and... why do you prefix these with _KABI? that's a mistake  
>>>> imo.  Don't bother with that. Really. Either these need  
>>>> exporting to  userspace, but then either use __ as prefix or  
>>>> don't use a prefix.  But KABI.. No.)
>>>
>>> According to the various standards all symbols beginning with __  
>>> are  reserved for "The Implementation", including the compiler,  
>>> the  standard library, the kernel, etc.  In order to avoid  
>>> clashing with  any/all of those, I picked the __KABI_ and __kabi_  
>>> prefixes for  uniqueness.  In theory I could just use __, but  
>>> there are problems  with that too.  For example, note how the  
>>> current compiler.h files  redefine __always_inline to mean  
>>> something kinda different.  The GCC manual says we should be able  
>>> to write this:
>>
>> __KABI_ everywhere will just make your headers totally  
>> unreadable.  Please don't do that.
>
> Ack, I agree.

Let me reiterate two facts:

(1)  The various C standards state that the implementation should  
restrict itself to symbols prefixed with "__", everything else is  
reserved for user code (Including symbols prefixed with a single  
underscore).
(2)  GCC predefines a large collection of symbols, macros, and  
functions for its own use, and this set is not constant (just look at  
the number of new __-prefixed symbols added between GCC 3 and 4.  In  
addition, we're not just compiling this code under GCC, but people  
will also be using it (hopefully unmodified) under tiny-cc, intel-cc,  
PGI, PathScale, Lahey, ARM Ltd, lcc, and possibly others.  It  
probably does not need to be stated that for something as userspace- 
sensitive as the KABI headers we should not risk colliding with  
predefined builtins in any of those compilers.

So my question to the list is this:
Can you come up with any way other than using a "__kabi_" prefix to  
reasonably avoid namespace collisions with that large list of  
compilers?  If you have some way, I'd be interested to hear it, but  
as a number of those compilers are commercial I'd have no way to test  
on them (and I suspect most people on this list would not either).

Of course, if the general consensus is that supporting non-GCC is not  
important, then that's ok with me.  Judging from the number of  
negative responses my earlier "[OT] Non-GCC compilers used for linux  
userspace" got, however, that doesn't seem to be the case.

Thanks for the advice!

Cheers,
Kyle Moffett

Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

[Arjan van de Ven]

On Sat, 2006-04-01 at 21:42 -0500, Kyle Moffett wrote:

> (1)  The various C standards state that the implementation should  
> restrict itself to symbols prefixed with "__", everything else is  
> reserved for user code (Including symbols prefixed with a single  
> underscore).

user code in this context includes, in my interpretation, headers for
specific ioctl structures and such (eg direct included headers; headers
that those headers include are a different matter) that the user wants.

Which is the vast majority of the kernel abi.
Exceptions are things like __u64 which are almost always "indirectly"
included. Eg the app wants "struct foo_ioctl" and foo_ioctl happens to
contain an __u64 which needs types.h


> (2)  GCC predefines a large collection of symbols, macros, and  
> functions for its own use, and this set is not constant (just look at  
> the number of new __-prefixed symbols added between GCC 3 and 4.  In  
> addition, we're not just compiling this code under GCC, but people  
> will also be using it (hopefully unmodified) under tiny-cc, intel-cc,  
> PGI, PathScale, Lahey, ARM Ltd, lcc, and possibly others.  It  
> probably does not need to be stated that for something as userspace- 
> sensitive as the KABI headers we should not risk colliding with  
> predefined builtins in any of those compilers.

the good news is that as kernel we have SOME power ;)
people who make compilers like that stay away from symbols the kernel
defines. Especially gcc is very careful in its namespace use and tends
to be very different from what linux kernel uses

> So my question to the list is this:
> Can you come up with any way other than using a "__kabi_" prefix to  
> reasonably avoid namespace collisions with that large list of  
> compilers?

First of all be realistic. Don't do silly things for places where it
doesn't matter. Again the ioctl structs come to mind.
Second, names the kernel ALREADY claims are of course free to use as
well; all those compilers ALREADY stay away from those.

What is left if you take those two big ones out of the picture?


> Of course, if the general consensus is that supporting non-GCC is not  
> important, then that's ok with me.  Judging from the number of  
> negative responses my earlier "[OT] Non-GCC compilers used for linux  
> userspace" got, however, that doesn't seem to be the case.

there is another dynamic; those other compilers in general try to
emulate gcc to a large degree, usually a very large degree. So the
"problem" you see is a lot smaller than you think.

Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

[Kyle Moffett]

On Apr 1, 2006, at 22:01:06, Arjan van de Ven wrote:
> On Sat, 2006-04-01 at 21:42 -0500, Kyle Moffett wrote:
>
>> (1)  The various C standards state that the implementation should  
>> restrict itself to symbols prefixed with "__", everything else is  
>> reserved for user code (Including symbols prefixed with a single  
>> underscore).
>
> user code in this context includes, in my interpretation, headers  
> for specific ioctl structures and such (eg direct included headers;  
> headers that those headers include are a different matter) that the  
> user wants.
>
> Which is the vast majority of the kernel abi. Exceptions are things  
> like __u64 which are almost always "indirectly" included. Eg the  
> app wants "struct foo_ioctl" and foo_ioctl happens to contain an  
> __u64 which needs types.h

I get the impression that you're confusing my KABI proposal with  
kernel-only code.  The point of KABI is to generalize some of the  
headers so that they work both in userspace and kernelspace, even if  
it means that the headers themselves aren't as clean as they might  
otherwise be.  In the kernel we have only one API where we have to  
deal with compatibility: the kernel/userspace boundary.  I'd like to  
strictly define all of that compatibility within a single set of  
_upstream_ headers in linux/include/linux-kabi or similar.

But returning to your statements above, user code in that (C  
standards) context means anything not standard-specified.  Basically,  
from userspace, a program that includes _any_ standard-specified  
header must bring in *only* those symbols specified by the standard  
as being present in that header, as well as any implementation- 
defined double-underscore symbols.  So unless I "#include  
<stdint.h>", my user program can typedef uint32_t to anything it  
wants, even a struct, if it likes.  I've actually done that before on  
a very strange Cray platform with non-GCC compiler and OS that only  
supported 64-bit integers (a char was a 64-bit integer with ops to  
mask out the upper 56 bits), just to make bzip2 work as expected.   
The same rules apply for _every_ other standard-specified header  
(stdlib.h, stdio.h, sys/types.h, etc).

> the good news is that as kernel we have SOME power ;) people who  
> make compilers like that stay away from symbols the kernel defines.  
> Especially gcc is very careful in its namespace use and tends to be  
> very different from what linux kernel uses.

As I said above, I'm talking about the proposed kernel ABI headers  
here.  The whole point of these is something that both the kernel  
_and_ userspace can include.  This means that the headers must allow  
a libc that includes them to be completely standards-compliant.  It  
also means that these headers must work not only for GCC, but also  
for tcc and icc and the myriad of other compilers I mentioned (the  
ones that I got so many emails about in the "Non-GCC compilers" thread).

>> So my question to the list is this:  Can you come up with any way  
>> other than using a "__kabi_" prefix to   reasonably avoid  
>> namespace collisions with that large list of compilers?
>
> First of all be realistic. Don't do silly things for places where  
> it doesn't matter. Again the ioctl structs come to mind. Second,  
> names the kernel ALREADY claims are of course free to use as well;  
> all those compilers ALREADY stay away from those.

Agreed (to some extent).  I'm still evaluating a lot of the code, I  
plan to submit several non-KABI cleanups first to prepare the way.   
I'll try to get you specific patches so you can give me specific  
rejections, ok? :-D  I have the feeling that we both see things the  
same way but have difficulty communicating due to the lack of examples.

> What is left if you take those two big ones out of the picture?

First of all, there seems to be a lot of little bits of code that the  
KABI headers really want to use but can't right now.  For example,  
the asm-*/bitops.h headers contain inline functions for a variety of  
nonatomic bitops that are _identical_ to the FD_SET code implemented  
in asm-*/posix-types.h.  Likewise it appears there are other standard- 
specified bitmask manipulation functions that KABI wants to implement  
using those.  Unfortunately those bitmask functions are only  
available in kernel space, although a few architectures were having  
extensive fun with header inclusion loops or implicit inclusion just  
to get access to them.  Unfortunately this made it impossible to use  
those headers in userspace.

It would be really nice if some of that architecture-specific  
duplication could be consolidated into a single header file useable  
from the userspace-pertinent KABI headers _and_ directly from kernel  
code.

> there is another dynamic; those other compilers in general try to  
> emulate gcc to a large degree, usually a very large degree. So the  
> "problem" you see is a lot smaller than you think.

Please read in detail the responses to the non-gcc compiler thread  
that I mentioned before.  Apparently a number of the compilers only  
take care to conform to the C89 or C99 specifications, which state  
that such symbols are perfectly ok for them to use, and as a result  
such "Just like GCC" behavior cannot be relied upon for kernel/ 
userspace-ABI headers.

Cheers,
Kyle Moffett

Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

[Arjan van de Ven]

On Sun, 2006-04-02 at 00:53 -0500, Kyle Moffett wrote:
> deal with compatibility: the kernel/userspace boundary.  I'd like to  
> strictly define all of that compatibility within a single set of  
> _upstream_ headers in linux/include/linux-kabi or similar.

yes that was what I was hoping you meant

> 
> But returning to your statements above, user code in that (C  
> standards) context means anything not standard-specified. 

I think you misread the C standard then. This only applies when you ONLY
include the headers defined in the standard, and no others.

>  Basically,  
> from userspace, a program that includes _any_ standard-specified  
> header must bring in *only* those symbols specified by the standard  
> as being present in that header, as well as any implementation- 
> defined double-underscore symbols.  

if you follow this argument then ANY library, be it glibc or gtk or
mysql or .. can only add __ things. That is not the case. Think of
"non-indirect" kernel header includes as including a header for a
library.

Maybe an example is needed: include/mtd/mtd-abi.h  
Well it's a slightly bad example since it uses stdint.h types not __u64
types. But assume it used __u64 types, then this header would be
perfectly fine, including from a standards point of view.



> So unless I "#include  
> <stdint.h>", my user program can typedef uint32_t to anything it  
> wants, even a struct, if it likes. 

yes this the example where __u64 needs underscores, and headers need to
use __u64 as they have to today.


>  I've actually done that before on  
> a very strange Cray platform with non-GCC compiler and OS that only  
> supported 64-bit integers (a char was a 64-bit integer with ops to  
> mask out the upper 56 bits), just to make bzip2 work as expected.   
> The same rules apply for _every_ other standard-specified header  
> (stdlib.h, stdio.h, sys/types.h, etc).

exactly. Nowhere is linux/* or asm/* in the standards as specified.


> to get access to them.  Unfortunately this made it impossible to use  
> those headers in userspace.

ok now I'm confused. How is providing FUNCTIONALITY such as setting
bitops the right thing for such headers? I would argue that the clean
headers should absolutely NOT do that but only provide the INTERFACE.

(this is also there for a license angle btw, if you actually start to
provide large bodies of code this way the lawyers will get unhappy wrt
the GPL; it's a LOT better to strictly do interface things only, eg
types, defines and prototypes)


In general:
I think you're focusing WAY too much on the __u64 kind of thing, which
is the vast minority of the ABI work, the ioctl structs are the big one.
For the __u64 kind of thing I would strongly suggest that you stick to
the currently already used symbols; anything existing knows to stay
clear of those already, so things are fine there so lets keep it simple
for those.

Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

[Pavel Machek]

Hi!

> >>__KABI_ everywhere will just make your headers totally  
> >>unreadable.  Please don't do that.
> >
> >Ack, I agree.
> 
> Let me reiterate two facts:
> 
> (1)  The various C standards state that the implementation should  
> restrict itself to symbols prefixed with "__", everything else is  
> reserved for user code (Including symbols prefixed with a single  
> underscore).
> (2)  GCC predefines a large collection of symbols, macros, and  
> functions for its own use, and this set is not constant (just look at  
> the number of new __-prefixed symbols added between GCC 3 and 4.  In  
> addition, we're not just compiling this code under GCC, but people  
> will also be using it (hopefully unmodified) under tiny-cc, intel-cc,  
> PGI, PathScale, Lahey, ARM Ltd, lcc, and possibly others.  It  
> probably does not need to be stated that for something as userspace- 
> sensitive as the KABI headers we should not risk colliding with  
> predefined builtins in any of those compilers.
> 
> So my question to the list is this:
> Can you come up with any way other than using a "__kabi_" prefix to  
> reasonably avoid namespace collisions with that large list of  
> compilers?  If you have some way, I'd be interested to hear it, but  
> as a number of those compilers are commercial I'd have no way to test  
> on them (and I suspect most people on this list would not either).

No, you should just not care about anything but
gcc. intel-cc-version-0.3.2.1.2.5 could use __kabi_struct_dirent or
whatever, and collide anyway. By adding __kabi you just make it less
likely.

I believe __ is enough. If there's one conflict with some obscure
compiler, we can simply fix the conflict (or even fix the compiler
:-).

If you feel __ is too dangerous, you may go __k ... It will not look
as ugly as __kabi_ , and should be very safe.
								Pavel

-- 
Picture of sleeping (Linux) penguin wanted...

Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

[Kyle Moffett]

On Apr 2, 2006, at 06:32:23, Pavel Machek wrote:
>> So my question to the list is this:
>> Can you come up with any way other than using a "__kabi_" prefix  
>> to reasonably avoid namespace collisions with that large list of  
>> compilers?  If you have some way, I'd be interested to hear it,  
>> but as a number of those compilers are commercial I'd have no way  
>> to test on them (and I suspect most people on this list would not  
>> either).
>
> No, you should just not care about anything but gcc. intel-cc- 
> version-0.3.2.1.2.5 could use __kabi_struct_dirent or whatever, and  
> collide anyway. By adding __kabi you just make it less likely.

At worst it would just go from "struct dirent" to "struct  
__kabi_dirent".  One reason for this distinction as I believe was  
highlighted in another email was so that eventually if necessary libc  
could export a "struct dirent" not the same as the kernel one, and  
translate between them internally.  That would be difficult or  
impossible now, given the way the kernel exports "struct dirent"  
directly.  I don't remember the specific case where this would have  
been convenient, but I seem to recall it was mentioned in one of the  
earlier iterations of this thread.

> I believe __ is enough. If there's one conflict with some obscure  
> compiler, we can simply fix the conflict (or even fix the  
> compiler :-).
>
> If you feel __ is too dangerous, you may go __k ... It will not  
> look as ugly as __kabi_ , and should be very safe.

I still disagree with you on this point, but I'll save the arguments  
for when I have some submittable patches I'd like to get feedback  
on.  I'm also fairly positive that in comparison to the ugliness in  
some of the necessary C89-compatibility macros, the __kabi_ prefix  
would be insignificant, but let's leave that discussion for another  
time as well.

Cheers,
Kyle Moffett

In any case, for reference, here are a few of the specific arguments  
for support for other compilers:

On Mar 28, 2006, at 12:28:47, Daniel Jacobowitz wrote:
> If you want glibc to ever include these things, they had better be  
> portable C and work without GCC.  Otherwise it's a non-starter.   
> Only GCC may be used to build glibc, but it deliberately supports  
> any conforming C compiler to build userspace code.

On Mar 28, 2006, at 12:56:27, Jesper Juhl wrote:
> Other compilers do exist.
>
> Over the years I've personally used a few to compile userspace apps
> for different projects (though never for compiling the kernel).
>
> Some of the compilers I have personally used for userspace apps on  
> Linux include: gcc, icc, lcc, tcc
> Others that I know of but have never used include: sdcc, Compaq C  
> for Linux, Open Watcom, vacpp, XL C/C++
>
> and I'm sure many more exist...

Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

[Sam Ravnborg]

On Sun, Mar 26, 2006 at 06:54:16AM -0500, Kyle Moffett wrote:
> Create initial kernel ABI header infrastructure
> 
> 
> diff --git a/Makefile b/Makefile
> index af6210d..8e9045a 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -787,13 +787,15 @@ ifneq ($(KBUILD_SRC),)
>  		/bin/false; \
>  	fi;
>  	$(Q)if [ ! -d include2 ]; then mkdir -p include2; fi;
> +	$(Q)if [ ! -d include2/kabi ]; then mkdir -p include2/kabi; fi;
>  	$(Q)ln -fsn $(srctree)/include/asm-$(ARCH) include2/asm
> +	$(Q)ln -fsn $(srctree)/include/kabi/arch-$(ARCH) include2/kabi/arch


No - we do not want another symlink.
Create something like:
include/i386/kabi-asm/  <= i386 specific files
include/kabi/           <= general files

Then we can do:
LINUXINCLUDE += -Iinclude/kabi-$(ARCH)
And the following would work like expected:
#include <kabi/foo.h>
#include <kabi-asm/foo.h>


But this leaves all existing users in the dark cold.
So a more involved approach could be to tkae the opposite approach.
To dedicate an area for kernel only header files and make sure this
directory is searched _before_ include/

We could do something like
kinclude/linux/         <= generic kernel include headers
kinclude/$(arch)/asm/   <= arch specific include headers

Then adding to the top-level Makefile:
LINUXINCLUDE := -Ikinclude -Ikinclude/$(ARCH)
LINUXINCLUDE += ...

would actually cut it.
Then we would not hurt existing users since they continue to use
include/linux/* + include/asm/*
And we could migrate one by one to the kernel clean part.

In other words - a solution that keeps backwards compatibility.
A solution that distingush hard between what is the ABI and
what is kernel stuff.
And a namespace that is not in use today.

And we have so many users of include/linux today. They do not need
a _kabi_ prefix so let it go.

	Sam

Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

[Kyle Moffett]

On Mar 26, 2006, at 15:05:37, Sam Ravnborg wrote:
> On Sun, Mar 26, 2006 at 06:54:16AM -0500, Kyle Moffett wrote:
>> Create initial kernel ABI header infrastructure
>>
>> diff --git a/Makefile b/Makefile
>> index af6210d..8e9045a 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -787,13 +787,15 @@ ifneq ($(KBUILD_SRC),)
>>  		/bin/false; \
>>  	fi;
>>  	$(Q)if [ ! -d include2 ]; then mkdir -p include2; fi;
>> +	$(Q)if [ ! -d include2/kabi ]; then mkdir -p include2/kabi; fi;
>>  	$(Q)ln -fsn $(srctree)/include/asm-$(ARCH) include2/asm
>> +	$(Q)ln -fsn $(srctree)/include/kabi/arch-$(ARCH) include2/kabi/arch
>
> No - we do not want another symlink.
> Create something like:
> include/i386/kabi-asm/  <= i386 specific files
> include/kabi/           <= general files
>
> Then we can do:
> LINUXINCLUDE += -Iinclude/kabi-$(ARCH)
> And the following would work like expected:
> #include <kabi/foo.h>
> #include <kabi-asm/foo.h>

Hmm, I see where you're going with this.  Thanks for the tip!

> But this leaves all existing users in the dark cold.  So a more  
> involved approach could be to take the opposite approach. To  
> dedicate an area for kernel only header files and make sure this  
> directory is searched _before_ include/

I'm not entirely sure I understand this bit.  The idea behind this  
kabi stuff is precisely to split out portions of the headers so that  
both userspace and kernelspace can get at them; to designate specific  
items as "userspace clean" by putting them in kabi; everything else  
need not care at all, and all those headers would remain in include/ 
linux where they are now.  No sense moving _everything_ in include/  
around, we just want the parts that userspace needs too.

> And we have so many users of include/linux today. They do not need  
> a _kabi_ prefix so let it go.

_Exactly_ the point.  I'd like to leave 99% of the kernel code  
alone.  Userspace doesn't work with the headers in the stock kernel  
as-is, so I see no problem with working to provide a clean source ABI  
even if it's partially incompatible with the old source ABI.  The  
binary ABI would stay the same, though, and I'd try to fix the old  
source ABI where possible.

I think I'll try something like your original proposal:
include/kabi/			<= Userspace-clean generic ABI includes
include/$(ARCH)/		<= New arch-specific include directory
include/$(ARCH)/kabi-arch/	<= Arch-specific ABI includes

Then the following Makefile snippet:
LINUXINCLUDE += -Iinclude/$(ARCH)

And then in the header file:
#include <kabi-arch/stddef.h>
#include <kabi/types.h>

It would also be possible to move files from include/asm-$(ARCH) to  
include/$(ARCH)/asm if desired, but that's a little offtopic for the  
kernel-ABI headers project I'm working on right now.

Thanks for the advice!
Kyle Moffett

Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

[Sam Ravnborg]

On Sun, Mar 26, 2006 at 03:39:32PM -0500, Kyle Moffett wrote:
> On Mar 26, 2006, at 15:05:37, Sam Ravnborg wrote:
> >On Sun, Mar 26, 2006 at 06:54:16AM -0500, Kyle Moffett wrote:
> >>Create initial kernel ABI header infrastructure
> >>
> >>diff --git a/Makefile b/Makefile
> >>index af6210d..8e9045a 100644
> >>--- a/Makefile
> >>+++ b/Makefile
> >>@@ -787,13 +787,15 @@ ifneq ($(KBUILD_SRC),)
> >> 		/bin/false; \
> >> 	fi;
> >> 	$(Q)if [ ! -d include2 ]; then mkdir -p include2; fi;
> >>+	$(Q)if [ ! -d include2/kabi ]; then mkdir -p include2/kabi; fi;
> >> 	$(Q)ln -fsn $(srctree)/include/asm-$(ARCH) include2/asm
> >>+	$(Q)ln -fsn $(srctree)/include/kabi/arch-$(ARCH) include2/kabi/arch
> >
> >No - we do not want another symlink.
> >Create something like:
> >include/i386/kabi-asm/  <= i386 specific files
> >include/kabi/           <= general files
> >
> >Then we can do:
> >LINUXINCLUDE += -Iinclude/kabi-$(ARCH)
> >And the following would work like expected:
> >#include <kabi/foo.h>
> >#include <kabi-asm/foo.h>
> 
> Hmm, I see where you're going with this.  Thanks for the tip!
> 
> >But this leaves all existing users in the dark cold.  So a more  
> >involved approach could be to take the opposite approach. To  
> >dedicate an area for kernel only header files and make sure this  
> >directory is searched _before_ include/
> 
> I'm not entirely sure I understand this bit.  The idea behind this  
> kabi stuff is precisely to split out portions of the headers so that  
> both userspace and kernelspace can get at them; to designate specific  
> items as "userspace clean" by putting them in kabi; everything else  
> need not care at all, and all those headers would remain in include/ 
> linux where they are now.  No sense moving _everything_ in include/  
> around, we just want the parts that userspace needs too.
> 

There are today a great number of users of the existing kernel headers.
Introducing a new namespace for the userspace suers will leave them in a
dilemma where they have to support kernels before kabi, and kernels with
kabi. That alone will limit the acceptance of this.

So shifting to another approach where you take everything that we have
today and clean it up step by step will allow us in small steps to go
from current mess to a clean ernel ABI.
But we will not have this clean and virgin set of headers until a few
years ahead of us.

Keeping include/linux for the kernel ABI will allow us NOT to break
existing users and it will allow a stepwise apporach at the same time.
But what we make clean and virgin is the kernel specific stuff, not the
userspace specific stuff. In final result will be the same but shifting
focus to the kernel internal stuff allows us to do ti in incremental
steps that does NOT break userspace every and so often.
And we will only break external modules that does not use kbuild to
compile - and for hese I really do not care a whit.

Keeping the breakage minimal is the key point here.

	Sam

Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

[Kyle Moffett]

On Mar 26, 2006, at 16:26:01, Sam Ravnborg wrote:
> On Sun, Mar 26, 2006 at 03:39:32PM -0500, Kyle Moffett wrote:
>> I'm not entirely sure I understand this bit.  The idea behind this  
>> kabi stuff is precisely to split out portions of the headers so  
>> that both userspace and kernelspace can get at them; to designate  
>> specific items as "userspace clean" by putting them in kabi;  
>> everything else need not care at all, and all those headers would  
>> remain in include/ linux where they are now.  No sense moving  
>> _everything_ in include/ around, we just want the parts that  
>> userspace needs too.
>
> There are today a great number of users of the existing kernel  
> headers.  Introducing a new namespace for the userspace suers will  
> leave them in a dilemma where they have to support kernels before  
> kabi, and kernels with kabi. That alone will limit the acceptance  
> of this.

I tried to talk to this in my other email, but for emphasis; I'd like  
to do two things:
1)  Preserve compatibility with <linux/*.h>, so that old programs  
still work fine
2)  Cause <kabi/*.h> to work on any distro, and kernel, not just  
kernel released after it is introduced.  In practice this means that  
a distro could just package up the kabi headers and install them in / 
usr/include.  A package that is altered to compile against only  
__kabi_* would be able to use information about which syscalls and  
structures are available in each version to either handle different  
kernels at runtime or have a configure-time kernel version specified  
by the user.

> Keeping include/linux for the kernel ABI will allow us NOT to break  
> existing users and it will allow a stepwise apporach at the same time.

I completely agree.  If you'll notice the patches I submitted were  
quite careful to preserve the exact same semantics for the linux/*.h  
headers, both in the interests of not breaking the kernel _and_ in  
the interests of allowing userspace to still include <linux/ 
types.h>.  IMHO, we should try to get the interface compatibility  
crap _out_ of linux source tree and into a separate package which  
depends only on the cleaned-up interface specification (IE: kabi/*).

Cheers,
Kyle Moffett

[RFC][PATCH 2/2] Generalize fd_set handling across architectures

[Kyle Moffett]

Generalize fd_set handling across architectures

The fd_set handling is identical across all architectures, but each has its
own unique set of ugly functions or macros to do it.  This cleans up all of
the ugliness and merges the duplicated code into a single KABI header,
include/kabi/fdset.h, which is then included in the old places to preserve
existing semantics.

Appears to compile correctly with allmodconfig on powerpc (32-bit)

Signed-off-by: Kyle Moffett <mrmacman_g4@mac.com>

---
commit c00dc3cc75d218b5c23cefcb5324197f34d93de3
tree b4baa9126e0f7490c782279eb069ff184e59a73f
parent affadc51c2b4c96785809a352973a9f6a51a4c37
author Kyle Moffett <mrmacman_g4@mac.com> Sun, 26 Mar 2006 06:25:07 -0500
committer Kyle Moffett <mrmacman_g4@mac.com> Sun, 26 Mar 2006 06:25:07 -0500

 include/asm-alpha/posix_types.h   |   81 +------------------------------------
 include/asm-arm/posix_types.h     |   22 +---------
 include/asm-arm26/posix_types.h   |   22 +---------
 include/asm-cris/posix_types.h    |   23 -----------
 include/asm-frv/posix_types.h     |   18 +-------
 include/asm-h8300/posix_types.h   |   18 +-------
 include/asm-i386/posix_types.h    |   35 +---------------
 include/asm-ia64/posix_types.h    |   79 +-----------------------------------
 include/asm-m32r/posix_types.h    |   75 +---------------------------------
 include/asm-m68k/posix_types.h    |   18 +-------
 include/asm-mips/posix_types.h    |   73 ---------------------------------
 include/asm-parisc/posix_types.h  |   74 +---------------------------------
 include/asm-powerpc/posix_types.h |   74 +---------------------------------
 include/asm-s390/posix_types.h    |   23 +----------
 include/asm-sh/posix_types.h      |   75 +---------------------------------
 include/asm-sh64/posix_types.h    |   75 +---------------------------------
 include/asm-sparc/posix_types.h   |   75 +---------------------------------
 include/asm-sparc64/posix_types.h |   75 +---------------------------------
 include/asm-v850/posix_types.h    |   23 +----------
 include/asm-x86_64/posix_types.h  |   75 +---------------------------------
 include/asm-xtensa/posix_types.h  |   68 +------------------------------
 include/kabi/fdset.h              |   49 ++++++++++++++++++++++
 include/linux/fdset.h             |   15 +++++++
 include/linux/posix_types.h       |   35 ----------------
 include/linux/time.h              |   14 +++---
 25 files changed, 112 insertions(+), 1102 deletions(-)

diff --git a/include/asm-alpha/posix_types.h b/include/asm-alpha/posix_types.h
index c78c04a..c303ade 100644
--- a/include/asm-alpha/posix_types.h
+++ b/include/asm-alpha/posix_types.h
@@ -1,6 +1,8 @@
 #ifndef _ALPHA_POSIX_TYPES_H
 #define _ALPHA_POSIX_TYPES_H
 
+#include <linux/fdset.h>
+
 /*
  * This file is generally used by user-level software, so you need to
  * be a little careful about namespace pollution etc.  Also, we cannot
@@ -41,83 +43,4 @@ typedef __kernel_gid_t __kernel_gid32_t;
 
 typedef unsigned int	__kernel_old_dev_t;
 
-#ifdef __KERNEL__
-
-#ifndef __GNUC__
-
-#define	__FD_SET(d, set)	((set)->fds_bits[__FDELT(d)] |= __FDMASK(d))
-#define	__FD_CLR(d, set)	((set)->fds_bits[__FDELT(d)] &= ~__FDMASK(d))
-#define	__FD_ISSET(d, set)	(((set)->fds_bits[__FDELT(d)] & __FDMASK(d)) != 0)
-#define	__FD_ZERO(set)	\
-  ((void) memset ((__ptr_t) (set), 0, sizeof (__kernel_fd_set)))
-
-#else /* __GNUC__ */
-
-/* With GNU C, use inline functions instead so args are evaluated only once: */
-
-#undef __FD_SET
-static __inline__ void __FD_SET(unsigned long fd, __kernel_fd_set *fdsetp)
-{
-	unsigned long _tmp = fd / __NFDBITS;
-	unsigned long _rem = fd % __NFDBITS;
-	fdsetp->fds_bits[_tmp] |= (1UL<<_rem);
-}
-
-#undef __FD_CLR
-static __inline__ void __FD_CLR(unsigned long fd, __kernel_fd_set *fdsetp)
-{
-	unsigned long _tmp = fd / __NFDBITS;
-	unsigned long _rem = fd % __NFDBITS;
-	fdsetp->fds_bits[_tmp] &= ~(1UL<<_rem);
-}
-
-#undef __FD_ISSET
-static __inline__ int __FD_ISSET(unsigned long fd, const __kernel_fd_set *p)
-{ 
-	unsigned long _tmp = fd / __NFDBITS;
-	unsigned long _rem = fd % __NFDBITS;
-	return (p->fds_bits[_tmp] & (1UL<<_rem)) != 0;
-}
-
-/*
- * This will unroll the loop for the normal constant case (8 ints,
- * for a 256-bit fd_set)
- */
-#undef __FD_ZERO
-static __inline__ void __FD_ZERO(__kernel_fd_set *p)
-{
-	unsigned long *tmp = p->fds_bits;
-	int i;
-
-	if (__builtin_constant_p(__FDSET_LONGS)) {
-		switch (__FDSET_LONGS) {
-		      case 16:
-			tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
-			tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
-			tmp[ 8] = 0; tmp[ 9] = 0; tmp[10] = 0; tmp[11] = 0;
-			tmp[12] = 0; tmp[13] = 0; tmp[14] = 0; tmp[15] = 0;
-			return;
-
-		      case 8:
-			tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
-			tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
-			return;
-
-		      case 4:
-			tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
-			return;
-		}
-	}
-	i = __FDSET_LONGS;
-	while (i) {
-		i--;
-		*tmp = 0;
-		tmp++;
-	}
-}
-
-#endif /* __GNUC__ */
-
-#endif /* __KERNEL__ */
-
 #endif /* _ALPHA_POSIX_TYPES_H */
diff --git a/include/asm-arm/posix_types.h b/include/asm-arm/posix_types.h
index e142a2a..918508b 100644
--- a/include/asm-arm/posix_types.h
+++ b/include/asm-arm/posix_types.h
@@ -13,6 +13,8 @@
 #ifndef __ARCH_ARM_POSIX_TYPES_H
 #define __ARCH_ARM_POSIX_TYPES_H
 
+#include <linux/fdset.h>
+
 /*
  * This file is generally used by user-level software, so you need to
  * be a little careful about namespace pollution etc.  Also, we cannot
@@ -58,24 +60,4 @@ typedef struct {
 #endif /* !defined(__KERNEL__) && !defined(__USE_ALL) */
 } __kernel_fsid_t;
 
-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef	__FD_SET
-#define __FD_SET(fd, fdsetp) \
-		(((fd_set *)(fdsetp))->fds_bits[(fd) >> 5] |= (1<<((fd) & 31)))
-
-#undef	__FD_CLR
-#define __FD_CLR(fd, fdsetp) \
-		(((fd_set *)(fdsetp))->fds_bits[(fd) >> 5] &= ~(1<<((fd) & 31)))
-
-#undef	__FD_ISSET
-#define __FD_ISSET(fd, fdsetp) \
-		((((fd_set *)(fdsetp))->fds_bits[(fd) >> 5] & (1<<((fd) & 31))) != 0)
-
-#undef	__FD_ZERO
-#define __FD_ZERO(fdsetp) \
-		(memset (fdsetp, 0, sizeof (*(fd_set *)(fdsetp))))
-
-#endif
-
 #endif
diff --git a/include/asm-arm26/posix_types.h b/include/asm-arm26/posix_types.h
index f8d1eb4..918508b 100644
--- a/include/asm-arm26/posix_types.h
+++ b/include/asm-arm26/posix_types.h
@@ -13,6 +13,8 @@
 #ifndef __ARCH_ARM_POSIX_TYPES_H
 #define __ARCH_ARM_POSIX_TYPES_H
 
+#include <linux/fdset.h>
+
 /*
  * This file is generally used by user-level software, so you need to
  * be a little careful about namespace pollution etc.  Also, we cannot
@@ -58,24 +60,4 @@ typedef struct {
 #endif /* !defined(__KERNEL__) && !defined(__USE_ALL) */
 } __kernel_fsid_t;
 
-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef	__FD_SET
-#define __FD_SET(fd, fdsetp) \
-		(((fd_set *)(fdsetp))->fds_bits[(fd) >> 5] |= (1<<((fd) & 31)))
-
-#undef	__FD_CLR
-#define __FD_CLR(fd, fdsetp) \
-		(((fd_set *)(fdsetp))->fds_bits[(fd) >> 5] &= ~(1<<((fd) & 31)))
-
-#undef	__FD_ISSET
-#define __FD_ISSET(fd, fdsetp) \
-		((((fd_set *)(fdsetp))->fds_bits[(fd) >> 5] & (1<<((fd) & 31))) != 0)
-
-#undef	__FD_ZERO
-#define __FD_ZERO(fdsetp) \
-		(memset ((fdsetp), 0, sizeof (*(fd_set *)(fdsetp))))
-
-#endif
-
 #endif
diff --git a/include/asm-cris/posix_types.h b/include/asm-cris/posix_types.h
index 6d26fee..fdf1e26 100644
--- a/include/asm-cris/posix_types.h
+++ b/include/asm-cris/posix_types.h
@@ -1,12 +1,9 @@
 /* $Id: posix_types.h,v 1.1 2000/07/10 16:32:31 bjornw Exp $ */
 
-/* We cheat a bit and use our C-coded bitops functions from asm/bitops.h */
-/* I guess we should write these in assembler because they are used often. */
-
 #ifndef __ARCH_CRIS_POSIX_TYPES_H
 #define __ARCH_CRIS_POSIX_TYPES_H
 
-#include <asm/bitops.h>
+#include <linux/fdset.h>
 
 /*
  * This file is generally used by user-level software, so you need to
@@ -53,22 +50,4 @@ typedef struct {
 #endif /* !defined(__KERNEL__) && !defined(__USE_ALL) */
 } __kernel_fsid_t;
 
-/* should this ifdef be here ?  */
-
-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef	__FD_SET
-#define __FD_SET(fd,fdsetp) set_bit(fd, (void *)(fdsetp))
-
-#undef	__FD_CLR
-#define __FD_CLR(fd,fdsetp) clear_bit(fd, (void *)(fdsetp))
-
-#undef	__FD_ISSET
-#define __FD_ISSET(fd,fdsetp) test_bit(fd, (void *)(fdsetp))
-
-#undef	__FD_ZERO
-#define __FD_ZERO(fdsetp) memset((void *)(fdsetp), 0, __FDSET_LONGS << 2)
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-
 #endif /* __ARCH_CRIS_POSIX_TYPES_H */
diff --git a/include/asm-frv/posix_types.h b/include/asm-frv/posix_types.h
index 73c2ba8..5b241f7 100644
--- a/include/asm-frv/posix_types.h
+++ b/include/asm-frv/posix_types.h
@@ -1,6 +1,8 @@
 #ifndef _ASM_POSIX_TYPES_H
 #define _ASM_POSIX_TYPES_H
 
+#include <linux/fdset.h>
+
 /*
  * This file is generally used by user-level software, so you need to
  * be a little careful about namespace pollution etc.  Also, we cannot
@@ -46,21 +48,5 @@ typedef struct {
 #endif /* !defined(__KERNEL__) && !defined(__USE_ALL) */
 } __kernel_fsid_t;
 
-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef	__FD_SET
-#define	__FD_SET(d, set)	((set)->fds_bits[__FDELT(d)] |= __FDMASK(d))
-
-#undef	__FD_CLR
-#define	__FD_CLR(d, set)	((set)->fds_bits[__FDELT(d)] &= ~__FDMASK(d))
-
-#undef	__FD_ISSET
-#define	__FD_ISSET(d, set)	(!!((set)->fds_bits[__FDELT(d)] & __FDMASK(d)))
-
-#undef	__FD_ZERO
-#define __FD_ZERO(fdsetp) (memset (fdsetp, 0, sizeof(*(fd_set *)fdsetp)))
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-
 #endif
 
diff --git a/include/asm-h8300/posix_types.h b/include/asm-h8300/posix_types.h
index 7de94b1..c65fbc9 100644
--- a/include/asm-h8300/posix_types.h
+++ b/include/asm-h8300/posix_types.h
@@ -1,6 +1,8 @@
 #ifndef __ARCH_H8300_POSIX_TYPES_H
 #define __ARCH_H8300_POSIX_TYPES_H
 
+#include <linux/fdset.h>
+
 /*
  * This file is generally used by user-level software, so you need to
  * be a little careful about namespace pollution etc.  Also, we cannot
@@ -45,20 +47,4 @@ typedef struct {
 #endif /* !defined(__KERNEL__) && !defined(__USE_ALL) */
 } __kernel_fsid_t;
 
-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef	__FD_SET
-#define	__FD_SET(d, set)	((set)->fds_bits[__FDELT(d)] |= __FDMASK(d))
-
-#undef	__FD_CLR
-#define	__FD_CLR(d, set)	((set)->fds_bits[__FDELT(d)] &= ~__FDMASK(d))
-
-#undef	__FD_ISSET
-#define	__FD_ISSET(d, set)	((set)->fds_bits[__FDELT(d)] & __FDMASK(d))
-
-#undef	__FD_ZERO
-#define __FD_ZERO(fdsetp) (memset (fdsetp, 0, sizeof(*(fd_set *)fdsetp)))
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-
 #endif
diff --git a/include/asm-i386/posix_types.h b/include/asm-i386/posix_types.h
index 4e47ed0..7148932 100644
--- a/include/asm-i386/posix_types.h
+++ b/include/asm-i386/posix_types.h
@@ -1,6 +1,8 @@
 #ifndef __ARCH_I386_POSIX_TYPES_H
 #define __ARCH_I386_POSIX_TYPES_H
 
+#include <linux/fdset.h>
+
 /*
  * This file is generally used by user-level software, so you need to
  * be a little careful about namespace pollution etc.  Also, we cannot
@@ -46,37 +48,4 @@ typedef struct {
 #endif /* !defined(__KERNEL__) && !defined(__USE_ALL) */
 } __kernel_fsid_t;
 
-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef	__FD_SET
-#define __FD_SET(fd,fdsetp) \
-		__asm__ __volatile__("btsl %1,%0": \
-			"=m" (*(__kernel_fd_set *) (fdsetp)):"r" ((int) (fd)))
-
-#undef	__FD_CLR
-#define __FD_CLR(fd,fdsetp) \
-		__asm__ __volatile__("btrl %1,%0": \
-			"=m" (*(__kernel_fd_set *) (fdsetp)):"r" ((int) (fd)))
-
-#undef	__FD_ISSET
-#define __FD_ISSET(fd,fdsetp) (__extension__ ({ \
-		unsigned char __result; \
-		__asm__ __volatile__("btl %1,%2 ; setb %0" \
-			:"=q" (__result) :"r" ((int) (fd)), \
-			"m" (*(__kernel_fd_set *) (fdsetp))); \
-		__result; }))
-
-#undef	__FD_ZERO
-#define __FD_ZERO(fdsetp) \
-do { \
-	int __d0, __d1; \
-	__asm__ __volatile__("cld ; rep ; stosl" \
-			:"=m" (*(__kernel_fd_set *) (fdsetp)), \
-			  "=&c" (__d0), "=&D" (__d1) \
-			:"a" (0), "1" (__FDSET_LONGS), \
-			"2" ((__kernel_fd_set *) (fdsetp)) : "memory"); \
-} while (0)
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-
 #endif
diff --git a/include/asm-ia64/posix_types.h b/include/asm-ia64/posix_types.h
index adb6227..15a0ad7 100644
--- a/include/asm-ia64/posix_types.h
+++ b/include/asm-ia64/posix_types.h
@@ -1,6 +1,8 @@
 #ifndef _ASM_IA64_POSIX_TYPES_H
 #define _ASM_IA64_POSIX_TYPES_H
 
+#include <linux/fdset.h>
+
 /*
  * This file is generally used by user-level software, so you need to
  * be a little careful about namespace pollution etc.  Also, we cannot
@@ -46,81 +48,4 @@ typedef __kernel_gid_t __kernel_gid32_t;
 
 typedef unsigned int	__kernel_old_dev_t;
 
-# ifdef __KERNEL__
-
-#  ifndef __GNUC__
-
-#define	__FD_SET(d, set)	((set)->fds_bits[__FDELT(d)] |= __FDMASK(d))
-#define	__FD_CLR(d, set)	((set)->fds_bits[__FDELT(d)] &= ~__FDMASK(d))
-#define	__FD_ISSET(d, set)	(((set)->fds_bits[__FDELT(d)] & __FDMASK(d)) != 0)
-#define	__FD_ZERO(set)	\
-  ((void) memset ((__ptr_t) (set), 0, sizeof (__kernel_fd_set)))
-
-#  else /* !__GNUC__ */
-
-/* With GNU C, use inline functions instead so args are evaluated only once: */
-
-#undef __FD_SET
-static __inline__ void __FD_SET(unsigned long fd, __kernel_fd_set *fdsetp)
-{
-	unsigned long _tmp = fd / __NFDBITS;
-	unsigned long _rem = fd % __NFDBITS;
-	fdsetp->fds_bits[_tmp] |= (1UL<<_rem);
-}
-
-#undef __FD_CLR
-static __inline__ void __FD_CLR(unsigned long fd, __kernel_fd_set *fdsetp)
-{
-	unsigned long _tmp = fd / __NFDBITS;
-	unsigned long _rem = fd % __NFDBITS;
-	fdsetp->fds_bits[_tmp] &= ~(1UL<<_rem);
-}
-
-#undef __FD_ISSET
-static __inline__ int __FD_ISSET(unsigned long fd, const __kernel_fd_set *p)
-{ 
-	unsigned long _tmp = fd / __NFDBITS;
-	unsigned long _rem = fd % __NFDBITS;
-	return (p->fds_bits[_tmp] & (1UL<<_rem)) != 0;
-}
-
-/*
- * This will unroll the loop for the normal constant case (8 ints,
- * for a 256-bit fd_set)
- */
-#undef __FD_ZERO
-static __inline__ void __FD_ZERO(__kernel_fd_set *p)
-{
-	unsigned long *tmp = p->fds_bits;
-	int i;
-
-	if (__builtin_constant_p(__FDSET_LONGS)) {
-		switch (__FDSET_LONGS) {
-		      case 16:
-			tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
-			tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
-			tmp[ 8] = 0; tmp[ 9] = 0; tmp[10] = 0; tmp[11] = 0;
-			tmp[12] = 0; tmp[13] = 0; tmp[14] = 0; tmp[15] = 0;
-			return;
-
-		      case 8:
-			tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
-			tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
-			return;
-
-		      case 4:
-			tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
-			return;
-		}
-	}
-	i = __FDSET_LONGS;
-	while (i) {
-		i--;
-		*tmp = 0;
-		tmp++;
-	}
-}
-
-#  endif /* !__GNUC__ */
-# endif /* __KERNEL__ */
 #endif /* _ASM_IA64_POSIX_TYPES_H */
diff --git a/include/asm-m32r/posix_types.h b/include/asm-m32r/posix_types.h
index 47e7e85..e79af7d 100644
--- a/include/asm-m32r/posix_types.h
+++ b/include/asm-m32r/posix_types.h
@@ -1,6 +1,8 @@
 #ifndef _ASM_M32R_POSIX_TYPES_H
 #define _ASM_M32R_POSIX_TYPES_H
 
+#include <linux/fdset.h>
+
 /* $Id$ */
 
 /* orig : i386, sh 2.4.18 */
@@ -50,77 +52,4 @@ typedef struct {
 #endif /* !defined(__KERNEL__) && !defined(__USE_ALL) */
 } __kernel_fsid_t;
 
-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef	__FD_SET
-static __inline__ void __FD_SET(unsigned long __fd, __kernel_fd_set *__fdsetp)
-{
-	unsigned long __tmp = __fd / __NFDBITS;
-	unsigned long __rem = __fd % __NFDBITS;
-	__fdsetp->fds_bits[__tmp] |= (1UL<<__rem);
-}
-
-#undef	__FD_CLR
-static __inline__ void __FD_CLR(unsigned long __fd, __kernel_fd_set *__fdsetp)
-{
-	unsigned long __tmp = __fd / __NFDBITS;
-	unsigned long __rem = __fd % __NFDBITS;
-	__fdsetp->fds_bits[__tmp] &= ~(1UL<<__rem);
-}
-
-
-#undef	__FD_ISSET
-static __inline__ int __FD_ISSET(unsigned long __fd, const __kernel_fd_set *__p)
-{
-	unsigned long __tmp = __fd / __NFDBITS;
-	unsigned long __rem = __fd % __NFDBITS;
-	return (__p->fds_bits[__tmp] & (1UL<<__rem)) != 0;
-}
-
-/*
- * This will unroll the loop for the normal constant case (8 ints,
- * for a 256-bit fd_set)
- */
-#undef	__FD_ZERO
-static __inline__ void __FD_ZERO(__kernel_fd_set *__p)
-{
-	unsigned long *__tmp = __p->fds_bits;
-	int __i;
-
-	if (__builtin_constant_p(__FDSET_LONGS)) {
-		switch (__FDSET_LONGS) {
-		case 16:
-			__tmp[ 0] = 0; __tmp[ 1] = 0;
-			__tmp[ 2] = 0; __tmp[ 3] = 0;
-			__tmp[ 4] = 0; __tmp[ 5] = 0;
-			__tmp[ 6] = 0; __tmp[ 7] = 0;
-			__tmp[ 8] = 0; __tmp[ 9] = 0;
-			__tmp[10] = 0; __tmp[11] = 0;
-			__tmp[12] = 0; __tmp[13] = 0;
-			__tmp[14] = 0; __tmp[15] = 0;
-			return;
-
-		case 8:
-			__tmp[ 0] = 0; __tmp[ 1] = 0;
-			__tmp[ 2] = 0; __tmp[ 3] = 0;
-			__tmp[ 4] = 0; __tmp[ 5] = 0;
-			__tmp[ 6] = 0; __tmp[ 7] = 0;
-			return;
-
-		case 4:
-			__tmp[ 0] = 0; __tmp[ 1] = 0;
-			__tmp[ 2] = 0; __tmp[ 3] = 0;
-			return;
-		}
-	}
-	__i = __FDSET_LONGS;
-	while (__i) {
-		__i--;
-		*__tmp = 0;
-		__tmp++;
-	}
-}
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-
 #endif  /* _ASM_M32R_POSIX_TYPES_H */
diff --git a/include/asm-m68k/posix_types.h b/include/asm-m68k/posix_types.h
index fa166ee..39b1b77 100644
--- a/include/asm-m68k/posix_types.h
+++ b/include/asm-m68k/posix_types.h
@@ -1,6 +1,8 @@
 #ifndef __ARCH_M68K_POSIX_TYPES_H
 #define __ARCH_M68K_POSIX_TYPES_H
 
+#include <linux/fdset.h>
+
 /*
  * This file is generally used by user-level software, so you need to
  * be a little careful about namespace pollution etc.  Also, we cannot
@@ -46,20 +48,4 @@ typedef struct {
 #endif /* !defined(__KERNEL__) && !defined(__USE_ALL) */
 } __kernel_fsid_t;
 
-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef	__FD_SET
-#define	__FD_SET(d, set)	((set)->fds_bits[__FDELT(d)] |= __FDMASK(d))
-
-#undef	__FD_CLR
-#define	__FD_CLR(d, set)	((set)->fds_bits[__FDELT(d)] &= ~__FDMASK(d))
-
-#undef	__FD_ISSET
-#define	__FD_ISSET(d, set)	((set)->fds_bits[__FDELT(d)] & __FDMASK(d))
-
-#undef	__FD_ZERO
-#define __FD_ZERO(fdsetp) (memset (fdsetp, 0, sizeof(*(fd_set *)fdsetp)))
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-
 #endif
diff --git a/include/asm-mips/posix_types.h b/include/asm-mips/posix_types.h
index c2e8a00..01ad0d7 100644
--- a/include/asm-mips/posix_types.h
+++ b/include/asm-mips/posix_types.h
@@ -10,6 +10,7 @@
 #define _ASM_POSIX_TYPES_H
 
 #include <asm/sgidefs.h>
+#include <linux/fdset.h>
 
 /*
  * This file is generally used by user-level software, so you need to
@@ -69,76 +70,4 @@ typedef struct {
 #endif
 } __kernel_fsid_t;
 
-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef __FD_SET
-static __inline__ void __FD_SET(unsigned long __fd, __kernel_fd_set *__fdsetp)
-{
-	unsigned long __tmp = __fd / __NFDBITS;
-	unsigned long __rem = __fd % __NFDBITS;
-	__fdsetp->fds_bits[__tmp] |= (1UL<<__rem);
-}
-
-#undef __FD_CLR
-static __inline__ void __FD_CLR(unsigned long __fd, __kernel_fd_set *__fdsetp)
-{
-	unsigned long __tmp = __fd / __NFDBITS;
-	unsigned long __rem = __fd % __NFDBITS;
-	__fdsetp->fds_bits[__tmp] &= ~(1UL<<__rem);
-}
-
-#undef __FD_ISSET
-static __inline__ int __FD_ISSET(unsigned long __fd, const __kernel_fd_set *__p)
-{
-	unsigned long __tmp = __fd / __NFDBITS;
-	unsigned long __rem = __fd % __NFDBITS;
-	return (__p->fds_bits[__tmp] & (1UL<<__rem)) != 0;
-}
-
-/*
- * This will unroll the loop for the normal constant case (8 ints,
- * for a 256-bit fd_set)
- */
-#undef __FD_ZERO
-static __inline__ void __FD_ZERO(__kernel_fd_set *__p)
-{
-	unsigned long *__tmp = __p->fds_bits;
-	int __i;
-
-	if (__builtin_constant_p(__FDSET_LONGS)) {
-		switch (__FDSET_LONGS) {
-		case 16:
-			__tmp[ 0] = 0; __tmp[ 1] = 0;
-			__tmp[ 2] = 0; __tmp[ 3] = 0;
-			__tmp[ 4] = 0; __tmp[ 5] = 0;
-			__tmp[ 6] = 0; __tmp[ 7] = 0;
-			__tmp[ 8] = 0; __tmp[ 9] = 0;
-			__tmp[10] = 0; __tmp[11] = 0;
-			__tmp[12] = 0; __tmp[13] = 0;
-			__tmp[14] = 0; __tmp[15] = 0;
-			return;
-
-		case 8:
-			__tmp[ 0] = 0; __tmp[ 1] = 0;
-			__tmp[ 2] = 0; __tmp[ 3] = 0;
-			__tmp[ 4] = 0; __tmp[ 5] = 0;
-			__tmp[ 6] = 0; __tmp[ 7] = 0;
-			return;
-
-		case 4:
-			__tmp[ 0] = 0; __tmp[ 1] = 0;
-			__tmp[ 2] = 0; __tmp[ 3] = 0;
-			return;
-		}
-	}
-	__i = __FDSET_LONGS;
-	while (__i) {
-		__i--;
-		*__tmp = 0;
-		__tmp++;
-	}
-}
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-
 #endif /* _ASM_POSIX_TYPES_H */
diff --git a/include/asm-parisc/posix_types.h b/include/asm-parisc/posix_types.h
index 9b19970..c58df37 100644
--- a/include/asm-parisc/posix_types.h
+++ b/include/asm-parisc/posix_types.h
@@ -1,6 +1,8 @@
 #ifndef __ARCH_PARISC_POSIX_TYPES_H
 #define __ARCH_PARISC_POSIX_TYPES_H
 
+#include <linux/fdset.h>
+
 /*
  * This file is generally used by user-level software, so you need to
  * be a little careful about namespace pollution etc.  Also, we cannot
@@ -58,76 +60,4 @@ typedef struct {
 typedef __kernel_uid_t __kernel_old_uid_t;
 typedef __kernel_gid_t __kernel_old_gid_t;
 
-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef __FD_SET
-static __inline__ void __FD_SET(unsigned long __fd, __kernel_fd_set *__fdsetp)
-{
-	unsigned long __tmp = __fd / __NFDBITS;
-	unsigned long __rem = __fd % __NFDBITS;
-	__fdsetp->fds_bits[__tmp] |= (1UL<<__rem);
-}
-
-#undef __FD_CLR
-static __inline__ void __FD_CLR(unsigned long __fd, __kernel_fd_set *__fdsetp)
-{
-	unsigned long __tmp = __fd / __NFDBITS;
-	unsigned long __rem = __fd % __NFDBITS;
-	__fdsetp->fds_bits[__tmp] &= ~(1UL<<__rem);
-}
-
-#undef __FD_ISSET
-static __inline__ int __FD_ISSET(unsigned long __fd, const __kernel_fd_set *__p)
-{ 
-	unsigned long __tmp = __fd / __NFDBITS;
-	unsigned long __rem = __fd % __NFDBITS;
-	return (__p->fds_bits[__tmp] & (1UL<<__rem)) != 0;
-}
-
-/*
- * This will unroll the loop for the normal constant case (8 ints,
- * for a 256-bit fd_set)
- */
-#undef __FD_ZERO
-static __inline__ void __FD_ZERO(__kernel_fd_set *__p)
-{
-	unsigned long *__tmp = __p->fds_bits;
-	int __i;
-
-	if (__builtin_constant_p(__FDSET_LONGS)) {
-		switch (__FDSET_LONGS) {
-		case 16:
-			__tmp[ 0] = 0; __tmp[ 1] = 0;
-			__tmp[ 2] = 0; __tmp[ 3] = 0;
-			__tmp[ 4] = 0; __tmp[ 5] = 0;
-			__tmp[ 6] = 0; __tmp[ 7] = 0;
-			__tmp[ 8] = 0; __tmp[ 9] = 0;
-			__tmp[10] = 0; __tmp[11] = 0;
-			__tmp[12] = 0; __tmp[13] = 0;
-			__tmp[14] = 0; __tmp[15] = 0;
-			return;
-
-		case 8:
-			__tmp[ 0] = 0; __tmp[ 1] = 0;
-			__tmp[ 2] = 0; __tmp[ 3] = 0;
-			__tmp[ 4] = 0; __tmp[ 5] = 0;
-			__tmp[ 6] = 0; __tmp[ 7] = 0;
-			return;
-
-		case 4:
-			__tmp[ 0] = 0; __tmp[ 1] = 0;
-			__tmp[ 2] = 0; __tmp[ 3] = 0;
-			return;
-		}
-	}
-	__i = __FDSET_LONGS;
-	while (__i) {
-		__i--;
-		*__tmp = 0;
-		__tmp++;
-	}
-}
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-
 #endif
diff --git a/include/asm-powerpc/posix_types.h b/include/asm-powerpc/posix_types.h
index c639107..7de6c18 100644
--- a/include/asm-powerpc/posix_types.h
+++ b/include/asm-powerpc/posix_types.h
@@ -1,6 +1,8 @@
 #ifndef _ASM_POWERPC_POSIX_TYPES_H
 #define _ASM_POWERPC_POSIX_TYPES_H
 
+#include <linux/fdset.h>
+
 /*
  * This file is generally used by user-level software, so you need to
  * be a little careful about namespace pollution etc.  Also, we cannot
@@ -54,76 +56,4 @@ typedef struct {
 	int	val[2];
 } __kernel_fsid_t;
 
-#ifndef __GNUC__
-
-#define	__FD_SET(d, set)	((set)->fds_bits[__FDELT(d)] |= __FDMASK(d))
-#define	__FD_CLR(d, set)	((set)->fds_bits[__FDELT(d)] &= ~__FDMASK(d))
-#define	__FD_ISSET(d, set)	(((set)->fds_bits[__FDELT(d)] & __FDMASK(d)) != 0)
-#define	__FD_ZERO(set)	\
-  ((void) memset ((__ptr_t) (set), 0, sizeof (__kernel_fd_set)))
-
-#else /* __GNUC__ */
-
-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) \
-    || (__GLIBC__ == 2 && __GLIBC_MINOR__ == 0)
-/* With GNU C, use inline functions instead so args are evaluated only once: */
-
-#undef __FD_SET
-static __inline__ void __FD_SET(unsigned long fd, __kernel_fd_set *fdsetp)
-{
-	unsigned long _tmp = fd / __NFDBITS;
-	unsigned long _rem = fd % __NFDBITS;
-	fdsetp->fds_bits[_tmp] |= (1UL<<_rem);
-}
-
-#undef __FD_CLR
-static __inline__ void __FD_CLR(unsigned long fd, __kernel_fd_set *fdsetp)
-{
-	unsigned long _tmp = fd / __NFDBITS;
-	unsigned long _rem = fd % __NFDBITS;
-	fdsetp->fds_bits[_tmp] &= ~(1UL<<_rem);
-}
-
-#undef __FD_ISSET
-static __inline__ int __FD_ISSET(unsigned long fd, __kernel_fd_set *p)
-{ 
-	unsigned long _tmp = fd / __NFDBITS;
-	unsigned long _rem = fd % __NFDBITS;
-	return (p->fds_bits[_tmp] & (1UL<<_rem)) != 0;
-}
-
-/*
- * This will unroll the loop for the normal constant case (8 ints,
- * for a 256-bit fd_set)
- */
-#undef __FD_ZERO
-static __inline__ void __FD_ZERO(__kernel_fd_set *p)
-{
-	unsigned long *tmp = (unsigned long *)p->fds_bits;
-	int i;
-
-	if (__builtin_constant_p(__FDSET_LONGS)) {
-		switch (__FDSET_LONGS) {
-		      case 16:
-			tmp[12] = 0; tmp[13] = 0; tmp[14] = 0; tmp[15] = 0;
-			tmp[ 8] = 0; tmp[ 9] = 0; tmp[10] = 0; tmp[11] = 0;
-
-		      case 8:
-			tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
-
-		      case 4:
-			tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
-			return;
-		}
-	}
-	i = __FDSET_LONGS;
-	while (i) {
-		i--;
-		*tmp = 0;
-		tmp++;
-	}
-}
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-#endif /* __GNUC__ */
 #endif /* _ASM_POWERPC_POSIX_TYPES_H */
diff --git a/include/asm-s390/posix_types.h b/include/asm-s390/posix_types.h
index 61788de..6cf74f1 100644
--- a/include/asm-s390/posix_types.h
+++ b/include/asm-s390/posix_types.h
@@ -9,6 +9,8 @@
 #ifndef __ARCH_S390_POSIX_TYPES_H
 #define __ARCH_S390_POSIX_TYPES_H
 
+#include <linux/fdset.h>
+
 /*
  * This file is generally used by user-level software, so you need to
  * be a little careful about namespace pollution etc.  Also, we cannot
@@ -75,25 +77,4 @@ typedef struct {
 #endif                       /* !defined(__KERNEL__) && !defined(__USE_ALL)*/
 } __kernel_fsid_t;
 
-
-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#ifndef _S390_BITOPS_H
-#include <asm/bitops.h>
-#endif
-
-#undef  __FD_SET
-#define __FD_SET(fd,fdsetp)  set_bit((fd),(fdsetp)->fds_bits)
-
-#undef  __FD_CLR
-#define __FD_CLR(fd,fdsetp)  clear_bit((fd),(fdsetp)->fds_bits)
-
-#undef  __FD_ISSET
-#define __FD_ISSET(fd,fdsetp)  test_bit((fd),(fdsetp)->fds_bits)
-
-#undef  __FD_ZERO
-#define __FD_ZERO(fdsetp) (memset ((fdsetp), 0, sizeof(*(fd_set *)(fdsetp))))
-
-#endif     /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)*/
-
 #endif
diff --git a/include/asm-sh/posix_types.h b/include/asm-sh/posix_types.h
index 0a3d2f5..4db22df 100644
--- a/include/asm-sh/posix_types.h
+++ b/include/asm-sh/posix_types.h
@@ -1,6 +1,8 @@
 #ifndef __ASM_SH_POSIX_TYPES_H
 #define __ASM_SH_POSIX_TYPES_H
 
+#include <linux/fdset.h>
+
 /*
  * This file is generally used by user-level software, so you need to
  * be a little careful about namespace pollution etc.  Also, we cannot
@@ -46,77 +48,4 @@ typedef struct {
 #endif /* !defined(__KERNEL__) && !defined(__USE_ALL) */
 } __kernel_fsid_t;
 
-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef	__FD_SET
-static __inline__ void __FD_SET(unsigned long __fd, __kernel_fd_set *__fdsetp)
-{
-	unsigned long __tmp = __fd / __NFDBITS;
-	unsigned long __rem = __fd % __NFDBITS;
-	__fdsetp->fds_bits[__tmp] |= (1UL<<__rem);
-}
-
-#undef	__FD_CLR
-static __inline__ void __FD_CLR(unsigned long __fd, __kernel_fd_set *__fdsetp)
-{
-	unsigned long __tmp = __fd / __NFDBITS;
-	unsigned long __rem = __fd % __NFDBITS;
-	__fdsetp->fds_bits[__tmp] &= ~(1UL<<__rem);
-}
-
-
-#undef	__FD_ISSET
-static __inline__ int __FD_ISSET(unsigned long __fd, const __kernel_fd_set *__p)
-{ 
-	unsigned long __tmp = __fd / __NFDBITS;
-	unsigned long __rem = __fd % __NFDBITS;
-	return (__p->fds_bits[__tmp] & (1UL<<__rem)) != 0;
-}
-
-/*
- * This will unroll the loop for the normal constant case (8 ints,
- * for a 256-bit fd_set)
- */
-#undef	__FD_ZERO
-static __inline__ void __FD_ZERO(__kernel_fd_set *__p)
-{
-	unsigned long *__tmp = __p->fds_bits;
-	int __i;
-
-	if (__builtin_constant_p(__FDSET_LONGS)) {
-		switch (__FDSET_LONGS) {
-		case 16:
-			__tmp[ 0] = 0; __tmp[ 1] = 0;
-			__tmp[ 2] = 0; __tmp[ 3] = 0;
-			__tmp[ 4] = 0; __tmp[ 5] = 0;
-			__tmp[ 6] = 0; __tmp[ 7] = 0;
-			__tmp[ 8] = 0; __tmp[ 9] = 0;
-			__tmp[10] = 0; __tmp[11] = 0;
-			__tmp[12] = 0; __tmp[13] = 0;
-			__tmp[14] = 0; __tmp[15] = 0;
-			return;
-
-		case 8:
-			__tmp[ 0] = 0; __tmp[ 1] = 0;
-			__tmp[ 2] = 0; __tmp[ 3] = 0;
-			__tmp[ 4] = 0; __tmp[ 5] = 0;
-			__tmp[ 6] = 0; __tmp[ 7] = 0;
-			return;
-
-		case 4:
-			__tmp[ 0] = 0; __tmp[ 1] = 0;
-			__tmp[ 2] = 0; __tmp[ 3] = 0;
-			return;
-		}
-	}
-	__i = __FDSET_LONGS;
-	while (__i) {
-		__i--;
-		*__tmp = 0;
-		__tmp++;
-	}
-}
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-
 #endif /* __ASM_SH_POSIX_TYPES_H */
diff --git a/include/asm-sh64/posix_types.h b/include/asm-sh64/posix_types.h
index 0620317..97448be 100644
--- a/include/asm-sh64/posix_types.h
+++ b/include/asm-sh64/posix_types.h
@@ -1,6 +1,8 @@
 #ifndef __ASM_SH64_POSIX_TYPES_H
 #define __ASM_SH64_POSIX_TYPES_H
 
+#include <linux/fdset.h>
+
 /*
  * This file is subject to the terms and conditions of the GNU General Public
  * License.  See the file "COPYING" in the main directory of this archive
@@ -55,77 +57,4 @@ typedef struct {
 #endif /* !defined(__KERNEL__) && !defined(__USE_ALL) */
 } __kernel_fsid_t;
 
-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef	__FD_SET
-static __inline__ void __FD_SET(unsigned long __fd, __kernel_fd_set *__fdsetp)
-{
-	unsigned long __tmp = __fd / __NFDBITS;
-	unsigned long __rem = __fd % __NFDBITS;
-	__fdsetp->fds_bits[__tmp] |= (1UL<<__rem);
-}
-
-#undef	__FD_CLR
-static __inline__ void __FD_CLR(unsigned long __fd, __kernel_fd_set *__fdsetp)
-{
-	unsigned long __tmp = __fd / __NFDBITS;
-	unsigned long __rem = __fd % __NFDBITS;
-	__fdsetp->fds_bits[__tmp] &= ~(1UL<<__rem);
-}
-
-
-#undef	__FD_ISSET
-static __inline__ int __FD_ISSET(unsigned long __fd, const __kernel_fd_set *__p)
-{
-	unsigned long __tmp = __fd / __NFDBITS;
-	unsigned long __rem = __fd % __NFDBITS;
-	return (__p->fds_bits[__tmp] & (1UL<<__rem)) != 0;
-}
-
-/*
- * This will unroll the loop for the normal constant case (8 ints,
- * for a 256-bit fd_set)
- */
-#undef	__FD_ZERO
-static __inline__ void __FD_ZERO(__kernel_fd_set *__p)
-{
-	unsigned long *__tmp = __p->fds_bits;
-	int __i;
-
-	if (__builtin_constant_p(__FDSET_LONGS)) {
-		switch (__FDSET_LONGS) {
-		case 16:
-			__tmp[ 0] = 0; __tmp[ 1] = 0;
-			__tmp[ 2] = 0; __tmp[ 3] = 0;
-			__tmp[ 4] = 0; __tmp[ 5] = 0;
-			__tmp[ 6] = 0; __tmp[ 7] = 0;
-			__tmp[ 8] = 0; __tmp[ 9] = 0;
-			__tmp[10] = 0; __tmp[11] = 0;
-			__tmp[12] = 0; __tmp[13] = 0;
-			__tmp[14] = 0; __tmp[15] = 0;
-			return;
-
-		case 8:
-			__tmp[ 0] = 0; __tmp[ 1] = 0;
-			__tmp[ 2] = 0; __tmp[ 3] = 0;
-			__tmp[ 4] = 0; __tmp[ 5] = 0;
-			__tmp[ 6] = 0; __tmp[ 7] = 0;
-			return;
-
-		case 4:
-			__tmp[ 0] = 0; __tmp[ 1] = 0;
-			__tmp[ 2] = 0; __tmp[ 3] = 0;
-			return;
-		}
-	}
-	__i = __FDSET_LONGS;
-	while (__i) {
-		__i--;
-		*__tmp = 0;
-		__tmp++;
-	}
-}
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-
 #endif /* __ASM_SH64_POSIX_TYPES_H */
diff --git a/include/asm-sparc/posix_types.h b/include/asm-sparc/posix_types.h
index 9ef1b3d..6e6b39e 100644
--- a/include/asm-sparc/posix_types.h
+++ b/include/asm-sparc/posix_types.h
@@ -1,6 +1,8 @@
 #ifndef __ARCH_SPARC_POSIX_TYPES_H
 #define __ARCH_SPARC_POSIX_TYPES_H
 
+#include <linux/fdset.h>
+
 /*
  * This file is generally used by user-level software, so you need to
  * be a little careful about namespace pollution etc.  Also, we cannot
@@ -46,77 +48,4 @@ typedef struct {
 #endif /* !defined(__KERNEL__) && !defined(__USE_ALL) */
 } __kernel_fsid_t;
 
-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef __FD_SET
-static __inline__ void __FD_SET(unsigned long fd, __kernel_fd_set *fdsetp)
-{
-	unsigned long _tmp = fd / __NFDBITS;
-	unsigned long _rem = fd % __NFDBITS;
-	fdsetp->fds_bits[_tmp] |= (1UL<<_rem);
-}
-
-#undef __FD_CLR
-static __inline__ void __FD_CLR(unsigned long fd, __kernel_fd_set *fdsetp)
-{
-	unsigned long _tmp = fd / __NFDBITS;
-	unsigned long _rem = fd % __NFDBITS;
-	fdsetp->fds_bits[_tmp] &= ~(1UL<<_rem);
-}
-
-#undef __FD_ISSET
-static __inline__ int __FD_ISSET(unsigned long fd, __const__ __kernel_fd_set *p)
-{ 
-	unsigned long _tmp = fd / __NFDBITS;
-	unsigned long _rem = fd % __NFDBITS;
-	return (p->fds_bits[_tmp] & (1UL<<_rem)) != 0;
-}
-
-/*
- * This will unroll the loop for the normal constant cases (8 or 32 longs,
- * for 256 and 1024-bit fd_sets respectively)
- */
-#undef __FD_ZERO
-static __inline__ void __FD_ZERO(__kernel_fd_set *p)
-{
-	unsigned long *tmp = p->fds_bits;
-	int i;
-
-	if (__builtin_constant_p(__FDSET_LONGS)) {
-		switch (__FDSET_LONGS) {
-			case 32:
-			  tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
-			  tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
-			  tmp[ 8] = 0; tmp[ 9] = 0; tmp[10] = 0; tmp[11] = 0;
-			  tmp[12] = 0; tmp[13] = 0; tmp[14] = 0; tmp[15] = 0;
-			  tmp[16] = 0; tmp[17] = 0; tmp[18] = 0; tmp[19] = 0;
-			  tmp[20] = 0; tmp[21] = 0; tmp[22] = 0; tmp[23] = 0;
-			  tmp[24] = 0; tmp[25] = 0; tmp[26] = 0; tmp[27] = 0;
-			  tmp[28] = 0; tmp[29] = 0; tmp[30] = 0; tmp[31] = 0;
-			  return;
-			case 16:
-			  tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
-			  tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
-			  tmp[ 8] = 0; tmp[ 9] = 0; tmp[10] = 0; tmp[11] = 0;
-			  tmp[12] = 0; tmp[13] = 0; tmp[14] = 0; tmp[15] = 0;
-			  return;
-			case 8:
-			  tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
-			  tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
-			  return;
-			case 4:
-			  tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
-			  return;
-		}
-	}
-	i = __FDSET_LONGS;
-	while (i) {
-		i--;
-		*tmp = 0;
-		tmp++;
-	}
-}
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-
 #endif /* !(__ARCH_SPARC_POSIX_TYPES_H) */
diff --git a/include/asm-sparc64/posix_types.h b/include/asm-sparc64/posix_types.h
index c86b945..3c99210 100644
--- a/include/asm-sparc64/posix_types.h
+++ b/include/asm-sparc64/posix_types.h
@@ -1,6 +1,8 @@
 #ifndef __ARCH_SPARC64_POSIX_TYPES_H
 #define __ARCH_SPARC64_POSIX_TYPES_H
 
+#include <linux/fdset.h>
+
 /*
  * This file is generally used by user-level software, so you need to
  * be a little careful about namespace pollution etc.  Also, we cannot
@@ -50,77 +52,4 @@ typedef struct {
 #endif /* !defined(__KERNEL__) && !defined(__USE_ALL) */
 } __kernel_fsid_t;
 
-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef __FD_SET
-static __inline__ void __FD_SET(unsigned long fd, __kernel_fd_set *fdsetp)
-{
-	unsigned long _tmp = fd / __NFDBITS;
-	unsigned long _rem = fd % __NFDBITS;
-	fdsetp->fds_bits[_tmp] |= (1UL<<_rem);
-}
-
-#undef __FD_CLR
-static __inline__ void __FD_CLR(unsigned long fd, __kernel_fd_set *fdsetp)
-{
-	unsigned long _tmp = fd / __NFDBITS;
-	unsigned long _rem = fd % __NFDBITS;
-	fdsetp->fds_bits[_tmp] &= ~(1UL<<_rem);
-}
-
-#undef __FD_ISSET
-static __inline__ int __FD_ISSET(unsigned long fd, __const__ __kernel_fd_set *p)
-{ 
-	unsigned long _tmp = fd / __NFDBITS;
-	unsigned long _rem = fd % __NFDBITS;
-	return (p->fds_bits[_tmp] & (1UL<<_rem)) != 0;
-}
-
-/*
- * This will unroll the loop for the normal constant cases (8 or 32 longs,
- * for 256 and 1024-bit fd_sets respectively)
- */
-#undef __FD_ZERO
-static __inline__ void __FD_ZERO(__kernel_fd_set *p)
-{
-	unsigned long *tmp = p->fds_bits;
-	int i;
-
-	if (__builtin_constant_p(__FDSET_LONGS)) {
-		switch (__FDSET_LONGS) {
-			case 32:
-			  tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
-			  tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
-			  tmp[ 8] = 0; tmp[ 9] = 0; tmp[10] = 0; tmp[11] = 0;
-			  tmp[12] = 0; tmp[13] = 0; tmp[14] = 0; tmp[15] = 0;
-			  tmp[16] = 0; tmp[17] = 0; tmp[18] = 0; tmp[19] = 0;
-			  tmp[20] = 0; tmp[21] = 0; tmp[22] = 0; tmp[23] = 0;
-			  tmp[24] = 0; tmp[25] = 0; tmp[26] = 0; tmp[27] = 0;
-			  tmp[28] = 0; tmp[29] = 0; tmp[30] = 0; tmp[31] = 0;
-			  return;
-			case 16:
-			  tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
-			  tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
-			  tmp[ 8] = 0; tmp[ 9] = 0; tmp[10] = 0; tmp[11] = 0;
-			  tmp[12] = 0; tmp[13] = 0; tmp[14] = 0; tmp[15] = 0;
-			  return;
-			case 8:
-			  tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
-			  tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
-			  return;
-			case 4:
-			  tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
-			  return;
-		}
-	}
-	i = __FDSET_LONGS;
-	while (i) {
-		i--;
-		*tmp = 0;
-		tmp++;
-	}
-}
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-
 #endif /* !(__ARCH_SPARC64_POSIX_TYPES_H) */
diff --git a/include/asm-v850/posix_types.h b/include/asm-v850/posix_types.h
index ccb7297..1c6e0bb 100644
--- a/include/asm-v850/posix_types.h
+++ b/include/asm-v850/posix_types.h
@@ -14,6 +14,8 @@
 #ifndef __V850_POSIX_TYPES_H__
 #define __V850_POSIX_TYPES_H__
 
+#include <linux/fdset.h>
+
 typedef unsigned long	__kernel_ino_t;
 typedef unsigned long long __kernel_ino64_t;
 typedef unsigned int	__kernel_mode_t;
@@ -52,25 +54,4 @@ typedef struct {
 } __kernel_fsid_t;
 
 
-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-/* We used to include <asm/bitops.h> here, which seems the right thing, but
-   it caused nasty include-file definition order problems.  Removing the
-   include seems to work, so fingers crossed...  */
-
-#undef	__FD_SET
-#define __FD_SET(fd, fd_set) \
-  __set_bit (fd, (void *)&((__kernel_fd_set *)fd_set)->fds_bits)
-#undef __FD_CLR
-#define __FD_CLR(fd, fd_set) \
-  __clear_bit (fd, (void *)&((__kernel_fd_set *)fd_set)->fds_bits)
-#undef	__FD_ISSET
-#define __FD_ISSET(fd, fd_set) \
-  __test_bit (fd, (void *)&((__kernel_fd_set *)fd_set)->fds_bits)
-#undef	__FD_ZERO
-#define __FD_ZERO(fd_set) \
-  memset (fd_set, 0, sizeof (*(fd_set *)fd_set))
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-
 #endif /* __V850_POSIX_TYPES_H__ */
diff --git a/include/asm-x86_64/posix_types.h b/include/asm-x86_64/posix_types.h
index 9926aa4..ce54799 100644
--- a/include/asm-x86_64/posix_types.h
+++ b/include/asm-x86_64/posix_types.h
@@ -1,6 +1,8 @@
 #ifndef _ASM_X86_64_POSIX_TYPES_H
 #define _ASM_X86_64_POSIX_TYPES_H
 
+#include <linux/fdset.h>
+
 /*
  * This file is generally used by user-level software, so you need to
  * be a little careful about namespace pollution etc.  Also, we cannot
@@ -43,77 +45,4 @@ typedef __kernel_gid_t __kernel_gid32_t;
 
 typedef unsigned long	__kernel_old_dev_t;
 
-#ifdef __KERNEL__
-
-#undef __FD_SET
-static __inline__ void __FD_SET(unsigned long fd, __kernel_fd_set *fdsetp)
-{
-	unsigned long _tmp = fd / __NFDBITS;
-	unsigned long _rem = fd % __NFDBITS;
-	fdsetp->fds_bits[_tmp] |= (1UL<<_rem);
-}
-
-#undef __FD_CLR
-static __inline__ void __FD_CLR(unsigned long fd, __kernel_fd_set *fdsetp)
-{
-	unsigned long _tmp = fd / __NFDBITS;
-	unsigned long _rem = fd % __NFDBITS;
-	fdsetp->fds_bits[_tmp] &= ~(1UL<<_rem);
-}
-
-#undef __FD_ISSET
-static __inline__ int __FD_ISSET(unsigned long fd, __const__ __kernel_fd_set *p)
-{
-	unsigned long _tmp = fd / __NFDBITS;
-	unsigned long _rem = fd % __NFDBITS;
-	return (p->fds_bits[_tmp] & (1UL<<_rem)) != 0;
-}
-
-/*
- * This will unroll the loop for the normal constant cases (8 or 32 longs,
- * for 256 and 1024-bit fd_sets respectively)
- */
-#undef __FD_ZERO
-static __inline__ void __FD_ZERO(__kernel_fd_set *p)
-{
-	unsigned long *tmp = p->fds_bits;
-	int i;
-
-	if (__builtin_constant_p(__FDSET_LONGS)) {
-		switch (__FDSET_LONGS) {
-			case 32:
-			  tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
-			  tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
-			  tmp[ 8] = 0; tmp[ 9] = 0; tmp[10] = 0; tmp[11] = 0;
-			  tmp[12] = 0; tmp[13] = 0; tmp[14] = 0; tmp[15] = 0;
-			  tmp[16] = 0; tmp[17] = 0; tmp[18] = 0; tmp[19] = 0;
-			  tmp[20] = 0; tmp[21] = 0; tmp[22] = 0; tmp[23] = 0;
-			  tmp[24] = 0; tmp[25] = 0; tmp[26] = 0; tmp[27] = 0;
-			  tmp[28] = 0; tmp[29] = 0; tmp[30] = 0; tmp[31] = 0;
-			  return;
-			case 16:
-			  tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
-			  tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
-			  tmp[ 8] = 0; tmp[ 9] = 0; tmp[10] = 0; tmp[11] = 0;
-			  tmp[12] = 0; tmp[13] = 0; tmp[14] = 0; tmp[15] = 0;
-			  return;
-			case 8:
-			  tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
-			  tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
-			  return;
-			case 4:
-			  tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
-			  return;
-		}
-	}
-	i = __FDSET_LONGS;
-	while (i) {
-		i--;
-		*tmp = 0;
-		tmp++;
-	}
-}
-
-#endif /* defined(__KERNEL__) */
-
 #endif
diff --git a/include/asm-xtensa/posix_types.h b/include/asm-xtensa/posix_types.h
index 2c816b0..3d1309a 100644
--- a/include/asm-xtensa/posix_types.h
+++ b/include/asm-xtensa/posix_types.h
@@ -13,6 +13,8 @@
 #ifndef _XTENSA_POSIX_TYPES_H
 #define _XTENSA_POSIX_TYPES_H
 
+#include <linux/fdset.h>
+
 /*
  * This file is generally used by user-level software, so you need to
  * be a little careful about namespace pollution etc.  Also, we cannot
@@ -54,70 +56,4 @@ typedef struct {
 	int	val[2];
 } __kernel_fsid_t;
 
-#ifndef __GNUC__
-
-#define	__FD_SET(d, set)	((set)->fds_bits[__FDELT(d)] |= __FDMASK(d))
-#define	__FD_CLR(d, set)	((set)->fds_bits[__FDELT(d)] &= ~__FDMASK(d))
-#define	__FD_ISSET(d, set)	((set)->fds_bits[__FDELT(d)] & __FDMASK(d))
-#define	__FD_ZERO(set)	\
-  ((void) memset ((__ptr_t) (set), 0, sizeof (__kernel_fd_set)))
-
-#else /* __GNUC__ */
-
-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) \
-    || (__GLIBC__ == 2 && __GLIBC_MINOR__ == 0)
-/* With GNU C, use inline functions instead so args are evaluated only once: */
-
-#undef __FD_SET
-static __inline__ void __FD_SET(unsigned long fd, __kernel_fd_set *fdsetp)
-{
-	unsigned long _tmp = fd / __NFDBITS;
-	unsigned long _rem = fd % __NFDBITS;
-	fdsetp->fds_bits[_tmp] |= (1UL<<_rem);
-}
-
-#undef __FD_CLR
-static __inline__ void __FD_CLR(unsigned long fd, __kernel_fd_set *fdsetp)
-{
-	unsigned long _tmp = fd / __NFDBITS;
-	unsigned long _rem = fd % __NFDBITS;
-	fdsetp->fds_bits[_tmp] &= ~(1UL<<_rem);
-}
-
-#undef __FD_ISSET
-static __inline__ int __FD_ISSET(unsigned long fd, __kernel_fd_set *p)
-{
-	unsigned long _tmp = fd / __NFDBITS;
-	unsigned long _rem = fd % __NFDBITS;
-	return (p->fds_bits[_tmp] & (1UL<<_rem)) != 0;
-}
-
-/*
- * This will unroll the loop for the normal constant case (8 ints,
- * for a 256-bit fd_set)
- */
-#undef __FD_ZERO
-static __inline__ void __FD_ZERO(__kernel_fd_set *p)
-{
-	unsigned int *tmp = (unsigned int *)p->fds_bits;
-	int i;
-
-	if (__builtin_constant_p(__FDSET_LONGS)) {
-		switch (__FDSET_LONGS) {
-			case 8:
-				tmp[0] = 0; tmp[1] = 0; tmp[2] = 0; tmp[3] = 0;
-				tmp[4] = 0; tmp[5] = 0; tmp[6] = 0; tmp[7] = 0;
-				return;
-		}
-	}
-	i = __FDSET_LONGS;
-	while (i) {
-		i--;
-		*tmp = 0;
-		tmp++;
-	}
-}
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-#endif /* __GNUC__ */
 #endif /* _XTENSA_POSIX_TYPES_H */
diff --git a/include/kabi/fdset.h b/include/kabi/fdset.h
new file mode 100644
index 0000000..57a9df9
--- /dev/null
+++ b/include/kabi/fdset.h
@@ -0,0 +1,49 @@
+#ifndef  __KABI_FDSET_H
+# define __KABI_FDSET_H 1
+
+# include <kabi/arch/stddef.h>
+
+/*
+ * This allows for 1024 file descriptors: if NR_OPEN is ever grown beyond 
+ * that you'll have to change this too. But 1024 fd's seem to be enough even 
+ * for such "real" unices like OSF/1, so hopefully this is one limit that 
+ * doesn't have to be changed [again].
+ *
+ * TODO: This should probably use the generic bitops in some form, since it's 
+ * using unsigned longs (as do the generic bitops), and has an identical 
+ * in-memory layout on all architectures.
+ */
+
+# define __KABI_NFDBITS     __KABI_BITS_PER_LONG
+# define __KABI_FD_SETSIZE  (1024UL)
+# define __KABI_FDSET_LONGS (__KABI_FD_SETSIZE/__KABI_NFDBITS)
+# define __KABI_FDELT(fd)   ((fd)/__KABI_NFDBITS)
+# define __KABI_FDMASK(fd)  (1UL << ((fd) % __KABI_NFDBITS))
+
+typedef struct {
+	unsigned long fds_bits [__KABI_FDSET_LONGS];
+} __kabi_fd_set;
+
+static __inline__ void __KABI_FD_SET(unsigned long fd, __kabi_fd_set *p)
+{
+	p->fds_bits[__KABI_FDELT(fd)] |= __KABI_FDMASK(fd);
+}
+
+static __inline__ void __KABI_FD_CLR(unsigned long fd, __kabi_fd_set *p)
+{
+	p->fds_bits[__KABI_FDELT(fd)] &= ~__KABI_FDMASK(fd);
+}
+
+static __inline__ int __KABI_FD_ISSET(unsigned long fd, __kabi_fd_set *p)
+{
+	return !!(p->fds_bits[__KABI_FDELT(fd)] & __KABI_FDMASK(fd));
+}
+
+static __inline__ void __KABI_FD_ZERO(__kabi_fd_set *p)
+{
+	unsigned long i;
+	for (i = 0; i < __KABI_FDSET_LONGS; i++)
+		p->fds_bits[i] = 0;
+}
+
+#endif /* not __KABI_FDSET_H */
diff --git a/include/linux/fdset.h b/include/linux/fdset.h
new file mode 100644
index 0000000..1e92ba4
--- /dev/null
+++ b/include/linux/fdset.h
@@ -0,0 +1,15 @@
+#ifndef  __LINUX_FDSET_H
+# define __LINUX_FDSET_H 1
+
+# include <kabi/fdset.h>
+
+# define __NFDBITS		__KABI_NFDBITS
+# define __FD_SETSIZE		__KABI_FD_SETSIZE
+# define __FD_SET(fd,set)	__KABI_FD_SET(fd,set)
+# define __FD_CLR(fd,set)	__KABI_FD_CLR(fd,set)
+# define __FD_ISSET(fd,set)	__KABI_FD_ISSET(fd,set)
+# define __FD_ZERO(set)		__KABI_FD_ZERO(set)
+
+typedef __kabi_fd_set __kernel_fd_set;
+
+#endif /* not __LINUX_FDSET_H */
diff --git a/include/linux/posix_types.h b/include/linux/posix_types.h
index f04c98c..1b571da 100644
--- a/include/linux/posix_types.h
+++ b/include/linux/posix_types.h
@@ -2,40 +2,7 @@
 #define _LINUX_POSIX_TYPES_H
 
 #include <linux/stddef.h>
-
-/*
- * This allows for 1024 file descriptors: if NR_OPEN is ever grown
- * beyond that you'll have to change this too. But 1024 fd's seem to be
- * enough even for such "real" unices like OSF/1, so hopefully this is
- * one limit that doesn't have to be changed [again].
- *
- * Note that POSIX wants the FD_CLEAR(fd,fdsetp) defines to be in
- * <sys/time.h> (and thus <linux/time.h>) - but this is a more logical
- * place for them. Solved by having dummy defines in <sys/time.h>.
- */
-
-/*
- * Those macros may have been defined in <gnu/types.h>. But we always
- * use the ones here. 
- */
-#undef __NFDBITS
-#define __NFDBITS	(8 * sizeof(unsigned long))
-
-#undef __FD_SETSIZE
-#define __FD_SETSIZE	1024
-
-#undef __FDSET_LONGS
-#define __FDSET_LONGS	(__FD_SETSIZE/__NFDBITS)
-
-#undef __FDELT
-#define	__FDELT(d)	((d) / __NFDBITS)
-
-#undef __FDMASK
-#define	__FDMASK(d)	(1UL << ((d) % __NFDBITS))
-
-typedef struct {
-	unsigned long fds_bits [__FDSET_LONGS];
-} __kernel_fd_set;
+#include <linux/fdset.h>
 
 /* Type of a signal handler.  */
 typedef void (*__kernel_sighandler_t)(int);
diff --git a/include/linux/time.h b/include/linux/time.h
index bf0e785..e2dbfa9 100644
--- a/include/linux/time.h
+++ b/include/linux/time.h
@@ -150,13 +150,13 @@ extern struct timeval ns_to_timeval(cons
 
 #endif /* __KERNEL__ */
 
-#define NFDBITS			__NFDBITS
-
-#define FD_SETSIZE		__FD_SETSIZE
-#define FD_SET(fd,fdsetp)	__FD_SET(fd,fdsetp)
-#define FD_CLR(fd,fdsetp)	__FD_CLR(fd,fdsetp)
-#define FD_ISSET(fd,fdsetp)	__FD_ISSET(fd,fdsetp)
-#define FD_ZERO(fdsetp)		__FD_ZERO(fdsetp)
+#include <kabi/fdset.h>
+#define NFDBITS			__KABI_NFDBITS
+#define FD_SETSIZE		__KABI_FD_SETSIZE
+#define FD_SET(fd,set)		__KABI_FD_SET(fd,set)
+#define FD_CLR(fd,set)		__KABI_FD_CLR(fd,set)
+#define FD_ISSET(fd,set)	__KABI_FD_ISSET(fd,set)
+#define FD_ZERO(set)		__KABI_FD_ZERO(set)
 
 /*
  * Names of the interval timers, and structure

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Kyle Moffett]

On Sun, 26 Mar 2006 06:52:05 -0500 Kyle Moffett <mrmacman_g4@mac.com> wrote:
> 2)  Since most of the headers are currently quite broken with respect to
>     GLIBC and userspace, I won't spend much extra time preserving
>     compatibility with GLIBC, userspace, or non-GCC compilers.

That didn't come out right, but what I meant to say was this:  Since the 
headers in include/linux are quite broken with respect to GLIBC and 
userspace, I won't let so-called "compatibility" code like this get in 
the way:
  #ifndef __GNUC__
  #define DO_SOMETHING(foo) ICKY_MACRO
  #else
  static __inline__ void DO_SOMETHING(int foo)
  {
  	sensible_inline_function();
  }
  #endif

You can see where I take that approach in the patches I sent.

One other thing I would like to point out:  The fd_set code wants to use
__set_bit and __clear_bit from <linux/bitops.h>, but those really should
not be accessible to userspace directly.  I would like to propose moving
that functionality into <__klib/*.h> from which it would be accessible
to both <linux/*.h> and <kabi/*.h>.  I think this would also help with 
the UML header issues by providing those kernel-internal APIs to the 
kernel when run from userspace. (Please correct me if I'm wrong).

I appreciate your comments and corrections, thanks!

Cheers,
Kyle Moffett

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Nix]

On Sun, 26 Mar 2006, Kyle Moffett said:
> On Sun, 26 Mar 2006 06:52:05 -0500 Kyle Moffett <mrmacman_g4@mac.com> wrote:
>> 2)  Since most of the headers are currently quite broken with respect to
>>     GLIBC and userspace, I won't spend much extra time preserving
>>     compatibility with GLIBC, userspace, or non-GCC compilers.
> 
> That didn't come out right, but what I meant to say was this:  Since the 
> headers in include/linux are quite broken with respect to GLIBC and 
> userspace, I won't let so-called "compatibility" code like this get in 
> the way:

As long as the eventual goal is something that *works* with glibc and
uClibc and the rest of them, that's good enough as far as I can tell.
Obviously compatibility crud *in the headers* which serves no obvious
purpose and which userspace doesn't actually need should die (and
there's certainly some like that lying around).

-- 
`Come now, you should know that whenever you plan the duration of your
 unplanned downtime, you should add in padding for random management
 freakouts.'

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Arjan van de Ven]

On Sun, 2006-03-26 at 06:52 -0500, Kyle Moffett wrote:
> On Fri, 24 Mar 2006 17:46:27 -0500 Kyle Moffett <mrmacman_g4@mac.com> wrote:
> > I'm working on some sample patches now which I'll try to post in a
> > few days if I get the time.
> 
> Ok, here's a sample of the KABI conversion and cleanup patches that I'm
> proposing.  I have a few fundamental goals for these patches:

is KABI the right name? I mean.. from the kernel pov it's the interface
to userspace ;)

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Arjan van de Ven]

On Sun, 2006-03-26 at 14:26 +0200, Arjan van de Ven wrote:
> On Sun, 2006-03-26 at 06:52 -0500, Kyle Moffett wrote:
> > On Fri, 24 Mar 2006 17:46:27 -0500 Kyle Moffett <mrmacman_g4@mac.com> wrote:
> > > I'm working on some sample patches now which I'll try to post in a
> > > few days if I get the time.
> > 
> > Ok, here's a sample of the KABI conversion and cleanup patches that I'm
> > proposing.  I have a few fundamental goals for these patches:
> 
> is KABI the right name? I mean.. from the kernel pov it's the interface
> to userspace ;)

ok proposal: just use "abi" without the K; it is THE abi, and the only
one the kernel has, so no need for prefixes or suffixes.

(BTW I'm 100% behind the idea of a clean set of seperate ABI headers)

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Kyle Moffett]

On Mar 26, 2006, at 07:26:48, Arjan van de Ven wrote:
> On Sun, 2006-03-26 at 06:52 -0500, Kyle Moffett wrote:
>> On Fri, 24 Mar 2006 17:46:27 -0500 Kyle Moffett  
>> <mrmacman_g4@mac.com> wrote:
>>> I'm working on some sample patches now which I'll try to post in  
>>> a few days if I get the time.
>>
>> Ok, here's a sample of the KABI conversion and cleanup patches  
>> that I'm proposing.  I have a few fundamental goals for these  
>> patches:
>
> is KABI the right name? I mean.. from the kernel pov it's the  
> interface to userspace ;)

Well I guess you could call it UABI, but that might also imply that  
it's _userspace_ that defines the interface, instead of the kernel.   
Since the headers themselves are rather tightly coupled with the  
kernel, I think I'll stick with the KABI name for now (unless  
somebody can come up with a better one, of course :-D).

Cheers,
Kyle Moffett

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Giuseppe Bilotta]

On Sun, 26 Mar 2006 07:34:12 -0500, Kyle Moffett wrote:

> On Mar 26, 2006, at 07:26:48, Arjan van de Ven wrote:
>> On Sun, 2006-03-26 at 06:52 -0500, Kyle Moffett wrote:
>>> On Fri, 24 Mar 2006 17:46:27 -0500 Kyle Moffett  
>>> <mrmacman_g4@mac.com> wrote:
>>>> I'm working on some sample patches now which I'll try to post in  
>>>> a few days if I get the time.
>>>
>>> Ok, here's a sample of the KABI conversion and cleanup patches  
>>> that I'm proposing.  I have a few fundamental goals for these  
>>> patches:
>>
>> is KABI the right name? I mean.. from the kernel pov it's the  
>> interface to userspace ;)
> 
> Well I guess you could call it UABI, but that might also imply that  
> it's _userspace_ that defines the interface, instead of the kernel.   
> Since the headers themselves are rather tightly coupled with the  
> kernel, I think I'll stick with the KABI name for now (unless  
> somebody can come up with a better one, of course :-D).

Well, since I guess UAKDABI (User Accessible Kernel Defined) is out of
the question, what about LABI? (Linux ABI)

-- 
Giuseppe "Oblomov" Bilotta

Axiom I of the Giuseppe Bilotta
theory of IT:
Anything is better than MS

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Avi Kivity]

Kyle Moffett wrote:
>
> Well I guess you could call it UABI, but that might also imply that 
> it's _userspace_ that defines the interface, instead of the kernel.  
> Since the headers themselves are rather tightly coupled with the 
> kernel, I think I'll stick with the KABI name for now (unless somebody 
> can come up with a better one, of course :-D).
>
How about __linux, or __linux_abi? There are ABIs for other components, 
and other OSes. Linux is the name of the project after all.

-- 
error compiling committee.c: too many arguments to function

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Kyle Moffett]

On Mar 26, 2006, at 08:29:49, Avi Kivity wrote:
> Kyle Moffett wrote:
>> Well I guess you could call it UABI, but that might also imply  
>> that it's _userspace_ that defines the interface, instead of the  
>> kernel.  Since the headers themselves are rather tightly coupled  
>> with the kernel, I think I'll stick with the KABI name for now  
>> (unless somebody can come up with a better one, of course :-D).
>
> How about __linux, or __linux_abi? There are ABIs for other  
> components, and other OSes. Linux is the name of the project after  
> all.

The other thing that I quickly noticed while writing up the patches  
is that it's kind of tedious typing __kabi_ over and over again.  I  
actually did first try with __linux_abi_ but the typing effort and  
finger cramps made me give up on that really quickly.

Cheers,
Kyle Moffett

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Giuseppe Bilotta]

On Sun, 26 Mar 2006 08:47:51 -0500, Kyle Moffett wrote:

> The other thing that I quickly noticed while writing up the patches  
> is that it's kind of tedious typing __kabi_ over and over again.  I  
> actually did first try with __linux_abi_ but the typing effort and  
> finger cramps made me give up on that really quickly.

What kind of barebone editor are you using that doesn't offer a way
(through macro or automatic completion or abbreviations or
substitution or plain copy and paste or whatever else) to shorten that
typing effort?

-- 
Giuseppe "Oblomov" Bilotta

[W]hat country can preserve its liberties, if its rulers are not
warned from time to time that [the] people preserve the spirit of
resistance? Let them take arms...The tree of liberty must be
refreshed from time to time, with the blood of patriots and
tyrants.
	-- Thomas Jefferson, letter to Col. William S. Smith, 1787

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Kyle Moffett]

On Mar 26, 2006, at 08:53:05, Giuseppe Bilotta wrote:
> On Sun, 26 Mar 2006 08:47:51 -0500, Kyle Moffett wrote:
>> The other thing that I quickly noticed while writing up the  
>> patches is that it's kind of tedious typing __kabi_ over and over  
>> again.  I actually did first try with __linux_abi_ but the typing  
>> effort and finger cramps made me give up on that really quickly.
>
> What kind of barebone editor are you using that doesn't offer a way  
> (through macro or automatic completion or abbreviations or  
> substitution or plain copy and paste or whatever else) to shorten  
> that typing effort?

Well, actually I was using vim, and although I know how to configure  
shortcuts like that; do we really want to force people to use  
shortcuts and abbreviations to be able to sanely edit the ABI code?   
Isn't that counterproductive with the desire to get the maintainers  
fixing up their own ABIs?  Plus all the extra verbiage tends to make  
the lines too long (80-column limit) and makes it harder to read.

Cheers,
Kyle Moffett

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Giuseppe Bilotta]

On Sun, 26 Mar 2006 09:30:32 -0500, Kyle Moffett wrote:

> On Mar 26, 2006, at 08:53:05, Giuseppe Bilotta wrote:
>> On Sun, 26 Mar 2006 08:47:51 -0500, Kyle Moffett wrote:
>>> The other thing that I quickly noticed while writing up the  
>>> patches is that it's kind of tedious typing __kabi_ over and over  
>>> again.  I actually did first try with __linux_abi_ but the typing  
>>> effort and finger cramps made me give up on that really quickly.
>>
>> What kind of barebone editor are you using that doesn't offer a way  
>> (through macro or automatic completion or abbreviations or  
>> substitution or plain copy and paste or whatever else) to shorten  
>> that typing effort?
> 
> Well, actually I was using vim, and although I know how to configure  
> shortcuts like that; do we really want to force people to use  
> shortcuts and abbreviations to be able to sanely edit the ABI code?   
> Isn't that counterproductive with the desire to get the maintainers  
> fixing up their own ABIs?  Plus all the extra verbiage tends to make  
> the lines too long (80-column limit) and makes it harder to read.

Oh, I'm not exactly a fan of verbosity myself (essentially for the
lines too long and hard to read), which is why I suggested __labi_ as
a prefix --I wasn't arguing in favour of __linux_abi_.

However, as far as I know all programmer-friendly editors have some
form of easy to use code-completion feature (e.g. in vim ctrl+n/ctrl+p
are very handy without the need to define specific shortcuts, and the
new completion code in the future vim 7 is extremely promising ...)
and while I do agree that __linux_abi_ is too long, you remarked
yourself __[ukl]abi_ is still annoying, albeit shorter :) *Any* prefix
chosen will benefit from completion ;)

-- 
Giuseppe "Oblomov" Bilotta

"They that can give up essential liberty to obtain
a little temporary safety deserve neither liberty
nor safety." Benjamin Franklin

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Avi Kivity]

Kyle Moffett wrote:
> On Mar 26, 2006, at 08:29:49, Avi Kivity wrote:
>> Kyle Moffett wrote:
>>> Well I guess you could call it UABI, but that might also imply that 
>>> it's _userspace_ that defines the interface, instead of the kernel.  
>>> Since the headers themselves are rather tightly coupled with the 
>>> kernel, I think I'll stick with the KABI name for now (unless 
>>> somebody can come up with a better one, of course :-D).
>>
>> How about __linux, or __linux_abi? There are ABIs for other 
>> components, and other OSes. Linux is the name of the project after all.
>
> The other thing that I quickly noticed while writing up the patches is 
> that it's kind of tedious typing __kabi_ over and over again.  I 
> actually did first try with __linux_abi_ but the typing effort and 
> finger cramps made me give up on that really quickly.
#define _LA(x) __linux_abi_##x

struct _LA(whatever) {
    int foo;
    int bar;
};

struct _LA(another) {
    ...
};

#undef _LA

?

-- 
error compiling committee.c: too many arguments to function

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Arjan van de Ven]

On Sun, 2006-03-26 at 19:24 +0200, Avi Kivity wrote:
> Kyle Moffett wrote:
> > On Mar 26, 2006, at 08:29:49, Avi Kivity wrote:
> >> Kyle Moffett wrote:
> >>> Well I guess you could call it UABI, but that might also imply that 
> >>> it's _userspace_ that defines the interface, instead of the kernel.  
> >>> Since the headers themselves are rather tightly coupled with the 
> >>> kernel, I think I'll stick with the KABI name for now (unless 
> >>> somebody can come up with a better one, of course :-D).
> >>
> >> How about __linux, or __linux_abi? There are ABIs for other 
> >> components, and other OSes. Linux is the name of the project after all.
> >
> > The other thing that I quickly noticed while writing up the patches is 
> > that it's kind of tedious typing __kabi_ over and over again.  I 
> > actually did first try with __linux_abi_ but the typing effort and 
> > finger cramps made me give up on that really quickly.
> #define _LA(x) __linux_abi_##x
> 
> struct _LA(whatever) {
>     int foo;
>     int bar;
> };
> 
> struct _LA(another) {
>     ...
> };

this is a good sign that this is all very over designed :)

namespace pollution is perhaps evil, but we also should not overreact.
Especially for struct names. *IF* they are in a "narrow enough" header,
the user of the header knows what he is doing, and accepts these to be
in his namespace.

The problem is things like u64 etc that is VERY common in all headers,
but then again __u64 etc are just fine, history has proven that already.

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Avi Kivity]

Arjan van de Ven wrote:
>> struct _LA(whatever) {
>>     int foo;
>>     int bar;
>> };
>>
>> struct _LA(another) {
>>     ...
>> };
>>     
>
> this is a good sign that this is all very over designed :)
>
>   
It's an eyesore, isn't it? :)
> namespace pollution is perhaps evil, but we also should not overreact.
> Especially for struct names. *IF* they are in a "narrow enough" header,
> the user of the header knows what he is doing, and accepts these to be
> in his namespace.
>   
This is true for a small enough application. But things grow, libraries 
are added, and includes keep pulling other includes in. Sooner or later 
you'll have a collision.
> The problem is things like u64 etc that is VERY common in all headers,
> but then again __u64 etc are just fine, history has proven that already.
>   
Agree. But to be on the safe side one can use uint64_t and friends 
(which the kernel can typedef to u64 and first degree relatives)

-- 
error compiling committee.c: too many arguments to function

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Arjan van de Ven]

\
> This is true for a small enough application. But things grow, libraries 
> are added, and includes keep pulling other includes in. Sooner or later 
> you'll have a collision.
> > The problem is things like u64 etc that is VERY common in all headers,
> > but then again __u64 etc are just fine, history has proven that already.
> >   
> Agree. But to be on the safe side one can use uint64_t and friends 
> (which the kernel can typedef to u64 and first degree relatives)


actually uint64_t is a bigger namespace pollution than __u64 ;)

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Rob Landley]

On Sunday 26 March 2006 12:57 pm, Avi Kivity wrote:
> This is true for a small enough application. But things grow, libraries
> are added, and includes keep pulling other includes in. Sooner or later
> you'll have a collision.

And you'll fix it when it happens.  Fact of life.

> > The problem is things like u64 etc that is VERY common in all headers,
> > but then again __u64 etc are just fine, history has proven that already.
>
> Agree. But to be on the safe side one can use uint64_t and friends
> (which the kernel can typedef to u64 and first degree relatives)

Now that the kernel no longer targets gcc before 3.2, c99 names are merely 
ugly rather than an actual problem. :)

Rob
-- 
Never bet against the cheap plastic solution.

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Kyle Moffett]

On Mar 26, 2006, at 16:18:29, Rob Landley wrote:
>>> The problem is things like u64 etc that is VERY common in all  
>>> headers, but then again __u64 etc are just fine, history has  
>>> proven that already.
>>
>> Agree. But to be on the safe side one can use uint64_t and friends  
>> (which the kernel can typedef to u64 and first degree relatives)
>
> Now that the kernel no longer targets gcc before 3.2, c99 names are  
> merely ugly rather than an actual problem. :)

No, they're still a problem in ABI headers because they have defined  
visibility.  A userspace program that does not include <stdint.h>  
will not have those types put in its namespace.  Including <sys/ 
time.h> _cannot_ bring <stdint.> in automatically, that breaks the  
POSIX symbol visibility rules.

Cheers,
Kyle Moffett

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Avi Kivity]

Rob Landley wrote:
> On Sunday 26 March 2006 12:57 pm, Avi Kivity wrote:
>   
>> This is true for a small enough application. But things grow, libraries
>> are added, and includes keep pulling other includes in. Sooner or later
>> you'll have a collision.
>>     
>
> And you'll fix it when it happens.  Fact of life.
>   
Fixing it will mean breaking either the ABI of the kernel or of the 
large library you pulled in.

An ABI bug causes pain far beyond its size. Look at the trouble caused 
when some interfaces uses unsigned instead of u64. In kernel APIs, you 
just replace the type, but in the ABI, you add a new syscall or do some 
other hack.

Much better to get it right the first time, even if it's ugly. It's an 
ABI, not a beauty contest nominee.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Rob Landley]

On Monday 27 March 2006 1:19 am, Avi Kivity wrote:
> Rob Landley wrote:
> > On Sunday 26 March 2006 12:57 pm, Avi Kivity wrote:
> >> This is true for a small enough application. But things grow, libraries
> >> are added, and includes keep pulling other includes in. Sooner or later
> >> you'll have a collision.
> >
> > And you'll fix it when it happens.  Fact of life.
>
> Fixing it will mean breaking either the ABI of the kernel or of the
> large library you pulled in.
>
> An ABI bug causes pain far beyond its size. Look at the trouble caused
> when some interfaces uses unsigned instead of u64. In kernel APIs, you
> just replace the type, but in the ABI, you add a new syscall or do some
> other hack.

One of busybox's users has a libc built with improperly sanitized kernel 
headers that leak all the kernel's CONFIG_ symbols into the standard 
namespace, and we found out because one of them clashes with a busybox CONFIG 
symbol and it broke his build.

The easiest thing to do was rename our CONFIG symbol, which we did.  (And 
asked him to fix his system, which he didn't.)  This was a while ago now...

> Much better to get it right the first time, even if it's ugly. It's an
> ABI, not a beauty contest nominee.

I've been cc'd on all this because I cared about Mazur's kernel headers enough 
to email him several times.  But I can't say I really care about this new 
project that's taken over the discussion.

Either it doesn't break the existing interface, in which case it has no impact 
on me and I can ignore it, or it requires all existing userspace programs 
that care about the current state of things to change to care about your new 
thing instead, in which case it's probably doomed.  Unless the kernel picks 
it up and imposes a flag day, at which point I'll find out about it from LWN 
like everybody else.

Either way, it's not sounding like something I can grab and build uClibc 
systems with any time soon, the way I could use Mazur's headers to build 
uClibc.  I'll probably wind up using the gentoo headers when the 2.6.14 
version ships.

http://www.gentoo.org/cgi-bin/viewcvs.cgi/src/patchsets/gentoo-headers/?root=gentoo

*shrug*.  Good luck with whatever it is you're trying to accomplish.

Rob
-- 
Never bet against the cheap plastic solution.

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Mariusz Mazur]

On Monday 27 March 2006 21:48, Rob Landley wrote:
> Either way, it's not sounding like something I can grab and build uClibc
> systems with any time soon, the way I could use Mazur's headers to build
> uClibc.  I'll probably wind up using the gentoo headers when the 2.6.14
> version ships.

That's the trouble. While I have nothing against someone (as in -- not me :) 
doing all that abi separation, it's not something I can use straight away. 
Hell, I don't even get where in all of it I'd end up with something I could 
use (granted, I haven't looked into the thread too closely). Not to mention, 
that I suspect, that if there were enough people to do it, it would have 
gotten done two years ago.

So unless anybody's got a better idea, I'll try releasing some initial version 
of that llh-ng thingie rather soonish and see how that'll work out. Anybody 
with me on that? :)


-- 
Judge others by their intentions and yourself by your results.
                                                                 Guy Kawasaki
Education is an admirable thing, but it is well to remember from
time to time that nothing that is worth knowing can be taught.
                                                                  Oscar Wilde

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Kyle Moffett]

On Mar 28, 2006, at 15:04:53, Mariusz Mazur wrote:
> On Monday 27 March 2006 21:48, Rob Landley wrote:
>> Either way, it's not sounding like something I can grab and build  
>> uClibc systems with any time soon, the way I could use Mazur's  
>> headers to build uClibc.  I'll probably wind up using the gentoo  
>> headers when the 2.6.14 version ships.
>
> That's the trouble. While I have nothing against someone (as in --  
> not me :) doing all that abi separation, it's not something I can  
> use straight away. Hell, I don't even get where in all of it I'd  
> end up with something I could use (granted, I haven't looked into  
> the thread too closely). Not to mention, that I suspect, that if  
> there were enough people to do it, it would have gotten done two  
> years ago.
>
> So unless anybody's got a better idea, I'll try releasing some  
> initial version of that llh-ng thingie rather soonish and see how  
> that'll work out. Anybody with me on that? :)

I agree with this approach.  I don't think the llh-ng method will be  
maintainable in the long-term, which is why I'm doing the KABI  
patches, but I think that in the short term that's the fastest way to  
get a working set of headers against which to build glibc.

Cheers,
Kyle Moffett

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Rob Landley]

On Tuesday 28 March 2006 3:04 pm, Mariusz Mazur wrote:
> On Monday 27 March 2006 21:48, Rob Landley wrote:
> > Either way, it's not sounding like something I can grab and build uClibc
> > systems with any time soon, the way I could use Mazur's headers to build
> > uClibc.  I'll probably wind up using the gentoo headers when the 2.6.14
> > version ships.
>
> That's the trouble. While I have nothing against someone (as in -- not me
> :) doing all that abi separation, it's not something I can use straight
> away. Hell, I don't even get where in all of it I'd end up with something I
> could use (granted, I haven't looked into the thread too closely). Not to
> mention, that I suspect, that if there were enough people to do it, it
> would have gotten done two years ago.
>
> So unless anybody's got a better idea, I'll try releasing some initial
> version of that llh-ng thingie rather soonish and see how that'll work out.
> Anybody with me on that? :)

I'm highly interested in seeing the result. :)

Rob
-- 
Never bet against the cheap plastic solution.

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Rob Landley]

On Sunday 26 March 2006 7:34 am, Kyle Moffett wrote:
> Well I guess you could call it UABI, but that might also imply that
> it's _userspace_ that defines the interface, instead of the kernel.
> Since the headers themselves are rather tightly coupled with the
> kernel, I think I'll stick with the KABI name for now (unless
> somebody can come up with a better one, of course :-D).

Ok, question, using an example I'm familiar with.  What's affected by the 
prefix?

The busybox code for associating loopback devices with files was a problem for 
_years_.  The standard advice was not to include linux/loop.h, but instead 
cut and paste the structure out of the header into your own code.  
(Apparently, you're allowed to be uncomfortable if somebody suggests that 
this is the right way to get portability between bsd/cygwin/linux, but 
between 2.4 and 2.6 it's best practices.  Ok.)

Unfortunately, the cut and paste approach turned out to be incredibly painful 
because busybox needs to be portable between hardware platforms and between 
kernel versions.

Problem 1: The actual type of __kernel_dev_t varies between 
x86/arm/alpha/ppc/etc.  Any attempt to define our own loop ioctl structure 
meant we needed to either use the kernel-defined data type for this, or have 
an #ifdef forest for every possible platform somebody might want to build 
busybox on.  (This was exploded into a mess instantly.)

But using the kernel kdev_t type sucked too, because between 2.4 and 2.6 it 
got renamed to old_kdev_t so then we needed #ifdefs to check the kernel 
version of the headers we'd imported to see which name it was using.  (And it 
still broke anyway in strange various corner cases depending on which distro 
people were building under.)

We wandered through dozens of different approaches trying to avoid "#include 
<linux/loop.h>".  We had #ifdefs for glibc and #ifdefs for uclibc...  It was 
terrible.  About half the suggestions got checked into the tree at various 
points:
http://www.busybox.net/cgi-bin/viewcvs.cgi/trunk/busybox/libbb/loop.c?rev=14554&view=log

There were more that merely showed up on the list...

We eventually got it all untangled.  Now we check if we're on 2.6 and if so 
use the 64 bit API (which is stable on 2.6), and otherwise use a copy of the 
32 bit structure that predates __kernel_dev_t being renamed.  And we trust 
the darn kernel headers to supply the types for us, because there's just no 
alternative.

My question is: will your new approach work with our existing code, or are you 
saying we need another #ifdef to check for yet another gratuitously 
incompatible name, ala _kabi__kernel_dev_t?  Because I _really_ don't want to 
go there.  Nobody will be able to build existing versions of busybox (at 
least the mount/umount/losetup commands) against your new thing.  New 
versions that actually depended on _kabi_blah wouldn't build on any older 
distros unless we add another #ifdef...

This is just an example.  There is currently no existing program out in 
userspace that is tries to talk to the kernel using names with _kabi_ 
prepended to them, that I am aware of.  If you're adding this prefix, you're 
writing a brand new interface which currently has exactly zero users.  Is 
this the intent?

> Cheers,
> Kyle Moffett

Rob
-- 
Never bet against the cheap plastic solution.

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Kyle Moffett]

On Mar 26, 2006, at 15:55:51, Rob Landley wrote:
> On Sunday 26 March 2006 7:34 am, Kyle Moffett wrote:
>> Well I guess you could call it UABI, but that might also imply  
>> that it's _userspace_ that defines the interface, instead of the  
>> kernel. Since the headers themselves are rather tightly coupled  
>> with the kernel, I think I'll stick with the KABI name for now  
>> (unless somebody can come up with a better one, of course :-D).
>
> Ok, question, using an example I'm familiar with.  What's affected  
> by the prefix?
>
> We eventually got it all untangled.  Now we check if we're on 2.6  
> and if so use the 64 bit API (which is stable on 2.6), and  
> otherwise use a copy of the 32 bit structure that predates  
> __kernel_dev_t being renamed.  And we trust the darn kernel headers  
> to supply the types for us, because there's just no alternative.
>
> My question is: will your new approach work with our existing code,  
> or are you saying we need another #ifdef to check for yet another  
> gratuitously incompatible name, ala _kabi__kernel_dev_t?

No, no need for a new #ifdef, at least not for a long time if you  
don't want to.  My plans for the immediate future are only to touch  
files that are currently broken with respect to !__KERNEL__, and  
linux/loop.h isn't.  When I do get to it, my plan is to bring the  
parts that userspace needs into kabi/loop.h, named as __kabi_*,  
however linux/loop.h will define the new symbols to their old names  
for compatibility with linux code and userspace code that happens to  
be using those interfaces, so you still won't need to change  
anything.  Hopefully this would also mean that if the shipped linux/ 
loop.h is currently broken for your uses, I would be able to fix it  
so you don't need a customized copy at all.

If you _did_ decide to switch busybox to use the __kabi_ symbols, you  
would be able to take advantage of the fact that the same set of kabi  
headers (which don't include kernelversion.h) would work unmodified  
for kernels all the way back to 0.99, as long as you base your  
decisions of what features are available on either a dynamically- 
detected version or a pre-configured minimum version from the user.   
I'm assuming here that it's possible to compile a busybox that works  
on a 2.2 or 2.4 kernel even using linux-libc-headers 2.6.*, as long  
as the user indicates such.

> This is just an example.  There is currently no existing program  
> out in userspace that is tries to talk to the kernel using names  
> with _kabi_ prepended to them, that I am aware of.  If you're  
> adding this prefix, you're writing a brand new interface which  
> currently has exactly zero users.  Is this the intent?

I'd like this project to have a twofold purpose.  First, in the short  
term I want to fix up the general __KERNEL__ vs !__KERNEL__  
brokenness in include/linux and clean up and document which APIs are  
userspace ABI by moving them to include/kabi.  If that first part  
works, then in the very _long_ term, (on the order of a couple years  
or so) I would like to encourage larger projects like klibc and GLIBC  
to migrate to using *only* the include/kabi headers to avoid  
namespace pollution and allow kernel devs to rearrange and adjust the  
linux/* headers as they see fit without caring about breaking  
external programs.

Once the libcs are using kabi/*, GLIBC can pick up a set of linux/*.h  
"emulation" files based on kabi/*.h that provide all of the old  
source-APIs for as long as anybody cares, and Linux can go back to  
screwing around with its header files in any way that it pleases  
(just as long as it keeps ABI stuff all in kabi).  One of my major  
goals for this rework is to ensure that the kabi/*.h header files  
actually define the _ABI_ going back to the very early releases of  
Linux.  As Linus said, it's an ABI, which means that adding new  
features is OK, but removing them is bad, so I'll try to make sure  
it's _perfectly_ _ok_ to use a 2.6 kabi/ on a 2.0 kernel.  It should  
also be perfectly ok to use a 2.6 kabi/ on a 5-years-down-the-road  
2.8 kernel as long as you restrict yourself to the subset of features  
that were available in 2.6.

Cheers,
Kyle Moffett

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Eric Piel]

26.03.2006 13:52, Kyle Moffett wrote/a écrit:
> On Fri, 24 Mar 2006 17:46:27 -0500 Kyle Moffett <mrmacman_g4@mac.com> wrote:
>> I'm working on some sample patches now which I'll try to post in a
>> few days if I get the time.
> 
> Ok, here's a sample of the KABI conversion and cleanup patches that I'm
> proposing.  I have a few fundamental goals for these patches:
> 1)  The Linux kernel compiles and works at every step along the way
> 2)  Since most of the headers are currently quite broken with respect to
>     GLIBC and userspace, I won't spend much extra time preserving
>     compatibility with GLIBC, userspace, or non-GCC compilers.
> 3)  Everything in include/kabi will have a __kabi_ or __KABI_ prefix.
> 4)  Headers in include/linux that need the KABI interfaces will include
>     the corresponding <kabi/*.h> header and define or typedef the
>     necessary KABI definitions to the names the kernel wants.
> 5)  The stuff in include/kabi/*.h should always be completely independent
>     of userspace/kernelspace and not require any includes outside of
>     <kabi/*>.  This means that the only preprocessor symbols that we can
>     assume are present are those provided by the compiler itself.
Hello,

I completely agree with rules 1, 2 and 5. However, IMHO rule 4 should 
just be the inverse of rule 5: The stuff in include/linux should always 
be independent from KABI (and userspace of course). Simply because the 
way we _implement_ things in the kernel has to be different from the 
things that we _specify_ in the kernel ABI. They just append to be both 
written in C language, but it's not a reason to mix them. The kernel 
developers has to be free of doing any kludge, clever things, 
compatibility workarounds without affecting the userspace applications. 
Otherwise, you'll end up with another include/linux after few months! 
Separating the implementation and the binary specification has the 
additional advantage that if some kernel hacker mistakenly change the 
ABI, it's easy to say : "see, after your commit xxxxxxxx, the linux 
header and the kabi header are semantically different. You did something 
Wrong".

As for rule 3, if you have independent headers, this should be much less 
necessary. Additionally, keeping all the names identical to what they 
are already called will allow userspace to just use include/kabi/ as the 
/usr/include/linux/ directory. Avoiding smelly things like:

linux/foo.h:
   #define __kabi_foo foo
   #include <kabi/foo.h>

That was my 2 cents :-)
Regards,
Eric

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Rob Landley]

On Sunday 26 March 2006 9:31 am, Eric Piel wrote:
> I completely agree with rules 1, 2 and 5. However, IMHO rule 4 should
> just be the inverse of rule 5: The stuff in include/linux should always
> be independent from KABI (and userspace of course). Simply because the
> way we _implement_ things in the kernel has to be different from the
> things that we _specify_ in the kernel ABI.

You know all the stuff that's marked __user?  It's all kernel ABI.  Having it 
defined in two places invites version skew, and the kernel needs it too 
because the kernel is parsing the stuff sent in by userspace and filling it 
out to send back to userspace.

Lots of syscalls and ioctls and such pass a structure back and forth from 
userspace to the kernel and back, right?  Doing a stat() fills out a 
structure, doing an losetup fills out a structure, and so on.

Userspace needs to know what's in this structure.  It may be wrapped in a libc 
function that fills out a different structure from the kernel structure, but 
the data that goes back and forth between the program and the kernel has to 
be defined in a header somewhere so the libc knows what the kernel's sending 
and the kernel knows what the libc is sending.  (And for those functions with 
no libc wrapper, the user program needs to know the structure directly, 
somehow.)

Having a data-marshalling ABI structure defined in two places invites version 
skew.  Userspace needs access to this (at least to build a libc), and the 
kernel needs access to this, because it's a _communication_mechanism_.  You 
can't have a communication mechanism that's only defined at one end.

> As for rule 3, if you have independent headers, this should be much less
> necessary. Additionally, keeping all the names identical to what they
> are already called will allow userspace to just use include/kabi/ as the
> /usr/include/linux/ directory. Avoiding smelly things like:
>
> linux/foo.h:
>    #define __kabi_foo foo
>    #include <kabi/foo.h>

The #define is, indeed, smelly.

Rob
-- 
Never bet against the cheap plastic solution.

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Eric Piel]

26.03.2006 23:09, Rob Landley wrote/a écrit:
> On Sunday 26 March 2006 9:31 am, Eric Piel wrote:
>> I completely agree with rules 1, 2 and 5. However, IMHO rule 4 should
>> just be the inverse of rule 5: The stuff in include/linux should always
>> be independent from KABI (and userspace of course). Simply because the
>> way we _implement_ things in the kernel has to be different from the
>> things that we _specify_ in the kernel ABI.
> 
> You know all the stuff that's marked __user?  It's all kernel ABI.  Having it 
> defined in two places invites version skew, and the kernel needs it too 
> because the kernel is parsing the stuff sent in by userspace and filling it 
> out to send back to userspace.
> 
> Lots of syscalls and ioctls and such pass a structure back and forth from 
> userspace to the kernel and back, right?  Doing a stat() fills out a 
> structure, doing an losetup fills out a structure, and so on.
> 
> Userspace needs to know what's in this structure.  It may be wrapped in a libc 
> function that fills out a different structure from the kernel structure, but 
> the data that goes back and forth between the program and the kernel has to 
> be defined in a header somewhere so the libc knows what the kernel's sending 
> and the kernel knows what the libc is sending.  (And for those functions with 
> no libc wrapper, the user program needs to know the structure directly, 
> somehow.)
> 
> Having a data-marshalling ABI structure defined in two places invites version 
> skew.  Userspace needs access to this (at least to build a libc), and the 
> kernel needs access to this, because it's a _communication_mechanism_.  You 
> can't have a communication mechanism that's only defined at one end.
Well, that's half true. Indeed, in general, having two separate 
definitions invites version skew. However, in this particular case, it's 
slightly different: because the principle of the ABI is to be stable, or 
more exactly _compatible_. This means that if one definition was right 
at some point in the time, it should always still be true ten years 
later. At worse, the ABI can be extended, but never changed. If the 
specification (KABI) and the implementation (Linux) are not compatible 
it means the kernel developers screwed up, not that the KABI maintainers 
haven't updated in time.

Of course, next to the theory, there is the reality. Some part of the 
ABI has already be changed (broken) in the past (like the alsa ABI, 
IIRC). In such case the KABI maintainers will have to handle the changes 
as promptly as possible, but the responsibility will be held by the 
kernel developer who has opted for breakage.

The real problem of sharing the same headers between kernel and KABI is 
that it will end up by having to re-implement the "#ifdef __KERNEL__"'s. 
Have a look at Kyle's second patch "Generalize fd_set handling across 
architectures". Some headers had a different version of the __FD_*() 
macros depending on the compiler. That's something you may want to have 
in the implementation but definitely not in the specification. In this 
situation, Kyle handled it nicely by writing versions compatible with 
any compiler. Good looking solution inside the kernel will not always be 
an option. IMHO, if KABI and the kernel share the same headers, it's 
just a matter of time before someone introduces an "#ifdef 
__KERNEL__"-alike mechanism, exactly what we've been trying to remove.

Eric

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Kyle Moffett]

On Mar 26, 2006, at 18:06:48, Eric Piel wrote:
> The real problem of sharing the same headers between kernel and  
> KABI is that it will end up by having to re-implement the "#ifdef  
> __KERNEL__"'s. Have a look at Kyle's second patch "Generalize  
> fd_set handling across architectures". Some headers had a different  
> version of the __FD_*() macros depending on the compiler. That's  
> something you may want to have in the implementation but definitely  
> not in the specification.

Actually, I think it's the other way around.  The <kabi/*.h> files  
should (eventually) be useable in basically any compilation  
environment thrown at it.  This means it should work from C and C++,  
using GCC, ICC, or some custom barely-standards-compliant compiler.   
I didn't bother with that part right now, since I still want to try  
to reuse the generic bitops if possible, but it's something I plan to  
address in future versions of the patchset (see below).

> In this situation, Kyle handled it nicely by writing versions  
> compatible with any compiler.

Eh, not really.  "__inline__" is GCC-specific and probably won't work  
in other compilers (unless you did "#define __inline__", which would  
bloat the code a lot).

This case highlights something else I'd like to do.  A good chunk of  
the functionality in the Linux kernel works both in userspace and  
kernelspace, and some of those arch-specific primitives (like the  
inline bitops) would be useful in defining the kabi headers.   
According to Jeff Dike, UML would like access to some of that stuff  
unrestricted by __KERNEL__ too.  In all of those cases, it's not an  
ABI and all the users are in-kernel so it could be changed at will.   
I'd like to try to put some of that into a "klib" directory (though  
with dependencies crossing between kabi and klib) so that it could be  
used in kabi and UML without duplicating functionality.  Naturally  
much of that would be C-only and depend on GCC, but I would have to  
be careful that the kabi portions used least-common-denominator  
functionality.

That brings up one final point: Does anybody actually use any  
compilers on Linux other than GCC?

Cheers,
Kyle Moffett

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Jeff Dike]

On Sun, Mar 26, 2006 at 07:40:14PM -0500, Kyle Moffett wrote:
> According to Jeff Dike, UML would like access to some of that stuff  
> unrestricted by __KERNEL__ too.  

Not quite true - I just want them physically separated from the
non-userspace-usable stuff, as in different files.  I don't care
whether they are inside #ifdef __KERNEL__, because that's defined in
the UML build.

				Jeff

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Jan Engelhardt]

>
> Eh, not really.  "__inline__" is GCC-specific and probably won't work in other
> compilers (unless you did "#define __inline__", which would bloat the code a
> lot).
>
But ___inline is a C99 keyword, isnot it?




Jan Engelhardt
-- 

[OT] Non-GCC compilers used for linux userspace

[Kyle Moffett]

On Mar 28, 2006, at 09:20:09, Jan Engelhardt wrote:
>> Eh, not really.  "__inline__" is GCC-specific and probably won't  
>> work in other compilers (unless you did "#define __inline__",  
>> which would bloat the code a lot).
>
> But ___inline is a C99 keyword, is not it?

Not even GCC fully supports C99 (although I think it does support  
that keyword when passed -std=c99 or -std=gnu99), and I suspect that  
a majority of the other compilers for which we would want to add  
support in the kernel headers would not support C99 or would do a  
poor job of handling inline functions.

But my question still stands.  Does anybody actually use any non-GCC  
compiler for userspace in Linux?

Cheers,
Kyle Moffett

Re: [OT] Non-GCC compilers used for linux userspace

[Eric Piel]

03/28/2006 05:57 PM, Kyle Moffett wrote/a écrit:
> 
> But my question still stands.  Does anybody actually use any non-GCC 
> compiler for userspace in Linux?
At least in the domain of HPC, I've seen people which were compiling 
mostly *everything* with the intel compiler (x86 and ia64) for 
performance reason. So... yes userspace is sometimes compiled with 
non-GCC compiler :-)

Eric

Re: [OT] Non-GCC compilers used for linux userspace

[Kyle Moffett]

On Mar 28, 2006, at 11:13:42, Eric Piel wrote:
> 03/28/2006 05:57 PM, Kyle Moffett wrote/a écrit:
>> But my question still stands.  Does anybody actually use any non- 
>> GCC compiler for userspace in Linux?
>
> At least in the domain of HPC, I've seen people which were  
> compiling mostly *everything* with the intel compiler (x86 and  
> ia64) for performance reason. So... yes userspace is sometimes  
> compiled with non-GCC compiler :-)

Ok, well, the Intel compiler actually ends up emulating GCC and  
supports most of its extensions; supposedly it can even be used to  
compile the kernel sources, as per include/linux/compiler-intel.h. (I  
don't know how recently this has been tested, though)  So does  
anybody compile userspace under anything other than GCC or Intel  
compilers?  Do any such compilers even exist?

Cheers,
Kyle Moffett

Re: [OT] Non-GCC compilers used for linux userspace

[Jason L Tibbitts III]

>>>>> "KM" == Kyle Moffett <mrmacman_g4@mac.com> writes:

KM> So does anybody compile userspace under anything other than GCC or
KM> Intel compilers?  Do any such compilers even exist?

PGI and PathScale are around.  Lahey, too, although they seem to just
do Fortran now.

I doubt you'd want to worry about compiling the entire userland with
these compilers, however.

 - J<

Re: [OT] Non-GCC compilers used for linux userspace

[Kyle Moffett]

On Mar 28, 2006, at 11:59:13, Jason L Tibbitts III wrote:
>>>>>> "KM" == Kyle Moffett <mrmacman_g4@mac.com> writes:
>> So does anybody compile userspace under anything other than GCC or  
>> Intel compilers?  Do any such compilers even exist?
>
> PGI and PathScale are around.  Lahey, too, although they seem to  
> just do Fortran now.
>
> I doubt you'd want to worry about compiling the entire userland  
> with these compilers, however.

Mainly I want to know if I should even bother making the kabi headers  
compile with anything other than GCC.  Judging from the apparently  
negligible number of users, it doesn't sound like something I should  
spend much or any time on, at least for now.

Cheers,
Kyle Moffett

Re: [OT] Non-GCC compilers used for linux userspace

[Daniel Jacobowitz]

On Tue, Mar 28, 2006 at 12:13:15PM -0500, Kyle Moffett wrote:
> On Mar 28, 2006, at 11:59:13, Jason L Tibbitts III wrote:
> >>>>>>"KM" == Kyle Moffett <mrmacman_g4@mac.com> writes:
> >>So does anybody compile userspace under anything other than GCC or  
> >>Intel compilers?  Do any such compilers even exist?
> >
> >PGI and PathScale are around.  Lahey, too, although they seem to  
> >just do Fortran now.
> >
> >I doubt you'd want to worry about compiling the entire userland  
> >with these compilers, however.
> 
> Mainly I want to know if I should even bother making the kabi headers  
> compile with anything other than GCC.  Judging from the apparently  
> negligible number of users, it doesn't sound like something I should  
> spend much or any time on, at least for now.

I'm not sure how you got to that conclusion.  People have already named
several non-GCC compilers that are used; and most of the users of
commercial compilers won't be reading this list.

If you want glibc to ever include these things, they had better be
portable C and work without GCC.  Otherwise it's a non-starter.
Only GCC may be used to build glibc, but it deliberately supports any
conforming C compiler to build userspace code.

-- 
Daniel Jacobowitz
CodeSourcery

Re: [OT] Non-GCC compilers used for linux userspace

[Kyle Moffett]

On Mar 28, 2006, at 12:28:47, Daniel Jacobowitz wrote:
> On Tue, Mar 28, 2006 at 12:13:15PM -0500, Kyle Moffett wrote:
>> On Mar 28, 2006, at 11:59:13, Jason L Tibbitts III wrote:
>>>>>>>> "KM" == Kyle Moffett <mrmacman_g4@mac.com> writes:
>>>> So does anybody compile userspace under anything other than GCC  
>>>> or Intel compilers?  Do any such compilers even exist?
>>>
>>> PGI and PathScale are around.  Lahey, too, although they seem to  
>>> just do Fortran now.
>>>
>>> I doubt you'd want to worry about compiling the entire userland  
>>> with these compilers, however.
>>
>> Mainly I want to know if I should even bother making the kabi  
>> headers compile with anything other than GCC.  Judging from the  
>> apparently negligible number of users, it doesn't sound like  
>> something I should spend much or any time on, at least for now.
>
> I'm not sure how you got to that conclusion.  People have already  
> named several non-GCC compilers that are used; and most of the  
> users of commercial compilers won't be reading this list.
>
> If you want glibc to ever include these things, they had better be  
> portable C and work without GCC.  Otherwise it's a non-starter.   
> Only GCC may be used to build glibc, but it deliberately supports  
> any conforming C compiler to build userspace code.

Ok, my email was a bit premature (I've gotten a couple other private  
emails about other compilers too) and if people see this as an issue  
then I'll try to make all the code C89-compliant from the start.  I  
just didn't want to go writing a whole bunch of compatibility macros  
only to find out that they never got used.

Cheers,
Kyle Moffett

Re: [OT] Non-GCC compilers used for linux userspace

[Bryan O'Sullivan]

On Tue, 2006-03-28 at 12:13 -0500, Kyle Moffett wrote:

> Mainly I want to know if I should even bother making the kabi headers  
> compile with anything other than GCC.

The PathScale compiler is gcc-compatible, so there should be no problems
for us using gcc-isms in those headers.

	<b

Re: [OT] Non-GCC compilers used for linux userspace

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 616 bytes --]

On Tue, 2006-03-28 11:20:50 -0500, Kyle Moffett <mrmacman_g4@mac.com> wrote:
> don't know how recently this has been tested, though)  So does  
> anybody compile userspace under anything other than GCC or Intel  
> compilers?  Do any such compilers even exist?

tcc

MfG, JBG

-- 
Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481             _ O _
"Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg  _ _ O
 für einen Freien Staat voll Freier Bürger"  | im Internet! |   im Irak!   O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [OT] Non-GCC compilers used for linux userspace

[Jesper Juhl]

On 3/28/06, Kyle Moffett <mrmacman_g4@mac.com> wrote:
> On Mar 28, 2006, at 11:13:42, Eric Piel wrote:
> > 03/28/2006 05:57 PM, Kyle Moffett wrote/a écrit:
> >> But my question still stands.  Does anybody actually use any non-
> >> GCC compiler for userspace in Linux?
> >
> > At least in the domain of HPC, I've seen people which were
> > compiling mostly *everything* with the intel compiler (x86 and
> > ia64) for performance reason. So... yes userspace is sometimes
> > compiled with non-GCC compiler :-)
>
> Ok, well, the Intel compiler actually ends up emulating GCC and
> supports most of its extensions; supposedly it can even be used to
> compile the kernel sources, as per include/linux/compiler-intel.h. (I
> don't know how recently this has been tested, though)  So does
> anybody compile userspace under anything other than GCC or Intel
> compilers?  Do any such compilers even exist?
>

Other compilers do exist.

Over the years I've personally used a few to compile userspace apps
for different projects (though never for compiling the kernel).

Some of the compilers I have personally used for userspace apps on
Linux include ;

lcc   - http://www.cs.princeton.edu/software/lcc/
tcc   - http://fabrice.bellard.free.fr/tcc/

(plus gcc & icc ofcourse)


Others that I know of but have never used include :

sdcc   - http://sdcc.sourceforge.net/
Compaq C for Linux   - http://h30097.www3.hp.com/linux/compaq_c/index.html
Open Watcom   - http://www.openwatcom.org/
vacpp   - http://www-306.ibm.com/software/awdtools/vacpp/
XL C/C++   - http://www.absoft.com/Products/Compilers/C_C++/Linux/XLC/XLC.html

and I'm sure many more exist...

--
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

Re: [OT] Non-GCC compilers used for linux userspace

[Rob Landley]

On Tuesday 28 March 2006 11:20 am, Kyle Moffett wrote:
> On Mar 28, 2006, at 11:13:42, Eric Piel wrote:
> > 03/28/2006 05:57 PM, Kyle Moffett wrote/a écrit:
> >> But my question still stands.  Does anybody actually use any non-
> >> GCC compiler for userspace in Linux?
> >
> > At least in the domain of HPC, I've seen people which were
> > compiling mostly *everything* with the intel compiler (x86 and
> > ia64) for performance reason. So... yes userspace is sometimes
> > compiled with non-GCC compiler :-)
>
> Ok, well, the Intel compiler actually ends up emulating GCC and
> supports most of its extensions; supposedly it can even be used to
> compile the kernel sources, as per include/linux/compiler-intel.h. (I
> don't know how recently this has been tested, though)  So does
> anybody compile userspace under anything other than GCC or Intel
> compilers?  Do any such compilers even exist?
>
> Cheers,
> Kyle Moffett

I play with Fabrice Bellard's Tiny C Compiler (http://www.tinycc.org), and 
hope to get a distro compiled with it someday, at least as a proof of 
concept.

That aims for full c99 and is already implementing a lot of gcc stuff too.

Rob
-- 
Never bet against the cheap plastic solution.

Re: [OT] Non-GCC compilers used for linux userspace

[Nix]

On Tue, 28 Mar 2006, Rob Landley announced authoritatively:
> I play with Fabrice Bellard's Tiny C Compiler (http://www.tinycc.org), and 
> hope to get a distro compiled with it someday, at least as a proof of 
> concept.

It's a nifty idea, a sort of uClibc of C compilers. Alas it's useless on
almost all my systems because it's x86-only by design...

> That aims for full c99 and is already implementing a lot of gcc stuff too.

Good for it, as long as it doesn't go on to define __GNUC__ like icc did
at one point (even though it doesn't implement all GCC
extensions)... but Fabrice is sane so I doubt he'd do anything that
loopy.

-- 
`Come now, you should know that whenever you plan the duration of your
 unplanned downtime, you should add in padding for random management
 freakouts.'

Re: [OT] Non-GCC compilers used for linux userspace

[Rob Landley]

On Wednesday 29 March 2006 4:23 pm, Nix wrote:
> On Tue, 28 Mar 2006, Rob Landley announced authoritatively:
> > I play with Fabrice Bellard's Tiny C Compiler (http://www.tinycc.org),
> > and hope to get a distro compiled with it someday, at least as a proof of
> > concept.
>
> It's a nifty idea, a sort of uClibc of C compilers. Alas it's useless on
> almost all my systems because it's x86-only by design...

Actually according to the changelog version 0.9.21 grew support for ARM, and I 
believe it supports some other platforms too.

Notice that Fabrice Bellard (who wrote it) is also the guy behind qemu.  My 
vague understanding of how things went is that at some point after doing 
tccboot, he was decoupled the C parser from the code generator in order to 
retarget tcc to produce code for other platforms, and got the idea of 
producing different front-ends too so it could input something other than C, 
namely machine code.

The result was qemu, which sort of compiles machine code to machine code 
dynamically, and which has taken up a large chunk of his time ever since.  
(The speed of tcc development has tailed off noticeably since, but he still 
spends a little time on it, and there are other developers...)

> > That aims for full c99 and is already implementing a lot of gcc stuff
> > too.
>
> Good for it, as long as it doesn't go on to define __GNUC__ like icc did
> at one point (even though it doesn't implement all GCC
> extensions)... but Fabrice is sane so I doubt he'd do anything that
> loopy.

That's more a header issue anyway.  That's the uClibc developers problem. :)

Rob
-- 
Never bet against the cheap plastic solution.

Re: [OT] Non-GCC compilers used for linux userspace

[Nix]

On Wed, 29 Mar 2006, Rob Landley whispered secretively:
> Actually according to the changelog version 0.9.21 grew support for ARM, and I 
> believe it supports some other platforms too.

That's... impressive. Of course the more generality it grows the slower
it must necessarily become, although it'll be a while until its IR is as
bloated and thus as slow as GCC's... (GCC's only slow because of cache
effects, really).

> The result was qemu, which sort of compiles machine code to machine code 
> dynamically, and which has taken up a large chunk of his time ever since.  
> (The speed of tcc development has tailed off noticeably since, but he still 
> spends a little time on it, and there are other developers...)

Well, I'd rather he spent time on qemu than tcc; there are other C compilers
but there's nothing quite like qemu (bochs doesn't work very well, valgrind
is similar in essence but very different in operation...)

>> > That aims for full c99 and is already implementing a lot of gcc stuff
>> > too.
>>
>> Good for it, as long as it doesn't go on to define __GNUC__ like icc did
>> at one point (even though it doesn't implement all GCC
>> extensions)... but Fabrice is sane so I doubt he'd do anything that
>> loopy.
> 
> That's more a header issue anyway.  That's the uClibc developers problem. :)

;)

-- 
`Come now, you should know that whenever you plan the duration of your
 unplanned downtime, you should add in padding for random management
 freakouts.'

Re: [OT] Non-GCC compilers used for linux userspace

[Rob Landley]

On Thursday 30 March 2006 2:24 am, Nix wrote:
> On Wed, 29 Mar 2006, Rob Landley whispered secretively:
> > Actually according to the changelog version 0.9.21 grew support for ARM,
> > and I believe it supports some other platforms too.
>
> That's... impressive. Of course the more generality it grows the slower
> it must necessarily become,

Not if compliation speed is the primary explicit design goal from day one, and 
they regression test with that in mind.

Keep in mind that the main use of tcc these days is to turn c into a scripting 
language.  Just start your C file with

#!/usr/bin/tcc -run

And notice that #! is a preprocessor comment line as far as tcc is 
concerned. :)

> > The result was qemu, which sort of compiles machine code to machine code
> > dynamically, and which has taken up a large chunk of his time ever since.
> > (The speed of tcc development has tailed off noticeably since, but he
> > still spends a little time on it, and there are other developers...)
>
> Well, I'd rather he spent time on qemu than tcc; there are other C
> compilers but there's nothing quite like qemu (bochs doesn't work very
> well, valgrind is similar in essence but very different in operation...)

They're still sort of related.  The sad part is that tcc is -><- this close to 
building an unmodified linux kernel, as tccboot demonstrates.  But Fabrice 
seems to have gone "ooh, shiny!" off in another direction, for entirely 
understandable reasons... :)

I'm sure somebody will go take a whack at it sooner or later.  (I haven't got 
time either.)

Rob
-- 
Never bet against the cheap plastic solution.

Re: [OT] Non-GCC compilers used for linux userspace

[Nix]

On Thu, 30 Mar 2006, Rob Landley suggested tentatively:
> On Thursday 30 March 2006 2:24 am, Nix wrote:
>> On Wed, 29 Mar 2006, Rob Landley whispered secretively:
>> > Actually according to the changelog version 0.9.21 grew support for ARM,
>> > and I believe it supports some other platforms too.
>>
>> That's... impressive. Of course the more generality it grows the slower
>> it must necessarily become,
> 
> Not if compliation speed is the primary explicit design goal from day one, and 
> they regression test with that in mind.

Aaah.

> Keep in mind that the main use of tcc these days is to turn c into a scripting 
> language.  Just start your C file with
> 
> #!/usr/bin/tcc -run

I feel distinctly queasy. (I can't easily think of a less suitable language for
scripting than C, either: perhaps COBOL...)

> And notice that #! is a preprocessor comment line as far as tcc is 
> concerned. :)

I noticed that, but I didn't think anyone actually *used* tcc for this.

-- 
`Come now, you should know that whenever you plan the duration of your
 unplanned downtime, you should add in padding for random management
 freakouts.'

Re: [OT] Non-GCC compilers used for linux userspace

[Harald Arnesen]

Nix <nix@esperi.org.uk> (Thu, 30 Mar 2006 23:02:59 +0100):

> > And notice that #! is a preprocessor comment line as far as tcc is 
> > concerned. :)
> 
> I noticed that, but I didn't think anyone actually *used* tcc for
> this.
> 

I did...
-- 
Hilsen Harald.

Re: [OT] Non-GCC compilers used for linux userspace

[Rob Landley]

On Thursday 30 March 2006 5:02 pm, Nix wrote:
> > Keep in mind that the main use of tcc these days is to turn c into a
> > scripting language.  Just start your C file with
> >
> > #!/usr/bin/tcc -run
>
> I feel distinctly queasy. (I can't easily think of a less suitable language
> for scripting than C, either: perhaps COBOL...)

No, the queasy bit is when you realize that you can stick library linkage 
directives on that command line and thus you can dynamically compile and run 
X apps.

They have examples of doing this. :)

> > And notice that #! is a preprocessor comment line as far as tcc is
> > concerned. :)
>
> I noticed that, but I didn't think anyone actually *used* tcc for this.

It's not exactly "what it's for", but it's up there.

Rob
-- 
Never bet against the cheap plastic solution.

Re: [OT] Non-GCC compilers used for linux userspace

[Mathieu Chouquet-Stringer]

mrmacman_g4@mac.com (Kyle Moffett) writes:
> So does  anybody compile userspace under anything other than GCC or Intel
> compilers?  Do any such compilers even exist?

ccc on alpha
-- 
Mathieu Chouquet-Stringer
    "Le disparu, si l'on vénère sa mémoire, est plus présent et
                 plus puissant que le vivant".
           -- Antoine de Saint-Exupéry, Citadelle --

Re: [OT] Non-GCC compilers used for linux userspace

[Eric W. Biederman]

Eric Piel <Eric.Piel@tremplin-utc.net> writes:

> 03/28/2006 05:57 PM, Kyle Moffett wrote/a écrit:
>> But my question still stands.  Does anybody actually use any non-GCC compiler
>> for userspace in Linux?
> At least in the domain of HPC, I've seen people which were compiling mostly
> *everything* with the intel compiler (x86 and ia64) for performance
> reason. So... yes userspace is sometimes compiled with non-GCC compiler :-)

The pathscale, and pgi compilers also get a reasonable amount of use.

pathscale has a gcc derived front-end so it isn't to much to worry about.

The pgi compiler as I recall is a fairly pedantic c90 compiler, that
doesn't try and support gcc extensions.

Eric

Re: [OT] Non-GCC compilers used for linux userspace

[Peter Chubb]

>>>>> "Kyle" == Kyle Moffett <mrmacman_g4@mac.com> writes:


Kyle> But my question still stands.  Does anybody actually use any
Kyle> non-GCC compiler for userspace in Linux?

Yes.  GCC produces much less than optimal code on Itanium, so the
Intel compiler often gets used.


--
Dr Peter Chubb  http://www.gelato.unsw.edu.au  peterc AT gelato.unsw.edu.au
http://www.ertos.nicta.com.au           ERTOS within National ICT Australia

RE: [OT] Non-GCC compilers used for linux userspace

[Roger Heflin]

 

> Not even GCC fully supports C99 (although I think it does 
> support that keyword when passed -std=c99 or -std=gnu99), and 
> I suspect that a majority of the other compilers for which we 
> would want to add support in the kernel headers would not 
> support C99 or would do a poor job of handling inline functions.
> 
> But my question still stands.  Does anybody actually use any 
> non-GCC compiler for userspace in Linux?
> 
> Cheers,
> Kyle Moffett

Sometimes.

I have customers that use Pathscale, Intel, and Portland
groups compilers.

Not all of them use the C compilers (Some just use the F90
compilers), but a number of them do use the C compiler, though
I would be that most of the applications are not going to care
about doing much against anything but standard ANSI C calls.

                                Roger

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Ben Pfaff]

Jan Engelhardt <jengelh@linux01.gwdg.de> writes:

>> Eh, not really.  "__inline__" is GCC-specific and probably won't work in other
>> compilers (unless you did "#define __inline__", which would bloat the code a
>> lot).
>>
> But ___inline is a C99 keyword, isnot it?

___inline is not, but inline is.
-- 
"Platonically Evil Monkey has been symbolically representing the darkest 
 fears of humanity since the dawn of literature and religion, and I think
 I speak for everyone when I give it a sidelong glance of uneasy recognition 
 this evening." --Scrymarch

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Catalin Marinas]

Kyle Moffett <mrmacman_g4@mac.com> wrote:
> That brings up one final point: Does anybody actually use any
> compilers on Linux other than GCC?

Yes, some people use (or plan to use) the ARM Ltd toolchain
(http://www.arm.com/products/DevTools/RVCT.html) to cross-compile
applications for Linux (mainly for a better size/performance).

-- 
Catalin

Arch-specific header inconsistency (asm-*/termios.h)

[Kyle Moffett]

I'm in the process of cleaning up various headers for use from  
userspace, and at the same time I'm working to clean up some of the  
duplication in the kernel-only portions of those headers.  I'm  
running into one oddity that I can't explain.  In the various asm-*/ 
termios.h files, there are user_termio_to_kernel_termios functions.   
The m68k header does this:

get_user((termios)->c_line, &(termio)->c_line);

The i386 header is missing that line, and after macro expansion the  
rest of the context is almost exactly identical.  It appears there  
are a couple other architectures that fall on both sides of the  
fence.  Can anyone explain this apparently aberrant behavior or is  
this a bug?  If there's a reason behind it, I'll make sure to put in  
a useful comment; otherwise I'll fix it.

Thanks for the help!

Cheers,
Kyle Moffett

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Sam Ravnborg]

On Sun, Mar 26, 2006 at 06:52:05AM -0500, Kyle Moffett wrote:
> On Fri, 24 Mar 2006 17:46:27 -0500 Kyle Moffett <mrmacman_g4@mac.com> wrote:
> > I'm working on some sample patches now which I'll try to post in a
> > few days if I get the time.
> 
> Ok, here's a sample of the KABI conversion and cleanup patches that I'm
> proposing.  I have a few fundamental goals for these patches:
> 1)  The Linux kernel compiles and works at every step along the way
> 2)  Since most of the headers are currently quite broken with respect to
>     GLIBC and userspace, I won't spend much extra time preserving
>     compatibility with GLIBC, userspace, or non-GCC compilers.
> 3)  Everything in include/kabi will have a __kabi_ or __KABI_ prefix.
> 4)  Headers in include/linux that need the KABI interfaces will include
>     the corresponding <kabi/*.h> header and define or typedef the
>     necessary KABI definitions to the names the kernel wants.
> 5)  The stuff in include/kabi/*.h should always be completely independent
>     of userspace/kernelspace and not require any includes outside of
>     <kabi/*>.  This means that the only preprocessor symbols that we can
>     assume are present are those provided by the compiler itself.
Hi Kyle.
Been watching this thread evolve for a while - and contributed a little
myself.
But I fail to find a rationale for the selected approach.

There are at minimum the following options:
1) Go through current set of headers and sanitize them - using
   __KERNEL__ to identify kernel only stuff.
2) Keep user space interface (KABI in your term?) in include/ and slowly
   move kernel specific definitions somewhere else. This has the great
   advantage to keep backward compaitibility.
3) 'Preprocess' include/ and generate a set of KABI files based on
   current set of (cleaned up) kernel header files.
   'Preporcess in '' because a C-preprocessor will not be sutilable.
4) Introduce a virgin KABI (your approach). The virgin KABI has no users
   today and does in no way preserve backward compatibility.

Can you try to explain why the virgin approach is superior to the
others.

	Sam

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Kyle Moffett]

On Apr 2, 2006, at 13:58:59, Sam Ravnborg wrote:
> Hi Kyle.
> Been watching this thread evolve for a while - and contributed a  
> little myself.  But I fail to find a rationale for the selected  
> approach.
>
> 1) Go through current set of headers and sanitize them - using  
> __KERNEL__ to identify kernel only stuff.

I'm working on this right now as step one.  I see two big problems  
with this approach.  Primarily it leaves no distinction or  
documentation for future maintenance; the sanitization doesn't last  
long at all, as we've seen by trying this approach in the past.

> 2) Keep user space interface (KABI in your term?) in include/ and  
> slowly move kernel specific definitions somewhere else.  This has  
> the great advantage to keep backward compaitibility.

The big problem with this approach is it starts off with the  
submission of a 1000-item change-the-world patchset to move all of  
the completely-kernel-only patches from linux/include to linux/ 
kinclude; destroying all mergeability in the process and completely  
serializing development.  Judging from the responses on this list  
when similar changes have happened before, that kind of thing is  
usually considered bad.

> 3) 'Preprocess' include/ and generate a set of KABI files based on  
> current set of (cleaned up) kernel header files.  'Preprocess in ''  
> because a C-preprocessor will not be suitable.

I got the idea of preprocessing, but your sentence seems to have  
gotten mangled somewhere.  Any chance you could clarify?  From what I  
can see by looking through the current headers, preprocessing will  
not solve some of the duplication I'd like to try to clean up.  One  
example being the duplication of bitops in various macros all over  
the place, FD_SET/FD_CLR/FD_ISSET being a prime example of that  
duplication.  It would be really nice to be able to implement those  
in terms of __set_bit, etc, however those macros themselves must be  
made userspace-clean, including adding C89-compat macros for non-GCC  
compilers and other mild ugliness, even though they'd never be  
directly exposed for user programs.

> 4) Introduce a virgin KABI (your approach). The virgin KABI has no  
> users today and does in no way preserve backward compatibility.
>
> Can you try to explain why the virgin approach is superior to the  
> others.

Your "#4" does not accurately describe what I feel I'm trying to do.   
In my first set of patches I'd like to create a set of KABI headers  
(recently I've been considering sticking them under "linux-klib" or  
similar) which provide a set of useful standards-clean compiler- 
independent typedefs, macros, and inline functions to both the kernel  
_and_ whatever ABI headers we decide to use.  (working on that now)   
Those headers would _not_ be for use directly from userspace, but  
only from within the kernel and any in-kernel ABI headers.  As a  
result, since they would be either shipped with the kernel which  
compiles against them or installed with the KABI headers which depend  
upon them, developers would be gleefully free from any obligations to  
maintain a stable interface within the linux-klib headers (although  
the code itself should be mostly C89 compliant).

As I do that I would like to adjust the kernel headers to use those  
klib includes, hopefully alleviating some of the unusual include- 
cycle problems in various headers (like the asm/posix_types.h wanting  
to use asm/bitops.h for FD_SET operations, which in some  
architectures indirectly includes asm/posix_types.h).  Along with  
that process I would be fixing up __KERNEL__ wherever they seemed  
misplaced.  Afterwards, I would try to clean up the actual guts of  
the ABI (like FD_SET, for example), I would migrate that to include/ 
linux-kabi headers or similar, trying to contain all the C89-compat  
code and macros for non-GCC compilers in include/linux-{kabi,klib}

For future maintenance it would hopefully provide a clean and easily  
noticeable break between "You can modify include/linux as much as you  
damn well please" and "This patch touches include/linux-kabi, let's  
check it carefully for ABI breakage."  I'd like to propose this as a  
natural extension to Greg KH's ABI documentation, sequestering the  
more sensitive code into a separate directory where it may be more  
carefully watched and documented.  Feel free to continue to criticize  
and dissent; even if it's eventually determined that my idea won't  
work, hopefully some other useful ideas will come up in the discussion.

Thanks for the comments!

Cheers,
Kyle Moffett

Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

[Arnd Bergmann]

On Sunday 02 April 2006 21:30, Kyle Moffett wrote:
> > 3) 'Preprocess' include/ and generate a set of KABI files based on  
> > current set of (cleaned up) kernel header files.  'Preprocess in ''  
> > because a C-preprocessor will not be suitable.
> 
> I got the idea of preprocessing, but your sentence seems to have  
> gotten mangled somewhere.  Any chance you could clarify?  From what I  
> can see by looking through the current headers, preprocessing will  
> not solve some of the duplication I'd like to try to clean up.  One  
> example being the duplication of bitops in various macros all over  
> the place, FD_SET/FD_CLR/FD_ISSET being a prime example of that  
> duplication.  It would be really nice to be able to implement those  
> in terms of __set_bit, etc, however those macros themselves must be  
> made userspace-clean, including adding C89-compat macros for non-GCC  
> compilers and other mild ugliness, even though they'd never be  
> directly exposed for user programs.

These seem to be two completely independent problems. Reducing the
amount of duplication is a good thing all by itself that can be done
independent from finding a better way to provide the headers to 
user space.

About one year ago, I started down this route in order to autogenerate
something like linux-libc-headers from the kernel tree, but never got
around to submitting anything because I could not keep up with the
changes.

What I did there was to put a Makefile into each include directory
with a list of the files that should get installed (the list from
llc) and then add a headers_install target to Makefile that would
run each header file through an open-coded version of
'unifdef -U__KERNEL__ < include/$FILE > $PREFIX/include/$FILE'.

The advantage of this approach is that it minimises the amount of
change to each of the current header files while making it relatively
easy to audit by looking for wrong stuff (namespace pollution,
CONFIG_* dependencies, ...) in the installed headers.

	Arnd <><