Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

Kyle Moffett
2006-04-02T05:53:55Z

On Apr 1, 2006, at 22:01:06, Arjan van de Ven wrote:
> On Sat, 2006-04-01 at 21:42 -0500, Kyle Moffett wrote:
>
>> (1) The various C standards state that the implementation should
>> restrict itself to symbols prefixed with "__", everything else is
>> reserved for user code (Including symbols prefixed with a single
>> underscore).
>
> user code in this context includes, in my interpretation, headers
> for specific ioctl structures and such (eg direct included headers;
> headers that those headers include are a different matter) that the
> user wants.
>
> Which is the vast majority of the kernel abi. Exceptions are things
> like __u64 which are almost always "indirectly" included. Eg the
> app wants "struct foo_ioctl" and foo_ioctl happens to contain an
> __u64 which needs types.h

I get the impression that you're confusing my KABI proposal with
kernel-only code. The point of KABI is to generalize some of the
headers so that they work both in userspace and kernelspace, even if
it means that the headers themselves aren't as clean as they might
otherwise be. In the kernel we have only one API where we have to
deal with compatibility: the kernel/userspace boundary. I'd like to
strictly define all of that compatibility within a single set of
_upstream_ headers in linux/include/linux-kabi or similar.

But returning to your statements above, user code in that (C
standards) context means anything not standard-specified. Basically,
from userspace, a program that includes _any_ standard-specified
header must bring in *only* those symbols specified by the standard
as being present in that header, as well as any implementation-
defined double-underscore symbols. So unless I "#include
<stdint.h>", my user program can typedef uint32_t to anything it
wants, even a struct, if it likes. I've actually done that before on
a very strange Cray platform with non-GCC compiler and OS that only
supported 64-bit integers (a char was a 64-bit integer with ops to
mask out the upper 56 bits), just to make bzip2 work as expected.
The same rules apply for _every_ other standard-specified header
(stdlib.h, stdio.h, sys/types.h, etc).

> the good news is that as kernel we have SOME power ;) people who
> make compilers like that stay away from symbols the kernel defines.
> Especially gcc is very careful in its namespace use and tends to be
> very different from what linux kernel uses.

As I said above, I'm talking about the proposed kernel ABI headers
here. The whole point of these is something that both the kernel
_and_ userspace can include. This means that the headers must allow
a libc that includes them to be completely standards-compliant. It
also means that these headers must work not only for GCC, but also
for tcc and icc and the myriad of other compilers I mentioned (the
ones that I got so many emails about in the "Non-GCC compilers" thread).

>> So my question to the list is this: Can you come up with any way
>> other than using a "__kabi_" prefix to reasonably avoid
>> namespace collisions with that large list of compilers?
>
> First of all be realistic. Don't do silly things for places where
> it doesn't matter. Again the ioctl structs come to mind. Second,
> names the kernel ALREADY claims are of course free to use as well;
> all those compilers ALREADY stay away from those.

Agreed (to some extent). I'm still evaluating a lot of the code, I
plan to submit several non-KABI cleanups first to prepare the way.
I'll try to get you specific patches so you can give me specific
rejections, ok? :-D I have the feeling that we both see things the
same way but have difficulty communicating due to the lack of examples.

> What is left if you take those two big ones out of the picture?

First of all, there seems to be a lot of little bits of code that the
KABI headers really want to use but can't right now. For example,
the asm-*/bitops.h headers contain inline functions for a variety of
nonatomic bitops that are _identical_ to the FD_SET code implemented
in asm-*/posix-types.h. Likewise it appears there are other standard-
specified bitmask manipulation functions that KABI wants to implement
using those. Unfortunately those bitmask functions are only
available in kernel space, although a few architectures were having
extensive fun with header inclusion loops or implicit inclusion just
to get access to them. Unfortunately this made it impossible to use
those headers in userspace.

It would be really nice if some of that architecture-specific
duplication could be consolidated into a single header file useable
from the userspace-pertinent KABI headers _and_ directly from kernel
code.

> there is another dynamic; those other compilers in general try to
> emulate gcc to a large degree, usually a very large degree. So the
> "problem" you see is a lot smaller than you think.

Please read in detail the responses to the non-gcc compiler thread
that I mentioned before. Apparently a number of the compilers only
take care to conform to the C89 or C99 specifications, which state
that such symbols are perfectly ok for them to use, and as a result
such "Just like GCC" behavior cannot be relied upon for kernel/
userspace-ABI headers.

Cheers,
Kyle Moffett