Re: [klibc] [PATCH] build: Define __EXPORTED_HEADER__

Re: [klibc] [PATCH] build: Define __EXPORTED_HEADER__

[Mike Waychison]

On Fri, Feb 25, 2011 at 2:40 AM, Arnd Bergmann <arnd@arndb.de> wrote:
> On Friday 25 February 2011, Mike Waychison wrote:
>> On Thu, Feb 24, 2011 at 6:19 AM, Arnd Bergmann <arnd@arndb.de> wrote:
>> > On Thursday 24 February 2011, Mike Waychison wrote:
>> >> The kernel header guard to ensure that headers aren't included from
>> >> userland moved in commit 69c8f52b.  This causes the following message to
>> >> be emitted when building klibc (which is designed to use kernel headers
>> >> explicitly):
>> >>
>> >> /usr/local/src/git/linux-2.6//include/linux/types.h:13:2: warning: #warning "Attempt to use kernel headers from user space, see http://kernelnewbies.org/KernelHeaders"
>> >>
>> >> In order to silence this warning, make sure we define
>> >> __EXPORTED_HEADERS__ when building klibc userland.
>> >>
>> >> Signed-off-by: Mike Waychison <mikew@google.com>
>> >
>> > Well, klibc uses the kernel headers, but I don't think it's a good idea
>> > to use them directly without make headers_install. Since klibc also
>> > doesn't set -D__KERNEL__, the result should be the same, except that
>> > it makes sure not to use any headers that are meant to be internal.
>>
>> I'm not sure I follow.  The warning happens when __KERNEL__ isn't set.
>>
>> Also, the build for klibc explicitly supports using kernel headers
>> without installing them first via setting KLIBCKERNELSRC.
>
> My point was that klibc should perhaps no longer support this. When klibc
> was initially written, we did not have a way to install kernel headers and
> distros shipped a set of hand-crafter modified kernel headers. This is no
> longer the case since the kernel headers now work for all other libc
> implementations after installing.
>
> Is there still any benefit in allowing klibc to be built without
> installing the headers?

Well, it's useful for users (like us) that have environments where we
build a single vmlinux image given upstream kernel.org linux and klibc
trees.

> For instance, it should be easy to change the klibc build process from
> requiring to link to the kernel source to automatically installing the
> kernel headers into the klibc object directory.

I'm not against this idea, but is this something that works (and that
people can actually do given the state of the klibc tree at this
point)?

Maybe I don't understand what benefit is had by having a separate
install_headers step that does not much more than copy headers from
the kernel source tree into the klibc object tree as part of the
vmlinux build.

I believe the original motivation of klibc was to maintain a minimal
libc implementation that would mature to the point where we are
capable of building userland binaries in the kernel source tree (so
that some hairy legacy functionality currently implemented in
kernel-space could be cleaned up and moved into early-userspace
targets). I don't know why this effort has subsided, though I can
imagine that lack of will and time were large factors.

What you are proposing seems to me like a step towards giving up on
this noble goal :(

*sigh* At the other end of the spectrum is the fact that we now build
userland targets out of the kernel source tree, but that seems to be
more of a convenience-in-versioning issue than anything else. The
build environment has become pretty reliant on there being a whole GNU
based LSB environment setup for userland, which on a per-unit basis
isn't the de-facto standard anymore.  It'd be nice to hit some middle
ground here.

Re: [klibc] [PATCH] build: Define __EXPORTED_HEADER__

[Sam Ravnborg]

> 
> > For instance, it should be easy to change the klibc build process from
> > requiring to link to the kernel source to automatically installing the
> > kernel headers into the klibc object directory.
> 
> I'm not against this idea, but is this something that works (and that
> people can actually do given the state of the klibc tree at this
> point)?
Yes - it worked last time I played around with it.
> 
> Maybe I don't understand what benefit is had by having a separate
> install_headers step that does not much more than copy headers from
> the kernel source tree into the klibc object tree as part of the
> vmlinux build.
The headers generated are the headers supposed to be used by user-space.
klibc and glibc is just two obvious candidates.


> I believe the original motivation of klibc was to maintain a minimal
> libc implementation that would mature to the point where we are
> capable of building userland binaries in the kernel source tree (so
> that some hairy legacy functionality currently implemented in
> kernel-space could be cleaned up and moved into early-userspace
> targets). I don't know why this effort has subsided, though I can
> imagine that lack of will and time were large factors.
> 
> What you are proposing seems to me like a step towards giving up on
> this noble goal :(
You seem to be confusing something here. Even if klibc is one day
included in the kernel source then we should continue to use
the headers generated using "make headers_install" for building klibc.

	Sam

Re: [klibc] [PATCH] build: Define __EXPORTED_HEADER__

[Arnd Bergmann]

On Saturday 26 February 2011, Sam Ravnborg wrote:
> > 
> > What you are proposing seems to me like a step towards giving up on
> > this noble goal :(
> You seem to be confusing something here. Even if klibc is one day
> included in the kernel source then we should continue to use
> the headers generated using "make headers_install" for building klibc.

Absolutely, yes.

Mike, just think of the generated header files as intermediate files
required for building, just like the .o files that are use to build
the library.

	Arnd

Re: [klibc] [PATCH] build: Define __EXPORTED_HEADER__

[Mike Waychison]

On Sun, Feb 27, 2011 at 12:07 PM, Arnd Bergmann <arnd@arndb.de> wrote:
> On Saturday 26 February 2011, Sam Ravnborg wrote:
>> >
>> > What you are proposing seems to me like a step towards giving up on
>> > this noble goal :(
>> You seem to be confusing something here. Even if klibc is one day
>> included in the kernel source then we should continue to use
>> the headers generated using "make headers_install" for building klibc.
>
> Absolutely, yes.
>
> Mike, just think of the generated header files as intermediate files
> required for building, just like the .o files that are use to build
> the library.

How is one expected to test if the files should be rebuild or not?
Forcing a headers_install as part of a build will cause the entire
userland to be rebuilt.

Re: [klibc] [PATCH] build: Define __EXPORTED_HEADER__

[Arnd Bergmann]

On Monday 28 February 2011, Mike Waychison wrote:
> How is one expected to test if the files should be rebuild or not?
> Forcing a headers_install as part of a build will cause the entire
> userland to be rebuilt.

How about changing headers_install to only install files that have
changed? Make headers_install alreadh knows the list of all files
it is installing.

	Arnd

Re: [klibc] [PATCH] build: Define __EXPORTED_HEADER__

[Sam Ravnborg]

On Mon, Feb 28, 2011 at 01:11:16PM +0100, Arnd Bergmann wrote:
> On Monday 28 February 2011, Mike Waychison wrote:
> > How is one expected to test if the files should be rebuild or not?
> > Forcing a headers_install as part of a build will cause the entire
> > userland to be rebuilt.
> 
> How about changing headers_install to only install files that have
> changed? Make headers_install alreadh knows the list of all files
> it is installing.

make headers_install already does so. Only changes fiels are updated.
So I consider this a non-issue.

	Sam

Re: [klibc] [PATCH] build: Define __EXPORTED_HEADER__

[Mike Waychison]

On Mon, Feb 28, 2011 at 4:22 AM, Sam Ravnborg <sam@ravnborg.org> wrote:
> On Mon, Feb 28, 2011 at 01:11:16PM +0100, Arnd Bergmann wrote:
>> On Monday 28 February 2011, Mike Waychison wrote:
>> > How is one expected to test if the files should be rebuild or not?
>> > Forcing a headers_install as part of a build will cause the entire
>> > userland to be rebuilt.
>>
>> How about changing headers_install to only install files that have
>> changed? Make headers_install alreadh knows the list of all files
>> it is installing.
>
> make headers_install already does so. Only changes fiels are updated.
> So I consider this a non-issue.

Aha, I missed this point.  I've updated our build to do a
headers_install and things seem to work fine.

Thanks for the pointers :)  Please forget about this patch.