Re: kbuild-permanently-fix-kernel-configuration-include-mess.patch added to -mm tree

Re: kbuild-permanently-fix-kernel-configuration-include-mess.patch added to -mm tree

[Coywolf Qi Hunt]

On 9/14/05, akpm@osdl.org <akpm@osdl.org> wrote:
> 
> The patch titled
> 
>      kbuild: permanently fix kernel configuration include mess.
> 
> has been added to the -mm tree.  Its filename is
> 
>      kbuild-permanently-fix-kernel-configuration-include-mess.patch
> 
> 
> From: Russell King <rmk+lkml@arm.linux.org.uk>
> 
> Include autoconf.h into every kernel compilation via the gcc command line
> using -imacros.  This ensures that we have the kernel configuration
> included from the start, rather than relying on each file having #include
> <linux/config.h> as appropriate.  History has shown that this is something
> which is difficult to get right.

Not all compilations need config.h included and this slows down gratuitously.
-- 
Coywolf Qi Hunt
http://sosdg.org/~coywolf/

Re: kbuild-permanently-fix-kernel-configuration-include-mess.patch added to -mm tree

[Russell King]

On Wed, Sep 14, 2005 at 05:05:57PM +0800, Coywolf Qi Hunt wrote:
> On 9/14/05, akpm@osdl.org <akpm@osdl.org> wrote:
> > 
> > The patch titled
> > 
> >      kbuild: permanently fix kernel configuration include mess.
> > 
> > has been added to the -mm tree.  Its filename is
> > 
> >      kbuild-permanently-fix-kernel-configuration-include-mess.patch
> > 
> > 
> > From: Russell King <rmk+lkml@arm.linux.org.uk>
> > 
> > Include autoconf.h into every kernel compilation via the gcc command line
> > using -imacros.  This ensures that we have the kernel configuration
> > included from the start, rather than relying on each file having #include
> > <linux/config.h> as appropriate.  History has shown that this is something
> > which is difficult to get right.
> 
> Not all compilations need config.h included and this slows down gratuitously.

That is a small price to pay, rather than having to continually maintain
"does this file need config.h included" - which I think can conclusively
be shown to be a total lost cause.  There are about 3450 configuration
include errors in the kernel as of -git last night.

Getting config.h includes wrong causes subtle bugs - for instance, one
file may be built with some feature enabled which changes a structure
size, and another filfe may be built with it disabled.

I put forward that maintaining correct config.h include across all
files is demonstratably impossible in such a large source base without
considerable work.

I also put forward that the percentage of compilations which do not need
config.h is small and probably realistically zero.

Therefore, I think that a small slowdown for the few (if any) files which
don't need linux/config.h including is a good tradeoff.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

Re: kbuild-permanently-fix-kernel-configuration-include-mess.patch added to -mm tree

[Alan Cox]

On Mer, 2005-09-14 at 10:20 +0100, Russell King wrote:
> That is a small price to pay, rather than having to continually maintain
> "does this file need config.h included" - which I think can conclusively
> be shown to be a total lost cause.  There are about 3450 configuration
> include errors in the kernel as of -git last night.

I think your proposal makes sense. The alternative is to do a config
check each build for a while after the config pass and refuse to build
if the header check fails 8). That I suspect would rapidly see config.h
directly or indirectly in every file.

Re: kbuild-permanently-fix-kernel-configuration-include-mess.patch added to -mm tree

[Sam Ravnborg]

>  # Use LINUXINCLUDE when you must reference the include/ directory.
>  # Needed to be compatible with the O= option
>  LINUXINCLUDE    := -Iinclude \
> -                   $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include)
> +                   $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \
> +		   -imacros include/linux/autoconf.h
>  
What is the purpose of using -imacros instead of -iinclude

o -iinclude is much more commonly used for this purpose.
o sparse has limited support(*) for -iinclude today
o -imacros will silently ignore any output caused by the file

	Sam

Re: kbuild-permanently-fix-kernel-configuration-include-mess.patch added to -mm tree

[Russell King]

On Wed, Sep 14, 2005 at 06:49:53PM +0200, Sam Ravnborg wrote:
> >  # Use LINUXINCLUDE when you must reference the include/ directory.
> >  # Needed to be compatible with the O= option
> >  LINUXINCLUDE    := -Iinclude \
> > -                   $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include)
> > +                   $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \
> > +		   -imacros include/linux/autoconf.h
> >  
> What is the purpose of using -imacros instead of -iinclude
> 
> o -iinclude is much more commonly used for this purpose.
> o sparse has limited support(*) for -iinclude today
> o -imacros will silently ignore any output caused by the file

autoconf.h should only be macro definitions and should not contain
any code, so -imacros seemed to be the correct tool for the job.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

Re: kbuild-permanently-fix-kernel-configuration-include-mess.patch added to -mm tree

[Sam Ravnborg]

On Wed, Sep 14, 2005 at 05:53:26PM +0100, Russell King wrote:
> On Wed, Sep 14, 2005 at 06:49:53PM +0200, Sam Ravnborg wrote:
> > >  # Use LINUXINCLUDE when you must reference the include/ directory.
> > >  # Needed to be compatible with the O= option
> > >  LINUXINCLUDE    := -Iinclude \
> > > -                   $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include)
> > > +                   $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) \
> > > +		   -imacros include/linux/autoconf.h
> > >  
> > What is the purpose of using -imacros instead of -iinclude
> > 
> > o -iinclude is much more commonly used for this purpose.
> > o sparse has limited support(*) for -iinclude today
> > o -imacros will silently ignore any output caused by the file
> 
> autoconf.h should only be macro definitions and should not contain
> any code, so -imacros seemed to be the correct tool for the job.

I will use -iinclude for a start. When sparse has supported -imacros
in a few weeks we can change it if we like. I will update the patch.

	Sam

Re: kbuild-permanently-fix-kernel-configuration-include-mess.patch added to -mm tree

[Sam Ravnborg]

> That is a small price to pay, rather than having to continually maintain
> "does this file need config.h included" - which I think can conclusively
> be shown to be a total lost cause.  There are about 3450 configuration
> include errors in the kernel as of -git last night.
Depends on how you count...
If all .h files followed the rule - they should be selfcontained. In
other words they should all include what they need this is correct.

The correct figure is much less.
I did a check with defconfig for i386.
There are 7 .o files where config.h is not included - all are correct.
lib/errno.c, arch/ia386/boot/bootsect.S + a few more.
That was out of 983 .o files.

There will be no slowdown introducing -iinclude (or -imacros) keeping
these figures in mind.



find -name '.*.o.cmd' | xargs grep __KERNEL__ to find number of .o files
build.

added grep -l 'include/linux/config.h' to find .o files build and where
config-h was included. Then a simple diff..

	Sam