[regressions] Re: kbuild: save ARCH & CROSS_COMPILE when building a kernel

[regressions] Re: kbuild: save ARCH & CROSS_COMPILE when building a kernel

[Ingo Molnar]

Sam,

that arch-cache thing introduced in:

 5755433: kbuild: save ARCH & CROSS_COMPILE when building a kernel

isnt working very well in my experience.

1) it's a nuisance in cross-builds (it broke several cross-build scripts 
   of mine)

2) it makes it hard to switch from 32-bit x86 builds to 64-bit ones: 
   previously i only needed 'make ARCH=i386 defconfig' and off we went - 
   now i have to add a 'make mrproper' step.

3) it can also get stuck:

  phoenix:~/linux/linux> make mrproper
  Makefile:571: /home/mingo/tip/arch/no/Makefile: No such file or directory
  make: *** No rule to make target `/home/mingo/tip/arch/no/Makefile'.  Stop.

  phoenix:~/linux/linux> cat include/generated/*
  no

  phoenix:~/linux/linux> 

 There's no way out of this state, only if i manually remove 
 include/generated. (i suspect this is fixable)

Dunno - why isnt the .config enough in terms of determining which 
architecture we are on? If we want more info we should put it into the 
.config - not some other hidden state. I do think a single file should 
carry all 'configuration state' - that makes it easier to reproduce 
things, etc. etc. Everything else is really just hindering us.

	Ingo

Re: [regressions] Re: kbuild: save ARCH & CROSS_COMPILE when building a kernel

[Sam Ravnborg]

On Sat, Oct 03, 2009 at 12:34:41PM +0200, Ingo Molnar wrote:
> 
> Sam,
> 
> that arch-cache thing introduced in:
> 
>  5755433: kbuild: save ARCH & CROSS_COMPILE when building a kernel
> 
> isnt working very well in my experience.
> 
> 1) it's a nuisance in cross-builds (it broke several cross-build scripts 
>    of mine)
On the contrary it has helped me big time on my cross builds.
No need to script is all to remember to set the correct ARCH + CROSS_COMPILE
settings.

We should make the default case easy. And I consider the default case when
people fire up their shell and type:

    make CROSS_COMPILE=arm-linux- ARCH=arm defconfig

When they hit something they fix it up and do a simple:

    make

This is what this commit enables.
This commit is know to break scripts that expect that they can
seamlessly change CROSS_COMPILE or ARCH settings.
More on that below.

It is a nusiance that we have to instruct people to do a

    make mrproper

But on the other hand what else can we do to make sure people base their
work on a clean tree when the change either ARCH or CROSS_COMPILE.
Albeit for the latter we could drop the check as kbuild would see the
change in the commandline anyway.


 
> 2) it makes it hard to switch from 32-bit x86 builds to 64-bit ones: 
>    previously i only needed 'make ARCH=i386 defconfig' and off we went - 
>    now i have to add a 'make mrproper' step.

x86 is a mess in respect to ARCH setting.
We have:

ARCH=i386 =>
		- ARCH is set to i386
		- 64BIT is forced to n
		- defconfig is i386_defconfig

ARCH=x86 =>
		- ARCH is set to x86
		- 64BIT is configurable
		- defconfig is set to x86_64_defconfig

ARCH=x86_64 =>
		- ARCH is set to x86_64
		- 64BIT is forced y
		- defconfig is set to x86_64_defconfig
		
ARCH=""
		- let `uname -m` determine ARCH setting and follow rules above


And now when um enters the scene everyone gets confused.
I think the correct way to fix this is to always set ARCH=x86 in the x86 case.
And i386/x86_64 is only a user-interface thing - not a valid ARCH setting.

Pushing this to sparc and sh would also simplify a few things.

We would loose one thing. The current ARCH setting is used to _remember_
how we were started out. So if you once used ARCH=i386 you will not
be allowed to change 64BIT to 'y'.
But using ARCH to remember this is ackward at best and I guess very few people
realise this.

I will try to take a look at this in the hope that the current 'um'
mess can be cleaned up as a side effect.

> 3) it can also get stuck:
> 
>   phoenix:~/linux/linux> make mrproper
>   Makefile:571: /home/mingo/tip/arch/no/Makefile: No such file or directory
>   make: *** No rule to make target `/home/mingo/tip/arch/no/Makefile'.  Stop.

So you tried to configure a kernel with ARCH=no
The version I have now will ask you to do a:

    make mrporper ARCH=no

But this is suboptimal.

> Dunno - why isnt the .config enough in terms of determining which 
> architecture we are on? If we want more info we should put it into the 
> .config - not some other hidden state. I do think a single file should 
> carry all 'configuration state' - that makes it easier to reproduce 
> things, etc. etc. Everything else is really just hindering us.

chicken and egg + a few other issues.
If I want to have ARCH moved to Kconfig then this better be a true option.
Which requires kconfig to read in Kconfig* for all architectures in one sweep.

There is so many corner cases around this that I gave up last time
I looked at it.
But maybe the correct solution would be to move CROSS_COMPILE to Kconfig space.
And only leave ARCH out...


	Sam

Re: [regressions] Re: kbuild: save ARCH & CROSS_COMPILE when building a kernel

[Ingo Molnar]

* Sam Ravnborg <sam@ravnborg.org> wrote:

> On Sat, Oct 03, 2009 at 12:34:41PM +0200, Ingo Molnar wrote:
> > 
> > Sam,
> > 
> > that arch-cache thing introduced in:
> > 
> >  5755433: kbuild: save ARCH & CROSS_COMPILE when building a kernel
> > 
> > isnt working very well in my experience.
> > 
> > 1) it's a nuisance in cross-builds (it broke several cross-build scripts 
> >    of mine)
>
> On the contrary it has helped me big time on my cross builds. [...]

Could you please try to do that without hurting the existing scripting 
and existing workflow of other people?

> No need to script is all to remember to set the correct ARCH + 
> CROSS_COMPILE settings.

Then put your new state into the .config instead of this stupid 
include/generated/ extra state that is a _big_ regression over v2.6.31 
in terms of cross-build usability.

	Ingo

Re: [regressions] Re: kbuild: save ARCH & CROSS_COMPILE when building a kernel

[Sam Ravnborg]

On Fri, Oct 09, 2009 at 11:26:48AM +0200, Ingo Molnar wrote:
> 
> * Sam Ravnborg <sam@ravnborg.org> wrote:
> 
> > On Sat, Oct 03, 2009 at 12:34:41PM +0200, Ingo Molnar wrote:
> > > 
> > > Sam,
> > > 
> > > that arch-cache thing introduced in:
> > > 
> > >  5755433: kbuild: save ARCH & CROSS_COMPILE when building a kernel
> > > 
> > > isnt working very well in my experience.
> > > 
> > > 1) it's a nuisance in cross-builds (it broke several cross-build scripts 
> > >    of mine)
> >
> > On the contrary it has helped me big time on my cross builds. [...]
> 
> Could you please try to do that without hurting the existing scripting 
> and existing workflow of other people?

The only message in the above was that this was also beneficial
to some - and not a nusiance only.

> 
> > No need to script is all to remember to set the correct ARCH + 
> > CROSS_COMPILE settings.
> 
> Then put your new state into the .config instead of this stupid 
> include/generated/ extra state that is a _big_ regression over v2.6.31 
> in terms of cross-build usability.

I cannot see why saving this in include/generated/*
has anything to do with your complains. I guess you did not look
at the patch in question.

What is hurting you and several other users are _not_ related
to the fact that we save ARCH and CROSS_COMPILE setting or where
we save it.
What hurt you is the consistency checks that was added
to try to catch people in doing something unexpected.

Unexpected like trying to build a 64 bit kernel where they
previosuly were building a 32 bit kernel.
Or when they suddenly used a different set of binutils
because they changed CROSS_COMPILE settings.


The patch in question is reverted in my next push to Linus.
This is not an argument about that.
This is solely an argument about why it had this negative
impact for you and others.

[PS - mails had not been ignored. Other stuff just had higher
priority]

	Sam

Re: [regressions] Re: kbuild: save ARCH & CROSS_COMPILE when building a kernel

[Ingo Molnar]

* Sam Ravnborg <sam@ravnborg.org> wrote:

> On Fri, Oct 09, 2009 at 11:26:48AM +0200, Ingo Molnar wrote:
> > 
> > * Sam Ravnborg <sam@ravnborg.org> wrote:
> > 
> > > On Sat, Oct 03, 2009 at 12:34:41PM +0200, Ingo Molnar wrote:
> > > > 
> > > > Sam,
> > > > 
> > > > that arch-cache thing introduced in:
> > > > 
> > > >  5755433: kbuild: save ARCH & CROSS_COMPILE when building a kernel
> > > > 
> > > > isnt working very well in my experience.
> > > > 
> > > > 1) it's a nuisance in cross-builds (it broke several cross-build scripts 
> > > >    of mine)
> > >
> > > On the contrary it has helped me big time on my cross builds. [...]
> > 
> > Could you please try to do that without hurting the existing scripting 
> > and existing workflow of other people?
> 
> The only message in the above was that this was also beneficial
> to some - and not a nusiance only.
> 
> > 
> > > No need to script is all to remember to set the correct ARCH + 
> > > CROSS_COMPILE settings.
> > 
> > Then put your new state into the .config instead of this stupid 
> > include/generated/ extra state that is a _big_ regression over v2.6.31 
> > in terms of cross-build usability.
> 
> I cannot see why saving this in include/generated/*
> has anything to do with your complains. I guess you did not look
> at the patch in question.
> 
> What is hurting you and several other users are _not_ related
> to the fact that we save ARCH and CROSS_COMPILE setting or where
> we save it.
> What hurt you is the consistency checks that was added
> to try to catch people in doing something unexpected.
> 
> Unexpected like trying to build a 64 bit kernel where they
> previosuly were building a 32 bit kernel.
> Or when they suddenly used a different set of binutils
> because they changed CROSS_COMPILE settings.
> 
> 
> The patch in question is reverted in my next push to Linus.

Thanks! Mind sending the fix/revert here too, so that we can test 
whether that resolves all the problems?

	Ingo

Re: [regressions] Re: kbuild: save ARCH & CROSS_COMPILE when building a kernel

[Sam Ravnborg]

> > 
> > The patch in question is reverted in my next push to Linus.
> 
> Thanks! Mind sending the fix/revert here too, so that we can test 
> whether that resolves all the problems?

Here it is.
I intended to send it out before sending pull request to Linus but I forgot.

	Sam

commit 2331d1a6cd3d6e580bc88b9a160066d9e1177fe1
Author: Sam Ravnborg <sam@ravnborg.org>
Date:   Sun Oct 11 23:22:58 2009 +0200

    kbuild: revert "save ARCH & CROSS_COMPILE ..."
    
    Revert commit 575543347b5baed0ca927cb90ba8807396fe9cc9
    
    It caused following issues:
    
    - On architectures where ARCH= setting is used to select between
      32 and 64 bit this was no longer possible without "make mrproper"
    - If ARCH was changed then kbuild refused to run "make mrproper"
      because ARCH had changed
    - When CROSS_COMPILE was changed people were asked to run "make mrproper"
      but kbuild refused to run "make mrproper" because CROSS_COMPILE changed.
    - Spaces in CROSS_COMPILE was not 'supported'
    - If an non-existing ARCH= was used kbuild could get stuck
    
    Lessons learned:
    . Despite being simple and straghtforward people uses very different
      approaches when building the kernel.
    
    . CROSS_COMPILE is sometimes used for ccache despite cache being
      only a CC frontend so one would have expected CC to be
      used for this purpose.
    
    . And obviously this was not tested widely enough.
    
    Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
    Cc: Pavel Machek <pavel@ucw.cz>
    Cc: Ingo Molnar <mingo@elte.hu>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Andrew Morton <akpm@linux-foundation.org>

diff --git a/Makefile b/Makefile
index e50569a..8e5aaee 100644
--- a/Makefile
+++ b/Makefile
@@ -179,46 +179,9 @@ SUBARCH := $(shell uname -m | sed -e s/i.86/i386/ -e s/sun4u/sparc64/ \
 # Alternatively CROSS_COMPILE can be set in the environment.
 # Default value for CROSS_COMPILE is not to prefix executables
 # Note: Some architectures assign CROSS_COMPILE in their arch/*/Makefile
-#
-# To force ARCH and CROSS_COMPILE settings include kernel.* files
-# in the kernel tree - do not patch this file.
 export KBUILD_BUILDHOST := $(SUBARCH)
-
-# Kbuild save the ARCH and CROSS_COMPILE setting in kernel.* files.
-# Restore these settings and check that user did not specify
-# conflicting values.
-
-saved_arch  := $(shell cat include/generated/kernel.arch  2> /dev/null)
-saved_cross := $(shell cat include/generated/kernel.cross 2> /dev/null)
-
-ifneq ($(CROSS_COMPILE),)
-        ifneq ($(saved_cross),)
-                ifneq ($(CROSS_COMPILE),$(saved_cross))
-                        $(error CROSS_COMPILE changed from \
-                                "$(saved_cross)" to \
-                                 to "$(CROSS_COMPILE)". \
-                                 Use "make mrproper" to fix it up)
-                endif
-        endif
-else
-    CROSS_COMPILE := $(saved_cross)
-endif
-
-ifneq ($(ARCH),)
-        ifneq ($(saved_arch),)
-                ifneq ($(saved_arch),$(ARCH))
-                        $(error ARCH changed from \
-                                "$(saved_arch)" to "$(ARCH)". \
-                                 Use "make mrproper" to fix it up)
-                endif
-        endif
-else
-        ifneq ($(saved_arch),)
-                ARCH := $(saved_arch)
-        else
-                ARCH := $(SUBARCH)
-        endif
-endif
+ARCH		?= $(SUBARCH)
+CROSS_COMPILE	?=
 
 # Architecture as present in compile.h
 UTS_MACHINE 	:= $(ARCH)
@@ -483,11 +446,6 @@ ifeq ($(config-targets),1)
 include $(srctree)/arch/$(SRCARCH)/Makefile
 export KBUILD_DEFCONFIG KBUILD_KCONFIG
 
-# save ARCH & CROSS_COMPILE settings
-$(shell mkdir -p include/generated &&                            \
-        echo $(ARCH)          > include/generated/kernel.arch && \
-        echo $(CROSS_COMPILE) > include/generated/kernel.cross)
-
 config: scripts_basic outputmakefile FORCE
 	$(Q)mkdir -p include/linux include/config
 	$(Q)$(MAKE) $(build)=scripts/kconfig $@

Re: [regressions] Re: kbuild: save ARCH & CROSS_COMPILE when building a kernel

[Ingo Molnar]

* Sam Ravnborg <sam@ravnborg.org> wrote:

> > > 
> > > The patch in question is reverted in my next push to Linus.
> > 
> > Thanks! Mind sending the fix/revert here too, so that we can test 
> > whether that resolves all the problems?
> 
> Here it is.
> I intended to send it out before sending pull request to Linus but I forgot.
> 
> 	Sam
> 
> commit 2331d1a6cd3d6e580bc88b9a160066d9e1177fe1
> Author: Sam Ravnborg <sam@ravnborg.org>
> Date:   Sun Oct 11 23:22:58 2009 +0200
> 
>     kbuild: revert "save ARCH & CROSS_COMPILE ..."
>     
>     Revert commit 575543347b5baed0ca927cb90ba8807396fe9cc9
>     
>     It caused following issues:

Thanks Sam!!

	Ingo

Re: [regressions] Re: kbuild: save ARCH & CROSS_COMPILE when building a kernel

[Peter Zijlstra]

On Sat, 2009-10-03 at 12:34 +0200, Ingo Molnar wrote:
> 
> that arch-cache thing introduced in:
> 
>  5755433: kbuild: save ARCH & CROSS_COMPILE when building a kernel
> 
> isnt working very well in my experience.

Its breaking things for me too. When I build a .config using distcc and
cross compilers on my build farm, and then try a make install on another
machine which doesn't have either it utterly shits itself.

Please revert this.

Re: [regressions] Re: kbuild: save ARCH & CROSS_COMPILE when building a kernel

[Ingo Molnar]

* Peter Zijlstra <peterz@infradead.org> wrote:

> On Sat, 2009-10-03 at 12:34 +0200, Ingo Molnar wrote:
> > 
> > that arch-cache thing introduced in:
> > 
> >  5755433: kbuild: save ARCH & CROSS_COMPILE when building a kernel
> > 
> > isnt working very well in my experience.
> 
> Its breaking things for me too. When I build a .config using distcc 
> and cross compilers on my build farm, and then try a make install on 
> another machine which doesn't have either it utterly shits itself.
> 
> Please revert this.

I'm hitting problems with this several times a day.

I'm used to switching from 32-bit to 64-bit x86 kernels and now i have 
to via the extra 'make mrproper' and that's quite annoying as i already 
tell kbuild that i want a 64-bit config via:

  make ARCH=x86_64 defconfig

Why the extra forced step?

Also, there's things that need even more steps: i often take a config 
that triggers a breakage on 32-bit and change it to 64-bit via 'make 
oldconfig' - to see whether the bug is bitness invariant. I have to do 
something like:

  mv .config .config.tmp
  make mrproper
  mv .config.tmp .config
  make ARCH=x86_64 oldconfig

This doesnt look right at all.

	Ingo

Re: [regressions] Re: kbuild: save ARCH & CROSS_COMPILE when building a kernel

[Peter Zijlstra]

On Fri, 2009-10-09 at 10:58 +0200, Peter Zijlstra wrote:
> On Sat, 2009-10-03 at 12:34 +0200, Ingo Molnar wrote:
> > 
> > that arch-cache thing introduced in:
> > 
> >  5755433: kbuild: save ARCH & CROSS_COMPILE when building a kernel
> > 
> > isnt working very well in my experience.
> 
> Its breaking things for me too. When I build a .config using distcc and
> cross compilers on my build farm, and then try a make install on another
> machine which doesn't have either it utterly shits itself.

It also wrecks the online/offline workflow I have.

When connected with 10gbe I use the build cluster from my laptop, when
wireless or offline I clearly cannot use distcc.

My scripts automagically do this right, except now they barf.

It basically means I have to now script around this crap by trap'ing the
build script and always deleting the $BUILD_DIR/include/generated crap.

Is there really anybody out there who hasn't scripted their distcc/cross
build stuff and types that incredibly long make line every time?

Re: [regressions] Re: kbuild: save ARCH & CROSS_COMPILE when building a kernel

[Pavel Machek]

On Fri 2009-10-09 10:58:51, Peter Zijlstra wrote:
> On Sat, 2009-10-03 at 12:34 +0200, Ingo Molnar wrote:
> > 
> > that arch-cache thing introduced in:
> > 
> >  5755433: kbuild: save ARCH & CROSS_COMPILE when building a kernel
> > 
> > isnt working very well in my experience.
> 
> Its breaking things for me too. When I build a .config using distcc and
> cross compilers on my build farm, and then try a make install on another
> machine which doesn't have either it utterly shits itself.

Well, I use CROSS_COMPILE="ccache " (note the space), and that breaks
build in a way where not even mrproper helps...  

> Please revert this.

Please do.
									Pavel

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html