linux-next: tip-core build failure

linux-next: tip-core build failure

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 1106 bytes --]

Hi Ingo,

Today's linux-next build (powerpc allnoconfig) failed like this:

In file included from arch/powerpc/include/asm/system.h:8,
                 from include/linux/list.h:7,
                 from include/linux/signal.h:8,
                 from arch/powerpc/kernel/asm-offsets.c:16:
include/linux/irqflags.h:89:1: warning: "local_irq_save" redefined
In file included from arch/powerpc/include/asm/irqflags.h:11,
                 from include/linux/irqflags.h:55,
                 from arch/powerpc/include/asm/system.h:8,
                 from include/linux/list.h:7,
                 from include/linux/signal.h:8,
                 from arch/powerpc/kernel/asm-offsets.c:16:
arch/powerpc/include/asm/hw_irq.h:115:1: warning: this is the location of the previous definition

And it went downhill from there.

Introduced by commit bd8fbdee6562ee526f3c2582a3b373ef195015dd ("lockdep:
fix compilation when CONFIG_TRACE_IRQFLAGS_SUPPORT is not set") which I
have reverted.

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au
http://www.canb.auug.org.au/~sfr/

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: linux-next: tip-core build failure

[Rui Sousa]

On Saturday 13 September 2008 03:14, Stephen Rothwell wrote:
> Hi Ingo,
>
> Today's linux-next build (powerpc allnoconfig) failed like this:
>
> In file included from arch/powerpc/include/asm/system.h:8,
>                  from include/linux/list.h:7,
>                  from include/linux/signal.h:8,
>                  from arch/powerpc/kernel/asm-offsets.c:16:
> include/linux/irqflags.h:89:1: warning: "local_irq_save" redefined
> In file included from arch/powerpc/include/asm/irqflags.h:11,
>                  from include/linux/irqflags.h:55,
>                  from arch/powerpc/include/asm/system.h:8,
>                  from include/linux/list.h:7,
>                  from include/linux/signal.h:8,
>                  from arch/powerpc/kernel/asm-offsets.c:16:
> arch/powerpc/include/asm/hw_irq.h:115:1: warning: this is the location of
> the previous definition
>
> And it went downhill from there.
>
> Introduced by commit bd8fbdee6562ee526f3c2582a3b373ef195015dd ("lockdep:
> fix compilation when CONFIG_TRACE_IRQFLAGS_SUPPORT is not set") which I
> have reverted.

Ingo,

Do you agree that the original (in include/linux/irqflags.h):

#define raw_local_irq_disable()        local_irq_disable()

should be

#define local_irq_disable()		raw_local_irq_disable()        

and that architecture specific files should only define the raw_xxx() macros?
It seems I missed a bunch of need changes in the original patch, but if it's
the right way to go I can work on a patch to update the other archs too.

Thanks,
Rui

Re: linux-next: tip-core build failure

[Ingo Molnar]

* Rui Sousa <rui.p.m.sousa@gmail.com> wrote:

> On Saturday 13 September 2008 03:14, Stephen Rothwell wrote:
> > Hi Ingo,
> >
> > Today's linux-next build (powerpc allnoconfig) failed like this:
> >
> > In file included from arch/powerpc/include/asm/system.h:8,
> >                  from include/linux/list.h:7,
> >                  from include/linux/signal.h:8,
> >                  from arch/powerpc/kernel/asm-offsets.c:16:
> > include/linux/irqflags.h:89:1: warning: "local_irq_save" redefined
> > In file included from arch/powerpc/include/asm/irqflags.h:11,
> >                  from include/linux/irqflags.h:55,
> >                  from arch/powerpc/include/asm/system.h:8,
> >                  from include/linux/list.h:7,
> >                  from include/linux/signal.h:8,
> >                  from arch/powerpc/kernel/asm-offsets.c:16:
> > arch/powerpc/include/asm/hw_irq.h:115:1: warning: this is the location of
> > the previous definition
> >
> > And it went downhill from there.
> >
> > Introduced by commit bd8fbdee6562ee526f3c2582a3b373ef195015dd ("lockdep:
> > fix compilation when CONFIG_TRACE_IRQFLAGS_SUPPORT is not set") which I
> > have reverted.
> 
> Ingo,
> 
> Do you agree that the original (in include/linux/irqflags.h):
> 
> #define raw_local_irq_disable()        local_irq_disable()
> 
> should be
> 
> #define local_irq_disable()		raw_local_irq_disable()        
> 
> and that architecture specific files should only define the raw_xxx() 
> macros? It seems I missed a bunch of need changes in the original 
> patch, but if it's the right way to go I can work on a patch to update 
> the other archs too.

yes, i agree with that direction. But it's difficult: the non-x86 
architectures do not get much testing in practice at the development 
stage so trivial build bugs like this get only found at the latest 
stage. That's not actually bad in itself - stuff that is not used does 
not matter nearly as much as stuff that does get used.

	Ingo

Re: linux-next: tip-core build failure

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 946 bytes --]

Hi Ingo,

On Sun, 14 Sep 2008 14:43:20 +0200 Ingo Molnar <mingo@elte.hu> wrote:
>
> yes, i agree with that direction. But it's difficult: the non-x86 
> architectures do not get much testing in practice at the development 
> stage so trivial build bugs like this get only found at the latest 
> stage. That's not actually bad in itself - stuff that is not used does 
> not matter nearly as much as stuff that does get used.

Well, its not really that difficult, you just have to remember that x86
is not the whole world and touching generic code just might affect the
other architectures - especially when the code you are touching is using
stuff that is clearly implemented by the architectures. Posting stuff
like this to linux-arch usually gets some attention (my apologies if you
did so, I don't remember seeing it).

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au
http://www.canb.auug.org.au/~sfr/

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: linux-next: tip-core build failure

[Ingo Molnar]

* Stephen Rothwell <sfr@canb.auug.org.au> wrote:

> Hi Ingo,
> 
> On Sun, 14 Sep 2008 14:43:20 +0200 Ingo Molnar <mingo@elte.hu> wrote:
> >
> > yes, i agree with that direction. But it's difficult: the non-x86 
> > architectures do not get much testing in practice at the development 
> > stage so trivial build bugs like this get only found at the latest 
> > stage. That's not actually bad in itself - stuff that is not used does 
> > not matter nearly as much as stuff that does get used.
> 
> Well, its not really that difficult, you just have to remember that x86
> is not the whole world [...]

[ ... just used by 90%+ of our active testers/developers ;-) ... ]

> [...] and touching generic code just might affect the other 
> architectures - especially when the code you are touching is using 
> stuff that is clearly implemented by the architectures. Posting stuff 
> like this to linux-arch usually gets some attention (my apologies if 
> you did so, I don't remember seeing it).

hm, you seem to have a bias for powerpc, but you should realize that 
cross-building for 20+ architectures (i.e. increasing the testing 
overhead twenty-fold), to cover the remaining <10% of the test space is 
unreasonable: for many developers it's not just virtually impossible in 
practice but also often a serious waste of time.

Most people who have ever tried to collect cross-compilers for _all_ 
architectures and managed to get them all to work, and then used them 
consistently for all their patches will agree with that.

We want to push unreasonable work to those who depend on the result of 
that unreasonable work - i.e. users/developers of those platforms - not 
everyone else. We dont want to hinder the progress of Linux with blindly 
requiring all patches that happen to touch common .c or .h files to 
successfully build on 20+ odd architectures.

... anyway, no real arguments about this specific case, if a fix/report 
is available we'll integrate/fix the issue.

	Ingo

Re: linux-next: tip-core build failure

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 1873 bytes --]

Hi Ingo,

On Sun, 14 Sep 2008 20:36:29 +0200 Ingo Molnar <mingo@elte.hu> wrote:
>
> * Stephen Rothwell <sfr@canb.auug.org.au> wrote:
> > 
> > Well, its not really that difficult, you just have to remember that x86
> > is not the whole world [...]
> 
> [ ... just used by 90%+ of our active testers/developers ;-) ... ]

An irrelevant argument in this case.

> hm, you seem to have a bias for powerpc, but you should realize that 

And you seem to have a bias for x86, so what!

> cross-building for 20+ architectures (i.e. increasing the testing 
> overhead twenty-fold), to cover the remaining <10% of the test space is 
> unreasonable: for many developers it's not just virtually impossible in 
> practice but also often a serious waste of time.

I am not asking for that.

> We want to push unreasonable work to those who depend on the result of 
> that unreasonable work - i.e. users/developers of those platforms - not 
> everyone else. We dont want to hinder the progress of Linux with blindly 
> requiring all patches that happen to touch common .c or .h files to 
> successfully build on 20+ odd architectures.

But doing at least a simple grep for usages of the thing you are
changing, that is not unreasonable ...  especially if you are changing
(usually not well defined in the first place) interfaces that the
architectures have had to implement (as was the case here).

> ... anyway, no real arguments about this specific case, if a fix/report 
> is available we'll integrate/fix the issue.

Thanks.

Besides, Ingo, many of the TIP trees (as I understand them) are not x86
specific, so expecting them to build on more than one architecture is
not unreasonable.  This is part of the job of the integrator ...

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au
http://www.canb.auug.org.au/~sfr/

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: linux-next: tip-core build failure

[Ingo Molnar]

* Stephen Rothwell <sfr@canb.auug.org.au> wrote:

> Hi Ingo,
> 
> On Sun, 14 Sep 2008 20:36:29 +0200 Ingo Molnar <mingo@elte.hu> wrote:
> >
> > * Stephen Rothwell <sfr@canb.auug.org.au> wrote:
> > > 
> > > Well, its not really that difficult, you just have to remember that x86
> > > is not the whole world [...]
> > 
> > [ ... just used by 90%+ of our active testers/developers ;-) ... ]
> 
> An irrelevant argument in this case.

why is the actual usage distribution of Linux irrelevant? We make 
maintenance and testing prioritization based on usage and popularity on 
a daily basis: for example an unfixed bug in ext3 can easily stall an 
-rc or a stable release - an unfixed bug in freevxfs wont.

> > hm, you seem to have a bias for powerpc, but you should realize that
> 
> And you seem to have a bias for x86, so what!

Well, having a bias towards the most popular code and most popular 
devices and platforms is not just acceptable and it is not just common 
sense - it's also a basic required skill from any Linux subsystem 
maintainer. Ignoring the most popular usage would be silly and 
counter-productive.

And that requirement can be turned around to a certain degree: having an 
unreasonable bias _against_ popular platforms is counterproductive. I.e. 
you should weigh whether forcing the unreasonable use of our testing and 
development resources is good for Linux.

> > cross-building for 20+ architectures (i.e. increasing the testing 
> > overhead twenty-fold), to cover the remaining <10% of the test space 
> > is unreasonable: for many developers it's not just virtually 
> > impossible in practice but also often a serious waste of time.
> 
> I am not asking for that.
>
> > We want to push unreasonable work to those who depend on the result 
> > of that unreasonable work - i.e. users/developers of those platforms 
> > - not everyone else. We dont want to hinder the progress of Linux 
> > with blindly requiring all patches that happen to touch common .c or 
> > .h files to successfully build on 20+ odd architectures.
> 
> But doing at least a simple grep for usages of the thing you are 
> changing, that is not unreasonable ...  especially if you are changing 
> (usually not well defined in the first place) interfaces that the 
> architectures have had to implement (as was the case here).

this assumes that the connection is realized. It was not realized in 
this case.

> > ... anyway, no real arguments about this specific case, if a 
> > fix/report is available we'll integrate/fix the issue.
> 
> Thanks.
> 
> Besides, Ingo, many of the TIP trees (as I understand them) are not 
> x86 specific, so expecting them to build on more than one architecture 
> is not unreasonable.  This is part of the job of the integrator ...

we cross-build them (and fix the bugs, if any are found), but it's all 
driven by demand really or when we suspect there might be cross-arch 
breakage (it wasnt in this case). If i have the choice between analysing 
and fixing a bug that was reported by a real user and spending hours on 
cross-builds i do chose the one that helps Linux more.

	Ingo

Re: linux-next: tip-core build failure

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 2789 bytes --]

Hi Ingo,

On Sun, 14 Sep 2008 21:17:09 +0200 Ingo Molnar <mingo@elte.hu> wrote:
>
> * Stephen Rothwell <sfr@canb.auug.org.au> wrote:
> 
> > On Sun, 14 Sep 2008 20:36:29 +0200 Ingo Molnar <mingo@elte.hu> wrote:
> > >
> > > [ ... just used by 90%+ of our active testers/developers ;-) ... ]
> > 
> > An irrelevant argument in this case.
> 
> why is the actual usage distribution of Linux irrelevant? We make 

Notice the "in this case"?  In this case the code being changed clearly
could affect other architectures, so checking that x86 is OK is clearly
not enough (and that does not mean building it for 20+ other
architectures, just using grep to see the consequences of the change and
maybe discussing it with the affected architecture maintainers or on
linux-arch).

> Well, having a bias towards the most popular code and most popular 
> devices and platforms is not just acceptable and it is not just common 
> sense - it's also a basic required skill from any Linux subsystem 
> maintainer. Ignoring the most popular usage would be silly and 
> counter-productive.

I agree with you whole heartedly, and if any other architecture
maintainer broke x86, I would be complaining as well. And that has
nothing to do with what I said.

> And that requirement can be turned around to a certain degree: having an 
> unreasonable bias _against_ popular platforms is counterproductive. I.e. 
> you should weigh whether forcing the unreasonable use of our testing and 
> development resources is good for Linux.

And again, this has nothing to do with what I said.  In fact, I build
every merge of linux-next for x86_64 allmodconfig ... in no way do I have
a bias against any architecture and I am not trying to force everyone to
do lots of cross builds, just, when making changes that could affect
other architectures, to make a little effort to reduce the pain.

> > But doing at least a simple grep for usages of the thing you are 
> > changing, that is not unreasonable ...  especially if you are changing 
> > (usually not well defined in the first place) interfaces that the 
> > architectures have had to implement (as was the case here).
> 
> this assumes that the connection is realized. It was not realized in 
> this case.

OK, then let move on.

> we cross-build them (and fix the bugs, if any are found), but it's all 
> driven by demand really or when we suspect there might be cross-arch 
> breakage (it wasnt in this case). If i have the choice between analysing 
> and fixing a bug that was reported by a real user and spending hours on 
> cross-builds i do chose the one that helps Linux more.

Good to know.

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au
http://www.canb.auug.org.au/~sfr/

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: linux-next: tip-core build failure

[Ingo Molnar]

* Stephen Rothwell <sfr@canb.auug.org.au> wrote:

> Hi Ingo,
> 
> On Sun, 14 Sep 2008 21:17:09 +0200 Ingo Molnar <mingo@elte.hu> wrote:
> >
> > * Stephen Rothwell <sfr@canb.auug.org.au> wrote:
> > 
> > > On Sun, 14 Sep 2008 20:36:29 +0200 Ingo Molnar <mingo@elte.hu> wrote:
> > > >
> > > > [ ... just used by 90%+ of our active testers/developers ;-) ... ]
> > > 
> > > An irrelevant argument in this case.
> > 
> > why is the actual usage distribution of Linux irrelevant? We make 
> 
> Notice the "in this case"?  In this case the code being changed 
> clearly could affect other architectures, so checking that x86 is OK 
> is clearly not enough (and that does not mean building it for 20+ 
> other architectures, just using grep to see the consequences of the 
> change and maybe discussing it with the affected architecture 
> maintainers or on linux-arch).

again, the side effects were not realized. Pretty much _any_ patch that 
is not strictly restricted to a single architecture 'could' affect other 
architectures. Any change to a common .h or .c file could do that - and 
90% of Linux's source code is in common files.

Hence your suggestion that we should have found this breakage makes no 
sense in practice as it's not reliably testable.

Yes, for things like sparseirq support we pretty much expected 
cross-arch fallout so we checked a ton of architectures. For other 
patches bugs can slip through, and that's OK as we really dont want to 
dog down pretty much every patch with the requirement to cross-build.

	Ingo

Re: linux-next: tip-core build failure

[Rui Sousa]

On Monday 15 September 2008 10:11, Ingo Molnar wrote:
> * Stephen Rothwell <sfr@canb.auug.org.au> wrote:
> > Hi Ingo,
> >
> > On Sun, 14 Sep 2008 21:17:09 +0200 Ingo Molnar <mingo@elte.hu> wrote:
> > > * Stephen Rothwell <sfr@canb.auug.org.au> wrote:
> > > > On Sun, 14 Sep 2008 20:36:29 +0200 Ingo Molnar <mingo@elte.hu> wrote:
> > > > > [ ... just used by 90%+ of our active testers/developers ;-) ... ]
> > > >
> > > > An irrelevant argument in this case.
> > >
> > > why is the actual usage distribution of Linux irrelevant? We make
> >
> > Notice the "in this case"?  In this case the code being changed
> > clearly could affect other architectures, so checking that x86 is OK
> > is clearly not enough (and that does not mean building it for 20+
> > other architectures, just using grep to see the consequences of the
> > change and maybe discussing it with the affected architecture
> > maintainers or on linux-arch).
>
> again, the side effects were not realized. Pretty much _any_ patch that
> is not strictly restricted to a single architecture 'could' affect other
> architectures. Any change to a common .h or .c file could do that - and
> 90% of Linux's source code is in common files.
>
> Hence your suggestion that we should have found this breakage makes no
> sense in practice as it's not reliably testable.
>
> Yes, for things like sparseirq support we pretty much expected
> cross-arch fallout so we checked a ton of architectures. For other
> patches bugs can slip through, and that's OK as we really dont want to
> dog down pretty much every patch with the requirement to cross-build.
>
> 	Ingo

Hi,

I have just subscribed to linux-arch and will work on a patch updating all 
archictures. Once I have it I will post it on the list for discussion/review.

One thing that would be nice, but I don't see how it can be done in practice,
is to have a public server, where people can submit their patches and at least
make sure they compile on all archictetures.

Thanks,
Rui