Re: PATCH: Add --size-check=[error|warning]

Re: PATCH: Add --size-check=[error|warning]

[Ingo Molnar]

(H.J. Lu, did you drop me from the Cc: line?)

* Jan Beulich <JBeulich@novell.com> wrote:

> >> Please make it so that it'll be a warning by default, and an error
> >> upon programmer request. Otherwise, for the very purpose of
> > 
> > I disagree. It should be error by default since the input is bogus,
> > Otherwise, those assembly bugs, benign or not, may not get
> > fixed.
> > 
> >> bisection, it won't help much as you would have to override
> >> compiler/assembler flags during that process.
> >>
> > 
> > They can use a wrapper to pass --size-check=warning to
> > assembler.  I think it is a small price to pay for those mistakes.
> 
> "Small" being relative here - it could be dozens if not hundreds of
> people having to learn that this is necessary, many of them
> possibly rather unfamiliar with gas and its command line options.
>
> Also, using a wrapper gets further complicated by the fact that
> you may have to pass an extra -B to the compiler (not everyone
> has full control over the file system of all the machines used to
> do development), making sure this doesn't have any other
> unwanted side effects.

Correct. In reality if the kernel does not build or boot then most people just wont 
continue with the bisection. So this change actively degrades debuggability, for no 
good reason.

The thing is, it is absolutely, breath-takingy incompetent for the new binutils 
version to break the Linux kernel build for 4 years of Linux kernel history 
retroactively (130,000 commits), just to 'warn' about a size bug in a few debug 
symbols that has no functional effects whatsoever and which few people care about.

The correct solution is to turn it into a warning as me and others have suggested.

No argument was offered *why* the build must be aborted. A warning serves the 
purpose of informing the developer just as much - and does not inconvenience the 
tester.

Thanks,

	Ingo

Re: PATCH: Add --size-check=[error|warning]

[Alan Modra]

On Mon, Mar 14, 2011 at 10:55:34AM +0100, Ingo Molnar wrote:
> The thing is, it is absolutely, breath-takingy incompetent for

kernel developers to write such poor asm!  And not notice the error
for 4 years!  Oh, and the binutils developers to write such a poor
assembler in the first place.  ;-)

Seriously, you are complaining because something is fixed??

> The correct solution is to turn it into a warning as me and others have suggested.

I disagree.  The whole world is not the linux kernel.  I think HJ is
bending over backwards to even offer a switch that turns the error
into a warning.

-- 
Alan Modra
Australia Development Lab, IBM

Re: PATCH: Add --size-check=[error|warning]

[Pekka Enberg]

Hi Alan,

On Mon, Mar 14, 2011 at 12:41 PM, Alan Modra <amodra@gmail.com> wrote:
> On Mon, Mar 14, 2011 at 10:55:34AM +0100, Ingo Molnar wrote:
>> The thing is, it is absolutely, breath-takingy incompetent for
>
> kernel developers to write such poor asm!  And not notice the error
> for 4 years!  Oh, and the binutils developers to write such a poor
> assembler in the first place.  ;-)
>
> Seriously, you are complaining because something is fixed??
>
>> The correct solution is to turn it into a warning as me and others have suggested.
>
> I disagree.  The whole world is not the linux kernel.  I think HJ is
> bending over backwards to even offer a switch that turns the error
> into a warning.

So what do you suggest that testers who want to, say, build old Linux
kernel versions with new binutils do?

                        Pekka

Re: PATCH: Add --size-check=[error|warning]

[Alan Cox]

> > I disagree.  The whole world is not the linux kernel.  I think HJ is
> > bending over backwards to even offer a switch that turns the error
> > into a warning.
> 
> So what do you suggest that testers who want to, say, build old Linux
> kernel versions with new binutils do?

Use an older binutils. It's already the case you can't build old kernels
with current gcc and binutils, there have been repeated such events over
time and nobody has had too much trouble.

Alan

Re: PATCH: Add --size-check=[error|warning]

[Jan Beulich]

>>> On 14.03.11 at 11:41, Alan Modra <amodra@gmail.com> wrote:
> On Mon, Mar 14, 2011 at 10:55:34AM +0100, Ingo Molnar wrote:
>> The thing is, it is absolutely, breath-takingy incompetent for
> 
> kernel developers to write such poor asm!  And not notice the error
> for 4 years!  Oh, and the binutils developers to write such a poor
> assembler in the first place.  ;-)
> 
> Seriously, you are complaining because something is fixed??
> 
>> The correct solution is to turn it into a warning as me and others have 
> suggested.
> 
> I disagree.  The whole world is not the linux kernel.  I think HJ is
> bending over backwards to even offer a switch that turns the error
> into a warning.

Just to repeat what I said in a previous reply - an error should be
issued if it is impossible for the assembler to produce valid output.
Anything less severe should be a warning.

In the case given, as also stated before, simply not issuing anything
to the object file if .size has an invalid operand is quite feasible for
the assembler to do, and won't produce invalid output. The warning
tells the programmer that not everything (s)he intended to be in
the object file actually made it there.

Jan

Re: PATCH: Add --size-check=[error|warning]

[Andreas Schwab]

Pekka Enberg <penberg@kernel.org> writes:

> So what do you suggest that testers who want to, say, build old Linux
> kernel versions with new binutils do?

The same that testers have to do in order to build old Linux kernel
versions with current versions of make.

Andreas.

-- 
Andreas Schwab, schwab@redhat.com
GPG Key fingerprint = D4E8 DBE3 3813 BB5D FA84  5EC7 45C6 250E 6F00 984E
"And now for something completely different."

Re: PATCH: Add --size-check=[error|warning]

[Pekka Enberg]

Hi Andreas,

Pekka Enberg <penberg@kernel.org> writes:
>> So what do you suggest that testers who want to, say, build old Linux
>> kernel versions with new binutils do?

On Mon, Mar 14, 2011 at 1:02 PM, Andreas Schwab <schwab@redhat.com> wrote:
> The same that testers have to do in order to build old Linux kernel
> versions with current versions of make.

Are you saying it's OK to screw over binutils users because GNU Make
did that too?

                       Pekka

Re: PATCH: Add --size-check=[error|warning]

[Andreas Schwab]

Pekka Enberg <penberg@kernel.org> writes:

> Hi Andreas,
>
> Pekka Enberg <penberg@kernel.org> writes:
>>> So what do you suggest that testers who want to, say, build old Linux
>>> kernel versions with new binutils do?
>
> On Mon, Mar 14, 2011 at 1:02 PM, Andreas Schwab <schwab@redhat.com> wrote:
>> The same that testers have to do in order to build old Linux kernel
>> versions with current versions of make.
>
> Are you saying it's OK to screw over binutils users because GNU Make
> did that too?

I'm just telling you the facts.

Andreas.

-- 
Andreas Schwab, schwab@redhat.com
GPG Key fingerprint = D4E8 DBE3 3813 BB5D FA84  5EC7 45C6 250E 6F00 984E
"And now for something completely different."

Re: PATCH: Add --size-check=[error|warning]

[Ingo Molnar]

* Andreas Schwab <schwab@redhat.com> wrote:

> Pekka Enberg <penberg@kernel.org> writes:
> 
> > Hi Andreas,
> >
> > Pekka Enberg <penberg@kernel.org> writes:
> >>> So what do you suggest that testers who want to, say, build old Linux
> >>> kernel versions with new binutils do?
> >
> > On Mon, Mar 14, 2011 at 1:02 PM, Andreas Schwab <schwab@redhat.com> wrote:
> >> The same that testers have to do in order to build old Linux kernel
> >> versions with current versions of make.
> >
> > Are you saying it's OK to screw over binutils users because GNU Make
> > did that too?
> 
> I'm just telling you the facts.

And you are wrong - latest Make does not break reasonably old kernel builds such as 
v2.6.30.

Latest binutils insta-breaks the build over 130,000 commits, including the latest 
released kernel.

Please change that .size build error to a build warning, to avoid this unnecessary 
collateral damage.

Thanks,

	Ingo

Re: PATCH: Add --size-check=[error|warning]

[Andreas Schwab]

Ingo Molnar <mingo@elte.hu> writes:

> * Andreas Schwab <schwab@redhat.com> wrote:
>
>> Pekka Enberg <penberg@kernel.org> writes:
>> 
>> > Hi Andreas,
>> >
>> > Pekka Enberg <penberg@kernel.org> writes:
>> >>> So what do you suggest that testers who want to, say, build old Linux
>> >>> kernel versions with new binutils do?
>> >
>> > On Mon, Mar 14, 2011 at 1:02 PM, Andreas Schwab <schwab@redhat.com> wrote:
>> >> The same that testers have to do in order to build old Linux kernel
>> >> versions with current versions of make.
>> >
>> > Are you saying it's OK to screw over binutils users because GNU Make
>> > did that too?
>> 
>> I'm just telling you the facts.
>
> And you are wrong

This is just ridiculous.  You are defining away facts just because they
don't fit your view.

> latest Make does not break reasonably old kernel builds such as
> v2.6.30.

$ git tag --contains 3c955b4 | head -1
v2.6.36

Andreas.

-- 
Andreas Schwab, schwab@redhat.com
GPG Key fingerprint = D4E8 DBE3 3813 BB5D FA84  5EC7 45C6 250E 6F00 984E
"And now for something completely different."

Re: PATCH: Add --size-check=[error|warning]

[Ingo Molnar]

* Andreas Schwab <schwab@redhat.com> wrote:

> Ingo Molnar <mingo@elte.hu> writes:
> 
> > * Andreas Schwab <schwab@redhat.com> wrote:
> >
> >> Pekka Enberg <penberg@kernel.org> writes:
> >> 
> >> > Hi Andreas,
> >> >
> >> > Pekka Enberg <penberg@kernel.org> writes:
> >> >>> So what do you suggest that testers who want to, say, build old Linux
> >> >>> kernel versions with new binutils do?
> >> >
> >> > On Mon, Mar 14, 2011 at 1:02 PM, Andreas Schwab <schwab@redhat.com> wrote:
> >> >> The same that testers have to do in order to build old Linux kernel
> >> >> versions with current versions of make.
> >> >
> >> > Are you saying it's OK to screw over binutils users because GNU Make
> >> > did that too?
> >> 
> >> I'm just telling you the facts.
> >
> > And you are wrong - latest Make does not break reasonably old kernel builds such 
> > as v2.6.30.
> 
> This is just ridiculous.
> You are defining away facts just because they don't fit your view.

No, i actually tried out the latest released Make version (3.82) and it still builds 
v2.6.30 fine:

    LD      arch/x86/boot/setup.elf
    OBJCOPY arch/x86/boot/setup.bin
    BUILD   arch/x86/boot/bzImage
  Root device is (8, 1)
  Setup is 12364 bytes (padded to 12800 bytes). 
  System is 3730 kB
  CRC 77bb7f2d
  Kernel: arch/x86/boot/bzImage is ready  (#105830)

The kernel build count is at 105830 because i build and test a lot of kernels on 
this box.

And the resulting kernel boots fine on a testbox:

  mercury:~> uname -a
  Linux mercury 2.6.30 #105830 SMP Mon Mar 14 12:29:06 CET 2011 x86_64 x86_64 x86_64 GNU/Linux

I have built and booted over half a million Linux kernels in the past 3-4 years so i 
generally have quite a bit of experience when it comes to build environment bugs and 
workflow problems. Breaking the build retroactively and unnecessarily like here is 
one of the worst things that can be done to testing quality and efficiency.

Thanks,

	Ingo

Re: PATCH: Add --size-check=[error|warning]

[Andreas Schwab]

Ingo Molnar <mingo@elte.hu> writes:

> * Andreas Schwab <schwab@redhat.com> wrote:
>
>> Ingo Molnar <mingo@elte.hu> writes:
>> 
>> > * Andreas Schwab <schwab@redhat.com> wrote:
>> >
>> >> Pekka Enberg <penberg@kernel.org> writes:
>> >> 
>> >> > Hi Andreas,
>> >> >
>> >> > Pekka Enberg <penberg@kernel.org> writes:
>> >> >>> So what do you suggest that testers who want to, say, build old Linux
>> >> >>> kernel versions with new binutils do?
>> >> >
>> >> > On Mon, Mar 14, 2011 at 1:02 PM, Andreas Schwab <schwab@redhat.com> wrote:
>> >> >> The same that testers have to do in order to build old Linux kernel
>> >> >> versions with current versions of make.
>> >> >
>> >> > Are you saying it's OK to screw over binutils users because GNU Make
>> >> > did that too?
>> >> 
>> >> I'm just telling you the facts.
>> >
>> > And you are wrong - latest Make does not break reasonably old kernel builds such 
>> > as v2.6.30.
>> 
>> This is just ridiculous.
>> You are defining away facts just because they don't fit your view.
>
> No, i actually tried out the latest released Make version (3.82) and it still builds 
> v2.6.30 fine:

That's a very convincing argument.

Andreas.

-- 
Andreas Schwab, schwab@redhat.com
GPG Key fingerprint = D4E8 DBE3 3813 BB5D FA84  5EC7 45C6 250E 6F00 984E
"And now for something completely different."

Re: PATCH: Add --size-check=[error|warning]

[Ingo Molnar]

* Pekka Enberg <penberg@kernel.org> wrote:

> Hi Andreas,
> 
> Pekka Enberg <penberg@kernel.org> writes:
> >> So what do you suggest that testers who want to, say, build old Linux
> >> kernel versions with new binutils do?
> 
> On Mon, Mar 14, 2011 at 1:02 PM, Andreas Schwab <schwab@redhat.com> wrote:
> > The same that testers have to do in order to build old Linux kernel
> > versions with current versions of make.
> 
> Are you saying it's OK to screw over binutils users because GNU Make
> did that too?

The claim is not even true.

While really old Linux versions wont build with fresh versions of Make, something 
like v2.6.30, which is a 2 years old kernel, will build just fine, using latest 
Make.

So lets stop coming up with all the wrong reasons to not fix this problem ...

Thanks,

	Ingo

Re: PATCH: Add --size-check=[error|warning]

[Ingo Molnar]

* Alan Modra <amodra@gmail.com> wrote:

> On Mon, Mar 14, 2011 at 10:55:34AM +0100, Ingo Molnar wrote:
> > The thing is, it is absolutely, breath-takingy incompetent for
> 
> kernel developers to write such poor asm!  And not notice the error
> for 4 years! [...]

It is not 'poor asm'.

The 'bug' is just a slight assymetry in ENTRY()/END() debug-symbols sequences, with 
lots of assembly code between the ENTRY() and the END(). Here's an example:
    
         ENTRY(xen_do_hypervisor_callback)   # do_hypervisor_callback(struct *pt_regs)
           ...
         END(do_hypervisor_callback)

Human reviewers almost never catch such small mismatches, and binutils never even 
warned about it either - for over a decade.

Now kernel bisections are insta-broken on latest binutils, and there's nothing to do 
about it on the kernel side as during bisection all later fixes are unfolded. The 
fix itself i already applied - but my argument was not about that:

> [...] Oh, and the binutils developers to write such a poor assembler in the first 
> place.  ;-)
> 
> Seriously, you are complaining because something is fixed??

No, i reported this bug because the kernel build gets broken going back 130,000 
commits, breaking bisection and causing other damage - while issuing a warning 
message would achieve the same effect of warning the developer about the mismatch.

> > The correct solution is to turn it into a warning as me and others have suggested.
> 
> I disagree.  The whole world is not the linux kernel.  I think HJ is
> bending over backwards to even offer a switch that turns the error
> into a warning.

It's not about a switch at all - it's to not break builds by default. I.e. the 
default behavior should be to issue a warning and ignore the directive.

This is a very simple concept of compatibility: the build environment should always 
be very permissive - stuff that build fine before should be allowed to build.

Also, i hope you are not suggesting to break projects just because they are not 
important to you personally? The fix is exceedingly simple to do for the binutils 
project - and impossible to do for the kernel project (because during bisection - 
which is a very powerful debugging tool - older versions of the source get checked 
out).

Thanks,

	Ingo

Re: PATCH: Add --size-check=[error|warning]

[Ingo Molnar]

(resend, fixed the To line)

* Alan Modra <amodra@gmail.com> wrote:
 
> On Mon, Mar 14, 2011 at 10:55:34AM +0100, Ingo Molnar wrote:
> > The thing is, it is absolutely, breath-takingy incompetent for
> 
> kernel developers to write such poor asm!  And not notice the error
> for 4 years! [...]

It is not 'poor asm'.

The 'bug' is just a slight assymetry in ENTRY()/END() debug-symbols sequences, with 
lots of assembly code between the ENTRY() and the END(). Here's an example:
    
         ENTRY(xen_do_hypervisor_callback)   # do_hypervisor_callback(struct *pt_regs)
           ...
         END(do_hypervisor_callback)

Human reviewers almost never catch such small mismatches, and binutils never even 
warned about it either - for over a decade.

Now kernel bisections are insta-broken on latest binutils, and there's nothing to do 
about it on the kernel side as during bisection all later fixes are unfolded. The 
fix itself i already applied - but my argument was not about that:

> [...] Oh, and the binutils developers to write such a poor assembler in the first 
> place.  ;-)
> 
> Seriously, you are complaining because something is fixed??
 
No, i reported this bug because the kernel build gets broken going back 130,000 
commits, breaking bisection and causing other damage - while issuing a warning 
message would achieve the same effect of warning the developer about the mismatch.

> > The correct solution is to turn it into a warning as me and others have suggested.
> 
> I disagree.  The whole world is not the linux kernel.  I think HJ is
> bending over backwards to even offer a switch that turns the error
> into a warning.
 
It's not about a switch at all - it's to not break builds by default. I.e. the 
default behavior should be to issue a warning and ignore the directive.
 
This is a very simple concept of compatibility: the build environment should always 
be very permissive - stuff that build fine before should be allowed to build.

Also, i hope you are not suggesting to break projects just because they are not 
important to you personally? The fix is exceedingly simple to do for the binutils 
project - and impossible to do for the kernel project (because during bisection - 
which is a very powerful debugging tool - older versions of the source get checked 
out).

Thanks,

	Ingo

git bisect plus fixes (was: PATCH: Add --size-check=[error|warning])

[Ralf Wildenhues]

[ adding the git list; this is
  http://thread.gmane.org/gmane.comp.gnu.binutils/52601/focus=1112779 ]

Hello,

* Ingo Molnar wrote on Mon, Mar 14, 2011 at 01:23:42PM CET:
> Also, i hope you are not suggesting to break projects just because
> they are not important to you personally? The fix is exceedingly
> simple to do for the binutils project - and impossible to do for the
> kernel project (because during bisection - which is a very powerful
> debugging tool - older versions of the source get checked out).

FWIW, I don't have an opinion on this particular binutils issue, but
it would be very helpful if 'git bisect' made it easy to denote
"when going back, you might also need some of these changes".
(I'd just use a patch -p1 with a here-file in the bisect script, but
that might not be enough for all practical use cases.)

This issue has come up several times with high-profile issues.

Thanks,
Ralf