Announce: gcc bogus warning repository

Announce: gcc bogus warning repository

[Jeff Garzik]

The level of warnings in a kernel build has lately increased to the 
point where it is hiding bugs and otherwise making life difficult.

In particular, recent gcc versions throw warnings when it thinks a 
variable "MAY be used uninitialized", which is not terribly helpful due 
to the fact that most of these warnings are bogus.

For those that may find this valuable, I have started a git repo that 
silences these bogus warnings, after careful auditing of code paths to 
ensure that the warning truly is bogus.

The results may be found in the "gccbug" branch of
git://git.kernel.org/pub/scm/linux/kernel/git/jgarzik/misc-2.6.git

This repository will NEVER EVER be pushed upstream.  It exists solely 
for those who want to decrease their build noise, thereby exposing true 
bugs.

The audit has already uncovered several minor bugs, lending credence to 
my theory that too many warnings hides bugs.

Re: Announce: gcc bogus warning repository

[Al Viro]

On Sun, Oct 01, 2006 at 09:44:55AM -0400, Jeff Garzik wrote:
> This repository will NEVER EVER be pushed upstream.  It exists solely 
> for those who want to decrease their build noise, thereby exposing true 
> bugs.

Another way to deal with that is to use remap-log...  See
git://git.kernel.org/pub/scm/linux/kernel/git/viro/remapper.git/
for current version; it's _very_ good at reducing noise in diffs
from line renumbering, leaving the real changes.  While the number
of bogus warnings is very high, indeed, the rate at which they
appear is not _that_ horrible...

Re: Announce: gcc bogus warning repository

[Daniel Walker]

On Sun, 2006-10-01 at 09:44 -0400, Jeff Garzik wrote:
> The level of warnings in a kernel build has lately increased to the 
> point where it is hiding bugs and otherwise making life difficult.
> 
> In particular, recent gcc versions throw warnings when it thinks a 
> variable "MAY be used uninitialized", which is not terribly helpful due 
> to the fact that most of these warnings are bogus.
> 
> For those that may find this valuable, I have started a git repo that 
> silences these bogus warnings, after careful auditing of code paths to 
> ensure that the warning truly is bogus.
> 
> The results may be found in the "gccbug" branch of
> git://git.kernel.org/pub/scm/linux/kernel/git/jgarzik/misc-2.6.git
> 

Steven Rostedt an I worked on this problem in May. Steven came up with,
a nice way to handle these warnings, which doesn't increase code size.
Here's the post if your interested.

http://lkml.org/lkml/2006/5/11/50

Daniel

Re: Announce: gcc bogus warning repository

[Andrew Morton]

On Sun, 01 Oct 2006 08:40:13 -0700
Daniel Walker <dwalker@mvista.com> wrote:

> On Sun, 2006-10-01 at 09:44 -0400, Jeff Garzik wrote:
> > The level of warnings in a kernel build has lately increased to the 
> > point where it is hiding bugs and otherwise making life difficult.
> > 
> > In particular, recent gcc versions throw warnings when it thinks a 
> > variable "MAY be used uninitialized", which is not terribly helpful due 
> > to the fact that most of these warnings are bogus.
> > 
> > For those that may find this valuable, I have started a git repo that 
> > silences these bogus warnings, after careful auditing of code paths to 
> > ensure that the warning truly is bogus.
> > 
> > The results may be found in the "gccbug" branch of
> > git://git.kernel.org/pub/scm/linux/kernel/git/jgarzik/misc-2.6.git
> > 

Funny - I started doing the exact same thing here just eight hours ago. 
I'll send you anything which isn't in that git tree, thanks.

> Steven Rostedt an I worked on this problem in May. Steven came up with,
> a nice way to handle these warnings, which doesn't increase code size.
> Here's the post if your interested.
> 
> http://lkml.org/lkml/2006/5/11/50

I think we should merge that and use it.  No overhead, self-documenting,
easily greppable for.

The downsides are that it muckies up the source a little and introduces a
very small risk that real use-uninitialised bugs will be hidden.  But I
believe the benefit outweighs those disadvantages.

Re: Announce: gcc bogus warning repository

[Jeff Garzik]

Andrew Morton wrote:
> The downsides are that it muckies up the source a little and introduces a
> very small risk that real use-uninitialised bugs will be hidden.  But I
> believe the benefit outweighs those disadvantages.

How about just marking the ones I've already done in #gccbug?

If I'm taking the time to audit the code, and separate out bogosities 
from real bugs, it would be nice not to see that effort wasted.

#gccbug includes _only_ the bogosities.  I didn't just blindly paper 
over everything with a 'may be used uninitialized' warning.  I'm well 
over halfway through the 'make allmodconfig' build, and as LKML emails 
can attest, have found several valid warnings.

	Jeff

Re: Announce: gcc bogus warning repository

[Daniel Walker]

On Sun, 2006-10-01 at 14:16 -0400, Jeff Garzik wrote:
> Andrew Morton wrote:
> > The downsides are that it muckies up the source a little and introduces a
> > very small risk that real use-uninitialised bugs will be hidden.  But I
> > believe the benefit outweighs those disadvantages.
> 
> How about just marking the ones I've already done in #gccbug?
> 
> If I'm taking the time to audit the code, and separate out bogosities 
> from real bugs, it would be nice not to see that effort wasted.

There was a long thread on this, it's not about anyone not reviewing the
code properly when the warning is first silenced. It's that future
changes might create new problems that are also silenced. I don't think
it's a huge concern, especially since there's was a config option to
turn the warning backs on.

Daniel

Re: Announce: gcc bogus warning repository

[Jeff Garzik]

Daniel Walker wrote:
> On Sun, 2006-10-01 at 14:16 -0400, Jeff Garzik wrote:
>> Andrew Morton wrote:
>>> The downsides are that it muckies up the source a little and introduces a
>>> very small risk that real use-uninitialised bugs will be hidden.  But I
>>> believe the benefit outweighs those disadvantages.
>> How about just marking the ones I've already done in #gccbug?
>>
>> If I'm taking the time to audit the code, and separate out bogosities 
>> from real bugs, it would be nice not to see that effort wasted.
> 
> There was a long thread on this, it's not about anyone not reviewing the
> code properly when the warning is first silenced. It's that future
> changes might create new problems that are also silenced. I don't think
> it's a huge concern, especially since there's was a config option to
> turn the warning backs on.

That doesn't address my question at all.

If there is no difference between real non-init bugs and bogus warnings, 
then a config option doesn't make any difference at all, does it?  Real 
bugs are still hidden either way:  if the warnings are turned on, the 
bugs are lost in the noise.  if the warnings are turned off, the bugs 
are completely hidden.

	Jeff

Re: Announce: gcc bogus warning repository

[Daniel Walker]

On Sun, 2006-10-01 at 14:45 -0400, Jeff Garzik wrote:

> That doesn't address my question at all.

Did you have a question?

> If there is no difference between real non-init bugs and bogus warnings, 
> then a config option doesn't make any difference at all, does it?  Real 
> bugs are still hidden either way:  if the warnings are turned on, the 
> bugs are lost in the noise.  if the warnings are turned off, the bugs 
> are completely hidden.

If you turn the warnings on, at least you have a chance to see a warning
even if it's mixed with others.

Daniel

Re: Announce: gcc bogus warning repository

[Al Viro]

On Sun, Oct 01, 2006 at 11:58:33AM -0700, Daniel Walker wrote:
> On Sun, 2006-10-01 at 14:45 -0400, Jeff Garzik wrote:
> 
> > That doesn't address my question at all.
> 
> Did you have a question?
> 
> > If there is no difference between real non-init bugs and bogus warnings, 
> > then a config option doesn't make any difference at all, does it?  Real 
> > bugs are still hidden either way:  if the warnings are turned on, the 
> > bugs are lost in the noise.  if the warnings are turned off, the bugs 
> > are completely hidden.
> 
> If you turn the warnings on, at least you have a chance to see a warning
> even if it's mixed with others.

And that's better than the current situation in which respects, exactly?

Re: Announce: gcc bogus warning repository

[Daniel Walker]

On Sun, 2006-10-01 at 20:00 +0100, Al Viro wrote:
> On Sun, Oct 01, 2006 at 11:58:33AM -0700, Daniel Walker wrote:
> > On Sun, 2006-10-01 at 14:45 -0400, Jeff Garzik wrote:
> > 
> > > That doesn't address my question at all.
> > 
> > Did you have a question?
> > 
> > > If there is no difference between real non-init bugs and bogus warnings, 
> > > then a config option doesn't make any difference at all, does it?  Real 
> > > bugs are still hidden either way:  if the warnings are turned on, the 
> > > bugs are lost in the noise.  if the warnings are turned off, the bugs 
> > > are completely hidden.
> > 
> > If you turn the warnings on, at least you have a chance to see a warning
> > even if it's mixed with others.
> 
> And that's better than the current situation in which respects, exactly?

Seeing the warnings is the current situation.

Daniel

Re: Announce: gcc bogus warning repository

[Al Viro]

On Sun, Oct 01, 2006 at 12:03:10PM -0700, Daniel Walker wrote:
> > And that's better than the current situation in which respects, exactly?
> 
> Seeing the warnings is the current situation.

I bow to your incredible power of observation.  Now that you've shared
that revelation with the list, could you explain what does blanket silencing
of these warnings buy you, oh wan^H^Hise one?

Re: Announce: gcc bogus warning repository

[Daniel Walker]

On Sun, 2006-10-01 at 20:07 +0100, Al Viro wrote:
> On Sun, Oct 01, 2006 at 12:03:10PM -0700, Daniel Walker wrote:
> > > And that's better than the current situation in which respects, exactly?
> > 
> > Seeing the warnings is the current situation.
> 
> I bow to your incredible power of observation.  Now that you've shared
> that revelation with the list, could you explain what does blanket silencing
> of these warnings buy you, oh wan^H^Hise one?

Did you see me silencing anything (with your crystal ball?) ? Cause I'm
not.

Daniel

Re: Announce: gcc bogus warning repository

[Al Viro]

On Sun, Oct 01, 2006 at 12:13:54PM -0700, Daniel Walker wrote:
> On Sun, 2006-10-01 at 20:07 +0100, Al Viro wrote:
> > On Sun, Oct 01, 2006 at 12:03:10PM -0700, Daniel Walker wrote:
> > > > And that's better than the current situation in which respects, exactly?
> > > 
> > > Seeing the warnings is the current situation.
> > 
> > I bow to your incredible power of observation.  Now that you've shared
> > that revelation with the list, could you explain what does blanket silencing
> > of these warnings buy you, oh wan^H^Hise one?
> 
> Did you see me silencing anything (with your crystal ball?) ? Cause I'm
> not.

And what, in your opinion, does the patch in question achieve if not
that?

PS: sorry, I leave crystal balls to RMS et.al. - never had been
into that kind of scam...

Re: Announce: gcc bogus warning repository

[Daniel Walker]

On Sun, 2006-10-01 at 20:20 +0100, Al Viro wrote:
> On Sun, Oct 01, 2006 at 12:13:54PM -0700, Daniel Walker wrote:
> > On Sun, 2006-10-01 at 20:07 +0100, Al Viro wrote:
> > > On Sun, Oct 01, 2006 at 12:03:10PM -0700, Daniel Walker wrote:
> > > > > And that's better than the current situation in which respects, exactly?
> > > > 
> > > > Seeing the warnings is the current situation.
> > > 
> > > I bow to your incredible power of observation.  Now that you've shared
> > > that revelation with the list, could you explain what does blanket silencing
> > > of these warnings buy you, oh wan^H^Hise one?
> > 
> > Did you see me silencing anything (with your crystal ball?) ? Cause I'm
> > not.
> 
> And what, in your opinion, does the patch in question achieve if not
> that?

You mean the patch from Steven posted in May? Well it does appear to
silence the warning. You didn't like the approach it seems? Please tell
me why .

Daniel

Re: Announce: gcc bogus warning repository

[Al Viro]

On Sun, Oct 01, 2006 at 12:25:50PM -0700, Daniel Walker wrote:
> > > > I bow to your incredible power of observation.  Now that you've shared
> > > > that revelation with the list, could you explain what does blanket silencing
> > > > of these warnings buy you, oh wan^H^Hise one?
> > > 
> > > Did you see me silencing anything (with your crystal ball?) ? Cause I'm
> > > not.
> > 
> > And what, in your opinion, does the patch in question achieve if not
> > that?
> 
> You mean the patch from Steven posted in May? Well it does appear to
> silence the warning. You didn't like the approach it seems? Please tell
> me why .

Read the list archives...

Re: Announce: gcc bogus warning repository

[Andrew Morton]

On Sun, 1 Oct 2006 20:33:30 +0100
Al Viro <viro@ftp.linux.org.uk> wrote:

> On Sun, Oct 01, 2006 at 12:25:50PM -0700, Daniel Walker wrote:
> > > > > I bow to your incredible power of observation.  Now that you've shared
> > > > > that revelation with the list, could you explain what does blanket silencing
> > > > > of these warnings buy you, oh wan^H^Hise one?
> > > > 
> > > > Did you see me silencing anything (with your crystal ball?) ? Cause I'm
> > > > not.
> > > 
> > > And what, in your opinion, does the patch in question achieve if not
> > > that?
> > 
> > You mean the patch from Steven posted in May? Well it does appear to
> > silence the warning. You didn't like the approach it seems? Please tell
> > me why .
> 
> Read the list archives...

There isn't much point in doing that if they're as useless as this thread.

Fact is, those bogus warnings are harmful and cause real problems to not be
noticed.  There is value in finding some way of making real warnings
apparent to all developers and testers.

An external post-processor isn't very useful because few people will run
it.  If we can integrate such a thing into the build system and make it
available to all developers and testers then OK.

Re: Announce: gcc bogus warning repository

[Roland Dreier]

    Andrew> The downsides are that it muckies up the source a little
    Andrew> and introduces a very small risk that real
    Andrew> use-uninitialised bugs will be hidden.  But I believe the
    Andrew> benefit outweighs those disadvantages.

Not sure I agree -- it adds one more thing that must be maintained
when reorganizing code.  I think there is a pretty high risk of this
sort of warning silencing hiding a bug introduced later, which would
have triggered an "is used uninitialized" warning.

Perhaps asking for a gcc flag that turns off "may be used" warnings
but leaves "is used" warnings would be useful (or does it already exist?)

Re: Announce: gcc bogus warning repository

[linux-os (Dick Johnson)]

On Sun, 1 Oct 2006, Daniel Walker wrote:

> On Sun, 2006-10-01 at 09:44 -0400, Jeff Garzik wrote:
>> The level of warnings in a kernel build has lately increased to the
>> point where it is hiding bugs and otherwise making life difficult.
>>
>> In particular, recent gcc versions throw warnings when it thinks a
>> variable "MAY be used uninitialized", which is not terribly helpful due
>> to the fact that most of these warnings are bogus.
>>
>> For those that may find this valuable, I have started a git repo that
>> silences these bogus warnings, after careful auditing of code paths to
>> ensure that the warning truly is bogus.
>>
>> The results may be found in the "gccbug" branch of
>> git://git.kernel.org/pub/scm/linux/kernel/git/jgarzik/misc-2.6.git
>>
>
> Steven Rostedt an I worked on this problem in May. Steven came up with,
> a nice way to handle these warnings, which doesn't increase code size.
> Here's the post if your interested.
>
> http://lkml.org/lkml/2006/5/11/50
>
> Daniel

I wouldn't hide this in a macro, though. This could cause future
maintainers a lot of problems after code is revised. If you are
going to assign a variable's value to itself to eliminate compiler
warnings, please do it openly, preferably with a comment.

Cheers,
Dick Johnson
Penguin : Linux version 2.6.16.24 on an i686 machine (5592.72 BogoMips).
New book: http://www.AbominableFirebug.com/
_
\x1a\x04

****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

Re: Announce: gcc bogus warning repository

[Randy Dunlap]

On Sun, 01 Oct 2006 09:44:55 -0400 Jeff Garzik wrote:

> 
> The level of warnings in a kernel build has lately increased to the 
> point where it is hiding bugs and otherwise making life difficult.
> 
> In particular, recent gcc versions throw warnings when it thinks a 
> variable "MAY be used uninitialized", which is not terribly helpful due 
> to the fact that most of these warnings are bogus.
> 
> For those that may find this valuable, I have started a git repo that 
> silences these bogus warnings, after careful auditing of code paths to 
> ensure that the warning truly is bogus.
> 
> The results may be found in the "gccbug" branch of
> git://git.kernel.org/pub/scm/linux/kernel/git/jgarzik/misc-2.6.git
> 
> This repository will NEVER EVER be pushed upstream.  It exists solely 
> for those who want to decrease their build noise, thereby exposing true 
> bugs.
> 
> The audit has already uncovered several minor bugs, lending credence to 
> my theory that too many warnings hides bugs.

I usually build with must_check etc. enabled then grep them
away if I want to look for other messages.  I think that the situation
is not so disastrous.

---
~Randy

Re: Announce: gcc bogus warning repository

[Jeff Garzik]

Randy Dunlap wrote:
> On Sun, 01 Oct 2006 09:44:55 -0400 Jeff Garzik wrote:
> 
>> The level of warnings in a kernel build has lately increased to the 
>> point where it is hiding bugs and otherwise making life difficult.
>>
>> In particular, recent gcc versions throw warnings when it thinks a 
>> variable "MAY be used uninitialized", which is not terribly helpful due 
>> to the fact that most of these warnings are bogus.
>>
>> For those that may find this valuable, I have started a git repo that 
>> silences these bogus warnings, after careful auditing of code paths to 
>> ensure that the warning truly is bogus.
>>
>> The results may be found in the "gccbug" branch of
>> git://git.kernel.org/pub/scm/linux/kernel/git/jgarzik/misc-2.6.git
>>
>> This repository will NEVER EVER be pushed upstream.  It exists solely 
>> for those who want to decrease their build noise, thereby exposing true 
>> bugs.
>>
>> The audit has already uncovered several minor bugs, lending credence to 
>> my theory that too many warnings hides bugs.
> 
> I usually build with must_check etc. enabled then grep them
> away if I want to look for other messages.  I think that the situation
> is not so disastrous.

I think it's both sad, and telling, that the high level of build noise 
has trained kernel hackers to tune out warnings, and/or build tools of 
ever-increasing sophistication just to pick out the useful messages from 
all the noise.

If you have to grep useful stuff out of the noise, you've already lost.

	Jeff

Re: Announce: gcc bogus warning repository

[Alistair John Strachan]

On Sunday 01 October 2006 18:20, Jeff Garzik wrote:
> Randy Dunlap wrote:
[snip]
> >> This repository will NEVER EVER be pushed upstream.  It exists solely
> >> for those who want to decrease their build noise, thereby exposing true
> >> bugs.
> >>
> >> The audit has already uncovered several minor bugs, lending credence to
> >> my theory that too many warnings hides bugs.
> >
> > I usually build with must_check etc. enabled then grep them
> > away if I want to look for other messages.  I think that the situation
> > is not so disastrous.
>
> I think it's both sad, and telling, that the high level of build noise
> has trained kernel hackers to tune out warnings, and/or build tools of
> ever-increasing sophistication just to pick out the useful messages from
> all the noise.
>
> If you have to grep useful stuff out of the noise, you've already lost.

The question is whether the GCC guys are actually doing anything about the 
problem. If they are, we should do nothing. If they aren't, maybe it's time 
for "x = x" hacks like Steven's.

Is GCC 4.2 any better with this class of warnings?

-- 
Cheers,
Alistair.

Final year Computer Science undergraduate.
1F2 55 South Clerk Street, Edinburgh, UK.

Re: Announce: gcc bogus warning repository

[Adrian Bunk]

On Sun, Oct 01, 2006 at 06:27:29PM +0100, Alistair John Strachan wrote:
> On Sunday 01 October 2006 18:20, Jeff Garzik wrote:
> > Randy Dunlap wrote:
> [snip]
> > >> This repository will NEVER EVER be pushed upstream.  It exists solely
> > >> for those who want to decrease their build noise, thereby exposing true
> > >> bugs.
> > >>
> > >> The audit has already uncovered several minor bugs, lending credence to
> > >> my theory that too many warnings hides bugs.
> > >
> > > I usually build with must_check etc. enabled then grep them
> > > away if I want to look for other messages.  I think that the situation
> > > is not so disastrous.
> >
> > I think it's both sad, and telling, that the high level of build noise
> > has trained kernel hackers to tune out warnings, and/or build tools of
> > ever-increasing sophistication just to pick out the useful messages from
> > all the noise.
> >
> > If you have to grep useful stuff out of the noise, you've already lost.
> 
> The question is whether the GCC guys are actually doing anything about the 
> problem. If they are, we should do nothing. If they aren't, maybe it's time 
> for "x = x" hacks like Steven's.
>...

Let's be fair to gcc:

gcc correctly tells it "may be used uninitialized" - that's different 
from cases where gcc tells "is used uninitialized".

Sometimes, it requires _much_ context seeing that a condition is 
actually not possible.

And there are even cases where it's technically impossible for a 
compiler to figure out itself that a condition is not possible.

> Cheers,
> Alistair.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: Announce: gcc bogus warning repository

[Randy Dunlap]

On Sun, 01 Oct 2006 13:20:45 -0400 Jeff Garzik wrote:

> Randy Dunlap wrote:
> > On Sun, 01 Oct 2006 09:44:55 -0400 Jeff Garzik wrote:
> > 
> >> The level of warnings in a kernel build has lately increased to the 
> >> point where it is hiding bugs and otherwise making life difficult.
> >>
> >> In particular, recent gcc versions throw warnings when it thinks a 
> >> variable "MAY be used uninitialized", which is not terribly helpful due 
> >> to the fact that most of these warnings are bogus.
> >>
> >> For those that may find this valuable, I have started a git repo that 
> >> silences these bogus warnings, after careful auditing of code paths to 
> >> ensure that the warning truly is bogus.
> >>
> >> The results may be found in the "gccbug" branch of
> >> git://git.kernel.org/pub/scm/linux/kernel/git/jgarzik/misc-2.6.git
> >>
> >> This repository will NEVER EVER be pushed upstream.  It exists solely 
> >> for those who want to decrease their build noise, thereby exposing true 
> >> bugs.
> >>
> >> The audit has already uncovered several minor bugs, lending credence to 
> >> my theory that too many warnings hides bugs.
> > 
> > I usually build with must_check etc. enabled then grep them
> > away if I want to look for other messages.  I think that the situation
> > is not so disastrous.
> 
> I think it's both sad, and telling, that the high level of build noise 
> has trained kernel hackers to tune out warnings, and/or build tools of 
> ever-increasing sophistication just to pick out the useful messages from 
> all the noise.
> 
> If you have to grep useful stuff out of the noise, you've already lost.

I often build with C=1 (sparse checking), so the amount of output
has to be grepped IMO.  It's certainly too much to read otherwise.
We just have different perspectives, I guess.

---
~Randy

Re: Announce: gcc bogus warning repository

[Jörn Engel]

On Sun, 1 October 2006 13:20:45 -0400, Jeff Garzik wrote:
> 
> If you have to grep useful stuff out of the noise, you've already lost.

My initial thought was to agree.  And I still almost agree with this
statement.

Almost, because every so often, it makes sense to see a shipload of
warnings and manually weed through them all.  99% will be false
positives, but there is the remaining 1% indicating real bugs.

Of course, once these have been dealt with, the warnings give 100%
noise and 0% useful output.  It is time to wait another year or four
before enough bugs have accumulated to make it worth the effort again.

That said, if you need to grep every day, you've definitely lost.

Jörn

-- 
Write programs that do one thing and do it well. Write programs to work
together. Write programs to handle text streams, because that is a
universal interface.
-- Doug MacIlroy