Tracking regressions for next release(s)

Tracking regressions for next release(s)

[Maciej Rutecki]

Hi

For a long time (dealing with colleagues) we tracking regressions in
stable RC versions of the kernel. Because 3.3 is out, and it is time
thinking about tracking regressions for
next release. I believe that we need to do some summary and decide on
further action.

I am interested in the opinion of the developers, testers, and
everyone involved in the development of the kernel, if they thing that
tracking regressions and monitoring the quality makes sense,
especially since I met several times (put it mildly) dislike of such
work and the bugs are repaired relatively slowly. Perhaps someone has
comments or proposals for change (in the way of work or me).


I care about the opinion, because without the acceptance of the
community work does not make sense. I also do not insist, because I'll
have more time for other tasks such as testing.

Regards
-- 
Maciej Rutecki
http://www.mrutecki.pl

Re: Tracking regressions for next release(s)

[Borislav Petkov]

On Fri, Mar 23, 2012 at 07:01:19AM +0100, Maciej Rutecki wrote:
> I am interested in the opinion of the developers, testers, and
> everyone involved in the development of the kernel, if they thing that
> tracking regressions and monitoring the quality makes sense,

Absolutely.

> especially since I met several times (put it mildly) dislike of such
> work and the bugs are repaired relatively slowly.

I can imagine people getting cranky when someone points out that there's
a "boring" bug they need to fix instead of them working on the cool new
feature they have thought of. It is the same old story we've been having
since forever: people don't really love to fix bugs, especially if the
code works for them and the bug doesn't appear on their boxes.

> Perhaps someone has comments or proposals for change (in the way of
> work or me).

Yeah, we need a big bad assh*le :) who screams at everyone until their
bugs is fixed.

But serioulsy, this hasn't changed: we definitely need a regression
list, I think it works even better when Linus goes over it and says
this is fixed, that is this commit, etc. because he pulls all the trees
in the end, ... so yeah, I think what you guys are doing is good and
important.

It would be even cooler if this list be expanded also to regressions in
kernel performance which people have noticed from running benchmarks on
different -rcs and have noticed differences there, maybe a website (not
bugzilla) which lists all those regressions for interested parties to
fix in addition to the LKML mails..., etc...

Thanks for your hard work, btw.

-- 
Regards/Gruss,
    Boris.

Re: Tracking regressions for next release(s)

[Maciej Rutecki]

Hi,

On piątek, 23 marca 2012 o 11:14:31 Borislav Petkov wrote:
> On Fri, Mar 23, 2012 at 07:01:19AM +0100, Maciej Rutecki wrote:
> > I am interested in the opinion of the developers, testers, and
> > everyone involved in the development of the kernel, if they thing that
> > tracking regressions and monitoring the quality makes sense,
> 
> Absolutely.
> 
> > especially since I met several times (put it mildly) dislike of such
> > work and the bugs are repaired relatively slowly.
> 
> I can imagine people getting cranky when someone points out that there's
> a "boring" bug they need to fix instead of them working on the cool new
> feature they have thought of. It is the same old story we've been having
> since forever: people don't really love to fix bugs, especially if the
> code works for them and the bug doesn't appear on their boxes.
> 
> > Perhaps someone has comments or proposals for change (in the way of
> > work or me).
> 
> Yeah, we need a big bad assh*le :) who screams at everyone until their
> bugs is fixed.
> 
> But serioulsy, this hasn't changed: we definitely need a regression
> list, I think it works even better when Linus goes over it and says
> this is fixed, that is this commit, etc. because he pulls all the trees
> in the end, ... so yeah, I think what you guys are doing is good and
> important.
> 
> It would be even cooler if this list be expanded also to regressions in
> kernel performance which people have noticed from running benchmarks on
> different -rcs and have noticed differences there, maybe a website (not
> bugzilla) which lists all those regressions for interested parties to
> fix in addition to the LKML mails..., etc...
> 
> Thanks for your hard work, btw.


Borislav,  Bjorn Helgaas: thank you for the answer, but observing the 
reactions I get the impression that tracking the regression is not likely 
anyone's interest. In addition - especially on the last release cycle - 
sometimes encountered difficulties in cooperation on this topic with developers: 
ignoring request to update the status of the regression, or even add your e-
mail to bugzilla.

I give up tracking the regression, but not the kernel testing. Even now I have 
a few hours per week more for it.

Regards
-- 
Maciej Rutecki
http://www.mrutecki.pl

Re: Tracking regressions for next release(s)

[Borislav Petkov]

On Mon, Mar 26, 2012 at 06:47:17PM +0200, Maciej Rutecki wrote:
> Hi,
> 
> On piątek, 23 marca 2012 o 11:14:31 Borislav Petkov wrote:
> > On Fri, Mar 23, 2012 at 07:01:19AM +0100, Maciej Rutecki wrote:
> > > I am interested in the opinion of the developers, testers, and
> > > everyone involved in the development of the kernel, if they thing that
> > > tracking regressions and monitoring the quality makes sense,
> > 
> > Absolutely.
> > 
> > > especially since I met several times (put it mildly) dislike of such
> > > work and the bugs are repaired relatively slowly.
> > 
> > I can imagine people getting cranky when someone points out that there's
> > a "boring" bug they need to fix instead of them working on the cool new
> > feature they have thought of. It is the same old story we've been having
> > since forever: people don't really love to fix bugs, especially if the
> > code works for them and the bug doesn't appear on their boxes.
> > 
> > > Perhaps someone has comments or proposals for change (in the way of
> > > work or me).
> > 
> > Yeah, we need a big bad assh*le :) who screams at everyone until their
> > bugs is fixed.
> > 
> > But serioulsy, this hasn't changed: we definitely need a regression
> > list, I think it works even better when Linus goes over it and says
> > this is fixed, that is this commit, etc. because he pulls all the trees
> > in the end, ... so yeah, I think what you guys are doing is good and
> > important.
> > 
> > It would be even cooler if this list be expanded also to regressions in
> > kernel performance which people have noticed from running benchmarks on
> > different -rcs and have noticed differences there, maybe a website (not
> > bugzilla) which lists all those regressions for interested parties to
> > fix in addition to the LKML mails..., etc...
> > 
> > Thanks for your hard work, btw.
> 
> 
> Borislav,  Bjorn Helgaas: thank you for the answer, but observing the 
> reactions I get the impression that tracking the regression is not likely 
> anyone's interest. In addition - especially on the last release cycle - 
> sometimes encountered difficulties in cooperation on this topic with developers: 
> ignoring request to update the status of the regression, or even add your e-
> mail to bugzilla.

Well, sounds like you've already decided and that's just sad :(. Let's
add some more people to Cc, see what they think.

Guys, thread starts here: http://lkml.org/lkml/2012/3/23/31

> I give up tracking the regression, but not the kernel testing. Even now I have 
> a few hours per week more for it.

Well, if you still can report your results from it, I think maintainers
who are still interested in the quality of their code will be interested
in your testing reports.

Thanks again for your work.

-- 
Regards/Gruss,
    Boris.

Re: Tracking regressions for next release(s)

[H. Peter Anvin]

On 03/26/2012 11:58 AM, Borislav Petkov wrote:
> 
> Well, if you still can report your results from it, I think maintainers
> who are still interested in the quality of their code will be interested
> in your testing reports.
> 
> Thanks again for your work.
> 

I agree; it is very easy to lose track of regressions and it is really
useful to be reminded of them.  That doesn't mean, however, that all
bugzilla reports are useful or valid, but that doesn't mean the overall
effort isn't useful.

	-hpa

Re: Tracking regressions for next release(s)

[Bjorn Helgaas]

On Fri, Mar 23, 2012 at 12:01 AM, Maciej Rutecki
<maciej.rutecki@gmail.com> wrote:
> Hi
>
> For a long time (dealing with colleagues) we tracking regressions in
> stable RC versions of the kernel. Because 3.3 is out, and it is time
> thinking about tracking regressions for
> next release. I believe that we need to do some summary and decide on
> further action.
>
> I am interested in the opinion of the developers, testers, and
> everyone involved in the development of the kernel, if they thing that
> tracking regressions and monitoring the quality makes sense,
> especially since I met several times (put it mildly) dislike of such
> work and the bugs are repaired relatively slowly. Perhaps someone has
> comments or proposals for change (in the way of work or me).

I personally find the regression tracking very valuable.  Regressions
are worse than other bugs because they make users (at least this one
:)) hesitant to move to newer kernels, and then we miss out on the
testing we depend on.

Your tracking work helps everybody pay attention to these issues.  I
don't understand opposition to it.

Bjorn