Re: linux-next: origin tree build failure

Benjamin Herrenschmidt
2009-06-12T13:57:18Z

On Fri, 2009-06-12 at 15:44 +0200, Ingo Molnar wrote:

> This is certainly doable for agreeable features - which is the bulk
> - and it is being done.
>
> But this is a catch-22 for _controversial_ new features - which
> perfcounters clearly was, in case you turned off your lkml
> subscription ;-)

I didn't :-) My point here is that Linus can make a decision with an
email -before- merging so that -next gets a chance, at least for a
couple of days, to do the integration testing once the controversy has
been sorted by his highness.

> And if you hit that build breakage during bisection you can do:
>
> git cherry-pick e14112d

Right, I can, you can, but can random tester who wants to track down
what his problem is ? I'm not sure...

> Also, you seem to brush off the notion that far more bugs slip
> through linux-next than get caught by it.

Less than without linux-next. We aren't perfect and no process will
solve everything. But this could have been easily avoided.

> So if you think linux-next matters in terms of _regression_ testing,
> the numbers dont seem to support that notion. This particular
> incident does support that notion though, granted - but it's taken
> out of context IMHO:
>
> In terms of test coverage, at least for our trees, less than 1% of
> the bugs we handle get reported in a linux-next context - and most
> of the bugs that get reported (against say the scheduler tree) are
> related to rare architectures.

But most obvious bugs will have been caught way before that, which
leaves the hard to catch ones or the configuration-specific ones. Those
will pass linux-next, I agree. But that isn't my point and that isn't
what linux-next will catch. What is will catch is that kind of really
simple mechanical problems, such as build breakage for other archs.

If perfcounters had been 1 or 2 days in -next before being merged, we
would have avoided that problem and made everybody's bisecting life
easier.

> In fact, i checked, there were _zero_ x86 bugs reported against
> linux-next and solved against it between v2.6.30-rc1 and v2.6.30:

No but Stephen caught a bunch of mechanical compile fails due to
integration problems.

> git log --grep=next -i v2.6.30-rc1..v2.6.30 arch/x86/
>
> Doing it over the full cycle shows one commit altogether - a Xen
> build failure. In fact, i just checked the whole stabilization cycle
> for the whole kernel (v2.6.30-rc1..v2.6.30-final), and there were
> only 5 linux-next originated patches, most of them build failures.
>
> I did this by looking at all occurances of 'next', in all commit
> logs:
>
> git log --grep=next -i v2.6.30-rc1..v2.6.30
>
> and then manually checking the context of all 'next' matches and
> counting the linux-next related commits.
>
> So lets be generous and say that because some people dont put the
> bug report originator into the changelog it was four times as many,
> 20 - but that's still dwarved by the sheer amount of post-rc1
> changes: thousands of changes and hundreds of regressions.
>
> linux-next is mostly useful (to me at least) not for the
> cross-builds it does, but in terms of mapping out upcoming conflicts
> - which also drives early detection of problematic patches and
> problematic conflicts.

Yes, it does. The problem is that it helps -you- that way, but won't
help -us- vs. that kind of mechanical problems unless -you- also play
the game and get your stuff in there for a little while before merging
it :-)

Cheers,
Ben.