Re: mistakes in code vs. maintainer flow mistakes (was: [ 00/19] 3.10.1-stable review)

Ingo Molnar
2013-07-19T10:03:42Z

* Ingo Molnar <mingo@kernel.org> wrote:

> [...]
>
> Mistakes in patches and code happen all the time. Linus rarely if ever
> flamed me for _that_ - sh*t happens.
>
> What he flames me for, and what you (with all due respect) still don't
> seem to understand, are _META_ mistakes. Top level maintainer level
> mistakes. Bad patterns of maintainer behavior that really should not
> occur because they could affect many patches in the future, such as:
>
> - trying to argue regressions away - i.e. not 'shutting up' in time,
> being a meta hindrance to problem resolution
>
> - doing a sloppy Git flow, repeatedly
>
> - not testing adequately, especially when the pull request occurs at a
> critical time (such as a couple of hours before -rc1)
>
> - [ and many other meta mistakes ]
>
> None of those arguments are about code and still I fully expect Linus to
> pin those on me if he notices a meta bug in my behavior and finds it
> dangerous.

And note that whenever I or a fellow -tip maintainer got such an unhappy
complaint from Linus in the past couple of years our response wasn't just
to fix some broken code.

Our response was to fix broken top level maintainer behavior, by applying
'meta fixes':

- changing our Git workflow

- adding more scripting to catch bad commits

- changing our flow of sending pull requests, adding fail-safes

- trying to think more neutrally about bug reports to avoid punishing
the messenger and to avoid arguing regressions away

- hardening our review process

- making sure at least one -tip maintainer watches lkml for bugreports

- tightening our controls to avoid missed patches

- thinking about the timing of pull requests

- etc., etc.

(And there's an even larger body of 'meta fixes' we applied without being
prodded by Linus.)

On the outside such incidents look like as if Linus flamed 'the person' in
a disrespectful way.

What Linus _really_ flamed us for in 95% of the cases was the meta
process, the 'meta code' of Linux, which is not actual source code but
mostly a social construct, informal patterns of human behavior - and those
are inextricably embedded in the person.

And because the 'meta fixes' too are often of social nature, what you see
when reading lkml is just a unidirectional stream of complaints from
Linus. You typically don't see patch notifications of changed behavior.
Nor do you see top level maintainers 'speaking up against Linus' very
often: these are bugreports from Linus and we simply fix them, there's not
much to speak up against.

Linus is very laissez-faire about maintainence, so whenever he _does_
erupt at us (at a clip of ~10,000 commits per cycle that do go in without
any complaint from Linus) it's justified in a large percentage of cases.

So despite the outside appearance this is not top level Linux maintainers
being oppressed by Linus or suffering from some sort of Stockholm Syndrome
:-)

We are just as stubborn as Linus and do speak up against Linus when needed
- it just rarely is necessary - in great part because Linus flames in
public and takes care he is upset for a good reason so he does not have to
walk back on his flame. Public embarrassment cuts both ways.

When Linus's complaint is unjustified top level maintainers _do_ speak up
- see Thomas Gleixner's recent example, which resulted in Linus
apologizing. (It's a rare occurance and we've archived all the emails for
the history books.)

Thanks,

Ingo