Re: [RFC] Results of Phase 1 of the Review Process study

[Ian Campbell <ian.campbell@citrix.com>]

On Wed, 2015-10-14 at 18:32 +0100, Lars Kurth wrote:
> C1) Only 60% percent of the reviews on the mailing list could be matched
> to commits. This can be improved going forward, but we felt that the
> dataset is big enough for statical analysis and didn't want to spend too
> much time to get the matching perfect at this stage. See "Coverage
> analysis" for more details

How strict or fuzzy is the matching?

Does it account for e.g. spelling, grammar and clarity changes and things
like adding a subsystem ("tools: libxc:") prefix, either upon commit or by
the author in vN+1 based on feedback?

I often both comment on such things during review and (with the authors
permission) tweak things upon commit.

If those changes are not being correlated then I expect that would skew the
figures of those for whom English is not their first language (and not a
small portion of native speakers even!) and newcomers who e.g. might not be
aware of the need to prefix things with the subsystem.

In a (smaller) number of cases a patch is abandoned in favour of a very
different approach, which I think would be essentially untrackable, at
least automatically.

> 
[...]
> The log scale diagrams are more interesting (See LogScale.png for a 
> legend).

The width of the box in these diagrams has no meaning, is that correct?

> 
[...]
> == Backlog Analysis ==
> This section shows us the total of patch series reviews that could be
> modelled (60%) over the project's life-time 

How does this interact with the 60% in caveat C1? Is it the same 60% or is
this 60% of that 60% (i.e. 36% overall)?

If it is the same 60% then how are stalled series distinguished from the
40% which are not mapped to a commit?

Separately, I suppose it is impossible to distinguish stalled from
abandoned (and perhaps in some senses they are the same thing so we don't
need to distinguish).

Ian.