Re: [RFC] Results of Phase 1 of the Review Process study

[Ian Campbell <ian.campbell@citrix.com>]

On Thu, 2015-10-15 at 10:06 +0100, Ian Campbell wrote:
> On Wed, 2015-10-14 at 18:32 +0100, Lars Kurth wrote:
> > C1) Only 60% percent of the reviews on the mailing list could be
> > matched
> > to commits. This can be improved going forward, but we felt that the
> > dataset is big enough for statical analysis and didn't want to spend
> > too
> > much time to get the matching perfect at this stage. See "Coverage
> > analysis" for more details
> 
> How strict or fuzzy is the matching?
> 
> Does it account for e.g. spelling, grammar and clarity changes and things
> like adding a subsystem ("tools: libxc:") prefix, either upon commit or
> by
> the author in vN+1 based on feedback?
> 
> I often both comment on such things during review and (with the authors
> permission) tweak things upon commit.
> 
> If those changes are not being correlated then I expect that would skew
> the
> figures of those for whom English is not their first language (and not a
> small portion of native speakers even!) and newcomers who e.g. might not
> be
> aware of the need to prefix things with the subsystem.
> 
> In a (smaller) number of cases a patch is abandoned in favour of a very
> different approach, which I think would be essentially untrackable, at
> least automatically.

Looking at the stuff in [47] marked as last reviewed in 2014 it seems the
majority of them (at least the ones for which I am involved as a maintainer
etc) can be explained by one of these factors, just going from my memory of
things having been fixed in one way or another.

There also looks to be identical titles (e.g. "x86: Full support of PAT")
being listed there more than once.

Lastly there is at least one ("Introducing Xen PV block driver to OVMF")
which was against another project.

The same seems to be true of e.g. [49] too.

I'm afraid that on that basis I think C1 has skewed the conclusion that
there are 600 stalled series, possibly by a considerable factor.

Ian.

[...]
> [...]
> > == Backlog Analysis ==
> > This section shows us the total of patch series reviews that could be
> > modelled (60%) over the project's life-time 
> 
> How does this interact with the 60% in caveat C1? Is it the same 60% or
> is
> this 60% of that 60% (i.e. 36% overall)?
> 
> If it is the same 60% then how are stalled series distinguished from the
> 40% which are not mapped to a commit?
> 
> Separately, I suppose it is impossible to distinguish stalled from
> abandoned (and perhaps in some senses they are the same thing so we don't
> need to distinguish).
> 
> Ian.
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel