Re: [RFC] Results of Phase 1 of the Review Process study

[Ian Campbell <ian.campbell@citrix.com>]

On Fri, 2015-10-16 at 00:32 +0200, Jesus M. Gonzalez-Barahona wrote:
> On Thu, 2015-10-15 at 22:36 +0100, Lars Kurth wrote:
> > > On 15 Oct 2015, at 10:26, Ian Campbell <Ian.Campbell@citrix.com>
> > > wrote:
> > > 
> > > On Thu, 2015-10-15 at 10:06 +0100, Ian Campbell wrote:
> 
> > > > On Wed, 2015-10-14 at 18:32 +0100, Lars Kurth wrote:
> > [...]
> 
> > That is correct and a case we need to look at if it's worthwhile and
> > possible to fix it. However the fact remains that pretty much all the
> > graphs cover data for completed reviews only, except for backlog
> > data. We have two options
> > 1) Spend some effort trying to fix it
> > 2) Accept that "stalled" reviews are not that meaningful
> > Not sure what the answer is at this stage
> 
> I guess there are three possible solutions here:
> 
> * To send some message to the mailing list to "abandon" or close a
> review. This would allow us to detect those reviews, and everyone to
> know that they are not really stalled. But this would require changes
> to your policy, I assume.

I don't think this one is practical, since things are often implicitly
abandoned by e.g. people changing jobs or changing their focus etc and
under such circumstances they are unlikely to take any explicit action.

> * To consider that when there is no activity for a certain period, the
> review is no longer going to progress, and can be considered abandoned.
> The main trouble with this could be that we have seen some patch series
> inactive for very long periods, and still coming back to life after
> that. But being a very small fraction of the cases, for statistical
> purposes those could be considered as abandoned.

Yes, and I suppose based on data mining you can see the longest delay
before "coming back to life", which would then feed into the selection of
the certain period.

> * To label "by hand" the reviews that are abandoned, by some of you
> knowing about the project. But this is (I assume) too time-consuming
> and probably error-prone....

As I just said in another reply, it would be interesting to know what the
incoming rate of these were, there's a chance it might not be too bad once
some simple matching heuristics have been applied to weed out the simple
tweaks to the commit subject.

Ian.