What's the staging review and acceptance process?

What's the staging review and acceptance process?

[Joe Perches]

Hi Greg.

Sice your fosdem talk and tuxradar article came out there
have been many patches against staging by new contributors
(I did some too) that seem to get no feedback or get dropped
without comment.  Because many of these patches are first
attempts, probably most of them should not be applied in the
first submitted form.  I comment on some every once in awhile,
but the number of patches appear to go unnoticed or untracked
seems quite high.

Do you have enough bandwidth to keep these new contributors
engaged?  Anything I can do to help?

What is your current review/notify/accept/reject workflow?

Perhaps a patchwork queue for staging might help track these
patches and with more feedback or reviewers, get them in
shape to be applied.

Re: What's the staging review and acceptance process?

[Greg KH]

On Wed, Apr 21, 2010 at 01:32:47PM -0700, Joe Perches wrote:
> Hi Greg.
> 
> Sice your fosdem talk and tuxradar article came out there
> have been many patches against staging by new contributors
> (I did some too) that seem to get no feedback or get dropped
> without comment.  Because many of these patches are first
> attempts, probably most of them should not be applied in the
> first submitted form.  I comment on some every once in awhile,
> but the number of patches appear to go unnoticed or untracked
> seems quite high.

I have around 600 still left to go through.  It was an unfortunate set
of timing, the talk, article, and then I up and moved a few hundred
miles and decided to attend a few conferences, as well as maintaining 4
stable trees at the same time.  That caused the huge backlog staring at
me right now.

> Do you have enough bandwidth to keep these new contributors
> engaged?  Anything I can do to help?

Your reviews like you have been doing, when you see something obviously
wrong, is great.  I just went through 100 patches, and only 40 of them
were "valid" and able to be applied, so it is a high rejection rate
which requires a lot of attention to be paid to them.

> What is your current review/notify/accept/reject workflow?

Like it's always been:
  - patch comes in
  - I get around to reviewing it
  - if valid, I apply and you get an email
  - if invalid, I reject and say why in email

> Perhaps a patchwork queue for staging might help track these
> patches and with more feedback or reviewers, get them in
> shape to be applied.

patchwork doesn't work well for my patch flow, but maybe that is because
I haven't spent enough time with it.  Right now I have all the patches,
it's just a matter of getting through them.

thanks,

greg k-h

Re: What's the staging review and acceptance process?

[Joe Perches]

On Wed, 2010-04-21 at 20:45 -0700, Greg KH wrote:
> That caused the huge backlog staring at
> me right now.

Which likely discouraged the new contributors who
submitted stuff still in that backlog.

> I just went through 100 patches, and only 40 of them
> were "valid" and able to be applied, so it is a high rejection rate
> which requires a lot of attention to be paid to them.
> 
> > What is your current review/notify/accept/reject workflow?
> 
> Like it's always been:
>   - patch comes in
>   - I get around to reviewing it
>   - if valid, I apply and you get an email
>   - if invalid, I reject and say why in email

Unfortunately, your process has been opaque until you
personally handle it.

Does the driverdev list or any list always receive a copy
or the feedback?

> > Perhaps a patchwork queue for staging might help track these
> > patches and with more feedback or reviewers, get them in
> > shape to be applied.

> patchwork doesn't work well for my patch flow, but maybe that is because
> I haven't spent enough time with it.  Right now I have all the patches,
> it's just a matter of getting through them.

Maybe you should get some advice on using patchwork use
from somebody like David Miller, who gets rather more
patches for networking than staging gets.  The patchwork
queue for networking always seems managed and it can use
delegation, which your process doesn't seem capable of
doing.  You're a voluntary bottleneck for staging, I
think you either need to find personal cycles or find
some other suckers willing to volunteer who'd make up
more overall cycles.

cheers, Joe

Re: What's the staging review and acceptance process?

[Greg KH]

On Wed, Apr 21, 2010 at 09:25:57PM -0700, Joe Perches wrote:
> On Wed, 2010-04-21 at 20:45 -0700, Greg KH wrote:
> > That caused the huge backlog staring at
> > me right now.
> 
> Which likely discouraged the new contributors who
> submitted stuff still in that backlog.

While a series of unfortunate events did happen to cause this, do you
have any evidence of this causing people to go away?  I have had a
number of queries about pending patches, all of which I quickly
responded to (travel being easy to write emails, just not do "real"
work).

> > I just went through 100 patches, and only 40 of them
> > were "valid" and able to be applied, so it is a high rejection rate
> > which requires a lot of attention to be paid to them.
> > 
> > > What is your current review/notify/accept/reject workflow?
> > 
> > Like it's always been:
> >   - patch comes in
> >   - I get around to reviewing it
> >   - if valid, I apply and you get an email
> >   - if invalid, I reject and say why in email
> 
> Unfortunately, your process has been opaque until you
> personally handle it.

Yes, like all subsystem maintainers, right?

> Does the driverdev list or any list always receive a copy
> or the feedback?

If it was copied, yes, it does.  I can't control who the original
submitters copy on their emails, but they usually do hit the driverdev
mailing list for the most part.

> > > Perhaps a patchwork queue for staging might help track these
> > > patches and with more feedback or reviewers, get them in
> > > shape to be applied.
> 
> > patchwork doesn't work well for my patch flow, but maybe that is because
> > I haven't spent enough time with it.  Right now I have all the patches,
> > it's just a matter of getting through them.
> 
> Maybe you should get some advice on using patchwork use
> from somebody like David Miller, who gets rather more
> patches for networking than staging gets.  The patchwork
> queue for networking always seems managed and it can use
> delegation, which your process doesn't seem capable of
> doing.  You're a voluntary bottleneck for staging, I
> think you either need to find personal cycles or find
> some other suckers willing to volunteer who'd make up
> more overall cycles.

Are you volunteering?

thanks,

greg k-h

Re: What's the staging review and acceptance process?

[Joe Perches]

On Wed, 2010-04-21 at 22:49 -0700, Greg KH wrote:
> On Wed, Apr 21, 2010 at 09:25:57PM -0700, Joe Perches wrote:
> > On Wed, 2010-04-21 at 20:45 -0700, Greg KH wrote:
> > > That caused the huge backlog staring at me right now.
> > Which likely discouraged the new contributors who
> > submitted stuff still in that backlog.
> do you have any evidence of this causing people to go away?

I think it's pretty established that response delays
cause losses of one type or another.

> > Unfortunately, your process has been opaque until you
> > personally handle it.
> Yes, like all subsystem maintainers, right?

Not really, patchwork shows status immediately.

> Are you volunteering?

Guess so.

Re: What's the staging review and acceptance process?

[Greg KH]

On Wed, Apr 21, 2010 at 10:55:18PM -0700, Joe Perches wrote:
> On Wed, 2010-04-21 at 22:49 -0700, Greg KH wrote:
> > On Wed, Apr 21, 2010 at 09:25:57PM -0700, Joe Perches wrote:
> > > On Wed, 2010-04-21 at 20:45 -0700, Greg KH wrote:
> > > > That caused the huge backlog staring at me right now.
> > > Which likely discouraged the new contributors who
> > > submitted stuff still in that backlog.
> > do you have any evidence of this causing people to go away?
> 
> I think it's pretty established that response delays
> cause losses of one type or another.

Established where?
What type of losses?

> > > Unfortunately, your process has been opaque until you
> > > personally handle it.
> > Yes, like all subsystem maintainers, right?
> 
> Not really, patchwork shows status immediately.

Immediately when someone does something with it, right?  So, the same as
my development cycle?

> > Are you volunteering?
> 
> Guess so.

For now, let me get through this patch queue.  If you could review
anything new coming in, that would be greatly appreciated, especially to
weed out the problem patches.  After everything is caught up, we can
revisit it, ok?

thanks,

greg k-h

Re: What's the staging review and acceptance process?

[Joe Perches]

On Wed, 2010-04-21 at 23:12 -0700, Greg KH wrote:
> On Wed, Apr 21, 2010 at 10:55:18PM -0700, Joe Perches wrote:
> > I think it's pretty established that response delays
> > cause losses of one type or another.
> Established where?
> What type of losses?

Seriously?
Google much?
Take or study economics or psychology?
Have children?

Re: What's the staging review and acceptance process?

[Jaya Kumar]

On Thu, Apr 22, 2010 at 2:28 PM, Joe Perches <joe@perches.com> wrote:
> On Wed, 2010-04-21 at 23:12 -0700, Greg KH wrote:
>> On Wed, Apr 21, 2010 at 10:55:18PM -0700, Joe Perches wrote:
>> > I think it's pretty established that response delays
>> > cause losses of one type or another.
>> Established where?
>> What type of losses?
>
> Seriously?
> Google much?
> Take or study economics or psychology?
> Have children?
>

On a lighter note, I read above as suggesting that response delays
caused one to have children. :-)

Re: What's the staging review and acceptance process?

[Olivier Galibert]

On Wed, Apr 21, 2010 at 11:12:42PM -0700, Greg KH wrote:
> On Wed, Apr 21, 2010 at 10:55:18PM -0700, Joe Perches wrote:
> > Not really, patchwork shows status immediately.
> 
> Immediately when someone does something with it, right?  So, the same as
> my development cycle?

I guess the main difference is that patchwork allows one contributor
to see that his patch has just not been checked yet, vs. missed/lost.

  OG.

Re: What's the staging review and acceptance process?

[John W. Linville]

On Thu, Apr 22, 2010 at 08:42:45AM +0200, Olivier Galibert wrote:
> On Wed, Apr 21, 2010 at 11:12:42PM -0700, Greg KH wrote:
> > On Wed, Apr 21, 2010 at 10:55:18PM -0700, Joe Perches wrote:
> > > Not really, patchwork shows status immediately.
> > 
> > Immediately when someone does something with it, right?  So, the same as
> > my development cycle?
> 
> I guess the main difference is that patchwork allows one contributor
> to see that his patch has just not been checked yet, vs. missed/lost.

In Greg's defense, I find patchwork to be fairly unwieldy (which is
why I don't use it).  It is certainly possible that I am missing some
key feature, but my limited experience with it suggested to me that all
the clicky-clicky stuff required to deal with the individual messages
queued in patchwork nearly doubled my time overhead associated with
merging patches.

John
-- 
John W. Linville		Someday the world will need a hero, and you
linville@tuxdriver.com			might be all we have.  Be ready.

Re: What's the staging review and acceptance process?

[Randy Dunlap]

John W. Linville wrote:
> On Thu, Apr 22, 2010 at 08:42:45AM +0200, Olivier Galibert wrote:
>> On Wed, Apr 21, 2010 at 11:12:42PM -0700, Greg KH wrote:
>>> On Wed, Apr 21, 2010 at 10:55:18PM -0700, Joe Perches wrote:
>>>> Not really, patchwork shows status immediately.
>>> Immediately when someone does something with it, right?  So, the same as
>>> my development cycle?
>> I guess the main difference is that patchwork allows one contributor
>> to see that his patch has just not been checked yet, vs. missed/lost.
> 
> In Greg's defense, I find patchwork to be fairly unwieldy (which is
> why I don't use it).  It is certainly possible that I am missing some
> key feature, but my limited experience with it suggested to me that all
> the clicky-clicky stuff required to deal with the individual messages
> queued in patchwork nearly doubled my time overhead associated with
> merging patches.


Just from a patch submitter perspective, I'd like to see some kind of
response.  I can believe what you say about patchwork, so I wouldn't
advocate it.

I agree to Joe's perspective that Greg is overbooked regarding time.
I think that this is an ongoing problem, not just a current one that
will go away, so for me, the question is what is Greg willing to do
about it?

---
~Randy

Re: What's the staging review and acceptance process?

[Greg KH]

On Thu, Apr 22, 2010 at 01:06:16PM -0700, Randy Dunlap wrote:
> John W. Linville wrote:
> > On Thu, Apr 22, 2010 at 08:42:45AM +0200, Olivier Galibert wrote:
> >> On Wed, Apr 21, 2010 at 11:12:42PM -0700, Greg KH wrote:
> >>> On Wed, Apr 21, 2010 at 10:55:18PM -0700, Joe Perches wrote:
> >>>> Not really, patchwork shows status immediately.
> >>> Immediately when someone does something with it, right?  So, the same as
> >>> my development cycle?
> >> I guess the main difference is that patchwork allows one contributor
> >> to see that his patch has just not been checked yet, vs. missed/lost.
> > 
> > In Greg's defense, I find patchwork to be fairly unwieldy (which is
> > why I don't use it).  It is certainly possible that I am missing some
> > key feature, but my limited experience with it suggested to me that all
> > the clicky-clicky stuff required to deal with the individual messages
> > queued in patchwork nearly doubled my time overhead associated with
> > merging patches.
> 
> 
> Just from a patch submitter perspective, I'd like to see some kind of
> response.  I can believe what you say about patchwork, so I wouldn't
> advocate it.
> 
> I agree to Joe's perspective that Greg is overbooked regarding time.
> I think that this is an ongoing problem, not just a current one that
> will go away, so for me, the question is what is Greg willing to do
> about it?

Greg is not going to take up 3 months of his life and sell and buy a
house and move.

Seriously, that took up all of my "free" time, and it's now over, and I
am catching up on things, give me a short ammount of time please.

thanks,

greg k-h

Re: What's the staging review and acceptance process?

[Stefan Richter]

On 4/22/2010 7:55 AM, Joe Perches wrote:
> On Wed, 2010-04-21 at 22:49 -0700, Greg KH wrote:
>> On Wed, Apr 21, 2010 at 09:25:57PM -0700, Joe Perches wrote:
>> > On Wed, 2010-04-21 at 20:45 -0700, Greg KH wrote:
>> > > That caused the huge backlog staring at me right now.
>> > Which likely discouraged the new contributors who
>> > submitted stuff still in that backlog.
>> do you have any evidence of this causing people to go away?
> 
> I think it's pretty established that response delays
> cause losses of one type or another.

Perhaps, perhaps not.  OTOH, the development process documentation
contains the advice not to be discouraged if feedback isn't immediate,
and if in doubt, simply resubmit sometime later.
http://lxr.linux.no/#linux-old+v2.4.0/Documentation/SubmittingPatches#L180
I.e. it is well known that delays may happen and that people should not
worry about them.

If people /do/ get discouraged nevertheless, then this is because they
do not want or cannot afford to spend the (little) time to resubmit or
just ask what the review progress is.  If this happened to a bug fix of
an existing feature, than that is sad and our loss.  But if this
happened to a patch that is meant to develop a new feature --- such as
fundamentally all patches to drivers/staging/ --- then there is not any
real loss of a potential contributor, IMO:  Mid-term or long-term
contributors need to be people who work with the reviewers and
maintainers, and that means to cope with occasional delays.

Of course, if there is a subsystem with _permanent_ huge delays or
actual patch loss or is orphaned entirely, then somebody needs to step
in and offer help with review (which is a hard task and only worthwhile
if the volunteering reviewer is credible) and perhaps with maintenance
duties (which is easier).
-- 
Stefan Richter
-=====-==-=- -=-- =-==-
http://arcgraph.de/sr/

Re: What's the staging review and acceptance process?

[Alan Cox]

On Wed, 21 Apr 2010 22:49:02 -0700
Greg KH <gregkh@suse.de> wrote:

> On Wed, Apr 21, 2010 at 09:25:57PM -0700, Joe Perches wrote:
> > On Wed, 2010-04-21 at 20:45 -0700, Greg KH wrote:
> > > That caused the huge backlog staring at
> > > me right now.
> > 
> > Which likely discouraged the new contributors who
> > submitted stuff still in that backlog.
> 
> While a series of unfortunate events did happen to cause this, do you
> have any evidence of this causing people to go away? 

Subjectively the answer is yes I think. More quantatively it was the case.
I did some measurements long ago with 2.4-ac and there was direct and
clear connection between two things and patch submission/activity levels.
One was 'cycle time' (ie time from submit->response->tree) - the other was
putting the name of the contributor in the per -ac patch summaries that
used to get mailed out.


Alan

Re: What's the staging review and acceptance process?

[Greg KH]

On Thu, Apr 22, 2010 at 10:17:08AM +0100, Alan Cox wrote:
> On Wed, 21 Apr 2010 22:49:02 -0700
> Greg KH <gregkh@suse.de> wrote:
> 
> > On Wed, Apr 21, 2010 at 09:25:57PM -0700, Joe Perches wrote:
> > > On Wed, 2010-04-21 at 20:45 -0700, Greg KH wrote:
> > > > That caused the huge backlog staring at
> > > > me right now.
> > > 
> > > Which likely discouraged the new contributors who
> > > submitted stuff still in that backlog.
> > 
> > While a series of unfortunate events did happen to cause this, do you
> > have any evidence of this causing people to go away? 
> 
> Subjectively the answer is yes I think. More quantatively it was the case.
> I did some measurements long ago with 2.4-ac and there was direct and
> clear connection between two things and patch submission/activity levels.
> One was 'cycle time' (ie time from submit->response->tree) - the other was
> putting the name of the contributor in the per -ac patch summaries that
> used to get mailed out.

Yeah, I know I liked seeing my name there, that was very nice to have :)

thanks,

greg k-h

Re: What's the staging review and acceptance process?

[Willy Tarreau]

On Thu, Apr 22, 2010 at 01:18:19PM -0700, Greg KH wrote:
> On Thu, Apr 22, 2010 at 10:17:08AM +0100, Alan Cox wrote:
> > On Wed, 21 Apr 2010 22:49:02 -0700
> > Greg KH <gregkh@suse.de> wrote:
> > 
> > > On Wed, Apr 21, 2010 at 09:25:57PM -0700, Joe Perches wrote:
> > > > On Wed, 2010-04-21 at 20:45 -0700, Greg KH wrote:
> > > > > That caused the huge backlog staring at
> > > > > me right now.
> > > > 
> > > > Which likely discouraged the new contributors who
> > > > submitted stuff still in that backlog.
> > > 
> > > While a series of unfortunate events did happen to cause this, do you
> > > have any evidence of this causing people to go away? 
> > 
> > Subjectively the answer is yes I think. More quantatively it was the case.
> > I did some measurements long ago with 2.4-ac and there was direct and
> > clear connection between two things and patch submission/activity levels.
> > One was 'cycle time' (ie time from submit->response->tree) - the other was
> > putting the name of the contributor in the per -ac patch summaries that
> > used to get mailed out.
> 
> Yeah, I know I liked seeing my name there, that was very nice to have :)

For this exact reason, I pay a lot of attention to credit the people who
contribute some work, even if I have to adapt it afterwards. I've long
observed that doing this is essential if you want to see them come again
with some nice work.

Concerning the delay, it's even a tunable. When I took over 2.4, I did
not realize that being too much reactive to mails became an incitation
for submitters to send many many things. After some time (primarily due
to lack of time), I started to work by batches and I noticed that the
submission rates dramatically dropped. So I totally agree with Alan here.

BTW Greg, I too remember the time when a buddy was excited to forward to
me an announce from Alan where my name was cited :-)

Regards,
Willy