Proposal for a Backport tracker

Proposal for a Backport tracker

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 4440 bytes --]

Hi,

When an issue such as http://tracker.ceph.com/issues/9806 needs backporting, 

 * its status is changed to Pending Backport
 * the backport field is modified to contain the target stable releases
 * the severity field is modified to reflect the urgency of the backport

(see also http://tracker.ceph.com/projects/ceph-releases/wiki/HOWTO_schedule_an_issue_for_backporting). When a backport is ready for a given stable release, a comment is added with something like:

  * firefly backport https://github.com/ceph/ceph/pull/XXX is added to the issue

(either manually or in a semi automated way, see http://tracker.ceph.com/projects/ceph-releases/wiki/HOWTO_backport_commits). 

From time to time the Pending Backport issues are scrubbed to resolve those that have been successfully backported to all the desired stable releases (see http://tracker.ceph.com/projects/ceph-releases/wiki/HOWTO_resolve_issues_that_are_Pending_Backport for details).

This is not too complicated to explain and not too tedious to maintain. The only pain point is that we use the original bug report to track backports. It would probably be easier to have a separate issue, in a separate tracker to followup on the backports. It could go like this:

When an issue such as http://tracker.ceph.com/issues/9806 needs backporting, 

  * its status is changed to Pending Backport
  * the backport field is modified to contain the target stable releases
  * a backport issue is created in the "Backport" tracker and is "Related" to the original issue (probably using a script parsing the content of the backport field and not manually)
    * the issue is assigned to the backporter who will work on it
    * the release field of the backport issue is set to the target stable release (hammer, firefly, ...)
    * the Priority field is modified to reflect the urgency of the backport (it may be urgent for hammer and not for firefly, a distinction that is currently lost and using the severity field is kind of confusing because most if not all developers and predefined queries are looking at the value of the Priority field, not the severity field)
 
When a backport is ready for a given stable release:

  * the "Backport URL" field of the backport issue is set to the corresponding pull request or commit (sometime backports are not via a pull request, but all backports do have at least one URL)
  * when the backport is merged the issue is closed

From time to time, the "Pending Backport" issues for which all related issues from the Backport tracker have been closed are also closed. This step could even be entirely automated.

An additional benefit of having this separate tracker is that issues that require a non trivial backport could be assigned to a developer instead of a backporter and be discussed during bug scrub when reviewing http://tracker.ceph.com/projects/rbd/issues?query_id=27. There only are a handfull of them: for instance http://tracker.ceph.com/issues/9806 could be assigned to Josh and during bug scrub it would show to be from the Backport tracker.

The developer would not have to create issues to schedule a backport: (s)he would just update the backport field as (s)he currently does. The backporter will then be in charge of creating the issues. Most likely with a script creating the issues based on the content of the backport field instead of doing it manually. Having the developer create one issue per backport would not be good for two reasons : it would require explaining and it would be more work.

The justification for having a different tracker instead of using an existing one for the backport issues is primarily because the tracker shows in a number of pre-existing custom queries such as http://tracker.ceph.com/projects/rbd/issues?query_id=27. It's a easy way to convey the information: this issue is about backporting. A secondary benefit is that it's possible to define a workflow specific to a tracker, which could be interesting in the future because the backport workflow is not the same as the bug fixing workflow.

This is just an idea and I did not think this through. I like it right now but it just occurred to me this morning, after reading a mail from Josh who suggested we need a way for issues such as #9806 to be visible during bug scrub. Feel free to criticize bluntly, I won't be offended ;-)

Cheers

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Proposal for a Backport tracker

[Nathan Cutler]

> * a backport issue is created in the "Backport" tracker and is 
> "Related" to the original issue

Hi Loic:

I like the idea of having backport tickets in a separate Redmine
subproject/issue tracker. In fact, I would go even a step further and
have separate subprojects for each target version (hammer backports,
firefly backports).

Having all, e.g. hammer, backports in one place is advantageous in
pretty much every way I can think of: easier to see what has been
done, what needs to be done, easier to search, easier to automate,
less risk of munging something not related to backports...

It also has the effect of removing all the backport-related tickets
from the bug tracker(s).

Nathan

Re: Proposal for a Backport tracker

[Abhishek L]

[-- Attachment #1: Type: text/plain, Size: 1356 bytes --]

Hi

Nathan Cutler writes:

>> * a backport issue is created in the "Backport" tracker and is 
>> "Related" to the original issue
>
> Hi Loic:
>
> I like the idea of having backport tickets in a separate Redmine
> subproject/issue tracker. In fact, I would go even a step further and
> have separate subprojects for each target version (hammer backports,
> firefly backports).

Agree with the idea of seperate subproject. Though I'm not having a
strong opinion on whether subprojects are needed for each version;
probably for issues which have multiple backports ie. backports to both
firefly & hammer for instance it might be a little bit more bookkeeping,
though from a maintainer standpoint; it is easier having a seperate
subproject, so either way is okay. 

> Having all, e.g. hammer, backports in one place is advantageous in
> pretty much every way I can think of: easier to see what has been
> done, what needs to be done, easier to search, easier to automate,
> less risk of munging something not related to backports...
>

Agree, from a scripting standpoint, we could develop some tools to ease
out the process better, which should it make it easier in backports down
the line.

> It also has the effect of removing all the backport-related tickets
> from the bug tracker(s).




>
> Nathan

-- 
Abhishek

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 489 bytes --]

Re: Proposal for a Backport tracker

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 2076 bytes --]



On 21/05/2015 15:38, Abhishek L wrote:
> Hi
> 
> Nathan Cutler writes:
> 
>>> * a backport issue is created in the "Backport" tracker and is 
>>> "Related" to the original issue
>>
>> Hi Loic:
>>
>> I like the idea of having backport tickets in a separate Redmine
>> subproject/issue tracker. In fact, I would go even a step further and
>> have separate subprojects for each target version (hammer backports,
>> firefly backports).
> 
> Agree with the idea of seperate subproject. 

Another (I mean in addition to what I wrote in reply to Nathan) against subprojects is that existing custom queries would no longer display the backport issues. For instance http://tracker.ceph.com/projects/ceph/issues?query_id=47 is frequently used and does *not* display subprojects because Ceph is the parent project of all other projects and it would display all their issues. If there was a subproject for backports, it would not show. And if I'm not mistaken there is no way to say : "display issues from these subprojects and ignore the others". 

> Though I'm not having a
> strong opinion on whether subprojects are needed for each version;
> probably for issues which have multiple backports ie. backports to both
> firefly & hammer for instance it might be a little bit more bookkeeping,
> though from a maintainer standpoint; it is easier having a seperate
> subproject, so either way is okay. 
> 
>> Having all, e.g. hammer, backports in one place is advantageous in
>> pretty much every way I can think of: easier to see what has been
>> done, what needs to be done, easier to search, easier to automate,
>> less risk of munging something not related to backports...
>>
> 
> Agree, from a scripting standpoint, we could develop some tools to ease
> out the process better, which should it make it easier in backports down
> the line.
> 

I think we can do that by creating a tracker. It would be in addition to the Bug, Fix, Documentation etc. trackers that are already in place.

Cheers

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Proposal for a Backport tracker

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 1915 bytes --]



On 21/05/2015 14:41, Nathan Cutler wrote:
>> * a backport issue is created in the "Backport" tracker and is 
>> "Related" to the original issue
> 
> Hi Loic:
> 
> I like the idea of having backport tickets in a separate Redmine
> subproject/issue tracker. 

It would just be in a separate tracker, in the same project. 

> In fact, I would go even a step further and
> have separate subprojects for each target version (hammer backports,
> firefly backports).

Separating them in a different project / subproject would create problems because redmine has ways to partition the subprojects that make some things difficult. For instance not all issues can be "Related" to issues in other projects which is kind of annoying. My general impression with redmine subprojects is that they tend to complexify and obscure the process rather than help. Or maybe it's just that my redmine skills are not what they should be. Do you have a different experience ?

> Having all, e.g. hammer, backports in one place is advantageous in
> pretty much every way I can think of: easier to see what has been
> done, what needs to be done, easier to search, easier to automate,
> less risk of munging something not related to backports...

Yes. And I think we can have that by grouping them in a tracker instead of a subproject. Unless there are things we can only do if that's a subproject ?

> It also has the effect of removing all the backport-related tickets
> from the bug tracker(s).

With the risk that developers who currently see and care for pending backport tickets would no longer see them. We would then have to seduce them into adding one more place to the list of places they have to watch. 

Maybe I'm being too conservative ? Too eager to not disrupt anything that could cause a problem to the developers ?

Cheers
> 
> Nathan
> 

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Proposal for a Backport tracker

[Sage Weil]

On Thu, 21 May 2015, Loic Dachary wrote:
> 
> On 21/05/2015 14:41, Nathan Cutler wrote:
> >> * a backport issue is created in the "Backport" tracker and is 
> >> "Related" to the original issue
> > 
> > Hi Loic:
> > 
> > I like the idea of having backport tickets in a separate Redmine
> > subproject/issue tracker. 
> 
> It would just be in a separate tracker, in the same project. 
> 
> > In fact, I would go even a step further and
> > have separate subprojects for each target version (hammer backports,
> > firefly backports).
> 
> Separating them in a different project / subproject would create 
> problems because redmine has ways to partition the subprojects that make 
> some things difficult. For instance not all issues can be "Related" to 
> issues in other projects which is kind of annoying. My general 
> impression with redmine subprojects is that they tend to complexify and 
> obscure the process rather than help. Or maybe it's just that my redmine 
> skills are not what they should be. Do you have a different experience ?

I agree.  Adding subprojects clutters up the project list, and a separate 
tracker captures this perfectly.  It also means you can rank by priority 
issues and see both bugs and backprots together (if you like).  And I 
think (?) the related issue only works within the same project (that is 
at least true with the 'duplicate' relationship).

Loic, I think this is a great idea (provided the bot does all the issue 
creation).  +1!

sage

Re: Proposal for a Backport tracker

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 1957 bytes --]



On 21/05/2015 17:46, Sage Weil wrote:
> On Thu, 21 May 2015, Loic Dachary wrote:
>>
>> On 21/05/2015 14:41, Nathan Cutler wrote:
>>>> * a backport issue is created in the "Backport" tracker and is 
>>>> "Related" to the original issue
>>>
>>> Hi Loic:
>>>
>>> I like the idea of having backport tickets in a separate Redmine
>>> subproject/issue tracker. 
>>
>> It would just be in a separate tracker, in the same project. 
>>
>>> In fact, I would go even a step further and
>>> have separate subprojects for each target version (hammer backports,
>>> firefly backports).
>>
>> Separating them in a different project / subproject would create 
>> problems because redmine has ways to partition the subprojects that make 
>> some things difficult. For instance not all issues can be "Related" to 
>> issues in other projects which is kind of annoying. My general 
>> impression with redmine subprojects is that they tend to complexify and 
>> obscure the process rather than help. Or maybe it's just that my redmine 
>> skills are not what they should be. Do you have a different experience ?
> 
> I agree.  Adding subprojects clutters up the project list, and a separate 
> tracker captures this perfectly.  It also means you can rank by priority 
> issues and see both bugs and backprots together (if you like).  And I 
> think (?) the related issue only works within the same project (that is 
> at least true with the 'duplicate' relationship).
> 
> Loic, I think this is a great idea (provided the bot does all the issue 
> creation).  +1!

I created the Backport tracker and we will migrate the pending issues & HOWTO accordingly.

Cheers

> 
> sage
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Proposal for a Backport tracker

[Nathan Cutler]

Hi Loic:

As I understand the new backport workflow, the process of creating 
tickets in the Backport tracker will be automated. This raises a question:

As the automation script loops over each Pending Backport ticket, it 
will first create a corresponding ticket (or tickets) in the Backport 
tracker. After that, I'm thinking it will need to change the status of 
the original ticket from Pending Backport to something else (so it 
doesn't pick up the same ticket again the next time the script is run).

So the question is, what will the original ticket's status be changed 
to? Resolved?

Nathan

Re: Proposal for a Backport tracker

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 969 bytes --]

Hi Nathan,

On 22/05/2015 09:06, Nathan Cutler wrote:> Hi Loic:
> 
> As I understand the new backport workflow, the process of creating tickets in the Backport tracker will be automated. This raises a question:
> 
> As the automation script loops over each Pending Backport ticket, it will first create a corresponding ticket (or tickets) in the Backport tracker. After that, I'm thinking it will need to change the status of the original ticket from Pending Backport to something else (so it doesn't pick up the same ticket again the next time the script is run).

We can probably keep the original ticket in the "Pending Backport" status until all backports are resolved, as we currently do. 

> So the question is, what will the original ticket's status be changed to? Resolved?

When all backports are resolved, then we can also resolve the original ticket. Do you forsee a problem with that ?

Cheers

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Proposal for a Backport tracker

[Nathan Cutler]

> We can probably keep the original ticket in the "Pending Backport" status until all backports are resolved, as we currently do.
>
>> So the question is, what will the original ticket's status be changed to? Resolved?
>
> When all backports are resolved, then we can also resolve the original ticket. Do you forsee a problem with that ?
>

Yes and no. It's a scripting problem, not a workflow problem. I'll try 
to describe the scenario I'm envisioning:

We have a simple script that loops over all tickets with status "Pending 
Backport". We run it once per week. The first week we run it, the script 
finds 3 such tickets. For each one it creates a ticket in the Backport 
tracker.

A week goes by, during which developers marked 4 more tickets "Pending 
Backport". It's time to run the script again. Since it is looping over 
all tickets marked "Pending Backport", it now finds 7 such tickets: 3 
from last week and the 4 new ones. For each one it dutifully creates a 
new ticket in the Backport tracker. Now we have duplicate Backport 
tickets for 3 bugfixes.

But I guess it will be trivial to make the script check if a Backport 
ticket is already open and refrain from opening a new one in this case.

The same script could also check further: if Backport tickets already 
exist for the bugfix and all of them are marked Resolved, then 
automatically mark the original ticket Resolved as well.

Nathan

Re: Proposal for a Backport tracker

[Joao Eduardo Luis]

On 05/22/2015 09:32 AM, Nathan Cutler wrote:
>> We can probably keep the original ticket in the "Pending Backport"
>> status until all backports are resolved, as we currently do.
>>
>>> So the question is, what will the original ticket's status be changed
>>> to? Resolved?
>>
>> When all backports are resolved, then we can also resolve the original
>> ticket. Do you forsee a problem with that ?
>>
> 
> Yes and no. It's a scripting problem, not a workflow problem. I'll try
> to describe the scenario I'm envisioning:
> 
> We have a simple script that loops over all tickets with status "Pending
> Backport". We run it once per week. The first week we run it, the script
> finds 3 such tickets. For each one it creates a ticket in the Backport
> tracker.
> 
> A week goes by, during which developers marked 4 more tickets "Pending
> Backport". It's time to run the script again. Since it is looping over
> all tickets marked "Pending Backport", it now finds 7 such tickets: 3
> from last week and the 4 new ones. For each one it dutifully creates a
> new ticket in the Backport tracker. Now we have duplicate Backport
> tickets for 3 bugfixes.
> 
> But I guess it will be trivial to make the script check if a Backport
> ticket is already open and refrain from opening a new one in this case.

Alternatively, adding a new state to the tracker that distinguishes
between tickets that are pending being picked up on by the stable
releases team vs those that are just pending the backport.  Say, "Mark
for Backport" vs "Pending Backport".

  -Joao

Re: Proposal for a Backport tracker

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 2065 bytes --]



On 22/05/2015 12:36, Joao Eduardo Luis wrote:
> On 05/22/2015 09:32 AM, Nathan Cutler wrote:
>>> We can probably keep the original ticket in the "Pending Backport"
>>> status until all backports are resolved, as we currently do.
>>>
>>>> So the question is, what will the original ticket's status be changed
>>>> to? Resolved?
>>>
>>> When all backports are resolved, then we can also resolve the original
>>> ticket. Do you forsee a problem with that ?
>>>
>>
>> Yes and no. It's a scripting problem, not a workflow problem. I'll try
>> to describe the scenario I'm envisioning:
>>
>> We have a simple script that loops over all tickets with status "Pending
>> Backport". We run it once per week. The first week we run it, the script
>> finds 3 such tickets. For each one it creates a ticket in the Backport
>> tracker.
>>
>> A week goes by, during which developers marked 4 more tickets "Pending
>> Backport". It's time to run the script again. Since it is looping over
>> all tickets marked "Pending Backport", it now finds 7 such tickets: 3
>> from last week and the 4 new ones. For each one it dutifully creates a
>> new ticket in the Backport tracker. Now we have duplicate Backport
>> tickets for 3 bugfixes.
>>
>> But I guess it will be trivial to make the script check if a Backport
>> ticket is already open and refrain from opening a new one in this case.
> 
> Alternatively, adding a new state to the tracker that distinguishes
> between tickets that are pending being picked up on by the stable
> releases team vs those that are just pending the backport.  Say, "Mark
> for Backport" vs "Pending Backport".
> 

I liked the idea, wrote a reply and then realized: that won't save the script from checking if the reality matches the status. Sometimes the "backports" field is updated to remove or add a new backport target. It may mean closing or creating the corresponding issue and for that the script will have to query the backport tracker.

>   -Joao
> 
> 

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Proposal for a Backport tracker

[Sage Weil]

On Fri, 22 May 2015, Loic Dachary wrote:
> On 22/05/2015 12:36, Joao Eduardo Luis wrote:
> > On 05/22/2015 09:32 AM, Nathan Cutler wrote:
> >>> We can probably keep the original ticket in the "Pending Backport"
> >>> status until all backports are resolved, as we currently do.
> >>>
> >>>> So the question is, what will the original ticket's status be changed
> >>>> to? Resolved?
> >>>
> >>> When all backports are resolved, then we can also resolve the original
> >>> ticket. Do you forsee a problem with that ?
> >>>
> >>
> >> Yes and no. It's a scripting problem, not a workflow problem. I'll try
> >> to describe the scenario I'm envisioning:
> >>
> >> We have a simple script that loops over all tickets with status "Pending
> >> Backport". We run it once per week. The first week we run it, the script
> >> finds 3 such tickets. For each one it creates a ticket in the Backport
> >> tracker.
> >>
> >> A week goes by, during which developers marked 4 more tickets "Pending
> >> Backport". It's time to run the script again. Since it is looping over
> >> all tickets marked "Pending Backport", it now finds 7 such tickets: 3
> >> from last week and the 4 new ones. For each one it dutifully creates a
> >> new ticket in the Backport tracker. Now we have duplicate Backport
> >> tickets for 3 bugfixes.
> >>
> >> But I guess it will be trivial to make the script check if a Backport
> >> ticket is already open and refrain from opening a new one in this case.
> > 
> > Alternatively, adding a new state to the tracker that distinguishes
> > between tickets that are pending being picked up on by the stable
> > releases team vs those that are just pending the backport.  Say, "Mark
> > for Backport" vs "Pending Backport".
> > 
> 
> I liked the idea, wrote a reply and then realized: that won't save the 
> script from checking if the reality matches the status. Sometimes the 
> "backports" field is updated to remove or add a new backport target. It 
> may mean closing or creating the corresponding issue and for that the 
> script will have to query the backport tracker.

Being dependent on the states feel fragile to me, especially adding a 
"Mark for Backport" state.  It seems like the bot, once it looks at an 
issue, can look at the related backport items vs the backport field and 
make sure they exist, without worrying about the state.

But, we can't do that for all issues since all the old ones don't have 
this structure.

How about: the bot only looks at Pending Backport, and

 - ensures that linked issues exist for everything in the backport field
 - if all linked issues are resolved, changes status to Resolved.

That's it.  The only workflow requirement is that the dev who fixes the 
original bug can't simply set the ticket to Resolved (they must do Pending 
Backport) or else the process is short circuited.  This is already true 
today anyway.

This is probably already what you have in mind?

sage

Re: Proposal for a Backport tracker

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 1692 bytes --]



On 22/05/2015 10:32, Nathan Cutler wrote:>> We can probably keep the original ticket in the "Pending Backport" status until all backports are resolved, as we currently do.
>>
>>> So the question is, what will the original ticket's status be changed to? Resolved?
>>
>> When all backports are resolved, then we can also resolve the original ticket. Do you forsee a problem with that ?
>>
> 
> Yes and no. It's a scripting problem, not a workflow problem. I'll try to describe the scenario I'm envisioning:
> 
> We have a simple script that loops over all tickets with status "Pending Backport". We run it once per week. The first week we run it, the script finds 3 such tickets. For each one it creates a ticket in the Backport tracker.
> 
> A week goes by, during which developers marked 4 more tickets "Pending Backport". It's time to run the script again. Since it is looping over all tickets marked "Pending Backport", it now finds 7 such tickets: 3 from last week and the 4 new ones. For each one it dutifully creates a new ticket in the Backport tracker. Now we have duplicate Backport tickets for 3 bugfixes.
> 
> But I guess it will be trivial to make the script check if a Backport ticket is already open and refrain from opening a new one in this case.

Yes, by investigating the "Related issues" and finding those that belong to the Backport tracker, we should be able to do that.

> 
> The same script could also check further: if Backport tickets already exist for the bugfix and all of them are marked Resolved, then automatically mark the original ticket Resolved as well.

Absolutely :-)

> 
> Nathan

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]