Ceph backports workflow update

[Loic Dachary <loic@dachary.org>]

[-- Attachment #1: Type: text/plain, Size: 4507 bytes --]

Hi Ceph,

A month ago the following workflow was posted and I began to implement it.

> 0. Developer follows normal process to land PR to master. Once complete and ticket is marked Pending Backport this process initiates.

There were a few inconsistencies but they were easy to fix. When the tag is missing I update it manually (the redmine API is broken http://tracker.ceph.com/issues/10727 otherwise I would probably have written a script to semi-manually do that).

> 1. I periodically polls Redmine to look for tickets in Pending Backport state and focus on the ones that are left unattended for too long

I focused on giant and dumpling and was able to help with a few backports. However, I've not yet visited the majority of the issues that need attention ( see http://workbench.dachary.org/ceph/ceph-backports/wikis/dumpling#issues-that-need-backporting for dumpling and http://workbench.dachary.org/ceph/ceph-backports/wikis/giant#issues-that-need-backporting for giant ).

> 1a. Under the supervision of the author of the original patch, I find the commits associated with the Redmine ticket and Cherry Pick to the backport integration branch off of the desired maintenance branch (Dumping, Firefly, etc).
> 1b. I resolve any merge conflicts with the cherry-picked commit

It turns out that finding the relevant commits in almost all backports is made possible by the cross references between pull requests, commits and issues. I was able to backport commits that are trivial. Most of the other backports were done by the original author of the patch because I did not understand enough of the context to be helpful. 

> 2. I merge all backports for a given branch in an integration branch

It is done with something like

git merge --strategy octopus backports/pull/3439/head backports/pull/3552/head backports/pull/3489/head

and there currently are two integration branches:

* pull requests in dumpling-backports http://workbench.dachary.org/ceph/ceph-backports/wikis/dumpling#included-and-tested-in-integration-branch
* pull requests in giant-backports http://workbench.dachary.org/ceph/ceph-backports/wikis/giant#included-and-tested-in-integration-branch

Only once did I face a merge conflict. It was trivial and I resolved it. I should go back to the author of the patch and warn him about this conflict so that it does not create difficulties when all branches are finally merged.

> 3. I ask the leads of each project to review the integration

I did not do that. If I had been able to backport the non trivial commits, it may have been necessary. But I've been in contact with the leads regarding the individual backports and reviewing the set of commits being integrated seemed redundant. 

> 4. Once satisfied with group of backported commits to integration branch, I notify QE.

Running rbd, rados, rgw and is the bulkd of the work. The progress of the test runs and analysis are:

* dumpling http://tracker.ceph.com/issues/10560
* giant http://tracker.ceph.com/issues/10501

This is usually done as a mail thread in the ceph-qa mailing list but I did not want to disturb the list with the backports. In addition I felt the need to see the past analysis on a single page to remember where I was after a few days doing something else. The mail thread format did not provide that and I prefered to create a ticket for that purpose. It's proven both useful and cumbersome, I'll try to figure out something that shows the same information without so much manual maintenance.

> 5. QE tests backport integration branch against appropriate suites
> 6a. If QE is satisfied with test results, they merge backport integration branch.
> 6b. If QE is NOT satisfied with the test results, they indicate backport integration branch is NOT ready to merge and return to me to work with original Developer to resolve issue and return to steps 2/3
> 7. Ticket is moved to Resolved once backport integration branch containing cherry-picked backport is merged to the desired mainteance branch(es)

I've not yet reached this point, to be continued :-)

Cheers

P.S. Manually investigating each backport proved to be extremely tedious and repetitive. Fortunately there are patterns that allowed me to grow a script that creates an inventory for each branch to be backported ( http://workbench.dachary.org/ceph/ceph-backports/wikis/dumpling etc. ) that I use as my landing page when working on backports.

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]