a home for backport snippets

a home for backport snippets

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 1060 bytes --]

Hi,

Today, Nathan and I briefly discussed the idea of collecting the backport snippets that are archived in the wiki at http://tracker.ceph.com/projects/ceph-releases/wiki/HOWTO. We all have copies on our local disks and although they don't diverge much, this is not very sustainable. It was really good as we established the backport workflows. And it would have been immensely painful to maintain a proper software while we were changing the workflow on a regular basis. But it looks like we now have something stable.

Early this year ceph-workbench[1] was started with the idea of helping with backports. It is a mostly empty shell we can now use to collect all the snippets we have. Instead of adding set-release[2] to the script directory of Ceph, it would be a subcommand of ceph-workbench, like so:

  ceph-workbench set-release --token $github_token --key $redmine_key

What do you think ?

Cheers

[1] https://pypi.python.org/pypi/ceph-workbench
[2] https://github.com/ceph/ceph/pull/6466
-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: a home for backport snippets

[Nathan Cutler]

>   ceph-workbench set-release --token $github_token --key $redmine_key
> 
> What do you think ?

Would be nice to be able to put $github_token and $redmine_key into a
configuration file.

-- 
Nathan Cutler
Software Engineer Distributed Storage
SUSE LINUX, s.r.o.
Tel.: +420 284 084 037

Re: a home for backport snippets

[Abhishek Varshney]

Hi Loic,

It is definitely a great idea to have all the backport snippets under
a roof. However, these snippets, which are mostly a set of commands,
provide great flexibility in terms of configuration and the ability to
partly execute them. For instance, if I see conflicts while preparing
an integration branch, I comment out the git fetch, checkout and reset
steps from [1] after resolving the conflicts and re-run the snippet.
It would be nice if we can incorporate all the snippets and take
ceph-workbench to that level of flexibility :)

[1] http://tracker.ceph.com/projects/ceph-releases/wiki/HOWTO_populate_the_integration_branch

Thanks
Abhishek

On Thu, Nov 5, 2015 at 6:50 PM, Loic Dachary <loic@dachary.org> wrote:
> Hi,
>
> Today, Nathan and I briefly discussed the idea of collecting the backport snippets that are archived in the wiki at http://tracker.ceph.com/projects/ceph-releases/wiki/HOWTO. We all have copies on our local disks and although they don't diverge much, this is not very sustainable. It was really good as we established the backport workflows. And it would have been immensely painful to maintain a proper software while we were changing the workflow on a regular basis. But it looks like we now have something stable.
>
> Early this year ceph-workbench[1] was started with the idea of helping with backports. It is a mostly empty shell we can now use to collect all the snippets we have. Instead of adding set-release[2] to the script directory of Ceph, it would be a subcommand of ceph-workbench, like so:
>
>   ceph-workbench set-release --token $github_token --key $redmine_key
>
> What do you think ?
>
> Cheers
>
> [1] https://pypi.python.org/pypi/ceph-workbench
> [2] https://github.com/ceph/ceph/pull/6466
> --
> Loïc Dachary, Artisan Logiciel Libre
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: a home for backport snippets

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 2281 bytes --]

Hi Abhishek,

On 06/11/2015 07:11, Abhishek Varshney wrote:
> Hi Loic,
> 
> It is definitely a great idea to have all the backport snippets under
> a roof. However, these snippets, which are mostly a set of commands,
> provide great flexibility in terms of configuration and the ability to
> partly execute them. For instance, if I see conflicts while preparing
> an integration branch, I comment out the git fetch, checkout and reset
> steps from [1] after resolving the conflicts and re-run the snippet.
> It would be nice if we can incorporate all the snippets and take
> ceph-workbench to that level of flexibility :)

This is a good point: I'm doing the same kind of copy / paste. We'll have to adjust the granularity.

   http://ceph-workbench.dachary.org/users/auth/github

was setup and is dedicated to the tool. Please let me know when you first login so that I can make you an admin.

Cheers

> 
> [1] http://tracker.ceph.com/projects/ceph-releases/wiki/HOWTO_populate_the_integration_branch
> 
> Thanks
> Abhishek
> 
> On Thu, Nov 5, 2015 at 6:50 PM, Loic Dachary <loic@dachary.org> wrote:
>> Hi,
>>
>> Today, Nathan and I briefly discussed the idea of collecting the backport snippets that are archived in the wiki at http://tracker.ceph.com/projects/ceph-releases/wiki/HOWTO. We all have copies on our local disks and although they don't diverge much, this is not very sustainable. It was really good as we established the backport workflows. And it would have been immensely painful to maintain a proper software while we were changing the workflow on a regular basis. But it looks like we now have something stable.
>>
>> Early this year ceph-workbench[1] was started with the idea of helping with backports. It is a mostly empty shell we can now use to collect all the snippets we have. Instead of adding set-release[2] to the script directory of Ceph, it would be a subcommand of ceph-workbench, like so:
>>
>>   ceph-workbench set-release --token $github_token --key $redmine_key
>>
>> What do you think ?
>>
>> Cheers
>>
>> [1] https://pypi.python.org/pypi/ceph-workbench
>> [2] https://github.com/ceph/ceph/pull/6466
>> --
>> Loïc Dachary, Artisan Logiciel Libre
>>

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: a home for backport snippets

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 4063 bytes --]

Hi,

The new snippets home is at https://pypi.python.org/pypi/ceph-workbench and http://ceph-workbench.dachary.org/root/ceph-workbench.

The first snippet was merged by Nathan yesterday[1], the backport documentation updated accordingly[2], and I used it after merging half a dozen hammer backport that were approved a few days ago.

Integration tests should provide the best help against regression we can hope for (they spawn a redmine instance every time they run and use a dedicated github user to create and destroy projects, pull requests etc.) and they are run on every merge request[3]. When integrated in ceph-workbench, the snippet is documented[4] and the implementation[5] is tested in full[6]. The merits of 100% coverage are often disputed as overkill. IMHO it's better to remove an untested line of code rather than taking the chance that it grows into something that does not work (or possibly never worked). In the case of this snippet, there is a dozen of safe guards and four lines of code to modify the issue. It would be bad to discover, after modifying hundreds of issues in the Ceph tracker, that it never worked as expected. I'm sure we'll find ways to *not* do the right thing even with integration tests. But we'll hopefully do the right thing more often ;-)

I'm not sure how much time it will take us to convert all the snippets we have, but it does not matter much as we can keep doing things manually in the meantime.

Cheers

P.S. We are using a GitLab instance, with an integrated CI, instead of github with a CI on jenkins.ceph.com roughly for the same reasons puppet-ceph is in https://github.com/openstack/puppet-ceph and uses the OpenStack gates. We have no expertise on jenkins-job-builder[7] and the learning curve is perceived as significantly higher than a GitLab with an integrated CI[8]. We also want to share administrative permissions on the CI with all members of the stable release team to share the maintenance workload.

[1] backport-set-release http://ceph-workbench.dachary.org/root/ceph-workbench/merge_requests/8
[2] Resolving an issue http://tracker.ceph.com/projects/ceph-releases/wiki/HOWTO_merge_commits_from_the_integration_branch#Resolving-the-matching-issue
[3] Continuous integration http://ceph-workbench.dachary.org/dachary/ceph-workbench/builds/53
[4] Documentation http://ceph-workbench.dachary.org/root/ceph-workbench/merge_requests/8/diffs#9f3ebf1fc38506b66593397f3baac514d515c496_73_75
[5] Implementation http://ceph-workbench.dachary.org/root/ceph-workbench/merge_requests/8/diffs#070f4537c6cef8a2dacef1911a7d39acd0ce1387_0_75
[6] Testing http://ceph-workbench.dachary.org/root/ceph-workbench/merge_requests/8/diffs#66bd83c5111f0ccc884ad791c4acaa926ab52c2a_0_64
[7] Jenkins Job Builder http://docs.openstack.org/infra/jenkins-job-builder/ 
[8] Configuration of your builds with .gitlab-ci.yml http://doc.gitlab.com/ci/yaml/README.html

On 05/11/2015 14:20, Loic Dachary wrote:
> Hi,
> 
> Today, Nathan and I briefly discussed the idea of collecting the backport snippets that are archived in the wiki at http://tracker.ceph.com/projects/ceph-releases/wiki/HOWTO. We all have copies on our local disks and although they don't diverge much, this is not very sustainable. It was really good as we established the backport workflows. And it would have been immensely painful to maintain a proper software while we were changing the workflow on a regular basis. But it looks like we now have something stable.
> 
> Early this year ceph-workbench[1] was started with the idea of helping with backports. It is a mostly empty shell we can now use to collect all the snippets we have. Instead of adding set-release[2] to the script directory of Ceph, it would be a subcommand of ceph-workbench, like so:
> 
>   ceph-workbench set-release --token $github_token --key $redmine_key
> 
> What do you think ?
> 
> Cheers
> 
> [1] https://pypi.python.org/pypi/ceph-workbench
> [2] https://github.com/ceph/ceph/pull/6466
> 

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]