ceph-ci.git?

ceph-ci.git?

[Sage Weil]

As the number of people contributing code grows, we've added more and more 
people to the github ceph org who have write access to ceph.git.  Those 
people can merge pull requests and can also push branches directly to the 
repo.

We also use ceph.git as a source for the test build infrastrucure 
(gitbuilders) to generate packages for QA or hot fixes and make check 
tests.  This leads to an every-growing body of wip-* branches in the repo 
(which is annoying), and also means that in order to build something to 
test in QA you also get the ability to (say) push directly to master.

How about we instead

 - create a second repo named something like ceph-ci.git (that's the best 
I can come up with at the moment) 
 - add this as a second source for all gitbuilders (they can poll a list)
 - move all wip-* branches here
 - create a new github team with contributing developers who can push to 
this repo and are trusted not to wreak havoc on the builders
 - remove all the cruft from ceph.git, so that it's just master, next, the 
stable branches, release tags, and anything else similarly important.
 - restrict ceph.git write access to core developers

This will improve security somewhat and reduce the risk of an accidental 
push to an important branch.

It may also reduce the risk associated with accidental force pushes 
(something we've hemmed and hawed about recently) by limiting the circle 
of people who can write to ceph.git and also changing workflows so that it 
is almost never used directly...

?
sage

Re: ceph-ci.git?

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 4441 bytes --]

Hi Sage,

On 18/05/2015 06:40, Sage Weil wrote:
> As the number of people contributing code grows, we've added more and more 
> people to the github ceph org who have write access to ceph.git.  Those 
> people can merge pull requests and can also push branches directly to the 
> repo.
> 
> We also use ceph.git as a source for the test build infrastrucure 
> (gitbuilders) to generate packages for QA or hot fixes and make check 
> tests.  This leads to an every-growing body of wip-* branches in the repo 
> (which is annoying), and also means that in order to build something to 
> test in QA you also get the ability to (say) push directly to master.
> 
> How about we instead
> 
>  - create a second repo named something like ceph-ci.git (that's the best 
> I can come up with at the moment) 
>  - add this as a second source for all gitbuilders (they can poll a list)
>  - move all wip-* branches here
>  - create a new github team with contributing developers who can push to 
> this repo and are trusted not to wreak havoc on the builders
>  - remove all the cruft from ceph.git, so that it's just master, next, the 
> stable branches, release tags, and anything else similarly important.
>  - restrict ceph.git write access to core developers
> 
> This will improve security somewhat and reduce the risk of an accidental 
> push to an important branch.
> 
> It may also reduce the risk associated with accidental force pushes 
> (something we've hemmed and hawed about recently) by limiting the circle 
> of people who can write to ceph.git and also changing workflows so that it 
> is almost never used directly...

Having a reference repository for releases only, would be a great move.

One problem with promoting http://github.com/ceph/ceph to that role is that while the migration is in progress there will be two queues of pull requests, the new one at http://github.com/ceph-ci/ceph and the old one at http://github.com/ceph/ceph. Given our current flow of pull requests the migration will probably take about three months. And after that, since it's not possible to forbid pull requests on github.com (you can disable issues but not pull requests on a given repository) there will forever be a flow of misdirected pull requests against the old repository.

Maybe it would be better to keep http://github.com/ceph/ceph as it is and create a release repository (say https://git.ceph.com/?p=ceph-release.git) to only have the stable, master and next branches. Contributors who accidentally force push master on http://github.com/ceph/ceph would have the comfort of knowing that master on https://git.ceph.com/?p=ceph-release.git can be trusted to be a reference.

Updating https://git.ceph.com/?p=ceph-release.git could be done by a gate instead of trusting a group of people with it. The gate could be as simple as a cron job mirroring a predefined list of stable branches from http://github.com/ceph/ceph to https://git.ceph.com/?p=ceph-release.git. Although it could evolve into something more sophisticated in the future (such as *not* mirroring a branch for which the gitbuilders are red), this simple minded mirror would already effectively protect us against a force push, because the mirroring command would be setup in a way that does not allow force push. In the event of a force push on http://github.com/ceph/ceph, the mirror would fail and the force pushed branch on http://github.com/ceph/ceph can conveniently be reset to the latest from https://git.ceph.com/?p=ceph-release.git.

Although https://git.ceph.com/?p=ceph-release.git could be setup as http://github.com/ceph/ceph-release, it has a few disadvantages (pull requests can be misdirected to it, the repository cannot be set to protect against force push).

As time passes existing workflows (releases for instance) can gradually/opportunistically migrate from using http://github.com/ceph/ceph as a source to https://git.ceph.com/?p=ceph-release.git, to benefit from a more stable source. But even if they don't, the worst that can happen is that they temporarily suffer from a forced push, just as they currently do.

Cheers

> 
> ?
> sage
> 
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: ceph-ci.git?

[Justin Erenkrantz]

On Mon, May 18, 2015 at 1:09 AM, Loic Dachary <loic@dachary.org> wrote:
> Maybe it would be better to keep http://github.com/ceph/ceph as it is and create a release repository (say https://git.ceph.com/?p=ceph-release.git) to only have the stable, master and next branches. Contributors who accidentally force push master on http://github.com/ceph/ceph would have the comfort of knowing that master on https://git.ceph.com/?p=ceph-release.git can be trusted to be a reference.

+1.  -- justin

Re: ceph-ci.git?

[John Spray]

On 18/05/2015 09:09, Loic Dachary wrote:
> One problem with promoting http://github.com/ceph/ceph to that role is that while the migration is in progress there will be two queues of pull requests, the new one at http://github.com/ceph-ci/ceph and the old one at http://github.com/ceph/ceph. Given our current flow of pull requests the migration will probably take about three months. And after that, since it's not possible to forbid pull requests on github.com (you can disable issues but not pull requests on a given repository) there will forever be a flow of misdirected pull requests against the old repository.
If I understand correctly, this is not an issue as pull requests would 
still be against the original ceph repository.  The ceph-ci repository 
would be where people could push their branches directly to have them 
built, and then they could create a PR to the ceph repository (either 
from their branch on ceph-ci or their original branch in their own 
github repo).

So my workflow as a developer would be to write some code locally, push 
my branch to /jcsp/ceph to "save" it, push it to /ceph/ceph-ci to have 
the gitbuilders pick it up and build it, and then ultimately make a PR 
to /ceph/ceph to merge it.

Apologies in advance for muddying the water if I've got any of that 
wrong :-)

Cheers,
John

Re: ceph-ci.git?

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 1627 bytes --]



On 18/05/2015 18:29, John Spray wrote:> 
> 
> On 18/05/2015 09:09, Loic Dachary wrote:
>> One problem with promoting http://github.com/ceph/ceph to that role is that while the migration is in progress there will be two queues of pull requests, the new one at http://github.com/ceph-ci/ceph and the old one at http://github.com/ceph/ceph. Given our current flow of pull requests the migration will probably take about three months. And after that, since it's not possible to forbid pull requests on github.com (you can disable issues but not pull requests on a given repository) there will forever be a flow of misdirected pull requests against the old repository.
> If I understand correctly, this is not an issue as pull requests would still be against the original ceph repository.  The ceph-ci repository would be where people could push their branches directly to have them built, and then they could create a PR to the ceph repository (either from their branch on ceph-ci or their original branch in their own github repo).
> 
> So my workflow as a developer would be to write some code locally, push my branch to /jcsp/ceph to "save" it, push it to /ceph/ceph-ci to have the gitbuilders pick it up and build it, and then ultimately make a PR to /ceph/ceph to merge it.
> 
> Apologies in advance for muddying the water if I've got any of that wrong :-)

I think you got it right, I misunderstood the original proposal. 

+1 on the original proposal then. It's less work than having a ceph-release repository. Sorry for the noise :-)

> 
> Cheers,
> John

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: ceph-ci.git?

[Joao Eduardo Luis]

On 18/05/15 05:40, Sage Weil wrote:
> As the number of people contributing code grows, we've added more and more
> people to the github ceph org who have write access to ceph.git.  Those
> people can merge pull requests and can also push branches directly to the
> repo.
>
> We also use ceph.git as a source for the test build infrastrucure
> (gitbuilders) to generate packages for QA or hot fixes and make check
> tests.  This leads to an every-growing body of wip-* branches in the repo
> (which is annoying), and also means that in order to build something to
> test in QA you also get the ability to (say) push directly to master.
>
> How about we instead
>
>   - create a second repo named something like ceph-ci.git (that's the best
> I can come up with at the moment)
>   - add this as a second source for all gitbuilders (they can poll a list)
>   - move all wip-* branches here
>   - create a new github team with contributing developers who can push to
> this repo and are trusted not to wreak havoc on the builders
>   - remove all the cruft from ceph.git, so that it's just master, next, the
> stable branches, release tags, and anything else similarly important.
>   - restrict ceph.git write access to core developers
>
> This will improve security somewhat and reduce the risk of an accidental
> push to an important branch.
>
> It may also reduce the risk associated with accidental force pushes
> (something we've hemmed and hawed about recently) by limiting the circle
> of people who can write to ceph.git and also changing workflows so that it
> is almost never used directly...

I think this is a great idea!

'ceph-ci' also looks like a simple enough name conveying its purpose, 
even though the 'continuous' part is not necessarily true (but it's 
trivial to type, so it gets my vote).

   -Joao

Re: ceph-ci.git?

[Sage Weil]

On Sat, 23 May 2015, Joao Eduardo Luis wrote:
> On 18/05/15 05:40, Sage Weil wrote:
> > As the number of people contributing code grows, we've added more and more
> > people to the github ceph org who have write access to ceph.git.  Those
> > people can merge pull requests and can also push branches directly to the
> > repo.
> > 
> > We also use ceph.git as a source for the test build infrastrucure
> > (gitbuilders) to generate packages for QA or hot fixes and make check
> > tests.  This leads to an every-growing body of wip-* branches in the repo
> > (which is annoying), and also means that in order to build something to
> > test in QA you also get the ability to (say) push directly to master.
> > 
> > How about we instead
> > 
> >   - create a second repo named something like ceph-ci.git (that's the best
> > I can come up with at the moment)
> >   - add this as a second source for all gitbuilders (they can poll a list)
> >   - move all wip-* branches here
> >   - create a new github team with contributing developers who can push to
> > this repo and are trusted not to wreak havoc on the builders
> >   - remove all the cruft from ceph.git, so that it's just master, next, the
> > stable branches, release tags, and anything else similarly important.
> >   - restrict ceph.git write access to core developers
> > 
> > This will improve security somewhat and reduce the risk of an accidental
> > push to an important branch.
> > 
> > It may also reduce the risk associated with accidental force pushes
> > (something we've hemmed and hawed about recently) by limiting the circle
> > of people who can write to ceph.git and also changing workflows so that it
> > is almost never used directly...
> 
> I think this is a great idea!
> 
> 'ceph-ci' also looks like a simple enough name conveying its purpose, even
> though the 'continuous' part is not necessarily true (but it's trivial to
> type, so it gets my vote).

I've created the ceph-gi repository.  I think the next step is to redo all 
the gitbuilders to pull form both sources.

	http://tracker.ceph.com/issues/11747

And then we'll need to change up the github groups and permissions

	http://tracker.ceph.com/issues/11748

sage