* Coping with the pull-before-you-push model @ 2010-09-09 4:47 Joshua Jensen 2010-09-09 13:06 ` Ævar Arnfjörð Bjarmason 0 siblings, 1 reply; 14+ messages in thread From: Joshua Jensen @ 2010-09-09 4:47 UTC (permalink / raw) To: git@vger.kernel.org After a deployment of Git on a centralized server at my place of business, the largest amount of grumbling has been with the pull-before-you-push model. Coming from the file-centric Perforce where you need only have latest of just the files you are submitting, the pull-before-you-push model has really been a pain in the neck for a large team. Even with topic branches being used, merges to master occur frequently. It can really be a frustrating battle to get your merged branch pushed to the central master branch. In the time it took you to pull, test, and push, someone has probably already pushed before you. To cope with this, people will pull, not bother testing, and immediately push their changes. Yes, this could result in build instability, but it is considered better than never being able to make your change live. (Let's ignore what we should or shouldn't be doing as far as 'development practices'. :) We're solving the problems one step at a time...) Gerrit provides a compelling model where branches are pushed to the code review server in the form refs/for/master, and the given push will always succeed. Code reviews are performed, someone sets the verified bit, and the change is submitted and merged to master by Gerrit itself in a queued fashion. Unfortunately, its general "requirement" to squash your branch down to a single commit is, possibly, a showstopper. If it treated a branch merge as a group of commits that MUST stay together, that would be perfect. What other tools are out there that would let users successfully push their branch to the server (without having the HEAD master commit), and the push would be automatically merged to the master branch? Is there another workflow that is successful for your large(-ish) enterprise team? Thanks for your insights! Josh ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Coping with the pull-before-you-push model 2010-09-09 4:47 Coping with the pull-before-you-push model Joshua Jensen @ 2010-09-09 13:06 ` Ævar Arnfjörð Bjarmason 2010-09-09 14:43 ` Joshua Jensen 0 siblings, 1 reply; 14+ messages in thread From: Ævar Arnfjörð Bjarmason @ 2010-09-09 13:06 UTC (permalink / raw) To: Joshua Jensen; +Cc: git@vger.kernel.org On Thu, Sep 9, 2010 at 04:47, Joshua Jensen <jjensen@workspacewhiz.com> wrote: > After a deployment of Git on a centralized server at my place of business, > the largest amount of grumbling has been with the pull-before-you-push > model. Coming from the file-centric Perforce where you need only have > latest of just the files you are submitting, the pull-before-you-push model > has really been a pain in the neck for a large team. > > Even with topic branches being used, merges to master occur frequently. It > can really be a frustrating battle to get your merged branch pushed to the > central master branch. In the time it took you to pull, test, and push, > someone has probably already pushed before you. To cope with this, people > will pull, not bother testing, and immediately push their changes. Yes, > this could result in build instability, but it is considered better than > never being able to make your change live. > > (Let's ignore what we should or shouldn't be doing as far as 'development > practices'. :) We're solving the problems one step at a time...) Let's not ignore that. Presumably you had exactly the same problem in perforce, i.e. because you only had have the files you were changing checked out in Perforce in the time between `hack && pull && test && push` someone else might have already pushed. Thus what you just submitted wasn't guaranteed to pass tests. So is the flow in Git where you don't run the tests again, rebase and push and hope for the best any different? > Gerrit provides a compelling model where branches are pushed to the code > review server in the form refs/for/master, and the given push will always > succeed. Code reviews are performed, someone sets the verified bit, and the > change is submitted and merged to master by Gerrit itself in a queued > fashion. Unfortunately, its general "requirement" to squash your branch > down to a single commit is, possibly, a showstopper. If it treated a branch > merge as a group of commits that MUST stay together, that would be perfect. This sounds like something that's configurable in Gerrit, or should be. > [..] > > Is there another workflow that is successful for your large(-ish) enterprise > team? Linux manages to deal with a huge number of commits, but does so by having subsystems. Maybe that's something you can use in your codebase? ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Coping with the pull-before-you-push model 2010-09-09 13:06 ` Ævar Arnfjörð Bjarmason @ 2010-09-09 14:43 ` Joshua Jensen 2010-09-10 5:35 ` Jon Seymour 0 siblings, 1 reply; 14+ messages in thread From: Joshua Jensen @ 2010-09-09 14:43 UTC (permalink / raw) To: Ævar Arnfjörð Bjarmason; +Cc: git@vger.kernel.org ----- Original Message ----- From: Ævar Arnfjörð Bjarmason Date: 9/9/2010 7:06 AM > On Thu, Sep 9, 2010 at 04:47, Joshua Jensen<jjensen@workspacewhiz.com> wrote: >> After a deployment of Git on a centralized server at my place of business, >> the largest amount of grumbling has been with the pull-before-you-push >> model. Coming from the file-centric Perforce where you need only have >> latest of just the files you are submitting, the pull-before-you-push model >> has really been a pain in the neck for a large team. >> >> Even with topic branches being used, merges to master occur frequently. It >> can really be a frustrating battle to get your merged branch pushed to the >> central master branch. In the time it took you to pull, test, and push, >> someone has probably already pushed before you. To cope with this, people >> will pull, not bother testing, and immediately push their changes. Yes, >> this could result in build instability, but it is considered better than >> never being able to make your change live. >> >> (Let's ignore what we should or shouldn't be doing as far as 'development >> practices'. :) We're solving the problems one step at a time...) > Let's not ignore that. Fair. > Presumably you had exactly the same problem in perforce, i.e. because > you only had have the files you were changing checked out in Perforce > in the time between `hack&& pull&& test&& push` someone else might > have already pushed. Thus what you just submitted wasn't guaranteed to > pass tests. > > So is the flow in Git where you don't run the tests again, rebase and > push and hope for the best any different? The end result is the same; submitted code is never really tested against latest in Perforce either. The primary difference between the two is that the Perforce submit is successful the majority of the time (odds of someone editing and submitting the same file as you are low), and the Git push fails the majority of the time. Don't get me wrong. I've given training on why Git's enforced pull-before-you-push model can be better than what we had before (reproducible state, fewer broken builds, etc). Nevertheless, the issue is very frequent, and that's why I am querying others. >> Gerrit provides a compelling model where branches are pushed to the code >> review server in the form refs/for/master, and the given push will always >> succeed. Code reviews are performed, someone sets the verified bit, and the >> change is submitted and merged to master by Gerrit itself in a queued >> fashion. Unfortunately, its general "requirement" to squash your branch >> down to a single commit is, possibly, a showstopper. If it treated a branch >> merge as a group of commits that MUST stay together, that would be perfect. > This sounds like something that's configurable in Gerrit, or should > be. Agreed, but it appears it is currently a missing feature. There have been discussions about it on their mailing list over the past months, and a feature request is in their tracker. I am unsure of their progress or even interest. >> [..] >> >> Is there another workflow that is successful for your large(-ish) enterprise >> team? > Linux manages to deal with a huge number of commits, but does so by > having subsystems. > > Maybe that's something you can use in your codebase? Unless I'm mistaken, though, it seems to work in reverse of what our working model has been for years. I'm grossly oversimplifying the process, but the Linux model seems to be built on hierarchical 'pull requests'. I can tell a subsystem maintainer I have some changes and then ask that maintainer to pull them from a certain location. When that person has time/inclination, the change is pulled, merged in, and then another pull request is sent to the upstream hierarchy. This _is_ compelling, but even if it would work within the company I work for, it is such a dramatic shift in workflow that I am certain it could not be done in one fell swoop. Thanks for your insights! Josh ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Coping with the pull-before-you-push model 2010-09-09 14:43 ` Joshua Jensen @ 2010-09-10 5:35 ` Jon Seymour 2010-09-10 14:15 ` Jeff King 2010-09-14 4:37 ` Joshua Jensen 0 siblings, 2 replies; 14+ messages in thread From: Jon Seymour @ 2010-09-10 5:35 UTC (permalink / raw) To: Joshua Jensen; +Cc: Ævar Arnfjörð Bjarmason, git@vger.kernel.org On Fri, Sep 10, 2010 at 12:43 AM, Joshua Jensen <jjensen@workspacewhiz.com> wrote:>> >> Presumably you had exactly the same problem in perforce, i.e. because >> you only had have the files you were changing checked out in Perforce >> in the time between `hack&& pull&& test&& push` someone else might >> have already pushed. Thus what you just submitted wasn't guaranteed to >> pass tests. >> >> So is the flow in Git where you don't run the tests again, rebase and >> push and hope for the best any different? > > The end result is the same; submitted code is never really tested against > latest in Perforce either. The primary difference between the two is that > the Perforce submit is successful the majority of the time (odds of someone > editing and submitting the same file as you are low), and the Git push fails > the majority of the time. > > Don't get me wrong. I've given training on why Git's enforced > pull-before-you-push model can be better than what we had before > (reproducible state, fewer broken builds, etc). Nevertheless, the issue is > very frequent, and that's why I am querying others. I am not sure that git does enforce a pull before push model, except if you attempt to use git without the equivalent of a maintainer - in essence devolving responsibility of maintaining the tip of the shared branch to the whole team. This is probably the cultural shift that is hardest for enterprises to accept - why do I need a _person_ to do this _manual_ work when tools like {insert favourite non-DVCS here} can do this for me? To management, this looks like a step-backwards. In our case, the team leaders act like the subsystem leads and the build team acts like the maintainer. Team leads have a more review oriented focus, where as the build team doesn't understand the code that well and will reject merge conflicts if they are non-trivial. Their focus is just to keep the daily build rhythm going. If you don't have a maintainer role, then everyone is obliged to pull, test, pull-again, then push because there is no-one else there to do it and no other time to do it. The delays inherent in the test process inevitably mean you have to cycle several times until your commit wins. But with a maintainer role on your project, you don't need to do this. Everyone develops on yesterday's baseline and verifies that they don't regress anything with respect to that baseline - there is no requirement to certify against the bleeding edge of the integration stream - that's the role of the maintainer/build team once the daily build is done, thereby amortising test execution effort across the whole team. > > I'm grossly oversimplifying the process, but the Linux model seems to be > built on hierarchical 'pull requests'. I can tell a subsystem maintainer I > have some changes and then ask that maintainer to pull them from a certain > location. When that person has time/inclination, the change is pulled, > merged in, and then another pull request is sent to the upstream hierarchy. > Note that one difference between the maintainer and maintainer-less model is that the tip of the reference branch is published relatively rarely - it is quite stable. This is quite helpful because the baseline doesn't keep shifting every 30 minutes. In the maintainer-less model, you never have stability, nor any opportunity to acquire it. > This _is_ compelling, but even if it would work within the company I work > for, it is such a dramatic shift in workflow that I am certain it could not > be done in one fell swoo > The big difference between commercial work and open source projects like git and linux is that the latter have just one constraint - quality whereas the former also have budgets and schedules to worry about. Unintegrated crap code has cost the open source project almost nothing. Commercial enterprises, on the other hand, pay lots of money for people to develop code whether it is crap or otherwise. The idea of leaving expensive code unintegrated causes management paroxysms of concern that are hard to ignore - a tool that blindly integrates crap code automatically is a soothing balm for such people. Hence the resistance to tools like git that encourage a maintainer role and, implicit in that, the possibility of review that might result in crap code being exposed for what it is. jon seymour. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Coping with the pull-before-you-push model 2010-09-10 5:35 ` Jon Seymour @ 2010-09-10 14:15 ` Jeff King 2010-09-14 4:47 ` Joshua Jensen 2010-09-14 4:37 ` Joshua Jensen 1 sibling, 1 reply; 14+ messages in thread From: Jeff King @ 2010-09-10 14:15 UTC (permalink / raw) To: Jon Seymour Cc: Joshua Jensen, Ævar Arnfjörð Bjarmason, git@vger.kernel.org On Fri, Sep 10, 2010 at 03:35:21PM +1000, Jon Seymour wrote: > This is probably the cultural shift that is hardest for enterprises to > accept - why do I need a _person_ to do this _manual_ work when tools > like {insert favourite non-DVCS here} can do this for me? To > management, this looks like a step-backwards. Bear in mind that you can still shift to a maintainer model, but keep the maintainer automated. That is, you can queue up "to-pull" heads, and then have an automated process pull them one by one and do some basic QA (does it merge, does it build, does it pass automated tests, etc). Which is not that different from what many shops do in the non-maintainer model, except that when you break the build, the maintainer process notices _before_ publishing the merged tip, so everybody won't try to build on your broken crap. I seem to recall that Gerrit does something like this, but I may be mis-remembering. I haven't actually used it for real work. I still prefer a human maintainer, because they can do things like reorder the queue manually (or outright reject flaky topics) to get more sensible merges, or do easy but non-trivial merges themselves to avoid kicking code back to the developer. -Peff ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Coping with the pull-before-you-push model 2010-09-10 14:15 ` Jeff King @ 2010-09-14 4:47 ` Joshua Jensen 2010-09-14 5:24 ` Jeff King 2010-09-14 12:12 ` Theodore Tso 0 siblings, 2 replies; 14+ messages in thread From: Joshua Jensen @ 2010-09-14 4:47 UTC (permalink / raw) To: Jeff King Cc: Jon Seymour, Ævar Arnfjörð Bjarmason, git@vger.kernel.org ----- Original Message ----- From: Jeff King Date: 9/10/2010 8:15 AM > On Fri, Sep 10, 2010 at 03:35:21PM +1000, Jon Seymour wrote: > >> This is probably the cultural shift that is hardest for enterprises to >> accept - why do I need a _person_ to do this _manual_ work when tools >> like {insert favourite non-DVCS here} can do this for me? To >> management, this looks like a step-backwards. > Bear in mind that you can still shift to a maintainer model, but keep > the maintainer automated. That is, you can queue up "to-pull" heads, and > then have an automated process pull them one by one and do some basic QA > (does it merge, does it build, does it pass automated tests, etc). Which > is not that different from what many shops do in the non-maintainer > model, except that when you break the build, the maintainer process > notices _before_ publishing the merged tip, so everybody won't try to > build on your broken crap. > Do you know of any existing software that does this? This may be ideal in the short term. > I still prefer a human maintainer, because they can do things like > reorder the queue manually (or outright reject flaky topics) to get more > sensible merges, or do easy but non-trivial merges themselves to avoid > kicking code back to the developer. > I agree with you concerning how valuable a human maintainer is. Josh ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Coping with the pull-before-you-push model 2010-09-14 4:47 ` Joshua Jensen @ 2010-09-14 5:24 ` Jeff King 2010-09-14 5:59 ` Avery Pennarun 2010-09-15 21:59 ` David Brown 2010-09-14 12:12 ` Theodore Tso 1 sibling, 2 replies; 14+ messages in thread From: Jeff King @ 2010-09-14 5:24 UTC (permalink / raw) To: Joshua Jensen Cc: Avery Pennarun, Shawn Pearce, Jon Seymour, Ævar Arnfjörð Bjarmason, git@vger.kernel.org On Mon, Sep 13, 2010 at 10:47:30PM -0600, Joshua Jensen wrote: > >Bear in mind that you can still shift to a maintainer model, but keep > >the maintainer automated. That is, you can queue up "to-pull" heads, and > >then have an automated process pull them one by one and do some basic QA > >(does it merge, does it build, does it pass automated tests, etc). Which > >is not that different from what many shops do in the non-maintainer > >model, except that when you break the build, the maintainer process > >notices _before_ publishing the merged tip, so everybody won't try to > >build on your broken crap. > > > Do you know of any existing software that does this? This may be > ideal in the short term. I think that Avery Pennarun's gitbuilder may do what you want: http://github.com/apenwarr/gitbuilder/ but I've never used it. I seem to recall from one of Shawn's presentations on Gerrit Code Review that it does something like this, too, but I can't seem to find any docs about it in my brief search: http://code.google.com/p/gerrit/ It may be that Gerrit doesn't handle building itself, but that the Android project is running something alongside it. Shawn may be able to say more. Basically, what we are talking about is continuous integration, with the slight twist that instead of developers pushing commits to a mainline branch which is built and tested, we would build and test their commits and then merge them to the mainline branch. Systems like Hudson that do continuous integration and support git may handle a workflow like this, but I don't know (I've only ever used Hudson in the everything-goes-to-svn-trunk model). There is also a small continuous integration system that lives in the contrib/continuous directory of git.git itself. It's quite old at this point, but I wouldn't be surprised if its descendant is what Gerrit uses. -Peff ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Coping with the pull-before-you-push model 2010-09-14 5:24 ` Jeff King @ 2010-09-14 5:59 ` Avery Pennarun 2010-09-15 21:59 ` David Brown 1 sibling, 0 replies; 14+ messages in thread From: Avery Pennarun @ 2010-09-14 5:59 UTC (permalink / raw) To: Jeff King Cc: Joshua Jensen, Shawn Pearce, Jon Seymour, Ævar Arnfjörð, git@vger.kernel.org On Mon, Sep 13, 2010 at 10:24 PM, Jeff King <peff@peff.net> wrote: > On Mon, Sep 13, 2010 at 10:47:30PM -0600, Joshua Jensen wrote: >> >Bear in mind that you can still shift to a maintainer model, but keep >> >the maintainer automated. That is, you can queue up "to-pull" heads, and >> >then have an automated process pull them one by one and do some basic QA >> >(does it merge, does it build, does it pass automated tests, etc). Which >> >is not that different from what many shops do in the non-maintainer >> >model, except that when you break the build, the maintainer process >> >notices _before_ publishing the merged tip, so everybody won't try to >> >build on your broken crap. >> >> Do you know of any existing software that does this? This may be >> ideal in the short term. > > I think that Avery Pennarun's gitbuilder may do what you want: > > http://github.com/apenwarr/gitbuilder/ > > but I've never used it. gitbuilder doesn't do this. However, it does let you automatically do user-defined things for each branch, and it stores the result for each one. So it would be a pretty short trip from there to auto-merging when a branch is ready. Have fun, Avery ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Coping with the pull-before-you-push model 2010-09-14 5:24 ` Jeff King 2010-09-14 5:59 ` Avery Pennarun @ 2010-09-15 21:59 ` David Brown 1 sibling, 0 replies; 14+ messages in thread From: David Brown @ 2010-09-15 21:59 UTC (permalink / raw) To: Jeff King Cc: Joshua Jensen, Avery Pennarun, Shawn Pearce, Jon Seymour, Ævar Arnfjörð Bjarmason, git@vger.kernel.org On Tue, Sep 14, 2010 at 01:24:51AM -0400, Jeff King wrote: > I seem to recall from one of Shawn's presentations on Gerrit Code Review > that it does something like this, too, but I can't seem to find any docs > about it in my brief search: > > http://code.google.com/p/gerrit/ > > It may be that Gerrit doesn't handle building itself, but that the > Android project is running something alongside it. Shawn may be able to > say more. > > Basically, what we are talking about is continuous integration, with the > slight twist that instead of developers pushing commits to a mainline > branch which is built and tested, we would build and test their commits > and then merge them to the mainline branch. Gerrit doesn't do the build and test, but it isn't all that difficult to hook into. It also allows the results of the build and test to record their state so that the developer can track what is happening. The nice part about it, compared to other CI systems I've seen is that it catches the problems before they are merged into master, rather than after. Internally, we actually do multiple levels of a CI-type thing. Every time a developer uploads a change, one machine performs a sanity build on it (with multiple configurations). These results are visible in Gerrit, and it keeps people from putting too much effort into reviewing code that doesn't even compile. Once the code has gotten through two levels of code review, a more throughough build and test system pulls the whole project (from numerous git repos) builds and runs tests. This takes long enough that it doesn't do individual changes, so failures take work to track down, but it does generally assure that the result of each 'git merge' somewhat works. The only real annoying part I've found with the gerrit model is that the tree is filled with lots of merge comments (generally a merge for every real commit). The other option is to let Gerrit rebase, which then gets annoying when a developer has pulled changes from other developers. It also has a nice link to a 'git pull' command to pull down an individual change. David ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Coping with the pull-before-you-push model 2010-09-14 4:47 ` Joshua Jensen 2010-09-14 5:24 ` Jeff King @ 2010-09-14 12:12 ` Theodore Tso 2010-09-14 15:51 ` Joshua Jensen 1 sibling, 1 reply; 14+ messages in thread From: Theodore Tso @ 2010-09-14 12:12 UTC (permalink / raw) To: Joshua Jensen Cc: Jeff King, Jon Seymour, Ævar Arnfjörð Bjarmason, git@vger.kernel.org On Sep 14, 2010, at 12:47 AM, Joshua Jensen wrote: >>> management, this looks like a step-backwards. >> Bear in mind that you can still shift to a maintainer model, but keep >> the maintainer automated. That is, you can queue up "to-pull" heads, and >> then have an automated process pull them one by one and do some basic QA >> (does it merge, does it build, does it pass automated tests, etc). Which >> is not that different from what many shops do in the non-maintainer >> model, except that when you break the build, the maintainer process >> notices _before_ publishing the merged tip, so everybody won't try to >> build on your broken crap. >> > Do you know of any existing software that does this? This may be ideal in the short term. Our workflow at $WORK involves pushing changes to gerrit to various "effort branches", and then once they are approved, we have a "Mergitator" script that will attempt to merge the effort branch with the merged master branch, and then attempt to do a build. If the build succeeds, then the changes will get pushed back to the publically visible merged master branch, and then the Mergitator will move on to the next effort branch that requires merging. If there is a merge conflict, the Mergitator will refuse the merge, and then give instructions on how to fix up the tree to avoid merge conflicts. The Mergitator code hasn't been released, and I suspect the main reason is that there's relatively little code that could be used outside of our environment, and a large amount of code which contains lots of details about our internal build system that would have to be stripped out and generalized before it could be released --- and no one has time to do it. So this probably doesn't help you since I suspect you meant to ask the question, "do you know of any existing publically available software", but I can tell you that it certainly can be done, and that software exists. Making it be software which is useful and usable to you would definitely take more work... -- Ted ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Coping with the pull-before-you-push model 2010-09-14 12:12 ` Theodore Tso @ 2010-09-14 15:51 ` Joshua Jensen 2010-09-14 16:24 ` Eugene Sajine 2010-09-14 16:49 ` Ted Ts'o 0 siblings, 2 replies; 14+ messages in thread From: Joshua Jensen @ 2010-09-14 15:51 UTC (permalink / raw) To: Theodore Tso Cc: Jeff King, Jon Seymour, Ævar Arnfjörð Bjarmason, git@vger.kernel.org ----- Original Message ----- From: Theodore Tso Date: 9/14/2010 6:12 AM > On Sep 14, 2010, at 12:47 AM, Joshua Jensen wrote: >>> Bear in mind that you can still shift to a maintainer model, but keep >>> the maintainer automated. That is, you can queue up "to-pull" heads, and >>> then have an automated process pull them one by one and do some basic QA >>> (does it merge, does it build, does it pass automated tests, etc). Which >> Do you know of any existing software that does this? This may be ideal in the short ter > Our workflow at $WORK involves pushing changes to gerrit to various "effort branches", and then once they are approved, we have a "Mergitator" script that will attempt to merge the effort branch with the merged master branch, and then attempt to do a build. If the build succeeds, then the changes will get pushed back to the publically visible merged master branch, and then the Mergitator will move on to the next effort branch that requires merging. If there is a merge conflict, the Mergitator will refuse the merge, and then give instructions on how to fix up the tree to avoid merge conflicts. > How does the integration with Gerrit work here? The only thing that comes to mind is that you do something like: git push gerrit HEAD:refs/for/merged-master Then the approvals are done. Afterward, Gerrit merges to the merged-master branch. I would suppose an external script is performing regular fetches. When it sees new code, it builds. No, this can't be right, but I'll leave my incorrect workflow here. > So this probably doesn't help you since I suspect you meant to ask the question, "do you know of any existing publically available software", but I can tell you that it certainly can be done, and that software exists. Making it be software which is useful and usable to you would definitely take more work... It's the branch queueing issue that is my current hang up. Gerrit's method is slick, but that won't work outside of JGit. I'm not opposed to JGit; I just haven't touched Java in years. So, perhaps, a web interface where the branch owner selects the (pushed to central server) branch name that is ready to go. A merge is attempted. If it succeeds, great. If it fails, then the merge is reset? Josh ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Coping with the pull-before-you-push model 2010-09-14 15:51 ` Joshua Jensen @ 2010-09-14 16:24 ` Eugene Sajine 2010-09-14 16:49 ` Ted Ts'o 1 sibling, 0 replies; 14+ messages in thread From: Eugene Sajine @ 2010-09-14 16:24 UTC (permalink / raw) To: Joshua Jensen Cc: Theodore Tso, Jeff King, Jon Seymour, Ævar Arnfjörð Bjarmason, git@vger.kernel.org On Tue, Sep 14, 2010 at 11:51 AM, Joshua Jensen <jjensen@workspacewhiz.com> wrote: > ----- Original Message ----- > From: Theodore Tso > Date: 9/14/2010 6:12 AM >> >> On Sep 14, 2010, at 12:47 AM, Joshua Jensen wrote: >>>> >>>> Bear in mind that you can still shift to a maintainer model, but keep >>>> the maintainer automated. That is, you can queue up "to-pull" heads, and >>>> then have an automated process pull them one by one and do some basic QA >>>> (does it merge, does it build, does it pass automated tests, etc). Which >>> >>> Do you know of any existing software that does this? This may be ideal >>> in the short ter >> >> Our workflow at $WORK involves pushing changes to gerrit to various >> "effort branches", and then once they are approved, we have a "Mergitator" >> script that will attempt to merge the effort branch with the merged master >> branch, and then attempt to do a build. If the build succeeds, then the >> changes will get pushed back to the publically visible merged master branch, >> and then the Mergitator will move on to the next effort branch that requires >> merging. If there is a merge conflict, the Mergitator will refuse the >> merge, and then give instructions on how to fix up the tree to avoid merge >> conflicts. >> > How does the integration with Gerrit work here? The only thing that comes > to mind is that you do something like: > > git push gerrit HEAD:refs/for/merged-master > > Then the approvals are done. Afterward, Gerrit merges to the merged-master > branch. > > I would suppose an external script is performing regular fetches. When it > sees new code, it builds. > > No, this can't be right, but I'll leave my incorrect workflow here. >> >> So this probably doesn't help you since I suspect you meant to ask the >> question, "do you know of any existing publically available software", but I >> can tell you that it certainly can be done, and that software exists. >> Making it be software which is useful and usable to you would definitely >> take more work... > > It's the branch queueing issue that is my current hang up. Gerrit's method > is slick, but that won't work outside of JGit. I'm not opposed to JGit; I > just haven't touched Java in years. > > So, perhaps, a web interface where the branch owner selects the (pushed to > central server) branch name that is ready to go. A merge is attempted. If > it succeeds, great. If it fails, then the merge is reset? > > Josh > -- > To unsubscribe from this list: send the line "unsubscribe git" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > In our case we have moved from CVS to git and had pretty much same problems. The only difference was that we are working with java and trying to have every project in separate repository. So we don't have many commits into common codebase. We have a server keeping bare mainline repos. In order to integrate the project with others we are using Hudson CI which performs builds upon each change detected on the server side (i.e. mainline repository) and also builds the projects that are dependent on the one you just changed. Those dependencies can be described inside each hudson job, but we are using Ivy for dependency management. If you have good testing infrastructure that is even better. So generally the amount of contributors to one project is pretty small. Some projects are solo, some have small teams coworking on one project code base. We are using two main branch model. The development goes in topic branches. The qa branch is a staging branch for the changes to be accumulated for the release. People are using direct pushes to qa branch to integrate or a maintainer. Hudson builds this codebase. When all release functionality is done the person responsible for the release is merging qa branch into the master, which gets released (this must be a fast-forward). So, i would say component oriented approach is the key here. HTH, Eugene ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Coping with the pull-before-you-push model 2010-09-14 15:51 ` Joshua Jensen 2010-09-14 16:24 ` Eugene Sajine @ 2010-09-14 16:49 ` Ted Ts'o 1 sibling, 0 replies; 14+ messages in thread From: Ted Ts'o @ 2010-09-14 16:49 UTC (permalink / raw) To: Joshua Jensen Cc: Jeff King, Jon Seymour, Ævar Arnfjörð Bjarmason, git@vger.kernel.org On Tue, Sep 14, 2010 at 09:51:22AM -0600, Joshua Jensen wrote: > How does the integration with Gerrit work here? The only thing that > comes to mind is that you do something like: > > git push gerrit HEAD:refs/for/merged-master So we'll push to something to, say, refs/heads/fs/ext4/for-merged on the gerrit server, and let gerrit do its thing. After a colleague approves all of the patches in the branch, gerrit will release the commits to the branch, and only then will the mergitator script attempt to do a trial merge of the effort branch with the merged/master branch. If the trial merge succeeds, then it will attempt to do a trial compile. If the trial compile succeeds then the merged/master branch will be updated with the commit id of the trial merge, and then the mergitator script will move on to the next effort branch will has been updated. If the mergitator fails to merge a particular branch, then an e-mail is sent out explanining the cause of the merge failure, so a human can fix things up. This could be done by cherry-picking a commit from merged/master which caused the merge conflict, and then fixing up the merge conflict, for example. A wise developer will do a trial merge on their own *before* submitting their effort branch to gerrit for code review, but this is not strictly speaking required. - Ted ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Coping with the pull-before-you-push model 2010-09-10 5:35 ` Jon Seymour 2010-09-10 14:15 ` Jeff King @ 2010-09-14 4:37 ` Joshua Jensen 1 sibling, 0 replies; 14+ messages in thread From: Joshua Jensen @ 2010-09-14 4:37 UTC (permalink / raw) To: Jon Seymour; +Cc: Ævar Arnfjörð Bjarmason, git@vger.kernel.org ----- Original Message ----- From: Jon Seymour Date: 9/9/2010 11:35 PM >> This _is_ compelling, but even if it would work within the company I work >> for, it is such a dramatic shift in workflow that I am certain it could not >> be done in one fell swo >> > The big difference between commercial work and open source projects > like git and linux is that the latter have just one constraint - > quality whereas the former also have budgets and schedules to worry > about. Unintegrated crap code has cost the open source project almost > nothing. Commercial enterprises, on the other hand, pay lots of money > for people to develop code whether it is crap or otherwise. The idea > of leaving expensive code unintegrated causes management paroxysms of > concern that are hard to ignore - a tool that blindly integrates crap > code automatically is a soothing balm for such people. Hence the > resistance to tools like git that encourage a maintainer role and, > implicit in that, the possibility of review that might result in crap > code being exposed for what it is. My apologies for the late response. I'm not ignoring this. I am trying to process it in the context of my organization as I establish the workflows we'll be using. I may have some follow up questions soon. Josh ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2010-09-15 22:00 UTC | newest] Thread overview: 14+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2010-09-09 4:47 Coping with the pull-before-you-push model Joshua Jensen 2010-09-09 13:06 ` Ævar Arnfjörð Bjarmason 2010-09-09 14:43 ` Joshua Jensen 2010-09-10 5:35 ` Jon Seymour 2010-09-10 14:15 ` Jeff King 2010-09-14 4:47 ` Joshua Jensen 2010-09-14 5:24 ` Jeff King 2010-09-14 5:59 ` Avery Pennarun 2010-09-15 21:59 ` David Brown 2010-09-14 12:12 ` Theodore Tso 2010-09-14 15:51 ` Joshua Jensen 2010-09-14 16:24 ` Eugene Sajine 2010-09-14 16:49 ` Ted Ts'o 2010-09-14 4:37 ` Joshua Jensen
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).