Automating svn<->git gateway

Automating svn<->git gateway

[Jan Hudec]

Hello Folks,

I want to set up a gateway between subversion and git, which would keep the
master synchonized with subversion trunk, both ways, and allow working with
any additional branches independent of subversion. For users it should behave
as any other shared git repository accessed by push and pull. And it needs to
be automatic.

Did anybody try to set up something like this?

Background:

At $work, we are considering switch from subversion to git. However to avoid
big disruptions in the work, we need to do it gradually. So the idea is to
switch to git one by one. The people who already switch need to be able to
test the final workflow with git, while other people still commit to the
subversion repository.

This basically rules out everybody just using git-svn, because individual
conversions are incompatible (or is there some way to make them compatible?),
so the people couldn't easily share their working branches.

That leaves me with creating one git-svn repository and having everybody
clone from that. Keeping the repository up-to-date from subversion side seems
trivial (just 'git svn fetch' to it from subversion's post-commit hook).

The trickier part is exporting changes pushed from the git side to
subversion. My plan is to write a post-receive hook, that will
'git svn dcommit' to svn trunk.

I suppose I will have to get the rewritten commit back from subversion and
merge it back to the master. I have not yet tested whether when dcommiting
a merge will properly keep the second parent in the rewritten commit or not.
I can do extra merge if it does not at the cost of slightly uglier history.

Thanks,
Jan

-- 
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

Re: Automating svn<->git gateway

[Joshua Shrader]

Check out the article and third-to-last comment (as of today) at
http://blog.emmanuelbernard.com/2010/05/git-how-my-life-has-improved-since-last-month-when-i-used-svn/comment-page-1/#comment-2248

The comment (by Josh) is mine, and details how I've approached this
problem.  It's worked very well for us, and follows from a workflow
presented in Jon Loeliger's "Version Control with Git."  I haven't
messed with any hooks, but I'd imagine that wouldn't be too difficult.
 Right now, we keep a couple legacy maintenance SVN branches sync'd up
with their corresponding Git branches.  The one thing that we haven't
attempted (and don't plan on) is creating a new branch in Git that we
want to duplicate in SVN.  We're only syncing branches that existed in
SVN prior to our move to Git.  Any new branches are "Git-only."

On Mon, Oct 11, 2010 at 3:30 PM, Jan Hudec <bulb@ucw.cz> wrote:
> Hello Folks,
>
> I want to set up a gateway between subversion and git, which would keep the
> master synchonized with subversion trunk, both ways, and allow working with
> any additional branches independent of subversion. For users it should behave
> as any other shared git repository accessed by push and pull. And it needs to
> be automatic.
>
> Did anybody try to set up something like this?
>
> Background:
>
> At $work, we are considering switch from subversion to git. However to avoid
> big disruptions in the work, we need to do it gradually. So the idea is to
> switch to git one by one. The people who already switch need to be able to
> test the final workflow with git, while other people still commit to the
> subversion repository.
>
> This basically rules out everybody just using git-svn, because individual
> conversions are incompatible (or is there some way to make them compatible?),
> so the people couldn't easily share their working branches.
>
> That leaves me with creating one git-svn repository and having everybody
> clone from that. Keeping the repository up-to-date from subversion side seems
> trivial (just 'git svn fetch' to it from subversion's post-commit hook).
>
> The trickier part is exporting changes pushed from the git side to
> subversion. My plan is to write a post-receive hook, that will
> 'git svn dcommit' to svn trunk.
>
> I suppose I will have to get the rewritten commit back from subversion and
> merge it back to the master. I have not yet tested whether when dcommiting
> a merge will properly keep the second parent in the rewritten commit or not.
> I can do extra merge if it does not at the cost of slightly uglier history.
>
> Thanks,
> Jan
>
> --
>                                                 Jan 'Bulb' Hudec <bulb@ucw.cz>
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Automating svn<->git gateway

[Jan Hudec]

On Mon, Oct 11, 2010 at 16:33:47 -0400, Joshua Shrader wrote:
> Check out the article and third-to-last comment (as of today) at
> http://blog.emmanuelbernard.com/2010/05/git-how-my-life-has-improved-since-last-month-when-i-used-svn/comment-page-1/#comment-2248
> 
> The comment (by Josh) is mine, and details how I've approached this
> problem.  It's worked very well for us, and follows from a workflow
> presented in Jon Loeliger's "Version Control with Git."  I haven't
> messed with any hooks, but I'd imagine that wouldn't be too difficult.
>  Right now, we keep a couple legacy maintenance SVN branches sync'd up
> with their corresponding Git branches.  The one thing that we haven't
> attempted (and don't plan on) is creating a new branch in Git that we
> want to duplicate in SVN.  We're only syncing branches that existed in
> SVN prior to our move to Git.  Any new branches are "Git-only."

Thanks. That's the workflow I had in mind. I already did something similar
ages ago with GNU arch and CVS, but the exact outline for git-svn will be
definitely helpful. Especially the exact order in which the merges need to be
done.

For my purpose I'll definitely need to automate it, because we often need to
see each other's changes quickly (often somebody needs a fix from somebody
else to be able to test something else or to base other changes on it). So
requiring to log in on the server and run the synchronization manually would
add too much overhead.

> On Mon, Oct 11, 2010 at 3:30 PM, Jan Hudec <bulb@ucw.cz> wrote:
> > Hello Folks,
> >
> > I want to set up a gateway between subversion and git, which would keep the
> > master synchonized with subversion trunk, both ways, and allow working with
> > any additional branches independent of subversion. For users it should behave
> > as any other shared git repository accessed by push and pull. And it needs to
> > be automatic.
> >
> > Did anybody try to set up something like this?
-- 
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

Re: Automating svn<->git gateway

[Jakub Narebski]

Jan Hudec <bulb@ucw.cz> writes:

> I want to set up a gateway between subversion and git, which would keep the
> master synchonized with subversion trunk, both ways, and allow working with
> any additional branches independent of subversion. For users it should behave
> as any other shared git repository accessed by push and pull. And it needs to
> be automatic.
> 
> Did anybody try to set up something like this?

It is a pity that "Native SVN support in git" project at Google Summer of
Code 2010 (https://git.wiki.kernel.org/index.php/SoC2010Projects) turned
out to be too much for one summer; though people are working on it.  It
would be perfect for this situation.

-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: Automating svn<->git gateway

[Jan Hudec]

On Tue, Oct 12, 2010 at 12:54:54 -0700, Jakub Narebski wrote:
> Jan Hudec <bulb@ucw.cz> writes:
> 
> > I want to set up a gateway between subversion and git, which would keep the
> > master synchonized with subversion trunk, both ways, and allow working with
> > any additional branches independent of subversion. For users it should behave
> > as any other shared git repository accessed by push and pull. And it needs to
> > be automatic.
> > 
> > Did anybody try to set up something like this?
> 
> It is a pity that "Native SVN support in git" project at Google Summer of
> Code 2010 (https://git.wiki.kernel.org/index.php/SoC2010Projects) turned
> out to be too much for one summer; though people are working on it.  It
> would be perfect for this situation.

I am not actually sure it would. What I want is that for people who already
switch to git may immediately stop caring about subversion. From their point
of view, it shouldn't be there.

So while I'm sure the native SVN support will solve the quirks and bugs of
git-svn, it will not do away with need for the gateway repository that will
somehow synchronize itself with subversion.

-- 
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

Re: Automating svn<->git gateway

[Jakub Narebski]

On wtorek 12. października 2010 22:31, Jan Hudec napisał:
> On Tue, Oct 12, 2010 at 12:54:54 -0700, Jakub Narebski wrote:
> > Jan Hudec <bulb@ucw.cz> writes:
> > 
> > > I want to set up a gateway between subversion and git, which would keep the
> > > master synchonized with subversion trunk, both ways, and allow working with
> > > any additional branches independent of subversion. For users it should behave
> > > as any other shared git repository accessed by push and pull. And it needs to
> > > be automatic.
> > > 
> > > Did anybody try to set up something like this?
> > 
> > It is a pity that "Native SVN support in git" project at Google Summer of
> > Code 2010 (https://git.wiki.kernel.org/index.php/SoC2010Projects) turned
> > out to be too much for one summer; though people are working on it.  It
> > would be perfect for this situation.
> 
> I am not actually sure it would. What I want is that for people who already
> switch to git may immediately stop caring about subversion. From their point
> of view, it shouldn't be there.
> 
> So while I'm sure the native SVN support will solve the quirks and bugs of
> git-svn, it will not do away with need for the gateway repository that will
> somehow synchronize itself with subversion.

Well, I think that native SVN support would allow to treat subversion
repository as one of repositories in the network of repositories.  Those
repositories could be set in that pushing to central git repository pushes
also to subversion repository, and like central git repository fetches
from leaf repositories, it would fetch from subversion repository.

-- 
Jakub Narebski
Poland

Re: Automating svn<->git gateway

[Jan Hudec]

On Wed, Oct 13, 2010 at 01:25:48 +0200, Jakub Narebski wrote:
> On wtorek 12. października 2010 22:31, Jan Hudec napisał:
> > So while I'm sure the native SVN support will solve the quirks and bugs of
> > git-svn, it will not do away with need for the gateway repository that will
> > somehow synchronize itself with subversion.
> 
> Well, I think that native SVN support would allow to treat subversion
> repository as one of repositories in the network of repositories.  Those
> repositories could be set in that pushing to central git repository pushes
> also to subversion repository, and like central git repository fetches
> from leaf repositories, it would fetch from subversion repository.

Yes. Except I don't know how to do the "pushing to central git repository
pushes also to..." part. Subversion or not.

Though, I guess if it worked well enough to preserve the merge commit (i.e.
when I push and pull, I see the commit I pushed, not any kind of rewrite),
pushing in update-hook would be quite many bits simpler.

-- 
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

Re: Automating svn<->git gateway

[Jakub Narebski]

On Wed, 13 Oct 2010, Jan Hudec wrote:
> On Wed, Oct 13, 2010 at 01:25:48 +0200, Jakub Narebski wrote:
> > On wtorek 12. października 2010 22:31, Jan Hudec napisał:

> > > So while I'm sure the native SVN support will solve the quirks and bugs of
> > > git-svn, it will not do away with need for the gateway repository that will
> > > somehow synchronize itself with subversion.
> > 
> > Well, I think that native SVN support would allow to treat subversion
> > repository as one of repositories in the network of repositories.  Those
> > repositories could be set in that pushing to central git repository pushes
> > also to subversion repository, and like central git repository fetches
> > from leaf repositories, it would fetch from subversion repository.
> 
> Yes. Except I don't know how to do the "pushing to central git repository
> pushes also to..." part. Subversion or not.
> 
> Though, I guess if it worked well enough to preserve the merge commit (i.e.
> when I push and pull, I see the commit I pushed, not any kind of rewrite),
> pushing in update-hook would be quite many bits simpler.

I was thinking about setting up appropriate hook for that...

...but it might have been also done "manually", i.e. by having maintainer[*]
do something like 'git pushall' to push to all distribution points (public
git repositories), including Subversion repository, when pushing from 
his/her private repository to public repository/repositories.  Maintainer
would fetch (pull) from all leaf repositories, including Subversion
repository.  This means that SVN repository is both leaf and distribution
point.

[*] I wonder if "The surgical team" idea (less known than "The mythical
    man-month" aka 'assigning more programmers to a project running behind
    schedule will make it even later') from seminal work by Fred Brooks'
    "The Mythical Man-Month" is still valid.  The 'surgeon' would be the
    maintainer / code reviewer, and would be responsible for merging in
    code from other developers.

-- 
Jakub Narebski
Poland