keep a subversion mirror in sync

keep a subversion mirror in sync

[Gaspard Bucher]

Hi !

I have been trying different strategies to keep subversion repository  
in sync with a git repository (on github).

Some details with pictures can be found here: http://zenadmin.org/en/documentation/page439.html 
.

I have tried to use a single database to pull/push from github and  
dcommit to the old subversion repository but I get lots of conflicts  
and duplicates (see link above).

I have then tried to create a "pull" only clone to dcommit, but it  
looses the commit messages and replaces then with a merge message. At  
least this solution keeps the github repository clean.

What is the canonical way to do this ?

Many thanks for your help.

Gaspard

Re: keep a subversion mirror in sync

[Avery Pennarun]

On Wed, Sep 17, 2008 at 7:58 AM, Gaspard Bucher <gaspard@teti.ch> wrote:
> I have been trying different strategies to keep subversion repository in
> sync with a git repository (on github).
>
> Some details with pictures can be found here:
> http://zenadmin.org/en/documentation/page439.html.
>
> I have tried to use a single database to pull/push from github and dcommit
> to the old subversion repository but I get lots of conflicts and duplicates
> (see link above).
>
> I have then tried to create a "pull" only clone to dcommit, but it looses
> the commit messages and replaces then with a merge message. At least this
> solution keeps the github repository clean.
>
> What is the canonical way to do this ?

The bad news is that a fully automatic two-way sync with svn is pretty
much impossible, AFAIK.  The act of committing to svn adds a
git-svn-id line to your commit, so your old commit object needs to be
thrown away and replaced with a new commit object.  This makes
everybody else's git commits totally confused.  It's possible that the
svn.noMetaData option will avoid this problem, but I've never tried
it, and it's dangerous: if you lose your svn rev-map database, you
can't get it back.

As for dcommits losing commit messages... yes.  There is no way to
represent the idea of parallel development in svn without creating
branches.  For example, if we fetch from svn at point A, and person #1
commits to svn based on point A producing point B, and person #2
commits to git based on point A producing point C, then in order to
have git-svn send point C back into svn, it will have to merge points
B and C.

Git can do the merge with no problem, but now it has to commit the
changes from the merged version into svn.  It doesn't make sense to
commit *each* of the commits from A to C, because it would have to do
it *on top of* B.  The best it can do it commit the entire set of
changes as a merge commit.

You have two options:

1) Manually git-rebase the changes from A..C onto B, then hurry up and
dcommit them before someone else commits to svn before you (otherwise
you'd have to rebase again).  git svn dcommit will then be smart
enough to commit each one separately.

2) Accept the merge commit.  The good news is that in the git repo,
history will still have all the interesting information; svn's history
isn't wrong, it's just incomplete.

You could also try to modify git-svn to implement a third solution:

3) Create a temporary branch in svn by copying commit A.  Commit each
of the changes from A..C in that branch one by one, then create a
merge back on the real branch that merges your temporary branch in,
using the commit message and files from your git merge commit.

Option #3 sounds hairy and gross, so it would be a lot of work and
there's no guarantee your patch would be accepted.

Basically, svn's limited ability to express history is preventing you
from committing your "full" git history back into svn.

At work, I currently use option #2.  As long as the full history is
available *somewhere*, the world is an okay place.

Have fun,

Avery

Re: keep a subversion mirror in sync

[Gaspard Bucher]

I am the only one writing to the subversion repository. It's just a  
public "read-only" mirror.

After I dcommit, I get a merge commit that's added to the master  
branch. The "origin/master" does not have this merge commit. When I  
pull from origin/master, I get a conflict
for all modified files. Reading the docs, it says "Running git-merge  
or git-pull is NOT recommended on a branch you plan to dcommit from".

I think I will use svk to handle this sync problem.

SETUP:
1. svk mirror svn://... //mirror/foo
2. svk sync //mirror/foo
3. svk co //mirror/foo
4. cd foo
5. svk ps svn:ignore ".git" .
6. cd ..
4. rm -rf foo
5. git clone git@... foo
6. cd foo

SYNC:
1. get the last commit message from git and use it to sync back
"git log -1 --pretty=short --no-color > /tmp/git_svk_msg && svk ci -- 
import -F /tmp/git_svk_msg && rm /tmp/git_svk_msg"

I have tried this setup and it works fine. The fact that svk and git  
totally ignore each other is really nice in this situation.

I thought there would be an easier path, but finally the solution here  
seems good, as long as the "master" branch does not contain uncommited  
content.

Gaspard

> On Wed, Sep 17, 2008 at 7:58 AM, Gaspard Bucher <gaspard@teti.ch>  
> wrote:
>> I have been trying different strategies to keep subversion  
>> repository in
>> sync with a git repository (on github).
>>
>> Some details with pictures can be found here:
>> http://zenadmin.org/en/documentation/page439.html.
>>
>> I have tried to use a single database to pull/push from github and  
>> dcommit
>> to the old subversion repository but I get lots of conflicts and  
>> duplicates
>> (see link above).
>>
>> I have then tried to create a "pull" only clone to dcommit, but it  
>> looses
>> the commit messages and replaces then with a merge message. At  
>> least this
>> solution keeps the github repository clean.
>>
>> What is the canonical way to do this ?
>
> The bad news is that a fully automatic two-way sync with svn is pretty
> much impossible, AFAIK.  The act of committing to svn adds a
> git-svn-id line to your commit, so your old commit object needs to be
> thrown away and replaced with a new commit object.  This makes
> everybody else's git commits totally confused.  It's possible that the
> svn.noMetaData option will avoid this problem, but I've never tried
> it, and it's dangerous: if you lose your svn rev-map database, you
> can't get it back.
>
> As for dcommits losing commit messages... yes.  There is no way to
> represent the idea of parallel development in svn without creating
> branches.  For example, if we fetch from svn at point A, and person #1
> commits to svn based on point A producing point B, and person #2
> commits to git based on point A producing point C, then in order to
> have git-svn send point C back into svn, it will have to merge points
> B and C.
>
> Git can do the merge with no problem, but now it has to commit the
> changes from the merged version into svn.  It doesn't make sense to
> commit *each* of the commits from A to C, because it would have to do
> it *on top of* B.  The best it can do it commit the entire set of
> changes as a merge commit.
>
> You have two options:
>
> 1) Manually git-rebase the changes from A..C onto B, then hurry up and
> dcommit them before someone else commits to svn before you (otherwise
> you'd have to rebase again).  git svn dcommit will then be smart
> enough to commit each one separately.
>
> 2) Accept the merge commit.  The good news is that in the git repo,
> history will still have all the interesting information; svn's history
> isn't wrong, it's just incomplete.
>
> You could also try to modify git-svn to implement a third solution:
>
> 3) Create a temporary branch in svn by copying commit A.  Commit each
> of the changes from A..C in that branch one by one, then create a
> merge back on the real branch that merges your temporary branch in,
> using the commit message and files from your git merge commit.
>
> Option #3 sounds hairy and gross, so it would be a lot of work and
> there's no guarantee your patch would be accepted.
>
> Basically, svn's limited ability to express history is preventing you
> from committing your "full" git history back into svn.
>
> At work, I currently use option #2.  As long as the full history is
> available *somewhere*, the world is an okay place.
>
> Have fun,
>
> Avery