Three issues from a Subversion-to-git migration

Three issues from a Subversion-to-git migration

[Eric Raymond]

I joined the git list a few hours ago because of experiences I had
while migrating one of my projects, GPSD, from Subversion to git.
GPSD haa a logical but unusual use case for distributed version
control; the best places to test GPS sensors are out-of-doors in cars
and other places where wire-line Internet is absent and one may well
be out of range of a WAP.

In the course of the migration, I encountered three issues which I
think could be addressed with a relatively small amount of work.

1. The git hook scripts for CIA.vc are broken (in a way that is,
fortunately, easy to fix) and generally seem to be in an unmaintained,
dusty state.

2. The git-svn migration logic does not handle unmodified SVN tag
trees well.

3. I'm prone to typos, so I quickly noticed that the existing
facilities for editing comments in commits (before they're pushed)
are clumsy and dangerous.

I will address these points in detail in separate mails.  Issues 1 and
3 I can fix with working code.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Three issues from a Subversion-to-git migration

[Thomas Rast]

Eric Raymond wrote:
> 2. The git-svn migration logic does not handle unmodified SVN tag
> trees well.

The problem here is that git-svn is designed to handle incremental
updates, where it can't know whether some insane SVN user decides to
modify the tag later on.

I've used the following hack to make real tags out of SVN "tags":

git for-each-ref --format="%(refname)" refs/remotes/tags/ |
while read tag; do
    GIT_COMMITTER_DATE="$(git log -1 --pretty=format:"%ad" "$tag")" \
    GIT_COMMITTER_EMAIL="$(git log -1 --pretty=format:"%ce" "$tag")" \
    GIT_COMMITTER_NAME="$(git log -1 --pretty=format:"%cn" "$tag")" \
    git tag -m "$(git log -1 --pretty=format:"%s%n%b" "$tag")" \
    "${tag#refs/remotes/tags/}" "$tag"
done

Disclaimer: it worked last time I used it.  Haven't checked if it got
dusty since.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch

Re: Three issues from a Subversion-to-git migration

[Eric Raymond]

Thomas Rast <trast@student.ethz.ch>:
> Eric Raymond wrote:
> > 2. The git-svn migration logic does not handle unmodified SVN tag
> > trees well.
> 
> The problem here is that git-svn is designed to handle incremental
> updates, where it can't know whether some insane SVN user decides to
> modify the tag later on.

Yes.  Ideally, I suppose, git-svn (or whatever replaces it) would have
behavior something like this:

1. Turn unmodified tag directories into git tags
2. Turn odified tags into branches.
3. Recognize when a formerly unmodified tag has been modified, remove
   the git tag, and turn it into a branch.
 
> I've used the following hack to make real tags out of SVN "tags":
> 
> git for-each-ref --format="%(refname)" refs/remotes/tags/ |
> while read tag; do
>     GIT_COMMITTER_DATE="$(git log -1 --pretty=format:"%ad" "$tag")" \
>     GIT_COMMITTER_EMAIL="$(git log -1 --pretty=format:"%ce" "$tag")" \
>     GIT_COMMITTER_NAME="$(git log -1 --pretty=format:"%cn" "$tag")" \
>     git tag -m "$(git log -1 --pretty=format:"%s%n%b" "$tag")" \
>     "${tag#refs/remotes/tags/}" "$tag"
> done
> 
> Disclaimer: it worked last time I used it.  Haven't checked if it got
> dusty since.

Wow, that's ugly. But it does look like it ought to work.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Three issues from a Subversion-to-git migration

[Thomas Rast]

Eric Raymond wrote:
> > I've used the following hack to make real tags out of SVN "tags":
> > 
> > git for-each-ref --format="%(refname)" refs/remotes/tags/ |
> > while read tag; do
> >     GIT_COMMITTER_DATE="$(git log -1 --pretty=format:"%ad" "$tag")" \
> >     GIT_COMMITTER_EMAIL="$(git log -1 --pretty=format:"%ce" "$tag")" \
> >     GIT_COMMITTER_NAME="$(git log -1 --pretty=format:"%cn" "$tag")" \
> >     git tag -m "$(git log -1 --pretty=format:"%s%n%b" "$tag")" \
> >     "${tag#refs/remotes/tags/}" "$tag"
> > done
> > 
> > Disclaimer: it worked last time I used it.  Haven't checked if it got
> > dusty since.
> 
> Wow, that's ugly. But it does look like it ought to work.

BTW, you'll also want to use some treatment that removes the empty
commit that is generated from the 'svn copy' SVN commit for tagging.
One option is to use 'git filter-branch --prune-empty ...', which will
also drop other no-op commits.  If you want to remove only the ones
that come from tagging, creative use of git-diff-tree in the above
loop will work.

I suppose I was never bothered by the lack of automatic tagging
because I rarely found a git-svn import to be immediately fit for
publishing.  Usually it took some grafting and other filtering to
bring the history into shape anyway.  Maybe now that the svn:mergeinfo
support obviates the need for grafting, it's worth thinking about the
rest.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch

Re: Three issues from a Subversion-to-git migration

[Eric Raymond]

Thomas Rast <trast@student.ethz.ch>:
> I suppose I was never bothered by the lack of automatic tagging
> because I rarely found a git-svn import to be immediately fit for
> publishing.  Usually it took some grafting and other filtering to
> bring the history into shape anyway.  Maybe now that the svn:mergeinfo
> support obviates the need for grafting, it's worth thinking about the
> rest.

Can't argue your first point at all, because my only large migration
so far did in fact need filtering - to fix up artifacts from the Emacs
VC front end that were fossilized in the Subversion history. And that
was all my own fault; I was the original author of VC back in the
early 1990s, and should have rewritten it to be changeset-aware years
sooner than I did. Alas, I was kind of busy being Mr. Famous Geek for
about a decade in there, and the VC rewrite was one of several
projects that got seriously sidetracked.  I finally got it done in
2008-2009, and git is one of the backend systems that benefits from
that.

Still. Even conceding that point, built-in support to further reduce
the amount of hand-work required in SVN conversion would be no bad
thing.  Tag conversion was unequivocally the biggest pain in the ass
when I migrated GPSD; I'm not claiming that will always be true, but I do
think it's the largest pain that could be *reliably mechanized
away*. That makes it a logical target.

One of the reasons this is still on my mind after the GPSD migration
is Battle For Wesnoth <http://www.wesnoth.org/>. I'm one of the senior
devs on that project, and it is becoming clear to all that we have
reached Subversion's limits there.  I'm the project's tools and
toolsmithing expert, and I've pretty much got the other devs convinced to
switch to a DVCS when we can screw up our courage to move to a forge
that supports one. 

This means I'm probably going to be the guy on the spot doing yet
another big ugly conversion away from SVN sometime within the next
year.  The state of conversion tools at that time might end up
determining whether Wesnoth goes with git or Mercurial.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Three issues from a Subversion-to-git migration

[Gabriel Filion]

Hello,

On 2010-03-29 05:10, Eric Raymond wrote:
> Thomas Rast <trast@student.ethz.ch>:
>> Eric Raymond wrote:
>>> 2. The git-svn migration logic does not handle unmodified SVN tag
>>> trees well.
>>
>> The problem here is that git-svn is designed to handle incremental
>> updates, where it can't know whether some insane SVN user decides to
>> modify the tag later on.
> 
> Yes.  Ideally, I suppose, git-svn (or whatever replaces it) would have
> behavior something like this:
> 
> 1. Turn unmodified tag directories into git tags
> 2. Turn odified tags into branches.
> 3. Recognize when a formerly unmodified tag has been modified, remove
>    the git tag, and turn it into a branch.
>  

The 3rd point seems a bit weird to me.. users don't expect tags to
disappear magically. Especially if it's done during a fetch while working.

Here's how I would change the scenario:

1. For each creation of a sub-directory in SVN's tag directory, create a
git tag on the revision that was referenced by the directory copy in SVN.
2. If (and only if) there are later modifications in the tag directory,
create a branch starting from that tag.

This way, the tag would be there but a branch would hold modifications
based on code at this point, if there is any.

The problem with my scenario, though is that it doesn't take care of tag
creation + modification in the same commit (yuukkkk, but it's possible
that it exists somewhere). If it could be possible to verify if
modifications were made during the tag creation, then we could make the
case hit both points.

The other big "thing" is that it expects a certain correct separation
into different directories (e.g. trunk/ tags/ branches/ ), which SVN
doesn't enforce.

-- 
Gabriel Filion

Re: Three issues from a Subversion-to-git migration

[Eric Raymond]

Gabriel Filion <lelutin@gmail.com>:
> > 1. Turn unmodified tag directories into git tags
> > 2. Turn odified tags into branches.
> > 3. Recognize when a formerly unmodified tag has been modified, remove
> >    the git tag, and turn it into a branch.
> 
> The 3rd point seems a bit weird to me.. users don't expect tags to
> disappear magically. Especially if it's done during a fetch while working.

A reasonable objection.
 
> Here's how I would change the scenario:
> 
> 1. For each creation of a sub-directory in SVN's tag directory, create a
> git tag on the revision that was referenced by the directory copy in SVN.
> 2. If (and only if) there are later modifications in the tag directory,
> create a branch starting from that tag.
> 
> This way, the tag would be there but a branch would hold modifications
> based on code at this point, if there is any.

That would work for me.
 
> The problem with my scenario, though is that it doesn't take care of tag
> creation + modification in the same commit (yuukkkk, but it's possible
> that it exists somewhere). If it could be possible to verify if
> modifications were made during the tag creation, then we could make the
> case hit both points.

Doesn't seem like that should be difficult.

> The other big "thing" is that it expects a certain correct separation
> into different directories (e.g. trunk/ tags/ branches/ ), which SVN
> doesn't enforce.

Are you suggesting that branch directory copies should be handled with
the same rule? I think I could live with that.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>