Fwd: [Bug 163341] Re: git-svn gets wrong parent revision for tags

Fwd: [Bug 163341] Re: git-svn gets wrong parent revision for tags

[Thomas Leonard]

Hi.

The git-svn Ubuntu packager asked me to forward this here.

Summary:

When converting a subversion repository to GIT using git-svn, the tags
do not have the right parent. Each tag should be identical to a trunk
revision (which it was copied from), but because git-svn uses an
earlier revision as the parent it appears that the same work was
duplicated on two branches.

Example:

git-svn clone https://zero-install.svn.sourceforge.net/svnroot/zero-install
-T trunk/0publish -r1890:2072 -b releases/0publish

The git branch comes from r1894, yet the svn log shows that in
includes files from r2070:

$ svn log https://zero-install.svn.sourceforge.net/svnroot/zero-install
-r2071 -v
r2071 | tal197 | 2007-11-10 19:40:45 +0000 (Sat, 10 Nov 2007) | 1 line
Changed paths:
   A /releases/0publish/0publish-0.12 (from /trunk/0publish:1968)
   R /releases/0publish/0publish-0.12/0publish (from
/trunk/0publish/0publish:2070)
   R /releases/0publish/0publish-0.12/0publish.xml (from
/trunk/0publish/0publish.xml:2070)
   R /releases/0publish/0publish-0.12/release.py (from
/trunk/0publish/release.py:2069)

Released 0.12

https://bugs.launchpad.net/ubuntu/+source/git-core/+bug/163341

Thanks,


---------- Forwarded message ----------
From: Emil Sit <sit@amsterdam.lcs.mit.edu>
Date: 7 Apr 2008 20:56
Subject: [Bug 163341] Re: git-svn gets wrong parent revision for tags


I know that git-svn may rely on various conventions for the use of git
 and use of svn but since I'm not really familiar with either, it sounds
 like this should best be handled by the upstream developers.  Git does
 not use a dedicated bug tracker but handles issues on the mailing list
 git@vger.kernel.org.  If you submit your patch there, you should get a
 pretty fast response and you can follow up with this bug.  In any event,
 if the bug is fixed upstream, the next version of Ubuntu will probably
 include the fix automatically.

 Can you handle the follow-up with the official git developers?  Thanks!


 --
 git-svn gets wrong parent revision for tags
 https://bugs.launchpad.net/bugs/163341
 You received this bug notification because you are a direct subscriber
 of the bug.


-- 
Dr Thomas Leonard		http://rox.sourceforge.net
GPG: 9242 9807 C985 3C07 44A6  8B9A AE07 8280 59A5 3CC1

Re: Fwd: [Bug 163341] Re: git-svn gets wrong parent revision for tags

[Björn Steinbrink]

On 2008.04.08 16:48:03 +0100, Thomas Leonard wrote:
> Hi.
> 
> The git-svn Ubuntu packager asked me to forward this here.
> 
> Summary:
> 
> When converting a subversion repository to GIT using git-svn, the tags
> do not have the right parent. Each tag should be identical to a trunk
> revision (which it was copied from), but because git-svn uses an
> earlier revision as the parent it appears that the same work was
> duplicated on two branches.
> 
> Example:
> 
> git-svn clone https://zero-install.svn.sourceforge.net/svnroot/zero-install
> -T trunk/0publish -r1890:2072 -b releases/0publish
> 
> The git branch comes from r1894, yet the svn log shows that in
> includes files from r2070:
> 
> $ svn log https://zero-install.svn.sourceforge.net/svnroot/zero-install
> -r2071 -v
> r2071 | tal197 | 2007-11-10 19:40:45 +0000 (Sat, 10 Nov 2007) | 1 line
> Changed paths:
>    A /releases/0publish/0publish-0.12 (from /trunk/0publish:1968)
>    R /releases/0publish/0publish-0.12/0publish (from
> /trunk/0publish/0publish:2070)
>    R /releases/0publish/0publish-0.12/0publish.xml (from
> /trunk/0publish/0publish.xml:2070)
>    R /releases/0publish/0publish-0.12/release.py (from
> /trunk/0publish/release.py:2069)

Well, SVN recorded useless, broken metadata.

SVN itself believes that the branch was created from revision 1968. As
that revision didn't introduce any changes to trunk/0publish, there's no
commit for that revision in the git branch, so git-svn took the most
recent one instead (1894).

For the other three files, SVN reports that the files were replaced by
versions from another branch. There's no immediate way to tell whether
those replacements make the branch equal to the more recent version of
trunk. So git-svn does it the safe way and reproduces what SVN told it
to reproduce: A commit that creates a branch and changes some files.

I guess sth. like this happened on the svn end:
svn cp trunk/0publish releases/0publish (at rev. 1968)
svn cp trunk/0publish/release.py releases/0publish (at rev. 2069)
...
svn commit (whenever)

So the branch was "incrementally" created locally and SVN decided to
record the "Uh, I copied this from here to there" metacrap^H^H^H^Hdata,
of which git-svn made use. Hooray for rename/copy tracking.

You could maybe ask for an enhancement so that git-svn tries to figure
out if the provided metadata is broken and looks up a matching revision
in such a case. But I guess it's not worth the trouble and could lead to
other breakage along the way...

Björn

Re: Fwd: [Bug 163341] Re: git-svn gets wrong parent revision for tags

[Thomas Leonard]

On 08/04/2008, Björn Steinbrink <B.Steinbrink@gmx.de> wrote:
> On 2008.04.08 16:48:03 +0100, Thomas Leonard wrote:
[...]
>  > When converting a subversion repository to GIT using git-svn, the tags
>  > do not have the right parent. Each tag should be identical to a trunk
>  > revision (which it was copied from), but because git-svn uses an
>  > earlier revision as the parent it appears that the same work was
>  > duplicated on two branches.
[...]
>  > git-svn clone https://zero-install.svn.sourceforge.net/svnroot/zero-install
>  > -T trunk/0publish -r1890:2072 -b releases/0publish
>  >
>  > The git branch comes from r1894, yet the svn log shows that in
>  > includes files from r2070:
>  >
>  > $ svn log https://zero-install.svn.sourceforge.net/svnroot/zero-install
>  > -r2071 -v
>  > r2071 | tal197 | 2007-11-10 19:40:45 +0000 (Sat, 10 Nov 2007) | 1 line
>  > Changed paths:
>  >    A /releases/0publish/0publish-0.12 (from /trunk/0publish:1968)
>  >    R /releases/0publish/0publish-0.12/0publish (from
>  > /trunk/0publish/0publish:2070)
>  >    R /releases/0publish/0publish-0.12/0publish.xml (from
>  > /trunk/0publish/0publish.xml:2070)
>  >    R /releases/0publish/0publish-0.12/release.py (from
>  > /trunk/0publish/release.py:2069)
>
> Well, SVN recorded useless, broken metadata.

This is certainly true ;-)

>  SVN itself believes that the branch was created from revision 1968. As
>  that revision didn't introduce any changes to trunk/0publish, there's no
>  commit for that revision in the git branch, so git-svn took the most
>  recent one instead (1894).

>  For the other three files, SVN reports that the files were replaced by
>  versions from another branch. There's no immediate way to tell whether
>  those replacements make the branch equal to the more recent version of
>  trunk. So git-svn does it the safe way and reproduces what SVN told it
>  to reproduce: A commit that creates a branch and changes some files.
>
>  I guess sth. like this happened on the svn end:
>  svn cp trunk/0publish releases/0publish (at rev. 1968)
>  svn cp trunk/0publish/release.py releases/0publish (at rev. 2069)
>  ...
>  svn commit (whenever)

The script that made the releases simply did this ("." is the working copy):

svn cp -m "Released $VERSION" . http://...

Since each file in an svn working copy has its own revision (the last
time it was changed), the branch ends up with multiple source
revisions, even if the working copy is fully up-to-date and has no
local changes.

Example:

cd /tmp
svnadmin create test-repo
svn mkdir file:///tmp/test-repo/trunk -m 'Setup'
svn co file:///tmp/test-repo/trunk
cd trunk
touch foo bar
svn add foo bar
svn ci -m 'Start'
echo hi > bar
svn ci -m 'Update'
svn cp . file:///tmp/test-repo/release -m 'Release'
svn log -v file:///tmp/test-repo

The log shows:

------------------------------------------------------------------------
r4 | talex | 2008-04-08 21:35:44 +0100 (Tue, 08 Apr 2008) | 1 line
Changed paths:
   A /release (from /trunk:1)
   A /release/bar (from /trunk/bar:3)
   A /release/foo (from /trunk/foo:2)

Release

So, I don't think the metadata is broken in this case. Maybe other
people don't create branches like this, but it seemed like the obvious
way to do it at the time. Given this behaviour of svn, perhaps it
would be better to take the highest version number as the branch
point?


-- 
Dr Thomas Leonard		http://rox.sourceforge.net
GPG: 9242 9807 C985 3C07 44A6  8B9A AE07 8280 59A5 3CC1

Re: Fwd: [Bug 163341] Re: git-svn gets wrong parent revision for tags

[Björn Steinbrink]

[Grmpf, the next time you drop from me Cc:, I'll not bother to
answer...]

On 2008.04.08 21:43:03 +0100, Thomas Leonard wrote:
> On 08/04/2008, Björn Steinbrink <B.Steinbrink@gmx.de> wrote:
> > On 2008.04.08 16:48:03 +0100, Thomas Leonard wrote:
> [...]
> >  > When converting a subversion repository to GIT using git-svn, the tags
> >  > do not have the right parent. Each tag should be identical to a trunk
> >  > revision (which it was copied from), but because git-svn uses an
> >  > earlier revision as the parent it appears that the same work was
> >  > duplicated on two branches.
> [...]
> >  > git-svn clone https://zero-install.svn.sourceforge.net/svnroot/zero-install
> >  > -T trunk/0publish -r1890:2072 -b releases/0publish
> >  >
> >  > The git branch comes from r1894, yet the svn log shows that in
> >  > includes files from r2070:
> >  >
> >  > $ svn log https://zero-install.svn.sourceforge.net/svnroot/zero-install
> >  > -r2071 -v
> >  > r2071 | tal197 | 2007-11-10 19:40:45 +0000 (Sat, 10 Nov 2007) | 1 line
> >  > Changed paths:
> >  >    A /releases/0publish/0publish-0.12 (from /trunk/0publish:1968)
> >  >    R /releases/0publish/0publish-0.12/0publish (from
> >  > /trunk/0publish/0publish:2070)
> >  >    R /releases/0publish/0publish-0.12/0publish.xml (from
> >  > /trunk/0publish/0publish.xml:2070)
> >  >    R /releases/0publish/0publish-0.12/release.py (from
> >  > /trunk/0publish/release.py:2069)
> >
> > Well, SVN recorded useless, broken metadata.
> 
> This is certainly true ;-)
> 
> >  SVN itself believes that the branch was created from revision 1968. As
> >  that revision didn't introduce any changes to trunk/0publish, there's no
> >  commit for that revision in the git branch, so git-svn took the most
> >  recent one instead (1894).
> 
> >  For the other three files, SVN reports that the files were replaced by
> >  versions from another branch. There's no immediate way to tell whether
> >  those replacements make the branch equal to the more recent version of
> >  trunk. So git-svn does it the safe way and reproduces what SVN told it
> >  to reproduce: A commit that creates a branch and changes some files.
> >
> >  I guess sth. like this happened on the svn end:
> >  svn cp trunk/0publish releases/0publish (at rev. 1968)
> >  svn cp trunk/0publish/release.py releases/0publish (at rev. 2069)
> >  ...
> >  svn commit (whenever)
> 
> The script that made the releases simply did this ("." is the working copy):
> 
> svn cp -m "Released $VERSION" . http://...
> 
> Since each file in an svn working copy has its own revision (the last
> time it was changed), the branch ends up with multiple source
> revisions, even if the working copy is fully up-to-date and has no
> local changes.
> 
> Example:
> 
> cd /tmp
> svnadmin create test-repo
> svn mkdir file:///tmp/test-repo/trunk -m 'Setup'
> svn co file:///tmp/test-repo/trunk
> cd trunk
> touch foo bar
> svn add foo bar
> svn ci -m 'Start'
> echo hi > bar
> svn ci -m 'Update'
> svn cp . file:///tmp/test-repo/release -m 'Release'
> svn log -v file:///tmp/test-repo
> 
> The log shows:
> 
> ------------------------------------------------------------------------
> r4 | talex | 2008-04-08 21:35:44 +0100 (Tue, 08 Apr 2008) | 1 line
> Changed paths:
>    A /release (from /trunk:1)
>    A /release/bar (from /trunk/bar:3)
>    A /release/foo (from /trunk/foo:2)
> 
> Release

Change the "." to "file:///tmp/test-repo/trunk" in the svn cp command
and you get:
Changed paths:
   A /release (from /trunk:3)

Seems more useful. No idea why SVN decides to record crap when you use
".", even when your working copy is clean.

> So, I don't think the metadata is broken in this case. Maybe other
> people don't create branches like this, but it seemed like the obvious
> way to do it at the time. Given this behaviour of svn, perhaps it
> would be better to take the highest version number as the branch
> point?

Uhm, and what happens then, when you actually _did_ just copy over a
few files, but not all of them? Right, you get a branch to start at a
later revision and see a bunch of reverts. Equally broken.

Björn

Re: Fwd: [Bug 163341] Re: git-svn gets wrong parent revision for tags

[Thomas Leonard]

On 09/04/2008, Björn Steinbrink <B.Steinbrink@gmx.de> wrote:
> [Grmpf, the next time you drop from me Cc:, I'll not bother to
>  answer...]

Fixed; sorry.

>  On 2008.04.08 21:43:03 +0100, Thomas Leonard wrote:
[...]
>  > The script that made the releases simply did this ("." is the working copy):
>  >
>  > svn cp -m "Released $VERSION" . http://...
>  >
>  > Since each file in an svn working copy has its own revision (the last
>  > time it was changed), the branch ends up with multiple source
>  > revisions, even if the working copy is fully up-to-date and has no
>  > local changes.
>  >
>  > Example:
>  >
>  > cd /tmp
>  > svnadmin create test-repo
>  > svn mkdir file:///tmp/test-repo/trunk -m 'Setup'
>  > svn co file:///tmp/test-repo/trunk
>  > cd trunk
>  > touch foo bar
>  > svn add foo bar
>  > svn ci -m 'Start'
>  > echo hi > bar
>  > svn ci -m 'Update'
>  > svn cp . file:///tmp/test-repo/release -m 'Release'
>  > svn log -v file:///tmp/test-repo
>  >
>  > The log shows:
>  >
>  > ------------------------------------------------------------------------
>  > r4 | talex | 2008-04-08 21:35:44 +0100 (Tue, 08 Apr 2008) | 1 line
>  > Changed paths:
>  >    A /release (from /trunk:1)
>  >    A /release/bar (from /trunk/bar:3)
>  >    A /release/foo (from /trunk/foo:2)
>  >
>  > Release
>
> Change the "." to "file:///tmp/test-repo/trunk" in the svn cp command
>  and you get:
>  Changed paths:
>    A /release (from /trunk:3)
>
>  Seems more useful. No idea why SVN decides to record crap when you use
>  ".", even when your working copy is clean.

That's good advice, but I'm wondering how to convert my existing svn
repositories to GIT.

>  > So, I don't think the metadata is broken in this case. Maybe other
>  > people don't create branches like this, but it seemed like the obvious
>  > way to do it at the time. Given this behaviour of svn, perhaps it
>  > would be better to take the highest version number as the branch
>  > point?
>
> Uhm, and what happens then, when you actually _did_ just copy over a
>  few files, but not all of them? Right, you get a branch to start at a
>  later revision and see a bunch of reverts. Equally broken.

Sure, if you create a situation that GIT can't represent then there's
nothing much git-svn can do.

However, when there are different source revisions:

- Taking the highest revision will give correct results in the common
case, and "equally broken" results in the uncommon case.

- Taking any other revision (the current behaviour) is guaranteed to
give the wrong result. There's no way a branch that was created with a
file from revision 6 can possibly be a branch of revision 5, for
example.

Also, the current behaviour can be wildly out (e.g. revision 1894 when
it should have been 2070, in the previous real example). The highest
revision is unlikely to be far off.


-- 
Dr Thomas Leonard		http://rox.sourceforge.net
GPG: 9242 9807 C985 3C07 44A6  8B9A AE07 8280 59A5 3CC1

Re: Fwd: [Bug 163341] Re: git-svn gets wrong parent revision for tags

[Björn Steinbrink]

On 2008.04.09 11:39:17 +0100, Thomas Leonard wrote:
> On 09/04/2008, Björn Steinbrink <B.Steinbrink@gmx.de> wrote:
> >  On 2008.04.08 21:43:03 +0100, Thomas Leonard wrote:
> >  > The log shows:
> >  >
> >  > ------------------------------------------------------------------------
> >  > r4 | talex | 2008-04-08 21:35:44 +0100 (Tue, 08 Apr 2008) | 1 line
> >  > Changed paths:
> >  >    A /release (from /trunk:1)
> >  >    A /release/bar (from /trunk/bar:3)
> >  >    A /release/foo (from /trunk/foo:2)
> >  >
> >  > Release
> >
> > Change the "." to "file:///tmp/test-repo/trunk" in the svn cp command
> >  and you get:
> >  Changed paths:
> >    A /release (from /trunk:3)
> >
> >  Seems more useful. No idea why SVN decides to record crap when you use
> >  ".", even when your working copy is clean.
> 
> That's good advice, but I'm wondering how to convert my existing svn
> repositories to GIT.

I'd probably patch git-svn locally once and revert it after the import,
if your suggested change really makes sense for your repo. Another
option would be to adjust the branches afterwards, but that's probably
awkward in your case. Maybe a patch that allows to toggle the branch
creation logic would be accepted as well, dunno.

> >  > So, I don't think the metadata is broken in this case. Maybe other
> >  > people don't create branches like this, but it seemed like the obvious
> >  > way to do it at the time. Given this behaviour of svn, perhaps it
> >  > would be better to take the highest version number as the branch
> >  > point?
> >
> > Uhm, and what happens then, when you actually _did_ just copy over a
> >  few files, but not all of them? Right, you get a branch to start at a
> >  later revision and see a bunch of reverts. Equally broken.
> 
> Sure, if you create a situation that GIT can't represent then there's
> nothing much git-svn can do.
> 
> However, when there are different source revisions:
> 
> - Taking the highest revision will give correct results in the common
> case, and "equally broken" results in the uncommon case.
> 
> - Taking any other revision (the current behaviour) is guaranteed to
> give the wrong result. There's no way a branch that was created with a
> file from revision 6 can possibly be a branch of revision 5, for
> example.

Sure it can be.

svn co $repo full-repo-here
cd full-repo-here
svn cp trunk branches/b1 (at rev 5)
echo 123 > trunk/foo # (new file)
echo 456 > trunk/bar
# 431 other files changed (exaggeration ;-))
svn commit trunk -m "Committing trunk only, with 433 files changed"
svn cp trunk/foo branches/b1
svn commit branches/b1 -m "Committing b1."

b1 branched at rev 5, but has a single file from rev 6.

Currently, git-svn would create the branch starting at rev5, with
the addition of "foo" on top.

Your suggested change would make it branch at rev6 and revert 432 files.

The fact that svn just has directory copies as branches makes all kinds
of junk possible ;-)

> Also, the current behaviour can be wildly out (e.g. revision 1894 when
> it should have been 2070, in the previous real example). The highest
> revision is unlikely to be far off.

How many "interesting" revisions was it off by? 3? As the revision
numbers in svn are shared throughout the whole repo, your example sounds
far worse than it really is.

And for locally prepared branches (I've actually heard of people doing
that, like in my example above), it can happen for your way to be way
more off than the current way.

Björn