cvsimport bug on branches [was: conversion to git]

cvsimport bug on branches [was: conversion to git]

[Eric Blake]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

According to Eric Blake on 9/19/2007 5:13 PM:
> I'm working on converting M4 to git.
> 
> The initial git conversion is complete.  Feel free to check it out:
> 
> http://git.savannah.gnu.org/gitweb/?p=m4.git

Unfortunately, after publishing this repository, I noticed that there is a
bug in git-cvsimport, when a vendor branch is created that contains fewer
files than the point it branched from.  Those files were not removed from
the commit on the new branch, which means the git repository contains too
many files compared to what the CVS repository had for any checkout along
the branch.

For example, look at the m4-1.4.1 tag.  In CVS,

http://cvs.savannah.gnu.org/viewvc/m4/?root=m4&hideattic=0&pathrev=m4-1_4_1

shows the following:
ABOUT-NLS (dead) 	No revisions exist on m4-1_4_1

and for the file ChangeLog, the predecessor of m4-1_4_1 (rev 1.1.1.1.2.1)
was m4-1_4 (rev 1.1.1.1):

http://cvs.savannah.gnu.org/viewvc/m4/ChangeLog?hideattic=0&view=log&root=m4&pathrev=m4-1_4_1


But after git-cvsimport (using git 1.5.3), tag m4-1.4.1 is shown on the
branch-1_4 branch, but with unrelated m4-1.4ppre2 as parent, instead of
the correct m4-1.4:

http://git.sv.gnu.org/gitweb/?p=m4.git;a=shortlog;h=ab2ce6be

and as a result, the tree includes ABOUT-NLS (which was correctly a part
of the m4-1.4ppre2 tag on the master branch):

http://git.sv.gnu.org/gitweb/?p=m4.git;a=tree;h=ab2ce6bec48;hb=ab2ce6bec4


Now that the repository is published, what's the best way to correct the
history for the git m4-1.4.1 tag, to correctly pick the m4-1.4 commit as
its parent, and to omit the files that were intentionally omitted from the
CVS branch it was imported from?

What needs to be done to git-cvsimport to make it recognize files that
were omitted from CVS branch creation points, to avoid this problem in the
first place?

- --
Don't work too hard, make some time for fun as well!

Eric Blake             ebb9@byu.net
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFG8xu684KuGfSFAYARAjd8AJ0eOWw1TFGt4zpQKIDFTGRjxzI/cgCghBpM
W91wfRwc8Q2hoMO1WCWYtzM=
=zzNR
-----END PGP SIGNATURE-----

Re: cvsimport bug on branches [was: conversion to git]

[Steffen Prohaska]

On Sep 21, 2007, at 3:17 AM, Eric Blake wrote:

> According to Eric Blake on 9/19/2007 5:13 PM:
>> I'm working on converting M4 to git.
>>
>> The initial git conversion is complete.  Feel free to check it out:
>>
>> http://git.savannah.gnu.org/gitweb/?p=m4.git
>
> Unfortunately, after publishing this repository, I noticed that  
> there is a
> bug in git-cvsimport, when a vendor branch is created that contains  
> fewer
> files than the point it branched from.  Those files were not  
> removed from
> the commit on the new branch, which means the git repository  
> contains too
> many files compared to what the CVS repository had for any checkout  
> along
> the branch.
>
> [...]

git-cvsimport may have more problems, see e.g.

http://marc.info/?l=git&m=118385565913953&w=2
http://marc.info/?l=git&m=118260312708709&w=2


> Now that the repository is published, what's the best way to  
> correct the
> history for the git m4-1.4.1 tag, to correctly pick the m4-1.4  
> commit as
> its parent, and to omit the files that were intentionally omitted  
> from the
> CVS branch it was imported from?

Hard to say. The best is to avoid git-cvsimport if you need
to import branches correctly.

And you should do validate the result of any import by comparing
a working copy checked out from CVS with a working copy checked
out from git.

Never trust the cvs to git importers.


> What needs to be done to git-cvsimport to make it recognize files that
> were omitted from CVS branch creation points, to avoid this problem  
> in the
> first place?

I'd recommend not to use git-cvsimport but better use
a different converter.

For me parsecvs [1] works.

I'd also give cvs2svn's git support [2] a try.

cvs2svn strives much harder to report errors and handle complex
history correctly. parsecvs for example just crashes on corrupted
RCS files, while cvs2svn reports the problem. But if your CVS
repository is good parsecvs is much faster.

fromcvs/togit was also mentioned a couple of times on the list.
It worked for me the last time I tried it. But I did no in depth
validation of the result.

I still believe it would be good to add a warning to the
documentation of git-cvsimport and provide links to alternatives.

	Steffen

[1] http://gitweb.freedesktop.org/?p=users/keithp/parsecvs.git
[2] http://marc.info/?l=git&m=118592701426175&w=2

Re: cvsimport bug on branches [was: conversion to git]

[Linus Torvalds]

On Fri, 21 Sep 2007, Steffen Prohaska wrote:
> 
> Hard to say. The best is to avoid git-cvsimport if you need
> to import branches correctly.

Yes. git-cvsimport started out as a pretty quick hack, and depends on 
cvsps which is another fairly hacky thing.

The big advantage of git-cvsimport is that it can do incremental imports, 
which I don't think the other methods do. But if there is any choice at 
all, and especially if you're not that interested in the incremental 
feature (ie you can cut over to git, and perhaps use git-cvsserver to 
"supprt" CVS users) the other CVS importers are likely to be much better.

			Linus

Re: cvsimport bug on branches [was: conversion to git]

[Robin Rosenberg]

fredag 21 september 2007 skrev Linus Torvalds:
> 
> On Fri, 21 Sep 2007, Steffen Prohaska wrote:
> > 
> > Hard to say. The best is to avoid git-cvsimport if you need
> > to import branches correctly.
> 
> Yes. git-cvsimport started out as a pretty quick hack, and depends on 
> cvsps which is another fairly hacky thing.
> 
> The big advantage of git-cvsimport is that it can do incremental imports, 
> which I don't think the other methods do. But if there is any choice at 
> all, and especially if you're not that interested in the incremental 
> feature (ie you can cut over to git, and perhaps use git-cvsserver to 
> "supprt" CVS users) the other CVS importers are likely to be much better.

fromcvs does incremental import and it's very fast and uses much less memory 
than cvsimport. It needs the rcs files however and will not convert 
non-branch tags.

-- robin

Re: cvsimport bug on branches [was: conversion to git]

[Johannes Schindelin]

Hi,

On Fri, 21 Sep 2007, Robin Rosenberg wrote:

> fredag 21 september 2007 skrev Linus Torvalds:
> > 
> > The big advantage of git-cvsimport is that it can do incremental 
> > imports, which I don't think the other methods do. But if there is any 
> > choice at all, and especially if you're not that interested in the 
> > incremental feature (ie you can cut over to git, and perhaps use 
> > git-cvsserver to "supprt" CVS users) the other CVS importers are 
> > likely to be much better.
> 
> fromcvs does incremental import and it's very fast and uses much less 
> memory than cvsimport. It needs the rcs files however and will not 
> convert non-branch tags.

Plus you have to install Ruby.  Just wanted people to know.

Ciao,
Dscho

Re: cvsimport bug on branches [was: conversion to git]

[Robin Rosenberg]

fredag 21 september 2007 skrev Johannes Schindelin:
> Hi,
> 
> On Fri, 21 Sep 2007, Robin Rosenberg wrote:
> 
> > fredag 21 september 2007 skrev Linus Torvalds:
> > > 
> > > The big advantage of git-cvsimport is that it can do incremental 
> > > imports, which I don't think the other methods do. But if there is any 
> > > choice at all, and especially if you're not that interested in the 
> > > incremental feature (ie you can cut over to git, and perhaps use 
> > > git-cvsserver to "supprt" CVS users) the other CVS importers are 
> > > likely to be much better.
> > 
> > fromcvs does incremental import and it's very fast and uses much less 
> > memory than cvsimport. It needs the rcs files however and will not 
> > convert non-branch tags.
> 
> Plus you have to install Ruby.  Just wanted people to know.

You don't like Ruby, do you? It worth it, really.

-- robin

Re: cvsimport bug on branches [was: conversion to git]

[Johannes Schindelin]

Hi,

On Fri, 21 Sep 2007, Robin Rosenberg wrote:

> fredag 21 september 2007 skrev Johannes Schindelin:
> > Hi,
> > 
> > On Fri, 21 Sep 2007, Robin Rosenberg wrote:
> > 
> > > fredag 21 september 2007 skrev Linus Torvalds:
> > > > 
> > > > The big advantage of git-cvsimport is that it can do incremental 
> > > > imports, which I don't think the other methods do. But if there is any 
> > > > choice at all, and especially if you're not that interested in the 
> > > > incremental feature (ie you can cut over to git, and perhaps use 
> > > > git-cvsserver to "supprt" CVS users) the other CVS importers are 
> > > > likely to be much better.
> > > 
> > > fromcvs does incremental import and it's very fast and uses much less 
> > > memory than cvsimport. It needs the rcs files however and will not 
> > > convert non-branch tags.
> > 
> > Plus you have to install Ruby.  Just wanted people to know.
> 
> You don't like Ruby, do you? It worth it, really.

No, it's yet another dependency.  And the quality of the code still 
depends on the programmer, not the language.

Ciao,
Dscho

Re: cvsimport bug on branches [was: conversion to git]

[Robin Rosenberg]

fredag 21 september 2007 skrev Johannes Schindelin:
> No, it's yet another dependency.  And the quality of the code still 
> depends on the programmer, not the language.

I think I can agree there, on both counts. But, if you want a good incremental 
CVS importer and have access to the rcs files, that's the one there is.

git-cvsimport has a dependency on cvsps, which isn't included. You have to
to look it up yourself chooing among a dozen unequally bad versions. 
Installing ruby isn't any harder.

The dependency excludes it from being included with Git, but it does not 
disqualify it as a tool on it's own.

-- robin

fromcvs installation [was: cvsimport bug on branches [was: conversion to git]]

[Steffen Prohaska]

On Sep 22, 2007, at 1:16 AM, Robin Rosenberg wrote:

> fredag 21 september 2007 skrev Johannes Schindelin:
>> No, it's yet another dependency.  And the quality of the code still
>> depends on the programmer, not the language.
>
> I think I can agree there, on both counts. But, if you want a good  
> incremental
> CVS importer and have access to the rcs files, that's the one there  
> is.
>
> git-cvsimport has a dependency on cvsps, which isn't included. You  
> have to
> to look it up yourself chooing among a dozen unequally bad versions.
> Installing ruby isn't any harder.
>
> The dependency excludes it from being included with Git, but it  
> does not
> disqualify it as a tool on it's own.

Nonetheless a more detailed description how to install fromcvs
would be helpful. I remember it was not obvious to me.

The README of fromcvs only says
'''
Prerequisites:
- ruby (1.8.5 known working)
- fromcvs, <http://ww2.fs.ei.tum.de/~corecode/hg/fromcvs>
- rcsparse, <http://ww2.fs.ei.tum.de/~corecode/hg/rcsparse> (ruby  
module)
- Ruby/RBTree, <http://raa.ruby-lang.org/project/ruby-rbtree/>
- for git: git-fast-import, part of git 1.5
- for db/commitset: sqlite3 + sqlite3-ruby (available as gem)
'''

I didn't use ruby before and had no idea how to install a ruby module
or what 'available as gem' means. And by the way, it was not obvious
for me that I can download the tip of the mercurial branch by clicking
on the 'gz' link at the top of the page.

I think a step by step explanation how to proceed would have helped me.

	Steffen

Re: fromcvs installation [was: cvsimport bug on branches [was: conversion to git]]

[Simon 'corecode' Schubert]

Steffen Prohaska wrote:
> Nonetheless a more detailed description how to install fromcvs
> would be helpful. I remember it was not obvious to me.
[..]
> I think a step by step explanation how to proceed would have helped me.

Could you write down what you had to do and where the problems are as an
extension to the current README?

Thanks,
  simon

Re: cvsimport bug on branches [was: conversion to git]

[Martin Langhoff]

On 9/22/07, Linus Torvalds <torvalds@linux-foundation.org> wrote:
> The big advantage of git-cvsimport is that it can do incremental imports,

Indeed. And it can take over incrementals on a GIT repo initially
imported by parsecvs or cvs2git. That's the recommended practice for
tracking non-trivial CVS repos -- IOWs "that's what I do" ;-)

cheers,


m