cvsimport/cvsps: wrong revisions in branch

cvsimport/cvsps: wrong revisions in branch

[Gerrit Pape]

Hi, I struggle with importing a CVS repository into git to switch over
development.  In some branches there're differences in the source code
after importing into git.  We tracked this down to tags and branches
that where tagged partially in CVS, first a tag was set on one
subdirectory tree (sources), later on another subdirectory tree (docs).
This confuses cvsimport or cvsps, the tag in git after import doesn't
match the tag in CVS.

I use cvsps from http://ydirson.free.fr/soft/git/cvsps.git, here's how
to reproduce:

$ cvs -d /tmp/cvs init
$ mkdir repo && cd repo/
$ cvs -d /tmp/cvs import -m initial repo vendor-tag release-tag

No conflicts created by this import

$ cd .. && rmdir repo 
$ cvs -d /tmp/cvs co repo  
cvs checkout: Updating repo
$ cd repo/
$ mkdir a && echo foo >a/a
$ cvs add a a/a
Directory /tmp/cvs/repo/a added to the repository
cvs add: scheduling file `a/a' for addition
cvs add: use `cvs commit' to add this file permanently
$ cvs ci -m foo
cvs commit: Examining .
cvs commit: Examining a
/tmp/cvs/repo/a/a,v  <--  a/a
initial revision: 1.1
$ cvs tag TAG0 a/
cvs tag: Tagging a
T a/a
$ echo 'not in TAG0' >a/a 
$ cvs ci -m 'not in TAG0'
cvs commit: Examining .
cvs commit: Examining a
/tmp/cvs/repo/a/a,v  <--  a/a
new revision: 1.2; previous revision: 1.1
$ mkdir b && echo bar >b/b
$ cvs add b b/b
Directory /tmp/cvs/repo/b added to the repository
cvs add: scheduling file `b/b' for addition
cvs add: use `cvs commit' to add this file permanently
$ cvs ci -m 'add b/b'
cvs commit: Examining .
cvs commit: Examining a
cvs commit: Examining b
/tmp/cvs/repo/b/b,v  <--  b/b
initial revision: 1.1
$ cvs tag TAG0 b/
cvs tag: Tagging b
T b/b
$ git-cvsimport -C git
Committing initial tree d59780674121a7f49784509ee85079a36c9954a3
$ cd git/
$ git checkout -f master
$ git checkout -b tmp TAG0
$ cat a/a 
not in TAG0
$ 

Thanks, Gerrit.

Re: cvsimport/cvsps: wrong revisions in branch

[Simon 'corecode' Schubert]

[-- Attachment #1: Type: text/plain, Size: 1456 bytes --]

Gerrit Pape wrote:
> Hi, I struggle with importing a CVS repository into git to switch over
> development.  In some branches there're differences in the source code
> after importing into git.  We tracked this down to tags and branches
> that where tagged partially in CVS, first a tag was set on one
> subdirectory tree (sources), later on another subdirectory tree (docs).
> This confuses cvsimport or cvsps, the tag in git after import doesn't
> match the tag in CVS.

how do you expect this to ever work?  your tag is not a time stable one.  the tag spans multiple change sets.  best forget the whole tag thing, seriously.  my cvs converter [1] already has a hard time reproducing branches correctly.  doing this with tags is impossible.  svn can do this because they don't have tags, so it is a kind of fake.  of course you could do this in git as well by actually creating a branch for a tag, but it is really messy.  just don't do that.

good luck
  simon

[1]: <http://ww2.fs.ei.tum.de/~corecode/hg/fromcvs>, not 100% yet, but probably the best you can get at the moment.  i'm a perfectionist, so i might never call it "done".

-- 
Serve - BSD     +++  RENT this banner advert  +++    ASCII Ribbon   /"\
Work - Mac      +++  space for low €€€ NOW!1  +++      Campaign     \ /
Party Enjoy Relax   |   http://dragonflybsd.org      Against  HTML   \
Dude 2c 2 the max   !   http://golden-apple.biz       Mail + News   / \


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 252 bytes --]

Re: cvsimport/cvsps: wrong revisions in branch

[Uwe Kleine-König]

Hi Simon,

Simon 'corecode' Schubert wrote:
> [1]: <http://ww2.fs.ei.tum.de/~corecode/hg/fromcvs>, not 100% yet, but 
> probably the best you can get at the moment.  i'm a perfectionist, so i 
> might never call it "done".
I don't know ruby nor hg, but I managed to clone your repo.  Now I'm
stuck with:

	zeisberg@cepheus:~/hsrc/fromcvs$ ruby1.8 fromcvs.rb 
	fromcvs.rb:1:in `require': no such file to load -- rcsfile (LoadError)
	        from fromcvs.rb:1

`locate rcsfile` only returns /usr/share/man/man5/rcsfile.5.gz from the
rcs package.  `apt-cache search rcsfile` returns nothing.  (This is on
Debian testing/unstable.)  `apt-cache search rcs ruby` only returns

	aswiki - WikiWikiWeb clone (Wiki Engine) written in Ruby

Google didn't help me either.

Is there some documentation how to get that to work?

Best regards
Uwe

-- 
Uwe Kleine-König

http://www.google.com/search?q=the+speed+of+light+in+m%2Fs

Re: cvsimport/cvsps: wrong revisions in branch

[Simon 'corecode' Schubert]

[-- Attachment #1: Type: text/plain, Size: 2115 bytes --]

Uwe Kleine-König wrote:
> I don't know ruby nor hg, but I managed to clone your repo.  Now I'm
> stuck with:
> 
> 	zeisberg@cepheus:~/hsrc/fromcvs$ ruby1.8 fromcvs.rb 
> 	fromcvs.rb:1:in `require': no such file to load -- rcsfile (LoadError)
> 	        from fromcvs.rb:1

yes, you need the rcsparse module, available from the same address.

> Is there some documentation how to get that to work?

I've written it a million times to different people on irc, but somehow I never got around to making a README out of it.  I'll write it up very condensed here and add this as a base for a README to the the repo.

From CVS to * in little time
============================

Prerequisites:
  - ruby (1.8.5 known working)
  - fromcvs, <http://ww2.fs.ei.tum.de/~corecode/hg/fromcvs>
  - rcsparse, <http://ww2.fs.ei.tum.de/~corecode/hg/rcsparse> (ruby module)
  - Ruby/RBTree, <http://raa.ruby-lang.org/project/ruby-rbtree/>
  - for git: git-fast-import, <git://repo.or.cz/git/fastimport.git/>
  - for hg: ruby-python, <http://ww2.fs.ei.tum.de/~corecode/hg/ruby-python>


How to convert a CVS repo:

There is still a nice frontend missing, so you have to invoke the scripts by hand.

% mkdir gitdest && (cd gitdest && git init)
% ruby togit.rb /path/to/cvsroot modulename gitdest
% cd gitdest && git-repack -a -f -d      # optional, saves 20% or so

If you want to get nice author names, create setup the file authormap:

cat > /path/to/cvsroot/CVSROOT/authormap
corecode	Simon 'corecode' Schubert <corecode@fs.ei.tum.de>
jdoe		Jon H. Doe <hannidoe@example.net>
branch-fixup	CVS repo conversion <flames@example.net>

that's about for the quickstart.  be sure to report conversion errors or runtime issues (possibly due to obscure CVS repo activity) to me.

cheers
  simon

-- 
Serve - BSD     +++  RENT this banner advert  +++    ASCII Ribbon   /"\
Work - Mac      +++  space for low €€€ NOW!1  +++      Campaign     \ /
Party Enjoy Relax   |   http://dragonflybsd.org      Against  HTML   \
Dude 2c 2 the max   !   http://golden-apple.biz       Mail + News   / \


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 252 bytes --]

Re: cvsimport/cvsps: wrong revisions in branch

[Shawn O. Pearce]

Simon 'corecode' Schubert <corecode@fs.ei.tum.de> wrote:
> Gerrit Pape wrote:
> >Hi, I struggle with importing a CVS repository into git to switch over
> >development.  In some branches there're differences in the source code
> >after importing into git.  We tracked this down to tags and branches
> >that where tagged partially in CVS, first a tag was set on one
> >subdirectory tree (sources), later on another subdirectory tree (docs).
> >This confuses cvsimport or cvsps, the tag in git after import doesn't
> >match the tag in CVS.
> 
> how do you expect this to ever work?  your tag is not a time stable one.  
> the tag spans multiple change sets.  best forget the whole tag thing, 
> seriously.  my cvs converter [1] already has a hard time reproducing 
> branches correctly.  doing this with tags is impossible.  svn can do this 
> because they don't have tags, so it is a kind of fake.  of course you could 
> do this in git as well by actually creating a branch for a tag, but it is 
> really messy.  just don't do that.

If you are importing your history you probably really do want
the tags.  As broken as they may be.  Because they represent real
state that you probably need to be able to recover in the future.

One could make the argument that most very old tags aren't worth
importing, but certainly tags related to recent releases that are
still in production use (and thus may need patching) are probably
needed.

In Git the only way to really do this is to create a branch from a
nearby commit, then modify the branch until it conforms to the tag.
Do a series of evil merges from the relevant commits of each file
until you get the branch looking the way it needs to.  *THEN*
you tag it.
 
> [1]: <http://ww2.fs.ei.tum.de/~corecode/hg/fromcvs>, not 100% yet, but 
> probably the best you can get at the moment.  i'm a perfectionist, so i 
> might never call it "done".

Also probably the fastest CVS->Git converter publiclly available,
as its backed by git-fast-import.  :)

-- 
Shawn.

Re: cvsimport/cvsps: wrong revisions in branch

[Martin Langhoff]

On 1/26/07, Simon 'corecode' Schubert <corecode@fs.ei.tum.de> wrote:
> how do you expect this to ever work?  your tag is not a time stable one.

I agree on that, but you _can_ try a range of importers. I think you
should give Keith Packard's parsecvs a try. It is memory-bound and
won't do incremental imports but it manages to import some repos
cvsimport can't do directly.

The other option is to use cvs2svn, which is the most conservative and
careful importer around, and most likely to be able to deal with your
insane tags. And then git-svnimport right after ;-)

cheers

m

Re: cvsimport/cvsps: wrong revisions in branch

[Simon 'corecode' Schubert]

[-- Attachment #1: Type: text/plain, Size: 1452 bytes --]

Martin Langhoff wrote:
> On 1/26/07, Simon 'corecode' Schubert <corecode@fs.ei.tum.de> wrote:
>> how do you expect this to ever work?  your tag is not a time stable one.
> I agree on that, but you _can_ try a range of importers. I think you
> should give Keith Packard's parsecvs a try. It is memory-bound and
> won't do incremental imports but it manages to import some repos
> cvsimport can't do directly.

whichever importer you try, it *will* fail to create pure tags equivalent to the CVS tag.  it is not possible.  of course you can do trickery and create branches to simulate crooked tags.

> The other option is to use cvs2svn, which is the most conservative and
> careful importer around, and most likely to be able to deal with your
> insane tags. And then git-svnimport right after ;-)

yes, cvs2svn is quite top notch regarding the brokenness in repos it tolerates.  however, going from svn to git is equally complicated again, as you can't nicely convert svn's copy-to-branch if it again copies files from different revisions.  you just converted the problem to svn, but you didn't make it go away.

cheers
  simon

-- 
Serve - BSD     +++  RENT this banner advert  +++    ASCII Ribbon   /"\
Work - Mac      +++  space for low €€€ NOW!1  +++      Campaign     \ /
Party Enjoy Relax   |   http://dragonflybsd.org      Against  HTML   \
Dude 2c 2 the max   !   http://golden-apple.biz       Mail + News   / \


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 252 bytes --]

Re: cvsimport/cvsps: wrong revisions in branch

[Robin Rosenberg]

fredag 26 januari 2007 07:20 skrev Simon 'corecode' Schubert:
> whichever importer you try, it *will* fail to create pure tags equivalent
> to the CVS tag.  it is not possible.  of course you can do trickery and
> create branches to simulate crooked tags.

Well, you could create a standalone tag not on any branch at all, I guess, 
just create a tree and a commit and tag corresponding to the tagged files. At 
least you wouldn't be lying about the history of the tag by implicating it 
was made through some magic merge.

-- robin

Re: cvsimport/cvsps: wrong revisions in branch

[Gerrit Pape]

On Fri, Jan 26, 2007 at 03:06:19PM +1300, Martin Langhoff wrote:
> The other option is to use cvs2svn, which is the most conservative and
> careful importer around, and most likely to be able to deal with your
> insane tags. And then git-svnimport right after ;-)

We tried that already, cvs2svn did manage the tags and branches
correctly, but then git-svnimport had the same problems.

Thanks, Gerrit.

Re: cvsimport/cvsps: wrong revisions in branch

[Jakub Narebski]

[Cc: git@vger.kernel.org]

Simon 'corecode' Schubert wrote:

> [1]: <http://ww2.fs.ei.tum.de/~corecode/hg/fromcvs>, not 100% yet, but
> probably the best you can get at the moment.  i'm a perfectionist, so
> I might never call it "done".  

Could you please add info about this work (so other can use it too)
to http://git.or.cz/gitwiki/InterfacesFrontendsAndTools in the
"Interaction with other Revision Control Systems" section? TIA.

P.S. Please use (if possible) word wrap around 72 - 76 columns.
-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git