Problem with git-cvsimport

Problem with git-cvsimport

[Thomas Pasch]

Hello,

using git-cvsimport (1.5.3.4), it dies with

Update
guidance-common/src/java/com/jentro/manager/guidance/common/servlet/IconServlet.java:
2104 bytes
Tree ID 01cb84cbee2e70a712459be6601b993603eed5bd
Parent ID dcd8dc76f4638d1994165070c9813202992d546a
Committed patch 3775 (bmw +0000 2004-10-14 11:10:43)
Commit ID 53c68066f71651b057884e1101cda3967070724d
Fetching
guidance-common/src/java/com/jentro/manager/guidance/common/serverapi/GuidanceException.java
  v 1.14.4.2
Update
guidance-common/src/java/com/jentro/manager/guidance/common/serverapi/GuidanceException.java:
3718 bytes
Tree ID 886268190ac2cb28b5f1e6cdb309054bcb8fa38e
Parent ID 53c68066f71651b057884e1101cda3967070724d
Merge parent branch: master
fatal: Not a valid object name master
Use of uninitialized value in chomp at /usr/bin/git-cvsimport line 766.
Use of uninitialized value in pattern match (m//) at
/usr/bin/git-cvsimport line 527.
Use of uninitialized value in concatenation (.) or string at
/usr/bin/git-cvsimport line 767.
Cannot get commit id ():

What can I do to avoid this problem?

Cheers,

Thomas

Re: Problem with git-cvsimport

[Jan Wielemaker]

On Tuesday 09 October 2007 11:25, Thomas Pasch wrote:
> Hello,
>
> using git-cvsimport (1.5.3.4), it dies with
>
> Update
> guidance-common/src/java/com/jentro/manager/guidance/common/servlet/IconSer
>vlet.java: 2104 bytes
> Tree ID 01cb84cbee2e70a712459be6601b993603eed5bd
> Parent ID dcd8dc76f4638d1994165070c9813202992d546a
> Committed patch 3775 (bmw +0000 2004-10-14 11:10:43)
> Commit ID 53c68066f71651b057884e1101cda3967070724d
> Fetching
> guidance-common/src/java/com/jentro/manager/guidance/common/serverapi/Guida
>nceException.java v 1.14.4.2
> Update
> guidance-common/src/java/com/jentro/manager/guidance/common/serverapi/Guida
>nceException.java: 3718 bytes
> Tree ID 886268190ac2cb28b5f1e6cdb309054bcb8fa38e
> Parent ID 53c68066f71651b057884e1101cda3967070724d
> Merge parent branch: master
> fatal: Not a valid object name master
> Use of uninitialized value in chomp at /usr/bin/git-cvsimport line 766.
> Use of uninitialized value in pattern match (m//) at
> /usr/bin/git-cvsimport line 527.
> Use of uninitialized value in concatenation (.) or string at
> /usr/bin/git-cvsimport line 767.
> Cannot get commit id ():
>
> What can I do to avoid this problem?

I've had some similar problem.  I've converted two big old repositories by
first converting to SVN using:

	cvs2svn -s myrepo-svn /path/to/cvsmodule
	git-svnimport -i -u -C /path/to-git file://myrepo-svn

Worked like a charm

	Cheers --- Jan

Re: Problem with git-cvsimport

[Gerald (Jerry) Carter]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jan Wielemaker wrote:

> I've had some similar problem.  I've converted two big old 
> repositories by first converting to SVN using:
> 
> 	cvs2svn -s myrepo-svn /path/to/cvsmodule
> 	git-svnimport -i -u -C /path/to-git file://myrepo-svn

I would actually plug using cvs2svn to convert directly to git.
See this thread for Michael's original announcement.

   http://marc.info/?l=git&m=118592701426175&w=2

I'm in the process of drafting Samba's own git repos from
the CVS and SVN history (http://gitweb.samba.org/).




cheers, jerry
=====================================================================
Samba                                    ------- http://www.samba.org
Centeris                         -----------  http://www.centeris.com
"What man is a man who does not make the world better?"      --Balian
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHC4BJIR7qMdg1EfYRAlQ4AKDctlXFv0kcT51sA6P99qjVrPJ+MgCfWkCB
wPSf6l06UIlz0HERasHbryg=
=zSSf
-----END PGP SIGNATURE-----

Re: Problem with git-cvsimport

[Eyvind Bernhardsen]

On 9. okt.. 2007, at 15.21, Gerald (Jerry) Carter wrote:

> I would actually plug using cvs2svn to convert directly to git.
> See this thread for Michael's original announcement.
>
>   http://marc.info/?l=git&m=118592701426175&w=2

Seconded!  I've tried git-cvsimport, parsecvs, fromcvs, and cvs2svn on  
my employer's many large CVS modules, and cvs2svn is the only one that  
has never mangled an import.

That said, it is a work in progress, so there are some caveats:

* Setting up the direct conversion to git is more work than it should  
(and probably will, eventually) be.

* There is no support for incremental importing, and git-cvsimport  
_will_ mess up your git repository sooner or later if you try to use  
it for subsequent incremental imports.

* Tags each get a branch with a single commit, with the actual tag  
pointing to that commit.  This makes it harder than necessary to  
figure out what the history looks like; gitk's default view won't show  
any tags, for example, since it only shows the master branch and not  
the single-commit tag branches.

* Branches all get a useless commit at their branch point.  All  
branches from the main branch appear to be merged from the vendor  
branch (ie, the useless commit has the vendor branch as an extra  
parent), which might make sense to someone who knows what the vendor  
branch is for, but makes no sense to me.  This combined with the  
previous point makes "gitk --all" look needlessly spaghetti-like if  
you have a slightly complicated CVS history.

To sum up, cvs2svn gets the important stuff right, but has some sharp  
corners you need to watch so you don't put an eye out.

Eyvind Bernhardsen

Re: Problem with git-cvsimport

[Michael Haggerty]

Eyvind Bernhardsen wrote:
> On 9. okt.. 2007, at 15.21, Gerald (Jerry) Carter wrote:
>> I would actually plug using cvs2svn to convert directly to git.
>> See this thread for Michael's original announcement.
>>
>>   http://marc.info/?l=git&m=118592701426175&w=2
> 
> Seconded!  I've tried git-cvsimport, parsecvs, fromcvs, and cvs2svn on
> my employer's many large CVS modules, and cvs2svn is the only one that
> has never mangled an import.

I'm glad this worked.

> That said, it is a work in progress, so there are some caveats:
> 
> [...]
> 
> * Tags each get a branch with a single commit, with the actual tag
> pointing to that commit.  This makes it harder than necessary to figure
> out what the history looks like; gitk's default view won't show any
> tags, for example, since it only shows the master branch and not the
> single-commit tag branches.

I just fixed this in cvs2svn trunk r4213.  Now it reuses a single branch
called 'refs/heads/TAG.FIXUP' whenever it needs to make a tag fixup
branch, and it resets that branch when done.  (Resetting the tag fixup
branch changes it to 0000000000000000000000000000000000000000 but
doesn't really delete it; I don't know the ramifications of that but at
least it doesn't appear in gitk output any more.)

> * Branches all get a useless commit at their branch point.  All branches
> from the main branch appear to be merged from the vendor branch (ie, the
> useless commit has the vendor branch as an extra parent), which might
> make sense to someone who knows what the vendor branch is for, but makes
> no sense to me.  This combined with the previous point makes "gitk
> --all" look needlessly spaghetti-like if you have a slightly complicated
> CVS history.

I assume that the "useless commit" that you are referring to is the one
with log message "This commit was manufactured by cvs2svn to create
branch 'BRANCH'."  Is that correct?

I'm not a git expert, so I don't know whether these commits are in fact
useless.  But let me explain the reason I put them in and you can tell
me whether it is nonsense.

When you branch a file in CVS, CVS notes that the branch exists in the
file but doesn't record an author, log message, or timestamp.  The
contents of a file just after it is branched are exactly the same as the
contents on the parent branch.  Moreover, different files can be added
to a branch from different parent branches.

The intended purpose of the "useless commit" in the git output of
cvs2svn is to record the fact that a branch was created, to record
exactly which files exist on the branch at the time it was created, and
to record the source branches of the file on the branch.

I imagine that *if* a branch is created with a single parent branch, and
*if* each and every file from the parent branch is added to the new
branch, then it is possible that the "useless commit" could be omitted.
 But this decision would require information that cvs2svn doesn't
currently have at that stage of the conversion, and keeping the
necessary records would be quite expensive.

But in the general case, it doesn't seem to me that the commits are
really useless.  Am I wrong?  If so, please tell me what should be
changed in the git-fast-import data that is output when a branch is
created (e.g., for the main-cvsrepos in the cvs2svn test suite).

Regarding the superfluous vendorbranch parent: vendor branches are an
obscure CVS feature for tracking upstream sources.  The file contents on
the vendorbranch are typically exactly the same as that on trunk, and if
a branch is created while the vendorbranch is active, CVS doesn't record
whether the branch's parent was trunk or the vendorbranch.  Haven't yet
built the heuristics into cvs2svn to make this decision more
intelligently, so sometimes "vendorbranch" is listed as a branch parent
when it could be omitted.

> To sum up, cvs2svn gets the important stuff right, but has some sharp
> corners you need to watch so you don't put an eye out.

Thanks for the feedback!

Michael

[1] http://www.kernel.org/pub/software/scm/git/docs/git-fast-import.html

Re: Problem with git-cvsimport

[Eyvind Bernhardsen]

On 10. okt.. 2007, at 04.34, Michael Haggerty wrote:

[...]

Thanks for the response!  I should have explained that my goal is to  
convince my coworkers that git is worth the effort to learn, and IMO,  
making imported git repositories look as "clean" as possible to  
someone familiar with the CVS repository is an important part of  
that.  My suggestions and complaints should be seen in that light :)

>> * Tags each get a branch with a single commit, with the actual tag
>> pointing to that commit.  This makes it harder than necessary to  
>> figure
>> out what the history looks like; gitk's default view won't show any
>> tags, for example, since it only shows the master branch and not the
>> single-commit tag branches.
>
> I just fixed this in cvs2svn trunk r4213.  Now it reuses a single  
> branch
> called 'refs/heads/TAG.FIXUP' whenever it needs to make a tag fixup
> branch, and it resets that branch when done.  (Resetting the tag fixup
> branch changes it to 0000000000000000000000000000000000000000 but
> doesn't really delete it; I don't know the ramifications of that but  
> at
> least it doesn't appear in gitk output any more.)

Great!  I'll try this out at the first opportunity.

The main problem I have with the current branched-tag behaviour is  
that nothing exists on the branch to show that a tag is branched off,  
so no tags are shown at all when running gitk on a single branch.  A  
git newbie who is familiar with the CVS repository will probably just  
assume that tags haven't been imported, which is unfortunate.

Tagging the branch point is an obvious solution to that problem, but I  
wonder if branching for tag fixups could be made optional.  Instead of  
representing a tag with differences to the commit it's closest to like  
this:

     o--*--o--...
         \--t

(where "*" represents the closest commit on the branch, and "t"  
represents the fixup commit), might the following be possible and/or  
desirable?

     o--*--t--r--o--...

"r" being a commit that reverts the changes in "t".

It might just be me, but I think of a tag as being "on" a branch even  
when it has been messed with (sliding, omitting files, etc), and a  
linear history preserves that illusion.

>> * Branches all get a useless commit at their branch point.  All  
>> branches
>> from the main branch appear to be merged from the vendor branch  
>> (ie, the
>> useless commit has the vendor branch as an extra parent), which might
>> make sense to someone who knows what the vendor branch is for, but  
>> makes
>> no sense to me.  This combined with the previous point makes "gitk
>> --all" look needlessly spaghetti-like if you have a slightly  
>> complicated
>> CVS history.
>
> I assume that the "useless commit" that you are referring to is the  
> one
> with log message "This commit was manufactured by cvs2svn to create
> branch 'BRANCH'."  Is that correct?

Yes.  And I'm regretting my choice of words already.  "Unexpected  
commit" would have been better.

> I'm not a git expert, so I don't know whether these commits are in  
> fact
> useless.  But let me explain the reason I put them in and you can tell
> me whether it is nonsense.

[...]

I know barely enough about git (or CVS, for that matter) to be  
dangerous, but your explanation makes sense to me.

> I imagine that *if* a branch is created with a single parent branch,  
> and
> *if* each and every file from the parent branch is added to the new
> branch, then it is possible that the "useless commit" could be  
> omitted.
> But this decision would require information that cvs2svn doesn't
> currently have at that stage of the conversion, and keeping the
> necessary records would be quite expensive.

Right; that is how I think of a branch, even if CVS doesn't.  To me,  
the extra commit is useless because it should be identical to the  
branch point commit on the parent branch.  I don't think having an  
extra commit there is a huge problem, though.

[...]

> Regarding the superfluous vendorbranch parent: vendor branches are an
> obscure CVS feature for tracking upstream sources.  The file  
> contents on
> the vendorbranch are typically exactly the same as that on trunk,  
> and if
> a branch is created while the vendorbranch is active, CVS doesn't  
> record
> whether the branch's parent was trunk or the vendorbranch.  Haven't  
> yet
> built the heuristics into cvs2svn to make this decision more
> intelligently, so sometimes "vendorbranch" is listed as a branch  
> parent
> when it could be omitted.

I don't think I've ever seen the vendor branch used for anything but  
the initial import (creation) of a repository, and then only because  
it is a required parameter to "cvs import".  Is there any chance you  
could add an option to ignore the vendor branch?

> Thanks for the feedback!

Thanks for making cvs2svn the best CVS-to-git conversion tool :)  Now  
if it would only support incremental importing...

Eyvind Bernhardsen

Re: Problem with git-cvsimport

[Mike Snitzer]

On 10/10/07, Eyvind Bernhardsen <eyvind-git-list@orakel.ntnu.no> wrote:
...
>
> Thanks for making cvs2svn the best CVS-to-git conversion tool :)  Now
> if it would only support incremental importing...

Michael,

I second this question: is there any chance incremental importing will
be implemented in cvs2svn?

I've not used cvs2svn much and when I did it was for svn not git; but
given that git-cvsimport is known to mess up your git repo (as Eyvind
pointed out earlier) there doesn't appear to be any reliable tools to
allow for incrementally importing from cvs to git.

Are others using a tool for reliably importing from cvs to git?

Re: Problem with git-cvsimport

[Mike Snitzer]

On 10/30/07, Mike Snitzer <snitzer@gmail.com> wrote:
> On 10/10/07, Eyvind Bernhardsen <eyvind-git-list@orakel.ntnu.no> wrote:
> ...
> >
> > Thanks for making cvs2svn the best CVS-to-git conversion tool :)  Now
> > if it would only support incremental importing...
>
> Michael,
>
> I second this question: is there any chance incremental importing will
> be implemented in cvs2svn?
>
> I've not used cvs2svn much and when I did it was for svn not git; but
> given that git-cvsimport is known to mess up your git repo (as Eyvind
> pointed out earlier) there doesn't appear to be any reliable tools to
> allow for incrementally importing from cvs to git.
>
> Are others using a tool for reliably importing from cvs to git?

After reading the fairly recent "cvs2svn conversion directly to git
ready for experimentation" thread it is clear that its doable but
hasn't been done (seeing as you were looking for volunteers to do it).

Sorry for the noise,
Mike

Re: Problem with git-cvsimport

[Robin Rosenberg]

tisdag 30 oktober 2007 skrev Mike Snitzer:
> On 10/10/07, Eyvind Bernhardsen <eyvind-git-list@orakel.ntnu.no> wrote:
> ...
> >
> > Thanks for making cvs2svn the best CVS-to-git conversion tool :)  Now
> > if it would only support incremental importing...
> 
> Michael,
> 
> I second this question: is there any chance incremental importing will
> be implemented in cvs2svn?
> 
> I've not used cvs2svn much and when I did it was for svn not git; but
> given that git-cvsimport is known to mess up your git repo (as Eyvind
> pointed out earlier) there doesn't appear to be any reliable tools to
> allow for incrementally importing from cvs to git.
> 
> Are others using a tool for reliably importing from cvs to git?

I use fromcvs which is *very* fast, and quite memory conservative compared to 
the others and seems reliable so far (six months). It probably breaks on 
exotic variants of branches though, but I don't have those / don't care about 
them.

Do not push into the same repo fromcvs works on. I don't understand why, but I 
pushed once and *poof* the conversion went bad. 

Drawbacks, more dependencies and access to the rcs files is required and tags 
are not converted.

-- robin

Re: Problem with git-cvsimport

[Aidan Van Dyk]

Robin Rosenberg wrote:

> I use fromcvs which is *very* fast, and quite memory conservative compared
> to the others and seems reliable so far (six months). It probably breaks
> on exotic variants of branches though, but I don't have those / don't care
> about them.

I actually use fromcvs for a few repositories, and actually started using it
on repositories where cvsps (and git-cvsimport) fail.

> Drawbacks, more dependencies and access to the rcs files is required and
> tags are not converted.

Most projects have "rsyncs" or "tarballs" of their CVS repository available,
making fromcvs possible on most of them.  And CVS tags, well they are about
as good as CVS $Keywords$.

Re: Problem with git-cvsimport

[Michael Haggerty]

Mike Snitzer wrote:
> On 10/10/07, Eyvind Bernhardsen <eyvind-git-list@orakel.ntnu.no> wrote:
> ...
>> Thanks for making cvs2svn the best CVS-to-git conversion tool :)  Now
>> if it would only support incremental importing...
> 
> I second this question: is there any chance incremental importing will
> be implemented in cvs2svn?

Unfortunately, no, there is not much chance that I will implement this.
 I wouldn't be interested in a works-most-of-the-time solution, and a
reliable solution would take weeks to implement.

If somebody else wants to implement this feature, I would be happy to
help him get started, answer questions, discuss the design, etc.  Or if
somebody wants to sponsor the work, I might be able to justify working
on it myself.  But otherwise, I'm afraid it is unlikely to happen.

> I've not used cvs2svn much and when I did it was for svn not git; but
> given that git-cvsimport is known to mess up your git repo (as Eyvind
> pointed out earlier) there doesn't appear to be any reliable tools to
> allow for incrementally importing from cvs to git.

That's because it is quite a tricky problem, especially since CVS allows
history to be changed retroactively; for example,

- shift a tag to a different file revision

- add an existing tag to a new file or remove it from an old file

- delete ("obsolete") old revisions

- change files from vendor branches to main line of development

- even nastier server-side repository manipulations like deleting an RCS
file, renaming a file, etc.

These things really happen in the topsy-turvy CVS world; indeed, they
are a part of many organizations' standard workflow.

cvs2svn uses repository-wide information in the heuristics that it uses
to determine changesets, choose branch parents, fix clock skew, etc.
Therefore the naive approach of running a full conversion a second time
and just skipping over the revisions that were handled during the first
conversion would not even begin to work.  (I believe that this is the
approach of cvsps, which uses mostly local information to determine
changesets.)

I think the correct approach would involve recording the "frontier" of
the CVS repository, then at the next incremental conversion:

1. compare the current CVS repository to the recorded information

2. emit "fixup" changesets to reflect any CVS changes that happened
behind the previous "frontier".

3. emit changesets to reflect CVS changes beyond the frontier.

It is step 2 that is IMO the trickiest because it is so open-ended, and
modern SCMs don't allow all of the corresponding operations in any
straightforward way.  Presumably one would have to prohibit some of the
nastier CVS tricks and abort the incremental conversion if any are detected.

Furthermore, for many use-cases of incremental conversion the conversion
would have to run quickly.  Therefore, the incremental conversion code
should be written with a strong emphasis on achieving good performance.

Michael

problem with git-cvsimport

[picca]

Hello

I tryed to import this sourceforge project with git-cvsimport
The command was:

git-cvsimport -v -d :pserver:anonymous@tango-cs.cvs.sourceforge.net:/cvsroot/tango-cs -C tango.git tango

unfortunetly the command stop with this error:

skip patchset 3713: 1159522283 before 1159522323
skip patchset 3714: 1159522323 before 1159522323
DONE.
fatal: Needed a single revision
fatal: Needed a single revision
Usage: /usr/bin/git-merge [-n] [--no-commit] [--squash] [-s <strategy>]... <merg
e-message> <head> <remote>+
Could not merge origin into the current branch.

I am using the debian testing git-core version aka 1.4.1.1

Why ?


Have a nice day.