git cvsimport branches not consistent with CVS branches

git cvsimport branches not consistent with CVS branches

[Gordon Heydon]

Hi,

After some investigation I found that git cvsimport was not importing 
branches 100% correctly with CVS.

Git and CVS do branching very differently in that CVS this is done at 
the file level (like everything else) and git does it repository wide.

So if I have a CVS repository with the files a, b and c and I branch b 
with a `cvs tab -b BRANCH test` on the branch test I will just have the 
file b.

If I do a git cvsimport on the test branch there will actually be the 
files a, b and c

What I think needs to happen is that when git cvsimport created the 
branch in the git repository it needs to delete all files from the 
branch that were not branched.

Thanks in advance.
Gordon.

Re: git cvsimport branches not consistent with CVS branches

[Brian Downing]

On Sun, Jul 08, 2007 at 10:45:10AM +1000, Gordon Heydon wrote:
> After some investigation I found that git cvsimport was not importing 
> branches 100% correctly with CVS.
> 
> Git and CVS do branching very differently in that CVS this is done at 
> the file level (like everything else) and git does it repository wide.
> 
> So if I have a CVS repository with the files a, b and c and I branch b 
> with a `cvs tab -b BRANCH test` on the branch test I will just have the 
> file b.
> 
> If I do a git cvsimport on the test branch there will actually be the 
> files a, b and c
> 
> What I think needs to happen is that when git cvsimport created the 
> branch in the git repository it needs to delete all files from the 
> branch that were not branched.

I've been vaguely working on Yet Another CVS Importer (an incremental
one; both git-cvsimport (thanks to cvsps) and tailor take about ten
minutes and a gigabyte of RAM to figure out that nothing has to happen
with my repository.  I think I can do better than that).

In thinking about this case, I think I've decided that you want an
option on what to do here.  For some repositories you're not going to
care about having extra files with the tag, and would greatly prefer
that to having to create a branch for each and every tag (assuming you
can arrange to have the correct files present otherwise; this isn't
always possible.)

For other cases, you really only want to get the subset of the files
that are tagged.  For this, I think the best arrangement would be to
make your branch, then make a commit that only deletes the files that
are not present in the CVS branch, as you said.  Then immediately make
an empty commit to the mainline (wherever you branched from) merging
from the deletion commit.  Then proceed to commit normally to the
branch.  This way, if a user chooses to merge from the branch in Git,
it won't try to delete a bunch of files in the target branch.  (I'm
having a hard time coming up with a situation where you'd want that
behavior assuming the missing files were never tagged in CVS.  A sane
usage pattern would be to tag the whole repository then "cvs rm" the
files you don't want otherwise...)

A fun third case is when a file in CVS doesn't have a branch tag,
but then you go and "cvs add" a file with the same name in a branch.
What CVS does then is to place the branch tag off the current HEAD.
I /think/ you can detect this situation by noticing that the date of
the HEAD revision and the date of the "file was created on branch ..."
revision are exactly the same.  You really need to detect this case,
because when this happens there is no real parent relationship between
the branch and the branch point.  Since CVS has that "wait for a second
to tick" delay at the end this might even be safe.  Sigh.

Don't get me started about the abomination that is vendor branches.

I /hate/ CVS.

-bcd

Re: git cvsimport branches not consistent with CVS branches

[Robin Rosenberg]

söndag 08 juli 2007 skrev Brian Downing:
> On Sun, Jul 08, 2007 at 10:45:10AM +1000, Gordon Heydon wrote:
> > After some investigation I found that git cvsimport was not importing 
> > branches 100% correctly with CVS.
> > 
> > Git and CVS do branching very differently in that CVS this is done at 
> > the file level (like everything else) and git does it repository wide.
> > 
> > So if I have a CVS repository with the files a, b and c and I branch b 
> > with a `cvs tab -b BRANCH test` on the branch test I will just have the 
> > file b.
> > 
> > If I do a git cvsimport on the test branch there will actually be the 
> > files a, b and c
> > 
> > What I think needs to happen is that when git cvsimport created the 
> > branch in the git repository it needs to delete all files from the 
> > branch that were not branched.
> 
> I've been vaguely working on Yet Another CVS Importer (an incremental
> one; both git-cvsimport (thanks to cvsps) and tailor take about ten
> minutes and a gigabyte of RAM to figure out that nothing has to happen
> with my repository.  I think I can do better than that).

Corecode's fromcvs is pretty fast and incremental and AFAIK accurate. I had 
plenty problems with cvsimport, but fromcvs keeps in sync with the CVS repo.
Get  it at http://ww2.fs.ei.tum.de/~corecode/hg/fromcvs/ .

It does not convert regular tags, only branches, however so there is something to 
do for those that want a complete cvs import.

> In thinking about this case, I think I've decided that you want an
> option on what to do here.  For some repositories you're not going to
> care about having extra files with the tag, and would greatly prefer
> that to having to create a branch for each and every tag (assuming you
> can arrange to have the correct files present otherwise; this isn't
> always possible.)
> 
> For other cases, you really only want to get the subset of the files
> that are tagged.  For this, I think the best arrangement would be to
> make your branch, then make a commit that only deletes the files that
> are not present in the CVS branch, as you said.  Then immediately make

fromcvs drops the files that do not have the branch tag. It is pretty simple to change
the behaviour ( http://rosenberg.homelinux.net/gitweb/gitweb.cgi?p=FROMCVS.git;a=commitdiff;h=eb1c159bcc0d79eab182da3d7040ac62b52fd297 )
although this should be a switch and not hard coded.

-- robin

Re: git cvsimport branches not consistent with CVS branches

[Johannes Schindelin]

Hi,

On Sun, 8 Jul 2007, Robin Rosenberg wrote:

> Corecode's fromcvs is pretty fast and incremental and AFAIK accurate.

The only problem is that it is a misnomer: it is not fromcvs, but fromrcs, 
since you have to have access at the _files_. This is not always 
possible.

Ciao,
Dscho

Re: git cvsimport branches not consistent with CVS branches

[Robin Rosenberg]

söndag 08 juli 2007 skrev Johannes Schindelin:
> Hi,
> 
> On Sun, 8 Jul 2007, Robin Rosenberg wrote:
> 
> > Corecode's fromcvs is pretty fast and incremental and AFAIK accurate.
> 
> The only problem is that it is a misnomer: it is not fromcvs, but fromrcs, 
> since you have to have access at the _files_. This is not always 
> possible.

A small problem considering that fromcvs works and the other don't. My guess
is that fromcvs kan be "fixed" to do remote access much more easily than cvsimport
can be "fixed" to import an archive properly.

-- robin

Re: git cvsimport branches not consistent with CVS branches

[Steffen Prohaska]

On Jul 8, 2007, at 12:53 PM, Robin Rosenberg wrote:

> Corecode's fromcvs is pretty fast and incremental and AFAIK  
> accurate. I had
> plenty problems with cvsimport, but fromcvs keeps in sync with the  
> CVS repo.
> Get  it at http://ww2.fs.ei.tum.de/~corecode/hg/fromcvs/ .
>
> It does not convert regular tags, only branches, however so there  
> is something to
> do for those that want a complete cvs import.

Did anyone compare
    * git-cvsimport with cvsps patches from [1]
    * parsecvs [2]
    * fromcvs
and can give a recommendation?

My experience with plain git-cvsimport (without cvsps patches from [1])
is that it has a lot of problems. I'd recommend not to use it for
incremental import and be very suspicious about the git repository
created by git-cvsimport. You need to carefully validate the repository.
It's likely that you need to fix imported branches. The trunk seems to
be ok.

	Steffen

[1] http://ydirson.free.fr/en/software/scm/cvsps.html
[2] http://anongit.freedesktop.org/git/users/keithp/repos/parsecvs.git/

Re: git cvsimport branches not consistent with CVS branches

[Brian Downing]

On Sun, Jul 08, 2007 at 12:47:16PM +0100, Johannes Schindelin wrote:
> The only problem is that it is a misnomer: it is not fromcvs, but
> fromrcs, since you have to have access at the _files_. This is not
> always possible.

For the problem space I'm interested in (a /fast/ incremental CVS
importer) I think access to the RCS files is a must.  You can use inode
signatures to know which files you have to look at at all since the last
run, and you can abort your parse at the end of the delta section if no
new revisions appeared (rather than continuing to parse the deltatext
section, the largest part of the file).

For the repository I'm interested in (work), running a "real" cvs update
takes two or three minutes on a LAN.  Running a full rlog takes about
10 or so.  With the tricks above, I can import a single file change in
seconds instead.  (That includes the time to rsync the CVS repository
to my local mirror.)

Granted, though, not everyone can get access to the RCS files.

(The name's not really a misnomer though.  There's tons of CVS-specific
crap you have to deal with in the RCS file to successfully import it --
magic branch revision symbols, horrible vendor branch behavior, etc.)

I will try fromcvs; I hadn't looked at it, since from the documentation
it looked like it wasn't incremental.

-bcd

Re: git cvsimport branches not consistent with CVS branches

[Josh Triplett]

Steffen Prohaska wrote:
> On Jul 8, 2007, at 12:53 PM, Robin Rosenberg wrote:
>> Corecode's fromcvs is pretty fast and incremental and AFAIK  
>> accurate. I had
>> plenty problems with cvsimport, but fromcvs keeps in sync with the  
>> CVS repo.
>> Get  it at http://ww2.fs.ei.tum.de/~corecode/hg/fromcvs/ .
>>
>> It does not convert regular tags, only branches, however so there  
>> is something to
>> do for those that want a complete cvs import.
> 
> Did anyone compare
>     * git-cvsimport with cvsps patches from [1]
>     * parsecvs [2]
>     * fromcvs
> and can give a recommendation?
> 
> My experience with plain git-cvsimport (without cvsps patches from [1])
> is that it has a lot of problems. I'd recommend not to use it for
> incremental import and be very suspicious about the git repository
> created by git-cvsimport. You need to carefully validate the repository.
> It's likely that you need to fix imported branches. The trunk seems to
> be ok.

I've used both parsecvs and git-cvsimport, and I can definitely say from
experience that cvsimport badly munges history, and parsecvs seems to handle
it correctly.  parsecvs does have the limitations that it requires the ,v
files and doesn't work incrementally, making it great for project repository
conversions and painful for just wanting to use git on a CVS-(mis)managed
project.

I've never used fromcvs.

One idea to mitigate one of the limitations: how about modifying the CVS
server to add an extension that supports downloading full ,v files?  CVS has
had server extensions in the past, such as for ls/rls.  Once support for this
extension makes its way out to most CVS servers, this problem goes away.

- Josh Triplett