parsecvs losing files

parsecvs losing files

[Alex Bennee]

Hi,

I've been using a daily cron job running parsecvs to convert an uber
CVS tree into git. It works quite well and is substantially faster
than all the other methods I have tried. It also re-creates the same
history every time which is something I haven't managed to do with the
other tools.

Anyway today I noticed it had failed to import a sub-directory of the
project (not a bit I usually build). However looking at the import log
from parsecvs I see that the file was read by parsecvs. It looks as
though it should have made it into the git repo. The only thing that
seems different from the other modules is that the files where
imported once and don't seem to have been touched since. This may of
caused parsecvs to get confused?

Questions:

1. Has anyone seen this behaviour with parsecvs?
2. Is there any way to list all the tree objects in a repo so I can
search to see if the files exist in a tree but just not this branch?

-- 
Alex, homepage: http://www.bennee.com/~alex/

Re: parsecvs losing files

[Alex Bennee]

On Wed, May 28, 2008 at 3:50 PM, Alex Bennee <kernel-hacker@bennee.com> wrote:
> Hi,
> <snip>
> Anyway today I noticed it had failed to import a sub-directory of the
> project (not a bit I usually build). However looking at the import log
> from parsecvs I see that the file was read by parsecvs. It looks as
> though it should have made it into the git repo. The only thing that
> seems different from the other modules is that the files where
> imported once and don't seem to have been touched since. This may of
> caused parsecvs to get confused?

Well in answer to myself parsecvs does get confused. In an example
failed to import file:

Load:                          third-party/libxml/runtest.c,v   8207 of 79070
/export/git/master.cvs/third-party/libxml/runtest.c,v spliced:
	 1.1.1.1
	 1.1
Sorted heads for /export/git/master.cvs/third-party/libxml/runtest.c,v
	master 1.1
	master > BRANCH-3-5-branch 1.1.1.1.0.2
	master > BRANCH-3-5-16-branch 1.1.1.1.0.4
building branches for /export/git/master.cvs/third-party/libxml/runtest.c,v
file sha1: b694d565caf10fedbc7566f2bf15b893c57d5a19
file sha1: b694d565caf10fedbc7566f2bf15b893c57d5a19
file has 2 revisions

An lo, looking at the branches mentioned these missing files are
there. Trouble is the files should be in a number of branches, looking
at the ,v file in question:

head	1.1;
branch	1.1.1;
access;
symbols
	BRANCH-3-5-26-1:1.1.1.1
	BRANCH-3-5-25-1:1.1.1.1
	BRANCH-3-5-24-1:1.1.1.1
	BRANCH-3-5-22-3:1.1.1.1
	BRANCH-3-5-22-1:1.1.1.1
	BRANCH-3-5-21-1:1.1.1.1
	BRANCH-3-5-20-1:1.1.1.1
	BRANCH-3-5-16-7:1.1.1.1
	BRANCH-3-5-19-1:1.1.1.1
	BRANCH-3-5-16-6:1.1.1.1
	BRANCH-3-5-16-5:1.1.1.1
	BRANCH-3-5-18-1:1.1.1.1
	BRANCH-3-5-16-4:1.1.1.1
	BRANCH-3-5-16-branch:1.1.1.1.0.4
	BRANCH-3-5-17-1:1.1.1.1
	BRANCH-3-5-16-3:1.1.1.1
	BRANCH-3-5-16-2:1.1.1.1
	BRANCH-3-5-16-1:1.1.1.1
	post-merge-of-ajz-post-3-5-branch:1.1.1.1
	pre-merge-of-ajz-post-3-5-branch:1.1.1.1
	BRANCH-3-5-15-1:1.1.1.1
	BRANCH-3-5-14-1:1.1.1.1
	BRANCH-3-5-branch:1.1.1.1.0.2
	BRANCH-3-5-13-1:1.1.1.1
	BRANCH-3-5-12-1:1.1.1.1
	BRANCH-3-5-11-1:1.1.1.1
	BRANCH-3-5-10-1:1.1.1.1
	BRANCH-3-5-9-2:1.1.1.1
	BRANCH-3-3-20-red-e1-opt-branch:1.1.1.1
	GNOME_LIBXML2_2_6_29:1.1.1.1
	GNOME:1.1.1;
locks; strict;
comment	@ * @;


1.1
date	2007.08.16.15.59.35;	author jbpn;	state Exp;
branches
	1.1.1.1;
next	;

1.1.1.1
date	2007.08.16.15.59.35;	author jbpn;	state Exp;
branches;
next	;


desc
@@



1.1
log
@Initial revision
<snip rest of file>

I'm not sure why parsecvs got this wrong, I'm digging through it but
I'm a little fuzzy where lex/bison/whatever is involved. I notice
looking at the log for some of the files that did make it that the CVS
revisions for all the branches have a .0.[something] suffix which the
missing branches in this case don't have. However CVS is very sure the
file exists in these other branches.

Could this be something to do with explicit branches and sticky branch
tags? Or just a straight bug in parsecvs?


-- 
Alex, homepage: http://www.bennee.com/~alex/

Re: parsecvs losing files

[Pierre Habouzit]

[-- Attachment #1: Type: text/plain, Size: 3891 bytes --]

  Sorry for the top posting but the git list isn't really the upstream
for parsecvs. I'm now Cc-ing keithp who is the author :)

On Wed, May 28, 2008 at 04:53:30PM +0000, Alex Bennee wrote:
> On Wed, May 28, 2008 at 3:50 PM, Alex Bennee <kernel-hacker@bennee.com> wrote:
> > Hi,
> > <snip>
> > Anyway today I noticed it had failed to import a sub-directory of the
> > project (not a bit I usually build). However looking at the import log
> > from parsecvs I see that the file was read by parsecvs. It looks as
> > though it should have made it into the git repo. The only thing that
> > seems different from the other modules is that the files where
> > imported once and don't seem to have been touched since. This may of
> > caused parsecvs to get confused?
> 
> Well in answer to myself parsecvs does get confused. In an example
> failed to import file:
> 
> Load:                          third-party/libxml/runtest.c,v   8207 of 79070
> /export/git/master.cvs/third-party/libxml/runtest.c,v spliced:
> 	 1.1.1.1
> 	 1.1
> Sorted heads for /export/git/master.cvs/third-party/libxml/runtest.c,v
> 	master 1.1
> 	master > BRANCH-3-5-branch 1.1.1.1.0.2
> 	master > BRANCH-3-5-16-branch 1.1.1.1.0.4
> building branches for /export/git/master.cvs/third-party/libxml/runtest.c,v
> file sha1: b694d565caf10fedbc7566f2bf15b893c57d5a19
> file sha1: b694d565caf10fedbc7566f2bf15b893c57d5a19
> file has 2 revisions
> 
> An lo, looking at the branches mentioned these missing files are
> there. Trouble is the files should be in a number of branches, looking
> at the ,v file in question:
> 
> head	1.1;
> branch	1.1.1;
> access;
> symbols
> 	BRANCH-3-5-26-1:1.1.1.1
> 	BRANCH-3-5-25-1:1.1.1.1
> 	BRANCH-3-5-24-1:1.1.1.1
> 	BRANCH-3-5-22-3:1.1.1.1
> 	BRANCH-3-5-22-1:1.1.1.1
> 	BRANCH-3-5-21-1:1.1.1.1
> 	BRANCH-3-5-20-1:1.1.1.1
> 	BRANCH-3-5-16-7:1.1.1.1
> 	BRANCH-3-5-19-1:1.1.1.1
> 	BRANCH-3-5-16-6:1.1.1.1
> 	BRANCH-3-5-16-5:1.1.1.1
> 	BRANCH-3-5-18-1:1.1.1.1
> 	BRANCH-3-5-16-4:1.1.1.1
> 	BRANCH-3-5-16-branch:1.1.1.1.0.4
> 	BRANCH-3-5-17-1:1.1.1.1
> 	BRANCH-3-5-16-3:1.1.1.1
> 	BRANCH-3-5-16-2:1.1.1.1
> 	BRANCH-3-5-16-1:1.1.1.1
> 	post-merge-of-ajz-post-3-5-branch:1.1.1.1
> 	pre-merge-of-ajz-post-3-5-branch:1.1.1.1
> 	BRANCH-3-5-15-1:1.1.1.1
> 	BRANCH-3-5-14-1:1.1.1.1
> 	BRANCH-3-5-branch:1.1.1.1.0.2
> 	BRANCH-3-5-13-1:1.1.1.1
> 	BRANCH-3-5-12-1:1.1.1.1
> 	BRANCH-3-5-11-1:1.1.1.1
> 	BRANCH-3-5-10-1:1.1.1.1
> 	BRANCH-3-5-9-2:1.1.1.1
> 	BRANCH-3-3-20-red-e1-opt-branch:1.1.1.1
> 	GNOME_LIBXML2_2_6_29:1.1.1.1
> 	GNOME:1.1.1;
> locks; strict;
> comment	@ * @;
> 
> 
> 1.1
> date	2007.08.16.15.59.35;	author jbpn;	state Exp;
> branches
> 	1.1.1.1;
> next	;
> 
> 1.1.1.1
> date	2007.08.16.15.59.35;	author jbpn;	state Exp;
> branches;
> next	;
> 
> 
> desc
> @@
> 
> 
> 
> 1.1
> log
> @Initial revision
> <snip rest of file>
> 
> I'm not sure why parsecvs got this wrong, I'm digging through it but
> I'm a little fuzzy where lex/bison/whatever is involved. I notice
> looking at the log for some of the files that did make it that the CVS
> revisions for all the branches have a .0.[something] suffix which the
> missing branches in this case don't have. However CVS is very sure the
> file exists in these other branches.
> 
> Could this be something to do with explicit branches and sticky branch
> tags? Or just a straight bug in parsecvs?
> 
> 
> -- 
> Alex, homepage: http://www.bennee.com/~alex/
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [WORKAROUND] parsecvs losing files

[Alex Bennee]

On Wed, May 28, 2008 at 6:03 PM, Pierre Habouzit <madcoder@debian.org> wrote:
>  Sorry for the top posting but the git list isn't really the upstream
> for parsecvs. I'm now Cc-ing keithp who is the author :)

I shall keep CC'ing git@ for visibility if anyone else comes across this :-)

>
> On Wed, May 28, 2008 at 04:53:30PM +0000, Alex Bennee wrote:
>> On Wed, May 28, 2008 at 3:50 PM, Alex Bennee <kernel-hacker@bennee.com> wrote:
>> > Hi,
>> > <snip>
>> > Anyway today I noticed it had failed to import a sub-directory of the
>> > project (not a bit I usually build).
<snip>
>>
>> Well in answer to myself parsecvs does get confused. In an example
>> failed to import file:
>>
>> Load:                          third-party/libxml/runtest.c,v   8207 of 79070
>> /export/git/master.cvs/third-party/libxml/runtest.c,v spliced:
>>        1.1.1.1
>>        1.1
>> Sorted heads for /export/git/master.cvs/third-party/libxml/runtest.c,v
>>       master 1.1
>>       master > BRANCH-3-5-branch 1.1.1.1.0.2
>>       master > BRANCH-3-5-16-branch 1.1.1.1.0.4
>> building branches for /export/git/master.cvs/third-party/libxml/runtest.c,v
>> file sha1: b694d565caf10fedbc7566f2bf15b893c57d5a19
>> file sha1: b694d565caf10fedbc7566f2bf15b893c57d5a19
>> file has 2 revisions
>>
>> An lo, looking at the branches mentioned these missing files are
>> there. Trouble is the files should be in a number of branches, looking
>> at the ,v file in question:
<snip>
>>       BRANCH-3-3-20-red-e1-opt-branch:1.1.1.1
<snip>
>> I notice
>> looking at the log for some of the files that did make it that the CVS
>> revisions for all the branches have a .0.[something] suffix which the
>> missing branches in this case don't have.
<snip>

So I understand how this has happened. This particular module was
imported directly into the working branch at the time
(BRANCH-3-3-20-red-e1-opt-branch) where as other modules where
imported into the CVS HEAD and then branched into the current working
branch. As a result the branch tag didn't have the magic 0 in it.

We where able to work around the import failure by deleting the branch
tag from the module and then branching it again. The new branch tag
became:

BRANCH-3-3-20-red-e1-opt-branch:1.1.1.1.0.6

which parsecvs was able to correctly parse and assign to the correct
GIT branch on import. I'm guessing this is a corner case that could do
with better handling but in our case it was solved by tweaking our CVS
repository.

-- 
Alex, homepage: http://www.bennee.com/~alex/