Imports without Tariffs

Imports without Tariffs

[mfwitten]

[INTRO]

	Hello,
	
	I have come across a problem that could be easily addressed
	in order to improve interoperability between CVS and git.
	
	
	I'm new to using git (and a novice with SCMs in general),
	and I really enjoy the way that git allows me to think.
	
	Unfortunately, I am forced to deal with CVS (for one of
	my classes), but I have been courageously trying to use
	git behind the scenes, as I can already tell CVS is a
	nightmare.
	

[SCENARIO]
	
	I would like to do the following sequence:
		
		(1) Checkout a CVS repository as a git working tree.
				
			=> git cvsimport -A /path -a -k -v -r cvs -C work module
			
		(2) Edit the git working tree and make commits there.
				
			=> cd work
			=> vim ... # emacs can go live with CVS ;-P
			=> git commit -a -m ...
				
		(3) Export my git working tree commits back to CVS.
			
			=> cd ..; cvs co modules; cd modules
			
			=> for each pertinent git commit hash, HASH:
				GIT_DIR=../work/.git git-cvsexportcommit -vc HASH
				
		(4) Update my git working tree from CVS (at some later time).
			
			=> cd ../work
			=> git cvsimport -a -k -v -r cvs -C . module
		
[PROBLEM]
	
	git-cvsimport creates new hash IDs for the same ol' commits.
	
	
	Running git-cvsimport does indeed do an incremental update of
	the 'cvs' remote in the 'work' git repository.
	
	However, the updates are brought in as brand new git commits
	with the CVS dates in the log (though changed to UTC +0000!!).
	
	Therefore, when the updated 'cvs' remote branches are merged into
	my local branches, git treats the commits I made with cvsexportcommit
	as separate history; thus, history is duplicated and merge commits
	appear where they shouldn't.

[PARTIAL SOLUTION]
	
	The trick is to populate the 'cvs' remote branches with commits that
	have the right hash IDs.
	
	I thought I could do this by hand.
	
	I updated the cvsexportcommit/cvsimport sequence by pushing
	my local branches into their respective 'cvs' remote branches,
	before running that last cvsimport:
		
		=> for each pertinent git commit hash, HASH:
			GIT_DIR=../work/.git git-cvsexportcommit -vc HASH
		
		=>=>=> git push . refs/heads/master:refs/remotes/cvs/master
		
		=> git cvsimport -a -k -v -r cvs -C . module
	
	Of course, git-cvsimport still adds the commits to the 'cvs'
	remote branches, duplicating (log) histories.
	
	The difference is that the 'cvs' remote branches become descendants
	of the local branches, causing a fast-forward merge with the local
	branches; this removes the split histories, but still duplicates
	information in an even dumber way, as changes are undone and then
	redone.

[SOLUTION]
	
	The trick is to make both git-cvsimport and the user smarter
		---- A TALL ORDER!
	
	
	To make things simple, I think all of the necessary machinery
	should be put into git-cvsimport.
	
	The user should first git-cvsexportcommit as necessary.
	
	Then the user should tell git-cvsimport to push from the pertinent git
	branches into the pertinent cvsimport branch. To make things even  
simpler,
	I think that git-cvsimport should make the -r option mandatory; then
	git-cvsimport would simply do the pushing as I did:
		
		git push . refs/heads/<branch1>:refs/remotes/<remote>/<branch2>
	
	Then git-cvsimport would mark for each pushed-into branch the timestamp
	for when the push occurred. These marks could be put in some special  
file,
	say .git/CVSIMPORT.
	
	The next time git-cvsimport is used as normal, it can consult this  
file for
	timestamps, rather than relying on git log timestamps, to determine  
if creat-
	ing a new commit is necessary.

[CONCLUSION]
	
	I feel that not much is being put into converting between git and  
CVS, which
	is unfortunate, because a lot of stuff is in CVS out there; here at  
MIT,
	*all* CS students have to use CVS for at least one semester of  
grueling programming
	labs.
	
	If conversion between git and CVS is hard, many students will just  
learn
	what seems easiest---CVS---and they'll decide to use CVS for their  
projects
	later on (I've already seen this happen); that's not a future I want  
to have!
	
	Sincerely,
	Michael Witten
	
[OTHER PROBLEMS]
	
	(1) git-cvsimport creates log entries with UTC times; maybe that's  
correct.
	(2) git-cvsimport's -A argument must be a full path.
	(3) git-cvsexportcommit should automatically handle contiguous commits.
	(4) git-cvsimport is written in the most unmaintainable perl ever!

Re: Imports without Tariffs

[Jeff King]

On Fri, Oct 12, 2007 at 04:36:29PM -0400, mfwitten@MIT.EDU wrote:

> 		(4) Update my git working tree from CVS (at some later time).
> 			
> 			=> cd ../work
> 			=> git cvsimport -a -k -v -r cvs -C . module
> 		
> [PROBLEM]
> 	
> 	git-cvsimport creates new hash IDs for the same ol' commits.
>
> [...]
> 	Therefore, when the updated 'cvs' remote branches are merged into
> 	my local branches, git treats the commits I made with cvsexportcommit
> 	as separate history; thus, history is duplicated and merge commits
> 	appear where they shouldn't.

You are presumably doing a 'git-pull' on the cvsimport-ed commits. Try
doing a git-rebase, which will filter out commits which make the same
changes. Yes, it means throwing away your original commits, but the new
ones should be morally equivalent (and are now the "official" upstream
of the CVS repository).

-Peff

(unknown),

[Michael Witten]

I apologize if this is received twice.
I did add some comments, though!



On 12 Oct 2007, at 10:49:10 PM, Jeff King wrote:
> You are presumably doing a 'git-pull' on the cvsimport-ed commits. Try
> doing a git-rebase, which will filter out commits which make the same
> changes. Yes, it means throwing away your original commits, but the  
> new
> ones should be morally equivalent (and are now the "official" upstream
> of the CVS repository).

Now that you mention it, I think the best approach would be to:
	
	(1) cvsexportcommit
	(2) git reset --hard LAST_CVS_IMPORT_AND_MERGE
	(3) git cvsimport ..... # and merge

I think this is what you mean; it seems to me that rebasing isn't  
quite that.

However, this will not preserve more complicated history such as merges
from another git repository.

Basically, I want to treat my git repository as the official  
repository; the CVS
repo is just their for the old farts to get the latest stuff ;-P

Thanks!

Michael

PS
Please send me other opinions.

Re: your mail

[Jeff King]

On Sat, Oct 13, 2007 at 12:01:04AM -0400, Michael Witten wrote:

> Now that you mention it, I think the best approach would be to:
> 	
> 	(1) cvsexportcommit
> 	(2) git reset --hard LAST_CVS_IMPORT_AND_MERGE
> 	(3) git cvsimport ..... # and merge
>
> I think this is what you mean; it seems to me that rebasing isn't quite 
> that.

No, I mean rebasing instead of merge. As in, you have a history like
this:

    /--C---D  <-- your master
A--B
    \--C'--D' <-- cvsimport merge tip

where "C" and "D" are your commits in git, and C' and D' are pulled in
from cvsimport. You want to rebase your work like this:

A--B--C'--D'--C--D

except that git-rebase is smart enough to realize that C == C' and skip
it (so it's a "safe" way of moving forward).

> However, this will not preserve more complicated history such as merges
> from another git repository.

Correct. Rebasing doesn't really handle merges, but I assumed you were
just making simple commits on top of a cvs master.

> Basically, I want to treat my git repository as the official
> repository; the CVS repo is just their for the old farts to get the
> latest stuff ;-P

Then my suggestion doesn't really work. You might consider using git as
the official server and letting the old farts use git-cvsserver.

HTH,
-Peff

Re: Imports without Tariffs

[Michael Witten]

I apologize if you receive this twice.
I have now changed my email client to
use plain text by default.

On 13 Oct 2007, at 12:07:12 AM, Jeff King wrote:

> except that git-rebase is smart enough to realize that C == C' and  
> skip
> it (so it's a "safe" way of moving forward).

This is good to know! The documentation should mention this case!

>> However, this will not preserve more complicated history such as  
>> merges
>> from another git repository.
>
> Correct. Rebasing doesn't really handle merges, but I assumed you were
> just making simple commits on top of a cvs master.

Yes, you are quite correct. Your solution will work for me.
However, I think a general solution should be sought out.

Basically, the imported cvs history should be treated like
a remote that's being tracked. It seems like the solution
I proposed kind of does this and would work for other SCM
imports too.

>> Basically, I want to treat my git repository as the official
>> repository; the CVS repo is just their for the old farts to get the
>> latest stuff ;-P
>
> Then my suggestion doesn't really work. You might consider using  
> git as
> the official server and letting the old farts use git-cvsserver.

Unfortunately, they are the ones running the servers; I have to do my
git work behind the scenes.

Thanks,
Michael

Re: Imports without Tariffs

[Michael Witten]

On 12 Oct 2007, at 4:36:29 PM, mfwitten@MIT.EDU wrote:

> To make things simple, I think all of the necessary machinery
> should be put into git-cvsimport.
>
> The user should first git-cvsexportcommit as necessary.


Now that I have considered, it makes more sense to put the
machinery in git-cvsexportcommit.

The user could use a -b flag to specify the git branch to push
into after the cvs commit occurs, and git-cvsexportcommit would
update the .git/SCM_IMPORT file (changed from CVS_IMPORT).

Of course, this introduces other troubles.

Sometimes I run cvsexportcommit using a git repo on another server.
So perhaps one should also be able to use cvsexportcommit for just
pushing and editing the .git/SCM_IMPORT file.

That way it's possible to update CVS and then notify any other git
repo by hand.

Michael

Re: Imports without Tariffs

[Jeff King]

On Sat, Oct 13, 2007 at 12:30:09AM -0400, Michael Witten wrote:

>> except that git-rebase is smart enough to realize that C == C' and skip
>> it (so it's a "safe" way of moving forward).
>
> This is good to know! The documentation should mention this case!

Yes, it probably should. Can you submit a patch describing the behavior
where you think it ought to go?

> Basically, the imported cvs history should be treated like
> a remote that's being tracked. It seems like the solution
> I proposed kind of does this and would work for other SCM
> imports too.

I think it's an interesting avenue to pursue, though I would worry a
little about robustness. I like the fact that after rebasing, the
commits have done a complete git->cvs->git loop and look identical, so I
am sure everything made it through intact. But perhaps I'm just being
paranoid.

As an alternate idea, why not try to have the CVS commit contain all
information necessary to create a particular git commit. IOW, describe
all of the data that goes into the commit hash as textual comments in
the CVS commit (committer name/time, author name/time). And then
theoretically git-cvsimport can reconstruct the exact same commits
again, and your git->cvs->git merge really _will_ be a fastforward.

> Unfortunately, they are the ones running the servers; I have to do my
> git work behind the scenes.

I've been in the same boat, and just used the rebase trick I described.
Of course it was a very tiny project, so I didn't mind losing some of
the full power of git.

-Peff

Re: Imports without Tariffs

[Michael Witten]

On 13 Oct 2007, at 3:57:23 AM, Jeff King wrote:

> On Sat, Oct 13, 2007 at 12:30:09AM -0400, Michael Witten wrote:
>
>>> except that git-rebase is smart enough to realize that C == C'  
>>> and skip
>>> it (so it's a "safe" way of moving forward).
>>
>> This is good to know! The documentation should mention this case!
>
> Yes, it probably should. Can you submit a patch describing the  
> behavior
> where you think it ought to go?

I can make a patch, but at the moment I'm swamped and I don't want to  
think
about doing that.

I'll get around to it eventually, I hope.

Do I just submit the patch to this list? How do I know it will be used?

>> Basically, the imported cvs history should be treated like
>> a remote that's being tracked. It seems like the solution
>> I proposed kind of does this and would work for other SCM
>> imports too.
>
> I think it's an interesting avenue to pursue, though I would worry a
> little about robustness. I like the fact that after rebasing, the
> commits have done a complete git->cvs->git loop and look identical

Frankly, I don't know how robust my idea is either, but it's simple
enough not to have many problems lurking in the shadows.

It would certainly be more useful than not.

> As an alternate idea, why not try to have the CVS commit contain all
> information necessary to create a particular git commit. IOW, describe
> all of the data that goes into the commit hash as textual comments in
> the CVS commit (committer name/time, author name/time). And then
> theoretically git-cvsimport can reconstruct the exact same commits
> again, and your git->cvs->git merge really _will_ be a fastforward.

I considered this too, but this exposes what we're doing. We don't
want the old farts to wonder what all these hash thingies are.

Michael Witten

Re: Imports without Tariffs

[Jeff King]

On Sat, Oct 13, 2007 at 07:04:52PM -0400, Michael Witten wrote:

> I can make a patch, but at the moment I'm swamped and I don't want to think
> about doing that.
>
> I'll get around to it eventually, I hope.
>
> Do I just submit the patch to this list? How do I know it will be used?

Yes, send a patch to the list and to Junio (the maintainer).  See
Documentation/SubmittingPatches for details.

> Frankly, I don't know how robust my idea is either, but it's simple
> enough not to have many problems lurking in the shadows.
>
> It would certainly be more useful than not.

Then submit a patch implementing it. :)

>> all of the data that goes into the commit hash as textual comments in
>> the CVS commit (committer name/time, author name/time). And then
> I considered this too, but this exposes what we're doing. We don't
> want the old farts to wonder what all these hash thingies are.

Heh. I like the idea of git secretly infiltrating institutions,
replacing CVS unbeknownst to management. It was the same for Linux in
the mid-90's ("our mail server is running on what!?").

-Peff

Re: Imports without Tariffs

[Michael Witten]

On 14 Oct 2007, at 12:40:01 PM, Jeff King wrote:

> On Sat, Oct 13, 2007 at 07:04:52PM -0400, Michael Witten wrote:
>> Do I just submit the patch to this list? How do I know it will be  
>> used?
>
> Yes, send a patch to the list and to Junio (the maintainer).  See
> Documentation/SubmittingPatches for details.

Indeed! RTFM, as they say. ;-)

>> Frankly, I don't know how robust my idea is either, but it's simple
>> enough not to have many problems lurking in the shadows.
>>
>> It would certainly be more useful than not.
>
> Then submit a patch implementing it. :)

Perl. Ugh. I'll see what I can do, though it may take 1.5 weeks.

> I like the idea of git secretly infiltrating institutions,
> replacing CVS unbeknownst to management. It was the same for Linux in
> the mid-90's ("our mail server is running on what!?").

:-)

Replace CVS or Bust!