Need help for migration from CVS to git in one go (ie, FORGETTING CVS history)

Need help for migration from CVS to git in one go (ie, FORGETTING CVS history)

[Francis Galiegue]

Hello list,

Since my first email where I said that we wanted to migrate from CVS to a 
distributed SCM, we have had a lot of discussions with developers and my 
chief (which happens to be a developer too). We have come to the following 
conclusions:

* git has credentials (heck, it "runs" the Linux kernel, Wine, Samba, and 
other);
* the "distributed" part of it is indeed an advantage (planned developments 
with huge technical/functional impacts);
* 52 CVS modules, fine; but then this can become one and 52 subdirectories in 
them and still act as separate modules from the build system point of view 
(which I have implemented, so I can change it);
* we have enough resources that the old CVS tree can be kept, even on tape, 
the day we decide to "scrap it all" and start anew;
* CVS "good practices" have barely been observed, so "we" might as well start 
again from a sane base.

So, git looks like a great solution. The Eclipse plugin evolves nicely, you 
still can decide on a "central" repository... Looks good.

There are, however, a few problems:
* first: I am not an Eclipse user, and the two main developers use Eclipse 
extensively (but even they are lost because of CVS, which is to "my" 
advantage), so some questions below will focus on the Eclipse plugin;
* second: even though this may be a "non problem", we use Bonsai, which has 
the ability to see what was commited by whom, given a time interval (from d1 
to d2): the base gitweb allows to search by commiter, which is good, but it 
has no date boundaries: do tools exist for git that can do this? If not, that 
wouldn't be a big deal, however...
* third: also Bonsai-related; Bonsai can link to Bugzilla by matching (wild 
guess) /\b(?:#?)(\d+)\b/ and transforming this into 
http://your.bugzilla.fqdn.here/show_bug.cgi?id=$1. Does gitweb have this 
built-in? (haven't looked yet) Is this planned, or has it been discussed and 
been considered not worth the hassle?

Finally, about the Eclipse plugin:
* the README says that it has no push capacity, the popup menu says otherwise 
(last checked last Saturday)... What should I believe?
* if the plugin can push, can you configure from within the plugin what 
branches can be pushed where, or should you edit .git/config by hand?
* "git stash": is it supported?
* can you "copy" a commit, or even a set of commits, and "cherry-pick" them 
into another branch? Or even rebase a branch onto antoher?

Thanks,
-- 
fge

Re: Need help for migration from CVS to git in one go (ie, FORGETTING CVS history)

[Shawn O. Pearce]

Francis Galiegue <fg@one2team.net> wrote:
> Finally, about the Eclipse plugin:
> * the README says that it has no push capacity, the popup menu says otherwise 
> (last checked last Saturday)... What should I believe?

The README is out of date.  Push is implemented, works, and folks
actually use it.  That was a nice success from our GSoC program
this year.

> * if the plugin can push, can you configure from within the plugin what 
> branches can be pushed where, or should you edit .git/config by hand?

I don't think you can edit saved configurations in the plugin UI.
The APIs are there underneath to do the reading/writing and most
of the UI to show a configuration is present, I just don't think
anyone has wired the two together for saved configurations inside
of .git/config.  So yea, edit by hand.

> * "git stash": is it supported?

Not in Eclipse, no.

> * can you "copy" a commit, or even a set of commits, and "cherry-pick" them 
> into another branch? Or even rebase a branch onto antoher?

Not in Eclipse, no.

-- 
Shawn.

Re: Need help for migration from CVS to git in one go (ie, FORGETTING CVS history)

[Francis Galiegue]

Le Thursday 06 November 2008 00:23:48 Shawn O. Pearce, vous avez écrit :
> Francis Galiegue <fg@one2team.net> wrote:
> > Finally, about the Eclipse plugin:
> > * the README says that it has no push capacity, the popup menu says
> > otherwise (last checked last Saturday)... What should I believe?
>
> The README is out of date.  Push is implemented, works, and folks
> actually use it.  That was a nice success from our GSoC program
> this year.
>

Good to hear :p Any chance that the README gets updated? ;)

> > * if the plugin can push, can you configure from within the plugin what
> > branches can be pushed where, or should you edit .git/config by hand?
>
> I don't think you can edit saved configurations in the plugin UI.
> The APIs are there underneath to do the reading/writing and most
> of the UI to show a configuration is present, I just don't think
> anyone has wired the two together for saved configurations inside
> of .git/config.  So yea, edit by hand.
>

Not a big deal, I guess. This can be handled... (BTW, what about "git clone"? 
Still not there, although _this_ can be dealt with also)

> > * "git stash": is it supported?
>
> Not in Eclipse, no.
>

What do you mean by this?

> > * can you "copy" a commit, or even a set of commits, and "cherry-pick"
> > them into another branch? Or even rebase a branch onto antoher?
>
> Not in Eclipse, no.

Same question... What exactly is the problem with Eclipse wrt this kind of 
operation?

-- 
fge

Re: Need help for migration from CVS to git in one go (ie, FORGETTING CVS history)

[Shawn O. Pearce]

Francis Galiegue <fg@one2team.net> wrote:
> Le Thursday 06 November 2008 00:23:48 Shawn O. Pearce, vous avez écrit :
> > Francis Galiegue <fg@one2team.net> wrote:
> > > Finally, about the Eclipse plugin:
> 
> > > * if the plugin can push, can you configure from within the plugin what
> > > branches can be pushed where, or should you edit .git/config by hand?
> >
> > I don't think you can edit saved configurations in the plugin UI.
> 
> Not a big deal, I guess. This can be handled... (BTW, what about "git clone"? 
> Still not there, although _this_ can be dealt with also)

File->Import... from Git.  That can clone a repository.  After it
finishes you'll have to use the import wizard again to actually
import existing .project files, or create new projects, etc.
 
> > > * "git stash": is it supported?
> >
> > Not in Eclipse, no.
> 
> What do you mean by this?

I mean there's no implementation of git stash.  Eclipse doesn't
support git stash, the notion of the stash, the branch the stash
is on.  Its not in the Eclipse plugin.
 
> > > * can you "copy" a commit, or even a set of commits, and "cherry-pick"
> > > them into another branch? Or even rebase a branch onto antoher?
> >
> > Not in Eclipse, no.
> 
> Same question... What exactly is the problem with Eclipse wrt this kind of 
> operation?

Its also not implemented in the eclipse UI.

-- 
Shawn.

Re: Need help for migration from CVS to git in one go (ie, FORGETTING CVS history)

[Francis Galiegue]

[...]
 
> > > > * "git stash": is it supported?
> > >
> > > Not in Eclipse, no.
> >
> > What do you mean by this?
>
> I mean there's no implementation of git stash.  Eclipse doesn't
> support git stash, the notion of the stash, the branch the stash
> is on.  Its not in the Eclipse plugin.
>

While I'm a total newbie to Eclipse, and not that fluent with Java, this looks 
surprising. Is it really Eclipse that is at fault here? Eclipse saves its 
workspace (whatever it means to "save a workspace") when it exits, isn't 
there an interface that you can implement that does "partial" saves, hence 
git stashes?

> > > > * can you "copy" a commit, or even a set of commits, and
> > > > "cherry-pick" them into another branch? Or even rebase a branch onto
> > > > antoher?
> > >
> > > Not in Eclipse, no.
> >
> > Same question... What exactly is the problem with Eclipse wrt this kind
> > of operation?
>
> Its also not implemented in the eclipse UI.

Eclipse has the ability to apply patches AFAIK... Even though this may not be 
equivalent to a git cherry-pick, wouldn't there be a way to extract at least 
the diffs and commit messages and apply?

-- 
fge

Re: Need help for migration from CVS to git in one go (ie, FORGETTING CVS history)

[Robin Rosenberg]

torsdag 06 november 2008 02:28:20 skrev Francis Galiegue:
> [...]
>  
> > > > > * "git stash": is it supported?
> > > >
> > > > Not in Eclipse, no.
> > >
> > > What do you mean by this?
> >
> > I mean there's no implementation of git stash.  Eclipse doesn't
> > support git stash, the notion of the stash, the branch the stash
> > is on.  Its not in the Eclipse plugin.
> >
> 
> While I'm a total newbie to Eclipse, and not that fluent with Java, this looks 
> surprising. Is it really Eclipse that is at fault here? Eclipse saves its 
> workspace (whatever it means to "save a workspace") when it exits, isn't 
> there an interface that you can implement that does "partial" saves, hence 
> git stashes?

The fault is that it has not been implemented yet, It's not Eclipse's fault. The
plugin simply lacks support for it. You can do that with Git though. I use both
tools side by side.

> > > > > * can you "copy" a commit, or even a set of commits, and
> > > > > "cherry-pick" them into another branch? Or even rebase a branch onto
> > > > > antoher?
> > > >
> > > > Not in Eclipse, no.
> > >
> > > Same question... What exactly is the problem with Eclipse wrt this kind
> > > of operation?
> >
> > Its also not implemented in the eclipse UI.
> 
> Eclipse has the ability to apply patches AFAIK... Even though this may not be 
> equivalent to a git cherry-pick, wouldn't there be a way to extract at least 
> the diffs and commit messages and apply?

Sure. Solving everything that is missing is a matter of actually coding the feature.
Unfortunately we have very little time at our disposal, which is why we welcome
contributions. 

-- robin


-- robin

Re: Need help for migration from CVS to git in one go (ie, FORGETTING CVS history)

[Petr Baudis]

  Hi,

On Thu, Nov 06, 2008 at 12:14:56AM +0100, Francis Galiegue wrote:
> * second: even though this may be a "non problem", we use Bonsai, which has 
> the ability to see what was commited by whom, given a time interval (from d1 
> to d2): the base gitweb allows to search by commiter, which is good, but it 
> has no date boundaries: do tools exist for git that can do this? If not, that 
> wouldn't be a big deal, however...

	git log --since='2 months ago' --until='1 month ago'

  There is no gitweb functionality for this right now, but I think an
implementation of that would be welcome in principle.

> * third: also Bonsai-related; Bonsai can link to Bugzilla by matching (wild 
> guess) /\b(?:#?)(\d+)\b/ and transforming this into 
> http://your.bugzilla.fqdn.here/show_bug.cgi?id=$1. Does gitweb have this 
> built-in? (haven't looked yet) Is this planned, or has it been discussed and 
> been considered not worth the hassle?

  This is planned and I think there are third-party patches for this
(cannot find them quickly, though), but upstream gitweb does not have
the support for this included yet.

-- 
				Petr "Pasky" Baudis
People who take cold baths never have rheumatism, but they have
cold baths.

Re: Need help for migration from CVS to git in one go (ie, FORGETTING CVS history)

[Francis Galiegue]

Le Thursday 06 November 2008 00:50:35 Petr Baudis, vous avez écrit :
>   Hi,
>
> On Thu, Nov 06, 2008 at 12:14:56AM +0100, Francis Galiegue wrote:
> > * second: even though this may be a "non problem", we use Bonsai, which
> > has the ability to see what was commited by whom, given a time interval
> > (from d1 to d2): the base gitweb allows to search by commiter, which is
> > good, but it has no date boundaries: do tools exist for git that can do
> > this? If not, that wouldn't be a big deal, however...
>
> 	git log --since='2 months ago' --until='1 month ago'
>
>   There is no gitweb functionality for this right now, but I think an
> implementation of that would be welcome in principle.
>

Well, right now, the search field is only a... field. Implementing such a 
change would require a whole search page... I don't think anyone would want 
to pollute the bare-bones page with more fields if they're of no use 90+% of 
the time.

A search page would make sense, I guess, since the date criterion is not the 
only one that you could use... But what other criteria would be useful? And 
then, if a search page is implemented, what of the current dropdown list? 
Drop it altogether and put a "Search commits..." link instead?

> > * third: also Bonsai-related; Bonsai can link to Bugzilla by matching
> > (wild guess) /\b(?:#?)(\d+)\b/ and transforming this into
> > http://your.bugzilla.fqdn.here/show_bug.cgi?id=$1. Does gitweb have this
> > built-in? (haven't looked yet) Is this planned, or has it been discussed
> > and been considered not worth the hassle?
>
>   This is planned and I think there are third-party patches for this
> (cannot find them quickly, though), but upstream gitweb does not have
> the support for this included yet.

I'd love to have a link to these patches, honestly. "Our" commit messages are 
poor enough that more often than not, the whole commit message is, for 
instance, "fix bug #xxxx". While I push hard for more explicit commit 
messages, this is just a pipe dream and that link to Bugzilla is vital to 
us...

-- 
fge

Re: Need help for migration from CVS to git in one go (ie, FORGETTING CVS history)

[Jakub Narebski]

Francis Galiegue <fg@one2team.net> writes:

> Since my first email where I said that we wanted to migrate from CVS to a 
> distributed SCM, we have had a lot of discussions with developers and my 
> chief (which happens to be a developer too). We have come to the following 
> conclusions:
> 
> * git has credentials (heck, it "runs" the Linux kernel, Wine,
>   Samba, and other);

The main contenders among top OSS version control systems seems to be
Git, Mercurial and Bazaar-NG for distributed SCM, and Subversion for
centralized SCM.

> * the "distributed" part of it is indeed an advantage (planned
>   developments with huge technical/functional impacts);

> * 52 CVS modules, fine; but then this can become one and 52
>   subdirectories in them and still act as separate modules from
>   the build system point of view (which I have implemented, so
>   I can change it);

I think that those CVS modules should become separate repositories,
perhaps joined together using submodules. This is one of more
difficult things during conversion.

Note that in Git commits are always whole tree (whole project)
commits.

> * second: even though this may be a "non problem", we use Bonsai,
> which has the ability to see what was commited by whom, given a time
> interval (from d1 to d2): the base gitweb allows to search by
> commiter, which is good, but it has no date boundaries: do tools
> exist for git that can do this? If not, that wouldn't be a big deal,
> however...

First, there are more web interfaces than gitweb, see
http://git.or.cz/gitweb/InterfacesFrontendsAndTools

Second, you can do this from command line, using combination of commit
limiting a..b and a...b, or --since=<date> or --after=<date> and
--before=<date>, commit message searching --author, --committer, and
--grep, and path limiting "git log -- <pathspec>".

Third, it would be not that hard to add more advanced search support
to gitweb; this is even one of planned features.

> * third: also Bonsai-related; Bonsai can link to Bugzilla by
> matching (wild guess) /\b(?:#?)(\d+)\b/ and transforming this into
> http://your.bugzilla.fqdn.here/show_bug.cgi?id=$1. Does gitweb have
> this built-in? (haven't looked yet) Is this planned, or has it been
> discussed and been considered not worth the hassle?

This is (under name of 'committags') in gitweb TODO; gitweb-xmms2
support this IIRC or supported this (for Mantis and not Bugzilla
though...)

-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: Need help for migration from CVS to git in one go (ie, FORGETTING CVS history)

[Francis Galiegue]

Le Thursday 06 November 2008 04:08:16 Jakub Narebski, vous avez écrit :
Hello,
[...]
> >
> > * 52 CVS modules, fine; but then this can become one and 52
> >   subdirectories in them and still act as separate modules from
> >   the build system point of view (which I have implemented, so
> >   I can change it);
>
> I think that those CVS modules should become separate repositories,
> perhaps joined together using submodules. This is one of more
> difficult things during conversion.
>
> Note that in Git commits are always whole tree (whole project)
> commits.
>

Honestly, I'm not fond of this approach. The problem with submodules as far as 
I'm concerned is that documentation is "not really there", and (unless the 
README of egit is _really_ outdated) that there's no support in egit.

I know about commits affecting the whole tree, and even branches and tags, and 
that's more of an advantage to my eyes, for two reasons:

* 99+% of queries currently done on the CVS tree (with Bonsai) cover all 
modules; only rarely is a single module concerned, and in this case you just 
fill in the appropriate field in the search page anyway;
* creating a branch is one command and that's it. It may also be one command 
with submodules, but again, the documentation makes me uncomfortable; with 
CVS, well... You get the picture.

What's more, I don't think we have the requirement of making specific 
per-module tags. Not as far as this has been discussed so far, anyway, and 
not as far as the history shows.

> > * second: even though this may be a "non problem", we use Bonsai,
> > which has the ability to see what was commited by whom, given a time
> > interval (from d1 to d2): the base gitweb allows to search by
> > commiter, which is good, but it has no date boundaries: do tools
> > exist for git that can do this? If not, that wouldn't be a big deal,
> > however...
>
> First, there are more web interfaces than gitweb, see
> http://git.or.cz/gitweb/InterfacesFrontendsAndTools
>

Yep, I've yet to try those... There are quite a few!

> Second, you can do this from command line, using combination of commit
> limiting a..b and a...b, or --since=<date> or --after=<date> and
> --before=<date>, commit message searching --author, --committer, and
> --grep, and path limiting "git log -- <pathspec>".
>

Well, a Web-based search engine is kind of a requirement. I'm the only one to 
do command line... Thanks for the hints, though!

> Third, it would be not that hard to add more advanced search support
> to gitweb; this is even one of planned features.
>

Which brings back to the subject of submodules, since as I said above, we 
generally search on the entire tree, and per-module searches are rare.

> > * third: also Bonsai-related; Bonsai can link to Bugzilla by
> > matching (wild guess) /\b(?:#?)(\d+)\b/ and transforming this into
> > http://your.bugzilla.fqdn.here/show_bug.cgi?id=$1. Does gitweb have
> > this built-in? (haven't looked yet) Is this planned, or has it been
> > discussed and been considered not worth the hassle?
>
> This is (under name of 'committags') in gitweb TODO; gitweb-xmms2
> support this IIRC or supported this (for Mantis and not Bugzilla
> though...)

Interesting... I'll have a look at it.

Thanks for the answers!
-- 
fge

Re: Need help for migration from CVS to git in one go (ie, FORGETTING CVS history)

[Jakub Narebski]

On Thu, 6 Nov 2008, Francis Galiegue wrote:
> Le Thursday 06 November 2008 04:08:16 Jakub Narebski, vous avez écrit :
> Hello,
> [...]
> > >
> > > * 52 CVS modules, fine; but then this can become one and 52
> > >   subdirectories in them and still act as separate modules from
> > >   the build system point of view (which I have implemented, so
> > >   I can change it);
> >
> > I think that those CVS modules should become separate repositories,
> > perhaps joined together using submodules. This is one of more
> > difficult things during conversion.
> >
> > Note that in Git commits are always whole tree (whole project)
> > commits.
> >
> 
> Honestly, I'm not fond of this approach. The problem with submodules as far as 
> I'm concerned is that documentation is "not really there", and (unless the 
> README of egit is _really_ outdated) that there's no support in egit.
> 
> I know about commits affecting the whole tree, and even branches and tags, and 
> that's more of an advantage to my eyes, for two reasons:
> 
> * 99+% of queries currently done on the CVS tree (with Bonsai) cover all 
> modules; only rarely is a single module concerned, and in this case you just 
> fill in the appropriate field in the search page anyway;

Well, the mapping of CVS modules into Git repositories, and perhaps
also later binding those Git repositories together using submodules
support is IMHO one of more difficult decisions when deciding on
migration from CVS to Git.

What you would have to ask yourself is which of those CVS modules
are independent, for example having independent version numbers (tags)
and independent branches. And if commit really affects whole tree...

> * creating a branch is one command and that's it. It may also be one command 
> with submodules, but again, the documentation makes me uncomfortable; with 
> CVS, well... You get the picture.

Submodules are Git repositories of their own. So you have branching
there (almost) as easy as otherwise in Git. The only problem is a bit
lacking UI for binding those submodules together.

> 
> What's more, I don't think we have the requirement of making specific 
> per-module tags. Not as far as this has been discussed so far, anyway, and 
> not as far as the history shows.

Well, that is one issue that would help in mapping CVS modules to Git
repositories (and submodules).

> 
> > > * second: even though this may be a "non problem", we use Bonsai,
> > > which has the ability to see what was commited by whom, given a time
> > > interval (from d1 to d2): the base gitweb allows to search by
> > > commiter, which is good, but it has no date boundaries: do tools
> > > exist for git that can do this? If not, that wouldn't be a big deal,
> > > however...
> >
> > First, there are more web interfaces than gitweb, see
> > http://git.or.cz/gitweb/InterfacesFrontendsAndTools
> 
> Yep, I've yet to try those... There are quite a few!
> 
> > Second, you can do this from command line, using combination of commit
> > limiting a..b and a...b, or --since=<date> or --after=<date> and
> > --before=<date>, commit message searching --author, --committer, and
> > --grep, and path limiting "git log -- <pathspec>".
> 
> Well, a Web-based search engine is kind of a requirement. I'm the only one to 
> do command line... Thanks for the hints, though!

You can also use one of GUI; qgit and gitk + git-gui seems to be quite
mature and cross-platform.

> > Third, it would be not that hard to add more advanced search support
> > to gitweb; this is even one of planned features.

I think there are two possible ways of doing it: have a kind of
"advanced search" form, where one can have fill search terms, like
date limit, path limiting etc; or have an option to limit search to
current view context (for example current file or current directory).

> Which brings back to the subject of submodules, since as I said above, we 
> generally search on the entire tree, and per-module searches are rare.

Hmmm...

> 
> > > * third: also Bonsai-related; Bonsai can link to Bugzilla by
> > > matching (wild guess) /\b(?:#?)(\d+)\b/ and transforming this into
> > > http://your.bugzilla.fqdn.here/show_bug.cgi?id=$1. Does gitweb have
> > > this built-in? (haven't looked yet) Is this planned, or has it been
> > > discussed and been considered not worth the hassle?
> >
> > This is (under name of 'committags') in gitweb TODO; gitweb-xmms2
> > support this IIRC or supported this (for Mantis and not Bugzilla
> > though...)
> 
> Interesting... I'll have a look at it.

Well, now I have bumped priority of this item in my gitweb TODO list...

-- 
Jakub Narebski
Poland