Re: Importing Mozilla CVS into git

["Jon Smirl" <jonsmirl@gmail.com>]

On 6/6/06, Jon Smirl <jonsmirl@gmail.com> wrote:
> On 6/6/06, Martin Langhoff <martin.langhoff@gmail.com> wrote:
> > On 6/3/06, Jon Smirl <jonsmirl@gmail.com> wrote:
> > > On 6/1/06, Jon Smirl <jonsmirl@gmail.com> wrote:
> > > > With the attached patch you can parse the entire Mozilla tree. The
> > > > tree has over 100,000 files in it and about 300 branches.
> > >
> > > I was a little low with these counts, more like 110,000 files and some
> > > parts of the tree have 1,000 branches. Total tree size is 3GB.
> >
> > I don't think it really has that many branches. If I am to believe
> > cvsps (which took 3GB to walk the history), it has some branches with
> > recursive loops in their ancestry (MANG_MATH_BRANCH and
> > SpiderMonkey140_BRANCH have eachother as ancestors!?), 197969 commits
> > and 796 branches.

My full import to svn just finished after a day and a half.
Here are the stats:

cvs2svn Statistics:
------------------
Total CVS Files:             99851
Total CVS Revisions:        948580
Total Unique Tags:            1505
Total Unique Branches:        1577
CVS Repos Size in KB:      2725843
Total SVN Commits:          205787
First Revision Date:    Fri Mar 27 21:13:08 1998
Last Revision Date:     Tue May 30 19:28:10 2006
------------------
Timings:
------------------
pass 1:  3602 seconds
pass 2:   227 seconds
pass 3:    66 seconds
pass 4:  1070 seconds
pass 8:124650 seconds
total: 124650 seconds
[jonsmirl@jonsmirl ~]$

[jonsmirl@jonsmirl svn]$ du -h
4.0K    ./svntest/dav
12K     ./svntest/locks
40K     ./svntest/hooks
16K     ./svntest/conf
7.4G    ./svntest/db/revs
808M    ./svntest/db/revprops
4.0K    ./svntest/db/transactions
8.2G    ./svntest/db
8.2G    ./svntest
8.2G    .

[jonsmirl@jonsmirl svn]$ find | wc
 411607  411607 10891057

There are two directories that each contain about 205k files. 205K
files in a single directory is causing svn problems on Ext3.

Bottom line, cvs2svn import tool works quite well. Highest memory
consumption I saw was 100MB and it used 6GB of extra disk while
running plus space need by svn.

I don't know quite enough about git yet to replace the svn commands it
uses with git equivalents but if that were done I think most of the
cvs import problems would be solved. Obviously the svn team has put a
great deal of work into this program.

I don't think replacing the svn commands is very hard, I just haven't
figured out the right way to build branches with low-level git yet and
I don't know Python. I'll bet someone already familiar with git and
cvs import could convert it in a couple of hours.

-- 
Jon Smirl
jonsmirl@gmail.com