Re: Incremental cvsimports

["Geoff Russell" <geoffrey.russell@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 1972 bytes --]

Dear Martin,


On 5/24/06, Martin Langhoff <martin.langhoff@gmail.com> wrote:
> On 5/24/06, Geoff Russell <geoffrey.russell@gmail.com> wrote:
> > Dear Git,
>
> Dear Geoff,
>
> if you look at the list archive for the last couple of days, you'll
> see there's been quite a bit of activity in tuning cvsimport so that
> it scales better with large imports like yours. We have been playing
> with a gentoo cvs repo with 300K commits / 1.6GB uncompressed.
>
> Don't split up the tree... that'll lead to something rather ackward.
> Instead, fetch and build git from Junio's 'master' branch which seems
> to have collected most (all?) of the patches posted, including one
> from Linus that will repack the repo every 1K commits -- keeping the
> import size down.

I got the latest git and yes, the size is kept down. I've only tried with
a smaller repository but it looks promising. When I ran git-cvsimport without a
CVS-module name (wanting the entire repository), it gave me a Usage message
indicating that the CVS-module name was optional - but it isn't :)

I did have to change
2 lines in git-cvsimport to get it to run with my 5.8.0 perl (problems with
POSIX errno). I've attached a patch but my work around isn't as quick as
what it replaced.

Many thanks, I'll have a go with the big repository at work tomorrow!

Cheers,
Geoff Russell

P.S. I've just started to look with git. We have wanted a cvs replacement for
a while but have been too scared to change (until now).



>
> You _will_ need a lot of memory though, as cvsps grows large (working
> on a workaround now) and cvsimport grows a bit over time (where is
> that last leak?!). And a fast machine -- specially fast IO. I've just
> switched from an old test machine to an AMD64 with fast disks, and
> it's importing around 10K commits per hour.

I

>
> You will probably want to run cvsps by hand, and later use the -P flag.
>
> cheers,
>
>
> martin
>
>

[-- Attachment #2: 999 --]
[-- Type: application/octet-stream, Size: 903 bytes --]

*** git-cvsimport	2006-05-24 20:13:19.000000000 +0930
--- /usr/local/bin/git-cvsimport	2006-05-24 20:22:27.000000000 +0930
*************** use File::Basename qw(basename dirname);
*** 23,29 ****
  use Time::Local;
  use IO::Socket;
  use IO::Pipe;
! use POSIX qw(strftime dup2 :errno_h);
  use IPC::Open2;
  
  $SIG{'PIPE'}="IGNORE";
--- 23,29 ----
  use Time::Local;
  use IO::Socket;
  use IO::Pipe;
! use POSIX qw(strftime dup2);
  use IPC::Open2;
  
  $SIG{'PIPE'}="IGNORE";
*************** sub get_headref ($$) {
*** 446,452 ****
  	    is_sha1($r) or die "Cannot get head id for $name ($r): $!";
  	    return $r;
      }
!     die "unable to open $f: $!" unless $! == POSIX::ENOENT;
      return undef;
  }
  
--- 446,452 ----
  	    is_sha1($r) or die "Cannot get head id for $name ($r): $!";
  	    return $r;
      }
!     die "unable to open $f: $!" if -f $f;
      return undef;
  }