[PATCH] cvsimport: rewritten in Perl

[PATCH] cvsimport: rewritten in Perl

[Matthias Urlichs]

I just got my machine blocked from a CVS server which didn't like
to get hammered with connections.

That was cvs2git's shell script. Which, by the way, is slow as hell.

Appended: a git-cvsimport script, written in Perl, which directly talks
to the CVS server. If the repository is local, it runs a "cvs server"
child. It produces the same git repository as Linus' version. It can do
incremental imports. And it's 20 times faster (on my system, with a
local CVS repository).

cvs2git is thus obsolete; this patch deletes it.

Signed-Off-By: Matthias Urlichs <smurf@smurf.noris.de>

--- 

diff --git a/Documentation/cvs-migration.txt b/Documentation/cvs-migration.txt
--- a/Documentation/cvs-migration.txt
+++ b/Documentation/cvs-migration.txt
@@ -63,18 +63,38 @@ Once you've gotten (and installed) cvsps
 any more familiar with it, but make sure it is in your path. After that,
 the magic command line is
 
-	git cvsimport <cvsroot> <module>
+	git cvsimport -v -d <cvsroot> <module> <destination>
 
 which will do exactly what you'd think it does: it will create a git
-archive of the named CVS module. The new archive will be created in a
-subdirectory named <module>.
+archive of the named CVS module. The new archive will be created in the
+subdirectory named <destination>; it'll be created if it doesn't exist.
+Default is the local directory.
 
 It can take some time to actually do the conversion for a large archive
 since it involves checking out from CVS every revision of every file,
-and the conversion script can be reasonably chatty, but on some not very
-scientific tests it averaged about eight revisions per second, so a
-medium-sized project should not take more than a couple of minutes.  For
-larger projects or remote repositories, the process may take longer.
+and the conversion script is reasonably chatty unless you omit the '-v'
+option, but on some not very scientific tests it averaged about twenty
+revisions per second, so a medium-sized project should not take more
+than a couple of minutes.  For larger projects or remote repositories,
+the process may take longer.
+
+After the (initial) import is done, the CVS archive's current head
+revision will be checked out -- thus, you can start adding your own
+changes right away.
+
+The import is incremental, i.e. if you call it again next month it'll
+fetch any CVS updates that have been happening in the meantime. The
+cut-off is date-based, so don't change the branches that were imported
+from CVS.
+
+You can merge those updates (or, in fact, a different CVS branch) into
+your main branch:
+
+	cg-merge <branch>
+
+The HEAD revision from CVS is named "origin", not "HEAD", because git
+already uses "HEAD". (If you don't like 'origin', use cvsimport's
+'-o' option to change it.)
 
 
 Emulating CVS behaviour
diff --git a/Makefile b/Makefile
--- a/Makefile
+++ b/Makefile
@@ -35,7 +35,7 @@ PROG=   git-update-cache git-diff-files 
 	git-http-pull git-ssh-push git-ssh-pull git-rev-list git-mktag \
 	git-diff-helper git-tar-tree git-local-pull git-write-blob \
 	git-get-tar-commit-id git-apply git-stripspace \
-	git-cvs2git git-diff-stages git-rev-parse git-patch-id \
+	git-diff-stages git-rev-parse git-patch-id \
 	git-pack-objects git-unpack-objects
 
 all: $(PROG)
@@ -118,7 +118,6 @@ git-diff-helper: diff-helper.c
 git-tar-tree: tar-tree.c
 git-write-blob: write-blob.c
 git-stripspace: stripspace.c
-git-cvs2git: cvs2git.c
 git-diff-stages: diff-stages.c
 git-rev-parse: rev-parse.c
 git-patch-id: patch-id.c
diff --git a/cvs2git.c b/cvs2git.c
deleted file mode 100644
--- a/cvs2git.c
+++ /dev/null
@@ -1,329 +0,0 @@
-/*
- * cvs2git
- *
- * Copyright (C) Linus Torvalds 2005
- */
-
-#include <stdio.h>
-#include <ctype.h>
-#include <string.h>
-#include <stdlib.h>
-#include <unistd.h>
-
-static int verbose = 0;
-
-/*
- * This is a really stupid program that takes cvsps output, and
- * generates a a long _shell_script_ that will create the GIT archive
- * from it. 
- *
- * You've been warned. I told you it was stupid.
- *
- * NOTE NOTE NOTE! In order to do branches correctly, this needs
- * the fixed cvsps that has the "Ancestor branch" tag output.
- * Hopefully David Mansfield will update his distribution soon
- * enough (he's the one who wrote the patch, so at least we don't
- * have to figt maintainer issues ;)
- *
- * Usage:
- *
- *	TZ=UTC cvsps -A |
- *		git-cvs2git --cvsroot=[root] --module=[module] > script
- *
- * Creates a shell script that will generate the .git archive of
- * the names CVS repository.
- *
- *	TZ=UTC cvsps -s 1234- -A |
- *		git-cvs2git -u --cvsroot=[root] --module=[module] > script
- *
- * Creates a shell script that will update the .git archive with
- * CVS changes from patchset 1234 until the last one.
- *
- * IMPORTANT NOTE ABOUT "cvsps"! This requires version 2.1 or better,
- * and the "TZ=UTC" and the "-A" flag is required for sane results!
- */
-enum state {
-	Header,
-	Log,
-	Members
-};
-
-static const char *cvsroot;
-static const char *cvsmodule;
-
-static char date[100];
-static char author[100];
-static char branch[100];
-static char ancestor[100];
-static char tag[100];
-static char log[32768];
-static int loglen = 0;
-static int initial_commit = 1;
-
-static void lookup_author(char *n, char **name, char **email)
-{
-	/*
-	 * FIXME!!! I'm lazy and stupid.
-	 *
-	 * This could be something like
-	 *
-	 *	printf("lookup_author '%s'\n", n);
-	 *	*name = "$author_name";
-	 *	*email = "$author_email";
-	 *
-	 * and that would allow the script to do its own
-	 * lookups at run-time.
-	 */
-	*name = n;
-	*email = n;
-}
-
-static void prepare_commit(void)
-{
-	char *author_name, *author_email;
-	char *src_branch;
-
-	lookup_author(author, &author_name, &author_email);
-
-	printf("export GIT_COMMITTER_NAME=%s\n", author_name);
-	printf("export GIT_COMMITTER_EMAIL=%s\n", author_email);
-	printf("export GIT_COMMITTER_DATE='+0000 %s'\n", date);
-
-	printf("export GIT_AUTHOR_NAME=%s\n", author_name);
-	printf("export GIT_AUTHOR_EMAIL=%s\n", author_email);
-	printf("export GIT_AUTHOR_DATE='+0000 %s'\n", date);
-
-	if (initial_commit)
-		return;
-
-	src_branch = *ancestor ? ancestor : branch;
-	if (!strcmp(src_branch, "HEAD"))
-		src_branch = "master";
-	printf("ln -sf refs/heads/'%s' .git/HEAD\n", src_branch);
-
-	/*
-	 * Even if cvsps claims an ancestor, we'll let the new
-	 * branch name take precedence if it already exists
-	 */
-	if (*ancestor) {
-		src_branch = branch;
-		if (!strcmp(src_branch, "HEAD"))
-			src_branch = "master";
-		printf("[ -e .git/refs/heads/'%s' ] && ln -sf refs/heads/'%s' .git/HEAD\n",
-			src_branch, src_branch);
-	}
-
-	printf("git-read-tree -m HEAD || exit 1\n");
-	printf("git-checkout-cache -f -u -a\n");
-}
-
-static void commit(void)
-{
-	const char *cmit_parent = initial_commit ? "" : "-p HEAD";
-	const char *dst_branch;
-	char *space;
-	int i;
-
-	printf("tree=$(git-write-tree)\n");
-	printf("cat > .cmitmsg <<EOFMSG\n");
-
-	/* Escape $ characters, and remove control characters */
-	for (i = 0; i < loglen; i++) {
-		unsigned char c = log[i];
-
-		switch (c) {
-		case '$':
-		case '\\':
-		case '`':
-			putchar('\\');
-			break;
-		case 0 ... 31:
-			if (c == '\n' || c == '\t')
-				break;
-		case 128 ... 159:
-			continue;
-		}
-		putchar(c);
-	}
-	printf("\nEOFMSG\n");
-	printf("commit=$(cat .cmitmsg | git-commit-tree $tree %s)\n", cmit_parent);
-
-	dst_branch = branch;
-	if (!strcmp(dst_branch, "HEAD"))
-		dst_branch = "master";
-
-	printf("echo $commit > .git/refs/heads/'%s'\n", dst_branch);
-
-	space = strchr(tag, ' ');
-	if (space)
-		*space = 0;
-	if (strcmp(tag, "(none)"))
-		printf("echo $commit > .git/refs/tags/'%s'\n", tag);
-
-	printf("echo 'Committed (to %s):' ; cat .cmitmsg; echo\n", dst_branch);
-
-	*date = 0;
-	*author = 0;
-	*branch = 0;
-	*ancestor = 0;
-	*tag = 0;
-	loglen = 0;
-
-	initial_commit = 0;
-}
-
-static void update_file(char *line)
-{
-	char *name, *version;
-	char *dir;
-
-	while (isspace(*line))
-		line++;
-	name = line;
-	line = strchr(line, ':');
-	if (!line)
-		return;
-	*line++ = 0;
-	line = strchr(line, '>');
-	if (!line)
-		return;
-	*line++ = 0;
-	version = line;
-	line = strchr(line, '(');
-	if (line) {	/* "(DEAD)" */
-		printf("git-update-cache --force-remove '%s'\n", name);
-		return;
-	}
-
-	dir = strrchr(name, '/');
-	if (dir)
-		printf("mkdir -p %.*s\n", (int)(dir - name), name);
-
-	printf("cvs -q -d %s checkout -d .git-tmp -r%s '%s/%s'\n", 
-		cvsroot, version, cvsmodule, name);
-	printf("mv -f .git-tmp/%s %s\n", dir ? dir+1 : name, name);
-	printf("rm -rf .git-tmp\n");
-	printf("git-update-cache --add -- '%s'\n", name);
-}
-
-struct hdrentry {
-	const char *name;
-	char *dest;
-} hdrs[] = {
-	{ "Date:", date },
-	{ "Author:", author },
-	{ "Branch:", branch },
-	{ "Ancestor branch:", ancestor },
-	{ "Tag:", tag },
-	{ "Log:", NULL },
-	{ NULL, NULL }
-};
-
-int main(int argc, char **argv)
-{
-	static char line[1000];
-	enum state state = Header;
-	int i;
-
-	for (i = 1; i < argc; i++) {
-		const char *arg = argv[i];
-		if (!memcmp(arg, "--cvsroot=", 10)) {
-			cvsroot = arg + 10;
-			continue;
-		}
-		if (!memcmp(arg, "--module=", 9)) {
-			cvsmodule = arg+9;
-			continue;
-		} 
-		if (!strcmp(arg, "-v")) {
-			verbose = 1;
-			continue;
-		}
-		if (!strcmp(arg, "-u")) {
-			initial_commit = 0;
-			continue;
-		}
-	}
-
-
-	if (!cvsroot)
-		cvsroot = getenv("CVSROOT");
-
-	if (!cvsmodule || !cvsroot) {
-		fprintf(stderr, "I need a CVSROOT and module name\n");
-		exit(1);
-	}
-
-	if (initial_commit) {
-		printf("[ -d .git ] && exit 1\n");
-		    printf("git-init-db\n");
-		printf("mkdir -p .git/refs/heads\n");
-		printf("mkdir -p .git/refs/tags\n");
-		printf("ln -sf refs/heads/master .git/HEAD\n");
-	}
-
-	while (fgets(line, sizeof(line), stdin) != NULL) {
-		int linelen = strlen(line);
-
-		while (linelen && isspace(line[linelen-1]))
-			line[--linelen] = 0;
-
-		switch (state) {
-		struct hdrentry *entry;
-
-		case Header:
-			if (verbose)
-				printf("# H: %s\n", line);
-			for (entry = hdrs ; entry->name ; entry++) {
-				int len = strlen(entry->name);
-				char *val;
-
-				if (memcmp(entry->name, line, len))
-					continue;
-				if (!entry->dest) {
-					state = Log;
-					break;
-				}
-				val = line + len;
-				linelen -= len;
-				while (isspace(*val)) {
-					val++;
-					linelen--;
-				}
-				memcpy(entry->dest, val, linelen+1);
-				break;
-			}
-			continue;
-
-		case Log:
-			if (verbose)
-				printf("# L: %s\n", line);
-			if (!strcmp(line, "Members:")) {
-				while (loglen && isspace(log[loglen-1]))
-					log[--loglen] = 0;
-				prepare_commit();
-				state = Members;
-				continue;
-			}
-				
-			if (loglen + linelen + 5 > sizeof(log))
-				continue;
-			memcpy(log + loglen, line, linelen);
-			loglen += linelen;
-			log[loglen++] = '\n';
-			continue;
-
-		case Members:
-			if (verbose)
-				printf("# M: %s\n", line);
-			if (!linelen) {
-				commit();
-				state = Header;
-				continue;
-			}
-			update_file(line);
-			continue;
-		}
-	}
-	return 0;
-}
diff --git a/git-cvsimport-script b/git-cvsimport-script
--- a/git-cvsimport-script
+++ b/git-cvsimport-script
@@ -1,38 +1,629 @@
-#!/bin/sh
+#!/usr/bin/perl -w
 
-usage () {
-	echo "Usage: git cvsimport [-v] [-z fuzz] <cvsroot> <module>"
-	exit 1
-}
-
-CVS2GIT=""
-CVSPS="--cvs-direct -x -A"
-while true; do
-	case "$1" in
-	-v) CVS2GIT="$1" ;;
-	-z) shift; CVSPS="$CVSPS -z $1" ;;
-	-*) usage ;;
-	*)  break ;;
-	esac
-	shift
-done
-
-export CVSROOT="$1"
-export MODULE="$2"
-if [ ! "$CVSROOT" ] || [ ! "$MODULE" ] ; then
-	usage
-fi
-
-cvsps -h 2>&1 | grep -q "cvsps version 2.1" >& /dev/null || {
-	echo "I need cvsps version 2.1"
-	exit 1
-}
-
-mkdir "$MODULE" || exit 1
-cd "$MODULE"
-
-TZ=UTC cvsps $CVSPS $MODULE > .git-cvsps-result
-[ -s .git-cvsps-result ] || exit 1
-git-cvs2git $CVS2GIT --cvsroot="$CVSROOT" --module="$MODULE" < .git-cvsps-result > .git-create-script || exit 1
-sh .git-create-script
+# This tool is copyright (c) 2005, Matthias Urlichs.
+# It is released under the Gnu Public License, version 2.
+#
+# The basic idea is to aggregate CVS check-ins into related changes.
+# Fortunately, "cvsps" does that for us; all we have to do is to parse
+# its output.
+#
+# Checking out the files is done by a single long-running CVS connection
+# / server process.
+#
+# The head revision is on branch "origin" by default.
+# You can change that with the '-o' option.
+
+use strict;
+use warnings;
+use Getopt::Std;
+use File::Path qw(mkpath);
+use File::Basename qw(basename dirname);
+use Time::Local;
+use IO::Socket;
+use IO::Pipe;
+use POSIX qw(strftime dup2);
+
+$SIG{'PIPE'}="IGNORE";
+$ENV{'TZ'}="UTC";
+
+our($opt_h,$opt_o,$opt_v,$opt_d);
+
+sub usage() {
+	print STDERR <<END;
+Usage: ${\basename $0}     # fetch/update GIT from CVS
+	   [ -o branch-for-HEAD ] [ -h ] [ -v ] [ -d CVSROOT ]
+       CVS_module [ GIT_repository ]
+END
+	exit(1);
+}
+
+getopts("hqvo:d:") or usage();
+usage if $opt_h;
+
+@ARGV == 1 or @ARGV == 2 or usage();
+
+my($cvs_tree, $git_tree) = @ARGV;
+
+if($opt_d) {
+	$ENV{"CVSROOT"} = $opt_d;
+} elsif($ENV{"CVSROOT"}) {
+	$opt_d = $ENV{"CVSROOT"};
+} else {
+	die "CVSROOT needs to be set";
+}
+$opt_o ||= "origin";
+$git_tree ||= ".";
+
+select(STDERR); $|=1; select(STDOUT);
+
+
+package CVSconn;
+# Basic CVS dialog.
+# We're only interested in connecting and downloading, so ...
+
+use POSIX qw(strftime dup2);
+
+sub new {
+	my($what,$repo,$subdir) = @_;
+	$what=ref($what) if ref($what);
+
+	my $self = {};
+	$self->{'buffer'} = "";
+	bless($self,$what);
+
+	$repo =~ s#/+$##;
+	$self->{'fullrep'} = $repo;
+	$self->conn();
+
+	$self->{'subdir'} = $subdir;
+	$self->{'lines'} = undef;
+
+	return $self;
+}
+
+sub conn {
+	my $self = shift;
+	my $repo = $self->{'fullrep'};
+	if($repo =~ s/^:pserver:(?:(.*?)(?::(.*?))?@)?([^:\/]*)(?::(\d*))?//) {
+		my($user,$pass,$serv,$port) = ($1,$2,$3,$4);
+		$user="anonymous" unless defined $user;
+		my $rr2 = "-";
+		unless($port) {
+			$rr2 = ":pserver:$user\@$serv:$repo";
+			$port=2401;
+		}
+		my $rr = ":pserver:$user\@$serv:$port$repo";
+
+		unless($pass) {
+			open(H,$ENV{'HOME'}."/.cvspass") and do {
+				# :pserver:cvs@mea.tmt.tele.fi:/cvsroot/zmailer Ah<Z
+				while(<H>) {
+					chomp;
+					s/^\/\d+\s+//;
+					my ($w,$p) = split(/\s/,$_,2);
+					if($w eq $rr or $w eq $rr2) {
+						$pass = $p;
+						last;
+					}
+				}
+			};
+		}
+		$pass="A" unless $pass;
+
+		my $s = IO::Socket::INET->new(PeerHost => $serv, PeerPort => $port);
+		die "Socket to $serv: $!\n" unless defined $s;
+		$s->write("BEGIN AUTH REQUEST\n$repo\n$user\n$pass\nEND AUTH REQUEST\n")
+			or die "Write to $serv: $!\n";
+		$s->flush();
+
+		my $rep = <$s>;
+
+		if($rep ne "I LOVE YOU\n") {
+			$rep="<unknown>" unless $rep;
+			die "AuthReply: $rep\n";
+		}
+		$self->{'socketo'} = $s;
+		$self->{'socketi'} = $s;
+	} else { # local: Fork off our own cvs server.
+		my $pr = IO::Pipe->new();
+		my $pw = IO::Pipe->new();
+		my $pid = fork();
+		die "Fork: $!\n" unless defined $pid;
+		unless($pid) {
+			$pr->writer();
+			$pw->reader();
+			dup2($pw->fileno(),0);
+			dup2($pr->fileno(),1);
+			$pr->close();
+			$pw->close();
+			exec("cvs","server");
+		}
+		$pw->writer();
+		$pr->reader();
+		$self->{'socketo'} = $pw;
+		$self->{'socketi'} = $pr;
+	}
+	$self->{'socketo'}->write("Root $repo\n");
+
+	# Trial and error says that this probably is the minimum set
+	$self->{'socketo'}->write("Valid-responses ok error Valid-requests Mode M Mbinary E F Checked-in Created Updated Merged Removed\n");
+
+	$self->{'socketo'}->write("valid-requests\n");
+	$self->{'socketo'}->flush();
+
+	chomp(my $rep=$self->readline());
+	if($rep !~ s/^Valid-requests\s*//) {
+		$rep="<unknown>" unless $rep;
+		die "Expected Valid-requests from server, but got: $rep\n";
+	}
+	chomp(my $res=$self->readline());
+	die "validReply: $res\n" if $res ne "ok";
+
+	$self->{'socketo'}->write("UseUnchanged\n") if $rep =~ /\bUseUnchanged\b/;
+	$self->{'repo'} = $repo;
+}
+
+sub readline {
+	my($self) = @_;
+	return $self->{'socketi'}->getline();
+}
+
+sub _file {
+	# Request a file with a given revision.
+	# Trial and error says this is a good way to do it. :-/
+	my($self,$fn,$rev) = @_;
+	$self->{'socketo'}->write("Argument -N\n") or return undef;
+	$self->{'socketo'}->write("Argument -P\n") or return undef;
+	# $self->{'socketo'}->write("Argument -ko\n") or return undef;
+	# -ko: Linus' version doesn't use it
+	$self->{'socketo'}->write("Argument -r\n") or return undef;
+	$self->{'socketo'}->write("Argument $rev\n") or return undef;
+	$self->{'socketo'}->write("Argument --\n") or return undef;
+	$self->{'socketo'}->write("Argument $self->{'subdir'}/$fn\n") or return undef;
+	$self->{'socketo'}->write("Directory .\n") or return undef;
+	$self->{'socketo'}->write("$self->{'repo'}\n") or return undef;
+	$self->{'socketo'}->write("Sticky T1.1\n") or return undef;
+	$self->{'socketo'}->write("co\n") or return undef;
+	$self->{'socketo'}->flush() or return undef;
+	$self->{'lines'} = 0;
+	return 1;
+}
+sub _line {
+	# Read a line from the server.
+	# ... except that 'line' may be an entire file. ;-)
+	my($self) = @_;
+	die "Not in lines" unless defined $self->{'lines'};
+
+	my $line;
+	my $res="";
+	while(defined($line = $self->readline())) {
+		# M U gnupg-cvs-rep/AUTHORS
+		# Updated gnupg-cvs-rep/
+		# /daten/src/rsync/gnupg-cvs-rep/AUTHORS
+		# /AUTHORS/1.1///T1.1
+		# u=rw,g=rw,o=rw
+		# 0
+		# ok
+
+		if($line =~ s/^(?:Created|Updated) //) {
+			$line = $self->readline(); # path
+			$line = $self->readline(); # Entries line
+			my $mode = $self->readline(); chomp $mode;
+			$self->{'mode'} = $mode;
+			defined (my $cnt = $self->readline())
+				or die "EOF from server after 'Changed'\n";
+			chomp $cnt;
+			die "Duh: Filesize $cnt" if $cnt !~ /^\d+$/;
+			$line="";
+			$res="";
+			while($cnt) {
+				my $buf;
+				my $num = $self->{'socketi'}->read($buf,$cnt);
+				die "Server: Filesize $cnt: $num: $!\n" if not defined $num or $num<=0;
+				$res .= $buf;
+				$cnt -= $num;
+			}
+		} elsif($line =~ s/^ //) {
+			$res .= $line;
+		} elsif($line =~ /^M\b/) {
+			# output, do nothing
+		} elsif($line =~ /^Mbinary\b/) {
+			my $cnt;
+			die "EOF from server after 'Mbinary'" unless defined ($cnt = $self->readline());
+			chomp $cnt;
+			die "Duh: Mbinary $cnt" if $cnt !~ /^\d+$/ or $cnt<1;
+			$line="";
+			while($cnt) {
+				my $buf;
+				my $num = $self->{'socketi'}->read($buf,$cnt);
+				die "S: Mbinary $cnt: $num: $!\n" if not defined $num or $num<=0;
+				$res .= $buf;
+				$cnt -= $num;
+			}
+		} else {
+			chomp $line;
+			if($line eq "ok") {
+				# print STDERR "S: ok (".length($res).")\n";
+				return $res;
+			} elsif($line =~ s/^E //) {
+				# print STDERR "S: $line\n";
+			} else {
+				die "Unknown: $line\n";
+			}
+		}
+	}
+}
+sub file {
+	my($self,$fn,$rev) = @_;
+	my $res;
+
+	if ($self->_file($fn,$rev)) {
+		$res = $self->_line();
+		return $res if defined $res;
+	}
+
+	# retry
+	$self->conn();
+	$self->_file($fn,$rev)
+		or die "No file command send\n";
+	$res = $self->_line();
+	die "No input: $fn $rev\n" unless defined $res;
+	return $res;
+}
+
+
+package main;
+
+my $cvs = CVSconn->new($opt_d, $cvs_tree);
+
+
+sub pdate($) {
+	my($d) = @_;
+	m#(\d{2,4})/(\d\d)/(\d\d)\s(\d\d):(\d\d)(?::(\d\d))?#
+		or die "Unparseable date: $d\n";
+	my $y=$1; $y-=1900 if $y>1900;
+	return timegm($6||0,$5,$4,$3,$2-1,$y);
+}
+
+sub pmode($) {
+	my($mode) = @_;
+	my $m = 0;
+	my $mm = 0;
+	my $um = 0;
+	for my $x(split(//,$mode)) {
+		if($x eq ",") {
+			$m |= $mm&$um;
+			$mm = 0;
+			$um = 0;
+		} elsif($x eq "u") { $um |= 0700;
+		} elsif($x eq "g") { $um |= 0070;
+		} elsif($x eq "o") { $um |= 0007;
+		} elsif($x eq "r") { $mm |= 0444;
+		} elsif($x eq "w") { $mm |= 0222;
+		} elsif($x eq "x") { $mm |= 0111;
+		} elsif($x eq "=") { # do nothing
+		} else { die "Unknown mode: $mode\n";
+		}
+	}
+	$m |= $mm&$um;
+	return $m;
+}
+
+my $tmpcv = "/var/cache/cvs";
+
+sub getwd() {
+	my $pwd = `pwd`;
+	chomp $pwd;
+	return $pwd;
+}
+
+-d $git_tree
+	or mkdir($git_tree,0777)
+	or die "Could not create $git_tree: $!";
+chdir($git_tree);
+
+my $last_branch = "";
+my $orig_branch = "";
+my %branch_date;
+
+my $git_dir = $ENV{"GIT_DIR"} || ".git";
+$git_dir = getwd()."/".$git_dir unless $git_dir =~ m#^/#;
+$ENV{"GIT_DIR"} = $git_dir;
+unless(-d $git_dir) {
+	system("git-init-db");
+	die "Cannot init the GIT db at $git_tree: $?\n" if $?;
+	system("git-read-tree");
+	die "Cannot init an empty tree: $?\n" if $?;
+
+	$last_branch = $opt_o;
+	$orig_branch = "";
+} else {
+	$last_branch = basename(readlink("$git_dir/HEAD"));
+	unless($last_branch) {
+		warn "Cannot read the last branch name: $! -- assuming 'master'\n";
+		$last_branch = "master";
+	}
+	$orig_branch = $last_branch;
+
+	# Get the last import timestamps
+	opendir(D,"$git_dir/refs/heads");
+	while(defined(my $head = readdir(D))) {
+		next if $head =~ /^\./;
+		open(F,"$git_dir/refs/heads/$head")
+			or die "Bad head branch: $head: $!\n";
+		chomp(my $ftag = <F>);
+		close(F);
+		open(F,"git-cat-file commit $ftag |");
+		while(<F>) {
+			next unless /^author\s.*\s(\d+)\s[-+]\d{4}$/;
+			$branch_date{$head} = $1;
+			last;
+		}
+		close(F);
+	}
+	closedir(D);
+}
+
+-d $git_dir
+	or die "Could not create git subdir ($git_dir).\n";
+
+my $pid = open(CVS,"-|");
+die "Cannot fork: $!\n" unless defined $pid;
+unless($pid) {
+	exec("cvsps","-A","--cvs-direct",$cvs_tree);
+	die "Could not start cvsps: $!\n";
+}
+
+
+## cvsps output:
+#---------------------
+#PatchSet 314
+#Date: 1999/09/18 13:03:59
+#Author: wkoch
+#Branch: STABLE-BRANCH-1-0
+#Ancestor branch: HEAD
+#Tag: (none)
+#Log:
+#    See ChangeLog: Sat Sep 18 13:03:28 CEST 1999  Werner Koch
+#Members:
+#	README:1.57->1.57.2.1
+#	VERSION:1.96->1.96.2.1
+#
+#---------------------
+
+my $state = 0;
+
+my($patchset,$date,$author,$branch,$ancestor,$tag,$logmsg);
+my(@old,@new);
+my $commit = sub {
+	my $pid;
+	system("git-update-cache","--force-remove","--",@old) if @old;
+	die "Cannot remove files: $?\n" if $?;
+	system("git-update-cache","--add","--",@new) if @new;
+	die "Cannot add files: $?\n" if $?;
+
+	$pid = open(C,"-|");
+	die "Cannot fork: $!" unless defined $pid;
+	unless($pid) {
+		exec("git-write-tree");
+		die "Cannot exec git-write-tree: $!\n";
+	}
+	chomp(my $tree = <C>);
+	length($tree) == 40
+		or die "Cannot get tree id ($tree): $!\n";
+	close(C)
+		or die "Error running git-write-tree: $?\n";
+	print "Tree ID $tree\n" if $opt_v;
+
+	my $parent = "";
+	if(open(C,"$git_dir/refs/heads/$last_branch")) {
+		chomp($parent = <C>);
+		close(C);
+		length($parent) == 40
+			or die "Cannot get parent id ($parent): $!\n";
+		print "Parent ID $parent\n" if $opt_v;
+	}
+
+	my $pr = IO::Pipe->new();
+	my $pw = IO::Pipe->new();
+	$pid = fork();
+	die "Fork: $!\n" unless defined $pid;
+	unless($pid) {
+		$pr->writer();
+		$pw->reader();
+		dup2($pw->fileno(),0);
+		dup2($pr->fileno(),1);
+		$pr->close();
+		$pw->close();
+
+		my @par = ();
+		@par = ("-p",$parent) if $parent;
+		exec("env",
+			"GIT_AUTHOR_NAME=$author",
+			"GIT_AUTHOR_EMAIL=$author",
+			"GIT_AUTHOR_DATE=".strftime("+0000 %Y-%m-%d %H:%M:%S",gmtime($date)),
+			"GIT_COMMITTER_NAME=$author",
+			"GIT_COMMITTER_EMAIL=$author",
+			"GIT_COMMITTER_DATE=".strftime("+0000 %Y-%m-%d %H:%M:%S",gmtime($date)),
+			"git-commit-tree", $tree,@par);
+		die "Cannot exec git-commit-tree: $!\n";
+	}
+	$pw->writer();
+	$pr->reader();
+	print $pw $logmsg
+		or die "Error writing to git-commit-tree: $!\n";
+	$pw->close();
+
+	print "Committed patch $patchset ($branch)\n" if $opt_v;
+	chomp(my $cid = <$pr>);
+	length($cid) == 40
+		or die "Cannot get commit id ($cid): $!\n";
+	print "Commit ID $cid\n" if $opt_v;
+	$pr->close();
+
+	waitpid($pid,0);
+	die "Error running git-commit-tree: $?\n" if $?;
+
+	open(C,">$git_dir/refs/heads/$branch")
+		or die "Cannot open branch $branch for update: $!\n";
+	print C "$cid\n"
+		or die "Cannot write branch $branch for update: $!\n";
+	close(C)
+		or die "Cannot write branch $branch for update: $!\n";
+
+	if($tag) {
+		open(C,">$git_dir/refs/tags/$tag")
+			or die "Cannot create tag $tag: $!\n";
+		print C "$cid\n"
+			or die "Cannot write tag $branch: $!\n";
+		close(C)
+			or die "Cannot write tag $branch: $!\n";
+		print "Created tag '$tag' on '$branch'\n" if $opt_v;
+	}
+
+	@old = ();
+	@new = ();
+};
+
+while(<CVS>) {
+	chomp;
+	if($state == 0 and /^-+$/) {
+		$state = 1;
+	} elsif($state == 0) {
+		$state = 1;
+		redo;
+	} elsif(($state==0 or $state==1) and s/^PatchSet\s+//) {
+		$patchset = 0+$_;
+		$state=2;
+	} elsif($state == 2 and s/^Date:\s+//) {
+		$date = pdate($_);
+		unless($date) {
+			print STDERR "Could not parse date: $_\n";
+			$state=0;
+			next;
+		}
+		$state=3;
+	} elsif($state == 3 and s/^Author:\s+//) {
+		s/\s+$//;
+		$author = $_;
+		$state = 4;
+	} elsif($state == 4 and s/^Branch:\s+//) {
+		s/\s+$//;
+		$branch = $_;
+		$state = 5;
+	} elsif($state == 5 and s/^Ancestor branch:\s+//) {
+		s/\s+$//;
+		$ancestor = $_;
+		$ancestor = $opt_o if $ancestor == "HEAD";
+		$state = 6;
+	} elsif($state == 5) {
+		$ancestor = undef;
+		$state = 6;
+		redo;
+	} elsif($state == 6 and s/^Tag:\s+//) {
+		s/\s+$//;
+		if($_ eq "(none)") {
+			$tag = undef;
+		} else {
+			$tag = $_;
+		}
+		$state = 7;
+	} elsif($state == 7 and /^Log:/) {
+		$logmsg = "";
+		$state = 8;
+	} elsif($state == 8 and /^Members:/) {
+		$branch = $opt_o if $branch eq "HEAD";
+		if(defined $branch_date{$branch} and $branch_date{$branch} >= $date) {
+			# skip
+			print "skip patchset $patchset: $date before $branch_date{$branch}\n";
+			$state = 11;
+			next;
+		}
+		if($ancestor) {
+			if(-f "$git_dir/refs/heads/$branch") {
+				print STDERR "Branch $branch already exists!\n";
+				$state=11;
+				next;
+			}
+			unless(open(H,"$git_dir/refs/heads/$ancestor")) {
+				print STDERR "Branch $ancestor does not exist!\n";
+				$state=11;
+				next;
+			}
+			chomp(my $id = <H>);
+			close(H);
+			unless(open(H,"> $git_dir/refs/heads/$branch")) {
+				print STDERR "Could not create branch $branch: $!\n";
+				$state=11;
+				next;
+			}
+			print H "$id\n"
+				or die "Could not write branch $branch: $!";
+			close(H)
+				or die "Could not write branch $branch: $!";
+		}
+		if(($ancestor || $branch) ne $last_branch) {
+			print "Switching from $last_branch to $branch\n" if $opt_v;
+			system("git-read-tree","-m","-u","$last_branch","$branch");
+			die "read-tree failed: $?\n" if $?;
+		}
+		if($branch ne $last_branch) {
+			unlink("$git_dir/HEAD");
+			symlink("refs/heads/$branch","$git_dir/HEAD");
+			$last_branch = $branch;
+		}
+		$state = 9;
+	} elsif($state == 8) {
+		$logmsg .= "$_\n";
+	} elsif($state == 9 and /^\s+(\S+):(INITIAL|\d(?:\.\d+)+)->(\d(?:\.\d+)+)\s*$/) {
+#	VERSION:1.96->1.96.2.1
+		my $init = ($2 eq "INITIAL");
+		my $fn = $1;
+		my $rev = $3;
+		$fn =~ s#^/+##;
+		my $data = $cvs->file($fn,$rev);
+		print "".($init ? "New" : "Update")." $fn: ".length($data)." bytes.\n";
+		mkpath(dirname($fn),$opt_v);
+		open(F,"> ./$fn")
+			or die "Cannot create '$fn': $!\n";
+		print F $data
+			or die "Cannot write to '$fn': $!\n";
+		close(F)
+			or die "Cannot write to '$fn': $!\n";
+		chmod(pmode($cvs->{'mode'}), $fn);
+		push(@new,$fn); # may be resurrected!
+	} elsif($state == 9 and /^\s+(\S+):\d(?:\.\d+)+->(\d(?:\.\d+)+)\(DEAD\)\s*$/) {
+		my $fn = $1;
+		$fn =~ s#^/+##;
+		push(@old,$fn);
+	} elsif($state == 9 and /^\s*$/) {
+		$state = 10;
+	} elsif(($state == 9 or $state == 10) and /^-+$/) {
+		&$commit();
+		$state = 1;
+	} elsif($state == 11 and /^-+$/) {
+		$state = 1;
+	} elsif(/^-+$/) { # end of unknown-line processing
+		$state = 1;
+	} elsif($state != 11) { # ignore stuff when skipping
+		print "* UNKNOWN LINE * $_\n";
+	}
+}
+&$commit() if $branch and $state != 11;
+
+# Now switch back to the branch we were in before all of this happened
+if($orig_branch) {
+	print "DONE; switching back to $orig_branch\n" if $opt_v;
+} else {
+	$orig_branch = "master";
+	print "DONE; creating $orig_branch branch\n" if $opt_v;
+	system("cp","$git_dir/refs/heads/$opt_o","$git_dir/refs/heads/master")
+		unless -f "$git_dir/refs/heads/master";
+}
+
+system("git-read-tree","-m","-u","$last_branch","$orig_branch");
+die "read-tree failed: $?\n" if $?;
+
+unlink("$git_dir/HEAD");
+symlink("refs/heads/$orig_branch","$git_dir/HEAD");
 

Re: [PATCH] cvsimport: rewritten in Perl

[Nicolas Pitre]

On Tue, 28 Jun 2005, Matthias Urlichs wrote:

> I just got my machine blocked from a CVS server which didn't like
> to get hammered with connections.
> 
> That was cvs2git's shell script. Which, by the way, is slow as hell.
> 
> Appended: a git-cvsimport script, written in Perl, which directly talks
> to the CVS server. If the repository is local, it runs a "cvs server"
> child. It produces the same git repository as Linus' version. It can do
> incremental imports. And it's 20 times faster (on my system, with a
> local CVS repository).

Tried it on the bkcvs repository from 
ftp.kernel.org/pub/scm/linux/kernel/bkcvs/linux-2.5/
(it can be retrieved with rsync as well)

Your script died after about 30 seconds with:

[...]
New scripts/lxdialog/Makefile: 0 bytes.
New scripts/lxdialog/checklist.c: 0 bytes.
New scripts/lxdialog/colors.h: 0 bytes.
New scripts/lxdialog/dialog.h: 0 bytes.
New scripts/lxdialog/inputbox.c: 0 bytes.
New scripts/lxdialog/lxdialog.c: 0 bytes.
New scripts/lxdialog/menubox.c: 0 bytes.
New scripts/lxdialog/msgbox.c: 0 bytes.
New scripts/lxdialog/textbox.c: 0 bytes.
New scripts/lxdialog/util.c: 0 bytes.
New scripts/lxdialog/yesno.c: 0 bytes.
Can't exec "git-update-cache": Argument list too long at /home/nico/bin/git-cvsimport-script line 402, <CVS> line 8254.
Cannot add files: -1

The original Linus version, although painfully slow, successfully 
converts the whole thing after a couple hours.

Also aren't those "0 bytes" a bit suspicious?


Nicolas

Re: [PATCH] cvsimport: rewritten in Perl

[Matthias Urlichs]

Hi, Nicolas Pitre wrote:

> ftp.kernel.org/pub/scm/linux/kernel/bkcvs/linux-2.5/

Oh well, I'll have a look. (I never bothered with bk2cvs; there's a
better tool which does bk2git directly.)

> New scripts/lxdialog/yesno.c: 0 bytes.

0-byte files are quite common in real-world repositories, so
throwing an error when I see one won't work.

> Can't exec "git-update-cache": Argument list too long at
> /home/nico/bin/git-cvsimport-script line 402, <CVS> line 8254.

That, at least, is easily fixable, I'll do a followup patch.

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
 - -
"A 'meaning of life' removes the overriding fear that existance is
 useless -- while denying the fact that living is its own greatest reward."
          [Fredric Rice, HolySmoke, December 1996]

Re: [PATCH] cvsimport: rewritten in Perl

[Matthias Urlichs]

Hi, Matthias Urlichs wrote:

> Hi, Nicolas Pitre wrote:
> 
>> ftp.kernel.org/pub/scm/linux/kernel/bkcvs/linux-2.5/
> 
> Oh well, I'll have a look. (I never bothered with bk2cvs; there's a
> better tool which does bk2git directly.)
> 
OK -- pulled, tested.

... though why you'd want the kernel via CVS instead of by direct
BK->GIT import is beyond me. ;-)

>> New scripts/lxdialog/yesno.c: 0 bytes.
> 
That's a feature. All the r1.1 CVS versions are zero-byte files, because
they're copied from BK/SCCS 1.0 versions, which are empty too (BK adds
the actual content in revision 1.1).

>> Can't exec "git-update-cache": Argument list too long at
>> /home/nico/bin/git-cvsimport-script line 402, <CVS> line 8254.
> 
Fixed; patch mailed separately.

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
 - -
My mother had a baby once.
					-- Jigger

[PATCH] cvsimport-in-Perl: Limit the number of arguments to git-update-cache

[Matthias Urlichs]

A small fix to git-cvsimport-script:

Limit the number of arguments to git-update-cache.
(Limiting their length may make a bit more sense, but I'm lazy.)

Signed-Off-By: Matthias Urlichs <smurf@smurf.noris.de>

---

diff --git a/git-cvsimport-script b/git-cvsimport-script
--- a/git-cvsimport-script
+++ b/git-cvsimport-script
@@ -395,14 +395,32 @@ my $state = 0;
 
 my($patchset,$date,$author,$branch,$ancestor,$tag,$logmsg);
 my(@old,@new);
 my $commit = sub {
 	my $pid;
-	system("git-update-cache","--force-remove","--",@old) if @old;
-	die "Cannot remove files: $?\n" if $?;
-	system("git-update-cache","--add","--",@new) if @new;
-	die "Cannot add files: $?\n" if $?;
+	while(@old) {
+		my @o2;
+		if(@old > 55) {
+			@o2 = splice(@old,0,50);
+		} else {
+			@o2 = @old;
+			@old = ();
+		}
+		system("git-update-cache","--force-remove","--",@o2);
+		die "Cannot remove files: $?\n" if $?;
+	}
+	while(@new) {
+		my @n2;
+		if(@new > 55) {
+			@n2 = splice(@new,0,50);
+		} else {
+			@n2 = @new;
+			@new = ();
+		}
+		system("git-update-cache","--add","--",@n2);
+		die "Cannot add files: $?\n" if $?;
+	}
 
 	$pid = open(C,"-|");
 	die "Cannot fork: $!" unless defined $pid;
 	unless($pid) {
 		exec("git-write-tree");
@@ -478,13 +496,10 @@ my $commit = sub {
 			or die "Cannot write tag $branch: $!\n";
 		close(C)
 			or die "Cannot write tag $branch: $!\n";
 		print "Created tag '$tag' on '$branch'\n" if $opt_v;
 	}
-
-	@old = ();
-	@new = ();
 };
 
 while(<CVS>) {
 	chomp;
 	if($state == 0 and /^-+$/) {

[PATCH] cvsimport: perform string comparison on "HEAD"

[Sven Verdoolaege]

--
diff --git a/git-cvsimport-script b/git-cvsimport-script
--- a/git-cvsimport-script
+++ b/git-cvsimport-script
@@ -529,7 +529,7 @@ while(<CVS>) {
 	} elsif($state == 5 and s/^Ancestor branch:\s+//) {
 		s/\s+$//;
 		$ancestor = $_;
-		$ancestor = $opt_o if $ancestor == "HEAD";
+		$ancestor = $opt_o if $ancestor eq "HEAD";
 		$state = 6;
 	} elsif($state == 5) {
 		$ancestor = undef;

Re: [PATCH] cvsimport: rewritten in Perl

[Sven Verdoolaege]

On Tue, Jun 28, 2005 at 09:23:23PM +0200, Matthias Urlichs wrote:
> I just got my machine blocked from a CVS server which didn't like
> to get hammered with connections.
> 
> That was cvs2git's shell script. Which, by the way, is slow as hell.
> 
> Appended: a git-cvsimport script, written in Perl, which directly talks
> to the CVS server. If the repository is local, it runs a "cvs server"
> child. It produces the same git repository as Linus' version. It can do
> incremental imports. And it's 20 times faster (on my system, with a
> local CVS repository).
> 

Could you try to make the resulting repository compatible
with a repository generated with the old cvs2git ?

This is the original version:
sh-3.00$ git-cat-file commit f6a92a7a774473bce12415200bab2788ea3b18f0
tree a0ec41a61461476c72c3967576225bd4772b6c8f
author risset <risset> 995295631 +0000
committer risset <risset> 995295631 +0000

Initial revision
sh-3.00$ 


This is your version:
sh-3.00$ git-cat-file commit db3540e3f670d4af4acefc723bab41a077c9300e
tree a0ec41a61461476c72c3967576225bd4772b6c8f
author risset <risset> 995295631 +0000
committer risset <risset> 995295631 +0000

Initial revision

sh-3.00$ 

Note the extra empty line.
Every commit is now different.

I'd really prefer it if I would not have to do the whole
conversion again.

skimo

Re: [PATCH] cvsimport: rewritten in Perl

[Matthias Urlichs]

[-- Attachment #1: Type: text/plain, Size: 610 bytes --]

Hi,

Sven Verdoolaege:
> Could you try to make the resulting repository compatible
> with a repository generated with the old cvs2git ?
> 
I believe I did... at least it worked that way two days ago. ;-)

> This is the original version:

Which repository is that?

> I'd really prefer it if I would not have to do the whole
> conversion again.
> 
So do I. ;-)

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
 - -
Imitation is the sincerest form of television.
		-- Fred Allen

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH] cvsimport: perform string comparison on "HEAD"

[Matthias Urlichs]

[-- Attachment #1: Type: text/plain, Size: 518 bytes --]

Hi,

Sven Verdoolaege:
> -		$ancestor = $opt_o if $ancestor == "HEAD";
> +		$ancestor = $opt_o if $ancestor eq "HEAD";

Duh. Thanks for spotting that. :-/

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
 - -
I am enough of an artist to draw freely upon my imagination.
Imagination is more important than knowledge.
Knowledge is limited. Imagination encircles the world.
		-- Albert Einstein

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH] cvsimport: rewritten in Perl

[Sven Verdoolaege]

On Thu, Jun 30, 2005 at 05:21:25PM +0200, Matthias Urlichs wrote:
> Sven Verdoolaege:
> > Could you try to make the resulting repository compatible
> > with a repository generated with the old cvs2git ?
> > 
> I believe I did... at least it worked that way two days ago. ;-)
> 
> > This is the original version:
> 
> Which repository is that?
> 

That was a private repository, but I tried it on another one
and I see that each commit message has an extra empty line
when compared to both an earlier cvs2git conversion and
"cvs log".
Presumably you have an extraneous "\n" somewhere.

-d :pserver:anonymous@exp-prolog.cs.kuleuven.ac.be:/cvs/dtse/ Polylib 

sverdool@pc117b:/local/kul/Polylib> cvs log typemap  |tail -n 15
make Polyhedron subclass of Domain
----------------------------
revision 1.2
date: 2001/11/16 16:01:58;  author: sven;  state: Exp;  lines: +15 -1
take array when matrix is needed
----------------------------
revision 1.1
date: 2001/11/06 15:19:44;  author: sven;  state: Exp;
branches:  1.1.1;
Initial revision
----------------------------
revision 1.1.1.1
date: 2001/11/06 15:19:44;  author: sven;  state: Exp;  lines: +0 -0

=============================================================================

sverdool@pc117b:/local/git/Polylib> cg-log typemap | tail -n 26
commit b53b0121ac62e9fa9cceb2c483e121d6ba1e24ed
tree 8d3d8125b8642a21cb9a5efb4e308f91ede06d56
parent 74d88525870de6c5648580a90920e85f294cae8c
author sven <sven> Mon, 14 Jan 2002 12:48:11 +0000
committer sven <sven> Mon, 14 Jan 2002 12:48:11 +0000

    make Polyhedron subclass of Domain
    

commit df6e38c6efe4f161ee35d60728813ddc0f1cb9cf
tree 43eef31ade1f7a88256e6dff9e66c2e7ca3201db
parent e38797be1f09c649600e681c8c7dc322e2db504d
author sven <sven> Fri, 16 Nov 2001 16:01:58 +0000
committer sven <sven> Fri, 16 Nov 2001 16:01:58 +0000

    take array when matrix is needed
    

commit 4c91382f379478f2ef8b9d875e7c81bddcb3aa57
tree e0ed0e7b11f9fc0341cdddaa95a8872fe5b01a5c
author sven <sven> Tue, 06 Nov 2001 15:19:44 +0000
committer sven <sven> Tue, 06 Nov 2001 15:19:44 +0000

    Initial revision
    

sverdool@pc117b:/local/git/Polylib> 

skimo

Re: [PATCH] cvsimport: rewritten in Perl

[Matthias Urlichs]

[-- Attachment #1: Type: text/plain, Size: 555 bytes --]

Hi,

Sven Verdoolaege:
> Presumably you have an extraneous "\n" somewhere.
> 
Strange.
That must have crept in when I switched from cg-commit to git-commit-tree.

I'll find it. However, you don't actually need to re-import your
existing CVS->GIT trees; as long as the dates and the branch names
match, my script will continue where the other left off.

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
 - -
You do not have mail.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH] cvsimport: rewritten in Perl

[Sven Verdoolaege]

On Thu, Jun 30, 2005 at 06:10:43PM +0200, Matthias Urlichs wrote:
> I'll find it. However, you don't actually need to re-import your
> existing CVS->GIT trees; as long as the dates and the branch names
> match, my script will continue where the other left off.

I wanted to check first whether it would do the right thing.
I'll wait for your update to do further checking.

skimo

Re: [PATCH] cvsimport: rewritten in Perl

[Matthias Urlichs]

[-- Attachment #1: Type: text/plain, Size: 528 bytes --]

Hi,

Sven Verdoolaege:
> I'll wait for your update to do further checking.
> 
Ah, found it -- that was my slightly modified version of cvsps, which
happens not to print an extra empty line at the end. cvs2git.c dutifully
strips these.

Duh. Will post an incremental patch shortly.

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
 - -
As I learn to trust the universe, I no longer need to carry a gun.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

[PATCH] cvsimport: Limit the log string to 32k

[Matthias Urlichs]

[-- Attachment #1: Type: text/plain, Size: 674 bytes --]

Teach the new cvsimport script to chop the log string's trailing whitespace.

Limit the log string to 32k, in order to be compatible with the old
cvs2git program.

Signed-Off-By: Matthias Urlichs <smurf@smurf.noris.de>

---

diff --git a/git-cvsimport-script b/git-cvsimport-script
--- a/git-cvsimport-script
+++ b/git-cvsimport-script
@@ -468,7 +468,12 @@ my $commit = sub {
 	}
 	$pw->writer();
 	$pr->reader();
-	print $pw $logmsg
+
+	# compatibility with git2cvs
+	substr($logmsg,32767) = "" if length($logmsg) > 32767;
+	$logmsg =~ s/[\s\n]+\z//;
+
+	print $pw "$logmsg\n"
 		or die "Error writing to git-commit-tree: $!\n";
 	$pw->close();
 

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH] cvsimport: rewritten in Perl

[Nicolas Pitre]

On Thu, 30 Jun 2005, Matthias Urlichs wrote:

> Hi, Matthias Urlichs wrote:
> 
> > Hi, Nicolas Pitre wrote:
> > 
> >> ftp.kernel.org/pub/scm/linux/kernel/bkcvs/linux-2.5/
> > 
> > Oh well, I'll have a look. (I never bothered with bk2cvs; there's a
> > better tool which does bk2git directly.)
> > 
> OK -- pulled, tested.
> 
> ... though why you'd want the kernel via CVS instead of by direct
> BK->GIT import is beyond me. ;-)

That's just a nice big test case.

You could try with, say, the gcc CVS as well for example.


Nicolas

Re: [PATCH] cvsimport-in-Perl: Limit the number of arguments to git-update-cache

[Nicolas Pitre]

On Thu, 30 Jun 2005, Matthias Urlichs wrote:

> A small fix to git-cvsimport-script:
> 
> Limit the number of arguments to git-update-cache.
> (Limiting their length may make a bit more sense, but I'm lazy.)

Why not using:

	write( "| xargs | git-update-cache --add --", @new)

or the like (I forget what the exact perl incantation is).


Nicolas

Re: [PATCH] cvsimport-in-Perl: Limit the number of arguments to git-update-cache

[Nicolas Pitre]

On Thu, 30 Jun 2005, Nicolas Pitre wrote:

> On Thu, 30 Jun 2005, Matthias Urlichs wrote:
> 
> > A small fix to git-cvsimport-script:
> > 
> > Limit the number of arguments to git-update-cache.
> > (Limiting their length may make a bit more sense, but I'm lazy.)
> 
> Why not using:
> 
> 	write( "| xargs | git-update-cache --add --", @new)

That example should be:

	write( "| xargs git-update-cache --add --", @new)

of course.


Nicolas

Re: [PATCH] cvsimport: rewritten in Perl

[Nicolas Pitre]

On Thu, 30 Jun 2005, Matthias Urlichs wrote:

> Duh. Will post an incremental patch shortly.

Until Linus merges it I'd suggest that you post the updated full patch 
instead.


Nicolas

Re: [PATCH] cvsimport-in-Perl: Limit the number of arguments to git-update-cache

[Matthias Urlichs]

[-- Attachment #1: Type: text/plain, Size: 630 bytes --]

Hi,

Nicolas Pitre:
> That example should be:
> 
> 	write( "| xargs git-update-cache --add --", @new)
> 
> of course.
> 
Actually, 'local $\ = "\0";' and 'xargs -0', which is roughly the point
where doing it all in Perl starts being more readable. ;-)

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
 - -
The only secure computer is one that's unplugged, locked in a safe,
and buried 20 feet under the ground in a secret location... and I'm
not even too sure about that one.
		-- Dennis Huges, FBI.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH] cvsimport: rewritten in Perl

[Stephen C. Tweedie]

Hi,

On Tue, 2005-06-28 at 20:23, Matthias Urlichs wrote:

> That was cvs2git's shell script. Which, by the way, is slow as hell.

I discovered why yesterday --- it's this patch:

commit deb153a75ae1f5eca628a38b911474a69edd242d
...
    [PATCH] cvs2git and file permissions
    git-cvs2git: propagate mode information
     
    Let cvs checkout in a temporary directory rather than
    using the pipe option to avoid loss of mode information.

This has a very unfortunate side effect --- when CVS checks out into a
new directory, it then waits for the time to advance to the next second
so that it can guarantee that future writes to the checked-out file
always change the timestamp from what it stores in CVS/Entries.

And when you're doing repeated checkouts, that means you get one per
second, max.  And so the git cvs import was taking one ... second ...
per ... revision ... per ... file, instead of the 8 complete changesets
or better I was getting before the above change.

Reverting the change back to checkout-via-pipe fixed the performance
problem for me, but obviously we're back to losing the mode information.

--Stephen

Re: [PATCH] cvsimport: rewritten in Perl

[Sven Verdoolaege]

On Thu, Jun 30, 2005 at 06:10:43PM +0200, Matthias Urlichs wrote:
> I'll find it. However, you don't actually need to re-import your
> existing CVS->GIT trees; as long as the dates and the branch names
> match, my script will continue where the other left off.

That seems to work, once I figured out I had to pass in the "-o master"
option (as cvs2git didn't create an "origin" branch).

If you don't, you get a rather cryptic message:
usage: git-read-tree (<sha> | -m [-u] <sha1> [<sha2> [<sha3>]])
read-tree failed: 256

You probably want to check whether the origin branch actually
exists.

Apparently you also need to pass the '-x' option to cvsps.
Otherwise, it won't look at anything new.

It would also be nice if the user could pass extra options
to cvsps (notably '-z').

skimo

Re: [PATCH] cvsimport: rewritten in Perl

[Matthias Urlichs]

[-- Attachment #1: Type: text/plain, Size: 1372 bytes --]

Hi,

Sven Verdoolaege:
> On Thu, Jun 30, 2005 at 06:10:43PM +0200, Matthias Urlichs wrote:
> > I'll find it. However, you don't actually need to re-import your
> > existing CVS->GIT trees; as long as the dates and the branch names
> > match, my script will continue where the other left off.
> 
> That seems to work, once I figured out I had to pass in the "-o master"
> option (as cvs2git didn't create an "origin" branch).
> 
Yes -- that's intentional, as the "master" branch is the one you're
going to add your own work to once the improt is finished. Using
"master" as CVS HEAD would mean that incremental imports no longer work.

> If you don't, you get a rather cryptic message:
> usage: git-read-tree (<sha> | -m [-u] <sha1> [<sha2> [<sha3>]])
> read-tree failed: 256
> 
> You probably want to check whether the origin branch actually
> exists.
> 
I'll add that, thanks.

> Apparently you also need to pass the '-x' option to cvsps.
> Otherwise, it won't look at anything new.
> 
Ditto.

> It would also be nice if the user could pass extra options
> to cvsps (notably '-z').
> 
Ditto.  ;-)

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
 - -
Don't let grass grow on the path of friendship.
		-- Blackfoot Indian

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH] cvsimport: rewritten in Perl

[Sven Verdoolaege]

On Thu, Jun 30, 2005 at 11:00:23PM +0200, Matthias Urlichs wrote:
> Sven Verdoolaege:
> > It would also be nice if the user could pass extra options
> > to cvsps (notably '-z').
> > 
> Ditto.  ;-)
> 

Actually, do you really have to call cvsps from within your
script ?  Why don't you just keep the small shell script
that links cvsps to cvs2git (your version) ?

skimo

Re: [PATCH] cvsimport: rewritten in Perl

[Matthias Urlichs]

[-- Attachment #1: Type: text/plain, Size: 1120 bytes --]

Hi,

Sven Verdoolaege:
> On Thu, Jun 30, 2005 at 11:00:23PM +0200, Matthias Urlichs wrote:
> > Sven Verdoolaege:
> > > It would also be nice if the user could pass extra options
> > > to cvsps (notably '-z').
> > > 
> > Ditto.  ;-)
> 
> Actually, do you really have to call cvsps from within your
> script ?  Why don't you just keep the small shell script
> that links cvsps to cvs2git (your version) ?

I dislike temporary files, a shell pipe can't catch errors in earlier
stages without major hackery, Linus didn't have a problem with ripping
it out, and in an earlier life this script was called cvs2bk and called
bk directly, so I kept that.

Enough reasons? ;-)  Sure, none of them really prevent me from doing it,
but OTOH I see no reason to resurrect the shell script either.

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
 - -
Christianity has nothing to offer a happy man
living in a natural, intelligible universe.
		-- George H. Smith, "Atheism: The Case Against God"

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: cvsimport: rewritten in Perl

[Matthias Urlichs]

Hi, Nicolas Pitre wrote:

> On Thu, 30 Jun 2005, Matthias Urlichs wrote:
> 
>> Duh. Will post an incremental patch shortly.
> 
> Until Linus merges it I'd suggest that you post the updated full patch 
> instead.

Personally, I'd prefer merging.

Linus/everybody_else  :-) : Please pull from

rsync://netz.smurf.noris.de/git.git#cvs2git

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
 - -
We were hungry when we got to Moscow, Soviet.
					-- Groucho Marx

Re: cvsimport: rewritten in Perl

[Sven Verdoolaege]

On Fri, Jul 01, 2005 at 11:43:34AM +0200, Matthias Urlichs wrote:
> Personally, I'd prefer merging.
> 
Pull this one then:

http://www.liacs.nl/~sverdool/git.git#cvs2git

skimo

[PATCH] Make specification of CVS module to convert optional.

[Sven Verdoolaege]

Make specification of CVS module to convert optional.

If we're inside a checked out CVS repository, there is
no need to explicitly specify the module as it is
available in CVS/Repository.
Also read CVS/Root if it's available and -d is not specified.
Finally, explicitly pass root to cvsps as CVS/Root takes
precedence over CVSROOT.

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>

---
commit f9714a4a0cd4ed0ccca3833743d98ea874a2232d
tree de5d7bba63538f29b8ea2b801d932b7679289b96
parent 1cd3674add10d1e511446f3034a1d233a3da7eab
author Sven Verdoolaege <skimo@kotnet.org> Sun, 03 Jul 2005 11:34:59 +0200
committer Sven Verdoolaege <skimo@kotnet.org> Sun, 03 Jul 2005 11:40:44 +0200

 Documentation/git-cvsimport-script.txt |    2 +-
 git-cvsimport-script                   |   34 ++++++++++++++++++++++++--------
 2 files changed, 27 insertions(+), 9 deletions(-)

diff --git a/Documentation/git-cvsimport-script.txt b/Documentation/git-cvsimport-script.txt
--- a/Documentation/git-cvsimport-script.txt
+++ b/Documentation/git-cvsimport-script.txt
@@ -11,7 +11,7 @@ SYNOPSIS
 --------
 'git-cvsimport-script' [ -o <branch-for-HEAD> ] [ -h ] [ -v ]
 			[ -d <CVSROOT> ] [ -p <options-for-cvsps> ]
-			<CVS_module> [ <GIT_repository> ]
+			[ -C <GIT_repository> ] [ <CVS_module> ]
 
 
 DESCRIPTION
diff --git a/git-cvsimport-script b/git-cvsimport-script
--- a/git-cvsimport-script
+++ b/git-cvsimport-script
@@ -26,35 +26,53 @@ use POSIX qw(strftime dup2);
 $SIG{'PIPE'}="IGNORE";
 $ENV{'TZ'}="UTC";
 
-our($opt_h,$opt_o,$opt_v,$opt_d,$opt_p);
+our($opt_h,$opt_o,$opt_v,$opt_d,$opt_p,$opt_C);
 
 sub usage() {
 	print STDERR <<END;
 Usage: ${\basename $0}     # fetch/update GIT from CVS
 	   [ -o branch-for-HEAD ] [ -h ] [ -v ] [ -d CVSROOT ]
-       [ -p opts-for-cvsps ]
-       CVS_module [ GIT_repository ]
+       [ -p opts-for-cvsps ] [ -C GIT_repository ]
+       [ CVS_module ]
 END
 	exit(1);
 }
 
-getopts("hqvo:d:p:") or usage();
+getopts("hqvo:d:p:C:") or usage();
 usage if $opt_h;
 
-@ARGV == 1 or @ARGV == 2 or usage();
-
-my($cvs_tree, $git_tree) = @ARGV;
+@ARGV <= 1 or usage();
 
 if($opt_d) {
 	$ENV{"CVSROOT"} = $opt_d;
+} elsif(-f 'CVS/Root') {
+	open my $f, '<', 'CVS/Root' or die 'Failed to open CVS/Root';
+	$opt_d = <$f>;
+	chomp $opt_d;
+	close $f;
+	$ENV{"CVSROOT"} = $opt_d;
 } elsif($ENV{"CVSROOT"}) {
 	$opt_d = $ENV{"CVSROOT"};
 } else {
 	die "CVSROOT needs to be set";
 }
 $opt_o ||= "origin";
+my $git_tree = $opt_C;
 $git_tree ||= ".";
 
+my $cvs_tree;
+if ($#ARGV == 0) {
+	$cvs_tree = $ARGV[0];
+} elsif (-f 'CVS/Repository') {
+	open my $f, '<', 'CVS/Repository' or 
+	    die 'Failed to open CVS/Repository';
+	$cvs_tree = <$f>;
+	chomp $cvs_tree;
+	close $f
+} else {
+	usage();
+}
+
 select(STDERR); $|=1; select(STDOUT);
 
 
@@ -378,7 +396,7 @@ die "Cannot fork: $!\n" unless defined $
 unless($pid) {
 	my @opt;
 	@opt = split(/,/,$opt_p) if defined $opt_p;
-	exec("cvsps",@opt,"-x","-A","--cvs-direct",$cvs_tree);
+	exec("cvsps",@opt,"-x","-A","--cvs-direct",'--root',$opt_d,$cvs_tree);
 	die "Could not start cvsps: $!\n";
 }
 

git-cvsimport-script: clean up documentation

[Sven Verdoolaege]

git-cvsimport-script: clean up documentation

Remove documentation of irrelevant "type" option.

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>

---
commit a92bebe6978edaea2885a627e7bef6f7f8b208c2
tree 25e9a3324ac489b7987be562923fd8e81a5de358
parent f9714a4a0cd4ed0ccca3833743d98ea874a2232d
author Sven Verdoolaege <skimo@kotnet.org> Sun, 03 Jul 2005 11:38:06 +0200
committer Sven Verdoolaege <skimo@kotnet.org> Sun, 03 Jul 2005 11:40:45 +0200

 Documentation/git-cvsimport-script.txt |    8 --------
 1 files changed, 0 insertions(+), 8 deletions(-)

diff --git a/Documentation/git-cvsimport-script.txt b/Documentation/git-cvsimport-script.txt
--- a/Documentation/git-cvsimport-script.txt
+++ b/Documentation/git-cvsimport-script.txt
@@ -48,14 +48,6 @@ OPTIONS
 <CVS_module>::
 	The CVS module you want to import. Relative to <CVSROOT>.
 
-<type>::
-	Typically this matches the real type of <object> but asking
-	for a type that can trivially dereferenced from the given
-	<object> is also permitted.  An example is to ask for a
-	"tree" with <object> being a commit object that contains it,
-	or to ask for a "blob" with <object> being a tag object that
-	points at it.
-
 -h::
 	Print a short usage message and exit.
 

git-cvsimport-script: Honour CVS_SERVER.

[Sven Verdoolaege]

Honour CVS_SERVER.

---
commit 8d0ea3117597933610e02907d14b443f8996ca3b
tree 8a8aba2772a770082e7d6bd47abd42c3e239ed2c
parent a92bebe6978edaea2885a627e7bef6f7f8b208c2
author Sven Verdoolaege <skimo@kotnet.org> Sun, 03 Jul 2005 12:26:51 +0200
committer Sven Verdoolaege <skimo@kotnet.org> Sun, 03 Jul 2005 12:26:51 +0200

 git-cvsimport-script |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/git-cvsimport-script b/git-cvsimport-script
--- a/git-cvsimport-script
+++ b/git-cvsimport-script
@@ -148,6 +148,8 @@ sub conn {
 		my $pw = IO::Pipe->new();
 		my $pid = fork();
 		die "Fork: $!\n" unless defined $pid;
+		my $cvs = 'cvs';
+		$cvs = $ENV{CVS_SERVER} if exists $ENV{CVS_SERVER};
 		unless($pid) {
 			$pr->writer();
 			$pw->reader();
@@ -155,7 +157,7 @@ sub conn {
 			dup2($pr->fileno(),1);
 			$pr->close();
 			$pw->close();
-			exec("cvs","server");
+			exec($cvs,"server");
 		}
 		$pw->writer();
 		$pr->reader();

[PATCH] git-cvsimport-script: Support :ext: access method.

[Sven Verdoolaege]

Support :ext: access method.

---
commit 34155390a576d8124e0adc864aaf2f11bbf5168b
tree 4918235816314f1d9981456cb05e395b6030c035
parent 8d0ea3117597933610e02907d14b443f8996ca3b
author Sven Verdoolaege <skimo@kotnet.org> Sun, 03 Jul 2005 13:02:06 +0200
committer Sven Verdoolaege <skimo@kotnet.org> Sun, 03 Jul 2005 13:02:06 +0200

 Documentation/git-cvsimport-script.txt |    3 ++-
 git-cvsimport-script                   |   23 +++++++++++++++++++++--
 2 files changed, 23 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-cvsimport-script.txt b/Documentation/git-cvsimport-script.txt
--- a/Documentation/git-cvsimport-script.txt
+++ b/Documentation/git-cvsimport-script.txt
@@ -26,7 +26,8 @@ OPTIONS
 -------
 -d <CVSROOT>::
 	The root of the CVS archive. May be local (a simple path) or remote;
-	currently, only the :pserver: access method is supported.
+	currently, only the :local:, :ext: and :pserver: access methods 
+	are supported.
 
 -o <branch-for-HEAD>::
 	The 'HEAD' branch from CVS is imported to the 'origin' branch within
diff --git a/git-cvsimport-script b/git-cvsimport-script
--- a/git-cvsimport-script
+++ b/git-cvsimport-script
@@ -143,13 +143,32 @@ sub conn {
 		}
 		$self->{'socketo'} = $s;
 		$self->{'socketi'} = $s;
-	} else { # local: Fork off our own cvs server.
+	} else { # local or ext: Fork off our own cvs server.
 		my $pr = IO::Pipe->new();
 		my $pw = IO::Pipe->new();
 		my $pid = fork();
 		die "Fork: $!\n" unless defined $pid;
 		my $cvs = 'cvs';
 		$cvs = $ENV{CVS_SERVER} if exists $ENV{CVS_SERVER};
+		my $rsh = 'rsh';
+		$rsh = $ENV{CVS_RSH} if exists $ENV{CVS_RSH};
+
+		my @cvs = ($cvs, 'server');
+		my ($local, $user, $host);
+		$local = $repo =~ s/:local://;
+		if (!$local) {
+		    $repo =~ s/:ext://;
+		    $local = !($repo =~ s/^(?:([^\@:]+)\@)?([^:]+)://);
+		    ($user, $host) = ($1, $2);
+		}
+		if (!$local) {
+		    if ($user) {
+			unshift @cvs, $rsh, '-l', $user, $host;
+		    } else {
+			unshift @cvs, $rsh, $host;
+		    }
+		}
+
 		unless($pid) {
 			$pr->writer();
 			$pw->reader();
@@ -157,7 +176,7 @@ sub conn {
 			dup2($pr->fileno(),1);
 			$pr->close();
 			$pw->close();
-			exec($cvs,"server");
+			exec(@cvs);
 		}
 		$pw->writer();
 		$pr->reader();

Re: cvsimport: rewritten in Perl

[Sven Verdoolaege]

On Sun, Jul 03, 2005 at 12:35:17PM +0200, Sven Verdoolaege wrote:
> On Fri, Jul 01, 2005 at 11:43:34AM +0200, Matthias Urlichs wrote:
> > Personally, I'd prefer merging.
> > 
> Pull this one then:
> 
> http://www.liacs.nl/~sverdool/git.git#cvs2git

That was a cogito branch, btw.
In git you'd say:

git-http-pull -c -v -t -a heads/cvs2git http://www.liacs.nl/~sverdool/git.git/

skimo

Re: cvsimport: rewritten in Perl

[Matthias Urlichs]

Hi, Sven Verdoolaege wrote:

>> http://www.liacs.nl/~sverdool/git.git#cvs2git
> 
> That was a cogito branch, btw.
> In git you'd say:
> 
> git-http-pull -c -v -t -a heads/cvs2git http://www.liacs.nl/~sverdool/git.git/
> 
Your patches make sense -- thanks; imported.

Linus: Please grab http://netz.smurf.noris.de/git/git.git/#cvs2git.

(A very git-ty URL, that.)

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
 - -
Quantum Mechanics is God's version of "Trust me."

Re: [PATCH] cvsimport: rewritten in Perl

[Sven Verdoolaege]

On Thu, Jun 30, 2005 at 05:21:25PM +0200, Matthias Urlichs wrote:
> Sven Verdoolaege:
> > Could you try to make the resulting repository compatible
> > with a repository generated with the old cvs2git ?
> > 
> I believe I did... at least it worked that way two days ago. ;-)
> 

I've found another difference.

On -p -z,60 -d :ext:anoncvs@savannah.gnu.org:/cvsroot/libtool libtool
on the gord branch,
the old cvs2git would create

[..]
commit 9d829ba90211956c53ebd40d2ba5daa454bedf03
tree e5aab59ed74c6e9fa1686dd914e2c7f7526f3bb6
parent 22acdc03db3de37ce19fac68babc1f2ec0011a42
author gord <gord> Tue, 01 Apr 1997 18:29:23 +0000
committer gord <gord> Tue, 01 Apr 1997 18:29:23 +0000

    *** empty log message ***

commit 22acdc03db3de37ce19fac68babc1f2ec0011a42
tree 4f8d0cc71683897d1332b485e59d164ff6a233c2
parent 27236f713eec57889949b0498cfaf446735b670d
author gord <gord> Tue, 01 Apr 1997 18:29:23 +0000
committer gord <gord> Tue, 01 Apr 1997 18:29:23 +0000


[..]
whereas your script creates:

[..]
commit f9450d4e45bdd760183383a4f698a70136e17f03
tree e5aab59ed74c6e9fa1686dd914e2c7f7526f3bb6
parent a8b919fa4784e5845f7cf61f53251804945c1b35
author gord <gord> Tue, 01 Apr 1997 18:29:23 +0000
committer gord <gord> Tue, 01 Apr 1997 18:29:23 +0000


commit a8b919fa4784e5845f7cf61f53251804945c1b35
tree b82881ae63c0cef645cd2d50050d2961d2c8871d
parent 27236f713eec57889949b0498cfaf446735b670d
author gord <gord> Tue, 01 Apr 1997 18:29:23 +0000
committer gord <gord> Tue, 01 Apr 1997 18:29:23 +0000

    *** empty log message ***

[..]

Note how the patchsets with the same date have somehow
been reversed.  Any ideas ?

I've also found another typo in your script; see my repo.

skimo

Re: [PATCH] cvsimport: rewritten in Perl

[Matthias Urlichs]

[-- Attachment #1: Type: text/plain, Size: 520 bytes --]

Hi,

Sven Verdoolaege:
> Note how the patchsets with the same date have somehow
> been reversed.  Any ideas ?
> 
No. I process the lines from cvsps in the order I get them...

> I've also found another typo in your script; see my repo.
> 
Thanks. I guess. ;-)

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
 - -
People who run down others are taking a roundabout way of praising themselves.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH] cvsimport: rewritten in Perl

[Sven Verdoolaege]

Hi,

On Mon, Jul 04, 2005 at 03:49:48AM +0200, Matthias Urlichs wrote:
> Sven Verdoolaege:
> > Note how the patchsets with the same date have somehow
> > been reversed.  Any ideas ?
> > 
> No. I process the lines from cvsps in the order I get them...
> 

Ok, I reran cvs2git and now I get the same order as with
your script.  Something must have changed in the output
of cvsps.

skimo

[PATCH] git-cvsimport-script: leave working directory alone.

[Sven Verdoolaege]

git-cvsimport-script: leave working directory alone.

---
commit 2eb6d82eaa869a1faf4ba3326fd628f76f9f93a8
tree 3e59c8c298e51a8ae20102b4b139c84c79fa54a0
parent 866d13108e969773347828daa9b7f3476ec70cb8
author Sven Verdoolaege <skimo@kotnet.org> Mon, 04 Jul 2005 00:43:26 +0200
committer Sven Verdoolaege <skimo@kotnet.org> Mon, 04 Jul 2005 00:43:26 +0200

 git-cvsimport-script |   81 ++++++++++++++++++++++++++++++--------------------
 1 files changed, 48 insertions(+), 33 deletions(-)

diff --git a/git-cvsimport-script b/git-cvsimport-script
--- a/git-cvsimport-script
+++ b/git-cvsimport-script
@@ -80,6 +80,8 @@ package CVSconn;
 # Basic CVS dialog.
 # We're only interested in connecting and downloading, so ...
 
+use File::Spec;
+use File::Temp qw(tempfile);
 use POSIX qw(strftime dup2);
 
 sub new {
@@ -231,11 +233,11 @@ sub _file {
 sub _line {
 	# Read a line from the server.
 	# ... except that 'line' may be an entire file. ;-)
-	my($self) = @_;
+	my($self, $fh) = @_;
 	die "Not in lines" unless defined $self->{'lines'};
 
 	my $line;
-	my $res="";
+	my $res=0;
 	while(defined($line = $self->readline())) {
 		# M U gnupg-cvs-rep/AUTHORS
 		# Updated gnupg-cvs-rep/
@@ -255,16 +257,18 @@ sub _line {
 			chomp $cnt;
 			die "Duh: Filesize $cnt" if $cnt !~ /^\d+$/;
 			$line="";
-			$res="";
+			$res=0;
 			while($cnt) {
 				my $buf;
 				my $num = $self->{'socketi'}->read($buf,$cnt);
 				die "Server: Filesize $cnt: $num: $!\n" if not defined $num or $num<=0;
-				$res .= $buf;
+				print $fh $buf;
+				$res += $num;
 				$cnt -= $num;
 			}
 		} elsif($line =~ s/^ //) {
-			$res .= $line;
+			print $fh $line;
+			$res += length($line);
 		} elsif($line =~ /^M\b/) {
 			# output, do nothing
 		} elsif($line =~ /^Mbinary\b/) {
@@ -277,7 +281,8 @@ sub _line {
 				my $buf;
 				my $num = $self->{'socketi'}->read($buf,$cnt);
 				die "S: Mbinary $cnt: $num: $!\n" if not defined $num or $num<=0;
-				$res .= $buf;
+				print $fh $buf;
+				$res += $num;
 				$cnt -= $num;
 			}
 		} else {
@@ -297,18 +302,21 @@ sub file {
 	my($self,$fn,$rev) = @_;
 	my $res;
 
-	if ($self->_file($fn,$rev)) {
-		$res = $self->_line();
-		return $res if defined $res;
+	my ($fh, $name) = tempfile('gitcvs.XXXXXX', 
+		    DIR => File::Spec->tmpdir(), UNLINK => 1);
+
+	$self->_file($fn,$rev) and $res = $self->_line($fh);
+
+	if (!defined $res) {
+	    # retry
+	    $self->conn();
+	    $self->_file($fn,$rev)
+		    or die "No file command send\n";
+	    $res = $self->_line($fh);
+	    die "No input: $fn $rev\n" unless defined $res;
 	}
 
-	# retry
-	$self->conn();
-	$self->_file($fn,$rev)
-		or die "No file command send\n";
-	$res = $self->_line();
-	die "No input: $fn $rev\n" unless defined $res;
-	return $res;
+	return ($name, $res);
 }
 
 
@@ -457,13 +465,14 @@ my $commit = sub {
 	}
 	while(@new) {
 		my @n2;
-		if(@new > 55) {
-			@n2 = splice(@new,0,50);
+		if(@new > 12) {
+			@n2 = splice(@new,0,10);
 		} else {
 			@n2 = @new;
 			@new = ();
 		}
-		system("git-update-cache","--add","--",@n2);
+		system("git-update-cache","--add",
+			(map { ('--cacheinfo', @$_) } @n2));
 		die "Cannot add files: $?\n" if $?;
 	}
 
@@ -631,7 +640,7 @@ while(<CVS>) {
 		}
 		if(($ancestor || $branch) ne $last_branch) {
 			print "Switching from $last_branch to $branch\n" if $opt_v;
-			system("git-read-tree","-m","-u","$last_branch","$branch");
+			system("git-read-tree","-m","$last_branch","$branch");
 			die "read-tree failed: $?\n" if $?;
 		}
 		if($branch ne $last_branch) {
@@ -648,17 +657,16 @@ while(<CVS>) {
 		my $fn = $1;
 		my $rev = $3;
 		$fn =~ s#^/+##;
-		my $data = $cvs->file($fn,$rev);
-		print "".($init ? "New" : "Update")." $fn: ".length($data)." bytes.\n" if $opt_v;
-		mkpath(dirname($fn),$opt_v);
-		open(F,"> ./$fn")
-			or die "Cannot create '$fn': $!\n";
-		print F $data
-			or die "Cannot write to '$fn': $!\n";
-		close(F)
-			or die "Cannot write to '$fn': $!\n";
-		chmod(pmode($cvs->{'mode'}), $fn);
-		push(@new,$fn); # may be resurrected!
+		my ($tmpname, $size) = $cvs->file($fn,$rev);
+		print "".($init ? "New" : "Update")." $fn: $size bytes.\n" if $opt_v;
+		open my $F, '-|', "git-write-blob $tmpname"
+			or die "Cannot create object: $!\n";
+		my $sha = <$F>;
+		chomp $sha;
+		close $F;
+		unlink($tmpname);
+		my $mode = pmode($cvs->{'mode'});
+		push(@new,[$mode, $sha, $fn]); # may be resurrected!
 	} elsif($state == 9 and /^\s+(\S+):\d(?:\.\d+)+->(\d(?:\.\d+)+)\(DEAD\)\s*$/) {
 		my $fn = $1;
 		$fn =~ s#^/+##;
@@ -688,8 +696,15 @@ if($orig_branch) {
 		unless -f "$git_dir/refs/heads/master";
 }
 
-system("git-read-tree","-m","-u","$last_branch","$orig_branch");
-die "read-tree failed: $?\n" if $?;
+if ($orig_branch) {
+	system("git-read-tree",$last_branch);
+	die "read-tree failed: $?\n" if $?;
+} else {
+	system('git-read-tree', $orig_branch);
+	die "read-tree failed: $?\n" if $?;
+	system('git-checkout-cache', '-a');
+	die "checkout-cache failed: $?\n" if $?;
+}
 
 unlink("$git_dir/HEAD");
 symlink("refs/heads/$orig_branch","$git_dir/HEAD");

[PATCH] git-cvsimport-script: use private index.

[Sven Verdoolaege]

git-cvsimport-script: use private index.

---
commit 79ee456cf222982f7ee3f003440c57b5f7cffa8b
tree c27c7f8bafa48d81a4d9f7562b851681984a9c7e
parent 2eb6d82eaa869a1faf4ba3326fd628f76f9f93a8
author Sven Verdoolaege <skimo@kotnet.org> Mon, 04 Jul 2005 13:36:59 +0200
committer Sven Verdoolaege <skimo@kotnet.org> Mon, 04 Jul 2005 13:36:59 +0200

 git-cvsimport-script |   43 +++++++++++++++++++++++--------------------
 1 files changed, 23 insertions(+), 20 deletions(-)

diff --git a/git-cvsimport-script b/git-cvsimport-script
--- a/git-cvsimport-script
+++ b/git-cvsimport-script
@@ -16,6 +16,8 @@
 use strict;
 use warnings;
 use Getopt::Std;
+use File::Spec;
+use File::Temp qw(tempfile);
 use File::Path qw(mkpath);
 use File::Basename qw(basename dirname);
 use Time::Local;
@@ -377,6 +379,12 @@ my %branch_date;
 my $git_dir = $ENV{"GIT_DIR"} || ".git";
 $git_dir = getwd()."/".$git_dir unless $git_dir =~ m#^/#;
 $ENV{"GIT_DIR"} = $git_dir;
+my $orig_git_index;
+$orig_git_index = $ENV{GIT_INDEX_FILE} if exists $ENV{GIT_INDEX_FILE};
+my ($git_ih, $git_index) = tempfile('gitXXXXXX', SUFFIX => '.idx',
+				    DIR => File::Spec->tmpdir());
+close ($git_ih);
+$ENV{GIT_INDEX_FILE} = $git_index;
 unless(-d $git_dir) {
 	system("git-init-db");
 	die "Cannot init the GIT db at $git_tree: $?\n" if $?;
@@ -398,6 +406,9 @@ unless(-d $git_dir) {
 	}
 	$orig_branch = $last_branch;
 
+	# populate index
+	system('git-read-tree', $last_branch);
+
 	# Get the last import timestamps
 	opendir(D,"$git_dir/refs/heads");
 	while(defined(my $head = readdir(D))) {
@@ -643,11 +654,6 @@ while(<CVS>) {
 			system("git-read-tree","-m","$last_branch","$branch");
 			die "read-tree failed: $?\n" if $?;
 		}
-		if($branch ne $last_branch) {
-			unlink("$git_dir/HEAD");
-			symlink("refs/heads/$branch","$git_dir/HEAD");
-			$last_branch = $branch;
-		}
 		$state = 9;
 	} elsif($state == 8) {
 		$logmsg .= "$_\n";
@@ -686,26 +692,23 @@ while(<CVS>) {
 }
 &$commit() if $branch and $state != 11;
 
+unlink($git_index);
+
 # Now switch back to the branch we were in before all of this happened
 if($orig_branch) {
-	print "DONE; switching back to $orig_branch\n" if $opt_v;
+	print "DONE\n" if $opt_v;
 } else {
 	$orig_branch = "master";
 	print "DONE; creating $orig_branch branch\n" if $opt_v;
 	system("cp","$git_dir/refs/heads/$opt_o","$git_dir/refs/heads/master")
 		unless -f "$git_dir/refs/heads/master";
+	unlink("$git_dir/HEAD");
+	symlink("refs/heads/$orig_branch","$git_dir/HEAD");
+	if (defined $orig_git_index) {
+	    $ENV{GIT_INDEX_FILE} = $orig_git_index;
+	} else {
+	    delete $ENV{GIT_INDEX_FILE};
+	}
+	system('git checkout');
+	die "checkout failed: $?\n" if $?;
 }
-
-if ($orig_branch) {
-	system("git-read-tree",$last_branch);
-	die "read-tree failed: $?\n" if $?;
-} else {
-	system('git-read-tree', $orig_branch);
-	die "read-tree failed: $?\n" if $?;
-	system('git-checkout-cache', '-a');
-	die "checkout-cache failed: $?\n" if $?;
-}
-
-unlink("$git_dir/HEAD");
-symlink("refs/heads/$orig_branch","$git_dir/HEAD");
-

Re: [PATCH] cvsimport: rewritten in Perl

[Sven Verdoolaege]

On Thu, Jun 30, 2005 at 11:00:23PM +0200, Matthias Urlichs wrote:
> Sven Verdoolaege:
> > It would also be nice if the user could pass extra options
> > to cvsps (notably '-z').
> > 
> Ditto.  ;-)

Why not an explicit '-z' option as in the current git-cvsimport-script ?

skimo

Re: [PATCH] cvsimport: rewritten in Perl

[Sven Verdoolaege]

On Mon, Jul 04, 2005 at 03:53:27PM +0200, Matthias Urlichs wrote:
> Hi,
> 
> Sven Verdoolaege:
> > Why not an explicit '-z' option as in the current git-cvsimport-script ?
> 
> Because my code doesn't support compressed cvs connections:
> a -z that doesn't work except for the rlog part would be a lie.

I was talking about the cvsps '-z' option (see current git-cvsimport-script).
Are you saying you want to reserve that option to signify compressed
cvs connections ?

skimo

Re: [PATCH] cvsimport: rewritten in Perl

[Matthias Urlichs]

[-- Attachment #1: Type: text/plain, Size: 529 bytes --]

Hi,

Sven Verdoolaege:
> Why not an explicit '-z' option as in the current git-cvsimport-script ?

Because my code doesn't support compressed cvs connections:
a -z that doesn't work except for the rlog part would be a lie.

Feel free to add that code. ;-)

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
 - -
Confidence is simply that quiet, assured feeling you have before you
fall flat on your face.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

[PATCH] git-cvsimport-script: fix branch switching

[Sven Verdoolaege]

git-cvsimport-script: fix branch switching

Previous patch broke branch switching.

---
commit 46e63efc072bc440e4c6aad33d3157b70f5172b6
tree 2c4fd7286e29e6041808d07874ef2151e3876676
parent 79ee456cf222982f7ee3f003440c57b5f7cffa8b
author Sven Verdoolaege <skimo@kotnet.org> Mon, 04 Jul 2005 15:28:36 +0200
committer Sven Verdoolaege <skimo@kotnet.org> Mon, 04 Jul 2005 15:28:36 +0200

 git-cvsimport-script |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/git-cvsimport-script b/git-cvsimport-script
--- a/git-cvsimport-script
+++ b/git-cvsimport-script
@@ -651,9 +651,10 @@ while(<CVS>) {
 		}
 		if(($ancestor || $branch) ne $last_branch) {
 			print "Switching from $last_branch to $branch\n" if $opt_v;
-			system("git-read-tree","-m","$last_branch","$branch");
+			system("git-read-tree", $branch);
 			die "read-tree failed: $?\n" if $?;
 		}
+		$last_branch = $branch if $branch ne $last_branch;
 		$state = 9;
 	} elsif($state == 8) {
 		$logmsg .= "$_\n";

[PATCH] git-cvsimport-script: update cvsps cache instead of rebuilding it

[Sven Verdoolaege]

git-cvsimport-script: update cvsps cache instead of rebuilding it

Updating the cache is sufficient for most purposes.
If users really want to rebuild the cache, they can specify
the option themselves.

---
commit 6e7e37b0bfc921aa1f0cb30560fc128e87a41966
tree 12a9303d4ba4566d9e081b2c375648685ce41e93
parent 46e63efc072bc440e4c6aad33d3157b70f5172b6
author Sven Verdoolaege <skimo@kotnet.org> Mon, 04 Jul 2005 15:35:30 +0200
committer Sven Verdoolaege <skimo@kotnet.org> Mon, 04 Jul 2005 15:35:30 +0200

 git-cvsimport-script |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/git-cvsimport-script b/git-cvsimport-script
--- a/git-cvsimport-script
+++ b/git-cvsimport-script
@@ -436,7 +436,7 @@ die "Cannot fork: $!\n" unless defined $
 unless($pid) {
 	my @opt;
 	@opt = split(/,/,$opt_p) if defined $opt_p;
-	exec("cvsps",@opt,"-x","-A","--cvs-direct",'--root',$opt_d,$cvs_tree);
+	exec("cvsps",@opt,"-u","-A","--cvs-direct",'--root',$opt_d,$cvs_tree);
 	die "Could not start cvsps: $!\n";
 }
 

Re: [PATCH] cvsimport: rewritten in Perl

[Matthias Urlichs]

[-- Attachment #1: Type: text/plain, Size: 966 bytes --]

Hi,

Sven Verdoolaege:
> > Because my code doesn't support compressed cvs connections:
> > a -z that doesn't work except for the rlog part would be a lie.
> 
> I was talking about the cvsps '-z' option (see current git-cvsimport-script).
> Are you saying you want to reserve that option to signify compressed
> cvs connections ?
> 
Sorry, I was confused -- with cvsps, -capital-Z says to compress.

Ideally, I'd prefer to recycle standard CVS options as much as possible, 
but given that the confusion is already there (worse: cvs' -z wants an
argument (compression level), cvsps' -Z doesn't) that may not actually
make sense. *Shrug*

I'm too happy when other people improve my tools to get hung up on
details like that. ;-)

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
 - -
You'll feel much better once you've given up hope.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH] cvsimport: rewritten in Perl

[Sven Verdoolaege]

On Mon, Jul 04, 2005 at 04:36:37PM +0200, Matthias Urlichs wrote:
> Ideally, I'd prefer to recycle standard CVS options as much as possible, 
> but given that the confusion is already there (worse: cvs' -z wants an
> argument (compression level), cvsps' -Z doesn't) that may not actually
> make sense. *Shrug*
> 
> I'm too happy when other people improve my tools to get hung up on
> details like that. ;-)

Here it is, then.

skimo
--
git-cvsimport-script: provide direct support for cvsps -z option

---
commit 28537171e7ec23c8677ea6e77c208583f95caa28
tree ca80ed2fad05b150984c14a5364dac8d3e307120
parent 6e7e37b0bfc921aa1f0cb30560fc128e87a41966
author Sven Verdoolaege <skimo@kotnet.org> Mon, 04 Jul 2005 17:10:06 +0200
committer Sven Verdoolaege <skimo@kotnet.org> Mon, 04 Jul 2005 17:10:06 +0200

 git-cvsimport-script |    9 +++++----
 1 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/git-cvsimport-script b/git-cvsimport-script
--- a/git-cvsimport-script
+++ b/git-cvsimport-script
@@ -28,19 +28,19 @@ use POSIX qw(strftime dup2);
 $SIG{'PIPE'}="IGNORE";
 $ENV{'TZ'}="UTC";
 
-our($opt_h,$opt_o,$opt_v,$opt_d,$opt_p,$opt_C);
+our($opt_h,$opt_o,$opt_v,$opt_d,$opt_p,$opt_C,$opt_z);
 
 sub usage() {
 	print STDERR <<END;
 Usage: ${\basename $0}     # fetch/update GIT from CVS
-	   [ -o branch-for-HEAD ] [ -h ] [ -v ] [ -d CVSROOT ]
-       [ -p opts-for-cvsps ] [ -C GIT_repository ]
+       [ -o branch-for-HEAD ] [ -h ] [ -v ] [ -d CVSROOT ]
+       [ -p opts-for-cvsps ] [ -C GIT_repository ] [ -z fuzz ]
        [ CVS_module ]
 END
 	exit(1);
 }
 
-getopts("hqvo:d:p:C:") or usage();
+getopts("hqvo:d:p:C:z:") or usage();
 usage if $opt_h;
 
 @ARGV <= 1 or usage();
@@ -436,6 +436,7 @@ die "Cannot fork: $!\n" unless defined $
 unless($pid) {
 	my @opt;
 	@opt = split(/,/,$opt_p) if defined $opt_p;
+	unshift @opt, '-z', $opt_z if defined $opt_z;
 	exec("cvsps",@opt,"-u","-A","--cvs-direct",'--root',$opt_d,$cvs_tree);
 	die "Could not start cvsps: $!\n";
 }

Re: cvsimport: rewritten in Perl

[Wolfgang Denk]

In message <pan.2005.07.03.13.44.46.995524@smurf.noris.de> you wrote:
> 
> Linus: Please grab http://netz.smurf.noris.de/git/git.git/#cvs2git.

I tested this on the U-Boot CVS repository (available at
http://cvs.sourceforge.net/cvstarballs/u-boot-cvsroot.tar.bz2).

I got:

-> ~/test/git-cvsimport-script -d ~/git/u-boot-SF/u-boot -C u-boot u-boot
cvs_direct initialized to CVSROOT /home/wd/git/u-boot-SF/u-boot
cvs rlog: Logging u-boot
cvs rlog: Logging u-boot/board
...
cvs rlog: Logging u-boot/tools/scripts
cvs rlog: Logging u-boot/tools/updater
WARNING: Invalid PatchSet 815, Tag LABEL_2003_12_08_MKR:
    include/configs/svm_sc8xx.h:1.3=after, CHANGELOG:1.170=before. Treated as 'before'
WARNING: Invalid PatchSet 816, Tag LABEL_2003_12_08_MKR:
    include/configs/rmu.h:1.4=after, CHANGELOG:1.171=before. Treated as 'before'
WARNING: Invalid PatchSet 815, Tag LABEL_2003_12_08_MKR:
    include/configs/TQM823M.h:1.3=after, CHANGELOG:1.170=before. Treated as 'before'
WARNING: Invalid PatchSet 815, Tag LABEL_2003_12_08_MKR:
    include/configs/TQM823L.h:1.9=after, CHANGELOG:1.170=before. Treated as 'before'
WARNING: Invalid PatchSet 816, Tag LABEL_2003_12_08_MKR:
    include/configs/IceCube.h:1.16=after, CHANGELOG:1.171=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_12_08_MKR:
    fs/fat/fat.c:1.5=after, CHANGELOG:1.165=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_12_08_MKR:
    fs/fat/fat.c:1.5=after, board/RRvision/RRvision.c:1.2=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_12_08_MKR:
    fs/fat/fat.c:1.5=after, board/atc/atc.c:1.6=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_12_08_MKR:
    fs/fat/fat.c:1.5=after, board/c2mon/c2mon.c:1.2=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_12_08_MKR:
    fs/fat/fat.c:1.5=after, board/etx094/etx094.c:1.2=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_12_08_MKR:
    fs/fat/fat.c:1.5=after, board/evb64260/flash.c:1.5=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_12_08_MKR:
    fs/fat/fat.c:1.5=after, board/rmu/rmu.c:1.3=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_12_08_MKR:
    fs/fat/fat.c:1.5=after, board/tqm8260/tqm8260.c:1.4=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_12_08_MKR:
    fs/fat/fat.c:1.5=after, board/tqm8xx/tqm8xx.c:1.4=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_12_08_MKR:
    fs/fat/fat.c:1.5=after, include/version.h:1.20=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_12_08_MKR:
    fs/fat/fat.c:1.5=after, lib_arm/board.c:1.19=before. Treated as 'before'
WARNING: Invalid PatchSet 815, Tag LABEL_2003_12_08_MKR:
    cpu/mpc8xx/lcd.c:1.14=after, CHANGELOG:1.170=before. Treated as 'before'
WARNING: Invalid PatchSet 817, Tag LABEL_2003_12_08_MKR:
    common/cmd_bootm.c:1.31=after, CHANGELOG:1.172=before. Treated as 'before'
WARNING: Invalid PatchSet 815, Tag LABEL_2003_12_08_MKR:
    board/tqm8xx/tqm8xx.c:1.5=after, CHANGELOG:1.170=before. Treated as 'before'
WARNING: Invalid PatchSet 815, Tag LABEL_2003_12_08_MKR:
    README:1.60=after, CHANGELOG:1.170=before. Treated as 'before'
WARNING: Invalid PatchSet 815, Tag LABEL_2003_12_08_MKR:
    Makefile:1.65=after, CHANGELOG:1.170=before. Treated as 'before'
WARNING: Invalid PatchSet 816, Tag LABEL_2003_12_08_MKR:
    include/configs/IceCube.h:1.16=after, CHANGELOG:1.171=before. Treated as 'before'
WARNING: Invalid PatchSet 815, Tag LABEL_2003_12_08_MKR:
    include/configs/TQM823M.h:1.3=after, CHANGELOG:1.170=before. Treated as 'before'
WARNING: Invalid PatchSet 815, Tag LABEL_2003_12_08_MKR:
    include/configs/svm_sc8xx.h:1.3=after, CHANGELOG:1.170=before. Treated as 'before'
WARNING: Invalid PatchSet 817, Tag LABEL_2003_12_08_MKR:
    common/cmd_bootm.c:1.31=after, CHANGELOG:1.172=before. Treated as 'before'
WARNING: Invalid PatchSet 815, Tag LABEL_2003_12_08_MKR:
    cpu/mpc8xx/lcd.c:1.14=after, CHANGELOG:1.170=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_12_08_MKR:
    fs/fat/fat.c:1.5=after, CHANGELOG:1.165=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_12_08_MKR:
    fs/fat/fat.c:1.5=after, board/RRvision/RRvision.c:1.2=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_12_08_MKR:
    fs/fat/fat.c:1.5=after, board/atc/atc.c:1.6=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_12_08_MKR:
    fs/fat/fat.c:1.5=after, board/c2mon/c2mon.c:1.2=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_12_08_MKR:
    fs/fat/fat.c:1.5=after, board/etx094/etx094.c:1.2=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_12_08_MKR:
    fs/fat/fat.c:1.5=after, board/evb64260/flash.c:1.5=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_12_08_MKR:
    fs/fat/fat.c:1.5=after, board/rmu/rmu.c:1.3=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_12_08_MKR:
    fs/fat/fat.c:1.5=after, board/tqm8260/tqm8260.c:1.4=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_12_08_MKR:
    fs/fat/fat.c:1.5=after, board/tqm8xx/tqm8xx.c:1.4=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_12_08_MKR:
    fs/fat/fat.c:1.5=after, include/version.h:1.20=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_12_08_MKR:
    fs/fat/fat.c:1.5=after, lib_arm/board.c:1.19=before. Treated as 'before'
WARNING: Invalid PatchSet 815, Tag LABEL_2003_12_08_MKR:
    include/configs/TQM823L.h:1.9=after, CHANGELOG:1.170=before. Treated as 'before'
WARNING: Invalid PatchSet 815, Tag LABEL_2003_12_08_MKR:
    board/tqm8xx/tqm8xx.c:1.5=after, CHANGELOG:1.170=before. Treated as 'before'
WARNING: Invalid PatchSet 815, Tag LABEL_2003_12_08_MKR:
    Makefile:1.65=after, CHANGELOG:1.170=before. Treated as 'before'
WARNING: Invalid PatchSet 816, Tag LABEL_2003_12_08_MKR:
    include/configs/rmu.h:1.4=after, CHANGELOG:1.171=before. Treated as 'before'
WARNING: Invalid PatchSet 815, Tag LABEL_2003_12_08_MKR:
    README:1.60=after, CHANGELOG:1.170=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_11_26_MKR:
    fs/fat/fat.c:1.5=after, CHANGELOG:1.165=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_11_26_MKR:
    fs/fat/fat.c:1.5=after, board/RRvision/RRvision.c:1.2=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_11_26_MKR:
    fs/fat/fat.c:1.5=after, board/atc/atc.c:1.6=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_11_26_MKR:
    fs/fat/fat.c:1.5=after, board/c2mon/c2mon.c:1.2=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_11_26_MKR:
    fs/fat/fat.c:1.5=after, board/etx094/etx094.c:1.2=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_11_26_MKR:
    fs/fat/fat.c:1.5=after, board/evb64260/flash.c:1.5=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_11_26_MKR:
    fs/fat/fat.c:1.5=after, board/rmu/rmu.c:1.3=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_11_26_MKR:
    fs/fat/fat.c:1.5=after, board/tqm8260/tqm8260.c:1.4=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_11_26_MKR:
    fs/fat/fat.c:1.5=after, board/tqm8xx/tqm8xx.c:1.4=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_11_26_MKR:
    fs/fat/fat.c:1.5=after, include/version.h:1.20=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_11_26_MKR:
    fs/fat/fat.c:1.5=after, lib_arm/board.c:1.19=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_11_26_MKR:
    fs/fat/fat.c:1.5=after, CHANGELOG:1.165=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_11_26_MKR:
    fs/fat/fat.c:1.5=after, board/RRvision/RRvision.c:1.2=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_11_26_MKR:
    fs/fat/fat.c:1.5=after, board/atc/atc.c:1.6=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_11_26_MKR:
    fs/fat/fat.c:1.5=after, board/c2mon/c2mon.c:1.2=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_11_26_MKR:
    fs/fat/fat.c:1.5=after, board/etx094/etx094.c:1.2=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_11_26_MKR:
    fs/fat/fat.c:1.5=after, board/evb64260/flash.c:1.5=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_11_26_MKR:
    fs/fat/fat.c:1.5=after, board/rmu/rmu.c:1.3=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_11_26_MKR:
    fs/fat/fat.c:1.5=after, board/tqm8260/tqm8260.c:1.4=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_11_26_MKR:
    fs/fat/fat.c:1.5=after, board/tqm8xx/tqm8xx.c:1.4=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_11_26_MKR:
    fs/fat/fat.c:1.5=after, include/version.h:1.20=before. Treated as 'before'
WARNING: Invalid PatchSet 810, Tag LABEL_2003_11_26_MKR:
    fs/fat/fat.c:1.5=after, lib_arm/board.c:1.19=before. Treated as 'before'
Committing initial tree 2183da96d548c02bd4f99b05e673395f81212878
Cannot create object: Too many open files
-> ulimit -n
1024

Any ideas?


Best regards,

Wolfgang Denk

-- 
Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de
"It is easier to port a shell than a shell script."      - Larry Wall

Re: cvsimport: rewritten in Perl

[Linus Torvalds]

On Wed, 6 Jul 2005, Wolfgang Denk wrote:
> 
> I tested this on the U-Boot CVS repository (available at
> http://cvs.sourceforge.net/cvstarballs/u-boot-cvsroot.tar.bz2).
>
> Committing initial tree 2183da96d548c02bd4f99b05e673395f81212878
> Cannot create object: Too many open files

If you make it print out its <pid> and then pause, you can use 

	ls -l /proc/<pid>/fd/

to get an idea of what the files may be. Looks like the new perl version 
is leaking file descriptors..

Matthias?

		Linus
	

Re: cvsimport: rewritten in Perl

[Sven Verdoolaege]

On Tue, Jul 05, 2005 at 07:41:30PM -0700, Linus Torvalds wrote:
> If you make it print out its <pid> and then pause, you can use 
> 
> 	ls -l /proc/<pid>/fd/
> 
> to get an idea of what the files may be. Looks like the new perl version 
> is leaking file descriptors..
> 
> Matthias?

That was my mistake, actually.
Thanks for spotting this.

skimo
--
git-cvsimport-script: close temporary file.

---
commit 6b6fdaa290f7dfd178a518fcafb9e14e652eb8ac
tree 725e7c6ecc75a0e90a6bc002ce540bd74dca999e
parent f4b3a4c30b5ea3a5de2a2597a3c53266017d02ba
author Sven Verdoolaege <skimo@kotnet.org> Wed, 06 Jul 2005 08:01:47 +0200
committer Sven Verdoolaege <skimo@kotnet.org> Wed, 06 Jul 2005 08:01:47 +0200

 git-cvsimport-script |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/git-cvsimport-script b/git-cvsimport-script
--- a/git-cvsimport-script
+++ b/git-cvsimport-script
@@ -317,6 +317,7 @@ sub file {
 	    $res = $self->_line($fh);
 	    die "No input: $fn $rev\n" unless defined $res;
 	}
+	close ($fh);
 
 	return ($name, $res);
 }

Re: cvsimport: rewritten in Perl

[Matthias Urlichs]

[-- Attachment #1: Type: text/plain, Size: 654 bytes --]

Hi,

Sven Verdoolaege:
> > to get an idea of what the files may be. Looks like the new perl version 
> > is leaking file descriptors..
> > 
> > Matthias?
> 
> That was my mistake, actually.
> Thanks for spotting this.
> 
Ouch. For me, the main danger of lots of Python programming is that
I tend not to see this kind of problem any more, because in Python it
Simply Doesn't Happen. 

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
 - -
Conscience is the inner voice that warns us somebody is looking
					-- H. L. Mencken

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: cvsimport: rewritten in Perl

[Wolfgang Denk]

[-- Attachment #1: Type: text/plain, Size: 663 bytes --]

In message <Pine.LNX.4.58.0507051936350.3570@g5.osdl.org> Linus Torvalds wrote:
> 
> If you make it print out its <pid> and then pause, you can use 
> 
> 	ls -l /proc/<pid>/fd/
> 
> to get an idea of what the files may be. Looks like the new perl version 
> is leaking file descriptors..

It does. In case it's still of interest:  log file attached.

Best regards,

Wolfgang Denk


Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de
It seems intuitively obvious to me, which  means  that  it  might  be
wrong.                                                 -- Chris Torek


[-- Attachment #2: git-cvsimport-script.log.gz --]
[-- Type: application/x-gzip , Size: 1925 bytes --]