Perforce support.

Perforce support.

[Govind Salinas]

Hi,

I am hoping to convince my co-workers to start using a Distributed
SCM, hopefully git, and I wanted to see what people had to say about
the Perforce-git interoperability.  To make it more fun we are doing
this on Windows.

I have been playing around with git for a month or so and have started
writing, what I hope will be, a nice GUI over git that works well on
Windows (Cygwin) and offers some feeling of familiarity to our
Perforce users.  That however is only half the problem.

We need to be able to go back and forth to our main Perforce depot,
and while I understand that git-svn support is very good, I have only
seen limited support of Perforce.  I was wondering if anyone has been
using git with p4 and how well did it work.  We have very complex and
somewhat large "clients" that do a lot of mapping of directories
(which strikes me as particularly insane) and I was wondering if any
of the tools support that.

If anyone has any suggestions/guidance on how to do this I would appreciate it.

Thank You,
Govind Salinas.

Re: Perforce support.

[Alex Riesen]

[-- Attachment #1: Type: text/plain, Size: 2699 bytes --]

On 7/12/07, Govind Salinas <govindsalinas@gmail.com> wrote:
> I am hoping to convince my co-workers to start using a Distributed
> SCM, hopefully git, and I wanted to see what people had to say about
> the Perforce-git interoperability.  To make it more fun we are doing
> this on Windows.

There are two scripts. git-p4import.py, is in standard distribution,
and is AFAIK not used very much Windows. The other, git-p4-import.bat
is never tried on anywhere but Windows and is maintained outside
of mainstream Git.

I maintain the second one, git-p4-import.bat. I don't dare to submit
it for distribution with Git: it is very centered on stu^H^H^H^H suboptimal
practices I have at work.

> I have been playing around with git for a month or so and have started
> writing, what I hope will be, a nice GUI over git that works well on
> Windows (Cygwin) and offers some feeling of familiarity to our
> Perforce users.  That however is only half the problem.

I trust you already seen git gui...

> We need to be able to go back and forth to our main Perforce depot,
> and while I understand that git-svn support is very good, I have only
> seen limited support of Perforce.

Because Perforce is, in some respects, even worse: it is closed.
It will never get the level of support SVN or even CVS have.

>   I was wondering if anyone has been
> using git with p4 and how well did it work.  We have very complex and
> somewhat large "clients" that do a lot of mapping of directories
> (which strikes me as particularly insane) and I was wondering if any
> of the tools support that.

I am in such situation. Perforce is so bad in managing it,
that people here had to write a handful of wrappers around
standard commands just to make it limping (and it is falling
apart right now).

I use the git-p4-import.bat to import and export the commits.
It is PITA though. Try it (attached), but don't expect too much.
It is a perl script, geared towards Windows and Activision Perl
(hopefully not too much and it still can work with cygwin's perl).
Some examples (I assume a useful shell, not cmd):

Import a range of P4 changelists, convert p4 path to local path

  $ git-p4-import.bat --p4-path //Very/stupid/perforce \
  --local-path "" --p4-range '@123,456'
Result can be found in the reference named p4/IMPORT.

Export some commits:

  $ git-p4-import.bat --merge my-branch -y --edit=vim
This prepare a change lists, let you edit the changelist description
in vim and submit it. I suggest you always rebase my-branch
before this: p4 linearizes the history, so in case you loose your
original git branch, the _useful_ parenthood information is lost with it.
Better not even have any hopes associated with this.

[-- Attachment #2: git-p4-import.bat.txt --]
[-- Type: text/plain, Size: 50773 bytes --]

@rem = 'NT: CMD.EXE vim: syntax=perl noet sw=4
@perl -x -s %0 -- %*
@goto __end_of_file__
@rem ';
#!perl -w
#line 7

local $VERBOSE = 0;
local $DRYRUN = 0;
local $SHOW_DIFFS = 0;
local $AUTO_COMMIT = 0;
local $JUST_COMMIT = 0;
local $GIT_DIR = undef;
local $P4CLIENT = undef;
local $EDIT_COMMIT = 0;
local $FORCE = 0;
local @FULL_IMPORT = 0;
local @DESC = ();
local $SPEC = undef;
local $P4HAVE_FILE = undef;
local %P4USERS = ();
local $FULL_DESC = 1;
local $HEAD_FROM_P4 = 0;
local $P4_EDIT_MERGE = 0;
local $P4_EDIT_CHANGED = 0;
local $P4_EDIT_HEAD = undef;
local $editor = undef;

local $P4_PATH = undef;
local $P4_RANGE = undef;
local $P4_LOCAL_PATH = undef;
local $START_POINT = undef;
local $UPDATE_BRANCH = undef;
local $P4_TAG = undef;

local $P4_SYNC_TO = undef;
# cache for refs/p4import references
local %P4IMPORT_HEADS = ();

sub read_args {
    my $no_opt = 0;
    my $no_EDIT_COMMIT = 0;
    while (my $f = shift) {
	goto _files if $no_opt;
	$no_opt=1, next if $f eq '--';

	$DRYRUN=1, next if $f eq '-n' or $f eq '--dry-run';
	$SHOW_DIFFS=$DRYRUN=1, next if $f eq '--diffs';
	$AUTO_COMMIT=1, next if $f eq '-y' or $f eq '--yes';
	$JUST_COMMIT=1, next if $f =~ /^--(just-)?commit$/;
	$EDIT_COMMIT=1, next if $f eq '-e' or $f eq '--edit';
	$no_EDIT_COMMIT=1, next if $f eq '-ne' or $f eq '--no-edit';
	$FORCE=1, next if $f eq '--force';
	if ($f =~ /^-(e|-edit=)(.*)/) {
	    $editor = $2;
	    $EDIT_COMMIT=1;
	    next;
	}
	$FULL_IMPORT=1, next if $f eq '--full';
	$FULL_DESC++, next if $f eq '--p4-desc';

	$VERBOSE++, next if $f eq '-v' or $f eq '--verbose';
	$P4CLIENT = shift, next if $f eq '--client' and @_;
	$P4CLIENT = $1, next if $f =~ /^--client=(.*)/;
	push(@DESC,'c'.shift), next if $f eq '-C' and @_;
	push(@DESC,"c$1"), next if $f =~ /^--changelist=(.*)/;
	push(@DESC,'f'.shift), next if $f eq '-F' and @_;
	push(@DESC,"f$1"), next if $f =~ /^--file=(.*)/;
	push(@DESC,'4'.shift), next if ($f eq '--ptr' or $f eq '--p4') and @_;
	push(@DESC,"4$2"), next if $f =~ /^--(p4|ptr)=(.*)/;

	$P4_EDIT_CHANGED=1, next if $f eq '--p4-edit-changed';
	if ($f =~ /^--p4-edit-changed=(.*)/) {
	    $P4_EDIT_CHANGED=1;
	    $P4_EDIT_HEAD=$1;
	    next 
	}
	if ($f =~ /^--merge=(.*)/ or
	    ($f eq '--merge' and @_)) {
	    $P4_EDIT_MERGE=1;
	    $P4_EDIT_CHANGED=1;
	    $EDIT_COMMIT=1;
	    $P4_EDIT_HEAD= $f eq '--merge' ? shift: $1;
	    next 
	}
	$P4_TAG=$1, next if $f =~ /^--p4-tag=(.+)/;
	$P4_TAG=shift, next if $f eq '--p4-tag' and @_;
	# $P4_EDIT_MERGE = 'submit' if $f eq '--submit';

	$P4_PATH=$1, next if $f =~ /^--p4-path=(.*)/;
	$P4_PATH=shift, next if $f eq '--p4-path' and @_;
	$P4_RANGE=$1, next if $f =~ /^--p4-range=(.*)/;
	$P4_RANGE=shift, next if $f eq '--p4-range' and @_;
	$P4_LOCAL_PATH=$1, next if $f =~ /^--local-path=(.*)/;
	$P4_LOCAL_PATH=shift, next if $f eq '--local-path' and @_;
	$START_POINT=$1, next if $f =~ /^--start=(.*)/;
	$START_POINT=shift, next if $f eq '--start' and @_;
	$UPDATE_BRANCH=$1, next if $f =~ /^--branch=(.*)/;
	$UPDATE_BRANCH=shift, next if $f eq '--branch' and @_;

	$P4_SYNC_TO=$1, next if $f =~ /--sync=(.*)/;
	$P4_SYNC_TO=shift, next if $f eq '--sync' and @_;

	if ($f eq '--help' or $f eq '-h') {
	    print <<EOF;
$0 [-n|--dry-run] [-y|--yes] [--client <client-name>] [--diffs] \
[-e|--edit] [--just-commit] [--full] [-v|--verbose] [-C <change-number>] \
[-F <filename>] [--ptr|--p4 <p4-path-and/or-revision>] [--p4-desc] \
[--] [<specification>]

Perforce client state importer. Creates a git commit on the current
branch from a state the given p4 client and working directory hold.

<specification> must be given and is expected to be a file which will be
stored on the side branch under the name "spec".
Remote-to-local mapping and the revisions of files are stored in "have",
and the client definition - in "client".

--client client Specify client name (saved in .git/p4/client for the next time)
--full          Perform full import, don't even try to figure out what changed
-y|--yes        Commit automatically (by default only index updated)
--just-commit   To be used after you forgot to run with --yes first time
-n|--dry-run    Do not update the index and do not commit
-e|--edit       Edit commit description before committing
-v|--verbose    Be more verbose. Can be given many times, increases verbosity

--file=file
-F file         Take description for the commit from a file in the
                next parameter

--changelist=change
-C change       Take description for the commit from this p4 change

--p4|--ptr=p4-path-and/or-revision
                Take description for the commit from the p4 change described
                by this p4 path, possibly including revision specification

--p4-path=p4-path
                Import changes directly from Perforce concerning the given
                p4-path. The first change list defines the starting state
                of imported path
--p4-range=revrange
                Restrict the changes to the revrange change lists.
                The first change list is always imported fully,
                all the subsequent - incrementally. If not given -
                all changes on the patch will be imported
--local-path=local-path
                Replace the p4-path given to --p4-path with local-path
                in the imported pathnames
--start=git-ref
                Start commiting at the given reference. If not given,
                the value of reference given to --branch is used,
                otherwise the current HEAD is assumed
--branch=git-branch
                After everything is imported and committed, update the
                reference git-branch with sha1 of the last commit.
                If not given - the last commit of the import is stored
                in the reference /p4/IMPORT

--p4-desc       Increase amount of junk from p4 change description

--diffs         Show files which are different between local filesystem, index,
                and the current HEAD. Does not do anything else

--merge=branch
                Merge with the given sha1 and prepare a p4 submission.
                Current HEAD will be git-merged with branch.
                The working directory must have no local changes

--p4-edit-changed
--p4-edit-changed=branch
                Read the given branch in the index and prepare a p4 submission.
                Current HEAD must fast-forward to sha1. If branch is omited,
                the current HEAD is assumed (useful after push on HEAD).
                The working directory must have no local changes

--sync=ref
                Sync the current client to the state given by ref:head
                file. The file must have been stored by previous import.
                The option considers all references under p4import when
                looking for the "have"-file


The descriptions taken from p4 changes given by -C and --p4 will
be concatenated if the options given multiple times.
"--" can be used to separate options from description files.

EOF
	    exit(0);
	}
	die "$0: unknown option $f\n" if $f =~ /^-/;
    _files:
	warn "$0: spec was already set, $SPEC ignored\n" if defined($SPEC);
	$SPEC = $f;
    }
    $EDIT_COMMIT = 0 if $no_EDIT_COMMIT;
}
read_args(@ARGV);

$editor = $ENV{VISUAL} unless defined($editor);
$editor = $ENV{EDITOR} unless defined($editor);
$editor = 'vim' unless defined($editor);
die "$0: no editor defined\n" if $EDIT_COMMIT and !defined($editor);

if (!defined($GIT_DIR)) {
    $GIT_DIR = git_rev_parse('--git-dir');
    die "$0: GIT_DIR not found\n" if !defined($GIT_DIR) or !-d $GIT_DIR;
}

exit(git_show_diffs() ? 1: 0) if $SHOW_DIFFS;

# the most commands below need GIT_DIR/p4, try to create it
mkdir "$GIT_DIR/p4", 0775;

if (defined($P4_PATH)) {
    $START_POINT = $UPDATE_BRANCH
	if !defined($START_POINT) and defined($UPDATE_BRANCH);
    my $parent = undef;
    $parent = git_rev_parse($START_POINT) if defined($START_POINT);
    $parent = '' if !defined($parent);

    $P4_RANGE = '#head' if !defined($P4_RANGE); # import all changes
    $P4_LOCAL_PATH = $P4_PATH if !defined($P4_LOCAL_PATH);
    $P4_LOCAL_PATH =~ s!^/+!!o;
    s!/+\.\.\.$!!o for ($P4_PATH,$P4_LOCAL_PATH);
    s!/+$!!o for ($P4_PATH,$P4_LOCAL_PATH);
    print "Path conversion:\n'$P4_PATH' ->\n'$P4_LOCAL_PATH'\n" if $VERBOSE > 1;
    import_p4_dir($P4_PATH,$P4_LOCAL_PATH,$P4_RANGE,$parent,$UPDATE_BRANCH);
    exit 0;
}

if (defined($P4_TAG)) {
    my $rc = git_p4_tag_HEAD($P4_TAG);
    exit 127 if $rc & 0xff;
    exit 1 if $rc;
    exit 0;
}

# P4 client was given in command-line. Store it
if ( defined($P4CLIENT) ) {
    mkdir "$GIT_DIR/p4", 0777;
    if ( open(F, '>', "$GIT_DIR/p4/client") ) {
	print F "$P4CLIENT\n";
	close(F);
    } else {
	die "$0: cannot store client name: $!\n"
    }
} else {
    if ( open(F, '<', "$GIT_DIR/p4/client") ) {
	($P4CLIENT) = <F>;
	close(F);
	$P4CLIENT =~ s/^\s*//,$P4CLIENT =~ s/\s*$// if defined($P4CLIENT);
    }
}
die "P4 client not defined\n" if !defined($P4CLIENT) or !length($P4CLIENT);
print "reading client $P4CLIENT\n" if $VERBOSE;
local ($P4ROOT, $p4clnt, $P4HOST);
open(my $fdo, '>', "$GIT_DIR/p4/client.def") or die "p4/client.def: $!\n";
binmode($fdo);
open(my $fdi, '-|', "p4 client -o $P4CLIENT") or die "p4 client: $!\n";
binmode($fdi);
my $last_line_len = 0;
while (<$fdi>) {
    next if /^#/o;
    if ( m/^\s*Root:\s*(\S+)[\\\/]*\s*$/so ) { $P4ROOT = $1 }
    elsif ( m/^\s*Client:\s*(\S+)/o ) { $p4clnt = $1 }
    elsif ( m/^\s*Host:\s*(\S+)/o ) { $P4HOST = $1 }
    ($VERBOSE and print), next if /^(Access|Update):/;
    s/\r?\n$//so;
    my $len = length($_);
    print $fdo "$_\n" if $len or $len != $last_line_len;
    $last_line_len = $len;
}
close($fdi);
close($fdo);

die "Client root not defined\n" unless defined($P4ROOT);
if ( $VERBOSE ) {
    use Cwd;
    print "GIT_DIR: $GIT_DIR\n";
    print "Root: $P4ROOT (cwd: ".cwd()."\n";
    print "Host: $P4HOST\n";
    print "Client: $p4clnt\n" if $p4clnt ne $P4CLIENT;
}

die if defined $P4_SYNC_TO;
#exit(git_sync_to($P4_SYNC_TO)) if defined $P4_SYNC_TO;

if ($P4_EDIT_CHANGED) {
    exit(git_p4_merge($P4_EDIT_HEAD, $P4_EDIT_MERGE, $AUTO_COMMIT));
}

my ($git_head,$git_p4_head,$git_p4_have) = git_p4_init();

if ($JUST_COMMIT) {
    git_p4_commit($git_head, $git_p4_head);
    exit 0;
}

local %gitignore_dirs = ();
$gitignore_dirs{'/'} = read_filter_file("$GIT_DIR/info/exclude");
push(@{$gitignore_dirs{'/'}}, @{read_filter_file('.gitignore')});

my %git_index = ();
$/ = "\0";
my @git_X = ();
print "Reading git file list(git ls-files @git_X --cached -z)...\n" if $VERBOSE;
foreach ( qx{git ls-files @git_X --cached -z} ) {
    chop; # chop \0
    next if m/^\.gitignore$/o;
    next if m/\/\.gitignore$/o;
    next if filtered($_);
    $git_index{$_} = 1;
}

my @git_add = ();
my @git_addx = ();
my @git_del = ();
my @git_upd = ();

print "Reading P4 file list...\n" if $VERBOSE;
local ($Conflicts,$Ignored,$Added,$Deleted,$Updated) = (0,0,0,0,0);
$/ = "\n";
my $in_name = 0;
my @root = split(/[\/\\]+/, $P4ROOT);
my %p4_index = ();
my %p4_a_lc = ();
my %lnames = ();
my %lconflicts = ();
if (opendir(DIR, '.')) {
    $lnames{'.'} = [grep {$_ ne '.' and $_ ne '..'} readdir(DIR)];
    closedir(DIR);
}
open(my $have, "p4 -G -c $P4CLIENT -H $P4HOST -d $P4ROOT have |") or
    die "$0: failed to start p4: $!\n";
binmode($have);
$P4HAVE_FILE = "$GIT_DIR/p4/have";
open(my $storedhave, '>', $P4HAVE_FILE) or die "$P4HAVE_FILE: $!\n";
binmode($storedhave);
my ($cnt,$err,$ent) = (0,0,undef);
while (defined($ent=read_pydict_entry($have))) {
    if (defined($ent->{code}) and defined($ent->{data})) {
	++$err if $ent->{code} eq 'error';
	print STDERR 'p4: '.$ent->{code}.': '.$ent->{data}."\n";
	next;
    }
    next if !defined($ent->{depotFile}) or !defined($ent->{clientFile});
    ++$cnt;
    my $a = $ent->{depotFile};
    $ent->{clientFile} =~ m!^//[^/]+/(.*)!o;
    my $b = $1;
    my @bb = split(/\/+/, $b);
    print $storedhave "$a\0$ent->{clientFile}\0$ent->{haveRev}\0\n";

    if ( $^O eq 'MSWin32' ) {
	# stupid windows, daft activestate, dumb P4
	# This piece below is checking for file name conflicts
	# which happen on windows because of it mangling the names.
	my $blc = lc $b;
	if ( $#bb > 0 ) {
	    my $path = '.';
	    foreach my $n (@bb[0 .. $#bb -1]) {
		my @conflicts =
		    grep {lc $_ eq lc $n and $_ ne $n} @{$lnames{$path}};
		if (@conflicts and !exists($lconflicts{"$path/$n"})) {
		    warn "warning: $a -> $b\n".
			 "warning: conflict between path \"$path/$n\" and ".
			 "local filesystem in \"@conflicts\"\n";
		    $Conflicts++;
		    $lconflicts{"$path/$n"} = 1;
		}
		$path .= "/$n";
		if (!exists($lnames{$path})) {
		    if (opendir(DIR, $path)) {
			$lnames{$path} =
			    [grep {$_ ne '.' and $_ ne '..'} readdir(DIR)];
			closedir(DIR);
			#print "read $path (",scalar(@{$lnames{$path}}),")\n";
		    }
		}
	    }
	}
	if (!exists($p4_a_lc{$blc})) {
	    $p4_a_lc{$blc} = [$a, $b];
	} else {
	    warn("warning: $a -> $b\n".
		 "warning: conflicts with ".
		 $p4_a_lc{$blc}->[0]." -> ".
		 $p4_a_lc{$blc}->[1]."\n");
	    $Conflicts++;
	    next;
	}
    }

    my $i;
    for ($i = 0; $i < $#bb; ++$i) {
	my $bdir = join('/',@bb[0 .. $i]) . '/';
	if ( !exists($gitignore_dirs{$bdir}) ) {
	    $gitignore_dirs{$bdir} = read_filter_file("$bdir.gitignore");
	}
    }
    if (filtered($b)) {
	print " i $b\n" if $VERBOSE > 3;
	$Ignored++;
	next
    }
    $p4_index{$b} = $a;
    if ( exists($git_index{$b}) ) {
	my $needup = 1;
	if (defined($git_p4_have)) {
	    $prev = $git_p4_have->{$a};
	    if (defined($prev)) {
		$prev->[0] =~ m!^//[^/]+/(.*)!o;
		$needup = 0 if ($b eq $1) and ($prev->[1] eq $ent->{haveRev});
		if ($needup and $VERBOSE > 1) {
		    my $reason;
		    $reason = 'local file' if $b ne $1;
		    $reason = 'revision' if $prev->[1] ne $ent->{haveRev};
		    print "$a ($reason changed)\n";
		}
	    }
	}
	if ($needup) {
	    $Updated++;
	    push(@git_upd, $b);
	}
    } else {
	$Added++;
	if ( $b =~ m/\.(bat|cmd|pl|sh|exe|dll)$/io )
	{ push(@git_addx, $b) } else { push(@git_add, $b) }
    }
}
close($storedhave);
close($have);
exit 1 if $err; # the error already reported
die "Nothing in the client $P4CLIENT\n" if !$cnt;

undef %p4_a_lc;

@git_del = grep { !exists($p4_index{$_}) } keys %git_index;
$Deleted = $#git_del + 1;

#foreach (keys %git_index)
#{ push(@git_del, $_) if !exists($p4_index{$_}) }

if ( $DRYRUN ) {
    print($#git_add+$#git_addx+ 2," files to add\n") if $VERBOSE;
    print map {" a $_\n"} @git_add if $VERBOSE > 2;
    print map {" a $_\n"} @git_addx if $VERBOSE > 2;
    print($#git_del+1," files to unreg\n") if $VERBOSE;
    print map {" d $_\n"} @git_del if $VERBOSE > 2;
    print($#git_upd+1," files to update\n") if $VERBOSE;
    print map {" u $_\n"} @git_upd if $VERBOSE > 2;
    print "added: $Added, unregd: $Deleted, updated: $Updated, ignored: $Ignored";
    print ", conflicts: $Conflicts" if $Conflicts;
    print "\n";
} else {
    if (@git_add || @git_addx) {
	print($#git_add+$#git_addx+ 2,
	      " files | git update-index --add -z --stdin\n")
	    if $VERBOSE;
	if (@git_add) {
	    open(GIT, '| git update-index --add --chmod=-x -z --stdin') or
		die "$0 git-update-index(add): $!\n";
	    print GIT map {print " a $_\n" if $VERBOSE > 1; "$_\0"} @git_add;
	    close(GIT);
	}
	if (@git_addx) {
	    open(GIT, '| git update-index --add --chmod=+x -z --stdin') or
		die "$0 git-update-index(add): $!\n";
	    print GIT map {print " a $_\n" if $VERBOSE > 1; "$_\0"} @git_addx;
	    close(GIT);
	}
    }

    if (@git_del) {
	print($#git_del+1," files | git update-index --remove -z --stdin\n")
	    if $VERBOSE;
	open(GIT, '| git update-index --force-remove -z --stdin') or
	    die "$0 git-update-index(del): $!\n";
	print GIT map {print " d $_\n" if $VERBOSE > 1; "$_\0"} @git_del;
	close(GIT);
    }

    if (@git_upd) {
	print($#git_upd+1," files | git update-index -z --stdin\n")
	    if $VERBOSE;
	open(GIT, '| git update-index -z --stdin') or
	    die "$0 git-update-index(upd): $!\n";
	print GIT map {print " u $_\n" if $VERBOSE > 1; "$_\0"} @git_upd;
	close(GIT);
    }

    print "added: $Added, unregd: $Deleted, updated: $Updated, ignored: $Ignored";
    print ", conflicts: $Conflicts" if $Conflicts;
    print "\n";
    git_p4_commit($git_head, $git_p4_head) if $AUTO_COMMIT;
}

exit 0;

sub run_or_exit {
    my $rc = system(@_);
    exit(127) if $rc & 0xff;
    exit(1) if $rc;
    return 0;
}

sub filtered {
    my $name = shift;
    study($name);
    my @path = split(/\/+/o, $name);
    my $dir = '';
    $name = '';

    foreach my $d (@path) {
	$name .= $d;
#	print STDERR "$dir: $name $d\n" if $v;
	foreach my $re (@{$gitignore_dirs{'/'}}) {
	    return 1 if $name =~ m/$re/;
	    return 1 if $d =~ m/$re/;
	}
	if ( length($dir) and exists($gitignore_dirs{$dir}) ) {
	    foreach my $re (@{$gitignore_dirs{$dir}}) {
		return 1 if $name =~ m/$re/;
		return 1 if $d =~ m/$re/;
	    }
	}
	$name .= '/';
	$dir = $name;
    }
#    print STDERR "$name not filtered\n" if $v;
    return 0;
}

sub read_filter_file {
    my @filts = ();
    my $file = shift;
    if ( open(my $if, '<', $file) ) {
	print "added ignore file $file\n" if $VERBOSE;
	$/ = "\n";
	while (my $l = <$if>) {
	    next if $l =~ /^\s*#/o;
	    next if $l =~ /^\s*$/o;
	    $l =~ s/[\r\n]+$//so;
	    $l =~ s/\./\\./go;
	    $l =~ s/\*/.*/go;
	    if ( $l =~ m/\// ) {
		$l = "^$l($|/)";
	    } else {
		$l = "(^|/)$l\$";
	    }
	    print " filter $l\n" if $VERBOSE > 1;
	    push(@filts, qr/$l/);
	}
	close($if);
    }
    return \@filts;
}

sub r_pystr
{
    my $fd = shift;
    my ($len,$str)=('','');
    my ($c,$rd,$b) = (4,0,'');
    while ($c > 0) {
	$rd = sysread($fd,$b,$c);
	warn("failed to read len: $!"), return undef if !defined($rd);
	warn("not enough data for len"), return undef if !$rd;
	$len .= $b;
	$c -= $rd;
    }
    $len = unpack('V',$len);
    while ($len > 0) {
	$rd = sysread($fd,$b,$len);
	warn("failed to read data: $!"), return undef if !defined($rd);
	warn("not enough data"), return undef if !$rd;
	$str .= $b;
	$len -= $rd;
    }
    return $str;
}

sub read_pydict_entry
{
    my $f = shift;
    my ($buf,$rd);
    FIL: while (1) {
	# object type identifier
	$rd = sysread($f, $buf, 1);
	last FIL if $rd == 0;
	warn("p4: object type: $!\n"),last if $rd != 1;
	# '{' is a python marshalled dict
	warn("p4: object type: not {\n"),last if $buf ne '{';
	my $ent = {};
	PAIR: while (1) {
	    my ($b,$key);
	    # key type identifier
	    $rd = sysread($f, $b, 1);
	    warn("p4: key type: $!\n"),last FIL if $rd != 1;
	    if ($b eq 's') { # length-prefixed string
		$key = r_pystr($f);
		warn("p4: key: $!\n"),last FIL if !defined($b);
	    } elsif ($b eq '0') { # NULL-element, end of entry
		last PAIR;
	    } else {
		die("p4: key type: not s (string)\n");
		last FIL;
	    }
	    # value type identifier
	    $rd = sysread($f, $b, 1);
	    warn("p4: $key value type: $!\n"),last FIL if $rd != 1;
	    if ($b eq 's') { # length-prefixed string
		$b = r_pystr($f);
		warn("p4: $key value: $!\n"),last FIL if !defined($b);
		$ent->{$key} = $b;
	    } elsif ($b eq 'i') { # 4-byte integer
		$rd = sysread($f, $b, 4);
		warn("p4: $key value data: $!\n"),last FIL if $rd != 4;
		$ent->{$key} = unpack('V',$b);
	    } else {
		warn("p4: $key value type: not s ($b)\n");
		last FIL;
	    }
	}
	return $ent;
    }
    return undef;
}

sub p4user_to_env {
    my $u = shift;
    $ENV{GIT_AUTHOR_NAME}  = '';
    $ENV{GIT_AUTHOR_EMAIL} = '';
    return if !defined($u);
    if (!exists($P4USERS{$u})) {
	my ($mail,$name) = grep {/^(Email|FullName):/} qx{p4 user -o $u};
	if ($? == 0 and defined($mail) and defined($name)) {
	    s/^\S+:	([^\r\n]*)\r?\n$/$1/so for ($mail,$name);
	    if (length($name) and length($mail)) {
		$P4USERS{$u} = {name=>$name, email=>$mail};
	    }
	}
    }
    if ($P4USERS{$u}) {
	$p4u = $P4USERS{$u};
	$ENV{GIT_AUTHOR_NAME}  = $p4u->{name};
	$ENV{GIT_AUTHOR_EMAIL} = $p4u->{email};
    }
    return 1;
}

sub p4_get_change {
    my ($fd,$p4);
    my $cl = shift;
    if (!open($fd, '>', "$GIT_DIR/p4/files")) {
	warn "p4/files: $!\n";
	return;
    }
    print $fd "-o\n$cl\n";
    close($fd);
    if (!open($p4, "p4 -x $GIT_DIR/p4/files change|")) {
	warn "p4: failed to read p4 change $cl: $!\n";
	return;
    }
    my @change = <$p4>;
    close($p4);
    return @change;
}

sub cl2msg {
    my $cl = shift;
    my($o1,$o2,$i);
    if(!open($o1, '>>', "$GIT_DIR/p4/msg")) {
	warn "p4/msg: $!\n";
	return;
    }
    binmode($o1);
    if(!open($o2, '>>', "$GIT_DIR/p4/p4msg")) {
	warn "p4/p4msg: $!\n";
	close($o1);
	return
    }
    binmode($o2);
    if(!open($i, '-|', "p4 describe -s $cl")){
	warn "p4 describe: $!\n";
	close($o1);
	close($o2);
	return
    }
    binmode($i);
    print $o1 "$cl: " if $FULL_DESC;
    print $o2 "$cl: ";
    my @a;
    my $u = undef;
    while (my $l = <$i>) {
	if ($l =~ /^Change \d+ by (\S+)@[^ ]* on ([^\r\n]*)/so) {
	    $u = $1;
	    $ENV{GIT_AUTHOR_DATE} = $2 if length($2);
	}
	last if $FULL_DESC < 2 and $l =~ /^\s*Affected files \.{3}\s*$/so;
	$l =~ s/\r?\n$//so;
	push @a, $l;
    }
    close($i);
    print $o2 substr($a[2],1),"\n"; # p4 side-branch commit description
    close($o2);
    # import branch commit description
    if ($FULL_DESC > 1) {
	# desc level 2+: keep the Change line
	print $o1 map {"$_\n"} (substr($a[2],1),"\n",@a);
    } else {
	# levels 0 and 1: remove the Change line
	print $o1 map { (length($_) ? substr($_,1):'')."\n" } @a[2..$#a];
    }
    close($o1);
    p4user_to_env($u);
}

# looks for p4import/ commit which points to the given reference
# returns undef if not found
sub git_find_p4info {
    my $branch = shift;
    if (!%P4IMPORT_HEADS) {
	foreach my $l (qx{git show-ref}) {
	    $P4IMPORT_HEADS{$2} = $1
		if $l =~ m!^([0-9a-f]{40}) refs/(p4import/[^\r\n]+)\r?\n!so;
	}
    }
    my ($commit,$parent,$p4commit,$p4parent);
    my $r = git_rev_parse($branch);
    return undef if !defined($r);
    while (my ($k,$p4head) = each %P4IMPORT_HEADS) {
	my $commit;
	do {
	    print "trying $k:$p4head\n" if $VERBOSE >3;
	    ($commit,$p4head) =
		grep { s/^parent ([0-9a-f]{40}).*/$1/s }
		qx{git cat-file commit $p4head};
	    $commit = $p4head = '' if $?;
	    return $p4head if $commit eq $r;
	} while (defined($p4head) && length($p4head));
    }
    warn "$branch is not imported from Perforce\n";
    return undef;
}

sub git_get_p4have {
    my $p4head = shift;
    my $p4have = undef;
    if (defined($p4head) and length($p4head) and
	open(my $f, '-|', "git cat-file blob $p4head:have")) {
	my $old = $/;
	$/ = "\0";
	my $cnt = 0;
	while(1) {
	    my $p4name = <$f>;
	    last if !defined($p4name);
	    $p4name =~ s/^.//so if $cnt; # remove \n
	    my $name = <$f>;
	    my $rev = <$f>;
	    last if !defined($name) or !defined($rev);
	    chop($p4name,$name,$rev);
	    ++$cnt;
	    if (defined($p4have)) {
		$p4have->{$p4name} = [$name,$rev];
	    } else {
		$p4have = {$p4name=>[$name,$rev]};
	    }
	}
	$/ = $old;
	close($f);
	print "loaded $cnt revisions from $p4head\n" if $VERBOSE;
    }
    return $p4have;
}

sub p4_get_have {
    print "reading state of $P4CLIENT\n" if $VERBOSE;
    my ($p4have, $fdi);
    open($fdi, "p4 -G -c $P4CLIENT -H $P4HOST -d $P4ROOT have|") or
	die "p4 have: $!\n";
    binmode($fdi);
    my ($err,$ent) = (0,undef);
    while (defined($ent=read_pydict_entry($fdi))) {
	if (defined($ent->{code}) and defined($ent->{data})) {
	    ++$err if $ent->{code} eq 'error';
	    print STDERR "p4: $ent->{code}: $ent->{data}\n";
	    next;
	}
	next if !defined($ent->{depotFile});
	next if !defined($ent->{clientFile});
	if (defined($p4have)) {
	    $p4have->{$ent->{depotFile}}=[$ent->{clientFile},$ent->{haveRev}];
	} else {
	    $p4have={$$ent->{depotFile}=>[$ent->{clientFile},$ent->{haveRev}]};
	}
    }
    close($fdi);
    return $p4have;
}

sub git_p4_init {
    my ($commit,$parent,$p4commit,$p4parent);
    my $HEAD = git_rev_parse('HEAD');
    $HEAD = '' if !defined($HEAD);
    my $p4head = git_rev_parse("refs/p4import/$P4CLIENT");
    $p4head = '' if !defined($p4head);
    die "No HEAD commit! Refusing to import.\n" if !length($HEAD);
    if (length($p4head)) {
	($commit,$p4parent) =
	    grep { s/^parent (.{40}).*/$1/s }
	    qx{git cat-file commit $p4head};
	$commit = $p4parent = '' if $?;
	$p4parent = '' if !defined($p4parent);
    } else {
	$commit = $p4parent = '';
    }
    while (($commit ne $HEAD) and length($p4parent)) {
	$p4head = $p4parent;
	($commit,$p4parent) =
	    grep { s/^parent (.{40}).*/$1/so }
	    qx{git cat-file commit $p4head};
	$commit = $p4parent = '' if $?;
	$p4parent = '' if !defined($p4parent);
	if ($VERBOSE and ($HEAD eq $commit)) {
	    print "found p4 import commit ";
	    system('git','name-rev',$p4head);
	}
    }
    if ($HEAD ne $commit) {
	$HEAD_FROM_P4 = 0;
	warn "Current HEAD is not from $P4CLIENT, doing full import\n";
    } else {
	$HEAD_FROM_P4 = 1;
    }
    my $p4have = undef;
    if (!$FULL_IMPORT and ($HEAD eq $commit) and length($p4head)) {
	if (open(my $f, '-|', "git cat-file blob $p4head:have")) {
	    my $old = $/;
	    $/ = "\0";
	    my $cnt = 0;
	    while(1) {
		my $p4name = <$f>;
		last if !defined($p4name);
		$p4name =~ s/^.//so if $cnt; # remove \n
		my $name = <$f>;
		my $rev = <$f>;
		last if !defined($name) or !defined($rev);
		chop($p4name,$name,$rev);
		++$cnt;
		if (defined($p4have)) {
		    $p4have->{$p4name} = [$name,$rev];
		} else {
		    $p4have = {$p4name=>[$name,$rev]};
		}
	    }
	    $/ = $old;
	    close($f);
	    print "loaded $cnt revisions from $p4head\n" if $VERBOSE;
	}
    }
    return ($HEAD, $p4head, $p4have);
}

sub get_one_line {
    my ($line) = qx{@_};
    return undef if $?;
    $line = '' if !defined($line);
    $line =~ s/\r?\n//gs;
    return $line;
}

sub git_rev_parse {
    return get_one_line('git', 'rev-parse', shift);
}

sub git_write_tree {
    my $sha1 = get_one_line('git','write-tree');
    return undef if !defined($sha1) or !length($sha1);
    return $sha1;
}

sub git_commit_tree {
    my $sha1 = get_one_line('git','commit-tree',@_);
    return undef if !defined($sha1) or !length($sha1);
    return $sha1;
}

sub git_hash_stdin {
    my $sha1 = get_one_line('git','hash-object','-t','blob','-w','--stdin');
    return undef if !defined($sha1) or !length($sha1);
    return $sha1;
}

sub git_hash_file {
    open(STDIN, '<', $_[0]) or die "$0: git_hash_file $_[0]: $!\n";
    return git_hash_stdin();
}

sub git_update_ref_directly {
    return system('git','update-ref',@_);
}
sub git_update_ref {
    my ($msg,$refname,$refval) = @_;
    if ($refname =~ m!^(ORIG_|FETCH_|MERGE_)?HEAD$!o) {}
    elsif ($refname =~ s!^/+!!o) {}
    elsif ($refname =~ m!^refs/!o) {}
    elsif ($refname =~ m!^(heads|remotes|tags|p4import)/!o) {
	$refname = "refs/$refname"
    } else { $refname = "refs/heads/$refname" }
    print STDERR "Updating $refname with $refval\n" if $VERBOSE > 1;
    return git_update_ref_directly('-m',$msg,$refname,$refval);
}

sub git_p4_commit {
    my ($HEAD, $p4head) = @_;
    my ($commit,$parent,$p4commit,$p4parent);

    my ($fdo,$fdi,$rc);
    $rc = system('git','diff-index','--exit-code','--quiet','--cached','HEAD');
    if ($rc == 0) {
	warn("No changes\n");
	return;
    }

    return if $DRYRUN;

    if (!@DESC && !$EDIT_COMMIT) {
	warn "$0: no commit description given\n";
	return;
    }

    my $p4x = "$GIT_DIR/p4/idx.tmp";
    unlink($p4x);

    $ENV{PAGER} = 'cat';

    if (!defined($SPEC) or !open(STDIN, '<', $SPEC)) {
	if ( $^O eq 'MSWin32' ) {
	    open(STDIN, '<', 'NUL') or die "$SPEC: $!\n";
	} else {
	    open(STDIN, '<', '/dev/null') or die "$SPEC: $!\n";
	}
    }
    my $p4spec = git_hash_stdin();
    die "Failed to store $SPEC in git repo\n" if !defined($p4spec);

    my $p4clnt = git_hash_file("$GIT_DIR/p4/client.def");
    die "Failed to save mappings of $P4CLIENT in git repo" if !defined($p4clnt);

    if (!defined($P4HAVE_FILE)) {
	print "reading state of $P4CLIENT\n" if $VERBOSE;
	$P4HAVE_FILE = "$GIT_DIR/p4/have";
	open($fdo, '>', $P4HAVE_FILE) or die "p4/have: $!\n";
	binmode($fdo);
	open($fdi, "p4 -G -c $P4CLIENT -H $P4HOST -d $P4ROOT have|") or
	    die "p4 have: $!\n";
	binmode($fdi);
	my ($cnt,$err,$ent) = (0,0,undef);
	while (defined($ent=read_pydict_entry($fdi))) {
	    if (defined($ent->{code}) and defined($ent->{data})) {
		++$err if $ent->{code} eq 'error';
		print STDERR 'p4: '.$ent->{code}.': '.$ent->{data}."\n";
		next;
	    }
	    next if !defined($ent->{depotFile});
	    next if !defined($ent->{clientFile});
	    ++$cnt;
	    print $fdo "$ent->{depotFile}\0",
		       "$ent->{clientFile}\0",
		       "$ent->{haveRev}\0\n";
	}
	close($fdi);
	close($fdo);
	exit 1 if $err; # the error already reported
	die "The client $P4CLIENT has nothing\n" if !$cnt;
    }

    my $p4have = git_hash_file($P4HAVE_FILE);
    die "Failed to save state of $P4CLIENT in git repo" if !defined($p4have);

    #
    # Prepare commit messages
    #
    unlink("$GIT_DIR/p4/msg", "$GIT_DIR/p4/p4msg");
    open($fdo, '>', "$GIT_DIR/p4/msg"); close($fdo);
    open($fdo, '>', "$GIT_DIR/p4/p4msg"); close($fdo);

    foreach my $i (@DESC) {
	$i =~ s/^(.)//o;
	if ('c' eq $1) {
	    print "reading changes for $i\n" if $VERBOSE;
	    cl2msg($i);
	} elsif ('f' eq $1) {
	    my($o1,$o2,$i);
	    if (open($o1, '>>', "$GIT_DIR/p4/msg")) {
		if (open($o2, '>>', "$GIT_DIR/p4/p4msg")) {
		    if (open($i, '<', $i)) {
			my $n = 0;
			while(<$i>) {
			    $n++;
			    print $o1 $_;
			    print $o2 $_ if $n == 1;
			}
			close($i);
		    }
		    close($o2);
		}
		close($o1);
	    }
	} elsif ('4' eq $1) {
	    print "reading changes for $i\n" if $VERBOSE;
	    my $change = get_one_line('p4', 'changes', '-m1', $i);
	    if (!defined($change) or $change !~ m/\s+(\d+)\s/) {
		die "$i does not resolve into a change number\n";
	    }
	    cl2msg($1);
	}
    }
    system("$editor $GIT_DIR/p4/msg") if $EDIT_COMMIT;

    # copy mirror-branch commit message into side-branch
    # commit message if no other description were given.
    if (!-s "$GIT_DIR/p4/p4msg") {
	open($fdi, '<', "$GIT_DIR/p4/msg") or die "$GIT_DIR/p4/msg: $!\n";
	sysread($fdi,$buf,-s "$GIT_DIR/p4/msg");
	close($fdi);
	open($fdo, '>>', "$GIT_DIR/p4/p4msg") or die "$GIT_DIR/p4/p4msg: $!\n";
	syswrite($fdo,$buf);
	close($fdo);
    }

    #
    # Store the imported file data
    #

    if ($VERBOSE < 2) {
	if ( $^O eq 'MSWin32' ) { open(STDERR, "NUL") }
	else { open(STDERR, "/dev/null") }
    }

    my $remove_merge_heads = 0;
    my $tree = git_write_tree();
    die "failed to write current tree\n" if !defined($tree);
    open(STDIN, '<', "$GIT_DIR/p4/msg") or die "p4/msg: $!\n";
    if (length($HEAD)) {
	my @mergeparents;
	if (open($fd, '<', "$GIT_DIR/MERGE_HEAD")) {
	    while(<$fd>) {
		s/\r?\n$//gs;
		push(@mergeparents, '-p', $_) if /^[0-9a-f]{40}$/;
	    }
	    close($fd);
	}
	$commit = git_commit_tree($tree, '-p', $HEAD, @mergeparents);
	die "failed to commit the merged tree\n" if !defined($commit);
	$remove_merge_heads = 1 if @mergeparents;
    } else {
	$commit = git_commit_tree($tree);
	die "failed to commit current tree\n" if !defined($commit);
    }
    print "current tree stored in commit $commit\n" if $VERBOSE;

    #
    # Storing import control data
    #
    $ENV{GIT_INDEX_FILE} = $p4x;
    open($fdo, '|-', 'git update-index --add --index-info') or
	die "could not start git update-index\n";
    binmode($fdo);
    print $fdo "100644 $p4spec\tspec\n";
    print $fdo "100644 $p4clnt\tclient\n";
    print $fdo "100644 $p4have\thave\n";
    close($fdo);
    if($?) {
	die "Failed to store $SPEC in p4import index and git repo\n".
	    "Failed to save mappings of $P4CLIENT in p4import index and git repo\n".
	    "Failed to save state of $P4CLIENT in p4import index and git repo\n"
    }
    my $p4tree = git_write_tree();
    die "Failed to store $SPEC (tree) in git repo\n" if $?;

    # Bind import control data to the file data
    open(STDIN, '<', "$GIT_DIR/p4/p4msg") or die "p4/p4msg: $!\n";
    $p4commit = length($p4head) ?
	git_commit_tree($p4tree, '-p', $commit, '-p', $p4head):
	git_commit_tree($p4tree, '-p', $commit);
    die "Failed to store $SPEC (commit) in git repo\n" if $?;

    # Finishing touches: update references
    if (!$DRYRUN) {
	git_update_ref('backup ref of current branch','/p4/backup-HEAD','HEAD');
	git_update_ref('backup ref of p4import',
		       '/p4/backup-p4import',"refs/p4import/$P4CLIENT");
	$rc = git_update_ref('data of p4import','HEAD',$commit);
	die "Failed to update HEAD\n" if $rc;
	unlink("$GIT_DIR/MERGE_HEAD") if $remove_merge_heads;
	$rc = git_update_ref('p4import',"p4import/$P4CLIENT",$p4commit);
	die "Failed to store $SPEC (reference) in git repo\n" if $rc;
    }

    $ENV{GIT_PAGER} = 'cat';
    if ($VERBOSE) {
	print STDOUT (grep {s/\r?\n//gs;s/.*?\s//} qx{git name-rev refs/p4import/$P4CLIENT}), ":\n";
	system('git','log','--max-count=1','--pretty=format:%h %s%n',$p4commit);
    }
    print STDOUT (grep {s/\r?\n//gs;s/.*?\s//} qx{git name-rev HEAD}),":\n";
    system('git','log','--max-count=1','--pretty=format:%h %s%n',$commit);
}

sub import_p4_dir {
    my ($p4path, $local_path, $revrange, $parent, $branch) = @_;
    my ($fd, $ent, $error, $rc) = (undef,undef,0,0);

    print "Running changes\n" if $VERBOSE;

    if (open($fd, '>', "$GIT_DIR/p4/files")) {
	print "looking for changes $p4path/...$revrange\n" if $VERBOSE > 1;
	print $fd "$p4path/...$revrange\n";
	close($fd);
    } else {
	die "$0: p4/files: $!\n"
    }
    open(STDIN, '<', "$GIT_DIR/p4/files") or die "$0: p4/files: $!\n";
    die "$0: changes $p4path: $!\n" if !open($fd, '-|', 'p4 -G -x - changes');
    my %CHANGES = ();
    while (defined($ent = read_pydict_entry($fd))) {
	next if $error;
	if ($ent->{code} eq 'error') {
	    warn "$0: p4: $ent->{data}\n";
	    $error = 1;
	    next;
	}
	print "change $ent->{change}\n" if $VERBOSE > 1;
	$CHANGES{$ent->{change}} = {
	    change =>$ent->{change},
	    desc   =>'', # have to read the desc anyway with describe
	    mtime  =>$ent->{'time'},
	    user   =>$ent->{user},
	    files  =>[],
	};
    }
    close($fd);
    warn("$0: nothing found\n"), exit(0) if !%CHANGES;
    exit 1 if $error;

    print "Running describe for ".scalar(keys %CHANGES)." change lists\n"
	if $VERBOSE;

    if (open($fd, '>', "$GIT_DIR/p4/files")) {
	print $fd map { "$_\n"} sort {$a <=> $b} keys %CHANGES;
	close($fd);
    } else {
	die "$0: p4/files: $!\n"
    }
    open(STDIN, '<', "$GIT_DIR/p4/files") or die "$0: p4/files: $!\n";
    die "$0: describe: $!\n" if !open($fd, '-|', 'p4 -G -x - describe');
    while (defined($ent = read_pydict_entry($fd))) {
	next if $error;
	if ($ent->{code} eq 'error') {
	    warn "$0: p4: $ent->{data}\n";
	    $error = 1;
	    next;
	}
	$CHANGES{$ent->{change}}->{desc} = $ent->{desc};
	$CHANGES{$ent->{change}}->{files} = {};
	for (my $i=0;; ++$i) {
	    my $fn = $ent->{"depotFile$i"};
	    last if !defined($fn);
	    next if $fn !~ m!^$p4path(/|$)!;
	    $CHANGES{$ent->{change}}->{files}->{$fn} = {
		action => $ent->{"action$i"},
	    };
	    # $ent->{"type$i"} : text, binary
	}
    }
    close($fd);
    exit 1 if $error;

    print "Reading file data and creating git history\n" if $VERBOSE;

    # Prepare clean index under the local path
    $ENV{GIT_INDEX_FILE} = "$GIT_DIR/p4/idx.tmp";
    unlink($ENV{GIT_INDEX_FILE});
    if (length($parent)) {
	$rc = system('git', 'read-tree', '-i', '--reset', $parent);
	die "git read-tree $parent\n" if $rc;
	$rc = system('git','update-index','--force-remove','--',$local_path);
	die "git update-index $local_path\n" if $rc;
    }
    # Read file data for each change list
    my $first = undef;
    my $ch;
    foreach my $k (sort {$a <=> $b} keys %CHANGES) {
	$ch = $CHANGES{$k};
	print "$k\n" if $VERBOSE > 1;
	# Read the full tree for the first change number
	if (open($fd, '>', "$GIT_DIR/p4/files")) {
	    if (!defined($first)) {
		print $fd "$p4path/...\@$k\n";
		$first = $k;
	    } else {
		foreach my $f (keys(%{$ch->{files}})) {
		    print $fd "$f\@$k\n";
		}
	    }
	    close($fd);
	} else {
	    die "$0: p4/files: $!\n"
	}
	open(STDIN, '<', "$GIT_DIR/p4/files") or die "$0: p4/files: $!\n";

	print "Reading file data for $k\n" if $VERBOSE > 1;

	die "$0: print: $!\n" if !open($fd, '-|', 'p4 -G -x - print');
	my $tmpfile = "$GIT_DIR/p4/cp.tmp";
	while (defined($ent = read_pydict_entry($fd))) {
	    next if $error;
	    if ($ent->{code} eq 'error') {
		warn "$0: p4: $ent->{data}\n";
		$error = 1;
		next;
	    }
	    if ($ent->{code} eq 'binary') {
		if (!defined($f)) {
		    warn "$0: file data without stat info\n";
		    $error = 1;
		    next;
		}
		$f->{size} += length($ent->{data});
		if (length($ent->{data})) {
		    if (!defined(syswrite($f->{tmpfd}, $ent->{data}))) {
			warn "$f->{depotFile}: $tmpfile: $!\n";
			close($f->{tmpfd});
			$f = undef;
			$error = 1;
			next;
		    }
		} else {
		    # FILE FINISHED IF AN EMPTY BINARY PACKET RECEIVED
		    close($f->{tmpfd});
		    my $fn = $f->{depotFile};
		    # put file data into git repo
		    my $tmpsha1 = git_hash_file($tmpfile);
		    die "Failed to save $fn in git repo\n" if !defined($tmpsha1);
		    unlink($tmpfile);
		    $f->{sha1} = $tmpsha1;

		    print "$k\t$tmpsha1\t$f->{size}\t$fn\n" if $VERBOSE > 2;
		    $ch->{files}->{$fn} = $f;

		    delete $f->{depotFile}; # cleanup
		    delete $f->{tmpfd}; # cleanup
		    $f = undef;
		}
		next;
	    }
	    if ($ent->{code} eq 'stat') {
		die "$0: file $f->{depotFile} truncated\n" if defined($f);
		die "$ent->{depotFile}: the leading path not expected\n"
		    if substr($ent->{depotFile},0,length($p4path)) ne $p4path;
		die "$ent->{depotFile}\@$ent->{change}: the change not expected\n"
		    if $ent->{change} > $k;

		my $prev = $ch->{files}->{$ent->{depotFile}};
		$f = {
		    action => defined($prev->{action}) ? $prev->{action}:'add',
		    change => $ent->{change},
		    depotFile => $ent->{depotFile},
		    mtime => $ent->{'time'},
		    size => 0,
		};
		my $ft;
		open($ft,'>',$tmpfile) or die "$tmpfile: $!\n";
		$f->{tmpfd} = $ft;
	    }
	}
	close($fd);
	exit 1 if $error;
	my $modcnt = 0;
	my @delfiles = ();
	my $fdo = undef;
	open($fdo, '|-', 'git update-index -z --replace --add --index-info') or
	    die "could not start git update-index\n";
	binmode($fdo);
	while (my ($fn,$f) = each %{$ch->{files}}) {
	    if (substr($fn,0,length($p4path)) ne $p4path) {
		die "$fn: the leading path not expected\n";
	    }
	    my $locfile = $fn;
	    substr($locfile,0,length($p4path)) = $local_path;
	    $locfile =~ s!^/+!!o;
	    push(@delfiles, $locfile), next if $f->{action} eq 'delete';
	    print "\t$f->{sha1}\t$locfile\n" if $VERBOSE > 1;
	    my $mode = '100644';
	    $mode = '100755' if $locfile =~ /\.(exe|bat|cmd|pl|dll|so)$/io;
	    print $fdo "$mode $f->{sha1}\t$locfile\0";
	    ++$modcnt;
	}
	close($fdo);
	die "failed to build the trees in $local_path for $k\n" if $?;
	if (@delfiles) {
	    open($fdo, '|-', 'git update-index -z --force-remove --stdin') or
		die "could not start git update-index to remove files\n";
	    binmode($fdo);
	    print $fdo map { "$_\0" } @delfiles;
	    close($fdo);
	    print map { "\tdelete $_\n" } @delfiles if $VERBOSE > 1;
	    die "failed to clean the trees in $local_path for $k\n" if $?;
	    $modcnt += scalar @delfiles;
	}
	my $tree = git_write_tree();
	die "Failed to store tree of $k in git repo\n" if !defined($tree);
	print "\ttree of $k written as $tree\n" if $VERBOSE > 1;
	# prepare commit
	open($fdo,'>',"$GIT_DIR/p4/msg") or die "p4/msg: $!\n";
	print $fdo "$k: $ch->{desc}";
	close($fdo);
	p4user_to_env($ch->{user});
	$ENV{GIT_AUTHOR_DATE} = $ch->{mtime};
	open(STDIN, '<', "$GIT_DIR/p4/msg") or die "p4/msg (commit): $!\n";
	my $commit = length($parent) ?
	    git_commit_tree($tree, '-p', $parent): git_commit_tree($tree);
	die "Failed to commit $k in git repo\n" if !defined($commit);
	print "$k committed as $commit";
	print " ($modcnt modification".($modcnt==1?'':'s').')' if $VERBOSE;
	print "\n";
	$parent = $commit;
    }
    if (!$DRYRUN and length($parent)) {
	git_update_ref("backup ref of $branch", "/p4/backup-$branch",$branch)
	    if defined($branch);
	$branch = '/p4/IMPORT' if !defined($branch);
	my $rc = git_update_ref("p4 import $branch", $branch, $parent);
	die "failed to update $branch with $parent\n" if $rc;
	$branch =~ s!^/+!!;
	print "Branch '$branch' updated with state from $ch->{change}\n";
	$ENV{GIT_PAGER} = 'cat';
	system('git','log','--max-count=1','--pretty=format:%h %s%n',$branch);
    }
}

sub git_show_diffs {
    my $sep = $/;
    $/="\0";
    my ($show, $cnt) = (0, 0);
    if (open(F, '-|', 'git diff-files -r --name-only -z')) {
	while (<F>) {
	    my $c = chop;
	    $_ .= $c if $c ne "\0";
	    print "Changed files:\n" if !$show;
	    print " $_\n";
	    $show = 1;
	    $cnt++;
	}
	close(F);
    }
    if (open(F, '-|', 'git diff-index --cached -r -z HEAD')) {
	$show = 0;
	my ($diff, $info) = (0, 1);
	while (<F>) {
	    my $c = chop;
	    $_ .= $c if $c ne "\0";
	    if ($info) {
		next if !/^:(\d{6}) (\d{6}) ([0-9a-f]{40}) ([0-9a-f]{40}) ./o;
		# show only content changes, p4 does not support exec-bit anyway
		$diff = $3 ne $4;
	    } elsif ($diff) {
		print "Changes between index and HEAD:\n" if !$show;
		print " $_\n";
		$show = 1;
		$cnt++;
	    }
	    $info = !$info;
	}
	close(F);
    }
    $/ = $sep;
    return $cnt;
}

sub git_sync_to {
    my $rc;
    my $branch = shift;
    if (!$FORCE) {
	$rc = system('git', 'diff', '--quiet');
	warn("There are changes in the working directory. Refusing to sync\n")
	    if $rc;
	$rc = system('git', 'diff', '--quiet', '--cached');
	warn("There are changes in the index. Refusing to sync\n") if $rc;
    }
    my $p4head = git_find_p4info($branch);
    print "found p4 import commit $p4head for $branch\n"
	if ($VERBOSE > 1 or $DRYRUN) and defined $p4head;
    return 0 if $DRYRUN and defined $p4head;
    my $p4have = git_get_p4have($p4head);
    die "$p4head has no p4 data\n" if !defined($p4have);
    my ($fd,$fdo);
    die "reading client $P4CLIENT: $!\n"
	if !open($fd, '-|', "p4 -G client -o $P4CLIENT");
    binmode($fd);

    my %cdata = (Client=>'', Description=>'', LineEnd=>'', Options=>'');

    while (defined($ent = read_pydict_entry($fd))) {
	next if $error;
	if ($ent->{code} eq 'error') {
	    warn "p4 client $P4CLIENT: $ent->{data}\n";
	    $error = 1;
	    next;
	}
	foreach my $k (keys %cdata) {
	    $cdata{$k} = $ent->{$k} if defined $ent->{$k};
	}
    }
    close($fd);
    die "client.def: $!\n" if !open($fdo, '>', "$GIT_DIR/p4/client.def");
    binmode($fdo);
    print $fdo "Client: $P4CLIENT\n\n";
    print $fdo "Description:\n";
    $cdata{Description} =~ s/\n/\n\t/gs;
    $cdata{Description} =~ s/\t$//s;
    print $fdo "\t$cdata{Description}\n";
    my $wd = cwd();
    $wd =~ s/\\/\//go;
    print $fdo "Root:\t$wd\n\n";
    print $fdo "LineEnd:\t$cdata{LineEnd}\n\n";
    print $fdo "Options:\t$cdata{Options}\n\n";
    print $fdo "View:\n";
    print $fdo
	map {
	    my $fn = $p4have->{$_}->[0];
	    $fn =~ s!^//[^/]*/!//$P4CLIENT/!o;
	    s/([ \t"'@*#%])/sprintf("%%%02x",ord($1))/geo for ($_,$fn);
	    "\t$_ $fn\n"
	} sort { $a cmp $b }
	keys %{$p4have};
    close($fdo);
    print "loading client $P4CLIENT\n";
    open(STDIN, '<', "$GIT_DIR/p4/client.def") or die "client.def: $!\n";
    run_or_exit('p4', 'client', '-i');
    die "files: $!\n" if !open($fdo, '>', "$GIT_DIR/p4/files");
    print $fdo
	map {
	    my $fn = $_;
	    s/([ \t"'@*#%])/sprintf("%%%02x",ord($1))/geo;
	    "$_#$p4have->{$fn}->[1]\n"
	} sort { $a cmp $b }
	keys %{$p4have};
    close($fdo);
    print "syncing client $P4CLIENT\n";
    open(STDIN, '<', "$GIT_DIR/p4/files") or die "p4/files: $!\n";
    run_or_exit('p4', '-c', $P4CLIENT, '-H', $P4HOST, '-d', $P4ROOT,'-x', '-', 'sync');

    return defined($p4head) ? 0: 1;
}

sub git_p4_tag_HEAD {
    my $cl = shift;
    my @change = p4_get_change($cl);
    return 0x100 if !@change;
    my $d=0;
    my ($first_line) = grep {
	if ($d==1) {
	    if (!/^\s*[\r\n]*$/so) { $d=0; 1 }
	    else { 0 }
	} else {
	    $d=1 if /^Description:/o; 0
	}
    } @change;
    my $fd;
    if (!open($fd, '>', "$GIT_DIR/p4/desc")) {
	warn "p4/desc: $!\n";
	return 0x100;
    }
    unshift(@change, '');
    $first_line =~ s/^\s+//;
    unshift(@change, $first_line);
    print $fd map {s/\r?\n$//so;"$_\n"} @change;
    close($fd);
    my $rc = system('git', 'tag', '-f', '-F', "$GIT_DIR/p4/desc", "p4/$cl");
    print "HEAD tagged p4/$cl\n" if !$rc;
    return $rc;
}

sub git_p4_merge {
    my ($source_head,$do_merge,$auto_submit) = @_;
    my $rc = system('git', 'diff-files', '--quiet');
    exit(127) if $rc & 0xff; # error starting the program
    die "There are changes in $P4ROOT. Clean them up first.\n" if $rc;
    $source_head = 'HEAD' if !defined($source_head);
    my $mergehead = git_rev_parse($source_head);
    exit(1) if !defined($mergehead) or !length($mergehead);
    my $fast_forward = 1;
    # Check if the given reference is a direct descendant of current branch
    if ($source_head ne 'HEAD') {
	my $sha1 = get_one_line('git', 'rev-list', '--max-count=1', "$mergehead..HEAD");
	exit 1 if !defined($sha1);
	if (length($sha1) and defined($sha1) and $sha1 =~ /^[0-9a-f]{40}\b/) {
	    my $msg = "HEAD does not fast-forward to $source_head\n";
	    $do_merge ? warn("Warning: $msg"): die("Fatal: $msg");
	    $fast_forward = 0;
	}
    }
    print "Checking out $source_head ($mergehead) for p4 edit\n" if $VERBOSE;
    my $cnt;
    my @files = ();
    my $sep = $/;
    $/="\0";
    if (open(F, '-|', "git diff-index -R --cached -r -z $mergehead")) {
	my ($diff, $info, $M) = (0, 1, '');
	while (<F>) {
	    my $c = chop;
	    $_ .= $c if $c ne "\0";
	    if ($info) {
		next if !/^:\d{6} \d{6} ([0-9a-f]{40}) ([0-9a-f]{40}) (\w+)/o;
		# use only content changes, p4 does not support exec-bit
		$diff = $1 ne $2;
		$M = $3; # change type marker
	    } elsif ($diff) {
		print "$M $_\n" if $VERBOSE;
		die "File contains characters which p4 cannot support\n"
		    if /[\n@#%*]/s;
		push @files, "$M$_";
		$cnt++;
	    }
	    $info = !$info;
	}
	close(F);
    }
    $/ = $sep;
    if (!$cnt) {
	warn "$0: No content changes found between HEAD and $source_head";
	return 0;
    }
    # Create a new changelist
    my $p4;
    open($p4, "p4 -c $P4CLIENT -H $P4HOST -d $P4ROOT change -o|") or
	die "$0: failed to create changelist\n";
    my @desc = map {s/\r?\n$//so; $_} <$p4>;
    close($p4);

    my $desc_pos = 0;
    foreach (@desc) {
	++$desc_pos;
	last if /^Description:/o;
    }
    my $editfd;
    die "$GIT_DIR/p4/desc.txt: $!\n"
	if !open($editfd, '>', "$GIT_DIR/p4/desc.txt");
    my $range = "..$mergehead";
    # because I have no information about what I am merging with:
    # this is the case when a push modified HEAD.
    $range = "${mergehead}^..$mergehead" if $source_head eq 'HEAD';
    if (open(my $fd, '-|', "git log $range")) {
	while(<$fd>) {
	    # I believe it is not possible to save this information
	    # in Perforce. They are primitive
	    next if /^(commit |Author:|Date:)/;
	    s/\r?\n$//so;
	    next if !length($_); # header/message separator line
	    s/^\s{4}//o;
	    print $editfd "$_\n" if defined($editfd);
	}
	close($fd);
    }
    print $editfd map { /^\s?(.*)/o;"$1\n" } @desc[$desc_pos..$#desc]; 
    close($editfd);

    system("$editor $GIT_DIR/p4/desc.txt") if $EDIT_COMMIT;
    die "$GIT_DIR/p4/desc.txt: $!\n"
	if !open($editfd, '<', "$GIT_DIR/p4/desc.txt");
    my @tmpdesc = <$editfd>;
    die "No change description, not merging.\n"
	if !grep {!m/^(\s|[\r\n])*$/so} @tmpdesc;
    splice(@desc, $desc_pos, $#desc + 1 - $desc_pos,
	   map {s/\r?\n$//so;" $_"} @tmpdesc);
    close($editfd);

    open($p4, '>', "$GIT_DIR/p4/changelist") or
	die "$GIT_DIR/p4/changelist: $!\n";
    print $p4 map {"$_\n"} @desc;
    close($p4);

    return 0 if $DRYRUN;

    open(STDIN, '<', "$GIT_DIR/p4/changelist") or
	die "$GIT_DIR/p4/changelist: $!\n";
    open($p4, "p4 -c $P4CLIENT -H $P4HOST -d $P4ROOT change -i|") or
	die "$0: failed to create changelist\n";
    my ($newchange) = grep {s/^Change (\d+) created\b.*/$1/so} <$p4>;
    close($p4);
    print "Checking out P4 files in changelist $newchange\n" if $VERBOSE;
    sub runp4 {
	run_or_exit('p4','-c',$P4CLIENT,'-H',$P4HOST,'-d',$P4ROOT,@_);
    }
    # open files for edit
    $cnt = 0;
    open($p4, '>', "$GIT_DIR/p4/files") or die "$GIT_DIR/p4/files: $!\n";
    print $p4 "-c\n$newchange\n";
    print $p4 (map {++$cnt; substr($_,1)."\n"} grep {/^M/} @files);
    close($p4);
    runp4('-x',"$GIT_DIR/p4/files", 'edit') if $cnt;
    exit(1) if $?;

    $cnt = 0;
    open($p4, '>', "$GIT_DIR/p4/files") or die "$GIT_DIR/p4/files: $!\n";
    print $p4 "-c\n$newchange\n";
    print $p4 (map {++$cnt; substr($_,1)."\n"} grep {/^A/} @files);
    close($p4);
    runp4('-x',"$GIT_DIR/p4/files", 'add') if $cnt;
    exit(1) if $?;

    $cnt = 0;
    open($p4, '>', "$GIT_DIR/p4/files") or die "$GIT_DIR/p4/files: $!\n";
    print $p4 "-c\n$newchange\n";
    print $p4 (map {++$cnt; substr($_,1)."\n"} grep {/^D/} @files);
    close($p4);
    runp4('-x',"$GIT_DIR/p4/files", 'delete') if $cnt;
    exit(1) if $?;

    # p4 modifies working directory on checkout, stupid thing
    system('git', 'update-index', '--refresh');
    if ($do_merge) {
	run_or_exit('git', 'merge', '--no-commit', $mergehead);
    } else {
	run_or_exit('git', 'read-tree', '-m', '-u', $mergehead);
	print "The state of $source_head($mergehead) is checked out.\n";
    }
    if (!$auto_submit) {
	print "A p4 changelist $newchange is prepared.\n";
	print "To submit:\n\tp4 submit -c $newchange\n";
	print "To commit:\n\tgit-p4-import.bat -y -C SUBMITTED_CHANGE\n"
	    if !$fast_forward;
    } else {
	die "p4/files: $!\n" if !open($p4, '>', "$GIT_DIR/p4/files");
	print $p4 "-c\n$newchange\n";
	close($p4);
	my @res;
	my $subchange = undef;
	@res = qx{p4 -c $P4CLIENT -H $P4HOST -d $P4ROOT -x $GIT_DIR/p4/files submit};
	die "p4: failed to run p4 submit -c $newchange\n" if $?;
	foreach(@res) {
	    print;
	    if (/^\s*Change (\d+) submitted\..*/s) {
		$subchange = $1;
	    }
	    elsif (/^\s*Change (\d+) renamed change (\d+) and submitted.*/s) {
		$subchange = $2 if $1 eq $newchange;
	    }
	}
	die "p4 submit behaved unexpectedly\n".
	    "To tag the current HEAD use \n".
	    "\tgit-p4-import --p4-tag <the new change>\n"
	    if !defined($subchange);
	$subchange =~ s/\r?\n$//so;
	print "Submitted as $subchange\n";
	if ($fast_forward) {
	    $rc = git_p4_tag_HEAD($subchange);
	    exit(127) if $rc & 0xff;
	    exit(1) if $rc;
	    print "Tagged as p4/$subchange\n";
	} else {
	    print "The merged tree is left in $P4ROOT.\n";
	    print "To commit:\n\tgit-p4-import.bat -y -C $subchange\n";
	}
    }
    return 0;
}

__END__
:__end_of_file__

Re: Perforce support.

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 1594 bytes --]

On Thu, Jul 12, 2007 at 11:34:56 -0700, Govind Salinas wrote:
> I have been playing around with git for a month or so and have started
> writing, what I hope will be, a nice GUI over git that works well on
> Windows (Cygwin) and offers some feeling of familiarity to our
> Perforce users.  That however is only half the problem.

There are already several guis for git. None of them covers complete git
functionality, but if you can start adding functionality to one of them
instead of starting from scratch, it would give you a good head start. At
least the git-gui (in Tcl/Tk, part of git distribution) and qgit4
(git://repo.or.cz/qgit4.git, discuss changes on the git mailing list -- se
also http://digilander.libero.it/mcostalba/, but that only discusses the
previous Qt3-based branch) work on Windows.

> We need to be able to go back and forth to our main Perforce depot,
> and while I understand that git-svn support is very good, I have only
> seen limited support of Perforce.  I was wondering if anyone has been
> using git with p4 and how well did it work.  We have very complex and
> somewhat large "clients" that do a lot of mapping of directories
> (which strikes me as particularly insane) and I was wondering if any
> of the tools support that.
>
> If anyone has any suggestions/guidance on how to do this I would appreciate 
> it.

Search the list. The archive at http://marc.info/?l=git provides quite
sensible search support. There were some interesting discussions about
perforce in last few months.

-- 
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]