Re: From P4 to Git

[Sam Vilain <sam@vilain.net>]

On Mon, 2009-08-03 at 15:50 +0200, Alex Riesen wrote:
> On Mon, Aug 3, 2009 at 13:30, Sam Vilain<sam@vilain.net> wrote:
> > On Mon, 2009-08-03 at 10:47 +0200, Alex Riesen wrote:
> >> Is it an import-once tool, or can the process be restarted? (because it looks
> >> like the script needs a complicated setup).
> >
> > It's fully restartable.  Not only that but it uses transaction
> > protection to make sure that its internal state doesn't get corrupted
> > when performing the various options.
> 
> "varios options"? Operations? As when working on a live server?
> Aren't P4 changelist numbers always increasing? Or you mean
> the protection against multiple running instances of p4raw,
> so it is also parallelizable?

The "live" parts are never touched - only the write-only files that
perforce writes; and the rcs files are read using rcs.

What I was referring to by "various options" was the commands in the
program which perform the migration.  It's a multi-step process - load
the metadata from perforce 'load', import the file images to git blobs
'export-blobs', trace through the path structure and decide which parts
are the roots, and make parents 'find-branches', etc.  All parts are
fully transaction protected, restartable and rewindable.  It also means
that if a command crashes that you are left in a sane state.  I had to
build it like that to keep my sanity writing and using it :-).  After
all it was something of a reverse engineering of Perforce's internals.

All you need is the RCS backing files, a 'checkpoint' and (optionally)
'journal' files.  No access to the live perforce DBs is required.

> > No, it's server only. ...
> Darn. I hoped it wasn't. Can't play with it, then.

Yeah, in theory you could derive the internal table information by
issuing zillions of 'p4 integrated', 'p4 filelog' commands etc.  I've
written p4raw sub-commands which can produce the output of those
commands but not tried to go the other way; I wasn't interested.

> > ... I think I did get around to implementing not having
> > to go through all the stages for branches you didn't care to import.  It's
> > difficult though, the stage which correlates those thousands of 'integrate'
> > records is never going to be fast.
> Maybe if it is done locally, it can be improved? You seem to use the
> Postgre for lookups, right?

I'm sure that the queries could be improved, but Postgres is actually
quite good at crunching the numbers.  It's just a lot of data to crunch.

> That's were I hoped your project could help. I thought, if I pull in all the
> needed changelists (selected by path/CL), there may be a way to
> recreate a mergeable history out of this dump. At least, one involving less
> labor then I have to do now.

Yeah... well the design of my tool is that it needs to have the perforce
internal information.  But really you can probably consider the
converter orphaned, I have no current need to work on it; it served a
purpose, which was converting perl's perforce to perl.git, and that's
history now.

Sam