fast-import: multiple processes possible?

fast-import: multiple processes possible?

[Ian Zimmerman]

Hi, I have a user question, I hope that's ok.  Actually, the end goal is
development of a fast-import frontend.  The fast-import manpage says
that it is safe to run "any other" git operation concurrently with
fast-import.  It doesn't say if it is possible and safe to run another
copy of fast-import itself.

The reason I'd like to do this is speedup.  With sequential approach my
frontend can only load each core on my system about 50%.  If I could run
multiple imports concurrently I could make it close to 100%, especially
when feeding it blobs (which take most of the time).

Thanks for the advice.

-- 
Ian Zimmerman <itz@buug.org>
gpg public key: 1024D/C6FF61AD 
fingerprint: 66DC D68F 5C1B 4D71 2EE5  BD03 8A00 786C C6FF 61AD
Ham is for reading, not for eating.

Re: fast-import: multiple processes possible?

[Shawn Pearce]

On Sat, Nov 27, 2010 at 2:07 PM, Ian Zimmerman <itz@buug.org> wrote:
>
> Hi, I have a user question, I hope that's ok.  Actually, the end goal is
> development of a fast-import frontend.  The fast-import manpage says
> that it is safe to run "any other" git operation concurrently with
> fast-import.  It doesn't say if it is possible and safe to run another
> copy of fast-import itself.

Yes, you can.

The only problem is, each fast-import cannot see the objects created
by the other concurrent fast-imports until they either terminate, or
execute "checkpoint" command received from the frontend.

-- 
Shawn.