Re: [PATCH 5/6] Introduce the git fast-import-helper

[Tomas Carnecky <tom@dbservice.com>]

On 10/3/10 5:53 PM, Sverre Rabbelier wrote:
>> I only need two new things from fast-import:
>>  1) support non-numeric marks (and even this is maybe not strictly
>> required)
> 
> If this can be avoided, or worked around somehow, it would be a boon
> to performance. The current marks implementation uses a hash table
> index by the mark number, which is O(1), very efficient.

I also use a hash table (struct hash_table from hash.h). It's indexed by
the atom. So it's about equally fast as the existing one but uses
slightly more memory. I measured the speed and fih is about 5% slower
than fi. Also, I found out that setting max_packfile to 32MB makes the
import much faster (from 10 minutes down to 3m to import the sources of
git itself).

>>  2) dump the mark->sha1 mapping immediately after creating the object
>> (I heard there is a patch somewhere that does just that)
> 
> Why do you need that? Wouldn't the "write created object name to
> stdout" not be sufficient?

I do: fprintf(stdout, "mark :%s %s\n", mark, sha1_to_hex(sha1));
One reason why not just write the plain hash is because that's the same
syntax as the fih accepts in its input. This way you can do:
  $ ( cat marks; cat fast-export-stream ) | git fast-import-helper >> marks
and can restart at any time. Also, making the output a bit more
structured allows it to be easily extended in the future.

tom