git-svn: persistent memoization

[Andrew Myrick <amyrick@apple.com>]

git-svn uses the Memoize perl module to cache the return values of a few functions.  This speeds up svn:mergeinfo processing considerably, but as currently implemented, this memoization table must be reconstructed on every run of git-svn.  This isn't a problem on small projects, but it introduces a delay of several minutes when fetching from large repositories with a lot of merge info.

The Memoize module has support for storing its cache persistently (more below, and see http://perldoc.perl.org/Memoize.html for details), and I would love to take advantage of this support in git-svn.  Unfortunately, it's not entirely obvious to me how to get this working.  Here are the questions I have so far, moving from high level design questions to implementation details:

1) Is there any reason not to store these caches on disk?

2) Are there situations where the caches would need to be invalidated?  Perhaps when git-svn rebuilds its metadata?

3) Where should I put these caches?  I was thinking something like ".git/svn/caches/<cachefile>.db" would be appropriate.

4) What's the correct way to reference the path to those caches?  I tried using "$ENV{GIT_DIR}/svn/caches/has_no_changes.db", but the memoize calls are in a BEGIN block, and it seems that $ENV{GIT_DIR} hadn't been initialized at that point.  This has taken me way past my limited knowledge of Perl.

5) What backend should I pick to store the cache?  Memoize supports storing to any tied hash that supports TIEHASH, FETCH, STORE, and EXISTS, like DB_File.  It also supports storing to SDBM_File and NDBM_File through glue modules (they lack EXISTS support), and Storable, which isn't a tied hash at all, but just a serializer for hashes.  I'm leaning towards using Storable since it seems simple and would fit the workload well, but I would appreciate any insight from someone with more domain knowledge here.

Regards,
Andrew