[PATCH] git-svn: persistent memoization

[PATCH] git-svn: persistent memoization

[Andrew Myrick]

Make memoization of the svn:mergeinfo processing functions persistent with
Memoize::Storable so that the memoization tables don't need to be regenerated
every time the user runs git-svn fetch.

The Memoize::Storable hashes are stored in ENV{GIT_DIR}/svn/caches.

Signed-off-by: Andrew Myrick <amyrick@apple.com>
---
 git-svn.perl |   42 +++++++++++++++++++++++++++++++++++++-----
 1 files changed, 37 insertions(+), 5 deletions(-)

diff --git a/git-svn.perl b/git-svn.perl
index 1f201e4..f7a9410 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -1632,10 +1632,11 @@ use vars qw/$default_repo_id $default_ref_id $_no_metadata $_follow_parent
             $_use_svnsync_props $no_reuse_existing $_minimize_url
 	    $_use_log_author $_add_author_from $_localtime/;
 use Carp qw/croak/;
-use File::Path qw/mkpath/;
+use File::Path qw/mkpath make_path/;
 use File::Copy qw/copy/;
 use IPC::Open3;
 use Memoize;  # core since 5.8.0, Jul 2002
+use Memoize::Storable;
 
 my ($_gc_nr, $_gc_period);
 
@@ -3078,10 +3079,39 @@ sub has_no_changes {
 		command_oneline("rev-parse", "$commit~1^{tree}"));
 }
 
-BEGIN {
-	memoize 'lookup_svn_merge';
-	memoize 'check_cherry_pick';
-	memoize 'has_no_changes';
+# The GIT_DIR environment variable is not always set until after the command
+# line arguments are processed, so we can't memoize in a BEGIN block.
+{
+	my $memoized = 0;
+
+	sub memoize_svn_mergeinfo_functions {
+		return if $memoized;
+		$memoized = 1;
+
+		my $cache_path = "$ENV{GIT_DIR}/svn/caches/";
+		make_path($cache_path) unless -d $cache_path;
+
+		tie my %lookup_svn_merge_cache =>
+			'Memoize::Storable',"$cache_path/lookup_svn_merge.db", 'nstore';
+		memoize 'lookup_svn_merge',
+			SCALAR_CACHE => 'FAULT',
+			LIST_CACHE => ['HASH' => \%lookup_svn_merge_cache],
+		;
+
+		tie my %check_cherry_pick_cache =>
+			'Memoize::Storable',"$cache_path/check_cherry_pick.db", 'nstore';
+		memoize 'check_cherry_pick',
+			SCALAR_CACHE => 'FAULT',
+			LIST_CACHE => ['HASH' => \%check_cherry_pick_cache],
+		;
+
+		tie my %has_no_changes_cache =>
+			'Memoize::Storable',"$cache_path/has_no_changes.db", 'nstore';
+		memoize 'has_no_changes',
+			SCALAR_CACHE => ['HASH' => \%has_no_changes_cache],
+			LIST_CACHE => 'FAULT',
+		;
+	}
 }
 
 sub parents_exclude {
@@ -3125,6 +3155,8 @@ sub find_extra_svn_parents {
 	my ($self, $ed, $mergeinfo, $parents) = @_;
 	# aha!  svk:merge property changed...
 
+	memoize_svn_mergeinfo_functions();
+
 	# We first search for merged tips which are not in our
 	# history.  Then, we figure out which git revisions are in
 	# that tip, but not this revision.  If all of those revisions
-- 
1.6.6.1.4.g3df0a5.dirty

Re: [PATCH] git-svn: persistent memoization

[Eric Wong]

Andrew Myrick <amyrick@apple.com> wrote:
> Make memoization of the svn:mergeinfo processing functions persistent with
> Memoize::Storable so that the memoization tables don't need to be regenerated
> every time the user runs git-svn fetch.
> 
> The Memoize::Storable hashes are stored in ENV{GIT_DIR}/svn/caches.

Hi Andrew,

Perhaps "$ENV{GIT_DIR}/svn/.caches" is better here since older versions
of git svn used "$ENV{GIT_DIR}/svn/$refname" in the top-level and
"caches" may conflict with existing repos.

> -use File::Path qw/mkpath/;
> +use File::Path qw/mkpath make_path/;

File::Path::make_path is very recent not in Perls distributed by most
vendors.  My 5.10.0 installation (Debian stable) doesn't have it, and I
also don't see a good reason to use it over the traditional mkpath.

I think I'll squash the following patch and Ack.  Let me know if
you have any objections, thanks.!
(also wraps long lines to 80 chars)

diff --git a/git-svn.perl b/git-svn.perl
index 0153439..265852f 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -1652,7 +1652,7 @@ use vars qw/$default_repo_id $default_ref_id $_no_metadata $_follow_parent
             $_use_svnsync_props $no_reuse_existing $_minimize_url
 	    $_use_log_author $_add_author_from $_localtime/;
 use Carp qw/croak/;
-use File::Path qw/mkpath make_path/;
+use File::Path qw/mkpath/;
 use File::Copy qw/copy/;
 use IPC::Open3;
 use Memoize;  # core since 5.8.0, Jul 2002
@@ -3126,25 +3126,25 @@ sub has_no_changes {
 		return if $memoized;
 		$memoized = 1;
 
-		my $cache_path = "$ENV{GIT_DIR}/svn/caches/";
-		make_path($cache_path) unless -d $cache_path;
+		my $cache_path = "$ENV{GIT_DIR}/svn/.caches/";
+		mkpath([$cache_path]) unless -d $cache_path;
 
-		tie my %lookup_svn_merge_cache =>
-			'Memoize::Storable',"$cache_path/lookup_svn_merge.db", 'nstore';
+		tie my %lookup_svn_merge_cache => 'Memoize::Storable',
+		    "$cache_path/lookup_svn_merge.db", 'nstore';
 		memoize 'lookup_svn_merge',
 			SCALAR_CACHE => 'FAULT',
 			LIST_CACHE => ['HASH' => \%lookup_svn_merge_cache],
 		;
 
-		tie my %check_cherry_pick_cache =>
-			'Memoize::Storable',"$cache_path/check_cherry_pick.db", 'nstore';
+		tie my %check_cherry_pick_cache => 'Memoize::Storable',
+		    "$cache_path/check_cherry_pick.db", 'nstore';
 		memoize 'check_cherry_pick',
 			SCALAR_CACHE => 'FAULT',
 			LIST_CACHE => ['HASH' => \%check_cherry_pick_cache],
 		;
 
-		tie my %has_no_changes_cache =>
-			'Memoize::Storable',"$cache_path/has_no_changes.db", 'nstore';
+		tie my %has_no_changes_cache => 'Memoize::Storable',
+		    "$cache_path/has_no_changes.db", 'nstore';
 		memoize 'has_no_changes',
 			SCALAR_CACHE => ['HASH' => \%has_no_changes_cache],
 			LIST_CACHE => 'FAULT',
-- 
Eric Wong

Re: [PATCH] git-svn: persistent memoization

[Andrew Myrick]

On Jan 31, 2010, at 8:03 PM, Eric Wong wrote:

> Andrew Myrick <amyrick@apple.com> wrote:
>> Make memoization of the svn:mergeinfo processing functions persistent with
>> Memoize::Storable so that the memoization tables don't need to be regenerated
>> every time the user runs git-svn fetch.
>> 
>> The Memoize::Storable hashes are stored in ENV{GIT_DIR}/svn/caches.
> 
> Hi Andrew,
> 
> Perhaps "$ENV{GIT_DIR}/svn/.caches" is better here since older versions
> of git svn used "$ENV{GIT_DIR}/svn/$refname" in the top-level and
> "caches" may conflict with existing repos.
> 
>> -use File::Path qw/mkpath/;
>> +use File::Path qw/mkpath make_path/;
> 
> File::Path::make_path is very recent not in Perls distributed by most
> vendors.  My 5.10.0 installation (Debian stable) doesn't have it, and I
> also don't see a good reason to use it over the traditional mkpath.
> 
> I think I'll squash the following patch and Ack.  Let me know if
> you have any objections, thanks.!
> (also wraps long lines to 80 chars)

Makes sense to me.  Thanks, Eric.

-Andrew

Re: [PATCH] git-svn: persistent memoization

[Eric Wong]

Andrew Myrick <amyrick@apple.com> wrote:
> On Jan 31, 2010, at 8:03 PM, Eric Wong wrote:
> > Andrew Myrick <amyrick@apple.com> wrote:
> >> Make memoization of the svn:mergeinfo processing functions persistent with
> >> Memoize::Storable so that the memoization tables don't need to be regenerated
> >> every time the user runs git-svn fetch.
> >> 
> >> The Memoize::Storable hashes are stored in ENV{GIT_DIR}/svn/caches.
> > 
> > Hi Andrew,
> > 
> > Perhaps "$ENV{GIT_DIR}/svn/.caches" is better here since older versions
> > of git svn used "$ENV{GIT_DIR}/svn/$refname" in the top-level and
> > "caches" may conflict with existing repos.
> > 
> >> -use File::Path qw/mkpath/;
> >> +use File::Path qw/mkpath make_path/;
> > 
> > File::Path::make_path is very recent not in Perls distributed by most
> > vendors.  My 5.10.0 installation (Debian stable) doesn't have it, and I
> > also don't see a good reason to use it over the traditional mkpath.
> > 
> > I think I'll squash the following patch and Ack.  Let me know if
> > you have any objections, thanks.!
> > (also wraps long lines to 80 chars)
> 
> Makes sense to me.  Thanks, Eric.

Hi Junio,

I've acked and pushed this out to git://git.bogomips.org/git-svn

  Andrew Myrick (1):
        git-svn: persistent memoization

Sorry for the delay, my mind has slipped.  Thanks to Andrew
for the patch and reminder.

-- 
Eric Wong