[PATCH] git-svn: Print revision while searching for earliest use of path

[PATCH] git-svn: Print revision while searching for earliest use of path

[Deskin Miller]

When initializing a git-svn repository from a Subversion repository, it
is common to be interested in a path which did not exist in the initial
commit to Subversion.  In a large repository like e.g. Apache, this may
take some time while the user receives no additional feedback.  Print
the highest revision number scanned thus far to let the user know
something is still happening.

Signed-off-by: Deskin Miller <deskinm@umich.edu>
---
This came about on account of patmaddox asking on #git why git-svn
seemed to be hung on clone.  Despite the admonition that this might take
a long time, I also like to have some indication that progress is being
made.  My first version of this printed using '\rChecked through
r$revision' but the subsequent output line when the path is found ends
up clobbered on the same line, and I'm not skilled enough at the
terminal or Perl to address this cleanly.  If the current version is
felt to be too verbose since it is printing a new line, I'd be up for
squelching the output to e.g. every 1000 revisions or so.

Anecdotally, it looks like Subversion looks for the path in blocks of
100 revisions, so we get the nice whole revision number for free.  I
couldn't find any documentation on the proper format of the error
message, so I just came up with the regular expressions to parse the
revision myself; if they need to be more explicit to avoid really
egregious path names, I can make an effort.

I tested on both http:// and file:// transport, to come up with the
different error strings; since the error number for file is the same as
svn:// I'm hoping that the error string is the same too.  If someone can
bounce this off a svn:// repo I'd appreciate it, otherwise I'll dig out
the documentation and set up a network-served svn repository myself
(which is really my job as the patch author anyway).

Deskin Miller

 git-svn.perl |    7 +++++++
 1 files changed, 7 insertions(+), 0 deletions(-)

diff --git a/git-svn.perl b/git-svn.perl
index 79888a0..60b56be 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -4542,6 +4542,12 @@ sub skip_unknown_revs {
 	#   More codes may be discovered later...
 	if ($errno == 175007 || $errno == 175002 || $errno == 160013) {
 		my $err_key = $err->expanded_message;
+		my $revision = $err_key;
+		if ($errno == 175007 || $errno == 175002 ) {
+			$revision =~ s/.*!svn\/bc\/(\d+).*/$1/;
+		} elsif ($errno == 160013) {
+			$revision =~ s/.*File not found: revision (\d+).*/$1/;
+		}
 		# revision numbers change every time, filter them out
 		$err_key =~ s/\d+/\0/g;
 		$err_key = "$errno\0$err_key";
@@ -4555,6 +4561,7 @@ sub skip_unknown_revs {
 			     "This may take a while on large repositories\n";
 			$ignored_err{$err_key} = 1;
 		}
+		print "Checked through r$revision\n";
 		return;
 	}
 	die "Error from SVN, ($errno): ", $err->expanded_message,"\n";
-- 
1.6.1.399.g0d272

Re: [PATCH] git-svn: Print revision while searching for earliest use of path

[Eric Wong]

Deskin Miller <deskinm@umich.edu> wrote:
> When initializing a git-svn repository from a Subversion repository, it
> is common to be interested in a path which did not exist in the initial
> commit to Subversion.  In a large repository like e.g. Apache, this may
> take some time while the user receives no additional feedback.  Print
> the highest revision number scanned thus far to let the user know
> something is still happening.
> 
> Signed-off-by: Deskin Miller <deskinm@umich.edu>
> ---
> This came about on account of patmaddox asking on #git why git-svn
> seemed to be hung on clone.  Despite the admonition that this might take
> a long time, I also like to have some indication that progress is being
> made.  My first version of this printed using '\rChecked through
> r$revision' but the subsequent output line when the path is found ends
> up clobbered on the same line, and I'm not skilled enough at the
> terminal or Perl to address this cleanly.  If the current version is
> felt to be too verbose since it is printing a new line, I'd be up for
> squelching the output to e.g. every 1000 revisions or so.

This is definitely useful on slow/large repositories.  The current
output with newlines is fine by me.

> Anecdotally, it looks like Subversion looks for the path in blocks of
> 100 revisions, so we get the nice whole revision number for free.  I
> couldn't find any documentation on the proper format of the error
> message, so I just came up with the regular expressions to parse the
> revision myself; if they need to be more explicit to avoid really
> egregious path names, I can make an effort.
> 
> I tested on both http:// and file:// transport, to come up with the
> different error strings; since the error number for file is the same as
> svn:// I'm hoping that the error string is the same too.  If someone can
> bounce this off a svn:// repo I'd appreciate it, otherwise I'll dig out
> the documentation and set up a network-served svn repository myself
> (which is really my job as the patch author anyway).

Couldn't we avoid the trouble of parsing the inconsistent error
messages by printing this status message after the get_log() calls
in gs_fetch_loop_common() ?

-- 
Eric Wong

Re: [PATCH] git-svn: Print revision while searching for earliest use of path

[Deskin Miller]

Resending due to temporary insanity with email, sorry Eric.

On Thu, Feb 5, 2009 at 01:42, Eric Wong <normalperson@yhbt.net> wrote:
> Deskin Miller <deskinm@umich.edu> wrote:
>> If the current version is
>> felt to be too verbose since it is printing a new line, I'd be up for
>> squelching the output to e.g. every 1000 revisions or so.
>
> This is definitely useful on slow/large repositories.  The current
> output with newlines is fine by me.

Ok.

> Couldn't we avoid the trouble of parsing the inconsistent error
> messages by printing this status message after the get_log() calls
> in gs_fetch_loop_common() ?

I didn't look at get_log closely, because I thought it executed only
once.  I definitely agree that it would be cleaner if it's possible.
Let me look at the code some more and I'll see what I can do, or feel
free to post an alternate implementation.

Deskin Miller

[PATCH v2] git-svn: Print revision while searching for earliest use of path

[Deskin Miller]

When initializing a git-svn repository from a Subversion repoository,
it is common to be interested in a path which did not exist in the
initial commit to Subversion.  In a large repository, the initial fetch
may take some looking for the earliest existence of the path time while
the user receives no additional feedback.  Print the highest revision
number scanned thus far to let the user know something is still
happening.

Signed-off-by: Deskin Miller <deskinm@umich.edu>
---
I'd never looked closely enough at gs_fetch_loop_common to grok what was
going on, and my previous experience with svn's get_log led me to think
that git-svn would slurp up log info for all commits at once.  Silly of
me.

Eric, things are much cleaner doing as you suggest.  I added a
carriage-return at the end of the message because I like it that way,
and the line will be overwritten once it actually starts fetching data
from svn.  I don't feel strongly about it though, so if it's better to
make it a newline, please feel free to change it and apply.

Deskin Miller

 git-svn.perl |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/git-svn.perl b/git-svn.perl
index 79888a0..601e2a3 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -4348,6 +4348,9 @@ sub gs_fetch_loop_common {
 		}
 		$self->get_log([$longest_path], $min, $max, 0, 1, 1,
 		               sub { $revs{$_[1]} = _cb(@_) });
+		if ($err) {
+			print "Checked through r$max\r";
+		}
 		if ($err && $max >= $head) {
 			print STDERR "Path '$longest_path' ",
 				     "was probably deleted:\n",
-- 
1.6.1.399.g0d272

Re: [PATCH v2] git-svn: Print revision while searching for earliest use of path

[Eric Wong]

Deskin Miller <deskinm@umich.edu> wrote:
> When initializing a git-svn repository from a Subversion repoository,
> it is common to be interested in a path which did not exist in the
> initial commit to Subversion.  In a large repository, the initial fetch
> may take some looking for the earliest existence of the path time while
> the user receives no additional feedback.  Print the highest revision
> number scanned thus far to let the user know something is still
> happening.
> 
> Signed-off-by: Deskin Miller <deskinm@umich.edu>

Thanks Deskin,

The carriage return actually looks quite good :)

Acked-by: From: Eric Wong <normalperson@yhbt.net>

> ---
> I'd never looked closely enough at gs_fetch_loop_common to grok what was
> going on, and my previous experience with svn's get_log led me to think
> that git-svn would slurp up log info for all commits at once.  Silly of
> me.
> 
> Eric, things are much cleaner doing as you suggest.  I added a
> carriage-return at the end of the message because I like it that way,
> and the line will be overwritten once it actually starts fetching data
> from svn.  I don't feel strongly about it though, so if it's better to
> make it a newline, please feel free to change it and apply.
> 
> Deskin Miller
> 
>  git-svn.perl |    3 +++
>  1 files changed, 3 insertions(+), 0 deletions(-)
> 
> diff --git a/git-svn.perl b/git-svn.perl
> index 79888a0..601e2a3 100755
> --- a/git-svn.perl
> +++ b/git-svn.perl
> @@ -4348,6 +4348,9 @@ sub gs_fetch_loop_common {
>  		}
>  		$self->get_log([$longest_path], $min, $max, 0, 1, 1,
>  		               sub { $revs{$_[1]} = _cb(@_) });
> +		if ($err) {
> +			print "Checked through r$max\r";
> +		}
>  		if ($err && $max >= $head) {
>  			print STDERR "Path '$longest_path' ",
>  				     "was probably deleted:\n",
> --