how to skip branches on git svn clone/fetch when there are errors

how to skip branches on git svn clone/fetch when there are errors

[Daniele Segato]

Hi,

I'm trying to clone a big repository.

I follow this steps:

git init
git svn init svn://svn.mydomain.com/path/to/repo -T HEAD -b BRANCHES -t TAGS

vim .git/config # edited the svn-remote config as follow (add /root to
branches and tag) to match the repo structure
[svn-remote "svn"]
       url = svn://svn.mydomain.com
       fetch = path/to/repo/HEAD/root:refs/remotes/svn/trunk
       branches = path/to/repo/BRANCHES/*/root:refs/remotes/svn/*
       tags = path/to/repo/TAGS/*/root:refs/remotes/svn/tags/*

git svn fetch

When I reach revision ~7500 (I don't remember the exact number) I get an error:

$ git svn fetch
Use of uninitialized value in concatenation (.) or string at
/usr/lib/perl5/SVN/Core.pm line 584.
Authorization failed:  at /usr/bin/git-svn line 1415


After some debugging I found out the reason is something strange on
the SVN server: there is a folder in the SVN that give an error when
trying to access:

$ svn info svn://svn.mydomain.com/path/to/repo/BRANCHES/V2.1-A
svn: Authorization failed

The same error with svn list.

I don't know what's wrong with that branch but I just want to skip it...

I tried modifying the .git/config svn-remote configuration adding this:

ignore-paths = path\\/to\\/repo\\/BRANCHES\\/V2\\.1-A

and re-launching git svn fetch.

I had no luck.

Ho do I skip a path on the svn repository?

(I can't provide the real svn repo because it is password protected
and I can't give you the access)

Thanks,
Regards,
Daniele

Re: how to skip branches on git svn clone/fetch when there are errors

[Eric Wong]

Daniele Segato <daniele.bilug@gmail.com> wrote:
> Hi,
> 
> I'm trying to clone a big repository.
> 
> I follow this steps:
> 
> git init
> git svn init svn://svn.mydomain.com/path/to/repo -T HEAD -b BRANCHES -t TAGS
> 
> vim .git/config # edited the svn-remote config as follow (add /root to
> branches and tag) to match the repo structure
> [svn-remote "svn"]
>        url = svn://svn.mydomain.com
>        fetch = path/to/repo/HEAD/root:refs/remotes/svn/trunk
>        branches = path/to/repo/BRANCHES/*/root:refs/remotes/svn/*
>        tags = path/to/repo/TAGS/*/root:refs/remotes/svn/tags/*
> 
> git svn fetch
> 
> When I reach revision ~7500 (I don't remember the exact number) I get an error:
> 
> $ git svn fetch
> Use of uninitialized value in concatenation (.) or string at
> /usr/lib/perl5/SVN/Core.pm line 584.
> Authorization failed:  at /usr/bin/git-svn line 1415
> 
> 
> After some debugging I found out the reason is something strange on
> the SVN server: there is a folder in the SVN that give an error when
> trying to access:
> 
> $ svn info svn://svn.mydomain.com/path/to/repo/BRANCHES/V2.1-A
> svn: Authorization failed
> 
> The same error with svn list.
> 
> I don't know what's wrong with that branch but I just want to skip it...

Ouch :(

> I tried modifying the .git/config svn-remote configuration adding this:
> 
> ignore-paths = path\\/to\\/repo\\/BRANCHES\\/V2\\.1-A

> and re-launching git svn fetch.

ignore-paths is only for paths that get converted into part of
the git tree

> I had no luck.
> 
> Ho do I skip a path on the svn repository?

It's unfortunate, but there's not yet an exclude/ignore directive
when globbing.  You'll have to change your $GIT_CONFIG to only
have a list of branches you want, something like this:

[svn-remote "svn"]
	url = svn://svn.mydomain.com
	fetch = path/to/repo/HEAD/root:refs/remotes/svn/trunk

	; have one "fetch" line for every branch except the one you want
	fetch = path/to/repo/BRANCHES/a/root:refs/remotes/svn/a
	fetch = path/to/repo/BRANCHES/b/root:refs/remotes/svn/b
	fetch = path/to/repo/BRANCHES/c/root:refs/remotes/svn/c

	; you can do the same for tags if you have the same problem
	tags = path/to/repo/TAGS/*/root:refs/remotes/svn/tags/*

But you shouldn't have to worry about having "fetch" entries for
stale/old branches/tags you've already imported.

-- 
Eric Wong

Re: how to skip branches on git svn clone/fetch when there are errors

[Daniele Segato]

Il giorno ven, 04/09/2009 alle 23.16 -0700, Eric Wong ha scritto:
> It's unfortunate, but there's not yet an exclude/ignore directive
> when globbing.  You'll have to change your $GIT_CONFIG to only
> have a list of branches you want, something like this:
> 
> [svn-remote "svn"]
> 	url = svn://svn.mydomain.com
> 	fetch = path/to/repo/HEAD/root:refs/remotes/svn/trunk
> 
> 	; have one "fetch" line for every branch except the one you want
> 	fetch = path/to/repo/BRANCHES/a/root:refs/remotes/svn/a
> 	fetch = path/to/repo/BRANCHES/b/root:refs/remotes/svn/b
> 	fetch = path/to/repo/BRANCHES/c/root:refs/remotes/svn/c
> 
> 	; you can do the same for tags if you have the same problem
> 	tags = path/to/repo/TAGS/*/root:refs/remotes/svn/tags/*
> 
> But you shouldn't have to worry about having "fetch" entries for
> stale/old branches/tags you've already imported.

I see...
That means that then I'll have to manually add new created branches,
right?

well... I tried to avoid this kind of configuration for weeks :) but
that's probably my best way with that repo....

thank you,
Daniele

Re: how to skip branches on git svn clone/fetch when there are errors

[Eric Wong]

Daniele Segato <daniele.bilug@gmail.com> wrote:
> Il giorno ven, 04/09/2009 alle 23.16 -0700, Eric Wong ha scritto:
> > It's unfortunate, but there's not yet an exclude/ignore directive
> > when globbing.  You'll have to change your $GIT_CONFIG to only
> > have a list of branches you want, something like this:
> > 
> > [svn-remote "svn"]
> > 	url = svn://svn.mydomain.com
> > 	fetch = path/to/repo/HEAD/root:refs/remotes/svn/trunk
> > 
> > 	; have one "fetch" line for every branch except the one you want
> > 	fetch = path/to/repo/BRANCHES/a/root:refs/remotes/svn/a
> > 	fetch = path/to/repo/BRANCHES/b/root:refs/remotes/svn/b
> > 	fetch = path/to/repo/BRANCHES/c/root:refs/remotes/svn/c
> > 
> > 	; you can do the same for tags if you have the same problem
> > 	tags = path/to/repo/TAGS/*/root:refs/remotes/svn/tags/*
> > 
> > But you shouldn't have to worry about having "fetch" entries for
> > stale/old branches/tags you've already imported.
> 
> I see...
> That means that then I'll have to manually add new created branches,
> right?

Yes.

-- 
Eric Wong

Re: how to skip branches on git svn clone/fetch when there are errors

[Daniele Segato]

On Mon, Aug 31, 2009 at 10:26 AM, Daniele
Segato<daniele.bilug@gmail.com> wrote:>
> git init
> git svn init svn://svn.mydomain.com/path/to/repo -T HEAD -b BRANCHES -t TAGS
> $ git svn fetch
> Use of uninitialized value in concatenation (.) or string at
> /usr/lib/perl5/SVN/Core.pm line 584.
> Authorization failed:  at /usr/bin/git-svn line 1415
>

more info on the error (enabled the confess instead of croak to the
Core.pm library)

Use of uninitialized value in concatenation (.) or string at
/usr/lib/perl5/SVN/Core.pm line 585.
Authorization failed:  at /usr/lib/perl5/SVN/Core.pm line 654
	SVN::Error::confess_on_error('_p_svn_error_t=SCALAR(0x9492a50)')
called at /usr/lib/perl5/SVN/Ra.pm line 492
	SVN::Ra::AUTOLOAD('Git::SVN::Ra=HASH(0x945dae8)',
'alfresco-enterprise-mirror/alfresco/BRANCHES/V2.1-A/root', 7738,
'SVN::Pool=REF(0x9492bc0)') called at /usr/bin/git-svn line 3760
	Git::SVN::Ra::check_path('Git::SVN::Ra=HASH(0x945dae8)',
'alfresco-enterprise-mirror/alfresco/BRANCHES/V2.1-A/root', 7738)
called at /usr/bin/git-svn line 4045
	Git::SVN::Ra::get_dir_check('Git::SVN::Ra=HASH(0x945dae8)',
'HASH(0x92131e0)', 'HASH(0x9388050)', 7738) called at /usr/bin/git-svn
line 4062
	Git::SVN::Ra::match_globs('Git::SVN::Ra=HASH(0x945dae8)',
'HASH(0x92131e0)', 'HASH(0x9463c00)', 'ARRAY(0x90bbc00)', 7738) called
at /usr/bin/git-svn line 3985
	Git::SVN::Ra::gs_fetch_loop_common('Git::SVN::Ra=HASH(0x945dae8)',
7737, 16113, 'ARRAY(0x90bbbe0)', 'ARRAY(0x90bbc00)') called at
/usr/bin/git-svn line 1415
	Git::SVN::fetch_all('svn', 'HASH(0x9464250)') called at
/usr/bin/git-svn line 372
	main::cmd_fetch() called at /usr/bin/git-svn line 253
	eval {...} called at /usr/bin/git-svn line 251


I'll keep looking at it to see if I can figure out a way to "skip" the
error myself and, eventually, provide a patch

Bye,
Daniele

Re: how to skip branches on git svn clone/fetch when there are errors

[Daniele Segato]

On Mon, Sep 7, 2009 at 11:30 AM, Daniele Segato<daniele.bilug@gmail.com> wrote:
> more info on the error (enabled the confess instead of croak to the
> Core.pm library)
>
> Use of uninitialized value in concatenation (.) or string at
> /usr/lib/perl5/SVN/Core.pm line 585.
> Authorization failed:  at /usr/lib/perl5/SVN/Core.pm line 654
>        SVN::Error::confess_on_error('_p_svn_error_t=SCALAR(0x9492a50)')
> called at /usr/lib/perl5/SVN/Ra.pm line 492
>        SVN::Ra::AUTOLOAD('Git::SVN::Ra=HASH(0x945dae8)',
> 'alfresco-enterprise-mirror/alfresco/BRANCHES/V2.1-A/root', 7738,
> 'SVN::Pool=REF(0x9492bc0)') called at /usr/bin/git-svn line 3760
>        Git::SVN::Ra::check_path('Git::SVN::Ra=HASH(0x945dae8)',
> 'alfresco-enterprise-mirror/alfresco/BRANCHES/V2.1-A/root', 7738)
> called at /usr/bin/git-svn line 4045
>        Git::SVN::Ra::get_dir_check('Git::SVN::Ra=HASH(0x945dae8)',
> 'HASH(0x92131e0)', 'HASH(0x9388050)', 7738) called at /usr/bin/git-svn
> line 4062
>        Git::SVN::Ra::match_globs('Git::SVN::Ra=HASH(0x945dae8)',
> 'HASH(0x92131e0)', 'HASH(0x9463c00)', 'ARRAY(0x90bbc00)', 7738) called
> at /usr/bin/git-svn line 3985
>        Git::SVN::Ra::gs_fetch_loop_common('Git::SVN::Ra=HASH(0x945dae8)',
> 7737, 16113, 'ARRAY(0x90bbbe0)', 'ARRAY(0x90bbc00)') called at
> /usr/bin/git-svn line 1415
>        Git::SVN::fetch_all('svn', 'HASH(0x9464250)') called at
> /usr/bin/git-svn line 372
>        main::cmd_fetch() called at /usr/bin/git-svn line 253
>        eval {...} called at /usr/bin/git-svn line 251
>
>
> I'll keep looking at it to see if I can figure out a way to "skip" the
> error myself and, eventually, provide a patch

I played a little with perl and modified the code

I attach the patch I created...
it probably sucks and doesn't take cares of a lot of thing that I
didn't thought about...

After applying it I was able to continue the git svn fetch from the
point I left skipping those problematics paths...

Still I get a lot of warnings with:
Use of uninitialized value in concatenation (.) or string at
/usr/lib/perl5/SVN/Core.pm line 584


I just want you to take a look at it
I will not be disappointed if you place this patch in the recycle bean :)

I cloned the git repo, checked out the v1.5.6.5 tag (which is my
current version) and modified the git-svn.perl file.

It probably wont help but I think I should share it anyway

(patch below my signature)

Bye,
Daniele

From e8a1a12e83b3f0b18ce842190d8fc8eddaa77f68 Mon Sep 17 00:00:00 2001
From: Daniele Segato <daniele.bilug@gmail.com>
Date: Mon, 7 Sep 2009 15:30:14 +0200
Subject: [PATCH] Ignore error 175007 authorization failed on checkpath

I don't know if this is the best solution to solve the issue but it does
let me download the repo skipping the problematics paths
---
 git-svn.perl |   16 ++++++++++++++++
 1 files changed, 16 insertions(+), 0 deletions(-)

diff --git a/git-svn.perl b/git-svn.perl
index a366c89..0ab6453 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -3756,7 +3756,23 @@ sub check_path {
 		return $cache->{data}->{$path};
 	}
 	my $pool = SVN::Pool->new;
+	my $err_handler = $SVN::Error::handler;
+	$SVN::Error::handler = sub {
+		(my $err) = @_;
+		my $errno = $err->apr_err();
+		my $err_key = $err->expanded_message;
+		if ($errno == 175007) {
+			warn "W: Ignoring error from SVN, path probably ",
+				"does not exist: ($errno): ",
+				$err->expanded_message,"\n";
+		}
+		return;
+	};
+
 	my $t = $self->SUPER::check_path($path, $r, $pool);
+
+	$SVN::Error::handler = $err_handler;
+
 	$pool->clear;
 	if ($r != $cache->{r}) {
 		%{$cache->{data}} = ();
-- 
1.5.6.5

Re: how to skip branches on git svn clone/fetch when there are errors

[Daniele Segato]

On Mon, Sep 7, 2009 at 3:34 PM, Daniele Segato<daniele.bilug@gmail.com> wrote:
> +       $SVN::Error::handler = sub {
> +               (my $err) = @_;
> +               my $errno = $err->apr_err();
> +               my $err_key = $err->expanded_message;
> +               if ($errno == 175007) {
> +                       warn "W: Ignoring error from SVN, path probably ",
> +                               "does not exist: ($errno): ",
> +                               $err->expanded_message,"\n";
> +               }
> +               return;



ups..
this should have been return inside the if and die otherwise

and the error is 170001 (wrong copy/paste)

please ignore the patch I posted before

this is the "right" one:


From a17dfb0e268e11ce70587ccb48c359348f22ad99 Mon Sep 17 00:00:00 2001
From: Daniele Segato <daniele.bilug@gmail.com>
Date: Mon, 7 Sep 2009 15:30:14 +0200
Subject: [PATCH] Ignore err:170001 authorizationfailed on checkpath

I don't know if this is the best solution to solve the issue but it does
let me download the repo skipping the problematics paths
---
 git-svn.perl |   17 +++++++++++++++++
 1 files changed, 17 insertions(+), 0 deletions(-)

diff --git a/git-svn.perl b/git-svn.perl
index a366c89..80f958d 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -3756,7 +3756,24 @@ sub check_path {
 		return $cache->{data}->{$path};
 	}
 	my $pool = SVN::Pool->new;
+	my $err_handler = $SVN::Error::handler;
+	$SVN::Error::handler = sub {
+		(my $err) = @_;
+		my $errno = $err->apr_err();
+		my $err_key = $err->expanded_message;
+		if ($errno == 170001) {
+			warn "W: Ignoring error from SVN, path probably ",
+				"does not exist: ($errno): ",
+				$err->expanded_message,"\n";
+			return;
+		}
+		die "Error from SVN, ($errno): ", $err->expanded_message,"\n";
+	};
+
 	my $t = $self->SUPER::check_path($path, $r, $pool);
+
+	$SVN::Error::handler = $err_handler;
+
 	$pool->clear;
 	if ($r != $cache->{r}) {
 		%{$cache->{data}} = ();
-- 
1.5.6.5

Re: how to skip branches on git svn clone/fetch when there are errors

[Paolo Ciarrocchi]

On Mon, Sep 7, 2009 at 3:34 PM, Daniele Segato<daniele.bilug@gmail.com> wrote:
[...]

Ciao Daniele,

> From e8a1a12e83b3f0b18ce842190d8fc8eddaa77f68 Mon Sep 17 00:00:00 2001
> From: Daniele Segato <daniele.bilug@gmail.com>
> Date: Mon, 7 Sep 2009 15:30:14 +0200
> Subject: [PATCH] Ignore error 175007 authorization failed on checkpath
>
> I don't know if this is the best solution to solve the issue but it does
> let me download the repo skipping the problematics paths

Usually these lines are for the commit message, I guess you don't want
to store these two lines in git.git :-)

> ---

So if you want to add comments like you did, write them after the 3
minus and before the content of the patch.

>  git-svn.perl |   16 ++++++++++++++++
>  1 files changed, 16 insertions(+), 0 deletions(-)
>
> diff --git a/git-svn.perl b/git-svn.perl
> index a366c89..0ab6453 100755

Ciao,
-- 
Paolo

Re: how to skip branches on git svn clone/fetch when there are errors

[Daniele Segato]

Il giorno lun, 07/09/2009 alle 17.53 +0200, Paolo Ciarrocchi ha scritto:
> > I don't know if this is the best solution to solve the issue but it does
> > let me download the repo skipping the problematics paths
> 
> Usually these lines are for the commit message, I guess you don't want
> to store these two lines in git.git :-)

no
that was my local commit
(and I initially copied the wrong one....)

If my patch will be accepted (and I really doubt this will be the case)
I'll fix that with a git commit amend / git rebase -i following the git
developers guide-lines :)