git-svn of both trunk and tags while having no access to the 'parent' of those

git-svn of both trunk and tags while having no access to the 'parent' of those

[Yaroslav Halchenko]

Dear Git People,

I've ran into a situation here:

there is a repository with trunk and releases (analog to tags there)
available for public, but the rest of directories and their parent is
not available without authentication... ie I can access

http://domain.com/svnrepo/trunk
http://domain.com/svnrepo/releases
but not
http://domain.com/svnrepo/

Whenever I use git-svn (1.6.5 in Debian):

git svn clone --prefix=upstream-svn/ -T trunk -t releases http://domain.com/svnrepo svnrepo.gitsvn

it asks for authentication... I guess, now I can only clone trunk and
releases separately? or may be there is some way to avoid the
problem, ie avoid looking 'into root'?

Thanks in advance!
-- 
Yaroslav O. Halchenko
Postdoctoral Fellow,   Department of Psychological and Brain Sciences
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        

Re: git-svn of both trunk and tags while having no access to the 'parent' of those

[Michael J Gruber]

Yaroslav Halchenko venit, vidit, dixit 17.11.2009 03:59:
> Dear Git People,
> 
> I've ran into a situation here:
> 
> there is a repository with trunk and releases (analog to tags there)
> available for public, but the rest of directories and their parent is
> not available without authentication... ie I can access
> 
> http://domain.com/svnrepo/trunk
> http://domain.com/svnrepo/releases
> but not
> http://domain.com/svnrepo/
> 
> Whenever I use git-svn (1.6.5 in Debian):
> 
> git svn clone --prefix=upstream-svn/ -T trunk -t releases http://domain.com/svnrepo svnrepo.gitsvn
> 
> it asks for authentication... I guess, now I can only clone trunk and
> releases separately? or may be there is some way to avoid the
> problem, ie avoid looking 'into root'?
> 
> Thanks in advance!

Your problem description seems to match perfectly with the description
of the "--no-minimize-url" option in git svn's man page. I'm sure it's
worth a try.

Michael

Re: git-svn of both trunk and tags while having no access to the 'parent' of those

[Yaroslav Halchenko]

On Wed, 18 Nov 2009, Michael J Gruber wrote:

> > git svn clone --prefix=upstream-svn/ -T trunk -t releases http://domain.com/svnrepo svnrepo.gitsvn

> Your problem description seems to match perfectly with the description
> of the "--no-minimize-url" option in git svn's man page. I'm sure it's
> worth a try.

;-) oh I did I did ;)
I've used

git svn clone --no-minimize-url --prefix=upstream-svn/ -T trunk -t releases  http://domain.com/svnrepo/trunk svnrepo.gitsvn

that initialized repository, went through a lengthy list of 

trace: built-in: git 'config' 'svn-remote.svn.tags-maxRev' '...'

with last command reported

trace: built-in: git 'gc' '--auto'

and resulted in nothing being cloned/checked out or even a single ref.
The only file under .git besides the ones created by git init

./svn/refs/remotes/upstream-svn/trunk/.rev_map.33fb83da-1015-0410-9b9b-96027f9a4af8

and if I omitted trunk/ from url -- the same story of needed
authentication

-- 
Yaroslav O. Halchenko
Postdoctoral Fellow,   Department of Psychological and Brain Sciences
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        

Re: git-svn of both trunk and tags while having no access to the 'parent' of those

[Michael J Gruber]

Yaroslav Halchenko venit, vidit, dixit 18.11.2009 14:32:
> 
> On Wed, 18 Nov 2009, Michael J Gruber wrote:
> 
>>> git svn clone --prefix=upstream-svn/ -T trunk -t releases http://domain.com/svnrepo svnrepo.gitsvn
> 
>> Your problem description seems to match perfectly with the description
>> of the "--no-minimize-url" option in git svn's man page. I'm sure it's
>> worth a try.
> 
> ;-) oh I did I did ;)
> I've used
> 
> git svn clone --no-minimize-url --prefix=upstream-svn/ -T trunk -t releases  http://domain.com/svnrepo/trunk svnrepo.gitsvn

Is the trunk really at svnrepo/trunk/trunk?

> that initialized repository, went through a lengthy list of 
> 
> trace: built-in: git 'config' 'svn-remote.svn.tags-maxRev' '...'
> 
> with last command reported
> 
> trace: built-in: git 'gc' '--auto'
> 
> and resulted in nothing being cloned/checked out or even a single ref.
> The only file under .git besides the ones created by git init
> 
> ./svn/refs/remotes/upstream-svn/trunk/.rev_map.33fb83da-1015-0410-9b9b-96027f9a4af8
> 
> and if I omitted trunk/ from url -- the same story of needed
> authentication

I would try both

git svn clone --no-minimize-url --prefix=upstream-svn/ -T trunk -t
releases  http://domain.com/svnrepo/ svnrepo.gitsvn

and also the seemingly equivalent

git svn clone --no-minimize-url --prefix=upstream-svn/ -T
http://domain.com/svnrepo/trunk -t http://domain.com/svnrepo/releases
svnrepo.gitsvn

Also, I assume you can svn list http://domain.com/svnrepo/trunk and
http://domain.com/svnrepo/releases ;)

Michael

Re: git-svn of both trunk and tags while having no access to the 'parent' of those

[Yaroslav Halchenko]

On Wed, 18 Nov 2009, Michael J Gruber wrote:
> > git svn clone --no-minimize-url --prefix=upstream-svn/ -T trunk -t releases  http://domain.com/svnrepo/trunk svnrepo.gitsvn
> Is the trunk really at svnrepo/trunk/trunk?
nope... it is just svnrepo/trunk but if I set url to point to parent --
git svn seeks authentication right away

> I would try both
> git svn clone --no-minimize-url --prefix=upstream-svn/ -T trunk -t
> releases  http://domain.com/svnrepo/ svnrepo.gitsvn

asks for authentication since there is no public access to
http://domain.com/svnrepo/

> and also the seemingly equivalent

> git svn clone --no-minimize-url --prefix=upstream-svn/ -T
> http://domain.com/svnrepo/trunk -t http://domain.com/svnrepo/releases
> svnrepo.gitsvn
seems to not work since it wants url as a parameter 

Bad URL passed to RA layer: Illegal repository URL svnrepo.gitsvn  at /usr/lib/git-core/git-svn line 940

> Also, I assume you can svn list http://domain.com/svnrepo/trunk and
> http://domain.com/svnrepo/releases ;)
yeap -- I can list both of those but not their parent.


-- 
Yaroslav O. Halchenko
Postdoctoral Fellow,   Department of Psychological and Brain Sciences
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        

Re: git-svn of both trunk and tags while having no access to the 'parent' of those

[Michael J Gruber]

Yaroslav Halchenko venit, vidit, dixit 18.11.2009 15:23:
> 
> On Wed, 18 Nov 2009, Michael J Gruber wrote:
>>> git svn clone --no-minimize-url --prefix=upstream-svn/ -T trunk -t releases  http://domain.com/svnrepo/trunk svnrepo.gitsvn
>> Is the trunk really at svnrepo/trunk/trunk?
> nope... it is just svnrepo/trunk but if I set url to point to parent --
> git svn seeks authentication right away
> 
>> I would try both
>> git svn clone --no-minimize-url --prefix=upstream-svn/ -T trunk -t
>> releases  http://domain.com/svnrepo/ svnrepo.gitsvn
> 
> asks for authentication since there is no public access to
> http://domain.com/svnrepo/
> 
>> and also the seemingly equivalent
> 
>> git svn clone --no-minimize-url --prefix=upstream-svn/ -T
>> http://domain.com/svnrepo/trunk -t http://domain.com/svnrepo/releases
>> svnrepo.gitsvn
> seems to not work since it wants url as a parameter 
> 
> Bad URL passed to RA layer: Illegal repository URL svnrepo.gitsvn  at /usr/lib/git-core/git-svn line 940
> 
>> Also, I assume you can svn list http://domain.com/svnrepo/trunk and
>> http://domain.com/svnrepo/releases ;)
> yeap -- I can list both of those but not their parent.
> 
> 

OK, so the way it's implemented --no-minimize-url might be half as
useful as it could be. One last try (before asking Eric...) would be

git svn clone --no-minimize-url --prefix=upstream-svn/ -T
http://domain.com/svnrepo/trunk -t http://domain.com/svnrepo/releases
http://domain.com/svnrepo/trunk svnrepo.gitsvn

because that involves accessible URLs only and trunk and branch URLs are
absolute.

[Meanwhile I git the actual URL PMed by Yaroslov.]

So, what happens with the above is that git-svn tries to set the URL
config again from the URL part of an absolute tags argument. I don't
know how absolute URLs (which are documented) for tags etc. could
possibly work if git-svn tries to do that. Eric?

I tried also with two svn sections to circumvent this:

[svn-remote "svn"]
        url = http://domain.com:/project/trunk
        fetch = :refs/remotes/trunk
[svn-remote "svnr"]
        url = http://domain.com:/project/releases
        tags = /*:refs/remotes/tags/*

Fetching -Rsvn works fine, but fetching -Rsvnr gives the same
authentication problems. And fetch does not accept --no-minimize-url as
an option. OTOH: If the option is not used it seems to me (from the
source) that not minimizing is the default, which leaves me really
stomped. Eric?? ;)

Michael

Re: git-svn of both trunk and tags while having no access to the 'parent' of those

[Eric Wong]

Michael J Gruber <git@drmicha.warpmail.net> wrote:
> Yaroslav Halchenko venit, vidit, dixit 18.11.2009 15:23:
> > 
> > On Wed, 18 Nov 2009, Michael J Gruber wrote:
> >>> git svn clone --no-minimize-url --prefix=upstream-svn/ -T trunk -t releases  http://domain.com/svnrepo/trunk svnrepo.gitsvn
> >> Is the trunk really at svnrepo/trunk/trunk?
> > nope... it is just svnrepo/trunk but if I set url to point to parent --
> > git svn seeks authentication right away
> > 
> >> I would try both
> >> git svn clone --no-minimize-url --prefix=upstream-svn/ -T trunk -t
> >> releases  http://domain.com/svnrepo/ svnrepo.gitsvn
> > 
> > asks for authentication since there is no public access to
> > http://domain.com/svnrepo/
> > 
> >> and also the seemingly equivalent
> > 
> >> git svn clone --no-minimize-url --prefix=upstream-svn/ -T
> >> http://domain.com/svnrepo/trunk -t http://domain.com/svnrepo/releases
> >> svnrepo.gitsvn
> > seems to not work since it wants url as a parameter 
> > 
> > Bad URL passed to RA layer: Illegal repository URL svnrepo.gitsvn  at /usr/lib/git-core/git-svn line 940

Hi Michael, Yaroslav,

I think that second command syntax only works for "git svn init" right
now, not sure if it ever worked with "clone".

> >> Also, I assume you can svn list http://domain.com/svnrepo/trunk and
> >> http://domain.com/svnrepo/releases ;)
> > yeap -- I can list both of those but not their parent.
> > 
> 
> OK, so the way it's implemented --no-minimize-url might be half as
> useful as it could be. One last try (before asking Eric...) would be
> 
> git svn clone --no-minimize-url --prefix=upstream-svn/ -T
> http://domain.com/svnrepo/trunk -t http://domain.com/svnrepo/releases
> http://domain.com/svnrepo/trunk svnrepo.gitsvn
> 
> because that involves accessible URLs only and trunk and branch URLs are
> absolute.
> 
> [Meanwhile I git the actual URL PMed by Yaroslov.]

Any chance I could get that URL?  (I probably won't get a chance to
spend time debugging this until Friday).  Might be worth it to get an
strace or tcpdump to see what exactly is going on.  Or at least some
print statements around Git::SVN::Ra::new()...

> So, what happens with the above is that git-svn tries to set the URL
> config again from the URL part of an absolute tags argument. I don't
> know how absolute URLs (which are documented) for tags etc. could
> possibly work if git-svn tries to do that. Eric?

It looks like support for specifying absolute URLs with "clone"
has been broken for a while.  Guess I'll need test cases...

> I tried also with two svn sections to circumvent this:
> 
> [svn-remote "svn"]
>         url = http://domain.com:/project/trunk
>         fetch = :refs/remotes/trunk
> [svn-remote "svnr"]
>         url = http://domain.com:/project/releases
>         tags = /*:refs/remotes/tags/*
> 
> Fetching -Rsvn works fine, but fetching -Rsvnr gives the same
> authentication problems. And fetch does not accept --no-minimize-url as
> an option. OTOH: If the option is not used it seems to me (from the
> source) that not minimizing is the default, which leaves me really
> stomped. Eric?? ;)

Odd that the svnr one doesn't work.  Does changing:
         tags = /*:refs/remotes/tags/*
to:
         tags = *:refs/remotes/tags/*
help at all?

Minimizing should be the default in 1.6.5.  I'm a bit stumped, too :x
I'd like to see what network traffic git svn is sending for
the "svnr" case vs "svn ls http://domain.com/project/releases"

-- 
Eric Wong

Re: git-svn of both trunk and tags while having no access to the 'parent' of those

[Eric Wong]

Eric Wong <normalperson@yhbt.net> wrote:
> Michael J Gruber <git@drmicha.warpmail.net> wrote:
> > I tried also with two svn sections to circumvent this:
> > 
> > [svn-remote "svn"]
> >         url = http://domain.com:/project/trunk
> >         fetch = :refs/remotes/trunk
> > [svn-remote "svnr"]
> >         url = http://domain.com:/project/releases
> >         tags = /*:refs/remotes/tags/*

Hi Yaroslav,

Weird, I'm a bit confused by this myself but I'm out of time
at the moment, but swapping '[svn-remote "svn"]' and "[svn-remote "svnr"]"
seems to work, so there's apparently a bug in our handling of
svn-remote != "svn" somewhere...

So the following config should work:

[svn-remote "svnt"]
        url = http://domain.com:/project/trunk
        fetch = :refs/remotes/trunk
[svn-remote "svn"]
        url = http://domain.com:/project/releases
        tags = /*:refs/remotes/tags/*

You should probably run "git svn fetch -R svnt" before running
"git svn fetch" to pick up the tagged releases.

-- 
Eric Wong

[PATCH] git svn: always reuse existing remotes on fetch

[Eric Wong]

Eric Wong <normalperson@yhbt.net> wrote:
> so there's apparently a bug in our handling of
> svn-remote != "svn" somewhere...

>From 4d0157d6995925ea55ff181ea94d058583333f90 Mon Sep 17 00:00:00 2001
From: Eric Wong <normalperson@yhbt.net>
Date: Sun, 22 Nov 2009 12:37:06 -0800
Subject: [PATCH] git svn: always reuse existing remotes on fetch

The internal no_reuse_existing flag is set to allow initializing
multiple remotes with the same URL, common with SVM users.

Unfortunately, this flag caused misbehavior when used
with the -R command-line flag for fetching.

Signed-off-by: Eric Wong <normalperson@yhbt.net>
---
 git-svn.perl |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/git-svn.perl b/git-svn.perl
index 2746895..7f7a56f 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -428,6 +428,7 @@ sub cmd_fetch {
 	if (@_ > 1) {
 		die "Usage: $0 fetch [--all] [--parent] [svn-remote]\n";
 	}
+	$Git::SVN::no_reuse_existing = undef;
 	if ($_fetch_parent) {
 		my ($url, $rev, $uuid, $gs) = working_head_info('HEAD');
 		unless ($gs) {
@@ -956,6 +957,7 @@ sub cmd_multi_init {
 }
 
 sub cmd_multi_fetch {
+	$Git::SVN::no_reuse_existing = undef;
 	my $remotes = Git::SVN::read_all_remotes();
 	foreach my $repo_id (sort keys %$remotes) {
 		if ($remotes->{$repo_id}->{url}) {
-- 
Eric Wong