Problem using git svn clone

Problem using git svn clone

[Gustaf Hendeby]

Hi!

I have a problem with git-svn, which I hope the list can shed some
light over.  I'm quite new to git (but so far I really like it, great
job guys!) and I never use SVN, so don't really know what information
may be needed to be able to help me on the right track.  I hope the
below problem description is appropriate.


The situation is as follows:

I plan to use git and git svn to track a module in the middle of a SVN
repository (which I have no control over) where I don't have read
access more than to this specific module and the base directory, ie
the layout is similar to this
	https://svn.foo.bar/a/b/c/trunk
	https://svn.foo.bar/a/b/c/tags
	https://svn.foo.bar/a/b/c/branches
and I have read access to https://svm.foo.bar/a,
https://svn.foo.bar/a/b/c and below, but nothing else.

What I want to do is clone the whole thing, including tags and
branches.  I expected to be able to use the following command to do
this (git v1.5.2):

$ git svn clone https://svn.foo.bar/a/b/c -T trunk -t tags -b branches

Initialized empty Git repository in .git/
Using higher level of URL: https://svn.foo.bar/a/b/c => https://svn.foo.bar/a

W: Ignoring error from SVN, path probably does not exist: (175002): RA
layer request failed: REPORT request failed on '/a/!svn/bc/100':
REPORT of '/a/!svn/bc/100': Could not read chunk size: Secure
connection truncated (https://svn.foo.bar)
r306 = 83f0c10b988a8f1e77a3f354126c52f0cfdecf76 (trunk)
[...]
Found possible branch point: https://svn.foo.bar/a/b/c/trunk =>
https://svn.foo.bar/a/b/c/tags/0.1, 314
Found branch parent: (tags/0.1) ad6a2361d0e69d6288ef226bb335bb4bf3bdd12e
Following parent with do_update
Successfully followed parent
r315 = 3b92c4885b9d6f60241533dd99fa5023eebb1c64 (tags/0.1)
r316 = 02e3b828b4f11c5fab9d10d85076a8ff209afa00 (trunk)
[...]

However, it seems I just get the info about the revisions (shows up
nicely in gitk) but no content at all.  The config file reads:

[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
[svn-remote "svn"]
        url = https://svn.foo.bar/a/b/c/trunk
        fetch = :refs/remotes/git-svn


If I instead do:

$ git svn clone https://svn.foo.bar/a/b/c/trunk
Initialized empty Git repository in .git/
W: Ignoring error from SVN, path probably does not exist: (175007):
HTTP Path Not Found: REPORT request failed on
'/a/!svn/bc/100/b/c/trunk': '/a/!svn/bc/100/b/c/trunk' path not found
W: +empty_dir: source
W: +empty_dir: test
r306 = 83f0c10b988a8f1e77a3f354126c52f0cfdecf76 (git-svn)
        A       resources/extensions/circuitparts.drext
[...]
r309 = 3a386e9b985c419d129461acbf24978795b36b96 (git-svn)
        A       tools/source/physical.cc
[...]

I get all content in trunk and their revision history, but as expected
no tag info.  The config file reads:

[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
[svn-remote "svn"]
        url = https://svn.foo.bar/a
        fetch = b/c/trunk:refs/remotes/trunk
        branches = b/c/branches/*:refs/remotes/*
        tags = b/c/tags/*:refs/remotes/tags/*


Removing trunk from the URL yields all the files in trunk, tags, and
branches with a similar output.

Please, let me know if information is missing.  I'd be very thankful
for any help that I can get to sort this out.

/Gustaf

Re: Problem using git svn clone

[Seth Falcon]

"Gustaf Hendeby" <hendeby@gmail.com> writes:

> Hi!
>
> I have a problem with git-svn, which I hope the list can shed some
> light over.  I'm quite new to git (but so far I really like it, great
> job guys!) and I never use SVN, so don't really know what information
> may be needed to be able to help me on the right track.  I hope the
> below problem description is appropriate.

Although you've described in detail what you did, it isn't clear to me
what the problem is...

> The situation is as follows:
>
> I plan to use git and git svn to track a module in the middle of a SVN
> repository (which I have no control over) where I don't have read
> access more than to this specific module and the base directory, ie
> the layout is similar to this
> 	https://svn.foo.bar/a/b/c/trunk
> 	https://svn.foo.bar/a/b/c/tags
> 	https://svn.foo.bar/a/b/c/branches
> and I have read access to https://svm.foo.bar/a,
> https://svn.foo.bar/a/b/c and below, but nothing else.
>
> What I want to do is clone the whole thing, including tags and
> branches.  I expected to be able to use the following command to do
> this (git v1.5.2):
>
> $ git svn clone https://svn.foo.bar/a/b/c -T trunk -t tags -b branches
>
> Initialized empty Git repository in .git/
> Using higher level of URL: https://svn.foo.bar/a/b/c => https://svn.foo.bar/a
>
> W: Ignoring error from SVN, path probably does not exist: (175002): RA
> layer request failed: REPORT request failed on '/a/!svn/bc/100':
> REPORT of '/a/!svn/bc/100': Could not read chunk size: Secure
> connection truncated (https://svn.foo.bar)
> r306 = 83f0c10b988a8f1e77a3f354126c52f0cfdecf76 (trunk)
> [...]
> Found possible branch point: https://svn.foo.bar/a/b/c/trunk =>
> https://svn.foo.bar/a/b/c/tags/0.1, 314
> Found branch parent: (tags/0.1) ad6a2361d0e69d6288ef226bb335bb4bf3bdd12e
> Following parent with do_update
> Successfully followed parent
> r315 = 3b92c4885b9d6f60241533dd99fa5023eebb1c64 (tags/0.1)
> r316 = 02e3b828b4f11c5fab9d10d85076a8ff209afa00 (trunk)
> [...]
>
> However, it seems I just get the info about the revisions (shows up
> nicely in gitk) but no content at all.  The config file reads:
>
> [core]
>        repositoryformatversion = 0
>        filemode = true
>        bare = false
>        logallrefupdates = true
> [svn-remote "svn"]
>        url = https://svn.foo.bar/a/b/c/trunk
>        fetch = :refs/remotes/git-svn
>
>
> If I instead do:
>
> $ git svn clone https://svn.foo.bar/a/b/c/trunk
> Initialized empty Git repository in .git/
> W: Ignoring error from SVN, path probably does not exist: (175007):
> HTTP Path Not Found: REPORT request failed on
> '/a/!svn/bc/100/b/c/trunk': '/a/!svn/bc/100/b/c/trunk' path not found
> W: +empty_dir: source
> W: +empty_dir: test
> r306 = 83f0c10b988a8f1e77a3f354126c52f0cfdecf76 (git-svn)
>        A       resources/extensions/circuitparts.drext
> [...]
> r309 = 3a386e9b985c419d129461acbf24978795b36b96 (git-svn)
>        A       tools/source/physical.cc
> [...]
>
> I get all content in trunk and their revision history, but as expected
> no tag info.  The config file reads:
>
> [core]
>        repositoryformatversion = 0
>        filemode = true
>        bare = false
>        logallrefupdates = true
> [svn-remote "svn"]
>        url = https://svn.foo.bar/a
>        fetch = b/c/trunk:refs/remotes/trunk
>        branches = b/c/branches/*:refs/remotes/*
>        tags = b/c/tags/*:refs/remotes/tags/*

That looks like what you want.  Isn't it?

With that config, I would expect:

git svn fetch
   pull updates for trunk and all branches and tags from the svn
   server.

git checkout -b trunk remotes/git-svn
   checkout the current trunk

git svn rebase
   fetch and rebase trunk or current branch.  git svn should determine
   the right git branch from refs/remotes to rebase against.

Does that help any?

+ seth

-- 
Seth Falcon | Computational Biology | Fred Hutchinson Cancer Research Center
http://bioconductor.org

Fwd: Problem using git svn clone

[Gustaf Hendeby]

Apparently gmail does some HTMLing on reply unless you watch out, so
this didn't reach the list.

/Gustaf

---------- Forwarded message ----------
From: Gustaf Hendeby <hendeby@gmail.com>
Date: May 25, 2007 12:39 AM
Subject: Re: Problem using git svn clone
To: Seth Falcon <sethfalcon@gmail.com>
Cc: git@vger.kernel.org


On 5/24/07, Seth Falcon <sethfalcon@gmail.com> wrote:
> "Gustaf Hendeby" <hendeby@gmail.com> writes:
>
> > I have a problem with git-svn, which I hope the list can shed some
> > light over.  I'm quite new to git (but so far I really like it, great
> > job guys!) and I never use SVN, so don't really know what information
> > may be needed to be able to help me on the right track.  I hope the
> > below problem description is appropriate.
>
> Although you've described in detail what you did, it isn't clear to me
> what the problem is...

Let me clarify. When trying to check out the module with tags, the
first example I only get the revisions, no content at all!  I have
added some more info on this below.

The second case works as I expected it, I included it as reference,
and to show that it worked (probably not the smartest thing to do I
guess).  Hence, probably just the first example is of interest.

> > The situation is as follows:
> >
> > I plan to use git and git svn to track a module in the middle of a SVN
> > repository (which I have no control over) where I don't have read
> > access more than to this specific module and the base directory, ie
> > the layout is similar to this
> >       https://svn.foo.bar/a/b/c/trunk
> >       https://svn.foo.bar/a/b/c/tags
> >       https://svn.foo.bar/a/b/c/branches
> > and I have read access to https://svm.foo.bar/a,
> > https://svn.foo.bar/a/b/c and below, but nothing else.
> >
> > What I want to do is clone the whole thing, including tags and
> > branches.  I expected to be able to use the following command to do
> > this (git v1.5.2):
> >
> > $ git svn clone https://svn.foo.bar/a/b/c -T trunk -t tags -b branches
> >
> > Initialized empty Git repository in .git/
> > Using higher level of URL: https://svn.foo.bar/a/b/c => https://svn.foo.bar/a
> >
> > W: Ignoring error from SVN, path probably does not exist: (175002): RA
> > layer request failed: REPORT request failed on '/a/!svn/bc/100':
> > REPORT of '/a/!svn/bc/100': Could not read chunk size: Secure
> > connection truncated (https://svn.foo.bar)
> > r306 = 83f0c10b988a8f1e77a3f354126c52f0cfdecf76 (trunk)
> > [...]
> > Found possible branch point: https://svn.foo.bar/a/b/c/trunk =>
> > https://svn.foo.bar/a/b/c/tags/0.1 , 314
> > Found branch parent: (tags/0.1) ad6a2361d0e69d6288ef226bb335bb4bf3bdd12e
> > Following parent with do_update
> > Successfully followed parent
> > r315 = 3b92c4885b9d6f60241533dd99fa5023eebb1c64 (tags/0.1)
> > r316 = 02e3b828b4f11c5fab9d10d85076a8ff209afa00 (trunk)
> > [...]
> >
> > However, it seems I just get the info about the revisions (shows up
> > nicely in gitk) but no content at all.  The config file reads:

I'm really sorry, I mixed up the config files. :(  So I actually get
this for the first example not the second:

[core]
        repositoryformatversion = 0
        filemode = true
         bare = false
        logallrefupdates = true
[svn-remote "svn"]
        url = https://foo.bar/a
        fetch = b/c/trunk:refs/remotes/trunk
        branches = b/c/branches/*:refs/remotes/*
        tags = b/c/tags/*:refs/remotes/tags/*

Which looks good I guess...  But to be a bit more clear about what
this actually gives me in the directory I just cloned to, and this is
the problem:

$ git checkout -f master
Already on branch "master"
$ ls -a
.  ..  .git
$ du -hs .git/objects
186K    .git/objects

There should be hundreds of files here, several MB of code.  So
something has gone seriously wrong here --- or at least did not work
the way I expected it to.  The .git/objects in the second example
contains 5.7 MB data. More stats:

$ git rev-list --all | wc -l
99
$ git tag -l
$ git branch -a
* master
  tags/0.1
  tags/0.1.1
  tags/0.2.0
  tags/0.2.1
  trunk

No tags (I mistook some of the branches for tags in the first post),
but some branches all containing no files, though, still claiming to
be clean...

> >
> >
> > If I instead do:
> >
> > $ git svn clone https://svn.foo.bar/a/b/c/trunk
> > I get all content in trunk and their revision history, but as expected

> That looks like what you want.  Isn't it?

Second example works just as expected.

I assume the listed commands below are assumed to work with the second
example?  They doesn't seem to do any good to my first example.

> With that config, I would expect:
>
> git svn fetch
>    pull updates for trunk and all branches and tags from the svn
>    server.
>
> git checkout -b trunk remotes/git-svn
>    checkout the current trunk
>
> git svn rebase
>    fetch and rebase trunk or current branch.  git svn should determine
>    the right git branch from refs/remotes to rebase against.
>
> Does that help any?

Given the bad info I posted you did a great job, I really didn't
intend to waste the lists time. Thanks!  Though, my problem still
remains.  Given the new information, do you get any other ideas on
what I do wrong?

Re: Problem using git svn clone

[Seth Falcon]

"Gustaf Hendeby" <hendeby@gmail.com> writes:
> Let me clarify. When trying to check out the module with tags, the first
> example I only get the revisions, no content at all!  I have added some more
> info on this below.

If you see the output of the revisions being pulled from svn this
seems very odd.  Does a 'du -sh' in the created directory confirm no
data is there?  Are you sure it isn't just a matter of nothing getting
checked out for you?

> The second case works as I expected it, I included it as reference, and to
> show that it worked (probably not the smartest thing to do I guess).  Hence,
> probably just the first example is of interest.
>
>> The situation is as follows:
>> >
>> > I plan to use git and git svn to track a module in the middle of a SVN
>> > repository (which I have no control over) where I don't have read
>> > access more than to this specific module and the base directory, ie
>> > the layout is similar to this
>> >       https://svn.foo.bar/a/b/c/trunk
>> >       https://svn.foo.bar/a/b/c/tags
>> >       https://svn.foo.bar/a/b/c/branches
>> > and I have read access to https://svm.foo.bar/a,
>> > https://svn.foo.bar/a/b/c and below, but nothing else.
>> >
>> > What I want to do is clone the whole thing, including tags and
>> > branches.  I expected to be able to use the following command to do
>> > this (git v1.5.2):
>> >
>> > $ git svn clone https://svn.foo.bar/a/b/c -T trunk -t tags -b branches
>> >
>> > Initialized empty Git repository in .git/
>> > Using higher level of URL: https://svn.foo.bar/a/b/c =>
>> https://svn.foo.bar/a
>> >
>> > W: Ignoring error from SVN, path probably does not exist: (175002): RA
>> > layer request failed: REPORT request failed on '/a/!svn/bc/100':
>> > REPORT of '/a/!svn/bc/100': Could not read chunk size: Secure
>> > connection truncated (https://svn.foo.bar)
>> > r306 = 83f0c10b988a8f1e77a3f354126c52f0cfdecf76 (trunk)
>> > [...]
>> > Found possible branch point: https://svn.foo.bar/a/b/c/trunk =>
>> > https://svn.foo.bar/a/b/c/tags/0.1, 314
>> > Found branch parent: (tags/0.1) ad6a2361d0e69d6288ef226bb335bb4bf3bdd12e
>> > Following parent with do_update
>> > Successfully followed parent
>> > r315 = 3b92c4885b9d6f60241533dd99fa5023eebb1c64 (tags/0.1)
>> > r316 = 02e3b828b4f11c5fab9d10d85076a8ff209afa00 (trunk)
>> > [...]
>> >
>> > However, it seems I just get the info about the revisions (shows up
>> > nicely in gitk) but no content at all.  The config file reads:
>
>
> I'm really sorry, I mixed up the config files. :(  So I actually get this
> for the first example not the second:
>
> [core]
>        repositoryformatversion = 0
>        filemode = true
>        bare = false
>        logallrefupdates = true
> [svn-remote "svn"]
>        url = https://foo.bar/a
>        fetch = b/c/trunk:refs/remotes/trunk
>        branches = b/c/branches/*:refs/remotes/*
>        tags = b/c/tags/*:refs/remotes/tags/*
>
> Which looks good I guess...  But to be a bit more clear about what this
> actually gives me in the directory I just cloned to, and this is the
> problem:
>
> $ git checkout -f master

What's in .git/refs/remotes?

Does 'git checkout -b mytrunk refs/remotes/git-svn' work?

> Already on branch "master"
> $ ls -a
> .  ..  .git
> $ du -hs .git/objects
> 186K    .git/objects

A du of the entire .git dir would be more informative.

+ seth

Re: Problem using git svn clone

[Gustaf Hendeby]

On 5/25/07, Seth Falcon <sethfalcon@gmail.com> wrote:
> "Gustaf Hendeby" <hendeby@gmail.com> writes:
> > Let me clarify. When trying to check out the module with tags, the first
> > example I only get the revisions, no content at all!  I have added some more
> > info on this below.
>
> If you see the output of the revisions being pulled from svn this
> seems very odd.  Does a 'du -sh' in the created directory confirm no
> data is there?  Are you sure it isn't just a matter of nothing getting
> checked out for you?

The following result is the same directly after the clone attempt and
after a git checkout -f master:

$ du -hs
357K    .
$ ls -a
 .  ..  .git

If it is just not checked out I don't know what I'm doing wrong trying
to check it out.

> > The second case works as I expected it, I included it as reference, and to
> > show that it worked (probably not the smartest thing to do I guess).  Hence,
> > probably just the first example is of interest.
> >
> >> The situation is as follows:
> >> >
> >> > I plan to use git and git svn to track a module in the middle of a SVN
> >> > repository (which I have no control over) where I don't have read
> >> > access more than to this specific module and the base directory, ie
> >> > the layout is similar to this
> >> >       https://svn.foo.bar/a/b/c/trunk
> >> >       https://svn.foo.bar/a/b/c/tags
> >> >       https://svn.foo.bar/a/b/c/branches
> >> > and I have read access to https://svm.foo.bar/a,
> >> > https://svn.foo.bar/a/b/c and below, but nothing else.
> >> >
> >> > What I want to do is clone the whole thing, including tags and
> >> > branches.  I expected to be able to use the following command to do
> >> > this (git v1.5.2):
> >> >
> >> > $ git svn clone https://svn.foo.bar/a/b/c -T trunk -t tags -b branches
> >> >
> >> > Initialized empty Git repository in .git/
> >> > Using higher level of URL: https://svn.foo.bar/a/b/c =>
> >> https://svn.foo.bar/a
> >> >
> >> > W: Ignoring error from SVN, path probably does not exist: (175002): RA
> >> > layer request failed: REPORT request failed on '/a/!svn/bc/100':
> >> > REPORT of '/a/!svn/bc/100': Could not read chunk size: Secure
> >> > connection truncated (https://svn.foo.bar)
> >> > r306 = 83f0c10b988a8f1e77a3f354126c52f0cfdecf76 (trunk)
> >> > [...]
> >> > Found possible branch point: https://svn.foo.bar/a/b/c/trunk =>
> >> > https://svn.foo.bar/a/b/c/tags/0.1, 314
> >> > Found branch parent: (tags/0.1) ad6a2361d0e69d6288ef226bb335bb4bf3bdd12e
> >> > Following parent with do_update
> >> > Successfully followed parent
> >> > r315 = 3b92c4885b9d6f60241533dd99fa5023eebb1c64 (tags/0.1)
> >> > r316 = 02e3b828b4f11c5fab9d10d85076a8ff209afa00 (trunk)
> >> > [...]
> >> >
> >> > However, it seems I just get the info about the revisions (shows up
> >> > nicely in gitk) but no content at all.  The config file reads:
> >
> >
> > I'm really sorry, I mixed up the config files. :(  So I actually get this
> > for the first example not the second:
> >
> > [core]
> >        repositoryformatversion = 0
> >        filemode = true
> >        bare = false
> >        logallrefupdates = true
> > [svn-remote "svn"]
> >        url = https://foo.bar/a
> >        fetch = b/c/trunk:refs/remotes/trunk
> >        branches = b/c/branches/*:refs/remotes/*
> >        tags = b/c/tags/*:refs/remotes/tags/*
> >
> > Which looks good I guess...  But to be a bit more clear about what this
> > actually gives me in the directory I just cloned to, and this is the
> > problem:
> >
> > $ git checkout -f master
>
> What's in .git/refs/remotes?

$ ls -R .git/refs/remotes/
.git/refs/remotes/:
tags  trunk

.git/refs/remotes/tags:
0.1  0.1.1  0.2.0  0.2.1

So, in some way there seems to be tag information, but git doesn't
seem to handle them as tags.

>
> Does 'git checkout -b mytrunk refs/remotes/git-svn' work?

$ git checkout -b mytrunk refs/remotes/git-svn
git checkout: updating paths is incompatible with switching branches/forcing
Did you intend to checkout 'refs/remotes/git-svn' which can not be
resolved as commit?

However
$ git checkout -b mytrunk refs/remotes/trunk
works but is without any content.

>
> > Already on branch "master"
> > $ ls -a
> > .  ..  .git
> > $ du -hs .git/objects
> > 186K    .git/objects
>
> A du of the entire .git dir would be more informative.

$ du -hs .git
356K    .git

For the checkout of the trunk, that works but don't obtain what I'm
looking for has:
$ du -hs .git
5.9M    .git
More stuff has been stored here.  The clone process here also
indicates lots of added and changed files, not just list SVN revision
numbers with matching SHA1s as in the case shown above.

Also, this may be totally natural, but I find the warning that I get
at the start of the clone quite strange (from above):

W: Ignoring error from SVN, path probably does not exist: (175002): RA
layer request failed: REPORT request failed on '/a/!svn/bc/100':
REPORT of '/a/!svn/bc/100': Could not read chunk size: Secure
connection truncated (https://svn.foo.bar)

>From where comes the !svn/bc/100 ?  Is it supposed to be like that?
Also, it feels a bit dangerous to me to fall back to a position in the
SVN tree where there is a level that I don't have any access to
between the base and the module I want and have access to.

/Gustaf

Re: Problem using git svn clone

[Seth Falcon]

"Gustaf Hendeby" <hendeby@gmail.com> writes:

> On 5/25/07, Seth Falcon <sethfalcon@gmail.com> wrote:
>> "Gustaf Hendeby" <hendeby@gmail.com> writes:
>> > Let me clarify. When trying to check out the module with tags, the first
>> > example I only get the revisions, no content at all!  I have added some more
>> > info on this below.
>>
>> If you see the output of the revisions being pulled from svn this
>> seems very odd.  Does a 'du -sh' in the created directory confirm no
>> data is there?  Are you sure it isn't just a matter of nothing getting
>> checked out for you?
>
> The following result is the same directly after the clone attempt and
> after a git checkout -f master:
>
> $ du -hs
> 357K    .
> $ ls -a
> .  ..  .git
>
> If it is just not checked out I don't know what I'm doing wrong trying
> to check it out.

That does seem like either something is broken or not doing what you
expect (how's that for non-informative?).

I'm not sure I have enough expertise to help further.  If you care
about getting to the bottom of this, then I suspect you need to help
specify something that can be reproduced by others (which can be
difficult).

In the meantime, if I understood your comments, you did find a way to
clone the repos as you wanted just with a different command line...

+ seth

-- 
Seth Falcon | Computational Biology | Fred Hutchinson Cancer Research Center
http://bioconductor.org

Re: Problem using git svn clone

[Gustaf Hendeby]

On 5/25/07, Seth Falcon <sethfalcon@gmail.com> wrote:
> "Gustaf Hendeby" <hendeby@gmail.com> writes:
> > On 5/25/07, Seth Falcon <sethfalcon@gmail.com> wrote:
> >> "Gustaf Hendeby" <hendeby@gmail.com> writes:
> I'm not sure I have enough expertise to help further.  If you care
> about getting to the bottom of this, then I suspect you need to help
> specify something that can be reproduced by others (which can be
> difficult).

Ok fair enough, I am *very* *thankful* for your attempt to help me!

>
> In the meantime, if I understood your comments, you did find a way to
> clone the repos as you wanted just with a different command line...
Well, not really what I want since I lose all tags, but close enough
to be useful.   That will have do for time being.

One again, thanks!

/Gustaf