Problems with git fetch confusing foo and foo.git repos

Problems with git fetch confusing foo and foo.git repos

[Richard Purdie]

A while ago I reported a problem[1] where having:

/somewhere/foo
and
/somewhere/foo.git

as bare repositories and trying to clone them using alternates could
cause git to confuse them.

The "conclusion" was that I needed to do:

git clone -s -n /somewhere/foo/ x

to stop it looking at the .git version. Ok, fine. Ugly but I can live
with it and we added the workaround[2].

I've now discovered we only half solved the problem. Whilst the
alternates might get setup correctly, the branch names and revisions get
fetched from the .git version still.

It appears that even if you have a repository setup with an origin url
of "/somewhere/foo/", when you run git fetch origin -f  refs/*:refs/*,
it will look at foo.git if it exists.

The problem is the trailing slash is stripped off by the code in
git-fetch itself. It appears to have done this since it was converted
from a .sh function. It means it appears impossible to fetch the
branchnames/revisions from foo when foo.git exists.

I work with build systems that build complete linux systems and we're
running into failures caused by this. I really need pre-existing
versions of git to work so I can't even patch git to work around the
problem without significant cost. So far the only way I've figured out
to avoid this is to create a symlink to /somewhere/foo/ and then set url
to point at the symlink. This way I can prevent it from finding the
other directory.

I thought I'd mention this in the hopes git can be fixed to behave
better in this situation and perhaps I can drop the hacks I'm going to
have to add sometime in the future.

If anyone has any ideas for better workarounds I'd love to hear about
them...

[1] "Alternates corruption issue", 2012/1/31
[2] http://git.yoctoproject.org/cgit.cgi/poky/commit/bitbake/lib/bb/fetch2/git.py?id=64662290d3e7deb0b6093b3959c3f3eddb873893

Cheers,

Richard

Re: Problems with git fetch confusing foo and foo.git repos

[Richard Purdie]

On Sat, 2012-08-18 at 15:25 +0100, Richard Purdie wrote:
> A while ago I reported a problem[1] where having:
> 
> /somewhere/foo
> and
> /somewhere/foo.git
> 
> as bare repositories and trying to clone them using alternates could
> cause git to confuse them.
> 
> The "conclusion" was that I needed to do:
> 
> git clone -s -n /somewhere/foo/ x
> 
> to stop it looking at the .git version. Ok, fine. Ugly but I can live
> with it and we added the workaround[2].
> 
> I've now discovered we only half solved the problem. Whilst the
> alternates might get setup correctly, the branch names and revisions get
> fetched from the .git version still.
> 
> It appears that even if you have a repository setup with an origin url
> of "/somewhere/foo/", when you run git fetch origin -f  refs/*:refs/*,
> it will look at foo.git if it exists.
> 
> The problem is the trailing slash is stripped off by the code in
> git-fetch itself. It appears to have done this since it was converted
> from a .sh function. It means it appears impossible to fetch the
> branchnames/revisions from foo when foo.git exists.
> 
> I work with build systems that build complete linux systems and we're
> running into failures caused by this. I really need pre-existing
> versions of git to work so I can't even patch git to work around the
> problem without significant cost. So far the only way I've figured out
> to avoid this is to create a symlink to /somewhere/foo/ and then set url
> to point at the symlink. This way I can prevent it from finding the
> other directory.
> 
> I thought I'd mention this in the hopes git can be fixed to behave
> better in this situation and perhaps I can drop the hacks I'm going to
> have to add sometime in the future.

I'd add that I think the commit made for the original problem[1] has
fixed this scenario since it now will prefer foo over foo.git also in
the fetch case even if the / is removed from the url. My test machines
don't have that version yet though and I'm left with a problem where git
is older than 1.7.9.2. 

[1]http://git.kernel.org/?p=git/git.git;a=commit;h=b3256eb8b35937192e85725d0c2bcb422295790c

Cheers,

Richard

Re: Problems with git fetch confusing foo and foo.git repos

[Junio C Hamano]

Richard Purdie <richard.purdie@linuxfoundation.org> writes:

> I'd add that I think the commit made for the original problem[1] has
> fixed this scenario since it now will prefer foo over foo.git also in
> the fetch case even if the / is removed from the url.

OK.

As understand it, these "check various possibilities e.g. $name,
$name.git $name/.git" were never meant to be a way to encourage
users to have multiple repositories next to each other under
confusing names in the first place.  It was merely to allow users to
have one of them (some may prefer $name/ that is with working tree,
so we allow $name/.git to be discovered, while some may want to have
a bare repository at $name.git that is bare, so we also allow it to
be discovered).  The recent tweak was to favor the case where the
name asked explicitly by the user is matched first, and it does not
fundamentally change the intent of the basic design in any way.

Thanks for confirming that the tweak works well for you.

> My test machines
> don't have that version yet though and I'm left with a problem where git
> is older than 1.7.9.2. 

So what do you want to see happen next?

Re: Problems with git fetch confusing foo and foo.git repos

[Richard Purdie]

On Sat, 2012-08-18 at 13:33 -0700, Junio C Hamano wrote:
> Richard Purdie <richard.purdie@linuxfoundation.org> writes:
> 
> > I'd add that I think the commit made for the original problem[1] has
> > fixed this scenario since it now will prefer foo over foo.git also in
> > the fetch case even if the / is removed from the url.
> 
> OK.
> 
> As understand it, these "check various possibilities e.g. $name,
> $name.git $name/.git" were never meant to be a way to encourage
> users to have multiple repositories next to each other under
> confusing names in the first place.  It was merely to allow users to
> have one of them (some may prefer $name/ that is with working tree,
> so we allow $name/.git to be discovered, while some may want to have
> a bare repository at $name.git that is bare, so we also allow it to
> be discovered).  The recent tweak was to favor the case where the
> name asked explicitly by the user is matched first, and it does not
> fundamentally change the intent of the basic design in any way.
> 
> Thanks for confirming that the tweak works well for you.

I'm responsible for "generic" source fetching infrastructure and
unfortunately I haven't been able to prevent users ending up with
"repositories next to each other under confusing names" much as I might
like to. Users tend to manage to find all the corner cases in something
like this :)

> > My test machines
> > don't have that version yet though and I'm left with a problem where git
> > is older than 1.7.9.2. 
> 
> So what do you want to see happen next?

I was a bit confused earlier whether there was any remaining issue. With
the recent versions I've now confirmed there isn't and the bug is fixed
(which I really appreciate). Sorry for the noise.

My remaining question was whether there was any better way to work
around this in older versions of git. I've ended up implementing the
symlink solution I mentioned which whilst ugly, will hopefully put this
issue to rest for me.

(http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=a86bd422641ce083ba0cdb4efe2a4c307eb36f7e in case anyone cares)

Cheers,

Richard