`.git` symlink makes `git submodule add` fail

`.git` symlink makes `git submodule add` fail

[Vít Novotný]

Hi,

	$ uname -a
	Linux witiko 4.3.0-1-amd64 #1 SMP Debian 4.3.5-1 (2016-02-06) x86_64 GNU/Linux
	$ git --version
	git version 2.7.0

I tend to make `.git` a symlink, when I need to maintain several Git
repositories inside a single directory. This was always transparent to Git, but
problems arose, when working with submodules:

	$ mkdir -p .repos/repoA
	$ ln -s .repos/repoA .git
	$ git init
	Initialized empty Git repository in /tmp/tmp.q3hkme6nm4/.repos/repoA/
	$ git submodule add https://github.com/witiko/git-parallel
	Cloning into 'git-parallel'...
	remote: Counting objects: 212, done.
	remote: Compressing objects: 100% (16/16), done.
	remote: Total 212 (delta 6), reused 1 (delta 1), pack-reused 195
	Receiving objects: 100% (212/212), 64.20 KiB | 0 bytes/s, done.
	Resolving deltas: 100% (120/120), done.
	Checking connectivity... done.
	fatal: Could not chdir to '../../../git-parallel': No such file or directory
	Unable to checkout submodule 'git-parallel'

Is this a bug, or is the ability to symlink `.git` just a happy coincidence?

Best Regards,
Vít

Re: `.git` symlink makes `git submodule add` fail

[Junio C Hamano]

Vít Novotný <witiko@gmail.com> writes:

> Is this a bug, or is the ability to symlink `.git` just a happy coincidence?

It has never been supported.

Re: `.git` symlink makes `git submodule add` fail

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Vít Novotný <witiko@gmail.com> writes:
>
>> Is this a bug, or is the ability to symlink `.git` just a happy coincidence?
>
> It has never been supported.

Oops, hit "send" too early.

We have support for a "gitdir:" facility that would work even on a
filesystem that cannot do symlinks (see gitrepository-layout(5)),
and both the higher-level submodule Porcelain and the more recent
"worktree" experimental code do use it.

Re: `.git` symlink makes `git submodule add` fail

[Jeff King]

On Fri, Feb 12, 2016 at 10:19:38AM -0800, Junio C Hamano wrote:

> >> Is this a bug, or is the ability to symlink `.git` just a happy coincidence?
> >
> > It has never been supported.
> 
> Oops, hit "send" too early.
> 
> We have support for a "gitdir:" facility that would work even on a
> filesystem that cannot do symlinks (see gitrepository-layout(5)),
> and both the higher-level submodule Porcelain and the more recent
> "worktree" experimental code do use it.

And the way to convince git to make the link for you is with clone's
"--separate-git-dir" option.

-Peff

Re: `.git` symlink makes `git submodule add` fail

[Vít Novotný]

On Fri, Feb 12, 2016 at 10:19:38AM -0800, Junio C Hamano wrote:
> Junio C Hamano <gitster@pobox.com> writes:
> 
> > Vít Novotný <witiko@gmail.com> writes:
> >
> >> Is this a bug, or is the ability to symlink `.git` just a happy coincidence?
> >
> > It has never been supported.
> 
> Oops, hit "send" too early.
> 
> We have support for a "gitdir:" facility that would work even on a
> filesystem that cannot do symlinks (see gitrepository-layout(5)),
> and both the higher-level submodule Porcelain and the more recent
> "worktree" experimental code do use it.
> 

That sounds ideal, thank you for letting me know. Still, I believe that
well-behaved applications should be indifferent to symlinks, unless the
distinction is significant. Would there be any objections, if I
submitted a patch / series of patches that would add proper symlink
support?

Re: `.git` symlink makes `git submodule add` fail

[Vít Novotný]

On Fri, Feb 12, 2016 at 01:27:33PM -0500, Jeff King wrote:
> On Fri, Feb 12, 2016 at 10:19:38AM -0800, Junio C Hamano wrote:
> 
> > >> Is this a bug, or is the ability to symlink `.git` just a happy coincidence?
> > >
> > > It has never been supported.
> > 
> > Oops, hit "send" too early.
> > 
> > We have support for a "gitdir:" facility that would work even on a
> > filesystem that cannot do symlinks (see gitrepository-layout(5)),
> > and both the higher-level submodule Porcelain and the more recent
> > "worktree" experimental code do use it.
> 
> And the way to convince git to make the link for you is with clone's
> "--separate-git-dir" option.
> 
> -Peff

What's curious is that this doesn't really work either, so the issue
doesn't seem to be the lack of symlink support, but rather the lack of
willingness on the part of Git to resolve a path recursively.

The following works flawlessly:

  $ mkdir repos
  $ git init
  $ mv .git repos/repoA
  $ printf 'gitdir: repos/repoA\n' >.git
  $ git submodule add https://github.com/witiko/git-parallel

There is, however, a minor pain, which makes the repository unportable:

  $ cat git-parallel/.git
  gitdir: /tmp/foobar/repos/repoA/modules/git-parallel

When I change the gitdir symlink to a relative path, everything works
fine as long as I don't make Git go over two gitdir symlinks:

  $ sed -i 's#^/tmp/foobar#..#' git-parallel/.git
  $ git status
  [works]
  $ sed -i 's#repos/repoA#.git#' git-parallel/.git
  $ git status
  fatal: Not a git repository: git-parallel/../.git/modules/git-parallel

Clearly, Git resolves the first gitdir symlink:

  git-parallel/.git => git-parallel/../.git/modules/git-parallel

but refuses to resolve recursively:

  git-parallel/../.git/modules/git-parallel =>
  git-parallel/../repos/repoA/modules/git-parallel

Re: `.git` symlink makes `git submodule add` fail

[Jeff King]

On Sat, Feb 13, 2016 at 05:20:09AM +0100, Vít Novotný wrote:

> On Fri, Feb 12, 2016 at 01:27:33PM -0500, Jeff King wrote:
> > On Fri, Feb 12, 2016 at 10:19:38AM -0800, Junio C Hamano wrote:
> > 
> > > >> Is this a bug, or is the ability to symlink `.git` just a happy coincidence?
> > > >
> > > > It has never been supported.
> > > 
> > > Oops, hit "send" too early.
> > > 
> > > We have support for a "gitdir:" facility that would work even on a
> > > filesystem that cannot do symlinks (see gitrepository-layout(5)),
> > > and both the higher-level submodule Porcelain and the more recent
> > > "worktree" experimental code do use it.
> > 
> > And the way to convince git to make the link for you is with clone's
> > "--separate-git-dir" option.
> 
> What's curious is that this doesn't really work either, so the issue
> doesn't seem to be the lack of symlink support, but rather the lack of
> willingness on the part of Git to resolve a path recursively.

Right. The fundamental problem is that if git needs to construct a
related path, symlinks mean it cannot do so textually. I'm sure there
are spots where we use real_path() to get a canonical path for a
particular $GIT_DIR, but from your results, it's clear we don't do so
consistently.

So you can call that "git doesn't support symlinks", or you can call it
"git supports symlinks, but needs to do so more consistently". :) I am
not sure whether the existing uses of real_path() are there
intentionally to support the kind of path-munging that submodules do or
not. But either way, canonicalizing the paths before creating relative
ones is probably going to be the solution.

I say all of that without having really looked at the code. You asked
earlier if there would be any objects to a patch to implement better
symlink support. That is hard to answer without somebody digging in to
see what the fix would look like, and whether there would be any
tradeoffs.

-Peff