BUG: "--work-tree blah" does not imply "--git-dir blah/.git" or fix misleading error message

BUG: "--work-tree blah" does not imply "--git-dir blah/.git" or fix misleading error message

[John Twilley]

Today someone asked me if there was a way to run git against a
directory other than the current directory.  I looked at the output of
--help and ran this:

$ git --work-tree blah status

I got the following output:

fatal: Not a git repository (or any parent up to mount parent /home)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).

I mistakenly thought the error message meant that blah was not a git
repository.  What it meant was that there was no .git in the current
directory or any parent directory up to /home.

This command worked as expected:

$ git --work-tree blah --git-dir blah/.git status

The documentation is somewhat fuzzy about what constitutes a git
repository.  The gittutorial describes the git repository as .git when
talking about "git init" while the Git User's Manual describes the git
repository as the working tree and the special top-level directory
named .git when talking about "git clone".

It's clear (to me at least) that --work-tree should be used to
identify the root of the working tree when not inside the working
tree.  I expected that the git directory would be automatically set to
.git in the root of the working tree, as that would match the
documentation.  Instead, the current directory and its parents were
checked -- which could provide dangerously misleading information to
the user.

I think that one of two things should be done:  either the --git-dir
default should be changed when the --work-tree option is set, or the
error message cited above should be changed to explicitly identify the
directory being tested as a potential git repository.  I personally
believe the first option is superior because it fulfills the
expectations of average users (folks who read git's documentation
instead of its source code) while permitting flexibility to those who
wish to refer to the current directory or some other directory for
their --git-dir value.  If the current behavior is somehow not a bug
but instead a critical and significant feature which if changed would
cause more harm than good, please consider the second option.

Jack.
--
mathuin at gmail dot com

Re: BUG: "--work-tree blah" does not imply "--git-dir blah/.git" or fix misleading error message

[Carlos Martín Nieto]

[-- Attachment #1: Type: text/plain, Size: 4175 bytes --]

On Wed, Nov 30, 2011 at 09:43:08AM -0800, John Twilley wrote:
> Today someone asked me if there was a way to run git against a
> directory other than the current directory.  I looked at the output of
> --help and ran this:
> 
> $ git --work-tree blah status
> 
> I got the following output:
> 
> fatal: Not a git repository (or any parent up to mount parent /home)
> Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
> 
> I mistakenly thought the error message meant that blah was not a git
> repository.  What it meant was that there was no .git in the current
> directory or any parent directory up to /home.

Yes, git looks at the current directory and .git/ to see if there's a
git repository there. This is what happens unless you tell git to look
for it somewhere else.

> 
> This command worked as expected:
> 
> $ git --work-tree blah --git-dir blah/.git status
> 
> The documentation is somewhat fuzzy about what constitutes a git
> repository.  The gittutorial describes the git repository as .git when
> talking about "git init" while the Git User's Manual describes the git
> repository as the working tree and the special top-level directory
> named .git when talking about "git clone".

A git repository is what's under .git/ (or in foo.git/ for bare
repos). Non-bare repositories have a working tree associated with
them, which by default lives just above the repository, because it'd
be silly to have it somewhere else by default. Often you can think of
both as the repository, as they're both. When you tell git to look for
the worktree somewhere else, you're only overriding that particular
variable, as git expects to be run from the repository (or just above,
in the worktree).

> 
> It's clear (to me at least) that --work-tree should be used to
> identify the root of the working tree when not inside the working
> tree.  I expected that the git directory would be automatically set to
> .git in the root of the working tree, as that would match the
> documentation.  Instead, the current directory and its parents were
> checked -- which could provide dangerously misleading information to
> the user.

What part of the documentation exactly? --work-tree tells git to look
for the working tree somewhere else. This option exists in order to
support a multiple-workdir workflow.

> 
> I think that one of two things should be done:  either the --git-dir
> default should be changed when the --work-tree option is set, or the
> error message cited above should be changed to explicitly identify the
> directory being tested as a potential git repository.  I personally

Git does tell you explicitly, but only when you specify a gitdir (via
GIT_DIR or --git-dir), otherwise it looks at the current directory.

> believe the first option is superior because it fulfills the
> expectations of average users (folks who read git's documentation
> instead of its source code) while permitting flexibility to those who

It's not likely that it will get changed because that would break
backwards-compatability in a very big way. If your concern is for
"average user", she shouldn't be using that option, but changing to
that directory instead. If you want your working tree to be ./foo/ and
your gitdir to be ./foo/.git, why don't you just cd to ./foo/?

> wish to refer to the current directory or some other directory for
> their --git-dir value.  If the current behavior is somehow not a bug
> but instead a critical and significant feature which if changed would
> cause more harm than good, please consider the second option.

You get two different messages depending on how git is looking for the
repository. The message you mentioned gets printed when git tries to
find it automatically. A "fatal: Not a git repository: '/tmp'" gets
printed if you've told git to look for it in a specific place. The
information is already there, though I guess you do have to know about
the difference. Adding the current directory to the "default" message
probably wouldn't hurt, as it's unlikely that a script is parsing
that, and might be useful.

   cmn

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: BUG: "--work-tree blah" does not imply "--git-dir blah/.git" or fix misleading error message

[John Twilley]

On Wed, Nov 30, 2011 at 10:22, Carlos Martín Nieto <cmn@elego.de> wrote:
> On Wed, Nov 30, 2011 at 09:43:08AM -0800, John Twilley wrote:
>> Today someone asked me if there was a way to run git against a
>> directory other than the current directory.  I looked at the output of
>> --help and ran this:
>>
>> $ git --work-tree blah status
>>
>> I got the following output:
>>
>> fatal: Not a git repository (or any parent up to mount parent /home)
>> Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
>>
>> I mistakenly thought the error message meant that blah was not a git
>> repository.  What it meant was that there was no .git in the current
>> directory or any parent directory up to /home.
>
> Yes, git looks at the current directory and .git/ to see if there's a
> git repository there. This is what happens unless you tell git to look
> for it somewhere else.

This makes perfect sense, because nearly every time I run git
commands, I am somewhere within the working tree.  The point of my
post was that I was using --work-tree to tell git to look for the git
repository somewhere else (the root of the specified working tree)
which is not what git expected.

>> This command worked as expected:
>>
>> $ git --work-tree blah --git-dir blah/.git status
>>
>> The documentation is somewhat fuzzy about what constitutes a git
>> repository.  The gittutorial describes the git repository as .git when
>> talking about "git init" while the Git User's Manual describes the git
>> repository as the working tree and the special top-level directory
>> named .git when talking about "git clone".
>
> A git repository is what's under .git/ (or in foo.git/ for bare
> repos). Non-bare repositories have a working tree associated with
> them, which by default lives just above the repository, because it'd
> be silly to have it somewhere else by default. Often you can think of
> both as the repository, as they're both. When you tell git to look for
> the worktree somewhere else, you're only overriding that particular
> variable, as git expects to be run from the repository (or just above,
> in the worktree).

And it's exactly this issue -- that sometimes the repository is just
the git directory, and sometimes the repository is the working tree
which contains the git directory at its root -- that caused the
confusion and unexpected behavior.  If I were to use a bare repository
directly (something I've never done), I guess I might use --git-dir
since the repository may not be named .git but instead something like
proj.git.  When I accessed a repository from outside its working tree,
I expected --work-tree to cover the whole shebang.  Obviously this
discussion is exposing my relatively limited experience with git, but
these assumptions do not seem unreasonable on their face.

>> It's clear (to me at least) that --work-tree should be used to
>> identify the root of the working tree when not inside the working
>> tree.  I expected that the git directory would be automatically set to
>> .git in the root of the working tree, as that would match the
>> documentation.  Instead, the current directory and its parents were
>> checked -- which could provide dangerously misleading information to
>> the user.
>
> What part of the documentation exactly? --work-tree tells git to look
> for the working tree somewhere else. This option exists in order to
> support a multiple-workdir workflow.

What you mention above was what I was thinking about when I mentioned
the possibility of this being a critical and significant feature.  If
it is important to support a workflow with one git directory and
multiple working trees, and that case is more common/important than
the one I experienced, then changing the behavior of --git-dir is
obviously not the right thing to do.

>> I think that one of two things should be done:  either the --git-dir
>> default should be changed when the --work-tree option is set, or the
>> error message cited above should be changed to explicitly identify the
>> directory being tested as a potential git repository.  I personally
>
> Git does tell you explicitly, but only when you specify a gitdir (via
> GIT_DIR or --git-dir), otherwise it looks at the current directory.

This is misleading if you don't know that the specification of a
working tree does not also implicitly specify a git directory.
Whether that lack of knowledge is the user's problem or the
software/documentation's problem is a separate question.

>> believe the first option is superior because it fulfills the
>> expectations of average users (folks who read git's documentation
>> instead of its source code) while permitting flexibility to those who
>
> It's not likely that it will get changed because that would break
> backwards-compatability in a very big way. If your concern is for
> "average user", she shouldn't be using that option, but changing to
> that directory instead. If you want your working tree to be ./foo/ and
> your gitdir to be ./foo/.git, why don't you just cd to ./foo/?

From that perspective, why have --work-tree at all?  Without that
option, either the git directory is in the root of your current
working tree, or it's not -- in which case --git-dir is all you need.
If you're going to keep the option, it's helpful to provide the
diagnostic output.  My suggestion would be more compelling if I could
provide a valid use case, but all I can come up with off the top of my
head are scripts and something like "(cd $worktree && git status)"
would probably work fine.

>> wish to refer to the current directory or some other directory for
>> their --git-dir value.  If the current behavior is somehow not a bug
>> but instead a critical and significant feature which if changed would
>> cause more harm than good, please consider the second option.
>
> You get two different messages depending on how git is looking for the
> repository. The message you mentioned gets printed when git tries to
> find it automatically. A "fatal: Not a git repository: '/tmp'" gets
> printed if you've told git to look for it in a specific place. The
> information is already there, though I guess you do have to know about
> the difference. Adding the current directory to the "default" message
> probably wouldn't hurt, as it's unlikely that a script is parsing
> that, and might be useful.

Fewer scripts would be broken if the additional output is only
displayed when --work-tree is used, but that might be too special-case
for this situation.

>   cmn

Jack.
--
mathuin at gmail dot com

Re: BUG: "--work-tree blah" does not imply "--git-dir blah/.git" or fix misleading error message

[Junio C Hamano]

John Twilley <mathuin@gmail.com> writes:

> Today someone asked me if there was a way to run git against a
> directory other than the current directory.  I looked at the output of
> --help and ran this:
>
> $ git --work-tree blah status
>
> I got the following output:
>
> fatal: Not a git repository (or any parent up to mount parent /home)

Yeah, that is a "this a use case that we didn't even intend to support,
and as a consequence we do a random thing" bug.

Originally, when GIT_DIR is set (from the environment, and then later we
added "git --git-dir=..." as another way to do so), Git always used the
current directory as the top of the working tree. There was no mechanism
for the user to say "No, I am not at the top level, but I am in a
subdirectory of the working tree. The top of working tree is there".  That
was the use case GIT_WORK_TREE (from the environment, and then later we
added "git --work-tree=..." as another way to do so) was introduced
for.

So in that sense, it is an unsupported mode of operation and it is not
surprising at all if Git did any random and meaningless things if you used
GIT_WORK_TREE without specifying GIT_DIR at all. In the same sense,
strictly speaking, setting GIT_WORK_TREE to somewhere that is not a parent
directory of the current directory (even if you set GIT_DIR) is also an
unsupported mode of operation.

When GIT_DIR is not set, I think we still run the normal GIT_DIR discovery
starting from the current working directory, and when we do not find one,
we would error out, as you saw. I am sympathetic that your particular case
might have resulted in a more pleasant user experience if the GIT_DIR
discovery started from the directory specified by GIT_WORK_TREE (i.e. the
subdirectory "blah/.git"), but I do not think this is likely to change, as
I suspect that people and scripts are relying on the current behaviour to
be able to do something like this:

    cd /pub/scm/git/git.git ;# this is a bare repository
    mkdir /var/tmp/git
    git --work-tree=/var/tmp/git checkout

to have a temporary checkout, and changing the GIT_DIR discovery logic
will break them, i.e. they now have to do:

    cd /pub/scm/git/git.git ;# this is a bare repository
    mkdir /var/tmp/git
    git --work-tree=/var/tmp/git --git-dir=$(pwd) checkout

or something. Instead, what you wanted to do is already supported by:

    (cd blah && git status)

so nothing is lost.

We could reword this:

> fatal: Not a git repository (or any parent up to mount parent /home)

to "fatal: /home/bar/baz (or any parent ...) is not a git repository" to
mention the current directory /home/bar/baz, but I am having a hard time
convincing myself that such a change is particularly good, because almost
always you know where you are (many people have it in their shell prompt).
Such a change makes the message longer to fit on a line without adding
much value.

Re: BUG: "--work-tree blah" does not imply "--git-dir blah/.git" or fix misleading error message

[Carlos Martín Nieto]

[-- Attachment #1: Type: text/plain, Size: 629 bytes --]

On Wed, Nov 30, 2011 at 11:21:29AM -0800, Junio C Hamano wrote:
> subdirectory "blah/.git"), but I do not think this is likely to change, as
> I suspect that people and scripts are relying on the current behaviour to
> be able to do something like this:
> 
>     cd /pub/scm/git/git.git ;# this is a bare repository
>     mkdir /var/tmp/git
>     git --work-tree=/var/tmp/git checkout
> 

This is in fact the way that many (or from what I can see the most
popular) tutorials for abusing git as a deployment system tell you to
run it (though more often than not setting GIT_WORKTREE in the
environment).

   cmn


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: BUG: "--work-tree blah" does not imply "--git-dir blah/.git" or fix misleading error message

[Junio C Hamano]

Carlos Martín Nieto <carlos@cmartin.tk> writes:

> On Wed, Nov 30, 2011 at 11:21:29AM -0800, Junio C Hamano wrote:
>> subdirectory "blah/.git"), but I do not think this is likely to change, as
>> I suspect that people and scripts are relying on the current behaviour to
>> be able to do something like this:
>> 
>>     cd /pub/scm/git/git.git ;# this is a bare repository
>>     mkdir /var/tmp/git
>>     git --work-tree=/var/tmp/git checkout
>
> This is in fact the way that many (or from what I can see the most
> popular) tutorials for abusing git as a deployment system tell you to
> run it (though more often than not setting GIT_WORKTREE in the
> environment).

Heh, *ab*using is a good description if they really mean to use it as a
deployment system. For one thing it won't do anything a proper deployment
should do if the target directory is not empty ;-)

Re: BUG: "--work-tree blah" does not imply "--git-dir blah/.git" or fix misleading error message

[Carlos Martín Nieto]

[-- Attachment #1: Type: text/plain, Size: 8595 bytes --]

On Wed, Nov 30, 2011 at 11:13:20AM -0800, John Twilley wrote:
> On Wed, Nov 30, 2011 at 10:22, Carlos Martín Nieto <cmn@elego.de> wrote:
> > On Wed, Nov 30, 2011 at 09:43:08AM -0800, John Twilley wrote:
> >> Today someone asked me if there was a way to run git against a
> >> directory other than the current directory.  I looked at the output of
> >> --help and ran this:
> >>
> >> $ git --work-tree blah status
> >>
> >> I got the following output:
> >>
> >> fatal: Not a git repository (or any parent up to mount parent /home)
> >> Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
> >>
> >> I mistakenly thought the error message meant that blah was not a git
> >> repository.  What it meant was that there was no .git in the current
> >> directory or any parent directory up to /home.
> >
> > Yes, git looks at the current directory and .git/ to see if there's a
> > git repository there. This is what happens unless you tell git to look
> > for it somewhere else.
> 
> This makes perfect sense, because nearly every time I run git
> commands, I am somewhere within the working tree.  The point of my
> post was that I was using --work-tree to tell git to look for the git
> repository somewhere else (the root of the specified working tree)
> which is not what git expected.

And as Junio said, git devs have never considered the case where you
want to run git from a directory, but have git look for the worktree +
repo in a different place. Since what you want to do is solved
trivially by cd'ing in a subshell "(cd blah && git status)" it's not
surprising that this hasn't come up.

> 
> >> This command worked as expected:
> >>
> >> $ git --work-tree blah --git-dir blah/.git status
> >>
> >> The documentation is somewhat fuzzy about what constitutes a git
> >> repository.  The gittutorial describes the git repository as .git when
> >> talking about "git init" while the Git User's Manual describes the git
> >> repository as the working tree and the special top-level directory
> >> named .git when talking about "git clone".
> >
> > A git repository is what's under .git/ (or in foo.git/ for bare
> > repos). Non-bare repositories have a working tree associated with
> > them, which by default lives just above the repository, because it'd
> > be silly to have it somewhere else by default. Often you can think of
> > both as the repository, as they're both. When you tell git to look for
> > the worktree somewhere else, you're only overriding that particular
> > variable, as git expects to be run from the repository (or just above,
> > in the worktree).
> 
> And it's exactly this issue -- that sometimes the repository is just
> the git directory, and sometimes the repository is the working tree
> which contains the git directory at its root -- that caused the
> confusion and unexpected behavior.  If I were to use a bare repository
> directly (something I've never done), I guess I might use --git-dir
> since the repository may not be named .git but instead something like
> proj.git.  When I accessed a repository from outside its working tree,
> I expected --work-tree to cover the whole shebang.  Obviously this
> discussion is exposing my relatively limited experience with git, but
> these assumptions do not seem unreasonable on their face.

You hardly ever need to use git directly on a bare repo, but when you
do, you can just it from inside the repo, but most of the time git
expects to be run from the place where the repo+worktree combination
is to be found. Anything else is a low-level configuration and is just
not expected to be used by people who aren't familiar with git's usage
of "worktree".

> 
> >> It's clear (to me at least) that --work-tree should be used to
> >> identify the root of the working tree when not inside the working
> >> tree.  I expected that the git directory would be automatically set to
> >> .git in the root of the working tree, as that would match the
> >> documentation.  Instead, the current directory and its parents were
> >> checked -- which could provide dangerously misleading information to
> >> the user.
> >
> > What part of the documentation exactly? --work-tree tells git to look
> > for the working tree somewhere else. This option exists in order to
> > support a multiple-workdir workflow.
> 
> What you mention above was what I was thinking about when I mentioned
> the possibility of this being a critical and significant feature.  If
> it is important to support a workflow with one git directory and
> multiple working trees, and that case is more common/important than
> the one I experienced, then changing the behavior of --git-dir is
> obviously not the right thing to do.

It's not just about it being a critical feature. Yes it would break
many scripts, but what you want to do is meant to be done with a
subshell and I fail to see why git should go out of its way to support
such a usercase.

> 
> >> I think that one of two things should be done:  either the --git-dir
> >> default should be changed when the --work-tree option is set, or the
> >> error message cited above should be changed to explicitly identify the
> >> directory being tested as a potential git repository.  I personally
> >
> > Git does tell you explicitly, but only when you specify a gitdir (via
> > GIT_DIR or --git-dir), otherwise it looks at the current directory.
> 
> This is misleading if you don't know that the specification of a
> working tree does not also implicitly specify a git directory.
> Whether that lack of knowledge is the user's problem or the
> software/documentation's problem is a separate question.

The documentation now thankfully doesn't talk about working copy
anymore, so it should be fairly consistent. The gitglossary definition
of worktree should also explain what a worktree is.

> 
> >> believe the first option is superior because it fulfills the
> >> expectations of average users (folks who read git's documentation
> >> instead of its source code) while permitting flexibility to those who
> >
> > It's not likely that it will get changed because that would break
> > backwards-compatability in a very big way. If your concern is for
> > "average user", she shouldn't be using that option, but changing to
> > that directory instead. If you want your working tree to be ./foo/ and
> > your gitdir to be ./foo/.git, why don't you just cd to ./foo/?
> 
> From that perspective, why have --work-tree at all?  Without that
> option, either the git directory is in the root of your current
> working tree, or it's not -- in which case --git-dir is all you need.

The reason for --work-tree (and the earlier GIT_WORKTREE) is to allow
you to temporarily set the worktree somewhere else. It's meant to tell
git to pretend $PWD is something else.

> If you're going to keep the option, it's helpful to provide the
> diagnostic output.  My suggestion would be more compelling if I could
> provide a valid use case, but all I can come up with off the top of my
> head are scripts and something like "(cd $worktree && git status)"
> would probably work fine.

This subshell method is exactly how everyone's always written this
kind of thing, and the way that you're supposed to (and I don't mean
for git, I mean for any tool). Adding the option explicitly to git
would introduce an further potential source of bugs and is unnecessary.

> 
> >> wish to refer to the current directory or some other directory
for
> >> their --git-dir value.  If the current behavior is somehow not a bug
> >> but instead a critical and significant feature which if changed would
> >> cause more harm than good, please consider the second option.
> >
> > You get two different messages depending on how git is looking for the
> > repository. The message you mentioned gets printed when git tries to
> > find it automatically. A "fatal: Not a git repository: '/tmp'" gets
> > printed if you've told git to look for it in a specific place. The
> > information is already there, though I guess you do have to know about
> > the difference. Adding the current directory to the "default" message
> > probably wouldn't hurt, as it's unlikely that a script is parsing
> > that, and might be useful.
> 
> Fewer scripts would be broken if the additional output is only
> displayed when --work-tree is used, but that might be too special-case
> for this situation.

That would connect unrelated code, I doubt it's reasonable.

   cmn

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]