builtin command's prefix question

builtin command's prefix question

[Nguyen Thai Ngoc Duy]

Hi,

I have been looking at setup_git_directory_gently() lately. From my
understanding, setup_git_directory* will return a string called
"prefix" that is passed to builtin commands. What is the exact meaning
of this prefix? Correct me if I'm wrong. In early (read: no worktree)
days, cwd was moved to working root directory and prefix contained
relative path from working root directory to the original cwd. So it
had a few implications:
 1. A non-empty prefix indicates cwd has been moved
 2. If cwd is moved, it is moved to working root directory
 3. cwd+prefix should point to the current directory at the time the
command was issued (let's call it "user cwd")

Things change a bit since the rise of worktree:
 - If GIT_DIR is set and GIT_WORK_TREE is not, prefix is relative to
the to-be-set-up worktree, but cwd is not changed, so point 3 is gone.
 - If GIT_DIR is not set and GiT_WORK_TREE is,
  - and it is found that user cwd is inside a gitdir (bare repo), cwd
has been moved and prefix is empty, cwd+prefix no longer point to user
cwd
  - for other cases, cwd may not be worktree (the real worktree will
be setup in setup_work_tree or setup_git_directory)

Now that setup_work_tree can move cwd, it should also change prefix to
meet point 3. But it does not.

I feel dizzy now. Were my assumptions wrong? What is expected behavior
of setup_git_directory_gently()?
-- 
Duy

Re: builtin command's prefix question

[Junio C Hamano]

"Nguyen Thai Ngoc Duy" <pclouds@gmail.com> writes:

> I have been looking at setup_git_directory_gently() lately. From my
> understanding, setup_git_directory* will return a string called
> "prefix" that is passed to builtin commands. What is the exact meaning
> of this prefix?

Some historical background is in order.  For a long time, we only worked
from the very top of the work tree and nowhere else.  The path you get
from the user on the command line was supposed to be relative to the top
of the work tree, the user was supposed to be at the top of the work
tree, no work from subdirectories.

This was limiting, so "setup" was introduced.  The commands originally
worked only from the top would chdir up to the top of the work tree if
it was run from a subdirectory.  This however needs adjustments to the
paths we get from the user from the command line (or stdin for commands
that take such).  If we claim we work from a subdirectory, we should
interpret path given by the user who is in a subdirectory as relative to
that subdirectory.  The way to do this adjustment is to prefix the
subdirectory path to the path given by the user.

So, if you start a command from Documentation/ subdirectory, like this:

	$ cd Documentation
	$ git ls-files howto

internally, ls-files would:

 * Notice it was run from Documentation/ subdirectory;

 * cd up to the top level;

 * prefix "Documentation/" to the pathspec given by the user
   (i.e. "howto"), to form "Documentation/howto";

 * Act as before, except that it strips "Documentation/" from its usual
   output, to retain the illusion of working from that subdirectory.

And prefix is "Documentation/" (note the trailing slash) in such a
case.  If you run from the top, it is NULL to signal that there is no
need to do any of these tricks.

> ... Correct me if I'm wrong. In early (read: no worktree)
> days, cwd was moved to working root directory and prefix contained
> relative path from working root directory to the original cwd. So it
> had a few implications:

>  1. A non-empty prefix indicates cwd has been moved

Correct (I do not know if we care, though)

>  2. If cwd is moved, it is moved to working root directory

Correct ("we always work from top of the tree internally" matters)

>  3. cwd+prefix should point to the current directory at the time the
> command was issued (let's call it "user cwd")

Correct.

> Things change a bit since the rise of worktree:
>  - If GIT_DIR is set and GIT_WORK_TREE is not, prefix is relative to
> the to-be-set-up worktree, but cwd is not changed, so point 3 is gone.
>  - If GIT_DIR is not set and GiT_WORK_TREE is,
>   - and it is found that user cwd is inside a gitdir (bare repo), cwd
> has been moved and prefix is empty, cwd+prefix no longer point to user
> cwd
>   - for other cases, cwd may not be worktree (the real worktree will
> be setup in setup_work_tree or setup_git_directory)

The intention is:

 * If GIT_DIR is set but not GIT_WORK_TREE (nor core.worktree in
   config), you are either inside project.git/ directory of bare
   repository or at the toplevel of worktree-full directory.  This has
   been the traditional behaviour before GIT_WORK_TREE and we shouldn't
   break the existing setups that assume this behaviour.  So in that
   sense, with this combination:

   - If the repository is bare, the value of the prefix should not
     matter; the command that wants to look at prefix by definition
     wants to run from a subdirectory but there is no notion of
     "the user directory being a subdirectory of the top of the work
     tree" in a bare repository;

   - If the repository is not bare, the user directory _MUST_ be at the
     top of the work tree, as that is what the traditional behaviour is.
     Anything else would break existing setups.

     IOW, if you use GIT_DIR and still want to run from a subdirectory
     of the worktree, you must have either GIT_WORK_TREE or
     core.worktree to tell where the top of the worktree is, and if you
     don't, then you must be at the top.

   So the right thing to do in this case is not going anywhere and using
   prefix=NULL.

 * I would say it is a misconfiguration if GIT_DIR is not set and
   GIT_WORK_TREE is, as the sole purpose of GIT_WORK_TREE is so that you
   can work from a subdirectory when you set GIT_DIR.  I may be missing
   an obvious use case that this is useful, but I do not think of any.
   Dscho may be able to correct me on this, as he fixed up the original
   work tree series that was even messier quite a bit during the last
   round.

Re: builtin command's prefix question

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

>  * I would say it is a misconfiguration if GIT_DIR is not set and
>    GIT_WORK_TREE is, as the sole purpose of GIT_WORK_TREE is so that you
>    can work from a subdirectory when you set GIT_DIR.  I may be missing
>    an obvious use case that this is useful, but I do not think of any.
>    Dscho may be able to correct me on this, as he fixed up the original
>    work tree series that was even messier quite a bit during the last
>    round.

I had a short discussion with Dscho on this.  One scenario that was
brought up was this.

You have a work tree of mixed contents that logically belong to
separate repository.  Think $HOME/.?*, and tracking .vimrc and
.pinerc as separate "projects".  You have $HOME/gits/vim.git and
$HOME/gits/pine.git bare-looking repositories.

The "kosher" way of doing this might be:

        $ cd $HOME
        $ GIT_WORK_TREE=$HOME; export GIT_WORK_TREE
        $ edit .vimrc
        $ GIT_DIR=gits/vim.git git commit .vimrc
        $ edit .pinerc
        $ GIT_DIR=gits/pine.git git commit .pinerc

However, if we define setup() to behave this way when GIT_DIR is not
defined and GIT_WORK_TREE is:

 (1) internally pretend as if GIT_DIR was specified to be the
     directory where the command was started from (iow, do getcwd()
     once upon startup);

 (2) chdir to GIT_WORK_TREE (which means "callers of setup() always
     run from the top of the work tree");

 (3) set prefix to NULL;

Then this workflow becomes possible:

	$ cd $HOME
        $ GIT_WORK_TREE=$HOME; export GIT_WORK_TREE
	$ edit .vimrc .pinerc
        $ cd $HOME/gits/vimrc.git && git commit .vimrc
        $ cd $HOME/gits/pinerc.git && git commit .pinerc

I am not convinced this is giving any natural user experience, nor an
alternative:

	$ cd $HOME
        $ GIT_WORK_TREE=$HOME; export GIT_WORK_TREE
        $ cd $HOME/gits/vimrc.git
	$ edit $HOME/.vimrc
        $ git commit .vimrc
        $ cd $HOME/gits/pinerc.git
	$ edit $HOME/.pinerc
        $ git commit .pinerc

While I still think the combination is simply crazy and does not make
any sense, if enough users on the list agrees that it makes sense, I
wouldn't mind setup() did (1) to (3) mentioned above.  The alternative
is simply to declare GIT_WORK_TREE without GIT_DIR is a nonsense and
either error error out or ignore GIT_WORK_TREE, which might be easier to
explain to people.

Opinions?

Re: builtin command's prefix question

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> I am not convinced this is giving any natural user experience, nor an
> alternative:
>
>       $ cd $HOME
>       $ GIT_WORK_TREE=$HOME; export GIT_WORK_TREE
>       $ cd $HOME/gits/vimrc.git
>       $ edit $HOME/.vimrc
>       $ git commit .vimrc
>       $ cd $HOME/gits/pinerc.git
>       $ edit $HOME/.pinerc
>       $ git commit .pinerc
>
> While I still think the combination is simply crazy and does not make
> any sense, if enough users on the list agrees that it makes sense, I
> wouldn't mind setup() did (1) to (3) mentioned above.  The alternative
> is simply to declare GIT_WORK_TREE without GIT_DIR is a nonsense and
> either error error out or ignore GIT_WORK_TREE, which might be easier to
> explain to people.
>
> Opinions?

Side note.

By saying the above, I do not mean it is nonsense to try supporting a
work tree that is an overlay of disjoint set of work tree files from
multiple repositories/projects.  I do think it is a worthwhile goal to
support such a layout.

What I do not like is the way the ugly workaround does it, by
encouraging (rather, requiring) to issue git commands from a location
that is completely separate from the actual editing of the content
happens.

An independent issue of supporting such a overlayed work tree layout is
what to do with .gitignore files.  I think, especially with the recent
addition of --exclude-standard to ls-files and setup_standard_excludes()
in dir.c, we could have a per repository configuration that names the
per repository exclude files, so that .gitignore-vim and .gitignore-pine
can co-exist in $HOME, each excluding everything other than the
project's own files.

Re: builtin command's prefix question

[Nguyen Thai Ngoc Duy]

On Dec 6, 2007 5:12 AM, Junio C Hamano <gitster@pobox.com> wrote:
> The intention is:
>
>  * If GIT_DIR is set but not GIT_WORK_TREE (nor core.worktree in
>    config), you are either inside project.git/ directory of bare
>    repository or at the toplevel of worktree-full directory.  This has
>    been the traditional behaviour before GIT_WORK_TREE and we shouldn't
>    break the existing setups that assume this behaviour.  So in that
>    sense, with this combination:
>
>    - If the repository is bare, the value of the prefix should not
>      matter; the command that wants to look at prefix by definition
>      wants to run from a subdirectory but there is no notion of
>      "the user directory being a subdirectory of the top of the work
>      tree" in a bare repository;
>
>    - If the repository is not bare, the user directory _MUST_ be at the
>      top of the work tree, as that is what the traditional behaviour is.
>      Anything else would break existing setups.
>
>      IOW, if you use GIT_DIR and still want to run from a subdirectory
>      of the worktree, you must have either GIT_WORK_TREE or
>      core.worktree to tell where the top of the worktree is, and if you
>      don't, then you must be at the top.
>
>    So the right thing to do in this case is not going anywhere and using
>    prefix=NULL.

You are right. It is quite obvious from the code. No idea why I had
that in my mind.

>  * I would say it is a misconfiguration if GIT_DIR is not set and
>    GIT_WORK_TREE is, as the sole purpose of GIT_WORK_TREE is so that you
>    can work from a subdirectory when you set GIT_DIR.  I may be missing
>    an obvious use case that this is useful, but I do not think of any.
>    Dscho may be able to correct me on this, as he fixed up the original
>    work tree series that was even messier quite a bit during the last
>    round.

On Dec 6, 2007 6:22 AM, Junio C Hamano <gitster@pobox.com> wrote:
> However, if we define setup() to behave this way when GIT_DIR is not
> defined and GIT_WORK_TREE is:
>
>  (1) internally pretend as if GIT_DIR was specified to be the
>      directory where the command was started from (iow, do getcwd()
>      once upon startup);
>
>  (2) chdir to GIT_WORK_TREE (which means "callers of setup() always
>      run from the top of the work tree");
>
>  (3) set prefix to NULL;

(1) is fine by me, even if it goes up to find a gitdir. But (3), no,
prefix should be set as relative path from worktree top directory to
user current directory, not NULL.

> ...
> While I still think the combination is simply crazy and does not make
> any sense, if enough users on the list agrees that it makes sense, I
> wouldn't mind setup() did (1) to (3) mentioned above.  The alternative
> is simply to declare GIT_WOR+1 on whatever way that makes worktree less complicated. so your alternative ++K_TREE without GIT_DIR is a nonsense and
> either error error out or ignore GIT_WORK_TREE, which might be easier to
> explain to people.

I don't use either way you mentioned. So no comment here. But again,
no (3) please.
-- 
Duy

Re: builtin command's prefix question

[Johannes Schindelin]

Hi,

On Thu, 6 Dec 2007, Nguyen Thai Ngoc Duy wrote:

> On Dec 6, 2007 6:22 AM, Junio C Hamano <gitster@pobox.com> wrote:
> > However, if we define setup() to behave this way when GIT_DIR is not 
> > defined and GIT_WORK_TREE is:
> >
> >  (1) internally pretend as if GIT_DIR was specified to be the
> >      directory where the command was started from (iow, do getcwd()
> >      once upon startup);
> >
> >  (2) chdir to GIT_WORK_TREE (which means "callers of setup() always
> >      run from the top of the work tree");
> >
> >  (3) set prefix to NULL;
> 
> (1) is fine by me, even if it goes up to find a gitdir. But (3), no, 
> prefix should be set as relative path from worktree top directory to 
> user current directory, not NULL.

If you expect "git <command> <filespec>" to work correctly from GIT_DIR, 
you will _have_ to set the prefix to NULL.

Ciao,
Dscho

Re: builtin command's prefix question

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> On Thu, 6 Dec 2007, Nguyen Thai Ngoc Duy wrote:
>
>> On Dec 6, 2007 6:22 AM, Junio C Hamano <gitster@pobox.com> wrote:
>> > However, if we define setup() to behave this way when GIT_DIR is not 
>> > defined and GIT_WORK_TREE is:
>> >
>> >  (1) internally pretend as if GIT_DIR was specified to be the
>> >      directory where the command was started from (iow, do getcwd()
>> >      once upon startup);
>> >
>> >  (2) chdir to GIT_WORK_TREE (which means "callers of setup() always
>> >      run from the top of the work tree");
>> >
>> >  (3) set prefix to NULL;
>> 
>> (1) is fine by me, even if it goes up to find a gitdir. But (3), no, 
>> prefix should be set as relative path from worktree top directory to 
>> user current directory, not NULL.
>
> If you expect "git <command> <filespec>" to work correctly from GIT_DIR, 
> you will _have_ to set the prefix to NULL.

That depends on the definition of "correctly".

As I said, I think the above "rule-looking things" implement an insane
behaviour where you are in one directory, and use that <filespec> to
name things relative to some other directory whose location is
completely unrelated to the directory you are in.  IOW, not a good set
of rules, and I do not necessarily agree with the statement that says
such a behaviour is working "correctly from GIT_DIR".