Re: `git worktree list` when bare repository is named `.git`

Re: `git worktree list` when bare repository is named `.git`

[Caleb White]

[-- Attachment #1: Type: text/plain, Size: 2487 bytes --]

> It seems that `.git` is always stripped from the `git worktree list` output (including in `--porcelain` mode). This becomes relevant with bare repositories. Here is a bare repository functioning as expected:
> 
> $ mkdir bare1
> $ cd bare1
> $ git init --bare
> Initialized empty Git repository in /private/tmp/bare1/
> $ git worktree list
> /private/tmp/bare1  (bare)
> 
> But if we create a bare repository in a directory named `.git`, `git worktree list` displays the parent directory as the worktree, even though `git status` doesn't recognize it as a worktree:
> 
> $ mkdir -p bare2/.git
> $ cd bare2/.git
> $ git init --bare
> Initialized empty Git repository in /private/tmp/bare2/.git/
> $ git worktree list
> /private/tmp/bare2  (bare)
> $ cd /tmp/bare2
> $ git status
> fatal: this operation must be run in a work tree
> 
> However, Git _will_ recognize the parent directory (/tmp/bare2) as a Git repository for the purposes of commands like `git rev-parse --git-dir`. I suspect this can be fixed in `get_main_worktree` by only stripping a `.git` suffix from the path if the main worktree is not bare.

The behavior you are seeing is correct and expected (and I use bare
repositories with worktrees like this).

The actual bare repository can be the directory, in a .git directory,
or in any other directory (as long as it has a .git file that points
to the actual bare repository). The directory will still be considered a
bare repository. For example:

    mkdir -p repo/.bare
    cd repo/.bare
    Initialized empty Git repository in /private/tmp/repo/.bare/
    git init --bare
    cd ..
    echo 'gitdir: .bare' >.git
    git worktree list
    git worktree add master
    /private/tmp/repo  (bare)
    /private/tmp/repo/master

Bare repositories by definition do not have a working tree, and therefore it is
expected that `git status` fails (can only be executed in a worktree) while
other commands succeed.

For the purposes of `git worktree list`, the `.git` suffix **should not** be
shown. You should never try to use `git worktree list` to get the actual path
of the repository, the docs give the following warning:

    See gitrepository-layout[5] for more information. The rule of thumb is do
    not make any assumption about whether a path belongs to $GIT_DIR or
    $GIT_COMMON_DIR when you need to directly access something inside $GIT_DIR.
    Use git rev-parse --git-path to get the final path.

Best,

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 509 bytes --]

Re: `git worktree list` when bare repository is named `.git`

[Rebecca Turner]

> The actual bare repository can be the directory, in a .git directory,
> or in any other directory (as long as it has a .git file that points
> to the actual bare repository). The directory will still be considered a
> bare repository.

Perhaps your example got scrambled, but I can't quite reproduce it:

$ mkdir -p repo/.bare
$ cd repo/.bare
$ git init --bare
Initialized empty Git repository in /private/tmp/repo/.bare/
$ git worktree list
/private/tmp/repo/.bare  (bare)
$ cd ..
$ git worktree list
fatal: not a git repository (or any of the parent directories): .git
$ echo 'gitdir: .bare' >.git
$ git worktree list
/private/tmp/repo/.bare  (bare)

It seems like the $GIT_DIR is shown in the worktree list here, and when I add
a `.git` file pointing to `.bare` manually, that doesn't get listed. (Which I
suppose makes sense, because it's not a `git-worktree(1)` worktree, but still
seems a little bit odd?)

> For the purposes of `git worktree list`, the `.git` suffix **should not** be
> shown. You should never try to use `git worktree list` to get the actual path
> of the repository, the docs give the following warning:
>
>     See gitrepository-layout[5] for more information. The rule of thumb is do
>     not make any assumption about whether a path belongs to $GIT_DIR or
>     $GIT_COMMON_DIR when you need to directly access something inside $GIT_DIR.
>     Use git rev-parse --git-path to get the final path.

I understood this note to be talking about paths _within_ the `$GIT_DIR` or
`$GIT_COMMON_DIR` itself; I see no reason why `git worktree list` wouldn't list
a bare repository _consistently_ as either the `$GIT_DIR` or the parent of the
`$GIT_DIR`.

What I'd like to do is get the path of the worktree so that I can move it. `git
worktree list` gives me this information _except_ for bare repositories in
directories named `.git`. I'm happy to have a special case for this, but I'd
like to understand the principle here.

Maybe I'm just not supposed to name a bare repository `.git`? The
`gitrepository-layout(5)` page does seem to imply this is mutually exclusive
with bare repositories:

> A Git repository comes in two different flavours:
>
> •    a .git directory at the root of the working tree;
>
> •    a <project>.git directory that is a bare repository (i.e. without its
>      own working tree), that is typically used for exchanging histories with
>      others by pushing into it and fetching from it.

Thanks for your help!

Re: `git worktree list` when bare repository is named `.git`

[Caleb White]

On Thu Oct 10, 2024 at 11:59 PM CDT, Rebecca Turner wrote:
> Perhaps your example got scrambled, but I can't quite reproduce it:
>
> $ mkdir -p repo/.bare
> $ cd repo/.bare
> $ git init --bare
> Initialized empty Git repository in /private/tmp/repo/.bare/
> $ git worktree list
> /private/tmp/repo/.bare  (bare)
> $ cd ..
> $ git worktree list
> fatal: not a git repository (or any of the parent directories): .git
> $ echo 'gitdir: .bare' >.git
> $ git worktree list
> /private/tmp/repo/.bare  (bare)

> It seems like the $GIT_DIR is shown in the worktree list here, and when I add
> a `.git` file pointing to `.bare` manually, that doesn't get listed. (Which I
> suppose makes sense, because it's not a `git-worktree(1)` worktree, but still
> seems a little bit odd?)

Ah, I was mistaken. It will show the actual path if the gitdir points
to a separate directory.

> I understood this note to be talking about paths _within_ the `$GIT_DIR` or
> `$GIT_COMMON_DIR` itself; I see no reason why `git worktree list` wouldn't list
> a bare repository _consistently_ as either the `$GIT_DIR` or the parent of the
> `$GIT_DIR`.
>
> What I'd like to do is get the path of the worktree so that I can move it. `git
> worktree list` gives me this information _except_ for bare repositories in
> directories named `.git`. I'm happy to have a special case for this, but I'd
> like to understand the principle here.

Why would you move the `.git` directory? If you're trying to move the
repository, then wouldn't you just move the directory that contains the
`.git` directory?

I think the main reason why the `.git` path is trimmed is because it
doesn't make sense to show it in non-bare repositories. No one wants to
see the `.git` path in a normal repository.

    # global git
    git worktree list
    ~/sources/git  3f20f8dd05 [wt_relative_paths]

    # locally modified git
    ./git worktree list
    ~/sources/git/.git  3f20f8dd05 [wt_relative_paths]

I would rather not have the `.git` show even in bare repositories,
if a user has moved the bare repository to `.git`, then that would
indicate that the *intent* is for the parent directory to essentially
act as the repository (and be moved as a cohesive unit if moving).

> Maybe I'm just not supposed to name a bare repository `.git`? The
> `gitrepository-layout(5)` page does seem to imply this is mutually exclusive
> with bare repositories:
>
>> A Git repository comes in two different flavours:
>>
>> •    a .git directory at the root of the working tree;
>>
>> •    a <project>.git directory that is a bare repository (i.e. without its
>>      own working tree), that is typically used for exchanging histories with
>>      others by pushing into it and fetching from it.

There's nothing wrong with naming a bare repository `.git`. I've been
doing it for a while now and it works just fine.

I've become a big fan of using bare repositories with worktrees,
particularly in high-trafficked repositories where I'm constantly
switching between branches.

When I was first doing research on this, I found a ton of articles with
all kinds of different ways to do it. Some folks put their worktrees in
the same directory as the actual repository (intermixed with their
code), some polluted the parent directory, some created a detached
commit that removed all files from the default working tree and then
created the worktrees, some used a bare repository but then just created
the worktrees in the same directory, etc. I finally came across an
article that showed the `.bare` method above and I thought that was the
cleanest method. However, after using it for a while, I realized that
I could just move `.bare` to `.git` and it would work just fine (and
I could remove an extra file). I've been using that method ever since.

Best,

Re: `git worktree list` when bare repository is named `.git`

[Rebecca Turner]

> Why would you move the `.git` directory? If you're trying to move the
> repository, then wouldn't you just move the directory that contains the
> `.git` directory?

Ah, I should give some context here. I'm using worktrees with the layout you
describe later in your email:

    my-repo/
      .git/      <- bare git directory
      main/      <- worktree for main branch
      feature1/  <- worktree for feature work
      ...

I'm writing a tool to manage these layouts for you. I want to provide two
features:

1. The ability to add a new worktree in a slightly more magical manner; in
   particular, I want to be able to do `git my-tool add feature2` and add a new
   worktree in the same directory as all the other worktrees.

   For a non-bare main worktree, that directory is the parent of the main
   worktree.

   For a bare main worktree named `.git`, it's the path of the main
   worktree. (Nothing in the `git worktree list` output indicates this is the
   case!)

   For other bare worktrees, it's the parent of the main worktree.

2. The ability to convert an existing repository in this layout.

   This requires separating the `$GIT_DIR` from the worktree and then
   reassociating them, in order to convert the non-bare main worktree into a
   bare main worktree and a second linked worktree. (In particular, I'd like to
   avoid the cost of copying all the files in a large checkout.)

> I think the main reason why the `.git` path is trimmed is because it
> doesn't make sense to show it in non-bare repositories. No one wants to
> see the `.git` path in a normal repository.
>
>     # global git
>     git worktree list
>     ~/sources/git  3f20f8dd05 [wt_relative_paths]
>
>     # locally modified git
>     ./git worktree list
>     ~/sources/git/.git  3f20f8dd05 [wt_relative_paths]

I definitely agree with this!

> I would rather not have the `.git` show even in bare repositories,
> if a user has moved the bare repository to `.git`, then that would
> indicate that the *intent* is for the parent directory to essentially
> act as the repository (and be moved as a cohesive unit if moving).

I suppose that makes sense? Perhaps this is an area where the documentation
needs some additional notes.

> When I was first doing research on this, I found a ton of articles with
> all kinds of different ways to do it. Some folks put their worktrees in
> the same directory as the actual repository (intermixed with their
> code), some polluted the parent directory, some created a detached
> commit that removed all files from the default working tree and then
> created the worktrees, some used a bare repository but then just created
> the worktrees in the same directory, etc. I finally came across an
> article that showed the `.bare` method above and I thought that was the
> cleanest method. However, after using it for a while, I realized that
> I could just move `.bare` to `.git` and it would work just fine (and
> I could remove an extra file). I've been using that method ever since.

Yes, exactly! My frustration with this technique is how difficult it is to use.
I have existing checkouts I'd like to convert to worktree repositories, and
`git clone --bare` doesn't create remote-tracking branches, so it's strangely
difficult to set up repositories like this. I'm hoping to ease some of this
with my new tool.

Thanks again for your help,
-- Rebecca

Re: `git worktree list` when bare repository is named `.git`

[Caleb White]

On Fri Oct 11, 2024 at 11:19 AM CDT, Rebecca Turner wrote:
> Ah, I should give some context here. I'm using worktrees with the layout you
> describe later in your email:
>
>     my-repo/
>       .git/      <- bare git directory
>       main/      <- worktree for main branch
>       feature1/  <- worktree for feature work
>       ...
>
> I'm writing a tool to manage these layouts for you. I want to provide two
> features:
>
> 1. The ability to add a new worktree in a slightly more magical manner; in
>    particular, I want to be able to do `git my-tool add feature2` and add a new
>    worktree in the same directory as all the other worktrees.

If your repository is already set up in the layout you describe, you can
just execute `git worktree add` (I have this aliased to `g w add`, I'm a
lazy typer haha) in the `my-repo` directory.

>    For a non-bare main worktree, that directory is the parent of the main
>    worktree.

I would *not* do this, you may just want to not support this case. I
imagine most folks have a common directory for all their repositories,
and polluting the parent directory with worktrees sounds like a bad idea.

>
>    For a bare main worktree named `.git`, it's the path of the main
>    worktree. (Nothing in the `git worktree list` output indicates this is the
>    case!)
>
>    For other bare worktrees, it's the parent of the main worktree.

Note that you can use `rev-parse` to get the actual directory:

    git rev-parse --absolute-git-dir
    ~/sources/bare-repo/.git

> 2. The ability to convert an existing repository in this layout.
>
>    This requires separating the `$GIT_DIR` from the worktree and then
>    reassociating them, in order to convert the non-bare main worktree into a
>    bare main worktree and a second linked worktree. (In particular, I'd like to
>    avoid the cost of copying all the files in a large checkout.)

To convert an existing repository to this layout, all you should have to
do is:
- Add `bare = true` to the `[core]` section of the `.git/config` file
- Remove everything except the `.git` directory
- Create a new worktree for the default branch
- Profit!

>> When I was first doing research on this, I found a ton of articles with
>> all kinds of different ways to do it. Some folks put their worktrees in
>> the same directory as the actual repository (intermixed with their
>> code), some polluted the parent directory, some created a detached
>> commit that removed all files from the default working tree and then
>> created the worktrees, some used a bare repository but then just created
>> the worktrees in the same directory, etc. I finally came across an
>> article that showed the `.bare` method above and I thought that was the
>> cleanest method. However, after using it for a while, I realized that
>> I could just move `.bare` to `.git` and it would work just fine (and
>> I could remove an extra file). I've been using that method ever since.
>
> Yes, exactly! My frustration with this technique is how difficult it is to use.
> I have existing checkouts I'd like to convert to worktree repositories, and
> `git clone --bare` doesn't create remote-tracking branches, so it's strangely
> difficult to set up repositories like this. I'm hoping to ease some of this
> with my new tool.

I've created a helper script[1] that allows me to clone a bare repository
to use with worktrees. In case you don't know, any script in your PATH
that starts with `git-` can be called as `git <script-name>`. I then
have a git alias for this script and so I can run the following to set
everything up:

    git cloneb https://github.com/git/git

The script also allows setting up an upstream remote at the same time
in case you've forked the repository.

    git cloneb --remote=https://github.com/git/git https://github.com/<user>/git

[1]: https://github.com/calebdw/dotfiles/blob/master/scripts/git-clone-bare-for-worktrees

Best,

Re: `git worktree list` when bare repository is named `.git`

[Patrick Callahan]

On Fri, Oct 11, 2024 at 12:21 PM Rebecca Turner <rbt@fastmail.com> wrote:
>
> > Why would you move the `.git` directory? If you're trying to move the
> > repository, then wouldn't you just move the directory that contains the
> > `.git` directory?
>
> Ah, I should give some context here. I'm using worktrees with the layout you
> describe later in your email:
>
>     my-repo/
>       .git/      <- bare git directory
>       main/      <- worktree for main branch
>       feature1/  <- worktree for feature work
>       ...
>
> I'm writing a tool to manage these layouts for you. I want to provide two
> features:
>
> 1. The ability to add a new worktree in a slightly more magical manner; in
>    particular, I want to be able to do `git my-tool add feature2` and add a new
>    worktree in the same directory as all the other worktrees.
>
>    For a non-bare main worktree, that directory is the parent of the main
>    worktree.
>
>    For a bare main worktree named `.git`, it's the path of the main
>    worktree. (Nothing in the `git worktree list` output indicates this is the
>    case!)
>
>    For other bare worktrees, it's the parent of the main worktree.


 Rebecca,

I'm working on a tool to manage worktrees as well. Can we compare notes?
My directory structure separates repositories and worktrees in
separate directories, but the goals seem similar.

git documentation seems to treat the bare repository concept as if one
only uses bare repositories on a server, not locally in a development
environment.

But, I've found that local worktree-based development is possible for
multiple applications, libraries, toolchains, and CI, but it isn't
very easy to maintain by hand, so tooling is a must. With some
scripting, it works well for many bare repositories, numerous branch
worktrees, and multiple build-and-run scenarios. Switching between
tasks is almost instantaneous, with no need to stash or un-stash
anything.

I've managed to come up with a bash implementation with just three
commands for starting an editor/ide, building and running.  Parameters
to these commands set the context for whatever it is I'm working on at
the moment. I'm working on separate commands to maintain the
environment.  I need to do such things as clone an upstream repo,
clone

-Pat Callahan
 Framingham, Ma

Here's the list of requirements I'm working with:

Overview

- Use only bare repositories and worktrees
- Start an Ide, a build, a run for whatever I'm working on at the moment
- Support development work on multiple applications, libraries, or tools
- No limit to the number of working contexts.
- Instant focus on a specific context: a set of repositories,
git-references. and worktrees
- Instant switch between different working contexts within the same
application or between different applications.

Repositories

- Bare clones of a set of official and forked repositories from
Github, Gitlab, or Sourceforge
- Local Bare Repos Only
- All Repos in one directory as repo-name.git
- Worktrees as needed for building

Worktrees

- All Worktrees in another single directory as repo-name.git-reference
 (git reference being a branch, tag, or commit
- Worktree synchronization by git pull upstream, or git pull upstream --rebase
- Directories
- Hierarchy as flat as possible
- Automatic setup of out-of-tree builds using symbolic links to one or
more worktrees

IDE or Code Editor support

- Automatic setup of multiple multi-root workspaces based on the list
of worktrees used to build a specific branch.

Building and Running

- A default build script and custom build scripts where appropriate
- A default application run script with custom run scripts where appropriate
- Straightforward, flexible command line syntax
  Example: Build four separate cmake build types: Debug,
RelWithDebInfo for each of two branches

    b app-name branch-name1 branch-name-2 d rd r m

Re: `git worktree list` when bare repository is named `.git`

[Rebecca Turner]

Pat,

> I'm working on a tool to manage worktrees as well. Can we compare notes?
I've mostly finished my tool. Here's the code: https://github.com/9999years/git-prole

I'm not sure the repository layout my tool creates will match your expectations, but it'll be an interesting point of comparison. Enjoy!

Best,
-- Rebecca

`git worktree list` when bare repository is named `.git`

[Rebecca Turner]

It seems that `.git` is always stripped from the `git worktree list` output (including in `--porcelain` mode). This becomes relevant with bare repositories. Here is a bare repository functioning as expected:

$ mkdir bare1
$ cd bare1
$ git init --bare
Initialized empty Git repository in /private/tmp/bare1/
$ git worktree list
/private/tmp/bare1  (bare)

But if we create a bare repository in a directory named `.git`, `git worktree list` displays the parent directory as the worktree, even though `git status` doesn't recognize it as a worktree:

$ mkdir -p bare2/.git
$ cd bare2/.git
$ git init --bare
Initialized empty Git repository in /private/tmp/bare2/.git/
$ git worktree list
/private/tmp/bare2  (bare)
$ cd /tmp/bare2
$ git status
fatal: this operation must be run in a work tree

However, Git _will_ recognize the parent directory (/tmp/bare2) as a Git repository for the purposes of commands like `git rev-parse --git-dir`. I suspect this can be fixed in `get_main_worktree` by only stripping a `.git` suffix from the path if the main worktree is not bare.