Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Tao Klerks]

Hi folks,

I work with a project that often, or typically, benefits from having
multiple worktrees where users are working with different versions of
the code.

Worktrees make particular sense in this project for all the usual
reasons, plus the fact that it is a very *large* repo - multiple gigs
of repo, and very fast-moving; cloning and fetching less often is
nice.

One problem that we found users to have, early on when we introduced
worktrees, was that it was not obvious to users that there was (or why
there was) a "main" worktree, containing the actual ".git" repo, and
"subsidiary" worktrees, in the same directory location. Git by default
makes worktrees *subfolders* of the initial/main worktree/folder, but
we put them alongside each other to make other project tooling be able
to treat all worktrees consistently; in daily use there is absolutely
no difference between them, in all the "project tooling" there cannot
be a practical difference... but if you choose to delete a worktree,
and it happens to be the "special" one that contains the repo...
you've just lost stuff that you didn't expect to lose (any local
branches and stashes, basically).

Because worktree use was so useful/widespread/critical on this
project, and we already had a custom cloning process that introduced
selective refspecs etc, we introduced a special clone topology: the
initial clone is a bare repo, and that folder gets a specific clear
name (ending in .git). Then we create worktrees attached to that bare
repo.

Generally speaking, this has worked *very* well: I would recommend it
generally as a recognized/supported local-repo-setup. The most
important thing that makes this *possible* is the fact that "git
rev-parse --is-bare-repository" returns True in the bare repo folder,
where the lack of index and HEAD shouldn't bother git, and it returns
False in any one of the worktrees. It feels like things were designed
to work this way, even though I can find no explicit mention of this
topology in the docs.

However, from time to time something weird happens: Today I finally
started to understand why I was seeing a crazy GC error about bitmaps,
intermittently: It seems to be because "git gc" wants to create
bitmaps in bare repos, but can't do so properly when we're in a
partial clone... or something like that?

EG repro:
```
git clone --bare https://github.com/ksylor/ohshitgit dangit_shared.git
--filter=blob:none
git -C dangit_shared.git worktree add $PWD/dangit_wt3
cd dangit_wt3/
echo "this is some new unique blob in the repo" > new_blob.txt
git add new_blob.txt && git commit -m "new blob"
cd ../dangit_shared.git/
git gc
```

This yields, at the end of the GC run:
```
warning: Failed to write bitmap index. Packfile doesn't have full
closure (object bf86ed1b2602ac3a8d4724bcdf6707b156673aac is missing)
fatal: failed to write bitmap index
fatal: failed to run repack
```

On the other hand, running "git gc" in one of the worktrees works fine
(except you first need to delete a couple of ".tmp-*" files from the
"objects/pack" folder, if you already got the error above).

I at first thought this was a bug - but as I realized the problematic
behavior was tied to the "core.bare" setting (and its expression at
runtime through "git rev-parse --is-bare-repository"), it became more
obvious that maybe the system could/should be able to make assumptions
about the kind of repo that has this "true", and assume there are no
local-object-derived non-promisor packfiles (or whatever it is about
this example that makes things unhappy).

So, I guess I'm confused as to what "core.bare" is supposed to mean:
Is it intended to mean "there is no index nor HEAD here, and that's
good, don't worry" (in which case my setup is presumably "supported",
and the gc behavior is buggy?), or is it intended to mean "this is the
kind of repository in which there are no worktrees" (in which case I
am abusing the system and get the errors I deserve)?

CC Dscho, who made the worktree support that I rely on work about 16
years ago it seems, and Taylor Blau and Patrick Steinhardt, who I
think have made changes in the area of the code concerned with whether
or not we try to create bitmaps, and might have an opinion as to
whether the assumptions made by "gc" at the moment are unsafe, or my
use or "core.bare" in this context is wrong.

Thanks for any feedback!
Tao

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Tao Klerks]

On Mon, Sep 4, 2023 at 4:41 PM Tao Klerks <tao@klerks.biz> wrote:
>
> we introduced a special clone topology: the
> initial clone is a bare repo, and that folder gets a specific clear
> name (ending in .git). Then we create worktrees attached to that bare
> repo.
>
> Generally speaking, this has worked *very* well: I would recommend it
> generally as a recognized/supported local-repo-setup. The most
> important thing that makes this *possible* is the fact that "git
> rev-parse --is-bare-repository" returns True in the bare repo folder,
> where the lack of index and HEAD shouldn't bother git, and it returns
> False in any one of the worktrees. It feels like things were designed
> to work this way, even though I can find no explicit mention of this
> topology in the docs.

I should add that I only recently discovered "git clone
--separate-git-dir", which I at first though was a formal expression
of this setup... until I understood that the relationship between the
"GITDIR" and the worktree that you end up with is not "Bare repo vs
worktree", but rather... "orphaned repo / repo that doesn't know about
its worktree, vs worktree".

It seems, to me, that "my setup" makes a lot more sense than what you
end up with when you use "--separate-git-dir", and that the behavior
there predates the current "mutual reference" model of
worktrees-to-their-repo. If "my" use of "core.bare" in the example
above is sound - then should the implementation of
"--separate-git-dir" be changed to produce a bare repo with a
"worktrees" folder, like you get if you clone bare and add a worktree
in two separate steps?

(I say "change the implementation", but I guess I really mean
introducing a new option for the new behavior, and deprecate the old
option)

Dscho, I assume you would have the strongest opinion about this?

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Tao Klerks]

On Mon, Sep 4, 2023 at 4:59 PM Tao Klerks <tao@klerks.biz> wrote:
>
>
> It seems, to me, that "my setup" makes a lot more sense than what you
> end up with when you use "--separate-git-dir", and that the behavior
> there predates the current "mutual reference" model of
> worktrees-to-their-repo. If "my" use of "core.bare" in the example
> above is sound - then should the implementation of
> "--separate-git-dir" be changed to produce a bare repo with a
> "worktrees" folder, like you get if you clone bare and add a worktree
> in two separate steps?
>

And to confuse matters further, I just stumbled across
https://github.com/git/git/blob/master/contrib/workdir/git-new-workdir
- I don't understand when you would want to use that vs, again, a bare
repo with one or more worktrees properly attached via two-way
references, their own indexes, their own reflogs, etc.

Is it the case that this contrib script predates the current "git
worktree" support?

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Kristoffer Haugsbakk]

On Mon, Sep 4, 2023, at 17:29, Tao Klerks wrote:
> On Mon, Sep 4, 2023 at 4:59 PM Tao Klerks <tao@klerks.biz> wrote:
>>
>>
>> [snip]
>>
>
> And to confuse matters further, I just stumbled across
> https://github.com/git/git/blob/master/contrib/workdir/git-new-workdir
> - I don't understand when you would want to use that vs, again, a bare
> repo with one or more worktrees properly attached via two-way
> references, their own indexes, their own reflogs, etc.
>
> Is it the case that this contrib script predates the current "git
> worktree" support?

Yes, according to VonC[1] and the 2.05 release notes.[2]

🔗 1: https://stackoverflow.com/a/30185564/1725151
🔗 2: https://github.com/git/git/blob/22aca1b3ac10af7188dccf033b44a36926f04d4b/Documentation/RelNotes/2.5.0.txt#L25-L27

-- 
Kristoffer Haugsbakk

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Kristoffer Haugsbakk]

Hi Tao

Context for my own use: I use the default clone (named the same as the
upstream repository) as the main worktree and name worktrees according to
some topic. So if the repository is named `work-application` then I might
have worktrees named things like `deployment-work`, `next-version-work`,
and things like that. All of them sibling directories since they are all
Intellij projects (to your point about making tooling treat them the same
way). I usually use the main worktree so I am fine with one worktree being
*special* (that it contains the `.git` directory).

I can understand that the main worktree/linked worktree dichotomy might
feel artificial if you use, say, ten different wotrkees equally often. Or
maybe one worktree per branch.

And then from that vantage point it might feel wasteful to dedicate an
unused main worktree—with its own working tree—to just sit somewhere for
its `.git` directory, essentially.

On Mon, Sep 4, 2023, at 16:41, Tao Klerks wrote:
> Because worktree use was so useful/widespread/critical on this project,
> and we already had a custom cloning process that introduced selective
> refspecs etc, we introduced a special clone topology: the initial clone
> is a bare repo, and that folder gets a specific clear name (ending in
> .git). Then we create worktrees attached to that bare repo.

This is interesting as a Git user. I've been encountering questions on
StackOverflow where the questioner is using a bare repository which they
make (or try to make) worktrees from. I've been telling them that making
worktrees from a bare repository is a contradiction:[1]

> Bare repositories don’t have worktrees per definition. Or at least
> that’s what `man gitglossary says`. Of course what `git worktree` allows
> you to do trumps that. But it might be ill-defined.

The glossary says under “worktree” (on Git 2.42):

> A repository can have zero (i.e. bare repository) or one or more
> worktrees attached to it.

And as someone who never has needed to use a bare repository + worktrees
I've just left it at that.

🔗 1: https://stackoverflow.com/a/76273222/1725151

Cheers

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Eric Sunshine]

On Mon, Sep 4, 2023 at 1:57 PM Kristoffer Haugsbakk
<code@khaugsbakk.name> wrote:
> On Mon, Sep 4, 2023, at 16:41, Tao Klerks wrote:
> > Because worktree use was so useful/widespread/critical on this project,
> > and we already had a custom cloning process that introduced selective
> > refspecs etc, we introduced a special clone topology: the initial clone
> > is a bare repo, and that folder gets a specific clear name (ending in
> > .git). Then we create worktrees attached to that bare repo.
>
> This is interesting as a Git user. I've been encountering questions on
> StackOverflow where the questioner is using a bare repository which they
> make (or try to make) worktrees from. I've been telling them that making
> worktrees from a bare repository is a contradiction:[1]

Not at all. The combination of bare repository and multiple-worktrees
is legitimate and supported intentionally. (There are tests in the Git
test suite validating support of this feature.) For people who
regularly work with multiple worktrees, it is quite natural to have
all the worktrees hanging off a bare repository, each with equal
importance, rather than having a single "blessed" worktree which has
priority over all others.

> > Bare repositories don’t have worktrees per definition. Or at least
> > that’s what `man gitglossary says`. Of course what `git worktree` allows
> > you to do trumps that. But it might be ill-defined.
>
> The glossary says under “worktree” (on Git 2.42):
>
> > A repository can have zero (i.e. bare repository) or one or more
> > worktrees attached to it.

Speaking as a person involved in the implementation of worktrees,
including support for them in combination with bare repositories, my
reading of this is perhaps biased so that I understand its intent.
However, if I squint hard, I suppose I can see how you could read it
as meaning that a bare repository can't have any worktrees associated
with it. So, perhaps, the documentation could use a bit of touch up.

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Kristoffer Haugsbakk]

On Tue, Sep 5, 2023, at 02:38, Eric Sunshine wrote:
> Speaking as a person involved in the implementation of worktrees,
> including support for them in combination with bare repositories, my
> reading of this is perhaps biased so that I understand its intent.
> However, if I squint hard, I suppose I can see how you could read it
> as meaning that a bare repository can't have any worktrees associated
> with it. So, perhaps, the documentation could use a bit of touch up.

My interpretation of the documentation leads to contradictions. So I
thought of another one: A bare repository *can* have worktrees, but if it
does will in that case only have *linked* worktrees, since the
`repository.git` directory by definition does not have a working tree and
it therefore cannot be considered a worktree itself. By extension, a
linked worktree might be linked to a bare repository. Thus there is no
contradiction.

For example: there are four directory trees associated with the `repo`
repository:

1. `repo.git`: the directory tree with the bare repository; no worktree in
   *that* directory tree
2. `a`: worktree with a gitfile that points to `repo.git`
3. `b`: worktree with a gitfile that points to `repo.git`
4. `c`: worktree with a gitfile that points to `repo.git`

So you have:

• One repository
• Four directory trees
• Three worktrees

This interpretation seems completely in line with “bare repository” in the
glossary:

  “ A bare repository is normally an appropriately named directory with a
    .git suffix that does not have a locally checked-out copy of any of
    the files under revision control. That is, all of the Git
    administrative and control files that would normally be present in the
    hidden .git sub-directory are directly present in the repository.git
    directory instead, and no other files are present and checked
    out. Usually publishers of public repositories make bare repositories
    available.

But not with “worktree”:

  “ A repository can have zero (i.e. bare repository) or one or more
    worktrees attached to it. ...

Since this entry claims that “bare repository” and “zero worktrees” are
equivalent.

Nothwithstanding any implementation/documentation disagreement, I think
that this interpretation at least is coherent.

But note how (for me) it is a bit awkward to refer to a “bare repository”
in this context since I need to add “the directory tree” in order to
emphasize that we are talking about `repo.git`; normally you can kind of
loosely talk about “the repository” and still get the precise meaning that
you intend across, but in this case we have four directory trees which are
all *the same repository*. (Right?) So just saying “the bare repository”
can be misleading since it might hint that the three worktrees are not
part of the repository. (Perhaps there isn't enough nomenclature to
clearly talk about this particular case/setup?)

But with all of that in mind, perhaps the glossary could read something
like this instead (no reflowing):[1][2]

(`man git-worktree` might also need to be updated.)

† 1: Applied onto 1fc548b2d6 (The sixth batch, 2023-09-05)
🔗 2: https://github.com/git/git/compare/master...LemmingAvalanche:git:bare-and-worktrees?expand=1

Cheers

Kristoffer

-- >8 --
Subject: [PATCH] Try to reword what a worktree is

---
 Documentation/glossary-content.txt | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/Documentation/glossary-content.txt b/Documentation/glossary-content.txt
index 5a537268e2..5e192fb5dc 100644
--- a/Documentation/glossary-content.txt
+++ b/Documentation/glossary-content.txt
@@ -694,10 +694,14 @@ The most notable example is `HEAD`.
 	plus any local changes that you have made but not yet committed.

 [[def_worktree]]worktree::
-	A repository can have zero (i.e. bare repository) or one or
+	A repository can have zero or one or
 	more worktrees attached to it. One "worktree" consists of a
 	"working tree" and repository metadata, most of which are
 	shared among other worktrees of a single repository, and
 	some of which are maintained separately per worktree
 	(e.g. the index, HEAD and pseudorefs like MERGE_HEAD,
 	per-worktree refs and per-worktree configuration file).
++
+Note that the directory tree of a <<def_bare_repository,bare_repository>>
+may have linked worktrees, but cannot itself be a worktree since it has no
+working tree.
--
2.42.0

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Sergey Organov]

"Kristoffer Haugsbakk" <code@khaugsbakk.name> writes:


[...]

> -- >8 --
> Subject: [PATCH] Try to reword what a worktree is
>
> ---
>  Documentation/glossary-content.txt | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/glossary-content.txt b/Documentation/glossary-content.txt
> index 5a537268e2..5e192fb5dc 100644
> --- a/Documentation/glossary-content.txt
> +++ b/Documentation/glossary-content.txt
> @@ -694,10 +694,14 @@ The most notable example is `HEAD`.
>  	plus any local changes that you have made but not yet committed.
>
>  [[def_worktree]]worktree::
> -	A repository can have zero (i.e. bare repository) or one or
> +	A repository can have zero or one or
>  	more worktrees attached to it. One "worktree" consists of a
>  	"working tree" and repository metadata, most of which are
>  	shared among other worktrees of a single repository, and
>  	some of which are maintained separately per worktree
>  	(e.g. the index, HEAD and pseudorefs like MERGE_HEAD,
>  	per-worktree refs and per-worktree configuration file).
> ++
> +Note that the directory tree of a <<def_bare_repository,bare_repository>>
> +may have linked worktrees, but cannot itself be a worktree since it has no
> +working tree.

Reading this with a fresh eye, I wonder if we'd better distinguish
between "inline" worktree and "attached" worktrees?

As I see it, in fact a repository can have zero (i.e. bare repository)
or one inline worktree, as well as zero or more attached worktrees.

-- 
Sergey Organov

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Kristoffer Haugsbakk]

On Wed, Sep 6, 2023, at 18:39, Sergey Organov wrote:
>> -- >8 --
>> Subject: [PATCH] Try to reword what a worktree is
>>
>> ---
>>  Documentation/glossary-content.txt | 6 +++++-
>>  1 file changed, 5 insertions(+), 1 deletion(-)
>>
>> diff --git a/Documentation/glossary-content.txt b/Documentation/glossary-content.txt
>> index 5a537268e2..5e192fb5dc 100644
>> --- a/Documentation/glossary-content.txt
>> +++ b/Documentation/glossary-content.txt
>> @@ -694,10 +694,14 @@ The most notable example is `HEAD`.
>>  	plus any local changes that you have made but not yet committed.
>>
>>  [[def_worktree]]worktree::
>> -	A repository can have zero (i.e. bare repository) or one or
>> +	A repository can have zero or one or
>>  	more worktrees attached to it. One "worktree" consists of a
>>  	"working tree" and repository metadata, most of which are
>>  	shared among other worktrees of a single repository, and
>>  	some of which are maintained separately per worktree
>>  	(e.g. the index, HEAD and pseudorefs like MERGE_HEAD,
>>  	per-worktree refs and per-worktree configuration file).
>> ++
>> +Note that the directory tree of a <<def_bare_repository,bare_repository>>
>> +may have linked worktrees, but cannot itself be a worktree since it has no
>> +working tree.
>
> Reading this with a fresh eye, I wonder if we'd better distinguish
> between "inline" worktree and "attached" worktrees?
>
> As I see it, in fact a repository can have zero (i.e. bare repository)
> or one inline worktree, as well as zero or more attached worktrees.

Ah, thank you. I felt like the glossary/nomenclature was missing a few
words and these ones seem to fill things in nicely.

Now I'm just skeptical of the other wording issue about “bare repository”,
which might be somewhat out of place in the face of zero-to-multiple
worktrees. Going back to my example in the previous email:

• `repository.git` is a *bare repository* which has no *inline worktree*
  and three *attached worktrees* [I really like how inline/attached work
  here]
• `a` is an *attached worktree* of `repository.git`
• `a`, `b`, `c` are all the *worktrees* of the *bare repository*
  `repository.git` [“bare” here just emphasizes that `repository.git` does
  not have a worktree (“what about the worktree in `repository.git`?”)]

Does that sound right? (Asking no one in particular.) Personally I think
that it sounds more coherent than before I wrote it (than I thought it 
would).

-- 
Kristoffer Haugsbakk

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Tao Klerks]

On Wed, Sep 6, 2023 at 8:00 PM Kristoffer Haugsbakk
<code@khaugsbakk.name> wrote:
>
> On Wed, Sep 6, 2023, at 18:39, Sergey Organov wrote:
<SNIP>
> >
> > As I see it, in fact a repository can have zero (i.e. bare repository)
> > or one inline worktree, as well as zero or more attached worktrees.
>
> Ah, thank you. I felt like the glossary/nomenclature was missing a few
> words and these ones seem to fill things in nicely.
>

Yes, I agree!!

I like the nomenclature, I like the simple "zero (i.e. bare) or one
inline worktree, zero or more attached worktrees" explanation.

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Junio C Hamano]

Tao Klerks <tao@klerks.biz> writes:

> I like the nomenclature, I like the simple "zero (i.e. bare) or one
> inline worktree, zero or more attached worktrees" explanation.

We have used "main worktree" to refer to the working tree part (plus
the repository) of a non-bare repository.  And it makes sense to
explain it together with the concept of "worktree", as the primary
one is very much special in that it cannot be removed.  You can see
that "git worktree remove" would stop you from removing it with an
error message:

	fatal: '../there' is a main working tree.

It probably does not add much value to introduce a new term
"inline".  Here is what "git worktree --help" has to say about it.

    A repository has one main worktree (if it's not a bare repository) and
    zero or more linked worktrees.

I applaud whoever wrote this sentence for packing so much good
information in a concise and easy-to-understand description.

We can read that (1) a non-bare repository itself is considered
its "main worktree", (2) a bare repository, by inference, has no
main worktree (otherwise we wouldn't have said "if it's not"), and
(3) both bare and non-bare repositories can have linked worktrees
(again, otherwise we wouldn't have brought up a bare repository in
the description).

Perhaps we should borrow it to update the glossary, like so?


 Documentation/glossary-content.txt | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git c/Documentation/glossary-content.txt w/Documentation/glossary-content.txt
index 5a537268e2..d9ba3bab88 100644
--- c/Documentation/glossary-content.txt
+++ w/Documentation/glossary-content.txt
@@ -694,8 +694,8 @@ The most notable example is `HEAD`.
 	plus any local changes that you have made but not yet committed.
 
 [[def_worktree]]worktree::
-	A repository can have zero (i.e. bare repository) or one or
-	more worktrees attached to it. One "worktree" consists of a
+	A repository has one main worktree (if it's not a bare
+	repository) and zero or more linked worktrees.  One "worktree" consists of a
 	"working tree" and repository metadata, most of which are
 	shared among other worktrees of a single repository, and
 	some of which are maintained separately per worktree

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Sergey Organov]

Junio C Hamano <gitster@pobox.com> writes:

> Tao Klerks <tao@klerks.biz> writes:
>
>> I like the nomenclature, I like the simple "zero (i.e. bare) or one
>> inline worktree, zero or more attached worktrees" explanation.
>

[...]

> It probably does not add much value to introduce a new term
> "inline".  Here is what "git worktree --help" has to say about it.
>
>     A repository has one main worktree (if it's not a bare repository) and
>     zero or more linked worktrees.
>
> I applaud whoever wrote this sentence for packing so much good
> information in a concise and easy-to-understand description.

I agree "inline" is not much better than "main", nor "attached" is
better than "linked". I just pulled mine out of thin air, and what's
already there is probably fine. That said, to be picky, "main" suggests
that linked worktrees are somehow inferior. Are they?

-- 
Sergey Organov

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Junio C Hamano]

Sergey Organov <sorganov@gmail.com> writes:

> I agree "inline" is not much better than "main", nor "attached" is
> better than "linked". I just pulled mine out of thin air, and what's
> already there is probably fine.

Heh, the initial draft of my message you are responding to used
"primary" (and "attached"), because they are the word I am
accustomed to use (out of thin air) on the list a few times, before
checking with the existing documentation to realize that we use
"main" for that.

> That said, to be picky, "main" suggests
> that linked worktrees are somehow inferior. Are they?

I'd say that 'main' is different, not necessarily superiour, from
all others and they are equally useful and usable.  The difference
is that it cannot be removed.  There may be other differences I am
forgetting, but I do not think it is about which is superiour and
which is inferiour.

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Sergey Organov]

Junio C Hamano <gitster@pobox.com> writes:

> Sergey Organov <sorganov@gmail.com> writes:
>
>> I agree "inline" is not much better than "main", nor "attached" is
>> better than "linked". I just pulled mine out of thin air, and what's
>> already there is probably fine.
>
> Heh, the initial draft of my message you are responding to used
> "primary" (and "attached"), because they are the word I am
> accustomed to use (out of thin air) on the list a few times, before
> checking with the existing documentation to realize that we use
> "main" for that.
>
>> That said, to be picky, "main" suggests
>> that linked worktrees are somehow inferior. Are they?
>
> I'd say that 'main' is different, not necessarily superiour, from
> all others and they are equally useful and usable.  The difference
> is that it cannot be removed.  There may be other differences I am
> forgetting, but I do not think it is about which is superiour and
> which is inferiour.

Well, if worktree created by "git clone/init" is not superior compared
to that created by "git worktree", just different, then "main" might be
not the best choice, but then, provided it's already in use, it's
probably not that big deal either.

As a note, "primary" also suggests the rest are "secondary", and then
"primary" one might not be there in the first place, leaving us with a
set of "secondary" without "primary", that is a bit confusing.

"Embedded", "integrated", or even "default" come to mind as
alternatives. However, if "attached" is decided upon, "inline" just
follows naturally.

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Tao Klerks]

On Wed, Sep 6, 2023 at 10:26 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Tao Klerks <tao@klerks.biz> writes:
>
> > I like the nomenclature, I like the simple "zero (i.e. bare) or one
> > inline worktree, zero or more attached worktrees" explanation.
>
> We have used "main worktree" to refer to the working tree part (plus
> the repository) of a non-bare repository.  And it makes sense to
> explain it together with the concept of "worktree", as the primary
> one is very much special in that it cannot be removed.  You can see
> that "git worktree remove" would stop you from removing it with an
> error message:
>
>         fatal: '../there' is a main working tree.
>
> It probably does not add much value to introduce a new term
> "inline".  Here is what "git worktree --help" has to say about it.
>
>     A repository has one main worktree (if it's not a bare repository) and
>     zero or more linked worktrees.

I've definitely changed my mind about "inline", I agree "main" is
better. I'm not convinced it's the best word we could come up with,
but if it's well-established, I'm happy with it.

The problem I (now) see with "inline" is that it seems to imply a
spatial proximity that doesn't necessarily hold true, with
"--separate-git-dir" or other ways to separate the main worktree from
its usual "just above the .git directory" location. "Inline" is still
a reasonable qualification of the main worktree's *metadata* in that
situation (index, etc), but I think the word would not be sufficiently
clear/representative overall.

>
> I applaud whoever wrote this sentence for packing so much good
> information in a concise and easy-to-understand description.

I also like this sentence, it's basically equivalent to Sergey's sentence above.

>
> We can read that (1) a non-bare repository itself is considered
> its "main worktree", (2) a bare repository, by inference, has no
> main worktree (otherwise we wouldn't have said "if it's not"), and
> (3) both bare and non-bare repositories can have linked worktrees
> (again, otherwise we wouldn't have brought up a bare repository in
> the description).
>
> Perhaps we should borrow it to update the glossary, like so?
>

Looks good to me, but that leaves me with a different nitpick: we say
'One "worktree" consists of a "working tree" and repository metadata,
most of which are shared among other worktrees of a single repository,
and some of which are maintained separately per worktree'

This claims that the *shared metadata* (presumably the refs, the
branch reflogs, the objects, the config, etc) are *part of the
worktree* (a worktree "consists of" them and other things). That seems
like a very strange way to conceive of things, to me.

I would find it reasonable to state that the main worktree is part of
the repo - certainly that's now most everyday users would think of it,
if they were made to think of the worktree concept at all - but not
that the shared repo metadata is part of the worktree, and especially
not that the shared repo metadata is part of the attached worktrees.

I imagine that this weird phrasing intends to allude to the fact that
a worktree is "broken" without the repository metadata folder that
contains both its worktree-specific metadata and the shared metadata
that it depends just as much on... but can we come up with better
"relationship words here?

* A repository "has" zero or more worktrees
* If it "has" a "main" worktree it is not a bare repository, otherwise it is.
* It can have any number of "attached" worktrees

If a repo "has" these worktrees, is it in the sense that I "have" arms
and legs, and I "consist of" a person with arms and legs and other
body parts, or is it in the sense that I "have" a lifetime, opinions,
legal rights, and other things that I have as a consequence of being a
person, but are not "part of" me?

Similarly, do we have a term for "the directory that contains the
'refs' and 'objects' folders and stuff", regardless of whether it is
in fact the entire bare repository, typically with a name other than
".git", or it is nested in a main worktree in the usual fashion as
".git", or it is separated (and again, typically differently-named) in
a "--separate-git-dir" topology? I called it a "repository metadata
folder" above, but I'm not sure whether there is a correct, succinct
term for it.

Wrt to the "shared among other worktrees" bit specifically, I agree
with Sergey that "shared among all" would be clearer, but it's still
weird, because all of that shared metadata is "inherent to the repo"
beyond any and all worktrees. If this happens to be a bare repo, and
we remove all the attached worktrees, the metadata is still just as
meaningful - so saying that it was "shared among the worktrees", while
true, seems to be unnecessarily implying a smaller purpose/meaning
than appropriate.

Sorry to continue nitpicking - I would love to see a clear
nomenclature and description of these parts and their relationships
for people (with less git experience) to "get it" more easily.

>
>  Documentation/glossary-content.txt | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git c/Documentation/glossary-content.txt w/Documentation/glossary-content.txt
> index 5a537268e2..d9ba3bab88 100644
> --- c/Documentation/glossary-content.txt
> +++ w/Documentation/glossary-content.txt
> @@ -694,8 +694,8 @@ The most notable example is `HEAD`.
>         plus any local changes that you have made but not yet committed.
>
>  [[def_worktree]]worktree::
> -       A repository can have zero (i.e. bare repository) or one or
> -       more worktrees attached to it. One "worktree" consists of a
> +       A repository has one main worktree (if it's not a bare
> +       repository) and zero or more linked worktrees.  One "worktree" consists of a
>         "working tree" and repository metadata, most of which are
>         shared among other worktrees of a single repository, and
>         some of which are maintained separately per worktree

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Sergey Organov]

Tao Klerks <tao@klerks.biz> writes:

> On Wed, Sep 6, 2023 at 10:26 PM Junio C Hamano <gitster@pobox.com> wrote:
>>
>> Tao Klerks <tao@klerks.biz> writes:
>>
>> > I like the nomenclature, I like the simple "zero (i.e. bare) or one
>> > inline worktree, zero or more attached worktrees" explanation.
>>
>> We have used "main worktree" to refer to the working tree part (plus
>> the repository) of a non-bare repository.  And it makes sense to
>> explain it together with the concept of "worktree", as the primary
>> one is very much special in that it cannot be removed.  You can see
>> that "git worktree remove" would stop you from removing it with an
>> error message:
>>
>>         fatal: '../there' is a main working tree.
>>
>> It probably does not add much value to introduce a new term
>> "inline".  Here is what "git worktree --help" has to say about it.
>>
>>     A repository has one main worktree (if it's not a bare repository) and
>>     zero or more linked worktrees.
>
> I've definitely changed my mind about "inline", I agree "main" is
> better. I'm not convinced it's the best word we could come up with,
> but if it's well-established, I'm happy with it.
>
> The problem I (now) see with "inline" is that it seems to imply a
> spatial proximity that doesn't necessarily hold true, with
> "--separate-git-dir" or other ways to separate the main worktree from
> its usual "just above the .git directory" location. "Inline" is still
> a reasonable qualification of the main worktree's *metadata* in that
> situation (index, etc), but I think the word would not be sufficiently
> clear/representative overall.

It's not to argue in favor of "inline", just to clarify: I took it from
inline-vs-attached as used in e-mail, where "inline" means that you see
attachment right here, inline with the rest of text.

I also admit I didn't happen to consider --separate-git-dir at the
time.

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Kristoffer Haugsbakk]

Hi again Tao

On Thu, Sep 7, 2023, at 06:53, Tao Klerks wrote:
> I've definitely changed my mind about "inline", I agree "main" is
> better. I'm not convinced it's the best word we could come up with,
> but if it's well-established, I'm happy with it.

Note that the conversation was forked:

1. New nomenclature to describe things more precisely
2. How good those words in themselves are at describing these things

I liked “inline” since it helped to clarify the case of a bare repository
with linked worktrees. Sure, emphasizing that it has no “inline worktree”
might be redundant (it can be inferred, perhaps), it's nice to be able to
emphasize things for pedagogical purposes.

(I'm less sure if “attached” is needed.)

This is what we can say for certain about a repository:

• There definitely is a repository somewhere (maybe not in whatever
  worktree you are in right now though)
• It has zero or more worktrees

So “main worktree” is optional. And that optionality makes things awkward
since it also used to describe where the repository lives—even when the
repository has no *inline worktree* (it is bare).

The bottom line is that it's nice if one can avoid having to get into
situations like this made-up conversation:

A: — I'm in the deployment worktree now. Where's the main
  worktree in our workflow? [I don't know how to use `git worktree`]
B: — That's `repository.git`.
A: — Okay nice. Is that worktree used for the mainline development?
B: — No, it has no worktree. It's bare.
A: — What? But didn't you say that it was the main worktree?
B: — Yes, in the sense that it's where the repository is. But it has no
    worktree itself.

>> We can read that (1) a non-bare repository itself is considered
>> its "main worktree", (2) a bare repository, by inference, has no
>> main worktree (otherwise we wouldn't have said "if it's not"), and
>> (3) both bare and non-bare repositories can have linked worktrees
>> (again, otherwise we wouldn't have brought up a bare repository in
>> the description).
>>
>> Perhaps we should borrow it to update the glossary, like so?
>>
>
> Looks good to me, but that leaves me with a different nitpick: we say
> 'One "worktree" consists of a "working tree" and repository metadata,
> most of which are shared among other worktrees of a single repository,
> and some of which are maintained separately per worktree'
>
> This claims that the *shared metadata* (presumably the refs, the
> branch reflogs, the objects, the config, etc) are *part of the
> worktree* (a worktree "consists of" them and other things). That seems
> like a very strange way to conceive of things, to me.
>
> I would find it reasonable to state that the main worktree is part of
> the repo - certainly that's now most everyday users would think of it,
> if they were made to think of the worktree concept at all - but not
> that the shared repo metadata is part of the worktree, and especially
> not that the shared repo metadata is part of the attached worktrees.

Without getting into the subtle distinctions between is-a and has-a: I
think it could make sense to think of this in terms of which one needs the
other one. A worktree needs a repository, so one could say that a worktree
“consists of” that. A repository on the other hand doesn't need to have
any worktrees.

(But the vice-versa also makes sense.)

> I imagine that this weird phrasing intends to allude to the fact that
> a worktree is "broken" without the repository metadata folder that
> contains both its worktree-specific metadata and the shared metadata
> that it depends just as much on... but can we come up with better
> "relationship words here?

I don't see why one needs to phrase or define things in terms of what
would make it corrupted or not-that-thing any more. Like, a “working tree”
without a Git repository is just a directory tree—it's got nothing to do
with Git whatsoever.

> Sorry to continue nitpicking - I would love to see a clear
> nomenclature and description of these parts and their relationships
> for people (with less git experience) to "get it" more easily.

As a Git user I think this is a very productive topic.

Cheers

-- 
Kristoffer Haugsbakk

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Kristoffer Haugsbakk]

Hi again Junio

On Wed, Sep 6, 2023, at 22:26, Junio C Hamano wrote:
> Tao Klerks <tao@klerks.biz> writes:
>
>> I like the nomenclature, I like the simple "zero (i.e. bare) or one
>> inline worktree, zero or more attached worktrees" explanation.
>
> We have used "main worktree" to refer to the working tree part (plus
> the repository) of a non-bare repository.  And it makes sense to
> explain it together with the concept of "worktree", as the primary
> one is very much special in that it cannot be removed.  You can see
> that "git worktree remove" would stop you from removing it with an
> error message:
>
> 	fatal: '../there' is a main working tree.

This gives the same error if `there` is a bare repository. Is that
intended?

This goes back to my point about missing nomenclature: it's weird if the
“main working tree” can be a bare repository.

PS: Is it correct that the error message says “main working tree” instead
of “main worktree”? (See cc73385cf6 (worktree remove: new command,
2018-02-12.) I was thinking of spelunking the history further but thought
that I would quickly ask in case I'm missing something obvious.

> It probably does not add much value to introduce a new term
> "inline".

The reason that I like it is because it lets you describe a bare
repository with linked worktrees. Not because it would replace “main
worktree”.

Although in light of Sergey's post about inline/attached, the “main
worktree” term *might* start to look a bit anachronistic. But I'm not
sure.

> Here is what "git worktree --help" has to say about it.
>
>     A repository has one main worktree (if it's not a bare repository) and
>     zero or more linked worktrees.
>
> I applaud whoever wrote this sentence for packing so much good
> information in a concise and easy-to-understand description.

I agree that it is very elegant.

> Perhaps we should borrow it to update the glossary, like so?

Certainly. But although this looks like it completely describes everything
that you want, I still think it is good to explicitly mention something
like:

  “ Note that a bare repository may have ...

Since although this can certainly be inferred from the text, it's good to
have some redundancy when it comes to non-obvious cases.

Cheers

-- 
Kristoffer Haugsbakk

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Junio C Hamano]

"Kristoffer Haugsbakk" <code@khaugsbakk.name> writes:

>> 	fatal: '../there' is a main working tree.
>
> This gives the same error if `there` is a bare repository. Is that
> intended?

I do not think so.  The above is from "git worktree remove" and the
candidates to be removed is listed by "git worktree list".  I do not
know if it is sensible to include the bare repository that the
worktrees are attached to in the "list" output, and I do not think
it makes sense to accept the path to the directory that is such a
bare repository and let the code proceed that far.  It should just
reject it saying it is *not* a worktree.

> PS: Is it correct that the error message says “main working tree” instead
> of “main worktree”? (See cc73385cf6 (worktree remove: new command,
> 2018-02-12.) I was thinking of spelunking the history further but thought
> that I would quickly ask in case I'm missing something obvious.

My understanding is that "working tree" refers to what "git
checkout" would give you to your "make" and compilers.  The
"worktree" is a mechanism to allow you to have multiple "working
tree"s that are connected to a single repository (be it a bare or a
non-bare one).

> Certainly. But although this looks like it completely describes everything
> that you want, I still think it is good to explicitly mention something
> like:
>
>   “ Note that a bare repository may have ...
>
> Since although this can certainly be inferred from the text, it's good to
> have some redundancy when it comes to non-obvious cases.

That is fine.

I think I've already said everything that I think should be in the
final text, and I do not mind if there are anything extra for
helping new readers that may be more than absolute minimum.

Thanks.

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Junio C Hamano]

"Kristoffer Haugsbakk" <code@khaugsbakk.name> writes:

> But not with “worktree”:
>
>   “ A repository can have zero (i.e. bare repository) or one or more
>     worktrees attached to it. ...
>
> Since this entry claims that “bare repository” and “zero worktrees” are
> equivalent.

I wrote that "(i.e. bare repository)" in 2df5387e (glossary:
describe "worktree", 2022-02-09) but did not mean that way.  

A non-bare repository can reduce the number of its worktrees, but it
cannot go below one, because the directory with working tree files
and the .git/ subdirectory, i.e. its primary worktree, must exist
for it to be a non-bare repository.  Consequently a repository with
zero worktree is by definition a bare repository.

But that does not have to mean all bare repositories can have no
worktrees.

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Kristoffer Haugsbakk]

On Wed, Sep 6, 2023, at 19:52, Junio C Hamano wrote:
> I wrote that "(i.e. bare repository)" in 2df5387e (glossary:
> describe "worktree", 2022-02-09) but did not mean that way.
>
> A non-bare repository can reduce the number of its worktrees, but it
> cannot go below one, because the directory with working tree files
> and the .git/ subdirectory, i.e. its primary worktree, must exist
> for it to be a non-bare repository.  Consequently a repository with
> zero worktree is by definition a bare repository.
>
> But that does not have to mean all bare repositories can have no
> worktrees.

I see. Zero worktrees implies bare repository, but bare repository does
not imply zero worktrees. I got my logical connectives mixed up.

Thanks

-- 
Kristoffer Haugsbakk

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> "Kristoffer Haugsbakk" <code@khaugsbakk.name> writes:
>
>> But not with “worktree”:
>>
>>   “ A repository can have zero (i.e. bare repository) or one or more
>>     worktrees attached to it. ...
>>
>> Since this entry claims that “bare repository” and “zero worktrees” are
>> equivalent.
>
> I wrote that "(i.e. bare repository)" in 2df5387e (glossary:
> describe "worktree", 2022-02-09) but did not mean that way.  
>
> A non-bare repository can reduce the number of its worktrees, but it
> cannot go below one, because the directory with working tree files
> and the .git/ subdirectory, i.e. its primary worktree, must exist
> for it to be a non-bare repository.  Consequently a repository with
> zero worktree is by definition a bare repository.
>
> But that does not have to mean all bare repositories can have no
> worktrees.

I re-read the glossary entry and I think the current text is mostly
OK, except that it does not even have to mention "bare" at that
position in the sentence.  A bare repository with zero worktrees is
totally uninteresting in the explanation of the worktree.

We need to say that the repository data (configuration, refs and
objecs) are mostly shared among worktrees while some data are kept
per-worktree, which the current text adequately covers, and what is
missing with respect to a bare repository is that we do not say
worktrees can be attached after the fact to a repository that was
created bare.

So, perhaps something along this line?

 Documentation/glossary-content.txt | 16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)

diff --git c/Documentation/glossary-content.txt w/Documentation/glossary-content.txt
index 5a537268e2..6dba68ffc0 100644
--- c/Documentation/glossary-content.txt
+++ w/Documentation/glossary-content.txt
@@ -694,10 +694,12 @@ The most notable example is `HEAD`.
 	plus any local changes that you have made but not yet committed.
 
 [[def_worktree]]worktree::
-	A repository can have zero (i.e. bare repository) or one or
-	more worktrees attached to it. One "worktree" consists of a
-	"working tree" and repository metadata, most of which are
-	shared among other worktrees of a single repository, and
-	some of which are maintained separately per worktree
-	(e.g. the index, HEAD and pseudorefs like MERGE_HEAD,
-	per-worktree refs and per-worktree configuration file).
+	A repository can have zero or more worktrees attached to it.
+	One "worktree" consists of a "working tree" and repository
+	metadata, most of which are shared among other worktrees of
+	a single repository, and some of which are maintained
+	separately per worktree (e.g. the index, HEAD and pseudorefs
+	like MERGE_HEAD, per-worktree refs and per-worktree
+	configuration file).
++
+Note that worktrees can be attached to an existing bare repository.

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Sergey Organov]

Junio C Hamano <gitster@pobox.com> writes:

> Junio C Hamano <gitster@pobox.com> writes:
>
>> "Kristoffer Haugsbakk" <code@khaugsbakk.name> writes:
>>
>>> But not with “worktree”:
>>>
>>>   “ A repository can have zero (i.e. bare repository) or one or more
>>>     worktrees attached to it. ...
>>>
>>> Since this entry claims that “bare repository” and “zero worktrees” are
>>> equivalent.
>>
>> I wrote that "(i.e. bare repository)" in 2df5387e (glossary:
>> describe "worktree", 2022-02-09) but did not mean that way.  
>>
>> A non-bare repository can reduce the number of its worktrees, but it
>> cannot go below one, because the directory with working tree files
>> and the .git/ subdirectory, i.e. its primary worktree, must exist
>> for it to be a non-bare repository.  Consequently a repository with
>> zero worktree is by definition a bare repository.
>>
>> But that does not have to mean all bare repositories can have no
>> worktrees.
>
> I re-read the glossary entry and I think the current text is mostly
> OK, except that it does not even have to mention "bare" at that
> position in the sentence.  A bare repository with zero worktrees is
> totally uninteresting in the explanation of the worktree.

Sounds reasonable.

>
> We need to say that the repository data (configuration, refs and
> objecs) are mostly shared among worktrees while some data are kept
> per-worktree, which the current text adequately covers, and what is
> missing with respect to a bare repository is that we do not say
> worktrees can be attached after the fact to a repository that was
> created bare.

Why? Worktree could be attached after the fact to any repository. I
don't see why we need to mention bareness here, as it's not special in
this regard.

>
> So, perhaps something along this line?
>
>  Documentation/glossary-content.txt | 16 +++++++++-------
>  1 file changed, 9 insertions(+), 7 deletions(-)
>
> diff --git c/Documentation/glossary-content.txt w/Documentation/glossary-content.txt
> index 5a537268e2..6dba68ffc0 100644
> --- c/Documentation/glossary-content.txt
> +++ w/Documentation/glossary-content.txt
> @@ -694,10 +694,12 @@ The most notable example is `HEAD`.
>  	plus any local changes that you have made but not yet committed.
>  
>  [[def_worktree]]worktree::
> -	A repository can have zero (i.e. bare repository) or one or
> -	more worktrees attached to it. One "worktree" consists of a
> -	"working tree" and repository metadata, most of which are
> -	shared among other worktrees of a single repository, and
> -	some of which are maintained separately per worktree
> -	(e.g. the index, HEAD and pseudorefs like MERGE_HEAD,
> -	per-worktree refs and per-worktree configuration file).
> +	A repository can have zero or more worktrees attached to it.
> +	One "worktree" consists of a "working tree" and repository
> +	metadata, most of which are shared among other worktrees of
> +	a single repository, and some of which are maintained
> +	separately per worktree (e.g. the index, HEAD and pseudorefs
> +	like MERGE_HEAD, per-worktree refs and per-worktree
> +	configuration file).
> ++
> +Note that worktrees can be attached to an existing bare repository.

"shared among other worktrees" -> "shared among all worktrees"?

Also, if we do have "main worktree" and "linked worktree" as concepts,
they need to be at least mentioned in the glossary, I believe.

Finally, if we do have "linked worktrees", then the phrasing should
better use "linked" instead of "attached"? Alternatively, if "attached"
fits better, let's call them "attached worktrees"?

-- 
Sergey Organov

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Tao Klerks]

On Mon, Sep 4, 2023 at 7:57 PM Kristoffer Haugsbakk
<code@khaugsbakk.name> wrote:
>
> And then from that vantage point it might feel wasteful to dedicate an
> unused main worktree—with its own working tree—to just sit somewhere for
> its `.git` directory, essentially.

Yep, especially in my case where a worktree contains 200,000 files and
5GB; especially on Windows, that's a high tax to pay pointlessly.

>
> The glossary says under “worktree” (on Git 2.42):
>
> > A repository can have zero (i.e. bare repository) or one or more
> > worktrees attached to it.
>
> And as someone who never has needed to use a bare repository + worktrees
> I've just left it at that.
>

Thank you for this reference - given Eric's later answers I'll aim to
change the text rather than respect its current implication, but
knowing what needs to change is hugely helpful :)

(not that I know what to change the text *to*, but I guess there's
time to think about that)

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Eric Sunshine]

On Mon, Sep 4, 2023 at 10:41 AM Tao Klerks <tao@klerks.biz> wrote:
> I work with a project that often, or typically, benefits from having
> multiple worktrees where users are working with different versions of
> the code.
>
> Worktrees make particular sense in this project for all the usual
> reasons, plus the fact that it is a very *large* repo - multiple gigs
> of repo, and very fast-moving; cloning and fetching less often is
> nice.
>
> One problem that we found users to have, early on when we introduced
> worktrees, was that it was not obvious to users that there was (or why
> there was) a "main" worktree, containing the actual ".git" repo, and
> "subsidiary" worktrees, in the same directory location. Git by default
> makes worktrees *subfolders* of the initial/main worktree/folder, but

This is not accurate. There is no default location for new worktrees;
git-worktree creates the new worktree at the location specified by the
user:

    git worktree add [<options>] <path> [<commit>]

where <path> -- the only mandatory argument -- specifies the location.

> we put them alongside each other to make other project tooling be able
> to treat all worktrees consistently; in daily use there is absolutely
> no difference between them, in all the "project tooling" there cannot
> be a practical difference... but if you choose to delete a worktree,
> and it happens to be the "special" one that contains the repo...
> you've just lost stuff that you didn't expect to lose (any local
> branches and stashes, basically).
>
> Because worktree use was so useful/widespread/critical on this
> project, and we already had a custom cloning process that introduced
> selective refspecs etc, we introduced a special clone topology: the
> initial clone is a bare repo, and that folder gets a specific clear
> name (ending in .git). Then we create worktrees attached to that bare
> repo.
>
> Generally speaking, this has worked *very* well: I would recommend it
> generally as a recognized/supported local-repo-setup. The most
> important thing that makes this *possible* is the fact that "git
> rev-parse --is-bare-repository" returns True in the bare repo folder,
> where the lack of index and HEAD shouldn't bother git, and it returns
> False in any one of the worktrees. It feels like things were designed
> to work this way, even though I can find no explicit mention of this
> topology in the docs.

It indeed was designed to work this way. It is perfectly legitimate to
create worktrees attached to a bare repository[1].

[1]: Support for bare repositories in conjunction with multiple-
worktrees, however, came after the initial implementation of multiple-
worktrees. An unfortunate side-effect is that established terminology
became somewhat confusing. In particular, in a bare repository
scenario, the term "main worktree" refers to the bare repository, not
to the "blessed" worktree containing the ".git/" directory (since
there is no such worktree in this case).

> However, from time to time something weird happens: Today I finally
> started to understand why I was seeing a crazy GC error about bitmaps,
> intermittently: It seems to be because "git gc" wants to create
> bitmaps in bare repos, but can't do so properly when we're in a
> partial clone... or something like that?
>
> EG repro:
> ```
> git clone --bare https://github.com/ksylor/ohshitgit dangit_shared.git
> --filter=blob:none
> git -C dangit_shared.git worktree add $PWD/dangit_wt3
> cd dangit_wt3/
> echo "this is some new unique blob in the repo" > new_blob.txt
> git add new_blob.txt && git commit -m "new blob"
> cd ../dangit_shared.git/
> git gc
> ```
>
> This yields, at the end of the GC run:
> ```
> warning: Failed to write bitmap index. Packfile doesn't have full
> closure (object bf86ed1b2602ac3a8d4724bcdf6707b156673aac is missing)
> fatal: failed to write bitmap index
> fatal: failed to run repack
> ```
>
> On the other hand, running "git gc" in one of the worktrees works fine
> (except you first need to delete a couple of ".tmp-*" files from the
> "objects/pack" folder, if you already got the error above).

Worktrees appear to be a red-herring. It's possible to reproduce this
error without them. For instance:

    % git clone --bare --filter=blob:none
https://github.com/ksylor/ohshitgit dangit_shared.git
    % git clone dangit_shared.git foop
    % cd foop
    % echo nothing >nothing
    % git add nothing
    % git commit -m nothing
    fatal: unable to read dbbb0682a7690b62ccf51b2a8648fa71ac671348
    % git push origin master
    % cd ../dangit_shared.git
    % git gc
    ...
    warning: Failed to write bitmap index. Packfile doesn't have full
closure (object bf86ed1b2602ac3a8d4724bcdf6707b156673aac is missing)
    fatal: failed to write bitmap index
    fatal: failed to run repack

> I at first thought this was a bug - but as I realized the problematic
> behavior was tied to the "core.bare" setting (and its expression at
> runtime through "git rev-parse --is-bare-repository"), it became more
> obvious that maybe the system could/should be able to make assumptions
> about the kind of repo that has this "true", and assume there are no
> local-object-derived non-promisor packfiles (or whatever it is about
> this example that makes things unhappy).
>
> So, I guess I'm confused as to what "core.bare" is supposed to mean:
> Is it intended to mean "there is no index nor HEAD here, and that's
> good, don't worry" (in which case my setup is presumably "supported",
> and the gc behavior is buggy?), or is it intended to mean "this is the
> kind of repository in which there are no worktrees" (in which case I
> am abusing the system and get the errors I deserve)?

The former, meaning that your setup should be supported. Citing
documentation for `core.bare`:

    If true this repository is assumed to be bare and has no working
    directory associated with it. If this is the case a number of
    commands that require a working directory will be disabled, such
    as git-add(1) or git-merge(1).

On Mon, Sep 4, 2023 at 10:59 AM Tao Klerks <tao@klerks.biz> wrote:
> I should add that I only recently discovered "git clone
> --separate-git-dir", which I at first though was a formal expression
> of this setup... until I understood that the relationship between the
> "GITDIR" and the worktree that you end up with is not "Bare repo vs
> worktree", but rather... "orphaned repo / repo that doesn't know about
> its worktree, vs worktree".
>
> It seems, to me, that "my setup" makes a lot more sense than what you
> end up with when you use "--separate-git-dir", and that the behavior
> there predates the current "mutual reference" model of
> worktrees-to-their-repo. If "my" use of "core.bare" in the example
> above is sound - then should the implementation of
> "--separate-git-dir" be changed to produce a bare repo with a
> "worktrees" folder, like you get if you clone bare and add a worktree
> in two separate steps?

`--separate-git-dir` predates multiple-worktree support by several
years and is distinct in purpose from --bare and multiple-worktrees
(in fact, a couple somewhat recent fixes [2,3] were needed to prevent
--separate-git-dir from breaking worktree administrative data). My
understand from scanning history is that --separate-git-dir was
introduced in aid of submodule support and perhaps other use-cases.

[2]: 42264bc841 (init: teach --separate-git-dir to repair linked
worktrees, 2020-08-31)
[3]: 59d876ccd6 (init: make --separate-git-dir work from within linked
worktree, 2020-08-31)

On Mon, Sep 4, 2023 at 11:29 AM Tao Klerks <tao@klerks.biz> wrote:
> And to confuse matters further, I just stumbled across
> https://github.com/git/git/blob/master/contrib/workdir/git-new-workdir
> - I don't understand when you would want to use that vs, again, a bare
> repo with one or more worktrees properly attached via two-way
> references, their own indexes, their own reflogs, etc.
>
> Is it the case that this contrib script predates the current "git
> worktree" support?

git-new-workdir predates git-worktree by quite a few years and, as I
understand it, remains in-tree because it fills a niche not entirely
filled by git-worktree.

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Junio C Hamano]

Eric Sunshine <sunshine@sunshineco.com> writes:

> This is not accurate. There is no default location for new worktrees;
> git-worktree creates the new worktree at the location specified by the
> user:
>
>     git worktree add [<options>] <path> [<commit>]
>
> where <path> -- the only mandatory argument -- specifies the location.

All correct.  The per-worktree part of the repository data does live
in a subdirectory of the ".git" directory and that was probably what
Tao had in mind, though.

> It indeed was designed to work this way. It is perfectly legitimate to
> create worktrees attached to a bare repository[1].
>
> [1]: Support for bare repositories in conjunction with multiple-
> worktrees, however, came after the initial implementation of multiple-
> worktrees. An unfortunate side-effect is that established terminology
> became somewhat confusing. In particular, in a bare repository
> scenario, the term "main worktree" refers to the bare repository, not
> to the "blessed" worktree containing the ".git/" directory (since
> there is no such worktree in this case).

Again all correct.

>> Is it the case that this contrib script predates the current "git
>> worktree" support?
>
> git-new-workdir predates git-worktree by quite a few years and, as I
> understand it, remains in-tree because it fills a niche not entirely
> filled by git-worktree.

I actually think there is no longer a valid workflow whose support
by "worktree" is still insufficient and the script has outlived its
usefulness.  I have been a heavy user of the new-workdir script to
maintain my build environments, but I always have the HEAD of these
workdir's detached, so I can easily switch my arrangement to use the
"git worktree" without losing any flexibility.

Perhaps we should remove it, possibly leaving a tombstone file like
how we removed stuff from the contrib/examples directory.

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Eric Sunshine]

On Mon, Sep 4, 2023 at 9:09 PM Junio C Hamano <gitster@pobox.com> wrote:
> Eric Sunshine <sunshine@sunshineco.com> writes:
> > This is not accurate. There is no default location for new worktrees;
> > git-worktree creates the new worktree at the location specified by the
> > user:
> >
> >     git worktree add [<options>] <path> [<commit>]
> >
> > where <path> -- the only mandatory argument -- specifies the location.
>
> All correct.  The per-worktree part of the repository data does live
> in a subdirectory of the ".git" directory and that was probably what
> Tao had in mind, though.

That could be. I read Tao's explanation as meaning that people do this:

    git clone foo.git foo
    cd foo
    git worktree add bar
    git worktree add baz

rather than (perhaps) this:

    git clone foo.git foo
    cd foo
    git worktree add ../bar
    git worktree add ../baz

But it's possible I misunderstood.

> >> Is it the case that this contrib script predates the current "git
> >> worktree" support?
> >
> > git-new-workdir predates git-worktree by quite a few years and, as I
> > understand it, remains in-tree because it fills a niche not entirely
> > filled by git-worktree.
>
> I actually think there is no longer a valid workflow whose support
> by "worktree" is still insufficient and the script has outlived its
> usefulness.  I have been a heavy user of the new-workdir script to
> maintain my build environments, but I always have the HEAD of these
> workdir's detached, so I can easily switch my arrangement to use the
> "git worktree" without losing any flexibility.

My response was based upon my recollection of the periodic message
which shows up on the mailing list reporting a bug or submitting an
improvement for git-new-workdir, accompanied by a statement that
git-new-workdir is still a better fit for the user's particular
use-case. But I've never used it myself, so it's good to hear from
someone (you) who does use it.

> Perhaps we should remove it, possibly leaving a tombstone file like
> how we removed stuff from the contrib/examples directory.

Perhaps.

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Junio C Hamano]

Eric Sunshine <sunshine@sunshineco.com> writes:

>> All correct.  The per-worktree part of the repository data does live
>> in a subdirectory of the ".git" directory and that was probably what
>> Tao had in mind, though.
>
> That could be. I read Tao's explanation as meaning that people do this:
>
>     git clone foo.git foo
>     cd foo
>     git worktree add bar
>     git worktree add baz
>
> rather than (perhaps) this:
>
>     git clone foo.git foo
>     cd foo
>     git worktree add ../bar
>     git worktree add ../baz

Ah, that reading does totally make sense.

But I am not sure it would lead to "we need to carefully protect the
primary worktree", because it is rather obvious, especially if you
bypass "git worktree remove" and use "rm -fr", you would lose
everybody underneath if you remove the "foo" in the "worktrees are
subdirectories of the primary" variant in the above examples.

Even though deriving the worktree(s) from a separate and protected
bare repositories does protect you from total disaster caused by
removing "rm -fr" and bypassing "git worktree remove", it still
should be discouraged, as the per-worktree states left behind in the
repository interfere with the operations in surviving worktrees.
Teaching folks not to do "rm -fr" would be the first step to a more
pleasant end-user experience, I would think.

Thanks.

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Tao Klerks]

On Tue, Sep 5, 2023 at 5:13 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Eric Sunshine <sunshine@sunshineco.com> writes:
>
> >> All correct.  The per-worktree part of the repository data does live
> >> in a subdirectory of the ".git" directory and that was probably what
> >> Tao had in mind, though.
> >
> > That could be. I read Tao's explanation as meaning that people do this:
> >
> >     git clone foo.git foo
> >     cd foo
> >     git worktree add bar
> >     git worktree add baz
> >
> > rather than (perhaps) this:
> >
> >     git clone foo.git foo
> >     cd foo
> >     git worktree add ../bar
> >     git worktree add ../baz
>
> Ah, that reading does totally make sense.
>

Fwiw, Eric's reading was my intended one. The people I have spoken
with, as well as myself, have started using "git worktree" by doing
the former, and only later felt really transgressive when placing the
worktrees explicitly on a higher level, on equal footing with the
"main worktree". To me it seemed natural that the "nested worktrees"
approach was the expected one, as otherwise it gets even harder to
explain/justify the operational difference between the "main worktree"
and the other worktrees - then leading to the bare+worktrees approach
to eliminate that operational difference.

> But I am not sure it would lead to "we need to carefully protect the
> primary worktree", because it is rather obvious, especially if you
> bypass "git worktree remove" and use "rm -fr", you would lose
> everybody underneath if you remove the "foo" in the "worktrees are
> subdirectories of the primary" variant in the above examples.

Right, sorry, too many poorly-expressed thoughts crammed together.

We need to start carefully protecting main worktree *when we start to
get clever* and actually add the worktrees as siblings to the main
worktree. That protection is indeed "implicit" before you start using
"../"... but then you have other issues of
git-worktree-within-git-worktree confusion.

Is there a manual for "expected typical usage of git worktree" somewhere?

>
> Even though deriving the worktree(s) from a separate and protected
> bare repositories does protect you from total disaster caused by
> removing "rm -fr" and bypassing "git worktree remove", it still
> should be discouraged, as the per-worktree states left behind in the
> repository interfere with the operations in surviving worktrees.

Right, that's fine. Of course you're going to encourage deleting the
worktrees carefully... but equally of-course, some people *will* do
"rm -fr that-worktree-I-dont-know-how-to-clean", and when they do,
telling them "just 'git worktree repair'" is much easier than telling
them to "recover deleted files 'cause your local branches just
evaporated"

> Teaching folks not to do "rm -fr" would be the first step to a more
> pleasant end-user experience, I would think.

The less arcane trivia you *need* to teach users for them to be
effective, the better the experience is for everyone.

The fact that "deleting a standalone git repo only deletes what's in
that standalone git repo the way you've done your whole life, but in
this environment what look like multiple repos are actually
'worktrees', if you ever delete one your life *might*, if you choose
the wrong one, suddenly be very unpleasant" is arcane trivia, in my
opinion. Better to set things up so they *can't* shoot themselves in
the foot with a bullet of that caliber.

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Kristoffer Haugsbakk]

On Tue, Sep 5, 2023, at 18:25, Tao Klerks wrote:
> Fwiw, Eric's reading was my intended one. The people I have spoken
> with, as well as myself, have started using "git worktree" by doing
> the former, and only later felt really transgressive when placing the
> worktrees explicitly on a higher level, on equal footing with the
> "main worktree". To me it seemed natural that the "nested worktrees"
> approach was the expected one, as otherwise it gets even harder to
> explain/justify the operational difference between the "main worktree"
> and the other worktrees - then leading to the bare+worktrees approach
> to eliminate that operational difference.

Let's consider a use-case that `git-worktrees` can replace: just cloning
the repository again to do some particular task.

My coworkers have to work on some branch `divergent` which requires a
complete rebuild of the project compared to the main branch. But they also
need to work on small derivative branches of the main branch. In order to
not rebuild all the time they simply cloned the project again and work in
*that* repository when working on `divergent`. And since this is an
Intellij project it was cloned in the same directory as the original
clone.

Replacing this use-case with `git worktrees` would be:

    git worktree ../divergent-project

In this case, `git worktrees` is a more streamlined and better version of
cloning a separate repository:

1. Only one object store
2. No need to remember to fetch from both
3. No risk of forgetting that you have some n-iary clone on your machine
   with original work

But crucially they also had the luxury of just cloning the project again
since it is less than 1GB; people with larger repositories (like
yourself) might have to immediately use the streamlined approaches like
`git worktree` since the straightforward approaches are too costly.

So I think the sibling directory tree makes sense when you arrive at this
command/workflow from the brute-force approach. But maybe that's not the
case when you have to design the proper way of doing it up-front(?)

> Is there a manual for "expected typical usage of git worktree" somewhere?

The example in “Description” of `man git-worktree` uses a sibling
directory: `git worktree ../hotfix`.

>> Even though deriving the worktree(s) from a separate and protected
>> bare repositories does protect you from total disaster caused by
>> removing "rm -fr" and bypassing "git worktree remove", it still
>> should be discouraged, as the per-worktree states left behind in the
>> repository interfere with the operations in surviving worktrees.
>
> Right, that's fine. Of course you're going to encourage deleting the
> worktrees carefully... but equally of-course, some people *will* do
> "rm -fr that-worktree-I-dont-know-how-to-clean", and when they do,
> telling them "just 'git worktree repair'" is much easier than telling
> them to "recover deleted files 'cause your local branches just
> evaporated"
>
>> Teaching folks not to do "rm -fr" would be the first step to a more
>> pleasant end-user experience, I would think.
>
> The less arcane trivia you *need* to teach users for them to be
> effective, the better the experience is for everyone.
>
> The fact that "deleting a standalone git repo only deletes what's in
> that standalone git repo the way you've done your whole life, but in
> this environment what look like multiple repos are actually
> 'worktrees', if you ever delete one your life *might*, if you choose
> the wrong one, suddenly be very unpleasant" is arcane trivia, in my
> opinion. Better to set things up so they *can't* shoot themselves in
> the foot with a bullet of that caliber.

I don't see how the principle of respecting the level of abstraction
doesn't apply here.

Before you might be able to delete a branch `b` by doing `git rm
.git/refs/heads/b` and be content when that either finishes successfully
or when it complains that the file doesn't exist; you can feel confident
that there is no more `b` branch. Now though the ref `b` might be a
*packed ref*, so you cannot do that and be sure that the ref was removed.

So if you create a worktree with `git worktree`, you should probably
remove it with the same command.

(Am I missing something? I probably am. I might not have understood the
context for wanting to run rm(1) on these directories.)

Personally I like to conceptualize worktrees as having equal status to
each other as far as me (the user) is concerned, since there isn't
anything you can do in the main worktree–or at least I haven't found
anything like that—that I *can't* do in a worktree *at that level of
abstraction* (directly manipulating files in the `.git` or
`repository.git` directory doesn't apply to this level of abstraction, so
that the linked worktrees only have a gitfile is irrelevant here).

-- 
Kristoffer Haugsbakk

Re: Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

[Tao Klerks]

On Tue, Sep 5, 2023 at 2:26 AM Eric Sunshine <sunshine@sunshineco.com> wrote:
>
> On Mon, Sep 4, 2023 at 10:41 AM Tao Klerks <tao@klerks.biz> wrote:
> >
> > One problem that we found users to have, early on when we introduced
> > worktrees, was that it was not obvious to users that there was (or why
> > there was) a "main" worktree, containing the actual ".git" repo, and
> > "subsidiary" worktrees, in the same directory location. Git by default
> > makes worktrees *subfolders* of the initial/main worktree/folder, but
>
> This is not accurate. There is no default location for new worktrees;
> git-worktree creates the new worktree at the location specified by the
> user:
>
>     git worktree add [<options>] <path> [<commit>]
>
> where <path> -- the only mandatory argument -- specifies the location.
>

Right - I know you *can* create worktrees in the "parent path", and
now that I revisit the doc I see there are even a couple examples that
do exactly that - but whenever I've talked with people who've tried
"git worktree add" independently, they ended up with the nested
worktrees and wondering why things work this weird way.

The idea that the "most trivial" example of creating a worktree would
be "git worktree add ../my_new_worktree" is, I believe, very
non-obvious.

The other thing that is I believe is non-obvious, in the current
solution, is that *if* you end up placing multiple worktrees at the
same level, *and* you end up using them "interchangeably" as you
presumably would in some or most scenarios, even though their *usage*
is identical, their "importance", or "lifetime" is very much not.

>
> It indeed was designed to work this way. It is perfectly legitimate to
> create worktrees attached to a bare repository[1].
>

Awesome, thx for confirming!

> in a bare repository
> scenario, the term "main worktree" refers to the bare repository, not
> to the "blessed" worktree containing the ".git/" directory (since
> there is no such worktree in this case).

Noted, thank you. We have been using the word "repository" vs worktree
- the (main) GITDIR is the repo, the worktrees are all just worktrees.
There is no "main worktree" in the way we talk about things, although
clearly the official nomenclature doesn't square with that, which I
might need to address at some point.


> Worktrees appear to be a red-herring. It's possible to reproduce this
> error without them. For instance:
>
>     % git clone --bare --filter=blob:none
> https://github.com/ksylor/ohshitgit dangit_shared.git
>     % git clone dangit_shared.git foop
>     % cd foop
>     % echo nothing >nothing
>     % git add nothing
>     % git commit -m nothing
>     fatal: unable to read dbbb0682a7690b62ccf51b2a8648fa71ac671348
>     % git push origin master
>     % cd ../dangit_shared.git
>     % git gc
>     ...
>     warning: Failed to write bitmap index. Packfile doesn't have full
> closure (object bf86ed1b2602ac3a8d4724bcdf6707b156673aac is missing)
>     fatal: failed to write bitmap index
>     fatal: failed to run repack
>

Hmm, I don't really understand what happened there, but it looks to me
like you went *much* further off the beaten path by cloning from a
partial clone. Afaik that's hard-not-supported...?

>
> The former, meaning that your setup should be supported. Citing
> documentation for `core.bare`:
>
>     If true this repository is assumed to be bare and has no working
>     directory associated with it. If this is the case a number of
>     commands that require a working directory will be disabled, such
>     as git-add(1) or git-merge(1).
>

Thanks again!

>
> `--separate-git-dir` predates multiple-worktree support by several
> years and is distinct in purpose from --bare and multiple-worktrees
> (in fact, a couple somewhat recent fixes [2,3] were needed to prevent
> --separate-git-dir from breaking worktree administrative data). My
> understand from scanning history is that --separate-git-dir was
> introduced in aid of submodule support and perhaps other use-cases.

OK, so if it is legitimate as-is... why doesn't it set
"core.worktree"? At least that way there'd be a solid two-way
reference like with "git worktree" worktrees.

Or is the *point* of the submodule and/or other use-cases that the
gitdir can't "know" its worktree??

> git-new-workdir predates git-worktree by quite a few years and, as I
> understand it, remains in-tree because it fills a niche not entirely
> filled by git-worktree.

OK, I'll keep my nose out of this one entirely either way, thx :)