Precious files and the .jj directory

Precious files and the .jj directory

[Jade Lovelace]

I'm aware of the many discussions about precious files [1] [2] [3],
but I wanted to highlight a particularly pernicious category of
precious files that are really hard to do the right thing about with
Git: namely, other version control systems. In particular, as I
learned in [4], `git clean -ixd` will of course list `.jj` to delete
and delete it if you have it in gitignore. But yet having it untracked
results in it possibly accidentally getting added and also clutters up
`git status`.

It's my understanding that git has more file deletion edge cases of
gitignored files than of untracked ones so the latter is theoretically
safer. Is that correct?

[1]: https://lore.kernel.org/git/pull.1627.git.1703643931314.gitgitgadget@gmail.com/
[2]: https://lore.kernel.org/git/871s7r4wuv.fsf@evledraar.gmail.com/
[3]: https://lore.kernel.org/git/7v4oepaup7.fsf@alter.siamese.dyndns.org/
[4]: https://maddie.wtf/posts/2025-07-21-jujutsu-for-busy-devs

The part about these that is especially pernicious is that git does
the right thing to `.git`, there are not that many of *these*
particular file patterns, and they generally merit the same treatment
as .git as deleting them results in somewhere between frustrating and
catastrophic loss of work. The one other example of one I've seen
other than .jj is .sl, though that's only colocated with git if you
are up to serious shimming shenanigans with broken tools (nix flakes
etc) as AFAIK it is not supposed to be used colocated normally.

Should these be special cased somehow? Should they be simply caught by
the precious-files work when it eventually gets done?

Regards,
Jade

Re: Precious files and the .jj directory

[Elijah Newren]

On Tue, Jul 22, 2025 at 12:31 AM Jade Lovelace <lists@jade.fyi> wrote:
>
> I'm aware of the many discussions about precious files [1] [2] [3],
> but I wanted to highlight a particularly pernicious category of
> precious files that are really hard to do the right thing about with
> Git: namely, other version control systems. In particular, as I
> learned in [4], `git clean -ixd` will of course list `.jj` to delete
> and delete it if you have it in gitignore. But yet having it untracked
> results in it possibly accidentally getting added and also clutters up
> `git status`.
>
> It's my understanding that git has more file deletion edge cases of
> gitignored files than of untracked ones so the latter is theoretically
> safer. Is that correct?

I'm not sure what you mean here.  What I can say is that ignored files
are treated as expendable; thus, for example, if another branch has a
file with the same name as an ignored file and you try to switch to
that branch, git will silently remove the ignored file in the way and
replace it with the file from the other branch.  However, I wouldn't
consider that an edge case.  I'd consider edge cases to be e.g. `git
stash` is implemented via forking a number of other git commands, and
one of those wasn't careful to avoid deleting files when our intention
was to avoid deleting them.  But, in the case of ignored files,
removing files in the way of the operation is not accidental; it's
documented as intended whenever that ignored file is in the way.

Untracked files are generally not treated as expendable, meaning we do
not intend to delete them.  The main caveat is that you can request
they be removed as needed by specifying a forcing flag (e.g. git
checkout --force, or git reset --hard) for git to delete those.  There
are a few edge cases, where commands invoke subcommands that might not
have been careful about flags they specify, resulting in the files
being deleted when they shouldn't be.  Most of these cases were fixed
a few years ago, though I documented a few extra cases.  Link [6] from
your Link [1] will lead you to those.

The precious file proposal is about splitting ignored files into two
categories -- trashable (what all ignored files currently fall under),
and precious (ignored but not expendable).  Until someone pushes that
effort, you have to decide whether it's more important to you that the
files aren't deleted (in which case I'd leave them as untracked) or
that they don't show up in `git status` and that you don't
accidentally add them when you're not careful about which files you
are adding (in which case you can mark them as ignored).

>
> [1]: https://lore.kernel.org/git/pull.1627.git.1703643931314.gitgitgadget@gmail.com/
> [2]: https://lore.kernel.org/git/871s7r4wuv.fsf@evledraar.gmail.com/
> [3]: https://lore.kernel.org/git/7v4oepaup7.fsf@alter.siamese.dyndns.org/
> [4]: https://maddie.wtf/posts/2025-07-21-jujutsu-for-busy-devs
>
> The part about these that is especially pernicious is that git does
> the right thing to `.git`, there are not that many of *these*
> particular file patterns, and they generally merit the same treatment
> as .git as deleting them results in somewhere between frustrating and
> catastrophic loss of work.

I can see that'd be pretty bad.  This probably arises in practice when
folks collectively put the files in all three categories, right?  In
other words, they start off as untracked, but someone accidentally
commits them (making them be tracked in some commits or branches), and
someone else decides to ignore them, and then when the person who
ignored these files tries to switch branches or bisect or rebase to or
on top of the other developer's accidental commit, then their files
are nuked.

If the files had only been in the combination of {untracked, ignored}
or {untracked, tracked} then you'd likely be fine.  You'd also be fine
if they were always ignored from the beginning, since that'd cause
everyone to be unlikely to commit them and make them be tracked.  It's
only when you end up with files that are both tracked in some commits
and ignored by some developers that you significantly risk running
into problems.

Or am I missing some case where this comes up?  Is one of the edge
cases for the handling of untracked files biting you?

> The one other example of one I've seen
> other than .jj is .sl, though that's only colocated with git if you
> are up to serious shimming shenanigans with broken tools (nix flakes
> etc) as AFAIK it is not supposed to be used colocated normally.
>
> Should these be special cased somehow? Should they be simply caught by
> the precious-files work when it eventually gets done?

I'd rather avoid the special-casing and instead have it be solved by
implementing precious files.  It is an interesting case where it gives
more motivation to the need for precious files; thanks for passing it
along.

Re: Precious files and the .jj directory

[Junio C Hamano]

Jade Lovelace <lists@jade.fyi> writes:

> I'm aware of the many discussions about precious files [1] [2] [3],
> but I wanted to highlight a particularly pernicious category of
> precious files that are really hard to do the right thing about with
> Git: namely, other version control systems. In particular, as I
> learned in [4], `git clean -ixd` will of course list `.jj` to delete
> and delete it if you have it in gitignore. But yet having it untracked
> results in it possibly accidentally getting added and also clutters up
> `git status`.

Yes, that is a very concise summary of what happens if you do not
support "ignored but precious" as a class of files distinct from
"ignored and expendable".  Git only supports the latter and we wish
we had also the former is where we are, after "many discussions" you
have read.

> It's my understanding that git has more file deletion edge cases of
> gitignored files than of untracked ones so the latter is theoretically
> safer. Is that correct?

Sorry, but I am not sure what apples and oranges you are comparing.

You list a thing in the .gitignore file or the .git/info/exclude
file (collectively known as "exclude mechanism") and the thing
becomes "ignored an dexpendable".  Your "add" will warn when you try
to add it, because it is ignored.  Your "checkout", "merge", etc.,
will happily overwrite such a path when it needs to be removed to
make room, because it is expendable.

You do not tell about a thing to the exclude mechanism, and the
thing is "untracked, not ignored".  Your "add" will happily add such
a path, because it is not ignored.  Your "checkout" and others will
play safer and more careful, avoid removing it, stop operation that
needs to remove the path in order to continue, because it is not
expendable.

The only solution would be to add proper support for ignored but
precious, I would think.

Re: Precious files and the .jj directory

[Martin von Zweigbergk]

On Tue, Jul 22, 2025 at 8:22 AM Elijah Newren <newren@gmail.com> wrote:
> Or am I missing some case where this comes up?  Is one of the edge
> cases for the handling of untracked files biting you?

If I understood the discussion correctly, it typically happens in what
we call "colocated repos" [1], where `.git/` and `.jj/` are siblings.
I just sent https://github.com/jj-vcs/jj/pull/7071 to at least warn
against running `git clean -fdx` when creating such setups.

[1] https://jj-vcs.github.io/jj/latest/git-compatibility/#co-located-jujutsugit-repos