* [RFC PATCH 1/6] precious-files.txt: new document proposing new precious file type
2026-10-08 12:06 [RFC PATCH 0/6] Introduce precious files Mirko Faina
@ 2026-10-08 12:06 ` Mirko Faina
2026-10-08 12:06 ` [RFC PATCH 2/6] dir.h: replace pattern macros with enum in attr.h Mirko Faina
` (5 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Mirko Faina @ 2026-10-08 12:06 UTC (permalink / raw)
To: git
Cc: Mirko Faina, Jeff King, Junio C Hamano, Elijah Newren,
Derrick Stolee, Sebastian Thiel
From: Elijah Newren <newren@gmail.com>
We have traditionally considered all ignored files to be expendable, but
users occasionally want ignored files that are not considered
expendable. Add a design document covering how to split ignored files
into two types: 'trashable' (what all ignored files are currently
considered) and 'precious' (the new type of ignored file).
Helped-by: Sebastian Thiel <sebastian.thiel@icloud.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Mirko Faina <mroik@delayed.space>
---
Was taken as is from [1]
[1] https://lore.kernel.org/git/pull.1627.git.1703643931314.gitgitgadget@gmail.com/
Documentation/technical/precious-files.txt | 540 +++++++++++++++++++++
1 file changed, 540 insertions(+)
create mode 100644 Documentation/technical/precious-files.txt
diff --git a/Documentation/technical/precious-files.txt b/Documentation/technical/precious-files.txt
new file mode 100644
index 0000000000..05c205b57b
--- /dev/null
+++ b/Documentation/technical/precious-files.txt
@@ -0,0 +1,540 @@
+Precious Files Design Document
+==============================
+
+Table of Contents
+ * Objective
+ * Background
+ * File categorization exceptions
+ * Proposal
+ * Precious file specification
+ * Breakdown of suggested behaviors by command
+ * Backward compatibility notes
+ * Slightly incompatible syntax
+ * Interaction with sparse-checkout parsing
+ * Behavior of traditional flags
+ * Interaction with older Git clients
+ * Commands with modified meaning
+ * Implementation hints
+ * Data structures
+ * Code areas
+ * Minimum
+ * Out of scope
+ * Previous discussions
+ * Alternatives considered
+
+Objective
+---------
+Support "Precious" Files in git, a set of files which are considered
+ignored (e.g. do not show up in "git status" output) but are not expendable
+(thus won't be removed to make room for a file when switching or merging
+branches).
+
+Background
+----------
+In git we have different types of files, with various subdivisions:
+ * tracked
+ * present (i.e. part of sparse checkout)
+ * not present (i.e. not part of sparse checkout)
+ * not tracked
+ * ignored (also treated as expendable)
+ * untracked (more precisely, not-tracked-and-not-ignored, but often
+ referred to as simply "untracked" despite the fact that such a term
+ is easily mistaken as a synonym to "not tracked". However, we haven't
+ been fully consistent, and some places like `git ls-files --others`
+ may use "untracked" to refer to the larger not-tracked category).
+ Not considered expendable.
+
+Over the years, the fact that ignored files are unconditionally treated as
+expendable (so that other operations like git checkout might wipe them out
+to make room for files on the other branch) has occasionally caused
+problems. Many have expressed a desire for subdividing the ignored class,
+so that we have both ignored-and-expendable (possibly referred to as
+"trashable", covering the only type of ignored file we have today) and
+introducing ignored-and-not-expendable (often referred to as "precious").
+
+File categorization exceptions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Our division above into nice categories is actually a bit of a lie.
+
+Once upon a time untracked files were considered expendable[1]. Even after
+that changed, we still had lots of edge cases where untracked files were
+deleted when they shouldn't be, and ignored files weren't deleted when they
+should be[2]. While that has been (mostly) fixed, despite the general
+intent to preserve untracked files, we have special cases that are
+documented as not preserving them[4,5]. There are also a few codepaths
+that have comments about locations that might (or definitely do)
+erroneously delete untracked paths[6]. And at least one code path that is
+known to erroneously delete untracked paths which has not been commented:
+`git checkout <tree> <pathspec>`. And there may be more.
+
+[1] https://lore.kernel.org/git/CABPp-BFyR19ch71W10oJDFuRX1OHzQ3si971pMn6dPtHKxJDXQ@mail.gmail.com/
+[2] https://lore.kernel.org/git/pull.1036.v3.git.1632760428.gitgitgadget@gmail.com/
+[3] https://lore.kernel.org/git/de416f887d7ce24f20ad3ad4cc838394d6523635.1632760428.git.gitgitgadget@gmail.com/
+[4] https://lore.kernel.org/git/xmqqr1e2ejs9.fsf@gitster.g/
+[5] https://lore.kernel.org/git/de416f887d7ce24f20ad3ad4cc838394d6523635.1632760428.git.gitgitgadget@gmail.com/
+[6] https://lore.kernel.org/git/6b42a80bf3d46e16980d0724e8b07101225239d0.1632760428.git.gitgitgadget@gmail.com/
+
+This history and these exceptions matter to this proposal because:
+ * it highlights how much work can be involved in trying to treat a class
+ of files as not expendable
+ * the existing corner cases where untracked files are erroneously
+ treated as expendable will probably also double as corner cases where
+ precious files are treated as expendable
+ * the past fixes for treating untracked files as precious will likely
+ highlight the needed types of code changes to treat ignored files as
+ precious
+
+Proposal
+--------
+We propose adding another class of files: ignored-but-not-expendable,
+referred to by the shorthand of "precious". The proposal is simple at a
+high level, but there are many details to consider:
+ * How to specify precious files (extended .gitignore syntax? attributes?)
+ * Which commands should be modified, and how?
+ * How to handle flags that are essentially a partial implementation of
+ a precious capability (e.g. [--[no-]overwrite-ignore])?
+ * How will older Git clients behave on a repo with precious files?
+The subsequent sections will try to address these questions in more detail.
+
+One thing to highlight here is that the class formerly called
+`ignored` now has two subtypes: (1) the type we already have,
+ignored-and-expendable (sometimes referred to below as "trashable")
+and (2) the new type, ignored-and-not-expendable (referred to as
+"precious").
+
+Precious file specification
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+As per [P2]:
+
+ """
+ Even though I referred to the precious _attribute_ in some of these
+ discussions, between the attribute mechanism and the ignore
+ mechanism, I am actually leaning toward suggesting to extend the
+ exclude/ignore mechanism to introduce the "precious" class. That
+ way, we can avoid possible snafu arising from marking a path in
+ .gitignore as ignored, and in .gitattrbutes as precious, and have to
+ figure out how these two settings are to work together.
+ """
+
+we specify precious files via an extension to .gitignore. In particular,
+lines starting with a '$' character specify that the file is precious.
+For example:
+ $.config
+would say the file `.config` is precious.
+
+Now that there are three types of files specified by .gitignore files --
+untracked, trashable (ignored-and-expendable), and precious
+(ignored-and-not-expendable), the meaning of `!` at the begining of a line
+needs careful clarification. It could be seen as "not ignored" or as "not
+trashable", given the subdivision of ignored files that has occurred. We
+specifically take it to mean "not ignored", i.e. "untracked".
+
+This leaves us with a simple set of rules to provide to users about lines
+in their '.gitignore' file:
+ * No special prefix character => ignored-and-expendable ("trashable")
+ * A '$' prefix character => ignored-and-not-expendable ("precious")
+ * A '!' prefix character => not ignored, i.e. untracked
+
+It's worth noting that the traditional use of '!' as a negation
+character needs updating, given the introduction of a ternary state
+("not trashable" could mean either untracked or precious, which is
+ambiguous). Refrain from referring to '!' as a negation character to
+avoid confusion. To assist users in making this mindset shift, flag
+any line beginning with '!$' as an error. As always,
+backslash-escaping remains an option, allowing users to specify
+entries like '!\$foo' to mark a file named '$foo' as untracked.
+
+Breakdown of suggested behaviors by command
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+See also "Out of Scope" section below, particularly for:
+ * apply, am [without -3]
+ * checkout/restore
+ * checkout-index
+ * additional information on merge backends
+
+Documentation:
+ * audit for references to "ignore" and "ignored", to see which ones need
+ to now replace those with either "ignored-and-expendable" (or
+ "trashable"), and which can remain "ignored".
+ * audit for "exclude" and "excluded" (the older terminology for ignored
+ files) and update them as well.
+ * add references to "precious" (and perhaps "trashable) as needed (don't
+ forget the glossary)
+ * rm: update the documentation:
+ "Ignored files are deemed expendable and won't stop" ->
+ "Ignored files, unless specifically marked precious, are by default
+ deemed expendable and won't stop"
+ * ensure all codepaths touched by 0e29222e0c2 ("Documentation: call out
+ commands that nuke untracked files/directories", 2021-09-27) also call
+ out that they'll nuke precious files in addition to untracked ones.
+ * change the documentation for '!' in gitignore to stop using the term
+ 'negates'; it's potentially misleading now (negating a ternary value
+ yields an ambiguous value). Instead, the prefix is used to mark
+ untracked (or "not ignored") files.
+ * note that the --[no-]overwrite-ignore option is deprecated, and, since
+ it predated the introduction of precious files is also a misnomer. The
+ correct name of the option would actually be --[no-]overwrite-trashable
+ but it is too late to rename.
+ * consider documenting that merge's --no-overwrite-ignore option is
+ virtually worthless (only works with the fast-forwarding backend).
+ * consider auditing the code for 'untracked' and fixing those to be
+ 'not tracked' in cases where both 'untracked' and 'ignored' files
+ are meant
+
+checkout/switch:
+ * will need to not overwrite precious files when they are in the way of
+ switching branches, unless --force/-f is specified.
+
+checkout/restore:
+ * when passed a <tree> as a source, do not overwrite precious files
+ (NOR untracked files!), unless --force/-f is specified. [Could be
+ considered a stretch goal...]
+
+merge:
+ * do not overwrite precious files when they are in the way of merging
+ branches. (Must be handled in each and every merge strategy;
+ user-defined merge strategies may get this wrong.)
+
+read-tree:
+ * -u: do not overwrite precious files when they are in the way, unless...
+ * --reset and -u: overwrite precious files as well as untracked files.
+ Add to the warning under --reset about overwritten untracked files to
+ note that precious files are also overwritten.
+
+am -3, cherry-pick, rebase, revert, : same as above for checkout/switch and
+ merge.
+
+add:
+ * same as today, just make sure when we split the ignored array (ignored &
+ ignored_nr) into multiple categories that it continues working
+
+rm:
+ * make sure submodules are not removed if precious files are present.
+ Currently, rm will remove submodules if only ignored files are present.
+
+check-ignore:
+ * since this command exists for debugging gitignore rules, there needs to
+ be some kind of mechanism for differentiating between trashable and
+ precious files. It is okay if this comes with a new command-line flag,
+ but there should be some tests showing how it behaves both with and
+ without that flag when precious files are present
+
+clean:
+ * clarify the meaning of -x and -X options: -X now means only remove
+ trashable files. -x means remove both untracked and trashable files.
+ (See also [P17])
+ * add a --all option for removing all not-tracked files: untracked,
+ trashable, and precious.
+ * Other than --all, it is not worth adding flags for cleaning subsets of
+ not-tracked files that include precious files (thus, no flag for just
+ precious, or trashable and precious, or untracked and precious)
+ * Patterns with a leading '$' can be passed to --exclude, if wanted.
+
+ls-files:
+ * --ignored/-i: shows every kind of ignored file (thus behaving the same
+ as today, since there is no way to distinguish between the types of
+ ignored in the output)
+ * add new `--ignored=precious` and `--ignored=trashable` flags for
+ differentiating. A plain `--ignored` is like having both
+ `--ignored=precious` and `--ignored=trashable` specified.
+ * --exclude,--exclude-from can now take patterns with a leading '$' and
+ the file will be considered precious rather than trashable.
+
+status:
+ * --ignored (without additional parameters) continues behaving as-is: it
+ prints both trashable and precious files in its "Ignored" category with
+ no distinguishing.
+ * --ignored --short will continue showing trashable files with '!!', but
+ will now show precious files using '$$'.
+ * --ignored --porcelain={v1,v2} will continue showing precious files
+ with the '!' character, since scripts may not be prepared to parse a
+ leading '$'. We can't break those scripts, even if it'd avoid the
+ off chance that those scripts act on the information about "ignored"
+ files and end up nuking precious files.
+ * --ignored --porcelain=v3 will need to be introduced to show precious
+ files with a leading '$'.
+
+sparse-checkout:
+ * the --rules-file option should be tested with a pattern with a leading
+ '$' to make sure it prints an expected error.
+ * it might be worth noting somewhere that sparse-checkout treats
+ ignored files as precious; when sparsifying, it attempts to remove
+ directories that do not match the sparse specification, but will
+ leave them present if any of the tracked files are modified, or if
+ there are any not-tracked files present. That includes ignored
+ files. That means no additional work is needed for precious
+ support; I just mention it for completeness.
+
+Backward compatibility notes
+----------------------------
+There are multiple issues that impinge on backward compatibility (either in
+terms of special care we need to take, or in terms of messaging we may need
+to send out about changes):
+ * Slightly incompatible syntax
+ * Interaction with sparse-checkout parsing
+ * Behavior of traditional flags
+ * Interaction with older Git clients
+ * Commands with modified meaning
+We'll discuss each in its own subsection below.
+
+Slightly incompatible syntax
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+This new syntax obviously breaks backward compatibility in that an ignored
+path named `$.config` would now have to be specified as `\$.config`. This
+is similar to how introducing `!` as a prefix in .gitignore files was a
+backward compatibility break. We expect and hope that the fallout will be
+minor. See also [P10].
+
+Interaction with sparse-checkout parsing
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+The $GIT_DIR/info/sparse-checkout file also makes use of gitignore syntax
+and the gitignore parsing to read the file. It differs in that the files
+specified are considered the files to be included (i.e. present in the
+working copy) rather than which files should be excluded, but otherwise
+has until now used identical syntax and parsing.
+
+However, for sparse-checkout there is no third type of file, so the '$'
+prefix makes no sense for it. As such, it should be an error for any
+lines to begin with '$' in a sparse-checkout file.
+
+(This also means that if anyone really did have a path beginning with '$'
+in sparse-checkout files previously, then they now need to backslash escape
+them, the same as with .gitignore files.)
+
+While we could theoretically avoid this small backward compatibility break
+for sparse-checkout parsing by just treating a leading '$' the way it
+traditionally has been done, I am worried about practically maintaining that
+solution:
+ * the gitignore parsing is peppered with references like 'exclude' that
+ are specific to the gitignore case
+ * because of the above, it is _heavily_ confusing to attempt to read and
+ understand the gitignore handling while considering the sparse-checkout
+ case. I've been tripped up by it *many* times.
+ * I think trying to reuse the existing parsing engine and have it handling
+ both old and new syntax is a recipe for failure. It'd be much cleaner
+ to have errors thrown if the processing turns up any "precious" files,
+ or perhaps if any line starts with '$'.
+ * I think making a copy of the existing parsing, and then letting them
+ diverge, means the two will eventually diverge even further, and we
+ would need to make a copy of all the documentation about gitignore rules
+ for sparse-checkout, all for the non-default non-cone case we are
+ already recommending users away from.
+
+Behavior of traditional flags
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+There are two flags to consider here: the --porcelain flag to git-status,
+and the --no-overwrite-ignore command to checkout & merge commands. For
+the --porcelain flag to git-status, see the "Breakdown of suggested
+behaviors by command" and look for git-status there. The rest of this
+section will focus on --[no-]overwrite-ignore.
+
+People have wanted precious files long enough, that they implemented an
+interim kludge of sorts -- a command line option that can be passed to
+various subcommands that treats all ignored files as precious:
+--no-overwrite-ignore.
+
+In particular, this flag can be passed to both git-checkout, and git-merge.
+However, in merge's case, the support depended the flag being passed to the
+backend and the backend supporting it. The builtin/merge.c code only ever
+bothered to pass this flag down to the fast-forwarding merge handling code,
+so it never worked with any backends that actually create a merge commit.
+
+We do need to keep these flags working, at least as much as they did
+previously. However, we don't want to consider them desired features,
+which would lead us to making related equivalents for precious files like
+--overwrite-precious. Instead we will:
+ * Keep --[no-]overwrite-ignore working, as much as it already was.
+ * Recommend users mark precious files in their gitignore files instead of
+ using these flags
+
+Interaction with older Git clients
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Older Git clients will not understand precious files. This means that:
+ * precious files will be considered untracked and not ignored.
+ * most comands will preserve these files, since untracked-and-not-ignored
+ are not considered expendable.
+ * git status will continue listing these files
+ * git add will add these files without requiring -f.
+
+This seems like a reasonable tradeoff that only has minor annoyances. The
+alternative of having the precious files treated as ignored has the very
+risky trade-off of deleting files which the users marked as important for
+us to keep.
+
+Commands with modified meaning
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+In clean, we adjust the meaning of both -x and -X:
+ -X: remove only trashable files
+ -x: remove untracked and trashable files (but preserve precious ones)
+
+Implementation hints
+--------------------
+
+Data structures
+~~~~~~~~~~~~~~~
+ * We will want to add a `precious` and `precious_nr` in dir_struct,
+ similar to the current entries/nr or ignored/ignored_nr.
+ * We may want to rename `ignored` and `ignored_nr` in dir_struct to
+ `trashable` and `trashable_nr`.
+
+Code areas
+~~~~~~~~~~
+ * "preserve_ignored", a flag in the code for handling the
+ --[no-]overwrite-ignore flag, is a very helpful marker about what needs
+ to be tweaked and how to tweak it to preserve more files. In particular,
+ note that --no-overwrite-ignore works by telling the machinery in dir.c
+ to not do the setup_standard_excludes() stuff, so that all ignored files
+ just look like untracked files. We'll need something slightly smarter,
+ which makes precious files look like untracked while trashable files
+ still appear in ignored. Shouldn't be too bad.
+ * we might need to add another entry to the unpack_trees_reset_type
+ enum. Or perhaps we still keep both UNPACK_RESET_PROTECT_UNTRACKED
+ and UNPACK_RESET_OVERWRITE_UNTRACKED but rename them with
+ s/UNTRACKED/NOT_EXPENDABLE/ so it is clear they handle both untracked and
+ precious files. Not sure which is needed yet.
+ * dir_struct->flags _might_ need new entries.
+ * ensure all relevant codepaths touched by 94b7f1563ac ("Comment important
+ codepaths regarding nuking untracked files/dirs", 2021-09-27) are either
+ fixed or also mention precious files
+ * am/rebase/checkout[without -f]: see 480d3d6bf90 ("Change unpack_trees'
+ 'reset' flag into an enum", 2021-09-27)
+ * Merge backends:
+ * (see also "Out of scope" section)
+ * merge-ort can be fixed by fixing the checkout code.
+ * merge-resolve and merge-octopus can probably be fixed by fixing
+ git-reset.
+ * stash:
+ * there is an existing --include-untracked option. There was no reason
+ to add a --include-ignored, because ignored files were trashable. Do
+ we need to add a --include-precious, though?
+ * this is a sad pile of shell-reimplemented-in-C. It's just awful.
+ See b34ab4a43ba ("stash: remove unnecessary process forking",
+ 2020-12-01) and ba359fd5070 ("stash: fix stash application in
+ sparse-checkouts", 2020-12-01) and 94b7f1563ac ("Comment important
+ codepaths regarding nuking untracked files/dirs", 2021-09-27).
+ Fixing stash to not nuke precious files (and to not nuke untracked
+ files either) might mean expunging the stupid
+ shell-reimplemented-in-C design, or at least moving things more in
+ that direction.
+ * rebase (merge backend), revert, cherry-pick, am -3: should automatically
+ be handled by getting merge-ort to work, which should work by making
+ checkout/switch work.
+ * bisect: should work by making checkout work
+
+Minimum
+~~~~~~~
+
+I think for a minimum implementation, we need to ensure that the following
+are handled:
+ * parsing:
+ * parsing of lines starting with '$' in .gitignore
+ * erroring on lines starting with '!$' in .gitignore
+ * erroring on lines starting with '$' in $GIT_DIR/info/sparse-checkout
+ * commands with support:
+ * switch/checkout
+ * merge when using the ort backend
+ * read-tree -u [without --reset] (due to internal use)
+ * ls-files
+
+Out of scope
+------------
+The following tasks are currently out of scope for this proposal:
+
+apply, am [without -3]: apply won't overwrite any file in the working
+ directory even when a new file is in the patch. It should overwrite
+ trashable files. We could log that bug via testcase, but make sure
+ there's a companion testcase that ensures overwriting untracked or
+ precious files continues to make apply throw an error. However, since
+ apply/am don't misbehave for precious files, we can defer this to later.
+
+checkout-index: similar to apply; won't overwrite any existing files, but
+ trashable files should be overwritten
+
+reset --hard:
+ * `git reset --hard` is a little funny and we have thought about changing
+ it[4]. However, that can be left for later and will not be tackled as
+ part of the work of introducing "precious" files as a concept.
+
+merge backends:
+ * it may make sense to try to make --no-overwrite-ignore work with more
+ merge backends, both because it's technically documented behavior, and
+ because doing so may be a step towards getting precious files supported
+ * when multiple merge strategies are specified, builtin/merge.c will
+ stash and restore state between the attempt of different strategies.
+ Since the reset_hard() function invokes `read-tree --reset -u`, there
+ might be a way to cause it to trash untracked files or to trash
+ precious files, depending on what the merge strategies did. It seems
+ unlikely (maybe the strategy handles D/F conflicts or rename
+ conflicts by renaming files in the way, and happens to rename a
+ precious file to a path that is considered either untracked or
+ precious -- merge-recursive certainly did this something like this
+ once upon a time and still might); we can probably ignore it for now.
+ * merge-recursive is a lost cause; it'd be a _huge_ amount of effort to
+ fix, but we intend to deprecate and delete it soon anyway (making all
+ requests for recursive just trigger ort instead).
+ * user-defined merge strategies are up to their authors to get right.
+ Odds are they won't, but odds are they already incorrectly nuke
+ untracked files too because who'd pay attention to a special case
+ like files being in the way of a merge? Anyway, "not our problem". :-)
+
+Previous discussions
+--------------------
+
+A far from exhaustive sampling of various past conversations on the topic:
+
+[P1] https://lore.kernel.org/git/7vipsnar23.fsf@alter.siamese.dyndns.org/
+[P2] https://lore.kernel.org/git/xmqqttqytnqb.fsf@gitster.g/
+[P3] https://lore.kernel.org/git/79901E6C-9839-4AB2-9360-9EBCA1AAE549@icloud.com/
+[P4] https://lore.kernel.org/git/87a6q9kacx.fsf@evledraar.gmail.com/
+[P5] https://lore.kernel.org/git/20190216114938.18843-1-pclouds@gmail.com/
+[P6] https://lore.kernel.org/git/87ftsi68ke.fsf@evledraar.gmail.com/
+[P7] https://lore.kernel.org/git/xmqqo7ub4sfh.fsf@gitster.g/
+[P8] https://lore.kernel.org/git/7v4oepaup7.fsf@alter.siamese.dyndns.org/
+[P9] https://lore.kernel.org/git/20181112232209.GK890086@genre.crustytoothpaste.net/
+[P10] https://lore.kernel.org/git/xmqqttqvg4lw.fsf@gitster.g/
+[P11] https://lore.kernel.org/git/xmqqk1hrr91s.fsf@gitster-ct.c.googlers.com/
+[P12] https://lore.kernel.org/git/9C4A2AFD-AAA2-4ABA-8A8B-2133FD870366@icloud.com/
+[P13] https://lore.kernel.org/git/xmqqfs2e3292.fsf@gitster.g/
+[P14] https://lore.kernel.org/git/0deee2bc-1775-4459-906d-1d44b3103499@gmail.com/
+[P15] https://lore.kernel.org/git/ZSkpOc%2FdcGcrFQNU@ugly/
+[P16] https://lore.kernel.org/git/xmqqil79t82q.fsf@gitster.g/
+[P17] https://lore.kernel.org/git/xmqqo7h6tnib.fsf@gitster.g/
+
+Alternatives considered
+-----------------------
+There have been multiple alternatives considered, along a few different
+axes:
+ * .gitattributes instead of .gitignore
+ * leaving sparse-checkout alone
+ * Trashable [P9,P11]
+ * Alternative gitignore syntax
+
+The choice of .gitattributes vs .gitignore was already addressed in the
+"Precious file specification" section.
+
+The choice to modify or leave alone the parsing of
+$GIT_DIR/info/sparse-checkout was already addressed in the "Interaction
+with sparse-checkout parsing" section.
+
+One alternative raised in the past was treating ignored files as not
+expendable by default, and then introducing a new category of
+ignored-but-expendable. This new category has been dubbed "trashable" in
+the past. That may have been a reasonable solution if Git did not have a
+large userbase already, but moving in this direction would cause severe
+problems for existing builds everywhere[P9] and would require users to
+doubly configure most files (since it is expected that
+ignored-but-expendable is a much larger class of files than
+ignored-but-precious). See also [P11].
+
+There have been multiple alternative suggestions for extending gitignore
+syntax to handle precious files and optionally future extensions as well.
+For example: [P10, P12, P13, P14, P15, P16] However:
+ * There have been on and off requests for precious files for about 14
+ years
+ * We are not aware of other types of extensions needed; there might
+ not be any
+ * The alternatives all seem much more complex to explain to users than
+ the simple proposal here.
+In particular, we like the simplicity of the providing the simple mapping
+to users from the penultimate paragraph of the "Precious file
+specification" section (the one regarding no-prefix vs. '!' vs '$').
--
2.56.0
^ permalink raw reply related [flat|nested] 8+ messages in thread* [RFC PATCH 2/6] dir.h: replace pattern macros with enum in attr.h
2026-10-08 12:06 [RFC PATCH 0/6] Introduce precious files Mirko Faina
2026-10-08 12:06 ` [RFC PATCH 1/6] precious-files.txt: new document proposing new precious file type Mirko Faina
@ 2026-10-08 12:06 ` Mirko Faina
2026-10-08 12:06 ` [RFC PATCH 3/6] dir.c: teach parse_path_pattern() precious files Mirko Faina
` (4 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Mirko Faina @ 2026-10-08 12:06 UTC (permalink / raw)
To: git; +Cc: Mirko Faina, Jeff King, Junio C Hamano, Elijah Newren,
Derrick Stolee
dir.h defines some macros that are used in the dir.c machinery for
.gitignore files. These macros are only used for the pattern struct
defined in attr.h so it really should belong there.
Remove the macro definitions from dir.h and replace them with the
pattern_flags enum in attr.h.
Signed-off-by: Mirko Faina <mroik@delayed.space>
---
attr.h | 9 ++++++++-
builtin/check-ignore.c | 1 +
builtin/sparse-checkout.c | 1 +
dir.c | 5 +++--
dir.h | 8 ++------
5 files changed, 15 insertions(+), 9 deletions(-)
diff --git a/attr.h b/attr.h
index a04a521092..c083d47df5 100644
--- a/attr.h
+++ b/attr.h
@@ -250,11 +250,18 @@ struct attr_state {
const char *setto;
};
+enum pattern_flags {
+ PATTERN_FLAG_NODIR = 1,
+ PATTERN_FLAG_ENDSWITH = 4,
+ PATTERN_FLAG_MUSTBEDIR = 8,
+ PATTERN_FLAG_NEGATIVE = 16,
+};
+
struct pattern {
const char *pattern;
int patternlen;
int nowildcardlen;
- unsigned flags; /* PATTERN_FLAG_* */
+ enum pattern_flags flags; /* PATTERN_FLAG_* */
};
/*
diff --git a/builtin/check-ignore.c b/builtin/check-ignore.c
index 644c9a414f..fdf7610178 100644
--- a/builtin/check-ignore.c
+++ b/builtin/check-ignore.c
@@ -9,6 +9,7 @@
#include "parse-options.h"
#include "submodule.h"
#include "write-or-die.h"
+#include "attr.h"
static int quiet, verbose, stdin_paths, show_non_matching, no_index;
static const char * const check_ignore_usage[] = {
diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c
index cb4a037b77..83981e2b1e 100644
--- a/builtin/sparse-checkout.c
+++ b/builtin/sparse-checkout.c
@@ -20,6 +20,7 @@
#include "setup.h"
#include "sparse-index.h"
#include "worktree.h"
+#include "attr.h"
static const char *empty_base = "";
diff --git a/dir.c b/dir.c
index d896e7be4b..c6342c882a 100644
--- a/dir.c
+++ b/dir.c
@@ -36,6 +36,7 @@
#include "trace2.h"
#include "tree.h"
#include "hex.h"
+#include "attr.h"
/*
* The maximum size of a pattern/exclude file. If the file exceeds this size
@@ -701,7 +702,7 @@ int no_wildcard(const char *string)
void parse_path_pattern(const char **pattern,
int *patternlen,
- unsigned *flags,
+ enum pattern_flags *flags,
int *nowildcardlen)
{
const char *p = *pattern;
@@ -979,7 +980,7 @@ void add_pattern(const char *string, const char *base,
{
struct path_pattern *pattern;
int patternlen;
- unsigned flags;
+ enum pattern_flags flags;
int nowildcardlen;
parse_path_pattern(&string, &patternlen, &flags, &nowildcardlen);
diff --git a/dir.h b/dir.h
index 83e0f648a8..210ee8a98d 100644
--- a/dir.h
+++ b/dir.h
@@ -6,6 +6,7 @@
#include "pathspec.h"
#include "statinfo.h"
#include "strbuf.h"
+#include "attr.h"
struct repository;
@@ -49,11 +50,6 @@ struct dir_entry {
char name[FLEX_ARRAY]; /* more */
};
-#define PATTERN_FLAG_NODIR 1
-#define PATTERN_FLAG_ENDSWITH 4
-#define PATTERN_FLAG_MUSTBEDIR 8
-#define PATTERN_FLAG_NEGATIVE 16
-
struct path_pattern {
/*
* This allows callers of last_matching_pattern() etc.
@@ -469,7 +465,7 @@ int add_patterns_from_blob_to_list(struct object_id *oid,
int add_patterns_from_buffer(char *buf, size_t size,
const char *base, int baselen,
struct pattern_list *pl);
-void parse_path_pattern(const char **string, int *patternlen, unsigned *flags, int *nowildcardlen);
+void parse_path_pattern(const char **string, int *patternlen, enum pattern_flags *flags, int *nowildcardlen);
void add_pattern(const char *string, const char *base,
int baselen, struct pattern_list *pl, int srcpos);
void clear_pattern_list(struct pattern_list *pl);
--
2.56.0
^ permalink raw reply related [flat|nested] 8+ messages in thread* [RFC PATCH 3/6] dir.c: teach parse_path_pattern() precious files
2026-10-08 12:06 [RFC PATCH 0/6] Introduce precious files Mirko Faina
2026-10-08 12:06 ` [RFC PATCH 1/6] precious-files.txt: new document proposing new precious file type Mirko Faina
2026-10-08 12:06 ` [RFC PATCH 2/6] dir.h: replace pattern macros with enum in attr.h Mirko Faina
@ 2026-10-08 12:06 ` Mirko Faina
2026-10-08 12:07 ` [RFC PATCH 4/6] dir.c: teach add_pattern() reject precious pattern Mirko Faina
` (3 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Mirko Faina @ 2026-10-08 12:06 UTC (permalink / raw)
To: git; +Cc: Mirko Faina, Jeff King, Junio C Hamano, Elijah Newren,
Derrick Stolee
parse_path_pattern() knows only to recognize patterns that are either
directories or normal files. It also knows that there are negated
patterns, but it doesn't know about patterns that indicate
precious files.
Teach about precious files to parse_path_pattern(). We teach it also to
detect the invalid patterns starting with '!$' which would mean a
negated precious file, that we don't allow. This is reported back to the
caller so we change its signature to return an int. We also teach the
callers parse_attr_line() and add_pattern() to handle such errors.
Signed-off-by: Mirko Faina <mroik@delayed.space>
---
attr.c | 8 ++++++--
attr.h | 1 +
dir.c | 26 +++++++++++++++++++++++---
dir.h | 2 +-
t/t7508-status.sh | 9 +++++++++
5 files changed, 40 insertions(+), 6 deletions(-)
diff --git a/attr.c b/attr.c
index 0e63f1b6de..5c60ca6b23 100644
--- a/attr.c
+++ b/attr.c
@@ -382,10 +382,14 @@ struct match_attr *parse_attr_line(const char *line, const char *src,
char *p = (char *)&(res->state[num_attr]);
memcpy(p, name, namelen);
res->u.pat.pattern = p;
- parse_path_pattern(&res->u.pat.pattern,
+ if (parse_path_pattern(&res->u.pat.pattern,
&res->u.pat.patternlen,
&res->u.pat.flags,
- &res->u.pat.nowildcardlen);
+ &res->u.pat.nowildcardlen)) {
+ warning(_("pattern '%s' is invalid, skipping"),
+ (char *)&res->u.pat.pattern);
+ goto fail_return;
+ }
if (res->u.pat.flags & PATTERN_FLAG_NEGATIVE) {
warning(_("Negative patterns are ignored in git attributes\n"
"Use '\\!' for literal leading exclamation."));
diff --git a/attr.h b/attr.h
index c083d47df5..d00939732b 100644
--- a/attr.h
+++ b/attr.h
@@ -255,6 +255,7 @@ enum pattern_flags {
PATTERN_FLAG_ENDSWITH = 4,
PATTERN_FLAG_MUSTBEDIR = 8,
PATTERN_FLAG_NEGATIVE = 16,
+ PATTERN_FLAG_PRECIOUS = 32,
};
struct pattern {
diff --git a/dir.c b/dir.c
index c6342c882a..c6f1bed429 100644
--- a/dir.c
+++ b/dir.c
@@ -700,7 +700,10 @@ int no_wildcard(const char *string)
return string[simple_length(string)] == '\0';
}
-void parse_path_pattern(const char **pattern,
+/*
+ * Returns 1 if the pattern is problematic, 0 otherwise
+ */
+int parse_path_pattern(const char **pattern,
int *patternlen,
enum pattern_flags *flags,
int *nowildcardlen)
@@ -709,7 +712,12 @@ void parse_path_pattern(const char **pattern,
size_t i, len;
*flags = 0;
- if (*p == '!') {
+ if (simple_length(p) >= 2 && p[0] == '!' && p[1] == '$') {
+ return 1;
+ } else if (*p == '$') {
+ *flags |= PATTERN_FLAG_PRECIOUS;
+ p++;
+ } else if (*p == '!') {
*flags |= PATTERN_FLAG_NEGATIVE;
p++;
}
@@ -736,6 +744,7 @@ void parse_path_pattern(const char **pattern,
*flags |= PATTERN_FLAG_ENDSWITH;
*pattern = p;
*patternlen = len;
+ return 0;
}
int pl_hashmap_cmp(const void *cmp_data UNUSED,
@@ -975,6 +984,14 @@ int hashmap_contains_parent(struct hashmap *map,
return 0;
}
+/*
+ * Parses the pattern for its type and sets flags accordingly, then adds it to
+ * the pattern list. If the pattern is invalid the function returns early with a
+ * warning.
+ *
+ * The only problematic patterns at the moment are the one starting with '!$'
+ * which are negated precious-files.
+ */
void add_pattern(const char *string, const char *base,
int baselen, struct pattern_list *pl, int srcpos)
{
@@ -983,7 +1000,10 @@ void add_pattern(const char *string, const char *base,
enum pattern_flags flags;
int nowildcardlen;
- parse_path_pattern(&string, &patternlen, &flags, &nowildcardlen);
+ if (parse_path_pattern(&string, &patternlen, &flags, &nowildcardlen)) {
+ warning(_("pattern '%s' is problematic, skipping"), string);
+ return;
+ }
FLEX_ALLOC_MEM(pattern, pattern, string, patternlen);
pattern->patternlen = patternlen;
pattern->nowildcardlen = nowildcardlen;
diff --git a/dir.h b/dir.h
index 210ee8a98d..5cda2cdba7 100644
--- a/dir.h
+++ b/dir.h
@@ -465,7 +465,7 @@ int add_patterns_from_blob_to_list(struct object_id *oid,
int add_patterns_from_buffer(char *buf, size_t size,
const char *base, int baselen,
struct pattern_list *pl);
-void parse_path_pattern(const char **string, int *patternlen, enum pattern_flags *flags, int *nowildcardlen);
+int parse_path_pattern(const char **string, int *patternlen, enum pattern_flags *flags, int *nowildcardlen);
void add_pattern(const char *string, const char *base,
int baselen, struct pattern_list *pl, int srcpos);
void clear_pattern_list(struct pattern_list *pl);
diff --git a/t/t7508-status.sh b/t/t7508-status.sh
index 0fd7c79911..aa251c6abd 100755
--- a/t/t7508-status.sh
+++ b/t/t7508-status.sh
@@ -301,6 +301,15 @@ EOF
test_cmp expect output
'
+test_expect_success 'parse invalid pattern' "
+ test_when_finished rm actual gitignore_backup &&
+ test_when_finished cp gitignore_backup .gitignore &&
+ cp .gitignore gitignore_backup &&
+ echo '!$/ciao' >> .gitignore &&
+ git status >actual 2>&1 &&
+ test_grep \"warning: pattern '!$/ciao' is problematic, skipping\" actual
+"
+
test_expect_success 'status with gitignore (nothing untracked)' '
{
echo ".gitignore" &&
--
2.56.0
^ permalink raw reply related [flat|nested] 8+ messages in thread* [RFC PATCH 4/6] dir.c: teach add_pattern() reject precious pattern
2026-10-08 12:06 [RFC PATCH 0/6] Introduce precious files Mirko Faina
` (2 preceding siblings ...)
2026-10-08 12:06 ` [RFC PATCH 3/6] dir.c: teach parse_path_pattern() precious files Mirko Faina
@ 2026-10-08 12:07 ` Mirko Faina
2026-10-08 12:07 ` [RFC PATCH 5/6] unpack-trees: teach check_ok_to_remove() precious Mirko Faina
` (2 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Mirko Faina @ 2026-10-08 12:07 UTC (permalink / raw)
To: git; +Cc: Mirko Faina, Jeff King, Junio C Hamano, Elijah Newren,
Derrick Stolee
There are some places where we want to disallow precious files patterns
altogether. This is the case with $GIT_DIR/info/sparse-checkout.
Teach add_pattern() a flag to indicate if we want it to reject
precious files patterns. If that's the case it fails the same way it
would when we encounter a pattern that starts with '!$', with a warning.
Signed-off-by: Mirko Faina <mroik@delayed.space>
---
builtin/clean.c | 4 ++--
builtin/ls-files.c | 2 +-
builtin/sparse-checkout.c | 15 +++++++++------
dir.c | 20 +++++++++++++++-----
dir.h | 5 +++--
t/helper/test-path-walk.c | 2 +-
t/t1091-sparse-checkout-builtin.sh | 8 ++++++++
7 files changed, 39 insertions(+), 17 deletions(-)
diff --git a/builtin/clean.c b/builtin/clean.c
index 1d5e7e5366..73408da5c2 100644
--- a/builtin/clean.c
+++ b/builtin/clean.c
@@ -707,7 +707,7 @@ static int filter_by_patterns_cmd(void)
item = &ignore_list.items[i];
if (!*item->string)
continue;
- add_pattern(item->string, "", 0, pl, -(i+1));
+ add_pattern(item->string, "", 0, pl, -(i+1), 0);
}
changed = 0;
@@ -1018,7 +1018,7 @@ int cmd_clean(int argc,
pl = add_pattern_list(&dir, EXC_CMDL, "--exclude option");
for (i = 0; i < exclude_list.nr; i++)
- add_pattern(exclude_list.items[i].string, "", 0, pl, -(i+1));
+ add_pattern(exclude_list.items[i].string, "", 0, pl, -(i+1), 0);
parse_pathspec(&pathspec, 0,
PATHSPEC_PREFER_CWD,
diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index b044520f9e..d1cc7e92f4 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -685,7 +685,7 @@ int cmd_ls_files(int argc,
ls_files_usage, 0);
pl = add_pattern_list(&dir, EXC_CMDL, "--exclude option");
for (i = 0; i < exclude_list.nr; i++) {
- add_pattern(exclude_list.items[i].string, "", 0, pl, --exclude_args);
+ add_pattern(exclude_list.items[i].string, "", 0, pl, --exclude_args, 0);
}
if (format && (show_stage || show_others || show_killed ||
diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c
index 83981e2b1e..615007148e 100644
--- a/builtin/sparse-checkout.c
+++ b/builtin/sparse-checkout.c
@@ -455,6 +455,8 @@ static struct sparse_checkout_init_opts {
int sparse_index;
} init_opts;
+#define PATTERN_DISALLOW_PRECIOUS (1<<1)
+
static int sparse_checkout_init(int argc, const char **argv, const char *prefix,
struct repository *repo)
{
@@ -487,7 +489,8 @@ static int sparse_checkout_init(int argc, const char **argv, const char *prefix,
memset(&pl, 0, sizeof(pl));
sparse_filename = get_sparse_checkout_filename();
- res = add_patterns_from_file_to_list(sparse_filename, "", 0, &pl, NULL, 0);
+ res = add_patterns_from_file_to_list(sparse_filename, "", 0, &pl, NULL,
+ PATTERN_DISALLOW_PRECIOUS);
/* If we already have a sparse-checkout file, use it. */
if (res >= 0) {
@@ -515,8 +518,8 @@ static int sparse_checkout_init(int argc, const char **argv, const char *prefix,
free(sparse_filename);
- add_pattern("/*", empty_base, 0, &pl, 0);
- add_pattern("!/*/", empty_base, 0, &pl, 0);
+ add_pattern("/*", empty_base, 0, &pl, 0, 0);
+ add_pattern("!/*/", empty_base, 0, &pl, 0, 0);
pl.use_cone_patterns = init_opts.cone_mode;
return write_patterns_and_update(repo, &pl);
@@ -618,12 +621,12 @@ static void add_patterns_from_input(struct pattern_list *pl,
struct strbuf line = STRBUF_INIT;
while (!strbuf_getline(&line, file))
- add_pattern(line.buf, empty_base, 0, pl, 0);
+ add_pattern(line.buf, empty_base, 0, pl, 0, 0);
strbuf_release(&line);
} else {
for (i = 0; i < argc; i++)
- add_pattern(argv[i], empty_base, 0, pl, 0);
+ add_pattern(argv[i], empty_base, 0, pl, 0, 0);
}
}
}
@@ -1079,7 +1082,7 @@ static int sparse_checkout_disable(int argc, const char **argv,
pl.use_cone_patterns = 0;
cfg->apply_sparse_checkout = 1;
- add_pattern("/*", empty_base, 0, &pl, 0);
+ add_pattern("/*", empty_base, 0, &pl, 0, 0);
prepare_repo_settings(the_repository);
repo->settings.sparse_index = 0;
diff --git a/dir.c b/dir.c
index c6f1bed429..9aba1716a6 100644
--- a/dir.c
+++ b/dir.c
@@ -993,7 +993,8 @@ int hashmap_contains_parent(struct hashmap *map,
* which are negated precious-files.
*/
void add_pattern(const char *string, const char *base,
- int baselen, struct pattern_list *pl, int srcpos)
+ int baselen, struct pattern_list *pl, int srcpos,
+ int disable_precious)
{
struct path_pattern *pattern;
int patternlen;
@@ -1004,6 +1005,12 @@ void add_pattern(const char *string, const char *base,
warning(_("pattern '%s' is problematic, skipping"), string);
return;
}
+
+ if (disable_precious && (flags & PATTERN_FLAG_PRECIOUS)) {
+ warning(_("'$%s' precious-files pattern not allowed here, skipping"),
+ string);
+ return;
+ }
FLEX_ALLOC_MEM(pattern, pattern, string, patternlen);
pattern->patternlen = patternlen;
pattern->nowildcardlen = nowildcardlen;
@@ -1162,6 +1169,7 @@ static void invalidate_directory(struct untracked_cache *uc,
/* Flags for add_patterns() */
#define PATTERN_NOFOLLOW (1<<0)
+#define PATTERN_DISALLOW_PRECIOUS (1<<1)
/*
* Given a file with name "fname", read it (either from disk, or from
@@ -1244,14 +1252,15 @@ static int add_patterns(const char *fname, const char *base, int baselen,
return -1;
}
- add_patterns_from_buffer(buf, size, base, baselen, pl);
+ add_patterns_from_buffer(buf, size, base, baselen, pl,
+ flags & PATTERN_DISALLOW_PRECIOUS);
free(buf);
return 0;
}
int add_patterns_from_buffer(char *buf, size_t size,
const char *base, int baselen,
- struct pattern_list *pl)
+ struct pattern_list *pl, int disable_precious)
{
char *orig = buf;
int i, lineno = 1;
@@ -1270,7 +1279,8 @@ int add_patterns_from_buffer(char *buf, size_t size,
if (entry != buf + i && entry[0] != '#') {
buf[i - (i && buf[i-1] == '\r')] = 0;
trim_trailing_spaces(entry);
- add_pattern(entry, base, baselen, pl, lineno);
+ add_pattern(entry, base, baselen, pl, lineno,
+ disable_precious);
}
lineno++;
entry = buf + i + 1;
@@ -1307,7 +1317,7 @@ int add_patterns_from_blob_to_list(
return -1;
}
- add_patterns_from_buffer(buf, size, base, baselen, pl);
+ add_patterns_from_buffer(buf, size, base, baselen, pl, 0);
free(buf);
return 0;
}
diff --git a/dir.h b/dir.h
index 5cda2cdba7..a6977149b8 100644
--- a/dir.h
+++ b/dir.h
@@ -464,10 +464,11 @@ int add_patterns_from_blob_to_list(struct object_id *oid,
struct pattern_list *pl);
int add_patterns_from_buffer(char *buf, size_t size,
const char *base, int baselen,
- struct pattern_list *pl);
+ struct pattern_list *pl, int disable_precious);
int parse_path_pattern(const char **string, int *patternlen, enum pattern_flags *flags, int *nowildcardlen);
void add_pattern(const char *string, const char *base,
- int baselen, struct pattern_list *pl, int srcpos);
+ int baselen, struct pattern_list *pl, int srcpos,
+ int disable_precious);
void clear_pattern_list(struct pattern_list *pl);
void dir_clear(struct dir_struct *dir);
diff --git a/t/helper/test-path-walk.c b/t/helper/test-path-walk.c
index 4233badb58..fefe885ec6 100644
--- a/t/helper/test-path-walk.c
+++ b/t/helper/test-path-walk.c
@@ -124,7 +124,7 @@ int cmd__path_walk(int argc, const char **argv)
info.pl->use_cone_patterns = 1;
strbuf_fread(&in, 2048, stdin);
- add_patterns_from_buffer(in.buf, in.len, "", 0, info.pl);
+ add_patterns_from_buffer(in.buf, in.len, "", 0, info.pl, 0);
strbuf_release(&in);
}
diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh
index 74b1761e0c..caae112037 100755
--- a/t/t1091-sparse-checkout-builtin.sh
+++ b/t/t1091-sparse-checkout-builtin.sh
@@ -252,6 +252,14 @@ test_expect_success 'sparse-checkout disable' '
check_files repo a deep folder1 folder2
'
+test_expect_success 'skip precious-file pattern in $GIT_DIR/info/sparse-checkout' "
+ test_when_finished rm actual .git/info/sparse-checkout &&
+ test_when_finished git sparse-checkout disable &&
+ echo \"$/ciao\" > .git/info/sparse-checkout &&
+ git sparse-checkout init >actual 2>&1 &&
+ test_grep \"warning: '$/ciao' precious-files pattern not allowed here, skipping\" actual
+"
+
test_expect_success 'sparse-index enabled and disabled' '
git -C repo sparse-checkout init --cone --sparse-index &&
test_cmp_config -C repo true index.sparse &&
--
2.56.0
^ permalink raw reply related [flat|nested] 8+ messages in thread* [RFC PATCH 5/6] unpack-trees: teach check_ok_to_remove() precious
2026-10-08 12:06 [RFC PATCH 0/6] Introduce precious files Mirko Faina
` (3 preceding siblings ...)
2026-10-08 12:07 ` [RFC PATCH 4/6] dir.c: teach add_pattern() reject precious pattern Mirko Faina
@ 2026-10-08 12:07 ` Mirko Faina
2026-10-08 12:07 ` [RFC PATCH 6/6] builtin/ls-files.c: support for precious files Mirko Faina
2026-10-08 12:12 ` [RFC PATCH 0/6] Introduce " Mirko Faina
6 siblings, 0 replies; 8+ messages in thread
From: Mirko Faina @ 2026-10-08 12:07 UTC (permalink / raw)
To: git; +Cc: Mirko Faina, Jeff King, Junio C Hamano, Elijah Newren,
Derrick Stolee
"git switch" and "git checkout" only clobber local changes when these
paths are marked as excluded. With the introduction of precious files we
want to make sure that the underlying machinery understands the
difference between a trashable and a precious file (which are both
excluded/ignored).
Teach check_ok_to_remove() to only target trashable files for clobbering
instead of targeting excluded files.
Signed-off-by: Mirko Faina <mroik@delayed.space>
---
This is missing some tests for clobbering, but I wasn't sure where to place
them.
dir.c | 19 ++++++++++++++++++-
dir.h | 4 ++++
unpack-trees.c | 7 ++++---
3 files changed, 26 insertions(+), 4 deletions(-)
diff --git a/dir.c b/dir.c
index 9aba1716a6..ffc1818533 100644
--- a/dir.c
+++ b/dir.c
@@ -1861,7 +1861,24 @@ struct path_pattern *last_matching_pattern(struct dir_struct *dir,
/*
* Loads the exclude lists for the directory containing pathname, then
- * scans all exclude lists to determine whether pathname is excluded.
+ * scans all exclude lists to determine whether pathname is trashable.
+ * Returns 1 if true, otherwise 0.
+ */
+int is_trashable(struct dir_struct *dir, struct index_state *istate,
+ const char *pathname, int *dtype_p)
+{
+ struct path_pattern *pattern =
+ last_matching_pattern(dir, istate, pathname, dtype_p);
+ if (pattern)
+ return pattern->flags &
+ (PATTERN_FLAG_NEGATIVE | PATTERN_FLAG_PRECIOUS) ? 0 : 1;
+ return 0;
+}
+
+/*
+ * Loads the exclude lists for the directory containing pathname, then
+ * scans all exclude lists to determine whether pathname is excluded (both
+ * trashable and precious).
* Returns 1 if true, otherwise 0.
*/
int is_excluded(struct dir_struct *dir, struct index_state *istate,
diff --git a/dir.h b/dir.h
index a6977149b8..5424036e4e 100644
--- a/dir.h
+++ b/dir.h
@@ -442,6 +442,10 @@ struct path_pattern *last_matching_pattern(struct dir_struct *dir,
struct index_state *istate,
const char *name, int *dtype);
+int is_trashable(struct dir_struct *dir,
+ struct index_state *istate,
+ const char *name, int *dtype);
+
int is_excluded(struct dir_struct *dir,
struct index_state *istate,
const char *name, int *dtype);
diff --git a/unpack-trees.c b/unpack-trees.c
index 1802809ad3..632fefd260 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -2434,11 +2434,12 @@ static int check_ok_to_remove(const char *name, int len, int dtype,
if (repo_ignore_case(the_repository) && icase_exists(o, name, len, st))
return 0;
+ /* Check if trashable */
if (o->internal.dir &&
- is_excluded(o->internal.dir, o->src_index, name, &dtype))
+ is_trashable(o->internal.dir, o->src_index, name, &dtype))
/*
- * ce->name is explicitly excluded, so it is Ok to
- * overwrite it.
+ * ce->name is explicitly marked as trashable,
+ * so it is Ok to overwrite it.
*/
return 0;
if (S_ISDIR(st->st_mode)) {
--
2.56.0
^ permalink raw reply related [flat|nested] 8+ messages in thread* [RFC PATCH 6/6] builtin/ls-files.c: support for precious files
2026-10-08 12:06 [RFC PATCH 0/6] Introduce precious files Mirko Faina
` (4 preceding siblings ...)
2026-10-08 12:07 ` [RFC PATCH 5/6] unpack-trees: teach check_ok_to_remove() precious Mirko Faina
@ 2026-10-08 12:07 ` Mirko Faina
2026-10-08 12:12 ` [RFC PATCH 0/6] Introduce " Mirko Faina
6 siblings, 0 replies; 8+ messages in thread
From: Mirko Faina @ 2026-10-08 12:07 UTC (permalink / raw)
To: git; +Cc: Mirko Faina, Jeff King, Junio C Hamano, Elijah Newren,
Derrick Stolee
Teach "git ls-files" new variants of "-i" with --ignored=trashable
and --ignored=precious, where git shows only trashable files with the
first and only precious files with the second. If no optional argument
is passed "ls-files" retains the current behaviour.
Signed-off-by: Mirko Faina <mroik@delayed.space>
---
builtin/ls-files.c | 73 ++++++++++++++++++++++++++---
dir.c | 74 ++++++++++++++++++++++++++----
dir.h | 34 +++++++++++++-
t/t2205-add-worktree-config.sh | 2 +-
t/t3001-ls-files-others-exclude.sh | 27 +++++++++++
5 files changed, 190 insertions(+), 20 deletions(-)
diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index d1cc7e92f4..63dd769385 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -163,8 +163,21 @@ static void show_dir_entry(struct index_state *istate,
write_name(ent->name);
}
-static void show_other_files(struct index_state *istate,
- const struct dir_struct *dir)
+static void show_precious_files(struct index_state *istate,
+ const struct dir_struct *dir)
+{
+ int i;
+
+ for (i = 0; i < dir->precious_nr; i++) {
+ struct dir_entry *ent = dir->precious[i];
+ if (!index_name_is_other(istate, ent->name, ent->len))
+ continue;
+ show_dir_entry(istate, tag_other, ent);
+ }
+}
+
+static void show_entries_files(struct index_state *istate,
+ const struct dir_struct *dir)
{
int i;
@@ -176,6 +189,24 @@ static void show_other_files(struct index_state *istate,
}
}
+static void show_ignored_files(struct index_state *istate,
+ const struct dir_struct *dir)
+{
+ show_entries_files(istate, dir);
+ show_precious_files(istate, dir);
+}
+
+static void show_other_files(struct index_state *istate,
+ const struct dir_struct *dir)
+{
+ if (dir->flags & DIR_SHOW_IGNORED)
+ show_ignored_files(istate, dir);
+ else if (dir->flags & DIR_SHOW_PRECIOUS)
+ show_precious_files(istate, dir);
+ else
+ show_entries_files(istate, dir);
+}
+
static void show_killed_files(struct index_state *istate,
const struct dir_struct *dir)
{
@@ -547,6 +578,30 @@ static const char * const ls_files_usage[] = {
NULL
};
+static int option_parse_ignored(const struct option *opt, const char *arg, int unset)
+{
+ enum dir_struct_flags *flags = opt->value;
+
+ BUG_ON_OPT_NEG(unset);
+
+ /*
+ * Reset in case --ignored=<type> is specified multiple times, we keep
+ * the last one.
+ */
+ *flags &= ~(DIR_SHOW_IGNORED | DIR_SHOW_TRASHABLE | DIR_SHOW_PRECIOUS);
+
+ if (!arg)
+ *flags |= DIR_SHOW_IGNORED;
+ else if (!strcmp(arg, "trashable"))
+ *flags |= DIR_SHOW_TRASHABLE;
+ else if (!strcmp(arg, "precious"))
+ *flags |= DIR_SHOW_PRECIOUS;
+ else
+ die(_("'%s' argument is not a valid value"), arg);
+
+ return 0;
+}
+
static int option_parse_exclude(const struct option *opt,
const char *arg, int unset)
{
@@ -615,9 +670,9 @@ int cmd_ls_files(int argc,
N_("show modified files in the output")),
OPT_BOOL('o', "others", &show_others,
N_("show other files in the output")),
- OPT_BIT('i', "ignored", &dir.flags,
+ OPT_CALLBACK_F('i', "ignored", &dir.flags, N_("type"),
N_("show ignored files in the output"),
- DIR_SHOW_IGNORED),
+ PARSE_OPT_OPTARG | PARSE_OPT_NONEG, option_parse_ignored),
OPT_BOOL('s', "stage", &show_stage,
N_("show staged contents' object name in the output")),
OPT_BOOL('k', "killed", &show_killed,
@@ -704,7 +759,9 @@ int cmd_ls_files(int argc,
tag_skip_worktree = "S ";
tag_resolve_undo = "U ";
}
- if (show_modified || show_others || show_deleted || (dir.flags & DIR_SHOW_IGNORED) || show_killed)
+ if (show_modified || show_others || show_deleted ||
+ (dir.flags & (DIR_SHOW_IGNORED | DIR_SHOW_TRASHABLE | DIR_SHOW_PRECIOUS)) ||
+ show_killed)
require_work_tree = 1;
if (show_unmerged)
/*
@@ -753,10 +810,12 @@ int cmd_ls_files(int argc,
if (pathspec.nr && error_unmatch)
ps_matched = xcalloc(pathspec.nr, 1);
- if ((dir.flags & DIR_SHOW_IGNORED) && !show_others && !show_cached)
+ if ((dir.flags & (DIR_SHOW_IGNORED | DIR_SHOW_TRASHABLE | DIR_SHOW_PRECIOUS)) &&
+ !show_others && !show_cached)
die("ls-files -i must be used with either -o or -c");
- if ((dir.flags & DIR_SHOW_IGNORED) && !exc_given)
+ if ((dir.flags & (DIR_SHOW_IGNORED | DIR_SHOW_TRASHABLE | DIR_SHOW_PRECIOUS)) &&
+ !exc_given)
die("ls-files --ignored needs some exclude pattern");
/* With no flags, we default to showing the cached files */
diff --git a/dir.c b/dir.c
index ffc1818533..274a934e12 100644
--- a/dir.c
+++ b/dir.c
@@ -281,10 +281,25 @@ int fill_directory(struct dir_struct *dir,
{
const char *matched_prefix;
size_t prefix_len;
+ unsigned exclusive;
+ int i;
- unsigned exclusive_flags = DIR_SHOW_IGNORED | DIR_SHOW_IGNORED_TOO;
- if ((dir->flags & exclusive_flags) == exclusive_flags)
- BUG("DIR_SHOW_IGNORED and DIR_SHOW_IGNORED_TOO are exclusive");
+ const unsigned exclusive_with_too[] = {
+ DIR_SHOW_IGNORED,
+ DIR_SHOW_TRASHABLE,
+ DIR_SHOW_PRECIOUS,
+ };
+ const char *exclusive_bug_text[] = {
+ "DIR_SHOW_IGNORED",
+ "DIR_SHOW_TRASHABLE",
+ "DIR_SHOW_PRECIOUS",
+ };
+ for (i = 0; i < 3; i++) {
+ exclusive = (exclusive_with_too[i] | DIR_SHOW_IGNORED_TOO);
+ if ((dir->flags & exclusive) == exclusive)
+ BUG("%s and DIR_SHOW_IGNORED_TOO are exclusive",
+ exclusive_bug_text[i]);
+ }
/*
* Calculate common prefix for the pathspec, and
@@ -1859,6 +1874,21 @@ struct path_pattern *last_matching_pattern(struct dir_struct *dir,
basename, dtype_p);
}
+/*
+ * Loads the exclude lists for the directory containing pathname, then
+ * scans all exclude lists to determine whether pathname is precious.
+ * Returns 1 if true, otherwise 0.
+ */
+int is_precious(struct dir_struct *dir, struct index_state *istate,
+ const char *pathname, int *dtype_p)
+{
+ struct path_pattern *pattern =
+ last_matching_pattern(dir, istate, pathname, dtype_p);
+ if (pattern)
+ return !!(pattern->flags & PATTERN_FLAG_PRECIOUS);
+ return 0;
+}
+
/*
* Loads the exclude lists for the directory containing pathname, then
* scans all exclude lists to determine whether pathname is trashable.
@@ -1911,6 +1941,17 @@ static struct dir_entry *dir_add_name(struct dir_struct *dir,
return dir->entries[dir->nr++] = dir_entry_new(pathname, len);
}
+static struct dir_entry *dir_add_precious(struct dir_struct *dir,
+ struct index_state *istate,
+ const char *pathname, int len)
+{
+ if (index_file_exists(istate, pathname, len, repo_ignore_case(the_repository)))
+ return NULL;
+
+ ALLOC_GROW(dir->precious, dir->precious_nr+1, dir->internal.precious_alloc);
+ return dir->precious[dir->precious_nr++] = dir_entry_new(pathname, len);
+}
+
struct dir_entry *dir_add_ignored(struct dir_struct *dir,
struct index_state *istate,
const char *pathname, int len)
@@ -2528,13 +2569,18 @@ static enum path_treatment treat_path(struct dir_struct *dir,
(directory_exists_in_index(istate, path->buf, path->len) == index_nonexistent))
return path_none;
- excluded = is_excluded(dir, istate, path->buf, &dtype);
+ excluded = is_excluded(dir, istate, path->buf, &dtype) ||
+ is_trashable(dir, istate, path->buf, &dtype) ||
+ is_precious(dir, istate, path->buf, &dtype);
/*
* Excluded? If we don't explicitly want to show
* ignored files, ignore it
*/
- if (excluded && !(dir->flags & (DIR_SHOW_IGNORED|DIR_SHOW_IGNORED_TOO)))
+ if (excluded && !(dir->flags & (DIR_SHOW_IGNORED|
+ DIR_SHOW_TRASHABLE|
+ DIR_SHOW_PRECIOUS|
+ DIR_SHOW_IGNORED_TOO)))
return path_excluded;
switch (dtype) {
@@ -2704,20 +2750,28 @@ static void add_path_to_appropriate_result_list(struct dir_struct *dir,
const struct pathspec *pathspec,
enum path_treatment state)
{
+ int dtype;
+
/* add the path to the appropriate result list */
switch (state) {
case path_excluded:
- if (dir->flags & DIR_SHOW_IGNORED)
- dir_add_name(dir, istate, path->buf, path->len);
- else if ((dir->flags & DIR_SHOW_IGNORED_TOO) ||
+ if (dir->flags & (DIR_SHOW_IGNORED | DIR_SHOW_TRASHABLE | DIR_SHOW_PRECIOUS)) {
+ dtype = resolve_dtype(cdir->d_type, istate, path->buf, path->len);
+ if (is_precious(dir, istate, path->buf, &dtype))
+ dir_add_precious(dir, istate, path->buf, path->len);
+ else
+ dir_add_name(dir, istate, path->buf, path->len);
+ } else if ((dir->flags & DIR_SHOW_IGNORED_TOO) ||
((dir->flags & DIR_COLLECT_IGNORED) &&
exclude_matches_pathspec(path->buf, path->len,
- pathspec)))
+ pathspec))) {
dir_add_ignored(dir, istate, path->buf, path->len);
+ }
break;
case path_untracked:
- if (dir->flags & DIR_SHOW_IGNORED)
+ if (dir->flags & (DIR_SHOW_IGNORED | DIR_SHOW_TRASHABLE |
+ DIR_SHOW_PRECIOUS))
break;
dir_add_name(dir, istate, path->buf, path->len);
if (cdir->fdir)
diff --git a/dir.h b/dir.h
index 5424036e4e..58e5467cda 100644
--- a/dir.h
+++ b/dir.h
@@ -212,14 +212,30 @@ struct untracked_cache {
struct dir_struct {
/* bit-field of options */
- enum {
+ enum dir_struct_flags {
/**
* Return just ignored files in `entries[]`, not untracked files.
* This flag is mutually exclusive with `DIR_SHOW_IGNORED_TOO`.
+ * This is a superset of DIR_SHOW_TRASHABLE and DIR_SHOW_PRECIOUS.
+ *
+ * This flag should probably be dropped throughout the codebase
+ * in favour of (DIR_SHOW_TRASHABLE | DIR_SHOW_PRECIOUS).
*/
DIR_SHOW_IGNORED = 1<<0,
+ /**
+ * Return just trashable files in `entries[]`, not untracked files.
+ * This flag is mutually exclusive with `DIR_SHOW_IGNORED_TOO`.
+ */
+ DIR_SHOW_TRASHABLE = 1<<10,
+
+ /**
+ * Return just precious files in `entries[]`, not untracked files.
+ * This flag is mutually exclusive with `DIR_SHOW_IGNORED_TOO`.
+ */
+ DIR_SHOW_PRECIOUS = 1<<11,
+
/* Include a directory that is not tracked. */
DIR_SHOW_OTHER_DIRECTORIES = 1<<1,
@@ -243,7 +259,8 @@ struct dir_struct {
/**
* Similar to `DIR_SHOW_IGNORED`, but return ignored files in
* `ignored[]` in addition to untracked files in `entries[]`.
- * This flag is mutually exclusive with `DIR_SHOW_IGNORED`.
+ * This flag is mutually exclusive with `DIR_SHOW_IGNORED`,
+ * `DIR_SHOW_TRASHABLE` and `DIR_SHOW_PRECIOUS`.
*/
DIR_SHOW_IGNORED_TOO = 1<<5,
@@ -278,12 +295,18 @@ struct dir_struct {
/* The number of members in `entries[]` array. */
int nr; /* output only */
+ /* The number of members in `precious[]` array. */
+ int precious_nr; /* output only */
+
/* The number of members in `ignored[]` array. */
int ignored_nr; /* output only */
/* An array of `struct dir_entry`, each element of which describes a path. */
struct dir_entry **entries; /* output only */
+ /* Used for listing precious files with `DIR_SHOW_PRECIOUS`. */
+ struct dir_entry **precious; /* output only */
+
/**
* used for ignored paths with the `DIR_SHOW_IGNORED_TOO` and
* `DIR_COLLECT_IGNORED` flags.
@@ -307,6 +330,9 @@ struct dir_struct {
/* Keeps track of allocation of `entries[]` array.*/
int alloc;
+ /* Keeps track of allocation of `precious[]` array.*/
+ int precious_alloc;
+
/* Keeps track of allocation of `ignored[]` array. */
int ignored_alloc;
@@ -442,6 +468,10 @@ struct path_pattern *last_matching_pattern(struct dir_struct *dir,
struct index_state *istate,
const char *name, int *dtype);
+int is_precious(struct dir_struct *dir,
+ struct index_state *istate,
+ const char *name, int *dtype);
+
int is_trashable(struct dir_struct *dir,
struct index_state *istate,
const char *name, int *dtype);
diff --git a/t/t2205-add-worktree-config.sh b/t/t2205-add-worktree-config.sh
index 43d950de64..a743470196 100755
--- a/t/t2205-add-worktree-config.sh
+++ b/t/t2205-add-worktree-config.sh
@@ -244,7 +244,7 @@ test_expect_success '3a: setup--add repo dir' '
test_expect_success '3b: ignored' '
(
cd test3 &&
- git --git-dir=repo/.git ls-files -io --directory --exclude-standard >actual-ignored-unsorted &&
+ git --git-dir=repo/.git ls-files -i -o --directory --exclude-standard >actual-ignored-unsorted &&
sort actual-ignored-unsorted >actual-ignored &&
sort expect-ignored-unsorted >expect-ignored &&
test_cmp expect-ignored actual-ignored
diff --git a/t/t3001-ls-files-others-exclude.sh b/t/t3001-ls-files-others-exclude.sh
index 29a0a25b30..977cbd6b6e 100755
--- a/t/t3001-ls-files-others-exclude.sh
+++ b/t/t3001-ls-files-others-exclude.sh
@@ -55,6 +55,7 @@ expect
!*.8' >.git/ignore
echo '*.1
+$wasder
/*.3
!*.6' >.gitignore
echo '*.2
@@ -76,6 +77,32 @@ test_expect_success 'git ls-files --others with various exclude options.' '
test_cmp expect output
'
+test_expect_success 'git ls-files -o -i' '
+ touch wasder &&
+ git ls-files -o -i --exclude-standard >output &&
+ cat output &&
+ test_grep "wasder" output
+'
+
+test_expect_success 'git ls-files -o --ignored=trashable' '
+ git ls-files -o --ignored=trashable --exclude-standard >output &&
+ cat output &&
+ test_grep ! "wasder" output
+'
+
+test_expect_success 'git ls-files -o --ignored=precious' '
+ git ls-files -o --ignored=precious --exclude-standard >output &&
+ cat output &&
+ test_grep "wasder" output
+'
+
+test_expect_success 'git ls-files -o --ignored=trashable' '
+ test_when_finished rm wasder &&
+ git ls-files -o --ignored=precious --ignored=trashable --exclude-standard >output &&
+ cat output &&
+ test_grep ! "wasder" output
+'
+
# Test \r\n (MSDOS-like systems)
printf '*.1\r\n/*.3\r\n!*.6\r\n' >.gitignore
--
2.56.0
^ permalink raw reply related [flat|nested] 8+ messages in thread* Re: [RFC PATCH 0/6] Introduce precious files
2026-10-08 12:06 [RFC PATCH 0/6] Introduce precious files Mirko Faina
` (5 preceding siblings ...)
2026-10-08 12:07 ` [RFC PATCH 6/6] builtin/ls-files.c: support for precious files Mirko Faina
@ 2026-10-08 12:12 ` Mirko Faina
6 siblings, 0 replies; 8+ messages in thread
From: Mirko Faina @ 2026-10-08 12:12 UTC (permalink / raw)
To: git; +Cc: Jeff King, Junio C Hamano, Elijah Newren, Derrick Stolee,
Mirko Faina
On Thu, Oct 08, 2026 at 02:06:56PM +0200, Mirko Faina wrote:
> Introduce precious files based on Elijah Newren's document [1].
>
> With this series documentation is not updated yet but I wanted some feedback
> before polishing and moving on to implement the changes for "git clean".
>
> [1] https://lore.kernel.org/git/pull.1627.git.1703643931314.gitgitgadget@gmail.com/
>
> [1/6] precious-files.txt: new document proposing new precious file type (Elijah Newren)
> [2/6] dir.h: replace pattern macros with enum in attr.h (Mirko Faina)
> [3/6] dir.c: teach parse_path_pattern() precious files (Mirko Faina)
> [4/6] dir.c: teach add_pattern() reject precious pattern (Mirko Faina)
> [5/6] unpack-trees: teach check_ok_to_remove() precious (Mirko Faina)
> [6/6] builtin/ls-files.c: support for precious files (Mirko Faina)
>
> Documentation/technical/precious-files.txt | 540 +++++++++++++++++++++
> attr.c | 8 +-
> attr.h | 10 +-
> builtin/check-ignore.c | 1 +
> builtin/clean.c | 4 +-
> builtin/ls-files.c | 75 ++-
> builtin/sparse-checkout.c | 16 +-
> dir.c | 144 +++++-
> dir.h | 51 +-
> t/helper/test-path-walk.c | 2 +-
> t/t1091-sparse-checkout-builtin.sh | 8 +
> t/t2205-add-worktree-config.sh | 2 +-
> t/t3001-ls-files-others-exclude.sh | 27 ++
> t/t7508-status.sh | 9 +
> unpack-trees.c | 7 +-
> 15 files changed, 849 insertions(+), 55 deletions(-)
> create mode 100644 Documentation/technical/precious-files.txt
>
> --
> 2.56.0
I forgot to specify the base. The series is based on c46c1e3772 (Start Git 2.98
cycle, 2026-09-30).
^ permalink raw reply [flat|nested] 8+ messages in thread