Lack of system-level excludesFile

Lack of system-level excludesFile

[Sparr]

I found myself in a discussion online about where and how it is
appropriate to ask git to ignore the .DS_Store file that is often
created by macOS. This led me to considering disparities in the
current state of git-config and gitignore. In particular, git-config
has system-level configuration in $(prefix)/etc/gitconfig which can be
overridden on a line by line basis by configuration at the user,
repository, or command levels. However, gitignore does not have an
equivalent system-level layer. You can set core.excludesfile in the
system-level config file, but the user can only override that whole
file by changing that config option, they have no way to override
single exclude entries or even just add to the list for themselves at
the user ("global") level.

I think that a system-level ignore/exclude file would be the ideal
place for the git package on a particular OS to put OS-specific rules
like the following examples, if they wanted to do so, or for a
sysadmin or root user to do the same to cover all local accounts by
default if their OS package opted to not provide any exclude rules.
https://www.toptal.com/developers/gitignore/api/macos
https://www.toptal.com/developers/gitignore/api/windows
https://www.toptal.com/developers/gitignore/api/linux

Was the decision to not allow or implement such a file intentional, or
is it just an emergent property of the way the config system works and
how core.excludesFile was implemented? Would an implementation of a
new feature supporting this sort of thing be worth discussing?

Re: Lack of system-level excludesFile

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1596 bytes --]

On 2024-10-03 at 00:23:17, Sparr wrote:
> Was the decision to not allow or implement such a file intentional, or
> is it just an emergent property of the way the config system works and
> how core.excludesFile was implemented? Would an implementation of a
> new feature supporting this sort of thing be worth discussing?

I wasn't part of the project when the exclude functionality was
implemented, so I can't say what the intent was, but I will note that
even CVS had ignore files.  My guess is that it wasn't seen as something
useful to implement, so nobody implemented it, but I think your argument
for doing so isn't bad.

I'll just note that typically I ignore things in the global include
which are likely to affect the environment I use, including my editor
and operating system.  So when I was tutoring an Emacs user, we
configured his machine to ignore `*~`, since his editor creates backup
files, and likewise, if I were still using macOS, I'd ignore
`.DS_Store`.

So basically, I think such a feature could be useful, but I also don't
see it as immediately necessary because the global (per-user) file is
also generally useful for this if the user is aware enough to know it
exists (which I admit they might not be).  One thing I would request is
that if you add this, you add an appropriate var to `git var` so that
people can find the location, which is important because different
versions of Git (say, Apple's vs. Homebrew's) may install the system
configuration in different locations.
-- 
brian m. carlson (they/them or he/him)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: Lack of system-level excludesFile

[Junio C Hamano]

"brian m. carlson" <sandals@crustytoothpaste.net> writes:

> So basically, I think such a feature could be useful, but I also don't
> see it as immediately necessary because the global (per-user) file is
> also generally useful for this if the user is aware enough to know it
> exists (which I admit they might not be).

I was around ;-) and the reasoning is exactly that.  

Per-user setting was considered the most appropriate (not "good
enough", but "more appropriate than system-wide") primarily because
the exclude list generally is just as personal as it is personal
what editor is used.

The same reason led to the thinking that it is perfectly fine to
have "*~" in the list of per-user exclude patterns for an Emacs user
but it is dubious to have in the list of system-wide exclude
patterns, if such a thing existed, when the system is used also by a
vi user.  I do not think it was explicitly argued, but it is natural
to lead to the conclusion that it was not worth adding such a
mechanism.

And that led to recommendation that these patterns of personal
nature should be left out of in-tree .gitignore file.  The Makefile
and the project source would dictate what build artifacts are
expected to be created in the working tree of a checkout of a
project, and in-tree .gitignore is the perfect mechanism to mark
these *.o, *.a, and *.pyo files to be ignored, but because not
everybody who work on the project uses Emacs, so "*~" falls into a
different category from these "usual build artifacts" and the
recommendation was to leave these outside the in-tree (or the list
of system-wide patterns, if we had a mechanism to specify such a
thing) .gitignore files.

But this thinking and recommendation changed over time, and I think
people (including us old-timers) take a more relaxed stance these
days.

If the contributor base of the project includes both Emacs and vi
users (and in this day and age of DEI, you are encourged to be
"inclusive" and prepare for the folks from other camps to join the
project in the future, even if your contributor base is solely vi
users), it will not be likely that the project wants to have any
files that match "*~" to be tracked at all, so having such a pattern
in in-tree .gitignore file to be shared by all participants would
not hurt anybody.  Emacs users will be helped by not having to worry
about configuring their per-user configuration, and vi users won't
be hurt as they are not going to add a tracked file foo~ to the
project (and alienate their friends in the project who happen to use
Emacs).

If you take the reasoning one step further [*], it may make sense to
have a list of ignored patterns that applies to everybody on the
system, regardless of who they are, what editor they use, and what
project they work on.

Having said all that, wouldn't that be already supported?  What
prevents you to set core.excludesFile in /etc/gitconfig?

Thanks.


[Footnote]

 * This may be a larger step though, as the set of projects that a
   single user works on is much narrower than the set of projects
   that any users on the same system work on (hence the types of
   files that are potentially tracked by these projects are much
   wider for system-wide exclude list to take into consideration).

   On the other hand, are massively multi-user systems (like the
   time-shared system studends 30-years ago used at Universities)
   still a thing?  If not, then having a pattern that may hurt
   people with other preference on the system-wide list would not be
   a problem at all---after all, you have the entire system without
   anybody else.  But at that point, system-wide and per-user would
   become the same thing ;-)

Re: Lack of system-level excludesFile

[Sparr]

On Thu, Oct 3, 2024 at 11:40 AM Junio C Hamano <gitster@pobox.com> wrote:
> If you take the reasoning one step further [*], it may make sense to
> have a list of ignored patterns that applies to everybody on the
> system, regardless of who they are, what editor they use, and what
> project they work on.
>
> Having said all that, wouldn't that be already supported?  What
> prevents you to set core.excludesFile in /etc/gitconfig?

I tried to cover this in my original email. You can set that and it
will apply to all users who haven't set up their own
core.excludesFile. But this has two problems.

1. It will disable the default behavior of a user being able to put
their own ignore rules in $XDG_CONFIG_HOME/git/ignore which I suspect
more than a few current git users rely on.

2. As soon as a user sets core.excludesFile in their account-level
config, to be able to create account-level ignore rules, such as based
on their desktop environment or editor or all the other stuff people
typically want to put in their excludesFile, they will lose the
functionality provided by the system level ignore file.

>  * This may be a larger step though, as the set of projects that a
>    single user works on is much narrower than the set of projects
>    that any users on the same system work on (hence the types of
>    files that are potentially tracked by these projects are much
>    wider for system-wide exclude list to take into consideration).

I agree that these two scopes are very different, and that's why I
want to be able to set them both! ~Every user on a macOS computer
(which might be a few on a family desktop, and was dozens last time I
managed a macOS server) wants to ignore `.DS_Store`, `.AppleDouble`,
and `.LSOverride` in ~every repo they work on.

>    On the other hand, are massively multi-user systems (like the
>    time-shared system studends 30-years ago used at Universities)
>    still a thing?  If not, then having a pattern that may hurt
>    people with other preference on the system-wide list would not be
>    a problem at all---after all, you have the entire system without
>    anybody else.  But at that point, system-wide and per-user would
>    become the same thing ;-)

That's not quite true even on single-user computers. It's pretty
common that one user might run things as their user account or as
root. It's less common but not unheard of for one user to have
multiple accounts (e.g. one for work and one for not work) or for
non-user accounts to perform git operations.

Re: Lack of system-level excludesFile

[Jeff King]

On Thu, Oct 03, 2024 at 01:45:07PM -0400, Sparr wrote:

> On Thu, Oct 3, 2024 at 11:40 AM Junio C Hamano <gitster@pobox.com> wrote:
> > If you take the reasoning one step further [*], it may make sense to
> > have a list of ignored patterns that applies to everybody on the
> > system, regardless of who they are, what editor they use, and what
> > project they work on.
> >
> > Having said all that, wouldn't that be already supported?  What
> > prevents you to set core.excludesFile in /etc/gitconfig?
> 
> I tried to cover this in my original email. You can set that and it
> will apply to all users who haven't set up their own
> core.excludesFile. But this has two problems.
> 
> 1. It will disable the default behavior of a user being able to put
> their own ignore rules in $XDG_CONFIG_HOME/git/ignore which I suspect
> more than a few current git users rely on.
> 
> 2. As soon as a user sets core.excludesFile in their account-level
> config, to be able to create account-level ignore rules, such as based
> on their desktop environment or editor or all the other stuff people
> typically want to put in their excludesFile, they will lose the
> functionality provided by the system level ignore file.

Yeah, I agree that the current system is not quite flexible enough. And
there is some prior art already in /etc/gitattributes, which we respect
in addition to core.attributesFile.

I did wonder briefly if we could let core.excludesFile be a multi-valued
list (so you could add a user one on top of the system value). But that
would break backwards compatibility for people who expect the current
last-one-wins behavior. And also would interact weirdly with the XDG
fallback, as you noted.

So I think it is reasonable to add a system-level version of gitignore.
I'm not sure how often it would be used, but it certainly fills a gap
and would make it consistent with config and attributes files. You can
grep for ETC_GITATTRIBUTES to see how the attributes version is
configured and used.

-Peff

PS If you want to get really wild, consider this: the exclude/ignore
   feature is really just a proper subset of the attributes system that
   came later. If we were designing today, we could ditch .gitignore
   entirely in favor of a special "ignored" attribute, and then you
   could just use /etc/gitattributes to do what you want.

   I'm not sure if people would find that more syntactically awkward; it
   is nice that you can just dump files one per line into .gitignore
   without special syntax. But it's possible that we could provide
   _both_ mechanisms, and let the user do whichever they find easier.

   That's obviously a much bigger project than just adding an
   /etc/gitignore file, though. :)

Re: Lack of system-level excludesFile

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> PS If you want to get really wild, consider this: the exclude/ignore
>    feature is really just a proper subset of the attributes system that
>    came later. If we were designing today, we could ditch .gitignore
>    entirely in favor of a special "ignored" attribute,

Yes, this was brought up in the past a few times.

Both the exclude stack (in dir.c) and attr stack (in attr.c) use a
similar approach to optimize the accesses to the data for callers
that traverse the paths in-order and ask if something is ignored (or
has this attribute) for each of the paths they encounter, so their
performance characteristics might be similar.  It certainly is a
tempting thought and indeed is a big project, especially if you want
to keep some sort of backward compatibility ;-)