[PATCH v3 1/2] .gitignore: update the command to check tracked files being ignored

[PATCH v3 1/2] .gitignore: update the command to check tracked files being ignored

[Masahiro Yamada]

Recent git versions do not accept the noted command.

  $ git ls-files -i --exclude-standard
  fatal: ls-files -i must be used with either -o or -c

The -c was implied before, but we need to make it explicit since
git commit b338e9f66873 ("ls-files: error out on -i unless -o or -c
are specified").

Also, replace --exclude-standard with --exclude-per-directory=.gitignore
so that everyone will get consistent results.

git-ls-files(1) says:

  --exclude-standard
      Add the standard Git exclusions: .git/info/exclude, .gitignore in
      each directory, and the user's global exclusion file.

We cannot predict what is locally added to .git/info/exclude or the
user's global exclusion file.

We can only manage .gitignore files committed to the repository.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Miguel Ojeda <ojeda@kernel.org>
---

Changes in v3:
  - Fix a typo. Update commit description per Miguel

 .gitignore | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.gitignore b/.gitignore
index 3ec73ead6757..2e2e3d1eeaee 100644
--- a/.gitignore
+++ b/.gitignore
@@ -4,7 +4,7 @@
 # subdirectories here. Add them in the ".gitignore" file
 # in that subdirectory instead.
 #
-# NOTE! Please use 'git ls-files -i --exclude-standard'
+# NOTE! Please use 'git ls-files -i -c --exclude-per-directory=.gitignore'
 # command after changing this file, to see if there are
 # any tracked files which get ignored after the change.
 #
-- 
2.34.1

[PATCH v3 2/2] kbuild: make W=1 warn files that are tracked but ignored by git

[Masahiro Yamada]

The top .gitignore comments about how to detect files breaking
.gitignore rules, but people rarely care about it.

Add a new W=1 warning to detect files that are tracked but ignored by
git. If git is not installed or the source tree is not tracked by git
at all, this script does not print anything.

Running it on v6.2-rc1 detected the following:

  $ make W=1 misc-check
  Documentation/devicetree/bindings/.yamllint: warning: ignored by one of the .gitignore files
  drivers/clk/.kunitconfig: warning: ignored by one of the .gitignore files
  drivers/gpu/drm/tests/.kunitconfig: warning: ignored by one of the .gitignore files
  drivers/hid/.kunitconfig: warning: ignored by one of the .gitignore files
  fs/ext4/.kunitconfig: warning: ignored by one of the .gitignore files
  fs/fat/.kunitconfig: warning: ignored by one of the .gitignore files
  kernel/kcsan/.kunitconfig: warning: ignored by one of the .gitignore files
  lib/kunit/.kunitconfig: warning: ignored by one of the .gitignore files
  mm/kfence/.kunitconfig: warning: ignored by one of the .gitignore files
  tools/testing/selftests/arm64/tags/.gitignore: warning: ignored by one of the .gitignore files
  tools/testing/selftests/arm64/tags/Makefile: warning: ignored by one of the .gitignore files
  tools/testing/selftests/arm64/tags/run_tags_test.sh: warning: ignored by one of the .gitignore files
  tools/testing/selftests/arm64/tags/tags_test.c: warning: ignored by one of the .gitignore files

These are ignored by the '.*' or 'tags' in the top .gitignore, but
there is no rule to negate it.

You might be tempted to do 'git add -f' but I want to have the real
issue fixed (by fixing a .gitignore, or by renaming files, etc.).

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
---

Changes in v3:
  - change working directory to srctree (Nicolas)

Changes in v2:
  - Add $(srctree)/ to make it work with O=

 Makefile           |  6 ++++++
 scripts/misc-check | 19 +++++++++++++++++++
 2 files changed, 25 insertions(+)
 create mode 100755 scripts/misc-check

diff --git a/Makefile b/Makefile
index acce5ec514f6..c0d7c75d8f14 100644
--- a/Makefile
+++ b/Makefile
@@ -1848,6 +1848,12 @@ rust-analyzer:
 # Misc
 # ---------------------------------------------------------------------------
 
+PHONY += misc-check
+misc-check:
+	$(Q)$(srctree)/scripts/misc-check
+
+all: misc-check
+
 PHONY += scripts_gdb
 scripts_gdb: prepare0
 	$(Q)$(MAKE) $(build)=scripts/gdb
diff --git a/scripts/misc-check b/scripts/misc-check
new file mode 100755
index 000000000000..d40d5484e0c5
--- /dev/null
+++ b/scripts/misc-check
@@ -0,0 +1,19 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0-only
+
+set -e
+
+# Detect files that are tracked but ignored by git. This is checked only when
+# ${KBUILD_EXTRA_WARN} contains 1, git is installed, and the source tree is
+# tracked by git.
+check_tracked_ignored_files () {
+	case "${KBUILD_EXTRA_WARN}" in
+	*1*) ;;
+	*) return;;
+	esac
+
+	git -C ${srctree:-.} ls-files -i -c --exclude-per-directory=.gitignore 2>/dev/null |
+		sed 's/$/: warning: ignored by one of the .gitignore files/' >&2
+}
+
+check_tracked_ignored_files
-- 
2.34.1

Re: [PATCH v3 2/2] kbuild: make W=1 warn files that are tracked but ignored by git

[Nicolas Schier]

[-- Attachment #1: Type: text/plain, Size: 2500 bytes --]

On Thu, Dec 29, 2022 at 04:43:10PM +0900 Masahiro Yamada wrote:
> The top .gitignore comments about how to detect files breaking
> .gitignore rules, but people rarely care about it.
> 
> Add a new W=1 warning to detect files that are tracked but ignored by
> git. If git is not installed or the source tree is not tracked by git
> at all, this script does not print anything.
> 
> Running it on v6.2-rc1 detected the following:
> 
>   $ make W=1 misc-check
>   Documentation/devicetree/bindings/.yamllint: warning: ignored by one of the .gitignore files
>   drivers/clk/.kunitconfig: warning: ignored by one of the .gitignore files
>   drivers/gpu/drm/tests/.kunitconfig: warning: ignored by one of the .gitignore files
>   drivers/hid/.kunitconfig: warning: ignored by one of the .gitignore files
>   fs/ext4/.kunitconfig: warning: ignored by one of the .gitignore files
>   fs/fat/.kunitconfig: warning: ignored by one of the .gitignore files
>   kernel/kcsan/.kunitconfig: warning: ignored by one of the .gitignore files
>   lib/kunit/.kunitconfig: warning: ignored by one of the .gitignore files
>   mm/kfence/.kunitconfig: warning: ignored by one of the .gitignore files
>   tools/testing/selftests/arm64/tags/.gitignore: warning: ignored by one of the .gitignore files
>   tools/testing/selftests/arm64/tags/Makefile: warning: ignored by one of the .gitignore files
>   tools/testing/selftests/arm64/tags/run_tags_test.sh: warning: ignored by one of the .gitignore files
>   tools/testing/selftests/arm64/tags/tags_test.c: warning: ignored by one of the .gitignore files
> 
> These are ignored by the '.*' or 'tags' in the top .gitignore, but
> there is no rule to negate it.
> 
> You might be tempted to do 'git add -f' but I want to have the real
> issue fixed (by fixing a .gitignore, or by renaming files, etc.).
> 
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> Reviewed-by: Nathan Chancellor <nathan@kernel.org>
> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
> ---
> 
> Changes in v3:
>   - change working directory to srctree (Nicolas)
> 
> Changes in v2:
>   - Add $(srctree)/ to make it work with O=
> 
>  Makefile           |  6 ++++++
>  scripts/misc-check | 19 +++++++++++++++++++
>  2 files changed, 25 insertions(+)
>  create mode 100755 scripts/misc-check

Tested-by: Nicolas Schier <nicolas@fjasle.eu>

out of curiosity: do you plan to implement more checks in the misc-checks
target?

Kind regards,
Nicolas


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v3 2/2] kbuild: make W=1 warn files that are tracked but ignored by git

[Masahiro Yamada]

On Thu, Dec 29, 2022 at 11:23 PM Nicolas Schier <nicolas@fjasle.eu> wrote:
>
> On Thu, Dec 29, 2022 at 04:43:10PM +0900 Masahiro Yamada wrote:
> > The top .gitignore comments about how to detect files breaking
> > .gitignore rules, but people rarely care about it.
> >
> > Add a new W=1 warning to detect files that are tracked but ignored by
> > git. If git is not installed or the source tree is not tracked by git
> > at all, this script does not print anything.
> >
> > Running it on v6.2-rc1 detected the following:
> >
> >   $ make W=1 misc-check
> >   Documentation/devicetree/bindings/.yamllint: warning: ignored by one of the .gitignore files
> >   drivers/clk/.kunitconfig: warning: ignored by one of the .gitignore files
> >   drivers/gpu/drm/tests/.kunitconfig: warning: ignored by one of the .gitignore files
> >   drivers/hid/.kunitconfig: warning: ignored by one of the .gitignore files
> >   fs/ext4/.kunitconfig: warning: ignored by one of the .gitignore files
> >   fs/fat/.kunitconfig: warning: ignored by one of the .gitignore files
> >   kernel/kcsan/.kunitconfig: warning: ignored by one of the .gitignore files
> >   lib/kunit/.kunitconfig: warning: ignored by one of the .gitignore files
> >   mm/kfence/.kunitconfig: warning: ignored by one of the .gitignore files
> >   tools/testing/selftests/arm64/tags/.gitignore: warning: ignored by one of the .gitignore files
> >   tools/testing/selftests/arm64/tags/Makefile: warning: ignored by one of the .gitignore files
> >   tools/testing/selftests/arm64/tags/run_tags_test.sh: warning: ignored by one of the .gitignore files
> >   tools/testing/selftests/arm64/tags/tags_test.c: warning: ignored by one of the .gitignore files
> >
> > These are ignored by the '.*' or 'tags' in the top .gitignore, but
> > there is no rule to negate it.
> >
> > You might be tempted to do 'git add -f' but I want to have the real
> > issue fixed (by fixing a .gitignore, or by renaming files, etc.).
> >
> > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> > Reviewed-by: Nathan Chancellor <nathan@kernel.org>
> > Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
> > ---
> >
> > Changes in v3:
> >   - change working directory to srctree (Nicolas)
> >
> > Changes in v2:
> >   - Add $(srctree)/ to make it work with O=
> >
> >  Makefile           |  6 ++++++
> >  scripts/misc-check | 19 +++++++++++++++++++
> >  2 files changed, 25 insertions(+)
> >  create mode 100755 scripts/misc-check
>
> Tested-by: Nicolas Schier <nicolas@fjasle.eu>
>
> out of curiosity: do you plan to implement more checks in the misc-checks
> target?


I chose a generic name so I can put more in this script, but
honestly I have nothing else in my mind for now.




>
> Kind regards,
> Nicolas
>


-- 
Best Regards
Masahiro Yamada

Re: [PATCH v3 2/2] kbuild: make W=1 warn files that are tracked but ignored by git

[Andy Shevchenko]

On Thu, Dec 29, 2022 at 04:43:10PM +0900, Masahiro Yamada wrote:
> The top .gitignore comments about how to detect files breaking
> .gitignore rules, but people rarely care about it.
> 
> Add a new W=1 warning to detect files that are tracked but ignored by
> git. If git is not installed or the source tree is not tracked by git
> at all, this script does not print anything.
> 
> Running it on v6.2-rc1 detected the following:

Since patch was published there is no sign it was ever meet Linux Next.
What's the plan?

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH v3 2/2] kbuild: make W=1 warn files that are tracked but ignored by git

[Masahiro Yamada]

On Fri, Jan 27, 2023 at 10:25 PM Andy Shevchenko
<andriy.shevchenko@intel.com> wrote:
>
> On Thu, Dec 29, 2022 at 04:43:10PM +0900, Masahiro Yamada wrote:
> > The top .gitignore comments about how to detect files breaking
> > .gitignore rules, but people rarely care about it.
> >
> > Add a new W=1 warning to detect files that are tracked but ignored by
> > git. If git is not installed or the source tree is not tracked by git
> > at all, this script does not print anything.
> >
> > Running it on v6.2-rc1 detected the following:
>
> Since patch was published there is no sign it was ever meet Linux Next.
> What's the plan?


Oh?
I can see this patch in linux-next.


$ git log next-20230127 -- scripts/misc-check


-- 
Best Regards
Masahiro Yamada

Re: [PATCH v3 2/2] kbuild: make W=1 warn files that are tracked but ignored by git

[Andy Shevchenko]

On Fri, Jan 27, 2023 at 11:31:07PM +0900, Masahiro Yamada wrote:
> On Fri, Jan 27, 2023 at 10:25 PM Andy Shevchenko
> <andriy.shevchenko@intel.com> wrote:
> >
> > On Thu, Dec 29, 2022 at 04:43:10PM +0900, Masahiro Yamada wrote:
> > > The top .gitignore comments about how to detect files breaking
> > > .gitignore rules, but people rarely care about it.
> > >
> > > Add a new W=1 warning to detect files that are tracked but ignored by
> > > git. If git is not installed or the source tree is not tracked by git
> > > at all, this script does not print anything.
> > >
> > > Running it on v6.2-rc1 detected the following:
> >
> > Since patch was published there is no sign it was ever meet Linux Next.
> > What's the plan?
> 
> Oh?

Sorry, my mistake. I need to understand why these patches do not fix
the issue I have.

> I can see this patch in linux-next.
> 
> $ git log next-20230127 -- scripts/misc-check

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH v3 2/2] kbuild: make W=1 warn files that are tracked but ignored by git

[Andy Shevchenko]

On Fri, Jan 27, 2023 at 04:41:37PM +0200, Andy Shevchenko wrote:
> On Fri, Jan 27, 2023 at 11:31:07PM +0900, Masahiro Yamada wrote:
> > On Fri, Jan 27, 2023 at 10:25 PM Andy Shevchenko
> > <andriy.shevchenko@intel.com> wrote:
> > >
> > > On Thu, Dec 29, 2022 at 04:43:10PM +0900, Masahiro Yamada wrote:
> > > > The top .gitignore comments about how to detect files breaking
> > > > .gitignore rules, but people rarely care about it.
> > > >
> > > > Add a new W=1 warning to detect files that are tracked but ignored by
> > > > git. If git is not installed or the source tree is not tracked by git
> > > > at all, this script does not print anything.
> > > >
> > > > Running it on v6.2-rc1 detected the following:
> > >
> > > Since patch was published there is no sign it was ever meet Linux Next.
> > > What's the plan?
> > 
> > Oh?
> 
> Sorry, my mistake. I need to understand why these patches do not fix
> the issue I have.

OK, after carefully reading the commit message it's actually the culprit of
the warnings I have.

So, it seems we need to wait maintainers / developers of the respective code
to go and fix this. Is it your intention?

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH v3 2/2] kbuild: make W=1 warn files that are tracked but ignored by git

[Masahiro Yamada]

On Fri, Jan 27, 2023 at 11:50 PM Andy Shevchenko
<andriy.shevchenko@intel.com> wrote:
>
> On Fri, Jan 27, 2023 at 04:41:37PM +0200, Andy Shevchenko wrote:
> > On Fri, Jan 27, 2023 at 11:31:07PM +0900, Masahiro Yamada wrote:
> > > On Fri, Jan 27, 2023 at 10:25 PM Andy Shevchenko
> > > <andriy.shevchenko@intel.com> wrote:
> > > >
> > > > On Thu, Dec 29, 2022 at 04:43:10PM +0900, Masahiro Yamada wrote:
> > > > > The top .gitignore comments about how to detect files breaking
> > > > > .gitignore rules, but people rarely care about it.
> > > > >
> > > > > Add a new W=1 warning to detect files that are tracked but ignored by
> > > > > git. If git is not installed or the source tree is not tracked by git
> > > > > at all, this script does not print anything.
> > > > >
> > > > > Running it on v6.2-rc1 detected the following:
> > > >
> > > > Since patch was published there is no sign it was ever meet Linux Next.
> > > > What's the plan?
> > >
> > > Oh?
> >
> > Sorry, my mistake. I need to understand why these patches do not fix
> > the issue I have.
>
> OK, after carefully reading the commit message it's actually the culprit of
> the warnings I have.
>
> So, it seems we need to wait maintainers / developers of the respective code
> to go and fix this. Is it your intention?


Yes.
I expect the 0day bot will block new breakages,
but actually the mainline got more warnings
under kselftest.




-- 
Best Regards
Masahiro Yamada