[PATCH 0/8] Some build system improvements

[PATCH 0/8] Some build system improvements

[Patrick Steinhardt]

Hi,

this patch series contains a small set of build system improvements:

  - The first couple patches introduce a new "tools/" directory that
    contains items related to our build infrastructure and to our
    developer tooling. This finally follows up on my promise to do this
    back when I did the spring clean of "contrib/". [1]

  - The last couple patches introduce precompiled headers into Meson for
    a nice compilation speedup of ~30%. It's 

The two topics are not really related with one another other than being
related to build systems. I decided to throw them in the same patch
series though so that I can introduce "precompiled.h" in "tools/".

Thanks!

Patrick

[1]: https://lore.kernel.org/git/20250506-pks-contrib-spring-cleanup-v1-0-e6d5ddd79a72@pks.im/

---
Patrick Steinhardt (8):
      Introduce new "tools/" directory
      contrib: move "coccinelle/" directory into "tools/"
      contrib: move "coverage-diff.sh" script into "tools/"
      contrib: move "update-unicode.sh" script into "tools/"
      builds: move build scripts into "tools/"
      git-compat-util.h: move warning infra to prepare for PCHs
      meson: compile compatibility sources separately
      meson: precompile "git-compat-util.h"

 Makefile                                           | 76 ++++++++---------
 ci/run-static-analysis.sh                          |  2 +-
 config.mak.dev                                     |  2 +-
 contrib/buildsystems/CMakeLists.txt                | 18 ++--
 contrib/meson.build                                |  1 -
 contrib/subtree/meson.build                        |  2 +-
 git-compat-util.h                                  |  8 +-
 meson.build                                        | 96 +++++++++++++---------
 tools/README.md                                    |  7 ++
 check-builtins.sh => tools/check-builtins.sh       |  0
 {contrib => tools}/coccinelle/.gitignore           |  0
 {contrib => tools}/coccinelle/README               |  2 +-
 {contrib => tools}/coccinelle/array.cocci          |  0
 {contrib => tools}/coccinelle/commit.cocci         |  0
 .../coccinelle/config_fn_ctx.pending.cocci         |  0
 {contrib => tools}/coccinelle/equals-null.cocci    |  0
 {contrib => tools}/coccinelle/flex_alloc.cocci     |  0
 {contrib => tools}/coccinelle/free.cocci           |  0
 .../coccinelle/git_config_number.cocci             |  0
 {contrib => tools}/coccinelle/hashmap.cocci        |  0
 .../coccinelle/index-compatibility.cocci           |  0
 {contrib => tools}/coccinelle/meson.build          |  0
 {contrib => tools}/coccinelle/object_id.cocci      |  0
 {contrib => tools}/coccinelle/preincr.cocci        |  0
 {contrib => tools}/coccinelle/qsort.cocci          |  0
 {contrib => tools}/coccinelle/refs.cocci           |  0
 {contrib => tools}/coccinelle/spatchcache          |  6 +-
 {contrib => tools}/coccinelle/strbuf.cocci         |  0
 {contrib => tools}/coccinelle/swap.cocci           |  0
 {contrib => tools}/coccinelle/tests/free.c         |  0
 {contrib => tools}/coccinelle/tests/free.res       |  0
 {contrib => tools}/coccinelle/the_repository.cocci |  0
 {contrib => tools}/coccinelle/xcalloc.cocci        |  0
 {contrib => tools}/coccinelle/xopen.cocci          |  0
 .../coccinelle/xstrdup_or_null.cocci               |  0
 {contrib => tools}/coccinelle/xstrncmpz.cocci      |  0
 {contrib => tools}/coverage-diff.sh                |  0
 detect-compiler => tools/detect-compiler           |  0
 generate-cmdlist.sh => tools/generate-cmdlist.sh   |  0
 .../generate-configlist.sh                         |  0
 generate-hooklist.sh => tools/generate-hooklist.sh |  0
 generate-perl.sh => tools/generate-perl.sh         |  0
 generate-python.sh => tools/generate-python.sh     |  0
 generate-script.sh => tools/generate-script.sh     |  0
 tools/meson.build                                  |  1 +
 tools/precompiled.h                                |  1 +
 {contrib => tools}/update-unicode/.gitignore       |  0
 {contrib => tools}/update-unicode/README           |  0
 .../update-unicode/update_unicode.sh               |  0
 49 files changed, 123 insertions(+), 99 deletions(-)


---
base-commit: af2c8a61818d773325ef2324dd135786a03ebca0
change-id: 20260304-b4-pks-build-infra-improvements-cc4012c5364e

[PATCH 1/8] Introduce new "tools/" directory

[Patrick Steinhardt]

According to its readme, the "contrib/" directory's main intent is to
collect stuff that is not an official part of Git, either because it is
too specialized or because it is still considered experimental. The
reality tells a bit of a different story though: while it _does_ contain
such things, it also contains other things:

  - Our credential helpers, which are being distributed by many
    packagers nowadays and which can be considered "stable".

  - A bunch of tooling that relates to our build and test
    infrastructure.

Especially the second category is somewhat of a sore spot. You really
wouldn't expect build-related tooling to be considered an optional part
of Git. Quite the opposite.

Create a new top-level "tools/" directory to fix this discrepancy. This
directory will contain all kind of tools that are related to our build
infrastructure and that Git developers are likely to use day to day.

For now, this directory doesn't contain anything yet except for a
readme and a Meson skeleton. This will change in subsequent commits.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Makefile          | 2 ++
 meson.build       | 1 +
 tools/README.md   | 7 +++++++
 tools/meson.build | 0
 4 files changed, 10 insertions(+)

diff --git a/Makefile b/Makefile
index f3264d0a37..c7cedbcd7c 100644
--- a/Makefile
+++ b/Makefile
@@ -1066,11 +1066,13 @@ SOURCES_CMD = ( \
 		'*.sh' \
 		':!*[tp][0-9][0-9][0-9][0-9]*' \
 		':!contrib' \
+		':!tools' \
 		2>/dev/null || \
 	$(FIND) . \
 		\( -name .git -type d -prune \) \
 		-o \( -name '[tp][0-9][0-9][0-9][0-9]*' -prune \) \
 		-o \( -name contrib -type d -prune \) \
+		-o \( -name tools -type d -prune \) \
 		-o \( -name build -type d -prune \) \
 		-o \( -name .build -type d -prune \) \
 		-o \( -name 'trash*' -type d -prune \) \
diff --git a/meson.build b/meson.build
index 4b536e0124..1d66b5181e 100644
--- a/meson.build
+++ b/meson.build
@@ -2149,6 +2149,7 @@ else
 endif
 
 subdir('contrib')
+subdir('tools')
 
 # Note that the target is intentionally configured after including the
 # 'contrib' directory, as some tool there also have their own manpages.
diff --git a/tools/README.md b/tools/README.md
new file mode 100644
index 0000000000..d732997136
--- /dev/null
+++ b/tools/README.md
@@ -0,0 +1,7 @@
+Developer Tooling
+-----------------
+
+This directory is expected to contain all sorts of tooling that
+relates to our build infrastructure. This includes scripts and
+inputs required by our build systems, but also scripts that
+developers are expected to run manually.
diff --git a/tools/meson.build b/tools/meson.build
new file mode 100644
index 0000000000..e69de29bb2

-- 
2.53.0.880.g73c4285caa.dirty

[PATCH 2/8] contrib: move "coccinelle/" directory into "tools/"

[Patrick Steinhardt]

The Coccinelle tool is an ingrained part of our build infrastructure. It
is executed by our CI to detect antipatterns and is used to detect
misuses of certain interfaces. It's presence in "contrib/" is thus
rather misleading.

Promote the configuration into the new "tools/" directory.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Makefile                                           | 40 +++++++++++-----------
 ci/run-static-analysis.sh                          |  2 +-
 contrib/meson.build                                |  1 -
 {contrib => tools}/coccinelle/.gitignore           |  0
 {contrib => tools}/coccinelle/README               |  2 +-
 {contrib => tools}/coccinelle/array.cocci          |  0
 {contrib => tools}/coccinelle/commit.cocci         |  0
 .../coccinelle/config_fn_ctx.pending.cocci         |  0
 {contrib => tools}/coccinelle/equals-null.cocci    |  0
 {contrib => tools}/coccinelle/flex_alloc.cocci     |  0
 {contrib => tools}/coccinelle/free.cocci           |  0
 .../coccinelle/git_config_number.cocci             |  0
 {contrib => tools}/coccinelle/hashmap.cocci        |  0
 .../coccinelle/index-compatibility.cocci           |  0
 {contrib => tools}/coccinelle/meson.build          |  0
 {contrib => tools}/coccinelle/object_id.cocci      |  0
 {contrib => tools}/coccinelle/preincr.cocci        |  0
 {contrib => tools}/coccinelle/qsort.cocci          |  0
 {contrib => tools}/coccinelle/refs.cocci           |  0
 {contrib => tools}/coccinelle/spatchcache          |  6 ++--
 {contrib => tools}/coccinelle/strbuf.cocci         |  0
 {contrib => tools}/coccinelle/swap.cocci           |  0
 {contrib => tools}/coccinelle/tests/free.c         |  0
 {contrib => tools}/coccinelle/tests/free.res       |  0
 {contrib => tools}/coccinelle/the_repository.cocci |  0
 {contrib => tools}/coccinelle/xcalloc.cocci        |  0
 {contrib => tools}/coccinelle/xopen.cocci          |  0
 .../coccinelle/xstrdup_or_null.cocci               |  0
 {contrib => tools}/coccinelle/xstrncmpz.cocci      |  0
 tools/meson.build                                  |  1 +
 30 files changed, 26 insertions(+), 26 deletions(-)

diff --git a/Makefile b/Makefile
index c7cedbcd7c..8564b1be36 100644
--- a/Makefile
+++ b/Makefile
@@ -1005,8 +1005,8 @@ SPATCH_TEST_FLAGS =
 # COMPUTE_HEADER_DEPENDENCIES=no this will be unset too.
 SPATCH_USE_O_DEPENDENCIES = YesPlease
 
-# Set SPATCH_CONCAT_COCCI to concatenate the contrib/cocci/*.cocci
-# files into a single contrib/cocci/ALL.cocci before running
+# Set SPATCH_CONCAT_COCCI to concatenate the tools/coccinelle/*.cocci
+# files into a single tools/coccinelle/ALL.cocci before running
 # "coccicheck".
 #
 # Pros:
@@ -1025,7 +1025,7 @@ SPATCH_USE_O_DEPENDENCIES = YesPlease
 #   generate a specific patch, e.g. this will always use strbuf.cocci,
 #   not ALL.cocci:
 #
-#	make contrib/coccinelle/strbuf.cocci.patch
+#	make tools/coccinelle/strbuf.cocci.patch
 SPATCH_CONCAT_COCCI = YesPlease
 
 # Rebuild 'coccicheck' if $(SPATCH), its flags etc. change
@@ -3457,15 +3457,15 @@ check:
 		exit 1; \
 	fi
 
-COCCI_GEN_ALL = .build/contrib/coccinelle/ALL.cocci
-COCCI_GLOB = $(wildcard contrib/coccinelle/*.cocci)
+COCCI_GEN_ALL = .build/tools/coccinelle/ALL.cocci
+COCCI_GLOB = $(wildcard tools/coccinelle/*.cocci)
 COCCI_RULES_TRACKED = $(COCCI_GLOB:%=.build/%)
 COCCI_RULES_TRACKED_NO_PENDING = $(filter-out %.pending.cocci,$(COCCI_RULES_TRACKED))
 COCCI_RULES =
 COCCI_RULES += $(COCCI_GEN_ALL)
 COCCI_RULES += $(COCCI_RULES_TRACKED)
 COCCI_NAMES =
-COCCI_NAMES += $(COCCI_RULES:.build/contrib/coccinelle/%.cocci=%)
+COCCI_NAMES += $(COCCI_RULES:.build/tools/coccinelle/%.cocci=%)
 
 COCCICHECK_PENDING = $(filter %.pending.cocci,$(COCCI_RULES))
 COCCICHECK = $(filter-out $(COCCICHECK_PENDING),$(COCCI_RULES))
@@ -3480,20 +3480,20 @@ COCCICHECK_PATCHES_PENDING_INTREE = $(COCCICHECK_PATCHES_PENDING:.build/%=%)
 # on $(MAKECMDGOALS) that match these $(COCCI_RULES)
 COCCI_RULES_GLOB =
 COCCI_RULES_GLOB += cocci%
-COCCI_RULES_GLOB += .build/contrib/coccinelle/%
+COCCI_RULES_GLOB += .build/tools/coccinelle/%
 COCCI_RULES_GLOB += $(COCCICHECK_PATCHES)
 COCCI_RULES_GLOB += $(COCCICHEC_PATCHES_PENDING)
 COCCI_RULES_GLOB += $(COCCICHECK_PATCHES_INTREE)
 COCCI_RULES_GLOB += $(COCCICHECK_PATCHES_PENDING_INTREE)
 COCCI_GOALS = $(filter $(COCCI_RULES_GLOB),$(MAKECMDGOALS))
 
-COCCI_TEST_RES = $(wildcard contrib/coccinelle/tests/*.res)
+COCCI_TEST_RES = $(wildcard tools/coccinelle/tests/*.res)
 
 $(COCCI_RULES_TRACKED): .build/% : %
 	$(call mkdir_p_parent_template)
 	$(QUIET_CP)cp $< $@
 
-.build/contrib/coccinelle/FOUND_H_SOURCES: $(FOUND_H_SOURCES)
+.build/tools/coccinelle/FOUND_H_SOURCES: $(FOUND_H_SOURCES)
 	$(call mkdir_p_parent_template)
 	$(QUIET_GEN) >$@
 
@@ -3507,12 +3507,12 @@ endif
 define cocci-rule
 
 ## Rule for .build/$(1).patch/$(2); Params:
-# $(1) = e.g. ".build/contrib/coccinelle/free.cocci"
+# $(1) = e.g. ".build/tools/coccinelle/free.cocci"
 # $(2) = e.g. "grep.c"
 # $(3) = e.g. "grep.o"
-COCCI_$(1:.build/contrib/coccinelle/%.cocci=%) += $(1).d/$(2).patch
+COCCI_$(1:.build/tools/coccinelle/%.cocci=%) += $(1).d/$(2).patch
 $(1).d/$(2).patch: GIT-SPATCH-DEFINES
-$(1).d/$(2).patch: $(if $(and $(SPATCH_USE_O_DEPENDENCIES),$(wildcard $(3))),$(3),.build/contrib/coccinelle/FOUND_H_SOURCES)
+$(1).d/$(2).patch: $(if $(and $(SPATCH_USE_O_DEPENDENCIES),$(wildcard $(3))),$(3),.build/tools/coccinelle/FOUND_H_SOURCES)
 $(1).d/$(2).patch: $(1)
 $(1).d/$(2).patch: $(1).d/%.patch : %
 	$$(call mkdir_p_parent_template)
@@ -3538,13 +3538,13 @@ endif
 
 define spatch-rule
 
-.build/contrib/coccinelle/$(1).cocci.patch: $$(COCCI_$(1))
+.build/tools/coccinelle/$(1).cocci.patch: $$(COCCI_$(1))
 	$$(QUIET_SPATCH_CAT)cat $$^ >$$@ && \
 	if test -s $$@; \
 	then \
 		echo '    ' SPATCH result: $$@; \
 	fi
-contrib/coccinelle/$(1).cocci.patch: .build/contrib/coccinelle/$(1).cocci.patch
+tools/coccinelle/$(1).cocci.patch: .build/tools/coccinelle/$(1).cocci.patch
 	$$(QUIET_CP)cp $$< $$@
 
 endef
@@ -3558,9 +3558,9 @@ $(COCCI_TEST_RES_GEN): GIT-SPATCH-DEFINES
 $(COCCI_TEST_RES_GEN): .build/%.res : %.c
 $(COCCI_TEST_RES_GEN): .build/%.res : %.res
 ifdef SPATCH_CONCAT_COCCI
-$(COCCI_TEST_RES_GEN): .build/contrib/coccinelle/tests/%.res : $(COCCI_GEN_ALL)
+$(COCCI_TEST_RES_GEN): .build/tools/coccinelle/tests/%.res : $(COCCI_GEN_ALL)
 else
-$(COCCI_TEST_RES_GEN): .build/contrib/coccinelle/tests/%.res : contrib/coccinelle/%.cocci
+$(COCCI_TEST_RES_GEN): .build/tools/coccinelle/tests/%.res : tools/coccinelle/%.cocci
 endif
 	$(call mkdir_p_parent_template)
 	$(QUIET_SPATCH_TEST)$(SPATCH) $(SPATCH_TEST_FLAGS) \
@@ -3576,14 +3576,14 @@ coccicheck-test: $(COCCI_TEST_RES_GEN)
 coccicheck: coccicheck-test
 
 ifdef SPATCH_CONCAT_COCCI
-COCCICHECK_PATCH_MUST_BE_EMPTY_FILES = contrib/coccinelle/ALL.cocci.patch
+COCCICHECK_PATCH_MUST_BE_EMPTY_FILES = tools/coccinelle/ALL.cocci.patch
 else
 COCCICHECK_PATCH_MUST_BE_EMPTY_FILES = $(COCCICHECK_PATCHES_INTREE)
 endif
 coccicheck: $(COCCICHECK_PATCH_MUST_BE_EMPTY_FILES)
 	! grep ^ $(COCCICHECK_PATCH_MUST_BE_EMPTY_FILES) /dev/null
 
-# See contrib/coccinelle/README
+# See tools/coccinelle/README
 coccicheck-pending: coccicheck-test
 coccicheck-pending: $(COCCICHECK_PATCHES_PENDING_INTREE)
 
@@ -3857,8 +3857,8 @@ profile-clean:
 
 cocciclean:
 	$(RM) GIT-SPATCH-DEFINES
-	$(RM) -r .build/contrib/coccinelle
-	$(RM) contrib/coccinelle/*.cocci.patch
+	$(RM) -r .build/tools/coccinelle
+	$(RM) tools/coccinelle/*.cocci.patch
 
 clean: profile-clean coverage-clean cocciclean
 	$(RM) -r .build $(UNIT_TEST_BIN)
diff --git a/ci/run-static-analysis.sh b/ci/run-static-analysis.sh
index 9e9c72681d..ba67e80b4d 100755
--- a/ci/run-static-analysis.sh
+++ b/ci/run-static-analysis.sh
@@ -10,7 +10,7 @@ make coccicheck
 set +x
 
 fail=
-for cocci_patch in contrib/coccinelle/*.patch
+for cocci_patch in tools/coccinelle/*.patch
 do
 	if test -s "$cocci_patch"
 	then
diff --git a/contrib/meson.build b/contrib/meson.build
index a88c5dfe09..569c23ee76 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -2,5 +2,4 @@ foreach feature : get_option('contrib')
   subdir(feature)
 endforeach
 
-subdir('coccinelle')
 subdir('credential')
diff --git a/contrib/coccinelle/.gitignore b/tools/coccinelle/.gitignore
similarity index 100%
rename from contrib/coccinelle/.gitignore
rename to tools/coccinelle/.gitignore
diff --git a/contrib/coccinelle/README b/tools/coccinelle/README
similarity index 98%
rename from contrib/coccinelle/README
rename to tools/coccinelle/README
index 055ad0e06a..fd0a543cc2 100644
--- a/contrib/coccinelle/README
+++ b/tools/coccinelle/README
@@ -38,7 +38,7 @@ that might be useful to developers.
    So to aid these large scale refactorings, semantic patches can be used.
    However we do not want to store them in the same place as the checks for
    bad patterns, as then automated builds would fail.
-   That is why semantic patches 'contrib/coccinelle/*.pending.cocci'
+   That is why semantic patches 'tools/coccinelle/*.pending.cocci'
    are ignored for checks, and can be applied using 'make coccicheck-pending'.
 
    This allows to expose plans of pending large scale refactorings without
diff --git a/contrib/coccinelle/array.cocci b/tools/coccinelle/array.cocci
similarity index 100%
rename from contrib/coccinelle/array.cocci
rename to tools/coccinelle/array.cocci
diff --git a/contrib/coccinelle/commit.cocci b/tools/coccinelle/commit.cocci
similarity index 100%
rename from contrib/coccinelle/commit.cocci
rename to tools/coccinelle/commit.cocci
diff --git a/contrib/coccinelle/config_fn_ctx.pending.cocci b/tools/coccinelle/config_fn_ctx.pending.cocci
similarity index 100%
rename from contrib/coccinelle/config_fn_ctx.pending.cocci
rename to tools/coccinelle/config_fn_ctx.pending.cocci
diff --git a/contrib/coccinelle/equals-null.cocci b/tools/coccinelle/equals-null.cocci
similarity index 100%
rename from contrib/coccinelle/equals-null.cocci
rename to tools/coccinelle/equals-null.cocci
diff --git a/contrib/coccinelle/flex_alloc.cocci b/tools/coccinelle/flex_alloc.cocci
similarity index 100%
rename from contrib/coccinelle/flex_alloc.cocci
rename to tools/coccinelle/flex_alloc.cocci
diff --git a/contrib/coccinelle/free.cocci b/tools/coccinelle/free.cocci
similarity index 100%
rename from contrib/coccinelle/free.cocci
rename to tools/coccinelle/free.cocci
diff --git a/contrib/coccinelle/git_config_number.cocci b/tools/coccinelle/git_config_number.cocci
similarity index 100%
rename from contrib/coccinelle/git_config_number.cocci
rename to tools/coccinelle/git_config_number.cocci
diff --git a/contrib/coccinelle/hashmap.cocci b/tools/coccinelle/hashmap.cocci
similarity index 100%
rename from contrib/coccinelle/hashmap.cocci
rename to tools/coccinelle/hashmap.cocci
diff --git a/contrib/coccinelle/index-compatibility.cocci b/tools/coccinelle/index-compatibility.cocci
similarity index 100%
rename from contrib/coccinelle/index-compatibility.cocci
rename to tools/coccinelle/index-compatibility.cocci
diff --git a/contrib/coccinelle/meson.build b/tools/coccinelle/meson.build
similarity index 100%
rename from contrib/coccinelle/meson.build
rename to tools/coccinelle/meson.build
diff --git a/contrib/coccinelle/object_id.cocci b/tools/coccinelle/object_id.cocci
similarity index 100%
rename from contrib/coccinelle/object_id.cocci
rename to tools/coccinelle/object_id.cocci
diff --git a/contrib/coccinelle/preincr.cocci b/tools/coccinelle/preincr.cocci
similarity index 100%
rename from contrib/coccinelle/preincr.cocci
rename to tools/coccinelle/preincr.cocci
diff --git a/contrib/coccinelle/qsort.cocci b/tools/coccinelle/qsort.cocci
similarity index 100%
rename from contrib/coccinelle/qsort.cocci
rename to tools/coccinelle/qsort.cocci
diff --git a/contrib/coccinelle/refs.cocci b/tools/coccinelle/refs.cocci
similarity index 100%
rename from contrib/coccinelle/refs.cocci
rename to tools/coccinelle/refs.cocci
diff --git a/contrib/coccinelle/spatchcache b/tools/coccinelle/spatchcache
similarity index 97%
rename from contrib/coccinelle/spatchcache
rename to tools/coccinelle/spatchcache
index 29e9352d8a..efbcbc3827 100755
--- a/contrib/coccinelle/spatchcache
+++ b/tools/coccinelle/spatchcache
@@ -30,7 +30,7 @@
 #	   out of control.
 #
 # This along with the general incremental "make" support for
-# "contrib/coccinelle" makes it viable to (re-)run coccicheck
+# "tools/coccinelle" makes it viable to (re-)run coccicheck
 # e.g. when merging integration branches.
 #
 # Note that the "--very-quiet" flag is currently critical. The cache
@@ -42,7 +42,7 @@
 # to change, so just supply "--very-quiet" for now.
 #
 # To use this, simply set SPATCH to
-# contrib/coccinelle/spatchcache. Then optionally set:
+# tools/coccinelle/spatchcache. Then optionally set:
 #
 #	[spatchCache]
 #		# Optional: path to a custom spatch
@@ -65,7 +65,7 @@
 #
 #	redis-cli FLUSHALL
 #	<make && make coccicheck, as above>
-#	grep -hore HIT -e MISS -e SET -e NOCACHE -e CANTCACHE .build/contrib/coccinelle | sort | uniq -c
+#	grep -hore HIT -e MISS -e SET -e NOCACHE -e CANTCACHE .build/tools/coccinelle | sort | uniq -c
 #	    600 CANTCACHE
 #	   7365 MISS
 #	   7365 SET
diff --git a/contrib/coccinelle/strbuf.cocci b/tools/coccinelle/strbuf.cocci
similarity index 100%
rename from contrib/coccinelle/strbuf.cocci
rename to tools/coccinelle/strbuf.cocci
diff --git a/contrib/coccinelle/swap.cocci b/tools/coccinelle/swap.cocci
similarity index 100%
rename from contrib/coccinelle/swap.cocci
rename to tools/coccinelle/swap.cocci
diff --git a/contrib/coccinelle/tests/free.c b/tools/coccinelle/tests/free.c
similarity index 100%
rename from contrib/coccinelle/tests/free.c
rename to tools/coccinelle/tests/free.c
diff --git a/contrib/coccinelle/tests/free.res b/tools/coccinelle/tests/free.res
similarity index 100%
rename from contrib/coccinelle/tests/free.res
rename to tools/coccinelle/tests/free.res
diff --git a/contrib/coccinelle/the_repository.cocci b/tools/coccinelle/the_repository.cocci
similarity index 100%
rename from contrib/coccinelle/the_repository.cocci
rename to tools/coccinelle/the_repository.cocci
diff --git a/contrib/coccinelle/xcalloc.cocci b/tools/coccinelle/xcalloc.cocci
similarity index 100%
rename from contrib/coccinelle/xcalloc.cocci
rename to tools/coccinelle/xcalloc.cocci
diff --git a/contrib/coccinelle/xopen.cocci b/tools/coccinelle/xopen.cocci
similarity index 100%
rename from contrib/coccinelle/xopen.cocci
rename to tools/coccinelle/xopen.cocci
diff --git a/contrib/coccinelle/xstrdup_or_null.cocci b/tools/coccinelle/xstrdup_or_null.cocci
similarity index 100%
rename from contrib/coccinelle/xstrdup_or_null.cocci
rename to tools/coccinelle/xstrdup_or_null.cocci
diff --git a/contrib/coccinelle/xstrncmpz.cocci b/tools/coccinelle/xstrncmpz.cocci
similarity index 100%
rename from contrib/coccinelle/xstrncmpz.cocci
rename to tools/coccinelle/xstrncmpz.cocci
diff --git a/tools/meson.build b/tools/meson.build
index e69de29bb2..f731f74312 100644
--- a/tools/meson.build
+++ b/tools/meson.build
@@ -0,0 +1 @@
+subdir('coccinelle')

-- 
2.53.0.880.g73c4285caa.dirty

[PATCH 3/8] contrib: move "coverage-diff.sh" script into "tools/"

[Patrick Steinhardt]

The "coverage-diff.sh" script can be used to get information about test
coverage fro the Git codebase. It is thus rather specific to our build
and test infrastructure and part of the developer-facing tooling. The
fact that this script is part of "contrib/" is thus rather misleading
and a historic wart.

Promote the tool into the new "tools/" directory.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 {contrib => tools}/coverage-diff.sh | 0
 1 file changed, 0 insertions(+), 0 deletions(-)

diff --git a/contrib/coverage-diff.sh b/tools/coverage-diff.sh
similarity index 100%
rename from contrib/coverage-diff.sh
rename to tools/coverage-diff.sh

-- 
2.53.0.880.g73c4285caa.dirty

[PATCH 4/8] contrib: move "update-unicode.sh" script into "tools/"

[Patrick Steinhardt]

The "update-unicode.sh" script is used to update the unicode data
compiled into Git whenever a new version of the Unicode standard has
been released. As such, it is a natural part of our developer-facing
tooling, and its presence in "contrib/" is misleading.

Promote the script into the new "tools/" directory.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 {contrib => tools}/update-unicode/.gitignore        | 0
 {contrib => tools}/update-unicode/README            | 0
 {contrib => tools}/update-unicode/update_unicode.sh | 0
 3 files changed, 0 insertions(+), 0 deletions(-)

diff --git a/contrib/update-unicode/.gitignore b/tools/update-unicode/.gitignore
similarity index 100%
rename from contrib/update-unicode/.gitignore
rename to tools/update-unicode/.gitignore
diff --git a/contrib/update-unicode/README b/tools/update-unicode/README
similarity index 100%
rename from contrib/update-unicode/README
rename to tools/update-unicode/README
diff --git a/contrib/update-unicode/update_unicode.sh b/tools/update-unicode/update_unicode.sh
similarity index 100%
rename from contrib/update-unicode/update_unicode.sh
rename to tools/update-unicode/update_unicode.sh

-- 
2.53.0.880.g73c4285caa.dirty

[PATCH 5/8] builds: move build scripts into "tools/"

[Patrick Steinhardt]

We have a bunch of scripts used by our different build systems that are
all located in the top-level directory. Now that we have introduced the
new "tools/" directory though we have a better home for them.

Move the scripts into the "tools/" directory.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Makefile                                           | 34 +++++++++++-----------
 config.mak.dev                                     |  2 +-
 contrib/buildsystems/CMakeLists.txt                | 18 ++++++------
 contrib/subtree/meson.build                        |  2 +-
 meson.build                                        | 14 ++++-----
 check-builtins.sh => tools/check-builtins.sh       |  0
 detect-compiler => tools/detect-compiler           |  0
 generate-cmdlist.sh => tools/generate-cmdlist.sh   |  0
 .../generate-configlist.sh                         |  0
 generate-hooklist.sh => tools/generate-hooklist.sh |  0
 generate-perl.sh => tools/generate-perl.sh         |  0
 generate-python.sh => tools/generate-python.sh     |  0
 generate-script.sh => tools/generate-script.sh     |  0
 13 files changed, 35 insertions(+), 35 deletions(-)

diff --git a/Makefile b/Makefile
index 8564b1be36..322f5940e3 100644
--- a/Makefile
+++ b/Makefile
@@ -2689,21 +2689,21 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
-config-list.h: generate-configlist.sh
+config-list.h: tools/generate-configlist.sh
 	@mkdir -p .depend
-	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh . $@ .depend/config-list.h.d
+	$(QUIET_GEN)$(SHELL_PATH) ./tools/generate-configlist.sh . $@ .depend/config-list.h.d
 
 -include .depend/config-list.h.d
 
-command-list.h: generate-cmdlist.sh command-list.txt
+command-list.h: tools/generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.adoc)
-	$(QUIET_GEN)$(SHELL_PATH) ./generate-cmdlist.sh \
+	$(QUIET_GEN)$(SHELL_PATH) ./tools/generate-cmdlist.sh \
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		. $@
 
-hook-list.h: generate-hooklist.sh Documentation/githooks.adoc
-	$(QUIET_GEN)$(SHELL_PATH) ./generate-hooklist.sh . $@
+hook-list.h: tools/generate-hooklist.sh Documentation/githooks.adoc
+	$(QUIET_GEN)$(SHELL_PATH) ./tools/generate-hooklist.sh . $@
 
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):\
 	$(localedir_SQ):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
@@ -2716,8 +2716,8 @@ GIT-SCRIPT-DEFINES: FORCE
 		echo "$$FLAGS" >$@; \
             fi
 
-$(SCRIPT_SH_GEN) $(SCRIPT_LIB) : % : %.sh generate-script.sh GIT-BUILD-OPTIONS GIT-SCRIPT-DEFINES
-	$(QUIET_GEN)./generate-script.sh "$<" "$@+" ./GIT-BUILD-OPTIONS && \
+$(SCRIPT_SH_GEN) $(SCRIPT_LIB) : % : %.sh tools/generate-script.sh GIT-BUILD-OPTIONS GIT-SCRIPT-DEFINES
+	$(QUIET_GEN)./tools/generate-script.sh "$<" "$@+" ./GIT-BUILD-OPTIONS && \
 	mv $@+ $@
 
 git.rc: git.rc.in GIT-VERSION-GEN GIT-VERSION-FILE
@@ -2757,8 +2757,8 @@ endif
 
 PERL_DEFINES += $(gitexecdir) $(perllibdir) $(localedir)
 
-$(SCRIPT_PERL_GEN): % : %.perl generate-perl.sh GIT-PERL-DEFINES GIT-PERL-HEADER GIT-VERSION-FILE
-	$(QUIET_GEN)$(SHELL_PATH) generate-perl.sh ./GIT-BUILD-OPTIONS ./GIT-VERSION-FILE GIT-PERL-HEADER "$<" "$@+" && \
+$(SCRIPT_PERL_GEN): % : %.perl tools/generate-perl.sh GIT-PERL-DEFINES GIT-PERL-HEADER GIT-VERSION-FILE
+	$(QUIET_GEN)$(SHELL_PATH) tools/generate-perl.sh ./GIT-BUILD-OPTIONS ./GIT-VERSION-FILE GIT-PERL-HEADER "$<" "$@+" && \
 	mv $@+ $@
 
 PERL_DEFINES := $(subst $(space),:,$(PERL_DEFINES))
@@ -2786,8 +2786,8 @@ GIT-PERL-HEADER: $(PERL_HEADER_TEMPLATE) GIT-PERL-DEFINES Makefile
 perllibdir:
 	@echo '$(perllibdir_SQ)'
 
-git-instaweb: git-instaweb.sh generate-script.sh GIT-BUILD-OPTIONS GIT-SCRIPT-DEFINES
-	$(QUIET_GEN)./generate-script.sh "$<" "$@+" ./GIT-BUILD-OPTIONS && \
+git-instaweb: git-instaweb.sh tools/generate-script.sh GIT-BUILD-OPTIONS GIT-SCRIPT-DEFINES
+	$(QUIET_GEN)./tools/generate-script.sh "$<" "$@+" ./GIT-BUILD-OPTIONS && \
 	chmod +x $@+ && \
 	mv $@+ $@
 else # NO_PERL
@@ -2804,9 +2804,9 @@ endif # NO_PERL
 $(SCRIPT_PYTHON_GEN): GIT-BUILD-OPTIONS
 
 ifndef NO_PYTHON
-$(SCRIPT_PYTHON_GEN): generate-python.sh
+$(SCRIPT_PYTHON_GEN): tools/generate-python.sh
 $(SCRIPT_PYTHON_GEN): % : %.py
-	$(QUIET_GEN)$(SHELL_PATH) generate-python.sh ./GIT-BUILD-OPTIONS "$<" "$@"
+	$(QUIET_GEN)$(SHELL_PATH) tools/generate-python.sh ./GIT-BUILD-OPTIONS "$<" "$@"
 else # NO_PYTHON
 $(SCRIPT_PYTHON_GEN): % : unimplemented.sh
 	$(QUIET_GEN) \
@@ -3226,9 +3226,9 @@ endif
 NO_PERL_CPAN_FALLBACKS_SQ = $(subst ','\'',$(NO_PERL_CPAN_FALLBACKS))
 endif
 
-perl/build/lib/%.pm: perl/%.pm generate-perl.sh GIT-BUILD-OPTIONS GIT-VERSION-FILE GIT-PERL-DEFINES
+perl/build/lib/%.pm: perl/%.pm tools/generate-perl.sh GIT-BUILD-OPTIONS GIT-VERSION-FILE GIT-PERL-DEFINES
 	$(call mkdir_p_parent_template)
-	$(QUIET_GEN)$(SHELL_PATH) generate-perl.sh ./GIT-BUILD-OPTIONS ./GIT-VERSION-FILE GIT-PERL-HEADER "$<" "$@"
+	$(QUIET_GEN)$(SHELL_PATH) tools/generate-perl.sh ./GIT-BUILD-OPTIONS ./GIT-VERSION-FILE GIT-PERL-HEADER "$<" "$@"
 
 perl/build/man/man3/Git.3pm: perl/Git.pm
 	$(call mkdir_p_parent_template)
@@ -3936,7 +3936,7 @@ check-docs::
 ### Make sure built-ins do not have dups and listed in git.c
 #
 check-builtins::
-	./check-builtins.sh
+	./tools/check-builtins.sh
 
 ### Test suite coverage testing
 #
diff --git a/config.mak.dev b/config.mak.dev
index e86b6e1b34..c8dcf78779 100644
--- a/config.mak.dev
+++ b/config.mak.dev
@@ -1,5 +1,5 @@
 ifndef COMPILER_FEATURES
-COMPILER_FEATURES := $(shell ./detect-compiler $(CC))
+COMPILER_FEATURES := $(shell ./tools/detect-compiler $(CC))
 endif
 
 ifeq ($(filter no-error,$(DEVOPTS)),)
diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt
index c6cfb874ef..81b4306e72 100644
--- a/contrib/buildsystems/CMakeLists.txt
+++ b/contrib/buildsystems/CMakeLists.txt
@@ -636,7 +636,7 @@ set(EXCLUSION_PROGS_CACHE ${EXCLUSION_PROGS} CACHE STRING "Programs not built" F
 if(NOT EXISTS ${CMAKE_BINARY_DIR}/command-list.h OR NOT EXCLUSION_PROGS_CACHE STREQUAL EXCLUSION_PROGS)
 	list(REMOVE_ITEM EXCLUSION_PROGS empty)
 	message("Generating command-list.h")
-	execute_process(COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/generate-cmdlist.sh"
+	execute_process(COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/tools/generate-cmdlist.sh"
 				${EXCLUSION_PROGS}
 				"${CMAKE_SOURCE_DIR}"
 				"${CMAKE_BINARY_DIR}/command-list.h")
@@ -644,14 +644,14 @@ endif()
 
 if(NOT EXISTS ${CMAKE_BINARY_DIR}/config-list.h)
 	message("Generating config-list.h")
-	execute_process(COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/generate-configlist.sh"
+	execute_process(COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/tools/generate-configlist.sh"
 				"${CMAKE_SOURCE_DIR}"
 				"${CMAKE_BINARY_DIR}/config-list.h")
 endif()
 
 if(NOT EXISTS ${CMAKE_BINARY_DIR}/hook-list.h)
 	message("Generating hook-list.h")
-	execute_process(COMMAND "${SH_EXE}" ${CMAKE_SOURCE_DIR}/generate-hooklist.sh
+	execute_process(COMMAND "${SH_EXE}" ${CMAKE_SOURCE_DIR}/tools/generate-hooklist.sh
 				"${CMAKE_SOURCE_DIR}"
 				"${CMAKE_BINARY_DIR}/hook-list.h")
 endif()
@@ -832,11 +832,11 @@ foreach(script ${git_shell_scripts})
 	endif()
 
 	add_custom_command(OUTPUT "${CMAKE_BINARY_DIR}/${shell_gen_path}"
-		COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/generate-script.sh"
+		COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/tools/generate-script.sh"
 			"${CMAKE_SOURCE_DIR}/${script}.sh"
 			"${CMAKE_BINARY_DIR}/${shell_gen_path}"
 			"${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS"
-		DEPENDS "${CMAKE_SOURCE_DIR}/generate-script.sh"
+		DEPENDS "${CMAKE_SOURCE_DIR}/tools/generate-script.sh"
 			"${CMAKE_SOURCE_DIR}/${script}.sh"
 		VERBATIM)
 	list(APPEND shell_gen ${CMAKE_BINARY_DIR}/${shell_gen_path})
@@ -875,13 +875,13 @@ foreach(script ${git_perl_scripts} ${perl_modules})
 	file(MAKE_DIRECTORY "${CMAKE_BINARY_DIR}/${perl_gen_dir}")
 
 	add_custom_command(OUTPUT "${CMAKE_BINARY_DIR}/${perl_gen_path}"
-		COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/generate-perl.sh"
+		COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/tools/generate-perl.sh"
 			"${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS"
 			"${CMAKE_BINARY_DIR}/GIT-VERSION-FILE"
 			"${CMAKE_BINARY_DIR}/GIT-PERL-HEADER"
 			"${CMAKE_SOURCE_DIR}/${script}"
 			"${CMAKE_BINARY_DIR}/${perl_gen_path}"
-		DEPENDS "${CMAKE_SOURCE_DIR}/generate-perl.sh"
+		DEPENDS "${CMAKE_SOURCE_DIR}/tools/generate-perl.sh"
 			"${CMAKE_SOURCE_DIR}/${script}"
 			"${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS"
 			"${CMAKE_BINARY_DIR}/GIT-VERSION-FILE"
@@ -892,11 +892,11 @@ add_custom_target(perl-gen ALL DEPENDS ${perl_gen})
 
 # Python script
 add_custom_command(OUTPUT "${CMAKE_BINARY_DIR}/git-p4"
-	COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/generate-python.sh"
+	COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/tools/generate-python.sh"
 		"${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS"
 		"${CMAKE_SOURCE_DIR}/git-p4.py"
 		"${CMAKE_BINARY_DIR}/git-p4"
-	DEPENDS "${CMAKE_SOURCE_DIR}/generate-python.sh"
+	DEPENDS "${CMAKE_SOURCE_DIR}/tools/generate-python.sh"
 		"${CMAKE_SOURCE_DIR}/git-p4.py"
 		"${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS"
 	VERBATIM)
diff --git a/contrib/subtree/meson.build b/contrib/subtree/meson.build
index 161435abeb..804c315894 100644
--- a/contrib/subtree/meson.build
+++ b/contrib/subtree/meson.build
@@ -3,7 +3,7 @@ git_subtree = custom_target(
   output: 'git-subtree',
   command: [
     shell,
-    meson.project_source_root() / 'generate-script.sh',
+    meson.project_source_root() / 'tools/generate-script.sh',
     '@INPUT@',
     '@OUTPUT@',
     meson.project_build_root() / 'GIT-BUILD-OPTIONS',
diff --git a/meson.build b/meson.build
index 1d66b5181e..604fe89d2d 100644
--- a/meson.build
+++ b/meson.build
@@ -554,7 +554,7 @@ libgit_sources = [
 libgit_sources += custom_target(
   input: 'command-list.txt',
   output: 'command-list.h',
-  command: [shell, meson.current_source_dir() + '/generate-cmdlist.sh', meson.current_source_dir(), '@OUTPUT@'],
+  command: [shell, meson.current_source_dir() + '/tools/generate-cmdlist.sh', meson.current_source_dir(), '@OUTPUT@'],
   env: script_environment,
 )
 
@@ -723,10 +723,10 @@ endif
 builtin_sources += custom_target(
   output: 'config-list.h',
   depfile: 'config-list.h.d',
-  depend_files: [ 'generate-configlist.sh' ],
+  depend_files: [ 'tools/generate-configlist.sh' ],
   command: [
     shell,
-    meson.current_source_dir() / 'generate-configlist.sh',
+    meson.current_source_dir() / 'tools/generate-configlist.sh',
     meson.current_source_dir(),
     '@OUTPUT@',
     '@DEPFILE@',
@@ -739,7 +739,7 @@ builtin_sources += custom_target(
   output: 'hook-list.h',
   command: [
     shell,
-    meson.current_source_dir() + '/generate-hooklist.sh',
+    meson.current_source_dir() + '/tools/generate-hooklist.sh',
     meson.current_source_dir(),
     '@OUTPUT@',
   ],
@@ -1959,7 +1959,7 @@ foreach script : scripts_sh
     output: fs.stem(script),
     command: [
       shell,
-      meson.project_source_root() / 'generate-script.sh',
+      meson.project_source_root() / 'tools/generate-script.sh',
       '@INPUT@',
       '@OUTPUT@',
       meson.project_build_root() / 'GIT-BUILD-OPTIONS',
@@ -2008,7 +2008,7 @@ if perl_features_enabled
 
   generate_perl_command = [
     shell,
-    meson.project_source_root() / 'generate-perl.sh',
+    meson.project_source_root() / 'tools/generate-perl.sh',
     meson.project_build_root() / 'GIT-BUILD-OPTIONS',
     git_version_file.full_path(),
     perl_header,
@@ -2057,7 +2057,7 @@ if target_python.found()
       output: fs.stem(script),
       command: [
         shell,
-        meson.project_source_root() / 'generate-python.sh',
+        meson.project_source_root() / 'tools/generate-python.sh',
         meson.project_build_root() / 'GIT-BUILD-OPTIONS',
         '@INPUT@',
         '@OUTPUT@',
diff --git a/check-builtins.sh b/tools/check-builtins.sh
similarity index 100%
rename from check-builtins.sh
rename to tools/check-builtins.sh
diff --git a/detect-compiler b/tools/detect-compiler
similarity index 100%
rename from detect-compiler
rename to tools/detect-compiler
diff --git a/generate-cmdlist.sh b/tools/generate-cmdlist.sh
similarity index 100%
rename from generate-cmdlist.sh
rename to tools/generate-cmdlist.sh
diff --git a/generate-configlist.sh b/tools/generate-configlist.sh
similarity index 100%
rename from generate-configlist.sh
rename to tools/generate-configlist.sh
diff --git a/generate-hooklist.sh b/tools/generate-hooklist.sh
similarity index 100%
rename from generate-hooklist.sh
rename to tools/generate-hooklist.sh
diff --git a/generate-perl.sh b/tools/generate-perl.sh
similarity index 100%
rename from generate-perl.sh
rename to tools/generate-perl.sh
diff --git a/generate-python.sh b/tools/generate-python.sh
similarity index 100%
rename from generate-python.sh
rename to tools/generate-python.sh
diff --git a/generate-script.sh b/tools/generate-script.sh
similarity index 100%
rename from generate-script.sh
rename to tools/generate-script.sh

-- 
2.53.0.880.g73c4285caa.dirty

[PATCH 6/8] git-compat-util.h: move warning infra to prepare for PCHs

[Patrick Steinhardt]

The "git-compat-util.h" header is supposed to be the first header
included by every code compilation unit. As such, a subsequent commit
will start to precompile this header to speed up compilation of Git.

This will cause an issue though with the way that we have set up the
"-Wsign-compare" warnings. It is expected that any compilation unit that
fails with that compiler warning sets `DISABLE_SIGN_COMPARE_WARNINGS`
before including "git-compat-util.h". If so, we'll disable the warning
right away via a compiler pragma.

But with precompiled headers we do not know ahead of time whether the
code unit wants to disable those warnings, and thus we'll have to
precompile the header without defining `DISABLE_SIGN_COMPARE_WARNINGS`.
But as the pragma statement is wrapped by our include guards, the second
include of that file will not have the desired effect of disabling the
warnings anymore.

We could fix this issue by declaring a new macro that compilation units
are expected to invoke after having included the file. In retrospect,
that would have been the better way to handle this as it allows for
more flexibility: we could for example toggle the warning for specific
code blocks, only. But changing this now would require a bunch of
changes, and the churn feels excessive for what we gain.

Instead, prepare for the precompiled headers by moving the code outside
of the include guards.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 git-compat-util.h | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/git-compat-util.h b/git-compat-util.h
index bebcf9f698..4b4ea2498f 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -34,10 +34,6 @@ struct strbuf;
 #  define DISABLE_WARNING(warning)
 #endif
 
-#ifdef DISABLE_SIGN_COMPARE_WARNINGS
-DISABLE_WARNING(-Wsign-compare)
-#endif
-
 #undef FLEX_ARRAY
 #define FLEX_ARRAY /* empty - weather balloon to require C99 FAM */
 
@@ -1099,3 +1095,7 @@ extern int not_supposed_to_survive;
 #endif /* CHECK_ASSERTION_SIDE_EFFECTS */
 
 #endif
+
+#ifdef DISABLE_SIGN_COMPARE_WARNINGS
+DISABLE_WARNING(-Wsign-compare)
+#endif

-- 
2.53.0.880.g73c4285caa.dirty

[PATCH 7/8] meson: compile compatibility sources separately

[Patrick Steinhardt]

In the next commit we're about to introduce a precompiled header for
"git-compat-util.h". The consequence of this change is that we'll
implicitly include that header for every compilation unit that uses the
precompiled headers.

This is okay for our "normal" library sources and our builtins. But some
of our compatibility sources do not include the header on purpose, and
doing so would cause compileir errors.

Prepare for this change by splitting out compatibility sources into
their static library. Like this we can selectively enable precompiled
headers for the library sources.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 meson.build | 79 +++++++++++++++++++++++++++++++++++--------------------------
 1 file changed, 45 insertions(+), 34 deletions(-)

diff --git a/meson.build b/meson.build
index 604fe89d2d..cd00be1c23 100644
--- a/meson.build
+++ b/meson.build
@@ -271,6 +271,13 @@ version_gen_environment.set('GIT_VERSION', get_option('version'))
 
 compiler = meson.get_compiler('c')
 
+compat_sources = [
+  'compat/nonblock.c',
+  'compat/obstack.c',
+  'compat/open.c',
+  'compat/terminal.c',
+]
+
 libgit_sources = [
   'abspath.c',
   'add-interactive.c',
@@ -304,10 +311,6 @@ libgit_sources = [
   'commit.c',
   'common-exit.c',
   'common-init.c',
-  'compat/nonblock.c',
-  'compat/obstack.c',
-  'compat/open.c',
-  'compat/terminal.c',
   'compiler-tricks/not-constant.c',
   'config.c',
   'connect.c',
@@ -1163,7 +1166,7 @@ endif
 
 if not has_poll_h and not has_sys_poll_h
   libgit_c_args += '-DNO_POLL'
-  libgit_sources += 'compat/poll/poll.c'
+  compat_sources += 'compat/poll/poll.c'
   libgit_include_directories += 'compat/poll'
 endif
 
@@ -1179,7 +1182,7 @@ endif
 # implementation to threat things like drive prefixes specially.
 if host_machine.system() == 'windows' or not compiler.has_header('libgen.h')
   libgit_c_args += '-DNO_LIBGEN_H'
-  libgit_sources += 'compat/basename.c'
+  compat_sources += 'compat/basename.c'
 endif
 
 if compiler.has_header('paths.h')
@@ -1209,7 +1212,7 @@ if host_machine.system() != 'windows'
   foreach symbol : ['inet_ntop', 'inet_pton', 'hstrerror']
     if not compiler.has_function(symbol, dependencies: networking_dependencies)
       libgit_c_args += '-DNO_' + symbol.to_upper()
-      libgit_sources += 'compat/' + symbol + '.c'
+      compat_sources += 'compat/' + symbol + '.c'
     endif
   endforeach
 endif
@@ -1251,18 +1254,18 @@ else
 endif
 
 if host_machine.system() == 'darwin'
-  libgit_sources += 'compat/precompose_utf8.c'
+  compat_sources += 'compat/precompose_utf8.c'
   libgit_c_args += '-DPRECOMPOSE_UNICODE'
   libgit_c_args += '-DPROTECT_HFS_DEFAULT'
 endif
 
 # Configure general compatibility wrappers.
 if host_machine.system() == 'cygwin'
-  libgit_sources += [
+  compat_sources += [
     'compat/win32/path-utils.c',
   ]
 elif host_machine.system() == 'windows'
-  libgit_sources += [
+  compat_sources += [
     'compat/winansi.c',
     'compat/win32/dirent.c',
     'compat/win32/flush.c',
@@ -1289,20 +1292,20 @@ elif host_machine.system() == 'windows'
   libgit_include_directories += 'compat/win32'
   if compiler.get_id() == 'msvc'
     libgit_include_directories += 'compat/vcbuild/include'
-    libgit_sources += 'compat/msvc.c'
+    compat_sources += 'compat/msvc.c'
   else
-    libgit_sources += 'compat/mingw.c'
+    compat_sources += 'compat/mingw.c'
   endif
 endif
 
 if host_machine.system() == 'linux'
-  libgit_sources += 'compat/linux/procinfo.c'
+  compat_sources += 'compat/linux/procinfo.c'
 elif host_machine.system() == 'windows'
-  libgit_sources += 'compat/win32/trace2_win32_process_info.c'
+  compat_sources += 'compat/win32/trace2_win32_process_info.c'
 elif host_machine.system() == 'darwin'
-  libgit_sources += 'compat/darwin/procinfo.c'
+  compat_sources += 'compat/darwin/procinfo.c'
 else
-  libgit_sources += 'compat/stub/procinfo.c'
+  compat_sources += 'compat/stub/procinfo.c'
 endif
 
 if host_machine.system() == 'cygwin' or host_machine.system() == 'windows'
@@ -1315,13 +1318,13 @@ endif
 
 # Configure the simple-ipc subsystem required fro the fsmonitor.
 if host_machine.system() == 'windows'
-  libgit_sources += [
+  compat_sources += [
     'compat/simple-ipc/ipc-shared.c',
     'compat/simple-ipc/ipc-win32.c',
   ]
   libgit_c_args += '-DSUPPORTS_SIMPLE_IPC'
 else
-  libgit_sources += [
+  compat_sources += [
     'compat/simple-ipc/ipc-shared.c',
     'compat/simple-ipc/ipc-unix-socket.c',
   ]
@@ -1339,7 +1342,7 @@ if fsmonitor_backend != ''
   libgit_c_args += '-DHAVE_FSMONITOR_DAEMON_BACKEND'
   libgit_c_args += '-DHAVE_FSMONITOR_OS_SETTINGS'
 
-  libgit_sources += [
+  compat_sources += [
     'compat/fsmonitor/fsm-health-' + fsmonitor_backend + '.c',
     'compat/fsmonitor/fsm-ipc-' + fsmonitor_backend + '.c',
     'compat/fsmonitor/fsm-listen-' + fsmonitor_backend + '.c',
@@ -1355,7 +1358,7 @@ if not get_option('b_sanitize').contains('address') and get_option('regex').allo
 
   if compiler.get_define('REG_ENHANCED', prefix: '#include <regex.h>') != ''
     libgit_c_args += '-DUSE_ENHANCED_BASIC_REGULAR_EXPRESSIONS'
-    libgit_sources += 'compat/regcomp_enhanced.c'
+    compat_sources += 'compat/regcomp_enhanced.c'
   endif
 elif not get_option('regex').enabled()
   libgit_c_args += [
@@ -1364,7 +1367,7 @@ elif not get_option('regex').enabled()
     '-DNO_MBSUPPORT',
   ]
   build_options_config.set('NO_REGEX', '1')
-  libgit_sources += 'compat/regex/regex.c'
+  compat_sources += 'compat/regex/regex.c'
   libgit_include_directories += 'compat/regex'
 else
     error('Native regex support requested but not found')
@@ -1428,7 +1431,7 @@ else
 
   if get_option('b_sanitize').contains('address')
     libgit_c_args += '-DNO_MMAP'
-    libgit_sources += 'compat/mmap.c'
+    compat_sources += 'compat/mmap.c'
   else
     checkfuncs += { 'mmap': ['mmap.c'] }
   endif
@@ -1438,7 +1441,7 @@ foreach func, impls : checkfuncs
   if not compiler.has_function(func)
     libgit_c_args += '-DNO_' + func.to_upper()
     foreach impl : impls
-      libgit_sources += 'compat/' + impl
+      compat_sources += 'compat/' + impl
     endforeach
   endif
 endforeach
@@ -1449,13 +1452,13 @@ endif
 
 if not compiler.has_function('strdup')
   libgit_c_args += '-DOVERRIDE_STRDUP'
-  libgit_sources += 'compat/strdup.c'
+  compat_sources += 'compat/strdup.c'
 endif
 
 if not compiler.has_function('qsort')
   libgit_c_args += '-DINTERNAL_QSORT'
 endif
-libgit_sources += 'compat/qsort_s.c'
+compat_sources += 'compat/qsort_s.c'
 
 if compiler.has_function('getdelim')
   libgit_c_args += '-DHAVE_GETDELIM'
@@ -1511,7 +1514,7 @@ if meson.can_run_host_binaries() and compiler.run('''
   }
 ''', name: 'fread reads directories').returncode() == 0
   libgit_c_args += '-DFREAD_READS_DIRECTORIES'
-  libgit_sources += 'compat/fopen.c'
+  compat_sources += 'compat/fopen.c'
 endif
 
 if not meson.is_cross_build() and fs.exists('/dev/tty')
@@ -1745,14 +1748,22 @@ else
 endif
 
 libgit = declare_dependency(
-  link_with: static_library('git',
-    sources: libgit_sources,
-    c_args: libgit_c_args + [
-      '-DGIT_VERSION_H="' + version_def_h.full_path() + '"',
-    ],
-    dependencies: libgit_dependencies,
-    include_directories: libgit_include_directories,
-  ),
+  link_with: [
+    static_library('compat',
+      sources: compat_sources,
+      c_args: libgit_c_args,
+      dependencies: libgit_dependencies,
+      include_directories: libgit_include_directories,
+    ),
+    static_library('git',
+      sources: libgit_sources,
+      c_args: libgit_c_args + [
+        '-DGIT_VERSION_H="' + version_def_h.full_path() + '"',
+      ],
+      dependencies: libgit_dependencies,
+      include_directories: libgit_include_directories,
+    ),
+  ],
   compile_args: libgit_c_args,
   dependencies: libgit_dependencies,
   include_directories: libgit_include_directories,

-- 
2.53.0.880.g73c4285caa.dirty

[PATCH 8/8] meson: precompile "git-compat-util.h"

[Patrick Steinhardt]

Every compilation unit in Git is expected to include "git-compat-util.h"
first, either directly or indirectly via "builtin.h". This header papers
over differences between platforms so that we can expect the typical
POSIX functions to exist. Furthermore, it provides functionality that we
end up using everywhere.

This header is thus quite heavy as a consequence. Preprocessing it as a
standalone unit via `clang -E git-compat-util.h` yields over 23,000
lines of code overall. Naturally, it takes quite some time to compile
all of this.

Luckily, this is exactly the kind of use case that precompiled headers
aim to solve: instead of recompiling it every single time, we compile it
once and then link the result into the executable. If include guards are
set up properly it means that the file won't need to be reprocessed.

Set up such a precompiled header for "git-compat-util.h" and wire it up
via Meson. This leads to a significant speedup when performing full
builds:

  Benchmark 1: ninja (rev = HEAD~)
  Time (mean ± σ):     14.467 s ±  0.126 s    [User: 248.133 s, System: 31.298 s]
  Range (min … max):   14.195 s … 14.633 s    10 runs

  Benchmark 2: ninja (rev = HEAD)
    Time (mean ± σ):     10.307 s ±  0.111 s    [User: 173.290 s, System: 23.998 s]
    Range (min … max):   10.030 s … 10.433 s    10 runs

  Summary
    ninja (rev = HEAD) ran
      1.40 ± 0.02 times faster than ninja (rev = HEAD~)

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 meson.build         | 2 ++
 tools/precompiled.h | 1 +
 2 files changed, 3 insertions(+)

diff --git a/meson.build b/meson.build
index cd00be1c23..4b3fd47061 100644
--- a/meson.build
+++ b/meson.build
@@ -1760,6 +1760,7 @@ libgit = declare_dependency(
       c_args: libgit_c_args + [
         '-DGIT_VERSION_H="' + version_def_h.full_path() + '"',
       ],
+      c_pch: [ 'tools/precompiled.h' ],
       dependencies: libgit_dependencies,
       include_directories: libgit_include_directories,
     ),
@@ -1820,6 +1821,7 @@ test_dependencies = [ ]
 
 git_builtin = executable('git',
   sources: builtin_sources + 'git.c',
+  c_pch: [ 'tools/precompiled.h' ],
   dependencies: [libgit_commonmain],
   install: true,
   install_dir: git_exec_path,
diff --git a/tools/precompiled.h b/tools/precompiled.h
new file mode 100644
index 0000000000..b2bec0d2b4
--- /dev/null
+++ b/tools/precompiled.h
@@ -0,0 +1 @@
+#include "git-compat-util.h"

-- 
2.53.0.880.g73c4285caa.dirty

Re: [PATCH 0/8] Some build system improvements

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

> Patrick Steinhardt (8):
>       Introduce new "tools/" directory
>       contrib: move "coccinelle/" directory into "tools/"

Even though "make coccicheck" may not care where the coccinelle
stuff lives, if this step changes the location of the resulting
coccinelle-generated patch in the tree, it will have fallouts to
developer workflows.  The blast radius may be limited, as those
other than Git developers will be hurt, though.

I mention this to forewarn others early, but I do not have a strong
objection against this step; at least not yet, even though I may
change my mind if the fallout turns out to be greater than I
anticipated.



>       contrib: move "coverage-diff.sh" script into "tools/"
>       contrib: move "update-unicode.sh" script into "tools/"
>       builds: move build scripts into "tools/"
>       git-compat-util.h: move warning infra to prepare for PCHs
>       meson: compile compatibility sources separately
>       meson: precompile "git-compat-util.h"
>
>  Makefile                                           | 76 ++++++++---------
>  ci/run-static-analysis.sh                          |  2 +-
>  config.mak.dev                                     |  2 +-
>  contrib/buildsystems/CMakeLists.txt                | 18 ++--
>  contrib/meson.build                                |  1 -
>  contrib/subtree/meson.build                        |  2 +-
>  git-compat-util.h                                  |  8 +-
>  meson.build                                        | 96 +++++++++++++---------
>  tools/README.md                                    |  7 ++
>  check-builtins.sh => tools/check-builtins.sh       |  0
>  {contrib => tools}/coccinelle/.gitignore           |  0
>  {contrib => tools}/coccinelle/README               |  2 +-
>  {contrib => tools}/coccinelle/array.cocci          |  0
>  {contrib => tools}/coccinelle/commit.cocci         |  0
>  .../coccinelle/config_fn_ctx.pending.cocci         |  0
>  {contrib => tools}/coccinelle/equals-null.cocci    |  0
>  {contrib => tools}/coccinelle/flex_alloc.cocci     |  0
>  {contrib => tools}/coccinelle/free.cocci           |  0
>  .../coccinelle/git_config_number.cocci             |  0
>  {contrib => tools}/coccinelle/hashmap.cocci        |  0
>  .../coccinelle/index-compatibility.cocci           |  0
>  {contrib => tools}/coccinelle/meson.build          |  0
>  {contrib => tools}/coccinelle/object_id.cocci      |  0
>  {contrib => tools}/coccinelle/preincr.cocci        |  0
>  {contrib => tools}/coccinelle/qsort.cocci          |  0
>  {contrib => tools}/coccinelle/refs.cocci           |  0
>  {contrib => tools}/coccinelle/spatchcache          |  6 +-
>  {contrib => tools}/coccinelle/strbuf.cocci         |  0
>  {contrib => tools}/coccinelle/swap.cocci           |  0
>  {contrib => tools}/coccinelle/tests/free.c         |  0
>  {contrib => tools}/coccinelle/tests/free.res       |  0
>  {contrib => tools}/coccinelle/the_repository.cocci |  0
>  {contrib => tools}/coccinelle/xcalloc.cocci        |  0
>  {contrib => tools}/coccinelle/xopen.cocci          |  0
>  .../coccinelle/xstrdup_or_null.cocci               |  0
>  {contrib => tools}/coccinelle/xstrncmpz.cocci      |  0
>  {contrib => tools}/coverage-diff.sh                |  0
>  detect-compiler => tools/detect-compiler           |  0
>  generate-cmdlist.sh => tools/generate-cmdlist.sh   |  0
>  .../generate-configlist.sh                         |  0
>  generate-hooklist.sh => tools/generate-hooklist.sh |  0
>  generate-perl.sh => tools/generate-perl.sh         |  0
>  generate-python.sh => tools/generate-python.sh     |  0
>  generate-script.sh => tools/generate-script.sh     |  0
>  tools/meson.build                                  |  1 +
>  tools/precompiled.h                                |  1 +
>  {contrib => tools}/update-unicode/.gitignore       |  0
>  {contrib => tools}/update-unicode/README           |  0
>  .../update-unicode/update_unicode.sh               |  0
>  49 files changed, 123 insertions(+), 99 deletions(-)
>
>
> ---
> base-commit: af2c8a61818d773325ef2324dd135786a03ebca0
> change-id: 20260304-b4-pks-build-infra-improvements-cc4012c5364e

Re: [PATCH 0/8] Some build system improvements

[Patrick Steinhardt]

On Tue, Mar 10, 2026 at 11:23:32AM -0700, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > Patrick Steinhardt (8):
> >       Introduce new "tools/" directory
> >       contrib: move "coccinelle/" directory into "tools/"
> 
> Even though "make coccicheck" may not care where the coccinelle
> stuff lives, if this step changes the location of the resulting
> coccinelle-generated patch in the tree, it will have fallouts to
> developer workflows.  The blast radius may be limited, as those
> other than Git developers will be hurt, though.
> 
> I mention this to forewarn others early, but I do not have a strong
> objection against this step; at least not yet, even though I may
> change my mind if the fallout turns out to be greater than I
> anticipated.

Yup, that's completely fair. Thanks!

Patrick

Re: [PATCH 7/8] meson: compile compatibility sources separately

[Phillip Wood]

On 10/03/2026 17:52, Patrick Steinhardt wrote:
> In the next commit we're about to introduce a precompiled header for
> "git-compat-util.h". The consequence of this change is that we'll
> implicitly include that header for every compilation unit that uses the
> precompiled headers.

Is that a meson thing? I know it defines precompiled headers on a 
per-target basis but does it somehow force each source file to include 
the precompiled header? Looking at the gcc documentation it seems like 
the precompiled header is only included where the original header is 
included. Splitting out the sources that do not depend on 
"git-compat-util.h" does mean we get some additional parallelism while 
we're precompiling the header which is probably a good thing.

Thanks

Phillip

> This is okay for our "normal" library sources and our builtins. But some
> of our compatibility sources do not include the header on purpose, and
> doing so would cause compileir errors.
> 
> Prepare for this change by splitting out compatibility sources into
> their static library. Like this we can selectively enable precompiled
> headers for the library sources.
> 
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>   meson.build | 79 +++++++++++++++++++++++++++++++++++--------------------------
>   1 file changed, 45 insertions(+), 34 deletions(-)
> 
> diff --git a/meson.build b/meson.build
> index 604fe89d2d..cd00be1c23 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -271,6 +271,13 @@ version_gen_environment.set('GIT_VERSION', get_option('version'))
>   
>   compiler = meson.get_compiler('c')
>   
> +compat_sources = [
> +  'compat/nonblock.c',
> +  'compat/obstack.c',
> +  'compat/open.c',
> +  'compat/terminal.c',
> +]
> +
>   libgit_sources = [
>     'abspath.c',
>     'add-interactive.c',
> @@ -304,10 +311,6 @@ libgit_sources = [
>     'commit.c',
>     'common-exit.c',
>     'common-init.c',
> -  'compat/nonblock.c',
> -  'compat/obstack.c',
> -  'compat/open.c',
> -  'compat/terminal.c',
>     'compiler-tricks/not-constant.c',
>     'config.c',
>     'connect.c',
> @@ -1163,7 +1166,7 @@ endif
>   
>   if not has_poll_h and not has_sys_poll_h
>     libgit_c_args += '-DNO_POLL'
> -  libgit_sources += 'compat/poll/poll.c'
> +  compat_sources += 'compat/poll/poll.c'
>     libgit_include_directories += 'compat/poll'
>   endif
>   
> @@ -1179,7 +1182,7 @@ endif
>   # implementation to threat things like drive prefixes specially.
>   if host_machine.system() == 'windows' or not compiler.has_header('libgen.h')
>     libgit_c_args += '-DNO_LIBGEN_H'
> -  libgit_sources += 'compat/basename.c'
> +  compat_sources += 'compat/basename.c'
>   endif
>   
>   if compiler.has_header('paths.h')
> @@ -1209,7 +1212,7 @@ if host_machine.system() != 'windows'
>     foreach symbol : ['inet_ntop', 'inet_pton', 'hstrerror']
>       if not compiler.has_function(symbol, dependencies: networking_dependencies)
>         libgit_c_args += '-DNO_' + symbol.to_upper()
> -      libgit_sources += 'compat/' + symbol + '.c'
> +      compat_sources += 'compat/' + symbol + '.c'
>       endif
>     endforeach
>   endif
> @@ -1251,18 +1254,18 @@ else
>   endif
>   
>   if host_machine.system() == 'darwin'
> -  libgit_sources += 'compat/precompose_utf8.c'
> +  compat_sources += 'compat/precompose_utf8.c'
>     libgit_c_args += '-DPRECOMPOSE_UNICODE'
>     libgit_c_args += '-DPROTECT_HFS_DEFAULT'
>   endif
>   
>   # Configure general compatibility wrappers.
>   if host_machine.system() == 'cygwin'
> -  libgit_sources += [
> +  compat_sources += [
>       'compat/win32/path-utils.c',
>     ]
>   elif host_machine.system() == 'windows'
> -  libgit_sources += [
> +  compat_sources += [
>       'compat/winansi.c',
>       'compat/win32/dirent.c',
>       'compat/win32/flush.c',
> @@ -1289,20 +1292,20 @@ elif host_machine.system() == 'windows'
>     libgit_include_directories += 'compat/win32'
>     if compiler.get_id() == 'msvc'
>       libgit_include_directories += 'compat/vcbuild/include'
> -    libgit_sources += 'compat/msvc.c'
> +    compat_sources += 'compat/msvc.c'
>     else
> -    libgit_sources += 'compat/mingw.c'
> +    compat_sources += 'compat/mingw.c'
>     endif
>   endif
>   
>   if host_machine.system() == 'linux'
> -  libgit_sources += 'compat/linux/procinfo.c'
> +  compat_sources += 'compat/linux/procinfo.c'
>   elif host_machine.system() == 'windows'
> -  libgit_sources += 'compat/win32/trace2_win32_process_info.c'
> +  compat_sources += 'compat/win32/trace2_win32_process_info.c'
>   elif host_machine.system() == 'darwin'
> -  libgit_sources += 'compat/darwin/procinfo.c'
> +  compat_sources += 'compat/darwin/procinfo.c'
>   else
> -  libgit_sources += 'compat/stub/procinfo.c'
> +  compat_sources += 'compat/stub/procinfo.c'
>   endif
>   
>   if host_machine.system() == 'cygwin' or host_machine.system() == 'windows'
> @@ -1315,13 +1318,13 @@ endif
>   
>   # Configure the simple-ipc subsystem required fro the fsmonitor.
>   if host_machine.system() == 'windows'
> -  libgit_sources += [
> +  compat_sources += [
>       'compat/simple-ipc/ipc-shared.c',
>       'compat/simple-ipc/ipc-win32.c',
>     ]
>     libgit_c_args += '-DSUPPORTS_SIMPLE_IPC'
>   else
> -  libgit_sources += [
> +  compat_sources += [
>       'compat/simple-ipc/ipc-shared.c',
>       'compat/simple-ipc/ipc-unix-socket.c',
>     ]
> @@ -1339,7 +1342,7 @@ if fsmonitor_backend != ''
>     libgit_c_args += '-DHAVE_FSMONITOR_DAEMON_BACKEND'
>     libgit_c_args += '-DHAVE_FSMONITOR_OS_SETTINGS'
>   
> -  libgit_sources += [
> +  compat_sources += [
>       'compat/fsmonitor/fsm-health-' + fsmonitor_backend + '.c',
>       'compat/fsmonitor/fsm-ipc-' + fsmonitor_backend + '.c',
>       'compat/fsmonitor/fsm-listen-' + fsmonitor_backend + '.c',
> @@ -1355,7 +1358,7 @@ if not get_option('b_sanitize').contains('address') and get_option('regex').allo
>   
>     if compiler.get_define('REG_ENHANCED', prefix: '#include <regex.h>') != ''
>       libgit_c_args += '-DUSE_ENHANCED_BASIC_REGULAR_EXPRESSIONS'
> -    libgit_sources += 'compat/regcomp_enhanced.c'
> +    compat_sources += 'compat/regcomp_enhanced.c'
>     endif
>   elif not get_option('regex').enabled()
>     libgit_c_args += [
> @@ -1364,7 +1367,7 @@ elif not get_option('regex').enabled()
>       '-DNO_MBSUPPORT',
>     ]
>     build_options_config.set('NO_REGEX', '1')
> -  libgit_sources += 'compat/regex/regex.c'
> +  compat_sources += 'compat/regex/regex.c'
>     libgit_include_directories += 'compat/regex'
>   else
>       error('Native regex support requested but not found')
> @@ -1428,7 +1431,7 @@ else
>   
>     if get_option('b_sanitize').contains('address')
>       libgit_c_args += '-DNO_MMAP'
> -    libgit_sources += 'compat/mmap.c'
> +    compat_sources += 'compat/mmap.c'
>     else
>       checkfuncs += { 'mmap': ['mmap.c'] }
>     endif
> @@ -1438,7 +1441,7 @@ foreach func, impls : checkfuncs
>     if not compiler.has_function(func)
>       libgit_c_args += '-DNO_' + func.to_upper()
>       foreach impl : impls
> -      libgit_sources += 'compat/' + impl
> +      compat_sources += 'compat/' + impl
>       endforeach
>     endif
>   endforeach
> @@ -1449,13 +1452,13 @@ endif
>   
>   if not compiler.has_function('strdup')
>     libgit_c_args += '-DOVERRIDE_STRDUP'
> -  libgit_sources += 'compat/strdup.c'
> +  compat_sources += 'compat/strdup.c'
>   endif
>   
>   if not compiler.has_function('qsort')
>     libgit_c_args += '-DINTERNAL_QSORT'
>   endif
> -libgit_sources += 'compat/qsort_s.c'
> +compat_sources += 'compat/qsort_s.c'
>   
>   if compiler.has_function('getdelim')
>     libgit_c_args += '-DHAVE_GETDELIM'
> @@ -1511,7 +1514,7 @@ if meson.can_run_host_binaries() and compiler.run('''
>     }
>   ''', name: 'fread reads directories').returncode() == 0
>     libgit_c_args += '-DFREAD_READS_DIRECTORIES'
> -  libgit_sources += 'compat/fopen.c'
> +  compat_sources += 'compat/fopen.c'
>   endif
>   
>   if not meson.is_cross_build() and fs.exists('/dev/tty')
> @@ -1745,14 +1748,22 @@ else
>   endif
>   
>   libgit = declare_dependency(
> -  link_with: static_library('git',
> -    sources: libgit_sources,
> -    c_args: libgit_c_args + [
> -      '-DGIT_VERSION_H="' + version_def_h.full_path() + '"',
> -    ],
> -    dependencies: libgit_dependencies,
> -    include_directories: libgit_include_directories,
> -  ),
> +  link_with: [
> +    static_library('compat',
> +      sources: compat_sources,
> +      c_args: libgit_c_args,
> +      dependencies: libgit_dependencies,
> +      include_directories: libgit_include_directories,
> +    ),
> +    static_library('git',
> +      sources: libgit_sources,
> +      c_args: libgit_c_args + [
> +        '-DGIT_VERSION_H="' + version_def_h.full_path() + '"',
> +      ],
> +      dependencies: libgit_dependencies,
> +      include_directories: libgit_include_directories,
> +    ),
> +  ],
>     compile_args: libgit_c_args,
>     dependencies: libgit_dependencies,
>     include_directories: libgit_include_directories,
> 

Re: [PATCH 8/8] meson: precompile "git-compat-util.h"

[Phillip Wood]

On 10/03/2026 17:52, Patrick Steinhardt wrote:
> Every compilation unit in Git is expected to include "git-compat-util.h"
> first, either directly or indirectly via "builtin.h". This header papers
> over differences between platforms so that we can expect the typical
> POSIX functions to exist. Furthermore, it provides functionality that we
> end up using everywhere.
> 
> This header is thus quite heavy as a consequence. Preprocessing it as a
> standalone unit via `clang -E git-compat-util.h` yields over 23,000
> lines of code overall. Naturally, it takes quite some time to compile
> all of this.
> 
> Luckily, this is exactly the kind of use case that precompiled headers
> aim to solve: instead of recompiling it every single time, we compile it
> once and then link the result into the executable. If include guards are
> set up properly it means that the file won't need to be reprocessed.
> 
> Set up such a precompiled header for "git-compat-util.h" and wire it up
> via Meson. This leads to a significant speedup when performing full
> builds:
> 
>    Benchmark 1: ninja (rev = HEAD~)
>    Time (mean ± σ):     14.467 s ±  0.126 s    [User: 248.133 s, System: 31.298 s]
>    Range (min … max):   14.195 s … 14.633 s    10 runs
> 
>    Benchmark 2: ninja (rev = HEAD)
>      Time (mean ± σ):     10.307 s ±  0.111 s    [User: 173.290 s, System: 23.998 s]
>      Range (min … max):   10.030 s … 10.433 s    10 runs
> 
>    Summary
>      ninja (rev = HEAD) ran
>        1.40 ± 0.02 times faster than ninja (rev = HEAD~)

This is a nice speedup for a full build. I'm not sure about dumping the 
precompiled header in tools/ though, it seems like an odd location for a 
header file. It is a requirement of meson that the source for the 
precompiled header lives in a separate directory to the rest of the 
sources but it might be better to adopt the suggestion in the 
documentation of a "pch" (or maybe "precompiled"?) directory rather than 
mixing it in with our build scripts.

Thanks

Phillip


> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>   meson.build         | 2 ++
>   tools/precompiled.h | 1 +
>   2 files changed, 3 insertions(+)
> 
> diff --git a/meson.build b/meson.build
> index cd00be1c23..4b3fd47061 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -1760,6 +1760,7 @@ libgit = declare_dependency(
>         c_args: libgit_c_args + [
>           '-DGIT_VERSION_H="' + version_def_h.full_path() + '"',
>         ],
> +      c_pch: [ 'tools/precompiled.h' ],
>         dependencies: libgit_dependencies,
>         include_directories: libgit_include_directories,
>       ),
> @@ -1820,6 +1821,7 @@ test_dependencies = [ ]
>   
>   git_builtin = executable('git',
>     sources: builtin_sources + 'git.c',
> +  c_pch: [ 'tools/precompiled.h' ],
>     dependencies: [libgit_commonmain],
>     install: true,
>     install_dir: git_exec_path,
> diff --git a/tools/precompiled.h b/tools/precompiled.h
> new file mode 100644
> index 0000000000..b2bec0d2b4
> --- /dev/null
> +++ b/tools/precompiled.h
> @@ -0,0 +1 @@
> +#include "git-compat-util.h"
> 

Re: [PATCH 7/8] meson: compile compatibility sources separately

[Phillip Wood]

On 11/03/2026 14:32, Phillip Wood wrote:
> On 10/03/2026 17:52, Patrick Steinhardt wrote:
>> In the next commit we're about to introduce a precompiled header for
>> "git-compat-util.h". The consequence of this change is that we'll
>> implicitly include that header for every compilation unit that uses the
>> precompiled headers.
> 
> Is that a meson thing? I know it defines precompiled headers on a per- 
> target basis but does it somehow force each source file to include the 
> precompiled header? Looking at the gcc documentation it seems like the 
> precompiled header is only included where the original header is 
> included.

Answering my own question the precompiled header is included via 
"-include" on the commandline. This is necessary in the general case 
because a precompiled header cannot be used once the first C token is seen.

As an aside in git we could probably get away without using "-include" 
because if we include "git-compat-util.h" it is always the first thing 
we do, or we inculde another file like "builtin.h" which immediately 
includes "git-compat-util.h" and so it is included before the first C 
token is seen. However meson cannot rely on that.

I notice the reftable sources don't seem to include "git-compat-util.h", 
do they need special handling here as well?

Thanks

Phillip

> Splitting out the sources that do not depend on "git-compat- 
> util.h" does mean we get some additional parallelism while we're 
> precompiling the header which is probably a good thing.
> 
> Thanks
> 
> Phillip
> 
>> This is okay for our "normal" library sources and our builtins. But some
>> of our compatibility sources do not include the header on purpose, and
>> doing so would cause compileir errors.
>>
>> Prepare for this change by splitting out compatibility sources into
>> their static library. Like this we can selectively enable precompiled
>> headers for the library sources.
>>
>> Signed-off-by: Patrick Steinhardt <ps@pks.im>
>> ---
>>   meson.build | 79 ++++++++++++++++++++++++++++++++++ 
>> +--------------------------
>>   1 file changed, 45 insertions(+), 34 deletions(-)
>>
>> diff --git a/meson.build b/meson.build
>> index 604fe89d2d..cd00be1c23 100644
>> --- a/meson.build
>> +++ b/meson.build
>> @@ -271,6 +271,13 @@ version_gen_environment.set('GIT_VERSION', 
>> get_option('version'))
>>   compiler = meson.get_compiler('c')
>> +compat_sources = [
>> +  'compat/nonblock.c',
>> +  'compat/obstack.c',
>> +  'compat/open.c',
>> +  'compat/terminal.c',
>> +]
>> +
>>   libgit_sources = [
>>     'abspath.c',
>>     'add-interactive.c',
>> @@ -304,10 +311,6 @@ libgit_sources = [
>>     'commit.c',
>>     'common-exit.c',
>>     'common-init.c',
>> -  'compat/nonblock.c',
>> -  'compat/obstack.c',
>> -  'compat/open.c',
>> -  'compat/terminal.c',
>>     'compiler-tricks/not-constant.c',
>>     'config.c',
>>     'connect.c',
>> @@ -1163,7 +1166,7 @@ endif
>>   if not has_poll_h and not has_sys_poll_h
>>     libgit_c_args += '-DNO_POLL'
>> -  libgit_sources += 'compat/poll/poll.c'
>> +  compat_sources += 'compat/poll/poll.c'
>>     libgit_include_directories += 'compat/poll'
>>   endif
>> @@ -1179,7 +1182,7 @@ endif
>>   # implementation to threat things like drive prefixes specially.
>>   if host_machine.system() == 'windows' or not 
>> compiler.has_header('libgen.h')
>>     libgit_c_args += '-DNO_LIBGEN_H'
>> -  libgit_sources += 'compat/basename.c'
>> +  compat_sources += 'compat/basename.c'
>>   endif
>>   if compiler.has_header('paths.h')
>> @@ -1209,7 +1212,7 @@ if host_machine.system() != 'windows'
>>     foreach symbol : ['inet_ntop', 'inet_pton', 'hstrerror']
>>       if not compiler.has_function(symbol, dependencies: 
>> networking_dependencies)
>>         libgit_c_args += '-DNO_' + symbol.to_upper()
>> -      libgit_sources += 'compat/' + symbol + '.c'
>> +      compat_sources += 'compat/' + symbol + '.c'
>>       endif
>>     endforeach
>>   endif
>> @@ -1251,18 +1254,18 @@ else
>>   endif
>>   if host_machine.system() == 'darwin'
>> -  libgit_sources += 'compat/precompose_utf8.c'
>> +  compat_sources += 'compat/precompose_utf8.c'
>>     libgit_c_args += '-DPRECOMPOSE_UNICODE'
>>     libgit_c_args += '-DPROTECT_HFS_DEFAULT'
>>   endif
>>   # Configure general compatibility wrappers.
>>   if host_machine.system() == 'cygwin'
>> -  libgit_sources += [
>> +  compat_sources += [
>>       'compat/win32/path-utils.c',
>>     ]
>>   elif host_machine.system() == 'windows'
>> -  libgit_sources += [
>> +  compat_sources += [
>>       'compat/winansi.c',
>>       'compat/win32/dirent.c',
>>       'compat/win32/flush.c',
>> @@ -1289,20 +1292,20 @@ elif host_machine.system() == 'windows'
>>     libgit_include_directories += 'compat/win32'
>>     if compiler.get_id() == 'msvc'
>>       libgit_include_directories += 'compat/vcbuild/include'
>> -    libgit_sources += 'compat/msvc.c'
>> +    compat_sources += 'compat/msvc.c'
>>     else
>> -    libgit_sources += 'compat/mingw.c'
>> +    compat_sources += 'compat/mingw.c'
>>     endif
>>   endif
>>   if host_machine.system() == 'linux'
>> -  libgit_sources += 'compat/linux/procinfo.c'
>> +  compat_sources += 'compat/linux/procinfo.c'
>>   elif host_machine.system() == 'windows'
>> -  libgit_sources += 'compat/win32/trace2_win32_process_info.c'
>> +  compat_sources += 'compat/win32/trace2_win32_process_info.c'
>>   elif host_machine.system() == 'darwin'
>> -  libgit_sources += 'compat/darwin/procinfo.c'
>> +  compat_sources += 'compat/darwin/procinfo.c'
>>   else
>> -  libgit_sources += 'compat/stub/procinfo.c'
>> +  compat_sources += 'compat/stub/procinfo.c'
>>   endif
>>   if host_machine.system() == 'cygwin' or host_machine.system() == 
>> 'windows'
>> @@ -1315,13 +1318,13 @@ endif
>>   # Configure the simple-ipc subsystem required fro the fsmonitor.
>>   if host_machine.system() == 'windows'
>> -  libgit_sources += [
>> +  compat_sources += [
>>       'compat/simple-ipc/ipc-shared.c',
>>       'compat/simple-ipc/ipc-win32.c',
>>     ]
>>     libgit_c_args += '-DSUPPORTS_SIMPLE_IPC'
>>   else
>> -  libgit_sources += [
>> +  compat_sources += [
>>       'compat/simple-ipc/ipc-shared.c',
>>       'compat/simple-ipc/ipc-unix-socket.c',
>>     ]
>> @@ -1339,7 +1342,7 @@ if fsmonitor_backend != ''
>>     libgit_c_args += '-DHAVE_FSMONITOR_DAEMON_BACKEND'
>>     libgit_c_args += '-DHAVE_FSMONITOR_OS_SETTINGS'
>> -  libgit_sources += [
>> +  compat_sources += [
>>       'compat/fsmonitor/fsm-health-' + fsmonitor_backend + '.c',
>>       'compat/fsmonitor/fsm-ipc-' + fsmonitor_backend + '.c',
>>       'compat/fsmonitor/fsm-listen-' + fsmonitor_backend + '.c',
>> @@ -1355,7 +1358,7 @@ if not 
>> get_option('b_sanitize').contains('address') and get_option('regex').allo
>>     if compiler.get_define('REG_ENHANCED', prefix: '#include 
>> <regex.h>') != ''
>>       libgit_c_args += '-DUSE_ENHANCED_BASIC_REGULAR_EXPRESSIONS'
>> -    libgit_sources += 'compat/regcomp_enhanced.c'
>> +    compat_sources += 'compat/regcomp_enhanced.c'
>>     endif
>>   elif not get_option('regex').enabled()
>>     libgit_c_args += [
>> @@ -1364,7 +1367,7 @@ elif not get_option('regex').enabled()
>>       '-DNO_MBSUPPORT',
>>     ]
>>     build_options_config.set('NO_REGEX', '1')
>> -  libgit_sources += 'compat/regex/regex.c'
>> +  compat_sources += 'compat/regex/regex.c'
>>     libgit_include_directories += 'compat/regex'
>>   else
>>       error('Native regex support requested but not found')
>> @@ -1428,7 +1431,7 @@ else
>>     if get_option('b_sanitize').contains('address')
>>       libgit_c_args += '-DNO_MMAP'
>> -    libgit_sources += 'compat/mmap.c'
>> +    compat_sources += 'compat/mmap.c'
>>     else
>>       checkfuncs += { 'mmap': ['mmap.c'] }
>>     endif
>> @@ -1438,7 +1441,7 @@ foreach func, impls : checkfuncs
>>     if not compiler.has_function(func)
>>       libgit_c_args += '-DNO_' + func.to_upper()
>>       foreach impl : impls
>> -      libgit_sources += 'compat/' + impl
>> +      compat_sources += 'compat/' + impl
>>       endforeach
>>     endif
>>   endforeach
>> @@ -1449,13 +1452,13 @@ endif
>>   if not compiler.has_function('strdup')
>>     libgit_c_args += '-DOVERRIDE_STRDUP'
>> -  libgit_sources += 'compat/strdup.c'
>> +  compat_sources += 'compat/strdup.c'
>>   endif
>>   if not compiler.has_function('qsort')
>>     libgit_c_args += '-DINTERNAL_QSORT'
>>   endif
>> -libgit_sources += 'compat/qsort_s.c'
>> +compat_sources += 'compat/qsort_s.c'
>>   if compiler.has_function('getdelim')
>>     libgit_c_args += '-DHAVE_GETDELIM'
>> @@ -1511,7 +1514,7 @@ if meson.can_run_host_binaries() and 
>> compiler.run('''
>>     }
>>   ''', name: 'fread reads directories').returncode() == 0
>>     libgit_c_args += '-DFREAD_READS_DIRECTORIES'
>> -  libgit_sources += 'compat/fopen.c'
>> +  compat_sources += 'compat/fopen.c'
>>   endif
>>   if not meson.is_cross_build() and fs.exists('/dev/tty')
>> @@ -1745,14 +1748,22 @@ else
>>   endif
>>   libgit = declare_dependency(
>> -  link_with: static_library('git',
>> -    sources: libgit_sources,
>> -    c_args: libgit_c_args + [
>> -      '-DGIT_VERSION_H="' + version_def_h.full_path() + '"',
>> -    ],
>> -    dependencies: libgit_dependencies,
>> -    include_directories: libgit_include_directories,
>> -  ),
>> +  link_with: [
>> +    static_library('compat',
>> +      sources: compat_sources,
>> +      c_args: libgit_c_args,
>> +      dependencies: libgit_dependencies,
>> +      include_directories: libgit_include_directories,
>> +    ),
>> +    static_library('git',
>> +      sources: libgit_sources,
>> +      c_args: libgit_c_args + [
>> +        '-DGIT_VERSION_H="' + version_def_h.full_path() + '"',
>> +      ],
>> +      dependencies: libgit_dependencies,
>> +      include_directories: libgit_include_directories,
>> +    ),
>> +  ],
>>     compile_args: libgit_c_args,
>>     dependencies: libgit_dependencies,
>>     include_directories: libgit_include_directories,
>>
> 
> 

Re: [PATCH 7/8] meson: compile compatibility sources separately

[SZEDER Gábor]

On Wed, Mar 11, 2026 at 02:56:24PM +0000, Phillip Wood wrote:
> On 11/03/2026 14:32, Phillip Wood wrote:
> > On 10/03/2026 17:52, Patrick Steinhardt wrote:
> > > In the next commit we're about to introduce a precompiled header for
> > > "git-compat-util.h". The consequence of this change is that we'll
> > > implicitly include that header for every compilation unit that uses the
> > > precompiled headers.
> > 
> > Is that a meson thing? I know it defines precompiled headers on a per-
> > target basis but does it somehow force each source file to include the
> > precompiled header? Looking at the gcc documentation it seems like the
> > precompiled header is only included where the original header is
> > included.
> 
> Answering my own question the precompiled header is included via "-include"
> on the commandline. This is necessary in the general case because a
> precompiled header cannot be used once the first C token is seen.
> 
> As an aside in git we could probably get away without using "-include"
> because if we include "git-compat-util.h" it is always the first thing we
> do, or we inculde another file like "builtin.h" which immediately includes
> "git-compat-util.h" and so it is included before the first C token is seen.

I couldn't find this in the GCC docs, but Make's documentation states
that "you cannot include a precompiled header from inside another
header." [1]

The strace of compiling a 'builtin/*.c' source file seems to confirm
it, the compiler (gcc-12) does look for 'builtin.h.gch', but doesn't
look for 'git-compat-util.h.gch':

  592662 newfstatat(AT_FDCWD, "builtin/builtin.h.gch", 0x7fff2066e610, 0) = -1 ENOENT (No such file or directory)
  592662 openat(AT_FDCWD, "builtin/builtin.h", O_RDONLY|O_NOCTTY) = -1 ENOENT (No such file or directory)
  592662 newfstatat(AT_FDCWD, "./builtin.h.gch", 0x7fff2066e610, 0) = -1 ENOENT (No such file or directory)
  592662 openat(AT_FDCWD, "./builtin.h", O_RDONLY|O_NOCTTY) = 4
  592662 newfstatat(4, "", {st_mode=S_IFREG|0664, st_size=17968, ...}, AT_EMPTY_PATH) = 0
  592662 read(4, "#ifndef BUILTIN_H\n#define BUILTI"..., 17968) = 17968
  592662 close(4)                         = 0
  592662 openat(AT_FDCWD, "./git-compat-util.h", O_RDONLY|O_NOCTTY) = 4


[1] Second bullet point at:
    https://gcc.gnu.org/onlinedocs/gcc/Precompiled-Headers.html

Re: [PATCH 8/8] meson: precompile "git-compat-util.h"

[Patrick Steinhardt]

On Wed, Mar 11, 2026 at 02:32:36PM +0000, Phillip Wood wrote:
> On 10/03/2026 17:52, Patrick Steinhardt wrote:
> > Every compilation unit in Git is expected to include "git-compat-util.h"
> > first, either directly or indirectly via "builtin.h". This header papers
> > over differences between platforms so that we can expect the typical
> > POSIX functions to exist. Furthermore, it provides functionality that we
> > end up using everywhere.
> > 
> > This header is thus quite heavy as a consequence. Preprocessing it as a
> > standalone unit via `clang -E git-compat-util.h` yields over 23,000
> > lines of code overall. Naturally, it takes quite some time to compile
> > all of this.
> > 
> > Luckily, this is exactly the kind of use case that precompiled headers
> > aim to solve: instead of recompiling it every single time, we compile it
> > once and then link the result into the executable. If include guards are
> > set up properly it means that the file won't need to be reprocessed.
> > 
> > Set up such a precompiled header for "git-compat-util.h" and wire it up
> > via Meson. This leads to a significant speedup when performing full
> > builds:
> > 
> >    Benchmark 1: ninja (rev = HEAD~)
> >    Time (mean ± σ):     14.467 s ±  0.126 s    [User: 248.133 s, System: 31.298 s]
> >    Range (min … max):   14.195 s … 14.633 s    10 runs
> > 
> >    Benchmark 2: ninja (rev = HEAD)
> >      Time (mean ± σ):     10.307 s ±  0.111 s    [User: 173.290 s, System: 23.998 s]
> >      Range (min … max):   10.030 s … 10.433 s    10 runs
> > 
> >    Summary
> >      ninja (rev = HEAD) ran
> >        1.40 ± 0.02 times faster than ninja (rev = HEAD~)
> 
> This is a nice speedup for a full build. I'm not sure about dumping the
> precompiled header in tools/ though, it seems like an odd location for a
> header file. It is a requirement of meson that the source for the
> precompiled header lives in a separate directory to the rest of the sources
> but it might be better to adopt the suggestion in the documentation of a
> "pch" (or maybe "precompiled"?) directory rather than mixing it in with our
> build scripts.

Yeah, it's a bit on the odd side. The reason I decided on "tools/"
though is that it now contains our build infra and developer tooling, so
it's an okayish fit. And I didn't feel like creating a directory for a
single file, only.

Patrick

Re: [PATCH 7/8] meson: compile compatibility sources separately

[Patrick Steinhardt]

On Thu, Mar 12, 2026 at 12:27:49AM +0100, SZEDER Gábor wrote:
> On Wed, Mar 11, 2026 at 02:56:24PM +0000, Phillip Wood wrote:
> > On 11/03/2026 14:32, Phillip Wood wrote:
> > > On 10/03/2026 17:52, Patrick Steinhardt wrote:
> > > > In the next commit we're about to introduce a precompiled header for
> > > > "git-compat-util.h". The consequence of this change is that we'll
> > > > implicitly include that header for every compilation unit that uses the
> > > > precompiled headers.
> > > 
> > > Is that a meson thing? I know it defines precompiled headers on a per-
> > > target basis but does it somehow force each source file to include the
> > > precompiled header? Looking at the gcc documentation it seems like the
> > > precompiled header is only included where the original header is
> > > included.
> > 
> > Answering my own question the precompiled header is included via "-include"
> > on the commandline. This is necessary in the general case because a
> > precompiled header cannot be used once the first C token is seen.
> > 
> > As an aside in git we could probably get away without using "-include"
> > because if we include "git-compat-util.h" it is always the first thing we
> > do, or we inculde another file like "builtin.h" which immediately includes
> > "git-compat-util.h" and so it is included before the first C token is seen.
> 
> I couldn't find this in the GCC docs, but Make's documentation states
> that "you cannot include a precompiled header from inside another
> header." [1]

I think you must have confused something, because the link _does_ point
to GCC's documentation, which does contain the quote.

But yes, overall there can only be one precompiled header, and it cannot
be used after the first token indeed. The use of "-include" is also
explicitly noted for "projects not designed with precompiled headers in
mind", which fits Git. After all, we still have the ability to compile
without precompiled headers, and I don't expect that to go away.

Patrick

> [1] Second bullet point at:
>     https://gcc.gnu.org/onlinedocs/gcc/Precompiled-Headers.html

Re: [PATCH 7/8] meson: compile compatibility sources separately

[Patrick Steinhardt]

On Wed, Mar 11, 2026 at 02:56:24PM +0000, Phillip Wood wrote:
> On 11/03/2026 14:32, Phillip Wood wrote:
> > On 10/03/2026 17:52, Patrick Steinhardt wrote:
> > > In the next commit we're about to introduce a precompiled header for
> > > "git-compat-util.h". The consequence of this change is that we'll
> > > implicitly include that header for every compilation unit that uses the
> > > precompiled headers.
> > 
> > Is that a meson thing? I know it defines precompiled headers on a per-
> > target basis but does it somehow force each source file to include the
> > precompiled header? Looking at the gcc documentation it seems like the
> > precompiled header is only included where the original header is
> > included.
> 
> Answering my own question the precompiled header is included via "-include"
> on the commandline. This is necessary in the general case because a
> precompiled header cannot be used once the first C token is seen.
> 
> As an aside in git we could probably get away without using "-include"
> because if we include "git-compat-util.h" it is always the first thing we
> do, or we inculde another file like "builtin.h" which immediately includes
> "git-compat-util.h" and so it is included before the first C token is seen.
> However meson cannot rely on that.
> 
> I notice the reftable sources don't seem to include "git-compat-util.h", do
> they need special handling here as well?

I don't see a strong reason to do so. The reason why we need to be
careful with "compat/" is that we redefine a bunch of standard symbols
there, and that requires us to play a couple of tricks with preprocessor
macros (see e.g. "compat/fopen.c").

We don't do anything like that in the reftable library, and we already
include "compat/posix.h". So in practice, it shouldn't have much of a
consueqence if we start to include "git-compat-util.h" implicitly over
there.

But if it ever does we can treat it the same as the compat library.

Patrick

Re: [PATCH 7/8] meson: compile compatibility sources separately

[Phillip Wood]

On 12/03/2026 06:22, Patrick Steinhardt wrote:
> On Wed, Mar 11, 2026 at 02:56:24PM +0000, Phillip Wood wrote:
>>
>> I notice the reftable sources don't seem to include "git-compat-util.h", do
>> they need special handling here as well?
> 
> I don't see a strong reason to do so. The reason why we need to be
> careful with "compat/" is that we redefine a bunch of standard symbols
> there, and that requires us to play a couple of tricks with preprocessor
> macros (see e.g. "compat/fopen.c").
> 
> We don't do anything like that in the reftable library, and we already
> include "compat/posix.h". So in practice, it shouldn't have much of a
> consueqence if we start to include "git-compat-util.h" implicitly over
> there.

It does mean we're using different includes when compiling with 
pre-compiled headers compared to compiling without them though which 
means contributors using per-compiled headers could accidentally depend 
on functions that are not included when compiling without them. Wasn't 
the idea behind "compat/posix.h" to avoid including "git-compat-util.h" 
in the reftable code? The commit message for 75a044f748f 
(git-compat-util.h: split out POSIX-emulating bits, 2025-02-18) says

     This intermixing is a bit of a problem for the reftable library as
     we don't want to recreate the POSIX-like interface there. But
     neither do we want to pull in the Git-specific functionality, as it
     is otherwise quite easy to start depending on the Git codebase
     again.

We could precompile "compat/posix.h" for the code that does not want 
"git-compat-util.h"

Thanks

Phillip

> But if it ever does we can treat it the same as the compat library.
> 
> Patrick
> 

Re: [PATCH 7/8] meson: compile compatibility sources separately

[Phillip Wood]

On 12/03/2026 06:21, Patrick Steinhardt wrote:
> On Thu, Mar 12, 2026 at 12:27:49AM +0100, SZEDER Gábor wrote:
>> On Wed, Mar 11, 2026 at 02:56:24PM +0000, Phillip Wood wrote:
>>> On 11/03/2026 14:32, Phillip Wood wrote:
>>>> On 10/03/2026 17:52, Patrick Steinhardt wrote:
>>>>> In the next commit we're about to introduce a precompiled header for
>>>>> "git-compat-util.h". The consequence of this change is that we'll
>>>>> implicitly include that header for every compilation unit that uses the
>>>>> precompiled headers.
>>>>
>>>> Is that a meson thing? I know it defines precompiled headers on a per-
>>>> target basis but does it somehow force each source file to include the
>>>> precompiled header? Looking at the gcc documentation it seems like the
>>>> precompiled header is only included where the original header is
>>>> included.
>>>
>>> Answering my own question the precompiled header is included via "-include"
>>> on the commandline. This is necessary in the general case because a
>>> precompiled header cannot be used once the first C token is seen.
>>>
>>> As an aside in git we could probably get away without using "-include"
>>> because if we include "git-compat-util.h" it is always the first thing we
>>> do, or we inculde another file like "builtin.h" which immediately includes
>>> "git-compat-util.h" and so it is included before the first C token is seen.
>>
>> I couldn't find this in the GCC docs, but Make's documentation states
>> that "you cannot include a precompiled header from inside another
>> header." [1]
> 
> I think you must have confused something, because the link _does_ point
> to GCC's documentation, which does contain the quote.
> 
> But yes, overall there can only be one precompiled header, and it cannot
> be used after the first token indeed. The use of "-include" is also
> explicitly noted for "projects not designed with precompiled headers in
> mind", which fits Git. After all, we still have the ability to compile
> without precompiled headers, and I don't expect that to go away.

Thanks to you both for the explanation. Should we explain this in the 
commit message?

Thanks

Phillip

> Patrick
> 
>> [1] Second bullet point at:
>>      https://gcc.gnu.org/onlinedocs/gcc/Precompiled-Headers.html

Re: [PATCH 0/8] Some build system improvements

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

> Hi,
>
> this patch series contains a small set of build system improvements:
>
>   - The first couple patches introduce a new "tools/" directory that
>     contains items related to our build infrastructure and to our
>     developer tooling. This finally follows up on my promise to do this
>     back when I did the spring clean of "contrib/". [1]
>
>   - The last couple patches introduce precompiled headers into Meson for
>     a nice compilation speedup of ~30%. It's 
>
> The two topics are not really related with one another other than being
> related to build systems. I decided to throw them in the same patch
> series though so that I can introduce "precompiled.h" in "tools/".

We saw a handful of exchanges a few days ago, but are there any more
comments?  Is the topic ready to advance?

Thanks.

Re: [PATCH 7/8] meson: compile compatibility sources separately

[Patrick Steinhardt]

On Fri, Mar 13, 2026 at 10:33:20AM +0000, Phillip Wood wrote:
> On 12/03/2026 06:22, Patrick Steinhardt wrote:
> > On Wed, Mar 11, 2026 at 02:56:24PM +0000, Phillip Wood wrote:
> > > 
> > > I notice the reftable sources don't seem to include "git-compat-util.h", do
> > > they need special handling here as well?
> > 
> > I don't see a strong reason to do so. The reason why we need to be
> > careful with "compat/" is that we redefine a bunch of standard symbols
> > there, and that requires us to play a couple of tricks with preprocessor
> > macros (see e.g. "compat/fopen.c").
> > 
> > We don't do anything like that in the reftable library, and we already
> > include "compat/posix.h". So in practice, it shouldn't have much of a
> > consueqence if we start to include "git-compat-util.h" implicitly over
> > there.
> 
> It does mean we're using different includes when compiling with pre-compiled
> headers compared to compiling without them though which means contributors
> using per-compiled headers could accidentally depend on functions that are
> not included when compiling without them. Wasn't the idea behind
> "compat/posix.h" to avoid including "git-compat-util.h" in the reftable
> code? The commit message for 75a044f748f (git-compat-util.h: split out
> POSIX-emulating bits, 2025-02-18) says
> 
>     This intermixing is a bit of a problem for the reftable library as
>     we don't want to recreate the POSIX-like interface there. But
>     neither do we want to pull in the Git-specific functionality, as it
>     is otherwise quite easy to start depending on the Git codebase
>     again.
> 
> We could precompile "compat/posix.h" for the code that does not want
> "git-compat-util.h"

In theory, yes. But in practice we'd notice this quite fast via other CI
jobs, and it feels a bit ugly to split out so many different libraries.
So I'd propose to keep this as-is for now, but iterate in case we notice
that it _does_ become a problem.

Does that work for you?

Thanks!

Patrick

Re: [PATCH 7/8] meson: compile compatibility sources separately

[Patrick Steinhardt]

On Fri, Mar 13, 2026 at 10:33:36AM +0000, Phillip Wood wrote:
> On 12/03/2026 06:21, Patrick Steinhardt wrote:
> > On Thu, Mar 12, 2026 at 12:27:49AM +0100, SZEDER Gábor wrote:
> > > On Wed, Mar 11, 2026 at 02:56:24PM +0000, Phillip Wood wrote:
> > > > On 11/03/2026 14:32, Phillip Wood wrote:
> > > > > On 10/03/2026 17:52, Patrick Steinhardt wrote:
> > > > > > In the next commit we're about to introduce a precompiled header for
> > > > > > "git-compat-util.h". The consequence of this change is that we'll
> > > > > > implicitly include that header for every compilation unit that uses the
> > > > > > precompiled headers.
> > > > > 
> > > > > Is that a meson thing? I know it defines precompiled headers on a per-
> > > > > target basis but does it somehow force each source file to include the
> > > > > precompiled header? Looking at the gcc documentation it seems like the
> > > > > precompiled header is only included where the original header is
> > > > > included.
> > > > 
> > > > Answering my own question the precompiled header is included via "-include"
> > > > on the commandline. This is necessary in the general case because a
> > > > precompiled header cannot be used once the first C token is seen.
> > > > 
> > > > As an aside in git we could probably get away without using "-include"
> > > > because if we include "git-compat-util.h" it is always the first thing we
> > > > do, or we inculde another file like "builtin.h" which immediately includes
> > > > "git-compat-util.h" and so it is included before the first C token is seen.
> > > 
> > > I couldn't find this in the GCC docs, but Make's documentation states
> > > that "you cannot include a precompiled header from inside another
> > > header." [1]
> > 
> > I think you must have confused something, because the link _does_ point
> > to GCC's documentation, which does contain the quote.
> > 
> > But yes, overall there can only be one precompiled header, and it cannot
> > be used after the first token indeed. The use of "-include" is also
> > explicitly noted for "projects not designed with precompiled headers in
> > mind", which fits Git. After all, we still have the ability to compile
> > without precompiled headers, and I don't expect that to go away.
> 
> Thanks to you both for the explanation. Should we explain this in the commit
> message?

Yeah, let's add a note for how this works internally. Let's also
highlight that we typically only have a single precompiled header by not
passing the headers as an array.

Patrick

Re: [PATCH 0/8] Some build system improvements

[Patrick Steinhardt]

On Fri, Mar 13, 2026 at 03:21:21PM -0700, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > Hi,
> >
> > this patch series contains a small set of build system improvements:
> >
> >   - The first couple patches introduce a new "tools/" directory that
> >     contains items related to our build infrastructure and to our
> >     developer tooling. This finally follows up on my promise to do this
> >     back when I did the spring clean of "contrib/". [1]
> >
> >   - The last couple patches introduce precompiled headers into Meson for
> >     a nice compilation speedup of ~30%. It's 
> >
> > The two topics are not really related with one another other than being
> > related to build systems. I decided to throw them in the same patch
> > series though so that I can introduce "precompiled.h" in "tools/".
> 
> We saw a handful of exchanges a few days ago, but are there any more
> comments?  Is the topic ready to advance?

I'll send a v2 with some minor adjustments in a bit. Thanks!

Patrick

[PATCH v2 0/8] Some build system improvements

[Patrick Steinhardt]

Hi,

this patch series contains a small set of build system improvements:

  - The first couple patches introduce a new "tools/" directory that
    contains items related to our build infrastructure and to our
    developer tooling. This finally follows up on my promise to do this
    back when I did the spring clean of "contrib/". [1]

  - The last couple patches introduce precompiled headers into Meson for
    a nice compilation speedup of ~30%. It's 

The two topics are not really related with one another other than being
related to build systems. I decided to throw them in the same patch
series though so that I can introduce "precompiled.h" in "tools/".

Changes in v2:
  - Turn array of precompiled headers into a simple string.
  - Point out in the commit message that the precompiled header is
    included implicitly.
  - Link to v1: https://lore.kernel.org/r/20260310-b4-pks-build-infra-improvements-v1-0-ec75d0710d6a@pks.im

Thanks!

Patrick

[1]: https://lore.kernel.org/git/20250506-pks-contrib-spring-cleanup-v1-0-e6d5ddd79a72@pks.im/

---
Patrick Steinhardt (8):
      Introduce new "tools/" directory
      contrib: move "coccinelle/" directory into "tools/"
      contrib: move "coverage-diff.sh" script into "tools/"
      contrib: move "update-unicode.sh" script into "tools/"
      builds: move build scripts into "tools/"
      git-compat-util.h: move warning infra to prepare for PCHs
      meson: compile compatibility sources separately
      meson: precompile "git-compat-util.h"

 Makefile                                           | 76 ++++++++---------
 ci/run-static-analysis.sh                          |  2 +-
 config.mak.dev                                     |  2 +-
 contrib/buildsystems/CMakeLists.txt                | 18 ++--
 contrib/meson.build                                |  1 -
 contrib/subtree/meson.build                        |  2 +-
 git-compat-util.h                                  |  8 +-
 meson.build                                        | 96 +++++++++++++---------
 tools/README.md                                    |  7 ++
 check-builtins.sh => tools/check-builtins.sh       |  0
 {contrib => tools}/coccinelle/.gitignore           |  0
 {contrib => tools}/coccinelle/README               |  2 +-
 {contrib => tools}/coccinelle/array.cocci          |  0
 {contrib => tools}/coccinelle/commit.cocci         |  0
 .../coccinelle/config_fn_ctx.pending.cocci         |  0
 {contrib => tools}/coccinelle/equals-null.cocci    |  0
 {contrib => tools}/coccinelle/flex_alloc.cocci     |  0
 {contrib => tools}/coccinelle/free.cocci           |  0
 .../coccinelle/git_config_number.cocci             |  0
 {contrib => tools}/coccinelle/hashmap.cocci        |  0
 .../coccinelle/index-compatibility.cocci           |  0
 {contrib => tools}/coccinelle/meson.build          |  0
 {contrib => tools}/coccinelle/object_id.cocci      |  0
 {contrib => tools}/coccinelle/preincr.cocci        |  0
 {contrib => tools}/coccinelle/qsort.cocci          |  0
 {contrib => tools}/coccinelle/refs.cocci           |  0
 {contrib => tools}/coccinelle/spatchcache          |  6 +-
 {contrib => tools}/coccinelle/strbuf.cocci         |  0
 {contrib => tools}/coccinelle/swap.cocci           |  0
 {contrib => tools}/coccinelle/tests/free.c         |  0
 {contrib => tools}/coccinelle/tests/free.res       |  0
 {contrib => tools}/coccinelle/the_repository.cocci |  0
 {contrib => tools}/coccinelle/xcalloc.cocci        |  0
 {contrib => tools}/coccinelle/xopen.cocci          |  0
 .../coccinelle/xstrdup_or_null.cocci               |  0
 {contrib => tools}/coccinelle/xstrncmpz.cocci      |  0
 {contrib => tools}/coverage-diff.sh                |  0
 detect-compiler => tools/detect-compiler           |  0
 generate-cmdlist.sh => tools/generate-cmdlist.sh   |  0
 .../generate-configlist.sh                         |  0
 generate-hooklist.sh => tools/generate-hooklist.sh |  0
 generate-perl.sh => tools/generate-perl.sh         |  0
 generate-python.sh => tools/generate-python.sh     |  0
 generate-script.sh => tools/generate-script.sh     |  0
 tools/meson.build                                  |  1 +
 tools/precompiled.h                                |  1 +
 {contrib => tools}/update-unicode/.gitignore       |  0
 {contrib => tools}/update-unicode/README           |  0
 .../update-unicode/update_unicode.sh               |  0
 49 files changed, 123 insertions(+), 99 deletions(-)

Range-diff versus v1:

1:  224e28be31 = 1:  9d09d2c39a Introduce new "tools/" directory
2:  b217df51e5 = 2:  1b96bfe0f4 contrib: move "coccinelle/" directory into "tools/"
3:  e371b6c221 = 3:  ed6e90bd36 contrib: move "coverage-diff.sh" script into "tools/"
4:  3efeda9fa0 = 4:  48d8275ed1 contrib: move "update-unicode.sh" script into "tools/"
5:  ee074c1396 = 5:  daafeb3462 builds: move build scripts into "tools/"
6:  d30d4a3119 = 6:  947fc0f7b4 git-compat-util.h: move warning infra to prepare for PCHs
7:  ca118197a9 = 7:  6a2fb99aae meson: compile compatibility sources separately
8:  a865a8650b ! 8:  857b478896 meson: precompile "git-compat-util.h"
    @@ Commit message
         set up properly it means that the file won't need to be reprocessed.
     
         Set up such a precompiled header for "git-compat-util.h" and wire it up
    -    via Meson. This leads to a significant speedup when performing full
    -    builds:
    +    via Meson. This causes Meson to implicitly include the precompiled
    +    header in all compilation units. With GCC and Clang for example this is
    +    done via the "-include" statement [1].
    +
    +    This leads to a significant speedup when performing full builds:
     
           Benchmark 1: ninja (rev = HEAD~)
           Time (mean ± σ):     14.467 s ±  0.126 s    [User: 248.133 s, System: 31.298 s]
    @@ Commit message
             ninja (rev = HEAD) ran
               1.40 ± 0.02 times faster than ninja (rev = HEAD~)
     
    +    [1]: https://gcc.gnu.org/onlinedocs/gcc/Precompiled-Headers.html
    +
         Signed-off-by: Patrick Steinhardt <ps@pks.im>
     
      ## meson.build ##
    @@ meson.build: libgit = declare_dependency(
            c_args: libgit_c_args + [
              '-DGIT_VERSION_H="' + version_def_h.full_path() + '"',
            ],
    -+      c_pch: [ 'tools/precompiled.h' ],
    ++      c_pch: 'tools/precompiled.h',
            dependencies: libgit_dependencies,
            include_directories: libgit_include_directories,
          ),
    @@ meson.build: test_dependencies = [ ]
      
      git_builtin = executable('git',
        sources: builtin_sources + 'git.c',
    -+  c_pch: [ 'tools/precompiled.h' ],
    ++  c_pch: 'tools/precompiled.h',
        dependencies: [libgit_commonmain],
        install: true,
        install_dir: git_exec_path,

---
base-commit: af2c8a61818d773325ef2324dd135786a03ebca0
change-id: 20260304-b4-pks-build-infra-improvements-cc4012c5364e

[PATCH v2 1/8] Introduce new "tools/" directory

[Patrick Steinhardt]

According to its readme, the "contrib/" directory's main intent is to
collect stuff that is not an official part of Git, either because it is
too specialized or because it is still considered experimental. The
reality tells a bit of a different story though: while it _does_ contain
such things, it also contains other things:

  - Our credential helpers, which are being distributed by many
    packagers nowadays and which can be considered "stable".

  - A bunch of tooling that relates to our build and test
    infrastructure.

Especially the second category is somewhat of a sore spot. You really
wouldn't expect build-related tooling to be considered an optional part
of Git. Quite the opposite.

Create a new top-level "tools/" directory to fix this discrepancy. This
directory will contain all kind of tools that are related to our build
infrastructure and that Git developers are likely to use day to day.

For now, this directory doesn't contain anything yet except for a
readme and a Meson skeleton. This will change in subsequent commits.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Makefile          | 2 ++
 meson.build       | 1 +
 tools/README.md   | 7 +++++++
 tools/meson.build | 0
 4 files changed, 10 insertions(+)

diff --git a/Makefile b/Makefile
index f3264d0a37..c7cedbcd7c 100644
--- a/Makefile
+++ b/Makefile
@@ -1066,11 +1066,13 @@ SOURCES_CMD = ( \
 		'*.sh' \
 		':!*[tp][0-9][0-9][0-9][0-9]*' \
 		':!contrib' \
+		':!tools' \
 		2>/dev/null || \
 	$(FIND) . \
 		\( -name .git -type d -prune \) \
 		-o \( -name '[tp][0-9][0-9][0-9][0-9]*' -prune \) \
 		-o \( -name contrib -type d -prune \) \
+		-o \( -name tools -type d -prune \) \
 		-o \( -name build -type d -prune \) \
 		-o \( -name .build -type d -prune \) \
 		-o \( -name 'trash*' -type d -prune \) \
diff --git a/meson.build b/meson.build
index 4b536e0124..1d66b5181e 100644
--- a/meson.build
+++ b/meson.build
@@ -2149,6 +2149,7 @@ else
 endif
 
 subdir('contrib')
+subdir('tools')
 
 # Note that the target is intentionally configured after including the
 # 'contrib' directory, as some tool there also have their own manpages.
diff --git a/tools/README.md b/tools/README.md
new file mode 100644
index 0000000000..d732997136
--- /dev/null
+++ b/tools/README.md
@@ -0,0 +1,7 @@
+Developer Tooling
+-----------------
+
+This directory is expected to contain all sorts of tooling that
+relates to our build infrastructure. This includes scripts and
+inputs required by our build systems, but also scripts that
+developers are expected to run manually.
diff --git a/tools/meson.build b/tools/meson.build
new file mode 100644
index 0000000000..e69de29bb2

-- 
2.53.0.959.g497ff81fa9.dirty

[PATCH v2 2/8] contrib: move "coccinelle/" directory into "tools/"

[Patrick Steinhardt]

The Coccinelle tool is an ingrained part of our build infrastructure. It
is executed by our CI to detect antipatterns and is used to detect
misuses of certain interfaces. It's presence in "contrib/" is thus
rather misleading.

Promote the configuration into the new "tools/" directory.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Makefile                                           | 40 +++++++++++-----------
 ci/run-static-analysis.sh                          |  2 +-
 contrib/meson.build                                |  1 -
 {contrib => tools}/coccinelle/.gitignore           |  0
 {contrib => tools}/coccinelle/README               |  2 +-
 {contrib => tools}/coccinelle/array.cocci          |  0
 {contrib => tools}/coccinelle/commit.cocci         |  0
 .../coccinelle/config_fn_ctx.pending.cocci         |  0
 {contrib => tools}/coccinelle/equals-null.cocci    |  0
 {contrib => tools}/coccinelle/flex_alloc.cocci     |  0
 {contrib => tools}/coccinelle/free.cocci           |  0
 .../coccinelle/git_config_number.cocci             |  0
 {contrib => tools}/coccinelle/hashmap.cocci        |  0
 .../coccinelle/index-compatibility.cocci           |  0
 {contrib => tools}/coccinelle/meson.build          |  0
 {contrib => tools}/coccinelle/object_id.cocci      |  0
 {contrib => tools}/coccinelle/preincr.cocci        |  0
 {contrib => tools}/coccinelle/qsort.cocci          |  0
 {contrib => tools}/coccinelle/refs.cocci           |  0
 {contrib => tools}/coccinelle/spatchcache          |  6 ++--
 {contrib => tools}/coccinelle/strbuf.cocci         |  0
 {contrib => tools}/coccinelle/swap.cocci           |  0
 {contrib => tools}/coccinelle/tests/free.c         |  0
 {contrib => tools}/coccinelle/tests/free.res       |  0
 {contrib => tools}/coccinelle/the_repository.cocci |  0
 {contrib => tools}/coccinelle/xcalloc.cocci        |  0
 {contrib => tools}/coccinelle/xopen.cocci          |  0
 .../coccinelle/xstrdup_or_null.cocci               |  0
 {contrib => tools}/coccinelle/xstrncmpz.cocci      |  0
 tools/meson.build                                  |  1 +
 30 files changed, 26 insertions(+), 26 deletions(-)

diff --git a/Makefile b/Makefile
index c7cedbcd7c..8564b1be36 100644
--- a/Makefile
+++ b/Makefile
@@ -1005,8 +1005,8 @@ SPATCH_TEST_FLAGS =
 # COMPUTE_HEADER_DEPENDENCIES=no this will be unset too.
 SPATCH_USE_O_DEPENDENCIES = YesPlease
 
-# Set SPATCH_CONCAT_COCCI to concatenate the contrib/cocci/*.cocci
-# files into a single contrib/cocci/ALL.cocci before running
+# Set SPATCH_CONCAT_COCCI to concatenate the tools/coccinelle/*.cocci
+# files into a single tools/coccinelle/ALL.cocci before running
 # "coccicheck".
 #
 # Pros:
@@ -1025,7 +1025,7 @@ SPATCH_USE_O_DEPENDENCIES = YesPlease
 #   generate a specific patch, e.g. this will always use strbuf.cocci,
 #   not ALL.cocci:
 #
-#	make contrib/coccinelle/strbuf.cocci.patch
+#	make tools/coccinelle/strbuf.cocci.patch
 SPATCH_CONCAT_COCCI = YesPlease
 
 # Rebuild 'coccicheck' if $(SPATCH), its flags etc. change
@@ -3457,15 +3457,15 @@ check:
 		exit 1; \
 	fi
 
-COCCI_GEN_ALL = .build/contrib/coccinelle/ALL.cocci
-COCCI_GLOB = $(wildcard contrib/coccinelle/*.cocci)
+COCCI_GEN_ALL = .build/tools/coccinelle/ALL.cocci
+COCCI_GLOB = $(wildcard tools/coccinelle/*.cocci)
 COCCI_RULES_TRACKED = $(COCCI_GLOB:%=.build/%)
 COCCI_RULES_TRACKED_NO_PENDING = $(filter-out %.pending.cocci,$(COCCI_RULES_TRACKED))
 COCCI_RULES =
 COCCI_RULES += $(COCCI_GEN_ALL)
 COCCI_RULES += $(COCCI_RULES_TRACKED)
 COCCI_NAMES =
-COCCI_NAMES += $(COCCI_RULES:.build/contrib/coccinelle/%.cocci=%)
+COCCI_NAMES += $(COCCI_RULES:.build/tools/coccinelle/%.cocci=%)
 
 COCCICHECK_PENDING = $(filter %.pending.cocci,$(COCCI_RULES))
 COCCICHECK = $(filter-out $(COCCICHECK_PENDING),$(COCCI_RULES))
@@ -3480,20 +3480,20 @@ COCCICHECK_PATCHES_PENDING_INTREE = $(COCCICHECK_PATCHES_PENDING:.build/%=%)
 # on $(MAKECMDGOALS) that match these $(COCCI_RULES)
 COCCI_RULES_GLOB =
 COCCI_RULES_GLOB += cocci%
-COCCI_RULES_GLOB += .build/contrib/coccinelle/%
+COCCI_RULES_GLOB += .build/tools/coccinelle/%
 COCCI_RULES_GLOB += $(COCCICHECK_PATCHES)
 COCCI_RULES_GLOB += $(COCCICHEC_PATCHES_PENDING)
 COCCI_RULES_GLOB += $(COCCICHECK_PATCHES_INTREE)
 COCCI_RULES_GLOB += $(COCCICHECK_PATCHES_PENDING_INTREE)
 COCCI_GOALS = $(filter $(COCCI_RULES_GLOB),$(MAKECMDGOALS))
 
-COCCI_TEST_RES = $(wildcard contrib/coccinelle/tests/*.res)
+COCCI_TEST_RES = $(wildcard tools/coccinelle/tests/*.res)
 
 $(COCCI_RULES_TRACKED): .build/% : %
 	$(call mkdir_p_parent_template)
 	$(QUIET_CP)cp $< $@
 
-.build/contrib/coccinelle/FOUND_H_SOURCES: $(FOUND_H_SOURCES)
+.build/tools/coccinelle/FOUND_H_SOURCES: $(FOUND_H_SOURCES)
 	$(call mkdir_p_parent_template)
 	$(QUIET_GEN) >$@
 
@@ -3507,12 +3507,12 @@ endif
 define cocci-rule
 
 ## Rule for .build/$(1).patch/$(2); Params:
-# $(1) = e.g. ".build/contrib/coccinelle/free.cocci"
+# $(1) = e.g. ".build/tools/coccinelle/free.cocci"
 # $(2) = e.g. "grep.c"
 # $(3) = e.g. "grep.o"
-COCCI_$(1:.build/contrib/coccinelle/%.cocci=%) += $(1).d/$(2).patch
+COCCI_$(1:.build/tools/coccinelle/%.cocci=%) += $(1).d/$(2).patch
 $(1).d/$(2).patch: GIT-SPATCH-DEFINES
-$(1).d/$(2).patch: $(if $(and $(SPATCH_USE_O_DEPENDENCIES),$(wildcard $(3))),$(3),.build/contrib/coccinelle/FOUND_H_SOURCES)
+$(1).d/$(2).patch: $(if $(and $(SPATCH_USE_O_DEPENDENCIES),$(wildcard $(3))),$(3),.build/tools/coccinelle/FOUND_H_SOURCES)
 $(1).d/$(2).patch: $(1)
 $(1).d/$(2).patch: $(1).d/%.patch : %
 	$$(call mkdir_p_parent_template)
@@ -3538,13 +3538,13 @@ endif
 
 define spatch-rule
 
-.build/contrib/coccinelle/$(1).cocci.patch: $$(COCCI_$(1))
+.build/tools/coccinelle/$(1).cocci.patch: $$(COCCI_$(1))
 	$$(QUIET_SPATCH_CAT)cat $$^ >$$@ && \
 	if test -s $$@; \
 	then \
 		echo '    ' SPATCH result: $$@; \
 	fi
-contrib/coccinelle/$(1).cocci.patch: .build/contrib/coccinelle/$(1).cocci.patch
+tools/coccinelle/$(1).cocci.patch: .build/tools/coccinelle/$(1).cocci.patch
 	$$(QUIET_CP)cp $$< $$@
 
 endef
@@ -3558,9 +3558,9 @@ $(COCCI_TEST_RES_GEN): GIT-SPATCH-DEFINES
 $(COCCI_TEST_RES_GEN): .build/%.res : %.c
 $(COCCI_TEST_RES_GEN): .build/%.res : %.res
 ifdef SPATCH_CONCAT_COCCI
-$(COCCI_TEST_RES_GEN): .build/contrib/coccinelle/tests/%.res : $(COCCI_GEN_ALL)
+$(COCCI_TEST_RES_GEN): .build/tools/coccinelle/tests/%.res : $(COCCI_GEN_ALL)
 else
-$(COCCI_TEST_RES_GEN): .build/contrib/coccinelle/tests/%.res : contrib/coccinelle/%.cocci
+$(COCCI_TEST_RES_GEN): .build/tools/coccinelle/tests/%.res : tools/coccinelle/%.cocci
 endif
 	$(call mkdir_p_parent_template)
 	$(QUIET_SPATCH_TEST)$(SPATCH) $(SPATCH_TEST_FLAGS) \
@@ -3576,14 +3576,14 @@ coccicheck-test: $(COCCI_TEST_RES_GEN)
 coccicheck: coccicheck-test
 
 ifdef SPATCH_CONCAT_COCCI
-COCCICHECK_PATCH_MUST_BE_EMPTY_FILES = contrib/coccinelle/ALL.cocci.patch
+COCCICHECK_PATCH_MUST_BE_EMPTY_FILES = tools/coccinelle/ALL.cocci.patch
 else
 COCCICHECK_PATCH_MUST_BE_EMPTY_FILES = $(COCCICHECK_PATCHES_INTREE)
 endif
 coccicheck: $(COCCICHECK_PATCH_MUST_BE_EMPTY_FILES)
 	! grep ^ $(COCCICHECK_PATCH_MUST_BE_EMPTY_FILES) /dev/null
 
-# See contrib/coccinelle/README
+# See tools/coccinelle/README
 coccicheck-pending: coccicheck-test
 coccicheck-pending: $(COCCICHECK_PATCHES_PENDING_INTREE)
 
@@ -3857,8 +3857,8 @@ profile-clean:
 
 cocciclean:
 	$(RM) GIT-SPATCH-DEFINES
-	$(RM) -r .build/contrib/coccinelle
-	$(RM) contrib/coccinelle/*.cocci.patch
+	$(RM) -r .build/tools/coccinelle
+	$(RM) tools/coccinelle/*.cocci.patch
 
 clean: profile-clean coverage-clean cocciclean
 	$(RM) -r .build $(UNIT_TEST_BIN)
diff --git a/ci/run-static-analysis.sh b/ci/run-static-analysis.sh
index 9e9c72681d..ba67e80b4d 100755
--- a/ci/run-static-analysis.sh
+++ b/ci/run-static-analysis.sh
@@ -10,7 +10,7 @@ make coccicheck
 set +x
 
 fail=
-for cocci_patch in contrib/coccinelle/*.patch
+for cocci_patch in tools/coccinelle/*.patch
 do
 	if test -s "$cocci_patch"
 	then
diff --git a/contrib/meson.build b/contrib/meson.build
index a88c5dfe09..569c23ee76 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -2,5 +2,4 @@ foreach feature : get_option('contrib')
   subdir(feature)
 endforeach
 
-subdir('coccinelle')
 subdir('credential')
diff --git a/contrib/coccinelle/.gitignore b/tools/coccinelle/.gitignore
similarity index 100%
rename from contrib/coccinelle/.gitignore
rename to tools/coccinelle/.gitignore
diff --git a/contrib/coccinelle/README b/tools/coccinelle/README
similarity index 98%
rename from contrib/coccinelle/README
rename to tools/coccinelle/README
index 055ad0e06a..fd0a543cc2 100644
--- a/contrib/coccinelle/README
+++ b/tools/coccinelle/README
@@ -38,7 +38,7 @@ that might be useful to developers.
    So to aid these large scale refactorings, semantic patches can be used.
    However we do not want to store them in the same place as the checks for
    bad patterns, as then automated builds would fail.
-   That is why semantic patches 'contrib/coccinelle/*.pending.cocci'
+   That is why semantic patches 'tools/coccinelle/*.pending.cocci'
    are ignored for checks, and can be applied using 'make coccicheck-pending'.
 
    This allows to expose plans of pending large scale refactorings without
diff --git a/contrib/coccinelle/array.cocci b/tools/coccinelle/array.cocci
similarity index 100%
rename from contrib/coccinelle/array.cocci
rename to tools/coccinelle/array.cocci
diff --git a/contrib/coccinelle/commit.cocci b/tools/coccinelle/commit.cocci
similarity index 100%
rename from contrib/coccinelle/commit.cocci
rename to tools/coccinelle/commit.cocci
diff --git a/contrib/coccinelle/config_fn_ctx.pending.cocci b/tools/coccinelle/config_fn_ctx.pending.cocci
similarity index 100%
rename from contrib/coccinelle/config_fn_ctx.pending.cocci
rename to tools/coccinelle/config_fn_ctx.pending.cocci
diff --git a/contrib/coccinelle/equals-null.cocci b/tools/coccinelle/equals-null.cocci
similarity index 100%
rename from contrib/coccinelle/equals-null.cocci
rename to tools/coccinelle/equals-null.cocci
diff --git a/contrib/coccinelle/flex_alloc.cocci b/tools/coccinelle/flex_alloc.cocci
similarity index 100%
rename from contrib/coccinelle/flex_alloc.cocci
rename to tools/coccinelle/flex_alloc.cocci
diff --git a/contrib/coccinelle/free.cocci b/tools/coccinelle/free.cocci
similarity index 100%
rename from contrib/coccinelle/free.cocci
rename to tools/coccinelle/free.cocci
diff --git a/contrib/coccinelle/git_config_number.cocci b/tools/coccinelle/git_config_number.cocci
similarity index 100%
rename from contrib/coccinelle/git_config_number.cocci
rename to tools/coccinelle/git_config_number.cocci
diff --git a/contrib/coccinelle/hashmap.cocci b/tools/coccinelle/hashmap.cocci
similarity index 100%
rename from contrib/coccinelle/hashmap.cocci
rename to tools/coccinelle/hashmap.cocci
diff --git a/contrib/coccinelle/index-compatibility.cocci b/tools/coccinelle/index-compatibility.cocci
similarity index 100%
rename from contrib/coccinelle/index-compatibility.cocci
rename to tools/coccinelle/index-compatibility.cocci
diff --git a/contrib/coccinelle/meson.build b/tools/coccinelle/meson.build
similarity index 100%
rename from contrib/coccinelle/meson.build
rename to tools/coccinelle/meson.build
diff --git a/contrib/coccinelle/object_id.cocci b/tools/coccinelle/object_id.cocci
similarity index 100%
rename from contrib/coccinelle/object_id.cocci
rename to tools/coccinelle/object_id.cocci
diff --git a/contrib/coccinelle/preincr.cocci b/tools/coccinelle/preincr.cocci
similarity index 100%
rename from contrib/coccinelle/preincr.cocci
rename to tools/coccinelle/preincr.cocci
diff --git a/contrib/coccinelle/qsort.cocci b/tools/coccinelle/qsort.cocci
similarity index 100%
rename from contrib/coccinelle/qsort.cocci
rename to tools/coccinelle/qsort.cocci
diff --git a/contrib/coccinelle/refs.cocci b/tools/coccinelle/refs.cocci
similarity index 100%
rename from contrib/coccinelle/refs.cocci
rename to tools/coccinelle/refs.cocci
diff --git a/contrib/coccinelle/spatchcache b/tools/coccinelle/spatchcache
similarity index 97%
rename from contrib/coccinelle/spatchcache
rename to tools/coccinelle/spatchcache
index 29e9352d8a..efbcbc3827 100755
--- a/contrib/coccinelle/spatchcache
+++ b/tools/coccinelle/spatchcache
@@ -30,7 +30,7 @@
 #	   out of control.
 #
 # This along with the general incremental "make" support for
-# "contrib/coccinelle" makes it viable to (re-)run coccicheck
+# "tools/coccinelle" makes it viable to (re-)run coccicheck
 # e.g. when merging integration branches.
 #
 # Note that the "--very-quiet" flag is currently critical. The cache
@@ -42,7 +42,7 @@
 # to change, so just supply "--very-quiet" for now.
 #
 # To use this, simply set SPATCH to
-# contrib/coccinelle/spatchcache. Then optionally set:
+# tools/coccinelle/spatchcache. Then optionally set:
 #
 #	[spatchCache]
 #		# Optional: path to a custom spatch
@@ -65,7 +65,7 @@
 #
 #	redis-cli FLUSHALL
 #	<make && make coccicheck, as above>
-#	grep -hore HIT -e MISS -e SET -e NOCACHE -e CANTCACHE .build/contrib/coccinelle | sort | uniq -c
+#	grep -hore HIT -e MISS -e SET -e NOCACHE -e CANTCACHE .build/tools/coccinelle | sort | uniq -c
 #	    600 CANTCACHE
 #	   7365 MISS
 #	   7365 SET
diff --git a/contrib/coccinelle/strbuf.cocci b/tools/coccinelle/strbuf.cocci
similarity index 100%
rename from contrib/coccinelle/strbuf.cocci
rename to tools/coccinelle/strbuf.cocci
diff --git a/contrib/coccinelle/swap.cocci b/tools/coccinelle/swap.cocci
similarity index 100%
rename from contrib/coccinelle/swap.cocci
rename to tools/coccinelle/swap.cocci
diff --git a/contrib/coccinelle/tests/free.c b/tools/coccinelle/tests/free.c
similarity index 100%
rename from contrib/coccinelle/tests/free.c
rename to tools/coccinelle/tests/free.c
diff --git a/contrib/coccinelle/tests/free.res b/tools/coccinelle/tests/free.res
similarity index 100%
rename from contrib/coccinelle/tests/free.res
rename to tools/coccinelle/tests/free.res
diff --git a/contrib/coccinelle/the_repository.cocci b/tools/coccinelle/the_repository.cocci
similarity index 100%
rename from contrib/coccinelle/the_repository.cocci
rename to tools/coccinelle/the_repository.cocci
diff --git a/contrib/coccinelle/xcalloc.cocci b/tools/coccinelle/xcalloc.cocci
similarity index 100%
rename from contrib/coccinelle/xcalloc.cocci
rename to tools/coccinelle/xcalloc.cocci
diff --git a/contrib/coccinelle/xopen.cocci b/tools/coccinelle/xopen.cocci
similarity index 100%
rename from contrib/coccinelle/xopen.cocci
rename to tools/coccinelle/xopen.cocci
diff --git a/contrib/coccinelle/xstrdup_or_null.cocci b/tools/coccinelle/xstrdup_or_null.cocci
similarity index 100%
rename from contrib/coccinelle/xstrdup_or_null.cocci
rename to tools/coccinelle/xstrdup_or_null.cocci
diff --git a/contrib/coccinelle/xstrncmpz.cocci b/tools/coccinelle/xstrncmpz.cocci
similarity index 100%
rename from contrib/coccinelle/xstrncmpz.cocci
rename to tools/coccinelle/xstrncmpz.cocci
diff --git a/tools/meson.build b/tools/meson.build
index e69de29bb2..f731f74312 100644
--- a/tools/meson.build
+++ b/tools/meson.build
@@ -0,0 +1 @@
+subdir('coccinelle')

-- 
2.53.0.959.g497ff81fa9.dirty

[PATCH v2 3/8] contrib: move "coverage-diff.sh" script into "tools/"

[Patrick Steinhardt]

The "coverage-diff.sh" script can be used to get information about test
coverage fro the Git codebase. It is thus rather specific to our build
and test infrastructure and part of the developer-facing tooling. The
fact that this script is part of "contrib/" is thus rather misleading
and a historic wart.

Promote the tool into the new "tools/" directory.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 {contrib => tools}/coverage-diff.sh | 0
 1 file changed, 0 insertions(+), 0 deletions(-)

diff --git a/contrib/coverage-diff.sh b/tools/coverage-diff.sh
similarity index 100%
rename from contrib/coverage-diff.sh
rename to tools/coverage-diff.sh

-- 
2.53.0.959.g497ff81fa9.dirty

[PATCH v2 4/8] contrib: move "update-unicode.sh" script into "tools/"

[Patrick Steinhardt]

The "update-unicode.sh" script is used to update the unicode data
compiled into Git whenever a new version of the Unicode standard has
been released. As such, it is a natural part of our developer-facing
tooling, and its presence in "contrib/" is misleading.

Promote the script into the new "tools/" directory.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 {contrib => tools}/update-unicode/.gitignore        | 0
 {contrib => tools}/update-unicode/README            | 0
 {contrib => tools}/update-unicode/update_unicode.sh | 0
 3 files changed, 0 insertions(+), 0 deletions(-)

diff --git a/contrib/update-unicode/.gitignore b/tools/update-unicode/.gitignore
similarity index 100%
rename from contrib/update-unicode/.gitignore
rename to tools/update-unicode/.gitignore
diff --git a/contrib/update-unicode/README b/tools/update-unicode/README
similarity index 100%
rename from contrib/update-unicode/README
rename to tools/update-unicode/README
diff --git a/contrib/update-unicode/update_unicode.sh b/tools/update-unicode/update_unicode.sh
similarity index 100%
rename from contrib/update-unicode/update_unicode.sh
rename to tools/update-unicode/update_unicode.sh

-- 
2.53.0.959.g497ff81fa9.dirty

[PATCH v2 5/8] builds: move build scripts into "tools/"

[Patrick Steinhardt]

We have a bunch of scripts used by our different build systems that are
all located in the top-level directory. Now that we have introduced the
new "tools/" directory though we have a better home for them.

Move the scripts into the "tools/" directory.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Makefile                                           | 34 +++++++++++-----------
 config.mak.dev                                     |  2 +-
 contrib/buildsystems/CMakeLists.txt                | 18 ++++++------
 contrib/subtree/meson.build                        |  2 +-
 meson.build                                        | 14 ++++-----
 check-builtins.sh => tools/check-builtins.sh       |  0
 detect-compiler => tools/detect-compiler           |  0
 generate-cmdlist.sh => tools/generate-cmdlist.sh   |  0
 .../generate-configlist.sh                         |  0
 generate-hooklist.sh => tools/generate-hooklist.sh |  0
 generate-perl.sh => tools/generate-perl.sh         |  0
 generate-python.sh => tools/generate-python.sh     |  0
 generate-script.sh => tools/generate-script.sh     |  0
 13 files changed, 35 insertions(+), 35 deletions(-)

diff --git a/Makefile b/Makefile
index 8564b1be36..322f5940e3 100644
--- a/Makefile
+++ b/Makefile
@@ -2689,21 +2689,21 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
-config-list.h: generate-configlist.sh
+config-list.h: tools/generate-configlist.sh
 	@mkdir -p .depend
-	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh . $@ .depend/config-list.h.d
+	$(QUIET_GEN)$(SHELL_PATH) ./tools/generate-configlist.sh . $@ .depend/config-list.h.d
 
 -include .depend/config-list.h.d
 
-command-list.h: generate-cmdlist.sh command-list.txt
+command-list.h: tools/generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.adoc)
-	$(QUIET_GEN)$(SHELL_PATH) ./generate-cmdlist.sh \
+	$(QUIET_GEN)$(SHELL_PATH) ./tools/generate-cmdlist.sh \
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		. $@
 
-hook-list.h: generate-hooklist.sh Documentation/githooks.adoc
-	$(QUIET_GEN)$(SHELL_PATH) ./generate-hooklist.sh . $@
+hook-list.h: tools/generate-hooklist.sh Documentation/githooks.adoc
+	$(QUIET_GEN)$(SHELL_PATH) ./tools/generate-hooklist.sh . $@
 
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):\
 	$(localedir_SQ):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
@@ -2716,8 +2716,8 @@ GIT-SCRIPT-DEFINES: FORCE
 		echo "$$FLAGS" >$@; \
             fi
 
-$(SCRIPT_SH_GEN) $(SCRIPT_LIB) : % : %.sh generate-script.sh GIT-BUILD-OPTIONS GIT-SCRIPT-DEFINES
-	$(QUIET_GEN)./generate-script.sh "$<" "$@+" ./GIT-BUILD-OPTIONS && \
+$(SCRIPT_SH_GEN) $(SCRIPT_LIB) : % : %.sh tools/generate-script.sh GIT-BUILD-OPTIONS GIT-SCRIPT-DEFINES
+	$(QUIET_GEN)./tools/generate-script.sh "$<" "$@+" ./GIT-BUILD-OPTIONS && \
 	mv $@+ $@
 
 git.rc: git.rc.in GIT-VERSION-GEN GIT-VERSION-FILE
@@ -2757,8 +2757,8 @@ endif
 
 PERL_DEFINES += $(gitexecdir) $(perllibdir) $(localedir)
 
-$(SCRIPT_PERL_GEN): % : %.perl generate-perl.sh GIT-PERL-DEFINES GIT-PERL-HEADER GIT-VERSION-FILE
-	$(QUIET_GEN)$(SHELL_PATH) generate-perl.sh ./GIT-BUILD-OPTIONS ./GIT-VERSION-FILE GIT-PERL-HEADER "$<" "$@+" && \
+$(SCRIPT_PERL_GEN): % : %.perl tools/generate-perl.sh GIT-PERL-DEFINES GIT-PERL-HEADER GIT-VERSION-FILE
+	$(QUIET_GEN)$(SHELL_PATH) tools/generate-perl.sh ./GIT-BUILD-OPTIONS ./GIT-VERSION-FILE GIT-PERL-HEADER "$<" "$@+" && \
 	mv $@+ $@
 
 PERL_DEFINES := $(subst $(space),:,$(PERL_DEFINES))
@@ -2786,8 +2786,8 @@ GIT-PERL-HEADER: $(PERL_HEADER_TEMPLATE) GIT-PERL-DEFINES Makefile
 perllibdir:
 	@echo '$(perllibdir_SQ)'
 
-git-instaweb: git-instaweb.sh generate-script.sh GIT-BUILD-OPTIONS GIT-SCRIPT-DEFINES
-	$(QUIET_GEN)./generate-script.sh "$<" "$@+" ./GIT-BUILD-OPTIONS && \
+git-instaweb: git-instaweb.sh tools/generate-script.sh GIT-BUILD-OPTIONS GIT-SCRIPT-DEFINES
+	$(QUIET_GEN)./tools/generate-script.sh "$<" "$@+" ./GIT-BUILD-OPTIONS && \
 	chmod +x $@+ && \
 	mv $@+ $@
 else # NO_PERL
@@ -2804,9 +2804,9 @@ endif # NO_PERL
 $(SCRIPT_PYTHON_GEN): GIT-BUILD-OPTIONS
 
 ifndef NO_PYTHON
-$(SCRIPT_PYTHON_GEN): generate-python.sh
+$(SCRIPT_PYTHON_GEN): tools/generate-python.sh
 $(SCRIPT_PYTHON_GEN): % : %.py
-	$(QUIET_GEN)$(SHELL_PATH) generate-python.sh ./GIT-BUILD-OPTIONS "$<" "$@"
+	$(QUIET_GEN)$(SHELL_PATH) tools/generate-python.sh ./GIT-BUILD-OPTIONS "$<" "$@"
 else # NO_PYTHON
 $(SCRIPT_PYTHON_GEN): % : unimplemented.sh
 	$(QUIET_GEN) \
@@ -3226,9 +3226,9 @@ endif
 NO_PERL_CPAN_FALLBACKS_SQ = $(subst ','\'',$(NO_PERL_CPAN_FALLBACKS))
 endif
 
-perl/build/lib/%.pm: perl/%.pm generate-perl.sh GIT-BUILD-OPTIONS GIT-VERSION-FILE GIT-PERL-DEFINES
+perl/build/lib/%.pm: perl/%.pm tools/generate-perl.sh GIT-BUILD-OPTIONS GIT-VERSION-FILE GIT-PERL-DEFINES
 	$(call mkdir_p_parent_template)
-	$(QUIET_GEN)$(SHELL_PATH) generate-perl.sh ./GIT-BUILD-OPTIONS ./GIT-VERSION-FILE GIT-PERL-HEADER "$<" "$@"
+	$(QUIET_GEN)$(SHELL_PATH) tools/generate-perl.sh ./GIT-BUILD-OPTIONS ./GIT-VERSION-FILE GIT-PERL-HEADER "$<" "$@"
 
 perl/build/man/man3/Git.3pm: perl/Git.pm
 	$(call mkdir_p_parent_template)
@@ -3936,7 +3936,7 @@ check-docs::
 ### Make sure built-ins do not have dups and listed in git.c
 #
 check-builtins::
-	./check-builtins.sh
+	./tools/check-builtins.sh
 
 ### Test suite coverage testing
 #
diff --git a/config.mak.dev b/config.mak.dev
index e86b6e1b34..c8dcf78779 100644
--- a/config.mak.dev
+++ b/config.mak.dev
@@ -1,5 +1,5 @@
 ifndef COMPILER_FEATURES
-COMPILER_FEATURES := $(shell ./detect-compiler $(CC))
+COMPILER_FEATURES := $(shell ./tools/detect-compiler $(CC))
 endif
 
 ifeq ($(filter no-error,$(DEVOPTS)),)
diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt
index c6cfb874ef..81b4306e72 100644
--- a/contrib/buildsystems/CMakeLists.txt
+++ b/contrib/buildsystems/CMakeLists.txt
@@ -636,7 +636,7 @@ set(EXCLUSION_PROGS_CACHE ${EXCLUSION_PROGS} CACHE STRING "Programs not built" F
 if(NOT EXISTS ${CMAKE_BINARY_DIR}/command-list.h OR NOT EXCLUSION_PROGS_CACHE STREQUAL EXCLUSION_PROGS)
 	list(REMOVE_ITEM EXCLUSION_PROGS empty)
 	message("Generating command-list.h")
-	execute_process(COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/generate-cmdlist.sh"
+	execute_process(COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/tools/generate-cmdlist.sh"
 				${EXCLUSION_PROGS}
 				"${CMAKE_SOURCE_DIR}"
 				"${CMAKE_BINARY_DIR}/command-list.h")
@@ -644,14 +644,14 @@ endif()
 
 if(NOT EXISTS ${CMAKE_BINARY_DIR}/config-list.h)
 	message("Generating config-list.h")
-	execute_process(COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/generate-configlist.sh"
+	execute_process(COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/tools/generate-configlist.sh"
 				"${CMAKE_SOURCE_DIR}"
 				"${CMAKE_BINARY_DIR}/config-list.h")
 endif()
 
 if(NOT EXISTS ${CMAKE_BINARY_DIR}/hook-list.h)
 	message("Generating hook-list.h")
-	execute_process(COMMAND "${SH_EXE}" ${CMAKE_SOURCE_DIR}/generate-hooklist.sh
+	execute_process(COMMAND "${SH_EXE}" ${CMAKE_SOURCE_DIR}/tools/generate-hooklist.sh
 				"${CMAKE_SOURCE_DIR}"
 				"${CMAKE_BINARY_DIR}/hook-list.h")
 endif()
@@ -832,11 +832,11 @@ foreach(script ${git_shell_scripts})
 	endif()
 
 	add_custom_command(OUTPUT "${CMAKE_BINARY_DIR}/${shell_gen_path}"
-		COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/generate-script.sh"
+		COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/tools/generate-script.sh"
 			"${CMAKE_SOURCE_DIR}/${script}.sh"
 			"${CMAKE_BINARY_DIR}/${shell_gen_path}"
 			"${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS"
-		DEPENDS "${CMAKE_SOURCE_DIR}/generate-script.sh"
+		DEPENDS "${CMAKE_SOURCE_DIR}/tools/generate-script.sh"
 			"${CMAKE_SOURCE_DIR}/${script}.sh"
 		VERBATIM)
 	list(APPEND shell_gen ${CMAKE_BINARY_DIR}/${shell_gen_path})
@@ -875,13 +875,13 @@ foreach(script ${git_perl_scripts} ${perl_modules})
 	file(MAKE_DIRECTORY "${CMAKE_BINARY_DIR}/${perl_gen_dir}")
 
 	add_custom_command(OUTPUT "${CMAKE_BINARY_DIR}/${perl_gen_path}"
-		COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/generate-perl.sh"
+		COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/tools/generate-perl.sh"
 			"${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS"
 			"${CMAKE_BINARY_DIR}/GIT-VERSION-FILE"
 			"${CMAKE_BINARY_DIR}/GIT-PERL-HEADER"
 			"${CMAKE_SOURCE_DIR}/${script}"
 			"${CMAKE_BINARY_DIR}/${perl_gen_path}"
-		DEPENDS "${CMAKE_SOURCE_DIR}/generate-perl.sh"
+		DEPENDS "${CMAKE_SOURCE_DIR}/tools/generate-perl.sh"
 			"${CMAKE_SOURCE_DIR}/${script}"
 			"${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS"
 			"${CMAKE_BINARY_DIR}/GIT-VERSION-FILE"
@@ -892,11 +892,11 @@ add_custom_target(perl-gen ALL DEPENDS ${perl_gen})
 
 # Python script
 add_custom_command(OUTPUT "${CMAKE_BINARY_DIR}/git-p4"
-	COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/generate-python.sh"
+	COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/tools/generate-python.sh"
 		"${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS"
 		"${CMAKE_SOURCE_DIR}/git-p4.py"
 		"${CMAKE_BINARY_DIR}/git-p4"
-	DEPENDS "${CMAKE_SOURCE_DIR}/generate-python.sh"
+	DEPENDS "${CMAKE_SOURCE_DIR}/tools/generate-python.sh"
 		"${CMAKE_SOURCE_DIR}/git-p4.py"
 		"${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS"
 	VERBATIM)
diff --git a/contrib/subtree/meson.build b/contrib/subtree/meson.build
index 161435abeb..804c315894 100644
--- a/contrib/subtree/meson.build
+++ b/contrib/subtree/meson.build
@@ -3,7 +3,7 @@ git_subtree = custom_target(
   output: 'git-subtree',
   command: [
     shell,
-    meson.project_source_root() / 'generate-script.sh',
+    meson.project_source_root() / 'tools/generate-script.sh',
     '@INPUT@',
     '@OUTPUT@',
     meson.project_build_root() / 'GIT-BUILD-OPTIONS',
diff --git a/meson.build b/meson.build
index 1d66b5181e..604fe89d2d 100644
--- a/meson.build
+++ b/meson.build
@@ -554,7 +554,7 @@ libgit_sources = [
 libgit_sources += custom_target(
   input: 'command-list.txt',
   output: 'command-list.h',
-  command: [shell, meson.current_source_dir() + '/generate-cmdlist.sh', meson.current_source_dir(), '@OUTPUT@'],
+  command: [shell, meson.current_source_dir() + '/tools/generate-cmdlist.sh', meson.current_source_dir(), '@OUTPUT@'],
   env: script_environment,
 )
 
@@ -723,10 +723,10 @@ endif
 builtin_sources += custom_target(
   output: 'config-list.h',
   depfile: 'config-list.h.d',
-  depend_files: [ 'generate-configlist.sh' ],
+  depend_files: [ 'tools/generate-configlist.sh' ],
   command: [
     shell,
-    meson.current_source_dir() / 'generate-configlist.sh',
+    meson.current_source_dir() / 'tools/generate-configlist.sh',
     meson.current_source_dir(),
     '@OUTPUT@',
     '@DEPFILE@',
@@ -739,7 +739,7 @@ builtin_sources += custom_target(
   output: 'hook-list.h',
   command: [
     shell,
-    meson.current_source_dir() + '/generate-hooklist.sh',
+    meson.current_source_dir() + '/tools/generate-hooklist.sh',
     meson.current_source_dir(),
     '@OUTPUT@',
   ],
@@ -1959,7 +1959,7 @@ foreach script : scripts_sh
     output: fs.stem(script),
     command: [
       shell,
-      meson.project_source_root() / 'generate-script.sh',
+      meson.project_source_root() / 'tools/generate-script.sh',
       '@INPUT@',
       '@OUTPUT@',
       meson.project_build_root() / 'GIT-BUILD-OPTIONS',
@@ -2008,7 +2008,7 @@ if perl_features_enabled
 
   generate_perl_command = [
     shell,
-    meson.project_source_root() / 'generate-perl.sh',
+    meson.project_source_root() / 'tools/generate-perl.sh',
     meson.project_build_root() / 'GIT-BUILD-OPTIONS',
     git_version_file.full_path(),
     perl_header,
@@ -2057,7 +2057,7 @@ if target_python.found()
       output: fs.stem(script),
       command: [
         shell,
-        meson.project_source_root() / 'generate-python.sh',
+        meson.project_source_root() / 'tools/generate-python.sh',
         meson.project_build_root() / 'GIT-BUILD-OPTIONS',
         '@INPUT@',
         '@OUTPUT@',
diff --git a/check-builtins.sh b/tools/check-builtins.sh
similarity index 100%
rename from check-builtins.sh
rename to tools/check-builtins.sh
diff --git a/detect-compiler b/tools/detect-compiler
similarity index 100%
rename from detect-compiler
rename to tools/detect-compiler
diff --git a/generate-cmdlist.sh b/tools/generate-cmdlist.sh
similarity index 100%
rename from generate-cmdlist.sh
rename to tools/generate-cmdlist.sh
diff --git a/generate-configlist.sh b/tools/generate-configlist.sh
similarity index 100%
rename from generate-configlist.sh
rename to tools/generate-configlist.sh
diff --git a/generate-hooklist.sh b/tools/generate-hooklist.sh
similarity index 100%
rename from generate-hooklist.sh
rename to tools/generate-hooklist.sh
diff --git a/generate-perl.sh b/tools/generate-perl.sh
similarity index 100%
rename from generate-perl.sh
rename to tools/generate-perl.sh
diff --git a/generate-python.sh b/tools/generate-python.sh
similarity index 100%
rename from generate-python.sh
rename to tools/generate-python.sh
diff --git a/generate-script.sh b/tools/generate-script.sh
similarity index 100%
rename from generate-script.sh
rename to tools/generate-script.sh

-- 
2.53.0.959.g497ff81fa9.dirty

[PATCH v2 6/8] git-compat-util.h: move warning infra to prepare for PCHs

[Patrick Steinhardt]

The "git-compat-util.h" header is supposed to be the first header
included by every code compilation unit. As such, a subsequent commit
will start to precompile this header to speed up compilation of Git.

This will cause an issue though with the way that we have set up the
"-Wsign-compare" warnings. It is expected that any compilation unit that
fails with that compiler warning sets `DISABLE_SIGN_COMPARE_WARNINGS`
before including "git-compat-util.h". If so, we'll disable the warning
right away via a compiler pragma.

But with precompiled headers we do not know ahead of time whether the
code unit wants to disable those warnings, and thus we'll have to
precompile the header without defining `DISABLE_SIGN_COMPARE_WARNINGS`.
But as the pragma statement is wrapped by our include guards, the second
include of that file will not have the desired effect of disabling the
warnings anymore.

We could fix this issue by declaring a new macro that compilation units
are expected to invoke after having included the file. In retrospect,
that would have been the better way to handle this as it allows for
more flexibility: we could for example toggle the warning for specific
code blocks, only. But changing this now would require a bunch of
changes, and the churn feels excessive for what we gain.

Instead, prepare for the precompiled headers by moving the code outside
of the include guards.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 git-compat-util.h | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/git-compat-util.h b/git-compat-util.h
index bebcf9f698..4b4ea2498f 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -34,10 +34,6 @@ struct strbuf;
 #  define DISABLE_WARNING(warning)
 #endif
 
-#ifdef DISABLE_SIGN_COMPARE_WARNINGS
-DISABLE_WARNING(-Wsign-compare)
-#endif
-
 #undef FLEX_ARRAY
 #define FLEX_ARRAY /* empty - weather balloon to require C99 FAM */
 
@@ -1099,3 +1095,7 @@ extern int not_supposed_to_survive;
 #endif /* CHECK_ASSERTION_SIDE_EFFECTS */
 
 #endif
+
+#ifdef DISABLE_SIGN_COMPARE_WARNINGS
+DISABLE_WARNING(-Wsign-compare)
+#endif

-- 
2.53.0.959.g497ff81fa9.dirty

[PATCH v2 7/8] meson: compile compatibility sources separately

[Patrick Steinhardt]

In the next commit we're about to introduce a precompiled header for
"git-compat-util.h". The consequence of this change is that we'll
implicitly include that header for every compilation unit that uses the
precompiled headers.

This is okay for our "normal" library sources and our builtins. But some
of our compatibility sources do not include the header on purpose, and
doing so would cause compileir errors.

Prepare for this change by splitting out compatibility sources into
their static library. Like this we can selectively enable precompiled
headers for the library sources.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 meson.build | 79 +++++++++++++++++++++++++++++++++++--------------------------
 1 file changed, 45 insertions(+), 34 deletions(-)

diff --git a/meson.build b/meson.build
index 604fe89d2d..cd00be1c23 100644
--- a/meson.build
+++ b/meson.build
@@ -271,6 +271,13 @@ version_gen_environment.set('GIT_VERSION', get_option('version'))
 
 compiler = meson.get_compiler('c')
 
+compat_sources = [
+  'compat/nonblock.c',
+  'compat/obstack.c',
+  'compat/open.c',
+  'compat/terminal.c',
+]
+
 libgit_sources = [
   'abspath.c',
   'add-interactive.c',
@@ -304,10 +311,6 @@ libgit_sources = [
   'commit.c',
   'common-exit.c',
   'common-init.c',
-  'compat/nonblock.c',
-  'compat/obstack.c',
-  'compat/open.c',
-  'compat/terminal.c',
   'compiler-tricks/not-constant.c',
   'config.c',
   'connect.c',
@@ -1163,7 +1166,7 @@ endif
 
 if not has_poll_h and not has_sys_poll_h
   libgit_c_args += '-DNO_POLL'
-  libgit_sources += 'compat/poll/poll.c'
+  compat_sources += 'compat/poll/poll.c'
   libgit_include_directories += 'compat/poll'
 endif
 
@@ -1179,7 +1182,7 @@ endif
 # implementation to threat things like drive prefixes specially.
 if host_machine.system() == 'windows' or not compiler.has_header('libgen.h')
   libgit_c_args += '-DNO_LIBGEN_H'
-  libgit_sources += 'compat/basename.c'
+  compat_sources += 'compat/basename.c'
 endif
 
 if compiler.has_header('paths.h')
@@ -1209,7 +1212,7 @@ if host_machine.system() != 'windows'
   foreach symbol : ['inet_ntop', 'inet_pton', 'hstrerror']
     if not compiler.has_function(symbol, dependencies: networking_dependencies)
       libgit_c_args += '-DNO_' + symbol.to_upper()
-      libgit_sources += 'compat/' + symbol + '.c'
+      compat_sources += 'compat/' + symbol + '.c'
     endif
   endforeach
 endif
@@ -1251,18 +1254,18 @@ else
 endif
 
 if host_machine.system() == 'darwin'
-  libgit_sources += 'compat/precompose_utf8.c'
+  compat_sources += 'compat/precompose_utf8.c'
   libgit_c_args += '-DPRECOMPOSE_UNICODE'
   libgit_c_args += '-DPROTECT_HFS_DEFAULT'
 endif
 
 # Configure general compatibility wrappers.
 if host_machine.system() == 'cygwin'
-  libgit_sources += [
+  compat_sources += [
     'compat/win32/path-utils.c',
   ]
 elif host_machine.system() == 'windows'
-  libgit_sources += [
+  compat_sources += [
     'compat/winansi.c',
     'compat/win32/dirent.c',
     'compat/win32/flush.c',
@@ -1289,20 +1292,20 @@ elif host_machine.system() == 'windows'
   libgit_include_directories += 'compat/win32'
   if compiler.get_id() == 'msvc'
     libgit_include_directories += 'compat/vcbuild/include'
-    libgit_sources += 'compat/msvc.c'
+    compat_sources += 'compat/msvc.c'
   else
-    libgit_sources += 'compat/mingw.c'
+    compat_sources += 'compat/mingw.c'
   endif
 endif
 
 if host_machine.system() == 'linux'
-  libgit_sources += 'compat/linux/procinfo.c'
+  compat_sources += 'compat/linux/procinfo.c'
 elif host_machine.system() == 'windows'
-  libgit_sources += 'compat/win32/trace2_win32_process_info.c'
+  compat_sources += 'compat/win32/trace2_win32_process_info.c'
 elif host_machine.system() == 'darwin'
-  libgit_sources += 'compat/darwin/procinfo.c'
+  compat_sources += 'compat/darwin/procinfo.c'
 else
-  libgit_sources += 'compat/stub/procinfo.c'
+  compat_sources += 'compat/stub/procinfo.c'
 endif
 
 if host_machine.system() == 'cygwin' or host_machine.system() == 'windows'
@@ -1315,13 +1318,13 @@ endif
 
 # Configure the simple-ipc subsystem required fro the fsmonitor.
 if host_machine.system() == 'windows'
-  libgit_sources += [
+  compat_sources += [
     'compat/simple-ipc/ipc-shared.c',
     'compat/simple-ipc/ipc-win32.c',
   ]
   libgit_c_args += '-DSUPPORTS_SIMPLE_IPC'
 else
-  libgit_sources += [
+  compat_sources += [
     'compat/simple-ipc/ipc-shared.c',
     'compat/simple-ipc/ipc-unix-socket.c',
   ]
@@ -1339,7 +1342,7 @@ if fsmonitor_backend != ''
   libgit_c_args += '-DHAVE_FSMONITOR_DAEMON_BACKEND'
   libgit_c_args += '-DHAVE_FSMONITOR_OS_SETTINGS'
 
-  libgit_sources += [
+  compat_sources += [
     'compat/fsmonitor/fsm-health-' + fsmonitor_backend + '.c',
     'compat/fsmonitor/fsm-ipc-' + fsmonitor_backend + '.c',
     'compat/fsmonitor/fsm-listen-' + fsmonitor_backend + '.c',
@@ -1355,7 +1358,7 @@ if not get_option('b_sanitize').contains('address') and get_option('regex').allo
 
   if compiler.get_define('REG_ENHANCED', prefix: '#include <regex.h>') != ''
     libgit_c_args += '-DUSE_ENHANCED_BASIC_REGULAR_EXPRESSIONS'
-    libgit_sources += 'compat/regcomp_enhanced.c'
+    compat_sources += 'compat/regcomp_enhanced.c'
   endif
 elif not get_option('regex').enabled()
   libgit_c_args += [
@@ -1364,7 +1367,7 @@ elif not get_option('regex').enabled()
     '-DNO_MBSUPPORT',
   ]
   build_options_config.set('NO_REGEX', '1')
-  libgit_sources += 'compat/regex/regex.c'
+  compat_sources += 'compat/regex/regex.c'
   libgit_include_directories += 'compat/regex'
 else
     error('Native regex support requested but not found')
@@ -1428,7 +1431,7 @@ else
 
   if get_option('b_sanitize').contains('address')
     libgit_c_args += '-DNO_MMAP'
-    libgit_sources += 'compat/mmap.c'
+    compat_sources += 'compat/mmap.c'
   else
     checkfuncs += { 'mmap': ['mmap.c'] }
   endif
@@ -1438,7 +1441,7 @@ foreach func, impls : checkfuncs
   if not compiler.has_function(func)
     libgit_c_args += '-DNO_' + func.to_upper()
     foreach impl : impls
-      libgit_sources += 'compat/' + impl
+      compat_sources += 'compat/' + impl
     endforeach
   endif
 endforeach
@@ -1449,13 +1452,13 @@ endif
 
 if not compiler.has_function('strdup')
   libgit_c_args += '-DOVERRIDE_STRDUP'
-  libgit_sources += 'compat/strdup.c'
+  compat_sources += 'compat/strdup.c'
 endif
 
 if not compiler.has_function('qsort')
   libgit_c_args += '-DINTERNAL_QSORT'
 endif
-libgit_sources += 'compat/qsort_s.c'
+compat_sources += 'compat/qsort_s.c'
 
 if compiler.has_function('getdelim')
   libgit_c_args += '-DHAVE_GETDELIM'
@@ -1511,7 +1514,7 @@ if meson.can_run_host_binaries() and compiler.run('''
   }
 ''', name: 'fread reads directories').returncode() == 0
   libgit_c_args += '-DFREAD_READS_DIRECTORIES'
-  libgit_sources += 'compat/fopen.c'
+  compat_sources += 'compat/fopen.c'
 endif
 
 if not meson.is_cross_build() and fs.exists('/dev/tty')
@@ -1745,14 +1748,22 @@ else
 endif
 
 libgit = declare_dependency(
-  link_with: static_library('git',
-    sources: libgit_sources,
-    c_args: libgit_c_args + [
-      '-DGIT_VERSION_H="' + version_def_h.full_path() + '"',
-    ],
-    dependencies: libgit_dependencies,
-    include_directories: libgit_include_directories,
-  ),
+  link_with: [
+    static_library('compat',
+      sources: compat_sources,
+      c_args: libgit_c_args,
+      dependencies: libgit_dependencies,
+      include_directories: libgit_include_directories,
+    ),
+    static_library('git',
+      sources: libgit_sources,
+      c_args: libgit_c_args + [
+        '-DGIT_VERSION_H="' + version_def_h.full_path() + '"',
+      ],
+      dependencies: libgit_dependencies,
+      include_directories: libgit_include_directories,
+    ),
+  ],
   compile_args: libgit_c_args,
   dependencies: libgit_dependencies,
   include_directories: libgit_include_directories,

-- 
2.53.0.959.g497ff81fa9.dirty

[PATCH v2 8/8] meson: precompile "git-compat-util.h"

[Patrick Steinhardt]

Every compilation unit in Git is expected to include "git-compat-util.h"
first, either directly or indirectly via "builtin.h". This header papers
over differences between platforms so that we can expect the typical
POSIX functions to exist. Furthermore, it provides functionality that we
end up using everywhere.

This header is thus quite heavy as a consequence. Preprocessing it as a
standalone unit via `clang -E git-compat-util.h` yields over 23,000
lines of code overall. Naturally, it takes quite some time to compile
all of this.

Luckily, this is exactly the kind of use case that precompiled headers
aim to solve: instead of recompiling it every single time, we compile it
once and then link the result into the executable. If include guards are
set up properly it means that the file won't need to be reprocessed.

Set up such a precompiled header for "git-compat-util.h" and wire it up
via Meson. This causes Meson to implicitly include the precompiled
header in all compilation units. With GCC and Clang for example this is
done via the "-include" statement [1].

This leads to a significant speedup when performing full builds:

  Benchmark 1: ninja (rev = HEAD~)
  Time (mean ± σ):     14.467 s ±  0.126 s    [User: 248.133 s, System: 31.298 s]
  Range (min … max):   14.195 s … 14.633 s    10 runs

  Benchmark 2: ninja (rev = HEAD)
    Time (mean ± σ):     10.307 s ±  0.111 s    [User: 173.290 s, System: 23.998 s]
    Range (min … max):   10.030 s … 10.433 s    10 runs

  Summary
    ninja (rev = HEAD) ran
      1.40 ± 0.02 times faster than ninja (rev = HEAD~)

[1]: https://gcc.gnu.org/onlinedocs/gcc/Precompiled-Headers.html

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 meson.build         | 2 ++
 tools/precompiled.h | 1 +
 2 files changed, 3 insertions(+)

diff --git a/meson.build b/meson.build
index cd00be1c23..2002f4795e 100644
--- a/meson.build
+++ b/meson.build
@@ -1760,6 +1760,7 @@ libgit = declare_dependency(
       c_args: libgit_c_args + [
         '-DGIT_VERSION_H="' + version_def_h.full_path() + '"',
       ],
+      c_pch: 'tools/precompiled.h',
       dependencies: libgit_dependencies,
       include_directories: libgit_include_directories,
     ),
@@ -1820,6 +1821,7 @@ test_dependencies = [ ]
 
 git_builtin = executable('git',
   sources: builtin_sources + 'git.c',
+  c_pch: 'tools/precompiled.h',
   dependencies: [libgit_commonmain],
   install: true,
   install_dir: git_exec_path,
diff --git a/tools/precompiled.h b/tools/precompiled.h
new file mode 100644
index 0000000000..b2bec0d2b4
--- /dev/null
+++ b/tools/precompiled.h
@@ -0,0 +1 @@
+#include "git-compat-util.h"

-- 
2.53.0.959.g497ff81fa9.dirty

Re: [PATCH 7/8] meson: compile compatibility sources separately

[Phillip Wood]

On 16/03/2026 08:09, Patrick Steinhardt wrote:
> On Fri, Mar 13, 2026 at 10:33:20AM +0000, Phillip Wood wrote:
>>
>> It does mean we're using different includes when compiling with pre-compiled
>> headers compared to compiling without them though which means contributors
>> using per-compiled headers could accidentally depend on functions that are
>> not included when compiling without them. Wasn't the idea behind
>> "compat/posix.h" to avoid including "git-compat-util.h" in the reftable
>> code? The commit message for 75a044f748f (git-compat-util.h: split out
>> POSIX-emulating bits, 2025-02-18) says
>>
>>      This intermixing is a bit of a problem for the reftable library as
>>      we don't want to recreate the POSIX-like interface there. But
>>      neither do we want to pull in the Git-specific functionality, as it
>>      is otherwise quite easy to start depending on the Git codebase
>>      again.
>>
>> We could precompile "compat/posix.h" for the code that does not want
>> "git-compat-util.h"
> 
> In theory, yes. But in practice we'd notice this quite fast via other CI
> jobs, and it feels a bit ugly to split out so many different libraries.
> So I'd propose to keep this as-is for now, but iterate in case we notice
> that it _does_ become a problem.

Fair enough, getting feedback via the CI when it could be given locally 
by the compiler isn't great but we don't expect this to be a common problem.

Thanks

Phillip


> Does that work for you?
> 
> Thanks!
> 
> Patrick
> 

Re: [PATCH v2 0/8] Some build system improvements

[Phillip Wood]

On 16/03/2026 10:07, Patrick Steinhardt wrote:
> 
> Changes in v2:
>    - Turn array of precompiled headers into a simple string.
>    - Point out in the commit message that the precompiled header is
>      included implicitly.
>    - Link to v1: https://lore.kernel.org/r/20260310-b4-pks-build-infra-improvements-v1-0-ec75d0710d6a@pks.im

Thanks for expanding the commit message in patch 8, the range-diff looks 
good.

Phillip

> Thanks!
> 
> Patrick
> 
> [1]: https://lore.kernel.org/git/20250506-pks-contrib-spring-cleanup-v1-0-e6d5ddd79a72@pks.im/
> 
> ---
> Patrick Steinhardt (8):
>        Introduce new "tools/" directory
>        contrib: move "coccinelle/" directory into "tools/"
>        contrib: move "coverage-diff.sh" script into "tools/"
>        contrib: move "update-unicode.sh" script into "tools/"
>        builds: move build scripts into "tools/"
>        git-compat-util.h: move warning infra to prepare for PCHs
>        meson: compile compatibility sources separately
>        meson: precompile "git-compat-util.h"
> 
>   Makefile                                           | 76 ++++++++---------
>   ci/run-static-analysis.sh                          |  2 +-
>   config.mak.dev                                     |  2 +-
>   contrib/buildsystems/CMakeLists.txt                | 18 ++--
>   contrib/meson.build                                |  1 -
>   contrib/subtree/meson.build                        |  2 +-
>   git-compat-util.h                                  |  8 +-
>   meson.build                                        | 96 +++++++++++++---------
>   tools/README.md                                    |  7 ++
>   check-builtins.sh => tools/check-builtins.sh       |  0
>   {contrib => tools}/coccinelle/.gitignore           |  0
>   {contrib => tools}/coccinelle/README               |  2 +-
>   {contrib => tools}/coccinelle/array.cocci          |  0
>   {contrib => tools}/coccinelle/commit.cocci         |  0
>   .../coccinelle/config_fn_ctx.pending.cocci         |  0
>   {contrib => tools}/coccinelle/equals-null.cocci    |  0
>   {contrib => tools}/coccinelle/flex_alloc.cocci     |  0
>   {contrib => tools}/coccinelle/free.cocci           |  0
>   .../coccinelle/git_config_number.cocci             |  0
>   {contrib => tools}/coccinelle/hashmap.cocci        |  0
>   .../coccinelle/index-compatibility.cocci           |  0
>   {contrib => tools}/coccinelle/meson.build          |  0
>   {contrib => tools}/coccinelle/object_id.cocci      |  0
>   {contrib => tools}/coccinelle/preincr.cocci        |  0
>   {contrib => tools}/coccinelle/qsort.cocci          |  0
>   {contrib => tools}/coccinelle/refs.cocci           |  0
>   {contrib => tools}/coccinelle/spatchcache          |  6 +-
>   {contrib => tools}/coccinelle/strbuf.cocci         |  0
>   {contrib => tools}/coccinelle/swap.cocci           |  0
>   {contrib => tools}/coccinelle/tests/free.c         |  0
>   {contrib => tools}/coccinelle/tests/free.res       |  0
>   {contrib => tools}/coccinelle/the_repository.cocci |  0
>   {contrib => tools}/coccinelle/xcalloc.cocci        |  0
>   {contrib => tools}/coccinelle/xopen.cocci          |  0
>   .../coccinelle/xstrdup_or_null.cocci               |  0
>   {contrib => tools}/coccinelle/xstrncmpz.cocci      |  0
>   {contrib => tools}/coverage-diff.sh                |  0
>   detect-compiler => tools/detect-compiler           |  0
>   generate-cmdlist.sh => tools/generate-cmdlist.sh   |  0
>   .../generate-configlist.sh                         |  0
>   generate-hooklist.sh => tools/generate-hooklist.sh |  0
>   generate-perl.sh => tools/generate-perl.sh         |  0
>   generate-python.sh => tools/generate-python.sh     |  0
>   generate-script.sh => tools/generate-script.sh     |  0
>   tools/meson.build                                  |  1 +
>   tools/precompiled.h                                |  1 +
>   {contrib => tools}/update-unicode/.gitignore       |  0
>   {contrib => tools}/update-unicode/README           |  0
>   .../update-unicode/update_unicode.sh               |  0
>   49 files changed, 123 insertions(+), 99 deletions(-)
> 
> Range-diff versus v1:
> 
> 1:  224e28be31 = 1:  9d09d2c39a Introduce new "tools/" directory
> 2:  b217df51e5 = 2:  1b96bfe0f4 contrib: move "coccinelle/" directory into "tools/"
> 3:  e371b6c221 = 3:  ed6e90bd36 contrib: move "coverage-diff.sh" script into "tools/"
> 4:  3efeda9fa0 = 4:  48d8275ed1 contrib: move "update-unicode.sh" script into "tools/"
> 5:  ee074c1396 = 5:  daafeb3462 builds: move build scripts into "tools/"
> 6:  d30d4a3119 = 6:  947fc0f7b4 git-compat-util.h: move warning infra to prepare for PCHs
> 7:  ca118197a9 = 7:  6a2fb99aae meson: compile compatibility sources separately
> 8:  a865a8650b ! 8:  857b478896 meson: precompile "git-compat-util.h"
>      @@ Commit message
>           set up properly it means that the file won't need to be reprocessed.
>       
>           Set up such a precompiled header for "git-compat-util.h" and wire it up
>      -    via Meson. This leads to a significant speedup when performing full
>      -    builds:
>      +    via Meson. This causes Meson to implicitly include the precompiled
>      +    header in all compilation units. With GCC and Clang for example this is
>      +    done via the "-include" statement [1].
>      +
>      +    This leads to a significant speedup when performing full builds:
>       
>             Benchmark 1: ninja (rev = HEAD~)
>             Time (mean ± σ):     14.467 s ±  0.126 s    [User: 248.133 s, System: 31.298 s]
>      @@ Commit message
>               ninja (rev = HEAD) ran
>                 1.40 ± 0.02 times faster than ninja (rev = HEAD~)
>       
>      +    [1]: https://gcc.gnu.org/onlinedocs/gcc/Precompiled-Headers.html
>      +
>           Signed-off-by: Patrick Steinhardt <ps@pks.im>
>       
>        ## meson.build ##
>      @@ meson.build: libgit = declare_dependency(
>              c_args: libgit_c_args + [
>                '-DGIT_VERSION_H="' + version_def_h.full_path() + '"',
>              ],
>      -+      c_pch: [ 'tools/precompiled.h' ],
>      ++      c_pch: 'tools/precompiled.h',
>              dependencies: libgit_dependencies,
>              include_directories: libgit_include_directories,
>            ),
>      @@ meson.build: test_dependencies = [ ]
>        
>        git_builtin = executable('git',
>          sources: builtin_sources + 'git.c',
>      -+  c_pch: [ 'tools/precompiled.h' ],
>      ++  c_pch: 'tools/precompiled.h',
>          dependencies: [libgit_commonmain],
>          install: true,
>          install_dir: git_exec_path,
> 
> ---
> base-commit: af2c8a61818d773325ef2324dd135786a03ebca0
> change-id: 20260304-b4-pks-build-infra-improvements-cc4012c5364e
> 
> 

Re: [PATCH 7/8] meson: compile compatibility sources separately

[Kristoffer Haugsbakk]

On Tue, Mar 10, 2026, at 18:52, Patrick Steinhardt wrote:
> In the next commit we're about to introduce a precompiled header for
> "git-compat-util.h". The consequence of this change is that we'll
> implicitly include that header for every compilation unit that uses the
> precompiled headers.
>
> This is okay for our "normal" library sources and our builtins. But some
> of our compatibility sources do not include the header on purpose, and
> doing so would cause compileir errors.

s/compileir/compiler/ (or /compilation)

>
> Prepare for this change by splitting out compatibility sources into
> their static library. Like this we can selectively enable precompiled

s/Like this/Like this,/ ?

> headers for the library sources.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
>[snip]

Re: [PATCH 7/8] meson: compile compatibility sources separately

[Patrick Steinhardt]

On Tue, Mar 17, 2026 at 04:38:31PM +0100, Kristoffer Haugsbakk wrote:
> On Tue, Mar 10, 2026, at 18:52, Patrick Steinhardt wrote:
> > In the next commit we're about to introduce a precompiled header for
> > "git-compat-util.h". The consequence of this change is that we'll
> > implicitly include that header for every compilation unit that uses the
> > precompiled headers.
> >
> > This is okay for our "normal" library sources and our builtins. But some
> > of our compatibility sources do not include the header on purpose, and
> > doing so would cause compileir errors.
> 
> s/compileir/compiler/ (or /compilation)
> 
> >
> > Prepare for this change by splitting out compatibility sources into
> > their static library. Like this we can selectively enable precompiled
> 
> s/Like this/Like this,/ ?

Yup, both of these make sense. Thanks!

Patric

[PATCH v3 0/8] Some build system improvements

[Patrick Steinhardt]

Hi,

this patch series contains a small set of build system improvements:

  - The first couple patches introduce a new "tools/" directory that
    contains items related to our build infrastructure and to our
    developer tooling. This finally follows up on my promise to do this
    back when I did the spring clean of "contrib/". [1]

  - The last couple patches introduce precompiled headers into Meson for
    a nice compilation speedup of ~30%. It's 

The two topics are not really related with one another other than being
related to build systems. I decided to throw them in the same patch
series though so that I can introduce "precompiled.h" in "tools/".

Changes in v3:
  - Improve commit message.
  - Link to v2: https://lore.kernel.org/r/20260316-b4-pks-build-infra-improvements-v2-0-4b2c2c0c0425@pks.im

Changes in v2:
  - Turn array of precompiled headers into a simple string.
  - Point out in the commit message that the precompiled header is
    included implicitly.
  - Link to v1: https://lore.kernel.org/r/20260310-b4-pks-build-infra-improvements-v1-0-ec75d0710d6a@pks.im

Thanks!

Patrick

[1]: https://lore.kernel.org/git/20250506-pks-contrib-spring-cleanup-v1-0-e6d5ddd79a72@pks.im/

---
Patrick Steinhardt (8):
      Introduce new "tools/" directory
      contrib: move "coccinelle/" directory into "tools/"
      contrib: move "coverage-diff.sh" script into "tools/"
      contrib: move "update-unicode.sh" script into "tools/"
      builds: move build scripts into "tools/"
      git-compat-util.h: move warning infra to prepare for PCHs
      meson: compile compatibility sources separately
      meson: precompile "git-compat-util.h"

 Makefile                                           | 76 ++++++++---------
 ci/run-static-analysis.sh                          |  2 +-
 config.mak.dev                                     |  2 +-
 contrib/buildsystems/CMakeLists.txt                | 18 ++--
 contrib/meson.build                                |  1 -
 contrib/subtree/meson.build                        |  2 +-
 git-compat-util.h                                  |  8 +-
 meson.build                                        | 96 +++++++++++++---------
 tools/README.md                                    |  7 ++
 check-builtins.sh => tools/check-builtins.sh       |  0
 {contrib => tools}/coccinelle/.gitignore           |  0
 {contrib => tools}/coccinelle/README               |  2 +-
 {contrib => tools}/coccinelle/array.cocci          |  0
 {contrib => tools}/coccinelle/commit.cocci         |  0
 .../coccinelle/config_fn_ctx.pending.cocci         |  0
 {contrib => tools}/coccinelle/equals-null.cocci    |  0
 {contrib => tools}/coccinelle/flex_alloc.cocci     |  0
 {contrib => tools}/coccinelle/free.cocci           |  0
 .../coccinelle/git_config_number.cocci             |  0
 {contrib => tools}/coccinelle/hashmap.cocci        |  0
 .../coccinelle/index-compatibility.cocci           |  0
 {contrib => tools}/coccinelle/meson.build          |  0
 {contrib => tools}/coccinelle/object_id.cocci      |  0
 {contrib => tools}/coccinelle/preincr.cocci        |  0
 {contrib => tools}/coccinelle/qsort.cocci          |  0
 {contrib => tools}/coccinelle/refs.cocci           |  0
 {contrib => tools}/coccinelle/spatchcache          |  6 +-
 {contrib => tools}/coccinelle/strbuf.cocci         |  0
 {contrib => tools}/coccinelle/swap.cocci           |  0
 {contrib => tools}/coccinelle/tests/free.c         |  0
 {contrib => tools}/coccinelle/tests/free.res       |  0
 {contrib => tools}/coccinelle/the_repository.cocci |  0
 {contrib => tools}/coccinelle/xcalloc.cocci        |  0
 {contrib => tools}/coccinelle/xopen.cocci          |  0
 .../coccinelle/xstrdup_or_null.cocci               |  0
 {contrib => tools}/coccinelle/xstrncmpz.cocci      |  0
 {contrib => tools}/coverage-diff.sh                |  0
 detect-compiler => tools/detect-compiler           |  0
 generate-cmdlist.sh => tools/generate-cmdlist.sh   |  0
 .../generate-configlist.sh                         |  0
 generate-hooklist.sh => tools/generate-hooklist.sh |  0
 generate-perl.sh => tools/generate-perl.sh         |  0
 generate-python.sh => tools/generate-python.sh     |  0
 generate-script.sh => tools/generate-script.sh     |  0
 tools/meson.build                                  |  1 +
 tools/precompiled.h                                |  1 +
 {contrib => tools}/update-unicode/.gitignore       |  0
 {contrib => tools}/update-unicode/README           |  0
 .../update-unicode/update_unicode.sh               |  0
 49 files changed, 123 insertions(+), 99 deletions(-)

Range-diff versus v2:

1:  051b66376f = 1:  bc18fe2f2d Introduce new "tools/" directory
2:  275a96c805 = 2:  393b42f433 contrib: move "coccinelle/" directory into "tools/"
3:  afc5a1f8b9 = 3:  00284934e9 contrib: move "coverage-diff.sh" script into "tools/"
4:  909d996f56 = 4:  9dceec07dd contrib: move "update-unicode.sh" script into "tools/"
5:  6396ae723e = 5:  af0ce83627 builds: move build scripts into "tools/"
6:  16b0e9f4fb = 6:  8cb7ea8245 git-compat-util.h: move warning infra to prepare for PCHs
7:  b4cd150fdf ! 7:  495335a97a meson: compile compatibility sources separately
    @@ Commit message
     
         This is okay for our "normal" library sources and our builtins. But some
         of our compatibility sources do not include the header on purpose, and
    -    doing so would cause compileir errors.
    +    doing so would cause compilation errors.
     
         Prepare for this change by splitting out compatibility sources into
    -    their static library. Like this we can selectively enable precompiled
    +    their static library. Like this, we can selectively enable precompiled
         headers for the library sources.
     
         Signed-off-by: Patrick Steinhardt <ps@pks.im>
8:  5899e0318b = 8:  6923592b62 meson: precompile "git-compat-util.h"

---
base-commit: af2c8a61818d773325ef2324dd135786a03ebca0
change-id: 20260304-b4-pks-build-infra-improvements-cc4012c5364e

[PATCH v3 1/8] Introduce new "tools/" directory

[Patrick Steinhardt]

According to its readme, the "contrib/" directory's main intent is to
collect stuff that is not an official part of Git, either because it is
too specialized or because it is still considered experimental. The
reality tells a bit of a different story though: while it _does_ contain
such things, it also contains other things:

  - Our credential helpers, which are being distributed by many
    packagers nowadays and which can be considered "stable".

  - A bunch of tooling that relates to our build and test
    infrastructure.

Especially the second category is somewhat of a sore spot. You really
wouldn't expect build-related tooling to be considered an optional part
of Git. Quite the opposite.

Create a new top-level "tools/" directory to fix this discrepancy. This
directory will contain all kind of tools that are related to our build
infrastructure and that Git developers are likely to use day to day.

For now, this directory doesn't contain anything yet except for a
readme and a Meson skeleton. This will change in subsequent commits.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Makefile          | 2 ++
 meson.build       | 1 +
 tools/README.md   | 7 +++++++
 tools/meson.build | 0
 4 files changed, 10 insertions(+)

diff --git a/Makefile b/Makefile
index f3264d0a37..c7cedbcd7c 100644
--- a/Makefile
+++ b/Makefile
@@ -1066,11 +1066,13 @@ SOURCES_CMD = ( \
 		'*.sh' \
 		':!*[tp][0-9][0-9][0-9][0-9]*' \
 		':!contrib' \
+		':!tools' \
 		2>/dev/null || \
 	$(FIND) . \
 		\( -name .git -type d -prune \) \
 		-o \( -name '[tp][0-9][0-9][0-9][0-9]*' -prune \) \
 		-o \( -name contrib -type d -prune \) \
+		-o \( -name tools -type d -prune \) \
 		-o \( -name build -type d -prune \) \
 		-o \( -name .build -type d -prune \) \
 		-o \( -name 'trash*' -type d -prune \) \
diff --git a/meson.build b/meson.build
index 4b536e0124..1d66b5181e 100644
--- a/meson.build
+++ b/meson.build
@@ -2149,6 +2149,7 @@ else
 endif
 
 subdir('contrib')
+subdir('tools')
 
 # Note that the target is intentionally configured after including the
 # 'contrib' directory, as some tool there also have their own manpages.
diff --git a/tools/README.md b/tools/README.md
new file mode 100644
index 0000000000..d732997136
--- /dev/null
+++ b/tools/README.md
@@ -0,0 +1,7 @@
+Developer Tooling
+-----------------
+
+This directory is expected to contain all sorts of tooling that
+relates to our build infrastructure. This includes scripts and
+inputs required by our build systems, but also scripts that
+developers are expected to run manually.
diff --git a/tools/meson.build b/tools/meson.build
new file mode 100644
index 0000000000..e69de29bb2

-- 
2.53.0.959.g497ff81fa9.dirty

[PATCH v3 2/8] contrib: move "coccinelle/" directory into "tools/"

[Patrick Steinhardt]

The Coccinelle tool is an ingrained part of our build infrastructure. It
is executed by our CI to detect antipatterns and is used to detect
misuses of certain interfaces. It's presence in "contrib/" is thus
rather misleading.

Promote the configuration into the new "tools/" directory.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Makefile                                           | 40 +++++++++++-----------
 ci/run-static-analysis.sh                          |  2 +-
 contrib/meson.build                                |  1 -
 {contrib => tools}/coccinelle/.gitignore           |  0
 {contrib => tools}/coccinelle/README               |  2 +-
 {contrib => tools}/coccinelle/array.cocci          |  0
 {contrib => tools}/coccinelle/commit.cocci         |  0
 .../coccinelle/config_fn_ctx.pending.cocci         |  0
 {contrib => tools}/coccinelle/equals-null.cocci    |  0
 {contrib => tools}/coccinelle/flex_alloc.cocci     |  0
 {contrib => tools}/coccinelle/free.cocci           |  0
 .../coccinelle/git_config_number.cocci             |  0
 {contrib => tools}/coccinelle/hashmap.cocci        |  0
 .../coccinelle/index-compatibility.cocci           |  0
 {contrib => tools}/coccinelle/meson.build          |  0
 {contrib => tools}/coccinelle/object_id.cocci      |  0
 {contrib => tools}/coccinelle/preincr.cocci        |  0
 {contrib => tools}/coccinelle/qsort.cocci          |  0
 {contrib => tools}/coccinelle/refs.cocci           |  0
 {contrib => tools}/coccinelle/spatchcache          |  6 ++--
 {contrib => tools}/coccinelle/strbuf.cocci         |  0
 {contrib => tools}/coccinelle/swap.cocci           |  0
 {contrib => tools}/coccinelle/tests/free.c         |  0
 {contrib => tools}/coccinelle/tests/free.res       |  0
 {contrib => tools}/coccinelle/the_repository.cocci |  0
 {contrib => tools}/coccinelle/xcalloc.cocci        |  0
 {contrib => tools}/coccinelle/xopen.cocci          |  0
 .../coccinelle/xstrdup_or_null.cocci               |  0
 {contrib => tools}/coccinelle/xstrncmpz.cocci      |  0
 tools/meson.build                                  |  1 +
 30 files changed, 26 insertions(+), 26 deletions(-)

diff --git a/Makefile b/Makefile
index c7cedbcd7c..8564b1be36 100644
--- a/Makefile
+++ b/Makefile
@@ -1005,8 +1005,8 @@ SPATCH_TEST_FLAGS =
 # COMPUTE_HEADER_DEPENDENCIES=no this will be unset too.
 SPATCH_USE_O_DEPENDENCIES = YesPlease
 
-# Set SPATCH_CONCAT_COCCI to concatenate the contrib/cocci/*.cocci
-# files into a single contrib/cocci/ALL.cocci before running
+# Set SPATCH_CONCAT_COCCI to concatenate the tools/coccinelle/*.cocci
+# files into a single tools/coccinelle/ALL.cocci before running
 # "coccicheck".
 #
 # Pros:
@@ -1025,7 +1025,7 @@ SPATCH_USE_O_DEPENDENCIES = YesPlease
 #   generate a specific patch, e.g. this will always use strbuf.cocci,
 #   not ALL.cocci:
 #
-#	make contrib/coccinelle/strbuf.cocci.patch
+#	make tools/coccinelle/strbuf.cocci.patch
 SPATCH_CONCAT_COCCI = YesPlease
 
 # Rebuild 'coccicheck' if $(SPATCH), its flags etc. change
@@ -3457,15 +3457,15 @@ check:
 		exit 1; \
 	fi
 
-COCCI_GEN_ALL = .build/contrib/coccinelle/ALL.cocci
-COCCI_GLOB = $(wildcard contrib/coccinelle/*.cocci)
+COCCI_GEN_ALL = .build/tools/coccinelle/ALL.cocci
+COCCI_GLOB = $(wildcard tools/coccinelle/*.cocci)
 COCCI_RULES_TRACKED = $(COCCI_GLOB:%=.build/%)
 COCCI_RULES_TRACKED_NO_PENDING = $(filter-out %.pending.cocci,$(COCCI_RULES_TRACKED))
 COCCI_RULES =
 COCCI_RULES += $(COCCI_GEN_ALL)
 COCCI_RULES += $(COCCI_RULES_TRACKED)
 COCCI_NAMES =
-COCCI_NAMES += $(COCCI_RULES:.build/contrib/coccinelle/%.cocci=%)
+COCCI_NAMES += $(COCCI_RULES:.build/tools/coccinelle/%.cocci=%)
 
 COCCICHECK_PENDING = $(filter %.pending.cocci,$(COCCI_RULES))
 COCCICHECK = $(filter-out $(COCCICHECK_PENDING),$(COCCI_RULES))
@@ -3480,20 +3480,20 @@ COCCICHECK_PATCHES_PENDING_INTREE = $(COCCICHECK_PATCHES_PENDING:.build/%=%)
 # on $(MAKECMDGOALS) that match these $(COCCI_RULES)
 COCCI_RULES_GLOB =
 COCCI_RULES_GLOB += cocci%
-COCCI_RULES_GLOB += .build/contrib/coccinelle/%
+COCCI_RULES_GLOB += .build/tools/coccinelle/%
 COCCI_RULES_GLOB += $(COCCICHECK_PATCHES)
 COCCI_RULES_GLOB += $(COCCICHEC_PATCHES_PENDING)
 COCCI_RULES_GLOB += $(COCCICHECK_PATCHES_INTREE)
 COCCI_RULES_GLOB += $(COCCICHECK_PATCHES_PENDING_INTREE)
 COCCI_GOALS = $(filter $(COCCI_RULES_GLOB),$(MAKECMDGOALS))
 
-COCCI_TEST_RES = $(wildcard contrib/coccinelle/tests/*.res)
+COCCI_TEST_RES = $(wildcard tools/coccinelle/tests/*.res)
 
 $(COCCI_RULES_TRACKED): .build/% : %
 	$(call mkdir_p_parent_template)
 	$(QUIET_CP)cp $< $@
 
-.build/contrib/coccinelle/FOUND_H_SOURCES: $(FOUND_H_SOURCES)
+.build/tools/coccinelle/FOUND_H_SOURCES: $(FOUND_H_SOURCES)
 	$(call mkdir_p_parent_template)
 	$(QUIET_GEN) >$@
 
@@ -3507,12 +3507,12 @@ endif
 define cocci-rule
 
 ## Rule for .build/$(1).patch/$(2); Params:
-# $(1) = e.g. ".build/contrib/coccinelle/free.cocci"
+# $(1) = e.g. ".build/tools/coccinelle/free.cocci"
 # $(2) = e.g. "grep.c"
 # $(3) = e.g. "grep.o"
-COCCI_$(1:.build/contrib/coccinelle/%.cocci=%) += $(1).d/$(2).patch
+COCCI_$(1:.build/tools/coccinelle/%.cocci=%) += $(1).d/$(2).patch
 $(1).d/$(2).patch: GIT-SPATCH-DEFINES
-$(1).d/$(2).patch: $(if $(and $(SPATCH_USE_O_DEPENDENCIES),$(wildcard $(3))),$(3),.build/contrib/coccinelle/FOUND_H_SOURCES)
+$(1).d/$(2).patch: $(if $(and $(SPATCH_USE_O_DEPENDENCIES),$(wildcard $(3))),$(3),.build/tools/coccinelle/FOUND_H_SOURCES)
 $(1).d/$(2).patch: $(1)
 $(1).d/$(2).patch: $(1).d/%.patch : %
 	$$(call mkdir_p_parent_template)
@@ -3538,13 +3538,13 @@ endif
 
 define spatch-rule
 
-.build/contrib/coccinelle/$(1).cocci.patch: $$(COCCI_$(1))
+.build/tools/coccinelle/$(1).cocci.patch: $$(COCCI_$(1))
 	$$(QUIET_SPATCH_CAT)cat $$^ >$$@ && \
 	if test -s $$@; \
 	then \
 		echo '    ' SPATCH result: $$@; \
 	fi
-contrib/coccinelle/$(1).cocci.patch: .build/contrib/coccinelle/$(1).cocci.patch
+tools/coccinelle/$(1).cocci.patch: .build/tools/coccinelle/$(1).cocci.patch
 	$$(QUIET_CP)cp $$< $$@
 
 endef
@@ -3558,9 +3558,9 @@ $(COCCI_TEST_RES_GEN): GIT-SPATCH-DEFINES
 $(COCCI_TEST_RES_GEN): .build/%.res : %.c
 $(COCCI_TEST_RES_GEN): .build/%.res : %.res
 ifdef SPATCH_CONCAT_COCCI
-$(COCCI_TEST_RES_GEN): .build/contrib/coccinelle/tests/%.res : $(COCCI_GEN_ALL)
+$(COCCI_TEST_RES_GEN): .build/tools/coccinelle/tests/%.res : $(COCCI_GEN_ALL)
 else
-$(COCCI_TEST_RES_GEN): .build/contrib/coccinelle/tests/%.res : contrib/coccinelle/%.cocci
+$(COCCI_TEST_RES_GEN): .build/tools/coccinelle/tests/%.res : tools/coccinelle/%.cocci
 endif
 	$(call mkdir_p_parent_template)
 	$(QUIET_SPATCH_TEST)$(SPATCH) $(SPATCH_TEST_FLAGS) \
@@ -3576,14 +3576,14 @@ coccicheck-test: $(COCCI_TEST_RES_GEN)
 coccicheck: coccicheck-test
 
 ifdef SPATCH_CONCAT_COCCI
-COCCICHECK_PATCH_MUST_BE_EMPTY_FILES = contrib/coccinelle/ALL.cocci.patch
+COCCICHECK_PATCH_MUST_BE_EMPTY_FILES = tools/coccinelle/ALL.cocci.patch
 else
 COCCICHECK_PATCH_MUST_BE_EMPTY_FILES = $(COCCICHECK_PATCHES_INTREE)
 endif
 coccicheck: $(COCCICHECK_PATCH_MUST_BE_EMPTY_FILES)
 	! grep ^ $(COCCICHECK_PATCH_MUST_BE_EMPTY_FILES) /dev/null
 
-# See contrib/coccinelle/README
+# See tools/coccinelle/README
 coccicheck-pending: coccicheck-test
 coccicheck-pending: $(COCCICHECK_PATCHES_PENDING_INTREE)
 
@@ -3857,8 +3857,8 @@ profile-clean:
 
 cocciclean:
 	$(RM) GIT-SPATCH-DEFINES
-	$(RM) -r .build/contrib/coccinelle
-	$(RM) contrib/coccinelle/*.cocci.patch
+	$(RM) -r .build/tools/coccinelle
+	$(RM) tools/coccinelle/*.cocci.patch
 
 clean: profile-clean coverage-clean cocciclean
 	$(RM) -r .build $(UNIT_TEST_BIN)
diff --git a/ci/run-static-analysis.sh b/ci/run-static-analysis.sh
index 9e9c72681d..ba67e80b4d 100755
--- a/ci/run-static-analysis.sh
+++ b/ci/run-static-analysis.sh
@@ -10,7 +10,7 @@ make coccicheck
 set +x
 
 fail=
-for cocci_patch in contrib/coccinelle/*.patch
+for cocci_patch in tools/coccinelle/*.patch
 do
 	if test -s "$cocci_patch"
 	then
diff --git a/contrib/meson.build b/contrib/meson.build
index a88c5dfe09..569c23ee76 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -2,5 +2,4 @@ foreach feature : get_option('contrib')
   subdir(feature)
 endforeach
 
-subdir('coccinelle')
 subdir('credential')
diff --git a/contrib/coccinelle/.gitignore b/tools/coccinelle/.gitignore
similarity index 100%
rename from contrib/coccinelle/.gitignore
rename to tools/coccinelle/.gitignore
diff --git a/contrib/coccinelle/README b/tools/coccinelle/README
similarity index 98%
rename from contrib/coccinelle/README
rename to tools/coccinelle/README
index 055ad0e06a..fd0a543cc2 100644
--- a/contrib/coccinelle/README
+++ b/tools/coccinelle/README
@@ -38,7 +38,7 @@ that might be useful to developers.
    So to aid these large scale refactorings, semantic patches can be used.
    However we do not want to store them in the same place as the checks for
    bad patterns, as then automated builds would fail.
-   That is why semantic patches 'contrib/coccinelle/*.pending.cocci'
+   That is why semantic patches 'tools/coccinelle/*.pending.cocci'
    are ignored for checks, and can be applied using 'make coccicheck-pending'.
 
    This allows to expose plans of pending large scale refactorings without
diff --git a/contrib/coccinelle/array.cocci b/tools/coccinelle/array.cocci
similarity index 100%
rename from contrib/coccinelle/array.cocci
rename to tools/coccinelle/array.cocci
diff --git a/contrib/coccinelle/commit.cocci b/tools/coccinelle/commit.cocci
similarity index 100%
rename from contrib/coccinelle/commit.cocci
rename to tools/coccinelle/commit.cocci
diff --git a/contrib/coccinelle/config_fn_ctx.pending.cocci b/tools/coccinelle/config_fn_ctx.pending.cocci
similarity index 100%
rename from contrib/coccinelle/config_fn_ctx.pending.cocci
rename to tools/coccinelle/config_fn_ctx.pending.cocci
diff --git a/contrib/coccinelle/equals-null.cocci b/tools/coccinelle/equals-null.cocci
similarity index 100%
rename from contrib/coccinelle/equals-null.cocci
rename to tools/coccinelle/equals-null.cocci
diff --git a/contrib/coccinelle/flex_alloc.cocci b/tools/coccinelle/flex_alloc.cocci
similarity index 100%
rename from contrib/coccinelle/flex_alloc.cocci
rename to tools/coccinelle/flex_alloc.cocci
diff --git a/contrib/coccinelle/free.cocci b/tools/coccinelle/free.cocci
similarity index 100%
rename from contrib/coccinelle/free.cocci
rename to tools/coccinelle/free.cocci
diff --git a/contrib/coccinelle/git_config_number.cocci b/tools/coccinelle/git_config_number.cocci
similarity index 100%
rename from contrib/coccinelle/git_config_number.cocci
rename to tools/coccinelle/git_config_number.cocci
diff --git a/contrib/coccinelle/hashmap.cocci b/tools/coccinelle/hashmap.cocci
similarity index 100%
rename from contrib/coccinelle/hashmap.cocci
rename to tools/coccinelle/hashmap.cocci
diff --git a/contrib/coccinelle/index-compatibility.cocci b/tools/coccinelle/index-compatibility.cocci
similarity index 100%
rename from contrib/coccinelle/index-compatibility.cocci
rename to tools/coccinelle/index-compatibility.cocci
diff --git a/contrib/coccinelle/meson.build b/tools/coccinelle/meson.build
similarity index 100%
rename from contrib/coccinelle/meson.build
rename to tools/coccinelle/meson.build
diff --git a/contrib/coccinelle/object_id.cocci b/tools/coccinelle/object_id.cocci
similarity index 100%
rename from contrib/coccinelle/object_id.cocci
rename to tools/coccinelle/object_id.cocci
diff --git a/contrib/coccinelle/preincr.cocci b/tools/coccinelle/preincr.cocci
similarity index 100%
rename from contrib/coccinelle/preincr.cocci
rename to tools/coccinelle/preincr.cocci
diff --git a/contrib/coccinelle/qsort.cocci b/tools/coccinelle/qsort.cocci
similarity index 100%
rename from contrib/coccinelle/qsort.cocci
rename to tools/coccinelle/qsort.cocci
diff --git a/contrib/coccinelle/refs.cocci b/tools/coccinelle/refs.cocci
similarity index 100%
rename from contrib/coccinelle/refs.cocci
rename to tools/coccinelle/refs.cocci
diff --git a/contrib/coccinelle/spatchcache b/tools/coccinelle/spatchcache
similarity index 97%
rename from contrib/coccinelle/spatchcache
rename to tools/coccinelle/spatchcache
index 29e9352d8a..efbcbc3827 100755
--- a/contrib/coccinelle/spatchcache
+++ b/tools/coccinelle/spatchcache
@@ -30,7 +30,7 @@
 #	   out of control.
 #
 # This along with the general incremental "make" support for
-# "contrib/coccinelle" makes it viable to (re-)run coccicheck
+# "tools/coccinelle" makes it viable to (re-)run coccicheck
 # e.g. when merging integration branches.
 #
 # Note that the "--very-quiet" flag is currently critical. The cache
@@ -42,7 +42,7 @@
 # to change, so just supply "--very-quiet" for now.
 #
 # To use this, simply set SPATCH to
-# contrib/coccinelle/spatchcache. Then optionally set:
+# tools/coccinelle/spatchcache. Then optionally set:
 #
 #	[spatchCache]
 #		# Optional: path to a custom spatch
@@ -65,7 +65,7 @@
 #
 #	redis-cli FLUSHALL
 #	<make && make coccicheck, as above>
-#	grep -hore HIT -e MISS -e SET -e NOCACHE -e CANTCACHE .build/contrib/coccinelle | sort | uniq -c
+#	grep -hore HIT -e MISS -e SET -e NOCACHE -e CANTCACHE .build/tools/coccinelle | sort | uniq -c
 #	    600 CANTCACHE
 #	   7365 MISS
 #	   7365 SET
diff --git a/contrib/coccinelle/strbuf.cocci b/tools/coccinelle/strbuf.cocci
similarity index 100%
rename from contrib/coccinelle/strbuf.cocci
rename to tools/coccinelle/strbuf.cocci
diff --git a/contrib/coccinelle/swap.cocci b/tools/coccinelle/swap.cocci
similarity index 100%
rename from contrib/coccinelle/swap.cocci
rename to tools/coccinelle/swap.cocci
diff --git a/contrib/coccinelle/tests/free.c b/tools/coccinelle/tests/free.c
similarity index 100%
rename from contrib/coccinelle/tests/free.c
rename to tools/coccinelle/tests/free.c
diff --git a/contrib/coccinelle/tests/free.res b/tools/coccinelle/tests/free.res
similarity index 100%
rename from contrib/coccinelle/tests/free.res
rename to tools/coccinelle/tests/free.res
diff --git a/contrib/coccinelle/the_repository.cocci b/tools/coccinelle/the_repository.cocci
similarity index 100%
rename from contrib/coccinelle/the_repository.cocci
rename to tools/coccinelle/the_repository.cocci
diff --git a/contrib/coccinelle/xcalloc.cocci b/tools/coccinelle/xcalloc.cocci
similarity index 100%
rename from contrib/coccinelle/xcalloc.cocci
rename to tools/coccinelle/xcalloc.cocci
diff --git a/contrib/coccinelle/xopen.cocci b/tools/coccinelle/xopen.cocci
similarity index 100%
rename from contrib/coccinelle/xopen.cocci
rename to tools/coccinelle/xopen.cocci
diff --git a/contrib/coccinelle/xstrdup_or_null.cocci b/tools/coccinelle/xstrdup_or_null.cocci
similarity index 100%
rename from contrib/coccinelle/xstrdup_or_null.cocci
rename to tools/coccinelle/xstrdup_or_null.cocci
diff --git a/contrib/coccinelle/xstrncmpz.cocci b/tools/coccinelle/xstrncmpz.cocci
similarity index 100%
rename from contrib/coccinelle/xstrncmpz.cocci
rename to tools/coccinelle/xstrncmpz.cocci
diff --git a/tools/meson.build b/tools/meson.build
index e69de29bb2..f731f74312 100644
--- a/tools/meson.build
+++ b/tools/meson.build
@@ -0,0 +1 @@
+subdir('coccinelle')

-- 
2.53.0.959.g497ff81fa9.dirty

[PATCH v3 3/8] contrib: move "coverage-diff.sh" script into "tools/"

[Patrick Steinhardt]

The "coverage-diff.sh" script can be used to get information about test
coverage fro the Git codebase. It is thus rather specific to our build
and test infrastructure and part of the developer-facing tooling. The
fact that this script is part of "contrib/" is thus rather misleading
and a historic wart.

Promote the tool into the new "tools/" directory.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 {contrib => tools}/coverage-diff.sh | 0
 1 file changed, 0 insertions(+), 0 deletions(-)

diff --git a/contrib/coverage-diff.sh b/tools/coverage-diff.sh
similarity index 100%
rename from contrib/coverage-diff.sh
rename to tools/coverage-diff.sh

-- 
2.53.0.959.g497ff81fa9.dirty

[PATCH v3 4/8] contrib: move "update-unicode.sh" script into "tools/"

[Patrick Steinhardt]

The "update-unicode.sh" script is used to update the unicode data
compiled into Git whenever a new version of the Unicode standard has
been released. As such, it is a natural part of our developer-facing
tooling, and its presence in "contrib/" is misleading.

Promote the script into the new "tools/" directory.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 {contrib => tools}/update-unicode/.gitignore        | 0
 {contrib => tools}/update-unicode/README            | 0
 {contrib => tools}/update-unicode/update_unicode.sh | 0
 3 files changed, 0 insertions(+), 0 deletions(-)

diff --git a/contrib/update-unicode/.gitignore b/tools/update-unicode/.gitignore
similarity index 100%
rename from contrib/update-unicode/.gitignore
rename to tools/update-unicode/.gitignore
diff --git a/contrib/update-unicode/README b/tools/update-unicode/README
similarity index 100%
rename from contrib/update-unicode/README
rename to tools/update-unicode/README
diff --git a/contrib/update-unicode/update_unicode.sh b/tools/update-unicode/update_unicode.sh
similarity index 100%
rename from contrib/update-unicode/update_unicode.sh
rename to tools/update-unicode/update_unicode.sh

-- 
2.53.0.959.g497ff81fa9.dirty

[PATCH v3 5/8] builds: move build scripts into "tools/"

[Patrick Steinhardt]

We have a bunch of scripts used by our different build systems that are
all located in the top-level directory. Now that we have introduced the
new "tools/" directory though we have a better home for them.

Move the scripts into the "tools/" directory.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Makefile                                           | 34 +++++++++++-----------
 config.mak.dev                                     |  2 +-
 contrib/buildsystems/CMakeLists.txt                | 18 ++++++------
 contrib/subtree/meson.build                        |  2 +-
 meson.build                                        | 14 ++++-----
 check-builtins.sh => tools/check-builtins.sh       |  0
 detect-compiler => tools/detect-compiler           |  0
 generate-cmdlist.sh => tools/generate-cmdlist.sh   |  0
 .../generate-configlist.sh                         |  0
 generate-hooklist.sh => tools/generate-hooklist.sh |  0
 generate-perl.sh => tools/generate-perl.sh         |  0
 generate-python.sh => tools/generate-python.sh     |  0
 generate-script.sh => tools/generate-script.sh     |  0
 13 files changed, 35 insertions(+), 35 deletions(-)

diff --git a/Makefile b/Makefile
index 8564b1be36..322f5940e3 100644
--- a/Makefile
+++ b/Makefile
@@ -2689,21 +2689,21 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
-config-list.h: generate-configlist.sh
+config-list.h: tools/generate-configlist.sh
 	@mkdir -p .depend
-	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh . $@ .depend/config-list.h.d
+	$(QUIET_GEN)$(SHELL_PATH) ./tools/generate-configlist.sh . $@ .depend/config-list.h.d
 
 -include .depend/config-list.h.d
 
-command-list.h: generate-cmdlist.sh command-list.txt
+command-list.h: tools/generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.adoc)
-	$(QUIET_GEN)$(SHELL_PATH) ./generate-cmdlist.sh \
+	$(QUIET_GEN)$(SHELL_PATH) ./tools/generate-cmdlist.sh \
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		. $@
 
-hook-list.h: generate-hooklist.sh Documentation/githooks.adoc
-	$(QUIET_GEN)$(SHELL_PATH) ./generate-hooklist.sh . $@
+hook-list.h: tools/generate-hooklist.sh Documentation/githooks.adoc
+	$(QUIET_GEN)$(SHELL_PATH) ./tools/generate-hooklist.sh . $@
 
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):\
 	$(localedir_SQ):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
@@ -2716,8 +2716,8 @@ GIT-SCRIPT-DEFINES: FORCE
 		echo "$$FLAGS" >$@; \
             fi
 
-$(SCRIPT_SH_GEN) $(SCRIPT_LIB) : % : %.sh generate-script.sh GIT-BUILD-OPTIONS GIT-SCRIPT-DEFINES
-	$(QUIET_GEN)./generate-script.sh "$<" "$@+" ./GIT-BUILD-OPTIONS && \
+$(SCRIPT_SH_GEN) $(SCRIPT_LIB) : % : %.sh tools/generate-script.sh GIT-BUILD-OPTIONS GIT-SCRIPT-DEFINES
+	$(QUIET_GEN)./tools/generate-script.sh "$<" "$@+" ./GIT-BUILD-OPTIONS && \
 	mv $@+ $@
 
 git.rc: git.rc.in GIT-VERSION-GEN GIT-VERSION-FILE
@@ -2757,8 +2757,8 @@ endif
 
 PERL_DEFINES += $(gitexecdir) $(perllibdir) $(localedir)
 
-$(SCRIPT_PERL_GEN): % : %.perl generate-perl.sh GIT-PERL-DEFINES GIT-PERL-HEADER GIT-VERSION-FILE
-	$(QUIET_GEN)$(SHELL_PATH) generate-perl.sh ./GIT-BUILD-OPTIONS ./GIT-VERSION-FILE GIT-PERL-HEADER "$<" "$@+" && \
+$(SCRIPT_PERL_GEN): % : %.perl tools/generate-perl.sh GIT-PERL-DEFINES GIT-PERL-HEADER GIT-VERSION-FILE
+	$(QUIET_GEN)$(SHELL_PATH) tools/generate-perl.sh ./GIT-BUILD-OPTIONS ./GIT-VERSION-FILE GIT-PERL-HEADER "$<" "$@+" && \
 	mv $@+ $@
 
 PERL_DEFINES := $(subst $(space),:,$(PERL_DEFINES))
@@ -2786,8 +2786,8 @@ GIT-PERL-HEADER: $(PERL_HEADER_TEMPLATE) GIT-PERL-DEFINES Makefile
 perllibdir:
 	@echo '$(perllibdir_SQ)'
 
-git-instaweb: git-instaweb.sh generate-script.sh GIT-BUILD-OPTIONS GIT-SCRIPT-DEFINES
-	$(QUIET_GEN)./generate-script.sh "$<" "$@+" ./GIT-BUILD-OPTIONS && \
+git-instaweb: git-instaweb.sh tools/generate-script.sh GIT-BUILD-OPTIONS GIT-SCRIPT-DEFINES
+	$(QUIET_GEN)./tools/generate-script.sh "$<" "$@+" ./GIT-BUILD-OPTIONS && \
 	chmod +x $@+ && \
 	mv $@+ $@
 else # NO_PERL
@@ -2804,9 +2804,9 @@ endif # NO_PERL
 $(SCRIPT_PYTHON_GEN): GIT-BUILD-OPTIONS
 
 ifndef NO_PYTHON
-$(SCRIPT_PYTHON_GEN): generate-python.sh
+$(SCRIPT_PYTHON_GEN): tools/generate-python.sh
 $(SCRIPT_PYTHON_GEN): % : %.py
-	$(QUIET_GEN)$(SHELL_PATH) generate-python.sh ./GIT-BUILD-OPTIONS "$<" "$@"
+	$(QUIET_GEN)$(SHELL_PATH) tools/generate-python.sh ./GIT-BUILD-OPTIONS "$<" "$@"
 else # NO_PYTHON
 $(SCRIPT_PYTHON_GEN): % : unimplemented.sh
 	$(QUIET_GEN) \
@@ -3226,9 +3226,9 @@ endif
 NO_PERL_CPAN_FALLBACKS_SQ = $(subst ','\'',$(NO_PERL_CPAN_FALLBACKS))
 endif
 
-perl/build/lib/%.pm: perl/%.pm generate-perl.sh GIT-BUILD-OPTIONS GIT-VERSION-FILE GIT-PERL-DEFINES
+perl/build/lib/%.pm: perl/%.pm tools/generate-perl.sh GIT-BUILD-OPTIONS GIT-VERSION-FILE GIT-PERL-DEFINES
 	$(call mkdir_p_parent_template)
-	$(QUIET_GEN)$(SHELL_PATH) generate-perl.sh ./GIT-BUILD-OPTIONS ./GIT-VERSION-FILE GIT-PERL-HEADER "$<" "$@"
+	$(QUIET_GEN)$(SHELL_PATH) tools/generate-perl.sh ./GIT-BUILD-OPTIONS ./GIT-VERSION-FILE GIT-PERL-HEADER "$<" "$@"
 
 perl/build/man/man3/Git.3pm: perl/Git.pm
 	$(call mkdir_p_parent_template)
@@ -3936,7 +3936,7 @@ check-docs::
 ### Make sure built-ins do not have dups and listed in git.c
 #
 check-builtins::
-	./check-builtins.sh
+	./tools/check-builtins.sh
 
 ### Test suite coverage testing
 #
diff --git a/config.mak.dev b/config.mak.dev
index e86b6e1b34..c8dcf78779 100644
--- a/config.mak.dev
+++ b/config.mak.dev
@@ -1,5 +1,5 @@
 ifndef COMPILER_FEATURES
-COMPILER_FEATURES := $(shell ./detect-compiler $(CC))
+COMPILER_FEATURES := $(shell ./tools/detect-compiler $(CC))
 endif
 
 ifeq ($(filter no-error,$(DEVOPTS)),)
diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt
index c6cfb874ef..81b4306e72 100644
--- a/contrib/buildsystems/CMakeLists.txt
+++ b/contrib/buildsystems/CMakeLists.txt
@@ -636,7 +636,7 @@ set(EXCLUSION_PROGS_CACHE ${EXCLUSION_PROGS} CACHE STRING "Programs not built" F
 if(NOT EXISTS ${CMAKE_BINARY_DIR}/command-list.h OR NOT EXCLUSION_PROGS_CACHE STREQUAL EXCLUSION_PROGS)
 	list(REMOVE_ITEM EXCLUSION_PROGS empty)
 	message("Generating command-list.h")
-	execute_process(COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/generate-cmdlist.sh"
+	execute_process(COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/tools/generate-cmdlist.sh"
 				${EXCLUSION_PROGS}
 				"${CMAKE_SOURCE_DIR}"
 				"${CMAKE_BINARY_DIR}/command-list.h")
@@ -644,14 +644,14 @@ endif()
 
 if(NOT EXISTS ${CMAKE_BINARY_DIR}/config-list.h)
 	message("Generating config-list.h")
-	execute_process(COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/generate-configlist.sh"
+	execute_process(COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/tools/generate-configlist.sh"
 				"${CMAKE_SOURCE_DIR}"
 				"${CMAKE_BINARY_DIR}/config-list.h")
 endif()
 
 if(NOT EXISTS ${CMAKE_BINARY_DIR}/hook-list.h)
 	message("Generating hook-list.h")
-	execute_process(COMMAND "${SH_EXE}" ${CMAKE_SOURCE_DIR}/generate-hooklist.sh
+	execute_process(COMMAND "${SH_EXE}" ${CMAKE_SOURCE_DIR}/tools/generate-hooklist.sh
 				"${CMAKE_SOURCE_DIR}"
 				"${CMAKE_BINARY_DIR}/hook-list.h")
 endif()
@@ -832,11 +832,11 @@ foreach(script ${git_shell_scripts})
 	endif()
 
 	add_custom_command(OUTPUT "${CMAKE_BINARY_DIR}/${shell_gen_path}"
-		COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/generate-script.sh"
+		COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/tools/generate-script.sh"
 			"${CMAKE_SOURCE_DIR}/${script}.sh"
 			"${CMAKE_BINARY_DIR}/${shell_gen_path}"
 			"${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS"
-		DEPENDS "${CMAKE_SOURCE_DIR}/generate-script.sh"
+		DEPENDS "${CMAKE_SOURCE_DIR}/tools/generate-script.sh"
 			"${CMAKE_SOURCE_DIR}/${script}.sh"
 		VERBATIM)
 	list(APPEND shell_gen ${CMAKE_BINARY_DIR}/${shell_gen_path})
@@ -875,13 +875,13 @@ foreach(script ${git_perl_scripts} ${perl_modules})
 	file(MAKE_DIRECTORY "${CMAKE_BINARY_DIR}/${perl_gen_dir}")
 
 	add_custom_command(OUTPUT "${CMAKE_BINARY_DIR}/${perl_gen_path}"
-		COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/generate-perl.sh"
+		COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/tools/generate-perl.sh"
 			"${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS"
 			"${CMAKE_BINARY_DIR}/GIT-VERSION-FILE"
 			"${CMAKE_BINARY_DIR}/GIT-PERL-HEADER"
 			"${CMAKE_SOURCE_DIR}/${script}"
 			"${CMAKE_BINARY_DIR}/${perl_gen_path}"
-		DEPENDS "${CMAKE_SOURCE_DIR}/generate-perl.sh"
+		DEPENDS "${CMAKE_SOURCE_DIR}/tools/generate-perl.sh"
 			"${CMAKE_SOURCE_DIR}/${script}"
 			"${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS"
 			"${CMAKE_BINARY_DIR}/GIT-VERSION-FILE"
@@ -892,11 +892,11 @@ add_custom_target(perl-gen ALL DEPENDS ${perl_gen})
 
 # Python script
 add_custom_command(OUTPUT "${CMAKE_BINARY_DIR}/git-p4"
-	COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/generate-python.sh"
+	COMMAND "${SH_EXE}" "${CMAKE_SOURCE_DIR}/tools/generate-python.sh"
 		"${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS"
 		"${CMAKE_SOURCE_DIR}/git-p4.py"
 		"${CMAKE_BINARY_DIR}/git-p4"
-	DEPENDS "${CMAKE_SOURCE_DIR}/generate-python.sh"
+	DEPENDS "${CMAKE_SOURCE_DIR}/tools/generate-python.sh"
 		"${CMAKE_SOURCE_DIR}/git-p4.py"
 		"${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS"
 	VERBATIM)
diff --git a/contrib/subtree/meson.build b/contrib/subtree/meson.build
index 161435abeb..804c315894 100644
--- a/contrib/subtree/meson.build
+++ b/contrib/subtree/meson.build
@@ -3,7 +3,7 @@ git_subtree = custom_target(
   output: 'git-subtree',
   command: [
     shell,
-    meson.project_source_root() / 'generate-script.sh',
+    meson.project_source_root() / 'tools/generate-script.sh',
     '@INPUT@',
     '@OUTPUT@',
     meson.project_build_root() / 'GIT-BUILD-OPTIONS',
diff --git a/meson.build b/meson.build
index 1d66b5181e..604fe89d2d 100644
--- a/meson.build
+++ b/meson.build
@@ -554,7 +554,7 @@ libgit_sources = [
 libgit_sources += custom_target(
   input: 'command-list.txt',
   output: 'command-list.h',
-  command: [shell, meson.current_source_dir() + '/generate-cmdlist.sh', meson.current_source_dir(), '@OUTPUT@'],
+  command: [shell, meson.current_source_dir() + '/tools/generate-cmdlist.sh', meson.current_source_dir(), '@OUTPUT@'],
   env: script_environment,
 )
 
@@ -723,10 +723,10 @@ endif
 builtin_sources += custom_target(
   output: 'config-list.h',
   depfile: 'config-list.h.d',
-  depend_files: [ 'generate-configlist.sh' ],
+  depend_files: [ 'tools/generate-configlist.sh' ],
   command: [
     shell,
-    meson.current_source_dir() / 'generate-configlist.sh',
+    meson.current_source_dir() / 'tools/generate-configlist.sh',
     meson.current_source_dir(),
     '@OUTPUT@',
     '@DEPFILE@',
@@ -739,7 +739,7 @@ builtin_sources += custom_target(
   output: 'hook-list.h',
   command: [
     shell,
-    meson.current_source_dir() + '/generate-hooklist.sh',
+    meson.current_source_dir() + '/tools/generate-hooklist.sh',
     meson.current_source_dir(),
     '@OUTPUT@',
   ],
@@ -1959,7 +1959,7 @@ foreach script : scripts_sh
     output: fs.stem(script),
     command: [
       shell,
-      meson.project_source_root() / 'generate-script.sh',
+      meson.project_source_root() / 'tools/generate-script.sh',
       '@INPUT@',
       '@OUTPUT@',
       meson.project_build_root() / 'GIT-BUILD-OPTIONS',
@@ -2008,7 +2008,7 @@ if perl_features_enabled
 
   generate_perl_command = [
     shell,
-    meson.project_source_root() / 'generate-perl.sh',
+    meson.project_source_root() / 'tools/generate-perl.sh',
     meson.project_build_root() / 'GIT-BUILD-OPTIONS',
     git_version_file.full_path(),
     perl_header,
@@ -2057,7 +2057,7 @@ if target_python.found()
       output: fs.stem(script),
       command: [
         shell,
-        meson.project_source_root() / 'generate-python.sh',
+        meson.project_source_root() / 'tools/generate-python.sh',
         meson.project_build_root() / 'GIT-BUILD-OPTIONS',
         '@INPUT@',
         '@OUTPUT@',
diff --git a/check-builtins.sh b/tools/check-builtins.sh
similarity index 100%
rename from check-builtins.sh
rename to tools/check-builtins.sh
diff --git a/detect-compiler b/tools/detect-compiler
similarity index 100%
rename from detect-compiler
rename to tools/detect-compiler
diff --git a/generate-cmdlist.sh b/tools/generate-cmdlist.sh
similarity index 100%
rename from generate-cmdlist.sh
rename to tools/generate-cmdlist.sh
diff --git a/generate-configlist.sh b/tools/generate-configlist.sh
similarity index 100%
rename from generate-configlist.sh
rename to tools/generate-configlist.sh
diff --git a/generate-hooklist.sh b/tools/generate-hooklist.sh
similarity index 100%
rename from generate-hooklist.sh
rename to tools/generate-hooklist.sh
diff --git a/generate-perl.sh b/tools/generate-perl.sh
similarity index 100%
rename from generate-perl.sh
rename to tools/generate-perl.sh
diff --git a/generate-python.sh b/tools/generate-python.sh
similarity index 100%
rename from generate-python.sh
rename to tools/generate-python.sh
diff --git a/generate-script.sh b/tools/generate-script.sh
similarity index 100%
rename from generate-script.sh
rename to tools/generate-script.sh

-- 
2.53.0.959.g497ff81fa9.dirty

[PATCH v3 6/8] git-compat-util.h: move warning infra to prepare for PCHs

[Patrick Steinhardt]

The "git-compat-util.h" header is supposed to be the first header
included by every code compilation unit. As such, a subsequent commit
will start to precompile this header to speed up compilation of Git.

This will cause an issue though with the way that we have set up the
"-Wsign-compare" warnings. It is expected that any compilation unit that
fails with that compiler warning sets `DISABLE_SIGN_COMPARE_WARNINGS`
before including "git-compat-util.h". If so, we'll disable the warning
right away via a compiler pragma.

But with precompiled headers we do not know ahead of time whether the
code unit wants to disable those warnings, and thus we'll have to
precompile the header without defining `DISABLE_SIGN_COMPARE_WARNINGS`.
But as the pragma statement is wrapped by our include guards, the second
include of that file will not have the desired effect of disabling the
warnings anymore.

We could fix this issue by declaring a new macro that compilation units
are expected to invoke after having included the file. In retrospect,
that would have been the better way to handle this as it allows for
more flexibility: we could for example toggle the warning for specific
code blocks, only. But changing this now would require a bunch of
changes, and the churn feels excessive for what we gain.

Instead, prepare for the precompiled headers by moving the code outside
of the include guards.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 git-compat-util.h | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/git-compat-util.h b/git-compat-util.h
index bebcf9f698..4b4ea2498f 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -34,10 +34,6 @@ struct strbuf;
 #  define DISABLE_WARNING(warning)
 #endif
 
-#ifdef DISABLE_SIGN_COMPARE_WARNINGS
-DISABLE_WARNING(-Wsign-compare)
-#endif
-
 #undef FLEX_ARRAY
 #define FLEX_ARRAY /* empty - weather balloon to require C99 FAM */
 
@@ -1099,3 +1095,7 @@ extern int not_supposed_to_survive;
 #endif /* CHECK_ASSERTION_SIDE_EFFECTS */
 
 #endif
+
+#ifdef DISABLE_SIGN_COMPARE_WARNINGS
+DISABLE_WARNING(-Wsign-compare)
+#endif

-- 
2.53.0.959.g497ff81fa9.dirty

[PATCH v3 7/8] meson: compile compatibility sources separately

[Patrick Steinhardt]

In the next commit we're about to introduce a precompiled header for
"git-compat-util.h". The consequence of this change is that we'll
implicitly include that header for every compilation unit that uses the
precompiled headers.

This is okay for our "normal" library sources and our builtins. But some
of our compatibility sources do not include the header on purpose, and
doing so would cause compilation errors.

Prepare for this change by splitting out compatibility sources into
their static library. Like this, we can selectively enable precompiled
headers for the library sources.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 meson.build | 79 +++++++++++++++++++++++++++++++++++--------------------------
 1 file changed, 45 insertions(+), 34 deletions(-)

diff --git a/meson.build b/meson.build
index 604fe89d2d..cd00be1c23 100644
--- a/meson.build
+++ b/meson.build
@@ -271,6 +271,13 @@ version_gen_environment.set('GIT_VERSION', get_option('version'))
 
 compiler = meson.get_compiler('c')
 
+compat_sources = [
+  'compat/nonblock.c',
+  'compat/obstack.c',
+  'compat/open.c',
+  'compat/terminal.c',
+]
+
 libgit_sources = [
   'abspath.c',
   'add-interactive.c',
@@ -304,10 +311,6 @@ libgit_sources = [
   'commit.c',
   'common-exit.c',
   'common-init.c',
-  'compat/nonblock.c',
-  'compat/obstack.c',
-  'compat/open.c',
-  'compat/terminal.c',
   'compiler-tricks/not-constant.c',
   'config.c',
   'connect.c',
@@ -1163,7 +1166,7 @@ endif
 
 if not has_poll_h and not has_sys_poll_h
   libgit_c_args += '-DNO_POLL'
-  libgit_sources += 'compat/poll/poll.c'
+  compat_sources += 'compat/poll/poll.c'
   libgit_include_directories += 'compat/poll'
 endif
 
@@ -1179,7 +1182,7 @@ endif
 # implementation to threat things like drive prefixes specially.
 if host_machine.system() == 'windows' or not compiler.has_header('libgen.h')
   libgit_c_args += '-DNO_LIBGEN_H'
-  libgit_sources += 'compat/basename.c'
+  compat_sources += 'compat/basename.c'
 endif
 
 if compiler.has_header('paths.h')
@@ -1209,7 +1212,7 @@ if host_machine.system() != 'windows'
   foreach symbol : ['inet_ntop', 'inet_pton', 'hstrerror']
     if not compiler.has_function(symbol, dependencies: networking_dependencies)
       libgit_c_args += '-DNO_' + symbol.to_upper()
-      libgit_sources += 'compat/' + symbol + '.c'
+      compat_sources += 'compat/' + symbol + '.c'
     endif
   endforeach
 endif
@@ -1251,18 +1254,18 @@ else
 endif
 
 if host_machine.system() == 'darwin'
-  libgit_sources += 'compat/precompose_utf8.c'
+  compat_sources += 'compat/precompose_utf8.c'
   libgit_c_args += '-DPRECOMPOSE_UNICODE'
   libgit_c_args += '-DPROTECT_HFS_DEFAULT'
 endif
 
 # Configure general compatibility wrappers.
 if host_machine.system() == 'cygwin'
-  libgit_sources += [
+  compat_sources += [
     'compat/win32/path-utils.c',
   ]
 elif host_machine.system() == 'windows'
-  libgit_sources += [
+  compat_sources += [
     'compat/winansi.c',
     'compat/win32/dirent.c',
     'compat/win32/flush.c',
@@ -1289,20 +1292,20 @@ elif host_machine.system() == 'windows'
   libgit_include_directories += 'compat/win32'
   if compiler.get_id() == 'msvc'
     libgit_include_directories += 'compat/vcbuild/include'
-    libgit_sources += 'compat/msvc.c'
+    compat_sources += 'compat/msvc.c'
   else
-    libgit_sources += 'compat/mingw.c'
+    compat_sources += 'compat/mingw.c'
   endif
 endif
 
 if host_machine.system() == 'linux'
-  libgit_sources += 'compat/linux/procinfo.c'
+  compat_sources += 'compat/linux/procinfo.c'
 elif host_machine.system() == 'windows'
-  libgit_sources += 'compat/win32/trace2_win32_process_info.c'
+  compat_sources += 'compat/win32/trace2_win32_process_info.c'
 elif host_machine.system() == 'darwin'
-  libgit_sources += 'compat/darwin/procinfo.c'
+  compat_sources += 'compat/darwin/procinfo.c'
 else
-  libgit_sources += 'compat/stub/procinfo.c'
+  compat_sources += 'compat/stub/procinfo.c'
 endif
 
 if host_machine.system() == 'cygwin' or host_machine.system() == 'windows'
@@ -1315,13 +1318,13 @@ endif
 
 # Configure the simple-ipc subsystem required fro the fsmonitor.
 if host_machine.system() == 'windows'
-  libgit_sources += [
+  compat_sources += [
     'compat/simple-ipc/ipc-shared.c',
     'compat/simple-ipc/ipc-win32.c',
   ]
   libgit_c_args += '-DSUPPORTS_SIMPLE_IPC'
 else
-  libgit_sources += [
+  compat_sources += [
     'compat/simple-ipc/ipc-shared.c',
     'compat/simple-ipc/ipc-unix-socket.c',
   ]
@@ -1339,7 +1342,7 @@ if fsmonitor_backend != ''
   libgit_c_args += '-DHAVE_FSMONITOR_DAEMON_BACKEND'
   libgit_c_args += '-DHAVE_FSMONITOR_OS_SETTINGS'
 
-  libgit_sources += [
+  compat_sources += [
     'compat/fsmonitor/fsm-health-' + fsmonitor_backend + '.c',
     'compat/fsmonitor/fsm-ipc-' + fsmonitor_backend + '.c',
     'compat/fsmonitor/fsm-listen-' + fsmonitor_backend + '.c',
@@ -1355,7 +1358,7 @@ if not get_option('b_sanitize').contains('address') and get_option('regex').allo
 
   if compiler.get_define('REG_ENHANCED', prefix: '#include <regex.h>') != ''
     libgit_c_args += '-DUSE_ENHANCED_BASIC_REGULAR_EXPRESSIONS'
-    libgit_sources += 'compat/regcomp_enhanced.c'
+    compat_sources += 'compat/regcomp_enhanced.c'
   endif
 elif not get_option('regex').enabled()
   libgit_c_args += [
@@ -1364,7 +1367,7 @@ elif not get_option('regex').enabled()
     '-DNO_MBSUPPORT',
   ]
   build_options_config.set('NO_REGEX', '1')
-  libgit_sources += 'compat/regex/regex.c'
+  compat_sources += 'compat/regex/regex.c'
   libgit_include_directories += 'compat/regex'
 else
     error('Native regex support requested but not found')
@@ -1428,7 +1431,7 @@ else
 
   if get_option('b_sanitize').contains('address')
     libgit_c_args += '-DNO_MMAP'
-    libgit_sources += 'compat/mmap.c'
+    compat_sources += 'compat/mmap.c'
   else
     checkfuncs += { 'mmap': ['mmap.c'] }
   endif
@@ -1438,7 +1441,7 @@ foreach func, impls : checkfuncs
   if not compiler.has_function(func)
     libgit_c_args += '-DNO_' + func.to_upper()
     foreach impl : impls
-      libgit_sources += 'compat/' + impl
+      compat_sources += 'compat/' + impl
     endforeach
   endif
 endforeach
@@ -1449,13 +1452,13 @@ endif
 
 if not compiler.has_function('strdup')
   libgit_c_args += '-DOVERRIDE_STRDUP'
-  libgit_sources += 'compat/strdup.c'
+  compat_sources += 'compat/strdup.c'
 endif
 
 if not compiler.has_function('qsort')
   libgit_c_args += '-DINTERNAL_QSORT'
 endif
-libgit_sources += 'compat/qsort_s.c'
+compat_sources += 'compat/qsort_s.c'
 
 if compiler.has_function('getdelim')
   libgit_c_args += '-DHAVE_GETDELIM'
@@ -1511,7 +1514,7 @@ if meson.can_run_host_binaries() and compiler.run('''
   }
 ''', name: 'fread reads directories').returncode() == 0
   libgit_c_args += '-DFREAD_READS_DIRECTORIES'
-  libgit_sources += 'compat/fopen.c'
+  compat_sources += 'compat/fopen.c'
 endif
 
 if not meson.is_cross_build() and fs.exists('/dev/tty')
@@ -1745,14 +1748,22 @@ else
 endif
 
 libgit = declare_dependency(
-  link_with: static_library('git',
-    sources: libgit_sources,
-    c_args: libgit_c_args + [
-      '-DGIT_VERSION_H="' + version_def_h.full_path() + '"',
-    ],
-    dependencies: libgit_dependencies,
-    include_directories: libgit_include_directories,
-  ),
+  link_with: [
+    static_library('compat',
+      sources: compat_sources,
+      c_args: libgit_c_args,
+      dependencies: libgit_dependencies,
+      include_directories: libgit_include_directories,
+    ),
+    static_library('git',
+      sources: libgit_sources,
+      c_args: libgit_c_args + [
+        '-DGIT_VERSION_H="' + version_def_h.full_path() + '"',
+      ],
+      dependencies: libgit_dependencies,
+      include_directories: libgit_include_directories,
+    ),
+  ],
   compile_args: libgit_c_args,
   dependencies: libgit_dependencies,
   include_directories: libgit_include_directories,

-- 
2.53.0.959.g497ff81fa9.dirty

[PATCH v3 8/8] meson: precompile "git-compat-util.h"

[Patrick Steinhardt]

Every compilation unit in Git is expected to include "git-compat-util.h"
first, either directly or indirectly via "builtin.h". This header papers
over differences between platforms so that we can expect the typical
POSIX functions to exist. Furthermore, it provides functionality that we
end up using everywhere.

This header is thus quite heavy as a consequence. Preprocessing it as a
standalone unit via `clang -E git-compat-util.h` yields over 23,000
lines of code overall. Naturally, it takes quite some time to compile
all of this.

Luckily, this is exactly the kind of use case that precompiled headers
aim to solve: instead of recompiling it every single time, we compile it
once and then link the result into the executable. If include guards are
set up properly it means that the file won't need to be reprocessed.

Set up such a precompiled header for "git-compat-util.h" and wire it up
via Meson. This causes Meson to implicitly include the precompiled
header in all compilation units. With GCC and Clang for example this is
done via the "-include" statement [1].

This leads to a significant speedup when performing full builds:

  Benchmark 1: ninja (rev = HEAD~)
  Time (mean ± σ):     14.467 s ±  0.126 s    [User: 248.133 s, System: 31.298 s]
  Range (min … max):   14.195 s … 14.633 s    10 runs

  Benchmark 2: ninja (rev = HEAD)
    Time (mean ± σ):     10.307 s ±  0.111 s    [User: 173.290 s, System: 23.998 s]
    Range (min … max):   10.030 s … 10.433 s    10 runs

  Summary
    ninja (rev = HEAD) ran
      1.40 ± 0.02 times faster than ninja (rev = HEAD~)

[1]: https://gcc.gnu.org/onlinedocs/gcc/Precompiled-Headers.html

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 meson.build         | 2 ++
 tools/precompiled.h | 1 +
 2 files changed, 3 insertions(+)

diff --git a/meson.build b/meson.build
index cd00be1c23..2002f4795e 100644
--- a/meson.build
+++ b/meson.build
@@ -1760,6 +1760,7 @@ libgit = declare_dependency(
       c_args: libgit_c_args + [
         '-DGIT_VERSION_H="' + version_def_h.full_path() + '"',
       ],
+      c_pch: 'tools/precompiled.h',
       dependencies: libgit_dependencies,
       include_directories: libgit_include_directories,
     ),
@@ -1820,6 +1821,7 @@ test_dependencies = [ ]
 
 git_builtin = executable('git',
   sources: builtin_sources + 'git.c',
+  c_pch: 'tools/precompiled.h',
   dependencies: [libgit_commonmain],
   install: true,
   install_dir: git_exec_path,
diff --git a/tools/precompiled.h b/tools/precompiled.h
new file mode 100644
index 0000000000..b2bec0d2b4
--- /dev/null
+++ b/tools/precompiled.h
@@ -0,0 +1 @@
+#include "git-compat-util.h"

-- 
2.53.0.959.g497ff81fa9.dirty

Re: [PATCH v3 3/8] contrib: move "coverage-diff.sh" script into "tools/"

[Toon Claes]

Patrick Steinhardt <ps@pks.im> writes:

> The "coverage-diff.sh" script can be used to get information about test
> coverage fro the Git codebase. It is thus rather specific to our build

Tiniest nit s/fro/for

> and test infrastructure and part of the developer-facing tooling. The
> fact that this script is part of "contrib/" is thus rather misleading
> and a historic wart.
>
> Promote the tool into the new "tools/" directory.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  {contrib => tools}/coverage-diff.sh | 0
>  1 file changed, 0 insertions(+), 0 deletions(-)
>
> diff --git a/contrib/coverage-diff.sh b/tools/coverage-diff.sh
> similarity index 100%
> rename from contrib/coverage-diff.sh
> rename to tools/coverage-diff.sh
>
> -- 
> 2.53.0.959.g497ff81fa9.dirty
>
>

-- 
Cheers,
Toon

Re: [PATCH v3 6/8] git-compat-util.h: move warning infra to prepare for PCHs

[Toon Claes]

Patrick Steinhardt <ps@pks.im> writes:

> The "git-compat-util.h" header is supposed to be the first header
> included by every code compilation unit. As such, a subsequent commit
> will start to precompile this header to speed up compilation of Git.
>
> This will cause an issue though with the way that we have set up the
> "-Wsign-compare" warnings. It is expected that any compilation unit that
> fails with that compiler warning sets `DISABLE_SIGN_COMPARE_WARNINGS`
> before including "git-compat-util.h". If so, we'll disable the warning
> right away via a compiler pragma.
>
> But with precompiled headers we do not know ahead of time whether the
> code unit wants to disable those warnings, and thus we'll have to
> precompile the header without defining `DISABLE_SIGN_COMPARE_WARNINGS`.
> But as the pragma statement is wrapped by our include guards, the second
> include of that file will not have the desired effect of disabling the
> warnings anymore.
>
> We could fix this issue by declaring a new macro that compilation units
> are expected to invoke after having included the file. In retrospect,
> that would have been the better way to handle this as it allows for
> more flexibility: we could for example toggle the warning for specific
> code blocks, only. But changing this now would require a bunch of
> changes, and the churn feels excessive for what we gain.
>
> Instead, prepare for the precompiled headers by moving the code outside
> of the include guards.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  git-compat-util.h | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/git-compat-util.h b/git-compat-util.h
> index bebcf9f698..4b4ea2498f 100644
> --- a/git-compat-util.h
> +++ b/git-compat-util.h
> @@ -34,10 +34,6 @@ struct strbuf;
>  #  define DISABLE_WARNING(warning)
>  #endif
>  
> -#ifdef DISABLE_SIGN_COMPARE_WARNINGS
> -DISABLE_WARNING(-Wsign-compare)
> -#endif
> -
>  #undef FLEX_ARRAY
>  #define FLEX_ARRAY /* empty - weather balloon to require C99 FAM */
>  
> @@ -1099,3 +1095,7 @@ extern int not_supposed_to_survive;
>  #endif /* CHECK_ASSERTION_SIDE_EFFECTS */
>  
>  #endif
> +
> +#ifdef DISABLE_SIGN_COMPARE_WARNINGS
> +DISABLE_WARNING(-Wsign-compare)
> +#endif

Okay, so with all patches applied, when a .c file is compiled,
tools/precompiled.h is included as the first one. That one includes
git-compat-util.h and processes everything inside the include guards.
Then it starts processing the contents of that files and that file might
#define DISABLE_SIGN_COMPARE_WARNINGS. Usually git-compat-util.h is then
included again, but thanks to the include guards, most of it is ignored,
except for this last bit.

Okay, makes sense.

-- 
Cheers,
Toon

Re: [PATCH v3 8/8] meson: precompile "git-compat-util.h"

[Toon Claes]

Patrick Steinhardt <ps@pks.im> writes:

> Every compilation unit in Git is expected to include "git-compat-util.h"
> first, either directly or indirectly via "builtin.h". This header papers
> over differences between platforms so that we can expect the typical
> POSIX functions to exist. Furthermore, it provides functionality that we
> end up using everywhere.
>
> This header is thus quite heavy as a consequence. Preprocessing it as a
> standalone unit via `clang -E git-compat-util.h` yields over 23,000
> lines of code overall. Naturally, it takes quite some time to compile
> all of this.
>
> Luckily, this is exactly the kind of use case that precompiled headers
> aim to solve: instead of recompiling it every single time, we compile it
> once and then link the result into the executable. If include guards are
> set up properly it means that the file won't need to be reprocessed.
>
> Set up such a precompiled header for "git-compat-util.h" and wire it up
> via Meson. This causes Meson to implicitly include the precompiled
> header in all compilation units. With GCC and Clang for example this is
> done via the "-include" statement [1].
>
> This leads to a significant speedup when performing full builds:
>
>   Benchmark 1: ninja (rev = HEAD~)
>   Time (mean ± σ):     14.467 s ±  0.126 s    [User: 248.133 s, System: 31.298 s]
>   Range (min … max):   14.195 s … 14.633 s    10 runs
>
>   Benchmark 2: ninja (rev = HEAD)
>     Time (mean ± σ):     10.307 s ±  0.111 s    [User: 173.290 s, System: 23.998 s]
>     Range (min … max):   10.030 s … 10.433 s    10 runs
>
>   Summary
>     ninja (rev = HEAD) ran
>       1.40 ± 0.02 times faster than ninja (rev = HEAD~)
>

Quite cool! The only nit I would have about this, this optimization is
not applied to building with Makefiles. While that isn't an issue, I
wouldn't have hurt if it was mentioned in the commit message.

Anyway, overall I got nothing that's holding back this series from
merging. Looks good!

-- 
Cheers,
Toon