[PATCH v2 0/4] configure: reduce number of files created

[PATCH v2 0/4] configure: reduce number of files created

[Juergen Gross]

Instead of creating lots of files during configure, create those
files (where possible) rather at build time. This reduces the need
to run configure when e.g. changing a man page containing a path
defined by the configure run.

Changes in V2:
- dropped patch 1 as already applied
- no longer rename files to *.src
- don't use pattern rule

Juergen Gross (4):
  build: add make macro for making file from file.in
  docs: replace @xxx@ markers at build time
  config: remove unused paths from config/Paths.mk.in
  tools: replace @xxx@ markers at build time

 .gitignore                     |  1 +
 Config.mk                      | 13 +++++++++++++
 config/Paths.mk.in             |  5 -----
 config/Tools-paths.mk.in       | 10 ++++++++++
 docs/Makefile                  |  8 +++++++-
 docs/configure                 |  7 +------
 docs/configure.ac              |  9 +--------
 tools/configure                | 21 ++-------------------
 tools/configure.ac             | 19 +------------------
 tools/hotplug/FreeBSD/Makefile |  7 ++++++-
 tools/hotplug/Linux/Makefile   | 10 +++++++++-
 tools/hotplug/NetBSD/Makefile  |  7 ++++++-
 tools/hotplug/common/Makefile  |  7 ++++++-
 tools/ocaml/libs/xs/Makefile   |  9 +++++++++
 tools/ocaml/xenstored/Makefile |  9 ++++++++-
 15 files changed, 80 insertions(+), 62 deletions(-)
 create mode 100644 config/Tools-paths.mk.in

-- 
2.51.0

[PATCH v2 1/4] build: add make macro for making file from file.in

[Juergen Gross]

Add a new make macro for creating <file> from <file>.in at build
time. To be used like this:

$(foreach file,$(IN_FILES),$(eval $(call apply-build-vars,$(file))))

This can be used instead of the current approach to perform the similar
step for file.in during ./configure.

This will avoid having to run ./configure just because of modifying a
file depending on a variable set by configure.

Prepare to have multiple files as source for the replacement patterns.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
V2:
- don't use pattern rule, but create explicit dependency in macro,
  don't require to rename source files (Jan Beulich, Andrew Cooper)
---
 Config.mk | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/Config.mk b/Config.mk
index e1556dfbfa..d21d67945a 100644
--- a/Config.mk
+++ b/Config.mk
@@ -159,6 +159,19 @@ define move-if-changed
 	if ! cmp -s $(1) $(2); then mv -f $(1) $(2); else rm -f $(1); fi
 endef
 
+PATH_FILES := Paths
+INC_FILES := $(foreach f, $(PATH_FILES), $(XEN_ROOT)/config/$(f).mk)
+
+include $(INC_FILES)
+
+BUILD_MAKE_VARS := $(foreach f, $(PATH_FILES), $(shell awk '$$2 == ":=" { print $$1; }' $(XEN_ROOT)/config/$(f).mk.in))
+
+# Replace @xxx@ markers in $(1).in with $(xxx) variable contents, write to $(1)
+define apply-build-vars
+ $(1): $(1).in
+	sed $$(foreach v, $$(BUILD_MAKE_VARS), -e 's#@$$(v)@#$$($$(v))#g') <$$< >$$@
+endef
+
 CFLAGS += -fno-strict-aliasing
 
 CFLAGS += -std=gnu99
-- 
2.51.0

[PATCH v2 2/4] docs: replace @xxx@ markers at build time

[Juergen Gross]

Use the apply-build-vars make macro to replace the @xxx@ markers in
*.in files only at build time.

This allows to change the affected document files without having to
run "configure" for making the change effective.

While at it add the generated files to the distclean make target.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
V2:
- don't rename source files
---
 docs/Makefile     | 8 +++++++-
 docs/configure    | 7 +------
 docs/configure.ac | 9 +--------
 3 files changed, 9 insertions(+), 15 deletions(-)

diff --git a/docs/Makefile b/docs/Makefile
index 37776d303c..e5f4a8ca86 100644
--- a/docs/Makefile
+++ b/docs/Makefile
@@ -8,8 +8,11 @@ DATE		:= $(call date,"+%Y-%m-%d")
 DOC_ARCHES      := arm ppc riscv x86_32 x86_64
 MAN_SECTIONS    := 1 5 7 8
 
+IN_FILES := man/xl-disk-configuration.5.pod man/xl-network-configuration.5.pod
+IN_FILES += man/xl.1.pod man/xl.cfg.5.pod man/xl.conf.5.pod
+
 # Documentation sources to build
-MAN-SRC-y := $(sort $(basename $(wildcard man/*.pod man/*.pandoc)))
+MAN-SRC-y := $(sort $(basename $(wildcard man/*.pod man/*.pandoc) $(IN_FILES)))
 
 RST-SRC-y := $(sort $(filter-out %index.rst,$(shell find * -type f -name '*.rst' -print)))
 
@@ -77,11 +80,14 @@ clean: clean-man-pages
 distclean: clean
 	rm -rf $(XEN_ROOT)/config/Docs.mk config.log config.status config.cache \
 		autom4te.cache
+	rm -f $(IN_FILES)
 
 # Top level install targets
 
 .PHONY: man-pages install-man-pages clean-man-pages uninstall-man-pages
 
+$(foreach file,$(IN_FILES),$(eval $(call apply-build-vars,$(file))))
+
 # Metarules for generating manpages.  Run with $(1) substitued for section
 define GENERATE_MANPAGE_RULES
 
diff --git a/docs/configure b/docs/configure
index 98dda3cd0f..8871914dcb 100755
--- a/docs/configure
+++ b/docs/configure
@@ -1794,7 +1794,7 @@ ac_compiler_gnu=$ac_cv_c_compiler_gnu
 
 
 
-ac_config_files="$ac_config_files ../config/Docs.mk man/xl.cfg.5.pod man/xl.1.pod man/xl-disk-configuration.5.pod man/xl-network-configuration.5.pod man/xl.conf.5.pod"
+ac_config_files="$ac_config_files ../config/Docs.mk"
 
 
 
@@ -3063,11 +3063,6 @@ for ac_config_target in $ac_config_targets
 do
   case $ac_config_target in
     "../config/Docs.mk") CONFIG_FILES="$CONFIG_FILES ../config/Docs.mk" ;;
-    "man/xl.cfg.5.pod") CONFIG_FILES="$CONFIG_FILES man/xl.cfg.5.pod" ;;
-    "man/xl.1.pod") CONFIG_FILES="$CONFIG_FILES man/xl.1.pod" ;;
-    "man/xl-disk-configuration.5.pod") CONFIG_FILES="$CONFIG_FILES man/xl-disk-configuration.5.pod" ;;
-    "man/xl-network-configuration.5.pod") CONFIG_FILES="$CONFIG_FILES man/xl-network-configuration.5.pod" ;;
-    "man/xl.conf.5.pod") CONFIG_FILES="$CONFIG_FILES man/xl.conf.5.pod" ;;
 
   *) as_fn_error $? "invalid argument: \`$ac_config_target'" "$LINENO" 5;;
   esac
diff --git a/docs/configure.ac b/docs/configure.ac
index c2e5edd3b3..43dc516056 100644
--- a/docs/configure.ac
+++ b/docs/configure.ac
@@ -5,14 +5,7 @@ AC_PREREQ([2.67])
 AC_INIT([Xen Hypervisor Documentation], m4_esyscmd([../version.sh ../xen/Makefile]),
     [xen-devel@lists.xen.org], [xen], [https://www.xen.org/])
 AC_CONFIG_SRCDIR([misc/xen-command-line.pandoc])
-AC_CONFIG_FILES([
-../config/Docs.mk
-man/xl.cfg.5.pod
-man/xl.1.pod
-man/xl-disk-configuration.5.pod
-man/xl-network-configuration.5.pod
-man/xl.conf.5.pod
-])
+AC_CONFIG_FILES([../config/Docs.mk])
 AC_CONFIG_AUX_DIR([../])
 
 # M4 Macro includes
-- 
2.51.0

[PATCH v2 3/4] config: remove unused paths from config/Paths.mk.in

[Juergen Gross]

Some paths in config/Paths.mk.in are used nowhere, so remove them.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 config/Paths.mk.in | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/config/Paths.mk.in b/config/Paths.mk.in
index bc42748b7a..668545be2f 100644
--- a/config/Paths.mk.in
+++ b/config/Paths.mk.in
@@ -20,10 +20,7 @@ libexecdir               := @libexecdir@
 datarootdir              := @datarootdir@
 mandir                   := @mandir@
 docdir                   := @docdir@
-dvidir                   := @dvidir@
 htmldir                  := @htmldir@
-pdfdir                   := @pdfdir@
-psdir                    := @psdir@
 includedir               := @includedir@
 localstatedir            := @localstatedir@
 sysconfdir               := @sysconfdir@
@@ -34,8 +31,6 @@ LIBEXEC_LIB              := @LIBEXEC_LIB@
 LIBEXEC_INC              := @LIBEXEC_INC@
 
 SHAREDIR                 := @SHAREDIR@
-MAN1DIR                  := $(mandir)/man1
-MAN8DIR                  := $(mandir)/man8
 
 XEN_RUN_DIR              := @XEN_RUN_DIR@
 XEN_LOG_DIR              := @XEN_LOG_DIR@
-- 
2.51.0

[PATCH v2 4/4] tools: replace @xxx@ markers at build time

[Juergen Gross]

Use the apply-build-vars make macro to replace the @xxx@ markers in
most *.in files only at build time.

As some of the markers are local to tools/configure, introduce
config/Tools-paths.mk.in and add the related make variables to it.
Add Tools-paths to the PATH_FILES make variable in order to include
the definitions for replacing them in the *.in files.

Add the generated files to the distclean target.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
V2:
- don't rename source files
---
 .gitignore                     |  1 +
 Config.mk                      |  2 +-
 config/Tools-paths.mk.in       | 10 ++++++++++
 tools/configure                | 21 ++-------------------
 tools/configure.ac             | 19 +------------------
 tools/hotplug/FreeBSD/Makefile |  7 ++++++-
 tools/hotplug/Linux/Makefile   | 10 +++++++++-
 tools/hotplug/NetBSD/Makefile  |  7 ++++++-
 tools/hotplug/common/Makefile  |  7 ++++++-
 tools/ocaml/libs/xs/Makefile   |  9 +++++++++
 tools/ocaml/xenstored/Makefile |  9 ++++++++-
 11 files changed, 59 insertions(+), 43 deletions(-)
 create mode 100644 config/Tools-paths.mk.in

diff --git a/.gitignore b/.gitignore
index d83427aba8..57d54f676f 100644
--- a/.gitignore
+++ b/.gitignore
@@ -47,6 +47,7 @@ config.status
 config.cache
 config/Toplevel.mk
 config/Paths.mk
+config/Tools-paths.mk
 
 dist/*
 extras/
diff --git a/Config.mk b/Config.mk
index d21d67945a..8d1368d25b 100644
--- a/Config.mk
+++ b/Config.mk
@@ -159,7 +159,7 @@ define move-if-changed
 	if ! cmp -s $(1) $(2); then mv -f $(1) $(2); else rm -f $(1); fi
 endef
 
-PATH_FILES := Paths
+PATH_FILES := Paths Tools-paths
 INC_FILES := $(foreach f, $(PATH_FILES), $(XEN_ROOT)/config/$(f).mk)
 
 include $(INC_FILES)
diff --git a/config/Tools-paths.mk.in b/config/Tools-paths.mk.in
new file mode 100644
index 0000000000..ac6298e761
--- /dev/null
+++ b/config/Tools-paths.mk.in
@@ -0,0 +1,10 @@
+-include $(XEN_ROOT)/config/Paths.mk
+
+XENSTORED                := @XENSTORED@
+XENSTORED_KVA            := @XENSTORED_KVA@
+XENSTORED_PORT           := @XENSTORED_PORT@
+XEN_RUN_STORED           := @XEN_RUN_STORED@
+
+LINUX_BACKEND_MODULES    := @LINUX_BACKEND_MODULES@
+
+qemu_xen_path            := @qemu_xen_path@
diff --git a/tools/configure b/tools/configure
index 3111f5688c..479c7c9a3c 100755
--- a/tools/configure
+++ b/tools/configure
@@ -2742,7 +2742,7 @@ ac_compiler_gnu=$ac_cv_c_compiler_gnu
 
 
 
-ac_config_files="$ac_config_files ../config/Tools.mk hotplug/common/hotplugpath.sh hotplug/FreeBSD/rc.d/xencommons hotplug/FreeBSD/rc.d/xendriverdomain hotplug/Linux/init.d/sysconfig.xencommons hotplug/Linux/init.d/sysconfig.xendomains hotplug/Linux/init.d/xen-watchdog hotplug/Linux/init.d/xencommons hotplug/Linux/init.d/xendomains hotplug/Linux/init.d/xendriverdomain hotplug/Linux/launch-xenstore hotplug/Linux/vif-setup hotplug/Linux/xen-hotplug-common.sh hotplug/Linux/xendomains hotplug/NetBSD/rc.d/xencommons hotplug/NetBSD/rc.d/xendriverdomain ocaml/libs/xs/paths.ml ocaml/xenstored/paths.ml ocaml/xenstored/oxenstored.conf"
+ac_config_files="$ac_config_files ../config/Tools.mk ../config/Tools-paths.mk"
 
 ac_config_headers="$ac_config_headers config.h"
 
@@ -11268,24 +11268,7 @@ for ac_config_target in $ac_config_targets
 do
   case $ac_config_target in
     "../config/Tools.mk") CONFIG_FILES="$CONFIG_FILES ../config/Tools.mk" ;;
-    "hotplug/common/hotplugpath.sh") CONFIG_FILES="$CONFIG_FILES hotplug/common/hotplugpath.sh" ;;
-    "hotplug/FreeBSD/rc.d/xencommons") CONFIG_FILES="$CONFIG_FILES hotplug/FreeBSD/rc.d/xencommons" ;;
-    "hotplug/FreeBSD/rc.d/xendriverdomain") CONFIG_FILES="$CONFIG_FILES hotplug/FreeBSD/rc.d/xendriverdomain" ;;
-    "hotplug/Linux/init.d/sysconfig.xencommons") CONFIG_FILES="$CONFIG_FILES hotplug/Linux/init.d/sysconfig.xencommons" ;;
-    "hotplug/Linux/init.d/sysconfig.xendomains") CONFIG_FILES="$CONFIG_FILES hotplug/Linux/init.d/sysconfig.xendomains" ;;
-    "hotplug/Linux/init.d/xen-watchdog") CONFIG_FILES="$CONFIG_FILES hotplug/Linux/init.d/xen-watchdog" ;;
-    "hotplug/Linux/init.d/xencommons") CONFIG_FILES="$CONFIG_FILES hotplug/Linux/init.d/xencommons" ;;
-    "hotplug/Linux/init.d/xendomains") CONFIG_FILES="$CONFIG_FILES hotplug/Linux/init.d/xendomains" ;;
-    "hotplug/Linux/init.d/xendriverdomain") CONFIG_FILES="$CONFIG_FILES hotplug/Linux/init.d/xendriverdomain" ;;
-    "hotplug/Linux/launch-xenstore") CONFIG_FILES="$CONFIG_FILES hotplug/Linux/launch-xenstore" ;;
-    "hotplug/Linux/vif-setup") CONFIG_FILES="$CONFIG_FILES hotplug/Linux/vif-setup" ;;
-    "hotplug/Linux/xen-hotplug-common.sh") CONFIG_FILES="$CONFIG_FILES hotplug/Linux/xen-hotplug-common.sh" ;;
-    "hotplug/Linux/xendomains") CONFIG_FILES="$CONFIG_FILES hotplug/Linux/xendomains" ;;
-    "hotplug/NetBSD/rc.d/xencommons") CONFIG_FILES="$CONFIG_FILES hotplug/NetBSD/rc.d/xencommons" ;;
-    "hotplug/NetBSD/rc.d/xendriverdomain") CONFIG_FILES="$CONFIG_FILES hotplug/NetBSD/rc.d/xendriverdomain" ;;
-    "ocaml/libs/xs/paths.ml") CONFIG_FILES="$CONFIG_FILES ocaml/libs/xs/paths.ml" ;;
-    "ocaml/xenstored/paths.ml") CONFIG_FILES="$CONFIG_FILES ocaml/xenstored/paths.ml" ;;
-    "ocaml/xenstored/oxenstored.conf") CONFIG_FILES="$CONFIG_FILES ocaml/xenstored/oxenstored.conf" ;;
+    "../config/Tools-paths.mk") CONFIG_FILES="$CONFIG_FILES ../config/Tools-paths.mk" ;;
     "config.h") CONFIG_HEADERS="$CONFIG_HEADERS config.h" ;;
     "hotplug/Linux/systemd/proc-xen.mount") CONFIG_FILES="$CONFIG_FILES hotplug/Linux/systemd/proc-xen.mount" ;;
     "hotplug/Linux/systemd/xen-init-dom0.service") CONFIG_FILES="$CONFIG_FILES hotplug/Linux/systemd/xen-init-dom0.service" ;;
diff --git a/tools/configure.ac b/tools/configure.ac
index 285b4ea128..ecd45e782e 100644
--- a/tools/configure.ac
+++ b/tools/configure.ac
@@ -7,24 +7,7 @@ AC_INIT([Xen Hypervisor Tools], m4_esyscmd([../version.sh ../xen/Makefile]),
 AC_CONFIG_SRCDIR([libs/light/libxl.c])
 AC_CONFIG_FILES([
 ../config/Tools.mk
-hotplug/common/hotplugpath.sh
-hotplug/FreeBSD/rc.d/xencommons
-hotplug/FreeBSD/rc.d/xendriverdomain
-hotplug/Linux/init.d/sysconfig.xencommons
-hotplug/Linux/init.d/sysconfig.xendomains
-hotplug/Linux/init.d/xen-watchdog
-hotplug/Linux/init.d/xencommons
-hotplug/Linux/init.d/xendomains
-hotplug/Linux/init.d/xendriverdomain
-hotplug/Linux/launch-xenstore
-hotplug/Linux/vif-setup
-hotplug/Linux/xen-hotplug-common.sh
-hotplug/Linux/xendomains
-hotplug/NetBSD/rc.d/xencommons
-hotplug/NetBSD/rc.d/xendriverdomain
-ocaml/libs/xs/paths.ml
-ocaml/xenstored/paths.ml
-ocaml/xenstored/oxenstored.conf
+../config/Tools-paths.mk
 ])
 AC_CONFIG_HEADERS([config.h])
 AC_CONFIG_AUX_DIR([../])
diff --git a/tools/hotplug/FreeBSD/Makefile b/tools/hotplug/FreeBSD/Makefile
index a6552c9884..8de923781c 100644
--- a/tools/hotplug/FreeBSD/Makefile
+++ b/tools/hotplug/FreeBSD/Makefile
@@ -8,9 +8,13 @@ XEN_SCRIPT_DATA :=
 
 XEN_RCD_PROG := rc.d/xencommons rc.d/xendriverdomain
 
+IN_FILES := rc.d/xencommons rc.d/xendriverdomain
+
 .PHONY: all
 all:
 
+$(foreach file,$(IN_FILES),$(eval $(call apply-build-vars,$(file))))
+
 .PHONY: install
 install: install-scripts install-rcd
 
@@ -35,7 +39,7 @@ uninstall-scripts:
 	rm -f $(addprefix $(DESTDIR)$(XEN_SCRIPT_DIR)/, $(XEN_SCRIPT_DATA))
 
 .PHONY: install-rcd
-install-rcd:
+install-rcd: $(IN_FILES)
 	$(INSTALL_DIR) $(DESTDIR)$(INITD_DIR)
 	set -e; for i in $(XEN_RCD_PROG); \
 	   do \
@@ -51,3 +55,4 @@ clean:
 
 .PHONY: distclean
 distclean: clean
+	rm -rf $(IN_FILES)
diff --git a/tools/hotplug/Linux/Makefile b/tools/hotplug/Linux/Makefile
index 9a7b3a3515..6fcf84a6f4 100644
--- a/tools/hotplug/Linux/Makefile
+++ b/tools/hotplug/Linux/Makefile
@@ -26,9 +26,16 @@ XEN_SCRIPT_DATA := xen-script-common.sh locking.sh logging.sh
 XEN_SCRIPT_DATA += xen-hotplug-common.sh xen-network-common.sh vif-common.sh
 XEN_SCRIPT_DATA += block-common.sh
 
+IN_FILES := launch-xenstore vif-setup xendomains xen-hotplug-common.sh
+IN_FILES += init.d/sysconfig.xendomains init.d/xen-watchdog
+IN_FILES += init.d/xencommons init.d/xendomains
+IN_FILES += init.d/xendriverdomain init.d/sysconfig.xencommons
+
 .PHONY: all
 all: subdirs-all
 
+$(foreach file,$(IN_FILES),$(eval $(call apply-build-vars,$(file))))
+
 .PHONY: install
 install: install-initd install-scripts subdirs-install
 
@@ -37,7 +44,7 @@ uninstall: uninstall-initd uninstall-scripts subdirs-uninstall
 
 # See docs/misc/distro_mapping.txt for INITD_DIR location
 .PHONY: install-initd
-install-initd:
+install-initd: $(IN_FILES)
 	$(INSTALL_DIR) $(DESTDIR)$(INITD_DIR)
 	$(INSTALL_DIR) $(DESTDIR)$(SYSCONFIG_DIR)
 	$(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN)
@@ -81,3 +88,4 @@ clean: subdirs-clean
 
 .PHONY: distclean
 distclean: clean
+	rm -rf $(IN_FILES)
diff --git a/tools/hotplug/NetBSD/Makefile b/tools/hotplug/NetBSD/Makefile
index 1cd3db2ccb..4ac6bd8fa7 100644
--- a/tools/hotplug/NetBSD/Makefile
+++ b/tools/hotplug/NetBSD/Makefile
@@ -12,9 +12,13 @@ XEN_SCRIPTS += qemu-ifup
 XEN_SCRIPT_DATA :=
 XEN_RCD_PROG := rc.d/xencommons rc.d/xendomains rc.d/xen-watchdog rc.d/xendriverdomain
 
+IN_FILES := rc.d/xencommons rc.d/xendriverdomain
+
 .PHONY: all
 all:
 
+$(foreach file,$(IN_FILES),$(eval $(call apply-build-vars,$(file))))
+
 .PHONY: install
 install: install-scripts install-rcd
 
@@ -39,7 +43,7 @@ uninstall-scripts:
 	rm -f $(addprefix $(DESTDIR)$(XEN_SCRIPT_DIR)/, $(XEN_SCRIPT_DATA))
 
 .PHONY: install-rcd
-install-rcd:
+install-rcd: $(IN_FILES)
 	$(INSTALL_DIR) $(DESTDIR)$(INITD_DIR)
 	set -e; for i in $(XEN_RCD_PROG); \
 	   do \
@@ -57,3 +61,4 @@ clean:
 
 .PHONY: distclean
 distclean: clean
+	rm -rf $(IN_FILES)
diff --git a/tools/hotplug/common/Makefile b/tools/hotplug/common/Makefile
index 62afe1019e..0017332293 100644
--- a/tools/hotplug/common/Makefile
+++ b/tools/hotplug/common/Makefile
@@ -7,9 +7,13 @@ include $(XEN_ROOT)/tools/Rules.mk
 XEN_SCRIPTS :=
 XEN_SCRIPT_DATA := hotplugpath.sh
 
+IN_FILES := hotplugpath.sh
+
 .PHONY: all
 all:
 
+$(foreach file,$(IN_FILES),$(eval $(call apply-build-vars,$(file))))
+
 .PHONY: install
 install: install-scripts
 
@@ -17,7 +21,7 @@ install: install-scripts
 uninstall: uninstall-scripts
 
 .PHONY: install-scripts
-install-scripts: all
+install-scripts: all $(IN_FILES)
 	$(INSTALL_DIR) $(DESTDIR)$(XEN_SCRIPT_DIR)
 	set -e; for i in $(XEN_SCRIPTS); \
 	   do \
@@ -38,3 +42,4 @@ clean:
 
 .PHONY: distclean
 distclean: clean
+	rm -rf $(IN_FILES)
diff --git a/tools/ocaml/libs/xs/Makefile b/tools/ocaml/libs/xs/Makefile
index e160e6a711..7d70d0904c 100644
--- a/tools/ocaml/libs/xs/Makefile
+++ b/tools/ocaml/libs/xs/Makefile
@@ -8,6 +8,8 @@ OCAMLOPTFLAGS += -for-pack Xenstore
 .NOTPARALLEL:
 # Ocaml is such a PITA!
 
+IN_FILES := paths.ml
+
 PREINTF = xsraw.cmi xst.cmi
 PREOBJS = queueop xsraw xst
 PRELIBS = $(foreach obj, $(PREOBJS),$(obj).cmo) $(foreach obj,$(PREOJBS),$(obj).cmx)
@@ -17,6 +19,10 @@ LIBS = xenstore.cma xenstore.cmxa
 
 all: $(PREINTF) $(PRELIBS) $(INTF) $(LIBS) $(PROGRAMS)
 
+$(foreach file,$(IN_FILES),$(eval $(call apply-build-vars,$(file))))
+
+$(OBJS): $(IN_FILES)
+
 bins: $(PROGRAMS)
 
 libs: $(LIBS)
@@ -43,4 +49,7 @@ install: $(LIBS) META
 uninstall:
 	$(OCAMLFIND) remove -destdir $(OCAMLDESTDIR) xenstore
 
+.PHONY: distclean
+	rm -rf $(IN_FILES)
+
 include $(OCAML_TOPLEVEL)/Makefile.rules
diff --git a/tools/ocaml/xenstored/Makefile b/tools/ocaml/xenstored/Makefile
index c333394a34..2576991030 100644
--- a/tools/ocaml/xenstored/Makefile
+++ b/tools/ocaml/xenstored/Makefile
@@ -72,6 +72,8 @@ XENSTOREDLIBS = \
 	-ccopt -L -ccopt $(OCAML_TOPLEVEL)/libs/xsd_glue $(OCAML_TOPLEVEL)/libs/xsd_glue/plugin_interface_v1.cmxa \
 	-ccopt -L -ccopt $(XEN_ROOT)/tools/libs/ctrl
 
+IN_FILES := paths.ml oxenstored.conf
+
 PROGRAMS = oxenstored
 
 oxenstored_LIBS = $(XENSTOREDLIBS)
@@ -83,7 +85,9 @@ oxenstored_OBJS = $(oxenstored_MLSORTED:.ml=)
 
 OCAML_PROGRAM = oxenstored
 
-all: $(INTF) $(LIBS) $(PROGRAMS)
+all: $(IN_FILES) $(INTF) $(LIBS) $(PROGRAMS)
+
+$(foreach file,$(IN_FILES),$(eval $(call apply-build-vars,$(file))))
 
 bins: $(PROGRAMS)
 
@@ -99,4 +103,7 @@ uninstall:
 	rm -f $(DESTDIR)$(XEN_CONFIG_DIR)/oxenstored.conf
 	rm -f $(DESTDIR)$(sbindir)/oxenstored
 
+distclean:
+	rm -rf $(IN_FILES)
+
 include $(OCAML_TOPLEVEL)/Makefile.rules
-- 
2.51.0

Re: [PATCH v2 2/4] docs: replace @xxx@ markers at build time

[Andrew Cooper]

On 14/11/2025 11:32 am, Juergen Gross wrote:
> diff --git a/docs/Makefile b/docs/Makefile
> index 37776d303c..e5f4a8ca86 100644
> --- a/docs/Makefile
> +++ b/docs/Makefile
> @@ -8,8 +8,11 @@ DATE		:= $(call date,"+%Y-%m-%d")
>  DOC_ARCHES      := arm ppc riscv x86_32 x86_64
>  MAN_SECTIONS    := 1 5 7 8
>  
> +IN_FILES := man/xl-disk-configuration.5.pod man/xl-network-configuration.5.pod
> +IN_FILES += man/xl.1.pod man/xl.cfg.5.pod man/xl.conf.5.pod

Sorry, I meant to say this on the previous revision.  Can we please list
these one per line, for the future ease of inserting/removing.

Is IN_FILES really correct?  These are the generated (non-.in) files,
rather than the .in files themselves.  GEN_FILES from v1 would seem to
be a better fit.

But, overall I think this is a nicer change.

> +
>  # Documentation sources to build
> -MAN-SRC-y := $(sort $(basename $(wildcard man/*.pod man/*.pandoc)))
> +MAN-SRC-y := $(sort $(basename $(wildcard man/*.pod man/*.pandoc) $(IN_FILES)))

Doesn't the man/*.pod wildcard do this already ?

~Andrew

Re: [PATCH v2 1/4] build: add make macro for making file from file.in

[Andrew Cooper]

On 14/11/2025 11:32 am, Juergen Gross wrote:
> diff --git a/Config.mk b/Config.mk
> index e1556dfbfa..d21d67945a 100644
> --- a/Config.mk
> +++ b/Config.mk
> @@ -159,6 +159,19 @@ define move-if-changed
>  	if ! cmp -s $(1) $(2); then mv -f $(1) $(2); else rm -f $(1); fi
>  endef
>  
> +PATH_FILES := Paths
> +INC_FILES := $(foreach f, $(PATH_FILES), $(XEN_ROOT)/config/$(f).mk)
> +
> +include $(INC_FILES)
> +
> +BUILD_MAKE_VARS := $(foreach f, $(PATH_FILES), $(shell awk '$$2 == ":=" { print $$1; }' $(XEN_ROOT)/config/$(f).mk.in))
> +
> +# Replace @xxx@ markers in $(1).in with $(xxx) variable contents, write to $(1)
> +define apply-build-vars
> + $(1): $(1).in
> +	sed $$(foreach v, $$(BUILD_MAKE_VARS), -e 's#@$$(v)@#$$($$(v))#g') <$$< >$$@
> +endef

Shouldn't this write to a tmp file, and use move-if-changed?  Most of
the time the markers won't have changed, and we'll want to short circuit
dependent rules.

~Andrew

Re: [PATCH v2 3/4] config: remove unused paths from config/Paths.mk.in

[Andrew Cooper]

On 14/11/2025 11:32 am, Juergen Gross wrote:
> Some paths in config/Paths.mk.in are used nowhere, so remove them.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>
> ---
>  config/Paths.mk.in | 5 -----
>  1 file changed, 5 deletions(-)
>
> diff --git a/config/Paths.mk.in b/config/Paths.mk.in
> index bc42748b7a..668545be2f 100644
> --- a/config/Paths.mk.in
> +++ b/config/Paths.mk.in
> @@ -20,10 +20,7 @@ libexecdir               := @libexecdir@
>  datarootdir              := @datarootdir@
>  mandir                   := @mandir@
>  docdir                   := @docdir@
> -dvidir                   := @dvidir@
>  htmldir                  := @htmldir@
> -pdfdir                   := @pdfdir@
> -psdir                    := @psdir@

The point I apparently didn't make very well was this:

We generate both HTML and PDFs, yet use neither of htmldir nor pdfdir. 
Whatever is done to one should be done to the other.

Leave just docdir, and let whomever tidies up the mess in docs/ adjust
the paths as necessary.

~Andrew

Re: [PATCH v2 4/4] tools: replace @xxx@ markers at build time

[Andrew Cooper]

On 14/11/2025 11:32 am, Juergen Gross wrote:
> diff --git a/tools/hotplug/FreeBSD/Makefile b/tools/hotplug/FreeBSD/Makefile
> index a6552c9884..8de923781c 100644
> --- a/tools/hotplug/FreeBSD/Makefile
> +++ b/tools/hotplug/FreeBSD/Makefile
> @@ -51,3 +55,4 @@ clean:
>  
>  .PHONY: distclean
>  distclean: clean
> +	rm -rf $(IN_FILES)

In addition to the points in patch 2, can we use $(RM) (which includes
-f IIRC) when adding these.  None of the generated files are
directories, so the -r wants dropping.

But again, this is far nicer than v1 overall.

> diff --git a/tools/hotplug/Linux/Makefile b/tools/hotplug/Linux/Makefile
> index 9a7b3a3515..6fcf84a6f4 100644
> --- a/tools/hotplug/Linux/Makefile
> +++ b/tools/hotplug/Linux/Makefile
> @@ -37,7 +44,7 @@ uninstall: uninstall-initd uninstall-scripts subdirs-uninstall
>  
>  # See docs/misc/distro_mapping.txt for INITD_DIR location
>  .PHONY: install-initd
> -install-initd:
> +install-initd: $(IN_FILES)
>  	$(INSTALL_DIR) $(DESTDIR)$(INITD_DIR)
>  	$(INSTALL_DIR) $(DESTDIR)$(SYSCONFIG_DIR)
>  	$(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN)

Hmm.  Logically, generating the files should be part of the build step,
not the install step.

I see you already had to adapt this way for Ocaml, where the generated
files are source files, not just config.

~Andrew

Re: [PATCH v2 1/4] build: add make macro for making file from file.in

[Jürgen Groß]

[-- Attachment #1.1.1: Type: text/plain, Size: 1695 bytes --]

On 14.11.25 12:42, Andrew Cooper wrote:
> On 14/11/2025 11:32 am, Juergen Gross wrote:
>> diff --git a/Config.mk b/Config.mk
>> index e1556dfbfa..d21d67945a 100644
>> --- a/Config.mk
>> +++ b/Config.mk
>> @@ -159,6 +159,19 @@ define move-if-changed
>>   	if ! cmp -s $(1) $(2); then mv -f $(1) $(2); else rm -f $(1); fi
>>   endef
>>   
>> +PATH_FILES := Paths
>> +INC_FILES := $(foreach f, $(PATH_FILES), $(XEN_ROOT)/config/$(f).mk)
>> +
>> +include $(INC_FILES)
>> +
>> +BUILD_MAKE_VARS := $(foreach f, $(PATH_FILES), $(shell awk '$$2 == ":=" { print $$1; }' $(XEN_ROOT)/config/$(f).mk.in))
>> +
>> +# Replace @xxx@ markers in $(1).in with $(xxx) variable contents, write to $(1)
>> +define apply-build-vars
>> + $(1): $(1).in
>> +	sed $$(foreach v, $$(BUILD_MAKE_VARS), -e 's#@$$(v)@#$$($$(v))#g') <$$< >$$@
>> +endef
> 
> Shouldn't this write to a tmp file, and use move-if-changed?  Most of
> the time the markers won't have changed, and we'll want to short circuit
> dependent rules.

I can see this being an advantage when e.g. generating header files, as
those being generated again would potentially cause lots of rebuilds.

In this case I can hardly see any case where make wouldn't do the right
thing already. Either the *.in file is newer than the generated file due
to a git update or a manual edit, so make will regenerate the target (and
this is what we want), or the *.in file hasn't changed, so make won't
regenerate the file as it is newer than the *.in file already.

Or did I miss some aspect?

What IS needed is probably a dependency on $(PATH_FILES) in case someone
did a new ./configure call without a make distclean.


Juergen

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3743 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [PATCH v2 2/4] docs: replace @xxx@ markers at build time

[Jürgen Groß]

[-- Attachment #1.1.1: Type: text/plain, Size: 1514 bytes --]

On 14.11.25 12:40, Andrew Cooper wrote:
> On 14/11/2025 11:32 am, Juergen Gross wrote:
>> diff --git a/docs/Makefile b/docs/Makefile
>> index 37776d303c..e5f4a8ca86 100644
>> --- a/docs/Makefile
>> +++ b/docs/Makefile
>> @@ -8,8 +8,11 @@ DATE		:= $(call date,"+%Y-%m-%d")
>>   DOC_ARCHES      := arm ppc riscv x86_32 x86_64
>>   MAN_SECTIONS    := 1 5 7 8
>>   
>> +IN_FILES := man/xl-disk-configuration.5.pod man/xl-network-configuration.5.pod
>> +IN_FILES += man/xl.1.pod man/xl.cfg.5.pod man/xl.conf.5.pod
> 
> Sorry, I meant to say this on the previous revision.  Can we please list
> these one per line, for the future ease of inserting/removing.

Okay.

> Is IN_FILES really correct?  These are the generated (non-.in) files,
> rather than the .in files themselves.  GEN_FILES from v1 would seem to
> be a better fit.

I wanted to make clear this is related to *.in files. And IMHO GEN_FILES
was too generic on a second thought.

GENERATED_FROM_IN_SUFFIXED_FILES seems a little bit clumsy. ;-)
Seriously, if you have any better name, I'd be happy to use it.

> But, overall I think this is a nicer change.
> 
>> +
>>   # Documentation sources to build
>> -MAN-SRC-y := $(sort $(basename $(wildcard man/*.pod man/*.pandoc)))
>> +MAN-SRC-y := $(sort $(basename $(wildcard man/*.pod man/*.pandoc) $(IN_FILES)))
> 
> Doesn't the man/*.pod wildcard do this already ?

Not if only the man/*.pod.in file is there, so the man/*.pod file needs to be
made first.


Juergen

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3743 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [PATCH v2 3/4] config: remove unused paths from config/Paths.mk.in

[Jürgen Groß]

[-- Attachment #1.1.1: Type: text/plain, Size: 1163 bytes --]

On 14.11.25 12:47, Andrew Cooper wrote:
> On 14/11/2025 11:32 am, Juergen Gross wrote:
>> Some paths in config/Paths.mk.in are used nowhere, so remove them.
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> ---
>>   config/Paths.mk.in | 5 -----
>>   1 file changed, 5 deletions(-)
>>
>> diff --git a/config/Paths.mk.in b/config/Paths.mk.in
>> index bc42748b7a..668545be2f 100644
>> --- a/config/Paths.mk.in
>> +++ b/config/Paths.mk.in
>> @@ -20,10 +20,7 @@ libexecdir               := @libexecdir@
>>   datarootdir              := @datarootdir@
>>   mandir                   := @mandir@
>>   docdir                   := @docdir@
>> -dvidir                   := @dvidir@
>>   htmldir                  := @htmldir@
>> -pdfdir                   := @pdfdir@
>> -psdir                    := @psdir@
> 
> The point I apparently didn't make very well was this:
> 
> We generate both HTML and PDFs, yet use neither of htmldir nor pdfdir.
> Whatever is done to one should be done to the other.
> 
> Leave just docdir, and let whomever tidies up the mess in docs/ adjust
> the paths as necessary.

Ah, okay.

No problem.


Juergen

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3743 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [PATCH v2 4/4] tools: replace @xxx@ markers at build time

[Jürgen Groß]

[-- Attachment #1.1.1: Type: text/plain, Size: 1372 bytes --]

On 14.11.25 12:54, Andrew Cooper wrote:
> On 14/11/2025 11:32 am, Juergen Gross wrote:
>> diff --git a/tools/hotplug/FreeBSD/Makefile b/tools/hotplug/FreeBSD/Makefile
>> index a6552c9884..8de923781c 100644
>> --- a/tools/hotplug/FreeBSD/Makefile
>> +++ b/tools/hotplug/FreeBSD/Makefile
>> @@ -51,3 +55,4 @@ clean:
>>   
>>   .PHONY: distclean
>>   distclean: clean
>> +	rm -rf $(IN_FILES)
> 
> In addition to the points in patch 2, can we use $(RM) (which includes
> -f IIRC) when adding these.  None of the generated files are
> directories, so the -r wants dropping.

Okay.

> 
> But again, this is far nicer than v1 overall.
> 
>> diff --git a/tools/hotplug/Linux/Makefile b/tools/hotplug/Linux/Makefile
>> index 9a7b3a3515..6fcf84a6f4 100644
>> --- a/tools/hotplug/Linux/Makefile
>> +++ b/tools/hotplug/Linux/Makefile
>> @@ -37,7 +44,7 @@ uninstall: uninstall-initd uninstall-scripts subdirs-uninstall
>>   
>>   # See docs/misc/distro_mapping.txt for INITD_DIR location
>>   .PHONY: install-initd
>> -install-initd:
>> +install-initd: $(IN_FILES)
>>   	$(INSTALL_DIR) $(DESTDIR)$(INITD_DIR)
>>   	$(INSTALL_DIR) $(DESTDIR)$(SYSCONFIG_DIR)
>>   	$(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN)
> 
> Hmm.  Logically, generating the files should be part of the build step,
> not the install step.

Agreed. Will change this.


Juergen

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3743 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [PATCH v2 1/4] build: add make macro for making file from file.in

[Jan Beulich]

On 14.11.2025 13:54, Jürgen Groß wrote:
> On 14.11.25 12:42, Andrew Cooper wrote:
>> On 14/11/2025 11:32 am, Juergen Gross wrote:
>>> diff --git a/Config.mk b/Config.mk
>>> index e1556dfbfa..d21d67945a 100644
>>> --- a/Config.mk
>>> +++ b/Config.mk
>>> @@ -159,6 +159,19 @@ define move-if-changed
>>>   	if ! cmp -s $(1) $(2); then mv -f $(1) $(2); else rm -f $(1); fi
>>>   endef
>>>   
>>> +PATH_FILES := Paths
>>> +INC_FILES := $(foreach f, $(PATH_FILES), $(XEN_ROOT)/config/$(f).mk)
>>> +
>>> +include $(INC_FILES)
>>> +
>>> +BUILD_MAKE_VARS := $(foreach f, $(PATH_FILES), $(shell awk '$$2 == ":=" { print $$1; }' $(XEN_ROOT)/config/$(f).mk.in))
>>> +
>>> +# Replace @xxx@ markers in $(1).in with $(xxx) variable contents, write to $(1)
>>> +define apply-build-vars
>>> + $(1): $(1).in
>>> +	sed $$(foreach v, $$(BUILD_MAKE_VARS), -e 's#@$$(v)@#$$($$(v))#g') <$$< >$$@
>>> +endef
>>
>> Shouldn't this write to a tmp file, and use move-if-changed?  Most of
>> the time the markers won't have changed, and we'll want to short circuit
>> dependent rules.
> 
> I can see this being an advantage when e.g. generating header files, as
> those being generated again would potentially cause lots of rebuilds.
> 
> In this case I can hardly see any case where make wouldn't do the right
> thing already. Either the *.in file is newer than the generated file due
> to a git update or a manual edit, so make will regenerate the target (and
> this is what we want), or the *.in file hasn't changed, so make won't
> regenerate the file as it is newer than the *.in file already.
> 
> Or did I miss some aspect?

Aren't some of the generated files Makefile fragments? Them being re-generated
means make re-invoking itself, which could be avoided if the contents don't
really change. (This isn't just a performance concern; this re-invocation has
been the source of, well, surprising behavior in certain cases.)

Jan

Re: [PATCH v2 1/4] build: add make macro for making file from file.in

[Jan Beulich]

On 14.11.2025 12:32, Juergen Gross wrote:
> Add a new make macro for creating <file> from <file>.in at build
> time. To be used like this:
> 
> $(foreach file,$(IN_FILES),$(eval $(call apply-build-vars,$(file))))
> 
> This can be used instead of the current approach to perform the similar
> step for file.in during ./configure.
> 
> This will avoid having to run ./configure just because of modifying a
> file depending on a variable set by configure.
> 
> Prepare to have multiple files as source for the replacement patterns.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>
> ---
> V2:
> - don't use pattern rule, but create explicit dependency in macro,
>   don't require to rename source files (Jan Beulich, Andrew Cooper)
> ---
>  Config.mk | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
> 
> diff --git a/Config.mk b/Config.mk
> index e1556dfbfa..d21d67945a 100644
> --- a/Config.mk
> +++ b/Config.mk
> @@ -159,6 +159,19 @@ define move-if-changed
>  	if ! cmp -s $(1) $(2); then mv -f $(1) $(2); else rm -f $(1); fi
>  endef
>  
> +PATH_FILES := Paths
> +INC_FILES := $(foreach f, $(PATH_FILES), $(XEN_ROOT)/config/$(f).mk)
> +
> +include $(INC_FILES)

Is any of the above part of introducing the macro? "Paths" is already a
specific case of holding patterns that want replacing. In turn ...

> +BUILD_MAKE_VARS := $(foreach f, $(PATH_FILES), $(shell awk '$$2 == ":=" { print $$1; }' $(XEN_ROOT)/config/$(f).mk.in))

... it's not quite clear to me how it can be $(PATH_FILES) here.

> +# Replace @xxx@ markers in $(1).in with $(xxx) variable contents, write to $(1)
> +define apply-build-vars
> + $(1): $(1).in

This being indented by a space looks a little unusual.

Jan

> +	sed $$(foreach v, $$(BUILD_MAKE_VARS), -e 's#@$$(v)@#$$($$(v))#g') <$$< >$$@
> +endef
> +
>  CFLAGS += -fno-strict-aliasing
>  
>  CFLAGS += -std=gnu99

Re: [PATCH v2 1/4] build: add make macro for making file from file.in

[Andrew Cooper]

On 17/11/2025 12:24 pm, Jan Beulich wrote:
> On 14.11.2025 13:54, Jürgen Groß wrote:
>> On 14.11.25 12:42, Andrew Cooper wrote:
>>> On 14/11/2025 11:32 am, Juergen Gross wrote:
>>>> diff --git a/Config.mk b/Config.mk
>>>> index e1556dfbfa..d21d67945a 100644
>>>> --- a/Config.mk
>>>> +++ b/Config.mk
>>>> @@ -159,6 +159,19 @@ define move-if-changed
>>>>   	if ! cmp -s $(1) $(2); then mv -f $(1) $(2); else rm -f $(1); fi
>>>>   endef
>>>>   
>>>> +PATH_FILES := Paths
>>>> +INC_FILES := $(foreach f, $(PATH_FILES), $(XEN_ROOT)/config/$(f).mk)
>>>> +
>>>> +include $(INC_FILES)
>>>> +
>>>> +BUILD_MAKE_VARS := $(foreach f, $(PATH_FILES), $(shell awk '$$2 == ":=" { print $$1; }' $(XEN_ROOT)/config/$(f).mk.in))
>>>> +
>>>> +# Replace @xxx@ markers in $(1).in with $(xxx) variable contents, write to $(1)
>>>> +define apply-build-vars
>>>> + $(1): $(1).in
>>>> +	sed $$(foreach v, $$(BUILD_MAKE_VARS), -e 's#@$$(v)@#$$($$(v))#g') <$$< >$$@
>>>> +endef
>>> Shouldn't this write to a tmp file, and use move-if-changed?  Most of
>>> the time the markers won't have changed, and we'll want to short circuit
>>> dependent rules.
>> I can see this being an advantage when e.g. generating header files, as
>> those being generated again would potentially cause lots of rebuilds.
>>
>> In this case I can hardly see any case where make wouldn't do the right
>> thing already. Either the *.in file is newer than the generated file due
>> to a git update or a manual edit, so make will regenerate the target (and
>> this is what we want), or the *.in file hasn't changed, so make won't
>> regenerate the file as it is newer than the *.in file already.
>>
>> Or did I miss some aspect?
> Aren't some of the generated files Makefile fragments? Them being re-generated
> means make re-invoking itself, which could be avoided if the contents don't
> really change. (This isn't just a performance concern; this re-invocation has
> been the source of, well, surprising behavior in certain cases.)

Having thought about this some more, it needs to be FORCE and
move-if-changed, or to express a dependency on Paths.mk

Otherwise, if you ./configure --libdir=something/else and then do an
incremental build, the generated files will be wrong.

~Andrew

Re: [PATCH v2 2/4] docs: replace @xxx@ markers at build time

[Jan Beulich]

On 14.11.2025 14:00, Jürgen Groß wrote:
> On 14.11.25 12:40, Andrew Cooper wrote:
>> On 14/11/2025 11:32 am, Juergen Gross wrote:
>>> diff --git a/docs/Makefile b/docs/Makefile
>>> index 37776d303c..e5f4a8ca86 100644
>>> --- a/docs/Makefile
>>> +++ b/docs/Makefile
>>> @@ -8,8 +8,11 @@ DATE		:= $(call date,"+%Y-%m-%d")
>>>   DOC_ARCHES      := arm ppc riscv x86_32 x86_64
>>>   MAN_SECTIONS    := 1 5 7 8
>>>   
>>> +IN_FILES := man/xl-disk-configuration.5.pod man/xl-network-configuration.5.pod
>>> +IN_FILES += man/xl.1.pod man/xl.cfg.5.pod man/xl.conf.5.pod
>>
>> Sorry, I meant to say this on the previous revision.  Can we please list
>> these one per line, for the future ease of inserting/removing.
> 
> Okay.
> 
>> Is IN_FILES really correct?  These are the generated (non-.in) files,
>> rather than the .in files themselves.  GEN_FILES from v1 would seem to
>> be a better fit.
> 
> I wanted to make clear this is related to *.in files. And IMHO GEN_FILES
> was too generic on a second thought.
> 
> GENERATED_FROM_IN_SUFFIXED_FILES seems a little bit clumsy. ;-)
> Seriously, if you have any better name, I'd be happy to use it.

GEN_POD_FILES, seeing they're all *.pod?

Jan

Re: [PATCH v2 1/4] build: add make macro for making file from file.in

[Jürgen Groß]

[-- Attachment #1.1.1: Type: text/plain, Size: 2855 bytes --]

On 17.11.25 13:24, Jan Beulich wrote:
> On 14.11.2025 13:54, Jürgen Groß wrote:
>> On 14.11.25 12:42, Andrew Cooper wrote:
>>> On 14/11/2025 11:32 am, Juergen Gross wrote:
>>>> diff --git a/Config.mk b/Config.mk
>>>> index e1556dfbfa..d21d67945a 100644
>>>> --- a/Config.mk
>>>> +++ b/Config.mk
>>>> @@ -159,6 +159,19 @@ define move-if-changed
>>>>    	if ! cmp -s $(1) $(2); then mv -f $(1) $(2); else rm -f $(1); fi
>>>>    endef
>>>>    
>>>> +PATH_FILES := Paths
>>>> +INC_FILES := $(foreach f, $(PATH_FILES), $(XEN_ROOT)/config/$(f).mk)
>>>> +
>>>> +include $(INC_FILES)
>>>> +
>>>> +BUILD_MAKE_VARS := $(foreach f, $(PATH_FILES), $(shell awk '$$2 == ":=" { print $$1; }' $(XEN_ROOT)/config/$(f).mk.in))
>>>> +
>>>> +# Replace @xxx@ markers in $(1).in with $(xxx) variable contents, write to $(1)
>>>> +define apply-build-vars
>>>> + $(1): $(1).in
>>>> +	sed $$(foreach v, $$(BUILD_MAKE_VARS), -e 's#@$$(v)@#$$($$(v))#g') <$$< >$$@
>>>> +endef
>>>
>>> Shouldn't this write to a tmp file, and use move-if-changed?  Most of
>>> the time the markers won't have changed, and we'll want to short circuit
>>> dependent rules.
>>
>> I can see this being an advantage when e.g. generating header files, as
>> those being generated again would potentially cause lots of rebuilds.
>>
>> In this case I can hardly see any case where make wouldn't do the right
>> thing already. Either the *.in file is newer than the generated file due
>> to a git update or a manual edit, so make will regenerate the target (and
>> this is what we want), or the *.in file hasn't changed, so make won't
>> regenerate the file as it is newer than the *.in file already.
>>
>> Or did I miss some aspect?
> 
> Aren't some of the generated files Makefile fragments? Them being re-generated

No.

Man-pages, shell scripts and some Ocaml files (one config file and one .ml file,
which is similar to an include file I believe).

> means make re-invoking itself, which could be avoided if the contents don't
> really change. (This isn't just a performance concern; this re-invocation has
> been the source of, well, surprising behavior in certain cases.)

I still don't see a case where make would consider rebuilding the file from
its .in file without the .in file having changed, thus resulting in the built
file to change, too. Well, with one probably very rare exception: in case a
different @marker@ is used in the .in file, but without changing the resulting
file due to old and new marker resulting in the same output.

In case we really care about such cases, we should think about using
move-if-changed everywhere, as e.g. building a program with $HOSTCC could
result in an unchanged binary even with source files having changed, and the
resulting program could be used to generate other files ...


Juergen


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3743 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [PATCH v2 1/4] build: add make macro for making file from file.in

[Jürgen Groß]

[-- Attachment #1.1.1: Type: text/plain, Size: 2183 bytes --]

On 17.11.25 13:29, Jan Beulich wrote:
> On 14.11.2025 12:32, Juergen Gross wrote:
>> Add a new make macro for creating <file> from <file>.in at build
>> time. To be used like this:
>>
>> $(foreach file,$(IN_FILES),$(eval $(call apply-build-vars,$(file))))
>>
>> This can be used instead of the current approach to perform the similar
>> step for file.in during ./configure.
>>
>> This will avoid having to run ./configure just because of modifying a
>> file depending on a variable set by configure.
>>
>> Prepare to have multiple files as source for the replacement patterns.
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> ---
>> V2:
>> - don't use pattern rule, but create explicit dependency in macro,
>>    don't require to rename source files (Jan Beulich, Andrew Cooper)
>> ---
>>   Config.mk | 13 +++++++++++++
>>   1 file changed, 13 insertions(+)
>>
>> diff --git a/Config.mk b/Config.mk
>> index e1556dfbfa..d21d67945a 100644
>> --- a/Config.mk
>> +++ b/Config.mk
>> @@ -159,6 +159,19 @@ define move-if-changed
>>   	if ! cmp -s $(1) $(2); then mv -f $(1) $(2); else rm -f $(1); fi
>>   endef
>>   
>> +PATH_FILES := Paths
>> +INC_FILES := $(foreach f, $(PATH_FILES), $(XEN_ROOT)/config/$(f).mk)
>> +
>> +include $(INC_FILES)
> 
> Is any of the above part of introducing the macro? "Paths" is already a
> specific case of holding patterns that want replacing. In turn ...
> 
>> +BUILD_MAKE_VARS := $(foreach f, $(PATH_FILES), $(shell awk '$$2 == ":=" { print $$1; }' $(XEN_ROOT)/config/$(f).mk.in))
> 
> ... it's not quite clear to me how it can be $(PATH_FILES) here.

See patch 4.

PATH_FILES is specifying the .mk files containing the marker definitions.
I need the ability to have multiple such files in order to be able to let
tools/configure build its own definitions.

> 
>> +# Replace @xxx@ markers in $(1).in with $(xxx) variable contents, write to $(1)
>> +define apply-build-vars
>> + $(1): $(1).in
> 
> This being indented by a space looks a little unusual.

There are instances of that in tools/Rules.mk and stunbdom/Makefile.

OTOH it might have been me introducing those. :-)


Juergen

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3743 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [PATCH v2 1/4] build: add make macro for making file from file.in

[Jürgen Groß]

[-- Attachment #1.1.1: Type: text/plain, Size: 2338 bytes --]

On 17.11.25 13:30, Andrew Cooper wrote:
> On 17/11/2025 12:24 pm, Jan Beulich wrote:
>> On 14.11.2025 13:54, Jürgen Groß wrote:
>>> On 14.11.25 12:42, Andrew Cooper wrote:
>>>> On 14/11/2025 11:32 am, Juergen Gross wrote:
>>>>> diff --git a/Config.mk b/Config.mk
>>>>> index e1556dfbfa..d21d67945a 100644
>>>>> --- a/Config.mk
>>>>> +++ b/Config.mk
>>>>> @@ -159,6 +159,19 @@ define move-if-changed
>>>>>    	if ! cmp -s $(1) $(2); then mv -f $(1) $(2); else rm -f $(1); fi
>>>>>    endef
>>>>>    
>>>>> +PATH_FILES := Paths
>>>>> +INC_FILES := $(foreach f, $(PATH_FILES), $(XEN_ROOT)/config/$(f).mk)
>>>>> +
>>>>> +include $(INC_FILES)
>>>>> +
>>>>> +BUILD_MAKE_VARS := $(foreach f, $(PATH_FILES), $(shell awk '$$2 == ":=" { print $$1; }' $(XEN_ROOT)/config/$(f).mk.in))
>>>>> +
>>>>> +# Replace @xxx@ markers in $(1).in with $(xxx) variable contents, write to $(1)
>>>>> +define apply-build-vars
>>>>> + $(1): $(1).in
>>>>> +	sed $$(foreach v, $$(BUILD_MAKE_VARS), -e 's#@$$(v)@#$$($$(v))#g') <$$< >$$@
>>>>> +endef
>>>> Shouldn't this write to a tmp file, and use move-if-changed?  Most of
>>>> the time the markers won't have changed, and we'll want to short circuit
>>>> dependent rules.
>>> I can see this being an advantage when e.g. generating header files, as
>>> those being generated again would potentially cause lots of rebuilds.
>>>
>>> In this case I can hardly see any case where make wouldn't do the right
>>> thing already. Either the *.in file is newer than the generated file due
>>> to a git update or a manual edit, so make will regenerate the target (and
>>> this is what we want), or the *.in file hasn't changed, so make won't
>>> regenerate the file as it is newer than the *.in file already.
>>>
>>> Or did I miss some aspect?
>> Aren't some of the generated files Makefile fragments? Them being re-generated
>> means make re-invoking itself, which could be avoided if the contents don't
>> really change. (This isn't just a performance concern; this re-invocation has
>> been the source of, well, surprising behavior in certain cases.)
> 
> Having thought about this some more, it needs to be FORCE and
> move-if-changed, or to express a dependency on Paths.mk

I said that already in the paragraph just after the part of my response you
cited.


Juergen

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3743 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [PATCH v2 1/4] build: add make macro for making file from file.in

[Jan Beulich]

On 17.11.2025 13:37, Jürgen Groß wrote:
> On 17.11.25 13:24, Jan Beulich wrote:
>> On 14.11.2025 13:54, Jürgen Groß wrote:
>>> On 14.11.25 12:42, Andrew Cooper wrote:
>>>> On 14/11/2025 11:32 am, Juergen Gross wrote:
>>>>> diff --git a/Config.mk b/Config.mk
>>>>> index e1556dfbfa..d21d67945a 100644
>>>>> --- a/Config.mk
>>>>> +++ b/Config.mk
>>>>> @@ -159,6 +159,19 @@ define move-if-changed
>>>>>    	if ! cmp -s $(1) $(2); then mv -f $(1) $(2); else rm -f $(1); fi
>>>>>    endef
>>>>>    
>>>>> +PATH_FILES := Paths
>>>>> +INC_FILES := $(foreach f, $(PATH_FILES), $(XEN_ROOT)/config/$(f).mk)
>>>>> +
>>>>> +include $(INC_FILES)
>>>>> +
>>>>> +BUILD_MAKE_VARS := $(foreach f, $(PATH_FILES), $(shell awk '$$2 == ":=" { print $$1; }' $(XEN_ROOT)/config/$(f).mk.in))
>>>>> +
>>>>> +# Replace @xxx@ markers in $(1).in with $(xxx) variable contents, write to $(1)
>>>>> +define apply-build-vars
>>>>> + $(1): $(1).in
>>>>> +	sed $$(foreach v, $$(BUILD_MAKE_VARS), -e 's#@$$(v)@#$$($$(v))#g') <$$< >$$@
>>>>> +endef
>>>>
>>>> Shouldn't this write to a tmp file, and use move-if-changed?  Most of
>>>> the time the markers won't have changed, and we'll want to short circuit
>>>> dependent rules.
>>>
>>> I can see this being an advantage when e.g. generating header files, as
>>> those being generated again would potentially cause lots of rebuilds.
>>>
>>> In this case I can hardly see any case where make wouldn't do the right
>>> thing already. Either the *.in file is newer than the generated file due
>>> to a git update or a manual edit, so make will regenerate the target (and
>>> this is what we want), or the *.in file hasn't changed, so make won't
>>> regenerate the file as it is newer than the *.in file already.
>>>
>>> Or did I miss some aspect?
>>
>> Aren't some of the generated files Makefile fragments? Them being re-generated
> 
> No.
> 
> Man-pages, shell scripts and some Ocaml files (one config file and one .ml file,
> which is similar to an include file I believe).
> 
>> means make re-invoking itself, which could be avoided if the contents don't
>> really change. (This isn't just a performance concern; this re-invocation has
>> been the source of, well, surprising behavior in certain cases.)
> 
> I still don't see a case where make would consider rebuilding the file from
> its .in file without the .in file having changed, thus resulting in the built
> file to change, too.

As Andrew indicated, Paths.mk might have changed, so at the very least an
explicit dependency would need adding. But as alluded to elsewhere, I'm not
quite convinced Paths.mk should be hard-coded as the sole source of patterns
in Config.mk. At the point further such file come into play, dealing with the
dependencies might get interesting / clumsy.

> Well, with one probably very rare exception: in case a
> different @marker@ is used in the .in file, but without changing the resulting
> file due to old and new marker resulting in the same output.
> 
> In case we really care about such cases, we should think about using
> move-if-changed everywhere, as e.g. building a program with $HOSTCC could
> result in an unchanged binary even with source files having changed, and the
> resulting program could be used to generate other files ...

For some of the cases this might actually be worthwhile. It all depends on
how much of a knock-on effect the re-building of a particular file has.

Jan

Re: [PATCH v2 1/4] build: add make macro for making file from file.in

[Jan Beulich]

On 17.11.2025 13:48, Jürgen Groß wrote:
> On 17.11.25 13:29, Jan Beulich wrote:
>> On 14.11.2025 12:32, Juergen Gross wrote:
>>> --- a/Config.mk
>>> +++ b/Config.mk
>>> @@ -159,6 +159,19 @@ define move-if-changed
>>>   	if ! cmp -s $(1) $(2); then mv -f $(1) $(2); else rm -f $(1); fi
>>>   endef
>>>   
>>> +PATH_FILES := Paths
>>> +INC_FILES := $(foreach f, $(PATH_FILES), $(XEN_ROOT)/config/$(f).mk)
>>> +
>>> +include $(INC_FILES)
>>
>> Is any of the above part of introducing the macro? "Paths" is already a
>> specific case of holding patterns that want replacing. In turn ...
>>
>>> +BUILD_MAKE_VARS := $(foreach f, $(PATH_FILES), $(shell awk '$$2 == ":=" { print $$1; }' $(XEN_ROOT)/config/$(f).mk.in))
>>
>> ... it's not quite clear to me how it can be $(PATH_FILES) here.
> 
> See patch 4.
> 
> PATH_FILES is specifying the .mk files containing the marker definitions.
> I need the ability to have multiple such files in order to be able to let
> tools/configure build its own definitions.

That's a good example - why would that affect the stubdom/ part of the tree?
Imo what pattern file(s) to use wants leaving to the invokee of the macro,
not pinning down globally for everyone.

Jan

Re: [PATCH v2 2/4] docs: replace @xxx@ markers at build time

[Jürgen Groß]

[-- Attachment #1.1.1: Type: text/plain, Size: 1542 bytes --]

On 17.11.25 13:33, Jan Beulich wrote:
> On 14.11.2025 14:00, Jürgen Groß wrote:
>> On 14.11.25 12:40, Andrew Cooper wrote:
>>> On 14/11/2025 11:32 am, Juergen Gross wrote:
>>>> diff --git a/docs/Makefile b/docs/Makefile
>>>> index 37776d303c..e5f4a8ca86 100644
>>>> --- a/docs/Makefile
>>>> +++ b/docs/Makefile
>>>> @@ -8,8 +8,11 @@ DATE		:= $(call date,"+%Y-%m-%d")
>>>>    DOC_ARCHES      := arm ppc riscv x86_32 x86_64
>>>>    MAN_SECTIONS    := 1 5 7 8
>>>>    
>>>> +IN_FILES := man/xl-disk-configuration.5.pod man/xl-network-configuration.5.pod
>>>> +IN_FILES += man/xl.1.pod man/xl.cfg.5.pod man/xl.conf.5.pod
>>>
>>> Sorry, I meant to say this on the previous revision.  Can we please list
>>> these one per line, for the future ease of inserting/removing.
>>
>> Okay.
>>
>>> Is IN_FILES really correct?  These are the generated (non-.in) files,
>>> rather than the .in files themselves.  GEN_FILES from v1 would seem to
>>> be a better fit.
>>
>> I wanted to make clear this is related to *.in files. And IMHO GEN_FILES
>> was too generic on a second thought.
>>
>> GENERATED_FROM_IN_SUFFIXED_FILES seems a little bit clumsy. ;-)
>> Seriously, if you have any better name, I'd be happy to use it.
> 
> GEN_POD_FILES, seeing they're all *.pod?

For this case, maybe. OTOH in case someone adds a .podman file we'd need
to rename again.

And I think using the same make variable name in all Makefiles needing to
specify *.in derived files would be preferable.

Maybe IN_TARGETS?


Juergen

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3743 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [PATCH v2 2/4] docs: replace @xxx@ markers at build time

[Jan Beulich]

On 17.11.2025 13:55, Jürgen Groß wrote:
> On 17.11.25 13:33, Jan Beulich wrote:
>> On 14.11.2025 14:00, Jürgen Groß wrote:
>>> On 14.11.25 12:40, Andrew Cooper wrote:
>>>> On 14/11/2025 11:32 am, Juergen Gross wrote:
>>>>> diff --git a/docs/Makefile b/docs/Makefile
>>>>> index 37776d303c..e5f4a8ca86 100644
>>>>> --- a/docs/Makefile
>>>>> +++ b/docs/Makefile
>>>>> @@ -8,8 +8,11 @@ DATE		:= $(call date,"+%Y-%m-%d")
>>>>>    DOC_ARCHES      := arm ppc riscv x86_32 x86_64
>>>>>    MAN_SECTIONS    := 1 5 7 8
>>>>>    
>>>>> +IN_FILES := man/xl-disk-configuration.5.pod man/xl-network-configuration.5.pod
>>>>> +IN_FILES += man/xl.1.pod man/xl.cfg.5.pod man/xl.conf.5.pod
>>>>
>>>> Sorry, I meant to say this on the previous revision.  Can we please list
>>>> these one per line, for the future ease of inserting/removing.
>>>
>>> Okay.
>>>
>>>> Is IN_FILES really correct?  These are the generated (non-.in) files,
>>>> rather than the .in files themselves.  GEN_FILES from v1 would seem to
>>>> be a better fit.
>>>
>>> I wanted to make clear this is related to *.in files. And IMHO GEN_FILES
>>> was too generic on a second thought.
>>>
>>> GENERATED_FROM_IN_SUFFIXED_FILES seems a little bit clumsy. ;-)
>>> Seriously, if you have any better name, I'd be happy to use it.
>>
>> GEN_POD_FILES, seeing they're all *.pod?
> 
> For this case, maybe. OTOH in case someone adds a .podman file we'd need
> to rename again.
> 
> And I think using the same make variable name in all Makefiles needing to
> specify *.in derived files would be preferable.
> 
> Maybe IN_TARGETS?

Better than IN_FILES, but still potentially ambiguous. How about sticking
to IN_FILES but indeed enumerating the .in there (zapping the suffix upon
use)? And/or would $(wildcard <path>/*.in) perhaps make sense to use?

Jan

Re: [PATCH v2 1/4] build: add make macro for making file from file.in

[Jürgen Groß]

[-- Attachment #1.1.1: Type: text/plain, Size: 4157 bytes --]

On 17.11.25 13:51, Jan Beulich wrote:
> On 17.11.2025 13:37, Jürgen Groß wrote:
>> On 17.11.25 13:24, Jan Beulich wrote:
>>> On 14.11.2025 13:54, Jürgen Groß wrote:
>>>> On 14.11.25 12:42, Andrew Cooper wrote:
>>>>> On 14/11/2025 11:32 am, Juergen Gross wrote:
>>>>>> diff --git a/Config.mk b/Config.mk
>>>>>> index e1556dfbfa..d21d67945a 100644
>>>>>> --- a/Config.mk
>>>>>> +++ b/Config.mk
>>>>>> @@ -159,6 +159,19 @@ define move-if-changed
>>>>>>     	if ! cmp -s $(1) $(2); then mv -f $(1) $(2); else rm -f $(1); fi
>>>>>>     endef
>>>>>>     
>>>>>> +PATH_FILES := Paths
>>>>>> +INC_FILES := $(foreach f, $(PATH_FILES), $(XEN_ROOT)/config/$(f).mk)
>>>>>> +
>>>>>> +include $(INC_FILES)
>>>>>> +
>>>>>> +BUILD_MAKE_VARS := $(foreach f, $(PATH_FILES), $(shell awk '$$2 == ":=" { print $$1; }' $(XEN_ROOT)/config/$(f).mk.in))
>>>>>> +
>>>>>> +# Replace @xxx@ markers in $(1).in with $(xxx) variable contents, write to $(1)
>>>>>> +define apply-build-vars
>>>>>> + $(1): $(1).in
>>>>>> +	sed $$(foreach v, $$(BUILD_MAKE_VARS), -e 's#@$$(v)@#$$($$(v))#g') <$$< >$$@
>>>>>> +endef
>>>>>
>>>>> Shouldn't this write to a tmp file, and use move-if-changed?  Most of
>>>>> the time the markers won't have changed, and we'll want to short circuit
>>>>> dependent rules.
>>>>
>>>> I can see this being an advantage when e.g. generating header files, as
>>>> those being generated again would potentially cause lots of rebuilds.
>>>>
>>>> In this case I can hardly see any case where make wouldn't do the right
>>>> thing already. Either the *.in file is newer than the generated file due
>>>> to a git update or a manual edit, so make will regenerate the target (and
>>>> this is what we want), or the *.in file hasn't changed, so make won't
>>>> regenerate the file as it is newer than the *.in file already.
>>>>
>>>> Or did I miss some aspect?
>>>
>>> Aren't some of the generated files Makefile fragments? Them being re-generated
>>
>> No.
>>
>> Man-pages, shell scripts and some Ocaml files (one config file and one .ml file,
>> which is similar to an include file I believe).
>>
>>> means make re-invoking itself, which could be avoided if the contents don't
>>> really change. (This isn't just a performance concern; this re-invocation has
>>> been the source of, well, surprising behavior in certain cases.)
>>
>> I still don't see a case where make would consider rebuilding the file from
>> its .in file without the .in file having changed, thus resulting in the built
>> file to change, too.
> 
> As Andrew indicated, Paths.mk might have changed, so at the very least an
> explicit dependency would need adding. But as alluded to elsewhere, I'm not

Yes, and I said that already.

> quite convinced Paths.mk should be hard-coded as the sole source of patterns
> in Config.mk. At the point further such file come into play, dealing with the
> dependencies might get interesting / clumsy.

See my answer to your next reply.

> 
>> Well, with one probably very rare exception: in case a
>> different @marker@ is used in the .in file, but without changing the resulting
>> file due to old and new marker resulting in the same output.
>>
>> In case we really care about such cases, we should think about using
>> move-if-changed everywhere, as e.g. building a program with $HOSTCC could
>> result in an unchanged binary even with source files having changed, and the
>> resulting program could be used to generate other files ...
> 
> For some of the cases this might actually be worthwhile. It all depends on
> how much of a knock-on effect the re-building of a particular file has.

As long as the effect is not WRONG (which isn't the case with my patch series),
I think we have to consider how often this would be the case.

Optimizing the build time for one case in 10.000 builds (and I think the ratio
in the case of my series is even more extreme) while making the build time even
only a tiny bit longer for all the other cases is a bad idea IMHO. And the build
time will be slower with using a tmp file and invoking the move-if-changed
macro.


Juergen

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3743 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [PATCH v2 1/4] build: add make macro for making file from file.in

[Jürgen Groß]

[-- Attachment #1.1.1: Type: text/plain, Size: 1596 bytes --]

On 17.11.25 13:54, Jan Beulich wrote:
> On 17.11.2025 13:48, Jürgen Groß wrote:
>> On 17.11.25 13:29, Jan Beulich wrote:
>>> On 14.11.2025 12:32, Juergen Gross wrote:
>>>> --- a/Config.mk
>>>> +++ b/Config.mk
>>>> @@ -159,6 +159,19 @@ define move-if-changed
>>>>    	if ! cmp -s $(1) $(2); then mv -f $(1) $(2); else rm -f $(1); fi
>>>>    endef
>>>>    
>>>> +PATH_FILES := Paths
>>>> +INC_FILES := $(foreach f, $(PATH_FILES), $(XEN_ROOT)/config/$(f).mk)
>>>> +
>>>> +include $(INC_FILES)
>>>
>>> Is any of the above part of introducing the macro? "Paths" is already a
>>> specific case of holding patterns that want replacing. In turn ...
>>>
>>>> +BUILD_MAKE_VARS := $(foreach f, $(PATH_FILES), $(shell awk '$$2 == ":=" { print $$1; }' $(XEN_ROOT)/config/$(f).mk.in))
>>>
>>> ... it's not quite clear to me how it can be $(PATH_FILES) here.
>>
>> See patch 4.
>>
>> PATH_FILES is specifying the .mk files containing the marker definitions.
>> I need the ability to have multiple such files in order to be able to let
>> tools/configure build its own definitions.
> 
> That's a good example - why would that affect the stubdom/ part of the tree?
> Imo what pattern file(s) to use wants leaving to the invokee of the macro,
> not pinning down globally for everyone.

Yes, I could add the tools specific marker file in the use cases under tools.

The question is whether adding it to 6 Makefiles is really worth that
optimization, especially as only building the man files would be effected
right now (which could change in future, of course).


Juergen

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3743 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [PATCH v2 2/4] docs: replace @xxx@ markers at build time

[Jürgen Groß]

[-- Attachment #1.1.1: Type: text/plain, Size: 2493 bytes --]

On 17.11.25 14:00, Jan Beulich wrote:
> On 17.11.2025 13:55, Jürgen Groß wrote:
>> On 17.11.25 13:33, Jan Beulich wrote:
>>> On 14.11.2025 14:00, Jürgen Groß wrote:
>>>> On 14.11.25 12:40, Andrew Cooper wrote:
>>>>> On 14/11/2025 11:32 am, Juergen Gross wrote:
>>>>>> diff --git a/docs/Makefile b/docs/Makefile
>>>>>> index 37776d303c..e5f4a8ca86 100644
>>>>>> --- a/docs/Makefile
>>>>>> +++ b/docs/Makefile
>>>>>> @@ -8,8 +8,11 @@ DATE		:= $(call date,"+%Y-%m-%d")
>>>>>>     DOC_ARCHES      := arm ppc riscv x86_32 x86_64
>>>>>>     MAN_SECTIONS    := 1 5 7 8
>>>>>>     
>>>>>> +IN_FILES := man/xl-disk-configuration.5.pod man/xl-network-configuration.5.pod
>>>>>> +IN_FILES += man/xl.1.pod man/xl.cfg.5.pod man/xl.conf.5.pod
>>>>>
>>>>> Sorry, I meant to say this on the previous revision.  Can we please list
>>>>> these one per line, for the future ease of inserting/removing.
>>>>
>>>> Okay.
>>>>
>>>>> Is IN_FILES really correct?  These are the generated (non-.in) files,
>>>>> rather than the .in files themselves.  GEN_FILES from v1 would seem to
>>>>> be a better fit.
>>>>
>>>> I wanted to make clear this is related to *.in files. And IMHO GEN_FILES
>>>> was too generic on a second thought.
>>>>
>>>> GENERATED_FROM_IN_SUFFIXED_FILES seems a little bit clumsy. ;-)
>>>> Seriously, if you have any better name, I'd be happy to use it.
>>>
>>> GEN_POD_FILES, seeing they're all *.pod?
>>
>> For this case, maybe. OTOH in case someone adds a .podman file we'd need
>> to rename again.
>>
>> And I think using the same make variable name in all Makefiles needing to
>> specify *.in derived files would be preferable.
>>
>> Maybe IN_TARGETS?
> 
> Better than IN_FILES, but still potentially ambiguous. How about sticking
> to IN_FILES but indeed enumerating the .in there (zapping the suffix upon
> use)? And/or would $(wildcard <path>/*.in) perhaps make sense to use?

Zapping the suffix upon use would be possible, but more clumsy (there are
normally at least 3 direct uses of IN_FILES in each affected Makefile, while
there are 0 use cases of the .in suffixed source files).

Using a local make variable for speeding that up would have the same problem
as before: how to name it?

And using $(wildcard <path>/*.in) is not an option, as that would reintroduce
the need to distinguish the configure-time and build-time *.in files, which
I solved in V1 of my series by renaming the build-time ones to *.src.


Juergen

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3743 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [PATCH v2 1/4] build: add make macro for making file from file.in

[Jan Beulich]

On 17.11.2025 14:10, Jürgen Groß wrote:
> On 17.11.25 13:54, Jan Beulich wrote:
>> On 17.11.2025 13:48, Jürgen Groß wrote:
>>> On 17.11.25 13:29, Jan Beulich wrote:
>>>> On 14.11.2025 12:32, Juergen Gross wrote:
>>>>> --- a/Config.mk
>>>>> +++ b/Config.mk
>>>>> @@ -159,6 +159,19 @@ define move-if-changed
>>>>>    	if ! cmp -s $(1) $(2); then mv -f $(1) $(2); else rm -f $(1); fi
>>>>>    endef
>>>>>    
>>>>> +PATH_FILES := Paths
>>>>> +INC_FILES := $(foreach f, $(PATH_FILES), $(XEN_ROOT)/config/$(f).mk)
>>>>> +
>>>>> +include $(INC_FILES)
>>>>
>>>> Is any of the above part of introducing the macro? "Paths" is already a
>>>> specific case of holding patterns that want replacing. In turn ...
>>>>
>>>>> +BUILD_MAKE_VARS := $(foreach f, $(PATH_FILES), $(shell awk '$$2 == ":=" { print $$1; }' $(XEN_ROOT)/config/$(f).mk.in))
>>>>
>>>> ... it's not quite clear to me how it can be $(PATH_FILES) here.
>>>
>>> See patch 4.
>>>
>>> PATH_FILES is specifying the .mk files containing the marker definitions.
>>> I need the ability to have multiple such files in order to be able to let
>>> tools/configure build its own definitions.
>>
>> That's a good example - why would that affect the stubdom/ part of the tree?
>> Imo what pattern file(s) to use wants leaving to the invokee of the macro,
>> not pinning down globally for everyone.
> 
> Yes, I could add the tools specific marker file in the use cases under tools.
> 
> The question is whether adding it to 6 Makefiles is really worth that
> optimization, especially as only building the man files would be effected
> right now (which could change in future, of course).

Sticking to Paths.mk (which sits in the global config/ subtree anyway) might
be okay. The new (tools/ specific aiui) file you add later doesn't really
belong there, though. Therefore the macro may want to be constructed such
that it can be used both ways.

Jan

Re: [PATCH v2 2/4] docs: replace @xxx@ markers at build time

[Jan Beulich]

On 17.11.2025 14:17, Jürgen Groß wrote:
> And using $(wildcard <path>/*.in) is not an option, as that would reintroduce
> the need to distinguish the configure-time and build-time *.in files,

Hmm, yes, if there's a mix of them in any one directory, that wouldn't work.

Jan

> which
> I solved in V1 of my series by renaming the build-time ones to *.src.
> 
> 
> Juergen