Re: [LTP] [PATCH] build: Parallelize subdirectory recursion in generic_trunk_target

[Petr Vorel <pvorel@suse.cz>]

Hi Andrea,

thanks for the speedup.

Reviewed-by: Petr Vorel <pvorel@suse.cz>
...

> +# Recurse into subdirectories in parallel instead of using a serial shell loop.
> +# Each subdirectory is mapped to a phony target (<dir>-all, <dir>-install, etc.).
> +# GNU Make can then schedule sibling directories concurrently under -jN.
> +all:     $(addsuffix -all,$(SUBDIRS))
> +install: $(addsuffix -install,$(SUBDIRS))
> +check:   $(addsuffix -check,$(SUBDIRS))
> +
> +.PHONY: $(addsuffix -all,$(SUBDIRS)) $(addsuffix -install,$(SUBDIRS)) $(addsuffix -check,$(SUBDIRS))
> +
> +# For each target (e.g. 'open-all'), strip the suffix to get the directory name,
> +# build trunk-local targets first, and ensure the subdirectory exists.
> +$(addsuffix -all,$(SUBDIRS)): trunk-all | $(SUBDIRS) $(MAKE_DEPS)
>  ifdef VERBOSE
> -	@set -e; for dir in $(SUBDIRS); do \
> -	    $(MAKE) -C $$dir -f "$(abs_srcdir)/$$dir/Makefile" $@; \
> -	done
Yeah, replacing shell for loop which is single process only to make processing
makes sense.

> +	$(MAKE) -C "$(patsubst %-all,%,$@)" -f "$(abs_srcdir)/$(patsubst %-all,%,$@)/Makefile" all
>  else
> -	@set -e; for dir in $(SUBDIRS); do \
> -	    $(MAKE) --no-print-directory -C $$dir -f "$(abs_srcdir)/$$dir/Makefile" $@; \
> -	done
> +	@$(MAKE) --no-print-directory -C "$(patsubst %-all,%,$@)" -f "$(abs_srcdir)/$(patsubst %-all,%,$@)/Makefile" all
> +endif
> +
> +$(addsuffix -install,$(SUBDIRS)): trunk-install | $(SUBDIRS) $(MAKE_DEPS)
> +ifdef VERBOSE
> +	$(MAKE) -C "$(patsubst %-install,%,$@)" -f "$(abs_srcdir)/$(patsubst %-install,%,$@)/Makefile" install
> +else
> +	@$(MAKE) --no-print-directory -C "$(patsubst %-install,%,$@)" -f "$(abs_srcdir)/$(patsubst %-install,%,$@)/Makefile" install
> +endif
> +
> +$(addsuffix -check,$(SUBDIRS)): trunk-check | $(SUBDIRS) $(MAKE_DEPS)
> +ifdef VERBOSE
> +	$(MAKE) -C "$(patsubst %-check,%,$@)" -f "$(abs_srcdir)/$(patsubst %-check,%,$@)/Makefile" check
> +else
> +	@$(MAKE) --no-print-directory -C "$(patsubst %-check,%,$@)" -f "$(abs_srcdir)/$(patsubst %-check,%,$@)/Makefile" check
>  endif
>  endif

> diff --git a/testcases/kernel/Makefile b/testcases/kernel/Makefile
> index 29e26742d..3bf5e433e 100644
> --- a/testcases/kernel/Makefile
> +++ b/testcases/kernel/Makefile
> @@ -54,3 +54,6 @@ FILTER_OUT_DIRS		+= containers controllers device-drivers fs io ipc mem \
>  endif

>  include $(top_srcdir)/include/mk/generic_trunk_target.mk
> +
> +# libkerntest.a in lib/ must be built before syscalls and mem consumers
> +syscalls-all mem-all: lib-all

I wonder if this can be autogenerated, because this was not needed before.

I'm ok with this explicit definition. Probably in the future we should move it
to libs/ (using it as LTPLIBS), which will solve the dependency.

> diff --git a/testcases/network/rpc/Makefile b/testcases/network/rpc/Makefile
> index 6ecb1a665..8d12bdf1e 100644
> --- a/testcases/network/rpc/Makefile
> +++ b/testcases/network/rpc/Makefile
> @@ -9,3 +9,6 @@ include $(top_srcdir)/include/mk/env_pre.mk
>  INSTALL_TARGETS		:= *.sh

>  include $(top_srcdir)/include/mk/generic_trunk_target.mk
> +
> +# librpc-tirpc.a in rpc-tirpc/ must be built before basic_tests
> +basic_tests-all: rpc-tirpc-all

Also librpc-tirpc.a should be moved into libs/ (if we still keep this code and
not move it to libtirpc project where it'd make more sense).

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp