[PATCH v3 0/3] bin-wrappers: test without dashed commands in PATH

[PATCH v3 0/3] bin-wrappers: test without dashed commands in PATH

[Matthew Ogilvie]

This patch series allows running the test suite and/or running an
uninstalled build without dashed commands in the PATH.

Changes since version 2:

   merged - The first 3 old patches (fixes to tests to avoid dashed
         commands, and added GIT_TEST_INSTALLED documentation) are
         now in master, so I am not duplicating them here.

   1/3 - Changed to use "@@BUILD_DIR@@" instead of "__GIT_EXEC_DIR__"
         for sed substitution, and to use single quotes around it
         when setting environment variables in wrap-for-bin.sh.
         But this patch still does not really try to handle
         strange characters in $(shell pwd); see earlier email
         thread about this.  (This is newer than pu.)

   2/3 - Fixed a couple of spelling errors in the --with-dashes
         documentation, and avoid "TOP".  (This is newer than pu.)

   3/3 - Replaced wording for running an uninstalled git.
         It now uses Junio's text: no change compared to his
         version of this patch that is currently in pu.

Matthew Ogilvie (3):
  build dashless "bin-wrappers" directory similar to installed bindir
  run test suite without dashed git-commands in PATH
  INSTALL: document a simpler way to run uninstalled builds

 .gitignore      |    1 +
 INSTALL         |   18 +++++++++++-------
 Makefile        |   49 ++++++++++++++++++++++++++++++++++++-------------
 t/README        |    9 +++++++++
 t/test-lib.sh   |   33 +++++++++++++++++++++------------
 wrap-for-bin.sh |   15 +++++++++++++++
 6 files changed, 93 insertions(+), 32 deletions(-)
 create mode 100644 wrap-for-bin.sh

[PATCH v3 1/3] build dashless "bin-wrappers" directory similar to installed bindir

[Matthew Ogilvie]

The new bin-wrappers directory contains wrapper scripts
for executables that will be installed into the standard
bindir.  It explicitly does not contain most dashed-commands.
The scripts automatically set environment variables to run out
of the source tree, not the installed directory.

This will allow running the test suite without dashed commands in
the PATH.  It also provides a simplified way to test run custom
built git executables without installing them first.

bin-wrappers also contains wrappers for some test suite support
executables, where the test suite will soon make use of them.

Signed-off-by: Matthew Ogilvie <mmogilvi_git@miniinfo.net>
---
 .gitignore      |    1 +
 Makefile        |   49 ++++++++++++++++++++++++++++++++++++-------------
 wrap-for-bin.sh |   15 +++++++++++++++
 3 files changed, 52 insertions(+), 13 deletions(-)
 create mode 100644 wrap-for-bin.sh

diff --git a/.gitignore b/.gitignore
index ac02a58..5d32289 100644
--- a/.gitignore
+++ b/.gitignore
@@ -2,6 +2,7 @@
 /GIT-CFLAGS
 /GIT-GUI-VARS
 /GIT-VERSION-FILE
+/bin-wrappers/
 /git
 /git-add
 /git-add--interactive
diff --git a/Makefile b/Makefile
index 4a1e5bc..378962e 100644
--- a/Makefile
+++ b/Makefile
@@ -427,6 +427,15 @@ ALL_PROGRAMS = $(PROGRAMS) $(SCRIPTS)
 # what 'all' will build but not install in gitexecdir
 OTHER_PROGRAMS = git$X
 
+# what test wrappers are needed and 'install' will install, in bindir
+BINDIR_PROGRAMS_NEED_X += git
+BINDIR_PROGRAMS_NEED_X += git-upload-pack
+BINDIR_PROGRAMS_NEED_X += git-receive-pack
+BINDIR_PROGRAMS_NEED_X += git-upload-archive
+BINDIR_PROGRAMS_NEED_X += git-shell
+
+BINDIR_PROGRAMS_NO_X += git-cvsserver
+
 # Set paths to tools early so that they can be used for version tests.
 ifndef SHELL_PATH
 	SHELL_PATH = /bin/sh
@@ -1714,19 +1723,30 @@ endif
 
 ### Testing rules
 
-TEST_PROGRAMS += test-chmtime$X
-TEST_PROGRAMS += test-ctype$X
-TEST_PROGRAMS += test-date$X
-TEST_PROGRAMS += test-delta$X
-TEST_PROGRAMS += test-dump-cache-tree$X
-TEST_PROGRAMS += test-genrandom$X
-TEST_PROGRAMS += test-match-trees$X
-TEST_PROGRAMS += test-parse-options$X
-TEST_PROGRAMS += test-path-utils$X
-TEST_PROGRAMS += test-sha1$X
-TEST_PROGRAMS += test-sigchain$X
+TEST_PROGRAMS_NEED_X += test-chmtime
+TEST_PROGRAMS_NEED_X += test-ctype
+TEST_PROGRAMS_NEED_X += test-date
+TEST_PROGRAMS_NEED_X += test-delta
+TEST_PROGRAMS_NEED_X += test-dump-cache-tree
+TEST_PROGRAMS_NEED_X += test-genrandom
+TEST_PROGRAMS_NEED_X += test-match-trees
+TEST_PROGRAMS_NEED_X += test-parse-options
+TEST_PROGRAMS_NEED_X += test-path-utils
+TEST_PROGRAMS_NEED_X += test-sha1
+TEST_PROGRAMS_NEED_X += test-sigchain
+
+TEST_PROGRAMS = $(patsubst %,%$X,$(TEST_PROGRAMS_NEED_X))
 
-all:: $(TEST_PROGRAMS)
+test_bindir_programs := $(patsubst %,bin-wrappers/%,$(BINDIR_PROGRAMS_NEED_X) $(BINDIR_PROGRAMS_NO_X) $(TEST_PROGRAMS_NEED_X))
+
+all:: $(TEST_PROGRAMS) $(test_bindir_programs)
+
+bin-wrappers/%: wrap-for-bin.sh
+	@mkdir -p bin-wrappers
+	$(QUIET_GEN)sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \
+	     -e 's|@@BUILD_DIR@@|$(shell pwd)|' \
+	     -e 's|@@PROG@@|$(@F)|' < $< > $@ && \
+	chmod +x $@
 
 # GNU make supports exporting all variables by "export" without parameters.
 # However, the environment gets quite big, and some programs have problems
@@ -1787,11 +1807,13 @@ endif
 gitexec_instdir_SQ = $(subst ','\'',$(gitexec_instdir))
 export gitexec_instdir
 
+install_bindir_programs := $(patsubst %,%$X,$(BINDIR_PROGRAMS_NEED_X)) $(BINDIR_PROGRAMS_NO_X)
+
 install: all
 	$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(bindir_SQ)'
 	$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
 	$(INSTALL) $(ALL_PROGRAMS) '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
-	$(INSTALL) git$X git-upload-pack$X git-receive-pack$X git-upload-archive$X git-shell$X git-cvsserver '$(DESTDIR_SQ)$(bindir_SQ)'
+	$(INSTALL) $(install_bindir_programs) '$(DESTDIR_SQ)$(bindir_SQ)'
 	$(MAKE) -C templates DESTDIR='$(DESTDIR_SQ)' install
 ifndef NO_PERL
 	$(MAKE) -C perl prefix='$(prefix_SQ)' DESTDIR='$(DESTDIR_SQ)' install
@@ -1902,6 +1924,7 @@ clean:
 		$(LIB_FILE) $(XDIFF_LIB)
 	$(RM) $(ALL_PROGRAMS) $(BUILT_INS) git$X
 	$(RM) $(TEST_PROGRAMS)
+	$(RM) -r bin-wrappers
 	$(RM) *.spec *.pyc *.pyo */*.pyc */*.pyo common-cmds.h TAGS tags cscope*
 	$(RM) -r autom4te.cache
 	$(RM) config.log config.mak.autogen config.mak.append config.status config.cache
diff --git a/wrap-for-bin.sh b/wrap-for-bin.sh
new file mode 100644
index 0000000..c5075c9
--- /dev/null
+++ b/wrap-for-bin.sh
@@ -0,0 +1,15 @@
+#!/bin/sh
+
+# wrap-for-bin.sh: Template for git executable wrapper scripts
+# to run test suite against sandbox, but with only bindir-installed
+# executables in PATH.  The Makefile copies this into various
+# files in bin-wrappers, substituting
+# @@BUILD_DIR@@ and @@PROG@@.
+
+GIT_EXEC_PATH='@@BUILD_DIR@@'
+GIT_TEMPLATE_DIR='@@BUILD_DIR@@/templates/blt'
+GITPERLLIB='@@BUILD_DIR@@/perl/blib/lib'
+PATH='@@BUILD_DIR@@/bin-wrappers:'"$PATH"
+export GIT_EXEC_PATH GIT_TEMPLATE_DIR GITPERLLIB PATH
+
+exec "${GIT_EXEC_PATH}/@@PROG@@" "$@"
-- 
1.6.6.rc1

[PATCH v3 2/3] run test suite without dashed git-commands in PATH

[Matthew Ogilvie]

Only put bin-wrappers in the PATH (not GIT_EXEC_PATH), to emulate the
default installed user environment, and ensure all the programs run
correctly in such an environment.  This is now the default, although
it can be overridden with a --with-dashes test option when running
tests.

Signed-off-by: Matthew Ogilvie <mmogilvi_git@miniinfo.net>
---
 t/README      |    9 +++++++++
 t/test-lib.sh |   33 +++++++++++++++++++++------------
 2 files changed, 30 insertions(+), 12 deletions(-)

diff --git a/t/README b/t/README
index 4e1d7dd..dcd3ebb 100644
--- a/t/README
+++ b/t/README
@@ -75,6 +75,15 @@ appropriately before running "make".
 	As the names depend on the tests' file names, it is safe to
 	run the tests with this option in parallel.
 
+--with-dashes::
+	By default tests are run without dashed forms of
+	commands (like git-commit) in the PATH (it only uses
+	wrappers from ../bin-wrappers).  Use this option to include
+	the build directory (..) in the PATH, which contains all
+	the dashed forms of commands.  This option is currently
+	implied by other options like --valgrind and
+	GIT_TEST_INSTALLED.
+
 You can also set the GIT_TEST_INSTALLED environment variable to
 the bindir of an existing git installation to test that installation.
 You still need to have built this git sandbox, from which various
diff --git a/t/test-lib.sh b/t/test-lib.sh
index ec3336a..85377c8 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -105,6 +105,8 @@ do
 		verbose=t; shift ;;
 	-q|--q|--qu|--qui|--quie|--quiet)
 		quiet=t; shift ;;
+	--with-dashes)
+		with_dashes=t; shift ;;
 	--no-color)
 		color=; shift ;;
 	--no-python)
@@ -551,19 +553,8 @@ test_done () {
 # Test the binaries we have just built.  The tests are kept in
 # t/ subdirectory and are run in 'trash directory' subdirectory.
 TEST_DIRECTORY=$(pwd)
-if test -z "$valgrind"
+if test -n "$valgrind"
 then
-	if test -z "$GIT_TEST_INSTALLED"
-	then
-		PATH=$TEST_DIRECTORY/..:$PATH
-		GIT_EXEC_PATH=$TEST_DIRECTORY/..
-	else
-		GIT_EXEC_PATH=$($GIT_TEST_INSTALLED/git --exec-path)  ||
-		error "Cannot run git from $GIT_TEST_INSTALLED."
-		PATH=$GIT_TEST_INSTALLED:$TEST_DIRECTORY/..:$PATH
-		GIT_EXEC_PATH=${GIT_TEST_EXEC_PATH:-$GIT_EXEC_PATH}
-	fi
-else
 	make_symlink () {
 		test -h "$2" &&
 		test "$1" = "$(readlink "$2")" || {
@@ -625,6 +616,24 @@ else
 	PATH=$GIT_VALGRIND/bin:$PATH
 	GIT_EXEC_PATH=$GIT_VALGRIND/bin
 	export GIT_VALGRIND
+elif test -n "$GIT_TEST_INSTALLED" ; then
+	GIT_EXEC_PATH=$($GIT_TEST_INSTALLED/git --exec-path)  ||
+	error "Cannot run git from $GIT_TEST_INSTALLED."
+	PATH=$GIT_TEST_INSTALLED:$TEST_DIRECTORY/..:$PATH
+	GIT_EXEC_PATH=${GIT_TEST_EXEC_PATH:-$GIT_EXEC_PATH}
+else # normal case, use ../bin-wrappers only unless $with_dashes:
+	git_bin_dir="$TEST_DIRECTORY/../bin-wrappers"
+	if ! test -x "$git_bin_dir/git" ; then
+		if test -z "$with_dashes" ; then
+			say "$git_bin_dir/git is not executable; using GIT_EXEC_PATH"
+		fi
+		with_dashes=t
+	fi
+	PATH="$git_bin_dir:$PATH"
+	GIT_EXEC_PATH=$TEST_DIRECTORY/..
+	if test -n "$with_dashes" ; then
+		PATH="$TEST_DIRECTORY/..:$PATH"
+	fi
 fi
 GIT_TEMPLATE_DIR=$(pwd)/../templates/blt
 unset GIT_CONFIG
-- 
1.6.6.rc1

[PATCH v3 3/3] INSTALL: document a simpler way to run uninstalled builds

[Matthew Ogilvie]

The new scripts automatically saved in the bin-wrappers directory allow
you to run a build when you have neither installed git nor tweaked
environment variables.  Mention this in INSTALL, along with the slight
performance issue of doing so.

This can be especially handy for manually testing network-invoked git
(from ssh, web servers, or similar), but it is also handy with a plain
command prompt.

Signed-off-by: Matthew Ogilvie <mmogilvi_git@miniinfo.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 INSTALL |   18 +++++++++++-------
 1 files changed, 11 insertions(+), 7 deletions(-)

diff --git a/INSTALL b/INSTALL
index be504c9..61086ab 100644
--- a/INSTALL
+++ b/INSTALL
@@ -38,13 +38,17 @@ Issues of note:
    Interactive Tools package still can install "git", but you can build it
    with --disable-transition option to avoid this.
 
- - You can use git after building but without installing if you
-   wanted to.  Various git commands need to find other git
-   commands and scripts to do their work, so you would need to
-   arrange a few environment variables to tell them that their
-   friends will be found in your built source area instead of at
-   their standard installation area.  Something like this works
-   for me:
+ - You can use git after building but without installing if you want
+   to test drive it.  Simply run git found in bin-wrappers directory
+   in the build directory, or prepend that directory to your $PATH.
+   This however is less efficient than running an installed git, as
+   you always need an extra fork+exec to run any git subcommand.
+
+   It is still possible to use git without installing by setting a few
+   environment variables, which was the way this was done
+   traditionally.  But using git found in bin-wrappers directory in
+   the build directory is far simpler.  As a historical reference, the
+   old way went like this:
 
 	GIT_EXEC_PATH=`pwd`
 	PATH=`pwd`:$PATH
-- 
1.6.6.rc1