[PATCH] rebase -i: add exec command to launch a shell command

[PATCH] rebase -i: add exec command to launch a shell command

[Matthieu Moy]

The typical usage pattern would be to run a test (or simply a compilation
command) at given points in history.

The shell command is ran (from the worktree root), and the rebase is
stopped when the command fails, to give the user an opportunity to fix
the problem before continuing with "git rebase --continue".

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
---

So, back to the "run from tree root", but that't now properly
documented and tested.

One notable difference with my first version is that the command is
ran in a subshell, defaulting to $SHELL (typically for users like me
with $SHELL=zsh who may want to take advantage of their shell's
advanced features)

 Documentation/git-rebase.txt  |   24 +++++++++++++++++++
 git-rebase--interactive.sh    |   20 ++++++++++++++++
 t/lib-rebase.sh               |    2 +
 t/t3404-rebase-interactive.sh |   50 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 96 insertions(+), 0 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index be23ad2..4bd4b66 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -459,6 +459,30 @@ sure that the current HEAD is "B", and call
 $ git rebase -i -p --onto Q O
 -----------------------------
 
+Reordering and editing commits usually creates untested intermediate
+steps.  You may want to check that your history editing did not break
+anything by running a test, or at least recompiling at intermediate
+points in history by using the "exec" command (shortcut "x").  You may
+do so by creating a todo list like this one:
+
+-------------------------------------------
+pick deadbee Implement feature XXX
+fixup f1a5c00 Fix to feature XXX
+exec make
+pick c0ffeee The oneline of the next commit
+edit deadbab The oneline of the commit after
+exec cd subdir; make test
+...
+-------------------------------------------
+
+The interactive rebase will stop when a command fails (i.e. exits with
+non-0 status) to give you an opportunity to fix the problem. You can
+continue with `git rebase --continue`.
+
+The "exec" command launches the command in a shell (the one specified
+in `$SHELL`, or the default shell if `$SHELL` is not set), so you can
+use usual shell commands like "cd". The command is run from the
+root of the working tree.
 
 SPLITTING COMMITS
 -----------------
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index b94c2a0..33d3087 100755
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -537,6 +537,25 @@ do_next () {
 		esac
 		record_in_rewritten $sha1
 		;;
+	x|"exec")
+		read -r command rest < "$TODO"
+		mark_action_done
+		printf 'Executing: %s\n' "$rest"
+		# "exec" command doesn't take a sha1 in the todo-list.
+		# => can't just use $sha1 here.
+		git rev-parse --verify HEAD > "$DOTEST"/stopped-sha
+		${SHELL:-@SHELL_PATH@} -c "$rest" # Actual execution
+		status=$?
+		if test "$status" -ne 0
+		then
+			warn "Execution failed: $rest"
+			warn "You can fix the problem, and then run"
+			warn
+			warn "	git rebase --continue"
+			warn
+			exit "$status"
+		fi
+		;;
 	*)
 		warn "Unknown command: $command $sha1 $rest"
 		if git rev-parse --verify -q "$sha1" >/dev/null
@@ -957,6 +976,7 @@ first and then run 'git rebase --continue' again."
 #  e, edit = use commit, but stop for amending
 #  s, squash = use commit, but meld into previous commit
 #  f, fixup = like "squash", but discard this commit's log message
+#  x <cmd>, exec <cmd> = Run a shell command <cmd>, and stop if it fails
 #
 # If you remove a line here THAT COMMIT WILL BE LOST.
 # However, if you remove everything, the rebase will be aborted.
diff --git a/t/lib-rebase.sh b/t/lib-rebase.sh
index 6aefe27..6ccf797 100644
--- a/t/lib-rebase.sh
+++ b/t/lib-rebase.sh
@@ -47,6 +47,8 @@ for line in $FAKE_LINES; do
 	case $line in
 	squash|fixup|edit|reword)
 		action="$line";;
+	exec*)
+		echo "$line" | sed 's/_/ /g' >> "$1";;
 	"#")
 		echo '# comment' >> "$1";;
 	">")
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index 9f03ce6..3b07850 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -64,6 +64,56 @@ test_expect_success 'setup' '
 	done
 '
 
+# debugging-friendly alternatives to "test -f" and "test ! -f"
+file_must_exist () {
+    if ! [ -f "$1" ]; then
+	echo "file $1 not created."
+	false
+    fi
+}
+
+file_must_not_exist () {
+    if [ -f "$1" ]; then
+	echo "file $1 created while it shouldn't have. $2"
+	false
+    fi
+}
+
+test_expect_success 'rebase -i with the exec command' '
+	git checkout master &&
+	FAKE_LINES="1 exec_touch_touch-one 2 exec_touch_touch-two exec_false exec_touch_touch-three 3 4
+		exec_touch_\"touch-file__name_with_spaces\";_touch_touch-after-semicolon 5" \
+		test_must_fail git rebase -i A &&
+	file_must_exist touch-one &&
+	file_must_exist touch-two &&
+	file_must_not_exist touch-three "(Should have stopped before)" &&
+	test $(git rev-parse C) = $(git rev-parse HEAD) || {
+		echo "Stopped at wrong revision:"
+		echo "($(git describe --tags HEAD) instead of C)"
+		false
+	} &&
+	git rebase --continue &&
+	file_must_exist touch-three &&
+	file_must_exist "touch-file  name with spaces" &&
+	file_must_exist touch-after-semicolon &&
+	test $(git rev-parse master) = $(git rev-parse HEAD) || {
+		echo "Stopped at wrong revision:"
+		echo "($(git describe --tags HEAD) instead of master)"
+		false
+	} &&
+	rm -f touch-*
+'
+
+test_expect_success 'rebase -i with the exec command runs from tree root' '
+	git checkout master &&
+	mkdir subdir && cd subdir &&
+	FAKE_LINES="1 exec_touch_touch-subdir" \
+		git rebase -i HEAD^ &&
+	cd .. &&
+	file_must_exist touch-subdir &&
+	rm -fr subdir
+'
+
 test_expect_success 'no changes are a nop' '
 	git checkout branch2 &&
 	git rebase -i F &&
-- 
1.7.2.1.30.g18195

Re: [PATCH] rebase -i: add exec command to launch a shell command

[Ævar Arnfjörð Bjarmason]

On Thu, Aug 5, 2010 at 13:00, Matthieu Moy <Matthieu.Moy@imag.fr> wrote:
> The typical usage pattern would be to run a test (or simply a compilation
> command) at given points in history.
>
> The shell command is ran (from the worktree root), and the rebase is
> stopped when the command fails, to give the user an opportunity to fix
> the problem before continuing with "git rebase --continue".
>
> Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
> ---
>
> So, back to the "run from tree root", but that't now properly
> documented and tested.
>
> One notable difference with my first version is that the command is
> ran in a subshell, defaulting to $SHELL (typically for users like me
> with $SHELL=zsh who may want to take advantage of their shell's
> advanced features)
>
>  Documentation/git-rebase.txt  |   24 +++++++++++++++++++
>  git-rebase--interactive.sh    |   20 ++++++++++++++++
>  t/lib-rebase.sh               |    2 +
>  t/t3404-rebase-interactive.sh |   50 +++++++++++++++++++++++++++++++++++++++++
>  4 files changed, 96 insertions(+), 0 deletions(-)
>
> diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
> index be23ad2..4bd4b66 100644
> --- a/Documentation/git-rebase.txt
> +++ b/Documentation/git-rebase.txt
> @@ -459,6 +459,30 @@ sure that the current HEAD is "B", and call
>  $ git rebase -i -p --onto Q O
>  -----------------------------
>
> +Reordering and editing commits usually creates untested intermediate
> +steps.  You may want to check that your history editing did not break
> +anything by running a test, or at least recompiling at intermediate
> +points in history by using the "exec" command (shortcut "x").  You may
> +do so by creating a todo list like this one:
> +
> +-------------------------------------------
> +pick deadbee Implement feature XXX
> +fixup f1a5c00 Fix to feature XXX
> +exec make
> +pick c0ffeee The oneline of the next commit
> +edit deadbab The oneline of the commit after
> +exec cd subdir; make test
> +...
> +-------------------------------------------
> +
> +The interactive rebase will stop when a command fails (i.e. exits with
> +non-0 status) to give you an opportunity to fix the problem. You can
> +continue with `git rebase --continue`.
> +
> +The "exec" command launches the command in a shell (the one specified
> +in `$SHELL`, or the default shell if `$SHELL` is not set), so you can
> +use usual shell commands like "cd". The command is run from the

I think that needs a definite article: ".. use the usual ..".

> +# debugging-friendly alternatives to "test -f" and "test ! -f"
> +file_must_exist () {
> +    if ! [ -f "$1" ]; then
> +       echo "file $1 not created."
> +       false
> +    fi
> +}
> +
> +file_must_not_exist () {
> +    if [ -f "$1" ]; then
> +       echo "file $1 created while it shouldn't have. $2"
> +       false
> +    fi
> +}

As I pointed out in a previous comment to the series this sort of
debug code should either be converted to use "test" or we should
incorporate it into test-lib.sh and use it everywhere.

I somewhat like the latter. It's sometimes hard to see what's going
wrong with our tests. It'd also translate to TAP subtests.

Otherwise this all looks good. Especially without the fragile mkdir/chdir
part present in a previous submission.

Re: [PATCH] rebase -i: add exec command to launch a shell command

[Matthieu Moy]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

>> +in `$SHELL`, or the default shell if `$SHELL` is not set), so you can
>> +use usual shell commands like "cd". The command is run from the
>
> I think that needs a definite article: ".. use the usual ..".

I don't think so, especially with a plural "commands" after.
Googlefight agrees with me ("use the usual commands" => 350 results,
"use usual commands" => 4900), but that's not a proof. Perhaps a
native speaker could help?

> As I pointed out in a previous comment to the series this sort of
> debug code should either be converted to use "test" or we should
> incorporate it into test-lib.sh and use it everywhere.
>
> I somewhat like the latter. It's sometimes hard to see what's going
> wrong with our tests. It'd also translate to TAP subtests.

I'll do both: use test -f in a first patch, and propose alternative in
the second. New patch serie follows.

I don't know TAP and TAP subtests, but if the functions exist, other
patches can be added on top to improve them.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

[PATCH 1/2] rebase -i: add exec command to launch a shell command

[Matthieu Moy]

The typical usage pattern would be to run a test (or simply a compilation
command) at given points in history.

The shell command is ran (from the worktree root), and the rebase is
stopped when the command fails, to give the user an opportunity to fix
the problem before continuing with "git rebase --continue".

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
---
 Documentation/git-rebase.txt  |   24 ++++++++++++++++++++++++
 git-rebase--interactive.sh    |   20 ++++++++++++++++++++
 t/lib-rebase.sh               |    2 ++
 t/t3404-rebase-interactive.sh |   35 +++++++++++++++++++++++++++++++++++
 4 files changed, 81 insertions(+), 0 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index be23ad2..4bd4b66 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -459,6 +459,30 @@ sure that the current HEAD is "B", and call
 $ git rebase -i -p --onto Q O
 -----------------------------
 
+Reordering and editing commits usually creates untested intermediate
+steps.  You may want to check that your history editing did not break
+anything by running a test, or at least recompiling at intermediate
+points in history by using the "exec" command (shortcut "x").  You may
+do so by creating a todo list like this one:
+
+-------------------------------------------
+pick deadbee Implement feature XXX
+fixup f1a5c00 Fix to feature XXX
+exec make
+pick c0ffeee The oneline of the next commit
+edit deadbab The oneline of the commit after
+exec cd subdir; make test
+...
+-------------------------------------------
+
+The interactive rebase will stop when a command fails (i.e. exits with
+non-0 status) to give you an opportunity to fix the problem. You can
+continue with `git rebase --continue`.
+
+The "exec" command launches the command in a shell (the one specified
+in `$SHELL`, or the default shell if `$SHELL` is not set), so you can
+use usual shell commands like "cd". The command is run from the
+root of the working tree.
 
 SPLITTING COMMITS
 -----------------
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index b94c2a0..33d3087 100755
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -537,6 +537,25 @@ do_next () {
 		esac
 		record_in_rewritten $sha1
 		;;
+	x|"exec")
+		read -r command rest < "$TODO"
+		mark_action_done
+		printf 'Executing: %s\n' "$rest"
+		# "exec" command doesn't take a sha1 in the todo-list.
+		# => can't just use $sha1 here.
+		git rev-parse --verify HEAD > "$DOTEST"/stopped-sha
+		${SHELL:-@SHELL_PATH@} -c "$rest" # Actual execution
+		status=$?
+		if test "$status" -ne 0
+		then
+			warn "Execution failed: $rest"
+			warn "You can fix the problem, and then run"
+			warn
+			warn "	git rebase --continue"
+			warn
+			exit "$status"
+		fi
+		;;
 	*)
 		warn "Unknown command: $command $sha1 $rest"
 		if git rev-parse --verify -q "$sha1" >/dev/null
@@ -957,6 +976,7 @@ first and then run 'git rebase --continue' again."
 #  e, edit = use commit, but stop for amending
 #  s, squash = use commit, but meld into previous commit
 #  f, fixup = like "squash", but discard this commit's log message
+#  x <cmd>, exec <cmd> = Run a shell command <cmd>, and stop if it fails
 #
 # If you remove a line here THAT COMMIT WILL BE LOST.
 # However, if you remove everything, the rebase will be aborted.
diff --git a/t/lib-rebase.sh b/t/lib-rebase.sh
index 6aefe27..6ccf797 100644
--- a/t/lib-rebase.sh
+++ b/t/lib-rebase.sh
@@ -47,6 +47,8 @@ for line in $FAKE_LINES; do
 	case $line in
 	squash|fixup|edit|reword)
 		action="$line";;
+	exec*)
+		echo "$line" | sed 's/_/ /g' >> "$1";;
 	"#")
 		echo '# comment' >> "$1";;
 	">")
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index 9f03ce6..bba220a 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -64,6 +64,41 @@ test_expect_success 'setup' '
 	done
 '
 
+test_expect_success 'rebase -i with the exec command' '
+	git checkout master &&
+	FAKE_LINES="1 exec_touch_touch-one 2 exec_touch_touch-two exec_false exec_touch_touch-three 3 4
+		exec_touch_\"touch-file__name_with_spaces\";_touch_touch-after-semicolon 5" \
+		test_must_fail git rebase -i A &&
+	test -f touch-one &&
+	test -f touch-two &&
+	! test -f touch-three &&
+	test $(git rev-parse C) = $(git rev-parse HEAD) || {
+		echo "Stopped at wrong revision:"
+		echo "($(git describe --tags HEAD) instead of C)"
+		false
+	} &&
+	git rebase --continue &&
+	test -f touch-three &&
+	test -f "touch-file  name with spaces" &&
+	test -f touch-after-semicolon &&
+	test $(git rev-parse master) = $(git rev-parse HEAD) || {
+		echo "Stopped at wrong revision:"
+		echo "($(git describe --tags HEAD) instead of master)"
+		false
+	} &&
+	rm -f touch-*
+'
+
+test_expect_success 'rebase -i with the exec command runs from tree root' '
+	git checkout master &&
+	mkdir subdir && cd subdir &&
+	FAKE_LINES="1 exec_touch_touch-subdir" \
+		git rebase -i HEAD^ &&
+	cd .. &&
+	test -f touch-subdir &&
+	rm -fr subdir
+'
+
 test_expect_success 'no changes are a nop' '
 	git checkout branch2 &&
 	git rebase -i F &&
-- 
1.7.2.1.30.g18195

[PATCH 2/2] test-lib: user-friendly alternatives to test [!] [-d|-f]

[Matthieu Moy]

The helper functions are implemented, documented, and used in a few
places to validate them, but not everywhere to avoid useless code churn.

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
---
 t/README                      |    8 ++++++++
 t/t3404-rebase-interactive.sh |   18 +++++++++---------
 t/t3407-rebase-abort.sh       |    6 +++---
 t/test-lib.sh                 |   32 ++++++++++++++++++++++++++++++++
 4 files changed, 52 insertions(+), 12 deletions(-)

diff --git a/t/README b/t/README
index 0d1183c..be760b9 100644
--- a/t/README
+++ b/t/README
@@ -467,6 +467,14 @@ library for your script to use.
    <expected> file.  This behaves like "cmp" but produces more
    helpful output when the test is run with "-v" option.
 
+ - test_file_must_exist <file> [<diagnosis>]
+   test_file_must_not_exist <file> [<diagnosis>]
+   test_dir_must_exist <dir> [<diagnosis>]
+   test_dir_must_not_exist <dir> [<diagnosis>]
+
+   check whether a file/directory exists or doesn't. <diagnosis> will
+   be displayed if the test fails.
+
  - test_when_finished <script>
 
    Prepend <script> to a list of commands to run to clean up
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index bba220a..50787c2 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -69,18 +69,18 @@ test_expect_success 'rebase -i with the exec command' '
 	FAKE_LINES="1 exec_touch_touch-one 2 exec_touch_touch-two exec_false exec_touch_touch-three 3 4
 		exec_touch_\"touch-file__name_with_spaces\";_touch_touch-after-semicolon 5" \
 		test_must_fail git rebase -i A &&
-	test -f touch-one &&
-	test -f touch-two &&
-	! test -f touch-three &&
+	test_file_must_exist touch-one &&
+	test_file_must_exist touch-two &&
+	test_file_must_not_exist touch-three "(Rebase should have stopped before)" &&
 	test $(git rev-parse C) = $(git rev-parse HEAD) || {
 		echo "Stopped at wrong revision:"
 		echo "($(git describe --tags HEAD) instead of C)"
 		false
 	} &&
 	git rebase --continue &&
-	test -f touch-three &&
-	test -f "touch-file  name with spaces" &&
-	test -f touch-after-semicolon &&
+	test_file_must_exist touch-three &&
+	test_file_must_exist "touch-file  name with spaces" &&
+	test_file_must_exist touch-after-semicolon &&
 	test $(git rev-parse master) = $(git rev-parse HEAD) || {
 		echo "Stopped at wrong revision:"
 		echo "($(git describe --tags HEAD) instead of master)"
@@ -95,7 +95,7 @@ test_expect_success 'rebase -i with the exec command runs from tree root' '
 	FAKE_LINES="1 exec_touch_touch-subdir" \
 		git rebase -i HEAD^ &&
 	cd .. &&
-	test -f touch-subdir &&
+	test_file_must_exist touch-subdir &&
 	rm -fr subdir
 '
 
@@ -178,7 +178,7 @@ test_expect_success 'abort' '
 	git rebase --abort &&
 	test $(git rev-parse new-branch1) = $(git rev-parse HEAD) &&
 	test "$(git symbolic-ref -q HEAD)" = "refs/heads/branch1" &&
-	! test -d .git/rebase-merge
+	test_dir_must_not_exist .git/rebase-merge
 '
 
 test_expect_success 'abort with error when new base cannot be checked out' '
@@ -187,7 +187,7 @@ test_expect_success 'abort with error when new base cannot be checked out' '
 	test_must_fail git rebase -i master > output 2>&1 &&
 	grep "Untracked working tree file .file1. would be overwritten" \
 		output &&
-	! test -d .git/rebase-merge &&
+	test_dir_must_not_exist .git/rebase-merge &&
 	git reset --hard HEAD^
 '
 
diff --git a/t/t3407-rebase-abort.sh b/t/t3407-rebase-abort.sh
index 2999e78..a1615b8 100755
--- a/t/t3407-rebase-abort.sh
+++ b/t/t3407-rebase-abort.sh
@@ -38,7 +38,7 @@ testrebase() {
 		# Clean up the state from the previous one
 		git reset --hard pre-rebase &&
 		test_must_fail git rebase$type master &&
-		test -d "$dotest" &&
+		test_dir_must_not_exist "$dotest" &&
 		git rebase --abort &&
 		test $(git rev-parse to-rebase) = $(git rev-parse pre-rebase) &&
 		test ! -d "$dotest"
@@ -49,7 +49,7 @@ testrebase() {
 		# Clean up the state from the previous one
 		git reset --hard pre-rebase &&
 		test_must_fail git rebase$type master &&
-		test -d "$dotest" &&
+		test_dir_must_not_exist "$dotest" &&
 		test_must_fail git rebase --skip &&
 		test $(git rev-parse HEAD) = $(git rev-parse master) &&
 		git rebase --abort &&
@@ -62,7 +62,7 @@ testrebase() {
 		# Clean up the state from the previous one
 		git reset --hard pre-rebase &&
 		test_must_fail git rebase$type master &&
-		test -d "$dotest" &&
+		test_dir_must_not_exist "$dotest" &&
 		echo c > a &&
 		echo d >> a &&
 		git add a &&
diff --git a/t/test-lib.sh b/t/test-lib.sh
index e8f21d5..3701f2d 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -541,6 +541,38 @@ test_external_without_stderr () {
 	fi
 }
 
+# debugging-friendly alternatives to "test [!] [-f|-d]"
+# The commands test the existence or non-existance of $1. $2 can be
+# given to provide a more precise diagnosis.
+test_file_must_exist () {
+    if ! [ -f "$1" ]; then
+	echo "file $1 doesn't exist. $*"
+	false
+    fi
+}
+
+test_file_must_not_exist () {
+    if [ -f "$1" ]; then
+	echo "file $1 exists. $*"
+	false
+    fi
+}
+
+test_dir_must_exist () {
+    if ! [ -d "$1" ]; then
+	echo "directory $1 doesn't exist. $*"
+	false
+    fi
+}
+
+test_file_must_not_exist () {
+    if [ -d "$1" ]; then
+	echo "directory $1 exists. $*"
+	false
+    fi
+}
+
+
 # This is not among top-level (test_expect_success | test_expect_failure)
 # but is a prefix that can be used in the test script, like:
 #
-- 
1.7.2.1.30.g18195

[PATCH v2] test-lib: user-friendly alternatives to test [!] [-d|-f]

[Matthieu Moy]

The helper functions are implemented, documented, and used in a few
places to validate them, but not everywhere to avoid useless code churn.

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
---
Sooooory, the first version was plain broken (the tests did not even
pass anymore), forget it. This should be better.

 t/README                      |    8 ++++++++
 t/t3404-rebase-interactive.sh |   18 +++++++++---------
 t/t3407-rebase-abort.sh       |    6 +++---
 t/test-lib.sh                 |   32 ++++++++++++++++++++++++++++++++
 4 files changed, 52 insertions(+), 12 deletions(-)

diff --git a/t/README b/t/README
index 0d1183c..be760b9 100644
--- a/t/README
+++ b/t/README
@@ -467,6 +467,14 @@ library for your script to use.
    <expected> file.  This behaves like "cmp" but produces more
    helpful output when the test is run with "-v" option.
 
+ - test_file_must_exist <file> [<diagnosis>]
+   test_file_must_not_exist <file> [<diagnosis>]
+   test_dir_must_exist <dir> [<diagnosis>]
+   test_dir_must_not_exist <dir> [<diagnosis>]
+
+   check whether a file/directory exists or doesn't. <diagnosis> will
+   be displayed if the test fails.
+
  - test_when_finished <script>
 
    Prepend <script> to a list of commands to run to clean up
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index bba220a..50787c2 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -69,18 +69,18 @@ test_expect_success 'rebase -i with the exec command' '
 	FAKE_LINES="1 exec_touch_touch-one 2 exec_touch_touch-two exec_false exec_touch_touch-three 3 4
 		exec_touch_\"touch-file__name_with_spaces\";_touch_touch-after-semicolon 5" \
 		test_must_fail git rebase -i A &&
-	test -f touch-one &&
-	test -f touch-two &&
-	! test -f touch-three &&
+	test_file_must_exist touch-one &&
+	test_file_must_exist touch-two &&
+	test_file_must_not_exist touch-three "(Rebase should have stopped before)" &&
 	test $(git rev-parse C) = $(git rev-parse HEAD) || {
 		echo "Stopped at wrong revision:"
 		echo "($(git describe --tags HEAD) instead of C)"
 		false
 	} &&
 	git rebase --continue &&
-	test -f touch-three &&
-	test -f "touch-file  name with spaces" &&
-	test -f touch-after-semicolon &&
+	test_file_must_exist touch-three &&
+	test_file_must_exist "touch-file  name with spaces" &&
+	test_file_must_exist touch-after-semicolon &&
 	test $(git rev-parse master) = $(git rev-parse HEAD) || {
 		echo "Stopped at wrong revision:"
 		echo "($(git describe --tags HEAD) instead of master)"
@@ -95,7 +95,7 @@ test_expect_success 'rebase -i with the exec command runs from tree root' '
 	FAKE_LINES="1 exec_touch_touch-subdir" \
 		git rebase -i HEAD^ &&
 	cd .. &&
-	test -f touch-subdir &&
+	test_file_must_exist touch-subdir &&
 	rm -fr subdir
 '
 
@@ -178,7 +178,7 @@ test_expect_success 'abort' '
 	git rebase --abort &&
 	test $(git rev-parse new-branch1) = $(git rev-parse HEAD) &&
 	test "$(git symbolic-ref -q HEAD)" = "refs/heads/branch1" &&
-	! test -d .git/rebase-merge
+	test_dir_must_not_exist .git/rebase-merge
 '
 
 test_expect_success 'abort with error when new base cannot be checked out' '
@@ -187,7 +187,7 @@ test_expect_success 'abort with error when new base cannot be checked out' '
 	test_must_fail git rebase -i master > output 2>&1 &&
 	grep "Untracked working tree file .file1. would be overwritten" \
 		output &&
-	! test -d .git/rebase-merge &&
+	test_dir_must_not_exist .git/rebase-merge &&
 	git reset --hard HEAD^
 '
 
diff --git a/t/t3407-rebase-abort.sh b/t/t3407-rebase-abort.sh
index 2999e78..0ca81fe 100755
--- a/t/t3407-rebase-abort.sh
+++ b/t/t3407-rebase-abort.sh
@@ -38,7 +38,7 @@ testrebase() {
 		# Clean up the state from the previous one
 		git reset --hard pre-rebase &&
 		test_must_fail git rebase$type master &&
-		test -d "$dotest" &&
+		test_dir_must_exist "$dotest" &&
 		git rebase --abort &&
 		test $(git rev-parse to-rebase) = $(git rev-parse pre-rebase) &&
 		test ! -d "$dotest"
@@ -49,7 +49,7 @@ testrebase() {
 		# Clean up the state from the previous one
 		git reset --hard pre-rebase &&
 		test_must_fail git rebase$type master &&
-		test -d "$dotest" &&
+		test_dir_must_exist "$dotest" &&
 		test_must_fail git rebase --skip &&
 		test $(git rev-parse HEAD) = $(git rev-parse master) &&
 		git rebase --abort &&
@@ -62,7 +62,7 @@ testrebase() {
 		# Clean up the state from the previous one
 		git reset --hard pre-rebase &&
 		test_must_fail git rebase$type master &&
-		test -d "$dotest" &&
+		test_dir_must_exist "$dotest" &&
 		echo c > a &&
 		echo d >> a &&
 		git add a &&
diff --git a/t/test-lib.sh b/t/test-lib.sh
index e8f21d5..694bbe8 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -541,6 +541,38 @@ test_external_without_stderr () {
 	fi
 }
 
+# debugging-friendly alternatives to "test [!] [-f|-d]"
+# The commands test the existence or non-existance of $1. $2 can be
+# given to provide a more precise diagnosis.
+test_file_must_exist () {
+	if ! [ -f "$1" ]; then
+		echo "file $1 doesn't exist. $*"
+		false
+	fi
+}
+
+test_file_must_not_exist () {
+	if [ -f "$1" ]; then
+		echo "file $1 exists. $*"
+		false
+	fi
+}
+
+test_dir_must_exist () {
+	if ! [ -d "$1" ]; then
+		echo "directory $1 doesn't exist. $*"
+		false
+	fi
+}
+
+test_dir_must_not_exist () {
+	if [ -d "$1" ]; then
+		echo "directory $1 exists. $*"
+		false
+	fi
+}
+
+
 # This is not among top-level (test_expect_success | test_expect_failure)
 # but is a prefix that can be used in the test script, like:
 #
-- 
1.7.2.1.30.g18195

Re: [PATCH v2] test-lib: user-friendly alternatives to test [!] [-d|-f]

[Jonathan Nieder]

Matthieu Moy wrote:

> The helper functions are implemented, documented, and used in a few
> places to validate them

When I first read this, I thought you were saying these helpers
already existed.  This is where the rationale goes, anyway, so maybe:

	Add new test_file_must_not_exist et al helpers for
	use by tests to more loudly diagnose failures that
	manifest themselves by the existence or nonexistence
	of a file or directory.

	So now you can use

		test_file_must_exist foo "so there"

	from your test, and when it fails due to foo being
	absent or being a symlink instead, instead of silence
	you will get (if debugging with "-v") the helpful message

		file foo does not exist. so there.

> +++ b/t/README
> @@ -467,6 +467,14 @@ library for your script to use.
>     <expected> file.  This behaves like "cmp" but produces more
>     helpful output when the test is run with "-v" option.
>  
> + - test_file_must_exist <file> [<diagnosis>]
> +   test_file_must_not_exist <file> [<diagnosis>]
> +   test_dir_must_exist <dir> [<diagnosis>]
> +   test_dir_must_not_exist <dir> [<diagnosis>]
> +
> +   check whether a file/directory exists or doesn't. <diagnosis> will
> +   be displayed if the test fails.

Maybe:

	- test_file_exists <name> [<diagnosis>]
	- test_dir_exists <name> [<diagnosis>]

	  Check that <name> exists and is a file or directory,
	  printing a diagnostic if it does not.  The <diagnosis>
	  if present will be used to give some added context to
	  the diagnostic.

	- test_does_not_exist <name> [<diagnosis>]

	  Check that <name> does not exist, printing a
	  diagnostic if it does.  The <diagnosis> will be
	  printed on failure as added context if present.

I think the ..._must_exist names put the emphasis in the
wrong place, and they look funny in "if" statements.

> +++ b/t/t3404-rebase-interactive.sh
> +++ b/t/t3407-rebase-abort.sh
[examples]

Makes sense.

> +++ b/t/test-lib.sh
> @@ -541,6 +541,38 @@ test_external_without_stderr () {
>  	fi
>  }
>  
> +# debugging-friendly alternatives to "test [!] [-f|-d]"
> +# The commands test the existence or non-existance of $1. $2 can be
> +# given to provide a more precise diagnosis.
> +test_file_must_exist () {
> +	if ! [ -f "$1" ]; then
> +		echo "file $1 doesn't exist. $*"
> +		false
> +	fi
> +}

Style nitpick: if statementss in the test-lib have tended to look like

 if [ foo ]
 then
	bar
 fi

so far.  Here the whole function is a glorified "test -f", so I wonder
if

	[ -f "$1" ] ||
	{
		echo >&2 "file $1 doesn't exist. $*"
		false
	}

would not be clearer.  I dunno.

> +test_file_must_not_exist () {
> +	if [ -f "$1" ]; then
> +		echo "file $1 exists. $*"
> +		false
> +	fi
> +}

What should happen if $1 exists and is not a file?

I have often run into silent test failures of the sort your patch
is designed to avoid.  Thanks for tackling it.

Re: [PATCH v2] test-lib: user-friendly alternatives to test [!] [-d|-f]

[Ævar Arnfjörð Bjarmason]

On Fri, Aug 6, 2010 at 22:57, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Matthieu Moy wrote:
>        - test_file_exists <name> [<diagnosis>]
>        - test_dir_exists <name> [<diagnosis>]
>
>          Check that <name> exists and is a file or directory,
>          printing a diagnostic if it does not.  The <diagnosis>
>          if present will be used to give some added context to
>          the diagnostic.
>
>        - test_does_not_exist <name> [<diagnosis>]
>
>          Check that <name> does not exist, printing a
>          diagnostic if it does.  The <diagnosis> will be
>          printed on failure as added context if present.
>
> I think the ..._must_exist names put the emphasis in the
> wrong place, and they look funny in "if" statements.

Personally I'd prefer something where I can just think in the "test"
bultin terms and still get debug info, something like (pseduocode)

    gittest() {
        test "$#" = 3 && { bool='!'; shift; } || bool=
        test=$1; shift
        args="$@"; shift
        case "$test" in
        -f)
            # Handle the common case of -f with a custom message
        ;;
        -d)
            # Same for -d
        ;;
        *)
            # Just pass a switch to test and say "test with the -X
switch", or something
esac
    }

    gittest ! -f ~/.gitconfig
    gittest -f ~/.gitconfig
    gittest -d /tmp
    gittest ! -d /tmp
    gittest -s /tmp

I'll never be able to fit more test_* functions in my brain, and I
wrote the docs :)

> Style nitpick: if statementss in the test-lib have tended to look like
>
>  if [ foo ]
>  then
>        bar
>  fi
>
> so far.  Here the whole function is a glorified "test -f", so I wonder
> if
>
>        [ -f "$1" ] ||
>        {
>                echo >&2 "file $1 doesn't exist. $*"
>                false
>        }
>
> would not be clearer.  I dunno.

This is the style we usually use:

    if ! test -f "$1"
    then
        echo >&2 "file $1 doesn't exist. $*"
        false
    fi

> I have often run into silent test failures of the sort your patch
> is designed to avoid.  Thanks for tackling it.

Yeah, having more intra-test progress is definitely good. Right now I
just remove things from the tests in an ad-hoc fashion until they
start passing if they fail when I debug them.

I mentioned that we could emit these test progress reports as TAP in a
previous E-Mail. Here's how that could look like:

    $ perl -MTest::More=no_plan -E '
        subtest "A git test" => sub {
            pass("doing test -f file");
            pass("git commit ...");
            pass("test_tick...");
            done_testing();
        } for 1 .. 2
    '
        ok 1 - doing test -f file
        ok 2 - git commit ...
        ok 3 - test_tick...
        1..3
    ok 1 - A git test
        ok 1 - doing test -f file
        ok 2 - git commit ...
        ok 3 - test_tick...
        1..3
    ok 2 - A git test
    1..2

I.e. we could make these intra-test progress reports machine readable.

As the example shows the obvious next step would be to make other
utility functions like test_commit() emit a progress status as well.

Re: [PATCH 2/2] test-lib: user-friendly alternatives to test [!] [-d|-f]

[Junio C Hamano]

Matthieu Moy <Matthieu.Moy@imag.fr> writes:

> +   test_file_must_not_exist <file> [<diagnosis>]
> +   test_dir_must_not_exist <dir> [<diagnosis>]

Should either of these pass?

    mkdir foo && test_file_must_not_exist foo
    rm -fr foo && >foo && test_dir_must_not_exist foo

I think in most of the test cases we want to write "must not exist" to
make sure what we are supposed to remove is gone, which would mean that
(1) we know what that thing is, and (2) we not only just do not want a
file "foo" when we say "file-must-not-exist foo", but we don't expect it
to be a directory either.

I'd say we would probably want three primitives instead of four:

    test_path_is_file        <path>
    test_path_is_directory   <path>
    test_path_is_missing     <path>

Thanks.

[PATCH] test-lib: user-friendly alternatives to test [-d|-f|-e]

[Matthieu Moy]

The helper functions are implemented, documented, and used in a few
places to validate them, but not everywhere to avoid useless code churn.

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
---
> I'd say we would probably want three primitives instead of four:
> 
>     test_path_is_file        <path>
>     test_path_is_directory   <path>
>     test_path_is_missing     <path>

I buy this. Thanks,

 t/README                      |    7 +++++++
 t/t3404-rebase-interactive.sh |   18 +++++++++---------
 t/t3407-rebase-abort.sh       |    6 +++---
 t/test-lib.sh                 |   32 ++++++++++++++++++++++++++++++++
 4 files changed, 51 insertions(+), 12 deletions(-)

diff --git a/t/README b/t/README
index 0d1183c..410499a 100644
--- a/t/README
+++ b/t/README
@@ -467,6 +467,13 @@ library for your script to use.
    <expected> file.  This behaves like "cmp" but produces more
    helpful output when the test is run with "-v" option.
 
+ - test_path_is_file <file> [<diagnosis>]
+   test_path_is_dir <dir> [<diagnosis>]
+   test_path_is_missing <path> [<diagnosis>]
+
+   Check whether a file/directory exists or doesn't. <diagnosis> will
+   be displayed if the test fails.
+
  - test_when_finished <script>
 
    Prepend <script> to a list of commands to run to clean up
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index 7b0026e..f78c364 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -69,18 +69,18 @@ test_expect_success 'rebase -i with the exec command' '
 	FAKE_LINES="1 exec_touch_touch-one 2 exec_touch_touch-two exec_false exec_touch_touch-three 3 4
 		exec_touch_\"touch-file__name_with_spaces\";_touch_touch-after-semicolon 5" \
 		test_must_fail git rebase -i A &&
-	test -f touch-one &&
-	test -f touch-two &&
-	! test -f touch-three &&
+	test_path_is_file touch-one &&
+	test_path_is_file touch-two &&
+	test_path_is_missing touch-three "(Rebase should have stopped before)" &&
 	test $(git rev-parse C) = $(git rev-parse HEAD) || {
 		echo "Stopped at wrong revision:"
 		echo "($(git describe --tags HEAD) instead of C)"
 		false
 	} &&
 	git rebase --continue &&
-	test -f touch-three &&
-	test -f "touch-file  name with spaces" &&
-	test -f touch-after-semicolon &&
+	test_path_is_file touch-three &&
+	test_path_is_file "touch-file  name with spaces" &&
+	test_path_is_file touch-after-semicolon &&
 	test $(git rev-parse master) = $(git rev-parse HEAD) || {
 		echo "Stopped at wrong revision:"
 		echo "($(git describe --tags HEAD) instead of master)"
@@ -95,7 +95,7 @@ test_expect_success 'rebase -i with the exec command runs from tree root' '
 	FAKE_LINES="1 exec_touch_touch-subdir" \
 		git rebase -i HEAD^ &&
 	cd .. &&
-	test -f touch-subdir &&
+	test_path_is_file touch-subdir &&
 	rm -fr subdir
 '
 
@@ -191,7 +191,7 @@ test_expect_success 'abort' '
 	git rebase --abort &&
 	test $(git rev-parse new-branch1) = $(git rev-parse HEAD) &&
 	test "$(git symbolic-ref -q HEAD)" = "refs/heads/branch1" &&
-	! test -d .git/rebase-merge
+	test_path_is_missing .git/rebase-merge
 '
 
 test_expect_success 'abort with error when new base cannot be checked out' '
@@ -200,7 +200,7 @@ test_expect_success 'abort with error when new base cannot be checked out' '
 	test_must_fail git rebase -i master > output 2>&1 &&
 	grep "Untracked working tree file .file1. would be overwritten" \
 		output &&
-	! test -d .git/rebase-merge &&
+	test_path_is_missing .git/rebase-merge &&
 	git reset --hard HEAD^
 '
 
diff --git a/t/t3407-rebase-abort.sh b/t/t3407-rebase-abort.sh
index 2999e78..fbb3f2e 100755
--- a/t/t3407-rebase-abort.sh
+++ b/t/t3407-rebase-abort.sh
@@ -38,7 +38,7 @@ testrebase() {
 		# Clean up the state from the previous one
 		git reset --hard pre-rebase &&
 		test_must_fail git rebase$type master &&
-		test -d "$dotest" &&
+		test_path_is_dir "$dotest" &&
 		git rebase --abort &&
 		test $(git rev-parse to-rebase) = $(git rev-parse pre-rebase) &&
 		test ! -d "$dotest"
@@ -49,7 +49,7 @@ testrebase() {
 		# Clean up the state from the previous one
 		git reset --hard pre-rebase &&
 		test_must_fail git rebase$type master &&
-		test -d "$dotest" &&
+		test_path_is_dir "$dotest" &&
 		test_must_fail git rebase --skip &&
 		test $(git rev-parse HEAD) = $(git rev-parse master) &&
 		git rebase --abort &&
@@ -62,7 +62,7 @@ testrebase() {
 		# Clean up the state from the previous one
 		git reset --hard pre-rebase &&
 		test_must_fail git rebase$type master &&
-		test -d "$dotest" &&
+		test_path_is_dir "$dotest" &&
 		echo c > a &&
 		echo d >> a &&
 		git add a &&
diff --git a/t/test-lib.sh b/t/test-lib.sh
index e8f21d5..a2173dd 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -541,6 +541,38 @@ test_external_without_stderr () {
 	fi
 }
 
+# debugging-friendly alternatives to "test [-f|-d|-e]"
+# The commands test the existence or non-existance of $1. $2 can be
+# given to provide a more precise diagnosis.
+test_path_is_file () {
+	if ! [ -f "$1" ]
+	then
+		echo "File $1 doesn't exist. $*"
+		false
+	fi
+}
+
+test_path_is_dir () {
+	if ! [ -d "$1" ]
+	then
+		echo "Directory $1 doesn't exist. $*"
+		false
+	fi
+}
+
+test_path_is_missing () {
+	if [ -e "$1" ]
+	then
+		echo "Path exists:"
+		ls -ld "$1"
+		if [ $# -ge 1 ]; then
+			echo "$*"
+		fi
+		false
+	fi
+}
+
+
 # This is not among top-level (test_expect_success | test_expect_failure)
 # but is a prefix that can be used in the test script, like:
 #
-- 
1.7.2.1.52.g95e25.dirty

Re: [PATCH] test-lib: user-friendly alternatives to test [-d|-f|-e]

[Joshua Juran]

On Aug 10, 2010, at 4:00 AM, Matthieu Moy wrote:

> +# debugging-friendly alternatives to "test [-f|-d|-e]"
> +# The commands test the existence or non-existance of $1. $2 can be

s/existance/existence/

Josh

[PATCH v3] test-lib: user-friendly alternatives to test [-d|-f|-e]

[Matthieu Moy]

The helper functions are implemented, documented, and used in a few
places to validate them, but not everywhere to avoid useless code churn.

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
---
With Junio's comment and a typo noticed by Joshua Juran.

 t/README                      |    7 +++++++
 t/t3404-rebase-interactive.sh |   18 +++++++++---------
 t/t3407-rebase-abort.sh       |    6 +++---
 t/test-lib.sh                 |   32 ++++++++++++++++++++++++++++++++
 4 files changed, 51 insertions(+), 12 deletions(-)

diff --git a/t/README b/t/README
index 0d1183c..410499a 100644
--- a/t/README
+++ b/t/README
@@ -467,6 +467,13 @@ library for your script to use.
    <expected> file.  This behaves like "cmp" but produces more
    helpful output when the test is run with "-v" option.
 
+ - test_path_is_file <file> [<diagnosis>]
+   test_path_is_dir <dir> [<diagnosis>]
+   test_path_is_missing <path> [<diagnosis>]
+
+   Check whether a file/directory exists or doesn't. <diagnosis> will
+   be displayed if the test fails.
+
  - test_when_finished <script>
 
    Prepend <script> to a list of commands to run to clean up
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index 7b0026e..f78c364 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -69,18 +69,18 @@ test_expect_success 'rebase -i with the exec command' '
 	FAKE_LINES="1 exec_touch_touch-one 2 exec_touch_touch-two exec_false exec_touch_touch-three 3 4
 		exec_touch_\"touch-file__name_with_spaces\";_touch_touch-after-semicolon 5" \
 		test_must_fail git rebase -i A &&
-	test -f touch-one &&
-	test -f touch-two &&
-	! test -f touch-three &&
+	test_path_is_file touch-one &&
+	test_path_is_file touch-two &&
+	test_path_is_missing touch-three "(Rebase should have stopped before)" &&
 	test $(git rev-parse C) = $(git rev-parse HEAD) || {
 		echo "Stopped at wrong revision:"
 		echo "($(git describe --tags HEAD) instead of C)"
 		false
 	} &&
 	git rebase --continue &&
-	test -f touch-three &&
-	test -f "touch-file  name with spaces" &&
-	test -f touch-after-semicolon &&
+	test_path_is_file touch-three &&
+	test_path_is_file "touch-file  name with spaces" &&
+	test_path_is_file touch-after-semicolon &&
 	test $(git rev-parse master) = $(git rev-parse HEAD) || {
 		echo "Stopped at wrong revision:"
 		echo "($(git describe --tags HEAD) instead of master)"
@@ -95,7 +95,7 @@ test_expect_success 'rebase -i with the exec command runs from tree root' '
 	FAKE_LINES="1 exec_touch_touch-subdir" \
 		git rebase -i HEAD^ &&
 	cd .. &&
-	test -f touch-subdir &&
+	test_path_is_file touch-subdir &&
 	rm -fr subdir
 '
 
@@ -191,7 +191,7 @@ test_expect_success 'abort' '
 	git rebase --abort &&
 	test $(git rev-parse new-branch1) = $(git rev-parse HEAD) &&
 	test "$(git symbolic-ref -q HEAD)" = "refs/heads/branch1" &&
-	! test -d .git/rebase-merge
+	test_path_is_missing .git/rebase-merge
 '
 
 test_expect_success 'abort with error when new base cannot be checked out' '
@@ -200,7 +200,7 @@ test_expect_success 'abort with error when new base cannot be checked out' '
 	test_must_fail git rebase -i master > output 2>&1 &&
 	grep "Untracked working tree file .file1. would be overwritten" \
 		output &&
-	! test -d .git/rebase-merge &&
+	test_path_is_missing .git/rebase-merge &&
 	git reset --hard HEAD^
 '
 
diff --git a/t/t3407-rebase-abort.sh b/t/t3407-rebase-abort.sh
index 2999e78..fbb3f2e 100755
--- a/t/t3407-rebase-abort.sh
+++ b/t/t3407-rebase-abort.sh
@@ -38,7 +38,7 @@ testrebase() {
 		# Clean up the state from the previous one
 		git reset --hard pre-rebase &&
 		test_must_fail git rebase$type master &&
-		test -d "$dotest" &&
+		test_path_is_dir "$dotest" &&
 		git rebase --abort &&
 		test $(git rev-parse to-rebase) = $(git rev-parse pre-rebase) &&
 		test ! -d "$dotest"
@@ -49,7 +49,7 @@ testrebase() {
 		# Clean up the state from the previous one
 		git reset --hard pre-rebase &&
 		test_must_fail git rebase$type master &&
-		test -d "$dotest" &&
+		test_path_is_dir "$dotest" &&
 		test_must_fail git rebase --skip &&
 		test $(git rev-parse HEAD) = $(git rev-parse master) &&
 		git rebase --abort &&
@@ -62,7 +62,7 @@ testrebase() {
 		# Clean up the state from the previous one
 		git reset --hard pre-rebase &&
 		test_must_fail git rebase$type master &&
-		test -d "$dotest" &&
+		test_path_is_dir "$dotest" &&
 		echo c > a &&
 		echo d >> a &&
 		git add a &&
diff --git a/t/test-lib.sh b/t/test-lib.sh
index e8f21d5..d584194 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -541,6 +541,38 @@ test_external_without_stderr () {
 	fi
 }
 
+# debugging-friendly alternatives to "test [-f|-d|-e]"
+# The commands test the existence or non-existence of $1. $2 can be
+# given to provide a more precise diagnosis.
+test_path_is_file () {
+	if ! [ -f "$1" ]
+	then
+		echo "File $1 doesn't exist. $*"
+		false
+	fi
+}
+
+test_path_is_dir () {
+	if ! [ -d "$1" ]
+	then
+		echo "Directory $1 doesn't exist. $*"
+		false
+	fi
+}
+
+test_path_is_missing () {
+	if [ -e "$1" ]
+	then
+		echo "Path exists:"
+		ls -ld "$1"
+		if [ $# -ge 1 ]; then
+			echo "$*"
+		fi
+		false
+	fi
+}
+
+
 # This is not among top-level (test_expect_success | test_expect_failure)
 # but is a prefix that can be used in the test script, like:
 #
-- 
1.7.2.1.52.g95e25.dirty

Re: [PATCH] rebase -i: add exec command to launch a shell command

[Erik Faye-Lund]

On Thu, Aug 5, 2010 at 6:47 PM, Matthieu Moy
<Matthieu.Moy@grenoble-inp.fr> wrote:
> Ęvar Arnfjörš Bjarmason <avarab@gmail.com> writes:
>
>>> +in `$SHELL`, or the default shell if `$SHELL` is not set), so you can
>>> +use usual shell commands like "cd". The command is run from the
>>
>> I think that needs a definite article: ".. use the usual ..".
>
> I don't think so, especially with a plural "commands" after.
> Googlefight agrees with me ("use the usual commands" => 350 results,
> "use usual commands" => 4900), but that's not a proof.

Being grammatically correct doesn't automatically make a sentence
good. "Use ususal" is a bit of a tongue-twister, so I'd rewrite that
to "use normal" for that purpose alone.

But I'm not a native English speaker, and the natives might disagree with me.

Re: [PATCH] rebase -i: add exec command to launch a shell command

[Jacob Helwig]

On Thu, Aug 5, 2010 at 09:47, Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> wrote:
> Ęvar Arnfjörš Bjarmason <avarab@gmail.com> writes:
>
>>> +in `$SHELL`, or the default shell if `$SHELL` is not set), so you can
>>> +use usual shell commands like "cd". The command is run from the
>>
>> I think that needs a definite article: ".. use the usual ..".
>
> I don't think so, especially with a plural "commands" after.
> Googlefight agrees with me ("use the usual commands" => 350 results,
> "use usual commands" => 4900), but that's not a proof. Perhaps a
> native speaker could help?
>

You could probably just drop "usual" entirely: ..., so you can use
shell commands like "cd".

I'm not sure that "usual" really adds anything there, and might
actually be confusing.  Does it mean that "unusual" ones won't work?
What are "unusual" ones?

Possibly make 'like "cd"', parenthetical to further show that it's an
example, and not saying that only commands along the lines of "cd"
will work?  ..., so you can use shell commands (for example: cd).

-Jacob

Re: [PATCH] rebase -i: add exec command to launch a shell command

[Junio C Hamano]

Jacob Helwig <jacob.helwig@gmail.com> writes:

> On Thu, Aug 5, 2010 at 09:47, Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> wrote:
>> Ęvar Arnfjörš Bjarmason <avarab@gmail.com> writes:
>>
>>>> +in `$SHELL`, or the default shell if `$SHELL` is not set), so you can
>>>> +use usual shell commands like "cd". The command is run from the
>>>
>>> I think that needs a definite article: ".. use the usual ..".
>> ...
> You could probably just drop "usual" entirely: ..., so you can use
> shell commands like "cd".

Sounds sane.  Will do, unless a native speaker stops me from doing so.

Re: [RFC/PATCH] rebase -i: add run command to launch a shell command

[Matthieu Moy]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> On Fri, Jul 30, 2010 at 15:24, Matthieu Moy
> <Matthieu.Moy@grenoble-inp.fr> wrote:
>> Marc Branchaud <marcnarc@xiplink.com> writes:
>>
>>>> The name of the command may be subject to discussions. I've chosen
>>>> "run", but maybe "shell" would be OK too. In both cases, it doesn't
>>>> allow the one-letter version since both "r" and "s" are already used.
>>>
>>> "exec" with one-letter "x"?
>>
>> Thanks, that sounds good, yes. Any other thought?
>
> I like "exec".

Yes, it's the best proposal IMHO. "x" is very often associated to
"execute", and I'd rather keep away from punctuation/shift combo.
There have been discussions in the past about giving "!" a semantics
(like "fixup!" for a variant of "fixup"). Let's keep this as an option
for the future by not using ! now.

> I think the docs need to elaborate on the environment the "exec"
> command gets executed in, what's its current working directory for
> instance? Wherever I happened to run git-rebase from? the project
> root?

That's a good question. My original patch was running the command from
the toplevel, which is the natural way to implement it. I've changed
my mind to execute the command from the place where "git rebase -i"
was started (which means this has to be memorized in a temporary file
to be persistant accross "git rebase --continue"). I think this makes
more sense for the user, and I've actually already been biten by the
old behavior, running "rebase -i" from a doc/ subdirectory, and
wondering why my "exec make" was rebuilding the code itself.

This comes with at least one drawback: if directory from which the
rebase was started didn't exist in the past, then we can't do a simple
"cd" to it. My implementation re-creates the directory temporarily, so
that the command can run, and cleans it up afterwards. The only really
problematic case is when the directory can not be created (like
directory/file conflict). It this case, the command is not ran, and
the script exits.

> your if ! eval .. error message also exits with 0, surely that should
> exit with 1?

I'm now exiting with the same status as the command itself in case of
failure.

New version follows. It should hopefully look more like a real patch
than an RFC now.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

[PATCH] rebase -i: add exec command to launch a shell command

[Matthieu Moy]

The typical usage pattern would be to run a test (or simply a compilation
command) at given points in history.

The shell command is ran (in the directory where the rebase was started
by the user), and the rebase is stopped when the command fails, to give
the user an opportunity to fix the problem before continuing with "git
rebase --continue".

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
---
 Documentation/git-rebase.txt  |   26 +++++++++++++
 git-rebase--interactive.sh    |   77 +++++++++++++++++++++++++++++++++++++++
 git-rebase.sh                 |    6 +++
 t/lib-rebase.sh               |    2 +
 t/t3404-rebase-interactive.sh |   79 +++++++++++++++++++++++++++++++++++++++++
 5 files changed, 190 insertions(+), 0 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index be23ad2..121f873 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -459,6 +459,32 @@ sure that the current HEAD is "B", and call
 $ git rebase -i -p --onto Q O
 -----------------------------
 
+Reordering and editing commits usually creates untested intermediate
+steps.  You may want to check that your history editting did not break
+anything by running a test, or at least recompiling at intermediate
+points in history by using the "exec" command (shortcut "x").  You may
+do so by creating a todo list like this one:
+
+-------------------------------------------
+pick deadbee Implement feature XXX
+fixup f1a5c00 Fix to feature XXX
+exec make
+pick c0ffeee The oneline of the next commit
+edit deadbab The oneline of the commit after
+exec make; make test
+...
+-------------------------------------------
+
+The interactive rebase will stop when a command fails (i.e. exists
+with non-0 status) to give you an opportunity to fix the problem.  You
+can continue with `git rebase --continue`.
+
+The "exec" command launches the command in a shell (the one specified
+in `$SHELL`, or the default shell if `$SHELL` is not set), so you can
+use usual shell commands like "cd". The commands are ran from the same
+directory as "rebase -i" was started (this directory is temporarily
+re-created if needed).
+
 
 SPLITTING COMMITS
 -----------------
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index b94c2a0..98a54e5 100755
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -104,6 +104,10 @@ AMEND="$DOTEST"/amend
 REWRITTEN_LIST="$DOTEST"/rewritten-list
 REWRITTEN_PENDING="$DOTEST"/rewritten-pending
 
+# The "exec" command needs to know from which directory the
+# interactive rebase started.
+START_PREFIX="$DOTEST"/start-prefix
+
 PRESERVE_MERGES=
 STRATEGY=
 ONTO=
@@ -448,6 +452,31 @@ record_in_rewritten() {
 	esac
 }
 
+# Like "mkdir -p", but outputs the largest prefix of "$1" which
+# already exists as a directory
+mkdir_parent_verbose() {
+	dir=$1
+	to_mk=.
+	while ! test -d "$dir"
+	do
+		to_mk=$(basename "$dir")/"$to_mk"
+		dir=$(dirname "$dir")
+	done
+	(cd "$dir" && mkdir -p "$to_mk" && pwd -P)
+}
+
+# Delete directory "$1", and its parent, until encountering a
+# non-empty directory or "$2".
+cleanup_directory() {
+	to_rm="$1"
+	while test -d "$1" && test -d "$2" &&
+		test "$(cd "$to_rm"; pwd -P)" != "$(cd "$2"; pwd -P)" &&
+		rmdir "$to_rm"
+	do
+		to_rm=$(dirname "$to_rm")
+	done
+}
+
 do_next () {
 	rm -f "$MSG" "$AUTHOR_SCRIPT" "$AMEND" || exit
 	read -r command sha1 rest < "$TODO"
@@ -537,6 +566,52 @@ do_next () {
 		esac
 		record_in_rewritten $sha1
 		;;
+	x|"exec")
+		read -r command rest < "$TODO"
+		mark_action_done
+		printf 'Executing: %s\n' "$rest"
+		# "exec" command doesn't take a sha1 in the todo-list.
+		# => can't just use $sha1 here.
+		git rev-parse --verify HEAD > "$DOTEST"/stopped-sha
+		# We run the command from the directory in which "git
+		# rebase -i" was first started.
+		prefix=$(cat "$START_PREFIX")
+		to_rm=
+		old_pwd=$(pwd)
+		cd "$prefix" 2>/dev/null || {
+			# The prefix from which the rebase started
+			# didn't exist in the past.  We create the
+			# directory to let the command run from there,
+			# and remember how to delete it in $to_rm.
+			to_rm=$(mkdir_parent_verbose "$prefix") && cd "$prefix"
+		} || {
+			warn "Cannot create directory $prefix"
+			warn "Aborting execution of command $rest"
+			warn "To continue, run"
+			warn
+			warn "	git rebase --continue"
+			warn
+			exit 1
+		}
+		${SHELL:-@SHELL_PATH@} -c "$rest" # Actual execution
+		status=$?
+		cd "$old_pwd"
+		# Don't leave empty directory around.
+		# Since cleanup_directory never deletes non-empty
+		# directories, this can't lose valuable data.
+		if [ -n "$to_rm" ]; then
+			cleanup_directory "$prefix" "$to_rm"
+		fi
+		if test "$status" -ne 0
+		then
+			warn "Execution failed: $rest"
+			warn "You can fix the problem, and then run"
+			warn
+			warn "	git rebase --continue"
+			warn
+			exit "$status"
+		fi
+		;;
 	*)
 		warn "Unknown command: $command $sha1 $rest"
 		if git rev-parse --verify -q "$sha1" >/dev/null
@@ -851,6 +926,7 @@ first and then run 'git rebase --continue' again."
 		echo $ONTO > "$DOTEST"/onto
 		test -z "$STRATEGY" || echo "$STRATEGY" > "$DOTEST"/strategy
 		test t = "$VERBOSE" && : > "$DOTEST"/verbose
+		echo "$GIT_PREFIX" > "$START_PREFIX"
 		if test t = "$PRESERVE_MERGES"
 		then
 			if test -z "$REBASE_ROOT"
@@ -957,6 +1033,7 @@ first and then run 'git rebase --continue' again."
 #  e, edit = use commit, but stop for amending
 #  s, squash = use commit, but meld into previous commit
 #  f, fixup = like "squash", but discard this commit's log message
+#  x <cmd>, exec <cmd> = Run a shell command <cmd>, and stop if it fails
 #
 # If you remove a line here THAT COMMIT WILL BE LOST.
 # However, if you remove everything, the rebase will be aborted.
diff --git a/git-rebase.sh b/git-rebase.sh
index ab4afa7..fff512c 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -32,6 +32,12 @@ OPTIONS_SPEC=
 . git-sh-setup
 set_reflog_action rebase
 require_work_tree
+
+# Keep the prefix path in an environment variable so that
+# git-rebase--interactive can find it.
+GIT_PREFIX=$(git rev-parse --show-prefix)
+export GIT_PREFIX
+
 cd_to_toplevel
 
 LF='
diff --git a/t/lib-rebase.sh b/t/lib-rebase.sh
index 6aefe27..6ccf797 100644
--- a/t/lib-rebase.sh
+++ b/t/lib-rebase.sh
@@ -47,6 +47,8 @@ for line in $FAKE_LINES; do
 	case $line in
 	squash|fixup|edit|reword)
 		action="$line";;
+	exec*)
+		echo "$line" | sed 's/_/ /g' >> "$1";;
 	"#")
 		echo '# comment' >> "$1";;
 	">")
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index 9f03ce6..649a400 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -64,6 +64,85 @@ test_expect_success 'setup' '
 	done
 '
 
+file_must_exist () {
+    if ! [ -f "$1" ]; then
+	echo "file $1 not created."
+	false
+    fi
+}
+
+file_must_not_exist () {
+    if [ -f "$1" ]; then
+	echo "file $1 created while it shouldn't have. $2"
+	false
+    fi
+}
+
+dir_must_exist () {
+    if ! [ -d "$1" ]; then
+	echo "dir $1 does not exist."
+	false
+    fi
+}
+
+dir_must_not_exist () {
+    if [ -d "$1" ]; then
+	echo "dir $1 exists while it shouldn't. $2"
+	false
+    fi
+}
+
+test_expect_success 'rebase -i with the exec command' '
+	git checkout master &&
+	FAKE_LINES="1 exec_touch_touch-one 2 exec_touch_touch-two exec_false exec_touch_touch-three 3 4
+		exec_touch_\"touch-file__name_with_spaces\";_touch_touch-after-semicolon 5" \
+		test_must_fail git rebase -i A &&
+	file_must_exist touch-one &&
+	file_must_exist touch-two &&
+	file_must_not_exist touch-three "(Should have stopped before)" &&
+	test $(git rev-parse C) = $(git rev-parse HEAD) || {
+		echo "Stopped at wrong revision:"
+		echo "($(git describe --tags HEAD) instead of C)"
+		false
+	} &&
+	git rebase --continue &&
+	file_must_exist touch-three &&
+	file_must_exist "touch-file  name with spaces" &&
+	file_must_exist touch-after-semicolon &&
+	test $(git rev-parse master) = $(git rev-parse HEAD) || {
+		echo "Stopped at wrong revision:"
+		echo "($(git describe --tags HEAD) instead of master)"
+		false
+	} &&
+	rm -f touch-*
+'
+
+test_expect_success 'rebase -i with exec in deleted directory' '
+	git checkout master &&
+	rm -f pwd.txt &&
+	mkdir tmpdir &&
+	cd tmpdir &&
+	pwd > ../expect &&
+	FAKE_LINES="1 exec_false_one
+		      exec_pwd>../actual exec_false_two
+		      exec_touch_tmpfile exec_false_three
+		      exec_pwd" \
+		test_must_fail git rebase -i HEAD^ &&
+	cd .. && rm -fr tmpdir &&
+	# Should re-create tmpdir temporarily, and clean it up:
+	test_must_fail git rebase --continue &&
+	test_cmp expect actual &&
+	dir_must_not_exist tmpdir &&
+	# Should not cleanup non-empty directory
+	test_must_fail git rebase --continue &&
+	file_must_exist tmpdir/tmpfile &&
+	rm -fr tmpdir && touch tmpdir &&
+	# Should fail because of D/F conflict
+	test_must_fail git rebase --continue &&
+	git rebase --continue &&
+	rm -fr tmpdir
+'
+
 test_expect_success 'no changes are a nop' '
 	git checkout branch2 &&
 	git rebase -i F &&
-- 
1.7.2.1.10.g5cb67a

Re: [PATCH] rebase -i: add exec command to launch a shell command

[Jared Hance]

On Mon, Aug 02, 2010 at 12:03:53PM +0200, Matthieu Moy wrote:
> The shell command is ran (in the directory where the rebase was started
> by the user)

I disagree with how intuitive this behavior is. "rebase" is a
repository level command - It modifies the objects in the repository.
As a side effect, the working tree attached to a (non-bare) repository
is changed, even outside of where the rebase was ran.

Since the rebase affects the entire repository, I think it makes sense
for the commands to be run out of the repository root.

Perhaps a command-line option for toggling this behavior would be a
good compromise.

Re: [PATCH] rebase -i: add exec command to launch a shell command

[Ævar Arnfjörð Bjarmason]

On Mon, Aug 2, 2010 at 10:03, Matthieu Moy <Matthieu.Moy@imag.fr> wrote:

> +use usual shell commands like "cd". The commands are ran from the same

"are run from the same"

> +directory as "rebase -i" was started (this directory is temporarily
> +re-created if needed).

"was started in"

> +file_must_exist () {
> +    if ! [ -f "$1" ]; then
> +       echo "file $1 not created."
> +       false
> +    fi
> +}
> +
> +file_must_not_exist () {
> +    if [ -f "$1" ]; then
> +       echo "file $1 created while it shouldn't have. $2"
> +       false
> +    fi
> +}
> +
> +dir_must_exist () {
> +    if ! [ -d "$1" ]; then
> +       echo "dir $1 does not exist."
> +       false
> +    fi
> +}
> +
> +dir_must_not_exist () {
> +    if [ -d "$1" ]; then
> +       echo "dir $1 exists while it shouldn't. $2"
> +       false
> +    fi
> +}

Leftover debugging code? Looks useful, but probably something we
should have in test-lib.sh as "git_test" as a wrapper for the "test"
built-in.

Re: [PATCH] rebase -i: add exec command to launch a shell command

[Johannes Sixt]

On Montag, 2. August 2010, Matthieu Moy wrote:
> The typical usage pattern would be to run a test (or simply a compilation
> command) at given points in history.

What happens if the command modifies the worktree and/or the index?

-- Hannes

Re: [PATCH] rebase -i: add exec command to launch a shell command

[Matthieu Moy]

Johannes Sixt <j6t@kdbg.org> writes:

> On Montag, 2. August 2010, Matthieu Moy wrote:
>> The typical usage pattern would be to run a test (or simply a compilation
>> command) at given points in history.
>
> What happens if the command modifies the worktree and/or the index?

Something terrible :-(. The next patch application fails, and the
patch is more or less silently dropped (I thought it would fail giving
the user an opportunity to fix and re-apply, but it doesn't).

New version checks working tree cleanness right after the command
finishes, and stops cleanly, in time (restart with "git rebase
--continue"). This comes with a new test.

(But the command can possibly create a new commit or amend the
previous one without problem)

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/