[PATCH v2] t9402: sed -i is not portable

[PATCH v2] t9402: sed -i is not portable

[Torsten Bögershausen]

On some systems sed allows the usage of e.g.
sed -i -e "s/line1/line2/" afile
to edit the file "in place".
Other systems don't allow that: one observed behaviour is that
sed -i -e "s/line1/line2/" afile
creates a backup file called afile-e, which breaks the test.
As sed -i is not part of POSIX, avoid it.

Use test_cmp, makes the test easier to debug.
Chain all shell commands with && to detect all kinds of failure.

Signed-off-by: Torsten Bögershausen <tboegi@web.de>
---
Changes since v1:
No correction of TABs to make it easier to review

If this is OK:
Matthew would you like to send a complete re-roll,
because the credit should be on you ?

 t/t9402-git-cvsserver-refs.sh | 44 ++++++++++++++++++++++---------------------
 1 file changed, 23 insertions(+), 21 deletions(-)

diff --git a/t/t9402-git-cvsserver-refs.sh b/t/t9402-git-cvsserver-refs.sh
index 858ef0f..5138f14 100755
--- a/t/t9402-git-cvsserver-refs.sh
+++ b/t/t9402-git-cvsserver-refs.sh
@@ -28,27 +28,26 @@ check_file() {
 }
 
 check_end_tree() {
-    sandbox="$1"
-    expectCount=$(wc -l < "$WORKDIR/check.list")
-    cvsCount=$(find "$sandbox" -name CVS -prune -o -type f -print | wc -l)
-    test x"$cvsCount" = x"$expectCount"
-    stat=$?
-    echo "check_end $sandbox : $stat cvs=$cvsCount expect=$expectCount" \
-	>> "$WORKDIR/check.log"
-    return $stat
+    sandbox="$1" &&
+    wc -l < "$WORKDIR/check.list" > expected &&
+    find "$sandbox" -type f | grep -v "/CVS" > "$WORKDIR/check.cvsCount" &&
+    wc -l < "$WORKDIR/check.cvsCount" >actual &&
+    test_cmp expected actual &&
+		rm expected actual &&
+		sort < "$WORKDIR/check.list" > expected &&
+		sort < "$WORKDIR/check.cvsCount" | sed -e "s%cvswork/%%" >actual &&
+    test_cmp expected actual &&
+		rm expected actual
 }
 
 check_end_full_tree() {
-    sandbox="$1"
-    ver="$2"
-    expectCount=$(wc -l < "$WORKDIR/check.list")
-    cvsCount=$(find "$sandbox" -name CVS -prune -o -type f -print | wc -l)
-    gitCount=$(git ls-tree -r "$2" | wc -l)
-    test x"$cvsCount" = x"$expectCount" -a x"$gitCount" = x"$expectCount"
-    stat=$?
-    echo "check_end $sandbox : $stat cvs=$cvsCount git=$gitCount expect=$expectCount" \
-	>> "$WORKDIR/check.log"
-    return $stat
+    sandbox="$1" &&
+    sort < "$WORKDIR/check.list" >expected &&
+    find "$sandbox" -name CVS -prune -o -type f -print | sed -e "s%$sandbox/%%" | sort >act1 &&
+		test_cmp expected act1 &&
+    git ls-tree -r "$2" | sed -e "s/^.*blob [0-9a-fA-F]*[	 ]*//" | sort > act2 &&
+		test_cmp expected act2 &&
+    rm expected act1 act2
 }
 
 #########
@@ -155,7 +154,8 @@ test_expect_success 'cvs co b1 [cvswork3]' '
 
 test_expect_success 'edit cvswork3 and save diff' '
     ( cd cvswork3 &&
-      sed -i -e "s/line1/line1 - data/" adir/afile &&
+      sed -e "s/line1/line1 - data/" adir/afile >adir/afileNEW &&
+			mv -f adir/afileNEW adir/afile &&
       echo "afile5" > adir/afile5 &&
       rm t2 &&
       cvs -f add adir/afile5 &&
@@ -168,7 +168,8 @@ test_expect_success 'setup v1.2 on b1' '
     git checkout b1 &&
     echo "new v1.2" > t3 &&
     rm t2 &&
-    sed -i -e "s/line3/line3 - more data/" adir/afile &&
+    sed -e "s/line3/line3 - more data/" adir/afile >adir/afileNEW &&
+		mv -f adir/afileNEW adir/afile &&
     rm adir/a2file &&
     echo "a3file" >> adir/a3file &&
     echo "bfile line 3" >> adir/bdir/bfile &&
@@ -300,7 +301,8 @@ test_expect_success 'root dir rm file [cvswork2]' '
 
 test_expect_success 'subdir edit/add/rm files [cvswork2' '
     ( cd cvswork2 &&
-      sed -i -e "s/line 1/line 1 (v2)/" adir/bdir/bfile &&
+      sed -e "s/line 1/line 1 (v2)/" adir/bdir/bfile >adir/bdir/bfileNEW &&
+      mv -f adir/bdir/bfileNEW adir/bdir/bfile &&
       rm adir/bdir/b2file &&
       cd adir &&
       cvs -f rm bdir/b2file &&
-- 
1.8.0.197.g5a90748

Re: [PATCH v2] t9402: sed -i is not portable

[Junio C Hamano]

Torsten Bögershausen <tboegi@web.de> writes:

> On some systems sed allows the usage of e.g.
> sed -i -e "s/line1/line2/" afile
> to edit the file "in place".
> Other systems don't allow that: one observed behaviour is that
> sed -i -e "s/line1/line2/" afile
> creates a backup file called afile-e, which breaks the test.
> As sed -i is not part of POSIX, avoid it.

Thanks.

The intention is good, but see comments on this part in the patch
below.

> Use test_cmp, makes the test easier to debug.

I see a few other remaining issues in the code after this patch is
applied.  If we are doing other fixes like these, we may want to fix
them as well:

 - test_must_fail is used to run cvs; it shouldn't (instead of
   saying "test_must_fail cvs ...", just say "! cvs ...").

 - A shell function should begin like this: "shellfunc () {"

 - Redirection should not have SP before the filename (i.e. ">out",
   not "> out").

 - It is OK to spell single-liner subshell like this:

    ( cd cvswork3 && ! cvs -f diff -N -u ) &&
    ...

   but a multi-line subshell should start its first command on a
   separate line, like this:

    (
        cd cvswork3 &&
        cvs update foo
    ) &&
    ...

 - quite a few "[cvswork$n]" comments are truncated to "[cvswork$n"
   in the test titles.

> Chain all shell commands with && to detect all kinds of failure.

I think you missed the one in 'merge early [cvswork3] b3 with b1'
where a merge is done in a subshell; a failed merge is not detected.

This is another reason why I asked you not to mix different things
in a single patch; if this patch is rerolled to cover missing "&&"
that you missed, it needs to be re-reviewed for the "sed -i" fix we
review during this round again.

> Signed-off-by: Torsten Bögershausen <tboegi@web.de>
> ---
> Changes since v1:
> No correction of TABs to make it easier to review
>
> If this is OK:
> Matthew would you like to send a complete re-roll,
> because the credit should be on you ?
>
>  t/t9402-git-cvsserver-refs.sh | 44 ++++++++++++++++++++++---------------------
>  1 file changed, 23 insertions(+), 21 deletions(-)
>
> diff --git a/t/t9402-git-cvsserver-refs.sh b/t/t9402-git-cvsserver-refs.sh
> index 858ef0f..5138f14 100755
> --- a/t/t9402-git-cvsserver-refs.sh
> +++ b/t/t9402-git-cvsserver-refs.sh
> @@ -28,27 +28,26 @@ check_file() {
>  }
>  
>  check_end_tree() {
> -    sandbox="$1"
> -    expectCount=$(wc -l < "$WORKDIR/check.list")
> -    cvsCount=$(find "$sandbox" -name CVS -prune -o -type f -print | wc -l)
> -    test x"$cvsCount" = x"$expectCount"
> -    stat=$?
> -    echo "check_end $sandbox : $stat cvs=$cvsCount expect=$expectCount" \
> -	>> "$WORKDIR/check.log"
> -    return $stat
> +    sandbox="$1" &&
> +    wc -l < "$WORKDIR/check.list" > expected &&
> +    find "$sandbox" -type f | grep -v "/CVS" > "$WORKDIR/check.cvsCount" &&
> +    wc -l < "$WORKDIR/check.cvsCount" >actual &&

You replaced the original "find" that was perfectly correct with
"pipe to grep -v" which is more expensive.  Why?

The file stores the lines to be counted; why is it called a "Count"?

As "check.list" is really the "expected list of files", it may want
to be renamed to "list.expected" or something like that.  Then the
output from this "find" would naturally become "list.actual", i.e.

	find "$sandbox" -name CVS -type d -prune \
            -o -type f -print >"$WORKDIR/list.actual"

Even though you are comparing only the number of lines in the file
in this function, are they expected to name the same set of paths?
The other function check_end_full_tree used to compare only the
count, but you changed it to compare the actul names of paths, which
may be a more robust and correct test, so it might make more sense
to redefine $WORKDIR/list.{expect,actual} to be a sorted list of
paths relative to the top of the working tree and do something like
this:

	(
        	cd "$sandbox" &&
	        find . -name CVS -type d -prune -o -type f -print
	) | sed -e "s|^./||" | sort >"$WORKDIR/list.actual"

and make sure "$WORKDIR/list.expect" is also sorted.  Then you can
lose the "wc -l" based check from here and compare the list of
paths.

> +    test_cmp expected actual &&
> +		rm expected actual &&
> +		sort < "$WORKDIR/check.list" > expected &&
> +		sort < "$WORKDIR/check.cvsCount" | sed -e "s%cvswork/%%" >actual &&

It is probably easier for a later patch to fix the indentation, if
these new lines followed the existing indentation.

> +    test_cmp expected actual &&
> +		rm expected actual
>  }
>  
>  check_end_full_tree() {
> -    sandbox="$1"
> -    ver="$2"
> -    expectCount=$(wc -l < "$WORKDIR/check.list")
> -    cvsCount=$(find "$sandbox" -name CVS -prune -o -type f -print | wc -l)
> -    gitCount=$(git ls-tree -r "$2" | wc -l)
> -    test x"$cvsCount" = x"$expectCount" -a x"$gitCount" = x"$expectCount"
> -    stat=$?
> -    echo "check_end $sandbox : $stat cvs=$cvsCount git=$gitCount expect=$expectCount" \
> -	>> "$WORKDIR/check.log"
> -    return $stat
> +    sandbox="$1" &&
> +    sort < "$WORKDIR/check.list" >expected &&
> +    find "$sandbox" -name CVS -prune -o -type f -print | sed -e "s%$sandbox/%%" | sort >act1 &&
> +		test_cmp expected act1 &&
> +    git ls-tree -r "$2" | sed -e "s/^.*blob [0-9a-fA-F]*[	 ]*//" | sort > act2 &&

You can say "git ls-tree --name-only -r" and lose the sed.