[PATCH] test-lib-functions: simplify `test_file_not_empty` failure message

[PATCH] test-lib-functions: simplify `test_file_not_empty` failure message

[Eric Sunshine]

From: Eric Sunshine <sunshine@sunshineco.com>

The function `test_file_not_empty` asserts that a file exists and is not
empty. When the assertion fails, it complains:

    'foo' is not a non-empty file.

which is difficult to interpret due to the double-negative. To make it
easier to understand the problem, simplify the message by dropping the
double-negative and stating the problem more directly:

    'foo' is empty but should not be

(The full-stop is also dropped from the message to reflect the style of
messages issued by other `test_path_*` functions.)

Note: Technically, the revised message is slightly less accurate since
the function asserts both that the file exists and that it is non-empty,
but the new message talks only about the emptiness of the file, not
whether it exists. A more accurate message might be "'foo' is empty but
should not be (or doesn't exist)", but that's unnecessarily long-winded
and adds little information that the test author couldn't discover by
noticing the file's absence.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
---

This is a tangential follow-up to the discussion at [1].

[1]: https://lore.kernel.org/git/CAPig+cQ+JNBwydUq0CsTZGs8mHs3L3fJDuSosd+-WdKwWWw=gg@mail.gmail.com/

 t/test-lib-functions.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index b5eaf7fdc1..9e97b324c5 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -991,7 +991,7 @@ test_file_not_empty () {
 	test "$#" = 2 && BUG "2 param"
 	if ! test -s "$1"
 	then
-		echo "'$1' is not a non-empty file."
+		echo "'$1' is empty but should not be"
 		false
 	fi
 }
-- 
2.44.0

Re: [PATCH] test-lib-functions: simplify `test_file_not_empty` failure message

[Junio C Hamano]

Eric Sunshine <ericsunshine@charter.net> writes:

> Note: Technically, the revised message is slightly less accurate since
> the function asserts both that the file exists and that it is non-empty,
> but the new message talks only about the emptiness of the file, not
> whether it exists.
>
> A more accurate message might be "'foo' is empty but
> should not be (or doesn't exist)", but that's unnecessarily long-winded
> and adds little information that the test author couldn't discover by
> noticing the file's absence.

Besides, that is way too confusing.  "<foo> is empty or it does not
exist" I may understand, but with your construct, I wouldn't be able
to tell how I am supposed to interpret the "(or doesn't exist)"
part.

> diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
> index b5eaf7fdc1..9e97b324c5 100644
> --- a/t/test-lib-functions.sh
> +++ b/t/test-lib-functions.sh
> @@ -991,7 +991,7 @@ test_file_not_empty () {
>  	test "$#" = 2 && BUG "2 param"
>  	if ! test -s "$1"
>  	then
> -		echo "'$1' is not a non-empty file."
> +		echo "'$1' is empty but should not be"

The "adds little information" version may be

		echo "'$1' is either missing or empty, but should not be"

And avoiding "X is Y, but should  be ~Y" construct, perhaps

		echo "'$1' should be a file with non-empty contents"

would work better?  I dunno.

Re: [PATCH] test-lib-functions: simplify `test_file_not_empty` failure message

[Eric Sunshine]

On Fri, Mar 1, 2024 at 5:11 PM Junio C Hamano <gitster@pobox.com> wrote:
> Eric Sunshine <ericsunshine@charter.net> writes:
> > A more accurate message might be "'foo' is empty but
> > should not be (or doesn't exist)", but that's unnecessarily long-winded
> > and adds little information that the test author couldn't discover by
> > noticing the file's absence.
>
> The "adds little information" version may be
>
>                 echo "'$1' is either missing or empty, but should not be"
>
> And avoiding "X is Y, but should  be ~Y" construct, perhaps
>
>                 echo "'$1' should be a file with non-empty contents"
>
> would work better?  I dunno.

I find "'$1' is either missing or empty, but should not be" suggestion
clear and easily understood. I'll reroll with that.

Re: [PATCH] test-lib-functions: simplify `test_file_not_empty` failure message

[Dirk Gouders]

Eric Sunshine <sunshine@sunshineco.com> writes:

> On Fri, Mar 1, 2024 at 5:11 PM Junio C Hamano <gitster@pobox.com> wrote:
>> Eric Sunshine <ericsunshine@charter.net> writes:
>> > A more accurate message might be "'foo' is empty but
>> > should not be (or doesn't exist)", but that's unnecessarily long-winded
>> > and adds little information that the test author couldn't discover by
>> > noticing the file's absence.
>>
>> The "adds little information" version may be
>>
>>                 echo "'$1' is either missing or empty, but should not be"
>>
>> And avoiding "X is Y, but should  be ~Y" construct, perhaps
>>
>>                 echo "'$1' should be a file with non-empty contents"
>>
>> would work better?  I dunno.
>
> I find "'$1' is either missing or empty, but should not be" suggestion
> clear and easily understood. I'll reroll with that.

This is a view from a position with more distance:

I find that not so easily understood -- the "but should not
be" part is rather unexpected and I feel, it doesn't provide necessary
information, e.g.:

test_path_is_executable () {
...
		echo "$1 is not executable"
...

also doesn't state what is wanted and I doubt that message doesn't
clearly describe the problem.

While I looked at it: there is another `test -s` in test_grep () that
perhaps could be fixed the same way:

	if test -s "$last_arg"
	then
		cat >&4 "$last_arg"
	else
		echo >&4 "<File '$last_arg' is empty>"
	fi

Dirk

Re: [PATCH] test-lib-functions: simplify `test_file_not_empty` failure message

[Rubén Justo]

On Fri, Mar 01, 2024 at 03:49:22PM -0500, Eric Sunshine wrote:
> From: Eric Sunshine <sunshine@sunshineco.com>
> 
> The function `test_file_not_empty` asserts that a file exists and is not
> empty. When the assertion fails, it complains:
> 
>     'foo' is not a non-empty file.
> 
> which is difficult to interpret due to the double-negative. To make it
> easier to understand the problem, simplify the message by dropping the
> double-negative and stating the problem more directly:
> 
>     'foo' is empty but should not be
> 
> (The full-stop is also dropped from the message to reflect the style of
> messages issued by other `test_path_*` functions.)
> 
> Note: Technically, the revised message is slightly less accurate since
> the function asserts both that the file exists and that it is non-empty,
> but the new message talks only about the emptiness of the file, not
> whether it exists. A more accurate message might be "'foo' is empty but
> should not be (or doesn't exist)", but that's unnecessarily long-winded
> and adds little information that the test author couldn't discover by
> noticing the file's absence.

To improve the accuracy of the message, I wonder if it is worth doing
what we do in test_must_be_empty:

	test_must_be_empty () {
		test "$#" -ne 1 && BUG "1 param"
		test_path_is_file "$1" &&
		if test -s "$1"
		then
			echo "'$1' is not empty, it contains:"
			cat "$1"
			return 1
		fi
	}

Perhaps:

diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index b5eaf7fdc1..5b5ee0dc1d 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -989,9 +989,10 @@ test_dir_is_empty () {
 # Check if the file exists and has a size greater than zero
 test_file_not_empty () {
        test "$#" = 2 && BUG "2 param"
+       test_path_is_file "$1" &&
        if ! test -s "$1"
        then
-               echo "'$1' is not a non-empty file."
+		echo "'$1' is empty but should not be"
                false
        fi
 }

> 
> Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
> ---
> 
> This is a tangential follow-up to the discussion at [1].
> 
> [1]: https://lore.kernel.org/git/CAPig+cQ+JNBwydUq0CsTZGs8mHs3L3fJDuSosd+-WdKwWWw=gg@mail.gmail.com/
> 
>  t/test-lib-functions.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
> index b5eaf7fdc1..9e97b324c5 100644
> --- a/t/test-lib-functions.sh
> +++ b/t/test-lib-functions.sh
> @@ -991,7 +991,7 @@ test_file_not_empty () {
>  	test "$#" = 2 && BUG "2 param"
>  	if ! test -s "$1"
>  	then
> -		echo "'$1' is not a non-empty file."
> +		echo "'$1' is empty but should not be"
>  		false
>  	fi
>  }
> -- 
> 2.44.0
> 

Re: [PATCH] test-lib-functions: simplify `test_file_not_empty` failure message

[Junio C Hamano]

Dirk Gouders <dirk@gouders.net> writes:

> Eric Sunshine <sunshine@sunshineco.com> writes:
>
>> On Fri, Mar 1, 2024 at 5:11 PM Junio C Hamano <gitster@pobox.com> wrote:
>>> Eric Sunshine <ericsunshine@charter.net> writes:
>>> > A more accurate message might be "'foo' is empty but
>>> > should not be (or doesn't exist)", but that's unnecessarily long-winded
>>> > and adds little information that the test author couldn't discover by
>>> > noticing the file's absence.
>>>
>>> The "adds little information" version may be
>>>
>>>                 echo "'$1' is either missing or empty, but should not be"
>>> ...
>> I find "'$1' is either missing or empty, but should not be" suggestion
>> clear and easily understood. I'll reroll with that.
>
> This is a view from a position with more distance:
>
> I find that not so easily understood -- the "but should not
> be" part is rather unexpected and I feel, it doesn't provide necessary
> information, e.g.:
>
> test_path_is_executable () {
> ...
> 		echo "$1 is not executable"
> ...
>
> also doesn't state what is wanted and I doubt that message doesn't
> clearly describe the problem.

I cannot tell if you really meant the double negative involving
"doubt", but assuming you did, you are saying that

    With "X is not Y", it is clear enough that we expect X to be Y
    (if it were not clear to somebody who read "X is not Y" that we
    want X to be Y, then "X is not Y, but it should be" may needed,
    but "X is not Y" is clear enough).

So you think "$1 is either missing or empty" is better without "but
should not be" added to the end?  Am I reading you correctly?

I think this takes us back pretty much to square one ;-) but that is
also fine.

But the above argument depends on an untold assumption.  The message
"X is not Y" must be clearly understood as a complaint, not a mere
statement of a fact.  I am not sure if that is the case.

Instead of "X is not Y, but it should be", the way to clarify these
messages may be to say "error: X is not Y", perhaps?

> While I looked at it: there is another `test -s` in test_grep () that
> perhaps could be fixed the same way:
>
> 	if test -s "$last_arg"
> 	then
> 		cat >&4 "$last_arg"
> 	else
> 		echo >&4 "<File '$last_arg' is empty>"
> 	fi

If you are worried about "test -s" failing because "$last_arg" does
not exist, then you are worried too much.  We upfront guard the
test_grep() helper with "test -f" of the same file and diagnoses the
lack of the file as a bug in the test.  And we do not assume gremlins
removing random files while we are running tests.

Re: [PATCH] test-lib-functions: simplify `test_file_not_empty` failure message

[Junio C Hamano]

Rubén Justo <rjusto@gmail.com> writes:

> To improve the accuracy of the message, I wonder if it is worth doing
> ...
> diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
> index b5eaf7fdc1..5b5ee0dc1d 100644
> --- a/t/test-lib-functions.sh
> +++ b/t/test-lib-functions.sh
> @@ -989,9 +989,10 @@ test_dir_is_empty () {
>  # Check if the file exists and has a size greater than zero
>  test_file_not_empty () {
>         test "$#" = 2 && BUG "2 param"
> +       test_path_is_file "$1" &&
>         if ! test -s "$1"
>         then
> -               echo "'$1' is not a non-empty file."
> +		echo "'$1' is empty but should not be"
>                 false
>         fi
>  }

Simple and effective to remove the need to have to worry about the
"missing" case.  The "but should not be" part may still be subject
to discussion, but I do not have a strong opinion there.

Re: [PATCH] test-lib-functions: simplify `test_file_not_empty` failure message

[Dirk Gouders]

Junio C Hamano <gitster@pobox.com> writes:

> Dirk Gouders <dirk@gouders.net> writes:
>
>> Eric Sunshine <sunshine@sunshineco.com> writes:
>>
>>> On Fri, Mar 1, 2024 at 5:11 PM Junio C Hamano <gitster@pobox.com> wrote:
>>>> Eric Sunshine <ericsunshine@charter.net> writes:
>>>> > A more accurate message might be "'foo' is empty but
>>>> > should not be (or doesn't exist)", but that's unnecessarily long-winded
>>>> > and adds little information that the test author couldn't discover by
>>>> > noticing the file's absence.
>>>>
>>>> The "adds little information" version may be
>>>>
>>>>                 echo "'$1' is either missing or empty, but should not be"
>>>> ...
>>> I find "'$1' is either missing or empty, but should not be" suggestion
>>> clear and easily understood. I'll reroll with that.
>>
>> This is a view from a position with more distance:
>>
>> I find that not so easily understood -- the "but should not
>> be" part is rather unexpected and I feel, it doesn't provide necessary
>> information, e.g.:
>>
>> test_path_is_executable () {
>> ...
>> 		echo "$1 is not executable"
>> ...
>>
>> also doesn't state what is wanted and I doubt that message doesn't
>> clearly describe the problem.
>
> I cannot tell if you really meant the double negative involving
> "doubt", but assuming you did, you are saying that

I'm sorry about that double negative which was probably wrong wording of
a non-native speaker.

>     With "X is not Y", it is clear enough that we expect X to be Y
>     (if it were not clear to somebody who read "X is not Y" that we
>     want X to be Y, then "X is not Y, but it should be" may needed,
>     but "X is not Y" is clear enough).
>
> So you think "$1 is either missing or empty" is better without "but
> should not be" added to the end?  Am I reading you correctly?
>
> I think this takes us back pretty much to square one ;-) but that is
> also fine.
>
> But the above argument depends on an untold assumption.  The message
> "X is not Y" must be clearly understood as a complaint, not a mere
> statement of a fact.  I am not sure if that is the case.
>
> Instead of "X is not Y, but it should be", the way to clarify these
> messages may be to say "error: X is not Y", perhaps?

That is exactly what came to my mind when I was later re-thinking
what I had written.

>> While I looked at it: there is another `test -s` in test_grep () that
>> perhaps could be fixed the same way:
>>
>> 	if test -s "$last_arg"
>> 	then
>> 		cat >&4 "$last_arg"
>> 	else
>> 		echo >&4 "<File '$last_arg' is empty>"
>> 	fi
>
> If you are worried about "test -s" failing because "$last_arg" does
> not exist, then you are worried too much.  We upfront guard the
> test_grep() helper with "test -f" of the same file and diagnoses the
> lack of the file as a bug in the test.  And we do not assume gremlins
> removing random files while we are running tests.

Yes, thank you for clarification and sorry for the noise.

Dirk