[PATCH 0/2] make chainlint output more newcomer-friendly

[PATCH 0/2] make chainlint output more newcomer-friendly

[Eric Sunshine]

From: Eric Sunshine <sunshine@sunshineco.com>

For the sake of newcomers to the project, I have several times over the
last couple years thought to update t/README to explain the rather
cryptic and terse problem annotations emitted by chainlint (i.e.
"?!FOO?!"), but I never got around to it. However, a review comment[*] I
posted recently suggesting an update to CodingGuidelines reminded me of
the need to update t/README.

As such, I set about to do so but quickly realized that it would be far
more useful to newcomers for chainlint to emit friendly problem
descriptions rather than expecting users to know to consult t/README to
interpret the existing cryptic annotations. This patch series, which
improves chainlint's output, is the result of that epiphany.

[*]: https://lore.kernel.org/git/CAPig+cQLr+vAzkt8UJNVCeE8osGEcEfFunG36oqxa0k8JamJzQ@mail.gmail.com/

Eric Sunshine (2):
  chainlint: make error messages self-explanatory
  chainlint: reduce annotation noise-factor

 t/chainlint.pl                                | 33 ++++++++++++++-----
 t/chainlint/arithmetic-expansion.expect       |  2 +-
 t/chainlint/block.expect                      |  8 ++---
 t/chainlint/broken-chain.expect               |  2 +-
 t/chainlint/case.expect                       |  4 +--
 t/chainlint/chain-break-false.expect          |  2 +-
 t/chainlint/chained-block.expect              |  2 +-
 t/chainlint/chained-subshell.expect           |  4 +--
 t/chainlint/command-substitution.expect       |  2 +-
 t/chainlint/complex-if-in-cuddled-loop.expect |  2 +-
 t/chainlint/cuddled.expect                    |  4 +--
 t/chainlint/for-loop.expect                   |  8 ++---
 t/chainlint/function.expect                   |  4 +--
 t/chainlint/here-doc-body-indent.expect       |  2 +-
 t/chainlint/here-doc-body-pathological.expect |  4 +--
 t/chainlint/here-doc-body.expect              |  4 +--
 t/chainlint/here-doc-double.expect            |  2 +-
 t/chainlint/here-doc-indent-operator.expect   |  2 +-
 .../here-doc-multi-line-command-subst.expect  |  2 +-
 t/chainlint/here-doc-multi-line-string.expect |  2 +-
 t/chainlint/if-condition-split.expect         |  2 +-
 t/chainlint/if-in-loop.expect                 |  4 +--
 t/chainlint/if-then-else.expect               |  4 +--
 t/chainlint/inline-comment.expect             |  2 +-
 t/chainlint/loop-detect-failure.expect        |  2 +-
 t/chainlint/loop-in-if.expect                 |  8 ++---
 t/chainlint/multi-line-string.expect          |  2 +-
 t/chainlint/negated-one-liner.expect          |  4 +--
 t/chainlint/nested-cuddled-subshell.expect    |  6 ++--
 t/chainlint/nested-here-doc.expect            |  2 +-
 t/chainlint/nested-loop-detect-failure.expect |  6 ++--
 t/chainlint/nested-subshell-comment.expect    |  2 +-
 t/chainlint/nested-subshell.expect            |  2 +-
 t/chainlint/not-heredoc.expect                |  2 +-
 t/chainlint/one-liner-for-loop.expect         |  2 +-
 t/chainlint/one-liner.expect                  |  6 ++--
 t/chainlint/pipe.expect                       |  2 +-
 t/chainlint/semicolon.expect                  | 12 +++----
 t/chainlint/subshell-here-doc.expect          |  2 +-
 t/chainlint/subshell-one-liner.expect         | 10 +++---
 t/chainlint/token-pasting.expect              |  8 ++---
 t/chainlint/unclosed-here-doc-indent.expect   |  2 +-
 t/chainlint/unclosed-here-doc.expect          |  2 +-
 t/chainlint/while-loop.expect                 |  8 ++---
 t/test-lib.sh                                 |  2 +-
 45 files changed, 108 insertions(+), 91 deletions(-)

-- 
2.46.0

[PATCH 1/2] chainlint: make error messages self-explanatory

[Eric Sunshine]

From: Eric Sunshine <sunshine@sunshineco.com>

The annotations emitted by chainlint to indicate detected problems are
overly terse, so much so that developers new to the project -- those who
should most benefit from the linting -- may find them baffling. For
instance, although the author of chainlint and seasoned Git developers
may understand that "?!AMP?!" is an abbreviation of "ampersand" and
indicates a break in the &&-chain, this may not be obvious to newcomers.

Similarly, although the annotation "?!LOOP?!" is understood by project
regulars to indicate a missing `|| return 1` (or `|| exit 1` in a
subshell), newcomers may find it more than a little perplexing. The
"?!LOOP?!" case is particularly serious since it is likely that some
newcomers are unaware that shell loops do not terminate automatically
upon error, and it is more difficult for a newcomer to figure out how to
correct the problem by examining surrounding code since `|| return 1`
appears in test scrips relatively infrequently (compared, for instance,
with &&-chaining).

Address these shortcomings by emitting human-consumable messages which
both explain the problem and give a strong hint about how to correct it.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
---
 t/chainlint.pl                                | 26 ++++++++++++++-----
 t/chainlint/arithmetic-expansion.expect       |  2 +-
 t/chainlint/block.expect                      |  8 +++---
 t/chainlint/broken-chain.expect               |  2 +-
 t/chainlint/case.expect                       |  4 +--
 t/chainlint/chain-break-false.expect          |  2 +-
 t/chainlint/chained-block.expect              |  2 +-
 t/chainlint/chained-subshell.expect           |  4 +--
 t/chainlint/command-substitution.expect       |  2 +-
 t/chainlint/complex-if-in-cuddled-loop.expect |  2 +-
 t/chainlint/cuddled.expect                    |  4 +--
 t/chainlint/for-loop.expect                   |  8 +++---
 t/chainlint/function.expect                   |  4 +--
 t/chainlint/here-doc-body-indent.expect       |  2 +-
 t/chainlint/here-doc-body-pathological.expect |  4 +--
 t/chainlint/here-doc-body.expect              |  4 +--
 t/chainlint/here-doc-double.expect            |  2 +-
 t/chainlint/here-doc-indent-operator.expect   |  2 +-
 .../here-doc-multi-line-command-subst.expect  |  2 +-
 t/chainlint/here-doc-multi-line-string.expect |  2 +-
 t/chainlint/if-condition-split.expect         |  2 +-
 t/chainlint/if-in-loop.expect                 |  4 +--
 t/chainlint/if-then-else.expect               |  4 +--
 t/chainlint/inline-comment.expect             |  2 +-
 t/chainlint/loop-detect-failure.expect        |  2 +-
 t/chainlint/loop-in-if.expect                 |  8 +++---
 t/chainlint/multi-line-string.expect          |  2 +-
 t/chainlint/negated-one-liner.expect          |  4 +--
 t/chainlint/nested-cuddled-subshell.expect    |  6 ++---
 t/chainlint/nested-here-doc.expect            |  2 +-
 t/chainlint/nested-loop-detect-failure.expect |  6 ++---
 t/chainlint/nested-subshell-comment.expect    |  2 +-
 t/chainlint/nested-subshell.expect            |  2 +-
 t/chainlint/not-heredoc.expect                |  2 +-
 t/chainlint/one-liner-for-loop.expect         |  2 +-
 t/chainlint/one-liner.expect                  |  6 ++---
 t/chainlint/pipe.expect                       |  2 +-
 t/chainlint/semicolon.expect                  | 12 ++++-----
 t/chainlint/subshell-here-doc.expect          |  2 +-
 t/chainlint/subshell-one-liner.expect         | 10 +++----
 t/chainlint/token-pasting.expect              |  8 +++---
 t/chainlint/unclosed-here-doc-indent.expect   |  2 +-
 t/chainlint/unclosed-here-doc.expect          |  2 +-
 t/chainlint/while-loop.expect                 |  8 +++---
 44 files changed, 102 insertions(+), 88 deletions(-)

diff --git a/t/chainlint.pl b/t/chainlint.pl
index 5361f23b1d..d79f183dfd 100755
--- a/t/chainlint.pl
+++ b/t/chainlint.pl
@@ -9,7 +9,7 @@
 # Input arguments are pathnames of shell scripts containing test definitions,
 # or globs referencing a collection of scripts. For each problem discovered,
 # the pathname of the script containing the test is printed along with the test
-# name and the test body with a `?!FOO?!` annotation at the location of each
+# name and the test body with a `?!ERR?!` annotation at the location of each
 # detected problem, where "FOO" is a tag such as "AMP" which indicates a broken
 # &&-chain. Returns zero if no problems are discovered, otherwise non-zero.
 
@@ -181,7 +181,7 @@ sub swallow_heredocs {
 			$self->{lineno} += () = $body =~ /\n/sg;
 			next;
 		}
-		push(@{$self->{parser}->{problems}}, ['UNCLOSED-HEREDOC', $tag]);
+		push(@{$self->{parser}->{problems}}, ['HEREDOC', $tag]);
 		$$b =~ /(?:\G|\n).*\z/gc; # consume rest of input
 		my $body = substr($$b, $start, pos($$b) - $start);
 		$self->{lineno} += () = $body =~ /\n/sg;
@@ -238,6 +238,7 @@ sub new {
 		stop => [],
 		output => [],
 		heredocs => {},
+		insubshell => 0,
 	} => $class;
 	$self->{lexer} = Lexer->new($self, $s);
 	return $self;
@@ -296,8 +297,11 @@ sub parse_group {
 
 sub parse_subshell {
 	my $self = shift @_;
-	return ($self->parse(qr/^\)$/),
-		$self->expect(')'));
+	$self->{insubshell}++;
+	my @tokens = ($self->parse(qr/^\)$/),
+		      $self->expect(')'));
+	$self->{insubshell}--;
+	return @tokens;
 }
 
 sub parse_case_pattern {
@@ -528,7 +532,7 @@ sub parse_loop_body {
 	return @tokens if ends_with(\@tokens, [qr/^\|\|$/, "\n", qr/^echo$/, qr/^.+$/]);
 	# flag missing "return/exit" handling explicit failure in loop body
 	my $n = find_non_nl(\@tokens);
-	push(@{$self->{problems}}, ['LOOP', $tokens[$n]]);
+	push(@{$self->{problems}}, [$self->{insubshell} ? 'LOOPEXIT' : 'LOOPRETURN', $tokens[$n]]);
 	return @tokens;
 }
 
@@ -619,6 +623,15 @@ sub unwrap {
 	return $s
 }
 
+sub format_problem {
+	local $_ = shift;
+	/^AMP$/ && return "missing '&&'";
+	/^LOOPRETURN$/ && return "missing '|| return 1'";
+	/^LOOPEXIT$/ && return "missing '|| exit 1'";
+	/^HEREDOC$/ && return 'unclosed heredoc';
+	die("unrecognized problem type '$_'\n");
+}
+
 sub check_test {
 	my $self = shift @_;
 	my $title = unwrap(shift @_);
@@ -641,7 +654,8 @@ sub check_test {
 	for (sort {$a->[1]->[2] <=> $b->[1]->[2]} @$problems) {
 		my ($label, $token) = @$_;
 		my $pos = $token->[2];
-		$checked .= substr($body, $start, $pos - $start) . " ?!$label?! ";
+		my $err = format_problem($label, $token);
+		$checked .= substr($body, $start, $pos - $start) . " ?!ERR $err?! ";
 		$start = $pos;
 	}
 	$checked .= substr($body, $start);
diff --git a/t/chainlint/arithmetic-expansion.expect b/t/chainlint/arithmetic-expansion.expect
index 338ecd5861..2efd65dcbd 100644
--- a/t/chainlint/arithmetic-expansion.expect
+++ b/t/chainlint/arithmetic-expansion.expect
@@ -4,6 +4,6 @@
 5 	baz
 6 ) &&
 7 (
-8 	bar=$((42 + 1)) ?!AMP?!
+8 	bar=$((42 + 1)) ?!ERR missing '&&'?!
 9 	baz
 10 )
diff --git a/t/chainlint/block.expect b/t/chainlint/block.expect
index b62e3d58c3..28410b33ef 100644
--- a/t/chainlint/block.expect
+++ b/t/chainlint/block.expect
@@ -1,20 +1,20 @@
 2 (
 3 	foo &&
 4 	{
-5 		echo a ?!AMP?!
+5 		echo a ?!ERR missing '&&'?!
 6 		echo b
 7 	} &&
 8 	bar &&
 9 	{
 10 		echo c
-11 	} ?!AMP?!
+11 	} ?!ERR missing '&&'?!
 12 	baz
 13 ) &&
 14 
 15 {
-16 	echo a; ?!AMP?! echo b
+16 	echo a; ?!ERR missing '&&'?! echo b
 17 } &&
-18 { echo a; ?!AMP?! echo b; } &&
+18 { echo a; ?!ERR missing '&&'?! echo b; } &&
 19 
 20 {
 21 	echo "${var}9" &&
diff --git a/t/chainlint/broken-chain.expect b/t/chainlint/broken-chain.expect
index 9a1838736f..2a209df0a7 100644
--- a/t/chainlint/broken-chain.expect
+++ b/t/chainlint/broken-chain.expect
@@ -1,6 +1,6 @@
 2 (
 3 	foo &&
-4 	bar ?!AMP?!
+4 	bar ?!ERR missing '&&'?!
 5 	baz &&
 6 	wop
 7 )
diff --git a/t/chainlint/case.expect b/t/chainlint/case.expect
index c04c61ff36..d00b67b766 100644
--- a/t/chainlint/case.expect
+++ b/t/chainlint/case.expect
@@ -9,11 +9,11 @@
 10 	case "$x" in
 11 	x) foo ;;
 12 	*) bar ;;
-13 	esac ?!AMP?!
+13 	esac ?!ERR missing '&&'?!
 14 	foobar
 15 ) &&
 16 (
 17 	case "$x" in 1) true;; esac &&
-18 	case "$y" in 2) false;; esac ?!AMP?!
+18 	case "$y" in 2) false;; esac ?!ERR missing '&&'?!
 19 	foobar
 20 )
diff --git a/t/chainlint/chain-break-false.expect b/t/chainlint/chain-break-false.expect
index 4f815f8e14..bfccc0d90f 100644
--- a/t/chainlint/chain-break-false.expect
+++ b/t/chainlint/chain-break-false.expect
@@ -4,6 +4,6 @@
 5 	echo failed!
 6 	false
 7 else
-8 	echo it went okay ?!AMP?!
+8 	echo it went okay ?!ERR missing '&&'?!
 9 	congratulate user
 10 fi
diff --git a/t/chainlint/chained-block.expect b/t/chainlint/chained-block.expect
index a546b714a6..293d9eac42 100644
--- a/t/chainlint/chained-block.expect
+++ b/t/chainlint/chained-block.expect
@@ -1,5 +1,5 @@
 2 echo nobody home && {
-3 	test the doohicky ?!AMP?!
+3 	test the doohicky ?!ERR missing '&&'?!
 4 	right now
 5 } &&
 6 
diff --git a/t/chainlint/chained-subshell.expect b/t/chainlint/chained-subshell.expect
index f78b268291..2f5de4fead 100644
--- a/t/chainlint/chained-subshell.expect
+++ b/t/chainlint/chained-subshell.expect
@@ -1,10 +1,10 @@
 2 mkdir sub && (
 3 	cd sub &&
-4 	foo the bar ?!AMP?!
+4 	foo the bar ?!ERR missing '&&'?!
 5 	nuff said
 6 ) &&
 7 
 8 cut "-d " -f actual | (read s1 s2 s3 &&
-9 test -f $s1 ?!AMP?!
+9 test -f $s1 ?!ERR missing '&&'?!
 10 test $(cat $s2) = tree2path1 &&
 11 test $(cat $s3) = tree3path1)
diff --git a/t/chainlint/command-substitution.expect b/t/chainlint/command-substitution.expect
index 5e31b36db6..511c918cb5 100644
--- a/t/chainlint/command-substitution.expect
+++ b/t/chainlint/command-substitution.expect
@@ -4,6 +4,6 @@
 5 	baz
 6 ) &&
 7 (
-8 	bar=$(gobble blocks) ?!AMP?!
+8 	bar=$(gobble blocks) ?!ERR missing '&&'?!
 9 	baz
 10 )
diff --git a/t/chainlint/complex-if-in-cuddled-loop.expect b/t/chainlint/complex-if-in-cuddled-loop.expect
index 3a740103db..eb855378a1 100644
--- a/t/chainlint/complex-if-in-cuddled-loop.expect
+++ b/t/chainlint/complex-if-in-cuddled-loop.expect
@@ -4,6 +4,6 @@
 5      :
 6    else
 7      echo >file
-8    fi ?!LOOP?!
+8    fi ?!ERR missing '|| exit 1'?!
 9  done) &&
 10 test ! -f file
diff --git a/t/chainlint/cuddled.expect b/t/chainlint/cuddled.expect
index b06d638311..65825c6879 100644
--- a/t/chainlint/cuddled.expect
+++ b/t/chainlint/cuddled.expect
@@ -2,7 +2,7 @@
 3 	bar
 4 ) &&
 5 
-6 (cd foo ?!AMP?!
+6 (cd foo ?!ERR missing '&&'?!
 7 	bar
 8 ) &&
 9 
@@ -13,5 +13,5 @@
 14 (cd foo &&
 15 	bar) &&
 16 
-17 (cd foo ?!AMP?!
+17 (cd foo ?!ERR missing '&&'?!
 18 	bar)
diff --git a/t/chainlint/for-loop.expect b/t/chainlint/for-loop.expect
index 908aeedf96..df6fc1a35f 100644
--- a/t/chainlint/for-loop.expect
+++ b/t/chainlint/for-loop.expect
@@ -1,14 +1,14 @@
 2 (
 3 	for i in a b c
 4 	do
-5 		echo $i ?!AMP?!
-6 		cat <<-\EOF ?!LOOP?!
+5 		echo $i ?!ERR missing '&&'?!
+6 		cat <<-\EOF ?!ERR missing '|| exit 1'?!
 7 		bar
 8 		EOF
-9 	done ?!AMP?!
+9 	done ?!ERR missing '&&'?!
 10 
 11 	for i in a b c; do
 12 		echo $i &&
-13 		cat $i ?!LOOP?!
+13 		cat $i ?!ERR missing '|| exit 1'?!
 14 	done
 15 )
diff --git a/t/chainlint/function.expect b/t/chainlint/function.expect
index c226246b25..7a15be745b 100644
--- a/t/chainlint/function.expect
+++ b/t/chainlint/function.expect
@@ -4,8 +4,8 @@
 5 
 6 remove_object() {
 7 	file=$(sha1_file "$*") &&
-8 	test -e "$file" ?!AMP?!
+8 	test -e "$file" ?!ERR missing '&&'?!
 9 	rm -f "$file"
-10 } ?!AMP?!
+10 } ?!ERR missing '&&'?!
 11 
 12 sha1_file arg && remove_object arg
diff --git a/t/chainlint/here-doc-body-indent.expect b/t/chainlint/here-doc-body-indent.expect
index 4323acc93d..1d7298c8ad 100644
--- a/t/chainlint/here-doc-body-indent.expect
+++ b/t/chainlint/here-doc-body-indent.expect
@@ -1,2 +1,2 @@
-2 	echo "we should find this" ?!AMP?!
+2 	echo "we should find this" ?!ERR missing '&&'?!
 3 	echo "even though our heredoc has its indent stripped"
diff --git a/t/chainlint/here-doc-body-pathological.expect b/t/chainlint/here-doc-body-pathological.expect
index a93a1fa3aa..828f232616 100644
--- a/t/chainlint/here-doc-body-pathological.expect
+++ b/t/chainlint/here-doc-body-pathological.expect
@@ -1,7 +1,7 @@
-2 	echo "outer here-doc does not allow indented end-tag" ?!AMP?!
+2 	echo "outer here-doc does not allow indented end-tag" ?!ERR missing '&&'?!
 3 	cat >file <<-\EOF &&
 4 	but this inner here-doc
 5 	does allow indented EOF
 6 	EOF
-7 	echo "missing chain after" ?!AMP?!
+7 	echo "missing chain after" ?!ERR missing '&&'?!
 8 	echo "but this line is OK because it's the end"
diff --git a/t/chainlint/here-doc-body.expect b/t/chainlint/here-doc-body.expect
index ddf1c412af..79b9603c1e 100644
--- a/t/chainlint/here-doc-body.expect
+++ b/t/chainlint/here-doc-body.expect
@@ -1,7 +1,7 @@
-2 	echo "missing chain before" ?!AMP?!
+2 	echo "missing chain before" ?!ERR missing '&&'?!
 3 	cat >file <<-\EOF &&
 4 	inside inner here-doc
 5 	these are not shell commands
 6 	EOF
-7 	echo "missing chain after" ?!AMP?!
+7 	echo "missing chain after" ?!ERR missing '&&'?!
 8 	echo "but this line is OK because it's the end"
diff --git a/t/chainlint/here-doc-double.expect b/t/chainlint/here-doc-double.expect
index 20dba4b452..9cb1a1a5e3 100644
--- a/t/chainlint/here-doc-double.expect
+++ b/t/chainlint/here-doc-double.expect
@@ -1,2 +1,2 @@
-8 	echo "actual test commands" ?!AMP?!
+8 	echo "actual test commands" ?!ERR missing '&&'?!
 9 	echo "that should be checked"
diff --git a/t/chainlint/here-doc-indent-operator.expect b/t/chainlint/here-doc-indent-operator.expect
index 277a11202d..2d61e5f49d 100644
--- a/t/chainlint/here-doc-indent-operator.expect
+++ b/t/chainlint/here-doc-indent-operator.expect
@@ -4,7 +4,7 @@
 5 chunks: oid_fanout oid_lookup commit_metadata generation_data bloom_indexes bloom_data
 6 EOF
 7 
-8 cat >expect << -EOF ?!AMP?!
+8 cat >expect << -EOF ?!ERR missing '&&'?!
 9 this is not indented
 10 -EOF
 11 
diff --git a/t/chainlint/here-doc-multi-line-command-subst.expect b/t/chainlint/here-doc-multi-line-command-subst.expect
index 41b55f6437..881e4d2098 100644
--- a/t/chainlint/here-doc-multi-line-command-subst.expect
+++ b/t/chainlint/here-doc-multi-line-command-subst.expect
@@ -3,6 +3,6 @@
 4 		fossil
 5 		vegetable
 6 		END
-7 		wiffle) ?!AMP?!
+7 		wiffle) ?!ERR missing '&&'?!
 8 	echo $x
 9 )
diff --git a/t/chainlint/here-doc-multi-line-string.expect b/t/chainlint/here-doc-multi-line-string.expect
index c71828589e..06c791e0a4 100644
--- a/t/chainlint/here-doc-multi-line-string.expect
+++ b/t/chainlint/here-doc-multi-line-string.expect
@@ -1,6 +1,6 @@
 2 (
 3 	cat <<-\TXT && echo "multi-line
-4 	string" ?!AMP?!
+4 	string" ?!ERR missing '&&'?!
 5 	fizzle
 6 	TXT
 7 	bap
diff --git a/t/chainlint/if-condition-split.expect b/t/chainlint/if-condition-split.expect
index 9daf3d294a..5688d93a4f 100644
--- a/t/chainlint/if-condition-split.expect
+++ b/t/chainlint/if-condition-split.expect
@@ -2,6 +2,6 @@
 3    marcia ||
 4    kevin
 5 then
-6 	echo "nomads" ?!AMP?!
+6 	echo "nomads" ?!ERR missing '&&'?!
 7 	echo "for sure"
 8 fi
diff --git a/t/chainlint/if-in-loop.expect b/t/chainlint/if-in-loop.expect
index ff8c60dbdb..253b461f87 100644
--- a/t/chainlint/if-in-loop.expect
+++ b/t/chainlint/if-in-loop.expect
@@ -5,8 +5,8 @@
 6 		then
 7 			echo "err"
 8 			exit 1
-9 		fi ?!AMP?!
+9 		fi ?!ERR missing '&&'?!
 10 		foo
-11 	done ?!AMP?!
+11 	done ?!ERR missing '&&'?!
 12 	bar
 13 )
diff --git a/t/chainlint/if-then-else.expect b/t/chainlint/if-then-else.expect
index 965d7e41a2..1b3162759f 100644
--- a/t/chainlint/if-then-else.expect
+++ b/t/chainlint/if-then-else.expect
@@ -1,7 +1,7 @@
 2 (
 3 	if test -n ""
 4 	then
-5 		echo very ?!AMP?!
+5 		echo very ?!ERR missing '&&'?!
 6 		echo empty
 7 	elif test -z ""
 8 	then
@@ -11,7 +11,7 @@
 12 		cat <<-\EOF
 13 		bar
 14 		EOF
-15 	fi ?!AMP?!
+15 	fi ?!ERR missing '&&'?!
 16 	echo poodle
 17 ) &&
 18 (
diff --git a/t/chainlint/inline-comment.expect b/t/chainlint/inline-comment.expect
index 0285c0b22c..fedc059a0c 100644
--- a/t/chainlint/inline-comment.expect
+++ b/t/chainlint/inline-comment.expect
@@ -1,6 +1,6 @@
 2 (
 3 	foobar && # comment 1
-4 	barfoo ?!AMP?! # wrong position for &&
+4 	barfoo ?!ERR missing '&&'?! # wrong position for &&
 5 	flibble "not a # comment"
 6 ) &&
 7 
diff --git a/t/chainlint/loop-detect-failure.expect b/t/chainlint/loop-detect-failure.expect
index 40c06f0d53..2d46f6d2eb 100644
--- a/t/chainlint/loop-detect-failure.expect
+++ b/t/chainlint/loop-detect-failure.expect
@@ -11,5 +11,5 @@
 12 do
 13 	printf "%"$n"s" X > r2/large.$n &&
 14 	git -C r2 add large.$n &&
-15 	git -C r2 commit -m "$n" ?!LOOP?!
+15 	git -C r2 commit -m "$n" ?!ERR missing '|| return 1'?!
 16 done
diff --git a/t/chainlint/loop-in-if.expect b/t/chainlint/loop-in-if.expect
index 4e8c67c914..8936d7ff2d 100644
--- a/t/chainlint/loop-in-if.expect
+++ b/t/chainlint/loop-in-if.expect
@@ -3,10 +3,10 @@
 4 	then
 5 		while true
 6 		do
-7 			echo "pop" ?!AMP?!
-8 			echo "glup" ?!LOOP?!
-9 		done ?!AMP?!
+7 			echo "pop" ?!ERR missing '&&'?!
+8 			echo "glup" ?!ERR missing '|| exit 1'?!
+9 		done ?!ERR missing '&&'?!
 10 		foo
-11 	fi ?!AMP?!
+11 	fi ?!ERR missing '&&'?!
 12 	bar
 13 )
diff --git a/t/chainlint/multi-line-string.expect b/t/chainlint/multi-line-string.expect
index 62c54e3a5e..3c3a1de75c 100644
--- a/t/chainlint/multi-line-string.expect
+++ b/t/chainlint/multi-line-string.expect
@@ -3,7 +3,7 @@
 4 		line 2
 5 		line 3" &&
 6 	y="line 1
-7 		line2" ?!AMP?!
+7 		line2" ?!ERR missing '&&'?!
 8 	foobar
 9 ) &&
 10 (
diff --git a/t/chainlint/negated-one-liner.expect b/t/chainlint/negated-one-liner.expect
index a6ce52a1da..12bd65264a 100644
--- a/t/chainlint/negated-one-liner.expect
+++ b/t/chainlint/negated-one-liner.expect
@@ -1,5 +1,5 @@
 2 ! (foo && bar) &&
 3 ! (foo && bar) >baz &&
 4 
-5 ! (foo; ?!AMP?! bar) &&
-6 ! (foo; ?!AMP?! bar) >baz
+5 ! (foo; ?!ERR missing '&&'?! bar) &&
+6 ! (foo; ?!ERR missing '&&'?! bar) >baz
diff --git a/t/chainlint/nested-cuddled-subshell.expect b/t/chainlint/nested-cuddled-subshell.expect
index 0191c9c294..3e947ea5e1 100644
--- a/t/chainlint/nested-cuddled-subshell.expect
+++ b/t/chainlint/nested-cuddled-subshell.expect
@@ -5,7 +5,7 @@
 6 
 7 	(cd foo &&
 8 		bar
-9 	) ?!AMP?!
+9 	) ?!ERR missing '&&'?!
 10 
 11 	(
 12 		cd foo &&
@@ -13,13 +13,13 @@
 14 
 15 	(
 16 		cd foo &&
-17 		bar) ?!AMP?!
+17 		bar) ?!ERR missing '&&'?!
 18 
 19 	(cd foo &&
 20 		bar) &&
 21 
 22 	(cd foo &&
-23 		bar) ?!AMP?!
+23 		bar) ?!ERR missing '&&'?!
 24 
 25 	foobar
 26 )
diff --git a/t/chainlint/nested-here-doc.expect b/t/chainlint/nested-here-doc.expect
index 70d9b68dc9..107e5afb01 100644
--- a/t/chainlint/nested-here-doc.expect
+++ b/t/chainlint/nested-here-doc.expect
@@ -18,7 +18,7 @@
 19 	toink
 20 	INPUT_END
 21 
-22 	cat <<-\EOT ?!AMP?!
+22 	cat <<-\EOT ?!ERR missing '&&'?!
 23 	text goes here
 24 	data <<EOF
 25 		data goes here
diff --git a/t/chainlint/nested-loop-detect-failure.expect b/t/chainlint/nested-loop-detect-failure.expect
index c13c4d2f90..26557b05a1 100644
--- a/t/chainlint/nested-loop-detect-failure.expect
+++ b/t/chainlint/nested-loop-detect-failure.expect
@@ -2,8 +2,8 @@
 3 do
 4 	for j in 0 1 2 3 4 5 6 7 8 9;
 5 	do
-6 		echo "$i$j" >"path$i$j" ?!LOOP?!
-7 	done ?!LOOP?!
+6 		echo "$i$j" >"path$i$j" ?!ERR missing '|| return 1'?!
+7 	done ?!ERR missing '|| return 1'?!
 8 done &&
 9 
 10 for i in 0 1 2 3 4 5 6 7 8 9;
@@ -18,7 +18,7 @@
 19 do
 20 	for j in 0 1 2 3 4 5 6 7 8 9;
 21 	do
-22 		echo "$i$j" >"path$i$j" ?!LOOP?!
+22 		echo "$i$j" >"path$i$j" ?!ERR missing '|| return 1'?!
 23 	done || return 1
 24 done &&
 25 
diff --git a/t/chainlint/nested-subshell-comment.expect b/t/chainlint/nested-subshell-comment.expect
index f89a8d03a8..c6891919c0 100644
--- a/t/chainlint/nested-subshell-comment.expect
+++ b/t/chainlint/nested-subshell-comment.expect
@@ -6,6 +6,6 @@
 7 		# minor numbers of cows (or do they?)
 8 		baz &&
 9 		snaff
-10 	) ?!AMP?!
+10 	) ?!ERR missing '&&'?!
 11 	fuzzy
 12 )
diff --git a/t/chainlint/nested-subshell.expect b/t/chainlint/nested-subshell.expect
index 811e8a7912..b98d723edf 100644
--- a/t/chainlint/nested-subshell.expect
+++ b/t/chainlint/nested-subshell.expect
@@ -7,7 +7,7 @@
 8 
 9 	cd foo &&
 10 	(
-11 		echo a ?!AMP?!
+11 		echo a ?!ERR missing '&&'?!
 12 		echo b
 13 	) >file
 14 )
diff --git a/t/chainlint/not-heredoc.expect b/t/chainlint/not-heredoc.expect
index 611b7b75cb..9910621103 100644
--- a/t/chainlint/not-heredoc.expect
+++ b/t/chainlint/not-heredoc.expect
@@ -9,6 +9,6 @@
 10 	echo ourside &&
 11 	echo "=======" &&
 12 	echo theirside &&
-13 	echo ">>>>>>> theirs" ?!AMP?!
+13 	echo ">>>>>>> theirs" ?!ERR missing '&&'?!
 14 	poodle
 15 ) >merged
diff --git a/t/chainlint/one-liner-for-loop.expect b/t/chainlint/one-liner-for-loop.expect
index 49dcf065ef..2eb2d5fcaf 100644
--- a/t/chainlint/one-liner-for-loop.expect
+++ b/t/chainlint/one-liner-for-loop.expect
@@ -3,7 +3,7 @@
 4 	cd dir-rename-and-content &&
 5 	test_write_lines 1 2 3 4 5 >foo &&
 6 	mkdir olddir &&
-7 	for i in a b c; do echo $i >olddir/$i; ?!LOOP?! done ?!AMP?!
+7 	for i in a b c; do echo $i >olddir/$i; ?!ERR missing '|| exit 1'?! done ?!ERR missing '&&'?!
 8 	git add foo olddir &&
 9 	git commit -m "original" &&
 10 )
diff --git a/t/chainlint/one-liner.expect b/t/chainlint/one-liner.expect
index 9861811283..2c5826e6c4 100644
--- a/t/chainlint/one-liner.expect
+++ b/t/chainlint/one-liner.expect
@@ -2,8 +2,8 @@
 3 (foo && bar) |
 4 (foo && bar) >baz &&
 5 
-6 (foo; ?!AMP?! bar) &&
-7 (foo; ?!AMP?! bar) |
-8 (foo; ?!AMP?! bar) >baz &&
+6 (foo; ?!ERR missing '&&'?! bar) &&
+7 (foo; ?!ERR missing '&&'?! bar) |
+8 (foo; ?!ERR missing '&&'?! bar) >baz &&
 9 
 10 (foo "bar; baz")
diff --git a/t/chainlint/pipe.expect b/t/chainlint/pipe.expect
index 1bbe5a2ce1..a198d5bdb2 100644
--- a/t/chainlint/pipe.expect
+++ b/t/chainlint/pipe.expect
@@ -4,7 +4,7 @@
 5 	baz &&
 6 
 7 	fish |
-8 	cow ?!AMP?!
+8 	cow ?!ERR missing '&&'?!
 9 
 10 	sunder
 11 )
diff --git a/t/chainlint/semicolon.expect b/t/chainlint/semicolon.expect
index 866438310c..e22920bf2c 100644
--- a/t/chainlint/semicolon.expect
+++ b/t/chainlint/semicolon.expect
@@ -1,19 +1,19 @@
 2 (
-3 	cat foo ; ?!AMP?! echo bar ?!AMP?!
-4 	cat foo ; ?!AMP?! echo bar
+3 	cat foo ; ?!ERR missing '&&'?! echo bar ?!ERR missing '&&'?!
+4 	cat foo ; ?!ERR missing '&&'?! echo bar
 5 ) &&
 6 (
-7 	cat foo ; ?!AMP?! echo bar &&
-8 	cat foo ; ?!AMP?! echo bar
+7 	cat foo ; ?!ERR missing '&&'?! echo bar &&
+8 	cat foo ; ?!ERR missing '&&'?! echo bar
 9 ) &&
 10 (
 11 	echo "foo; bar" &&
-12 	cat foo; ?!AMP?! echo bar
+12 	cat foo; ?!ERR missing '&&'?! echo bar
 13 ) &&
 14 (
 15 	foo;
 16 ) &&
 17 (cd foo &&
 18 	for i in a b c; do
-19 		echo; ?!LOOP?!
+19 		echo; ?!ERR missing '|| exit 1'?!
 20 	done)
diff --git a/t/chainlint/subshell-here-doc.expect b/t/chainlint/subshell-here-doc.expect
index 5647500c82..953d8084e5 100644
--- a/t/chainlint/subshell-here-doc.expect
+++ b/t/chainlint/subshell-here-doc.expect
@@ -6,7 +6,7 @@
 7 	nevermore...
 8 	EOF
 9 
-10 	cat <<EOF >bip ?!AMP?!
+10 	cat <<EOF >bip ?!ERR missing '&&'?!
 11 	fish fly high
 12 EOF
 13 
diff --git a/t/chainlint/subshell-one-liner.expect b/t/chainlint/subshell-one-liner.expect
index 214316c6a0..f82296db66 100644
--- a/t/chainlint/subshell-one-liner.expect
+++ b/t/chainlint/subshell-one-liner.expect
@@ -3,17 +3,17 @@
 4 	(foo && bar) |
 5 	(foo && bar) >baz &&
 6 
-7 	(foo; ?!AMP?! bar) &&
-8 	(foo; ?!AMP?! bar) |
-9 	(foo; ?!AMP?! bar) >baz &&
+7 	(foo; ?!ERR missing '&&'?! bar) &&
+8 	(foo; ?!ERR missing '&&'?! bar) |
+9 	(foo; ?!ERR missing '&&'?! bar) >baz &&
 10 
 11 	(foo || exit 1) &&
 12 	(foo || exit 1) |
 13 	(foo || exit 1) >baz &&
 14 
-15 	(foo && bar) ?!AMP?!
+15 	(foo && bar) ?!ERR missing '&&'?!
 16 
-17 	(foo && bar; ?!AMP?! baz) ?!AMP?!
+17 	(foo && bar; ?!ERR missing '&&'?! baz) ?!ERR missing '&&'?!
 18 
 19 	foobar
 20 )
diff --git a/t/chainlint/token-pasting.expect b/t/chainlint/token-pasting.expect
index 64f3235d26..aa64cf75f3 100644
--- a/t/chainlint/token-pasting.expect
+++ b/t/chainlint/token-pasting.expect
@@ -2,13 +2,13 @@
 3 git config filter.rot13.clean ./rot13.sh &&
 4 
 5 {
-6     echo "*.t filter=rot13" ?!AMP?!
+6     echo "*.t filter=rot13" ?!ERR missing '&&'?!
 7     echo "*.i ident"
 8 } >.gitattributes &&
 9 
 10 {
-11     echo a b c d e f g h i j k l m ?!AMP?!
-12     echo n o p q r s t u v w x y z ?!AMP?!
+11     echo a b c d e f g h i j k l m ?!ERR missing '&&'?!
+12     echo n o p q r s t u v w x y z ?!ERR missing '&&'?!
 13     echo '$Id$'
 14 } >test &&
 15 cat test >test.t &&
@@ -19,7 +19,7 @@
 20 git checkout -- test test.t test.i &&
 21 
 22 echo "content-test2" >test2.o &&
-23 echo "content-test3 - filename with special characters" >"test3 'sq',$x=.o" ?!AMP?!
+23 echo "content-test3 - filename with special characters" >"test3 'sq',$x=.o" ?!ERR missing '&&'?!
 24 
 25 downstream_url_for_sed=$(
 26 	printf "%sn" "$downstream_url" |
diff --git a/t/chainlint/unclosed-here-doc-indent.expect b/t/chainlint/unclosed-here-doc-indent.expect
index f78e23cb63..d5b9ab52ee 100644
--- a/t/chainlint/unclosed-here-doc-indent.expect
+++ b/t/chainlint/unclosed-here-doc-indent.expect
@@ -1,4 +1,4 @@
 2 command_which_is_run &&
-3 cat >expect <<-\EOF ?!UNCLOSED-HEREDOC?! &&
+3 cat >expect <<-\EOF ?!ERR unclosed heredoc?! &&
 4 we forget to end the here-doc
 5 command_which_is_gobbled
diff --git a/t/chainlint/unclosed-here-doc.expect b/t/chainlint/unclosed-here-doc.expect
index 51304672cf..8f6d260544 100644
--- a/t/chainlint/unclosed-here-doc.expect
+++ b/t/chainlint/unclosed-here-doc.expect
@@ -1,5 +1,5 @@
 2 command_which_is_run &&
-3 cat >expect <<\EOF ?!UNCLOSED-HEREDOC?! &&
+3 cat >expect <<\EOF ?!ERR unclosed heredoc?! &&
 4 	we try to end the here-doc below,
 5 	but the indentation throws us off
 6 	since the operator is not "<<-".
diff --git a/t/chainlint/while-loop.expect b/t/chainlint/while-loop.expect
index 5ffabd5a93..1cfd17b3c2 100644
--- a/t/chainlint/while-loop.expect
+++ b/t/chainlint/while-loop.expect
@@ -1,14 +1,14 @@
 2 (
 3 	while true
 4 	do
-5 		echo foo ?!AMP?!
-6 		cat <<-\EOF ?!LOOP?!
+5 		echo foo ?!ERR missing '&&'?!
+6 		cat <<-\EOF ?!ERR missing '|| exit 1'?!
 7 		bar
 8 		EOF
-9 	done ?!AMP?!
+9 	done ?!ERR missing '&&'?!
 10 
 11 	while true; do
 12 		echo foo &&
-13 		cat bar ?!LOOP?!
+13 		cat bar ?!ERR missing '|| exit 1'?!
 14 	done
 15 )
-- 
2.46.0

[PATCH 2/2] chainlint: reduce annotation noise-factor

[Eric Sunshine]

From: Eric Sunshine <sunshine@sunshineco.com>

When chainlint detects a problem in a test definition, it highlights the
offending code with an "?!ERR ...?!" annotation. The rather curious "?!"
delimiter was chosen to draw the reader's attention to the problem area.

Later, chainlint learned to color its output when sent to a terminal.
Problem annotations are colored with a red background which stands out
well from surrounding text, thus easily draws the reader's attention. As
such, the additional "?!" decoration became superfluous (when output is
colored), however the decoration was retained since it serves as a good
needle when using the terminal's search feature to "jump" to the next
problem.

Nevertheless, the "?!" decoration is noisy and ugly and makes it
unnecessarily difficult for the reader to pluck the problem description
from the annotation. For instance, it is easier to see at a glance what
the problem is in:

    ERR missing '&&'

than in the noisier:

    ?!ERR missing '&&'?!

Therefore drop the "!?" decoration when output is colored (but retain it
otherwise).

Note that the preceding change gave all problem annotations a uniform
"ERR" prefix which serves as a reasonably suitable replacement needle
when searching in a terminal, so loss of "?!" in the output should not
be overly problematic.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
---
 t/chainlint.pl | 7 +++++--
 t/test-lib.sh  | 2 +-
 2 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/t/chainlint.pl b/t/chainlint.pl
index d79f183dfd..971ab9212a 100755
--- a/t/chainlint.pl
+++ b/t/chainlint.pl
@@ -591,6 +591,7 @@ sub new {
 	my $class = shift @_;
 	my $self = $class->SUPER::new(@_);
 	$self->{ntests} = 0;
+	$self->{nerrs} = 0;
 	return $self;
 }
 
@@ -647,8 +648,10 @@ sub check_test {
 	my $parser = TestParser->new(\$body);
 	my @tokens = $parser->parse();
 	my $problems = $parser->{problems};
+	$self->{nerrs} += @$problems;
 	return unless $emit_all || @$problems;
 	my $c = main::fd_colors(1);
+	my ($erropen, $errclose) = -t 1 ? ("$c->{rev}$c->{red}", $c->{reset}) : ('?!', '?!');
 	my $start = 0;
 	my $checked = '';
 	for (sort {$a->[1]->[2] <=> $b->[1]->[2]} @$problems) {
@@ -663,7 +666,7 @@ sub check_test {
 	$checked =~ s/^\d+ \n//;
 	$checked =~ s/(\s) \?!/$1?!/mg;
 	$checked =~ s/\?! (\s)/?!$1/mg;
-	$checked =~ s/(\?![^?]+\?!)/$c->{rev}$c->{red}$1$c->{reset}/mg;
+	$checked =~ s/\?!([^?]+)\?!/$erropen$1$errclose/mg;
 	$checked =~ s/^\d+/$c->{dim}$&$c->{reset}/mg;
 	$checked .= "\n" unless $checked =~ /\n$/;
 	push(@{$self->{output}}, "$c->{blue}# chainlint: $title$c->{reset}\n$checked");
@@ -805,9 +808,9 @@ sub check_script {
 			my $c = fd_colors(1);
 			my $s = join('', @{$parser->{output}});
 			$emit->("$c->{bold}$c->{blue}# chainlint: $path$c->{reset}\n" . $s);
-			$nerrs += () = $s =~ /\?![^?]+\?!/g;
 		}
 		$ntests += $parser->{ntests};
+		$nerrs += $parser->{nerrs};
 	}
 	return [$id, $nscripts, $ntests, $nerrs];
 }
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 54247604cb..b652cb98cd 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -1606,7 +1606,7 @@ if test "${GIT_TEST_CHAIN_LINT:-1}" != 0 &&
    test "${GIT_TEST_EXT_CHAIN_LINT:-1}" != 0
 then
 	"$PERL_PATH" "$TEST_DIRECTORY/chainlint.pl" "$0" ||
-		BUG "lint error (see '?!...!? annotations above)"
+		BUG "lint error (see 'ERR' annotations above)"
 fi
 
 # Last-minute variable setup
-- 
2.46.0

Re: [PATCH 1/2] chainlint: make error messages self-explanatory

[Patrick Steinhardt]

On Thu, Aug 29, 2024 at 05:16:24AM -0400, Eric Sunshine wrote:
> From: Eric Sunshine <sunshine@sunshineco.com>
> 
> The annotations emitted by chainlint to indicate detected problems are
> overly terse, so much so that developers new to the project -- those who
> should most benefit from the linting -- may find them baffling. For
> instance, although the author of chainlint and seasoned Git developers
> may understand that "?!AMP?!" is an abbreviation of "ampersand" and
> indicates a break in the &&-chain, this may not be obvious to newcomers.
> 
> Similarly, although the annotation "?!LOOP?!" is understood by project
> regulars to indicate a missing `|| return 1` (or `|| exit 1` in a
> subshell), newcomers may find it more than a little perplexing. The
> "?!LOOP?!" case is particularly serious since it is likely that some
> newcomers are unaware that shell loops do not terminate automatically
> upon error, and it is more difficult for a newcomer to figure out how to
> correct the problem by examining surrounding code since `|| return 1`
> appears in test scrips relatively infrequently (compared, for instance,
> with &&-chaining).
> 
> Address these shortcomings by emitting human-consumable messages which
> both explain the problem and give a strong hint about how to correct it.

A worthwhile goal indeed. As you say, especially figuring out how to fix
the loop annotations is not exactly straight forward.

[snip]
> diff --git a/t/chainlint.pl b/t/chainlint.pl
> index 5361f23b1d..d79f183dfd 100755
> --- a/t/chainlint.pl
> +++ b/t/chainlint.pl
> @@ -9,7 +9,7 @@
>  # Input arguments are pathnames of shell scripts containing test definitions,
>  # or globs referencing a collection of scripts. For each problem discovered,
>  # the pathname of the script containing the test is printed along with the test
> -# name and the test body with a `?!FOO?!` annotation at the location of each
> +# name and the test body with a `?!ERR?!` annotation at the location of each
>  # detected problem, where "FOO" is a tag such as "AMP" which indicates a broken
>  # &&-chain. Returns zero if no problems are discovered, otherwise non-zero.
>  
> @@ -181,7 +181,7 @@ sub swallow_heredocs {
>  			$self->{lineno} += () = $body =~ /\n/sg;
>  			next;
>  		}
> -		push(@{$self->{parser}->{problems}}, ['UNCLOSED-HEREDOC', $tag]);
> +		push(@{$self->{parser}->{problems}}, ['HEREDOC', $tag]);
>  		$$b =~ /(?:\G|\n).*\z/gc; # consume rest of input
>  		my $body = substr($$b, $start, pos($$b) - $start);
>  		$self->{lineno} += () = $body =~ /\n/sg;

I was wondering why this is being changed here, as I found the old name
to be easier to understand. Then I saw further down that you essentially
use those as identifiers for the actual problem.

Is there a specific reason why we now have the separate translation
step? Couldn't we instead push the translated message here, directly?

> @@ -296,8 +297,11 @@ sub parse_group {
>  
>  sub parse_subshell {
>  	my $self = shift @_;
> -	return ($self->parse(qr/^\)$/),
> -		$self->expect(')'));
> +	$self->{insubshell}++;
> +	my @tokens = ($self->parse(qr/^\)$/),
> +		      $self->expect(')'));
> +	$self->{insubshell}--;
> +	return @tokens;
>  }
>  
>  sub parse_case_pattern {

Okay. The subshell recursion level tracking here is required such that
we can discern LOOPEXIT vs LOOPRETURN cases. Makes sense.

> @@ -641,7 +654,8 @@ sub check_test {
>  	for (sort {$a->[1]->[2] <=> $b->[1]->[2]} @$problems) {
>  		my ($label, $token) = @$_;
>  		my $pos = $token->[2];
> -		$checked .= substr($body, $start, $pos - $start) . " ?!$label?! ";
> +		my $err = format_problem($label, $token);
> +		$checked .= substr($body, $start, $pos - $start) . " ?!ERR $err?! ";
>  		$start = $pos;
>  	}
>  	$checked .= substr($body, $start);
> diff --git a/t/chainlint/arithmetic-expansion.expect b/t/chainlint/arithmetic-expansion.expect
> index 338ecd5861..2efd65dcbd 100644
> --- a/t/chainlint/arithmetic-expansion.expect
> +++ b/t/chainlint/arithmetic-expansion.expect
> @@ -4,6 +4,6 @@
>  5 	baz
>  6 ) &&
>  7 (
> -8 	bar=$((42 + 1)) ?!AMP?!
> +8 	bar=$((42 + 1)) ?!ERR missing '&&'?!
>  9 	baz
>  10 )

I find the resulting error messages a bit confusing: to me it reads as
if "ERR" is missing the ampersands. Is it actually useful to have the
ERR prefix in the first place? We do not output anything but errors, so
it feels somewhat redundant.

Patrick

Re: [PATCH 2/2] chainlint: reduce annotation noise-factor

[Patrick Steinhardt]

On Thu, Aug 29, 2024 at 05:16:25AM -0400, Eric Sunshine wrote:
> From: Eric Sunshine <sunshine@sunshineco.com>
> 
> When chainlint detects a problem in a test definition, it highlights the
> offending code with an "?!ERR ...?!" annotation. The rather curious "?!"
> delimiter was chosen to draw the reader's attention to the problem area.
> 
> Later, chainlint learned to color its output when sent to a terminal.
> Problem annotations are colored with a red background which stands out
> well from surrounding text, thus easily draws the reader's attention. As
> such, the additional "?!" decoration became superfluous (when output is
> colored), however the decoration was retained since it serves as a good
> needle when using the terminal's search feature to "jump" to the next
> problem.
> 
> Nevertheless, the "?!" decoration is noisy and ugly and makes it
> unnecessarily difficult for the reader to pluck the problem description
> from the annotation. For instance, it is easier to see at a glance what
> the problem is in:
> 
>     ERR missing '&&'
> 
> than in the noisier:
> 
>     ?!ERR missing '&&'?!
> 
> Therefore drop the "!?" decoration when output is colored (but retain it
> otherwise).
> 
> Note that the preceding change gave all problem annotations a uniform
> "ERR" prefix which serves as a reasonably suitable replacement needle
> when searching in a terminal, so loss of "?!" in the output should not
> be overly problematic.

Okay, now the "ERR" prefix becomes a bit more important because we drop
the other punctuation. I'm still not much of a fan of it, though. Makes
me wonder whether we want to take a clue from how compilers nowadays
format this, e.g. by using "pointers".

So this:

    2 (
    3 	foo |
    4 	bar |
    5 	baz &&
    6 
    7 	fish |
    8 	cow ?!AMP?!
    9 
    10 	sunder
    11 )

Would become this:

    t/chainlint/pipe.actual:8: error: expected ampersands (&&)
    7 	fish |
    8 	cow 
            ^
    9

While this would be neat, I guess it would also be way more work than
the current series you have posted. And whether that work is ultimately
really worth it may be another question. Probably not.

So overall, I'm fine with the direction that your patch series takes.

Patrick

Re: [PATCH 1/2] chainlint: make error messages self-explanatory

[Junio C Hamano]

Eric Sunshine <ericsunshine@charter.net> writes:

> "?!LOOP?!" case is particularly serious since it is likely that some
> newcomers are unaware that shell loops do not terminate automatically
> upon error, and it is more difficult for a newcomer to figure out how to
> correct the problem by examining surrounding code since `|| return 1`
> appears in test scrips relatively infrequently (compared, for instance,
> with &&-chaining).

"scrips" -> "scripts"

I'd prefer to see "some newcomes are unaware that" part rewritten
and toned down, as it is not our primary business to help total
newbies to learn shells, it certainly is not what the chain lint
checker should bend over backwards to do.

    ... particularly serious, as it does not convey that returning
    control with "|| return 1" (or "|| exit 1" from a subshell)
    immediately after we detect an error is the canonical way we
    chose in this project to handle errors in a loop.  Because it
    happens relatively infrequently, this norm is harder to figure
    out for a new person on their own than other patterns (like
    &&-chaining).

> Address these shortcomings by emitting human-consumable messages which
> both explain the problem and give a strong hint about how to correct it.

"consumable" -> "readable".

>
> Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
> ...
>  # Input arguments are pathnames of shell scripts containing test definitions,
>  # or globs referencing a collection of scripts. For each problem discovered,
>  # the pathname of the script containing the test is printed along with the test
> -# name and the test body with a `?!FOO?!` annotation at the location of each
> +# name and the test body with a `?!ERR?!` annotation at the location of each
>  # detected problem, where "FOO" is a tag such as "AMP" which indicates a broken

"FOO" -> "ERR"?

> @@ -619,6 +623,15 @@ sub unwrap {
>  	return $s
>  }
>  
> +sub format_problem {
> +	local $_ = shift;
> +	/^AMP$/ && return "missing '&&'";
> +	/^LOOPRETURN$/ && return "missing '|| return 1'";
> +	/^LOOPEXIT$/ && return "missing '|| exit 1'";
> +	/^HEREDOC$/ && return 'unclosed heredoc';
> +	die("unrecognized problem type '$_'\n");
> +}
> +
>  sub check_test {
>  	my $self = shift @_;
>  	my $title = unwrap(shift @_);
> @@ -641,7 +654,8 @@ sub check_test {
>  	for (sort {$a->[1]->[2] <=> $b->[1]->[2]} @$problems) {
>  		my ($label, $token) = @$_;
>  		my $pos = $token->[2];
> -		$checked .= substr($body, $start, $pos - $start) . " ?!$label?! ";
> +		my $err = format_problem($label, $token);
> +		$checked .= substr($body, $start, $pos - $start) . " ?!ERR $err?! ";
>  		$start = $pos;
>  	}
>  	$checked .= substr($body, $start);

With the hunks omitted before the above two that let us tell between
RETURN vs EXIT, the above two makes the problems much easier to
read.

All the "examples" (self tests) and changes to them looked sensible.

Thanks.

Re: [PATCH 2/2] chainlint: reduce annotation noise-factor

[Junio C Hamano]

Eric Sunshine <ericsunshine@charter.net> writes:

> From: Eric Sunshine <sunshine@sunshineco.com>
>
> When chainlint detects a problem in a test definition, it highlights the
> offending code with an "?!ERR ...?!" annotation. The rather curious "?!"
> delimiter was chosen to draw the reader's attention to the problem area.
>
> Later, chainlint learned to color its output when sent to a terminal.
> Problem annotations are colored with a red background which stands out
> well from surrounding text, thus easily draws the reader's attention. As
> such, the additional "?!" decoration became superfluous (when output is
> colored), however the decoration was retained since it serves as a good
> needle when using the terminal's search feature to "jump" to the next
> problem.
>
> Nevertheless, the "?!" decoration is noisy and ugly and makes it
> unnecessarily difficult for the reader to pluck the problem description
> from the annotation. For instance, it is easier to see at a glance what
> the problem is in:
>
>     ERR missing '&&'
>
> than in the noisier:
>
>     ?!ERR missing '&&'?!
>
> Therefore drop the "!?" decoration when output is colored (but retain it
> otherwise).

Wait.  That does not qualify "Therefore".

We talked about a "good needle" and then complained how ugly the
string that was happened to be chosen as good needle is.  That is
not enough to explain why it is justified to "lose" the needle.  The
only thing you justified is to move away from the ugly pattern, as a
typical "terminal's search feature" does not give us an easy way to
"jump to the next text painted yellow".

> Note that the preceding change gave all problem annotations a uniform
> "ERR" prefix which serves as a reasonably suitable replacement needle
> when searching in a terminal, so loss of "?!" in the output should not
> be overly problematic.

Drop this separate paragraph, promote its contents up from "Note"
status and as a proper part of the previous sentence in its rewrite,
something like:

    Since the errors are all uniformly prefixed with "ERR", which
    can be used as the "good needle" instead, lose the "!?"
    decoration when output is colored.

to replace "Therefore" and everything that follow.

> @@ -663,7 +666,7 @@ sub check_test {
>  	$checked =~ s/^\d+ \n//;
>  	$checked =~ s/(\s) \?!/$1?!/mg;
>  	$checked =~ s/\?! (\s)/?!$1/mg;
> -	$checked =~ s/(\?![^?]+\?!)/$c->{rev}$c->{red}$1$c->{reset}/mg;
> +	$checked =~ s/\?!([^?]+)\?!/$erropen$1$errclose/mg;

Hmph.  With $erropen and $errclose, I was hoping that we can shed
the reliance on the "?!" mark even internally.  This is especially
true that in the early part of this sub, the problem description was
very much structured piece of data, not something the consuming code
need to pick out of an already formatted text like this, risking to
get confused by the payload (i.e. the text that came from the
problematic test script inside "substr($body, $start, $pos-$start)"
may contain anything, including "?!", right?).

>  	$checked =~ s/^\d+/$c->{dim}$&$c->{reset}/mg;
>  	$checked .= "\n" unless $checked =~ /\n$/;
>  	push(@{$self->{output}}, "$c->{blue}# chainlint: $title$c->{reset}\n$checked");
> @@ -805,9 +808,9 @@ sub check_script {
>  			my $c = fd_colors(1);
>  			my $s = join('', @{$parser->{output}});
>  			$emit->("$c->{bold}$c->{blue}# chainlint: $path$c->{reset}\n" . $s);
> -			$nerrs += () = $s =~ /\?![^?]+\?!/g;
>  		}
>  		$ntests += $parser->{ntests};
> +		$nerrs += $parser->{nerrs};
>  	}
>  	return [$id, $nscripts, $ntests, $nerrs];
>  }
> diff --git a/t/test-lib.sh b/t/test-lib.sh
> index 54247604cb..b652cb98cd 100644
> --- a/t/test-lib.sh
> +++ b/t/test-lib.sh
> @@ -1606,7 +1606,7 @@ if test "${GIT_TEST_CHAIN_LINT:-1}" != 0 &&
>     test "${GIT_TEST_EXT_CHAIN_LINT:-1}" != 0
>  then
>  	"$PERL_PATH" "$TEST_DIRECTORY/chainlint.pl" "$0" ||
> -		BUG "lint error (see '?!...!? annotations above)"
> +		BUG "lint error (see 'ERR' annotations above)"
>  fi
>  
>  # Last-minute variable setup

Overall the two patches looked great.
Thanks.

Re: [PATCH 1/2] chainlint: make error messages self-explanatory

[Jeff King]

On Thu, Aug 29, 2024 at 12:03:33PM +0200, Patrick Steinhardt wrote:

> > diff --git a/t/chainlint/arithmetic-expansion.expect b/t/chainlint/arithmetic-expansion.expect
> > index 338ecd5861..2efd65dcbd 100644
> > --- a/t/chainlint/arithmetic-expansion.expect
> > +++ b/t/chainlint/arithmetic-expansion.expect
> > @@ -4,6 +4,6 @@
> >  5 	baz
> >  6 ) &&
> >  7 (
> > -8 	bar=$((42 + 1)) ?!AMP?!
> > +8 	bar=$((42 + 1)) ?!ERR missing '&&'?!
> >  9 	baz
> >  10 )
> 
> I find the resulting error messages a bit confusing: to me it reads as
> if "ERR" is missing the ampersands. Is it actually useful to have the
> ERR prefix in the first place? We do not output anything but errors, so
> it feels somewhat redundant.

I wonder if coloring "ERR" differently, or perhaps even adding a colon,
like "ERR: ", would make it stand out more.

FWIW, I find the existing error messages pretty readable, but that is
probably a sign that my mind has been poisoned by using chainlint too
much already. ;)

-Peff

Re: [PATCH 2/2] chainlint: reduce annotation noise-factor

[Jeff King]

On Thu, Aug 29, 2024 at 12:03:37PM +0200, Patrick Steinhardt wrote:

> Okay, now the "ERR" prefix becomes a bit more important because we drop
> the other punctuation. I'm still not much of a fan of it, though. Makes
> me wonder whether we want to take a clue from how compilers nowadays
> format this, e.g. by using "pointers".
> 
> So this:
> 
>     2 (
>     3 	foo |
>     4 	bar |
>     5 	baz &&
>     6 
>     7 	fish |
>     8 	cow ?!AMP?!
>     9 
>     10 	sunder
>     11 )
> 
> Would become this:
> 
>     t/chainlint/pipe.actual:8: error: expected ampersands (&&)
>     7 	fish |
>     8 	cow 
>             ^
>     9
> 
> While this would be neat, I guess it would also be way more work than
> the current series you have posted. And whether that work is ultimately
> really worth it may be another question. Probably not.

I think that output is quite readable. One bonus is that it follows the
usual "quickfix" format, so there's editor support for jumping to the
problematic spot.

It probably is more verbose if you have multiple errors right next to
each other (since now we just show the annotated source text). But that
is going to be relatively rare compared to single mistakes, I'd think.

-Peff

Re: [PATCH 1/2] chainlint: make error messages self-explanatory

[Eric Sunshine]

On Thu, Aug 29, 2024 at 6:03 AM Patrick Steinhardt <ps@pks.im> wrote:
> On Thu, Aug 29, 2024 at 05:16:24AM -0400, Eric Sunshine wrote:
> > -             push(@{$self->{parser}->{problems}}, ['UNCLOSED-HEREDOC', $tag]);
> > +             push(@{$self->{parser}->{problems}}, ['HEREDOC', $tag]);
> >               $$b =~ /(?:\G|\n).*\z/gc; # consume rest of input
> >               my $body = substr($$b, $start, pos($$b) - $start);
> >               $self->{lineno} += () = $body =~ /\n/sg;
>
> I was wondering why this is being changed here, as I found the old name
> to be easier to understand. Then I saw further down that you essentially
> use those as identifiers for the actual problem.

Peff chose[1] the longer "UNCLOSED-HEREDOC" over the (perhaps too)
terse "HERE" I had chosen[2], however, now that this is an internal
detail of the script -- not part of the user-facing output -- such
verbosity is unneeded. As programmers, just as we choose shorter
variable names (say, "i" instead of "record_index" in a for-loop), I
find "HEREDOC" easier to read in a code context than the longer
"UNCLOSED-HEREDOC", hence this (admittedly unnecessary) change.

[1]: https://lore.kernel.org/git/20230330193031.GC27989@coredump.intra.peff.net/
[2]: https://lore.kernel.org/git/CAPig+cQiOGrDSUc34jHEBp87Rx-dnXNcPcF76bu0SJoOzD+1hw@mail.gmail.com/

> Is there a specific reason why we now have the separate translation
> step? Couldn't we instead push the translated message here, directly?

I considered that but, although this instance is a simple "push"
operation, some heuristics scan and modify the `problems` array by
looking for and removing specific items. There are numerous instances
in (older) scripts similar to this:

    if condition not satisified
    then
        echo it did not work...
        echo failed!
        return 1
    fi

which prints an error message and then explicitly signals failure with
`return 1` (or `exit 1` or `false`) as the final command in an `if`
branch or `case` arm. In these cases, the tests don't bother
maintaining the &&-chain between `echo` and the explicit "test failed"
indicator.

As chainlint processes the token stream, it correctly pushes "AMP"
annotations onto the `problems` array for each of the `echo` lines,
but when it encounters the explicit `return 1`, the heuristic kicks in
and notices that the broken &&-chain leading up to `return 1` is
immaterial since the construct is manually signaling failure, thus the
&&-chain breakage is legitimate and safe. Requiring test authors to
add "&&" to each such line would just be making busy-work for them.
Hence, the heuristic actively removes the preceding "AMP" annotations
from `problems`. For the removal, it's easier to search `problems` for
a simple token such as "AMP" than to search for a user-facing message
such as "ERR missing '?!'".

> > -8    bar=$((42 + 1)) ?!AMP?!
> > +8    bar=$((42 + 1)) ?!ERR missing '&&'?!
>
> I find the resulting error messages a bit confusing: to me it reads as
> if "ERR" is missing the ampersands. Is it actually useful to have the
> ERR prefix in the first place? We do not output anything but errors, so
> it feels somewhat redundant.

As you mentioned in your review of [2/2], the "ERR" prefix serves as a
useful target for searches in a terminal.

Regarding possible confusion, my first draft placed a colon after the
prefix, i.e.:

    ERR: missing '&&'

but it seemed unnecessarily noisy, so I dropped the colon since:

    ERR missing '&&'

seemed clear enough. However, I don't feel too strongly about it and
can add the colon back if people think it would make the message
clearer.

Re: [PATCH 1/2] chainlint: make error messages self-explanatory

[Eric Sunshine]

On Thu, Aug 29, 2024 at 1:07 PM Jeff King <peff@peff.net> wrote:
> On Thu, Aug 29, 2024 at 12:03:33PM +0200, Patrick Steinhardt wrote:
> > I find the resulting error messages a bit confusing: to me it reads as
> > if "ERR" is missing the ampersands. Is it actually useful to have the
> > ERR prefix in the first place? We do not output anything but errors, so
> > it feels somewhat redundant.
>
> I wonder if coloring "ERR" differently, or perhaps even adding a colon,
> like "ERR: ", would make it stand out more.

I considered both of these ideas. Coloring "ERR" was dismissed almost
immediately due to the (admittedly tiny) bit of extra complexity and
the minimal color palette available (and since I couldn't trust myself
to not waste an inordinate amount of time trying to arrive at the
perfect color combination).

My first draft did place a colon after "ERR", but it seemed
unnecessarily noisy, so I dropped it. However, I don't feel overly
strongly about it and can add it back if people think it would be
helpful.

> FWIW, I find the existing error messages pretty readable, but that is
> probably a sign that my mind has been poisoned by using chainlint too
> much already. ;)

Goal achieved.

Re: [PATCH 2/2] chainlint: reduce annotation noise-factor

[Eric Sunshine]

On Thu, Aug 29, 2024 at 6:03 AM Patrick Steinhardt <ps@pks.im> wrote:
> On Thu, Aug 29, 2024 at 05:16:25AM -0400, Eric Sunshine wrote:
> > Note that the preceding change gave all problem annotations a uniform
> > "ERR" prefix which serves as a reasonably suitable replacement needle
> > when searching in a terminal, so loss of "?!" in the output should not
> > be overly problematic.
>
> Okay, now the "ERR" prefix becomes a bit more important because we drop
> the other punctuation. I'm still not much of a fan of it, though. Makes
> me wonder whether we want to take a clue from how compilers nowadays
> format this, e.g. by using "pointers".
>
> So this:
>     7   fish |
>     8   cow ?!AMP?!
>
> Would become this:
>     t/chainlint/pipe.actual:8: error: expected ampersands (&&)
>     7   fish |
>     8   cow
>             ^
>
> While this would be neat, I guess it would also be way more work than
> the current series you have posted. And whether that work is ultimately
> really worth it may be another question. Probably not.

Interestingly, I'm not always a fan of the sort of compiler output you
suggest since I often have more difficulty interpreting the output and
locating the actual problem[*] than if the annotation was merely
inline, sitting immediately next to the problem itself.

Also, the vast majority of the time, chainlint will be flagging a
missing "&&" at the end of line, so with the inline annotation, it's
very easy to see (especially when colored) exactly where the problem
is at a glance.

Hence, the cost of implementing "^" doesn't feel particularly
worthwhile (and, with my limited Git time these days, I'm unlikely to
do so).

[*] This is especially so when dealing with foreign code which is
wider than my 80-column terminal or 80-column editor window, in which
the source text and the "^" may wrap over multiple lines.

Re: [PATCH 2/2] chainlint: reduce annotation noise-factor

[Eric Sunshine]

On Thu, Aug 29, 2024 at 1:10 PM Jeff King <peff@peff.net> wrote:
> On Thu, Aug 29, 2024 at 12:03:37PM +0200, Patrick Steinhardt wrote:
> > Would become this:
> >
> >     t/chainlint/pipe.actual:8: error: expected ampersands (&&)
> >     7         fish |
> >     8         cow
> >             ^
> >     9
> >
> > While this would be neat, I guess it would also be way more work than
> > the current series you have posted. And whether that work is ultimately
> > really worth it may be another question. Probably not.
>
> I think that output is quite readable. One bonus is that it follows the
> usual "quickfix" format, so there's editor support for jumping to the
> problematic spot.

The "quickfix" notation has more appeal (to me) than the "^"
annotation since it is immediately useful in an editor and doesn't
require extra cogitation. A couple concerns which come to mind are
that it makes the output even more noisy (which could be distracting),
and that the path, if relative, might not correspond to the editor's
"cwd" or to a path in the editor's search list, so the promised "jump
to next problem" automation may not materialize.

> It probably is more verbose if you have multiple errors right next to
> each other (since now we just show the annotated source text). But that
> is going to be relatively rare compared to single mistakes, I'd think.

Maybe. Maybe not. A first-draft test by a newcomer might very well
break the &&-chain in many spots.

Nevertheless, I'm not likely to implement this any time soon (or at all).

Re: [PATCH 1/2] chainlint: make error messages self-explanatory

[Eric Sunshine]

On Thu, Aug 29, 2024 at 11:39 AM Junio C Hamano <gitster@pobox.com> wrote:
> Eric Sunshine <ericsunshine@charter.net> writes:
> > "?!LOOP?!" case is particularly serious since it is likely that some
> > newcomers are unaware that shell loops do not terminate automatically
> > upon error, and it is more difficult for a newcomer to figure out how to
> > correct the problem by examining surrounding code since `|| return 1`
> > appears in test scrips relatively infrequently (compared, for instance,
> > with &&-chaining).
>
> I'd prefer to see "some newcomes are unaware that" part rewritten
> and toned down, as it is not our primary business to help total
> newbies to learn shells, it certainly is not what the chain lint
> checker should bend over backwards to do.
>
>     ... particularly serious, as it does not convey that returning
>     control with "|| return 1" (or "|| exit 1" from a subshell)
>     immediately after we detect an error is the canonical way we
>     chose in this project to handle errors in a loop.  Because it
>     happens relatively infrequently, this norm is harder to figure
>     out for a new person on their own than other patterns (like
>     &&-chaining).

How about this?

    The "?!LOOP?!" case is particularly serious because that terse
    single word does nothing to convey that the loop body should end
    with "|| return 1" (or "|| exit 1" in a subshell) to ensure that a
    failing command in the body aborts the loop immediately, which is
    important since a shell loop does not automatically terminate when
    an error occurs within its body. Moreover, unlike &&-chaining
    which is ubiquitous in Git tests, the "|| return 1" idiom is
    relatively infrequent, thus may be harder for a newcomer to
    discover by consulting nearby code.

> > -# name and the test body with a `?!FOO?!` annotation at the location of each
> > +# name and the test body with a `?!ERR?!` annotation at the location of each
> >  # detected problem, where "FOO" is a tag such as "AMP" which indicates a broken
>
> "FOO" -> "ERR"?

Yep. Sharp eyes.

Re: [PATCH 1/2] chainlint: make error messages self-explanatory

[Junio C Hamano]

Eric Sunshine <sunshine@sunshineco.com> writes:

> How about this?
>
>     The "?!LOOP?!" case is particularly serious because that terse
>     single word does nothing to convey that the loop body should end
>     with "|| return 1" (or "|| exit 1" in a subshell) to ensure that a
>     failing command in the body aborts the loop immediately, which is
>     important since a shell loop does not automatically terminate when
>     an error occurs within its body. Moreover, unlike &&-chaining
>     which is ubiquitous in Git tests, the "|| return 1" idiom is
>     relatively infrequent, thus may be harder for a newcomer to
>     discover by consulting nearby code.

Strike ", which is important since .*\ its body." and the above
reads perfect.

>> > -# name and the test body with a `?!FOO?!` annotation at the location of each
>> > +# name and the test body with a `?!ERR?!` annotation at the location of each
>> >  # detected problem, where "FOO" is a tag such as "AMP" which indicates a broken
>>
>> "FOO" -> "ERR"?
>
> Yep. Sharp eyes.

OK.  I'll mark the topic to be expecting a reroll for these small
messaging plus "ERR" -> "ERR:" but without other larger changes
mentioned in the thread.

Thanks.

Re: [PATCH 2/2] chainlint: reduce annotation noise-factor

[Eric Sunshine]

On Thu, Aug 29, 2024 at 11:55 AM Junio C Hamano <gitster@pobox.com> wrote:
> Eric Sunshine <ericsunshine@charter.net> writes:
> > When chainlint detects a problem in a test definition, it highlights the
> > offending code with an "?!ERR ...?!" annotation. The rather curious "?!"
> > delimiter was chosen to draw the reader's attention to the problem area.
> >
> > Later, chainlint learned to color its output when sent to a terminal.
> > Problem annotations are colored with a red background which stands out
> > well from surrounding text, thus easily draws the reader's attention. As
> > such, the additional "?!" decoration became superfluous (when output is
> > colored), however the decoration was retained since it serves as a good
> > needle when using the terminal's search feature to "jump" to the next
> > problem.
> >
> > Nevertheless, the "?!" decoration is noisy and ugly and makes it
> > unnecessarily difficult for the reader to pluck the problem description
> > from the annotation. For instance, it is easier to see at a glance what
> > the problem is in:
> >     ERR missing '&&'
> > than in the noisier:
> >     ?!ERR missing '&&'?!
> > Therefore drop the "!?" decoration when output is colored (but retain it
> > otherwise).
>
> Wait.  That does not qualify "Therefore".
>
> We talked about a "good needle" and then complained how ugly the
> string that was happened to be chosen as good needle is.  That is
> not enough to explain why it is justified to "lose" the needle.  The
> only thing you justified is to move away from the ugly pattern, as a
> typical "terminal's search feature" does not give us an easy way to
> "jump to the next text painted yellow".
>
> > Note that the preceding change gave all problem annotations a uniform
> > "ERR" prefix which serves as a reasonably suitable replacement needle
> > when searching in a terminal, so loss of "?!" in the output should not
> > be overly problematic.
>
> Drop this separate paragraph, promote its contents up from "Note"
> status and as a proper part of the previous sentence in its rewrite,
> something like:
>
>     Since the errors are all uniformly prefixed with "ERR", which
>     can be used as the "good needle" instead, lose the "!?"
>     decoration when output is colored.
>
> to replace "Therefore" and everything that follow.

Perhaps the following would make for a more palatable commit message?

    When chainlint detects a problem in a test definition, it
    highlights the offending code with a "?!...?!" annotation. The
    rather curious "?!" decoration was chosen to draw the reader's
    attention to the problem area and to act as a good "needle" when
    using the terminal's search feature to "jump" to the next problem.

    Later, chainlint learned to color its output when sent to a
    terminal. Problem annotations are colored with a red background
    which stands out well from surrounding text, thus easily draws the
    reader's attention. Together with the preceding change which gave
    all problem annotations a uniform "ERR" prefix, the noisy "?!"
    decoration has become superfluous as a search "needle" so omit it
    when output is colored.

> > @@ -663,7 +666,7 @@ sub check_test {
> >       $checked =~ s/(\s) \?!/$1?!/mg;
> >       $checked =~ s/\?! (\s)/?!$1/mg;
> > -     $checked =~ s/(\?![^?]+\?!)/$c->{rev}$c->{red}$1$c->{reset}/mg;
> > +     $checked =~ s/\?!([^?]+)\?!/$erropen$1$errclose/mg;
>
> Hmph.  With $erropen and $errclose, I was hoping that we can shed
> the reliance on the "?!" mark even internally.

Good point. Just above the shown context:

    $checked .= substr($body, $start, $pos - $start) . " ?!ERR $err?! ";

unconditionally adds the "?!" decorations even when the output is
colored, and then the:

    $checked =~ s/\?!([^?]+)\?!/$erropen$1$errclose/mg;

removes them. It would be nice to add the "?!" decoration only when
not coloring output, however, together with the explicit space added
before and after "?!...?!" by:

    $checked .= substr($body, $start, $pos - $start) . " ?!ERR $err?! ";

the related:

    $checked =~ s/(\s) \?!/$1?!/mg;
    $checked =~ s/\?! (\s)/?!$1/mg;

ensure, for aesthetic reasons, that there is one, and only one, space
before and after the annotation.

It may be possible to do something like this instead (untested), but
I'm not sure it's worth the complexity:

    $checked .= substr($body, $start, $pos - $start);
    $checked .= ' ' unless $checked =~ /\s$/;
    $checked .= "$erropenERR $err$errclose";
    $checked .= ' ' unless $pos + 1 >= length($body) ||
        substr($body, $pos + 1, 1) =~ /\s/;

> This is especially
> true that in the early part of this sub, the problem description was
> very much structured piece of data, not something the consuming code
> need to pick out of an already formatted text like this, risking to
> get confused by the payload (i.e. the text that came from the
> problematic test script inside "substr($body, $start, $pos-$start)"
> may contain anything, including "?!", right?).

As first implemented, there was no structured "problem description".
chainlint originally just output a stream of raw parse tokens (not the
original test text), and when a problem was discovered the "?!...?!"
annotations were embedded directly in the output stream. This was
still the case even when colored output was implemented[1]; in fact,
the annotations were colored after-the-fact by searching for "?!...?!"
in the output stream. It was only when chainlint was taught to output
the original test text verbatim[2] that problem descriptions became
structured data.

I was never overly concerned about "?!" appearing as part of the
actual payload, partly because it is an unusual character sequence to
be present in shell code anyhow (indeed, it only appears in t5510),
partly because it requires "?!" to be doubled up (i.e. "?!...?!"), and
partly because this processing kicks in only when a linting problem is
actually discovered,

[1]: 7c04aa7390 (chainlint: colorize problem annotations and test
delimiters, 2022-09-13)
[2]: 73c768dae9 (chainlint: annotate original test definition rather
than token stream, 2022-11-08)

Re: [PATCH 2/2] chainlint: reduce annotation noise-factor

[Junio C Hamano]

Eric Sunshine <sunshine@sunshineco.com> writes:

> It may be possible to do something like this instead (untested), but
> I'm not sure it's worth the complexity:
>
>     $checked .= substr($body, $start, $pos - $start);
>     $checked .= ' ' unless $checked =~ /\s$/;
>     $checked .= "$erropenERR $err$errclose";
>     $checked .= ' ' unless $pos + 1 >= length($body) ||
>         substr($body, $pos + 1, 1) =~ /\s/;

I think the complexity you mention is the updates to existing code
to get to the above end state?  Using some setup like ...

	($erropen, errclose) = 
		$colored_output ? ("?!", "?!") : ("<RED>", "<RESET>");

... and then using a code like the above would be quite
straightforward and the end result cannot become simpler than that
;-)

> As first implemented, there was no structured "problem description".
> chainlint originally just output a stream of raw parse tokens (not the
> original test text), and when a problem was discovered the "?!...?!"
> annotations were embedded directly in the output stream. This was
> still the case even when colored output was implemented[1]; in fact,
> the annotations were colored after-the-fact by searching for "?!...?!"
> in the output stream. It was only when chainlint was taught to output
> the original test text verbatim[2] that problem descriptions became
> structured data.

Exactly.

Thanks.

[PATCH v2 0/3] make chainlint output more newcomer-friendly

[Eric Sunshine]

From: Eric Sunshine <sunshine@sunshineco.com>

This is a reroll of [1] which (hopefully) makes chainlint's problem
annotations more friendly and helpful for newcomers. In particular, it
ditches the cryptic "AMP", "LOOP", etc. and replaces them with proper
error messages.

Changes since v1:

* new patch [1/3] -- motivated by Junio's observation[2] about
  availability of structured problem information -- takes advantage of
  that information directly rather than post-processing "?!...?!"
  sequences in the output stream

* old patch [2/2] (now [3/3]) which drops "?!" decorations when emitting
  colored output to a terminal partially justified the change by
  claiming that the new "ERR" (or "ERR:") prefix is a good "needle" for
  a terminal's search feature, thus the noisy "?!" is no longer needed;
  however, I realized that "ERR" (or "ERR:") is, in fact, an awful
  needle since the string "err" (or "err:") is quite likely to
  legitimately appear in source text, hence I changed the prefix to
  "LINT:" (with the colon since Patrick found lack of colon
  confusing[3])

* rewrote commit messages based upon feedback from Junio[2,4]

* dropped an unused argument from the call to format_problem() which was
  an artifact used briefly during development of v1

Unfortunately, the included range-diff is a mess and pretty much useless
since half the changes from old patch [2/2] (now [3/3]) migrated to new
patch [1/3], and range-diff thinks that [1/3] is a rewrite of old [2/2]
and that old [2/2] is now entirely new [3/3] which is exactly opposite
what really happened, and I wasn't able to convince range-diff to
reconsider. Hence, I also included an interdiff which is easier to read
but doesn't show improvements to the commit messages.

[1]: https://lore.kernel.org/git/20240829091625.41297-1-ericsunshine@charter.net/
[2]: https://lore.kernel.org/git/xmqqv7zjwcgq.fsf@gitster.g/
[3]: https://lore.kernel.org/git/ZtBHbftK7vdTEz93@tanuki/
[4]: https://lore.kernel.org/git/xmqq7cbzxrry.fsf@gitster.g/

Eric Sunshine (3):
  chainlint: don't be fooled by "?!...?!" in test body
  chainlint: make error messages self-explanatory
  chainlint: reduce annotation noise-factor

 t/chainlint.pl                                | 42 +++++++++++++------
 t/chainlint/arithmetic-expansion.expect       |  2 +-
 t/chainlint/block.expect                      |  8 ++--
 t/chainlint/broken-chain.expect               |  2 +-
 t/chainlint/case.expect                       |  4 +-
 t/chainlint/chain-break-false.expect          |  2 +-
 t/chainlint/chained-block.expect              |  2 +-
 t/chainlint/chained-subshell.expect           |  4 +-
 t/chainlint/command-substitution.expect       |  2 +-
 t/chainlint/complex-if-in-cuddled-loop.expect |  2 +-
 t/chainlint/cuddled.expect                    |  4 +-
 t/chainlint/for-loop.expect                   |  8 ++--
 t/chainlint/function.expect                   |  4 +-
 t/chainlint/here-doc-body-indent.expect       |  2 +-
 t/chainlint/here-doc-body-pathological.expect |  4 +-
 t/chainlint/here-doc-body.expect              |  4 +-
 t/chainlint/here-doc-double.expect            |  2 +-
 t/chainlint/here-doc-indent-operator.expect   |  2 +-
 .../here-doc-multi-line-command-subst.expect  |  2 +-
 t/chainlint/here-doc-multi-line-string.expect |  2 +-
 t/chainlint/if-condition-split.expect         |  2 +-
 t/chainlint/if-in-loop.expect                 |  4 +-
 t/chainlint/if-then-else.expect               |  4 +-
 t/chainlint/inline-comment.expect             |  2 +-
 t/chainlint/loop-detect-failure.expect        |  2 +-
 t/chainlint/loop-in-if.expect                 |  8 ++--
 t/chainlint/multi-line-string.expect          |  2 +-
 t/chainlint/negated-one-liner.expect          |  4 +-
 t/chainlint/nested-cuddled-subshell.expect    |  6 +--
 t/chainlint/nested-here-doc.expect            |  2 +-
 t/chainlint/nested-loop-detect-failure.expect |  6 +--
 t/chainlint/nested-subshell-comment.expect    |  2 +-
 t/chainlint/nested-subshell.expect            |  2 +-
 t/chainlint/not-heredoc.expect                |  2 +-
 t/chainlint/one-liner-for-loop.expect         |  2 +-
 t/chainlint/one-liner.expect                  |  6 +--
 t/chainlint/pipe.expect                       |  2 +-
 t/chainlint/semicolon.expect                  | 12 +++---
 t/chainlint/subshell-here-doc.expect          |  2 +-
 t/chainlint/subshell-one-liner.expect         | 10 ++---
 t/chainlint/token-pasting.expect              |  8 ++--
 t/chainlint/unclosed-here-doc-indent.expect   |  2 +-
 t/chainlint/unclosed-here-doc.expect          |  2 +-
 t/chainlint/while-loop.expect                 |  8 ++--
 t/test-lib.sh                                 |  2 +-
 45 files changed, 113 insertions(+), 95 deletions(-)


Interdiff against v1:
diff --git a/t/chainlint.pl b/t/chainlint.pl
index 971ab9212a..f0598e3934 100755
--- a/t/chainlint.pl
+++ b/t/chainlint.pl
@@ -9,9 +9,9 @@
 # Input arguments are pathnames of shell scripts containing test definitions,
 # or globs referencing a collection of scripts. For each problem discovered,
 # the pathname of the script containing the test is printed along with the test
-# name and the test body with a `?!ERR?!` annotation at the location of each
-# detected problem, where "FOO" is a tag such as "AMP" which indicates a broken
-# &&-chain. Returns zero if no problems are discovered, otherwise non-zero.
+# name and the test body with a `?!LINT: ...?!` annotation at the location of
+# each detected problem, where "..." is an explanation of the problem. Returns
+# zero if no problems are discovered, otherwise non-zero.
 
 use warnings;
 use strict;
@@ -657,16 +657,17 @@ sub check_test {
 	for (sort {$a->[1]->[2] <=> $b->[1]->[2]} @$problems) {
 		my ($label, $token) = @$_;
 		my $pos = $token->[2];
-		my $err = format_problem($label, $token);
-		$checked .= substr($body, $start, $pos - $start) . " ?!ERR $err?! ";
+		my $err = format_problem($label);
+		$checked .= substr($body, $start, $pos - $start);
+		$checked .= ' ' unless $checked =~ /\s$/;
+		$checked .= "${erropen}LINT: $err$errclose";
+		$checked .= ' ' unless $pos >= length($body) ||
+		    substr($body, $pos, 1) =~ /^\s/;
 		$start = $pos;
 	}
 	$checked .= substr($body, $start);
 	$checked =~ s/^/$lineno++ . ' '/mge;
 	$checked =~ s/^\d+ \n//;
-	$checked =~ s/(\s) \?!/$1?!/mg;
-	$checked =~ s/\?! (\s)/?!$1/mg;
-	$checked =~ s/\?!([^?]+)\?!/$erropen$1$errclose/mg;
 	$checked =~ s/^\d+/$c->{dim}$&$c->{reset}/mg;
 	$checked .= "\n" unless $checked =~ /\n$/;
 	push(@{$self->{output}}, "$c->{blue}# chainlint: $title$c->{reset}\n$checked");
diff --git a/t/chainlint/arithmetic-expansion.expect b/t/chainlint/arithmetic-expansion.expect
index 2efd65dcbd..5677e16cad 100644
--- a/t/chainlint/arithmetic-expansion.expect
+++ b/t/chainlint/arithmetic-expansion.expect
@@ -4,6 +4,6 @@
 5 	baz
 6 ) &&
 7 (
-8 	bar=$((42 + 1)) ?!ERR missing '&&'?!
+8 	bar=$((42 + 1)) ?!LINT: missing '&&'?!
 9 	baz
 10 )
diff --git a/t/chainlint/block.expect b/t/chainlint/block.expect
index 28410b33ef..3d3f854c0d 100644
--- a/t/chainlint/block.expect
+++ b/t/chainlint/block.expect
@@ -1,20 +1,20 @@
 2 (
 3 	foo &&
 4 	{
-5 		echo a ?!ERR missing '&&'?!
+5 		echo a ?!LINT: missing '&&'?!
 6 		echo b
 7 	} &&
 8 	bar &&
 9 	{
 10 		echo c
-11 	} ?!ERR missing '&&'?!
+11 	} ?!LINT: missing '&&'?!
 12 	baz
 13 ) &&
 14 
 15 {
-16 	echo a; ?!ERR missing '&&'?! echo b
+16 	echo a; ?!LINT: missing '&&'?! echo b
 17 } &&
-18 { echo a; ?!ERR missing '&&'?! echo b; } &&
+18 { echo a; ?!LINT: missing '&&'?! echo b; } &&
 19 
 20 {
 21 	echo "${var}9" &&
diff --git a/t/chainlint/broken-chain.expect b/t/chainlint/broken-chain.expect
index 2a209df0a7..b7b1ce8509 100644
--- a/t/chainlint/broken-chain.expect
+++ b/t/chainlint/broken-chain.expect
@@ -1,6 +1,6 @@
 2 (
 3 	foo &&
-4 	bar ?!ERR missing '&&'?!
+4 	bar ?!LINT: missing '&&'?!
 5 	baz &&
 6 	wop
 7 )
diff --git a/t/chainlint/case.expect b/t/chainlint/case.expect
index d00b67b766..0a3b09e470 100644
--- a/t/chainlint/case.expect
+++ b/t/chainlint/case.expect
@@ -9,11 +9,11 @@
 10 	case "$x" in
 11 	x) foo ;;
 12 	*) bar ;;
-13 	esac ?!ERR missing '&&'?!
+13 	esac ?!LINT: missing '&&'?!
 14 	foobar
 15 ) &&
 16 (
 17 	case "$x" in 1) true;; esac &&
-18 	case "$y" in 2) false;; esac ?!ERR missing '&&'?!
+18 	case "$y" in 2) false;; esac ?!LINT: missing '&&'?!
 19 	foobar
 20 )
diff --git a/t/chainlint/chain-break-false.expect b/t/chainlint/chain-break-false.expect
index bfccc0d90f..f6a0a301e9 100644
--- a/t/chainlint/chain-break-false.expect
+++ b/t/chainlint/chain-break-false.expect
@@ -4,6 +4,6 @@
 5 	echo failed!
 6 	false
 7 else
-8 	echo it went okay ?!ERR missing '&&'?!
+8 	echo it went okay ?!LINT: missing '&&'?!
 9 	congratulate user
 10 fi
diff --git a/t/chainlint/chained-block.expect b/t/chainlint/chained-block.expect
index 293d9eac42..f2501bba90 100644
--- a/t/chainlint/chained-block.expect
+++ b/t/chainlint/chained-block.expect
@@ -1,5 +1,5 @@
 2 echo nobody home && {
-3 	test the doohicky ?!ERR missing '&&'?!
+3 	test the doohicky ?!LINT: missing '&&'?!
 4 	right now
 5 } &&
 6 
diff --git a/t/chainlint/chained-subshell.expect b/t/chainlint/chained-subshell.expect
index 2f5de4fead..93fb1a6578 100644
--- a/t/chainlint/chained-subshell.expect
+++ b/t/chainlint/chained-subshell.expect
@@ -1,10 +1,10 @@
 2 mkdir sub && (
 3 	cd sub &&
-4 	foo the bar ?!ERR missing '&&'?!
+4 	foo the bar ?!LINT: missing '&&'?!
 5 	nuff said
 6 ) &&
 7 
 8 cut "-d " -f actual | (read s1 s2 s3 &&
-9 test -f $s1 ?!ERR missing '&&'?!
+9 test -f $s1 ?!LINT: missing '&&'?!
 10 test $(cat $s2) = tree2path1 &&
 11 test $(cat $s3) = tree3path1)
diff --git a/t/chainlint/command-substitution.expect b/t/chainlint/command-substitution.expect
index 511c918cb5..73809fd585 100644
--- a/t/chainlint/command-substitution.expect
+++ b/t/chainlint/command-substitution.expect
@@ -4,6 +4,6 @@
 5 	baz
 6 ) &&
 7 (
-8 	bar=$(gobble blocks) ?!ERR missing '&&'?!
+8 	bar=$(gobble blocks) ?!LINT: missing '&&'?!
 9 	baz
 10 )
diff --git a/t/chainlint/complex-if-in-cuddled-loop.expect b/t/chainlint/complex-if-in-cuddled-loop.expect
index eb855378a1..e66bb2d5d0 100644
--- a/t/chainlint/complex-if-in-cuddled-loop.expect
+++ b/t/chainlint/complex-if-in-cuddled-loop.expect
@@ -4,6 +4,6 @@
 5      :
 6    else
 7      echo >file
-8    fi ?!ERR missing '|| exit 1'?!
+8    fi ?!LINT: missing '|| exit 1'?!
 9  done) &&
 10 test ! -f file
diff --git a/t/chainlint/cuddled.expect b/t/chainlint/cuddled.expect
index 65825c6879..1864b3fc8b 100644
--- a/t/chainlint/cuddled.expect
+++ b/t/chainlint/cuddled.expect
@@ -2,7 +2,7 @@
 3 	bar
 4 ) &&
 5 
-6 (cd foo ?!ERR missing '&&'?!
+6 (cd foo ?!LINT: missing '&&'?!
 7 	bar
 8 ) &&
 9 
@@ -13,5 +13,5 @@
 14 (cd foo &&
 15 	bar) &&
 16 
-17 (cd foo ?!ERR missing '&&'?!
+17 (cd foo ?!LINT: missing '&&'?!
 18 	bar)
diff --git a/t/chainlint/for-loop.expect b/t/chainlint/for-loop.expect
index df6fc1a35f..5029eacce3 100644
--- a/t/chainlint/for-loop.expect
+++ b/t/chainlint/for-loop.expect
@@ -1,14 +1,14 @@
 2 (
 3 	for i in a b c
 4 	do
-5 		echo $i ?!ERR missing '&&'?!
-6 		cat <<-\EOF ?!ERR missing '|| exit 1'?!
+5 		echo $i ?!LINT: missing '&&'?!
+6 		cat <<-\EOF ?!LINT: missing '|| exit 1'?!
 7 		bar
 8 		EOF
-9 	done ?!ERR missing '&&'?!
+9 	done ?!LINT: missing '&&'?!
 10 
 11 	for i in a b c; do
 12 		echo $i &&
-13 		cat $i ?!ERR missing '|| exit 1'?!
+13 		cat $i ?!LINT: missing '|| exit 1'?!
 14 	done
 15 )
diff --git a/t/chainlint/function.expect b/t/chainlint/function.expect
index 7a15be745b..9e46a3554a 100644
--- a/t/chainlint/function.expect
+++ b/t/chainlint/function.expect
@@ -4,8 +4,8 @@
 5 
 6 remove_object() {
 7 	file=$(sha1_file "$*") &&
-8 	test -e "$file" ?!ERR missing '&&'?!
+8 	test -e "$file" ?!LINT: missing '&&'?!
 9 	rm -f "$file"
-10 } ?!ERR missing '&&'?!
+10 } ?!LINT: missing '&&'?!
 11 
 12 sha1_file arg && remove_object arg
diff --git a/t/chainlint/here-doc-body-indent.expect b/t/chainlint/here-doc-body-indent.expect
index 1d7298c8ad..4306faee86 100644
--- a/t/chainlint/here-doc-body-indent.expect
+++ b/t/chainlint/here-doc-body-indent.expect
@@ -1,2 +1,2 @@
-2 	echo "we should find this" ?!ERR missing '&&'?!
+2 	echo "we should find this" ?!LINT: missing '&&'?!
 3 	echo "even though our heredoc has its indent stripped"
diff --git a/t/chainlint/here-doc-body-pathological.expect b/t/chainlint/here-doc-body-pathological.expect
index 828f232616..2f8ea03a47 100644
--- a/t/chainlint/here-doc-body-pathological.expect
+++ b/t/chainlint/here-doc-body-pathological.expect
@@ -1,7 +1,7 @@
-2 	echo "outer here-doc does not allow indented end-tag" ?!ERR missing '&&'?!
+2 	echo "outer here-doc does not allow indented end-tag" ?!LINT: missing '&&'?!
 3 	cat >file <<-\EOF &&
 4 	but this inner here-doc
 5 	does allow indented EOF
 6 	EOF
-7 	echo "missing chain after" ?!ERR missing '&&'?!
+7 	echo "missing chain after" ?!LINT: missing '&&'?!
 8 	echo "but this line is OK because it's the end"
diff --git a/t/chainlint/here-doc-body.expect b/t/chainlint/here-doc-body.expect
index 79b9603c1e..df8d79bc0a 100644
--- a/t/chainlint/here-doc-body.expect
+++ b/t/chainlint/here-doc-body.expect
@@ -1,7 +1,7 @@
-2 	echo "missing chain before" ?!ERR missing '&&'?!
+2 	echo "missing chain before" ?!LINT: missing '&&'?!
 3 	cat >file <<-\EOF &&
 4 	inside inner here-doc
 5 	these are not shell commands
 6 	EOF
-7 	echo "missing chain after" ?!ERR missing '&&'?!
+7 	echo "missing chain after" ?!LINT: missing '&&'?!
 8 	echo "but this line is OK because it's the end"
diff --git a/t/chainlint/here-doc-double.expect b/t/chainlint/here-doc-double.expect
index 9cb1a1a5e3..e5e981889f 100644
--- a/t/chainlint/here-doc-double.expect
+++ b/t/chainlint/here-doc-double.expect
@@ -1,2 +1,2 @@
-8 	echo "actual test commands" ?!ERR missing '&&'?!
+8 	echo "actual test commands" ?!LINT: missing '&&'?!
 9 	echo "that should be checked"
diff --git a/t/chainlint/here-doc-indent-operator.expect b/t/chainlint/here-doc-indent-operator.expect
index 2d61e5f49d..ec0e61505b 100644
--- a/t/chainlint/here-doc-indent-operator.expect
+++ b/t/chainlint/here-doc-indent-operator.expect
@@ -4,7 +4,7 @@
 5 chunks: oid_fanout oid_lookup commit_metadata generation_data bloom_indexes bloom_data
 6 EOF
 7 
-8 cat >expect << -EOF ?!ERR missing '&&'?!
+8 cat >expect << -EOF ?!LINT: missing '&&'?!
 9 this is not indented
 10 -EOF
 11 
diff --git a/t/chainlint/here-doc-multi-line-command-subst.expect b/t/chainlint/here-doc-multi-line-command-subst.expect
index 881e4d2098..8128f15b92 100644
--- a/t/chainlint/here-doc-multi-line-command-subst.expect
+++ b/t/chainlint/here-doc-multi-line-command-subst.expect
@@ -3,6 +3,6 @@
 4 		fossil
 5 		vegetable
 6 		END
-7 		wiffle) ?!ERR missing '&&'?!
+7 		wiffle) ?!LINT: missing '&&'?!
 8 	echo $x
 9 )
diff --git a/t/chainlint/here-doc-multi-line-string.expect b/t/chainlint/here-doc-multi-line-string.expect
index 06c791e0a4..a03a04ff3d 100644
--- a/t/chainlint/here-doc-multi-line-string.expect
+++ b/t/chainlint/here-doc-multi-line-string.expect
@@ -1,6 +1,6 @@
 2 (
 3 	cat <<-\TXT && echo "multi-line
-4 	string" ?!ERR missing '&&'?!
+4 	string" ?!LINT: missing '&&'?!
 5 	fizzle
 6 	TXT
 7 	bap
diff --git a/t/chainlint/if-condition-split.expect b/t/chainlint/if-condition-split.expect
index 5688d93a4f..6d2a03dfdb 100644
--- a/t/chainlint/if-condition-split.expect
+++ b/t/chainlint/if-condition-split.expect
@@ -2,6 +2,6 @@
 3    marcia ||
 4    kevin
 5 then
-6 	echo "nomads" ?!ERR missing '&&'?!
+6 	echo "nomads" ?!LINT: missing '&&'?!
 7 	echo "for sure"
 8 fi
diff --git a/t/chainlint/if-in-loop.expect b/t/chainlint/if-in-loop.expect
index 253b461f87..7e3ba740de 100644
--- a/t/chainlint/if-in-loop.expect
+++ b/t/chainlint/if-in-loop.expect
@@ -5,8 +5,8 @@
 6 		then
 7 			echo "err"
 8 			exit 1
-9 		fi ?!ERR missing '&&'?!
+9 		fi ?!LINT: missing '&&'?!
 10 		foo
-11 	done ?!ERR missing '&&'?!
+11 	done ?!LINT: missing '&&'?!
 12 	bar
 13 )
diff --git a/t/chainlint/if-then-else.expect b/t/chainlint/if-then-else.expect
index 1b3162759f..924caa2e4e 100644
--- a/t/chainlint/if-then-else.expect
+++ b/t/chainlint/if-then-else.expect
@@ -1,7 +1,7 @@
 2 (
 3 	if test -n ""
 4 	then
-5 		echo very ?!ERR missing '&&'?!
+5 		echo very ?!LINT: missing '&&'?!
 6 		echo empty
 7 	elif test -z ""
 8 	then
@@ -11,7 +11,7 @@
 12 		cat <<-\EOF
 13 		bar
 14 		EOF
-15 	fi ?!ERR missing '&&'?!
+15 	fi ?!LINT: missing '&&'?!
 16 	echo poodle
 17 ) &&
 18 (
diff --git a/t/chainlint/inline-comment.expect b/t/chainlint/inline-comment.expect
index fedc059a0c..4b4080124e 100644
--- a/t/chainlint/inline-comment.expect
+++ b/t/chainlint/inline-comment.expect
@@ -1,6 +1,6 @@
 2 (
 3 	foobar && # comment 1
-4 	barfoo ?!ERR missing '&&'?! # wrong position for &&
+4 	barfoo ?!LINT: missing '&&'?! # wrong position for &&
 5 	flibble "not a # comment"
 6 ) &&
 7 
diff --git a/t/chainlint/loop-detect-failure.expect b/t/chainlint/loop-detect-failure.expect
index 2d46f6d2eb..7d846b878d 100644
--- a/t/chainlint/loop-detect-failure.expect
+++ b/t/chainlint/loop-detect-failure.expect
@@ -11,5 +11,5 @@
 12 do
 13 	printf "%"$n"s" X > r2/large.$n &&
 14 	git -C r2 add large.$n &&
-15 	git -C r2 commit -m "$n" ?!ERR missing '|| return 1'?!
+15 	git -C r2 commit -m "$n" ?!LINT: missing '|| return 1'?!
 16 done
diff --git a/t/chainlint/loop-in-if.expect b/t/chainlint/loop-in-if.expect
index 8936d7ff2d..32e076ad1b 100644
--- a/t/chainlint/loop-in-if.expect
+++ b/t/chainlint/loop-in-if.expect
@@ -3,10 +3,10 @@
 4 	then
 5 		while true
 6 		do
-7 			echo "pop" ?!ERR missing '&&'?!
-8 			echo "glup" ?!ERR missing '|| exit 1'?!
-9 		done ?!ERR missing '&&'?!
+7 			echo "pop" ?!LINT: missing '&&'?!
+8 			echo "glup" ?!LINT: missing '|| exit 1'?!
+9 		done ?!LINT: missing '&&'?!
 10 		foo
-11 	fi ?!ERR missing '&&'?!
+11 	fi ?!LINT: missing '&&'?!
 12 	bar
 13 )
diff --git a/t/chainlint/multi-line-string.expect b/t/chainlint/multi-line-string.expect
index 3c3a1de75c..9d33297525 100644
--- a/t/chainlint/multi-line-string.expect
+++ b/t/chainlint/multi-line-string.expect
@@ -3,7 +3,7 @@
 4 		line 2
 5 		line 3" &&
 6 	y="line 1
-7 		line2" ?!ERR missing '&&'?!
+7 		line2" ?!LINT: missing '&&'?!
 8 	foobar
 9 ) &&
 10 (
diff --git a/t/chainlint/negated-one-liner.expect b/t/chainlint/negated-one-liner.expect
index 12bd65264a..0a6f3c29b2 100644
--- a/t/chainlint/negated-one-liner.expect
+++ b/t/chainlint/negated-one-liner.expect
@@ -1,5 +1,5 @@
 2 ! (foo && bar) &&
 3 ! (foo && bar) >baz &&
 4 
-5 ! (foo; ?!ERR missing '&&'?! bar) &&
-6 ! (foo; ?!ERR missing '&&'?! bar) >baz
+5 ! (foo; ?!LINT: missing '&&'?! bar) &&
+6 ! (foo; ?!LINT: missing '&&'?! bar) >baz
diff --git a/t/chainlint/nested-cuddled-subshell.expect b/t/chainlint/nested-cuddled-subshell.expect
index 3e947ea5e1..fec2c74274 100644
--- a/t/chainlint/nested-cuddled-subshell.expect
+++ b/t/chainlint/nested-cuddled-subshell.expect
@@ -5,7 +5,7 @@
 6 
 7 	(cd foo &&
 8 		bar
-9 	) ?!ERR missing '&&'?!
+9 	) ?!LINT: missing '&&'?!
 10 
 11 	(
 12 		cd foo &&
@@ -13,13 +13,13 @@
 14 
 15 	(
 16 		cd foo &&
-17 		bar) ?!ERR missing '&&'?!
+17 		bar) ?!LINT: missing '&&'?!
 18 
 19 	(cd foo &&
 20 		bar) &&
 21 
 22 	(cd foo &&
-23 		bar) ?!ERR missing '&&'?!
+23 		bar) ?!LINT: missing '&&'?!
 24 
 25 	foobar
 26 )
diff --git a/t/chainlint/nested-here-doc.expect b/t/chainlint/nested-here-doc.expect
index 107e5afb01..571f4c9514 100644
--- a/t/chainlint/nested-here-doc.expect
+++ b/t/chainlint/nested-here-doc.expect
@@ -18,7 +18,7 @@
 19 	toink
 20 	INPUT_END
 21 
-22 	cat <<-\EOT ?!ERR missing '&&'?!
+22 	cat <<-\EOT ?!LINT: missing '&&'?!
 23 	text goes here
 24 	data <<EOF
 25 		data goes here
diff --git a/t/chainlint/nested-loop-detect-failure.expect b/t/chainlint/nested-loop-detect-failure.expect
index 26557b05a1..b4aaa621a2 100644
--- a/t/chainlint/nested-loop-detect-failure.expect
+++ b/t/chainlint/nested-loop-detect-failure.expect
@@ -2,8 +2,8 @@
 3 do
 4 	for j in 0 1 2 3 4 5 6 7 8 9;
 5 	do
-6 		echo "$i$j" >"path$i$j" ?!ERR missing '|| return 1'?!
-7 	done ?!ERR missing '|| return 1'?!
+6 		echo "$i$j" >"path$i$j" ?!LINT: missing '|| return 1'?!
+7 	done ?!LINT: missing '|| return 1'?!
 8 done &&
 9 
 10 for i in 0 1 2 3 4 5 6 7 8 9;
@@ -18,7 +18,7 @@
 19 do
 20 	for j in 0 1 2 3 4 5 6 7 8 9;
 21 	do
-22 		echo "$i$j" >"path$i$j" ?!ERR missing '|| return 1'?!
+22 		echo "$i$j" >"path$i$j" ?!LINT: missing '|| return 1'?!
 23 	done || return 1
 24 done &&
 25 
diff --git a/t/chainlint/nested-subshell-comment.expect b/t/chainlint/nested-subshell-comment.expect
index c6891919c0..078c6f275f 100644
--- a/t/chainlint/nested-subshell-comment.expect
+++ b/t/chainlint/nested-subshell-comment.expect
@@ -6,6 +6,6 @@
 7 		# minor numbers of cows (or do they?)
 8 		baz &&
 9 		snaff
-10 	) ?!ERR missing '&&'?!
+10 	) ?!LINT: missing '&&'?!
 11 	fuzzy
 12 )
diff --git a/t/chainlint/nested-subshell.expect b/t/chainlint/nested-subshell.expect
index b98d723edf..a8d85d5d5b 100644
--- a/t/chainlint/nested-subshell.expect
+++ b/t/chainlint/nested-subshell.expect
@@ -7,7 +7,7 @@
 8 
 9 	cd foo &&
 10 	(
-11 		echo a ?!ERR missing '&&'?!
+11 		echo a ?!LINT: missing '&&'?!
 12 		echo b
 13 	) >file
 14 )
diff --git a/t/chainlint/not-heredoc.expect b/t/chainlint/not-heredoc.expect
index 9910621103..5d51705a7a 100644
--- a/t/chainlint/not-heredoc.expect
+++ b/t/chainlint/not-heredoc.expect
@@ -9,6 +9,6 @@
 10 	echo ourside &&
 11 	echo "=======" &&
 12 	echo theirside &&
-13 	echo ">>>>>>> theirs" ?!ERR missing '&&'?!
+13 	echo ">>>>>>> theirs" ?!LINT: missing '&&'?!
 14 	poodle
 15 ) >merged
diff --git a/t/chainlint/one-liner-for-loop.expect b/t/chainlint/one-liner-for-loop.expect
index 2eb2d5fcaf..e1fcbd3639 100644
--- a/t/chainlint/one-liner-for-loop.expect
+++ b/t/chainlint/one-liner-for-loop.expect
@@ -3,7 +3,7 @@
 4 	cd dir-rename-and-content &&
 5 	test_write_lines 1 2 3 4 5 >foo &&
 6 	mkdir olddir &&
-7 	for i in a b c; do echo $i >olddir/$i; ?!ERR missing '|| exit 1'?! done ?!ERR missing '&&'?!
+7 	for i in a b c; do echo $i >olddir/$i; ?!LINT: missing '|| exit 1'?! done ?!LINT: missing '&&'?!
 8 	git add foo olddir &&
 9 	git commit -m "original" &&
 10 )
diff --git a/t/chainlint/one-liner.expect b/t/chainlint/one-liner.expect
index 2c5826e6c4..5deeb05070 100644
--- a/t/chainlint/one-liner.expect
+++ b/t/chainlint/one-liner.expect
@@ -2,8 +2,8 @@
 3 (foo && bar) |
 4 (foo && bar) >baz &&
 5 
-6 (foo; ?!ERR missing '&&'?! bar) &&
-7 (foo; ?!ERR missing '&&'?! bar) |
-8 (foo; ?!ERR missing '&&'?! bar) >baz &&
+6 (foo; ?!LINT: missing '&&'?! bar) &&
+7 (foo; ?!LINT: missing '&&'?! bar) |
+8 (foo; ?!LINT: missing '&&'?! bar) >baz &&
 9 
 10 (foo "bar; baz")
diff --git a/t/chainlint/pipe.expect b/t/chainlint/pipe.expect
index a198d5bdb2..d947c76584 100644
--- a/t/chainlint/pipe.expect
+++ b/t/chainlint/pipe.expect
@@ -4,7 +4,7 @@
 5 	baz &&
 6 
 7 	fish |
-8 	cow ?!ERR missing '&&'?!
+8 	cow ?!LINT: missing '&&'?!
 9 
 10 	sunder
 11 )
diff --git a/t/chainlint/semicolon.expect b/t/chainlint/semicolon.expect
index e22920bf2c..2b499fbe70 100644
--- a/t/chainlint/semicolon.expect
+++ b/t/chainlint/semicolon.expect
@@ -1,19 +1,19 @@
 2 (
-3 	cat foo ; ?!ERR missing '&&'?! echo bar ?!ERR missing '&&'?!
-4 	cat foo ; ?!ERR missing '&&'?! echo bar
+3 	cat foo ; ?!LINT: missing '&&'?! echo bar ?!LINT: missing '&&'?!
+4 	cat foo ; ?!LINT: missing '&&'?! echo bar
 5 ) &&
 6 (
-7 	cat foo ; ?!ERR missing '&&'?! echo bar &&
-8 	cat foo ; ?!ERR missing '&&'?! echo bar
+7 	cat foo ; ?!LINT: missing '&&'?! echo bar &&
+8 	cat foo ; ?!LINT: missing '&&'?! echo bar
 9 ) &&
 10 (
 11 	echo "foo; bar" &&
-12 	cat foo; ?!ERR missing '&&'?! echo bar
+12 	cat foo; ?!LINT: missing '&&'?! echo bar
 13 ) &&
 14 (
 15 	foo;
 16 ) &&
 17 (cd foo &&
 18 	for i in a b c; do
-19 		echo; ?!ERR missing '|| exit 1'?!
+19 		echo; ?!LINT: missing '|| exit 1'?!
 20 	done)
diff --git a/t/chainlint/subshell-here-doc.expect b/t/chainlint/subshell-here-doc.expect
index 953d8084e5..e450caf948 100644
--- a/t/chainlint/subshell-here-doc.expect
+++ b/t/chainlint/subshell-here-doc.expect
@@ -6,7 +6,7 @@
 7 	nevermore...
 8 	EOF
 9 
-10 	cat <<EOF >bip ?!ERR missing '&&'?!
+10 	cat <<EOF >bip ?!LINT: missing '&&'?!
 11 	fish fly high
 12 EOF
 13 
diff --git a/t/chainlint/subshell-one-liner.expect b/t/chainlint/subshell-one-liner.expect
index f82296db66..265d996a21 100644
--- a/t/chainlint/subshell-one-liner.expect
+++ b/t/chainlint/subshell-one-liner.expect
@@ -3,17 +3,17 @@
 4 	(foo && bar) |
 5 	(foo && bar) >baz &&
 6 
-7 	(foo; ?!ERR missing '&&'?! bar) &&
-8 	(foo; ?!ERR missing '&&'?! bar) |
-9 	(foo; ?!ERR missing '&&'?! bar) >baz &&
+7 	(foo; ?!LINT: missing '&&'?! bar) &&
+8 	(foo; ?!LINT: missing '&&'?! bar) |
+9 	(foo; ?!LINT: missing '&&'?! bar) >baz &&
 10 
 11 	(foo || exit 1) &&
 12 	(foo || exit 1) |
 13 	(foo || exit 1) >baz &&
 14 
-15 	(foo && bar) ?!ERR missing '&&'?!
+15 	(foo && bar) ?!LINT: missing '&&'?!
 16 
-17 	(foo && bar; ?!ERR missing '&&'?! baz) ?!ERR missing '&&'?!
+17 	(foo && bar; ?!LINT: missing '&&'?! baz) ?!LINT: missing '&&'?!
 18 
 19 	foobar
 20 )
diff --git a/t/chainlint/token-pasting.expect b/t/chainlint/token-pasting.expect
index aa64cf75f3..387189b6de 100644
--- a/t/chainlint/token-pasting.expect
+++ b/t/chainlint/token-pasting.expect
@@ -2,13 +2,13 @@
 3 git config filter.rot13.clean ./rot13.sh &&
 4 
 5 {
-6     echo "*.t filter=rot13" ?!ERR missing '&&'?!
+6     echo "*.t filter=rot13" ?!LINT: missing '&&'?!
 7     echo "*.i ident"
 8 } >.gitattributes &&
 9 
 10 {
-11     echo a b c d e f g h i j k l m ?!ERR missing '&&'?!
-12     echo n o p q r s t u v w x y z ?!ERR missing '&&'?!
+11     echo a b c d e f g h i j k l m ?!LINT: missing '&&'?!
+12     echo n o p q r s t u v w x y z ?!LINT: missing '&&'?!
 13     echo '$Id$'
 14 } >test &&
 15 cat test >test.t &&
@@ -19,7 +19,7 @@
 20 git checkout -- test test.t test.i &&
 21 
 22 echo "content-test2" >test2.o &&
-23 echo "content-test3 - filename with special characters" >"test3 'sq',$x=.o" ?!ERR missing '&&'?!
+23 echo "content-test3 - filename with special characters" >"test3 'sq',$x=.o" ?!LINT: missing '&&'?!
 24 
 25 downstream_url_for_sed=$(
 26 	printf "%sn" "$downstream_url" |
diff --git a/t/chainlint/unclosed-here-doc-indent.expect b/t/chainlint/unclosed-here-doc-indent.expect
index d5b9ab52ee..156906c85a 100644
--- a/t/chainlint/unclosed-here-doc-indent.expect
+++ b/t/chainlint/unclosed-here-doc-indent.expect
@@ -1,4 +1,4 @@
 2 command_which_is_run &&
-3 cat >expect <<-\EOF ?!ERR unclosed heredoc?! &&
+3 cat >expect <<-\EOF ?!LINT: unclosed heredoc?! &&
 4 we forget to end the here-doc
 5 command_which_is_gobbled
diff --git a/t/chainlint/unclosed-here-doc.expect b/t/chainlint/unclosed-here-doc.expect
index 8f6d260544..752c608862 100644
--- a/t/chainlint/unclosed-here-doc.expect
+++ b/t/chainlint/unclosed-here-doc.expect
@@ -1,5 +1,5 @@
 2 command_which_is_run &&
-3 cat >expect <<\EOF ?!ERR unclosed heredoc?! &&
+3 cat >expect <<\EOF ?!LINT: unclosed heredoc?! &&
 4 	we try to end the here-doc below,
 5 	but the indentation throws us off
 6 	since the operator is not "<<-".
diff --git a/t/chainlint/while-loop.expect b/t/chainlint/while-loop.expect
index 1cfd17b3c2..2ba5582165 100644
--- a/t/chainlint/while-loop.expect
+++ b/t/chainlint/while-loop.expect
@@ -1,14 +1,14 @@
 2 (
 3 	while true
 4 	do
-5 		echo foo ?!ERR missing '&&'?!
-6 		cat <<-\EOF ?!ERR missing '|| exit 1'?!
+5 		echo foo ?!LINT: missing '&&'?!
+6 		cat <<-\EOF ?!LINT: missing '|| exit 1'?!
 7 		bar
 8 		EOF
-9 	done ?!ERR missing '&&'?!
+9 	done ?!LINT: missing '&&'?!
 10 
 11 	while true; do
 12 		echo foo &&
-13 		cat bar ?!ERR missing '|| exit 1'?!
+13 		cat bar ?!LINT: missing '|| exit 1'?!
 14 	done
 15 )
diff --git a/t/test-lib.sh b/t/test-lib.sh
index b652cb98cd..278d1215f1 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -1606,7 +1606,7 @@ if test "${GIT_TEST_CHAIN_LINT:-1}" != 0 &&
    test "${GIT_TEST_EXT_CHAIN_LINT:-1}" != 0
 then
 	"$PERL_PATH" "$TEST_DIRECTORY/chainlint.pl" "$0" ||
-		BUG "lint error (see 'ERR' annotations above)"
+		BUG "lint error (see 'LINT' annotations above)"
 fi
 
 # Last-minute variable setup


Range-diff against v1:
2:  93305d0bdf ! 1:  260a877ce1 chainlint: reduce annotation noise-factor
    @@ Metadata
     Author: Eric Sunshine <sunshine@sunshineco.com>
     
      ## Commit message ##
    -    chainlint: reduce annotation noise-factor
    +    chainlint: don't be fooled by "?!...?!" in test body
     
    -    When chainlint detects a problem in a test definition, it highlights the
    -    offending code with an "?!ERR ...?!" annotation. The rather curious "?!"
    -    delimiter was chosen to draw the reader's attention to the problem area.
    +    As originally implemented, chainlint did not collect structured
    +    information about detected problems. Instead, it merely emitted raw
    +    parse tokens (not the original test text), along with a "?!...?!"
    +    annotation directly into the output stream each time a problem was
    +    discovered. In order to report statistics (in --stats mode) and to
    +    adjust its exit code to indicate success or failure, it merely counts
    +    the number of times "?!...?!" appears in the output stream. An obvious
    +    shortcoming of this approach is that it can be fooled by a legitimate
    +    "?!...?!" sequence in the body of a test (though, only if an actual
    +    problem is detected in the test).
     
    -    Later, chainlint learned to color its output when sent to a terminal.
    -    Problem annotations are colored with a red background which stands out
    -    well from surrounding text, thus easily draws the reader's attention. As
    -    such, the additional "?!" decoration became superfluous (when output is
    -    colored), however the decoration was retained since it serves as a good
    -    needle when using the terminal's search feature to "jump" to the next
    -    problem.
    +    The situation did not improve when 7c04aa7390 (chainlint: colorize
    +    problem annotations and test delimiters, 2022-09-13) colored the
    +    annotations after-the-fact by searching for "?!...?!" in the output
    +    stream and inserting color codes. As above, a shortcoming is that this
    +    approach can incorrectly color a legitimate "?!...?!" sequence in a test
    +    body as if it is an error.
     
    -    Nevertheless, the "?!" decoration is noisy and ugly and makes it
    -    unnecessarily difficult for the reader to pluck the problem description
    -    from the annotation. For instance, it is easier to see at a glance what
    -    the problem is in:
    +    However, when 73c768dae9 (chainlint: annotate original test definition
    +    rather than token stream, 2022-11-08) taught chainlint to output the
    +    original test text verbatim, it started collecting structured
    +    information about detected problems.
     
    -        ERR missing '&&'
    -
    -    than in the noisier:
    -
    -        ?!ERR missing '&&'?!
    -
    -    Therefore drop the "!?" decoration when output is colored (but retain it
    -    otherwise).
    -
    -    Note that the preceding change gave all problem annotations a uniform
    -    "ERR" prefix which serves as a reasonably suitable replacement needle
    -    when searching in a terminal, so loss of "?!" in the output should not
    -    be overly problematic.
    +    Now that it is available, take advantage of the structured problem
    +    information to deterministically count the number of problems detected
    +    and to color the annotations directly, rather than scanning the output
    +    stream for "?!...?!" and performing these operations after-the-fact.
     
         Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
     
    @@ t/chainlint.pl: sub check_test {
     +	$self->{nerrs} += @$problems;
      	return unless $emit_all || @$problems;
      	my $c = main::fd_colors(1);
    -+	my ($erropen, $errclose) = -t 1 ? ("$c->{rev}$c->{red}", $c->{reset}) : ('?!', '?!');
      	my $start = 0;
    - 	my $checked = '';
    - 	for (sort {$a->[1]->[2] <=> $b->[1]->[2]} @$problems) {
     @@ t/chainlint.pl: sub check_test {
    + 	for (sort {$a->[1]->[2] <=> $b->[1]->[2]} @$problems) {
    + 		my ($label, $token) = @$_;
    + 		my $pos = $token->[2];
    +-		$checked .= substr($body, $start, $pos - $start) . " ?!$label?! ";
    ++		$checked .= substr($body, $start, $pos - $start);
    ++		$checked .= ' ' unless $checked =~ /\s$/;
    ++		$checked .= "$c->{rev}$c->{red}?!$label?!$c->{reset}";
    ++		$checked .= ' ' unless $pos >= length($body) ||
    ++		    substr($body, $pos, 1) =~ /^\s/;
    + 		$start = $pos;
    + 	}
    + 	$checked .= substr($body, $start);
    + 	$checked =~ s/^/$lineno++ . ' '/mge;
      	$checked =~ s/^\d+ \n//;
    - 	$checked =~ s/(\s) \?!/$1?!/mg;
    - 	$checked =~ s/\?! (\s)/?!$1/mg;
    +-	$checked =~ s/(\s) \?!/$1?!/mg;
    +-	$checked =~ s/\?! (\s)/?!$1/mg;
     -	$checked =~ s/(\?![^?]+\?!)/$c->{rev}$c->{red}$1$c->{reset}/mg;
    -+	$checked =~ s/\?!([^?]+)\?!/$erropen$1$errclose/mg;
      	$checked =~ s/^\d+/$c->{dim}$&$c->{reset}/mg;
      	$checked .= "\n" unless $checked =~ /\n$/;
      	push(@{$self->{output}}, "$c->{blue}# chainlint: $title$c->{reset}\n$checked");
    @@ t/chainlint.pl: sub check_script {
      	}
      	return [$id, $nscripts, $ntests, $nerrs];
      }
    -
    - ## t/test-lib.sh ##
    -@@ t/test-lib.sh: if test "${GIT_TEST_CHAIN_LINT:-1}" != 0 &&
    -    test "${GIT_TEST_EXT_CHAIN_LINT:-1}" != 0
    - then
    - 	"$PERL_PATH" "$TEST_DIRECTORY/chainlint.pl" "$0" ||
    --		BUG "lint error (see '?!...!? annotations above)"
    -+		BUG "lint error (see 'ERR' annotations above)"
    - fi
    - 
    - # Last-minute variable setup
1:  0530fc13c6 ! 2:  5a4c1bd31a chainlint: make error messages self-explanatory
    @@ Commit message
         may understand that "?!AMP?!" is an abbreviation of "ampersand" and
         indicates a break in the &&-chain, this may not be obvious to newcomers.
     
    -    Similarly, although the annotation "?!LOOP?!" is understood by project
    -    regulars to indicate a missing `|| return 1` (or `|| exit 1` in a
    -    subshell), newcomers may find it more than a little perplexing. The
    -    "?!LOOP?!" case is particularly serious since it is likely that some
    -    newcomers are unaware that shell loops do not terminate automatically
    -    upon error, and it is more difficult for a newcomer to figure out how to
    -    correct the problem by examining surrounding code since `|| return 1`
    -    appears in test scrips relatively infrequently (compared, for instance,
    -    with &&-chaining).
    +    The "?!LOOP?!" case is particularly serious because that terse single
    +    word does nothing to convey that the loop body should end with
    +    "|| return 1" (or "|| exit 1" in a subshell) to ensure that a failing
    +    command in the body aborts the loop immediately. Moreover, unlike
    +    &&-chaining which is ubiquitous in Git tests, the "|| return 1" idiom is
    +    relatively infrequent, thus may be harder for a newcomer to discover by
    +    consulting nearby code.
     
    -    Address these shortcomings by emitting human-consumable messages which
    +    Address these shortcomings by emitting human-readable messages which
         both explain the problem and give a strong hint about how to correct it.
     
         Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
    @@ t/chainlint.pl
      # or globs referencing a collection of scripts. For each problem discovered,
      # the pathname of the script containing the test is printed along with the test
     -# name and the test body with a `?!FOO?!` annotation at the location of each
    -+# name and the test body with a `?!ERR?!` annotation at the location of each
    - # detected problem, where "FOO" is a tag such as "AMP" which indicates a broken
    - # &&-chain. Returns zero if no problems are discovered, otherwise non-zero.
    +-# detected problem, where "FOO" is a tag such as "AMP" which indicates a broken
    +-# &&-chain. Returns zero if no problems are discovered, otherwise non-zero.
    ++# name and the test body with a `?!LINT: ...?!` annotation at the location of
    ++# each detected problem, where "..." is an explanation of the problem. Returns
    ++# zero if no problems are discovered, otherwise non-zero.
      
    + use warnings;
    + use strict;
     @@ t/chainlint.pl: sub swallow_heredocs {
      			$self->{lineno} += () = $body =~ /\n/sg;
      			next;
    @@ t/chainlint.pl: sub check_test {
      	for (sort {$a->[1]->[2] <=> $b->[1]->[2]} @$problems) {
      		my ($label, $token) = @$_;
      		my $pos = $token->[2];
    --		$checked .= substr($body, $start, $pos - $start) . " ?!$label?! ";
    -+		my $err = format_problem($label, $token);
    -+		$checked .= substr($body, $start, $pos - $start) . " ?!ERR $err?! ";
    ++		my $err = format_problem($label);
    + 		$checked .= substr($body, $start, $pos - $start);
    + 		$checked .= ' ' unless $checked =~ /\s$/;
    +-		$checked .= "$c->{rev}$c->{red}?!$label?!$c->{reset}";
    ++		$checked .= "$c->{rev}$c->{red}?!LINT: $err?!$c->{reset}";
    + 		$checked .= ' ' unless $pos >= length($body) ||
    + 		    substr($body, $pos, 1) =~ /^\s/;
      		$start = $pos;
    - 	}
    - 	$checked .= substr($body, $start);
     
      ## t/chainlint/arithmetic-expansion.expect ##
     @@
    @@ t/chainlint/arithmetic-expansion.expect
      6 ) &&
      7 (
     -8 	bar=$((42 + 1)) ?!AMP?!
    -+8 	bar=$((42 + 1)) ?!ERR missing '&&'?!
    ++8 	bar=$((42 + 1)) ?!LINT: missing '&&'?!
      9 	baz
      10 )
     
    @@ t/chainlint/block.expect
      3 	foo &&
      4 	{
     -5 		echo a ?!AMP?!
    -+5 		echo a ?!ERR missing '&&'?!
    ++5 		echo a ?!LINT: missing '&&'?!
      6 		echo b
      7 	} &&
      8 	bar &&
      9 	{
      10 		echo c
     -11 	} ?!AMP?!
    -+11 	} ?!ERR missing '&&'?!
    ++11 	} ?!LINT: missing '&&'?!
      12 	baz
      13 ) &&
      14 
      15 {
     -16 	echo a; ?!AMP?! echo b
    -+16 	echo a; ?!ERR missing '&&'?! echo b
    ++16 	echo a; ?!LINT: missing '&&'?! echo b
      17 } &&
     -18 { echo a; ?!AMP?! echo b; } &&
    -+18 { echo a; ?!ERR missing '&&'?! echo b; } &&
    ++18 { echo a; ?!LINT: missing '&&'?! echo b; } &&
      19 
      20 {
      21 	echo "${var}9" &&
    @@ t/chainlint/broken-chain.expect
      2 (
      3 	foo &&
     -4 	bar ?!AMP?!
    -+4 	bar ?!ERR missing '&&'?!
    ++4 	bar ?!LINT: missing '&&'?!
      5 	baz &&
      6 	wop
      7 )
    @@ t/chainlint/case.expect
      11 	x) foo ;;
      12 	*) bar ;;
     -13 	esac ?!AMP?!
    -+13 	esac ?!ERR missing '&&'?!
    ++13 	esac ?!LINT: missing '&&'?!
      14 	foobar
      15 ) &&
      16 (
      17 	case "$x" in 1) true;; esac &&
     -18 	case "$y" in 2) false;; esac ?!AMP?!
    -+18 	case "$y" in 2) false;; esac ?!ERR missing '&&'?!
    ++18 	case "$y" in 2) false;; esac ?!LINT: missing '&&'?!
      19 	foobar
      20 )
     
    @@ t/chainlint/chain-break-false.expect
      6 	false
      7 else
     -8 	echo it went okay ?!AMP?!
    -+8 	echo it went okay ?!ERR missing '&&'?!
    ++8 	echo it went okay ?!LINT: missing '&&'?!
      9 	congratulate user
      10 fi
     
    @@ t/chainlint/chained-block.expect
     @@
      2 echo nobody home && {
     -3 	test the doohicky ?!AMP?!
    -+3 	test the doohicky ?!ERR missing '&&'?!
    ++3 	test the doohicky ?!LINT: missing '&&'?!
      4 	right now
      5 } &&
      6 
    @@ t/chainlint/chained-subshell.expect
      2 mkdir sub && (
      3 	cd sub &&
     -4 	foo the bar ?!AMP?!
    -+4 	foo the bar ?!ERR missing '&&'?!
    ++4 	foo the bar ?!LINT: missing '&&'?!
      5 	nuff said
      6 ) &&
      7 
      8 cut "-d " -f actual | (read s1 s2 s3 &&
     -9 test -f $s1 ?!AMP?!
    -+9 test -f $s1 ?!ERR missing '&&'?!
    ++9 test -f $s1 ?!LINT: missing '&&'?!
      10 test $(cat $s2) = tree2path1 &&
      11 test $(cat $s3) = tree3path1)
     
    @@ t/chainlint/command-substitution.expect
      6 ) &&
      7 (
     -8 	bar=$(gobble blocks) ?!AMP?!
    -+8 	bar=$(gobble blocks) ?!ERR missing '&&'?!
    ++8 	bar=$(gobble blocks) ?!LINT: missing '&&'?!
      9 	baz
      10 )
     
    @@ t/chainlint/complex-if-in-cuddled-loop.expect
      6    else
      7      echo >file
     -8    fi ?!LOOP?!
    -+8    fi ?!ERR missing '|| exit 1'?!
    ++8    fi ?!LINT: missing '|| exit 1'?!
      9  done) &&
      10 test ! -f file
     
    @@ t/chainlint/cuddled.expect
      4 ) &&
      5 
     -6 (cd foo ?!AMP?!
    -+6 (cd foo ?!ERR missing '&&'?!
    ++6 (cd foo ?!LINT: missing '&&'?!
      7 	bar
      8 ) &&
      9 
    @@ t/chainlint/cuddled.expect
      15 	bar) &&
      16 
     -17 (cd foo ?!AMP?!
    -+17 (cd foo ?!ERR missing '&&'?!
    ++17 (cd foo ?!LINT: missing '&&'?!
      18 	bar)
     
      ## t/chainlint/for-loop.expect ##
    @@ t/chainlint/for-loop.expect
      4 	do
     -5 		echo $i ?!AMP?!
     -6 		cat <<-\EOF ?!LOOP?!
    -+5 		echo $i ?!ERR missing '&&'?!
    -+6 		cat <<-\EOF ?!ERR missing '|| exit 1'?!
    ++5 		echo $i ?!LINT: missing '&&'?!
    ++6 		cat <<-\EOF ?!LINT: missing '|| exit 1'?!
      7 		bar
      8 		EOF
     -9 	done ?!AMP?!
    -+9 	done ?!ERR missing '&&'?!
    ++9 	done ?!LINT: missing '&&'?!
      10 
      11 	for i in a b c; do
      12 		echo $i &&
     -13 		cat $i ?!LOOP?!
    -+13 		cat $i ?!ERR missing '|| exit 1'?!
    ++13 		cat $i ?!LINT: missing '|| exit 1'?!
      14 	done
      15 )
     
    @@ t/chainlint/function.expect
      6 remove_object() {
      7 	file=$(sha1_file "$*") &&
     -8 	test -e "$file" ?!AMP?!
    -+8 	test -e "$file" ?!ERR missing '&&'?!
    ++8 	test -e "$file" ?!LINT: missing '&&'?!
      9 	rm -f "$file"
     -10 } ?!AMP?!
    -+10 } ?!ERR missing '&&'?!
    ++10 } ?!LINT: missing '&&'?!
      11 
      12 sha1_file arg && remove_object arg
     
      ## t/chainlint/here-doc-body-indent.expect ##
     @@
     -2 	echo "we should find this" ?!AMP?!
    -+2 	echo "we should find this" ?!ERR missing '&&'?!
    ++2 	echo "we should find this" ?!LINT: missing '&&'?!
      3 	echo "even though our heredoc has its indent stripped"
     
      ## t/chainlint/here-doc-body-pathological.expect ##
     @@
     -2 	echo "outer here-doc does not allow indented end-tag" ?!AMP?!
    -+2 	echo "outer here-doc does not allow indented end-tag" ?!ERR missing '&&'?!
    ++2 	echo "outer here-doc does not allow indented end-tag" ?!LINT: missing '&&'?!
      3 	cat >file <<-\EOF &&
      4 	but this inner here-doc
      5 	does allow indented EOF
      6 	EOF
     -7 	echo "missing chain after" ?!AMP?!
    -+7 	echo "missing chain after" ?!ERR missing '&&'?!
    ++7 	echo "missing chain after" ?!LINT: missing '&&'?!
      8 	echo "but this line is OK because it's the end"
     
      ## t/chainlint/here-doc-body.expect ##
     @@
     -2 	echo "missing chain before" ?!AMP?!
    -+2 	echo "missing chain before" ?!ERR missing '&&'?!
    ++2 	echo "missing chain before" ?!LINT: missing '&&'?!
      3 	cat >file <<-\EOF &&
      4 	inside inner here-doc
      5 	these are not shell commands
      6 	EOF
     -7 	echo "missing chain after" ?!AMP?!
    -+7 	echo "missing chain after" ?!ERR missing '&&'?!
    ++7 	echo "missing chain after" ?!LINT: missing '&&'?!
      8 	echo "but this line is OK because it's the end"
     
      ## t/chainlint/here-doc-double.expect ##
     @@
     -8 	echo "actual test commands" ?!AMP?!
    -+8 	echo "actual test commands" ?!ERR missing '&&'?!
    ++8 	echo "actual test commands" ?!LINT: missing '&&'?!
      9 	echo "that should be checked"
     
      ## t/chainlint/here-doc-indent-operator.expect ##
    @@ t/chainlint/here-doc-indent-operator.expect
      6 EOF
      7 
     -8 cat >expect << -EOF ?!AMP?!
    -+8 cat >expect << -EOF ?!ERR missing '&&'?!
    ++8 cat >expect << -EOF ?!LINT: missing '&&'?!
      9 this is not indented
      10 -EOF
      11 
    @@ t/chainlint/here-doc-multi-line-command-subst.expect
      5 		vegetable
      6 		END
     -7 		wiffle) ?!AMP?!
    -+7 		wiffle) ?!ERR missing '&&'?!
    ++7 		wiffle) ?!LINT: missing '&&'?!
      8 	echo $x
      9 )
     
    @@ t/chainlint/here-doc-multi-line-string.expect
      2 (
      3 	cat <<-\TXT && echo "multi-line
     -4 	string" ?!AMP?!
    -+4 	string" ?!ERR missing '&&'?!
    ++4 	string" ?!LINT: missing '&&'?!
      5 	fizzle
      6 	TXT
      7 	bap
    @@ t/chainlint/if-condition-split.expect
      4    kevin
      5 then
     -6 	echo "nomads" ?!AMP?!
    -+6 	echo "nomads" ?!ERR missing '&&'?!
    ++6 	echo "nomads" ?!LINT: missing '&&'?!
      7 	echo "for sure"
      8 fi
     
    @@ t/chainlint/if-in-loop.expect
      7 			echo "err"
      8 			exit 1
     -9 		fi ?!AMP?!
    -+9 		fi ?!ERR missing '&&'?!
    ++9 		fi ?!LINT: missing '&&'?!
      10 		foo
     -11 	done ?!AMP?!
    -+11 	done ?!ERR missing '&&'?!
    ++11 	done ?!LINT: missing '&&'?!
      12 	bar
      13 )
     
    @@ t/chainlint/if-then-else.expect
      3 	if test -n ""
      4 	then
     -5 		echo very ?!AMP?!
    -+5 		echo very ?!ERR missing '&&'?!
    ++5 		echo very ?!LINT: missing '&&'?!
      6 		echo empty
      7 	elif test -z ""
      8 	then
    @@ t/chainlint/if-then-else.expect
      13 		bar
      14 		EOF
     -15 	fi ?!AMP?!
    -+15 	fi ?!ERR missing '&&'?!
    ++15 	fi ?!LINT: missing '&&'?!
      16 	echo poodle
      17 ) &&
      18 (
    @@ t/chainlint/inline-comment.expect
      2 (
      3 	foobar && # comment 1
     -4 	barfoo ?!AMP?! # wrong position for &&
    -+4 	barfoo ?!ERR missing '&&'?! # wrong position for &&
    ++4 	barfoo ?!LINT: missing '&&'?! # wrong position for &&
      5 	flibble "not a # comment"
      6 ) &&
      7 
    @@ t/chainlint/loop-detect-failure.expect
      13 	printf "%"$n"s" X > r2/large.$n &&
      14 	git -C r2 add large.$n &&
     -15 	git -C r2 commit -m "$n" ?!LOOP?!
    -+15 	git -C r2 commit -m "$n" ?!ERR missing '|| return 1'?!
    ++15 	git -C r2 commit -m "$n" ?!LINT: missing '|| return 1'?!
      16 done
     
      ## t/chainlint/loop-in-if.expect ##
    @@ t/chainlint/loop-in-if.expect
     -7 			echo "pop" ?!AMP?!
     -8 			echo "glup" ?!LOOP?!
     -9 		done ?!AMP?!
    -+7 			echo "pop" ?!ERR missing '&&'?!
    -+8 			echo "glup" ?!ERR missing '|| exit 1'?!
    -+9 		done ?!ERR missing '&&'?!
    ++7 			echo "pop" ?!LINT: missing '&&'?!
    ++8 			echo "glup" ?!LINT: missing '|| exit 1'?!
    ++9 		done ?!LINT: missing '&&'?!
      10 		foo
     -11 	fi ?!AMP?!
    -+11 	fi ?!ERR missing '&&'?!
    ++11 	fi ?!LINT: missing '&&'?!
      12 	bar
      13 )
     
    @@ t/chainlint/multi-line-string.expect
      5 		line 3" &&
      6 	y="line 1
     -7 		line2" ?!AMP?!
    -+7 		line2" ?!ERR missing '&&'?!
    ++7 		line2" ?!LINT: missing '&&'?!
      8 	foobar
      9 ) &&
      10 (
    @@ t/chainlint/negated-one-liner.expect
      4 
     -5 ! (foo; ?!AMP?! bar) &&
     -6 ! (foo; ?!AMP?! bar) >baz
    -+5 ! (foo; ?!ERR missing '&&'?! bar) &&
    -+6 ! (foo; ?!ERR missing '&&'?! bar) >baz
    ++5 ! (foo; ?!LINT: missing '&&'?! bar) &&
    ++6 ! (foo; ?!LINT: missing '&&'?! bar) >baz
     
      ## t/chainlint/nested-cuddled-subshell.expect ##
     @@
    @@ t/chainlint/nested-cuddled-subshell.expect
      7 	(cd foo &&
      8 		bar
     -9 	) ?!AMP?!
    -+9 	) ?!ERR missing '&&'?!
    ++9 	) ?!LINT: missing '&&'?!
      10 
      11 	(
      12 		cd foo &&
    @@ t/chainlint/nested-cuddled-subshell.expect
      15 	(
      16 		cd foo &&
     -17 		bar) ?!AMP?!
    -+17 		bar) ?!ERR missing '&&'?!
    ++17 		bar) ?!LINT: missing '&&'?!
      18 
      19 	(cd foo &&
      20 		bar) &&
      21 
      22 	(cd foo &&
     -23 		bar) ?!AMP?!
    -+23 		bar) ?!ERR missing '&&'?!
    ++23 		bar) ?!LINT: missing '&&'?!
      24 
      25 	foobar
      26 )
    @@ t/chainlint/nested-here-doc.expect
      20 	INPUT_END
      21 
     -22 	cat <<-\EOT ?!AMP?!
    -+22 	cat <<-\EOT ?!ERR missing '&&'?!
    ++22 	cat <<-\EOT ?!LINT: missing '&&'?!
      23 	text goes here
      24 	data <<EOF
      25 		data goes here
    @@ t/chainlint/nested-loop-detect-failure.expect
      5 	do
     -6 		echo "$i$j" >"path$i$j" ?!LOOP?!
     -7 	done ?!LOOP?!
    -+6 		echo "$i$j" >"path$i$j" ?!ERR missing '|| return 1'?!
    -+7 	done ?!ERR missing '|| return 1'?!
    ++6 		echo "$i$j" >"path$i$j" ?!LINT: missing '|| return 1'?!
    ++7 	done ?!LINT: missing '|| return 1'?!
      8 done &&
      9 
      10 for i in 0 1 2 3 4 5 6 7 8 9;
    @@ t/chainlint/nested-loop-detect-failure.expect
      20 	for j in 0 1 2 3 4 5 6 7 8 9;
      21 	do
     -22 		echo "$i$j" >"path$i$j" ?!LOOP?!
    -+22 		echo "$i$j" >"path$i$j" ?!ERR missing '|| return 1'?!
    ++22 		echo "$i$j" >"path$i$j" ?!LINT: missing '|| return 1'?!
      23 	done || return 1
      24 done &&
      25 
    @@ t/chainlint/nested-subshell-comment.expect
      8 		baz &&
      9 		snaff
     -10 	) ?!AMP?!
    -+10 	) ?!ERR missing '&&'?!
    ++10 	) ?!LINT: missing '&&'?!
      11 	fuzzy
      12 )
     
    @@ t/chainlint/nested-subshell.expect
      9 	cd foo &&
      10 	(
     -11 		echo a ?!AMP?!
    -+11 		echo a ?!ERR missing '&&'?!
    ++11 		echo a ?!LINT: missing '&&'?!
      12 		echo b
      13 	) >file
      14 )
    @@ t/chainlint/not-heredoc.expect
      11 	echo "=======" &&
      12 	echo theirside &&
     -13 	echo ">>>>>>> theirs" ?!AMP?!
    -+13 	echo ">>>>>>> theirs" ?!ERR missing '&&'?!
    ++13 	echo ">>>>>>> theirs" ?!LINT: missing '&&'?!
      14 	poodle
      15 ) >merged
     
    @@ t/chainlint/one-liner-for-loop.expect
      5 	test_write_lines 1 2 3 4 5 >foo &&
      6 	mkdir olddir &&
     -7 	for i in a b c; do echo $i >olddir/$i; ?!LOOP?! done ?!AMP?!
    -+7 	for i in a b c; do echo $i >olddir/$i; ?!ERR missing '|| exit 1'?! done ?!ERR missing '&&'?!
    ++7 	for i in a b c; do echo $i >olddir/$i; ?!LINT: missing '|| exit 1'?! done ?!LINT: missing '&&'?!
      8 	git add foo olddir &&
      9 	git commit -m "original" &&
      10 )
    @@ t/chainlint/one-liner.expect
     -6 (foo; ?!AMP?! bar) &&
     -7 (foo; ?!AMP?! bar) |
     -8 (foo; ?!AMP?! bar) >baz &&
    -+6 (foo; ?!ERR missing '&&'?! bar) &&
    -+7 (foo; ?!ERR missing '&&'?! bar) |
    -+8 (foo; ?!ERR missing '&&'?! bar) >baz &&
    ++6 (foo; ?!LINT: missing '&&'?! bar) &&
    ++7 (foo; ?!LINT: missing '&&'?! bar) |
    ++8 (foo; ?!LINT: missing '&&'?! bar) >baz &&
      9 
      10 (foo "bar; baz")
     
    @@ t/chainlint/pipe.expect
      6 
      7 	fish |
     -8 	cow ?!AMP?!
    -+8 	cow ?!ERR missing '&&'?!
    ++8 	cow ?!LINT: missing '&&'?!
      9 
      10 	sunder
      11 )
    @@ t/chainlint/semicolon.expect
      2 (
     -3 	cat foo ; ?!AMP?! echo bar ?!AMP?!
     -4 	cat foo ; ?!AMP?! echo bar
    -+3 	cat foo ; ?!ERR missing '&&'?! echo bar ?!ERR missing '&&'?!
    -+4 	cat foo ; ?!ERR missing '&&'?! echo bar
    ++3 	cat foo ; ?!LINT: missing '&&'?! echo bar ?!LINT: missing '&&'?!
    ++4 	cat foo ; ?!LINT: missing '&&'?! echo bar
      5 ) &&
      6 (
     -7 	cat foo ; ?!AMP?! echo bar &&
     -8 	cat foo ; ?!AMP?! echo bar
    -+7 	cat foo ; ?!ERR missing '&&'?! echo bar &&
    -+8 	cat foo ; ?!ERR missing '&&'?! echo bar
    ++7 	cat foo ; ?!LINT: missing '&&'?! echo bar &&
    ++8 	cat foo ; ?!LINT: missing '&&'?! echo bar
      9 ) &&
      10 (
      11 	echo "foo; bar" &&
     -12 	cat foo; ?!AMP?! echo bar
    -+12 	cat foo; ?!ERR missing '&&'?! echo bar
    ++12 	cat foo; ?!LINT: missing '&&'?! echo bar
      13 ) &&
      14 (
      15 	foo;
    @@ t/chainlint/semicolon.expect
      17 (cd foo &&
      18 	for i in a b c; do
     -19 		echo; ?!LOOP?!
    -+19 		echo; ?!ERR missing '|| exit 1'?!
    ++19 		echo; ?!LINT: missing '|| exit 1'?!
      20 	done)
     
      ## t/chainlint/subshell-here-doc.expect ##
    @@ t/chainlint/subshell-here-doc.expect
      8 	EOF
      9 
     -10 	cat <<EOF >bip ?!AMP?!
    -+10 	cat <<EOF >bip ?!ERR missing '&&'?!
    ++10 	cat <<EOF >bip ?!LINT: missing '&&'?!
      11 	fish fly high
      12 EOF
      13 
    @@ t/chainlint/subshell-one-liner.expect
     -7 	(foo; ?!AMP?! bar) &&
     -8 	(foo; ?!AMP?! bar) |
     -9 	(foo; ?!AMP?! bar) >baz &&
    -+7 	(foo; ?!ERR missing '&&'?! bar) &&
    -+8 	(foo; ?!ERR missing '&&'?! bar) |
    -+9 	(foo; ?!ERR missing '&&'?! bar) >baz &&
    ++7 	(foo; ?!LINT: missing '&&'?! bar) &&
    ++8 	(foo; ?!LINT: missing '&&'?! bar) |
    ++9 	(foo; ?!LINT: missing '&&'?! bar) >baz &&
      10 
      11 	(foo || exit 1) &&
      12 	(foo || exit 1) |
      13 	(foo || exit 1) >baz &&
      14 
     -15 	(foo && bar) ?!AMP?!
    -+15 	(foo && bar) ?!ERR missing '&&'?!
    ++15 	(foo && bar) ?!LINT: missing '&&'?!
      16 
     -17 	(foo && bar; ?!AMP?! baz) ?!AMP?!
    -+17 	(foo && bar; ?!ERR missing '&&'?! baz) ?!ERR missing '&&'?!
    ++17 	(foo && bar; ?!LINT: missing '&&'?! baz) ?!LINT: missing '&&'?!
      18 
      19 	foobar
      20 )
    @@ t/chainlint/token-pasting.expect
      4 
      5 {
     -6     echo "*.t filter=rot13" ?!AMP?!
    -+6     echo "*.t filter=rot13" ?!ERR missing '&&'?!
    ++6     echo "*.t filter=rot13" ?!LINT: missing '&&'?!
      7     echo "*.i ident"
      8 } >.gitattributes &&
      9 
      10 {
     -11     echo a b c d e f g h i j k l m ?!AMP?!
     -12     echo n o p q r s t u v w x y z ?!AMP?!
    -+11     echo a b c d e f g h i j k l m ?!ERR missing '&&'?!
    -+12     echo n o p q r s t u v w x y z ?!ERR missing '&&'?!
    ++11     echo a b c d e f g h i j k l m ?!LINT: missing '&&'?!
    ++12     echo n o p q r s t u v w x y z ?!LINT: missing '&&'?!
      13     echo '$Id$'
      14 } >test &&
      15 cat test >test.t &&
    @@ t/chainlint/token-pasting.expect
      21 
      22 echo "content-test2" >test2.o &&
     -23 echo "content-test3 - filename with special characters" >"test3 'sq',$x=.o" ?!AMP?!
    -+23 echo "content-test3 - filename with special characters" >"test3 'sq',$x=.o" ?!ERR missing '&&'?!
    ++23 echo "content-test3 - filename with special characters" >"test3 'sq',$x=.o" ?!LINT: missing '&&'?!
      24 
      25 downstream_url_for_sed=$(
      26 	printf "%sn" "$downstream_url" |
    @@ t/chainlint/unclosed-here-doc-indent.expect
     @@
      2 command_which_is_run &&
     -3 cat >expect <<-\EOF ?!UNCLOSED-HEREDOC?! &&
    -+3 cat >expect <<-\EOF ?!ERR unclosed heredoc?! &&
    ++3 cat >expect <<-\EOF ?!LINT: unclosed heredoc?! &&
      4 we forget to end the here-doc
      5 command_which_is_gobbled
     
    @@ t/chainlint/unclosed-here-doc.expect
     @@
      2 command_which_is_run &&
     -3 cat >expect <<\EOF ?!UNCLOSED-HEREDOC?! &&
    -+3 cat >expect <<\EOF ?!ERR unclosed heredoc?! &&
    ++3 cat >expect <<\EOF ?!LINT: unclosed heredoc?! &&
      4 	we try to end the here-doc below,
      5 	but the indentation throws us off
      6 	since the operator is not "<<-".
    @@ t/chainlint/while-loop.expect
      4 	do
     -5 		echo foo ?!AMP?!
     -6 		cat <<-\EOF ?!LOOP?!
    -+5 		echo foo ?!ERR missing '&&'?!
    -+6 		cat <<-\EOF ?!ERR missing '|| exit 1'?!
    ++5 		echo foo ?!LINT: missing '&&'?!
    ++6 		cat <<-\EOF ?!LINT: missing '|| exit 1'?!
      7 		bar
      8 		EOF
     -9 	done ?!AMP?!
    -+9 	done ?!ERR missing '&&'?!
    ++9 	done ?!LINT: missing '&&'?!
      10 
      11 	while true; do
      12 		echo foo &&
     -13 		cat bar ?!LOOP?!
    -+13 		cat bar ?!ERR missing '|| exit 1'?!
    ++13 		cat bar ?!LINT: missing '|| exit 1'?!
      14 	done
      15 )
-:  ---------- > 3:  eee1e7fac7 chainlint: reduce annotation noise-factor
-- 
2.46.0

[PATCH v2 1/3] chainlint: don't be fooled by "?!...?!" in test body

[Eric Sunshine]

From: Eric Sunshine <sunshine@sunshineco.com>

As originally implemented, chainlint did not collect structured
information about detected problems. Instead, it merely emitted raw
parse tokens (not the original test text), along with a "?!...?!"
annotation directly into the output stream each time a problem was
discovered. In order to report statistics (in --stats mode) and to
adjust its exit code to indicate success or failure, it merely counts
the number of times "?!...?!" appears in the output stream. An obvious
shortcoming of this approach is that it can be fooled by a legitimate
"?!...?!" sequence in the body of a test (though, only if an actual
problem is detected in the test).

The situation did not improve when 7c04aa7390 (chainlint: colorize
problem annotations and test delimiters, 2022-09-13) colored the
annotations after-the-fact by searching for "?!...?!" in the output
stream and inserting color codes. As above, a shortcoming is that this
approach can incorrectly color a legitimate "?!...?!" sequence in a test
body as if it is an error.

However, when 73c768dae9 (chainlint: annotate original test definition
rather than token stream, 2022-11-08) taught chainlint to output the
original test text verbatim, it started collecting structured
information about detected problems.

Now that it is available, take advantage of the structured problem
information to deterministically count the number of problems detected
and to color the annotations directly, rather than scanning the output
stream for "?!...?!" and performing these operations after-the-fact.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
---
 t/chainlint.pl | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/t/chainlint.pl b/t/chainlint.pl
index 5361f23b1d..1a7611ad43 100755
--- a/t/chainlint.pl
+++ b/t/chainlint.pl
@@ -587,6 +587,7 @@ sub new {
 	my $class = shift @_;
 	my $self = $class->SUPER::new(@_);
 	$self->{ntests} = 0;
+	$self->{nerrs} = 0;
 	return $self;
 }
 
@@ -634,6 +635,7 @@ sub check_test {
 	my $parser = TestParser->new(\$body);
 	my @tokens = $parser->parse();
 	my $problems = $parser->{problems};
+	$self->{nerrs} += @$problems;
 	return unless $emit_all || @$problems;
 	my $c = main::fd_colors(1);
 	my $start = 0;
@@ -641,15 +643,16 @@ sub check_test {
 	for (sort {$a->[1]->[2] <=> $b->[1]->[2]} @$problems) {
 		my ($label, $token) = @$_;
 		my $pos = $token->[2];
-		$checked .= substr($body, $start, $pos - $start) . " ?!$label?! ";
+		$checked .= substr($body, $start, $pos - $start);
+		$checked .= ' ' unless $checked =~ /\s$/;
+		$checked .= "$c->{rev}$c->{red}?!$label?!$c->{reset}";
+		$checked .= ' ' unless $pos >= length($body) ||
+		    substr($body, $pos, 1) =~ /^\s/;
 		$start = $pos;
 	}
 	$checked .= substr($body, $start);
 	$checked =~ s/^/$lineno++ . ' '/mge;
 	$checked =~ s/^\d+ \n//;
-	$checked =~ s/(\s) \?!/$1?!/mg;
-	$checked =~ s/\?! (\s)/?!$1/mg;
-	$checked =~ s/(\?![^?]+\?!)/$c->{rev}$c->{red}$1$c->{reset}/mg;
 	$checked =~ s/^\d+/$c->{dim}$&$c->{reset}/mg;
 	$checked .= "\n" unless $checked =~ /\n$/;
 	push(@{$self->{output}}, "$c->{blue}# chainlint: $title$c->{reset}\n$checked");
@@ -791,9 +794,9 @@ sub check_script {
 			my $c = fd_colors(1);
 			my $s = join('', @{$parser->{output}});
 			$emit->("$c->{bold}$c->{blue}# chainlint: $path$c->{reset}\n" . $s);
-			$nerrs += () = $s =~ /\?![^?]+\?!/g;
 		}
 		$ntests += $parser->{ntests};
+		$nerrs += $parser->{nerrs};
 	}
 	return [$id, $nscripts, $ntests, $nerrs];
 }
-- 
2.46.0

[PATCH v2 2/3] chainlint: make error messages self-explanatory

[Eric Sunshine]

From: Eric Sunshine <sunshine@sunshineco.com>

The annotations emitted by chainlint to indicate detected problems are
overly terse, so much so that developers new to the project -- those who
should most benefit from the linting -- may find them baffling. For
instance, although the author of chainlint and seasoned Git developers
may understand that "?!AMP?!" is an abbreviation of "ampersand" and
indicates a break in the &&-chain, this may not be obvious to newcomers.

The "?!LOOP?!" case is particularly serious because that terse single
word does nothing to convey that the loop body should end with
"|| return 1" (or "|| exit 1" in a subshell) to ensure that a failing
command in the body aborts the loop immediately. Moreover, unlike
&&-chaining which is ubiquitous in Git tests, the "|| return 1" idiom is
relatively infrequent, thus may be harder for a newcomer to discover by
consulting nearby code.

Address these shortcomings by emitting human-readable messages which
both explain the problem and give a strong hint about how to correct it.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
---
 t/chainlint.pl                                | 30 ++++++++++++++-----
 t/chainlint/arithmetic-expansion.expect       |  2 +-
 t/chainlint/block.expect                      |  8 ++---
 t/chainlint/broken-chain.expect               |  2 +-
 t/chainlint/case.expect                       |  4 +--
 t/chainlint/chain-break-false.expect          |  2 +-
 t/chainlint/chained-block.expect              |  2 +-
 t/chainlint/chained-subshell.expect           |  4 +--
 t/chainlint/command-substitution.expect       |  2 +-
 t/chainlint/complex-if-in-cuddled-loop.expect |  2 +-
 t/chainlint/cuddled.expect                    |  4 +--
 t/chainlint/for-loop.expect                   |  8 ++---
 t/chainlint/function.expect                   |  4 +--
 t/chainlint/here-doc-body-indent.expect       |  2 +-
 t/chainlint/here-doc-body-pathological.expect |  4 +--
 t/chainlint/here-doc-body.expect              |  4 +--
 t/chainlint/here-doc-double.expect            |  2 +-
 t/chainlint/here-doc-indent-operator.expect   |  2 +-
 .../here-doc-multi-line-command-subst.expect  |  2 +-
 t/chainlint/here-doc-multi-line-string.expect |  2 +-
 t/chainlint/if-condition-split.expect         |  2 +-
 t/chainlint/if-in-loop.expect                 |  4 +--
 t/chainlint/if-then-else.expect               |  4 +--
 t/chainlint/inline-comment.expect             |  2 +-
 t/chainlint/loop-detect-failure.expect        |  2 +-
 t/chainlint/loop-in-if.expect                 |  8 ++---
 t/chainlint/multi-line-string.expect          |  2 +-
 t/chainlint/negated-one-liner.expect          |  4 +--
 t/chainlint/nested-cuddled-subshell.expect    |  6 ++--
 t/chainlint/nested-here-doc.expect            |  2 +-
 t/chainlint/nested-loop-detect-failure.expect |  6 ++--
 t/chainlint/nested-subshell-comment.expect    |  2 +-
 t/chainlint/nested-subshell.expect            |  2 +-
 t/chainlint/not-heredoc.expect                |  2 +-
 t/chainlint/one-liner-for-loop.expect         |  2 +-
 t/chainlint/one-liner.expect                  |  6 ++--
 t/chainlint/pipe.expect                       |  2 +-
 t/chainlint/semicolon.expect                  | 12 ++++----
 t/chainlint/subshell-here-doc.expect          |  2 +-
 t/chainlint/subshell-one-liner.expect         | 10 +++----
 t/chainlint/token-pasting.expect              |  8 ++---
 t/chainlint/unclosed-here-doc-indent.expect   |  2 +-
 t/chainlint/unclosed-here-doc.expect          |  2 +-
 t/chainlint/while-loop.expect                 |  8 ++---
 44 files changed, 104 insertions(+), 90 deletions(-)

diff --git a/t/chainlint.pl b/t/chainlint.pl
index 1a7611ad43..ad26499478 100755
--- a/t/chainlint.pl
+++ b/t/chainlint.pl
@@ -9,9 +9,9 @@
 # Input arguments are pathnames of shell scripts containing test definitions,
 # or globs referencing a collection of scripts. For each problem discovered,
 # the pathname of the script containing the test is printed along with the test
-# name and the test body with a `?!FOO?!` annotation at the location of each
-# detected problem, where "FOO" is a tag such as "AMP" which indicates a broken
-# &&-chain. Returns zero if no problems are discovered, otherwise non-zero.
+# name and the test body with a `?!LINT: ...?!` annotation at the location of
+# each detected problem, where "..." is an explanation of the problem. Returns
+# zero if no problems are discovered, otherwise non-zero.
 
 use warnings;
 use strict;
@@ -181,7 +181,7 @@ sub swallow_heredocs {
 			$self->{lineno} += () = $body =~ /\n/sg;
 			next;
 		}
-		push(@{$self->{parser}->{problems}}, ['UNCLOSED-HEREDOC', $tag]);
+		push(@{$self->{parser}->{problems}}, ['HEREDOC', $tag]);
 		$$b =~ /(?:\G|\n).*\z/gc; # consume rest of input
 		my $body = substr($$b, $start, pos($$b) - $start);
 		$self->{lineno} += () = $body =~ /\n/sg;
@@ -238,6 +238,7 @@ sub new {
 		stop => [],
 		output => [],
 		heredocs => {},
+		insubshell => 0,
 	} => $class;
 	$self->{lexer} = Lexer->new($self, $s);
 	return $self;
@@ -296,8 +297,11 @@ sub parse_group {
 
 sub parse_subshell {
 	my $self = shift @_;
-	return ($self->parse(qr/^\)$/),
-		$self->expect(')'));
+	$self->{insubshell}++;
+	my @tokens = ($self->parse(qr/^\)$/),
+		      $self->expect(')'));
+	$self->{insubshell}--;
+	return @tokens;
 }
 
 sub parse_case_pattern {
@@ -528,7 +532,7 @@ sub parse_loop_body {
 	return @tokens if ends_with(\@tokens, [qr/^\|\|$/, "\n", qr/^echo$/, qr/^.+$/]);
 	# flag missing "return/exit" handling explicit failure in loop body
 	my $n = find_non_nl(\@tokens);
-	push(@{$self->{problems}}, ['LOOP', $tokens[$n]]);
+	push(@{$self->{problems}}, [$self->{insubshell} ? 'LOOPEXIT' : 'LOOPRETURN', $tokens[$n]]);
 	return @tokens;
 }
 
@@ -620,6 +624,15 @@ sub unwrap {
 	return $s
 }
 
+sub format_problem {
+	local $_ = shift;
+	/^AMP$/ && return "missing '&&'";
+	/^LOOPRETURN$/ && return "missing '|| return 1'";
+	/^LOOPEXIT$/ && return "missing '|| exit 1'";
+	/^HEREDOC$/ && return 'unclosed heredoc';
+	die("unrecognized problem type '$_'\n");
+}
+
 sub check_test {
 	my $self = shift @_;
 	my $title = unwrap(shift @_);
@@ -643,9 +656,10 @@ sub check_test {
 	for (sort {$a->[1]->[2] <=> $b->[1]->[2]} @$problems) {
 		my ($label, $token) = @$_;
 		my $pos = $token->[2];
+		my $err = format_problem($label);
 		$checked .= substr($body, $start, $pos - $start);
 		$checked .= ' ' unless $checked =~ /\s$/;
-		$checked .= "$c->{rev}$c->{red}?!$label?!$c->{reset}";
+		$checked .= "$c->{rev}$c->{red}?!LINT: $err?!$c->{reset}";
 		$checked .= ' ' unless $pos >= length($body) ||
 		    substr($body, $pos, 1) =~ /^\s/;
 		$start = $pos;
diff --git a/t/chainlint/arithmetic-expansion.expect b/t/chainlint/arithmetic-expansion.expect
index 338ecd5861..5677e16cad 100644
--- a/t/chainlint/arithmetic-expansion.expect
+++ b/t/chainlint/arithmetic-expansion.expect
@@ -4,6 +4,6 @@
 5 	baz
 6 ) &&
 7 (
-8 	bar=$((42 + 1)) ?!AMP?!
+8 	bar=$((42 + 1)) ?!LINT: missing '&&'?!
 9 	baz
 10 )
diff --git a/t/chainlint/block.expect b/t/chainlint/block.expect
index b62e3d58c3..3d3f854c0d 100644
--- a/t/chainlint/block.expect
+++ b/t/chainlint/block.expect
@@ -1,20 +1,20 @@
 2 (
 3 	foo &&
 4 	{
-5 		echo a ?!AMP?!
+5 		echo a ?!LINT: missing '&&'?!
 6 		echo b
 7 	} &&
 8 	bar &&
 9 	{
 10 		echo c
-11 	} ?!AMP?!
+11 	} ?!LINT: missing '&&'?!
 12 	baz
 13 ) &&
 14 
 15 {
-16 	echo a; ?!AMP?! echo b
+16 	echo a; ?!LINT: missing '&&'?! echo b
 17 } &&
-18 { echo a; ?!AMP?! echo b; } &&
+18 { echo a; ?!LINT: missing '&&'?! echo b; } &&
 19 
 20 {
 21 	echo "${var}9" &&
diff --git a/t/chainlint/broken-chain.expect b/t/chainlint/broken-chain.expect
index 9a1838736f..b7b1ce8509 100644
--- a/t/chainlint/broken-chain.expect
+++ b/t/chainlint/broken-chain.expect
@@ -1,6 +1,6 @@
 2 (
 3 	foo &&
-4 	bar ?!AMP?!
+4 	bar ?!LINT: missing '&&'?!
 5 	baz &&
 6 	wop
 7 )
diff --git a/t/chainlint/case.expect b/t/chainlint/case.expect
index c04c61ff36..0a3b09e470 100644
--- a/t/chainlint/case.expect
+++ b/t/chainlint/case.expect
@@ -9,11 +9,11 @@
 10 	case "$x" in
 11 	x) foo ;;
 12 	*) bar ;;
-13 	esac ?!AMP?!
+13 	esac ?!LINT: missing '&&'?!
 14 	foobar
 15 ) &&
 16 (
 17 	case "$x" in 1) true;; esac &&
-18 	case "$y" in 2) false;; esac ?!AMP?!
+18 	case "$y" in 2) false;; esac ?!LINT: missing '&&'?!
 19 	foobar
 20 )
diff --git a/t/chainlint/chain-break-false.expect b/t/chainlint/chain-break-false.expect
index 4f815f8e14..f6a0a301e9 100644
--- a/t/chainlint/chain-break-false.expect
+++ b/t/chainlint/chain-break-false.expect
@@ -4,6 +4,6 @@
 5 	echo failed!
 6 	false
 7 else
-8 	echo it went okay ?!AMP?!
+8 	echo it went okay ?!LINT: missing '&&'?!
 9 	congratulate user
 10 fi
diff --git a/t/chainlint/chained-block.expect b/t/chainlint/chained-block.expect
index a546b714a6..f2501bba90 100644
--- a/t/chainlint/chained-block.expect
+++ b/t/chainlint/chained-block.expect
@@ -1,5 +1,5 @@
 2 echo nobody home && {
-3 	test the doohicky ?!AMP?!
+3 	test the doohicky ?!LINT: missing '&&'?!
 4 	right now
 5 } &&
 6 
diff --git a/t/chainlint/chained-subshell.expect b/t/chainlint/chained-subshell.expect
index f78b268291..93fb1a6578 100644
--- a/t/chainlint/chained-subshell.expect
+++ b/t/chainlint/chained-subshell.expect
@@ -1,10 +1,10 @@
 2 mkdir sub && (
 3 	cd sub &&
-4 	foo the bar ?!AMP?!
+4 	foo the bar ?!LINT: missing '&&'?!
 5 	nuff said
 6 ) &&
 7 
 8 cut "-d " -f actual | (read s1 s2 s3 &&
-9 test -f $s1 ?!AMP?!
+9 test -f $s1 ?!LINT: missing '&&'?!
 10 test $(cat $s2) = tree2path1 &&
 11 test $(cat $s3) = tree3path1)
diff --git a/t/chainlint/command-substitution.expect b/t/chainlint/command-substitution.expect
index 5e31b36db6..73809fd585 100644
--- a/t/chainlint/command-substitution.expect
+++ b/t/chainlint/command-substitution.expect
@@ -4,6 +4,6 @@
 5 	baz
 6 ) &&
 7 (
-8 	bar=$(gobble blocks) ?!AMP?!
+8 	bar=$(gobble blocks) ?!LINT: missing '&&'?!
 9 	baz
 10 )
diff --git a/t/chainlint/complex-if-in-cuddled-loop.expect b/t/chainlint/complex-if-in-cuddled-loop.expect
index 3a740103db..e66bb2d5d0 100644
--- a/t/chainlint/complex-if-in-cuddled-loop.expect
+++ b/t/chainlint/complex-if-in-cuddled-loop.expect
@@ -4,6 +4,6 @@
 5      :
 6    else
 7      echo >file
-8    fi ?!LOOP?!
+8    fi ?!LINT: missing '|| exit 1'?!
 9  done) &&
 10 test ! -f file
diff --git a/t/chainlint/cuddled.expect b/t/chainlint/cuddled.expect
index b06d638311..1864b3fc8b 100644
--- a/t/chainlint/cuddled.expect
+++ b/t/chainlint/cuddled.expect
@@ -2,7 +2,7 @@
 3 	bar
 4 ) &&
 5 
-6 (cd foo ?!AMP?!
+6 (cd foo ?!LINT: missing '&&'?!
 7 	bar
 8 ) &&
 9 
@@ -13,5 +13,5 @@
 14 (cd foo &&
 15 	bar) &&
 16 
-17 (cd foo ?!AMP?!
+17 (cd foo ?!LINT: missing '&&'?!
 18 	bar)
diff --git a/t/chainlint/for-loop.expect b/t/chainlint/for-loop.expect
index 908aeedf96..5029eacce3 100644
--- a/t/chainlint/for-loop.expect
+++ b/t/chainlint/for-loop.expect
@@ -1,14 +1,14 @@
 2 (
 3 	for i in a b c
 4 	do
-5 		echo $i ?!AMP?!
-6 		cat <<-\EOF ?!LOOP?!
+5 		echo $i ?!LINT: missing '&&'?!
+6 		cat <<-\EOF ?!LINT: missing '|| exit 1'?!
 7 		bar
 8 		EOF
-9 	done ?!AMP?!
+9 	done ?!LINT: missing '&&'?!
 10 
 11 	for i in a b c; do
 12 		echo $i &&
-13 		cat $i ?!LOOP?!
+13 		cat $i ?!LINT: missing '|| exit 1'?!
 14 	done
 15 )
diff --git a/t/chainlint/function.expect b/t/chainlint/function.expect
index c226246b25..9e46a3554a 100644
--- a/t/chainlint/function.expect
+++ b/t/chainlint/function.expect
@@ -4,8 +4,8 @@
 5 
 6 remove_object() {
 7 	file=$(sha1_file "$*") &&
-8 	test -e "$file" ?!AMP?!
+8 	test -e "$file" ?!LINT: missing '&&'?!
 9 	rm -f "$file"
-10 } ?!AMP?!
+10 } ?!LINT: missing '&&'?!
 11 
 12 sha1_file arg && remove_object arg
diff --git a/t/chainlint/here-doc-body-indent.expect b/t/chainlint/here-doc-body-indent.expect
index 4323acc93d..4306faee86 100644
--- a/t/chainlint/here-doc-body-indent.expect
+++ b/t/chainlint/here-doc-body-indent.expect
@@ -1,2 +1,2 @@
-2 	echo "we should find this" ?!AMP?!
+2 	echo "we should find this" ?!LINT: missing '&&'?!
 3 	echo "even though our heredoc has its indent stripped"
diff --git a/t/chainlint/here-doc-body-pathological.expect b/t/chainlint/here-doc-body-pathological.expect
index a93a1fa3aa..2f8ea03a47 100644
--- a/t/chainlint/here-doc-body-pathological.expect
+++ b/t/chainlint/here-doc-body-pathological.expect
@@ -1,7 +1,7 @@
-2 	echo "outer here-doc does not allow indented end-tag" ?!AMP?!
+2 	echo "outer here-doc does not allow indented end-tag" ?!LINT: missing '&&'?!
 3 	cat >file <<-\EOF &&
 4 	but this inner here-doc
 5 	does allow indented EOF
 6 	EOF
-7 	echo "missing chain after" ?!AMP?!
+7 	echo "missing chain after" ?!LINT: missing '&&'?!
 8 	echo "but this line is OK because it's the end"
diff --git a/t/chainlint/here-doc-body.expect b/t/chainlint/here-doc-body.expect
index ddf1c412af..df8d79bc0a 100644
--- a/t/chainlint/here-doc-body.expect
+++ b/t/chainlint/here-doc-body.expect
@@ -1,7 +1,7 @@
-2 	echo "missing chain before" ?!AMP?!
+2 	echo "missing chain before" ?!LINT: missing '&&'?!
 3 	cat >file <<-\EOF &&
 4 	inside inner here-doc
 5 	these are not shell commands
 6 	EOF
-7 	echo "missing chain after" ?!AMP?!
+7 	echo "missing chain after" ?!LINT: missing '&&'?!
 8 	echo "but this line is OK because it's the end"
diff --git a/t/chainlint/here-doc-double.expect b/t/chainlint/here-doc-double.expect
index 20dba4b452..e5e981889f 100644
--- a/t/chainlint/here-doc-double.expect
+++ b/t/chainlint/here-doc-double.expect
@@ -1,2 +1,2 @@
-8 	echo "actual test commands" ?!AMP?!
+8 	echo "actual test commands" ?!LINT: missing '&&'?!
 9 	echo "that should be checked"
diff --git a/t/chainlint/here-doc-indent-operator.expect b/t/chainlint/here-doc-indent-operator.expect
index 277a11202d..ec0e61505b 100644
--- a/t/chainlint/here-doc-indent-operator.expect
+++ b/t/chainlint/here-doc-indent-operator.expect
@@ -4,7 +4,7 @@
 5 chunks: oid_fanout oid_lookup commit_metadata generation_data bloom_indexes bloom_data
 6 EOF
 7 
-8 cat >expect << -EOF ?!AMP?!
+8 cat >expect << -EOF ?!LINT: missing '&&'?!
 9 this is not indented
 10 -EOF
 11 
diff --git a/t/chainlint/here-doc-multi-line-command-subst.expect b/t/chainlint/here-doc-multi-line-command-subst.expect
index 41b55f6437..8128f15b92 100644
--- a/t/chainlint/here-doc-multi-line-command-subst.expect
+++ b/t/chainlint/here-doc-multi-line-command-subst.expect
@@ -3,6 +3,6 @@
 4 		fossil
 5 		vegetable
 6 		END
-7 		wiffle) ?!AMP?!
+7 		wiffle) ?!LINT: missing '&&'?!
 8 	echo $x
 9 )
diff --git a/t/chainlint/here-doc-multi-line-string.expect b/t/chainlint/here-doc-multi-line-string.expect
index c71828589e..a03a04ff3d 100644
--- a/t/chainlint/here-doc-multi-line-string.expect
+++ b/t/chainlint/here-doc-multi-line-string.expect
@@ -1,6 +1,6 @@
 2 (
 3 	cat <<-\TXT && echo "multi-line
-4 	string" ?!AMP?!
+4 	string" ?!LINT: missing '&&'?!
 5 	fizzle
 6 	TXT
 7 	bap
diff --git a/t/chainlint/if-condition-split.expect b/t/chainlint/if-condition-split.expect
index 9daf3d294a..6d2a03dfdb 100644
--- a/t/chainlint/if-condition-split.expect
+++ b/t/chainlint/if-condition-split.expect
@@ -2,6 +2,6 @@
 3    marcia ||
 4    kevin
 5 then
-6 	echo "nomads" ?!AMP?!
+6 	echo "nomads" ?!LINT: missing '&&'?!
 7 	echo "for sure"
 8 fi
diff --git a/t/chainlint/if-in-loop.expect b/t/chainlint/if-in-loop.expect
index ff8c60dbdb..7e3ba740de 100644
--- a/t/chainlint/if-in-loop.expect
+++ b/t/chainlint/if-in-loop.expect
@@ -5,8 +5,8 @@
 6 		then
 7 			echo "err"
 8 			exit 1
-9 		fi ?!AMP?!
+9 		fi ?!LINT: missing '&&'?!
 10 		foo
-11 	done ?!AMP?!
+11 	done ?!LINT: missing '&&'?!
 12 	bar
 13 )
diff --git a/t/chainlint/if-then-else.expect b/t/chainlint/if-then-else.expect
index 965d7e41a2..924caa2e4e 100644
--- a/t/chainlint/if-then-else.expect
+++ b/t/chainlint/if-then-else.expect
@@ -1,7 +1,7 @@
 2 (
 3 	if test -n ""
 4 	then
-5 		echo very ?!AMP?!
+5 		echo very ?!LINT: missing '&&'?!
 6 		echo empty
 7 	elif test -z ""
 8 	then
@@ -11,7 +11,7 @@
 12 		cat <<-\EOF
 13 		bar
 14 		EOF
-15 	fi ?!AMP?!
+15 	fi ?!LINT: missing '&&'?!
 16 	echo poodle
 17 ) &&
 18 (
diff --git a/t/chainlint/inline-comment.expect b/t/chainlint/inline-comment.expect
index 0285c0b22c..4b4080124e 100644
--- a/t/chainlint/inline-comment.expect
+++ b/t/chainlint/inline-comment.expect
@@ -1,6 +1,6 @@
 2 (
 3 	foobar && # comment 1
-4 	barfoo ?!AMP?! # wrong position for &&
+4 	barfoo ?!LINT: missing '&&'?! # wrong position for &&
 5 	flibble "not a # comment"
 6 ) &&
 7 
diff --git a/t/chainlint/loop-detect-failure.expect b/t/chainlint/loop-detect-failure.expect
index 40c06f0d53..7d846b878d 100644
--- a/t/chainlint/loop-detect-failure.expect
+++ b/t/chainlint/loop-detect-failure.expect
@@ -11,5 +11,5 @@
 12 do
 13 	printf "%"$n"s" X > r2/large.$n &&
 14 	git -C r2 add large.$n &&
-15 	git -C r2 commit -m "$n" ?!LOOP?!
+15 	git -C r2 commit -m "$n" ?!LINT: missing '|| return 1'?!
 16 done
diff --git a/t/chainlint/loop-in-if.expect b/t/chainlint/loop-in-if.expect
index 4e8c67c914..32e076ad1b 100644
--- a/t/chainlint/loop-in-if.expect
+++ b/t/chainlint/loop-in-if.expect
@@ -3,10 +3,10 @@
 4 	then
 5 		while true
 6 		do
-7 			echo "pop" ?!AMP?!
-8 			echo "glup" ?!LOOP?!
-9 		done ?!AMP?!
+7 			echo "pop" ?!LINT: missing '&&'?!
+8 			echo "glup" ?!LINT: missing '|| exit 1'?!
+9 		done ?!LINT: missing '&&'?!
 10 		foo
-11 	fi ?!AMP?!
+11 	fi ?!LINT: missing '&&'?!
 12 	bar
 13 )
diff --git a/t/chainlint/multi-line-string.expect b/t/chainlint/multi-line-string.expect
index 62c54e3a5e..9d33297525 100644
--- a/t/chainlint/multi-line-string.expect
+++ b/t/chainlint/multi-line-string.expect
@@ -3,7 +3,7 @@
 4 		line 2
 5 		line 3" &&
 6 	y="line 1
-7 		line2" ?!AMP?!
+7 		line2" ?!LINT: missing '&&'?!
 8 	foobar
 9 ) &&
 10 (
diff --git a/t/chainlint/negated-one-liner.expect b/t/chainlint/negated-one-liner.expect
index a6ce52a1da..0a6f3c29b2 100644
--- a/t/chainlint/negated-one-liner.expect
+++ b/t/chainlint/negated-one-liner.expect
@@ -1,5 +1,5 @@
 2 ! (foo && bar) &&
 3 ! (foo && bar) >baz &&
 4 
-5 ! (foo; ?!AMP?! bar) &&
-6 ! (foo; ?!AMP?! bar) >baz
+5 ! (foo; ?!LINT: missing '&&'?! bar) &&
+6 ! (foo; ?!LINT: missing '&&'?! bar) >baz
diff --git a/t/chainlint/nested-cuddled-subshell.expect b/t/chainlint/nested-cuddled-subshell.expect
index 0191c9c294..fec2c74274 100644
--- a/t/chainlint/nested-cuddled-subshell.expect
+++ b/t/chainlint/nested-cuddled-subshell.expect
@@ -5,7 +5,7 @@
 6 
 7 	(cd foo &&
 8 		bar
-9 	) ?!AMP?!
+9 	) ?!LINT: missing '&&'?!
 10 
 11 	(
 12 		cd foo &&
@@ -13,13 +13,13 @@
 14 
 15 	(
 16 		cd foo &&
-17 		bar) ?!AMP?!
+17 		bar) ?!LINT: missing '&&'?!
 18 
 19 	(cd foo &&
 20 		bar) &&
 21 
 22 	(cd foo &&
-23 		bar) ?!AMP?!
+23 		bar) ?!LINT: missing '&&'?!
 24 
 25 	foobar
 26 )
diff --git a/t/chainlint/nested-here-doc.expect b/t/chainlint/nested-here-doc.expect
index 70d9b68dc9..571f4c9514 100644
--- a/t/chainlint/nested-here-doc.expect
+++ b/t/chainlint/nested-here-doc.expect
@@ -18,7 +18,7 @@
 19 	toink
 20 	INPUT_END
 21 
-22 	cat <<-\EOT ?!AMP?!
+22 	cat <<-\EOT ?!LINT: missing '&&'?!
 23 	text goes here
 24 	data <<EOF
 25 		data goes here
diff --git a/t/chainlint/nested-loop-detect-failure.expect b/t/chainlint/nested-loop-detect-failure.expect
index c13c4d2f90..b4aaa621a2 100644
--- a/t/chainlint/nested-loop-detect-failure.expect
+++ b/t/chainlint/nested-loop-detect-failure.expect
@@ -2,8 +2,8 @@
 3 do
 4 	for j in 0 1 2 3 4 5 6 7 8 9;
 5 	do
-6 		echo "$i$j" >"path$i$j" ?!LOOP?!
-7 	done ?!LOOP?!
+6 		echo "$i$j" >"path$i$j" ?!LINT: missing '|| return 1'?!
+7 	done ?!LINT: missing '|| return 1'?!
 8 done &&
 9 
 10 for i in 0 1 2 3 4 5 6 7 8 9;
@@ -18,7 +18,7 @@
 19 do
 20 	for j in 0 1 2 3 4 5 6 7 8 9;
 21 	do
-22 		echo "$i$j" >"path$i$j" ?!LOOP?!
+22 		echo "$i$j" >"path$i$j" ?!LINT: missing '|| return 1'?!
 23 	done || return 1
 24 done &&
 25 
diff --git a/t/chainlint/nested-subshell-comment.expect b/t/chainlint/nested-subshell-comment.expect
index f89a8d03a8..078c6f275f 100644
--- a/t/chainlint/nested-subshell-comment.expect
+++ b/t/chainlint/nested-subshell-comment.expect
@@ -6,6 +6,6 @@
 7 		# minor numbers of cows (or do they?)
 8 		baz &&
 9 		snaff
-10 	) ?!AMP?!
+10 	) ?!LINT: missing '&&'?!
 11 	fuzzy
 12 )
diff --git a/t/chainlint/nested-subshell.expect b/t/chainlint/nested-subshell.expect
index 811e8a7912..a8d85d5d5b 100644
--- a/t/chainlint/nested-subshell.expect
+++ b/t/chainlint/nested-subshell.expect
@@ -7,7 +7,7 @@
 8 
 9 	cd foo &&
 10 	(
-11 		echo a ?!AMP?!
+11 		echo a ?!LINT: missing '&&'?!
 12 		echo b
 13 	) >file
 14 )
diff --git a/t/chainlint/not-heredoc.expect b/t/chainlint/not-heredoc.expect
index 611b7b75cb..5d51705a7a 100644
--- a/t/chainlint/not-heredoc.expect
+++ b/t/chainlint/not-heredoc.expect
@@ -9,6 +9,6 @@
 10 	echo ourside &&
 11 	echo "=======" &&
 12 	echo theirside &&
-13 	echo ">>>>>>> theirs" ?!AMP?!
+13 	echo ">>>>>>> theirs" ?!LINT: missing '&&'?!
 14 	poodle
 15 ) >merged
diff --git a/t/chainlint/one-liner-for-loop.expect b/t/chainlint/one-liner-for-loop.expect
index 49dcf065ef..e1fcbd3639 100644
--- a/t/chainlint/one-liner-for-loop.expect
+++ b/t/chainlint/one-liner-for-loop.expect
@@ -3,7 +3,7 @@
 4 	cd dir-rename-and-content &&
 5 	test_write_lines 1 2 3 4 5 >foo &&
 6 	mkdir olddir &&
-7 	for i in a b c; do echo $i >olddir/$i; ?!LOOP?! done ?!AMP?!
+7 	for i in a b c; do echo $i >olddir/$i; ?!LINT: missing '|| exit 1'?! done ?!LINT: missing '&&'?!
 8 	git add foo olddir &&
 9 	git commit -m "original" &&
 10 )
diff --git a/t/chainlint/one-liner.expect b/t/chainlint/one-liner.expect
index 9861811283..5deeb05070 100644
--- a/t/chainlint/one-liner.expect
+++ b/t/chainlint/one-liner.expect
@@ -2,8 +2,8 @@
 3 (foo && bar) |
 4 (foo && bar) >baz &&
 5 
-6 (foo; ?!AMP?! bar) &&
-7 (foo; ?!AMP?! bar) |
-8 (foo; ?!AMP?! bar) >baz &&
+6 (foo; ?!LINT: missing '&&'?! bar) &&
+7 (foo; ?!LINT: missing '&&'?! bar) |
+8 (foo; ?!LINT: missing '&&'?! bar) >baz &&
 9 
 10 (foo "bar; baz")
diff --git a/t/chainlint/pipe.expect b/t/chainlint/pipe.expect
index 1bbe5a2ce1..d947c76584 100644
--- a/t/chainlint/pipe.expect
+++ b/t/chainlint/pipe.expect
@@ -4,7 +4,7 @@
 5 	baz &&
 6 
 7 	fish |
-8 	cow ?!AMP?!
+8 	cow ?!LINT: missing '&&'?!
 9 
 10 	sunder
 11 )
diff --git a/t/chainlint/semicolon.expect b/t/chainlint/semicolon.expect
index 866438310c..2b499fbe70 100644
--- a/t/chainlint/semicolon.expect
+++ b/t/chainlint/semicolon.expect
@@ -1,19 +1,19 @@
 2 (
-3 	cat foo ; ?!AMP?! echo bar ?!AMP?!
-4 	cat foo ; ?!AMP?! echo bar
+3 	cat foo ; ?!LINT: missing '&&'?! echo bar ?!LINT: missing '&&'?!
+4 	cat foo ; ?!LINT: missing '&&'?! echo bar
 5 ) &&
 6 (
-7 	cat foo ; ?!AMP?! echo bar &&
-8 	cat foo ; ?!AMP?! echo bar
+7 	cat foo ; ?!LINT: missing '&&'?! echo bar &&
+8 	cat foo ; ?!LINT: missing '&&'?! echo bar
 9 ) &&
 10 (
 11 	echo "foo; bar" &&
-12 	cat foo; ?!AMP?! echo bar
+12 	cat foo; ?!LINT: missing '&&'?! echo bar
 13 ) &&
 14 (
 15 	foo;
 16 ) &&
 17 (cd foo &&
 18 	for i in a b c; do
-19 		echo; ?!LOOP?!
+19 		echo; ?!LINT: missing '|| exit 1'?!
 20 	done)
diff --git a/t/chainlint/subshell-here-doc.expect b/t/chainlint/subshell-here-doc.expect
index 5647500c82..e450caf948 100644
--- a/t/chainlint/subshell-here-doc.expect
+++ b/t/chainlint/subshell-here-doc.expect
@@ -6,7 +6,7 @@
 7 	nevermore...
 8 	EOF
 9 
-10 	cat <<EOF >bip ?!AMP?!
+10 	cat <<EOF >bip ?!LINT: missing '&&'?!
 11 	fish fly high
 12 EOF
 13 
diff --git a/t/chainlint/subshell-one-liner.expect b/t/chainlint/subshell-one-liner.expect
index 214316c6a0..265d996a21 100644
--- a/t/chainlint/subshell-one-liner.expect
+++ b/t/chainlint/subshell-one-liner.expect
@@ -3,17 +3,17 @@
 4 	(foo && bar) |
 5 	(foo && bar) >baz &&
 6 
-7 	(foo; ?!AMP?! bar) &&
-8 	(foo; ?!AMP?! bar) |
-9 	(foo; ?!AMP?! bar) >baz &&
+7 	(foo; ?!LINT: missing '&&'?! bar) &&
+8 	(foo; ?!LINT: missing '&&'?! bar) |
+9 	(foo; ?!LINT: missing '&&'?! bar) >baz &&
 10 
 11 	(foo || exit 1) &&
 12 	(foo || exit 1) |
 13 	(foo || exit 1) >baz &&
 14 
-15 	(foo && bar) ?!AMP?!
+15 	(foo && bar) ?!LINT: missing '&&'?!
 16 
-17 	(foo && bar; ?!AMP?! baz) ?!AMP?!
+17 	(foo && bar; ?!LINT: missing '&&'?! baz) ?!LINT: missing '&&'?!
 18 
 19 	foobar
 20 )
diff --git a/t/chainlint/token-pasting.expect b/t/chainlint/token-pasting.expect
index 64f3235d26..387189b6de 100644
--- a/t/chainlint/token-pasting.expect
+++ b/t/chainlint/token-pasting.expect
@@ -2,13 +2,13 @@
 3 git config filter.rot13.clean ./rot13.sh &&
 4 
 5 {
-6     echo "*.t filter=rot13" ?!AMP?!
+6     echo "*.t filter=rot13" ?!LINT: missing '&&'?!
 7     echo "*.i ident"
 8 } >.gitattributes &&
 9 
 10 {
-11     echo a b c d e f g h i j k l m ?!AMP?!
-12     echo n o p q r s t u v w x y z ?!AMP?!
+11     echo a b c d e f g h i j k l m ?!LINT: missing '&&'?!
+12     echo n o p q r s t u v w x y z ?!LINT: missing '&&'?!
 13     echo '$Id$'
 14 } >test &&
 15 cat test >test.t &&
@@ -19,7 +19,7 @@
 20 git checkout -- test test.t test.i &&
 21 
 22 echo "content-test2" >test2.o &&
-23 echo "content-test3 - filename with special characters" >"test3 'sq',$x=.o" ?!AMP?!
+23 echo "content-test3 - filename with special characters" >"test3 'sq',$x=.o" ?!LINT: missing '&&'?!
 24 
 25 downstream_url_for_sed=$(
 26 	printf "%sn" "$downstream_url" |
diff --git a/t/chainlint/unclosed-here-doc-indent.expect b/t/chainlint/unclosed-here-doc-indent.expect
index f78e23cb63..156906c85a 100644
--- a/t/chainlint/unclosed-here-doc-indent.expect
+++ b/t/chainlint/unclosed-here-doc-indent.expect
@@ -1,4 +1,4 @@
 2 command_which_is_run &&
-3 cat >expect <<-\EOF ?!UNCLOSED-HEREDOC?! &&
+3 cat >expect <<-\EOF ?!LINT: unclosed heredoc?! &&
 4 we forget to end the here-doc
 5 command_which_is_gobbled
diff --git a/t/chainlint/unclosed-here-doc.expect b/t/chainlint/unclosed-here-doc.expect
index 51304672cf..752c608862 100644
--- a/t/chainlint/unclosed-here-doc.expect
+++ b/t/chainlint/unclosed-here-doc.expect
@@ -1,5 +1,5 @@
 2 command_which_is_run &&
-3 cat >expect <<\EOF ?!UNCLOSED-HEREDOC?! &&
+3 cat >expect <<\EOF ?!LINT: unclosed heredoc?! &&
 4 	we try to end the here-doc below,
 5 	but the indentation throws us off
 6 	since the operator is not "<<-".
diff --git a/t/chainlint/while-loop.expect b/t/chainlint/while-loop.expect
index 5ffabd5a93..2ba5582165 100644
--- a/t/chainlint/while-loop.expect
+++ b/t/chainlint/while-loop.expect
@@ -1,14 +1,14 @@
 2 (
 3 	while true
 4 	do
-5 		echo foo ?!AMP?!
-6 		cat <<-\EOF ?!LOOP?!
+5 		echo foo ?!LINT: missing '&&'?!
+6 		cat <<-\EOF ?!LINT: missing '|| exit 1'?!
 7 		bar
 8 		EOF
-9 	done ?!AMP?!
+9 	done ?!LINT: missing '&&'?!
 10 
 11 	while true; do
 12 		echo foo &&
-13 		cat bar ?!LOOP?!
+13 		cat bar ?!LINT: missing '|| exit 1'?!
 14 	done
 15 )
-- 
2.46.0

[PATCH v2 3/3] chainlint: reduce annotation noise-factor

[Eric Sunshine]

From: Eric Sunshine <sunshine@sunshineco.com>

When chainlint detects a problem in a test definition, it highlights the
offending code with a "?!...?!" annotation. The rather curious "?!"
decoration was chosen to draw the reader's attention to the problem area
and to act as a good "needle" when using the terminal's search feature
to "jump" to the next problem.

Later, chainlint learned to color its output when sent to a terminal.
Problem annotations are colored with a red background which stands out
well from surrounding text, thus easily draws the reader's attention.
Together with the preceding change which gave all problem annotations a
uniform "LINT:" prefix, the noisy "?!" decoration has become superfluous
as a search "needle" so omit it when output is colored.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
---
 t/chainlint.pl | 3 ++-
 t/test-lib.sh  | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/t/chainlint.pl b/t/chainlint.pl
index ad26499478..f0598e3934 100755
--- a/t/chainlint.pl
+++ b/t/chainlint.pl
@@ -651,6 +651,7 @@ sub check_test {
 	$self->{nerrs} += @$problems;
 	return unless $emit_all || @$problems;
 	my $c = main::fd_colors(1);
+	my ($erropen, $errclose) = -t 1 ? ("$c->{rev}$c->{red}", $c->{reset}) : ('?!', '?!');
 	my $start = 0;
 	my $checked = '';
 	for (sort {$a->[1]->[2] <=> $b->[1]->[2]} @$problems) {
@@ -659,7 +660,7 @@ sub check_test {
 		my $err = format_problem($label);
 		$checked .= substr($body, $start, $pos - $start);
 		$checked .= ' ' unless $checked =~ /\s$/;
-		$checked .= "$c->{rev}$c->{red}?!LINT: $err?!$c->{reset}";
+		$checked .= "${erropen}LINT: $err$errclose";
 		$checked .= ' ' unless $pos >= length($body) ||
 		    substr($body, $pos, 1) =~ /^\s/;
 		$start = $pos;
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 54247604cb..278d1215f1 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -1606,7 +1606,7 @@ if test "${GIT_TEST_CHAIN_LINT:-1}" != 0 &&
    test "${GIT_TEST_EXT_CHAIN_LINT:-1}" != 0
 then
 	"$PERL_PATH" "$TEST_DIRECTORY/chainlint.pl" "$0" ||
-		BUG "lint error (see '?!...!? annotations above)"
+		BUG "lint error (see 'LINT' annotations above)"
 fi
 
 # Last-minute variable setup
-- 
2.46.0

Re: [PATCH v2 0/3] make chainlint output more newcomer-friendly

[Jeff King]

On Tue, Sep 10, 2024 at 12:10:10AM -0400, Eric Sunshine wrote:

> Changes since v1:
> 
> * new patch [1/3] -- motivated by Junio's observation[2] about
>   availability of structured problem information -- takes advantage of
>   that information directly rather than post-processing "?!...?!"
>   sequences in the output stream
> 
> * old patch [2/2] (now [3/3]) which drops "?!" decorations when emitting
>   colored output to a terminal partially justified the change by
>   claiming that the new "ERR" (or "ERR:") prefix is a good "needle" for
>   a terminal's search feature, thus the noisy "?!" is no longer needed;
>   however, I realized that "ERR" (or "ERR:") is, in fact, an awful
>   needle since the string "err" (or "err:") is quite likely to
>   legitimately appear in source text, hence I changed the prefix to
>   "LINT:" (with the colon since Patrick found lack of colon
>   confusing[3])
> 
> * rewrote commit messages based upon feedback from Junio[2,4]
> 
> * dropped an unused argument from the call to format_problem() which was
>   an artifact used briefly during development of v1

Very nice. I think the "LINT:" prefix does a good job of standing out,
after spot-checking the output of a few of the tests.

I read through the commits themselves and didn't have any suggestions.

-Peff

Re: [PATCH v2 2/3] chainlint: make error messages self-explanatory

[Patrick Steinhardt]

On Tue, Sep 10, 2024 at 12:10:12AM -0400, Eric Sunshine wrote:
> diff --git a/t/chainlint/arithmetic-expansion.expect b/t/chainlint/arithmetic-expansion.expect
> index 338ecd5861..5677e16cad 100644
> --- a/t/chainlint/arithmetic-expansion.expect
> +++ b/t/chainlint/arithmetic-expansion.expect
> @@ -4,6 +4,6 @@
>  5 	baz
>  6 ) &&
>  7 (
> -8 	bar=$((42 + 1)) ?!AMP?!
> +8 	bar=$((42 + 1)) ?!LINT: missing '&&'?!
>  9 	baz
>  10 )

This looks a lot nicer than both the old state and the first iteration.
I certainly like it! I'm not really able to comment on the Perl code,
which mostly looks like gibberish to me (which isn't your fault). I'll
leave it to others to comment on that.

Patrick

Re: [PATCH v2 3/3] chainlint: reduce annotation noise-factor

[Patrick Steinhardt]

On Tue, Sep 10, 2024 at 12:10:13AM -0400, Eric Sunshine wrote:
> From: Eric Sunshine <sunshine@sunshineco.com>
> 
> When chainlint detects a problem in a test definition, it highlights the
> offending code with a "?!...?!" annotation. The rather curious "?!"
> decoration was chosen to draw the reader's attention to the problem area
> and to act as a good "needle" when using the terminal's search feature
> to "jump" to the next problem.
> 
> Later, chainlint learned to color its output when sent to a terminal.
> Problem annotations are colored with a red background which stands out
> well from surrounding text, thus easily draws the reader's attention.
> Together with the preceding change which gave all problem annotations a
> uniform "LINT:" prefix, the noisy "?!" decoration has become superfluous
> as a search "needle" so omit it when output is colored.
> 
> Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
> ---
>  t/chainlint.pl | 3 ++-
>  t/test-lib.sh  | 2 +-
>  2 files changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/t/chainlint.pl b/t/chainlint.pl
> index ad26499478..f0598e3934 100755
> --- a/t/chainlint.pl
> +++ b/t/chainlint.pl
> @@ -651,6 +651,7 @@ sub check_test {
>  	$self->{nerrs} += @$problems;
>  	return unless $emit_all || @$problems;
>  	my $c = main::fd_colors(1);
> +	my ($erropen, $errclose) = -t 1 ? ("$c->{rev}$c->{red}", $c->{reset}) : ('?!', '?!');
>  	my $start = 0;
>  	my $checked = '';
>  	for (sort {$a->[1]->[2] <=> $b->[1]->[2]} @$problems) {

I was first wondering why we didn't have to change our tests. But this
seems to use either coloring or the `?!` decorations based on whether or
not we output to a terminal. And as our tests output to a non-terminal
they indeed see the old format, and as such they don't have to change.

One thing I don't like about this is that we now have different output
depending on whether or not you happen to pipe output to e.g. less(1),
which I do quite frequently. So I'd propose to just drop the markers
unconditionally.

Patrick

Re: [PATCH v2 3/3] chainlint: reduce annotation noise-factor

[Eric Sunshine]

On Tue, Sep 10, 2024 at 3:48 AM Patrick Steinhardt <ps@pks.im> wrote:
> On Tue, Sep 10, 2024 at 12:10:13AM -0400, Eric Sunshine wrote:
> > [...]
> > Later, chainlint learned to color its output when sent to a terminal.
> > Problem annotations are colored with a red background which stands out
> > well from surrounding text, thus easily draws the reader's attention.
> > Together with the preceding change which gave all problem annotations a
> > uniform "LINT:" prefix, the noisy "?!" decoration has become superfluous
> > as a search "needle" so omit it when output is colored.
> >
> > Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
> > ---
> > +     my ($erropen, $errclose) = -t 1 ? ("$c->{rev}$c->{red}", $c->{reset}) : ('?!', '?!');
>
> I was first wondering why we didn't have to change our tests. But this
> seems to use either coloring or the `?!` decorations based on whether or
> not we output to a terminal. And as our tests output to a non-terminal
> they indeed see the old format, and as such they don't have to change.

Correct.

> One thing I don't like about this is that we now have different output
> depending on whether or not you happen to pipe output to e.g. less(1),
> which I do quite frequently. So I'd propose to just drop the markers
> unconditionally.

My knee-jerk reaction is that the "?!" decoration is still handy for
drawing the eye when scanning non-colored output visually (not using a
search feature), so I'm hesitant to drop it. However, on reflection,
I'm not sure I feel very strongly about it. What do others think?

Re: [PATCH v2 3/3] chainlint: reduce annotation noise-factor

[Junio C Hamano]

Eric Sunshine <sunshine@sunshineco.com> writes:

>> One thing I don't like about this is that we now have different output
>> depending on whether or not you happen to pipe output to e.g. less(1),
>> which I do quite frequently. So I'd propose to just drop the markers
>> unconditionally.
>
> My knee-jerk reaction is that the "?!" decoration is still handy for
> drawing the eye when scanning non-colored output visually (not using a
> search feature), so I'm hesitant to drop it. However, on reflection,
> I'm not sure I feel very strongly about it. What do others think?

Unlike ERR, LINT is distinct enough, even when mixed with snippets
taken from the test scripts that are full of words that hints
errors, checking, etc., so I'd expect that new readers who have
never seen the "?!" eye-magnets would not find the output too hard
to read.  For those of us whose eyes are so used to, we might miss
them for a while, but I do not see much upside in keeping it.

Thanks.

Re: [PATCH v2 1/3] chainlint: don't be fooled by "?!...?!" in test body

[Junio C Hamano]

Eric Sunshine <ericsunshine@charter.net> writes:

> However, when 73c768dae9 (chainlint: annotate original test definition
> rather than token stream, 2022-11-08) taught chainlint to output the
> original test text verbatim, it started collecting structured
> information about detected problems.
>
> Now that it is available, take advantage of the structured problem
> information to deterministically count the number of problems detected
> and to color the annotations directly, rather than scanning the output
> stream for "?!...?!" and performing these operations after-the-fact.

Makes sense.  Nicely done.

Will queue.  Thanks.

Re: [PATCH v2 0/3] make chainlint output more newcomer-friendly

[Junio C Hamano]

Eric Sunshine <ericsunshine@charter.net> writes:

> * new patch [1/3] -- motivated by Junio's observation[2] about
>   availability of structured problem information -- takes advantage of
>   that information directly rather than post-processing "?!...?!"
>   sequences in the output stream

;-).

> * old patch [2/2] (now [3/3]) which drops "?!" decorations when emitting
>   colored output to a terminal partially justified the change by
>   claiming that the new "ERR" (or "ERR:") prefix is a good "needle" for
>   a terminal's search feature, thus the noisy "?!" is no longer needed;
>   however, I realized that "ERR" (or "ERR:") is, in fact, an awful
>   needle since the string "err" (or "err:") is quite likely to
>   legitimately appear in source text, hence I changed the prefix to
>   "LINT:" (with the colon since Patrick found lack of colon
>   confusing[3])

Nice; I prefer LINT over ERR quite a lot.

> Unfortunately, the included range-diff is a mess and pretty much useless

That's expected and OK after a large update of any series, which
often deserves to be read from cover to cover anyway.

> -	$checked =~ s/(\s) \?!/$1?!/mg;
> -	$checked =~ s/\?! (\s)/?!$1/mg;
> -	$checked =~ s/\?!([^?]+)\?!/$erropen$1$errclose/mg;

;-)

Re: [PATCH v2 3/3] chainlint: reduce annotation noise-factor

[Eric Sunshine]

On Tue, Sep 10, 2024 at 11:42 AM Junio C Hamano <gitster@pobox.com> wrote:
> Eric Sunshine <sunshine@sunshineco.com> writes:
> > On Tue, Sep 10, 2024 at 3:48 AM Patrick Steinhardt <ps@pks.im> wrote:
> >> One thing I don't like about this is that we now have different output
> >> depending on whether or not you happen to pipe output to e.g. less(1),
> >> which I do quite frequently. So I'd propose to just drop the markers
> >> unconditionally.
> >
> > My knee-jerk reaction is that the "?!" decoration is still handy for
> > drawing the eye when scanning non-colored output visually (not using a
> > search feature), so I'm hesitant to drop it. However, on reflection,
> > I'm not sure I feel very strongly about it. What do others think?
>
> Unlike ERR, LINT is distinct enough, even when mixed with snippets
> taken from the test scripts that are full of words that hints
> errors, checking, etc., so I'd expect that new readers who have
> never seen the "?!" eye-magnets would not find the output too hard
> to read.  For those of us whose eyes are so used to, we might miss
> them for a while, but I do not see much upside in keeping it.

Okay, thanks for weighing in. I'll reroll and make [3/3] drop the "?!"
decoration unconditionally.