[PATCH v4 0/2] ci: link failure and leak annotations to the test script

["Harald Nordgren via GitGitGadget" <gitgitgadget@gmail.com>]

Link failure and leak annotations in CI to the test script, so both can be
found from the job summary.

V4 CI Job where failures and leaks are reported:
https://github.com/git/git/actions/runs/36760014277/job/110039964565?pr=2426

Changes in v4:

 * Clarify commit messages and simplify escaping logic.

Changes in v3:

 * Fixed bug in the --immediate exit ordering: the --immediate &&
   --invert-exit-code path called exit 0 before the test's annotation was
   written, now a single unconditional call covers both exit paths.
 * github_escape_message_ no longer relies on \r being a portable sed escape
   sequence (not POSIX-guaranteed and BSD sed implementations can differ),
   it splices in the literal carriage-return byte via printf instead.
 * Reverted unrelated test-tool line back to its original form.

Changes in v2:

 * Split into two commits, each explaining its own reasoning.
 * Leak output is no longer capped or embedded in the message, it's now an
   uncapped fold, so multiple leaks in the same test both show in full. A
   second leak in a different test still won't show in the same run,
   --immediate stops the script at the first failure, but it no longer gets
   buried under every later test falsely reporting "not ok" either.
 * Drops the giant unfolded message that annotations used to carry, which is
   what probably caused the scrolling behavior.

Harald Nordgren (2):
  ci: annotate leaks and stop a leak-sanitizer script at its first
    failure
  ci: point test failures and fixed known breakages at their file and
    line

 ci/lib.sh                            |  1 +
 t/test-lib-github-workflow-markup.sh | 54 ++++++++++++++++++++++++----
 t/test-lib.sh                        |  6 +++-
 3 files changed, 54 insertions(+), 7 deletions(-)


base-commit: a018953688f1b10bddf91bff8747068f5f4746a4
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2419%2FHaraldNordgren%2Fci-annotation-file-line-v4
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2419/HaraldNordgren/ci-annotation-file-line-v4
Pull-Request: https://github.com/git/git/pull/2419

Range-diff vs v3:

 1:  b6a36820ae ! 1:  138394b48b ci: annotate leaks and stop a leak-sanitizer script at its first failure
     @@ Commit message
      
          Give a leak its own annotation. Point it at the test script, the exact
          line isn't known, only which script the leak turned up in, and put the
     -    full sanitizer report in a log group next to it, so it stays visible
     -    and isn't capped to a handful of lines.
     +    sanitizer report in a log group next to it, so it stays visible.
      
          Once a script has one leak, it keeps running: the sanitizer log
          directory is never cleared between tests, so every later test in the
 2:  750c360512 ! 2:  8ec2b53d82 ci: point test failures and fixed known breakages at their file and line
     @@ Commit message
          ci: point test failures and fixed known breakages at their file and line
      
          A test failure or a fixed known breakage gets an annotation that names
     -    the test but carries no file or line, so there is nothing to click
     -    through to from the GitHub UI.
     +    the test but says nothing about where it's defined, so a reviewer has
     +    to search the script by hand to find it.
      
          Find the line a test is defined on by searching the script for its
     -    description as a fixed string, using the first match. A description
     -    can contain characters like `[` or `*` that a regex search would
     -    misread, so match it literally. Fall back to line 1 when the
     -    description is not found verbatim, which happens when a test builds
     -    its description at runtime instead of writing it out literally.
     +    description as a fixed string, using the first match. Fall back to
     +    line 1 when the description is not found verbatim, which happens when
     +    a test builds its description at runtime instead of writing it out
     +    literally.
      
     -    A GitHub annotation is a single line, and a test description is always
     -    one line too, so only a `%` or a stray carriage return in it needs
     -    percent-encoding to keep the annotation intact. Escape `%` first, or a
     -    carriage return's own encoding would be mangled by a `%` substitution
     -    that ran after it.
     +    A GitHub annotation is a single line, so a `%` in a test description
     +    has to be percent-encoded as `%25`, or GitHub misreads it as its own
     +    escape sequence. for-each-ref's format atoms use plenty of them, e.g.
     +    `%(raw)`.
      
          Signed-off-by: Harald Nordgren <haraldnordgren@gmail.com>
      
     @@ t/test-lib-github-workflow-markup.sh: start_test_output () {
       }
       
      +github_escape_message_ () {
     -+	# A test description is always one line, so only % and CR need
     -+	# escaping here. Escape % first, or CR's own %-encoding gets mangled.
     -+	# \r is not a portable sed escape, so splice in the actual byte.
     -+	sed -e 's/%/%25/g' -e "s/$(printf '\r')/%0D/g"
     ++	# % has to be escaped or GitHub misreads it as the start of its own
     ++	# percent-encoding (e.g. a literal %(raw) in a for-each-ref test
     ++	# description).
     ++	sed -e 's/%/%25/g'
      +}
      +
      +find_test_case_line_ () {
      +	# A description can contain characters like [ or * that would
      +	# corrupt a regex search, so match it literally and take the first
     -+	# hit; -- keeps a description starting with "-" from being read as
     -+	# an option.
     ++	# hit. The -- keeps a description starting with "-" from being read
     ++	# as an option.
      +	grep -n -F -- "$1" "$TEST_DIRECTORY/$github_markup_script_name" |
      +	head -n 1 | cut -d: -f1
      +}

-- 
gitgitgadget