Re: [PATCH 0/9] ci: make Git's GitHub workflow output much more helpful

["Ævar Arnfjörð Bjarmason" <avarab@gmail.com>]

On Mon, Mar 07 2022, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
>>> One thing I forgot to mention was that when you expand a failing test it shows
>>> the test script twice before the output e.g.
>>>
>>> Error: failed: t7527.35 Matrix[uc:false][fsm:true] enable fsmonitor
>>> failure: t7527.35 Matrix[uc:false][fsm:true] enable fsmonitor
>>>   				git config core.fsmonitor true &&
>>>   				git fsmonitor--daemon start &&
>>>   				git update-index --fsmonitor
>>>
>>>   expecting success of 7527.35 'Matrix[uc:false][fsm:true] enable fsmonitor':
>>>   				git config core.fsmonitor true &&
>>>   				git fsmonitor--daemon start &&
>>>   				git update-index --fsmonitor
>>>
>>>  ++ git config core.fsmonitor true
>>>  ++ git fsmonitor--daemon start
>>>  ...
>>>
>>> I don't know how easy it would be to fix that so that we only show "expecting
>>> success of ..." without the test being printed first
>>
>> It's not _super_ easy: right now, the patch series does not touch the code
>
> In other words, it is not a new issue introduced by this series, right?

It is a new issue in this series, specifically how
"finalize_test_case_output" interacts with "test_{ok,failure}_" and
friends.

>> The easiest workaround would probably to add a flag that suppresses the
>> header `expecting success` in case we're running with the
>> `--github-workflow-markup` option.
>
> If that is the case, let's leave it outside the series.
>
> If we do not have to hide the solution behind any option specific to
> "--github-workflow-markup", then even users (like me) who reguarly
> run "cd t && sh ./t1234-a-particular-test.sh -i -v" would benefit if
> we no longer have to look at the duplicated test script in the
> output.

Unless you invoke it with --github-workflow-markup you won't see the
duplication.

I had some comments about inherent limitations in the approach in this
series vis-a-vis parsing markup after the fact[1]. But that really
doesn't seem to apply here. We're just printing the test source into the
*.markup file twice for no particular reason, aren't we?

*tests locally*

Hrm, so first this is a bug:
    
    $ ./t0002-gitfile.sh  --github-workflow-markup
    FATAL: Unexpected exit with code 1
    FATAL: Unexpected exit with code 1
    
Seems it wants --tee but doesn't declare it, this works:

    $ rm -rf test-results/; ./t0002-gitfile.sh  --github-workflow-markup --tee; cat test-results/t0002-gitfile.markup

Isn't this a matter of making finalize_test_case_output not print the
full $* (including the test source) for failures?

1. https://lore.kernel.org/git/220309.861qzbnymn.gmgdl@evledraar.gmail.com/