Re: [PATCH v2 2/6] t/test-lib: don't print shell traces to stdout

[Patrick Steinhardt <ps@pks.im>]

On Tue, May 27, 2025 at 12:47:00PM -0700, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> > We have several flags like "--verbose", "--verbose-only" or "-x" that
> > cause us to generate shell traces. The generated tracing output is split
> > up in these cases so that the test's stdout is printed to file
> > descriptor 3 whereas its stderr is printed to file descriptor 4.
> > Depending on which options have been given, we then end up either:
> >
> >   - Redirecting both file descriptors to a file.
> >
> >   - Redirecting them to stdout and stderr, respectively.
> >
> >   - Closing them in case we're running in none-verbose mode.
> >
> > The second case causes problems though when passing output to a TAP
> > parser. We print the test's stdout to the console's stdout, and that
> > results in broken TAP output.
> >
> > Fix the issue by instead redirecting the test's stdout to the shell's
> > stderr. This makes it impossible to discern stdout from stderr, but
> > going by my own experience I never came across a usecase where I would
> > have needed this distinction.
> 
> OK, so both stdout and stderr go to stderr, mixing everything into a
> single stream.  Do we need to worry about funny buffering making the
> test output harder to verify?  I mean, we only have to care about
> the ordering of lines within the original standard output (or
> standard error) stream independently, but now if the test thinks it
> wrote A to its stderr, then B to its stdout, and then C to its
> stderr, would we get them in the single output stream as A followed
> by B followed by C, or can sometimes buffered output can give us A
> then C then finally B?

Good question -- I think that the answer is yes, this can indeed cause
intermingled output. The redirect to the best of my knowledge would not
impact the buffering behaviour of stdout/stderr. I'm not really sure how
to address this though.

> Just an idle thought.  What makes me more confused is that the
> updated t0000 tests seem to say that we now check standard output
> and standard error separately.

To stdout goes any output that is expected to be parsed in the TAP
format, so all the "ok 1" or "not ok 2" lines. Everything else goes to
stderr.

Patrick