Re: [PATCH] tests: handle "funny" exit code 127 produced by MSVC-compiled exes

[Jeff King <peff@peff.net>]

On Mon, Oct 30, 2023 at 03:45:32PM +0000, Johannes Schindelin via GitGitGadget wrote:

> Now, `jk/tree-name-and-depth-limit` introduces a pair of test cases that
> expect a command that produces a stack overflow to fail, which it
> typically does with exit code 139 (which means SIGSEGV).

I think you're misinterpreting the purpose of the tests from that
series; they're not intended to segfault. Quoting from t6700:

  # We'll test against two depths here: a small one that will let us check the
  # behavior of the config setting easily, and a large one that should be
  # forbidden by default. Testing the default depth will let us know whether our
  # default is enough to prevent segfaults on systems that run the tests.

So for the "big tree" tests in that file, we are looking for a
controlled failure rather than a segfault. And indeed, the end of that
series already lowered the default to accommodate the msys windows
build; see the discussion in 4d5693ba05 (lower core.maxTreeDepth default
to 2048, 2023-08-31).

So I think the test is working as designed here: it is showing us that
the default value is not sufficient to protect MSVC builds from running
out of stack space. There are a few options there:

  1. We can lower the default everywhere.

  2. We can lower it just for MSVC builds.

  3. We can accept the situation and skip the tests for that build.

There's a bit more discussion in the commit I referenced above.

> Let's work around this by:
> 
> 1) recording which C compiler was used, and
> 
> 2) adding an MSVC-only exception to `test_must_fail` to treat 127 as a
>    regular failure.
> 
> There is a slight downside of this approach in that a real missing
> command could be mistaken for a failure. However, this would be caught
> on other platforms, and besides, we use `test_must_fail` only for `git`
> and `scalar` anymore, and we can be pretty certain that both are there.

I think there is another much worse downside to your patch: we will stop
noticing when MSVC builds segfault in the tests. The purpose of
test_must_fail is to allow controlled and expected failure returns from
the command, but still report on unexpected situations (signal death,
command not found, and so on).

-Peff