Re: [PATCH 2/3] ci: avoid bare "gcc" for osx-gcc job

Jeff King
2024-05-16T07:27:55Z

On Thu, May 16, 2024 at 03:19:30AM -0400, Jeff King wrote:

> On Sat, May 11, 2024 at 07:21:28PM +0200, Patrick Steinhardt wrote:
>
> > On Fri, May 10, 2024 at 03:47:39PM -0700, Junio C Hamano wrote:
> > > Jeff King <peff@peff.net> writes:
> > [snip]
> > > > [1] Another quirk is that we run the whole test suite for both
> > > > compilers, which is probably overkill. The main value in comparing
> > > > gcc vs clang is that we don't use any constructs that the compiler
> > > > complains about. It's _possible_ for there to be a construct that
> > > > the compiler does not notice but which causes a runtime difference
> > > > (say, undefined behavior which happens to work out on one compiler),
> > > > but I think we're again hitting diminishing returns.
> > >
> > > Yeah, that is a very good point.
> >
> > On Linux, we have the "pedantic" job that runs Fedora and only compiles
> > the sources with DEVOPTS=pedantic without running any of the tests. We
> > could do the same on macOS.
>
> Yeah, I think the infrastructure is there (looks like just resetting
> $run_tests). We probably could stand to use it in more places. E.g., is
> there even value in running the tests for linux-gcc and linux-clang?
> It's _possible_ for there to be a run-time difference in the compiler
> outputs, but we may be hitting diminishing returns. The main value I
> think is just seeing what the compilers complain about.

Actually, scratch that. I forgot we dropped linux-clang last summer in
d88d727143 (ci: drop linux-clang job, 2023-06-01), since it was mostly
redundant with the sanitizer builds (which use clang).

There is still "linux-gcc" and "linux-gcc-default", the former of which
uses an older version of the compiler. And again, it's not clear to me
that running the actual tests is uncovering useful stuff there.
Certainly it is possible, but I wonder if that is the best bang for the
buck (e.g., if we wanted to spend a "make test" worth of CPU, then
something like osx-sha256 would IMHO be more likely to uncover something
useful).

-Peff