Re: [PATCH v2 11/11] ci: run expensive tests on push builds to integration branches

[Junio C Hamano <gitster@pobox.com>]

Derrick Stolee <stolee@gmail.com> writes:

> On 5/4/2026 1:08 PM, Johannes Schindelin via GitGitGadget wrote:
>> From: Johannes Schindelin <johannes.schindelin@gmx.de>
>> 
>> Derrick Stolee suggested [1] that expensive tests should be run at a
>> regular cadence rather than on every PR iteration. Gate GIT_TEST_LONG
>> on push builds to the integration branches (next, master, main, maint)
>> so that the EXPENSIVE prereq is satisfied there but not during PR
>> validation, where the extra minutes of wall-clock time do not justify
>> themselves.
> I like that this will be run as part of regular updates to the
> important branches. The important bit after that is whether or
> not a human pays attention to the signal of these builds.
>
> Junio: Do you pay attention to CI breaks when you push to
> 'master'?

Well, it is way too late to notice breakage when the faulty update
hits 'master'.  CI failures should be noticed before breakage hits
'next'.

I often notice and complain when I see failures on 'seen', and
sometimes I help original submitter by bisecting, but I do not
necessarily have enough time and bandwidth to help everybody.

Quite honestly, the best place to give widest test coverage is much
closer to the source of the problems than in my tree and mixed with
other topics, i.e., at individual contributor's CI.  That way, I
presume that GitGitGadget can also help submitters avoid sending a
faulty series, reducing the load on the list and the maintainer.

Ideally the CI tests by the integrator should only be catching any
mismerges and unexpected inter-topic interactions, as they cannot be
caught by contributor's standalone tests, so I do not mind widening
coverage of CI tests when I push the integration results out.  But
so far, the majority of what I have seen and reported back to the
list have been something that the authors should be equipped to spot
in their topic without getting mixed with other topics into any
integration branches.

> One way to help this procedure could be to have GitHub CI
> failures trigger new issues, which could then be more easily
> viewed and noticed by the community watching the repo. This
> is of course out-of-scope for this patch series, but could be
> considered in the future.

I think a better way to help would be to arrange the workflow so
that we do not even have to trigger an issue, and stop before the
patches leave the original authors' hand.  They can of course ask
for help saying "here is my topic in my fork of the repository and
failing in this way for macOS that I do not have access to.  Could
anybody help me figuring out what macOS peculiarity my changes are
tickling?", or something like that.

It would be best to find problems early, and make it easier for
individual contributors to help each other by having a concrete CI
failure reports in their forks that they can point at when they ask
for help.  And CI run when I push 'seen' or 'master' out would not
help as much as CI run when they publish their forked branches would.

By the way, please expect slow responses as I am (officially) still
mostly offline for the rest of the week.

Thanks.