Re: [PATCH 0/5] ci: add GitLab CI definition

[Junio C Hamano <gitster@pobox.com>]

Patrick Steinhardt <ps@pks.im> writes:

>> So I have some hesitation about trying to mirror this rather complicated
>> set of build rules in another CI environment. My primary concern would
>> be that the two might fall out of sync and a series that is green on
>> GitHub would be red on GitLab, or vice-versa. Importantly, this can
>> happen even without changes to the build definitions, since (AFAICT)
>> both forges distribute new images automatically, so the set of packages
>> installed in GitHub may not exactly match what's in GitLab (and
>> vice-versa).
>
> Yup, that's a valid concern.

Is it?

I rather naïvely think different set of build options and tools
running the tests would mean we gain wider test coverage.  Even with
the current setup that relies on whatever GitHub offers, we already
see "this version passes all tests except for the job on macOS" and
"the version that was passing yesterday is not broken today---perhas
the image of the test environment has been updated and we need to
adjust to it" every once in a while.

> As mentioned, this patch series does not have the intent to make
> GitLab CI a second authoritative CI platform.  GitHub Actions
> should remain the source of truth of whether a pipeline passes or
> not.

I am not sure I follow.  Often we take a version that happened to
have failed Actions tests when we know the reason of the failure
has nothing to do with the new code.  From time to time people help
to make CI tests less flakey, but flakes are expected.

> Most importantly, I do not want to require the maintainer
> to now watch both pipelines on GitHub and GitLab.

I don't even make tests by GitHub Actions force me to do anything,
so there is no worry here.

> This might be another indicator that the pipeline should rather be
> in "contrib/", so that people don't start to treat it as
> authoritative.

Let me step back and as more basic questions.

 - What do you mean by "authoritative"?  For an authoritative CI
   test, contributors whose changes do not pass it should take it as
   a sign that their changes need more work?  If so, isn't it a
   natural expectation and a good thing?  Unless you expect the CI
   tests to be extra flakey, that is.

 - Are there reasons why you do not trust the CI tests at GitLab
   more than those run at GitHub?

> Last but not least, I actually think that having multiple supported CI
> platforms also has the benefit that people can more readily set it up
> for themselves. In theory, this has the potential to broaden the set of
> people willing to contribute to our `ci/` scripts, which would in the
> end also benefit GitHub Actions.

Yes, assuming that we can do so without much cutting and pasting but
with a clear sharing of the infrastructure code, and the multiple
supported CI environments are not too flakey, I am with this rather
naïve worldview that the more we have the merrier we would be.

> I understand your points, and especially the point about not having a
> second authoritative CI platform. I'm very much on the same page as you
> are here, and would be happy to move the definitions to "contrib/" if
> you want me to.
>
> But I think we should also see the potential benefit of having a second
> CI platform, as it enables a more diverse set of people to contribute.
> which can ultimately end up benefitting our CI infra for both GitHub
> Actions and GitLab CI.

I do *not* want to add new things, if we were to use them ourselves,
to "contrib/".  We have passed that stage long time ago that keeping
everything in my tree gives wider exposure and foster cooperation.

Thanks.