Re: [PATCH 0/5] ci: add GitLab CI definition

[Patrick Steinhardt <ps@pks.im>]

[-- Attachment #1: Type: text/plain, Size: 4890 bytes --]

On Wed, Nov 01, 2023 at 09:15:51AM +0900, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> 
> >> So I have some hesitation about trying to mirror this rather complicated
> >> set of build rules in another CI environment. My primary concern would
> >> be that the two might fall out of sync and a series that is green on
> >> GitHub would be red on GitLab, or vice-versa. Importantly, this can
> >> happen even without changes to the build definitions, since (AFAICT)
> >> both forges distribute new images automatically, so the set of packages
> >> installed in GitHub may not exactly match what's in GitLab (and
> >> vice-versa).
> >
> > Yup, that's a valid concern.
> 
> Is it?
> 
> I rather naïvely think different set of build options and tools
> running the tests would mean we gain wider test coverage.  Even with
> the current setup that relies on whatever GitHub offers, we already
> see "this version passes all tests except for the job on macOS" and
> "the version that was passing yesterday is not broken today---perhas
> the image of the test environment has been updated and we need to
> adjust to it" every once in a while.
> 
> > As mentioned, this patch series does not have the intent to make
> > GitLab CI a second authoritative CI platform.  GitHub Actions
> > should remain the source of truth of whether a pipeline passes or
> > not.
> 
> I am not sure I follow.  Often we take a version that happened to
> have failed Actions tests when we know the reason of the failure
> has nothing to do with the new code.  From time to time people help
> to make CI tests less flakey, but flakes are expected.
> 
> > Most importantly, I do not want to require the maintainer
> > to now watch both pipelines on GitHub and GitLab.
> 
> I don't even make tests by GitHub Actions force me to do anything,
> so there is no worry here.

Okay.

> > This might be another indicator that the pipeline should rather be
> > in "contrib/", so that people don't start to treat it as
> > authoritative.
> 
> Let me step back and as more basic questions.
> 
>  - What do you mean by "authoritative"?  For an authoritative CI
>    test, contributors whose changes do not pass it should take it as
>    a sign that their changes need more work?  If so, isn't it a
>    natural expectation and a good thing?  Unless you expect the CI
>    tests to be extra flakey, that is.

I was assuming that GitHub Actions was considered to be "the" CI
platform of the Git project. But with your explanations above I think
that assumption may not necessarily hold, or at least not to the extent
I assumed.

>  - Are there reasons why you do not trust the CI tests at GitLab
>    more than those run at GitHub?

No. Based on the above assumption I was simply treading carefully here.
Most importantly, I didn't want to create the impression that either:

    - "Now you have to watch two pipelines", doubling the effort that CI
      infrastructure creates for you as a maintainer.

    - "I want to eventually replace GitHub Actions".

This carefulness probably also comes from the fact that GitLab and
GitHub are direct competitors, so I was trying to preempt any kind of
implied agenda here. There is none, I just want to make sure that it
becomes easier for us at GitLab and other potential contributors that
use GitLab to contribute to Git.

Hope that makes sense.

> > Last but not least, I actually think that having multiple supported CI
> > platforms also has the benefit that people can more readily set it up
> > for themselves. In theory, this has the potential to broaden the set of
> > people willing to contribute to our `ci/` scripts, which would in the
> > end also benefit GitHub Actions.
> 
> Yes, assuming that we can do so without much cutting and pasting but
> with a clear sharing of the infrastructure code, and the multiple
> supported CI environments are not too flakey, I am with this rather
> naïve worldview that the more we have the merrier we would be.
> 
> > I understand your points, and especially the point about not having a
> > second authoritative CI platform. I'm very much on the same page as you
> > are here, and would be happy to move the definitions to "contrib/" if
> > you want me to.
> >
> > But I think we should also see the potential benefit of having a second
> > CI platform, as it enables a more diverse set of people to contribute.
> > which can ultimately end up benefitting our CI infra for both GitHub
> > Actions and GitLab CI.
> 
> I do *not* want to add new things, if we were to use them ourselves,
> to "contrib/".  We have passed that stage long time ago that keeping
> everything in my tree gives wider exposure and foster cooperation.

Fair enough.

Thanks for taking the time to make your thoughts clearer to me!

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]