Re: Should we maybe move Cirrus-CI jobs away from Gitlab again?

[Thomas Huth <thuth@redhat.com>]

On 27/09/2022 21.10, Stefan Hajnoczi wrote:
> On Tue, 27 Sept 2022 at 15:04, Thomas Huth <thuth@redhat.com> wrote:
>>
>> On 27/09/2022 20.47, Stefan Hajnoczi wrote:
>>> On Tue, 27 Sept 2022 at 14:40, Thomas Huth <thuth@redhat.com> wrote:
>>>>
>>>> On 27/09/2022 19.57, Daniel P. Berrangé wrote:
>>>>> On Tue, Sep 27, 2022 at 01:36:20PM -0400, Stefan Hajnoczi wrote:
>>>>>> On Tue, 27 Sept 2022 at 11:54, Daniel P. Berrangé <berrange@redhat.com> wrote:
>>>>>>>
>>>>>>> On Tue, Sep 27, 2022 at 11:44:45AM -0400, Stefan Hajnoczi wrote:
>>>>>>>> On Tue, 27 Sept 2022 at 05:02, Thomas Huth <thuth@redhat.com> wrote:
>>>>>>>>> now that Gitlab is giving us pressure on the amount of free CI minutes, I
>>>>>>>>> wonder whether we should maybe move the Cirrus-CI jobs out of the gitlab-CI
>>>>>>>>> dashboard again? We could add the jobs to our .cirrus-ci.yml file instead,
>>>>>>>>> like we did it in former times...
>>>>>>>>>
>>>>>>>>> Big advantage would be of course that the time for those jobs would not
>>>>>>>>> count in the Gitlab-CI minutes anymore. Disadvantage is of course that they
>>>>>>>>> do not show up in the gitlab-CI dashboard anymore, so there is no more
>>>>>>>>> e-mail notification about failed jobs, and you have to push to github, too,
>>>>>>>>> and finally check the results manually on cirrus-ci.com ...
>>>>>>>>
>>>>>>>> My understanding is that .gitlab-ci.d/cirrus.yml uses a GitLab CI job
>>>>>>>> to run the cirrus-run container image that forwards jobs to Cirrus-CI.
>>>>>>>> So GitLab CI resources are consumed waiting for Cirrus-CI to finish.
>>>>>>>>
>>>>>>>> This shouldn't affect gitlab.com/qemu-project where there are private
>>>>>>>> runners that do not consume GitLab CI minutes.
>>>>>>>>
>>>>>>>> Individual developers are affected though because they most likely
>>>>>>>> rely on the GitLab shared runner minutes quota.
>>>>>>>
>>>>>>> NB, none of the jobs should ever be run automatically anymore in
>>>>>>> QEMU CI pipelines. It always requires the maintainer to set the
>>>>>>> env var when pushing to git, to explicitly create a pipeline.
>>>>>>> You can then selectively start each individual job as desired.
>>>>>>
>>>>>> Cirrus CI is not automatically started when pushing to a personal
>>>>>> GitLab repo? If starting it requires manual action anyway then I think
>>>>>> nothing needs to be changed here.
>>>>>
>>>>> No pipeline at all is created unless you do
>>>>>
>>>>>      git push -o ci.variable=QEMU_CI=1 <your-fork-remote>
>>>>>
>>>>> that creates the pipeliune but doesn't run any jobs - they're manual
>>>>> start.
>>>>
>>>> Yes, sure, the jobs are not started automatically. But I *do* want to run
>>>> the jobs before sending pull requests - but since the gitlab-CI minutes are
>>>> now very limited, I'd like to avoid burning these minutes via gitlab and
>>>> start those jobs directly on cirrus-ci.com again. For that the jobs would
>>>> need to be moved to our .cirrus-ci.yml file again.
>>>>
>>>> Well, maybe we could also have both, jobs via cirrus-run for those who want
>>>> to see them in their gitlab-CI dashboard, and via .cirrus-ci.yml for those
>>>> who want to avoid burning CI minutes on Gitlab. It's a little bit of
>>>> double-maintenance, but maybe acceptable?
>>>
>>> I just noticed that qemu.git/master doesn't run Cirrus-CI. I guess it
>>> hasn't been set up in our GitLab project.
>>>
>>> Since it's not enabled for qemu.git/master nothing will change from my
>>> perspective. Feel free to change it as you wish.
>>
>> It's only run for the "staging" branch, I think. The idea was that things
>> get tested before merge on the "staging" branch, then there is no need
>> anymore to rerun everything when it gets pushed into the "master" branch.
> 
> I don't see a cirrus job:
> https://gitlab.com/qemu-project/qemu/-/pipelines/652051335

That's likely because it had been disabled on github when we switched to 
cirrus-run via gitlab. I still have it enabled for my forked repo on github, 
so you can see runs here:

  https://cirrus-ci.com/github/huth/qemu/

  Thomas