Re: [PATCH v6 4/4] Jobs based on custom runners: add job definitions for QEMU's machines

["Alex Bennée" <alex.bennee@linaro.org>]

Wainer dos Santos Moschetta <wainersm@redhat.com> writes:

> Hi,
>
> On 6/8/21 12:14 AM, Cleber Rosa wrote:
>> The QEMU project has two machines (aarch64 and s390x) that can be used
>> for jobs that do build and run tests.  This introduces those jobs,
>> which are a mapping of custom scripts used for the same purpose.
>>
>> Signed-off-by: Cleber Rosa <crosa@redhat.com>
>> ---
>>   .gitlab-ci.d/custom-runners.yml | 208 ++++++++++++++++++++++++++++++++
>>   1 file changed, 208 insertions(+)
>>
>> diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml
>> index a07b27384c..061d3cdfed 100644
>> --- a/.gitlab-ci.d/custom-runners.yml
>> +++ b/.gitlab-ci.d/custom-runners.yml
>> @@ -12,3 +12,211 @@
>>   # guarantees a fresh repository on each job run.
>>   variables:
>>     GIT_STRATEGY: clone
>> +
>> +# All ubuntu-18.04 jobs should run successfully in an environment
>> +# setup by the scripts/ci/setup/build-environment.yml task
>> +# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
>> +ubuntu-18.04-s390x-all-linux-static:
>> + allow_failure: true
>> + needs: []
>> + stage: build
>> + tags:
>> + - ubuntu_18.04
>> + - s390x
>> + rules:
>> + - if: '$CI_COMMIT_BRANCH =~ /^staging/'
>
> Should it restrict the job for pushes to qemu-project only? If yes,
> then it probably needs the statement:
>
> '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
>
> If you change that here, you will end it changing all over the jobs.
> In general, there are many boilerplates in this file. I'm ok to merge
> it as is as long as it is followed by another series to refactor the
> code.
>
>> + script:
>> + # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763
>> + # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages
>> + - mkdir build
>> + - cd build
>> + - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh
>> + - make --output-sync -j`nproc`
>> + - make --output-sync -j`nproc` check V=1
>> + - make --output-sync -j`nproc` check-tcg V=1
>> +
>> +ubuntu-18.04-s390x-all:
>> + allow_failure: true
>> + needs: []
>> + stage: build
>> + tags:
>> + - ubuntu_18.04
>> + - s390x
>> + rules:
>> + - if: '$CI_COMMIT_BRANCH =~ /^staging/'
>> + script:
>> + - mkdir build
>> + - cd build
>> + - ../configure --disable-libssh
>> + - make --output-sync -j`nproc`
>> + - make --output-sync -j`nproc` check V=1
>> +
>> +ubuntu-18.04-s390x-alldbg:
> Maybe we don't need both ubuntu-18.04-s390x-all and
> ubuntu-18.04-s390x-alldbg jobs.
>> + allow_failure: true
>> + needs: []
>> + stage: build
>> + tags:
>> + - ubuntu_18.04
>> + - s390x
>> + rules:
>> + - if: '$CI_COMMIT_BRANCH =~ /^staging/'
>> + script:
>> + - mkdir build
>> + - cd build
>> + - ../configure --enable-debug --disable-libssh
>> + - make clean
>> + - make --output-sync -j`nproc`
>> + - make --output-sync -j`nproc` check V=1
>> +ubuntu-18.04-s390x-clang:
>> + allow_failure: true
>> + needs: []
>> + stage: build
>> + tags:
>> + - ubuntu_18.04
>> + - s390x
>> + rules:
>> + - if: '$CI_COMMIT_BRANCH =~ /^staging/'
>> +   when: manual
>> + script:
>> + - mkdir build
>> + - cd build
>> + - ../configure --disable-libssh --cc=clang --cxx=clang++ --enable-sanitizers
>> + - make --output-sync -j`nproc`
>> + - make --output-sync -j`nproc` check V=1
>> +
>> +ubuntu-18.04-s390x-tci:
>> + allow_failure: true
>> + needs: []
>> + stage: build
>> + tags:
>> + - ubuntu_18.04
>> + - s390x
>> + rules:
>> + - if: '$CI_COMMIT_BRANCH =~ /^staging/'
>> + script:
>> + - mkdir build
>> + - cd build
>> + - ../configure --disable-libssh --enable-tcg-interpreter
>> + - make --output-sync -j`nproc`
> I think it needs to `make check-tcg` at least. See "build-tci" in
> `.gitlab-ci.d/buildtest.yml` for other tests being executed on shared
> runners.

To get anything other than the s390x-linux-user tests we will need the
cross compilers installed. Currently we don't really use docker for
anything other than x86_64 hosts (and some aarch64 which I've tested).

>> +
>> +ubuntu-18.04-s390x-notcg:
> The "build-tcg-disabled" in `.gitlab-ci.d/buildtest.yml` could be
> mimic-ed here too.
>> + allow_failure: true
>> + needs: []
>> + stage: build
>> + tags:
>> + - ubuntu_18.04
>> + - s390x
>> + rules:
>> + - if: '$CI_COMMIT_BRANCH =~ /^staging/'
>> +   when: manual
>> + script:
>> + - mkdir build
>> + - cd build
>> + - ../configure --disable-libssh --disable-tcg
>> + - make --output-sync -j`nproc`
>> + - make --output-sync -j`nproc` check V=1
>> +
>> +# All ubuntu-20.04 jobs should run successfully in an environment
>> +# setup by the scripts/ci/setup/qemu/build-environment.yml task
>> +# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
>> +ubuntu-20.04-aarch64-all-linux-static:
>> + allow_failure: true
>> + needs: []
>> + stage: build
>> + tags:
>> + - ubuntu_20.04
>> + - aarch64
>> + rules:
>> + - if: '$CI_COMMIT_BRANCH =~ /^staging/'
>> + script:
>> + # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763
>> + # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages
>> + - mkdir build
>> + - cd build
>> + - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh
>> + - make --output-sync -j`nproc`
>> + - make --output-sync -j`nproc` check V=1
>> + - make --output-sync -j`nproc` check-tcg V=1
>> +
>> +ubuntu-20.04-aarch64-all:
>> + allow_failure: true
>> + needs: []
>> + stage: build
>> + tags:
>> + - ubuntu_20.04
>> + - aarch64
>> + rules:
>> + - if: '$CI_COMMIT_BRANCH =~ /^staging/'
>> + script:
>> + - mkdir build
>> + - cd build
>> + - ../configure --disable-libssh
>> + - make --output-sync -j`nproc`
>> + - make --output-sync -j`nproc` check V=1
>> +
>> +ubuntu-20.04-aarch64-alldbg:
>> + allow_failure: true
>> + needs: []
>> + stage: build
>> + tags:
>> + - ubuntu_20.04
>> + - aarch64
>> + rules:
>> + - if: '$CI_COMMIT_BRANCH =~ /^staging/'
>> + script:
>> + - mkdir build
>> + - cd build
>> + - ../configure --enable-debug --disable-libssh
>> + - make clean
>> + - make --output-sync -j`nproc`
>> + - make --output-sync -j`nproc` check V=1
>> +
>> +ubuntu-20.04-aarch64-clang:
>> + allow_failure: true
>> + needs: []
>> + stage: build
>> + tags:
>> + - ubuntu_20.04
>> + - aarch64
>> + rules:
>> + - if: '$CI_COMMIT_BRANCH =~ /^staging/'
>> +   when: manual
>> + script:
>> + - mkdir build
>> + - cd build
>> + - ../configure --disable-libssh --cc=clang-10 --cxx=clang++-10 --enable-sanitizers
>> + - make --output-sync -j`nproc`
>> + - make --output-sync -j`nproc` check V=1
>> +
>> +ubuntu-20.04-aarch64-tci:
>> + allow_failure: true
>> + needs: []
>> + stage: build
>> + tags:
>> + - ubuntu_20.04
>> + - aarch64
>> + rules:
>> + - if: '$CI_COMMIT_BRANCH =~ /^staging/'
>> + script:
>> + - mkdir build
>> + - cd build
>> + - ../configure --disable-libssh --enable-tcg-interpreter
>> + - make --output-sync -j`nproc`
>> +
>> +ubuntu-20.04-aarch64-notcg:
>> + allow_failure: true
>> + needs: []
>> + stage: build
>> + tags:
>> + - ubuntu_20.04
>> + - aarch64
>> + rules:
>> + - if: '$CI_COMMIT_BRANCH =~ /^staging/'
>> +   when: manual
>> + script:
>> + - mkdir build
>> + - cd build
>> + - ../configure --disable-libssh --disable-tcg
>> + - make --output-sync -j`nproc`
>> + - make --output-sync -j`nproc` check V=1


-- 
Alex Bennée