Re: [PATCH 42/42] CI: Allow running tests on sjg lab

[Andrejs Cainikovs <andrejs.cainikovs@toradex.com>]

On Tue, Jun 11, 2024 at 02:01:56PM -0600, Simon Glass wrote:
> Add a way to run tests on a real hardware lab. This is in the very early
> experimental stages. There are only 23 boards and 3 of those are broken!
> (bob, ff3399, samus). A fourth fails due to problems with the TPM tests.
> 
> To try this, assuming you have gitlab access, set SJG_LAB=1, e.g.:
> 
>    git push -o ci.variable="SJG_LAB=1" dm HEAD:try
> 
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---
> 
>  .gitlab-ci.yml | 151 +++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 151 insertions(+)
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 165f765a833..6c362340341 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -17,6 +17,7 @@ stages:
>    - testsuites
>    - test.py
>    - world build
> +  - sjg-lab
>  
>  .buildman_and_testpy_template: &buildman_and_testpy_dfn
>    stage: test.py
> @@ -482,3 +483,153 @@ coreboot test.py:
>      TEST_PY_TEST_SPEC: "not sleep"
>      TEST_PY_ID: "--id qemu"
>    <<: *buildman_and_testpy_dfn
> +
> +.lab_template: &lab_dfn
> +  stage: sjg-lab
> +  rules:
> +    - when: always
> +  tags: [ 'lab' ]
> +  script:
> +    - if [[ -z "${SJG_LAB}" ]]; then
> +        exit 0;
> +      fi

Hi Simon,

Perhaps it would be better to move the check to rules:if?
I do know there's a lot of GitLab CI limitation when it comes to
variables, so not completely sure if this will work, but:

rules:
  - if: $SJG_LAB == "1"
    when: always
  - when: manual

My concern is that every of these jobs you added will spin up a docker
image even if you don't want to.

/Andrejs

> +    # Environment:
> +    #   SRC  - source tree
> +    #   OUT  - output directory for builds
> +    - export SRC="$(pwd)"
> +    - export OUT="${SRC}/build/${BOARD}"
> +    - export PATH=$PATH:~/bin
> +    - export PATH=$PATH:/vid/software/devel/ubtest/u-boot-test-hooks/bin
> +
> +    # Load it on the device
> +    - ret=0
> +    - echo "role ${ROLE}"
> +    - export strategy="-s uboot -e off"
> +    # export verbose="-v"
> +    - ${SRC}/test/py/test.py --role ${ROLE} --build-dir "${OUT}"
> +        --capture=tee-sys -k "not bootstd"|| ret=$?
> +    - U_BOOT_BOARD_IDENTITY="${ROLE}" u-boot-test-release || true
> +    - if [[ $ret -ne 0 ]]; then
> +        exit $ret;
> +      fi
> +  artifacts:
> +    when: always
> +    paths:
> +      - "build/${BOARD}/test-log.html"
> +      - "build/${BOARD}/multiplexed_log.css"
> +    expire_in: 1 week
> +
> +rpi3:
> +  variables:
> +    ROLE: rpi3
> +  <<: *lab_dfn
> +
> +opi_pc:
> +  variables:
> +    ROLE: opi_pc
> +  <<: *lab_dfn
> +
> +pcduino3_nano:
> +  variables:
> +    ROLE: pcduino3_nano
> +  <<: *lab_dfn
> +
> +samus:
> +  variables:
> +    ROLE: samus
> +  <<: *lab_dfn
> +
> +link:
> +  variables:
> +    ROLE: link
> +  <<: *lab_dfn
> +
> +jerry:
> +  variables:
> +    ROLE: jerry
> +  <<: *lab_dfn
> +
> +minnowmax:
> +  variables:
> +    ROLE: minnowmax
> +  <<: *lab_dfn
> +
> +opi_pc2:
> +  variables:
> +    ROLE: opi_pc2
> +  <<: *lab_dfn
> +
> +bpi:
> +  variables:
> +    ROLE: bpi
> +  <<: *lab_dfn
> +
> +rpi2:
> +  variables:
> +    ROLE: rpi2
> +  <<: *lab_dfn
> +
> +bob:
> +  variables:
> +    ROLE: bob
> +  <<: *lab_dfn
> +
> +ff3399:
> +  variables:
> +    ROLE: ff3399
> +  <<: *lab_dfn
> +
> +coral:
> +  variables:
> +    ROLE: coral
> +  <<: *lab_dfn
> +
> +rpi3z:
> +  variables:
> +    ROLE: rpi3z
> +  <<: *lab_dfn
> +
> +bbb:
> +  variables:
> +    ROLE: bbb
> +  <<: *lab_dfn
> +
> +kevin:
> +  variables:
> +    ROLE: kevin
> +  <<: *lab_dfn
> +
> +pine64:
> +  variables:
> +    ROLE: pine64
> +  <<: *lab_dfn
> +
> +c4:
> +  variables:
> +    ROLE: c4
> +  <<: *lab_dfn
> +
> +rpi4:
> +  variables:
> +    ROLE: rpi4
> +  <<: *lab_dfn
> +
> +rpi0:
> +  variables:
> +    ROLE: rpi0
> +  <<: *lab_dfn
> +
> +snow:
> +  variables:
> +    ROLE: snow
> +  <<: *lab_dfn
> +
> +pcduino3:
> +  variables:
> +    ROLE: pcduino3
> +  <<: *lab_dfn
> +
> +nyan-big:
> +  variables:
> +    ROLE: nyan-big
> +  <<: *lab_dfn
> -- 
> 2.34.1
>