[PATCH v1] automation: edit pipeline to prevent running non-selected jobs

[PATCH v1] automation: edit pipeline to prevent running non-selected jobs

[victorm.lira]

From: Victor Lira <victorm.lira@amd.com>

Filtering jobs using the selected jobs regex is missing for
qemu-export/yocto- jobs when running regular pipelines and eclair jobs
when running scheduled pipelines.

Add the missing rules to filter out those jobs, and set a default value
for the selected jobs regex to remove the need to always check if the
variable is empty.

Signed-off-by: Victor Lira <victorm.lira@amd.com>
---
example of the problem:
  - https://gitlab.com/xen-project/people/sstabellini/xen/-/pipelines/2018353899
  - SELECTED_JOBS_ONLY=/alpine-3.18-gcc$/ should produce 1 job only
note:
  - I tested only on sstabellini but the logic should work for hardware/staging
    too
---
Cc: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Cc: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: Anthony PERARD <anthony.perard@vates.tech>
Cc: Doug Goldstein <cardoe@cardoe.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org
---
 .gitlab-ci.yml                    | 1 +
 automation/gitlab-ci/analyze.yaml | 5 +++--
 automation/gitlab-ci/build.yaml   | 9 ++++++---
 3 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 7974ac4e82..64bed300a6 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -2,6 +2,7 @@ variables:
   XEN_REGISTRY: registry.gitlab.com/xen-project/xen
   SELECTED_JOBS_ONLY:
     description: "Regex to select only some jobs, must be enclosed with /. For example /job1|job2/"
+    value: "/.*/"

 workflow:
   name: "$CI_PIPELINE_SCHEDULE_DESCRIPTION"
diff --git a/automation/gitlab-ci/analyze.yaml b/automation/gitlab-ci/analyze.yaml
index d507210067..1f58e13cb2 100644
--- a/automation/gitlab-ci/analyze.yaml
+++ b/automation/gitlab-ci/analyze.yaml
@@ -31,8 +31,7 @@
   rules:
     - if: $CI_PIPELINE_SOURCE == "schedule"
       when: never
-    - if: $SELECTED_JOBS_ONLY && $CI_JOB_NAME =~ $SELECTED_JOBS_ONLY
-    - if: $SELECTED_JOBS_ONLY
+    - if: $CI_JOB_NAME !~ $SELECTED_JOBS_ONLY
       when: never
     - if: $WTOKEN && $CI_PROJECT_PATH =~ /^xen-project\/people\/.*$/
       when: manual
@@ -126,6 +125,8 @@ eclair-ARM64:
   rules:
     - if: $CI_PIPELINE_SOURCE != "schedule"
       when: never
+    - if: $CI_JOB_NAME !~ $SELECTED_JOBS_ONLY
+      when: never
     - !reference [.eclair-analysis, rules]

 eclair-x86_64:on-schedule:
diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index ab5211f77e..b2f96c1fe0 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -226,6 +226,9 @@
       - binaries/
     when: always
   needs: []
+  rules:
+    - if: $CI_JOB_NAME =~ $SELECTED_JOBS_ONLY
+      when: manual

 .yocto-test-arm64:
   extends: .yocto-test
@@ -261,6 +264,9 @@
 .test-jobs-artifact-common:
   stage: build
   needs: []
+  rules:
+    - if: $CI_JOB_NAME =~ $SELECTED_JOBS_ONLY
+      when: on_success

 # Arm test artifacts

@@ -468,20 +474,17 @@ yocto-qemuarm64:
   extends: .yocto-test-arm64
   variables:
     YOCTO_BOARD: qemuarm64
-  when: manual

 yocto-qemuarm:
   extends: .yocto-test-arm64
   variables:
     YOCTO_BOARD: qemuarm
     YOCTO_OUTPUT: --copy-output
-  when: manual

 yocto-qemux86-64:
   extends: .yocto-test-x86-64
   variables:
     YOCTO_BOARD: qemux86-64
-  when: manual

 # Cppcheck analysis jobs

--
2.50.GIT

Re: [PATCH v1] automation: edit pipeline to prevent running non-selected jobs

[Nicola Vetrini]

On 2025-09-03 03:49, victorm.lira@amd.com wrote:
> From: Victor Lira <victorm.lira@amd.com>
> 
> Filtering jobs using the selected jobs regex is missing for
> qemu-export/yocto- jobs when running regular pipelines and eclair jobs
> when running scheduled pipelines.
> 
> Add the missing rules to filter out those jobs, and set a default value
> for the selected jobs regex to remove the need to always check if the
> variable is empty.
> 
> Signed-off-by: Victor Lira <victorm.lira@amd.com>

Reviewed-by: Nicola Vetrini <nicola.vetrini@bugseng.com> # ECLAIR

If this goes in before [1] (which is likely), then I should rebase 
because it will probably conflict

[1] 
https://lore.kernel.org/xen-devel/d4a0924c84e78b3f677b0d987c2f8e4b3f6b80a5.1758226234.git.nicola.vetrini@bugseng.com/T/#u

> ---
> example of the problem:
>   - 
> https://gitlab.com/xen-project/people/sstabellini/xen/-/pipelines/2018353899
>   - SELECTED_JOBS_ONLY=/alpine-3.18-gcc$/ should produce 1 job only
> note:
>   - I tested only on sstabellini but the logic should work for 
> hardware/staging
>     too
> ---
> Cc: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> Cc: Nicola Vetrini <nicola.vetrini@bugseng.com>
> Cc: Anthony PERARD <anthony.perard@vates.tech>
> Cc: Doug Goldstein <cardoe@cardoe.com>
> Cc: Stefano Stabellini <sstabellini@kernel.org>
> Cc: xen-devel@lists.xenproject.org
> ---
>  .gitlab-ci.yml                    | 1 +
>  automation/gitlab-ci/analyze.yaml | 5 +++--
>  automation/gitlab-ci/build.yaml   | 9 ++++++---
>  3 files changed, 10 insertions(+), 5 deletions(-)
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 7974ac4e82..64bed300a6 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -2,6 +2,7 @@ variables:
>    XEN_REGISTRY: registry.gitlab.com/xen-project/xen
>    SELECTED_JOBS_ONLY:
>      description: "Regex to select only some jobs, must be enclosed 
> with /. For example /job1|job2/"
> +    value: "/.*/"
> 
>  workflow:
>    name: "$CI_PIPELINE_SCHEDULE_DESCRIPTION"
> diff --git a/automation/gitlab-ci/analyze.yaml 
> b/automation/gitlab-ci/analyze.yaml
> index d507210067..1f58e13cb2 100644
> --- a/automation/gitlab-ci/analyze.yaml
> +++ b/automation/gitlab-ci/analyze.yaml
> @@ -31,8 +31,7 @@
>    rules:
>      - if: $CI_PIPELINE_SOURCE == "schedule"
>        when: never
> -    - if: $SELECTED_JOBS_ONLY && $CI_JOB_NAME =~ $SELECTED_JOBS_ONLY
> -    - if: $SELECTED_JOBS_ONLY
> +    - if: $CI_JOB_NAME !~ $SELECTED_JOBS_ONLY
>        when: never
>      - if: $WTOKEN && $CI_PROJECT_PATH =~ /^xen-project\/people\/.*$/
>        when: manual
> @@ -126,6 +125,8 @@ eclair-ARM64:
>    rules:
>      - if: $CI_PIPELINE_SOURCE != "schedule"
>        when: never
> +    - if: $CI_JOB_NAME !~ $SELECTED_JOBS_ONLY
> +      when: never
>      - !reference [.eclair-analysis, rules]
> 
>  eclair-x86_64:on-schedule:
> diff --git a/automation/gitlab-ci/build.yaml 
> b/automation/gitlab-ci/build.yaml
> index ab5211f77e..b2f96c1fe0 100644
> --- a/automation/gitlab-ci/build.yaml
> +++ b/automation/gitlab-ci/build.yaml
> @@ -226,6 +226,9 @@
>        - binaries/
>      when: always
>    needs: []
> +  rules:
> +    - if: $CI_JOB_NAME =~ $SELECTED_JOBS_ONLY
> +      when: manual
> 
>  .yocto-test-arm64:
>    extends: .yocto-test
> @@ -261,6 +264,9 @@
>  .test-jobs-artifact-common:
>    stage: build
>    needs: []
> +  rules:
> +    - if: $CI_JOB_NAME =~ $SELECTED_JOBS_ONLY
> +      when: on_success
> 
>  # Arm test artifacts
> 
> @@ -468,20 +474,17 @@ yocto-qemuarm64:
>    extends: .yocto-test-arm64
>    variables:
>      YOCTO_BOARD: qemuarm64
> -  when: manual
> 
>  yocto-qemuarm:
>    extends: .yocto-test-arm64
>    variables:
>      YOCTO_BOARD: qemuarm
>      YOCTO_OUTPUT: --copy-output
> -  when: manual
> 
>  yocto-qemux86-64:
>    extends: .yocto-test-x86-64
>    variables:
>      YOCTO_BOARD: qemux86-64
> -  when: manual
> 
>  # Cppcheck analysis jobs
> 
> --
> 2.50.GIT

-- 
Nicola Vetrini, B.Sc.
Software Engineer
BUGSENG (https://bugseng.com)
LinkedIn: https://www.linkedin.com/in/nicola-vetrini-a42471253