* Re: [PATCH-for-6.1] gitlab-ci: Extract EDK2 job rules to reusable section
2021-07-14 10:10 [PATCH-for-6.1] gitlab-ci: Extract EDK2 job rules to reusable section Philippe Mathieu-Daudé
@ 2021-07-14 12:24 ` Daniel P. Berrangé
2021-07-14 17:02 ` Willian Rampazzo
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Daniel P. Berrangé @ 2021-07-14 12:24 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: Willian Rampazzo, Thomas Huth, Alex Bennée, qemu-devel,
Wainer dos Santos Moschetta
On Wed, Jul 14, 2021 at 12:10:03PM +0200, Philippe Mathieu-Daudé wrote:
> All jobs depending on 'docker-edk2' job must use at most all
> the rules that triggers it. The simplest way to ensure that
> is to always use the same rules. Extract all the rules to a
> reusable section, and include this section (with the 'extends'
> keyword) in both 'docker-edk2' and 'build-edk2' jobs.
>
> The problem was introduced in commit 71920809cea ("gitlab-ci.yml:
> Add jobs to build EDK2 firmware binaries"), but was revealed in
> commit 1925468ddbf ("docker: EDK2 build job depends on EDK2
> container") and eventually failed on CI:
> https://gitlab.com/qemu-project/qemu/-/pipelines/335995843
>
> Reported-by: Daniel P. Berrangé <berrange@redhat.com>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
> If this is a correct fix, I'll send a similar fix for the
> OpenSBI jobs.
> ---
> .gitlab-ci.d/edk2.yml | 27 ++++++++++++++++-----------
> 1 file changed, 16 insertions(+), 11 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH-for-6.1] gitlab-ci: Extract EDK2 job rules to reusable section
2021-07-14 10:10 [PATCH-for-6.1] gitlab-ci: Extract EDK2 job rules to reusable section Philippe Mathieu-Daudé
2021-07-14 12:24 ` Daniel P. Berrangé
@ 2021-07-14 17:02 ` Willian Rampazzo
2021-07-14 18:14 ` Cleber Rosa
2021-07-14 20:33 ` Philippe Mathieu-Daudé
3 siblings, 0 replies; 6+ messages in thread
From: Willian Rampazzo @ 2021-07-14 17:02 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: Daniel P . Berrangé, Thomas Huth, Alex Bennée,
qemu-devel, Wainer dos Santos Moschetta
On Wed, Jul 14, 2021 at 7:10 AM Philippe Mathieu-Daudé
<philmd@redhat.com> wrote:
>
> All jobs depending on 'docker-edk2' job must use at most all
> the rules that triggers it. The simplest way to ensure that
> is to always use the same rules. Extract all the rules to a
> reusable section, and include this section (with the 'extends'
> keyword) in both 'docker-edk2' and 'build-edk2' jobs.
>
> The problem was introduced in commit 71920809cea ("gitlab-ci.yml:
> Add jobs to build EDK2 firmware binaries"), but was revealed in
> commit 1925468ddbf ("docker: EDK2 build job depends on EDK2
> container") and eventually failed on CI:
> https://gitlab.com/qemu-project/qemu/-/pipelines/335995843
>
> Reported-by: Daniel P. Berrangé <berrange@redhat.com>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
> If this is a correct fix, I'll send a similar fix for the
> OpenSBI jobs.
> ---
> .gitlab-ci.d/edk2.yml | 27 ++++++++++++++++-----------
> 1 file changed, 16 insertions(+), 11 deletions(-)
>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH-for-6.1] gitlab-ci: Extract EDK2 job rules to reusable section
2021-07-14 10:10 [PATCH-for-6.1] gitlab-ci: Extract EDK2 job rules to reusable section Philippe Mathieu-Daudé
2021-07-14 12:24 ` Daniel P. Berrangé
2021-07-14 17:02 ` Willian Rampazzo
@ 2021-07-14 18:14 ` Cleber Rosa
2021-07-14 20:24 ` Philippe Mathieu-Daudé
2021-07-14 20:33 ` Philippe Mathieu-Daudé
3 siblings, 1 reply; 6+ messages in thread
From: Cleber Rosa @ 2021-07-14 18:14 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: Thomas Huth, Daniel P . Berrangé, qemu-devel,
Wainer dos Santos Moschetta, Willian Rampazzo, Alex Bennée
Philippe Mathieu-Daudé writes:
> All jobs depending on 'docker-edk2' job must use at most all
> the rules that triggers it. The simplest way to ensure that
> is to always use the same rules. Extract all the rules to a
> reusable section, and include this section (with the 'extends'
> keyword) in both 'docker-edk2' and 'build-edk2' jobs.
>
> The problem was introduced in commit 71920809cea ("gitlab-ci.yml:
> Add jobs to build EDK2 firmware binaries"), but was revealed in
> commit 1925468ddbf ("docker: EDK2 build job depends on EDK2
> container") and eventually failed on CI:
> https://gitlab.com/qemu-project/qemu/-/pipelines/335995843
>
> Reported-by: Daniel P. Berrangé <berrange@redhat.com>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
> If this is a correct fix, I'll send a similar fix for the
> OpenSBI jobs.
> ---
> .gitlab-ci.d/edk2.yml | 27 ++++++++++++++++-----------
> 1 file changed, 16 insertions(+), 11 deletions(-)
>
> diff --git a/.gitlab-ci.d/edk2.yml b/.gitlab-ci.d/edk2.yml
> index ba7280605c4..aae2f7ad880 100644
> --- a/.gitlab-ci.d/edk2.yml
> +++ b/.gitlab-ci.d/edk2.yml
> @@ -1,10 +1,22 @@
> -docker-edk2:
> - stage: containers
> - rules: # Only run this job when the Dockerfile is modified
> +# All jobs needing docker-edk2 must use the same rules it uses.
> +.edk2_job_rules:
> + rules: # Only run this job when ...
> - changes:
> + # this file is modified
> - .gitlab-ci.d/edk2.yml
> + # or the Dockerfile is modified
> - .gitlab-ci.d/edk2/Dockerfile
> + # or roms/edk2/ is modified (submodule updated)
> + - roms/edk2/*
> when: always
> + - if: '$CI_COMMIT_REF_NAME =~ /^edk2/' # or the branch/tag starts with 'edk2'
> + when: always
> + - if: '$CI_COMMIT_MESSAGE =~ /edk2/i' # or last commit description contains 'EDK2'
> + when: always
> +
> +docker-edk2:
> + extends: .edk2_job_rules
> + stage: containers
> image: docker:19.03.1
> services:
> - docker:19.03.1-dind
> @@ -24,16 +36,9 @@ docker-edk2:
> - docker push $IMAGE_TAG
>
> build-edk2:
> + extends: .edk2_job_rules
> stage: build
> needs: ['docker-edk2']
> - rules: # Only run this job when ...
> - - changes: # ... roms/edk2/ is modified (submodule updated)
> - - roms/edk2/*
> - when: always
> - - if: '$CI_COMMIT_REF_NAME =~ /^edk2/' # or the branch/tag starts with 'edk2'
> - when: always
> - - if: '$CI_COMMIT_MESSAGE =~ /edk2/i' # or last commit description contains 'EDK2'
> - when: always
> artifacts:
> paths: # 'artifacts.zip' will contains the following files:
> - pc-bios/edk2*bz2
Hi Phil,
This looks good, but have you triggered a pipeline with these changes?
It'd be helpful for an extra level of assurance for the intended results.
Thanks!
--
Cleber Rosa
[ Sr Software Engineer - Virtualization Team - Red Hat ]
[ Avocado Test Framework - avocado-framework.github.io ]
[ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ]
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH-for-6.1] gitlab-ci: Extract EDK2 job rules to reusable section
2021-07-14 18:14 ` Cleber Rosa
@ 2021-07-14 20:24 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 6+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-07-14 20:24 UTC (permalink / raw)
To: Cleber Rosa
Cc: Thomas Huth, Daniel P . Berrangé, qemu-devel,
Wainer dos Santos Moschetta, Willian Rampazzo, Alex Bennée
On 7/14/21 8:14 PM, Cleber Rosa wrote:
>
> Philippe Mathieu-Daudé writes:
>
>> All jobs depending on 'docker-edk2' job must use at most all
>> the rules that triggers it. The simplest way to ensure that
>> is to always use the same rules. Extract all the rules to a
>> reusable section, and include this section (with the 'extends'
>> keyword) in both 'docker-edk2' and 'build-edk2' jobs.
>>
>> The problem was introduced in commit 71920809cea ("gitlab-ci.yml:
>> Add jobs to build EDK2 firmware binaries"), but was revealed in
>> commit 1925468ddbf ("docker: EDK2 build job depends on EDK2
>> container") and eventually failed on CI:
>> https://gitlab.com/qemu-project/qemu/-/pipelines/335995843
>>
>> Reported-by: Daniel P. Berrangé <berrange@redhat.com>
>> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
>> ---
>> If this is a correct fix, I'll send a similar fix for the
>> OpenSBI jobs.
>> ---
>> .gitlab-ci.d/edk2.yml | 27 ++++++++++++++++-----------
>> 1 file changed, 16 insertions(+), 11 deletions(-)
>>
>> diff --git a/.gitlab-ci.d/edk2.yml b/.gitlab-ci.d/edk2.yml
>> index ba7280605c4..aae2f7ad880 100644
>> --- a/.gitlab-ci.d/edk2.yml
>> +++ b/.gitlab-ci.d/edk2.yml
>> @@ -1,10 +1,22 @@
>> -docker-edk2:
>> - stage: containers
>> - rules: # Only run this job when the Dockerfile is modified
>> +# All jobs needing docker-edk2 must use the same rules it uses.
>> +.edk2_job_rules:
>> + rules: # Only run this job when ...
>> - changes:
>> + # this file is modified
>> - .gitlab-ci.d/edk2.yml
>> + # or the Dockerfile is modified
>> - .gitlab-ci.d/edk2/Dockerfile
>> + # or roms/edk2/ is modified (submodule updated)
>> + - roms/edk2/*
>> when: always
>> + - if: '$CI_COMMIT_REF_NAME =~ /^edk2/' # or the branch/tag starts with 'edk2'
>> + when: always
>> + - if: '$CI_COMMIT_MESSAGE =~ /edk2/i' # or last commit description contains 'EDK2'
>> + when: always
>> +
>> +docker-edk2:
>> + extends: .edk2_job_rules
>> + stage: containers
>> image: docker:19.03.1
>> services:
>> - docker:19.03.1-dind
>> @@ -24,16 +36,9 @@ docker-edk2:
>> - docker push $IMAGE_TAG
>>
>> build-edk2:
>> + extends: .edk2_job_rules
>> stage: build
>> needs: ['docker-edk2']
>> - rules: # Only run this job when ...
>> - - changes: # ... roms/edk2/ is modified (submodule updated)
>> - - roms/edk2/*
>> - when: always
>> - - if: '$CI_COMMIT_REF_NAME =~ /^edk2/' # or the branch/tag starts with 'edk2'
>> - when: always
>> - - if: '$CI_COMMIT_MESSAGE =~ /edk2/i' # or last commit description contains 'EDK2'
>> - when: always
>> artifacts:
>> paths: # 'artifacts.zip' will contains the following files:
>> - pc-bios/edk2*bz2
>
>
> Hi Phil,
>
> This looks good, but have you triggered a pipeline with these changes?
> It'd be helpful for an extra level of assurance for the intended results.
Yes, before sending this patch ;)
https://gitlab.com/philmd_rh/qemu/-/pipelines/336800660
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH-for-6.1] gitlab-ci: Extract EDK2 job rules to reusable section
2021-07-14 10:10 [PATCH-for-6.1] gitlab-ci: Extract EDK2 job rules to reusable section Philippe Mathieu-Daudé
` (2 preceding siblings ...)
2021-07-14 18:14 ` Cleber Rosa
@ 2021-07-14 20:33 ` Philippe Mathieu-Daudé
3 siblings, 0 replies; 6+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-07-14 20:33 UTC (permalink / raw)
To: qemu-devel
Cc: Willian Rampazzo, Daniel P . Berrangé, Thomas Huth,
Alex Bennée, Wainer dos Santos Moschetta
On 7/14/21 12:10 PM, Philippe Mathieu-Daudé wrote:
> All jobs depending on 'docker-edk2' job must use at most all
> the rules that triggers it. The simplest way to ensure that
> is to always use the same rules. Extract all the rules to a
> reusable section, and include this section (with the 'extends'
> keyword) in both 'docker-edk2' and 'build-edk2' jobs.
>
> The problem was introduced in commit 71920809cea ("gitlab-ci.yml:
> Add jobs to build EDK2 firmware binaries"), but was revealed in
> commit 1925468ddbf ("docker: EDK2 build job depends on EDK2
> container") and eventually failed on CI:
> https://gitlab.com/qemu-project/qemu/-/pipelines/335995843
>
> Reported-by: Daniel P. Berrangé <berrange@redhat.com>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
> If this is a correct fix, I'll send a similar fix for the
> OpenSBI jobs.
> ---
> .gitlab-ci.d/edk2.yml | 27 ++++++++++++++++-----------
> 1 file changed, 16 insertions(+), 11 deletions(-)
Queued to fw-edk2 tree.
^ permalink raw reply [flat|nested] 6+ messages in thread