* [PATCH v2 1/2] roms/edk2: Only init brotli submodule to build BaseTools
2021-10-18 10:58 [PATCH v2 0/2] roms/edk2: Avoid cloning unused cmocka submodule Philippe Mathieu-Daudé
@ 2021-10-18 10:58 ` Philippe Mathieu-Daudé
2021-10-18 10:58 ` [PATCH v2 2/2] roms/edk2: Only initialize required submodules Philippe Mathieu-Daudé
` (2 subsequent siblings)
3 siblings, 0 replies; 7+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-10-18 10:58 UTC (permalink / raw)
To: qemu-devel
Cc: Thomas Huth, Michael Roth, Richard Henderson,
Philippe Mathieu-Daudé, Gerd Hoffmann, Paolo Bonzini,
Alex Bennée
Since EDK2 BaseTools only require the brotli submodule,
we don't need to initialize other submodules to build it.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
roms/Makefile | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/roms/Makefile b/roms/Makefile
index eeb5970348c..b967b53bb76 100644
--- a/roms/Makefile
+++ b/roms/Makefile
@@ -143,7 +143,8 @@ build-efi-roms: build-pxe-roms
# efirom
#
edk2-basetools:
- cd edk2/BaseTools && git submodule update --init --force
+ cd edk2/BaseTools && git submodule update --init --force \
+ Source/C/BrotliCompress/brotli
$(MAKE) -C edk2/BaseTools \
PYTHON_COMMAND=$${EDK2_PYTHON_COMMAND:-python3} \
EXTRA_OPTFLAGS='$(EDK2_BASETOOLS_OPTFLAGS)' \
--
2.31.1
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH v2 2/2] roms/edk2: Only initialize required submodules
2021-10-18 10:58 [PATCH v2 0/2] roms/edk2: Avoid cloning unused cmocka submodule Philippe Mathieu-Daudé
2021-10-18 10:58 ` [PATCH v2 1/2] roms/edk2: Only init brotli submodule to build BaseTools Philippe Mathieu-Daudé
@ 2021-10-18 10:58 ` Philippe Mathieu-Daudé
2021-10-18 12:12 ` [PATCH v2 0/2] roms/edk2: Avoid cloning unused cmocka submodule Gerd Hoffmann
2021-10-18 18:10 ` Richard Henderson
3 siblings, 0 replies; 7+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-10-18 10:58 UTC (permalink / raw)
To: qemu-devel
Cc: Thomas Huth, Michael Roth, Richard Henderson,
Philippe Mathieu-Daudé, Gerd Hoffmann, Paolo Bonzini,
Alex Bennée
The EDK2 firmware images built to test QEMU do not require
the following submodules:
- MdeModulePkg/Universal/RegularExpressionDxe/oniguruma
- UnitTestFrameworkPkg/Library/CmockaLib/cmocka
The only submodules required are:
- ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3
- BaseTools/Source/C/BrotliCompress/brotli
- CryptoPkg/Library/OpensslLib/openssl
- MdeModulePkg/Library/BrotliCustomDecompressLib/brotli
Adapt the buildsys machinery to only initialize the required
submodules.
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
Cc: Michael Roth <michael.roth@amd.com>
---
.gitlab-ci.d/edk2.yml | 6 +++++-
roms/Makefile.edk2 | 7 ++++++-
scripts/make-release | 7 ++++++-
3 files changed, 17 insertions(+), 3 deletions(-)
diff --git a/.gitlab-ci.d/edk2.yml b/.gitlab-ci.d/edk2.yml
index 62497ba47fb..13d0f8b019f 100644
--- a/.gitlab-ci.d/edk2.yml
+++ b/.gitlab-ci.d/edk2.yml
@@ -50,7 +50,11 @@ build-edk2:
GIT_DEPTH: 3
script: # Clone the required submodules and build EDK2
- git submodule update --init roms/edk2
- - git -C roms/edk2 submodule update --init
+ - git -C roms/edk2 submodule update --init --
+ ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3
+ BaseTools/Source/C/BrotliCompress/brotli
+ CryptoPkg/Library/OpensslLib/openssl
+ MdeModulePkg/Library/BrotliCustomDecompressLib/brotli
- export JOBS=$(($(getconf _NPROCESSORS_ONLN) + 1))
- echo "=== Using ${JOBS} simultaneous jobs ==="
- make -j${JOBS} -C roms efi 2>&1 1>edk2-stdout.log | tee -a edk2-stderr.log >&2
diff --git a/roms/Makefile.edk2 b/roms/Makefile.edk2
index a8ed3255757..fdae0b511f5 100644
--- a/roms/Makefile.edk2
+++ b/roms/Makefile.edk2
@@ -51,7 +51,12 @@ all: $(foreach flashdev,$(flashdevs),../pc-bios/edk2-$(flashdev).fd.bz2) \
# make-release/tarball scripts.
submodules:
if test -d edk2/.git; then \
- cd edk2 && git submodule update --init --force; \
+ cd edk2 && git submodule update --init --force -- \
+ ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3 \
+ BaseTools/Source/C/BrotliCompress/brotli \
+ CryptoPkg/Library/OpensslLib/openssl \
+ MdeModulePkg/Library/BrotliCustomDecompressLib/brotli \
+ ; \
fi
# See notes on the ".NOTPARALLEL" target and the "+" indicator in
diff --git a/scripts/make-release b/scripts/make-release
index a2a8cda33c0..05b14ecc95e 100755
--- a/scripts/make-release
+++ b/scripts/make-release
@@ -27,7 +27,12 @@ git submodule update --init
# don't necessarily have much control over how a submodule handles its
# submodule dependencies, so we continue to handle these on a case-by-case
# basis for now.
-(cd roms/edk2 && git submodule update --init)
+(cd roms/edk2 && \
+ git submodule update --init -- \
+ ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3 \
+ BaseTools/Source/C/BrotliCompress/brotli \
+ CryptoPkg/Library/OpensslLib/openssl \
+ MdeModulePkg/Library/BrotliCustomDecompressLib/brotli)
popd
tar --exclude=.git -cjf ${destination}.tar.bz2 ${destination}
rm -rf ${destination}
--
2.31.1
^ permalink raw reply related [flat|nested] 7+ messages in thread* Re: [PATCH v2 0/2] roms/edk2: Avoid cloning unused cmocka submodule
2021-10-18 10:58 [PATCH v2 0/2] roms/edk2: Avoid cloning unused cmocka submodule Philippe Mathieu-Daudé
2021-10-18 10:58 ` [PATCH v2 1/2] roms/edk2: Only init brotli submodule to build BaseTools Philippe Mathieu-Daudé
2021-10-18 10:58 ` [PATCH v2 2/2] roms/edk2: Only initialize required submodules Philippe Mathieu-Daudé
@ 2021-10-18 12:12 ` Gerd Hoffmann
2021-10-18 18:10 ` Richard Henderson
3 siblings, 0 replies; 7+ messages in thread
From: Gerd Hoffmann @ 2021-10-18 12:12 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: Thomas Huth, Michael Roth, Richard Henderson, qemu-devel,
Paolo Bonzini, Alex Bennée
On Mon, Oct 18, 2021 at 12:58:14PM +0200, Philippe Mathieu-Daudé wrote:
> cmocka website SSL certificate expired, making CI pipelines
> fail [*]. However EDK2 images built to test QEMU don't need
> cmocka, nor oniguruma. Avoid cloning them.
>
> Note: scripts/make-release is neither covered in MAINTAINERS
> nor in our CI.
>
> [*] https://gitlab.com/rth7680/qemu/-/jobs/1685387520
> fatal: unable to access 'https://git.cryptomilk.org/projects/cmocka.git/': server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
> fatal: clone of 'https://git.cryptomilk.org/projects/cmocka.git' into submodule path 'UnitTestFrameworkPkg/Library/CmockaLib/cmocka' failed
>
> Since v1:
> - Fixed typo (thuth)
> - Added missing '--' shell separator
Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v2 0/2] roms/edk2: Avoid cloning unused cmocka submodule
2021-10-18 10:58 [PATCH v2 0/2] roms/edk2: Avoid cloning unused cmocka submodule Philippe Mathieu-Daudé
` (2 preceding siblings ...)
2021-10-18 12:12 ` [PATCH v2 0/2] roms/edk2: Avoid cloning unused cmocka submodule Gerd Hoffmann
@ 2021-10-18 18:10 ` Richard Henderson
2021-10-18 18:53 ` Philippe Mathieu-Daudé
3 siblings, 1 reply; 7+ messages in thread
From: Richard Henderson @ 2021-10-18 18:10 UTC (permalink / raw)
To: Philippe Mathieu-Daudé, qemu-devel
Cc: Paolo Bonzini, Thomas Huth, Alex Bennée, Gerd Hoffmann,
Michael Roth
On 10/18/21 3:58 AM, Philippe Mathieu-Daudé wrote:
> cmocka website SSL certificate expired, making CI pipelines
> fail [*]. However EDK2 images built to test QEMU don't need
> cmocka, nor oniguruma. Avoid cloning them.
>
> Note: scripts/make-release is neither covered in MAINTAINERS
> nor in our CI.
>
> [*] https://gitlab.com/rth7680/qemu/-/jobs/1685387520
> fatal: unable to access 'https://git.cryptomilk.org/projects/cmocka.git/': server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
> fatal: clone of 'https://git.cryptomilk.org/projects/cmocka.git' into submodule path 'UnitTestFrameworkPkg/Library/CmockaLib/cmocka' failed
>
> Since v1:
> - Fixed typo (thuth)
> - Added missing '--' shell separator
>
> Philippe Mathieu-Daudé (2):
> roms/edk2: Only init brotli submodule to build BaseTools
> roms/edk2: Only initialize required submodules
Thanks a lot.
Tested-by: Richard Henderson <richard.henderson@linaro.org>
I'll queue this to target-arm.next.
r~
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH v2 0/2] roms/edk2: Avoid cloning unused cmocka submodule
2021-10-18 18:10 ` Richard Henderson
@ 2021-10-18 18:53 ` Philippe Mathieu-Daudé
2021-10-18 18:55 ` Richard Henderson
0 siblings, 1 reply; 7+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-10-18 18:53 UTC (permalink / raw)
To: Richard Henderson, qemu-devel
Cc: Paolo Bonzini, Thomas Huth, Alex Bennée, Gerd Hoffmann,
Michael Roth
On 10/18/21 20:10, Richard Henderson wrote:
> On 10/18/21 3:58 AM, Philippe Mathieu-Daudé wrote:
>> cmocka website SSL certificate expired, making CI pipelines
>> fail [*]. However EDK2 images built to test QEMU don't need
>> cmocka, nor oniguruma. Avoid cloning them.
>>
>> Note: scripts/make-release is neither covered in MAINTAINERS
>> nor in our CI.
>>
>> [*] https://gitlab.com/rth7680/qemu/-/jobs/1685387520
>> fatal: unable to access
>> 'https://git.cryptomilk.org/projects/cmocka.git/': server certificate
>> verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt
>> CRLfile: none
>> fatal: clone of 'https://git.cryptomilk.org/projects/cmocka.git' into
>> submodule path 'UnitTestFrameworkPkg/Library/CmockaLib/cmocka' failed
>>
>> Since v1:
>> - Fixed typo (thuth)
>> - Added missing '--' shell separator
>>
>> Philippe Mathieu-Daudé (2):
>> roms/edk2: Only init brotli submodule to build BaseTools
>> roms/edk2: Only initialize required submodules
>
> Thanks a lot.
>
> Tested-by: Richard Henderson <richard.henderson@linaro.org>
>
> I'll queue this to target-arm.next.
If you don't have anything in your queue I can send a pullreq
tomorrow, otherwise thanks for taking care of it.
^ permalink raw reply [flat|nested] 7+ messages in thread