* [Buildroot] [PATCH RFC 1/1] boot/edk2: unset MAKEFLAGS in build environment
@ 2023-12-31 18:14 Julien Olivain
2024-01-01 21:27 ` Yann E. MORIN
2024-01-10 20:09 ` Peter Korsgaard
0 siblings, 2 replies; 3+ messages in thread
From: Julien Olivain @ 2023-12-31 18:14 UTC (permalink / raw)
To: buildroot; +Cc: Julien Olivain, Yann E . MORIN
Yann reported in [1] that edk2 build could sometimes fail. The issue
can be reproduced when per-package directories is enabled, or also
when building on a system with GNU Make >= 4.4 using the
"--shuffle=reverse" option (such as Fedora 39). Those are pointing
toward a Makefile dependency issue.
The issue can be reproduced with commands:
cat > .config <<EOF
BR2_riscv=y
BR2_RISCV_64=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_TARGET_EDK2=y
EOF
make olddefconfig
Then, building either with:
make --shuffle=reverse
Or:
utils/config -e BR2_PER_PACKAGE_DIRECTORIES
make olddefconfig
make -j$(nproc)
It is interesting to mention that when using "make --shuffle=reverse"
to build, the build can be completed if restarted only with "make". It
will not pull any other Buildroot package. This fact hints toward a
Makefile dependency issue internal to the EDK2 build system, rather
than in the Buildroot recipe.
The EDK2 build system is quite unique. See [2]. It generates files,
makefiles and internally uses GNU Make to compile code. This system is
likely not tested as being a sub-Make process in a complex Makefile
such as Buildroot.
In order to prevent Buildroot to pass unexpected Make flags to the
EDK2 sub-Make, this commit unset the MAKEFLAGS variable in the EDK2
build environment. This will put the EDK2 build script in a more
common and tested state. See GNU Make documentation about recursive use
of Make, more specifically [3].
Note: as mentioned, the build failure is likely due to an internal
issue of the EDK2 build system. The failure points to a missing
dependency in the EDK2 generator itself. This commit does not fix this
issue, but rather put the EDK2 build system in a normalized
environment, avoiding Buildroot flags being passed to the internal
EDK2 sub-Make invocation. The upstream EDK2 build system most likely
need a fix too.
Fixes:
make[2]: *** No rule to make target '/buildroot/output/build/edk2-edk2-stable202308/Build/RiscVVirtQemu/RELEASE_GCC5/RISCV64/MdeModulePkg/Application/UiApp/UiApp/DEBUG/UiApp.efi', needed by '/buildroot/output/build/edk2-edk2-stable202308/Build/RiscVVirtQemu/RELEASE_GCC5/FV/Ffs/462CAA21-7614-4503-836E-8AB6F4662331UiApp/UiApp.offset'. Stop.
build.py...
: error 7000: Failed to execute command
make tbuild [/buildroot/output/build/edk2-edk2-stable202308/Build/RiscVVirtQemu/RELEASE_GCC5/RISCV64/MdeModulePkg/Application/UiApp/UiApp]
build.py...
: error F002: Failed to build module
/buildroot/output/build/edk2-edk2-stable202308/MdeModulePkg/Application/UiApp/UiApp.inf [RISCV64, GCC5, RELEASE]
[1] https://lists.buildroot.org/pipermail/buildroot/2023-December/681507.html
[2] https://tianocore-docs.github.io/edk2-BuildSpecification/draft/4_edk_ii_build_process_overview/42_build_process_overview.html
[3] https://www.gnu.org/software/make/manual/make.html#Options_002fRecursion
Reported-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
---
boot/edk2/edk2.mk | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/boot/edk2/edk2.mk b/boot/edk2/edk2.mk
index efdb4d02ce..87ac80c88f 100644
--- a/boot/edk2/edk2.mk
+++ b/boot/edk2/edk2.mk
@@ -150,7 +150,13 @@ EDK2_BASETOOLS_OPTS = \
EDK2_PACKAGES_PATH = $(subst $(space),:,$(strip $(EDK2_PACKAGES_PATHS)))
+# EDK2 "build" script internally uses and calls "make", which controls
+# its own flags. It is mainly tested while not being a sub-make. In
+# order to stay in that configuration, we avoid leaking top-level
+# Buildroot make flags into EDK2 build by clearing the MAKEFLAGS
+# environment variable.
EDK2_BUILD_ENV += \
+ MAKEFLAGS= \
WORKSPACE=$(@D) \
PACKAGES_PATH=$(EDK2_PACKAGES_PATH) \
PYTHON_COMMAND=$(HOST_DIR)/bin/python3 \
--
2.43.0
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply related [flat|nested] 3+ messages in thread* Re: [Buildroot] [PATCH RFC 1/1] boot/edk2: unset MAKEFLAGS in build environment
2023-12-31 18:14 [Buildroot] [PATCH RFC 1/1] boot/edk2: unset MAKEFLAGS in build environment Julien Olivain
@ 2024-01-01 21:27 ` Yann E. MORIN
2024-01-10 20:09 ` Peter Korsgaard
1 sibling, 0 replies; 3+ messages in thread
From: Yann E. MORIN @ 2024-01-01 21:27 UTC (permalink / raw)
To: Julien Olivain; +Cc: buildroot
Julien, All,
On 2023-12-31 19:14 +0100, Julien Olivain spake thusly:
> Yann reported in [1] that edk2 build could sometimes fail. The issue
> can be reproduced when per-package directories is enabled, or also
> when building on a system with GNU Make >= 4.4 using the
> "--shuffle=reverse" option (such as Fedora 39). Those are pointing
> toward a Makefile dependency issue.
>
> The issue can be reproduced with commands:
>
> cat > .config <<EOF
> BR2_riscv=y
> BR2_RISCV_64=y
> BR2_TOOLCHAIN_EXTERNAL=y
> BR2_TARGET_EDK2=y
> EOF
> make olddefconfig
>
> Then, building either with:
>
> make --shuffle=reverse
>
> Or:
>
> utils/config -e BR2_PER_PACKAGE_DIRECTORIES
> make olddefconfig
> make -j$(nproc)
>
> It is interesting to mention that when using "make --shuffle=reverse"
> to build, the build can be completed if restarted only with "make". It
> will not pull any other Buildroot package. This fact hints toward a
> Makefile dependency issue internal to the EDK2 build system, rather
> than in the Buildroot recipe.
>
> The EDK2 build system is quite unique. See [2]. It generates files,
> makefiles and internally uses GNU Make to compile code. This system is
> likely not tested as being a sub-Make process in a complex Makefile
> such as Buildroot.
>
> In order to prevent Buildroot to pass unexpected Make flags to the
> EDK2 sub-Make, this commit unset the MAKEFLAGS variable in the EDK2
> build environment. This will put the EDK2 build script in a more
> common and tested state. See GNU Make documentation about recursive use
> of Make, more specifically [3].
>
> Note: as mentioned, the build failure is likely due to an internal
> issue of the EDK2 build system. The failure points to a missing
> dependency in the EDK2 generator itself. This commit does not fix this
> issue, but rather put the EDK2 build system in a normalized
> environment, avoiding Buildroot flags being passed to the internal
> EDK2 sub-Make invocation. The upstream EDK2 build system most likely
> need a fix too.
>
> Fixes:
>
> make[2]: *** No rule to make target '/buildroot/output/build/edk2-edk2-stable202308/Build/RiscVVirtQemu/RELEASE_GCC5/RISCV64/MdeModulePkg/Application/UiApp/UiApp/DEBUG/UiApp.efi', needed by '/buildroot/output/build/edk2-edk2-stable202308/Build/RiscVVirtQemu/RELEASE_GCC5/FV/Ffs/462CAA21-7614-4503-836E-8AB6F4662331UiApp/UiApp.offset'. Stop.
>
> build.py...
> : error 7000: Failed to execute command
> make tbuild [/buildroot/output/build/edk2-edk2-stable202308/Build/RiscVVirtQemu/RELEASE_GCC5/RISCV64/MdeModulePkg/Application/UiApp/UiApp]
>
> build.py...
> : error F002: Failed to build module
> /buildroot/output/build/edk2-edk2-stable202308/MdeModulePkg/Application/UiApp/UiApp.inf [RISCV64, GCC5, RELEASE]
>
> [1] https://lists.buildroot.org/pipermail/buildroot/2023-December/681507.html
> [2] https://tianocore-docs.github.io/edk2-BuildSpecification/draft/4_edk_ii_build_process_overview/42_build_process_overview.html
> [3] https://www.gnu.org/software/make/manual/make.html#Options_002fRecursion
>
> Reported-by: Yann E. MORIN <yann.morin.1998@free.fr>
> Signed-off-by: Julien Olivain <ju.o@free.fr>
Thank you very much for this very detailed explanations! I see you
marked is as RFC, but I believe it is good to go as-is: the workaround
is perfectly sensible.
Of course, if upstream were to fix that issue, it would be better, but
given the complexity of it, we are probably not in a position to
properly fix it in a timely manner.
Applied to master, thanks.
Regards,
Yann E. MORIN.
> ---
> boot/edk2/edk2.mk | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/boot/edk2/edk2.mk b/boot/edk2/edk2.mk
> index efdb4d02ce..87ac80c88f 100644
> --- a/boot/edk2/edk2.mk
> +++ b/boot/edk2/edk2.mk
> @@ -150,7 +150,13 @@ EDK2_BASETOOLS_OPTS = \
>
> EDK2_PACKAGES_PATH = $(subst $(space),:,$(strip $(EDK2_PACKAGES_PATHS)))
>
> +# EDK2 "build" script internally uses and calls "make", which controls
> +# its own flags. It is mainly tested while not being a sub-make. In
> +# order to stay in that configuration, we avoid leaking top-level
> +# Buildroot make flags into EDK2 build by clearing the MAKEFLAGS
> +# environment variable.
> EDK2_BUILD_ENV += \
> + MAKEFLAGS= \
> WORKSPACE=$(@D) \
> PACKAGES_PATH=$(EDK2_PACKAGES_PATH) \
> PYTHON_COMMAND=$(HOST_DIR)/bin/python3 \
> --
> 2.43.0
>
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [Buildroot] [PATCH RFC 1/1] boot/edk2: unset MAKEFLAGS in build environment
2023-12-31 18:14 [Buildroot] [PATCH RFC 1/1] boot/edk2: unset MAKEFLAGS in build environment Julien Olivain
2024-01-01 21:27 ` Yann E. MORIN
@ 2024-01-10 20:09 ` Peter Korsgaard
1 sibling, 0 replies; 3+ messages in thread
From: Peter Korsgaard @ 2024-01-10 20:09 UTC (permalink / raw)
To: Julien Olivain; +Cc: Yann E . MORIN, buildroot
>>>>> "Julien" == Julien Olivain <ju.o@free.fr> writes:
> Yann reported in [1] that edk2 build could sometimes fail. The issue
> can be reproduced when per-package directories is enabled, or also
> when building on a system with GNU Make >= 4.4 using the
> "--shuffle=reverse" option (such as Fedora 39). Those are pointing
> toward a Makefile dependency issue.
> The issue can be reproduced with commands:
> cat > .config <<EOF
> BR2_riscv=y
> BR2_RISCV_64=y
> BR2_TOOLCHAIN_EXTERNAL=y
> BR2_TARGET_EDK2=y
> EOF
> make olddefconfig
> Then, building either with:
> make --shuffle=reverse
> Or:
> utils/config -e BR2_PER_PACKAGE_DIRECTORIES
> make olddefconfig
> make -j$(nproc)
> It is interesting to mention that when using "make --shuffle=reverse"
> to build, the build can be completed if restarted only with "make". It
> will not pull any other Buildroot package. This fact hints toward a
> Makefile dependency issue internal to the EDK2 build system, rather
> than in the Buildroot recipe.
> The EDK2 build system is quite unique. See [2]. It generates files,
> makefiles and internally uses GNU Make to compile code. This system is
> likely not tested as being a sub-Make process in a complex Makefile
> such as Buildroot.
> In order to prevent Buildroot to pass unexpected Make flags to the
> EDK2 sub-Make, this commit unset the MAKEFLAGS variable in the EDK2
> build environment. This will put the EDK2 build script in a more
> common and tested state. See GNU Make documentation about recursive use
> of Make, more specifically [3].
> Note: as mentioned, the build failure is likely due to an internal
> issue of the EDK2 build system. The failure points to a missing
> dependency in the EDK2 generator itself. This commit does not fix this
> issue, but rather put the EDK2 build system in a normalized
> environment, avoiding Buildroot flags being passed to the internal
> EDK2 sub-Make invocation. The upstream EDK2 build system most likely
> need a fix too.
> Fixes:
> make[2]: *** No rule to make target '/buildroot/output/build/edk2-edk2-stable202308/Build/RiscVVirtQemu/RELEASE_GCC5/RISCV64/MdeModulePkg/Application/UiApp/UiApp/DEBUG/UiApp.efi', needed by '/buildroot/output/build/edk2-edk2-stable202308/Build/RiscVVirtQemu/RELEASE_GCC5/FV/Ffs/462CAA21-7614-4503-836E-8AB6F4662331UiApp/UiApp.offset'. Stop.
> build.py...
> : error 7000: Failed to execute command
> make tbuild [/buildroot/output/build/edk2-edk2-stable202308/Build/RiscVVirtQemu/RELEASE_GCC5/RISCV64/MdeModulePkg/Application/UiApp/UiApp]
> build.py...
> : error F002: Failed to build module
> /buildroot/output/build/edk2-edk2-stable202308/MdeModulePkg/Application/UiApp/UiApp.inf [RISCV64, GCC5, RELEASE]
> [1] https://lists.buildroot.org/pipermail/buildroot/2023-December/681507.html
> [2] https://tianocore-docs.github.io/edk2-BuildSpecification/draft/4_edk_ii_build_process_overview/42_build_process_overview.html
> [3] https://www.gnu.org/software/make/manual/make.html#Options_002fRecursion
> Reported-by: Yann E. MORIN <yann.morin.1998@free.fr>
> Signed-off-by: Julien Olivain <ju.o@free.fr>
Nice description!
Committed to 2023.02.x and 2023.11.x, thanks.
--
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2024-01-10 20:09 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-12-31 18:14 [Buildroot] [PATCH RFC 1/1] boot/edk2: unset MAKEFLAGS in build environment Julien Olivain
2024-01-01 21:27 ` Yann E. MORIN
2024-01-10 20:09 ` Peter Korsgaard
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox