* [PATCH v2] riscv: add dependency among Image(.gz), loader(.bin), and vmlinuz.efi
@ 2023-11-19 10:00 Masahiro Yamada
2023-11-21 16:36 ` Ard Biesheuvel
` (2 more replies)
0 siblings, 3 replies; 4+ messages in thread
From: Masahiro Yamada @ 2023-11-19 10:00 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Albert Ou, linux-riscv
Cc: linux-kernel, linux-kbuild, Ard Biesheuvel, Simon Glass,
Masahiro Yamada
A common issue in Makefile is a race in parallel building.
You need to be careful to prevent multiple threads from writing to the
same file simultaneously.
Commit 3939f3345050 ("ARM: 8418/1: add boot image dependencies to not
generate invalid images") addressed such a bad scenario.
A similar symptom occurs with the following command:
$ make -j$(nproc) ARCH=riscv Image Image.gz loader loader.bin vmlinuz.efi
[ snip ]
SORTTAB vmlinux
OBJCOPY arch/riscv/boot/Image
OBJCOPY arch/riscv/boot/Image
OBJCOPY arch/riscv/boot/Image
OBJCOPY arch/riscv/boot/Image
OBJCOPY arch/riscv/boot/Image
GZIP arch/riscv/boot/Image.gz
AS arch/riscv/boot/loader.o
AS arch/riscv/boot/loader.o
Kernel: arch/riscv/boot/Image is ready
PAD arch/riscv/boot/vmlinux.bin
GZIP arch/riscv/boot/vmlinuz
Kernel: arch/riscv/boot/loader is ready
OBJCOPY arch/riscv/boot/loader.bin
Kernel: arch/riscv/boot/loader.bin is ready
Kernel: arch/riscv/boot/Image.gz is ready
OBJCOPY arch/riscv/boot/vmlinuz.o
LD arch/riscv/boot/vmlinuz.efi.elf
OBJCOPY arch/riscv/boot/vmlinuz.efi
Kernel: arch/riscv/boot/vmlinuz.efi is ready
The log "OBJCOPY arch/riscv/boot/Image" is displayed 5 times.
(also "AS arch/riscv/boot/loader.o" twice.)
It indicates that 5 threads simultaneously enter arch/riscv/boot/
and write to arch/riscv/boot/Image.
It occasionally leads to a build failure:
$ make -j$(nproc) ARCH=riscv Image Image.gz loader loader.bin vmlinuz.efi
[ snip ]
SORTTAB vmlinux
OBJCOPY arch/riscv/boot/Image
OBJCOPY arch/riscv/boot/Image
OBJCOPY arch/riscv/boot/Image
OBJCOPY arch/riscv/boot/Image
PAD arch/riscv/boot/vmlinux.bin
truncate: Invalid number: 'arch/riscv/boot/vmlinux.bin'
make[2]: *** [drivers/firmware/efi/libstub/Makefile.zboot:13: arch/riscv/boot/vmlinux.bin] Error 1
make[2]: *** Deleting file 'arch/riscv/boot/vmlinux.bin'
make[1]: *** [arch/riscv/Makefile:167: vmlinuz.efi] Error 2
make[1]: *** Waiting for unfinished jobs....
Kernel: arch/riscv/boot/Image is ready
GZIP arch/riscv/boot/Image.gz
AS arch/riscv/boot/loader.o
AS arch/riscv/boot/loader.o
Kernel: arch/riscv/boot/loader is ready
OBJCOPY arch/riscv/boot/loader.bin
Kernel: arch/riscv/boot/loader.bin is ready
Kernel: arch/riscv/boot/Image.gz is ready
make: *** [Makefile:234: __sub-make] Error 2
Image.gz, loader, vmlinuz.efi depend on Image. loader.bin depends
on loader. Such dependencies are not specified in arch/riscv/Makefile.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---
Changes in v2:
- Fix commit log
arch/riscv/Makefile | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile
index 5cbe596345c1..1d6ed27e0a2a 100644
--- a/arch/riscv/Makefile
+++ b/arch/riscv/Makefile
@@ -163,6 +163,8 @@ BOOT_TARGETS := Image Image.gz loader loader.bin xipImage vmlinuz.efi
all: $(notdir $(KBUILD_IMAGE))
+loader.bin: loader
+Image.gz loader vmlinuz.efi: Image
$(BOOT_TARGETS): vmlinux
$(Q)$(MAKE) $(build)=$(boot) $(boot)/$@
@$(kecho) ' Kernel: $(boot)/$@ is ready'
--
2.40.1
^ permalink raw reply related [flat|nested] 4+ messages in thread* Re: [PATCH v2] riscv: add dependency among Image(.gz), loader(.bin), and vmlinuz.efi
2023-11-19 10:00 [PATCH v2] riscv: add dependency among Image(.gz), loader(.bin), and vmlinuz.efi Masahiro Yamada
@ 2023-11-21 16:36 ` Ard Biesheuvel
2023-11-21 17:51 ` Samuel Holland
2024-01-20 21:09 ` patchwork-bot+linux-riscv
2 siblings, 0 replies; 4+ messages in thread
From: Ard Biesheuvel @ 2023-11-21 16:36 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, linux-riscv,
linux-kernel, linux-kbuild, Simon Glass
On Sun, 19 Nov 2023 at 05:00, Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> A common issue in Makefile is a race in parallel building.
>
> You need to be careful to prevent multiple threads from writing to the
> same file simultaneously.
>
> Commit 3939f3345050 ("ARM: 8418/1: add boot image dependencies to not
> generate invalid images") addressed such a bad scenario.
>
> A similar symptom occurs with the following command:
>
> $ make -j$(nproc) ARCH=riscv Image Image.gz loader loader.bin vmlinuz.efi
> [ snip ]
> SORTTAB vmlinux
> OBJCOPY arch/riscv/boot/Image
> OBJCOPY arch/riscv/boot/Image
> OBJCOPY arch/riscv/boot/Image
> OBJCOPY arch/riscv/boot/Image
> OBJCOPY arch/riscv/boot/Image
> GZIP arch/riscv/boot/Image.gz
> AS arch/riscv/boot/loader.o
> AS arch/riscv/boot/loader.o
> Kernel: arch/riscv/boot/Image is ready
> PAD arch/riscv/boot/vmlinux.bin
> GZIP arch/riscv/boot/vmlinuz
> Kernel: arch/riscv/boot/loader is ready
> OBJCOPY arch/riscv/boot/loader.bin
> Kernel: arch/riscv/boot/loader.bin is ready
> Kernel: arch/riscv/boot/Image.gz is ready
> OBJCOPY arch/riscv/boot/vmlinuz.o
> LD arch/riscv/boot/vmlinuz.efi.elf
> OBJCOPY arch/riscv/boot/vmlinuz.efi
> Kernel: arch/riscv/boot/vmlinuz.efi is ready
>
> The log "OBJCOPY arch/riscv/boot/Image" is displayed 5 times.
> (also "AS arch/riscv/boot/loader.o" twice.)
>
> It indicates that 5 threads simultaneously enter arch/riscv/boot/
> and write to arch/riscv/boot/Image.
>
> It occasionally leads to a build failure:
>
> $ make -j$(nproc) ARCH=riscv Image Image.gz loader loader.bin vmlinuz.efi
> [ snip ]
> SORTTAB vmlinux
> OBJCOPY arch/riscv/boot/Image
> OBJCOPY arch/riscv/boot/Image
> OBJCOPY arch/riscv/boot/Image
> OBJCOPY arch/riscv/boot/Image
> PAD arch/riscv/boot/vmlinux.bin
> truncate: Invalid number: 'arch/riscv/boot/vmlinux.bin'
> make[2]: *** [drivers/firmware/efi/libstub/Makefile.zboot:13: arch/riscv/boot/vmlinux.bin] Error 1
> make[2]: *** Deleting file 'arch/riscv/boot/vmlinux.bin'
> make[1]: *** [arch/riscv/Makefile:167: vmlinuz.efi] Error 2
> make[1]: *** Waiting for unfinished jobs....
> Kernel: arch/riscv/boot/Image is ready
> GZIP arch/riscv/boot/Image.gz
> AS arch/riscv/boot/loader.o
> AS arch/riscv/boot/loader.o
> Kernel: arch/riscv/boot/loader is ready
> OBJCOPY arch/riscv/boot/loader.bin
> Kernel: arch/riscv/boot/loader.bin is ready
> Kernel: arch/riscv/boot/Image.gz is ready
> make: *** [Makefile:234: __sub-make] Error 2
>
> Image.gz, loader, vmlinuz.efi depend on Image. loader.bin depends
> on loader. Such dependencies are not specified in arch/riscv/Makefile.
>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---
>
> Changes in v2:
> - Fix commit log
>
Acked-by: Ard Biesheuvel <ardb@kernel.org>
> arch/riscv/Makefile | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile
> index 5cbe596345c1..1d6ed27e0a2a 100644
> --- a/arch/riscv/Makefile
> +++ b/arch/riscv/Makefile
> @@ -163,6 +163,8 @@ BOOT_TARGETS := Image Image.gz loader loader.bin xipImage vmlinuz.efi
>
> all: $(notdir $(KBUILD_IMAGE))
>
> +loader.bin: loader
> +Image.gz loader vmlinuz.efi: Image
> $(BOOT_TARGETS): vmlinux
> $(Q)$(MAKE) $(build)=$(boot) $(boot)/$@
> @$(kecho) ' Kernel: $(boot)/$@ is ready'
> --
> 2.40.1
>
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: [PATCH v2] riscv: add dependency among Image(.gz), loader(.bin), and vmlinuz.efi
2023-11-19 10:00 [PATCH v2] riscv: add dependency among Image(.gz), loader(.bin), and vmlinuz.efi Masahiro Yamada
2023-11-21 16:36 ` Ard Biesheuvel
@ 2023-11-21 17:51 ` Samuel Holland
2024-01-20 21:09 ` patchwork-bot+linux-riscv
2 siblings, 0 replies; 4+ messages in thread
From: Samuel Holland @ 2023-11-21 17:51 UTC (permalink / raw)
To: Masahiro Yamada
Cc: linux-kernel, linux-kbuild, Ard Biesheuvel, Simon Glass,
Paul Walmsley, Palmer Dabbelt, Albert Ou, linux-riscv
On 2023-11-19 4:00 AM, Masahiro Yamada wrote:
> A common issue in Makefile is a race in parallel building.
>
> You need to be careful to prevent multiple threads from writing to the
> same file simultaneously.
>
> Commit 3939f3345050 ("ARM: 8418/1: add boot image dependencies to not
> generate invalid images") addressed such a bad scenario.
>
> A similar symptom occurs with the following command:
>
> $ make -j$(nproc) ARCH=riscv Image Image.gz loader loader.bin vmlinuz.efi
> [ snip ]
> SORTTAB vmlinux
> OBJCOPY arch/riscv/boot/Image
> OBJCOPY arch/riscv/boot/Image
> OBJCOPY arch/riscv/boot/Image
> OBJCOPY arch/riscv/boot/Image
> OBJCOPY arch/riscv/boot/Image
> GZIP arch/riscv/boot/Image.gz
> AS arch/riscv/boot/loader.o
> AS arch/riscv/boot/loader.o
> Kernel: arch/riscv/boot/Image is ready
> PAD arch/riscv/boot/vmlinux.bin
> GZIP arch/riscv/boot/vmlinuz
> Kernel: arch/riscv/boot/loader is ready
> OBJCOPY arch/riscv/boot/loader.bin
> Kernel: arch/riscv/boot/loader.bin is ready
> Kernel: arch/riscv/boot/Image.gz is ready
> OBJCOPY arch/riscv/boot/vmlinuz.o
> LD arch/riscv/boot/vmlinuz.efi.elf
> OBJCOPY arch/riscv/boot/vmlinuz.efi
> Kernel: arch/riscv/boot/vmlinuz.efi is ready
>
> The log "OBJCOPY arch/riscv/boot/Image" is displayed 5 times.
> (also "AS arch/riscv/boot/loader.o" twice.)
>
> It indicates that 5 threads simultaneously enter arch/riscv/boot/
> and write to arch/riscv/boot/Image.
>
> It occasionally leads to a build failure:
>
> $ make -j$(nproc) ARCH=riscv Image Image.gz loader loader.bin vmlinuz.efi
> [ snip ]
> SORTTAB vmlinux
> OBJCOPY arch/riscv/boot/Image
> OBJCOPY arch/riscv/boot/Image
> OBJCOPY arch/riscv/boot/Image
> OBJCOPY arch/riscv/boot/Image
> PAD arch/riscv/boot/vmlinux.bin
> truncate: Invalid number: 'arch/riscv/boot/vmlinux.bin'
> make[2]: *** [drivers/firmware/efi/libstub/Makefile.zboot:13: arch/riscv/boot/vmlinux.bin] Error 1
> make[2]: *** Deleting file 'arch/riscv/boot/vmlinux.bin'
> make[1]: *** [arch/riscv/Makefile:167: vmlinuz.efi] Error 2
> make[1]: *** Waiting for unfinished jobs....
> Kernel: arch/riscv/boot/Image is ready
> GZIP arch/riscv/boot/Image.gz
> AS arch/riscv/boot/loader.o
> AS arch/riscv/boot/loader.o
> Kernel: arch/riscv/boot/loader is ready
> OBJCOPY arch/riscv/boot/loader.bin
> Kernel: arch/riscv/boot/loader.bin is ready
> Kernel: arch/riscv/boot/Image.gz is ready
> make: *** [Makefile:234: __sub-make] Error 2
>
> Image.gz, loader, vmlinuz.efi depend on Image. loader.bin depends
> on loader. Such dependencies are not specified in arch/riscv/Makefile.
>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---
>
> Changes in v2:
> - Fix commit log
>
> arch/riscv/Makefile | 2 ++
> 1 file changed, 2 insertions(+)
Reviewed-by: Samuel Holland <samuel.holland@sifive.com>
Tested-by: Samuel Holland <samuel.holland@sifive.com>
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: [PATCH v2] riscv: add dependency among Image(.gz), loader(.bin), and vmlinuz.efi
2023-11-19 10:00 [PATCH v2] riscv: add dependency among Image(.gz), loader(.bin), and vmlinuz.efi Masahiro Yamada
2023-11-21 16:36 ` Ard Biesheuvel
2023-11-21 17:51 ` Samuel Holland
@ 2024-01-20 21:09 ` patchwork-bot+linux-riscv
2 siblings, 0 replies; 4+ messages in thread
From: patchwork-bot+linux-riscv @ 2024-01-20 21:09 UTC (permalink / raw)
To: Masahiro Yamada
Cc: linux-riscv, paul.walmsley, palmer, aou, linux-kernel,
linux-kbuild, ardb, sjg
Hello:
This patch was applied to riscv/linux.git (fixes)
by Palmer Dabbelt <palmer@rivosinc.com>:
On Sun, 19 Nov 2023 19:00:24 +0900 you wrote:
> A common issue in Makefile is a race in parallel building.
>
> You need to be careful to prevent multiple threads from writing to the
> same file simultaneously.
>
> Commit 3939f3345050 ("ARM: 8418/1: add boot image dependencies to not
> generate invalid images") addressed such a bad scenario.
>
> [...]
Here is the summary with links:
- [v2] riscv: add dependency among Image(.gz), loader(.bin), and vmlinuz.efi
https://git.kernel.org/riscv/c/c4db7ff7a9ed
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2024-01-20 21:09 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-11-19 10:00 [PATCH v2] riscv: add dependency among Image(.gz), loader(.bin), and vmlinuz.efi Masahiro Yamada
2023-11-21 16:36 ` Ard Biesheuvel
2023-11-21 17:51 ` Samuel Holland
2024-01-20 21:09 ` patchwork-bot+linux-riscv
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox