Re: [PATCH] kbuild: Fix lto build issue for aarch64

Re: [PATCH] kbuild: Fix lto build issue for aarch64

[Masahiro Yamada]

On Fri, Jan 31, 2025 at 7:25 AM Yonghong Song <yonghong.song@linux.dev> wrote:
>
> Commit bede169618c6 ("kbuild: enable objtool for *.mod.o and additional kernel objects")
> added objtool support for various files like *.mod.o, vmlinux.export.o, etc.
> But unfortunately it also breaks lto build for aarch64 where a common source file,
> e.g., kernel/bpf/syscall.c, is compiled with command:
>   clang ... -flto=thin ... -c -o kernel/bpf/syscall.o /home/yhs/work/bpf-next/kernel/bpf/syscall.c  ;
>   ld.lld -EL  -maarch64elf -z norelro -mllvm -import-instr-limit=5 -z noexecstack   \
>       -r -o kernel/bpf/.tmp_syscall.o kernel/bpf/syscall.o;
>   mv kernel/bpf/.tmp_syscall.o kernel/bpf/syscall.o
>
> The above command line completely destroyed thin-lto since file kernel/bpf/syscall.o
> is expected to be a llvm bitcode file and later ld.lld can do cross-file inlining.
>
> The issue has been reported in [1] and Masahiro actually provided a suggested fix.
> But unfortunately that suggested fix does not work.

Thanks for the report, but your patch does not fix the real issue.

I think my original [1] worked, although it was a bit redundant
(extra $(LD) is executed where it is unnecessary).


> This patch made improvement on top of [2] and the issue for aarch64 lto is fixed.
> The x86_64 is also tested without any issue.

You are not fixing the issue.
You are just disabling objtool (i.e. reverting  bede169618c6).


>
> In [1], commit 1b466b29a3bf ("kbuild: re-enable KCSAN for autogenerated *.mod.c intermediaries")
> enabled kcsan for *.mod.c files. This patch actually disabled that.
> Should we revert commit 1b466b29a3bf as well?

No.
I do not want to revert 1b466b29a3bf.

If 1b466b29a3bf were reverted, there would be no point
in applying bede169618c6.


>
>   [1] https://lore.kernel.org/lkml/20241113234526.402738-1-masahiroy@kernel.org/
>   [2] https://lore.kernel.org/lkml/CAK7LNATpu5zYwx7kmaknsPGLXt8n8uCXyFpdi5vZeFZiBxYkGw@mail.gmail.com/
>
> Fixes: bede169618c6 ("kbuild: enable objtool for *.mod.o and additional kernel objects")
> Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
> ---
>  scripts/Makefile.build | 2 ++
>  scripts/Makefile.lib   | 4 +---
>  2 files changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/scripts/Makefile.build b/scripts/Makefile.build
> index c16e4cf54d77..e9ae38b71a08 100644
> --- a/scripts/Makefile.build
> +++ b/scripts/Makefile.build
> @@ -129,6 +129,8 @@ $(obj)/%.ll: $(obj)/%.c FORCE
>
>  is-single-obj-m = $(and $(part-of-module),$(filter $@, $(obj-m)),y)
>
> +need-ld-single = $(is-single-obj-m)
> +
>  ifdef CONFIG_MODVERSIONS
>  # When module versioning is enabled the following steps are executed:
>  # o compile a <file>.o from <file>.c
> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> index 7395200538da..f04672bfe068 100644
> --- a/scripts/Makefile.lib
> +++ b/scripts/Makefile.lib
> @@ -302,11 +302,9 @@ endef
>  # ===========================================================================
>  # These are shared by some Makefile.* files.
>
> -objtool-enabled := y
> -

This is wrong.

You are simply disabling objtool for *.mod.o

I am pretty sure the problem reported by 54babdc
will come back.


I submitted a proper fix.



>  ifdef CONFIG_LTO_CLANG
>  # objtool cannot process LLVM IR. Make $(LD) covert LLVM IR to ELF here.
> -cmd_ld_single = $(if $(objtool-enabled), ; $(LD) $(ld_flags) -r -o $(tmp-target) $@; mv $(tmp-target) $@)
> +cmd_ld_single = $(if $(need-ld-single), ; $(LD) $(ld_flags) -r -o $(tmp-target) $@; mv $(tmp-target) $@)
>  endif
>
>  quiet_cmd_cc_o_c = CC $(quiet_modtag)  $@
> --
> 2.43.5
>


--
Best Regards

Masahiro Yamada

Re: [PATCH] kbuild: Fix lto build issue for aarch64

[Yonghong Song]

On 1/31/25 6:08 AM, Masahiro Yamada wrote:
> On Fri, Jan 31, 2025 at 7:25 AM Yonghong Song <yonghong.song@linux.dev> wrote:
>> Commit bede169618c6 ("kbuild: enable objtool for *.mod.o and additional kernel objects")
>> added objtool support for various files like *.mod.o, vmlinux.export.o, etc.
>> But unfortunately it also breaks lto build for aarch64 where a common source file,
>> e.g., kernel/bpf/syscall.c, is compiled with command:
>>    clang ... -flto=thin ... -c -o kernel/bpf/syscall.o /home/yhs/work/bpf-next/kernel/bpf/syscall.c  ;
>>    ld.lld -EL  -maarch64elf -z norelro -mllvm -import-instr-limit=5 -z noexecstack   \
>>        -r -o kernel/bpf/.tmp_syscall.o kernel/bpf/syscall.o;
>>    mv kernel/bpf/.tmp_syscall.o kernel/bpf/syscall.o
>>
>> The above command line completely destroyed thin-lto since file kernel/bpf/syscall.o
>> is expected to be a llvm bitcode file and later ld.lld can do cross-file inlining.
>>
>> The issue has been reported in [1] and Masahiro actually provided a suggested fix.
>> But unfortunately that suggested fix does not work.
> Thanks for the report, but your patch does not fix the real issue.
>
> I think my original [1] worked, although it was a bit redundant
> (extra $(LD) is executed where it is unnecessary).
>
>
>> This patch made improvement on top of [2] and the issue for aarch64 lto is fixed.
>> The x86_64 is also tested without any issue.
> You are not fixing the issue.
> You are just disabling objtool (i.e. reverting  bede169618c6).

You are correct. Essentially my patch just reverted bede169618c6
since I am not familar with Makefile.build and Makefile.lib so
won't be able to come up with a proper solution.

>
>
>> In [1], commit 1b466b29a3bf ("kbuild: re-enable KCSAN for autogenerated *.mod.c intermediaries")
>> enabled kcsan for *.mod.c files. This patch actually disabled that.
>> Should we revert commit 1b466b29a3bf as well?
> No.
> I do not want to revert 1b466b29a3bf.
>
> If 1b466b29a3bf were reverted, there would be no point
> in applying bede169618c6.
>
>
>>    [1] https://lore.kernel.org/lkml/20241113234526.402738-1-masahiroy@kernel.org/
>>    [2] https://lore.kernel.org/lkml/CAK7LNATpu5zYwx7kmaknsPGLXt8n8uCXyFpdi5vZeFZiBxYkGw@mail.gmail.com/
>>
>> Fixes: bede169618c6 ("kbuild: enable objtool for *.mod.o and additional kernel objects")
>> Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
>> ---
>>   scripts/Makefile.build | 2 ++
>>   scripts/Makefile.lib   | 4 +---
>>   2 files changed, 3 insertions(+), 3 deletions(-)
>>
>> diff --git a/scripts/Makefile.build b/scripts/Makefile.build
>> index c16e4cf54d77..e9ae38b71a08 100644
>> --- a/scripts/Makefile.build
>> +++ b/scripts/Makefile.build
>> @@ -129,6 +129,8 @@ $(obj)/%.ll: $(obj)/%.c FORCE
>>
>>   is-single-obj-m = $(and $(part-of-module),$(filter $@, $(obj-m)),y)
>>
>> +need-ld-single = $(is-single-obj-m)
>> +
>>   ifdef CONFIG_MODVERSIONS
>>   # When module versioning is enabled the following steps are executed:
>>   # o compile a <file>.o from <file>.c
>> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
>> index 7395200538da..f04672bfe068 100644
>> --- a/scripts/Makefile.lib
>> +++ b/scripts/Makefile.lib
>> @@ -302,11 +302,9 @@ endef
>>   # ===========================================================================
>>   # These are shared by some Makefile.* files.
>>
>> -objtool-enabled := y
>> -
> This is wrong.
>
> You are simply disabling objtool for *.mod.o
>
> I am pretty sure the problem reported by 54babdc
> will come back.
>
>
> I submitted a proper fix.

I tested the patch with arm64/x86_64 + lto/no-lto. Everything works fine.
I have replied with my Tested-by tag. Thanks!

>
>
>
>>   ifdef CONFIG_LTO_CLANG
>>   # objtool cannot process LLVM IR. Make $(LD) covert LLVM IR to ELF here.
>> -cmd_ld_single = $(if $(objtool-enabled), ; $(LD) $(ld_flags) -r -o $(tmp-target) $@; mv $(tmp-target) $@)
>> +cmd_ld_single = $(if $(need-ld-single), ; $(LD) $(ld_flags) -r -o $(tmp-target) $@; mv $(tmp-target) $@)
>>   endif
>>
>>   quiet_cmd_cc_o_c = CC $(quiet_modtag)  $@
>> --
>> 2.43.5
>>
>
> --
> Best Regards
>
> Masahiro Yamada