[PATCH] package.py: warn if target is not a valid ELF file in dwarfsrcfiles

[PATCH] package.py: warn if target is not a valid ELF file in dwarfsrcfiles

[mark.yang]

From: "mark.yang" <mark.yang@lge.com>

When using the Clang toolchain with LTO, intermediate .o files are generated as
LLVM bitcode files instead of ELF files. While .so libraries are ultimately
ELF, .a files are not. In this case, dwarfsrcfiles always fails.

Adding the -ffat-lto-objects option allows the Clang toolchain to produce .o
files that are recognized as ELF, but this is not always a suitable solution
as it causes issues when used in conjunction with -fsanitize=cfi.

Therefore, we need to handle cases where dwarfsrcfiles encounters a file that
is not a valid ELF. Change the behavior from a fatal error to a warning to
notify the user.

Signed-off-by: mark.yang <mark.yang@lge.com>
---
 meta/lib/oe/package.py | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py
index baaa0cba02..56b9bcd546 100644
--- a/meta/lib/oe/package.py
+++ b/meta/lib/oe/package.py
@@ -782,8 +782,12 @@ def source_info(file, d, fatal=True):
     if retval != 0 and retval != 255:
         msg = "dwarfsrcfiles failed with exit code %s (cmd was %s)%s" % (retval, cmd, ":\n%s" % output if output else "")
         if fatal:
-            bb.fatal(msg)
-        bb.note(msg)
+            if "not a valid ELF file" in output:
+                bb.warn(msg)
+            else:
+                bb.fatal(msg)
+        else:
+            bb.note(msg)
 
     debugsources = parse_debugsources_from_dwarfsrcfiles_output(output)
 

Re: [OE-core] [PATCH] package.py: warn if target is not a valid ELF file in dwarfsrcfiles

[Alexander Kanavin]

Can you please provide a way to reproduce and observe the issue? I'm
somewhat confused as to what the original issue is: is dwarfsrcfiles
called on an .a file? That is not an ELF file regardless of whether
clang-with-lto is enabled, so why does it work in one case but not in
the other?

Alex
On Fri, 26 Dec 2025 at 08:36, mark.yang via lists.openembedded.org
<mark.yang=lge.com@lists.openembedded.org> wrote:
>
> From: "mark.yang" <mark.yang@lge.com>
>
> When using the Clang toolchain with LTO, intermediate .o files are generated as
> LLVM bitcode files instead of ELF files. While .so libraries are ultimately
> ELF, .a files are not. In this case, dwarfsrcfiles always fails.
>
> Adding the -ffat-lto-objects option allows the Clang toolchain to produce .o
> files that are recognized as ELF, but this is not always a suitable solution
> as it causes issues when used in conjunction with -fsanitize=cfi.
>
> Therefore, we need to handle cases where dwarfsrcfiles encounters a file that
> is not a valid ELF. Change the behavior from a fatal error to a warning to
> notify the user.
>
> Signed-off-by: mark.yang <mark.yang@lge.com>
> ---
>  meta/lib/oe/package.py | 8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py
> index baaa0cba02..56b9bcd546 100644
> --- a/meta/lib/oe/package.py
> +++ b/meta/lib/oe/package.py
> @@ -782,8 +782,12 @@ def source_info(file, d, fatal=True):
>      if retval != 0 and retval != 255:
>          msg = "dwarfsrcfiles failed with exit code %s (cmd was %s)%s" % (retval, cmd, ":\n%s" % output if output else "")
>          if fatal:
> -            bb.fatal(msg)
> -        bb.note(msg)
> +            if "not a valid ELF file" in output:
> +                bb.warn(msg)
> +            else:
> +                bb.fatal(msg)
> +        else:
> +            bb.note(msg)
>
>      debugsources = parse_debugsources_from_dwarfsrcfiles_output(output)
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#228547): https://lists.openembedded.org/g/openembedded-core/message/228547
> Mute This Topic: https://lists.openembedded.org/mt/116947501/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [PATCH] package.py: warn if target is not a valid ELF file in dwarfsrcfiles

[mark.yang]

[-- Attachment #1: Type: text/plain, Size: 3065 bytes --]

My error report is as follows.
https://errors.yoctoproject.org/Errors/Details/892988/

I am using only the openembedded-core layer, with nodistro.
local.conf

> 
> PREFERRED_TOOLCHAIN_TARGET = "clang"
> DISTRO_FEATURES:append = " ld-is-lld"
> require conf/distro/include/lto.inc
> DISTRO_FEATURES:append = " lto"
> # The settings below were taken from meta-clang.
> LTO:toolchain-clang:class-target =
> "${@bb.utils.contains('DISTRO_FEATURES', 'thin-lto', '-flto=thin', '-flto
> -fuse-ld=lld', d)}"
> LTO:toolchain-clang:class-nativesdk =
> "${@bb.utils.contains('DISTRO_FEATURES', 'thin-lto', '-flto=thin', '-flto
> -fuse-ld=lld', d)}"
> 

dwarfsrcfiles checks .a files during the package process.
Whether using LTO or not, DWARF information is extracted by iterating through .o files inside .a files.
However, they must be ELF objects.

To summarize, it is as follows.
The problem is that when using clang + LTO, the .o files are not ELF but LLVM IR bitcode.

On Sat, Dec 27, 2025 at 03:31 AM, Alexander Kanavin wrote:

> 
> Can you please provide a way to reproduce and observe the issue? I'm
> somewhat confused as to what the original issue is: is dwarfsrcfiles
> called on an .a file? That is not an ELF file regardless of whether
> clang-with-lto is enabled, so why does it work in one case but not in
> the other?
> 
> Alex
> On Fri, 26 Dec 2025 at 08:36, mark.yang via lists.openembedded.org
> <mark.yang=lge.com@lists.openembedded.org> wrote:
> 
>> 
>> From: "mark.yang" <mark.yang@lge.com>
>> 
>> When using the Clang toolchain with LTO, intermediate .o files are
>> generated as
>> LLVM bitcode files instead of ELF files. While .so libraries are
>> ultimately
>> ELF, .a files are not. In this case, dwarfsrcfiles always fails.
>> 
>> Adding the -ffat-lto-objects option allows the Clang toolchain to produce
>> .o
>> files that are recognized as ELF, but this is not always a suitable
>> solution
>> as it causes issues when used in conjunction with -fsanitize=cfi.
>> 
>> Therefore, we need to handle cases where dwarfsrcfiles encounters a file
>> that
>> is not a valid ELF. Change the behavior from a fatal error to a warning to
>> 
>> notify the user.
>> 
>> Signed-off-by: mark.yang <mark.yang@lge.com>
>> ---
>> meta/lib/oe/package.py | 8 ++++++--
>> 1 file changed, 6 insertions(+), 2 deletions(-)
>> 
>> diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py
>> index baaa0cba02..56b9bcd546 100644
>> --- a/meta/lib/oe/package.py
>> +++ b/meta/lib/oe/package.py
>> @@ -782,8 +782,12 @@ def source_info(file, d, fatal=True):
>> if retval != 0 and retval != 255:
>> msg = "dwarfsrcfiles failed with exit code %s (cmd was %s)%s" % (retval,
>> cmd, ":\n%s" % output if output else "")
>> if fatal:
>> - bb.fatal(msg)
>> - bb.note(msg)
>> + if "not a valid ELF file" in output:
>> + bb.warn(msg)
>> + else:
>> + bb.fatal(msg)
>> + else:
>> + bb.note(msg)
>> 
>> debugsources = parse_debugsources_from_dwarfsrcfiles_output(output)
>> 
>> 
>> 
>> 
> 
>

[-- Attachment #2: Type: text/html, Size: 3469 bytes --]

Re: [OE-core] [PATCH] package.py: warn if target is not a valid ELF file in dwarfsrcfiles

[Alexander Kanavin]

On Sat, 27 Dec 2025 at 09:46, mark.yang via lists.openembedded.org
<mark.yang=lge.com@lists.openembedded.org> wrote:
>
> My error report is as follows.
> https://errors.yoctoproject.org/Errors/Details/892988/
>
> I am using only the openembedded-core layer, with nodistro.
> local.conf
>
> PREFERRED_TOOLCHAIN_TARGET = "clang"
> DISTRO_FEATURES:append = " ld-is-lld"
> require conf/distro/include/lto.inc
> DISTRO_FEATURES:append = " lto"
> # The settings below were taken from meta-clang.
> LTO:toolchain-clang:class-target = "${@bb.utils.contains('DISTRO_FEATURES', 'thin-lto', '-flto=thin', '-flto -fuse-ld=lld', d)}"
> LTO:toolchain-clang:class-nativesdk = "${@bb.utils.contains('DISTRO_FEATURES', 'thin-lto', '-flto=thin', '-flto -fuse-ld=lld', d)}"
>
>
> dwarfsrcfiles checks .a files during the package process.
> Whether using LTO or not, DWARF information is extracted by iterating through .o files inside .a files.
> However, they must be ELF objects.
>
> To summarize, it is as follows.
> The problem is that when using clang + LTO, the .o files are not ELF but LLVM IR bitcode.

Thanks for the reproduction instructions and the error message.

The fix is somewhat problematic:
- it relies on the particular console output from dwarfsrcfiles which
may change over time (maybe subtly), and then the code around it will
quietly cease to work;
- it obscures other possible issues that would be exposed by the same
error from dwarfsrcfiles: there can be many reasons for something not
being a valid elf file, and they should remain fatal errors;
- conversely for this particular use case it still emits warnings,
even though there's nothing the user can do about it - should it
rather say nothing if .a has only llvm-ir files?

Is it possible to rather guard the call to dwarfsrcfiles with a set of
conditions: inspecting .a file, clang as toolchain and lto in
distro_features, and skip the call if all conditions are true?

Alex

Re: [PATCH] package.py: warn if target is not a valid ELF file in dwarfsrcfiles

[mark.yang]

[-- Attachment #1: Type: text/plain, Size: 185 bytes --]

Thank you for reviewing this, Alex.

I have incorporated it into
https://lists.openembedded.org/g/openembedded-core/topic/patch_package_py_skip/116968059
and uploaded a new patch.

[-- Attachment #2: Type: text/html, Size: 376 bytes --]