Re: [OE-core] [PATCH 0/4] Oe-selftest for Clang, LLVM, LLD

[Deepesh Varatharajan <deepesh.varatharajan@windriver.com>]

On 21-02-2026 13:53, Mathieu Dubois-Briand wrote:
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
> On Fri Feb 20, 2026 at 7:02 AM CET, Deepesh via lists.openembedded.org Varatharajan wrote:
>> From: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
>>
>> This patch series introduces a test framework to run selftests for
>> target builds of Clang, LLVM, and LLD.
>>
>> Summary of changes:
>> -Patch 1: Disable clang-tools-extra tests, as Clang's CMake cannot
>>            detect the llvm-bcanalyzer target.
>> -Patch 2: Enabled the necessary CMake flags for clang, llvm & lld tests
>> -Patch 3: Selftest script for Clang, LLVM, and LLD including exclude list
>> -Patch 4: Fix rust build failure by replacing target llvm-config with native
>>
>> Testing:
>> 1. The test suite has been successfully validated on x86-64, arm64, riscv64
>> and arm32 for all Clang, LLD & LLVM components.
>> 2. Clang and LLD tests passed successfully on x86. LLVM tests are skipped
>> on x86 since it's broken. A follow-up bug will be filed and addressed.
>> 3. Clang, LLVM & LLD all together ~80K tests are there and we currently
>> ~250 tests to exclude list which are failing (0.32%).
>>
>> Impact on build time:
>> These builds were executed in a controlled environment without poky-sstate
>> enabled.
>>
>> Default(without changes):
>> |  Target   | Run 1      | Run 2      | Run 3      |    Average     |
>> | --------- | ---------- | ---------- | ---------- | -------------- |
>> |   llvm    | 22m18.441s | 22m31.905s | 22m23.718s |   22m24.688s   |
>> |   clang   | 30m09.774s | 30m22.680s | 30m15.312s |   30m15.922s   |
>> |   lld     | 1m26.994s  | 1m30.415s  | 1m29.006s  |   1m28.805s    |
>>
>> With the testsuite changes applied:
>> |  Target   | Run 1      | Run 2      | Run 3      |    Average     |
>> | --------- | ---------- | ---------- | ---------- | -------------- |
>> |   llvm    | 23m32.104s | 23m45.887s | 23m38.157s |   23m38.716s   |
>> |   clang   | 30m31.402s | 30m46.198s | 30m39.445s |   30m39.015s   |
>> |   lld     | 1m29.845s  | 1m34.212s  | 1m31.925s  |   1m31.994s    |
>>
>> Comparison:
>> |  Target   | Default Avg | With Changes Avg |     **Diff**     |
>> | --------- | ----------- | ---------------- | ---------------- |
>> |   llvm    | 22m24.688s  | 23m38.716s       |   +1m14.028s     |
>> |   clang   | 30m15.922s  | 30m39.015s       |   +0m23.093s     |
>> |   lld     | 1m28.805s   | 1m31.994s        |   +0m3.189s      |
>>
>> Impact on tmpdir size:
>> | Metric                               | Size  |
>> | ------------------------------------ | ----- |
>> | Default target `tmpdir` size         | 74 GB |
>> | Target `tmpdir` size (after changes) | 79 GB |
>> | Difference                           | +5 GB |
>>
>> Deepesh Varatharajan (4):
> Hi Deepesh,
>
> It looks like some selftests are failing:
>
> 2026-02-20 20:32:17,178 - oe-selftest - INFO - clang.LLVMSelfTestSystemEmulated.test_llvm (subunit.RemotedTestCase)
> 2026-02-20 20:32:17,179 - oe-selftest - INFO -  ... FAIL
> ...
> 2026-02-20 20:32:17,179 - oe-selftest - INFO - 3: 1/1 8/21 (298.44s) (0 failed) (clang.LLVMSelfTestSystemEmulated.test_llvm)
> 2026-02-20 20:32:17,179 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
>    File "/srv/pokybuild/yocto-worker/qemux86-64-tc/build/layers/openembedded-core/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
>      return func(*args, **kwargs)
>    File "/srv/pokybuild/yocto-worker/qemux86-64-tc/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/clang.py", line 184, in test_llvm
>      raise AssertionError(f"llvm-lit failed for LLVM: {output}")
> AssertionError: llvm-lit failed for LLVM: sh: cd: line 0: can't cd to /srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-177928/tmp/work/x86-64-v3-poky-linux/llvm/21.1.8/build/bin: No such file or directory
> ...
> 2026-02-20 20:32:20,596 - oe-selftest - INFO - clang.LLDSelfTestSystemEmulated.test_lld (subunit.RemotedTestCase)
> 2026-02-20 20:32:20,597 - oe-selftest - INFO -  ... FAIL
> ...
> 2026-02-20 20:32:20,814 - oe-selftest - INFO - clang.ClangSelfTestSystemEmulated.test_clang (subunit.RemotedTestCase)
> 2026-02-20 20:32:20,815 - oe-selftest - INFO -  ... FAIL
>
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/66/builds/3243
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/58/builds/1069
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/42/builds/3147
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/28/builds/3165
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/5/builds/3151
>
> Can you have a look at these failures?
Hi Mathieu,

These failures appear to be intermittent and were not caught during our 
testing.
We’ve been able to reproduce the issue locally and have implemented a 
fix. V2
incoming.

Regards,
Deepesh
>
> Thanks,
> Mathieu
>
> --
> Mathieu Dubois-Briand, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>