Re: [PATCH 0/4 V2] Oe-selftest for Clang, LLVM, LLD

[Deepesh Varatharajan <deepesh.varatharajan@windriver.com>]

Hi All,

Please ignore this patch. The llvm_sysroot_preprocess_testsuite function 
should be added
to llvm:class-target, not llvm:class-nativesdk. I will send the 
corrected version in V3.

Regards,
Deepesh

On 02-03-2026 12:19, Deepesh.Varatharajan@windriver.com wrote:
> From: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
>
> This patch series introduces a test framework to run selftests for
> target builds of Clang, LLVM, and LLD.
>
> v2 changes:
> - Enable required CMake flags only when running the testsuite.
> - Ensure default build time and package sizes remain unaffected.
>
> Summary of changes:
> -Patch 1: Disable clang-tools-extra tests, as Clang's CMake cannot
>            detect the llvm-bcanalyzer target.
> -Patch 2: Enabled the necessary CMake flags for clang, llvm & lld tests
>            when CLANG_ENABLE_TESTSUITE is set to "1".
> -Patch 3: Selftest script for Clang, LLVM, and LLD including exclude list
> -Patch 4: Fix rust build failure by replacing target llvm-config with native
>
> Testing:
> 1. The test suite has been successfully validated on x86-64, arm64, riscv64
> and arm32 for all Clang, LLD & LLVM components.
> 2. Clang and LLD tests passed successfully on x86. LLVM tests are skipped
> on x86 since it's broken. A follow-up bug will be filed and addressed.
> 3. Clang, LLVM & LLD all together ~80K tests are there and we currently
> ~250 tests to exclude list which are failing (0.32%).
>
> Impact on build time:
> These builds were executed in a controlled environment without poky-sstate
> enabled.
>
> Default(without changes):
> +-----------+------------+------------+------------+----------------+
> |  Target   | Run 1      | Run 2      | Run 3      |    Average     |
> | --------- | ---------- | ---------- | ---------- | -------------- |
> |   llvm    | 22m18.441s | 22m31.905s | 22m23.718s |   22m24.688s   |
> |   clang   | 30m09.774s | 30m22.680s | 30m15.312s |   30m15.922s   |
> |   lld     | 1m26.994s  | 1m30.415s  | 1m29.006s  |   1m28.805s    |
> +-----------+------------+------------+------------+----------------+
>
> With the testsuite changes applied:
> +-----------+------------+------------+------------+----------------+
> |  Target   | Run 1      | Run 2      | Run 3      |    Average     |
> | --------- | ---------- | ---------- | ---------- | -------------- |
> |   llvm    | 23m32.104s | 23m45.887s | 23m38.157s |   23m38.716s   |
> |   clang   | 30m31.402s | 30m46.198s | 30m39.445s |   30m39.015s   |
> |   lld     | 1m29.845s  | 1m34.212s  | 1m31.925s  |   1m31.994s    |
> +-----------+------------+------------+------------+----------------+
>
> Comparison:
> The additional overhead is minimal and primarily due to enabling
> test-related build configurations.
> +-----------+-------------+------------------+------------------+
> |  Target   | Default Avg | With Changes Avg |     **Diff**     |
> | --------- | ----------- | ---------------- | ---------------- |
> |   llvm    | 22m24.688s  | 23m38.716s       |   +1m14.028s     |
> |   clang   | 30m15.922s  | 30m39.015s       |   +0m23.093s     |
> |   lld     | 1m28.805s   | 1m31.994s        |   +0m3.189s      |
> +-----------+-------------+------------------+------------------+
>
> Impact on package size and Fix:
> The testsuite changes has a negative impact on certain package sizes,
> as below:
> +---------------------+----------+----------+-----------+------------+
> | Package             | Old (MB) | New (MB) | Diff (MB) | Change (%) |
> +=====================+==========+==========+===========+============+
> | clang/clang-dbg     |    44.99 |    59.79 |    +14.81 |    +32.92% |
> | clang/clang-dev     |    30.22 |    30.22 |     -0.00 |     -0.00% |
> | clang/clang-tools   |     5.05 |    29.58 |    +24.53 |   +485.87% |
> | llvm/llvm-bin       |    16.40 |    71.77 |    +55.36 |   +337.50% |
> | llvm/llvm-dbg       |   386.79 |   646.12 |   +259.33 |    +67.05% |
> | llvm/llvm-dev       |    32.67 |    33.67 |     +1.00 |     +3.05% |
> | llvm/llvm-src       |   387.54 |   389.44 |     +1.90 |     +0.49% |
> | llvm/llvm-staticdev |   609.68 |   616.80 |     +7.12 |     +1.17% |
> +---------------------+----------+----------+-----------+------------+
>
> FIX:
> All testsuite-related changes are now gated by the CLANG_ENABLE_TESTSUITE
> variable (default "0"), ensuring:
> - Test-specific CMake flags are applied only during test execution.
> - Standard builds remain fully unaffected in both build time and package size.
>
> Deepesh Varatharajan (4):
>    clang-tools-extra: disable tests
>    clang: Use CLANG_ENABLE_TESTSUITE to enable LLVM, Clang, and LLD tests
>    oeqa/selftest/clang: Add oe-seltests for Clang/LLVM/LLD
>    rust: Fix rust build failure
>
>   meta/lib/oeqa/selftest/cases/clang.py         | 299 ++++++++++++++++++
>   .../0042-guard-clang-tools-extra-test.patch   |  74 +++++
>   meta/recipes-devtools/clang/clang_git.bb      |   8 +
>   meta/recipes-devtools/clang/common.inc        |   1 +
>   meta/recipes-devtools/clang/lld_git.bb        |   5 +
>   meta/recipes-devtools/clang/llvm_git.bb       |  18 +-
>   meta/recipes-devtools/rust/rust_1.93.0.bb     |   5 +-
>   7 files changed, 407 insertions(+), 3 deletions(-)
>   create mode 100644 meta/lib/oeqa/selftest/cases/clang.py
>   create mode 100644 meta/recipes-devtools/clang/clang/0042-guard-clang-tools-extra-test.patch
>