DEFINE_FLEX_test: EXPECTATION FAILED at lib/overflow_kunit.c:1200:

DEFINE_FLEX_test: EXPECTATION FAILED at lib/overflow_kunit.c:1200:

[Naresh Kamboju]

The overflow_DEFINE_FLEX_test KUnit test case. This test consistently
passes when built with GCC-13 but fails when using Clang-19 or
Clang-nightly.

Test Case: overflow_DEFINE_FLEX_test
Compilers: Passing: GCC-13
           Failing: Clang-19, Clang-nightly
Observed Behavior: The test failure is reproducible with Clang builds,
                   while GCC builds produce consistent success.

This inconsistency suggests a potential issue either in the Clang toolchain
or in the test implementation that is exposed by Clang's compilation behavior.

Test log:
----------
<6>[   92.471692]     # castable_to_type_test: 103 castable_to_type()
tests finished
<6>[   92.474933]     ok 21 castable_to_type_test
<3>[   92.476715]     # DEFINE_FLEX_test: EXPECTATION FAILED at
lib/overflow_kunit.c:1200
<3>[   92.476715]     Expected
__builtin_dynamic_object_size(two_but_zero, 0) == expected_raw_size,
but
<3>[   92.476715]         __builtin_dynamic_object_size(two_but_zero,
0) == 12 (0xc)
<3>[   92.476715]         expected_raw_size == 8 (0x8)
<6>[   92.480178]     not ok 22 DEFINE_FLEX_test
<6>[   92.483020] # overflow: pass:21 fail:1 skip:0 total:22


Test failed log with clang-19 and clang-nightly :
-----------------------------------------
- https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20241119/testrun/25908897/suite/kunit/test/overflow_DEFINE_FLEX_test/log
- https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20241119/testrun/25908897/suite/kunit/test/overflow_DEFINE_FLEX_test/details/
- https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20241119/testrun/25908335/suite/kunit/test/overflow_DEFINE_FLEX_test/log
- https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20241119/testrun/25908335/suite/kunit/test/overflow_DEFINE_FLEX_test/details/

Test history:
----------
- https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20241119/testrun/25908897/suite/kunit/test/overflow_DEFINE_FLEX_test/history/

Test pass log with gcc-13
----------
- https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20241119/testrun/25908488/suite/kunit/test/overflow_DEFINE_FLEX_test/log
- https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20241119/testrun/25908488/suite/kunit/test/overflow_DEFINE_FLEX_test/details/

Build image:
-----------
- https://storage.tuxsuite.com/public/linaro/lkft/builds/2p3Z8y7TVGQ8pXwjhYTnnMNxFpZ/

Steps to reproduce test run:
------------
- https://tuxapi.tuxsuite.com/v1/groups/linaro/projects/lkft/tests/2p3ZAtZdJgM7AbNB5vGAGDf73vx/reproducer
- https://tuxapi.tuxsuite.com/v1/groups/linaro/projects/lkft/tests/2p3ZAtZdJgM7AbNB5vGAGDf73vx/tux_plan

Steps to reproduce builds:
----------
- tuxmake --runtime podman \
          --target-arch arm64 \
          --toolchain clang-19 \
          --kconfig defconfig \
          --kconfig-add CONFIG_KASAN=y \
          --kconfig-add CONFIG_KUNIT=y \
          --kconfig-add CONFIG_KUNIT_ALL_TESTS=y LLVM=1 LLVM_IAS=1

metadata:
----
git repo: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
git sha: 414c97c966b69e4a6ea7b32970fa166b2f9b9ef0
kernel config:
https://storage.tuxsuite.com/public/linaro/lkft/builds/2p3Z8y7TVGQ8pXwjhYTnnMNxFpZ/config
build url: https://storage.tuxsuite.com/public/linaro/lkft/builds/2p3Z8y7TVGQ8pXwjhYTnnMNxFpZ/
toolchain: clang-19 and clang-nightly
config: defconfig + KASAN + KUNIT + KUNIT_ALL_TESTS
arch: arm64, armv5

Please let me know if more information or logs are required.

--
Linaro LKFT
https://lkft.linaro.org

Re: DEFINE_FLEX_test: EXPECTATION FAILED at lib/overflow_kunit.c:1200:

[Nathan Chancellor]

Hi Naresh,

+ Kees (it does not look like you own lib/overflow_kunit.c, should that
be updated?)

On Tue, Nov 19, 2024 at 04:17:41PM +0530, Naresh Kamboju wrote:
> The overflow_DEFINE_FLEX_test KUnit test case. This test consistently
> passes when built with GCC-13 but fails when using Clang-19 or
> Clang-nightly.
> 
> Test Case: overflow_DEFINE_FLEX_test
> Compilers: Passing: GCC-13
>            Failing: Clang-19, Clang-nightly
> Observed Behavior: The test failure is reproducible with Clang builds,
>                    while GCC builds produce consistent success.
> 
> This inconsistency suggests a potential issue either in the Clang toolchain
> or in the test implementation that is exposed by Clang's compilation behavior.
> 
> Test log:
> ----------
> <6>[   92.471692]     # castable_to_type_test: 103 castable_to_type()
> tests finished
> <6>[   92.474933]     ok 21 castable_to_type_test
> <3>[   92.476715]     # DEFINE_FLEX_test: EXPECTATION FAILED at
> lib/overflow_kunit.c:1200
> <3>[   92.476715]     Expected
> __builtin_dynamic_object_size(two_but_zero, 0) == expected_raw_size,
> but
> <3>[   92.476715]         __builtin_dynamic_object_size(two_but_zero,
> 0) == 12 (0xc)
> <3>[   92.476715]         expected_raw_size == 8 (0x8)
> <6>[   92.480178]     not ok 22 DEFINE_FLEX_test
> <6>[   92.483020] # overflow: pass:21 fail:1 skip:0 total:22

I can reproduce this with Clang 19.1.3 on 6.12, so it does not appear to
be a recent problem.

  $ printf 'CONFIG_%s=y\n' KUNIT OVERFLOW_KUNIT_TEST >kernel/configs/overflow_kunit.config

  $ make -skj"$(nproc)" ARCH=arm64 LLVM=1 mrproper {def,hardening.,overflow_kunit.}config Image.gz

  $ boot-qemu.py ...
  [    0.000000] Linux version 6.12.0 (nathan@thelio-3990X) (ClangBuiltLinux clang version 19.1.3 (https://github.com/llvm/llvm-project.git ab51eccf88f5321e7c60591c5546b254b6afab99), ClangBuiltLinux LLD 19.1.3 (https://github.com/llvm/llvm-project.git ab51eccf88f5321e7c60591c5546b254b6afab99)) #1 SMP PREEMPT Tue Nov 19 07:28:39 MST 2024
  ...
  [    4.184764]     # DEFINE_FLEX_test: EXPECTATION FAILED at lib/overflow_kunit.c:1200
  [    4.184764]     Expected __builtin_dynamic_object_size(two_but_zero, 0) == expected_raw_size, but
  [    4.184764]         __builtin_dynamic_object_size(two_but_zero, 0) == 12 (0xc)
  [    4.184764]         expected_raw_size == 8 (0x8)
  [    4.190023]     not ok 22 DEFINE_FLEX_test
  [    4.206181] # overflow: pass:21 fail:1 skip:0 total:22
  [    4.208635] # Totals: pass:21 fail:1 skip:0 total:22
  [    4.212218] not ok 1 overflow
  ...

I do not really understand how __builtin_dynamic_object_size() can
return 12 for two_but_zero with __counted_by() because DEFINE_RAW_FLEX()
does not initialize the counter so it should be zero... Kees? I guess
maybe something changed on the LLVM side, I will see if I can bisect
later (all the boxes are tied up with other compilations at the moment).

Cheers,
Nathan

Re: DEFINE_FLEX_test: EXPECTATION FAILED at lib/overflow_kunit.c:1200:

[Kees Cook]

On Tue, Nov 19, 2024 at 08:05:16AM -0700, Nathan Chancellor wrote:
> Hi Naresh,
> 
> + Kees (it does not look like you own lib/overflow_kunit.c, should that
> be updated?)

Yeah, though I thought the selftest tree was moving a bunch of these
into a subdirectory? Maybe that didn't happen for v6.12?

> On Tue, Nov 19, 2024 at 04:17:41PM +0530, Naresh Kamboju wrote:
> > The overflow_DEFINE_FLEX_test KUnit test case. This test consistently
> > passes when built with GCC-13 but fails when using Clang-19 or
> > Clang-nightly.
> > 
> > Test Case: overflow_DEFINE_FLEX_test
> > Compilers: Passing: GCC-13
> >            Failing: Clang-19, Clang-nightly
> > Observed Behavior: The test failure is reproducible with Clang builds,
> >                    while GCC builds produce consistent success.
> > 
> > This inconsistency suggests a potential issue either in the Clang toolchain
> > or in the test implementation that is exposed by Clang's compilation behavior.
> > 
> > Test log:
> > ----------
> > <6>[   92.471692]     # castable_to_type_test: 103 castable_to_type()
> > tests finished
> > <6>[   92.474933]     ok 21 castable_to_type_test
> > <3>[   92.476715]     # DEFINE_FLEX_test: EXPECTATION FAILED at
> > lib/overflow_kunit.c:1200
> > <3>[   92.476715]     Expected
> > __builtin_dynamic_object_size(two_but_zero, 0) == expected_raw_size,
> > but
> > <3>[   92.476715]         __builtin_dynamic_object_size(two_but_zero,
> > 0) == 12 (0xc)
> > <3>[   92.476715]         expected_raw_size == 8 (0x8)
> > <6>[   92.480178]     not ok 22 DEFINE_FLEX_test
> > <6>[   92.483020] # overflow: pass:21 fail:1 skip:0 total:22
> 
> I can reproduce this with Clang 19.1.3 on 6.12, so it does not appear to
> be a recent problem.
> 
>   $ printf 'CONFIG_%s=y\n' KUNIT OVERFLOW_KUNIT_TEST >kernel/configs/overflow_kunit.config
> 
>   $ make -skj"$(nproc)" ARCH=arm64 LLVM=1 mrproper {def,hardening.,overflow_kunit.}config Image.gz
> 
>   $ boot-qemu.py ...
>   [    0.000000] Linux version 6.12.0 (nathan@thelio-3990X) (ClangBuiltLinux clang version 19.1.3 (https://github.com/llvm/llvm-project.git ab51eccf88f5321e7c60591c5546b254b6afab99), ClangBuiltLinux LLD 19.1.3 (https://github.com/llvm/llvm-project.git ab51eccf88f5321e7c60591c5546b254b6afab99)) #1 SMP PREEMPT Tue Nov 19 07:28:39 MST 2024
>   ...
>   [    4.184764]     # DEFINE_FLEX_test: EXPECTATION FAILED at lib/overflow_kunit.c:1200
>   [    4.184764]     Expected __builtin_dynamic_object_size(two_but_zero, 0) == expected_raw_size, but
>   [    4.184764]         __builtin_dynamic_object_size(two_but_zero, 0) == 12 (0xc)
>   [    4.184764]         expected_raw_size == 8 (0x8)
>   [    4.190023]     not ok 22 DEFINE_FLEX_test
>   [    4.206181] # overflow: pass:21 fail:1 skip:0 total:22
>   [    4.208635] # Totals: pass:21 fail:1 skip:0 total:22
>   [    4.212218] not ok 1 overflow
>   ...
> 
> I do not really understand how __builtin_dynamic_object_size() can
> return 12 for two_but_zero with __counted_by() because DEFINE_RAW_FLEX()
> does not initialize the counter so it should be zero... Kees? I guess
> maybe something changed on the LLVM side, I will see if I can bisect
> later (all the boxes are tied up with other compilations at the moment).

Hmm. I assume this is related to recent bdos vs counted_by changes in
Clang 19.1.2 (or .3?) But I'm going to have to track down which is
causing it.

The test is supposed to check this...

if counted_by is supported, DEFINE_RAW_FLEX will init counted_by to 0,
so the object is expected to be seen as sizeof(int) + sizeof(u32) (8).

if counted_by is NOT supported, then bdos will return the on-stack size
of the object (8 + sizeof(s16) * 2) == 12.

If LLVM switch to "max of counted_by or bos", then returning 12 would
make sense again.

I will check behaviors and compare it to GCC 15...

-- 
Kees Cook

Re: DEFINE_FLEX_test: EXPECTATION FAILED at lib/overflow_kunit.c:1200:

[David Gow]

[-- Attachment #1: Type: text/plain, Size: 473 bytes --]

On Wed, 20 Nov 2024 at 01:38, Kees Cook <kees@kernel.org> wrote:
>
> On Tue, Nov 19, 2024 at 08:05:16AM -0700, Nathan Chancellor wrote:
> > Hi Naresh,
> >
> > + Kees (it does not look like you own lib/overflow_kunit.c, should that
> > be updated?)
>
> Yeah, though I thought the selftest tree was moving a bunch of these
> into a subdirectory? Maybe that didn't happen for v6.12?
>

This should be happening post 6.13-rc1, in order to dodge a bunch of conflicts.

-- David

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5294 bytes --]