From: "Sundeep KOKKONDA" <sundeep.kokkonda@gmail.com>
To: Randy MacLeod <randy.macleod@windriver.com>,
openembedded-core@lists.openembedded.org,
"steve@sakoman.com" <steve@sakoman.com>
Subject: Re: [OE-core] [kirkstone 00/26] Patch review
Date: Mon, 20 Jun 2022 08:39:25 +0530 [thread overview]
Message-ID: <4b1d26ac-6fe2-50cf-4cbd-23fcee49b0b8@gmail.com> (raw)
In-Reply-To: <e449115f-00a6-e11a-cd8d-f502db8a0b0d@windriver.com>
[-- Attachment #1.1: Type: text/plain, Size: 2860 bytes --]
Hello Randy,
On 16-06-2022 19:49, Randy MacLeod wrote:
> On 2022-06-15 22:19, Sundeep KOKKONDA wrote:
>> Hello,
>>
>> GCC regression test performed on /Kirkstone/ with gcc-11.3 &
>> gcc-11.2. The results are *NOT* identical. Below is test summary.
>>
>> *gcc-11.3:*
>> / === gcc Summary ===/
>> /# of expected passes 126552/
>> /# of unexpected failures 24295/
>> /# of unexpected successes 77/
>> /# of expected failures 782/
>> /# of unresolved testcases 10/
>> /# of unsupported tests 2760/
>> //home/bft1/skokkonda/kirkstone/poky/build/sdk/sysroots/x86_64-pokysdk-linux/usr/bin/x86_64-poky-linux/x86_64-poky-linux-gcc
>> version 11.3.0 (GCC)/
>>
>> *gcc-11.2:*
>> / === gcc Summary ===/
>> /# of expected passes 125933/
>> /# of unexpected failures 24215/
>> /# of unexpected successes 76/
>> /# of expected failures 773/
>> /# of unresolved testcases 10/
>> /# of unsupported tests 2779/
>> //home/bft1/skokkonda/kirkstone-gcc-11.2/poky/build/sdk/sysroots/x86_64-pokysdk-linux/usr/bin/x86_64-poky-linux/x86_64-poky-linux-gcc
>> version 11.2.0 (GCC)
>>
>>
>> /
>
> /11.3: # of expected passes 126552/
> //11.2: # of expected passes 125933//
>
> So >= 619 additional tests were added and they mostly pass (as you'd
> expect).
> That's good news.
>
>
> /11.3: # of unexpected failures 24295/
> /11.2: # of unexpected failures 24215/
> /
> /
> /So 80 additional unexpected failures - can you summarize what these
> failures are?
> /
There are 83 tests failed with GCC-11.3 and out of them most of the
failed cases (75 tests) are /execution /tests and the remaining are
/test for excess errors/ (Tests with excess messages output to stderr).
The Failed tests list is attached.
> /Were any of them fixed after the 11.3 tag?
> /
Does it mean, do you want me to test on 11.4 and compare?
> /
> Are they related to one part of the toolchain or spread across various
> features?
> Are they reproducible when buiding outside of bitbake?/
The test failures are from gcc.dg/torture, gcc.c-torture & gcc.target
testsuites.
> /
> /
> /The other differences are less significant.
> If you post similar data in future emails, please do the math for people!
> /
> /
> /
> //
>> gcc-11.3 & gcc-11.2 test summaries are attached.
>
> I didn't look at these yet but thanks for providing them.
>
> Since we're not seeing any runtime regression in our Yocto
> test suite, I suspect that we shouldn't panic but should instead
> just work to understand, fix, upstream and monitor for future regressions.
>
> ../Randy
>
>>
>>
>> --
>> Thanks,
>> Sundeep K.
>>
>>
>>
>
> --
> # Randy MacLeod
> # Wind River Linux
[-- Attachment #1.2: Type: text/html, Size: 6852 bytes --]
[-- Attachment #2: FailedTests_GCC-11.3 --]
[-- Type: text/plain, Size: 5983 bytes --]
FAIL: gcc.c-torture/execute/bitfld-10.c -O0 execution test
FAIL: gcc.c-torture/execute/bitfld-10.c -O1 execution test
FAIL: gcc.c-torture/execute/bitfld-10.c -O2 execution test
FAIL: gcc.c-torture/execute/bitfld-10.c -O3 -g execution test
FAIL: gcc.c-torture/execute/bitfld-10.c -Os execution test
FAIL: gcc.c-torture/execute/bitfld-10.c -O2 -flto -fno-use-linker-plugin -flto-partition=none execution test
FAIL: gcc.c-torture/execute/bitfld-10.c -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects execution test
FAIL: gcc.c-torture/execute/pr103052.c -O0 execution test
FAIL: gcc.c-torture/execute/pr103052.c -O1 execution test
FAIL: gcc.c-torture/execute/pr103052.c -O2 execution test
FAIL: gcc.c-torture/execute/pr103052.c -O3 -g execution test
FAIL: gcc.c-torture/execute/pr103052.c -Os execution test
FAIL: gcc.c-torture/execute/pr103052.c -O2 -flto -fno-use-linker-plugin -flto-partition=none execution test
FAIL: gcc.c-torture/execute/pr103052.c -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects execution test
FAIL: gcc.c-torture/execute/pr103255.c -O0 execution test
FAIL: gcc.c-torture/execute/pr103255.c -O1 execution test
FAIL: gcc.c-torture/execute/pr103255.c -O2 execution test
FAIL: gcc.c-torture/execute/pr103255.c -O3 -fomit-frame-pointer -funroll-loops -fpeel-loops -ftracer -finline-functions execution test
FAIL: gcc.c-torture/execute/pr103255.c -O3 -g execution test
FAIL: gcc.c-torture/execute/pr103255.c -Os execution test
FAIL: gcc.c-torture/execute/pr103255.c -O2 -flto -fno-use-linker-plugin -flto-partition=none execution test
FAIL: gcc.c-torture/execute/pr103255.c -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects execution test
FAIL: gcc.c-torture/execute/pr104814.c -O0 execution test
FAIL: gcc.c-torture/execute/pr104814.c -O1 execution test
FAIL: gcc.c-torture/execute/pr104814.c -O2 execution test
FAIL: gcc.c-torture/execute/pr104814.c -O3 -fomit-frame-pointer -funroll-loops -fpeel-loops -ftracer -finline-functions execution test
FAIL: gcc.c-torture/execute/pr104814.c -O3 -g execution test
FAIL: gcc.c-torture/execute/pr104814.c -Os execution test
FAIL: gcc.c-torture/execute/pr104814.c -O2 -flto -fno-use-linker-plugin -flto-partition=none execution test
FAIL: gcc.c-torture/execute/pr104814.c -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects execution test
FAIL: gcc.dg/analyzer/data-model-1.c (test for warnings, line 254)
FAIL: gcc.dg/analyzer/strndup-1.c (test for warnings, line 11)
FAIL: c-c++-common/cpp/pr104147.c -Wc++-compat execution test
FAIL: gcc.dg/pr102224.c execution test
FAIL: gcc.dg/pr102798.c execution test
FAIL: gcc.dg/pr103860.c execution test
FAIL: gcc.dg/sso-16.c execution test
FAIL: gcc.dg/ipa/pr103083-1.c execution test
FAIL: gcc.dg/lto/pr101868 c_lto_pr101868_0.o-c_lto_pr101868_3.o execute -O2 -fno-strict-aliasing -flto
FAIL: gcc.dg/lto/pr101949 c_lto_pr101949_0.o-c_lto_pr101949_1.o execute -O2 -fipa-pta -flto -flto-partition=1to1
FAIL: gcc.dg/torture/pr102139.c -O0 execution test
FAIL: gcc.dg/torture/pr102139.c -O1 execution test
FAIL: gcc.dg/torture/pr102139.c -O2 execution test
FAIL: gcc.dg/torture/pr102139.c -O3 -g execution test
FAIL: gcc.dg/torture/pr102139.c -Os execution test
FAIL: gcc.dg/torture/pr102139.c -O2 -flto -fno-use-linker-plugin -flto-partition=none execution test
FAIL: gcc.dg/torture/pr102139.c -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects execution test
FAIL: gcc.dg/torture/pr103181.c -O0 execution test
FAIL: gcc.dg/torture/pr103181.c -O1 execution test
FAIL: gcc.dg/torture/pr103181.c -O2 execution test
FAIL: gcc.dg/torture/pr103181.c -O3 -g execution test
FAIL: gcc.dg/torture/pr103181.c -Os execution test
FAIL: gcc.dg/torture/pr103181.c -O2 -flto -fno-use-linker-plugin -flto-partition=none execution test
FAIL: gcc.dg/torture/pr103181.c -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects execution test
FAIL: gcc.dg/torture/pr103237.c -O0 execution test
FAIL: gcc.dg/torture/pr103237.c -O1 execution test
FAIL: gcc.dg/torture/pr103237.c -O2 execution test
FAIL: gcc.dg/torture/pr103237.c -O3 -fomit-frame-pointer -funroll-loops -fpeel-loops -ftracer -finline-functions execution test
FAIL: gcc.dg/torture/pr103237.c -O3 -g execution test
FAIL: gcc.dg/torture/pr103237.c -Os execution test
FAIL: gcc.dg/torture/pr103237.c -O2 -flto -fno-use-linker-plugin -flto-partition=none execution test
FAIL: gcc.dg/torture/pr103237.c -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects execution test
FAIL: gcc.dg/torture/pr105198.c -O0 execution test
FAIL: gcc.dg/torture/pr105198.c -O1 execution test
FAIL: gcc.dg/torture/pr105198.c -O2 execution test
FAIL: gcc.dg/torture/pr105198.c -O3 -fomit-frame-pointer -funroll-loops -fpeel-loops -ftracer -finline-functions execution test
FAIL: gcc.dg/torture/pr105198.c -O3 -g execution test
FAIL: gcc.dg/torture/pr105198.c -Os execution test
FAIL: gcc.dg/torture/pr105198.c -O2 -flto -fno-use-linker-plugin -flto-partition=none execution test
FAIL: gcc.dg/torture/pr105198.c -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects execution test
FAIL: gcc.dg/vect/bb-slp-pr101615-2.c execution test
FAIL: gcc.dg/vect/bb-slp-pr101615-2.c -flto -ffat-lto-objects execution test
FAIL: gcc.target/i386/avx-pr102224.c execution test
FAIL: gcc.target/i386/avx512f-pr101471.c (test for excess errors)
FAIL: gcc.target/i386/avx512f-pr101471.c execution test
FAIL: gcc.target/i386/pr102498.c execution test
FAIL: gcc.target/i386/pr102627.c execution test
FAIL: gcc.target/i386/pr104971.c execution test
FAIL: gcc.target/i386/pr105000-1.c (test for excess errors)
FAIL: gcc.target/i386/pr105000-2.c (test for excess errors)
FAIL: gcc.target/i386/pr105000-3.c (test for excess errors)
FAIL: gcc.target/i386/pr99754-1.c execution test
FAIL: gcc.target/i386/pr99754-2.c execution test
next prev parent reply other threads:[~2022-06-20 3:09 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-06-02 16:51 [OE-core][kirkstone 00/26] Patch review Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 01/26] vim: Upgrade 8.2.4912 -> 8.2.5034 to fix 9 CVEs Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 02/26] tiff: Add jbig PACKAGECONFIG and clarify CVE-2022-1210 Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 03/26] libxslt: Mark CVE-2022-29824 as not applying Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 04/26] cve-check.bbclass: Added do_populate_sdk[recrdeptask] Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 05/26] cve-check: Add helper for symlink handling Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 06/26] cve-check: Only include installed packages for rootfs manifest Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 07/26] cve-extra-exclusions: Add kernel CVEs Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 08/26] cve-check: Allow warnings to be disabled Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 09/26] linux-yocto/5.15: update to v5.15.37 Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 10/26] linux-yocto/5.10: update to v5.10.113 Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 11/26] linux-yocto/5.15: update to v5.15.38 Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 12/26] linux-yocto/5.10: update to v5.10.114 Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 13/26] libpcre2: upgrade 10.39 -> 10.40 Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 14/26] ncurses: update to patchlevel 20220423 Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 15/26] mesa.inc: package 00-radv-defaults.conf Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 16/26] staging.bbclass: process direct dependencies in deterministic order Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 17/26] libseccomp: Add missing files for ptests Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 18/26] insane.bbclass: make sure to close .patch files Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 19/26] pciutils: avoid lspci conflict with busybox Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 20/26] ovmf: Fix native build with gcc-12 Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 21/26] rust-common: Fix sstate signatures between arm hf and non-hf Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 22/26] rust-common: Ensure sstate signatures have correct dependencues for do_rust_gen_targets Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 23/26] rust-common: Fix for target definitions returning 'NoneType' for arm Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 24/26] rust-common: Drop LLVM_TARGET and simplify Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 25/26] rust-common: Fix native signature dependency issues Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 26/26] gcc: Upgrade to 11.3 release Steve Sakoman
2022-06-10 8:39 ` [kirkstone 00/26] Patch review Sundeep KOKKONDA
2022-06-10 9:12 ` [OE-core] " Martin Jansa
2022-06-10 9:13 ` Martin Jansa
2022-06-10 14:19 ` Steve Sakoman
2022-06-16 2:19 ` Sundeep KOKKONDA
2022-06-16 14:19 ` [OE-core] " Randy MacLeod
2022-06-20 3:09 ` Sundeep KOKKONDA [this message]
2022-06-27 12:12 ` Randy MacLeod
-- strict thread matches above, loose matches on Subject: below --
2022-08-19 2:42 [OE-core][kirkstone " Steve Sakoman
2026-01-20 13:37 Yoann Congal
2026-01-20 19:03 ` Yoann Congal
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4b1d26ac-6fe2-50cf-4cbd-23fcee49b0b8@gmail.com \
--to=sundeep.kokkonda@gmail.com \
--cc=openembedded-core@lists.openembedded.org \
--cc=randy.macleod@windriver.com \
--cc=steve@sakoman.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox