On 2022-06-15 22:19, Sundeep KOKKONDA wrote:
Hello,

GCC regression test performed on Kirkstone with gcc-11.3 & gcc-11.2. The results are NOT identical. Below is test summary.

gcc-11.3:
                === gcc Summary ===
 
# of expected passes            126552
# of unexpected failures        24295
# of unexpected successes       77
# of expected failures          782
# of unresolved testcases       10
# of unsupported tests          2760
/home/bft1/skokkonda/kirkstone/poky/build/sdk/sysroots/x86_64-pokysdk-linux/usr/bin/x86_64-poky-linux/x86_64-poky-linux-gcc  version 11.3.0 (GCC)

gcc-11.2:
                === gcc Summary ===
 
# of expected passes            125933
# of unexpected failures        24215
# of unexpected successes       76
# of expected failures          773
# of unresolved testcases       10
# of unsupported tests          2779
/home/bft1/skokkonda/kirkstone-gcc-11.2/poky/build/sdk/sysroots/x86_64-pokysdk-linux/usr/bin/x86_64-poky-linux/x86_64-poky-linux-gcc  version 11.2.0 (GCC)


11.3: # of expected passes            126552
11.2: # of expected passes            125933

So >= 619 additional tests were added and they mostly pass (as you'd expect).
That's good news.


11.3: # of unexpected failures        24295
11.2: # of unexpected failures        24215

So 80 additional unexpected failures - can you summarize what these failures are?
Were any of them fixed after the 11.3 tag?

Are they related to one part of the toolchain or spread across various features?
Are they reproducible when buiding outside of bitbake?

The other differences are less significant.
If you post similar data in future emails, please do the math for people!

gcc-11.3 & gcc-11.2 test summaries are attached.

I didn't look at these yet but  thanks for providing them.

Since we're not seeing any runtime regression in our Yocto
test suite, I suspect that we shouldn't panic but should instead
just work to understand, fix, upstream and monitor for future regressions.

../Randy



--
Thanks,
Sundeep K.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166997): https://lists.openembedded.org/g/openembedded-core/message/166997
Mute This Topic: https://lists.openembedded.org/mt/91503792/3616765
Group Owner: openembedded-core+owner@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [randy.macleod@windriver.com]
-=-=-=-=-=-=-=-=-=-=-=-


-- 
# Randy MacLeod
# Wind River Linux