Re: [OE-core] [kirkstone 00/26] Patch review

[Randy MacLeod <randy.macleod@windriver.com>]

[-- Attachment #1: Type: text/plain, Size: 4204 bytes --]

On 2022-06-19 23:09, Sundeep KOKKONDA wrote:
> Hello Randy,
>
> On 16-06-2022 19:49, Randy MacLeod wrote:
>> On 2022-06-15 22:19, Sundeep KOKKONDA wrote:
>>> Hello,
>>>
>>> GCC regression test performed on /Kirkstone/ with gcc-11.3 & 
>>> gcc-11.2. The results are *NOT* identical. Below is test summary.
>>>
>>> *gcc-11.3:*
>>> /                === gcc Summary ===/
>>> /# of expected passes            126552/
>>> /# of unexpected failures        24295/
>>> /# of unexpected successes       77/
>>> /# of expected failures          782/
>>> /# of unresolved testcases       10/
>>> /# of unsupported tests          2760/
>>> //home/bft1/skokkonda/kirkstone/poky/build/sdk/sysroots/x86_64-pokysdk-linux/usr/bin/x86_64-poky-linux/x86_64-poky-linux-gcc 
>>> version 11.3.0 (GCC)/
>>>
>>> *gcc-11.2:*
>>> /                === gcc Summary ===/
>>> /# of expected passes            125933/
>>> /# of unexpected failures        24215/
>>> /# of unexpected successes       76/
>>> /# of expected failures          773/
>>> /# of unresolved testcases       10/
>>> /# of unsupported tests          2779/
>>> //home/bft1/skokkonda/kirkstone-gcc-11.2/poky/build/sdk/sysroots/x86_64-pokysdk-linux/usr/bin/x86_64-poky-linux/x86_64-poky-linux-gcc 
>>> version 11.2.0 (GCC)
>>>
>>>
>>> /
>>
>> /11.3: # of expected passes            126552/
>> //11.2: # of expected passes            125933//
>>
>> So >= 619 additional tests were added and they mostly pass (as you'd 
>> expect).
>> That's good news.
>>
>>
>> /11.3: # of unexpected failures        24295/
>> /11.2: # of unexpected failures        24215/
>> /
>> /
>> /So 80 additional unexpected failures - can you summarize what these 
>> failures are?
>> /
> There are 83 tests failed with GCC-11.3 and out of them most of the 
> failed cases (75 tests) are /execution /tests and the remaining are 
> /test for excess errors/ (Tests with excess messages output to 
> stderr). The Failed tests list is attached. 



>> /Were any of them fixed after the 11.3 tag?
>> /
> Does it mean, do you want me to test on 11.4 and compare?


When it's out, yes certainly, but until then can you take a brief look at:
$ git log --oneline releases/gcc-11.3.0...
and the actual code changes and tell us if any of the
failed tests are issues that others have fixed.

Do you have the test pass/fail/skip/... numbers for 11.2 ?
If so, please compare the 11.3 numbers to them.

>> /
>> Are they related to one part of the toolchain or spread across 
>> various features?
>> Are they reproducible when buiding outside of bitbake?/
> The test failures are from gcc.dg/torture, gcc.c-torture & gcc.target 
> <https://urldefense.com/v3/__http://gcc.target__;!!AjveYdw8EvQ!b_lsS40IeXan8BPrk7uRlgocKqibMQVnhzcDoWw4Y4cAWykDtRGiBACxR9Ii9jcUyww-mrPYHPd2Vn-dS0pMqj_bvdyA_Xop$> 
> testsuites.

Ok, that sounds like the parts of a test suite that might fail!

../Randy


>> /
>> /
>> /The other differences are less significant.
>> If you post similar data in future emails, please do the math for 
>> people!
>> /
>> /
>> /
>> //
>>> gcc-11.3 & gcc-11.2 test summaries are attached.
>>
>> I didn't look at these yet but  thanks for providing them.
>>
>> Since we're not seeing any runtime regression in our Yocto
>> test suite, I suspect that we shouldn't panic but should instead
>> just work to understand, fix, upstream and monitor for future 
>> regressions.
>>
>> ../Randy
>>
>>>
>>>
>>> --
>>> Thanks,
>>> Sundeep K.
>>>
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>> Links: You receive all messages sent to this group.
>>> View/Reply Online (#166997):https://lists.openembedded.org/g/openembedded-core/message/166997
>>> Mute This Topic:https://lists.openembedded.org/mt/91503792/3616765
>>> Group Owner:openembedded-core+owner@lists.openembedded.org
>>> Unsubscribe:https://lists.openembedded.org/g/openembedded-core/unsub  [randy.macleod@windriver.com]
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>
>>
>> -- 
>> # Randy MacLeod
>> # Wind River Linux
>
>

-- 
# Randy MacLeod
# Wind River Linux

[-- Attachment #2: Type: text/html, Size: 9724 bytes --]