RE: [OE-core] [kirkstone] Binutils, Glibc: Updated with latest commits

[sundeep.kokkonda@gmail.com]

Hello Richard,

> -----Original Message-----
> From: Richard Purdie <richard.purdie@linuxfoundation.org>
> Sent: 04 July 2022 14:20
> To: Sundeep KOKKONDA <sundeep.kokkonda@gmail.com>; openembedded-
> core@lists.openembedded.org
> Subject: Re: [OE-core] [kirkstone] Binutils, Glibc: Updated with latest commits
> 
> On Mon, 2022-07-04 at 00:51 -0700, Sundeep KOKKONDA wrote:
> > The regression test for Binutils and Glibc is performed.
> >
> > 1) Binutils test shows no regression after update.
> >
> > 2) Glibc test result:
> > Test summary before glibc update on Kirkstone:
> >     204 FAIL
> >    4320 PASS
> >      43 UNSUPPORTED
> >      12 XFAIL
> >       6 XPASS
> >
> > Test summary after glibc update on Kirkstone:
> >     204 FAIL
> >    4339 PASS
> >      41 UNSUPPORTED
> >      12 XFAIL
> >       6 XPASS
> >
> > Test results analysis:
> > Total 4585 test cases before Glibc update & Total 4602 test cases
> > after Glibc update.
> > 17 new test cases added (out of those 16 PASS & 1 FAIL)
> > 12 new FAIL test cases after glibc update
> > 14 new PASS test cases after glibc update
> 
> Do we know why some of the failing test cass now pass and why some of the
> passing test cases now fail? Is this some issue with the way we're testing things?
> I'm wondering if this is "normal" noise between test runs and if there is some
> underlying instability or glibc changes do cause changes in pass/fails of tests like
> this in general?


Hello Richard,

The test results are impacted by Execution procedure OR by Changes in glibc.

Execution procedure:
I tested the glibc with the cross-compiler method given in https://sourceware.org/glibc/wiki/Testing/Testsuite for the aarch64 architecture on qemu.
However, based on our past test experience, we're following a few additional steps like copying the 'glibc-locale/2.28-r0/image/usr/*' to booted qemu image /usr/ directory (This step is not mentioned the glibc wiki) etc.
Also, the glibc test is aborted when the -Werror=format-security flag is used and to get the test done we've removed this error flag and compiled glibc sources.

Since the glibc test not performed very frequently, I am not sure the additional steps what we followed are obsolete or not. I am now working on adhering to the test procedure mentioned in the glibc test wiki and will see the results.

Changes in glibc:
To comment on the code change impact on the test failures I've to look into each failed test case by case and what are the changes in glibc and how it is impacting the test.


I'll keep you posted on this and since the glibc test execution is taking a lot of time my response may be delayed.



Thanks,
Sundeep K.


> 
> Cheers,
> 
> Richard
> 
> 
>