Re: [OE-core] [PATCH v6] Rust Oe-Selftest implementation

pgowda cve
2022-06-03T13:07:27Z

Hi Luca,

Regarding the issue
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14818

I ran the Oe-Selftest for Rust as well as completed packages and did
not come across any issues.
a@yow-fedora-builder1 build]$ uname -a
Linux yow-fedora-builder1 5.16.18-200.fc35.x86_64 #1 SMP PREEMPT Mon
Mar 28 14:10:07 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

The log of tests can be seen as follows:-
The command used to run the test was as mentioned in the Bugzilla.
oe-selftest -a -t machine -t toolchain-system -j 15

test_libatomic: PASSED (113.76s)
2022-06-03 08:39:19,084 - oe-selftest - INFO - RESULTS -
gcc.GccLibGompSelfTestSystemEmulated.test_libgomp: PASSED (637.21s)
2022-06-03 08:39:19,084 - oe-selftest - INFO - RESULTS -
gcc.GccLibItmSelfTestSystemEmulated.test_libitm: PASSED (133.31s)
2022-06-03 08:39:19,084 - oe-selftest - INFO - RESULTS -
gcc.GccLibStdCxxSelfTestSystemEmulated.test_libstdcxx: PASSED
(2546.63s)
2022-06-03 08:39:19,084 - oe-selftest - INFO - RESULTS -
gcc.GxxCrossSelfTestSystemEmulated.test_cross_gxx: PASSED (744.56s)
2022-06-03 08:39:19,084 - oe-selftest - INFO - RESULTS -
glibc.GlibcSelfTestSystemEmulated.test_glibc: PASSED (1101.43s)
2022-06-03 08:39:19,085 - oe-selftest - INFO - RESULTS -
meta_ide.MetaIDE.test_meta_ide_can_build_cpio_project: PASSED (54.33s)
2022-06-03 08:39:19,085 - oe-selftest - INFO - RESULTS -
meta_ide.MetaIDE.test_meta_ide_can_compile_c_program: PASSED (0.19s)
2022-06-03 08:39:19,085 - oe-selftest - INFO - RESULTS -
meta_ide.MetaIDE.test_meta_ide_had_installed_meta_ide_support: PASSED
(0.01s)
2022-06-03 08:39:19,085 - oe-selftest - INFO - RESULTS -
runqemu.QemuTest.test_qemu_can_shutdown: PASSED (45.98s)
2022-06-03 08:39:19,085 - oe-selftest - INFO - RESULTS -
rust.RustSelfTestSystemEmulated.test_rust: PASSED (8381.70s)
2022-06-03 08:39:19,085 - oe-selftest - INFO - RESULTS -
gcc.GccLibSspSelfTestSystemEmulated.test_libssp: SKIPPED (3.92s)
2022-06-03 08:39:22,336 - oe-selftest - INFO - SUMMARY:
2022-06-03 08:39:22,336 - oe-selftest - INFO - oe-selftest () - Ran 16
tests in 19613.479s
2022-06-03 08:39:22,336 - oe-selftest - INFO - oe-selftest - OK - All
required tests passed (successes=15, skipped=1, failures=0, errors=0)

I also ran the Rust Oe-Selftest individually and did not find any issues.
Can you please share the environment file so that I can reproduce and
fix the issue?

Thanks,
Pgowda

On Tue, May 31, 2022 at 6:35 PM Luca Ceresoli <luca.ceresoli@bootlin.com> wrote:
>
> Hi Pgowda,
>
> Il giorno Wed, 25 May 2022 09:57:48 +0200
> "Luca Ceresoli via lists.openembedded.org"
> <luca.ceresoli=bootlin.com@lists.openembedded.org> ha scritto:
>
> > Pgowda,
> >
> > Il giorno Tue, 24 May 2022 21:32:58 -0700
> > "Pgowda" <pgowda.cve@gmail.com> ha scritto:
> >
> > > The patch implements Rust testing framework similar to other selftest,
> > > specifically the gcc selftest in OE. It uses the client and server
> > > based method to test the binaries for cross-target on the image.
> > > The test framework is a wrapper around the Rust build system as ./x.py
> > > test.
> > > It tests many functionalities of Rust distribution like tools,
> > > documentation, libraries, packages, tools, Cargo, Crater etc.
> > > Please refer the following link for detailed description of Rust
> > > testing:-
> > > https://rustc-dev-guide.rust-lang.org/tests/intro.html#tool-tests
> > >
> > > To support the rust tests in oe-core, the following functions were
> > > added:-
> > > setup_cargo_environment(): Build bootstrap and some early stage tools.
> > > do_rust_setup_snapshot(): Install the snapshot version of rust
> > > binaries. do_configure(): To generate config.toml
> > > do_compile(): To build "remote-test-server" for qemu target image.
> > >
> > > Approximate Number of Tests Run in the Rust Testsuite :- 18000
> > > Approximate Number of Tests that FAIL in bitbake environment :-
> > > 100-150 Normally majority of the testcases are present in major
> > > folder "test/" It contributes to more than 80% of the testcases
> > > present in Rust test framework. These tests pass as expected on any
> > > Rust versions without much fuss. The tests that fail are of less
> > > important and contribute to less than 2% of the total testcases.
> > > These minor tests are observed to work on some versions and fail on
> > > others. They have to be added, ignored or excluded for different
> > > versions as per the behavior. These tests have been ignored or
> > > excluded in the Rust selftest environment to generate success of
> > > completing the testsuite.
> > >
> > > These tests work in parallel mode even in the skipped test mode as
> > > expected. Although the patch to disable tests is large, it is very
> > > simple in that it only disables tests. When updating to a newer
> > > version of Rust, the patch can usually be ported in a day.
> > >
> > > Signed-off-by: pgowda <pgowda.cve@gmail.com>
> > > Signed-off-by: Vinay Kumar <vinay.m.engg@gmail.com>
> > > ---
> > > meta/conf/distro/include/maintainers.inc | 1 +
> > > meta/lib/oeqa/selftest/cases/rust.py | 57 +++
> > > meta/recipes-devtools/rust/rust-common.inc | 1 -
> > > meta/recipes-devtools/rust/rust-testsuite.inc | 163 +++++++
> > > .../rust-testsuite/rust-oe-selftest.patch | 417
> > > ++++++++++++++++++ .../rust/rust-testsuite_1.60.0.bb |
> > > 3 + 6 files changed, 641 insertions(+), 1 deletion(-)
> > > create mode 100644 meta/lib/oeqa/selftest/cases/rust.py
> > > create mode 100644 meta/recipes-devtools/rust/rust-testsuite.inc
> > > create mode 100644
> > > meta/recipes-devtools/rust/rust-testsuite/rust-oe-selftest.patch
> > > create mode 100644 meta/recipes-devtools/rust/rust-testsuite_1.60.0.bb
> > >
> > > diff --git a/meta/conf/distro/include/maintainers.inc
> > > b/meta/conf/distro/include/maintainers.inc index
> > > 3990d1d507..cecb02b1c6 100644 ---
> > > a/meta/conf/distro/include/maintainers.inc +++
> > > b/meta/conf/distro/include/maintainers.inc @@ -719,6 +719,7 @@
> > > RECIPE_MAINTAINER:pn-rust-crosssdk-${SDK_ARCH}-glibc = "Randy MacLeod
> > > <Randy.Mac
> > > RECIPE_MAINTAINER:pn-rust-cross-canadian-${TRANSLATED_TARGET_ARCH} =
> > > "Randy MacLeod <Randy.MacLeod@windriver.com>"
> > > RECIPE_MAINTAINER:pn-rust-hello-world = "Randy MacLeod
> > > <Randy.MacLeod@windriver.com>" RECIPE_MAINTAINER:pn-rust-llvm =
> > > "Randy MacLeod <Randy.MacLeod@windriver.com>"
> > > +RECIPE_MAINTAINER:pn-rust-testsuite = "Randy MacLeod
> > > <Randy.MacLeod@windriver.com>"
> > > RECIPE_MAINTAINER:pn-rust-tools-cross-canadian-${TRANSLATED_TARGET_ARCH}
> > > = "Randy MacLeod <Randy.MacLeod@windriver.com>"
> > > RECIPE_MAINTAINER:pn-rxvt-unicode = "Unassigned
> > > <unassigned@yoctoproject.org>" RECIPE_MAINTAINER:pn-sato-screenshot =
> > > "Ross Burton <ross.burton@arm.com>" diff --git
> > > a/meta/lib/oeqa/selftest/cases/rust.py
> > > b/meta/lib/oeqa/selftest/cases/rust.py new file mode 100644 index
> > > 0000000000..a37e3e37c5 --- /dev/null +++
> > > b/meta/lib/oeqa/selftest/cases/rust.py @@ -0,0 +1,57 @@ +#
> > > SPDX-License-Identifier: MIT +import os +import subprocess +from
> > > oeqa.core.decorator import OETestTag +from oeqa.core.case import
> > > OEPTestResultTestCase +from oeqa.selftest.case import
> > > OESelftestTestCase +from oeqa.utils.commands import runCmd, bitbake,
> > > get_bb_var, get_bb_vars, runqemu, Command +from oeqa.utils.sshcontrol
> > > import SSHControl + +# Total time taken for testing is of about 2hr
> > > 20min, with PARALLEL_MAKE set to 40 number of jobs. +class
> > > RustSelfTestBase(OESelftestTestCase, OEPTestResultTestCase): +
> > > + def run_check_emulated(self, *args, **kwargs):
> > > + # build remote-test-server before image build
> >
> > Weird indentation here, 7 spaces for level 1, 15 for level 2... v5 had
> > a much cleaner tab-based indentation. Is it possible that you
> > manipulated the patch manually? It's a bad idea, I hurt myself a few
> > times when I used to do that! :) I recommend using plain 'git
> > format-patch' / 'git send-email'.
> >
> > You should resend with a 4-spaces indentation, which is the Python
> > standard. However I'm taking the patch anyway for testing, so it's
> > more efficient if you just wait for the test results before resending,
> > so you can also include any bugfixes (if needed) in your next iteration.
>
> In addition to the above comments, this patch is causing intermittent
> failures, as noted on this bug:
>
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14818
>
> So far it has failed only when building on CentOS and Fedora hosts.
> Hopefully this will help you reproduce and fix the problem.
>
> --
> Luca Ceresoli, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com