Re: [OE-core] [PATCH v13] Rust Oe-Selftest implementation

[Luca Ceresoli <luca.ceresoli@bootlin.com>]

Hello Yash,

On Tue, 18 Apr 2023 14:40:13 +0530
"Yash Shinde" <yashinde145@gmail.com> wrote:

> The patch implements Rust testing framework similar to other selftest,
> specifically the gcc selftest in OE. It uses the client and server
> based method to test the binaries for cross-target on the image.
> The test framework is a wrapper around the Rust build system as ./x.py
> test. It tests many functionalities of Rust distribution like tools,
> documentation, libraries, packages, tools, Cargo, Crater etc.
> Please refer the following link for detailed description of Rust
> testing:-
> https://rustc-dev-guide.rust-lang.org/tests/intro.html#tool-tests
> 
> To support the rust tests in oe-core, the following functions were
> added:-
> setup_cargo_environment(): Build bootstrap and some early stage tools.
> do_rust_setup_snapshot(): Install the snapshot version of rust binaries.
> do_configure(): To generate config.toml
> do_compile(): To build "remote-test-server" for qemu target image.
> 
> Approximate Number of Tests Run in the Rust Testsuite :- 18000
> Approximate Number of Tests that FAIL in bitbake environment :- 100-150
> Normally majority of the testcases are present in major folder "test/"
> It contributes to more than 80% of the testcases present in Rust test
> framework. These tests pass as expected on any Rust versions without
> much fuss. The tests that fail are of less important and contribute to
> less than 2% of the total testcases. These minor tests are observed to
> work on some versions and fail on others. They have to be added, ignored
> or excluded for different versions as per the behavior.
> These tests have been ignored or excluded in the Rust selftest
> environment to generate success of completing the testsuite.
> 
> These tests work in parallel mode even in the skipped test mode as
> expected. Although the patch to disable tests is large, it is very simple
> in that it only disables tests. When updating to a newer version of Rust,
> the patch can usually be ported in a day.
> 
> Tested for X86, X86-64, ARM, ARM64 and MIPS64 on CentOS release 6.10
> 
> Signed-off-by: pgowda <pgowda.cve@gmail.com>
> Signed-off-by: Vinay Kumar <vinay.m.engg@gmail.com>
> Signed-off-by: Yash Shinde <yashinde145@gmail.com>

Testing with this patch is causing a lot of failures. Here are some
logs:

https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/6944/steps/12/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/97/builds/6344/steps/16/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/100/builds/4337/steps/12/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/6998/steps/11/logs/stdio

And here's the entire a-full build in case you need more info:

https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/5194

Best regards,
Luca

-- 
Luca Ceresoli, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com