From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0315C433EF for ; Tue, 31 May 2022 13:05:37 +0000 (UTC) Received: from relay12.mail.gandi.net (relay12.mail.gandi.net [217.70.178.232]) by mx.groups.io with SMTP id smtpd.web11.49100.1654002334193617443 for ; Tue, 31 May 2022 06:05:34 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@bootlin.com header.s=gm1 header.b=RlCiy6Cf; spf=pass (domain: bootlin.com, ip: 217.70.178.232, mailfrom: luca.ceresoli@bootlin.com) Received: (Authenticated sender: luca.ceresoli@bootlin.com) by mail.gandi.net (Postfix) with ESMTPSA id 6F0AF20000A; Tue, 31 May 2022 13:05:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=gm1; t=1654002332; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eOhKNE8xSy/rXOY7DnO9CRuAbvy65CQ7vzfd6WybyNk=; b=RlCiy6CfT4zN68bZk34trYZVHGPVEcJAqOXFsGAsRm+xHZdLE4Z2iFIuFAX3+AYnKNc0pL NHvUm8OMGuMaiNa7DtIjl0IR598fS6aRmnXMq/mw3KZi8jJPXFpzXl/Z18UDnfRy2UsPo/ 03omuPVdv0hZ/oELZDgFaEZmWPZBkCTq0Slxvee2DA2F4c0aRtvILdxr1mAAiYvJ9eLP3o iPGTinwt/q0mlBOA0FA9CL27UyiWnyMOAti5USdZsoVm/UtfaQVNRLvBGGSe5MWRHMSkTi +OviZcp5VtItmxosHy67GKkW+kXD4igvjiPRTz+/1j8BxEYIctO6g+mG6HhqAA== Date: Tue, 31 May 2022 15:05:29 +0200 From: Luca Ceresoli To: "Luca Ceresoli via lists.openembedded.org" Cc: luca.ceresoli@bootlin.com, "Pgowda" , openembedded-core@lists.openembedded.org, richard.purdie@linuxfoundation.org, rwmacleod@gmail.com, alex.kanavin@gmail.com, vinay.m.engg@gmail.com Subject: Re: [OE-core] [PATCH v6] Rust Oe-Selftest implementation Message-ID: <20220531150529.705390b9@melee> In-Reply-To: <16F24A16BC7F056F.12338@lists.openembedded.org> References: <20220524225603.50d60963@melee> <20220525043258.8949-1-pgowda.cve@gmail.com> <16F24A16BC7F056F.12338@lists.openembedded.org> Organization: Bootlin X-Mailer: Claws Mail 3.17.5 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 31 May 2022 13:05:37 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/166310 Hi Pgowda, Il giorno Wed, 25 May 2022 09:57:48 +0200 "Luca Ceresoli via lists.openembedded.org" ha scritto: > Pgowda, > > Il giorno Tue, 24 May 2022 21:32:58 -0700 > "Pgowda" ha scritto: > > > The patch implements Rust testing framework similar to other selftest, > > specifically the gcc selftest in OE. It uses the client and server > > based method to test the binaries for cross-target on the image. > > The test framework is a wrapper around the Rust build system as ./x.py > > test. > > It tests many functionalities of Rust distribution like tools, > > documentation, libraries, packages, tools, Cargo, Crater etc. > > Please refer the following link for detailed description of Rust > > testing:- > > https://rustc-dev-guide.rust-lang.org/tests/intro.html#tool-tests > > > > To support the rust tests in oe-core, the following functions were > > added:- > > setup_cargo_environment(): Build bootstrap and some early stage tools. > > do_rust_setup_snapshot(): Install the snapshot version of rust > > binaries. do_configure(): To generate config.toml > > do_compile(): To build "remote-test-server" for qemu target image. > > > > Approximate Number of Tests Run in the Rust Testsuite :- 18000 > > Approximate Number of Tests that FAIL in bitbake environment :- > > 100-150 Normally majority of the testcases are present in major > > folder "test/" It contributes to more than 80% of the testcases > > present in Rust test framework. These tests pass as expected on any > > Rust versions without much fuss. The tests that fail are of less > > important and contribute to less than 2% of the total testcases. > > These minor tests are observed to work on some versions and fail on > > others. They have to be added, ignored or excluded for different > > versions as per the behavior. These tests have been ignored or > > excluded in the Rust selftest environment to generate success of > > completing the testsuite. > > > > These tests work in parallel mode even in the skipped test mode as > > expected. Although the patch to disable tests is large, it is very > > simple in that it only disables tests. When updating to a newer > > version of Rust, the patch can usually be ported in a day. > > > > Signed-off-by: pgowda > > Signed-off-by: Vinay Kumar > > --- > > meta/conf/distro/include/maintainers.inc | 1 + > > meta/lib/oeqa/selftest/cases/rust.py | 57 +++ > > meta/recipes-devtools/rust/rust-common.inc | 1 - > > meta/recipes-devtools/rust/rust-testsuite.inc | 163 +++++++ > > .../rust-testsuite/rust-oe-selftest.patch | 417 > > ++++++++++++++++++ .../rust/rust-testsuite_1.60.0.bb | > > 3 + 6 files changed, 641 insertions(+), 1 deletion(-) > > create mode 100644 meta/lib/oeqa/selftest/cases/rust.py > > create mode 100644 meta/recipes-devtools/rust/rust-testsuite.inc > > create mode 100644 > > meta/recipes-devtools/rust/rust-testsuite/rust-oe-selftest.patch > > create mode 100644 meta/recipes-devtools/rust/rust-testsuite_1.60.0.bb > > > > diff --git a/meta/conf/distro/include/maintainers.inc > > b/meta/conf/distro/include/maintainers.inc index > > 3990d1d507..cecb02b1c6 100644 --- > > a/meta/conf/distro/include/maintainers.inc +++ > > b/meta/conf/distro/include/maintainers.inc @@ -719,6 +719,7 @@ > > RECIPE_MAINTAINER:pn-rust-crosssdk-${SDK_ARCH}-glibc = "Randy MacLeod > > > RECIPE_MAINTAINER:pn-rust-cross-canadian-${TRANSLATED_TARGET_ARCH} = > > "Randy MacLeod " > > RECIPE_MAINTAINER:pn-rust-hello-world = "Randy MacLeod > > " RECIPE_MAINTAINER:pn-rust-llvm = > > "Randy MacLeod " > > +RECIPE_MAINTAINER:pn-rust-testsuite = "Randy MacLeod > > " > > RECIPE_MAINTAINER:pn-rust-tools-cross-canadian-${TRANSLATED_TARGET_ARCH} > > = "Randy MacLeod " > > RECIPE_MAINTAINER:pn-rxvt-unicode = "Unassigned > > " RECIPE_MAINTAINER:pn-sato-screenshot = > > "Ross Burton " diff --git > > a/meta/lib/oeqa/selftest/cases/rust.py > > b/meta/lib/oeqa/selftest/cases/rust.py new file mode 100644 index > > 0000000000..a37e3e37c5 --- /dev/null +++ > > b/meta/lib/oeqa/selftest/cases/rust.py @@ -0,0 +1,57 @@ +# > > SPDX-License-Identifier: MIT +import os +import subprocess +from > > oeqa.core.decorator import OETestTag +from oeqa.core.case import > > OEPTestResultTestCase +from oeqa.selftest.case import > > OESelftestTestCase +from oeqa.utils.commands import runCmd, bitbake, > > get_bb_var, get_bb_vars, runqemu, Command +from oeqa.utils.sshcontrol > > import SSHControl + +# Total time taken for testing is of about 2hr > > 20min, with PARALLEL_MAKE set to 40 number of jobs. +class > > RustSelfTestBase(OESelftestTestCase, OEPTestResultTestCase): + > > + def run_check_emulated(self, *args, **kwargs): > > + # build remote-test-server before image build > > Weird indentation here, 7 spaces for level 1, 15 for level 2... v5 had > a much cleaner tab-based indentation. Is it possible that you > manipulated the patch manually? It's a bad idea, I hurt myself a few > times when I used to do that! :) I recommend using plain 'git > format-patch' / 'git send-email'. > > You should resend with a 4-spaces indentation, which is the Python > standard. However I'm taking the patch anyway for testing, so it's > more efficient if you just wait for the test results before resending, > so you can also include any bugfixes (if needed) in your next iteration. In addition to the above comments, this patch is causing intermittent failures, as noted on this bug: https://bugzilla.yoctoproject.org/show_bug.cgi?id=14818 So far it has failed only when building on CentOS and Fedora hosts. Hopefully this will help you reproduce and fix the problem. -- Luca Ceresoli, Bootlin Embedded Linux and Kernel engineering https://bootlin.com