From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) by mx.groups.io with SMTP id smtpd.web11.29653.1629749376560005594 for ; Mon, 23 Aug 2021 13:09:37 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@denx.de header.s=phobos-20191101 header.b=joQJQoyf; spf=pass (domain: denx.de, ip: 85.214.62.61, mailfrom: lukma@denx.de) Received: from ktm (85-222-111-42.dynamic.chello.pl [85.222.111.42]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: lukma@denx.de) by phobos.denx.de (Postfix) with ESMTPSA id C980480394; Mon, 23 Aug 2021 22:09:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=denx.de; s=phobos-20191101; t=1629749374; bh=awoS4xMY1L6YNhgqCpMdEUhhW3G3lBNssBogJXyg3Xs=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=joQJQoyfyA8XpbmAc6bYa8C88sdiR8jtWXlD2uFpQSDk8UIxjiydFG+DKO3o4gvQA 9cuOXqYH3KcKo1/U+XgbOjPKIG8Ag7fp0e/F3Hx0lomsUs9i6CMRYXV1GL2xmCo1j1 E6AdIvLma1bJ+XUYdc738FBknDM6+pzKROsPaKumoZX6bwFuZqUNryxkeZAjwdU4jt RQ/Xdk34nvueZe36Pjwmv5pN6cPYkx3qL1o4q8cCkq855ow4vURN0yDqNyoZi4KKIG S7kIUV2SkkCd+5cd2L5Jmp0wCs1ai5z3cFbQ2NOg1H6JoxNgrww4N6Iflt4KH29lAP AZsU//xGoszbA== Date: Mon, 23 Aug 2021 22:09:26 +0200 From: "?ukasz Majewski" To: Khem Raj Cc: openembedded-core@lists.openembedded.org Subject: Re: [OE-core] [PATCH 2/2] glibc: ptest: Add support for running glibc test suite with ptest Message-ID: <20210823220926.37d87121@ktm> In-Reply-To: References: <20210823150853.2817-1-lukma@denx.de> <20210823150853.2817-2-lukma@denx.de> <20210823202414.731c6749@ktm> Organization: denx.de X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 X-Virus-Scanned: clamav-milter 0.103.2 at phobos.denx.de X-Virus-Status: Clean X-Groupsio-MsgNum: 155191 Content-Type: multipart/signed; micalg=pgp-sha512; boundary="Sig_/egxHoZSlxhH=xGPPF0OdrHW"; protocol="application/pgp-signature" --Sig_/egxHoZSlxhH=xGPPF0OdrHW Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Mon, 23 Aug 2021 12:52:44 -0700 Khem Raj wrote: > On Mon, Aug 23, 2021 at 11:24 AM Lukasz Majewski > wrote: >=20 > > Hi Khem, > > =20 > > > On 8/23/21 8:08 AM, ?ukasz Majewski wrote: =20 > > > > This patch introduces new recipe - namely 'glibc-tests', which > > > > builds and installs glibc test suite to OE/Yocto built image. > > > > > > > > It reuses code from already available 'glibc-testsuite' recipe, > > > > which is run with 'bitbake glibc-testsuite -c check' and uses > > > > qemu to execute remotely (via SSH) tests on some emulated > > > > machine. > > > > > > > > This recipe installs eligible tests on some rootfs image. > > > > Afterwards, either all of them or only time related subset, > > > > those tests can be executed on the real hardware, to facilitate > > > > validation of this platform with for example Y2038 problem > > > > compliance. > > > > > > > > By default all tests are executed, with 'ptest-runner > > > > glibc-tests'. To test only time related subset - one needs to > > > > call: cd /usr/lib/glibc-tests/ptest/; rm run-ptest; \ > > > > ln -s run-ptest-time run-ptest; cd -; ptest-runner glibc-tests > > > > > > > > To facilitate debugging, source files are provided by default > > > > with the unstripped debugging symbols. Such approach would > > > > reduce the already complex recipe (as it inherits base glibc > > > > one), so there is no need to also install *-dbg and *-src > > > > packages. =20 > > > > > > > > > does it have to be a separate recipe I wonder if we can have it > > > built as part of glibc itself controlled via ptest knob =20 > > > > I've followed the glibc-testsuite recipe to provide tests for ptest. > > Test creation is similar to it, but doesn't require QEMU run (tests > > are executed on the HW). > > > > Another rationale was to have some kind of "features" separation in > > different file (liked glibc-mtrace), which is easier to maintain and > > review. > > > > Last but not least - separate recipe (and built binaries) allow some > > kind of magic with selection of used test programs (this may be > > useful if one would like to have different sets of tests in > > different packages)=20 >=20 > That=E2=80=99s seems ok I think the names are quite confusing now not tha= t it > was not so much better before but now we have glibc-tests and > glibc-testsuites which do same things but in very different way maybe > glibc-testsuite should be renamed to something like glibc-tests-crosd > or some such I think that glibc-testsuite_2.34.bb shall be renamed to glibc-tests-qemu_2.34.bb as it is more descriptive. Then, glibc-tests_2.34.bb could be left as it is now, as inheriting ptest requires it to be added as 'glibc-tests-ptest' in meta/conf/distro/include/ptest-packagelists.inc (I've tried to rename it to glibc-tests-ptest_2.34.bb, but then it was required to add glibc-tests-ptest-ptest to the ptest-packagelist.inc file, which is a bit clumsy IMHO). >=20 > > =20 > > > =20 > > > > > > > > Signed-off-by: Lukasz Majewski > > > > --- > > > > .../distro/include/ptest-packagelists.inc | 1 + > > > > meta/recipes-core/glibc/glibc-tests_2.34.bb | 101 > > > > ++++++++++++++++++ meta/recipes-core/glibc/glibc/run-ptest-all > > > > | 25 +++++ meta/recipes-core/glibc/glibc/run-ptest-time | 30 > > > > ++++++ 4 files changed, 157 insertions(+) > > > > create mode 100644 meta/recipes-core/glibc/glibc-tests_2.34.bb > > > > create mode 100755 meta/recipes-core/glibc/glibc/run-ptest-all > > > > create mode 100755 > > > > meta/recipes-core/glibc/glibc/run-ptest-time > > > > > > > > diff --git a/meta/conf/distro/include/ptest-packagelists.inc > > > > b/meta/conf/distro/include/ptest-packagelists.inc index > > > > 3872bdc942..e460ad1c8b 100644 --- > > > > a/meta/conf/distro/include/ptest-packagelists.inc +++ > > > > b/meta/conf/distro/include/ptest-packagelists.inc @@ -59,6 > > > > +59,7 @@ PTESTS_FAST =3D "\ slang-ptest \ > > > > wayland-ptest \ > > > > zlib-ptest \ > > > > + glibc-tests-ptest \ > > > > " > > > > PTESTS_FAST:remove:mips64 =3D "qemu-ptest" > > > > PTESTS_PROBLEMS:append:mips64 =3D "qemu-ptest" > > > > diff --git a/meta/recipes-core/glibc/glibc-tests_2.34.bb > > > > b/meta/recipes-core/glibc/glibc-tests_2.34.bb new file mode > > > > 100644 index 0000000000..896809f700 > > > > --- /dev/null > > > > +++ b/meta/recipes-core/glibc/glibc-tests_2.34.bb > > > > @@ -0,0 +1,101 @@ > > > > +require glibc_${PV}.bb > > > > +require glibc-tests.inc > > > > + > > > > +inherit ptest > > > > + > > > > +SRC_URI:append =3D " \ > > > > + file://run-ptest-all file://run-ptest-time \ > > > > +" > > > > + > > > > +SUMMARY =3D "glibc tests to be run with ptest" > > > > + > > > > +# Erase some variables already set by glibc_${PV} > > > > +python __anonymous() { > > > > + # Remove packages provided by glibc build, we only need > > > > a subset of them > > > > + d.setVar("PACKAGES", "${PN}") > > > > + > > > > + d.setVar("PROVIDES", "${PN}") > > > > + d.setVar("RPROVIDES", "${PN} glibc-ptest") > > > > + > > > > + d.setVar("RRECOMMENDS", "") > > > > +} > > > > + > > > > +# Remove any lefovers from original glibc recipe > > > > +RPROVIDES:${PN} =3D "${PN}" > > > > +RRECOMMENDS:${PN} =3D "" > > > > +RDEPENDS:${PN} =3D "sed" > > > > + > > > > +# Just build tests for target - do not run them > > > > +do_check:append () { > > > > + oe_runmake -i check run-built-tests=3Dno > > > > +} > > > > +addtask do_check after do_compile before do_install_ptest_base > > > > + > > > > +glibc_strip_build_directory () { > > > > + # Delete all non executable files from build directory > > > > + find ${B} ! -executable -type f -delete > > > > + > > > > + # Remove build dynamic libraries and links to them as > > > > + # those are already installed in the target device > > > > + find ${B} -type f -name "*.so" -delete > > > > + find ${B} -type l -name "*.so*" -delete > > > > + > > > > + # Remove headers (installed with glibc) > > > > + find ${B} -type f -name "*.h" -delete > > > > + > > > > + find ${B} -type f -name "isomac" -delete > > > > + find ${B} -type f -name "annexc" -delete > > > > +} > > > > + > > > > +do_install_ptest_base () { > > > > + glibc_strip_build_directory > > > > + > > > > + # Install build test programs to the image > > > > + install -d ${D}${PTEST_PATH}/tests/glibc-ptest/ > > > > + cp -r ${B}/* ${D}${PTEST_PATH}/tests/glibc-ptest/ > > > > + > > > > + install -d ${D}${PTEST_PATH} > > > > + cp ${WORKDIR}/run-ptest-* ${D}${PTEST_PATH}/ > > > > + > > > > + # By default run all glibc tests > > > > + cd ${D}${PTEST_PATH}/ > > > > + ln -s run-ptest-all run-ptest > > > > +} > > > > + > > > > +# The datadir directory is required to allow core (and reused) > > > > +# glibc cleanup function to finish correctly, as this directory > > > > +# is not created for ptests > > > > +stash_locale_package_cleanup:prepend () { > > > > + mkdir -p ${PKGD}${datadir} > > > > +} > > > > + > > > > +stash_locale_sysroot_cleanup:prepend () { > > > > + mkdir -p ${SYSROOT_DESTDIR}${datadir} > > > > +} > > > > + > > > > +# Prevent the do_package() task to set 'libc6' prefix > > > > +# for glibc tests related packages > > > > +python populate_packages:prepend () { > > > > + if d.getVar('DEBIAN_NAMES'): > > > > + d.setVar('DEBIAN_NAMES', '') > > > > +} > > > > + > > > > +FILES:${PN} =3D "${PTEST_PATH}/* /usr/src/debug/glibc-tests/*" > > > > + > > > > +EXCLUDE_FROM_SHLIBS =3D "1" > > > > + > > > > +# Install debug data in .debug and sources in /usr/src/debug > > > > +# It is more handy to have _all_ the sources and symbols in one > > > > +# place (package) as this recipe will be used for validation > > > > and +# debugging. > > > > +PACKAGE_DEBUG_SPLIT_STYLE =3D "debug" > > > > + > > > > +# glibc test cases violate by default some Yocto/OE checks > > > > (staticdev, +# textrel) > > > > +# 'debug-files' - add everything (including debug) into one > > > > package +# (no need to install/build *-src > > > > package) +INSANE_SKIP:${PN} +=3D "staticdev textrel debug-files > > > > rpaths" + > > > > +deltask do_stash_locale > > > > +do_install[noexec] =3D "1" > > > > +do_populate_sysroot[noexec] =3D "1" > > > > diff --git a/meta/recipes-core/glibc/glibc/run-ptest-all > > > > b/meta/recipes-core/glibc/glibc/run-ptest-all new file mode > > > > 100755 index 0000000000..47bfe5238c > > > > --- /dev/null > > > > +++ b/meta/recipes-core/glibc/glibc/run-ptest-all > > > > @@ -0,0 +1,25 @@ > > > > +#!/bin/sh > > > > +# ptest script for glibc - to run time related tests to > > > > +# facilitate Y2038 validation > > > > +# Run with 'ptest-runner glibc-tests' > > > > + > > > > +output() { > > > > + retcode=3D$? > > > > + if [ $retcode -eq 0 ] > > > > + then echo "PASS: $i" > > > > + elif [ $retcode -eq 77 ] > > > > + then echo "SKIP: $i" > > > > + else echo "FAIL: $i" > > > > + fi > > > > +} > > > > + > > > > +# Allow altering time on the target > > > > +export GLIBC_TEST_ALLOW_TIME_SETTING=3D"1" > > > > + > > > > +tst=3D$(ls -r ${PWD}/tests/glibc-ptest/*/*) > > > > + > > > > +for i in ${tst} > > > > +do > > > > + $i >/dev/null 2>&1 > > > > + output > > > > +done > > > > diff --git a/meta/recipes-core/glibc/glibc/run-ptest-time > > > > b/meta/recipes-core/glibc/glibc/run-ptest-time new file mode > > > > 100755 index 0000000000..24dedc6e32 > > > > --- /dev/null > > > > +++ b/meta/recipes-core/glibc/glibc/run-ptest-time > > > > @@ -0,0 +1,30 @@ > > > > +#!/bin/sh > > > > +# ptest script for glibc - to run time related tests to > > > > +# facilitate Y2038 validation > > > > +# Run with 'ptest-runner glibc-tests' > > > > + > > > > +output() { > > > > + retcode=3D$? > > > > + if [ $retcode -eq 0 ] > > > > + then echo "PASS: $i" > > > > + elif [ $retcode -eq 77 ] > > > > + then echo "SKIP: $i" > > > > + else echo "FAIL: $i" > > > > + fi > > > > +} > > > > + > > > > +# Allow altering time on the target > > > > +export GLIBC_TEST_ALLOW_TIME_SETTING=3D"1" > > > > + > > > > +tst_time64=3D$(ls -r ${PWD}/tests/glibc-ptest/*/*-time64) > > > > + > > > > +# Remove '-time64' suffix - those tests are also time > > > > +# related > > > > +tst_time_tmp=3D$(sed -e "s/-time64$//" <<< ${tst_time64}) > > > > +tst_time=3D"${tst_time_tmp} ${tst_time64}" > > > > + > > > > +for i in ${tst_time} > > > > +do > > > > + $i >/dev/null 2>&1 > > > > + output > > > > +done > > > > > > > > > > > > > > > > > > > > =20 > > > > > > > > > > Best regards, > > > > Lukasz Majewski > > > > -- > > > > DENX Software Engineering GmbH, Managing Director: Wolfgang > > Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, > > Germany Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: > > lukma@denx.de=20 Best regards, Lukasz Majewski -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de --Sig_/egxHoZSlxhH=xGPPF0OdrHW Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iQEzBAEBCgAdFiEEgAyFJ+N6uu6+XupJAR8vZIA0zr0FAmEkAHYACgkQAR8vZIA0 zr0rHwgAkSIQr0F07EShl9bZ0MEo7ANshzPxlNc5MEmyb8m82D7bFVtFdryQjscH V0ELk3bJsM1QCMzbt0x0uOU1Zh0urkQFOuuX6qGfO8ibGZtL1vYp9xFWsDAJcWx7 md/us+38H9LnzG4jzNNRi5SkHmja+qANi0gDFTTbhYf3ppIDQRp3+u6BkIYYF0ia CwvR7Zooa2SVD0HM3aZ6pXaPVPtTx2zdu2vp1pLBZUia2Sgzw4Eo72VNzs6Jp9Gp kFO3KiErugiyt48jrkBJib26rZ93yjJ1Mapj/Ml7Pd4VUi05g3jzF6HQOup6lw/h LekC9oYUMUhLvrTC1ftt9Caf9e/eWw== =hUJ3 -----END PGP SIGNATURE----- --Sig_/egxHoZSlxhH=xGPPF0OdrHW--