* [PATCHV2 1/3] ptest-cargo.bbclass: create class
2023-04-24 13:04 [PATCHV2 0/3] Create class for building and installing rust tests frederic.martinsons
@ 2023-04-24 13:04 ` frederic.martinsons
2023-04-24 13:04 ` [PATCHV2 2/3] python3-bcrypt: enable build of unit tests frederic.martinsons
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: frederic.martinsons @ 2023-04-24 13:04 UTC (permalink / raw)
To: openembedded-core
From: Frederic Martinsons <frederic.martinsons@gmail.com>
This new class offer the capbility to build rust tests and
find them correctly.
Due to non deterministic name of generated binaries, a custom
parsing of build result must be performed.
See https://github.com/rust-lang/cargo/issues/1924
All rust project will generate a test binary even if there are
not test defined in source code (the binary will just output
that it ran 0 tests)
Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
---
meta/classes-recipe/ptest-cargo.bbclass | 129 ++++++++++++++++++++++++
1 file changed, 129 insertions(+)
create mode 100644 meta/classes-recipe/ptest-cargo.bbclass
diff --git a/meta/classes-recipe/ptest-cargo.bbclass b/meta/classes-recipe/ptest-cargo.bbclass
new file mode 100644
index 0000000000..f26d4225d3
--- /dev/null
+++ b/meta/classes-recipe/ptest-cargo.bbclass
@@ -0,0 +1,129 @@
+inherit cargo ptest
+
+CARGO_TEST_BINARIES_FILES ?= "${B}/test_binaries_list"
+
+# sadly generated test binary have no deterministic names (https://github.com/rust-lang/cargo/issues/1924)
+# which force us to parse the cargo output in json format to find those test binaries
+python do_compile_ptest_cargo() {
+ import subprocess
+ import json
+
+ cargo = bb.utils.which(d.getVar("PATH"), d.getVar("CARGO", True))
+ cargo_build_flags = d.getVar("CARGO_BUILD_FLAGS", True)
+ rust_flags = d.getVar("RUSTFLAGS", True)
+ manifest_path = d.getVar("MANIFEST_PATH", True)
+
+ env = os.environ.copy()
+ env['RUSTFLAGS'] = rust_flags
+ cmd = f"{cargo} build --tests --message-format json {cargo_build_flags}"
+ bb.note(f"Building tests with cargo ({cmd})")
+
+ try:
+ proc = subprocess.Popen(cmd, shell=True, env=env, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
+ except subprocess.CalledProcessError as e:
+ bb.fatal(f"Cannot build test with cargo: {e}")
+
+ lines = []
+ for line in proc.stdout:
+ data = line.decode('utf-8').strip('\n')
+ lines.append(data)
+ bb.note(data)
+ proc.communicate()
+ if proc.returncode != 0:
+ bb.fatal(f"Unable to compile test with cargo, '{cmd}' failed")
+
+ # Definition of the format: https://doc.rust-lang.org/cargo/reference/external-tools.html#json-messages
+ test_bins = []
+ for line in lines:
+ try:
+ data = json.loads(line)
+ except json.JSONDecodeError:
+ # skip lines that are not a json
+ pass
+ else:
+ try:
+ # Filter the test packages coming from the current manifest
+ current_manifest_path = os.path.normpath(data['manifest_path'])
+ project_manifest_path = os.path.normpath(manifest_path)
+ if current_manifest_path == project_manifest_path:
+ if data['target']['test'] or data['target']['doctest'] and data['executable']:
+ test_bins.append(data['executable'])
+ except KeyError as e:
+ # skip lines that do not meet the requirements
+ pass
+
+ # All rust project will genrate at least one unit test binary
+ # It will just run a test suite with 0 tests if the project didn't define some
+ # So it is not expected to have an empty list here
+ if not test_bins:
+ bb.fatal("Unable to find any test binaries")
+
+ cargo_test_binaries_file = d.getVar('CARGO_TEST_BINARIES_FILES', True)
+ bb.note(f"Found {len(test_bins)} tests, write their path into {cargo_test_binaries_file}")
+ with open(cargo_test_binaries_file, "w") as f:
+ for test_bin in test_bins:
+ f.write(f"{test_bin}\n")
+
+}
+
+python do_install_ptest_cargo() {
+ import shutil
+
+ dest_dir = d.getVar("D", True)
+ pn = d.getVar("PN", True)
+ ptest_path = d.getVar("PTEST_PATH", True)
+ cargo_test_binaries_file = d.getVar('CARGO_TEST_BINARIES_FILES', True)
+
+ ptest_dir = os.path.join(dest_dir, ptest_path.lstrip('/'))
+ os.makedirs(ptest_dir, exist_ok=True)
+
+ test_bins = []
+ with open(cargo_test_binaries_file, "r") as f:
+ for line in f.readlines():
+ test_bins.append(line.strip('\n'))
+
+ test_paths = []
+ for test_bin in test_bins:
+ shutil.copy2(test_bin, ptest_dir)
+ test_paths.append(os.path.join(ptest_path, os.path.basename(test_bin)))
+
+ ptest_script = os.path.join(ptest_dir, "run-ptest")
+ if os.path.exists(ptest_script):
+ with open(ptest_script, "a") as f:
+ f.write(f"\necho \"\"\n")
+ f.write(f"echo \"## starting to run rust tests ##\"\n")
+ for test_path in test_paths:
+ f.write(f"{test_path}\n")
+ else:
+ with open(ptest_script, "a") as f:
+ f.write("#!/bin/sh\n")
+ for test_path in test_paths:
+ f.write(f"{test_path}\n")
+ os.chmod(ptest_script, 0o755)
+
+ # this is chown -R root:root ${D}${PTEST_PATH}
+ for root, dirs, files in os.walk(ptest_dir):
+ for d in dirs:
+ shutil.chown(os.path.join(root, d), "root", "root")
+ for f in files:
+ shutil.chown(os.path.join(root, f), "root", "root")
+}
+
+do_install_ptest_cargo[dirs] = "${B}"
+do_install_ptest_cargo[doc] = "Create or update the run-ptest script with rust test binaries generated"
+do_compile_ptest_cargo[dirs] = "${B}"
+do_compile_ptest_cargo[doc] = "Generate rust test binaries through cargo"
+
+addtask compile_ptest_cargo after do_compile before do_compile_ptest_base
+addtask install_ptest_cargo after do_install_ptest_base before do_package
+
+python () {
+ if not bb.data.inherits_class('native', d) and not bb.data.inherits_class('cross', d):
+ d.setVarFlag('do_install_ptest_cargo', 'fakeroot', '1')
+ d.setVarFlag('do_install_ptest_cargp', 'umask', '022')
+
+ # Remove all '*ptest_cargo' tasks when ptest is not enabled
+ if not(d.getVar('PTEST_ENABLED') == "1"):
+ for i in ['do_compile_ptest_cargo', 'do_install_ptest_cargo']:
+ bb.build.deltask(i, d)
+}
--
2.34.1
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCHV2 3/3] zvariant: add ptest feature for zvariant test suite
2023-04-24 13:04 [PATCHV2 0/3] Create class for building and installing rust tests frederic.martinsons
2023-04-24 13:04 ` [PATCHV2 1/3] ptest-cargo.bbclass: create class frederic.martinsons
2023-04-24 13:04 ` [PATCHV2 2/3] python3-bcrypt: enable build of unit tests frederic.martinsons
@ 2023-04-24 13:04 ` frederic.martinsons
2023-04-26 21:10 ` [OE-core] [PATCHV2 0/3] Create class for building and installing rust tests Luca Ceresoli
3 siblings, 0 replies; 6+ messages in thread
From: frederic.martinsons @ 2023-04-24 13:04 UTC (permalink / raw)
To: openembedded-core
From: Frederic Martinsons <frederic.martinsons@gmail.com>
Below is the output of run-ptest script under qemu
root@qemux86-64:~# /usr/lib/zvariant/ptest/run-ptest
running 37 tests
test framing_offset_size::tests::framing_offset_size_bump ... ok$<2>
test owned_value::tests::from_value ... ok$<2>
test object_path::unit::owned_from_reader ... ok$<2>
test str::tests::from_string ... ok$<2>
test signature::tests::signature_slicing ... ok$<2>
test str::tests::test_ordering ... ok$<2>
test owned_value::tests::map_conversion ... ok$<2>
test owned_value::tests::serde ... ok$<2>
test tests::enums ... ok$<2>
test tests::derive ... ok$<2>
test tests::f64_value ... ok$<2>
test tests::i16_value ... ok$<2>
test tests::fd_value ... ok$<2>
test tests::i32_value ... ok$<2>
test tests::i8_value ... ok$<2>
test tests::i64_value ... ok$<2>
test tests::ip_addr ... ok$<2>
test tests::issue_59 ... ok$<2>
test tests::issue_99 ... ok$<2>
test tests::array_value ... ok$<2>
test tests::object_path_value ... ok$<2>
test tests::dict_value ... ok$<2>
test tests::signature_value ... ok$<2>
test tests::serialized_size ... ok$<2>
test tests::struct_byte_array ... ok$<2>
test tests::struct_ref ... ok$<2>
test tests::str_value ... ok$<2>
test tests::option_value ... ok$<2>
test tests::struct_value ... ok$<2>
test tests::struct_with_hashmap ... ok$<2>
test tests::u16_value ... ok$<2>
test tests::u32_value ... ok$<2>
test tests::unit ... ok$<2>
test tests::u8_value ... ok$<2>
test tests::unit_fds ... ok$<2>
test tests::value_value ... ok$<2>
test tests::recursion_limits ... ok$<2>
test result: ok$<2>. 37 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.21s
root@qemux86-64:~#
Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
---
.../recipes-extended/zvariant/zvariant_3.12.0.bb | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/meta-selftest/recipes-extended/zvariant/zvariant_3.12.0.bb b/meta-selftest/recipes-extended/zvariant/zvariant_3.12.0.bb
index 4285d11b72..36f2a07ecf 100644
--- a/meta-selftest/recipes-extended/zvariant/zvariant_3.12.0.bb
+++ b/meta-selftest/recipes-extended/zvariant/zvariant_3.12.0.bb
@@ -7,6 +7,8 @@ HOMEPAGE = "https://gitlab.freedesktop.org/dbus/zbus/"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://LICENSE;md5=b377b220f43d747efdec40d69fcaa69d"
+DEPENDS += "${@bb.utils.contains('DISTRO_FEATURES', 'ptest', 'glib-2.0', '', d)}"
+
SRC_URI = " \
git://gitlab.freedesktop.org/dbus/zbus;protocol=https;branch=main;subpath=zvariant \
file://0001-Tweak-zvariant-crate-config.patch;striplevel=2 \
@@ -21,10 +23,15 @@ python do_clean_lic_file_symlink() {
addtask clean_lic_file_symlink after do_unpack before do_patch
-inherit cargo cargo-update-recipe-crates
+inherit ptest-cargo pkgconfig cargo-update-recipe-crates
# Remove this when the recipe is reproducible
EXCLUDE_FROM_WORLD = "1"
require ${BPN}-crates.inc
require ${BPN}-git-crates.inc
+
+# zvariant is an indermediate product for the zbus project
+# and so, it provided only a static lib (rlib) which fall only
+# in -dev package
+ALLOW_EMPTY:${PN} = "1"
--
2.34.1
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [OE-core] [PATCHV2 0/3] Create class for building and installing rust tests
2023-04-24 13:04 [PATCHV2 0/3] Create class for building and installing rust tests frederic.martinsons
` (2 preceding siblings ...)
2023-04-24 13:04 ` [PATCHV2 3/3] zvariant: add ptest feature for zvariant test suite frederic.martinsons
@ 2023-04-26 21:10 ` Luca Ceresoli
2023-04-27 4:42 ` Frédéric Martinsons
3 siblings, 1 reply; 6+ messages in thread
From: Luca Ceresoli @ 2023-04-26 21:10 UTC (permalink / raw)
To: Frederic Martinsons; +Cc: openembedded-core
Hello Frederic,
On Mon, 24 Apr 2023 15:04:10 +0200
"Frederic Martinsons" <frederic.martinsons@gmail.com> wrote:
> From: Frederic Martinsons <frederic.martinsons@gmail.com>
>
> This brings the possibility to use this class to build and ship
> unit tests of rust projects, the class also create (or modified)
> standard run-ptest script to run the generated rust test suite.
>
> It has been tested successfully with core-image-sato under qemu
> for zvariant-ptest and python3-bcrypt-ptest (though the last one
> didn't define any unit tests).
>
> Note that I tried to do the same with python3-cryptography but fail
> to build the test suite and I don't know how to do it with the rust
> extension module build by python setuptools. There must be some kind
> of way for doing that so maybe someone will put some work in it (
> because python3-cryptography rust extension do have unit tests).
>
> Moreover, in the class, I didn't manage to share data between
> do_compile_ptest_base and do_install_ptest_base cleanly (I tried
> to d.setVar in compile and d.getVar in install but it seems that
> the data store doens't recognize my new variable) so I used a file
> for that. I'm sure there is a clever way for doing that, so feel
> free to tell me.
A build error appeared on the autobuilders with these patches applied:
File "/home/pokybuild/yocto-worker/oe-selftest-armhost/build/meta/lib/oeqa/selftest/cases/devtool.py", line 879, in test_devtool_modify_git_crates_subpath
self._test_recipe_contents(recipefile, {}, ['cargo'])
File "/home/pokybuild/yocto-worker/oe-selftest-armhost/build/meta/lib/oeqa/selftest/cases/devtool.py", line 169, in _test_recipe_contents
self.assertIn(inherit, inherits, 'Missing inherit of %s' % inherit)
File "/usr/lib/python3.8/unittest/case.py", line 1179, in assertIn
self.fail(self._formatMessage(msg, standardMsg))
File "/usr/lib/python3.8/unittest/case.py", line 753, in fail
raise self.failureException(msg)
AssertionError: 'cargo' not found in {'pkgconfig', 'cargo-update-recipe-crates', 'ptest-cargo'} : Missing inherit of cargo
Log:
https://autobuilder.yoctoproject.org/typhoon/#/builders/127/builds/1321/steps/14/logs/stdio
Best regards,
Luca
--
Luca Ceresoli, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 6+ messages in thread