[PATCH 0/3] Create class for building rust unit test

[PATCH 0/3] Create class for building rust unit test

[frederic.martinsons]

From: Frederic Martinsons <frederic.martinsons@gmail.com>

This brings the possibility to use this class to build and ship
unit tests of rust projects, the class also create (or modified)
standard run-ptest script to run the generated rust test suite.

It has been tested successfully with core-image-sato under qemu
for zvariant-ptest and python3-bcrypt-ptest (though the last one
didn't define any unit tests).

Note that I tried to do the same with python3-cryptography but fail
to build the test suite and I don't know how to do it with the rust
extension module build by python setuptools. There must be some kind
of way for doing that so maybe someone will put some work in it (
because python3-cryptography rust extension do have unit tests).

Moreover, in the class, I didn't manage to share data between
do_compile_ptest_base and do_install_ptest_base cleanly (I tried
to d.setVar in compile and d.getVar in install but it seems that
the data store doens't recognize my new variable) so I used a file
for that. I'm sure there is a clever way for doing that, so feel
free to tell me.

The following changes since commit 45a8bb6e4676899d40525e7d5ad1c1ddefee3185:

  apt-util: Fix ptest on musl (2023-04-20 11:56:03 +0100)

are available in the Git repository at:

  https://gitlab.com/fmartinsons/openembedded-core cargo-add-ptest

Frederic Martinsons (3):
  ptest-cargo.bbclass: create class
  python3-bcrypt: enable build of unit tests
  zvariant: add ptest feature for zvariant test suite

 .../zvariant/zvariant_3.12.0.bb               |  11 +-
 meta/classes-recipe/ptest-cargo.bbclass       | 108 ++++++++++++++++++
 .../python/python3-bcrypt_4.0.1.bb            |   4 +-
 3 files changed, 121 insertions(+), 2 deletions(-)
 create mode 100644 meta/classes-recipe/ptest-cargo.bbclass

-- 
2.34.1

[PATCH 1/3] ptest-cargo.bbclass: create class

[frederic.martinsons]

From: Frederic Martinsons <frederic.martinsons@gmail.com>

This new class offer the capbility to build rust tests and
find them correctly.
Due to non deterministic name of generated binaries, a custom
parsing of build result must be performed.
See https://github.com/rust-lang/cargo/issues/1924

All rust project will generate a test binary even if there are
not test defined in source code (the binary will just output
that it ran 0 tests)

Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
---
 meta/classes-recipe/ptest-cargo.bbclass | 108 ++++++++++++++++++++++++
 1 file changed, 108 insertions(+)
 create mode 100644 meta/classes-recipe/ptest-cargo.bbclass

diff --git a/meta/classes-recipe/ptest-cargo.bbclass b/meta/classes-recipe/ptest-cargo.bbclass
new file mode 100644
index 0000000000..a76b06b46a
--- /dev/null
+++ b/meta/classes-recipe/ptest-cargo.bbclass
@@ -0,0 +1,108 @@
+inherit cargo ptest
+
+CARGO_TEST_BINARIES_FILES ?= "${B}/test_binaries_list"
+
+# sadly generated test binary have no deterministic names (https://github.com/rust-lang/cargo/issues/1924)
+# which force us to parse the cargo output in json format to find those test binaries
+python do_compile_ptest_base() {
+    import subprocess
+    import json
+
+    cargo = bb.utils.which(d.getVar("PATH"), d.getVar("CARGO", True))
+    cargo_build_flags = d.getVar("CARGO_BUILD_FLAGS", True)
+    rust_flags = d.getVar("RUSTFLAGS", True)
+    manifest_path = d.getVar("MANIFEST_PATH", True)
+
+    env = os.environ.copy()
+    env['RUSTFLAGS'] = rust_flags
+    cmd = f"{cargo} build --tests --message-format json {cargo_build_flags}"
+    bb.note(f"Building tests with cargo ({cmd})")
+
+    try:
+        proc = subprocess.Popen(cmd, shell=True, env=env, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
+    except subprocess.CalledProcessError as e:
+        bb.fatal(f"Cannot build test with cargo: {e}")
+
+    lines = []
+    for line in proc.stdout:
+        data = line.decode('utf-8').strip('\n')
+        lines.append(data)
+        bb.note(data)
+    proc.communicate()
+    if proc.returncode != 0:
+        bb.fatal(f"Unable to compile test with cargo, '{cmd}' failed")
+
+    # Definition of the format: https://doc.rust-lang.org/cargo/reference/external-tools.html#json-messages
+    test_bins = []
+    for line in lines:
+        try:
+            data = json.loads(line)
+        except json.JSONDecodeError:
+            # skip lines that are not a json
+            pass
+        else:
+            try:
+                # Filter the test packages coming from the current manifest
+                current_manifest_path = os.path.normpath(data['manifest_path'])
+                project_manifest_path = os.path.normpath(manifest_path)
+                if current_manifest_path == project_manifest_path:
+                    if data['target']['test'] or data['target']['doctest'] and data['executable']:
+                        test_bins.append(data['executable'])
+            except KeyError as e:
+                # skip lines that do not meet the requirements
+                pass
+
+    # All rust project will genrate at least one unit test binary
+    # It will just run a test suite with 0 tests if the project didn't define some
+    # So it is not expected to have an empty list here
+    if not test_bins:
+        bb.fatal("Unable to find any test binaries")
+
+    cargo_test_binaries_file = d.getVar('CARGO_TEST_BINARIES_FILES', True)
+    bb.note(f"Found {len(test_bins)} tests, write their path into {cargo_test_binaries_file}")
+    with open(cargo_test_binaries_file, "w") as f:
+        for test_bin in test_bins:
+            f.write(f"{test_bin}\n")
+
+}
+
+python do_install_ptest_base() {
+    import shutil
+
+    dest_dir = d.getVar("D", True)
+    pn = d.getVar("PN", True)
+    ptest_path = d.getVar("PTEST_PATH", True)
+    cargo_test_binaries_file = d.getVar('CARGO_TEST_BINARIES_FILES', True)
+
+    ptest_dir = os.path.join(dest_dir, ptest_path.lstrip('/'))
+    os.makedirs(ptest_dir, exist_ok=True)
+
+    test_bins = []
+    with open(cargo_test_binaries_file, "r") as f:
+        for line in f.readlines():
+            test_bins.append(line.strip('\n'))
+
+    test_paths = []
+    for test_bin in test_bins:
+        shutil.copy2(test_bin, ptest_dir)
+        test_paths.append(os.path.join(ptest_path, os.path.basename(test_bin)))
+
+    ptest_script = os.path.join(ptest_dir, "run-ptest")
+    if os.path.exists(ptest_script):
+        with open(ptest_script, "a") as f:
+            for test_path in test_paths:
+                f.write(f"{test_path}\n")
+    else:
+        with open(ptest_script, "a") as f:
+            f.write("#!/bin/sh\n")
+            for test_path in test_paths:
+                f.write(f"{test_path}\n")
+    os.chmod(ptest_script, 0o755)
+
+    # this is chown -R root:root ${D}${PTEST_PATH}
+    for root, dirs, files in os.walk(ptest_dir):
+        for d in dirs:
+            shutil.chown(os.path.join(root, d), "root", "root")
+        for f in files:
+            shutil.chown(os.path.join(root, f), "root", "root")
+}
-- 
2.34.1

[PATCH 2/3] python3-bcrypt: enable build of unit tests

[frederic.martinsons]

From: Frederic Martinsons <frederic.martinsons@gmail.com>

The source code of bcrypt extension doesn't define any tests
but it is to show the ptest-cargo usage

Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
---
 meta/recipes-devtools/python/python3-bcrypt_4.0.1.bb | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/python/python3-bcrypt_4.0.1.bb b/meta/recipes-devtools/python/python3-bcrypt_4.0.1.bb
index 21f2eb6ba4..b4f245530d 100644
--- a/meta/recipes-devtools/python/python3-bcrypt_4.0.1.bb
+++ b/meta/recipes-devtools/python/python3-bcrypt_4.0.1.bb
@@ -7,12 +7,14 @@ DEPENDS += "${PYTHON_PN}-cffi-native"
 
 SRC_URI[sha256sum] = "27d375903ac8261cfe4047f6709d16f7d18d39b1ec92aaf72af989552a650ebd"
 
-inherit pypi python_setuptools3_rust ptest cargo-update-recipe-crates
+inherit pypi python_setuptools3_rust ptest-cargo cargo-update-recipe-crates
 
 SRC_URI += " \
 	file://run-ptest \
 "
 
+CARGO_SRC_DIR = "src/_bcrypt"
+
 require ${BPN}-crates.inc
 
 RDEPENDS:${PN}-ptest += " \
-- 
2.34.1

[PATCH 3/3] zvariant: add ptest feature for zvariant test suite

[frederic.martinsons]

From: Frederic Martinsons <frederic.martinsons@gmail.com>

Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
---
 .../recipes-extended/zvariant/zvariant_3.12.0.bb      | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/meta-selftest/recipes-extended/zvariant/zvariant_3.12.0.bb b/meta-selftest/recipes-extended/zvariant/zvariant_3.12.0.bb
index 4285d11b72..6c69c80940 100644
--- a/meta-selftest/recipes-extended/zvariant/zvariant_3.12.0.bb
+++ b/meta-selftest/recipes-extended/zvariant/zvariant_3.12.0.bb
@@ -7,6 +7,8 @@ HOMEPAGE = "https://gitlab.freedesktop.org/dbus/zbus/"
 LICENSE = "MIT"
 LIC_FILES_CHKSUM = "file://LICENSE;md5=b377b220f43d747efdec40d69fcaa69d"
 
+DEPENDS += "${@bb.utils.contains('DISTRO_FEATURES', 'ptest', 'glib-2.0', '', d)}"
+
 SRC_URI = " \
     git://gitlab.freedesktop.org/dbus/zbus;protocol=https;branch=main;subpath=zvariant \
     file://0001-Tweak-zvariant-crate-config.patch;striplevel=2 \
@@ -21,10 +23,17 @@ python do_clean_lic_file_symlink() {
 
 addtask clean_lic_file_symlink after do_unpack before do_patch
 
-inherit cargo cargo-update-recipe-crates
+inherit ptest-cargo pkgconfig cargo-update-recipe-crates
 
 # Remove this when the recipe is reproducible
 EXCLUDE_FROM_WORLD = "1"
 
 require ${BPN}-crates.inc
 require ${BPN}-git-crates.inc
+
+# zvariant is an indermediate product for the zbus project
+# and so, it provided only a static lib (rlib) which fall only
+# in -dev package
+ALLOW_EMPTY:${PN} = "1"
+RDEPENDS:${PN}-ptest:remove = "${PN}"
+
-- 
2.34.1

Re: [OE-core] [PATCH 3/3] zvariant: add ptest feature for zvariant test suite

[Alexander Kanavin]

On Sat, 22 Apr 2023 at 18:00, Frederic Martinsons
<frederic.martinsons@gmail.com> wrote:

> +# zvariant is an indermediate product for the zbus project
> +# and so, it provided only a static lib (rlib) which fall only
> +# in -dev package
> +ALLOW_EMPTY:${PN} = "1"
> +RDEPENDS:${PN}-ptest:remove = "${PN}"

Only the first should be enough, why also the second? We generally try
to avoid :remove, as issues can be usually solved in better ways.

Alex

Re: [OE-core] [PATCH 0/3] Create class for building rust unit test

[Alexander Kanavin]

If you add ptests, please also add them to appropriate lists in
meta/conf/distro/include/ptest-packagelists.inc, so they will be
actually built and executed in yocto CI.

Can you include a sample output of the tests you added into commit messages?

Alex

On Sat, 22 Apr 2023 at 17:59, Frederic Martinsons
<frederic.martinsons@gmail.com> wrote:
>
> From: Frederic Martinsons <frederic.martinsons@gmail.com>
>
> This brings the possibility to use this class to build and ship
> unit tests of rust projects, the class also create (or modified)
> standard run-ptest script to run the generated rust test suite.
>
> It has been tested successfully with core-image-sato under qemu
> for zvariant-ptest and python3-bcrypt-ptest (though the last one
> didn't define any unit tests).
>
> Note that I tried to do the same with python3-cryptography but fail
> to build the test suite and I don't know how to do it with the rust
> extension module build by python setuptools. There must be some kind
> of way for doing that so maybe someone will put some work in it (
> because python3-cryptography rust extension do have unit tests).
>
> Moreover, in the class, I didn't manage to share data between
> do_compile_ptest_base and do_install_ptest_base cleanly (I tried
> to d.setVar in compile and d.getVar in install but it seems that
> the data store doens't recognize my new variable) so I used a file
> for that. I'm sure there is a clever way for doing that, so feel
> free to tell me.
>
> The following changes since commit 45a8bb6e4676899d40525e7d5ad1c1ddefee3185:
>
>   apt-util: Fix ptest on musl (2023-04-20 11:56:03 +0100)
>
> are available in the Git repository at:
>
>   https://gitlab.com/fmartinsons/openembedded-core cargo-add-ptest
>
> Frederic Martinsons (3):
>   ptest-cargo.bbclass: create class
>   python3-bcrypt: enable build of unit tests
>   zvariant: add ptest feature for zvariant test suite
>
>  .../zvariant/zvariant_3.12.0.bb               |  11 +-
>  meta/classes-recipe/ptest-cargo.bbclass       | 108 ++++++++++++++++++
>  .../python/python3-bcrypt_4.0.1.bb            |   4 +-
>  3 files changed, 121 insertions(+), 2 deletions(-)
>  create mode 100644 meta/classes-recipe/ptest-cargo.bbclass
>
> --
> 2.34.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#180319): https://lists.openembedded.org/g/openembedded-core/message/180319
> Mute This Topic: https://lists.openembedded.org/mt/98436057/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-core] [PATCH 0/3] Create class for building rust unit test

[Frédéric Martinsons]

[-- Attachment #1: Type: text/plain, Size: 2941 bytes --]

On Mon, 24 Apr 2023 at 09:51, Alexander Kanavin <alex.kanavin@gmail.com>
wrote:

> If you add ptests, please also add them to appropriate lists in
> meta/conf/distro/include/ptest-packagelists.inc, so they will be
> actually built and executed in yocto CI.
>
> Can you include a sample output of the tests you added into commit
> messages?
>
> Alex
>

Ok I will issue a V2 with the modifications asked.


> On Sat, 22 Apr 2023 at 17:59, Frederic Martinsons
> <frederic.martinsons@gmail.com> wrote:
> >
> > From: Frederic Martinsons <frederic.martinsons@gmail.com>
> >
> > This brings the possibility to use this class to build and ship
> > unit tests of rust projects, the class also create (or modified)
> > standard run-ptest script to run the generated rust test suite.
> >
> > It has been tested successfully with core-image-sato under qemu
> > for zvariant-ptest and python3-bcrypt-ptest (though the last one
> > didn't define any unit tests).
> >
> > Note that I tried to do the same with python3-cryptography but fail
> > to build the test suite and I don't know how to do it with the rust
> > extension module build by python setuptools. There must be some kind
> > of way for doing that so maybe someone will put some work in it (
> > because python3-cryptography rust extension do have unit tests).
> >
> > Moreover, in the class, I didn't manage to share data between
> > do_compile_ptest_base and do_install_ptest_base cleanly (I tried
> > to d.setVar in compile and d.getVar in install but it seems that
> > the data store doens't recognize my new variable) so I used a file
> > for that. I'm sure there is a clever way for doing that, so feel
> > free to tell me.
> >
> > The following changes since commit
> 45a8bb6e4676899d40525e7d5ad1c1ddefee3185:
> >
> >   apt-util: Fix ptest on musl (2023-04-20 11:56:03 +0100)
> >
> > are available in the Git repository at:
> >
> >   https://gitlab.com/fmartinsons/openembedded-core cargo-add-ptest
> >
> > Frederic Martinsons (3):
> >   ptest-cargo.bbclass: create class
> >   python3-bcrypt: enable build of unit tests
> >   zvariant: add ptest feature for zvariant test suite
> >
> >  .../zvariant/zvariant_3.12.0.bb               |  11 +-
> >  meta/classes-recipe/ptest-cargo.bbclass       | 108 ++++++++++++++++++
> >  .../python/python3-bcrypt_4.0.1.bb            |   4 +-
> >  3 files changed, 121 insertions(+), 2 deletions(-)
> >  create mode 100644 meta/classes-recipe/ptest-cargo.bbclass
> >
> > --
> > 2.34.1
> >
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#180319):
> https://lists.openembedded.org/g/openembedded-core/message/180319
> > Mute This Topic: https://lists.openembedded.org/mt/98436057/1686489
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> alex.kanavin@gmail.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >
>

[-- Attachment #2: Type: text/html, Size: 4709 bytes --]

Re: [OE-core] [PATCH 3/3] zvariant: add ptest feature for zvariant test suite

[Frédéric Martinsons]

[-- Attachment #1: Type: text/plain, Size: 754 bytes --]

On Mon, 24 Apr 2023 at 09:47, Alexander Kanavin <alex.kanavin@gmail.com>
wrote:

> On Sat, 22 Apr 2023 at 18:00, Frederic Martinsons
> <frederic.martinsons@gmail.com> wrote:
>
> > +# zvariant is an indermediate product for the zbus project
> > +# and so, it provided only a static lib (rlib) which fall only
> > +# in -dev package
> > +ALLOW_EMPTY:${PN} = "1"
> > +RDEPENDS:${PN}-ptest:remove = "${PN}"
>
> Only the first should be enough, why also the second? We generally try
> to avoid :remove, as issues can be usually solved in better ways.
>
> Alex
>

Ok will remove it in a V2. It seems that I had a wrong state if I must had
add this line, but I did
a "bitbake -c cleanall zvariant && bitbake core-image-sato" and I didn't
have any rootfs
issue.

[-- Attachment #2: Type: text/html, Size: 1275 bytes --]

Re: [OE-core] [PATCH 0/3] Create class for building rust unit test

[Frédéric Martinsons]

[-- Attachment #1: Type: text/plain, Size: 3363 bytes --]

On Mon, 24 Apr 2023 at 10:20, Frederic Martinsons via lists.openembedded.org
<frederic.martinsons=gmail.com@lists.openembedded.org> wrote:

>
> On Mon, 24 Apr 2023 at 09:51, Alexander Kanavin <alex.kanavin@gmail.com>
> wrote:
>
>> If you add ptests, please also add them to appropriate lists in
>> meta/conf/distro/include/ptest-packagelists.inc, so they will be
>> actually built and executed in yocto CI.
>>
>> Can you include a sample output of the tests you added into commit
>> messages?
>>
>> Alex
>>
>
> Ok I will issue a V2 with the modifications asked.
>
>
I looked again and python3-bcrypt is already in PTESTS_FAST , for zvariant
, the recipe is in meta-selftest
so this layer is not always included, is this will be a problem to add
zvariant into PTESTS_FAST ?

On Sat, 22 Apr 2023 at 17:59, Frederic Martinsons
>> <frederic.martinsons@gmail.com> wrote:
>> >
>> > From: Frederic Martinsons <frederic.martinsons@gmail.com>
>> >
>> > This brings the possibility to use this class to build and ship
>> > unit tests of rust projects, the class also create (or modified)
>> > standard run-ptest script to run the generated rust test suite.
>> >
>> > It has been tested successfully with core-image-sato under qemu
>> > for zvariant-ptest and python3-bcrypt-ptest (though the last one
>> > didn't define any unit tests).
>> >
>> > Note that I tried to do the same with python3-cryptography but fail
>> > to build the test suite and I don't know how to do it with the rust
>> > extension module build by python setuptools. There must be some kind
>> > of way for doing that so maybe someone will put some work in it (
>> > because python3-cryptography rust extension do have unit tests).
>> >
>> > Moreover, in the class, I didn't manage to share data between
>> > do_compile_ptest_base and do_install_ptest_base cleanly (I tried
>> > to d.setVar in compile and d.getVar in install but it seems that
>> > the data store doens't recognize my new variable) so I used a file
>> > for that. I'm sure there is a clever way for doing that, so feel
>> > free to tell me.
>> >
>> > The following changes since commit
>> 45a8bb6e4676899d40525e7d5ad1c1ddefee3185:
>> >
>> >   apt-util: Fix ptest on musl (2023-04-20 11:56:03 +0100)
>> >
>> > are available in the Git repository at:
>> >
>> >   https://gitlab.com/fmartinsons/openembedded-core cargo-add-ptest
>> >
>> > Frederic Martinsons (3):
>> >   ptest-cargo.bbclass: create class
>> >   python3-bcrypt: enable build of unit tests
>> >   zvariant: add ptest feature for zvariant test suite
>> >
>> >  .../zvariant/zvariant_3.12.0.bb               |  11 +-
>> >  meta/classes-recipe/ptest-cargo.bbclass       | 108 ++++++++++++++++++
>> >  .../python/python3-bcrypt_4.0.1.bb            |   4 +-
>> >  3 files changed, 121 insertions(+), 2 deletions(-)
>> >  create mode 100644 meta/classes-recipe/ptest-cargo.bbclass
>> >
>> > --
>> > 2.34.1
>> >
>> >
>> >
>> >
>>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#180341):
> https://lists.openembedded.org/g/openembedded-core/message/180341
> Mute This Topic: https://lists.openembedded.org/mt/98436057/6213388
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> frederic.martinsons@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 5642 bytes --]

Re: [OE-core] [PATCH 0/3] Create class for building rust unit test

[Alexander Kanavin]

On Mon, 24 Apr 2023 at 11:44, Frédéric Martinsons
<frederic.martinsons@gmail.com> wrote:
> I looked again and python3-bcrypt is already in PTESTS_FAST , for zvariant , the recipe is in meta-selftest
> so this layer is not always included, is this will be a problem to add zvariant into PTESTS_FAST ?

zvariant could be skipped for now then. Are the newly expanded bcrypt
tests still quick (under kvm)?

Alex

Re: [OE-core] [PATCH 0/3] Create class for building rust unit test

[Frédéric Martinsons]

[-- Attachment #1: Type: text/plain, Size: 774 bytes --]

Le lun. 24 avr. 2023, 11:53, Alexander Kanavin <alex.kanavin@gmail.com> a
écrit :

> On Mon, 24 Apr 2023 at 11:44, Frédéric Martinsons
> <frederic.martinsons@gmail.com> wrote:
> > I looked again and python3-bcrypt is already in PTESTS_FAST , for
> zvariant , the recipe is in meta-selftest
> > so this layer is not always included, is this will be a problem to add
> zvariant into PTESTS_FAST ?
>
> zvariant could be skipped for now then. Are the newly expanded bcrypt
> tests still quick (under kvm)?
>
> Alex
>

Really quick since there are 0 test in the rust code ;)
But a test binary is guaranteed to be generated anyway.
I'll take another look to see if why python3-bcrypt was already in the test
suite and if I didn't overwrite something there.

[-- Attachment #2: Type: text/html, Size: 1451 bytes --]