[RESEND 0/1] Add ptest support for libexif

[RESEND 0/1] Add ptest support for libexif

[Julien Stephan]

Hi all,

Resending this series. The required series [1] has been merged, so this one is now ready to be merged.

Previous message:

This series add ptest support for libexif.

This series depend on [1]. This fix is required here because one of the
test program, listed in check_PROGRAMS is not listed in TESTS, because
it is called through a wrapper script. So we are missing one binary and
we get a runtime error.

Ptest was executed on a core-image-minimal:
root@qemux86-64:~# ptest-runner libexif
START: ptest-runner
2023-07-12T15:45
BEGIN: /usr/lib/libexif/ptest
[...]
i====================
All 13 tests passed
(1 test was not run)
====================
[...]
DURATION: 35
END: /usr/lib/libexif/ptest
2023-07-12T15:45
STOP: ptest-runner
TOTAL: 1 FAIL: 0

My working branch is [2]

Best
Julien

[1]: https://lists.openembedded.org/g/openembedded-core/message/184201
[2]: https://git.yoctoproject.org/poky-contrib/log/?h=jstephan/add-libexif-ptest-support

Julien Stephan (1):
  libexif: add ptest support

 .../distro/include/ptest-packagelists.inc     |  1 +
 ...-serial-tests-config-needed-by-ptest.patch | 26 ++++++++++++++++
 .../recipes-support/libexif/libexif/run-ptest |  3 ++
 .../recipes-support/libexif/libexif_0.6.24.bb | 30 ++++++++++++++++++-
 4 files changed, 59 insertions(+), 1 deletion(-)
 create mode 100644 meta/recipes-support/libexif/libexif/0001-Add-serial-tests-config-needed-by-ptest.patch
 create mode 100644 meta/recipes-support/libexif/libexif/run-ptest

--
2.41.0

[RESEND 1/1] libexif: add ptest support

[Julien Stephan]

adds ptest support for libexif

Ptest was executed on a core-image-minimal:
root@qemux86-64:~# ptest-runner libexif
START: ptest-runner
2023-07-12T15:45
BEGIN: /usr/lib/libexif/ptest
[...]
i====================
All 13 tests passed
(1 test was not run)
====================
[...]
DURATION: 35
END: /usr/lib/libexif/ptest
2023-07-12T15:45
STOP: ptest-runner
TOTAL: 1 FAIL: 0

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
---
 .../distro/include/ptest-packagelists.inc     |  1 +
 ...-serial-tests-config-needed-by-ptest.patch | 26 ++++++++++++++++
 .../recipes-support/libexif/libexif/run-ptest |  3 ++
 .../recipes-support/libexif/libexif_0.6.24.bb | 30 ++++++++++++++++++-
 4 files changed, 59 insertions(+), 1 deletion(-)
 create mode 100644 meta/recipes-support/libexif/libexif/0001-Add-serial-tests-config-needed-by-ptest.patch
 create mode 100644 meta/recipes-support/libexif/libexif/run-ptest

diff --git a/meta/conf/distro/include/ptest-packagelists.inc b/meta/conf/distro/include/ptest-packagelists.inc
index 6250cf081e0..bbbef5b0431 100644
--- a/meta/conf/distro/include/ptest-packagelists.inc
+++ b/meta/conf/distro/include/ptest-packagelists.inc
@@ -75,6 +75,7 @@ PTESTS_FAST = "\
     slang \
     wayland \
     zlib \
+    libexif \
 "
 PTESTS_FAST:append:libc-glibc = " glibc-tests"
 PTESTS_PROBLEMS:remove:libc-glibc = "glibc-tests"
diff --git a/meta/recipes-support/libexif/libexif/0001-Add-serial-tests-config-needed-by-ptest.patch b/meta/recipes-support/libexif/libexif/0001-Add-serial-tests-config-needed-by-ptest.patch
new file mode 100644
index 00000000000..505aa073309
--- /dev/null
+++ b/meta/recipes-support/libexif/libexif/0001-Add-serial-tests-config-needed-by-ptest.patch
@@ -0,0 +1,26 @@
+From 1ee7217c8ae724d793f9a9876c3608057a2ccbf8 Mon Sep 17 00:00:00 2001
+From: Julien Stephan <jstephan@baylibre.com>
+Date: Tue, 11 Jul 2023 16:07:54 +0200
+Subject: [PATCH] Add serial-tests config needed by ptest
+
+Add serial-tests support, ptest needs it.
+
+Upstream-Status: Inappropriate [oe specific]
+
+Signed-off-by: Julien Stephan <jstephan@baylibre.com>
+---
+ configure.ac | 1 +
+ 1 file changed, 1 insertion(+)
+
+diff --git a/configure.ac b/configure.ac
+index cd48047..5413907 100644
+--- a/configure.ac
++++ b/configure.ac
+@@ -16,6 +16,7 @@ AM_INIT_AUTOMAKE([
+   dist-zip
+   check-news
+   subdir-objects
++  serial-tests
+ ])
+ AM_MAINTAINER_MODE
+ 
diff --git a/meta/recipes-support/libexif/libexif/run-ptest b/meta/recipes-support/libexif/libexif/run-ptest
new file mode 100644
index 00000000000..2d23159eb03
--- /dev/null
+++ b/meta/recipes-support/libexif/libexif/run-ptest
@@ -0,0 +1,3 @@
+#!/bin/sh
+
+make -o Makefile runtest-TESTS
diff --git a/meta/recipes-support/libexif/libexif_0.6.24.bb b/meta/recipes-support/libexif/libexif_0.6.24.bb
index 367f012f8cf..27e369a89a5 100644
--- a/meta/recipes-support/libexif/libexif_0.6.24.bb
+++ b/meta/recipes-support/libexif/libexif_0.6.24.bb
@@ -11,12 +11,40 @@ def version_underscore(v):
     return "_".join(v.split("."))
 
 SRC_URI = "${GITHUB_BASE_URI}/download/v${PV}/libexif-${PV}.tar.bz2 \
+           file://0001-Add-serial-tests-config-needed-by-ptest.patch \
+           file://run-ptest \
            "
 
 SRC_URI[sha256sum] = "d47564c433b733d83b6704c70477e0a4067811d184ec565258ac563d8223f6ae"
 
-inherit autotools gettext github-releases
+inherit autotools gettext github-releases ptest
 
 EXTRA_OECONF += "--disable-docs"
 
+do_compile_ptest() {
+    oe_runmake -C test buildtest-TESTS
+}
+
+do_install_ptest() {
+    install ${B}/test/test*[!\.o] ${D}${PTEST_PATH}
+    for f in ${D}${PTEST_PATH}/test*; do
+        sed -i "s/\(LD_LIBRARY_PATH=\).*\(:\$LD_LIBRARY_PATH\)\"/\1.\2/" $f
+    done
+
+    install ${B}/test/Makefile ${D}${PTEST_PATH}
+    sed -i -e "/^srcdir/c srcdir = \$\{PWD\}" ${D}${PTEST_PATH}/Makefile
+
+    install -d ${D}${PTEST_PATH}/nls
+    install ${B}/test/nls/*[!\.o] ${D}${PTEST_PATH}/nls
+    install -d ${D}${PTEST_PATH}/.libs
+    install ${B}/test/.libs/* ${D}${PTEST_PATH}/.libs
+
+    install ${S}/test/*.sh ${D}${PTEST_PATH}
+
+    install -d ${D}${PTEST_PATH}/testdata
+    install ${S}/test/testdata/* ${D}${PTEST_PATH}/testdata
+}
+
+RDEPENDS:${PN}-ptest += "make bash"
+
 BBCLASSEXTEND = "native nativesdk"
-- 
2.41.0

Re: [OE-core] [RESEND 1/1] libexif: add ptest support

[Alexander Kanavin]

What do the tests actually do? If they test libexif functionality in
isolation, is there a need to run them, if upstream does this anyway?
ptests are primarily meant for integration testing, e.g. exposing
issues that arise in a complete customized system.

Alex

On Thu, 3 Aug 2023 at 15:29, Julien Stephan <jstephan@baylibre.com> wrote:
>
> adds ptest support for libexif
>
> Ptest was executed on a core-image-minimal:
> root@qemux86-64:~# ptest-runner libexif
> START: ptest-runner
> 2023-07-12T15:45
> BEGIN: /usr/lib/libexif/ptest
> [...]
> i====================
> All 13 tests passed
> (1 test was not run)
> ====================
> [...]
> DURATION: 35
> END: /usr/lib/libexif/ptest
> 2023-07-12T15:45
> STOP: ptest-runner
> TOTAL: 1 FAIL: 0
>
> Signed-off-by: Julien Stephan <jstephan@baylibre.com>
> ---
>  .../distro/include/ptest-packagelists.inc     |  1 +
>  ...-serial-tests-config-needed-by-ptest.patch | 26 ++++++++++++++++
>  .../recipes-support/libexif/libexif/run-ptest |  3 ++
>  .../recipes-support/libexif/libexif_0.6.24.bb | 30 ++++++++++++++++++-
>  4 files changed, 59 insertions(+), 1 deletion(-)
>  create mode 100644 meta/recipes-support/libexif/libexif/0001-Add-serial-tests-config-needed-by-ptest.patch
>  create mode 100644 meta/recipes-support/libexif/libexif/run-ptest
>
> diff --git a/meta/conf/distro/include/ptest-packagelists.inc b/meta/conf/distro/include/ptest-packagelists.inc
> index 6250cf081e0..bbbef5b0431 100644
> --- a/meta/conf/distro/include/ptest-packagelists.inc
> +++ b/meta/conf/distro/include/ptest-packagelists.inc
> @@ -75,6 +75,7 @@ PTESTS_FAST = "\
>      slang \
>      wayland \
>      zlib \
> +    libexif \
>  "
>  PTESTS_FAST:append:libc-glibc = " glibc-tests"
>  PTESTS_PROBLEMS:remove:libc-glibc = "glibc-tests"
> diff --git a/meta/recipes-support/libexif/libexif/0001-Add-serial-tests-config-needed-by-ptest.patch b/meta/recipes-support/libexif/libexif/0001-Add-serial-tests-config-needed-by-ptest.patch
> new file mode 100644
> index 00000000000..505aa073309
> --- /dev/null
> +++ b/meta/recipes-support/libexif/libexif/0001-Add-serial-tests-config-needed-by-ptest.patch
> @@ -0,0 +1,26 @@
> +From 1ee7217c8ae724d793f9a9876c3608057a2ccbf8 Mon Sep 17 00:00:00 2001
> +From: Julien Stephan <jstephan@baylibre.com>
> +Date: Tue, 11 Jul 2023 16:07:54 +0200
> +Subject: [PATCH] Add serial-tests config needed by ptest
> +
> +Add serial-tests support, ptest needs it.
> +
> +Upstream-Status: Inappropriate [oe specific]
> +
> +Signed-off-by: Julien Stephan <jstephan@baylibre.com>
> +---
> + configure.ac | 1 +
> + 1 file changed, 1 insertion(+)
> +
> +diff --git a/configure.ac b/configure.ac
> +index cd48047..5413907 100644
> +--- a/configure.ac
> ++++ b/configure.ac
> +@@ -16,6 +16,7 @@ AM_INIT_AUTOMAKE([
> +   dist-zip
> +   check-news
> +   subdir-objects
> ++  serial-tests
> + ])
> + AM_MAINTAINER_MODE
> +
> diff --git a/meta/recipes-support/libexif/libexif/run-ptest b/meta/recipes-support/libexif/libexif/run-ptest
> new file mode 100644
> index 00000000000..2d23159eb03
> --- /dev/null
> +++ b/meta/recipes-support/libexif/libexif/run-ptest
> @@ -0,0 +1,3 @@
> +#!/bin/sh
> +
> +make -o Makefile runtest-TESTS
> diff --git a/meta/recipes-support/libexif/libexif_0.6.24.bb b/meta/recipes-support/libexif/libexif_0.6.24.bb
> index 367f012f8cf..27e369a89a5 100644
> --- a/meta/recipes-support/libexif/libexif_0.6.24.bb
> +++ b/meta/recipes-support/libexif/libexif_0.6.24.bb
> @@ -11,12 +11,40 @@ def version_underscore(v):
>      return "_".join(v.split("."))
>
>  SRC_URI = "${GITHUB_BASE_URI}/download/v${PV}/libexif-${PV}.tar.bz2 \
> +           file://0001-Add-serial-tests-config-needed-by-ptest.patch \
> +           file://run-ptest \
>             "
>
>  SRC_URI[sha256sum] = "d47564c433b733d83b6704c70477e0a4067811d184ec565258ac563d8223f6ae"
>
> -inherit autotools gettext github-releases
> +inherit autotools gettext github-releases ptest
>
>  EXTRA_OECONF += "--disable-docs"
>
> +do_compile_ptest() {
> +    oe_runmake -C test buildtest-TESTS
> +}
> +
> +do_install_ptest() {
> +    install ${B}/test/test*[!\.o] ${D}${PTEST_PATH}
> +    for f in ${D}${PTEST_PATH}/test*; do
> +        sed -i "s/\(LD_LIBRARY_PATH=\).*\(:\$LD_LIBRARY_PATH\)\"/\1.\2/" $f
> +    done
> +
> +    install ${B}/test/Makefile ${D}${PTEST_PATH}
> +    sed -i -e "/^srcdir/c srcdir = \$\{PWD\}" ${D}${PTEST_PATH}/Makefile
> +
> +    install -d ${D}${PTEST_PATH}/nls
> +    install ${B}/test/nls/*[!\.o] ${D}${PTEST_PATH}/nls
> +    install -d ${D}${PTEST_PATH}/.libs
> +    install ${B}/test/.libs/* ${D}${PTEST_PATH}/.libs
> +
> +    install ${S}/test/*.sh ${D}${PTEST_PATH}
> +
> +    install -d ${D}${PTEST_PATH}/testdata
> +    install ${S}/test/testdata/* ${D}${PTEST_PATH}/testdata
> +}
> +
> +RDEPENDS:${PN}-ptest += "make bash"
> +
>  BBCLASSEXTEND = "native nativesdk"
> --
> 2.41.0
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#185477): https://lists.openembedded.org/g/openembedded-core/message/185477
> Mute This Topic: https://lists.openembedded.org/mt/100525801/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-core] [RESEND 1/1] libexif: add ptest support

[Julien Stephan]

Le jeu. 3 août 2023 à 15:51, Alexander Kanavin
<alex.kanavin@gmail.com> a écrit :
>
> What do the tests actually do? If they test libexif functionality in
> isolation, is there a need to run them, if upstream does this anyway?
> ptests are primarily meant for integration testing, e.g. exposing
> issues that arise in a complete customized system.
>
> Alex

Hi Alexander,

Didn't catch this before, makes sense. I thought we would like to have
ptest enabled for as many packages as possible.
From libexif github pages:

> The short test programs in the test directory illustrates how to create
> valid EXIF data from scratch, how to save EXIF data and how to load EXIF
> data from data in memory. To run the test programs, just run make check.
> There are also a few simple example programs available in the
> contrib/examples/ directory.  Don't hesitate to contact
> us at <libexif-devel@lists.sourceforge.net> if you have any questions
> on how to use libexif.

So what should we do about this patch? just drop it?
Best

Julien

>
> On Thu, 3 Aug 2023 at 15:29, Julien Stephan <jstephan@baylibre.com> wrote:
> >
> > adds ptest support for libexif
> >
> > Ptest was executed on a core-image-minimal:
> > root@qemux86-64:~# ptest-runner libexif
> > START: ptest-runner
> > 2023-07-12T15:45
> > BEGIN: /usr/lib/libexif/ptest
> > [...]
> > i====================
> > All 13 tests passed
> > (1 test was not run)
> > ====================
> > [...]
> > DURATION: 35
> > END: /usr/lib/libexif/ptest
> > 2023-07-12T15:45
> > STOP: ptest-runner
> > TOTAL: 1 FAIL: 0
> >
> > Signed-off-by: Julien Stephan <jstephan@baylibre.com>
> > ---
> >  .../distro/include/ptest-packagelists.inc     |  1 +
> >  ...-serial-tests-config-needed-by-ptest.patch | 26 ++++++++++++++++
> >  .../recipes-support/libexif/libexif/run-ptest |  3 ++
> >  .../recipes-support/libexif/libexif_0.6.24.bb | 30 ++++++++++++++++++-
> >  4 files changed, 59 insertions(+), 1 deletion(-)
> >  create mode 100644 meta/recipes-support/libexif/libexif/0001-Add-serial-tests-config-needed-by-ptest.patch
> >  create mode 100644 meta/recipes-support/libexif/libexif/run-ptest
> >
> > diff --git a/meta/conf/distro/include/ptest-packagelists.inc b/meta/conf/distro/include/ptest-packagelists.inc
> > index 6250cf081e0..bbbef5b0431 100644
> > --- a/meta/conf/distro/include/ptest-packagelists.inc
> > +++ b/meta/conf/distro/include/ptest-packagelists.inc
> > @@ -75,6 +75,7 @@ PTESTS_FAST = "\
> >      slang \
> >      wayland \
> >      zlib \
> > +    libexif \
> >  "
> >  PTESTS_FAST:append:libc-glibc = " glibc-tests"
> >  PTESTS_PROBLEMS:remove:libc-glibc = "glibc-tests"
> > diff --git a/meta/recipes-support/libexif/libexif/0001-Add-serial-tests-config-needed-by-ptest.patch b/meta/recipes-support/libexif/libexif/0001-Add-serial-tests-config-needed-by-ptest.patch
> > new file mode 100644
> > index 00000000000..505aa073309
> > --- /dev/null
> > +++ b/meta/recipes-support/libexif/libexif/0001-Add-serial-tests-config-needed-by-ptest.patch
> > @@ -0,0 +1,26 @@
> > +From 1ee7217c8ae724d793f9a9876c3608057a2ccbf8 Mon Sep 17 00:00:00 2001
> > +From: Julien Stephan <jstephan@baylibre.com>
> > +Date: Tue, 11 Jul 2023 16:07:54 +0200
> > +Subject: [PATCH] Add serial-tests config needed by ptest
> > +
> > +Add serial-tests support, ptest needs it.
> > +
> > +Upstream-Status: Inappropriate [oe specific]
> > +
> > +Signed-off-by: Julien Stephan <jstephan@baylibre.com>
> > +---
> > + configure.ac | 1 +
> > + 1 file changed, 1 insertion(+)
> > +
> > +diff --git a/configure.ac b/configure.ac
> > +index cd48047..5413907 100644
> > +--- a/configure.ac
> > ++++ b/configure.ac
> > +@@ -16,6 +16,7 @@ AM_INIT_AUTOMAKE([
> > +   dist-zip
> > +   check-news
> > +   subdir-objects
> > ++  serial-tests
> > + ])
> > + AM_MAINTAINER_MODE
> > +
> > diff --git a/meta/recipes-support/libexif/libexif/run-ptest b/meta/recipes-support/libexif/libexif/run-ptest
> > new file mode 100644
> > index 00000000000..2d23159eb03
> > --- /dev/null
> > +++ b/meta/recipes-support/libexif/libexif/run-ptest
> > @@ -0,0 +1,3 @@
> > +#!/bin/sh
> > +
> > +make -o Makefile runtest-TESTS
> > diff --git a/meta/recipes-support/libexif/libexif_0.6.24.bb b/meta/recipes-support/libexif/libexif_0.6.24.bb
> > index 367f012f8cf..27e369a89a5 100644
> > --- a/meta/recipes-support/libexif/libexif_0.6.24.bb
> > +++ b/meta/recipes-support/libexif/libexif_0.6.24.bb
> > @@ -11,12 +11,40 @@ def version_underscore(v):
> >      return "_".join(v.split("."))
> >
> >  SRC_URI = "${GITHUB_BASE_URI}/download/v${PV}/libexif-${PV}.tar.bz2 \
> > +           file://0001-Add-serial-tests-config-needed-by-ptest.patch \
> > +           file://run-ptest \
> >             "
> >
> >  SRC_URI[sha256sum] = "d47564c433b733d83b6704c70477e0a4067811d184ec565258ac563d8223f6ae"
> >
> > -inherit autotools gettext github-releases
> > +inherit autotools gettext github-releases ptest
> >
> >  EXTRA_OECONF += "--disable-docs"
> >
> > +do_compile_ptest() {
> > +    oe_runmake -C test buildtest-TESTS
> > +}
> > +
> > +do_install_ptest() {
> > +    install ${B}/test/test*[!\.o] ${D}${PTEST_PATH}
> > +    for f in ${D}${PTEST_PATH}/test*; do
> > +        sed -i "s/\(LD_LIBRARY_PATH=\).*\(:\$LD_LIBRARY_PATH\)\"/\1.\2/" $f
> > +    done
> > +
> > +    install ${B}/test/Makefile ${D}${PTEST_PATH}
> > +    sed -i -e "/^srcdir/c srcdir = \$\{PWD\}" ${D}${PTEST_PATH}/Makefile
> > +
> > +    install -d ${D}${PTEST_PATH}/nls
> > +    install ${B}/test/nls/*[!\.o] ${D}${PTEST_PATH}/nls
> > +    install -d ${D}${PTEST_PATH}/.libs
> > +    install ${B}/test/.libs/* ${D}${PTEST_PATH}/.libs
> > +
> > +    install ${S}/test/*.sh ${D}${PTEST_PATH}
> > +
> > +    install -d ${D}${PTEST_PATH}/testdata
> > +    install ${S}/test/testdata/* ${D}${PTEST_PATH}/testdata
> > +}
> > +
> > +RDEPENDS:${PN}-ptest += "make bash"
> > +
> >  BBCLASSEXTEND = "native nativesdk"
> > --
> > 2.41.0
> >
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#185477): https://lists.openembedded.org/g/openembedded-core/message/185477
> > Mute This Topic: https://lists.openembedded.org/mt/100525801/1686489
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >

Re: [OE-core] [RESEND 1/1] libexif: add ptest support

[Richard Purdie]

On Thu, 2023-08-03 at 15:50 +0200, Alexander Kanavin wrote:
> What do the tests actually do? If they test libexif functionality in
> isolation, is there a need to run them, if upstream does this anyway?
> ptests are primarily meant for integration testing, e.g. exposing
> issues that arise in a complete customized system.

I've noticed a few replies about this and I do disagree slightly.

ptests are useful to ensure that the recipes in question actually work.
I'm sure most upstreams do test things but probably not on weirder
targets like mips/powerpc or with musl.

For something like libexif, if the ptests test the library functions
properly, returning the right test data for a sample for example, then
that is a useful for us.

Cheers,

Richard