* [master][PATCH v2] libsdl2: Add tests package and ptest support
@ 2025-09-04 0:07 a-christidis
2025-09-04 8:41 ` Mathieu Dubois-Briand
2025-09-04 11:27 ` Alexander Kanavin
0 siblings, 2 replies; 6+ messages in thread
From: a-christidis @ 2025-09-04 0:07 UTC (permalink / raw)
To: openembedded-core, mathieu.dubois-briand; +Cc: alex, alex.kanavin
From: Antonios Christidis <a-christidis@ti.com>
The SDL2 software comes with its own set of tests. Following that
introduce support for ptest.
Signed-off-by: Antonios Christidis <a-christidis@ti.com>
---
v2:
- Updated run-ptest to be posix compliant
- Use SDL_INSTALL_TESTS cmake config parameter
meta/conf/distro/include/ptest-packagelists.inc | 1 +
meta/recipes-graphics/libsdl2/libsdl2/run-ptest | 16 ++++++++++++++++
meta/recipes-graphics/libsdl2/libsdl2_2.32.8.bb | 13 +++++++++++--
3 files changed, 28 insertions(+), 2 deletions(-)
create mode 100644 meta/recipes-graphics/libsdl2/libsdl2/run-ptest
diff --git a/meta/conf/distro/include/ptest-packagelists.inc b/meta/conf/distro/include/ptest-packagelists.inc
index 9a7b25a916..a5476927b5 100644
--- a/meta/conf/distro/include/ptest-packagelists.inc
+++ b/meta/conf/distro/include/ptest-packagelists.inc
@@ -32,6 +32,7 @@ PTESTS_FAST = "\
libgpg-error\
libnl \
libpcre \
+ libsdl2 \
libssh2 \
libtest-fatal-perl \
libtest-needs-perl \
diff --git a/meta/recipes-graphics/libsdl2/libsdl2/run-ptest b/meta/recipes-graphics/libsdl2/libsdl2/run-ptest
new file mode 100644
index 0000000000..acdfa951e9
--- /dev/null
+++ b/meta/recipes-graphics/libsdl2/libsdl2/run-ptest
@@ -0,0 +1,16 @@
+#!/bin/sh
+
+test_array="testdraw2 testgeometry testsprite2 testgles2 testoffscreen"
+
+
+test_path="/usr/libexec/installed-tests/SDL2/"
+
+for test in ${test_array}
+do
+ timeout --preserve-status 10 "$test_path""$test"
+ if [ $? -eq 0 ]; then
+ echo "PASS: $test"
+ else
+ echo "FAIL: $test"
+ fi
+done
diff --git a/meta/recipes-graphics/libsdl2/libsdl2_2.32.8.bb b/meta/recipes-graphics/libsdl2/libsdl2_2.32.8.bb
index 98291e0f80..9134a408c6 100644
--- a/meta/recipes-graphics/libsdl2/libsdl2_2.32.8.bb
+++ b/meta/recipes-graphics/libsdl2/libsdl2_2.32.8.bb
@@ -21,13 +21,15 @@ LIC_FILES_CHKSUM:append = " ${@bb.utils.contains('PACKAGECONFIG', 'arm-neon', 'f
PROVIDES = "virtual/libsdl2"
-SRC_URI = "https://www.libsdl.org/release/SDL2-${PV}.tar.gz"
+SRC_URI = "https://www.libsdl.org/release/SDL2-${PV}.tar.gz \
+ file://run-ptest \
+ "
S = "${UNPACKDIR}/SDL2-${PV}"
SRC_URI[sha256sum] = "0ca83e9c9b31e18288c7ec811108e58bac1f1bb5ec6577ad386830eac51c787e"
-inherit cmake lib_package binconfig-disabled pkgconfig upstream-version-is-even
+inherit cmake lib_package binconfig-disabled pkgconfig upstream-version-is-even ptest
UPSTREAM_CHECK_REGEX = "SDL2-(?P<pver>\d+\.(\d*[02468])+(\.\d+)+)\.tar"
BINCONFIG = "${bindir}/sdl2-config"
@@ -48,6 +50,8 @@ EXTRA_OECMAKE = "-DSDL_OSS=OFF -DSDL_ESD=OFF -DSDL_ARTS=OFF \
-DSDL_X11_XRANDR=OFF \
-DSDL_X11_XSCRNSAVER=OFF \
-DSDL_X11_XSHAPE=OFF \
+ -DSDL_TESTS=ON \
+ -DSDL_INSTALL_TESTS=ON \
"
# opengl packageconfig factored out to make it easy for distros
@@ -82,6 +86,11 @@ PACKAGECONFIG[x11] = "-DSDL_X11=ON,-DSDL_X11=OFF,virtual/libx11 libxext l
CFLAGS:append:class-native = " -DNO_SHARED_MEMORY"
+PACKAGE_BEFORE_PN = "${PN}-tests"
+
FILES:${PN} += "${datadir}/licenses/SDL2/LICENSE.txt"
+FILES:${PN}-tests += "${libexecdir} /usr/share/installed-tests/SDL2"
+
+RDEPENDS:${PN}-ptest = "${PN}-tests"
BBCLASSEXTEND = "native nativesdk"
--
2.34.1
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [master][PATCH v2] libsdl2: Add tests package and ptest support
2025-09-04 0:07 [master][PATCH v2] libsdl2: Add tests package and ptest support a-christidis
@ 2025-09-04 8:41 ` Mathieu Dubois-Briand
2025-09-04 8:52 ` [OE-core] " Joao Marcos Costa
2025-09-04 11:27 ` Alexander Kanavin
1 sibling, 1 reply; 6+ messages in thread
From: Mathieu Dubois-Briand @ 2025-09-04 8:41 UTC (permalink / raw)
To: a-christidis, openembedded-core; +Cc: alex, alex.kanavin
On Thu Sep 4, 2025 at 2:07 AM CEST, a-christidis wrote:
> From: Antonios Christidis <a-christidis@ti.com>
>
> The SDL2 software comes with its own set of tests. Following that
> introduce support for ptest.
>
> Signed-off-by: Antonios Christidis <a-christidis@ti.com>
> ---
Hi Antonios,
Thanks for the new version.
It looks like we still have a QA issue:
ERROR: nativesdk-libsdl2-2.32.8-r0 do_package: QA Issue: nativesdk-libsdl2: Files/directories were installed but not shipped in any package:
/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests
/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2
/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testsurround.test
/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testerror.test
/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testtimer.test
/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testatomic.test
/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testfilesystem.test
/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testthread.test
/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testaudioinfo.test
/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testlocale.test
/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testevdev.test
/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testpower.test
/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testqsort.test
/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testplatform.test
/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testautomation.test
/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testver.test
Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install.
https://autobuilder.yoctoproject.org/valkyrie/#/builders/19/builds/2327
Can you fix it please?
Thanks,
Mathieu
--
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [OE-core] [master][PATCH v2] libsdl2: Add tests package and ptest support
2025-09-04 8:41 ` Mathieu Dubois-Briand
@ 2025-09-04 8:52 ` Joao Marcos Costa
0 siblings, 0 replies; 6+ messages in thread
From: Joao Marcos Costa @ 2025-09-04 8:52 UTC (permalink / raw)
To: mathieu.dubois-briand, a-christidis, openembedded-core; +Cc: alex, alex.kanavin
Hello,
I hope this email finds you well.
On 9/4/25 10:41, Mathieu Dubois-Briand via lists.openembedded.org wrote:
> On Thu Sep 4, 2025 at 2:07 AM CEST, a-christidis wrote:
>> From: Antonios Christidis <a-christidis@ti.com>
>>
>> The SDL2 software comes with its own set of tests. Following that
>> introduce support for ptest.
>>
>> Signed-off-by: Antonios Christidis <a-christidis@ti.com>
>> ---
>
> Hi Antonios,
>
> Thanks for the new version.
>
> It looks like we still have a QA issue:
>
> ERROR: nativesdk-libsdl2-2.32.8-r0 do_package: QA Issue: nativesdk-libsdl2: Files/directories were installed but not shipped in any package:
> /usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests
> /usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2
> /usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testsurround.test
> /usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testerror.test
> /usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testtimer.test
> /usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testatomic.test
> /usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testfilesystem.test
> /usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testthread.test
> /usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testaudioinfo.test
> /usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testlocale.test
> /usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testevdev.test
> /usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testpower.test
> /usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testqsort.test
> /usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testplatform.test
> /usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testautomation.test
> /usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/installed-tests/SDL2/testver.test
> Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install.
>
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/19/builds/2327
>
> Can you fix it please?
>
> Thanks,
> Mathieu
Just in case, there are some ptest failures as well:
Failed ptests:
{'libsdl2': ['testdraw2', 'testgeometry', 'testsprite2',
'testgles2', 'testoffscreen']}
https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/2220/steps/12/logs/stdio
--
Best regards,
João Marcos Costa
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [master][PATCH v2] libsdl2: Add tests package and ptest support
2025-09-04 0:07 [master][PATCH v2] libsdl2: Add tests package and ptest support a-christidis
2025-09-04 8:41 ` Mathieu Dubois-Briand
@ 2025-09-04 11:27 ` Alexander Kanavin
2025-09-12 20:54 ` [EXTERNAL] " Antonios Christidis
1 sibling, 1 reply; 6+ messages in thread
From: Alexander Kanavin @ 2025-09-04 11:27 UTC (permalink / raw)
To: a-christidis; +Cc: openembedded-core, mathieu.dubois-briand, alex
On Thu, 4 Sept 2025 at 02:07, <a-christidis@ti.com> wrote:
> +test_array="testdraw2 testgeometry testsprite2 testgles2 testoffscreen"
> +test_path="/usr/libexec/installed-tests/SDL2/"
> +
> +for test in ${test_array}
> +do
> + timeout --preserve-status 10 "$test_path""$test"
> + if [ $? -eq 0 ]; then
> + echo "PASS: $test"
> + else
> + echo "FAIL: $test"
> + fi
> +done
Hardcoding a list of tests is prone to regressions and incomplete coverage.
I applied the patch and checked what actually gets installed. SDL
conforms to gnome's installed tests spec, which means you can inherit
ptest-gnome, and run-ptest should be as simple as:
#! /bin/sh
gnome-desktop-testing-runner SDL2
There are examples in oe-core if you need them for reference.
Obviously you should run it too and check that it passes.
meta-oe has a recipe for libsdl3, whether you want to forward-port the
test support to that recipe is at your discretion, but at some point
libsdl2 will be obsoleted and replaced by that in core.
Alex
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [EXTERNAL] Re: [master][PATCH v2] libsdl2: Add tests package and ptest support
2025-09-04 11:27 ` Alexander Kanavin
@ 2025-09-12 20:54 ` Antonios Christidis
2025-09-16 9:45 ` Alexander Kanavin
0 siblings, 1 reply; 6+ messages in thread
From: Antonios Christidis @ 2025-09-12 20:54 UTC (permalink / raw)
To: Alexander Kanavin; +Cc: openembedded-core, alex
Dear Alexander,
I ran the tests using the gnome-desktop-testing-runner. Here is what I
found:
libsdl2 has 70 tests. Currently, following the gnome standard and being
packaged under /usr/libexec/installed-tests .
18 out of the 70 tests are executed with the
gnome-desktop-testing-runner. The standard only covers tests that run
for a finite amount of time, not infinitely, and requires ".test" files
under /usr/share.
However, when running the 18 tests on qemux86-64 qemuarm, the following
tests fail:
testaudioinfo
testautomation
testlocale
testsurround
Most of the test failures are due to the audio stack, which I do not
have experience with in order to debug.
Here's my questions, what should I do here ?
It is acceptable to create a oe-core only patch for libsdl2, that would
block the generation of the ".test" files for the above tests, ".test"
files are what dictate which tests are run?
Or, I can take a look into updating the version of gnome-desktop-testing
? More recent versions of the software, have a CLI option to exclude
tests. The current packages version within oe-core does not.
I'm leaning on the 2nd option, but want to get your opinion on this
since you have more experience on ptest and related topics.
Kind Regards,
Antonios
On 9/4/25 6:27 AM, Alexander Kanavin wrote:
> On Thu, 4 Sept 2025 at 02: 07, <a-christidis@ ti. com> wrote: >
> +test_array="testdraw2 testgeometry testsprite2 testgles2
> testoffscreen" > +test_path="/usr/libexec/installed-tests/SDL2/" > + >
> +for test in ${test_array} > +do
> ZjQcmQRYFpfptBannerStart
> This message was sent from outside of Texas Instruments.
> Do not click links or open attachments unless you recognize the source
> of this email and know the content is safe.
> Report Suspicious
> <https://us-phishalarm-ewt.proofpoint.com/EWT/v1/G3vK!tDdkczhAu0BQaazEFn6sssEtcdxUhEvtTan3pBnTy9YsgLjvWwOA7YJejpfF1TdW-I8bxkDedD2ZVkPZUvwfaMUJ9SDMz1bIjJfXyt52mx6YGg_dzNkYWXvJTDg$>
>
> ZjQcmQRYFpfptBannerEnd
> On Thu, 4 Sept 2025 at 02:07, <a-christidis@ti.com> wrote:
> > +test_array="testdraw2 testgeometry testsprite2 testgles2 testoffscreen"
> > +test_path="/usr/libexec/installed-tests/SDL2/"
> > +
> > +for test in ${test_array}
> > +do
> > + timeout --preserve-status 10 "$test_path""$test"
> > + if [ $? -eq 0 ]; then
> > + echo "PASS: $test"
> > + else
> > + echo "FAIL: $test"
> > + fi
> > +done
>
> Hardcoding a list of tests is prone to regressions and incomplete coverage.
>
> I applied the patch and checked what actually gets installed. SDL
> conforms to gnome's installed tests spec, which means you can inherit
> ptest-gnome, and run-ptest should be as simple as:
>
> #! /bin/sh
>
> gnome-desktop-testing-runner SDL2
>
> There are examples in oe-core if you need them for reference.
> Obviously you should run it too and check that it passes.
>
> meta-oe has a recipe for libsdl3, whether you want to forward-port the
> test support to that recipe is at your discretion, but at some point
> libsdl2 will be obsoleted and replaced by that in core.
>
> Alex
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [EXTERNAL] Re: [master][PATCH v2] libsdl2: Add tests package and ptest support
2025-09-12 20:54 ` [EXTERNAL] " Antonios Christidis
@ 2025-09-16 9:45 ` Alexander Kanavin
0 siblings, 0 replies; 6+ messages in thread
From: Alexander Kanavin @ 2025-09-16 9:45 UTC (permalink / raw)
To: Antonios Christidis; +Cc: openembedded-core
On Fri, 12 Sept 2025 at 22:54, Antonios Christidis <a-christidis@ti.com> wrote:
> However, when running the 18 tests on qemux86-64 qemuarm, the following
> tests fail:
>
> testaudioinfo
>
> testautomation
>
> testlocale
>
> testsurround
>
> Most of the test failures are due to the audio stack, which I do not
> have experience with in order to debug.
>
> Here's my questions, what should I do here ?
>
> It is acceptable to create a oe-core only patch for libsdl2, that would
> block the generation of the ".test" files for the above tests, ".test"
> files are what dictate which tests are run?
>
> Or, I can take a look into updating the version of gnome-desktop-testing
> ? More recent versions of the software, have a CLI option to exclude
> tests. The current packages version within oe-core does not.
>
> I'm leaning on the 2nd option, but want to get your opinion on this
> since you have more experience on ptest and related topics.
If these tests are failing, then it is not right to just go and
disable them without investigating why.
Maybe the failures are due to real bugs, maybe something is
misconfigured, or a test dependency is missing, we should try and get
to the bottom of it. Otherwise what's the point of running tests in
the first place, if any test failure results in adding the test to the
skip list?
Basically, show what the failures are, provide the patch and the steps
to reproduce them, and we'll try to find a fix. I've fixed many a
ptest over the years, and have a bag of tricks in my head.
Alex
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2025-09-16 9:45 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-09-04 0:07 [master][PATCH v2] libsdl2: Add tests package and ptest support a-christidis
2025-09-04 8:41 ` Mathieu Dubois-Briand
2025-09-04 8:52 ` [OE-core] " Joao Marcos Costa
2025-09-04 11:27 ` Alexander Kanavin
2025-09-12 20:54 ` [EXTERNAL] " Antonios Christidis
2025-09-16 9:45 ` Alexander Kanavin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox