* [PATCH v4] curl : Add ptest
@ 2022-07-06 10:18 Yogesh Tyagi
2022-07-14 13:45 ` [OE-core] " Alexandre Belloni
0 siblings, 1 reply; 2+ messages in thread
From: Yogesh Tyagi @ 2022-07-06 10:18 UTC (permalink / raw)
To: openembedded-core
- curl-ptest is taking around 200 seconds to execute so
added curl-ptest to PTESTS_SLOW
- This patch is rework on an existing patch provided
by Maxin B. John (maxin.john@intel.com)
https://www.openembedded.org/pipermail/openembedded-core/2017-July/139176.html
- Below is the run log of curl-ptest
START: ptest-runner
2022-07-03T15:52
BEGIN: /usr/lib/curl/ptest
********* System characteristics ********
* curl 7.83.1 (x86_64-poky-linux-gnu)
* libcurl/7.83.1 OpenSSL/3.0.3 zlib/1.2.12 libidn2/2.3.2
* Features: alt-svc AsynchDNS Debug HSTS HTTPS-proxy IDN Largefile libz NTLM SSL TLS-SRP UnixSockets
* Disabled: headers-api
* Host: qemux86-64
* System: Linux qemux86-64 5.15.44-yocto-standard #1 SMP PREEMPT Tue May 31 20:28:59 UTC 2022 x86_64 GNU/Linux
* OS: linux
* Servers: HTTP-unix
* Env:
* Seed: 238593
*****************************************
PASS: test 0001 (1 out of 1466, remaining: 25:07, took 1.029s, duration: 00:01)
PASS: test 0002 (2 out of 1466, remaining: 13:21, took 0.065s, duration: 00:01)
...
...
PASS: test 3019 (1460 out of 1466, remaining: 00:00, took 0.012s, duration: 03:16)
PASS: test 3020 (1461 out of 1466, remaining: 00:00, took 0.011s, duration: 03:16)
test 3025...The tool set in the test case for this: 'lib3025' does not exist
TESTDONE: 1280 tests were considered during 197 seconds.
TESTDONE: 783 tests out of PASS: 783 report: 100%
DURATION: 202
END: /usr/lib/curl/ptest
2022-07-03T15:56
STOP: ptest-runner
TOTAL: 1 FAIL: 0
- disable the curl tests that are expected to fail
- remove the generated file configurehelp.pm from curl test beacuse it is causing reproducible build failure.
this file is used by some curl tests to scan symbols from curl headers. we are anyway not installing curl
headers and already have disabled those tests.
[YOCTO #6707]
Signed-off-by: Yogesh Tyagi <yogesh.tyagi@intel.com>
---
v4:removed configurehelp.pm from curl test which is causing reproducible build error
.../distro/include/ptest-packagelists.inc | 1 +
meta/recipes-support/curl/curl/disable-tests | 28 +++++++++++++++++++
meta/recipes-support/curl/curl/run-ptest | 6 ++++
meta/recipes-support/curl/curl_7.84.0.bb | 21 +++++++++++++-
4 files changed, 55 insertions(+), 1 deletion(-)
create mode 100644 meta/recipes-support/curl/curl/disable-tests
create mode 100644 meta/recipes-support/curl/curl/run-ptest
diff --git a/meta/conf/distro/include/ptest-packagelists.inc b/meta/conf/distro/include/ptest-packagelists.inc
index a1ead90649..6c4339e3e1 100644
--- a/meta/conf/distro/include/ptest-packagelists.inc
+++ b/meta/conf/distro/include/ptest-packagelists.inc
@@ -81,6 +81,7 @@ PTESTS_SLOW = "\
babeltrace2-ptest \
busybox-ptest \
coreutils-ptest \
+ curl-ptest \
dbus-ptest \
e2fsprogs-ptest \
findutils-ptest \
diff --git a/meta/recipes-support/curl/curl/disable-tests b/meta/recipes-support/curl/curl/disable-tests
new file mode 100644
index 0000000000..92056bd8ca
--- /dev/null
+++ b/meta/recipes-support/curl/curl/disable-tests
@@ -0,0 +1,28 @@
+# These CRL test (alt-avc) are failing
+356
+412
+413
+# These CRL tests are scanning docs
+971
+1119
+1132
+1135
+# These CRL tests are scnning headers
+1167
+# These CRL tests are scanning man pages
+1139
+1140
+1173
+1177
+# This CRL test is looking for m4 files
+1165
+# This CRL test is looking for src files
+1185
+# These CRL tests need --libcurl option to be enabled
+1400
+1401
+1402
+1403
+1404
+1405
+1465
diff --git a/meta/recipes-support/curl/curl/run-ptest b/meta/recipes-support/curl/curl/run-ptest
new file mode 100644
index 0000000000..614e822922
--- /dev/null
+++ b/meta/recipes-support/curl/curl/run-ptest
@@ -0,0 +1,6 @@
+#!/bin/sh
+cd tests
+./runtests.pl -a -n -s | sed \
+ -e 's|\([^ ]* *\) \([^ ]* *\)...OK|PASS: \1 \2|' \
+ -e 's|\([^ ]* *\) \([^ ]* *\)...FAILED|FAIL: \1 \2|' \
+ -e 's/Warning: test[0-9]\+ not present in tests\/data\/Makefile.inc//'
diff --git a/meta/recipes-support/curl/curl_7.84.0.bb b/meta/recipes-support/curl/curl_7.84.0.bb
index 74a19d20c9..0d829cdf23 100644
--- a/meta/recipes-support/curl/curl_7.84.0.bb
+++ b/meta/recipes-support/curl/curl_7.84.0.bb
@@ -12,13 +12,15 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=190c514872597083303371684954f238"
SRC_URI = " \
https://curl.se/download/${BP}.tar.xz \
file://0001-easy_lock.h-include-sched.h-if-available-to-fix-buil.patch \
+ file://run-ptest \
+ file://disable-tests \
"
SRC_URI[sha256sum] = "2d118b43f547bfe5bae806d8d47b4e596ea5b25a6c1f080aef49fbcd817c5db8"
# Curl has used many names over the years...
CVE_PRODUCT = "haxx:curl haxx:libcurl curl:curl curl:libcurl libcurl:libcurl daniel_stenberg:curl"
-inherit autotools pkgconfig binconfig multilib_header
+inherit autotools pkgconfig binconfig multilib_header ptest
# Entropy source for random PACKAGECONFIG option
RANDOM ?= "/dev/urandom"
@@ -83,6 +85,23 @@ do_install:append:class-target() {
${D}${bindir}/curl-config
}
+do_compile_ptest() {
+ oe_runmake test
+ oe_runmake -C ${B}/tests/server
+}
+
+do_install_ptest() {
+ cat ${WORKDIR}/disable-tests >> ${S}/tests/data/DISABLED
+ rm ${B}/tests/configurehelp.pm
+ cp -rf ${B}/tests ${D}${PTEST_PATH}
+ cp -rf ${S}/tests ${D}${PTEST_PATH}
+ install -d ${D}${PTEST_PATH}/src
+ ln -sf ${bindir}/curl ${D}${PTEST_PATH}/src/curl
+ cp -rf ${D}${bindir}/curl-config ${D}${PTEST_PATH}
+}
+
+RDEPENDS:${PN}-ptest += "bash perl-modules"
+
PACKAGES =+ "lib${BPN}"
FILES:lib${BPN} = "${libdir}/lib*.so.*"
--
2.36.1
^ permalink raw reply related [flat|nested] 2+ messages in thread* Re: [OE-core] [PATCH v4] curl : Add ptest
2022-07-06 10:18 [PATCH v4] curl : Add ptest Yogesh Tyagi
@ 2022-07-14 13:45 ` Alexandre Belloni
0 siblings, 0 replies; 2+ messages in thread
From: Alexandre Belloni @ 2022-07-14 13:45 UTC (permalink / raw)
To: Yogesh Tyagi; +Cc: openembedded-core
Hello,
This made it in master and we found a reproducibility issue. I've opened
this bug to track that:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14863
Could you have a look?
I guess that if the tests don't need the Makefiles, we could just drop
them.
Thanks!
On 06/07/2022 18:18:27+0800, Yogesh Tyagi wrote:
> - curl-ptest is taking around 200 seconds to execute so
> added curl-ptest to PTESTS_SLOW
> - This patch is rework on an existing patch provided
> by Maxin B. John (maxin.john@intel.com)
> https://www.openembedded.org/pipermail/openembedded-core/2017-July/139176.html
> - Below is the run log of curl-ptest
> START: ptest-runner
> 2022-07-03T15:52
> BEGIN: /usr/lib/curl/ptest
> ********* System characteristics ********
> * curl 7.83.1 (x86_64-poky-linux-gnu)
> * libcurl/7.83.1 OpenSSL/3.0.3 zlib/1.2.12 libidn2/2.3.2
> * Features: alt-svc AsynchDNS Debug HSTS HTTPS-proxy IDN Largefile libz NTLM SSL TLS-SRP UnixSockets
> * Disabled: headers-api
> * Host: qemux86-64
> * System: Linux qemux86-64 5.15.44-yocto-standard #1 SMP PREEMPT Tue May 31 20:28:59 UTC 2022 x86_64 GNU/Linux
> * OS: linux
> * Servers: HTTP-unix
> * Env:
> * Seed: 238593
> *****************************************
> PASS: test 0001 (1 out of 1466, remaining: 25:07, took 1.029s, duration: 00:01)
> PASS: test 0002 (2 out of 1466, remaining: 13:21, took 0.065s, duration: 00:01)
> ...
> ...
> PASS: test 3019 (1460 out of 1466, remaining: 00:00, took 0.012s, duration: 03:16)
> PASS: test 3020 (1461 out of 1466, remaining: 00:00, took 0.011s, duration: 03:16)
> test 3025...The tool set in the test case for this: 'lib3025' does not exist
> TESTDONE: 1280 tests were considered during 197 seconds.
> TESTDONE: 783 tests out of PASS: 783 report: 100%
> DURATION: 202
> END: /usr/lib/curl/ptest
> 2022-07-03T15:56
> STOP: ptest-runner
> TOTAL: 1 FAIL: 0
> - disable the curl tests that are expected to fail
> - remove the generated file configurehelp.pm from curl test beacuse it is causing reproducible build failure.
> this file is used by some curl tests to scan symbols from curl headers. we are anyway not installing curl
> headers and already have disabled those tests.
> [YOCTO #6707]
>
> Signed-off-by: Yogesh Tyagi <yogesh.tyagi@intel.com>
> ---
> v4:removed configurehelp.pm from curl test which is causing reproducible build error
>
> .../distro/include/ptest-packagelists.inc | 1 +
> meta/recipes-support/curl/curl/disable-tests | 28 +++++++++++++++++++
> meta/recipes-support/curl/curl/run-ptest | 6 ++++
> meta/recipes-support/curl/curl_7.84.0.bb | 21 +++++++++++++-
> 4 files changed, 55 insertions(+), 1 deletion(-)
> create mode 100644 meta/recipes-support/curl/curl/disable-tests
> create mode 100644 meta/recipes-support/curl/curl/run-ptest
>
> diff --git a/meta/conf/distro/include/ptest-packagelists.inc b/meta/conf/distro/include/ptest-packagelists.inc
> index a1ead90649..6c4339e3e1 100644
> --- a/meta/conf/distro/include/ptest-packagelists.inc
> +++ b/meta/conf/distro/include/ptest-packagelists.inc
> @@ -81,6 +81,7 @@ PTESTS_SLOW = "\
> babeltrace2-ptest \
> busybox-ptest \
> coreutils-ptest \
> + curl-ptest \
> dbus-ptest \
> e2fsprogs-ptest \
> findutils-ptest \
> diff --git a/meta/recipes-support/curl/curl/disable-tests b/meta/recipes-support/curl/curl/disable-tests
> new file mode 100644
> index 0000000000..92056bd8ca
> --- /dev/null
> +++ b/meta/recipes-support/curl/curl/disable-tests
> @@ -0,0 +1,28 @@
> +# These CRL test (alt-avc) are failing
> +356
> +412
> +413
> +# These CRL tests are scanning docs
> +971
> +1119
> +1132
> +1135
> +# These CRL tests are scnning headers
> +1167
> +# These CRL tests are scanning man pages
> +1139
> +1140
> +1173
> +1177
> +# This CRL test is looking for m4 files
> +1165
> +# This CRL test is looking for src files
> +1185
> +# These CRL tests need --libcurl option to be enabled
> +1400
> +1401
> +1402
> +1403
> +1404
> +1405
> +1465
> diff --git a/meta/recipes-support/curl/curl/run-ptest b/meta/recipes-support/curl/curl/run-ptest
> new file mode 100644
> index 0000000000..614e822922
> --- /dev/null
> +++ b/meta/recipes-support/curl/curl/run-ptest
> @@ -0,0 +1,6 @@
> +#!/bin/sh
> +cd tests
> +./runtests.pl -a -n -s | sed \
> + -e 's|\([^ ]* *\) \([^ ]* *\)...OK|PASS: \1 \2|' \
> + -e 's|\([^ ]* *\) \([^ ]* *\)...FAILED|FAIL: \1 \2|' \
> + -e 's/Warning: test[0-9]\+ not present in tests\/data\/Makefile.inc//'
> diff --git a/meta/recipes-support/curl/curl_7.84.0.bb b/meta/recipes-support/curl/curl_7.84.0.bb
> index 74a19d20c9..0d829cdf23 100644
> --- a/meta/recipes-support/curl/curl_7.84.0.bb
> +++ b/meta/recipes-support/curl/curl_7.84.0.bb
> @@ -12,13 +12,15 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=190c514872597083303371684954f238"
> SRC_URI = " \
> https://curl.se/download/${BP}.tar.xz \
> file://0001-easy_lock.h-include-sched.h-if-available-to-fix-buil.patch \
> + file://run-ptest \
> + file://disable-tests \
> "
> SRC_URI[sha256sum] = "2d118b43f547bfe5bae806d8d47b4e596ea5b25a6c1f080aef49fbcd817c5db8"
>
> # Curl has used many names over the years...
> CVE_PRODUCT = "haxx:curl haxx:libcurl curl:curl curl:libcurl libcurl:libcurl daniel_stenberg:curl"
>
> -inherit autotools pkgconfig binconfig multilib_header
> +inherit autotools pkgconfig binconfig multilib_header ptest
>
> # Entropy source for random PACKAGECONFIG option
> RANDOM ?= "/dev/urandom"
> @@ -83,6 +85,23 @@ do_install:append:class-target() {
> ${D}${bindir}/curl-config
> }
>
> +do_compile_ptest() {
> + oe_runmake test
> + oe_runmake -C ${B}/tests/server
> +}
> +
> +do_install_ptest() {
> + cat ${WORKDIR}/disable-tests >> ${S}/tests/data/DISABLED
> + rm ${B}/tests/configurehelp.pm
> + cp -rf ${B}/tests ${D}${PTEST_PATH}
> + cp -rf ${S}/tests ${D}${PTEST_PATH}
> + install -d ${D}${PTEST_PATH}/src
> + ln -sf ${bindir}/curl ${D}${PTEST_PATH}/src/curl
> + cp -rf ${D}${bindir}/curl-config ${D}${PTEST_PATH}
> +}
> +
> +RDEPENDS:${PN}-ptest += "bash perl-modules"
> +
> PACKAGES =+ "lib${BPN}"
>
> FILES:lib${BPN} = "${libdir}/lib*.so.*"
> --
> 2.36.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#167716): https://lists.openembedded.org/g/openembedded-core/message/167716
> Mute This Topic: https://lists.openembedded.org/mt/92203371/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2022-07-14 13:45 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-07-06 10:18 [PATCH v4] curl : Add ptest Yogesh Tyagi
2022-07-14 13:45 ` [OE-core] " Alexandre Belloni
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox