[meta-oe][PATCH V1] lz4: Add ptest support

[meta-oe][PATCH V1] lz4: Add ptest support

[qiutt]

From: Qiu Tingting <qiutt@fujitsu.com>

Add a ptest for lz4.
- It is taking around 25 min to execute so added it to PTEST_SLOW.
- It contains two case: test_frame, test_fuzzer.
- Below is the run log:
  START: ptest-runner
  2023-04-04T08:28
  BEGIN: /usr/lib/lz4/ptest
  Starting lz4frame tester (64-bits, 1.9.4)
  Seed = 485
  Basic tests completed
  All tests completed
  PASS: lz4/test_frame
  Starting LZ4 fuzzer (64-bits, v1.9.4)
  Seed = 7154
  Overflow tests : 64 bits mode : no overflow
  All unit tests completed successfully compressionLevel=9
  Overflow tests : 64 bits mode : no overflow
  All unit tests completed successfully compressionLevel=10
  generating low buffer at address 0x1000
    65536 /  65536   - all tests completed successfully
  compression ratio: 54.326%
  HC compression ratio: 51.129%
  ratio with dict: 34.335%
  PASS: lz4/test_fuzzer
  DURATION: 1518
  END: /usr/lib/lz4/ptest
  2023-04-04T08:53
  STOP: ptest-runner
  TOTAL: 1 FAIL: 0

Signed-off-by: Qiu Tingting <qiutt@fujitsu.com>
---
 .../distro/include/ptest-packagelists.inc     |  3 +--
 meta/recipes-support/lz4/files/run-ptest      | 26 +++++++++++++++++++
 meta/recipes-support/lz4/lz4_1.9.4.bb         | 21 ++++++++++++++-
 3 files changed, 47 insertions(+), 3 deletions(-)
 create mode 100644 meta/recipes-support/lz4/files/run-ptest

diff --git a/meta/conf/distro/include/ptest-packagelists.inc b/meta/conf/distro/include/ptest-packagelists.inc
index d60ebb38fc..d6351466f2 100644
--- a/meta/conf/distro/include/ptest-packagelists.inc
+++ b/meta/conf/distro/include/ptest-packagelists.inc
@@ -105,6 +105,7 @@ PTESTS_SLOW = "\
     tcl \
     util-linux \
     valgrind \
+    lz4 \
 "
 
 PTESTS_SLOW:remove:riscv64 = "valgrind"
@@ -114,7 +115,6 @@ PTESTS_SLOW:remove:x86 = "valgrind"
 PTESTS_PROBLEMS:append:x86 = " valgrind"
 
 #    ruby \ # Timeout
-#    lz4 \ # Needs a rewrite
 #    rt-tests \ # Needs to be checked whether it runs at all
 #    bash \ # Test outcomes are non-deterministic by design
 #    ifupdown \ # Tested separately in lib/oeqa/selftest/cases/imagefeatures.py
@@ -127,7 +127,6 @@ PTESTS_PROBLEMS:append:x86 = " valgrind"
 
 PTESTS_PROBLEMS = "\
     ruby \
-    lz4 \
     rt-tests \
     bash \
     ifupdown \
diff --git a/meta/recipes-support/lz4/files/run-ptest b/meta/recipes-support/lz4/files/run-ptest
new file mode 100644
index 0000000000..278db179ac
--- /dev/null
+++ b/meta/recipes-support/lz4/files/run-ptest
@@ -0,0 +1,26 @@
+#!/bin/sh
+
+# Define test work dir
+WORKDIR=@PTEST_PATH@/tests/
+
+# Run test
+cd ${WORKDIR}
+
+case_name="test_frame"
+./frametest 
+ret=$?
+if [[ ${ret} == "0" ]]; then
+    echo "PASS: lz4/$case_name"
+else
+    echo "FAIL: lz4/$case_name"
+fi
+
+case_name="test_fuzzer"
+./fuzzer
+ret=$?
+if [[ ${ret} == "0" ]]; then
+    echo "PASS: lz4/$case_name"
+else
+    echo "FAIL: lz4/$case_name"
+fi
+
diff --git a/meta/recipes-support/lz4/lz4_1.9.4.bb b/meta/recipes-support/lz4/lz4_1.9.4.bb
index a2a178bab5..5b57dd4639 100644
--- a/meta/recipes-support/lz4/lz4_1.9.4.bb
+++ b/meta/recipes-support/lz4/lz4_1.9.4.bb
@@ -12,11 +12,15 @@ PE = "1"
 
 SRCREV = "5ff839680134437dbf4678f3d0c7b371d84f4964"
 
-SRC_URI = "git://github.com/lz4/lz4.git;branch=release;protocol=https"
+SRC_URI = "git://github.com/lz4/lz4.git;branch=release;protocol=https \
+	   file://run-ptest \
+	   "
 UPSTREAM_CHECK_GITTAGREGEX = "v(?P<pver>.*)"
 
 S = "${WORKDIR}/git"
 
+inherit ptest
+
 # Fixed in r118, which is larger than the current version.
 CVE_CHECK_IGNORE += "CVE-2014-4715"
 
@@ -27,3 +31,18 @@ do_install() {
 }
 
 BBCLASSEXTEND = "native nativesdk"
+
+RDEPENDS:${PN}-ptest += "bash"
+
+do_compile_ptest() {
+        oe_runmake -C ${B}/tests/
+}
+
+do_install_ptest() {
+	install -d ${D}${PTEST_PATH}/tests/
+	install --mode=755 ${B}/tests/frametest ${D}${PTEST_PATH}/tests/
+	install --mode=755 ${B}/tests/fuzzer ${D}${PTEST_PATH}/tests/
+	sed -i "s#@PTEST_PATH@#${PTEST_PATH}#g" ${D}${PTEST_PATH}/run-ptest
+
+}
+
-- 
2.25.1

Re: [OE-core] [meta-oe][PATCH V1] lz4: Add ptest support

[Richard Purdie]

On Tue, 2023-04-04 at 17:59 +0800, qiutt@fujitsu.com wrote:
> From: Qiu Tingting <qiutt@fujitsu.com>
> 
> Add a ptest for lz4.
> - It is taking around 25 min to execute so added it to PTEST_SLOW.
> - It contains two case: test_frame, test_fuzzer.
> - Below is the run log:
>   START: ptest-runner
>   2023-04-04T08:28
>   BEGIN: /usr/lib/lz4/ptest
>   Starting lz4frame tester (64-bits, 1.9.4)
>   Seed = 485
>   Basic tests completed
>   All tests completed
>   PASS: lz4/test_frame
>   Starting LZ4 fuzzer (64-bits, v1.9.4)
>   Seed = 7154
>   Overflow tests : 64 bits mode : no overflow
>   All unit tests completed successfully compressionLevel=9
>   Overflow tests : 64 bits mode : no overflow
>   All unit tests completed successfully compressionLevel=10
>   generating low buffer at address 0x1000
>     65536 /  65536   - all tests completed successfully
>   compression ratio: 54.326%
>   HC compression ratio: 51.129%
>   ratio with dict: 34.335%
>   PASS: lz4/test_fuzzer
>   DURATION: 1518
>   END: /usr/lib/lz4/ptest
>   2023-04-04T08:53
>   STOP: ptest-runner
>   TOTAL: 1 FAIL: 0
> 
> Signed-off-by: Qiu Tingting <qiutt@fujitsu.com>

25 minutes seems slow, is that with KVM? How much of the time is in the
frame test and how much is in the fuzzer?

We should probably just run the frame tests and skip the fuzzing as
we're interested in integration testing, not random tests!

Cheers,

Richard

RE: [OE-core] [meta-oe][PATCH V1] lz4: Add ptest support

[qiutt]

Hi, Richard

> -----Original Message-----
> From: Richard Purdie <richard.purdie@linuxfoundation.org>
> Sent: Tuesday, April 4, 2023 10:17 PM
> To: Qiu, Tingting/仇 婷婷 <qiutt@fujitsu.com>;
> openembedded-core@lists.openembedded.org
> Cc: FNST fnstml-fujitsuten <fnstml-fujitsuten@fujitsu.com>
> Subject: Re: [OE-core] [meta-oe][PATCH V1] lz4: Add ptest support
> 
> On Tue, 2023-04-04 at 17:59 +0800, qiutt@fujitsu.com wrote:
> > From: Qiu Tingting <qiutt@fujitsu.com>
> >
> > Add a ptest for lz4.
> > - It is taking around 25 min to execute so added it to PTEST_SLOW.
> > - It contains two case: test_frame, test_fuzzer.
> > - Below is the run log:
> >   START: ptest-runner
> >   2023-04-04T08:28
> >   BEGIN: /usr/lib/lz4/ptest
> >   Starting lz4frame tester (64-bits, 1.9.4)
> >   Seed = 485
> >   Basic tests completed
> >   All tests completed
> >   PASS: lz4/test_frame
> >   Starting LZ4 fuzzer (64-bits, v1.9.4)
> >   Seed = 7154
> >   Overflow tests : 64 bits mode : no overflow
> >   All unit tests completed successfully compressionLevel=9
> >   Overflow tests : 64 bits mode : no overflow
> >   All unit tests completed successfully compressionLevel=10
> >   generating low buffer at address 0x1000
> >     65536 /  65536   - all tests completed successfully
> >   compression ratio: 54.326%
> >   HC compression ratio: 51.129%
> >   ratio with dict: 34.335%
> >   PASS: lz4/test_fuzzer
> >   DURATION: 1518
> >   END: /usr/lib/lz4/ptest
> >   2023-04-04T08:53
> >   STOP: ptest-runner
> >   TOTAL: 1 FAIL: 0
> >
> > Signed-off-by: Qiu Tingting <qiutt@fujitsu.com>
> 
> 25 minutes seems slow, is that with KVM? How much of the time is in the
> frame test and how much is in the fuzzer?
Yes, that is with KVM.
Frame test : 9 min
Fuzzer test: 16 min

> 
> We should probably just run the frame tests and skip the fuzzing as we're
> interested in integration testing, not random tests!

I will send a new patch for frame tests only.

Best regards,
Qiu Tingting

> 
> Cheers,
> 
> Richard
>