[PATCH v2] libssh2: Upgrade to v1.11.0

[PATCH v2] libssh2: Upgrade to v1.11.0

[Sudip Mukherjee]

From: Sudip Mukherjee <sudipm.mukherjee@gmail.com>

Changes:
Update license checksum: change in copyright year.
Remove patch fixed by upstream.
Disable rpath, hardcoded rpath failed curl-native build.
Adjust ptests.
Remove --disable-static from OECONF

Signed-off-by: Sudip Mukherjee <sudipm.mukherjee@gmail.com>
---

v2: Remove --disable-static from OECONF.
Only needed for poky as its using --disable-static and that fails ptest builds.


 .../libssh2/libssh2/fix-ssh2-test.patch       | 23 -------------------
 .../recipes-support/libssh2/libssh2/run-ptest |  2 +-
 .../{libssh2_1.10.0.bb => libssh2_1.11.0.bb}  | 17 +++++++++-----
 3 files changed, 12 insertions(+), 30 deletions(-)
 delete mode 100644 meta/recipes-support/libssh2/libssh2/fix-ssh2-test.patch
 rename meta/recipes-support/libssh2/{libssh2_1.10.0.bb => libssh2_1.11.0.bb} (65%)

diff --git a/meta/recipes-support/libssh2/libssh2/fix-ssh2-test.patch b/meta/recipes-support/libssh2/libssh2/fix-ssh2-test.patch
deleted file mode 100644
index ee916c42d4..0000000000
--- a/meta/recipes-support/libssh2/libssh2/fix-ssh2-test.patch
+++ /dev/null
@@ -1,23 +0,0 @@
-In 8.8 OpenSSH disabled sha1 rsa-sha keys out of the box,
-so we need to re-enable them as a workaround for the test
-suite until upstream updates the tests.
-
-See: https://github.com/libssh2/libssh2/issues/630
-
-Upstream-Status: Backport [alternative fixes merged upstream]
-
-Patch taken from https://github.com/mirror-rpm/libssh2/commit/47f7114f7d0780f3075bad51a71881f45cc933c5
-
---- a/tests/ssh2.sh
-+++ b/tests/ssh2.sh
-@@ -25,7 +25,8 @@ $SSHD -f /dev/null -h "$srcdir"/etc/host
-     -o 'Port 4711' \
-     -o 'Protocol 2' \
-     -o "AuthorizedKeysFile $srcdir/etc/user.pub" \
--    -o 'UsePrivilegeSeparation no' \
-+    -o 'HostKeyAlgorithms +ssh-rsa' \
-+    -o 'PubkeyAcceptedAlgorithms +ssh-rsa' \
-     -o 'StrictModes no' \
-     -D \
-     $libssh2_sshd_params &
-
diff --git a/meta/recipes-support/libssh2/libssh2/run-ptest b/meta/recipes-support/libssh2/libssh2/run-ptest
index 5e7426f79d..ed87074fcd 100644
--- a/meta/recipes-support/libssh2/libssh2/run-ptest
+++ b/meta/recipes-support/libssh2/libssh2/run-ptest
@@ -2,7 +2,7 @@
 
 ptestdir=$(dirname "$(readlink -f "$0")")
 cd tests
-for test in simple mansyntax.sh ssh2.sh
+for test in test_simple mansyntax.sh test_sshd.test
 do
 	./../test-driver --test-name $test --log-file ../$test.log --trs-file ../$test.trs --color-tests no --enable-hard-errors yes --expect-failure no -- ./$test
 done
diff --git a/meta/recipes-support/libssh2/libssh2_1.10.0.bb b/meta/recipes-support/libssh2/libssh2_1.11.0.bb
similarity index 65%
rename from meta/recipes-support/libssh2/libssh2_1.10.0.bb
rename to meta/recipes-support/libssh2/libssh2_1.11.0.bb
index d5513373b0..eff2f043a5 100644
--- a/meta/recipes-support/libssh2/libssh2_1.10.0.bb
+++ b/meta/recipes-support/libssh2/libssh2_1.11.0.bb
@@ -5,21 +5,22 @@ SECTION = "libs"
 DEPENDS = "zlib"
 
 LICENSE = "BSD-3-Clause"
-LIC_FILES_CHKSUM = "file://COPYING;md5=3e089ad0cf27edf1e7f261dfcd06acc7"
+LIC_FILES_CHKSUM = "file://COPYING;md5=24a33237426720395ebb1dd1349ca225"
 
 SRC_URI = "http://www.libssh2.org/download/${BP}.tar.gz \
-           file://fix-ssh2-test.patch \
            file://run-ptest \
            "
 
-SRC_URI[sha256sum] = "2d64e90f3ded394b91d3a2e774ca203a4179f69aebee03003e5a6fa621e41d51"
+SRC_URI[sha256sum] = "3736161e41e2693324deb38c26cfdc3efe6209d634ba4258db1cecff6a5ad461"
 
 inherit autotools pkgconfig ptest
 
 EXTRA_OECONF += "\
                  --with-libz \
                  --with-libz-prefix=${STAGING_LIBDIR} \
+                 --disable-rpath \
                 "
+EXTRA_OECONF:remove = "--disable-static"
 
 # only one of openssl and gcrypt could be set
 PACKAGECONFIG ??= "openssl"
@@ -41,9 +42,13 @@ do_install_ptest() {
 	install -d ${D}${PTEST_PATH}/tests
 	install -m 0755 ${S}/test-driver ${D}${PTEST_PATH}/
 	cp -rf ${B}/tests/.libs/* ${D}${PTEST_PATH}/tests/
-	cp -rf ${S}/tests/mansyntax.sh  ${D}${PTEST_PATH}/tests/
-	cp -rf ${S}/tests/ssh2.sh  ${D}${PTEST_PATH}/tests/
-	cp -rf ${S}/tests/etc ${D}${PTEST_PATH}/tests/
+	cp -rf ${B}/tests/test_simple ${D}${PTEST_PATH}/tests/
+	cp -rf ${S}/tests/mansyntax.sh ${D}${PTEST_PATH}/tests/
+	cp -rf ${S}/tests/test_sshd.test  ${D}${PTEST_PATH}/tests/
+	cp -rf ${S}/tests/openssh_server ${D}${PTEST_PATH}/tests/
+	cp -rf ${S}/tests/key* ${D}${PTEST_PATH}/tests/
+# Makefile.inc required as test_sshd.test generates the list of test from this
+	cp -rf ${S}/tests/Makefile.inc ${D}${PTEST_PATH}/tests/
 	mkdir -p ${D}${PTEST_PATH}/docs
 	cp -r ${S}/docs/* ${D}${PTEST_PATH}/docs/
 }
-- 
2.30.2

Re: [OE-core] [PATCH v2] libssh2: Upgrade to v1.11.0

[Ross Burton]

On 13 Jun 2023, at 21:52, Sudip Mukherjee via lists.openembedded.org <sudipm.mukherjee=gmail.com@lists.openembedded.org> wrote:
> # only one of openssl and gcrypt could be set
> PACKAGECONFIG ??= "openssl"
> @@ -41,9 +42,13 @@ do_install_ptest() {
> install -d ${D}${PTEST_PATH}/tests
> install -m 0755 ${S}/test-driver ${D}${PTEST_PATH}/
> cp -rf ${B}/tests/.libs/* ${D}${PTEST_PATH}/tests/
> - cp -rf ${S}/tests/mansyntax.sh  ${D}${PTEST_PATH}/tests/
> - cp -rf ${S}/tests/ssh2.sh  ${D}${PTEST_PATH}/tests/
> - cp -rf ${S}/tests/etc ${D}${PTEST_PATH}/tests/
> + cp -rf ${B}/tests/test_simple ${D}${PTEST_PATH}/tests/
> + cp -rf ${S}/tests/mansyntax.sh ${D}${PTEST_PATH}/tests/
> + cp -rf ${S}/tests/test_sshd.test  ${D}${PTEST_PATH}/tests/
> + cp -rf ${S}/tests/openssh_server ${D}${PTEST_PATH}/tests/
> + cp -rf ${S}/tests/key* ${D}${PTEST_PATH}/tests/
> +# Makefile.inc required as test_sshd.test generates the list of test from this
> + cp -rf ${S}/tests/Makefile.inc ${D}${PTEST_PATH}/tests/
> mkdir -p ${D}${PTEST_PATH}/docs
> cp -r ${S}/docs/* ${D}${PTEST_PATH}/docs/
> }

As you’ve clearly been looking at the tests, why does the ptest support need the documentation?

Ross

Re: [OE-core] [PATCH v2] libssh2: Upgrade to v1.11.0

[Sudip Mukherjee]

Hi Ross,

On Wed, 14 Jun 2023 at 14:53, Ross Burton <ross.burton@arm.com> wrote:
>
> On 13 Jun 2023, at 21:52, Sudip Mukherjee via lists.openembedded.org <sudipm.mukherjee=gmail.com@lists.openembedded.org> wrote:
> > # only one of openssl and gcrypt could be set
> > PACKAGECONFIG ??= "openssl"
> > @@ -41,9 +42,13 @@ do_install_ptest() {
> > install -d ${D}${PTEST_PATH}/tests
> > install -m 0755 ${S}/test-driver ${D}${PTEST_PATH}/
> > cp -rf ${B}/tests/.libs/* ${D}${PTEST_PATH}/tests/
> > - cp -rf ${S}/tests/mansyntax.sh  ${D}${PTEST_PATH}/tests/
> > - cp -rf ${S}/tests/ssh2.sh  ${D}${PTEST_PATH}/tests/
> > - cp -rf ${S}/tests/etc ${D}${PTEST_PATH}/tests/
> > + cp -rf ${B}/tests/test_simple ${D}${PTEST_PATH}/tests/
> > + cp -rf ${S}/tests/mansyntax.sh ${D}${PTEST_PATH}/tests/
> > + cp -rf ${S}/tests/test_sshd.test  ${D}${PTEST_PATH}/tests/
> > + cp -rf ${S}/tests/openssh_server ${D}${PTEST_PATH}/tests/
> > + cp -rf ${S}/tests/key* ${D}${PTEST_PATH}/tests/
> > +# Makefile.inc required as test_sshd.test generates the list of test from this
> > + cp -rf ${S}/tests/Makefile.inc ${D}${PTEST_PATH}/tests/
> > mkdir -p ${D}${PTEST_PATH}/docs
> > cp -r ${S}/docs/* ${D}${PTEST_PATH}/docs/
> > }
>
> As you’ve clearly been looking at the tests, why does the ptest support need the documentation?

There is a test for manpage syntax "mansyntax.sh" which needs the
generated man pages in the docs folder.
compile_ptest_base will not fail, but actual tests will fail.

-- 
Regards
Sudip

Re: [OE-core] [PATCH v2] libssh2: Upgrade to v1.11.0

[Ross Burton]

On 14 Jun 2023, at 16:13, Sudip Mukherjee <sudipm.mukherjee@gmail.com> wrote:
> 
> Hi Ross,
> 
> On Wed, 14 Jun 2023 at 14:53, Ross Burton <ross.burton@arm.com> wrote:
>> 
>> On 13 Jun 2023, at 21:52, Sudip Mukherjee via lists.openembedded.org <sudipm.mukherjee=gmail.com@lists.openembedded.org> wrote:
>>> # only one of openssl and gcrypt could be set
>>> PACKAGECONFIG ??= "openssl"
>>> @@ -41,9 +42,13 @@ do_install_ptest() {
>>> install -d ${D}${PTEST_PATH}/tests
>>> install -m 0755 ${S}/test-driver ${D}${PTEST_PATH}/
>>> cp -rf ${B}/tests/.libs/* ${D}${PTEST_PATH}/tests/
>>> - cp -rf ${S}/tests/mansyntax.sh  ${D}${PTEST_PATH}/tests/
>>> - cp -rf ${S}/tests/ssh2.sh  ${D}${PTEST_PATH}/tests/
>>> - cp -rf ${S}/tests/etc ${D}${PTEST_PATH}/tests/
>>> + cp -rf ${B}/tests/test_simple ${D}${PTEST_PATH}/tests/
>>> + cp -rf ${S}/tests/mansyntax.sh ${D}${PTEST_PATH}/tests/
>>> + cp -rf ${S}/tests/test_sshd.test  ${D}${PTEST_PATH}/tests/
>>> + cp -rf ${S}/tests/openssh_server ${D}${PTEST_PATH}/tests/
>>> + cp -rf ${S}/tests/key* ${D}${PTEST_PATH}/tests/
>>> +# Makefile.inc required as test_sshd.test generates the list of test from this
>>> + cp -rf ${S}/tests/Makefile.inc ${D}${PTEST_PATH}/tests/
>>> mkdir -p ${D}${PTEST_PATH}/docs
>>> cp -r ${S}/docs/* ${D}${PTEST_PATH}/docs/
>>> }
>> 
>> As you’ve clearly been looking at the tests, why does the ptest support need the documentation?
> 
> There is a test for manpage syntax "mansyntax.sh" which needs the
> generated man pages in the docs folder.
> compile_ptest_base will not fail, but actual tests will fail.

This is where I start to say that we should be trimming the tests and only running useful tests for integration purposes, not the complete upstream test suite.

Ross

Re: [OE-core] [PATCH v2] libssh2: Upgrade to v1.11.0

[Sudip Mukherjee]

On Wed, 14 Jun 2023 at 16:47, Ross Burton <Ross.Burton@arm.com> wrote:
>
> On 14 Jun 2023, at 16:13, Sudip Mukherjee <sudipm.mukherjee@gmail.com> wrote:
> >
> > Hi Ross,
> >
> > On Wed, 14 Jun 2023 at 14:53, Ross Burton <ross.burton@arm.com> wrote:
> >>
> >> On 13 Jun 2023, at 21:52, Sudip Mukherjee via lists.openembedded.org <sudipm.mukherjee=gmail.com@lists.openembedded.org> wrote:
> >>> # only one of openssl and gcrypt could be set

<snip>

> >>> mkdir -p ${D}${PTEST_PATH}/docs
> >>> cp -r ${S}/docs/* ${D}${PTEST_PATH}/docs/
> >>> }
> >>
> >> As you’ve clearly been looking at the tests, why does the ptest support need the documentation?
> >
> > There is a test for manpage syntax "mansyntax.sh" which needs the
> > generated man pages in the docs folder.
> > compile_ptest_base will not fail, but actual tests will fail.
>
> This is where I start to say that we should be trimming the tests and only running useful tests for integration purposes, not the complete upstream test suite.

Sure, and as confirmed over irc I will send a followup patch removing that test.

-- 
Regards
Sudip

Re: [OE-core] [PATCH v2] libssh2: Upgrade to v1.11.0

[Alexandre Belloni]

On 14/06/2023 21:29:56+0100, Sudip Mukherjee wrote:
> On Wed, 14 Jun 2023 at 16:47, Ross Burton <Ross.Burton@arm.com> wrote:
> >
> > On 14 Jun 2023, at 16:13, Sudip Mukherjee <sudipm.mukherjee@gmail.com> wrote:
> > >
> > > Hi Ross,
> > >
> > > On Wed, 14 Jun 2023 at 14:53, Ross Burton <ross.burton@arm.com> wrote:
> > >>
> > >> On 13 Jun 2023, at 21:52, Sudip Mukherjee via lists.openembedded.org <sudipm.mukherjee=gmail.com@lists.openembedded.org> wrote:
> > >>> # only one of openssl and gcrypt could be set
> 
> <snip>
> 
> > >>> mkdir -p ${D}${PTEST_PATH}/docs
> > >>> cp -r ${S}/docs/* ${D}${PTEST_PATH}/docs/
> > >>> }
> > >>
> > >> As you’ve clearly been looking at the tests, why does the ptest support need the documentation?
> > >
> > > There is a test for manpage syntax "mansyntax.sh" which needs the
> > > generated man pages in the docs folder.
> > > compile_ptest_base will not fail, but actual tests will fail.
> >
> > This is where I start to say that we should be trimming the tests and only running useful tests for integration purposes, not the complete upstream test suite.
> 
> Sure, and as confirmed over irc I will send a followup patch removing that test.
> 
> -- 
> Regards
> Sudip

Note that test_sshd.test is failing, full log here:
https://autobuilder.yocto.io/pub/non-release/20230614-11/testresults/qemux86-64-ptest/core-image-ptest-libssh2/log.do_testimage.50695.20230614084246

-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH v2] libssh2: Upgrade to v1.11.0

[Sudip Mukherjee]

[-- Attachment #1: Type: text/plain, Size: 2145 bytes --]

On Wed, 14 Jun 2023 at 23:38, Alexandre Belloni
<alexandre.belloni@bootlin.com> wrote:
>
> On 14/06/2023 21:29:56+0100, Sudip Mukherjee wrote:
> > On Wed, 14 Jun 2023 at 16:47, Ross Burton <Ross.Burton@arm.com> wrote:
> > >
> > > On 14 Jun 2023, at 16:13, Sudip Mukherjee <sudipm.mukherjee@gmail.com> wrote:
> > > >
> > > > Hi Ross,
> > > >
> > > > On Wed, 14 Jun 2023 at 14:53, Ross Burton <ross.burton@arm.com> wrote:
> > > >>
> > > >> On 13 Jun 2023, at 21:52, Sudip Mukherjee via lists.openembedded.org <sudipm.mukherjee=gmail.com@lists.openembedded.org> wrote:
> > > >>> # only one of openssl and gcrypt could be set
> >
> > <snip>
> >
> > > >>> mkdir -p ${D}${PTEST_PATH}/docs
> > > >>> cp -r ${S}/docs/* ${D}${PTEST_PATH}/docs/
> > > >>> }
> > > >>
> > > >> As you’ve clearly been looking at the tests, why does the ptest support need the documentation?
> > > >
> > > > There is a test for manpage syntax "mansyntax.sh" which needs the
> > > > generated man pages in the docs folder.
> > > > compile_ptest_base will not fail, but actual tests will fail.
> > >
> > > This is where I start to say that we should be trimming the tests and only running useful tests for integration purposes, not the complete upstream test suite.
> >
> > Sure, and as confirmed over irc I will send a followup patch removing that test.
> >
> > --
> > Regards
> > Sudip
>
> Note that test_sshd.test is failing, full log here:
> https://autobuilder.yocto.io/pub/non-release/20230614-11/testresults/qemux86-64-ptest/core-image-ptest-libssh2/log.do_testimage.50695.20230614084246

So, I think I will need some help here as I am unable to reproduce
this failure. All tests are passing in my setup.
I have applied the libssh2 patch on top of
3da65f786c0a370ead26022e9cf716c1b18bd5cb in poky.
And result from 'do_testimage' shows:

BEGIN: /usr/lib/libssh2/ptest
PASS: test_simple
PASS: mansyntax.sh
PASS: test_sshd.test
DURATION: 9
END: /usr/lib/libssh2/ptest

I am attaching the log from my test. Will really appreciate it if you
can please give me the steps to reproduce the failure.


-- 
Regards
Sudip

[-- Attachment #2: log.do_testimage.gz --]
[-- Type: application/gzip, Size: 111769 bytes --]