QEMU cirrus freebsd 13 CI failing with 'Undefined symbol "rl_set_timeout"'

QEMU cirrus freebsd 13 CI failing with 'Undefined symbol "rl_set_timeout"'

[Peter Maydell]

I've just noticed that our (optional) FreeBSD 13 CI job is
failing while running the qemu-iotests, like this:

+ld-elf.so.1: /usr/local/bin/bash: Undefined symbol "rl_set_timeout"

Full job logs from a couple of sample builds:
https://cirrus-ci.com/task/6541458329567232
https://cirrus-ci.com/task/6036627739377664

Any idea what this is about? It looks at first glance like
the bash on the CI system is busted because it can't find
libreadline, but maybe I'm missing something. Are we missing
some runtime shared library from a config file?

(I didn't notice this immediately because our 'make check'
logging is so voluminous that gitlab truncates it, so you
have to click through to the gitlab job log, find the cirrus
URL, then go look at it in Cirrus. We still really need to
trim down the amount of output we're emitting here.)

thanks
-- PMM

Re: QEMU cirrus freebsd 13 CI failing with 'Undefined symbol "rl_set_timeout"'

[Daniel P. Berrangé]

On Mon, Jan 09, 2023 at 05:14:38PM +0000, Peter Maydell wrote:
> I've just noticed that our (optional) FreeBSD 13 CI job is
> failing while running the qemu-iotests, like this:
> 
> +ld-elf.so.1: /usr/local/bin/bash: Undefined symbol "rl_set_timeout"
> 
> Full job logs from a couple of sample builds:
> https://cirrus-ci.com/task/6541458329567232
> https://cirrus-ci.com/task/6036627739377664
> 
> Any idea what this is about? It looks at first glance like
> the bash on the CI system is busted because it can't find
> libreadline, but maybe I'm missing something. Are we missing
> some runtime shared library from a config file?

Usually this kind of thing happens when FreeBSD issue a new
minor release. The ports build will pick up a dependency on
an API in the new release, and the ports package manager
never checks that it is running on the current base image.

In this case though, we're already running on FreeBSD 13.1,
which is most current release, and 13.2 isn't due for at
least 2 months. So my usual fix of updating the base image
version won't solve this.

It does feel like the ports 'bash' build is broken.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: QEMU cirrus freebsd 13 CI failing with 'Undefined symbol "rl_set_timeout"'

[Li-Wen Hsu]

[-- Attachment #1: Type: text/plain, Size: 1856 bytes --]

On Tue, Jan 10, 2023 at 2:25 AM Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> On Mon, Jan 09, 2023 at 05:14:38PM +0000, Peter Maydell wrote:
> > I've just noticed that our (optional) FreeBSD 13 CI job is
> > failing while running the qemu-iotests, like this:
> >
> > +ld-elf.so.1: /usr/local/bin/bash: Undefined symbol "rl_set_timeout"
> >
> > Full job logs from a couple of sample builds:
> > https://cirrus-ci.com/task/6541458329567232
> > https://cirrus-ci.com/task/6036627739377664
> >
> > Any idea what this is about? It looks at first glance like
> > the bash on the CI system is busted because it can't find
> > libreadline, but maybe I'm missing something. Are we missing
> > some runtime shared library from a config file?
>
> Usually this kind of thing happens when FreeBSD issue a new
> minor release. The ports build will pick up a dependency on
> an API in the new release, and the ports package manager
> never checks that it is running on the current base image.
>
> In this case though, we're already running on FreeBSD 13.1,
> which is most current release, and 13.2 isn't due for at
> least 2 months. So my usual fix of updating the base image
> version won't solve this.
>
> It does feel like the ports 'bash' build is broken.

TL;DR: The workaround would be upgrading readline package or more
extensively, do a `pkg upgrade -y` before install other packages.

rl_set_timeout is in libreadline.so which is installed by readline
pacakge, and that package was pre-installed when installing other
packages needed by the GCP image, which is used for cirrus-ci.  It
seems that the readline package on image is too old and doesn't meet
the requirement of bash and caused this issue.

I can reproduce this and confirmed the workaround.  I attached a patch
and hope that it helps.

Best,
Li-Wen

[-- Attachment #2: 0001-Upgrade-all-packages-in-the-VM-to-ensure-the-freshne.patch --]
[-- Type: text/x-patch, Size: 1060 bytes --]

From def139241759d2bcba70ed5cc7dff522f4cb6753 Mon Sep 17 00:00:00 2001
From: Li-Wen Hsu <lwhsu@lwhsu.org>
Date: Tue, 10 Jan 2023 04:32:01 +0800
Subject: [PATCH] Upgrade all packages in the VM to ensure the freshness

Signed-off-by: Li-Wen Hsu <lwhsu@lwhsu.org>
---
 .gitlab-ci.d/cirrus.yml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml
index 785b163aa6..502dfd612c 100644
--- a/.gitlab-ci.d/cirrus.yml
+++ b/.gitlab-ci.d/cirrus.yml
@@ -53,7 +53,7 @@ x64-freebsd-12-build:
     CIRRUS_VM_IMAGE_NAME: freebsd-12-4
     CIRRUS_VM_CPUS: 8
     CIRRUS_VM_RAM: 8G
-    UPDATE_COMMAND: pkg update
+    UPDATE_COMMAND: pkg update; pkg upgrade -y
     INSTALL_COMMAND: pkg install -y
     TEST_TARGETS: check
 
@@ -66,7 +66,7 @@ x64-freebsd-13-build:
     CIRRUS_VM_IMAGE_NAME: freebsd-13-1
     CIRRUS_VM_CPUS: 8
     CIRRUS_VM_RAM: 8G
-    UPDATE_COMMAND: pkg update
+    UPDATE_COMMAND: pkg update; pkg upgrade -y
     INSTALL_COMMAND: pkg install -y
     TEST_TARGETS: check
 
-- 
2.39.0