Re: intermittent failure, s390 host, x86-64 guest, vhost-user-blk test

[Peter Maydell <peter.maydell@linaro.org>]

On Thu, 5 Aug 2021 at 18:39, Peter Maydell <peter.maydell@linaro.org> wrote:
>
> On Wed, 4 Aug 2021 at 15:33, Peter Maydell <peter.maydell@linaro.org> wrote:
> >
> > On Wed, 4 Aug 2021 at 14:43, Michael S. Tsirkin <mst@redhat.com> wrote:
> > >
> > > On Wed, Aug 04, 2021 at 01:40:37PM +0100, Peter Maydell wrote:
> > > > Saw this intermittent as part of my ad-hoc CI test setup.
> > > > The backtrace says that the QEMU process has somehow hung
> > > > during 'realize' of the vhost-user device...
> > >
> > > Hmm seems to have to do with the daemon not responding ...
> > > Cc a bunch more people ...
> >
> > Here's another one; same thing, but a ppc64 guest, I think.
>
> Ran into this again; I'll spare you the backtraces but it was the
> same 'hang in realize', this one for qemu-system-i386. Same s390 host.
> This seems to be a fairly frequent intermittent -- can somebody
> have a look at it ?

I think I have also now seen this hang on aarch64 host:

pm       2507739  0.0  0.0   7300  3104 ?        S    09:29   0:00
             \_ bash -o pipefail -c echo
'MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}
QTEST_QEMU_IMG=./qemu-img
G_TEST_DBUS_DAEMON=/home/pm/qemu/tests/dbus-vmstate-daemon.sh
QTEST_QEMU_BINARY=./qemu-system-i386
QTEST_QEMU_STORAGE_DAEMON_BINARY=./storage-daemon/qemu-storage-daemon
tests/qtest/qos-test --tap -k' &&
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}
QTEST_QEMU_IMG=./qemu-img
G_TEST_DBUS_DAEMON=/home/pm/qemu/tests/dbus-vmstate-daemon.sh
QTEST_QEMU_BINARY=./qemu-system-i386
QTEST_QEMU_STORAGE_DAEMON_BINARY=./storage-daemon/qemu-storage-daemon
tests/qtest/qos-test --tap -k -m quick < /dev/null |
./scripts/tap-driver.pl --test-name="qtest-i386/qos-test"
pm       2507744  0.0  0.0  88952  8292 ?        Sl   09:29   0:00
                 \_ tests/qtest/qos-test --tap -k -m quick
pm       2515688  0.0  0.0 333288 21044 ?        Sl   09:30   0:00
                 |   \_ ./storage-daemon/qemu-storage-daemon
--blockdev driver=file,node-name=disk0,filename=qtest.EjWt12 --export
type=vhost-user-blk,id=disk0,addr.type=unix,addr.path=/tmp/qtest-2507744-sock.s5wii0,node-name=disk0,writable=on,num-queues=1
pm       2515689  0.0  0.0 869272 65868 ?        Sl   09:30   0:00
                 |   \_ ./qemu-system-i386 -qtest
unix:/tmp/qtest-2507744.sock -qtest-log /dev/null -chardev
socket,path=/tmp/qtest-2507744.qmp,id=char0 -mon
chardev=char0,mode=control -display none -M pc -device
vhost-user-blk-pci,id=drv0,chardev=char1,addr=4.0 -object
memory-backend-memfd,id=mem,size=256M,share=on -M memory-backend=mem
-m 256M -chardev socket,id=char1,path=/tmp/qtest-2507744-sock.s5wii0
-accel qtest
pm       2507745  0.0  0.0  17404 11244 ?        S    09:29   0:00
                 \_ perl ./scripts/tap-driver.pl
--test-name=qtest-i386/qos-test


Not 100% sure it's the same hang, because the machine doesn't let
me do ptrace-attach-to-process, but it seems likely.

-- PMM