Re: [qemu-mainline test] 62028: regressions - FAIL

[Ian Campbell <ian.campbell@citrix.com>]

Stefano, Anthony,

This looks like a real upstream regression.

On Thu, 2015-09-17 at 13:44 +0000, osstest service owner wrote:
> flight 62028 qemu-mainline real [real]
> http://logs.test-lab.xenproject.org/osstest/logs/62028/
> 
> Regressions :-(
> 
> Tests which did not succeed and are blocking,
> including tests which could not be run:
>  test-amd64-amd64-xl-pvh-amd   6 xen-boot                  fail REGR. vs. 61666
>  test-amd64-amd64-xl-pvh-intel  6 xen-boot                 fail REGR. vs. 61666
>  test-armhf-armhf-xl-vhd       6 xen-boot                  fail REGR. vs. 61666
[...many more for every test-*...]

I looked at test-*-*-xl and they all seem to have hung at "Starting QEMU as
disk backend for dom0".

In this flight http://logs.test-lab.xenproject.org/osstest/logs/62028/test-
amd64-amd64-xl/serial-chardonnay1.log ends with:

Sep 17 11:58:48.589051 Starting QEMU as disk backend for dom0
Sep 17 11:58:48.589081
Sep 17 11:59:41.890612 <client 0x93c470 connected - now 1 clients>
(the new client is going to hit the debug keys after the timeout)

By comparison 61666 has in http://logs.test-lab.xenproject.org/osstest/logs
/61666/test-amd64-amd64-xl/serial-elbling0.log

    Sep 10 13:31:52.273142 Starting QEMU as disk backend for dom0
    Sep 10 13:31:52.273183 Starting NTP server: ntpd.
    Sep 10 13:32:18.613286 ^[[r^[[H^[[J

    Sep 10 13:32:18.652996 Debian GNU/Linux 7 elbling0 hvc0

i.e. normally dom0's qemu starts almost instantly.

Given the number of failures here this seems unlikely to be host specific
and almost certain represents a bug in qemu-mainline.

It was noted on xen-users this morning that there is no logging from this
qemu instance -- fixing that would be well worthwhile I think.

http://logs.test-lab.xenproject.org/osstest/logs/62028/test-amd64-amd64-xl/chardonnay1-output-ps_wwwaxf_-eo_pid%2Ctty%2Cstat%2Ctime%2Cnice%2Cpsr%2Cpcpu%2Cpmem%2Cnwchan%2Cwchan%2325%2Cargs

shows the process:

3638 ?        SL   00:00:00   0   3  0.3  0.5 1d1054 poll_schedule_timeout      \_ startpar -p 4 -t 20 -T 3 -M start -P N -R 2
 3815 ?        S    00:00:00   0   0  0.0  0.3  b0482 wait                           \_ /bin/bash /etc/init.d/xencommons start
 4053 ?        Sl   00:00:00   0   1  0.0  1.6 ffffff pipe_wait                          \_ /usr/local/lib/xen/bin/qemu-system-i386 -xen-domid 0 -xen-attach -name dom0 -nographic -M xenpv -daemonize -monitor /dev/null -serial /dev/null -parallel /dev/null -pidfile /var/run/qemu-dom0.pid
 4171 ?        Ss   00:00:00   0   2  0.0  0.5 113cd9 futex_wait_queue_me                    \_ /usr/local/lib/xen/bin/qemu-system-i386 -xen-domid 0 -xen-attach -name dom0 -nographic -M xenpv -daemonize -monitor /dev/null -serial /dev/null -parallel /dev/null -pidfile /var/run/qemu-dom0.pid

The bisector seems to have started working on this in
http://logs.test-lab.xenproject.org/osstest/results/bisect/qemu-mainline/te
st-amd64-amd64-xl-pvh-amd.xen-boot.html


Ian.