From mboxrd@z Thu Jan  1 00:00:00 1970
From: =?UTF-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>
Subject: [BUG] Emulation issues
Date: Wed, 29 Jul 2015 12:17:09 +0200
Message-ID: <55B8A825.5020608@citrix.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Return-path: <xen-devel-bounces@lists.xen.org>
Received: from mail6.bemta3.messagelabs.com ([195.245.230.39])
	by lists.xen.org with esmtp (Exim 4.72)
	(envelope-from <prvs=645e8fec1=roger.pau@citrix.com>)
	id 1ZKOQF-0003FD-Ih
	for xen-devel@lists.xenproject.org; Wed, 29 Jul 2015 10:17:15 +0000
List-Unsubscribe: <http://lists.xen.org/cgi-bin/mailman/options/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xen.org>
List-Help: <mailto:xen-devel-request@lists.xen.org?subject=help>
List-Subscribe: <http://lists.xen.org/cgi-bin/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=subscribe>
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: xen-devel <xen-devel@lists.xenproject.org>, Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul.durrant@citrix.com>
List-Id: xen-devel@lists.xenproject.org

Hello,

While trying to debug a hotplug scripts issue, I came across what seems
to be an emulation bug inside of Xen. The result of this is a bunch of
repeated messages on the serial console:

[...]
(XEN) io.c:165:d56v0 Weird HVM ioemulation status 1.
(XEN) domain_crash called from io.c:166
(XEN) io.c:165:d54v0 Weird HVM ioemulation status 1.
(XEN) domain_crash called from io.c:166
(XEN) io.c:165:d56v0 Weird HVM ioemulation status 1.
(XEN) domain_crash called from io.c:166
(XEN) io.c:165:d54v0 Weird HVM ioemulation status 1.
(XEN) domain_crash called from io.c:166
(XEN) io.c:165:d56v0 Weird HVM ioemulation status 1.
(XEN) domain_crash called from io.c:166
(XEN) io.c:165:d54v0 Weird HVM ioemulation status 1.
(XEN) domain_crash called from io.c:166
(XEN) io.c:165:d56v0 Weird HVM ioemulation status 1.
(XEN) domain_crash called from io.c:166
(XEN) io.c:165:d54v0 Weird HVM ioemulation status 1.
(XEN) domain_crash called from io.c:166
(XEN) io.c:165:d56v0 Weird HVM ioemulation status 1.
(XEN) domain_crash called from io.c:166
(XEN) io.c:165:d54v0 Weird HVM ioemulation status 1.
(XEN) domain_crash called from io.c:166
(XEN) io.c:165:d56v0 Weird HVM ioemulation status 1.
(XEN) domain_crash called from io.c:166
(XEN) io.c:165:d54v0 Weird HVM ioemulation status 1.
(XEN) io.c:165:d56v0 Weird HVM ioemulation status 1.
(XEN) domain_crash called from io.c:166
(XEN) domain_crash called from io.c:166
(XEN) io.c:165:d54v0 Weird HVM ioemulation status 1.
(XEN) io.c:165:d56v0 Weird HVM ioemulation status 1.
(XEN) domain_crash called from io.c:166
(XEN) domain_crash called from io.c:166
(XEN) io.c:165:d54v0 Weird HVM ioemulation status 1.
(XEN) io.c:165:d56v0 Weird HVM ioemulation status 1.
(XEN) domain_crash called from io.c:166
(XEN) domain_crash called from io.c:166
(XEN) io.c:165:d54v0 Weird HVM ioemulation status 1.
(XEN) domain_crash called from io.c:166
(XEN) io.c:165:d56v0 Weird HVM ioemulation status 1.
(XEN) domain_crash called from io.c:166
(XEN) io.c:165:d54v0 Weird HVM ioemulation status 1.
(XEN) domain_crash called from io.c:166
(XEN) io.c:165:d54v0 Weird HVM ioemulation status 1.
(XEN) domain_crash called from io.c:166
(XEN) io.c:165:d54v0 Weird HVM ioemulation status 1.
(XEN) io.c:165:d56v0 Weird HVM ioemulation status 1.
(XEN) domain_crash called from io.c:166
(XEN) domain_crash called from io.c:166
(XEN) io.c:165:d54v0 Weird HVM ioemulation status 1.
(XEN) io.c:165:d56v0 Weird HVM ioemulation status 1.
(XEN) domain_crash called from io.c:166
(XEN) domain_crash called from io.c:166
(XEN) io.c:165:d54v0 Weird HVM ioemulation status 1.
(XEN) io.c:165:d56v0 Weird HVM ioemulation status 1.
(XEN) domain_crash called from io.c:166
(XEN) domain_crash called from io.c:166
(XEN) io.c:165:d54v0 Weird HVM ioemulation status 1.
(XEN) io.c:165:d56v0 Weird HVM ioemulation status 1.
(XEN) domain_crash called from io.c:166
(XEN) domain_crash called from io.c:166
(XEN) io.c:165:d54v0 Weird HVM ioemulation status 1.
(XEN) io.c:165:d56v0 Weird HVM ioemulation status 1.
(XEN) domain_crash called from io.c:166
(XEN) domain_crash called from io.c:166
(XEN) io.c:165:d56v0 Weird HVM ioemulation status 1.
(XEN) domain_crash called from io.c:166
[...]

This is quite spammy, I've been able to reliably trigger this by using
the following config file:

boot="d"
disk = [
'file:/root/test/FreeBSD-10.2-RC1-amd64-bootonly.iso,hdc:cdrom,r',
	]
memory=256
vcpus=1
name = "freebsd@i@"
builder='hvm'
on_crash="preserve"
on_reboot="preserve"
vnc=1
vnclisten="0.0.0.0"
serial='pty'

And launching them in a loop using the following script:

for i in `seq 1 20`; do
	sed s/@i@/$i/g freebsd.cfg > freebsd$i.cfg
	xl -vvv create freebsd$i.cfg
done

The bug happens when all 20 domains are running and I run a `xl shutdown
-w -a`, then some of them (not all) start triggering this emulation
issues. For the reference, the FreeBSD image can be obtained from:

ftp://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/10.2/FreeBSD-10.2-RC1-amd64-bootonly.iso

Also, I'm using latest staging changeset, 44313a.

Roger.