From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:44917) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UfYXO-0000vb-Nc for qemu-devel@nongnu.org; Thu, 23 May 2013 12:38:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UfYX9-0004se-Jf for qemu-devel@nongnu.org; Thu, 23 May 2013 12:38:46 -0400 Received: from e7.ny.us.ibm.com ([32.97.182.137]:42752) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UfYX9-0004sa-De for qemu-devel@nongnu.org; Thu, 23 May 2013 12:38:31 -0400 Received: from /spool/local by e7.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 23 May 2013 12:38:31 -0400 Received: from d01relay03.pok.ibm.com (d01relay03.pok.ibm.com [9.56.227.235]) by d01dlp01.pok.ibm.com (Postfix) with ESMTP id 96D7838C804A for ; Thu, 23 May 2013 12:38:27 -0400 (EDT) Received: from d03av05.boulder.ibm.com (d03av05.boulder.ibm.com [9.17.195.85]) by d01relay03.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id r4NGcR5s260142 for ; Thu, 23 May 2013 12:38:28 -0400 Received: from d03av05.boulder.ibm.com (loopback [127.0.0.1]) by d03av05.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id r4NGcKQh031864 for ; Thu, 23 May 2013 10:38:21 -0600 Message-ID: <519E45F3.4070709@us.ibm.com> Date: Thu, 23 May 2013 09:38:11 -0700 From: Badari Pulavarty MIME-Version: 1.0 References: <519D6478.8060403@us.ibm.com> <20130523005355.GA11949@hj.localdomain> <20130523094809.GM4725@redhat.com> <519E2C21.3000103@us.ibm.com> <519E2E7D.4080502@redhat.com> <20130523152758.GA21158@hj.localdomain> <519E361B.9070702@redhat.com> <519E3FCE.5040401@us.ibm.com> <519E4197.7030605@redhat.com> In-Reply-To: <519E4197.7030605@redhat.com> Content-Type: multipart/alternative; boundary="------------010609070505060007040200" Subject: Re: [Qemu-devel] qemu seabios issue with vhost-scsi List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Paolo Bonzini Cc: Stefan Hajnoczi , Asias He , qemu-devel , "Nicholas A. Bellinger" , Gleb Natapov This is a multi-part message in MIME format. --------------010609070505060007040200 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On 05/23/2013 09:19 AM, Paolo Bonzini wrote: > Il 23/05/2013 18:11, Badari Pulavarty ha scritto: >> On 05/23/2013 08:30 AM, Paolo Bonzini wrote: >>> Il 23/05/2013 17:27, Asias He ha scritto: >>>> On Thu, May 23, 2013 at 04:58:05PM +0200, Paolo Bonzini wrote: >>>>> Il 23/05/2013 16:48, Badari Pulavarty ha scritto: >>>>>>> The common virtio-scsi code in QEMU should guard against this. In >>>>>>> virtio-blk data plane I hit a similar case and ended up starting the >>>>>>> data plane thread (equivalent to vhost here) *before* the status >>>>>>> register is set to DRIVER_OK. >>>>>> Thats exactly what my debug in vhost_scsi_set_status() shows. >>>>>> >>>>>> set status started 0 val 0 >>>>>> set status started 0 val 0 >>>>>> set status started 0 val 0 >>>>>> set status started 0 val 0 >>>>>> set status started 0 val 0 >>>>>> set status started 0 val 3 >>>>>> Program received signal SIGSEGV, Segmentation fault. >>>>>> >>>>>> We never got a chance to call vhost_scsi_start() as we are waiting >>>>>> for DRIVER_OK. >>>> Reproduced the SIGSEGV and verified that replacing the bios.bin with the >>>> one from seabios.git makes the guest boot. >>> This should fix it: >>> >>> diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c >>> index 08dd3f3..3139355 100644 >>> --- a/hw/scsi/virtio-scsi.c >>> +++ b/hw/scsi/virtio-scsi.c >>> @@ -266,7 +266,7 @@ fail: >>> >>> static void virtio_scsi_handle_ctrl(VirtIODevice *vdev, VirtQueue *vq) >>> { >>> - VirtIOSCSI *s = (VirtIOSCSI *)vdev; >>> + VirtIOSCSI *s = VIRTIO_SCSI(vdev); >>> VirtIOSCSIReq *req; >>> >>> while ((req = virtio_scsi_pop_req(s, vq))) { >>> @@ -347,9 +347,8 @@ static void virtio_scsi_fail_cmd_req(VirtIOSCSIReq >>> *req) >>> >>> static void virtio_scsi_handle_cmd(VirtIODevice *vdev, VirtQueue *vq) >>> { >>> - /* use non-QOM casts in the data path */ >>> - VirtIOSCSI *s = (VirtIOSCSI *)vdev; >>> - VirtIOSCSICommon *vs = &s->parent_obj; >>> + VirtIOSCSI *s = VIRTIO_SCSI(vdev); >>> + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(vdev); >>> >>> VirtIOSCSIReq *req; >>> int n; >>> >>> Paolo >>> >> Hmm.. Not quite.. > If that is with the old SeaBIOS, then SIGABRT is intended. :) The guest > is buggy, the problem in QEMU only lies in _how_ it fails. > > Paolo > > I am confused now. Without above changes, seabios fix makes the guest boot. But with the above changes, I run into this (even with seabios fix) (gdb) run -L /root/bios2 -cpu qemu64 --enable-kvm -m 4096 -drive file=/var/lib/libvirt/images/lnx.img,if=ide,cache=writethrough -device vhost-scsi-pci,wwpn=naa.6001405bd4e8476d,event_idx=off -vnc :10 -boot d Starting program: /root/qemu/x86_64-softmmu/qemu-system-x86_64 -L /root/bios2 -cpu qemu64 --enable-kvm -m 4096 -drive file=/var/lib/libvirt/images/lnx.img,if=ide,cache=writethrough -device vhost-scsi-pci,wwpn=naa.6001405bd4e8476d,event_idx=off -vnc :10 -boot d warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7ffff7ffa000 [Thread debugging using libthread_db enabled] [New Thread 0x7ffff1c1c700 (LWP 3299)] [New Thread 0x7ffff1239700 (LWP 3300)] [New Thread 0x7fffeb7ff700 (LWP 3303)] set status started 0 val 0 set status started 0 val 0 set status started 0 val 0 set status started 0 val 0 set status started 0 val 0 set status started 0 val 3 set status started 0 val 7 set status started 1 val 0 /root/qemu/hw/scsi/virtio-scsi.c:356:virtio_scsi_handle_cmd: Object 0x55555659d918 is not an instance of type virtio-scsi-device Program received signal SIGABRT, Aborted. [Switching to Thread 0x7ffff1239700 (LWP 3300)] 0x00007ffff5cf18a5 in raise () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install cyrus-sasl-gssapi-2.1.23-13.el6_3.1.x86_64 cyrus-sasl-lib-2.1.23-13.el6_3.1.x86_64 cyrus-sasl-md5-2.1.23-13.el6_3.1.x86_64 cyrus-sasl-plain-2.1.23-13.el6_3.1.x86_64 db4-4.7.25-17.el6.x86_64 glib2-2.22.5-7.el6.x86_64 glibc-2.12-1.107.el6.x86_64 gnutls-2.8.5-10.el6.x86_64 keyutils-libs-1.4-4.el6.x86_64 krb5-libs-1.10.3-10.el6.x86_64 libcom_err-1.41.12-14.el6.x86_64 libcurl-7.19.7-35.el6.x86_64 libgcrypt-1.4.5-9.el6_2.2.x86_64 libgpg-error-1.7-4.el6.x86_64 libidn-1.18-2.el6.x86_64 libpng-1.2.49-1.el6_2.x86_64 libselinux-2.0.94-5.3.el6.x86_64 libssh2-1.4.2-1.el6.x86_64 libtasn1-2.3-3.el6_2.1.x86_64 ncurses-libs-5.7-3.20090208.el6.x86_64 nspr-4.9.2-1.el6.x86_64 nss-3.14.0.0-12.el6.x86_64 nss-softokn-freebl-3.12.9-11.el6.x86_64 nss-util-3.14.0.0-2.el6.x86_64 openldap-2.4.23-31.el6.x86_64 openssl-1.0.0-27.el6.x86_64 pixman-0.26.2-4.el6.x86_64 zlib-1.2.3-29.el6.x86_64 (gdb) bt #0 0x00007ffff5cf18a5 in raise () from /lib64/libc.so.6 #1 0x00007ffff5cf3085 in abort () from /lib64/libc.so.6 #2 0x00005555557230d0 in object_dynamic_cast_assert (obj=0x55555659d918, typename=0x5555558a56e5 "virtio-scsi-device", file=0x5555558bda30 "/root/qemu/hw/scsi/virtio-scsi.c", line=356, func=) at qom/object.c:456 #3 0x00005555557a5ef1 in virtio_scsi_handle_cmd (vdev=0x55555659d918, vq=0x5555565929d0) at /root/qemu/hw/scsi/virtio-scsi.c:356 #4 0x00005555556e467d in virtio_pci_set_host_notifier_internal (proxy=0x55555659d120, n=2, assign=false, set_handler=false) at hw/virtio/virtio-pci.c:200 #5 0x00005555557a9663 in vhost_dev_disable_notifiers (hdev=0x55555659da38, vdev=) at /root/qemu/hw/virtio/vhost.c:955 #6 0x00005555557a48a1 in vhost_scsi_stop (vdev=0x55555659d918, val=) at /root/qemu/hw/scsi/vhost-scsi.c:136 #7 vhost_scsi_set_status (vdev=0x55555659d918, val=) at /root/qemu/hw/scsi/vhost-scsi.c:198 #8 0x00005555557acb97 in virtio_set_status (vdev=0x55555659d918, val=0 '\000') at /root/qemu/hw/virtio/virtio.c:529 #9 0x00005555556e50d6 in virtio_ioport_write (opaque=0x55555659d120, addr=, val=, size=) at hw/virtio/virtio-pci.c:300 #10 virtio_pci_config_write (opaque=0x55555659d120, addr=, val=, size=) at hw/virtio/virtio-pci.c:422 #11 0x00005555557b3a60 in access_with_adjusted_size (addr=18, value=0x7ffff1238b78, size=1, access_size_min=, access_size_max=, access= 0x5555557b51d0 , opaque=0x55555659d7d0) at /root/qemu/memory.c:364 #12 0x00005555557b408b in memory_region_iorange_write (iorange=, offset=, width=, data=0) at /root/qemu/memory.c:439 #13 0x00005555557b2e61 in kvm_handle_io (env=0x555556521af0) at /root/qemu/kvm-all.c:1482 #14 kvm_cpu_exec (env=0x555556521af0) at /root/qemu/kvm-all.c:1634 #15 0x000055555576148e in qemu_kvm_cpu_thread_fn (arg=0x555556521af0) at /root/qemu/cpus.c:759 #16 0x00007ffff6059851 in start_thread () from /lib64/libpthread.so.0 #17 0x00007ffff5da790d in clone () from /lib64/libc.so.6 --------------010609070505060007040200 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit
On 05/23/2013 09:19 AM, Paolo Bonzini wrote:
Il 23/05/2013 18:11, Badari Pulavarty ha scritto:

On 05/23/2013 08:30 AM, Paolo Bonzini wrote:

Il 23/05/2013 17:27, Asias He ha scritto:

On Thu, May 23, 2013 at 04:58:05PM +0200, Paolo Bonzini wrote:

Il 23/05/2013 16:48, Badari Pulavarty ha scritto:

The common virtio-scsi code in QEMU should guard against this.  In
virtio-blk data plane I hit a similar case and ended up starting the
data plane thread (equivalent to vhost here) *before* the status
register is set to DRIVER_OK.

Thats exactly what my debug in vhost_scsi_set_status() shows.

set status started 0 val 0
set status started 0 val 0
set status started 0 val 0
set status started 0 val 0
set status started 0 val 0
set status started 0 val 3
Program received signal SIGSEGV, Segmentation fault.

We never got a chance to call vhost_scsi_start() as we are waiting
for DRIVER_OK.

Reproduced the SIGSEGV and verified that replacing the bios.bin with the
one from seabios.git makes the guest boot.

This should fix it:

diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
index 08dd3f3..3139355 100644
--- a/hw/scsi/virtio-scsi.c
+++ b/hw/scsi/virtio-scsi.c
@@ -266,7 +266,7 @@ fail:

  static void virtio_scsi_handle_ctrl(VirtIODevice *vdev, VirtQueue *vq)
  {
-    VirtIOSCSI *s = (VirtIOSCSI *)vdev;
+    VirtIOSCSI *s = VIRTIO_SCSI(vdev);
      VirtIOSCSIReq *req;

      while ((req = virtio_scsi_pop_req(s, vq))) {
@@ -347,9 +347,8 @@ static void virtio_scsi_fail_cmd_req(VirtIOSCSIReq
*req)

  static void virtio_scsi_handle_cmd(VirtIODevice *vdev, VirtQueue *vq)
  {
-    /* use non-QOM casts in the data path */
-    VirtIOSCSI *s = (VirtIOSCSI *)vdev;
-    VirtIOSCSICommon *vs = &s->parent_obj;
+    VirtIOSCSI *s = VIRTIO_SCSI(vdev);
+    VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(vdev);

      VirtIOSCSIReq *req;
      int n;

Paolo


Hmm.. Not quite..

If that is with the old SeaBIOS, then SIGABRT is intended. :)  The guest
is buggy, the problem in QEMU only lies in _how_ it fails.

Paolo

I am confused now. Without above changes, seabios fix makes the
guest boot. But with the above changes, I run into this (even with seabios fix)


(gdb) run  -L /root/bios2 -cpu qemu64 --enable-kvm -m 4096 -drive file=/var/lib/libvirt/images/lnx.img,if=ide,cache=writethrough -device vhost-scsi-pci,wwpn=naa.6001405bd4e8476d,event_idx=off -vnc :10 -boot d
Starting program: /root/qemu/x86_64-softmmu/qemu-system-x86_64 -L /root/bios2 -cpu qemu64 --enable-kvm -m 4096 -drive file=/var/lib/libvirt/images/lnx.img,if=ide,cache=writethrough -device vhost-scsi-pci,wwpn=naa.6001405bd4e8476d,event_idx=off -vnc :10 -boot d
warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7ffff7ffa000
[Thread debugging using libthread_db enabled]
[New Thread 0x7ffff1c1c700 (LWP 3299)]
[New Thread 0x7ffff1239700 (LWP 3300)]
[New Thread 0x7fffeb7ff700 (LWP 3303)]
set status started 0 val 0
set status started 0 val 0
set status started 0 val 0
set status started 0 val 0
set status started 0 val 0
set status started 0 val 3
set status started 0 val 7
set status started 1 val 0
/root/qemu/hw/scsi/virtio-scsi.c:356:virtio_scsi_handle_cmd: Object 0x55555659d918 is not an instance of type virtio-scsi-device

Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7ffff1239700 (LWP 3300)]
0x00007ffff5cf18a5 in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install cyrus-sasl-gssapi-2.1.23-13.el6_3.1.x86_64 cyrus-sasl-lib-2.1.23-13.el6_3.1.x86_64 cyrus-sasl-md5-2.1.23-13.el6_3.1.x86_64 cyrus-sasl-plain-2.1.23-13.el6_3.1.x86_64 db4-4.7.25-17.el6.x86_64 glib2-2.22.5-7.el6.x86_64 glibc-2.12-1.107.el6.x86_64 gnutls-2.8.5-10.el6.x86_64 keyutils-libs-1.4-4.el6.x86_64 krb5-libs-1.10.3-10.el6.x86_64 libcom_err-1.41.12-14.el6.x86_64 libcurl-7.19.7-35.el6.x86_64 libgcrypt-1.4.5-9.el6_2.2.x86_64 libgpg-error-1.7-4.el6.x86_64 libidn-1.18-2.el6.x86_64 libpng-1.2.49-1.el6_2.x86_64 libselinux-2.0.94-5.3.el6.x86_64 libssh2-1.4.2-1.el6.x86_64 libtasn1-2.3-3.el6_2.1.x86_64 ncurses-libs-5.7-3.20090208.el6.x86_64 nspr-4.9.2-1.el6.x86_64 nss-3.14.0.0-12.el6.x86_64 nss-softokn-freebl-3.12.9-11.el6.x86_64 nss-util-3.14.0.0-2.el6.x86_64 openldap-2.4.23-31.el6.x86_64 openssl-1.0.0-27.el6.x86_64 pixman-0.26.2-4.el6.x86_64 zlib-1.2.3-29.el6.x86_64
(gdb) bt
#0  0x00007ffff5cf18a5 in raise () from /lib64/libc.so.6
#1  0x00007ffff5cf3085 in abort () from /lib64/libc.so.6
#2  0x00005555557230d0 in object_dynamic_cast_assert (obj=0x55555659d918, typename=0x5555558a56e5 "virtio-scsi-device", file=0x5555558bda30 "/root/qemu/hw/scsi/virtio-scsi.c", line=356,
    func=<value optimized out>) at qom/object.c:456
#3  0x00005555557a5ef1 in virtio_scsi_handle_cmd (vdev=0x55555659d918, vq=0x5555565929d0) at /root/qemu/hw/scsi/virtio-scsi.c:356
#4  0x00005555556e467d in virtio_pci_set_host_notifier_internal (proxy=0x55555659d120, n=2, assign=false, set_handler=false) at hw/virtio/virtio-pci.c:200
#5  0x00005555557a9663 in vhost_dev_disable_notifiers (hdev=0x55555659da38, vdev=<value optimized out>) at /root/qemu/hw/virtio/vhost.c:955
#6  0x00005555557a48a1 in vhost_scsi_stop (vdev=0x55555659d918, val=<value optimized out>) at /root/qemu/hw/scsi/vhost-scsi.c:136
#7  vhost_scsi_set_status (vdev=0x55555659d918, val=<value optimized out>) at /root/qemu/hw/scsi/vhost-scsi.c:198
#8  0x00005555557acb97 in virtio_set_status (vdev=0x55555659d918, val=0 '\000') at /root/qemu/hw/virtio/virtio.c:529
#9  0x00005555556e50d6 in virtio_ioport_write (opaque=0x55555659d120, addr=<value optimized out>, val=<value optimized out>, size=<value optimized out>) at hw/virtio/virtio-pci.c:300
#10 virtio_pci_config_write (opaque=0x55555659d120, addr=<value optimized out>, val=<value optimized out>, size=<value optimized out>) at hw/virtio/virtio-pci.c:422
#11 0x00005555557b3a60 in access_with_adjusted_size (addr=18, value=0x7ffff1238b78, size=1, access_size_min=<value optimized out>, access_size_max=<value optimized out>, access=
    0x5555557b51d0 <memory_region_write_accessor>, opaque=0x55555659d7d0) at /root/qemu/memory.c:364
#12 0x00005555557b408b in memory_region_iorange_write (iorange=<value optimized out>, offset=<value optimized out>, width=<value optimized out>, data=0) at /root/qemu/memory.c:439
#13 0x00005555557b2e61 in kvm_handle_io (env=0x555556521af0) at /root/qemu/kvm-all.c:1482
#14 kvm_cpu_exec (env=0x555556521af0) at /root/qemu/kvm-all.c:1634
#15 0x000055555576148e in qemu_kvm_cpu_thread_fn (arg=0x555556521af0) at /root/qemu/cpus.c:759
#16 0x00007ffff6059851 in start_thread () from /lib64/libpthread.so.0
#17 0x00007ffff5da790d in clone () from /lib64/libc.so.6


--------------010609070505060007040200--