[Qemu-devel] qemu seabios issue with vhost-scsi

[Qemu-devel] qemu seabios issue with vhost-scsi

[Badari]

Hi,

While testing vhost-scsi in the current qemu git, ran into an earlier issue
with seabios. I had to disable scsi support in seabios to get it working.

I was hoping this issue got resolved when vhost-scsi support got
merged into qemu. Is this still being worked on ?

Thanks,
Badari

[root ~]# gdb /root/qemu/x86_64-softmmu/qemu-system-x86_64
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-60.el6)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /root/qemu/x86_64-softmmu/qemu-system-x86_64...done.
(gdb) run  --cpu qemu64 --enable-kvm  -m 4096 -drive 
file=/var/lib/libvirt/images/lnx.img,if=ide,cache=writethrough -device 
vhost-scsi-pci,wwpn=naa.6001405bd4e8476d,event_idx=off -vnc :10 -boot d
Starting program: /root/qemu/x86_64-softmmu/qemu-system-x86_64 --cpu 
qemu64 --enable-kvm  -m 4096 -drive 
file=/var/lib/libvirt/images/window.img,if=ide,cache=writethrough 
-device vhost-scsi-pci,wwpn=naa.6001405bd4e8476d,event_idx=off -vnc :10 
-boot d
warning: no loadable sections found in added symbol-file system-supplied 
DSO at 0x7ffff7ffa000
[Thread debugging using libthread_db enabled]
[New Thread 0x7ffff1c1c700 (LWP 4725)]
[New Thread 0x7ffff1239700 (LWP 4726)]
[New Thread 0x7fffeb7ff700 (LWP 4729)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff1239700 (LWP 4726)]
0x00005555556b3191 in scsi_device_find (bus=0x5555565abb50, channel=0, id=0,
     lun=0) at hw/scsi/scsi-bus.c:1744
1744        QTAILQ_FOREACH_REVERSE(kid, &bus->qbus.children, 
ChildrenHead, sibling) {
Missing separate debuginfos, use: debuginfo-install 
cyrus-sasl-gssapi-2.1.23-13.el6_3.1.x86_64 
cyrus-sasl-lib-2.1.23-13.el6_3.1.x86_64 
cyrus-sasl-md5-2.1.23-13.el6_3.1.x86_64 
cyrus-sasl-plain-2.1.23-13.el6_3.1.x86_64 db4-4.7.25-17.el6.x86_64 
glib2-2.22.5-7.el6.x86_64 glibc-2.12-1.107.el6.x86_64 
gnutls-2.8.5-10.el6.x86_64 keyutils-libs-1.4-4.el6.x86_64 
krb5-libs-1.10.3-10.el6.x86_64 libcom_err-1.41.12-14.el6.x86_64 
libcurl-7.19.7-35.el6.x86_64 libgcrypt-1.4.5-9.el6_2.2.x86_64 
libgpg-error-1.7-4.el6.x86_64 libidn-1.18-2.el6.x86_64 
libpng-1.2.49-1.el6_2.x86_64 libselinux-2.0.94-5.3.el6.x86_64 
libssh2-1.4.2-1.el6.x86_64 libtasn1-2.3-3.el6_2.1.x86_64 
ncurses-libs-5.7-3.20090208.el6.x86_64 nspr-4.9.2-1.el6.x86_64 
nss-3.14.0.0-12.el6.x86_64 nss-softokn-freebl-3.12.9-11.el6.x86_64 
nss-util-3.14.0.0-2.el6.x86_64 openldap-2.4.23-31.el6.x86_64 
openssl-1.0.0-27.el6.x86_64 pixman-0.26.2-4.el6.x86_64 
zlib-1.2.3-29.el6.x86_64
(gdb) bt
#0  0x00005555556b3191 in scsi_device_find (bus=0x5555565abb50, 
channel=0, id=
     0, lun=0) at hw/scsi/scsi-bus.c:1744
#1  0x00005555557a59f0 in virtio_scsi_device_find (vdev=0x5555565aba38, vq=
     0x5555565d1150) at /root/qemu/hw/scsi/virtio-scsi.c:56
#2  virtio_scsi_handle_cmd (vdev=0x5555565aba38, vq=0x5555565d1150)
     at /root/qemu/hw/scsi/virtio-scsi.c:376
#3  0x00005555557b3410 in access_with_adjusted_size (addr=16, value=
     0x7ffff1238b78, size=2, access_size_min=<value optimized out>,
     access_size_max=<value optimized out>, access=
     0x5555557b4b80 <memory_region_write_accessor>, opaque=0x5555565ab8f0)
     at /root/qemu/memory.c:364
#4  0x00005555557b3a3b in memory_region_iorange_write (
     iorange=<value optimized out>, offset=<value optimized out>,
     width=<value optimized out>, data=2) at /root/qemu/memory.c:439
#5  0x00005555557b29a6 in kvm_handle_io (env=0x555556520aa0)
     at /root/qemu/kvm-all.c:1485
#6  kvm_cpu_exec (env=0x555556520aa0) at /root/qemu/kvm-all.c:1634
#7  0x000055555576108e in qemu_kvm_cpu_thread_fn (arg=0x555556520aa0)
     at /root/qemu/cpus.c:759
#8  0x00007ffff6059851 in start_thread () from /lib64/libpthread.so.0
#9  0x00007ffff5da790d in clone () from /lib64/libc.so.6

Re: [Qemu-devel] qemu seabios issue with vhost-scsi

[Asias He]

On Wed, May 22, 2013 at 05:36:08PM -0700, Badari wrote:
> Hi,
> 
> While testing vhost-scsi in the current qemu git, ran into an earlier issue
> with seabios. I had to disable scsi support in seabios to get it working.
> 
> I was hoping this issue got resolved when vhost-scsi support got
> merged into qemu. Is this still being worked on ?

Hmm, can you try seabios.git? Not sure if seabios shipped by qemu picked
up the fixes for vhost-scsi.

> Thanks,
> Badari
> 
> [root ~]# gdb /root/qemu/x86_64-softmmu/qemu-system-x86_64
> GNU gdb (GDB) Red Hat Enterprise Linux (7.2-60.el6)
> Copyright (C) 2010 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later
> <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-redhat-linux-gnu".
> For bug reporting instructions, please see:
> <http://www.gnu.org/software/gdb/bugs/>...
> Reading symbols from /root/qemu/x86_64-softmmu/qemu-system-x86_64...done.
> (gdb) run  --cpu qemu64 --enable-kvm  -m 4096 -drive
> file=/var/lib/libvirt/images/lnx.img,if=ide,cache=writethrough
> -device vhost-scsi-pci,wwpn=naa.6001405bd4e8476d,event_idx=off -vnc
> :10 -boot d
> Starting program: /root/qemu/x86_64-softmmu/qemu-system-x86_64 --cpu
> qemu64 --enable-kvm  -m 4096 -drive
> file=/var/lib/libvirt/images/window.img,if=ide,cache=writethrough
> -device vhost-scsi-pci,wwpn=naa.6001405bd4e8476d,event_idx=off -vnc
> :10 -boot d
> warning: no loadable sections found in added symbol-file
> system-supplied DSO at 0x7ffff7ffa000
> [Thread debugging using libthread_db enabled]
> [New Thread 0x7ffff1c1c700 (LWP 4725)]
> [New Thread 0x7ffff1239700 (LWP 4726)]
> [New Thread 0x7fffeb7ff700 (LWP 4729)]
> 
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7ffff1239700 (LWP 4726)]
> 0x00005555556b3191 in scsi_device_find (bus=0x5555565abb50, channel=0, id=0,
>     lun=0) at hw/scsi/scsi-bus.c:1744
> 1744        QTAILQ_FOREACH_REVERSE(kid, &bus->qbus.children,
> ChildrenHead, sibling) {
> Missing separate debuginfos, use: debuginfo-install
> cyrus-sasl-gssapi-2.1.23-13.el6_3.1.x86_64
> cyrus-sasl-lib-2.1.23-13.el6_3.1.x86_64
> cyrus-sasl-md5-2.1.23-13.el6_3.1.x86_64
> cyrus-sasl-plain-2.1.23-13.el6_3.1.x86_64 db4-4.7.25-17.el6.x86_64
> glib2-2.22.5-7.el6.x86_64 glibc-2.12-1.107.el6.x86_64
> gnutls-2.8.5-10.el6.x86_64 keyutils-libs-1.4-4.el6.x86_64
> krb5-libs-1.10.3-10.el6.x86_64 libcom_err-1.41.12-14.el6.x86_64
> libcurl-7.19.7-35.el6.x86_64 libgcrypt-1.4.5-9.el6_2.2.x86_64
> libgpg-error-1.7-4.el6.x86_64 libidn-1.18-2.el6.x86_64
> libpng-1.2.49-1.el6_2.x86_64 libselinux-2.0.94-5.3.el6.x86_64
> libssh2-1.4.2-1.el6.x86_64 libtasn1-2.3-3.el6_2.1.x86_64
> ncurses-libs-5.7-3.20090208.el6.x86_64 nspr-4.9.2-1.el6.x86_64
> nss-3.14.0.0-12.el6.x86_64 nss-softokn-freebl-3.12.9-11.el6.x86_64
> nss-util-3.14.0.0-2.el6.x86_64 openldap-2.4.23-31.el6.x86_64
> openssl-1.0.0-27.el6.x86_64 pixman-0.26.2-4.el6.x86_64
> zlib-1.2.3-29.el6.x86_64
> (gdb) bt
> #0  0x00005555556b3191 in scsi_device_find (bus=0x5555565abb50,
> channel=0, id=
>     0, lun=0) at hw/scsi/scsi-bus.c:1744
> #1  0x00005555557a59f0 in virtio_scsi_device_find (vdev=0x5555565aba38, vq=
>     0x5555565d1150) at /root/qemu/hw/scsi/virtio-scsi.c:56
> #2  virtio_scsi_handle_cmd (vdev=0x5555565aba38, vq=0x5555565d1150)
>     at /root/qemu/hw/scsi/virtio-scsi.c:376
> #3  0x00005555557b3410 in access_with_adjusted_size (addr=16, value=
>     0x7ffff1238b78, size=2, access_size_min=<value optimized out>,
>     access_size_max=<value optimized out>, access=
>     0x5555557b4b80 <memory_region_write_accessor>, opaque=0x5555565ab8f0)
>     at /root/qemu/memory.c:364
> #4  0x00005555557b3a3b in memory_region_iorange_write (
>     iorange=<value optimized out>, offset=<value optimized out>,
>     width=<value optimized out>, data=2) at /root/qemu/memory.c:439
> #5  0x00005555557b29a6 in kvm_handle_io (env=0x555556520aa0)
>     at /root/qemu/kvm-all.c:1485
> #6  kvm_cpu_exec (env=0x555556520aa0) at /root/qemu/kvm-all.c:1634
> #7  0x000055555576108e in qemu_kvm_cpu_thread_fn (arg=0x555556520aa0)
>     at /root/qemu/cpus.c:759
> #8  0x00007ffff6059851 in start_thread () from /lib64/libpthread.so.0
> #9  0x00007ffff5da790d in clone () from /lib64/libc.so.6
> 
> 

-- 
Asias

Re: [Qemu-devel] qemu seabios issue with vhost-scsi

[Gleb Natapov]

On Thu, May 23, 2013 at 08:53:55AM +0800, Asias He wrote:
> On Wed, May 22, 2013 at 05:36:08PM -0700, Badari wrote:
> > Hi,
> > 
> > While testing vhost-scsi in the current qemu git, ran into an earlier issue
> > with seabios. I had to disable scsi support in seabios to get it working.
> > 
> > I was hoping this issue got resolved when vhost-scsi support got
> > merged into qemu. Is this still being worked on ?
> 
> Hmm, can you try seabios.git? Not sure if seabios shipped by qemu picked
> up the fixes for vhost-scsi.
> 
Nothing in seabios should crash qemu.

> > Thanks,
> > Badari
> > 
> > [root ~]# gdb /root/qemu/x86_64-softmmu/qemu-system-x86_64
> > GNU gdb (GDB) Red Hat Enterprise Linux (7.2-60.el6)
> > Copyright (C) 2010 Free Software Foundation, Inc.
> > License GPLv3+: GNU GPL version 3 or later
> > <http://gnu.org/licenses/gpl.html>
> > This is free software: you are free to change and redistribute it.
> > There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
> > and "show warranty" for details.
> > This GDB was configured as "x86_64-redhat-linux-gnu".
> > For bug reporting instructions, please see:
> > <http://www.gnu.org/software/gdb/bugs/>...
> > Reading symbols from /root/qemu/x86_64-softmmu/qemu-system-x86_64...done.
> > (gdb) run  --cpu qemu64 --enable-kvm  -m 4096 -drive
> > file=/var/lib/libvirt/images/lnx.img,if=ide,cache=writethrough
> > -device vhost-scsi-pci,wwpn=naa.6001405bd4e8476d,event_idx=off -vnc
> > :10 -boot d
> > Starting program: /root/qemu/x86_64-softmmu/qemu-system-x86_64 --cpu
> > qemu64 --enable-kvm  -m 4096 -drive
> > file=/var/lib/libvirt/images/window.img,if=ide,cache=writethrough
> > -device vhost-scsi-pci,wwpn=naa.6001405bd4e8476d,event_idx=off -vnc
> > :10 -boot d
> > warning: no loadable sections found in added symbol-file
> > system-supplied DSO at 0x7ffff7ffa000
> > [Thread debugging using libthread_db enabled]
> > [New Thread 0x7ffff1c1c700 (LWP 4725)]
> > [New Thread 0x7ffff1239700 (LWP 4726)]
> > [New Thread 0x7fffeb7ff700 (LWP 4729)]
> > 
> > Program received signal SIGSEGV, Segmentation fault.
> > [Switching to Thread 0x7ffff1239700 (LWP 4726)]
> > 0x00005555556b3191 in scsi_device_find (bus=0x5555565abb50, channel=0, id=0,
> >     lun=0) at hw/scsi/scsi-bus.c:1744
> > 1744        QTAILQ_FOREACH_REVERSE(kid, &bus->qbus.children,
> > ChildrenHead, sibling) {
> > Missing separate debuginfos, use: debuginfo-install
> > cyrus-sasl-gssapi-2.1.23-13.el6_3.1.x86_64
> > cyrus-sasl-lib-2.1.23-13.el6_3.1.x86_64
> > cyrus-sasl-md5-2.1.23-13.el6_3.1.x86_64
> > cyrus-sasl-plain-2.1.23-13.el6_3.1.x86_64 db4-4.7.25-17.el6.x86_64
> > glib2-2.22.5-7.el6.x86_64 glibc-2.12-1.107.el6.x86_64
> > gnutls-2.8.5-10.el6.x86_64 keyutils-libs-1.4-4.el6.x86_64
> > krb5-libs-1.10.3-10.el6.x86_64 libcom_err-1.41.12-14.el6.x86_64
> > libcurl-7.19.7-35.el6.x86_64 libgcrypt-1.4.5-9.el6_2.2.x86_64
> > libgpg-error-1.7-4.el6.x86_64 libidn-1.18-2.el6.x86_64
> > libpng-1.2.49-1.el6_2.x86_64 libselinux-2.0.94-5.3.el6.x86_64
> > libssh2-1.4.2-1.el6.x86_64 libtasn1-2.3-3.el6_2.1.x86_64
> > ncurses-libs-5.7-3.20090208.el6.x86_64 nspr-4.9.2-1.el6.x86_64
> > nss-3.14.0.0-12.el6.x86_64 nss-softokn-freebl-3.12.9-11.el6.x86_64
> > nss-util-3.14.0.0-2.el6.x86_64 openldap-2.4.23-31.el6.x86_64
> > openssl-1.0.0-27.el6.x86_64 pixman-0.26.2-4.el6.x86_64
> > zlib-1.2.3-29.el6.x86_64
> > (gdb) bt
> > #0  0x00005555556b3191 in scsi_device_find (bus=0x5555565abb50,
> > channel=0, id=
> >     0, lun=0) at hw/scsi/scsi-bus.c:1744
> > #1  0x00005555557a59f0 in virtio_scsi_device_find (vdev=0x5555565aba38, vq=
> >     0x5555565d1150) at /root/qemu/hw/scsi/virtio-scsi.c:56
> > #2  virtio_scsi_handle_cmd (vdev=0x5555565aba38, vq=0x5555565d1150)
> >     at /root/qemu/hw/scsi/virtio-scsi.c:376
> > #3  0x00005555557b3410 in access_with_adjusted_size (addr=16, value=
> >     0x7ffff1238b78, size=2, access_size_min=<value optimized out>,
> >     access_size_max=<value optimized out>, access=
> >     0x5555557b4b80 <memory_region_write_accessor>, opaque=0x5555565ab8f0)
> >     at /root/qemu/memory.c:364
> > #4  0x00005555557b3a3b in memory_region_iorange_write (
> >     iorange=<value optimized out>, offset=<value optimized out>,
> >     width=<value optimized out>, data=2) at /root/qemu/memory.c:439
> > #5  0x00005555557b29a6 in kvm_handle_io (env=0x555556520aa0)
> >     at /root/qemu/kvm-all.c:1485
> > #6  kvm_cpu_exec (env=0x555556520aa0) at /root/qemu/kvm-all.c:1634
> > #7  0x000055555576108e in qemu_kvm_cpu_thread_fn (arg=0x555556520aa0)
> >     at /root/qemu/cpus.c:759
> > #8  0x00007ffff6059851 in start_thread () from /lib64/libpthread.so.0
> > #9  0x00007ffff5da790d in clone () from /lib64/libc.so.6
> > 
> > 
> 
> -- 
> Asias

--
			Gleb.

Re: [Qemu-devel] qemu seabios issue with vhost-scsi

[Stefan Hajnoczi]

On Thu, May 23, 2013 at 11:48 AM, Gleb Natapov <gleb@redhat.com> wrote:
> On Thu, May 23, 2013 at 08:53:55AM +0800, Asias He wrote:
>> On Wed, May 22, 2013 at 05:36:08PM -0700, Badari wrote:
>> > Hi,
>> >
>> > While testing vhost-scsi in the current qemu git, ran into an earlier issue
>> > with seabios. I had to disable scsi support in seabios to get it working.
>> >
>> > I was hoping this issue got resolved when vhost-scsi support got
>> > merged into qemu. Is this still being worked on ?
>>
>> Hmm, can you try seabios.git? Not sure if seabios shipped by qemu picked
>> up the fixes for vhost-scsi.
>>
> Nothing in seabios should crash qemu.

Agreed.  I'm pretty sure it's the scenario that I posted in my other
reply to this thread.

The common virtio-scsi code in QEMU should guard against this.  In
virtio-blk data plane I hit a similar case and ended up starting the
data plane thread (equivalent to vhost here) *before* the status
register is set to DRIVER_OK.

Stefan

Re: [Qemu-devel] qemu seabios issue with vhost-scsi

[Badari Pulavarty]

On 05/23/2013 06:32 AM, Stefan Hajnoczi wrote:
> On Thu, May 23, 2013 at 11:48 AM, Gleb Natapov <gleb@redhat.com> wrote:
>> On Thu, May 23, 2013 at 08:53:55AM +0800, Asias He wrote:
>>> On Wed, May 22, 2013 at 05:36:08PM -0700, Badari wrote:
>>>> Hi,
>>>>
>>>> While testing vhost-scsi in the current qemu git, ran into an earlier issue
>>>> with seabios. I had to disable scsi support in seabios to get it working.
>>>>
>>>> I was hoping this issue got resolved when vhost-scsi support got
>>>> merged into qemu. Is this still being worked on ?
>>> Hmm, can you try seabios.git? Not sure if seabios shipped by qemu picked
>>> up the fixes for vhost-scsi.
>>>
>> Nothing in seabios should crash qemu.
> Agreed.  I'm pretty sure it's the scenario that I posted in my other
> reply to this thread.
>
> The common virtio-scsi code in QEMU should guard against this.  In
> virtio-blk data plane I hit a similar case and ended up starting the
> data plane thread (equivalent to vhost here) *before* the status
> register is set to DRIVER_OK.
>

Thats exactly what my debug in vhost_scsi_set_status() shows.

set status started 0 val 0
set status started 0 val 0
set status started 0 val 0
set status started 0 val 0
set status started 0 val 0
set status started 0 val 3
Program received signal SIGSEGV, Segmentation fault.

We never got a chance to call vhost_scsi_start() as we are waiting
for DRIVER_OK.

Thanks,
Badari

Re: [Qemu-devel] qemu seabios issue with vhost-scsi

[Paolo Bonzini]

Il 23/05/2013 16:48, Badari Pulavarty ha scritto:
>> The common virtio-scsi code in QEMU should guard against this.  In
>> virtio-blk data plane I hit a similar case and ended up starting the
>> data plane thread (equivalent to vhost here) *before* the status
>> register is set to DRIVER_OK.
> 
> Thats exactly what my debug in vhost_scsi_set_status() shows.
> 
> set status started 0 val 0
> set status started 0 val 0
> set status started 0 val 0
> set status started 0 val 0
> set status started 0 val 0
> set status started 0 val 3
> Program received signal SIGSEGV, Segmentation fault.
> 
> We never got a chance to call vhost_scsi_start() as we are waiting
> for DRIVER_OK.

This is the fix in SeaBIOS:

commit 5a7730db57ab0715223421e65b54fb50d6fefe5c
Author: Asias He <asias@redhat.com>
Date:   Fri Mar 15 09:45:15 2013 +0800

    virtio-scsi: Set _DRIVER_OK flag before scsi target scanning

    Before we start scsi target scanning, we need to set the
    VIRTIO_CONFIG_S_DRIVER_OK flag so the device can do setup properly.

    This fix a bug when booting tcm_vhost with seabios.

    Signed-off-by: Asias He <asias@redhat.com>
    Acked-by: Paolo Bonzini <pbonzini@redhat.com>



Still, Gleb is right that SeaBIOS should not be able to crash QEMU;
exit(1) is fine, SIGSEGV is not.

Paolo

Re: [Qemu-devel] qemu seabios issue with vhost-scsi

[Asias He]

On Thu, May 23, 2013 at 04:58:05PM +0200, Paolo Bonzini wrote:
> Il 23/05/2013 16:48, Badari Pulavarty ha scritto:
> >> The common virtio-scsi code in QEMU should guard against this.  In
> >> virtio-blk data plane I hit a similar case and ended up starting the
> >> data plane thread (equivalent to vhost here) *before* the status
> >> register is set to DRIVER_OK.
> > 
> > Thats exactly what my debug in vhost_scsi_set_status() shows.
> > 
> > set status started 0 val 0
> > set status started 0 val 0
> > set status started 0 val 0
> > set status started 0 val 0
> > set status started 0 val 0
> > set status started 0 val 3
> > Program received signal SIGSEGV, Segmentation fault.
> > 
> > We never got a chance to call vhost_scsi_start() as we are waiting
> > for DRIVER_OK.

Reproduced the SIGSEGV and verified that replacing the bios.bin with the
one from seabios.git makes the guest boot.

> This is the fix in SeaBIOS:
> 
> commit 5a7730db57ab0715223421e65b54fb50d6fefe5c
> Author: Asias He <asias@redhat.com>
> Date:   Fri Mar 15 09:45:15 2013 +0800
> 
>     virtio-scsi: Set _DRIVER_OK flag before scsi target scanning
> 
>     Before we start scsi target scanning, we need to set the
>     VIRTIO_CONFIG_S_DRIVER_OK flag so the device can do setup properly.
> 
>     This fix a bug when booting tcm_vhost with seabios.
> 
>     Signed-off-by: Asias He <asias@redhat.com>
>     Acked-by: Paolo Bonzini <pbonzini@redhat.com>
> 
> 
> 
> Still, Gleb is right that SeaBIOS should not be able to crash QEMU;
> exit(1) is fine, SIGSEGV is not.

Agree too.

> Paolo

-- 
Asias

Re: [Qemu-devel] qemu seabios issue with vhost-scsi

[Paolo Bonzini]

Il 23/05/2013 17:27, Asias He ha scritto:
> On Thu, May 23, 2013 at 04:58:05PM +0200, Paolo Bonzini wrote:
>> Il 23/05/2013 16:48, Badari Pulavarty ha scritto:
>>>> The common virtio-scsi code in QEMU should guard against this.  In
>>>> virtio-blk data plane I hit a similar case and ended up starting the
>>>> data plane thread (equivalent to vhost here) *before* the status
>>>> register is set to DRIVER_OK.
>>>
>>> Thats exactly what my debug in vhost_scsi_set_status() shows.
>>>
>>> set status started 0 val 0
>>> set status started 0 val 0
>>> set status started 0 val 0
>>> set status started 0 val 0
>>> set status started 0 val 0
>>> set status started 0 val 3
>>> Program received signal SIGSEGV, Segmentation fault.
>>>
>>> We never got a chance to call vhost_scsi_start() as we are waiting
>>> for DRIVER_OK.
> 
> Reproduced the SIGSEGV and verified that replacing the bios.bin with the
> one from seabios.git makes the guest boot.

This should fix it:

diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
index 08dd3f3..3139355 100644
--- a/hw/scsi/virtio-scsi.c
+++ b/hw/scsi/virtio-scsi.c
@@ -266,7 +266,7 @@ fail:
 
 static void virtio_scsi_handle_ctrl(VirtIODevice *vdev, VirtQueue *vq)
 {
-    VirtIOSCSI *s = (VirtIOSCSI *)vdev;
+    VirtIOSCSI *s = VIRTIO_SCSI(vdev);
     VirtIOSCSIReq *req;
 
     while ((req = virtio_scsi_pop_req(s, vq))) {
@@ -347,9 +347,8 @@ static void virtio_scsi_fail_cmd_req(VirtIOSCSIReq *req)
 
 static void virtio_scsi_handle_cmd(VirtIODevice *vdev, VirtQueue *vq)
 {
-    /* use non-QOM casts in the data path */
-    VirtIOSCSI *s = (VirtIOSCSI *)vdev;
-    VirtIOSCSICommon *vs = &s->parent_obj;
+    VirtIOSCSI *s = VIRTIO_SCSI(vdev);
+    VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(vdev);
 
     VirtIOSCSIReq *req;
     int n;

Paolo

Re: [Qemu-devel] qemu seabios issue with vhost-scsi

[Badari Pulavarty]

On 05/23/2013 08:30 AM, Paolo Bonzini wrote:
> Il 23/05/2013 17:27, Asias He ha scritto:
>> On Thu, May 23, 2013 at 04:58:05PM +0200, Paolo Bonzini wrote:
>>> Il 23/05/2013 16:48, Badari Pulavarty ha scritto:
>>>>> The common virtio-scsi code in QEMU should guard against this.  In
>>>>> virtio-blk data plane I hit a similar case and ended up starting the
>>>>> data plane thread (equivalent to vhost here) *before* the status
>>>>> register is set to DRIVER_OK.
>>>> Thats exactly what my debug in vhost_scsi_set_status() shows.
>>>>
>>>> set status started 0 val 0
>>>> set status started 0 val 0
>>>> set status started 0 val 0
>>>> set status started 0 val 0
>>>> set status started 0 val 0
>>>> set status started 0 val 3
>>>> Program received signal SIGSEGV, Segmentation fault.
>>>>
>>>> We never got a chance to call vhost_scsi_start() as we are waiting
>>>> for DRIVER_OK.
>> Reproduced the SIGSEGV and verified that replacing the bios.bin with the
>> one from seabios.git makes the guest boot.
> This should fix it:
>
> diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
> index 08dd3f3..3139355 100644
> --- a/hw/scsi/virtio-scsi.c
> +++ b/hw/scsi/virtio-scsi.c
> @@ -266,7 +266,7 @@ fail:
>
>   static void virtio_scsi_handle_ctrl(VirtIODevice *vdev, VirtQueue *vq)
>   {
> -    VirtIOSCSI *s = (VirtIOSCSI *)vdev;
> +    VirtIOSCSI *s = VIRTIO_SCSI(vdev);
>       VirtIOSCSIReq *req;
>
>       while ((req = virtio_scsi_pop_req(s, vq))) {
> @@ -347,9 +347,8 @@ static void virtio_scsi_fail_cmd_req(VirtIOSCSIReq *req)
>
>   static void virtio_scsi_handle_cmd(VirtIODevice *vdev, VirtQueue *vq)
>   {
> -    /* use non-QOM casts in the data path */
> -    VirtIOSCSI *s = (VirtIOSCSI *)vdev;
> -    VirtIOSCSICommon *vs = &s->parent_obj;
> +    VirtIOSCSI *s = VIRTIO_SCSI(vdev);
> +    VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(vdev);
>
>       VirtIOSCSIReq *req;
>       int n;
>
> Paolo
>
Hmm.. Not quite..

(gdb) run -cpu qemu64 --enable-kvm -m 4096 -drive 
file=/var/lib/libvirt/images/lnx.img,if=ide,cache=writethrough -device 
vhost-scsi-pci,wwpn=naa.6001405bd4e8476d,event_idx=off -vnc :10 -boot d
Starting program: /root/qemu/x86_64-softmmu/qemu-system-x86_64 -cpu 
qemu64 --enable-kvm -m 4096 -drive 
file=/var/lib/libvirt/images/lnx.img,if=ide,cache=writethrough -device 
vhost-scsi-pci,wwpn=naa.6001405bd4e8476d,event_idx=off -vnc :10 -boot d
warning: no loadable sections found in added symbol-file system-supplied 
DSO at 0x7ffff7ffa000
[Thread debugging using libthread_db enabled]
[New Thread 0x7ffff1c1c700 (LWP 2458)]
[New Thread 0x7ffff1239700 (LWP 2459)]
[New Thread 0x7fffeb7ff700 (LWP 2462)]
set status started 0 val 0
set status started 0 val 0
set status started 0 val 0
set status started 0 val 0
set status started 0 val 0
set status started 0 val 3
/root/qemu/hw/scsi/virtio-scsi.c:356:virtio_scsi_handle_cmd: Object 
0x5555565aca88 is not an instance of type virtio-scsi-device

Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7ffff1239700 (LWP 2459)]
0x00007ffff5cf18a5 in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install 
cyrus-sasl-gssapi-2.1.23-13.el6_3.1.x86_64 
cyrus-sasl-lib-2.1.23-13.el6_3.1.x86_64 
cyrus-sasl-md5-2.1.23-13.el6_3.1.x86_64 
cyrus-sasl-plain-2.1.23-13.el6_3.1.x86_64 db4-4.7.25-17.el6.x86_64 
glib2-2.22.5-7.el6.x86_64 glibc-2.12-1.107.el6.x86_64 
gnutls-2.8.5-10.el6.x86_64 keyutils-libs-1.4-4.el6.x86_64 
krb5-libs-1.10.3-10.el6.x86_64 libcom_err-1.41.12-14.el6.x86_64 
libcurl-7.19.7-35.el6.x86_64 libgcrypt-1.4.5-9.el6_2.2.x86_64 
libgpg-error-1.7-4.el6.x86_64 libidn-1.18-2.el6.x86_64 
libpng-1.2.49-1.el6_2.x86_64 libselinux-2.0.94-5.3.el6.x86_64 
libssh2-1.4.2-1.el6.x86_64 libtasn1-2.3-3.el6_2.1.x86_64 
ncurses-libs-5.7-3.20090208.el6.x86_64 nspr-4.9.2-1.el6.x86_64 
nss-3.14.0.0-12.el6.x86_64 nss-softokn-freebl-3.12.9-11.el6.x86_64 
nss-util-3.14.0.0-2.el6.x86_64 openldap-2.4.23-31.el6.x86_64 
openssl-1.0.0-27.el6.x86_64 pixman-0.26.2-4.el6.x86_64 
zlib-1.2.3-29.el6.x86_64
(gdb) bt
#0  0x00007ffff5cf18a5 in raise () from /lib64/libc.so.6
#1  0x00007ffff5cf3085 in abort () from /lib64/libc.so.6
#2  0x00005555557230d0 in object_dynamic_cast_assert 
(obj=0x5555565aca88, typename=0x5555558a56e5 "virtio-scsi-device", 
file=0x5555558bda30 "/root/qemu/hw/scsi/virtio-scsi.c", line=356,
     func=<value optimized out>) at qom/object.c:456
#3  0x00005555557a5ef1 in virtio_scsi_handle_cmd (vdev=0x5555565aca88, 
vq=0x5555565d2160) at /root/qemu/hw/scsi/virtio-scsi.c:356
#4  0x00005555557b3a60 in access_with_adjusted_size (addr=16, 
value=0x7ffff1238b78, size=2, access_size_min=<value optimized out>, 
access_size_max=<value optimized out>, access=
     0x5555557b51d0 <memory_region_write_accessor>, 
opaque=0x5555565ac940) at /root/qemu/memory.c:364
#5  0x00005555557b408b in memory_region_iorange_write (iorange=<value 
optimized out>, offset=<value optimized out>, width=<value optimized 
out>, data=2) at /root/qemu/memory.c:439
#6  0x00005555557b2ff6 in kvm_handle_io (env=0x555556521af0) at 
/root/qemu/kvm-all.c:1485
#7  kvm_cpu_exec (env=0x555556521af0) at /root/qemu/kvm-all.c:1634
#8  0x000055555576148e in qemu_kvm_cpu_thread_fn (arg=0x555556521af0) at 
/root/qemu/cpus.c:759
#9  0x00007ffff6059851 in start_thread () from /lib64/libpthread.so.0
#10 0x00007ffff5da790d in clone () from /lib64/libc.so.6

Re: [Qemu-devel] qemu seabios issue with vhost-scsi

[Paolo Bonzini]

Il 23/05/2013 18:11, Badari Pulavarty ha scritto:
> On 05/23/2013 08:30 AM, Paolo Bonzini wrote:
>> Il 23/05/2013 17:27, Asias He ha scritto:
>>> On Thu, May 23, 2013 at 04:58:05PM +0200, Paolo Bonzini wrote:
>>>> Il 23/05/2013 16:48, Badari Pulavarty ha scritto:
>>>>>> The common virtio-scsi code in QEMU should guard against this.  In
>>>>>> virtio-blk data plane I hit a similar case and ended up starting the
>>>>>> data plane thread (equivalent to vhost here) *before* the status
>>>>>> register is set to DRIVER_OK.
>>>>> Thats exactly what my debug in vhost_scsi_set_status() shows.
>>>>>
>>>>> set status started 0 val 0
>>>>> set status started 0 val 0
>>>>> set status started 0 val 0
>>>>> set status started 0 val 0
>>>>> set status started 0 val 0
>>>>> set status started 0 val 3
>>>>> Program received signal SIGSEGV, Segmentation fault.
>>>>>
>>>>> We never got a chance to call vhost_scsi_start() as we are waiting
>>>>> for DRIVER_OK.
>>> Reproduced the SIGSEGV and verified that replacing the bios.bin with the
>>> one from seabios.git makes the guest boot.
>> This should fix it:
>>
>> diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
>> index 08dd3f3..3139355 100644
>> --- a/hw/scsi/virtio-scsi.c
>> +++ b/hw/scsi/virtio-scsi.c
>> @@ -266,7 +266,7 @@ fail:
>>
>>   static void virtio_scsi_handle_ctrl(VirtIODevice *vdev, VirtQueue *vq)
>>   {
>> -    VirtIOSCSI *s = (VirtIOSCSI *)vdev;
>> +    VirtIOSCSI *s = VIRTIO_SCSI(vdev);
>>       VirtIOSCSIReq *req;
>>
>>       while ((req = virtio_scsi_pop_req(s, vq))) {
>> @@ -347,9 +347,8 @@ static void virtio_scsi_fail_cmd_req(VirtIOSCSIReq
>> *req)
>>
>>   static void virtio_scsi_handle_cmd(VirtIODevice *vdev, VirtQueue *vq)
>>   {
>> -    /* use non-QOM casts in the data path */
>> -    VirtIOSCSI *s = (VirtIOSCSI *)vdev;
>> -    VirtIOSCSICommon *vs = &s->parent_obj;
>> +    VirtIOSCSI *s = VIRTIO_SCSI(vdev);
>> +    VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(vdev);
>>
>>       VirtIOSCSIReq *req;
>>       int n;
>>
>> Paolo
>>
> Hmm.. Not quite..

If that is with the old SeaBIOS, then SIGABRT is intended. :)  The guest
is buggy, the problem in QEMU only lies in _how_ it fails.

Paolo

> (gdb) run -cpu qemu64 --enable-kvm -m 4096 -drive
> file=/var/lib/libvirt/images/lnx.img,if=ide,cache=writethrough -device
> vhost-scsi-pci,wwpn=naa.6001405bd4e8476d,event_idx=off -vnc :10 -boot d
> Starting program: /root/qemu/x86_64-softmmu/qemu-system-x86_64 -cpu
> qemu64 --enable-kvm -m 4096 -drive
> file=/var/lib/libvirt/images/lnx.img,if=ide,cache=writethrough -device
> vhost-scsi-pci,wwpn=naa.6001405bd4e8476d,event_idx=off -vnc :10 -boot d
> warning: no loadable sections found in added symbol-file system-supplied
> DSO at 0x7ffff7ffa000
> [Thread debugging using libthread_db enabled]
> [New Thread 0x7ffff1c1c700 (LWP 2458)]
> [New Thread 0x7ffff1239700 (LWP 2459)]
> [New Thread 0x7fffeb7ff700 (LWP 2462)]
> set status started 0 val 0
> set status started 0 val 0
> set status started 0 val 0
> set status started 0 val 0
> set status started 0 val 0
> set status started 0 val 3
> /root/qemu/hw/scsi/virtio-scsi.c:356:virtio_scsi_handle_cmd: Object
> 0x5555565aca88 is not an instance of type virtio-scsi-device
> 
> Program received signal SIGABRT, Aborted.
> [Switching to Thread 0x7ffff1239700 (LWP 2459)]
> 0x00007ffff5cf18a5 in raise () from /lib64/libc.so.6
> Missing separate debuginfos, use: debuginfo-install
> cyrus-sasl-gssapi-2.1.23-13.el6_3.1.x86_64
> cyrus-sasl-lib-2.1.23-13.el6_3.1.x86_64
> cyrus-sasl-md5-2.1.23-13.el6_3.1.x86_64
> cyrus-sasl-plain-2.1.23-13.el6_3.1.x86_64 db4-4.7.25-17.el6.x86_64
> glib2-2.22.5-7.el6.x86_64 glibc-2.12-1.107.el6.x86_64
> gnutls-2.8.5-10.el6.x86_64 keyutils-libs-1.4-4.el6.x86_64
> krb5-libs-1.10.3-10.el6.x86_64 libcom_err-1.41.12-14.el6.x86_64
> libcurl-7.19.7-35.el6.x86_64 libgcrypt-1.4.5-9.el6_2.2.x86_64
> libgpg-error-1.7-4.el6.x86_64 libidn-1.18-2.el6.x86_64
> libpng-1.2.49-1.el6_2.x86_64 libselinux-2.0.94-5.3.el6.x86_64
> libssh2-1.4.2-1.el6.x86_64 libtasn1-2.3-3.el6_2.1.x86_64
> ncurses-libs-5.7-3.20090208.el6.x86_64 nspr-4.9.2-1.el6.x86_64
> nss-3.14.0.0-12.el6.x86_64 nss-softokn-freebl-3.12.9-11.el6.x86_64
> nss-util-3.14.0.0-2.el6.x86_64 openldap-2.4.23-31.el6.x86_64
> openssl-1.0.0-27.el6.x86_64 pixman-0.26.2-4.el6.x86_64
> zlib-1.2.3-29.el6.x86_64
> (gdb) bt
> #0  0x00007ffff5cf18a5 in raise () from /lib64/libc.so.6
> #1  0x00007ffff5cf3085 in abort () from /lib64/libc.so.6
> #2  0x00005555557230d0 in object_dynamic_cast_assert
> (obj=0x5555565aca88, typename=0x5555558a56e5 "virtio-scsi-device",
> file=0x5555558bda30 "/root/qemu/hw/scsi/virtio-scsi.c", line=356,
>     func=<value optimized out>) at qom/object.c:456
> #3  0x00005555557a5ef1 in virtio_scsi_handle_cmd (vdev=0x5555565aca88,
> vq=0x5555565d2160) at /root/qemu/hw/scsi/virtio-scsi.c:356
> #4  0x00005555557b3a60 in access_with_adjusted_size (addr=16,
> value=0x7ffff1238b78, size=2, access_size_min=<value optimized out>,
> access_size_max=<value optimized out>, access=
>     0x5555557b51d0 <memory_region_write_accessor>,
> opaque=0x5555565ac940) at /root/qemu/memory.c:364
> #5  0x00005555557b408b in memory_region_iorange_write (iorange=<value
> optimized out>, offset=<value optimized out>, width=<value optimized
> out>, data=2) at /root/qemu/memory.c:439
> #6  0x00005555557b2ff6 in kvm_handle_io (env=0x555556521af0) at
> /root/qemu/kvm-all.c:1485
> #7  kvm_cpu_exec (env=0x555556521af0) at /root/qemu/kvm-all.c:1634
> #8  0x000055555576148e in qemu_kvm_cpu_thread_fn (arg=0x555556521af0) at
> /root/qemu/cpus.c:759
> #9  0x00007ffff6059851 in start_thread () from /lib64/libpthread.so.0
> #10 0x00007ffff5da790d in clone () from /lib64/libc.so.6
> 

Re: [Qemu-devel] qemu seabios issue with vhost-scsi

[Badari Pulavarty]

[-- Attachment #1: Type: text/plain, Size: 6970 bytes --]

On 05/23/2013 09:19 AM, Paolo Bonzini wrote:
> Il 23/05/2013 18:11, Badari Pulavarty ha scritto:
>> On 05/23/2013 08:30 AM, Paolo Bonzini wrote:
>>> Il 23/05/2013 17:27, Asias He ha scritto:
>>>> On Thu, May 23, 2013 at 04:58:05PM +0200, Paolo Bonzini wrote:
>>>>> Il 23/05/2013 16:48, Badari Pulavarty ha scritto:
>>>>>>> The common virtio-scsi code in QEMU should guard against this.  In
>>>>>>> virtio-blk data plane I hit a similar case and ended up starting the
>>>>>>> data plane thread (equivalent to vhost here) *before* the status
>>>>>>> register is set to DRIVER_OK.
>>>>>> Thats exactly what my debug in vhost_scsi_set_status() shows.
>>>>>>
>>>>>> set status started 0 val 0
>>>>>> set status started 0 val 0
>>>>>> set status started 0 val 0
>>>>>> set status started 0 val 0
>>>>>> set status started 0 val 0
>>>>>> set status started 0 val 3
>>>>>> Program received signal SIGSEGV, Segmentation fault.
>>>>>>
>>>>>> We never got a chance to call vhost_scsi_start() as we are waiting
>>>>>> for DRIVER_OK.
>>>> Reproduced the SIGSEGV and verified that replacing the bios.bin with the
>>>> one from seabios.git makes the guest boot.
>>> This should fix it:
>>>
>>> diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
>>> index 08dd3f3..3139355 100644
>>> --- a/hw/scsi/virtio-scsi.c
>>> +++ b/hw/scsi/virtio-scsi.c
>>> @@ -266,7 +266,7 @@ fail:
>>>
>>>    static void virtio_scsi_handle_ctrl(VirtIODevice *vdev, VirtQueue *vq)
>>>    {
>>> -    VirtIOSCSI *s = (VirtIOSCSI *)vdev;
>>> +    VirtIOSCSI *s = VIRTIO_SCSI(vdev);
>>>        VirtIOSCSIReq *req;
>>>
>>>        while ((req = virtio_scsi_pop_req(s, vq))) {
>>> @@ -347,9 +347,8 @@ static void virtio_scsi_fail_cmd_req(VirtIOSCSIReq
>>> *req)
>>>
>>>    static void virtio_scsi_handle_cmd(VirtIODevice *vdev, VirtQueue *vq)
>>>    {
>>> -    /* use non-QOM casts in the data path */
>>> -    VirtIOSCSI *s = (VirtIOSCSI *)vdev;
>>> -    VirtIOSCSICommon *vs = &s->parent_obj;
>>> +    VirtIOSCSI *s = VIRTIO_SCSI(vdev);
>>> +    VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(vdev);
>>>
>>>        VirtIOSCSIReq *req;
>>>        int n;
>>>
>>> Paolo
>>>
>> Hmm.. Not quite..
> If that is with the old SeaBIOS, then SIGABRT is intended. :)  The guest
> is buggy, the problem in QEMU only lies in _how_ it fails.
>
> Paolo
>
>

I am confused now. Without above changes, seabios fix makes the
guest boot. But with the above changes, I run into this (even with 
seabios fix)


(gdb) run  -L /root/bios2 -cpu qemu64 --enable-kvm -m 4096 -drive 
file=/var/lib/libvirt/images/lnx.img,if=ide,cache=writethrough -device 
vhost-scsi-pci,wwpn=naa.6001405bd4e8476d,event_idx=off -vnc :10 -boot d
Starting program: /root/qemu/x86_64-softmmu/qemu-system-x86_64 -L 
/root/bios2 -cpu qemu64 --enable-kvm -m 4096 -drive 
file=/var/lib/libvirt/images/lnx.img,if=ide,cache=writethrough -device 
vhost-scsi-pci,wwpn=naa.6001405bd4e8476d,event_idx=off -vnc :10 -boot d
warning: no loadable sections found in added symbol-file system-supplied 
DSO at 0x7ffff7ffa000
[Thread debugging using libthread_db enabled]
[New Thread 0x7ffff1c1c700 (LWP 3299)]
[New Thread 0x7ffff1239700 (LWP 3300)]
[New Thread 0x7fffeb7ff700 (LWP 3303)]
set status started 0 val 0
set status started 0 val 0
set status started 0 val 0
set status started 0 val 0
set status started 0 val 0
set status started 0 val 3
set status started 0 val 7
set status started 1 val 0
/root/qemu/hw/scsi/virtio-scsi.c:356:virtio_scsi_handle_cmd: Object 
0x55555659d918 is not an instance of type virtio-scsi-device

Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7ffff1239700 (LWP 3300)]
0x00007ffff5cf18a5 in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install 
cyrus-sasl-gssapi-2.1.23-13.el6_3.1.x86_64 
cyrus-sasl-lib-2.1.23-13.el6_3.1.x86_64 
cyrus-sasl-md5-2.1.23-13.el6_3.1.x86_64 
cyrus-sasl-plain-2.1.23-13.el6_3.1.x86_64 db4-4.7.25-17.el6.x86_64 
glib2-2.22.5-7.el6.x86_64 glibc-2.12-1.107.el6.x86_64 
gnutls-2.8.5-10.el6.x86_64 keyutils-libs-1.4-4.el6.x86_64 
krb5-libs-1.10.3-10.el6.x86_64 libcom_err-1.41.12-14.el6.x86_64 
libcurl-7.19.7-35.el6.x86_64 libgcrypt-1.4.5-9.el6_2.2.x86_64 
libgpg-error-1.7-4.el6.x86_64 libidn-1.18-2.el6.x86_64 
libpng-1.2.49-1.el6_2.x86_64 libselinux-2.0.94-5.3.el6.x86_64 
libssh2-1.4.2-1.el6.x86_64 libtasn1-2.3-3.el6_2.1.x86_64 
ncurses-libs-5.7-3.20090208.el6.x86_64 nspr-4.9.2-1.el6.x86_64 
nss-3.14.0.0-12.el6.x86_64 nss-softokn-freebl-3.12.9-11.el6.x86_64 
nss-util-3.14.0.0-2.el6.x86_64 openldap-2.4.23-31.el6.x86_64 
openssl-1.0.0-27.el6.x86_64 pixman-0.26.2-4.el6.x86_64 
zlib-1.2.3-29.el6.x86_64
(gdb) bt
#0  0x00007ffff5cf18a5 in raise () from /lib64/libc.so.6
#1  0x00007ffff5cf3085 in abort () from /lib64/libc.so.6
#2  0x00005555557230d0 in object_dynamic_cast_assert 
(obj=0x55555659d918, typename=0x5555558a56e5 "virtio-scsi-device", 
file=0x5555558bda30 "/root/qemu/hw/scsi/virtio-scsi.c", line=356,
     func=<value optimized out>) at qom/object.c:456
#3  0x00005555557a5ef1 in virtio_scsi_handle_cmd (vdev=0x55555659d918, 
vq=0x5555565929d0) at /root/qemu/hw/scsi/virtio-scsi.c:356
#4  0x00005555556e467d in virtio_pci_set_host_notifier_internal 
(proxy=0x55555659d120, n=2, assign=false, set_handler=false) at 
hw/virtio/virtio-pci.c:200
#5  0x00005555557a9663 in vhost_dev_disable_notifiers 
(hdev=0x55555659da38, vdev=<value optimized out>) at 
/root/qemu/hw/virtio/vhost.c:955
#6  0x00005555557a48a1 in vhost_scsi_stop (vdev=0x55555659d918, 
val=<value optimized out>) at /root/qemu/hw/scsi/vhost-scsi.c:136
#7  vhost_scsi_set_status (vdev=0x55555659d918, val=<value optimized 
out>) at /root/qemu/hw/scsi/vhost-scsi.c:198
#8  0x00005555557acb97 in virtio_set_status (vdev=0x55555659d918, val=0 
'\000') at /root/qemu/hw/virtio/virtio.c:529
#9  0x00005555556e50d6 in virtio_ioport_write (opaque=0x55555659d120, 
addr=<value optimized out>, val=<value optimized out>, size=<value 
optimized out>) at hw/virtio/virtio-pci.c:300
#10 virtio_pci_config_write (opaque=0x55555659d120, addr=<value 
optimized out>, val=<value optimized out>, size=<value optimized out>) 
at hw/virtio/virtio-pci.c:422
#11 0x00005555557b3a60 in access_with_adjusted_size (addr=18, 
value=0x7ffff1238b78, size=1, access_size_min=<value optimized out>, 
access_size_max=<value optimized out>, access=
     0x5555557b51d0 <memory_region_write_accessor>, 
opaque=0x55555659d7d0) at /root/qemu/memory.c:364
#12 0x00005555557b408b in memory_region_iorange_write (iorange=<value 
optimized out>, offset=<value optimized out>, width=<value optimized 
out>, data=0) at /root/qemu/memory.c:439
#13 0x00005555557b2e61 in kvm_handle_io (env=0x555556521af0) at 
/root/qemu/kvm-all.c:1482
#14 kvm_cpu_exec (env=0x555556521af0) at /root/qemu/kvm-all.c:1634
#15 0x000055555576148e in qemu_kvm_cpu_thread_fn (arg=0x555556521af0) at 
/root/qemu/cpus.c:759
#16 0x00007ffff6059851 in start_thread () from /lib64/libpthread.so.0
#17 0x00007ffff5da790d in clone () from /lib64/libc.so.6



[-- Attachment #2: Type: text/html, Size: 9016 bytes --]

Re: [Qemu-devel] qemu seabios issue with vhost-scsi

[Paolo Bonzini]

Il 23/05/2013 18:38, Badari Pulavarty ha scritto:
>> If that is with the old SeaBIOS, then SIGABRT is intended. :)  The guest
>> is buggy, the problem in QEMU only lies in _how_ it fails.
>>
>> Paolo
>>
>>
> 
> I am confused now. Without above changes, seabios fix makes the
> guest boot. But with the above changes, I run into this (even with
> seabios fix)

Ah, okay.  I understood it crashed only with the SeaBIOS fix.

I'll work on a more proper fix then.

Paolo

Re: [Qemu-devel] qemu seabios issue with vhost-scsi

[Stefan Hajnoczi]

On Thu, May 23, 2013 at 6:47 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
> Il 23/05/2013 18:38, Badari Pulavarty ha scritto:
>>> If that is with the old SeaBIOS, then SIGABRT is intended. :)  The guest
>>> is buggy, the problem in QEMU only lies in _how_ it fails.
>>>
>>> Paolo
>>>
>>>
>>
>> I am confused now. Without above changes, seabios fix makes the
>> guest boot. But with the above changes, I run into this (even with
>> seabios fix)
>
> Ah, okay.  I understood it crashed only with the SeaBIOS fix.
>
> I'll work on a more proper fix then.

Maybe use the same approach as data plane - activate vhost when
userspace sees the first virtqueue kick.  That way it works with weird
guests and never aborts.

Stefan

Re: [Qemu-devel] qemu seabios issue with vhost-scsi

[Paolo Bonzini]

----- Messaggio originale -----
> Da: "Stefan Hajnoczi" <stefanha@gmail.com>
> A: "Paolo Bonzini" <pbonzini@redhat.com>
> Cc: "Badari Pulavarty" <pbadari@us.ibm.com>, "Asias He" <asias@redhat.com>, "Nicholas A. Bellinger"
> <nab@linux-iscsi.org>, "qemu-devel" <qemu-devel@nongnu.org>, "Gleb Natapov" <gleb@redhat.com>
> Inviato: Giovedì, 23 maggio 2013 19:18:26
> Oggetto: Re: qemu seabios issue with vhost-scsi
> 
> On Thu, May 23, 2013 at 6:47 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
> > Il 23/05/2013 18:38, Badari Pulavarty ha scritto:
> >>> If that is with the old SeaBIOS, then SIGABRT is intended. :)  The guest
> >>> is buggy, the problem in QEMU only lies in _how_ it fails.
> >>>
> >>> Paolo
> >>>
> >>>
> >>
> >> I am confused now. Without above changes, seabios fix makes the
> >> guest boot. But with the above changes, I run into this (even with
> >> seabios fix)
> >
> > Ah, okay.  I understood it crashed only with the SeaBIOS fix.
> >
> > I'll work on a more proper fix then.
> 
> Maybe use the same approach as data plane - activate vhost when
> userspace sees the first virtqueue kick.  That way it works with weird
> guests and never aborts.

Good idea.

Paolo

Re: [Qemu-devel] qemu seabios issue with vhost-scsi

[Asias He]

On Thu, May 23, 2013 at 01:31:12PM -0400, Paolo Bonzini wrote:
> 
> 
> ----- Messaggio originale -----
> > Da: "Stefan Hajnoczi" <stefanha@gmail.com>
> > A: "Paolo Bonzini" <pbonzini@redhat.com>
> > Cc: "Badari Pulavarty" <pbadari@us.ibm.com>, "Asias He" <asias@redhat.com>, "Nicholas A. Bellinger"
> > <nab@linux-iscsi.org>, "qemu-devel" <qemu-devel@nongnu.org>, "Gleb Natapov" <gleb@redhat.com>
> > Inviato: Giovedì, 23 maggio 2013 19:18:26
> > Oggetto: Re: qemu seabios issue with vhost-scsi
> > 
> > On Thu, May 23, 2013 at 6:47 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
> > > Il 23/05/2013 18:38, Badari Pulavarty ha scritto:
> > >>> If that is with the old SeaBIOS, then SIGABRT is intended. :)  The guest
> > >>> is buggy, the problem in QEMU only lies in _how_ it fails.
> > >>>
> > >>> Paolo
> > >>>
> > >>>
> > >>
> > >> I am confused now. Without above changes, seabios fix makes the
> > >> guest boot. But with the above changes, I run into this (even with
> > >> seabios fix)
> > >
> > > Ah, okay.  I understood it crashed only with the SeaBIOS fix.
> > >
> > > I'll work on a more proper fix then.
> > 
> > Maybe use the same approach as data plane - activate vhost when
> > userspace sees the first virtqueue kick.  That way it works with weird
> > guests and never aborts.
> 
> Good idea.

I remember you mentioned that you wanted to start vhost earlier when you
were debugging vhost-scsi a few days ago.

> Paolo

-- 
Asias

Re: [Qemu-devel] qemu seabios issue with vhost-scsi

[Badari Pulavarty]

On 05/23/2013 07:58 AM, Paolo Bonzini wrote:
> Il 23/05/2013 16:48, Badari Pulavarty ha scritto:
>>> The common virtio-scsi code in QEMU should guard against this.  In
>>> virtio-blk data plane I hit a similar case and ended up starting the
>>> data plane thread (equivalent to vhost here) *before* the status
>>> register is set to DRIVER_OK.
>> Thats exactly what my debug in vhost_scsi_set_status() shows.
>>
>> set status started 0 val 0
>> set status started 0 val 0
>> set status started 0 val 0
>> set status started 0 val 0
>> set status started 0 val 0
>> set status started 0 val 3
>> Program received signal SIGSEGV, Segmentation fault.
>>
>> We never got a chance to call vhost_scsi_start() as we are waiting
>> for DRIVER_OK.
> This is the fix in SeaBIOS:
>
> commit 5a7730db57ab0715223421e65b54fb50d6fefe5c
> Author: Asias He <asias@redhat.com>
> Date:   Fri Mar 15 09:45:15 2013 +0800
>
>      virtio-scsi: Set _DRIVER_OK flag before scsi target scanning
>
>      Before we start scsi target scanning, we need to set the
>      VIRTIO_CONFIG_S_DRIVER_OK flag so the device can do setup properly.
>
>      This fix a bug when booting tcm_vhost with seabios.
>
>      Signed-off-by: Asias He <asias@redhat.com>
>      Acked-by: Paolo Bonzini <pbonzini@redhat.com>
>
>
>
> Still, Gleb is right that SeaBIOS should not be able to crash QEMU;
> exit(1) is fine, SIGSEGV is not.
>
> Paolo
>
This fixed the issue and makes the guest boot.

Thanks
Badari

Re: [Qemu-devel] qemu seabios issue with vhost-scsi

[Stefan Hajnoczi]

On Wed, May 22, 2013 at 05:36:08PM -0700, Badari wrote:
> Hi,
> 
> While testing vhost-scsi in the current qemu git, ran into an earlier issue
> with seabios. I had to disable scsi support in seabios to get it working.
> 
> I was hoping this issue got resolved when vhost-scsi support got
> merged into qemu. Is this still being worked on ?
> 
> Thanks,
> Badari
> 
> [root ~]# gdb /root/qemu/x86_64-softmmu/qemu-system-x86_64
> GNU gdb (GDB) Red Hat Enterprise Linux (7.2-60.el6)
> Copyright (C) 2010 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later
> <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-redhat-linux-gnu".
> For bug reporting instructions, please see:
> <http://www.gnu.org/software/gdb/bugs/>...
> Reading symbols from /root/qemu/x86_64-softmmu/qemu-system-x86_64...done.
> (gdb) run  --cpu qemu64 --enable-kvm  -m 4096 -drive
> file=/var/lib/libvirt/images/lnx.img,if=ide,cache=writethrough
> -device vhost-scsi-pci,wwpn=naa.6001405bd4e8476d,event_idx=off -vnc
> :10 -boot d
> Starting program: /root/qemu/x86_64-softmmu/qemu-system-x86_64 --cpu
> qemu64 --enable-kvm  -m 4096 -drive
> file=/var/lib/libvirt/images/window.img,if=ide,cache=writethrough
> -device vhost-scsi-pci,wwpn=naa.6001405bd4e8476d,event_idx=off -vnc
> :10 -boot d
> warning: no loadable sections found in added symbol-file
> system-supplied DSO at 0x7ffff7ffa000
> [Thread debugging using libthread_db enabled]
> [New Thread 0x7ffff1c1c700 (LWP 4725)]
> [New Thread 0x7ffff1239700 (LWP 4726)]
> [New Thread 0x7fffeb7ff700 (LWP 4729)]
> 
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7ffff1239700 (LWP 4726)]
> 0x00005555556b3191 in scsi_device_find (bus=0x5555565abb50, channel=0, id=0,
>     lun=0) at hw/scsi/scsi-bus.c:1744
> 1744        QTAILQ_FOREACH_REVERSE(kid, &bus->qbus.children,
> ChildrenHead, sibling) {
> Missing separate debuginfos, use: debuginfo-install
> cyrus-sasl-gssapi-2.1.23-13.el6_3.1.x86_64
> cyrus-sasl-lib-2.1.23-13.el6_3.1.x86_64
> cyrus-sasl-md5-2.1.23-13.el6_3.1.x86_64
> cyrus-sasl-plain-2.1.23-13.el6_3.1.x86_64 db4-4.7.25-17.el6.x86_64
> glib2-2.22.5-7.el6.x86_64 glibc-2.12-1.107.el6.x86_64
> gnutls-2.8.5-10.el6.x86_64 keyutils-libs-1.4-4.el6.x86_64
> krb5-libs-1.10.3-10.el6.x86_64 libcom_err-1.41.12-14.el6.x86_64
> libcurl-7.19.7-35.el6.x86_64 libgcrypt-1.4.5-9.el6_2.2.x86_64
> libgpg-error-1.7-4.el6.x86_64 libidn-1.18-2.el6.x86_64
> libpng-1.2.49-1.el6_2.x86_64 libselinux-2.0.94-5.3.el6.x86_64
> libssh2-1.4.2-1.el6.x86_64 libtasn1-2.3-3.el6_2.1.x86_64
> ncurses-libs-5.7-3.20090208.el6.x86_64 nspr-4.9.2-1.el6.x86_64
> nss-3.14.0.0-12.el6.x86_64 nss-softokn-freebl-3.12.9-11.el6.x86_64
> nss-util-3.14.0.0-2.el6.x86_64 openldap-2.4.23-31.el6.x86_64
> openssl-1.0.0-27.el6.x86_64 pixman-0.26.2-4.el6.x86_64
> zlib-1.2.3-29.el6.x86_64
> (gdb) bt
> #0  0x00005555556b3191 in scsi_device_find (bus=0x5555565abb50,
> channel=0, id=
>     0, lun=0) at hw/scsi/scsi-bus.c:1744
> #1  0x00005555557a59f0 in virtio_scsi_device_find (vdev=0x5555565aba38, vq=
>     0x5555565d1150) at /root/qemu/hw/scsi/virtio-scsi.c:56
> #2  virtio_scsi_handle_cmd (vdev=0x5555565aba38, vq=0x5555565d1150)
>     at /root/qemu/hw/scsi/virtio-scsi.c:376

We should never get here with vhost-scsi.  This function is processing
the command virtqueue in QEMU userspace - if vhost is active then we
shouldn't reach this.

AFAICT the s->bus was not initialized in the vhost codepath.  Therefore
the crash in scsi_device_find(bus, ...).

Can you check vhost_scsi_set_status() was called and if it successfully
enabled vhost?

Is it possible that the guest is notifying the virtqueue before setting
the status register to DRIVER_OK?  That would explain why vhost hasn't
been activated yet.

Stefan