[Qemu-devel] virtio_stw_phys_cached crash

[Qemu-devel] virtio_stw_phys_cached crash

[Fam Zheng]

On master (13b7b188501) and v3.0.0, booting the OpenBSD install63.iso with a
virtio-net (slirp backend) device gives this assertion failure:

(gdb) bt
#0  0x00007f6f25703feb in raise () at /lib64/libc.so.6
#1  0x00007f6f256ee5c1 in abort () at /lib64/libc.so.6
#2  0x00007f6f256ee491 in _nl_load_domain.cold.0 () at /lib64/libc.so.6
#3  0x00007f6f256fc752 in  () at /lib64/libc.so.6
#4  0x000055d966f5b5d6 in address_space_stw_le_cached (cache=0x7f6f040fd700, addr=516, val=1, attrs=..., result=0x0)
    at /stor/work/qemu/include/exec/memory_ldst_cached.inc.h:85
#5  0x000055d966f5b804 in stw_le_phys_cached (cache=0x7f6f040fd700, addr=516, val=1) at /stor/work/qemu/include/exec/memory_ldst_phys.inc.h:121
#6  0x000055d966f5c3e7 in virtio_stw_phys_cached (vdev=0x55d969aeeff0, cache=0x7f6f040fd700, pa=516, value=1) at /stor/work/qemu/include/hw/virtio/virtio-access.h:195
#7  0x000055d966f5cef8 in vring_set_avail_event (vq=0x7f6f181df110, val=1) at /stor/work/qemu/hw/virtio/virtio.c:324
#8  0x000055d966f5e1c8 in virtqueue_pop (vq=0x7f6f181df110, sz=48) at /stor/work/qemu/hw/virtio/virtio.c:883
#9  0x000055d966f2d69b in virtio_net_handle_ctrl (vdev=0x55d969aeeff0, vq=0x7f6f181df110) at /stor/work/qemu/hw/net/virtio-net.c:986
#10 0x000055d966f5ffb9 in virtio_queue_notify_vq (vq=0x7f6f181df110) at /stor/work/qemu/hw/virtio/virtio.c:1534
#11 0x000055d966f6224d in virtio_queue_host_notifier_read (n=0x7f6f181df178) at /stor/work/qemu/hw/virtio/virtio.c:2451
#12 0x000055d9673ef460 in aio_dispatch_handlers (ctx=0x55d96891af90) at /stor/work/qemu/util/aio-posix.c:406
#13 0x000055d9673ef5f3 in aio_dispatch (ctx=0x55d96891af90) at /stor/work/qemu/util/aio-posix.c:437
#14 0x000055d9673eae25 in aio_ctx_dispatch (source=0x55d96891af90, callback=0x0, user_data=0x0) at /stor/work/qemu/util/async.c:261
#15 0x00007f6f28b578ad in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
#16 0x000055d9673ee040 in glib_pollfds_poll () at /stor/work/qemu/util/main-loop.c:215
#17 0x000055d9673ee0ae in os_host_main_loop_wait (timeout=0) at /stor/work/qemu/util/main-loop.c:238
#18 0x000055d9673ee167 in main_loop_wait (nonblocking=0) at /stor/work/qemu/util/main-loop.c:497
#19 0x000055d96702efd0 in main_loop () at /stor/work/qemu/vl.c:1866
#20 0x000055d9670368bd in main (argc=22, argv=0x7ffdef327638, envp=0x7ffdef3276f0) at /stor/work/qemu/vl.c:4644

No surprise, bisection points to

commit 48564041a73adbbff52834f9edbe3806fceefab7
Author: Paolo Bonzini <pbonzini@redhat.com>
Date:   Sun Mar 18 18:26:36 2018 +0100

    exec: reintroduce MemoryRegion caching

Fam

Re: [Qemu-devel] virtio_stw_phys_cached crash

[Paolo Bonzini]

On 22/08/2018 16:57, Fam Zheng wrote:
> On master (13b7b188501) and v3.0.0, booting the OpenBSD install63.iso with a
> virtio-net (slirp backend) device gives this assertion failure:
> 
> (gdb) bt
> #0  0x00007f6f25703feb in raise () at /lib64/libc.so.6
> #1  0x00007f6f256ee5c1 in abort () at /lib64/libc.so.6
> #2  0x00007f6f256ee491 in _nl_load_domain.cold.0 () at /lib64/libc.so.6
> #3  0x00007f6f256fc752 in  () at /lib64/libc.so.6
> #4  0x000055d966f5b5d6 in address_space_stw_le_cached (cache=0x7f6f040fd700, addr=516, val=1, attrs=..., result=0x0)
>     at /stor/work/qemu/include/exec/memory_ldst_cached.inc.h:85

Does this help?

diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index d4e4d98b59..5982678c75 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -2006,14 +2006,25 @@ static int virtio_set_features_nocheck(VirtIODevice *vdev, uint64_t val)
 
 int virtio_set_features(VirtIODevice *vdev, uint64_t val)
 {
-   /*
+    int ret;
+    /*
      * The driver must not attempt to set features after feature negotiation
      * has finished.
      */
     if (vdev->status & VIRTIO_CONFIG_S_FEATURES_OK) {
         return -EINVAL;
     }
-    return virtio_set_features_nocheck(vdev, val);
+    ret = virtio_set_features_nocheck(vdev, val);
+    if (!ret && (val & VIRTIO_RING_F_EVENT_IDX)) {
+        /* VIRTIO_RING_F_EVENT_IDX changes the size of the caches.  */
+        int i;
+        for (i = 0; i < VIRTIO_QUEUE_MAX; i++) {
+            if (vdev->vq[i].vring.num != 0) {
+                virtio_init_region_cache(vdev, i);
+            }
+        }
+    }
+    return ret;
 }
 
 int virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id)

(I haven't tried to reproduce, or checked the spec to see if this could
be a guest bug too.  Of course assertion failures are wrong anyway, so
we might as well work around it as above).

Paolo

Re: [Qemu-devel] virtio_stw_phys_cached crash

[Fam Zheng]

On Wed, 08/22 19:45, Paolo Bonzini wrote:
> On 22/08/2018 16:57, Fam Zheng wrote:
> > On master (13b7b188501) and v3.0.0, booting the OpenBSD install63.iso with a
> > virtio-net (slirp backend) device gives this assertion failure:
> > 
> > (gdb) bt
> > #0  0x00007f6f25703feb in raise () at /lib64/libc.so.6
> > #1  0x00007f6f256ee5c1 in abort () at /lib64/libc.so.6
> > #2  0x00007f6f256ee491 in _nl_load_domain.cold.0 () at /lib64/libc.so.6
> > #3  0x00007f6f256fc752 in  () at /lib64/libc.so.6
> > #4  0x000055d966f5b5d6 in address_space_stw_le_cached (cache=0x7f6f040fd700, addr=516, val=1, attrs=..., result=0x0)
> >     at /stor/work/qemu/include/exec/memory_ldst_cached.inc.h:85
> 
> Does this help?

Nope. The crash happens the same (once you press "Auto Install" at the first
prompt).

Fam

> 
> diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
> index d4e4d98b59..5982678c75 100644
> --- a/hw/virtio/virtio.c
> +++ b/hw/virtio/virtio.c
> @@ -2006,14 +2006,25 @@ static int virtio_set_features_nocheck(VirtIODevice *vdev, uint64_t val)
>  
>  int virtio_set_features(VirtIODevice *vdev, uint64_t val)
>  {
> -   /*
> +    int ret;
> +    /*
>       * The driver must not attempt to set features after feature negotiation
>       * has finished.
>       */
>      if (vdev->status & VIRTIO_CONFIG_S_FEATURES_OK) {
>          return -EINVAL;
>      }
> -    return virtio_set_features_nocheck(vdev, val);
> +    ret = virtio_set_features_nocheck(vdev, val);
> +    if (!ret && (val & VIRTIO_RING_F_EVENT_IDX)) {
> +        /* VIRTIO_RING_F_EVENT_IDX changes the size of the caches.  */
> +        int i;
> +        for (i = 0; i < VIRTIO_QUEUE_MAX; i++) {
> +            if (vdev->vq[i].vring.num != 0) {
> +                virtio_init_region_cache(vdev, i);
> +            }
> +        }
> +    }
> +    return ret;
>  }
>  
>  int virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id)
> 
> (I haven't tried to reproduce, or checked the spec to see if this could
> be a guest bug too.  Of course assertion failures are wrong anyway, so
> we might as well work around it as above).
> 
> Paolo

Re: [Qemu-devel] virtio_stw_phys_cached crash

[Paolo Bonzini]

On 23/08/2018 10:43, Fam Zheng wrote:
>> Does this help?
> Nope. The crash happens the same (once you press "Auto Install" at the first
> prompt).

The patch has a typo; I have now sent it properly.

Paolo