[Qemu-devel] [PATCH v2 for-2.8 0/3] virtio fixes

[Qemu-devel] [PATCH v2 for-2.8 0/3] virtio fixes

[Paolo Bonzini]

Patch 1 fixes vhost, patches 2-3 fix Windows hibernation.

Paolo

v1->v2: more comments [Cornelia]
        squash syntax error fix from patch 3 into patch 2 [Christian]

Paolo Bonzini (3):
  virtio: introduce grab/release_ioeventfd to fix vhost
  virtio: access ISR atomically
  virtio: set ISR on dataplane notifications

 hw/block/dataplane/virtio-blk.c |  4 +--
 hw/scsi/virtio-scsi-dataplane.c |  7 -----
 hw/scsi/virtio-scsi.c           |  2 +-
 hw/virtio/trace-events          |  2 +-
 hw/virtio/vhost.c               | 14 +++++-----
 hw/virtio/virtio-bus.c          | 58 +++++++++++++++++++++++++++++++++--------
 hw/virtio/virtio-mmio.c         |  6 ++---
 hw/virtio/virtio-pci.c          |  9 +++----
 hw/virtio/virtio.c              | 46 +++++++++++++++++++++++++-------
 include/hw/virtio/virtio-bus.h  | 14 ++++++++++
 include/hw/virtio/virtio-scsi.h |  1 -
 include/hw/virtio/virtio.h      |  4 ++-
 12 files changed, 117 insertions(+), 50 deletions(-)

-- 
2.9.3

[Qemu-devel] [PATCH 1/3] virtio: introduce grab/release_ioeventfd to fix vhost

[Paolo Bonzini]

Following the recent refactoring of virtio notifiers [1], more specifically
the patch ed08a2a0b ("virtio: use virtio_bus_set_host_notifier to
start/stop ioeventfd") that uses virtio_bus_set_host_notifier [2]
by default, core virtio code requires 'ioeventfd_started' to be set
to true/false when the host notifiers are configured.

When vhost is stopped and started, however, there is a stop followed by
another start. Since ioeventfd_started was never set to true, the 'stop'
operation triggered by virtio_bus_set_host_notifier() will not result
in a call to virtio_pci_ioeventfd_assign(assign=false). This leaves
the memory regions with stale notifiers and results on the next start
triggering the following assertion:

  kvm_mem_ioeventfd_add: error adding ioeventfd: File exists
  Aborted

This patch reintroduces (hopefully in a cleaner way) the concept
that was present with ioeventfd_disabled before the refactoring.
When ioeventfd_grabbed>0, ioeventfd_started tracks whether ioeventfd
should be enabled or not, but ioeventfd is actually not started at
all until vhost releases the host notifiers.

[1] http://lists.nongnu.org/archive/html/qemu-devel/2016-10/msg07748.html
[2] http://lists.nongnu.org/archive/html/qemu-devel/2016-10/msg07760.html

Reported-by: Felipe Franciosi <felipe@nutanix.com>
Reported-by: Christian Borntraeger <borntraeger@de.ibm.com>
Reported-by: Alex Williamson <alex.williamson@redhat.com>
Fixes: ed08a2a0b ("virtio: use virtio_bus_set_host_notifier to start/stop ioeventfd")
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20161111192855.26350-1-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
        v1->v2: more comments [Cornelia]

 hw/virtio/vhost.c              | 14 +++++-----
 hw/virtio/virtio-bus.c         | 58 ++++++++++++++++++++++++++++++++++--------
 hw/virtio/virtio.c             | 16 ++++++++++++
 include/hw/virtio/virtio-bus.h | 14 ++++++++++
 include/hw/virtio/virtio.h     |  2 ++
 5 files changed, 86 insertions(+), 18 deletions(-)

diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index 30aee88..f7f7023 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -1214,17 +1214,17 @@ void vhost_dev_cleanup(struct vhost_dev *hdev)
 int vhost_dev_enable_notifiers(struct vhost_dev *hdev, VirtIODevice *vdev)
 {
     BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev)));
-    VirtioBusState *vbus = VIRTIO_BUS(qbus);
-    VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(vbus);
     int i, r, e;
 
-    if (!k->ioeventfd_assign) {
+    /* We will pass the notifiers to the kernel, make sure that QEMU
+     * doesn't interfere.
+     */
+    r = virtio_device_grab_ioeventfd(vdev);
+    if (r < 0) {
         error_report("binding does not support host notifiers");
-        r = -ENOSYS;
         goto fail;
     }
 
-    virtio_device_stop_ioeventfd(vdev);
     for (i = 0; i < hdev->nvqs; ++i) {
         r = virtio_bus_set_host_notifier(VIRTIO_BUS(qbus), hdev->vq_index + i,
                                          true);
@@ -1244,7 +1244,7 @@ fail_vq:
         }
         assert (e >= 0);
     }
-    virtio_device_start_ioeventfd(vdev);
+    virtio_device_release_ioeventfd(vdev);
 fail:
     return r;
 }
@@ -1267,7 +1267,7 @@ void vhost_dev_disable_notifiers(struct vhost_dev *hdev, VirtIODevice *vdev)
         }
         assert (r >= 0);
     }
-    virtio_device_start_ioeventfd(vdev);
+    virtio_device_release_ioeventfd(vdev);
 }
 
 /* Test and clear event pending status.
diff --git a/hw/virtio/virtio-bus.c b/hw/virtio/virtio-bus.c
index bf61f66..d6c0c72 100644
--- a/hw/virtio/virtio-bus.c
+++ b/hw/virtio/virtio-bus.c
@@ -147,6 +147,39 @@ void virtio_bus_set_vdev_config(VirtioBusState *bus, uint8_t *config)
     }
 }
 
+/* On success, ioeventfd ownership belongs to the caller.  */
+int virtio_bus_grab_ioeventfd(VirtioBusState *bus)
+{
+    VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(bus);
+
+    /* vhost can be used even if ioeventfd=off in the proxy device,
+     * so do not check k->ioeventfd_enabled.
+     */
+    if (!k->ioeventfd_assign) {
+        return -ENOSYS;
+    }
+
+    if (bus->ioeventfd_grabbed == 0 && bus->ioeventfd_started) {
+        virtio_bus_stop_ioeventfd(bus);
+        /* Remember that we need to restart ioeventfd
+         * when ioeventfd_grabbed becomes zero.
+         */
+        bus->ioeventfd_started = true;
+    }
+    bus->ioeventfd_grabbed++;
+    return 0;
+}
+
+void virtio_bus_release_ioeventfd(VirtioBusState *bus)
+{
+    assert(bus->ioeventfd_grabbed != 0);
+    if (--bus->ioeventfd_grabbed == 0 && bus->ioeventfd_started) {
+        /* Force virtio_bus_start_ioeventfd to act.  */
+        bus->ioeventfd_started = false;
+        virtio_bus_start_ioeventfd(bus);
+    }
+}
+
 int virtio_bus_start_ioeventfd(VirtioBusState *bus)
 {
     VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(bus);
@@ -161,10 +194,14 @@ int virtio_bus_start_ioeventfd(VirtioBusState *bus)
     if (bus->ioeventfd_started) {
         return 0;
     }
-    r = vdc->start_ioeventfd(vdev);
-    if (r < 0) {
-        error_report("%s: failed. Fallback to userspace (slower).", __func__);
-        return r;
+
+    /* Only set our notifier if we have ownership.  */
+    if (!bus->ioeventfd_grabbed) {
+        r = vdc->start_ioeventfd(vdev);
+        if (r < 0) {
+            error_report("%s: failed. Fallback to userspace (slower).", __func__);
+            return r;
+        }
     }
     bus->ioeventfd_started = true;
     return 0;
@@ -179,9 +216,12 @@ void virtio_bus_stop_ioeventfd(VirtioBusState *bus)
         return;
     }
 
-    vdev = virtio_bus_get_device(bus);
-    vdc = VIRTIO_DEVICE_GET_CLASS(vdev);
-    vdc->stop_ioeventfd(vdev);
+    /* Only remove our notifier if we have ownership.  */
+    if (!bus->ioeventfd_grabbed) {
+        vdev = virtio_bus_get_device(bus);
+        vdc = VIRTIO_DEVICE_GET_CLASS(vdev);
+        vdc->stop_ioeventfd(vdev);
+    }
     bus->ioeventfd_started = false;
 }
 
@@ -211,7 +251,6 @@ int virtio_bus_set_host_notifier(VirtioBusState *bus, int n, bool assign)
     }
 
     if (assign) {
-        assert(!bus->ioeventfd_started);
         r = event_notifier_init(notifier, 1);
         if (r < 0) {
             error_report("%s: unable to init event notifier: %s (%d)",
@@ -225,9 +264,6 @@ int virtio_bus_set_host_notifier(VirtioBusState *bus, int n, bool assign)
         }
         return 0;
     } else {
-        if (!bus->ioeventfd_started) {
-            return 0;
-        }
         k->ioeventfd_assign(proxy, notifier, n, false);
     }
 
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index 55a00cd..b7d5828 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -2191,6 +2191,22 @@ void virtio_device_stop_ioeventfd(VirtIODevice *vdev)
     virtio_bus_stop_ioeventfd(vbus);
 }
 
+int virtio_device_grab_ioeventfd(VirtIODevice *vdev)
+{
+    BusState *qbus = qdev_get_parent_bus(DEVICE(vdev));
+    VirtioBusState *vbus = VIRTIO_BUS(qbus);
+
+    return virtio_bus_grab_ioeventfd(vbus);
+}
+
+void virtio_device_release_ioeventfd(VirtIODevice *vdev)
+{
+    BusState *qbus = qdev_get_parent_bus(DEVICE(vdev));
+    VirtioBusState *vbus = VIRTIO_BUS(qbus);
+
+    virtio_bus_release_ioeventfd(vbus);
+}
+
 static void virtio_device_class_init(ObjectClass *klass, void *data)
 {
     /* Set the default value here. */
diff --git a/include/hw/virtio/virtio-bus.h b/include/hw/virtio/virtio-bus.h
index fdf7fda..8a51e2c 100644
--- a/include/hw/virtio/virtio-bus.h
+++ b/include/hw/virtio/virtio-bus.h
@@ -97,6 +97,16 @@ struct VirtioBusState {
      * Set if ioeventfd has been started.
      */
     bool ioeventfd_started;
+
+    /*
+     * Set if ioeventfd has been grabbed by vhost.  When ioeventfd
+     * is grabbed by vhost, we track its started/stopped state (which
+     * depends in turn on the virtio status register), but do not
+     * register a handler for the ioeventfd.  When ioeventfd is
+     * released, if ioeventfd_started is true we finally register
+     * the handler so that QEMU's device model can use ioeventfd.
+     */
+    int ioeventfd_grabbed;
 };
 
 void virtio_bus_device_plugged(VirtIODevice *vdev, Error **errp);
@@ -131,6 +141,10 @@ bool virtio_bus_ioeventfd_enabled(VirtioBusState *bus);
 int virtio_bus_start_ioeventfd(VirtioBusState *bus);
 /* Stop the ioeventfd. */
 void virtio_bus_stop_ioeventfd(VirtioBusState *bus);
+/* Tell the bus that vhost is grabbing the ioeventfd. */
+int virtio_bus_grab_ioeventfd(VirtioBusState *bus);
+/* bus that vhost is not using the ioeventfd anymore. */
+void virtio_bus_release_ioeventfd(VirtioBusState *bus);
 /* Switch from/to the generic ioeventfd handler */
 int virtio_bus_set_host_notifier(VirtioBusState *bus, int n, bool assign);
 
diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index 5951997..835b085 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -272,6 +272,8 @@ void virtio_queue_set_guest_notifier_fd_handler(VirtQueue *vq, bool assign,
                                                 bool with_irqfd);
 int virtio_device_start_ioeventfd(VirtIODevice *vdev);
 void virtio_device_stop_ioeventfd(VirtIODevice *vdev);
+int virtio_device_grab_ioeventfd(VirtIODevice *vdev);
+void virtio_device_release_ioeventfd(VirtIODevice *vdev);
 bool virtio_device_ioeventfd_enabled(VirtIODevice *vdev);
 EventNotifier *virtio_queue_get_host_notifier(VirtQueue *vq);
 void virtio_queue_host_notifier_read(EventNotifier *n);
-- 
2.9.3

Re: [Qemu-devel] [PATCH 1/3] virtio: introduce grab/release_ioeventfd to fix vhost

[Alexey Kardashevskiy]

On 17/11/16 05:05, Paolo Bonzini wrote:
> Following the recent refactoring of virtio notifiers [1], more specifically
> the patch ed08a2a0b ("virtio: use virtio_bus_set_host_notifier to
> start/stop ioeventfd") that uses virtio_bus_set_host_notifier [2]
> by default, core virtio code requires 'ioeventfd_started' to be set
> to true/false when the host notifiers are configured.
> 
> When vhost is stopped and started, however, there is a stop followed by
> another start. Since ioeventfd_started was never set to true, the 'stop'
> operation triggered by virtio_bus_set_host_notifier() will not result
> in a call to virtio_pci_ioeventfd_assign(assign=false). This leaves
> the memory regions with stale notifiers and results on the next start
> triggering the following assertion:
> 
>   kvm_mem_ioeventfd_add: error adding ioeventfd: File exists
>   Aborted
> 
> This patch reintroduces (hopefully in a cleaner way) the concept
> that was present with ioeventfd_disabled before the refactoring.
> When ioeventfd_grabbed>0, ioeventfd_started tracks whether ioeventfd
> should be enabled or not, but ioeventfd is actually not started at
> all until vhost releases the host notifiers.
> 
> [1] http://lists.nongnu.org/archive/html/qemu-devel/2016-10/msg07748.html
> [2] http://lists.nongnu.org/archive/html/qemu-devel/2016-10/msg07760.html
> 
> Reported-by: Felipe Franciosi <felipe@nutanix.com>
> Reported-by: Christian Borntraeger <borntraeger@de.ibm.com>
> Reported-by: Alex Williamson <alex.williamson@redhat.com>
> Fixes: ed08a2a0b ("virtio: use virtio_bus_set_host_notifier to start/stop ioeventfd")
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> Message-Id: <20161111192855.26350-1-pbonzini@redhat.com>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>


As mentioned in another thread, this fixed vhost on ppc64/pseries.


Tested-by: Alexey Kardashevskiy <aik@ozlabs.ru>


> ---
>         v1->v2: more comments [Cornelia]
> 
>  hw/virtio/vhost.c              | 14 +++++-----
>  hw/virtio/virtio-bus.c         | 58 ++++++++++++++++++++++++++++++++++--------
>  hw/virtio/virtio.c             | 16 ++++++++++++
>  include/hw/virtio/virtio-bus.h | 14 ++++++++++
>  include/hw/virtio/virtio.h     |  2 ++
>  5 files changed, 86 insertions(+), 18 deletions(-)
> 
> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> index 30aee88..f7f7023 100644
> --- a/hw/virtio/vhost.c
> +++ b/hw/virtio/vhost.c
> @@ -1214,17 +1214,17 @@ void vhost_dev_cleanup(struct vhost_dev *hdev)
>  int vhost_dev_enable_notifiers(struct vhost_dev *hdev, VirtIODevice *vdev)
>  {
>      BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev)));
> -    VirtioBusState *vbus = VIRTIO_BUS(qbus);
> -    VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(vbus);
>      int i, r, e;
>  
> -    if (!k->ioeventfd_assign) {
> +    /* We will pass the notifiers to the kernel, make sure that QEMU
> +     * doesn't interfere.
> +     */
> +    r = virtio_device_grab_ioeventfd(vdev);
> +    if (r < 0) {
>          error_report("binding does not support host notifiers");
> -        r = -ENOSYS;
>          goto fail;
>      }
>  
> -    virtio_device_stop_ioeventfd(vdev);
>      for (i = 0; i < hdev->nvqs; ++i) {
>          r = virtio_bus_set_host_notifier(VIRTIO_BUS(qbus), hdev->vq_index + i,
>                                           true);
> @@ -1244,7 +1244,7 @@ fail_vq:
>          }
>          assert (e >= 0);
>      }
> -    virtio_device_start_ioeventfd(vdev);
> +    virtio_device_release_ioeventfd(vdev);
>  fail:
>      return r;
>  }
> @@ -1267,7 +1267,7 @@ void vhost_dev_disable_notifiers(struct vhost_dev *hdev, VirtIODevice *vdev)
>          }
>          assert (r >= 0);
>      }
> -    virtio_device_start_ioeventfd(vdev);
> +    virtio_device_release_ioeventfd(vdev);
>  }
>  
>  /* Test and clear event pending status.
> diff --git a/hw/virtio/virtio-bus.c b/hw/virtio/virtio-bus.c
> index bf61f66..d6c0c72 100644
> --- a/hw/virtio/virtio-bus.c
> +++ b/hw/virtio/virtio-bus.c
> @@ -147,6 +147,39 @@ void virtio_bus_set_vdev_config(VirtioBusState *bus, uint8_t *config)
>      }
>  }
>  
> +/* On success, ioeventfd ownership belongs to the caller.  */
> +int virtio_bus_grab_ioeventfd(VirtioBusState *bus)
> +{
> +    VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(bus);
> +
> +    /* vhost can be used even if ioeventfd=off in the proxy device,
> +     * so do not check k->ioeventfd_enabled.
> +     */
> +    if (!k->ioeventfd_assign) {
> +        return -ENOSYS;
> +    }
> +
> +    if (bus->ioeventfd_grabbed == 0 && bus->ioeventfd_started) {
> +        virtio_bus_stop_ioeventfd(bus);
> +        /* Remember that we need to restart ioeventfd
> +         * when ioeventfd_grabbed becomes zero.
> +         */
> +        bus->ioeventfd_started = true;
> +    }
> +    bus->ioeventfd_grabbed++;
> +    return 0;
> +}
> +
> +void virtio_bus_release_ioeventfd(VirtioBusState *bus)
> +{
> +    assert(bus->ioeventfd_grabbed != 0);
> +    if (--bus->ioeventfd_grabbed == 0 && bus->ioeventfd_started) {
> +        /* Force virtio_bus_start_ioeventfd to act.  */
> +        bus->ioeventfd_started = false;
> +        virtio_bus_start_ioeventfd(bus);
> +    }
> +}
> +
>  int virtio_bus_start_ioeventfd(VirtioBusState *bus)
>  {
>      VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(bus);
> @@ -161,10 +194,14 @@ int virtio_bus_start_ioeventfd(VirtioBusState *bus)
>      if (bus->ioeventfd_started) {
>          return 0;
>      }
> -    r = vdc->start_ioeventfd(vdev);
> -    if (r < 0) {
> -        error_report("%s: failed. Fallback to userspace (slower).", __func__);
> -        return r;
> +
> +    /* Only set our notifier if we have ownership.  */
> +    if (!bus->ioeventfd_grabbed) {
> +        r = vdc->start_ioeventfd(vdev);
> +        if (r < 0) {
> +            error_report("%s: failed. Fallback to userspace (slower).", __func__);
> +            return r;
> +        }
>      }
>      bus->ioeventfd_started = true;
>      return 0;
> @@ -179,9 +216,12 @@ void virtio_bus_stop_ioeventfd(VirtioBusState *bus)
>          return;
>      }
>  
> -    vdev = virtio_bus_get_device(bus);
> -    vdc = VIRTIO_DEVICE_GET_CLASS(vdev);
> -    vdc->stop_ioeventfd(vdev);
> +    /* Only remove our notifier if we have ownership.  */
> +    if (!bus->ioeventfd_grabbed) {
> +        vdev = virtio_bus_get_device(bus);
> +        vdc = VIRTIO_DEVICE_GET_CLASS(vdev);
> +        vdc->stop_ioeventfd(vdev);
> +    }
>      bus->ioeventfd_started = false;
>  }
>  
> @@ -211,7 +251,6 @@ int virtio_bus_set_host_notifier(VirtioBusState *bus, int n, bool assign)
>      }
>  
>      if (assign) {
> -        assert(!bus->ioeventfd_started);
>          r = event_notifier_init(notifier, 1);
>          if (r < 0) {
>              error_report("%s: unable to init event notifier: %s (%d)",
> @@ -225,9 +264,6 @@ int virtio_bus_set_host_notifier(VirtioBusState *bus, int n, bool assign)
>          }
>          return 0;
>      } else {
> -        if (!bus->ioeventfd_started) {
> -            return 0;
> -        }
>          k->ioeventfd_assign(proxy, notifier, n, false);
>      }
>  
> diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
> index 55a00cd..b7d5828 100644
> --- a/hw/virtio/virtio.c
> +++ b/hw/virtio/virtio.c
> @@ -2191,6 +2191,22 @@ void virtio_device_stop_ioeventfd(VirtIODevice *vdev)
>      virtio_bus_stop_ioeventfd(vbus);
>  }
>  
> +int virtio_device_grab_ioeventfd(VirtIODevice *vdev)
> +{
> +    BusState *qbus = qdev_get_parent_bus(DEVICE(vdev));
> +    VirtioBusState *vbus = VIRTIO_BUS(qbus);
> +
> +    return virtio_bus_grab_ioeventfd(vbus);
> +}
> +
> +void virtio_device_release_ioeventfd(VirtIODevice *vdev)
> +{
> +    BusState *qbus = qdev_get_parent_bus(DEVICE(vdev));
> +    VirtioBusState *vbus = VIRTIO_BUS(qbus);
> +
> +    virtio_bus_release_ioeventfd(vbus);
> +}
> +
>  static void virtio_device_class_init(ObjectClass *klass, void *data)
>  {
>      /* Set the default value here. */
> diff --git a/include/hw/virtio/virtio-bus.h b/include/hw/virtio/virtio-bus.h
> index fdf7fda..8a51e2c 100644
> --- a/include/hw/virtio/virtio-bus.h
> +++ b/include/hw/virtio/virtio-bus.h
> @@ -97,6 +97,16 @@ struct VirtioBusState {
>       * Set if ioeventfd has been started.
>       */
>      bool ioeventfd_started;
> +
> +    /*
> +     * Set if ioeventfd has been grabbed by vhost.  When ioeventfd
> +     * is grabbed by vhost, we track its started/stopped state (which
> +     * depends in turn on the virtio status register), but do not
> +     * register a handler for the ioeventfd.  When ioeventfd is
> +     * released, if ioeventfd_started is true we finally register
> +     * the handler so that QEMU's device model can use ioeventfd.
> +     */
> +    int ioeventfd_grabbed;
>  };
>  
>  void virtio_bus_device_plugged(VirtIODevice *vdev, Error **errp);
> @@ -131,6 +141,10 @@ bool virtio_bus_ioeventfd_enabled(VirtioBusState *bus);
>  int virtio_bus_start_ioeventfd(VirtioBusState *bus);
>  /* Stop the ioeventfd. */
>  void virtio_bus_stop_ioeventfd(VirtioBusState *bus);
> +/* Tell the bus that vhost is grabbing the ioeventfd. */
> +int virtio_bus_grab_ioeventfd(VirtioBusState *bus);
> +/* bus that vhost is not using the ioeventfd anymore. */
> +void virtio_bus_release_ioeventfd(VirtioBusState *bus);
>  /* Switch from/to the generic ioeventfd handler */
>  int virtio_bus_set_host_notifier(VirtioBusState *bus, int n, bool assign);
>  
> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> index 5951997..835b085 100644
> --- a/include/hw/virtio/virtio.h
> +++ b/include/hw/virtio/virtio.h
> @@ -272,6 +272,8 @@ void virtio_queue_set_guest_notifier_fd_handler(VirtQueue *vq, bool assign,
>                                                  bool with_irqfd);
>  int virtio_device_start_ioeventfd(VirtIODevice *vdev);
>  void virtio_device_stop_ioeventfd(VirtIODevice *vdev);
> +int virtio_device_grab_ioeventfd(VirtIODevice *vdev);
> +void virtio_device_release_ioeventfd(VirtIODevice *vdev);
>  bool virtio_device_ioeventfd_enabled(VirtIODevice *vdev);
>  EventNotifier *virtio_queue_get_host_notifier(VirtQueue *vq);
>  void virtio_queue_host_notifier_read(EventNotifier *n);
> 


-- 
Alexey

Re: [Qemu-devel] [PATCH 1/3] virtio: introduce grab/release_ioeventfd to fix vhost

[Cornelia Huck]

On Wed, 16 Nov 2016 19:05:49 +0100
Paolo Bonzini <pbonzini@redhat.com> wrote:

> Following the recent refactoring of virtio notifiers [1], more specifically
> the patch ed08a2a0b ("virtio: use virtio_bus_set_host_notifier to
> start/stop ioeventfd") that uses virtio_bus_set_host_notifier [2]
> by default, core virtio code requires 'ioeventfd_started' to be set
> to true/false when the host notifiers are configured.
> 
> When vhost is stopped and started, however, there is a stop followed by
> another start. Since ioeventfd_started was never set to true, the 'stop'
> operation triggered by virtio_bus_set_host_notifier() will not result
> in a call to virtio_pci_ioeventfd_assign(assign=false). This leaves
> the memory regions with stale notifiers and results on the next start
> triggering the following assertion:
> 
>   kvm_mem_ioeventfd_add: error adding ioeventfd: File exists
>   Aborted
> 
> This patch reintroduces (hopefully in a cleaner way) the concept
> that was present with ioeventfd_disabled before the refactoring.
> When ioeventfd_grabbed>0, ioeventfd_started tracks whether ioeventfd
> should be enabled or not, but ioeventfd is actually not started at
> all until vhost releases the host notifiers.
> 
> [1] http://lists.nongnu.org/archive/html/qemu-devel/2016-10/msg07748.html
> [2] http://lists.nongnu.org/archive/html/qemu-devel/2016-10/msg07760.html
> 
> Reported-by: Felipe Franciosi <felipe@nutanix.com>
> Reported-by: Christian Borntraeger <borntraeger@de.ibm.com>
> Reported-by: Alex Williamson <alex.williamson@redhat.com>
> Fixes: ed08a2a0b ("virtio: use virtio_bus_set_host_notifier to start/stop ioeventfd")
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> Message-Id: <20161111192855.26350-1-pbonzini@redhat.com>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>         v1->v2: more comments [Cornelia]
> 
>  hw/virtio/vhost.c              | 14 +++++-----
>  hw/virtio/virtio-bus.c         | 58 ++++++++++++++++++++++++++++++++++--------
>  hw/virtio/virtio.c             | 16 ++++++++++++
>  include/hw/virtio/virtio-bus.h | 14 ++++++++++
>  include/hw/virtio/virtio.h     |  2 ++
>  5 files changed, 86 insertions(+), 18 deletions(-)

Reviewed-by: Cornelia Huck <cornelia.huck@de.ibm.com>

Re: [Qemu-devel] [PATCH 1/3] virtio: introduce grab/release_ioeventfd to fix vhost

[Christian Borntraeger]

On 11/16/2016 07:05 PM, Paolo Bonzini wrote:
> Following the recent refactoring of virtio notifiers [1], more specifically
> the patch ed08a2a0b ("virtio: use virtio_bus_set_host_notifier to
> start/stop ioeventfd") that uses virtio_bus_set_host_notifier [2]
> by default, core virtio code requires 'ioeventfd_started' to be set
> to true/false when the host notifiers are configured.
> 
> When vhost is stopped and started, however, there is a stop followed by
> another start. Since ioeventfd_started was never set to true, the 'stop'
> operation triggered by virtio_bus_set_host_notifier() will not result
> in a call to virtio_pci_ioeventfd_assign(assign=false). This leaves
> the memory regions with stale notifiers and results on the next start
> triggering the following assertion:
> 
>   kvm_mem_ioeventfd_add: error adding ioeventfd: File exists
>   Aborted
> 
> This patch reintroduces (hopefully in a cleaner way) the concept
> that was present with ioeventfd_disabled before the refactoring.
> When ioeventfd_grabbed>0, ioeventfd_started tracks whether ioeventfd
> should be enabled or not, but ioeventfd is actually not started at
> all until vhost releases the host notifiers.
> 
> [1] http://lists.nongnu.org/archive/html/qemu-devel/2016-10/msg07748.html
> [2] http://lists.nongnu.org/archive/html/qemu-devel/2016-10/msg07760.html
> 
> Reported-by: Felipe Franciosi <felipe@nutanix.com>
> Reported-by: Christian Borntraeger <borntraeger@de.ibm.com>
> Reported-by: Alex Williamson <alex.williamson@redhat.com>
> Fixes: ed08a2a0b ("virtio: use virtio_bus_set_host_notifier to start/stop ioeventfd")
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> Message-Id: <20161111192855.26350-1-pbonzini@redhat.com>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>         v1->v2: more comments [Cornelia]


As this seems to fix a functional issues, is there any chance to apply this
patch now and not wait for the discussion about patch 2 and 3 to calm down?

Re: [Qemu-devel] [PATCH 1/3] virtio: introduce grab/release_ioeventfd to fix vhost

[Michael S. Tsirkin]

On Fri, Nov 18, 2016 at 09:15:32AM +0100, Christian Borntraeger wrote:
> On 11/16/2016 07:05 PM, Paolo Bonzini wrote:
> > Following the recent refactoring of virtio notifiers [1], more specifically
> > the patch ed08a2a0b ("virtio: use virtio_bus_set_host_notifier to
> > start/stop ioeventfd") that uses virtio_bus_set_host_notifier [2]
> > by default, core virtio code requires 'ioeventfd_started' to be set
> > to true/false when the host notifiers are configured.
> > 
> > When vhost is stopped and started, however, there is a stop followed by
> > another start. Since ioeventfd_started was never set to true, the 'stop'
> > operation triggered by virtio_bus_set_host_notifier() will not result
> > in a call to virtio_pci_ioeventfd_assign(assign=false). This leaves
> > the memory regions with stale notifiers and results on the next start
> > triggering the following assertion:
> > 
> >   kvm_mem_ioeventfd_add: error adding ioeventfd: File exists
> >   Aborted
> > 
> > This patch reintroduces (hopefully in a cleaner way) the concept
> > that was present with ioeventfd_disabled before the refactoring.
> > When ioeventfd_grabbed>0, ioeventfd_started tracks whether ioeventfd
> > should be enabled or not, but ioeventfd is actually not started at
> > all until vhost releases the host notifiers.
> > 
> > [1] http://lists.nongnu.org/archive/html/qemu-devel/2016-10/msg07748.html
> > [2] http://lists.nongnu.org/archive/html/qemu-devel/2016-10/msg07760.html
> > 
> > Reported-by: Felipe Franciosi <felipe@nutanix.com>
> > Reported-by: Christian Borntraeger <borntraeger@de.ibm.com>
> > Reported-by: Alex Williamson <alex.williamson@redhat.com>
> > Fixes: ed08a2a0b ("virtio: use virtio_bus_set_host_notifier to start/stop ioeventfd")
> > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> > Message-Id: <20161111192855.26350-1-pbonzini@redhat.com>
> > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> > ---
> >         v1->v2: more comments [Cornelia]
> 
> 
> As this seems to fix a functional issues, is there any chance to apply this
> patch now and not wait for the discussion about patch 2 and 3 to calm down?

It's in my tree, will be in the next pull.

[Qemu-devel] [PATCH 2/3] virtio: access ISR atomically

[Paolo Bonzini]

This will be needed once dataplane will be able to set it outside
the big QEMU lock.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
        v1->v2: squash syntax error fix from patch 3 [Christian]

 hw/virtio/virtio-mmio.c |  6 +++---
 hw/virtio/virtio-pci.c  |  9 +++------
 hw/virtio/virtio.c      | 18 +++++++++++++-----
 3 files changed, 19 insertions(+), 14 deletions(-)

diff --git a/hw/virtio/virtio-mmio.c b/hw/virtio/virtio-mmio.c
index a30270f..17412cb 100644
--- a/hw/virtio/virtio-mmio.c
+++ b/hw/virtio/virtio-mmio.c
@@ -191,7 +191,7 @@ static uint64_t virtio_mmio_read(void *opaque, hwaddr offset, unsigned size)
         return virtio_queue_get_addr(vdev, vdev->queue_sel)
             >> proxy->guest_page_shift;
     case VIRTIO_MMIO_INTERRUPTSTATUS:
-        return vdev->isr;
+        return atomic_read(&vdev->isr);
     case VIRTIO_MMIO_STATUS:
         return vdev->status;
     case VIRTIO_MMIO_HOSTFEATURESSEL:
@@ -299,7 +299,7 @@ static void virtio_mmio_write(void *opaque, hwaddr offset, uint64_t value,
         }
         break;
     case VIRTIO_MMIO_INTERRUPTACK:
-        vdev->isr &= ~value;
+        atomic_and(&vdev->isr, ~value);
         virtio_update_irq(vdev);
         break;
     case VIRTIO_MMIO_STATUS:
@@ -347,7 +347,7 @@ static void virtio_mmio_update_irq(DeviceState *opaque, uint16_t vector)
     if (!vdev) {
         return;
     }
-    level = (vdev->isr != 0);
+    level = (atomic_read(&vdev->isr) != 0);
     DPRINTF("virtio_mmio setting IRQ %d\n", level);
     qemu_set_irq(proxy->irq, level);
 }
diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
index 97b32fe..521ba0b 100644
--- a/hw/virtio/virtio-pci.c
+++ b/hw/virtio/virtio-pci.c
@@ -73,7 +73,7 @@ static void virtio_pci_notify(DeviceState *d, uint16_t vector)
         msix_notify(&proxy->pci_dev, vector);
     else {
         VirtIODevice *vdev = virtio_bus_get_device(&proxy->bus);
-        pci_set_irq(&proxy->pci_dev, vdev->isr & 1);
+        pci_set_irq(&proxy->pci_dev, atomic_read(&vdev->isr) & 1);
     }
 }
 
@@ -449,8 +449,7 @@ static uint32_t virtio_ioport_read(VirtIOPCIProxy *proxy, uint32_t addr)
         break;
     case VIRTIO_PCI_ISR:
         /* reading from the ISR also clears it. */
-        ret = vdev->isr;
-        vdev->isr = 0;
+        ret = atomic_xchg(&vdev->isr, 0);
         pci_irq_deassert(&proxy->pci_dev);
         break;
     case VIRTIO_MSI_CONFIG_VECTOR:
@@ -1379,9 +1378,7 @@ static uint64_t virtio_pci_isr_read(void *opaque, hwaddr addr,
 {
     VirtIOPCIProxy *proxy = opaque;
     VirtIODevice *vdev = virtio_bus_get_device(&proxy->bus);
-    uint64_t val = vdev->isr;
-
-    vdev->isr = 0;
+    uint64_t val = atomic_xchg(&vdev->isr, 0);
     pci_irq_deassert(&proxy->pci_dev);
 
     return val;
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index b7d5828..ecf13bd 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -945,7 +945,7 @@ void virtio_reset(void *opaque)
     vdev->guest_features = 0;
     vdev->queue_sel = 0;
     vdev->status = 0;
-    vdev->isr = 0;
+    atomic_set(&vdev->isr, 0);
     vdev->config_vector = VIRTIO_NO_VECTOR;
     virtio_notify_vector(vdev, vdev->config_vector);
 
@@ -1318,10 +1318,18 @@ void virtio_del_queue(VirtIODevice *vdev, int n)
     vdev->vq[n].vring.num_default = 0;
 }
 
+static void virtio_set_isr(VirtIODevice *vdev, int value)
+{
+    uint8_t old = atomic_read(&vdev->isr);
+    if ((old & value) != value) {
+        atomic_or(&vdev->isr, value);
+    }
+}
+
 void virtio_irq(VirtQueue *vq)
 {
     trace_virtio_irq(vq);
-    vq->vdev->isr |= 0x01;
+    virtio_set_isr(vq->vdev, 0x1);
     virtio_notify_vector(vq->vdev, vq->vector);
 }
 
@@ -1355,7 +1363,7 @@ void virtio_notify(VirtIODevice *vdev, VirtQueue *vq)
     }
 
     trace_virtio_notify(vdev, vq);
-    vdev->isr |= 0x01;
+    virtio_set_isr(vq->vdev, 0x1);
     virtio_notify_vector(vdev, vq->vector);
 }
 
@@ -1364,7 +1372,7 @@ void virtio_notify_config(VirtIODevice *vdev)
     if (!(vdev->status & VIRTIO_CONFIG_S_DRIVER_OK))
         return;
 
-    vdev->isr |= 0x03;
+    virtio_set_isr(vdev, 0x3);
     vdev->generation++;
     virtio_notify_vector(vdev, vdev->config_vector);
 }
@@ -1895,7 +1903,7 @@ void virtio_init(VirtIODevice *vdev, const char *name,
 
     vdev->device_id = device_id;
     vdev->status = 0;
-    vdev->isr = 0;
+    atomic_set(&vdev->isr, 0);
     vdev->queue_sel = 0;
     vdev->config_vector = VIRTIO_NO_VECTOR;
     vdev->vq = g_malloc0(sizeof(VirtQueue) * VIRTIO_QUEUE_MAX);
-- 
2.9.3

Re: [Qemu-devel] [PATCH 2/3] virtio: access ISR atomically

[Michael S. Tsirkin]

On Wed, Nov 16, 2016 at 07:05:50PM +0100, Paolo Bonzini wrote:
> @@ -1318,10 +1318,18 @@ void virtio_del_queue(VirtIODevice *vdev, int n)
>      vdev->vq[n].vring.num_default = 0;
>  }
>  
> +static void virtio_set_isr(VirtIODevice *vdev, int value)
> +{
> +    uint8_t old = atomic_read(&vdev->isr);
> +    if ((old & value) != value) {
> +        atomic_or(&vdev->isr, value);
> +    }
> +}
> +

Paolo, can you pls comment on why is it done like this,
as opposed to a single atomic_or?

-- 
MST

Re: [Qemu-devel] [PATCH 2/3] virtio: access ISR atomically

[Paolo Bonzini]

----- Original Message -----
> From: "Michael S. Tsirkin" <mst@redhat.com>
> To: "Paolo Bonzini" <pbonzini@redhat.com>
> Cc: qemu-devel@nongnu.org, "alex williamson" <alex.williamson@redhat.com>, borntraeger@de.ibm.com, felipe@nutanix.com
> Sent: Thursday, November 17, 2016 6:55:50 PM
> Subject: Re: [PATCH 2/3] virtio: access ISR atomically
> 
> On Wed, Nov 16, 2016 at 07:05:50PM +0100, Paolo Bonzini wrote:
> > @@ -1318,10 +1318,18 @@ void virtio_del_queue(VirtIODevice *vdev, int n)
> >      vdev->vq[n].vring.num_default = 0;
> >  }
> >  
> > +static void virtio_set_isr(VirtIODevice *vdev, int value)
> > +{
> > +    uint8_t old = atomic_read(&vdev->isr);
> > +    if ((old & value) != value) {
> > +        atomic_or(&vdev->isr, value);
> > +    }
> > +}
> > +
> 
> Paolo, can you pls comment on why is it done like this,
> as opposed to a single atomic_or?

To avoid cacheline bouncing in the common case where MSI is active
but the host doesn't read ISR.

Paolo

Re: [Qemu-devel] [PATCH 2/3] virtio: access ISR atomically

[Michael S. Tsirkin]

On Thu, Nov 17, 2016 at 02:49:58PM -0500, Paolo Bonzini wrote:
> 
> 
> ----- Original Message -----
> > From: "Michael S. Tsirkin" <mst@redhat.com>
> > To: "Paolo Bonzini" <pbonzini@redhat.com>
> > Cc: qemu-devel@nongnu.org, "alex williamson" <alex.williamson@redhat.com>, borntraeger@de.ibm.com, felipe@nutanix.com
> > Sent: Thursday, November 17, 2016 6:55:50 PM
> > Subject: Re: [PATCH 2/3] virtio: access ISR atomically
> > 
> > On Wed, Nov 16, 2016 at 07:05:50PM +0100, Paolo Bonzini wrote:
> > > @@ -1318,10 +1318,18 @@ void virtio_del_queue(VirtIODevice *vdev, int n)
> > >      vdev->vq[n].vring.num_default = 0;
> > >  }
> > >  
> > > +static void virtio_set_isr(VirtIODevice *vdev, int value)
> > > +{
> > > +    uint8_t old = atomic_read(&vdev->isr);
> > > +    if ((old & value) != value) {
> > > +        atomic_or(&vdev->isr, value);
> > > +    }
> > > +}
> > > +
> > 
> > Paolo, can you pls comment on why is it done like this,
> > as opposed to a single atomic_or?
> 
> To avoid cacheline bouncing in the common case where MSI is active
> but the host doesn't read ISR.
> 
> Paolo

You mean the guest does not read ISR?
And so this helps keep the field read-mostly for all CPUs,
sharing the cache line.
OK but this is worth documenting.

-- 
MST

Re: [Qemu-devel] [PATCH 2/3] virtio: access ISR atomically

[Paolo Bonzini]

----- Original Message -----
> From: "Michael S. Tsirkin" <mst@redhat.com>
> To: "Paolo Bonzini" <pbonzini@redhat.com>
> Cc: qemu-devel@nongnu.org, "alex williamson" <alex.williamson@redhat.com>, borntraeger@de.ibm.com, felipe@nutanix.com
> Sent: Thursday, November 17, 2016 11:33:24 PM
> Subject: Re: [PATCH 2/3] virtio: access ISR atomically
> 
> On Thu, Nov 17, 2016 at 02:49:58PM -0500, Paolo Bonzini wrote:
> > 
> > 
> > ----- Original Message -----
> > > From: "Michael S. Tsirkin" <mst@redhat.com>
> > > To: "Paolo Bonzini" <pbonzini@redhat.com>
> > > Cc: qemu-devel@nongnu.org, "alex williamson"
> > > <alex.williamson@redhat.com>, borntraeger@de.ibm.com, felipe@nutanix.com
> > > Sent: Thursday, November 17, 2016 6:55:50 PM
> > > Subject: Re: [PATCH 2/3] virtio: access ISR atomically
> > > 
> > > On Wed, Nov 16, 2016 at 07:05:50PM +0100, Paolo Bonzini wrote:
> > > > @@ -1318,10 +1318,18 @@ void virtio_del_queue(VirtIODevice *vdev, int
> > > > n)
> > > >      vdev->vq[n].vring.num_default = 0;
> > > >  }
> > > >  
> > > > +static void virtio_set_isr(VirtIODevice *vdev, int value)
> > > > +{
> > > > +    uint8_t old = atomic_read(&vdev->isr);
> > > > +    if ((old & value) != value) {
> > > > +        atomic_or(&vdev->isr, value);
> > > > +    }
> > > > +}
> > > > +
> > > 
> > > Paolo, can you pls comment on why is it done like this,
> > > as opposed to a single atomic_or?
> > 
> > To avoid cacheline bouncing in the common case where MSI is active
> > but the host doesn't read ISR.
> 
> You mean the guest does not read ISR?
> And so this helps keep the field read-mostly for all CPUs,
> sharing the cache line. OK but this is worth documenting.

Yes, v3 on the way.

Paolo

[Qemu-devel] [PATCH 3/3] virtio: set ISR on dataplane notifications

[Paolo Bonzini]

Dataplane has been omitting forever the step of setting ISR when
an interrupt is raised.  This caused little breakage, because the
specification actually says that ISR may not be updated in MSI mode.

Some versions of the Windows drivers however didn't clear MSI mode
correctly, and proceeded using polling mode (using ISR, not the used
ring index!) for crashdump and hibernation.  If it were just crashdump
and hibernation it would not be a big deal, but recent releases of
Windows do not really shut down, but rather log out and hibernate to
make the next startup faster.  Hence, this manifested as a more serious
hang during shutdown with e.g. Windows 8.1 and virtio-win 1.8.0 RPMs.
Newer versions fixed this, while older versions do not use MSI at all.

The failure has always been there for virtio dataplane, but it became
visible after commits 9ffe337 ("virtio-blk: always use dataplane path
if ioeventfd is active", 2016-10-30) and ad07cd6 ("virtio-scsi: always
use dataplane path if ioeventfd is active", 2016-10-30) made virtio-blk
and virtio-scsi always use the dataplane code under KVM.  The good news
therefore is that it was not a bug in the patches---they were doing
exactly what they were meant for, i.e. shake out remaining dataplane bugs.

The fix is not hard, so it's worth arranging for the broken drivers.
The virtio_should_notify+event_notifier_set pair that is common to
virtio-blk and virtio-scsi dataplane is replaced with a new public
function virtio_notify_irqfd that also sets ISR.  The irqfd emulation
code now need not set ISR anymore, so virtio_irq is removed.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 hw/block/dataplane/virtio-blk.c |  4 +---
 hw/scsi/virtio-scsi-dataplane.c |  7 -------
 hw/scsi/virtio-scsi.c           |  2 +-
 hw/virtio/trace-events          |  2 +-
 hw/virtio/virtio.c              | 20 ++++++++++++--------
 include/hw/virtio/virtio-scsi.h |  1 -
 include/hw/virtio/virtio.h      |  2 +-
 7 files changed, 16 insertions(+), 22 deletions(-)

diff --git a/hw/block/dataplane/virtio-blk.c b/hw/block/dataplane/virtio-blk.c
index 90ef557..d1f9f63 100644
--- a/hw/block/dataplane/virtio-blk.c
+++ b/hw/block/dataplane/virtio-blk.c
@@ -68,9 +68,7 @@ static void notify_guest_bh(void *opaque)
             unsigned i = j + ctzl(bits);
             VirtQueue *vq = virtio_get_queue(s->vdev, i);
 
-            if (virtio_should_notify(s->vdev, vq)) {
-                event_notifier_set(virtio_queue_get_guest_notifier(vq));
-            }
+            virtio_notify_irqfd(s->vdev, vq);
 
             bits &= bits - 1; /* clear right-most bit */
         }
diff --git a/hw/scsi/virtio-scsi-dataplane.c b/hw/scsi/virtio-scsi-dataplane.c
index f2ea29d..6b8d0f0 100644
--- a/hw/scsi/virtio-scsi-dataplane.c
+++ b/hw/scsi/virtio-scsi-dataplane.c
@@ -95,13 +95,6 @@ static int virtio_scsi_vring_init(VirtIOSCSI *s, VirtQueue *vq, int n,
     return 0;
 }
 
-void virtio_scsi_dataplane_notify(VirtIODevice *vdev, VirtIOSCSIReq *req)
-{
-    if (virtio_should_notify(vdev, req->vq)) {
-        event_notifier_set(virtio_queue_get_guest_notifier(req->vq));
-    }
-}
-
 /* assumes s->ctx held */
 static void virtio_scsi_clear_aio(VirtIOSCSI *s)
 {
diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
index 3e5ae6a..10fd687 100644
--- a/hw/scsi/virtio-scsi.c
+++ b/hw/scsi/virtio-scsi.c
@@ -69,7 +69,7 @@ static void virtio_scsi_complete_req(VirtIOSCSIReq *req)
     qemu_iovec_from_buf(&req->resp_iov, 0, &req->resp, req->resp_size);
     virtqueue_push(vq, &req->elem, req->qsgl.size + req->resp_iov.size);
     if (s->dataplane_started && !s->dataplane_fenced) {
-        virtio_scsi_dataplane_notify(vdev, req);
+        virtio_notify_irqfd(vdev, vq);
     } else {
         virtio_notify(vdev, vq);
     }
diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events
index 8756cef..7b6f55e 100644
--- a/hw/virtio/trace-events
+++ b/hw/virtio/trace-events
@@ -5,7 +5,7 @@ virtqueue_fill(void *vq, const void *elem, unsigned int len, unsigned int idx) "
 virtqueue_flush(void *vq, unsigned int count) "vq %p count %u"
 virtqueue_pop(void *vq, void *elem, unsigned int in_num, unsigned int out_num) "vq %p elem %p in_num %u out_num %u"
 virtio_queue_notify(void *vdev, int n, void *vq) "vdev %p n %d vq %p"
-virtio_irq(void *vq) "vq %p"
+virtio_notify_irqfd(void *vdev, void *vq) "vdev %p vq %p"
 virtio_notify(void *vdev, void *vq) "vdev %p vq %p"
 virtio_set_status(void *vdev, uint8_t val) "vdev %p val %u"
 
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index ecf13bd..860ebdb 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -1326,13 +1326,6 @@ static void virtio_set_isr(VirtIODevice *vdev, int value)
     }
 }
 
-void virtio_irq(VirtQueue *vq)
-{
-    trace_virtio_irq(vq);
-    virtio_set_isr(vq->vdev, 0x1);
-    virtio_notify_vector(vq->vdev, vq->vector);
-}
-
 bool virtio_should_notify(VirtIODevice *vdev, VirtQueue *vq)
 {
     uint16_t old, new;
@@ -1356,6 +1349,17 @@ bool virtio_should_notify(VirtIODevice *vdev, VirtQueue *vq)
     return !v || vring_need_event(vring_get_used_event(vq), new, old);
 }
 
+void virtio_notify_irqfd(VirtIODevice *vdev, VirtQueue *vq)
+{
+    if (!virtio_should_notify(vdev, vq)) {
+        return;
+    }
+
+    trace_virtio_notify_irqfd(vdev, vq);
+    virtio_set_isr(vq->vdev, 0x1);
+    event_notifier_set(&vq->guest_notifier);
+}
+
 void virtio_notify(VirtIODevice *vdev, VirtQueue *vq)
 {
     if (!virtio_should_notify(vdev, vq)) {
@@ -1990,7 +1994,7 @@ static void virtio_queue_guest_notifier_read(EventNotifier *n)
 {
     VirtQueue *vq = container_of(n, VirtQueue, guest_notifier);
     if (event_notifier_test_and_clear(n)) {
-        virtio_irq(vq);
+        virtio_notify_vector(vq->vdev, vq->vector);
     }
 }
 
diff --git a/include/hw/virtio/virtio-scsi.h b/include/hw/virtio/virtio-scsi.h
index 9fbc7d7..7375196 100644
--- a/include/hw/virtio/virtio-scsi.h
+++ b/include/hw/virtio/virtio-scsi.h
@@ -137,6 +137,5 @@ void virtio_scsi_push_event(VirtIOSCSI *s, SCSIDevice *dev,
 void virtio_scsi_dataplane_setup(VirtIOSCSI *s, Error **errp);
 int virtio_scsi_dataplane_start(VirtIODevice *s);
 void virtio_scsi_dataplane_stop(VirtIODevice *s);
-void virtio_scsi_dataplane_notify(VirtIODevice *vdev, VirtIOSCSIReq *req);
 
 #endif /* QEMU_VIRTIO_SCSI_H */
diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index 835b085..ab0e030 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -181,6 +181,7 @@ void virtqueue_get_avail_bytes(VirtQueue *vq, unsigned int *in_bytes,
                                unsigned max_in_bytes, unsigned max_out_bytes);
 
 bool virtio_should_notify(VirtIODevice *vdev, VirtQueue *vq);
+void virtio_notify_irqfd(VirtIODevice *vdev, VirtQueue *vq);
 void virtio_notify(VirtIODevice *vdev, VirtQueue *vq);
 
 void virtio_save(VirtIODevice *vdev, QEMUFile *f);
@@ -280,7 +281,6 @@ void virtio_queue_host_notifier_read(EventNotifier *n);
 void virtio_queue_aio_set_host_notifier_handler(VirtQueue *vq, AioContext *ctx,
                                                 void (*fn)(VirtIODevice *,
                                                            VirtQueue *));
-void virtio_irq(VirtQueue *vq);
 VirtQueue *virtio_vector_first_queue(VirtIODevice *vdev, uint16_t vector);
 VirtQueue *virtio_vector_next_queue(VirtQueue *vq);
 
-- 
2.9.3

Re: [Qemu-devel] [PATCH 3/3] virtio: set ISR on dataplane notifications

[Michael S. Tsirkin]

On Wed, Nov 16, 2016 at 07:05:51PM +0100, Paolo Bonzini wrote:
> Dataplane has been omitting forever the step of setting ISR when
> an interrupt is raised.  This caused little breakage, because the
> specification actually says that ISR may not be updated in MSI mode.
> 
> Some versions of the Windows drivers however didn't clear MSI mode
> correctly, and proceeded using polling mode (using ISR, not the used
> ring index!) for crashdump and hibernation.  If it were just crashdump
> and hibernation it would not be a big deal, but recent releases of
> Windows do not really shut down, but rather log out and hibernate to
> make the next startup faster.  Hence, this manifested as a more serious
> hang during shutdown with e.g. Windows 8.1 and virtio-win 1.8.0 RPMs.
> Newer versions fixed this, while older versions do not use MSI at all.
> 
> The failure has always been there for virtio dataplane, but it became
> visible after commits 9ffe337 ("virtio-blk: always use dataplane path
> if ioeventfd is active", 2016-10-30) and ad07cd6 ("virtio-scsi: always
> use dataplane path if ioeventfd is active", 2016-10-30) made virtio-blk
> and virtio-scsi always use the dataplane code under KVM.  The good news
> therefore is that it was not a bug in the patches---they were doing
> exactly what they were meant for, i.e. shake out remaining dataplane bugs.
> 
> The fix is not hard, so it's worth arranging for the broken drivers.
> The virtio_should_notify+event_notifier_set pair that is common to
> virtio-blk and virtio-scsi dataplane is replaced with a new public
> function virtio_notify_irqfd that also sets ISR.  The irqfd emulation
> code now need not set ISR anymore, so virtio_irq is removed.
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  hw/block/dataplane/virtio-blk.c |  4 +---
>  hw/scsi/virtio-scsi-dataplane.c |  7 -------
>  hw/scsi/virtio-scsi.c           |  2 +-
>  hw/virtio/trace-events          |  2 +-
>  hw/virtio/virtio.c              | 20 ++++++++++++--------
>  include/hw/virtio/virtio-scsi.h |  1 -
>  include/hw/virtio/virtio.h      |  2 +-
>  7 files changed, 16 insertions(+), 22 deletions(-)
> 
> diff --git a/hw/block/dataplane/virtio-blk.c b/hw/block/dataplane/virtio-blk.c
> index 90ef557..d1f9f63 100644
> --- a/hw/block/dataplane/virtio-blk.c
> +++ b/hw/block/dataplane/virtio-blk.c
> @@ -68,9 +68,7 @@ static void notify_guest_bh(void *opaque)
>              unsigned i = j + ctzl(bits);
>              VirtQueue *vq = virtio_get_queue(s->vdev, i);
>  
> -            if (virtio_should_notify(s->vdev, vq)) {
> -                event_notifier_set(virtio_queue_get_guest_notifier(vq));
> -            }
> +            virtio_notify_irqfd(s->vdev, vq);
>  
>              bits &= bits - 1; /* clear right-most bit */
>          }
> diff --git a/hw/scsi/virtio-scsi-dataplane.c b/hw/scsi/virtio-scsi-dataplane.c
> index f2ea29d..6b8d0f0 100644
> --- a/hw/scsi/virtio-scsi-dataplane.c
> +++ b/hw/scsi/virtio-scsi-dataplane.c
> @@ -95,13 +95,6 @@ static int virtio_scsi_vring_init(VirtIOSCSI *s, VirtQueue *vq, int n,
>      return 0;
>  }
>  
> -void virtio_scsi_dataplane_notify(VirtIODevice *vdev, VirtIOSCSIReq *req)
> -{
> -    if (virtio_should_notify(vdev, req->vq)) {
> -        event_notifier_set(virtio_queue_get_guest_notifier(req->vq));
> -    }
> -}
> -
>  /* assumes s->ctx held */
>  static void virtio_scsi_clear_aio(VirtIOSCSI *s)
>  {
> diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
> index 3e5ae6a..10fd687 100644
> --- a/hw/scsi/virtio-scsi.c
> +++ b/hw/scsi/virtio-scsi.c
> @@ -69,7 +69,7 @@ static void virtio_scsi_complete_req(VirtIOSCSIReq *req)
>      qemu_iovec_from_buf(&req->resp_iov, 0, &req->resp, req->resp_size);
>      virtqueue_push(vq, &req->elem, req->qsgl.size + req->resp_iov.size);
>      if (s->dataplane_started && !s->dataplane_fenced) {
> -        virtio_scsi_dataplane_notify(vdev, req);
> +        virtio_notify_irqfd(vdev, vq);
>      } else {
>          virtio_notify(vdev, vq);
>      }
> diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events
> index 8756cef..7b6f55e 100644
> --- a/hw/virtio/trace-events
> +++ b/hw/virtio/trace-events
> @@ -5,7 +5,7 @@ virtqueue_fill(void *vq, const void *elem, unsigned int len, unsigned int idx) "
>  virtqueue_flush(void *vq, unsigned int count) "vq %p count %u"
>  virtqueue_pop(void *vq, void *elem, unsigned int in_num, unsigned int out_num) "vq %p elem %p in_num %u out_num %u"
>  virtio_queue_notify(void *vdev, int n, void *vq) "vdev %p n %d vq %p"
> -virtio_irq(void *vq) "vq %p"
> +virtio_notify_irqfd(void *vdev, void *vq) "vdev %p vq %p"
>  virtio_notify(void *vdev, void *vq) "vdev %p vq %p"
>  virtio_set_status(void *vdev, uint8_t val) "vdev %p val %u"
>  
> diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
> index ecf13bd..860ebdb 100644
> --- a/hw/virtio/virtio.c
> +++ b/hw/virtio/virtio.c
> @@ -1326,13 +1326,6 @@ static void virtio_set_isr(VirtIODevice *vdev, int value)
>      }
>  }
>  
> -void virtio_irq(VirtQueue *vq)
> -{
> -    trace_virtio_irq(vq);
> -    virtio_set_isr(vq->vdev, 0x1);
> -    virtio_notify_vector(vq->vdev, vq->vector);
> -}
> -
>  bool virtio_should_notify(VirtIODevice *vdev, VirtQueue *vq)
>  {
>      uint16_t old, new;
> @@ -1356,6 +1349,17 @@ bool virtio_should_notify(VirtIODevice *vdev, VirtQueue *vq)
>      return !v || vring_need_event(vring_get_used_event(vq), new, old);
>  }
>  
> +void virtio_notify_irqfd(VirtIODevice *vdev, VirtQueue *vq)
> +{
> +    if (!virtio_should_notify(vdev, vq)) {
> +        return;
> +    }
> +
> +    trace_virtio_notify_irqfd(vdev, vq);
> +    virtio_set_isr(vq->vdev, 0x1);

So here, I think we need a comment with parts of
the commit log.

/*
 * virtio spec 1.0 says ISR bit 0 should be ignored with MSI, but
 * windows drivers included in virtio-win 1.8.0 (circa 2015)
 * for Windows 8.1 only are incorrectly polling this bit during shutdown
 * in MSI mode, causing a hang if this bit is never updated.
 * Next driver release from 2016 fixed this problem, so working around it
 * is not a must, but it's easy to do so let's do it here.
 *
 * Note: it's safe to update ISR from any thread as it was switched
 * to an atomic operation.
 */




> +    event_notifier_set(&vq->guest_notifier);
> +}
> +
>  void virtio_notify(VirtIODevice *vdev, VirtQueue *vq)
>  {
>      if (!virtio_should_notify(vdev, vq)) {
> @@ -1990,7 +1994,7 @@ static void virtio_queue_guest_notifier_read(EventNotifier *n)
>  {
>      VirtQueue *vq = container_of(n, VirtQueue, guest_notifier);
>      if (event_notifier_test_and_clear(n)) {
> -        virtio_irq(vq);
> +        virtio_notify_vector(vq->vdev, vq->vector);
>      }
>  }
>  
> diff --git a/include/hw/virtio/virtio-scsi.h b/include/hw/virtio/virtio-scsi.h
> index 9fbc7d7..7375196 100644
> --- a/include/hw/virtio/virtio-scsi.h
> +++ b/include/hw/virtio/virtio-scsi.h
> @@ -137,6 +137,5 @@ void virtio_scsi_push_event(VirtIOSCSI *s, SCSIDevice *dev,
>  void virtio_scsi_dataplane_setup(VirtIOSCSI *s, Error **errp);
>  int virtio_scsi_dataplane_start(VirtIODevice *s);
>  void virtio_scsi_dataplane_stop(VirtIODevice *s);
> -void virtio_scsi_dataplane_notify(VirtIODevice *vdev, VirtIOSCSIReq *req);
>  
>  #endif /* QEMU_VIRTIO_SCSI_H */
> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> index 835b085..ab0e030 100644
> --- a/include/hw/virtio/virtio.h
> +++ b/include/hw/virtio/virtio.h
> @@ -181,6 +181,7 @@ void virtqueue_get_avail_bytes(VirtQueue *vq, unsigned int *in_bytes,
>                                 unsigned max_in_bytes, unsigned max_out_bytes);
>  
>  bool virtio_should_notify(VirtIODevice *vdev, VirtQueue *vq);
> +void virtio_notify_irqfd(VirtIODevice *vdev, VirtQueue *vq);
>  void virtio_notify(VirtIODevice *vdev, VirtQueue *vq);
>  
>  void virtio_save(VirtIODevice *vdev, QEMUFile *f);
> @@ -280,7 +281,6 @@ void virtio_queue_host_notifier_read(EventNotifier *n);
>  void virtio_queue_aio_set_host_notifier_handler(VirtQueue *vq, AioContext *ctx,
>                                                  void (*fn)(VirtIODevice *,
>                                                             VirtQueue *));
> -void virtio_irq(VirtQueue *vq);
>  VirtQueue *virtio_vector_first_queue(VirtIODevice *vdev, uint16_t vector);
>  VirtQueue *virtio_vector_next_queue(VirtQueue *vq);
>  
> -- 
> 2.9.3

Re: [Qemu-devel] [PATCH 3/3] virtio: set ISR on dataplane notifications

[Paolo Bonzini]

> > +void virtio_notify_irqfd(VirtIODevice *vdev, VirtQueue *vq)
> > +{
> > +    if (!virtio_should_notify(vdev, vq)) {
> > +        return;
> > +    }
> > +
> > +    trace_virtio_notify_irqfd(vdev, vq);
> > +    virtio_set_isr(vq->vdev, 0x1);
> 
> So here, I think we need a comment with parts of
> the commit log.
> 
> /*
>  * virtio spec 1.0 says ISR bit 0 should be ignored with MSI, but
>  * windows drivers included in virtio-win 1.8.0 (circa 2015)
>  * for Windows 8.1 only are incorrectly polling this bit during shutdown
         ^^^^^^^^^^^^^^^^

Not sure it's only for Windows 8.1, in fact probably not.
Looks good if you replace this line with

"are incorrectly polling this bit during crashdump or hibernation"

Paolo

>  * in MSI mode, causing a hang if this bit is never updated.
>  * Next driver release from 2016 fixed this problem, so working around it
>  * is not a must, but it's easy to do so let's do it here.
>  *
>  * Note: it's safe to update ISR from any thread as it was switched
>  * to an atomic operation.
>  */


> 
> 
> 
> > +    event_notifier_set(&vq->guest_notifier);
> > +}
> > +
> >  void virtio_notify(VirtIODevice *vdev, VirtQueue *vq)
> >  {
> >      if (!virtio_should_notify(vdev, vq)) {
> > @@ -1990,7 +1994,7 @@ static void
> > virtio_queue_guest_notifier_read(EventNotifier *n)
> >  {
> >      VirtQueue *vq = container_of(n, VirtQueue, guest_notifier);
> >      if (event_notifier_test_and_clear(n)) {
> > -        virtio_irq(vq);
> > +        virtio_notify_vector(vq->vdev, vq->vector);
> >      }
> >  }
> >  
> > diff --git a/include/hw/virtio/virtio-scsi.h
> > b/include/hw/virtio/virtio-scsi.h
> > index 9fbc7d7..7375196 100644
> > --- a/include/hw/virtio/virtio-scsi.h
> > +++ b/include/hw/virtio/virtio-scsi.h
> > @@ -137,6 +137,5 @@ void virtio_scsi_push_event(VirtIOSCSI *s, SCSIDevice
> > *dev,
> >  void virtio_scsi_dataplane_setup(VirtIOSCSI *s, Error **errp);
> >  int virtio_scsi_dataplane_start(VirtIODevice *s);
> >  void virtio_scsi_dataplane_stop(VirtIODevice *s);
> > -void virtio_scsi_dataplane_notify(VirtIODevice *vdev, VirtIOSCSIReq *req);
> >  
> >  #endif /* QEMU_VIRTIO_SCSI_H */
> > diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> > index 835b085..ab0e030 100644
> > --- a/include/hw/virtio/virtio.h
> > +++ b/include/hw/virtio/virtio.h
> > @@ -181,6 +181,7 @@ void virtqueue_get_avail_bytes(VirtQueue *vq, unsigned
> > int *in_bytes,
> >                                 unsigned max_in_bytes, unsigned
> >                                 max_out_bytes);
> >  
> >  bool virtio_should_notify(VirtIODevice *vdev, VirtQueue *vq);
> > +void virtio_notify_irqfd(VirtIODevice *vdev, VirtQueue *vq);
> >  void virtio_notify(VirtIODevice *vdev, VirtQueue *vq);
> >  
> >  void virtio_save(VirtIODevice *vdev, QEMUFile *f);
> > @@ -280,7 +281,6 @@ void virtio_queue_host_notifier_read(EventNotifier *n);
> >  void virtio_queue_aio_set_host_notifier_handler(VirtQueue *vq, AioContext
> >  *ctx,
> >                                                  void (*fn)(VirtIODevice *,
> >                                                             VirtQueue *));
> > -void virtio_irq(VirtQueue *vq);
> >  VirtQueue *virtio_vector_first_queue(VirtIODevice *vdev, uint16_t vector);
> >  VirtQueue *virtio_vector_next_queue(VirtQueue *vq);
> >  
> > --
> > 2.9.3
> 

Re: [Qemu-devel] [PATCH 3/3] virtio: set ISR on dataplane notifications

[Michael S. Tsirkin]

On Wed, Nov 16, 2016 at 03:38:11PM -0500, Paolo Bonzini wrote:
> > > +void virtio_notify_irqfd(VirtIODevice *vdev, VirtQueue *vq)
> > > +{
> > > +    if (!virtio_should_notify(vdev, vq)) {
> > > +        return;
> > > +    }
> > > +
> > > +    trace_virtio_notify_irqfd(vdev, vq);
> > > +    virtio_set_isr(vq->vdev, 0x1);
> > 
> > So here, I think we need a comment with parts of
> > the commit log.
> > 
> > /*
> >  * virtio spec 1.0 says ISR bit 0 should be ignored with MSI, but
> >  * windows drivers included in virtio-win 1.8.0 (circa 2015)
> >  * for Windows 8.1 only are incorrectly polling this bit during shutdown
>          ^^^^^^^^^^^^^^^^
> 
> Not sure it's only for Windows 8.1, in fact probably not.

8.1 on shutdown and others on crashdump or hibernation?

> Looks good if you replace this line with
> 
> "are incorrectly polling this bit during crashdump or hibernation"
> 
> Paolo
> 
> >  * in MSI mode, causing a hang if this bit is never updated.
> >  * Next driver release from 2016 fixed this problem, so working around it
> >  * is not a must, but it's easy to do so let's do it here.
> >  *
> >  * Note: it's safe to update ISR from any thread as it was switched
> >  * to an atomic operation.
> >  */
> 
> 
> > 
> > 
> > 
> > > +    event_notifier_set(&vq->guest_notifier);
> > > +}
> > > +
> > >  void virtio_notify(VirtIODevice *vdev, VirtQueue *vq)
> > >  {
> > >      if (!virtio_should_notify(vdev, vq)) {
> > > @@ -1990,7 +1994,7 @@ static void
> > > virtio_queue_guest_notifier_read(EventNotifier *n)
> > >  {
> > >      VirtQueue *vq = container_of(n, VirtQueue, guest_notifier);
> > >      if (event_notifier_test_and_clear(n)) {
> > > -        virtio_irq(vq);
> > > +        virtio_notify_vector(vq->vdev, vq->vector);
> > >      }
> > >  }
> > >  
> > > diff --git a/include/hw/virtio/virtio-scsi.h
> > > b/include/hw/virtio/virtio-scsi.h
> > > index 9fbc7d7..7375196 100644
> > > --- a/include/hw/virtio/virtio-scsi.h
> > > +++ b/include/hw/virtio/virtio-scsi.h
> > > @@ -137,6 +137,5 @@ void virtio_scsi_push_event(VirtIOSCSI *s, SCSIDevice
> > > *dev,
> > >  void virtio_scsi_dataplane_setup(VirtIOSCSI *s, Error **errp);
> > >  int virtio_scsi_dataplane_start(VirtIODevice *s);
> > >  void virtio_scsi_dataplane_stop(VirtIODevice *s);
> > > -void virtio_scsi_dataplane_notify(VirtIODevice *vdev, VirtIOSCSIReq *req);
> > >  
> > >  #endif /* QEMU_VIRTIO_SCSI_H */
> > > diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> > > index 835b085..ab0e030 100644
> > > --- a/include/hw/virtio/virtio.h
> > > +++ b/include/hw/virtio/virtio.h
> > > @@ -181,6 +181,7 @@ void virtqueue_get_avail_bytes(VirtQueue *vq, unsigned
> > > int *in_bytes,
> > >                                 unsigned max_in_bytes, unsigned
> > >                                 max_out_bytes);
> > >  
> > >  bool virtio_should_notify(VirtIODevice *vdev, VirtQueue *vq);
> > > +void virtio_notify_irqfd(VirtIODevice *vdev, VirtQueue *vq);
> > >  void virtio_notify(VirtIODevice *vdev, VirtQueue *vq);
> > >  
> > >  void virtio_save(VirtIODevice *vdev, QEMUFile *f);
> > > @@ -280,7 +281,6 @@ void virtio_queue_host_notifier_read(EventNotifier *n);
> > >  void virtio_queue_aio_set_host_notifier_handler(VirtQueue *vq, AioContext
> > >  *ctx,
> > >                                                  void (*fn)(VirtIODevice *,
> > >                                                             VirtQueue *));
> > > -void virtio_irq(VirtQueue *vq);
> > >  VirtQueue *virtio_vector_first_queue(VirtIODevice *vdev, uint16_t vector);
> > >  VirtQueue *virtio_vector_next_queue(VirtQueue *vq);
> > >  
> > > --
> > > 2.9.3
> > 

Re: [Qemu-devel] [PATCH 3/3] virtio: set ISR on dataplane notifications

[Paolo Bonzini]

----- Original Message -----
> From: "Michael S. Tsirkin" <mst@redhat.com>
> To: "Paolo Bonzini" <pbonzini@redhat.com>
> Cc: qemu-devel@nongnu.org, "alex williamson" <alex.williamson@redhat.com>, borntraeger@de.ibm.com, felipe@nutanix.com
> Sent: Wednesday, November 16, 2016 9:39:24 PM
> Subject: Re: [PATCH 3/3] virtio: set ISR on dataplane notifications
> 
> On Wed, Nov 16, 2016 at 03:38:11PM -0500, Paolo Bonzini wrote:
> > > > +void virtio_notify_irqfd(VirtIODevice *vdev, VirtQueue *vq)
> > > > +{
> > > > +    if (!virtio_should_notify(vdev, vq)) {
> > > > +        return;
> > > > +    }
> > > > +
> > > > +    trace_virtio_notify_irqfd(vdev, vq);
> > > > +    virtio_set_isr(vq->vdev, 0x1);
> > > 
> > > So here, I think we need a comment with parts of
> > > the commit log.
> > > 
> > > /*
> > >  * virtio spec 1.0 says ISR bit 0 should be ignored with MSI, but
> > >  * windows drivers included in virtio-win 1.8.0 (circa 2015)
> > >  * for Windows 8.1 only are incorrectly polling this bit during shutdown
> >          ^^^^^^^^^^^^^^^^
> > 
> > Not sure it's only for Windows 8.1, in fact probably not.
> 
> 8.1 on shutdown and others on crashdump or hibernation?

Even 8.1 is really a hibernation hidden behind a "Shut down" menu item.

Paolo

> > Looks good if you replace this line with
> > 
> > "are incorrectly polling this bit during crashdump or hibernation"
> > 
> > Paolo
> > 
> > >  * in MSI mode, causing a hang if this bit is never updated.
> > >  * Next driver release from 2016 fixed this problem, so working around it
> > >  * is not a must, but it's easy to do so let's do it here.
> > >  *
> > >  * Note: it's safe to update ISR from any thread as it was switched
> > >  * to an atomic operation.
> > >  */
> > 
> > 
> > > 
> > > 
> > > 
> > > > +    event_notifier_set(&vq->guest_notifier);
> > > > +}
> > > > +
> > > >  void virtio_notify(VirtIODevice *vdev, VirtQueue *vq)
> > > >  {
> > > >      if (!virtio_should_notify(vdev, vq)) {
> > > > @@ -1990,7 +1994,7 @@ static void
> > > > virtio_queue_guest_notifier_read(EventNotifier *n)
> > > >  {
> > > >      VirtQueue *vq = container_of(n, VirtQueue, guest_notifier);
> > > >      if (event_notifier_test_and_clear(n)) {
> > > > -        virtio_irq(vq);
> > > > +        virtio_notify_vector(vq->vdev, vq->vector);
> > > >      }
> > > >  }
> > > >  
> > > > diff --git a/include/hw/virtio/virtio-scsi.h
> > > > b/include/hw/virtio/virtio-scsi.h
> > > > index 9fbc7d7..7375196 100644
> > > > --- a/include/hw/virtio/virtio-scsi.h
> > > > +++ b/include/hw/virtio/virtio-scsi.h
> > > > @@ -137,6 +137,5 @@ void virtio_scsi_push_event(VirtIOSCSI *s,
> > > > SCSIDevice
> > > > *dev,
> > > >  void virtio_scsi_dataplane_setup(VirtIOSCSI *s, Error **errp);
> > > >  int virtio_scsi_dataplane_start(VirtIODevice *s);
> > > >  void virtio_scsi_dataplane_stop(VirtIODevice *s);
> > > > -void virtio_scsi_dataplane_notify(VirtIODevice *vdev, VirtIOSCSIReq
> > > > *req);
> > > >  
> > > >  #endif /* QEMU_VIRTIO_SCSI_H */
> > > > diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> > > > index 835b085..ab0e030 100644
> > > > --- a/include/hw/virtio/virtio.h
> > > > +++ b/include/hw/virtio/virtio.h
> > > > @@ -181,6 +181,7 @@ void virtqueue_get_avail_bytes(VirtQueue *vq,
> > > > unsigned
> > > > int *in_bytes,
> > > >                                 unsigned max_in_bytes, unsigned
> > > >                                 max_out_bytes);
> > > >  
> > > >  bool virtio_should_notify(VirtIODevice *vdev, VirtQueue *vq);
> > > > +void virtio_notify_irqfd(VirtIODevice *vdev, VirtQueue *vq);
> > > >  void virtio_notify(VirtIODevice *vdev, VirtQueue *vq);
> > > >  
> > > >  void virtio_save(VirtIODevice *vdev, QEMUFile *f);
> > > > @@ -280,7 +281,6 @@ void virtio_queue_host_notifier_read(EventNotifier
> > > > *n);
> > > >  void virtio_queue_aio_set_host_notifier_handler(VirtQueue *vq,
> > > >  AioContext
> > > >  *ctx,
> > > >                                                  void
> > > >                                                  (*fn)(VirtIODevice *,
> > > >                                                             VirtQueue
> > > >                                                             *));
> > > > -void virtio_irq(VirtQueue *vq);
> > > >  VirtQueue *virtio_vector_first_queue(VirtIODevice *vdev, uint16_t
> > > >  vector);
> > > >  VirtQueue *virtio_vector_next_queue(VirtQueue *vq);
> > > >  
> > > > --
> > > > 2.9.3
> > > 
> 

Re: [Qemu-devel] [PATCH 3/3] virtio: set ISR on dataplane notifications

[Michael S. Tsirkin]

On Wed, Nov 16, 2016 at 04:05:31PM -0500, Paolo Bonzini wrote:
> 
> 
> ----- Original Message -----
> > From: "Michael S. Tsirkin" <mst@redhat.com>
> > To: "Paolo Bonzini" <pbonzini@redhat.com>
> > Cc: qemu-devel@nongnu.org, "alex williamson" <alex.williamson@redhat.com>, borntraeger@de.ibm.com, felipe@nutanix.com
> > Sent: Wednesday, November 16, 2016 9:39:24 PM
> > Subject: Re: [PATCH 3/3] virtio: set ISR on dataplane notifications
> > 
> > On Wed, Nov 16, 2016 at 03:38:11PM -0500, Paolo Bonzini wrote:
> > > > > +void virtio_notify_irqfd(VirtIODevice *vdev, VirtQueue *vq)
> > > > > +{
> > > > > +    if (!virtio_should_notify(vdev, vq)) {
> > > > > +        return;
> > > > > +    }
> > > > > +
> > > > > +    trace_virtio_notify_irqfd(vdev, vq);
> > > > > +    virtio_set_isr(vq->vdev, 0x1);
> > > > 
> > > > So here, I think we need a comment with parts of
> > > > the commit log.
> > > > 
> > > > /*
> > > >  * virtio spec 1.0 says ISR bit 0 should be ignored with MSI, but
> > > >  * windows drivers included in virtio-win 1.8.0 (circa 2015)
> > > >  * for Windows 8.1 only are incorrectly polling this bit during shutdown
> > >          ^^^^^^^^^^^^^^^^
> > > 
> > > Not sure it's only for Windows 8.1, in fact probably not.
> > 
> > 8.1 on shutdown and others on crashdump or hibernation?
> 
> Even 8.1 is really a hibernation hidden behind a "Shut down" menu item.
> 
> Paolo

what does "hang during shutdown" in your commit log refer to then?

> > > Looks good if you replace this line with
> > > 
> > > "are incorrectly polling this bit during crashdump or hibernation"
> > > 
> > > Paolo
> > > 
> > > >  * in MSI mode, causing a hang if this bit is never updated.
> > > >  * Next driver release from 2016 fixed this problem, so working around it
> > > >  * is not a must, but it's easy to do so let's do it here.
> > > >  *
> > > >  * Note: it's safe to update ISR from any thread as it was switched
> > > >  * to an atomic operation.
> > > >  */
> > > 
> > > 
> > > > 
> > > > 
> > > > 
> > > > > +    event_notifier_set(&vq->guest_notifier);
> > > > > +}
> > > > > +
> > > > >  void virtio_notify(VirtIODevice *vdev, VirtQueue *vq)
> > > > >  {
> > > > >      if (!virtio_should_notify(vdev, vq)) {
> > > > > @@ -1990,7 +1994,7 @@ static void
> > > > > virtio_queue_guest_notifier_read(EventNotifier *n)
> > > > >  {
> > > > >      VirtQueue *vq = container_of(n, VirtQueue, guest_notifier);
> > > > >      if (event_notifier_test_and_clear(n)) {
> > > > > -        virtio_irq(vq);
> > > > > +        virtio_notify_vector(vq->vdev, vq->vector);
> > > > >      }
> > > > >  }
> > > > >  
> > > > > diff --git a/include/hw/virtio/virtio-scsi.h
> > > > > b/include/hw/virtio/virtio-scsi.h
> > > > > index 9fbc7d7..7375196 100644
> > > > > --- a/include/hw/virtio/virtio-scsi.h
> > > > > +++ b/include/hw/virtio/virtio-scsi.h
> > > > > @@ -137,6 +137,5 @@ void virtio_scsi_push_event(VirtIOSCSI *s,
> > > > > SCSIDevice
> > > > > *dev,
> > > > >  void virtio_scsi_dataplane_setup(VirtIOSCSI *s, Error **errp);
> > > > >  int virtio_scsi_dataplane_start(VirtIODevice *s);
> > > > >  void virtio_scsi_dataplane_stop(VirtIODevice *s);
> > > > > -void virtio_scsi_dataplane_notify(VirtIODevice *vdev, VirtIOSCSIReq
> > > > > *req);
> > > > >  
> > > > >  #endif /* QEMU_VIRTIO_SCSI_H */
> > > > > diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> > > > > index 835b085..ab0e030 100644
> > > > > --- a/include/hw/virtio/virtio.h
> > > > > +++ b/include/hw/virtio/virtio.h
> > > > > @@ -181,6 +181,7 @@ void virtqueue_get_avail_bytes(VirtQueue *vq,
> > > > > unsigned
> > > > > int *in_bytes,
> > > > >                                 unsigned max_in_bytes, unsigned
> > > > >                                 max_out_bytes);
> > > > >  
> > > > >  bool virtio_should_notify(VirtIODevice *vdev, VirtQueue *vq);
> > > > > +void virtio_notify_irqfd(VirtIODevice *vdev, VirtQueue *vq);
> > > > >  void virtio_notify(VirtIODevice *vdev, VirtQueue *vq);
> > > > >  
> > > > >  void virtio_save(VirtIODevice *vdev, QEMUFile *f);
> > > > > @@ -280,7 +281,6 @@ void virtio_queue_host_notifier_read(EventNotifier
> > > > > *n);
> > > > >  void virtio_queue_aio_set_host_notifier_handler(VirtQueue *vq,
> > > > >  AioContext
> > > > >  *ctx,
> > > > >                                                  void
> > > > >                                                  (*fn)(VirtIODevice *,
> > > > >                                                             VirtQueue
> > > > >                                                             *));
> > > > > -void virtio_irq(VirtQueue *vq);
> > > > >  VirtQueue *virtio_vector_first_queue(VirtIODevice *vdev, uint16_t
> > > > >  vector);
> > > > >  VirtQueue *virtio_vector_next_queue(VirtQueue *vq);
> > > > >  
> > > > > --
> > > > > 2.9.3
> > > > 
> > 

Re: [Qemu-devel] [PATCH 3/3] virtio: set ISR on dataplane notifications

[Paolo Bonzini]

> > > > > /*
> > > > >  * virtio spec 1.0 says ISR bit 0 should be ignored with MSI, but
> > > > >  * windows drivers included in virtio-win 1.8.0 (circa 2015)
> > > > >  * for Windows 8.1 only are incorrectly polling this bit during
> > > > >  shutdown
> > > >          ^^^^^^^^^^^^^^^^
> > > > 
> > > > Not sure it's only for Windows 8.1, in fact probably not.
> > > 
> > > 8.1 on shutdown and others on crashdump or hibernation?
> > 
> > Even 8.1 is really a hibernation hidden behind a "Shut down" menu item.
> > 
> > Paolo
> 
> what does "hang during shutdown" in your commit log refer to then?

The full text from the commit log is:

    recent releases of
    Windows do not really shut down, but rather log out and hibernate to
    make the next startup faster.  Hence, this manifested as a more serious
    hang during shutdown with e.g. Windows 8.1 and virtio-win 1.8.0 RPMs.

Shutdown in the commit log just means "clicking Shut down".  The previous
sentence explains why.  Also note the "e.g.", I've not tested other versions
of Windows.

Paolo

> > > > Looks good if you replace this line with
> > > > 
> > > > "are incorrectly polling this bit during crashdump or hibernation"
> > > > 
> > > > Paolo
> > > > 
> > > > >  * in MSI mode, causing a hang if this bit is never updated.
> > > > >  * Next driver release from 2016 fixed this problem, so working
> > > > >  around it
> > > > >  * is not a must, but it's easy to do so let's do it here.
> > > > >  *
> > > > >  * Note: it's safe to update ISR from any thread as it was switched
> > > > >  * to an atomic operation.
> > > > >  */
> > > > 
> > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > > +    event_notifier_set(&vq->guest_notifier);
> > > > > > +}
> > > > > > +
> > > > > >  void virtio_notify(VirtIODevice *vdev, VirtQueue *vq)
> > > > > >  {
> > > > > >      if (!virtio_should_notify(vdev, vq)) {
> > > > > > @@ -1990,7 +1994,7 @@ static void
> > > > > > virtio_queue_guest_notifier_read(EventNotifier *n)
> > > > > >  {
> > > > > >      VirtQueue *vq = container_of(n, VirtQueue, guest_notifier);
> > > > > >      if (event_notifier_test_and_clear(n)) {
> > > > > > -        virtio_irq(vq);
> > > > > > +        virtio_notify_vector(vq->vdev, vq->vector);
> > > > > >      }
> > > > > >  }
> > > > > >  
> > > > > > diff --git a/include/hw/virtio/virtio-scsi.h
> > > > > > b/include/hw/virtio/virtio-scsi.h
> > > > > > index 9fbc7d7..7375196 100644
> > > > > > --- a/include/hw/virtio/virtio-scsi.h
> > > > > > +++ b/include/hw/virtio/virtio-scsi.h
> > > > > > @@ -137,6 +137,5 @@ void virtio_scsi_push_event(VirtIOSCSI *s,
> > > > > > SCSIDevice
> > > > > > *dev,
> > > > > >  void virtio_scsi_dataplane_setup(VirtIOSCSI *s, Error **errp);
> > > > > >  int virtio_scsi_dataplane_start(VirtIODevice *s);
> > > > > >  void virtio_scsi_dataplane_stop(VirtIODevice *s);
> > > > > > -void virtio_scsi_dataplane_notify(VirtIODevice *vdev,
> > > > > > VirtIOSCSIReq
> > > > > > *req);
> > > > > >  
> > > > > >  #endif /* QEMU_VIRTIO_SCSI_H */
> > > > > > diff --git a/include/hw/virtio/virtio.h
> > > > > > b/include/hw/virtio/virtio.h
> > > > > > index 835b085..ab0e030 100644
> > > > > > --- a/include/hw/virtio/virtio.h
> > > > > > +++ b/include/hw/virtio/virtio.h
> > > > > > @@ -181,6 +181,7 @@ void virtqueue_get_avail_bytes(VirtQueue *vq,
> > > > > > unsigned
> > > > > > int *in_bytes,
> > > > > >                                 unsigned max_in_bytes, unsigned
> > > > > >                                 max_out_bytes);
> > > > > >  
> > > > > >  bool virtio_should_notify(VirtIODevice *vdev, VirtQueue *vq);
> > > > > > +void virtio_notify_irqfd(VirtIODevice *vdev, VirtQueue *vq);
> > > > > >  void virtio_notify(VirtIODevice *vdev, VirtQueue *vq);
> > > > > >  
> > > > > >  void virtio_save(VirtIODevice *vdev, QEMUFile *f);
> > > > > > @@ -280,7 +281,6 @@ void
> > > > > > virtio_queue_host_notifier_read(EventNotifier
> > > > > > *n);
> > > > > >  void virtio_queue_aio_set_host_notifier_handler(VirtQueue *vq,
> > > > > >  AioContext
> > > > > >  *ctx,
> > > > > >                                                  void
> > > > > >                                                  (*fn)(VirtIODevice
> > > > > >                                                  *,
> > > > > >                                                             VirtQueue
> > > > > >                                                             *));
> > > > > > -void virtio_irq(VirtQueue *vq);
> > > > > >  VirtQueue *virtio_vector_first_queue(VirtIODevice *vdev, uint16_t
> > > > > >  vector);
> > > > > >  VirtQueue *virtio_vector_next_queue(VirtQueue *vq);
> > > > > >  
> > > > > > --
> > > > > > 2.9.3
> > > > > 
> > > 
> 

Re: [Qemu-devel] [PATCH 3/3] virtio: set ISR on dataplane notifications

[Michael S. Tsirkin]

On Thu, Nov 17, 2016 at 04:04:26AM -0500, Paolo Bonzini wrote:
> > > > > > /*
> > > > > >  * virtio spec 1.0 says ISR bit 0 should be ignored with MSI, but
> > > > > >  * windows drivers included in virtio-win 1.8.0 (circa 2015)
> > > > > >  * for Windows 8.1 only are incorrectly polling this bit during
> > > > > >  shutdown
> > > > >          ^^^^^^^^^^^^^^^^
> > > > > 
> > > > > Not sure it's only for Windows 8.1, in fact probably not.
> > > > 
> > > > 8.1 on shutdown and others on crashdump or hibernation?
> > > 
> > > Even 8.1 is really a hibernation hidden behind a "Shut down" menu item.
> > > 
> > > Paolo
> > 
> > what does "hang during shutdown" in your commit log refer to then?
> 
> The full text from the commit log is:
> 
>     recent releases of
>     Windows do not really shut down, but rather log out and hibernate to
>     make the next startup faster.  Hence, this manifested as a more serious
>     hang during shutdown with e.g. Windows 8.1 and virtio-win 1.8.0 RPMs.
> 
> Shutdown in the commit log just means "clicking Shut down".  The previous
> sentence explains why.  Also note the "e.g.", I've not tested other versions
> of Windows.
> 
> Paolo

AFAIK the specific version of drivers did not support windows 10,
and IIRC windows 7 does shutdown normally.
So that only leaves windows 8 and 8.1.


> > > > > Looks good if you replace this line with
> > > > > 
> > > > > "are incorrectly polling this bit during crashdump or hibernation"

... which on windows 8 and 8.1 is activated by default through the
shutdown menu

> > > > > Paolo
> > > > > 
> > > > > >  * in MSI mode, causing a hang if this bit is never updated.
> > > > > >  * Next driver release from 2016 fixed this problem, so working
> > > > > >  around it
> > > > > >  * is not a must, but it's easy to do so let's do it here.
> > > > > >  *
> > > > > >  * Note: it's safe to update ISR from any thread as it was switched
> > > > > >  * to an atomic operation.
> > > > > >  */
> > > > > 

Re: [Qemu-devel] [PATCH 3/3] virtio: set ISR on dataplane notifications

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1440 bytes --]

On Wed, Nov 16, 2016 at 10:15:25PM +0200, Michael S. Tsirkin wrote:
> On Wed, Nov 16, 2016 at 07:05:51PM +0100, Paolo Bonzini wrote:
> > @@ -1356,6 +1349,17 @@ bool virtio_should_notify(VirtIODevice *vdev, VirtQueue *vq)
> >      return !v || vring_need_event(vring_get_used_event(vq), new, old);
> >  }
> >  
> > +void virtio_notify_irqfd(VirtIODevice *vdev, VirtQueue *vq)
> > +{
> > +    if (!virtio_should_notify(vdev, vq)) {
> > +        return;
> > +    }
> > +
> > +    trace_virtio_notify_irqfd(vdev, vq);
> > +    virtio_set_isr(vq->vdev, 0x1);
> 
> So here, I think we need a comment with parts of
> the commit log.
> 
> /*
>  * virtio spec 1.0 says ISR bit 0 should be ignored with MSI, but
>  * windows drivers included in virtio-win 1.8.0 (circa 2015)
>  * for Windows 8.1 only are incorrectly polling this bit during shutdown
>  * in MSI mode, causing a hang if this bit is never updated.
>  * Next driver release from 2016 fixed this problem, so working around it
>  * is not a must, but it's easy to do so let's do it here.
>  *
>  * Note: it's safe to update ISR from any thread as it was switched
>  * to an atomic operation.
>  */

I agree.  The commit description is nice but the information needs to be
in the code.

I remember explicitly omitting the ISR update when writing the dataplane
code because I saw the spec does not require it for MSI.

A comment is necessary.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [Qemu-devel] [PATCH v2 for-2.8 0/3] virtio fixes

[no-reply]

Hi,

Your series failed automatic build test. Please find the testing commands and
their output below. If you have docker installed, you can probably reproduce it
locally.

Type: series
Subject: [Qemu-devel] [PATCH v2 for-2.8 0/3] virtio fixes
Message-id: 20161116180551.9611-1-pbonzini@redhat.com

=== TEST SCRIPT BEGIN ===
#!/bin/bash
set -e
git submodule update --init dtc
# Let docker tests dump environment info
export SHOW_ENV=1
export J=16
make docker-test-quick@centos6
make docker-test-mingw@fedora
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
4476079 virtio: set ISR on dataplane notifications
f45efd4 virtio: access ISR atomically
9fd4e4a virtio: introduce grab/release_ioeventfd to fix vhost

=== OUTPUT BEGIN ===
Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc'
Cloning into 'dtc'...
Submodule path 'dtc': checked out '65cc4d2748a2c2e6f27f1cf39e07a5dbabd80ebf'
  BUILD   centos6
make[1]: Entering directory `/var/tmp/patchew-tester-tmp-5tzxa5rp/src'
  ARCHIVE qemu.tgz
  ARCHIVE dtc.tgz
  COPY    RUNNER
    RUN test-quick in qemu:centos6 
Packages installed:
SDL-devel-1.2.14-7.el6_7.1.x86_64
ccache-3.1.6-2.el6.x86_64
epel-release-6-8.noarch
gcc-4.4.7-17.el6.x86_64
git-1.7.1-4.el6_7.1.x86_64
glib2-devel-2.28.8-5.el6.x86_64
libfdt-devel-1.4.0-1.el6.x86_64
make-3.81-23.el6.x86_64
package g++ is not installed
pixman-devel-0.32.8-1.el6.x86_64
tar-1.23-15.el6_8.x86_64
zlib-devel-1.2.3-29.el6.x86_64

Environment variables:
PACKAGES=libfdt-devel ccache     tar git make gcc g++     zlib-devel glib2-devel SDL-devel pixman-devel     epel-release
HOSTNAME=4fce3ac805f5
TERM=xterm
MAKEFLAGS= -j16
HISTSIZE=1000
J=16
USER=root
CCACHE_DIR=/var/tmp/ccache
EXTRA_CONFIGURE_OPTS=
V=
SHOW_ENV=1
MAIL=/var/spool/mail/root
PATH=/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
LANG=en_US.UTF-8
TARGET_LIST=
HISTCONTROL=ignoredups
SHLVL=1
HOME=/root
TEST_DIR=/tmp/qemu-test
LOGNAME=root
LESSOPEN=||/usr/bin/lesspipe.sh %s
FEATURES= dtc
DEBUG=
G_BROKEN_FILENAMES=1
CCACHE_HASHDIR=
_=/usr/bin/env

Configure options:
--enable-werror --target-list=x86_64-softmmu,aarch64-softmmu --prefix=/var/tmp/qemu-build/install
No C++ compiler available; disabling C++ specific optional code
Install prefix    /var/tmp/qemu-build/install
BIOS directory    /var/tmp/qemu-build/install/share/qemu
binary directory  /var/tmp/qemu-build/install/bin
library directory /var/tmp/qemu-build/install/lib
module directory  /var/tmp/qemu-build/install/lib/qemu
libexec directory /var/tmp/qemu-build/install/libexec
include directory /var/tmp/qemu-build/install/include
config directory  /var/tmp/qemu-build/install/etc
local state directory   /var/tmp/qemu-build/install/var
Manual directory  /var/tmp/qemu-build/install/share/man
ELF interp prefix /usr/gnemul/qemu-%M
Source path       /tmp/qemu-test/src
C compiler        cc
Host C compiler   cc
C++ compiler      
Objective-C compiler cc
ARFLAGS           rv
CFLAGS            -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g 
QEMU_CFLAGS       -I/usr/include/pixman-1    -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include   -fPIE -DPIE -m64 -mcx16 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv  -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-all
LDFLAGS           -Wl,--warn-common -Wl,-z,relro -Wl,-z,now -pie -m64 -g 
make              make
install           install
python            python -B
smbd              /usr/sbin/smbd
module support    no
host CPU          x86_64
host big endian   no
target list       x86_64-softmmu aarch64-softmmu
tcg debug enabled no
gprof enabled     no
sparse enabled    no
strip binaries    yes
profiler          no
static build      no
pixman            system
SDL support       yes (1.2.14)
GTK support       no 
GTK GL support    no
VTE support       no 
TLS priority      NORMAL
GNUTLS support    no
GNUTLS rnd        no
libgcrypt         no
libgcrypt kdf     no
nettle            no 
nettle kdf        no
libtasn1          no
curses support    no
virgl support     no
curl support      no
mingw32 support   no
Audio drivers     oss
Block whitelist (rw) 
Block whitelist (ro) 
VirtFS support    no
VNC support       yes
VNC SASL support  no
VNC JPEG support  no
VNC PNG support   no
xen support       no
brlapi support    no
bluez  support    no
Documentation     no
PIE               yes
vde support       no
netmap support    no
Linux AIO support no
ATTR/XATTR support yes
Install blobs     yes
KVM support       yes
COLO support      yes
RDMA support      no
TCG interpreter   no
fdt support       yes
preadv support    yes
fdatasync         yes
madvise           yes
posix_madvise     yes
libcap-ng support no
vhost-net support yes
vhost-scsi support yes
vhost-vsock support yes
Trace backends    log
spice support     no 
rbd support       no
xfsctl support    no
smartcard support no
libusb            no
usb net redir     no
OpenGL support    no
OpenGL dmabufs    no
libiscsi support  no
libnfs support    no
build guest agent yes
QGA VSS support   no
QGA w32 disk info no
QGA MSI support   no
seccomp support   no
coroutine backend ucontext
coroutine pool    yes
debug stack usage no
GlusterFS support no
Archipelago support no
gcov              gcov
gcov enabled      no
TPM support       yes
libssh2 support   no
TPM passthrough   yes
QOM debugging     yes
lzo support       no
snappy support    no
bzip2 support     no
NUMA host support no
tcmalloc support  no
jemalloc support  no
avx2 optimization no
replication support yes
  GEN     x86_64-softmmu/config-devices.mak.tmp
  GEN     aarch64-softmmu/config-devices.mak.tmp
  GEN     config-host.h
  GEN     qemu-options.def
  GEN     qmp-commands.h
  GEN     qapi-types.h
  GEN     qapi-event.h
  GEN     qapi-visit.h
  GEN     qmp-introspect.h
  GEN     x86_64-softmmu/config-devices.mak
  GEN     aarch64-softmmu/config-devices.mak
  GEN     module_block.h
  GEN     tests/test-qapi-types.h
  GEN     tests/test-qapi-visit.h
  GEN     tests/test-qmp-commands.h
  GEN     tests/test-qapi-event.h
  GEN     tests/test-qmp-introspect.h
  GEN     config-all-devices.mak
  GEN     trace/generated-tracers.h
  GEN     trace/generated-tcg-tracers.h
  GEN     trace/generated-helpers-wrappers.h
  GEN     trace/generated-helpers.h
  CC      tests/qemu-iotests/socket_scm_helper.o
  GEN     qga/qapi-generated/qga-qapi-types.h
  GEN     qga/qapi-generated/qga-qapi-visit.h
  GEN     qga/qapi-generated/qga-qmp-commands.h
  GEN     qga/qapi-generated/qga-qapi-types.c
  GEN     qga/qapi-generated/qga-qapi-visit.c
  GEN     qga/qapi-generated/qga-qmp-marshal.c
  GEN     qmp-introspect.c
  GEN     qapi-types.c
  GEN     qapi-visit.c
  GEN     qapi-event.c
  CC      qapi/qapi-visit-core.o
  CC      qapi/qapi-dealloc-visitor.o
  CC      qapi/qobject-input-visitor.o
  CC      qapi/qobject-output-visitor.o
  CC      qapi/qmp-registry.o
  CC      qapi/qmp-dispatch.o
  CC      qapi/string-input-visitor.o
  CC      qapi/string-output-visitor.o
  CC      qapi/opts-visitor.o
  CC      qapi/qapi-clone-visitor.o
  CC      qapi/qmp-event.o
  CC      qapi/qapi-util.o
  CC      qobject/qnull.o
  CC      qobject/qint.o
  CC      qobject/qstring.o
  CC      qobject/qdict.o
  CC      qobject/qlist.o
  CC      qobject/qfloat.o
  CC      qobject/qbool.o
  CC      qobject/qjson.o
  CC      qobject/qobject.o
  CC      qobject/json-lexer.o
  CC      qobject/json-streamer.o
  CC      qobject/json-parser.o
  GEN     trace/generated-tracers.c
  CC      trace/control.o
  CC      trace/qmp.o
  CC      util/osdep.o
  CC      util/cutils.o
  CC      util/unicode.o
  CC      util/qemu-timer-common.o
  CC      util/bufferiszero.o
  CC      util/compatfd.o
  CC      util/event_notifier-posix.o
  CC      util/mmap-alloc.o
  CC      util/oslib-posix.o
  CC      util/qemu-openpty.o
  CC      util/qemu-thread-posix.o
  CC      util/memfd.o
  CC      util/envlist.o
  CC      util/path.o
  CC      util/module.o
  CC      util/bitmap.o
  CC      util/bitops.o
  CC      util/hbitmap.o
  CC      util/fifo8.o
  CC      util/acl.o
  CC      util/error.o
  CC      util/qemu-error.o
  CC      util/id.o
  CC      util/iov.o
  CC      util/qemu-config.o
  CC      util/qemu-sockets.o
  CC      util/uri.o
  CC      util/notify.o
  CC      util/qemu-option.o
  CC      util/qemu-progress.o
  CC      util/hexdump.o
  CC      util/crc32c.o
  CC      util/uuid.o
  CC      util/throttle.o
  CC      util/getauxval.o
  CC      util/readline.o
  CC      util/rcu.o
  CC      util/qemu-coroutine.o
  CC      util/qemu-coroutine-lock.o
  CC      util/qemu-coroutine-io.o
  CC      util/qemu-coroutine-sleep.o
  CC      util/coroutine-ucontext.o
  CC      util/buffer.o
  CC      util/timed-average.o
  CC      util/base64.o
  CC      util/log.o
  CC      util/qdist.o
  CC      util/qht.o
  CC      util/range.o
  CC      crypto/pbkdf-stub.o
  CC      stubs/arch-query-cpu-def.o
  CC      stubs/arch-query-cpu-model-expansion.o
  CC      stubs/arch-query-cpu-model-comparison.o
  CC      stubs/arch-query-cpu-model-baseline.o
  CC      stubs/bdrv-next-monitor-owned.o
  CC      stubs/blk-commit-all.o
  CC      stubs/blockdev-close-all-bdrv-states.o
  CC      stubs/cpu-get-clock.o
  CC      stubs/clock-warp.o
  CC      stubs/cpu-get-icount.o
  CC      stubs/dump.o
  CC      stubs/error-printf.o
  CC      stubs/fdset-add-fd.o
  CC      stubs/fdset-find-fd.o
  CC      stubs/fdset-remove-fd.o
  CC      stubs/fdset-get-fd.o
  CC      stubs/gdbstub.o
  CC      stubs/get-fd.o
  CC      stubs/get-next-serial.o
  CC      stubs/get-vm-name.o
  CC      stubs/iothread-lock.o
  CC      stubs/iothread.o
  CC      stubs/is-daemonized.o
  CC      stubs/machine-init-done.o
  CC      stubs/migr-blocker.o
  CC      stubs/mon-is-qmp.o
  CC      stubs/qtest.o
  CC      stubs/monitor-init.o
  CC      stubs/notify-event.o
  CC      stubs/replay.o
  CC      stubs/runstate-check.o
  CC      stubs/replay-user.o
  CC      stubs/reset.o
  CC      stubs/set-fd-handler.o
  CC      stubs/slirp.o
  CC      stubs/sysbus.o
  CC      stubs/trace-control.o
  CC      stubs/uuid.o
  CC      stubs/vm-stop.o
  CC      stubs/vmstate.o
  CC      stubs/cpus.o
  CC      stubs/kvm.o
  CC      stubs/qmp_pc_dimm_device_list.o
  CC      stubs/target-monitor-defs.o
  CC      stubs/target-get-monitor-def.o
  CC      stubs/iohandler.o
  CC      stubs/vhost.o
  CC      stubs/smbios_type_38.o
  CC      stubs/ipmi.o
  CC      stubs/pc_madt_cpu_entry.o
  CC      contrib/ivshmem-client/ivshmem-client.o
  CC      stubs/migration-colo.o
  CC      contrib/ivshmem-client/main.o
  CC      contrib/ivshmem-server/ivshmem-server.o
  CC      contrib/ivshmem-server/main.o
  CC      qemu-nbd.o
  CC      async.o
  CC      block.o
  CC      thread-pool.o
  CC      blockjob.o
  CC      main-loop.o
  CC      iohandler.o
  CC      qemu-timer.o
  CC      aio-posix.o
  CC      qemu-io-cmds.o
  CC      replication.o
  CC      block/raw_bsd.o
  CC      block/qcow.o
  CC      block/vdi.o
  CC      block/vmdk.o
  CC      block/cloop.o
  CC      block/bochs.o
  CC      block/vpc.o
  CC      block/vvfat.o
  CC      block/dmg.o
  CC      block/qcow2.o
  CC      block/qcow2-refcount.o
  CC      block/qcow2-cluster.o
  CC      block/qcow2-snapshot.o
  CC      block/qcow2-cache.o
  CC      block/qed.o
  CC      block/qed-gencb.o
  CC      block/qed-l2-cache.o
  CC      block/qed-table.o
  CC      block/qed-cluster.o
  CC      block/qed-check.o
  CC      block/vhdx.o
  CC      block/vhdx-endian.o
  CC      block/vhdx-log.o
  CC      block/quorum.o
  CC      block/parallels.o
  CC      block/blkdebug.o
  CC      block/blkverify.o
  CC      block/blkreplay.o
  CC      block/block-backend.o
  CC      block/snapshot.o
  CC      block/qapi.o
  CC      block/raw-posix.o
  CC      block/null.o
  CC      block/mirror.o
  CC      block/commit.o
  CC      block/io.o
  CC      block/throttle-groups.o
  CC      block/nbd.o
  CC      block/nbd-client.o
  CC      block/sheepdog.o
  CC      block/accounting.o
  CC      block/dirty-bitmap.o
  CC      block/write-threshold.o
  CC      block/backup.o
  CC      block/replication.o
  CC      block/crypto.o
  CC      nbd/server.o
  CC      nbd/client.o
  CC      nbd/common.o
  CC      crypto/init.o
  CC      crypto/hash.o
  CC      crypto/hash-glib.o
  CC      crypto/aes.o
  CC      crypto/desrfb.o
  CC      crypto/cipher.o
  CC      crypto/tlscreds.o
  CC      crypto/tlscredsanon.o
  CC      crypto/tlscredsx509.o
  CC      crypto/tlssession.o
  CC      crypto/secret.o
  CC      crypto/random-platform.o
  CC      crypto/pbkdf.o
  CC      crypto/ivgen.o
  CC      crypto/ivgen-essiv.o
  CC      crypto/ivgen-plain.o
  CC      crypto/ivgen-plain64.o
  CC      crypto/afsplit.o
  CC      crypto/xts.o
  CC      crypto/block.o
  CC      crypto/block-qcow.o
  CC      crypto/block-luks.o
  CC      io/channel.o
  CC      io/channel-buffer.o
  CC      io/channel-command.o
  CC      io/channel-file.o
  CC      io/channel-socket.o
  CC      io/channel-tls.o
  CC      io/channel-watch.o
  CC      io/channel-websock.o
  CC      io/channel-util.o
  CC      io/task.o
  CC      qom/object.o
  CC      qom/container.o
  CC      qom/qom-qobject.o
  CC      qom/object_interfaces.o
  GEN     qemu-img-cmds.h
  CC      qemu-io.o
  CC      qemu-bridge-helper.o
  CC      blockdev.o
  CC      blockdev-nbd.o
  CC      iothread.o
  CC      qdev-monitor.o
  CC      device-hotplug.o
  CC      os-posix.o
  CC      qemu-char.o
  CC      page_cache.o
  CC      accel.o
  CC      bt-host.o
  CC      bt-vhci.o
  CC      dma-helpers.o
  CC      vl.o
  CC      tpm.o
  CC      device_tree.o
  GEN     qmp-marshal.c
  CC      qmp.o
  CC      hmp.o
  CC      cpus-common.o
  CC      audio/audio.o
  CC      audio/noaudio.o
  CC      audio/wavaudio.o
  CC      audio/mixeng.o
  CC      audio/sdlaudio.o
  CC      audio/ossaudio.o
  CC      audio/wavcapture.o
  CC      backends/rng.o
  CC      backends/rng-egd.o
  CC      backends/rng-random.o
  CC      backends/msmouse.o
  CC      backends/testdev.o
  CC      backends/tpm.o
  CC      backends/hostmem.o
  CC      backends/hostmem-ram.o
  CC      backends/hostmem-file.o
  CC      backends/cryptodev.o
  CC      backends/cryptodev-builtin.o
  CC      block/stream.o
  CC      disas/arm.o
  CC      disas/i386.o
  CC      fsdev/qemu-fsdev-dummy.o
  CC      fsdev/qemu-fsdev-opts.o
  CC      hw/acpi/core.o
  CC      hw/acpi/piix4.o
  CC      hw/acpi/pcihp.o
  CC      hw/acpi/ich9.o
  CC      hw/acpi/tco.o
  CC      hw/acpi/cpu_hotplug.o
  CC      hw/acpi/memory_hotplug.o
  CC      hw/acpi/memory_hotplug_acpi_table.o
  CC      hw/acpi/cpu.o
  CC      hw/acpi/nvdimm.o
  CC      hw/acpi/acpi_interface.o
  CC      hw/acpi/bios-linker-loader.o
  CC      hw/acpi/aml-build.o
  CC      hw/acpi/ipmi.o
  CC      hw/audio/sb16.o
  CC      hw/audio/es1370.o
  CC      hw/audio/ac97.o
  CC      hw/audio/fmopl.o
  CC      hw/audio/adlib.o
  CC      hw/audio/gus.o
  CC      hw/audio/gusemu_hal.o
  CC      hw/audio/gusemu_mixer.o
  CC      hw/audio/cs4231a.o
  CC      hw/audio/intel-hda.o
  CC      hw/audio/hda-codec.o
  CC      hw/audio/pcspk.o
  CC      hw/audio/wm8750.o
  CC      hw/audio/pl041.o
  CC      hw/audio/lm4549.o
  CC      hw/audio/marvell_88w8618.o
  CC      hw/block/block.o
  CC      hw/block/cdrom.o
  CC      hw/block/hd-geometry.o
  CC      hw/block/fdc.o
  CC      hw/block/nand.o
  CC      hw/block/pflash_cfi01.o
  CC      hw/block/m25p80.o
  CC      hw/block/pflash_cfi02.o
  CC      hw/block/ecc.o
  CC      hw/block/onenand.o
  CC      hw/block/nvme.o
  CC      hw/bt/core.o
  CC      hw/bt/l2cap.o
  CC      hw/bt/sdp.o
  CC      hw/bt/hci.o
  CC      hw/bt/hid.o
  CC      hw/bt/hci-csr.o
  CC      hw/char/ipoctal232.o
  CC      hw/char/parallel.o
  CC      hw/char/pl011.o
  CC      hw/char/serial.o
  CC      hw/char/serial-isa.o
  CC      hw/char/serial-pci.o
  CC      hw/char/virtio-console.o
  CC      hw/char/cadence_uart.o
  CC      hw/char/debugcon.o
  CC      hw/char/imx_serial.o
  CC      hw/core/qdev.o
  CC      hw/core/qdev-properties.o
  CC      hw/core/bus.o
  CC      hw/core/fw-path-provider.o
  CC      hw/core/irq.o
  CC      hw/core/hotplug.o
  CC      hw/core/ptimer.o
  CC      hw/core/sysbus.o
  CC      hw/core/null-machine.o
  CC      hw/core/machine.o
  CC      hw/core/loader.o
  CC      hw/core/qdev-properties-system.o
  CC      hw/core/register.o
  CC      hw/core/or-irq.o
  CC      hw/core/platform-bus.o
  CC      hw/display/ads7846.o
  CC      hw/display/cirrus_vga.o
  CC      hw/display/pl110.o
  CC      hw/display/ssd0303.o
  CC      hw/display/ssd0323.o
  CC      hw/display/vga-pci.o
  CC      hw/display/vga-isa.o
  CC      hw/display/vmware_vga.o
  CC      hw/display/blizzard.o
  CC      hw/display/exynos4210_fimd.o
  CC      hw/display/framebuffer.o
  CC      hw/display/tc6393xb.o
  CC      hw/dma/pl330.o
  CC      hw/dma/pl080.o
  CC      hw/dma/i8257.o
  CC      hw/dma/xlnx-zynq-devcfg.o
  CC      hw/gpio/max7310.o
  CC      hw/gpio/pl061.o
  CC      hw/gpio/zaurus.o
  CC      hw/i2c/core.o
  CC      hw/gpio/gpio_key.o
  CC      hw/i2c/smbus.o
  CC      hw/i2c/smbus_eeprom.o
  CC      hw/i2c/i2c-ddc.o
  CC      hw/i2c/versatile_i2c.o
  CC      hw/i2c/smbus_ich9.o
  CC      hw/i2c/pm_smbus.o
  CC      hw/i2c/bitbang_i2c.o
  CC      hw/i2c/exynos4210_i2c.o
  CC      hw/i2c/imx_i2c.o
  CC      hw/i2c/aspeed_i2c.o
  CC      hw/ide/core.o
  CC      hw/ide/atapi.o
  CC      hw/ide/qdev.o
  CC      hw/ide/pci.o
  CC      hw/ide/isa.o
  CC      hw/ide/piix.o
  CC      hw/ide/microdrive.o
  CC      hw/ide/ahci.o
  CC      hw/ide/ich.o
  CC      hw/input/hid.o
  CC      hw/input/lm832x.o
  CC      hw/input/pckbd.o
  CC      hw/input/pl050.o
  CC      hw/input/ps2.o
  CC      hw/input/stellaris_input.o
  CC      hw/input/tsc2005.o
  CC      hw/input/vmmouse.o
  CC      hw/input/virtio-input.o
  CC      hw/input/virtio-input-hid.o
  CC      hw/input/virtio-input-host.o
  CC      hw/intc/i8259_common.o
  CC      hw/intc/i8259.o
  CC      hw/intc/pl190.o
  CC      hw/intc/imx_avic.o
  CC      hw/intc/realview_gic.o
  CC      hw/intc/ioapic_common.o
  CC      hw/intc/arm_gic.o
  CC      hw/intc/arm_gic_common.o
  CC      hw/intc/arm_gicv2m.o
  CC      hw/intc/arm_gicv3_common.o
  CC      hw/intc/arm_gicv3.o
  CC      hw/intc/arm_gicv3_dist.o
  CC      hw/intc/arm_gicv3_redist.o
  CC      hw/intc/arm_gicv3_its_common.o
  CC      hw/intc/intc.o
  CC      hw/ipack/ipack.o
  CC      hw/ipack/tpci200.o
  CC      hw/ipmi/ipmi.o
  CC      hw/ipmi/ipmi_bmc_sim.o
  CC      hw/ipmi/ipmi_bmc_extern.o
  CC      hw/ipmi/isa_ipmi_kcs.o
  CC      hw/ipmi/isa_ipmi_bt.o
  CC      hw/isa/isa-bus.o
  CC      hw/isa/apm.o
  CC      hw/mem/nvdimm.o
  CC      hw/mem/pc-dimm.o
  CC      hw/misc/applesmc.o
  CC      hw/misc/max111x.o
  CC      hw/misc/tmp105.o
  CC      hw/misc/debugexit.o
  CC      hw/misc/sga.o
  CC      hw/misc/pc-testdev.o
  CC      hw/misc/pci-testdev.o
  CC      hw/misc/arm_l2x0.o
  CC      hw/misc/arm_integrator_debug.o
  CC      hw/misc/a9scu.o
  CC      hw/misc/arm11scu.o
  CC      hw/net/ne2000.o
  CC      hw/net/eepro100.o
  CC      hw/net/pcnet-pci.o
  CC      hw/net/pcnet.o
  CC      hw/net/e1000.o
  CC      hw/net/e1000x_common.o
  CC      hw/net/net_tx_pkt.o
  CC      hw/net/net_rx_pkt.o
  CC      hw/net/e1000e.o
  CC      hw/net/e1000e_core.o
  CC      hw/net/rtl8139.o
  CC      hw/net/vmxnet3.o
  CC      hw/net/smc91c111.o
  CC      hw/net/lan9118.o
  CC      hw/net/ne2000-isa.o
  CC      hw/net/xgmac.o
  CC      hw/net/allwinner_emac.o
  CC      hw/net/imx_fec.o
  CC      hw/net/cadence_gem.o
  CC      hw/net/stellaris_enet.o
  CC      hw/net/rocker/rocker.o
  CC      hw/net/rocker/rocker_fp.o
  CC      hw/net/rocker/rocker_desc.o
  CC      hw/net/rocker/rocker_world.o
  CC      hw/net/rocker/rocker_of_dpa.o
  CC      hw/nvram/eeprom93xx.o
  CC      hw/nvram/fw_cfg.o
  CC      hw/nvram/chrp_nvram.o
  CC      hw/pci-bridge/pci_bridge_dev.o
  CC      hw/pci-bridge/pci_expander_bridge.o
  CC      hw/pci-bridge/xio3130_upstream.o
  CC      hw/pci-bridge/ioh3420.o
  CC      hw/pci-bridge/xio3130_downstream.o
  CC      hw/pci-bridge/i82801b11.o
  CC      hw/pci-host/pam.o
  CC      hw/pci-host/versatile.o
  CC      hw/pci-host/piix.o
  CC      hw/pci-host/q35.o
  CC      hw/pci-host/gpex.o
  CC      hw/pci/pci.o
  CC      hw/pci/pci_bridge.o
  CC      hw/pci/msix.o
  CC      hw/pci/msi.o
  CC      hw/pci/shpc.o
  CC      hw/pci/slotid_cap.o
  CC      hw/pci/pci_host.o
  CC      hw/pci/pcie_host.o
  CC      hw/pci/pcie.o
  CC      hw/pci/pcie_aer.o
  CC      hw/pci/pcie_port.o
  CC      hw/pci/pci-stub.o
  CC      hw/pcmcia/pcmcia.o
  CC      hw/scsi/scsi-disk.o
  CC      hw/scsi/scsi-generic.o
  CC      hw/scsi/scsi-bus.o
  CC      hw/scsi/lsi53c895a.o
  CC      hw/scsi/mptsas.o
  CC      hw/scsi/mptconfig.o
  CC      hw/scsi/mptendian.o
  CC      hw/scsi/megasas.o
  CC      hw/scsi/vmw_pvscsi.o
  CC      hw/scsi/esp.o
  CC      hw/scsi/esp-pci.o
  CC      hw/sd/pl181.o
  CC      hw/sd/ssi-sd.o
  CC      hw/sd/sd.o
  CC      hw/sd/core.o
  CC      hw/sd/sdhci.o
  CC      hw/smbios/smbios.o
  CC      hw/smbios/smbios_type_38.o
  CC      hw/ssi/pl022.o
  CC      hw/ssi/ssi.o
  CC      hw/ssi/xilinx_spips.o
  CC      hw/ssi/aspeed_smc.o
  CC      hw/ssi/stm32f2xx_spi.o
  CC      hw/timer/arm_timer.o
  CC      hw/timer/arm_mptimer.o
  CC      hw/timer/a9gtimer.o
  CC      hw/timer/cadence_ttc.o
  CC      hw/timer/ds1338.o
  CC      hw/timer/hpet.o
  CC      hw/timer/i8254_common.o
  CC      hw/timer/i8254.o
  CC      hw/timer/pl031.o
  CC      hw/timer/twl92230.o
  CC      hw/timer/imx_epit.o
  CC      hw/timer/imx_gpt.o
  CC      hw/timer/stm32f2xx_timer.o
  CC      hw/timer/aspeed_timer.o
  CC      hw/tpm/tpm_tis.o
  CC      hw/tpm/tpm_passthrough.o
  CC      hw/tpm/tpm_util.o
  CC      hw/usb/core.o
  CC      hw/usb/bus.o
  CC      hw/usb/combined-packet.o
/tmp/qemu-test/src/hw/nvram/fw_cfg.c: In function ‘fw_cfg_dma_transfer’:
/tmp/qemu-test/src/hw/nvram/fw_cfg.c:329: warning: ‘read’ may be used uninitialized in this function
  CC      hw/usb/libhw.o
  CC      hw/usb/desc.o
  CC      hw/usb/desc-msos.o
  CC      hw/usb/hcd-uhci.o
  CC      hw/usb/hcd-ohci.o
  CC      hw/usb/hcd-ehci.o
  CC      hw/usb/hcd-ehci-pci.o
  CC      hw/usb/hcd-ehci-sysbus.o
  CC      hw/usb/hcd-xhci.o
  CC      hw/usb/hcd-musb.o
  CC      hw/usb/dev-hub.o
  CC      hw/usb/dev-hid.o
  CC      hw/usb/dev-wacom.o
  CC      hw/usb/dev-storage.o
  CC      hw/usb/dev-uas.o
  CC      hw/usb/dev-audio.o
  CC      hw/usb/dev-network.o
  CC      hw/usb/dev-serial.o
  CC      hw/usb/dev-bluetooth.o
  CC      hw/usb/dev-smartcard-reader.o
  CC      hw/usb/dev-mtp.o
  CC      hw/usb/host-stub.o
  CC      hw/virtio/virtio-rng.o
  CC      hw/virtio/virtio-pci.o
  CC      hw/virtio/virtio-bus.o
  CC      hw/virtio/virtio-mmio.o
  CC      hw/watchdog/watchdog.o
  CC      hw/watchdog/wdt_i6300esb.o
  CC      hw/watchdog/wdt_ib700.o
  CC      migration/migration.o
  CC      migration/socket.o
  CC      migration/fd.o
  CC      migration/exec.o
  CC      migration/tls.o
  CC      migration/colo-comm.o
  CC      migration/colo.o
  CC      migration/colo-failover.o
  CC      migration/vmstate.o
  CC      migration/qemu-file.o
  CC      migration/qemu-file-channel.o
  CC      migration/xbzrle.o
  CC      migration/postcopy-ram.o
  CC      migration/qjson.o
  CC      migration/block.o
  CC      net/net.o
  CC      net/queue.o
  CC      net/checksum.o
  CC      net/util.o
  CC      net/hub.o
  CC      net/socket.o
  CC      net/dump.o
  CC      net/eth.o
  CC      net/l2tpv3.o
  CC      net/vhost-user.o
  CC      net/tap.o
  CC      net/tap-linux.o
  CC      net/slirp.o
  CC      net/filter.o
  CC      net/filter-buffer.o
  CC      net/colo-compare.o
  CC      net/filter-mirror.o
  CC      net/colo.o
  CC      net/filter-rewriter.o
  CC      qom/cpu.o
  CC      replay/replay.o
  CC      replay/replay-internal.o
/tmp/qemu-test/src/replay/replay-internal.c: In function ‘replay_put_array’:
/tmp/qemu-test/src/replay/replay-internal.c:65: warning: ignoring return value of ‘fwrite’, declared with attribute warn_unused_result
  CC      replay/replay-events.o
  CC      replay/replay-time.o
  CC      replay/replay-input.o
  CC      replay/replay-snapshot.o
  CC      replay/replay-char.o
  CC      slirp/cksum.o
  CC      slirp/if.o
  CC      slirp/ip_icmp.o
  CC      slirp/ip6_icmp.o
  CC      slirp/ip6_input.o
  CC      slirp/ip6_output.o
  CC      slirp/ip_input.o
  CC      slirp/ip_output.o
  CC      slirp/dnssearch.o
  CC      slirp/dhcpv6.o
  CC      slirp/slirp.o
  CC      slirp/misc.o
  CC      slirp/mbuf.o
  CC      slirp/sbuf.o
  CC      slirp/socket.o
  CC      slirp/tcp_input.o
  CC      slirp/tcp_output.o
  CC      slirp/tcp_subr.o
/tmp/qemu-test/src/slirp/tcp_input.c: In function ‘tcp_input’:
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_p’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_len’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_tos’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_id’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_off’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_ttl’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_sum’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_src.s_addr’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_dst.s_addr’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:220: warning: ‘save_ip6.ip_nh’ may be used uninitialized in this function
  CC      slirp/tcp_timer.o
  CC      slirp/udp.o
  CC      slirp/udp6.o
  CC      slirp/bootp.o
  CC      slirp/tftp.o
  CC      slirp/arp_table.o
  CC      slirp/ndp_table.o
  CC      ui/keymaps.o
  CC      ui/console.o
  CC      ui/cursor.o
  CC      ui/qemu-pixman.o
  CC      ui/input.o
  CC      ui/input-keymap.o
  CC      ui/input-legacy.o
  CC      ui/input-linux.o
  CC      ui/sdl.o
  CC      ui/sdl_zoom.o
  CC      ui/vnc.o
  CC      ui/x_keymap.o
  CC      ui/vnc-enc-zlib.o
  CC      ui/vnc-enc-hextile.o
  CC      ui/vnc-enc-tight.o
  CC      ui/vnc-palette.o
  CC      ui/vnc-enc-zrle.o
  CC      ui/vnc-auth-vencrypt.o
  CC      ui/vnc-ws.o
  CC      ui/vnc-jobs.o
  LINK    tests/qemu-iotests/socket_scm_helper
  CC      qga/commands.o
  CC      qga/guest-agent-command-state.o
  CC      qga/main.o
  CC      qga/commands-posix.o
  CC      qga/channel-posix.o
  CC      qga/qapi-generated/qga-qapi-types.o
  CC      qga/qapi-generated/qga-qapi-visit.o
  CC      qga/qapi-generated/qga-qmp-marshal.o
  CC      qmp-introspect.o
  CC      qapi-visit.o
  CC      qapi-types.o
  CC      qapi-event.o
  CC      qemu-img.o
  AR      libqemustub.a
  CC      qmp-marshal.o
  CC      trace/generated-tracers.o
  AS      optionrom/multiboot.o
  AS      optionrom/kvmvapic.o
  AS      optionrom/linuxboot.o
  CC      optionrom/linuxboot_dma.o
cc: unrecognized option '-no-integrated-as'
cc: unrecognized option '-no-integrated-as'
  BUILD   optionrom/linuxboot_dma.img
  BUILD   optionrom/linuxboot_dma.raw
  BUILD   optionrom/linuxboot.img
  BUILD   optionrom/multiboot.img
  BUILD   optionrom/linuxboot.raw
  SIGN    optionrom/linuxboot_dma.bin
  BUILD   optionrom/kvmvapic.img
  BUILD   optionrom/multiboot.raw
  SIGN    optionrom/linuxboot.bin
  BUILD   optionrom/kvmvapic.raw
  SIGN    optionrom/multiboot.bin
  SIGN    optionrom/kvmvapic.bin
  AR      libqemuutil.a
  LINK    qemu-ga
  LINK    ivshmem-client
  LINK    ivshmem-server
  LINK    qemu-nbd
  LINK    qemu-img
  LINK    qemu-io
  LINK    qemu-bridge-helper
  GEN     x86_64-softmmu/hmp-commands.h
  GEN     x86_64-softmmu/hmp-commands-info.h
  GEN     x86_64-softmmu/config-target.h
  GEN     aarch64-softmmu/hmp-commands.h
  GEN     aarch64-softmmu/hmp-commands-info.h
  GEN     aarch64-softmmu/config-target.h
  CC      x86_64-softmmu/exec.o
  CC      x86_64-softmmu/translate-all.o
  CC      x86_64-softmmu/cpu-exec.o
  CC      x86_64-softmmu/translate-common.o
  CC      x86_64-softmmu/cpu-exec-common.o
  CC      x86_64-softmmu/tcg/tcg.o
  CC      x86_64-softmmu/tcg/tcg-op.o
  CC      x86_64-softmmu/tcg/optimize.o
  CC      x86_64-softmmu/tcg/tcg-common.o
  CC      x86_64-softmmu/arch_init.o
  CC      x86_64-softmmu/fpu/softfloat.o
  CC      x86_64-softmmu/disas.o
  CC      x86_64-softmmu/tcg-runtime.o
  CC      x86_64-softmmu/cpus.o
  CC      x86_64-softmmu/monitor.o
  CC      x86_64-softmmu/gdbstub.o
  CC      x86_64-softmmu/balloon.o
  CC      x86_64-softmmu/ioport.o
  CC      x86_64-softmmu/numa.o
  CC      x86_64-softmmu/qtest.o
  CC      x86_64-softmmu/bootdevice.o
  CC      x86_64-softmmu/kvm-all.o
  CC      x86_64-softmmu/memory.o
  CC      x86_64-softmmu/cputlb.o
  CC      aarch64-softmmu/exec.o
  CC      aarch64-softmmu/translate-all.o
  CC      x86_64-softmmu/memory_mapping.o
  CC      aarch64-softmmu/cpu-exec.o
  CC      aarch64-softmmu/translate-common.o
  CC      aarch64-softmmu/cpu-exec-common.o
  CC      aarch64-softmmu/tcg/tcg.o
  CC      x86_64-softmmu/dump.o
  CC      aarch64-softmmu/tcg/tcg-op.o
  CC      x86_64-softmmu/migration/ram.o
  CC      aarch64-softmmu/tcg/optimize.o
  CC      aarch64-softmmu/tcg/tcg-common.o
  CC      aarch64-softmmu/fpu/softfloat.o
  CC      x86_64-softmmu/migration/savevm.o
  CC      x86_64-softmmu/xen-common-stub.o
  CC      x86_64-softmmu/xen-hvm-stub.o
  CC      x86_64-softmmu/hw/block/virtio-blk.o
  CC      aarch64-softmmu/disas.o
  CC      aarch64-softmmu/tcg-runtime.o
  GEN     aarch64-softmmu/gdbstub-xml.c
  CC      aarch64-softmmu/kvm-stub.o
  CC      x86_64-softmmu/hw/block/dataplane/virtio-blk.o
  CC      aarch64-softmmu/arch_init.o
  CC      aarch64-softmmu/cpus.o
  CC      aarch64-softmmu/monitor.o
  CC      aarch64-softmmu/gdbstub.o
  CC      x86_64-softmmu/hw/char/virtio-serial-bus.o
  CC      aarch64-softmmu/balloon.o
  CC      aarch64-softmmu/ioport.o
  CC      x86_64-softmmu/hw/core/nmi.o
  CC      x86_64-softmmu/hw/core/generic-loader.o
  CC      x86_64-softmmu/hw/cpu/core.o
  CC      x86_64-softmmu/hw/display/vga.o
  CC      aarch64-softmmu/numa.o
  CC      x86_64-softmmu/hw/display/virtio-gpu.o
  CC      x86_64-softmmu/hw/display/virtio-gpu-3d.o
  CC      aarch64-softmmu/qtest.o
  CC      aarch64-softmmu/bootdevice.o
  CC      aarch64-softmmu/memory.o
  CC      aarch64-softmmu/cputlb.o
  CC      aarch64-softmmu/memory_mapping.o
  CC      aarch64-softmmu/dump.o
  CC      aarch64-softmmu/migration/ram.o
  CC      x86_64-softmmu/hw/display/virtio-gpu-pci.o
  CC      aarch64-softmmu/migration/savevm.o
  CC      aarch64-softmmu/xen-common-stub.o
  CC      aarch64-softmmu/xen-hvm-stub.o
  CC      aarch64-softmmu/hw/adc/stm32f2xx_adc.o
  CC      aarch64-softmmu/hw/block/virtio-blk.o
  CC      aarch64-softmmu/hw/block/dataplane/virtio-blk.o
  CC      x86_64-softmmu/hw/display/virtio-vga.o
  CC      x86_64-softmmu/hw/intc/apic.o
  CC      x86_64-softmmu/hw/intc/apic_common.o
  CC      x86_64-softmmu/hw/intc/ioapic.o
  CC      aarch64-softmmu/hw/char/exynos4210_uart.o
  CC      aarch64-softmmu/hw/char/omap_uart.o
  CC      aarch64-softmmu/hw/char/digic-uart.o
  CC      aarch64-softmmu/hw/char/stm32f2xx_usart.o
  CC      x86_64-softmmu/hw/isa/lpc_ich9.o
  CC      aarch64-softmmu/hw/char/bcm2835_aux.o
  CC      aarch64-softmmu/hw/char/virtio-serial-bus.o
  CC      aarch64-softmmu/hw/core/nmi.o
  CC      aarch64-softmmu/hw/core/generic-loader.o
  CC      aarch64-softmmu/hw/cpu/arm11mpcore.o
  CC      x86_64-softmmu/hw/misc/vmport.o
  CC      aarch64-softmmu/hw/cpu/realview_mpcore.o
  CC      aarch64-softmmu/hw/cpu/a9mpcore.o
  CC      aarch64-softmmu/hw/cpu/a15mpcore.o
  CC      aarch64-softmmu/hw/cpu/core.o
  CC      aarch64-softmmu/hw/display/omap_dss.o
  CC      aarch64-softmmu/hw/display/omap_lcdc.o
  CC      aarch64-softmmu/hw/display/pxa2xx_lcd.o
  CC      aarch64-softmmu/hw/display/bcm2835_fb.o
  CC      aarch64-softmmu/hw/display/vga.o
  CC      x86_64-softmmu/hw/misc/ivshmem.o
  CC      x86_64-softmmu/hw/misc/pvpanic.o
  CC      x86_64-softmmu/hw/misc/edu.o
  CC      x86_64-softmmu/hw/misc/hyperv_testdev.o
  CC      x86_64-softmmu/hw/net/virtio-net.o
  CC      x86_64-softmmu/hw/net/vhost_net.o
  CC      x86_64-softmmu/hw/scsi/virtio-scsi.o
  CC      aarch64-softmmu/hw/display/virtio-gpu.o
  CC      x86_64-softmmu/hw/scsi/virtio-scsi-dataplane.o
  CC      aarch64-softmmu/hw/display/virtio-gpu-3d.o
  CC      x86_64-softmmu/hw/scsi/vhost-scsi.o
  CC      x86_64-softmmu/hw/timer/mc146818rtc.o
  CC      x86_64-softmmu/hw/vfio/common.o
  CC      aarch64-softmmu/hw/display/virtio-gpu-pci.o
  CC      x86_64-softmmu/hw/vfio/pci.o
  CC      x86_64-softmmu/hw/vfio/pci-quirks.o
  CC      x86_64-softmmu/hw/vfio/platform.o
  CC      aarch64-softmmu/hw/display/dpcd.o
  CC      x86_64-softmmu/hw/vfio/calxeda-xgmac.o
  CC      aarch64-softmmu/hw/display/xlnx_dp.o
  CC      aarch64-softmmu/hw/dma/xlnx_dpdma.o
  CC      aarch64-softmmu/hw/dma/soc_dma.o
  CC      aarch64-softmmu/hw/dma/omap_dma.o
  CC      aarch64-softmmu/hw/dma/pxa2xx_dma.o
  CC      aarch64-softmmu/hw/dma/bcm2835_dma.o
  CC      x86_64-softmmu/hw/vfio/amd-xgbe.o
  CC      x86_64-softmmu/hw/vfio/spapr.o
  CC      x86_64-softmmu/hw/virtio/virtio.o
  CC      aarch64-softmmu/hw/gpio/omap_gpio.o
  CC      aarch64-softmmu/hw/gpio/imx_gpio.o
  CC      aarch64-softmmu/hw/i2c/omap_i2c.o
  CC      aarch64-softmmu/hw/input/pxa2xx_keypad.o
  CC      x86_64-softmmu/hw/virtio/virtio-balloon.o
  CC      aarch64-softmmu/hw/input/tsc210x.o
  CC      x86_64-softmmu/hw/virtio/vhost.o
  CC      x86_64-softmmu/hw/virtio/vhost-backend.o
  CC      aarch64-softmmu/hw/intc/armv7m_nvic.o
  CC      aarch64-softmmu/hw/intc/exynos4210_gic.o
  CC      aarch64-softmmu/hw/intc/exynos4210_combiner.o
  CC      aarch64-softmmu/hw/intc/omap_intc.o
  CC      aarch64-softmmu/hw/intc/bcm2835_ic.o
  CC      aarch64-softmmu/hw/intc/bcm2836_control.o
  CC      aarch64-softmmu/hw/intc/allwinner-a10-pic.o
  CC      aarch64-softmmu/hw/intc/aspeed_vic.o
  CC      x86_64-softmmu/hw/virtio/vhost-user.o
  CC      x86_64-softmmu/hw/virtio/vhost-vsock.o
  CC      x86_64-softmmu/hw/virtio/virtio-crypto.o
  CC      x86_64-softmmu/hw/virtio/virtio-crypto-pci.o
  CC      x86_64-softmmu/hw/i386/multiboot.o
  CC      aarch64-softmmu/hw/intc/arm_gicv3_cpuif.o
  CC      aarch64-softmmu/hw/misc/ivshmem.o
  CC      aarch64-softmmu/hw/misc/arm_sysctl.o
  CC      aarch64-softmmu/hw/misc/cbus.o
  CC      x86_64-softmmu/hw/i386/pc.o
  CC      aarch64-softmmu/hw/misc/exynos4210_pmu.o
  CC      x86_64-softmmu/hw/i386/pc_piix.o
  CC      x86_64-softmmu/hw/i386/pc_q35.o
  CC      aarch64-softmmu/hw/misc/imx_ccm.o
/tmp/qemu-test/src/hw/i386/pc_piix.c: In function ‘igd_passthrough_isa_bridge_create’:
/tmp/qemu-test/src/hw/i386/pc_piix.c:1046: warning: ‘pch_rev_id’ may be used uninitialized in this function
  CC      aarch64-softmmu/hw/misc/imx31_ccm.o
  CC      x86_64-softmmu/hw/i386/pc_sysfw.o
  CC      x86_64-softmmu/hw/i386/x86-iommu.o
  CC      aarch64-softmmu/hw/misc/imx25_ccm.o
  CC      aarch64-softmmu/hw/misc/imx6_ccm.o
  CC      x86_64-softmmu/hw/i386/intel_iommu.o
  CC      x86_64-softmmu/hw/i386/amd_iommu.o
  CC      aarch64-softmmu/hw/misc/imx6_src.o
  CC      x86_64-softmmu/hw/i386/kvmvapic.o
  CC      x86_64-softmmu/hw/i386/acpi-build.o
  CC      aarch64-softmmu/hw/misc/mst_fpga.o
  CC      x86_64-softmmu/hw/i386/pci-assign-load-rom.o
  CC      aarch64-softmmu/hw/misc/omap_clk.o
/tmp/qemu-test/src/hw/i386/acpi-build.c: In function ‘build_append_pci_bus_devices’:
/tmp/qemu-test/src/hw/i386/acpi-build.c:501: warning: ‘notify_method’ may be used uninitialized in this function
  CC      x86_64-softmmu/hw/i386/kvm/clock.o
  CC      aarch64-softmmu/hw/misc/omap_gpmc.o
  CC      aarch64-softmmu/hw/misc/omap_l4.o
  CC      aarch64-softmmu/hw/misc/omap_sdrc.o
  CC      x86_64-softmmu/hw/i386/kvm/apic.o
  CC      x86_64-softmmu/hw/i386/kvm/i8259.o
  CC      aarch64-softmmu/hw/misc/omap_tap.o
  CC      x86_64-softmmu/hw/i386/kvm/ioapic.o
  CC      aarch64-softmmu/hw/misc/bcm2835_mbox.o
  CC      aarch64-softmmu/hw/misc/bcm2835_property.o
  CC      aarch64-softmmu/hw/misc/zynq_slcr.o
  CC      x86_64-softmmu/hw/i386/kvm/i8254.o
  CC      x86_64-softmmu/hw/i386/kvm/pci-assign.o
  CC      x86_64-softmmu/target-i386/translate.o
  CC      aarch64-softmmu/hw/misc/zynq-xadc.o
  CC      aarch64-softmmu/hw/misc/stm32f2xx_syscfg.o
  CC      aarch64-softmmu/hw/misc/edu.o
  CC      x86_64-softmmu/target-i386/helper.o
  CC      aarch64-softmmu/hw/misc/auxbus.o
  CC      aarch64-softmmu/hw/misc/aspeed_scu.o
  CC      aarch64-softmmu/hw/misc/aspeed_sdmc.o
  CC      aarch64-softmmu/hw/net/virtio-net.o
  CC      aarch64-softmmu/hw/net/vhost_net.o
  CC      aarch64-softmmu/hw/pcmcia/pxa2xx.o
  CC      x86_64-softmmu/target-i386/cpu.o
  CC      x86_64-softmmu/target-i386/bpt_helper.o
  CC      x86_64-softmmu/target-i386/excp_helper.o
  CC      aarch64-softmmu/hw/scsi/virtio-scsi.o
  CC      x86_64-softmmu/target-i386/fpu_helper.o
  CC      x86_64-softmmu/target-i386/cc_helper.o
  CC      aarch64-softmmu/hw/scsi/virtio-scsi-dataplane.o
  CC      aarch64-softmmu/hw/scsi/vhost-scsi.o
  CC      aarch64-softmmu/hw/sd/omap_mmc.o
  CC      aarch64-softmmu/hw/sd/pxa2xx_mmci.o
  CC      x86_64-softmmu/target-i386/int_helper.o
  CC      aarch64-softmmu/hw/ssi/omap_spi.o
  CC      aarch64-softmmu/hw/ssi/imx_spi.o
  CC      aarch64-softmmu/hw/timer/exynos4210_mct.o
  CC      aarch64-softmmu/hw/timer/exynos4210_pwm.o
  CC      aarch64-softmmu/hw/timer/exynos4210_rtc.o
  CC      aarch64-softmmu/hw/timer/omap_gptimer.o
  CC      aarch64-softmmu/hw/timer/omap_synctimer.o
  CC      aarch64-softmmu/hw/timer/pxa2xx_timer.o
  CC      x86_64-softmmu/target-i386/svm_helper.o
  CC      x86_64-softmmu/target-i386/smm_helper.o
  CC      x86_64-softmmu/target-i386/misc_helper.o
  CC      aarch64-softmmu/hw/timer/digic-timer.o
  CC      x86_64-softmmu/target-i386/mem_helper.o
  CC      aarch64-softmmu/hw/timer/allwinner-a10-pit.o
  CC      aarch64-softmmu/hw/usb/tusb6010.o
  CC      x86_64-softmmu/target-i386/seg_helper.o
  CC      aarch64-softmmu/hw/vfio/pci.o
  CC      aarch64-softmmu/hw/vfio/common.o
  CC      x86_64-softmmu/target-i386/mpx_helper.o
  CC      aarch64-softmmu/hw/vfio/pci-quirks.o
  CC      x86_64-softmmu/target-i386/gdbstub.o
  CC      x86_64-softmmu/target-i386/machine.o
  CC      x86_64-softmmu/target-i386/arch_memory_mapping.o
  CC      x86_64-softmmu/target-i386/arch_dump.o
  CC      x86_64-softmmu/target-i386/monitor.o
  CC      aarch64-softmmu/hw/vfio/platform.o
  CC      x86_64-softmmu/target-i386/kvm.o
  CC      aarch64-softmmu/hw/vfio/calxeda-xgmac.o
  CC      x86_64-softmmu/target-i386/hyperv.o
  CC      aarch64-softmmu/hw/vfio/amd-xgbe.o
  CC      aarch64-softmmu/hw/vfio/spapr.o
  GEN     trace/generated-helpers.c
  CC      x86_64-softmmu/trace/control-target.o
  CC      aarch64-softmmu/hw/virtio/virtio.o
  CC      aarch64-softmmu/hw/virtio/virtio-balloon.o
  CC      aarch64-softmmu/hw/virtio/vhost.o
  CC      aarch64-softmmu/hw/virtio/vhost-backend.o
  CC      aarch64-softmmu/hw/virtio/vhost-user.o
  CC      aarch64-softmmu/hw/virtio/vhost-vsock.o
  CC      aarch64-softmmu/hw/virtio/virtio-crypto.o
  CC      x86_64-softmmu/trace/generated-helpers.o
  CC      aarch64-softmmu/hw/virtio/virtio-crypto-pci.o
  CC      aarch64-softmmu/hw/arm/boot.o
  CC      aarch64-softmmu/hw/arm/collie.o
  CC      aarch64-softmmu/hw/arm/exynos4_boards.o
  CC      aarch64-softmmu/hw/arm/gumstix.o
  CC      aarch64-softmmu/hw/arm/highbank.o
  CC      aarch64-softmmu/hw/arm/digic_boards.o
  CC      aarch64-softmmu/hw/arm/integratorcp.o
  CC      aarch64-softmmu/hw/arm/mainstone.o
  CC      aarch64-softmmu/hw/arm/musicpal.o
  CC      aarch64-softmmu/hw/arm/nseries.o
  CC      aarch64-softmmu/hw/arm/palm.o
  CC      aarch64-softmmu/hw/arm/omap_sx1.o
  CC      aarch64-softmmu/hw/arm/realview.o
  CC      aarch64-softmmu/hw/arm/spitz.o
  CC      aarch64-softmmu/hw/arm/stellaris.o
  CC      aarch64-softmmu/hw/arm/tosa.o
  CC      aarch64-softmmu/hw/arm/versatilepb.o
  CC      aarch64-softmmu/hw/arm/vexpress.o
  CC      aarch64-softmmu/hw/arm/virt.o
  CC      aarch64-softmmu/hw/arm/xilinx_zynq.o
  CC      aarch64-softmmu/hw/arm/z2.o
  CC      aarch64-softmmu/hw/arm/virt-acpi-build.o
  CC      aarch64-softmmu/hw/arm/netduino2.o
  CC      aarch64-softmmu/hw/arm/sysbus-fdt.o
  CC      aarch64-softmmu/hw/arm/armv7m.o
  CC      aarch64-softmmu/hw/arm/exynos4210.o
  CC      aarch64-softmmu/hw/arm/pxa2xx.o
  CC      aarch64-softmmu/hw/arm/pxa2xx_gpio.o
  CC      aarch64-softmmu/hw/arm/pxa2xx_pic.o
  CC      aarch64-softmmu/hw/arm/digic.o
  CC      aarch64-softmmu/hw/arm/omap1.o
  CC      aarch64-softmmu/hw/arm/omap2.o
  CC      aarch64-softmmu/hw/arm/strongarm.o
  CC      aarch64-softmmu/hw/arm/allwinner-a10.o
  CC      aarch64-softmmu/hw/arm/cubieboard.o
  CC      aarch64-softmmu/hw/arm/bcm2835_peripherals.o
  CC      aarch64-softmmu/hw/arm/bcm2836.o
  CC      aarch64-softmmu/hw/arm/raspi.o
  CC      aarch64-softmmu/hw/arm/stm32f205_soc.o
  CC      aarch64-softmmu/hw/arm/xlnx-zynqmp.o
  CC      aarch64-softmmu/hw/arm/xlnx-ep108.o
  CC      aarch64-softmmu/hw/arm/fsl-imx25.o
  CC      aarch64-softmmu/hw/arm/imx25_pdk.o
  CC      aarch64-softmmu/hw/arm/fsl-imx31.o
  CC      aarch64-softmmu/hw/arm/kzm.o
  CC      aarch64-softmmu/hw/arm/fsl-imx6.o
  CC      aarch64-softmmu/hw/arm/sabrelite.o
  CC      aarch64-softmmu/hw/arm/aspeed_soc.o
  CC      aarch64-softmmu/hw/arm/aspeed.o
  CC      aarch64-softmmu/target-arm/arm-semi.o
  CC      aarch64-softmmu/target-arm/machine.o
  CC      aarch64-softmmu/target-arm/psci.o
  CC      aarch64-softmmu/target-arm/arch_dump.o
  CC      aarch64-softmmu/target-arm/monitor.o
  CC      aarch64-softmmu/target-arm/kvm-stub.o
  CC      aarch64-softmmu/target-arm/translate.o
  CC      aarch64-softmmu/target-arm/op_helper.o
  CC      aarch64-softmmu/target-arm/helper.o
  CC      aarch64-softmmu/target-arm/cpu.o
  CC      aarch64-softmmu/target-arm/neon_helper.o
  CC      aarch64-softmmu/target-arm/iwmmxt_helper.o
  CC      aarch64-softmmu/target-arm/gdbstub.o
  CC      aarch64-softmmu/target-arm/cpu64.o
  CC      aarch64-softmmu/target-arm/translate-a64.o
  CC      aarch64-softmmu/target-arm/helper-a64.o
  CC      aarch64-softmmu/target-arm/crypto_helper.o
  CC      aarch64-softmmu/target-arm/gdbstub64.o
  CC      aarch64-softmmu/target-arm/arm-powerctl.o
  GEN     trace/generated-helpers.c
  CC      aarch64-softmmu/trace/control-target.o
  CC      aarch64-softmmu/gdbstub-xml.o
  CC      aarch64-softmmu/trace/generated-helpers.o
  LINK    x86_64-softmmu/qemu-system-x86_64
/tmp/qemu-test/src/target-arm/translate-a64.c: In function ‘handle_shri_with_rndacc’:
/tmp/qemu-test/src/target-arm/translate-a64.c:6395: warning: ‘tcg_src_hi’ may be used uninitialized in this function
/tmp/qemu-test/src/target-arm/translate-a64.c: In function ‘disas_simd_scalar_two_reg_misc’:
/tmp/qemu-test/src/target-arm/translate-a64.c:8122: warning: ‘rmode’ may be used uninitialized in this function
  LINK    aarch64-softmmu/qemu-system-aarch64
  TEST    tests/qapi-schema/alternate-any.out
  TEST    tests/qapi-schema/alternate-array.out
  TEST    tests/qapi-schema/alternate-base.out
  TEST    tests/qapi-schema/alternate-clash.out
  TEST    tests/qapi-schema/alternate-conflict-dict.out
  TEST    tests/qapi-schema/alternate-conflict-string.out
  TEST    tests/qapi-schema/alternate-empty.out
  TEST    tests/qapi-schema/alternate-nested.out
  TEST    tests/qapi-schema/alternate-unknown.out
  TEST    tests/qapi-schema/args-alternate.out
  TEST    tests/qapi-schema/args-any.out
  TEST    tests/qapi-schema/args-array-empty.out
  TEST    tests/qapi-schema/args-array-unknown.out
  TEST    tests/qapi-schema/args-bad-boxed.out
  TEST    tests/qapi-schema/args-boxed-anon.out
  TEST    tests/qapi-schema/args-boxed-empty.out
  TEST    tests/qapi-schema/args-boxed-string.out
  TEST    tests/qapi-schema/args-int.out
  TEST    tests/qapi-schema/args-member-array-bad.out
  TEST    tests/qapi-schema/args-invalid.out
  TEST    tests/qapi-schema/args-member-case.out
  TEST    tests/qapi-schema/args-member-unknown.out
  TEST    tests/qapi-schema/args-name-clash.out
  TEST    tests/qapi-schema/args-union.out
  TEST    tests/qapi-schema/args-unknown.out
  TEST    tests/qapi-schema/bad-base.out
  TEST    tests/qapi-schema/bad-data.out
  TEST    tests/qapi-schema/bad-ident.out
  TEST    tests/qapi-schema/bad-type-bool.out
  TEST    tests/qapi-schema/bad-type-dict.out
  TEST    tests/qapi-schema/bad-type-int.out
  TEST    tests/qapi-schema/base-cycle-direct.out
  TEST    tests/qapi-schema/base-cycle-indirect.out
  TEST    tests/qapi-schema/command-int.out
  TEST    tests/qapi-schema/comments.out
  TEST    tests/qapi-schema/double-data.out
  TEST    tests/qapi-schema/double-type.out
  TEST    tests/qapi-schema/duplicate-key.out
  TEST    tests/qapi-schema/empty.out
  TEST    tests/qapi-schema/enum-bad-name.out
  TEST    tests/qapi-schema/enum-bad-prefix.out
  TEST    tests/qapi-schema/enum-clash-member.out
  TEST    tests/qapi-schema/enum-dict-member.out
  TEST    tests/qapi-schema/enum-member-case.out
  TEST    tests/qapi-schema/enum-int-member.out
  TEST    tests/qapi-schema/enum-wrong-data.out
  TEST    tests/qapi-schema/enum-missing-data.out
  TEST    tests/qapi-schema/escape-outside-string.out
  TEST    tests/qapi-schema/escape-too-big.out
  TEST    tests/qapi-schema/escape-too-short.out
  TEST    tests/qapi-schema/event-boxed-empty.out
  TEST    tests/qapi-schema/event-case.out
  TEST    tests/qapi-schema/event-nest-struct.out
  TEST    tests/qapi-schema/flat-union-array-branch.out
  TEST    tests/qapi-schema/flat-union-bad-base.out
  TEST    tests/qapi-schema/flat-union-bad-discriminator.out
  TEST    tests/qapi-schema/flat-union-clash-member.out
  TEST    tests/qapi-schema/flat-union-base-any.out
  TEST    tests/qapi-schema/flat-union-base-union.out
  TEST    tests/qapi-schema/flat-union-empty.out
  TEST    tests/qapi-schema/flat-union-incomplete-branch.out
  TEST    tests/qapi-schema/flat-union-inline.out
  TEST    tests/qapi-schema/flat-union-int-branch.out
  TEST    tests/qapi-schema/flat-union-invalid-branch-key.out
  TEST    tests/qapi-schema/flat-union-invalid-discriminator.out
  TEST    tests/qapi-schema/flat-union-no-base.out
  TEST    tests/qapi-schema/flat-union-optional-discriminator.out
  TEST    tests/qapi-schema/flat-union-string-discriminator.out
  TEST    tests/qapi-schema/ident-with-escape.out
  TEST    tests/qapi-schema/funny-char.out
  TEST    tests/qapi-schema/include-before-err.out
  TEST    tests/qapi-schema/include-cycle.out
  TEST    tests/qapi-schema/include-format-err.out
  TEST    tests/qapi-schema/include-nested-err.out
  TEST    tests/qapi-schema/include-no-file.out
  TEST    tests/qapi-schema/include-non-file.out
  TEST    tests/qapi-schema/include-relpath.out
  TEST    tests/qapi-schema/include-repetition.out
  TEST    tests/qapi-schema/include-self-cycle.out
  TEST    tests/qapi-schema/include-simple.out
  TEST    tests/qapi-schema/indented-expr.out
  TEST    tests/qapi-schema/leading-comma-list.out
  TEST    tests/qapi-schema/leading-comma-object.out
  TEST    tests/qapi-schema/missing-colon.out
  TEST    tests/qapi-schema/missing-comma-list.out
  TEST    tests/qapi-schema/missing-comma-object.out
  TEST    tests/qapi-schema/missing-type.out
  TEST    tests/qapi-schema/nested-struct-data.out
  TEST    tests/qapi-schema/non-objects.out
  TEST    tests/qapi-schema/qapi-schema-test.out
  TEST    tests/qapi-schema/quoted-structural-chars.out
  TEST    tests/qapi-schema/redefined-builtin.out
  TEST    tests/qapi-schema/redefined-event.out
  TEST    tests/qapi-schema/redefined-command.out
  TEST    tests/qapi-schema/redefined-type.out
  TEST    tests/qapi-schema/reserved-enum-q.out
  TEST    tests/qapi-schema/reserved-command-q.out
  TEST    tests/qapi-schema/reserved-member-has.out
  TEST    tests/qapi-schema/reserved-member-q.out
  TEST    tests/qapi-schema/reserved-member-underscore.out
  TEST    tests/qapi-schema/reserved-member-u.out
  TEST    tests/qapi-schema/reserved-type-kind.out
  TEST    tests/qapi-schema/reserved-type-list.out
  TEST    tests/qapi-schema/returns-alternate.out
  TEST    tests/qapi-schema/returns-array-bad.out
  TEST    tests/qapi-schema/returns-dict.out
  TEST    tests/qapi-schema/returns-unknown.out
  TEST    tests/qapi-schema/returns-whitelist.out
  TEST    tests/qapi-schema/struct-base-clash-deep.out
  TEST    tests/qapi-schema/struct-base-clash.out
  TEST    tests/qapi-schema/struct-data-invalid.out
  TEST    tests/qapi-schema/struct-member-invalid.out
  TEST    tests/qapi-schema/trailing-comma-list.out
  TEST    tests/qapi-schema/trailing-comma-object.out
  TEST    tests/qapi-schema/type-bypass-bad-gen.out
  TEST    tests/qapi-schema/unclosed-list.out
  TEST    tests/qapi-schema/unclosed-object.out
  TEST    tests/qapi-schema/unclosed-string.out
  TEST    tests/qapi-schema/unicode-str.out
  TEST    tests/qapi-schema/union-base-no-discriminator.out
  TEST    tests/qapi-schema/union-branch-case.out
  TEST    tests/qapi-schema/union-clash-branches.out
  TEST    tests/qapi-schema/union-empty.out
  TEST    tests/qapi-schema/union-invalid-base.out
  TEST    tests/qapi-schema/union-optional-branch.out
  TEST    tests/qapi-schema/union-unknown.out
  TEST    tests/qapi-schema/unknown-escape.out
  TEST    tests/qapi-schema/unknown-expr-key.out
  CC      tests/check-qdict.o
  CC      tests/test-char.o
  CC      tests/check-qfloat.o
  CC      tests/check-qint.o
  CC      tests/check-qstring.o
  CC      tests/check-qlist.o
  CC      tests/check-qnull.o
  CC      tests/check-qjson.o
  GEN     tests/test-qapi-visit.c
  CC      tests/test-qobject-output-visitor.o
  GEN     tests/test-qapi-types.c
  GEN     tests/test-qapi-event.c
  GEN     tests/test-qmp-introspect.c
  CC      tests/test-clone-visitor.o
  CC      tests/test-qobject-input-visitor.o
  CC      tests/test-qobject-input-strict.o
  CC      tests/test-qmp-commands.o
  GEN     tests/test-qmp-marshal.c
  CC      tests/test-string-input-visitor.o
  CC      tests/test-string-output-visitor.o
  CC      tests/test-qmp-event.o
  CC      tests/test-opts-visitor.o
  CC      tests/test-coroutine.o
  CC      tests/test-visitor-serialization.o
  CC      tests/test-iov.o
  CC      tests/test-throttle.o
  CC      tests/test-aio.o
  CC      tests/test-thread-pool.o
  CC      tests/test-hbitmap.o
  CC      tests/test-blockjob.o
  CC      tests/test-blockjob-txn.o
  CC      tests/test-x86-cpuid.o
  CC      tests/test-vmstate.o
  CC      tests/test-xbzrle.o
  CC      tests/test-int128.o
  CC      tests/test-mul64.o
  CC      tests/test-cutils.o
  CC      tests/rcutorture.o
  CC      tests/test-rcu-list.o
  CC      tests/test-qdist.o
/tmp/qemu-test/src/tests/test-int128.c:180: warning: ‘__noclone__’ attribute directive ignored
  CC      tests/test-qht.o
  CC      tests/qht-bench.o
  CC      tests/test-qht-par.o
  CC      tests/test-bitops.o
  CC      tests/check-qom-interface.o
  CC      tests/test-qemu-opts.o
  CC      tests/check-qom-proplist.o
  CC      tests/test-write-threshold.o
  CC      tests/test-crypto-hash.o
  CC      tests/test-crypto-cipher.o
  CC      tests/test-crypto-secret.o
  CC      tests/test-qga.o
  CC      tests/libqtest.o
  CC      tests/test-io-task.o
  CC      tests/test-timed-average.o
  CC      tests/test-io-channel-socket.o
  CC      tests/io-channel-helpers.o
  CC      tests/test-io-channel-file.o
  CC      tests/test-io-channel-command.o
  CC      tests/test-io-channel-buffer.o
  CC      tests/test-base64.o
  CC      tests/test-crypto-ivgen.o
  CC      tests/test-crypto-afsplit.o
  CC      tests/test-crypto-xts.o
  CC      tests/test-crypto-block.o
  CC      tests/test-logging.o
  CC      tests/test-replication.o
  CC      tests/test-uuid.o
  CC      tests/test-bufferiszero.o
  CC      tests/ptimer-test.o
  CC      tests/ptimer-test-stubs.o
  CC      tests/vhost-user-test.o
  CC      tests/libqos/fw_cfg.o
  CC      tests/libqos/pci.o
  CC      tests/libqos/malloc.o
  CC      tests/libqos/i2c.o
  CC      tests/libqos/libqos.o
  CC      tests/libqos/malloc-spapr.o
  CC      tests/libqos/libqos-spapr.o
  CC      tests/libqos/rtas.o
  CC      tests/libqos/pci-spapr.o
  CC      tests/libqos/pci-pc.o
  CC      tests/libqos/malloc-pc.o
  CC      tests/libqos/libqos-pc.o
  CC      tests/libqos/ahci.o
  CC      tests/libqos/virtio.o
  CC      tests/libqos/virtio-pci.o
  CC      tests/libqos/virtio-mmio.o
  CC      tests/libqos/malloc-generic.o
  CC      tests/ide-test.o
  CC      tests/endianness-test.o
  CC      tests/fdc-test.o
  CC      tests/hd-geo-test.o
  CC      tests/ahci-test.o
  CC      tests/boot-order-test.o
  CC      tests/boot-sector.o
  CC      tests/boot-serial-test.o
  CC      tests/bios-tables-test.o
  CC      tests/pxe-test.o
  CC      tests/ipmi-kcs-test.o
  CC      tests/rtc-test.o
  CC      tests/ipmi-bt-test.o
  CC      tests/i440fx-test.o
  CC      tests/fw_cfg-test.o
  CC      tests/drive_del-test.o
/tmp/qemu-test/src/tests/ide-test.c: In function ‘cdrom_pio_impl’:
/tmp/qemu-test/src/tests/ide-test.c:791: warning: ignoring return value of ‘fwrite’, declared with attribute warn_unused_result
/tmp/qemu-test/src/tests/ide-test.c: In function ‘test_cdrom_dma’:
/tmp/qemu-test/src/tests/ide-test.c:886: warning: ignoring return value of ‘fwrite’, declared with attribute warn_unused_result
  CC      tests/wdt_ib700-test.o
  CC      tests/tco-test.o
  CC      tests/e1000-test.o
  CC      tests/e1000e-test.o
  CC      tests/rtl8139-test.o
  CC      tests/pcnet-test.o
  CC      tests/eepro100-test.o
  CC      tests/ne2000-test.o
  CC      tests/nvme-test.o
  CC      tests/ac97-test.o
  CC      tests/virtio-net-test.o
  CC      tests/es1370-test.o
  CC      tests/virtio-balloon-test.o
  CC      tests/virtio-blk-test.o
  CC      tests/virtio-rng-test.o
  CC      tests/virtio-scsi-test.o
  CC      tests/virtio-serial-test.o
  CC      tests/tpci200-test.o
  CC      tests/virtio-console-test.o
  CC      tests/ipoctal232-test.o
  CC      tests/display-vga-test.o
  CC      tests/intel-hda-test.o
  CC      tests/ivshmem-test.o
  CC      tests/vmxnet3-test.o
  CC      tests/pvpanic-test.o
  CC      tests/i82801b11-test.o
  CC      tests/ioh3420-test.o
  CC      tests/usb-hcd-ohci-test.o
  CC      tests/libqos/usb.o
  CC      tests/usb-hcd-uhci-test.o
  CC      tests/usb-hcd-ehci-test.o
  CC      tests/pc-cpu-test.o
  CC      tests/usb-hcd-xhci-test.o
  CC      tests/q35-test.o
  CC      tests/test-netfilter.o
  CC      tests/test-filter-mirror.o
  CC      tests/test-filter-redirector.o
  CC      tests/test-x86-cpuid-compat.o
  CC      tests/postcopy-test.o
  CC      tests/device-introspect-test.o
  CC      tests/qom-test.o
  LINK    tests/check-qdict
  LINK    tests/test-char
  LINK    tests/check-qfloat
  LINK    tests/check-qint
  LINK    tests/check-qstring
  LINK    tests/check-qlist
  LINK    tests/check-qnull
  LINK    tests/check-qjson
  CC      tests/test-qapi-visit.o
  CC      tests/test-qapi-types.o
  CC      tests/test-qapi-event.o
  CC      tests/test-qmp-introspect.o
  CC      tests/test-qmp-marshal.o
  LINK    tests/test-coroutine
  LINK    tests/test-visitor-serialization
  LINK    tests/test-iov
  LINK    tests/test-aio
  LINK    tests/test-throttle
  LINK    tests/test-thread-pool
  LINK    tests/test-hbitmap
  LINK    tests/test-blockjob
  LINK    tests/test-blockjob-txn
  LINK    tests/test-x86-cpuid
  LINK    tests/test-xbzrle
  LINK    tests/test-vmstate
  LINK    tests/test-cutils
  LINK    tests/test-mul64
  LINK    tests/test-int128
  LINK    tests/rcutorture
  LINK    tests/test-rcu-list
  LINK    tests/test-qdist
  LINK    tests/test-qht
  LINK    tests/qht-bench
  LINK    tests/test-bitops
  LINK    tests/check-qom-interface
  LINK    tests/check-qom-proplist
  LINK    tests/test-qemu-opts
  LINK    tests/test-write-threshold
  LINK    tests/test-crypto-hash
  LINK    tests/test-crypto-cipher
  LINK    tests/test-crypto-secret
  LINK    tests/test-qga
  LINK    tests/test-timed-average
  LINK    tests/test-io-task
  LINK    tests/test-io-channel-socket
  LINK    tests/test-io-channel-file
  LINK    tests/test-io-channel-command
  LINK    tests/test-io-channel-buffer
  LINK    tests/test-base64
  LINK    tests/test-crypto-ivgen
  LINK    tests/test-crypto-afsplit
  LINK    tests/test-crypto-xts
  LINK    tests/test-crypto-block
  LINK    tests/test-logging
  LINK    tests/test-replication
  LINK    tests/test-bufferiszero
  LINK    tests/test-uuid
  LINK    tests/ptimer-test
  LINK    tests/vhost-user-test
  LINK    tests/endianness-test
  LINK    tests/fdc-test
  LINK    tests/ide-test
  LINK    tests/ahci-test
  LINK    tests/hd-geo-test
  LINK    tests/boot-order-test
  LINK    tests/bios-tables-test
  LINK    tests/boot-serial-test
  LINK    tests/pxe-test
  LINK    tests/rtc-test
  LINK    tests/ipmi-kcs-test
  LINK    tests/ipmi-bt-test
  LINK    tests/i440fx-test
  LINK    tests/fw_cfg-test
  LINK    tests/drive_del-test
  LINK    tests/wdt_ib700-test
  LINK    tests/tco-test
  LINK    tests/e1000-test
  LINK    tests/e1000e-test
  LINK    tests/rtl8139-test
  LINK    tests/pcnet-test
  LINK    tests/eepro100-test
  LINK    tests/ne2000-test
  LINK    tests/nvme-test
  LINK    tests/ac97-test
  LINK    tests/es1370-test
  LINK    tests/virtio-net-test
  LINK    tests/virtio-balloon-test
  LINK    tests/virtio-blk-test
  LINK    tests/virtio-rng-test
  LINK    tests/virtio-scsi-test
  LINK    tests/virtio-serial-test
  LINK    tests/virtio-console-test
  LINK    tests/tpci200-test
  LINK    tests/ipoctal232-test
  LINK    tests/display-vga-test
  LINK    tests/intel-hda-test
  LINK    tests/ivshmem-test
  LINK    tests/vmxnet3-test
  LINK    tests/pvpanic-test
  LINK    tests/i82801b11-test
  LINK    tests/ioh3420-test
  LINK    tests/usb-hcd-ohci-test
  LINK    tests/usb-hcd-uhci-test
  LINK    tests/usb-hcd-ehci-test
  LINK    tests/usb-hcd-xhci-test
  LINK    tests/pc-cpu-test
  LINK    tests/q35-test
  LINK    tests/test-netfilter
  LINK    tests/test-filter-mirror
  LINK    tests/test-filter-redirector
  LINK    tests/postcopy-test
  LINK    tests/test-x86-cpuid-compat
  LINK    tests/device-introspect-test
  LINK    tests/qom-test
  GTESTER tests/check-qdict
  GTESTER tests/test-char
  GTESTER tests/check-qfloat
  GTESTER tests/check-qint
  GTESTER tests/check-qstring
  GTESTER tests/check-qlist
  GTESTER tests/check-qnull
  GTESTER tests/check-qjson
  LINK    tests/test-qobject-output-visitor
  LINK    tests/test-clone-visitor
  LINK    tests/test-qobject-input-visitor
  LINK    tests/test-qobject-input-strict
  LINK    tests/test-qmp-commands
  LINK    tests/test-string-input-visitor
  LINK    tests/test-string-output-visitor
  LINK    tests/test-qmp-event
  LINK    tests/test-opts-visitor
  GTESTER tests/test-throttle
  GTESTER tests/test-coroutine
  GTESTER tests/test-visitor-serialization
  GTESTER tests/test-iov
  GTESTER tests/test-aio
  GTESTER tests/test-thread-pool
  GTESTER tests/test-hbitmap
  GTESTER tests/test-blockjob-txn
  GTESTER tests/test-blockjob
  GTESTER tests/test-x86-cpuid
  GTESTER tests/test-xbzrle
  GTESTER tests/test-vmstate
  GTESTER tests/test-cutils
  GTESTER tests/test-mul64
  GTESTER tests/test-int128
Failed to load simple/primitive:b_1
Failed to load simple/primitive:i64_2
Failed to load simple/primitive:i32_1
Failed to load simple/primitive:i32_1
  GTESTER tests/rcutorture
  GTESTER tests/test-rcu-list
  GTESTER tests/test-qdist
  GTESTER tests/test-qht
  LINK    tests/test-qht-par
  GTESTER tests/test-bitops
  GTESTER tests/check-qom-interface
  GTESTER tests/check-qom-proplist
  GTESTER tests/test-qemu-opts
  GTESTER tests/test-write-threshold
  GTESTER tests/test-crypto-hash
  GTESTER tests/test-crypto-cipher
  GTESTER tests/test-qga
  GTESTER tests/test-crypto-secret
  GTESTER tests/test-timed-average
  GTESTER tests/test-io-task
  GTESTER tests/test-io-channel-socket
  GTESTER tests/test-io-channel-command
  GTESTER tests/test-io-channel-file
  GTESTER tests/test-io-channel-buffer
  GTESTER tests/test-base64
  GTESTER tests/test-crypto-ivgen
  GTESTER tests/test-crypto-afsplit
  GTESTER tests/test-crypto-xts
  GTESTER tests/test-crypto-block
  GTESTER tests/test-logging
  GTESTER tests/test-replication
  GTESTER tests/test-bufferiszero
  GTESTER tests/test-uuid
  GTESTER tests/ptimer-test
  GTESTER check-qtest-x86_64
  GTESTER check-qtest-aarch64
  GTESTER tests/test-qobject-output-visitor
  GTESTER tests/test-clone-visitor
  GTESTER tests/test-qobject-input-visitor
  GTESTER tests/test-qobject-input-strict
  GTESTER tests/test-qmp-commands
  GTESTER tests/test-string-input-visitor
  GTESTER tests/test-qmp-event
  GTESTER tests/test-opts-visitor
  GTESTER tests/test-qht-par
  GTESTER tests/test-string-output-visitor
ftruncate: Permission denied
ftruncate: Permission denied
ftruncate: Permission denied
**
ERROR:/tmp/qemu-test/src/tests/vhost-user-test.c:668:test_migrate: assertion failed: (qdict_haskey(rsp, "return"))
GTester: last random seed: R02S572bd0e04eb0104cb64103e6a8c60549
ftruncate: Permission denied
ftruncate: Permission denied
**
ERROR:/tmp/qemu-test/src/tests/test-thread-pool.c:207:do_test_cancel: assertion failed (data[i].n == 2): (5 == 2)
GTester: last random seed: R02S0e2846dcad47dd06f6ef255794dbbfc6
make: *** [check-tests/test-thread-pool] Error 1
make: *** Waiting for unfinished jobs....
Could not access KVM kernel module: No such file or directory
failed to initialize KVM: No such file or directory
Back to tcg accelerator.
Could not access KVM kernel module: No such file or directory
failed to initialize KVM: No such file or directory
Back to tcg accelerator.
Could not access KVM kernel module: No such file or directory
failed to initialize KVM: No such file or directory
Back to tcg accelerator.
Could not access KVM kernel module: No such file or directory
failed to initialize KVM: No such file or directory
Back to tcg accelerator.
Could not access KVM kernel module: No such file or directory
failed to initialize KVM: No such file or directory
Back to tcg accelerator.
Could not access KVM kernel module: No such file or directory
failed to initialize KVM: No such file or directory
Back to tcg accelerator.
Could not access KVM kernel module: No such file or directory
failed to initialize KVM: No such file or directory
Back to tcg accelerator.
Could not access KVM kernel module: No such file or directory
failed to initialize KVM: No such file or directory
Back to tcg accelerator.
Could not access KVM kernel module: No such file or directory
failed to initialize KVM: No such file or directory
Back to tcg accelerator.
Could not access KVM kernel module: No such file or directory
failed to initialize KVM: No such file or directory
Back to tcg accelerator.
make[1]: *** [docker-run] Error 2
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-5tzxa5rp/src'
make: *** [docker-run-test-quick@centos6] Error 2
=== OUTPUT END ===

Test command exited with code: 2


---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@freelists.org