[PATCH RFC 0/3] virtio fix up started checks

[PATCH RFC 0/3] virtio fix up started checks

[Michael S. Tsirkin]

This is an attempt to fix up device started checks.
Unfortunately this causes failures in CI
and I could not figure it out.

The simplest way to test is to set QEMU_CI to 2
on gitlab, then push there.

Alternatively, push to gitlab, then
create pipeline while setting QEMU_CI to 1,
then run amd64-fedora-container and then clang-system -
that slows things down enough to make the failures
trigger.

See: https://gitlab.com/mstredhat/qemu/-/jobs/3279537476


Alex, Viresh, need your help here. Thanks!

Alex, pls note that same failures are triggered by your RFC - if we know the
root cause we can discuss solutions. So if you prefer pls go ahead and
debug that. Thanks!

Michael S. Tsirkin (3):
  virtio: distinguish between started and running
  gpio: use virtio_device_running
  virtio: revert changes to virtio_device_started

 include/hw/virtio/virtio.h   | 7 ++++++-
 hw/virtio/vhost-user-fs.c    | 2 +-
 hw/virtio/vhost-user-gpio.c  | 4 ++--
 hw/virtio/vhost-user-i2c.c   | 4 ++--
 hw/virtio/vhost-user-rng.c   | 4 ++--
 hw/virtio/vhost-user-vsock.c | 2 +-
 hw/virtio/vhost-vsock.c      | 2 +-
 7 files changed, 15 insertions(+), 10 deletions(-)

-- 
MST

[PATCH RFC 1/3] virtio: distinguish between started and running

[Michael S. Tsirkin]

virtio core often needs to know whether device is started, this is what
virtio_device_started already did. However, backends want to know
whether virtio is actually running which also depends on whether vm is
running. To address this we moved the check to virtio_device_started,
but this changes virtio core behavior which wasn't intentional.  Let's
add a new API virtio_device_running just for the backends.
Follow up patch will revert the change to virtio_device_started.

Further, the old API was actually ignoring vm running state
most of the time (when use_started property is set),
The new API takes vm running state into account properly.

Fixes: 9f6bcfd99f ("hw/virtio: move vm_running check to virtio_device_started")
Cc: "Alex Bennée" <alex.bennee@linaro.org>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/hw/virtio/virtio.h   | 9 +++++++++
 hw/virtio/vhost-user-fs.c    | 2 +-
 hw/virtio/vhost-user-i2c.c   | 4 ++--
 hw/virtio/vhost-user-rng.c   | 4 ++--
 hw/virtio/vhost-user-vsock.c | 2 +-
 hw/virtio/vhost-vsock.c      | 2 +-
 6 files changed, 16 insertions(+), 7 deletions(-)

diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index 1423dba379..634c24513f 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -402,6 +402,15 @@ static inline bool virtio_device_started(VirtIODevice *vdev, uint8_t status)
     return status & VIRTIO_CONFIG_S_DRIVER_OK;
 }
 
+static inline bool virtio_device_running(VirtIODevice *vdev, uint8_t status)
+{
+    if (!vdev->vm_running) {
+        return false;
+    }
+
+    return virtio_device_started(vdev, status);
+}
+
 static inline void virtio_set_started(VirtIODevice *vdev, bool started)
 {
     if (started) {
diff --git a/hw/virtio/vhost-user-fs.c b/hw/virtio/vhost-user-fs.c
index ad0f91c607..174c968179 100644
--- a/hw/virtio/vhost-user-fs.c
+++ b/hw/virtio/vhost-user-fs.c
@@ -123,7 +123,7 @@ static void vuf_stop(VirtIODevice *vdev)
 static void vuf_set_status(VirtIODevice *vdev, uint8_t status)
 {
     VHostUserFS *fs = VHOST_USER_FS(vdev);
-    bool should_start = virtio_device_started(vdev, status);
+    bool should_start = virtio_device_running(vdev, status);
 
     if (vhost_dev_is_started(&fs->vhost_dev) == should_start) {
         return;
diff --git a/hw/virtio/vhost-user-i2c.c b/hw/virtio/vhost-user-i2c.c
index bc58b6c0d1..cf4fd26cf4 100644
--- a/hw/virtio/vhost-user-i2c.c
+++ b/hw/virtio/vhost-user-i2c.c
@@ -93,7 +93,7 @@ static void vu_i2c_stop(VirtIODevice *vdev)
 static void vu_i2c_set_status(VirtIODevice *vdev, uint8_t status)
 {
     VHostUserI2C *i2c = VHOST_USER_I2C(vdev);
-    bool should_start = virtio_device_started(vdev, status);
+    bool should_start = virtio_device_running(vdev, status);
 
     if (vhost_dev_is_started(&i2c->vhost_dev) == should_start) {
         return;
@@ -157,7 +157,7 @@ static int vu_i2c_connect(DeviceState *dev)
     i2c->connected = true;
 
     /* restore vhost state */
-    if (virtio_device_started(vdev, vdev->status)) {
+    if (virtio_device_running(vdev, vdev->status)) {
         vu_i2c_start(vdev);
     }
 
diff --git a/hw/virtio/vhost-user-rng.c b/hw/virtio/vhost-user-rng.c
index bc1f36c5ac..ba548dc83c 100644
--- a/hw/virtio/vhost-user-rng.c
+++ b/hw/virtio/vhost-user-rng.c
@@ -90,7 +90,7 @@ static void vu_rng_stop(VirtIODevice *vdev)
 static void vu_rng_set_status(VirtIODevice *vdev, uint8_t status)
 {
     VHostUserRNG *rng = VHOST_USER_RNG(vdev);
-    bool should_start = virtio_device_started(vdev, status);
+    bool should_start = virtio_device_running(vdev, status);
 
     if (vhost_dev_is_started(&rng->vhost_dev) == should_start) {
         return;
@@ -144,7 +144,7 @@ static void vu_rng_connect(DeviceState *dev)
     rng->connected = true;
 
     /* restore vhost state */
-    if (virtio_device_started(vdev, vdev->status)) {
+    if (virtio_device_running(vdev, vdev->status)) {
         vu_rng_start(vdev);
     }
 }
diff --git a/hw/virtio/vhost-user-vsock.c b/hw/virtio/vhost-user-vsock.c
index 7b67e29d83..30228eaa21 100644
--- a/hw/virtio/vhost-user-vsock.c
+++ b/hw/virtio/vhost-user-vsock.c
@@ -55,7 +55,7 @@ const VhostDevConfigOps vsock_ops = {
 static void vuv_set_status(VirtIODevice *vdev, uint8_t status)
 {
     VHostVSockCommon *vvc = VHOST_VSOCK_COMMON(vdev);
-    bool should_start = virtio_device_started(vdev, status);
+    bool should_start = virtio_device_running(vdev, status);
 
     if (vhost_dev_is_started(&vvc->vhost_dev) == should_start) {
         return;
diff --git a/hw/virtio/vhost-vsock.c b/hw/virtio/vhost-vsock.c
index 7dc3c73931..1c768ee196 100644
--- a/hw/virtio/vhost-vsock.c
+++ b/hw/virtio/vhost-vsock.c
@@ -70,7 +70,7 @@ static int vhost_vsock_set_running(VirtIODevice *vdev, int start)
 static void vhost_vsock_set_status(VirtIODevice *vdev, uint8_t status)
 {
     VHostVSockCommon *vvc = VHOST_VSOCK_COMMON(vdev);
-    bool should_start = virtio_device_started(vdev, status);
+    bool should_start = virtio_device_running(vdev, status);
     int ret;
 
     if (vhost_dev_is_started(&vvc->vhost_dev) == should_start) {
-- 
MST

[PATCH RFC 2/3] gpio: use virtio_device_running

[Michael S. Tsirkin]

Same as other vhost-user devices, vhost-user-gpio cares whether
device is running not whether frontend is started.
Switch to that.

Fixes: 27ba7b027f ("hw/virtio: add boilerplate for vhost-user-gpio device")
Cc: "Viresh Kumar" <viresh.kumar@linaro.org>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/virtio/vhost-user-gpio.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/hw/virtio/vhost-user-gpio.c b/hw/virtio/vhost-user-gpio.c
index 8b40fe450c..f34ee59b6e 100644
--- a/hw/virtio/vhost-user-gpio.c
+++ b/hw/virtio/vhost-user-gpio.c
@@ -152,7 +152,7 @@ static void vu_gpio_stop(VirtIODevice *vdev)
 static void vu_gpio_set_status(VirtIODevice *vdev, uint8_t status)
 {
     VHostUserGPIO *gpio = VHOST_USER_GPIO(vdev);
-    bool should_start = virtio_device_started(vdev, status);
+    bool should_start = virtio_device_running(vdev, status);
 
     trace_virtio_gpio_set_status(status);
 
@@ -228,7 +228,7 @@ static int vu_gpio_connect(DeviceState *dev, Error **errp)
     }
 
     /* restore vhost state */
-    if (virtio_device_started(vdev, vdev->status)) {
+    if (virtio_device_running(vdev, vdev->status)) {
         vu_gpio_start(vdev);
     }
 
-- 
MST

[PATCH RFC 3/3] virtio: revert changes to virtio_device_started

[Michael S. Tsirkin]

virtio core often needs to know whether device is started, this is what
virtio_device_started already did. However, backends want to know
whether virtio is actually running which also depends on whether vm is
running. To address this we moved the check to virtio_device_started,
but this changes virtio core behavior which wasn't intentional.
Now that backends use the new virtio_device_running API,
revert the change to virtio_device_started.

Fixes: 9f6bcfd99f ("hw/virtio: move vm_running check to virtio_device_started")
Cc: "Alex Bennée" <alex.bennee@linaro.org>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/hw/virtio/virtio.h | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index 634c24513f..de8d78af12 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -395,10 +395,6 @@ static inline bool virtio_device_started(VirtIODevice *vdev, uint8_t status)
         return vdev->started;
     }
 
-    if (!vdev->vm_running) {
-        return false;
-    }
-
     return status & VIRTIO_CONFIG_S_DRIVER_OK;
 }
 
-- 
MST

Re: [PATCH RFC 0/3] virtio fix up started checks

[Michael S. Tsirkin]

On Sat, Nov 05, 2022 at 02:16:29PM -0400, Michael S. Tsirkin wrote:
> This is an attempt to fix up device started checks.
> Unfortunately this causes failures in CI
> and I could not figure it out.
> 
> The simplest way to test is to set QEMU_CI to 2
> on gitlab, then push there.
> 
> Alternatively, push to gitlab, then
> create pipeline while setting QEMU_CI to 1,
> then run amd64-fedora-container and then clang-system -
> that slows things down enough to make the failures
> trigger.
> 
> See: https://gitlab.com/mstredhat/qemu/-/jobs/3279537476

And here is a backtrace:

――――――――――――――――――――――――――――――――――――― ✀  ―――――――――――――――――――――――――――――――――――――
stderr:
qemu-system-arm: Failed to write msg. Wrote -1 instead of 20.
qemu-system-arm: vhost VQ 0 ring restore failed: -22: Invalid argument (22)
qemu-system-arm: Failed to set msg fds.
qemu-system-arm: vhost VQ 1 ring restore failed: -22: Invalid argument (22)
qemu-system-arm: -chardev socket,id=chr-reconnect,path=/tmp/vhost-test-PIIDV1/reconnect.sock,server=on: info: QEMU waiting for connection on: disconnected:unix:/tmp/vhost-test-PIIDV1/reconnect.sock,server=on
qemu-system-arm: Failed to write msg. Wrote -1 instead of 20.
qemu-system-arm: vhost VQ 0 ring restore failed: -22: Invalid argument (22)
qemu-system-arm: Failed to set msg fds.
qemu-system-arm: vhost VQ 1 ring restore failed: -22: Invalid argument (22)
qemu-system-arm: -chardev socket,id=chr-connect-fail,path=/tmp/vhost-test-U7IGV1/connect-fail.sock,server=on: info: QEMU waiting for connection on: disconnected:unix:/tmp/vhost-test-U7IGV1/connect-fail.sock,server=on
qemu-system-arm: -netdev vhost-user,id=hs0,chardev=chr-connect-fail,vhostforce=on: Failed to read msg header. Read 0 instead of 12. Original request 1.
qemu-system-arm: -netdev vhost-user,id=hs0,chardev=chr-connect-fail,vhostforce=on: vhost_backend_init failed: Protocol error
qemu-system-arm: -netdev vhost-user,id=hs0,chardev=chr-connect-fail,vhostforce=on: failed to init vhost_net for queue 0
qemu-system-arm: -netdev vhost-user,id=hs0,chardev=chr-connect-fail,vhostforce=on: info: QEMU waiting for connection on: disconnected:unix:/tmp/vhost-test-U7IGV1/connect-fail.sock,server=on
qemu-system-arm: Failed to write msg. Wrote -1 instead of 20.
qemu-system-arm: vhost VQ 0 ring restore failed: -22: Invalid argument (22)
qemu-system-arm: Failed to set msg fds.
qemu-system-arm: vhost VQ 1 ring restore failed: -22: Invalid argument (22)
qemu-system-arm: -chardev socket,id=chr-flags-mismatch,path=/tmp/vhost-test-BUYEV1/flags-mismatch.sock,server=on: info: QEMU waiting for connection on: disconnected:unix:/tmp/vhost-test-BUYEV1/flags-mismatch.sock,server=on
qemu-system-arm: Failed to write msg. Wrote -1 instead of 52.
qemu-system-arm: vhost_set_mem_table failed: Invalid argument (22)
qemu-system-arm: unable to start vhost net: 22: falling back on userspace virtio
vhost lacks feature mask 0x40000000 for backend
qemu-system-arm: failed to init vhost_net for queue 0
qemu-system-arm: Failed to set msg fds.
qemu-system-arm: vhost VQ 0 ring restore failed: -22: Invalid argument (22)
qemu-system-arm: Failed to set msg fds.
qemu-system-arm: vhost VQ 1 ring restore failed: -22: Invalid argument (22)
qemu-system-arm: Failed to write msg. Wrote -1 instead of 20.
qemu-system-arm: vhost VQ 0 ring restore failed: -22: Invalid argument (22)
qemu-system-arm: Failed to set msg fds.
qemu-system-arm: vhost VQ 1 ring restore failed: -22: Invalid argument (22)
qemu-system-arm: Failed to set msg fds.
qemu-system-arm: vhost VQ 2 ring restore failed: -22: Invalid argument (22)
qemu-system-arm: Failed to set msg fds.
qemu-system-arm: vhost VQ 3 ring restore failed: -22: Invalid argument (22)
qemu-system-arm: Failed to write msg. Wrote -1 instead of 20.
qemu-system-arm: vhost VQ 0 ring restore failed: -22: Invalid argument (22)
qemu-system-arm: Failed to set msg fds.
qemu-system-arm: vhost VQ 1 ring restore failed: -22: Invalid argument (22)
qemu-system-arm: Failed to write msg. Wrote -1 instead of 20.
qemu-system-arm: vhost VQ 0 ring restore failed: -22: Invalid argument (22)
qemu-system-arm: Failed to set msg fds.
qemu-system-arm: vhost VQ 1 ring restore failed: -22: Invalid argument (22)
qemu-system-arm: Failed to set msg fds.
qemu-system-arm: vhost VQ 0 ring restore failed: -22: Invalid argument (22)
qemu-system-arm: Failed to set msg fds.
qemu-system-arm: vhost VQ 1 ring restore failed: -22: Invalid argument (22)
qemu-system-arm: Failed to set msg fds.
qemu-system-arm: vhost_set_vring_call failed: Invalid argument (22)
qemu-system-arm: Failed to set msg fds.
qemu-system-arm: vhost_set_vring_call failed: Invalid argument (22)
qemu-system-arm: Failed to write msg. Wrote -1 instead of 20.
qemu-system-arm: vhost VQ 0 ring restore failed: -5: Input/output error (5)
UndefinedBehaviorSanitizer:DEADLYSIGNAL
==8747==ERROR: UndefinedBehaviorSanitizer: SEGV on unknown address 0x0000000000fc (pc 0x55b8ada1276d bp 0x000000000007 sp 0x7ffd127cf5f0 T8747)
==8747==The signal is caused by a WRITE memory access.
==8747==Hint: address points to the zero page.
    #0 0x55b8ada1276d in virtio_bus_release_ioeventfd /builds/mstredhat/qemu/build/../hw/virtio/virtio-bus.c:216:30
    #1 0x55b8ade97b51 in vu_gpio_set_status /builds/mstredhat/qemu/build/../hw/virtio/vhost-user-gpio.c:172:9
    #2 0x55b8ade593f9 in virtio_set_status /builds/mstredhat/qemu/build/../hw/virtio/virtio.c:2442:9
    #3 0x55b8ada4d3d7 in vm_state_notify /builds/mstredhat/qemu/build/../softmmu/runstate.c:334:13
    #4 0x55b8ada4459a in do_vm_stop /builds/mstredhat/qemu/build/../softmmu/cpus.c:262:9
    #5 0x55b8ada4e2db in qemu_cleanup /builds/mstredhat/qemu/build/../softmmu/runstate.c:827:5
    #6 0x55b8ad6054fc in qemu_default_main /builds/mstredhat/qemu/build/../softmmu/main.c:38:5
    #7 0x7f3da8999eaf in __libc_start_call_main (/lib64/libc.so.6+0x3feaf)
    #8 0x7f3da8999f5f in __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x3ff5f)
    #9 0x55b8ad5dc094 in _start (/builds/mstredhat/qemu/build/qemu-system-arm+0xc17094)
UndefinedBehaviorSanitizer can not provide additional info.
SUMMARY: UndefinedBehaviorSanitizer: SEGV /builds/mstredhat/qemu/build/../hw/virtio/virtio-bus.c:216:30 in virtio_bus_release_ioeventfd
==8747==ABORTING
../tests/qtest/libqtest.c:179: kill_qemu() tried to terminate QEMU process but encountered exit status 1 (expected 0)
**
ERROR:../tests/qtest/qos-test.c:191:subprocess_run_one_test: child process (/arm/virt/virtio-mmio/virtio-bus/vhost-user-gpio-device/vhost-user-gpio/vhost-user-gpio-tests/read-guest-mem/memfile/subprocess [8737]) failed unexpectedly
(test program exited with status code -6)
――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――



https://gitlab.com/mstredhat/qemu/-/jobs/3279637541


gpio attempts to stop backend when notifiers are not enabled.

No clue how that triggers.



> 
> Alex, Viresh, need your help here. Thanks!
> 
> Alex, pls note that same failures are triggered by your RFC - if we know the
> root cause we can discuss solutions. So if you prefer pls go ahead and
> debug that. Thanks!
> 
> Michael S. Tsirkin (3):
>   virtio: distinguish between started and running
>   gpio: use virtio_device_running
>   virtio: revert changes to virtio_device_started
> 
>  include/hw/virtio/virtio.h   | 7 ++++++-
>  hw/virtio/vhost-user-fs.c    | 2 +-
>  hw/virtio/vhost-user-gpio.c  | 4 ++--
>  hw/virtio/vhost-user-i2c.c   | 4 ++--
>  hw/virtio/vhost-user-rng.c   | 4 ++--
>  hw/virtio/vhost-user-vsock.c | 2 +-
>  hw/virtio/vhost-vsock.c      | 2 +-
>  7 files changed, 15 insertions(+), 10 deletions(-)
> 
> -- 
> MST
>