[PATCH 0/8] Support VIRTIO_F_RING_RESET for vhost-user in virtio pci-modern

[PATCH 0/8] Support VIRTIO_F_RING_RESET for vhost-user in virtio pci-modern

[Kangjie Xu]

This patch set is based on the patch set that supports VIRTIO_F_RING_RESET for vhost-kernel:
    https://lore.kernel.org/qemu-devel/cover.1661414345.git.kangjie.xu@linux.alibaba.com/T/

The virtio queue reset function has already been defined in the virtio spec 1.2.
The relevant virtio spec information is here:

    https://github.com/oasis-tcs/virtio-spec/issues/124
    https://github.com/oasis-tcs/virtio-spec/issues/139

This patch set is to support this function for vhost-user in QEMU. It consists of several parts:
Patch 1: docs about vhost-user message VHOST_USER_RESET_VRING.
Patch 2: rename vhost_set_vring_enable to vhost_set_dev_enable.
Patches 3-4: support in vhost-user module.
Patches 5-6: support in vhost-net module.
Patch 7: support in virtio-net module.
Patch 8: add feature negotitation support.

The process of virtqueue reset can be concluded as:
1. The virtqueue is disabled when VIRTIO_PCI_COMMON_Q_RESET is written.
2. Then the virtqueue can be optionally restarted(re-enabled).

The detailed process is listed below:
1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
    -> virtio_queue_reset() [virtio]
        -> virtio_net_queue_reset() [virtio-net]
            -> vhost_net_virtqueue_reset() [vhost-net]
                -> vhost_user_reset_vring() [vhost-user]
                    -> send VHOST_USER_RESET_VRING to the device
                -> vhost_virtqueue_unmap()
        -> __virtio_queue_reset()
2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
    -> virtio_queue_enable() [virtio]
        -> virtio_net_queue_enable() [virtio-net]
            -> vhost_net_virtqueue_restart() [vhost-net]
                -> vhost_virtqueue_start()
                -> vhost_user_set_vring_enable [vhost-user]
                    -> send VHOST_USER_SET_VRING_ENABLE to the device
    -> set enabled, reset status of vq.

Test environment:
    Qemu: QEMU emulator version 7.0.50
    Guest: 5.19.0-rc3 (With vq reset support)
    DPDK: 22.07-rc1 (With vq reset support)
    Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;

    The drvier can resize the virtio queue, then virtio queue reset function should
    be triggered.

    The default is split mode, modify Qemu virtio-net to add PACKED feature to 
    test packed mode.

Guest Kernel Patch:
    https://lore.kernel.org/bpf/20220801063902.129329-1-xuanzhuo@linux.alibaba.com/

DPDK Patch:
    https://github.com/middaywords/dpdk/compare/72206323a5dd3182b13f61b25a64abdddfee595c...eabadfac7953da66bc10ffb8284b490d09bb7ec7

changelog:(based the series https://lore.kernel.org/qemu-devel/cover.1658141552.git.kangjie.xu@linux.alibaba.com/T/#t)
1. rename vhost_set_vring_enable to vhost_set_dev_enable.
2. add vhost-user message VHOST_USER_RESET_VRING
3. remove restart/reset functions of virtqueue in vhost module.

Kangjie Xu (8):
  docs: vhost-user: add VHOST_USER_RESET_VRING message
  net: virtio: rename vhost_set_vring_enable to vhost_set_dev_enable
  vhost-user: add op to enable or disable a single vring
  vhost-user: introduce vhost_reset_vring() interface
  vhost-net: vhost-user: update vhost_net_virtqueue_reset()
  vhost-net: vhost-user: update vhost_net_virtqueue_restart()
  virtio-net: vhost-user: update queue_reset and queue_enable
  vhost: vhost-user: enable vq reset feature

 backends/cryptodev-vhost.c        | 12 +++---
 docs/interop/vhost-user.rst       | 10 +++++
 hw/net/vhost_net-stub.c           |  2 +-
 hw/net/vhost_net.c                | 34 +++++++++++++---
 hw/net/virtio-net.c               | 10 +++--
 hw/virtio/vhost-user.c            | 68 +++++++++++++++++++++++++++----
 include/hw/virtio/vhost-backend.h |  8 +++-
 include/net/vhost_net.h           |  2 +-
 8 files changed, 119 insertions(+), 27 deletions(-)

-- 
2.32.0

[PATCH 1/8] docs: vhost-user: add VHOST_USER_RESET_VRING message

[Kangjie Xu]

To support the reset operation for an individual virtqueue,
we introduce a new message VHOST_USER_RESET_VRING. This
message is submitted by the front-end to reset an individual
virtqueue to initial states in the back-end. The reply is
needed to ensure that the reset operation is complete.

Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 docs/interop/vhost-user.rst | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst
index 3f18ab424e..ce7991b9d3 100644
--- a/docs/interop/vhost-user.rst
+++ b/docs/interop/vhost-user.rst
@@ -1422,6 +1422,16 @@ Front-end message types
   query the back-end for its device status as defined in the Virtio
   specification.
 
+``VHOST_USER_RESET_VRING``
+  :id: 41
+  :equivalent ioctl: N/A
+  :request payload: vring state description
+  :reply payload: ``u64``
+
+  When the feature ``VIRTIO_F_RING_RESET`` feature has been successfully
+  negotiated, this message is submitted by the front-end to reset an
+  individual virtqueue to initial states in the back-end. It will ask
+  for a reply to ensure the virtqueue is successfully reset in the back-end.
 
 Back-end message types
 ----------------------
-- 
2.32.0

[PATCH 2/8] net: virtio: rename vhost_set_vring_enable to vhost_set_dev_enable

[Kangjie Xu]

Previously, vhost_set_vring_enable will enable/disable all vrings
in a device, which causes ambiguity. So we rename it to
vhost_set_dev_enable.

Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 backends/cryptodev-vhost.c        | 12 ++++++------
 hw/net/vhost_net-stub.c           |  2 +-
 hw/net/vhost_net.c                |  8 ++++----
 hw/net/virtio-net.c               |  4 ++--
 hw/virtio/vhost-user.c            |  4 ++--
 include/hw/virtio/vhost-backend.h |  6 +++---
 include/net/vhost_net.h           |  2 +-
 7 files changed, 19 insertions(+), 19 deletions(-)

diff --git a/backends/cryptodev-vhost.c b/backends/cryptodev-vhost.c
index bc13e466b4..b83e939760 100644
--- a/backends/cryptodev-vhost.c
+++ b/backends/cryptodev-vhost.c
@@ -147,9 +147,9 @@ cryptodev_vhost_set_vq_index(CryptoDevBackendVhost *crypto,
 }
 
 static int
-vhost_set_vring_enable(CryptoDevBackendClient *cc,
-                            CryptoDevBackend *b,
-                            uint16_t queue, int enable)
+vhost_set_dev_enable(CryptoDevBackendClient *cc,
+                     CryptoDevBackend *b,
+                     uint16_t queue, int enable)
 {
     CryptoDevBackendVhost *crypto =
                        cryptodev_get_vhost(cc, b, queue);
@@ -162,8 +162,8 @@ vhost_set_vring_enable(CryptoDevBackendClient *cc,
     }
 
     vhost_ops = crypto->dev.vhost_ops;
-    if (vhost_ops->vhost_set_vring_enable) {
-        return vhost_ops->vhost_set_vring_enable(&crypto->dev, enable);
+    if (vhost_ops->vhost_set_dev_enable) {
+        return vhost_ops->vhost_set_dev_enable(&crypto->dev, enable);
     }
 
     return 0;
@@ -219,7 +219,7 @@ int cryptodev_vhost_start(VirtIODevice *dev, int total_queues)
 
         if (cc->vring_enable) {
             /* restore vring enable state */
-            r = vhost_set_vring_enable(cc, b, i, cc->vring_enable);
+            r = vhost_set_dev_enable(cc, b, i, cc->vring_enable);
 
             if (r < 0) {
                 goto err_start;
diff --git a/hw/net/vhost_net-stub.c b/hw/net/vhost_net-stub.c
index 89d71cfb8e..ac5f217dc1 100644
--- a/hw/net/vhost_net-stub.c
+++ b/hw/net/vhost_net-stub.c
@@ -92,7 +92,7 @@ VHostNetState *get_vhost_net(NetClientState *nc)
     return 0;
 }
 
-int vhost_set_vring_enable(NetClientState *nc, int enable)
+int vhost_set_dev_enable(NetClientState *nc, int enable)
 {
     return 0;
 }
diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index 74c5147d6e..c0c1456172 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -379,7 +379,7 @@ int vhost_net_start(VirtIODevice *dev, NetClientState *ncs,
 
         if (peer->vring_enable) {
             /* restore vring enable state */
-            r = vhost_set_vring_enable(peer, peer->vring_enable);
+            r = vhost_set_dev_enable(peer, peer->vring_enable);
 
             if (r < 0) {
                 vhost_net_stop_one(get_vhost_net(peer), dev);
@@ -491,15 +491,15 @@ VHostNetState *get_vhost_net(NetClientState *nc)
     return vhost_net;
 }
 
-int vhost_set_vring_enable(NetClientState *nc, int enable)
+int vhost_set_dev_enable(NetClientState *nc, int enable)
 {
     VHostNetState *net = get_vhost_net(nc);
     const VhostOps *vhost_ops = net->dev.vhost_ops;
 
     nc->vring_enable = enable;
 
-    if (vhost_ops && vhost_ops->vhost_set_vring_enable) {
-        return vhost_ops->vhost_set_vring_enable(&net->dev, enable);
+    if (vhost_ops && vhost_ops->vhost_set_dev_enable) {
+        return vhost_ops->vhost_set_dev_enable(&net->dev, enable);
     }
 
     return 0;
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 7817206596..6ab796b399 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -696,7 +696,7 @@ static int peer_attach(VirtIONet *n, int index)
     }
 
     if (nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_USER) {
-        vhost_set_vring_enable(nc->peer, 1);
+        vhost_set_dev_enable(nc->peer, 1);
     }
 
     if (nc->peer->info->type != NET_CLIENT_DRIVER_TAP) {
@@ -719,7 +719,7 @@ static int peer_detach(VirtIONet *n, int index)
     }
 
     if (nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_USER) {
-        vhost_set_vring_enable(nc->peer, 0);
+        vhost_set_dev_enable(nc->peer, 0);
     }
 
     if (nc->peer->info->type !=  NET_CLIENT_DRIVER_TAP) {
diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index bd24741be8..794519359b 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -1198,7 +1198,7 @@ static int vhost_user_set_vring_base(struct vhost_dev *dev,
     return vhost_set_vring(dev, VHOST_USER_SET_VRING_BASE, ring);
 }
 
-static int vhost_user_set_vring_enable(struct vhost_dev *dev, int enable)
+static int vhost_user_set_dev_enable(struct vhost_dev *dev, int enable)
 {
     int i;
 
@@ -2627,7 +2627,7 @@ const VhostOps user_ops = {
         .vhost_set_owner = vhost_user_set_owner,
         .vhost_reset_device = vhost_user_reset_device,
         .vhost_get_vq_index = vhost_user_get_vq_index,
-        .vhost_set_vring_enable = vhost_user_set_vring_enable,
+        .vhost_set_dev_enable = vhost_user_set_dev_enable,
         .vhost_requires_shm_log = vhost_user_requires_shm_log,
         .vhost_migration_done = vhost_user_migration_done,
         .vhost_backend_can_merge = vhost_user_can_merge,
diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h
index eab46d7f0b..b49432045f 100644
--- a/include/hw/virtio/vhost-backend.h
+++ b/include/hw/virtio/vhost-backend.h
@@ -81,8 +81,8 @@ typedef int (*vhost_set_backend_cap_op)(struct vhost_dev *dev);
 typedef int (*vhost_set_owner_op)(struct vhost_dev *dev);
 typedef int (*vhost_reset_device_op)(struct vhost_dev *dev);
 typedef int (*vhost_get_vq_index_op)(struct vhost_dev *dev, int idx);
-typedef int (*vhost_set_vring_enable_op)(struct vhost_dev *dev,
-                                         int enable);
+typedef int (*vhost_set_dev_enable_op)(struct vhost_dev *dev,
+                                       int enable);
 typedef bool (*vhost_requires_shm_log_op)(struct vhost_dev *dev);
 typedef int (*vhost_migration_done_op)(struct vhost_dev *dev,
                                        char *mac_addr);
@@ -155,7 +155,7 @@ typedef struct VhostOps {
     vhost_set_owner_op vhost_set_owner;
     vhost_reset_device_op vhost_reset_device;
     vhost_get_vq_index_op vhost_get_vq_index;
-    vhost_set_vring_enable_op vhost_set_vring_enable;
+    vhost_set_dev_enable_op vhost_set_dev_enable;
     vhost_requires_shm_log_op vhost_requires_shm_log;
     vhost_migration_done_op vhost_migration_done;
     vhost_backend_can_merge_op vhost_backend_can_merge;
diff --git a/include/net/vhost_net.h b/include/net/vhost_net.h
index 40b9a40074..22a1fcd39e 100644
--- a/include/net/vhost_net.h
+++ b/include/net/vhost_net.h
@@ -42,7 +42,7 @@ void vhost_net_virtqueue_mask(VHostNetState *net, VirtIODevice *dev,
 int vhost_net_notify_migration_done(VHostNetState *net, char* mac_addr);
 VHostNetState *get_vhost_net(NetClientState *nc);
 
-int vhost_set_vring_enable(NetClientState * nc, int enable);
+int vhost_set_dev_enable(NetClientState *nc, int enable);
 
 uint64_t vhost_net_get_acked_features(VHostNetState *net);
 
-- 
2.32.0

[PATCH 3/8] vhost-user: add op to enable or disable a single vring

[Kangjie Xu]

There is only vhost_set_dev_enable op in VhostOps. Thus, we introduce
the interface vhost_set_vring_enable to set the enable status for a
single vring.

Resetting a single vq will rely on this interface.

Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 hw/virtio/vhost-user.c            | 25 ++++++++++++++++++-------
 include/hw/virtio/vhost-backend.h |  3 +++
 2 files changed, 21 insertions(+), 7 deletions(-)

diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index 794519359b..3f140d5085 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -1198,6 +1198,22 @@ static int vhost_user_set_vring_base(struct vhost_dev *dev,
     return vhost_set_vring(dev, VHOST_USER_SET_VRING_BASE, ring);
 }
 
+static int vhost_user_set_vring_enable(struct vhost_dev *dev,
+                                       int index,
+                                       int enable)
+{
+    if (index < dev->vq_index || index >= dev->vq_index + dev->nvqs) {
+        return -EINVAL;
+    }
+
+    struct vhost_vring_state state = {
+        .index = index,
+        .num   = enable,
+    };
+
+    return vhost_set_vring(dev, VHOST_USER_SET_VRING_ENABLE, &state);
+}
+
 static int vhost_user_set_dev_enable(struct vhost_dev *dev, int enable)
 {
     int i;
@@ -1207,13 +1223,7 @@ static int vhost_user_set_dev_enable(struct vhost_dev *dev, int enable)
     }
 
     for (i = 0; i < dev->nvqs; ++i) {
-        int ret;
-        struct vhost_vring_state state = {
-            .index = dev->vq_index + i,
-            .num   = enable,
-        };
-
-        ret = vhost_set_vring(dev, VHOST_USER_SET_VRING_ENABLE, &state);
+        int ret = vhost_user_set_vring_enable(dev, dev->vq_index + i, enable);
         if (ret < 0) {
             /*
              * Restoring the previous state is likely infeasible, as well as
@@ -2627,6 +2637,7 @@ const VhostOps user_ops = {
         .vhost_set_owner = vhost_user_set_owner,
         .vhost_reset_device = vhost_user_reset_device,
         .vhost_get_vq_index = vhost_user_get_vq_index,
+        .vhost_set_vring_enable = vhost_user_set_vring_enable,
         .vhost_set_dev_enable = vhost_user_set_dev_enable,
         .vhost_requires_shm_log = vhost_user_requires_shm_log,
         .vhost_migration_done = vhost_user_migration_done,
diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h
index b49432045f..dad7191bac 100644
--- a/include/hw/virtio/vhost-backend.h
+++ b/include/hw/virtio/vhost-backend.h
@@ -81,6 +81,8 @@ typedef int (*vhost_set_backend_cap_op)(struct vhost_dev *dev);
 typedef int (*vhost_set_owner_op)(struct vhost_dev *dev);
 typedef int (*vhost_reset_device_op)(struct vhost_dev *dev);
 typedef int (*vhost_get_vq_index_op)(struct vhost_dev *dev, int idx);
+typedef int (*vhost_set_vring_enable_op)(struct vhost_dev *dev,
+                                         int index, int enable);
 typedef int (*vhost_set_dev_enable_op)(struct vhost_dev *dev,
                                        int enable);
 typedef bool (*vhost_requires_shm_log_op)(struct vhost_dev *dev);
@@ -155,6 +157,7 @@ typedef struct VhostOps {
     vhost_set_owner_op vhost_set_owner;
     vhost_reset_device_op vhost_reset_device;
     vhost_get_vq_index_op vhost_get_vq_index;
+    vhost_set_vring_enable_op vhost_set_vring_enable;
     vhost_set_dev_enable_op vhost_set_dev_enable;
     vhost_requires_shm_log_op vhost_requires_shm_log;
     vhost_migration_done_op vhost_migration_done;
-- 
2.32.0

[PATCH 4/8] vhost-user: introduce vhost_reset_vring() interface

[Kangjie Xu]

Introduce the interface vhost_reset_vring(). The interface is a wrapper
to send a VHOST_USER_RESET_VRING message to the back-end. It will reset
an individual vring in the back-end. Meanwhile, it will wait for a reply
to ensure the reset has been completed.

Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 hw/virtio/vhost-user.c            | 41 +++++++++++++++++++++++++++++++
 include/hw/virtio/vhost-backend.h |  3 +++
 2 files changed, 44 insertions(+)

diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index 3f140d5085..b49076fdc4 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -126,6 +126,7 @@ typedef enum VhostUserRequest {
     VHOST_USER_GET_MAX_MEM_SLOTS = 36,
     VHOST_USER_ADD_MEM_REG = 37,
     VHOST_USER_REM_MEM_REG = 38,
+    VHOST_USER_RESET_VRING = 41,
     VHOST_USER_MAX
 } VhostUserRequest;
 
@@ -1508,6 +1509,45 @@ static int vhost_user_get_max_memslots(struct vhost_dev *dev,
     return 0;
 }
 
+static int vhost_user_reset_vring(struct vhost_dev *dev,
+                                  struct vhost_vring_state *ring)
+{
+    int ret;
+    VhostUserMsg msg = {
+        .hdr.request = VHOST_USER_RESET_VRING,
+        .hdr.flags = VHOST_USER_VERSION,
+        .payload.state = *ring,
+        .hdr.size = sizeof(msg.payload.state),
+    };
+
+    if (!virtio_has_feature(dev->acked_features, VIRTIO_F_RING_RESET)) {
+        return -ENOTSUP;
+    }
+
+    ret = vhost_user_write(dev, &msg, NULL, 0);
+    if (ret < 0) {
+        return ret;
+    }
+
+    ret = vhost_user_read(dev, &msg);
+    if (ret < 0) {
+        return ret;
+    }
+
+    if (msg.hdr.request != VHOST_USER_RESET_VRING) {
+        error_report("Received unexpected msg type. Expected %d received %d",
+                     VHOST_USER_RESET_VRING, msg.hdr.request);
+        return -EPROTO;
+    }
+
+    if (msg.hdr.size != sizeof(msg.payload.state)) {
+        error_report("Received bad msg size.");
+        return -EPROTO;
+    }
+
+    return 0;
+}
+
 static int vhost_user_reset_device(struct vhost_dev *dev)
 {
     VhostUserMsg msg = {
@@ -2635,6 +2675,7 @@ const VhostOps user_ops = {
         .vhost_set_features = vhost_user_set_features,
         .vhost_get_features = vhost_user_get_features,
         .vhost_set_owner = vhost_user_set_owner,
+        .vhost_reset_vring = vhost_user_reset_vring,
         .vhost_reset_device = vhost_user_reset_device,
         .vhost_get_vq_index = vhost_user_get_vq_index,
         .vhost_set_vring_enable = vhost_user_set_vring_enable,
diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h
index dad7191bac..ec65a55a77 100644
--- a/include/hw/virtio/vhost-backend.h
+++ b/include/hw/virtio/vhost-backend.h
@@ -79,6 +79,8 @@ typedef int (*vhost_get_features_op)(struct vhost_dev *dev,
                                      uint64_t *features);
 typedef int (*vhost_set_backend_cap_op)(struct vhost_dev *dev);
 typedef int (*vhost_set_owner_op)(struct vhost_dev *dev);
+typedef int (*vhost_reset_vring_op)(struct vhost_dev *dev,
+                                    struct vhost_vring_state *ring);
 typedef int (*vhost_reset_device_op)(struct vhost_dev *dev);
 typedef int (*vhost_get_vq_index_op)(struct vhost_dev *dev, int idx);
 typedef int (*vhost_set_vring_enable_op)(struct vhost_dev *dev,
@@ -156,6 +158,7 @@ typedef struct VhostOps {
     vhost_set_backend_cap_op vhost_set_backend_cap;
     vhost_set_owner_op vhost_set_owner;
     vhost_reset_device_op vhost_reset_device;
+    vhost_reset_vring_op vhost_reset_vring;
     vhost_get_vq_index_op vhost_get_vq_index;
     vhost_set_vring_enable_op vhost_set_vring_enable;
     vhost_set_dev_enable_op vhost_set_dev_enable;
-- 
2.32.0

[PATCH 5/8] vhost-net: vhost-user: update vhost_net_virtqueue_reset()

[Kangjie Xu]

Update vhost_net_virtqueue_reset() for vhost-user scenario.

In order to reuse some functions, we process the idx for
vhost-user scenario because vhost_get_vq_index behave
differently for vhost-user.

Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 hw/net/vhost_net.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index c0c1456172..8ad5743f7c 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -522,19 +522,28 @@ void vhost_net_virtqueue_reset(VirtIODevice *vdev, NetClientState *nc,
     VHostNetState *net = get_vhost_net(nc->peer);
     const VhostOps *vhost_ops = net->dev.vhost_ops;
     struct vhost_vring_file file = { .fd = -1 };
-    int idx;
+    struct vhost_vring_state state;
+    int idx, r;
 
     /* should only be called after backend is connected */
     assert(vhost_ops);
 
     idx = vhost_ops->vhost_get_vq_index(&net->dev, vq_index);
+    if (net->nc->info->type == NET_CLIENT_DRIVER_VHOST_USER) {
+        idx -= net->dev.vq_index;
+    }
 
     if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) {
         file.index = idx;
-        int r = vhost_net_set_backend(&net->dev, &file);
+        r = vhost_net_set_backend(&net->dev, &file);
         assert(r >= 0);
     }
 
+    if (vhost_ops->vhost_reset_vring) {
+        state.index = net->dev.vq_index + idx;
+        r = vhost_ops->vhost_reset_vring(&net->dev, &state);
+    }
+
     vhost_virtqueue_unmap(&net->dev, vdev, net->dev.vqs + idx, idx);
 }
 
-- 
2.32.0

[PATCH 6/8] vhost-net: vhost-user: update vhost_net_virtqueue_restart()

[Kangjie Xu]

Update vhost_net_virtqueue_restart() for vhost-user scenario.

In order to reuse some functions, we process the idx for
vhost-user case. It is because vhost_get_vq_index behave
differently in vhost-user.

Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 hw/net/vhost_net.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index 8ad5743f7c..13b9c11e68 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -563,6 +563,9 @@ int vhost_net_virtqueue_restart(VirtIODevice *vdev, NetClientState *nc,
     assert(vhost_ops);
 
     idx = vhost_ops->vhost_get_vq_index(&net->dev, vq_index);
+    if (net->nc->info->type == NET_CLIENT_DRIVER_VHOST_USER) {
+        idx -= net->dev.vq_index;
+    }
 
     r = vhost_virtqueue_start(&net->dev,
                               vdev,
@@ -572,6 +575,15 @@ int vhost_net_virtqueue_restart(VirtIODevice *vdev, NetClientState *nc,
         goto err_start;
     }
 
+    if (vhost_ops->vhost_set_vring_enable) {
+        r = vhost_ops->vhost_set_vring_enable(&net->dev,
+                                              net->dev.vq_index + idx,
+                                              1);
+        if (r < 0) {
+            goto err_start;
+        }
+    }
+
     if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) {
         file.index = idx;
         file.fd = net->backend;
-- 
2.32.0

[PATCH 7/8] virtio-net: vhost-user: update queue_reset and queue_enable

[Kangjie Xu]

Update virtio_net_queue_reset() and virtio_net_queue_enable()
for vhost-user scenario.

Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 hw/net/virtio-net.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 6ab796b399..19a2132180 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -550,7 +550,8 @@ static void virtio_net_queue_reset(VirtIODevice *vdev, uint32_t queue_index)
     }
 
     if (get_vhost_net(nc->peer) &&
-        nc->peer->info->type == NET_CLIENT_DRIVER_TAP) {
+        (nc->peer->info->type == NET_CLIENT_DRIVER_TAP ||
+         nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_USER)) {
         vhost_net_virtqueue_reset(vdev, nc, queue_index);
     }
 
@@ -568,7 +569,8 @@ static void virtio_net_queue_enable(VirtIODevice *vdev, uint32_t queue_index)
     }
 
     if (get_vhost_net(nc->peer) &&
-        nc->peer->info->type == NET_CLIENT_DRIVER_TAP) {
+        (nc->peer->info->type == NET_CLIENT_DRIVER_TAP ||
+         nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_USER)) {
         r = vhost_net_virtqueue_restart(vdev, nc, queue_index);
         if (r < 0) {
             error_report("unable to restart vhost net virtqueue: %d, "
-- 
2.32.0

[PATCH 8/8] vhost: vhost-user: enable vq reset feature

[Kangjie Xu]

Add virtqueue reset feature for vhost-user.

Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
 hw/net/vhost_net.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index 13b9c11e68..745cb4375b 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -74,6 +74,7 @@ static const int user_feature_bits[] = {
     VIRTIO_NET_F_MTU,
     VIRTIO_F_IOMMU_PLATFORM,
     VIRTIO_F_RING_PACKED,
+    VIRTIO_F_RING_RESET,
     VIRTIO_NET_F_RSS,
     VIRTIO_NET_F_HASH_REPORT,
 
-- 
2.32.0