[RFC PATCH 0/9] Net Control VQ support in vDPA SVQ

[RFC PATCH 0/9] Net Control VQ support in vDPA SVQ

[Eugenio Pérez]

Control virtqueue is used by networking device for accepting various
commands from the driver. It's a must to support multiqueue and other
configurations.

Shadow VirtQueue (SVQ) [1] already makes possible migration of virtqueue
states, effectively intercepting them so qemu can track what regions of memory
are dirty because device action and needs migration. However, this does not
solve networking device state seen by the driver because CVQ messages, like
changes on MAC addresses from the driver.

To solve that, this series uses SVQ infraestructure proposed at SVQ [1] to
intercept networking control messages used by the device. This way, qemu is
able to update VirtIONet device model and to migrate it. This series needs to
be applied on top of [1].

Ideally, only the control VQ would be shadowed for all the run of qemu and the
rest of virtqueues would be passthrough unless it's migration time. However,
this requires the vDPA device to support address translations from more than
one address space, something that is not possible at the moment with the
current vhost-vDPA API. The API change has been proposed at [2], but use of it
is left for future series.

Sending this as a RFC so some details like error control is still not 100%
tested. Comments are welcomed on every aspect of the patch.

[1] https://lore.kernel.org/qemu-devel/20220121202733.404989-1-eperezma@redhat.com/
[2] https://lkml.org/lkml/2020/9/23/1243

Eugenio Pérez (9):
  virtio-net: Expose ctrl virtqueue logic
  vdpa: Extract get geatures part from vhost_vdpa_get_max_queue_pairs
  virtio: Make virtqueue_alloc_element non-static
  vhost: Add SVQElement
  vhost: Add custom used buffer callback
  vdpa: Add map/unmap operation callback to SVQ
  vhost: Add vhost_svq_inject
  vhost: Add vhost_svq_start_op
  vdpa: control virtqueue support on shadow virtqueue

 hw/virtio/vhost-shadow-virtqueue.h |  25 +++-
 include/hw/virtio/vhost-vdpa.h     |   2 +
 include/hw/virtio/virtio-net.h     |   3 +
 include/hw/virtio/virtio.h         |   1 +
 hw/net/virtio-net.c                |  83 ++++++-----
 hw/virtio/vhost-shadow-virtqueue.c | 217 +++++++++++++++++++++++------
 hw/virtio/vhost-vdpa.c             |  22 ++-
 hw/virtio/virtio.c                 |   2 +-
 net/vhost-vdpa.c                   | 140 +++++++++++++++++--
 9 files changed, 405 insertions(+), 90 deletions(-)

-- 
2.27.0

[RFC PATCH 1/9] virtio-net: Expose ctrl virtqueue logic

[Eugenio Pérez]

This allows external vhost-net devices to modify the state of the
VirtIO device model once vhost-vdpa device has acknowledge the control
commands.

Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
---
 include/hw/virtio/virtio-net.h |  3 ++
 hw/net/virtio-net.c            | 83 ++++++++++++++++++++--------------
 2 files changed, 52 insertions(+), 34 deletions(-)

diff --git a/include/hw/virtio/virtio-net.h b/include/hw/virtio/virtio-net.h
index eb87032627..e62f9e227f 100644
--- a/include/hw/virtio/virtio-net.h
+++ b/include/hw/virtio/virtio-net.h
@@ -218,6 +218,9 @@ struct VirtIONet {
     struct EBPFRSSContext ebpf_rss;
 };
 
+unsigned virtio_net_handle_ctrl_iov(VirtIODevice *vdev,
+                                    const struct iovec *in_sg, size_t in_num,
+                                    struct iovec *out_sg, unsigned out_num);
 void virtio_net_set_netclient_name(VirtIONet *n, const char *name,
                                    const char *type);
 
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 49cd13314a..a9027b3373 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -1417,56 +1417,71 @@ static int virtio_net_handle_mq(VirtIONet *n, uint8_t cmd,
     return VIRTIO_NET_OK;
 }
 
-static void virtio_net_handle_ctrl(VirtIODevice *vdev, VirtQueue *vq)
+unsigned virtio_net_handle_ctrl_iov(VirtIODevice *vdev,
+                                    const struct iovec *in_sg, size_t in_num,
+                                    struct iovec *out_sg, unsigned out_num)
 {
     VirtIONet *n = VIRTIO_NET(vdev);
     struct virtio_net_ctrl_hdr ctrl;
     virtio_net_ctrl_ack status = VIRTIO_NET_ERR;
-    VirtQueueElement *elem;
     size_t s;
     struct iovec *iov, *iov2;
-    unsigned int iov_cnt;
+
+    if (iov_size(in_sg, in_num) < sizeof(status) ||
+        iov_size(out_sg, out_num) < sizeof(ctrl)) {
+        virtio_error(vdev, "virtio-net ctrl missing headers");
+        return 0;
+    }
+
+    iov2 = iov = g_memdup(out_sg, sizeof(struct iovec) * out_num);
+    s = iov_to_buf(iov, out_num, 0, &ctrl, sizeof(ctrl));
+    iov_discard_front(&iov, &out_num, sizeof(ctrl));
+    if (s != sizeof(ctrl)) {
+        status = VIRTIO_NET_ERR;
+    } else if (ctrl.class == VIRTIO_NET_CTRL_RX) {
+        status = virtio_net_handle_rx_mode(n, ctrl.cmd, iov, out_num);
+    } else if (ctrl.class == VIRTIO_NET_CTRL_MAC) {
+        status = virtio_net_handle_mac(n, ctrl.cmd, iov, out_num);
+    } else if (ctrl.class == VIRTIO_NET_CTRL_VLAN) {
+        status = virtio_net_handle_vlan_table(n, ctrl.cmd, iov, out_num);
+    } else if (ctrl.class == VIRTIO_NET_CTRL_ANNOUNCE) {
+        status = virtio_net_handle_announce(n, ctrl.cmd, iov, out_num);
+    } else if (ctrl.class == VIRTIO_NET_CTRL_MQ) {
+        status = virtio_net_handle_mq(n, ctrl.cmd, iov, out_num);
+    } else if (ctrl.class == VIRTIO_NET_CTRL_GUEST_OFFLOADS) {
+        status = virtio_net_handle_offloads(n, ctrl.cmd, iov, out_num);
+    }
+
+    s = iov_from_buf(in_sg, in_num, 0, &status, sizeof(status));
+    assert(s == sizeof(status));
+
+    g_free(iov2);
+    return sizeof(status);
+}
+
+static void virtio_net_handle_ctrl(VirtIODevice *vdev, VirtQueue *vq)
+{
+    VirtQueueElement *elem;
 
     for (;;) {
+        unsigned written;
         elem = virtqueue_pop(vq, sizeof(VirtQueueElement));
         if (!elem) {
             break;
         }
-        if (iov_size(elem->in_sg, elem->in_num) < sizeof(status) ||
-            iov_size(elem->out_sg, elem->out_num) < sizeof(ctrl)) {
-            virtio_error(vdev, "virtio-net ctrl missing headers");
+
+        written = virtio_net_handle_ctrl_iov(vdev, elem->in_sg, elem->in_num,
+                                             elem->out_sg, elem->out_num);
+        if (written > 0) {
+            virtqueue_push(vq, elem, written);
+            virtio_notify(vdev, vq);
+            g_free(elem);
+        } else {
             virtqueue_detach_element(vq, elem, 0);
             g_free(elem);
             break;
-        }
 
-        iov_cnt = elem->out_num;
-        iov2 = iov = g_memdup(elem->out_sg, sizeof(struct iovec) * elem->out_num);
-        s = iov_to_buf(iov, iov_cnt, 0, &ctrl, sizeof(ctrl));
-        iov_discard_front(&iov, &iov_cnt, sizeof(ctrl));
-        if (s != sizeof(ctrl)) {
-            status = VIRTIO_NET_ERR;
-        } else if (ctrl.class == VIRTIO_NET_CTRL_RX) {
-            status = virtio_net_handle_rx_mode(n, ctrl.cmd, iov, iov_cnt);
-        } else if (ctrl.class == VIRTIO_NET_CTRL_MAC) {
-            status = virtio_net_handle_mac(n, ctrl.cmd, iov, iov_cnt);
-        } else if (ctrl.class == VIRTIO_NET_CTRL_VLAN) {
-            status = virtio_net_handle_vlan_table(n, ctrl.cmd, iov, iov_cnt);
-        } else if (ctrl.class == VIRTIO_NET_CTRL_ANNOUNCE) {
-            status = virtio_net_handle_announce(n, ctrl.cmd, iov, iov_cnt);
-        } else if (ctrl.class == VIRTIO_NET_CTRL_MQ) {
-            status = virtio_net_handle_mq(n, ctrl.cmd, iov, iov_cnt);
-        } else if (ctrl.class == VIRTIO_NET_CTRL_GUEST_OFFLOADS) {
-            status = virtio_net_handle_offloads(n, ctrl.cmd, iov, iov_cnt);
-        }
-
-        s = iov_from_buf(elem->in_sg, elem->in_num, 0, &status, sizeof(status));
-        assert(s == sizeof(status));
-
-        virtqueue_push(vq, elem, sizeof(status));
-        virtio_notify(vdev, vq);
-        g_free(iov2);
-        g_free(elem);
+        }
     }
 }
 
-- 
2.27.0

[RFC PATCH 2/9] vdpa: Extract get geatures part from vhost_vdpa_get_max_queue_pairs

[Eugenio Pérez]

To know the device features is also needed for CVQ SVQ. Extract from
vhost_vdpa_get_max_queue_pairs so we can reuse it.

Report errno in case of failure getting them while we're at it.

Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
---
 net/vhost-vdpa.c | 30 ++++++++++++++++++++----------
 1 file changed, 20 insertions(+), 10 deletions(-)

diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c
index 420614d590..fc844a7ce6 100644
--- a/net/vhost-vdpa.c
+++ b/net/vhost-vdpa.c
@@ -241,20 +241,24 @@ static NetClientState *net_vhost_vdpa_init(NetClientState *peer,
     return nc;
 }
 
-static int vhost_vdpa_get_max_queue_pairs(int fd, int *has_cvq, Error **errp)
+static int vhost_vdpa_get_features(int fd, uint64_t *features, Error **errp)
+{
+    int ret = ioctl(fd, VHOST_GET_FEATURES, features);
+    if (ret) {
+        error_setg_errno(errp, errno,
+                              "Fail to query features from vhost-vDPA device");
+    }
+    return ret;
+}
+
+static int vhost_vdpa_get_max_queue_pairs(int fd, uint64_t features,
+                                          int *has_cvq, Error **errp)
 {
     unsigned long config_size = offsetof(struct vhost_vdpa_config, buf);
     g_autofree struct vhost_vdpa_config *config = NULL;
     __virtio16 *max_queue_pairs;
-    uint64_t features;
     int ret;
 
-    ret = ioctl(fd, VHOST_GET_FEATURES, &features);
-    if (ret) {
-        error_setg(errp, "Fail to query features from vhost-vDPA device");
-        return ret;
-    }
-
     if (features & (1 << VIRTIO_NET_F_CTRL_VQ)) {
         *has_cvq = 1;
     } else {
@@ -285,10 +289,11 @@ int net_init_vhost_vdpa(const Netdev *netdev, const char *name,
                         NetClientState *peer, Error **errp)
 {
     const NetdevVhostVDPAOptions *opts;
+    uint64_t features;
     int vdpa_device_fd;
     g_autofree NetClientState **ncs = NULL;
     NetClientState *nc;
-    int queue_pairs, i, has_cvq = 0;
+    int queue_pairs, r, i, has_cvq = 0;
     g_autoptr(VhostIOVATree) iova_tree = NULL;
     struct vhost_vdpa_iova_range iova_range;
 
@@ -304,7 +309,12 @@ int net_init_vhost_vdpa(const Netdev *netdev, const char *name,
         return -errno;
     }
 
-    queue_pairs = vhost_vdpa_get_max_queue_pairs(vdpa_device_fd,
+    r = vhost_vdpa_get_features(vdpa_device_fd, &features, errp);
+    if (r) {
+        return r;
+    }
+
+    queue_pairs = vhost_vdpa_get_max_queue_pairs(vdpa_device_fd, features,
                                                  &has_cvq, errp);
     if (queue_pairs < 0) {
         qemu_close(vdpa_device_fd);
-- 
2.27.0

[RFC PATCH 3/9] virtio: Make virtqueue_alloc_element non-static

[Eugenio Pérez]

So SVQ can allocate elements using it

Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
---
 include/hw/virtio/virtio.h | 1 +
 hw/virtio/virtio.c         | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index f095637058..6f4cccfd42 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -195,6 +195,7 @@ void virtqueue_fill(VirtQueue *vq, const VirtQueueElement *elem,
                     unsigned int len, unsigned int idx);
 
 void virtqueue_map(VirtIODevice *vdev, VirtQueueElement *elem);
+void *virtqueue_alloc_element(size_t sz, unsigned out_num, unsigned in_num);
 void *virtqueue_pop(VirtQueue *vq, size_t sz);
 unsigned int virtqueue_drop_all(VirtQueue *vq);
 void *qemu_get_virtqueue_element(VirtIODevice *vdev, QEMUFile *f, size_t sz);
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index aae72fb8b7..ca7d1f0613 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -1376,7 +1376,7 @@ void virtqueue_map(VirtIODevice *vdev, VirtQueueElement *elem)
                                                                         false);
 }
 
-static void *virtqueue_alloc_element(size_t sz, unsigned out_num, unsigned in_num)
+void *virtqueue_alloc_element(size_t sz, unsigned out_num, unsigned in_num)
 {
     VirtQueueElement *elem;
     size_t in_addr_ofs = QEMU_ALIGN_UP(sz, __alignof__(elem->in_addr[0]));
-- 
2.27.0

[RFC PATCH 4/9] vhost: Add SVQElement

[Eugenio Pérez]

This allows SVQ to add metadata to the different queue elements

Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
---
 hw/virtio/vhost-shadow-virtqueue.c | 52 ++++++++++++++++++------------
 1 file changed, 31 insertions(+), 21 deletions(-)

diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c
index 873e39e9cf..ea32b7ae7f 100644
--- a/hw/virtio/vhost-shadow-virtqueue.c
+++ b/hw/virtio/vhost-shadow-virtqueue.c
@@ -17,6 +17,10 @@
 #include "qemu/error-report.h"
 #include "qemu/main-loop.h"
 
+typedef struct SVQElement {
+    VirtQueueElement elem;
+} SVQElement;
+
 /* Shadow virtqueue to relay notifications */
 struct VhostShadowVirtqueue {
     /* Shadow vring */
@@ -50,10 +54,10 @@ struct VhostShadowVirtqueue {
     VhostIOVATree *iova_tree;
 
     /* Map for returning guest's descriptors */
-    VirtQueueElement **ring_id_maps;
+    SVQElement **ring_id_maps;
 
     /* Next VirtQueue element that guest made available */
-    VirtQueueElement *next_guest_avail_elem;
+    SVQElement *next_guest_avail_elem;
 
     /* Next head to expose to device */
     uint16_t avail_idx_shadow;
@@ -281,9 +285,10 @@ static void vhost_vring_write_descs(VhostShadowVirtqueue *svq,
 }
 
 static bool vhost_svq_add_split(VhostShadowVirtqueue *svq,
-                                VirtQueueElement *elem,
+                                SVQElement *svq_elem,
                                 unsigned *head)
 {
+    const VirtQueueElement *elem = &svq_elem->elem;
     unsigned avail_idx;
     vring_avail_t *avail = svq->vring.avail;
     bool ok;
@@ -324,7 +329,7 @@ static bool vhost_svq_add_split(VhostShadowVirtqueue *svq,
     return true;
 }
 
-static bool vhost_svq_add(VhostShadowVirtqueue *svq, VirtQueueElement *elem)
+static bool vhost_svq_add(VhostShadowVirtqueue *svq, SVQElement *elem)
 {
     unsigned qemu_head;
     bool ok = vhost_svq_add_split(svq, elem, &qemu_head);
@@ -372,19 +377,21 @@ static void vhost_handle_guest_kick(VhostShadowVirtqueue *svq)
         }
 
         while (true) {
+            SVQElement *svq_elem;
             VirtQueueElement *elem;
             bool ok;
 
             if (svq->next_guest_avail_elem) {
-                elem = g_steal_pointer(&svq->next_guest_avail_elem);
+                svq_elem = g_steal_pointer(&svq->next_guest_avail_elem);
             } else {
-                elem = virtqueue_pop(svq->vq, sizeof(*elem));
+                svq_elem = virtqueue_pop(svq->vq, sizeof(*elem));
             }
 
-            if (!elem) {
+            if (!svq_elem) {
                 break;
             }
 
+            elem = &svq_elem->elem;
             if (elem->out_num + elem->in_num >
                 vhost_svq_available_slots(svq)) {
                 /*
@@ -398,11 +405,11 @@ static void vhost_handle_guest_kick(VhostShadowVirtqueue *svq)
                  * queue the current guest descriptor and ignore further kicks
                  * until some elements are used.
                  */
-                svq->next_guest_avail_elem = elem;
+                svq->next_guest_avail_elem = svq_elem;
                 return;
             }
 
-            ok = vhost_svq_add(svq, elem);
+            ok = vhost_svq_add(svq, svq_elem);
             if (unlikely(!ok)) {
                 /* VQ is broken, just return and ignore any other kicks */
                 return;
@@ -437,7 +444,7 @@ static bool vhost_svq_more_used(VhostShadowVirtqueue *svq)
     return svq->last_used_idx != svq->shadow_used_idx;
 }
 
-static VirtQueueElement *vhost_svq_get_buf(VhostShadowVirtqueue *svq)
+static SVQElement *vhost_svq_get_buf(VhostShadowVirtqueue *svq)
 {
     vring_desc_t *descs = svq->vring.desc;
     const vring_used_t *used = svq->vring.used;
@@ -471,7 +478,7 @@ static VirtQueueElement *vhost_svq_get_buf(VhostShadowVirtqueue *svq)
     descs[used_elem.id].next = svq->free_head;
     svq->free_head = used_elem.id;
 
-    svq->ring_id_maps[used_elem.id]->len = used_elem.len;
+    svq->ring_id_maps[used_elem.id]->elem.len = used_elem.len;
     return g_steal_pointer(&svq->ring_id_maps[used_elem.id]);
 }
 
@@ -486,11 +493,13 @@ static void vhost_svq_flush(VhostShadowVirtqueue *svq,
 
         vhost_svq_set_notification(svq, false);
         while (true) {
-            g_autofree VirtQueueElement *elem = vhost_svq_get_buf(svq);
-            if (!elem) {
+            g_autofree SVQElement *svq_elem = vhost_svq_get_buf(svq);
+            VirtQueueElement *elem;
+            if (!svq_elem) {
                 break;
             }
 
+            elem = &svq_elem->elem;
             if (unlikely(i >= svq->vring.num)) {
                 virtio_error(svq->vdev,
                          "More than %u used buffers obtained in a %u size SVQ",
@@ -667,7 +676,7 @@ void vhost_svq_start(VhostShadowVirtqueue *svq, VirtIODevice *vdev,
 void vhost_svq_stop(VhostShadowVirtqueue *svq)
 {
     event_notifier_set_handler(&svq->svq_kick, NULL);
-    g_autofree VirtQueueElement *next_avail_elem = NULL;
+    g_autofree SVQElement *next_avail_elem = NULL;
 
     if (!svq->vq) {
         return;
@@ -677,17 +686,18 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq)
     vhost_svq_flush(svq, false);
 
     for (unsigned i = 0; i < svq->vring.num; ++i) {
-        g_autofree VirtQueueElement *elem = NULL;
-        elem = g_steal_pointer(&svq->ring_id_maps[i]);
-        if (elem) {
-            virtqueue_detach_element(svq->vq, elem, elem->len);
+        g_autofree SVQElement *svq_elem = NULL;
+        svq_elem = g_steal_pointer(&svq->ring_id_maps[i]);
+        if (svq_elem)  {
+            virtqueue_detach_element(svq->vq, &svq_elem->elem,
+                                     svq_elem->elem.len);
         }
     }
 
     next_avail_elem = g_steal_pointer(&svq->next_guest_avail_elem);
     if (next_avail_elem) {
-        virtqueue_detach_element(svq->vq, next_avail_elem,
-                                 next_avail_elem->len);
+        virtqueue_detach_element(svq->vq, &next_avail_elem->elem,
+                                 next_avail_elem->elem.len);
     }
 }
 
@@ -735,7 +745,7 @@ VhostShadowVirtqueue *vhost_svq_new(uint16_t qsize, VhostIOVATree *iova_tree)
     svq->vring.used = qemu_memalign(qemu_real_host_page_size, device_size);
     memset(svq->vring.used, 0, device_size);
     svq->iova_tree = iova_tree;
-    svq->ring_id_maps = g_new0(VirtQueueElement *, qsize);
+    svq->ring_id_maps = g_new0(SVQElement *, qsize);
     event_notifier_set_handler(&svq->hdev_call, vhost_svq_handle_call);
     return g_steal_pointer(&svq);
 
-- 
2.27.0

[RFC PATCH 5/9] vhost: Add custom used buffer callback

[Eugenio Pérez]

The callback allows SVQ users to know the VirtQueue requests and
responses. QEMU can use this to synchronize virtio device model state,
allowing to migrate it with minimum changes to the migration code.

In the case of networking, this will be used to inspect control
virtqueue messages.

Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
---
 hw/virtio/vhost-shadow-virtqueue.h | 10 +++++++++-
 include/hw/virtio/vhost-vdpa.h     |  2 ++
 hw/virtio/vhost-shadow-virtqueue.c | 15 ++++++++++++++-
 hw/virtio/vhost-vdpa.c             |  4 ++--
 4 files changed, 27 insertions(+), 4 deletions(-)

diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-virtqueue.h
index a2b0c6434d..f23fb93c20 100644
--- a/hw/virtio/vhost-shadow-virtqueue.h
+++ b/hw/virtio/vhost-shadow-virtqueue.h
@@ -16,6 +16,13 @@
 
 typedef struct VhostShadowVirtqueue VhostShadowVirtqueue;
 
+typedef void (*VirtQueueElementCallback)(VirtIODevice *vdev,
+                                         const VirtQueueElement *elem);
+
+typedef struct VhostShadowVirtqueueOps {
+    VirtQueueElementCallback used_elem_handler;
+} VhostShadowVirtqueueOps;
+
 bool vhost_svq_valid_device_features(uint64_t *features);
 bool vhost_svq_valid_guest_features(uint64_t *features);
 bool vhost_svq_ack_guest_features(uint64_t dev_features,
@@ -39,7 +46,8 @@ void vhost_svq_start(VhostShadowVirtqueue *svq, VirtIODevice *vdev,
                      VirtQueue *vq);
 void vhost_svq_stop(VhostShadowVirtqueue *svq);
 
-VhostShadowVirtqueue *vhost_svq_new(uint16_t qsize, VhostIOVATree *iova_map);
+VhostShadowVirtqueue *vhost_svq_new(uint16_t qsize, VhostIOVATree *iova_map,
+                                    const VhostShadowVirtqueueOps *ops);
 
 void vhost_svq_free(VhostShadowVirtqueue *vq);
 
diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h
index cd2388b3be..a0271534e6 100644
--- a/include/hw/virtio/vhost-vdpa.h
+++ b/include/hw/virtio/vhost-vdpa.h
@@ -17,6 +17,7 @@
 #include "hw/virtio/vhost-iova-tree.h"
 #include "hw/virtio/virtio.h"
 #include "standard-headers/linux/vhost_types.h"
+#include "hw/virtio/vhost-shadow-virtqueue.h"
 
 typedef struct VhostVDPAHostNotifier {
     MemoryRegion mr;
@@ -34,6 +35,7 @@ typedef struct vhost_vdpa {
     /* IOVA mapping used by Shadow Virtqueue */
     VhostIOVATree *iova_tree;
     GPtrArray *shadow_vqs;
+    const VhostShadowVirtqueueOps *shadow_vq_ops;
     struct vhost_dev *dev;
     VhostVDPAHostNotifier notifier[VIRTIO_QUEUE_MAX];
 } VhostVDPA;
diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c
index ea32b7ae7f..5665947d1a 100644
--- a/hw/virtio/vhost-shadow-virtqueue.c
+++ b/hw/virtio/vhost-shadow-virtqueue.c
@@ -59,6 +59,12 @@ struct VhostShadowVirtqueue {
     /* Next VirtQueue element that guest made available */
     SVQElement *next_guest_avail_elem;
 
+    /* Optional callbacks */
+    const VhostShadowVirtqueueOps *ops;
+
+    /* Optional custom used virtqueue element handler */
+    VirtQueueElementCallback used_elem_cb;
+
     /* Next head to expose to device */
     uint16_t avail_idx_shadow;
 
@@ -509,6 +515,10 @@ static void vhost_svq_flush(VhostShadowVirtqueue *svq,
                 i = 0;
             }
             virtqueue_fill(vq, elem, elem->len, i++);
+
+            if (svq->ops && svq->ops->used_elem_handler) {
+                svq->ops->used_elem_handler(svq->vdev, elem);
+            }
         }
 
         virtqueue_flush(vq, i);
@@ -707,12 +717,14 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq)
  *
  * @qsize Shadow VirtQueue size
  * @iova_tree Tree to perform descriptors translations
+ * @used_cb Optional callback for each device's used buffer
  *
  * Returns the new virtqueue or NULL.
  *
  * In case of error, reason is reported through error_report.
  */
-VhostShadowVirtqueue *vhost_svq_new(uint16_t qsize, VhostIOVATree *iova_tree)
+VhostShadowVirtqueue *vhost_svq_new(uint16_t qsize, VhostIOVATree *iova_tree,
+                                    const VhostShadowVirtqueueOps *ops)
 {
     size_t desc_size = sizeof(vring_desc_t) * qsize;
     size_t device_size, driver_size;
@@ -747,6 +759,7 @@ VhostShadowVirtqueue *vhost_svq_new(uint16_t qsize, VhostIOVATree *iova_tree)
     svq->iova_tree = iova_tree;
     svq->ring_id_maps = g_new0(SVQElement *, qsize);
     event_notifier_set_handler(&svq->hdev_call, vhost_svq_handle_call);
+    svq->ops = ops;
     return g_steal_pointer(&svq);
 
 err_init_hdev_call:
diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c
index c64a6802b7..5707b1952d 100644
--- a/hw/virtio/vhost-vdpa.c
+++ b/hw/virtio/vhost-vdpa.c
@@ -17,7 +17,6 @@
 #include "hw/virtio/vhost.h"
 #include "hw/virtio/vhost-backend.h"
 #include "hw/virtio/virtio-net.h"
-#include "hw/virtio/vhost-shadow-virtqueue.h"
 #include "hw/virtio/vhost-vdpa.h"
 #include "exec/address-spaces.h"
 #include "qemu/main-loop.h"
@@ -1219,7 +1218,8 @@ static int vhost_vdpa_init_svq(struct vhost_dev *hdev, struct vhost_vdpa *v,
     for (unsigned n = 0; n < hdev->nvqs; ++n) {
         DMAMap device_region, driver_region;
         struct vhost_vring_addr addr;
-        VhostShadowVirtqueue *svq = vhost_svq_new(qsize, v->iova_tree);
+        VhostShadowVirtqueue *svq = vhost_svq_new(qsize, v->iova_tree,
+                                                  v->shadow_vq_ops);
         if (unlikely(!svq)) {
             error_setg(errp, "Cannot create svq %u", n);
             return -1;
-- 
2.27.0

[RFC PATCH 6/9] vdpa: Add map/unmap operation callback to SVQ

[Eugenio Pérez]

Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
---
 hw/virtio/vhost-shadow-virtqueue.h | 13 ++++++++++++-
 hw/virtio/vhost-shadow-virtqueue.c | 12 +++++++++++-
 hw/virtio/vhost-vdpa.c             | 20 +++++++++++++++++++-
 3 files changed, 42 insertions(+), 3 deletions(-)

diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-virtqueue.h
index f23fb93c20..bf3b658889 100644
--- a/hw/virtio/vhost-shadow-virtqueue.h
+++ b/hw/virtio/vhost-shadow-virtqueue.h
@@ -23,6 +23,15 @@ typedef struct VhostShadowVirtqueueOps {
     VirtQueueElementCallback used_elem_handler;
 } VhostShadowVirtqueueOps;
 
+typedef int (*vhost_svq_map_op)(hwaddr iova, hwaddr size, void *vaddr,
+                                bool readonly, void *opaque);
+typedef int (*vhost_svq_unmap_op)(hwaddr iova, hwaddr size, void *opaque);
+
+typedef struct VhostShadowVirtqueueMapOps {
+    vhost_svq_map_op map;
+    vhost_svq_unmap_op unmap;
+} VhostShadowVirtqueueMapOps;
+
 bool vhost_svq_valid_device_features(uint64_t *features);
 bool vhost_svq_valid_guest_features(uint64_t *features);
 bool vhost_svq_ack_guest_features(uint64_t dev_features,
@@ -47,7 +56,9 @@ void vhost_svq_start(VhostShadowVirtqueue *svq, VirtIODevice *vdev,
 void vhost_svq_stop(VhostShadowVirtqueue *svq);
 
 VhostShadowVirtqueue *vhost_svq_new(uint16_t qsize, VhostIOVATree *iova_map,
-                                    const VhostShadowVirtqueueOps *ops);
+                                    const VhostShadowVirtqueueOps *ops,
+                                    const VhostShadowVirtqueueMapOps *map_ops,
+                                    void *map_ops_opaque);
 
 void vhost_svq_free(VhostShadowVirtqueue *vq);
 
diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c
index 5665947d1a..2ba3c2966a 100644
--- a/hw/virtio/vhost-shadow-virtqueue.c
+++ b/hw/virtio/vhost-shadow-virtqueue.c
@@ -62,6 +62,12 @@ struct VhostShadowVirtqueue {
     /* Optional callbacks */
     const VhostShadowVirtqueueOps *ops;
 
+    /* Device memory mapping callbacks */
+    const VhostShadowVirtqueueMapOps *map_ops;
+
+    /* Device memory mapping callbacks opaque */
+    void *map_ops_opaque;
+
     /* Optional custom used virtqueue element handler */
     VirtQueueElementCallback used_elem_cb;
 
@@ -724,7 +730,9 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq)
  * In case of error, reason is reported through error_report.
  */
 VhostShadowVirtqueue *vhost_svq_new(uint16_t qsize, VhostIOVATree *iova_tree,
-                                    const VhostShadowVirtqueueOps *ops)
+                                    const VhostShadowVirtqueueOps *ops,
+                                    const VhostShadowVirtqueueMapOps *map_ops,
+                                    void *map_ops_opaque)
 {
     size_t desc_size = sizeof(vring_desc_t) * qsize;
     size_t device_size, driver_size;
@@ -760,6 +768,8 @@ VhostShadowVirtqueue *vhost_svq_new(uint16_t qsize, VhostIOVATree *iova_tree,
     svq->ring_id_maps = g_new0(SVQElement *, qsize);
     event_notifier_set_handler(&svq->hdev_call, vhost_svq_handle_call);
     svq->ops = ops;
+    svq->map_ops = map_ops;
+    svq->map_ops_opaque = map_ops_opaque;
     return g_steal_pointer(&svq);
 
 err_init_hdev_call:
diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c
index 5707b1952d..4142a1ce5d 100644
--- a/hw/virtio/vhost-vdpa.c
+++ b/hw/virtio/vhost-vdpa.c
@@ -1178,6 +1178,22 @@ static int vhost_vdpa_get_max_queue_size(struct vhost_dev *dev,
     return vhost_vdpa_call(dev, VHOST_VDPA_GET_VRING_NUM, qsize);
 }
 
+static int vhost_vdpa_svq_map(hwaddr iova, hwaddr size, void *vaddr,
+                              bool readonly, void *opaque)
+{
+    return vhost_vdpa_dma_map(opaque, iova, size, vaddr, readonly);
+}
+
+static int vhost_vdpa_svq_unmap(hwaddr iova, hwaddr size, void *opaque)
+{
+    return vhost_vdpa_dma_unmap(opaque, iova, size);
+}
+
+static const VhostShadowVirtqueueMapOps vhost_vdpa_svq_map_ops = {
+    .map = vhost_vdpa_svq_map,
+    .unmap = vhost_vdpa_svq_unmap,
+};
+
 static int vhost_vdpa_init_svq(struct vhost_dev *hdev, struct vhost_vdpa *v,
                                Error **errp)
 {
@@ -1219,7 +1235,9 @@ static int vhost_vdpa_init_svq(struct vhost_dev *hdev, struct vhost_vdpa *v,
         DMAMap device_region, driver_region;
         struct vhost_vring_addr addr;
         VhostShadowVirtqueue *svq = vhost_svq_new(qsize, v->iova_tree,
-                                                  v->shadow_vq_ops);
+                                                  v->shadow_vq_ops,
+                                                  &vhost_vdpa_svq_map_ops,
+                                                  v);
         if (unlikely(!svq)) {
             error_setg(errp, "Cannot create svq %u", n);
             return -1;
-- 
2.27.0

[RFC PATCH 7/9] vhost: Add vhost_svq_inject

[Eugenio Pérez]

This allows qemu to inject packets to the device without guest's notice.

This will be use to inject net CVQ messages to restore status in the destination

Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
---
 hw/virtio/vhost-shadow-virtqueue.h |   2 +
 hw/virtio/vhost-shadow-virtqueue.c | 142 ++++++++++++++++++++++++-----
 2 files changed, 123 insertions(+), 21 deletions(-)

diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-virtqueue.h
index bf3b658889..767b0a82ba 100644
--- a/hw/virtio/vhost-shadow-virtqueue.h
+++ b/hw/virtio/vhost-shadow-virtqueue.h
@@ -38,6 +38,8 @@ bool vhost_svq_ack_guest_features(uint64_t dev_features,
                                   uint64_t guest_features,
                                   uint64_t *acked_features);
 
+bool vhost_svq_inject(VhostShadowVirtqueue *svq, const struct iovec *iov,
+                      size_t out_num, size_t in_num);
 void vhost_svq_set_svq_kick_fd(VhostShadowVirtqueue *svq, int svq_kick_fd);
 void vhost_svq_set_guest_call_notifier(VhostShadowVirtqueue *svq, int call_fd);
 const EventNotifier *vhost_svq_get_dev_kick_notifier(
diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c
index 2ba3c2966a..26938b059c 100644
--- a/hw/virtio/vhost-shadow-virtqueue.c
+++ b/hw/virtio/vhost-shadow-virtqueue.c
@@ -16,9 +16,11 @@
 
 #include "qemu/error-report.h"
 #include "qemu/main-loop.h"
+#include "qemu/iov.h"
 
 typedef struct SVQElement {
     VirtQueueElement elem;
+    bool not_from_guest;
 } SVQElement;
 
 /* Shadow virtqueue to relay notifications */
@@ -311,19 +313,53 @@ static bool vhost_svq_add_split(VhostShadowVirtqueue *svq,
     /* We need some descriptors here */
     assert(elem->out_num || elem->in_num);
 
-    ok = vhost_svq_translate_addr(svq, sgs, elem->out_sg, elem->out_num);
-    if (unlikely(!ok)) {
-        return false;
+    if (elem->out_sg && svq_elem->not_from_guest) {
+        DMAMap map = {
+            .translated_addr = (hwaddr)svq_elem->elem.out_sg->iov_base,
+            .size = ROUND_UP(elem->out_sg->iov_len, 4096) - 1,
+            .perm = IOMMU_RO,
+        };
+        int r = vhost_iova_tree_map_alloc(svq->iova_tree, &map);
+
+        elem->out_addr[0] = map.iova;
+        assert(elem->out_num == 1);
+        assert(r == IOVA_OK);
+
+        r = svq->map_ops->map(map.iova, map.size, (void *)map.translated_addr,
+                              true, svq->map_ops_opaque);
+        assert(r == 0);
+        sgs[0] = (void *)map.iova;
+    } else {
+        ok = vhost_svq_translate_addr(svq, sgs, elem->out_sg, elem->out_num);
+        if (unlikely(!ok)) {
+            return false;
+        }
     }
     vhost_vring_write_descs(svq, sgs, elem->out_sg, elem->out_num,
                             elem->in_num > 0, false);
 
+    if (elem->in_sg && svq_elem->not_from_guest) {
+        DMAMap map = {
+            .translated_addr = (hwaddr)svq_elem->elem.in_sg->iov_base,
+            .size = ROUND_UP(elem->in_sg->iov_len, 4096) - 1,
+            .perm = IOMMU_RW,
+        };
+        int r = vhost_iova_tree_map_alloc(svq->iova_tree, &map);
 
-    ok = vhost_svq_translate_addr(svq, sgs, elem->in_sg, elem->in_num);
-    if (unlikely(!ok)) {
-        return false;
-    }
+        elem->in_addr[0] = map.iova;
+        assert(elem->out_num == 1);
+        assert(r == IOVA_OK);
 
+        r = svq->map_ops->map(map.iova, map.size, (void *)map.translated_addr,
+                              false, svq->map_ops_opaque);
+        assert(r == 0);
+        sgs[0] = (void *)map.iova;
+    } else {
+        ok = vhost_svq_translate_addr(svq, sgs, elem->in_sg, elem->in_num);
+        if (unlikely(!ok)) {
+            return false;
+        }
+    }
     vhost_vring_write_descs(svq, sgs, elem->in_sg, elem->in_num, false, true);
 
     /*
@@ -364,6 +400,43 @@ static void vhost_svq_kick(VhostShadowVirtqueue *svq)
     event_notifier_set(&svq->hdev_kick);
 }
 
+bool vhost_svq_inject(VhostShadowVirtqueue *svq, const struct iovec *iov,
+                      size_t out_num, size_t in_num)
+{
+    size_t out_size = iov_size(iov, out_num);
+    size_t out_buf_size = ROUND_UP(out_size, 4096);
+    size_t in_size = iov_size(iov + out_num, in_num);
+    size_t in_buf_size = ROUND_UP(in_size, 4096);
+    SVQElement *svq_elem;
+    uint16_t num_slots = (in_num ? 1 : 0) + (out_num ? 1 : 0);
+
+    if (unlikely(num_slots == 0 || svq->next_guest_avail_elem ||
+        vhost_svq_available_slots(svq) < num_slots)) {
+        return false;
+    }
+
+    svq_elem = virtqueue_alloc_element(sizeof(SVQElement), 1, 1);
+    if (out_num) {
+        void *out = qemu_memalign(4096, out_buf_size);
+        svq_elem->elem.out_sg[0].iov_base = out;
+        svq_elem->elem.out_sg[0].iov_len = out_size;
+        iov_to_buf(iov, out_num, 0, out, out_size);
+        memset(out + out_size, 0, out_buf_size - out_size);
+    }
+    if (in_num) {
+        void *in = qemu_memalign(4096, in_buf_size);
+        svq_elem->elem.in_sg[0].iov_base = in;
+        svq_elem->elem.in_sg[0].iov_len = in_size;
+        memset(in, 0, in_buf_size);
+    }
+
+    svq_elem->not_from_guest = true;
+    vhost_svq_add(svq, svq_elem);
+    vhost_svq_kick(svq);
+
+    return true;
+}
+
 /**
  * Forward available buffers.
  *
@@ -512,23 +585,50 @@ static void vhost_svq_flush(VhostShadowVirtqueue *svq,
             }
 
             elem = &svq_elem->elem;
-            if (unlikely(i >= svq->vring.num)) {
-                virtio_error(svq->vdev,
-                         "More than %u used buffers obtained in a %u size SVQ",
-                         i, svq->vring.num);
-                virtqueue_fill(vq, elem, elem->len, i);
-                virtqueue_flush(vq, i);
-                i = 0;
-            }
-            virtqueue_fill(vq, elem, elem->len, i++);
-
             if (svq->ops && svq->ops->used_elem_handler) {
                 svq->ops->used_elem_handler(svq->vdev, elem);
             }
+
+            if (svq_elem->not_from_guest) {
+                const DMAMap out_map = {
+                    .iova = elem->out_addr[0],
+                    .translated_addr = (hwaddr)elem->out_sg[0].iov_base,
+                    .size = elem->out_sg[0].iov_len,
+                };
+                const DMAMap in_map = {
+                    .iova = elem->in_addr[0],
+                    .translated_addr = (hwaddr)elem->in_sg[0].iov_base,
+                    .size = elem->in_sg[0].iov_len,
+                };
+                vhost_iova_tree_remove(svq->iova_tree, &out_map);
+                if (svq->map_ops->unmap) {
+                    svq->map_ops->unmap(out_map.iova, in_map.size,
+                                        svq->map_ops_opaque);
+                }
+                qemu_vfree(elem->out_sg[0].iov_base);
+                vhost_iova_tree_remove(svq->iova_tree, &in_map);
+                if (svq->map_ops->unmap) {
+                    svq->map_ops->unmap(out_map.iova, out_map.size,
+                                        svq->map_ops_opaque);
+                }
+                qemu_vfree(elem->in_sg[0].iov_base);
+            } else {
+                if (unlikely(i >= svq->vring.num)) {
+                    virtio_error(svq->vdev,
+                        "More than %u used buffers obtained in a %u size SVQ",
+                        i, svq->vring.num);
+                    virtqueue_fill(vq, elem, elem->len, i);
+                    virtqueue_flush(vq, i);
+                    i = 0;
+                }
+                virtqueue_fill(vq, elem, elem->len, i++);
+            }
         }
 
-        virtqueue_flush(vq, i);
-        event_notifier_set(&svq->svq_call);
+        if (i > 0) {
+            virtqueue_flush(vq, i);
+            event_notifier_set(&svq->svq_call);
+        }
 
         if (check_for_avail_queue && svq->next_guest_avail_elem) {
             /*
@@ -704,14 +804,14 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq)
     for (unsigned i = 0; i < svq->vring.num; ++i) {
         g_autofree SVQElement *svq_elem = NULL;
         svq_elem = g_steal_pointer(&svq->ring_id_maps[i]);
-        if (svq_elem)  {
+        if (svq_elem && !svq_elem->not_from_guest)  {
             virtqueue_detach_element(svq->vq, &svq_elem->elem,
                                      svq_elem->elem.len);
         }
     }
 
     next_avail_elem = g_steal_pointer(&svq->next_guest_avail_elem);
-    if (next_avail_elem) {
+    if (next_avail_elem && !next_avail_elem->not_from_guest) {
         virtqueue_detach_element(svq->vq, &next_avail_elem->elem,
                                  next_avail_elem->elem.len);
     }
-- 
2.27.0

[RFC PATCH 8/9] vhost: Add vhost_svq_start_op

[Eugenio Pérez]

Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
---
 hw/virtio/vhost-shadow-virtqueue.h | 4 +++-
 hw/virtio/vhost-shadow-virtqueue.c | 4 ++++
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-virtqueue.h
index 767b0a82ba..8f2377bbc4 100644
--- a/hw/virtio/vhost-shadow-virtqueue.h
+++ b/hw/virtio/vhost-shadow-virtqueue.h
@@ -15,11 +15,13 @@
 #include "hw/virtio/vhost-iova-tree.h"
 
 typedef struct VhostShadowVirtqueue VhostShadowVirtqueue;
-
+typedef bool (*vhost_svq_start_op)(VhostShadowVirtqueue *svq,
+                                   VirtIODevice *vdev);
 typedef void (*VirtQueueElementCallback)(VirtIODevice *vdev,
                                          const VirtQueueElement *elem);
 
 typedef struct VhostShadowVirtqueueOps {
+    vhost_svq_start_op start;
     VirtQueueElementCallback used_elem_handler;
 } VhostShadowVirtqueueOps;
 
diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c
index 26938b059c..18cdb35ea3 100644
--- a/hw/virtio/vhost-shadow-virtqueue.c
+++ b/hw/virtio/vhost-shadow-virtqueue.c
@@ -783,6 +783,10 @@ void vhost_svq_start(VhostShadowVirtqueue *svq, VirtIODevice *vdev,
     for (unsigned i = 0; i < svq->vring.num - 1; i++) {
         svq->vring.desc[i].next = cpu_to_le16(i + 1);
     }
+
+    if (svq->ops && svq->ops->start) {
+        svq->ops->start(svq, vdev);
+    }
 }
 
 /**
-- 
2.27.0

[RFC PATCH 9/9] vdpa: control virtqueue support on shadow virtqueue

[Eugenio Pérez]

Introduce the control virtqueue support for vDPA shadow virtqueue. This
is needed for advanced networking features like multiqueue.

To demonstrate command handling, VIRTIO_NET_F_CTROL_MACADDR is
implemented. If vDPA device is started with SVQ support, and MAC changes
in the source VM, it will be transfered with the rest of properties in
the emulated virtio-net device model.

A new CVQ command will be reproduced at destination so that NIC is aware
of the changed MAC.

Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
---
 net/vhost-vdpa.c | 110 ++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 108 insertions(+), 2 deletions(-)

diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c
index fc844a7ce6..ea4e489070 100644
--- a/net/vhost-vdpa.c
+++ b/net/vhost-vdpa.c
@@ -11,6 +11,7 @@
 
 #include "qemu/osdep.h"
 #include "clients.h"
+#include "hw/virtio/virtio-net.h"
 #include "net/vhost_net.h"
 #include "net/vhost-vdpa.h"
 #include "hw/virtio/vhost-vdpa.h"
@@ -70,6 +71,28 @@ const int vdpa_feature_bits[] = {
     VHOST_INVALID_FEATURE_BIT
 };
 
+/** Supported device specific feature bits with SVQ */
+static const uint64_t vdpa_svq_device_features =
+    BIT_ULL(VIRTIO_NET_F_CSUM) |
+    BIT_ULL(VIRTIO_NET_F_GUEST_CSUM) |
+    BIT_ULL(VIRTIO_NET_F_CTRL_GUEST_OFFLOADS) |
+    BIT_ULL(VIRTIO_NET_F_MTU) |
+    BIT_ULL(VIRTIO_NET_F_MAC) |
+    BIT_ULL(VIRTIO_NET_F_GUEST_TSO4) |
+    BIT_ULL(VIRTIO_NET_F_GUEST_TSO6) |
+    BIT_ULL(VIRTIO_NET_F_GUEST_ECN) |
+    BIT_ULL(VIRTIO_NET_F_GUEST_UFO) |
+    BIT_ULL(VIRTIO_NET_F_HOST_TSO4) |
+    BIT_ULL(VIRTIO_NET_F_HOST_TSO6) |
+    BIT_ULL(VIRTIO_NET_F_HOST_ECN) |
+    BIT_ULL(VIRTIO_NET_F_HOST_UFO) |
+    BIT_ULL(VIRTIO_NET_F_MRG_RXBUF) |
+    BIT_ULL(VIRTIO_NET_F_STATUS) |
+    BIT_ULL(VIRTIO_NET_F_CTRL_VQ) |
+    BIT_ULL(VIRTIO_NET_F_CTRL_MAC_ADDR) |
+    BIT_ULL(VIRTIO_NET_F_RSC_EXT) |
+    BIT_ULL(VIRTIO_NET_F_STANDBY);
+
 VHostNetState *vhost_vdpa_get_vhost_net(NetClientState *nc)
 {
     VhostVDPAState *s = DO_UPCAST(VhostVDPAState, nc, nc);
@@ -203,6 +226,79 @@ static void vhost_vdpa_get_iova_range(int fd,
     }
 }
 
+static bool vhost_vdpa_start_control_svq(VhostShadowVirtqueue *svq,
+                                        VirtIODevice *vdev)
+{
+    VirtIONet *n = VIRTIO_NET(vdev);
+    NetClientState *nc = qemu_get_subqueue(n->nic, n->max_queue_pairs);
+    VhostVDPAState *s = DO_UPCAST(VhostVDPAState, nc, nc->peer);
+    uint64_t features = vdev->host_features;
+    assert(s->nc.info->type == NET_CLIENT_DRIVER_VHOST_VDPA);
+
+    if (features & BIT_ULL(VIRTIO_NET_F_CTRL_MAC_ADDR)) {
+        const struct virtio_net_ctrl_hdr ctrl = {
+            .class = VIRTIO_NET_CTRL_MAC,
+            .cmd = VIRTIO_NET_CTRL_MAC_ADDR_SET,
+        };
+        uint8_t mac[6];
+        const struct iovec data[] = {
+            {
+                .iov_base = (void *)&ctrl,
+                .iov_len = sizeof(ctrl),
+            },{
+                .iov_base = mac,
+                .iov_len = sizeof(mac),
+            },{
+                .iov_base = NULL,
+                .iov_len = sizeof(virtio_net_ctrl_ack),
+            }
+        };
+        bool ret;
+
+        /* TODO: Only best effort? */
+        memcpy(mac, n->mac, sizeof(mac));
+        ret = vhost_svq_inject(svq, data, 2, 1);
+        if (!ret) {
+            return false;
+        }
+    }
+
+    return true;
+}
+
+static void vhost_vdpa_net_handle_ctrl(VirtIODevice *vdev,
+                                       const VirtQueueElement *elem)
+{
+    struct virtio_net_ctrl_hdr ctrl;
+    virtio_net_ctrl_ack status = VIRTIO_NET_ERR;
+    size_t s;
+    struct iovec in = {
+        .iov_base = &status,
+        .iov_len = sizeof(status),
+    };
+
+    s = iov_to_buf(elem->out_sg, elem->out_num, 0, &ctrl, sizeof(ctrl.class));
+    if (s != sizeof(ctrl.class) ||
+        ctrl.class != VIRTIO_NET_CTRL_MAC_ADDR_SET) {
+        return;
+    }
+    s = iov_to_buf(elem->in_sg, elem->in_num, 0, &status, sizeof(status));
+    if (s != sizeof(status) || status != VIRTIO_NET_OK) {
+        return;
+    }
+
+    status = VIRTIO_NET_ERR;
+    virtio_net_handle_ctrl_iov(vdev, &in, 1, elem->out_sg, elem->out_num);
+    if (status != VIRTIO_NET_OK) {
+        error_report("Bad CVQ processing in model");
+    }
+}
+
+static const VhostShadowVirtqueueOps vhost_vdpa_net_svq_ops = {
+    .start = vhost_vdpa_start_control_svq,
+    .used_elem_handler = vhost_vdpa_net_handle_ctrl,
+};
+
 static NetClientState *net_vhost_vdpa_init(NetClientState *peer,
                                        const char *device,
                                        const char *name,
@@ -232,6 +328,9 @@ static NetClientState *net_vhost_vdpa_init(NetClientState *peer,
     s->vhost_vdpa.index = queue_pair_index;
     s->vhost_vdpa.iova_range = iova_range;
     s->vhost_vdpa.shadow_vqs_enabled = svq;
+    if (!is_datapath) {
+        s->vhost_vdpa.shadow_vq_ops = &vhost_vdpa_net_svq_ops;
+    }
     s->vhost_vdpa.iova_tree = iova_tree;
     ret = vhost_vdpa_add(nc, (void *)&s->vhost_vdpa, queue_pair_index, nvqs);
     if (ret) {
@@ -322,8 +421,15 @@ int net_init_vhost_vdpa(const Netdev *netdev, const char *name,
     }
     vhost_vdpa_get_iova_range(vdpa_device_fd, &iova_range);
     if (opts->x_svq) {
-        if (has_cvq) {
-            error_setg(errp, "vdpa svq does not work with cvq");
+        uint64_t invalid_dev_features = features &
+            ~vdpa_svq_device_features &
+            /* Transport are all accepted at this point */
+            ~MAKE_64BIT_MASK(VIRTIO_TRANSPORT_F_START,
+                            VIRTIO_TRANSPORT_F_END - VIRTIO_TRANSPORT_F_START);
+
+        if (invalid_dev_features) {
+            error_setg(errp, "vdpa svq does not work with features 0x%" PRIx64,
+                       invalid_dev_features);
             goto err_svq;
         }
         iova_tree = vhost_iova_tree_new(iova_range.first, iova_range.last);
-- 
2.27.0

Re: [RFC PATCH 7/9] vhost: Add vhost_svq_inject

[Eugenio Perez Martin]

On Mon, Feb 14, 2022 at 8:28 PM Eugenio Pérez <eperezma@redhat.com> wrote:
>
> This allows qemu to inject packets to the device without guest's notice.
>
> This will be use to inject net CVQ messages to restore status in the destination
>
> Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
> ---
>  hw/virtio/vhost-shadow-virtqueue.h |   2 +
>  hw/virtio/vhost-shadow-virtqueue.c | 142 ++++++++++++++++++++++++-----
>  2 files changed, 123 insertions(+), 21 deletions(-)
>
> diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-virtqueue.h
> index bf3b658889..767b0a82ba 100644
> --- a/hw/virtio/vhost-shadow-virtqueue.h
> +++ b/hw/virtio/vhost-shadow-virtqueue.h
> @@ -38,6 +38,8 @@ bool vhost_svq_ack_guest_features(uint64_t dev_features,
>                                    uint64_t guest_features,
>                                    uint64_t *acked_features);
>
> +bool vhost_svq_inject(VhostShadowVirtqueue *svq, const struct iovec *iov,
> +                      size_t out_num, size_t in_num);
>  void vhost_svq_set_svq_kick_fd(VhostShadowVirtqueue *svq, int svq_kick_fd);
>  void vhost_svq_set_guest_call_notifier(VhostShadowVirtqueue *svq, int call_fd);
>  const EventNotifier *vhost_svq_get_dev_kick_notifier(
> diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c
> index 2ba3c2966a..26938b059c 100644
> --- a/hw/virtio/vhost-shadow-virtqueue.c
> +++ b/hw/virtio/vhost-shadow-virtqueue.c
> @@ -16,9 +16,11 @@
>
>  #include "qemu/error-report.h"
>  #include "qemu/main-loop.h"
> +#include "qemu/iov.h"
>
>  typedef struct SVQElement {
>      VirtQueueElement elem;
> +    bool not_from_guest;
>  } SVQElement;
>
>  /* Shadow virtqueue to relay notifications */
> @@ -311,19 +313,53 @@ static bool vhost_svq_add_split(VhostShadowVirtqueue *svq,
>      /* We need some descriptors here */
>      assert(elem->out_num || elem->in_num);
>
> -    ok = vhost_svq_translate_addr(svq, sgs, elem->out_sg, elem->out_num);
> -    if (unlikely(!ok)) {
> -        return false;
> +    if (elem->out_sg && svq_elem->not_from_guest) {
> +        DMAMap map = {
> +            .translated_addr = (hwaddr)svq_elem->elem.out_sg->iov_base,
> +            .size = ROUND_UP(elem->out_sg->iov_len, 4096) - 1,
> +            .perm = IOMMU_RO,
> +        };
> +        int r = vhost_iova_tree_map_alloc(svq->iova_tree, &map);
> +
> +        elem->out_addr[0] = map.iova;
> +        assert(elem->out_num == 1);
> +        assert(r == IOVA_OK);
> +
> +        r = svq->map_ops->map(map.iova, map.size, (void *)map.translated_addr,
> +                              true, svq->map_ops_opaque);
> +        assert(r == 0);
> +        sgs[0] = (void *)map.iova;
> +    } else {
> +        ok = vhost_svq_translate_addr(svq, sgs, elem->out_sg, elem->out_num);
> +        if (unlikely(!ok)) {
> +            return false;
> +        }
>      }
>      vhost_vring_write_descs(svq, sgs, elem->out_sg, elem->out_num,
>                              elem->in_num > 0, false);
>
> +    if (elem->in_sg && svq_elem->not_from_guest) {
> +        DMAMap map = {
> +            .translated_addr = (hwaddr)svq_elem->elem.in_sg->iov_base,
> +            .size = ROUND_UP(elem->in_sg->iov_len, 4096) - 1,
> +            .perm = IOMMU_RW,
> +        };
> +        int r = vhost_iova_tree_map_alloc(svq->iova_tree, &map);
>
> -    ok = vhost_svq_translate_addr(svq, sgs, elem->in_sg, elem->in_num);
> -    if (unlikely(!ok)) {
> -        return false;
> -    }
> +        elem->in_addr[0] = map.iova;
> +        assert(elem->out_num == 1);
> +        assert(r == IOVA_OK);
>
> +        r = svq->map_ops->map(map.iova, map.size, (void *)map.translated_addr,
> +                              false, svq->map_ops_opaque);
> +        assert(r == 0);
> +        sgs[0] = (void *)map.iova;
> +    } else {
> +        ok = vhost_svq_translate_addr(svq, sgs, elem->in_sg, elem->in_num);
> +        if (unlikely(!ok)) {
> +            return false;
> +        }
> +    }
>      vhost_vring_write_descs(svq, sgs, elem->in_sg, elem->in_num, false, true);
>
>      /*
> @@ -364,6 +400,43 @@ static void vhost_svq_kick(VhostShadowVirtqueue *svq)
>      event_notifier_set(&svq->hdev_kick);
>  }
>
> +bool vhost_svq_inject(VhostShadowVirtqueue *svq, const struct iovec *iov,
> +                      size_t out_num, size_t in_num)
> +{
> +    size_t out_size = iov_size(iov, out_num);
> +    size_t out_buf_size = ROUND_UP(out_size, 4096);
> +    size_t in_size = iov_size(iov + out_num, in_num);
> +    size_t in_buf_size = ROUND_UP(in_size, 4096);
> +    SVQElement *svq_elem;
> +    uint16_t num_slots = (in_num ? 1 : 0) + (out_num ? 1 : 0);
> +
> +    if (unlikely(num_slots == 0 || svq->next_guest_avail_elem ||
> +        vhost_svq_available_slots(svq) < num_slots)) {
> +        return false;
> +    }
> +
> +    svq_elem = virtqueue_alloc_element(sizeof(SVQElement), 1, 1);
> +    if (out_num) {
> +        void *out = qemu_memalign(4096, out_buf_size);
> +        svq_elem->elem.out_sg[0].iov_base = out;
> +        svq_elem->elem.out_sg[0].iov_len = out_size;
> +        iov_to_buf(iov, out_num, 0, out, out_size);
> +        memset(out + out_size, 0, out_buf_size - out_size);
> +    }
> +    if (in_num) {
> +        void *in = qemu_memalign(4096, in_buf_size);
> +        svq_elem->elem.in_sg[0].iov_base = in;
> +        svq_elem->elem.in_sg[0].iov_len = in_size;
> +        memset(in, 0, in_buf_size);
> +    }
> +
> +    svq_elem->not_from_guest = true;
> +    vhost_svq_add(svq, svq_elem);
> +    vhost_svq_kick(svq);
> +
> +    return true;
> +}
> +
>  /**
>   * Forward available buffers.
>   *
> @@ -512,23 +585,50 @@ static void vhost_svq_flush(VhostShadowVirtqueue *svq,
>              }
>
>              elem = &svq_elem->elem;
> -            if (unlikely(i >= svq->vring.num)) {
> -                virtio_error(svq->vdev,
> -                         "More than %u used buffers obtained in a %u size SVQ",
> -                         i, svq->vring.num);
> -                virtqueue_fill(vq, elem, elem->len, i);
> -                virtqueue_flush(vq, i);
> -                i = 0;
> -            }
> -            virtqueue_fill(vq, elem, elem->len, i++);
> -
>              if (svq->ops && svq->ops->used_elem_handler) {
>                  svq->ops->used_elem_handler(svq->vdev, elem);
>              }
> +
> +            if (svq_elem->not_from_guest) {
> +                const DMAMap out_map = {
> +                    .iova = elem->out_addr[0],
> +                    .translated_addr = (hwaddr)elem->out_sg[0].iov_base,
> +                    .size = elem->out_sg[0].iov_len,
> +                };
> +                const DMAMap in_map = {
> +                    .iova = elem->in_addr[0],
> +                    .translated_addr = (hwaddr)elem->in_sg[0].iov_base,
> +                    .size = elem->in_sg[0].iov_len,
> +                };
> +                vhost_iova_tree_remove(svq->iova_tree, &out_map);
> +                if (svq->map_ops->unmap) {

This is actually bad, as is removing the SVQ element size, not the map size.

Same for the "in_map". I will fix it for the next revision.

Thanks!

> +                    svq->map_ops->unmap(out_map.iova, in_map.size,
> +                                        svq->map_ops_opaque);
> +                }
> +                qemu_vfree(elem->out_sg[0].iov_base);
> +                vhost_iova_tree_remove(svq->iova_tree, &in_map);
> +                if (svq->map_ops->unmap) {
> +                    svq->map_ops->unmap(out_map.iova, out_map.size,
> +                                        svq->map_ops_opaque);
> +                }
> +                qemu_vfree(elem->in_sg[0].iov_base);
> +            } else {
> +                if (unlikely(i >= svq->vring.num)) {
> +                    virtio_error(svq->vdev,
> +                        "More than %u used buffers obtained in a %u size SVQ",
> +                        i, svq->vring.num);
> +                    virtqueue_fill(vq, elem, elem->len, i);
> +                    virtqueue_flush(vq, i);
> +                    i = 0;
> +                }
> +                virtqueue_fill(vq, elem, elem->len, i++);
> +            }
>          }
>
> -        virtqueue_flush(vq, i);
> -        event_notifier_set(&svq->svq_call);
> +        if (i > 0) {
> +            virtqueue_flush(vq, i);
> +            event_notifier_set(&svq->svq_call);
> +        }
>
>          if (check_for_avail_queue && svq->next_guest_avail_elem) {
>              /*
> @@ -704,14 +804,14 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq)
>      for (unsigned i = 0; i < svq->vring.num; ++i) {
>          g_autofree SVQElement *svq_elem = NULL;
>          svq_elem = g_steal_pointer(&svq->ring_id_maps[i]);
> -        if (svq_elem)  {
> +        if (svq_elem && !svq_elem->not_from_guest)  {
>              virtqueue_detach_element(svq->vq, &svq_elem->elem,
>                                       svq_elem->elem.len);
>          }
>      }
>
>      next_avail_elem = g_steal_pointer(&svq->next_guest_avail_elem);
> -    if (next_avail_elem) {
> +    if (next_avail_elem && !next_avail_elem->not_from_guest) {
>          virtqueue_detach_element(svq->vq, &next_avail_elem->elem,
>                                   next_avail_elem->elem.len);
>      }
> --
> 2.27.0
>
>

Re: [RFC PATCH 0/9] Net Control VQ support in vDPA SVQ

[Eugenio Perez Martin]

On Mon, Feb 14, 2022 at 8:38 PM Eugenio Pérez <eperezma@redhat.com> wrote:
>
> Control virtqueue is used by networking device for accepting various
> commands from the driver. It's a must to support multiqueue and other
> configurations.
>
> Shadow VirtQueue (SVQ) [1] already makes possible migration of virtqueue
> states, effectively intercepting them so qemu can track what regions of memory
> are dirty because device action and needs migration. However, this does not
> solve networking device state seen by the driver because CVQ messages, like
> changes on MAC addresses from the driver.
>
> To solve that, this series uses SVQ infraestructure proposed at SVQ [1] to
> intercept networking control messages used by the device. This way, qemu is
> able to update VirtIONet device model and to migrate it. This series needs to
> be applied on top of [1].
>
> Ideally, only the control VQ would be shadowed for all the run of qemu and the
> rest of virtqueues would be passthrough unless it's migration time. However,
> this requires the vDPA device to support address translations from more than
> one address space, something that is not possible at the moment with the
> current vhost-vDPA API. The API change has been proposed at [2], but use of it
> is left for future series.
>
> Sending this as a RFC so some details like error control is still not 100%
> tested. Comments are welcomed on every aspect of the patch.
>
> [1] https://lore.kernel.org/qemu-devel/20220121202733.404989-1-eperezma@redhat.com/
> [2] https://lkml.org/lkml/2020/9/23/1243
>
> Eugenio Pérez (9):
>   virtio-net: Expose ctrl virtqueue logic
>   vdpa: Extract get geatures part from vhost_vdpa_get_max_queue_pairs
>   virtio: Make virtqueue_alloc_element non-static
>   vhost: Add SVQElement
>   vhost: Add custom used buffer callback
>   vdpa: Add map/unmap operation callback to SVQ
>   vhost: Add vhost_svq_inject
>   vhost: Add vhost_svq_start_op
>   vdpa: control virtqueue support on shadow virtqueue
>
>  hw/virtio/vhost-shadow-virtqueue.h |  25 +++-
>  include/hw/virtio/vhost-vdpa.h     |   2 +
>  include/hw/virtio/virtio-net.h     |   3 +
>  include/hw/virtio/virtio.h         |   1 +
>  hw/net/virtio-net.c                |  83 ++++++-----
>  hw/virtio/vhost-shadow-virtqueue.c | 217 +++++++++++++++++++++++------
>  hw/virtio/vhost-vdpa.c             |  22 ++-
>  hw/virtio/virtio.c                 |   2 +-
>  net/vhost-vdpa.c                   | 140 +++++++++++++++++--
>  9 files changed, 405 insertions(+), 90 deletions(-)
>

This series is available to test at [1].

Thanks!

[1] https://github.com/eugpermar/qemu/releases/tag/vdpa_sw_live_migration.d%2Fcvq-v1

> --
> 2.27.0
>
>
>