[Qemu-devel] [PATCH V8 0/9] Support more virtio queues

[Qemu-devel] [PATCH V8 0/9] Support more virtio queues

[Jason Wang]

We current limit the max virtio queues to 64. This is not sufficient
to support multiqueue devices (e.g recent Linux support up to 256
tap queues). So this series tries to let virtio to support more
queues.

No much works need to be done except:

- Increase the limit from 64 to 1024
- Let device_plugged() fail for virtio-ccw and s390-virtio when trying
  to plug virtio device with more than 64 virt queues. Since they
  currently don't support this.

With this patch, we can support up to 256 queues. Since x86 can only
allow about 240 interrupt vectors for MSI-X, current Linux driver can
only have about 80 queue pairs has their private MSI-X interrupt
vectors. With sharing IRQ with queue pairs (RFC posted in
https://lkml.org/lkml/2014/12/25/169), Linux driver can have up
to about 186 queue pairs has their private MSI-X interrupt vectors.

Test:
- Stress test on virtio-pci with x86_64-softmmu.
- Test s390-virtio and virtio-ccw can't start a virtio-net with more
  than 64 virtqueues.
- Compile test for other

Cc: Amit Shah <amit.shah@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Alexander Graf <agraf@suse.de>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Cornelia Huck <cornelia.huck@de.ibm.com>
Cc: Christian Borntraeger <borntraeger@de.ibm.com>

Please review

Thanks

Changes from V7:
- Remove the patches that has been accepted.
- Adding all virtqueues in .realize() for virtio-net
- Let device_plugged() can fail
- Incrase the global limit from 64 to 1024 while let virtio-ccw and
  s390-virtio's device_plugged() fail when the number of queues is
  greater than 64
- Drop the migration capability stuffs.

Changes from V6:
- Tweak some commit logs
- Replace the missed VIRTIO_PCI_MAX in ccw with a bus specific limit
- Bump the pci queue limit from 513 to 1024

Changes from V5:
- Rebase to HEAD
- Drop the patch that introduce helper to get vq index since there has
  already one
- Don't try to solve the migration compatibility issue for invalid
  configuration like vectors > 128 with only 64 virto queues. This is
  done by let msix calaucalte the correct bar size and drop the patch
  18 that intrdouces the auto_msix_bar_size.
- Cleanup the bar size and pba offset calcuation code
- Add a comment to explain the new virtio pci queue limit number 513

Changes from V4:
- Drop the patch of bus limitation validation and send it as an
  independent patch
- Don't let the virtio-pci to calculate the MSI-X bar size, instead,
  tell the msix code whether or not to use legacy bar size and let it
  to do this
- Make sure the MSI-X bar size is at least 4K
- Subject typos fixes for patch 8
- Mention adapter routes changes in the commit log of patch 9
- Only use the vector to queue list for virtio-pci to avoid the
  unnecessary possible overhead on other transport.

Changes from V3:
- rebase to master and target to 2.4
- handling compat issues for spapr
- fixes for hmp command completion when we have a nic with 256 queues
- using VIRTIO_NO_VECTOR instead of 0 for invalid virtqueue in
  virtio-ccw
- fix compile issues for ppc64-softmmu
- don't export VIRTIO_CCW_QUEUE_MAX by introducing a gsi limit and
  inherit in ccw
- use transport specific queue limit in virtio-serial
- correct the stale comment for AdapterRoutes and move it to ccw patch
- replace 128 with queue_max + 64 and add a comment for this in
  virtio_ccw_notify()

Changes from V2:
- move transport specific limitation to their implementation. (The
  left is VIRTIO_CCW_QUEUE_MAX since I fail to find a common header
  files other than virtio.h)
- use virtio_get_queue_ma) if possible in virtio.c
- AdapterRoutes should use ccw limit
- introduce a vector to queue mapping for virito devices and speedup
  the MSI-X masking and unmasking through this.

Changes from V1:
- add a validation against the bus limitation
- switch to use a bus specific queue limit instead of a global one,
  this will allow us to just increase the limit of one transport
  without disturbing others.
- only increase the queue limit of virtio-pci
- limit the maximum number of virtio queues to 64 for legacy machine
  types

Jason Wang (9):
  virtio-net: adding all queues in .realize()
  virtio: device_plugged() can fail
  virtio: introduce virtio_get_num_queues()
  virtio-ccw: introduce ccw specific queue limit
  virtio-ccw: validate the number of queues against bus limitation
  virtio-s390: introduce virito s390 queue limit
  virtio-s390: introduce virtio_s390_device_plugged()
  virtio: rename VIRTIO_PCI_QUEUE_MAX to VIRTIO_QUEUE_MAX
  virtio: increase the queue limit to 1024

 hw/char/virtio-serial-bus.c    |  2 +-
 hw/net/virtio-net.c            | 63 +++++++++++++-----------------------------
 hw/s390x/s390-virtio-bus.c     | 20 ++++++++++++--
 hw/s390x/s390-virtio-ccw.c     |  2 +-
 hw/s390x/virtio-ccw.c          | 23 ++++++++++-----
 hw/scsi/virtio-scsi.c          |  4 +--
 hw/virtio/virtio-bus.c         |  6 ++--
 hw/virtio/virtio-mmio.c        |  6 ++--
 hw/virtio/virtio-pci.c         | 12 ++++----
 hw/virtio/virtio.c             | 46 ++++++++++++++++++++----------
 include/hw/s390x/s390_flic.h   |  5 +++-
 include/hw/virtio/virtio-bus.h |  4 +--
 include/hw/virtio/virtio.h     |  3 +-
 13 files changed, 108 insertions(+), 88 deletions(-)

-- 
2.1.4

[Qemu-devel] [PATCH V8 1/9] virtio-net: adding all queues in .realize()

[Jason Wang]

Instead of adding queues for multiqueue during feature set. This patch
did this in .realize(), this will help the following patches that
count the number of virtqueues used in .device_plugged() callback.

Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 hw/net/virtio-net.c | 59 +++++++++++++++--------------------------------------
 1 file changed, 17 insertions(+), 42 deletions(-)

diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 3af6faf..3edb1c4 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -1301,39 +1301,8 @@ static void virtio_net_tx_bh(void *opaque)
 
 static void virtio_net_set_multiqueue(VirtIONet *n, int multiqueue)
 {
-    VirtIODevice *vdev = VIRTIO_DEVICE(n);
-    int i, max = multiqueue ? n->max_queues : 1;
-
     n->multiqueue = multiqueue;
 
-    for (i = 2; i < n->max_queues * 2 + 1; i++) {
-        virtio_del_queue(vdev, i);
-    }
-
-    for (i = 1; i < max; i++) {
-        n->vqs[i].rx_vq = virtio_add_queue(vdev, 256, virtio_net_handle_rx);
-        if (n->vqs[i].tx_timer) {
-            n->vqs[i].tx_vq =
-                virtio_add_queue(vdev, 256, virtio_net_handle_tx_timer);
-            n->vqs[i].tx_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL,
-                                                   virtio_net_tx_timer,
-                                                   &n->vqs[i]);
-        } else {
-            n->vqs[i].tx_vq =
-                virtio_add_queue(vdev, 256, virtio_net_handle_tx_bh);
-            n->vqs[i].tx_bh = qemu_bh_new(virtio_net_tx_bh, &n->vqs[i]);
-        }
-
-        n->vqs[i].tx_waiting = 0;
-        n->vqs[i].n = n;
-    }
-
-    /* Note: Minux Guests (version 3.2.1) use ctrl vq but don't ack
-     * VIRTIO_NET_F_CTRL_VQ. Create ctrl vq unconditionally to avoid
-     * breaking them.
-     */
-    n->ctrl_vq = virtio_add_queue(vdev, 64, virtio_net_handle_ctrl);
-
     virtio_net_set_queues(n);
 }
 
@@ -1594,9 +1563,7 @@ static void virtio_net_device_realize(DeviceState *dev, Error **errp)
         return;
     }
     n->vqs = g_malloc0(sizeof(VirtIONetQueue) * n->max_queues);
-    n->vqs[0].rx_vq = virtio_add_queue(vdev, 256, virtio_net_handle_rx);
     n->curr_queues = 1;
-    n->vqs[0].n = n;
     n->tx_timeout = n->net_conf.txtimer;
 
     if (n->net_conf.tx && strcmp(n->net_conf.tx, "timer")
@@ -1607,16 +1574,24 @@ static void virtio_net_device_realize(DeviceState *dev, Error **errp)
         error_report("Defaulting to \"bh\"");
     }
 
-    if (n->net_conf.tx && !strcmp(n->net_conf.tx, "timer")) {
-        n->vqs[0].tx_vq = virtio_add_queue(vdev, 256,
-                                           virtio_net_handle_tx_timer);
-        n->vqs[0].tx_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL, virtio_net_tx_timer,
-                                               &n->vqs[0]);
-    } else {
-        n->vqs[0].tx_vq = virtio_add_queue(vdev, 256,
-                                           virtio_net_handle_tx_bh);
-        n->vqs[0].tx_bh = qemu_bh_new(virtio_net_tx_bh, &n->vqs[0]);
+    for (i = 0; i < n->max_queues; i++) {
+        n->vqs[i].rx_vq = virtio_add_queue(vdev, 256, virtio_net_handle_rx);
+        if (n->net_conf.tx && !strcmp(n->net_conf.tx, "timer")) {
+            n->vqs[i].tx_vq =
+                virtio_add_queue(vdev, 256, virtio_net_handle_tx_timer);
+            n->vqs[i].tx_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL,
+                                              virtio_net_tx_timer,
+                                              &n->vqs[i]);
+        } else {
+            n->vqs[i].tx_vq =
+                virtio_add_queue(vdev, 256, virtio_net_handle_tx_bh);
+            n->vqs[i].tx_bh = qemu_bh_new(virtio_net_tx_bh, &n->vqs[i]);
+        }
+
+        n->vqs[i].tx_waiting = 0;
+        n->vqs[i].n = n;
     }
+
     n->ctrl_vq = virtio_add_queue(vdev, 64, virtio_net_handle_ctrl);
     qemu_macaddr_default_if_unset(&n->nic_conf.macaddr);
     memcpy(&n->mac[0], &n->nic_conf.macaddr, sizeof(n->mac));
-- 
2.1.4

[Qemu-devel] [PATCH V8 2/9] virtio: device_plugged() can fail

[Jason Wang]

This patch passes error pointer to transport specific device_plugged()
callback. Through this way, device_plugged() can do some transport
specific check and fail. This will be uesd by following patches that
check the number of virtqueues against the transport limitation.

Cc: Cornelia Huck <cornelia.huck@de.ibm.com>
Cc: Christian Borntraeger <borntraeger@de.ibm.com>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Alexander Graf <agraf@suse.de>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 hw/s390x/virtio-ccw.c          | 2 +-
 hw/virtio/virtio-bus.c         | 6 ++----
 hw/virtio/virtio-mmio.c        | 2 +-
 hw/virtio/virtio-pci.c         | 2 +-
 hw/virtio/virtio.c             | 7 ++++++-
 include/hw/virtio/virtio-bus.h | 4 ++--
 6 files changed, 13 insertions(+), 10 deletions(-)

diff --git a/hw/s390x/virtio-ccw.c b/hw/s390x/virtio-ccw.c
index c96101a..994284c 100644
--- a/hw/s390x/virtio-ccw.c
+++ b/hw/s390x/virtio-ccw.c
@@ -1413,7 +1413,7 @@ static int virtio_ccw_load_config(DeviceState *d, QEMUFile *f)
 }
 
 /* This is called by virtio-bus just after the device is plugged. */
-static void virtio_ccw_device_plugged(DeviceState *d)
+static void virtio_ccw_device_plugged(DeviceState *d, Error **errp)
 {
     VirtioCcwDevice *dev = VIRTIO_CCW_DEVICE(d);
     SubchDev *sch = dev->sch;
diff --git a/hw/virtio/virtio-bus.c b/hw/virtio/virtio-bus.c
index be886e7..ceabe99 100644
--- a/hw/virtio/virtio-bus.c
+++ b/hw/virtio/virtio-bus.c
@@ -38,7 +38,7 @@ do { printf("virtio_bus: " fmt , ## __VA_ARGS__); } while (0)
 #endif
 
 /* A VirtIODevice is being plugged */
-int virtio_bus_device_plugged(VirtIODevice *vdev)
+void virtio_bus_device_plugged(VirtIODevice *vdev, Error **errp)
 {
     DeviceState *qdev = DEVICE(vdev);
     BusState *qbus = BUS(qdev_get_parent_bus(qdev));
@@ -47,10 +47,8 @@ int virtio_bus_device_plugged(VirtIODevice *vdev)
     DPRINTF("%s: plug device.\n", qbus->name);
 
     if (klass->device_plugged != NULL) {
-        klass->device_plugged(qbus->parent);
+        klass->device_plugged(qbus->parent, errp);
     }
-
-    return 0;
 }
 
 /* Reset the virtio_bus */
diff --git a/hw/virtio/virtio-mmio.c b/hw/virtio/virtio-mmio.c
index 10123f3..dd3f80b 100644
--- a/hw/virtio/virtio-mmio.c
+++ b/hw/virtio/virtio-mmio.c
@@ -345,7 +345,7 @@ static void virtio_mmio_reset(DeviceState *d)
 /* virtio-mmio device */
 
 /* This is called by virtio-bus just after the device is plugged. */
-static void virtio_mmio_device_plugged(DeviceState *opaque)
+static void virtio_mmio_device_plugged(DeviceState *opaque, Error **errp)
 {
     VirtIOMMIOProxy *proxy = VIRTIO_MMIO(opaque);
 
diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
index 867c9d1..b88c8e0 100644
--- a/hw/virtio/virtio-pci.c
+++ b/hw/virtio/virtio-pci.c
@@ -918,7 +918,7 @@ static int virtio_pci_query_nvectors(DeviceState *d)
 }
 
 /* This is called by virtio-bus just after the device is plugged. */
-static void virtio_pci_device_plugged(DeviceState *d)
+static void virtio_pci_device_plugged(DeviceState *d, Error **errp)
 {
     VirtIOPCIProxy *proxy = VIRTIO_PCI(d);
     VirtioBusState *bus = &proxy->bus;
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index 6985e76..0b50f9d 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -1328,7 +1328,12 @@ static void virtio_device_realize(DeviceState *dev, Error **errp)
             return;
         }
     }
-    virtio_bus_device_plugged(vdev);
+
+    virtio_bus_device_plugged(vdev, &err);
+    if (err != NULL) {
+        error_propagate(errp, err);
+        return;
+    }
 }
 
 static void virtio_device_unrealize(DeviceState *dev, Error **errp)
diff --git a/include/hw/virtio/virtio-bus.h b/include/hw/virtio/virtio-bus.h
index a4588ca..e5bbfbf 100644
--- a/include/hw/virtio/virtio-bus.h
+++ b/include/hw/virtio/virtio-bus.h
@@ -56,7 +56,7 @@ typedef struct VirtioBusClass {
      * transport independent init function.
      * This is called by virtio-bus just after the device is plugged.
      */
-    void (*device_plugged)(DeviceState *d);
+    void (*device_plugged)(DeviceState *d, Error **errp);
     /*
      * transport independent exit function.
      * This is called by virtio-bus just before the device is unplugged.
@@ -75,7 +75,7 @@ struct VirtioBusState {
     BusState parent_obj;
 };
 
-int virtio_bus_device_plugged(VirtIODevice *vdev);
+void virtio_bus_device_plugged(VirtIODevice *vdev, Error **errp);
 void virtio_bus_reset(VirtioBusState *bus);
 void virtio_bus_device_unplugged(VirtIODevice *bus);
 /* Get the device id of the plugged device. */
-- 
2.1.4

[Qemu-devel] [PATCH V8 3/9] virtio: introduce virtio_get_num_queues()

[Jason Wang]

This patch introduces virtio_get_num_queues() which iterates the vqs
array and return the number of virtqueues used by device.

Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 hw/virtio/virtio.c         | 13 +++++++++++++
 include/hw/virtio/virtio.h |  1 +
 2 files changed, 14 insertions(+)

diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index 0b50f9d..81d8905 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -746,6 +746,19 @@ int virtio_queue_get_num(VirtIODevice *vdev, int n)
     return vdev->vq[n].vring.num;
 }
 
+int virtio_get_num_queues(VirtIODevice *vdev)
+{
+    int i;
+
+    for (i = 0; i < VIRTIO_PCI_QUEUE_MAX; i++) {
+        if (!virtio_queue_get_num(vdev, i)) {
+            break;
+        }
+    }
+
+    return i;
+}
+
 int virtio_queue_get_id(VirtQueue *vq)
 {
     VirtIODevice *vdev = vq->vdev;
diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index 8210cb3..175195d 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -175,6 +175,7 @@ void virtio_queue_set_addr(VirtIODevice *vdev, int n, hwaddr addr);
 hwaddr virtio_queue_get_addr(VirtIODevice *vdev, int n);
 void virtio_queue_set_num(VirtIODevice *vdev, int n, int num);
 int virtio_queue_get_num(VirtIODevice *vdev, int n);
+int virtio_get_num_queues(VirtIODevice *vdev);
 void virtio_queue_set_align(VirtIODevice *vdev, int n, int align);
 void virtio_queue_notify(VirtIODevice *vdev, int n);
 uint16_t virtio_queue_vector(VirtIODevice *vdev, int n);
-- 
2.1.4

[Qemu-devel] [PATCH V8 4/9] virtio-ccw: introduce ccw specific queue limit

[Jason Wang]

Cc: Alexander Graf <agraf@suse.de>
Cc: Cornelia Huck <cornelia.huck@de.ibm.com>
Cc: Christian Borntraeger <borntraeger@de.ibm.com>
Cc: Richard Henderson <rth@twiddle.net>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 hw/s390x/s390-virtio-ccw.c   |  2 +-
 hw/s390x/virtio-ccw.c        | 12 ++++++------
 include/hw/s390x/s390_flic.h |  5 ++++-
 3 files changed, 11 insertions(+), 8 deletions(-)

diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index afb539a..8a565f6 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -67,7 +67,7 @@ static int virtio_ccw_hcall_notify(const uint64_t *args)
     if (!sch || !css_subch_visible(sch)) {
         return -EINVAL;
     }
-    if (queue >= VIRTIO_PCI_QUEUE_MAX) {
+    if (queue >= VIRTIO_CCW_QUEUE_MAX) {
         return -EINVAL;
     }
     virtio_queue_notify(virtio_ccw_get_vdev(sch), queue);
diff --git a/hw/s390x/virtio-ccw.c b/hw/s390x/virtio-ccw.c
index 994284c..aaa9833 100644
--- a/hw/s390x/virtio-ccw.c
+++ b/hw/s390x/virtio-ccw.c
@@ -170,7 +170,7 @@ static void virtio_ccw_start_ioeventfd(VirtioCcwDevice *dev)
         return;
     }
     vdev = virtio_bus_get_device(&dev->bus);
-    for (n = 0; n < VIRTIO_PCI_QUEUE_MAX; n++) {
+    for (n = 0; n < VIRTIO_CCW_QUEUE_MAX; n++) {
         if (!virtio_queue_get_num(vdev, n)) {
             continue;
         }
@@ -205,7 +205,7 @@ static void virtio_ccw_stop_ioeventfd(VirtioCcwDevice *dev)
         return;
     }
     vdev = virtio_bus_get_device(&dev->bus);
-    for (n = 0; n < VIRTIO_PCI_QUEUE_MAX; n++) {
+    for (n = 0; n < VIRTIO_CCW_QUEUE_MAX; n++) {
         if (!virtio_queue_get_num(vdev, n)) {
             continue;
         }
@@ -266,7 +266,7 @@ static int virtio_ccw_set_vqs(SubchDev *sch, uint64_t addr, uint32_t align,
 {
     VirtIODevice *vdev = virtio_ccw_get_vdev(sch);
 
-    if (index >= VIRTIO_PCI_QUEUE_MAX) {
+    if (index >= VIRTIO_CCW_QUEUE_MAX) {
         return -EINVAL;
     }
 
@@ -291,7 +291,7 @@ static int virtio_ccw_set_vqs(SubchDev *sch, uint64_t addr, uint32_t align,
         virtio_queue_set_vector(vdev, index, index);
     }
     /* tell notify handler in case of config change */
-    vdev->config_vector = VIRTIO_PCI_QUEUE_MAX;
+    vdev->config_vector = VIRTIO_CCW_QUEUE_MAX;
     return 0;
 }
 
@@ -573,7 +573,7 @@ static int virtio_ccw_cb(SubchDev *sch, CCW1 ccw)
                                                     ccw.cda,
                                                     MEMTXATTRS_UNSPECIFIED,
                                                     NULL);
-            if (vq_config.index >= VIRTIO_PCI_QUEUE_MAX) {
+            if (vq_config.index >= VIRTIO_CCW_QUEUE_MAX) {
                 ret = -EINVAL;
                 break;
             }
@@ -1052,7 +1052,7 @@ static void virtio_ccw_notify(DeviceState *d, uint16_t vector)
         return;
     }
 
-    if (vector < VIRTIO_PCI_QUEUE_MAX) {
+    if (vector < VIRTIO_CCW_QUEUE_MAX) {
         if (!dev->indicators) {
             return;
         }
diff --git a/include/hw/s390x/s390_flic.h b/include/hw/s390x/s390_flic.h
index 489d73b..200e7e9 100644
--- a/include/hw/s390x/s390_flic.h
+++ b/include/hw/s390x/s390_flic.h
@@ -17,10 +17,13 @@
 #include "hw/s390x/adapter.h"
 #include "hw/virtio/virtio.h"
 
+#define ADAPTER_ROUTES_MAX_GSI 64
+#define VIRTIO_CCW_QUEUE_MAX ADAPTER_ROUTES_MAX_GSI
+
 typedef struct AdapterRoutes {
     AdapterInfo adapter;
     int num_routes;
-    int gsi[VIRTIO_PCI_QUEUE_MAX];
+    int gsi[ADAPTER_ROUTES_MAX_GSI];
 } AdapterRoutes;
 
 #define TYPE_S390_FLIC_COMMON "s390-flic"
-- 
2.1.4

[Qemu-devel] [PATCH V8 5/9] virtio-ccw: validate the number of queues against bus limitation

[Jason Wang]

Cc: Cornelia Huck <cornelia.huck@de.ibm.com>
Cc: Christian Borntraeger <borntraeger@de.ibm.com>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Alexander Graf <agraf@suse.de>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 hw/s390x/virtio-ccw.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/hw/s390x/virtio-ccw.c b/hw/s390x/virtio-ccw.c
index aaa9833..18fc697 100644
--- a/hw/s390x/virtio-ccw.c
+++ b/hw/s390x/virtio-ccw.c
@@ -1416,7 +1416,16 @@ static int virtio_ccw_load_config(DeviceState *d, QEMUFile *f)
 static void virtio_ccw_device_plugged(DeviceState *d, Error **errp)
 {
     VirtioCcwDevice *dev = VIRTIO_CCW_DEVICE(d);
+    VirtIODevice *vdev = virtio_bus_get_device(&dev->bus);
     SubchDev *sch = dev->sch;
+    int n = virtio_get_num_queues(vdev);
+
+    if (virtio_get_num_queues(vdev) > VIRTIO_CCW_QUEUE_MAX) {
+        error_setg(errp, "The nubmer of virtqueues %d "
+                   "exceeds ccw limit %d", n,
+                   VIRTIO_CCW_QUEUE_MAX);
+        return;
+    }
 
     sch->id.cu_model = virtio_bus_get_vdev_id(&dev->bus);
 
-- 
2.1.4

[Qemu-devel] [PATCH V8 6/9] virtio-s390: introduce virito s390 queue limit

[Jason Wang]

Cc: Alexander Graf <agraf@suse.de>
Cc: Richard Henderson <rth@twiddle.net>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 hw/s390x/s390-virtio-bus.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/hw/s390x/s390-virtio-bus.c b/hw/s390x/s390-virtio-bus.c
index 0e35ac9..3801c73 100644
--- a/hw/s390x/s390-virtio-bus.c
+++ b/hw/s390x/s390-virtio-bus.c
@@ -45,6 +45,8 @@
     do { } while (0)
 #endif
 
+#define VIRTIO_S390_QUEUE_MAX 64
+
 static void virtio_s390_bus_new(VirtioBusState *bus, size_t bus_size,
                                 VirtIOS390Device *dev);
 
@@ -354,7 +356,7 @@ static ram_addr_t s390_virtio_device_num_vq(VirtIOS390Device *dev)
     VirtIODevice *vdev = dev->vdev;
     int num_vq;
 
-    for (num_vq = 0; num_vq < VIRTIO_PCI_QUEUE_MAX; num_vq++) {
+    for (num_vq = 0; num_vq < VIRTIO_S390_QUEUE_MAX; num_vq++) {
         if (!virtio_queue_get_num(vdev, num_vq)) {
             break;
         }
@@ -476,7 +478,7 @@ VirtIOS390Device *s390_virtio_bus_find_vring(VirtIOS390Bus *bus,
     QTAILQ_FOREACH(kid, &bus->bus.children, sibling) {
         VirtIOS390Device *dev = (VirtIOS390Device *)kid->child;
 
-        for(i = 0; i < VIRTIO_PCI_QUEUE_MAX; i++) {
+        for (i = 0; i < VIRTIO_S390_QUEUE_MAX; i++) {
             if (!virtio_queue_get_addr(dev->vdev, i))
                 break;
             if (virtio_queue_get_addr(dev->vdev, i) == mem) {
-- 
2.1.4

[Qemu-devel] [PATCH V8 7/9] virtio-s390: introduce virtio_s390_device_plugged()

[Jason Wang]

This patch introduce a virtio-s390 specific device_plugged() function
and doing the number of virtqueue validation inside.

Cc: Alexander Graf <agraf@suse.de>
Cc: Richard Henderson <rth@twiddle.net>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 hw/s390x/s390-virtio-bus.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/hw/s390x/s390-virtio-bus.c b/hw/s390x/s390-virtio-bus.c
index 3801c73..cbb3f92 100644
--- a/hw/s390x/s390-virtio-bus.c
+++ b/hw/s390x/s390-virtio-bus.c
@@ -536,6 +536,19 @@ static unsigned virtio_s390_get_features(DeviceState *d)
     return dev->host_features;
 }
 
+static void virtio_s390_device_plugged(DeviceState *d, Error **errp)
+{
+    VirtIOS390Device *dev = to_virtio_s390_device(d);
+    VirtIODevice *vdev = virtio_bus_get_device(&dev->bus);
+    int n = virtio_get_num_queues(vdev);
+
+    if (n > VIRTIO_S390_QUEUE_MAX) {
+        error_setg(errp, "The nubmer of virtqueues %d "
+                   "exceeds s390 limit %d", n,
+                   VIRTIO_S390_QUEUE_MAX);
+    }
+}
+
 /**************** S390 Virtio Bus Device Descriptions *******************/
 
 static void s390_virtio_net_class_init(ObjectClass *klass, void *data)
@@ -736,6 +749,7 @@ static void virtio_s390_bus_class_init(ObjectClass *klass, void *data)
     bus_class->max_dev = 1;
     k->notify = virtio_s390_notify;
     k->get_features = virtio_s390_get_features;
+    k->device_plugged = virtio_s390_device_plugged;
 }
 
 static const TypeInfo virtio_s390_bus_info = {
-- 
2.1.4

[Qemu-devel] [PATCH V8 8/9] virtio: rename VIRTIO_PCI_QUEUE_MAX to VIRTIO_QUEUE_MAX

[Jason Wang]

VIRTIO_PCI_QUEUE_MAX is not only used for pci, so rename it be generic.

Cc: Amit Shah <amit.shah@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 hw/char/virtio-serial-bus.c |  2 +-
 hw/net/virtio-net.c         |  4 ++--
 hw/scsi/virtio-scsi.c       |  4 ++--
 hw/virtio/virtio-mmio.c     |  4 ++--
 hw/virtio/virtio-pci.c      | 10 +++++-----
 hw/virtio/virtio.c          | 28 ++++++++++++++--------------
 include/hw/virtio/virtio.h  |  2 +-
 7 files changed, 27 insertions(+), 27 deletions(-)

diff --git a/hw/char/virtio-serial-bus.c b/hw/char/virtio-serial-bus.c
index 6e2ad82..333339e 100644
--- a/hw/char/virtio-serial-bus.c
+++ b/hw/char/virtio-serial-bus.c
@@ -973,7 +973,7 @@ static void virtio_serial_device_realize(DeviceState *dev, Error **errp)
     }
 
     /* Each port takes 2 queues, and one pair is for the control queue */
-    max_supported_ports = VIRTIO_PCI_QUEUE_MAX / 2 - 1;
+    max_supported_ports = VIRTIO_QUEUE_MAX / 2 - 1;
 
     if (vser->serial.max_virtserial_ports > max_supported_ports) {
         error_setg(errp, "maximum ports supported: %u", max_supported_ports);
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 3edb1c4..6dca726 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -1555,10 +1555,10 @@ static void virtio_net_device_realize(DeviceState *dev, Error **errp)
     virtio_init(vdev, "virtio-net", VIRTIO_ID_NET, n->config_size);
 
     n->max_queues = MAX(n->nic_conf.peers.queues, 1);
-    if (n->max_queues * 2 + 1 > VIRTIO_PCI_QUEUE_MAX) {
+    if (n->max_queues * 2 + 1 > VIRTIO_QUEUE_MAX) {
         error_setg(errp, "Invalid number of queues (= %" PRIu32 "), "
                    "must be a positive integer less than %d.",
-                   n->max_queues, (VIRTIO_PCI_QUEUE_MAX - 1) / 2);
+                   n->max_queues, (VIRTIO_QUEUE_MAX - 1) / 2);
         virtio_cleanup(vdev);
         return;
     }
diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
index e242fef..5818159 100644
--- a/hw/scsi/virtio-scsi.c
+++ b/hw/scsi/virtio-scsi.c
@@ -830,10 +830,10 @@ void virtio_scsi_common_realize(DeviceState *dev, Error **errp,
                 sizeof(VirtIOSCSIConfig));
 
     if (s->conf.num_queues == 0 ||
-            s->conf.num_queues > VIRTIO_PCI_QUEUE_MAX - 2) {
+            s->conf.num_queues > VIRTIO_QUEUE_MAX - 2) {
         error_setg(errp, "Invalid number of queues (= %" PRIu32 "), "
                          "must be a positive integer less than %d.",
-                   s->conf.num_queues, VIRTIO_PCI_QUEUE_MAX - 2);
+                   s->conf.num_queues, VIRTIO_QUEUE_MAX - 2);
         virtio_cleanup(vdev);
         return;
     }
diff --git a/hw/virtio/virtio-mmio.c b/hw/virtio/virtio-mmio.c
index dd3f80b..bd2ee26 100644
--- a/hw/virtio/virtio-mmio.c
+++ b/hw/virtio/virtio-mmio.c
@@ -237,7 +237,7 @@ static void virtio_mmio_write(void *opaque, hwaddr offset, uint64_t value,
                 proxy->guest_page_shift);
         break;
     case VIRTIO_MMIO_QUEUESEL:
-        if (value < VIRTIO_PCI_QUEUE_MAX) {
+        if (value < VIRTIO_QUEUE_MAX) {
             vdev->queue_sel = value;
         }
         break;
@@ -257,7 +257,7 @@ static void virtio_mmio_write(void *opaque, hwaddr offset, uint64_t value,
         }
         break;
     case VIRTIO_MMIO_QUEUENOTIFY:
-        if (value < VIRTIO_PCI_QUEUE_MAX) {
+        if (value < VIRTIO_QUEUE_MAX) {
             virtio_queue_notify(vdev, value);
         }
         break;
diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
index b88c8e0..5eb5f46 100644
--- a/hw/virtio/virtio-pci.c
+++ b/hw/virtio/virtio-pci.c
@@ -171,7 +171,7 @@ static void virtio_pci_start_ioeventfd(VirtIOPCIProxy *proxy)
         return;
     }
 
-    for (n = 0; n < VIRTIO_PCI_QUEUE_MAX; n++) {
+    for (n = 0; n < VIRTIO_QUEUE_MAX; n++) {
         if (!virtio_queue_get_num(vdev, n)) {
             continue;
         }
@@ -207,7 +207,7 @@ static void virtio_pci_stop_ioeventfd(VirtIOPCIProxy *proxy)
         return;
     }
 
-    for (n = 0; n < VIRTIO_PCI_QUEUE_MAX; n++) {
+    for (n = 0; n < VIRTIO_QUEUE_MAX; n++) {
         if (!virtio_queue_get_num(vdev, n)) {
             continue;
         }
@@ -243,11 +243,11 @@ static void virtio_ioport_write(void *opaque, uint32_t addr, uint32_t val)
             virtio_queue_set_addr(vdev, vdev->queue_sel, pa);
         break;
     case VIRTIO_PCI_QUEUE_SEL:
-        if (val < VIRTIO_PCI_QUEUE_MAX)
+        if (val < VIRTIO_QUEUE_MAX)
             vdev->queue_sel = val;
         break;
     case VIRTIO_PCI_QUEUE_NOTIFY:
-        if (val < VIRTIO_PCI_QUEUE_MAX) {
+        if (val < VIRTIO_QUEUE_MAX) {
             virtio_queue_notify(vdev, val);
         }
         break;
@@ -750,7 +750,7 @@ static int virtio_pci_set_guest_notifiers(DeviceState *d, int nvqs, bool assign)
     bool with_irqfd = msix_enabled(&proxy->pci_dev) &&
         kvm_msi_via_irqfd_enabled();
 
-    nvqs = MIN(nvqs, VIRTIO_PCI_QUEUE_MAX);
+    nvqs = MIN(nvqs, VIRTIO_QUEUE_MAX);
 
     /* When deassigning, pass a consistent nvqs value
      * to avoid leaking notifiers.
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index 81d8905..7092aa5 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -600,7 +600,7 @@ void virtio_reset(void *opaque)
     vdev->config_vector = VIRTIO_NO_VECTOR;
     virtio_notify_vector(vdev, vdev->config_vector);
 
-    for(i = 0; i < VIRTIO_PCI_QUEUE_MAX; i++) {
+    for(i = 0; i < VIRTIO_QUEUE_MAX; i++) {
         vdev->vq[i].vring.desc = 0;
         vdev->vq[i].vring.avail = 0;
         vdev->vq[i].vring.used = 0;
@@ -750,7 +750,7 @@ int virtio_get_num_queues(VirtIODevice *vdev)
 {
     int i;
 
-    for (i = 0; i < VIRTIO_PCI_QUEUE_MAX; i++) {
+    for (i = 0; i < VIRTIO_QUEUE_MAX; i++) {
         if (!virtio_queue_get_num(vdev, i)) {
             break;
         }
@@ -762,7 +762,7 @@ int virtio_get_num_queues(VirtIODevice *vdev)
 int virtio_queue_get_id(VirtQueue *vq)
 {
     VirtIODevice *vdev = vq->vdev;
-    assert(vq >= &vdev->vq[0] && vq < &vdev->vq[VIRTIO_PCI_QUEUE_MAX]);
+    assert(vq >= &vdev->vq[0] && vq < &vdev->vq[VIRTIO_QUEUE_MAX]);
     return vq - &vdev->vq[0];
 }
 
@@ -798,7 +798,7 @@ void virtio_queue_notify(VirtIODevice *vdev, int n)
 
 uint16_t virtio_queue_vector(VirtIODevice *vdev, int n)
 {
-    return n < VIRTIO_PCI_QUEUE_MAX ? vdev->vq[n].vector :
+    return n < VIRTIO_QUEUE_MAX ? vdev->vq[n].vector :
         VIRTIO_NO_VECTOR;
 }
 
@@ -806,7 +806,7 @@ void virtio_queue_set_vector(VirtIODevice *vdev, int n, uint16_t vector)
 {
     VirtQueue *vq = &vdev->vq[n];
 
-    if (n < VIRTIO_PCI_QUEUE_MAX) {
+    if (n < VIRTIO_QUEUE_MAX) {
         if (vdev->vector_queues &&
             vdev->vq[n].vector != VIRTIO_NO_VECTOR) {
             QLIST_REMOVE(vq, node);
@@ -824,12 +824,12 @@ VirtQueue *virtio_add_queue(VirtIODevice *vdev, int queue_size,
 {
     int i;
 
-    for (i = 0; i < VIRTIO_PCI_QUEUE_MAX; i++) {
+    for (i = 0; i < VIRTIO_QUEUE_MAX; i++) {
         if (vdev->vq[i].vring.num == 0)
             break;
     }
 
-    if (i == VIRTIO_PCI_QUEUE_MAX || queue_size > VIRTQUEUE_MAX_SIZE)
+    if (i == VIRTIO_QUEUE_MAX || queue_size > VIRTQUEUE_MAX_SIZE)
         abort();
 
     vdev->vq[i].vring.num = queue_size;
@@ -841,7 +841,7 @@ VirtQueue *virtio_add_queue(VirtIODevice *vdev, int queue_size,
 
 void virtio_del_queue(VirtIODevice *vdev, int n)
 {
-    if (n < 0 || n >= VIRTIO_PCI_QUEUE_MAX) {
+    if (n < 0 || n >= VIRTIO_QUEUE_MAX) {
         abort();
     }
 
@@ -951,14 +951,14 @@ void virtio_save(VirtIODevice *vdev, QEMUFile *f)
     qemu_put_be32(f, vdev->config_len);
     qemu_put_buffer(f, vdev->config, vdev->config_len);
 
-    for (i = 0; i < VIRTIO_PCI_QUEUE_MAX; i++) {
+    for (i = 0; i < VIRTIO_QUEUE_MAX; i++) {
         if (vdev->vq[i].vring.num == 0)
             break;
     }
 
     qemu_put_be32(f, i);
 
-    for (i = 0; i < VIRTIO_PCI_QUEUE_MAX; i++) {
+    for (i = 0; i < VIRTIO_QUEUE_MAX; i++) {
         if (vdev->vq[i].vring.num == 0)
             break;
 
@@ -1023,7 +1023,7 @@ int virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id)
     qemu_get_8s(f, &vdev->status);
     qemu_get_8s(f, &vdev->isr);
     qemu_get_be16s(f, &vdev->queue_sel);
-    if (vdev->queue_sel >= VIRTIO_PCI_QUEUE_MAX) {
+    if (vdev->queue_sel >= VIRTIO_QUEUE_MAX) {
         return -1;
     }
     qemu_get_be32s(f, &features);
@@ -1050,7 +1050,7 @@ int virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id)
 
     num = qemu_get_be32(f);
 
-    if (num > VIRTIO_PCI_QUEUE_MAX) {
+    if (num > VIRTIO_QUEUE_MAX) {
         error_report("Invalid number of PCI queues: 0x%x", num);
         return -1;
     }
@@ -1176,9 +1176,9 @@ void virtio_init(VirtIODevice *vdev, const char *name,
     vdev->isr = 0;
     vdev->queue_sel = 0;
     vdev->config_vector = VIRTIO_NO_VECTOR;
-    vdev->vq = g_malloc0(sizeof(VirtQueue) * VIRTIO_PCI_QUEUE_MAX);
+    vdev->vq = g_malloc0(sizeof(VirtQueue) * VIRTIO_QUEUE_MAX);
     vdev->vm_running = runstate_is_running();
-    for (i = 0; i < VIRTIO_PCI_QUEUE_MAX; i++) {
+    for (i = 0; i < VIRTIO_QUEUE_MAX; i++) {
         vdev->vq[i].vector = VIRTIO_NO_VECTOR;
         vdev->vq[i].vdev = vdev;
         vdev->vq[i].queue_index = i;
diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index 175195d..8fdaa72 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -48,7 +48,7 @@ typedef struct VirtQueueElement
     struct iovec out_sg[VIRTQUEUE_MAX_SIZE];
 } VirtQueueElement;
 
-#define VIRTIO_PCI_QUEUE_MAX 64
+#define VIRTIO_QUEUE_MAX 64
 
 #define VIRTIO_NO_VECTOR 0xffff
 
-- 
2.1.4

[Qemu-devel] [PATCH V8 9/9] virtio: increase the queue limit to 1024

[Jason Wang]

Increase the queue limit to 1024. But virtio-ccw and s390-virtio won't
support this, this is done through failing device_plugged() for those
two transports if the number of virtqueues is greater than 64.

Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 include/hw/virtio/virtio.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index 8fdaa72..74c411f 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -48,7 +48,7 @@ typedef struct VirtQueueElement
     struct iovec out_sg[VIRTQUEUE_MAX_SIZE];
 } VirtQueueElement;
 
-#define VIRTIO_QUEUE_MAX 64
+#define VIRTIO_QUEUE_MAX 1024
 
 #define VIRTIO_NO_VECTOR 0xffff
 
-- 
2.1.4