From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50811) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YyDa5-0005ae-2S for qemu-devel@nongnu.org; Fri, 29 May 2015 02:15:46 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YyDa1-0004KJ-RW for qemu-devel@nongnu.org; Fri, 29 May 2015 02:15:45 -0400 Received: from mx1.redhat.com ([209.132.183.28]:37066) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YyDa1-0004K3-Jd for qemu-devel@nongnu.org; Fri, 29 May 2015 02:15:41 -0400 From: Jason Wang Date: Fri, 29 May 2015 14:15:23 +0800 Message-Id: <1432880132-28477-1-git-send-email-jasowang@redhat.com> Subject: [Qemu-devel] [PATCH V8 0/9] Support more virtio queues List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: mst@redhat.com, Jason Wang , Alexander Graf , Amit Shah , cornelia.huck@de.ibm.com, Paolo Bonzini , Christian Borntraeger , Richard Henderson We current limit the max virtio queues to 64. This is not sufficient to support multiqueue devices (e.g recent Linux support up to 256 tap queues). So this series tries to let virtio to support more queues. No much works need to be done except: - Increase the limit from 64 to 1024 - Let device_plugged() fail for virtio-ccw and s390-virtio when trying to plug virtio device with more than 64 virt queues. Since they currently don't support this. With this patch, we can support up to 256 queues. Since x86 can only allow about 240 interrupt vectors for MSI-X, current Linux driver can only have about 80 queue pairs has their private MSI-X interrupt vectors. With sharing IRQ with queue pairs (RFC posted in https://lkml.org/lkml/2014/12/25/169), Linux driver can have up to about 186 queue pairs has their private MSI-X interrupt vectors. Test: - Stress test on virtio-pci with x86_64-softmmu. - Test s390-virtio and virtio-ccw can't start a virtio-net with more than 64 virtqueues. - Compile test for other Cc: Amit Shah Cc: Paolo Bonzini Cc: Alexander Graf Cc: Richard Henderson Cc: Cornelia Huck Cc: Christian Borntraeger Please review Thanks Changes from V7: - Remove the patches that has been accepted. - Adding all virtqueues in .realize() for virtio-net - Let device_plugged() can fail - Incrase the global limit from 64 to 1024 while let virtio-ccw and s390-virtio's device_plugged() fail when the number of queues is greater than 64 - Drop the migration capability stuffs. Changes from V6: - Tweak some commit logs - Replace the missed VIRTIO_PCI_MAX in ccw with a bus specific limit - Bump the pci queue limit from 513 to 1024 Changes from V5: - Rebase to HEAD - Drop the patch that introduce helper to get vq index since there has already one - Don't try to solve the migration compatibility issue for invalid configuration like vectors > 128 with only 64 virto queues. This is done by let msix calaucalte the correct bar size and drop the patch 18 that intrdouces the auto_msix_bar_size. - Cleanup the bar size and pba offset calcuation code - Add a comment to explain the new virtio pci queue limit number 513 Changes from V4: - Drop the patch of bus limitation validation and send it as an independent patch - Don't let the virtio-pci to calculate the MSI-X bar size, instead, tell the msix code whether or not to use legacy bar size and let it to do this - Make sure the MSI-X bar size is at least 4K - Subject typos fixes for patch 8 - Mention adapter routes changes in the commit log of patch 9 - Only use the vector to queue list for virtio-pci to avoid the unnecessary possible overhead on other transport. Changes from V3: - rebase to master and target to 2.4 - handling compat issues for spapr - fixes for hmp command completion when we have a nic with 256 queues - using VIRTIO_NO_VECTOR instead of 0 for invalid virtqueue in virtio-ccw - fix compile issues for ppc64-softmmu - don't export VIRTIO_CCW_QUEUE_MAX by introducing a gsi limit and inherit in ccw - use transport specific queue limit in virtio-serial - correct the stale comment for AdapterRoutes and move it to ccw patch - replace 128 with queue_max + 64 and add a comment for this in virtio_ccw_notify() Changes from V2: - move transport specific limitation to their implementation. (The left is VIRTIO_CCW_QUEUE_MAX since I fail to find a common header files other than virtio.h) - use virtio_get_queue_ma) if possible in virtio.c - AdapterRoutes should use ccw limit - introduce a vector to queue mapping for virito devices and speedup the MSI-X masking and unmasking through this. Changes from V1: - add a validation against the bus limitation - switch to use a bus specific queue limit instead of a global one, this will allow us to just increase the limit of one transport without disturbing others. - only increase the queue limit of virtio-pci - limit the maximum number of virtio queues to 64 for legacy machine types Jason Wang (9): virtio-net: adding all queues in .realize() virtio: device_plugged() can fail virtio: introduce virtio_get_num_queues() virtio-ccw: introduce ccw specific queue limit virtio-ccw: validate the number of queues against bus limitation virtio-s390: introduce virito s390 queue limit virtio-s390: introduce virtio_s390_device_plugged() virtio: rename VIRTIO_PCI_QUEUE_MAX to VIRTIO_QUEUE_MAX virtio: increase the queue limit to 1024 hw/char/virtio-serial-bus.c | 2 +- hw/net/virtio-net.c | 63 +++++++++++++----------------------------- hw/s390x/s390-virtio-bus.c | 20 ++++++++++++-- hw/s390x/s390-virtio-ccw.c | 2 +- hw/s390x/virtio-ccw.c | 23 ++++++++++----- hw/scsi/virtio-scsi.c | 4 +-- hw/virtio/virtio-bus.c | 6 ++-- hw/virtio/virtio-mmio.c | 6 ++-- hw/virtio/virtio-pci.c | 12 ++++---- hw/virtio/virtio.c | 46 ++++++++++++++++++++---------- include/hw/s390x/s390_flic.h | 5 +++- include/hw/virtio/virtio-bus.h | 4 +-- include/hw/virtio/virtio.h | 3 +- 13 files changed, 108 insertions(+), 88 deletions(-) -- 2.1.4