From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35710) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YYUwf-0000Ws-Vl for qemu-devel@nongnu.org; Thu, 19 Mar 2015 03:32:47 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YYUwc-0000fZ-0B for qemu-devel@nongnu.org; Thu, 19 Mar 2015 03:32:45 -0400 Date: Thu, 19 Mar 2015 08:32:31 +0100 From: "Michael S. Tsirkin" Message-ID: <20150319083220-mutt-send-email-mst@redhat.com> References: <1426671309-13645-1-git-send-email-jasowang@redhat.com> <20150318135804-mutt-send-email-mst@redhat.com> <1426742693.5002.5@smtp.corp.redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1426742693.5002.5@smtp.corp.redhat.com> Subject: Re: [Qemu-devel] [PATCH V4 00/19] Support more virtio queues List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Jason Wang Cc: Kevin Wolf , Amit Shah , Alexander Graf , qemu-devel@nongnu.org, Keith Busch , Christian Borntraeger , qemu-ppc@nongnu.org, Stefan Hajnoczi , cornelia.huck@de.ibm.com, Paolo Bonzini , Richard Henderson On Thu, Mar 19, 2015 at 01:24:53PM +0800, Jason Wang wrote: > > > On Wed, Mar 18, 2015 at 8:58 PM, Michael S. Tsirkin wrote: > >On Wed, Mar 18, 2015 at 05:34:50PM +0800, Jason Wang wrote: > >> We current limit the max virtio queues to 64. This is not sufficient > >> to support multiqueue devices (e.g recent Linux support up to 256 > >> tap queues). So this series tries to let virtio to support more queues. > >> No much works need to be done except: > >> - Introducing transport specific queue limitation. > >> - Let each virtio transport to use specific limit. > >> - Speedup the MSI-X masking and unmasking through per vector queue > >> list, and increase the maximum MSI-X vectors supported by qemu. > >> - With the above enhancements, increase the maximum number of > >> virtqueues supported by PCI from 64 to 513. > >> - Compat the changes for legacy machine types. > > > >What are the compatibility considerations here? > > Two considerations: > 1) To keep msix bar size to 4K for legacy machine types > 2) Limit the pci queue max to 64 for legacy machine types 2 seems not relevant to me. > > > > > >> With this patch, we can support up to 256 queues. Since x86 can only > >> allow about 240 interrupt vectors for MSI-X, current Linux driver can > >> only have about 80 queue pairs has their private MSI-X interrupt > >> vectors. With sharing IRQ with queue pairs (RFC posted in > >> https://lkml.org/lkml/2014/12/25/169), Linux driver can have up > >> to about 186 queue pairs has their private MSI-X interrupt vectors. > >> Stress/migration test on virtio-pci, compile test on other > >> targets. And make check on s390x-softmmu and ppc64-softmmu. > >> Cc: Paolo Bonzini > >> Cc: Richard Henderson > >> Cc: Michael S. Tsirkin > >> Cc: Alexander Graf > >> Cc: Keith Busch > >> Cc: Kevin Wolf > >> Cc: Stefan Hajnoczi > >> Cc: Christian Borntraeger > >> Cc: Cornelia Huck > >> Cc: Amit Shah > >> Cc: qemu-ppc@nongnu.org > >> Please review > >> Thanks > >> Changes from V3: > >> - rebase to master and target to 2.4 > >> - handling compat issues for spapr > >> - fixes for hmp command completion when we have a nic with 256 queues > >> - using VIRTIO_NO_VECTOR instead of 0 for invalid virtqueue in > >> virtio-ccw > >> - fix compile issues for ppc64-softmmu > >> - don't export VIRTIO_CCW_QUEUE_MAX by introducing a gsi limit and > >> inherit in ccw > >> - use transport specific queue limit in virtio-serial > >> - correct the stale comment for AdapterRoutes and move it to ccw patch > >> - replace 128 with queue_max + 64 and add a comment for this in > >> virtio_ccw_notify() > >> Changes from V2: > >> - move transport specific limitation to their implementation. (The > >> left is VIRTIO_CCW_QUEUE_MAX since I fail to find a common header > >> files other than virtio.h) > >> - use virtio_get_queue_max() if possible in virtio.c > >> - AdapterRoutes should use ccw limit > >> - introduce a vector to queue mapping for virito devices and speedup > >> the MSI-X masking and unmasking through this. > >> Changes from V1: > >> - add a validation against the bus limitation > >> - switch to use a bus specific queue limit instead of a global one, > >> this will allow us to just increase the limit of one transport > >> without disturbing others. > >> - only increase the queue limit of virtio-pci > >> - limit the maximum number of virtio queues to 64 for legacy machine > >> types > >> Jason Wang (19): > >> pc: add 2.4 machine types > >> spapr: add machine type specific instance init function > >> ppc: spapr: add 2.4 machine type > >> monitor: replace the magic number 255 with MAX_QUEUE_NUM > >> monitor: check return value of qemu_find_net_clients_except() > >> virtio-ccw: using VIRTIO_NO_VECTOR instead of 0 for invalid virtqueue > >> virtio-net: validate backend queue numbers against bus limitation > >> virtio-net: fix the upper bound when trying to delete queues > >> virito: introduce bus specific queue limit > >> virtio-ccw: introduce ccw specific queue limit > >> virtio-s390: switch to bus specific queue limit > >> virtio-mmio: switch to bus specific queue limit > >> virtio-pci: switch to use bus specific queue limit > >> virtio: introduce vector to virtqueues mapping > >> virtio: introduce virtio_queue_get_index() > >> virtio-pci: speedup MSI-X masking and unmasking > >> virtio-pci: increase the maximum number of virtqueues to 513 > >> pci: remove hard-coded bar size in msix_init_exclusive_bar() > >> virtio-pci: introduce auto_msix_bar_size property > >> hw/block/nvme.c | 2 +- > >> hw/char/virtio-serial-bus.c | 2 +- > >> hw/i386/pc_piix.c | 42 ++++++++++++++++++++--- > >> hw/i386/pc_q35.c | 39 +++++++++++++++++++-- > >> hw/misc/ivshmem.c | 2 +- > >> hw/net/virtio-net.c | 9 ++++- > >> hw/pci/msix.c | 18 ++++------ > >> hw/ppc/spapr.c | 70 > >>+++++++++++++++++++++++++++++++++++-- > >> hw/s390x/s390-virtio-bus.c | 7 ++-- > >> hw/s390x/s390-virtio-ccw.c | 7 ++-- > >> hw/s390x/virtio-ccw.c | 20 +++++++---- > >> hw/scsi/virtio-scsi.c | 4 +-- > >> hw/virtio/virtio-mmio.c | 7 ++-- > >> hw/virtio/virtio-pci.c | 78 > >>++++++++++++++++++++++++++++-------------- > >> hw/virtio/virtio-pci.h | 3 ++ > >> hw/virtio/virtio.c | 75 > >>+++++++++++++++++++++++++++++++--------- > >> include/hw/compat.h | 11 ++++++ > >> include/hw/pci/msix.h | 2 +- > >> include/hw/s390x/s390_flic.h | 4 ++- > >> include/hw/virtio/virtio-bus.h | 2 ++ > >> include/hw/virtio/virtio.h | 7 ++-- > >> monitor.c | 25 ++++++++------ > >> 22 files changed, 339 insertions(+), 97 deletions(-) > >> -- 2.1.0