Re: [Qemu-devel] [PATCH 00/12] Multiqueue virtio-net

[Anthony Liguori <aliguori@us.ibm.com>]

Jason Wang <jasowang@redhat.com> writes:

> Hello all:
>
> This seires is an update of last version of multiqueue virtio-net support.
>
> Recently, linux tap gets multiqueue support. This series implements basic
> support for multiqueue tap, nic and vhost. Then use it as an infrastructure to
> enable the multiqueue support for virtio-net.
>
> Both vhost and userspace multiqueue were implemented for virtio-net, but
> userspace could be get much benefits since dataplane like parallized mechanism
> were not implemented.
>
> User could start a multiqueue virtio-net card through adding a "queues"
> parameter to tap.
>
> ./qemu -netdev tap,id=hn0,queues=2,vhost=on -device virtio-net-pci,netdev=hn0
>
> Management tools such as libvirt can pass multiple pre-created fds through
>
> ./qemu -netdev tap,id=hn0,queues=2,fd=X,fd=Y -device
> virtio-net-pci,netdev=hn0

I'm confused/frightened that this syntax works.  You shouldn't be
allowed to have two values for the same property.  Better to have a
syntax like fd[0]=X,fd[1]=Y or something along those lines.

Regards,

Anthony Liguori

>
> You can fetch and try the code from:
> git://github.com/jasowang/qemu.git
>
> Patch 1 adds a generic method of creating multiqueue taps and implement the
> linux part.
> Patch 2 - 4 introduce some helpers which could be used to refactor the nic
> emulation codes to support multiqueue.
> Patch 5 introduces multiqueue support for qemu networking code: each peers of
> NetClientState were abstracted as a queue. Though this, most of the codes could
> be reusued without change.
> Patch 6 adds basic multiqueue support for vhost which could let vhost just
> handle a subset of all virtqueues.
> Patch 7-8 introduce new helpers of virtio which is needed by multiqueue
> virtio-net.
> Patch 9-12 implement the multiqueue support of virtio-net
>
> Changes from RFC v2:
> - rebase the codes to latest qemu
> - align the multiqueue virtio-net implementation to virtio spec
> - split the patches into more smaller patches
> - set_link and hotplug support
>
> Changes from RFC V1:
> - rebase to the latest
> - fix memory leak in parse_netdev
> - fix guest notifiers assignment/de-assignment
> - changes the command lines to:
>    qemu -netdev tap,queues=2 -device virtio-net-pci,queues=2
>
> Reference:
> v2: http://lists.gnu.org/archive/html/qemu-devel/2012-06/msg04108.html
> v1: http://comments.gmane.org/gmane.comp.emulators.qemu/100481
>
> Perf Numbers:
>
> Two Intel Xeon 5620 with direct connected intel 82599EB
> Host/Guest kernel: David net tree
> vhost enabled
>
> - lots of improvents of both latency and cpu utilization in request-reponse test
> - get regression of guest sending small packets which because TCP tends to batch
>   less when the latency were improved
>
> 1q/2q/4q
> TCP_RR
>  size #sessions trans.rate  norm trans.rate  norm trans.rate  norm
> 1 1     9393.26   595.64  9408.18   597.34  9375.19   584.12
> 1 20    72162.1   2214.24 129880.22 2456.13 196949.81 2298.13
> 1 50    107513.38 2653.99 139721.93 2490.58 259713.82 2873.57
> 1 100   126734.63 2676.54 145553.5  2406.63 265252.68 2943
> 64 1    9453.42   632.33  9371.37   616.13  9338.19   615.97
> 64 20   70620.03  2093.68 125155.75 2409.15 191239.91 2253.32
> 64 50   106966    2448.29 146518.67 2514.47 242134.07 2720.91
> 64 100  117046.35 2394.56 190153.09 2696.82 238881.29 2704.41
> 256 1   8733.29   736.36  8701.07   680.83  8608.92   530.1
> 256 20  69279.89  2274.45 115103.07 2299.76 144555.16 1963.53
> 256 50  97676.02  2296.09 150719.57 2522.92 254510.5  3028.44
> 256 100 150221.55 2949.56 197569.3  2790.92 300695.78 3494.83
> TCP_CRR
>  size #sessions trans.rate  norm trans.rate  norm trans.rate  norm
> 1 1     2848.37  163.41 2230.39  130.89 2013.09  120.47
> 1 20    23434.5  562.11 31057.43 531.07 49488.28 564.41
> 1 50    28514.88 582.17 40494.23 605.92 60113.35 654.97
> 1 100   28827.22 584.73 48813.25 661.6  61783.62 676.56
> 64 1    2780.08  159.4  2201.07  127.96 2006.8   117.63
> 64 20   23318.51 564.47 30982.44 530.24 49734.95 566.13
> 64 50   28585.72 582.54 40576.7  610.08 60167.89 656.56
> 64 100  28747.37 584.17 49081.87 667.87 60612.94 662
> 256 1   2772.08  160.51 2231.84  131.05 2003.62  113.45
> 256 20  23086.35 559.8  30929.09 528.16 48454.9  555.22
> 256 50  28354.7  579.85 40578.31 607    60261.71 657.87
> 256 100 28844.55 585.67 48541.86 659.08 61941.07 676.72
> TCP_STREAM guest receiving
>  size #sessions throughput  norm throughput  norm throughput  norm
> 1 1     16.27   1.33   16.1    1.12   16.13   0.99
> 1 2     33.04   2.08   32.96   2.19   32.75   1.98
> 1 4     66.62   6.83   68.3    5.56   66.14   2.65
> 64 1    896.55  56.67  914.02  58.14  898.9   61.56
> 64 2    1830.46 91.02  1812.02 64.59  1835.57 66.26
> 64 4    3626.61 142.55 3636.25 100.64 3607.46 75.03
> 256 1   2619.49 131.23 2543.19 129.03 2618.69 132.39
> 256 2   5136.58 203.02 5163.31 141.11 5236.51 149.4
> 256 4   7063.99 242.83 9365.4  208.49 9421.03 159.94
> 512 1   3592.43 165.24 3603.12 167.19 3552.5  169.57
> 512 2   7042.62 246.59 7068.46 180.87 7258.52 186.3
> 512 4   6996.08 241.49 9298.34 206.12 9418.52 159.33
> 1024 1  4339.54 192.95 4370.2  191.92 4211.72 192.49
> 1024 2  7439.45 254.77 9403.99 215.24 9120.82 222.67
> 1024 4  7953.86 272.11 9403.87 208.23 9366.98 159.49
> 4096 1  7696.28 272.04 7611.41 270.38 7778.71 267.76
> 4096 2  7530.35 261.1  8905.43 246.27 8990.18 267.57
> 4096 4  7121.6  247.02 9411.75 206.71 9654.96 184.67
> 16384 1 7795.73 268.54 7780.94 267.2  7634.26 260.73
> 16384 2 7436.57 255.81 9381.86 220.85 9392    220.36
> 16384 4 7199.07 247.81 9420.96 205.87 9373.69 159.57
> TCP_MAERTS guest sending
>  size #sessions throughput  norm throughput  norm throughput  norm
> 1 1     15.94   0.62   15.55   0.61   15.13   0.59
> 1 2     36.11   0.83   32.46   0.69   32.28   0.69
> 1 4     71.59   1      68.91   0.94   61.52   0.77
> 64 1    630.71  22.52  622.11  22.35  605.09  21.84
> 64 2    1442.36 30.57  1292.15 25.82  1282.67 25.55
> 64 4    3186.79 42.59  2844.96 36.03  2529.69 30.06
> 256 1   1760.96 58.07  1738.44 57.43  1695.99 56.19
> 256 2   4834.23 95.19  3524.85 64.21  3511.94 64.45
> 256 4   9324.63 145.74 8956.49 116.39 6720.17 73.86
> 512 1   2678.03 84.1   2630.68 82.93  2636.54 82.57
> 512 2   9368.17 195.61 9408.82 204.53 5316.3  92.99
> 512 4   9186.34 209.68 9358.72 183.82 9489.29 160.42
> 1024 1  3620.71 109.88 3625.54 109.83 3606.61 112.35
> 1024 2  9429    258.32 7082.79 120.55 7403.53 134.78
> 1024 4  9430.66 290.44 9499.29 232.31 9414.6  190.92
> 4096 1  9339.28 296.48 9374.23 372.88 9348.76 298.49
> 4096 2  9410.53 378.69 9412.61 286.18 9409.75 278.31
> 4096 4  9487.35 374.1  9556.91 288.81 9441.94 221.64
> 16384 1 9380.43 403.8  9379.78 399.13 9382.42 393.55
> 16384 2 9367.69 406.93 9415.04 312.68 9409.29 300.9
> 16384 4 9391.96 405.17 9695.12 310.54 9423.76 223.47
>
>
> Jason Wang (12):
>   tap: multiqueue support
>   net: introduce qemu_get_queue()
>   net: introduce qemu_get_nic()
>   net: intorduce qemu_del_nic()
>   net: multiqueue support
>   vhost: multiqueue support
>   virtio: introduce virtio_queue_del()
>   virtio: add a queue_index to VirtQueue
>   virtio-net: separate virtqueue from VirtIONet
>   virtio-net: multiqueue support
>   virtio-net: migration support for multiqueue
>   virtio-net: compat multiqueue support
>
>  hw/cadence_gem.c        |   16 +-
>  hw/dp8393x.c            |   16 +-
>  hw/e1000.c              |   28 ++--
>  hw/eepro100.c           |   18 +-
>  hw/etraxfs_eth.c        |   10 +-
>  hw/lan9118.c            |   16 +-
>  hw/lance.c              |    2 +-
>  hw/mcf_fec.c            |   12 +-
>  hw/milkymist-minimac2.c |   10 +-
>  hw/mipsnet.c            |   10 +-
>  hw/musicpal.c           |    6 +-
>  hw/ne2000-isa.c         |    4 +-
>  hw/ne2000.c             |   12 +-
>  hw/opencores_eth.c      |   12 +-
>  hw/pc_piix.c            |    4 +
>  hw/pcnet-pci.c          |    4 +-
>  hw/pcnet.c              |   12 +-
>  hw/qdev-properties.c    |   46 ++++-
>  hw/qdev-properties.h    |    6 +-
>  hw/rtl8139.c            |   20 +-
>  hw/smc91c111.c          |   10 +-
>  hw/spapr_llan.c         |    8 +-
>  hw/stellaris_enet.c     |   10 +-
>  hw/usb/dev-network.c    |   16 +-
>  hw/vhost.c              |   52 +++--
>  hw/vhost.h              |    2 +
>  hw/vhost_net.c          |    7 +-
>  hw/vhost_net.h          |    2 +-
>  hw/virtio-net.c         |  523 ++++++++++++++++++++++++++++++++++-------------
>  hw/virtio-net.h         |   27 +++-
>  hw/virtio.c             |   17 ++
>  hw/virtio.h             |    3 +
>  hw/xen_nic.c            |   14 +-
>  hw/xgmac.c              |   10 +-
>  hw/xilinx_axienet.c     |   10 +-
>  hw/xilinx_ethlite.c     |   10 +-
>  net.c                   |  198 ++++++++++++++----
>  net.h                   |   31 +++-
>  net/tap-aix.c           |   18 ++-
>  net/tap-bsd.c           |   18 ++-
>  net/tap-haiku.c         |   18 ++-
>  net/tap-linux.c         |   70 ++++++-
>  net/tap-linux.h         |    4 +
>  net/tap-solaris.c       |   18 ++-
>  net/tap-win32.c         |   10 +
>  net/tap.c               |  248 ++++++++++++++++-------
>  net/tap.h               |    8 +-
>  qapi-schema.json        |    5 +-
>  savevm.c                |    2 +-
>  49 files changed, 1177 insertions(+), 456 deletions(-)
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html