From: Jason Wang <jasowang@redhat.com>
To: mst@redhat.com, aliguori@us.ibm.com, stefanha@redhat.com,
qemu-devel@nongnu.org
Cc: krkumar2@in.ibm.com, kvm@vger.kernel.org, mprivozn@redhat.com,
Jason Wang <jasowang@redhat.com>,
rusty@rustcorp.com.au, jwhan@filewood.snu.ac.kr,
shiyer@redhat.com
Subject: [Qemu-devel] [PATCH 00/12] Multiqueue virtio-net
Date: Fri, 28 Dec 2012 18:31:52 +0800 [thread overview]
Message-ID: <1356690724-37891-1-git-send-email-jasowang@redhat.com> (raw)
Hello all:
This seires is an update of last version of multiqueue virtio-net support.
Recently, linux tap gets multiqueue support. This series implements basic
support for multiqueue tap, nic and vhost. Then use it as an infrastructure to
enable the multiqueue support for virtio-net.
Both vhost and userspace multiqueue were implemented for virtio-net, but
userspace could be get much benefits since dataplane like parallized mechanism
were not implemented.
User could start a multiqueue virtio-net card through adding a "queues"
parameter to tap.
./qemu -netdev tap,id=hn0,queues=2,vhost=on -device virtio-net-pci,netdev=hn0
Management tools such as libvirt can pass multiple pre-created fds through
./qemu -netdev tap,id=hn0,queues=2,fd=X,fd=Y -device virtio-net-pci,netdev=hn0
You can fetch and try the code from:
git://github.com/jasowang/qemu.git
Patch 1 adds a generic method of creating multiqueue taps and implement the
linux part.
Patch 2 - 4 introduce some helpers which could be used to refactor the nic
emulation codes to support multiqueue.
Patch 5 introduces multiqueue support for qemu networking code: each peers of
NetClientState were abstracted as a queue. Though this, most of the codes could
be reusued without change.
Patch 6 adds basic multiqueue support for vhost which could let vhost just
handle a subset of all virtqueues.
Patch 7-8 introduce new helpers of virtio which is needed by multiqueue
virtio-net.
Patch 9-12 implement the multiqueue support of virtio-net
Changes from RFC v2:
- rebase the codes to latest qemu
- align the multiqueue virtio-net implementation to virtio spec
- split the patches into more smaller patches
- set_link and hotplug support
Changes from RFC V1:
- rebase to the latest
- fix memory leak in parse_netdev
- fix guest notifiers assignment/de-assignment
- changes the command lines to:
qemu -netdev tap,queues=2 -device virtio-net-pci,queues=2
Reference:
v2: http://lists.gnu.org/archive/html/qemu-devel/2012-06/msg04108.html
v1: http://comments.gmane.org/gmane.comp.emulators.qemu/100481
Perf Numbers:
Two Intel Xeon 5620 with direct connected intel 82599EB
Host/Guest kernel: David net tree
vhost enabled
- lots of improvents of both latency and cpu utilization in request-reponse test
- get regression of guest sending small packets which because TCP tends to batch
less when the latency were improved
1q/2q/4q
TCP_RR
size #sessions trans.rate norm trans.rate norm trans.rate norm
1 1 9393.26 595.64 9408.18 597.34 9375.19 584.12
1 20 72162.1 2214.24 129880.22 2456.13 196949.81 2298.13
1 50 107513.38 2653.99 139721.93 2490.58 259713.82 2873.57
1 100 126734.63 2676.54 145553.5 2406.63 265252.68 2943
64 1 9453.42 632.33 9371.37 616.13 9338.19 615.97
64 20 70620.03 2093.68 125155.75 2409.15 191239.91 2253.32
64 50 106966 2448.29 146518.67 2514.47 242134.07 2720.91
64 100 117046.35 2394.56 190153.09 2696.82 238881.29 2704.41
256 1 8733.29 736.36 8701.07 680.83 8608.92 530.1
256 20 69279.89 2274.45 115103.07 2299.76 144555.16 1963.53
256 50 97676.02 2296.09 150719.57 2522.92 254510.5 3028.44
256 100 150221.55 2949.56 197569.3 2790.92 300695.78 3494.83
TCP_CRR
size #sessions trans.rate norm trans.rate norm trans.rate norm
1 1 2848.37 163.41 2230.39 130.89 2013.09 120.47
1 20 23434.5 562.11 31057.43 531.07 49488.28 564.41
1 50 28514.88 582.17 40494.23 605.92 60113.35 654.97
1 100 28827.22 584.73 48813.25 661.6 61783.62 676.56
64 1 2780.08 159.4 2201.07 127.96 2006.8 117.63
64 20 23318.51 564.47 30982.44 530.24 49734.95 566.13
64 50 28585.72 582.54 40576.7 610.08 60167.89 656.56
64 100 28747.37 584.17 49081.87 667.87 60612.94 662
256 1 2772.08 160.51 2231.84 131.05 2003.62 113.45
256 20 23086.35 559.8 30929.09 528.16 48454.9 555.22
256 50 28354.7 579.85 40578.31 607 60261.71 657.87
256 100 28844.55 585.67 48541.86 659.08 61941.07 676.72
TCP_STREAM guest receiving
size #sessions throughput norm throughput norm throughput norm
1 1 16.27 1.33 16.1 1.12 16.13 0.99
1 2 33.04 2.08 32.96 2.19 32.75 1.98
1 4 66.62 6.83 68.3 5.56 66.14 2.65
64 1 896.55 56.67 914.02 58.14 898.9 61.56
64 2 1830.46 91.02 1812.02 64.59 1835.57 66.26
64 4 3626.61 142.55 3636.25 100.64 3607.46 75.03
256 1 2619.49 131.23 2543.19 129.03 2618.69 132.39
256 2 5136.58 203.02 5163.31 141.11 5236.51 149.4
256 4 7063.99 242.83 9365.4 208.49 9421.03 159.94
512 1 3592.43 165.24 3603.12 167.19 3552.5 169.57
512 2 7042.62 246.59 7068.46 180.87 7258.52 186.3
512 4 6996.08 241.49 9298.34 206.12 9418.52 159.33
1024 1 4339.54 192.95 4370.2 191.92 4211.72 192.49
1024 2 7439.45 254.77 9403.99 215.24 9120.82 222.67
1024 4 7953.86 272.11 9403.87 208.23 9366.98 159.49
4096 1 7696.28 272.04 7611.41 270.38 7778.71 267.76
4096 2 7530.35 261.1 8905.43 246.27 8990.18 267.57
4096 4 7121.6 247.02 9411.75 206.71 9654.96 184.67
16384 1 7795.73 268.54 7780.94 267.2 7634.26 260.73
16384 2 7436.57 255.81 9381.86 220.85 9392 220.36
16384 4 7199.07 247.81 9420.96 205.87 9373.69 159.57
TCP_MAERTS guest sending
size #sessions throughput norm throughput norm throughput norm
1 1 15.94 0.62 15.55 0.61 15.13 0.59
1 2 36.11 0.83 32.46 0.69 32.28 0.69
1 4 71.59 1 68.91 0.94 61.52 0.77
64 1 630.71 22.52 622.11 22.35 605.09 21.84
64 2 1442.36 30.57 1292.15 25.82 1282.67 25.55
64 4 3186.79 42.59 2844.96 36.03 2529.69 30.06
256 1 1760.96 58.07 1738.44 57.43 1695.99 56.19
256 2 4834.23 95.19 3524.85 64.21 3511.94 64.45
256 4 9324.63 145.74 8956.49 116.39 6720.17 73.86
512 1 2678.03 84.1 2630.68 82.93 2636.54 82.57
512 2 9368.17 195.61 9408.82 204.53 5316.3 92.99
512 4 9186.34 209.68 9358.72 183.82 9489.29 160.42
1024 1 3620.71 109.88 3625.54 109.83 3606.61 112.35
1024 2 9429 258.32 7082.79 120.55 7403.53 134.78
1024 4 9430.66 290.44 9499.29 232.31 9414.6 190.92
4096 1 9339.28 296.48 9374.23 372.88 9348.76 298.49
4096 2 9410.53 378.69 9412.61 286.18 9409.75 278.31
4096 4 9487.35 374.1 9556.91 288.81 9441.94 221.64
16384 1 9380.43 403.8 9379.78 399.13 9382.42 393.55
16384 2 9367.69 406.93 9415.04 312.68 9409.29 300.9
16384 4 9391.96 405.17 9695.12 310.54 9423.76 223.47
Jason Wang (12):
tap: multiqueue support
net: introduce qemu_get_queue()
net: introduce qemu_get_nic()
net: intorduce qemu_del_nic()
net: multiqueue support
vhost: multiqueue support
virtio: introduce virtio_queue_del()
virtio: add a queue_index to VirtQueue
virtio-net: separate virtqueue from VirtIONet
virtio-net: multiqueue support
virtio-net: migration support for multiqueue
virtio-net: compat multiqueue support
hw/cadence_gem.c | 16 +-
hw/dp8393x.c | 16 +-
hw/e1000.c | 28 ++--
hw/eepro100.c | 18 +-
hw/etraxfs_eth.c | 10 +-
hw/lan9118.c | 16 +-
hw/lance.c | 2 +-
hw/mcf_fec.c | 12 +-
hw/milkymist-minimac2.c | 10 +-
hw/mipsnet.c | 10 +-
hw/musicpal.c | 6 +-
hw/ne2000-isa.c | 4 +-
hw/ne2000.c | 12 +-
hw/opencores_eth.c | 12 +-
hw/pc_piix.c | 4 +
hw/pcnet-pci.c | 4 +-
hw/pcnet.c | 12 +-
hw/qdev-properties.c | 46 ++++-
hw/qdev-properties.h | 6 +-
hw/rtl8139.c | 20 +-
hw/smc91c111.c | 10 +-
hw/spapr_llan.c | 8 +-
hw/stellaris_enet.c | 10 +-
hw/usb/dev-network.c | 16 +-
hw/vhost.c | 52 +++--
hw/vhost.h | 2 +
hw/vhost_net.c | 7 +-
hw/vhost_net.h | 2 +-
hw/virtio-net.c | 523 ++++++++++++++++++++++++++++++++++-------------
hw/virtio-net.h | 27 +++-
hw/virtio.c | 17 ++
hw/virtio.h | 3 +
hw/xen_nic.c | 14 +-
hw/xgmac.c | 10 +-
hw/xilinx_axienet.c | 10 +-
hw/xilinx_ethlite.c | 10 +-
net.c | 198 ++++++++++++++----
net.h | 31 +++-
net/tap-aix.c | 18 ++-
net/tap-bsd.c | 18 ++-
net/tap-haiku.c | 18 ++-
net/tap-linux.c | 70 ++++++-
net/tap-linux.h | 4 +
net/tap-solaris.c | 18 ++-
net/tap-win32.c | 10 +
net/tap.c | 248 ++++++++++++++++-------
net/tap.h | 8 +-
qapi-schema.json | 5 +-
savevm.c | 2 +-
49 files changed, 1177 insertions(+), 456 deletions(-)
next reply other threads:[~2012-12-28 10:41 UTC|newest]
Thread overview: 58+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-12-28 10:31 Jason Wang [this message]
2012-12-28 10:31 ` [Qemu-devel] [PATCH 01/12] tap: multiqueue support Jason Wang
2013-01-09 9:56 ` Stefan Hajnoczi
2013-01-09 15:25 ` Jason Wang
2013-01-10 8:32 ` Stefan Hajnoczi
2013-01-10 10:28 ` Stefan Hajnoczi
2013-01-10 13:52 ` Jason Wang
2012-12-28 10:31 ` [Qemu-devel] [PATCH 02/12] net: introduce qemu_get_queue() Jason Wang
2012-12-28 10:31 ` [Qemu-devel] [PATCH 03/12] net: introduce qemu_get_nic() Jason Wang
2012-12-28 10:31 ` [Qemu-devel] [PATCH 04/12] net: intorduce qemu_del_nic() Jason Wang
2012-12-28 10:31 ` [Qemu-devel] [PATCH 05/12] net: multiqueue support Jason Wang
2012-12-28 18:06 ` Blue Swirl
2012-12-28 10:31 ` [Qemu-devel] [PATCH 06/12] vhost: " Jason Wang
2012-12-28 10:31 ` [Qemu-devel] [PATCH 07/12] virtio: introduce virtio_queue_del() Jason Wang
2013-01-08 7:14 ` Michael S. Tsirkin
2013-01-08 9:28 ` Jason Wang
2012-12-28 10:32 ` [Qemu-devel] [PATCH 08/12] virtio: add a queue_index to VirtQueue Jason Wang
2012-12-28 10:32 ` [Qemu-devel] [PATCH 09/12] virtio-net: separate virtqueue from VirtIONet Jason Wang
2012-12-28 10:32 ` [Qemu-devel] [PATCH 10/12] virtio-net: multiqueue support Jason Wang
2012-12-28 17:52 ` Blue Swirl
2013-01-04 5:12 ` Jason Wang
2013-01-04 20:41 ` Blue Swirl
2013-01-08 9:07 ` Wanlong Gao
2013-01-08 9:29 ` Jason Wang
2013-01-08 9:32 ` Wanlong Gao
2013-01-08 9:49 ` Wanlong Gao
2013-01-08 9:51 ` Jason Wang
2013-01-08 10:00 ` Wanlong Gao
2013-01-08 10:14 ` Jason Wang
2013-01-08 11:24 ` Wanlong Gao
2013-01-09 3:11 ` Jason Wang
2013-01-09 8:23 ` Wanlong Gao
2013-01-09 9:30 ` Jason Wang
2013-01-09 10:01 ` Wanlong Gao
2013-01-09 15:26 ` Jason Wang
2013-01-10 6:43 ` Jason Wang
2013-01-10 6:49 ` Wanlong Gao
2013-01-10 7:16 ` Jason Wang
2013-01-10 9:06 ` Wanlong Gao
2013-01-10 9:40 ` Jason Wang
2012-12-28 10:32 ` [Qemu-devel] [PATCH 11/12] virtio-net: migration support for multiqueue Jason Wang
2013-01-08 7:10 ` Michael S. Tsirkin
2013-01-08 9:27 ` Jason Wang
2012-12-28 10:32 ` [Qemu-devel] [PATCH 12/12] virtio-net: compat multiqueue support Jason Wang
2013-01-09 14:29 ` [Qemu-devel] [PATCH 00/12] Multiqueue virtio-net Stefan Hajnoczi
2013-01-09 15:32 ` Michael S. Tsirkin
2013-01-09 15:33 ` Jason Wang
2013-01-10 8:44 ` Stefan Hajnoczi
2013-01-10 9:34 ` Jason Wang
2013-01-10 11:49 ` Stefan Hajnoczi
2013-01-10 14:15 ` Jason Wang
2013-01-14 19:44 ` Anthony Liguori
2013-01-15 10:12 ` Jason Wang
2013-01-16 15:09 ` Anthony Liguori
2013-01-16 15:19 ` Michael S. Tsirkin
2013-01-16 16:14 ` Anthony Liguori
2013-01-16 16:48 ` Michael S. Tsirkin
2013-01-17 10:31 ` Michael S. Tsirkin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1356690724-37891-1-git-send-email-jasowang@redhat.com \
--to=jasowang@redhat.com \
--cc=aliguori@us.ibm.com \
--cc=jwhan@filewood.snu.ac.kr \
--cc=krkumar2@in.ibm.com \
--cc=kvm@vger.kernel.org \
--cc=mprivozn@redhat.com \
--cc=mst@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=rusty@rustcorp.com.au \
--cc=shiyer@redhat.com \
--cc=stefanha@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).