From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mark McLoughlin Subject: [PATCH 0/6] Kill off the virtio_net tx mitigation timer Date: Thu, 30 Oct 2008 17:51:47 +0000 Message-ID: <1225389113-28332-1-git-send-email-markmc@redhat.com> Cc: kvm@vger.kernel.org To: Avi Kivity Return-path: Received: from mail19.svc.cra.dublin.eircom.net ([159.134.118.218]:37583 "HELO mail19.svc.cra.dublin.eircom.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S1758119AbYJ3RxX (ORCPT ); Thu, 30 Oct 2008 13:53:23 -0400 In-Reply-To: <> References: <> Sender: kvm-owner@vger.kernel.org List-ID: Hey, The main patch in this series is 5/6 - it just kills off the virtio_net tx mitigation timer and does all the tx I/O in the I/O thread. Below are the results I got from benchmarking guest->host and host->guest on my machine. There's enough numbers there to make anyone blind, but basically there are results for current kvm-userspace.git, with the no-tx-timer patch applied and with the drop-the-mutex patch applied. Also, I've included results that show what difference some tuning makes with all the patches applied. The tuning basically just involves pinning the I/O thread and the netperf/netserver processes in both the host and guest to two physical CPUs which share a L2 cache. (Yes, the 1k buffer size results are weird - we think there's a bug in recent kernels that causes us not to coalesce these small buffers into a large GSO packet before sending) Anyway, the results in all their glory: | guest->host tput | host->guest tput netperf, 10x20s runs (Gb/s) | min/ mean/ max/stddev | min/ mean/ max/stddev ------------------------------+----------------------------+--------------------------- kvm-userspace.git, 1k | 0.600/ 0.645/ 0.670/ 0.025 | 5.170/ 5.285/ 5.470/ 0.087 kvm-userspace.git, 16k | 3.070/ 3.350/ 3.710/ 0.248 | 5.950/ 6.374/ 6.760/ 0.261 kvm-userspace.git, 65k | 4.950/ 6.041/ 7.170/ 0.639 | 5.480/ 5.642/ 5.810/ 0.092 no tx timer, 1k | 0.720/ 0.790/ 0.850/ 0.040 | 4.950/ 5.172/ 5.370/ 0.128 no tx timer, 16k | 4.120/ 4.512/ 4.740/ 0.190 | 4.900/ 5.480/ 6.230/ 0.416 no tx timer, 65k | 5.510/ 7.702/ 9.600/ 1.153 | 4.490/ 5.208/ 5.690/ 0.408 drop mutex, 1k | 0.810/ 0.847/ 0.910/ 0.030 | 5.140/ 5.416/ 5.660/ 0.145 drop mutex, 16k | 5.110/ 5.713/ 6.480/ 0.440 | 6.050/ 6.658/ 7.490/ 0.443 drop mutex, 65k | 7.070/ 8.054/ 9.210/ 0.671 | 4.470/ 5.922/ 7.200/ 0.930 tuned, 1k | 4.260/ 6.739/ 8.020/ 1.075 | 3.640/ 3.777/ 4.020/ 0.110 tuned, 16k | 8.530/ 8.867/ 9.110/ 0.146 | 7.220/ 7.276/ 7.360/ 0.040 tuned, 65k | 9.200/ 9.282/ 9.870/ 0.198 | 7.850/ 7.924/ 8.000/ 0.045 | guest->host %idle | host->guest %idle netperf, 10x20s runs (Gb/s) | avg/cpu#0/cpu#1/cpu#2/cpu#3 | avg/cpu#0/cpu#1/cpu#2/cpu#3 ------------------------------+--------------------------------+----------------------------- kvm-userspace.git, 1k | 64.93/65.94/61.06/72.89/60.31 | 39.36/52.52/19.04/63.94/21.35 kvm-userspace.git, 16k | 61.47/66.22/52.76/66.99/60.02 | 43.94/58.12/27.45/55.85/33.64 kvm-userspace.git, 65k | 61.37/82.58/51.55/51.15/60.16 | 49.30/69.80/37.14/51.75/38.24 no tx timer, 1k | 57.99/59.93/59.68/71.17/42.03 | 42.13/60.85/31.13/55.06/22.94 no tx timer, 16k | 54.87/55.57/55.88/61.35/46.67 | 49.18/54.35/40.56/59.42/43.46 no tx timer, 65k | 47.53/73.63/33.00/46.36/37.14 | 53.67/66.69/40.92/47.73/59.89 drop mutex, 1k | 52.94/61.90/48.44/64.35/38.25 | 39.31/53.29/26.35/60.90/17.53 drop mutex, 16k | 49.28/59.47/37.16/50.07/50.16 | 42.18/52.26/26.76/55.95/34.26 drop mutex, 65k | 49.65/65.94/41.38/47.55/42.55 | 46.36/42.51/60.55/45.15/37.91 tuned, 1k | 57.68/99.81/99.96/18.76/21.91 | 65.49/99.84/99.97/15.20/52.46 tuned, 16k | 61.66/99.85/99.98/13.01/40.83 | 61.43/99.79/99.93/0.97/47.61 tuned, 65k | 62.57/99.99/99.95/13.13/39.28 | 60.15/99.81/99.97/1.27/43.10 Cheers, Mark.