From mboxrd@z Thu Jan 1 00:00:00 1970 From: Rusty Russell Subject: Re: [PATCH 0/9][RFC] KVM virtio_net performance Date: Sun, 27 Jul 2008 16:48:00 +1000 Message-ID: <200807271648.00438.rusty@rustcorp.com.au> References: <1216899979-32532-1-git-send-email-markmc@redhat.com> <488AF240.2060208@qumranet.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8BIT Cc: Mark McLoughlin , kvm@vger.kernel.org, Herbert Xu To: Avi Kivity Return-path: Received: from ozlabs.org ([203.10.76.45]:44317 "EHLO ozlabs.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752749AbYG0GsL convert rfc822-to-8bit (ORCPT ); Sun, 27 Jul 2008 02:48:11 -0400 In-Reply-To: <488AF240.2060208@qumranet.com> Content-Disposition: inline Sender: kvm-owner@vger.kernel.org List-ID: On Saturday 26 July 2008 19:45:36 Avi Kivity wrote: > Mark McLoughlin wrote: > > Hey, > > Here's a bunch of patches attempting to improve the performance > > of virtio_net. This is more an RFC rather than a patch submission > > since, as can be seen below, not all patches actually improve the > > perfomance measurably. > > > > I've tried hard to test each of these patches with as stable and > > informative a benchmark as I could find. The first benchmark is a > > netperf[1] based throughput benchmark and the second uses a flood > > ping[2] to measure latency differences. > > > > Each set of figures is min/average/max/standard deviation. The > > first set is Gb/s and the second is milliseconds. > > > > The network configuration used was very simple - the guest with > > a virtio_net interface and the host with a tap interface and static > > IP addresses assigned to both - e.g. there was no bridge in the host > > involved and iptables was disable in both the host and guest. > > > > I used: > > > > 1) kvm-71-26-g6152996 with the patches that follow > > > > 2) Linus's v2.6.26-5752-g93ded9b with Rusty's virtio patches from > > 219:bbd2611289c5 applied; these are the patches have just been > > submitted to Linus > > > > The conclusions I draw are: > > > > 1) The length of the tx mitigation timer makes quite a difference to > > throughput achieved; we probably need a good heuristic for > > adjusting this on the fly. > > The tx mitigation timer is just one part of the equation; the other is > the virtio ring window size, which is now fixed. > > Using a maximum sized window is good when the guest and host are running > flat out, doing nothing but networking. When throughput drops (because > the guest is spending cpu on processing, or simply because the other > side is not keeping up), we need to drop the windows size so as to > retain acceptable latencies. > > The tx timer can then be set to "a bit after the end of the window", > acting as a safety belt in case the throughput changes. Interestingly, I played a little with a threshold patch. Unfortunately it seemed to just add YA variable to the mix; it'd take real research to figure out how to adjust the threshold and timeout values appropriately for different situations. > This isn't too good. Low latency is important for nfs clients (or other > request/response workloads). I think we can keep these low by adjusting > the virtio window (for example, on an idle system it should be 1), so > that the tx mitigation timer only fires when the workload transitions > from throughput to request/response. >>From reading this thread, this seems like an implementation bug. Don't suppress notifications on an empty ring (ie. threshold will be 1 when nothing is happening). Rusty.