Re: [PATCH 0/9][RFC] KVM virtio_net performance

[Rusty Russell <rusty@rustcorp.com.au>]

On Saturday 26 July 2008 19:45:36 Avi Kivity wrote:
> Mark McLoughlin wrote:
> > Hey,
> >       Here's a bunch of patches attempting to improve the performance
> > of virtio_net. This is more an RFC rather than a patch submission
> > since, as can be seen below, not all patches actually improve the
> > perfomance measurably.
> >
> >       I've tried hard to test each of these patches with as stable and
> > informative a benchmark as I could find. The first benchmark is a
> > netperf[1] based throughput benchmark and the second uses a flood
> > ping[2] to measure latency differences.
> >
> >       Each set of figures is min/average/max/standard deviation. The
> > first set is Gb/s and the second is milliseconds.
> >
> >       The network configuration used was very simple - the guest with
> > a virtio_net interface and the host with a tap interface and static
> > IP addresses assigned to both - e.g. there was no bridge in the host
> > involved and iptables was disable in both the host and guest.
> >
> >       I used:
> >
> >   1) kvm-71-26-g6152996 with the patches that follow
> >
> >   2) Linus's v2.6.26-5752-g93ded9b with Rusty's virtio patches from
> >      219:bbd2611289c5 applied; these are the patches have just been
> >      submitted to Linus
> >
> >       The conclusions I draw are:
> >
> >   1) The length of the tx mitigation timer makes quite a difference to
> >      throughput achieved; we probably need a good heuristic for
> >      adjusting this on the fly.
>
> The tx mitigation timer is just one part of the equation; the other is
> the virtio ring window size, which is now fixed.
>
> Using a maximum sized window is good when the guest and host are running
> flat out, doing nothing but networking.  When throughput drops (because
> the guest is spending cpu on processing, or simply because the other
> side is not keeping up), we need to drop the windows size so as to
> retain acceptable latencies.
>
> The tx timer can then be set to "a bit after the end of the window",
> acting as a safety belt in case the throughput changes.

Interestingly, I played a little with a threshold patch.  Unfortunately it 
seemed to just add YA variable to the mix; it'd take real research to figure 
out how to adjust the threshold and timeout values appropriately for 
different situations.

> This isn't too good.  Low latency is important for nfs clients (or other
> request/response workloads).  I think we can keep these low by adjusting
> the virtio window (for example, on an idle system it should be 1), so
> that the tx mitigation timer only fires when the workload transitions
> from throughput to request/response.

>From reading this thread, this seems like an implementation bug.  Don't 
suppress notifications on an empty ring (ie. threshold will be 1 when nothing 
is happening).

Rusty.