From mboxrd@z Thu Jan 1 00:00:00 1970 From: Avi Kivity Subject: Re: [PATCH 0/6] Kill off the virtio_net tx mitigation timer Date: Thu, 06 Nov 2008 19:38:21 +0200 Message-ID: <49132B8D.2070300@redhat.com> References: <> <1225389113-28332-1-git-send-email-markmc@redhat.com> <490D7754.4070807@redhat.com> <1225715009.5904.39.camel@blaa> <490EF141.8040005@redhat.com> <1225724694.5904.63.camel@blaa> <490F1690.6060509@redhat.com> <1225989962.10879.19.camel@blaa> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: kvm@vger.kernel.org To: Mark McLoughlin Return-path: Received: from mx2.redhat.com ([66.187.237.31]:33889 "EHLO mx2.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750793AbYKFRiU (ORCPT ); Thu, 6 Nov 2008 12:38:20 -0500 Received: from int-mx2.corp.redhat.com (int-mx2.corp.redhat.com [172.16.27.26]) by mx2.redhat.com (8.13.8/8.13.8) with ESMTP id mA6HcDli003443 for ; Thu, 6 Nov 2008 12:38:13 -0500 In-Reply-To: <1225989962.10879.19.camel@blaa> Sender: kvm-owner@vger.kernel.org List-ID: Mark McLoughlin wrote: > Hey, > So, I went off and spent some time gathering more data on this stuff > and putting it together in a more consumable fashion. > > Here are some graphs showing the effect some of these changes have on > throughput, cpu utilization and vmexit rate: > > http://markmc.fedorapeople.org/virtio-netperf/2008-11-06/ > > This is very helpful. > The results are a little surprising, and I'm not sure I've fully > digested them yet but some conclusions: > > 1) Disabling notifications from the guest for longer helps; you see > an increase in cpu utilization and vmexit rate, but that can be > accounted for by the extra data we're transferring > > Graphing cpu/bandwidth (cycles/bit) will show that nicely. > 2) Flushing (when the ring is full) in the I/O thread doesn't seem to > help anything; strangely, it has a detrimental effect on > host->guest traffic where I would expect us to hit this case at > all. > > I suspect we may not actually be hitting the full ring condition > in these tests at all. > > That's good; ring full == stall, especially with smp guests. > 4) Removing the tx timer doesn't have a huge affect on guest->host, > except for 32 byte buffers where we see a huge increase in vmexits > and a drop in throughput. Bizarrely, we don't see this effect with > 64 byte buffers. > Wierd. Cacheline size effects? the host must copy twice the number of cachelines for the same throughput, when moving between 32 and 64. > > However, it does have a pretty significant impact on host->guest, > which makes sense since in that case we'll just have a steady > stream of TCP ACK packets so if small guest->host packets are > affected badly, so will the ACK packets. > no-tx-timer is good for two workloads: streaming gso packets, where the packet is so large the vmexit count is low anyway, and small, latency sensitive packets, where you need the vmexits. I'm worried about the workloads in between, which is why I'm pushing for the dynamic window. > 5) The drop-mutex patch is a nice win overall, except for a huge > increase in vmexits for sub-4k guest->host packets. Strange. > What types of vmexits are these? virtio pio or mmu? and what's the test length (interested in vmexits/sec and vmexits/bit). Maybe the allocator changes its behavior and we're faulting in pages. -- I have a truly marvellous patch that fixes the bug which this signature is too narrow to contain.