Re: [PATCH 0/6] Kill off the virtio_net tx mitigation timer

[Avi Kivity <avi@redhat.com>]

Mark McLoughlin wrote:
> Hey,
> 	So, I went off and spent some time gathering more data on this stuff
> and putting it together in a more consumable fashion.
>
> 	Here are some graphs showing the effect some of these changes have on
> throughput, cpu utilization and vmexit rate:
>
>   http://markmc.fedorapeople.org/virtio-netperf/2008-11-06/
>
>   

This is very helpful.

> 	The results are a little surprising, and I'm not sure I've fully
> digested them yet but some conclusions:
>
>   1) Disabling notifications from the guest for longer helps; you see 
>      an increase in cpu utilization and vmexit rate, but that can be 
>      accounted for by the extra data we're transferring
>
>   

Graphing cpu/bandwidth (cycles/bit) will show that nicely.

>   2) Flushing (when the ring is full) in the I/O thread doesn't seem to
>      help anything; strangely, it has a detrimental effect on 
>      host->guest traffic where I would expect us to hit this case at 
>      all.
>
>      I suspect we may not actually be hitting the full ring condition
>      in these tests at all.
>
>   

That's good; ring full == stall, especially with smp guests.


>   4) Removing the tx timer doesn't have a huge affect on guest->host, 
>      except for 32 byte buffers where we see a huge increase in vmexits 
>      and a drop in throughput. Bizarrely, we don't see this effect with 
>      64 byte buffers.
>   

Wierd.  Cacheline size effects?  the host must copy twice the number of 
cachelines for the same throughput, when moving between 32 and 64.

> 	
>      However, it does have a pretty significant impact on host->guest, 
>      which makes sense since in that case we'll just have a steady
>      stream of TCP ACK packets so if small guest->host packets are 
>      affected badly, so will the ACK packets.
>   

no-tx-timer is good for two workloads: streaming gso packets, where the 
packet is so large the vmexit count is low anyway, and small, latency 
sensitive packets, where you need the vmexits.  I'm worried about the 
workloads in between, which is why I'm pushing for the dynamic window.

>   5) The drop-mutex patch is a nice win overall, except for a huge 
>      increase in vmexits for sub-4k guest->host packets. Strange.
>   

What types of vmexits are these?  virtio pio or mmu?  and what's the 
test length (interested in vmexits/sec and vmexits/bit).

Maybe the allocator changes its behavior and we're faulting in pages.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.