Re: performance of virtual functions compared to virtio

[David Ahern <dsahern@gmail.com>]

On 04/25/11 14:27, Alex Williamson wrote:
> On Mon, 2011-04-25 at 13:49 -0600, David Ahern wrote:
>>
>> On 04/25/11 13:29, Alex Williamson wrote:
>>> So we're effectively getting host-host latency/throughput for the VF,
>>> it's just that in the 82576 implementation of SR-IOV, the VF takes a
>>> latency hit that puts it pretty close to virtio.  Unfortunate.  I think
>>
>> For host-to-VM using VFs is worse than virtio which is counterintuitive.
> 
> On the same host, just think about the data path of one versus the
> other.  On the guest side, there's virtio vs a physical NIC.  virtio is
> designed to be virtualization friendly, so hopefully has less context
> switches in setting up and processing transactions.  Once the packet
> leaves the assigned physical NIC, it has to come back up the entire host
> I/O stack, while the virtio device is connected to an internal bridge
> and bypasses all but the upper level network routing.

I get the virtio path, but you lost me on the physical NIC. I thought
the point of VFs is to bypass the host from having to touch the packet,
so the processing path with a VM using a VF would be the same as a non-VM.

David


> 
>>> you'll find that passing the PF to the guests should be pretty close to
>>> that 185us latency.  I would assume (hope) the higher end NICs reduce
>>
>> About that 185usec: do you know where the bottleneck is? It seems as if
>> the packet is held in some queue waiting for an event/timeout before it
>> is transmitted.
> 
> I don't know specifically, I don't do much network performance tuning.
> Interrupt coalescing could be a factor, along with various offload
> settings, and of course latency of the physical NIC hardware and
> interconnects.  Thanks,
> 
> Alex
>