[Qemu-devel] macvtap performance: good when writing from guest, abysmal when reading on guest (~ 700kB/s)

[Qemu-devel] macvtap performance: good when writing from guest, abysmal when reading on guest (~ 700kB/s)

[Lutz Vieweg]

Hi,

I've been using qemu-kvm along with ordinary tap-devices and software bridges
for quite some time. When I recently noticed that a certain TCP connection between
a guest and a remote physical host was limited to ~ 80MB/s, I thought it would
be a good idea to check whether by using "macvtap", instead, the performance
would get better.

So I setup a guest on a host that has a direct peer-to-peer 10G cable to
another host, and configured it to use a macvtap device.

Then I did some benchmarks, using "nc" on both sides, just reading from /dev/zero,
writing to /dev/null.

When the guest VM is writing into a TCP connection to the physical host (linux-3.1.6),
the performance is ~ 140MB/s - not great, but better than with ordinary
tap devices.

But to my big surprise, the performance when the physical host is writing,
and the guest VM is reading is abysmal, only ~ 700kB/s!
No bottleneck is obvious - the CPU usage and NIC utilization of both
the VM, its host, and the other host is all quite low.
"strace" on qemu process indicates that from time to time, there are "pauses" of ~ 0.5
seconds in between the many reads from /dev/tapX, but I am not sure whether
this is the whole reason for the bad performance.

Any ideas?

Or should I rather stay with ordinary tap/brctl, or try yet another
virtual NIC technique?

Regards,

Lutz Vieweg

Re: [Qemu-devel] macvtap performance: good when writing from guest, abysmal when reading on guest (~ 700kB/s)

[Lutz Vieweg]

Adding to my symptom description: I re-tested with "vhost=on" in addition
and verified this feature was actually used.
But this didn't change the benchmark results.

Regards,

Lutz Vieweg

On 01/20/2012 04:10 PM, Lutz Vieweg wrote:
> Hi,
>
> I've been using qemu-kvm along with ordinary tap-devices and software bridges
> for quite some time. When I recently noticed that a certain TCP connection between
> a guest and a remote physical host was limited to ~ 80MB/s, I thought it would
> be a good idea to check whether by using "macvtap", instead, the performance
> would get better.
>
> So I setup a guest on a host that has a direct peer-to-peer 10G cable to
> another host, and configured it to use a macvtap device.
>
> Then I did some benchmarks, using "nc" on both sides, just reading from /dev/zero,
> writing to /dev/null.
>
> When the guest VM is writing into a TCP connection to the physical host (linux-3.1.6),
> the performance is ~ 140MB/s - not great, but better than with ordinary
> tap devices.
>
> But to my big surprise, the performance when the physical host is writing,
> and the guest VM is reading is abysmal, only ~ 700kB/s!
> No bottleneck is obvious - the CPU usage and NIC utilization of both
> the VM, its host, and the other host is all quite low.
> "strace" on qemu process indicates that from time to time, there are "pauses" of ~ 0.5
> seconds in between the many reads from /dev/tapX, but I am not sure whether
> this is the whole reason for the bad performance.
>
> Any ideas?
>
> Or should I rather stay with ordinary tap/brctl, or try yet another
> virtual NIC technique?
>
> Regards,
>
> Lutz Vieweg
>
>
>
>