Re: network performance between host and guest/iscsi to the rescue

[Erik Brakkee <erik@brakkee.org>]

Erik Brakkee wrote:
> Brian Jackson wrote:
>> On 12/17/2010 4:29 PM, Erik Brakkee wrote:
>>> Hi,
>>>
>>>
>>> For a backup of data from a VM to a USB mounted disk I want to 
>>> circumvent the USB 1.1 limitations on the guest and instead copy the 
>>> data over to the host using scp/ssh. I have setup a network using 
>>> virtio and NAT like this:
>>>
>>> <interface type='network'>
>>> <mac address='52:54:00:6b:0d:36'/>
>>> <source network='default'/>
>>> <target dev='vnet1'/>
>>> <model type='virtio'/>
>>> <alias name='net1'/>
>>> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' 
>>> function='0x0'/>
>>> </interface>
>>
>>
>> What does that equate to in command line options? Check libvirt logs 
>> maybe. What version of qemu-kvm? Guest details? Host details?
>>
>>
>>>
>>> When I now create a 1GB file using dd and copy it over from the 
>>> guest to the host, I am seeing a performance between 25-30 MB/s.
>>
>>
>> Is it to and from the same disk? If so, maybe you could try a tmpfs 
>> in the guest or host so you aren't constantly seeking back and forth 
>> on the same disk.
>>
>> Also have you tried something like rsyncd instead of scp? Maybe you 
>> are hitting some sort of encryption limitation.
>>
>>
>>>
>>> My question is if this is normal because I have seen others on the 
>>> internet achieve far greater speeds.
>>
>>
>> Depends on a lot of factors. Certainly raw bandwidth wise, virtio-net 
>> is capable of a lot more than that. With vhost-net here, I can get 
>> over 5gbps guest to host. And that's on crappy old first gen cpus (no 
>> ept/etc.).
>>
>>
>>>
>>> In any case the speeds are comparable to current USB 2.0 speeds but 
>>> I intend on using USB 3.0 so would like to get a little bit more out 
>>> of it.
>>>
>>> What would I use to speed this up a bit futher?
>>>
>>> Cheers
>>>   Erik
>>>
>>> -- 
>>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
> You are right. I was indeed not so smart of me. I was assuming that 
> the overhead of SSH would be negligible. However, I am seeing similar 
> transfer speeds when I copy a file on the localhost to the localhost 
> using SSH (on the host, not even on a guest). Netperf tests of the 
> network show much higher speeds (approx 900Mbps) so the network is not 
> the problem. The bottleneck is definitely SSH.
I have also experimented with rsh (not really secure) and that gave some 
improvements but not much. For instance, bandwidth increasing from 
approx 25MB/s to 35MB/s.

Then I thought about using iscsi and this really works great. Since KVM 
does not allow passing of new storage to a running guest (did not work 
for me in any case), iscsi is the way out. I simply configured a target 
on the host and an initiator on the guest and I am achieving native 
speeds for disk access.

When I overwrite a 900MB file (the iscsi target is 1GB) multiple times 
using  'dd if=/dev/zero bs=1M count=900 of=test.bin' I am seeing a 
performance of approx. 90MB/s on the iscsi initiator and approx. 75MB/s 
on the root partition. Both partitions are ext4 file systems and the 
root file system is using virtio. Also the network interface is using 
virtio.

My test setup is as follows: Sony Vaio F11 laptop, 8GB memory, Intel 
Core i7 Q720 (1.6GHz), host and guest both run Opensuse 11.3 64bit.
I created a 1GB logical volume on the host and exposed this as a target 
to the guest. The network interface is using virtio with NAT networking.

In any case my problem is solved. I intend to mount a SATA backup disk 
over USB 3.0 on the host and then do a backup from the guest directly 
onto this backup disk. Currently, I am using USB 2.0 and I am getting 
speeds of at most 30MB/s. In any case, this promises something for the 
bandwidth that I can get for backup from a guest in the future.

The interesting thing about this is that iscsi over a virtio network 
seems to be a bit faster in this case than a virtio disk. This makes it 
tempting to setup an iSCSI boot on the guest if not for the additional 
complexity this brings (had some experiences before with opensuse 11.1 
and it wasn't mature then). Also, I don't know how this would scale over 
multiple VMs. Alternatively, perhaps the implementation of virtio disks 
could be (transparently) based on iscsi or on iscsi concepts. I am 
curious to hear what you think of these results and if you also have 
similar experiences.