[Qemu-devel] Performance of USB2.0

[Qemu-devel] Performance of USB2.0

[Til Obes]

Hello all,

i want to use a virtual router which is connected to a cable box
via an usb ethernet controller. The device itself runs properly
at normal usage. But inside the virtual machine i get only about 7MBit.
I configured the the usb device with the following xml syntax of
libvirt:
<controller type='usb' index='0' model='ehci'>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x07' 
function='0x0'/>
     </controller>
<hostdev mode='subsystem' type='usb' managed='no'>
       <source>
         <vendor id='0x9710'/>
         <product id='0x7830'/>
       </source>
     </hostdev>

Inside the guest i get this output of lsusb:
Bus 001 Device 002: ID 9710:7830 MosChip Semiconductor MCS7830 10/100 
Mbps Ethernet adapter
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

lsusb -v shows this output for the root hub:
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Device Descriptor:
   bLength                18
   bDescriptorType         1
   bcdUSB               2.00
   bDeviceClass            9 Hub
   bDeviceSubClass         0 Unused
   bDeviceProtocol         0 Full speed (or root) hub
   bMaxPacketSize0        64
   idVendor           0x1d6b Linux Foundation
   idProduct          0x0002 2.0 root hub
   bcdDevice            2.06
   iManufacturer           3 Linux 2.6.32-5-amd64 ehci_hcd
   iProduct                2 EHCI Host Controller
   iSerial                 1 0000:00:07.0

Does this "Full speed" mean, that it is still running with 12MBit?

What is the status of High Speed USB (480Mbit) inside the guest?
What versions do i need? I currently use Debian Sid:
qemu-kvm (0.15.1)
libvirt0-dbg (0.9.6-2)
Kernel: Linux server 3.0.0-2-amd64 #1 SMP x86_64 GNU/Linux

Regards Til

Re: [Qemu-devel] Performance of USB2.0

[Hans de Goede]

Hi,

On 10/31/2011 10:27 AM, Til Obes wrote:
> Hello all,
>
> i want to use a virtual router which is connected to a cable box
> via an usb ethernet controller. The device itself runs properly
> at normal usage. But inside the virtual machine i get only about 7MBit.
> I configured the the usb device with the following xml syntax of
> libvirt:
> <controller type='usb' index='0' model='ehci'>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
> </controller>
> <hostdev mode='subsystem' type='usb' managed='no'>
> <source>
> <vendor id='0x9710'/>
> <product id='0x7830'/>
> </source>
> </hostdev>
>
> Inside the guest i get this output of lsusb:
> Bus 001 Device 002: ID 9710:7830 MosChip Semiconductor MCS7830 10/100 Mbps Ethernet adapter
> Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
>
> lsusb -v shows this output for the root hub:
> Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
> Device Descriptor:
> bLength 18
> bDescriptorType 1
> bcdUSB 2.00
> bDeviceClass 9 Hub
> bDeviceSubClass 0 Unused
> bDeviceProtocol 0 Full speed (or root) hub
> bMaxPacketSize0 64
> idVendor 0x1d6b Linux Foundation
> idProduct 0x0002 2.0 root hub
> bcdDevice 2.06
> iManufacturer 3 Linux 2.6.32-5-amd64 ehci_hcd
> iProduct 2 EHCI Host Controller
> iSerial 1 0000:00:07.0
>
> Does this "Full speed" mean, that it is still running with 12MBit?

No, as lsusb says bDeviceProtocol 0 is standard for root hubs even
for usb2 (so high speed / 480mbit capable) root hubs.

> What is the status of High Speed USB (480Mbit) inside the guest?

It should work fine since qemu 0.15.

If you're using libvirt to start qemu, then it will also pass
-usb to qemu, so you will have both a usb-1 (uhci) and a usb-2
(ehci) controller inside your vm, since your hostdev xml code does
not specify a bus the hostdev will likely get connected to the first
usb bus which is the one attached to the uhci controller, although
your lsusb output suggests otherwise (to my surprise).

So assuming that I'm reading your lsusb output correct, the device
does seem to be connected to the virtual ehci controller rather then
to the virtual uhci controller, which more or less rules that out
as the cause.

This means that the likely cause is just that usb emulation / pass
through causes quite a bit of overhead, which is not unexpected since
both the usb protocol and the ehci controller interface are both quite
hard to emulate.

It is likely better to just use the usb nic directly from the host,
and then pass it through the virtio-net using bridging.

Regards,

Hans

Re: [Qemu-devel] Performance of USB2.0

[Til Obes]

Hi.

Am 31.10.2011 13:46, schrieb Hans de Goede:
> If you're using libvirt to start qemu, then it will also pass
> -usb to qemu, so you will have both a usb-1 (uhci) and a usb-2
> (ehci) controller inside your vm, since your hostdev xml code does
> not specify a bus the hostdev will likely get connected to the first
> usb bus which is the one attached to the uhci controller, although
> your lsusb output suggests otherwise (to my surprise).
 >
 > So assuming that I'm reading your lsusb output correct, the device
 > does seem to be connected to the virtual ehci controller rather then
 > to the virtual uhci controller, which more or less rules that out
 > as the cause.


The kernel output says:
router:~# dmesg|grep usb
[    0.418314] usbcore: registered new interface driver usbfs
[    0.418401] usbcore: registered new interface driver hub
[    0.419256] usbcore: registered new device driver usb
[    0.488137] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
[    0.488144] usb usb1: New USB device strings: Mfr=3, Product=2, 
SerialNumber=1
[    0.488150] usb usb1: Product: EHCI Host Controller
[    0.488155] usb usb1: Manufacturer: Linux 2.6.32-5-amd64 ehci_hcd
[    0.488160] usb usb1: SerialNumber: 0000:00:07.0
[    0.488437] usb usb1: configuration #1 chosen from 1 choice
[    1.155604] usb 1-1: new high speed USB device using ehci_hcd and 
address 2
[    1.665760] usb 1-1: New USB device found, idVendor=9710, idProduct=7830
[    1.665781] usb 1-1: New USB device strings: Mfr=1, Product=2, 
SerialNumber=3
[    1.665784] usb 1-1: Product: USB-MAC Controller
[    1.665786] usb 1-1: Manufacturer: Moschip Semiconductor
[    1.665787] usb 1-1: SerialNumber: 3b0c00d1
[    1.665886] usb 1-1: configuration #1 chosen from 1 choice
[    1.877906] usb 1-1: applying rev.C fixup
[    1.890249] usb 1-1: applying rev.C fixup
[    1.905011] eth2: register 'MOSCHIP usb-ethernet driver' at 
usb-0000:00:07.0-1, MOSCHIP 7830/7730 usb-NET adapter, 00:13:3b:0c:00:d1
[    1.905028] usbcore: registered new interface driver MOSCHIP 
usb-ethernet driver

So it should work with my configuration...


> This means that the likely cause is just that usb emulation / pass
> through causes quite a bit of overhead, which is not unexpected since
> both the usb protocol and the ehci controller interface are both quite
> hard to emulate.

But 6Mbit from 480MBit is quite low, at least 100MBit should be 
realistic or? How can i debug this?

> It is likely better to just use the usb nic directly from the host,
> and then pass it through the virtio-net using bridging.

Well the cable modem is only talking to the first mac address it gets.
In my setup eth1 is the bridged nic to the cable box and it isnt
working. Any suggestions how to solve this?

Regards Til

Re: [Qemu-devel] Performance of USB2.0

[Hans de Goede]

Hi,

On 10/31/2011 02:37 PM, Til Obes wrote:
> Hi.
>
> Am 31.10.2011 13:46, schrieb Hans de Goede:
>> If you're using libvirt to start qemu, then it will also pass
>> -usb to qemu, so you will have both a usb-1 (uhci) and a usb-2
>> (ehci) controller inside your vm, since your hostdev xml code does
>> not specify a bus the hostdev will likely get connected to the first
>> usb bus which is the one attached to the uhci controller, although
>> your lsusb output suggests otherwise (to my surprise).
>  >
>  > So assuming that I'm reading your lsusb output correct, the device
>  > does seem to be connected to the virtual ehci controller rather then
>  > to the virtual uhci controller, which more or less rules that out
>  > as the cause.
>
>
> The kernel output says:
> router:~# dmesg|grep usb
> [ 0.418314] usbcore: registered new interface driver usbfs
> [ 0.418401] usbcore: registered new interface driver hub
> [ 0.419256] usbcore: registered new device driver usb
> [ 0.488137] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
> [ 0.488144] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
> [ 0.488150] usb usb1: Product: EHCI Host Controller
> [ 0.488155] usb usb1: Manufacturer: Linux 2.6.32-5-amd64 ehci_hcd
> [ 0.488160] usb usb1: SerialNumber: 0000:00:07.0
> [ 0.488437] usb usb1: configuration #1 chosen from 1 choice
> [ 1.155604] usb 1-1: new high speed USB device using ehci_hcd and address 2
> [ 1.665760] usb 1-1: New USB device found, idVendor=9710, idProduct=7830
> [ 1.665781] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
> [ 1.665784] usb 1-1: Product: USB-MAC Controller
> [ 1.665786] usb 1-1: Manufacturer: Moschip Semiconductor
> [ 1.665787] usb 1-1: SerialNumber: 3b0c00d1
> [ 1.665886] usb 1-1: configuration #1 chosen from 1 choice
> [ 1.877906] usb 1-1: applying rev.C fixup
> [ 1.890249] usb 1-1: applying rev.C fixup
> [ 1.905011] eth2: register 'MOSCHIP usb-ethernet driver' at usb-0000:00:07.0-1, MOSCHIP 7830/7730 usb-NET adapter, 00:13:3b:0c:00:d1
> [ 1.905028] usbcore: registered new interface driver MOSCHIP usb-ethernet driver
>
> So it should work with my configuration...
>

Agreed.

>> This means that the likely cause is just that usb emulation / pass
>> through causes quite a bit of overhead, which is not unexpected since
>> both the usb protocol and the ehci controller interface are both quite
>> hard to emulate.
>
> But 6Mbit from 480MBit is quite low, at least 100MBit should be realistic or?

That depends on how the protocol between the NIC and the usb host works, it likely
is using bulk transfers, and likely only 1 at a time, which means that latency
becomes an important issue, and usb redirection of bulk transfers causes quite
a bit of latency.


>
>> It is likely better to just use the usb nic directly from the host,
>> and then pass it through the virtio-net using bridging.
>
> Well the cable modem is only talking to the first mac address it gets.
> In my setup eth1 is the bridged nic to the cable box and it isnt
> working. Any suggestions how to solve this?

If the cable box wants to use the usb-nic, then why not let the host
drive the usb-host and bridge the usb-nic rather then eth1 to the virtual machine?

Regards,

Hans

Re: [Qemu-devel] Performance of USB2.0

[Gerd Hoffmann]

  Hi,

> This means that the likely cause is just that usb emulation / pass
> through causes quite a bit of overhead, which is not unexpected since
> both the usb protocol and the ehci controller interface are both quite
> hard to emulate.

I think the main issue here is that we don't do buffering / pipelining
for bulk transfers at the moment.  We grab a single transfer request
from the guest, pass it to the kernel, when it is done pass it back to
the guest, then look look for the next one.  Instead we could queue up
all transfer requests from the guest to the kernel, which would give a
noticable better throughput.  The qemu usb subsystem can't handle that
(yet).  Fixing that is one the TODO list though.

Additionally all bulk xfer processing is done in the 1000 Hz frame
timer, which combined with the above limits the number of packets to
1000 packets per second (and direction), i.e. with a MTU of 1500 you'll
get 1500 * 1000 = 1.5 MB/s max.  This you can expect to get with the
current code.

Add some protocol and other overhead to the 7 MBit/s you are actually
seeing and the numbers are pretty close, so there isn't much room to
improve things.  The only option I see is to operate the device with an
larger MTU if the usb device and your network setup can handle that.

HTH,
  Gerd

Re: [Qemu-devel] Performance of USB2.0

[Til Obes]

Hi.

Am 01.11.2011 10:37, schrieb Gerd Hoffmann:
> I think the main issue here is that we don't do buffering / pipelining
> for bulk transfers at the moment.  We grab a single transfer request
> from the guest, pass it to the kernel, when it is done pass it back to
> the guest, then look look for the next one.  Instead we could queue up
> all transfer requests from the guest to the kernel, which would give a
> noticable better throughput.  The qemu usb subsystem can't handle that
> (yet).  Fixing that is one the TODO list though.
>
> Additionally all bulk xfer processing is done in the 1000 Hz frame
> timer, which combined with the above limits the number of packets to
> 1000 packets per second (and direction), i.e. with a MTU of 1500 you'll
> get 1500 * 1000 = 1.5 MB/s max.  This you can expect to get with the
> current code.
>
> Add some protocol and other overhead to the 7 MBit/s you are actually
> seeing and the numbers are pretty close, so there isn't much room to
> improve things.  The only option I see is to operate the device with an
> larger MTU if the usb device and your network setup can handle that.

That helps and explains why there is no speed difference between 1.1
and 2.0 at my tests. How can i help to prioritize this issue? ;)

Regards Til

Re: [Qemu-devel] Performance of USB2.0

[Til Obes]

Hello.

Am 01.11.2011 10:37, schrieb Gerd Hoffmann:
> I think the main issue here is that we don't do buffering / pipelining
> for bulk transfers at the moment.  We grab a single transfer request
> from the guest, pass it to the kernel, when it is done pass it back to
> the guest, then look look for the next one.  Instead we could queue up
> all transfer requests from the guest to the kernel, which would give a
> noticable better throughput.  The qemu usb subsystem can't handle that
> (yet).  Fixing that is one the TODO list though.

Any news about this topic?

Regards Til