USB passthrough burning 5% of CPU?

USB passthrough burning 5% of CPU?

[Gordan Bobic]

 Ever since I switched from passing a PCI USB controller
 through to a domU in favour of passing USB devices using
 the "usbdevice" config parameter, the domU affected
 seems to have qemu-dm "idling" at about 5% CPU usage in
 dom0. With the PCI USB controller passed through, this
 does not happen.

 Is this normal/expected? Does a similar continuous CPU
 overhead invisibly apply to PCI passthrough devices as
 well? Or is this CPU tax only applicable to USB
 passthrough?

 Gordan

Re: USB passthrough burning 5% of CPU?

[George Dunlap]

On Mon, Aug 5, 2013 at 1:28 PM, Gordan Bobic <gordan@bobich.net> wrote:
> Ever since I switched from passing a PCI USB controller
> through to a domU in favour of passing USB devices using
> the "usbdevice" config parameter, the domU affected
> seems to have qemu-dm "idling" at about 5% CPU usage in
> dom0. With the PCI USB controller passed through, this
> does not happen.
>
> Is this normal/expected? Does a similar continuous CPU
> overhead invisibly apply to PCI passthrough devices as
> well? Or is this CPU tax only applicable to USB
> passthrough?

Do you have the emulated USB controller enabled when you're passing
through the PCI USB controller?

My understanding was that certain USB devices must, by specification,
be polled something like a hundred times per second.  QEMU has to
emulate every one of these.  One of the things XenServer does for
really high-density VDI deployments, I believe, is disable USB in the
guests entirely.

Felipe, any comments?

 -George

Re: USB passthrough burning 5% of CPU?

[Ian Campbell]

On Mon, 2013-08-05 at 13:28 +0100, Gordan Bobic wrote:
>  Ever since I switched from passing a PCI USB controller
>  through to a domU in favour of passing USB devices using
>  the "usbdevice" config parameter, the domU affected
>  seems to have qemu-dm "idling" at about 5% CPU usage in
>  dom0. With the PCI USB controller passed through, this
>  does not happen.
> 
>  Is this normal/expected? Does a similar continuous CPU
>  overhead invisibly apply to PCI passthrough devices as
>  well? Or is this CPU tax only applicable to USB
>  passthrough?

I've heard that USB host controllers are expensive to virtualise because
USB has an effective 100ms (?) poll at the hardware level, which needs
to be managed (something to do with Isochronous URBs and/or the USB
schedule? I'm not 100% sure -- deeper USB voodoo than I can manage
anyway).

It had never occurred to me that this might also apply to usbdevice
style passthrough but since in that case, AIUI, you are running an
emulated host controller with a real device behind it it seems plausible
that this configuration would have that sort of overhead too.

Ian.

Re: USB passthrough burning 5% of CPU?

[Gordan Bobic]

 On Mon, 5 Aug 2013 14:57:09 +0100, George Dunlap <dunlapg@umich.edu> 
 wrote:
> On Mon, Aug 5, 2013 at 1:28 PM, Gordan Bobic <gordan@bobich.net> 
> wrote:
>> Ever since I switched from passing a PCI USB controller
>> through to a domU in favour of passing USB devices using
>> the "usbdevice" config parameter, the domU affected
>> seems to have qemu-dm "idling" at about 5% CPU usage in
>> dom0. With the PCI USB controller passed through, this
>> does not happen.
>>
>> Is this normal/expected? Does a similar continuous CPU
>> overhead invisibly apply to PCI passthrough devices as
>> well? Or is this CPU tax only applicable to USB
>> passthrough?
>
> Do you have the emulated USB controller enabled when you're passing
> through the PCI USB controller?

 Yes, but nothing is hanging off it (well, other than maybe the
 emulated VNC mouse/keyboard).

> My understanding was that certain USB devices must, by specification,
> be polled something like a hundred times per second.  QEMU has to
> emulate every one of these.  One of the things XenServer does for
> really high-density VDI deployments, I believe, is disable USB in the
> guests entirely.

 So it stands to reason that with no physical USB devices passed through
 the USB controller polling gets optimized out.

 Gordan

Re: USB passthrough burning 5% of CPU?

[George Dunlap]

On Mon, Aug 5, 2013 at 3:01 PM, Gordan Bobic <gordan@bobich.net> wrote:
> On Mon, 5 Aug 2013 14:57:09 +0100, George Dunlap <dunlapg@umich.edu> wrote:
>>
>> On Mon, Aug 5, 2013 at 1:28 PM, Gordan Bobic <gordan@bobich.net> wrote:
>>>
>>> Ever since I switched from passing a PCI USB controller
>>> through to a domU in favour of passing USB devices using
>>> the "usbdevice" config parameter, the domU affected
>>> seems to have qemu-dm "idling" at about 5% CPU usage in
>>> dom0. With the PCI USB controller passed through, this
>>> does not happen.
>>>
>>> Is this normal/expected? Does a similar continuous CPU
>>> overhead invisibly apply to PCI passthrough devices as
>>> well? Or is this CPU tax only applicable to USB
>>> passthrough?
>>
>>
>> Do you have the emulated USB controller enabled when you're passing
>> through the PCI USB controller?
>
>
> Yes, but nothing is hanging off it (well, other than maybe the
> emulated VNC mouse/keyboard).

>> My understanding was that certain USB devices must, by specification,
>> be polled something like a hundred times per second.  QEMU has to
>> emulate every one of these.  One of the things XenServer does for
>> really high-density VDI deployments, I believe, is disable USB in the
>> guests entirely.
>
>
> So it stands to reason that with no physical USB devices passed through
> the USB controller polling gets optimized out.

You could probably verify this by taking a xentrace and then running
xenalyze on it.  xenalyze can be found here:

http://xenbits.xensource.com/ext/xenalyze.hg

There's a very minimalist .html doc in the repo.  You want "--summary
--with-mmio-enumeration" I think.

It may be that the device you're passing through is more "active" wrt polling...

 -George

Re: USB passthrough burning 5% of CPU?

[Konrad Rzeszutek Wilk]

On Mon, Aug 05, 2013 at 01:28:39PM +0100, Gordan Bobic wrote:
> Ever since I switched from passing a PCI USB controller
> through to a domU in favour of passing USB devices using
> the "usbdevice" config parameter, the domU affected
> seems to have qemu-dm "idling" at about 5% CPU usage in
> dom0. With the PCI USB controller passed through, this
> does not happen.
> 
> Is this normal/expected? Does a similar continuous CPU
> overhead invisibly apply to PCI passthrough devices as
> well? Or is this CPU tax only applicable to USB
> passthrough?

I think that is about right. USB1 sucks at virtualization so
QEMU has to scan a list of memory regions to figure out if
the guest has decided that "hey, I've a got a packet of
data'. The CPU overhead for PCI passthrough does not happen
as the PCI device can detect where the request went and
do it stuff.

USB2.0 is not any better either. The XHCI on the other hand
is - but I am not sure if that is implemented in QEMU yet.


Ah, Gerd here has a nice update:
http://www.linux-kvm.org/wiki/images/b/be/2012-forum-qemu-usb-status-update.pdf

> 
> Gordan
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel