[Qemu-devel] UHCI idle detection

[Qemu-devel] UHCI idle detection

[Stefan Hajnoczi]

CPU utilization is a known issue with UHCI emulation.  I spent a short
time poking around the code and USB specifications trying to come up
with a way to detect "idle" periods where we don't need to poll the
frame list at 1000 Hz.

I was hoping to find a solution to detect an "idle" UHCI state, i.e. a
stable state where the guest is waiting for UHCI to report events and
the guest isn't issuing new transfers.  If the idle state can be
detected, then UHCI stops its frame timer and protects the frame list
and other control structure guest memory pages.

When the guest writes to those memory pages again in order to issue a
new USB transaction, we catch the write.  UHCI unprotects the guest
memory pages and turns the frame timer back on.

Unfortunately the UHCI frame list, transfer descriptor, and QH
structure is complicated.  I didn't figure out whether it is feasible
to protect all the necessary guest memory pages.

Also, I'm not sure how easy it is for QEMU to protect guest memory in
this fashion (for TCG and KVM).  It seems like the VGA dirty memory
stuff polls - it accumulates dirty memory bits and gets checked by the
VGA emulation code in the necessary places - rather than intercepts
writes.

Any thoughts on this approach?

Stefan

[Qemu-devel] Re: UHCI idle detection

[Gerd Hoffmann]

On 01/04/11 12:48, Stefan Hajnoczi wrote:
> CPU utilization is a known issue with UHCI emulation.  I spent a short
> time poking around the code and USB specifications trying to come up
> with a way to detect "idle" periods where we don't need to poll the
> frame list at 1000 Hz.

Check out

http://cgit.freedesktop.org/spice/qemu/log/?h=usb.3

It adds remote wakeup support to uhci, hub and hid emulation.  This 
allows the guest to suspend the usb bus when it is idle, the 1000 Hz 
wakeup goes away then.

Only problem with that is guests don't do that by default because there 
is way too much broken hardware out there.  That includes the current 
qemu usb hid emulation which advertises remote wakeup support although 
it isn't implemented :(

Windows guests needs some registry hackery and Linux guests some udev 
rules to enable remote wakeup permanently.

> I was hoping to find a solution to detect an "idle" UHCI state, i.e. a
> stable state where the guest is waiting for UHCI to report events and
> the guest isn't issuing new transfers.  If the idle state can be
> detected, then UHCI stops its frame timer and protects the frame list
> and other control structure guest memory pages.
 >
> When the guest writes to those memory pages again in order to issue a
> new USB transaction, we catch the write.  UHCI unprotects the guest
> memory pages and turns the frame timer back on.

It isn't that simple.  "idle" state depends on the usb devices too. 
With a usb tablet connected (and no remote-wakeup being used) uhci will 
poll the tablet in regular intervals and the usb tablet will either send 
the next event in the queue or NACK the transfer in case there is no 
event available.

cheers,
   Gerd

[Qemu-devel] Re: UHCI idle detection

[Stefan Hajnoczi]

On Tue, Jan 4, 2011 at 12:13 PM, Gerd Hoffmann <kraxel@redhat.com> wrote:
> On 01/04/11 12:48, Stefan Hajnoczi wrote:
>>
>> CPU utilization is a known issue with UHCI emulation.  I spent a short
>> time poking around the code and USB specifications trying to come up
>> with a way to detect "idle" periods where we don't need to poll the
>> frame list at 1000 Hz.
>
> Check out
>
> http://cgit.freedesktop.org/spice/qemu/log/?h=usb.3
>
> It adds remote wakeup support to uhci, hub and hid emulation.  This allows
> the guest to suspend the usb bus when it is idle, the 1000 Hz wakeup goes
> away then.
>
> Only problem with that is guests don't do that by default because there is
> way too much broken hardware out there.  That includes the current qemu usb
> hid emulation which advertises remote wakeup support although it isn't
> implemented :(
>
> Windows guests needs some registry hackery and Linux guests some udev rules
> to enable remote wakeup permanently.

That commit inspired me to look at UHCI.  If the solution requires
modifying the guest then it is not widely useful.

>> When the guest writes to those memory pages again in order to issue a
>> new USB transaction, we catch the write.  UHCI unprotects the guest
>> memory pages and turns the frame timer back on.
>
> It isn't that simple.  "idle" state depends on the usb devices too. With a
> usb tablet connected (and no remote-wakeup being used) uhci will poll the
> tablet in regular intervals and the usb tablet will either send the next
> event in the queue or NACK the transfer in case there is no event available.

Is there something preventing USB tablet emulation from using a
non-polling approach?  Devices need to be able to kick UHCI when data
becomes available.

Stefan

[Qemu-devel] Re: UHCI idle detection

[Gerd Hoffmann]

   Hi,

>> Windows guests needs some registry hackery and Linux guests some udev rules
>> to enable remote wakeup permanently.
>
> That commit inspired me to look at UHCI.  If the solution requires
> modifying the guest then it is not widely useful.

Well, long-term this shouldn't be a big issue.  I expect guest agents 
become commonplace soonish as some features require guest cooperation, 
so the guest agents can also care about this kind of tweaks.  Also for 
linux we can try to send the changes to upstream udev to have it spread 
into linux distros.

>>> When the guest writes to those memory pages again in order to issue a
>>> new USB transaction, we catch the write.  UHCI unprotects the guest
>>> memory pages and turns the frame timer back on.
>>
>> It isn't that simple.  "idle" state depends on the usb devices too. With a
>> usb tablet connected (and no remote-wakeup being used) uhci will poll the
>> tablet in regular intervals and the usb tablet will either send the next
>> event in the queue or NACK the transfer in case there is no event available.
>
> Is there something preventing USB tablet emulation from using a
> non-polling approach?  Devices need to be able to kick UHCI when data
> becomes available.

remote wakeup is exactly that.  Except that it doesn't kick the uhci 
directly but the port it is connected to, which isn't the same in case 
the device is connected using a usb hub.  Which probably makes it tricky 
to re-use the remote wakeup callback for the idle detection.

cheers,
   Gerd

[Qemu-devel] Re: UHCI idle detection

[Anthony Liguori]

On 01/04/2011 07:43 AM, Gerd Hoffmann wrote:
>   Hi,
>
>>> Windows guests needs some registry hackery and Linux guests some 
>>> udev rules
>>> to enable remote wakeup permanently.
>>
>> That commit inspired me to look at UHCI.  If the solution requires
>> modifying the guest then it is not widely useful.
>
> Well, long-term this shouldn't be a big issue.  I expect guest agents 
> become commonplace soonish as some features require guest cooperation, 
> so the guest agents can also care about this kind of tweaks.  Also for 
> linux we can try to send the changes to upstream udev to have it 
> spread into linux distros.

I think we're long overdue for a paravirtual mouse.  Basically, a virtio 
version of xenkbd-front.c.  In fact, it's probably possible to reuse the 
protocol.

Regards,

Anthony Liguori

>
>>>> When the guest writes to those memory pages again in order to issue a
>>>> new USB transaction, we catch the write.  UHCI unprotects the guest
>>>> memory pages and turns the frame timer back on.
>>>
>>> It isn't that simple.  "idle" state depends on the usb devices too. 
>>> With a
>>> usb tablet connected (and no remote-wakeup being used) uhci will 
>>> poll the
>>> tablet in regular intervals and the usb tablet will either send the 
>>> next
>>> event in the queue or NACK the transfer in case there is no event 
>>> available.
>>
>> Is there something preventing USB tablet emulation from using a
>> non-polling approach?  Devices need to be able to kick UHCI when data
>> becomes available.
>
> remote wakeup is exactly that.  Except that it doesn't kick the uhci 
> directly but the port it is connected to, which isn't the same in case 
> the device is connected using a usb hub.  Which probably makes it 
> tricky to re-use the remote wakeup callback for the idle detection.
>
> cheers,
>   Gerd

[Qemu-devel] Re: UHCI idle detection

[Gerd Hoffmann]

On 01/04/11 14:49, Anthony Liguori wrote:
> On 01/04/2011 07:43 AM, Gerd Hoffmann wrote:
>> Hi,
>>
>>>> Windows guests needs some registry hackery and Linux guests some
>>>> udev rules
>>>> to enable remote wakeup permanently.
>>>
>>> That commit inspired me to look at UHCI. If the solution requires
>>> modifying the guest then it is not widely useful.
>>
>> Well, long-term this shouldn't be a big issue. I expect guest agents
>> become commonplace soonish as some features require guest cooperation,
>> so the guest agents can also care about this kind of tweaks. Also for
>> linux we can try to send the changes to upstream udev to have it
>> spread into linux distros.
>
> I think we're long overdue for a paravirtual mouse. Basically, a virtio
> version of xenkbd-front.c. In fact, it's probably possible to reuse the
> protocol.

Oh, there already is one.  vmmouse.  Recent Xorg versions even use it 
automagically.  With Fedora 14 (as guest) you can drop the usb tablet 
and you still have an absolute mouse pointer because of that ;)

Windows is more tricky as it doesn't work out-of-the-box but that 
wouldn't be different with a virtio-based mouse.

cheers,
   Gerd

[Qemu-devel] Re: UHCI idle detection

[Anthony Liguori]

On 01/04/2011 08:16 AM, Gerd Hoffmann wrote:
> On 01/04/11 14:49, Anthony Liguori wrote:
>> On 01/04/2011 07:43 AM, Gerd Hoffmann wrote:
>>> Hi,
>>>
>>>>> Windows guests needs some registry hackery and Linux guests some
>>>>> udev rules
>>>>> to enable remote wakeup permanently.
>>>>
>>>> That commit inspired me to look at UHCI. If the solution requires
>>>> modifying the guest then it is not widely useful.
>>>
>>> Well, long-term this shouldn't be a big issue. I expect guest agents
>>> become commonplace soonish as some features require guest cooperation,
>>> so the guest agents can also care about this kind of tweaks. Also for
>>> linux we can try to send the changes to upstream udev to have it
>>> spread into linux distros.
>>
>> I think we're long overdue for a paravirtual mouse. Basically, a virtio
>> version of xenkbd-front.c. In fact, it's probably possible to reuse the
>> protocol.
>
> Oh, there already is one.  vmmouse.  Recent Xorg versions even use it 
> automagically.  With Fedora 14 (as guest) you can drop the usb tablet 
> and you still have an absolute mouse pointer because of that ;)
>
> Windows is more tricky as it doesn't work out-of-the-box but that 
> wouldn't be different with a virtio-based mouse.

vmmouse relies on the VMware backdoor interface.  The backdoor interface 
requires that a certain PIO port be accessible from CPL=3 even if IOPL=0.

It works in Linux because Xorg generally changes IOPL=3 because it 
implements device drivers but in the long run, I hope that no longer 
becomes true.  With Windows, I've always been under the impression that 
their input driver runs in CPL=3 and there is no interface to change IOPL.

Supporting the backdoor interface properly in KVM would be really ugly.  
We'd have to shadow the IDT which means write protecting it and all of 
the ugliness that goes along with it.

Regards,

Anthony Liguori

> cheers,
>   Gerd
>

Re: [Qemu-devel] Re: UHCI idle detection

[Alexander Graf]

On 04.01.2011, at 15:22, Anthony Liguori wrote:

> On 01/04/2011 08:16 AM, Gerd Hoffmann wrote:
>> On 01/04/11 14:49, Anthony Liguori wrote:
>>> On 01/04/2011 07:43 AM, Gerd Hoffmann wrote:
>>>> Hi,
>>>> 
>>>>>> Windows guests needs some registry hackery and Linux guests some
>>>>>> udev rules
>>>>>> to enable remote wakeup permanently.
>>>>> 
>>>>> That commit inspired me to look at UHCI. If the solution requires
>>>>> modifying the guest then it is not widely useful.
>>>> 
>>>> Well, long-term this shouldn't be a big issue. I expect guest agents
>>>> become commonplace soonish as some features require guest cooperation,
>>>> so the guest agents can also care about this kind of tweaks. Also for
>>>> linux we can try to send the changes to upstream udev to have it
>>>> spread into linux distros.
>>> 
>>> I think we're long overdue for a paravirtual mouse. Basically, a virtio
>>> version of xenkbd-front.c. In fact, it's probably possible to reuse the
>>> protocol.
>> 
>> Oh, there already is one.  vmmouse.  Recent Xorg versions even use it automagically.  With Fedora 14 (as guest) you can drop the usb tablet and you still have an absolute mouse pointer because of that ;)
>> 
>> Windows is more tricky as it doesn't work out-of-the-box but that wouldn't be different with a virtio-based mouse.
> 
> vmmouse relies on the VMware backdoor interface.  The backdoor interface requires that a certain PIO port be accessible from CPL=3 even if IOPL=0.
> 
> It works in Linux because Xorg generally changes IOPL=3 because it implements device drivers but in the long run, I hope that no longer becomes true.  With Windows, I've always been under the impression that their input driver runs in CPL=3 and there is no interface to change IOPL.
> 
> Supporting the backdoor interface properly in KVM would be really ugly.  We'd have to shadow the IDT which means write protecting it and all of the ugliness that goes along with it.

Are you sure this is still true for recent versions of VMware? I don't see how they'd hold onto this with vmx.


Alex