[Qemu-devel] Re: [RFC] Moving the kvm ioapic, pic, and pit back to userspace

[Avi Kivity <avi@redhat.com>]

On 06/08/2010 01:23 AM, Anthony Liguori wrote:
>>> A better example would be a generic counter kernel mechanism.  I can 
>>> envision such a device as doing nothing more than providing a 
>>> read-only view of a counter with a userspace configurable divider 
>>> and width.  Any write to the counter or read of any other byte 
>>> outside the counter register would result in a trap to userspace.
>>
>> What about latches?  byte access to word registers?  There will be as 
>> many special cases as there are timers.
>>
>> If the kernel supported a bytecode/jit facility I'd happily use that 
>> to download portions of the device model into the kernel.
>>
>>>
>>> That should allow both the PIT and the HPET to be accelerated with 
>>> minimal effort in the kernel.
>>
>> IMO it's probably more effort than porting HPET to the kernel.  Try 
>> outlining an interface that supports PIT, HPET, RTC, and ACPI PMTIMER.
>
>
> I was referring specifically to time sources, not time events.
>
> An accelerated counter for HPET is pretty trivial.  It's a 32-bit 
> register that's actually a nanosecond value in qemu.  We need to be 
> able to set an offset from the host wall clock time, a means to stop 
> it, and a means to start it.
>
> The PIT is latched so the kernel needs to know enough about how to 
> decode the PIT state to understand the latching.  There's very little 
> state associated with latching though so I don't think this is a huge 
> problem.  It's a fixed value write to a fixed register followed by a 
> read to a fixed register.  The act of latching doesn't effect the 
> state beyond the fact that you need to save the latched value in the 
> event that you have a live migration before reading the latched value.
>
> The PMTIMER is also pretty straight forward.  It's a variable port 
> address (that's fixed during execution).
>
> Even if we require three separate interfaces, the interfaces are so 
> simply that it seems like an obvious win.

So a non-generic interface - 4x the interfaces (including RTC).

Those counters raise interrupts when they expire, and set various status 
bits in their hardware.  So we need 4x of:

   set counter value, frequency, and reload interval
   raise alarm to userspace on expiration
   set counter memory/ioport location and availability
   read counter value

and we haven't solved interrupt coalescing.

>
>>>
>>>> 5. Risk
>>>>
>>>> We may find out after all this is implemented that performance is 
>>>> not acceptable and all the work will have to be dropped.
>>>
>>> That's another advantage to a straight port to userspace.  We can 
>>> collect performance data with only a modest amount of engineering 
>>> effort.
>>
>> Port what exactly?  We have a userspace irqchip implementation.  What 
>> we don't have is just the ioapic/pic/pit in userspace, and the only 
>> way to try it out is to implement the whole thing.
>
> If you take the kernel code and do a pretty straight port: switching 
> kernel functions to libc functions and maintaining all the existing 
> locking via pthreads, you could then implement a very simple MMIO/PIO 
> dispatch mechanism in the kvm code that shortcutted those devices 
> before we ever hit the qemu_mutex and the traditional qemu code 
> paths.  It should be a relatively easy conversion and it gives a 
> proper vehicle for doing experimentations.

Those devices don't exist independently of the rest of the devices.  If 
they need to post interrupts, they will need the traditional qemu code 
paths.

(I'm trying to view the move from the POV of the kernel first, assuming 
userspace is as efficient as possible; so I'm not arguing qemu 
inefficiencies should prevent us from doing it.  But they do add up 
considerably to the amount of work involved)

>
> In fact, you could pretty quickly determine viability by porting the 
> PIT to userspace and implementing a vpit interface in the kernel that 
> allowed the channel 0 counters to be latched and read within 
> lightweight exits.


Just looking at it shows the interface is incredibly messy.  You have to 
maintain the control word in the kernel (since it tells you which 
counter to read or write), so now you need a userspace interface to read 
and write the control word.  With the current interface, you have the 
entire thing in a black box that you don't need to worry about (except 
for the speaker port...).


-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.