Re: [RFC PATCH 2/4] KVM: x86: Add KVM exit for IOAPIC EOIs

[Avi Kivity <avi.kivity@gmail.com>]

On 05/27/2015 05:06 AM, Steve Rutherford wrote:
> On Sun, May 24, 2015 at 07:46:03PM +0300, Avi Kivity wrote:
>> On 05/13/2015 04:47 AM, Steve Rutherford wrote:
>>> Adds KVM_EXIT_IOAPIC_EOI which passes the interrupt vector up to
>>> userspace.
>>>
>>> Uses a per VCPU exit bitmap to decide whether or not the IOAPIC needs
>>> to be informed (which is identical to the EOI_EXIT_BITMAP field used
>>> by modern x86 processors, but can also be used to elide kvm IOAPIC EOI
>>> exits on older processors).
>>>
>>> [Note: A prototype using ResampleFDs found that decoupling the EOI
>> >from the VCPU's thread made it possible for the VCPU to not see a
>>> recent EOI after reentering the guest. This does not match real
>>> hardware.]
>>>
>>> Compile tested for Intel x86.
>>>
>>> Signed-off-by: Steve Rutherford <srutherford@google.com>
>>> ---
>>>   Documentation/virtual/kvm/api.txt | 10 ++++++++++
>>>   arch/x86/include/asm/kvm_host.h   |  3 +++
>>>   arch/x86/kvm/lapic.c              |  9 +++++++++
>>>   arch/x86/kvm/x86.c                | 11 +++++++++++
>>>   include/linux/kvm_host.h          |  1 +
>>>   include/uapi/linux/kvm.h          |  5 +++++
>>>   6 files changed, 39 insertions(+)
>>>
>>> diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
>>> index 0744b4e..dd92996 100644
>>> --- a/Documentation/virtual/kvm/api.txt
>>> +++ b/Documentation/virtual/kvm/api.txt
>>> @@ -3285,6 +3285,16 @@ Valid values for 'type' are:
>>>   	 */
>>>   	__u64 kvm_valid_regs;
>>>   	__u64 kvm_dirty_regs;
>>> +
>>> +	/* KVM_EXIT_IOAPIC_EOI */
>>> +        struct {
>>> +	       __u8 vector;
>>> +        } eoi;
>>> +
>>> +Indicates that an eoi of a level triggered IOAPIC interrupt on vector has
>>> +occurred, which should be handled by the userspace IOAPIC. Triggers when
>>> +the Irqchip has been split between userspace and the kernel.
>>> +
>> The ioapic is a global resource, so it doesn't make sense for
>> information about it to be returned in a per-vcpu structure
> EOI exits are a per-vcpu behavior, so this doesn't seem all that strange.
>
>> (or to block the vcpu while it is being processed).
> Blocking doesn't feel clean, but doesn't seem all that bad, given
> that these operations are relatively rare on modern configurations.

Agree, maybe the realtime people have an interest here.

>> The way I'd model it is to emulate the APIC bus that connects local
>> APICs and the IOAPIC, using a socket pair.  When the user-space
>> ioapic wants to inject an interrupt, it sends a message to the local
>> APICs which then inject it, and when it's ack'ed the EOI is sent
>> back on the same bus.
> Although I'm not certain about this, it sounds to me like this would
> require a kernel thread to be waiting (in some way) on this socket, which
> seems rather heavy handed.

It's been a while since I did kernel programming, but I think you can 
queue a callback to be called when an I/O is ready, and not require a 
thread.  IIRC we do that with irqfd to cause an interrupt to be injected.