Re: [PATCH v3] enable x2APIC without interrupt remapping under KVM

[ebiederm@xmission.com (Eric W. Biederman)]

Avi Kivity <avi@redhat.com> writes:

> On 06/30/2009 07:44 PM, Eric W. Biederman wrote:
>>>> To support that seriously you need interrupt remapping.
>>>>
>>>>        
>>> Can you explain why?
>>>      
>>
>> Because ioapics don't fully function according to spec,
>> and the interrupt code on the hotplug path is a horrible
>> terrible broken hack for ioapics.
>>
>> It is better than nothing but it certainly is not something
>> I would expect to work all of the time.
>>    
>
> Can you elaborate?  For kvm guests, the hardware is reasonably will implemented
> and if not we will fix it.  We need not cripple a feature just because some
> hardware is broken.

The short version is I don't know what work arounds we will ultimately
decide to deploy to work with real hardware.  

I have been seriously contemplating causing a cpu hot-unplug request
to fail if we are in ioapic mode and we have irqs routed to the cpu
that is being unplugged.

Even with perfectly working hardware it is not possible in the general
case to migrate an ioapic irq from one cpu to another outside of an
interrupt handler without without risking dropping an interrupt.
There is no general way to know you have seen the last interrupt
floating around your system.  PCI ordering rules don't help because
the ioapics can potentially take an out of band channel.

The interrupt remapping hardware is pci and is only present on systems
with in-band interrupts, so we can flush in-flight interrupts using
pci reads.  We can safely and do reprogram the hardware to point at
different cpus at arbitrary times.

So if you possibly can please emulate hardware that is known to work
reliably for cpu hotplug.

Eric