Re: [kvm-ia64-devel] IRQ assignment

[Avi Kivity <avi@qumranet.com>]

Alexander Graf wrote:
>
> On May 21, 2008, at 6:07 PM, Xu, Anthony wrote:
>
>> Avi Kivity wrote:
>>> Xu, Anthony wrote:
>>>> Xiantao and I have found the root cause,
>>>> Qemu emulates PIIX chipset, all pci devices can only use irq 10.11,
>>>> which is confiured inside chipset interrupt routing table. Even
>>>> though IOAPIC have 24 interrupt pins.
>>>> While KVM/IA64 use the same Guest Firmware with what XEN/IA64 which
>>>> use different "interrupt routing algorithm".
>>>> Means the pci device irq doesn't match between qemu and Guest
>>>> Firmware in KVM/IA64. So guest didn't get pci device interrupt.
>>>>
>>>> Obviously there are two ways to fix it.
>>>> 1. modify qemu side, all pci devices use irq larger than or equal to
>>>> 16, we need to come out an algorithm to calculate irq from pci
>>>> device(bus number,device number, function number),
>>>>    then we also need to modify IA32 Guest BIOS to present the same
>>>> pci device irq (use same algorithm) to guest OS. Avi seems not want
>>>> to modify qemu a lot.
>>>>
>>>> 2. modify IA64 guest firmware, two pros, (1)all pci devices use only
>>>> 10,11 two irqs, so if there are many pci device, there are a lot of
>>>> interrupt sharing, which impact performance negatively
>>>>    (2) We need to maintain two versions fo IA64 guest firmwares, one
>>>> for KVM/IA64, the other for XEN/IA64, which is not what I want.
>>>>
>>>>
>>>> What's your suggestion?
>>>>
>>>>
>>>
>>> Allowing qemu to use all ioapic interrupt pins will reduce interrupt
>>> sharing on x86, which is a good thing, so I prefer the first option
>>> too.
>>
>>
>> Thanks for your support, I preper option #1,
>> Any suggestion for the mapping from BDF to irq.
>>
>> In XEN both in IA64/IA32,
>>
>> BIOS provides a 48 pin IOAPIC ( usually it is 24) to reduce irq sharing.
>
> Most mainboards these days provide two IOAPICs, which would sum up to 
> 48 again. I think that should be the preferred way of implementing it 
> virtually too.
>

I agree.  ia64 has a preference for a single 48-pin ioapic for Xen 
compatibility, but x86 and ia64 needn't be exactly equal.

On the other hand, adding a new ioapic will be more difficult than 
extending an existing one.

On yet another hand, two ioapics (each with its own lock) will improve 
scaling.

On the fourth hand (anyone for Bridge?), if we hit ioapic scalability 
problems, each pin should have its own lock.

> The idea is great! I tried extending the IRQ logic to a "full IOAPIC" 
> myself recently, but failed miserably. The biggest hurdle is that 
> currently the code is reversed in qemu. If an interrupt occurs, the 
> PIC is asked if it's destined to go there and if not it gets rerouted 
> to the IOAPIC. Unfortunately this breaks with IRQs > 16.

Shouldn't each irq be routed to *both* chips, and the OS disables one or 
the other?

-- 
error compiling committee.c: too many arguments to function