Re: [kvm-ia64-devel] IRQ assignment

Re: [kvm-ia64-devel] IRQ assignment

[Avi Kivity]

Xu, Anthony wrote:
>> Allowing qemu to use all ioapic interrupt pins will reduce interrupt
>> sharing on x86, which is a good thing, so I prefer the first option
>> too. 
>>     
>
>
> Thanks for your support, I preper option #1,
> Any suggestion for the mapping from BDF to irq.
>
> In XEN both in IA64/IA32,
>
> BIOS provides a 48 pin IOAPIC ( usually it is 24) to reduce irq sharing.
> 0~15 are reserved for legacy devices.
> Pci devices use 16~47,
> The mapping is like
>   ((bdf >> 3) *4) %(48-16) + 16
> Means every pci interrup pin( irqA, irqB, irqC, irqD) of every pci
> device use different irq pin of IOAPIC if number of pci devices is less
> than 8.
> I think it can avoid interrupt sharing in most case.
>
>   

With 24 free pins, that's fine.  With 8 free pins, less so.  We'll need 
to mix in more high bits.

I guess we need to increase the number of pins on x86 too.

> If use this method, we can share same IA64 guest BIOS between XEN/IA64
> and KVM/IA64.
>   

You can use this method for ia64, and we'll have a different function 
for x86 (perhaps two functions, if we later increase the number of pins 
to 48 (or even more); the DSDT will need to select the appropriate 
routing table according to what's present on the hardware).

-- 
error compiling committee.c: too many arguments to function

RE: [kvm-ia64-devel] IRQ assignment

[Xu, Anthony]

Avi Kivity wrote:
> 
> With 24 free pins, that's fine.  With 8 free pins, less so.  We'll
> need to mix in more high bits.
> 
> I guess we need to increase the number of pins on x86 too.
> 
>> If use this method, we can share same IA64 guest BIOS between
>> XEN/IA64 and KVM/IA64. 
>> 
> 
> You can use this method for ia64, and we'll have a different function
> for x86 (perhaps two functions, if we later increase the number of
> pins to 48 (or even more); the DSDT will need to select the
> appropriate routing table according to what's present on the
> hardware). 
It can work if X86 and ia64 implement different "->map" function.

Use this kind of "fixed" algorithm may waste IOAPIC interrupt pin, due
to not every pci device will use up 4 irq.

Is it possible to let qemu dynamically build route table in DSDT?

Anthony


> 
> --
> error compiling committee.c: too many arguments to function

Re: [kvm-ia64-devel] IRQ assignment

[Avi Kivity]

Xu, Anthony wrote:
>>
>> You can use this method for ia64, and we'll have a different function
>> for x86 (perhaps two functions, if we later increase the number of
>> pins to 48 (or even more); the DSDT will need to select the
>> appropriate routing table according to what's present on the
>> hardware). 
>>     
> It can work if X86 and ia64 implement different "->map" function.
>
> Use this kind of "fixed" algorithm may waste IOAPIC interrupt pin, due
> to not every pci device will use up 4 irq.
>
> Is it possible to let qemu dynamically build route table in DSDT?
>   

x86 and ia64 have different DSDTs, so I don't see the need for dynamic 
generation.  The x86 DSDT can return different routing tables depending 
on whether one or two ioapics are present (this can be detected at runtime).

-- 
error compiling committee.c: too many arguments to function

RE: [kvm-ia64-devel] IRQ assignment

[Xu, Anthony]

Avi Kivity wrote:
> 
> x86 and ia64 have different DSDTs, so I don't see the need for dynamic
> generation.  The x86 DSDT can return different routing tables
> depending 
> on whether one or two ioapics are present (this can be detected at
> runtime). 

Okay, use fixed DSDT

- Anthony

Re: [kvm-ia64-devel] IRQ assignment

[Avi Kivity]

Alexander Graf wrote:
>
> On May 21, 2008, at 6:07 PM, Xu, Anthony wrote:
>
>> Avi Kivity wrote:
>>> Xu, Anthony wrote:
>>>> Xiantao and I have found the root cause,
>>>> Qemu emulates PIIX chipset, all pci devices can only use irq 10.11,
>>>> which is confiured inside chipset interrupt routing table. Even
>>>> though IOAPIC have 24 interrupt pins.
>>>> While KVM/IA64 use the same Guest Firmware with what XEN/IA64 which
>>>> use different "interrupt routing algorithm".
>>>> Means the pci device irq doesn't match between qemu and Guest
>>>> Firmware in KVM/IA64. So guest didn't get pci device interrupt.
>>>>
>>>> Obviously there are two ways to fix it.
>>>> 1. modify qemu side, all pci devices use irq larger than or equal to
>>>> 16, we need to come out an algorithm to calculate irq from pci
>>>> device(bus number,device number, function number),
>>>>    then we also need to modify IA32 Guest BIOS to present the same
>>>> pci device irq (use same algorithm) to guest OS. Avi seems not want
>>>> to modify qemu a lot.
>>>>
>>>> 2. modify IA64 guest firmware, two pros, (1)all pci devices use only
>>>> 10,11 two irqs, so if there are many pci device, there are a lot of
>>>> interrupt sharing, which impact performance negatively
>>>>    (2) We need to maintain two versions fo IA64 guest firmwares, one
>>>> for KVM/IA64, the other for XEN/IA64, which is not what I want.
>>>>
>>>>
>>>> What's your suggestion?
>>>>
>>>>
>>>
>>> Allowing qemu to use all ioapic interrupt pins will reduce interrupt
>>> sharing on x86, which is a good thing, so I prefer the first option
>>> too.
>>
>>
>> Thanks for your support, I preper option #1,
>> Any suggestion for the mapping from BDF to irq.
>>
>> In XEN both in IA64/IA32,
>>
>> BIOS provides a 48 pin IOAPIC ( usually it is 24) to reduce irq sharing.
>
> Most mainboards these days provide two IOAPICs, which would sum up to 
> 48 again. I think that should be the preferred way of implementing it 
> virtually too.
>

I agree.  ia64 has a preference for a single 48-pin ioapic for Xen 
compatibility, but x86 and ia64 needn't be exactly equal.

On the other hand, adding a new ioapic will be more difficult than 
extending an existing one.

On yet another hand, two ioapics (each with its own lock) will improve 
scaling.

On the fourth hand (anyone for Bridge?), if we hit ioapic scalability 
problems, each pin should have its own lock.

> The idea is great! I tried extending the IRQ logic to a "full IOAPIC" 
> myself recently, but failed miserably. The biggest hurdle is that 
> currently the code is reversed in qemu. If an interrupt occurs, the 
> PIC is asked if it's destined to go there and if not it gets rerouted 
> to the IOAPIC. Unfortunately this breaks with IRQs > 16.

Shouldn't each irq be routed to *both* chips, and the OS disables one or 
the other?

-- 
error compiling committee.c: too many arguments to function