Re: [PATCH 00/53] dyn_array/nr_irqs/sparse_irq support v10

[ebiederm@xmission.com (Eric W. Biederman)]

"Yinghai Lu" <yhlu.kernel@gmail.com> writes:

> On Thu, Aug 14, 2008 at 5:11 PM, Eric W. Biederman
> <ebiederm@xmission.com> wrote:
>> "Yinghai Lu" <yhlu.kernel@gmail.com> writes:
>>
>>> find something interesting:
>>>
>>> found new irq_cfg for irq 20
>>>  0 add_pin_to_irq: irq 20 --> apic 0 pin 20
>>> assign_irq_vector: irq 20 vector 0x59 cpu  5
>>> IOAPIC[0]: Set routing entry (0-20 -> 0x59 -> IRQ 20 Mode:1 Active:1)
>>> found new irq_desc for irq 20
>>> pci 0000:00:02.1: PCI INT B -> Link[LUS2] -> GSI 20 (level, low) -> IRQ 20
>>>
>>> IO APIC #0......
>>> .... register #00: 00000000
>>> .......    : physical APIC id: 00
>>> .......    : Delivery Type: 0
>>> .......    : LTS          : 0
>>> .... register #01: 00170011
>>> .......     : max redirection entries: 0017
>>> .......     : PRQ implemented: 0
>>> .......     : IO APIC version: 0011
>>> .... register #02: 00000000
>>> .......     : arbitration: 00
>>> .... IRQ redirection table:
>>>  NR Dst Mask Trig IRR Pol Stat Dmod Deli Vect:
>>> ...
>>>  14 09  1    1    0   1   0    0    0    59
>>> ...
>>>
>>> ehci_hcd 0000:00:02.1: USB 2.0 started, EHCI 1.00, driver 10 Dec 2004
>>> do_IRQ: cannot handle IRQ -1 vector 0x59 cpu 0
>>> ------------[ cut here ]------------
>>> Kernel BUG at 40206b11 [verbose debug info unavailable]
>>> invalid opcode: 0000 [#1] SMP
>>> Modules linked in:
>>>
>>> Pid: 70, comm: kasyncinit Not tainted (2.6.27-rc3-tip-00191-g98ccb89-dirty
> #23)
>>> EIP: 0060:[<40206b11>] EFLAGS: 00010092 CPU: 0
>>> EIP is at do_IRQ+0x6b/0xae
>>> EAX: 00000032 EBX: 00001d28 ECX: 00003434 EDX: 00000046
>>> ESI: 00000000 EDI: 00000059 EBP: c7a37d3c ESP: c7a37d14
>>>  DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
>>> Process kasyncinit (pid: 70, ti=c7a36000 task=c79c9860 task.ti=c7a36000)
>>> Stack: 40a74317 40814ea0 ffffffff 00000059 00000000 00000000 ffffffff
> c421bd20
>>> c7a37d9c c7a37dac c7a37d7c 4020555f c421bd20 00000000 c421bd20 c7a37d9c
>>> c7a37dac c7a37d7c 40b717f8 0000007b 0000007b 000000d8 ffffffa6 4080db6c
>>> Call Trace:
>>>  [<4020555f>] ? common_interrupt+0x23/0x28
>>>
>>>
>>> it is on 16cores system with 32bit bigsmp, so it is using phy_flat
>>> cpu 5 has apicid 9, and ioapic reg setting right with Dmod= 0 ( phys)
>>>
>>> but io_apic controller deliver that interrupt to cpu0 (with apicid =
>>> 4) instead of cpu 5 (with apic id = 9)
>>>
>>> look at the 64 bit, TARGET_CPUS for phys_flat is cpu_online_map
>>>
>>> and 32bit bigsmp TARGET_CPUS is only one cpu set and rotating with online
> cpu...
>>>
>>> Change 32bit bigsmp TARGE_CPUS ?
>>
>> Set vector_allocation_domain to CPU_MASK_ALL on 32bit.  That doesn't give us
>> the benefit of per cpu vectors right now, but in my research there has not
>> been a 32bit kernel yet that has needed it.  We have never shared vectors
>> between 2 gsi on 32bit x86, we have only collapsed the irq space.
>
> TARGET_CPUS only used by ioapic_xx.c
>
> vector_allocator_domain will return cpumask_of_cpu(cpu)..

Note: Only vector_allocation_domain matters.  That is where
we setup the irq to be received.  TARGET_CPUS is just a request
to send the irq to someplace in that mask.

Eric