Re: Questions on vpic, vlapic, vioapic and line 0 (aka timer)

[Don Slutz <dslutz@verizon.com>]

On 02/20/14 03:58, Jan Beulich wrote:
>>>> On 19.02.14 at 16:14, Don Slutz <dslutz@verizon.com> wrote:
>> For some TBD reason (very very rarely) the routine timer_irq_works() in linux
>> is reporting that the timer IRQ does not work:
>>
>> ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
>> ..MP-BIOS bug: 8254 timer not connected to IO-APIC
>> ...trying to set up timer (IRQ0) through the 8259A ...
>> ..... (found apic 0 pin 2) ...
>> ....... failed.
>> ...trying to set up timer as Virtual Wire IRQ...
>> ..... failed.
>> ...trying to set up timer as ExtINT IRQ...
>>
>> hangs and xen's console is spewing:
>>
>> vioapic.c:352:d1 Unsupported delivery mode 7
>> vioapic.c:352:d1 Unsupported delivery mode 7
>> vioapic.c:352:d1 Unsupported delivery mode 7
>> ...
>>
>> I have determined that the lines (in linux):
>>
>> #ifdef CONFIG_X86_IO_APIC
>>           no_timer_check = 1;
>> #endif
>>
>> that KVM has are missing for Xen.  The simple workaround is to specify
>> "no_timer_check" on the kernel command args.
>>
>> The reason for the email is that I have found the routine
>> __vlapic_accept_pic_intr:
>>
>>       /* We deliver 8259 interrupts to the appropriate CPU as follows. */
>>       return ((/* IOAPIC pin0 is unmasked and routing to this LAPIC? */
>>                ((redir0.fields.delivery_mode == dest_ExtINT) &&
>>                 !redir0.fields.mask &&
>>                 redir0.fields.dest_id == VLAPIC_ID(vlapic) &&
>>                 !vlapic_disabled(vlapic)) ||
>>                /* LAPIC has LVT0 unmasked for ExtInts? */
>>                ((lvt0 & (APIC_MODE_MASK|APIC_LVT_MASKED)) == APIC_DM_EXTINT) ||
>>                /* LAPIC is fully disabled? */
>>                vlapic_hw_disabled(vlapic)));
>> }
>>
>>
>> Which looks to imply that the vioapic supports "delivery mode 7"
>> (dest_ExtINT), but this case is missing (the message logged above).
> Not really - the code above suggests the LAPIC emulation is
> prepared for the IOAPIC emulation to support that mode, not
> that the IOAPIC one supports it. At present only LAPIC LVT0
> really supports that delivery mode.

Well, the linux kernel that I have been testing with only sets LAPIC LVT0 to 7:


         init_8259A(0);
         make_8259A_irq(0);
         apic_write(APIC_LVT0, APIC_DM_EXTINT);


And that agrees with xen-hvmctx:

dcs-xen-54:~/xen>sudo xen-hvmctx 8 | grep -A7 LINT0
       LVT LINT0  : 00000700
         mask     = 0
         trigger  = edge
         rem irr  = 0
         polarty  = active hi
         status   = idle
         delivry  = ExtINT
         vector   = 00

(Patch in the works for this decode).

And the ioapic does NOT have this mode set:

     IOAPIC: base_address 0xfec00000, ioregsel 0x20 id 0
         pin dsm dsa msk trig rir polarity dls  dlm    vec raw
         00: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         01: phys 00  0  edge   0 activehi idle Fixed   49 0x0000000000000031
         02: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         03: phys 00  0  edge   0 activehi idle Fixed   51 0x0000000000000033
         04: phys 00  0  edge   0 activehi idle Fixed   52 0x0000000000000034
         05: phys 00  1  level  0 activelo idle Fixed   53 0x000000000001a035
         06: phys 00  0  edge   0 activehi idle Fixed   54 0x0000000000000036
         07: phys 00  0  edge   0 activehi idle Fixed   55 0x0000000000000037
         08: phys 00  0  edge   0 activehi idle Fixed   56 0x0000000000000038
         09: phys 00  1  level  0 activelo idle Fixed   57 0x000000000001a039
         10: phys 00  1  level  0 activelo idle Fixed   58 0x000000000001a03a
         11: phys 00  1  level  0 activelo idle Fixed   59 0x000000000001a03b
         12: phys 00  0  edge   0 activehi idle Fixed   60 0x000000000000003c
         13: phys 00  0  edge   0 activehi idle Fixed   61 0x000000000000003d
         14: phys 00  0  edge   0 activehi idle Fixed   62 0x000000000000003e
         15: phys 00  0  edge   0 activehi idle Fixed   63 0x000000000000003f
         16: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         17: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         18: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         19: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         20: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         21: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         22: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         23: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         24: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         25: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         26: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         27: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         28: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         29: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         30: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         31: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         32: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         33: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         34: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         35: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         36: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         37: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         38: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         39: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         40: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         41: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         42: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         43: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         44: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         45: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         46: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000
         47: phys 00  1  edge   0 activehi idle Fixed    0 0x0000000000010000

So I am now more confused that vioapic.c is reporting this message.


>> Also linux and xen both set "LAPIC has LVT0" to APIC_DM_FIXED for "Virtual
>> Wire IRQ" usage, but this code only allows for APIC_DM_EXTINT.  I have been
>> able to get "Virtual Wire IRQ" usage to work by adding:
>>
>>                /* LAPIC has LVT0 unmasked for Fixed? */
>>                ((lvt0 & (APIC_MODE_MASK|APIC_LVT_MASKED)) == APIC_DM_FIXED) ||
>>
>> It is not clear to me if it should be added or just changed.
> Adding this to __vlapic_accept_pic_intr() would be contrary to
> the purpose of the function afaict - fixed delivery mode is
> unrelated to delivering PIC interrupts.

So where would be a better place to add this kind of check?

Or is the PIT and HPET not "connected" to LINTIN0 of the lapic?

>> This code looks to state that:
>>
>> ...trying to set up timer (IRQ0) through the 8259A ...
>>
>> is expected to fail.  Is this by design?  (QEMU does allow this case.)
> But in the end I think you're barking up the wrong tree: All of this
> code would be of no interest at all if Linux didn't for some reason go
> into that fallback code. And with that fallback code existing mainly
> (if not exclusively) to deal with (half-)broken hardware/firmware, we
> should really make sure our emulation isn't broken (i.e. the fallback
> never being made use of). So finding the "TBD reason" is what is
> really needed.
>
> Jan
>

I am now sure that I found the base reason in my case.  See:

http://lists.xen.org/archives/html/xen-devel/2014-02/msg02408.html

For my proposed fix.

    -Don Slutz