Re: Interrupt injection with ISR set on Intel hardware

["Roger Pau Monné" <roger.pau@citrix.com>]

On Thu, Dec 13, 2018 at 01:28:23AM +0000, Tian, Kevin wrote:
> > From: Roger Pau Monné [mailto:roger.pau@citrix.com]
> > Sent: Wednesday, December 12, 2018 8:18 PM
> > 
> > On Wed, Dec 12, 2018 at 11:48:52AM +0000, Tian, Kevin wrote:
> > > > From: Roger Pau Monné [mailto:roger.pau@citrix.com]
> > > > Sent: Wednesday, December 12, 2018 7:25 PM
> > > >
> > > > On Wed, Dec 12, 2018 at 10:36:44AM +0000, Tian, Kevin wrote:
> > > > > > From: Roger Pau Monné [mailto:roger.pau@citrix.com]
> > > > > > Sent: Monday, October 15, 2018 6:30 PM
> > > > > > (XEN)   [22642] POWER    TYPE 4
> > > > > > (XEN)   [22643] IDLE     PPR 0x00000020
> > > > > > (XEN)                    IRR
> > > > > >
> > > >
> > 00000000000000000000000000000000000000000000000000000000000000
> > > > > > 00
> > > > > > (XEN)                    ISR
> > > > > >
> > > >
> > 00000000020000000000000000000000000000000000000000000000000000
> > > > > > 00
> > > > > > (XEN)   [22644] WAKE     PPR 0x00000020
> > > > > > (XEN)                    IRR
> > > > > >
> > > >
> > 00000000020000000000000000000000000000000000000000000000000000
> > > > > > 00
> > > > > > (XEN)                    ISR
> > > > > >
> > > >
> > 00000000020000000000000000000000000000000000000000000000000000
> > > > > > 00
> > > > >
> > > > > looks pending IRR (0x21) doesn't always trigger a spurious interrupt?
> > > >
> > > > Yes, that's correct. Having a pending IRR and going idle doesn't
> > > > always trigger the spurious interrupt re-injection.
> > > >
> > > > > is it a fixed pattern after how many rounds of Cstate enter/exit with
> > > > > pending IRR(0x21) then you see assertion happened (in this example
> > > > > it happens at 3rd time)?
> > > >
> > > > It's not a fixed pattern, here's another trace with IRR(0x21) being
> > > > pending just once during the Cstate transitions:
> > >
> > > did you observe a case where such asset may occur when IRR(0x21)
> > > is cleared but ISR (0x21) is set?
> > 
> > No, I've always seen both ISR and IRR set when the interrupt injection
> > happens. This of course doesn't mean it's not possible, but I have not
> > seen any trace with ISR(0x21) set and IRR(0x21) clear.
> > 
> 
> sorry but let me double confirm. You always see ISR[21]/IRR[21] being
> set "before and after entering C3" to hit the problem, right?

Yes, that's correct.

> When 
> interrupt injection happens later, ISR[21] is set but IRR[21] is cleared (as 
> expected for normal interrupt delivery process).
> 
> btw I checked your original mail:
> 
> (XEN)    [<ffff82d0802e2403>] mwait-idle.c#mwait_idle+0x2a5/0x381
> xen/arch/x86/cpu/mwait-idle.c:802
> 
>    788		if (cpu_is_haltable(cpu))
>    789			mwait_idle_with_hints(eax, MWAIT_ECX_INTERRUPT_BREAK);
>    790
>    791		after = cpuidle_get_tick();
>    792
>    793		cstate_restore_tsc();
>    794		trace_exit_reason(irq_traced);
>    795		TRACE_6D(TRC_PM_IDLE_EXIT, cx->type, after,
>    796			irq_traced[0], irq_traced[1], irq_traced[2], irq_traced[3]);
>    797
>    798		/* Now back in C0. */
>    799		update_idle_stats(power, cx, before, after);
>    800		local_irq_enable();
>    801
> -> 802		if (!(lapic_timer_reliable_states & (1 << cstate)))
>    803			lapic_timer_on();
>    804
>    805		sched_tick_resume();
>    806		cpufreq_dbs_timer_resume();
> 
> Looks above code is different from staging:

The code matches staging at the point where I posted the original bug
report, 2 months ago.

> acpi_processor_idle:
> 	acpi_idle_do_entry:
> 		acpi_processor_ffh_cstate_enter:
> 			mwait_idle_with_hints

There's another caller of mwait_idle_with_hints, the mwait_idle
function. This gets setup by mwait_idle_init.

> there is no mwait_idle alone.

There's a mwait_idle in current staging code, check:

xen/arch/x86/cpu/mwait-idle.c:718

http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/x86/cpu/mwait-idle.c;h=f89c52f2565e8deb7d4aec309124fe6fbd57b27d;hb=refs/heads/staging#l718

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel