[Qemu-devel] ARM64 Interrupt handling on QEMU

[Qemu-devel] ARM64 Interrupt handling on QEMU

[Brijen Raval]

I am booting up a custom kernel on QEMU ARM64, upon completion of its
initial boot up it looks like it enters the arch_idle() state

I enabled the -d int logging to understand what is going on, I see the
following repeated many times continuosly here after

Taking exception 5 [IRQ]
...from EL1 to EL1
...with ESR 0x15/0x56000000
...with ELR 0xffffffff0000349c
...to EL1 PC 0xffffffff00008280 PSTATE 0x3c5

Here's the dissassembly for the relevant piece of code:

 ffffffff00003498 <arch_idle>:
 arch_idle():
 ../../kernel/arch/arm64/arch.cpp:182
 ffffffff00003498:       d503207f        wfi
 ffffffff0000349c:       d65f03c0        ret

I am trying to understand what exceptions are occurring exactly when kernel
is idle (timer?). According to above ELR is pointing to arch_idle(), but I
believe "wfi" instruction would not be an IRQ but a sync abort which is
handle differently right?

Also from ESR, it looks like a SVC instruction but if I am not wrong for
IRQs ESRs are not updated (considered)

One more thing, is there a way in QEMU I could find out what exception 5 is
corresponding to?

Re: [Qemu-devel] ARM64 Interrupt handling on QEMU

[Peter Maydell]

On 15 March 2018 at 03:07, Brijen Raval <bjraval123@gmail.com> wrote:
> I am booting up a custom kernel on QEMU ARM64, upon completion of its
> initial boot up it looks like it enters the arch_idle() state
>
> I enabled the -d int logging to understand what is going on, I see the
> following repeated many times continuosly here after
>
> Taking exception 5 [IRQ]
> ...from EL1 to EL1
> ...with ESR 0x15/0x56000000
> ...with ELR 0xffffffff0000349c
> ...to EL1 PC 0xffffffff00008280 PSTATE 0x3c5
>
> Here's the dissassembly for the relevant piece of code:
>
>  ffffffff00003498 <arch_idle>:
>  arch_idle():
>  ../../kernel/arch/arm64/arch.cpp:182
>  ffffffff00003498:       d503207f        wfi
>  ffffffff0000349c:       d65f03c0        ret
>
> I am trying to understand what exceptions are occurring exactly when kernel
> is idle (timer?). According to above ELR is pointing to arch_idle(), but I
> believe "wfi" instruction would not be an IRQ but a sync abort which is
> handle differently right?

WFI is neither an IRQ nor an abort. It's just a hint to the CPU
that it doesn't need to execute any more instructions until the
next interrupt occurs. (It's a valid implementation for it to just
be a NOP.) QEMU does implement WFI to be "don't execute more insns
until the next interrupt", which is why you're seeing the ELR for
the interrupt generally being the ret insn: if the guest is mostly
idle and its idle loop includes a WFI then almost all the time an
incoming interrupt will find that the CPU was in the WFI insn.

Correctly written software will not ever issue a WFI unless it
has some mechanism for being woken up from it, which is typically
an outstanding interrupt, perhaps timer or an interrupt for
completed disk or network operation. For an SMP kernel it could also be
some other CPU sending this CPU an inter-processor-interrupt to
wake us up). "idle" for an OS just means "nothing to do right now".

> Also from ESR, it looks like a SVC instruction but if I am not wrong for
> IRQs ESRs are not updated (considered)

QEMU's logging prints the ESR value for all exceptions, even those
where architecturally the ESR is not updated. In those cases the
printed value can be ignored.

> One more thing, is there a way in QEMU I could find out what exception 5 is
> corresponding to?

The logging tells you:
> Taking exception 5 [IRQ]

Exception 5 is IRQ. (These numbers are all internal to QEMU, and
don't have any architectural or guest-visible relevance. They're
the EXCP_* constants defined at the top of target/arm/cpu.h.)

thanks
-- PMM

Re: [Qemu-devel] ARM64 Interrupt handling on QEMU

[Brijen Raval]

On Thu, Mar 15, 2018 at 2:59 AM Peter Maydell <peter.maydell@linaro.org>
wrote:

> On 15 March 2018 at 03:07, Brijen Raval <bjraval123@gmail.com> wrote:
> > I am booting up a custom kernel on QEMU ARM64, upon completion of its
> > initial boot up it looks like it enters the arch_idle() state
> >
> > I enabled the -d int logging to understand what is going on, I see the
> > following repeated many times continuosly here after
> >
> > Taking exception 5 [IRQ]
> > ...from EL1 to EL1
> > ...with ESR 0x15/0x56000000
> > ...with ELR 0xffffffff0000349c
> > ...to EL1 PC 0xffffffff00008280 PSTATE 0x3c5
> >
> > Here's the dissassembly for the relevant piece of code:
> >
> >  ffffffff00003498 <arch_idle>:
> >  arch_idle():
> >  ../../kernel/arch/arm64/arch.cpp:182
> >  ffffffff00003498:       d503207f        wfi
> >  ffffffff0000349c:       d65f03c0        ret
> >
> > I am trying to understand what exceptions are occurring exactly when
> kernel
> > is idle (timer?). According to above ELR is pointing to arch_idle(), but
> I
> > believe "wfi" instruction would not be an IRQ but a sync abort which is
> > handle differently right?
>
> WFI is neither an IRQ nor an abort. It's just a hint to the CPU
> that it doesn't need to execute any more instructions until the
> next interrupt occurs. (It's a valid implementation for it to just
> be a NOP.) QEMU does implement WFI to be "don't execute more insns
> until the next interrupt", which is why you're seeing the ELR for
> the interrupt generally being the ret insn: if the guest is mostly
> idle and its idle loop includes a WFI then almost all the time an
> incoming interrupt will find that the CPU was in the WFI insn.
>
> Correctly written software will not ever issue a WFI unless it
> has some mechanism for being woken up from it, which is typically
> an outstanding interrupt, perhaps timer or an interrupt for
> completed disk or network operation. For an SMP kernel it could also be
> some other CPU sending this CPU an inter-processor-interrupt to
> wake us up). "idle" for an OS just means "nothing to do right now".
>

OK, that makes sense


>
> > Also from ESR, it looks like a SVC instruction but if I am not wrong for
> > IRQs ESRs are not updated (considered)
>
> QEMU's logging prints the ESR value for all exceptions, even those
> where architecturally the ESR is not updated. In those cases the
> printed value can be ignored.
>

Thought so, thanks for the clarification

>
> > One more thing, is there a way in QEMU I could find out what exception 5
> is
> > corresponding to?
>
> The logging tells you:
> > Taking exception 5 [IRQ]
>
> Exception 5 is IRQ. (These numbers are all internal to QEMU, and
> don't have any architectural or guest-visible relevance. They're
> the EXCP_* constants defined at the top of target/arm/cpu.h.)
>

Yup I checked out the QEMU source and confirmed above. So is there any way
to find out what is the IRQ for?


>
> thanks
> -- PMM
>

Re: [Qemu-devel] ARM64 Interrupt handling on QEMU

[Peter Maydell]

On 15 March 2018 at 20:24, Brijen Raval <bjraval123@gmail.com> wrote:
> On Thu, Mar 15, 2018 at 2:59 AM Peter Maydell <peter.maydell@linaro.org>
> wrote:
>> Exception 5 is IRQ. (These numbers are all internal to QEMU, and
>> don't have any architectural or guest-visible relevance. They're
>> the EXCP_* constants defined at the top of target/arm/cpu.h.)
>
>
> Yup I checked out the QEMU source and confirmed above. So is there any way
> to find out what is the IRQ for?

You would need to look at what the state of the interrupt controller
is. You can turn on all the tracepoints in the GIC with -d trace:gic*
(though to understand what it's saying you may need to have
some familiarity with the GIC spec and/or look at the code).

thanks
-- PMM