Question about edge-triggered interrupt

Question about edge-triggered interrupt

[LIU Zhiwei]

[-- Attachment #1: Type: text/plain, Size: 935 bytes --]

Hi folks,

Currently, I am writing an interrupt controller (CLIC) for RISC-V. I 
can't find a good way to process edge-triggered interrupt.

According to edge-triggered definition, if I select an edge-triggered  
interrupt to serve , it will clean its pending status. However after 
serving the interrupt,  there is no chance to select other pending 
interrupts.

I have two methods.

 1. One is to add a timer for interrupt controller, so that every
    pending interrupt can be served at some point.
 2. The other is that  always pull a pending interrupt to serve at the
    interrupt return instruction.

But both are not so good. The first one we can not server the interrupts 
immediately. The second one we have to add some
code when executing the guest code.

I want to know if there is a better way to handle edge-triggered 
interrupt on QEMU.

Thanks very much for your reading and look forward to your response.

Zhiwei



[-- Attachment #2: Type: text/html, Size: 1271 bytes --]

Re: Question about edge-triggered interrupt

[Peter Maydell]

On Thu, 11 Mar 2021 at 02:59, LIU Zhiwei <zhiwei_liu@c-sky.com> wrote:
> Currently, I am writing an interrupt controller (CLIC) for RISC-V.  I can't find a good way to process edge-triggered interrupt.
>
> According to edge-triggered definition, if I select an edge-triggered  interrupt to serve , it will clean its pending status. However after serving the interrupt,  there is no chance to select other pending interrupts.
>
> I have two methods.
>
> One is to add a timer for interrupt controller, so that every pending interrupt can be served at some point.
> The other is that  always pull a pending interrupt to serve at the interrupt return instruction.

You should do what the hardware you're modelling does. I'm
pretty sure it won't be using a timer, at any rate. Whether
it does something on interrupt-return insns will depend on
the architecture and how tightly the interrupt controller
is integrated with the CPU.

In general edge-triggered interrupts for an interrupt
controller just mean "when it sees a 0->1 transition
on its input line it does something, eg mark the interrupt
pending", and also usually "when the guest OS acknowledges
the interrupt (eg via a register write to the interrupt
controller), mark the interrupt as no longer pending", and
a "keep telling the OS that there is an available interrupt
of some kind until all the interrupts are no longer pending".

The details of how this works depend entirely on the
interrupt controller -- hopefully it has a specification
document which will tell you the actual behaviour.

thanks
-- PMM

Re: Question about edge-triggered interrupt

[LIU Zhiwei]

[-- Attachment #1: Type: text/plain, Size: 2364 bytes --]



On 2021/3/11 17:33, Peter Maydell wrote:
> On Thu, 11 Mar 2021 at 02:59, LIU Zhiwei <zhiwei_liu@c-sky.com> wrote:
>> Currently, I am writing an interrupt controller (CLIC) for RISC-V.  I can't find a good way to process edge-triggered interrupt.
>>
>> According to edge-triggered definition, if I select an edge-triggered  interrupt to serve , it will clean its pending status. However after serving the interrupt,  there is no chance to select other pending interrupts.
>>
>> I have two methods.
>>
>> One is to add a timer for interrupt controller, so that every pending interrupt can be served at some point.
>> The other is that  always pull a pending interrupt to serve at the interrupt return instruction.
> You should do what the hardware you're modelling does. I'm
> pretty sure it won't be using a timer, at any rate. Whether
> it does something on interrupt-return insns will depend on
> the architecture and how tightly the interrupt controller
> is integrated with the CPU.
>
> In general edge-triggered interrupts for an interrupt
> controller just mean "when it sees a 0->1 transition
> on its input line it does something, eg mark the interrupt
> pending", and also usually "when the guest OS acknowledges
> the interrupt (eg via a register write to the interrupt
> controller), mark the interrupt as no longer pending", and
> a "keep telling the OS that there is an available interrupt
> of some kind until all the interrupts are no longer pending".
Thanks very much. It solves the most problem. However, I still have a 
little one.


 From the specification, I find that software will not clean the pending 
bit on interrupt controller via a register write.

"When a vectored interrupt is selected and serviced, the hardware will automatically clear the

corresponding pending bit in edge-triggered mode. In this case, software does not need to clear

pending bit at all in the service routine."

Hardware can always select a pending interrupt as it is cycle-driven. 
But QEMU is event-driven.
I don't know if there is a chance to select other pending interrupts 
after serving the selected interrupt.

Look forward to your reply. Thanks very much.

Zhiwei
> The details of how this works depend entirely on the
> interrupt controller -- hopefully it has a specification
> document which will tell you the actual behaviour.
>
> thanks
> -- PMM


[-- Attachment #2: Type: text/html, Size: 3218 bytes --]

Re: Question about edge-triggered interrupt

[Peter Maydell]

On Thu, 11 Mar 2021 at 12:59, LIU Zhiwei <zhiwei_liu@c-sky.com> wrote:
> From the specification, I find that software will not clean the pending bit on interrupt controller via a register write.
>
> "When a vectored interrupt is selected and serviced, the hardware will automatically clear the
>
> corresponding pending bit in edge-triggered mode. In this case, software does not need to clear
>
> pending bit at all in the service routine."
>
> Hardware can always select a pending interrupt as it is cycle-driven. But QEMU is event-driven.
> I don't know if there is a chance to select other pending interrupts after serving the selected interrupt.

Something must change that the hardware (and thus the model) can use to
determine that it needs to do something else. The interrupt controller
must be able to tell when the interrupt is "selected and serviced"
somehow.

thanks
-- PMM

Re: Question about edge-triggered interrupt

[LIU Zhiwei]

On 2021/3/11 21:17, Peter Maydell wrote:
> On Thu, 11 Mar 2021 at 12:59, LIU Zhiwei <zhiwei_liu@c-sky.com> wrote:
>>  From the specification, I find that software will not clean the pending bit on interrupt controller via a register write.
>>
>> "When a vectored interrupt is selected and serviced, the hardware will automatically clear the
>>
>> corresponding pending bit in edge-triggered mode. In this case, software does not need to clear
>>
>> pending bit at all in the service routine."
>>
>> Hardware can always select a pending interrupt as it is cycle-driven. But QEMU is event-driven.
>> I don't know if there is a chance to select other pending interrupts after serving the selected interrupt.
> Something must change that the hardware (and thus the model) can use to
> determine that it needs to do something else. The interrupt controller
> must be able to tell when the interrupt is "selected and serviced"
> somehow.
That's a case I can't understand.

1.  An device causes an edge-triggered interrupt A.
2.  The interrupt controller sample the interrupt A, and setting pending 
bit for A.
3.  The interrupt controller select the interrupt  A to CPU and clean 
the pending bit for A(according to the specification).
4.  CPU start to execute the A's service routine.
5.  Another device causes interrupt B.
6.  The interrupt controller sample the interrupt B, and setting pending 
bit for B.
7.  As B's priority is lower than A, it can't preempt  A's service routine.
8.  A's service routine return.
9. No other interrupt comes after B.

After the sequence, B is pending in interrupt controller, and has no 
chance to be selected by interrupt controller.
A's service routine will neither write interrupt controller's register 
nor device's register.

In my RTOS case, the interrupt A here is really a software interrupt. 
Just for clarity here.

Thanks for sharing your ideas.

Zhiwei
>
> thanks
> -- PMM

Re: Question about edge-triggered interrupt

[Peter Maydell]

On Thu, 11 Mar 2021 at 16:01, LIU Zhiwei <zhiwei_liu@c-sky.com> wrote:
> That's a case I can't understand.
>
> 1.  An device causes an edge-triggered interrupt A.
> 2.  The interrupt controller sample the interrupt A, and setting pending
> bit for A.
> 3.  The interrupt controller select the interrupt  A to CPU and clean
> the pending bit for A(according to the specification).
> 4.  CPU start to execute the A's service routine.
> 5.  Another device causes interrupt B.
> 6.  The interrupt controller sample the interrupt B, and setting pending
> bit for B.
> 7.  As B's priority is lower than A, it can't preempt  A's service routine.
> 8.  A's service routine return.
> 9. No other interrupt comes after B.
>
> After the sequence, B is pending in interrupt controller, and has no
> chance to be selected by interrupt controller.
> A's service routine will neither write interrupt controller's register
> nor device's register.

I'm pretty sure that there is some interaction between the CPU and the
interrupt controller on a return-from-interrupt that you are
missing. There are several ways to do it in principle; you need
a risc-v expert to answer your question more specifically.

thanks
-- PMM

Re: Question about edge-triggered interrupt

[LIU Zhiwei]

On 2021/3/12 0:07, Peter Maydell wrote:
> On Thu, 11 Mar 2021 at 16:01, LIU Zhiwei <zhiwei_liu@c-sky.com> wrote:
>> That's a case I can't understand.
>>
>> 1.  An device causes an edge-triggered interrupt A.
>> 2.  The interrupt controller sample the interrupt A, and setting pending
>> bit for A.
>> 3.  The interrupt controller select the interrupt  A to CPU and clean
>> the pending bit for A(according to the specification).
>> 4.  CPU start to execute the A's service routine.
>> 5.  Another device causes interrupt B.
>> 6.  The interrupt controller sample the interrupt B, and setting pending
>> bit for B.
>> 7.  As B's priority is lower than A, it can't preempt  A's service routine.
>> 8.  A's service routine return.
>> 9. No other interrupt comes after B.
>>
>> After the sequence, B is pending in interrupt controller, and has no
>> chance to be selected by interrupt controller.
>> A's service routine will neither write interrupt controller's register
>> nor device's register.
> I'm pretty sure that there is some interaction between the CPU and the
> interrupt controller on a return-from-interrupt that you are
> missing.
I have asked my CPU design workmates. They said, "you have to pull a 
pending interrupt at the interrupt return instruction".

But I don't want to do this way, because it influences the interrupt 
return instruction's behavior. The specification doesn't
point that explicitly. It's just our CPU implementation.

If I lock the iothread in interrupt return instruction helper function, 
and pull a pending interrupt. It really works, but I don't think
it will be appropriate for other CPU implementation.
> There are several ways to do it in principle;
I am curious about the ways. If you like, a simple list please.
> you need
> a risc-v expert to answer your question more specifically.
Thanks for your advice.

Zhiwei
>
> thanks
> -- PMM

Re: Question about edge-triggered interrupt

[Peter Maydell]

On Thu, 11 Mar 2021 at 16:40, LIU Zhiwei <zhiwei_liu@c-sky.com> wrote:
>
>
>
> On 2021/3/12 0:07, Peter Maydell wrote:
> > On Thu, 11 Mar 2021 at 16:01, LIU Zhiwei <zhiwei_liu@c-sky.com> wrote:
> > I'm pretty sure that there is some interaction between the CPU and the
> > interrupt controller on a return-from-interrupt that you are
> > missing.
> I have asked my CPU design workmates. They said, "you have to pull a
> pending interrupt at the interrupt return instruction".
>
> But I don't want to do this way, because it influences the interrupt
> return instruction's behavior. The specification doesn't
> point that explicitly. It's just our CPU implementation.

If the hardware does it like that, your model of the hardware
should do it like that. Don't invent QEMU-specific weirdness
if you can avoid it.

More generally, if the specification says that something has to
happen when the CPU does an interrupt-return instruction, then the
obvious implementation is to put suitable code in the helper function
for the interrupt return instruction.

> If I lock the iothread in interrupt return instruction helper function,
> and pull a pending interrupt. It really works, but I don't think
> it will be appropriate for other CPU implementation.
> > There are several ways to do it in principle;
> I am curious about the ways. If you like, a simple list please.

So for instance:
You can have an interrupt controller which holds a signal line
to the CPU high while it has any pending interrupt. When the
CPU executes return-from-interrupt this involves unblocking interrupts,
and so if there's still another pending interrupt the CPU will just
immediately take it, because the signal line is still high. Arm FIQ/IRQ
with a GICv2 works like this. (It's also how QEMU handles the GICv3,
which is not exactly the same as the typical h/w implementation.)

You can have a more complicated scheme where the interrupt
controller and the CPU are very tightly integrated, and so
'return from interrupt' executed by the CPU means "look at whatever
is next up as the next highest priority interrupt and do that,
possibly avoiding a bunch of stack frame unstack/stack actions".
The Arm M-profile NVIC is like that.

thanks
-- PMM

Re: Question about edge-triggered interrupt

[LIU Zhiwei]

On 2021/3/12 0:50, Peter Maydell wrote:
> On Thu, 11 Mar 2021 at 16:40, LIU Zhiwei <zhiwei_liu@c-sky.com> wrote:
>>
>>
>> On 2021/3/12 0:07, Peter Maydell wrote:
>>> On Thu, 11 Mar 2021 at 16:01, LIU Zhiwei <zhiwei_liu@c-sky.com> wrote:
>>> I'm pretty sure that there is some interaction between the CPU and the
>>> interrupt controller on a return-from-interrupt that you are
>>> missing.
>> I have asked my CPU design workmates. They said, "you have to pull a
>> pending interrupt at the interrupt return instruction".
>>
>> But I don't want to do this way, because it influences the interrupt
>> return instruction's behavior. The specification doesn't
>> point that explicitly. It's just our CPU implementation.
> If the hardware does it like that, your model of the hardware
> should do it like that. Don't invent QEMU-specific weirdness
> if you can avoid it.
>
> More generally, if the specification says that something has to
> happen when the CPU does an interrupt-return instruction, then the
> obvious implementation is to put suitable code in the helper function
> for the interrupt return instruction.
Good principle. Get it, thanks.
>> If I lock the iothread in interrupt return instruction helper function,
>> and pull a pending interrupt. It really works, but I don't think
>> it will be appropriate for other CPU implementation.
>>> There are several ways to do it in principle;
>> I am curious about the ways. If you like, a simple list please.
> So for instance:
> You can have an interrupt controller which holds a signal line
> to the CPU high while it has any pending interrupt. When the
> CPU executes return-from-interrupt this involves unblocking interrupts,
> and so if there's still another pending interrupt the CPU will just
> immediately take it, because the signal line is still high. Arm FIQ/IRQ
> with a GICv2 works like this. (It's also how QEMU handles the GICv3,
> which is not exactly the same as the typical h/w implementation.)
>
> You can have a more complicated scheme where the interrupt
> controller and the CPU are very tightly integrated, and so
> 'return from interrupt' executed by the CPU means "look at whatever
> is next up as the next highest priority interrupt and do that,
> possibly avoiding a bunch of stack frame unstack/stack actions".
> The Arm M-profile NVIC is like that.
Both work for me.  I think the CLIC is more similar to NVIC and the 
solution in GICv2
is more general. I will firstly add an implementation according to the 
NVIC solution.

Thanks very much. By the way, in my opinion, the signal line in GICv2 
solution is
something like "QEMU-specific weirdness" 😁.

Zhiwei
>
> thanks
> -- PMM

Re: Question about edge-triggered interrupt

[Peter Maydell]

On Fri, 12 Mar 2021 at 09:20, LIU Zhiwei <zhiwei_liu@c-sky.com> wrote:
>
> Thanks very much. By the way, in my opinion, the signal line in GICv2
> solution is
> something like "QEMU-specific weirdness" .

No, for the GICv2 that really is how the hardware works -- the interrupt
controller talks to the CPU via the classic IRQ and FIQ lines, and the
CPU itself has no particular access to what the interrupt controller
itself is doing. For GICv3 you have more of a point...

thanks
-- PMM

Re: Question about edge-triggered interrupt

[Alistair Francis]

On Thu, Mar 11, 2021 at 11:21 AM LIU Zhiwei <zhiwei_liu@c-sky.com> wrote:
>
>
>
> On 2021/3/11 21:17, Peter Maydell wrote:
> > On Thu, 11 Mar 2021 at 12:59, LIU Zhiwei <zhiwei_liu@c-sky.com> wrote:
> >>  From the specification, I find that software will not clean the pending bit on interrupt controller via a register write.
> >>
> >> "When a vectored interrupt is selected and serviced, the hardware will automatically clear the
> >>
> >> corresponding pending bit in edge-triggered mode. In this case, software does not need to clear
> >>
> >> pending bit at all in the service routine."
> >>
> >> Hardware can always select a pending interrupt as it is cycle-driven. But QEMU is event-driven.
> >> I don't know if there is a chance to select other pending interrupts after serving the selected interrupt.
> > Something must change that the hardware (and thus the model) can use to
> > determine that it needs to do something else. The interrupt controller
> > must be able to tell when the interrupt is "selected and serviced"
> > somehow.
> That's a case I can't understand.
>
> 1.  An device causes an edge-triggered interrupt A.
> 2.  The interrupt controller sample the interrupt A, and setting pending
> bit for A.
> 3.  The interrupt controller select the interrupt  A to CPU and clean
> the pending bit for A(according to the specification).

I'm not a CLIC expert but from what I can tell I think the interrupt
is only cleared after being read.

The key lines in the spec are:

"When a vectored interrupt is selected and serviced, the hardware will
automatically clear the corresponding pending bit in edge-triggered
mode."

and

"The xnxti CSR can be used by software to service the next horizontal
interrupt..."

The C-ABI code (https://github.com/riscv/riscv-fast-interrupt/blob/master/clic.adoc#81-c-abi-trampoline-code)
seems to say something similar


    csrrsi a0, mnxti, MIE   # Get highest current interrupt and enable
interrupts.
                            # Will return original interrupt if no
others appear. (6)

Does that sound right?

Alistair


> 4.  CPU start to execute the A's service routine.
> 5.  Another device causes interrupt B.
> 6.  The interrupt controller sample the interrupt B, and setting pending
> bit for B.
> 7.  As B's priority is lower than A, it can't preempt  A's service routine.
> 8.  A's service routine return.
> 9. No other interrupt comes after B.
>
> After the sequence, B is pending in interrupt controller, and has no
> chance to be selected by interrupt controller.
> A's service routine will neither write interrupt controller's register
> nor device's register.
>
> In my RTOS case, the interrupt A here is really a software interrupt.
> Just for clarity here.
>
> Thanks for sharing your ideas.
>
> Zhiwei
> >
> > thanks
> > -- PMM
>
>

Re: Question about edge-triggered interrupt

[LIU Zhiwei]

On 2021/3/12 4:45, Alistair Francis wrote:
> On Thu, Mar 11, 2021 at 11:21 AM LIU Zhiwei<zhiwei_liu@c-sky.com>  wrote:
>> On 2021/3/11 21:17, Peter Maydell wrote:
>>> On Thu, 11 Mar 2021 at 12:59, LIU Zhiwei<zhiwei_liu@c-sky.com>  wrote:
>>>>   From the specification, I find that software will not clean the pending bit on interrupt controller via a register write.
>>>>
>>>> "When a vectored interrupt is selected and serviced, the hardware will automatically clear the
>>>>
>>>> corresponding pending bit in edge-triggered mode. In this case, software does not need to clear
>>>>
>>>> pending bit at all in the service routine."
>>>>
>>>> Hardware can always select a pending interrupt as it is cycle-driven. But QEMU is event-driven.
>>>> I don't know if there is a chance to select other pending interrupts after serving the selected interrupt.
>>> Something must change that the hardware (and thus the model) can use to
>>> determine that it needs to do something else. The interrupt controller
>>> must be able to tell when the interrupt is "selected and serviced"
>>> somehow.
>> That's a case I can't understand.
>>
>> 1.  An device causes an edge-triggered interrupt A.
>> 2.  The interrupt controller sample the interrupt A, and setting pending
>> bit for A.
>> 3.  The interrupt controller select the interrupt  A to CPU and clean
>> the pending bit for A(according to the specification).

> I'm not a CLIC expert but from what I can tell I think the interrupt
> is only cleared after being read.
>
> The key lines in the spec are:
>
> "When a vectored interrupt is selected and serviced, the hardware will
> automatically clear the corresponding pending bit in edge-triggered
> mode."
>
> and
>
> "The xnxti CSR can be used by software to service the next horizontal
> interrupt..."
>
> The C-ABI code (https://github.com/riscv/riscv-fast-interrupt/blob/master/clic.adoc#81-c-abi-trampoline-code)
> seems to say something similar
>
>
>      csrrsi a0, mnxti, MIE   # Get highest current interrupt and enable
> interrupts.
>                              # Will return original interrupt if no
> others appear. (6)
Hi Alistair,

Thanks for your reply.

It is right if the interrupt routine really takes the C-ABI code. 
However, there may
be many other implementations for interrupt routine.

As the pending bit has been automatically cleaned by hardware, it will 
be unnecessary to
read xnxti in interrupt routine or access other registers to clean it, 
e.g. my RTOS case
never reads xnxti.

Zhiwei
> Does that sound right?
>
> Alistair
>
>
>> 4.  CPU start to execute the A's service routine.
>> 5.  Another device causes interrupt B.
>> 6.  The interrupt controller sample the interrupt B, and setting pending
>> bit for B.
>> 7.  As B's priority is lower than A, it can't preempt  A's service routine.
>> 8.  A's service routine return.
>> 9. No other interrupt comes after B.
>>
>> After the sequence, B is pending in interrupt controller, and has no
>> chance to be selected by interrupt controller.
>> A's service routine will neither write interrupt controller's register
>> nor device's register.
>>
>> In my RTOS case, the interrupt A here is really a software interrupt.
>> Just for clarity here.
>>
>> Thanks for sharing your ideas.
>>
>> Zhiwei
>>> thanks
>>> -- PMM