threaded interrupt handlers: what if hardirq part not disables device interrupt?

threaded interrupt handlers: what if hardirq part not disables device interrupt?

[Remy Bohmer]

Hello Thomas, Peter,

While getting the new RT6-patch working properly on a Atmel AT91
board, I ran into the problem that the atmel-serial driver misses
characters since its interrupt hander runs in thread context and it
cannot keep up with the 1-byte RX 'FIFO' of the DBGU port.
It makes the terminal slow and broken.

To solve this issue, I thought to make proper use of the threaded
interrupt mechanism and make:
* 1 part run in hard-irq context, only handling the RX-fifo, push it
in a queue and
*  wake the irq-thread to process the data from the queue.

Digging through the interrupt code I read that the hard-irq part
should disable the device interrupt, while the irq-thread needs to
reenable it after it has handled it completely.
If I am going to use the queue idea as depicted above, the hard-irq
should keep on being enabled forever, and just wake up the irq-thread
once it has pushed data in the queue, on which the irq-thread could
process it on its own.
This seem to be against the principles as on which the current
interrupt handling code is based. It does not even work properly if I
do it this way.
Disabling the device interrupt in the hard-irq until the irq-thread
reenables it again, does not solve the problem since characters will
be missed.

As alternative solution I can do the following as well:
The driver now has a tasklet used for processing the queue. If I add
IRQF_NO_THREAD as irq-flag, the thread is not created and the handler
runs completely in hard-irq context. This seems to work okay, but it
does not feel as the right solution.

How is this supposed to work? Do you have better ideas how to solve
situations like this?

Kind regards,

Remy

Re: threaded interrupt handlers: what if hardirq part not disables device interrupt?

[Peter Zijlstra]

On Sun, 2011-07-31 at 16:15 +0200, Remy Bohmer wrote:
> As alternative solution I can do the following as well:
> The driver now has a tasklet used for processing the queue. If I add
> IRQF_NO_THREAD as irq-flag, the thread is not created and the handler
> runs completely in hard-irq context. This seems to work okay, but it
> does not feel as the right solution.
> 
> How is this supposed to work? Do you have better ideas how to solve
> situations like this? 

This sounds like the total trainwreck known as tty/console, I believe
jwessel and tglx had some ideas on that. Maybe Jason can share what he
knows, I'm not up to speed on that issue.

Re: threaded interrupt handlers: what if hardirq part not disables device interrupt?

[Jason Wessel]

On 08/01/2011 07:44 AM, Peter Zijlstra wrote:
> On Sun, 2011-07-31 at 16:15 +0200, Remy Bohmer wrote:
>> As alternative solution I can do the following as well:
>> The driver now has a tasklet used for processing the queue. If I add
>> IRQF_NO_THREAD as irq-flag, the thread is not created and the handler
>> runs completely in hard-irq context. This seems to work okay, but it
>> does not feel as the right solution.
>>
>> How is this supposed to work? Do you have better ideas how to solve
>> situations like this? 
> This sounds like the total trainwreck known as tty/console, I believe
> jwessel and tglx had some ideas on that. Maybe Jason can share what he
> knows, I'm not up to speed on that issue.
>
>

I talked with Thomas a bit about this last week.  While we did not talk specifically about implementation details we agree on the top level problem.  Talking to the serial hardware for any reason in an atomic context (as you often might do with printk) is simply forbidden, and yet we really want any kind critical messages, when taking an oops or using the kernel debugger to be able to be emitted from the serial port.  Design wise these goals couldn't be further apart.

The infrastructure we have in the kernel today for the serial console and tty plumbing simply does not meet both needs.  The discussion went on to conclude that for the normal mode of operation a serial HW driver should be structured to put everything into a queue for deferred work using a worker thread.  The hard IRQ context would make use of a kfifo if needed.   In terms of the oops / kernel debug path we already have some plumbing for the CONSOLE_POLL interface where we can directly read/write to the serial hardware and effectively don't care how much latency or time is lost in between.   The general idea would be to make this interface a required part of the change so we could have low latency operation as well as "don't care about latency" operation.

All that being said, I am not sure it actually answers the original problem of the atmel driver.  It seems to me that the driver needs to change just a bit with the same sort of principle in mind.  We have a kfifo implementation in the kernel that is generic and employed by many of the USB serial devices today.  It would seem that the logical approach to your immediate problem would be to use the kfifo and schedule the work without a tasklet and instead use the workqueue interface to either stick work on the default queue or make use of a separate queue specific to the atmel HW.  This approach should allow for the maximum preemption and in theory the lowest latency at the expense of some scheduling overhead as is to be expected for the price of using the RT kernel.

Jason.