locking issue (hardirq+softirq+user)

locking issue (hardirq+softirq+user)

[Jiri Slaby]

Hi!

an user still gets NMI watchdog warning, that the machine deadlocked.

The code is something like this:

DEFINE_SPINLOCK(lock);

isr() /* i.e. hardirq context */
{
spin_lock(&lock);
...
spin_unlock(&lock);
}

timer() /* i.e. softirq context */
{
unsigned int f;
spin_lock_irqsave(&lock, f) /* stack shows, that it locks here */
...
spin_unlock_irqrestore(&lock, f)
...
mod_timer();
}

tty_open_or_whatever() /* i.e. user context */
{
unsigned int f;
spin_lock_irqsave(&lock, f)
...
spin_unlock_irqrestore(&lock, f)
}

init()
{
mod_timer();
request_irq();
register_that_open_with_something();
}

What's the correct locking approach in this situation? Is that correct (I tried
to go through Rusty Russel's guide to locking, but I didn't get it in this
case)? There were many spin_lock recursions in the driver
(drivers/char/isicom.c), which I removed, but it still deadlocks on SMP.

thanks,
-- 
http://www.fi.muni.cz/~xslaby/            Jiri Slaby
faculty of informatics, masaryk university, brno, cz
e-mail: jirislaby gmail com, gpg pubkey fingerprint:
B674 9967 0407 CE62 ACC8  22A0 32CC 55C3 39D4 7A7E

Re: locking issue (hardirq+softirq+user)

[Arjan van de Ven]

On Wed, 2006-12-20 at 14:35 +0100, Jiri Slaby wrote:
> Hi!
> 
> an user still gets NMI watchdog warning, that the machine deadlocked.

have you tried enabling LOCKDEP ?


> isr() /* i.e. hardirq context */
> {
> spin_lock(&lock);
> ...
> spin_unlock(&lock);
> }

this is ok if you are 100% sure that this never gets called in any other
way

> 
> timer() /* i.e. softirq context */
> {
> unsigned int f;
> spin_lock_irqsave(&lock, f) /* stack shows, that it locks here */

this is a bug, the flags are an "unsigned long" not "unsigned int"!
It may do really bad stuff!

Greetings,
   Arjan van de Ven

-- 
if you want to mail me at work (you don't), use arjan (at) linux.intel.com
Test the interaction between Linux and your BIOS via http://www.linuxfirmwarekit.org

Re: locking issue (hardirq+softirq+user)

[Jiri Slaby]

Arjan van de Ven wrote:
> On Wed, 2006-12-20 at 14:35 +0100, Jiri Slaby wrote:
>> Hi!
>>
>> an user still gets NMI watchdog warning, that the machine deadlocked.
> 
> have you tried enabling LOCKDEP ?

No, only PROVE_LOCKING, I apply him to turn this on too.

>> isr() /* i.e. hardirq context */
>> {
>> spin_lock(&lock);
>> ...
>> spin_unlock(&lock);
>> }
> 
> this is ok if you are 100% sure that this never gets called in any other
> way

It holds. Only in request_irq isr function name occurs.

>> timer() /* i.e. softirq context */
>> {
>> unsigned int f;
>> spin_lock_irqsave(&lock, f) /* stack shows, that it locks here */
> 
> this is a bug, the flags are an "unsigned long" not "unsigned int"!
> It may do really bad stuff!

Aah, sorry, I misread the code, I saw flags, but it was port->flags;
board->flags, which is used in irq{save,restore} functions, is ulong.

thanks for the quick reply,
-- 
http://www.fi.muni.cz/~xslaby/            Jiri Slaby
faculty of informatics, masaryk university, brno, cz
e-mail: jirislaby gmail com, gpg pubkey fingerprint:
B674 9967 0407 CE62 ACC8  22A0 32CC 55C3 39D4 7A7E

Re: locking issue (hardirq+softirq+user)

[linux-os (Dick Johnson)]

On Wed, 20 Dec 2006, Jiri Slaby wrote:

> Hi!
>
> an user still gets NMI watchdog warning, that the machine deadlocked.
>
> The code is something like this:
>
> DEFINE_SPINLOCK(lock);
>
> isr() /* i.e. hardirq context */
> {
> spin_lock(&lock);
> ...
> spin_unlock(&lock);
> }
>
> timer() /* i.e. softirq context */
> {
> unsigned int f;
> spin_lock_irqsave(&lock, f) /* stack shows, that it locks here */
> ...
> spin_unlock_irqrestore(&lock, f)
> ...
> mod_timer();
> }
>
> tty_open_or_whatever() /* i.e. user context */
> {
> unsigned int f;
> spin_lock_irqsave(&lock, f)
> ...
> spin_unlock_irqrestore(&lock, f)
> }
>
> init()
> {
> mod_timer();
> request_irq();
> register_that_open_with_something();
> }
>
> What's the correct locking approach in this situation? Is that correct (I tried
> to go through Rusty Russel's guide to locking, but I didn't get it in this
> case)? There were many spin_lock recursions in the driver
> (drivers/char/isicom.c), which I removed, but it still deadlocks on SMP.
>
> thanks,
> -- 
> http://www.fi.muni.cz/~xslaby/            Jiri Slaby
> faculty of informatics, masaryk university, brno, cz
> e-mail: jirislaby gmail com, gpg pubkey fingerprint:
> B674 9967 0407 CE62 ACC8  22A0 32CC 55C3 39D4 7A7E

Well the locking is okay. The lock in the ISR, which doesn't require saving
and restoring flags, will prevent any other spin-lock from being entered
by another CPU when an interrupt is being handled. The spin-locks within
the timer will not clash as long as you are truly accessing the same lock 
variable.

The watchdog (NMI) hang detector runs when a CPU instruction-pointer seems
to be in the same place for too long. It could be that your ISR does not
return the correct 'done' flag or that you have some do-forever loop in your
code.

Note that if you start your timer from within the ISR, the timer (on another
CPU) may start instantly. This is usually not a problem, but you need
to make sure this doesn't create a wait-forever loop as well. Nevertheless
the basic spin-locking that you have shown is correct.

Cheers,
Dick Johnson
Penguin : Linux version 2.6.16.24 on an i686 machine (5592.72 BogoMips).
New book: http://www.AbominableFirebug.com/
_
\x1a\x04

****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.