newbie questions about while (1) in kernel mode and spinlocks

newbie questions about while (1) in kernel mode and spinlocks

[Sorin Manolache]

Dear list,

I am in the process of learning how to write linux device drivers.

I have a 2.6.16.5 kernel running on a monoprocessor machine.
#CONFIG_SMP is not set
CONFIG_DEBUG_SPINLOCK=y.
CONFIG_PREEMPT=y
CONFIG_PREEMPT_BKL=y

First question:

I wrote

while (1)
    ;

in the read function of a test device driver. I expect the calling
process to freeze, and then a timer interrupt to preempt the kernel
and to schedule another process. This does not happen, the whole
system freezes. I see no effect from pressing keys or moving the
mouse. Why? The hardware interrupts are not disabled, are they? Why do
the interrupt handlers not get executed?

Second question:

I wrote

spin_lock(&lck);
down(&sem); /* I know that one shouldn't sleep when holding a lock */
                    /* but I want to understand why */
spin_unlock(&lck);

in the read function and

up(&sem)

in the write function. The semaphore is initially locked, so the first
process invoking down will sleep.

I invoke

cat /dev/test

and the process sleeps on the semaphore. Then I invoke

echo 1 > /dev/test

and I wake up the "cat" process.

Then I intend to invoke _two_ cat processes. I expect the first one to
sleep on the semaphore and the second on to spin at the spin_lock.
Then I expect to wake up the first process by invoking an echo, the
first process to release the lock and the second process to sleep on
the semaphore. What I get is that the system freezes as soon as I
invoke the second "cat" process. Again, no effect from key presses or
mouse movements. Why? Shouldn't the timer interrupt preempt the second
"cat" process that spins on the spinlock and give control to something
else, for example to the console where I could wake up the first "cat"
process? Why do I not see any effect from mouse movements? Hardware
interrupts are not disabled, are they?

Third question:

The Linux Device Drivers book says that a spin_lock should not be
shared between a process and an interrupt handler. The explanation is
that the process may hold the lock, an interrupt occurs, the interrupt
handler spins on the lock held by the process and the system freezes.
Why should it freeze? Isn't it possible for the interrupt handler to
re-enable interrupts as its first thing, then to spin at the lock, the
timer interrupt to preempt the interrupt handler and to relinquish
control to the process which in turn will finish its critical section
and release the lock, making way for the interrupt handler to
continue.

Thank you very much for clarifying these issues.

Regards,
Sorin

Re: newbie questions about while (1) in kernel mode and spinlocks

[Paolo Ornati]

On Thu, 21 Dec 2006 10:41:44 +0100
"Sorin Manolache" <sorinm@gmail.com> wrote:

> The Linux Device Drivers book says that a spin_lock should not be
> shared between a process and an interrupt handler. The explanation is
> that the process may hold the lock, an interrupt occurs, the interrupt
> handler spins on the lock held by the process and the system freezes.
> Why should it freeze? Isn't it possible for the interrupt handler to
> re-enable interrupts as its first thing, then to spin at the lock, the
> timer interrupt to preempt the interrupt handler and to relinquish
> control to the process which in turn will finish its critical section
> and release the lock, making way for the interrupt handler to
> continue.

Iterrupt handlers are executend in the process context (on top of the
process that they interrupted).

So, if you have a proccess A that does:

	Usual Kernel Code		Interrupt Handler

	...
	spin_lock(my_lock);
	...
		-------interrupt----->	...
					spin_lock(my_lock); // deadlock!
					...
		<------ back --------- 
	----
	spin_unlock(my_lock);


See?

If the interrupt comes in when process A is running and holding the
lock PREEMPTION can't do anything.

-- 
	Paolo Ornati
	Linux 2.6.20-rc1-g99f5e971 on x86_64

Re: newbie questions about while (1) in kernel mode and spinlocks

[Paolo Ornati]

On Thu, 21 Dec 2006 10:41:44 +0100
"Sorin Manolache" <sorinm@gmail.com> wrote:

> spin_lock(&lck);
> down(&sem); /* I know that one shouldn't sleep when holding a lock */
>                     /* but I want to understand why */

I suppose because the lock is held for an indefinite amount of time and
any other process that try to get that lock will "spin" and burn CPU
without doing anything useful (locking the process in kernel mode and
preventing the execution of other processes on that CPU if there
isn't any type of PREEMPTION).

:)

spin_lock is a "while(1) {...}" thing...

-- 
	Paolo Ornati
	Linux 2.6.20-rc1-g99f5e971 on x86_64

Re: newbie questions about while (1) in kernel mode and spinlocks

[Paolo Ornati]

On Thu, 21 Dec 2006 10:41:44 +0100
"Sorin Manolache" <sorinm@gmail.com> wrote:

> while (1)
>     ;
> 
> in the read function of a test device driver. I expect the calling
> process to freeze, and then a timer interrupt to preempt the kernel
> and to schedule another process. This does not happen, the whole
> system freezes. I see no effect from pressing keys or moving the
> mouse. Why? The hardware interrupts are not disabled, are they? Why do
> the interrupt handlers not get executed?

Here I'm not sure. I think that interrupts are enabled and processed
correctly BUT the process cannot be Preempted because there is some
lock held (every lock will increment the preemption count).

The mouse doesn't move because X cannot be executed...

A quick test you can do is to enable CONFIG_MAGIC_SYSRQ and try with
"ALT + Stamp + B" when the system "freezes". If it reboots then
interrupts work  :)

-- 
	Paolo Ornati
	Linux 2.6.20-rc1-g99f5e971 on x86_64

Re: newbie questions about while (1) in kernel mode and spinlocks

[Steven Rostedt]

On Thu, 2006-12-21 at 10:41 +0100, Sorin Manolache wrote:
> Dear list,
> 
> I am in the process of learning how to write linux device drivers.
> 
> I have a 2.6.16.5 kernel running on a monoprocessor machine.

We usually call that a Uniprocessor or just UP.

> #CONFIG_SMP is not set
> CONFIG_DEBUG_SPINLOCK=y.
> CONFIG_PREEMPT=y
> CONFIG_PREEMPT_BKL=y
> 
> First question:
> 
> I wrote
> 
> while (1)
>     ;
> 
> in the read function of a test device driver. I expect the calling
> process to freeze, and then a timer interrupt to preempt the kernel
> and to schedule another process. This does not happen, the whole
> system freezes. I see no effect from pressing keys or moving the
> mouse. Why? The hardware interrupts are not disabled, are they? Why do
> the interrupt handlers not get executed?

Matters what you have in that code. Are you sure interrupts are not
handled? The can be, and you just don't notice, because the programs
that get affected by the interrupts are not able to run.

I don't know what your read function looks like, or how you got there,
but a while(1) would only slow the system down quite a bit. It shouldn't
lock it up. (/me goes to try it out on a dummy driver after writing
this).

> 
> Second question:
> 
> I wrote
> 
> spin_lock(&lck);
> down(&sem); /* I know that one shouldn't sleep when holding a lock */
>                     /* but I want to understand why */

Well a spin_lock is just that. It spins.  What happens if you schedule,
and the next process that goes to run tries to grab that same spin_lock.
It spins, thinking the lock holder is on another CPU and it will be
released shortly. But then, the other CPU (assuming a 2x system) has a
process that tries to grab this same spin_lock, now the system is truely
dead locked. All CPUS are spinning waiting for the non-running process
to let go of the spin lock.

> spin_unlock(&lck);
> 
> in the read function and
> 
> up(&sem)
> 
> in the write function. The semaphore is initially locked, so the first
> process invoking down will sleep.
> 
> I invoke
> 
> cat /dev/test
> 
> and the process sleeps on the semaphore. Then I invoke
> 
> echo 1 > /dev/test
> 
> and I wake up the "cat" process.
> 
> Then I intend to invoke _two_ cat processes. I expect the first one to
> sleep on the semaphore and the second on to spin at the spin_lock.
> Then I expect to wake up the first process by invoking an echo, the
> first process to release the lock and the second process to sleep on
> the semaphore. What I get is that the system freezes as soon as I
> invoke the second "cat" process. Again, no effect from key presses or
> mouse movements. Why? Shouldn't the timer interrupt preempt the second
> "cat" process that spins on the spinlock and give control to something
> else, for example to the console where I could wake up the first "cat"
> process? Why do I not see any effect from mouse movements? Hardware
> interrupts are not disabled, are they?

A spin_lock will not preempt. So if you are doing this on a UP system, a
spin lock will only be a preempt disable (with CONFIG_PREEMPT=y).
There's no need for spin_locks for UP.

I'd have to take a look at the actual code to explain exactly what mess
you are making for yourself ;)

> 
> Third question:
> 
> The Linux Device Drivers book says that a spin_lock should not be
> shared between a process and an interrupt handler. The explanation is
> that the process may hold the lock, an interrupt occurs, the interrupt
> handler spins on the lock held by the process and the system freezes.
> Why should it freeze? Isn't it possible for the interrupt handler to
> re-enable interrupts as its first thing, then to spin at the lock, the
> timer interrupt to preempt the interrupt handler and to relinquish
> control to the process which in turn will finish its critical section
> and release the lock, making way for the interrupt handler to
> continue.

I believe that Paolo explained this, but I can go into more details if
you need.

> 
> Thank you very much for clarifying these issues.

No prob.

-- Steve

Re: newbie questions about while (1) in kernel mode and spinlocks

[Robert Hancock]

Sorin Manolache wrote:
> 
> The Linux Device Drivers book says that a spin_lock should not be
> shared between a process and an interrupt handler. The explanation is
> that the process may hold the lock, an interrupt occurs, the interrupt
> handler spins on the lock held by the process and the system freezes.
> Why should it freeze? Isn't it possible for the interrupt handler to
> re-enable interrupts as its first thing, then to spin at the lock, the
> timer interrupt to preempt the interrupt handler and to relinquish
> control to the process which in turn will finish its critical section
> and release the lock, making way for the interrupt handler to
> continue.

When the timer interrupt finishes, it's not going to return control to 
the process, it's going to return control to what it interrupted which 
is the interrupt handler in your code which will just continue spinning. 
Interrupt handlers are not preemptible by anything other than other 
interrupt handlers (and then only in some cases) so it will sit spinning 
on that lock forever.

-- 
Robert Hancock      Saskatoon, SK, Canada
To email, remove "nospam" from hancockr@nospamshaw.ca
Home Page: http://www.roberthancock.com/

RE: newbie questions about while (1) in kernel mode and spinlocks

[SR, Krishna]

Hi,

Well, Spin lock itself is a while loop. So in case a process on another
CPU has the semaphore and you get a  spin lock and try to wait on a
semaphore and the other CPU also tries to get the spin lock then you are
in a dead loop.


CPU 1                   CPU 2
		            Get sem-1 (success)
				......
Spin lock (success)	.......

...				.....
...				Spin lock??? No you are locked....
Get sem -1 (block)
You are blocked now

That is the reason we always make sure that we never do any thing that
can sleep in a place after we have spin lock.

And as Paolo tells, all these if PREEMPTION is disabled. But in any case
if the signals are not handled then the processes are blocked.

Thanks and Best Regards,
KK
-----Original Message-----
From: Paolo Ornati [mailto:ornati@fastwebnet.it] 
Sent: Thursday, December 21, 2006 2:20 AM
To: Sorin Manolache
Cc: linux-kernel@vger.kernel.org
Subject: Re: newbie questions about while (1) in kernel mode and
spinlocks

On Thu, 21 Dec 2006 10:41:44 +0100
"Sorin Manolache" <sorinm@gmail.com> wrote:

> spin_lock(&lck);
> down(&sem); /* I know that one shouldn't sleep when holding a lock */
>                     /* but I want to understand why */

I suppose because the lock is held for an indefinite amount of time and
any other process that try to get that lock will "spin" and burn CPU
without doing anything useful (locking the process in kernel mode and
preventing the execution of other processes on that CPU if there
isn't any type of PREEMPTION).

:)

spin_lock is a "while(1) {...}" thing...

-- 
	Paolo Ornati
	Linux 2.6.20-rc1-g99f5e971 on x86_64
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/