Re: what's the semaphore in requests for?

Peter T. Breuer
2001-08-03T23:37:44Z

OK, It begins to look to me as though something takes the io_request
spinlock and then sleeps. I thought it was me, but now I'm not so sure.

New readers, set scene here:

"ptb wrote:"
> > > > 2 processors + 2 userspace helper daemon on device = bug (lockup)
> > > > 1 processors + 1 userspace helper daemon on device = no bug

And I now believe the explanation is as follows:

cpu1 cpu2

ioctl takes private spinlock something takes io_request spinlock
IDE interrupt takes io spinlock schedule
deadlock dev req fn tries for private spinlock
deadlock

This deadlock relies on

1) me not protecting against interrupts in my spinlocks (reasonable,
since my spinlock is not taken from an interrupt).
2) something taking the io spinlock and sleeping while it's held.

I thought at first that (2) was me, but it may not be so.

I surmised that when I held the io_request spinlock that the
end_that_request_first function may have been sleeping occasionally ..
but I moved the spinlock to cover only end_that_request_last and I still
see deadlocks when (1) is true. When (1) is not true, no deadlock,
wherever I put the spinlock in end_request. So the indication is that
it's not (only) me who's taking the io_request spinlock and then
sleeping.

Does anyone have any comment? I am running 2.4.6 with the latest
aic7xxx patches.

My grounds for believing in the scenario above are that when a
lockup occurs, in the kernel debugger I see both my helper daemon
threads scheduled, and a userspace function deadlocked in IDE_do_irq in
a spinlock, and one cpu not responding to NMI. Since I don't schedule
or sleep when holding my spinlock, my spinlock cannot be held at this
point, and the deadlock must be against another spinlock. That must be
the io_request spinlock (it's the only other spinlock in my code too).
And the deadlock only happens when I don't mask interrupts when taking
my spinlock, so it must happen in an interrupt manager .. and it's
clearly in the IDE manager, and I believe it's against the io_request
spinlock. Also the deadlock must be a cross-cpu deadlock, as it only
happened when I ran two helper client daemons, not one.

The lockup happens about 1 in every 250000 requests.

My SMP machine is already slightly unstable under 2.4 kernels, about
one spontaneous reboot per four to seven days.

Peter