* side effects of calling interruptible_sleep_on_timeout()
@ 2012-04-24 9:54 devendra rawat
2012-04-24 22:06 ` Philipp Ittershagen
0 siblings, 1 reply; 6+ messages in thread
From: devendra rawat @ 2012-04-24 9:54 UTC (permalink / raw)
To: kernelnewbies
Hi,
A switch driver is causing soft lockup on Montavista Linux Kernel 2.6.10
system.
While browsing through the code of the driver. I came across a snippet
where after disabling the interrupts
a call is made to interruptible_sleep_on_timeout().
The code snippet is like
cli();
init_waitqueue_head(&queue);
interruptible_sleep_on_timeout(&queue, USEC_TO_JIFFIES(usec));
thread_check_signals();
sti();
I need to know the side effect of this sort of code, can it be responsible
for the softlockup of the system ? Its a PowerPC based system.
Regards,
Devendra
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20120424/5ddbd029/attachment.html
^ permalink raw reply [flat|nested] 6+ messages in thread* side effects of calling interruptible_sleep_on_timeout() 2012-04-24 9:54 side effects of calling interruptible_sleep_on_timeout() devendra rawat @ 2012-04-24 22:06 ` Philipp Ittershagen 2012-04-25 10:26 ` Srivatsa S. Bhat 0 siblings, 1 reply; 6+ messages in thread From: Philipp Ittershagen @ 2012-04-24 22:06 UTC (permalink / raw) To: kernelnewbies Hi Devendra, On Tue, Apr 24, 2012 at 03:24:23PM +0530, devendra rawat wrote: > Hi, > A switch driver is causing soft lockup on Montavista Linux Kernel > 2.6.10 system. > While browsing through the code of the driver. I came across a snippet > where after disabling the interrupts > a call is made to interruptible_sleep_on_timeout(). > The code snippet is like > cli(); > init_waitqueue_head(&queue); > interruptible_sleep_on_timeout(&queue, USEC_TO_JIFFIES(usec)); > thread_check_signals(); > sti(); > I need to know the side effect of this sort of code, can it be > responsible for the softlockup of the system ? Its a PowerPC based > system. you cannot call sleep functions after disabling interrupts, because no interrupt will arrive for the scheduler to see the timeout and resume your task. Greetings, Philipp ^ permalink raw reply [flat|nested] 6+ messages in thread
* side effects of calling interruptible_sleep_on_timeout() 2012-04-24 22:06 ` Philipp Ittershagen @ 2012-04-25 10:26 ` Srivatsa S. Bhat 2012-04-26 4:33 ` Arun KS 0 siblings, 1 reply; 6+ messages in thread From: Srivatsa S. Bhat @ 2012-04-25 10:26 UTC (permalink / raw) To: kernelnewbies On 04/25/2012 03:36 AM, Philipp Ittershagen wrote: > Hi Devendra, > > On Tue, Apr 24, 2012 at 03:24:23PM +0530, devendra rawat wrote: >> Hi, >> A switch driver is causing soft lockup on Montavista Linux Kernel >> 2.6.10 system. >> While browsing through the code of the driver. I came across a snippet >> where after disabling the interrupts >> a call is made to interruptible_sleep_on_timeout(). >> The code snippet is like >> cli(); >> init_waitqueue_head(&queue); >> interruptible_sleep_on_timeout(&queue, USEC_TO_JIFFIES(usec)); >> thread_check_signals(); >> sti(); >> I need to know the side effect of this sort of code, can it be >> responsible for the softlockup of the system ? Its a PowerPC based >> system. > > you cannot call sleep functions after disabling interrupts, because no > interrupt will arrive for the scheduler to see the timeout and resume your > task. > Yes, that's right. Also, in general, sleeping inside atomic sections (eg., sections with interrupts disabled or preempt disabled) is wrong. There is a config option in the kernel that you can use to enable sleep-inside-atomic-section-checking (CONFIG_DEBUG_ATOMIC_SLEEP I believe), which can help you pin-point such bugs easily. Regards, Srivatsa S. Bhat ^ permalink raw reply [flat|nested] 6+ messages in thread
* side effects of calling interruptible_sleep_on_timeout() 2012-04-25 10:26 ` Srivatsa S. Bhat @ 2012-04-26 4:33 ` Arun KS 2012-04-26 8:03 ` Srivatsa S. Bhat [not found] ` <E351E450E8B9F54684A699D42DC5ADF21FB079C4@MPBAGVEX02.corp.mphasis.com> 0 siblings, 2 replies; 6+ messages in thread From: Arun KS @ 2012-04-26 4:33 UTC (permalink / raw) To: kernelnewbies Hi Srivatsa, On Wed, Apr 25, 2012 at 3:56 PM, Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com> wrote: > On 04/25/2012 03:36 AM, Philipp Ittershagen wrote: > >> Hi Devendra, >> >> On Tue, Apr 24, 2012 at 03:24:23PM +0530, devendra rawat wrote: >>> ? ?Hi, >>> ? ?A switch driver is causing soft lockup on Montavista Linux Kernel >>> ? ?2.6.10 system. >>> ? ?While browsing through the code of the driver. I came across a snippet >>> ? ?where after disabling the interrupts >>> ? ?a call is made to interruptible_sleep_on_timeout(). >>> ? ?The code snippet is like >>> ? ?cli(); >>> ? ?init_waitqueue_head(&queue); >>> ? ? ? ? ? ?interruptible_sleep_on_timeout(&queue, USEC_TO_JIFFIES(usec)); >>> ? ? ? ? ? ?thread_check_signals(); >>> ? ?sti(); >>> ? ?I need to know the side effect of this sort of code, can it be >>> ? ?responsible for the softlockup of the system ? Its a PowerPC based >>> ? ?system. >> >> you cannot call sleep functions after disabling interrupts, because no >> interrupt will arrive for the scheduler to see the timeout and resume your >> task. >> > > > Yes, that's right. Also, in general, sleeping inside atomic sections (eg., > sections with interrupts disabled or preempt disabled) is wrong. There is a > config option in the kernel that you can use to enable > sleep-inside-atomic-section-checking (CONFIG_DEBUG_ATOMIC_SLEEP I believe), > which can help you pin-point such bugs easily. I tired an experiment to check this. /* disable interrupts and preemption */ spin_lock_irqsave(&lock, flags); /* enable preemption, but interrupt still disabled */ spin_unlock(&lock); /* Now schedule something else */ schedule_timeout(10 * HZ); But this is not causing any harm. I m able to call schedule with interrupt disabled and system works fine afterwards. So when I looked inside the schedule() function, it checks only whether preemption is disabled or not. schedule calls BUG() only if preemption is disabled and not if interrupts are disabled. And AFAIK there is no fuction inside the kernel which tells you that interrupt are disabled. So explantion why system works fine after calling a schedule with interrupt disabled go here, There is a raw_spin_lock_irq(&rq->lock) inside the __schedule() which in turn calls local_irq_disable(). local_irq_disable/enable() functions are not nested. We dont have reference counting. One call to local_irq_enable is enough to enable multiple calls of local_irq_disable(). So my inference is that if you call a schedule with interrupt disable will not cause any problem. Because schedule function enable it back before we really schedules out. But call to schedule() with preemtion disabled will end up in famous BUG scheduling while atomic. NB: Kernel version used is 3.0.15 Thanks, Arun > > Regards, > Srivatsa S. Bhat > > > _______________________________________________ > Kernelnewbies mailing list > Kernelnewbies at kernelnewbies.org > http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies ^ permalink raw reply [flat|nested] 6+ messages in thread
* side effects of calling interruptible_sleep_on_timeout() 2012-04-26 4:33 ` Arun KS @ 2012-04-26 8:03 ` Srivatsa S. Bhat [not found] ` <E351E450E8B9F54684A699D42DC5ADF21FB079C4@MPBAGVEX02.corp.mphasis.com> 1 sibling, 0 replies; 6+ messages in thread From: Srivatsa S. Bhat @ 2012-04-26 8:03 UTC (permalink / raw) To: kernelnewbies On 04/26/2012 10:03 AM, Arun KS wrote: > Hi Srivatsa, > > On Wed, Apr 25, 2012 at 3:56 PM, Srivatsa S. Bhat > <srivatsa.bhat@linux.vnet.ibm.com> wrote: >> On 04/25/2012 03:36 AM, Philipp Ittershagen wrote: >> >>> Hi Devendra, >>> >>> On Tue, Apr 24, 2012 at 03:24:23PM +0530, devendra rawat wrote: >>>> Hi, >>>> A switch driver is causing soft lockup on Montavista Linux Kernel >>>> 2.6.10 system. >>>> While browsing through the code of the driver. I came across a snippet >>>> where after disabling the interrupts >>>> a call is made to interruptible_sleep_on_timeout(). >>>> The code snippet is like >>>> cli(); >>>> init_waitqueue_head(&queue); >>>> interruptible_sleep_on_timeout(&queue, USEC_TO_JIFFIES(usec)); >>>> thread_check_signals(); >>>> sti(); >>>> I need to know the side effect of this sort of code, can it be >>>> responsible for the softlockup of the system ? Its a PowerPC based >>>> system. >>> >>> you cannot call sleep functions after disabling interrupts, because no >>> interrupt will arrive for the scheduler to see the timeout and resume your >>> task. >>> >> >> >> Yes, that's right. Also, in general, sleeping inside atomic sections (eg., >> sections with interrupts disabled or preempt disabled) is wrong. There is a >> config option in the kernel that you can use to enable >> sleep-inside-atomic-section-checking (CONFIG_DEBUG_ATOMIC_SLEEP I believe), >> which can help you pin-point such bugs easily. > > I tired an experiment to check this. > > /* disable interrupts and preemption */ > spin_lock_irqsave(&lock, flags); > /* enable preemption, but interrupt still disabled */ > spin_unlock(&lock); > /* Now schedule something else */ > schedule_timeout(10 * HZ); > > But this is not causing any harm. I m able to call schedule with > interrupt disabled and system works fine afterwards. > > So when I looked inside the schedule() function, it checks only > whether preemption is disabled or not. schedule calls BUG() only if > preemption is disabled and not if interrupts are disabled. > > And AFAIK there is no fuction inside the kernel which tells you that > interrupt are disabled. > > So explantion why system works fine after calling a schedule with > interrupt disabled go here, > > There is a raw_spin_lock_irq(&rq->lock) inside the __schedule() which > in turn calls local_irq_disable(). > > local_irq_disable/enable() functions are not nested. We dont have > reference counting. > One call to local_irq_enable is enough to enable multiple calls of > local_irq_disable(). > > So my inference is that if you call a schedule with interrupt disable > will not cause any problem. Because schedule function enable it back > before we really schedules out. > But call to schedule() with preemtion disabled will end up in famous > BUG scheduling while atomic. > Indeed, you are right! And your experiment and analysis is perfect too! Sorry for the confusion - I had used the term "atomic" quite loosely. But your careful experiment of just re-enabling preemption, while still keeping the interrupts disabled was a very good one! And to add to what you said above, the __schedule() also does a preempt_enable() to re-enable preemption (which it had disabled at the beginning). But since preempt_disable() can nest, if we had called __schedule() with preemption already disabled, then we end up in trouble - and hence the BUG is fired in such cases. Thanks for the clarification! Regards, Srivatsa S. Bhat ^ permalink raw reply [flat|nested] 6+ messages in thread
[parent not found: <E351E450E8B9F54684A699D42DC5ADF21FB079C4@MPBAGVEX02.corp.mphasis.com>]
[parent not found: <CABOM9ZoJk5FMSPeFGrZhbS_raUdnpXYjZraJd0Ddm0E5E+n00A@mail.gmail.com>]
[parent not found: <E351E450E8B9F54684A699D42DC5ADF21FB0812D@MPBAGVEX02.corp.mphasis.com>]
[parent not found: <CABOM9ZqEyZh8Oe3hKp8wb6OhzqP+7mRiXRAYuD3+iWBPBNkUBg@mail.gmail.com>]
[parent not found: <E351E450E8B9F54684A699D42DC5ADF21FB777E1@MPBAGVEX02.corp.mphasis.com>]
* side effects of calling interruptible_sleep_on_timeout() [not found] ` <E351E450E8B9F54684A699D42DC5ADF21FB777E1@MPBAGVEX02.corp.mphasis.com> @ 2012-04-26 12:17 ` Arun KS 0 siblings, 0 replies; 6+ messages in thread From: Arun KS @ 2012-04-26 12:17 UTC (permalink / raw) To: kernelnewbies On Thu, Apr 26, 2012 at 3:48 PM, Shashidhara Shamaiah <Shashidhara.Shamaiah@mphasis.com> wrote: > Hi Arun, > > My question is how will the scheduler run, if interrupts are disabled on the local CPU. As timer interrupt controls the CPU? scheduler is nothing but a piece of code( __schedule()) which is just like any other function. If interrupts are disabled, you wont be getting the timer interrupts. So time slice will never get elapsed, which in turn allows current process to run for ever until it calls schedule again or exits. But as we figured out that __scheduler is enabling the interrupts back before we really schedules out. So this problem won't occur in our Linux kernel. Thanks, Arun > > Regards > Shashidhara > > -----Original Message----- > From: Arun KS [mailto:getarunks at gmail.com] > Sent: Thursday, April 26, 2012 11:36 AM > To: Shashidhara Shamaiah > Subject: Re: side effects of calling interruptible_sleep_on_timeout() > > Hi, > > My code is, > > /* disable interrupts and preemption */ > spin_lock_irqsave(&lock, flags); > /* enable preemption, but interrupt still disabled */ > spin_unlock(&lock); > /* Now schedule something else */ > schedule_timeout(10 * HZ); > > If you see the second step spin_unlock(&lock). This will enable the > preemtion back. > > Arun > > On Thu, Apr 26, 2012 at 11:26 AM, Shashidhara Shamaiah > <Shashidhara.Shamaiah@mphasis.com> wrote: >> Hi Arun, >> >> I think if you run the code on a SMP machine, then the BUG() in schedule() will be triggered as spinlock_irqsave() disables preemption for a SMP. >> >> Regards >> Shashidhara >> >> -----Original Message----- >> From: Arun KS [mailto:getarunks at gmail.com] >> Sent: Thursday, April 26, 2012 10:41 AM >> To: Shashidhara Shamaiah >> Subject: Re: side effects of calling interruptible_sleep_on_timeout() >> >> Hello Shashidhara, >> >> My machine is UP. >> >> thanks, >> Arun >> >> On Thu, Apr 26, 2012 at 10:23 AM, Shashidhara Shamaiah >> <Shashidhara.Shamaiah@mphasis.com> wrote: >>> Hi Arun, >>> >>> One more question regarding your experiment. Were you running your code on a UP or SMP machine? >>> >>> Regards >>> Shashidhara >>> >>> -----Original Message----- >>> From: kernelnewbies-bounces at kernelnewbies.org [mailto:kernelnewbies-bounces at kernelnewbies.org] On Behalf Of Arun KS >>> Sent: Thursday, April 26, 2012 10:03 AM >>> To: Srivatsa S. Bhat >>> Cc: vinayak menon; Philipp Ittershagen; devendra rawat; kernelnewbies at kernelnewbies.org >>> Subject: Re: side effects of calling interruptible_sleep_on_timeout() >>> >>> Hi Srivatsa, >>> >>> On Wed, Apr 25, 2012 at 3:56 PM, Srivatsa S. Bhat >>> <srivatsa.bhat@linux.vnet.ibm.com> wrote: >>>> On 04/25/2012 03:36 AM, Philipp Ittershagen wrote: >>>> >>>>> Hi Devendra, >>>>> >>>>> On Tue, Apr 24, 2012 at 03:24:23PM +0530, devendra rawat wrote: >>>>>> ? ?Hi, >>>>>> ? ?A switch driver is causing soft lockup on Montavista Linux Kernel >>>>>> ? ?2.6.10 system. >>>>>> ? ?While browsing through the code of the driver. I came across a snippet >>>>>> ? ?where after disabling the interrupts >>>>>> ? ?a call is made to interruptible_sleep_on_timeout(). >>>>>> ? ?The code snippet is like >>>>>> ? ?cli(); >>>>>> ? ?init_waitqueue_head(&queue); >>>>>> ? ? ? ? ? ?interruptible_sleep_on_timeout(&queue, USEC_TO_JIFFIES(usec)); >>>>>> ? ? ? ? ? ?thread_check_signals(); >>>>>> ? ?sti(); >>>>>> ? ?I need to know the side effect of this sort of code, can it be >>>>>> ? ?responsible for the softlockup of the system ? Its a PowerPC based >>>>>> ? ?system. >>>>> >>>>> you cannot call sleep functions after disabling interrupts, because no >>>>> interrupt will arrive for the scheduler to see the timeout and resume your >>>>> task. >>>>> >>>> >>>> >>>> Yes, that's right. Also, in general, sleeping inside atomic sections (eg., >>>> sections with interrupts disabled or preempt disabled) is wrong. There is a >>>> config option in the kernel that you can use to enable >>>> sleep-inside-atomic-section-checking (CONFIG_DEBUG_ATOMIC_SLEEP I believe), >>>> which can help you pin-point such bugs easily. >>> >>> I tired an experiment to check this. >>> >>> /* disable interrupts and preemption */ >>> spin_lock_irqsave(&lock, flags); >>> /* enable preemption, but interrupt still disabled */ >>> spin_unlock(&lock); >>> /* Now schedule something else */ >>> schedule_timeout(10 * HZ); >>> >>> But this is not causing any harm. I m able to call schedule with >>> interrupt disabled and system works fine afterwards. >>> >>> So when I looked inside the schedule() function, it checks only >>> whether preemption is disabled or not. schedule calls ?BUG() only if >>> preemption is disabled and not if interrupts are disabled. >>> >>> And AFAIK there is no fuction inside the kernel which tells you that >>> interrupt are disabled. >>> >>> So explantion why system works fine after calling a schedule with >>> interrupt disabled go here, >>> >>> There is a raw_spin_lock_irq(&rq->lock) inside the __schedule() which >>> in turn calls local_irq_disable(). >>> >>> local_irq_disable/enable() functions are not nested. We dont have >>> reference counting. >>> One call to local_irq_enable is enough to enable multiple calls of >>> local_irq_disable(). >>> >>> So my inference is that if you call a schedule with interrupt disable >>> will not cause any problem. Because schedule function enable it back >>> before we really schedules out. >>> But call to schedule() with preemtion disabled will end up in famous >>> BUG scheduling while atomic. >>> >>> NB: Kernel version used is 3.0.15 >>> >>> Thanks, >>> Arun >>> >>> >>>> >>>> Regards, >>>> Srivatsa S. Bhat >>>> >>>> >>>> _______________________________________________ >>>> Kernelnewbies mailing list >>>> Kernelnewbies at kernelnewbies.org >>>> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies >>> >>> _______________________________________________ >>> Kernelnewbies mailing list >>> Kernelnewbies at kernelnewbies.org >>> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies >>> >>> Information transmitted by this e-mail is proprietary to MphasiS, its associated companies and/ or its customers and is intended >>> for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or >>> exempt from disclosure under applicable law. If you are not the intended recipient or it appears that this mail has been forwarded >>> to you without proper authority, you are notified that any use or dissemination of this information in any manner is strictly >>> prohibited. In such cases, please notify us immediately at mailmaster at mphasis.com and delete this mail from your records. >>> >> >> Information transmitted by this e-mail is proprietary to MphasiS, its associated companies and/ or its customers and is intended >> for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or >> exempt from disclosure under applicable law. If you are not the intended recipient or it appears that this mail has been forwarded >> to you without proper authority, you are notified that any use or dissemination of this information in any manner is strictly >> prohibited. In such cases, please notify us immediately at mailmaster at mphasis.com and delete this mail from your records. >> > > Information transmitted by this e-mail is proprietary to MphasiS, its associated companies and/ or its customers and is intended > for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or > exempt from disclosure under applicable law. If you are not the intended recipient or it appears that this mail has been forwarded > to you without proper authority, you are notified that any use or dissemination of this information in any manner is strictly > prohibited. In such cases, please notify us immediately at mailmaster at mphasis.com and delete this mail from your records. > ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2012-04-26 12:17 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-04-24 9:54 side effects of calling interruptible_sleep_on_timeout() devendra rawat
2012-04-24 22:06 ` Philipp Ittershagen
2012-04-25 10:26 ` Srivatsa S. Bhat
2012-04-26 4:33 ` Arun KS
2012-04-26 8:03 ` Srivatsa S. Bhat
[not found] ` <E351E450E8B9F54684A699D42DC5ADF21FB079C4@MPBAGVEX02.corp.mphasis.com>
[not found] ` <CABOM9ZoJk5FMSPeFGrZhbS_raUdnpXYjZraJd0Ddm0E5E+n00A@mail.gmail.com>
[not found] ` <E351E450E8B9F54684A699D42DC5ADF21FB0812D@MPBAGVEX02.corp.mphasis.com>
[not found] ` <CABOM9ZqEyZh8Oe3hKp8wb6OhzqP+7mRiXRAYuD3+iWBPBNkUBg@mail.gmail.com>
[not found] ` <E351E450E8B9F54684A699D42DC5ADF21FB777E1@MPBAGVEX02.corp.mphasis.com>
2012-04-26 12:17 ` Arun KS
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).