* spin_lock and scheduler confusion [not found] <D69C90565D53114396BF743585AF5A09122E61E90F@VSHINMSMBX01.vshodc.lntinfotech.com> @ 2011-01-05 22:23 ` Viral Mehta 2011-01-07 3:51 ` Alexandre Courbot ` (2 more replies) 0 siblings, 3 replies; 15+ messages in thread From: Viral Mehta @ 2011-01-05 22:23 UTC (permalink / raw) To: kernelnewbies Hi , I need your help to solve below confusion. 1. I know that spin_lock_irqsave, disables premption and local_irq. 2. There is some below piece of kernel code running on Quad core Machine. spin_lock_irqsave(&lock, flags); some_large_critical_section //sleep of 5 minutes //I know there should be as small as possible task in critical section // treat this as an example only spin_lock_irqrestore(); 3. One of the CPU core tries to execute this code and so acquires the lock. 4. Now, second core is also goes to execute same piece of code and so will keep spinning for lock Now, the question is Will EVER second CPU core get chance to execute another task ? Ever if timeslice is over for the current task ? What if scheduler code is running on CPU core-3 and sees that timeslice for task running on CPU core-2 has expired ? I guess timeslice expire case is not as same as preemption. Or may be I am terribly wrong. Please help, Thanks, Viral ________________________________ The contents of this e-mail and any attachment(s) may contain confidential or privileged information for the intended recipient(s). Unintended recipients are prohibited from taking action on the basis of information in this e-mail and using or disseminating the information, and must notify the sender and delete it from their system. L&T Infotech will not accept responsibility or liability for the accuracy or completeness of, or the presence of any virus or disabling code in this e-mail" ______________________________________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110106/2de907f8/attachment-0001.html ^ permalink raw reply [flat|nested] 15+ messages in thread
* spin_lock and scheduler confusion 2011-01-05 22:23 ` spin_lock and scheduler confusion Viral Mehta @ 2011-01-07 3:51 ` Alexandre Courbot 2011-01-07 3:56 ` Mulyadi Santosa 2011-01-07 5:28 ` Dave Hylands 2 siblings, 0 replies; 15+ messages in thread From: Alexandre Courbot @ 2011-01-07 3:51 UTC (permalink / raw) To: kernelnewbies Hi, > 2. There is?some below piece of kernel?code running on Quad core Machine. > ??? spin_lock_irqsave(&lock, flags); > ????some_large_critical_section //sleep of 5 minutes > ??? //I know there should be as small as possible task in critical section > ??? // treat this as an example only > ??? spin_lock_irqrestore(); > 3. One of the CPU core tries to execute this code and so acquires the lock. > 4. Now, second core is also goes to execute same piece of code and so will > keep spinning for lock > > Now, the question is > Will?EVER?second CPU core get chance to execute another task ? > Ever if?timeslice is over for the current task ? As far as I understand (from quickly reading include/linux/spinlock*.h), kernel preemption is disabled as long as a spin lock is used (and in your case, as soon as spin_lock_irqsave is entered). This means that a core that started acquiring a spinlock will not execute another task not only until the lock is acquired, but until the lock is released. > What if scheduler code is running on CPU core-3 and sees that > timeslice?for task running on CPU core-2 has expired ? AFAIK (that is, not very far) the scheduler code only cares for the current core and does not assign tasks to others. But even if it did, it would not have any way to stop the second core until it releases the spin lock. So in any case, both core 1 and core 2 will be blocked until they release the lock. Alex. ^ permalink raw reply [flat|nested] 15+ messages in thread
* spin_lock and scheduler confusion 2011-01-05 22:23 ` spin_lock and scheduler confusion Viral Mehta 2011-01-07 3:51 ` Alexandre Courbot @ 2011-01-07 3:56 ` Mulyadi Santosa 2011-01-07 5:28 ` Dave Hylands 2 siblings, 0 replies; 15+ messages in thread From: Mulyadi Santosa @ 2011-01-07 3:56 UTC (permalink / raw) To: kernelnewbies Dear Viral... On Thu, Jan 6, 2011 at 05:23, Viral Mehta <Viral.Mehta@lntinfotech.com> wrote: > > Hi , > > I need your help to solve below confusion. > > 1. I know that spin_lock_irqsave, disables?premption and local_irq. > 2. There is?some below piece of kernel?code running on Quad core Machine. > ??? spin_lock_irqsave(&lock, flags); > ????some_large_critical_section //sleep of 5 minutes > ??? //I know there should be as small as possible task in critical section > ??? // treat this as an example only > ??? spin_lock_irqrestore(); > 3. One of the CPU core tries to execute this code and so acquires the lock. > 4. Now, second core is also goes to execute same piece of code and so will > keep spinning for lock > > Now, the question is > Will?EVER?second CPU core get chance to execute another task ? > Ever if?timeslice is over for the current task ? If interrupt is also disabled, then I think it's a game over...preemption won't happen on that CPU.....i mean, not system triggered preemption...the process must voluntarily yield itself, thus giving others time to run. This, assuming HPET signaling also based on interrupt...hence affected by irq disabling... -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com ^ permalink raw reply [flat|nested] 15+ messages in thread
* spin_lock and scheduler confusion 2011-01-05 22:23 ` spin_lock and scheduler confusion Viral Mehta 2011-01-07 3:51 ` Alexandre Courbot 2011-01-07 3:56 ` Mulyadi Santosa @ 2011-01-07 5:28 ` Dave Hylands 2011-01-07 6:27 ` Tayade, Nilesh 2011-01-07 17:46 ` Viral Mehta 2 siblings, 2 replies; 15+ messages in thread From: Dave Hylands @ 2011-01-07 5:28 UTC (permalink / raw) To: kernelnewbies Hi Viral, On Wed, Jan 5, 2011 at 2:23 PM, Viral Mehta <Viral.Mehta@lntinfotech.com> wrote: > > Hi , > > I need your help to solve below confusion. > > 1. I know that spin_lock_irqsave, disables?premption and local_irq. > 2. There is?some below piece of kernel?code running on Quad core Machine. > ??? spin_lock_irqsave(&lock, flags); > ????some_large_critical_section //sleep of 5 minutes > ??? //I know there should be as small as possible task in critical section > ??? // treat this as an example only Note that you can't sleep while you hold a spinlock. You're not allowed to perform any type of blocking operations. If you're holding the spinlock for any significant length of time, then you're using the wrong design. > ??? spin_lock_irqrestore(); > 3. One of the CPU core tries to execute this code and so acquires the lock. > 4. Now, second core is also goes to execute same piece of code and so will > keep spinning for lock > > Now, the question is > Will?EVER?second CPU core get chance to execute another task ? Not while it's holding the spinlock or waiting for the spinlock. > Ever if?timeslice is over for the current task ? The time tick interrupt is what determines when the timeslice is over. Since you have interrupts disabled, the timer interrupt can't happen. > What if scheduler code is running on CPU core-3 and sees that > timeslice?for task running on CPU core-2 has expired ? Each core only considers the timeslices for its own core. > I guess?timeslice expire case is not as same as preemption. Or may be I am > terribly wrong. You shouldn't be holding a spinlock for periods of time approaching the length of a timeslice. The timer interrupt is what determines the end of a timeslice. No timer interrupt, no end of a timeslice. Preemption is also triggered by the timer interrupt, or by releasing a resource that a higher priority task is waiting for. Dave Hylands ^ permalink raw reply [flat|nested] 15+ messages in thread
* spin_lock and scheduler confusion 2011-01-07 5:28 ` Dave Hylands @ 2011-01-07 6:27 ` Tayade, Nilesh 2011-01-07 7:02 ` Tirtha Ghosh 2011-01-07 15:51 ` Dave Hylands 2011-01-07 17:46 ` Viral Mehta 1 sibling, 2 replies; 15+ messages in thread From: Tayade, Nilesh @ 2011-01-07 6:27 UTC (permalink / raw) To: kernelnewbies Hi, > -----Original Message----- > From: kernelnewbies-bounces at kernelnewbies.org [mailto:kernelnewbies- > bounces at kernelnewbies.org] On Behalf Of Dave Hylands > Sent: Friday, January 07, 2011 10:59 AM > To: Viral Mehta > Cc: kernelnewbies at kernelnewbies.org > Subject: Re: spin_lock and scheduler confusion > > Hi Viral, > > On Wed, Jan 5, 2011 at 2:23 PM, Viral Mehta > <Viral.Mehta@lntinfotech.com> wrote: > > > > Hi , > > > > I need your help to solve below confusion. > > [...] > > Note that you can't sleep while you hold a spinlock. You're not > allowed to perform any type of blocking operations. If you're holding > the spinlock for any significant length of time, then you're using the > wrong design. > > > ??? spin_lock_irqrestore(); > > 3. One of the CPU core tries to execute this code and so acquires the > lock. > > 4. Now, second core is also goes to execute same piece of code and so > will [...] > > Not while it's holding the spinlock or waiting for the spinlock. > > > Ever if?timeslice is over for the current task ? > > The time tick interrupt is what determines when the timeslice is over. > Since you have interrupts disabled, the timer interrupt can't happen. > > > What if scheduler code is running on CPU core-3 and sees that > > timeslice?for task running on CPU core-2 has expired ? > > Each core only considers the timeslices for its own core. > > > I guess?timeslice expire case is not as same as preemption. Or may be > I am > > terribly wrong. > > You shouldn't be holding a spinlock for periods of time approaching > the length of a timeslice. The timer interrupt is what determines the > end of a timeslice. No timer interrupt, no end of a timeslice. > Preemption is also triggered by the timer interrupt, or by releasing a > resource that a higher priority task is waiting for. May be my understanding is incorrect, but wouldn't we hit the NMI watchdog here(assuming we are running on x86/x86_64)? We have a system lockup for long time. http://lxr.linux.no/#linux+v2.6.37/Documentation/nmi_watchdog.txt Could someone please clarify? > > Dave Hylands -- Thanks, Nilesh ^ permalink raw reply [flat|nested] 15+ messages in thread
* spin_lock and scheduler confusion 2011-01-07 6:27 ` Tayade, Nilesh @ 2011-01-07 7:02 ` Tirtha Ghosh 2011-01-07 7:35 ` Rajat Sharma 2011-01-07 15:51 ` Dave Hylands 1 sibling, 1 reply; 15+ messages in thread From: Tirtha Ghosh @ 2011-01-07 7:02 UTC (permalink / raw) To: kernelnewbies NMI has greater priority over spinlock, cause this is non maskable and NMI watchdog can be used for debugging spinlock deadlocks (CONFIG_DEBUG_SPINLOCK). So we will hit NMI watchdog even if spinlock is acquired. On Fri, Jan 7, 2011 at 11:57 AM, Tayade, Nilesh <Nilesh.Tayade@netscout.com>wrote: > Hi, > > > -----Original Message----- > > From: kernelnewbies-bounces at kernelnewbies.org [mailto:kernelnewbies- > > bounces at kernelnewbies.org] On Behalf Of Dave Hylands > > Sent: Friday, January 07, 2011 10:59 AM > > To: Viral Mehta > > Cc: kernelnewbies at kernelnewbies.org > > Subject: Re: spin_lock and scheduler confusion > > > > Hi Viral, > > > > On Wed, Jan 5, 2011 at 2:23 PM, Viral Mehta > > <Viral.Mehta@lntinfotech.com> wrote: > > > > > > Hi , > > > > > > I need your help to solve below confusion. > > > > [...] > > > > Note that you can't sleep while you hold a spinlock. You're not > > allowed to perform any type of blocking operations. If you're holding > > the spinlock for any significant length of time, then you're using the > > wrong design. > > > > > spin_lock_irqrestore(); > > > 3. One of the CPU core tries to execute this code and so acquires the > > lock. > > > 4. Now, second core is also goes to execute same piece of code and so > > will > [...] > > > > Not while it's holding the spinlock or waiting for the spinlock. > > > > > Ever if timeslice is over for the current task ? > > > > The time tick interrupt is what determines when the timeslice is over. > > Since you have interrupts disabled, the timer interrupt can't happen. > > > > > What if scheduler code is running on CPU core-3 and sees that > > > timeslice for task running on CPU core-2 has expired ? > > > > Each core only considers the timeslices for its own core. > > > > > I guess timeslice expire case is not as same as preemption. Or may be > > I am > > > terribly wrong. > > > > You shouldn't be holding a spinlock for periods of time approaching > > the length of a timeslice. The timer interrupt is what determines the > > end of a timeslice. No timer interrupt, no end of a timeslice. > > Preemption is also triggered by the timer interrupt, or by releasing a > > resource that a higher priority task is waiting for. > > May be my understanding is incorrect, but wouldn't we hit the NMI watchdog > here(assuming we are running on x86/x86_64)? > We have a system lockup for long time. > http://lxr.linux.no/#linux+v2.6.37/Documentation/nmi_watchdog.txt > > Could someone please clarify? > > > > Dave Hylands > > -- > Thanks, > Nilesh > > _______________________________________________ > Kernelnewbies mailing list > Kernelnewbies at kernelnewbies.org > http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110107/0e313cbf/attachment.html ^ permalink raw reply [flat|nested] 15+ messages in thread
* spin_lock and scheduler confusion 2011-01-07 7:02 ` Tirtha Ghosh @ 2011-01-07 7:35 ` Rajat Sharma 2011-01-07 7:49 ` nilesh 2011-01-07 15:49 ` Dave Hylands 0 siblings, 2 replies; 15+ messages in thread From: Rajat Sharma @ 2011-01-07 7:35 UTC (permalink / raw) To: kernelnewbies As I remember timer interrupt as well is an NMI so, it is possible (although not advised) to call schedule function while holding spinlock on same core. spin_lock_irqsave(); schedule(); spin_lock_irqrestore(); however if you have debugging options turned on like CONFIG_DEBUG_SPINLOCK, you may likely get kernel warning for 'scheduling in atomic context'. Then what can happen if this core is allowed to switched to new process? Consider the case where new process as well tries to aquire same spin_lock() which new process can not aquire and start spinning for the lock for ever :). Likewise, other cores will also get locked down. However stil you can detect softlockup through NMI watchdog. Rajat On Fri, Jan 7, 2011 at 12:32 PM, Tirtha Ghosh <gtirtha@gmail.com> wrote: > NMI has greater priority over spinlock, cause this is non maskable and NMI > watchdog can be used for debugging spinlock deadlocks > (CONFIG_DEBUG_SPINLOCK). > > So we will hit NMI watchdog even if spinlock is acquired. > > > > On Fri, Jan 7, 2011 at 11:57 AM, Tayade, Nilesh < > Nilesh.Tayade at netscout.com> wrote: > >> Hi, >> >> > -----Original Message----- >> > From: kernelnewbies-bounces at kernelnewbies.org [mailto:kernelnewbies- >> > bounces at kernelnewbies.org] On Behalf Of Dave Hylands >> > Sent: Friday, January 07, 2011 10:59 AM >> > To: Viral Mehta >> > Cc: kernelnewbies at kernelnewbies.org >> > Subject: Re: spin_lock and scheduler confusion >> > >> > Hi Viral, >> > >> > On Wed, Jan 5, 2011 at 2:23 PM, Viral Mehta >> > <Viral.Mehta@lntinfotech.com> wrote: >> > > >> > > Hi , >> > > >> > > I need your help to solve below confusion. >> > > >> [...] >> > >> > Note that you can't sleep while you hold a spinlock. You're not >> > allowed to perform any type of blocking operations. If you're holding >> > the spinlock for any significant length of time, then you're using the >> > wrong design. >> > >> > > spin_lock_irqrestore(); >> > > 3. One of the CPU core tries to execute this code and so acquires the >> > lock. >> > > 4. Now, second core is also goes to execute same piece of code and so >> > will >> [...] >> > >> > Not while it's holding the spinlock or waiting for the spinlock. >> > >> > > Ever if timeslice is over for the current task ? >> > >> > The time tick interrupt is what determines when the timeslice is over. >> > Since you have interrupts disabled, the timer interrupt can't happen. >> > >> > > What if scheduler code is running on CPU core-3 and sees that >> > > timeslice for task running on CPU core-2 has expired ? >> > >> > Each core only considers the timeslices for its own core. >> > >> > > I guess timeslice expire case is not as same as preemption. Or may be >> > I am >> > > terribly wrong. >> > >> > You shouldn't be holding a spinlock for periods of time approaching >> > the length of a timeslice. The timer interrupt is what determines the >> > end of a timeslice. No timer interrupt, no end of a timeslice. >> > Preemption is also triggered by the timer interrupt, or by releasing a >> > resource that a higher priority task is waiting for. >> >> May be my understanding is incorrect, but wouldn't we hit the NMI watchdog >> here(assuming we are running on x86/x86_64)? >> We have a system lockup for long time. >> http://lxr.linux.no/#linux+v2.6.37/Documentation/nmi_watchdog.txt >> >> Could someone please clarify? >> > >> > Dave Hylands >> >> -- >> Thanks, >> Nilesh >> >> _______________________________________________ >> Kernelnewbies mailing list >> Kernelnewbies at kernelnewbies.org >> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies >> > > > _______________________________________________ > Kernelnewbies mailing list > Kernelnewbies at kernelnewbies.org > http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110107/41aaf7b1/attachment-0001.html ^ permalink raw reply [flat|nested] 15+ messages in thread
* spin_lock and scheduler confusion 2011-01-07 7:35 ` Rajat Sharma @ 2011-01-07 7:49 ` nilesh 2011-01-07 8:03 ` anish singh 2011-01-07 15:49 ` Dave Hylands 1 sibling, 1 reply; 15+ messages in thread From: nilesh @ 2011-01-07 7:49 UTC (permalink / raw) To: kernelnewbies On Fri, 2011-01-07 at 13:05 +0530, Rajat Sharma wrote: > As I remember timer interrupt as well is an NMI so, it is possible > (although not advised) to call schedule function while holding > spinlock on same core. > > spin_lock_irqsave(); > schedule(); > spin_lock_irqrestore(); > > however if you have debugging options turned on like > CONFIG_DEBUG_SPINLOCK, you may likely get kernel warning for > 'scheduling in atomic context'. > > Then what can happen if this core is allowed to switched to new > process? Consider the case where new process as well tries to aquire > same spin_lock() which new process can not aquire and start spinning > for the lock for ever :). Likewise, other cores will also get locked > down. > > However stil you can detect softlockup through NMI watchdog. Sorry if I am building up the confusion here. But as Dave Hylands initially mentioned, there will be no timer interrupt. So shouldn't the NMI watchdog get triggered then? No interrupts -> system freeze -> NMI Wdt reboot. http://slacksite.com/slackware/nmi.html > > Rajat -- Thanks, Nilesh ^ permalink raw reply [flat|nested] 15+ messages in thread
* spin_lock and scheduler confusion 2011-01-07 7:49 ` nilesh @ 2011-01-07 8:03 ` anish singh 2011-01-07 9:31 ` nilesh 0 siblings, 1 reply; 15+ messages in thread From: anish singh @ 2011-01-07 8:03 UTC (permalink / raw) To: kernelnewbies On Fri, Jan 7, 2011 at 1:19 PM, nilesh <nilesh.tayade@netscout.com> wrote: > On Fri, 2011-01-07 at 13:05 +0530, Rajat Sharma wrote: > > As I remember timer interrupt as well is an NMI so, it is possible > > (although not advised) to call schedule function while holding > > spinlock on same core. > > > > spin_lock_irqsave(); > > schedule(); > > spin_lock_irqrestore(); > > > > however if you have debugging options turned on like > > CONFIG_DEBUG_SPINLOCK, you may likely get kernel warning for > > 'scheduling in atomic context'. > > > > Then what can happen if this core is allowed to switched to new > > process? Consider the case where new process as well tries to aquire > > same spin_lock() which new process can not aquire and start spinning > > for the lock for ever :). Likewise, other cores will also get locked > > down. > > > > However stil you can detect softlockup through NMI watchdog. > > >>Sorry if I am building up the confusion here. But as Dave Hylands > >>initially mentioned, there will be no timer interrupt. So shouldn't the > >>NMI watchdog get triggered then? No interrupts -> system freeze -> NMI > >>Wdt reboot. > In my opinion(uninformed ) NMI watchdog will be triggered only in case where you are holding a spinlock.It will not be triggered just because timer interrupts are disabled due to holding a spinlock. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110107/bbae4309/attachment.html ^ permalink raw reply [flat|nested] 15+ messages in thread
* spin_lock and scheduler confusion 2011-01-07 8:03 ` anish singh @ 2011-01-07 9:31 ` nilesh 2011-01-07 13:58 ` Rajat Sharma 0 siblings, 1 reply; 15+ messages in thread From: nilesh @ 2011-01-07 9:31 UTC (permalink / raw) To: kernelnewbies On Fri, 2011-01-07 at 13:33 +0530, anish singh wrote: > > > On Fri, Jan 7, 2011 at 1:19 PM, nilesh <nilesh.tayade@netscout.com> > wrote: > On Fri, 2011-01-07 at 13:05 +0530, Rajat Sharma wrote: > > As I remember timer interrupt as well is an NMI so, it is > possible > > (although not advised) to call schedule function while > holding > > spinlock on same core. > > > > spin_lock_irqsave(); > > schedule(); > > spin_lock_irqrestore(); > > > > however if you have debugging options turned on like > > CONFIG_DEBUG_SPINLOCK, you may likely get kernel warning for > > 'scheduling in atomic context'. > > > > Then what can happen if this core is allowed to switched to > new > > process? Consider the case where new process as well tries > to aquire > > same spin_lock() which new process can not aquire and start > spinning > > for the lock for ever :). Likewise, other cores will also > get locked > > down. > > > > However stil you can detect softlockup through NMI watchdog. > > > >>Sorry if I am building up the confusion here. But as Dave > Hylands > >>initially mentioned, there will be no timer interrupt. So > shouldn't the > >>NMI watchdog get triggered then? No interrupts -> system > freeze -> NMI > >>Wdt reboot. > > In my opinion(uninformed ) NMI watchdog will be triggered only in case > where you are holding a spinlock.It will not be triggered just because > timer interrupts are disabled due to holding a spinlock. > No, what I meant is - we have masked all the interrupts before holding the spinlock (and not even getting the timer interrupts) so it's as good as system freeze. And we should trigger the NMI watchdog, isn't it? -- Thanks, Nilesh ^ permalink raw reply [flat|nested] 15+ messages in thread
* spin_lock and scheduler confusion 2011-01-07 9:31 ` nilesh @ 2011-01-07 13:58 ` Rajat Sharma 0 siblings, 0 replies; 15+ messages in thread From: Rajat Sharma @ 2011-01-07 13:58 UTC (permalink / raw) To: kernelnewbies > As I remember timer interrupt as well is an NMI so, it is > possible (although not advised) to call schedule function while > holding spinlock on same core. > > spin_lock_irqsave(); > schedule(); > spin_lock_irqrestore(); sorry for confusion created by my last mail, looks like its only watchdog timer which are NMI and above function call to schedule() does not require any help from scheduler's timer interrupts (not sure if same timer hardware is used for watchdog and scheduler timer interrupts, my guess is both may share the hardware, but different IRQL line, one maskable other non-maskable). However it is still possible to voluntarily schedule the CPU core by currently executing code even with spin_locks disabled. Rajat On Fri, Jan 7, 2011 at 3:01 PM, nilesh <nilesh.tayade@netscout.com> wrote: > On Fri, 2011-01-07 at 13:33 +0530, anish singh wrote: > > > > > > On Fri, Jan 7, 2011 at 1:19 PM, nilesh <nilesh.tayade@netscout.com> > > wrote: > > On Fri, 2011-01-07 at 13:05 +0530, Rajat Sharma wrote: > > > As I remember timer interrupt as well is an NMI so, it is > > possible > > > (although not advised) to call schedule function while > > holding > > > spinlock on same core. > > > > > > spin_lock_irqsave(); > > > schedule(); > > > spin_lock_irqrestore(); > > > > > > however if you have debugging options turned on like > > > CONFIG_DEBUG_SPINLOCK, you may likely get kernel warning for > > > 'scheduling in atomic context'. > > > > > > Then what can happen if this core is allowed to switched to > > new > > > process? Consider the case where new process as well tries > > to aquire > > > same spin_lock() which new process can not aquire and start > > spinning > > > for the lock for ever :). Likewise, other cores will also > > get locked > > > down. > > > > > > However stil you can detect softlockup through NMI watchdog. > > > > > > >>Sorry if I am building up the confusion here. But as Dave > > Hylands > > >>initially mentioned, there will be no timer interrupt. So > > shouldn't the > > >>NMI watchdog get triggered then? No interrupts -> system > > freeze -> NMI > > >>Wdt reboot. > > > > In my opinion(uninformed ) NMI watchdog will be triggered only in case > > where you are holding a spinlock.It will not be triggered just because > > timer interrupts are disabled due to holding a spinlock. > > > No, what I meant is - we have masked all the interrupts before holding > the spinlock (and not even getting the timer interrupts) so it's as good > as system freeze. And we should trigger the NMI watchdog, isn't it? > > -- > Thanks, > Nilesh > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110107/112a60ea/attachment-0001.html ^ permalink raw reply [flat|nested] 15+ messages in thread
* spin_lock and scheduler confusion 2011-01-07 7:35 ` Rajat Sharma 2011-01-07 7:49 ` nilesh @ 2011-01-07 15:49 ` Dave Hylands 1 sibling, 0 replies; 15+ messages in thread From: Dave Hylands @ 2011-01-07 15:49 UTC (permalink / raw) To: kernelnewbies Hi Rajat, On Thu, Jan 6, 2011 at 11:35 PM, Rajat Sharma <fs.rajat@gmail.com> wrote: > As I remember timer interrupt as well is an NMI so, it is possible (although > not advised) to call schedule function while holding spinlock on same core. > > spin_lock_irqsave(); > schedule(); > spin_lock_irqrestore(); You're not supposed to call schedule with interrupts disbled. If you follow the code though schedule, you'll see that it calls schedule_debug, which in turns checks to see if its being called from an atomic context and if it is, it will cause the BUG: scheduling while atomic: message along with a stack dump. > > however if you have debugging options turned on like CONFIG_DEBUG_SPINLOCK, > you may likely get kernel warning for 'scheduling in atomic context'. > > Then what can happen if this core is allowed to switched to new process? > Consider the case where new process as well tries to aquire same spin_lock() > which new process can not aquire and start spinning for the lock for ever > :). Likewise, other cores will also get locked down. > > However stil you can detect softlockup through NMI watchdog. > > Rajat > > On Fri, Jan 7, 2011 at 12:32 PM, Tirtha Ghosh <gtirtha@gmail.com> wrote: >> >> NMI has greater priority over spinlock, cause this is non maskable and NMI >> watchdog can be used for debugging spinlock deadlocks >> (CONFIG_DEBUG_SPINLOCK). >> >> So we will hit NMI watchdog even if spinlock is acquired. >> >> >> >> On Fri, Jan 7, 2011 at 11:57 AM, Tayade, Nilesh >> <Nilesh.Tayade@netscout.com> wrote: >>> >>> Hi, >>> >>> > -----Original Message----- >>> > From: kernelnewbies-bounces at kernelnewbies.org [mailto:kernelnewbies- >>> > bounces at kernelnewbies.org] On Behalf Of Dave Hylands >>> > Sent: Friday, January 07, 2011 10:59 AM >>> > To: Viral Mehta >>> > Cc: kernelnewbies at kernelnewbies.org >>> > Subject: Re: spin_lock and scheduler confusion >>> > >>> > Hi Viral, >>> > >>> > On Wed, Jan 5, 2011 at 2:23 PM, Viral Mehta >>> > <Viral.Mehta@lntinfotech.com> wrote: >>> > > >>> > > Hi , >>> > > >>> > > I need your help to solve below confusion. >>> > > >>> [...] >>> > >>> > Note that you can't sleep while you hold a spinlock. You're not >>> > allowed to perform any type of blocking operations. If you're holding >>> > the spinlock for any significant length of time, then you're using the >>> > wrong design. >>> > >>> > > ??? spin_lock_irqrestore(); >>> > > 3. One of the CPU core tries to execute this code and so acquires the >>> > lock. >>> > > 4. Now, second core is also goes to execute same piece of code and so >>> > will >>> [...] >>> > >>> > Not while it's holding the spinlock or waiting for the spinlock. >>> > >>> > > Ever if?timeslice is over for the current task ? >>> > >>> > The time tick interrupt is what determines when the timeslice is over. >>> > Since you have interrupts disabled, the timer interrupt can't happen. >>> > >>> > > What if scheduler code is running on CPU core-3 and sees that >>> > > timeslice?for task running on CPU core-2 has expired ? >>> > >>> > Each core only considers the timeslices for its own core. >>> > >>> > > I guess?timeslice expire case is not as same as preemption. Or may be >>> > I am >>> > > terribly wrong. >>> > >>> > You shouldn't be holding ?a spinlock for periods of time approaching >>> > the length of a timeslice. The timer interrupt is what determines the >>> > end of a timeslice. No timer interrupt, no end of a timeslice. >>> > Preemption is also triggered by the timer interrupt, or by releasing a >>> > resource that a higher priority task is waiting for. >>> >>> May be my understanding is incorrect, but wouldn't we hit the NMI >>> watchdog here(assuming we are running on x86/x86_64)? >>> We have a system lockup for long time. >>> http://lxr.linux.no/#linux+v2.6.37/Documentation/nmi_watchdog.txt >>> >>> Could someone please clarify? >>> > >>> > Dave Hylands >>> >>> -- >>> Thanks, >>> Nilesh >>> >>> _______________________________________________ >>> Kernelnewbies mailing list >>> Kernelnewbies at kernelnewbies.org >>> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies >> >> >> _______________________________________________ >> Kernelnewbies mailing list >> Kernelnewbies at kernelnewbies.org >> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies >> > > ^ permalink raw reply [flat|nested] 15+ messages in thread
* spin_lock and scheduler confusion 2011-01-07 6:27 ` Tayade, Nilesh 2011-01-07 7:02 ` Tirtha Ghosh @ 2011-01-07 15:51 ` Dave Hylands 1 sibling, 0 replies; 15+ messages in thread From: Dave Hylands @ 2011-01-07 15:51 UTC (permalink / raw) To: kernelnewbies Hi Nilesh. Using reply all this time... On Thu, Jan 6, 2011 at 10:27 PM, Tayade, Nilesh <Nilesh.Tayade@netscout.com> wrote: ...snip... >> You shouldn't be holding a spinlock for periods of time approaching >> the length of a timeslice. The timer interrupt is what determines the >> end of a timeslice. No timer interrupt, no end of a timeslice. >> Preemption is also triggered by the timer interrupt, or by releasing a >> resource that a higher priority task is waiting for. > > May be my understanding is incorrect, but wouldn't we hit the NMI watchdog here(assuming we are running on x86/x86_64)? > We have a system lockup for long time. > http://lxr.linux.no/#linux+v2.6.37/Documentation/nmi_watchdog.txt > > Could someone please clarify? Did you enable it by passing in nmi_watchdog=1 on the kernel command line? (and verify that it was working fine - as described in the nmi__watchdog.txt file) Dave Hylands ^ permalink raw reply [flat|nested] 15+ messages in thread
* spin_lock and scheduler confusion 2011-01-07 5:28 ` Dave Hylands 2011-01-07 6:27 ` Tayade, Nilesh @ 2011-01-07 17:46 ` Viral Mehta 2011-01-08 10:58 ` Rajat Sharma 1 sibling, 1 reply; 15+ messages in thread From: Viral Mehta @ 2011-01-07 17:46 UTC (permalink / raw) To: kernelnewbies Hi, ________________________________________ From: Dave Hylands [dhylands at gmail.com] Subject: Re: spin_lock and scheduler confusion >> I guess timeslice expire case is not as same as preemption. Or may be I am >> terribly wrong. > >You shouldn't be holding a spinlock for periods of time approaching >the length of a timeslice. The timer interrupt is what determines the >end of a timeslice. No timer interrupt, no end of a timeslice. >Preemption is also triggered by the timer interrupt, or by releasing a >resource that a higher priority task is waiting for. thanks, it is much clear now. >Dave Hylands Thanks, Viral http://groups.google.com/group/fundamental-discussion?hl=en The contents of this e-mail and any attachment(s) may contain confidential or privileged information for the intended recipient(s). Unintended recipients are prohibited from taking action on the basis of information in this e-mail and using or disseminating the information, and must notify the sender and delete it from their system. L&T Infotech will not accept responsibility or liability for the accuracy or completeness of, or the presence of any virus or disabling code in this e-mail" ______________________________________________________________________ ^ permalink raw reply [flat|nested] 15+ messages in thread
* spin_lock and scheduler confusion 2011-01-07 17:46 ` Viral Mehta @ 2011-01-08 10:58 ` Rajat Sharma 0 siblings, 0 replies; 15+ messages in thread From: Rajat Sharma @ 2011-01-08 10:58 UTC (permalink / raw) To: kernelnewbies Hi Dave, > > spin_lock_irqsave(); > > schedule(); > > spin_lock_irqrestore(); > > You're not supposed to call schedule with interrupts disbled. > > If you follow the code though schedule, you'll see that it calls > schedule_debug, which in turns checks to see if its being called from > an atomic context and if it is, it will cause the > > BUG: scheduling while atomic: Please refer to my first mail in this thread, I have already mentioned about scheduling while atomic in my first mail: > however if you have debugging options turned on like CONFIG_DEBUG_SPINLOCK, you may likely get kernel warning for 'scheduling > in atomic context'. And also its just a warning print with stack dump but not the panic() or BUG(), so system is still responsive. Its anyways not a good practice, but what I was trying to highlight is what can happen if we call schedule() function while holding spin_lock_irqsave. To kill the curiosity, I tried out following module: #include <linux/module.h> #include <linux/kernel.h> #include <linux/spinlock.h> #include <linux/sched.h> MODULE_LICENSE("GPL"); static DEFINE_SPINLOCK(sleepy_lock); static int __init sleepy_init(void) { unsigned long flags; printk("SLEEPTEST: loading sleepytest\n"); spin_lock_irqsave(&sleepy_lock, flags); schedule(); spin_unlock_irqrestore(&sleepy_lock, flags); return 0; } static void __exit sleepy_exit(void) { printk("SLEEPTEST: unloading sleepytest\n"); } module_init(sleepy_init); module_exit(sleepy_exit); And on my system, module silently prints messages without any atomic schedule warning: [ 6850.499940] SLEEPTEST: loading sleepytest [ 6871.822539] SLEEPTEST: unloading sleepytest I am not sure what config option turns on __schedule_bug(), but I didn't see this message on my system: CONFIG_SCHED_DEBUG=y # CONFIG_DEBUG_SPINLOCK is not set # CONFIG_DEBUG_SPINLOCK_SLEEP is not set Please try this module and let us know if you can hit schedule_bug(). Rajat On Fri, Jan 7, 2011 at 11:16 PM, Viral Mehta <Viral.Mehta@lntinfotech.com> wrote: > Hi, > ________________________________________ > From: Dave Hylands [dhylands at gmail.com] > Subject: Re: spin_lock and scheduler confusion > >>> I guess timeslice expire case is not as same as preemption. Or may be I am >>> terribly wrong. >> >>You shouldn't be holding ?a spinlock for periods of time approaching >>the length of a timeslice. The timer interrupt is what determines the >>end of a timeslice. No timer interrupt, no end of a timeslice. >>Preemption is also triggered by the timer interrupt, or by releasing a >>resource that a higher priority task is waiting for. > > thanks, > it is much clear now. > >>Dave Hylands > > Thanks, > Viral > > http://groups.google.com/group/fundamental-discussion?hl=en > > The contents of this e-mail and any attachment(s) may contain confidential or privileged information for the intended recipient(s). Unintended recipients are prohibited from taking action on the basis of information in this e-mail and ?using or disseminating the information, ?and must notify the sender and delete it from their system. L&T Infotech will not accept responsibility or liability for the accuracy or completeness of, or the presence of any virus or disabling code in this e-mail" > > ______________________________________________________________________ > > _______________________________________________ > Kernelnewbies mailing list > Kernelnewbies at kernelnewbies.org > http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies > ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2011-01-08 10:58 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <D69C90565D53114396BF743585AF5A09122E61E90F@VSHINMSMBX01.vshodc.lntinfotech.com>
2011-01-05 22:23 ` spin_lock and scheduler confusion Viral Mehta
2011-01-07 3:51 ` Alexandre Courbot
2011-01-07 3:56 ` Mulyadi Santosa
2011-01-07 5:28 ` Dave Hylands
2011-01-07 6:27 ` Tayade, Nilesh
2011-01-07 7:02 ` Tirtha Ghosh
2011-01-07 7:35 ` Rajat Sharma
2011-01-07 7:49 ` nilesh
2011-01-07 8:03 ` anish singh
2011-01-07 9:31 ` nilesh
2011-01-07 13:58 ` Rajat Sharma
2011-01-07 15:49 ` Dave Hylands
2011-01-07 15:51 ` Dave Hylands
2011-01-07 17:46 ` Viral Mehta
2011-01-08 10:58 ` Rajat Sharma
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).