spin_lock and scheduler confusion

spin_lock and scheduler confusion

[Viral Mehta]

Hi ,

I need your help to solve below confusion.

1. I know that spin_lock_irqsave, disables premption and local_irq.
2. There is some below piece of kernel code running on Quad core Machine.
    spin_lock_irqsave(&lock, flags);
    some_large_critical_section //sleep of 5 minutes
    //I know there should be as small as possible task in critical section
    // treat this as an example only
    spin_lock_irqrestore();
3. One of the CPU core tries to execute this code and so acquires the lock.
4. Now, second core is also goes to execute same piece of code and so will keep spinning for lock

Now, the question is
Will EVER second CPU core get chance to execute another task ?
Ever if timeslice is over for the current task ?

What if scheduler code is running on CPU core-3 and sees that
timeslice for task running on CPU core-2 has expired ?

I guess timeslice expire case is not as same as preemption. Or may be I am terribly wrong.

Please help,
Thanks,
Viral


________________________________
The contents of this e-mail and any attachment(s) may contain confidential or privileged information for the intended recipient(s). Unintended recipients are prohibited from taking action on the basis of information in this e-mail and using or disseminating the information, and must notify the sender and delete it from their system. L&T Infotech will not accept responsibility or liability for the accuracy or completeness of, or the presence of any virus or disabling code in this e-mail"

______________________________________________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110106/2de907f8/attachment-0001.html 

spin_lock and scheduler confusion

[Alexandre Courbot]

Hi,

> 2. There is?some below piece of kernel?code running on Quad core Machine.
> ??? spin_lock_irqsave(&lock, flags);
> ????some_large_critical_section //sleep of 5 minutes
> ??? //I know there should be as small as possible task in critical section
> ??? // treat this as an example only
> ??? spin_lock_irqrestore();
> 3. One of the CPU core tries to execute this code and so acquires the lock.
> 4. Now, second core is also goes to execute same piece of code and so will
> keep spinning for lock
>
> Now, the question is
> Will?EVER?second CPU core get chance to execute another task ?
> Ever if?timeslice is over for the current task ?

As far as I understand (from quickly reading
include/linux/spinlock*.h), kernel preemption is disabled as long as a
spin lock is used (and in your case, as soon as spin_lock_irqsave is
entered). This means that a core that started acquiring a spinlock
will not execute another task not only until the lock is acquired, but
until the lock is released.

> What if scheduler code is running on CPU core-3 and sees that
> timeslice?for task running on CPU core-2 has expired ?

AFAIK (that is, not very far) the scheduler code only cares for the
current core and does not assign tasks to others. But even if it did,
it would not have any way to stop the second core until it releases
the spin lock. So in any case, both core 1 and core 2 will be blocked
until they release the lock.

Alex.

spin_lock and scheduler confusion

[Mulyadi Santosa]

Dear Viral...

On Thu, Jan 6, 2011 at 05:23, Viral Mehta <Viral.Mehta@lntinfotech.com> wrote:
>
> Hi ,
>
> I need your help to solve below confusion.
>
> 1. I know that spin_lock_irqsave, disables?premption and local_irq.
> 2. There is?some below piece of kernel?code running on Quad core Machine.
> ??? spin_lock_irqsave(&lock, flags);
> ????some_large_critical_section //sleep of 5 minutes
> ??? //I know there should be as small as possible task in critical section
> ??? // treat this as an example only
> ??? spin_lock_irqrestore();
> 3. One of the CPU core tries to execute this code and so acquires the lock.
> 4. Now, second core is also goes to execute same piece of code and so will
> keep spinning for lock
>
> Now, the question is
> Will?EVER?second CPU core get chance to execute another task ?
> Ever if?timeslice is over for the current task ?


If interrupt is also disabled, then I think it's a game
over...preemption won't happen on that CPU.....i mean, not system
triggered preemption...the process must voluntarily yield itself, thus
giving others time to run.

This, assuming HPET signaling also based on interrupt...hence affected
by irq disabling...

-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

spin_lock and scheduler confusion

[Dave Hylands]

Hi Viral,

On Wed, Jan 5, 2011 at 2:23 PM, Viral Mehta <Viral.Mehta@lntinfotech.com> wrote:
>
> Hi ,
>
> I need your help to solve below confusion.
>
> 1. I know that spin_lock_irqsave, disables?premption and local_irq.
> 2. There is?some below piece of kernel?code running on Quad core Machine.
> ??? spin_lock_irqsave(&lock, flags);
> ????some_large_critical_section //sleep of 5 minutes
> ??? //I know there should be as small as possible task in critical section
> ??? // treat this as an example only

Note that you can't sleep while you hold a spinlock. You're not
allowed to perform any type of blocking operations. If you're holding
the spinlock for any significant length of time, then you're using the
wrong design.

> ??? spin_lock_irqrestore();
> 3. One of the CPU core tries to execute this code and so acquires the lock.
> 4. Now, second core is also goes to execute same piece of code and so will
> keep spinning for lock
>
> Now, the question is
> Will?EVER?second CPU core get chance to execute another task ?

Not while it's holding the spinlock or waiting for the spinlock.

> Ever if?timeslice is over for the current task ?

The time tick interrupt is what determines when the timeslice is over.
Since you have interrupts disabled, the timer interrupt can't happen.

> What if scheduler code is running on CPU core-3 and sees that
> timeslice?for task running on CPU core-2 has expired ?

Each core only considers the timeslices for its own core.

> I guess?timeslice expire case is not as same as preemption. Or may be I am
> terribly wrong.

You shouldn't be holding  a spinlock for periods of time approaching
the length of a timeslice. The timer interrupt is what determines the
end of a timeslice. No timer interrupt, no end of a timeslice.
Preemption is also triggered by the timer interrupt, or by releasing a
resource that a higher priority task is waiting for.

Dave Hylands

spin_lock and scheduler confusion

[Tayade, Nilesh]

Hi,

> -----Original Message-----
> From: kernelnewbies-bounces at kernelnewbies.org [mailto:kernelnewbies-
> bounces at kernelnewbies.org] On Behalf Of Dave Hylands
> Sent: Friday, January 07, 2011 10:59 AM
> To: Viral Mehta
> Cc: kernelnewbies at kernelnewbies.org
> Subject: Re: spin_lock and scheduler confusion
> 
> Hi Viral,
> 
> On Wed, Jan 5, 2011 at 2:23 PM, Viral Mehta
> <Viral.Mehta@lntinfotech.com> wrote:
> >
> > Hi ,
> >
> > I need your help to solve below confusion.
> >
[...]
> 
> Note that you can't sleep while you hold a spinlock. You're not
> allowed to perform any type of blocking operations. If you're holding
> the spinlock for any significant length of time, then you're using the
> wrong design.
> 
> > ??? spin_lock_irqrestore();
> > 3. One of the CPU core tries to execute this code and so acquires the
> lock.
> > 4. Now, second core is also goes to execute same piece of code and so
> will
[...]
> 
> Not while it's holding the spinlock or waiting for the spinlock.
> 
> > Ever if?timeslice is over for the current task ?
> 
> The time tick interrupt is what determines when the timeslice is over.
> Since you have interrupts disabled, the timer interrupt can't happen.
> 
> > What if scheduler code is running on CPU core-3 and sees that
> > timeslice?for task running on CPU core-2 has expired ?
> 
> Each core only considers the timeslices for its own core.
> 
> > I guess?timeslice expire case is not as same as preemption. Or may be
> I am
> > terribly wrong.
> 
> You shouldn't be holding  a spinlock for periods of time approaching
> the length of a timeslice. The timer interrupt is what determines the
> end of a timeslice. No timer interrupt, no end of a timeslice.
> Preemption is also triggered by the timer interrupt, or by releasing a
> resource that a higher priority task is waiting for.

May be my understanding is incorrect, but wouldn't we hit the NMI watchdog here(assuming we are running on x86/x86_64)? 
We have a system lockup for long time.
http://lxr.linux.no/#linux+v2.6.37/Documentation/nmi_watchdog.txt

Could someone please clarify?
> 
> Dave Hylands

--
Thanks,
Nilesh

spin_lock and scheduler confusion

[Tirtha Ghosh]

NMI has greater priority over spinlock, cause this is non maskable and NMI
watchdog can be used for debugging spinlock deadlocks
(CONFIG_DEBUG_SPINLOCK).

So we will hit NMI watchdog even if spinlock is acquired.



On Fri, Jan 7, 2011 at 11:57 AM, Tayade, Nilesh
<Nilesh.Tayade@netscout.com>wrote:

> Hi,
>
> > -----Original Message-----
> > From: kernelnewbies-bounces at kernelnewbies.org [mailto:kernelnewbies-
> > bounces at kernelnewbies.org] On Behalf Of Dave Hylands
> > Sent: Friday, January 07, 2011 10:59 AM
> > To: Viral Mehta
> > Cc: kernelnewbies at kernelnewbies.org
> > Subject: Re: spin_lock and scheduler confusion
> >
> > Hi Viral,
> >
> > On Wed, Jan 5, 2011 at 2:23 PM, Viral Mehta
> > <Viral.Mehta@lntinfotech.com> wrote:
> > >
> > > Hi ,
> > >
> > > I need your help to solve below confusion.
> > >
> [...]
> >
> > Note that you can't sleep while you hold a spinlock. You're not
> > allowed to perform any type of blocking operations. If you're holding
> > the spinlock for any significant length of time, then you're using the
> > wrong design.
> >
> > >     spin_lock_irqrestore();
> > > 3. One of the CPU core tries to execute this code and so acquires the
> > lock.
> > > 4. Now, second core is also goes to execute same piece of code and so
> > will
> [...]
> >
> > Not while it's holding the spinlock or waiting for the spinlock.
> >
> > > Ever if timeslice is over for the current task ?
> >
> > The time tick interrupt is what determines when the timeslice is over.
> > Since you have interrupts disabled, the timer interrupt can't happen.
> >
> > > What if scheduler code is running on CPU core-3 and sees that
> > > timeslice for task running on CPU core-2 has expired ?
> >
> > Each core only considers the timeslices for its own core.
> >
> > > I guess timeslice expire case is not as same as preemption. Or may be
> > I am
> > > terribly wrong.
> >
> > You shouldn't be holding  a spinlock for periods of time approaching
> > the length of a timeslice. The timer interrupt is what determines the
> > end of a timeslice. No timer interrupt, no end of a timeslice.
> > Preemption is also triggered by the timer interrupt, or by releasing a
> > resource that a higher priority task is waiting for.
>
> May be my understanding is incorrect, but wouldn't we hit the NMI watchdog
> here(assuming we are running on x86/x86_64)?
> We have a system lockup for long time.
> http://lxr.linux.no/#linux+v2.6.37/Documentation/nmi_watchdog.txt
>
> Could someone please clarify?
> >
> > Dave Hylands
>
> --
> Thanks,
> Nilesh
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110107/0e313cbf/attachment.html 

spin_lock and scheduler confusion

[Rajat Sharma]

As I remember timer interrupt as well is an NMI so, it is possible (although
not advised) to call schedule function while holding spinlock on same core.

spin_lock_irqsave();
schedule();
spin_lock_irqrestore();

however if you have debugging options turned on like CONFIG_DEBUG_SPINLOCK,
you may likely get kernel warning for 'scheduling in atomic context'.

Then what can happen if this core is allowed to switched to new process?
Consider the case where new process as well tries to aquire same spin_lock()
which new process can not aquire and start spinning for the lock for ever
:). Likewise, other cores will also get locked down.

However stil you can detect softlockup through NMI watchdog.

Rajat

On Fri, Jan 7, 2011 at 12:32 PM, Tirtha Ghosh <gtirtha@gmail.com> wrote:

> NMI has greater priority over spinlock, cause this is non maskable and NMI
> watchdog can be used for debugging spinlock deadlocks
> (CONFIG_DEBUG_SPINLOCK).
>
> So we will hit NMI watchdog even if spinlock is acquired.
>
>
>
> On Fri, Jan 7, 2011 at 11:57 AM, Tayade, Nilesh <
> Nilesh.Tayade at netscout.com> wrote:
>
>> Hi,
>>
>> > -----Original Message-----
>> > From: kernelnewbies-bounces at kernelnewbies.org [mailto:kernelnewbies-
>> > bounces at kernelnewbies.org] On Behalf Of Dave Hylands
>> > Sent: Friday, January 07, 2011 10:59 AM
>> > To: Viral Mehta
>> > Cc: kernelnewbies at kernelnewbies.org
>> > Subject: Re: spin_lock and scheduler confusion
>> >
>> > Hi Viral,
>> >
>> > On Wed, Jan 5, 2011 at 2:23 PM, Viral Mehta
>> > <Viral.Mehta@lntinfotech.com> wrote:
>> > >
>> > > Hi ,
>> > >
>> > > I need your help to solve below confusion.
>> > >
>> [...]
>> >
>> > Note that you can't sleep while you hold a spinlock. You're not
>> > allowed to perform any type of blocking operations. If you're holding
>> > the spinlock for any significant length of time, then you're using the
>> > wrong design.
>> >
>> > >     spin_lock_irqrestore();
>> > > 3. One of the CPU core tries to execute this code and so acquires the
>> > lock.
>> > > 4. Now, second core is also goes to execute same piece of code and so
>> > will
>> [...]
>> >
>> > Not while it's holding the spinlock or waiting for the spinlock.
>> >
>> > > Ever if timeslice is over for the current task ?
>> >
>> > The time tick interrupt is what determines when the timeslice is over.
>> > Since you have interrupts disabled, the timer interrupt can't happen.
>> >
>> > > What if scheduler code is running on CPU core-3 and sees that
>> > > timeslice for task running on CPU core-2 has expired ?
>> >
>> > Each core only considers the timeslices for its own core.
>> >
>> > > I guess timeslice expire case is not as same as preemption. Or may be
>> > I am
>> > > terribly wrong.
>> >
>> > You shouldn't be holding  a spinlock for periods of time approaching
>> > the length of a timeslice. The timer interrupt is what determines the
>> > end of a timeslice. No timer interrupt, no end of a timeslice.
>> > Preemption is also triggered by the timer interrupt, or by releasing a
>> > resource that a higher priority task is waiting for.
>>
>> May be my understanding is incorrect, but wouldn't we hit the NMI watchdog
>> here(assuming we are running on x86/x86_64)?
>> We have a system lockup for long time.
>> http://lxr.linux.no/#linux+v2.6.37/Documentation/nmi_watchdog.txt
>>
>> Could someone please clarify?
>> >
>> > Dave Hylands
>>
>> --
>> Thanks,
>> Nilesh
>>
>> _______________________________________________
>> Kernelnewbies mailing list
>> Kernelnewbies at kernelnewbies.org
>> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>>
>
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110107/41aaf7b1/attachment-0001.html 

spin_lock and scheduler confusion

[nilesh]

On Fri, 2011-01-07 at 13:05 +0530, Rajat Sharma wrote:
> As I remember timer interrupt as well is an NMI so, it is possible
> (although not advised) to call schedule function while holding
> spinlock on same core.
> 
> spin_lock_irqsave();
> schedule();
> spin_lock_irqrestore();
> 
> however if you have debugging options turned on like
> CONFIG_DEBUG_SPINLOCK, you may likely get kernel warning for
> 'scheduling in atomic context'.
> 
> Then what can happen if this core is allowed to switched to new
> process? Consider the case where new process as well tries to aquire
> same spin_lock() which new process can not aquire and start spinning
> for the lock for ever :). Likewise, other cores will also get locked
> down.
> 
> However stil you can detect softlockup through NMI watchdog.

Sorry if I am building up the confusion here. But as Dave Hylands
initially mentioned, there will be no timer interrupt. So shouldn't the
NMI watchdog get triggered then? No interrupts -> system freeze -> NMI
Wdt reboot. 

http://slacksite.com/slackware/nmi.html

> 
> Rajat

-- 
Thanks,
Nilesh

spin_lock and scheduler confusion

[anish singh]

On Fri, Jan 7, 2011 at 1:19 PM, nilesh <nilesh.tayade@netscout.com> wrote:

> On Fri, 2011-01-07 at 13:05 +0530, Rajat Sharma wrote:
> > As I remember timer interrupt as well is an NMI so, it is possible
> > (although not advised) to call schedule function while holding
> > spinlock on same core.
> >
> > spin_lock_irqsave();
> > schedule();
> > spin_lock_irqrestore();
> >
> > however if you have debugging options turned on like
> > CONFIG_DEBUG_SPINLOCK, you may likely get kernel warning for
> > 'scheduling in atomic context'.
> >
> > Then what can happen if this core is allowed to switched to new
> > process? Consider the case where new process as well tries to aquire
> > same spin_lock() which new process can not aquire and start spinning
> > for the lock for ever :). Likewise, other cores will also get locked
> > down.
> >
> > However stil you can detect softlockup through NMI watchdog.
>
> >>Sorry if I am building up the confusion here. But as Dave Hylands
> >>initially mentioned, there will be no timer interrupt. So shouldn't the
> >>NMI watchdog get triggered then? No interrupts -> system freeze -> NMI
> >>Wdt reboot.
>

In my opinion(uninformed ) NMI watchdog will be triggered only in case where
you are holding a spinlock.It will not be triggered just because timer
interrupts are disabled due to holding a spinlock.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110107/bbae4309/attachment.html 

spin_lock and scheduler confusion

[nilesh]

On Fri, 2011-01-07 at 13:33 +0530, anish singh wrote:
> 
> 
> On Fri, Jan 7, 2011 at 1:19 PM, nilesh <nilesh.tayade@netscout.com>
> wrote:
>         On Fri, 2011-01-07 at 13:05 +0530, Rajat Sharma wrote:
>         > As I remember timer interrupt as well is an NMI so, it is
>         possible
>         > (although not advised) to call schedule function while
>         holding
>         > spinlock on same core.
>         >
>         > spin_lock_irqsave();
>         > schedule();
>         > spin_lock_irqrestore();
>         >
>         > however if you have debugging options turned on like
>         > CONFIG_DEBUG_SPINLOCK, you may likely get kernel warning for
>         > 'scheduling in atomic context'.
>         >
>         > Then what can happen if this core is allowed to switched to
>         new
>         > process? Consider the case where new process as well tries
>         to aquire
>         > same spin_lock() which new process can not aquire and start
>         spinning
>         > for the lock for ever :). Likewise, other cores will also
>         get locked
>         > down.
>         >
>         > However stil you can detect softlockup through NMI watchdog.
>         
>         
>         >>Sorry if I am building up the confusion here. But as Dave
>         Hylands
>         >>initially mentioned, there will be no timer interrupt. So
>         shouldn't the
>         >>NMI watchdog get triggered then? No interrupts -> system
>         freeze -> NMI
>         >>Wdt reboot.
> 
> In my opinion(uninformed ) NMI watchdog will be triggered only in case
> where you are holding a spinlock.It will not be triggered just because
> timer interrupts are disabled due to holding a spinlock.
> 
No, what I meant is - we have masked all the interrupts before holding
the spinlock (and not even getting the timer interrupts) so it's as good
as system freeze. And we should trigger the NMI watchdog, isn't it?

-- 
Thanks,
Nilesh

spin_lock and scheduler confusion

[Rajat Sharma]

> As I remember timer interrupt as well is an NMI so, it is
> possible (although not advised) to call schedule function while
> holding spinlock on same core.
>
> spin_lock_irqsave();
> schedule();
> spin_lock_irqrestore();

sorry for confusion created by my last mail, looks like its only watchdog
timer which are NMI and above function call to schedule() does not require
any help from scheduler's timer interrupts (not sure if same timer hardware
is used for watchdog and scheduler timer interrupts, my guess is both may
share the hardware, but different IRQL line, one maskable other
non-maskable). However it is still possible to voluntarily schedule the CPU
core by currently executing code even with spin_locks disabled.

Rajat

On Fri, Jan 7, 2011 at 3:01 PM, nilesh <nilesh.tayade@netscout.com> wrote:

> On Fri, 2011-01-07 at 13:33 +0530, anish singh wrote:
> >
> >
> > On Fri, Jan 7, 2011 at 1:19 PM, nilesh <nilesh.tayade@netscout.com>
> > wrote:
> >         On Fri, 2011-01-07 at 13:05 +0530, Rajat Sharma wrote:
> >         > As I remember timer interrupt as well is an NMI so, it is
> >         possible
> >         > (although not advised) to call schedule function while
> >         holding
> >         > spinlock on same core.
> >         >
> >         > spin_lock_irqsave();
> >         > schedule();
> >         > spin_lock_irqrestore();
> >         >
> >         > however if you have debugging options turned on like
> >         > CONFIG_DEBUG_SPINLOCK, you may likely get kernel warning for
> >         > 'scheduling in atomic context'.
> >         >
> >         > Then what can happen if this core is allowed to switched to
> >         new
> >         > process? Consider the case where new process as well tries
> >         to aquire
> >         > same spin_lock() which new process can not aquire and start
> >         spinning
> >         > for the lock for ever :). Likewise, other cores will also
> >         get locked
> >         > down.
> >         >
> >         > However stil you can detect softlockup through NMI watchdog.
> >
> >
> >         >>Sorry if I am building up the confusion here. But as Dave
> >         Hylands
> >         >>initially mentioned, there will be no timer interrupt. So
> >         shouldn't the
> >         >>NMI watchdog get triggered then? No interrupts -> system
> >         freeze -> NMI
> >         >>Wdt reboot.
> >
> > In my opinion(uninformed ) NMI watchdog will be triggered only in case
> > where you are holding a spinlock.It will not be triggered just because
> > timer interrupts are disabled due to holding a spinlock.
> >
> No, what I meant is - we have masked all the interrupts before holding
> the spinlock (and not even getting the timer interrupts) so it's as good
> as system freeze. And we should trigger the NMI watchdog, isn't it?
>
> --
> Thanks,
> Nilesh
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110107/112a60ea/attachment-0001.html 

spin_lock and scheduler confusion

[Dave Hylands]

Hi Rajat,

On Thu, Jan 6, 2011 at 11:35 PM, Rajat Sharma <fs.rajat@gmail.com> wrote:
> As I remember timer interrupt as well is an NMI so, it is possible (although
> not advised) to call schedule function while holding spinlock on same core.
>
> spin_lock_irqsave();
> schedule();
> spin_lock_irqrestore();

You're not supposed to call schedule with interrupts disbled.

If you follow the code though schedule, you'll see that it calls
schedule_debug, which in turns checks to see if its being called from
an atomic context and if it is, it will cause the

BUG: scheduling while atomic:

message along with a stack dump.

>
> however if you have debugging options turned on like CONFIG_DEBUG_SPINLOCK,
> you may likely get kernel warning for 'scheduling in atomic context'.
>
> Then what can happen if this core is allowed to switched to new process?
> Consider the case where new process as well tries to aquire same spin_lock()
> which new process can not aquire and start spinning for the lock for ever
> :). Likewise, other cores will also get locked down.
>
> However stil you can detect softlockup through NMI watchdog.
>
> Rajat
>
> On Fri, Jan 7, 2011 at 12:32 PM, Tirtha Ghosh <gtirtha@gmail.com> wrote:
>>
>> NMI has greater priority over spinlock, cause this is non maskable and NMI
>> watchdog can be used for debugging spinlock deadlocks
>> (CONFIG_DEBUG_SPINLOCK).
>>
>> So we will hit NMI watchdog even if spinlock is acquired.
>>
>>
>>
>> On Fri, Jan 7, 2011 at 11:57 AM, Tayade, Nilesh
>> <Nilesh.Tayade@netscout.com> wrote:
>>>
>>> Hi,
>>>
>>> > -----Original Message-----
>>> > From: kernelnewbies-bounces at kernelnewbies.org [mailto:kernelnewbies-
>>> > bounces at kernelnewbies.org] On Behalf Of Dave Hylands
>>> > Sent: Friday, January 07, 2011 10:59 AM
>>> > To: Viral Mehta
>>> > Cc: kernelnewbies at kernelnewbies.org
>>> > Subject: Re: spin_lock and scheduler confusion
>>> >
>>> > Hi Viral,
>>> >
>>> > On Wed, Jan 5, 2011 at 2:23 PM, Viral Mehta
>>> > <Viral.Mehta@lntinfotech.com> wrote:
>>> > >
>>> > > Hi ,
>>> > >
>>> > > I need your help to solve below confusion.
>>> > >
>>> [...]
>>> >
>>> > Note that you can't sleep while you hold a spinlock. You're not
>>> > allowed to perform any type of blocking operations. If you're holding
>>> > the spinlock for any significant length of time, then you're using the
>>> > wrong design.
>>> >
>>> > > ??? spin_lock_irqrestore();
>>> > > 3. One of the CPU core tries to execute this code and so acquires the
>>> > lock.
>>> > > 4. Now, second core is also goes to execute same piece of code and so
>>> > will
>>> [...]
>>> >
>>> > Not while it's holding the spinlock or waiting for the spinlock.
>>> >
>>> > > Ever if?timeslice is over for the current task ?
>>> >
>>> > The time tick interrupt is what determines when the timeslice is over.
>>> > Since you have interrupts disabled, the timer interrupt can't happen.
>>> >
>>> > > What if scheduler code is running on CPU core-3 and sees that
>>> > > timeslice?for task running on CPU core-2 has expired ?
>>> >
>>> > Each core only considers the timeslices for its own core.
>>> >
>>> > > I guess?timeslice expire case is not as same as preemption. Or may be
>>> > I am
>>> > > terribly wrong.
>>> >
>>> > You shouldn't be holding ?a spinlock for periods of time approaching
>>> > the length of a timeslice. The timer interrupt is what determines the
>>> > end of a timeslice. No timer interrupt, no end of a timeslice.
>>> > Preemption is also triggered by the timer interrupt, or by releasing a
>>> > resource that a higher priority task is waiting for.
>>>
>>> May be my understanding is incorrect, but wouldn't we hit the NMI
>>> watchdog here(assuming we are running on x86/x86_64)?
>>> We have a system lockup for long time.
>>> http://lxr.linux.no/#linux+v2.6.37/Documentation/nmi_watchdog.txt
>>>
>>> Could someone please clarify?
>>> >
>>> > Dave Hylands
>>>
>>> --
>>> Thanks,
>>> Nilesh
>>>
>>> _______________________________________________
>>> Kernelnewbies mailing list
>>> Kernelnewbies at kernelnewbies.org
>>> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>>
>>
>> _______________________________________________
>> Kernelnewbies mailing list
>> Kernelnewbies at kernelnewbies.org
>> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>>
>
>

spin_lock and scheduler confusion

[Dave Hylands]

Hi Nilesh.

Using reply all this time...

On Thu, Jan 6, 2011 at 10:27 PM, Tayade, Nilesh
<Nilesh.Tayade@netscout.com> wrote:
...snip...
>> You shouldn't be holding  a spinlock for periods of time approaching
>> the length of a timeslice. The timer interrupt is what determines the
>> end of a timeslice. No timer interrupt, no end of a timeslice.
>> Preemption is also triggered by the timer interrupt, or by releasing a
>> resource that a higher priority task is waiting for.
>
> May be my understanding is incorrect, but wouldn't we hit the NMI watchdog here(assuming we are running on x86/x86_64)?
> We have a system lockup for long time.
> http://lxr.linux.no/#linux+v2.6.37/Documentation/nmi_watchdog.txt
>
> Could someone please clarify?

Did you enable it by passing in nmi_watchdog=1 on the kernel command
line? (and verify that it was working fine - as described in the
nmi__watchdog.txt file)

Dave Hylands

spin_lock and scheduler confusion

[Viral Mehta]

Hi,
________________________________________
From: Dave Hylands [dhylands at gmail.com]
Subject: Re: spin_lock and scheduler confusion

>> I guess timeslice expire case is not as same as preemption. Or may be I am
>> terribly wrong.
>
>You shouldn't be holding  a spinlock for periods of time approaching
>the length of a timeslice. The timer interrupt is what determines the
>end of a timeslice. No timer interrupt, no end of a timeslice.
>Preemption is also triggered by the timer interrupt, or by releasing a
>resource that a higher priority task is waiting for.

thanks,
it is much clear now.

>Dave Hylands

Thanks,
Viral

http://groups.google.com/group/fundamental-discussion?hl=en

The contents of this e-mail and any attachment(s) may contain confidential or privileged information for the intended recipient(s). Unintended recipients are prohibited from taking action on the basis of information in this e-mail and  using or disseminating the information,  and must notify the sender and delete it from their system. L&T Infotech will not accept responsibility or liability for the accuracy or completeness of, or the presence of any virus or disabling code in this e-mail"

______________________________________________________________________

spin_lock and scheduler confusion

[Rajat Sharma]

Hi Dave,

> > spin_lock_irqsave();
> > schedule();
> > spin_lock_irqrestore();
>
> You're not supposed to call schedule with interrupts disbled.
>
> If you follow the code though schedule, you'll see that it calls
> schedule_debug, which in turns checks to see if its being called from
> an atomic context and if it is, it will cause the
>
> BUG: scheduling while atomic:

Please refer to my first mail in this thread, I have already mentioned
about scheduling while atomic in my first mail:

> however if you have debugging options turned on like CONFIG_DEBUG_SPINLOCK, you may likely get kernel warning for 'scheduling > in atomic context'.

And also its just a warning print with stack dump but not the panic()
or BUG(), so system is still responsive. Its anyways not a good
practice, but what I was trying to highlight is what can happen if we
call schedule() function while holding spin_lock_irqsave. To kill the
curiosity, I tried out following module:

#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/spinlock.h>
#include <linux/sched.h>

MODULE_LICENSE("GPL");

static DEFINE_SPINLOCK(sleepy_lock);

static int __init sleepy_init(void)
{
	unsigned long flags;

	printk("SLEEPTEST: loading sleepytest\n");

	spin_lock_irqsave(&sleepy_lock, flags);
	schedule();
	spin_unlock_irqrestore(&sleepy_lock, flags);

	return 0;
}

static void __exit sleepy_exit(void)
{
	printk("SLEEPTEST: unloading sleepytest\n");
}

module_init(sleepy_init);
module_exit(sleepy_exit);

And on my system, module silently prints messages without any atomic
schedule warning:

[ 6850.499940] SLEEPTEST: loading sleepytest
[ 6871.822539] SLEEPTEST: unloading sleepytest

I am not sure what config option turns on __schedule_bug(), but I
didn't see this message on my system:

CONFIG_SCHED_DEBUG=y
# CONFIG_DEBUG_SPINLOCK is not set
# CONFIG_DEBUG_SPINLOCK_SLEEP is not set

Please try this module and let us know if you can hit schedule_bug().

Rajat

On Fri, Jan 7, 2011 at 11:16 PM, Viral Mehta
<Viral.Mehta@lntinfotech.com> wrote:
> Hi,
> ________________________________________
> From: Dave Hylands [dhylands at gmail.com]
> Subject: Re: spin_lock and scheduler confusion
>
>>> I guess timeslice expire case is not as same as preemption. Or may be I am
>>> terribly wrong.
>>
>>You shouldn't be holding ?a spinlock for periods of time approaching
>>the length of a timeslice. The timer interrupt is what determines the
>>end of a timeslice. No timer interrupt, no end of a timeslice.
>>Preemption is also triggered by the timer interrupt, or by releasing a
>>resource that a higher priority task is waiting for.
>
> thanks,
> it is much clear now.
>
>>Dave Hylands
>
> Thanks,
> Viral
>
> http://groups.google.com/group/fundamental-discussion?hl=en
>
> The contents of this e-mail and any attachment(s) may contain confidential or privileged information for the intended recipient(s). Unintended recipients are prohibited from taking action on the basis of information in this e-mail and ?using or disseminating the information, ?and must notify the sender and delete it from their system. L&T Infotech will not accept responsibility or liability for the accuracy or completeness of, or the presence of any virus or disabling code in this e-mail"
>
> ______________________________________________________________________
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>