BUG: at kernel/sched.c:4031 __schedule()

BUG: at kernel/sched.c:4031 __schedule()

[Oleg Kechin]

Hi
I run 2.6.20-rt8 kernel on board like innovator1510 (OMAP5910 cpu)
I notice in boot string:
BUG: at kernel/sched.c:4031 __schedule()
kernel complain interrupts are enabled after schedule.

digging in sched.c I figured out that interrupts are disabled when cpu 
enter __sched(),
but are enabled when it went out.
Playing with the code i can find out, that problem is in the next 
fragment: (line 4024)

   if (likely(prev != next)) {
       next->timestamp = now;
       rq->nr_switches++;
       rq->curr = next;
       ++*switch_count;

       prepare_task_switch(rq, next);
       prev = context_switch(rq, prev, next);
       barrier();
       trace_special_pid(prev->pid, PRIO(prev), PRIO(current));
       /*
        * this_rq must be evaluated again because prev may have moved
        * CPUs since it called schedule(), thus the 'rq' on its stack
        * frame will be invalid.
        */
       finish_task_switch(this_rq(), prev);
       __preempt_enable_no_resched();
   } else {

in prepare_task_switch(rq, next); interrupts are enabled,
but in finish_task_switch(this_rq(), prev); interrupts not disabled
(it look not symmetrical for me).

Can anybody tell - is this right behavior?
I think interrupts should be disabled somewhere in this fragment,
or may be, scheduler should not complain?

Oleg Kechin.

Re: BUG: at kernel/sched.c:4031 __schedule()

[Steven Rostedt]

On Thu, 2007-06-07 at 17:07 +0300, Oleg Kechin wrote:

> in prepare_task_switch(rq, next); interrupts are enabled,

That's a bug.  __schedule should not have interrupts enabled.
What's the back trace of this?

-- Steve

> but in finish_task_switch(this_rq(), prev); interrupts not disabled
> (it look not symmetrical for me).
> 
> Can anybody tell - is this right behavior?
> I think interrupts should be disabled somewhere in this fragment,
> or may be, scheduler should not complain?
> 
> Oleg Kechin.
> -
> To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: BUG: at kernel/sched.c:4031 __schedule()

[Jan Altenberg]

Hi,

> > in prepare_task_switch(rq, next); interrupts are enabled,
> 
> That's a bug.  __schedule should not have interrupts enabled.
> What's the back trace of this?

I've also seen this in the past. Interrupts are enabled in
prepare_lock_switch because ARM defines __ARCH_WANT_INTERRUPTS_ON_CTXSW.

IIRC Thomas fixed this for the current -rt release.

Cheers,
	Jan

Re: BUG: at kernel/sched.c:4031 __schedule()

[Steven Rostedt]

On Thu, 2007-06-07 at 17:58 +0200, Jan Altenberg wrote:
> Hi,
> 
> > > in prepare_task_switch(rq, next); interrupts are enabled,
> > 
> > That's a bug.  __schedule should not have interrupts enabled.
> > What's the back trace of this?
> 
> I've also seen this in the past. Interrupts are enabled in
> prepare_lock_switch because ARM defines __ARCH_WANT_INTERRUPTS_ON_CTXSW.
> 

Heh, silly me.  I've been focused so much on x86 that I forgot what
other archs do.

> IIRC Thomas fixed this for the current -rt release.

Thomas, perhaps you want to post the fix.

What arch is innovator1510 (OMAP5910 cpu) anyway?

/me has been away from embedded systems for some time.

-- Steve

Re: BUG: at kernel/sched.c:4031 __schedule()

[Daniel Walker]

On Thu, 2007-06-07 at 12:13 -0400, Steven Rostedt wrote:

> 
> What arch is innovator1510 (OMAP5910 cpu) anyway?

It's an ARM .

> /me has been away from embedded systems for some time.
> 

The "fix" went into -rt6 , then was dropped in -rt7 .. I guess a better
fix is being working on, not sure..

Daniel

Re: BUG: at kernel/sched.c:4031 __schedule()

[Daniel Walker]

On Thu, 2007-06-07 at 12:13 -0400, Steven Rostedt wrote:
> On Thu, 2007-06-07 at 17:58 +0200, Jan Altenberg wrote:
> > Hi,
> > 
> > > > in prepare_task_switch(rq, next); interrupts are enabled,
> > > 
> > > That's a bug.  __schedule should not have interrupts enabled.
> > > What's the back trace of this?
> > 
> > I've also seen this in the past. Interrupts are enabled in
> > prepare_lock_switch because ARM defines __ARCH_WANT_INTERRUPTS_ON_CTXSW.
> > 
> 
> Heh, silly me.  I've been focused so much on x86 that I forgot what
> other archs do.
> 
> > IIRC Thomas fixed this for the current -rt release.
> 
> Thomas, perhaps you want to post the fix.
> 
> What arch is innovator1510 (OMAP5910 cpu) anyway?
> 
> /me has been away from embedded systems for some time.

I was wrong, I guess it's still in there .. Below I think..

+ *
+ * The new debugging code in __schedule() expects that interrupts
+ * are disabled across a switch_to. Need to investigate which
+ * consequences this has.
  */
-#if 0 // preempt_rt hack, because warn_on in shedule.c triggers
+#ifndef CONFIG_PREEMPT_RT
 #define __ARCH_WANT_INTERRUPTS_ON_CTXSW
 #endif

Re: BUG: at kernel/sched.c:4031 __schedule()

[Daniel Walker]

On Thu, 2007-06-07 at 09:21 -0700, Daniel Walker wrote:
> .
> 
> + *
> + * The new debugging code in __schedule() expects that interrupts
> + * are disabled across a switch_to. Need to investigate which
> + * consequences this has.
>   */
> -#if 0 // preempt_rt hack, because warn_on in shedule.c triggers
> +#ifndef CONFIG_PREEMPT_RT
>  #define __ARCH_WANT_INTERRUPTS_ON_CTXSW
>  #endif
> 

It is starting to look like a real fix now, thanks for updating it
Thomas .

Daniel

Re: BUG: at kernel/sched.c:4031 __schedule()

[Oleg Kechin]

On 6/7/07, Daniel Walker <dwalker@mvista.com> wrote:
> On Thu, 2007-06-07 at 09:21 -0700, Daniel Walker wrote:
> > .
> >
> > + *
> > + * The new debugging code in __schedule() expects that interrupts
> > + * are disabled across a switch_to. Need to investigate which
> > + * consequences this has.
> >   */
> > -#if 0 // preempt_rt hack, because warn_on in shedule.c triggers
> > +#ifndef CONFIG_PREEMPT_RT
> >  #define __ARCH_WANT_INTERRUPTS_ON_CTXSW
> >  #endif
> >
>
> It is starting to look like a real fix now, thanks for updating it
> Thomas .
>
> Daniel
>
>
Sorry, OMAP5910 CPU is ARM 925 (v4).
Thank you, Daniel, it's disapear.
There is stil a bit unclearness for me. When I set config other than
PREEMPT_RT this bug occure again.
May be it's ok - if someone want -rt patch on ARM arch, he will want
only PREEMPT_RT or I am wrong?

Oleg Kechin.

Re: BUG: at kernel/sched.c:4031 __schedule()

[Thomas Gleixner]

On Fri, 2007-06-08 at 13:08 +0300, Oleg Kechin wrote:
> Sorry, OMAP5910 CPU is ARM 925 (v4).
> Thank you, Daniel, it's disapear.
> There is stil a bit unclearness for me. When I set config other than
> PREEMPT_RT this bug occure again.
> May be it's ok - if someone want -rt patch on ARM arch, he will want
> only PREEMPT_RT or I am wrong?

My bad. The check in schedule() is there whether PREEMPT_RT is enabled
or not. I'm going to look at this.

Thanks,

	tglx

BUG: at kernel/sched.c:4031 __schedule()

[Oleg Kechin]

Hi
I run 2.6.20-rt8 kernel on board like innovator1510 (OMAP5910 cpu)
I notice in boot string:
BUG: at kernel/sched.c:4031 __schedule()
kernel complain interrupts are enabled after schedule.

digging in sched.c I figured out that interrupts are disabled when cpu 
enter __sched(),
but are enabled when it went out.
Playing with the code i can find out, that problem is in the next 
fragment: (line 4024)

    if (likely(prev != next)) {
        next->timestamp = now;
        rq->nr_switches++;
        rq->curr = next;
        ++*switch_count;

        prepare_task_switch(rq, next);
        prev = context_switch(rq, prev, next);
        barrier();
        trace_special_pid(prev->pid, PRIO(prev), PRIO(current));
        /*
         * this_rq must be evaluated again because prev may have moved
         * CPUs since it called schedule(), thus the 'rq' on its stack
         * frame will be invalid.
         */
        finish_task_switch(this_rq(), prev);
        __preempt_enable_no_resched();
    } else {

in prepare_task_switch(rq, next); interrupts are enabled,
but in finish_task_switch(this_rq(), prev); interrupts not disabled
(it look not symmetrical for me).

Can anybody tell - is this right behavior?
I think interrupts should be disabled somewhere in this fragment,
or may be, scheduler should not complain?

Oleg Kechin.

Re: BUG: at kernel/sched.c:4031 __schedule()

[Dirk Behme]

Oleg Kechin wrote:
> Hi
> I run 2.6.20-rt8 kernel on board like innovator1510 (OMAP5910 cpu)
> I notice in boot string:
> BUG: at kernel/sched.c:4031 __schedule()
> kernel complain interrupts are enabled after schedule.
> 
> digging in sched.c I figured out that interrupts are disabled when cpu 
> enter __sched(),
> but are enabled when it went out.
> Playing with the code i can find out, that problem is in the next 
> fragment: (line 4024)
> 
>    if (likely(prev != next)) {
>        next->timestamp = now;
>        rq->nr_switches++;
>        rq->curr = next;
>        ++*switch_count;
> 
>        prepare_task_switch(rq, next);
>        prev = context_switch(rq, prev, next);
>        barrier();
>        trace_special_pid(prev->pid, PRIO(prev), PRIO(current));
>        /*
>         * this_rq must be evaluated again because prev may have moved
>         * CPUs since it called schedule(), thus the 'rq' on its stack
>         * frame will be invalid.
>         */
>        finish_task_switch(this_rq(), prev);
>        __preempt_enable_no_resched();
>    } else {
> 
> in prepare_task_switch(rq, next); interrupts are enabled,
> but in finish_task_switch(this_rq(), prev); interrupts not disabled
> (it look not symmetrical for me).
> 
> Can anybody tell - is this right behavior?
> I think interrupts should be disabled somewhere in this fragment,
> or may be, scheduler should not complain?

Looks to me that this isn't an OMAP specific question. I 
think linux-rt-users@vger.kernel.org list [1] is a better 
place for this question.

Did you try a more recent -rt patch? They are updated 
frequently with additional fixes.

Regards

Dirk

[1] http://rt.wiki.kernel.org/index.php/Mailinglists