2.5.8: preemption + SMP broken ?

2.5.8: preemption + SMP broken ?

[Dipankar Sarma]

My machine (4cpu x86) hangs while booting a kernel with SMP
and preemption enabled. It hangs while executing one of
the initcalls, probably BIO since that is where the
next boot message comes from during a successful boot with SMP
(or preemption) disabled.

Has anyone tried out preemption with SMP ?

Thanks
-- 
Dipankar Sarma  <dipankar@in.ibm.com> http://lse.sourceforge.net
Linux Technology Center, IBM Software Lab, Bangalore, India.

Re: 2.5.8: preemption + SMP broken ?

[Robert Love]

On Wed, 2002-04-17 at 13:52, Dipankar Sarma wrote:
> My machine (4cpu x86) hangs while booting a kernel with SMP
> and preemption enabled. It hangs while executing one of
> the initcalls, probably BIO since that is where the
> next boot message comes from during a successful boot with SMP
> (or preemption) disabled.
> 
> Has anyone tried out preemption with SMP ?

Sure, all my testing is on SMP.  The problem manifested itself in
2.5.8-pre when some changes where made to the migration code.  The race
is in the migration code - I am not sure it is preempts fault, per se,
but the attached patch should fix it.  It is pending with Linus for the
next release.

	Robert Love

diff -urN linux-2.5.8/kernel/sched.c linux/kernel/sched.c
--- linux-2.5.8/kernel/sched.c	Sun Apr 14 15:18:47 2002
+++ linux/kernel/sched.c	Mon Apr 15 14:47:18 2002
@@ -1649,6 +1649,7 @@
 	if (!new_mask)
 		BUG();
 
+	preempt_disable();
 	rq = task_rq_lock(p, &flags);
 	p->cpus_allowed = new_mask;
 	/*
@@ -1657,7 +1658,7 @@
 	 */
 	if (new_mask & (1UL << p->thread_info->cpu)) {
 		task_rq_unlock(rq, &flags);
-		return;
+		goto out;
 	}
 
 	init_MUTEX_LOCKED(&req.sem);
@@ -1667,6 +1668,8 @@
 	wake_up_process(rq->migration_thread);
 
 	down(&req.sem);
+out:
+	preempt_enable();
 }
 
 static volatile unsigned long migration_mask;

Re: 2.5.8: preemption + SMP broken ?

[Dipankar Sarma]

On Wed, Apr 17, 2002 at 02:25:57PM -0400, Robert Love wrote:
> On Wed, 2002-04-17 at 13:52, Dipankar Sarma wrote:
> > Has anyone tried out preemption with SMP ?
> 
> Sure, all my testing is on SMP.  The problem manifested itself in
> 2.5.8-pre when some changes where made to the migration code.  The race
> is in the migration code - I am not sure it is preempts fault, per se,
> but the attached patch should fix it.  It is pending with Linus for the
> next release.

It worked! Thanks a lot, Robert.

Thanks
-- 
Dipankar Sarma  <dipankar@in.ibm.com> http://lse.sourceforge.net
Linux Technology Center, IBM Software Lab, Bangalore, India.

Re: 2.5.8: preemption + SMP broken ?

[Pavel Machek]

Hi!

> > My machine (4cpu x86) hangs while booting a kernel with SMP
> > and preemption enabled. It hangs while executing one of
> > the initcalls, probably BIO since that is where the
> > next boot message comes from during a successful boot with SMP
> > (or preemption) disabled.
> > 
> > Has anyone tried out preemption with SMP ?
> 
> Sure, all my testing is on SMP.  The problem manifested itself in
> 2.5.8-pre when some changes where made to the migration code.  The race
> is in the migration code - I am not sure it is preempts fault, per se,
> but the attached patch should fix it.  It is pending with Linus for the
> next release.

Wel, if race is in migration code, you should fix migration, not add
preempt_disable(), right? [Or is race preempt-onlly?]

-- 
Philips Velo 1: 1"x4"x8", 300gram, 60, 12MB, 40bogomips, linux, mutt,
details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.

Re: 2.5.8: preemption + SMP broken ?

[Robert Love]

On Sun, 2002-04-21 at 14:57, Pavel Machek wrote:

> Wel, if race is in migration code, you should fix migration, not add
> preempt_disable(), right? [Or is race preempt-onlly?]

I was not sure where the race was - I really did not see a preempt-only
race there, but I could not pinpoint the cause for an SMP race.  Since
it was only showing itself as a preempt-related race, I disabled
preemption.

Erich Focht later pointed out that interrupts should be disabled in
migration_init and we sent a patch off to Linus.  I will do some testing
and see if this fixes it (I suspect it may) and subsequently remove the
preempt code.

	Robert Love