init_idle reaped before final call

init_idle reaped before final call

[Kip Walker]

[cross-posted to linux-kernel in a separate mail (oops)]

I'm working with a (approximately) 2.4.17 kernel from the mips-linux
tree (oss.sgi.com).

I'd like to propose removing the "__init" designation from init_idle in
kernel/sched.c, since this is called from rest_init via cpu_idle. 
Notice that rest_init isn't in an init section, and explicitly mentions
that it's avoiding a race with free_initmem.  In my kernel (an SMP
kernel running on a system with only 1 available CPU), cpu_idle isn't
getting called until after free_initmem().

My CPU is MIPS, but it looks like x86 could experience the same problem.

Kip

Index: kernel/sched.c
===================================================================
RCS file:
/projects/bbp/cvsroot/systemsw/linux/src/kernel/kernel/sched.c,v
retrieving revision 1.10
diff -u -r1.10 sched.c
--- kernel/sched.c      2002/01/15 04:13:43     1.10
+++ kernel/sched.c      2002/03/05 19:40:14
@@ -1304,7 +1304,7 @@
 
 extern unsigned long wait_init_idle;
 
-void __init init_idle(void)
+void init_idle(void)
 {
        struct schedule_data * sched_data;
        sched_data = &aligned_data[smp_processor_id()].schedule_data;

Re: init_idle reaped before final call

[Kip Walker]

"Martin J. Bligh" wrote:
> 
> > I'm working with a (approximately) 2.4.17 kernel from the mips-linux
> > tree (oss.sgi.com).
> >
> > I'd like to propose removing the "__init" designation from init_idle in
> > kernel/sched.c, since this is called from rest_init via cpu_idle.
> > Notice that rest_init isn't in an init section, and explicitly mentions
> > that it's avoiding a race with free_initmem.  In my kernel (an SMP
> > kernel running on a system with only 1 available CPU), cpu_idle isn't
> > getting called until after free_initmem().
> >
> > My CPU is MIPS, but it looks like x86 could experience the same problem.
> 
> I fixed something in this area for x86, looks like the same code path
> for MIPS unless I'm misreading.
> 
> smp_init spins waiting on wait_init_idle until every cpu has done
> init_idle. rest_init() isn't called until smp_init returns, so I'm not sure
> how you could hit this (possibly there's a minute window after init_idle
> clears the bit, but before it returns?).

This synchronization doesn't help: cpu0 (even in the multi-cpu case)
calls init_idle twice -- once from smp_init (through smp_boot_cpus), and
then again from cpu_idle.  In my failing case (CONFIG_SMP=y, but only 1
cpu in the system) the second call, the one from cpu_idle, doesn't
happen until long after the init kernel thread has been running and has
freed the initmem.

Maybe a better fix is to avoid this double calling of init_idle for the
"master" CPU?  From my reading the code, x86 seems to behave the same.

Kip

Re: init_idle reaped before final call

[Justin Carlson]

[-- Attachment #1: Type: text/plain, Size: 907 bytes --]

On Tue, 2002-03-05 at 17:15, Kip Walker wrote:
> 
> Maybe a better fix is to avoid this double calling of init_idle for the
> "master" CPU?  From my reading the code, x86 seems to behave the same.
> 

Looks to me like the clean fix would be to call init_idle() from
rest_init() before the init() thread is spawned, and remove it from
cpu_idle().  It looks like a pretty straightforward race condition that
no one else has happened to trigger in a bad way.  I'm no scheduler pro,
but I don't see any problems with calling init_idle() earlier.

That fix assumes that bringup of non-primary cpus on other architectures
call init_idle() explicitly before allowing smp_init() to return; this
is true of mips, but I can't vouch for any other arch's.

I'd submit a patch, but I'm sadly lacking in SMP machines for testing. 
Anyone who wants to rectify that, I'm open to charity.  :)

-Justin


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 232 bytes --]