[Linux-ia64] Re: CPU hotplug boot change for IA64

[Linux-ia64] Re: CPU hotplug boot change for IA64

[David Mosberger]

What's the reason for putting cpu_initialize in setup.c instead of
smp.c?  Also, I much prefer to have all global variables declared in
header files; especially for special things that are marked __init or
__devinit, etc.  Nothing worse than someone accidentally trying to use
the variable after its gone.  (Yes, there are other offenders, but
that's no reason to add to the problem.)

	--david

Re: [Linux-ia64] Re: CPU hotplug boot change for IA64

[Kimio Suganuma]

Hi David,

Sorry, I missed to remove that. :-(
cpu_initialized (or something like that) is needed when
CPU Hot Plug is implemented.
__cpu_up() must detect that the CPU has already been initialized
and but offlined or the CPU hasn't been initialized.
But for 2.5.29, such the variable isn't necessary.
And as you pointed out, it should not be in setup.c.
I'll update the patch and send it later.

Regards,
Kimi

On Fri, 2 Aug 2002 14:44:16 -0700
David Mosberger <davidm@napali.hpl.hp.com> wrote:

> What's the reason for putting cpu_initialize in setup.c instead of
> smp.c?  Also, I much prefer to have all global variables declared in
> header files; especially for special things that are marked __init or
> __devinit, etc.  Nothing worse than someone accidentally trying to use
> the variable after its gone.  (Yes, there are other offenders, but
> that's no reason to add to the problem.)

-- 
Kimio Suganuma <k-suganuma@mvj.biglobe.ne.jp>

Re: [Linux-ia64] Re: CPU hotplug boot change for IA64

[Kimio Suganuma]

[-- Attachment #1: Type: text/plain, Size: 718 bytes --]

Hi David,

I removed cpu_initialized and fixed several bugs.
I tested SMP and UP kernels on 2-way Bigsur.
Please let me know if there is any problem on the patch.

Thanks,
Kimi

On Fri, 02 Aug 2002 15:18:01 -0700
Kimio Suganuma <k-suganuma@mvj.biglobe.ne.jp> wrote:

> Sorry, I missed to remove that. :-(
> cpu_initialized (or something like that) is needed when
> CPU Hot Plug is implemented.
> __cpu_up() must detect that the CPU has already been initialized
> and but offlined or the CPU hasn't been initialized.
> But for 2.5.29, such the variable isn't necessary.
> And as you pointed out, it should not be in setup.c.
> I'll update the patch and send it later.

-- 
Kimio Suganuma <k-suganuma@mvj.biglobe.ne.jp>

[-- Attachment #2: hotplug-boot-change-ia64.patch.gz --]
[-- Type: application/octet-stream, Size: 2473 bytes --]

[Linux-ia64] Re: CPU hotplug boot change for IA64

[Rusty Russell]

In message <20020802134915.ED77.K-SUGANUMA@mvj.biglobe.ne.jp> you write:
> Bit for CPU#0 on cpu_online_map was set at smp_boot_cpus() before.
> But on the 2.5.29, place of calling smp_boot_cpus() is changed to
> later timing and this seems to make a chance that smp_call_function()
> is called before setting cpu_online_map, (I guess.)
> In that case, smp_call_function won't return because num_online_cpus()
> returns 0 and the system will hang.
> 
> I have an experience of this problem only on a large system
> like Tiger, but it may occur on another system.
> 
> Rusty, I guess this modification should be needed for i386, too.

You should *not* be calling smp_call_function before smp_prepare_cpus.

I know kgdb can do this, but it's a special case.

Rusty.
--
  Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

Re: [Linux-ia64] Re: CPU hotplug boot change for IA64

[Kimio Suganuma]

Hi Rusty,

On Sat, 03 Aug 2002 13:29:02 +1000
Rusty Russell <rusty@rustcorp.com.au> wrote:

> > Rusty, I guess this modification should be needed for i386, too.
> 
> You should *not* be calling smp_call_function before smp_prepare_cpus.
> 
> I know kgdb can do this, but it's a special case.

OK.
I cannot remember who was calling smp_call_function, but I think
it might be kdb. I'm using kdb for debugging in usual.

Thanks,
Kimi

-- 
Kimio Suganuma <k-suganuma@mvj.biglobe.ne.jp>

Re: [Linux-ia64] Re: CPU hotplug boot change for IA64

[Rusty Russell]

In message <20020802210541.0AB4.K-SUGANUMA@mvj.biglobe.ne.jp> you write:
> Hi Rusty,
> 
> On Sat, 03 Aug 2002 13:29:02 +1000
> Rusty Russell <rusty@rustcorp.com.au> wrote:
> 
> > > Rusty, I guess this modification should be needed for i386, too.
> > 
> > You should *not* be calling smp_call_function before smp_prepare_cpus.
> > 
> > I know kgdb can do this, but it's a special case.
> 
> OK.
> I cannot remember who was calling smp_call_function, but I think
> it might be kdb. I'm using kdb for debugging in usual.

Yes, in which case the fix I asked Andrew Morton to use was:

	/* Before smp_prepare_cpus, cpu_online not usable */
	if (num_online_cpus() > 1)
		smp_call_function(...)

Cheers,
Rusty.
--
  Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

Re: [Linux-ia64] Re: CPU hotplug boot change for IA64

[Andrew Morton]

Rusty Russell wrote:
> 
> In message <20020802210541.0AB4.K-SUGANUMA@mvj.biglobe.ne.jp> you write:
> > Hi Rusty,
> >
> > On Sat, 03 Aug 2002 13:29:02 +1000
> > Rusty Russell <rusty@rustcorp.com.au> wrote:
> >
> > > > Rusty, I guess this modification should be needed for i386, too.
> > >
> > > You should *not* be calling smp_call_function before smp_prepare_cpus.
> > >
> > > I know kgdb can do this, but it's a special case.
> >
> > OK.
> > I cannot remember who was calling smp_call_function, but I think
> > it might be kdb. I'm using kdb for debugging in usual.
> 
> Yes, in which case the fix I asked Andrew Morton to use was:
> 
>         /* Before smp_prepare_cpus, cpu_online not usable */
>         if (num_online_cpus() > 1)
>                 smp_call_function(...)

It works, too.  http://www.zip.com.au/~akpm/linux/patches/2.5/2.5.30/kgdb.patch

Re: [Linux-ia64] Re: CPU hotplug boot change for IA64

[Erich Focht]

Hi Kimi,

On Saturday 03 August 2002 02:53, Kimio Suganuma wrote:
> Hi David,
>
> I removed cpu_initialized and fixed several bugs.
> I tested SMP and UP kernels on 2-way Bigsur.
> Please let me know if there is any problem on the patch.

The patch looks fine. Before applying it to 2.5.29 my BigSur ended up
with no tasks beeing scheduled to cpu#1 and no migration_task and
ksoftirqd for the same cpu. But the system came up. After applying it 
the system hangs in the boot sequence right after "Freeing unused
kernel memory". But CPU#1 was woken up correctly. Sorry, have no kdb
to be able to tell you more...

Does anybody else have these problems?

Thanks,
Erich

Re: [Linux-ia64] Re: CPU hotplug boot change for IA64

[Kimio Suganuma]

Hi Erich,

On Tue, 6 Aug 2002 20:33:09 +0200
Erich Focht <efocht@ess.nec.de> wrote:

> > I removed cpu_initialized and fixed several bugs.
> > I tested SMP and UP kernels on 2-way Bigsur.
> > Please let me know if there is any problem on the patch.
> 
> The patch looks fine. Before applying it to 2.5.29 my BigSur ended up
> with no tasks beeing scheduled to cpu#1 and no migration_task and
> ksoftirqd for the same cpu. But the system came up. After applying it 
> the system hangs in the boot sequence right after "Freeing unused
> kernel memory". But CPU#1 was woken up correctly. Sorry, have no kdb
> to be able to tell you more...
> 
> Does anybody else have these problems?

I had a problem that the system hang at booting on 2.5.29 without
my patch. After that, I turned off CONFIGs as much as possible
and then I succeeded to boot the kernel.
I suspect IDE/ATAPI driver or something was causing the problem
(I'm not sure, though.)
Anyway, I'd like to reproduce the problem. Could you send me your
.config?

Regards,
Kimi

-- 
Kimio Suganuma <k-suganuma@mvj.biglobe.ne.jp>

Re: [Linux-ia64] Re: CPU hotplug boot change for IA64

[Kimio Suganuma]

Hi,

On Tue, 6 Aug 2002 20:33:09 +0200
Erich Focht <efocht@ess.nec.de> wrote:

> The patch looks fine. Before applying it to 2.5.29 my BigSur ended up
> with no tasks beeing scheduled to cpu#1 and no migration_task and
> ksoftirqd for the same cpu.
>
> But the system came up. After applying it 
> the system hangs in the boot sequence right after "Freeing unused
> kernel memory". But CPU#1 was woken up correctly. Sorry, have no kdb
> to be able to tell you more...
> 
> Does anybody else have these problems?

ksoftirqd and migration_thread are spawned in cpu_up() in kernel/cpu.c.
But, cpu_up() wasn't called because the cpu had already been running
at that time (see arch/ia64/kernel/smpboot.c and init/main.c.)
I tried to change init/main.c to call cpu_up() for APs, and I made sure
that the kernel caused the system hangs as you reported.
This means that 2.5.29 kernel is not stable on SMP regardless of
CPU Hot Plug patch applying.
I found that kudzu(1) and shutdown of ethernet may cause system hangs
but I'm not sure why.

Regards,
Kimi

-- 
Kimio Suganuma <k-suganuma@mvj.biglobe.ne.jp>