2 CPUs, but only one is used

2 CPUs, but only one is used

[Giuliano Pochini]

I have a dual G4 MDD mac running linux 2.6.19. When I boot with maxcpus=1 and
then I issue "echo 1 >/sys/devices/system/cpu/cpu1/online" the second CPU is
put online without errors, but it stays idle+wait=100% and of course
user+system+nice=0%. It works fine without maxcpus=1. No problems with 2.6.18.

I saw that there are some cpu hotplug related changes in 2.6.19, but I have
no access to a non-ppc dual cpu box to check if the problem also affects
other architectures right now.


--
Giuliano.

Re: 2 CPUs, but only one is used

[Giuliano Pochini]

On Sun, 17 Dec 2006 14:41:27 +0100
Giuliano Pochini <pochini@shiny.it> wrote:

> I have a dual G4 MDD mac running linux 2.6.19. When I boot with maxcpus=1 and
> then I issue "echo 1 >/sys/devices/system/cpu/cpu1/online" the second CPU is
> put online without errors, but it stays idle+wait=100% and of course
> user+system+nice=0%. It works fine without maxcpus=1. No problems with 2.6.18.
>
> I saw that there are some cpu hotplug related changes in 2.6.19, but I have
> no access to a non-ppc dual cpu box to check if the problem also affects
> other architectures right now.

I tried to reproduce the problem on an intel box, but I couldn't. /sys
contains only the cpu0 file if it boots with maxcpus=1.


--
Giuliano.

Re: 2 CPUs, but only one is used

[Haren Myneni]

Giuliano Pochini wrote:

>On Sun, 17 Dec 2006 14:41:27 +0100
>Giuliano Pochini <pochini@shiny.it> wrote:
>
>  
>
>>I have a dual G4 MDD mac running linux 2.6.19. When I boot with maxcpus=1 and
>>then I issue "echo 1 >/sys/devices/system/cpu/cpu1/online" the second CPU is
>>put online without errors, but it stays idle+wait=100% and of course
>>user+system+nice=0%. It works fine without maxcpus=1. No problems with 2.6.18.
>>
>>I saw that there are some cpu hotplug related changes in 2.6.19, but I have
>>no access to a non-ppc dual cpu box to check if the problem also affects
>>other architectures right now.
>>    
>>
>
>I tried to reproduce the problem on an intel box, but I couldn't. /sys
>contains only the cpu0 file if it boots with maxcpus=1.
>  
>
On powerpc, when maxcpus=1 is passed, it does not reflect that change in 
cpu_possible_map. This possible bitmap is created based on device-tree. 
Where as on i386, it does. I.e, cpu_possible_map contains only one cpu. 
/sys entries will be created based on cpu_possible_map (I think). Hence, 
you noticed only cpu0 entry on Intel box. 

- Haren

>
>--
>Giuliano.
>_______________________________________________
>Linuxppc-dev mailing list
>Linuxppc-dev@ozlabs.org
>https://ozlabs.org/mailman/listinfo/linuxppc-dev
>  
>

Re: 2 CPUs, but only one is used

[Nathan Lynch]

Giuliano Pochini wrote:
> On Sun, 17 Dec 2006 14:41:27 +0100
> Giuliano Pochini <pochini@shiny.it> wrote:
> 
> > I have a dual G4 MDD mac running linux 2.6.19. When I boot with maxcpus=1 and
> > then I issue "echo 1 >/sys/devices/system/cpu/cpu1/online" the second CPU is
> > put online without errors, but it stays idle+wait=100% and of course
> > user+system+nice=0%. It works fine without maxcpus=1. No problems with 2.6.18.
> >
> > I saw that there are some cpu hotplug related changes in 2.6.19, but I have
> > no access to a non-ppc dual cpu box to check if the problem also affects
> > other architectures right now.
> 
> I tried to reproduce the problem on an intel box, but I couldn't. /sys
> contains only the cpu0 file if it boots with maxcpus=1.

Can you try to git bisect to find the change which caused the
regression?

Re: 2 CPUs, but only one is used

[Giuliano Pochini]

On Wed, 20 Dec 2006 08:16:55 -0600
Nathan Lynch <ntl@pobox.com> wrote:

> Giuliano Pochini wrote:
> > On Sun, 17 Dec 2006 14:41:27 +0100
> > Giuliano Pochini <pochini@shiny.it> wrote:
> > 
> > > I have a dual G4 MDD mac running linux 2.6.19. When I boot with maxcpus=1 and
> > > then I issue "echo 1 >/sys/devices/system/cpu/cpu1/online" the second CPU is
> > > put online without errors, but it stays idle+wait=100% and of course
> > > user+system+nice=0%. It works fine without maxcpus=1. No problems with 2.6.18.
> > >
> > > I saw that there are some cpu hotplug related changes in 2.6.19, but I have
> > > no access to a non-ppc dual cpu box to check if the problem also affects
> > > other architectures right now.
> > 
> > I tried to reproduce the problem on an intel box, but I couldn't. /sys
> > contains only the cpu0 file if it boots with maxcpus=1.
> 
> Can you try to git bisect to find the change which caused the
> regression?

*sigh* There are too much changes in the scheduler of 2.6.19 and in the ppc
code. I'm trying to find the problem in another way.


--
Giuliano.

Re: 2 CPUs, but only one is used

[Giuliano Pochini]

On Wed, 20 Dec 2006 08:16:55 -0600
Nathan Lynch <ntl@pobox.com> wrote:

> > > I have a dual G4 MDD mac running linux 2.6.19. When I boot with maxcpus=1 and
> > > then I issue "echo 1 >/sys/devices/system/cpu/cpu1/online" the second CPU is
> > > put online without errors, but it stays idle+wait=100% and of course
> > > user+system+nice=0%. It works fine without maxcpus=1. No problems with 2.6.18.
> > >
> > > I saw that there are some cpu hotplug related changes in 2.6.19, but I have
> > > no access to a non-ppc dual cpu box to check if the problem also affects
> > > other architectures right now.
> > 
> > I tried to reproduce the problem on an intel box, but I couldn't. /sys
> > contains only the cpu0 file if it boots with maxcpus=1.
> 
> Can you try to git bisect to find the change which caused the
> regression?
> 

I finally have some time to spend on this issue.

At the end of kernel/cpu.c::cpu_up() both cpus are set as present, possible
and online. The problem is that in kernel/sched.c::find_idlest_group()
p->cpus_allowed is always 1 (it should be 3). I tried to put a dump_stack()
or a printk(mask) in set_cpu_mask() but it crashes badly when mask is not 3,
ie. anything different from khelper or kthread, so I couldn't know what sets
the wrong cpumask.


--
Giuliano.

Re: 2 CPUs, but only one is used

[Nathan Lynch]

Giuliano Pochini wrote:
> I have a dual G4 MDD mac running linux 2.6.19. When I boot with maxcpus=1 and
> then I issue "echo 1 >/sys/devices/system/cpu/cpu1/online" the second CPU is
> put online without errors, but it stays idle+wait=100% and of course
> user+system+nice=0%. It works fine without maxcpus=1. No problems with 2.6.18.

...

> At the end of kernel/cpu.c::cpu_up() both cpus are set as present, possible
> and online. The problem is that in kernel/sched.c::find_idlest_group()
> p->cpus_allowed is always 1 (it should be 3). I tried to put a dump_stack()
> or a printk(mask) in set_cpu_mask() but it crashes badly when mask is not 3,
> ie. anything different from khelper or kthread, so I couldn't know what sets
> the wrong cpumask.

Does this fix it?

diff --git a/kernel/sched.c b/kernel/sched.c
index b515e3c..3c8b1c5 100644
--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -6875,7 +6875,7 @@ void __init sched_init_smp(void)
 
 	lock_cpu_hotplug();
 	arch_init_sched_domains(&cpu_online_map);
-	cpus_andnot(non_isolated_cpus, cpu_online_map, cpu_isolated_map);
+	cpus_andnot(non_isolated_cpus, cpu_possible_map, cpu_isolated_map);
 	if (cpus_empty(non_isolated_cpus))
 		cpu_set(smp_processor_id(), non_isolated_cpus);
 	unlock_cpu_hotplug();

Re: 2 CPUs, but only one is used

[Giuliano Pochini]

On Sun, 24 Dec 2006 23:52:51 -0600
Nathan Lynch <ntl@pobox.com> wrote:

> > At the end of kernel/cpu.c::cpu_up() both cpus are set as present, possible
> > and online. The problem is that in kernel/sched.c::find_idlest_group()
> > p->cpus_allowed is always 1 (it should be 3). I tried to put a dump_stack()
> > or a printk(mask) in set_cpu_mask() but it crashes badly when mask is not 3,
> > ie. anything different from khelper or kthread, so I couldn't know what sets
> > the wrong cpumask.
> 
> Does this fix it?
> 
> diff --git a/kernel/sched.c b/kernel/sched.c
> index b515e3c..3c8b1c5 100644
> --- a/kernel/sched.c
> +++ b/kernel/sched.c
> @@ -6875,7 +6875,7 @@ void __init sched_init_smp(void)
>  
>  	lock_cpu_hotplug();
>  	arch_init_sched_domains(&cpu_online_map);
> -	cpus_andnot(non_isolated_cpus, cpu_online_map, cpu_isolated_map);
> +	cpus_andnot(non_isolated_cpus, cpu_possible_map, cpu_isolated_map);
>  	if (cpus_empty(non_isolated_cpus))
>  		cpu_set(smp_processor_id(), non_isolated_cpus);
>  	unlock_cpu_hotplug();

Yes, it does. I also tested if tasks are migrated to/off cpu1 when I
enable/disable it and when I boot with or without maxcpus=1. It seems to
work correctly in all cases. Thank you.


--
Giuliano.

Re: 2 CPUs, but only one is used

[Giuliano Pochini]

[I resend this msg because I didn't receive it back from the lppc ml]

On Sun, 24 Dec 2006 23:52:51 -0600
Nathan Lynch <ntl@pobox.com> wrote:

> > At the end of kernel/cpu.c::cpu_up() both cpus are set as present, possible
> > and online. The problem is that in kernel/sched.c::find_idlest_group()
> > p->cpus_allowed is always 1 (it should be 3). I tried to put a dump_stack()
> > or a printk(mask) in set_cpu_mask() but it crashes badly when mask is not 3,
> > ie. anything different from khelper or kthread, so I couldn't know what sets
> > the wrong cpumask.
> 
> Does this fix it?
> 
> diff --git a/kernel/sched.c b/kernel/sched.c
> index b515e3c..3c8b1c5 100644
> --- a/kernel/sched.c
> +++ b/kernel/sched.c
> @@ -6875,7 +6875,7 @@ void __init sched_init_smp(void)
>  
>  	lock_cpu_hotplug();
>  	arch_init_sched_domains(&cpu_online_map);
> -	cpus_andnot(non_isolated_cpus, cpu_online_map, cpu_isolated_map);
> +	cpus_andnot(non_isolated_cpus, cpu_possible_map, cpu_isolated_map);
>  	if (cpus_empty(non_isolated_cpus))
>  		cpu_set(smp_processor_id(), non_isolated_cpus);
>  	unlock_cpu_hotplug();

Yes, it does. I also tested if tasks are migrated to/off cpu1 when I
enable/disable it and when I boot with or without maxcpus=1. It seems to
work correctly in all cases. Thank you.


--
Giuliano.