[RFC] mark Netlogic XLR chip as SMT capable

[RFC] mark Netlogic XLR chip as SMT capable

[Hillf Danton]

Netlogic XLR chip has multiple cores. Each core includes four integrated
hardware threads, and they share L1 data and instruction caches.

If XLR chip is marked to be SMT capable, linux scheduler then could do more,
say idle load balancing.

Any comment is welcom, thanks.

Signed-off-by: Hillf Danton <dhillf@gmail.com>
---

--- a/arch/mips/netlogic/xlr/smp.c	Sun Oct  2 14:15:28 2011
+++ b/arch/mips/netlogic/xlr/smp.c	Sun Oct  2 14:15:58 2011
@@ -176,6 +176,7 @@ void __init nlm_smp_setup(void)

 void nlm_prepare_cpus(unsigned int max_cpus)
 {
+	smp_num_siblings = 4;
 }

 struct plat_smp_ops nlm_smp_ops = {
--- a/arch/mips/kernel/smp.c	Sun Oct  2 14:12:09 2011
+++ b/arch/mips/kernel/smp.c	Sun Oct  2 14:14:58 2011
@@ -73,7 +73,12 @@ static inline void set_cpu_sibling_map(i

 	if (smp_num_siblings > 1) {
 		for_each_cpu_mask(i, cpu_sibling_setup_map) {
-			if (cpu_data[cpu].core == cpu_data[i].core) {
+			if (current_cpu_type() == CPU_XLR) {
+				if (((i>>2) & 0x7) == ((cpu>>2) & 0x7))
+					goto set;
+			}
+			else if (cpu_data[cpu].core == cpu_data[i].core) {
+set:
 				cpu_set(i, cpu_sibling_map[cpu]);
 				cpu_set(cpu, cpu_sibling_map[i]);
 			}

Re: [RFC] mark Netlogic XLR chip as SMT capable

[Jayachandran C.]

On Sun, Oct 02, 2011 at 03:26:14PM +0800, Hillf Danton wrote:
> Netlogic XLR chip has multiple cores. Each core includes four integrated
> hardware threads, and they share L1 data and instruction caches.
> 
> If XLR chip is marked to be SMT capable, linux scheduler then could do more,
> say idle load balancing.
> 
> Any comment is welcom, thanks.

I may be missing something here, but how about just setting cpu_data[].core in
the init_secondary method?  That would avoid the change to kernel/smp.c. 

> 
> Signed-off-by: Hillf Danton <dhillf@gmail.com>
> ---
> 
> --- a/arch/mips/netlogic/xlr/smp.c	Sun Oct  2 14:15:28 2011
> +++ b/arch/mips/netlogic/xlr/smp.c	Sun Oct  2 14:15:58 2011
> @@ -176,6 +176,7 @@ void __init nlm_smp_setup(void)
> 
>  void nlm_prepare_cpus(unsigned int max_cpus)
>  {
> +	smp_num_siblings = 4;
>  }
> 
>  struct plat_smp_ops nlm_smp_ops = {
> --- a/arch/mips/kernel/smp.c	Sun Oct  2 14:12:09 2011
> +++ b/arch/mips/kernel/smp.c	Sun Oct  2 14:14:58 2011
> @@ -73,7 +73,12 @@ static inline void set_cpu_sibling_map(i
> 
>  	if (smp_num_siblings > 1) {
>  		for_each_cpu_mask(i, cpu_sibling_setup_map) {
> -			if (cpu_data[cpu].core == cpu_data[i].core) {
> +			if (current_cpu_type() == CPU_XLR) {
> +				if (((i>>2) & 0x7) == ((cpu>>2) & 0x7))
> +					goto set;
> +			}
> +			else if (cpu_data[cpu].core == cpu_data[i].core) {
> +set:
>  				cpu_set(i, cpu_sibling_map[cpu]);
>  				cpu_set(cpu, cpu_sibling_map[i]);
>  			}

JC.

Re: [RFC] mark Netlogic XLR chip as SMT capable

[Jayachandran C.]

On Sun, Oct 02, 2011 at 03:26:14PM +0800, Hillf Danton wrote:
> Netlogic XLR chip has multiple cores. Each core includes four integrated
> hardware threads, and they share L1 data and instruction caches.
> 
> If XLR chip is marked to be SMT capable, linux scheduler then could do more,
> say idle load balancing.
> 
> Any comment is welcom, thanks.

I may be missing something here, but how about just setting cpu_data[].core in
the init_secondary method?  That would avoid the change to kernel/smp.c. 

> 
> Signed-off-by: Hillf Danton <dhillf@gmail.com>
> ---
> 
> --- a/arch/mips/netlogic/xlr/smp.c	Sun Oct  2 14:15:28 2011
> +++ b/arch/mips/netlogic/xlr/smp.c	Sun Oct  2 14:15:58 2011
> @@ -176,6 +176,7 @@ void __init nlm_smp_setup(void)
> 
>  void nlm_prepare_cpus(unsigned int max_cpus)
>  {
> +	smp_num_siblings = 4;
>  }
> 
>  struct plat_smp_ops nlm_smp_ops = {
> --- a/arch/mips/kernel/smp.c	Sun Oct  2 14:12:09 2011
> +++ b/arch/mips/kernel/smp.c	Sun Oct  2 14:14:58 2011
> @@ -73,7 +73,12 @@ static inline void set_cpu_sibling_map(i
> 
>  	if (smp_num_siblings > 1) {
>  		for_each_cpu_mask(i, cpu_sibling_setup_map) {
> -			if (cpu_data[cpu].core == cpu_data[i].core) {
> +			if (current_cpu_type() == CPU_XLR) {
> +				if (((i>>2) & 0x7) == ((cpu>>2) & 0x7))
> +					goto set;
> +			}
> +			else if (cpu_data[cpu].core == cpu_data[i].core) {
> +set:
>  				cpu_set(i, cpu_sibling_map[cpu]);
>  				cpu_set(cpu, cpu_sibling_map[i]);
>  			}

JC.

Re: [RFC] mark Netlogic XLR chip as SMT capable

[Hillf Danton]

On Sun, Oct 2, 2011 at 4:30 PM, Jayachandran C.
<jayachandranc@netlogicmicro.com> wrote:
> On Sun, Oct 02, 2011 at 03:26:14PM +0800, Hillf Danton wrote:
>> Netlogic XLR chip has multiple cores. Each core includes four integrated
>> hardware threads, and they share L1 data and instruction caches.
>>
>> If XLR chip is marked to be SMT capable, linux scheduler then could do more,
>> say idle load balancing.
>>
>> Any comment is welcom, thanks.
>
> I may be missing something here, but how about just setting cpu_data[].core in
> the init_secondary method?  That would avoid the change to kernel/smp.c.
>

Got and thanks. It is re-prepared as the following:)

Hillf
----------------------------------------------------------------------------
Subject: [RFC] Mark Netlogic XLR chip to be SMT capable

Netlogic XLR chip has multiple cores. Each core includes four integrated
hardware threads, and they share L1 data and instruction caches.

If XLR chip is marked to be SMT capable, scheduler then could do more, say,
idle load balancing.

According to JC's comment changes are now confined only to the code of XLR.

Signed-off-by: Hillf Danton <dhillf@gmail.com>
---

--- a/arch/mips/netlogic/xlr/smp.c	Sun Oct  2 14:15:28 2011
+++ b/arch/mips/netlogic/xlr/smp.c	Mon Oct  3 13:33:02 2011
@@ -104,6 +104,12 @@ void nlm_early_init_secondary(void)
  */
 static void __cpuinit nlm_init_secondary(void)
 {
+	unsigned int cpu, core_id;
+
+	cpu = smp_processor_id();
+	core_id = (cpu >> 2) & 0x7;
+	cpu_data[cpu].core = core_id;
+
 	nlm_smp_irq_init();
 }

@@ -176,6 +182,8 @@ void __init nlm_smp_setup(void)

 void nlm_prepare_cpus(unsigned int max_cpus)
 {
+	/* declare we are SMT capable */
+	smp_num_siblings = 4;
 }

 struct plat_smp_ops nlm_smp_ops = {

Re: [RFC] mark Netlogic XLR chip as SMT capable

[Ralf Baechle]

On Mon, Oct 03, 2011 at 01:46:46PM +0800, Hillf Danton wrote:

> +	unsigned int cpu, core_id;
> +
> +	cpu = smp_processor_id();
> +	core_id = (cpu >> 2) & 0x7;
> +	cpu_data[cpu].core = core_id;

This is going to break in setups where Linux is not being booted on
what the hardware considers CPU core 0.  Which is not uncommon in embedded
setups.  You may want to probe the hardware for the core ID rather than
relying on smp_processor_id() here.

  Ralf

Re: [RFC] mark Netlogic XLR chip as SMT capable

[Jayachandran C.]

On Mon, Oct 03, 2011 at 11:32:04AM +0100, Ralf Baechle wrote:
> On Mon, Oct 03, 2011 at 01:46:46PM +0800, Hillf Danton wrote:
> 
> > +	unsigned int cpu, core_id;
> > +
> > +	cpu = smp_processor_id();
> > +	core_id = (cpu >> 2) & 0x7;
> > +	cpu_data[cpu].core = core_id;
> 
> This is going to break in setups where Linux is not being booted on
> what the hardware considers CPU core 0.  Which is not uncommon in embedded
> setups.  You may want to probe the hardware for the core ID rather than
> relying on smp_processor_id() here.

Yes, the function hard_smp_processor_id() from netlogic/mips-extns.h has to
be used here.

This also conflicts with the recent patch-set for XLP support, but I don't
know the status of that yet.

JC.

Re: [RFC] mark Netlogic XLR chip as SMT capable

[Jayachandran C.]

On Mon, Oct 03, 2011 at 11:32:04AM +0100, Ralf Baechle wrote:
> On Mon, Oct 03, 2011 at 01:46:46PM +0800, Hillf Danton wrote:
> 
> > +	unsigned int cpu, core_id;
> > +
> > +	cpu = smp_processor_id();
> > +	core_id = (cpu >> 2) & 0x7;
> > +	cpu_data[cpu].core = core_id;
> 
> This is going to break in setups where Linux is not being booted on
> what the hardware considers CPU core 0.  Which is not uncommon in embedded
> setups.  You may want to probe the hardware for the core ID rather than
> relying on smp_processor_id() here.

Yes, the function hard_smp_processor_id() from netlogic/mips-extns.h has to
be used here.

This also conflicts with the recent patch-set for XLP support, but I don't
know the status of that yet.

JC.

Re: [RFC] mark Netlogic XLR chip as SMT capable

[Hillf Danton]

On Mon, Oct 3, 2011 at 6:39 PM, Jayachandran C.
<jayachandranc@netlogicmicro.com> wrote:
> On Mon, Oct 03, 2011 at 11:32:04AM +0100, Ralf Baechle wrote:
>> On Mon, Oct 03, 2011 at 01:46:46PM +0800, Hillf Danton wrote:
>>
>> > +   unsigned int cpu, core_id;
>> > +
>> > +   cpu = smp_processor_id();
>> > +   core_id = (cpu >> 2) & 0x7;
>> > +   cpu_data[cpu].core = core_id;
>>
>> This is going to break in setups where Linux is not being booted on
>> what the hardware considers CPU core 0.  Which is not uncommon in embedded
>> setups.  You may want to probe the hardware for the core ID rather than
>> relying on smp_processor_id() here.
>
> Yes, the function hard_smp_processor_id() from netlogic/mips-extns.h has to
> be used here.
>
> This also conflicts with the recent patch-set for XLP support, but I don't
> know the status of that yet.
>

Thanks JC and Ralf. Your comments are included in the following version.

Hillf
----------------------------------------------------------------------------
Subject: [RFC] Mark Netlogic XLR chip to be SMT capable

Netlogic XLR chip has multiple cores. Each core includes four integrated
hardware threads, and they share L1 data and instruction caches.

If the chip is marked to be SMT capable, scheduler then could do more, say,
idle load balancing.

Changes are now confined only to the code of XLR, and hardware is probed
to get core ID for correct setup.

Signed-off-by: Hillf Danton <dhillf@gmail.com>
---

--- a/arch/mips/netlogic/xlr/smp.c	Sun Oct  2 14:15:28 2011
+++ b/arch/mips/netlogic/xlr/smp.c	Mon Oct  3 13:33:02 2011
@@ -104,6 +104,12 @@ void nlm_early_init_secondary(void)
  */
 static void __cpuinit nlm_init_secondary(void)
 {
+	unsigned int cpu, core_id;
+
+	cpu = hard_smp_processor_id();
+	core_id = (cpu >> 2) & 7;
+	cpu_data[cpu].core = core_id;
+
 	nlm_smp_irq_init();
 }

@@ -176,6 +182,8 @@ void __init nlm_smp_setup(void)

 void nlm_prepare_cpus(unsigned int max_cpus)
 {
+	/* declare we are SMT capable */
+	smp_num_siblings = 4;
 }

 struct plat_smp_ops nlm_smp_ops = {