Linux MIPS Architecture development
 help / color / mirror / Atom feed
From: James Hogan <james.hogan@imgtec.com>
To: Ralf Baechle <ralf@linux-mips.org>
Cc: James Hogan <james.hogan@imgtec.com>,
	Paul Burton <paul.burton@imgtec.com>,
	Leonid Yegoshin <leonid.yegoshin@imgtec.com>,
	Felix Fietkau <nbd@nbd.name>,
	Jayachandran C. <jchandra@broadcom.com>,
	<linux-mips@linux-mips.org>
Subject: [PATCH 09/13] MIPS: c-r4k: Exclude sibling CPUs in SMP calls
Date: Wed, 13 Jul 2016 14:12:52 +0100	[thread overview]
Message-ID: <1468415576-12600-10-git-send-email-james.hogan@imgtec.com> (raw)
In-Reply-To: <1468415576-12600-1-git-send-email-james.hogan@imgtec.com>

When performing SMP calls to foreign cores, exclude sibling CPUs from
the provided map, as we already handle the local core on the current
CPU. This prevents an SMP call from for example core 0, VPE 1 to VPE 0
on the same core.

In the process the cpu_foreign_map cpumask is turned into an array of
cpumasks, so that each CPU has its own version of it which excludes
sibling CPUs. r4k_op_needs_ipi() is also updated to reflect that cache
management SMP calls are not needed when all CPUs are siblings (i.e.
there are no foreign CPUs according to the new cpu_foreign_map[]
semantics which exclude siblings).

Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com>
Cc: Felix Fietkau <nbd@nbd.name>
Cc: Jayachandran C. <jchandra@broadcom.com>
Cc: linux-mips@linux-mips.org
---
 arch/mips/include/asm/smp.h |  2 +-
 arch/mips/kernel/smp.c      |  6 ++++--
 arch/mips/mm/c-r4k.c        | 17 +++++++++++++----
 3 files changed, 18 insertions(+), 7 deletions(-)

diff --git a/arch/mips/include/asm/smp.h b/arch/mips/include/asm/smp.h
index 0c534a03bb36..8bc6c70a4030 100644
--- a/arch/mips/include/asm/smp.h
+++ b/arch/mips/include/asm/smp.h
@@ -23,7 +23,7 @@
 extern int smp_num_siblings;
 extern cpumask_t cpu_sibling_map[];
 extern cpumask_t cpu_core_map[];
-extern cpumask_t cpu_foreign_map;
+extern cpumask_t cpu_foreign_map[];
 
 #define raw_smp_processor_id() (current_thread_info()->cpu)
 
diff --git a/arch/mips/kernel/smp.c b/arch/mips/kernel/smp.c
index 783d5f50ab9d..f95f094f36e4 100644
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -72,7 +72,7 @@ EXPORT_SYMBOL(cpu_core_map);
  * A logcal cpu mask containing only one VPE per core to
  * reduce the number of IPIs on large MT systems.
  */
-cpumask_t cpu_foreign_map __read_mostly;
+cpumask_t cpu_foreign_map[NR_CPUS] __read_mostly;
 EXPORT_SYMBOL(cpu_foreign_map);
 
 /* representing cpus for which sibling maps can be computed */
@@ -141,7 +141,9 @@ void calculate_cpu_foreign_map(void)
 			cpumask_set_cpu(i, &temp_foreign_map);
 	}
 
-	cpumask_copy(&cpu_foreign_map, &temp_foreign_map);
+	for_each_online_cpu(i)
+		cpumask_andnot(&cpu_foreign_map[i],
+			       &temp_foreign_map, &cpu_sibling_map[i]);
 }
 
 struct plat_smp_ops *mp_ops;
diff --git a/arch/mips/mm/c-r4k.c b/arch/mips/mm/c-r4k.c
index 2a4bb5057ebc..57374f0c33f2 100644
--- a/arch/mips/mm/c-r4k.c
+++ b/arch/mips/mm/c-r4k.c
@@ -56,7 +56,9 @@
  * @type:	Type of cache operations (R4K_HIT or R4K_INDEX).
  *
  * Decides whether a cache op needs to be performed on every core in the system.
- * This may change depending on the @type of cache operation.
+ * This may change depending on the @type of cache operation, as well as the set
+ * of online CPUs, so preemption should be disabled by the caller to prevent CPU
+ * hotplug from changing the result.
  *
  * Returns:	1 if the cache operation @type should be done on every core in
  *		the system.
@@ -71,9 +73,15 @@ static inline bool r4k_op_needs_ipi(unsigned int type)
 
 	/*
 	 * Hardware doesn't globalize the required cache ops, so SMP calls may
-	 * be needed.
+	 * be needed, but only if there are foreign CPUs (non-siblings with
+	 * separate caches).
 	 */
-	return true;
+	/* cpu_foreign_map[] undeclared when !CONFIG_SMP */
+#ifdef CONFIG_SMP
+	return !cpumask_empty(&cpu_foreign_map[0]);
+#else
+	return false;
+#endif
 }
 
 /*
@@ -90,7 +98,8 @@ static inline void r4k_on_each_cpu(unsigned int type,
 {
 	preempt_disable();
 	if (r4k_op_needs_ipi(type))
-		smp_call_function_many(&cpu_foreign_map, func, info, 1);
+		smp_call_function_many(&cpu_foreign_map[smp_processor_id()],
+				       func, info, 1);
 	func(info);
 	preempt_enable();
 }
-- 
2.4.10

WARNING: multiple messages have this Message-ID (diff)
From: James Hogan <james.hogan@imgtec.com>
To: Ralf Baechle <ralf@linux-mips.org>
Cc: James Hogan <james.hogan@imgtec.com>,
	Paul Burton <paul.burton@imgtec.com>,
	Leonid Yegoshin <leonid.yegoshin@imgtec.com>,
	Felix Fietkau <nbd@nbd.name>,
	"Jayachandran C." <jchandra@broadcom.com>,
	linux-mips@linux-mips.org
Subject: [PATCH 09/13] MIPS: c-r4k: Exclude sibling CPUs in SMP calls
Date: Wed, 13 Jul 2016 14:12:52 +0100	[thread overview]
Message-ID: <1468415576-12600-10-git-send-email-james.hogan@imgtec.com> (raw)
Message-ID: <20160713131252.ew-hy2bAbRN_5bpZI5OkscSdDkScAhroHzmYDQvyhLE@z> (raw)
In-Reply-To: <1468415576-12600-1-git-send-email-james.hogan@imgtec.com>

When performing SMP calls to foreign cores, exclude sibling CPUs from
the provided map, as we already handle the local core on the current
CPU. This prevents an SMP call from for example core 0, VPE 1 to VPE 0
on the same core.

In the process the cpu_foreign_map cpumask is turned into an array of
cpumasks, so that each CPU has its own version of it which excludes
sibling CPUs. r4k_op_needs_ipi() is also updated to reflect that cache
management SMP calls are not needed when all CPUs are siblings (i.e.
there are no foreign CPUs according to the new cpu_foreign_map[]
semantics which exclude siblings).

Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com>
Cc: Felix Fietkau <nbd@nbd.name>
Cc: Jayachandran C. <jchandra@broadcom.com>
Cc: linux-mips@linux-mips.org
---
 arch/mips/include/asm/smp.h |  2 +-
 arch/mips/kernel/smp.c      |  6 ++++--
 arch/mips/mm/c-r4k.c        | 17 +++++++++++++----
 3 files changed, 18 insertions(+), 7 deletions(-)

diff --git a/arch/mips/include/asm/smp.h b/arch/mips/include/asm/smp.h
index 0c534a03bb36..8bc6c70a4030 100644
--- a/arch/mips/include/asm/smp.h
+++ b/arch/mips/include/asm/smp.h
@@ -23,7 +23,7 @@
 extern int smp_num_siblings;
 extern cpumask_t cpu_sibling_map[];
 extern cpumask_t cpu_core_map[];
-extern cpumask_t cpu_foreign_map;
+extern cpumask_t cpu_foreign_map[];
 
 #define raw_smp_processor_id() (current_thread_info()->cpu)
 
diff --git a/arch/mips/kernel/smp.c b/arch/mips/kernel/smp.c
index 783d5f50ab9d..f95f094f36e4 100644
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -72,7 +72,7 @@ EXPORT_SYMBOL(cpu_core_map);
  * A logcal cpu mask containing only one VPE per core to
  * reduce the number of IPIs on large MT systems.
  */
-cpumask_t cpu_foreign_map __read_mostly;
+cpumask_t cpu_foreign_map[NR_CPUS] __read_mostly;
 EXPORT_SYMBOL(cpu_foreign_map);
 
 /* representing cpus for which sibling maps can be computed */
@@ -141,7 +141,9 @@ void calculate_cpu_foreign_map(void)
 			cpumask_set_cpu(i, &temp_foreign_map);
 	}
 
-	cpumask_copy(&cpu_foreign_map, &temp_foreign_map);
+	for_each_online_cpu(i)
+		cpumask_andnot(&cpu_foreign_map[i],
+			       &temp_foreign_map, &cpu_sibling_map[i]);
 }
 
 struct plat_smp_ops *mp_ops;
diff --git a/arch/mips/mm/c-r4k.c b/arch/mips/mm/c-r4k.c
index 2a4bb5057ebc..57374f0c33f2 100644
--- a/arch/mips/mm/c-r4k.c
+++ b/arch/mips/mm/c-r4k.c
@@ -56,7 +56,9 @@
  * @type:	Type of cache operations (R4K_HIT or R4K_INDEX).
  *
  * Decides whether a cache op needs to be performed on every core in the system.
- * This may change depending on the @type of cache operation.
+ * This may change depending on the @type of cache operation, as well as the set
+ * of online CPUs, so preemption should be disabled by the caller to prevent CPU
+ * hotplug from changing the result.
  *
  * Returns:	1 if the cache operation @type should be done on every core in
  *		the system.
@@ -71,9 +73,15 @@ static inline bool r4k_op_needs_ipi(unsigned int type)
 
 	/*
 	 * Hardware doesn't globalize the required cache ops, so SMP calls may
-	 * be needed.
+	 * be needed, but only if there are foreign CPUs (non-siblings with
+	 * separate caches).
 	 */
-	return true;
+	/* cpu_foreign_map[] undeclared when !CONFIG_SMP */
+#ifdef CONFIG_SMP
+	return !cpumask_empty(&cpu_foreign_map[0]);
+#else
+	return false;
+#endif
 }
 
 /*
@@ -90,7 +98,8 @@ static inline void r4k_on_each_cpu(unsigned int type,
 {
 	preempt_disable();
 	if (r4k_op_needs_ipi(type))
-		smp_call_function_many(&cpu_foreign_map, func, info, 1);
+		smp_call_function_many(&cpu_foreign_map[smp_processor_id()],
+				       func, info, 1);
 	func(info);
 	preempt_enable();
 }
-- 
2.4.10

  parent reply	other threads:[~2016-07-13 13:16 UTC|newest]

Thread overview: 28+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-07-13 13:12 [PATCH 00/13] MIPS: c-r4k: SMP cache management fixes James Hogan
2016-07-13 13:12 ` James Hogan
2016-07-13 13:12 ` [PATCH 01/13] MIPS: SMP: Clear ASID without confusing has_valid_asid() James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 02/13] MIPS: SMP: Update cpu_foreign_map on CPU disable James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 03/13] MIPS: SMP: Drop stop_this_cpu() cpu_foreign_map hack James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 04/13] MIPS: c-r4k: Fix protected_writeback_scache_line for EVA James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 05/13] MIPS: c-r4k: Fix sigtramp SMP call to use kmap James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 06/13] MIPS: c-r4k: Avoid dcache flush for sigtramps James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 07/13] MIPS: c-r4k: Add r4k_on_each_cpu cache op type arg James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 08/13] MIPS: c-r4k: Fix valid ASID optimisation James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` James Hogan [this message]
2016-07-13 13:12   ` [PATCH 09/13] MIPS: c-r4k: Exclude sibling CPUs in SMP calls James Hogan
2016-07-13 13:12 ` [PATCH 10/13] MIPS: c-r4k: Split r4k_flush_kernel_vmap_range() James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 11/13] MIPS: c-r4k: Local flush_icache_range cache op override James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 12/13] MIPS: c-r4k: Avoid small flush_icache_range SMP calls James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 13/13] MIPS: c-r4k: Use SMP calls for CM indexed cache ops James Hogan
2016-07-13 13:12   ` James Hogan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1468415576-12600-10-git-send-email-james.hogan@imgtec.com \
    --to=james.hogan@imgtec.com \
    --cc=jchandra@broadcom.com \
    --cc=leonid.yegoshin@imgtec.com \
    --cc=linux-mips@linux-mips.org \
    --cc=nbd@nbd.name \
    --cc=paul.burton@imgtec.com \
    --cc=ralf@linux-mips.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox