Linux MIPS Architecture development
 help / color / mirror / Atom feed
From: James Hogan <james.hogan@imgtec.com>
To: Ralf Baechle <ralf@linux-mips.org>
Cc: James Hogan <james.hogan@imgtec.com>, <linux-mips@linux-mips.org>,
	Paul Burton <paul.burton@imgtec.com>,
	Leonid Yegoshin <leonid.yegoshin@imgtec.com>
Subject: [PATCH 03/13] MIPS: SMP: Drop stop_this_cpu() cpu_foreign_map hack
Date: Wed, 13 Jul 2016 14:12:46 +0100	[thread overview]
Message-ID: <1468415576-12600-4-git-send-email-james.hogan@imgtec.com> (raw)
In-Reply-To: <1468415576-12600-1-git-send-email-james.hogan@imgtec.com>

Commit cccf34e9411c ("MIPS: c-r4k: Fix cache flushing for MT cores")
added the cpu_foreign_map cpumask containing a single VPE from each
online core, and recalculated it when secondary CPUs are brought up.

stop_this_cpu() was also updated to recalculate cpu_foreign_map, but
with an additional hack before marking the CPU as offline to copy
cpu_online_mask into cpu_foreign_map and perform an SMP memory barrier.

This appears to have been intended to prevent cache management IPIs
being missed when the VPE representing the core in cpu_foreign_map is
taken offline while other VPEs remain online. Unfortunately there is
nothing in this hack to prevent r4k_on_each_cpu() from reading the old
cpu_foreign_map, and smp_call_function_many() from reading that new
cpu_online_mask with the core's representative VPE marked offline. It
then wouldn't send an IPI to any online VPEs of that core.

stop_this_cpu() is only actually called in panic and system shutdown /
halt / reboot situations, in which case all CPUs are going down and we
don't really need to care about cache management, so drop this hack.

Note that the __cpu_disable() case for CPU hotplug is handled in the
previous commit, and no synchronisation is needed there due to the use
of stop_machine() which prevents hotplug from taking place while any CPU
has disabled preemption (as r4k_on_each_cpu() does).

Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com>
---
 arch/mips/kernel/smp.c | 9 +--------
 1 file changed, 1 insertion(+), 8 deletions(-)

diff --git a/arch/mips/kernel/smp.c b/arch/mips/kernel/smp.c
index a4d4309ecff2..783d5f50ab9d 100644
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -344,16 +344,9 @@ asmlinkage void start_secondary(void)
 static void stop_this_cpu(void *dummy)
 {
 	/*
-	 * Remove this CPU. Be a bit slow here and
-	 * set the bits for every online CPU so we don't miss
-	 * any IPI whilst taking this VPE down.
+	 * Remove this CPU:
 	 */
 
-	cpumask_copy(&cpu_foreign_map, cpu_online_mask);
-
-	/* Make it visible to every other CPU */
-	smp_mb();
-
 	set_cpu_online(smp_processor_id(), false);
 	calculate_cpu_foreign_map();
 	local_irq_disable();
-- 
2.4.10

WARNING: multiple messages have this Message-ID (diff)
From: James Hogan <james.hogan@imgtec.com>
To: Ralf Baechle <ralf@linux-mips.org>
Cc: James Hogan <james.hogan@imgtec.com>,
	linux-mips@linux-mips.org, Paul Burton <paul.burton@imgtec.com>,
	Leonid Yegoshin <leonid.yegoshin@imgtec.com>
Subject: [PATCH 03/13] MIPS: SMP: Drop stop_this_cpu() cpu_foreign_map hack
Date: Wed, 13 Jul 2016 14:12:46 +0100	[thread overview]
Message-ID: <1468415576-12600-4-git-send-email-james.hogan@imgtec.com> (raw)
Message-ID: <20160713131246.w9zH1XCQPrWzirBxYpOgKIZD9heLViFvtsbvQodAEJg@z> (raw)
In-Reply-To: <1468415576-12600-1-git-send-email-james.hogan@imgtec.com>

Commit cccf34e9411c ("MIPS: c-r4k: Fix cache flushing for MT cores")
added the cpu_foreign_map cpumask containing a single VPE from each
online core, and recalculated it when secondary CPUs are brought up.

stop_this_cpu() was also updated to recalculate cpu_foreign_map, but
with an additional hack before marking the CPU as offline to copy
cpu_online_mask into cpu_foreign_map and perform an SMP memory barrier.

This appears to have been intended to prevent cache management IPIs
being missed when the VPE representing the core in cpu_foreign_map is
taken offline while other VPEs remain online. Unfortunately there is
nothing in this hack to prevent r4k_on_each_cpu() from reading the old
cpu_foreign_map, and smp_call_function_many() from reading that new
cpu_online_mask with the core's representative VPE marked offline. It
then wouldn't send an IPI to any online VPEs of that core.

stop_this_cpu() is only actually called in panic and system shutdown /
halt / reboot situations, in which case all CPUs are going down and we
don't really need to care about cache management, so drop this hack.

Note that the __cpu_disable() case for CPU hotplug is handled in the
previous commit, and no synchronisation is needed there due to the use
of stop_machine() which prevents hotplug from taking place while any CPU
has disabled preemption (as r4k_on_each_cpu() does).

Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com>
---
 arch/mips/kernel/smp.c | 9 +--------
 1 file changed, 1 insertion(+), 8 deletions(-)

diff --git a/arch/mips/kernel/smp.c b/arch/mips/kernel/smp.c
index a4d4309ecff2..783d5f50ab9d 100644
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -344,16 +344,9 @@ asmlinkage void start_secondary(void)
 static void stop_this_cpu(void *dummy)
 {
 	/*
-	 * Remove this CPU. Be a bit slow here and
-	 * set the bits for every online CPU so we don't miss
-	 * any IPI whilst taking this VPE down.
+	 * Remove this CPU:
 	 */
 
-	cpumask_copy(&cpu_foreign_map, cpu_online_mask);
-
-	/* Make it visible to every other CPU */
-	smp_mb();
-
 	set_cpu_online(smp_processor_id(), false);
 	calculate_cpu_foreign_map();
 	local_irq_disable();
-- 
2.4.10

  parent reply	other threads:[~2016-07-13 13:13 UTC|newest]

Thread overview: 28+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-07-13 13:12 [PATCH 00/13] MIPS: c-r4k: SMP cache management fixes James Hogan
2016-07-13 13:12 ` James Hogan
2016-07-13 13:12 ` [PATCH 01/13] MIPS: SMP: Clear ASID without confusing has_valid_asid() James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 02/13] MIPS: SMP: Update cpu_foreign_map on CPU disable James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` James Hogan [this message]
2016-07-13 13:12   ` [PATCH 03/13] MIPS: SMP: Drop stop_this_cpu() cpu_foreign_map hack James Hogan
2016-07-13 13:12 ` [PATCH 04/13] MIPS: c-r4k: Fix protected_writeback_scache_line for EVA James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 05/13] MIPS: c-r4k: Fix sigtramp SMP call to use kmap James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 06/13] MIPS: c-r4k: Avoid dcache flush for sigtramps James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 07/13] MIPS: c-r4k: Add r4k_on_each_cpu cache op type arg James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 08/13] MIPS: c-r4k: Fix valid ASID optimisation James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 09/13] MIPS: c-r4k: Exclude sibling CPUs in SMP calls James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 10/13] MIPS: c-r4k: Split r4k_flush_kernel_vmap_range() James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 11/13] MIPS: c-r4k: Local flush_icache_range cache op override James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 12/13] MIPS: c-r4k: Avoid small flush_icache_range SMP calls James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 13/13] MIPS: c-r4k: Use SMP calls for CM indexed cache ops James Hogan
2016-07-13 13:12   ` James Hogan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1468415576-12600-4-git-send-email-james.hogan@imgtec.com \
    --to=james.hogan@imgtec.com \
    --cc=leonid.yegoshin@imgtec.com \
    --cc=linux-mips@linux-mips.org \
    --cc=paul.burton@imgtec.com \
    --cc=ralf@linux-mips.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox