linux-mips.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: James Hogan <james.hogan@imgtec.com>
To: Ralf Baechle <ralf@linux-mips.org>
Cc: James Hogan <james.hogan@imgtec.com>,
	Paul Burton <paul.burton@imgtec.com>,
	Leonid Yegoshin <leonid.yegoshin@imgtec.com>,
	<linux-mips@linux-mips.org>
Subject: [PATCH 01/13] MIPS: SMP: Clear ASID without confusing has_valid_asid()
Date: Wed, 13 Jul 2016 14:12:44 +0100	[thread overview]
Message-ID: <1468415576-12600-2-git-send-email-james.hogan@imgtec.com> (raw)
In-Reply-To: <1468415576-12600-1-git-send-email-james.hogan@imgtec.com>

The SMP flush_tlb_*() functions may clear the memory map's ASIDs for
other CPUs if the mm has only a single user (the current CPU) in order
to avoid SMP calls. However this makes it appear to has_valid_asid(),
which is used by various cache flush functions, as if the CPUs have
never run in the mm, and therefore can't have cached any of its memory.

For flush_tlb_mm() this doesn't sound unreasonable.

flush_tlb_range() corresponds to flush_cache_range() which does do full
indexed cache flushes, but only on the icache if the specified mapping
is executable, otherwise it doesn't guarantee that there are no cache
contents left for the mm.

flush_tlb_page() corresponds to flush_cache_page(), which will perform
address based cache ops on the specified page only, and also only
touches the icache if the page is executable. It does not guarantee that
there are no cache contents left for the mm.

For example, this affects flush_cache_range() which uses the
has_valid_asid() optimisation. It is required to flush the icache when
mappings are made executable (e.g. using mprotect) so they are
immediately usable. If some code is changed to non executable in order
to be modified then it will not be flushed from the icache during that
time, but the ASID on other CPUs may still be cleared for TLB flushing.
When the code is changed back to executable, flush_cache_range() will
assume the code hasn't run on those other CPUs due to the zero ASID, and
won't invalidate the icache on them.

This is fixed by clearing the other CPUs ASIDs to 1 instead of 0 for the
above two flush_tlb_*() functions when the corresponding cache flushes
are likely to be incomplete (non executable range flush, or any page
flush). This ASID appears valid to has_valid_asid(), but still triggers
ASID regeneration due to the upper ASID version bits being 0, which is
less than the minimum ASID version of 1 and so always treated as stale.

Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com>
Cc: linux-mips@linux-mips.org
---
 arch/mips/kernel/smp.c | 17 +++++++++++++++--
 arch/mips/mm/c-r4k.c   |  4 ++++
 2 files changed, 19 insertions(+), 2 deletions(-)

diff --git a/arch/mips/kernel/smp.c b/arch/mips/kernel/smp.c
index f9d01e953acb..0c98b4a313be 100644
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -512,10 +512,17 @@ void flush_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned l
 		smp_on_other_tlbs(flush_tlb_range_ipi, &fd);
 	} else {
 		unsigned int cpu;
+		int exec = vma->vm_flags & VM_EXEC;
 
 		for_each_online_cpu(cpu) {
+			/*
+			 * flush_cache_range() will only fully flush icache if
+			 * the VMA is executable, otherwise we must invalidate
+			 * ASID without it appearing to has_valid_asid() as if
+			 * mm has been completely unused by that CPU.
+			 */
 			if (cpu != smp_processor_id() && cpu_context(cpu, mm))
-				cpu_context(cpu, mm) = 0;
+				cpu_context(cpu, mm) = !exec;
 		}
 	}
 	local_flush_tlb_range(vma, start, end);
@@ -560,8 +567,14 @@ void flush_tlb_page(struct vm_area_struct *vma, unsigned long page)
 		unsigned int cpu;
 
 		for_each_online_cpu(cpu) {
+			/*
+			 * flush_cache_page() only does partial flushes, so
+			 * invalidate ASID without it appearing to
+			 * has_valid_asid() as if mm has been completely unused
+			 * by that CPU.
+			 */
 			if (cpu != smp_processor_id() && cpu_context(cpu, vma->vm_mm))
-				cpu_context(cpu, vma->vm_mm) = 0;
+				cpu_context(cpu, vma->vm_mm) = 1;
 		}
 	}
 	local_flush_tlb_page(vma, page);
diff --git a/arch/mips/mm/c-r4k.c b/arch/mips/mm/c-r4k.c
index 645c69c95c9c..9204d4e4f02f 100644
--- a/arch/mips/mm/c-r4k.c
+++ b/arch/mips/mm/c-r4k.c
@@ -484,6 +484,10 @@ static void r4k__flush_cache_vunmap(void)
 	r4k_blast_dcache();
 }
 
+/*
+ * Note: flush_tlb_range() assumes flush_cache_range() sufficiently flushes
+ * whole caches when vma is executable.
+ */
 static inline void local_r4k_flush_cache_range(void * args)
 {
 	struct vm_area_struct *vma = args;
-- 
2.4.10

WARNING: multiple messages have this Message-ID (diff)
From: James Hogan <james.hogan@imgtec.com>
To: Ralf Baechle <ralf@linux-mips.org>
Cc: James Hogan <james.hogan@imgtec.com>,
	Paul Burton <paul.burton@imgtec.com>,
	Leonid Yegoshin <leonid.yegoshin@imgtec.com>,
	linux-mips@linux-mips.org
Subject: [PATCH 01/13] MIPS: SMP: Clear ASID without confusing has_valid_asid()
Date: Wed, 13 Jul 2016 14:12:44 +0100	[thread overview]
Message-ID: <1468415576-12600-2-git-send-email-james.hogan@imgtec.com> (raw)
Message-ID: <20160713131244.MJV8BWKdv9erQPtvPD2DxbGVX-lNf5Sh81ddtckWfJ8@z> (raw)
In-Reply-To: <1468415576-12600-1-git-send-email-james.hogan@imgtec.com>

The SMP flush_tlb_*() functions may clear the memory map's ASIDs for
other CPUs if the mm has only a single user (the current CPU) in order
to avoid SMP calls. However this makes it appear to has_valid_asid(),
which is used by various cache flush functions, as if the CPUs have
never run in the mm, and therefore can't have cached any of its memory.

For flush_tlb_mm() this doesn't sound unreasonable.

flush_tlb_range() corresponds to flush_cache_range() which does do full
indexed cache flushes, but only on the icache if the specified mapping
is executable, otherwise it doesn't guarantee that there are no cache
contents left for the mm.

flush_tlb_page() corresponds to flush_cache_page(), which will perform
address based cache ops on the specified page only, and also only
touches the icache if the page is executable. It does not guarantee that
there are no cache contents left for the mm.

For example, this affects flush_cache_range() which uses the
has_valid_asid() optimisation. It is required to flush the icache when
mappings are made executable (e.g. using mprotect) so they are
immediately usable. If some code is changed to non executable in order
to be modified then it will not be flushed from the icache during that
time, but the ASID on other CPUs may still be cleared for TLB flushing.
When the code is changed back to executable, flush_cache_range() will
assume the code hasn't run on those other CPUs due to the zero ASID, and
won't invalidate the icache on them.

This is fixed by clearing the other CPUs ASIDs to 1 instead of 0 for the
above two flush_tlb_*() functions when the corresponding cache flushes
are likely to be incomplete (non executable range flush, or any page
flush). This ASID appears valid to has_valid_asid(), but still triggers
ASID regeneration due to the upper ASID version bits being 0, which is
less than the minimum ASID version of 1 and so always treated as stale.

Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com>
Cc: linux-mips@linux-mips.org
---
 arch/mips/kernel/smp.c | 17 +++++++++++++++--
 arch/mips/mm/c-r4k.c   |  4 ++++
 2 files changed, 19 insertions(+), 2 deletions(-)

diff --git a/arch/mips/kernel/smp.c b/arch/mips/kernel/smp.c
index f9d01e953acb..0c98b4a313be 100644
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -512,10 +512,17 @@ void flush_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned l
 		smp_on_other_tlbs(flush_tlb_range_ipi, &fd);
 	} else {
 		unsigned int cpu;
+		int exec = vma->vm_flags & VM_EXEC;
 
 		for_each_online_cpu(cpu) {
+			/*
+			 * flush_cache_range() will only fully flush icache if
+			 * the VMA is executable, otherwise we must invalidate
+			 * ASID without it appearing to has_valid_asid() as if
+			 * mm has been completely unused by that CPU.
+			 */
 			if (cpu != smp_processor_id() && cpu_context(cpu, mm))
-				cpu_context(cpu, mm) = 0;
+				cpu_context(cpu, mm) = !exec;
 		}
 	}
 	local_flush_tlb_range(vma, start, end);
@@ -560,8 +567,14 @@ void flush_tlb_page(struct vm_area_struct *vma, unsigned long page)
 		unsigned int cpu;
 
 		for_each_online_cpu(cpu) {
+			/*
+			 * flush_cache_page() only does partial flushes, so
+			 * invalidate ASID without it appearing to
+			 * has_valid_asid() as if mm has been completely unused
+			 * by that CPU.
+			 */
 			if (cpu != smp_processor_id() && cpu_context(cpu, vma->vm_mm))
-				cpu_context(cpu, vma->vm_mm) = 0;
+				cpu_context(cpu, vma->vm_mm) = 1;
 		}
 	}
 	local_flush_tlb_page(vma, page);
diff --git a/arch/mips/mm/c-r4k.c b/arch/mips/mm/c-r4k.c
index 645c69c95c9c..9204d4e4f02f 100644
--- a/arch/mips/mm/c-r4k.c
+++ b/arch/mips/mm/c-r4k.c
@@ -484,6 +484,10 @@ static void r4k__flush_cache_vunmap(void)
 	r4k_blast_dcache();
 }
 
+/*
+ * Note: flush_tlb_range() assumes flush_cache_range() sufficiently flushes
+ * whole caches when vma is executable.
+ */
 static inline void local_r4k_flush_cache_range(void * args)
 {
 	struct vm_area_struct *vma = args;
-- 
2.4.10

  parent reply	other threads:[~2016-07-13 13:13 UTC|newest]

Thread overview: 28+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-07-13 13:12 [PATCH 00/13] MIPS: c-r4k: SMP cache management fixes James Hogan
2016-07-13 13:12 ` James Hogan
2016-07-13 13:12 ` James Hogan [this message]
2016-07-13 13:12   ` [PATCH 01/13] MIPS: SMP: Clear ASID without confusing has_valid_asid() James Hogan
2016-07-13 13:12 ` [PATCH 02/13] MIPS: SMP: Update cpu_foreign_map on CPU disable James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 03/13] MIPS: SMP: Drop stop_this_cpu() cpu_foreign_map hack James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 04/13] MIPS: c-r4k: Fix protected_writeback_scache_line for EVA James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 05/13] MIPS: c-r4k: Fix sigtramp SMP call to use kmap James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 06/13] MIPS: c-r4k: Avoid dcache flush for sigtramps James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 07/13] MIPS: c-r4k: Add r4k_on_each_cpu cache op type arg James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 08/13] MIPS: c-r4k: Fix valid ASID optimisation James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 09/13] MIPS: c-r4k: Exclude sibling CPUs in SMP calls James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 10/13] MIPS: c-r4k: Split r4k_flush_kernel_vmap_range() James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 11/13] MIPS: c-r4k: Local flush_icache_range cache op override James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 12/13] MIPS: c-r4k: Avoid small flush_icache_range SMP calls James Hogan
2016-07-13 13:12   ` James Hogan
2016-07-13 13:12 ` [PATCH 13/13] MIPS: c-r4k: Use SMP calls for CM indexed cache ops James Hogan
2016-07-13 13:12   ` James Hogan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1468415576-12600-2-git-send-email-james.hogan@imgtec.com \
    --to=james.hogan@imgtec.com \
    --cc=leonid.yegoshin@imgtec.com \
    --cc=linux-mips@linux-mips.org \
    --cc=paul.burton@imgtec.com \
    --cc=ralf@linux-mips.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).