[PATCH v4 0/2] alpha: complete the direct-mm shootdown fixes

[PATCH v4 0/2] alpha: complete the direct-mm shootdown fixes

[Magnus Lindholm]

This is the second of two series. Apply all eight patches of "alpha: fix
stale TLB translations breaking copy-on-write and writeback" v4 first,
then these two patches. The base-commit and prerequisite-patch-id lines
below describe that ordering from for-next at 33465f6ab697.

Patch 1 removes the remote context-clearing loop after the migration
shootdown rendezvous. The callbacks have already updated their own slots;
clearing them remotely can hide a CPU still running the mm.

Patch 2 fixes the mm_users <= 1 shootdown shortcut. A kthread_use_mm()
borrower takes mmgrab(), not mmget(), so mm_users does not count it.
Inspect remote context slots instead of clearing them, and skip IPIs
only when no other CPU has a context. Together the two patches remove
runtime remote writes to those slots.

The full-mm, icache and migration callbacks now test current->mm, like
the page-flush IPI handler. A CPU retaining the mm only lazily clears its
slot instead of publishing a new ASN. The migration callback still does
the per-VA tbi() afterwards. Returning to the mm allocates a new ASN
through ev5_switch_mm(), or through the direct-load path for an explicit
borrow by current.

The caller-side tests in flush_tlb_mm() and flush_icache_user_page()
remain current->active_mm. A lazy caller can publish its own slot, but
it also performs the shortcut check, which excludes the calling CPU.
The claim about retiring lazy slots applies to the callbacks; it does
not mean that every path now tests current->mm.

Context publication from ev5_switch_mm() is ordered before use of the
mm by the mb() in Alpha's arch_spin_unlock(), when finish_lock_switch()
drops the rq lock. This is an Alpha property, not a generic guarantee
that unlocking the scheduler lock is a full barrier. A comment at the
store now records that dependency. The smp_mb() added to
__load_new_mm_context() in v3 remains necessary for publications from
check_mmu_context(), which runs after the scheduler has dropped the lock.

Changes since v3:

  - Drop the stale v2 Message-ID trailer from patch 1. Its code is
    unchanged and Matt's Reviewed-by is retained.
  - Test current->mm in ipi_flush_mm_and_page(), so migration callbacks
    also retire lazy slots rather than repopulating them on each page.
  - Explain why the caller-side active_mm tests remain unchanged.
  - Name Alpha's arch_spin_unlock() mb() in the ordering argument and
    document the dependency next to the ev5_switch_mm() publication.
  - Add Matt's Reviewed-by to patch 2 with the migration-handler change
    he requested as the condition for that tag.

Testing

Fresh fork-throughput measurements and an SMP run under compaction are
still pending for this revision. Targeted Alpha SMP, UP and SMP with
compaction builds passed, as did a full ES40 CONFIG_COMPACTION=y build. The earlier fork numbers do not measure
these handler changes and are not presented as results for v4.

For comparison only, on 2026-10-09 the preceding MMU v3 plus stale-TLB v4
passed nine lost-store rounds, nine stale-read rounds and ten writeback
rounds on an ES40, EV68AL Tsunami, with four CPUs online and
CONFIG_COMPACTION=n. No new kernel errors were observed. That run did not
exercise the migration callback changed here and is not validation of
this revision under compaction.

The direct context-load fix moved to stale-TLB v4 patch 1. Its existing
review/test tags remain there; it is not duplicated in this series.
Both patches here retain Cc: stable, including the migration cleanup as
the shortcut's prerequisite.

Thanks to Matt Turner for the review and EV7 testing.

Earlier revision:
  v2: https://lore.kernel.org/linux-alpha/20261008195608.965266-1-linmag7@gmail.com/
  v3: https://lore.kernel.org/linux-alpha/20261009212501.971158-1-linmag7@gmail.com/

Magnus Lindholm (2):
  alpha: do not clear remote MMU contexts in migrate_flush_tlb_page()
  alpha: do not skip TLB shootdown IPIs for kthread-borrowed mms

 arch/alpha/include/asm/mmu_context.h |  7 +++-
 arch/alpha/kernel/smp.c              | 52 ++++++++++++++--------------
 arch/alpha/mm/fault.c                |  4 ++-
 arch/alpha/mm/tlbflush.c             | 18 +---------
 4 files changed, 36 insertions(+), 45 deletions(-)


base-commit: 33465f6ab697abceafd9a340067a544d551c4412
prerequisite-patch-id: d194ee1170a45793fa8533400dbc5599fc0e59ae
prerequisite-patch-id: 17df30a2da268e1bd07c7b44de4ef061cb744c50
prerequisite-patch-id: 94856bd5734d96521ef19e59cdb77af1d577fefa
prerequisite-patch-id: 7326de756f1a933cd3c44e4f7669c5a7c4519678
prerequisite-patch-id: 9e39c93f5b7267d1e8ffd176988803681ba03eeb
prerequisite-patch-id: c3f48cfa74e60b966610f88f73f7a393b5adf0b2
prerequisite-patch-id: 73a4faf81a8a0929ba8ba03a6d9a3286e0d9bd2b
prerequisite-patch-id: 55fb49cef84ea661b5340f14c56d73c04556f565
-- 
2.43.0

[PATCH v4 1/2] alpha: do not clear remote MMU contexts in migrate_flush_tlb_page()

[Magnus Lindholm]

After the on_each_cpu() rendezvous, migrate_flush_tlb_page() walks the
other CPUs and zeroes their mm->context[cpu] when mm_users is at most one,
described as mimicking flush_tlb_mm()'s mm_users<=1 optimization.

It is not one. flush_tlb_mm() tests mm_users before deciding whether to
send the IPIs; here every CPU has already been visited and waited for, so
nothing is saved. The callback has also just set each CPU's own slot
correctly, so the loop only overwrites it, and does so while that CPU may
still be running in the address space.

Remove it. The shootdown shortcuts in flush_tlb_mm(), flush_tlb_page()
and flush_icache_user_page() clear remote slots too; once the next patch
removes those as well, every runtime update of mm->context[cpu] is made by
CPU cpu itself. That patch relies on the invariant when it reads those
slots to decide whether a shootdown can be skipped, so this one is tagged
for stable as its prerequisite.

Fixes: dd5712f3379c ("alpha: fix user-space corruption during memory compaction")
Cc: stable@vger.kernel.org
Reviewed-by: Matt Turner <mattst88@gmail.com>
Signed-off-by: Magnus Lindholm <linmag7@gmail.com>
---
 arch/alpha/mm/tlbflush.c | 16 ----------------
 1 file changed, 16 deletions(-)

diff --git a/arch/alpha/mm/tlbflush.c b/arch/alpha/mm/tlbflush.c
index ccbc317b9a34..239c72b8a741 100644
--- a/arch/alpha/mm/tlbflush.c
+++ b/arch/alpha/mm/tlbflush.c
@@ -90,22 +90,6 @@ void migrate_flush_tlb_page(struct vm_area_struct *vma, unsigned long addr)
 	 */
 	preempt_disable();
 	on_each_cpu(ipi_flush_mm_and_page, &d, 1);
-
-	/*
-	 * mimic flush_tlb_mm()'s mm_users<=1 optimization.
-	 */
-	if (atomic_read(&mm->mm_users) <= 1) {
-
-		int cpu, this_cpu;
-		this_cpu = smp_processor_id();
-
-		for (cpu = 0; cpu < NR_CPUS; cpu++) {
-			if (!cpu_online(cpu) || cpu == this_cpu)
-				continue;
-			if (READ_ONCE(mm->context[cpu]))
-				WRITE_ONCE(mm->context[cpu], 0);
-		}
-	}
 	preempt_enable();
 }
 
-- 
2.43.0

[PATCH v4 2/2] alpha: do not skip TLB shootdown IPIs for kthread-borrowed mms

[Magnus Lindholm]

flush_tlb_mm(), flush_tlb_page() and flush_icache_user_page() skip the
shootdown IPI when mm_users <= 1, on the assumption that no other CPU can
be running the mm. A task that borrows an mm through kthread_use_mm()
takes mmgrab() rather than mmget(), so it never appears in mm_users, and
kthread_use_mm() may be handed an mm that is already the caller's
active_mm and loaded on that CPU. Such a CPU was not only left without
an IPI, its mm->context[cpu] was cleared from under it.

mm->context[cpu] is already the record of which CPUs hold an ASN for the
mm. Test it rather than clearing it: take the shortcut only when no
other CPU has one, and otherwise fall through to the IPI, which
invalidates those CPUs properly. A CPU that merely ran the mm in the
past also holds a context and now costs an IPI, which errs in the safe
direction.

Change ipi_flush_tlb_mm(), ipi_flush_icache_page() and the migration
callback ipi_flush_mm_and_page() to test current->mm, like
ipi_flush_tlb_page(). A CPU that only retains the mm lazily then clears
its own slot through flush_tlb_other(), rather than publishing a new ASN
on every IPI or migration rendezvous. The migration callback still does
its per-VA tbi() afterwards. Returning to the mm allocates a fresh ASN
through ev5_switch_mm(), or through the direct-load path when
kthread_use_mm() switches current's mm.

Once these historical slots are gone, the shortcut is available again
if mm_users remains at most one and no other CPU has taken or kept a
context. An explicit kthread_use_mm() borrower has current->mm set and
still receives the invalidate it needs.

The caller-side tests in flush_tlb_mm() and flush_icache_user_page()
remain current->active_mm. A kernel thread that flushes its lazily held
mm can therefore publish its own context slot. Those branches also
perform the shortcut check, which excludes the calling CPU's slot, so
leave them unchanged. This is distinct from a remote lazy CPU keeping
its slot populated through repeated callbacks.

Together with the preceding patch, dropping these clearing loops leaves
every runtime write to mm->context[] targeting the writing CPU's own slot,
so the array becomes a lockless publication of which CPUs may be using the
mm, with a single writer per slot. Mark the two stores that are now
observed from other CPUs; flush_tlb_other() already uses WRITE_ONCE().
The uniprocessor flush_icache_user_page() is left alone, as nothing reads
another CPU's slot there, and init_new_context() runs before the mm is
shared.

Order the slot reads after the page-table changes with smp_mb(). On the
publishing CPU, the store in ev5_switch_mm() precedes the mb() in Alpha's
arch_spin_unlock(), when finish_lock_switch() drops the rq lock. This
relies on Alpha's unlock implementation, not a generic scheduler promise
of a full barrier on unlock; document that reliance at the store.
Also issue smp_mb() after
publishing in __load_new_mm_context(), before loading and using the mm.
This covers every caller, including check_mmu_context(), which can
publish a replacement after finish_lock_switch() has already run.

This requires the preceding removal of remote slot clearing from
migrate_flush_tlb_page(). It also requires the stale-TLB series to set
need_new_asn on both the reused and newly allocated ASN paths. Otherwise
an IPI between finish_lock_switch() and check_mmu_context() can zero a
new slot while asn_lock is set, and the task can return to user space
with a live ASN that the shortcut cannot see.

Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2")
Cc: stable@vger.kernel.org
Reviewed-by: Matt Turner <mattst88@gmail.com>
Signed-off-by: Magnus Lindholm <linmag7@gmail.com>
---
 arch/alpha/include/asm/mmu_context.h |  7 +++-
 arch/alpha/kernel/smp.c              | 52 ++++++++++++++--------------
 arch/alpha/mm/fault.c                |  4 ++-
 arch/alpha/mm/tlbflush.c             |  2 +-
 4 files changed, 36 insertions(+), 29 deletions(-)

diff --git a/arch/alpha/include/asm/mmu_context.h b/arch/alpha/include/asm/mmu_context.h
index e5a5737506db..46b9b82f3895 100644
--- a/arch/alpha/include/asm/mmu_context.h
+++ b/arch/alpha/include/asm/mmu_context.h
@@ -158,7 +158,12 @@ ev5_switch_mm(struct mm_struct *prev_mm, struct mm_struct *next_mm,
 	mmc = next_mm->context[cpu];
 	if ((mmc ^ asn) & ~HARDWARE_ASN_MASK) {
 		mmc = __get_new_mm_context(next_mm, cpu);
-		next_mm->context[cpu] = mmc;
+		/*
+		 * Ordered before any use of the mm by the mb() in
+		 * arch_spin_unlock(), when finish_lock_switch() drops
+		 * the rq lock.
+		 */
+		WRITE_ONCE(next_mm->context[cpu], mmc);
 	}
 #ifdef CONFIG_SMP
 	/* A deferred shootdown can also invalidate a newly allocated ASN. */
diff --git a/arch/alpha/kernel/smp.c b/arch/alpha/kernel/smp.c
index e21bc3920bec..11768aa292a4 100644
--- a/arch/alpha/kernel/smp.c
+++ b/arch/alpha/kernel/smp.c
@@ -625,12 +625,30 @@ static void
 ipi_flush_tlb_mm(void *x)
 {
 	struct mm_struct *mm = x;
-	if (mm == current->active_mm && !asn_locked())
+	if (mm == current->mm && !asn_locked())
 		flush_tlb_current(mm);
 	else
 		flush_tlb_other(mm);
 }
 
+/* True if a CPU other than this one holds an ASN for MM.  */
+static bool
+mm_context_elsewhere(struct mm_struct *mm)
+{
+	int cpu, this_cpu = smp_processor_id();
+
+	/* Pairs with the barrier the publishing CPU issues before using MM.  */
+	smp_mb();
+
+	for_each_online_cpu(cpu) {
+		if (cpu == this_cpu)
+			continue;
+		if (READ_ONCE(mm->context[cpu]))
+			return true;
+	}
+	return false;
+}
+
 void
 flush_tlb_mm(struct mm_struct *mm)
 {
@@ -638,14 +656,8 @@ flush_tlb_mm(struct mm_struct *mm)
 
 	if (mm == current->active_mm) {
 		flush_tlb_current(mm);
-		if (atomic_read(&mm->mm_users) <= 1) {
-			int cpu, this_cpu = smp_processor_id();
-			for (cpu = 0; cpu < NR_CPUS; cpu++) {
-				if (!cpu_online(cpu) || cpu == this_cpu)
-					continue;
-				if (mm->context[cpu])
-					mm->context[cpu] = 0;
-			}
+		if (atomic_read(&mm->mm_users) <= 1 &&
+		    !mm_context_elsewhere(mm)) {
 			preempt_enable();
 			return;
 		}
@@ -690,14 +702,8 @@ flush_tlb_page(struct vm_area_struct *vma, unsigned long addr)
 	/* As in ipi_flush_tlb_page(): a targeted tbi() needs MM current.  */
 	if (mm == current->mm) {
 		flush_tlb_current_page(mm, vma, addr);
-		if (atomic_read(&mm->mm_users) <= 1) {
-			int cpu, this_cpu = smp_processor_id();
-			for (cpu = 0; cpu < NR_CPUS; cpu++) {
-				if (!cpu_online(cpu) || cpu == this_cpu)
-					continue;
-				if (mm->context[cpu])
-					mm->context[cpu] = 0;
-			}
+		if (atomic_read(&mm->mm_users) <= 1 &&
+		    !mm_context_elsewhere(mm)) {
 			preempt_enable();
 			return;
 		}
@@ -728,7 +734,7 @@ static void
 ipi_flush_icache_page(void *x)
 {
 	struct mm_struct *mm = (struct mm_struct *) x;
-	if (mm == current->active_mm && !asn_locked())
+	if (mm == current->mm && !asn_locked())
 		__load_new_mm_context(mm);
 	else
 		flush_tlb_other(mm);
@@ -747,14 +753,8 @@ flush_icache_user_page(struct vm_area_struct *vma, struct page *page,
 
 	if (mm == current->active_mm) {
 		__load_new_mm_context(mm);
-		if (atomic_read(&mm->mm_users) <= 1) {
-			int cpu, this_cpu = smp_processor_id();
-			for (cpu = 0; cpu < NR_CPUS; cpu++) {
-				if (!cpu_online(cpu) || cpu == this_cpu)
-					continue;
-				if (mm->context[cpu])
-					mm->context[cpu] = 0;
-			}
+		if (atomic_read(&mm->mm_users) <= 1 &&
+		    !mm_context_elsewhere(mm)) {
 			preempt_enable();
 			return;
 		}
diff --git a/arch/alpha/mm/fault.c b/arch/alpha/mm/fault.c
index dfe427d93072..5b62bdeabe2a 100644
--- a/arch/alpha/mm/fault.c
+++ b/arch/alpha/mm/fault.c
@@ -69,7 +69,9 @@ __load_new_mm_context(struct mm_struct *next_mm)
 	struct pcb_struct *pcb;
 
 	mmc = __get_new_mm_context(next_mm, smp_processor_id());
-	next_mm->context[smp_processor_id()] = mmc;
+	WRITE_ONCE(next_mm->context[smp_processor_id()], mmc);
+	/* Publish the context before this CPU can access the mm. */
+	smp_mb();
 
 	pcb = &current_thread_info()->pcb;
 	pcb->asn = mmc & HARDWARE_ASN_MASK;
diff --git a/arch/alpha/mm/tlbflush.c b/arch/alpha/mm/tlbflush.c
index 239c72b8a741..a2aef90e6234 100644
--- a/arch/alpha/mm/tlbflush.c
+++ b/arch/alpha/mm/tlbflush.c
@@ -66,7 +66,7 @@ static void ipi_flush_mm_and_page(void *x)
 	struct tlb_mm_and_addr *d = x;
 
 	/* Part 1: mm context side (Alpha uses ASN/context as a key mechanism). */
-	if (d->mm == current->active_mm && !asn_locked())
+	if (d->mm == current->mm && !asn_locked())
 		__load_new_mm_context(d->mm);
 	else
 		flush_tlb_other(d->mm);
-- 
2.43.0

Re: [PATCH v4 2/2] alpha: do not skip TLB shootdown IPIs for kthread-borrowed mms

[Matt Turner]

On Sat, Oct 10, 2026 at 16:33 +0200, Magnus Lindholm wrote:
> This requires the preceding removal of remote slot clearing from
> migrate_flush_tlb_page(). It also requires the stale-TLB series to set
> need_new_asn on both the reused and newly allocated ASN paths.
[...]
> Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2")
> Cc: stable@vger.kernel.org

With that Fixes tag this will be tried on every stable tree, and the
dependency is only in the prose. The hunk context includes comments the
stale-TLB series adds, so it will fail to apply rather than misapply,
which is the safe outcome. Naming the prerequisites on the Cc: stable
line would still save a round of FAILED mails.

> +             /*
> +              * Ordered before any use of the mm by the mb() in
> +              * arch_spin_unlock(), when finish_lock_switch() drops
> +              * the rq lock.
> +              */
> +             WRITE_ONCE(next_mm->context[cpu], mmc);

"Any use of the mm" is slightly too strong: swpctx loads the new ptbr
and ASN in switch_to(), before the unlock. What the ordering has to
precede is the first TLB fill for a user address, so maybe:

                /*
                 * Ordered before any user access through the mm by the
                 * mb() in arch_spin_unlock(), when finish_lock_switch()
                 * drops the rq lock.
                 */

Fine as a fixup, no need for a v5 on my account.

> +             if (atomic_read(&mm->mm_users) <= 1 &&
> +                 !mm_context_elsewhere(mm)) {

Not for this series: once every slot has a single writer, I think the
slot test is sufficient by itself and mm_users only narrows the
shortcut. Dropping it later would extend the shortcut to a
multithreaded process whose threads all sit on one CPU. Worth a look
after this has had some soak time.

Nit: the changelog paragraph starting "Order the slot reads" has a short
line at "Also issue smp_mb() after" that wants refilling.

Re: [PATCH v4 2/2] alpha: do not skip TLB shootdown IPIs for kthread-borrowed mms

[Magnus Lindholm]

Hi Matt,

On Sun, Oct 11, 2026 at 1:34 AM Matt Turner <mattst88@gmail.com> wrote:
>
> On Sat, Oct 10, 2026 at 16:33 +0200, Magnus Lindholm wrote:
> > This requires the preceding removal of remote slot clearing from
> > migrate_flush_tlb_page(). It also requires the stale-TLB series to set
> > need_new_asn on both the reused and newly allocated ASN paths.
> [...]
> > Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2")
> > Cc: stable@vger.kernel.org
>
> With that Fixes tag this will be tried on every stable tree, and the
> dependency is only in the prose. The hunk context includes comments the
> stale-TLB series adds, so it will fail to apply rather than misapply,
> which is the safe outcome. Naming the prerequisites on the Cc: stable
> line would still save a round of FAILED mails.
>

Thanks, fixed in for-next. I've added the prerequisite commits to the
Cc: stable annotations.


> > +             /*
> > +              * Ordered before any use of the mm by the mb() in
> > +              * arch_spin_unlock(), when finish_lock_switch() drops
> > +              * the rq lock.
> > +              */
> > +             WRITE_ONCE(next_mm->context[cpu], mmc);
>
> "Any use of the mm" is slightly too strong: swpctx loads the new ptbr
> and ASN in switch_to(), before the unlock. What the ordering has to
> precede is the first TLB fill for a user address, so maybe:
>
>                 /*
>                  * Ordered before any user access through the mm by the
>                  * mb() in arch_spin_unlock(), when finish_lock_switch()
>                  * drops the rq lock.
>                  */
>
> Fine as a fixup, no need for a v5 on my account.
>

I've changed the comment to "before any user access
through the mm" as suggested, and refilled the changelog paragraph.
No functional changes, your Reviewed-by is retained.

> > +             if (atomic_read(&mm->mm_users) <= 1 &&
> > +                 !mm_context_elsewhere(mm)) {
>
> Not for this series: once every slot has a single writer, I think the
> slot test is sufficient by itself and mm_users only narrows the
> shortcut. Dropping it later would extend the shortcut to a
> multithreaded process whose threads all sit on one CPU. Worth a look
> after this has had some soak time.
>

Interesting, that is for sure worth looking into.

>
> Nit: the changelog paragraph starting "Order the slot reads" has a short
> line at "Also issue smp_mb() after" that wants refilling.

The combined TLB v5 and MMU v4 stack has now completed an SMP compaction
soak on the ES40 with three CPUs online and CONFIG_COMPACTION=y.

Regards

Magnus

Re: [PATCH v4 0/2] alpha: complete the direct-mm shootdown fixes

[Matt Turner]

On Sat, Oct 10, 2026 at 16:33 +0200, Magnus Lindholm wrote:
> This is the second of two series. Apply all eight patches of "alpha: fix
> stale TLB translations breaking copy-on-write and writeback" v4 first,
> then these two patches. The base-commit and prerequisite-patch-id lines
> below describe that ordering from for-next at 33465f6ab697.

The first two prerequisite-patch-id lines (d194ee1170a4, 17df30a2da26) do
not match what is in for-next. for-next carries the v5 versions of "load
the MMU context when switch_mm() switches the current task" and "run
check_mmu_context() from finish_arch_post_lock_switch()". The other six
match, and both patches here apply unchanged on top, so this only matters
to someone following the cover letter literally.

> Fresh fork-throughput measurements and an SMP run under compaction are
> still pending for this revision. Targeted Alpha SMP, UP and SMP with
> compaction builds passed, as did a full ES40 CONFIG_COMPACTION=y build.

Thanks for being explicit about this. The current->mm test in
ipi_flush_mm_and_page() is the one change in v4 that no run has
exercised, and the series is in for-next already. I would like to see an
SMP soak with CONFIG_COMPACTION=y and compaction actually triggered
before this goes further.

I went through v4 again and both tags stand. In particular I checked
that the only caller reaching the ev5_switch_mm() store is
context_switch(); kthread_use_mm(), sched_force_init_mm() and
idle_task_exit() all pass current and take the __load_new_mm_context()
branch, so the arch_spin_unlock() argument covers every publication from
that store. Two small comments on patch 2.