Re: [PATCH 3/4] arm64: errata: Work around early CME DVMSync acknowledgement

Catalin Marinas
2026-03-06T12:19:16Z

On Fri, Mar 06, 2026 at 12:00:30PM +0000, Catalin Marinas wrote:
> On Thu, Mar 05, 2026 at 02:32:11PM +0000, Will Deacon wrote:
> > On Mon, Mar 02, 2026 at 04:57:56PM +0000, Catalin Marinas wrote:
> > > +void sme_do_dvmsync(void)
> > > +{
> > > + /*
> > > + * This is called from the TLB maintenance functions after the DSB ISH
> > > + * to send hardware DVMSync message. If this CPU sees the mask as
> > > + * empty, the remote CPU executing sme_set_active() would have seen
> > > + * the DVMSync and no IPI required.
> > > + */
> > > + if (cpumask_empty(sme_active_cpus))
> > > + return;
> > > +
> > > + preempt_disable();
> > > + smp_call_function_many(sme_active_cpus, sme_dvmsync_ipi, NULL, true);
> > > + preempt_enable();
> > > +}
> >
> > Why do we care about all CPUs using SME, rather than limiting it to the
> > set of CPUs using SME with the mm we've invalidated? This looks like it
> > will result in unnecessary cross-calls when multiple tasks are using SME
> > (especially as the mm flag is only cleared on fork).
>
> Yes, it's a possibility but I traded it for simplicity. We also have the
> TTU case where we don't have an mm and we don't want to broadcast to all
> CPUs either, hence an sme_active_cpus mask. As I just replied on patch
> 2, for the TLB batching we wouldn't be able to use a cpumask in the
> batching structure since, per the ordering above, we need the DVMSync
> before checking if/where to send the IPI to.
>
> For the typical TLBI (not TTU), we can track a per-mm mask passed down
> to this function (I have patches doing this but it didn't make a
> significant difference in benchmarks).

Reusing the current mm_cpumask(), something like below. We could also
scrap the MMCF_SME_DVMSYNC flag, though we end up always call
sme_do_dvmsync() and checking the mask, probably more expensive than a
flag check.

diff --git a/arch/arm64/include/asm/tlbflush.h b/arch/arm64/include/asm/tlbflush.h
index e3ea0246a4f4..2c77ca41cb14 100644
--- a/arch/arm64/include/asm/tlbflush.h
+++ b/arch/arm64/include/asm/tlbflush.h
@@ -81,7 +81,7 @@ static inline unsigned long get_trans_granule(void)
}

#ifdef CONFIG_ARM64_ERRATUM_SME_DVMSYNC
-void sme_do_dvmsync(void);
+void sme_do_dvmsync(struct mm_struct *mm);

static inline void sme_dvmsync(struct mm_struct *mm)
{
@@ -90,7 +90,7 @@ static inline void sme_dvmsync(struct mm_struct *mm)
if (mm && !test_bit(ilog2(MMCF_SME_DVMSYNC), &mm->context.flags))
return;

- sme_do_dvmsync();
+ sme_do_dvmsync(mm);
}
#else
static inline void sme_dvmsync(struct mm_struct *mm) { }
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index 90015fc29722..37e215cd0f39 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -1378,6 +1378,7 @@ void sme_set_active(unsigned int cpu)
if (!test_bit(ilog2(MMCF_SME_DVMSYNC), &current->mm->context.flags))
set_bit(ilog2(MMCF_SME_DVMSYNC), &current->mm->context.flags);

+ cpumask_set_cpu(cpu, mm_cpumask(current->mm));
cpumask_set_cpu(cpu, sme_active_cpus);

/*
@@ -1398,6 +1399,7 @@ void sme_clear_active(unsigned int cpu)
* With SCTLR_EL1.IESB enabled, the SME memory transactions are
* completed on entering EL1.
*/
+ cpumask_clear_cpu(cpu, mm_cpumask(current->mm));
cpumask_clear_cpu(cpu, sme_active_cpus);
}

@@ -1410,19 +1412,25 @@ static void sme_dvmsync_ipi(void *unused)
*/
}

-void sme_do_dvmsync(void)
+void sme_do_dvmsync(struct mm_struct *mm)
{
/*
* This is called from the TLB maintenance functions after the DSB ISH
* to send hardware DVMSync message. If this CPU sees the mask as
* empty, the remote CPU executing sme_set_active() would have seen
* the DVMSync and no IPI required.
+ *
+ * When an mm is provided, limit the IPI to CPUs that are actively
+ * running SME code for that mm (recorded in mm_cpumask()), otherwise
+ * fall back to the global sme_active_cpus mask.
*/
- if (cpumask_empty(sme_active_cpus))
+ const struct cpumask *mask = mm ? mm_cpumask(mm) : sme_active_cpus;
+
+ if (cpumask_empty(mask))
return;

preempt_disable();
- smp_call_function_many(sme_active_cpus, sme_dvmsync_ipi, NULL, true);
+ smp_call_function_many(mask, sme_dvmsync_ipi, NULL, true);
preempt_enable();
}