[PATCH] arm64: errata: Add NXP iMX8QM workaround for A53 Cache coherency issue

[PATCH] arm64: errata: Add NXP iMX8QM workaround for A53 Cache coherency issue

[Ivan T. Ivanov]

According to NXP errata document[1] i.MX8QuadMax SoC suffers from
serious cache coherence issue. It was also mentioned in initial
support[2] for imx8qm mek machine.

I chose to use an ALTERNATIVE() framework, instead downstream solution[3],
for this issue with the hope to reduce effect of this fix on unaffected
platforms.

Unfortunately I was unable to find a way to identify SoC ID using
registers. Boot CPU MIDR_EL1 is equal to 0x410fd034. So I fallback to
using devicetree compatible strings for this.

I know this fix is a suboptimal solution for affected machines, but I
haven't been able to come up with a less intrusive fix.  And I hope once
TLB caches are invalidated any immediate attempt to invalidate them again
will be close to NOP operation (flush_tlb_kernel_range())

I have run few simple benchmarks and perf tests on affected and unaffected
machines and I was not able see any obvious issues. iMX8QM "performance"
was nearly doubled with 2 A72 bringed online.

Following is excerpt from NXP IMX8_1N94W "Mask Set Errata" document
Rev. 5, 3/2023. Just in case it gets lost somehow.

---
"ERR050104: Arm/A53: Cache coherency issue"

Description

Some maintenance operations exchanged between the A53 and A72
core clusters, involving some Translation Look-aside Buffer
Invalidate (TLBI) and Instruction Cache (IC) instructions can
be corrupted. The upper bits, above bit-35, of ARADDR and ACADDR
buses within in Arm A53 sub-system have been incorrectly connected.
Therefore ARADDR and ACADDR address bits above bit-35 should not
be used.

Workaround

The following software instructions are required to be downgraded
to TLBI VMALLE1IS:  TLBI ASIDE1, TLBI ASIDE1IS, TLBI VAAE1,
TLBI VAAE1IS, TLBI VAALE1, TLBI VAALE1IS, TLBI VAE1, TLBI VAE1IS,
TLBI VALE1, TLBI VALE1IS

The following software instructions are required to be downgraded
to TLBI VMALLS12E1IS: TLBI IPAS2E1IS, TLBI IPAS2LE1IS

The following software instructions are required to be downgraded
to TLBI ALLE2IS: TLBI VAE2IS, TLBI VALE2IS.

The following software instructions are required to be downgraded
to TLBI ALLE3IS: TLBI VAE3IS, TLBI VALE3IS.

The following software instructions are required to be downgraded
to TLBI VMALLE1IS when the Force Broadcast (FB) bit [9] of the
Hypervisor Configuration Register (HCR_EL2) is set:
TLBI ASIDE1, TLBI VAAE1, TLBI VAALE1, TLBI VAE1, TLBI VALE1

The following software instruction is required to be downgraded
to IC IALLUIS: IC IVAU, Xt

Specifically for the IC IVAU, Xt downgrade, setting SCTLR_EL1.UCI
to 0 will disable EL0 access to this instruction. Any attempt to
execute from EL0 will generate an EL1 trap, where the downgrade to
IC ALLUIS can be implemented.
--

[1] https://www.nxp.com/docs/en/errata/IMX8_1N94W.pdf
[2] 307fd14d4b14 ("arm64: dts: imx: add imx8qm mek support")
[3] https://github.com/nxp-imx/linux-imx/blob/lf-6.1.y/arch/arm64/include/asm/tlbflush.h#L19

Signed-off-by: Ivan T. Ivanov <iivanov@suse.de>
---
 Documentation/arm64/silicon-errata.rst |  2 ++
 arch/arm64/Kconfig                     | 10 ++++++++++
 arch/arm64/include/asm/cpufeature.h    |  3 ++-
 arch/arm64/include/asm/tlbflush.h      |  6 +++++-
 arch/arm64/kernel/cpu_errata.c         | 18 ++++++++++++++++++
 arch/arm64/kernel/traps.c              | 22 +++++++++++++++++++++-
 arch/arm64/tools/cpucaps               |  1 +
 7 files changed, 59 insertions(+), 3 deletions(-)

diff --git a/Documentation/arm64/silicon-errata.rst b/Documentation/arm64/silicon-errata.rst
index ec5f889d7681..fce231797184 100644
--- a/Documentation/arm64/silicon-errata.rst
+++ b/Documentation/arm64/silicon-errata.rst
@@ -175,6 +175,8 @@ stable kernels.
 +----------------+-----------------+-----------------+-----------------------------+
 | Freescale/NXP  | LS2080A/LS1043A | A-008585        | FSL_ERRATUM_A008585         |
 +----------------+-----------------+-----------------+-----------------------------+
+| Freescale/NXP  | i.MX 8QuadMax   | ERR050104       | NXP_IMX8QM_ERRATUM_ERR050104|
++----------------+-----------------+-----------------+-----------------------------+
 +----------------+-----------------+-----------------+-----------------------------+
 | Hisilicon      | Hip0{5,6,7}     | #161010101      | HISILICON_ERRATUM_161010101 |
 +----------------+-----------------+-----------------+-----------------------------+
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 1023e896d46b..437cb53f8753 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -1159,6 +1159,16 @@ config SOCIONEXT_SYNQUACER_PREITS
 
 	  If unsure, say Y.
 
+config NXP_IMX8QM_ERRATUM_ERR050104
+	bool "NXP iMX8QM: Workaround for Arm/A53 Cache coherency issue"
+	default n
+	help
+	  Some maintenance operations exchanged between the A53 and A72 core
+	  clusters, involving some Translation Look-aside Buffer Invalidate
+	  (TLBI) and Instruction Cache (IC) instructions can be corrupted.
+
+	  If unsure, say N.
+
 endmenu # "ARM errata workarounds via the alternatives framework"
 
 choice
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 6bf013fb110d..1ed648f7f29a 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -835,7 +835,8 @@ static inline bool system_supports_bti(void)
 static inline bool system_supports_tlb_range(void)
 {
 	return IS_ENABLED(CONFIG_ARM64_TLB_RANGE) &&
-		cpus_have_const_cap(ARM64_HAS_TLB_RANGE);
+		cpus_have_const_cap(ARM64_HAS_TLB_RANGE) &&
+		!cpus_have_const_cap(ARM64_WORKAROUND_NXP_ERR050104);
 }
 
 int do_emulate_mrs(struct pt_regs *regs, u32 sys_reg, u32 rt);
diff --git a/arch/arm64/include/asm/tlbflush.h b/arch/arm64/include/asm/tlbflush.h
index 412a3b9a3c25..12055b859ce3 100644
--- a/arch/arm64/include/asm/tlbflush.h
+++ b/arch/arm64/include/asm/tlbflush.h
@@ -37,7 +37,11 @@
 			    : : )
 
 #define __TLBI_1(op, arg) asm (ARM64_ASM_PREAMBLE			       \
-			       "tlbi " #op ", %0\n"			       \
+		   ALTERNATIVE("nop\n	nop\n  tlbi " #op ", %0",              \
+			       "tlbi vmalle1is\n  dsb ish\n  isb",	       \
+			       ARM64_WORKAROUND_NXP_ERR050104)		       \
+			    : : "r" (arg));				       \
+			  asm (ARM64_ASM_PREAMBLE			       \
 		   ALTERNATIVE("nop\n			nop",		       \
 			       "dsb ish\n		tlbi " #op ", %0",     \
 			       ARM64_WORKAROUND_REPEAT_TLBI,		       \
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 307faa2b4395..7b702a79bf60 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -8,6 +8,7 @@
 #include <linux/arm-smccc.h>
 #include <linux/types.h>
 #include <linux/cpu.h>
+#include <linux/of.h>
 #include <asm/cpu.h>
 #include <asm/cputype.h>
 #include <asm/cpufeature.h>
@@ -55,6 +56,14 @@ is_kryo_midr(const struct arm64_cpu_capabilities *entry, int scope)
 	return model == entry->midr_range.model;
 }
 
+static bool __maybe_unused
+is_imx8qm_soc(const struct arm64_cpu_capabilities *entry, int scope)
+{
+	WARN_ON(preemptible());
+
+	return of_machine_is_compatible("fsl,imx8qm");
+}
+
 static bool
 has_mismatched_cache_type(const struct arm64_cpu_capabilities *entry,
 			  int scope)
@@ -729,6 +738,15 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
 		MIDR_FIXED(MIDR_CPU_VAR_REV(1,1), BIT(25)),
 		.cpu_enable = cpu_clear_bf16_from_user_emulation,
 	},
+#endif
+#ifdef CONFIG_NXP_IMX8QM_ERRATUM_ERR050104
+	{
+		.desc = "NXP A53 cache coherency issue",
+		.capability = ARM64_WORKAROUND_NXP_ERR050104,
+		.type = ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE,
+		.matches = is_imx8qm_soc,
+		.cpu_enable = cpu_enable_cache_maint_trap,
+	},
 #endif
 	{
 	}
diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
index 4a79ba100799..4858f8c86fd5 100644
--- a/arch/arm64/kernel/traps.c
+++ b/arch/arm64/kernel/traps.c
@@ -529,6 +529,26 @@ void do_el1_fpac(struct pt_regs *regs, unsigned long esr)
 		uaccess_ttbr0_disable();			\
 	}
 
+#define __user_instruction_cache_maint(address, res)		\
+do {								\
+	if (address >= TASK_SIZE_MAX) {				\
+		res = -EFAULT;					\
+	} else {						\
+		uaccess_ttbr0_enable();				\
+		asm volatile (					\
+			"1:\n"					\
+	    ALTERNATIVE("	ic	ivau, %1\n",		\
+			"	ic	ialluis\n",		\
+			ARM64_WORKAROUND_NXP_ERR050104)		\
+			"	mov	%w0, #0\n"		\
+			"2:\n"					\
+			_ASM_EXTABLE_UACCESS_ERR(1b, 2b, %w0)	\
+			: "=r" (res)				\
+			: "r" (address));			\
+		uaccess_ttbr0_disable();			\
+	}							\
+} while (0)
+
 static void user_cache_maint_handler(unsigned long esr, struct pt_regs *regs)
 {
 	unsigned long tagged_address, address;
@@ -556,7 +576,7 @@ static void user_cache_maint_handler(unsigned long esr, struct pt_regs *regs)
 		__user_cache_maint("dc civac", address, ret);
 		break;
 	case ESR_ELx_SYS64_ISS_CRM_IC_IVAU:	/* IC IVAU */
-		__user_cache_maint("ic ivau", address, ret);
+		__user_instruction_cache_maint(address, ret);
 		break;
 	default:
 		force_signal_inject(SIGILL, ILL_ILLOPC, regs->pc, 0);
diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps
index 37b1340e9646..e225f1cd1005 100644
--- a/arch/arm64/tools/cpucaps
+++ b/arch/arm64/tools/cpucaps
@@ -90,3 +90,4 @@ WORKAROUND_NVIDIA_CARMEL_CNP
 WORKAROUND_QCOM_FALKOR_E1003
 WORKAROUND_REPEAT_TLBI
 WORKAROUND_SPECULATIVE_AT
+WORKAROUND_NXP_ERR050104
-- 
2.35.3


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH] arm64: errata: Add NXP iMX8QM workaround for A53 Cache coherency issue

[Mark Rutland]

Hi,

On Wed, Apr 12, 2023 at 03:55:06PM +0300, Ivan T. Ivanov wrote:
> According to NXP errata document[1] i.MX8QuadMax SoC suffers from
> serious cache coherence issue. It was also mentioned in initial
> support[2] for imx8qm mek machine.

That's a fairly horrid bug.

> I chose to use an ALTERNATIVE() framework, instead downstream solution[3],
> for this issue with the hope to reduce effect of this fix on unaffected
> platforms.
> 
> Unfortunately I was unable to find a way to identify SoC ID using
> registers. Boot CPU MIDR_EL1 is equal to 0x410fd034. So I fallback to
> using devicetree compatible strings for this.

How does this work with KVM?

VMs have no idea that the host platform is, and so will have no idea that this
erratum applies, so they're going to blow up spectacularly.

So we should probably be disabling KVM (or at the very least, printing a
gigantic warning).

> I know this fix is a suboptimal solution for affected machines, but I
> haven't been able to come up with a less intrusive fix.  And I hope once
> TLB caches are invalidated any immediate attempt to invalidate them again
> will be close to NOP operation (flush_tlb_kernel_range())
> 
> I have run few simple benchmarks and perf tests on affected and unaffected
> machines and I was not able see any obvious issues. iMX8QM "performance"
> was nearly doubled with 2 A72 bringed online.
> 
> Following is excerpt from NXP IMX8_1N94W "Mask Set Errata" document
> Rev. 5, 3/2023. Just in case it gets lost somehow.
> 
> ---
> "ERR050104: Arm/A53: Cache coherency issue"
> 
> Description
> 
> Some maintenance operations exchanged between the A53 and A72
> core clusters, involving some Translation Look-aside Buffer
> Invalidate (TLBI) and Instruction Cache (IC) instructions can
> be corrupted. The upper bits, above bit-35, of ARADDR and ACADDR
> buses within in Arm A53 sub-system have been incorrectly connected.
> Therefore ARADDR and ACADDR address bits above bit-35 should not
> be used.
> 
> Workaround
> 
> The following software instructions are required to be downgraded
> to TLBI VMALLE1IS:  TLBI ASIDE1, TLBI ASIDE1IS, TLBI VAAE1,
> TLBI VAAE1IS, TLBI VAALE1, TLBI VAALE1IS, TLBI VAE1, TLBI VAE1IS,
> TLBI VALE1, TLBI VALE1IS
> 
> The following software instructions are required to be downgraded
> to TLBI VMALLS12E1IS: TLBI IPAS2E1IS, TLBI IPAS2LE1IS
> 
> The following software instructions are required to be downgraded
> to TLBI ALLE2IS: TLBI VAE2IS, TLBI VALE2IS.
> 
> The following software instructions are required to be downgraded
> to TLBI ALLE3IS: TLBI VAE3IS, TLBI VALE3IS.
> 
> The following software instructions are required to be downgraded
> to TLBI VMALLE1IS when the Force Broadcast (FB) bit [9] of the
> Hypervisor Configuration Register (HCR_EL2) is set:
> TLBI ASIDE1, TLBI VAAE1, TLBI VAALE1, TLBI VAE1, TLBI VALE1
> 
> The following software instruction is required to be downgraded
> to IC IALLUIS: IC IVAU, Xt
> 
> Specifically for the IC IVAU, Xt downgrade, setting SCTLR_EL1.UCI
> to 0 will disable EL0 access to this instruction. Any attempt to
> execute from EL0 will generate an EL1 trap, where the downgrade to
> IC ALLUIS can be implemented.
> --
> 
> [1] https://www.nxp.com/docs/en/errata/IMX8_1N94W.pdf
> [2] 307fd14d4b14 ("arm64: dts: imx: add imx8qm mek support")
> [3] https://github.com/nxp-imx/linux-imx/blob/lf-6.1.y/arch/arm64/include/asm/tlbflush.h#L19
> 
> Signed-off-by: Ivan T. Ivanov <iivanov@suse.de>
> ---
>  Documentation/arm64/silicon-errata.rst |  2 ++
>  arch/arm64/Kconfig                     | 10 ++++++++++
>  arch/arm64/include/asm/cpufeature.h    |  3 ++-
>  arch/arm64/include/asm/tlbflush.h      |  6 +++++-
>  arch/arm64/kernel/cpu_errata.c         | 18 ++++++++++++++++++
>  arch/arm64/kernel/traps.c              | 22 +++++++++++++++++++++-
>  arch/arm64/tools/cpucaps               |  1 +
>  7 files changed, 59 insertions(+), 3 deletions(-)
> 
> diff --git a/Documentation/arm64/silicon-errata.rst b/Documentation/arm64/silicon-errata.rst
> index ec5f889d7681..fce231797184 100644
> --- a/Documentation/arm64/silicon-errata.rst
> +++ b/Documentation/arm64/silicon-errata.rst
> @@ -175,6 +175,8 @@ stable kernels.
>  +----------------+-----------------+-----------------+-----------------------------+
>  | Freescale/NXP  | LS2080A/LS1043A | A-008585        | FSL_ERRATUM_A008585         |
>  +----------------+-----------------+-----------------+-----------------------------+
> +| Freescale/NXP  | i.MX 8QuadMax   | ERR050104       | NXP_IMX8QM_ERRATUM_ERR050104|
> ++----------------+-----------------+-----------------+-----------------------------+
>  +----------------+-----------------+-----------------+-----------------------------+
>  | Hisilicon      | Hip0{5,6,7}     | #161010101      | HISILICON_ERRATUM_161010101 |
>  +----------------+-----------------+-----------------+-----------------------------+
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index 1023e896d46b..437cb53f8753 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -1159,6 +1159,16 @@ config SOCIONEXT_SYNQUACER_PREITS
>  
>  	  If unsure, say Y.
>  
> +config NXP_IMX8QM_ERRATUM_ERR050104
> +	bool "NXP iMX8QM: Workaround for Arm/A53 Cache coherency issue"

Please use the erratum number, e.g.

	bool "NXP iMX8QM ERR050104: broken cache/tlb invalidation"

> +	default n
> +	help
> +	  Some maintenance operations exchanged between the A53 and A72 core
> +	  clusters, involving some Translation Look-aside Buffer Invalidate
> +	  (TLBI) and Instruction Cache (IC) instructions can be corrupted.

Likewise, please add a more compelte description here, e.g.

	help
	  On iMX8QM, addresses above bit 35 are not broadcast correctly for
	  TLBI or IC operations, making TLBI and IC unreliable.

	  Work around this erratum by using TLBI *ALL*IS and IC IALLUIS
	  operations. EL0 use of IC IVAU is trapped and upgraded to IC IALLUIS.

> +
> +	  If unsure, say N.
> +
>  endmenu # "ARM errata workarounds via the alternatives framework"
>  
>  choice
> diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
> index 6bf013fb110d..1ed648f7f29a 100644
> --- a/arch/arm64/include/asm/cpufeature.h
> +++ b/arch/arm64/include/asm/cpufeature.h
> @@ -835,7 +835,8 @@ static inline bool system_supports_bti(void)
>  static inline bool system_supports_tlb_range(void)
>  {
>  	return IS_ENABLED(CONFIG_ARM64_TLB_RANGE) &&
> -		cpus_have_const_cap(ARM64_HAS_TLB_RANGE);
> +		cpus_have_const_cap(ARM64_HAS_TLB_RANGE) &&
> +		!cpus_have_const_cap(ARM64_WORKAROUND_NXP_ERR050104);
>  }

It'd be better to handle this in the detection of ARM64_HAS_TLB_RANGE, as we
have for CNP where has_useable_cnp() checks for ARM64_WORKAROUND_NVIDIA_CARMEL_CNP.

>  
>  int do_emulate_mrs(struct pt_regs *regs, u32 sys_reg, u32 rt);
> diff --git a/arch/arm64/include/asm/tlbflush.h b/arch/arm64/include/asm/tlbflush.h
> index 412a3b9a3c25..12055b859ce3 100644
> --- a/arch/arm64/include/asm/tlbflush.h
> +++ b/arch/arm64/include/asm/tlbflush.h
> @@ -37,7 +37,11 @@
>  			    : : )
>  
>  #define __TLBI_1(op, arg) asm (ARM64_ASM_PREAMBLE			       \
> -			       "tlbi " #op ", %0\n"			       \
> +		   ALTERNATIVE("nop\n	nop\n  tlbi " #op ", %0",              \
> +			       "tlbi vmalle1is\n  dsb ish\n  isb",	       \
> +			       ARM64_WORKAROUND_NXP_ERR050104)		       \
> +			    : : "r" (arg));				       \

Why do you need the DSB ISH + ISB here? It's up to the caller to issue those,
and the ARM64_WORKAROUND_REPEAT_TLBI workaround only has DSB ISH to ensure that
the first op completes before the second is issued.

> +			  asm (ARM64_ASM_PREAMBLE			       \
>  		   ALTERNATIVE("nop\n			nop",		       \
>  			       "dsb ish\n		tlbi " #op ", %0",     \
>  			       ARM64_WORKAROUND_REPEAT_TLBI,		       \
> diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
> index 307faa2b4395..7b702a79bf60 100644
> --- a/arch/arm64/kernel/cpu_errata.c
> +++ b/arch/arm64/kernel/cpu_errata.c
> @@ -8,6 +8,7 @@
>  #include <linux/arm-smccc.h>
>  #include <linux/types.h>
>  #include <linux/cpu.h>
> +#include <linux/of.h>
>  #include <asm/cpu.h>
>  #include <asm/cputype.h>
>  #include <asm/cpufeature.h>
> @@ -55,6 +56,14 @@ is_kryo_midr(const struct arm64_cpu_capabilities *entry, int scope)
>  	return model == entry->midr_range.model;
>  }
>  
> +static bool __maybe_unused
> +is_imx8qm_soc(const struct arm64_cpu_capabilities *entry, int scope)
> +{
> +	WARN_ON(preemptible());
> +
> +	return of_machine_is_compatible("fsl,imx8qm");
> +}

As above, what is going to be done for VMs, where this won't be present?

> +
>  static bool
>  has_mismatched_cache_type(const struct arm64_cpu_capabilities *entry,
>  			  int scope)
> @@ -729,6 +738,15 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
>  		MIDR_FIXED(MIDR_CPU_VAR_REV(1,1), BIT(25)),
>  		.cpu_enable = cpu_clear_bf16_from_user_emulation,
>  	},
> +#endif
> +#ifdef CONFIG_NXP_IMX8QM_ERRATUM_ERR050104
> +	{
> +		.desc = "NXP A53 cache coherency issue",

Please use the erratum number, i.e. 

		.desc = "NXP erratum ERR050104",

> +		.capability = ARM64_WORKAROUND_NXP_ERR050104,
> +		.type = ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE,
> +		.matches = is_imx8qm_soc,
> +		.cpu_enable = cpu_enable_cache_maint_trap,
> +	},
>  #endif
>  	{
>  	}
> diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
> index 4a79ba100799..4858f8c86fd5 100644
> --- a/arch/arm64/kernel/traps.c
> +++ b/arch/arm64/kernel/traps.c
> @@ -529,6 +529,26 @@ void do_el1_fpac(struct pt_regs *regs, unsigned long esr)
>  		uaccess_ttbr0_disable();			\
>  	}
>  
> +#define __user_instruction_cache_maint(address, res)		\
> +do {								\
> +	if (address >= TASK_SIZE_MAX) {				\
> +		res = -EFAULT;					\
> +	} else {						\
> +		uaccess_ttbr0_enable();				\
> +		asm volatile (					\
> +			"1:\n"					\
> +	    ALTERNATIVE("	ic	ivau, %1\n",		\
> +			"	ic	ialluis\n",		\
> +			ARM64_WORKAROUND_NXP_ERR050104)		\
> +			"	mov	%w0, #0\n"		\
> +			"2:\n"					\
> +			_ASM_EXTABLE_UACCESS_ERR(1b, 2b, %w0)	\
> +			: "=r" (res)				\
> +			: "r" (address));			\
> +		uaccess_ttbr0_disable();			\
> +	}							\
> +} while (0)
> +
>  static void user_cache_maint_handler(unsigned long esr, struct pt_regs *regs)
>  {
>  	unsigned long tagged_address, address;
> @@ -556,7 +576,7 @@ static void user_cache_maint_handler(unsigned long esr, struct pt_regs *regs)
>  		__user_cache_maint("dc civac", address, ret);
>  		break;
>  	case ESR_ELx_SYS64_ISS_CRM_IC_IVAU:	/* IC IVAU */
> -		__user_cache_maint("ic ivau", address, ret);
> +		__user_instruction_cache_maint(address, ret);
>  		break;

Hmm... this will silently change any 'IC IVAU' to never fault. That's probably
not the end of the world, since it's IMP-DEF whether IC would raise a
permission fault, but it is a change of behaviour.

It would be a bit simpler to handle this inline within the switch, e.g.

	case ESR_ELx_SYS64_ISS_CRM_IC_IVAU:     /* IC IVAU */
		if (cpus_have_final_cap(ARM64_WORKAROUND_NXP_ERR050104)) {
			asm volatile("ic ialluis");
			ret = 0;
			break;
		}
		__user_instruction_cache_maint(address, ret);
		break;

... as that would avoid duplicating the bulk of the __user_cache_maint() macro.


>  	default:
>  		force_signal_inject(SIGILL, ILL_ILLOPC, regs->pc, 0);
> diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps
> index 37b1340e9646..e225f1cd1005 100644
> --- a/arch/arm64/tools/cpucaps
> +++ b/arch/arm64/tools/cpucaps
> @@ -90,3 +90,4 @@ WORKAROUND_NVIDIA_CARMEL_CNP
>  WORKAROUND_QCOM_FALKOR_E1003
>  WORKAROUND_REPEAT_TLBI
>  WORKAROUND_SPECULATIVE_AT
> +WORKAROUND_NXP_ERR050104

These definitions are expected to be ordered alphabetically, so this should be
earlier in the list.

Thanks,
Mark.

> -- 
> 2.35.3
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH] arm64: errata: Add NXP iMX8QM workaround for A53 Cache coherency issue

[Robin Murphy]

On 2023-04-13 12:19, Mark Rutland wrote:
[...]
>> diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
>> index 6bf013fb110d..1ed648f7f29a 100644
>> --- a/arch/arm64/include/asm/cpufeature.h
>> +++ b/arch/arm64/include/asm/cpufeature.h
>> @@ -835,7 +835,8 @@ static inline bool system_supports_bti(void)
>>   static inline bool system_supports_tlb_range(void)
>>   {
>>   	return IS_ENABLED(CONFIG_ARM64_TLB_RANGE) &&
>> -		cpus_have_const_cap(ARM64_HAS_TLB_RANGE);
>> +		cpus_have_const_cap(ARM64_HAS_TLB_RANGE) &&
>> +		!cpus_have_const_cap(ARM64_WORKAROUND_NXP_ERR050104);
>>   }
> 
> It'd be better to handle this in the detection of ARM64_HAS_TLB_RANGE, as we
> have for CNP where has_useable_cnp() checks for ARM64_WORKAROUND_NVIDIA_CARMEL_CNP.

It's not needed in either place, since neither Cortex-A53 or Cortex-A72 
support FEAT_TLBIRANGE, so this could never be true on affected 
platforms anyway.

Tangentially, I understand this platform has an SMMU[1], so I'd say it 
would also be worth checking what SMMU_IDR0.BTM reports. With any luck 
it might be 0, but if it's 1 then strictly it would want to be 
overridden as part of a complete workaround as well. That wouldn't be a 
practical issue right now, not least since the current Linux driver 
doesn't even use BTM, but it's something which could need to be borne in 
mind in future.

Robin.

[1] 
https://lore.kernel.org/linux-arm-kernel/20210807104517.24066-1-peng.fan@oss.nxp.com/

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH] arm64: errata: Add NXP iMX8QM workaround for A53 Cache coherency issue

[Peng Fan]

+Frank, who had worked on downstream solution.

On 4/12/2023 8:55 PM, Ivan T. Ivanov wrote:
> According to NXP errata document[1] i.MX8QuadMax SoC suffers from
> serious cache coherence issue. It was also mentioned in initial
> support[2] for imx8qm mek machine.
> 
> I chose to use an ALTERNATIVE() framework, instead downstream solution[3],
> for this issue with the hope to reduce effect of this fix on unaffected
> platforms.
> 
> Unfortunately I was unable to find a way to identify SoC ID using
> registers. Boot CPU MIDR_EL1 is equal to 0x410fd034. So I fallback to
> using devicetree compatible strings for this.
> 
> I know this fix is a suboptimal solution for affected machines, but I
> haven't been able to come up with a less intrusive fix.  And I hope once
> TLB caches are invalidated any immediate attempt to invalidate them again
> will be close to NOP operation (flush_tlb_kernel_range())
> 
> I have run few simple benchmarks and perf tests on affected and unaffected
> machines and I was not able see any obvious issues. iMX8QM "performance"
> was nearly doubled with 2 A72 bringed online.
> 
> Following is excerpt from NXP IMX8_1N94W "Mask Set Errata" document
> Rev. 5, 3/2023. Just in case it gets lost somehow.
> 
> ---
> "ERR050104: Arm/A53: Cache coherency issue"
> 
> Description
> 
> Some maintenance operations exchanged between the A53 and A72
> core clusters, involving some Translation Look-aside Buffer
> Invalidate (TLBI) and Instruction Cache (IC) instructions can
> be corrupted. The upper bits, above bit-35, of ARADDR and ACADDR
> buses within in Arm A53 sub-system have been incorrectly connected.
> Therefore ARADDR and ACADDR address bits above bit-35 should not
> be used.
> 
> Workaround
> 
> The following software instructions are required to be downgraded
> to TLBI VMALLE1IS:  TLBI ASIDE1, TLBI ASIDE1IS, TLBI VAAE1,
> TLBI VAAE1IS, TLBI VAALE1, TLBI VAALE1IS, TLBI VAE1, TLBI VAE1IS,
> TLBI VALE1, TLBI VALE1IS
> 
> The following software instructions are required to be downgraded
> to TLBI VMALLS12E1IS: TLBI IPAS2E1IS, TLBI IPAS2LE1IS
> 
> The following software instructions are required to be downgraded
> to TLBI ALLE2IS: TLBI VAE2IS, TLBI VALE2IS.
> 
> The following software instructions are required to be downgraded
> to TLBI ALLE3IS: TLBI VAE3IS, TLBI VALE3IS.
> 
> The following software instructions are required to be downgraded
> to TLBI VMALLE1IS when the Force Broadcast (FB) bit [9] of the
> Hypervisor Configuration Register (HCR_EL2) is set:
> TLBI ASIDE1, TLBI VAAE1, TLBI VAALE1, TLBI VAE1, TLBI VALE1
> 
> The following software instruction is required to be downgraded
> to IC IALLUIS: IC IVAU, Xt
> 
> Specifically for the IC IVAU, Xt downgrade, setting SCTLR_EL1.UCI
> to 0 will disable EL0 access to this instruction. Any attempt to
> execute from EL0 will generate an EL1 trap, where the downgrade to
> IC ALLUIS can be implemented.
> --
> 
> [1] https://www.nxp.com/docs/en/errata/IMX8_1N94W.pdf
> [2] 307fd14d4b14 ("arm64: dts: imx: add imx8qm mek support")
> [3] https://github.com/nxp-imx/linux-imx/blob/lf-6.1.y/arch/arm64/include/asm/tlbflush.h#L19
> 
> Signed-off-by: Ivan T. Ivanov <iivanov@suse.de>
> ---
>   Documentation/arm64/silicon-errata.rst |  2 ++
>   arch/arm64/Kconfig                     | 10 ++++++++++
>   arch/arm64/include/asm/cpufeature.h    |  3 ++-
>   arch/arm64/include/asm/tlbflush.h      |  6 +++++-
>   arch/arm64/kernel/cpu_errata.c         | 18 ++++++++++++++++++
>   arch/arm64/kernel/traps.c              | 22 +++++++++++++++++++++-
>   arch/arm64/tools/cpucaps               |  1 +
>   7 files changed, 59 insertions(+), 3 deletions(-)
> 
> diff --git a/Documentation/arm64/silicon-errata.rst b/Documentation/arm64/silicon-errata.rst
> index ec5f889d7681..fce231797184 100644
> --- a/Documentation/arm64/silicon-errata.rst
> +++ b/Documentation/arm64/silicon-errata.rst
> @@ -175,6 +175,8 @@ stable kernels.
>   +----------------+-----------------+-----------------+-----------------------------+
>   | Freescale/NXP  | LS2080A/LS1043A | A-008585        | FSL_ERRATUM_A008585         |
>   +----------------+-----------------+-----------------+-----------------------------+
> +| Freescale/NXP  | i.MX 8QuadMax   | ERR050104       | NXP_IMX8QM_ERRATUM_ERR050104|
> ++----------------+-----------------+-----------------+-----------------------------+
>   +----------------+-----------------+-----------------+-----------------------------+
>   | Hisilicon      | Hip0{5,6,7}     | #161010101      | HISILICON_ERRATUM_161010101 |
>   +----------------+-----------------+-----------------+-----------------------------+
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index 1023e896d46b..437cb53f8753 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -1159,6 +1159,16 @@ config SOCIONEXT_SYNQUACER_PREITS
>   
>   	  If unsure, say Y.
>   
> +config NXP_IMX8QM_ERRATUM_ERR050104
> +	bool "NXP iMX8QM: Workaround for Arm/A53 Cache coherency issue"
> +	default n
> +	help
> +	  Some maintenance operations exchanged between the A53 and A72 core
> +	  clusters, involving some Translation Look-aside Buffer Invalidate
> +	  (TLBI) and Instruction Cache (IC) instructions can be corrupted.
> +
> +	  If unsure, say N.
> +
>   endmenu # "ARM errata workarounds via the alternatives framework"
>   
>   choice
> diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
> index 6bf013fb110d..1ed648f7f29a 100644
> --- a/arch/arm64/include/asm/cpufeature.h
> +++ b/arch/arm64/include/asm/cpufeature.h
> @@ -835,7 +835,8 @@ static inline bool system_supports_bti(void)
>   static inline bool system_supports_tlb_range(void)
>   {
>   	return IS_ENABLED(CONFIG_ARM64_TLB_RANGE) &&
> -		cpus_have_const_cap(ARM64_HAS_TLB_RANGE);
> +		cpus_have_const_cap(ARM64_HAS_TLB_RANGE) &&
> +		!cpus_have_const_cap(ARM64_WORKAROUND_NXP_ERR050104);
>   }
>   
>   int do_emulate_mrs(struct pt_regs *regs, u32 sys_reg, u32 rt);
> diff --git a/arch/arm64/include/asm/tlbflush.h b/arch/arm64/include/asm/tlbflush.h
> index 412a3b9a3c25..12055b859ce3 100644
> --- a/arch/arm64/include/asm/tlbflush.h
> +++ b/arch/arm64/include/asm/tlbflush.h
> @@ -37,7 +37,11 @@
>   			    : : )
>   
>   #define __TLBI_1(op, arg) asm (ARM64_ASM_PREAMBLE			       \
> -			       "tlbi " #op ", %0\n"			       \
> +		   ALTERNATIVE("nop\n	nop\n  tlbi " #op ", %0",              \
> +			       "tlbi vmalle1is\n  dsb ish\n  isb",	       \
> +			       ARM64_WORKAROUND_NXP_ERR050104)		       \
> +			    : : "r" (arg));				       \
> +			  asm (ARM64_ASM_PREAMBLE			       \
>   		   ALTERNATIVE("nop\n			nop",		       \
>   			       "dsb ish\n		tlbi " #op ", %0",     \
>   			       ARM64_WORKAROUND_REPEAT_TLBI,		       \
> diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
> index 307faa2b4395..7b702a79bf60 100644
> --- a/arch/arm64/kernel/cpu_errata.c
> +++ b/arch/arm64/kernel/cpu_errata.c
> @@ -8,6 +8,7 @@
>   #include <linux/arm-smccc.h>
>   #include <linux/types.h>
>   #include <linux/cpu.h>
> +#include <linux/of.h>
>   #include <asm/cpu.h>
>   #include <asm/cputype.h>
>   #include <asm/cpufeature.h>
> @@ -55,6 +56,14 @@ is_kryo_midr(const struct arm64_cpu_capabilities *entry, int scope)
>   	return model == entry->midr_range.model;
>   }
>   
> +static bool __maybe_unused
> +is_imx8qm_soc(const struct arm64_cpu_capabilities *entry, int scope)
> +{
> +	WARN_ON(preemptible());
> +
> +	return of_machine_is_compatible("fsl,imx8qm");
> +}
> +
>   static bool
>   has_mismatched_cache_type(const struct arm64_cpu_capabilities *entry,
>   			  int scope)
> @@ -729,6 +738,15 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
>   		MIDR_FIXED(MIDR_CPU_VAR_REV(1,1), BIT(25)),
>   		.cpu_enable = cpu_clear_bf16_from_user_emulation,
>   	},
> +#endif
> +#ifdef CONFIG_NXP_IMX8QM_ERRATUM_ERR050104
> +	{
> +		.desc = "NXP A53 cache coherency issue",
> +		.capability = ARM64_WORKAROUND_NXP_ERR050104,
> +		.type = ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE,
> +		.matches = is_imx8qm_soc,
> +		.cpu_enable = cpu_enable_cache_maint_trap,
> +	},
>   #endif
>   	{
>   	}
> diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
> index 4a79ba100799..4858f8c86fd5 100644
> --- a/arch/arm64/kernel/traps.c
> +++ b/arch/arm64/kernel/traps.c
> @@ -529,6 +529,26 @@ void do_el1_fpac(struct pt_regs *regs, unsigned long esr)
>   		uaccess_ttbr0_disable();			\
>   	}
>   
> +#define __user_instruction_cache_maint(address, res)		\
> +do {								\
> +	if (address >= TASK_SIZE_MAX) {				\
> +		res = -EFAULT;					\
> +	} else {						\
> +		uaccess_ttbr0_enable();				\
> +		asm volatile (					\
> +			"1:\n"					\
> +	    ALTERNATIVE("	ic	ivau, %1\n",		\
> +			"	ic	ialluis\n",		\
> +			ARM64_WORKAROUND_NXP_ERR050104)		\
> +			"	mov	%w0, #0\n"		\
> +			"2:\n"					\
> +			_ASM_EXTABLE_UACCESS_ERR(1b, 2b, %w0)	\
> +			: "=r" (res)				\
> +			: "r" (address));			\
> +		uaccess_ttbr0_disable();			\
> +	}							\
> +} while (0)
> +
>   static void user_cache_maint_handler(unsigned long esr, struct pt_regs *regs)
>   {
>   	unsigned long tagged_address, address;
> @@ -556,7 +576,7 @@ static void user_cache_maint_handler(unsigned long esr, struct pt_regs *regs)
>   		__user_cache_maint("dc civac", address, ret);
>   		break;
>   	case ESR_ELx_SYS64_ISS_CRM_IC_IVAU:	/* IC IVAU */
> -		__user_cache_maint("ic ivau", address, ret);
> +		__user_instruction_cache_maint(address, ret);
>   		break;
>   	default:
>   		force_signal_inject(SIGILL, ILL_ILLOPC, regs->pc, 0);
> diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps
> index 37b1340e9646..e225f1cd1005 100644
> --- a/arch/arm64/tools/cpucaps
> +++ b/arch/arm64/tools/cpucaps
> @@ -90,3 +90,4 @@ WORKAROUND_NVIDIA_CARMEL_CNP
>   WORKAROUND_QCOM_FALKOR_E1003
>   WORKAROUND_REPEAT_TLBI
>   WORKAROUND_SPECULATIVE_AT
> +WORKAROUND_NXP_ERR050104

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

RE: [PATCH] arm64: errata: Add NXP iMX8QM workaround for A53 Cache coherency issue

[Frank Li]

> -----Original Message-----
> From: Peng Fan (OSS) <peng.fan@oss.nxp.com>
> Sent: Sunday, April 16, 2023 10:08 PM
> To: Ivan T. Ivanov <iivanov@suse.de>; Catalin Marinas
> <catalin.marinas@arm.com>; Will Deacon <will@kernel.org>; Frank Li
> <frank.li@nxp.com>
> Cc: Mark Brown <broonie@kernel.org>; Mark Rutland
> <mark.rutland@arm.com>; Shawn Guo <shawnguo@kernel.org>; Aisheng
> Dong <aisheng.dong@nxp.com>; linux-arm-kernel@lists.infradead.org; dl-
> linux-imx <linux-imx@nxp.com>
> Subject: Re: [PATCH] arm64: errata: Add NXP iMX8QM workaround for A53
> Cache coherency issue
> 
> +Frank, who had worked on downstream solution.

+ Jason Liu, who actually work on this. 

Best regards
Frank Li

> 
> On 4/12/2023 8:55 PM, Ivan T. Ivanov wrote:
> > According to NXP errata document[1] i.MX8QuadMax SoC suffers from
> > serious cache coherence issue. It was also mentioned in initial
> > support[2] for imx8qm mek machine.
> >
> > I chose to use an ALTERNATIVE() framework, instead downstream
> solution[3],
> > for this issue with the hope to reduce effect of this fix on unaffected
> > platforms.
> >
> > Unfortunately I was unable to find a way to identify SoC ID using
> > registers. Boot CPU MIDR_EL1 is equal to 0x410fd034. So I fallback to
> > using devicetree compatible strings for this.
> >
> > I know this fix is a suboptimal solution for affected machines, but I
> > haven't been able to come up with a less intrusive fix.  And I hope once
> > TLB caches are invalidated any immediate attempt to invalidate them again
> > will be close to NOP operation (flush_tlb_kernel_range())
> >
> > I have run few simple benchmarks and perf tests on affected and
> unaffected
> > machines and I was not able see any obvious issues. iMX8QM
> "performance"
> > was nearly doubled with 2 A72 bringed online.
> >
> > Following is excerpt from NXP IMX8_1N94W "Mask Set Errata" document
> > Rev. 5, 3/2023. Just in case it gets lost somehow.
> >
> > ---
> > "ERR050104: Arm/A53: Cache coherency issue"
> >
> > Description
> >
> > Some maintenance operations exchanged between the A53 and A72
> > core clusters, involving some Translation Look-aside Buffer
> > Invalidate (TLBI) and Instruction Cache (IC) instructions can
> > be corrupted. The upper bits, above bit-35, of ARADDR and ACADDR
> > buses within in Arm A53 sub-system have been incorrectly connected.
> > Therefore ARADDR and ACADDR address bits above bit-35 should not
> > be used.
> >
> > Workaround
> >
> > The following software instructions are required to be downgraded
> > to TLBI VMALLE1IS:  TLBI ASIDE1, TLBI ASIDE1IS, TLBI VAAE1,
> > TLBI VAAE1IS, TLBI VAALE1, TLBI VAALE1IS, TLBI VAE1, TLBI VAE1IS,
> > TLBI VALE1, TLBI VALE1IS
> >
> > The following software instructions are required to be downgraded
> > to TLBI VMALLS12E1IS: TLBI IPAS2E1IS, TLBI IPAS2LE1IS
> >
> > The following software instructions are required to be downgraded
> > to TLBI ALLE2IS: TLBI VAE2IS, TLBI VALE2IS.
> >
> > The following software instructions are required to be downgraded
> > to TLBI ALLE3IS: TLBI VAE3IS, TLBI VALE3IS.
> >
> > The following software instructions are required to be downgraded
> > to TLBI VMALLE1IS when the Force Broadcast (FB) bit [9] of the
> > Hypervisor Configuration Register (HCR_EL2) is set:
> > TLBI ASIDE1, TLBI VAAE1, TLBI VAALE1, TLBI VAE1, TLBI VALE1
> >
> > The following software instruction is required to be downgraded
> > to IC IALLUIS: IC IVAU, Xt
> >
> > Specifically for the IC IVAU, Xt downgrade, setting SCTLR_EL1.UCI
> > to 0 will disable EL0 access to this instruction. Any attempt to
> > execute from EL0 will generate an EL1 trap, where the downgrade to
> > IC ALLUIS can be implemented.
> > --
> >
> > [1] https://www.nxp.com/docs/en/errata/IMX8_1N94W.pdf
> > [2] 307fd14d4b14 ("arm64: dts: imx: add imx8qm mek support")
> > [3] https://github.com/nxp-imx/linux-imx/blob/lf-
> 6.1.y/arch/arm64/include/asm/tlbflush.h#L19
> >
> > Signed-off-by: Ivan T. Ivanov <iivanov@suse.de>
> > ---
> >   Documentation/arm64/silicon-errata.rst |  2 ++
> >   arch/arm64/Kconfig                     | 10 ++++++++++
> >   arch/arm64/include/asm/cpufeature.h    |  3 ++-
> >   arch/arm64/include/asm/tlbflush.h      |  6 +++++-
> >   arch/arm64/kernel/cpu_errata.c         | 18 ++++++++++++++++++
> >   arch/arm64/kernel/traps.c              | 22 +++++++++++++++++++++-
> >   arch/arm64/tools/cpucaps               |  1 +
> >   7 files changed, 59 insertions(+), 3 deletions(-)
> >
> > diff --git a/Documentation/arm64/silicon-errata.rst
> b/Documentation/arm64/silicon-errata.rst
> > index ec5f889d7681..fce231797184 100644
> > --- a/Documentation/arm64/silicon-errata.rst
> > +++ b/Documentation/arm64/silicon-errata.rst
> > @@ -175,6 +175,8 @@ stable kernels.
> >   +----------------+-----------------+-----------------+-----------------------------+
> >   | Freescale/NXP  | LS2080A/LS1043A | A-008585        |
> FSL_ERRATUM_A008585         |
> >   +----------------+-----------------+-----------------+-----------------------------+
> > +| Freescale/NXP  | i.MX 8QuadMax   | ERR050104       |
> NXP_IMX8QM_ERRATUM_ERR050104|
> > ++----------------+-----------------+-----------------+-----------------------------+
> >   +----------------+-----------------+-----------------+-----------------------------+
> >   | Hisilicon      | Hip0{5,6,7}     | #161010101      |
> HISILICON_ERRATUM_161010101 |
> >   +----------------+-----------------+-----------------+-----------------------------+
> > diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> > index 1023e896d46b..437cb53f8753 100644
> > --- a/arch/arm64/Kconfig
> > +++ b/arch/arm64/Kconfig
> > @@ -1159,6 +1159,16 @@ config SOCIONEXT_SYNQUACER_PREITS
> >
> >   	  If unsure, say Y.
> >
> > +config NXP_IMX8QM_ERRATUM_ERR050104
> > +	bool "NXP iMX8QM: Workaround for Arm/A53 Cache coherency
> issue"
> > +	default n
> > +	help
> > +	  Some maintenance operations exchanged between the A53 and
> A72 core
> > +	  clusters, involving some Translation Look-aside Buffer Invalidate
> > +	  (TLBI) and Instruction Cache (IC) instructions can be corrupted.
> > +
> > +	  If unsure, say N.
> > +
> >   endmenu # "ARM errata workarounds via the alternatives framework"
> >
> >   choice
> > diff --git a/arch/arm64/include/asm/cpufeature.h
> b/arch/arm64/include/asm/cpufeature.h
> > index 6bf013fb110d..1ed648f7f29a 100644
> > --- a/arch/arm64/include/asm/cpufeature.h
> > +++ b/arch/arm64/include/asm/cpufeature.h
> > @@ -835,7 +835,8 @@ static inline bool system_supports_bti(void)
> >   static inline bool system_supports_tlb_range(void)
> >   {
> >   	return IS_ENABLED(CONFIG_ARM64_TLB_RANGE) &&
> > -		cpus_have_const_cap(ARM64_HAS_TLB_RANGE);
> > +		cpus_have_const_cap(ARM64_HAS_TLB_RANGE) &&
> > +
> 	!cpus_have_const_cap(ARM64_WORKAROUND_NXP_ERR050104);
> >   }
> >
> >   int do_emulate_mrs(struct pt_regs *regs, u32 sys_reg, u32 rt);
> > diff --git a/arch/arm64/include/asm/tlbflush.h
> b/arch/arm64/include/asm/tlbflush.h
> > index 412a3b9a3c25..12055b859ce3 100644
> > --- a/arch/arm64/include/asm/tlbflush.h
> > +++ b/arch/arm64/include/asm/tlbflush.h
> > @@ -37,7 +37,11 @@
> >   			    : : )
> >
> >   #define __TLBI_1(op, arg) asm (ARM64_ASM_PREAMBLE
> 	       \
> > -			       "tlbi " #op ", %0\n"			       \
> > +		   ALTERNATIVE("nop\n	nop\n  tlbi " #op ", %0",              \
> > +			       "tlbi vmalle1is\n  dsb ish\n  isb",	       \
> > +			       ARM64_WORKAROUND_NXP_ERR050104)
> 	       \
> > +			    : : "r" (arg));				       \
> > +			  asm (ARM64_ASM_PREAMBLE
> 	       \
> >   		   ALTERNATIVE("nop\n			nop",		       \
> >   			       "dsb ish\n		tlbi " #op ", %0",     \
> >   			       ARM64_WORKAROUND_REPEAT_TLBI,
> 	       \
> > diff --git a/arch/arm64/kernel/cpu_errata.c
> b/arch/arm64/kernel/cpu_errata.c
> > index 307faa2b4395..7b702a79bf60 100644
> > --- a/arch/arm64/kernel/cpu_errata.c
> > +++ b/arch/arm64/kernel/cpu_errata.c
> > @@ -8,6 +8,7 @@
> >   #include <linux/arm-smccc.h>
> >   #include <linux/types.h>
> >   #include <linux/cpu.h>
> > +#include <linux/of.h>
> >   #include <asm/cpu.h>
> >   #include <asm/cputype.h>
> >   #include <asm/cpufeature.h>
> > @@ -55,6 +56,14 @@ is_kryo_midr(const struct arm64_cpu_capabilities
> *entry, int scope)
> >   	return model == entry->midr_range.model;
> >   }
> >
> > +static bool __maybe_unused
> > +is_imx8qm_soc(const struct arm64_cpu_capabilities *entry, int scope)
> > +{
> > +	WARN_ON(preemptible());
> > +
> > +	return of_machine_is_compatible("fsl,imx8qm");
> > +}
> > +
> >   static bool
> >   has_mismatched_cache_type(const struct arm64_cpu_capabilities *entry,
> >   			  int scope)
> > @@ -729,6 +738,15 @@ const struct arm64_cpu_capabilities arm64_errata[]
> = {
> >   		MIDR_FIXED(MIDR_CPU_VAR_REV(1,1), BIT(25)),
> >   		.cpu_enable = cpu_clear_bf16_from_user_emulation,
> >   	},
> > +#endif
> > +#ifdef CONFIG_NXP_IMX8QM_ERRATUM_ERR050104
> > +	{
> > +		.desc = "NXP A53 cache coherency issue",
> > +		.capability = ARM64_WORKAROUND_NXP_ERR050104,
> > +		.type = ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE,
> > +		.matches = is_imx8qm_soc,
> > +		.cpu_enable = cpu_enable_cache_maint_trap,
> > +	},
> >   #endif
> >   	{
> >   	}
> > diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
> > index 4a79ba100799..4858f8c86fd5 100644
> > --- a/arch/arm64/kernel/traps.c
> > +++ b/arch/arm64/kernel/traps.c
> > @@ -529,6 +529,26 @@ void do_el1_fpac(struct pt_regs *regs, unsigned
> long esr)
> >   		uaccess_ttbr0_disable();			\
> >   	}
> >
> > +#define __user_instruction_cache_maint(address, res)		\
> > +do {								\
> > +	if (address >= TASK_SIZE_MAX) {				\
> > +		res = -EFAULT;					\
> > +	} else {						\
> > +		uaccess_ttbr0_enable();				\
> > +		asm volatile (					\
> > +			"1:\n"					\
> > +	    ALTERNATIVE("	ic	ivau, %1\n",		\
> > +			"	ic	ialluis\n",		\
> > +			ARM64_WORKAROUND_NXP_ERR050104)
> 	\
> > +			"	mov	%w0, #0\n"		\
> > +			"2:\n"					\
> > +			_ASM_EXTABLE_UACCESS_ERR(1b, 2b, %w0)	\
> > +			: "=r" (res)				\
> > +			: "r" (address));			\
> > +		uaccess_ttbr0_disable();			\
> > +	}							\
> > +} while (0)
> > +
> >   static void user_cache_maint_handler(unsigned long esr, struct pt_regs
> *regs)
> >   {
> >   	unsigned long tagged_address, address;
> > @@ -556,7 +576,7 @@ static void user_cache_maint_handler(unsigned
> long esr, struct pt_regs *regs)
> >   		__user_cache_maint("dc civac", address, ret);
> >   		break;
> >   	case ESR_ELx_SYS64_ISS_CRM_IC_IVAU:	/* IC IVAU */
> > -		__user_cache_maint("ic ivau", address, ret);
> > +		__user_instruction_cache_maint(address, ret);
> >   		break;
> >   	default:
> >   		force_signal_inject(SIGILL, ILL_ILLOPC, regs->pc, 0);
> > diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps
> > index 37b1340e9646..e225f1cd1005 100644
> > --- a/arch/arm64/tools/cpucaps
> > +++ b/arch/arm64/tools/cpucaps
> > @@ -90,3 +90,4 @@ WORKAROUND_NVIDIA_CARMEL_CNP
> >   WORKAROUND_QCOM_FALKOR_E1003
> >   WORKAROUND_REPEAT_TLBI
> >   WORKAROUND_SPECULATIVE_AT
> > +WORKAROUND_NXP_ERR050104
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH] arm64: errata: Add NXP iMX8QM workaround for A53 Cache coherency issue

[Mark Rutland]

On Fri, Apr 14, 2023 at 12:36:53PM +0100, Robin Murphy wrote:
> On 2023-04-13 12:19, Mark Rutland wrote:
> [...]
> > > diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
> > > index 6bf013fb110d..1ed648f7f29a 100644
> > > --- a/arch/arm64/include/asm/cpufeature.h
> > > +++ b/arch/arm64/include/asm/cpufeature.h
> > > @@ -835,7 +835,8 @@ static inline bool system_supports_bti(void)
> > >   static inline bool system_supports_tlb_range(void)
> > >   {
> > >   	return IS_ENABLED(CONFIG_ARM64_TLB_RANGE) &&
> > > -		cpus_have_const_cap(ARM64_HAS_TLB_RANGE);
> > > +		cpus_have_const_cap(ARM64_HAS_TLB_RANGE) &&
> > > +		!cpus_have_const_cap(ARM64_WORKAROUND_NXP_ERR050104);
> > >   }
> > 
> > It'd be better to handle this in the detection of ARM64_HAS_TLB_RANGE, as we
> > have for CNP where has_useable_cnp() checks for ARM64_WORKAROUND_NVIDIA_CARMEL_CNP.
> 
> It's not needed in either place, since neither Cortex-A53 or Cortex-A72
> support FEAT_TLBIRANGE, so this could never be true on affected platforms
> anyway.

Ah, even better -- we can just drop it.

> Tangentially, I understand this platform has an SMMU[1], so I'd say it would
> also be worth checking what SMMU_IDR0.BTM reports. With any luck it might be
> 0, but if it's 1 then strictly it would want to be overridden as part of a
> complete workaround as well. That wouldn't be a practical issue right now,
> not least since the current Linux driver doesn't even use BTM, but it's
> something which could need to be borne in mind in future.

Absolutely.

Mark.

> 
> Robin.
> 
> [1] https://lore.kernel.org/linux-arm-kernel/20210807104517.24066-1-peng.fan@oss.nxp.com/

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH] arm64: errata: Add NXP iMX8QM workaround for A53 Cache coherency issue

[Ivan T. Ivanov]

On 04-13 12:19, Mark Rutland wrote:
> > 
> > Unfortunately I was unable to find a way to identify SoC ID using
> > registers. Boot CPU MIDR_EL1 is equal to 0x410fd034. So I fallback to
> > using devicetree compatible strings for this.
> 
> How does this work with KVM?
> 
> VMs have no idea that the host platform is, and so will have no idea that this
> erratum applies, so they're going to blow up spectacularly.

I don't think this platform it is supposed to run KVM guests, but
who knows?! Someone suggested using SMBIOS tables or something around
this, but I am not sure how reliable way this is.

> 
> So we should probably be disabling KVM (or at the very least, printing a
> gigantic warning).

I will see if I can come up with something. Suggestions are of course
welcomed.

> 
> >  
> > +config NXP_IMX8QM_ERRATUM_ERR050104
> > +	bool "NXP iMX8QM: Workaround for Arm/A53 Cache coherency issue"
> 
> Please use the erratum number, e.g.
> 

Ok.

> 	bool "NXP iMX8QM ERR050104: broken cache/tlb invalidation"
> 
> > +	default n
> > +	help
> > +	  Some maintenance operations exchanged between the A53 and A72 core
> > +	  clusters, involving some Translation Look-aside Buffer Invalidate
> > +	  (TLBI) and Instruction Cache (IC) instructions can be corrupted.
> 
> Likewise, please add a more compelte description here, e.g.
> 
> 	help
> 	  On iMX8QM, addresses above bit 35 are not broadcast correctly for
> 	  TLBI or IC operations, making TLBI and IC unreliable.
> 
> 	  Work around this erratum by using TLBI *ALL*IS and IC IALLUIS
> 	  operations. EL0 use of IC IVAU is trapped and upgraded to IC IALLUIS.
>

Ok.

> > +
> > +	  If unsure, say N.
> > +
> >  endmenu # "ARM errata workarounds via the alternatives framework"
> >  
> >  choice
> > diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
> > index 6bf013fb110d..1ed648f7f29a 100644
> > --- a/arch/arm64/include/asm/cpufeature.h
> > +++ b/arch/arm64/include/asm/cpufeature.h
> > @@ -835,7 +835,8 @@ static inline bool system_supports_bti(void)
> >  static inline bool system_supports_tlb_range(void)
> >  {
> >  	return IS_ENABLED(CONFIG_ARM64_TLB_RANGE) &&
> > -		cpus_have_const_cap(ARM64_HAS_TLB_RANGE);
> > +		cpus_have_const_cap(ARM64_HAS_TLB_RANGE) &&
> > +		!cpus_have_const_cap(ARM64_WORKAROUND_NXP_ERR050104);
> >  }
> 
> It'd be better to handle this in the detection of ARM64_HAS_TLB_RANGE, as we
> have for CNP where has_useable_cnp() checks for ARM64_WORKAROUND_NVIDIA_CARMEL_CNP.

Thanks, I will rework this.

> 
> >  
> >  int do_emulate_mrs(struct pt_regs *regs, u32 sys_reg, u32 rt);
> > diff --git a/arch/arm64/include/asm/tlbflush.h b/arch/arm64/include/asm/tlbflush.h
> > index 412a3b9a3c25..12055b859ce3 100644
> > --- a/arch/arm64/include/asm/tlbflush.h
> > +++ b/arch/arm64/include/asm/tlbflush.h
> > @@ -37,7 +37,11 @@
> >  			    : : )
> >  
> >  #define __TLBI_1(op, arg) asm (ARM64_ASM_PREAMBLE			       \
> > -			       "tlbi " #op ", %0\n"			       \
> > +		   ALTERNATIVE("nop\n	nop\n  tlbi " #op ", %0",              \
> > +			       "tlbi vmalle1is\n  dsb ish\n  isb",	       \
> > +			       ARM64_WORKAROUND_NXP_ERR050104)		       \
> > +			    : : "r" (arg));				       \
> 
> Why do you need the DSB ISH + ISB here? It's up to the caller to issue those,
> and the ARM64_WORKAROUND_REPEAT_TLBI workaround only has DSB ISH to ensure that
> the first op completes before the second is issued.

Right, perhaps I was gone too far here :-)

> 
> > +			  asm (ARM64_ASM_PREAMBLE			       \
> >  		   ALTERNATIVE("nop\n			nop",		       \
> >  			       "dsb ish\n		tlbi " #op ", %0",     \
> >  			       ARM64_WORKAROUND_REPEAT_TLBI,		       \
> > diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
> > index 307faa2b4395..7b702a79bf60 100644
> > --- a/arch/arm64/kernel/cpu_errata.c
> > +++ b/arch/arm64/kernel/cpu_errata.c
> > @@ -8,6 +8,7 @@
> >  #include <linux/arm-smccc.h>
> >  #include <linux/types.h>
> >  #include <linux/cpu.h>
> > +#include <linux/of.h>
> >  #include <asm/cpu.h>
> >  #include <asm/cputype.h>
> >  #include <asm/cpufeature.h>
> > @@ -55,6 +56,14 @@ is_kryo_midr(const struct arm64_cpu_capabilities *entry, int scope)
> >  	return model == entry->midr_range.model;
> >  }
> >  
> > +static bool __maybe_unused
> > +is_imx8qm_soc(const struct arm64_cpu_capabilities *entry, int scope)
> > +{
> > +	WARN_ON(preemptible());
> > +
> > +	return of_machine_is_compatible("fsl,imx8qm");
> > +}
> 
> As above, what is going to be done for VMs, where this won't be present?

As I said, not sure this is usable for running VM's, but I am open for
suggestions. Maybe NXP has some register which could be used for SoC 
identification, but judging by downstream implementation there is none.

> 
> > +
> >  static bool
> >  has_mismatched_cache_type(const struct arm64_cpu_capabilities *entry,
> >  			  int scope)
> > @@ -729,6 +738,15 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
> >  		MIDR_FIXED(MIDR_CPU_VAR_REV(1,1), BIT(25)),
> >  		.cpu_enable = cpu_clear_bf16_from_user_emulation,
> >  	},
> > +#endif
> > +#ifdef CONFIG_NXP_IMX8QM_ERRATUM_ERR050104
> > +	{
> > +		.desc = "NXP A53 cache coherency issue",
> 
> Please use the erratum number, i.e. 
> 
> 		.desc = "NXP erratum ERR050104",

Ok.

> 
> > +		.capability = ARM64_WORKAROUND_NXP_ERR050104,
> > +		.type = ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE,
> > +		.matches = is_imx8qm_soc,
> > +		.cpu_enable = cpu_enable_cache_maint_trap,
> > +	},
> >  #endif
> >  	{
> >  	}
> > diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
> > index 4a79ba100799..4858f8c86fd5 100644
> > --- a/arch/arm64/kernel/traps.c
> > +++ b/arch/arm64/kernel/traps.c
> > @@ -529,6 +529,26 @@ void do_el1_fpac(struct pt_regs *regs, unsigned long esr)
> >  		uaccess_ttbr0_disable();			\
> >  	}
> >  
> > +#define __user_instruction_cache_maint(address, res)		\
> > +do {								\
> > +	if (address >= TASK_SIZE_MAX) {				\
> > +		res = -EFAULT;					\
> > +	} else {						\
> > +		uaccess_ttbr0_enable();				\
> > +		asm volatile (					\
> > +			"1:\n"					\
> > +	    ALTERNATIVE("	ic	ivau, %1\n",		\
> > +			"	ic	ialluis\n",		\
> > +			ARM64_WORKAROUND_NXP_ERR050104)		\
> > +			"	mov	%w0, #0\n"		\
> > +			"2:\n"					\
> > +			_ASM_EXTABLE_UACCESS_ERR(1b, 2b, %w0)	\
> > +			: "=r" (res)				\
> > +			: "r" (address));			\
> > +		uaccess_ttbr0_disable();			\
> > +	}							\
> > +} while (0)
> > +
> >  static void user_cache_maint_handler(unsigned long esr, struct pt_regs *regs)
> >  {
> >  	unsigned long tagged_address, address;
> > @@ -556,7 +576,7 @@ static void user_cache_maint_handler(unsigned long esr, struct pt_regs *regs)
> >  		__user_cache_maint("dc civac", address, ret);
> >  		break;
> >  	case ESR_ELx_SYS64_ISS_CRM_IC_IVAU:	/* IC IVAU */
> > -		__user_cache_maint("ic ivau", address, ret);
> > +		__user_instruction_cache_maint(address, ret);
> >  		break;
> 
> Hmm... this will silently change any 'IC IVAU' to never fault. That's probably
> not the end of the world, since it's IMP-DEF whether IC would raise a
> permission fault, but it is a change of behaviour.
> 
> It would be a bit simpler to handle this inline within the switch, e.g.
> 
> 	case ESR_ELx_SYS64_ISS_CRM_IC_IVAU:     /* IC IVAU */
> 		if (cpus_have_final_cap(ARM64_WORKAROUND_NXP_ERR050104)) {
> 			asm volatile("ic ialluis");
> 			ret = 0;
> 			break;
> 		}
> 		__user_instruction_cache_maint(address, ret);
> 		break;
> 
> ... as that would avoid duplicating the bulk of the __user_cache_maint() macro.

Ok. 

> 
> 
> >  	default:
> >  		force_signal_inject(SIGILL, ILL_ILLOPC, regs->pc, 0);
> > diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps
> > index 37b1340e9646..e225f1cd1005 100644
> > --- a/arch/arm64/tools/cpucaps
> > +++ b/arch/arm64/tools/cpucaps
> > @@ -90,3 +90,4 @@ WORKAROUND_NVIDIA_CARMEL_CNP
> >  WORKAROUND_QCOM_FALKOR_E1003
> >  WORKAROUND_REPEAT_TLBI
> >  WORKAROUND_SPECULATIVE_AT
> > +WORKAROUND_NXP_ERR050104
> 
> These definitions are expected to be ordered alphabetically, so this should be
> earlier in the list.
> 

Ok. Thanks!
Ivan


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH] arm64: errata: Add NXP iMX8QM workaround for A53 Cache coherency issue

[Ivan T. Ivanov]

On 04-17 16:35, Mark Rutland wrote:

> > > > @@ -835,7 +835,8 @@ static inline bool system_supports_bti(void)
> > > >   static inline bool system_supports_tlb_range(void)
> > > >   {
> > > >   	return IS_ENABLED(CONFIG_ARM64_TLB_RANGE) &&
> > > > -		cpus_have_const_cap(ARM64_HAS_TLB_RANGE);
> > > > +		cpus_have_const_cap(ARM64_HAS_TLB_RANGE) &&
> > > > +		!cpus_have_const_cap(ARM64_WORKAROUND_NXP_ERR050104);
> > > >   }
> > > 
> > > It'd be better to handle this in the detection of ARM64_HAS_TLB_RANGE, as we
> > > have for CNP where has_useable_cnp() checks for ARM64_WORKAROUND_NVIDIA_CARMEL_CNP.
> > 
> > It's not needed in either place, since neither Cortex-A53 or Cortex-A72
> > support FEAT_TLBIRANGE, so this could never be true on affected platforms
> > anyway.
> 
> Ah, even better -- we can just drop it.

Ok.

> 
> > Tangentially, I understand this platform has an SMMU[1], so I'd say it would
> > also be worth checking what SMMU_IDR0.BTM reports. With any luck it might be
> > 0, but if it's 1 then strictly it would want to be overridden as part of a
> > complete workaround as well. That wouldn't be a practical issue right now,
> > not least since the current Linux driver doesn't even use BTM, but it's
> > something which could need to be borne in mind in future.
> 
> Absolutely.

I don't completely understand implication of this, but for SMMU inside 
iMX8QM report that "Broadcast TLB maintenance is supported"

Regards,
Ivan

> 
> > 
> > [1] https://lore.kernel.org/linux-arm-kernel/20210807104517.24066-1-peng.fan@oss.nxp.com/

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH] arm64: errata: Add NXP iMX8QM workaround for A53 Cache coherency issue

[Robin Murphy]

On 18/04/2023 5:54 pm, Ivan T. Ivanov wrote:
> On 04-17 16:35, Mark Rutland wrote:
> 
>>>>> @@ -835,7 +835,8 @@ static inline bool system_supports_bti(void)
>>>>>    static inline bool system_supports_tlb_range(void)
>>>>>    {
>>>>>    	return IS_ENABLED(CONFIG_ARM64_TLB_RANGE) &&
>>>>> -		cpus_have_const_cap(ARM64_HAS_TLB_RANGE);
>>>>> +		cpus_have_const_cap(ARM64_HAS_TLB_RANGE) &&
>>>>> +		!cpus_have_const_cap(ARM64_WORKAROUND_NXP_ERR050104);
>>>>>    }
>>>>
>>>> It'd be better to handle this in the detection of ARM64_HAS_TLB_RANGE, as we
>>>> have for CNP where has_useable_cnp() checks for ARM64_WORKAROUND_NVIDIA_CARMEL_CNP.
>>>
>>> It's not needed in either place, since neither Cortex-A53 or Cortex-A72
>>> support FEAT_TLBIRANGE, so this could never be true on affected platforms
>>> anyway.
>>
>> Ah, even better -- we can just drop it.
> 
> Ok.
> 
>>
>>> Tangentially, I understand this platform has an SMMU[1], so I'd say it would
>>> also be worth checking what SMMU_IDR0.BTM reports. With any luck it might be
>>> 0, but if it's 1 then strictly it would want to be overridden as part of a
>>> complete workaround as well. That wouldn't be a practical issue right now,
>>> not least since the current Linux driver doesn't even use BTM, but it's
>>> something which could need to be borne in mind in future.
>>
>> Absolutely.
> 
> I don't completely understand implication of this, but for SMMU inside
> iMX8QM report that "Broadcast TLB maintenance is supported"

Aha, in fact it seems we might be OK. I double-checked and it turns out 
that thanks to an MMU-500 erratum, we can't necessarily believe IDR0.BTM 
anyway. Thus anyone who adds DVM support to an SMMUv2 driver is going to 
need to implement some new firmware property or platform detection, at 
which point the chances of it getting silently enabled for i.MX8 (if 
indeed BTM isn't already a lie there) seem sufficiently remote that I'd 
feel fairly comfortable not doing anything explicit for now.

(The case of concern wouldn't be so much the expected use of DVM to 
share CPU pagetables with the SMMU, which in principle the regular CPU 
workaround should already cover, but more anyone trying to play clever 
tricks using DVM and CPU instructions to maintain private SMMU contexts 
rather than register-based commands.)

Thanks,
Robin.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH] arm64: errata: Add NXP iMX8QM workaround for A53 cache coherency issue

[Peng Fan (OSS)]

From: Peng Fan <peng.fan@nxp.com>

According to NXP errata document IMX8_1N94W[1], the i.MX8QuadMax SoC
suffers from a serious cache coherency issue (ERR050104). The upper
bits, above bit 35, of the ARADDR and ACADDR buses within the Arm A53
subsystem have been incorrectly connected. This causes some TLBI and IC
maintenance operations exchanged between the A53 and A72 core clusters
to be corrupted.

The workaround requires:

  - Downgrading targeted TLBI operations to broadcast-all variants.
    Instead of patching the low-level __TLBI_1 macro (which interferes
    with the REPEAT_TLBI workaround and causes excessive over-
    invalidation), redirect high-level TLB flush functions
    (flush_tlb_mm, __do_flush_tlb_range, flush_tlb_kernel_range,
    __flush_tlb_kernel_pgtable) to use VMALLE1IS via static key checks.

  - Upgrading IC IVAU to IC IALLUIS for both kernel (via ALTERNATIVE in
    invalidate_icache_by_line) and EL0 userspace (via trap-and-upgrade
    in user_cache_maint_handler with SCTLR_EL1.UCI=0).

  - Disabling KVM since correct TLB maintenance cannot be guaranteed
    for guests without the proper devicetree.

  - No need to touch SMMU Broadcast TLB Maintenance (BTM) since i.MX8QM
    does not support broadcast TLB.

SoC detection uses devicetree compatible string "fsl,imx8qm" since the
boot CPU MIDR_EL1 (0x410fd034) and AIDR_EL1 (0) are not unique to this
SoC.

[1] https://www.nxp.com/docs/en/errata/IMX8_1N94W.pdf

[ Reworked per review feedback from Will Deacon and Mark Rutland:
  - Move TLBI workaround from __TLBI_1 macro to high-level flush
    functions to avoid REPEAT_TLBI interaction issues
  - Add kernel IC IVAU upgrade via ALTERNATIVE in assembler.h
  - Add cpucap_is_possible() entry for compile-time elimination]

Co-developed-by: Ivan T. Ivanov <iivanov@suse.de>
Signed-off-by: Ivan T. Ivanov <iivanov@suse.de>
Link: https://lore.kernel.org/all/20230420112952.28340-1-iivanov@suse.de/
Signed-off-by: Peng Fan <peng.fan@nxp.com>
---
This picks up the work originally done by Ivan T. Ivanov in 2023 [1],
reworked to address review feedback from Will Deacon and Mark Rutland,
and rebased onto linux-next (next-20260723).

Changes from v2 [1]:
  - Moved TLBI workaround from __TLBI_1 macro to high-level flush
    functions (flush_tlb_mm, __do_flush_tlb_range, flush_tlb_kernel_range,
    __flush_tlb_kernel_pgtable) to avoid REPEAT_TLBI interaction issues,
    per Will Deacon suggestion to use static keys at the higher level.
  - Added kernel IC IVAU -> IC IALLUIS upgrade via ALTERNATIVE in
    assembler.h, per Mark Rutland review noting that only EL0 traps
    were handled in v2.
  - Added cpucap_is_possible() entry for compile-time elimination when
    CONFIG_NXP_IMX8QM_ERRATUM_ERR050104 is not set.
  - Dropped SMMU BTM changes since iMX8QM does not support broadcast TLB.

[1] https://lore.kernel.org/all/20230420112952.28340-1-iivanov@suse.de/
[2] https://www.nxp.com/docs/en/errata/IMX8_1N94W.pdf
---
 Documentation/arch/arm64/silicon-errata.rst |  2 +
 arch/arm64/Kconfig                          | 16 ++++++++
 arch/arm64/include/asm/assembler.h          |  5 +++
 arch/arm64/include/asm/cpucaps.h            |  2 +
 arch/arm64/include/asm/tlbflush.h           | 61 +++++++++++++++++------------
 arch/arm64/kernel/cpu_errata.c              | 20 ++++++++++
 arch/arm64/kernel/traps.c                   |  6 +++
 arch/arm64/kvm/arm.c                        |  5 +++
 arch/arm64/tools/cpucaps                    |  1 +
 9 files changed, 94 insertions(+), 24 deletions(-)

diff --git a/Documentation/arch/arm64/silicon-errata.rst b/Documentation/arch/arm64/silicon-errata.rst
index 868bd9eed9a6..a90fd2ec532b 100644
--- a/Documentation/arch/arm64/silicon-errata.rst
+++ b/Documentation/arch/arm64/silicon-errata.rst
@@ -321,6 +321,8 @@ stable kernels.
 +----------------+-----------------+-----------------+-----------------------------+
 | Freescale/NXP  | LS2080A/LS1043A | A-008585        | FSL_ERRATUM_A008585         |
 +----------------+-----------------+-----------------+-----------------------------+
+| Freescale/NXP  | i.MX 8QuadMax   | ERR050104       | NXP_IMX8QM_ERRATUM_ERR050104|
++----------------+-----------------+-----------------+-----------------------------+
 +----------------+-----------------+-----------------+-----------------------------+
 | Hisilicon      | Hip0{5,6,7}     | #161010101      | HISILICON_ERRATUM_161010101 |
 +----------------+-----------------+-----------------+-----------------------------+
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index faccdf847a67..eea6774baf0d 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -1454,6 +1454,22 @@ config ROCKCHIP_ERRATUM_3588001
 
 	  If unsure, say Y.
 
+config NXP_IMX8QM_ERRATUM_ERR050104
+	bool "NXP iMX8QM ERR050104: broken cache/TLB invalidation broadcast"
+	default y
+	help
+	  On iMX8QM, address bits above bit 35 in the A53 subsystem ARADDR and
+	  ACADDR buses are incorrectly connected. This corrupts targeted TLBI
+	  and IC broadcasts exchanged between the A53 and A72 core clusters.
+
+	  Work around this by redirecting targeted TLBI operations to
+	  broadcast-all variants (VMALLE1IS) in the high-level TLB flush
+	  functions, upgrading IC IVAU to IC IALLUIS for both kernel and
+	  user-space, and disabling KVM since correct TLB maintenance
+	  cannot be guaranteed for guests.
+
+	  If unsure, say Y.
+
 config SOCIONEXT_SYNQUACER_PREITS
 	bool "Socionext Synquacer: Workaround for GICv3 pre-ITS"
 	default y
diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h
index effae53e9739..b35c9308e32b 100644
--- a/arch/arm64/include/asm/assembler.h
+++ b/arch/arm64/include/asm/assembler.h
@@ -455,6 +455,10 @@ alternative_else_nop_endif
  * 	Corrupts:	tmp1, tmp2
  */
 	.macro invalidate_icache_by_line start, end, tmp1, tmp2, fixup
+alternative_if ARM64_WORKAROUND_NXP_ERR050104
+	ic	ialluis
+	b	.Licache_done\@
+alternative_else_nop_endif
 	icache_line_size \tmp1, \tmp2
 	sub	\tmp2, \tmp1, #1
 	bic	\tmp2, \start, \tmp2
@@ -463,6 +467,7 @@ alternative_else_nop_endif
 	add	\tmp2, \tmp2, \tmp1
 	cmp	\tmp2, \end
 	b.lo	.Licache_op\@
+.Licache_done\@:
 	dsb	ish
 	isb
 
diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h
index 76350b38f0d7..0c3bfbe99aad 100644
--- a/arch/arm64/include/asm/cpucaps.h
+++ b/arch/arm64/include/asm/cpucaps.h
@@ -66,6 +66,8 @@ cpucap_is_possible(const unsigned int cap)
 		return IS_ENABLED(CONFIG_ARM64_ERRATUM_3194386);
 	case ARM64_WORKAROUND_4193714:
 		return IS_ENABLED(CONFIG_ARM64_ERRATUM_4193714);
+	case ARM64_WORKAROUND_NXP_ERR050104:
+		return IS_ENABLED(CONFIG_NXP_IMX8QM_ERRATUM_ERR050104);
 	case ARM64_MPAM:
 		/*
 		 * KVM MPAM support doesn't rely on the host kernel supporting MPAM.
diff --git a/arch/arm64/include/asm/tlbflush.h b/arch/arm64/include/asm/tlbflush.h
index 14a78ac0f800..87843e602127 100644
--- a/arch/arm64/include/asm/tlbflush.h
+++ b/arch/arm64/include/asm/tlbflush.h
@@ -378,9 +378,13 @@ static inline void flush_tlb_mm(struct mm_struct *mm)
 	unsigned long asid;
 
 	dsb(ishst);
-	asid = __TLBI_VADDR(0, ASID(mm));
-	__tlbi(aside1is, asid);
-	__tlbi_user(aside1is, asid);
+	if (alternative_has_cap_unlikely(ARM64_WORKAROUND_NXP_ERR050104)) {
+		__tlbi(vmalle1is);
+	} else {
+		asid = __TLBI_VADDR(0, ASID(mm));
+		__tlbi(aside1is, asid);
+		__tlbi_user(aside1is, asid);
+	}
 	__tlbi_sync_s1ish(mm);
 	mmu_notifier_arch_invalidate_secondary_tlbs(mm, 0, -1UL);
 }
@@ -580,23 +584,27 @@ static __always_inline void __do_flush_tlb_range(struct vm_area_struct *vma,
 
 	asid = ASID(mm);
 
-	switch (flags & (TLBF_NOWALKCACHE | TLBF_NOBROADCAST)) {
-	case TLBF_NONE:
-		__flush_s1_tlb_range_op(vae1is, start, pages, stride,
-					asid, tlb_level);
-		break;
-	case TLBF_NOWALKCACHE:
-		__flush_s1_tlb_range_op(vale1is, start, pages, stride,
-					asid, tlb_level);
-		break;
-	case TLBF_NOBROADCAST:
-		/* Combination unused */
-		BUG();
-		break;
-	case TLBF_NOWALKCACHE | TLBF_NOBROADCAST:
-		__flush_s1_tlb_range_op(vale1, start, pages, stride,
-					asid, tlb_level);
-		break;
+	if (alternative_has_cap_unlikely(ARM64_WORKAROUND_NXP_ERR050104)) {
+		__tlbi(vmalle1is);
+	} else {
+		switch (flags & (TLBF_NOWALKCACHE | TLBF_NOBROADCAST)) {
+		case TLBF_NONE:
+			__flush_s1_tlb_range_op(vae1is, start, pages, stride,
+						asid, tlb_level);
+			break;
+		case TLBF_NOWALKCACHE:
+			__flush_s1_tlb_range_op(vale1is, start, pages, stride,
+						asid, tlb_level);
+			break;
+		case TLBF_NOBROADCAST:
+			/* Combination unused */
+			BUG();
+			break;
+		case TLBF_NOWALKCACHE | TLBF_NOBROADCAST:
+			__flush_s1_tlb_range_op(vale1, start, pages, stride,
+						asid, tlb_level);
+			break;
+		}
 	}
 
 	if (!(flags & TLBF_NONOTIFY))
@@ -657,7 +665,8 @@ static inline void flush_tlb_kernel_range(unsigned long start, unsigned long end
 	end = round_up(end, stride);
 	pages = (end - start) >> PAGE_SHIFT;
 
-	if (__flush_tlb_range_limit_excess(pages, stride)) {
+	if (alternative_has_cap_unlikely(ARM64_WORKAROUND_NXP_ERR050104) ||
+	    __flush_tlb_range_limit_excess(pages, stride)) {
 		flush_tlb_all();
 		return;
 	}
@@ -675,10 +684,14 @@ static inline void flush_tlb_kernel_range(unsigned long start, unsigned long end
  */
 static inline void __flush_tlb_kernel_pgtable(unsigned long kaddr)
 {
-	unsigned long addr = __TLBI_VADDR(kaddr, 0);
-
 	dsb(ishst);
-	__tlbi(vaae1is, addr);
+	if (alternative_has_cap_unlikely(ARM64_WORKAROUND_NXP_ERR050104)) {
+		__tlbi(vmalle1is);
+	} else {
+		unsigned long addr = __TLBI_VADDR(kaddr, 0);
+
+		__tlbi(vaae1is, addr);
+	}
 	__tlbi_sync_s1ish_kernel();
 	isb();
 }
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 5db8f0619e4b..1866eeec27ca 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -6,6 +6,7 @@
  */
 
 #include <linux/arm-smccc.h>
+#include <linux/of.h>
 #include <linux/types.h>
 #include <linux/cpu.h>
 #include <asm/cpu.h>
@@ -200,6 +201,16 @@ cpu_enable_cache_maint_trap(const struct arm64_cpu_capabilities *__unused)
 	sysreg_clear_set(sctlr_el1, SCTLR_EL1_UCI, 0);
 }
 
+#ifdef CONFIG_NXP_IMX8QM_ERRATUM_ERR050104
+static bool
+is_imx8qm_soc(const struct arm64_cpu_capabilities *entry, int scope)
+{
+	WARN_ON(preemptible());
+
+	return of_machine_is_compatible("fsl,imx8qm");
+}
+#endif
+
 #define CAP_MIDR_RANGE(model, v_min, r_min, v_max, r_max)	\
 	.matches = is_affected_midr_range,			\
 	.midr_range = MIDR_RANGE(model, v_min, r_min, v_max, r_max)
@@ -1030,6 +1041,15 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
 		.type = ARM64_CPUCAP_SYSTEM_FEATURE,
 		.matches = has_broken_gic_v3_seis,
 	},
+#ifdef CONFIG_NXP_IMX8QM_ERRATUM_ERR050104
+	{
+		.desc = "NXP erratum ERR050104",
+		.capability = ARM64_WORKAROUND_NXP_ERR050104,
+		.type = ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE,
+		.matches = is_imx8qm_soc,
+		.cpu_enable = cpu_enable_cache_maint_trap,
+	},
+#endif
 	{
 	}
 };
diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
index 914282016069..beb658a007d5 100644
--- a/arch/arm64/kernel/traps.c
+++ b/arch/arm64/kernel/traps.c
@@ -585,6 +585,12 @@ static void user_cache_maint_handler(unsigned long esr, struct pt_regs *regs)
 		__user_cache_maint("dc civac", address, ret);
 		break;
 	case ESR_ELx_SYS64_ISS_CRM_IC_IVAU:	/* IC IVAU */
+		if (cpus_have_final_cap(ARM64_WORKAROUND_NXP_ERR050104)) {
+			/* ERR050104: upgrade IC IVAU to IC IALLUIS */
+			asm volatile("ic ialluis");
+			ret = 0;
+			break;
+		}
 		__user_cache_maint("ic ivau", address, ret);
 		break;
 	default:
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 9a6c72a18672..734ccd7ec056 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -3032,6 +3032,11 @@ static __init int kvm_arm_init(void)
 
 	in_hyp_mode = is_kernel_in_hyp_mode();
 
+	if (cpus_have_final_cap(ARM64_WORKAROUND_NXP_ERR050104)) {
+		kvm_info("KVM not supported on NXP iMX8QM (ERR050104)\n");
+		return -ENODEV;
+	}
+
 	if (cpus_have_final_cap(ARM64_WORKAROUND_DEVICE_LOAD_ACQUIRE) ||
 	    cpus_have_final_cap(ARM64_WORKAROUND_1508412))
 		kvm_info("Guests without required CPU erratum workarounds can deadlock system!\n" \
diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps
index fbf27e887e71..fb3fe9736cc6 100644
--- a/arch/arm64/tools/cpucaps
+++ b/arch/arm64/tools/cpucaps
@@ -123,6 +123,7 @@ WORKAROUND_DEVICE_LOAD_ACQUIRE
 WORKAROUND_DISABLE_CNP
 WORKAROUND_GICv3_BROKEN_SEIS
 WORKAROUND_NVIDIA_OLYMPUS_1027
+WORKAROUND_NXP_ERR050104
 WORKAROUND_PMUV3_IMPDEF_TRAPS
 WORKAROUND_QCOM_FALKOR_E1003
 WORKAROUND_QCOM_ORYON_CNTVOFF

---
base-commit: 9eebf259d5352b87080d67758f483583d9e763d7
change-id: 20260729-imx8qm-cache-coherency-3129d5da3de1

Best regards,
--  
Peng Fan <peng.fan@nxp.com>

Re: [PATCH] arm64: errata: Add NXP iMX8QM workaround for A53 cache coherency issue

[Mark Rutland]

On Wed, Jul 29, 2026 at 11:28:52AM +0800, Peng Fan (OSS) wrote:
> From: Peng Fan <peng.fan@nxp.com>
> 
> According to NXP errata document IMX8_1N94W[1], the i.MX8QuadMax SoC
> suffers from a serious cache coherency issue (ERR050104). The upper
> bits, above bit 35, of the ARADDR and ACADDR buses within the Arm A53
> subsystem have been incorrectly connected. This causes some TLBI and IC
> maintenance operations exchanged between the A53 and A72 core clusters
> to be corrupted.
> 
> The workaround requires:
> 
>   - Downgrading targeted TLBI operations to broadcast-all variants.
>     Instead of patching the low-level __TLBI_1 macro (which interferes
>     with the REPEAT_TLBI workaround and causes excessive over-
>     invalidation), redirect high-level TLB flush functions
>     (flush_tlb_mm, __do_flush_tlb_range, flush_tlb_kernel_range,
>     __flush_tlb_kernel_pgtable) to use VMALLE1IS via static key checks.
> 
>   - Upgrading IC IVAU to IC IALLUIS for both kernel (via ALTERNATIVE in
>     invalidate_icache_by_line) and EL0 userspace (via trap-and-upgrade
>     in user_cache_maint_handler with SCTLR_EL1.UCI=0).
> 
>   - Disabling KVM since correct TLB maintenance cannot be guaranteed
>     for guests without the proper devicetree.
> 
>   - No need to touch SMMU Broadcast TLB Maintenance (BTM) since i.MX8QM
>     does not support broadcast TLB.
> 
> SoC detection uses devicetree compatible string "fsl,imx8qm" since the
> boot CPU MIDR_EL1 (0x410fd034) and AIDR_EL1 (0) are not unique to this
> SoC.
> 
> [1] https://www.nxp.com/docs/en/errata/IMX8_1N94W.pdf

[...]

> +	if (alternative_has_cap_unlikely(ARM64_WORKAROUND_NXP_ERR050104)) {
> +		__tlbi(vmalle1is);
> +	} else {
> +		switch (flags & (TLBF_NOWALKCACHE | TLBF_NOBROADCAST)) {
> +		case TLBF_NONE:
> +			__flush_s1_tlb_range_op(vae1is, start, pages, stride,
> +						asid, tlb_level);
> +			break;
> +		case TLBF_NOWALKCACHE:
> +			__flush_s1_tlb_range_op(vale1is, start, pages, stride,
> +						asid, tlb_level);
> +			break;
> +		case TLBF_NOBROADCAST:
> +			/* Combination unused */
> +			BUG();
> +			break;
> +		case TLBF_NOWALKCACHE | TLBF_NOBROADCAST:
> +			__flush_s1_tlb_range_op(vale1, start, pages, stride,
> +						asid, tlb_level);
> +			break;
> +		}
>  	}

Are you sure the workaround needs to be applied for the TLBF_NOBROADCAST
cases? The NXP document lists non-broadcast TLBI instructions, but I
strongly suspect that's in error, as those shouldn't result in
transactions over the core's external interfaces.

[...]

> @@ -585,6 +585,12 @@ static void user_cache_maint_handler(unsigned long esr, struct pt_regs *regs)
>  		__user_cache_maint("dc civac", address, ret);
>  		break;
>  	case ESR_ELx_SYS64_ISS_CRM_IC_IVAU:	/* IC IVAU */
> +		if (cpus_have_final_cap(ARM64_WORKAROUND_NXP_ERR050104)) {
> +			/* ERR050104: upgrade IC IVAU to IC IALLUIS */
> +			asm volatile("ic ialluis");
> +			ret = 0;
> +			break;
> +		}
>  		__user_cache_maint("ic ivau", address, ret);
>  		break;

This doesn't address my feedback on v1:

  https://lore.kernel.org/all/ZIHisLL9FXlbuMLJ@FVFF77S0Q05N/

... which I see Sashiko has also spotted.

Please make this:

	 case ESR_ELx_SYS64_ISS_CRM_IC_IVAU:     /* IC IVAU */
	 	__user_cache_maint("ic ivau", address, ret)
		if (cpus_have_final_cap(ARM64_WORKAROUND_NXP_ERR050104) && !ret)
			asm volatile("ic ialluis");
		break;

If there's a reason that doesn't work, please explain that reason.

Mark.

Re: [PATCH] arm64: errata: Add NXP iMX8QM workaround for A53 cache coherency issue

[Peng Fan]

Hi Mark,

Thanks for your reviewing.

On Wed, Jul 29, 2026 at 10:50:56AM +0100, Mark Rutland wrote:
>On Wed, Jul 29, 2026 at 11:28:52AM +0800, Peng Fan (OSS) wrote:
>> From: Peng Fan <peng.fan@nxp.com>
>> 
>> According to NXP errata document IMX8_1N94W[1], the i.MX8QuadMax SoC
>> suffers from a serious cache coherency issue (ERR050104). The upper
>> bits, above bit 35, of the ARADDR and ACADDR buses within the Arm A53
>> subsystem have been incorrectly connected. This causes some TLBI and IC
>> maintenance operations exchanged between the A53 and A72 core clusters
>> to be corrupted.
>> 
>> The workaround requires:
>> 
>>   - Downgrading targeted TLBI operations to broadcast-all variants.
>>     Instead of patching the low-level __TLBI_1 macro (which interferes
>>     with the REPEAT_TLBI workaround and causes excessive over-
>>     invalidation), redirect high-level TLB flush functions
>>     (flush_tlb_mm, __do_flush_tlb_range, flush_tlb_kernel_range,
>>     __flush_tlb_kernel_pgtable) to use VMALLE1IS via static key checks.
>> 
>>   - Upgrading IC IVAU to IC IALLUIS for both kernel (via ALTERNATIVE in
>>     invalidate_icache_by_line) and EL0 userspace (via trap-and-upgrade
>>     in user_cache_maint_handler with SCTLR_EL1.UCI=0).
>> 
>>   - Disabling KVM since correct TLB maintenance cannot be guaranteed
>>     for guests without the proper devicetree.
>> 
>>   - No need to touch SMMU Broadcast TLB Maintenance (BTM) since i.MX8QM
>>     does not support broadcast TLB.
>> 
>> SoC detection uses devicetree compatible string "fsl,imx8qm" since the
>> boot CPU MIDR_EL1 (0x410fd034) and AIDR_EL1 (0) are not unique to this
>> SoC.
>> 
>> [1] https://www.nxp.com/docs/en/errata/IMX8_1N94W.pdf
>
>[...]
>
>> +	if (alternative_has_cap_unlikely(ARM64_WORKAROUND_NXP_ERR050104)) {
>> +		__tlbi(vmalle1is);
>> +	} else {
>> +		switch (flags & (TLBF_NOWALKCACHE | TLBF_NOBROADCAST)) {
>> +		case TLBF_NONE:
>> +			__flush_s1_tlb_range_op(vae1is, start, pages, stride,
>> +						asid, tlb_level);
>> +			break;
>> +		case TLBF_NOWALKCACHE:
>> +			__flush_s1_tlb_range_op(vale1is, start, pages, stride,
>> +						asid, tlb_level);
>> +			break;
>> +		case TLBF_NOBROADCAST:
>> +			/* Combination unused */
>> +			BUG();
>> +			break;
>> +		case TLBF_NOWALKCACHE | TLBF_NOBROADCAST:
>> +			__flush_s1_tlb_range_op(vale1, start, pages, stride,
>> +						asid, tlb_level);
>> +			break;
>> +		}
>>  	}
>
>Are you sure the workaround needs to be applied for the TLBF_NOBROADCAST
>cases? The NXP document lists non-broadcast TLBI instructions, but I
>strongly suspect that's in error, as those shouldn't result in
>transactions over the core's external interfaces.

I think you are right, TLBF_NOBROADCAST should not be applied with the
workaround.

I will use below if this looks good to you.

 if (alternative_has_cap_unlikely(ARM64_WORKAROUND_NXP_ERR050104) &&
     !(flags & TLBF_NOBROADCAST)) {
         __tlbi(vmalle1is);
 } else {
 ...
 }

>
>[...]
>
>> @@ -585,6 +585,12 @@ static void user_cache_maint_handler(unsigned long esr, struct pt_regs *regs)
>>  		__user_cache_maint("dc civac", address, ret);
>>  		break;
>>  	case ESR_ELx_SYS64_ISS_CRM_IC_IVAU:	/* IC IVAU */
>> +		if (cpus_have_final_cap(ARM64_WORKAROUND_NXP_ERR050104)) {
>> +			/* ERR050104: upgrade IC IVAU to IC IALLUIS */
>> +			asm volatile("ic ialluis");
>> +			ret = 0;
>> +			break;
>> +		}
>>  		__user_cache_maint("ic ivau", address, ret);
>>  		break;
>
>This doesn't address my feedback on v1:
>
>  https://lore.kernel.org/all/ZIHisLL9FXlbuMLJ@FVFF77S0Q05N/

Sorry for missing this one.

>
>... which I see Sashiko has also spotted.
>
>Please make this:
>
>	 case ESR_ELx_SYS64_ISS_CRM_IC_IVAU:     /* IC IVAU */
>	 	__user_cache_maint("ic ivau", address, ret)
>		if (cpus_have_final_cap(ARM64_WORKAROUND_NXP_ERR050104) && !ret)
>			asm volatile("ic ialluis");
>		break;
>
>If there's a reason that doesn't work, please explain that reason.

It should be ok.  I will take your suggestion in next version.

Thanks,
Peng

>
>Mark.
>
>