Re: [PATCH 3/3] arm64: errata: Expand speculative SSBS workaround (again)

[Anshuman Khandual <anshuman.khandual@arm.com>]

On 8/1/24 15:48, Mark Rutland wrote:
> A number of Arm Ltd CPUs suffer from errata whereby an MSR to the SSBS
> special-purpose register does not affect subsequent speculative
> instructions, permitting speculative store bypassing for a window of
> time.
> 
> We worked around this for a number of CPUs in commits:
> 
> * 7187bb7d0b5c7dfa ("arm64: errata: Add workaround for Arm errata 3194386 and 3312417")
> * 75b3c43eab594bfb ("arm64: errata: Expand speculative SSBS workaround")
> 
> Since then, similar errata have been published for a number of other Arm
> Ltd CPUs, for which the same mitigation is sufficient. This is described
> in their respective Software Developer Errata Notice (SDEN) documents:
> 
> * Cortex-A76 (MP052) SDEN v31.0, erratum 3324349
>   https://developer.arm.com/documentation/SDEN-885749/3100/
> 
> * Cortex-A77 (MP074) SDEN v19.0, erratum 3324348
>   https://developer.arm.com/documentation/SDEN-1152370/1900/
> 
> * Cortex-A78 (MP102) SDEN v21.0, erratum 3324344
>   https://developer.arm.com/documentation/SDEN-1401784/2100/
> 
> * Cortex-A78C (MP138) SDEN v16.0, erratum 3324346
>   https://developer.arm.com/documentation/SDEN-1707916/1600/
> 
> * Cortex-A78C (MP154) SDEN v10.0, erratum 3324347
>   https://developer.arm.com/documentation/SDEN-2004089/1000/
> 
> * Cortex-A725 (MP190) SDEN v5.0, erratum 3456106
>   https://developer.arm.com/documentation/SDEN-2832921/0500/
> 
> * Cortex-X1 (MP077) SDEN v21.0, erratum 3324344
>   https://developer.arm.com/documentation/SDEN-1401782/2100/
> 
> * Cortex-X1C (MP136) SDEN v16.0, erratum 3324346
>   https://developer.arm.com/documentation/SDEN-1707914/1600/
> 
> * Neoverse-N1 (MP050) SDEN v32.0, erratum 3324349
>   https://developer.arm.com/documentation/SDEN-885747/3200/
> 
> * Neoverse-V1 (MP076) SDEN v19.0, erratum 3324341
>   https://developer.arm.com/documentation/SDEN-1401781/1900/
> 
> Note that due to the manner in which Arm develops IP and tracks errata,
> some CPUs share a common erratum number and some CPUs have multiple
> erratum numbers for the same HW issue.
> 
> On parts without SB, it is necessary to use ISB for the workaround. The
> spec_bar() macro used in the mitigation will expand to a "DSB SY; ISB"
> sequence in this case, which is sufficient on all affected parts.
> 
> Enable the existing mitigation by adding the relevant MIDRs to
> erratum_spec_ssbs_list. The list is sorted alphanumerically (involving
> moving Neoverse-V3 after Neoverse-V2) so that this is easy to audit and
> potentially extend again in future. The Kconfig text is also updated to
> clarify the set of affected parts and the mitigation.
> 
> Signed-off-by: Mark Rutland <mark.rutland@arm.com>
> Cc: James Morse <james.morse@arm.com>
> Cc: Will Deacon <will@kernel.org>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> ---
>  Documentation/arch/arm64/silicon-errata.rst | 18 +++++++++++++++++
>  arch/arm64/Kconfig                          | 22 +++++++++++++++------
>  arch/arm64/kernel/cpu_errata.c              | 11 ++++++++++-
>  3 files changed, 44 insertions(+), 7 deletions(-)
> 
> diff --git a/Documentation/arch/arm64/silicon-errata.rst b/Documentation/arch/arm64/silicon-errata.rst
> index bb83c5d8c6755..50327c05be8d1 100644
> --- a/Documentation/arch/arm64/silicon-errata.rst
> +++ b/Documentation/arch/arm64/silicon-errata.rst
> @@ -122,10 +122,18 @@ stable kernels.
>  +----------------+-----------------+-----------------+-----------------------------+
>  | ARM            | Cortex-A76      | #1490853        | N/A                         |
>  +----------------+-----------------+-----------------+-----------------------------+
> +| ARM            | Cortex-A76      | #3324349        | ARM64_ERRATUM_3194386       |
> ++----------------+-----------------+-----------------+-----------------------------+
>  | ARM            | Cortex-A77      | #1491015        | N/A                         |
>  +----------------+-----------------+-----------------+-----------------------------+
>  | ARM            | Cortex-A77      | #1508412        | ARM64_ERRATUM_1508412       |
>  +----------------+-----------------+-----------------+-----------------------------+
> +| ARM            | Cortex-A77      | #3324348        | ARM64_ERRATUM_3194386       |
> ++----------------+-----------------+-----------------+-----------------------------+
> +| ARM            | Cortex-A78      | #3324344        | ARM64_ERRATUM_3194386       |
> ++----------------+-----------------+-----------------+-----------------------------+
> +| ARM            | Cortex-A78C     | #3324346,3324347| ARM64_ERRATUM_3194386       |
> ++----------------+-----------------+-----------------+-----------------------------+
>  | ARM            | Cortex-A710     | #2119858        | ARM64_ERRATUM_2119858       |
>  +----------------+-----------------+-----------------+-----------------------------+
>  | ARM            | Cortex-A710     | #2054223        | ARM64_ERRATUM_2054223       |
> @@ -138,8 +146,14 @@ stable kernels.
>  +----------------+-----------------+-----------------+-----------------------------+
>  | ARM            | Cortex-A720     | #3456091        | ARM64_ERRATUM_3194386       |
>  +----------------+-----------------+-----------------+-----------------------------+
> +| ARM            | Cortex-A725     | #3456106        | ARM64_ERRATUM_3194386       |
> ++----------------+-----------------+-----------------+-----------------------------+
>  | ARM            | Cortex-X1       | #1502854        | N/A                         |
>  +----------------+-----------------+-----------------+-----------------------------+
> +| ARM            | Cortex-X1       | #3324344        | ARM64_ERRATUM_3194386       |
> ++----------------+-----------------+-----------------+-----------------------------+
> +| ARM            | Cortex-X1C      | #3324346        | ARM64_ERRATUM_3194386       |
> ++----------------+-----------------+-----------------+-----------------------------+
>  | ARM            | Cortex-X2       | #2119858        | ARM64_ERRATUM_2119858       |
>  +----------------+-----------------+-----------------+-----------------------------+
>  | ARM            | Cortex-X2       | #2224489        | ARM64_ERRATUM_2224489       |
> @@ -160,6 +174,8 @@ stable kernels.
>  +----------------+-----------------+-----------------+-----------------------------+
>  | ARM            | Neoverse-N1     | #1542419        | ARM64_ERRATUM_1542419       |
>  +----------------+-----------------+-----------------+-----------------------------+
> +| ARM            | Neoverse-N1     | #3324349        | ARM64_ERRATUM_3194386       |
> ++----------------+-----------------+-----------------+-----------------------------+
>  | ARM            | Neoverse-N2     | #2139208        | ARM64_ERRATUM_2139208       |
>  +----------------+-----------------+-----------------+-----------------------------+
>  | ARM            | Neoverse-N2     | #2067961        | ARM64_ERRATUM_2067961       |
> @@ -170,6 +186,8 @@ stable kernels.
>  +----------------+-----------------+-----------------+-----------------------------+
>  | ARM            | Neoverse-V1     | #1619801        | N/A                         |
>  +----------------+-----------------+-----------------+-----------------------------+
> +| ARM            | Neoverse-V1     | #3324341        | ARM64_ERRATUM_3194386       |
> ++----------------+-----------------+-----------------+-----------------------------+
>  | ARM            | Neoverse-V2     | #3324336        | ARM64_ERRATUM_3194386       |
>  +----------------+-----------------+-----------------+-----------------------------+
>  | ARM            | Neoverse-V3     | #3312417        | ARM64_ERRATUM_3194386       |
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index b3fc891f15442..a2f8ff354ca67 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -1069,18 +1069,28 @@ config ARM64_ERRATUM_3117295
>  	  If unsure, say Y.
>  
>  config ARM64_ERRATUM_3194386
> -	bool "Cortex-{A720,X4,X925}/Neoverse-V3: workaround for MSR SSBS not self-synchronizing"
> +	bool "Cortex-*/Neoverse-*: workaround for MSR SSBS not self-synchronizing"
>  	default y
>  	help
>  	  This option adds the workaround for the following errata:
>  
> +	  * ARM Cortex-A76 erratum 3324349
> +	  * ARM Cortex-A77 erratum 3324348
> +	  * ARM Cortex-A78 erratum 3324344
> +	  * ARM Cortex-A78C erratum 3324346
> +	  * ARM Cortex-A78C erratum 3324347
>  	  * ARM Cortex-A710 erratam 3324338
>  	  * ARM Cortex-A720 erratum 3456091
> +	  * ARM Cortex-A725 erratum 3456106
> +	  * ARM Cortex-X1 erratum 3324344
> +	  * ARM Cortex-X1C erratum 3324346
>  	  * ARM Cortex-X2 erratum 3324338
>  	  * ARM Cortex-X3 erratum 3324335
>  	  * ARM Cortex-X4 erratum 3194386
>  	  * ARM Cortex-X925 erratum 3324334
> +	  * ARM Neoverse-N1 erratum 3324349
>  	  * ARM Neoverse N2 erratum 3324339
> +	  * ARM Neoverse-V1 erratum 3324341
>  	  * ARM Neoverse V2 erratum 3324336
>  	  * ARM Neoverse-V3 erratum 3312417
>  
> @@ -1088,11 +1098,11 @@ config ARM64_ERRATUM_3194386
>  	  subsequent speculative instructions, which may permit unexepected
>  	  speculative store bypassing.
>  
> -	  Work around this problem by placing a speculation barrier after
> -	  kernel changes to SSBS. The presence of the SSBS special-purpose
> -	  register is hidden from hwcaps and EL0 reads of ID_AA64PFR1_EL1, such
> -	  that userspace will use the PR_SPEC_STORE_BYPASS prctl to change
> -	  SSBS.
> +	  Work around this problem by placing a Speculation Barrier (SB) or
> +	  Instruction Synchronization Barrier (ISB) after kernel changes to
> +	  SSBS. The presence of the SSBS special-purpose register is hidden
> +	  from hwcaps and EL0 reads of ID_AA64PFR1_EL1, such that userspace
> +	  will use the PR_SPEC_STORE_BYPASS prctl to change SSBS.
>  
>  	  If unsure, say Y.
>  
> diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
> index 617424b73f8c3..f6b6b45073571 100644
> --- a/arch/arm64/kernel/cpu_errata.c
> +++ b/arch/arm64/kernel/cpu_errata.c
> @@ -434,15 +434,24 @@ static const struct midr_range erratum_spec_unpriv_load_list[] = {
>  
>  #ifdef CONFIG_ARM64_ERRATUM_3194386
>  static const struct midr_range erratum_spec_ssbs_list[] = {
> +	MIDR_ALL_VERSIONS(MIDR_CORTEX_A76),
> +	MIDR_ALL_VERSIONS(MIDR_CORTEX_A77),
> +	MIDR_ALL_VERSIONS(MIDR_CORTEX_A78),
> +	MIDR_ALL_VERSIONS(MIDR_CORTEX_A78C),
>  	MIDR_ALL_VERSIONS(MIDR_CORTEX_A710),
>  	MIDR_ALL_VERSIONS(MIDR_CORTEX_A720),
> +	MIDR_ALL_VERSIONS(MIDR_CORTEX_A725),
> +	MIDR_ALL_VERSIONS(MIDR_CORTEX_X1),
> +	MIDR_ALL_VERSIONS(MIDR_CORTEX_X1C),
>  	MIDR_ALL_VERSIONS(MIDR_CORTEX_X2),
>  	MIDR_ALL_VERSIONS(MIDR_CORTEX_X3),
>  	MIDR_ALL_VERSIONS(MIDR_CORTEX_X4),
>  	MIDR_ALL_VERSIONS(MIDR_CORTEX_X925),
> +	MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N1),
>  	MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N2),
> -	MIDR_ALL_VERSIONS(MIDR_NEOVERSE_V3),
> +	MIDR_ALL_VERSIONS(MIDR_NEOVERSE_V1),
>  	MIDR_ALL_VERSIONS(MIDR_NEOVERSE_V2),
> +	MIDR_ALL_VERSIONS(MIDR_NEOVERSE_V3),
>  	{}
>  };
>  #endif

Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>