[PATCH v4 00/36] Attack vector controls

[PATCH v4 00/36] Attack vector controls

[David Kaplan]

This series restructures arch/x86/kernel/cpu/bugs.c and proposes new
command line options to make it easier to control which CPU mitigations
are applied.  These options select relevant mitigations based on chosen
attack vectors, which are hopefully easier for users to understand.

There are two parts to this patch series:

The first 16 patches restructure the existing mitigation selection logic
to use a uniform set of functions.  First, the "select" function is
called for each mitigation to select an appropriate mitigation.  Unless
a mitigation is explicitly selected or disabled with a command line
option, the default mitigation is AUTO and the "select" function will
then choose the best mitigation.  After the "select" function is called
for each mitigation, some mitigations define an "update" function which
can be used to update the selection, based on the choices made by other
mitigations.  Finally, the "apply" function is called which enables the
chosen mitigation.

This structure simplifies the mitigation control logic, especially when
there are dependencies between multiple vulnerabilities.  It also
prepares the code for the second set of patches.

The rest of the patches define new "attack vector" options
to make it easier to select appropriate mitigations based on the usage
of the system.  While many users may not be intimately familiar with the
details of these CPU vulnerabilities, they are likely better able to
understand the intended usage of their system.  As a result, unneeded
mitigations may be disabled, allowing users to recoup more performance.
New documentation is included with recommendations on what to consider
when choosing which attack vectors to enable/disable.

In this series, attack vector options are chosen using the mitigations=
command line.  Attack vectors may be individually disabled such as
'mitigations=auto;no_user_kernel,no_user_user'.  The 'mitigations=off'
option is equivalent to disabling all attack vectors.  'mitigations=off'
therefore disables all mitigations, unless bug-specific command line
options are used to re-enable some.

Note that this patch series does not change any of the existing
mitigation defaults.

Changes in v4:
   - New command line interface for attack vector selection
   - Rework of smt mitigations into on/auto/off
   - Print mitigated attack vectors in dmesg and sysfs
   - Various bug fixes and clean up

Changes in v3:
   - Moved command line options to be x86-only
   - Fix bugs related to ucode detection for taa/mmio/rfds
   - Various clean up

Changes in v2:
   - Removed new enum, just use X86_BUG* to identify vulnerabilities
   - Mitigate gds if cross-thread protection is selected as pointed out
     by Andrew Cooper
   - Simplifications around verw-based mitigation handling
   - Various bug fixes

David Kaplan (36):
  x86/bugs: Restructure mds mitigation
  x86/bugs: Restructure taa mitigation
  x86/bugs: Restructure mmio mitigation
  x86/bugs: Restructure rfds mitigation
  x86/bugs: Remove md_clear_*_mitigation()
  x86/bugs: Restructure srbds mitigation
  x86/bugs: Restructure gds mitigation
  x86/bugs: Restructure spectre_v1 mitigation
  x86/bugs: Only allow retbleed=stuff on Intel
  x86/bugs: Restructure retbleed mitigation
  x86/bugs: Restructure spectre_v2_user mitigation
  x86/bugs: Restructure bhi mitigation
  x86/bugs: Restructure spectre_v2 mitigation
  x86/bugs: Restructure ssb mitigation
  x86/bugs: Restructure l1tf mitigation
  x86/bugs: Restructure srso mitigation
  Documentation/x86: Document the new attack vector controls
  cpu: Define attack vectors
  x86/Kconfig: Arch attack vector support
  x86/bugs: Determine relevant vulnerabilities based on attack vector
    controls.
  x86/bugs: Add attack vector controls for mds
  x86/bugs: Add attack vector controls for taa
  x86/bugs: Add attack vector controls for mmio
  x86/bugs: Add attack vector controls for rfds
  x86/bugs: Add attack vector controls for srbds
  x86/bugs: Add attack vector controls for gds
  x86/bugs: Add attack vector controls for spectre_v1
  x86/bugs: Add attack vector controls for retbleed
  x86/bugs: Add attack vector controls for spectre_v2_user
  x86/bugs: Add attack vector controls for bhi
  x86/bugs: Add attack vector controls for spectre_v2
  x86/bugs: Add attack vector controls for l1tf
  x86/bugs: Add attack vector controls for srso
  x86/pti: Add attack vector controls for pti
  x86/bugs: Print enabled attack vectors
  cpu: Show attack vectors in sysfs

 .../hw-vuln/attack_vector_controls.rst        |  236 +++
 Documentation/admin-guide/hw-vuln/index.rst   |    1 +
 arch/Kconfig                                  |    3 +
 arch/x86/Kconfig                              |    1 +
 arch/x86/include/asm/processor.h              |    1 +
 arch/x86/kernel/cpu/bugs.c                    | 1302 ++++++++++-------
 arch/x86/kvm/vmx/vmx.c                        |    2 +
 arch/x86/mm/pti.c                             |    4 +-
 drivers/base/cpu.c                            |   67 +
 include/linux/cpu.h                           |   20 +
 kernel/cpu.c                                  |  129 +-
 11 files changed, 1248 insertions(+), 518 deletions(-)
 create mode 100644 Documentation/admin-guide/hw-vuln/attack_vector_controls.rst

-- 
2.34.1

[PATCH v4 01/36] x86/bugs: Restructure mds mitigation

[David Kaplan]

Restructure mds mitigation selection to use select/update/apply
functions to create consistent vulnerability handling.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 55 ++++++++++++++++++++++++++++++++++++--
 1 file changed, 53 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 4386aa6c69e1..71da57c4f83b 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -34,6 +34,25 @@
 
 #include "cpu.h"
 
+/*
+ * Speculation Vulnerability Handling
+ *
+ * Each vulnerability is handled with the following functions:
+ *   <vuln>_select_mitigation() -- Selects a mitigation to use.  This should
+ *				   take into account all relevant command line
+ *				   options.
+ *   <vuln>_update_mitigation() -- This is called after all vulnerabilities have
+ *				   selected a mitigation, in case the selection
+ *				   may want to change based on other choices
+ *				   made.  This function is optional.
+ *   <vuln>_apply_mitigation() -- Enable the selected mitigation.
+ *
+ * The compile-time mitigation in all cases should be AUTO.  An explicit
+ * command-line option can override AUTO.  If no such option is
+ * provided, <vuln>_select_mitigation() will override AUTO to the best
+ * mitigation option.
+ */
+
 static void __init spectre_v1_select_mitigation(void);
 static void __init spectre_v2_select_mitigation(void);
 static void __init retbleed_select_mitigation(void);
@@ -41,6 +60,8 @@ static void __init spectre_v2_user_select_mitigation(void);
 static void __init ssb_select_mitigation(void);
 static void __init l1tf_select_mitigation(void);
 static void __init mds_select_mitigation(void);
+static void __init mds_update_mitigation(void);
+static void __init mds_apply_mitigation(void);
 static void __init md_clear_update_mitigation(void);
 static void __init md_clear_select_mitigation(void);
 static void __init taa_select_mitigation(void);
@@ -169,6 +190,7 @@ void __init cpu_select_mitigations(void)
 	spectre_v2_user_select_mitigation();
 	ssb_select_mitigation();
 	l1tf_select_mitigation();
+	mds_select_mitigation();
 	md_clear_select_mitigation();
 	srbds_select_mitigation();
 	l1d_flush_select_mitigation();
@@ -179,6 +201,14 @@ void __init cpu_select_mitigations(void)
 	 */
 	srso_select_mitigation();
 	gds_select_mitigation();
+
+	/*
+	 * After mitigations are selected, some may need to update their
+	 * choices.
+	 */
+	mds_update_mitigation();
+
+	mds_apply_mitigation();
 }
 
 /*
@@ -281,6 +311,9 @@ enum rfds_mitigations {
 static enum rfds_mitigations rfds_mitigation __ro_after_init =
 	IS_ENABLED(CONFIG_MITIGATION_RFDS) ? RFDS_MITIGATION_AUTO : RFDS_MITIGATION_OFF;
 
+/* Set if any of MDS/TAA/MMIO/RFDS are going to enable VERW. */
+static bool verw_mitigation_selected __ro_after_init;
+
 static void __init mds_select_mitigation(void)
 {
 	if (!boot_cpu_has_bug(X86_BUG_MDS) || cpu_mitigations_off()) {
@@ -291,12 +324,31 @@ static void __init mds_select_mitigation(void)
 	if (mds_mitigation == MDS_MITIGATION_AUTO)
 		mds_mitigation = MDS_MITIGATION_FULL;
 
+	verw_mitigation_selected = true;
+}
+
+static void __init mds_update_mitigation(void)
+{
+	if (!boot_cpu_has_bug(X86_BUG_MDS) || cpu_mitigations_off())
+		return;
+
+	/* If TAA, MMIO, or RFDS are being mitigated, MDS gets mitigated too. */
+	if (verw_mitigation_selected)
+		mds_mitigation = MDS_MITIGATION_FULL;
+
 	if (mds_mitigation == MDS_MITIGATION_FULL) {
 		if (!boot_cpu_has(X86_FEATURE_MD_CLEAR))
 			mds_mitigation = MDS_MITIGATION_VMWERV;
+	}
 
-		setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF);
+	pr_info("%s\n", mds_strings[mds_mitigation]);
+}
 
+static void __init mds_apply_mitigation(void)
+{
+	if (mds_mitigation == MDS_MITIGATION_FULL ||
+	    mds_mitigation == MDS_MITIGATION_VMWERV) {
+		setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF);
 		if (!boot_cpu_has(X86_BUG_MSBDS_ONLY) &&
 		    (mds_nosmt || cpu_mitigations_auto_nosmt()))
 			cpu_smt_disable(false);
@@ -599,7 +651,6 @@ static void __init md_clear_update_mitigation(void)
 
 static void __init md_clear_select_mitigation(void)
 {
-	mds_select_mitigation();
 	taa_select_mitigation();
 	mmio_select_mitigation();
 	rfds_select_mitigation();
-- 
2.34.1

[PATCH v4 02/36] x86/bugs: Restructure taa mitigation

[David Kaplan]

Restructure taa mitigation to use select/update/apply functions to
create consistent vulnerability handling.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 94 ++++++++++++++++++++++++--------------
 1 file changed, 59 insertions(+), 35 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 71da57c4f83b..2fd58b7089c4 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -65,6 +65,8 @@ static void __init mds_apply_mitigation(void);
 static void __init md_clear_update_mitigation(void);
 static void __init md_clear_select_mitigation(void);
 static void __init taa_select_mitigation(void);
+static void __init taa_update_mitigation(void);
+static void __init taa_apply_mitigation(void);
 static void __init mmio_select_mitigation(void);
 static void __init srbds_select_mitigation(void);
 static void __init l1d_flush_select_mitigation(void);
@@ -191,6 +193,7 @@ void __init cpu_select_mitigations(void)
 	ssb_select_mitigation();
 	l1tf_select_mitigation();
 	mds_select_mitigation();
+	taa_select_mitigation();
 	md_clear_select_mitigation();
 	srbds_select_mitigation();
 	l1d_flush_select_mitigation();
@@ -207,8 +210,10 @@ void __init cpu_select_mitigations(void)
 	 * choices.
 	 */
 	mds_update_mitigation();
+	taa_update_mitigation();
 
 	mds_apply_mitigation();
+	taa_apply_mitigation();
 }
 
 /*
@@ -388,6 +393,11 @@ static const char * const taa_strings[] = {
 	[TAA_MITIGATION_TSX_DISABLED]	= "Mitigation: TSX disabled",
 };
 
+static bool __init taa_vulnerable(void)
+{
+	return boot_cpu_has_bug(X86_BUG_TAA) && boot_cpu_has(X86_FEATURE_RTM);
+}
+
 static void __init taa_select_mitigation(void)
 {
 	if (!boot_cpu_has_bug(X86_BUG_TAA)) {
@@ -401,48 +411,63 @@ static void __init taa_select_mitigation(void)
 		return;
 	}
 
-	if (cpu_mitigations_off()) {
+	if (cpu_mitigations_off())
 		taa_mitigation = TAA_MITIGATION_OFF;
-		return;
-	}
 
-	/*
-	 * TAA mitigation via VERW is turned off if both
-	 * tsx_async_abort=off and mds=off are specified.
-	 */
-	if (taa_mitigation == TAA_MITIGATION_OFF &&
-	    mds_mitigation == MDS_MITIGATION_OFF)
+	/* Microcode will be checked in taa_update_mitigation(). */
+	if (taa_mitigation == TAA_MITIGATION_AUTO)
+		taa_mitigation = TAA_MITIGATION_VERW;
+
+	if (taa_mitigation != TAA_MITIGATION_OFF)
+		verw_mitigation_selected = true;
+}
+
+static void __init taa_update_mitigation(void)
+{
+	if (!taa_vulnerable() || cpu_mitigations_off())
 		return;
 
-	if (boot_cpu_has(X86_FEATURE_MD_CLEAR))
+	if (verw_mitigation_selected)
 		taa_mitigation = TAA_MITIGATION_VERW;
-	else
-		taa_mitigation = TAA_MITIGATION_UCODE_NEEDED;
 
-	/*
-	 * VERW doesn't clear the CPU buffers when MD_CLEAR=1 and MDS_NO=1.
-	 * A microcode update fixes this behavior to clear CPU buffers. It also
-	 * adds support for MSR_IA32_TSX_CTRL which is enumerated by the
-	 * ARCH_CAP_TSX_CTRL_MSR bit.
-	 *
-	 * On MDS_NO=1 CPUs if ARCH_CAP_TSX_CTRL_MSR is not set, microcode
-	 * update is required.
-	 */
-	if ( (x86_arch_cap_msr & ARCH_CAP_MDS_NO) &&
-	    !(x86_arch_cap_msr & ARCH_CAP_TSX_CTRL_MSR))
-		taa_mitigation = TAA_MITIGATION_UCODE_NEEDED;
+	if (taa_mitigation == TAA_MITIGATION_VERW) {
+		/* Check if the requisite ucode is available. */
+		if (!boot_cpu_has(X86_FEATURE_MD_CLEAR))
+			taa_mitigation = TAA_MITIGATION_UCODE_NEEDED;
 
-	/*
-	 * TSX is enabled, select alternate mitigation for TAA which is
-	 * the same as MDS. Enable MDS static branch to clear CPU buffers.
-	 *
-	 * For guests that can't determine whether the correct microcode is
-	 * present on host, enable the mitigation for UCODE_NEEDED as well.
-	 */
-	setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF);
+		/*
+		 * VERW doesn't clear the CPU buffers when MD_CLEAR=1 and MDS_NO=1.
+		 * A microcode update fixes this behavior to clear CPU buffers. It also
+		 * adds support for MSR_IA32_TSX_CTRL which is enumerated by the
+		 * ARCH_CAP_TSX_CTRL_MSR bit.
+		 *
+		 * On MDS_NO=1 CPUs if ARCH_CAP_TSX_CTRL_MSR is not set, microcode
+		 * update is required.
+		 */
+		if ((x86_arch_cap_msr & ARCH_CAP_MDS_NO) &&
+		   !(x86_arch_cap_msr & ARCH_CAP_TSX_CTRL_MSR))
+			taa_mitigation = TAA_MITIGATION_UCODE_NEEDED;
+	}
 
-	if (taa_nosmt || cpu_mitigations_auto_nosmt())
-		cpu_smt_disable(false);
+	pr_info("%s\n", taa_strings[taa_mitigation]);
+}
+
+static void __init taa_apply_mitigation(void)
+{
+	if (taa_mitigation == TAA_MITIGATION_VERW ||
+	    taa_mitigation == TAA_MITIGATION_UCODE_NEEDED) {
+		/*
+		 * TSX is enabled, select alternate mitigation for TAA which is
+		 * the same as MDS. Enable MDS static branch to clear CPU buffers.
+		 *
+		 * For guests that can't determine whether the correct microcode is
+		 * present on host, enable the mitigation for UCODE_NEEDED as well.
+		 */
+		setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF);
+
+		if (taa_nosmt || cpu_mitigations_auto_nosmt())
+			cpu_smt_disable(false);
+	}
 }
 
 static int __init tsx_async_abort_parse_cmdline(char *str)
@@ -651,7 +676,6 @@ static void __init md_clear_update_mitigation(void)
 
 static void __init md_clear_select_mitigation(void)
 {
-	taa_select_mitigation();
 	mmio_select_mitigation();
 	rfds_select_mitigation();
 
-- 
2.34.1

[PATCH v4 03/36] x86/bugs: Restructure mmio mitigation

[David Kaplan]

Restructure mmio mitigation to use select/update/apply functions to
create consistent vulnerability handling.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 77 +++++++++++++++++++++++++-------------
 1 file changed, 51 insertions(+), 26 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 2fd58b7089c4..a727f7998bec 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -68,6 +68,8 @@ static void __init taa_select_mitigation(void);
 static void __init taa_update_mitigation(void);
 static void __init taa_apply_mitigation(void);
 static void __init mmio_select_mitigation(void);
+static void __init mmio_update_mitigation(void);
+static void __init mmio_apply_mitigation(void);
 static void __init srbds_select_mitigation(void);
 static void __init l1d_flush_select_mitigation(void);
 static void __init srso_select_mitigation(void);
@@ -194,6 +196,7 @@ void __init cpu_select_mitigations(void)
 	l1tf_select_mitigation();
 	mds_select_mitigation();
 	taa_select_mitigation();
+	mmio_select_mitigation();
 	md_clear_select_mitigation();
 	srbds_select_mitigation();
 	l1d_flush_select_mitigation();
@@ -211,9 +214,11 @@ void __init cpu_select_mitigations(void)
 	 */
 	mds_update_mitigation();
 	taa_update_mitigation();
+	mmio_update_mitigation();
 
 	mds_apply_mitigation();
 	taa_apply_mitigation();
+	mmio_apply_mitigation();
 }
 
 /*
@@ -511,24 +516,60 @@ static void __init mmio_select_mitigation(void)
 		return;
 	}
 
-	if (mmio_mitigation == MMIO_MITIGATION_OFF)
-		return;
+	/* Microcode will be checked in mmio_update_mitigation(). */
+	if (mmio_mitigation == MMIO_MITIGATION_AUTO)
+		mmio_mitigation = MMIO_MITIGATION_VERW;
 
 	/*
 	 * Enable CPU buffer clear mitigation for host and VMM, if also affected
-	 * by MDS or TAA. Otherwise, enable mitigation for VMM only.
+	 * by MDS or TAA.
 	 */
-	if (boot_cpu_has_bug(X86_BUG_MDS) || (boot_cpu_has_bug(X86_BUG_TAA) &&
-					      boot_cpu_has(X86_FEATURE_RTM)))
-		setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF);
+	if (boot_cpu_has_bug(X86_BUG_MDS) || taa_vulnerable())
+		verw_mitigation_selected = true;
+}
+
+static void __init mmio_update_mitigation(void)
+{
+	if (!boot_cpu_has_bug(X86_BUG_MMIO_STALE_DATA) || cpu_mitigations_off())
+		return;
+
+	if (verw_mitigation_selected)
+		mmio_mitigation = MMIO_MITIGATION_VERW;
+
+	if (mmio_mitigation == MMIO_MITIGATION_VERW) {
+		/*
+		 * Check if the system has the right microcode.
+		 *
+		 * CPU Fill buffer clear mitigation is enumerated by either an explicit
+		 * FB_CLEAR or by the presence of both MD_CLEAR and L1D_FLUSH on MDS
+		 * affected systems.
+		 */
+		if (!((x86_arch_cap_msr & ARCH_CAP_FB_CLEAR) ||
+		      (boot_cpu_has(X86_FEATURE_MD_CLEAR) &&
+		       boot_cpu_has(X86_FEATURE_FLUSH_L1D) &&
+		     !(x86_arch_cap_msr & ARCH_CAP_MDS_NO))))
+			mmio_mitigation = MMIO_MITIGATION_UCODE_NEEDED;
+	}
+
+	if (boot_cpu_has_bug(X86_BUG_MMIO_UNKNOWN))
+		pr_info("Unknown: No mitigations\n");
+	else
+		pr_info("%s\n", mmio_strings[mmio_mitigation]);
+}
+
+static void __init mmio_apply_mitigation(void)
+{
+	if (mmio_mitigation == MMIO_MITIGATION_OFF)
+		return;
 
 	/*
-	 * X86_FEATURE_CLEAR_CPU_BUF could be enabled by other VERW based
-	 * mitigations, disable KVM-only mitigation in that case.
+	 * Only enable the VMM mitigation if the CPU buffer clear mitigation is
+	 * not being used.
 	 */
-	if (boot_cpu_has(X86_FEATURE_CLEAR_CPU_BUF))
+	if (verw_mitigation_selected) {
+		setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF);
 		static_branch_disable(&mmio_stale_data_clear);
-	else
+	} else
 		static_branch_enable(&mmio_stale_data_clear);
 
 	/*
@@ -539,21 +580,6 @@ static void __init mmio_select_mitigation(void)
 	if (!(x86_arch_cap_msr & ARCH_CAP_FBSDP_NO))
 		static_branch_enable(&mds_idle_clear);
 
-	/*
-	 * Check if the system has the right microcode.
-	 *
-	 * CPU Fill buffer clear mitigation is enumerated by either an explicit
-	 * FB_CLEAR or by the presence of both MD_CLEAR and L1D_FLUSH on MDS
-	 * affected systems.
-	 */
-	if ((x86_arch_cap_msr & ARCH_CAP_FB_CLEAR) ||
-	    (boot_cpu_has(X86_FEATURE_MD_CLEAR) &&
-	     boot_cpu_has(X86_FEATURE_FLUSH_L1D) &&
-	     !(x86_arch_cap_msr & ARCH_CAP_MDS_NO)))
-		mmio_mitigation = MMIO_MITIGATION_VERW;
-	else
-		mmio_mitigation = MMIO_MITIGATION_UCODE_NEEDED;
-
 	if (mmio_nosmt || cpu_mitigations_auto_nosmt())
 		cpu_smt_disable(false);
 }
@@ -676,7 +702,6 @@ static void __init md_clear_update_mitigation(void)
 
 static void __init md_clear_select_mitigation(void)
 {
-	mmio_select_mitigation();
 	rfds_select_mitigation();
 
 	/*
-- 
2.34.1

[PATCH v4 04/36] x86/bugs: Restructure rfds mitigation

[David Kaplan]

Restructure rfds mitigation to use select/update/apply functions to
create consistent vulnerability handling.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 40 ++++++++++++++++++++++++++++++++------
 1 file changed, 34 insertions(+), 6 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index a727f7998bec..de431f2bb012 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -70,6 +70,9 @@ static void __init taa_apply_mitigation(void);
 static void __init mmio_select_mitigation(void);
 static void __init mmio_update_mitigation(void);
 static void __init mmio_apply_mitigation(void);
+static void __init rfds_select_mitigation(void);
+static void __init rfds_update_mitigation(void);
+static void __init rfds_apply_mitigation(void);
 static void __init srbds_select_mitigation(void);
 static void __init l1d_flush_select_mitigation(void);
 static void __init srso_select_mitigation(void);
@@ -197,6 +200,7 @@ void __init cpu_select_mitigations(void)
 	mds_select_mitigation();
 	taa_select_mitigation();
 	mmio_select_mitigation();
+	rfds_select_mitigation();
 	md_clear_select_mitigation();
 	srbds_select_mitigation();
 	l1d_flush_select_mitigation();
@@ -215,10 +219,12 @@ void __init cpu_select_mitigations(void)
 	mds_update_mitigation();
 	taa_update_mitigation();
 	mmio_update_mitigation();
+	rfds_update_mitigation();
 
 	mds_apply_mitigation();
 	taa_apply_mitigation();
 	mmio_apply_mitigation();
+	rfds_apply_mitigation();
 }
 
 /*
@@ -614,22 +620,45 @@ static const char * const rfds_strings[] = {
 	[RFDS_MITIGATION_UCODE_NEEDED]		= "Vulnerable: No microcode",
 };
 
+static bool __init rfds_has_ucode(void)
+{
+	return (x86_arch_cap_msr & ARCH_CAP_RFDS_CLEAR);
+}
+
 static void __init rfds_select_mitigation(void)
 {
 	if (!boot_cpu_has_bug(X86_BUG_RFDS) || cpu_mitigations_off()) {
 		rfds_mitigation = RFDS_MITIGATION_OFF;
 		return;
 	}
-	if (rfds_mitigation == RFDS_MITIGATION_OFF)
-		return;
 
 	if (rfds_mitigation == RFDS_MITIGATION_AUTO)
 		rfds_mitigation = RFDS_MITIGATION_VERW;
 
-	if (x86_arch_cap_msr & ARCH_CAP_RFDS_CLEAR)
+	if (rfds_has_ucode())
+		verw_mitigation_selected = true;
+}
+
+static void __init rfds_update_mitigation(void)
+{
+	if (!boot_cpu_has_bug(X86_BUG_RFDS) || cpu_mitigations_off())
+		return;
+
+	if (verw_mitigation_selected)
+		rfds_mitigation = RFDS_MITIGATION_VERW;
+
+	if (rfds_mitigation == RFDS_MITIGATION_VERW) {
+		if (!rfds_has_ucode())
+			rfds_mitigation = RFDS_MITIGATION_UCODE_NEEDED;
+	}
+
+	pr_info("%s\n", rfds_strings[rfds_mitigation]);
+}
+
+static void __init rfds_apply_mitigation(void)
+{
+	if (rfds_mitigation == RFDS_MITIGATION_VERW)
 		setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF);
-	else
-		rfds_mitigation = RFDS_MITIGATION_UCODE_NEEDED;
 }
 
 static __init int rfds_parse_cmdline(char *str)
@@ -702,7 +731,6 @@ static void __init md_clear_update_mitigation(void)
 
 static void __init md_clear_select_mitigation(void)
 {
-	rfds_select_mitigation();
 
 	/*
 	 * As these mitigations are inter-related and rely on VERW instruction
-- 
2.34.1

[PATCH v4 05/36] x86/bugs: Remove md_clear_*_mitigation()

[David Kaplan]

The functionality in md_clear_update_mitigation() and
md_clear_select_mitigation() is now integrated into the select/update
functions for the MDS, TAA, MMIO, and RFDS vulnerabilities.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 65 --------------------------------------
 1 file changed, 65 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index de431f2bb012..841ab123a180 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -62,8 +62,6 @@ static void __init l1tf_select_mitigation(void);
 static void __init mds_select_mitigation(void);
 static void __init mds_update_mitigation(void);
 static void __init mds_apply_mitigation(void);
-static void __init md_clear_update_mitigation(void);
-static void __init md_clear_select_mitigation(void);
 static void __init taa_select_mitigation(void);
 static void __init taa_update_mitigation(void);
 static void __init taa_apply_mitigation(void);
@@ -201,7 +199,6 @@ void __init cpu_select_mitigations(void)
 	taa_select_mitigation();
 	mmio_select_mitigation();
 	rfds_select_mitigation();
-	md_clear_select_mitigation();
 	srbds_select_mitigation();
 	l1d_flush_select_mitigation();
 
@@ -678,68 +675,6 @@ static __init int rfds_parse_cmdline(char *str)
 }
 early_param("reg_file_data_sampling", rfds_parse_cmdline);
 
-#undef pr_fmt
-#define pr_fmt(fmt)     "" fmt
-
-static void __init md_clear_update_mitigation(void)
-{
-	if (cpu_mitigations_off())
-		return;
-
-	if (!boot_cpu_has(X86_FEATURE_CLEAR_CPU_BUF))
-		goto out;
-
-	/*
-	 * X86_FEATURE_CLEAR_CPU_BUF is now enabled. Update MDS, TAA and MMIO
-	 * Stale Data mitigation, if necessary.
-	 */
-	if (mds_mitigation == MDS_MITIGATION_OFF &&
-	    boot_cpu_has_bug(X86_BUG_MDS)) {
-		mds_mitigation = MDS_MITIGATION_FULL;
-		mds_select_mitigation();
-	}
-	if (taa_mitigation == TAA_MITIGATION_OFF &&
-	    boot_cpu_has_bug(X86_BUG_TAA)) {
-		taa_mitigation = TAA_MITIGATION_VERW;
-		taa_select_mitigation();
-	}
-	/*
-	 * MMIO_MITIGATION_OFF is not checked here so that mmio_stale_data_clear
-	 * gets updated correctly as per X86_FEATURE_CLEAR_CPU_BUF state.
-	 */
-	if (boot_cpu_has_bug(X86_BUG_MMIO_STALE_DATA)) {
-		mmio_mitigation = MMIO_MITIGATION_VERW;
-		mmio_select_mitigation();
-	}
-	if (rfds_mitigation == RFDS_MITIGATION_OFF &&
-	    boot_cpu_has_bug(X86_BUG_RFDS)) {
-		rfds_mitigation = RFDS_MITIGATION_VERW;
-		rfds_select_mitigation();
-	}
-out:
-	if (boot_cpu_has_bug(X86_BUG_MDS))
-		pr_info("MDS: %s\n", mds_strings[mds_mitigation]);
-	if (boot_cpu_has_bug(X86_BUG_TAA))
-		pr_info("TAA: %s\n", taa_strings[taa_mitigation]);
-	if (boot_cpu_has_bug(X86_BUG_MMIO_STALE_DATA))
-		pr_info("MMIO Stale Data: %s\n", mmio_strings[mmio_mitigation]);
-	else if (boot_cpu_has_bug(X86_BUG_MMIO_UNKNOWN))
-		pr_info("MMIO Stale Data: Unknown: No mitigations\n");
-	if (boot_cpu_has_bug(X86_BUG_RFDS))
-		pr_info("Register File Data Sampling: %s\n", rfds_strings[rfds_mitigation]);
-}
-
-static void __init md_clear_select_mitigation(void)
-{
-
-	/*
-	 * As these mitigations are inter-related and rely on VERW instruction
-	 * to clear the microarchitural buffers, update and print their status
-	 * after mitigation selection is done for each of these vulnerabilities.
-	 */
-	md_clear_update_mitigation();
-}
-
 #undef pr_fmt
 #define pr_fmt(fmt)	"SRBDS: " fmt
 
-- 
2.34.1

[PATCH v4 06/36] x86/bugs: Restructure srbds mitigation

[David Kaplan]

Restructure srbds to use select/apply functions to create consistent
vulnerability handling.

Define new AUTO mitigation for SRBDS.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 20 ++++++++++++++++----
 1 file changed, 16 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 841ab123a180..fd04de518fd0 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -72,6 +72,7 @@ static void __init rfds_select_mitigation(void);
 static void __init rfds_update_mitigation(void);
 static void __init rfds_apply_mitigation(void);
 static void __init srbds_select_mitigation(void);
+static void __init srbds_apply_mitigation(void);
 static void __init l1d_flush_select_mitigation(void);
 static void __init srso_select_mitigation(void);
 static void __init gds_select_mitigation(void);
@@ -222,6 +223,7 @@ void __init cpu_select_mitigations(void)
 	taa_apply_mitigation();
 	mmio_apply_mitigation();
 	rfds_apply_mitigation();
+	srbds_apply_mitigation();
 }
 
 /*
@@ -680,6 +682,7 @@ early_param("reg_file_data_sampling", rfds_parse_cmdline);
 
 enum srbds_mitigations {
 	SRBDS_MITIGATION_OFF,
+	SRBDS_MITIGATION_AUTO,
 	SRBDS_MITIGATION_UCODE_NEEDED,
 	SRBDS_MITIGATION_FULL,
 	SRBDS_MITIGATION_TSX_OFF,
@@ -687,7 +690,7 @@ enum srbds_mitigations {
 };
 
 static enum srbds_mitigations srbds_mitigation __ro_after_init =
-	IS_ENABLED(CONFIG_MITIGATION_SRBDS) ? SRBDS_MITIGATION_FULL : SRBDS_MITIGATION_OFF;
+	IS_ENABLED(CONFIG_MITIGATION_SRBDS) ? SRBDS_MITIGATION_AUTO : SRBDS_MITIGATION_OFF;
 
 static const char * const srbds_strings[] = {
 	[SRBDS_MITIGATION_OFF]		= "Vulnerable",
@@ -738,8 +741,13 @@ void update_srbds_msr(void)
 
 static void __init srbds_select_mitigation(void)
 {
-	if (!boot_cpu_has_bug(X86_BUG_SRBDS))
+	if (!boot_cpu_has_bug(X86_BUG_SRBDS) || cpu_mitigations_off()) {
+		srbds_mitigation = SRBDS_MITIGATION_OFF;
 		return;
+	}
+
+	if (srbds_mitigation == SRBDS_MITIGATION_AUTO)
+		srbds_mitigation = SRBDS_MITIGATION_FULL;
 
 	/*
 	 * Check to see if this is one of the MDS_NO systems supporting TSX that
@@ -753,13 +761,17 @@ static void __init srbds_select_mitigation(void)
 		srbds_mitigation = SRBDS_MITIGATION_HYPERVISOR;
 	else if (!boot_cpu_has(X86_FEATURE_SRBDS_CTRL))
 		srbds_mitigation = SRBDS_MITIGATION_UCODE_NEEDED;
-	else if (cpu_mitigations_off() || srbds_off)
+	else if (srbds_off)
 		srbds_mitigation = SRBDS_MITIGATION_OFF;
 
-	update_srbds_msr();
 	pr_info("%s\n", srbds_strings[srbds_mitigation]);
 }
 
+static void __init srbds_apply_mitigation(void)
+{
+	update_srbds_msr();
+}
+
 static int __init srbds_parse_cmdline(char *str)
 {
 	if (!str)
-- 
2.34.1

[PATCH v4 07/36] x86/bugs: Restructure gds mitigation

[David Kaplan]

Restructure gds mitigation to use select/apply functions to create
consistent vulnerability handling.

Define new AUTO mitigation for gds.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 43 +++++++++++++++++++++++++-------------
 1 file changed, 29 insertions(+), 14 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index fd04de518fd0..ee7a8009a188 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -76,6 +76,7 @@ static void __init srbds_apply_mitigation(void);
 static void __init l1d_flush_select_mitigation(void);
 static void __init srso_select_mitigation(void);
 static void __init gds_select_mitigation(void);
+static void __init gds_apply_mitigation(void);
 
 /* The base value of the SPEC_CTRL MSR without task-specific bits set */
 u64 x86_spec_ctrl_base;
@@ -224,6 +225,7 @@ void __init cpu_select_mitigations(void)
 	mmio_apply_mitigation();
 	rfds_apply_mitigation();
 	srbds_apply_mitigation();
+	gds_apply_mitigation();
 }
 
 /*
@@ -818,6 +820,7 @@ early_param("l1d_flush", l1d_flush_parse_cmdline);
 
 enum gds_mitigations {
 	GDS_MITIGATION_OFF,
+	GDS_MITIGATION_AUTO,
 	GDS_MITIGATION_UCODE_NEEDED,
 	GDS_MITIGATION_FORCE,
 	GDS_MITIGATION_FULL,
@@ -826,7 +829,7 @@ enum gds_mitigations {
 };
 
 static enum gds_mitigations gds_mitigation __ro_after_init =
-	IS_ENABLED(CONFIG_MITIGATION_GDS) ? GDS_MITIGATION_FULL : GDS_MITIGATION_OFF;
+	IS_ENABLED(CONFIG_MITIGATION_GDS) ? GDS_MITIGATION_AUTO : GDS_MITIGATION_OFF;
 
 static const char * const gds_strings[] = {
 	[GDS_MITIGATION_OFF]		= "Vulnerable",
@@ -867,6 +870,7 @@ void update_gds_msr(void)
 	case GDS_MITIGATION_FORCE:
 	case GDS_MITIGATION_UCODE_NEEDED:
 	case GDS_MITIGATION_HYPERVISOR:
+	case GDS_MITIGATION_AUTO:
 		return;
 	}
 
@@ -890,26 +894,21 @@ static void __init gds_select_mitigation(void)
 
 	if (boot_cpu_has(X86_FEATURE_HYPERVISOR)) {
 		gds_mitigation = GDS_MITIGATION_HYPERVISOR;
-		goto out;
+		return;
 	}
 
 	if (cpu_mitigations_off())
 		gds_mitigation = GDS_MITIGATION_OFF;
 	/* Will verify below that mitigation _can_ be disabled */
 
+	if (gds_mitigation == GDS_MITIGATION_AUTO)
+		gds_mitigation = GDS_MITIGATION_FULL;
+
 	/* No microcode */
 	if (!(x86_arch_cap_msr & ARCH_CAP_GDS_CTRL)) {
-		if (gds_mitigation == GDS_MITIGATION_FORCE) {
-			/*
-			 * This only needs to be done on the boot CPU so do it
-			 * here rather than in update_gds_msr()
-			 */
-			setup_clear_cpu_cap(X86_FEATURE_AVX);
-			pr_warn("Microcode update needed! Disabling AVX as mitigation.\n");
-		} else {
+		if (gds_mitigation != GDS_MITIGATION_FORCE)
 			gds_mitigation = GDS_MITIGATION_UCODE_NEEDED;
-		}
-		goto out;
+		return;
 	}
 
 	/* Microcode has mitigation, use it */
@@ -930,9 +929,25 @@ static void __init gds_select_mitigation(void)
 		 */
 		gds_mitigation = GDS_MITIGATION_FULL_LOCKED;
 	}
+}
+
+static void __init gds_apply_mitigation(void)
+{
+	if (!boot_cpu_has_bug(X86_BUG_GDS))
+		return;
+
+	/* Microcode is present */
+	if (x86_arch_cap_msr & ARCH_CAP_GDS_CTRL)
+		update_gds_msr();
+	else if (gds_mitigation == GDS_MITIGATION_FORCE) {
+		/*
+		 * This only needs to be done on the boot CPU so do it
+		 * here rather than in update_gds_msr()
+		 */
+		setup_clear_cpu_cap(X86_FEATURE_AVX);
+		pr_warn("Microcode update needed! Disabling AVX as mitigation.\n");
+	}
 
-	update_gds_msr();
-out:
 	pr_info("%s\n", gds_strings[gds_mitigation]);
 }
 
-- 
2.34.1

[PATCH v4 08/36] x86/bugs: Restructure spectre_v1 mitigation

[David Kaplan]

Restructure spectre_v1 to use select/apply functions to create
consistent vulnerability handling.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index ee7a8009a188..de120ecb752d 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -54,6 +54,7 @@
  */
 
 static void __init spectre_v1_select_mitigation(void);
+static void __init spectre_v1_apply_mitigation(void);
 static void __init spectre_v2_select_mitigation(void);
 static void __init retbleed_select_mitigation(void);
 static void __init spectre_v2_user_select_mitigation(void);
@@ -220,6 +221,7 @@ void __init cpu_select_mitigations(void)
 	mmio_update_mitigation();
 	rfds_update_mitigation();
 
+	spectre_v1_apply_mitigation();
 	mds_apply_mitigation();
 	taa_apply_mitigation();
 	mmio_apply_mitigation();
@@ -1008,10 +1010,14 @@ static bool smap_works_speculatively(void)
 
 static void __init spectre_v1_select_mitigation(void)
 {
-	if (!boot_cpu_has_bug(X86_BUG_SPECTRE_V1) || cpu_mitigations_off()) {
+	if (!boot_cpu_has_bug(X86_BUG_SPECTRE_V1) || cpu_mitigations_off())
 		spectre_v1_mitigation = SPECTRE_V1_MITIGATION_NONE;
+}
+
+static void __init spectre_v1_apply_mitigation(void)
+{
+	if (!boot_cpu_has_bug(X86_BUG_SPECTRE_V1) || cpu_mitigations_off())
 		return;
-	}
 
 	if (spectre_v1_mitigation == SPECTRE_V1_MITIGATION_AUTO) {
 		/*
-- 
2.34.1

[PATCH v4 09/36] x86/bugs: Only allow retbleed=stuff on Intel

[David Kaplan]

The retbleed=stuff mitigation is only applicable for Intel CPUs affected
by retbleed.  If this option is selected for another vendor, print a
warning and fall back to the AUTO option.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index de120ecb752d..4af342d226c8 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -1178,6 +1178,10 @@ static void __init retbleed_select_mitigation(void)
 	case RETBLEED_CMD_STUFF:
 		if (IS_ENABLED(CONFIG_MITIGATION_CALL_DEPTH_TRACKING) &&
 		    spectre_v2_enabled == SPECTRE_V2_RETPOLINE) {
+			if (boot_cpu_data.x86_vendor != X86_VENDOR_INTEL) {
+				pr_err("WARNING: retbleed=stuff only supported for Intel CPUs.\n");
+				goto do_cmd_auto;
+			}
 			retbleed_mitigation = RETBLEED_MITIGATION_STUFF;
 
 		} else {
-- 
2.34.1

[PATCH v4 10/36] x86/bugs: Restructure retbleed mitigation

[David Kaplan]

Restructure retbleed mitigation to use select/update/apply functions to
create consistent vulnerability handling.  The retbleed_update_mitigation()
simplifies the dependency between spectre_v2 and retbleed.

The command line options now directly select a preferred mitigation
which simplifies the logic.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 180 ++++++++++++++++++-------------------
 1 file changed, 90 insertions(+), 90 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 4af342d226c8..80b554249d85 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -57,6 +57,8 @@ static void __init spectre_v1_select_mitigation(void);
 static void __init spectre_v1_apply_mitigation(void);
 static void __init spectre_v2_select_mitigation(void);
 static void __init retbleed_select_mitigation(void);
+static void __init retbleed_update_mitigation(void);
+static void __init retbleed_apply_mitigation(void);
 static void __init spectre_v2_user_select_mitigation(void);
 static void __init ssb_select_mitigation(void);
 static void __init l1tf_select_mitigation(void);
@@ -184,11 +186,6 @@ void __init cpu_select_mitigations(void)
 	/* Select the proper CPU mitigations before patching alternatives: */
 	spectre_v1_select_mitigation();
 	spectre_v2_select_mitigation();
-	/*
-	 * retbleed_select_mitigation() relies on the state set by
-	 * spectre_v2_select_mitigation(); specifically it wants to know about
-	 * spectre_v2=ibrs.
-	 */
 	retbleed_select_mitigation();
 	/*
 	 * spectre_v2_user_select_mitigation() relies on the state set by
@@ -216,12 +213,14 @@ void __init cpu_select_mitigations(void)
 	 * After mitigations are selected, some may need to update their
 	 * choices.
 	 */
+	retbleed_update_mitigation();
 	mds_update_mitigation();
 	taa_update_mitigation();
 	mmio_update_mitigation();
 	rfds_update_mitigation();
 
 	spectre_v1_apply_mitigation();
+	retbleed_apply_mitigation();
 	mds_apply_mitigation();
 	taa_apply_mitigation();
 	mmio_apply_mitigation();
@@ -1072,6 +1071,7 @@ enum spectre_v2_mitigation spectre_v2_enabled __ro_after_init = SPECTRE_V2_NONE;
 
 enum retbleed_mitigation {
 	RETBLEED_MITIGATION_NONE,
+	RETBLEED_MITIGATION_AUTO,
 	RETBLEED_MITIGATION_UNRET,
 	RETBLEED_MITIGATION_IBPB,
 	RETBLEED_MITIGATION_IBRS,
@@ -1079,14 +1079,6 @@ enum retbleed_mitigation {
 	RETBLEED_MITIGATION_STUFF,
 };
 
-enum retbleed_mitigation_cmd {
-	RETBLEED_CMD_OFF,
-	RETBLEED_CMD_AUTO,
-	RETBLEED_CMD_UNRET,
-	RETBLEED_CMD_IBPB,
-	RETBLEED_CMD_STUFF,
-};
-
 static const char * const retbleed_strings[] = {
 	[RETBLEED_MITIGATION_NONE]	= "Vulnerable",
 	[RETBLEED_MITIGATION_UNRET]	= "Mitigation: untrained return thunk",
@@ -1097,9 +1089,7 @@ static const char * const retbleed_strings[] = {
 };
 
 static enum retbleed_mitigation retbleed_mitigation __ro_after_init =
-	RETBLEED_MITIGATION_NONE;
-static enum retbleed_mitigation_cmd retbleed_cmd __ro_after_init =
-	IS_ENABLED(CONFIG_MITIGATION_RETBLEED) ? RETBLEED_CMD_AUTO : RETBLEED_CMD_OFF;
+	IS_ENABLED(CONFIG_MITIGATION_RETBLEED) ? RETBLEED_MITIGATION_AUTO : RETBLEED_MITIGATION_NONE;
 
 static int __ro_after_init retbleed_nosmt = false;
 
@@ -1116,15 +1106,15 @@ static int __init retbleed_parse_cmdline(char *str)
 		}
 
 		if (!strcmp(str, "off")) {
-			retbleed_cmd = RETBLEED_CMD_OFF;
+			retbleed_mitigation = RETBLEED_MITIGATION_NONE;
 		} else if (!strcmp(str, "auto")) {
-			retbleed_cmd = RETBLEED_CMD_AUTO;
+			retbleed_mitigation = RETBLEED_MITIGATION_AUTO;
 		} else if (!strcmp(str, "unret")) {
-			retbleed_cmd = RETBLEED_CMD_UNRET;
+			retbleed_mitigation = RETBLEED_MITIGATION_UNRET;
 		} else if (!strcmp(str, "ibpb")) {
-			retbleed_cmd = RETBLEED_CMD_IBPB;
+			retbleed_mitigation = RETBLEED_MITIGATION_IBPB;
 		} else if (!strcmp(str, "stuff")) {
-			retbleed_cmd = RETBLEED_CMD_STUFF;
+			retbleed_mitigation = RETBLEED_MITIGATION_STUFF;
 		} else if (!strcmp(str, "nosmt")) {
 			retbleed_nosmt = true;
 		} else if (!strcmp(str, "force")) {
@@ -1145,57 +1135,42 @@ early_param("retbleed", retbleed_parse_cmdline);
 
 static void __init retbleed_select_mitigation(void)
 {
-	bool mitigate_smt = false;
-
-	if (!boot_cpu_has_bug(X86_BUG_RETBLEED) || cpu_mitigations_off())
-		return;
-
-	switch (retbleed_cmd) {
-	case RETBLEED_CMD_OFF:
+	if (!boot_cpu_has_bug(X86_BUG_RETBLEED) || cpu_mitigations_off()) {
+		retbleed_mitigation = RETBLEED_MITIGATION_NONE;
 		return;
+	}
 
-	case RETBLEED_CMD_UNRET:
-		if (IS_ENABLED(CONFIG_MITIGATION_UNRET_ENTRY)) {
-			retbleed_mitigation = RETBLEED_MITIGATION_UNRET;
-		} else {
+	switch (retbleed_mitigation) {
+	case RETBLEED_MITIGATION_UNRET:
+		if (!IS_ENABLED(CONFIG_MITIGATION_UNRET_ENTRY)) {
+			retbleed_mitigation = RETBLEED_MITIGATION_AUTO;
 			pr_err("WARNING: kernel not compiled with MITIGATION_UNRET_ENTRY.\n");
-			goto do_cmd_auto;
 		}
 		break;
-
-	case RETBLEED_CMD_IBPB:
+	case RETBLEED_MITIGATION_IBPB:
 		if (!boot_cpu_has(X86_FEATURE_IBPB)) {
 			pr_err("WARNING: CPU does not support IBPB.\n");
-			goto do_cmd_auto;
-		} else if (IS_ENABLED(CONFIG_MITIGATION_IBPB_ENTRY)) {
-			retbleed_mitigation = RETBLEED_MITIGATION_IBPB;
-		} else {
+			retbleed_mitigation = RETBLEED_MITIGATION_AUTO;
+		} else if (!IS_ENABLED(CONFIG_MITIGATION_IBPB_ENTRY)) {
 			pr_err("WARNING: kernel not compiled with MITIGATION_IBPB_ENTRY.\n");
-			goto do_cmd_auto;
+			retbleed_mitigation = RETBLEED_MITIGATION_AUTO;
 		}
 		break;
-
-	case RETBLEED_CMD_STUFF:
-		if (IS_ENABLED(CONFIG_MITIGATION_CALL_DEPTH_TRACKING) &&
-		    spectre_v2_enabled == SPECTRE_V2_RETPOLINE) {
-			if (boot_cpu_data.x86_vendor != X86_VENDOR_INTEL) {
-				pr_err("WARNING: retbleed=stuff only supported for Intel CPUs.\n");
-				goto do_cmd_auto;
-			}
-			retbleed_mitigation = RETBLEED_MITIGATION_STUFF;
-
-		} else {
-			if (IS_ENABLED(CONFIG_MITIGATION_CALL_DEPTH_TRACKING))
-				pr_err("WARNING: retbleed=stuff depends on spectre_v2=retpoline\n");
-			else
-				pr_err("WARNING: kernel not compiled with MITIGATION_CALL_DEPTH_TRACKING.\n");
-
-			goto do_cmd_auto;
+	case RETBLEED_MITIGATION_STUFF:
+		if (!IS_ENABLED(CONFIG_MITIGATION_CALL_DEPTH_TRACKING)) {
+			pr_err("WARNING: kernel not compiled with MITIGATION_CALL_DEPTH_TRACKING.\n");
+			retbleed_mitigation = RETBLEED_MITIGATION_AUTO;
+		} else if (boot_cpu_data.x86_vendor != X86_VENDOR_INTEL) {
+			pr_err("WARNING: retbleed=stuff only supported for Intel CPUs.\n");
+			retbleed_mitigation = RETBLEED_MITIGATION_AUTO;
 		}
 		break;
+	default:
+		break;
+	}
 
-do_cmd_auto:
-	case RETBLEED_CMD_AUTO:
+	if (retbleed_mitigation == RETBLEED_MITIGATION_AUTO) {
+		/* Intel mitigation selected in retbleed_update_mitigation() */
 		if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD ||
 		    boot_cpu_data.x86_vendor == X86_VENDOR_HYGON) {
 			if (IS_ENABLED(CONFIG_MITIGATION_UNRET_ENTRY))
@@ -1203,18 +1178,65 @@ static void __init retbleed_select_mitigation(void)
 			else if (IS_ENABLED(CONFIG_MITIGATION_IBPB_ENTRY) &&
 				 boot_cpu_has(X86_FEATURE_IBPB))
 				retbleed_mitigation = RETBLEED_MITIGATION_IBPB;
+			else
+				retbleed_mitigation = RETBLEED_MITIGATION_NONE;
 		}
+	}
+}
 
-		/*
-		 * The Intel mitigation (IBRS or eIBRS) was already selected in
-		 * spectre_v2_select_mitigation().  'retbleed_mitigation' will
-		 * be set accordingly below.
-		 */
+static void __init retbleed_update_mitigation(void)
+{
+	if (!boot_cpu_has_bug(X86_BUG_RETBLEED) || cpu_mitigations_off())
+		return;
 
-		break;
+	if (retbleed_mitigation == RETBLEED_MITIGATION_NONE)
+		goto out;
+
+	/*
+	 * retbleed=stuff is only allowed on Intel.  If stuffing can't be used
+	 * then a different mitigation will be selected below.
+	 */
+	if (retbleed_mitigation == RETBLEED_MITIGATION_STUFF) {
+		if (spectre_v2_enabled != SPECTRE_V2_RETPOLINE) {
+			pr_err("WARNING: retbleed=stuff depends on spectre_v2=retpoline\n");
+			retbleed_mitigation = RETBLEED_MITIGATION_AUTO;
+		}
+	}
+	/*
+	 * Let IBRS trump all on Intel without affecting the effects of the
+	 * retbleed= cmdline option except for call depth based stuffing
+	 */
+	if (boot_cpu_data.x86_vendor == X86_VENDOR_INTEL) {
+		switch (spectre_v2_enabled) {
+		case SPECTRE_V2_IBRS:
+			retbleed_mitigation = RETBLEED_MITIGATION_IBRS;
+			break;
+		case SPECTRE_V2_EIBRS:
+		case SPECTRE_V2_EIBRS_RETPOLINE:
+		case SPECTRE_V2_EIBRS_LFENCE:
+			retbleed_mitigation = RETBLEED_MITIGATION_EIBRS;
+			break;
+		default:
+			if (retbleed_mitigation != RETBLEED_MITIGATION_STUFF)
+				pr_err(RETBLEED_INTEL_MSG);
+		}
+		/* If nothing has set the mitigation yet, default to NONE. */
+		if (retbleed_mitigation == RETBLEED_MITIGATION_AUTO)
+			retbleed_mitigation = RETBLEED_MITIGATION_NONE;
 	}
+out:
+	pr_info("%s\n", retbleed_strings[retbleed_mitigation]);
+}
+
+
+static void __init retbleed_apply_mitigation(void)
+{
+	bool mitigate_smt = false;
 
 	switch (retbleed_mitigation) {
+	case RETBLEED_MITIGATION_NONE:
+		return;
+
 	case RETBLEED_MITIGATION_UNRET:
 		setup_force_cpu_cap(X86_FEATURE_RETHUNK);
 		setup_force_cpu_cap(X86_FEATURE_UNRET);
@@ -1264,28 +1286,6 @@ static void __init retbleed_select_mitigation(void)
 	if (mitigate_smt && !boot_cpu_has(X86_FEATURE_STIBP) &&
 	    (retbleed_nosmt || cpu_mitigations_auto_nosmt()))
 		cpu_smt_disable(false);
-
-	/*
-	 * Let IBRS trump all on Intel without affecting the effects of the
-	 * retbleed= cmdline option except for call depth based stuffing
-	 */
-	if (boot_cpu_data.x86_vendor == X86_VENDOR_INTEL) {
-		switch (spectre_v2_enabled) {
-		case SPECTRE_V2_IBRS:
-			retbleed_mitigation = RETBLEED_MITIGATION_IBRS;
-			break;
-		case SPECTRE_V2_EIBRS:
-		case SPECTRE_V2_EIBRS_RETPOLINE:
-		case SPECTRE_V2_EIBRS_LFENCE:
-			retbleed_mitigation = RETBLEED_MITIGATION_EIBRS;
-			break;
-		default:
-			if (retbleed_mitigation != RETBLEED_MITIGATION_STUFF)
-				pr_err(RETBLEED_INTEL_MSG);
-		}
-	}
-
-	pr_info("%s\n", retbleed_strings[retbleed_mitigation]);
 }
 
 #undef pr_fmt
@@ -1839,8 +1839,8 @@ static void __init spectre_v2_select_mitigation(void)
 
 		if (IS_ENABLED(CONFIG_MITIGATION_IBRS_ENTRY) &&
 		    boot_cpu_has_bug(X86_BUG_RETBLEED) &&
-		    retbleed_cmd != RETBLEED_CMD_OFF &&
-		    retbleed_cmd != RETBLEED_CMD_STUFF &&
+		    retbleed_mitigation != RETBLEED_MITIGATION_NONE &&
+		    retbleed_mitigation != RETBLEED_MITIGATION_STUFF &&
 		    boot_cpu_has(X86_FEATURE_IBRS) &&
 		    boot_cpu_data.x86_vendor == X86_VENDOR_INTEL) {
 			mode = SPECTRE_V2_IBRS;
@@ -1989,7 +1989,7 @@ static void __init spectre_v2_select_mitigation(void)
 	    (boot_cpu_data.x86_vendor == X86_VENDOR_AMD ||
 	     boot_cpu_data.x86_vendor == X86_VENDOR_HYGON)) {
 
-		if (retbleed_cmd != RETBLEED_CMD_IBPB) {
+		if (retbleed_mitigation != RETBLEED_MITIGATION_IBPB) {
 			setup_force_cpu_cap(X86_FEATURE_USE_IBPB_FW);
 			pr_info("Enabling Speculation Barrier for firmware calls\n");
 		}
-- 
2.34.1

[PATCH v4 11/36] x86/bugs: Restructure spectre_v2_user mitigation

[David Kaplan]

Restructure spectre_v2_user to use select/update/apply functions to
create consistent vulnerability handling.

The ibpb/stibp choices are first decided based on the spectre_v2_user
command line but can be modified by the spectre_v2 command line option
as well.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 152 +++++++++++++++++++++----------------
 1 file changed, 85 insertions(+), 67 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 80b554249d85..623a3a3d3008 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -60,6 +60,8 @@ static void __init retbleed_select_mitigation(void);
 static void __init retbleed_update_mitigation(void);
 static void __init retbleed_apply_mitigation(void);
 static void __init spectre_v2_user_select_mitigation(void);
+static void __init spectre_v2_user_update_mitigation(void);
+static void __init spectre_v2_user_apply_mitigation(void);
 static void __init ssb_select_mitigation(void);
 static void __init l1tf_select_mitigation(void);
 static void __init mds_select_mitigation(void);
@@ -187,11 +189,6 @@ void __init cpu_select_mitigations(void)
 	spectre_v1_select_mitigation();
 	spectre_v2_select_mitigation();
 	retbleed_select_mitigation();
-	/*
-	 * spectre_v2_user_select_mitigation() relies on the state set by
-	 * retbleed_select_mitigation(); specifically the STIBP selection is
-	 * forced for UNRET or IBPB.
-	 */
 	spectre_v2_user_select_mitigation();
 	ssb_select_mitigation();
 	l1tf_select_mitigation();
@@ -214,6 +211,8 @@ void __init cpu_select_mitigations(void)
 	 * choices.
 	 */
 	retbleed_update_mitigation();
+	/* spectre_v2_user_update_mitigation() depends on retbleed_mitigation */
+	spectre_v2_user_update_mitigation();
 	mds_update_mitigation();
 	taa_update_mitigation();
 	mmio_update_mitigation();
@@ -221,6 +220,7 @@ void __init cpu_select_mitigations(void)
 
 	spectre_v1_apply_mitigation();
 	retbleed_apply_mitigation();
+	spectre_v2_user_apply_mitigation();
 	mds_apply_mitigation();
 	taa_apply_mitigation();
 	mmio_apply_mitigation();
@@ -1365,6 +1365,8 @@ enum spectre_v2_mitigation_cmd {
 	SPECTRE_V2_CMD_IBRS,
 };
 
+static enum spectre_v2_mitigation_cmd spectre_v2_cmd __ro_after_init = SPECTRE_V2_CMD_AUTO;
+
 enum spectre_v2_user_cmd {
 	SPECTRE_V2_USER_CMD_NONE,
 	SPECTRE_V2_USER_CMD_AUTO,
@@ -1403,31 +1405,19 @@ static void __init spec_v2_user_print_cond(const char *reason, bool secure)
 		pr_info("spectre_v2_user=%s forced on command line.\n", reason);
 }
 
-static __ro_after_init enum spectre_v2_mitigation_cmd spectre_v2_cmd;
-
 static enum spectre_v2_user_cmd __init
 spectre_v2_parse_user_cmdline(void)
 {
-	enum spectre_v2_user_cmd mode;
 	char arg[20];
 	int ret, i;
 
-	mode = IS_ENABLED(CONFIG_MITIGATION_SPECTRE_V2) ?
-		SPECTRE_V2_USER_CMD_AUTO : SPECTRE_V2_USER_CMD_NONE;
-
-	switch (spectre_v2_cmd) {
-	case SPECTRE_V2_CMD_NONE:
+	if (cpu_mitigations_off() || !IS_ENABLED(CONFIG_MITIGATION_SPECTRE_V2))
 		return SPECTRE_V2_USER_CMD_NONE;
-	case SPECTRE_V2_CMD_FORCE:
-		return SPECTRE_V2_USER_CMD_FORCE;
-	default:
-		break;
-	}
 
 	ret = cmdline_find_option(boot_command_line, "spectre_v2_user",
 				  arg, sizeof(arg));
 	if (ret < 0)
-		return mode;
+		return SPECTRE_V2_USER_CMD_AUTO;
 
 	for (i = 0; i < ARRAY_SIZE(v2_user_options); i++) {
 		if (match_option(arg, ret, v2_user_options[i].option)) {
@@ -1438,7 +1428,7 @@ spectre_v2_parse_user_cmdline(void)
 	}
 
 	pr_err("Unknown user space protection option (%s). Switching to default\n", arg);
-	return mode;
+	return SPECTRE_V2_USER_CMD_AUTO;
 }
 
 static inline bool spectre_v2_in_ibrs_mode(enum spectre_v2_mitigation mode)
@@ -1446,10 +1436,10 @@ static inline bool spectre_v2_in_ibrs_mode(enum spectre_v2_mitigation mode)
 	return spectre_v2_in_eibrs_mode(mode) || mode == SPECTRE_V2_IBRS;
 }
 
+
 static void __init
 spectre_v2_user_select_mitigation(void)
 {
-	enum spectre_v2_user_mitigation mode = SPECTRE_V2_USER_NONE;
 	enum spectre_v2_user_cmd cmd;
 
 	if (!boot_cpu_has(X86_FEATURE_IBPB) && !boot_cpu_has(X86_FEATURE_STIBP))
@@ -1458,48 +1448,61 @@ spectre_v2_user_select_mitigation(void)
 	cmd = spectre_v2_parse_user_cmdline();
 	switch (cmd) {
 	case SPECTRE_V2_USER_CMD_NONE:
-		goto set_mode;
+		return;
 	case SPECTRE_V2_USER_CMD_FORCE:
-		mode = SPECTRE_V2_USER_STRICT;
+		spectre_v2_user_ibpb = SPECTRE_V2_USER_STRICT;
+		spectre_v2_user_stibp = SPECTRE_V2_USER_STRICT;
 		break;
 	case SPECTRE_V2_USER_CMD_AUTO:
 	case SPECTRE_V2_USER_CMD_PRCTL:
+		spectre_v2_user_ibpb = SPECTRE_V2_USER_PRCTL;
+		spectre_v2_user_stibp = SPECTRE_V2_USER_PRCTL;
+		break;
 	case SPECTRE_V2_USER_CMD_PRCTL_IBPB:
-		mode = SPECTRE_V2_USER_PRCTL;
+		spectre_v2_user_ibpb = SPECTRE_V2_USER_STRICT;
+		spectre_v2_user_stibp = SPECTRE_V2_USER_PRCTL;
 		break;
 	case SPECTRE_V2_USER_CMD_SECCOMP:
-	case SPECTRE_V2_USER_CMD_SECCOMP_IBPB:
 		if (IS_ENABLED(CONFIG_SECCOMP))
-			mode = SPECTRE_V2_USER_SECCOMP;
+			spectre_v2_user_ibpb = SPECTRE_V2_USER_SECCOMP;
 		else
-			mode = SPECTRE_V2_USER_PRCTL;
+			spectre_v2_user_ibpb = SPECTRE_V2_USER_PRCTL;
+		spectre_v2_user_stibp = spectre_v2_user_ibpb;
+		break;
+	case SPECTRE_V2_USER_CMD_SECCOMP_IBPB:
+		spectre_v2_user_ibpb = SPECTRE_V2_USER_STRICT;
+		spectre_v2_user_stibp = SPECTRE_V2_USER_PRCTL;
 		break;
 	}
 
-	/* Initialize Indirect Branch Prediction Barrier */
-	if (boot_cpu_has(X86_FEATURE_IBPB)) {
-		static_branch_enable(&switch_vcpu_ibpb);
+	/*
+	 * At this point, an STIBP mode other than "off" has been set.
+	 * If STIBP support is not being forced, check if STIBP always-on
+	 * is preferred.
+	 */
+	if (spectre_v2_user_stibp != SPECTRE_V2_USER_STRICT &&
+	    boot_cpu_has(X86_FEATURE_AMD_STIBP_ALWAYS_ON))
+		spectre_v2_user_stibp = SPECTRE_V2_USER_STRICT_PREFERRED;
+}
 
-		spectre_v2_user_ibpb = mode;
-		switch (cmd) {
-		case SPECTRE_V2_USER_CMD_NONE:
-			break;
-		case SPECTRE_V2_USER_CMD_FORCE:
-		case SPECTRE_V2_USER_CMD_PRCTL_IBPB:
-		case SPECTRE_V2_USER_CMD_SECCOMP_IBPB:
-			static_branch_enable(&switch_mm_always_ibpb);
-			spectre_v2_user_ibpb = SPECTRE_V2_USER_STRICT;
-			break;
-		case SPECTRE_V2_USER_CMD_PRCTL:
-		case SPECTRE_V2_USER_CMD_AUTO:
-		case SPECTRE_V2_USER_CMD_SECCOMP:
-			static_branch_enable(&switch_mm_cond_ibpb);
-			break;
-		}
+static void __init spectre_v2_user_update_mitigation(void)
+{
+	bool smt_possible = IS_ENABLED(CONFIG_SMP);
 
-		pr_info("mitigation: Enabling %s Indirect Branch Prediction Barrier\n",
-			static_key_enabled(&switch_mm_always_ibpb) ?
-			"always-on" : "conditional");
+	if (!boot_cpu_has(X86_FEATURE_IBPB) && !boot_cpu_has(X86_FEATURE_STIBP))
+		return;
+
+	if (cpu_smt_control == CPU_SMT_FORCE_DISABLED ||
+	    cpu_smt_control == CPU_SMT_NOT_SUPPORTED)
+		smt_possible = false;
+
+	/* The spectre_v2 cmd line can override spectre_v2_user options */
+	if (spectre_v2_cmd == SPECTRE_V2_CMD_NONE) {
+		spectre_v2_user_ibpb = SPECTRE_V2_USER_NONE;
+		spectre_v2_user_stibp = SPECTRE_V2_USER_NONE;
+	} else if (spectre_v2_cmd == SPECTRE_V2_CMD_FORCE) {
+		spectre_v2_user_ibpb = SPECTRE_V2_USER_STRICT;
+		spectre_v2_user_stibp = SPECTRE_V2_USER_STRICT;
 	}
 
 	/*
@@ -1517,30 +1520,45 @@ spectre_v2_user_select_mitigation(void)
 	if (!boot_cpu_has(X86_FEATURE_STIBP) ||
 	    !cpu_smt_possible() ||
 	    (spectre_v2_in_eibrs_mode(spectre_v2_enabled) &&
-	     !boot_cpu_has(X86_FEATURE_AUTOIBRS)))
+	     !boot_cpu_has(X86_FEATURE_AUTOIBRS))) {
+		spectre_v2_user_stibp = SPECTRE_V2_USER_NONE;
 		return;
+	}
 
-	/*
-	 * At this point, an STIBP mode other than "off" has been set.
-	 * If STIBP support is not being forced, check if STIBP always-on
-	 * is preferred.
-	 */
-	if (mode != SPECTRE_V2_USER_STRICT &&
-	    boot_cpu_has(X86_FEATURE_AMD_STIBP_ALWAYS_ON))
-		mode = SPECTRE_V2_USER_STRICT_PREFERRED;
-
-	if (retbleed_mitigation == RETBLEED_MITIGATION_UNRET ||
-	    retbleed_mitigation == RETBLEED_MITIGATION_IBPB) {
-		if (mode != SPECTRE_V2_USER_STRICT &&
-		    mode != SPECTRE_V2_USER_STRICT_PREFERRED)
+	if (spectre_v2_user_stibp != SPECTRE_V2_USER_NONE &&
+	    (retbleed_mitigation == RETBLEED_MITIGATION_UNRET ||
+	     retbleed_mitigation == RETBLEED_MITIGATION_IBPB)) {
+		if (spectre_v2_user_stibp != SPECTRE_V2_USER_STRICT &&
+		    spectre_v2_user_stibp != SPECTRE_V2_USER_STRICT_PREFERRED)
 			pr_info("Selecting STIBP always-on mode to complement retbleed mitigation\n");
-		mode = SPECTRE_V2_USER_STRICT_PREFERRED;
+		spectre_v2_user_stibp = SPECTRE_V2_USER_STRICT_PREFERRED;
 	}
+	pr_info("%s\n", spectre_v2_user_strings[spectre_v2_user_stibp]);
+}
 
-	spectre_v2_user_stibp = mode;
+static void __init spectre_v2_user_apply_mitigation(void)
+{
+	/* Initialize Indirect Branch Prediction Barrier */
+	if (boot_cpu_has(X86_FEATURE_IBPB) &&
+	    spectre_v2_user_ibpb != SPECTRE_V2_USER_NONE) {
+		static_branch_enable(&switch_vcpu_ibpb);
 
-set_mode:
-	pr_info("%s\n", spectre_v2_user_strings[mode]);
+		switch (spectre_v2_user_ibpb) {
+		case SPECTRE_V2_USER_STRICT:
+			static_branch_enable(&switch_mm_always_ibpb);
+			break;
+		case SPECTRE_V2_USER_PRCTL:
+		case SPECTRE_V2_USER_SECCOMP:
+			static_branch_enable(&switch_mm_cond_ibpb);
+			break;
+		default:
+			break;
+		}
+
+		pr_info("mitigation: Enabling %s Indirect Branch Prediction Barrier\n",
+			static_key_enabled(&switch_mm_always_ibpb) ?
+			"always-on" : "conditional");
+	}
 }
 
 static const char * const spectre_v2_strings[] = {
-- 
2.34.1

[PATCH v4 12/36] x86/bugs: Restructure bhi mitigation

[David Kaplan]

Restructure bhi mitigation to use select/apply functions to create
consistent vulnerability handling.

Define new AUTO mitigation for bhi.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 623a3a3d3008..96cb2ac70245 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -82,6 +82,8 @@ static void __init l1d_flush_select_mitigation(void);
 static void __init srso_select_mitigation(void);
 static void __init gds_select_mitigation(void);
 static void __init gds_apply_mitigation(void);
+static void __init bhi_select_mitigation(void);
+static void __init bhi_apply_mitigation(void);
 
 /* The base value of the SPEC_CTRL MSR without task-specific bits set */
 u64 x86_spec_ctrl_base;
@@ -205,6 +207,7 @@ void __init cpu_select_mitigations(void)
 	 */
 	srso_select_mitigation();
 	gds_select_mitigation();
+	bhi_select_mitigation();
 
 	/*
 	 * After mitigations are selected, some may need to update their
@@ -227,6 +230,7 @@ void __init cpu_select_mitigations(void)
 	rfds_apply_mitigation();
 	srbds_apply_mitigation();
 	gds_apply_mitigation();
+	bhi_apply_mitigation();
 }
 
 /*
@@ -1775,12 +1779,13 @@ static bool __init spec_ctrl_bhi_dis(void)
 
 enum bhi_mitigations {
 	BHI_MITIGATION_OFF,
+	BHI_MITIGATION_AUTO,
 	BHI_MITIGATION_ON,
 	BHI_MITIGATION_VMEXIT_ONLY,
 };
 
 static enum bhi_mitigations bhi_mitigation __ro_after_init =
-	IS_ENABLED(CONFIG_MITIGATION_SPECTRE_BHI) ? BHI_MITIGATION_ON : BHI_MITIGATION_OFF;
+	IS_ENABLED(CONFIG_MITIGATION_SPECTRE_BHI) ? BHI_MITIGATION_AUTO : BHI_MITIGATION_OFF;
 
 static int __init spectre_bhi_parse_cmdline(char *str)
 {
@@ -1801,6 +1806,15 @@ static int __init spectre_bhi_parse_cmdline(char *str)
 early_param("spectre_bhi", spectre_bhi_parse_cmdline);
 
 static void __init bhi_select_mitigation(void)
+{
+	if (!boot_cpu_has(X86_BUG_BHI) || cpu_mitigations_off())
+		bhi_mitigation = BHI_MITIGATION_OFF;
+
+	if (bhi_mitigation == BHI_MITIGATION_AUTO)
+		bhi_mitigation = BHI_MITIGATION_ON;
+}
+
+static void __init bhi_apply_mitigation(void)
 {
 	if (bhi_mitigation == BHI_MITIGATION_OFF)
 		return;
@@ -1942,9 +1956,6 @@ static void __init spectre_v2_select_mitigation(void)
 	    mode == SPECTRE_V2_RETPOLINE)
 		spec_ctrl_disable_kernel_rrsba();
 
-	if (boot_cpu_has(X86_BUG_BHI))
-		bhi_select_mitigation();
-
 	spectre_v2_enabled = mode;
 	pr_info("%s\n", spectre_v2_strings[mode]);
 
-- 
2.34.1

[PATCH v4 13/36] x86/bugs: Restructure spectre_v2 mitigation

[David Kaplan]

Restructure spectre_v2 to use select/update/apply functions to create
consistent vulnerability handling.

The spectre_v2 mitigation may be updated based on the selected retbleed
mitigation.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 79 +++++++++++++++++++++++---------------
 1 file changed, 48 insertions(+), 31 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 96cb2ac70245..b4a72ddf159c 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -56,6 +56,8 @@
 static void __init spectre_v1_select_mitigation(void);
 static void __init spectre_v1_apply_mitigation(void);
 static void __init spectre_v2_select_mitigation(void);
+static void __init spectre_v2_update_mitigation(void);
+static void __init spectre_v2_apply_mitigation(void);
 static void __init retbleed_select_mitigation(void);
 static void __init retbleed_update_mitigation(void);
 static void __init retbleed_apply_mitigation(void);
@@ -212,7 +214,12 @@ void __init cpu_select_mitigations(void)
 	/*
 	 * After mitigations are selected, some may need to update their
 	 * choices.
+	 *
+	 * Note that retbleed_update_mitigation() relies on the state set by
+	 * spectre_v2_update_mitigation(); specifically it wants to know about
+	 * spectre_v2=ibrs.
 	 */
+	spectre_v2_update_mitigation();
 	retbleed_update_mitigation();
 	/* spectre_v2_user_update_mitigation() depends on retbleed_mitigation */
 	spectre_v2_user_update_mitigation();
@@ -222,6 +229,7 @@ void __init cpu_select_mitigations(void)
 	rfds_update_mitigation();
 
 	spectre_v1_apply_mitigation();
+	spectre_v2_apply_mitigation();
 	retbleed_apply_mitigation();
 	spectre_v2_user_apply_mitigation();
 	mds_apply_mitigation();
@@ -1847,18 +1855,18 @@ static void __init bhi_apply_mitigation(void)
 
 static void __init spectre_v2_select_mitigation(void)
 {
-	enum spectre_v2_mitigation_cmd cmd = spectre_v2_parse_cmdline();
 	enum spectre_v2_mitigation mode = SPECTRE_V2_NONE;
+	spectre_v2_cmd = spectre_v2_parse_cmdline();
 
 	/*
 	 * If the CPU is not affected and the command line mode is NONE or AUTO
 	 * then nothing to do.
 	 */
 	if (!boot_cpu_has_bug(X86_BUG_SPECTRE_V2) &&
-	    (cmd == SPECTRE_V2_CMD_NONE || cmd == SPECTRE_V2_CMD_AUTO))
+	    (spectre_v2_cmd == SPECTRE_V2_CMD_NONE || spectre_v2_cmd == SPECTRE_V2_CMD_AUTO))
 		return;
 
-	switch (cmd) {
+	switch (spectre_v2_cmd) {
 	case SPECTRE_V2_CMD_NONE:
 		return;
 
@@ -1869,16 +1877,6 @@ static void __init spectre_v2_select_mitigation(void)
 			break;
 		}
 
-		if (IS_ENABLED(CONFIG_MITIGATION_IBRS_ENTRY) &&
-		    boot_cpu_has_bug(X86_BUG_RETBLEED) &&
-		    retbleed_mitigation != RETBLEED_MITIGATION_NONE &&
-		    retbleed_mitigation != RETBLEED_MITIGATION_STUFF &&
-		    boot_cpu_has(X86_FEATURE_IBRS) &&
-		    boot_cpu_data.x86_vendor == X86_VENDOR_INTEL) {
-			mode = SPECTRE_V2_IBRS;
-			break;
-		}
-
 		mode = spectre_v2_select_retpoline();
 		break;
 
@@ -1912,10 +1910,32 @@ static void __init spectre_v2_select_mitigation(void)
 		break;
 	}
 
-	if (mode == SPECTRE_V2_EIBRS && unprivileged_ebpf_enabled())
+	spectre_v2_enabled = mode;
+}
+
+static void __init spectre_v2_update_mitigation(void)
+{
+	if (spectre_v2_cmd == SPECTRE_V2_CMD_AUTO) {
+		if (IS_ENABLED(CONFIG_MITIGATION_IBRS_ENTRY) &&
+		    boot_cpu_has_bug(X86_BUG_RETBLEED) &&
+		    retbleed_mitigation != RETBLEED_MITIGATION_NONE &&
+		    retbleed_mitigation != RETBLEED_MITIGATION_STUFF &&
+		    boot_cpu_has(X86_FEATURE_IBRS) &&
+		    boot_cpu_data.x86_vendor == X86_VENDOR_INTEL) {
+			spectre_v2_enabled = SPECTRE_V2_IBRS;
+		}
+	}
+
+	if (boot_cpu_has_bug(X86_BUG_SPECTRE_V2) && !cpu_mitigations_off())
+		pr_info("%s\n", spectre_v2_strings[spectre_v2_enabled]);
+}
+
+static void __init spectre_v2_apply_mitigation(void)
+{
+	if (spectre_v2_enabled == SPECTRE_V2_EIBRS && unprivileged_ebpf_enabled())
 		pr_err(SPECTRE_V2_EIBRS_EBPF_MSG);
 
-	if (spectre_v2_in_ibrs_mode(mode)) {
+	if (spectre_v2_in_ibrs_mode(spectre_v2_enabled)) {
 		if (boot_cpu_has(X86_FEATURE_AUTOIBRS)) {
 			msr_set_bit(MSR_EFER, _EFER_AUTOIBRS);
 		} else {
@@ -1924,8 +1944,10 @@ static void __init spectre_v2_select_mitigation(void)
 		}
 	}
 
-	switch (mode) {
+	switch (spectre_v2_enabled) {
 	case SPECTRE_V2_NONE:
+		return;
+
 	case SPECTRE_V2_EIBRS:
 		break;
 
@@ -1951,14 +1973,11 @@ static void __init spectre_v2_select_mitigation(void)
 	 * JMPs gets protection against BHI and Intramode-BTI, but RET
 	 * prediction from a non-RSB predictor is still a risk.
 	 */
-	if (mode == SPECTRE_V2_EIBRS_LFENCE ||
-	    mode == SPECTRE_V2_EIBRS_RETPOLINE ||
-	    mode == SPECTRE_V2_RETPOLINE)
+	if (spectre_v2_enabled == SPECTRE_V2_EIBRS_LFENCE ||
+	    spectre_v2_enabled == SPECTRE_V2_EIBRS_RETPOLINE ||
+	    spectre_v2_enabled == SPECTRE_V2_RETPOLINE)
 		spec_ctrl_disable_kernel_rrsba();
 
-	spectre_v2_enabled = mode;
-	pr_info("%s\n", spectre_v2_strings[mode]);
-
 	/*
 	 * If Spectre v2 protection has been enabled, fill the RSB during a
 	 * context switch.  In general there are two types of RSB attacks
@@ -2000,7 +2019,7 @@ static void __init spectre_v2_select_mitigation(void)
 	setup_force_cpu_cap(X86_FEATURE_RSB_CTXSW);
 	pr_info("Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch\n");
 
-	spectre_v2_determine_rsb_fill_type_at_vmexit(mode);
+	spectre_v2_determine_rsb_fill_type_at_vmexit(spectre_v2_enabled);
 
 	/*
 	 * Retpoline protects the kernel, but doesn't protect firmware.  IBRS
@@ -2008,10 +2027,10 @@ static void __init spectre_v2_select_mitigation(void)
 	 * firmware calls only when IBRS / Enhanced / Automatic IBRS aren't
 	 * otherwise enabled.
 	 *
-	 * Use "mode" to check Enhanced IBRS instead of boot_cpu_has(), because
-	 * the user might select retpoline on the kernel command line and if
-	 * the CPU supports Enhanced IBRS, kernel might un-intentionally not
-	 * enable IBRS around firmware calls.
+	 * Use "spectre_v2_enabled" to check Enhanced IBRS instead of
+	 * boot_cpu_has(), because the user might select retpoline on the kernel
+	 * command line and if the CPU supports Enhanced IBRS, kernel might
+	 * un-intentionally not enable IBRS around firmware calls.
 	 */
 	if (boot_cpu_has_bug(X86_BUG_RETBLEED) &&
 	    boot_cpu_has(X86_FEATURE_IBPB) &&
@@ -2023,13 +2042,11 @@ static void __init spectre_v2_select_mitigation(void)
 			pr_info("Enabling Speculation Barrier for firmware calls\n");
 		}
 
-	} else if (boot_cpu_has(X86_FEATURE_IBRS) && !spectre_v2_in_ibrs_mode(mode)) {
+	} else if (boot_cpu_has(X86_FEATURE_IBRS) &&
+		   !spectre_v2_in_ibrs_mode(spectre_v2_enabled)) {
 		setup_force_cpu_cap(X86_FEATURE_USE_IBRS_FW);
 		pr_info("Enabling Restricted Speculation for firmware calls\n");
 	}
-
-	/* Set up IBPB and STIBP depending on the general spectre V2 command */
-	spectre_v2_cmd = cmd;
 }
 
 static void update_stibp_msr(void * __unused)
-- 
2.34.1

[PATCH v4 14/36] x86/bugs: Restructure ssb mitigation

[David Kaplan]

Restructure ssb to use select/apply functions to create consistent
vulnerability handling.

Remove __ssb_select_mitigation() and split the functionality between the
select/apply functions.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 36 +++++++++++++++++-------------------
 1 file changed, 17 insertions(+), 19 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index b4a72ddf159c..1d77747e0f74 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -65,6 +65,7 @@ static void __init spectre_v2_user_select_mitigation(void);
 static void __init spectre_v2_user_update_mitigation(void);
 static void __init spectre_v2_user_apply_mitigation(void);
 static void __init ssb_select_mitigation(void);
+static void __init ssb_apply_mitigation(void);
 static void __init l1tf_select_mitigation(void);
 static void __init mds_select_mitigation(void);
 static void __init mds_update_mitigation(void);
@@ -232,6 +233,7 @@ void __init cpu_select_mitigations(void)
 	spectre_v2_apply_mitigation();
 	retbleed_apply_mitigation();
 	spectre_v2_user_apply_mitigation();
+	ssb_apply_mitigation();
 	mds_apply_mitigation();
 	taa_apply_mitigation();
 	mmio_apply_mitigation();
@@ -2235,19 +2237,18 @@ static enum ssb_mitigation_cmd __init ssb_parse_cmdline(void)
 	return cmd;
 }
 
-static enum ssb_mitigation __init __ssb_select_mitigation(void)
+static void ssb_select_mitigation(void)
 {
-	enum ssb_mitigation mode = SPEC_STORE_BYPASS_NONE;
 	enum ssb_mitigation_cmd cmd;
 
 	if (!boot_cpu_has(X86_FEATURE_SSBD))
-		return mode;
+		goto out;
 
 	cmd = ssb_parse_cmdline();
 	if (!boot_cpu_has_bug(X86_BUG_SPEC_STORE_BYPASS) &&
 	    (cmd == SPEC_STORE_BYPASS_CMD_NONE ||
 	     cmd == SPEC_STORE_BYPASS_CMD_AUTO))
-		return mode;
+		return;
 
 	switch (cmd) {
 	case SPEC_STORE_BYPASS_CMD_SECCOMP:
@@ -2256,28 +2257,35 @@ static enum ssb_mitigation __init __ssb_select_mitigation(void)
 		 * enabled.
 		 */
 		if (IS_ENABLED(CONFIG_SECCOMP))
-			mode = SPEC_STORE_BYPASS_SECCOMP;
+			ssb_mode = SPEC_STORE_BYPASS_SECCOMP;
 		else
-			mode = SPEC_STORE_BYPASS_PRCTL;
+			ssb_mode = SPEC_STORE_BYPASS_PRCTL;
 		break;
 	case SPEC_STORE_BYPASS_CMD_ON:
-		mode = SPEC_STORE_BYPASS_DISABLE;
+		ssb_mode = SPEC_STORE_BYPASS_DISABLE;
 		break;
 	case SPEC_STORE_BYPASS_CMD_AUTO:
 	case SPEC_STORE_BYPASS_CMD_PRCTL:
-		mode = SPEC_STORE_BYPASS_PRCTL;
+		ssb_mode = SPEC_STORE_BYPASS_PRCTL;
 		break;
 	case SPEC_STORE_BYPASS_CMD_NONE:
 		break;
 	}
 
+out:
+	if (boot_cpu_has_bug(X86_BUG_SPEC_STORE_BYPASS))
+		pr_info("%s\n", ssb_strings[ssb_mode]);
+}
+
+static void __init ssb_apply_mitigation(void)
+{
 	/*
 	 * We have three CPU feature flags that are in play here:
 	 *  - X86_BUG_SPEC_STORE_BYPASS - CPU is susceptible.
 	 *  - X86_FEATURE_SSBD - CPU is able to turn off speculative store bypass
 	 *  - X86_FEATURE_SPEC_STORE_BYPASS_DISABLE - engage the mitigation
 	 */
-	if (mode == SPEC_STORE_BYPASS_DISABLE) {
+	if (ssb_mode == SPEC_STORE_BYPASS_DISABLE) {
 		setup_force_cpu_cap(X86_FEATURE_SPEC_STORE_BYPASS_DISABLE);
 		/*
 		 * Intel uses the SPEC CTRL MSR Bit(2) for this, while AMD may
@@ -2291,16 +2299,6 @@ static enum ssb_mitigation __init __ssb_select_mitigation(void)
 			update_spec_ctrl(x86_spec_ctrl_base);
 		}
 	}
-
-	return mode;
-}
-
-static void ssb_select_mitigation(void)
-{
-	ssb_mode = __ssb_select_mitigation();
-
-	if (boot_cpu_has_bug(X86_BUG_SPEC_STORE_BYPASS))
-		pr_info("%s\n", ssb_strings[ssb_mode]);
 }
 
 #undef pr_fmt
-- 
2.34.1

[PATCH v4 15/36] x86/bugs: Restructure l1tf mitigation

[David Kaplan]

Restructure l1tf to use select/apply functions to create consistent
vulnerability handling.

Define new AUTO mitigation for l1tf.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/include/asm/processor.h |  1 +
 arch/x86/kernel/cpu/bugs.c       | 25 +++++++++++++++++++------
 arch/x86/kvm/vmx/vmx.c           |  2 ++
 3 files changed, 22 insertions(+), 6 deletions(-)

diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
index 5d2f7e5aff26..0973bed22172 100644
--- a/arch/x86/include/asm/processor.h
+++ b/arch/x86/include/asm/processor.h
@@ -734,6 +734,7 @@ void store_cpu_caps(struct cpuinfo_x86 *info);
 
 enum l1tf_mitigations {
 	L1TF_MITIGATION_OFF,
+	L1TF_MITIGATION_AUTO,
 	L1TF_MITIGATION_FLUSH_NOWARN,
 	L1TF_MITIGATION_FLUSH,
 	L1TF_MITIGATION_FLUSH_NOSMT,
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 1d77747e0f74..b9b7f5967f1e 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -67,6 +67,7 @@ static void __init spectre_v2_user_apply_mitigation(void);
 static void __init ssb_select_mitigation(void);
 static void __init ssb_apply_mitigation(void);
 static void __init l1tf_select_mitigation(void);
+static void __init l1tf_apply_mitigation(void);
 static void __init mds_select_mitigation(void);
 static void __init mds_update_mitigation(void);
 static void __init mds_apply_mitigation(void);
@@ -234,6 +235,7 @@ void __init cpu_select_mitigations(void)
 	retbleed_apply_mitigation();
 	spectre_v2_user_apply_mitigation();
 	ssb_apply_mitigation();
+	l1tf_apply_mitigation();
 	mds_apply_mitigation();
 	taa_apply_mitigation();
 	mmio_apply_mitigation();
@@ -2554,7 +2556,7 @@ EXPORT_SYMBOL_GPL(itlb_multihit_kvm_mitigation);
 
 /* Default mitigation for L1TF-affected CPUs */
 enum l1tf_mitigations l1tf_mitigation __ro_after_init =
-	IS_ENABLED(CONFIG_MITIGATION_L1TF) ? L1TF_MITIGATION_FLUSH : L1TF_MITIGATION_OFF;
+	IS_ENABLED(CONFIG_MITIGATION_L1TF) ? L1TF_MITIGATION_AUTO : L1TF_MITIGATION_OFF;
 #if IS_ENABLED(CONFIG_KVM_INTEL)
 EXPORT_SYMBOL_GPL(l1tf_mitigation);
 #endif
@@ -2601,23 +2603,34 @@ static void override_cache_bits(struct cpuinfo_x86 *c)
 }
 
 static void __init l1tf_select_mitigation(void)
+{
+	if (!boot_cpu_has_bug(X86_BUG_L1TF) || cpu_mitigations_off()) {
+		l1tf_mitigation = L1TF_MITIGATION_OFF;
+		return;
+	}
+
+	if (l1tf_mitigation == L1TF_MITIGATION_AUTO) {
+		if (cpu_mitigations_auto_nosmt())
+			l1tf_mitigation = L1TF_MITIGATION_FLUSH_NOSMT;
+		else
+			l1tf_mitigation = L1TF_MITIGATION_FLUSH;
+	}
+}
+
+static void __init l1tf_apply_mitigation(void)
 {
 	u64 half_pa;
 
 	if (!boot_cpu_has_bug(X86_BUG_L1TF))
 		return;
 
-	if (cpu_mitigations_off())
-		l1tf_mitigation = L1TF_MITIGATION_OFF;
-	else if (cpu_mitigations_auto_nosmt())
-		l1tf_mitigation = L1TF_MITIGATION_FLUSH_NOSMT;
-
 	override_cache_bits(&boot_cpu_data);
 
 	switch (l1tf_mitigation) {
 	case L1TF_MITIGATION_OFF:
 	case L1TF_MITIGATION_FLUSH_NOWARN:
 	case L1TF_MITIGATION_FLUSH:
+	case L1TF_MITIGATION_AUTO:
 		break;
 	case L1TF_MITIGATION_FLUSH_NOSMT:
 	case L1TF_MITIGATION_FULL:
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 3dd9007ae685..bd61e88ba445 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -273,6 +273,7 @@ static int vmx_setup_l1d_flush(enum vmx_l1d_flush_state l1tf)
 		case L1TF_MITIGATION_OFF:
 			l1tf = VMENTER_L1D_FLUSH_NEVER;
 			break;
+		case L1TF_MITIGATION_AUTO:
 		case L1TF_MITIGATION_FLUSH_NOWARN:
 		case L1TF_MITIGATION_FLUSH:
 		case L1TF_MITIGATION_FLUSH_NOSMT:
@@ -7652,6 +7653,7 @@ int vmx_vm_init(struct kvm *kvm)
 		case L1TF_MITIGATION_FLUSH_NOWARN:
 			/* 'I explicitly don't care' is set */
 			break;
+		case L1TF_MITIGATION_AUTO:
 		case L1TF_MITIGATION_FLUSH:
 		case L1TF_MITIGATION_FLUSH_NOSMT:
 		case L1TF_MITIGATION_FULL:
-- 
2.34.1

[PATCH v4 16/36] x86/bugs: Restructure srso mitigation

[David Kaplan]

Restructure srso to use select/update/apply functions to create
consistent vulnerability handling.  Like with retbleed, the command line
options directly select mitigations which can later be modified.

Also fix the ibpb-vmexit case to look at CONFIG_MITIGATION_IBPB_ENTRY
instead of CONFIG_MITIGATION_SRSO.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 213 +++++++++++++++++--------------------
 1 file changed, 97 insertions(+), 116 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index b9b7f5967f1e..d48b0a941b2d 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -84,6 +84,8 @@ static void __init srbds_select_mitigation(void);
 static void __init srbds_apply_mitigation(void);
 static void __init l1d_flush_select_mitigation(void);
 static void __init srso_select_mitigation(void);
+static void __init srso_update_mitigation(void);
+static void __init srso_apply_mitigation(void);
 static void __init gds_select_mitigation(void);
 static void __init gds_apply_mitigation(void);
 static void __init bhi_select_mitigation(void);
@@ -204,11 +206,6 @@ void __init cpu_select_mitigations(void)
 	rfds_select_mitigation();
 	srbds_select_mitigation();
 	l1d_flush_select_mitigation();
-
-	/*
-	 * srso_select_mitigation() depends and must run after
-	 * retbleed_select_mitigation().
-	 */
 	srso_select_mitigation();
 	gds_select_mitigation();
 	bhi_select_mitigation();
@@ -229,6 +226,8 @@ void __init cpu_select_mitigations(void)
 	taa_update_mitigation();
 	mmio_update_mitigation();
 	rfds_update_mitigation();
+	/* srso_update_mitigation() relies on retbleed_mitigation. */
+	srso_update_mitigation();
 
 	spectre_v1_apply_mitigation();
 	spectre_v2_apply_mitigation();
@@ -241,6 +240,7 @@ void __init cpu_select_mitigations(void)
 	mmio_apply_mitigation();
 	rfds_apply_mitigation();
 	srbds_apply_mitigation();
+	srso_apply_mitigation();
 	gds_apply_mitigation();
 	bhi_apply_mitigation();
 }
@@ -2690,6 +2690,7 @@ early_param("l1tf", l1tf_cmdline);
 
 enum srso_mitigation {
 	SRSO_MITIGATION_NONE,
+	SRSO_MITIGATION_AUTO,
 	SRSO_MITIGATION_UCODE_NEEDED,
 	SRSO_MITIGATION_SAFE_RET_UCODE_NEEDED,
 	SRSO_MITIGATION_MICROCODE,
@@ -2699,14 +2700,6 @@ enum srso_mitigation {
 	SRSO_MITIGATION_BP_SPEC_REDUCE,
 };
 
-enum srso_mitigation_cmd {
-	SRSO_CMD_OFF,
-	SRSO_CMD_MICROCODE,
-	SRSO_CMD_SAFE_RET,
-	SRSO_CMD_IBPB,
-	SRSO_CMD_IBPB_ON_VMEXIT,
-};
-
 static const char * const srso_strings[] = {
 	[SRSO_MITIGATION_NONE]			= "Vulnerable",
 	[SRSO_MITIGATION_UCODE_NEEDED]		= "Vulnerable: No microcode",
@@ -2718,8 +2711,7 @@ static const char * const srso_strings[] = {
 	[SRSO_MITIGATION_BP_SPEC_REDUCE]	= "Mitigation: Reduced Speculation"
 };
 
-static enum srso_mitigation srso_mitigation __ro_after_init = SRSO_MITIGATION_NONE;
-static enum srso_mitigation_cmd srso_cmd __ro_after_init = SRSO_CMD_SAFE_RET;
+static enum srso_mitigation srso_mitigation __ro_after_init = SRSO_MITIGATION_AUTO;
 
 static int __init srso_parse_cmdline(char *str)
 {
@@ -2727,15 +2719,15 @@ static int __init srso_parse_cmdline(char *str)
 		return -EINVAL;
 
 	if (!strcmp(str, "off"))
-		srso_cmd = SRSO_CMD_OFF;
+		srso_mitigation = SRSO_MITIGATION_NONE;
 	else if (!strcmp(str, "microcode"))
-		srso_cmd = SRSO_CMD_MICROCODE;
+		srso_mitigation = SRSO_MITIGATION_MICROCODE;
 	else if (!strcmp(str, "safe-ret"))
-		srso_cmd = SRSO_CMD_SAFE_RET;
+		srso_mitigation = SRSO_MITIGATION_SAFE_RET;
 	else if (!strcmp(str, "ibpb"))
-		srso_cmd = SRSO_CMD_IBPB;
+		srso_mitigation = SRSO_MITIGATION_IBPB;
 	else if (!strcmp(str, "ibpb-vmexit"))
-		srso_cmd = SRSO_CMD_IBPB_ON_VMEXIT;
+		srso_mitigation = SRSO_MITIGATION_IBPB_ON_VMEXIT;
 	else
 		pr_err("Ignoring unknown SRSO option (%s).", str);
 
@@ -2749,130 +2741,75 @@ static void __init srso_select_mitigation(void)
 {
 	bool has_microcode = boot_cpu_has(X86_FEATURE_IBPB_BRTYPE);
 
-	if (!boot_cpu_has_bug(X86_BUG_SRSO) ||
-	    cpu_mitigations_off() ||
-	    srso_cmd == SRSO_CMD_OFF) {
-		if (boot_cpu_has(X86_FEATURE_SBPB))
-			x86_pred_cmd = PRED_CMD_SBPB;
-		goto out;
-	}
+	if (!boot_cpu_has_bug(X86_BUG_SRSO) || cpu_mitigations_off())
+		srso_mitigation = SRSO_MITIGATION_NONE;
+
+	if (srso_mitigation == SRSO_MITIGATION_NONE)
+		return;
+
+	if (srso_mitigation == SRSO_MITIGATION_AUTO)
+		srso_mitigation = SRSO_MITIGATION_SAFE_RET;
 
 	if (has_microcode) {
 		/*
 		 * Zen1/2 with SMT off aren't vulnerable after the right
 		 * IBPB microcode has been applied.
-		 *
-		 * Zen1/2 don't have SBPB, no need to try to enable it here.
 		 */
 		if (boot_cpu_data.x86 < 0x19 && !cpu_smt_possible()) {
 			setup_force_cpu_cap(X86_FEATURE_SRSO_NO);
-			goto out;
-		}
-
-		if (retbleed_mitigation == RETBLEED_MITIGATION_IBPB) {
-			srso_mitigation = SRSO_MITIGATION_IBPB;
-			goto out;
+			srso_mitigation = SRSO_MITIGATION_NONE;
+			return;
 		}
 	} else {
 		pr_warn("IBPB-extending microcode not applied!\n");
 		pr_warn(SRSO_NOTICE);
-
-		/* may be overwritten by SRSO_CMD_SAFE_RET below */
-		srso_mitigation = SRSO_MITIGATION_UCODE_NEEDED;
 	}
 
-	switch (srso_cmd) {
-	case SRSO_CMD_MICROCODE:
-		if (has_microcode) {
-			srso_mitigation = SRSO_MITIGATION_MICROCODE;
-			pr_warn(SRSO_NOTICE);
-		}
-		break;
-
-	case SRSO_CMD_SAFE_RET:
-		if (boot_cpu_has(X86_FEATURE_SRSO_USER_KERNEL_NO))
-			goto ibpb_on_vmexit;
-
-		if (IS_ENABLED(CONFIG_MITIGATION_SRSO)) {
-			/*
-			 * Enable the return thunk for generated code
-			 * like ftrace, static_call, etc.
-			 */
-			setup_force_cpu_cap(X86_FEATURE_RETHUNK);
-			setup_force_cpu_cap(X86_FEATURE_UNRET);
-
-			if (boot_cpu_data.x86 == 0x19) {
-				setup_force_cpu_cap(X86_FEATURE_SRSO_ALIAS);
-				x86_return_thunk = srso_alias_return_thunk;
-			} else {
-				setup_force_cpu_cap(X86_FEATURE_SRSO);
-				x86_return_thunk = srso_return_thunk;
-			}
-			if (has_microcode)
-				srso_mitigation = SRSO_MITIGATION_SAFE_RET;
-			else
-				srso_mitigation = SRSO_MITIGATION_SAFE_RET_UCODE_NEEDED;
-		} else {
+	switch (srso_mitigation) {
+	case SRSO_MITIGATION_SAFE_RET:
+		if (!IS_ENABLED(CONFIG_MITIGATION_SRSO))
 			pr_err("WARNING: kernel not compiled with MITIGATION_SRSO.\n");
+		else if (boot_cpu_has(X86_FEATURE_SRSO_USER_KERNEL_NO)) {
+			srso_mitigation = SRSO_MITIGATION_IBPB_ON_VMEXIT;
+			goto ibpb_on_vmexit;
 		}
-		break;
 
-	case SRSO_CMD_IBPB:
-		if (IS_ENABLED(CONFIG_MITIGATION_IBPB_ENTRY)) {
-			if (has_microcode) {
-				setup_force_cpu_cap(X86_FEATURE_ENTRY_IBPB);
-				setup_force_cpu_cap(X86_FEATURE_IBPB_ON_VMEXIT);
-				srso_mitigation = SRSO_MITIGATION_IBPB;
-
-				/*
-				 * IBPB on entry already obviates the need for
-				 * software-based untraining so clear those in case some
-				 * other mitigation like Retbleed has selected them.
-				 */
-				setup_clear_cpu_cap(X86_FEATURE_UNRET);
-				setup_clear_cpu_cap(X86_FEATURE_RETHUNK);
-
-				/*
-				 * There is no need for RSB filling: entry_ibpb() ensures
-				 * all predictions, including the RSB, are invalidated,
-				 * regardless of IBPB implementation.
-				 */
-				setup_clear_cpu_cap(X86_FEATURE_RSB_VMEXIT);
-			}
-		} else {
-			pr_err("WARNING: kernel not compiled with MITIGATION_IBPB_ENTRY.\n");
-		}
+		if (!has_microcode)
+			srso_mitigation = SRSO_MITIGATION_SAFE_RET_UCODE_NEEDED;
 		break;
-
 ibpb_on_vmexit:
-	case SRSO_CMD_IBPB_ON_VMEXIT:
+	case SRSO_MITIGATION_IBPB_ON_VMEXIT:
 		if (boot_cpu_has(X86_FEATURE_SRSO_BP_SPEC_REDUCE)) {
 			pr_notice("Reducing speculation to address VM/HV SRSO attack vector.\n");
 			srso_mitigation = SRSO_MITIGATION_BP_SPEC_REDUCE;
 			break;
 		}
-
-		if (IS_ENABLED(CONFIG_MITIGATION_IBPB_ENTRY)) {
-			if (has_microcode) {
-				setup_force_cpu_cap(X86_FEATURE_IBPB_ON_VMEXIT);
-				srso_mitigation = SRSO_MITIGATION_IBPB_ON_VMEXIT;
-
-				/*
-				 * There is no need for RSB filling: entry_ibpb() ensures
-				 * all predictions, including the RSB, are invalidated,
-				 * regardless of IBPB implementation.
-				 */
-				setup_clear_cpu_cap(X86_FEATURE_RSB_VMEXIT);
-			}
-		} else {
+		fallthrough;
+	case SRSO_MITIGATION_IBPB:
+		if (!IS_ENABLED(CONFIG_MITIGATION_IBPB_ENTRY))
 			pr_err("WARNING: kernel not compiled with MITIGATION_IBPB_ENTRY.\n");
-		}
+
+		if (!has_microcode)
+			srso_mitigation = SRSO_MITIGATION_UCODE_NEEDED;
 		break;
 	default:
 		break;
 	}
+}
 
-out:
+static void __init srso_update_mitigation(void)
+{
+	/* If retbleed is using IBPB, that works for SRSO as well */
+	if (retbleed_mitigation == RETBLEED_MITIGATION_IBPB &&
+	    boot_cpu_has(X86_FEATURE_IBPB_BRTYPE))
+		srso_mitigation = SRSO_MITIGATION_IBPB;
+
+	if (boot_cpu_has_bug(X86_BUG_SRSO) && !cpu_mitigations_off())
+		pr_info("%s\n", srso_strings[srso_mitigation]);
+}
+
+static void __init srso_apply_mitigation(void)
+{
 	/*
 	 * Clear the feature flag if this mitigation is not selected as that
 	 * feature flag controls the BpSpecReduce MSR bit toggling in KVM.
@@ -2880,8 +2817,52 @@ static void __init srso_select_mitigation(void)
 	if (srso_mitigation != SRSO_MITIGATION_BP_SPEC_REDUCE)
 		setup_clear_cpu_cap(X86_FEATURE_SRSO_BP_SPEC_REDUCE);
 
-	if (srso_mitigation != SRSO_MITIGATION_NONE)
-		pr_info("%s\n", srso_strings[srso_mitigation]);
+	if (srso_mitigation == SRSO_MITIGATION_NONE) {
+		if (boot_cpu_has(X86_FEATURE_SBPB))
+			x86_pred_cmd = PRED_CMD_SBPB;
+		return;
+	}
+
+	switch (srso_mitigation) {
+	case SRSO_MITIGATION_SAFE_RET:
+	case SRSO_MITIGATION_SAFE_RET_UCODE_NEEDED:
+		/*
+		 * Enable the return thunk for generated code
+		 * like ftrace, static_call, etc.
+		 */
+		setup_force_cpu_cap(X86_FEATURE_RETHUNK);
+		setup_force_cpu_cap(X86_FEATURE_UNRET);
+
+		if (boot_cpu_data.x86 == 0x19) {
+			setup_force_cpu_cap(X86_FEATURE_SRSO_ALIAS);
+			x86_return_thunk = srso_alias_return_thunk;
+		} else {
+			setup_force_cpu_cap(X86_FEATURE_SRSO);
+			x86_return_thunk = srso_return_thunk;
+		}
+		break;
+	case SRSO_MITIGATION_IBPB:
+		setup_force_cpu_cap(X86_FEATURE_ENTRY_IBPB);
+		/*
+		 * IBPB on entry already obviates the need for
+		 * software-based untraining so clear those in case some
+		 * other mitigation like Retbleed has selected them.
+		 */
+		setup_clear_cpu_cap(X86_FEATURE_UNRET);
+		setup_clear_cpu_cap(X86_FEATURE_RETHUNK);
+		fallthrough;
+	case SRSO_MITIGATION_IBPB_ON_VMEXIT:
+		setup_force_cpu_cap(X86_FEATURE_IBPB_ON_VMEXIT);
+		/*
+		 * There is no need for RSB filling: entry_ibpb() ensures
+		 * all predictions, including the RSB, are invalidated,
+		 * regardless of IBPB implementation.
+		 */
+		setup_clear_cpu_cap(X86_FEATURE_RSB_VMEXIT);
+		break;
+	default:
+		break;
+	}
 }
 
 #undef pr_fmt
-- 
2.34.1

[PATCH v4 17/36] Documentation/x86: Document the new attack vector controls

[David Kaplan]

Document the 5 new attack vector command line options, how they
interact with existing vulnerability controls, and recommendations on when
they can be disabled.

Note that while mitigating against untrusted userspace requires both
user-to-kernel and user-to-user protection, these are kept separate.  The
kernel can control what code executes inside of it and that may affect the
risk associated with vulnerabilities especially if new kernel mitigations
are implemented.  The same isn't typically true of userspace.

In other words, the risk associated with user-to-user or guest-to-guest
attacks is unlikely to change over time.  While the risk associated with
user-to-kernel or guest-to-host attacks may change.  Therefore, these
controls are separated.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 .../hw-vuln/attack_vector_controls.rst        | 236 ++++++++++++++++++
 Documentation/admin-guide/hw-vuln/index.rst   |   1 +
 2 files changed, 237 insertions(+)
 create mode 100644 Documentation/admin-guide/hw-vuln/attack_vector_controls.rst

diff --git a/Documentation/admin-guide/hw-vuln/attack_vector_controls.rst b/Documentation/admin-guide/hw-vuln/attack_vector_controls.rst
new file mode 100644
index 000000000000..6a581503bc5d
--- /dev/null
+++ b/Documentation/admin-guide/hw-vuln/attack_vector_controls.rst
@@ -0,0 +1,236 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+Attack Vector Controls
+======================
+
+Attack vector controls provide a simple method to configure only the mitigations
+for CPU vulnerabilities which are relevant given the intended use of a system.
+Administrators are encouraged to consider which attack vectors are relevant and
+disable all others in order to recoup system performance.
+
+When new relevant CPU vulnerabilities are found, they will be added to these
+attack vector controls so administrators will likely not need to reconfigure
+their command line parameters as mitigations will continue to be correctly
+applied based on the chosen attack vector controls.
+
+Attack Vectors
+--------------
+
+There are 5 sets of attack-vector mitigations currently supported by the kernel:
+
+#. :ref:`user_kernel`
+#. :ref:`user_user`
+#. :ref:`guest_host`
+#. :ref:`guest_guest`
+#. :ref:`smt`
+
+To control the enabled attack vectors, see :ref:`cmdline`.
+
+.. _user_kernel:
+
+User-to-Kernel
+^^^^^^^^^^^^^^
+
+The user-to-kernel attack vector involves a malicious userspace program
+attempting to leak kernel data into userspace by exploiting a CPU vulnerability.
+The kernel data involved might be limited to certain kernel memory, or include
+all memory in the system, depending on the vulnerability exploited.
+
+If no untrusted userspace applications are being run, such as with single-user
+systems, consider disabling user-to-kernel mitigations.
+
+Note that the CPU vulnerabilities mitigated by Linux have generally not been
+shown to be exploitable from browser-based sandboxes.  User-to-kernel
+mitigations are therefore mostly relevant if unknown userspace applications may
+be run by untrusted users.
+
+*user-to-kernel mitigations are enabled by default*
+
+.. _user_user:
+
+User-to-User
+^^^^^^^^^^^^
+
+The user-to-user attack vector involves a malicious userspace program attempting
+to influence the behavior of another unsuspecting userspace program in order to
+exfiltrate data.  The vulnerability of a userspace program is based on the
+program itself and the interfaces it provides.
+
+If no untrusted userspace applications are being run, consider disabling
+user-to-user mitigations.
+
+Note that because the Linux kernel contains a mapping of all physical memory,
+preventing a malicious userspace program from leaking data from another
+userspace program requires mitigating user-to-kernel attacks as well for
+complete protection.
+
+*user-to-user mitigations are enabled by default*
+
+.. _guest_host:
+
+Guest-to-Host
+^^^^^^^^^^^^^
+
+The guest-to-host attack vector involves a malicious VM attempting to leak
+hypervisor data into the VM.  The data involved may be limited, or may
+potentially include all memory in the system, depending on the vulnerability
+exploited.
+
+If no untrusted VMs are being run, consider disabling guest-to-host mitigations.
+
+*guest-to-host mitigations are enabled by default if KVM support is present*
+
+.. _guest_guest:
+
+Guest-to-Guest
+^^^^^^^^^^^^^^
+
+The guest-to-guest attack vector involves a malicious VM attempting to influence
+the behavior of another unsuspecting VM in order to exfiltrate data.  The
+vulnerability of a VM is based on the code inside the VM itself and the
+interfaces it provides.
+
+If no untrusted VMs, or only a single VM is being run, consider disabling
+guest-to-guest mitigations.
+
+Similar to the user-to-user attack vector, preventing a malicious VM from
+leaking data from another VM requires mitigating guest-to-host attacks as well
+due to the Linux kernel phys map.
+
+*guest-to-guest mitigations are enabled by default if KVM support is present*
+
+.. _smt:
+
+Cross-Thread
+^^^^^^^^^^^^
+
+The cross-thread attack vector involves a malicious userspace program or
+malicious VM either observing or attempting to influence the behavior of code
+running on the SMT sibling thread in order to exfiltrate data.
+
+Many cross-thread attacks can only be mitigated if SMT is disabled, which will
+result in reduced CPU core count and reduced performance.
+
+If cross-thread mitigations are fully enabled ('auto,nosmt'), all mitigations
+for cross-thread attacks will be enabled.  SMT may be disabled depending on
+which vulnerabilities are present in the CPU.
+
+If cross-thread mitigations are partially enabled ('auto'), mitigations for
+cross-thread attacks will be enabled but SMT will not be disabled.
+
+If cross-thread mitigations are disabled, no mitigations for cross-thread
+attacks will be enabled.
+
+Cross-thread mitigation may not be required if core-scheduling or similar
+techniques are used to prevent untrusted workloads from running on SMT siblings.
+
+*cross-thread mitigations default to partially enabled*
+
+.. _cmdline:
+
+Command Line Controls
+---------------------
+
+Attack vectors are controlled through the mitigations= command line option.  The
+value provided begins with a global option and then may optionally include one
+or more options to disable various attack vectors.
+
+Format:
+	| ``mitigations=[global]``
+	| ``mitigations=[global];[attack vectors]``
+
+Global options:
+
+============ =============================================================
+Option       Description
+============ =============================================================
+'off'        All attack vectors disabled.
+'auto'       All attack vectors enabled, partial cross-thread mitigations.
+'auto,nosmt' All attack vectors enabled, full cross-thread mitigations.
+============ =============================================================
+
+Attack vector options:
+
+================= =======================================
+Option            Description
+================= =======================================
+'no_user_kernel'  Disables user-to-kernel mitigations.
+'no_user_user'    Disables user-to-user mitigations.
+'no_guest_host'   Disables guest-to-host mitigations.
+'no_guest_guest'  Disables guest-to-guest mitigations
+'no_cross_thread' Disables all cross-thread mitigations.
+================= =======================================
+
+Multiple attack vector options may be specified in a comma-separated list.  If
+the global option is not specified, it defaults to 'auto'.  The global option
+'off' is equivalent to disabling all attack vectors.
+
+Examples:
+	| ``mitigations=auto;no_user_kernel``
+
+	Enable all attack vectors except user-to-kernel.  Partial cross-thread
+	mitigations.
+
+	| ``mitigations=auto,nosmt;no_guest_host,no_guest_guest``
+
+	Enable all attack vectors and cross-thread mitigations except for
+	guest-to-host and guest-to-guest mitigations.
+
+	| ``mitigations=;no_cross_thread``
+
+	Enable all attack vectors but not cross-thread mitigations.
+
+Interactions with command-line options
+--------------------------------------
+
+Vulnerability-specific controls (e.g. "retbleed=off") take precedence over all
+attack vector controls.  Mitigations for individual vulnerabilities may be
+turned on or off via their command-line options regardless of the attack vector
+controls.
+
+Summary of attack-vector mitigations
+------------------------------------
+
+When a vulnerability is mitigated due to an attack-vector control, the default
+mitigation option for that particular vulnerability is used.  To use a different
+mitigation, please use the vulnerability-specific command line option.
+
+The table below summarizes which vulnerabilities are mitigated when different
+attack vectors are enabled and assuming the CPU is vulnerable.
+
+=============== ============== ============ ============= ============== ============
+Vulnerability   User-to-Kernel User-to-User Guest-to-Host Guest-to-Guest Cross-Thread
+=============== ============== ============ ============= ============== ============
+BHI                   X                           X
+GDS                   X              X            X              X        (Note 1)
+L1TF                  X                           X                       (Note 2)
+MDS                   X              X            X              X        (Note 2)
+MMIO                  X              X            X              X        (Note 2)
+Meltdown              X
+Retbleed              X                           X                       (Note 3)
+RFDS                  X              X            X              X
+Spectre_v1            X
+Spectre_v2            X                           X
+Spectre_v2_user                      X                           X        (Note 1)
+SRBDS                 X              X            X              X
+SRSO                  X                           X
+SSB (Note 4)
+TAA                   X              X            X              X        (Note 2)
+=============== ============== ============ ============= ============== ============
+
+Notes:
+   1 --  Can be mitigated without disabling SMT.
+
+   2 --  Disables SMT if cross-thread mitigations are fully enabled  and the CPU
+   is vulnerable
+
+   3 --  Disables SMT if cross-thread mitigations are fully enabled, the CPU is
+   vulnerable, and STIBP is not supported
+
+   4 --  Speculative store bypass is always enabled by default (no kernel
+   mitigation applied) unless overridden with spec_store_bypass_disable option
+
+When an attack-vector is disabled, all mitigations for the vulnerabilities
+listed in the above table are disabled, unless mitigation is required for a
+different enabled attack-vector or a mitigation is explicitly selected via a
+vulnerability-specific command line option.
diff --git a/Documentation/admin-guide/hw-vuln/index.rst b/Documentation/admin-guide/hw-vuln/index.rst
index ff0b440ef2dc..1add4a0baeb0 100644
--- a/Documentation/admin-guide/hw-vuln/index.rst
+++ b/Documentation/admin-guide/hw-vuln/index.rst
@@ -9,6 +9,7 @@ are configurable at compile, boot or run time.
 .. toctree::
    :maxdepth: 1
 
+   attack_vector_controls
    spectre
    l1tf
    mds
-- 
2.34.1

[PATCH v4 18/36] cpu: Define attack vectors

[David Kaplan]

Define 4 new attack vectors that are used for controlling CPU speculation
mitigations.  These may be individually disabled as part of the
mitigations= command line.  Attack vector controls are combined with global
options like 'auto' or 'auto,nosmt' like 'mitigations=auto;no_user_kernel'.

Cross-thread mitigations can either remain enabled fully, including
potentially disabling SMT ('auto,nosmt'), remain enabled except for
disabling SMT ('auto'), or entirely disabled through the new
'no_cross_thread' attack vector option.

The default settings for these attack vectors are consistent with existing
kernel defaults, other than the automatic disabling of VM-based attack
vectors if KVM support is not present.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 include/linux/cpu.h |  20 +++++++
 kernel/cpu.c        | 129 ++++++++++++++++++++++++++++++++++++++++----
 2 files changed, 138 insertions(+), 11 deletions(-)

diff --git a/include/linux/cpu.h b/include/linux/cpu.h
index 6a0a8f1c7c90..476c6dfe2c03 100644
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -190,9 +190,25 @@ void cpuhp_report_idle_dead(void);
 static inline void cpuhp_report_idle_dead(void) { }
 #endif /* #ifdef CONFIG_HOTPLUG_CPU */
 
+enum cpu_attack_vectors {
+	CPU_MITIGATE_USER_KERNEL,
+	CPU_MITIGATE_USER_USER,
+	CPU_MITIGATE_GUEST_HOST,
+	CPU_MITIGATE_GUEST_GUEST,
+	NR_CPU_ATTACK_VECTORS,
+};
+
+enum smt_mitigations {
+	SMT_MITIGATIONS_OFF,
+	SMT_MITIGATIONS_AUTO,
+	SMT_MITIGATIONS_ON,
+};
+
 #ifdef CONFIG_CPU_MITIGATIONS
 extern bool cpu_mitigations_off(void);
 extern bool cpu_mitigations_auto_nosmt(void);
+extern bool cpu_mitigate_attack_vector(enum cpu_attack_vectors v);
+extern enum smt_mitigations smt_mitigations;
 #else
 static inline bool cpu_mitigations_off(void)
 {
@@ -202,6 +218,10 @@ static inline bool cpu_mitigations_auto_nosmt(void)
 {
 	return false;
 }
+static inline bool cpu_mitigate_attack_vector(enum cpu_attack_vectors v)
+{
+	return false;
+}
 #endif
 
 #endif /* _LINUX_CPU_H_ */
diff --git a/kernel/cpu.c b/kernel/cpu.c
index ad755db29efd..fe00ab81d682 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -37,6 +37,7 @@
 #include <linux/cpuset.h>
 #include <linux/random.h>
 #include <linux/cc_platform.h>
+#include <linux/parser.h>
 
 #include <trace/events/power.h>
 #define CREATE_TRACE_POINTS
@@ -3178,8 +3179,38 @@ void __init boot_cpu_hotplug_init(void)
 
 #ifdef CONFIG_CPU_MITIGATIONS
 /*
- * These are used for a global "mitigations=" cmdline option for toggling
- * optional CPU mitigations.
+ * All except the cross-thread attack vector are mitigated by default.
+ * Cross-thread mitigation often requires disabling SMT which is too expensive
+ * to be enabled by default.
+ *
+ * Guest-to-Host and Guest-to-Guest vectors are only needed if KVM support is
+ * present.
+ */
+static bool attack_vectors[NR_CPU_ATTACK_VECTORS] __ro_after_init = {
+	[CPU_MITIGATE_USER_KERNEL] = true,
+	[CPU_MITIGATE_USER_USER] = true,
+	[CPU_MITIGATE_GUEST_HOST] = IS_ENABLED(CONFIG_KVM),
+	[CPU_MITIGATE_GUEST_GUEST] = IS_ENABLED(CONFIG_KVM),
+};
+
+bool cpu_mitigate_attack_vector(enum cpu_attack_vectors v)
+{
+	if (v < NR_CPU_ATTACK_VECTORS)
+		return attack_vectors[v];
+
+	WARN_ON_ONCE(v >= NR_CPU_ATTACK_VECTORS);
+	return false;
+}
+
+/*
+ * There are 3 global options, 'off', 'auto', 'auto,nosmt'.
+ * These may optionally be combined with attack-vector disables after a ';'.
+ *
+ * Examples:
+ *   mitigations=auto;no_user_kernel,no_user_user,no_cross_thread
+ *   mitigations=auto,nosmt;no_guest_host,no_guest_guest
+ *
+ * mitigations=off is equivalent to disabling all attack vectors.
  */
 enum cpu_mitigations {
 	CPU_MITIGATIONS_OFF,
@@ -3187,19 +3218,95 @@ enum cpu_mitigations {
 	CPU_MITIGATIONS_AUTO_NOSMT,
 };
 
+enum {
+	NO_USER_KERNEL,
+	NO_USER_USER,
+	NO_GUEST_HOST,
+	NO_GUEST_GUEST,
+	NO_CROSS_THREAD,
+	NR_VECTOR_PARAMS,
+};
+
+enum smt_mitigations smt_mitigations __ro_after_init = SMT_MITIGATIONS_AUTO;
 static enum cpu_mitigations cpu_mitigations __ro_after_init = CPU_MITIGATIONS_AUTO;
 
+static const match_table_t global_mitigations = {
+	{ CPU_MITIGATIONS_AUTO_NOSMT,	"auto,nosmt"},
+	{ CPU_MITIGATIONS_AUTO,		"auto"},
+	{ CPU_MITIGATIONS_OFF,		"off"},
+};
+
+static const match_table_t vector_mitigations = {
+	{ NO_USER_KERNEL,	"no_user_kernel"},
+	{ NO_USER_USER,		"no_user_user"},
+	{ NO_GUEST_HOST,	"no_guest_host"},
+	{ NO_GUEST_GUEST,	"no_guest_guest"},
+	{ NO_CROSS_THREAD,	"no_cross_thread"},
+	{ NR_VECTOR_PARAMS,	NULL},
+};
+
+static int __init mitigations_parse_global_opt(char *arg)
+{
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(global_mitigations); i++) {
+		const char *pattern = global_mitigations[i].pattern;
+
+		if (!strncmp(arg, pattern, strlen(pattern))) {
+			cpu_mitigations = global_mitigations[i].token;
+			return strlen(pattern);
+		}
+	}
+
+	return 0;
+}
+
 static int __init mitigations_parse_cmdline(char *arg)
 {
-	if (!strcmp(arg, "off"))
-		cpu_mitigations = CPU_MITIGATIONS_OFF;
-	else if (!strcmp(arg, "auto"))
-		cpu_mitigations = CPU_MITIGATIONS_AUTO;
-	else if (!strcmp(arg, "auto,nosmt"))
-		cpu_mitigations = CPU_MITIGATIONS_AUTO_NOSMT;
-	else
-		pr_crit("Unsupported mitigations=%s, system may still be vulnerable\n",
-			arg);
+	char *s, *p;
+	int len;
+
+	len = mitigations_parse_global_opt(arg);
+
+	if (cpu_mitigations_off()) {
+		memset(attack_vectors, 0, sizeof(attack_vectors));
+		smt_mitigations = SMT_MITIGATIONS_OFF;
+	} else if (cpu_mitigations_auto_nosmt())
+		smt_mitigations = SMT_MITIGATIONS_ON;
+
+	p = arg + len;
+
+	if (!*p)
+		return 0;
+
+	/* Attack vector controls may come after a ';' */
+	if (*p++ != ';' || !IS_ENABLED(CONFIG_ARCH_HAS_CPU_ATTACK_VECTORS)) {
+		pr_crit("Unsupported mitigations=%s, system may still be vulnerable\n",	arg);
+		return 0;
+	}
+
+	while ((s = strsep(&p, ",")) != NULL) {
+		switch (match_token(s, vector_mitigations, NULL)) {
+		case NO_USER_KERNEL:
+			attack_vectors[CPU_MITIGATE_USER_KERNEL] = false;
+			break;
+		case NO_USER_USER:
+			attack_vectors[CPU_MITIGATE_USER_USER] = false;
+			break;
+		case NO_GUEST_HOST:
+			attack_vectors[CPU_MITIGATE_GUEST_HOST] = false;
+			break;
+		case NO_GUEST_GUEST:
+			attack_vectors[CPU_MITIGATE_GUEST_GUEST] = false;
+			break;
+		case NO_CROSS_THREAD:
+			smt_mitigations = SMT_MITIGATIONS_OFF;
+			break;
+		default:
+			pr_crit("Unsupported mitigations options %s\n",	s);
+			return 0;
+		}
+	}
 
 	return 0;
 }
-- 
2.34.1

[PATCH v4 19/36] x86/Kconfig: Arch attack vector support

[David Kaplan]

ARCH_HAS_CPU_ATTACK_VECTORS should be set for architectures which implement
the new attack-vector based controls for CPU mitigations.  If an arch does
not support attack-vector based controls then an attempt to use them
results in a warning.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/Kconfig     | 3 +++
 arch/x86/Kconfig | 1 +
 2 files changed, 4 insertions(+)

diff --git a/arch/Kconfig b/arch/Kconfig
index 9f6eb09ef12d..cfa65367a08b 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -1748,4 +1748,7 @@ config ARCH_WANTS_PRE_LINK_VMLINUX
 	  An architecture can select this if it provides arch/<arch>/tools/Makefile
 	  with .arch.vmlinux.o target to be linked into vmlinux.
 
+config ARCH_HAS_CPU_ATTACK_VECTORS
+	bool
+
 endmenu
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 1665ebaba251..cb1c03021683 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -72,6 +72,7 @@ config X86
 	select ARCH_ENABLE_SPLIT_PMD_PTLOCK if (PGTABLE_LEVELS > 2) && (X86_64 || X86_PAE)
 	select ARCH_ENABLE_THP_MIGRATION if X86_64 && TRANSPARENT_HUGEPAGE
 	select ARCH_HAS_ACPI_TABLE_UPGRADE	if ACPI
+	select ARCH_HAS_CPU_ATTACK_VECTORS	if CPU_MITIGATIONS
 	select ARCH_HAS_CACHE_LINE_SIZE
 	select ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
 	select ARCH_HAS_CPU_FINALIZE_INIT
-- 
2.34.1

[PATCH v4 20/36] x86/bugs: Determine relevant vulnerabilities based on attack vector controls.

[David Kaplan]

The function should_mitigate_vuln() defines which vulnerabilities should
be mitigated based on the selected attack vector controls.  The
selections here are based on the individual characteristics of each
vulnerability.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 55 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 55 insertions(+)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index d48b0a941b2d..2323bfbcd694 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -300,6 +300,61 @@ static void x86_amd_ssb_disable(void)
 #undef pr_fmt
 #define pr_fmt(fmt)	"MDS: " fmt
 
+/*
+ * Returns true if vulnerability should be mitigated based on the
+ * selected attack vector controls.
+ *
+ * See Documentation/admin-guide/hw-vuln/attack_vector_controls.rst
+ */
+static bool __init should_mitigate_vuln(unsigned int bug)
+{
+	switch (bug) {
+	/*
+	 * The only runtime-selected spectre_v1 mitigations in the kernel are
+	 * related to SWAPGS protection on kernel entry.  Therefore, protection
+	 * is only required for the user->kernel attack vector.
+	 */
+	case X86_BUG_SPECTRE_V1:
+		return cpu_mitigate_attack_vector(CPU_MITIGATE_USER_KERNEL);
+
+	case X86_BUG_SPECTRE_V2:
+	case X86_BUG_RETBLEED:
+	case X86_BUG_SRSO:
+	case X86_BUG_L1TF:
+		return cpu_mitigate_attack_vector(CPU_MITIGATE_USER_KERNEL) ||
+		       cpu_mitigate_attack_vector(CPU_MITIGATE_GUEST_HOST);
+
+	case X86_BUG_SPECTRE_V2_USER:
+		return cpu_mitigate_attack_vector(CPU_MITIGATE_USER_USER) ||
+		       cpu_mitigate_attack_vector(CPU_MITIGATE_GUEST_GUEST);
+
+	/*
+	 * All the vulnerabilities below allow potentially leaking data
+	 * across address spaces.  Therefore, mitigation is required for
+	 * any of these 4 attack vectors.
+	 */
+	case X86_BUG_MDS:
+	case X86_BUG_TAA:
+	case X86_BUG_MMIO_STALE_DATA:
+	case X86_BUG_RFDS:
+	case X86_BUG_SRBDS:
+		return cpu_mitigate_attack_vector(CPU_MITIGATE_USER_KERNEL) ||
+		       cpu_mitigate_attack_vector(CPU_MITIGATE_GUEST_HOST) ||
+		       cpu_mitigate_attack_vector(CPU_MITIGATE_USER_USER) ||
+		       cpu_mitigate_attack_vector(CPU_MITIGATE_GUEST_GUEST);
+
+	case X86_BUG_GDS:
+		return cpu_mitigate_attack_vector(CPU_MITIGATE_USER_KERNEL) ||
+		       cpu_mitigate_attack_vector(CPU_MITIGATE_GUEST_HOST) ||
+		       cpu_mitigate_attack_vector(CPU_MITIGATE_USER_USER) ||
+		       cpu_mitigate_attack_vector(CPU_MITIGATE_GUEST_GUEST) ||
+		       (smt_mitigations != SMT_MITIGATIONS_OFF);
+	default:
+		WARN(1, "Unknown bug %x\n", bug);
+		return false;
+	}
+}
+
 /* Default mitigation for MDS-affected CPUs */
 static enum mds_mitigations mds_mitigation __ro_after_init =
 	IS_ENABLED(CONFIG_MITIGATION_MDS) ? MDS_MITIGATION_AUTO : MDS_MITIGATION_OFF;
-- 
2.34.1

[PATCH v4 21/36] x86/bugs: Add attack vector controls for mds

[David Kaplan]

Use attack vector controls to determine if mds mitigation is required.
The global mitigations=off command now simply disables all attack vectors
so explicit checking of mitigations=off is no longer needed.

If cross-thread attack mitigations are required, disable SMT.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 2323bfbcd694..197ef9f2cbce 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -405,20 +405,25 @@ static bool verw_mitigation_selected __ro_after_init;
 
 static void __init mds_select_mitigation(void)
 {
-	if (!boot_cpu_has_bug(X86_BUG_MDS) || cpu_mitigations_off()) {
+	if (!boot_cpu_has_bug(X86_BUG_MDS)) {
 		mds_mitigation = MDS_MITIGATION_OFF;
 		return;
 	}
 
-	if (mds_mitigation == MDS_MITIGATION_AUTO)
-		mds_mitigation = MDS_MITIGATION_FULL;
+	if (mds_mitigation == MDS_MITIGATION_AUTO) {
+		if (should_mitigate_vuln(X86_BUG_MDS))
+			mds_mitigation = MDS_MITIGATION_FULL;
+		else
+			mds_mitigation = MDS_MITIGATION_OFF;
+	}
 
-	verw_mitigation_selected = true;
+	if (mds_mitigation != MDS_MITIGATION_OFF)
+		verw_mitigation_selected = true;
 }
 
 static void __init mds_update_mitigation(void)
 {
-	if (!boot_cpu_has_bug(X86_BUG_MDS) || cpu_mitigations_off())
+	if (!boot_cpu_has_bug(X86_BUG_MDS))
 		return;
 
 	/* If TAA, MMIO, or RFDS are being mitigated, MDS gets mitigated too. */
@@ -439,7 +444,7 @@ static void __init mds_apply_mitigation(void)
 	    mds_mitigation == MDS_MITIGATION_VMWERV) {
 		setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF);
 		if (!boot_cpu_has(X86_BUG_MSBDS_ONLY) &&
-		    (mds_nosmt || cpu_mitigations_auto_nosmt()))
+		    (mds_nosmt || smt_mitigations == SMT_MITIGATIONS_ON))
 			cpu_smt_disable(false);
 	}
 }
-- 
2.34.1

[PATCH v4 22/36] x86/bugs: Add attack vector controls for taa

[David Kaplan]

Use attack vector controls to determine if taa mitigation is required.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 197ef9f2cbce..56d3bf343d1f 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -500,12 +500,13 @@ static void __init taa_select_mitigation(void)
 		return;
 	}
 
-	if (cpu_mitigations_off())
-		taa_mitigation = TAA_MITIGATION_OFF;
-
 	/* Microcode will be checked in taa_update_mitigation(). */
-	if (taa_mitigation == TAA_MITIGATION_AUTO)
-		taa_mitigation = TAA_MITIGATION_VERW;
+	if (taa_mitigation == TAA_MITIGATION_AUTO) {
+		if (should_mitigate_vuln(X86_BUG_TAA))
+			taa_mitigation = TAA_MITIGATION_VERW;
+		else
+			taa_mitigation = TAA_MITIGATION_OFF;
+	}
 
 	if (taa_mitigation != TAA_MITIGATION_OFF)
 		verw_mitigation_selected = true;
@@ -513,7 +514,7 @@ static void __init taa_select_mitigation(void)
 
 static void __init taa_update_mitigation(void)
 {
-	if (!taa_vulnerable() || cpu_mitigations_off())
+	if (!taa_vulnerable())
 		return;
 
 	if (verw_mitigation_selected)
@@ -554,7 +555,7 @@ static void __init taa_apply_mitigation(void)
 		 */
 		setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF);
 
-		if (taa_nosmt || cpu_mitigations_auto_nosmt())
+		if (taa_nosmt || smt_mitigations == SMT_MITIGATIONS_ON)
 			cpu_smt_disable(false);
 	}
 }
-- 
2.34.1

[PATCH v4 23/36] x86/bugs: Add attack vector controls for mmio

[David Kaplan]

Use attack vectors controls to determine if mmio mitigation is required.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 56d3bf343d1f..ba7c2a1ea98f 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -595,15 +595,21 @@ static const char * const mmio_strings[] = {
 static void __init mmio_select_mitigation(void)
 {
 	if (!boot_cpu_has_bug(X86_BUG_MMIO_STALE_DATA) ||
-	     boot_cpu_has_bug(X86_BUG_MMIO_UNKNOWN) ||
-	     cpu_mitigations_off()) {
+	     boot_cpu_has_bug(X86_BUG_MMIO_UNKNOWN)) {
 		mmio_mitigation = MMIO_MITIGATION_OFF;
 		return;
 	}
 
 	/* Microcode will be checked in mmio_update_mitigation(). */
-	if (mmio_mitigation == MMIO_MITIGATION_AUTO)
-		mmio_mitigation = MMIO_MITIGATION_VERW;
+	if (mmio_mitigation == MMIO_MITIGATION_AUTO) {
+		if (should_mitigate_vuln(X86_BUG_MMIO_STALE_DATA))
+			mmio_mitigation = MMIO_MITIGATION_VERW;
+		else
+			mmio_mitigation = MMIO_MITIGATION_OFF;
+	}
+
+	if (mmio_mitigation == MMIO_MITIGATION_OFF)
+		return;
 
 	/*
 	 * Enable CPU buffer clear mitigation for host and VMM, if also affected
@@ -615,7 +621,7 @@ static void __init mmio_select_mitigation(void)
 
 static void __init mmio_update_mitigation(void)
 {
-	if (!boot_cpu_has_bug(X86_BUG_MMIO_STALE_DATA) || cpu_mitigations_off())
+	if (!boot_cpu_has_bug(X86_BUG_MMIO_STALE_DATA))
 		return;
 
 	if (verw_mitigation_selected)
@@ -665,7 +671,7 @@ static void __init mmio_apply_mitigation(void)
 	if (!(x86_arch_cap_msr & ARCH_CAP_FBSDP_NO))
 		static_branch_enable(&mds_idle_clear);
 
-	if (mmio_nosmt || cpu_mitigations_auto_nosmt())
+	if (mmio_nosmt || smt_mitigations == SMT_MITIGATIONS_ON)
 		cpu_smt_disable(false);
 }
 
-- 
2.34.1

[PATCH v4 24/36] x86/bugs: Add attack vector controls for rfds

[David Kaplan]

Use attack vector controls to determine if rfds mitigation is required.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index ba7c2a1ea98f..32f850c80b86 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -712,13 +712,20 @@ static bool __init rfds_has_ucode(void)
 
 static void __init rfds_select_mitigation(void)
 {
-	if (!boot_cpu_has_bug(X86_BUG_RFDS) || cpu_mitigations_off()) {
+	if (!boot_cpu_has_bug(X86_BUG_RFDS)) {
 		rfds_mitigation = RFDS_MITIGATION_OFF;
 		return;
 	}
 
-	if (rfds_mitigation == RFDS_MITIGATION_AUTO)
-		rfds_mitigation = RFDS_MITIGATION_VERW;
+	if (rfds_mitigation == RFDS_MITIGATION_AUTO) {
+		if (should_mitigate_vuln(X86_BUG_RFDS))
+			rfds_mitigation = RFDS_MITIGATION_VERW;
+		else
+			rfds_mitigation = RFDS_MITIGATION_OFF;
+	}
+
+	if (rfds_mitigation == RFDS_MITIGATION_OFF)
+		return;
 
 	if (rfds_has_ucode())
 		verw_mitigation_selected = true;
@@ -726,7 +733,7 @@ static void __init rfds_select_mitigation(void)
 
 static void __init rfds_update_mitigation(void)
 {
-	if (!boot_cpu_has_bug(X86_BUG_RFDS) || cpu_mitigations_off())
+	if (!boot_cpu_has_bug(X86_BUG_RFDS))
 		return;
 
 	if (verw_mitigation_selected)
-- 
2.34.1

[PATCH v4 25/36] x86/bugs: Add attack vector controls for srbds

[David Kaplan]

Use attack vector controls to determine if srbds mitigation is required.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 32f850c80b86..148cac36a24b 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -834,13 +834,19 @@ void update_srbds_msr(void)
 
 static void __init srbds_select_mitigation(void)
 {
-	if (!boot_cpu_has_bug(X86_BUG_SRBDS) || cpu_mitigations_off()) {
+	if (!boot_cpu_has_bug(X86_BUG_SRBDS)) {
 		srbds_mitigation = SRBDS_MITIGATION_OFF;
 		return;
 	}
 
-	if (srbds_mitigation == SRBDS_MITIGATION_AUTO)
-		srbds_mitigation = SRBDS_MITIGATION_FULL;
+	if (srbds_mitigation == SRBDS_MITIGATION_AUTO) {
+		if (should_mitigate_vuln(X86_BUG_SRBDS))
+			srbds_mitigation = SRBDS_MITIGATION_FULL;
+		else {
+			srbds_mitigation = SRBDS_MITIGATION_OFF;
+			return;
+		}
+	}
 
 	/*
 	 * Check to see if this is one of the MDS_NO systems supporting TSX that
-- 
2.34.1

[PATCH v4 26/36] x86/bugs: Add attack vector controls for gds

[David Kaplan]

Use attack vector controls to determine if gds mitigation is required.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 148cac36a24b..5803c45dc15c 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -994,12 +994,15 @@ static void __init gds_select_mitigation(void)
 		return;
 	}
 
-	if (cpu_mitigations_off())
-		gds_mitigation = GDS_MITIGATION_OFF;
 	/* Will verify below that mitigation _can_ be disabled */
-
-	if (gds_mitigation == GDS_MITIGATION_AUTO)
-		gds_mitigation = GDS_MITIGATION_FULL;
+	if (gds_mitigation == GDS_MITIGATION_AUTO) {
+		if (should_mitigate_vuln(X86_BUG_GDS))
+			gds_mitigation = GDS_MITIGATION_FULL;
+		else {
+			gds_mitigation = GDS_MITIGATION_OFF;
+			return;
+		}
+	}
 
 	/* No microcode */
 	if (!(x86_arch_cap_msr & ARCH_CAP_GDS_CTRL)) {
-- 
2.34.1

[PATCH v4 27/36] x86/bugs: Add attack vector controls for spectre_v1

[David Kaplan]

Use attack vector controls to determine if spectre_v1 mitigation is
required.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 5803c45dc15c..00f679dcc28a 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -1108,13 +1108,16 @@ static bool smap_works_speculatively(void)
 
 static void __init spectre_v1_select_mitigation(void)
 {
-	if (!boot_cpu_has_bug(X86_BUG_SPECTRE_V1) || cpu_mitigations_off())
+	if (!boot_cpu_has_bug(X86_BUG_SPECTRE_V1))
+		spectre_v1_mitigation = SPECTRE_V1_MITIGATION_NONE;
+
+	if (!should_mitigate_vuln(X86_BUG_SPECTRE_V1))
 		spectre_v1_mitigation = SPECTRE_V1_MITIGATION_NONE;
 }
 
 static void __init spectre_v1_apply_mitigation(void)
 {
-	if (!boot_cpu_has_bug(X86_BUG_SPECTRE_V1) || cpu_mitigations_off())
+	if (!boot_cpu_has_bug(X86_BUG_SPECTRE_V1))
 		return;
 
 	if (spectre_v1_mitigation == SPECTRE_V1_MITIGATION_AUTO) {
-- 
2.34.1

[PATCH v4 28/36] x86/bugs: Add attack vector controls for retbleed

[David Kaplan]

Use attack vector controls to determine if retbleed mitigation is
required.

Disable SMT if cross-thread protection is desired and STIBP is not
available.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 30 +++++++++++++++++-------------
 1 file changed, 17 insertions(+), 13 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 00f679dcc28a..4d72c6dd66e8 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -1237,7 +1237,7 @@ early_param("retbleed", retbleed_parse_cmdline);
 
 static void __init retbleed_select_mitigation(void)
 {
-	if (!boot_cpu_has_bug(X86_BUG_RETBLEED) || cpu_mitigations_off()) {
+	if (!boot_cpu_has_bug(X86_BUG_RETBLEED)) {
 		retbleed_mitigation = RETBLEED_MITIGATION_NONE;
 		return;
 	}
@@ -1272,23 +1272,27 @@ static void __init retbleed_select_mitigation(void)
 	}
 
 	if (retbleed_mitigation == RETBLEED_MITIGATION_AUTO) {
-		/* Intel mitigation selected in retbleed_update_mitigation() */
-		if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD ||
-		    boot_cpu_data.x86_vendor == X86_VENDOR_HYGON) {
-			if (IS_ENABLED(CONFIG_MITIGATION_UNRET_ENTRY))
-				retbleed_mitigation = RETBLEED_MITIGATION_UNRET;
-			else if (IS_ENABLED(CONFIG_MITIGATION_IBPB_ENTRY) &&
-				 boot_cpu_has(X86_FEATURE_IBPB))
-				retbleed_mitigation = RETBLEED_MITIGATION_IBPB;
-			else
-				retbleed_mitigation = RETBLEED_MITIGATION_NONE;
+		if (should_mitigate_vuln(X86_BUG_RETBLEED)) {
+			/* Intel mitigation selected in retbleed_update_mitigation() */
+			if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD ||
+			    boot_cpu_data.x86_vendor == X86_VENDOR_HYGON) {
+				if (IS_ENABLED(CONFIG_MITIGATION_UNRET_ENTRY))
+					retbleed_mitigation = RETBLEED_MITIGATION_UNRET;
+				else if (IS_ENABLED(CONFIG_MITIGATION_IBPB_ENTRY) &&
+					 boot_cpu_has(X86_FEATURE_IBPB))
+					retbleed_mitigation = RETBLEED_MITIGATION_IBPB;
+				else
+					retbleed_mitigation = RETBLEED_MITIGATION_NONE;
+			}
+		} else {
+			retbleed_mitigation = RETBLEED_MITIGATION_NONE;
 		}
 	}
 }
 
 static void __init retbleed_update_mitigation(void)
 {
-	if (!boot_cpu_has_bug(X86_BUG_RETBLEED) || cpu_mitigations_off())
+	if (!boot_cpu_has_bug(X86_BUG_RETBLEED))
 		return;
 
 	if (retbleed_mitigation == RETBLEED_MITIGATION_NONE)
@@ -1386,7 +1390,7 @@ static void __init retbleed_apply_mitigation(void)
 	}
 
 	if (mitigate_smt && !boot_cpu_has(X86_FEATURE_STIBP) &&
-	    (retbleed_nosmt || cpu_mitigations_auto_nosmt()))
+	    (retbleed_nosmt || smt_mitigations == SMT_MITIGATIONS_ON))
 		cpu_smt_disable(false);
 }
 
-- 
2.34.1

[PATCH v4 29/36] x86/bugs: Add attack vector controls for spectre_v2_user

[David Kaplan]

Use attack vector controls to determine if spectre_v2_user mitigation is
required.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 4d72c6dd66e8..e06dee765fd5 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -1517,7 +1517,7 @@ spectre_v2_parse_user_cmdline(void)
 	char arg[20];
 	int ret, i;
 
-	if (cpu_mitigations_off() || !IS_ENABLED(CONFIG_MITIGATION_SPECTRE_V2))
+	if (!IS_ENABLED(CONFIG_MITIGATION_SPECTRE_V2))
 		return SPECTRE_V2_USER_CMD_NONE;
 
 	ret = cmdline_find_option(boot_command_line, "spectre_v2_user",
@@ -1560,6 +1560,13 @@ spectre_v2_user_select_mitigation(void)
 		spectre_v2_user_stibp = SPECTRE_V2_USER_STRICT;
 		break;
 	case SPECTRE_V2_USER_CMD_AUTO:
+		if (!should_mitigate_vuln(X86_BUG_SPECTRE_V2_USER))
+			return;
+		spectre_v2_user_ibpb = SPECTRE_V2_USER_PRCTL;
+		if (smt_mitigations == SMT_MITIGATIONS_OFF)
+			return;
+		spectre_v2_user_stibp = SPECTRE_V2_USER_PRCTL;
+		break;
 	case SPECTRE_V2_USER_CMD_PRCTL:
 		spectre_v2_user_ibpb = SPECTRE_V2_USER_PRCTL;
 		spectre_v2_user_stibp = SPECTRE_V2_USER_PRCTL;
-- 
2.34.1

[PATCH v4 30/36] x86/bugs: Add attack vector controls for bhi

[David Kaplan]

There are two BHI mitigations, one for SYSCALL and one for VMEXIT.
Split these up so they can be selected individually based on attack
vector.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 40 +++++++++++++++++++++++++-------------
 1 file changed, 27 insertions(+), 13 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index e06dee765fd5..4912367f2334 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -1889,8 +1889,9 @@ static bool __init spec_ctrl_bhi_dis(void)
 enum bhi_mitigations {
 	BHI_MITIGATION_OFF,
 	BHI_MITIGATION_AUTO,
-	BHI_MITIGATION_ON,
-	BHI_MITIGATION_VMEXIT_ONLY,
+	BHI_MITIGATION_FULL,
+	BHI_MITIGATION_VMEXIT,
+	BHI_MITIGATION_SYSCALL
 };
 
 static enum bhi_mitigations bhi_mitigation __ro_after_init =
@@ -1904,9 +1905,9 @@ static int __init spectre_bhi_parse_cmdline(char *str)
 	if (!strcmp(str, "off"))
 		bhi_mitigation = BHI_MITIGATION_OFF;
 	else if (!strcmp(str, "on"))
-		bhi_mitigation = BHI_MITIGATION_ON;
+		bhi_mitigation = BHI_MITIGATION_FULL;
 	else if (!strcmp(str, "vmexit"))
-		bhi_mitigation = BHI_MITIGATION_VMEXIT_ONLY;
+		bhi_mitigation = BHI_MITIGATION_VMEXIT;
 	else
 		pr_err("Ignoring unknown spectre_bhi option (%s)", str);
 
@@ -1916,11 +1917,20 @@ early_param("spectre_bhi", spectre_bhi_parse_cmdline);
 
 static void __init bhi_select_mitigation(void)
 {
-	if (!boot_cpu_has(X86_BUG_BHI) || cpu_mitigations_off())
+	if (!boot_cpu_has(X86_BUG_BHI))
 		bhi_mitigation = BHI_MITIGATION_OFF;
 
-	if (bhi_mitigation == BHI_MITIGATION_AUTO)
-		bhi_mitigation = BHI_MITIGATION_ON;
+	if (bhi_mitigation == BHI_MITIGATION_AUTO) {
+		if (cpu_mitigate_attack_vector(CPU_MITIGATE_USER_KERNEL)) {
+			if (cpu_mitigate_attack_vector(CPU_MITIGATE_GUEST_HOST))
+				bhi_mitigation = BHI_MITIGATION_FULL;
+			else
+				bhi_mitigation = BHI_MITIGATION_SYSCALL;
+		} else if (cpu_mitigate_attack_vector(CPU_MITIGATE_GUEST_HOST))
+			bhi_mitigation = BHI_MITIGATION_VMEXIT;
+		else
+			bhi_mitigation = BHI_MITIGATION_OFF;
+	}
 }
 
 static void __init bhi_apply_mitigation(void)
@@ -1943,15 +1953,19 @@ static void __init bhi_apply_mitigation(void)
 	if (!IS_ENABLED(CONFIG_X86_64))
 		return;
 
-	if (bhi_mitigation == BHI_MITIGATION_VMEXIT_ONLY) {
-		pr_info("Spectre BHI mitigation: SW BHB clearing on VM exit only\n");
+	/* Mitigate KVM if guest->host protection is desired */
+	if (bhi_mitigation == BHI_MITIGATION_FULL ||
+	    bhi_mitigation == BHI_MITIGATION_VMEXIT) {
 		setup_force_cpu_cap(X86_FEATURE_CLEAR_BHB_LOOP_ON_VMEXIT);
-		return;
+		pr_info("Spectre BHI mitigation: SW BHB clearing on VM exit\n");
 	}
 
-	pr_info("Spectre BHI mitigation: SW BHB clearing on syscall and VM exit\n");
-	setup_force_cpu_cap(X86_FEATURE_CLEAR_BHB_LOOP);
-	setup_force_cpu_cap(X86_FEATURE_CLEAR_BHB_LOOP_ON_VMEXIT);
+	/* Mitigate syscalls if user->kernel protection is desired */
+	if (bhi_mitigation == BHI_MITIGATION_FULL ||
+	    bhi_mitigation == BHI_MITIGATION_SYSCALL) {
+		setup_force_cpu_cap(X86_FEATURE_CLEAR_BHB_LOOP);
+		pr_info("Spectre BHI mitigation: SW BHB clearing on syscall\n");
+	}
 }
 
 static void __init spectre_v2_select_mitigation(void)
-- 
2.34.1

[PATCH v4 31/36] x86/bugs: Add attack vector controls for spectre_v2

[David Kaplan]

Use attack vector controls to determine if spectre_v2 mitigation is
required.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 4912367f2334..e35f7059f6fc 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -1715,8 +1715,7 @@ static enum spectre_v2_mitigation_cmd __init spectre_v2_parse_cmdline(void)
 	int ret, i;
 
 	cmd = IS_ENABLED(CONFIG_MITIGATION_SPECTRE_V2) ?  SPECTRE_V2_CMD_AUTO : SPECTRE_V2_CMD_NONE;
-	if (cmdline_find_option_bool(boot_command_line, "nospectre_v2") ||
-	    cpu_mitigations_off())
+	if (cmdline_find_option_bool(boot_command_line, "nospectre_v2"))
 		return SPECTRE_V2_CMD_NONE;
 
 	ret = cmdline_find_option(boot_command_line, "spectre_v2", arg, sizeof(arg));
@@ -1985,8 +1984,11 @@ static void __init spectre_v2_select_mitigation(void)
 	case SPECTRE_V2_CMD_NONE:
 		return;
 
-	case SPECTRE_V2_CMD_FORCE:
 	case SPECTRE_V2_CMD_AUTO:
+		if (!should_mitigate_vuln(X86_BUG_SPECTRE_V2))
+			break;
+		fallthrough;
+	case SPECTRE_V2_CMD_FORCE:
 		if (boot_cpu_has(X86_FEATURE_IBRS_ENHANCED)) {
 			mode = SPECTRE_V2_EIBRS;
 			break;
@@ -2041,7 +2043,7 @@ static void __init spectre_v2_update_mitigation(void)
 		}
 	}
 
-	if (boot_cpu_has_bug(X86_BUG_SPECTRE_V2) && !cpu_mitigations_off())
+	if (boot_cpu_has_bug(X86_BUG_SPECTRE_V2))
 		pr_info("%s\n", spectre_v2_strings[spectre_v2_enabled]);
 }
 
-- 
2.34.1

[PATCH v4 32/36] x86/bugs: Add attack vector controls for l1tf

[David Kaplan]

Use attack vector controls to determine if l1tf mitigation is required.

Disable SMT if cross-thread attack vector option is selected.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index e35f7059f6fc..e38529f3022b 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -2717,16 +2717,20 @@ static void override_cache_bits(struct cpuinfo_x86 *c)
 
 static void __init l1tf_select_mitigation(void)
 {
-	if (!boot_cpu_has_bug(X86_BUG_L1TF) || cpu_mitigations_off()) {
+	if (!boot_cpu_has_bug(X86_BUG_L1TF)) {
 		l1tf_mitigation = L1TF_MITIGATION_OFF;
 		return;
 	}
 
 	if (l1tf_mitigation == L1TF_MITIGATION_AUTO) {
-		if (cpu_mitigations_auto_nosmt())
-			l1tf_mitigation = L1TF_MITIGATION_FLUSH_NOSMT;
-		else
-			l1tf_mitigation = L1TF_MITIGATION_FLUSH;
+		if (!should_mitigate_vuln(X86_BUG_L1TF))
+			l1tf_mitigation = L1TF_MITIGATION_OFF;
+		else {
+			if (smt_mitigations == SMT_MITIGATIONS_ON)
+				l1tf_mitigation = L1TF_MITIGATION_FLUSH_NOSMT;
+			else
+				l1tf_mitigation = L1TF_MITIGATION_FLUSH;
+		}
 	}
 }
 
-- 
2.34.1

[PATCH v4 33/36] x86/bugs: Add attack vector controls for srso

[David Kaplan]

Use attack vector controls to determine if srso mitigation is required.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index e38529f3022b..4e1fc1468870 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -2858,14 +2858,19 @@ static void __init srso_select_mitigation(void)
 {
 	bool has_microcode = boot_cpu_has(X86_FEATURE_IBPB_BRTYPE);
 
-	if (!boot_cpu_has_bug(X86_BUG_SRSO) || cpu_mitigations_off())
+	if (!boot_cpu_has_bug(X86_BUG_SRSO)) {
 		srso_mitigation = SRSO_MITIGATION_NONE;
-
-	if (srso_mitigation == SRSO_MITIGATION_NONE)
 		return;
+	}
 
-	if (srso_mitigation == SRSO_MITIGATION_AUTO)
-		srso_mitigation = SRSO_MITIGATION_SAFE_RET;
+	if (srso_mitigation == SRSO_MITIGATION_AUTO) {
+		if (should_mitigate_vuln(X86_BUG_SRSO))
+			srso_mitigation = SRSO_MITIGATION_SAFE_RET;
+		else {
+			srso_mitigation = SRSO_MITIGATION_NONE;
+			return;
+		}
+	}
 
 	if (has_microcode) {
 		/*
@@ -2921,7 +2926,7 @@ static void __init srso_update_mitigation(void)
 	    boot_cpu_has(X86_FEATURE_IBPB_BRTYPE))
 		srso_mitigation = SRSO_MITIGATION_IBPB;
 
-	if (boot_cpu_has_bug(X86_BUG_SRSO) && !cpu_mitigations_off())
+	if (boot_cpu_has_bug(X86_BUG_SRSO))
 		pr_info("%s\n", srso_strings[srso_mitigation]);
 }
 
-- 
2.34.1

[PATCH v4 34/36] x86/pti: Add attack vector controls for pti

[David Kaplan]

Disable PTI mitigation if user->kernel attack vector mitigations are
disabled.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/mm/pti.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/x86/mm/pti.c b/arch/x86/mm/pti.c
index 5f0d579932c6..e5b96233aa58 100644
--- a/arch/x86/mm/pti.c
+++ b/arch/x86/mm/pti.c
@@ -38,6 +38,7 @@
 #include <asm/desc.h>
 #include <asm/sections.h>
 #include <asm/set_memory.h>
+#include <asm/bugs.h>
 
 #undef pr_fmt
 #define pr_fmt(fmt)     "Kernel/User page tables isolation: " fmt
@@ -84,7 +85,8 @@ void __init pti_check_boottime_disable(void)
 		return;
 	}
 
-	if (cpu_mitigations_off())
+	if (pti_mode == PTI_AUTO &&
+	    !cpu_mitigate_attack_vector(CPU_MITIGATE_USER_KERNEL))
 		pti_mode = PTI_FORCE_OFF;
 	if (pti_mode == PTI_FORCE_OFF) {
 		pti_print_if_insecure("disabled on command line.");
-- 
2.34.1

[PATCH v4 35/36] x86/bugs: Print enabled attack vectors

[David Kaplan]

Print the status of enabled attack vectors and SMT mitigation status in the
boot log for easier reporting and debugging.  This information will also be
available through sysfs.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/kernel/cpu/bugs.c | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 4e1fc1468870..7ee9b8bf05e5 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -173,6 +173,34 @@ DEFINE_STATIC_KEY_FALSE(switch_mm_cond_l1d_flush);
 DEFINE_STATIC_KEY_FALSE(mmio_stale_data_clear);
 EXPORT_SYMBOL_GPL(mmio_stale_data_clear);
 
+#undef pr_fmt
+#define pr_fmt(fmt)	"mitigations: " fmt
+
+static void __init cpu_print_attack_vectors(void)
+{
+	pr_info("Enabled attack vectors: ");
+	if (cpu_mitigate_attack_vector(CPU_MITIGATE_USER_KERNEL))
+		pr_cont("user_kernel, ");
+	if (cpu_mitigate_attack_vector(CPU_MITIGATE_USER_USER))
+		pr_cont("user_user, ");
+	if (cpu_mitigate_attack_vector(CPU_MITIGATE_GUEST_HOST))
+		pr_cont("guest_host, ");
+	if (cpu_mitigate_attack_vector(CPU_MITIGATE_GUEST_GUEST))
+		pr_cont("guest_guest, ");
+
+	pr_cont("SMT mitigations: ");
+	switch (smt_mitigations) {
+	case SMT_MITIGATIONS_OFF:
+		pr_cont("off\n");
+		break;
+	case SMT_MITIGATIONS_AUTO:
+		pr_cont("auto\n");
+		break;
+	case SMT_MITIGATIONS_ON:
+		pr_cont("on\n");
+	}
+}
+
 void __init cpu_select_mitigations(void)
 {
 	/*
@@ -193,6 +221,8 @@ void __init cpu_select_mitigations(void)
 
 	x86_arch_cap_msr = x86_read_arch_cap_msr();
 
+	cpu_print_attack_vectors();
+
 	/* Select the proper CPU mitigations before patching alternatives: */
 	spectre_v1_select_mitigation();
 	spectre_v2_select_mitigation();
-- 
2.34.1

[PATCH v4 36/36] cpu: Show attack vectors in sysfs

[David Kaplan]

Show the status of currently mitigated attack vectors in
/sys/devices/system/cpu/vector_mitigations/

Note that these files are not under the vulnerabilities directory so they
will not be printed by 'lscpu'.

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 drivers/base/cpu.c | 67 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 67 insertions(+)

diff --git a/drivers/base/cpu.c b/drivers/base/cpu.c
index a7e511849875..e9bb9c3edf91 100644
--- a/drivers/base/cpu.c
+++ b/drivers/base/cpu.c
@@ -20,6 +20,7 @@
 #include <linux/tick.h>
 #include <linux/pm_qos.h>
 #include <linux/delay.h>
+#include <linux/string_choices.h>
 #include <linux/sched/isolation.h>
 
 #include "base.h"
@@ -641,6 +642,70 @@ static const struct attribute_group cpu_root_vulnerabilities_group = {
 	.attrs = cpu_root_vulnerabilities_attrs,
 };
 
+static const char *attack_vector_state(enum cpu_attack_vectors v)
+{
+	return str_on_off(cpu_mitigate_attack_vector(v));
+}
+
+static ssize_t cpu_show_user_kernel_vector(struct device *dev,
+				struct device_attribute *attr, char *buf)
+{
+	return sysfs_emit(buf, "%s\n",  attack_vector_state(CPU_MITIGATE_USER_KERNEL));
+}
+
+static ssize_t cpu_show_user_user_vector(struct device *dev,
+				struct device_attribute *attr, char *buf)
+{
+	return sysfs_emit(buf, "%s\n", attack_vector_state(CPU_MITIGATE_USER_USER));
+}
+
+static ssize_t cpu_show_guest_host_vector(struct device *dev,
+				struct device_attribute *attr, char *buf)
+{
+	return sysfs_emit(buf, "%s\n", attack_vector_state(CPU_MITIGATE_GUEST_HOST));
+}
+
+static ssize_t cpu_show_guest_guest_vector(struct device *dev,
+				struct device_attribute *attr, char *buf)
+{
+	return sysfs_emit(buf, "%s\n", attack_vector_state(CPU_MITIGATE_GUEST_GUEST));
+}
+
+static ssize_t cpu_show_smt_vector(struct device *dev,
+				struct device_attribute *attr, char *buf)
+{
+	switch (smt_mitigations) {
+	case SMT_MITIGATIONS_OFF:
+		return sysfs_emit(buf, "off\n");
+	case SMT_MITIGATIONS_ON:
+		return sysfs_emit(buf, "on\n");
+	case SMT_MITIGATIONS_AUTO:
+		return sysfs_emit(buf, "auto\n");
+	}
+
+	return 0;
+}
+
+static DEVICE_ATTR(user_kernel, 0444, cpu_show_user_kernel_vector, NULL);
+static DEVICE_ATTR(user_user, 0444, cpu_show_user_user_vector, NULL);
+static DEVICE_ATTR(guest_host, 0444, cpu_show_guest_host_vector, NULL);
+static DEVICE_ATTR(guest_guest, 0444, cpu_show_guest_guest_vector, NULL);
+static DEVICE_ATTR(smt, 0444, cpu_show_smt_vector, NULL);
+
+static struct attribute *cpu_vector_mitigations_attrs[] = {
+	&dev_attr_user_kernel.attr,
+	&dev_attr_user_user.attr,
+	&dev_attr_guest_host.attr,
+	&dev_attr_guest_guest.attr,
+	&dev_attr_smt.attr,
+	NULL
+};
+
+static const struct attribute_group cpu_vector_mitigations_group = {
+	.name  = "vector_mitigations",
+	.attrs = cpu_vector_mitigations_attrs,
+};
+
 static void __init cpu_register_vulnerabilities(void)
 {
 	struct device *dev = bus_get_dev_root(&cpu_subsys);
@@ -648,6 +713,8 @@ static void __init cpu_register_vulnerabilities(void)
 	if (dev) {
 		if (sysfs_create_group(&dev->kobj, &cpu_root_vulnerabilities_group))
 			pr_err("Unable to register CPU vulnerabilities\n");
+		if (sysfs_create_group(&dev->kobj, &cpu_vector_mitigations_group))
+			pr_err("Unable to register CPU attack vectors\n");
 		put_device(dev);
 	}
 }
-- 
2.34.1

Re: [PATCH v4 00/36] Attack vector controls

[Ingo Molnar]

* David Kaplan <david.kaplan@amd.com> wrote:

> David Kaplan (36):
>   x86/bugs: Restructure mds mitigation
>   x86/bugs: Restructure taa mitigation
>   x86/bugs: Restructure mmio mitigation
>   x86/bugs: Restructure rfds mitigation
>   x86/bugs: Remove md_clear_*_mitigation()
>   x86/bugs: Restructure srbds mitigation
>   x86/bugs: Restructure gds mitigation
>   x86/bugs: Restructure spectre_v1 mitigation
>   x86/bugs: Only allow retbleed=stuff on Intel
>   x86/bugs: Restructure retbleed mitigation
>   x86/bugs: Restructure spectre_v2_user mitigation
>   x86/bugs: Restructure bhi mitigation
>   x86/bugs: Restructure spectre_v2 mitigation
>   x86/bugs: Restructure ssb mitigation
>   x86/bugs: Restructure l1tf mitigation
>   x86/bugs: Restructure srso mitigation
>   Documentation/x86: Document the new attack vector controls
>   cpu: Define attack vectors
>   x86/Kconfig: Arch attack vector support
>   x86/bugs: Determine relevant vulnerabilities based on attack vector
>     controls.
>   x86/bugs: Add attack vector controls for mds
>   x86/bugs: Add attack vector controls for taa
>   x86/bugs: Add attack vector controls for mmio
>   x86/bugs: Add attack vector controls for rfds
>   x86/bugs: Add attack vector controls for srbds
>   x86/bugs: Add attack vector controls for gds
>   x86/bugs: Add attack vector controls for spectre_v1
>   x86/bugs: Add attack vector controls for retbleed
>   x86/bugs: Add attack vector controls for spectre_v2_user
>   x86/bugs: Add attack vector controls for bhi
>   x86/bugs: Add attack vector controls for spectre_v2
>   x86/bugs: Add attack vector controls for l1tf
>   x86/bugs: Add attack vector controls for srso
>   x86/pti: Add attack vector controls for pti
>   x86/bugs: Print enabled attack vectors
>   cpu: Show attack vectors in sysfs

Just an overall meta comment: could you please capitalize all the 
acronyms properly and consistently in titles, changelogs and the code 
itself:

  s/mds
   /MDS

  s/srso
   /SRSO

etc. For single patches we maintainers will routinely fix such issues, 
but for a 36-patch series it's rather tedious ...

Also, please put no periods into titles, and titles should begin with a 
verb.

Thanks,

	Ingo

RE: [PATCH v4 00/36] Attack vector controls

[Kaplan, David]

[AMD Official Use Only - AMD Internal Distribution Only]

> -----Original Message-----
> From: Ingo Molnar <mingo@kernel.org>
> Sent: Monday, March 10, 2025 1:45 PM
> To: Kaplan, David <David.Kaplan@amd.com>
> Cc: Thomas Gleixner <tglx@linutronix.de>; Borislav Petkov <bp@alien8.de>; Peter
> Zijlstra <peterz@infradead.org>; Josh Poimboeuf <jpoimboe@kernel.org>; Pawan
> Gupta <pawan.kumar.gupta@linux.intel.com>; Ingo Molnar <mingo@redhat.com>;
> Dave Hansen <dave.hansen@linux.intel.com>; x86@kernel.org; H . Peter Anvin
> <hpa@zytor.com>; linux-kernel@vger.kernel.org; Brendan Jackman
> <jackmanb@google.com>; Derek Manwaring <derekmn@amazon.com>
> Subject: Re: [PATCH v4 00/36] Attack vector controls
>
> Caution: This message originated from an External Source. Use proper caution
> when opening attachments, clicking links, or responding.
>
>
> * David Kaplan <david.kaplan@amd.com> wrote:
>
> > David Kaplan (36):
> >   x86/bugs: Restructure mds mitigation
> >   x86/bugs: Restructure taa mitigation
> >   x86/bugs: Restructure mmio mitigation
> >   x86/bugs: Restructure rfds mitigation
> >   x86/bugs: Remove md_clear_*_mitigation()
> >   x86/bugs: Restructure srbds mitigation
> >   x86/bugs: Restructure gds mitigation
> >   x86/bugs: Restructure spectre_v1 mitigation
> >   x86/bugs: Only allow retbleed=stuff on Intel
> >   x86/bugs: Restructure retbleed mitigation
> >   x86/bugs: Restructure spectre_v2_user mitigation
> >   x86/bugs: Restructure bhi mitigation
> >   x86/bugs: Restructure spectre_v2 mitigation
> >   x86/bugs: Restructure ssb mitigation
> >   x86/bugs: Restructure l1tf mitigation
> >   x86/bugs: Restructure srso mitigation
> >   Documentation/x86: Document the new attack vector controls
> >   cpu: Define attack vectors
> >   x86/Kconfig: Arch attack vector support
> >   x86/bugs: Determine relevant vulnerabilities based on attack vector
> >     controls.
> >   x86/bugs: Add attack vector controls for mds
> >   x86/bugs: Add attack vector controls for taa
> >   x86/bugs: Add attack vector controls for mmio
> >   x86/bugs: Add attack vector controls for rfds
> >   x86/bugs: Add attack vector controls for srbds
> >   x86/bugs: Add attack vector controls for gds
> >   x86/bugs: Add attack vector controls for spectre_v1
> >   x86/bugs: Add attack vector controls for retbleed
> >   x86/bugs: Add attack vector controls for spectre_v2_user
> >   x86/bugs: Add attack vector controls for bhi
> >   x86/bugs: Add attack vector controls for spectre_v2
> >   x86/bugs: Add attack vector controls for l1tf
> >   x86/bugs: Add attack vector controls for srso
> >   x86/pti: Add attack vector controls for pti
> >   x86/bugs: Print enabled attack vectors
> >   cpu: Show attack vectors in sysfs
>
> Just an overall meta comment: could you please capitalize all the acronyms properly
> and consistently in titles, changelogs and the code
> itself:
>
>   s/mds
>    /MDS
>
>   s/srso
>    /SRSO
>
> etc. For single patches we maintainers will routinely fix such issues, but for a 36-
> patch series it's rather tedious ...
>
> Also, please put no periods into titles, and titles should begin with a verb.
>

Ack, will fix.

Thanks --David Kaplan

Re: [PATCH v4 11/36] x86/bugs: Restructure spectre_v2_user mitigation

[kernel test robot]

Hi David,

kernel test robot noticed the following build warnings:

[auto build test WARNING on tip/master]
[cannot apply to tip/x86/core linus/master tip/auto-latest tip/smp/core v6.14-rc6 next-20250311]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/David-Kaplan/x86-bugs-Restructure-mds-mitigation/20250311-005151
base:   tip/master
patch link:    https://lore.kernel.org/r/20250310164023.779191-12-david.kaplan%40amd.com
patch subject: [PATCH v4 11/36] x86/bugs: Restructure spectre_v2_user mitigation
config: x86_64-allnoconfig (https://download.01.org/0day-ci/archive/20250312/202503121721.1nslvluh-lkp@intel.com/config)
compiler: clang version 19.1.7 (https://github.com/llvm/llvm-project cd708029e0b2869e80abe31ddb175f7c35361f90)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250312/202503121721.1nslvluh-lkp@intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202503121721.1nslvluh-lkp@intel.com/

All warnings (new ones prefixed by >>):

>> arch/x86/kernel/cpu/bugs.c:1490:7: warning: variable 'smt_possible' set but not used [-Wunused-but-set-variable]
    1490 |         bool smt_possible = IS_ENABLED(CONFIG_SMP);
         |              ^
   1 warning generated.


vim +/smt_possible +1490 arch/x86/kernel/cpu/bugs.c

  1487	
  1488	static void __init spectre_v2_user_update_mitigation(void)
  1489	{
> 1490		bool smt_possible = IS_ENABLED(CONFIG_SMP);
  1491	
  1492		if (!boot_cpu_has(X86_FEATURE_IBPB) && !boot_cpu_has(X86_FEATURE_STIBP))
  1493			return;
  1494	
  1495		if (cpu_smt_control == CPU_SMT_FORCE_DISABLED ||
  1496		    cpu_smt_control == CPU_SMT_NOT_SUPPORTED)
  1497			smt_possible = false;
  1498	
  1499		/* The spectre_v2 cmd line can override spectre_v2_user options */
  1500		if (spectre_v2_cmd == SPECTRE_V2_CMD_NONE) {
  1501			spectre_v2_user_ibpb = SPECTRE_V2_USER_NONE;
  1502			spectre_v2_user_stibp = SPECTRE_V2_USER_NONE;
  1503		} else if (spectre_v2_cmd == SPECTRE_V2_CMD_FORCE) {
  1504			spectre_v2_user_ibpb = SPECTRE_V2_USER_STRICT;
  1505			spectre_v2_user_stibp = SPECTRE_V2_USER_STRICT;
  1506		}
  1507	
  1508		/*
  1509		 * If no STIBP, Intel enhanced IBRS is enabled, or SMT impossible, STIBP
  1510		 * is not required.
  1511		 *
  1512		 * Intel's Enhanced IBRS also protects against cross-thread branch target
  1513		 * injection in user-mode as the IBRS bit remains always set which
  1514		 * implicitly enables cross-thread protections.  However, in legacy IBRS
  1515		 * mode, the IBRS bit is set only on kernel entry and cleared on return
  1516		 * to userspace.  AMD Automatic IBRS also does not protect userspace.
  1517		 * These modes therefore disable the implicit cross-thread protection,
  1518		 * so allow for STIBP to be selected in those cases.
  1519		 */
  1520		if (!boot_cpu_has(X86_FEATURE_STIBP) ||
  1521		    !cpu_smt_possible() ||
  1522		    (spectre_v2_in_eibrs_mode(spectre_v2_enabled) &&
  1523		     !boot_cpu_has(X86_FEATURE_AUTOIBRS))) {
  1524			spectre_v2_user_stibp = SPECTRE_V2_USER_NONE;
  1525			return;
  1526		}
  1527	
  1528		if (spectre_v2_user_stibp != SPECTRE_V2_USER_NONE &&
  1529		    (retbleed_mitigation == RETBLEED_MITIGATION_UNRET ||
  1530		     retbleed_mitigation == RETBLEED_MITIGATION_IBPB)) {
  1531			if (spectre_v2_user_stibp != SPECTRE_V2_USER_STRICT &&
  1532			    spectre_v2_user_stibp != SPECTRE_V2_USER_STRICT_PREFERRED)
  1533				pr_info("Selecting STIBP always-on mode to complement retbleed mitigation\n");
  1534			spectre_v2_user_stibp = SPECTRE_V2_USER_STRICT_PREFERRED;
  1535		}
  1536		pr_info("%s\n", spectre_v2_user_strings[spectre_v2_user_stibp]);
  1537	}
  1538	

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

Re: [PATCH v4 20/36] x86/bugs: Determine relevant vulnerabilities based on attack vector controls.

[kernel test robot]

Hi David,

kernel test robot noticed the following build errors:

[auto build test ERROR on tip/master]
[cannot apply to tip/x86/core linus/master tip/auto-latest tip/smp/core v6.14-rc6 next-20250311]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/David-Kaplan/x86-bugs-Restructure-mds-mitigation/20250311-005151
base:   tip/master
patch link:    https://lore.kernel.org/r/20250310164023.779191-21-david.kaplan%40amd.com
patch subject: [PATCH v4 20/36] x86/bugs: Determine relevant vulnerabilities based on attack vector controls.
config: x86_64-allnoconfig (https://download.01.org/0day-ci/archive/20250312/202503121906.IF0k61bY-lkp@intel.com/config)
compiler: clang version 19.1.7 (https://github.com/llvm/llvm-project cd708029e0b2869e80abe31ddb175f7c35361f90)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250312/202503121906.IF0k61bY-lkp@intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202503121906.IF0k61bY-lkp@intel.com/

All errors (new ones prefixed by >>):

>> arch/x86/kernel/cpu/bugs.c:351:11: error: use of undeclared identifier 'smt_mitigations'; did you mean 'l1tf_mitigation'?
     351 |                        (smt_mitigations != SMT_MITIGATIONS_OFF);
         |                         ^~~~~~~~~~~~~~~
         |                         l1tf_mitigation
   arch/x86/include/asm/processor.h:745:30: note: 'l1tf_mitigation' declared here
     745 | extern enum l1tf_mitigations l1tf_mitigation;
         |                              ^
   arch/x86/kernel/cpu/bugs.c:1561:7: warning: variable 'smt_possible' set but not used [-Wunused-but-set-variable]
    1561 |         bool smt_possible = IS_ENABLED(CONFIG_SMP);
         |              ^
   1 warning and 1 error generated.


vim +351 arch/x86/kernel/cpu/bugs.c

   302	
   303	/*
   304	 * Returns true if vulnerability should be mitigated based on the
   305	 * selected attack vector controls.
   306	 *
   307	 * See Documentation/admin-guide/hw-vuln/attack_vector_controls.rst
   308	 */
   309	static bool __init should_mitigate_vuln(unsigned int bug)
   310	{
   311		switch (bug) {
   312		/*
   313		 * The only runtime-selected spectre_v1 mitigations in the kernel are
   314		 * related to SWAPGS protection on kernel entry.  Therefore, protection
   315		 * is only required for the user->kernel attack vector.
   316		 */
   317		case X86_BUG_SPECTRE_V1:
   318			return cpu_mitigate_attack_vector(CPU_MITIGATE_USER_KERNEL);
   319	
   320		case X86_BUG_SPECTRE_V2:
   321		case X86_BUG_RETBLEED:
   322		case X86_BUG_SRSO:
   323		case X86_BUG_L1TF:
   324			return cpu_mitigate_attack_vector(CPU_MITIGATE_USER_KERNEL) ||
   325			       cpu_mitigate_attack_vector(CPU_MITIGATE_GUEST_HOST);
   326	
   327		case X86_BUG_SPECTRE_V2_USER:
   328			return cpu_mitigate_attack_vector(CPU_MITIGATE_USER_USER) ||
   329			       cpu_mitigate_attack_vector(CPU_MITIGATE_GUEST_GUEST);
   330	
   331		/*
   332		 * All the vulnerabilities below allow potentially leaking data
   333		 * across address spaces.  Therefore, mitigation is required for
   334		 * any of these 4 attack vectors.
   335		 */
   336		case X86_BUG_MDS:
   337		case X86_BUG_TAA:
   338		case X86_BUG_MMIO_STALE_DATA:
   339		case X86_BUG_RFDS:
   340		case X86_BUG_SRBDS:
   341			return cpu_mitigate_attack_vector(CPU_MITIGATE_USER_KERNEL) ||
   342			       cpu_mitigate_attack_vector(CPU_MITIGATE_GUEST_HOST) ||
   343			       cpu_mitigate_attack_vector(CPU_MITIGATE_USER_USER) ||
   344			       cpu_mitigate_attack_vector(CPU_MITIGATE_GUEST_GUEST);
   345	
   346		case X86_BUG_GDS:
   347			return cpu_mitigate_attack_vector(CPU_MITIGATE_USER_KERNEL) ||
   348			       cpu_mitigate_attack_vector(CPU_MITIGATE_GUEST_HOST) ||
   349			       cpu_mitigate_attack_vector(CPU_MITIGATE_USER_USER) ||
   350			       cpu_mitigate_attack_vector(CPU_MITIGATE_GUEST_GUEST) ||
 > 351			       (smt_mitigations != SMT_MITIGATIONS_OFF);
   352		default:
   353			WARN(1, "Unknown bug %x\n", bug);
   354			return false;
   355		}
   356	}
   357	

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

Re: [PATCH v4 03/36] x86/bugs: Restructure mmio mitigation

[Borislav Petkov]

On Mon, Mar 10, 2025 at 11:39:50AM -0500, David Kaplan wrote:
> @@ -511,24 +516,60 @@ static void __init mmio_select_mitigation(void)
>  		return;
>  	}
>  
> -	if (mmio_mitigation == MMIO_MITIGATION_OFF)
> -		return;
> +	/* Microcode will be checked in mmio_update_mitigation(). */
> +	if (mmio_mitigation == MMIO_MITIGATION_AUTO)
> +		mmio_mitigation = MMIO_MITIGATION_VERW;
>  
>  	/*
>  	 * Enable CPU buffer clear mitigation for host and VMM, if also affected
> -	 * by MDS or TAA. Otherwise, enable mitigation for VMM only.
> +	 * by MDS or TAA.
>  	 */
> -	if (boot_cpu_has_bug(X86_BUG_MDS) || (boot_cpu_has_bug(X86_BUG_TAA) &&
> -					      boot_cpu_has(X86_FEATURE_RTM)))
> -		setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF);
> +	if (boot_cpu_has_bug(X86_BUG_MDS) || taa_vulnerable())
> +		verw_mitigation_selected = true;
> +}

So applied this reads strange:

        if (mmio_mitigation == MMIO_MITIGATION_AUTO)
                mmio_mitigation = MMIO_MITIGATION_VERW;

        if (boot_cpu_has_bug(X86_BUG_MDS) || taa_vulnerable())
                verw_mitigation_selected = true;

I'd expect to see:

	if (mmio_mitigation == MMIO_MITIGATION_AUTO) {
                mmio_mitigation = MMIO_MITIGATION_VERW;
		verw_mitigation_selected = true;
	}

        if (boot_cpu_has_bug(X86_BUG_MDS) || taa_vulnerable())
                verw_mitigation_selected = true;

because the above branch already selected MMIO_MITIGATION_VERW so we might as
well set verw_mitigation_selected, right?

> +static void __init mmio_update_mitigation(void)
> +{
> +	if (!boot_cpu_has_bug(X86_BUG_MMIO_STALE_DATA) || cpu_mitigations_off())
> +		return;
> +
> +	if (verw_mitigation_selected)
> +		mmio_mitigation = MMIO_MITIGATION_VERW;

... and the above change would obviate this one.

Looking at that verw_mitigation_selected switch - that seems like the higher
prio thing we should be looking at first as in: *something* selected VERW
mitigation so we must honor it.

And then the *_select_mitigation() functions will simply use the respective
*_mitigation variable to perhaps override it only when really needed.

I think.

Or maybe I'm missing an aspect.

Because if we make verw_mitigation_selected the higher prio thing, we can
remove some of that additional checking.

Or?

> +
> +	if (mmio_mitigation == MMIO_MITIGATION_VERW) {

I.e., in mmio_update_mitigation(), the only check you need to do is:

	if (verw_mitigation_selected)

and then adjust mmio_mitigation depending on microcode presence or not.

> +		/*
> +		 * Check if the system has the right microcode.
> +		 *
> +		 * CPU Fill buffer clear mitigation is enumerated by either an explicit
> +		 * FB_CLEAR or by the presence of both MD_CLEAR and L1D_FLUSH on MDS
> +		 * affected systems.
> +		 */
> +		if (!((x86_arch_cap_msr & ARCH_CAP_FB_CLEAR) ||
> +		      (boot_cpu_has(X86_FEATURE_MD_CLEAR) &&
> +		       boot_cpu_has(X86_FEATURE_FLUSH_L1D) &&
> +		     !(x86_arch_cap_msr & ARCH_CAP_MDS_NO))))
> +			mmio_mitigation = MMIO_MITIGATION_UCODE_NEEDED;

... as you do here.

> +	}
> +
> +	if (boot_cpu_has_bug(X86_BUG_MMIO_UNKNOWN))

Btw, that UNKNOWN thing is just silly. Looking at git history:

7df548840c49 ("x86/bugs: Add "unknown" reporting for MMIO Stale Data")

this was added just so that it doesn't say "Not affected" about those CPUs but
"unknown."

But

  "Mitigation is not deployed when the status is unknown."

so if it is only about reporting, I think we can synthesize the logic of this:

        if (!arch_cap_mmio_immune(x86_arch_cap_msr)) {
                if (cpu_matches(cpu_vuln_blacklist, MMIO))
                        setup_force_cpu_bug(X86_BUG_MMIO_STALE_DATA);
                else if (!cpu_matches(cpu_vuln_whitelist, NO_MMIO))
                        setup_force_cpu_bug(X86_BUG_MMIO_UNKNOWN);
        }

into a separate function and get rid of that X86_BUG_MMIO_UNKNOWN thing.

Pawan?

I'll try to whack it later to see how ugly it gets.

> +		pr_info("Unknown: No mitigations\n");
> +	else
> +		pr_info("%s\n", mmio_strings[mmio_mitigation]);
> +}
> +
> +static void __init mmio_apply_mitigation(void)
> +{
> +	if (mmio_mitigation == MMIO_MITIGATION_OFF)
> +		return;
>  
>  	/*
> -	 * X86_FEATURE_CLEAR_CPU_BUF could be enabled by other VERW based
> -	 * mitigations, disable KVM-only mitigation in that case.
> +	 * Only enable the VMM mitigation if the CPU buffer clear mitigation is
> +	 * not being used.

So this comment doesn't fit with what the code now does...

>  	 */
> -	if (boot_cpu_has(X86_FEATURE_CLEAR_CPU_BUF))
> +	if (verw_mitigation_selected) {

... which is to check whether something enabled the VERW mitigation...

> +		setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF);
>  		static_branch_disable(&mmio_stale_data_clear);
> -	else
> +	} else
>  		static_branch_enable(&mmio_stale_data_clear);

{ } around the else branch too pls.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

RE: [PATCH v4 03/36] x86/bugs: Restructure mmio mitigation

[Kaplan, David]

[AMD Official Use Only - AMD Internal Distribution Only]

> -----Original Message-----
> From: Borislav Petkov <bp@alien8.de>
> Sent: Thursday, March 13, 2025 4:36 AM
> To: Kaplan, David <David.Kaplan@amd.com>; Pawan Gupta
> <pawan.kumar.gupta@linux.intel.com>
> Cc: Thomas Gleixner <tglx@linutronix.de>; Peter Zijlstra <peterz@infradead.org>;
> Josh Poimboeuf <jpoimboe@kernel.org>; Ingo Molnar <mingo@redhat.com>; Dave
> Hansen <dave.hansen@linux.intel.com>; x86@kernel.org; H . Peter Anvin
> <hpa@zytor.com>; linux-kernel@vger.kernel.org; Brendan Jackman
> <jackmanb@google.com>; Derek Manwaring <derekmn@amazon.com>
> Subject: Re: [PATCH v4 03/36] x86/bugs: Restructure mmio mitigation
>
> Caution: This message originated from an External Source. Use proper caution
> when opening attachments, clicking links, or responding.
>
>
> On Mon, Mar 10, 2025 at 11:39:50AM -0500, David Kaplan wrote:
> > @@ -511,24 +516,60 @@ static void __init mmio_select_mitigation(void)
> >               return;
> >       }
> >
> > -     if (mmio_mitigation == MMIO_MITIGATION_OFF)
> > -             return;
> > +     /* Microcode will be checked in mmio_update_mitigation(). */
> > +     if (mmio_mitigation == MMIO_MITIGATION_AUTO)
> > +             mmio_mitigation = MMIO_MITIGATION_VERW;
> >
> >       /*
> >        * Enable CPU buffer clear mitigation for host and VMM, if also affected
> > -      * by MDS or TAA. Otherwise, enable mitigation for VMM only.
> > +      * by MDS or TAA.
> >        */
> > -     if (boot_cpu_has_bug(X86_BUG_MDS) ||
> (boot_cpu_has_bug(X86_BUG_TAA) &&
> > -                                           boot_cpu_has(X86_FEATURE_RTM)))
> > -             setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF);
> > +     if (boot_cpu_has_bug(X86_BUG_MDS) || taa_vulnerable())
> > +             verw_mitigation_selected = true; }
>
> So applied this reads strange:
>
>         if (mmio_mitigation == MMIO_MITIGATION_AUTO)
>                 mmio_mitigation = MMIO_MITIGATION_VERW;
>
>         if (boot_cpu_has_bug(X86_BUG_MDS) || taa_vulnerable())
>                 verw_mitigation_selected = true;
>
> I'd expect to see:
>
>         if (mmio_mitigation == MMIO_MITIGATION_AUTO) {
>                 mmio_mitigation = MMIO_MITIGATION_VERW;
>                 verw_mitigation_selected = true;
>         }
>
>         if (boot_cpu_has_bug(X86_BUG_MDS) || taa_vulnerable())
>                 verw_mitigation_selected = true;
>
> because the above branch already selected MMIO_MITIGATION_VERW so we
> might as well set verw_mitigation_selected, right?

Based on my understanding from the existing code, X86_FEATURE_CLEAR_CPU_BUF is only to be used if the CPU is also affected by MDS or TAA (as noted by the comment).  verw_mitigation_selected really means that X86_FEATURE_CLEAR_CPU_BUF is going to be set.

>
> > +static void __init mmio_update_mitigation(void) {
> > +     if (!boot_cpu_has_bug(X86_BUG_MMIO_STALE_DATA) ||
> cpu_mitigations_off())
> > +             return;
> > +
> > +     if (verw_mitigation_selected)
> > +             mmio_mitigation = MMIO_MITIGATION_VERW;
>
> ... and the above change would obviate this one.
>
> Looking at that verw_mitigation_selected switch - that seems like the higher prio
> thing we should be looking at first as in: *something* selected VERW mitigation so
> we must honor it.
>
> And then the *_select_mitigation() functions will simply use the respective
> *_mitigation variable to perhaps override it only when really needed.
>
> I think.
>
> Or maybe I'm missing an aspect.
>
> Because if we make verw_mitigation_selected the higher prio thing, we can remove
> some of that additional checking.
>
> Or?

Not quite sure I follow.  verw_mitigation_selected can be set by any of the VERW-related bugs.  The update function runs after all the select functions so it knows if anybody else wanted VERW.  If so (and the CPU is vulnerable), we update the mmio_mitigation option.  But maybe I missed what you were trying to say here.

>
> > +
> > +     if (mmio_mitigation == MMIO_MITIGATION_VERW) {
>
> I.e., in mmio_update_mitigation(), the only check you need to do is:
>
>         if (verw_mitigation_selected)
>
> and then adjust mmio_mitigation depending on microcode presence or not.

Maybe this gets back to the point above that MMIO_MITIGATION_VERW does not necessarily mean verw_mitigation_selected.

MMIO_MITIGATION_VERW can imply just the need for the static branches to clear stale data in the VMM.

>
> > +             /*
> > +              * Check if the system has the right microcode.
> > +              *
> > +              * CPU Fill buffer clear mitigation is enumerated by either an explicit
> > +              * FB_CLEAR or by the presence of both MD_CLEAR and L1D_FLUSH
> on MDS
> > +              * affected systems.
> > +              */
> > +             if (!((x86_arch_cap_msr & ARCH_CAP_FB_CLEAR) ||
> > +                   (boot_cpu_has(X86_FEATURE_MD_CLEAR) &&
> > +                    boot_cpu_has(X86_FEATURE_FLUSH_L1D) &&
> > +                  !(x86_arch_cap_msr & ARCH_CAP_MDS_NO))))
> > +                     mmio_mitigation = MMIO_MITIGATION_UCODE_NEEDED;
>
> ... as you do here.
>
> > +     }
> > +
> > +     if (boot_cpu_has_bug(X86_BUG_MMIO_UNKNOWN))
>
> Btw, that UNKNOWN thing is just silly. Looking at git history:
>
> 7df548840c49 ("x86/bugs: Add "unknown" reporting for MMIO Stale Data")
>
> this was added just so that it doesn't say "Not affected" about those CPUs but
> "unknown."
>
> But
>
>   "Mitigation is not deployed when the status is unknown."
>
> so if it is only about reporting, I think we can synthesize the logic of this:
>
>         if (!arch_cap_mmio_immune(x86_arch_cap_msr)) {
>                 if (cpu_matches(cpu_vuln_blacklist, MMIO))
>                         setup_force_cpu_bug(X86_BUG_MMIO_STALE_DATA);
>                 else if (!cpu_matches(cpu_vuln_whitelist, NO_MMIO))
>                         setup_force_cpu_bug(X86_BUG_MMIO_UNKNOWN);
>         }
>
> into a separate function and get rid of that X86_BUG_MMIO_UNKNOWN thing.
>
> Pawan?
>
> I'll try to whack it later to see how ugly it gets.
>
> > +             pr_info("Unknown: No mitigations\n");
> > +     else
> > +             pr_info("%s\n", mmio_strings[mmio_mitigation]); }
> > +
> > +static void __init mmio_apply_mitigation(void) {
> > +     if (mmio_mitigation == MMIO_MITIGATION_OFF)
> > +             return;
> >
> >       /*
> > -      * X86_FEATURE_CLEAR_CPU_BUF could be enabled by other VERW
> based
> > -      * mitigations, disable KVM-only mitigation in that case.
> > +      * Only enable the VMM mitigation if the CPU buffer clear mitigation is
> > +      * not being used.
>
> So this comment doesn't fit with what the code now does...
>
> >        */
> > -     if (boot_cpu_has(X86_FEATURE_CLEAR_CPU_BUF))
> > +     if (verw_mitigation_selected) {
>
> ... which is to check whether something enabled the VERW mitigation...

I think the comment does fit.  The code is only enabling the static branch for KVM if X86_FEATURE_CLEAR_CPU_BUF is not being used.

>
> > +             setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF);
> >               static_branch_disable(&mmio_stale_data_clear);
> > -     else
> > +     } else
> >               static_branch_enable(&mmio_stale_data_clear);
>
> { } around the else branch too pls.

Ok

--David Kaplan

Re: [PATCH v4 03/36] x86/bugs: Restructure mmio mitigation

[Pawan Gupta]

On Thu, Mar 13, 2025 at 10:36:17AM +0100, Borislav Petkov wrote:
> I'd expect to see:
> 
> 	if (mmio_mitigation == MMIO_MITIGATION_AUTO) {
>                 mmio_mitigation = MMIO_MITIGATION_VERW;
> 		verw_mitigation_selected = true;
> 	}
> 
>         if (boot_cpu_has_bug(X86_BUG_MDS) || taa_vulnerable())
>                 verw_mitigation_selected = true;
> 
> because the above branch already selected MMIO_MITIGATION_VERW so we might as
> well set verw_mitigation_selected, right?

There is a subtle difference between setting verw_mitigation_selected and
MMIO_MITIGATION_VERW. The former is a system-wide switch that indicates
VERW is needed at both kernel-exit and VMenter. MMIO Stale Data is
different from other VERW based mitigations because it only requires VERW
at VMenter, when not affected by MDS/TAA. So, turning the system-wide knob
here would be wrong.

> > +static void __init mmio_update_mitigation(void)
> > +{
> > +	if (!boot_cpu_has_bug(X86_BUG_MMIO_STALE_DATA) || cpu_mitigations_off())
> > +		return;
> > +
> > +	if (verw_mitigation_selected)
> > +		mmio_mitigation = MMIO_MITIGATION_VERW;
[...]
> > +	if (boot_cpu_has_bug(X86_BUG_MMIO_UNKNOWN))
> 
> Btw, that UNKNOWN thing is just silly. Looking at git history:
> 
> 7df548840c49 ("x86/bugs: Add "unknown" reporting for MMIO Stale Data")
> 
> this was added just so that it doesn't say "Not affected" about those CPUs but
> "unknown."
> 
> But
> 
>   "Mitigation is not deployed when the status is unknown."
> 
> so if it is only about reporting, I think we can synthesize the logic of this:
> 
>         if (!arch_cap_mmio_immune(x86_arch_cap_msr)) {
>                 if (cpu_matches(cpu_vuln_blacklist, MMIO))
>                         setup_force_cpu_bug(X86_BUG_MMIO_STALE_DATA);
>                 else if (!cpu_matches(cpu_vuln_whitelist, NO_MMIO))
>                         setup_force_cpu_bug(X86_BUG_MMIO_UNKNOWN);
>         }
> 
> into a separate function and get rid of that X86_BUG_MMIO_UNKNOWN thing.

Hmm, that would not be straightforward, specially for sysfs status. The
above logic requires parsing the cpu_vuln_whitelist which is not available
after init. Moreover, sysfs reads would become slower if it has to read an
MSR and parse tables.

Also, cpu_show_common() by default shows "Not affected" in the absence of
bug bit. So setting X86_BUG_MMIO_UNKNOWN is simpler overall.

cpu_show_common(bug)
{
	if (!boot_cpu_has_bug(bug))
		return sysfs_emit(buf, "Not affected\n");

MMIO and VERW

[Borislav Petkov]

Carving this thing out into a separate thread:

On Thu, Mar 13, 2025 at 12:26:06PM -0700, Pawan Gupta wrote:
> On Thu, Mar 13, 2025 at 10:36:17AM +0100, Borislav Petkov wrote:
> > I'd expect to see:
> > 
> > 	if (mmio_mitigation == MMIO_MITIGATION_AUTO) {
> >                 mmio_mitigation = MMIO_MITIGATION_VERW;
> > 		verw_mitigation_selected = true;
> > 	}
> > 
> >         if (boot_cpu_has_bug(X86_BUG_MDS) || taa_vulnerable())
> >                 verw_mitigation_selected = true;
> > 
> > because the above branch already selected MMIO_MITIGATION_VERW so we might as
> > well set verw_mitigation_selected, right?
> 
> There is a subtle difference between setting verw_mitigation_selected and
> MMIO_MITIGATION_VERW. The former is a system-wide switch that indicates
> VERW is needed at both kernel-exit and VMenter. MMIO Stale Data is
> different from other VERW based mitigations because it only requires VERW
> at VMenter, when not affected by MDS/TAA. So, turning the system-wide knob
> here would be wrong.

Realistically speaking, do we have a machine where you *only* enable VERW on
VMENTER?

I'm not talking about some experimentation scenario where one measures which
mitigations cost how much.

Do we have a real-life hw configuration where the *only* VERW mitigation
needed is at VMENTER because that machine is affected *only* by MMIO and no
other VERW-based mitigation is needed?

Because if not, we might as well drop that too-special distinction and
simplify that maze of nasty conditional spaghetti...

Hmmm?

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: MMIO and VERW

[Pawan Gupta]

On Tue, Mar 18, 2025 at 03:16:59PM +0100, Borislav Petkov wrote:
> Carving this thing out into a separate thread:
> 
> On Thu, Mar 13, 2025 at 12:26:06PM -0700, Pawan Gupta wrote:
> > On Thu, Mar 13, 2025 at 10:36:17AM +0100, Borislav Petkov wrote:
> > > I'd expect to see:
> > > 
> > > 	if (mmio_mitigation == MMIO_MITIGATION_AUTO) {
> > >                 mmio_mitigation = MMIO_MITIGATION_VERW;
> > > 		verw_mitigation_selected = true;
> > > 	}
> > > 
> > >         if (boot_cpu_has_bug(X86_BUG_MDS) || taa_vulnerable())
> > >                 verw_mitigation_selected = true;
> > > 
> > > because the above branch already selected MMIO_MITIGATION_VERW so we might as
> > > well set verw_mitigation_selected, right?
> > 
> > There is a subtle difference between setting verw_mitigation_selected and
> > MMIO_MITIGATION_VERW. The former is a system-wide switch that indicates
> > VERW is needed at both kernel-exit and VMenter. MMIO Stale Data is
> > different from other VERW based mitigations because it only requires VERW
> > at VMenter, when not affected by MDS/TAA. So, turning the system-wide knob
> > here would be wrong.
> 
> Realistically speaking, do we have a machine where you *only* enable VERW on
> VMENTER?

Yes, more on it below.

> I'm not talking about some experimentation scenario where one measures which
> mitigations cost how much.
> 
> Do we have a real-life hw configuration where the *only* VERW mitigation
> needed is at VMENTER because that machine is affected *only* by MMIO and no
> other VERW-based mitigation is needed?

Rocket Lake, Comet Lake, Ice Lake with tsx=off only require VERW at
VMENTER. There are other MMIO affected CPUs that are not affected by MDS
and do not support TSX or disable it by default.

Re: MMIO and VERW

[Borislav Petkov]

On Tue, Mar 18, 2025 at 09:25:05AM -0700, Pawan Gupta wrote:
> Rocket Lake, Comet Lake, Ice Lake with tsx=off only require VERW at
> VMENTER. There are other MMIO affected CPUs that are not affected by MDS
> and do not support TSX or disable it by default.

So all those CPUs are only affected by MMIO and not affected by neither of
those:

TAA, RFDS, MDS

?

Or is that the case only when TSX is not enabled/not present there?

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: MMIO and VERW

[Pawan Gupta]

On Tue, Mar 18, 2025 at 05:34:51PM +0100, Borislav Petkov wrote:
> On Tue, Mar 18, 2025 at 09:25:05AM -0700, Pawan Gupta wrote:
> > Rocket Lake, Comet Lake, Ice Lake with tsx=off only require VERW at
> > VMENTER. There are other MMIO affected CPUs that are not affected by MDS
> > and do not support TSX or disable it by default.
> 
> So all those CPUs are only affected by MMIO and not affected by neither of
> those:
> 
> TAA, RFDS, MDS

That is correct, they are not affected by MDS, TAA and RFDS.

> Or is that the case only when TSX is not enabled/not present there?

As per the affected CPU table [1], Ice Lake is not affected by TAA even if
TSX is enabled.

[1] https://www.intel.com/content/www/us/en/developer/topic-technology/software-security-guidance/processors-affected-consolidated-product-cpu-model.html#tab-blade-1-2

Re: MMIO and VERW

[Borislav Petkov]

On Tue, Mar 18, 2025 at 09:56:45AM -0700, Pawan Gupta wrote:
> On Tue, Mar 18, 2025 at 05:34:51PM +0100, Borislav Petkov wrote:
> > On Tue, Mar 18, 2025 at 09:25:05AM -0700, Pawan Gupta wrote:
> > > Rocket Lake, Comet Lake, Ice Lake with tsx=off only require VERW at
> > > VMENTER. There are other MMIO affected CPUs that are not affected by MDS
> > > and do not support TSX or disable it by default.
> > 
> > So all those CPUs are only affected by MMIO and not affected by neither of
> > those:
> > 
> > TAA, RFDS, MDS
> 
> That is correct, they are not affected by MDS, TAA and RFDS.
> 
> > Or is that the case only when TSX is not enabled/not present there?
> 
> As per the affected CPU table [1], Ice Lake is not affected by TAA even if
> TSX is enabled.

That table is insane - I need at least 4 monitors to stare at it properly. :-P

Anyway, so I'm wondering if we special-case those CPUs and have them select
a special

MMIO_MITIGATION_VERW_VM

case and keep them separate from that whole
CPUs-can-be-affected-by-multiple-vulns and the mitigation for all of them is
VERW.

They will enable mmio_stale_data_clear and will be out of the equation.

Which will make this other logic simpler.

Hmm...

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: MMIO and VERW

[Pawan Gupta]

On Tue, Mar 18, 2025 at 06:37:08PM +0100, Borislav Petkov wrote:
> On Tue, Mar 18, 2025 at 09:56:45AM -0700, Pawan Gupta wrote:
> > On Tue, Mar 18, 2025 at 05:34:51PM +0100, Borislav Petkov wrote:
> > > On Tue, Mar 18, 2025 at 09:25:05AM -0700, Pawan Gupta wrote:
> > > > Rocket Lake, Comet Lake, Ice Lake with tsx=off only require VERW at
> > > > VMENTER. There are other MMIO affected CPUs that are not affected by MDS
> > > > and do not support TSX or disable it by default.
> > > 
> > > So all those CPUs are only affected by MMIO and not affected by neither of
> > > those:
> > > 
> > > TAA, RFDS, MDS
> > 
> > That is correct, they are not affected by MDS, TAA and RFDS.
> > 
> > > Or is that the case only when TSX is not enabled/not present there?
> > 
> > As per the affected CPU table [1], Ice Lake is not affected by TAA even if
> > TSX is enabled.
> 
> That table is insane - I need at least 4 monitors to stare at it properly. :-P

:D Totally agree. A machine readable format is here:
https://github.com/intel/Intel-affected-processor-list/blob/main/Intel_affected_processor_list.csv

> Anyway, so I'm wondering if we special-case those CPUs and have them select
> a special
> 
> MMIO_MITIGATION_VERW_VM
> 
> case and keep them separate from that whole
> CPUs-can-be-affected-by-multiple-vulns and the mitigation for all of them is
> VERW.
> 
> They will enable mmio_stale_data_clear and will be out of the equation.
> 
> Which will make this other logic simpler.

Likely yes, I will give this a shot and see how it compares with the
currrent implementation. Thanks for the suggestion.

Re: MMIO and VERW

[Borislav Petkov]

On Tue, Mar 18, 2025 at 11:15:15AM -0700, Pawan Gupta wrote:
> :D Totally agree. A machine readable format is here:
> https://github.com/intel/Intel-affected-processor-list/blob/main/Intel_affected_processor_list.csv

Nice.

> Likely yes, I will give this a shot and see how it compares with the
> currrent implementation. Thanks for the suggestion.

Thanks!

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH v4 03/36] x86/bugs: Restructure mmio mitigation

[Borislav Petkov]

On Thu, Mar 13, 2025 at 12:26:06PM -0700, Pawan Gupta wrote:
> Hmm, that would not be straightforward, specially for sysfs status.

See below:

- the unknown thing is done only for this vuln and not for the others

- it doesn't do anything besides reporting things differently - it doesn't
  apply any mitigations - it is simply causing unnecessary complications which
  don't bring anything besides maintenance overhead. Unless I'm missing an
  angle...

- all the currently unaffected CPUs can also be in "unknown" status so why is
  this special?

IOW, just whack the thing.

diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index 46935f29805c..75d77a5c28d7 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -518,7 +518,7 @@
 #define X86_BUG_ITLB_MULTIHIT		X86_BUG(23) /* "itlb_multihit" CPU may incur MCE during certain page attribute changes */
 #define X86_BUG_SRBDS			X86_BUG(24) /* "srbds" CPU may leak RNG bits if not mitigated */
 #define X86_BUG_MMIO_STALE_DATA		X86_BUG(25) /* "mmio_stale_data" CPU is affected by Processor MMIO Stale Data vulnerabilities */
-#define X86_BUG_MMIO_UNKNOWN		X86_BUG(26) /* "mmio_unknown" CPU is too old and its MMIO Stale Data status is unknown */
+/* unused, was #define X86_BUG_MMIO_UNKNOWN		X86_BUG(26) "mmio_unknown" CPU is too old and its MMIO Stale Data status is unknown */
 #define X86_BUG_RETBLEED		X86_BUG(27) /* "retbleed" CPU is affected by RETBleed */
 #define X86_BUG_EIBRS_PBRSB		X86_BUG(28) /* "eibrs_pbrsb" EIBRS is vulnerable to Post Barrier RSB Predictions */
 #define X86_BUG_SMT_RSB			X86_BUG(29) /* "smt_rsb" CPU is vulnerable to Cross-Thread Return Address Predictions */
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 4386aa6c69e1..a91a1cac6183 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -428,7 +428,6 @@ static const char * const mmio_strings[] = {
 static void __init mmio_select_mitigation(void)
 {
 	if (!boot_cpu_has_bug(X86_BUG_MMIO_STALE_DATA) ||
-	     boot_cpu_has_bug(X86_BUG_MMIO_UNKNOWN) ||
 	     cpu_mitigations_off()) {
 		mmio_mitigation = MMIO_MITIGATION_OFF;
 		return;
@@ -591,8 +590,6 @@ static void __init md_clear_update_mitigation(void)
 		pr_info("TAA: %s\n", taa_strings[taa_mitigation]);
 	if (boot_cpu_has_bug(X86_BUG_MMIO_STALE_DATA))
 		pr_info("MMIO Stale Data: %s\n", mmio_strings[mmio_mitigation]);
-	else if (boot_cpu_has_bug(X86_BUG_MMIO_UNKNOWN))
-		pr_info("MMIO Stale Data: Unknown: No mitigations\n");
 	if (boot_cpu_has_bug(X86_BUG_RFDS))
 		pr_info("Register File Data Sampling: %s\n", rfds_strings[rfds_mitigation]);
 }
@@ -2819,9 +2816,6 @@ static ssize_t tsx_async_abort_show_state(char *buf)
 
 static ssize_t mmio_stale_data_show_state(char *buf)
 {
-	if (boot_cpu_has_bug(X86_BUG_MMIO_UNKNOWN))
-		return sysfs_emit(buf, "Unknown: No mitigations\n");
-
 	if (mmio_mitigation == MMIO_MITIGATION_OFF)
 		return sysfs_emit(buf, "%s\n", mmio_strings[mmio_mitigation]);
 
@@ -3006,7 +3000,6 @@ static ssize_t cpu_show_common(struct device *dev, struct device_attribute *attr
 		return srbds_show_state(buf);
 
 	case X86_BUG_MMIO_STALE_DATA:
-	case X86_BUG_MMIO_UNKNOWN:
 		return mmio_stale_data_show_state(buf);
 
 	case X86_BUG_RETBLEED:
@@ -3075,10 +3068,7 @@ ssize_t cpu_show_srbds(struct device *dev, struct device_attribute *attr, char *
 
 ssize_t cpu_show_mmio_stale_data(struct device *dev, struct device_attribute *attr, char *buf)
 {
-	if (boot_cpu_has_bug(X86_BUG_MMIO_UNKNOWN))
-		return cpu_show_common(dev, attr, buf, X86_BUG_MMIO_UNKNOWN);
-	else
-		return cpu_show_common(dev, attr, buf, X86_BUG_MMIO_STALE_DATA);
+	return cpu_show_common(dev, attr, buf, X86_BUG_MMIO_STALE_DATA);
 }
 
 ssize_t cpu_show_retbleed(struct device *dev, struct device_attribute *attr, char *buf)
diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index 12126adbc3a9..4ada55f126ae 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -1402,15 +1402,10 @@ static void __init cpu_set_bug_bits(struct cpuinfo_x86 *c)
 	 * Affected CPU list is generally enough to enumerate the vulnerability,
 	 * but for virtualization case check for ARCH_CAP MSR bits also, VMM may
 	 * not want the guest to enumerate the bug.
-	 *
-	 * Set X86_BUG_MMIO_UNKNOWN for CPUs that are neither in the blacklist,
-	 * nor in the whitelist and also don't enumerate MSR ARCH_CAP MMIO bits.
 	 */
 	if (!arch_cap_mmio_immune(x86_arch_cap_msr)) {
 		if (cpu_matches(cpu_vuln_blacklist, MMIO))
 			setup_force_cpu_bug(X86_BUG_MMIO_STALE_DATA);
-		else if (!cpu_matches(cpu_vuln_whitelist, NO_MMIO))
-			setup_force_cpu_bug(X86_BUG_MMIO_UNKNOWN);
 	}
 
 	if (!cpu_has(c, X86_FEATURE_BTC_NO)) {
diff --git a/tools/arch/x86/include/asm/cpufeatures.h b/tools/arch/x86/include/asm/cpufeatures.h
index 9e3fa7942e7d..e88500d90309 100644
--- a/tools/arch/x86/include/asm/cpufeatures.h
+++ b/tools/arch/x86/include/asm/cpufeatures.h
@@ -508,7 +508,7 @@
 #define X86_BUG_ITLB_MULTIHIT		X86_BUG(23) /* "itlb_multihit" CPU may incur MCE during certain page attribute changes */
 #define X86_BUG_SRBDS			X86_BUG(24) /* "srbds" CPU may leak RNG bits if not mitigated */
 #define X86_BUG_MMIO_STALE_DATA		X86_BUG(25) /* "mmio_stale_data" CPU is affected by Processor MMIO Stale Data vulnerabilities */
-#define X86_BUG_MMIO_UNKNOWN		X86_BUG(26) /* "mmio_unknown" CPU is too old and its MMIO Stale Data status is unknown */
+/* unused, was #define X86_BUG_MMIO_UNKNOWN		X86_BUG(26) * "mmio_unknown" CPU is too old and its MMIO Stale Data status is unknown */
 #define X86_BUG_RETBLEED		X86_BUG(27) /* "retbleed" CPU is affected by RETBleed */
 #define X86_BUG_EIBRS_PBRSB		X86_BUG(28) /* "eibrs_pbrsb" EIBRS is vulnerable to Post Barrier RSB Predictions */
 #define X86_BUG_SMT_RSB			X86_BUG(29) /* "smt_rsb" CPU is vulnerable to Cross-Thread Return Address Predictions */

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH v4 03/36] x86/bugs: Restructure mmio mitigation

[Pawan Gupta]

On Mon, Mar 24, 2025 at 10:29:15AM +0100, Borislav Petkov wrote:
> On Thu, Mar 13, 2025 at 12:26:06PM -0700, Pawan Gupta wrote:
> > Hmm, that would not be straightforward, specially for sysfs status.
> 
> See below:
> 
> - the unknown thing is done only for this vuln and not for the others
> 
> - it doesn't do anything besides reporting things differently - it doesn't
>   apply any mitigations - it is simply causing unnecessary complications which
>   don't bring anything besides maintenance overhead. Unless I'm missing an
>   angle...

"Unknown" status reporting was requested by Andrew Cooper. I am not able to
find that conversation though. IIRC, the reason was out-of-service CPUs
were not tested for the presence of vulnerability. Adding Andrew to Cc.

> - all the currently unaffected CPUs can also be in "unknown" status so why is
>   this special?

Makes sense.

> IOW, just whack the thing.

I will let Andrew comment on this.

Re: [PATCH v4 03/36] x86/bugs: Restructure mmio mitigation

[Borislav Petkov]

On Mon, Mar 24, 2025 at 10:41:15AM -0700, Pawan Gupta wrote:
> "Unknown" status reporting was requested by Andrew Cooper. I am not able to
> find that conversation though. IIRC, the reason was out-of-service CPUs
> were not tested for the presence of vulnerability. Adding Andrew to Cc.

Right, except we don't do that for other vulns... let's wait for him to
respond though...

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH v4 04/36] x86/bugs: Restructure rfds mitigation

[Josh Poimboeuf]

On Mon, Mar 10, 2025 at 11:39:51AM -0500, David Kaplan wrote:
>  static void __init rfds_select_mitigation(void)
>  {
>  	if (!boot_cpu_has_bug(X86_BUG_RFDS) || cpu_mitigations_off()) {
>  		rfds_mitigation = RFDS_MITIGATION_OFF;
>  		return;
>  	}
> -	if (rfds_mitigation == RFDS_MITIGATION_OFF)
> -		return;

The removal of this RFDS_MITIGATION_OFF check can cause
verw_mitigation_selected to get wrongly enabled below if it was
RFDS_MITIGATION_OFF to begin with.

I think it's only a bisection issue, that gets re-added later with
"x86/bugs: Add attack vector controls for rfds".

>  
>  	if (rfds_mitigation == RFDS_MITIGATION_AUTO)
>  		rfds_mitigation = RFDS_MITIGATION_VERW;
>  
> -	if (x86_arch_cap_msr & ARCH_CAP_RFDS_CLEAR)
> +	if (rfds_has_ucode())
> +		verw_mitigation_selected = true;
> +}

-- 
Josh

Re: [PATCH v4 11/36] x86/bugs: Restructure spectre_v2_user mitigation

[Josh Poimboeuf]

On Mon, Mar 10, 2025 at 11:39:58AM -0500, David Kaplan wrote:
>  static inline bool spectre_v2_in_ibrs_mode(enum spectre_v2_mitigation mode)
> @@ -1446,10 +1436,10 @@ static inline bool spectre_v2_in_ibrs_mode(enum spectre_v2_mitigation mode)
>  	return spectre_v2_in_eibrs_mode(mode) || mode == SPECTRE_V2_IBRS;
>  }
>  
> +

Extra newline here.

>  	case SPECTRE_V2_USER_CMD_SECCOMP:
> -	case SPECTRE_V2_USER_CMD_SECCOMP_IBPB:
>  		if (IS_ENABLED(CONFIG_SECCOMP))
> -			mode = SPECTRE_V2_USER_SECCOMP;
> +			spectre_v2_user_ibpb = SPECTRE_V2_USER_SECCOMP;
>  		else
> -			mode = SPECTRE_V2_USER_PRCTL;
> +			spectre_v2_user_ibpb = SPECTRE_V2_USER_PRCTL;
> +		spectre_v2_user_stibp = spectre_v2_user_ibpb;
> +		break;
> +	case SPECTRE_V2_USER_CMD_SECCOMP_IBPB:
> +		spectre_v2_user_ibpb = SPECTRE_V2_USER_STRICT;
> +		spectre_v2_user_stibp = SPECTRE_V2_USER_PRCTL;
>  		break;
>  	}

For SPECTRE_V2_USER_CMD_SECCOMP_IBPB, shouldn't spectre_v2_user_stibp
be SPECTRE_V2_USER_SECCOMP if CONFIG_SECCOMP?

Also I think spectre_v2_user_ibpb needs to be cleared here if
X86_FEATURE_IBPB isn't set.  And similar for spectre_v2_user_stibp and
X86_FEATURE_STIBP.

> -	/* Initialize Indirect Branch Prediction Barrier */
> -	if (boot_cpu_has(X86_FEATURE_IBPB)) {
> -		static_branch_enable(&switch_vcpu_ibpb);
> +	/*
> +	 * At this point, an STIBP mode other than "off" has been set.
> +	 * If STIBP support is not being forced, check if STIBP always-on
> +	 * is preferred.
> +	 */
> +	if (spectre_v2_user_stibp != SPECTRE_V2_USER_STRICT &&
> +	    boot_cpu_has(X86_FEATURE_AMD_STIBP_ALWAYS_ON))
> +		spectre_v2_user_stibp = SPECTRE_V2_USER_STRICT_PREFERRED;

Instead of checking for !SPECTRE_V2_USER_STRICT it would probably be
better to check for SPECTRE_V2_USER_PRCTL or SPECTRE_V2_USER_SECCOMP
directly.

Then the returns added to the SPECTRE_V2_USER_CMD_AUTO case in
"x86/bugs: Add attack vector controls for spectre_v2_user" can be
converted to breaks, which simplifies the control flow and also allows
the above-suggested X86_FEATURE_IBPB/X86_FEATURE_STIBP checks to keep
working.

-- 
Josh

Re: [PATCH v4 11/36] x86/bugs: Restructure spectre_v2_user mitigation

[Josh Poimboeuf]

On Mon, Mar 10, 2025 at 11:39:58AM -0500, David Kaplan wrote:
> @@ -214,6 +211,8 @@ void __init cpu_select_mitigations(void)
>  	 * choices.
>  	 */
>  	retbleed_update_mitigation();
> +	/* spectre_v2_user_update_mitigation() depends on retbleed_mitigation */
> +	spectre_v2_user_update_mitigation();

Function names need trailing parentheses: "retbleed_mitigation()"

-- 
Josh

Re: [PATCH v4 13/36] x86/bugs: Restructure spectre_v2 mitigation

[Josh Poimboeuf]

On Mon, Mar 10, 2025 at 11:40:00AM -0500, David Kaplan wrote:
> Restructure spectre_v2 to use select/update/apply functions to create
> consistent vulnerability handling.
> 
> The spectre_v2 mitigation may be updated based on the selected retbleed
> mitigation.
> 
> Signed-off-by: David Kaplan <david.kaplan@amd.com>
> ---
>  arch/x86/kernel/cpu/bugs.c | 79 +++++++++++++++++++++++---------------
>  1 file changed, 48 insertions(+), 31 deletions(-)
> 
> diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
> index 96cb2ac70245..b4a72ddf159c 100644
> --- a/arch/x86/kernel/cpu/bugs.c
> +++ b/arch/x86/kernel/cpu/bugs.c
> @@ -56,6 +56,8 @@
>  static void __init spectre_v1_select_mitigation(void);
>  static void __init spectre_v1_apply_mitigation(void);
>  static void __init spectre_v2_select_mitigation(void);
> +static void __init spectre_v2_update_mitigation(void);
> +static void __init spectre_v2_apply_mitigation(void);
>  static void __init retbleed_select_mitigation(void);
>  static void __init retbleed_update_mitigation(void);
>  static void __init retbleed_apply_mitigation(void);
> @@ -212,7 +214,12 @@ void __init cpu_select_mitigations(void)
>  	/*
>  	 * After mitigations are selected, some may need to update their
>  	 * choices.
> +	 *
> +	 * Note that retbleed_update_mitigation() relies on the state set by
> +	 * spectre_v2_update_mitigation(); specifically it wants to know about
> +	 * spectre_v2=ibrs.
>  	 */
> +	spectre_v2_update_mitigation();
>  	retbleed_update_mitigation();

I'd suggest moving that dependency comment to above
retbleed_update_mitigaton() and making it more concise:

 	/*
 	 * After mitigations are selected, some may need to update their
 	 * choices.
	 */
	spectre_v2_update_mitigation();
	/* retbleed_update_mitigation() depends on spectre_v2_update_mitigation() */
	retbleed_update_mitigation();

-- 
Josh

Re: [PATCH v4 16/36] x86/bugs: Restructure srso mitigation

[Josh Poimboeuf]

On Mon, Mar 10, 2025 at 11:40:03AM -0500, David Kaplan wrote:
> @@ -229,6 +226,8 @@ void __init cpu_select_mitigations(void)
>  	taa_update_mitigation();
>  	mmio_update_mitigation();
>  	rfds_update_mitigation();
> +	/* srso_update_mitigation() relies on retbleed_mitigation. */
> +	srso_update_mitigation();

"relies on" -> "depends on" for consistency with the other comments.
Also it's "retbleed_update_mitigation".  Also needs parentheses:

	/* srso_update_mitigation() depends on retbleed_update_mitigation() */

> +	switch (srso_mitigation) {
> +	case SRSO_MITIGATION_SAFE_RET:
> +		if (!IS_ENABLED(CONFIG_MITIGATION_SRSO))
>  			pr_err("WARNING: kernel not compiled with MITIGATION_SRSO.\n");

srso_mitigation should be reset to NONE here?

> -		} else {
> -			pr_err("WARNING: kernel not compiled with MITIGATION_IBPB_ENTRY.\n");

Same here.

-- 
Josh

Re: [PATCH v4 18/36] cpu: Define attack vectors

[Josh Poimboeuf]

On Mon, Mar 10, 2025 at 11:40:05AM -0500, David Kaplan wrote:
> @@ -3178,8 +3179,38 @@ void __init boot_cpu_hotplug_init(void)
>  
>  #ifdef CONFIG_CPU_MITIGATIONS
>  /*
> - * These are used for a global "mitigations=" cmdline option for toggling
> - * optional CPU mitigations.
> + * All except the cross-thread attack vector are mitigated by default.
> + * Cross-thread mitigation often requires disabling SMT which is too expensive
> + * to be enabled by default.

Cross-thread is *partially* mitigated by default (everything except
disabling SMT).

> +bool cpu_mitigate_attack_vector(enum cpu_attack_vectors v)
> +{
> +	if (v < NR_CPU_ATTACK_VECTORS)
> +		return attack_vectors[v];
> +
> +	WARN_ON_ONCE(v >= NR_CPU_ATTACK_VECTORS);

This can be a WARN_ONCE(), v is already known to be invalid here.

>  static int __init mitigations_parse_cmdline(char *arg)
>  {
> -	if (!strcmp(arg, "off"))
> -		cpu_mitigations = CPU_MITIGATIONS_OFF;
> -	else if (!strcmp(arg, "auto"))
> -		cpu_mitigations = CPU_MITIGATIONS_AUTO;
> -	else if (!strcmp(arg, "auto,nosmt"))
> -		cpu_mitigations = CPU_MITIGATIONS_AUTO_NOSMT;
> -	else
> -		pr_crit("Unsupported mitigations=%s, system may still be vulnerable\n",
> -			arg);
> +	char *s, *p;
> +	int len;
> +
> +	len = mitigations_parse_global_opt(arg);
> +
> +	if (cpu_mitigations_off()) {
> +		memset(attack_vectors, 0, sizeof(attack_vectors));
> +		smt_mitigations = SMT_MITIGATIONS_OFF;
> +	} else if (cpu_mitigations_auto_nosmt())
> +		smt_mitigations = SMT_MITIGATIONS_ON;

Kernel coding style wants consistent braces for if-then-else:

	if (cpu_mitigations_off()) {
		memset(attack_vectors, 0, sizeof(attack_vectors));
		smt_mitigations = SMT_MITIGATIONS_OFF;
	} else if (cpu_mitigations_auto_nosmt()) {
		smt_mitigations = SMT_MITIGATIONS_ON;
	}

-- 
Josh

Re: [PATCH v4 17/36] Documentation/x86: Document the new attack vector controls

[Josh Poimboeuf]

On Mon, Mar 10, 2025 at 11:40:04AM -0500, David Kaplan wrote:
> +=============== ============== ============ ============= ============== ============
> +Vulnerability   User-to-Kernel User-to-User Guest-to-Host Guest-to-Guest Cross-Thread
> +=============== ============== ============ ============= ============== ============
> +BHI                   X                           X
> +GDS                   X              X            X              X        (Note 1)
> +L1TF                  X                           X                       (Note 2)
> +MDS                   X              X            X              X        (Note 2)
> +MMIO                  X              X            X              X        (Note 2)
> +Meltdown              X
> +Retbleed              X                           X                       (Note 3)
> +RFDS                  X              X            X              X
> +Spectre_v1            X
> +Spectre_v2            X                           X
> +Spectre_v2_user                      X                           X        (Note 1)
> +SRBDS                 X              X            X              X
> +SRSO                  X                           X
> +SSB (Note 4)

Any reason not to put the "Note 4" in the same column as the others?

-- 
Josh

RE: [PATCH v4 04/36] x86/bugs: Restructure rfds mitigation

[Kaplan, David]

[AMD Official Use Only - AMD Internal Distribution Only]

> -----Original Message-----
> From: Josh Poimboeuf <jpoimboe@kernel.org>
> Sent: Thursday, April 10, 2025 11:25 AM
> To: Kaplan, David <David.Kaplan@amd.com>
> Cc: Thomas Gleixner <tglx@linutronix.de>; Borislav Petkov <bp@alien8.de>;
> Peter Zijlstra <peterz@infradead.org>; Pawan Gupta
> <pawan.kumar.gupta@linux.intel.com>; Ingo Molnar <mingo@redhat.com>; Dave
> Hansen <dave.hansen@linux.intel.com>; x86@kernel.org; H . Peter Anvin
> <hpa@zytor.com>; linux-kernel@vger.kernel.org; Brendan Jackman
> <jackmanb@google.com>; Derek Manwaring <derekmn@amazon.com>
> Subject: Re: [PATCH v4 04/36] x86/bugs: Restructure rfds mitigation
>
> Caution: This message originated from an External Source. Use proper caution
> when opening attachments, clicking links, or responding.
>
>
> On Mon, Mar 10, 2025 at 11:39:51AM -0500, David Kaplan wrote:
> >  static void __init rfds_select_mitigation(void)  {
> >       if (!boot_cpu_has_bug(X86_BUG_RFDS) || cpu_mitigations_off()) {
> >               rfds_mitigation = RFDS_MITIGATION_OFF;
> >               return;
> >       }
> > -     if (rfds_mitigation == RFDS_MITIGATION_OFF)
> > -             return;
>
> The removal of this RFDS_MITIGATION_OFF check can cause
> verw_mitigation_selected to get wrongly enabled below if it was
> RFDS_MITIGATION_OFF to begin with.
>
> I think it's only a bisection issue, that gets re-added later with
> "x86/bugs: Add attack vector controls for rfds".
>

Ah, good catch.  Will fix.

Thanks --David Kaplan

RE: [PATCH v4 11/36] x86/bugs: Restructure spectre_v2_user mitigation

[Kaplan, David]

[AMD Official Use Only - AMD Internal Distribution Only]

> -----Original Message-----
> From: Josh Poimboeuf <jpoimboe@kernel.org>
> Sent: Thursday, April 10, 2025 11:26 AM
> To: Kaplan, David <David.Kaplan@amd.com>
> Cc: Thomas Gleixner <tglx@linutronix.de>; Borislav Petkov <bp@alien8.de>;
> Peter Zijlstra <peterz@infradead.org>; Pawan Gupta
> <pawan.kumar.gupta@linux.intel.com>; Ingo Molnar <mingo@redhat.com>; Dave
> Hansen <dave.hansen@linux.intel.com>; x86@kernel.org; H . Peter Anvin
> <hpa@zytor.com>; linux-kernel@vger.kernel.org; Brendan Jackman
> <jackmanb@google.com>; Derek Manwaring <derekmn@amazon.com>
> Subject: Re: [PATCH v4 11/36] x86/bugs: Restructure spectre_v2_user mitigation
>
> Caution: This message originated from an External Source. Use proper caution
> when opening attachments, clicking links, or responding.
>
>
> On Mon, Mar 10, 2025 at 11:39:58AM -0500, David Kaplan wrote:
> >  static inline bool spectre_v2_in_ibrs_mode(enum spectre_v2_mitigation
> > mode) @@ -1446,10 +1436,10 @@ static inline bool
> spectre_v2_in_ibrs_mode(enum spectre_v2_mitigation mode)
> >       return spectre_v2_in_eibrs_mode(mode) || mode ==
> > SPECTRE_V2_IBRS;  }
> >
> > +
>
> Extra newline here.

Ack

>
> >       case SPECTRE_V2_USER_CMD_SECCOMP:
> > -     case SPECTRE_V2_USER_CMD_SECCOMP_IBPB:
> >               if (IS_ENABLED(CONFIG_SECCOMP))
> > -                     mode = SPECTRE_V2_USER_SECCOMP;
> > +                     spectre_v2_user_ibpb = SPECTRE_V2_USER_SECCOMP;
> >               else
> > -                     mode = SPECTRE_V2_USER_PRCTL;
> > +                     spectre_v2_user_ibpb = SPECTRE_V2_USER_PRCTL;
> > +             spectre_v2_user_stibp = spectre_v2_user_ibpb;
> > +             break;
> > +     case SPECTRE_V2_USER_CMD_SECCOMP_IBPB:
> > +             spectre_v2_user_ibpb = SPECTRE_V2_USER_STRICT;
> > +             spectre_v2_user_stibp = SPECTRE_V2_USER_PRCTL;
> >               break;
> >       }
>
> For SPECTRE_V2_USER_CMD_SECCOMP_IBPB, shouldn't
> spectre_v2_user_stibp be SPECTRE_V2_USER_SECCOMP if
> CONFIG_SECCOMP?

Yes, you're right.

>
> Also I think spectre_v2_user_ibpb needs to be cleared here if
> X86_FEATURE_IBPB isn't set.  And similar for spectre_v2_user_stibp and
> X86_FEATURE_STIBP.

Ack.

>
> > -     /* Initialize Indirect Branch Prediction Barrier */
> > -     if (boot_cpu_has(X86_FEATURE_IBPB)) {
> > -             static_branch_enable(&switch_vcpu_ibpb);
> > +     /*
> > +      * At this point, an STIBP mode other than "off" has been set.
> > +      * If STIBP support is not being forced, check if STIBP always-on
> > +      * is preferred.
> > +      */
> > +     if (spectre_v2_user_stibp != SPECTRE_V2_USER_STRICT &&
> > +         boot_cpu_has(X86_FEATURE_AMD_STIBP_ALWAYS_ON))
> > +             spectre_v2_user_stibp =
> > + SPECTRE_V2_USER_STRICT_PREFERRED;
>
> Instead of checking for !SPECTRE_V2_USER_STRICT it would probably be
> better to check for SPECTRE_V2_USER_PRCTL or
> SPECTRE_V2_USER_SECCOMP directly.
>
> Then the returns added to the SPECTRE_V2_USER_CMD_AUTO case in
> "x86/bugs: Add attack vector controls for spectre_v2_user" can be converted to
> breaks, which simplifies the control flow and also allows the above-suggested
> X86_FEATURE_IBPB/X86_FEATURE_STIBP checks to keep working.
>

Got it, yeah will do that.

Thanks --David Kaplan

RE: [PATCH v4 11/36] x86/bugs: Restructure spectre_v2_user mitigation

[Kaplan, David]

[AMD Official Use Only - AMD Internal Distribution Only]

> -----Original Message-----
> From: Josh Poimboeuf <jpoimboe@kernel.org>
> Sent: Thursday, April 10, 2025 11:41 AM
> To: Kaplan, David <David.Kaplan@amd.com>
> Cc: Thomas Gleixner <tglx@linutronix.de>; Borislav Petkov <bp@alien8.de>;
> Peter Zijlstra <peterz@infradead.org>; Pawan Gupta
> <pawan.kumar.gupta@linux.intel.com>; Ingo Molnar <mingo@redhat.com>; Dave
> Hansen <dave.hansen@linux.intel.com>; x86@kernel.org; H . Peter Anvin
> <hpa@zytor.com>; linux-kernel@vger.kernel.org; Brendan Jackman
> <jackmanb@google.com>; Derek Manwaring <derekmn@amazon.com>
> Subject: Re: [PATCH v4 11/36] x86/bugs: Restructure spectre_v2_user mitigation
>
> Caution: This message originated from an External Source. Use proper caution
> when opening attachments, clicking links, or responding.
>
>
> On Mon, Mar 10, 2025 at 11:39:58AM -0500, David Kaplan wrote:
> > @@ -214,6 +211,8 @@ void __init cpu_select_mitigations(void)
> >        * choices.
> >        */
> >       retbleed_update_mitigation();
> > +     /* spectre_v2_user_update_mitigation() depends on retbleed_mitigation */
> > +     spectre_v2_user_update_mitigation();
>
> Function names need trailing parentheses: "retbleed_mitigation()"
>

That one is not actually a function name, it's the name of the file-scope variable.

--David Kaplan

RE: [PATCH v4 13/36] x86/bugs: Restructure spectre_v2 mitigation

[Kaplan, David]

[AMD Official Use Only - AMD Internal Distribution Only]

> -----Original Message-----
> From: Josh Poimboeuf <jpoimboe@kernel.org>
> Sent: Thursday, April 10, 2025 12:09 PM
> To: Kaplan, David <David.Kaplan@amd.com>
> Cc: Thomas Gleixner <tglx@linutronix.de>; Borislav Petkov <bp@alien8.de>;
> Peter Zijlstra <peterz@infradead.org>; Pawan Gupta
> <pawan.kumar.gupta@linux.intel.com>; Ingo Molnar <mingo@redhat.com>; Dave
> Hansen <dave.hansen@linux.intel.com>; x86@kernel.org; H . Peter Anvin
> <hpa@zytor.com>; linux-kernel@vger.kernel.org; Brendan Jackman
> <jackmanb@google.com>; Derek Manwaring <derekmn@amazon.com>
> Subject: Re: [PATCH v4 13/36] x86/bugs: Restructure spectre_v2 mitigation
>
> Caution: This message originated from an External Source. Use proper caution
> when opening attachments, clicking links, or responding.
>
>
> On Mon, Mar 10, 2025 at 11:40:00AM -0500, David Kaplan wrote:
> > Restructure spectre_v2 to use select/update/apply functions to create
> > consistent vulnerability handling.
> >
> > The spectre_v2 mitigation may be updated based on the selected
> > retbleed mitigation.
> >
> > Signed-off-by: David Kaplan <david.kaplan@amd.com>
> > ---
> >  arch/x86/kernel/cpu/bugs.c | 79
> > +++++++++++++++++++++++---------------
> >  1 file changed, 48 insertions(+), 31 deletions(-)
> >
> > diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
> > index 96cb2ac70245..b4a72ddf159c 100644
> > --- a/arch/x86/kernel/cpu/bugs.c
> > +++ b/arch/x86/kernel/cpu/bugs.c
> > @@ -56,6 +56,8 @@
> >  static void __init spectre_v1_select_mitigation(void);
> >  static void __init spectre_v1_apply_mitigation(void);
> >  static void __init spectre_v2_select_mitigation(void);
> > +static void __init spectre_v2_update_mitigation(void);
> > +static void __init spectre_v2_apply_mitigation(void);
> >  static void __init retbleed_select_mitigation(void);  static void
> > __init retbleed_update_mitigation(void);  static void __init
> > retbleed_apply_mitigation(void); @@ -212,7 +214,12 @@ void __init
> > cpu_select_mitigations(void)
> >       /*
> >        * After mitigations are selected, some may need to update their
> >        * choices.
> > +      *
> > +      * Note that retbleed_update_mitigation() relies on the state set by
> > +      * spectre_v2_update_mitigation(); specifically it wants to know about
> > +      * spectre_v2=ibrs.
> >        */
> > +     spectre_v2_update_mitigation();
> >       retbleed_update_mitigation();
>
> I'd suggest moving that dependency comment to above
> retbleed_update_mitigaton() and making it more concise:
>
>         /*
>          * After mitigations are selected, some may need to update their
>          * choices.
>          */
>         spectre_v2_update_mitigation();
>         /* retbleed_update_mitigation() depends on spectre_v2_update_mitigation() */
>         retbleed_update_mitigation();
>

Ack

--David Kaplan

RE: [PATCH v4 16/36] x86/bugs: Restructure srso mitigation

[Kaplan, David]

[AMD Official Use Only - AMD Internal Distribution Only]

> -----Original Message-----
> From: Josh Poimboeuf <jpoimboe@kernel.org>
> Sent: Thursday, April 10, 2025 12:39 PM
> To: Kaplan, David <David.Kaplan@amd.com>
> Cc: Thomas Gleixner <tglx@linutronix.de>; Borislav Petkov <bp@alien8.de>;
> Peter Zijlstra <peterz@infradead.org>; Pawan Gupta
> <pawan.kumar.gupta@linux.intel.com>; Ingo Molnar <mingo@redhat.com>; Dave
> Hansen <dave.hansen@linux.intel.com>; x86@kernel.org; H . Peter Anvin
> <hpa@zytor.com>; linux-kernel@vger.kernel.org; Brendan Jackman
> <jackmanb@google.com>; Derek Manwaring <derekmn@amazon.com>
> Subject: Re: [PATCH v4 16/36] x86/bugs: Restructure srso mitigation
>
> Caution: This message originated from an External Source. Use proper caution
> when opening attachments, clicking links, or responding.
>
>
> On Mon, Mar 10, 2025 at 11:40:03AM -0500, David Kaplan wrote:
> > @@ -229,6 +226,8 @@ void __init cpu_select_mitigations(void)
> >       taa_update_mitigation();
> >       mmio_update_mitigation();
> >       rfds_update_mitigation();
> > +     /* srso_update_mitigation() relies on retbleed_mitigation. */
> > +     srso_update_mitigation();
>
> "relies on" -> "depends on" for consistency with the other comments.
> Also it's "retbleed_update_mitigation".  Also needs parentheses:

Ack.  I was referring to the variable here, but I can refer to the function which probably makes more sense given that is what is called here.

>
>         /* srso_update_mitigation() depends on retbleed_update_mitigation() */
>
> > +     switch (srso_mitigation) {
> > +     case SRSO_MITIGATION_SAFE_RET:
> > +             if (!IS_ENABLED(CONFIG_MITIGATION_SRSO))
> >                       pr_err("WARNING: kernel not compiled with
> MITIGATION_SRSO.\n");
>
> srso_mitigation should be reset to NONE here?
>
> > -             } else {
> > -                     pr_err("WARNING: kernel not compiled with
> MITIGATION_IBPB_ENTRY.\n");
>
> Same here.
>

Ack to both.

Thanks --David Kaplan

RE: [PATCH v4 17/36] Documentation/x86: Document the new attack vector controls

[Kaplan, David]

[AMD Official Use Only - AMD Internal Distribution Only]

> -----Original Message-----
> From: Josh Poimboeuf <jpoimboe@kernel.org>
> Sent: Thursday, April 10, 2025 1:15 PM
> To: Kaplan, David <David.Kaplan@amd.com>
> Cc: Thomas Gleixner <tglx@linutronix.de>; Borislav Petkov <bp@alien8.de>;
> Peter Zijlstra <peterz@infradead.org>; Pawan Gupta
> <pawan.kumar.gupta@linux.intel.com>; Ingo Molnar <mingo@redhat.com>; Dave
> Hansen <dave.hansen@linux.intel.com>; x86@kernel.org; H . Peter Anvin
> <hpa@zytor.com>; linux-kernel@vger.kernel.org; Brendan Jackman
> <jackmanb@google.com>; Derek Manwaring <derekmn@amazon.com>
> Subject: Re: [PATCH v4 17/36] Documentation/x86: Document the new attack
> vector controls
>
> Caution: This message originated from an External Source. Use proper caution
> when opening attachments, clicking links, or responding.
>
>
> On Mon, Mar 10, 2025 at 11:40:04AM -0500, David Kaplan wrote:
> > +=============== ============== ============ =============
> ============== ============
> > +Vulnerability   User-to-Kernel User-to-User Guest-to-Host Guest-to-Guest
> Cross-Thread
> > +=============== ============== ============ =============
> ============== ============
> > +BHI                   X                           X
> > +GDS                   X              X            X              X        (Note 1)
> > +L1TF                  X                           X                       (Note 2)
> > +MDS                   X              X            X              X        (Note 2)
> > +MMIO                  X              X            X              X        (Note 2)
> > +Meltdown              X
> > +Retbleed              X                           X                       (Note 3)
> > +RFDS                  X              X            X              X
> > +Spectre_v1            X
> > +Spectre_v2            X                           X
> > +Spectre_v2_user                      X                           X        (Note 1)
> > +SRBDS                 X              X            X              X
> > +SRSO                  X                           X
> > +SSB (Note 4)
>
> Any reason not to put the "Note 4" in the same column as the others?
>

The other notes are about cross-thread mitigation specifically and those notes refer to the SMT aspects of those issues.

Note 4 in this case is about the SSB vulnerability itself, explaining that by default there is no mitigation for any case.  I was concerned that including SSB but without any X's in any of the columns would be confusing, so the note attempted to explain that there were no default mitigations for SSB under any attack vector.

--David Kaplan

RE: [PATCH v4 18/36] cpu: Define attack vectors

[Kaplan, David]

[AMD Official Use Only - AMD Internal Distribution Only]

> -----Original Message-----
> From: Josh Poimboeuf <jpoimboe@kernel.org>
> Sent: Thursday, April 10, 2025 1:12 PM
> To: Kaplan, David <David.Kaplan@amd.com>
> Cc: Thomas Gleixner <tglx@linutronix.de>; Borislav Petkov <bp@alien8.de>;
> Peter Zijlstra <peterz@infradead.org>; Pawan Gupta
> <pawan.kumar.gupta@linux.intel.com>; Ingo Molnar <mingo@redhat.com>; Dave
> Hansen <dave.hansen@linux.intel.com>; x86@kernel.org; H . Peter Anvin
> <hpa@zytor.com>; linux-kernel@vger.kernel.org; Brendan Jackman
> <jackmanb@google.com>; Derek Manwaring <derekmn@amazon.com>
> Subject: Re: [PATCH v4 18/36] cpu: Define attack vectors
>
> Caution: This message originated from an External Source. Use proper caution
> when opening attachments, clicking links, or responding.
>
>
> On Mon, Mar 10, 2025 at 11:40:05AM -0500, David Kaplan wrote:
> > @@ -3178,8 +3179,38 @@ void __init boot_cpu_hotplug_init(void)
> >
> >  #ifdef CONFIG_CPU_MITIGATIONS
> >  /*
> > - * These are used for a global "mitigations=" cmdline option for
> > toggling
> > - * optional CPU mitigations.
> > + * All except the cross-thread attack vector are mitigated by default.
> > + * Cross-thread mitigation often requires disabling SMT which is too
> > + expensive
> > + * to be enabled by default.
>
> Cross-thread is *partially* mitigated by default (everything except disabling SMT).
>

Will fix comment

> > +bool cpu_mitigate_attack_vector(enum cpu_attack_vectors v) {
> > +     if (v < NR_CPU_ATTACK_VECTORS)
> > +             return attack_vectors[v];
> > +
> > +     WARN_ON_ONCE(v >= NR_CPU_ATTACK_VECTORS);
>
> This can be a WARN_ONCE(), v is already known to be invalid here.

Ack

>
> >  static int __init mitigations_parse_cmdline(char *arg)  {
> > -     if (!strcmp(arg, "off"))
> > -             cpu_mitigations = CPU_MITIGATIONS_OFF;
> > -     else if (!strcmp(arg, "auto"))
> > -             cpu_mitigations = CPU_MITIGATIONS_AUTO;
> > -     else if (!strcmp(arg, "auto,nosmt"))
> > -             cpu_mitigations = CPU_MITIGATIONS_AUTO_NOSMT;
> > -     else
> > -             pr_crit("Unsupported mitigations=%s, system may still be vulnerable\n",
> > -                     arg);
> > +     char *s, *p;
> > +     int len;
> > +
> > +     len = mitigations_parse_global_opt(arg);
> > +
> > +     if (cpu_mitigations_off()) {
> > +             memset(attack_vectors, 0, sizeof(attack_vectors));
> > +             smt_mitigations = SMT_MITIGATIONS_OFF;
> > +     } else if (cpu_mitigations_auto_nosmt())
> > +             smt_mitigations = SMT_MITIGATIONS_ON;
>
> Kernel coding style wants consistent braces for if-then-else:
>
>         if (cpu_mitigations_off()) {
>                 memset(attack_vectors, 0, sizeof(attack_vectors));
>                 smt_mitigations = SMT_MITIGATIONS_OFF;
>         } else if (cpu_mitigations_auto_nosmt()) {
>                 smt_mitigations = SMT_MITIGATIONS_ON;
>         }
>

Ack

Thanks --David Kaplan

Re: [PATCH v4 11/36] x86/bugs: Restructure spectre_v2_user mitigation

[Josh Poimboeuf]

On Mon, Apr 14, 2025 at 07:20:59PM +0000, Kaplan, David wrote:
> [AMD Official Use Only - AMD Internal Distribution Only]
> 
> > -----Original Message-----
> > From: Josh Poimboeuf <jpoimboe@kernel.org>
> > Sent: Thursday, April 10, 2025 11:41 AM
> > To: Kaplan, David <David.Kaplan@amd.com>
> > Cc: Thomas Gleixner <tglx@linutronix.de>; Borislav Petkov <bp@alien8.de>;
> > Peter Zijlstra <peterz@infradead.org>; Pawan Gupta
> > <pawan.kumar.gupta@linux.intel.com>; Ingo Molnar <mingo@redhat.com>; Dave
> > Hansen <dave.hansen@linux.intel.com>; x86@kernel.org; H . Peter Anvin
> > <hpa@zytor.com>; linux-kernel@vger.kernel.org; Brendan Jackman
> > <jackmanb@google.com>; Derek Manwaring <derekmn@amazon.com>
> > Subject: Re: [PATCH v4 11/36] x86/bugs: Restructure spectre_v2_user mitigation
> >
> > Caution: This message originated from an External Source. Use proper caution
> > when opening attachments, clicking links, or responding.
> >
> >
> > On Mon, Mar 10, 2025 at 11:39:58AM -0500, David Kaplan wrote:
> > > @@ -214,6 +211,8 @@ void __init cpu_select_mitigations(void)
> > >        * choices.
> > >        */
> > >       retbleed_update_mitigation();
> > > +     /* spectre_v2_user_update_mitigation() depends on retbleed_mitigation */
> > > +     spectre_v2_user_update_mitigation();
> >
> > Function names need trailing parentheses: "retbleed_mitigation()"
> >
> 
> That one is not actually a function name, it's the name of the file-scope variable.

Why refer to a variable?  Isn't it the function dependencies which
matter here?

-- 
Josh

Re: [PATCH v4 17/36] Documentation/x86: Document the new attack vector controls

[Josh Poimboeuf]

On Mon, Apr 14, 2025 at 09:15:54PM +0000, Kaplan, David wrote:
> [AMD Official Use Only - AMD Internal Distribution Only]
> 
> > -----Original Message-----
> > From: Josh Poimboeuf <jpoimboe@kernel.org>
> > Sent: Thursday, April 10, 2025 1:15 PM
> > To: Kaplan, David <David.Kaplan@amd.com>
> > Cc: Thomas Gleixner <tglx@linutronix.de>; Borislav Petkov <bp@alien8.de>;
> > Peter Zijlstra <peterz@infradead.org>; Pawan Gupta
> > <pawan.kumar.gupta@linux.intel.com>; Ingo Molnar <mingo@redhat.com>; Dave
> > Hansen <dave.hansen@linux.intel.com>; x86@kernel.org; H . Peter Anvin
> > <hpa@zytor.com>; linux-kernel@vger.kernel.org; Brendan Jackman
> > <jackmanb@google.com>; Derek Manwaring <derekmn@amazon.com>
> > Subject: Re: [PATCH v4 17/36] Documentation/x86: Document the new attack
> > vector controls
> >
> > Caution: This message originated from an External Source. Use proper caution
> > when opening attachments, clicking links, or responding.
> >
> >
> > On Mon, Mar 10, 2025 at 11:40:04AM -0500, David Kaplan wrote:
> > > +=============== ============== ============ =============
> > ============== ============
> > > +Vulnerability   User-to-Kernel User-to-User Guest-to-Host Guest-to-Guest
> > Cross-Thread
> > > +=============== ============== ============ =============
> > ============== ============
> > > +BHI                   X                           X
> > > +GDS                   X              X            X              X        (Note 1)
> > > +L1TF                  X                           X                       (Note 2)
> > > +MDS                   X              X            X              X        (Note 2)
> > > +MMIO                  X              X            X              X        (Note 2)
> > > +Meltdown              X
> > > +Retbleed              X                           X                       (Note 3)
> > > +RFDS                  X              X            X              X
> > > +Spectre_v1            X
> > > +Spectre_v2            X                           X
> > > +Spectre_v2_user                      X                           X        (Note 1)
> > > +SRBDS                 X              X            X              X
> > > +SRSO                  X                           X
> > > +SSB (Note 4)
> >
> > Any reason not to put the "Note 4" in the same column as the others?
> >
> 
> The other notes are about cross-thread mitigation specifically and those notes refer to the SMT aspects of those issues.
> 
> Note 4 in this case is about the SSB vulnerability itself, explaining
> that by default there is no mitigation for any case.  I was concerned
> that including SSB but without any X's in any of the columns would be
> confusing, so the note attempted to explain that there were no default
> mitigations for SSB under any attack vector.

Putting the note there makes it a lot harder to see it.  And I think the
lack of X's is accurate, no?

-- 
Josh

RE: [PATCH v4 11/36] x86/bugs: Restructure spectre_v2_user mitigation

[Kaplan, David]

[AMD Official Use Only - AMD Internal Distribution Only]

> -----Original Message-----
> From: Josh Poimboeuf <jpoimboe@kernel.org>
> Sent: Monday, April 14, 2025 6:49 PM
> To: Kaplan, David <David.Kaplan@amd.com>
> Cc: Thomas Gleixner <tglx@linutronix.de>; Borislav Petkov <bp@alien8.de>;
> Peter Zijlstra <peterz@infradead.org>; Pawan Gupta
> <pawan.kumar.gupta@linux.intel.com>; Ingo Molnar <mingo@redhat.com>; Dave
> Hansen <dave.hansen@linux.intel.com>; x86@kernel.org; H . Peter Anvin
> <hpa@zytor.com>; linux-kernel@vger.kernel.org; Brendan Jackman
> <jackmanb@google.com>; Derek Manwaring <derekmn@amazon.com>
> Subject: Re: [PATCH v4 11/36] x86/bugs: Restructure spectre_v2_user mitigation
>
> Caution: This message originated from an External Source. Use proper caution
> when opening attachments, clicking links, or responding.
>
>
> On Mon, Apr 14, 2025 at 07:20:59PM +0000, Kaplan, David wrote:
> > [AMD Official Use Only - AMD Internal Distribution Only]
> >
> > > -----Original Message-----
> > > From: Josh Poimboeuf <jpoimboe@kernel.org>
> > > Sent: Thursday, April 10, 2025 11:41 AM
> > > To: Kaplan, David <David.Kaplan@amd.com>
> > > Cc: Thomas Gleixner <tglx@linutronix.de>; Borislav Petkov
> > > <bp@alien8.de>; Peter Zijlstra <peterz@infradead.org>; Pawan Gupta
> > > <pawan.kumar.gupta@linux.intel.com>; Ingo Molnar <mingo@redhat.com>;
> > > Dave Hansen <dave.hansen@linux.intel.com>; x86@kernel.org; H . Peter
> > > Anvin <hpa@zytor.com>; linux-kernel@vger.kernel.org; Brendan Jackman
> > > <jackmanb@google.com>; Derek Manwaring <derekmn@amazon.com>
> > > Subject: Re: [PATCH v4 11/36] x86/bugs: Restructure spectre_v2_user
> > > mitigation
> > >
> > > Caution: This message originated from an External Source. Use proper
> > > caution when opening attachments, clicking links, or responding.
> > >
> > >
> > > On Mon, Mar 10, 2025 at 11:39:58AM -0500, David Kaplan wrote:
> > > > @@ -214,6 +211,8 @@ void __init cpu_select_mitigations(void)
> > > >        * choices.
> > > >        */
> > > >       retbleed_update_mitigation();
> > > > +     /* spectre_v2_user_update_mitigation() depends on retbleed_mitigation
> */
> > > > +     spectre_v2_user_update_mitigation();
> > >
> > > Function names need trailing parentheses: "retbleed_mitigation()"
> > >
> >
> > That one is not actually a function name, it's the name of the file-scope variable.
>
> Why refer to a variable?  Isn't it the function dependencies which matter here?
>

Yeah, I agree.  The variable is what gets checked in the dependent function, but given this code block is all about the order of these various function calls, then using a function name makes more sense.

--David Kaplan

RE: [PATCH v4 17/36] Documentation/x86: Document the new attack vector controls

[Kaplan, David]

[AMD Official Use Only - AMD Internal Distribution Only]

> -----Original Message-----
> From: Josh Poimboeuf <jpoimboe@kernel.org>
> Sent: Monday, April 14, 2025 6:51 PM
> To: Kaplan, David <David.Kaplan@amd.com>
> Cc: Thomas Gleixner <tglx@linutronix.de>; Borislav Petkov <bp@alien8.de>;
> Peter Zijlstra <peterz@infradead.org>; Pawan Gupta
> <pawan.kumar.gupta@linux.intel.com>; Ingo Molnar <mingo@redhat.com>; Dave
> Hansen <dave.hansen@linux.intel.com>; x86@kernel.org; H . Peter Anvin
> <hpa@zytor.com>; linux-kernel@vger.kernel.org; Brendan Jackman
> <jackmanb@google.com>; Derek Manwaring <derekmn@amazon.com>
> Subject: Re: [PATCH v4 17/36] Documentation/x86: Document the new attack
> vector controls
>
> Caution: This message originated from an External Source. Use proper caution
> when opening attachments, clicking links, or responding.
>
>
> On Mon, Apr 14, 2025 at 09:15:54PM +0000, Kaplan, David wrote:
> > [AMD Official Use Only - AMD Internal Distribution Only]
> >
> > > -----Original Message-----
> > > From: Josh Poimboeuf <jpoimboe@kernel.org>
> > > Sent: Thursday, April 10, 2025 1:15 PM
> > > To: Kaplan, David <David.Kaplan@amd.com>
> > > Cc: Thomas Gleixner <tglx@linutronix.de>; Borislav Petkov
> > > <bp@alien8.de>; Peter Zijlstra <peterz@infradead.org>; Pawan Gupta
> > > <pawan.kumar.gupta@linux.intel.com>; Ingo Molnar <mingo@redhat.com>;
> > > Dave Hansen <dave.hansen@linux.intel.com>; x86@kernel.org; H . Peter
> > > Anvin <hpa@zytor.com>; linux-kernel@vger.kernel.org; Brendan Jackman
> > > <jackmanb@google.com>; Derek Manwaring <derekmn@amazon.com>
> > > Subject: Re: [PATCH v4 17/36] Documentation/x86: Document the new
> > > attack vector controls
> > >
> > > Caution: This message originated from an External Source. Use proper
> > > caution when opening attachments, clicking links, or responding.
> > >
> > >
> > > On Mon, Mar 10, 2025 at 11:40:04AM -0500, David Kaplan wrote:
> > > > +=============== ============== ============ =============
> > > ============== ============
> > > > +Vulnerability   User-to-Kernel User-to-User Guest-to-Host Guest-to-Guest
> > > Cross-Thread
> > > > +=============== ============== ============ =============
> > > ============== ============
> > > > +BHI                   X                           X
> > > > +GDS                   X              X            X              X        (Note 1)
> > > > +L1TF                  X                           X                       (Note 2)
> > > > +MDS                   X              X            X              X        (Note 2)
> > > > +MMIO                  X              X            X              X        (Note 2)
> > > > +Meltdown              X
> > > > +Retbleed              X                           X                       (Note 3)
> > > > +RFDS                  X              X            X              X
> > > > +Spectre_v1            X
> > > > +Spectre_v2            X                           X
> > > > +Spectre_v2_user                      X                           X        (Note 1)
> > > > +SRBDS                 X              X            X              X
> > > > +SRSO                  X                           X
> > > > +SSB (Note 4)
> > >
> > > Any reason not to put the "Note 4" in the same column as the others?
> > >
> >
> > The other notes are about cross-thread mitigation specifically and those notes
> refer to the SMT aspects of those issues.
> >
> > Note 4 in this case is about the SSB vulnerability itself, explaining
> > that by default there is no mitigation for any case.  I was concerned
> > that including SSB but without any X's in any of the columns would be
> > confusing, so the note attempted to explain that there were no default
> > mitigations for SSB under any attack vector.
>
> Putting the note there makes it a lot harder to see it.  And I think the lack of X's is
> accurate, no?
>

It is, it's just rather unique compared to the other bugs.  I could remove the note entirely, but I was concerned that might look odd because it'd be the only bug that isn't mitigated under any of the attack vectors.  And that's really just because the current default is not to mitigate that one.

--David Kaplan

Re: [PATCH v4 17/36] Documentation/x86: Document the new attack vector controls

[Josh Poimboeuf]

On Tue, Apr 15, 2025 at 02:59:32PM +0000, Kaplan, David wrote:
> > > > > +BHI                   X                           X
> > > > > +GDS                   X              X            X              X        (Note 1)
> > > > > +L1TF                  X                           X                       (Note 2)
> > > > > +MDS                   X              X            X              X        (Note 2)
> > > > > +MMIO                  X              X            X              X        (Note 2)
> > > > > +Meltdown              X
> > > > > +Retbleed              X                           X                       (Note 3)
> > > > > +RFDS                  X              X            X              X
> > > > > +Spectre_v1            X
> > > > > +Spectre_v2            X                           X
> > > > > +Spectre_v2_user                      X                           X        (Note 1)
> > > > > +SRBDS                 X              X            X              X
> > > > > +SRSO                  X                           X
> > > > > +SSB (Note 4)
> > > >
> > > > Any reason not to put the "Note 4" in the same column as the others?
> > > >
> > >
> > > The other notes are about cross-thread mitigation specifically and those notes
> > refer to the SMT aspects of those issues.
> > >
> > > Note 4 in this case is about the SSB vulnerability itself, explaining
> > > that by default there is no mitigation for any case.  I was concerned
> > > that including SSB but without any X's in any of the columns would be
> > > confusing, so the note attempted to explain that there were no default
> > > mitigations for SSB under any attack vector.
> >
> > Putting the note there makes it a lot harder to see it.  And I think the lack of X's is
> > accurate, no?
> >
> 
> It is, it's just rather unique compared to the other bugs.  I could
> remove the note entirely, but I was concerned that might look odd
> because it'd be the only bug that isn't mitigated under any of the
> attack vectors.  And that's really just because the current default is
> not to mitigate that one.

I think the note is helpful, it attempts to explain why there are no
X's.  I was just thinking that it seems more logical to put it in the
same column as the others.  And that would also help make it more clear
that yes, the X's are missing.  Which is indeed odd, but it's also the
reality.

-- 
Josh

RE: [PATCH v4 17/36] Documentation/x86: Document the new attack vector controls

[Kaplan, David]

[AMD Official Use Only - AMD Internal Distribution Only]

> -----Original Message-----
> From: Josh Poimboeuf <jpoimboe@kernel.org>
> Sent: Tuesday, April 15, 2025 10:32 AM
> To: Kaplan, David <David.Kaplan@amd.com>
> Cc: Thomas Gleixner <tglx@linutronix.de>; Borislav Petkov <bp@alien8.de>;
> Peter Zijlstra <peterz@infradead.org>; Pawan Gupta
> <pawan.kumar.gupta@linux.intel.com>; Ingo Molnar <mingo@redhat.com>; Dave
> Hansen <dave.hansen@linux.intel.com>; x86@kernel.org; H . Peter Anvin
> <hpa@zytor.com>; linux-kernel@vger.kernel.org; Brendan Jackman
> <jackmanb@google.com>; Derek Manwaring <derekmn@amazon.com>
> Subject: Re: [PATCH v4 17/36] Documentation/x86: Document the new attack
> vector controls
>
> Caution: This message originated from an External Source. Use proper caution
> when opening attachments, clicking links, or responding.
>
>
> On Tue, Apr 15, 2025 at 02:59:32PM +0000, Kaplan, David wrote:
> > > > > > +BHI                   X                           X
> > > > > > +GDS                   X              X            X              X        (Note 1)
> > > > > > +L1TF                  X                           X                       (Note 2)
> > > > > > +MDS                   X              X            X              X        (Note 2)
> > > > > > +MMIO                  X              X            X              X        (Note 2)
> > > > > > +Meltdown              X
> > > > > > +Retbleed              X                           X                       (Note 3)
> > > > > > +RFDS                  X              X            X              X
> > > > > > +Spectre_v1            X
> > > > > > +Spectre_v2            X                           X
> > > > > > +Spectre_v2_user                      X                           X        (Note 1)
> > > > > > +SRBDS                 X              X            X              X
> > > > > > +SRSO                  X                           X
> > > > > > +SSB (Note 4)
> > > > >
> > > > > Any reason not to put the "Note 4" in the same column as the others?
> > > > >
> > > >
> > > > The other notes are about cross-thread mitigation specifically and
> > > > those notes
> > > refer to the SMT aspects of those issues.
> > > >
> > > > Note 4 in this case is about the SSB vulnerability itself,
> > > > explaining that by default there is no mitigation for any case.  I
> > > > was concerned that including SSB but without any X's in any of the
> > > > columns would be confusing, so the note attempted to explain that
> > > > there were no default mitigations for SSB under any attack vector.
> > >
> > > Putting the note there makes it a lot harder to see it.  And I think
> > > the lack of X's is accurate, no?
> > >
> >
> > It is, it's just rather unique compared to the other bugs.  I could
> > remove the note entirely, but I was concerned that might look odd
> > because it'd be the only bug that isn't mitigated under any of the
> > attack vectors.  And that's really just because the current default is
> > not to mitigate that one.
>
> I think the note is helpful, it attempts to explain why there are no X's.  I was just
> thinking that it seems more logical to put it in the same column as the others.  And
> that would also help make it more clear that yes, the X's are missing.  Which is
> indeed odd, but it's also the reality.
>

Right, except that the last column is about the cross-thread vector, which is irrelevant for SSB.  All the other notes specifically pertain to SMT leakage.

I could put the '(Note 4)' text in every column, but that might be even weirder.  I could also remove SSB entirely from the table since it isn't technically relevant for any of the attack vector controls?

--David Kaplan

Re: [PATCH v4 17/36] Documentation/x86: Document the new attack vector controls

[Josh Poimboeuf]

On Tue, Apr 15, 2025 at 04:10:49PM +0000, Kaplan, David wrote:
> > I think the note is helpful, it attempts to explain why there are no X's.  I was just
> > thinking that it seems more logical to put it in the same column as the others.  And
> > that would also help make it more clear that yes, the X's are missing.  Which is
> > indeed odd, but it's also the reality.
> >
> 
> Right, except that the last column is about the cross-thread vector,
> which is irrelevant for SSB.  All the other notes specifically pertain
> to SMT leakage.

Ah.  Can we give the column a broader heading like "Notes"?

> I could put the '(Note 4)' text in every column, but that might be
> even weirder.  I could also remove SSB entirely from the table since
> it isn't technically relevant for any of the attack vector controls?

I'm thinking the table should list all the mitigations, regardless of
whether they're affected by these controls, so the controls are
well-defined without any ambiguity.

-- 
Josh

RE: [PATCH v4 17/36] Documentation/x86: Document the new attack vector controls

[Kaplan, David]

[AMD Official Use Only - AMD Internal Distribution Only]

> -----Original Message-----
> From: Josh Poimboeuf <jpoimboe@kernel.org>
> Sent: Tuesday, April 15, 2025 11:48 AM
> To: Kaplan, David <David.Kaplan@amd.com>
> Cc: Thomas Gleixner <tglx@linutronix.de>; Borislav Petkov <bp@alien8.de>;
> Peter Zijlstra <peterz@infradead.org>; Pawan Gupta
> <pawan.kumar.gupta@linux.intel.com>; Ingo Molnar <mingo@redhat.com>; Dave
> Hansen <dave.hansen@linux.intel.com>; x86@kernel.org; H . Peter Anvin
> <hpa@zytor.com>; linux-kernel@vger.kernel.org; Brendan Jackman
> <jackmanb@google.com>; Derek Manwaring <derekmn@amazon.com>
> Subject: Re: [PATCH v4 17/36] Documentation/x86: Document the new attack
> vector controls
>
> Caution: This message originated from an External Source. Use proper caution
> when opening attachments, clicking links, or responding.
>
>
> On Tue, Apr 15, 2025 at 04:10:49PM +0000, Kaplan, David wrote:
> > > I think the note is helpful, it attempts to explain why there are no
> > > X's.  I was just thinking that it seems more logical to put it in
> > > the same column as the others.  And that would also help make it
> > > more clear that yes, the X's are missing.  Which is indeed odd, but it's also the
> reality.
> > >
> >
> > Right, except that the last column is about the cross-thread vector,
> > which is irrelevant for SSB.  All the other notes specifically pertain
> > to SMT leakage.
>
> Ah.  Can we give the column a broader heading like "Notes"?

That's a decent idea.  Maybe a new column would be clearer, and I could just put an asterisk in the Cross-thread column for the issues that otherwise have notes there (just to indicate it's not a simple yes/no like in the others).

>
> > I could put the '(Note 4)' text in every column, but that might be
> > even weirder.  I could also remove SSB entirely from the table since
> > it isn't technically relevant for any of the attack vector controls?
>
> I'm thinking the table should list all the mitigations, regardless of whether they're
> affected by these controls, so the controls are well-defined without any ambiguity.
>

Ok

--David Kaplan