[PATCH v4 04/10] cpu/bugs: Allow spectre_v2=ibrs on x86 vendors other than Intel

[Kim Phillips <kim.phillips@amd.com>]

Prepare for legacy IBRS toggling on AMD, where the BTB Isolation
SEV-SNP feature uses it to optimize the VM exit-to-re-entry path.
There is no reason this wasn't allowed in the first place, hence
the Fixes: and stable tags.

Keep the IBRS-trumps-retbleed logic in retbleed_update_mitigation()
Intel-only.  Legacy SPEC_CTRL.IBRS does not mitigate AMD's Branch
Type Confusion RETBleed variant (RET prediction uses the Return
Address Predictor, not the indirect branch predictors IBRS
restricts), so letting SPECTRE_V2_IBRS trump retbleed on AMD would
silently drop the UNRET/IBPB mitigation that does cover it.

On AMD the decoupling is total: retbleed mitigation selection never
consults spectre_v2=, so spectre_v2=ibrs neither adds nor removes
RETBleed coverage.  A kernel built without MITIGATION_UNRET_ENTRY and
MITIGATION_IBPB_ENTRY already reports RETBleed as "Vulnerable" via the
retbleed sysfs node and boot log regardless of the spectre_v2= value,
so there is no silent gap in the spectre_v2=ibrs path to warn about --
and a warning there would wrongly imply the Intel-style IBRS/RETBleed
coupling exists on AMD.

Also drop CPU_SUP_INTEL from CONFIG_MITIGATION_IBRS_ENTRY's depends
line: the IBRS_ENTER/IBRS_EXIT macros are vendor-neutral, and the
Intel-only restriction would silently redirect spectre_v2=ibrs to
AUTO on AMD-only kernels.

Explicitly set or clear EFER.AUTOIBRS to match the selected mitigation
on AutoIBRS-capable CPUs: set it in eIBRS mode, and clear it in every
other mode -- NONE, retpoline, LFENCE and the legacy IBRS path alike.
head_64.S preserves incoming EFER bits, so a kexec from an
eIBRS/AutoIBRS kernel carries EFER.AUTOIBRS into the new kernel; without
an explicit clear the CPU stays in AutoIBRS mode while sysfs reports
e.g. "Mitigation: IBRS" or a retpoline mode, diverging from the actual
hardware state.  On a normal cold boot the bit is already clear, so the
msr_clear_bit() is a no-op there.  Clearing on the boot CPU suffices for
APs, since it precedes the init_real_mode() EFER snapshot used by the AP
trampoline.

Fixes: 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")
Reported-by: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
Cc: Borislav Petkov (AMD) <bp@alien8.de>
Cc: stable@kernel.org
Signed-off-by: Kim Phillips <kim.phillips@amd.com>
Assisted-by: ClaudeCode:claude-opus-4-7
---
 arch/x86/Kconfig           |  7 ++++---
 arch/x86/kernel/cpu/bugs.c | 39 ++++++++++++++++++++++++++------------
 2 files changed, 31 insertions(+), 15 deletions(-)

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index e725b439d0a2..4bd91d41dbba 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -2492,12 +2492,13 @@ config MITIGATION_IBPB_ENTRY
 
 config MITIGATION_IBRS_ENTRY
 	bool "Enable IBRS on kernel entry"
-	depends on CPU_SUP_INTEL && X86_64
+	depends on X86_64
 	default y
 	help
 	  Compile the kernel with support for the spectre_v2=ibrs mitigation.
-	  This mitigates both spectre_v2 and retbleed at great cost to
-	  performance.
+	  This mitigates spectre_v2 at great cost to performance.  On Intel,
+	  it also mitigates retbleed.  On AMD/Hygon, retbleed mitigation
+	  requires MITIGATION_UNRET_ENTRY or MITIGATION_IBPB_ENTRY.
 
 config MITIGATION_SRSO
 	bool "Mitigate speculative RAS overflow on AMD"
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 6a036b509f0b..245de4ea8d60 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -1305,7 +1305,14 @@ static void __init retbleed_update_mitigation(void)
 
 	/*
 	 * Let IBRS trump all on Intel without affecting the effects of the
-	 * retbleed= cmdline option except for call depth based stuffing
+	 * retbleed= cmdline option except for call depth based stuffing.
+	 *
+	 * On AMD/Hygon, legacy SPEC_CTRL.IBRS toggling does not mitigate the
+	 * Branch Type Confusion RETBleed variant: RET prediction comes from
+	 * the Return Address Predictor, not the restricted indirect branch
+	 * predictors that IBRS controls.  So keep this Intel-only and leave
+	 * AMD's software return-thunk mitigation (UNRET/IBPB) in place even
+	 * when spectre_v2=ibrs is selected.
 	 */
 	if (boot_cpu_data.x86_vendor == X86_VENDOR_INTEL) {
 		switch (spectre_v2_enabled) {
@@ -2164,11 +2171,6 @@ static void __init spectre_v2_select_mitigation(void)
 		spectre_v2_cmd = SPECTRE_V2_CMD_AUTO;
 	}
 
-	if (spectre_v2_cmd == SPECTRE_V2_CMD_IBRS && boot_cpu_data.x86_vendor != X86_VENDOR_INTEL) {
-		pr_err("IBRS selected but not Intel CPU. Switching to AUTO select\n");
-		spectre_v2_cmd = SPECTRE_V2_CMD_AUTO;
-	}
-
 	if (spectre_v2_cmd == SPECTRE_V2_CMD_IBRS && !boot_cpu_has(X86_FEATURE_IBRS)) {
 		pr_err("IBRS selected but CPU doesn't have IBRS. Switching to AUTO select\n");
 		spectre_v2_cmd = SPECTRE_V2_CMD_AUTO;
@@ -2297,13 +2299,26 @@ static void __init spectre_v2_apply_mitigation(void)
 	if (spectre_v2_enabled == SPECTRE_V2_EIBRS && unprivileged_ebpf_enabled())
 		pr_err(SPECTRE_V2_EIBRS_EBPF_MSG);
 
-	if (spectre_v2_in_ibrs_mode(spectre_v2_enabled)) {
-		if (boot_cpu_has(X86_FEATURE_AUTOIBRS)) {
+	/*
+	 * head_64.S preserves EFER.AUTOIBRS across boot, so a kexec from a
+	 * kernel that ran in AutoIBRS mode carries the bit into the new kernel.
+	 * Explicitly set or clear it to match the selected mitigation, regardless
+	 * of which mode is in effect.  The boot CPU does this before
+	 * init_real_mode() snapshots EFER for the AP trampoline, so APs inherit
+	 * the correct value too.
+	 */
+	if (boot_cpu_has(X86_FEATURE_AUTOIBRS)) {
+		if (spectre_v2_in_eibrs_mode(spectre_v2_enabled))
 			msr_set_bit(MSR_EFER, _EFER_AUTOIBRS);
-		} else {
-			x86_spec_ctrl_base |= SPEC_CTRL_IBRS;
-			update_spec_ctrl(x86_spec_ctrl_base);
-		}
+		else
+			msr_clear_bit(MSR_EFER, _EFER_AUTOIBRS);
+	}
+
+	if (spectre_v2_in_ibrs_mode(spectre_v2_enabled) &&
+	    !(boot_cpu_has(X86_FEATURE_AUTOIBRS) &&
+	      spectre_v2_in_eibrs_mode(spectre_v2_enabled))) {
+		x86_spec_ctrl_base |= SPEC_CTRL_IBRS;
+		update_spec_ctrl(x86_spec_ctrl_base);
 	}
 
 	if (spectre_v2_in_eibrs_mode(spectre_v2_enabled) &&
-- 
2.43.0