[PATCH v5 2/8] x86/bugs: Allow spectre_v2=ibrs on x86 vendors other than Intel

[Kim Phillips <kim.phillips@amd.com>]

Prepare for legacy IBRS toggling on AMD, where the BTB Isolation
SEV-SNP feature uses it to optimize the VM exit-to-re-entry path.
Commit 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to
support Kernel IBRS") restricted the option to Intel because that was the
only vendor that needed it at the time; nothing about the mechanism
is Intel-specific.

Keep the IBRS-trumps-retbleed logic in retbleed_update_mitigation()
Intel-only.  Legacy SPEC_CTRL.IBRS does not mitigate AMD's Branch
Type Confusion RETBleed variant (RET prediction uses the Return
Address Predictor, not the indirect branch predictors IBRS
restricts), so letting SPECTRE_V2_IBRS trump retbleed on AMD would
silently drop the UNRET/IBPB mitigation that does cover it.

On AMD the decoupling is total: retbleed mitigation selection never
consults spectre_v2=, so spectre_v2=ibrs neither adds nor removes
RETBleed coverage.  A kernel built without MITIGATION_UNRET_ENTRY and
MITIGATION_IBPB_ENTRY already reports RETBleed as "Vulnerable" via the
retbleed sysfs node and boot log regardless of the spectre_v2= value,
so there is no silent gap in the spectre_v2=ibrs path to warn about --
and a warning there would wrongly imply the Intel-style IBRS/RETBleed
coupling exists on AMD.

Also drop CPU_SUP_INTEL from CONFIG_MITIGATION_IBRS_ENTRY's depends
line: the IBRS_ENTER/IBRS_EXIT macros are vendor-neutral, and the
Intel-only restriction would silently redirect spectre_v2=ibrs to
AUTO on AMD-only kernels.

In spectre_v2_apply_mitigation(), route AutoIBRS-capable CPUs to
EFER.AUTOIBRS only for the eIBRS modes.  Previously any IBRS mode used
EFER.AUTOIBRS when the CPU had AutoIBRS, which was unreachable while
spectre_v2=ibrs was Intel-only, but would now hand spectre_v2=ibrs the
always-on AutoIBRS behaviour instead of the toggleable SPEC_CTRL.IBRS
the option asks for.

Finally, clear EFER.AUTOIBRS at the top of cpu_select_mitigations(),
alongside the existing SPEC_CTRL kexec cleanup.  head_64.S preserves
incoming EFER bits, so a kexec from a kernel that ran in AutoIBRS mode
carries the bit into the new kernel; without an explicit clear the CPU
stays in AutoIBRS mode while sysfs reports e.g. "Mitigation: IBRS" or a
retpoline mode, diverging from the actual hardware state.  On a normal
cold boot the bit is already clear, so the msr_clear_bit() is a no-op
there.  Clearing on the boot CPU suffices for APs, since it precedes the
init_real_mode() EFER snapshot used by the AP trampoline.

Reported-by: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
Cc: Borislav Petkov (AMD) <bp@alien8.de>
Signed-off-by: Kim Phillips <kim.phillips@amd.com>
Assisted-by: ClaudeCode:claude-opus-4-7
---
 arch/x86/Kconfig           |  7 +++---
 arch/x86/kernel/cpu/bugs.c | 44 +++++++++++++++++++++++++++-----------
 2 files changed, 35 insertions(+), 16 deletions(-)

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 15fd9ec5ecac..b9a7ddef4cba 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -2496,12 +2496,13 @@ config MITIGATION_IBPB_ENTRY
 
 config MITIGATION_IBRS_ENTRY
 	bool "Enable IBRS on kernel entry"
-	depends on CPU_SUP_INTEL && X86_64
+	depends on X86_64
 	default y
 	help
 	  Compile the kernel with support for the spectre_v2=ibrs mitigation.
-	  This mitigates both spectre_v2 and retbleed at great cost to
-	  performance.
+	  This mitigates spectre_v2 at great cost to performance.  On Intel,
+	  it also mitigates retbleed.  On AMD/Hygon, retbleed mitigation
+	  requires MITIGATION_UNRET_ENTRY or MITIGATION_IBPB_ENTRY.
 
 config MITIGATION_SRSO
 	bool "Mitigate speculative RAS overflow on AMD"
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 48eb1872af18..08780e0d37ec 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -1305,7 +1305,14 @@ static void __init retbleed_update_mitigation(void)
 
 	/*
 	 * Let IBRS trump all on Intel without affecting the effects of the
-	 * retbleed= cmdline option except for call depth based stuffing
+	 * retbleed= cmdline option except for call depth based stuffing.
+	 *
+	 * On AMD/Hygon, legacy SPEC_CTRL.IBRS toggling does not mitigate the
+	 * Branch Type Confusion RETBleed variant: RET prediction comes from
+	 * the Return Address Predictor, not the restricted indirect branch
+	 * predictors that IBRS controls.  So keep this Intel-only and leave
+	 * AMD's software return-thunk mitigation (UNRET/IBPB) in place even
+	 * when spectre_v2=ibrs is selected.
 	 */
 	if (boot_cpu_data.x86_vendor == X86_VENDOR_INTEL) {
 		switch (spectre_v2_enabled) {
@@ -2166,11 +2173,6 @@ static void __init spectre_v2_select_mitigation(void)
 		spectre_v2_cmd = SPECTRE_V2_CMD_AUTO;
 	}
 
-	if (spectre_v2_cmd == SPECTRE_V2_CMD_IBRS && boot_cpu_data.x86_vendor != X86_VENDOR_INTEL) {
-		pr_err("IBRS selected but not Intel CPU. Switching to AUTO select\n");
-		spectre_v2_cmd = SPECTRE_V2_CMD_AUTO;
-	}
-
 	if (spectre_v2_cmd == SPECTRE_V2_CMD_IBRS && !boot_cpu_has(X86_FEATURE_IBRS)) {
 		pr_err("IBRS selected but CPU doesn't have IBRS. Switching to AUTO select\n");
 		spectre_v2_cmd = SPECTRE_V2_CMD_AUTO;
@@ -2286,13 +2288,18 @@ static void __init spectre_v2_apply_mitigation(void)
 	if (spectre_v2_enabled == SPECTRE_V2_EIBRS && unprivileged_ebpf_enabled())
 		pr_err(SPECTRE_V2_EIBRS_EBPF_MSG);
 
-	if (spectre_v2_in_ibrs_mode(spectre_v2_enabled)) {
-		if (boot_cpu_has(X86_FEATURE_AUTOIBRS)) {
-			msr_set_bit(MSR_EFER, _EFER_AUTOIBRS);
-		} else {
-			x86_spec_ctrl_base |= SPEC_CTRL_IBRS;
-			update_spec_ctrl(x86_spec_ctrl_base);
-		}
+	/*
+	 * On AutoIBRS-capable CPUs, eIBRS is enabled through EFER.AUTOIBRS
+	 * rather than SPEC_CTRL.IBRS.  Legacy spectre_v2=ibrs keeps using
+	 * SPEC_CTRL.IBRS even there, as it needs to be toggled on kernel
+	 * entry/exit.
+	 */
+	if (spectre_v2_in_eibrs_mode(spectre_v2_enabled) &&
+	    boot_cpu_has(X86_FEATURE_AUTOIBRS)) {
+		msr_set_bit(MSR_EFER, _EFER_AUTOIBRS);
+	} else if (spectre_v2_in_ibrs_mode(spectre_v2_enabled)) {
+		x86_spec_ctrl_base |= SPEC_CTRL_IBRS;
+		update_spec_ctrl(x86_spec_ctrl_base);
 	}
 
 	if (spectre_v2_in_eibrs_mode(spectre_v2_enabled) &&
@@ -3297,6 +3304,17 @@ void __init cpu_select_mitigations(void)
 		x86_spec_ctrl_base &= ~SPEC_CTRL_MITIGATIONS_MASK;
 	}
 
+	/*
+	 * Likewise for EFER.AUTOIBRS: head_64.S preserves the incoming EFER
+	 * bits, so a kexec from a kernel that ran in AutoIBRS mode carries the
+	 * bit into this one.  Clear it and let the mitigation selection below
+	 * rediscover it.  This also runs before init_real_mode() snapshots
+	 * EFER for the AP trampoline, so APs inherit whatever this kernel
+	 * settles on rather than the previous kernel's choice.
+	 */
+	if (cpu_feature_enabled(X86_FEATURE_AUTOIBRS))
+		msr_clear_bit(MSR_EFER, _EFER_AUTOIBRS);
+
 	x86_arch_cap_msr = x86_read_arch_cap_msr();
 
 	cpu_print_attack_vectors();
-- 
2.43.0