[PATCH v2] x86/speculation: Simplify and make CALL_NOSPEC consistent

[PATCH v2] x86/speculation: Simplify and make CALL_NOSPEC consistent

[Pawan Gupta]

CALL_NOSPEC macro is used to generate Spectre-v2 mitigation friendly
indirect branches. At compile time the macro defaults to indirect branch,
and at runtime those can be patched to thunk based mitigations.

This approach is opposite of what is done for the rest of the kernel, where
the compile time default is to replace indirect calls with retpoline thunk
calls.

Make CALL_NOSPEC consistent with the rest of the kernel, default to
retpoline thunk at compile time when CONFIG_MITIGATION_RETPOLINE is
enabled.

Also add the missing __CS_PREFIX to the CALL_NOSPEC macro to make it
compatible with -mindirect-branch-cs-prefix.

Signed-off-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
---
v2:
- Fixed the inconsistency in the use of "\" in __CS_PREFIX. (Andrew)
- Fixed the comment to reflect that __CS_PREFIX only emits the prefix and
  not the JMP/CALL. (Andrew)

v1: https://lore.kernel.org/r/20250226-call-nospec-v1-1-4dde04a5c7a7@linux.intel.com
---
 arch/x86/include/asm/nospec-branch.h | 32 +++++++++++++++++++-------------
 1 file changed, 19 insertions(+), 13 deletions(-)

diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h
index 7e8bf78c03d5..aee26bb8230f 100644
--- a/arch/x86/include/asm/nospec-branch.h
+++ b/arch/x86/include/asm/nospec-branch.h
@@ -198,9 +198,8 @@
 .endm
 
 /*
- * Equivalent to -mindirect-branch-cs-prefix; emit the 5 byte jmp/call
- * to the retpoline thunk with a CS prefix when the register requires
- * a RAX prefix byte to encode. Also see apply_retpolines().
+ * Emits a conditional CS prefix that is compatible with
+ * -mindirect-branch-cs-prefix.
  */
 .macro __CS_PREFIX reg:req
 	.irp rs,r8,r9,r10,r11,r12,r13,r14,r15
@@ -420,20 +419,27 @@ static inline void call_depth_return_thunk(void) {}
 
 #ifdef CONFIG_X86_64
 
+/*
+ * Emits a conditional CS prefix that is compatible with
+ * -mindirect-branch-cs-prefix.
+ */
+#define __CS_PREFIX(reg)				\
+	".irp rs,r8,r9,r10,r11,r12,r13,r14,r15\n"	\
+	".ifc \\rs," reg "\n"				\
+	".byte 0x2e\n"					\
+	".endif\n"					\
+	".endr\n"
+
 /*
  * Inline asm uses the %V modifier which is only in newer GCC
  * which is ensured when CONFIG_MITIGATION_RETPOLINE is defined.
  */
-# define CALL_NOSPEC						\
-	ALTERNATIVE_2(						\
-	ANNOTATE_RETPOLINE_SAFE					\
-	"call *%[thunk_target]\n",				\
-	"call __x86_indirect_thunk_%V[thunk_target]\n",		\
-	X86_FEATURE_RETPOLINE,					\
-	"lfence;\n"						\
-	ANNOTATE_RETPOLINE_SAFE					\
-	"call *%[thunk_target]\n",				\
-	X86_FEATURE_RETPOLINE_LFENCE)
+#ifdef CONFIG_MITIGATION_RETPOLINE
+#define CALL_NOSPEC	__CS_PREFIX("%V[thunk_target]")	\
+			"call __x86_indirect_thunk_%V[thunk_target]\n"
+#else
+#define CALL_NOSPEC	"call *%[thunk_target]\n"
+#endif
 
 # define THUNK_TARGET(addr) [thunk_target] "r" (addr)
 

---
base-commit: d082ecbc71e9e0bf49883ee4afd435a77a5101b6
change-id: 20250226-call-nospec-b94808f0dc75

Best regards,
-- 
Pawan

Re: [PATCH v2] x86/speculation: Simplify and make CALL_NOSPEC consistent

[Ingo Molnar]

* Pawan Gupta <pawan.kumar.gupta@linux.intel.com> wrote:

> CALL_NOSPEC macro is used to generate Spectre-v2 mitigation friendly
> indirect branches. At compile time the macro defaults to indirect branch,
> and at runtime those can be patched to thunk based mitigations.
>
> This approach is opposite of what is done for the rest of the kernel, where
> the compile time default is to replace indirect calls with retpoline thunk
> calls.
> 
> Make CALL_NOSPEC consistent with the rest of the kernel, default to
> retpoline thunk at compile time when CONFIG_MITIGATION_RETPOLINE is
> enabled.
> 
> Also add the missing __CS_PREFIX to the CALL_NOSPEC macro to make it
> compatible with -mindirect-branch-cs-prefix.

The __CS_PREFIX change should probably be a separate patch.

Thanks,

	Ingo

Re: [PATCH v2] x86/speculation: Simplify and make CALL_NOSPEC consistent

[Pawan Gupta]

On Fri, Feb 28, 2025 at 10:36:43AM +0100, Ingo Molnar wrote:
> 
> * Pawan Gupta <pawan.kumar.gupta@linux.intel.com> wrote:
> 
> > CALL_NOSPEC macro is used to generate Spectre-v2 mitigation friendly
> > indirect branches. At compile time the macro defaults to indirect branch,
> > and at runtime those can be patched to thunk based mitigations.
> >
> > This approach is opposite of what is done for the rest of the kernel, where
> > the compile time default is to replace indirect calls with retpoline thunk
> > calls.
> > 
> > Make CALL_NOSPEC consistent with the rest of the kernel, default to
> > retpoline thunk at compile time when CONFIG_MITIGATION_RETPOLINE is
> > enabled.
> > 
> > Also add the missing __CS_PREFIX to the CALL_NOSPEC macro to make it
> > compatible with -mindirect-branch-cs-prefix.
> 
> The __CS_PREFIX change should probably be a separate patch.

Yes, I will split it into a separate patch.