[RFC] x86/retpoline: Add clang support for 64-bit builds

[RFC] x86/retpoline: Add clang support for 64-bit builds

[Guenter Roeck]

clang has its own set of compiler options for retpoline support.
Also, the thunks required by C code have their own function names.

For 64-bit builds, there is only a single thunk, which is easy
to support. Support for 32-bit builds is more complicated - in
addition to various register thunks, there is also a thunk
named __llvm_external_retpoline_push which is more challenging.
Play it safe and only support 64-bit clang builds for now.

Link: https://github.com/llvm-mirror/clang/commit/0d816739a82da29748caf88570affb9715e18b69
Cc: David Woodhouse <dwmw@amazon.co.uk>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: gnomes@lxorguk.ukuu.org.uk
Cc: Rik van Riel <riel@redhat.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: thomas.lendacky@amd.com
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Jiri Kosina <jikos@kernel.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Kees Cook <keescook@google.com>
Cc: Tim Chen <tim.c.chen@linux.intel.com>
Cc: Greg Kroah-Hartman <gregkh@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Signed-off-by: Guenter Roeck <linux@roeck-us.net>
---
Sent as RFC because I am not sure if the 64-bit only solution
is acceptable.

 arch/x86/Makefile        |  5 ++++-
 arch/x86/lib/retpoline.S | 24 ++++++++++++++++++++----
 2 files changed, 24 insertions(+), 5 deletions(-)

diff --git a/arch/x86/Makefile b/arch/x86/Makefile
index fad55160dcb9..536dd6775988 100644
--- a/arch/x86/Makefile
+++ b/arch/x86/Makefile
@@ -232,7 +232,10 @@ KBUILD_CFLAGS += -fno-asynchronous-unwind-tables
 
 # Avoid indirect branches in kernel to deal with Spectre
 ifdef CONFIG_RETPOLINE
-    RETPOLINE_CFLAGS += $(call cc-option,-mindirect-branch=thunk-extern -mindirect-branch-register)
+    RETPOLINE_CFLAGS = $(call cc-option,-mindirect-branch=thunk-extern -mindirect-branch-register)
+    ifeq ($(RETPOLINE_CFLAGS)$(CONFIG_X86_32),)
+	RETPOLINE_CFLAGS = $(call cc-option,-mretpoline -mretpoline-external-thunk)
+    endif
     ifneq ($(RETPOLINE_CFLAGS),)
         KBUILD_CFLAGS += $(RETPOLINE_CFLAGS) -DRETPOLINE
     endif
diff --git a/arch/x86/lib/retpoline.S b/arch/x86/lib/retpoline.S
index 480edc3a5e03..f77738b13481 100644
--- a/arch/x86/lib/retpoline.S
+++ b/arch/x86/lib/retpoline.S
@@ -9,14 +9,22 @@
 #include <asm/nospec-branch.h>
 #include <asm/bitsperlong.h>
 
-.macro THUNK reg
+.macro _THUNK prefix, reg
 	.section .text.__x86.indirect_thunk
 
-ENTRY(__x86_indirect_thunk_\reg)
+ENTRY(\prefix\reg)
 	CFI_STARTPROC
 	JMP_NOSPEC %\reg
 	CFI_ENDPROC
-ENDPROC(__x86_indirect_thunk_\reg)
+ENDPROC(\prefix\reg)
+.endm
+
+.macro THUNK reg
+_THUNK __x86_indirect_thunk_ \reg
+.endm
+
+.macro CLANG_THUNK reg
+_THUNK __llvm_external_retpoline_ \reg
 .endm
 
 /*
@@ -27,8 +35,11 @@ ENDPROC(__x86_indirect_thunk_\reg)
  * the simple and nasty way...
  */
 #define __EXPORT_THUNK(sym) _ASM_NOKPROBE(sym); EXPORT_SYMBOL(sym)
-#define EXPORT_THUNK(reg) __EXPORT_THUNK(__x86_indirect_thunk_ ## reg)
+#define _EXPORT_THUNK(thunk, reg) __EXPORT_THUNK(thunk ## reg)
+#define EXPORT_THUNK(reg) _EXPORT_THUNK(__x86_indirect_thunk_, reg)
 #define GENERATE_THUNK(reg) THUNK reg ; EXPORT_THUNK(reg)
+#define EXPORT_CLANG_THUNK(reg) _EXPORT_THUNK(__llvm_external_retpoline_, reg)
+#define GENERATE_CLANG_THUNK(reg) CLANG_THUNK reg ; EXPORT_CLANG_THUNK(reg)
 
 GENERATE_THUNK(_ASM_AX)
 GENERATE_THUNK(_ASM_BX)
@@ -46,6 +57,11 @@ GENERATE_THUNK(r12)
 GENERATE_THUNK(r13)
 GENERATE_THUNK(r14)
 GENERATE_THUNK(r15)
+
+#ifdef __clang__
+GENERATE_CLANG_THUNK(r11)
+#endif
+
 #endif
 
 /*
-- 
2.7.4

Re: [RFC] x86/retpoline: Add clang support for 64-bit builds

[Guenter Roeck]

On Fri, Feb 02, 2018 at 03:30:40PM -0800, Guenter Roeck wrote:
> clang has its own set of compiler options for retpoline support.
> Also, the thunks required by C code have their own function names.
> 
> For 64-bit builds, there is only a single thunk, which is easy
> to support. Support for 32-bit builds is more complicated - in
> addition to various register thunks, there is also a thunk
> named __llvm_external_retpoline_push which is more challenging.
> Play it safe and only support 64-bit clang builds for now.
> 
Feedback, anyone ?

I understand that there is no love for the ABI differences between clang
and gcc, but that doesn't help me. Even if the patch is unacceptable
as-is, I would like to have some feedback to get an idea if I am on the
right track, and what I would have to change to at least move into the
right direction.

Thanks,
Guenter

> Link: https://github.com/llvm-mirror/clang/commit/0d816739a82da29748caf88570affb9715e18b69
> Cc: David Woodhouse <dwmw@amazon.co.uk>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@kernel.org>
> Cc: gnomes@lxorguk.ukuu.org.uk
> Cc: Rik van Riel <riel@redhat.com>
> Cc: Andi Kleen <ak@linux.intel.com>
> Cc: Josh Poimboeuf <jpoimboe@redhat.com>
> Cc: thomas.lendacky@amd.com
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Cc: Jiri Kosina <jikos@kernel.org>
> Cc: Andy Lutomirski <luto@amacapital.net>
> Cc: Dave Hansen <dave.hansen@intel.com>
> Cc: Kees Cook <keescook@google.com>
> Cc: Tim Chen <tim.c.chen@linux.intel.com>
> Cc: Greg Kroah-Hartman <gregkh@linux-foundation.org>
> Cc: Paul Turner <pjt@google.com>
> Signed-off-by: Guenter Roeck <linux@roeck-us.net>
> ---
> Sent as RFC because I am not sure if the 64-bit only solution
> is acceptable.
> 
>  arch/x86/Makefile        |  5 ++++-
>  arch/x86/lib/retpoline.S | 24 ++++++++++++++++++++----
>  2 files changed, 24 insertions(+), 5 deletions(-)
> 
> diff --git a/arch/x86/Makefile b/arch/x86/Makefile
> index fad55160dcb9..536dd6775988 100644
> --- a/arch/x86/Makefile
> +++ b/arch/x86/Makefile
> @@ -232,7 +232,10 @@ KBUILD_CFLAGS += -fno-asynchronous-unwind-tables
>  
>  # Avoid indirect branches in kernel to deal with Spectre
>  ifdef CONFIG_RETPOLINE
> -    RETPOLINE_CFLAGS += $(call cc-option,-mindirect-branch=thunk-extern -mindirect-branch-register)
> +    RETPOLINE_CFLAGS = $(call cc-option,-mindirect-branch=thunk-extern -mindirect-branch-register)
> +    ifeq ($(RETPOLINE_CFLAGS)$(CONFIG_X86_32),)
> +	RETPOLINE_CFLAGS = $(call cc-option,-mretpoline -mretpoline-external-thunk)
> +    endif
>      ifneq ($(RETPOLINE_CFLAGS),)
>          KBUILD_CFLAGS += $(RETPOLINE_CFLAGS) -DRETPOLINE
>      endif
> diff --git a/arch/x86/lib/retpoline.S b/arch/x86/lib/retpoline.S
> index 480edc3a5e03..f77738b13481 100644
> --- a/arch/x86/lib/retpoline.S
> +++ b/arch/x86/lib/retpoline.S
> @@ -9,14 +9,22 @@
>  #include <asm/nospec-branch.h>
>  #include <asm/bitsperlong.h>
>  
> -.macro THUNK reg
> +.macro _THUNK prefix, reg
>  	.section .text.__x86.indirect_thunk
>  
> -ENTRY(__x86_indirect_thunk_\reg)
> +ENTRY(\prefix\reg)
>  	CFI_STARTPROC
>  	JMP_NOSPEC %\reg
>  	CFI_ENDPROC
> -ENDPROC(__x86_indirect_thunk_\reg)
> +ENDPROC(\prefix\reg)
> +.endm
> +
> +.macro THUNK reg
> +_THUNK __x86_indirect_thunk_ \reg
> +.endm
> +
> +.macro CLANG_THUNK reg
> +_THUNK __llvm_external_retpoline_ \reg
>  .endm
>  
>  /*
> @@ -27,8 +35,11 @@ ENDPROC(__x86_indirect_thunk_\reg)
>   * the simple and nasty way...
>   */
>  #define __EXPORT_THUNK(sym) _ASM_NOKPROBE(sym); EXPORT_SYMBOL(sym)
> -#define EXPORT_THUNK(reg) __EXPORT_THUNK(__x86_indirect_thunk_ ## reg)
> +#define _EXPORT_THUNK(thunk, reg) __EXPORT_THUNK(thunk ## reg)
> +#define EXPORT_THUNK(reg) _EXPORT_THUNK(__x86_indirect_thunk_, reg)
>  #define GENERATE_THUNK(reg) THUNK reg ; EXPORT_THUNK(reg)
> +#define EXPORT_CLANG_THUNK(reg) _EXPORT_THUNK(__llvm_external_retpoline_, reg)
> +#define GENERATE_CLANG_THUNK(reg) CLANG_THUNK reg ; EXPORT_CLANG_THUNK(reg)
>  
>  GENERATE_THUNK(_ASM_AX)
>  GENERATE_THUNK(_ASM_BX)
> @@ -46,6 +57,11 @@ GENERATE_THUNK(r12)
>  GENERATE_THUNK(r13)
>  GENERATE_THUNK(r14)
>  GENERATE_THUNK(r15)
> +
> +#ifdef __clang__
> +GENERATE_CLANG_THUNK(r11)
> +#endif
> +
>  #endif
>  
>  /*
> -- 
> 2.7.4
> 

Re: [RFC] x86/retpoline: Add clang support for 64-bit builds

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 824 bytes --]

On Tue, 2018-02-06 at 12:32 -0800, Guenter Roeck wrote:
> 
> > 
> Feedback, anyone ?
> 
> I understand that there is no love for the ABI differences between clang
> and gcc, but that doesn't help me. Even if the patch is unacceptable
> as-is, I would like to have some feedback to get an idea if I am on the
> right track, and what I would have to change to at least move into the
> right direction.

I'm going for just a comprehensive NACK. If they fix clang so that at
least the thunk symbols __x86_indirect_thunk_\reg are compatible, then
we could tolerate command line differences. Otherwise, just no.

Unless you really fancy trying to do some kind of alias hack in modules
so that they link to the external thunk by its proper name, even when
built with clang? I reserve the right to hate that too.

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5206 bytes --]

Re: [RFC] x86/retpoline: Add clang support for 64-bit builds

[Andy Lutomirski]

On Tue, Feb 6, 2018 at 8:52 PM, David Woodhouse <david@woodhou.se> wrote:
> On Tue, 2018-02-06 at 12:32 -0800, Guenter Roeck wrote:
>>
>> >
>> Feedback, anyone ?
>>
>> I understand that there is no love for the ABI differences between clang
>> and gcc, but that doesn't help me. Even if the patch is unacceptable
>> as-is, I would like to have some feedback to get an idea if I am on the
>> right track, and what I would have to change to at least move into the
>> right direction.
>
> I'm going for just a comprehensive NACK. If they fix clang so that at
> least the thunk symbols __x86_indirect_thunk_\reg are compatible, then
> we could tolerate command line differences. Otherwise, just no.
>
> Unless you really fancy trying to do some kind of alias hack in modules
> so that they link to the external thunk by its proper name, even when
> built with clang? I reserve the right to hate that too.

I agree.  Linux already has too much history of bending over backwards
to deal with poor ABIs from compilers rather than asking compilers to
fix their ABI.  For example, stack canaries.  Linux should never have
supported GCC's 32-bit x86 stack canary crap ABI in the first place.

--Andy