[PATCH v2] x86/retpoline: Add clang support

[PATCH v2] x86/retpoline: Add clang support

[Guenter Roeck]

clang has its own set of compiler options for retpoline support.

Link: https://github.com/llvm-mirror/clang/commit/0d816739a82da29748caf88570affb9715e18b69
Link: https://github.com/llvm-mirror/llvm/commit/fd5a8723ce9f2a6b250e85972ef859e4253ea95d
Link: https://github.com/llvm-mirror/llvm/commit/59b64490fda69d29bb42cfdf7eec37bcc31ff833
Cc: David Woodhouse <dwmw@amazon.co.uk>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: gnomes@lxorguk.ukuu.org.uk
Cc: Rik van Riel <riel@redhat.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: thomas.lendacky@amd.com
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Jiri Kosina <jikos@kernel.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Kees Cook <keescook@google.com>
Cc: Tim Chen <tim.c.chen@linux.intel.com>
Cc: Greg Kroah-Hartman <gregkh@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Signed-off-by: Guenter Roeck <linux@roeck-us.net>
---
v2: llvm has been updated to use the same thunk names as gcc.
    Tested with:
    clang version 7.0.0
	(https://git.llvm.org/git/clang.git/
		848874aed95a913fb45f363120500cebfe54e2ef)
	(https://git.llvm.org/git/llvm.git/
		3afd566557f3616881505db0d69f5d19bf55ae14)
    cross-checked with gcc 7.3.0 (x86_64-linux-gcc.br_real (Buildroot
    2018.02-rc1) 7.3.0).

Tested with 64-bit builds only; 32-bit images fail to build with clang
with various unrelated errors and are difficult to test.

I had to change '+=' to '=' below since make otherwise sets
RETPOLINE_CFLAGS to " ", and the subsequent ifneq would always match.
This is also the reason for the "ifeq ($(RETPOLINE_CFLAGS),)".
If there is another/different/better way to handle this, please let
me know.

There are curently lots of warnings when building an image with clang.

./include/linux/init.h:134:6: warning:
	unknown attribute 'indirect_branch' ignored

I was inclined to add "&& !defined(__clang__)" to the condition for the
__noretpoline define to fix the problem, but concluded that this should
be a separate patch unless it can be addressed in clang.

 arch/x86/Makefile | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/arch/x86/Makefile b/arch/x86/Makefile
index fad55160dcb9..c1a35cea88e1 100644
--- a/arch/x86/Makefile
+++ b/arch/x86/Makefile
@@ -232,7 +232,12 @@ KBUILD_CFLAGS += -fno-asynchronous-unwind-tables
 
 # Avoid indirect branches in kernel to deal with Spectre
 ifdef CONFIG_RETPOLINE
-    RETPOLINE_CFLAGS += $(call cc-option,-mindirect-branch=thunk-extern -mindirect-branch-register)
+    # gcc
+    RETPOLINE_CFLAGS = $(call cc-option,-mindirect-branch=thunk-extern -mindirect-branch-register)
+    ifeq ($(RETPOLINE_CFLAGS),)
+	# clang
+	RETPOLINE_CFLAGS = $(call cc-option,-mretpoline-external-thunk)
+    endif
     ifneq ($(RETPOLINE_CFLAGS),)
         KBUILD_CFLAGS += $(RETPOLINE_CFLAGS) -DRETPOLINE
     endif
-- 
2.7.4

Re: [PATCH v2] x86/retpoline: Add clang support

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 2791 bytes --]



On Wed, 2018-02-07 at 13:52 -0800, Guenter Roeck wrote:
> clang has its own set of compiler options for retpoline support.
> 
> Link: https://github.com/llvm-mirror/clang/commit/0d816739a82da29748caf88570affb9715e18b69
> Link: https://github.com/llvm-mirror/llvm/commit/fd5a8723ce9f2a6b250e85972ef859e4253ea95d
> Link: https://github.com/llvm-mirror/llvm/commit/59b64490fda69d29bb42cfdf7eec37bcc31ff833
> Cc: David Woodhouse <dwmw@amazon.co.uk>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@kernel.org>
> Cc: gnomes@lxorguk.ukuu.org.uk
> Cc: Rik van Riel <riel@redhat.com>
> Cc: Andi Kleen <ak@linux.intel.com>
> Cc: Josh Poimboeuf <jpoimboe@redhat.com>
> Cc: thomas.lendacky@amd.com
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Cc: Jiri Kosina <jikos@kernel.org>
> Cc: Andy Lutomirski <luto@amacapital.net>
> Cc: Dave Hansen <dave.hansen@intel.com>
> Cc: Kees Cook <keescook@google.com>
> Cc: Tim Chen <tim.c.chen@linux.intel.com>
> Cc: Greg Kroah-Hartman <gregkh@linux-foundation.org>
> Cc: Paul Turner <pjt@google.com>
> Signed-off-by: Guenter Roeck <linux@roeck-us.net>
> ---
> v2: llvm has been updated to use the same thunk names as gcc.
>     Tested with:
>     clang version 7.0.0
> 	(https://git.llvm.org/git/clang.git/
> 		848874aed95a913fb45f363120500cebfe54e2ef)
> 	(https://git.llvm.org/git/llvm.git/
> 		3afd566557f3616881505db0d69f5d19bf55ae14)
>     cross-checked with gcc 7.3.0 (x86_64-linux-gcc.br_real (Buildroot
>     2018.02-rc1) 7.3.0).
> 
> Tested with 64-bit builds only; 32-bit images fail to build with clang
> with various unrelated errors and are difficult to test.
> 
> I had to change '+=' to '=' below since make otherwise sets
> RETPOLINE_CFLAGS to " ", and the subsequent ifneq would always match.
> This is also the reason for the "ifeq ($(RETPOLINE_CFLAGS),)".
> If there is another/different/better way to handle this, please let
> me know.

See
http://git.infradead.org/users/dwmw2/linux-retpoline.git/commitdiff/82a1f41600


> There are curently lots of warnings when building an image with clang.
> 
> ./include/linux/init.h:134:6: warning:
> 	unknown attribute 'indirect_branch' ignored
> 
> I was inclined to add "&& !defined(__clang__)" to the condition for the
> __noretpoline define to fix the problem, but concluded that this should
> be a separate patch unless it can be addressed in clang.

... where I fixed that too as well as providing the
__x86_indirect_thunk for 32-bit builds (which are broken for other
reasons anyway).

We do still need the %V asm constraint modifier to work in clang
though, because otherwise it still doesn't build if you have any
hypervisor guest support.

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5213 bytes --]

Re: [PATCH v2] x86/retpoline: Add clang support

[Guenter Roeck]

On Wed, Feb 07, 2018 at 10:28:38PM +0000, David Woodhouse wrote:
> 
> 
> On Wed, 2018-02-07 at 13:52 -0800, Guenter Roeck wrote:
> > clang has its own set of compiler options for retpoline support.
> > 
> > Link: https://github.com/llvm-mirror/clang/commit/0d816739a82da29748caf88570affb9715e18b69
> > Link: https://github.com/llvm-mirror/llvm/commit/fd5a8723ce9f2a6b250e85972ef859e4253ea95d
> > Link: https://github.com/llvm-mirror/llvm/commit/59b64490fda69d29bb42cfdf7eec37bcc31ff833
> > Cc: David Woodhouse <dwmw@amazon.co.uk>
> > Cc: Thomas Gleixner <tglx@linutronix.de>
> > Cc: Ingo Molnar <mingo@kernel.org>
> > Cc: gnomes@lxorguk.ukuu.org.uk
> > Cc: Rik van Riel <riel@redhat.com>
> > Cc: Andi Kleen <ak@linux.intel.com>
> > Cc: Josh Poimboeuf <jpoimboe@redhat.com>
> > Cc: thomas.lendacky@amd.com
> > Cc: Peter Zijlstra <peterz@infradead.org>
> > Cc: Linus Torvalds <torvalds@linux-foundation.org>
> > Cc: Jiri Kosina <jikos@kernel.org>
> > Cc: Andy Lutomirski <luto@amacapital.net>
> > Cc: Dave Hansen <dave.hansen@intel.com>
> > Cc: Kees Cook <keescook@google.com>
> > Cc: Tim Chen <tim.c.chen@linux.intel.com>
> > Cc: Greg Kroah-Hartman <gregkh@linux-foundation.org>
> > Cc: Paul Turner <pjt@google.com>
> > Signed-off-by: Guenter Roeck <linux@roeck-us.net>
> > ---
> > v2: llvm has been updated to use the same thunk names as gcc.
> >     Tested with:
> >     clang version 7.0.0
> > 	(https://git.llvm.org/git/clang.git/
> > 		848874aed95a913fb45f363120500cebfe54e2ef)
> > 	(https://git.llvm.org/git/llvm.git/
> > 		3afd566557f3616881505db0d69f5d19bf55ae14)
> >     cross-checked with gcc 7.3.0 (x86_64-linux-gcc.br_real (Buildroot
> >     2018.02-rc1) 7.3.0).
> > 
> > Tested with 64-bit builds only; 32-bit images fail to build with clang
> > with various unrelated errors and are difficult to test.
> > 
> > I had to change '+=' to '=' below since make otherwise sets
> > RETPOLINE_CFLAGS to " ", and the subsequent ifneq would always match.
> > This is also the reason for the "ifeq ($(RETPOLINE_CFLAGS),)".
> > If there is another/different/better way to handle this, please let
> > me know.
> 
> See
> http://git.infradead.org/users/dwmw2/linux-retpoline.git/commitdiff/82a1f41600
> 
Nice.
> 
> > There are curently lots of warnings when building an image with clang.
> > 
> > ./include/linux/init.h:134:6: warning:
> > 	unknown attribute 'indirect_branch' ignored
> > 
> > I was inclined to add "&& !defined(__clang__)" to the condition for the
> > __noretpoline define to fix the problem, but concluded that this should
> > be a separate patch unless it can be addressed in clang.
> 
> ... where I fixed that too as well as providing the
> __x86_indirect_thunk for 32-bit builds (which are broken for other
> reasons anyway).
> 

Much better and more comprehensive than my patch. Let's go with yours.

> We do still need the %V asm constraint modifier to work in clang
> though, because otherwise it still doesn't build if you have any
> hypervisor guest support.

Let's see what Chandler's feedback is. Worst case we can handle that
in a separate patch; we won't be able to fix all clang problems in
a single patch anyway.

Thanks,
Guenter

Re: [PATCH v2] x86/retpoline: Add clang support

[Guenter Roeck]

On Wed, Feb 07, 2018 at 10:28:38PM +0000, David Woodhouse wrote:
> 
> 
> On Wed, 2018-02-07 at 13:52 -0800, Guenter Roeck wrote:
> > clang has its own set of compiler options for retpoline support.
> > 
> > Link: https://github.com/llvm-mirror/clang/commit/0d816739a82da29748caf88570affb9715e18b69
> > Link: https://github.com/llvm-mirror/llvm/commit/fd5a8723ce9f2a6b250e85972ef859e4253ea95d
> > Link: https://github.com/llvm-mirror/llvm/commit/59b64490fda69d29bb42cfdf7eec37bcc31ff833
> > Cc: David Woodhouse <dwmw@amazon.co.uk>
> > Cc: Thomas Gleixner <tglx@linutronix.de>
> > Cc: Ingo Molnar <mingo@kernel.org>
> > Cc: gnomes@lxorguk.ukuu.org.uk
> > Cc: Rik van Riel <riel@redhat.com>
> > Cc: Andi Kleen <ak@linux.intel.com>
> > Cc: Josh Poimboeuf <jpoimboe@redhat.com>
> > Cc: thomas.lendacky@amd.com
> > Cc: Peter Zijlstra <peterz@infradead.org>
> > Cc: Linus Torvalds <torvalds@linux-foundation.org>
> > Cc: Jiri Kosina <jikos@kernel.org>
> > Cc: Andy Lutomirski <luto@amacapital.net>
> > Cc: Dave Hansen <dave.hansen@intel.com>
> > Cc: Kees Cook <keescook@google.com>
> > Cc: Tim Chen <tim.c.chen@linux.intel.com>
> > Cc: Greg Kroah-Hartman <gregkh@linux-foundation.org>
> > Cc: Paul Turner <pjt@google.com>
> > Signed-off-by: Guenter Roeck <linux@roeck-us.net>
> > ---
> > v2: llvm has been updated to use the same thunk names as gcc.
> >     Tested with:
> >     clang version 7.0.0
> > 	(https://git.llvm.org/git/clang.git/
> > 		848874aed95a913fb45f363120500cebfe54e2ef)
> > 	(https://git.llvm.org/git/llvm.git/
> > 		3afd566557f3616881505db0d69f5d19bf55ae14)
> >     cross-checked with gcc 7.3.0 (x86_64-linux-gcc.br_real (Buildroot
> >     2018.02-rc1) 7.3.0).
> > 
> > Tested with 64-bit builds only; 32-bit images fail to build with clang
> > with various unrelated errors and are difficult to test.
> > 
> > I had to change '+=' to '=' below since make otherwise sets
> > RETPOLINE_CFLAGS to " ", and the subsequent ifneq would always match.
> > This is also the reason for the "ifeq ($(RETPOLINE_CFLAGS),)".
> > If there is another/different/better way to handle this, please let
> > me know.
> 
> See
> http://git.infradead.org/users/dwmw2/linux-retpoline.git/commitdiff/82a1f41600
> 

You have __x86_indirect_thunk within #ifdef CONFIG_64BIT.
I think it should be in the else case.

Guenter

Re: [PATCH v2] x86/retpoline: Add clang support

[Guenter Roeck]

Hi David,

On Wed, Feb 07, 2018 at 03:32:11PM -0800, Guenter Roeck wrote:
[ ... ]
> > 
> > See
> > http://git.infradead.org/users/dwmw2/linux-retpoline.git/commitdiff/82a1f41600

Any chance to see a patch with this change anytime soon ?

Thanks,
Guenter

Re: [PATCH v2] x86/retpoline: Add clang support

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 951 bytes --]



On Tue, 2018-02-13 at 15:41 -0800, Guenter Roeck wrote:
> Hi David,
> 
> On Wed, Feb 07, 2018 at 03:32:11PM -0800, Guenter Roeck wrote:
> [ ... ]
> > > 
> > > See
> > > http://git.infradead.org/users/dwmw2/linux-retpoline.git/commitdiff/82a1f41600
> 
> Any chance to see a patch with this change anytime soon ?

We were waiting for a fix for
https://bugs.llvm.org/show_bug.cgi?id=36329

I tested the candidate fix tonight and reported success; the
corresponding patch is at the top of my tree. Since I think we now have
a consensus on what the clang build will look like, I'll post that in
the morning along with the other bits I have ready to send upstream.

We're going to need the percpu.h fix too, and I'd also like to see the
status of the i915 build failure you mentioned. Is there a bug filed
for that already, and is it on the blocker list for 6.0? If not, why
not?

We also need to resolve the asm-goto thing.


[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5213 bytes --]

Re: [PATCH v2] x86/retpoline: Add clang support

[Guenter Roeck]

On 02/13/2018 04:10 PM, David Woodhouse wrote:
> 
> 
> On Tue, 2018-02-13 at 15:41 -0800, Guenter Roeck wrote:
>> Hi David,
>>
>> On Wed, Feb 07, 2018 at 03:32:11PM -0800, Guenter Roeck wrote:
>> [ ... ]
>>>>   
>>>> See
>>>> http://git.infradead.org/users/dwmw2/linux-retpoline.git/commitdiff/82a1f41600
>>
>> Any chance to see a patch with this change anytime soon ?
> 
> We were waiting for a fix for
> https://bugs.llvm.org/show_bug.cgi?id=36329
> 
> I tested the candidate fix tonight and reported success; the
> corresponding patch is at the top of my tree. Since I think we now have
> a consensus on what the clang build will look like, I'll post that in
> the morning along with the other bits I have ready to send upstream.
> 
Excellent, thanks a lot! That the retpoline.S changes for clang are
no longer required is excellent news.

> We're going to need the percpu.h fix too, and I'd also like to see the
> status of the i915 build failure you mentioned. Is there a bug filed
> for that already, and is it on the blocker list for 6.0? If not, why
> not?
> 
For my part I completely forgot about it :-(. Did you expect me to file a bug ?
If so, I'll need to retest and try to figure out how to do that. Let me know;
I should be able to get that done tomorrow.

Thanks,
Guenter

Re: [PATCH v2] x86/retpoline: Add clang support

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 1003 bytes --]



On Tue, 2018-02-13 at 18:18 -0800, Guenter Roeck wrote:
> 
> > We're going to need the percpu.h fix too, and I'd also like to see the
> > status of the i915 build failure you mentioned. Is there a bug filed
> > for that already, and is it on the blocker list for 6.0? If not, why
> > not?
> > 
> For my part I completely forgot about it :-(. Did you expect me to file a bug ?
> If so, I'll need to retest and try to figure out how to do that. Let me know;
> I should be able to get that done tomorrow.

I *thought* you had mentioned that it didn't build; in that case
absolutely, I expected there to be a bug filed already. As a general
rule, nobody should *ever* mention that something is broken, without
first ensuring the bug is filed.

Looking back, I can't now see where you said that; maybe I made it up
and I only observed this failure myself. Either way, I already filed
https://bugs.llvm.org/show_bug.cgi?id=36378 after searching bugzilla
and not finding it.

Thanks.

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5213 bytes --]

Re: [PATCH v2] x86/retpoline: Add clang support

[Guenter Roeck]

On Wed, Feb 14, 2018 at 01:09:29PM +0000, David Woodhouse wrote:
> 
> 
> On Tue, 2018-02-13 at 18:18 -0800, Guenter Roeck wrote:
> > 
> > > We're going to need the percpu.h fix too, and I'd also like to see the
> > > status of the i915 build failure you mentioned. Is there a bug filed
> > > for that already, and is it on the blocker list for 6.0? If not, why
> > > not?
> > > 
> > For my part I completely forgot about it :-(. Did you expect me to file a bug ?
> > If so, I'll need to retest and try to figure out how to do that. Let me know;
> > I should be able to get that done tomorrow.
> 
> I *thought* you had mentioned that it didn't build; in that case
> absolutely, I expected there to be a bug filed already. As a general
> rule, nobody should *ever* mention that something is broken, without
> first ensuring the bug is filed.
> 
> Looking back, I can't now see where you said that; maybe I made it up
> and I only observed this failure myself. Either way, I already filed

I found it in my e-mail logs, so it definitely was me. Or maybe both
of us in parallel.

> https://bugs.llvm.org/show_bug.cgi?id=36378 after searching bugzilla
> and not finding it.
> 
Thanks!
Guenter