From: Nadav Amit <namit@vmware.com>
To: linux-kernel@vger.kernel.org, x86@kernel.org
Cc: Nadav Amit <namit@vmware.com>,
Christopher Li <sparse@chrisli.org>,
linux-sparse@vger.kernel.org
Subject: [PATCH v5 2/9] x86: objtool: use asm macro for better compiler decisions
Date: Tue, 19 Jun 2018 12:48:47 -0700 [thread overview]
Message-ID: <20180619194854.69486-3-namit@vmware.com> (raw)
In-Reply-To: <20180619194854.69486-1-namit@vmware.com>
GCC considers the number of statements in inlined assembly blocks,
according to new-lines and semicolons, as an indication to the cost of
the block in time and space. This data is distorted by the kernel code,
which puts information in alternative sections. As a result, the
compiler may perform incorrect inlining and branch optimizations.
In the case of objtool, this distortion is extreme, since anyhow the
annotations of objtool are discarded during linkage.
The solution is to set an assembly macro and call it from the inline
assembly block. As a result GCC considers the inline assembly block as
a single instruction.
This patch slightly increases the kernel size.
text data bss dec hex filename
18140829 10224724 2957312 31322865 1ddf2f1 ./vmlinux before
18140970 10225412 2957312 31323694 1ddf62e ./vmlinux after (+829)
Static text symbols:
Before: 40321
After: 40302 (-19)
Cc: Christopher Li <sparse@chrisli.org>
Cc: linux-sparse@vger.kernel.org
Reviewed-by: Josh Poimboeuf <jpoimboe@redhat.com>
Signed-off-by: Nadav Amit <namit@vmware.com>
---
arch/x86/kernel/macros.S | 2 ++
include/linux/compiler.h | 56 ++++++++++++++++++++++++++++++----------
2 files changed, 45 insertions(+), 13 deletions(-)
diff --git a/arch/x86/kernel/macros.S b/arch/x86/kernel/macros.S
index cfc1c7d1a6eb..cee28c3246dc 100644
--- a/arch/x86/kernel/macros.S
+++ b/arch/x86/kernel/macros.S
@@ -5,3 +5,5 @@
* commonly used. The macros are precompiled into assmebly file which is later
* assembled together with each compiled file.
*/
+
+#include <linux/compiler.h>
diff --git a/include/linux/compiler.h b/include/linux/compiler.h
index 42506e4d1f53..2688f0d826e9 100644
--- a/include/linux/compiler.h
+++ b/include/linux/compiler.h
@@ -99,22 +99,13 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val,
* unique, to convince GCC not to merge duplicate inline asm statements.
*/
#define annotate_reachable() ({ \
- asm volatile("%c0:\n\t" \
- ".pushsection .discard.reachable\n\t" \
- ".long %c0b - .\n\t" \
- ".popsection\n\t" : : "i" (__COUNTER__)); \
+ asm volatile("ANNOTATE_REACHABLE counter=%c0" \
+ : : "i" (__COUNTER__)); \
})
#define annotate_unreachable() ({ \
- asm volatile("%c0:\n\t" \
- ".pushsection .discard.unreachable\n\t" \
- ".long %c0b - .\n\t" \
- ".popsection\n\t" : : "i" (__COUNTER__)); \
+ asm volatile("ANNOTATE_UNREACHABLE counter=%c0" \
+ : : "i" (__COUNTER__)); \
})
-#define ASM_UNREACHABLE \
- "999:\n\t" \
- ".pushsection .discard.unreachable\n\t" \
- ".long 999b - .\n\t" \
- ".popsection\n\t"
#else
#define annotate_reachable()
#define annotate_unreachable()
@@ -280,6 +271,45 @@ unsigned long read_word_at_a_time(const void *addr)
#endif /* __KERNEL__ */
+#else /* __ASSEMBLY__ */
+
+#ifdef __KERNEL__
+#ifndef LINKER_SCRIPT
+
+#ifdef CONFIG_STACK_VALIDATION
+.macro ANNOTATE_UNREACHABLE counter:req
+\counter:
+ .pushsection .discard.unreachable
+ .long \counter\()b -.
+ .popsection
+.endm
+
+.macro ANNOTATE_REACHABLE counter:req
+\counter:
+ .pushsection .discard.reachable
+ .long \counter\()b -.
+ .popsection
+.endm
+
+.macro ASM_UNREACHABLE
+999:
+ .pushsection .discard.unreachable
+ .long 999b - .
+ .popsection
+.endm
+#else /* CONFIG_STACK_VALIDATION */
+.macro ANNOTATE_UNREACHABLE counter:req
+.endm
+
+.macro ANNOTATE_REACHABLE counter:req
+.endm
+
+.macro ASM_UNREACHABLE
+.endm
+#endif /* CONFIG_STACK_VALIDATION */
+
+#endif /* LINKER_SCRIPT */
+#endif /* __KERNEL__ */
#endif /* __ASSEMBLY__ */
#ifndef __optimize
--
2.17.0
WARNING: multiple messages have this Message-ID (diff)
From: Nadav Amit <namit@vmware.com>
To: <linux-kernel@vger.kernel.org>, <x86@kernel.org>
Cc: Nadav Amit <namit@vmware.com>,
Christopher Li <sparse@chrisli.org>,
<linux-sparse@vger.kernel.org>
Subject: [PATCH v5 2/9] x86: objtool: use asm macro for better compiler decisions
Date: Tue, 19 Jun 2018 12:48:47 -0700 [thread overview]
Message-ID: <20180619194854.69486-3-namit@vmware.com> (raw)
In-Reply-To: <20180619194854.69486-1-namit@vmware.com>
GCC considers the number of statements in inlined assembly blocks,
according to new-lines and semicolons, as an indication to the cost of
the block in time and space. This data is distorted by the kernel code,
which puts information in alternative sections. As a result, the
compiler may perform incorrect inlining and branch optimizations.
In the case of objtool, this distortion is extreme, since anyhow the
annotations of objtool are discarded during linkage.
The solution is to set an assembly macro and call it from the inline
assembly block. As a result GCC considers the inline assembly block as
a single instruction.
This patch slightly increases the kernel size.
text data bss dec hex filename
18140829 10224724 2957312 31322865 1ddf2f1 ./vmlinux before
18140970 10225412 2957312 31323694 1ddf62e ./vmlinux after (+829)
Static text symbols:
Before: 40321
After: 40302 (-19)
Cc: Christopher Li <sparse@chrisli.org>
Cc: linux-sparse@vger.kernel.org
Reviewed-by: Josh Poimboeuf <jpoimboe@redhat.com>
Signed-off-by: Nadav Amit <namit@vmware.com>
---
arch/x86/kernel/macros.S | 2 ++
include/linux/compiler.h | 56 ++++++++++++++++++++++++++++++----------
2 files changed, 45 insertions(+), 13 deletions(-)
diff --git a/arch/x86/kernel/macros.S b/arch/x86/kernel/macros.S
index cfc1c7d1a6eb..cee28c3246dc 100644
--- a/arch/x86/kernel/macros.S
+++ b/arch/x86/kernel/macros.S
@@ -5,3 +5,5 @@
* commonly used. The macros are precompiled into assmebly file which is later
* assembled together with each compiled file.
*/
+
+#include <linux/compiler.h>
diff --git a/include/linux/compiler.h b/include/linux/compiler.h
index 42506e4d1f53..2688f0d826e9 100644
--- a/include/linux/compiler.h
+++ b/include/linux/compiler.h
@@ -99,22 +99,13 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val,
* unique, to convince GCC not to merge duplicate inline asm statements.
*/
#define annotate_reachable() ({ \
- asm volatile("%c0:\n\t" \
- ".pushsection .discard.reachable\n\t" \
- ".long %c0b - .\n\t" \
- ".popsection\n\t" : : "i" (__COUNTER__)); \
+ asm volatile("ANNOTATE_REACHABLE counter=%c0" \
+ : : "i" (__COUNTER__)); \
})
#define annotate_unreachable() ({ \
- asm volatile("%c0:\n\t" \
- ".pushsection .discard.unreachable\n\t" \
- ".long %c0b - .\n\t" \
- ".popsection\n\t" : : "i" (__COUNTER__)); \
+ asm volatile("ANNOTATE_UNREACHABLE counter=%c0" \
+ : : "i" (__COUNTER__)); \
})
-#define ASM_UNREACHABLE \
- "999:\n\t" \
- ".pushsection .discard.unreachable\n\t" \
- ".long 999b - .\n\t" \
- ".popsection\n\t"
#else
#define annotate_reachable()
#define annotate_unreachable()
@@ -280,6 +271,45 @@ unsigned long read_word_at_a_time(const void *addr)
#endif /* __KERNEL__ */
+#else /* __ASSEMBLY__ */
+
+#ifdef __KERNEL__
+#ifndef LINKER_SCRIPT
+
+#ifdef CONFIG_STACK_VALIDATION
+.macro ANNOTATE_UNREACHABLE counter:req
+\counter:
+ .pushsection .discard.unreachable
+ .long \counter\()b -.
+ .popsection
+.endm
+
+.macro ANNOTATE_REACHABLE counter:req
+\counter:
+ .pushsection .discard.reachable
+ .long \counter\()b -.
+ .popsection
+.endm
+
+.macro ASM_UNREACHABLE
+999:
+ .pushsection .discard.unreachable
+ .long 999b - .
+ .popsection
+.endm
+#else /* CONFIG_STACK_VALIDATION */
+.macro ANNOTATE_UNREACHABLE counter:req
+.endm
+
+.macro ANNOTATE_REACHABLE counter:req
+.endm
+
+.macro ASM_UNREACHABLE
+.endm
+#endif /* CONFIG_STACK_VALIDATION */
+
+#endif /* LINKER_SCRIPT */
+#endif /* __KERNEL__ */
#endif /* __ASSEMBLY__ */
#ifndef __optimize
--
2.17.0
next prev parent reply other threads:[~2018-06-19 19:48 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-06-19 19:48 [PATCH v5 0/9] x86: macrofying inline asm for better compilation Nadav Amit
2018-06-19 19:48 ` Nadav Amit
2018-06-19 19:48 ` [PATCH v5 1/9] Makefile: Prepare for using macros for inline asm Nadav Amit
2018-06-19 19:48 ` Nadav Amit
2018-06-19 19:48 ` Nadav Amit [this message]
2018-06-19 19:48 ` [PATCH v5 2/9] x86: objtool: use asm macro for better compiler decisions Nadav Amit
2018-06-19 19:48 ` [PATCH v5 3/9] x86: refcount: prevent gcc distortions Nadav Amit
2018-06-19 19:48 ` [PATCH v5 4/9] x86: alternatives: macrofy locks for better inlining Nadav Amit
2018-06-19 19:48 ` [PATCH v5 5/9] x86: bug: prevent gcc distortions Nadav Amit
2018-06-19 19:48 ` [PATCH v5 6/9] x86: prevent inline distortion by paravirt ops Nadav Amit
2018-06-20 5:05 ` kbuild test robot
2018-06-20 5:05 ` kbuild test robot
2018-06-20 10:10 ` Peter Zijlstra
2018-06-20 10:10 ` Peter Zijlstra
2018-06-19 19:48 ` [PATCH v5 7/9] x86: extable: use macros instead of inline assembly Nadav Amit
2018-06-19 19:48 ` [PATCH v5 8/9] x86: cpufeature: " Nadav Amit
2018-06-19 19:48 ` [PATCH v5 9/9] x86: jump-labels: " Nadav Amit
2018-06-20 13:06 ` Peter Zijlstra
2018-06-20 10:11 ` [PATCH v5 0/9] x86: macrofying inline asm for better compilation Peter Zijlstra
2018-06-20 10:11 ` Peter Zijlstra
2018-06-20 16:47 ` Nadav Amit
2018-06-20 16:47 ` Nadav Amit
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180619194854.69486-3-namit@vmware.com \
--to=namit@vmware.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-sparse@vger.kernel.org \
--cc=sparse@chrisli.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.