From: Peter Zijlstra <peterz@infradead.org>
To: bpf@vger.kernel.org
Subject: [PATCH bpf-next 12/17] x86/alternative: Handle Jcc __x86_indirect_thunk_\reg
Date: Wed, 27 Oct 2021 10:52:55 +0200 [thread overview]
Message-ID: <20211027085520.838533099@infradead.org> (raw)
In-Reply-To: 20211027085243.008677168@infradead.org
Handle the rare cases where the compiler (clang) does an indirect
conditional tail-call using:
Jcc __x86_indirect_thunk_\reg
For the !RETPOLINE case this can be rewritten to fit the original (6
byte) instruction like:
Jncc.d8 1f
JMP *%\reg
NOP
1:
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
arch/x86/kernel/alternative.c | 38 ++++++++++++++++++++++++++++++++++----
1 file changed, 34 insertions(+), 4 deletions(-)
--- a/arch/x86/kernel/alternative.c
+++ b/arch/x86/kernel/alternative.c
@@ -393,7 +393,8 @@ static int emit_indirect(int op, int reg
static int patch_retpoline(void *addr, struct insn *insn, u8 *bytes)
{
retpoline_thunk_t *target;
- int reg, i = 0;
+ int reg, ret, i = 0;
+ u8 op, cc;
target = addr + insn->length + insn->immediate.value;
reg = target - __x86_indirect_thunk_array;
@@ -407,9 +408,34 @@ static int patch_retpoline(void *addr, s
if (cpu_feature_enabled(X86_FEATURE_RETPOLINE))
return -1;
- i = emit_indirect(insn->opcode.bytes[0], reg, bytes);
- if (i < 0)
- return i;
+ op = insn->opcode.bytes[0];
+
+ /*
+ * Convert:
+ *
+ * Jcc.d32 __x86_indirect_thunk_\reg
+ *
+ * into:
+ *
+ * Jncc.d8 1f
+ * JMP *%\reg
+ * NOP
+ * 1:
+ */
+ if (op == 0x0f && (insn->opcode.bytes[1] & 0xf0) == 0x80) {
+ cc = insn->opcode.bytes[1] & 0xf;
+ cc ^= 1; /* invert condition */
+
+ bytes[i++] = 0x70 + cc; /* Jcc.d8 */
+ bytes[i++] = insn->length - 2;
+
+ op = JMP32_INSN_OPCODE;
+ }
+
+ ret = emit_indirect(op, reg, bytes + i);
+ if (ret < 0)
+ return ret;
+ i += ret;
for (; i < insn->length;)
bytes[i++] = BYTES_NOP1;
@@ -443,6 +469,10 @@ void __init_or_module noinline apply_ret
case JMP32_INSN_OPCODE:
break;
+ case 0x0f: /* escape */
+ if (op2 >= 0x80 && op2 <= 0x8f)
+ break;
+ fallthrough;
default:
WARN_ON_ONCE(1);
continue;
next prev parent reply other threads:[~2021-10-27 9:00 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-10-27 8:52 [PATCH bpf-next 00/17] x86: Rewrite the retpoline rewrite logic Peter Zijlstra
2021-10-27 8:52 ` [PATCH bpf-next 01/17] objtool: Handle __sanitize_cov*() tail calls Peter Zijlstra
2021-10-27 8:52 ` [PATCH bpf-next 02/17] objtool: Classify symbols Peter Zijlstra
2021-10-27 8:52 ` [PATCH bpf-next 03/17] objtool: Explicitly avoid self modifying code in .altinstr_replacement Peter Zijlstra
2021-10-27 8:52 ` [PATCH bpf-next 04/17] objtool: Shrink struct instruction Peter Zijlstra
2021-10-27 8:52 ` [PATCH bpf-next 05/17] objtool,x86: Replace alternatives with .retpoline_sites Peter Zijlstra
2021-10-27 8:52 ` [PATCH bpf-next 06/17] x86/retpoline: Remove unused replacement symbols Peter Zijlstra
2021-10-27 8:52 ` [PATCH bpf-next 07/17] x86/asm: Fix register order Peter Zijlstra
2021-10-27 8:52 ` [PATCH bpf-next 08/17] x86/asm: Fixup odd GEN-for-each-reg.h usage Peter Zijlstra
2021-10-27 8:52 ` [PATCH bpf-next 09/17] x86/retpoline: Move the retpoline thunk declarations to nospec-branch.h Peter Zijlstra
2021-10-27 8:52 ` [PATCH bpf-next 10/17] x86/retpoline: Create a retpoline thunk array Peter Zijlstra
2021-10-27 8:52 ` [PATCH bpf-next 11/17] x86/alternative: Implement .retpoline_sites support Peter Zijlstra
2021-10-27 8:52 ` Peter Zijlstra [this message]
2021-10-27 8:52 ` [PATCH bpf-next 13/17] x86/alternative: Try inline spectre_v2=retpoline,amd Peter Zijlstra
2021-10-27 8:52 ` [PATCH bpf-next 14/17] x86/alternative: Add debug prints to apply_retpolines() Peter Zijlstra
2021-10-27 8:52 ` [PATCH bpf-next 15/17] x86,bugs: Unconditionally allow spectre_v2=retpoline,amd Peter Zijlstra
2021-10-27 8:52 ` [PATCH bpf-next 16/17] bpf, x86: Simplify computing label offsets Peter Zijlstra
2021-10-27 8:53 ` [PATCH bpf-next 17/17] bpf,x86: Respect X86_FEATURE_RETPOLINE* Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20211027085520.838533099@infradead.org \
--to=peterz@infradead.org \
--cc=bpf@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).