[PATCH 9/9] bpf,x86: Respect X86_FEATURE_RETPOLINE*

[Peter Zijlstra <peterz@infradead.org>]

Current BPF codegen doesn't respect X86_FEATURE_RETPOLINE* flags and
unconditionally emits a thunk call, this is sub-optimal and doesn't
match the regular, compiler generated, code.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
 arch/x86/net/bpf_jit_comp.c |   18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -2123,14 +2123,18 @@ static int emit_fallback_jump(u8 **pprog
 	int err = 0;
 
 #ifdef CONFIG_RETPOLINE
-	/* Note that this assumes the the compiler uses external
-	 * thunks for indirect calls. Both clang and GCC use the same
-	 * naming convention for external thunks.
-	 */
-	err = emit_jump(&prog, __x86_indirect_thunk_rdx, prog);
-#else
-	EMIT2(0xFF, 0xE2);	/* jmp rdx */
+	if (cpu_feature_enabled(X86_FEATURE_RETPOLINE)) {
+		if (cpu_feature_enabled(X86_FEATURE_RETPOLINE_AMD)) {
+			/* The AMD retpoline can be easily emitted inline. */
+			EMIT3(0x0F, 0xAE, 0xE8);	/* lfence */
+			EMIT2(0xFF, 0xE2);		/* jmp rdx */
+		} else {
+			/* Call the retpoline thunk */
+			err = emit_jump(&prog, __x86_indirect_thunk_rdx, prog);
+		}
+	} else
 #endif
+	EMIT2(0xFF, 0xE2);	/* jmp rdx */
 	*pprog = prog;
 	return err;
 }